2023-09-30T22:00:07.058 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-30T22:00:07.058 DEBUG:teuthology.run:Teuthology command: teuthology --name dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi --owner scheduled_dparmar@teuthology --verbose --description fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/fsstress}}} --archive /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035 -- /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/orig.config.yaml 2023-09-30T22:00:07.110 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T22:00:07.190 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035 branch: fix-nfs-apply-err-reporting description: fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/fsstress}}} email: null first_in_suite: false job_id: '7408035' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: fix-nfs-apply-err-reporting ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62953 sha1: 6973b24e3d99fd494b753449d2ea12077599cbd6 owner: scheduled_dparmar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 seed: 2683 sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 sleep_before_teardown: 0 suite: fs:upgrade suite_branch: wip-62953 suite_path: /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa suite_relpath: qa suite_repo: https://github.com/dparmar18/ceph suite_sha1: 6973b24e3d99fd494b753449d2ea12077599cbd6 targets: smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHT6UO4ioFtlIcI3t7G0ooNN2C7WcTF/HmMzTJpxs4Rz3xSMq8jsklewTvcYjzA7i+VieIsSZql1hajeiQ40DdU= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGySYt+c1ONBPxgHRkiQTLkeNpBlx2n5G1xBpVB2OApOEJ7I2MiR9s06RV4zXlQcYi699ouZk420QaSwHM7Enqc= tasks: - install: branch: pacific exclude_packages: - cephadm - ceph-mgr-cephadm - ceph-immutable-object-cache - python3-rados - python3-rgw - python3-rbd - python3-cephfs - ceph-volume extra_packages: - python-rados - python-rgw - python-rbd - python-cephfs extra_system_packages: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - print: '**** done installing pacific' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client pacific - print: '**** done ceph' - ceph-fuse: null - mds_pre_upgrade: null - print: '**** done mds pre-upgrade sequence' - install.upgrade: mon.a: branch: reef - print: '**** done install.upgrade the host' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - ceph.stop: - mds.* - ceph.restart: daemons: - mds.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph osd dump -f json-pretty - ceph versions - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph osd require-osd-release reef - ceph.healthy: null - print: '**** done ceph.restart' - ceph-fuse: client.0: mounted: false - print: '**** done unmount client.0' - install.upgrade: client.0: null - print: '**** done install.upgrade on client.0' - ceph-fuse: client.0: null - print: '**** done remount client.0' - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: branch: pacific fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end - if not is_fuse() then reject() end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-30_21:53:45 tube: smithi user: dparmar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-30T22:00:07.190 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa; will attempt to use it 2023-09-30T22:00:07.191 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks 2023-09-30T22:00:07.191 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-30T22:00:07.193 INFO:teuthology.task.internal:Checking packages... 2023-09-30T22:00:07.215 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d6eb91756d99e822eeed7c6b783104a6ad86054' 2023-09-30T22:00:07.215 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:00:07.215 INFO:teuthology.packaging:ref: None 2023-09-30T22:00:07.215 INFO:teuthology.packaging:tag: None 2023-09-30T22:00:07.215 INFO:teuthology.packaging:branch: fix-nfs-apply-err-reporting 2023-09-30T22:00:07.215 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:00:07.215 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=fix-nfs-apply-err-reporting 2023-09-30T22:00:07.440 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5795.g3d6eb917 2023-09-30T22:00:07.440 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-30T22:00:07.447 INFO:teuthology.task.internal:no buildpackages task found 2023-09-30T22:00:07.448 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-30T22:00:07.454 INFO:teuthology.task.internal:Saving configuration 2023-09-30T22:00:07.473 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-30T22:00:07.478 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:00:07.503 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 21:55:08.688025', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHT6UO4ioFtlIcI3t7G0ooNN2C7WcTF/HmMzTJpxs4Rz3xSMq8jsklewTvcYjzA7i+VieIsSZql1hajeiQ40DdU='} 2023-09-30T22:00:07.530 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 21:55:08.689429', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGySYt+c1ONBPxgHRkiQTLkeNpBlx2n5G1xBpVB2OApOEJ7I2MiR9s06RV4zXlQcYi699ouZk420QaSwHM7Enqc='} 2023-09-30T22:00:07.531 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-30T22:00:07.537 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2023-09-30T22:00:07.537 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['client.0'] 2023-09-30T22:00:07.537 INFO:teuthology.run_tasks:Running task console_log... 2023-09-30T22:00:07.613 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff47e791f70>, signals=[15]) 2023-09-30T22:00:07.613 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-30T22:00:07.620 INFO:teuthology.task.internal:Opening connections... 2023-09-30T22:00:07.620 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:00:07.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:00:07.691 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2023-09-30T22:00:07.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:00:07.759 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-30T22:00:07.766 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2023-09-30T22:00:07.789 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2023-09-30T22:00:07.789 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-09-30T22:00:07.850 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2023-09-30T22:00:07.851 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T22:00:07.852 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T22:00:07.852 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-30T22:00:07.852 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-09-30T22:00:07.871 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2023-09-30T22:00:07.893 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2023-09-30T22:00:07.893 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2023-09-30T22:00:07.955 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T22:00:07.956 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T22:00:07.956 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-30T22:00:07.956 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2023-09-30T22:00:07.979 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-30T22:00:07.987 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-30T22:00:07.993 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-30T22:00:07.993 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2023-09-30T22:00:07.995 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2023-09-30T22:00:08.015 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-30T22:00:08.022 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-30T22:00:08.022 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2023-09-30T22:00:08.057 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2023-09-30T22:00:08.089 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-30T22:00:08.146 INFO:teuthology.run_tasks:Running task kernel... 2023-09-30T22:00:08.162 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-30T22:00:08.162 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-30T22:00:08.162 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T22:00:08.163 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T22:00:08.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:00:08.184 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-09-30T22:00:08.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:00:08.185 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2023-09-30T22:00:08.244 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-489.el8.x86_64 2023-09-30T22:00:08.245 INFO:teuthology.task.kernel:Running kernel on smithi089: 4.18.0-489.el8.x86_64 2023-09-30T22:00:08.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum install -y kernel 2023-09-30T22:00:08.246 INFO:teuthology.orchestra.run.smithi134.stdout:4.18.0-489.el8.x86_64 2023-09-30T22:00:08.246 INFO:teuthology.task.kernel:Running kernel on smithi134: 4.18.0-489.el8.x86_64 2023-09-30T22:00:08.246 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum install -y kernel 2023-09-30T22:00:13.688 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - AppStream 12 MB/s | 33 MB 00:02 2023-09-30T22:00:13.968 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - AppStream 11 MB/s | 33 MB 00:03 2023-09-30T22:00:23.521 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 48 MB 00:03 2023-09-30T22:00:29.809 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - BaseOS 4.9 MB/s | 48 MB 00:09 2023-09-30T22:00:30.879 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - Extras 35 kB/s | 18 kB 00:00 2023-09-30T22:00:31.896 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - Extras common packages 8.3 kB/s | 6.9 kB 00:00 2023-09-30T22:00:32.214 INFO:teuthology.orchestra.run.smithi134.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-09-30T22:00:33.998 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 16 MB 00:01 2023-09-30T22:00:36.851 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - Extras 101 kB/s | 18 kB 00:00 2023-09-30T22:00:37.251 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - Extras common packages 32 kB/s | 6.9 kB 00:00 2023-09-30T22:00:37.545 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-09-30T22:00:37.790 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 406 kB/s | 24 kB 00:00 2023-09-30T22:00:38.422 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-09-30T22:00:41.356 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-09-30T22:00:41.356 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-09-30T22:00:41.605 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-09-30T22:00:41.606 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-09-30T22:00:41.606 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2023-09-30T22:00:41.606 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-09-30T22:00:41.606 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2023-09-30T22:00:41.606 INFO:teuthology.orchestra.run.smithi134.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-09-30T22:00:41.606 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-09-30T22:00:41.606 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-09-30T22:00:41.607 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-09-30T22:00:41.607 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:00:41.607 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-09-30T22:00:41.607 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-09-30T22:00:41.607 INFO:teuthology.orchestra.run.smithi134.stdout:Install 3 Packages 2023-09-30T22:00:41.607 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:00:41.608 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 88 M 2023-09-30T22:00:41.608 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 96 M 2023-09-30T22:00:41.608 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-09-30T22:00:42.278 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 423 kB/s | 24 kB 00:00 2023-09-30T22:00:42.650 INFO:teuthology.orchestra.run.smithi134.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 12 MB/s | 10 MB 00:00 2023-09-30T22:00:43.675 INFO:teuthology.orchestra.run.smithi134.stdout:(2/3): kernel-core-4.18.0-514.el8.x86_64.rpm 23 MB/s | 43 MB 00:01 2023-09-30T22:00:44.517 INFO:teuthology.orchestra.run.smithi134.stdout:(3/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 13 MB/s | 35 MB 00:02 2023-09-30T22:00:44.517 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:00:44.517 INFO:teuthology.orchestra.run.smithi134.stdout:Total 30 MB/s | 88 MB 00:02 2023-09-30T22:00:45.192 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-09-30T22:00:45.606 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-09-30T22:00:45.606 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-09-30T22:00:45.795 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-09-30T22:00:45.795 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-09-30T22:00:46.054 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-09-30T22:00:46.054 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-09-30T22:00:46.055 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2023-09-30T22:00:46.055 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-09-30T22:00:46.055 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-09-30T22:00:46.055 INFO:teuthology.orchestra.run.smithi089.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-09-30T22:00:46.055 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-09-30T22:00:46.055 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-09-30T22:00:46.055 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-09-30T22:00:46.056 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:00:46.056 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-09-30T22:00:46.056 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-09-30T22:00:46.056 INFO:teuthology.orchestra.run.smithi089.stdout:Install 3 Packages 2023-09-30T22:00:46.056 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:00:46.057 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 88 M 2023-09-30T22:00:46.057 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 96 M 2023-09-30T22:00:46.057 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-09-30T22:00:46.833 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-09-30T22:00:46.856 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-09-30T22:00:47.197 INFO:teuthology.orchestra.run.smithi089.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 10 MB/s | 10 MB 00:00 2023-09-30T22:00:48.980 INFO:teuthology.orchestra.run.smithi089.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 13 MB/s | 35 MB 00:02 2023-09-30T22:00:49.464 INFO:teuthology.orchestra.run.smithi089.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 13 MB/s | 43 MB 00:03 2023-09-30T22:00:49.464 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:00:49.464 INFO:teuthology.orchestra.run.smithi089.stdout:Total 26 MB/s | 88 MB 00:03 2023-09-30T22:00:49.599 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-09-30T22:00:50.118 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:00:50.130 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-09-30T22:00:50.558 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-09-30T22:00:50.558 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-09-30T22:00:51.795 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-09-30T22:00:51.841 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-09-30T22:00:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:00:54.075 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:00:54.599 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-09-30T22:00:55.129 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:00:57.454 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:00:57.746 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:00:58.693 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:00:59.078 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:01:02.461 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:01:02.741 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:19.739 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:19.763 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:20.189 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:20.189 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:01:20.189 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:01:20.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:20.936 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:01:20.936 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-09-30T22:01:20.936 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-09-30T22:01:20.936 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-09-30T22:01:20.937 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:01:20.937 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-09-30T22:01:21.080 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:01:21.295 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:01:21.296 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:01:21.296 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-09-30T22:01:21.296 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T22:01:21.296 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T22:01:21.296 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-09-30T22:01:21.296 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T22:01:21.296 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:01:21.504 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:01:21.504 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:01:21.504 DEBUG:teuthology.orchestra.run.smithi134:> sudo rpm -qi grub2-tools 2023-09-30T22:01:21.538 INFO:teuthology.orchestra.run.smithi134.stdout:Name : grub2-tools 2023-09-30T22:01:21.538 INFO:teuthology.orchestra.run.smithi134.stdout:Epoch : 1 2023-09-30T22:01:21.538 INFO:teuthology.orchestra.run.smithi134.stdout:Version : 2.02 2023-09-30T22:01:21.538 INFO:teuthology.orchestra.run.smithi134.stdout:Release : 129.el8 2023-09-30T22:01:21.538 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: x86_64 2023-09-30T22:01:21.538 INFO:teuthology.orchestra.run.smithi134.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Group : System Environment/Base 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Size : 9265586 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:License : GPLv3+ 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Build Host : kbuilder.bsys.centos.org 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Relocations : (not relocatable) 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Packager : CentOS BuildSystem 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Vendor : CentOS 2023-09-30T22:01:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout:Summary : Support tools for GRUB. 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout:Description : 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout:hardware devices. 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:01:21.540 INFO:teuthology.orchestra.run.smithi134.stdout:This subpackage provides tools for support of all platforms. 2023-09-30T22:01:21.541 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-30T22:01:21.541 INFO:teuthology.task.kernel:Updating grub on smithi134 to boot 4.18.0-514.el8.x86_64 2023-09-30T22:01:21.541 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-30T22:01:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2023-09-30T22:01:24.530 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:24.555 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:24.885 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:24.886 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:01:24.886 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:01:25.198 INFO:teuthology.orchestra.run.smithi134.stderr:done 2023-09-30T22:01:25.200 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2023-09-30T22:01:25.222 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.3dT8nAjYU2 2023-09-30T22:01:25.222 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3dT8nAjYU2 2023-09-30T22:01:25.297 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.3dT8nAjYU2 2023-09-30T22:01:25.443 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.3dT8nAjYU2 is 6KB 2023-09-30T22:01:25.466 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.3dT8nAjYU2 2023-09-30T22:01:25.483 DEBUG:teuthology.orchestra.run.smithi134:> sudo /bin/ls /boot/loader/entries || true 2023-09-30T22:01:25.556 INFO:teuthology.orchestra.run.smithi134.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-09-30T22:01:25.556 INFO:teuthology.orchestra.run.smithi134.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-09-30T22:01:25.556 INFO:teuthology.orchestra.run.smithi134.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-09-30T22:01:25.556 INFO:teuthology.orchestra.run.smithi134.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-09-30T22:01:25.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-09-30T22:01:25.589 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:01:25.590 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:01:25.590 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-09-30T22:01:25.590 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-09-30T22:01:25.590 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-09-30T22:01:25.590 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:01:25.590 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-09-30T22:01:25.725 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:01:25.726 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2023-09-30T22:01:25.943 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:01:25.944 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:01:25.944 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-09-30T22:01:25.944 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T22:01:25.944 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T22:01:25.944 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-30T22:01:25.945 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi089.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T22:01:25.945 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:01:26.151 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:01:26.152 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:01:26.152 DEBUG:teuthology.orchestra.run.smithi089:> sudo rpm -qi grub2-tools 2023-09-30T22:01:26.185 INFO:teuthology.orchestra.run.smithi089.stdout:Name : grub2-tools 2023-09-30T22:01:26.185 INFO:teuthology.orchestra.run.smithi089.stdout:Epoch : 1 2023-09-30T22:01:26.185 INFO:teuthology.orchestra.run.smithi089.stdout:Version : 2.02 2023-09-30T22:01:26.185 INFO:teuthology.orchestra.run.smithi089.stdout:Release : 129.el8 2023-09-30T22:01:26.185 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: x86_64 2023-09-30T22:01:26.185 INFO:teuthology.orchestra.run.smithi089.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-09-30T22:01:26.185 INFO:teuthology.orchestra.run.smithi089.stdout:Group : System Environment/Base 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Size : 9265586 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:License : GPLv3+ 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Build Host : kbuilder.bsys.centos.org 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Relocations : (not relocatable) 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Packager : CentOS BuildSystem 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Vendor : CentOS 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Summary : Support tools for GRUB. 2023-09-30T22:01:26.186 INFO:teuthology.orchestra.run.smithi089.stdout:Description : 2023-09-30T22:01:26.187 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:01:26.187 INFO:teuthology.orchestra.run.smithi089.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-30T22:01:26.187 INFO:teuthology.orchestra.run.smithi089.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-30T22:01:26.187 INFO:teuthology.orchestra.run.smithi089.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-30T22:01:26.187 INFO:teuthology.orchestra.run.smithi089.stdout:hardware devices. 2023-09-30T22:01:26.187 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:01:26.187 INFO:teuthology.orchestra.run.smithi089.stdout:This subpackage provides tools for support of all platforms. 2023-09-30T22:01:26.188 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-30T22:01:26.188 INFO:teuthology.task.kernel:Updating grub on smithi089 to boot 4.18.0-514.el8.x86_64 2023-09-30T22:01:26.188 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-30T22:01:27.771 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2023-09-30T22:01:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:done 2023-09-30T22:01:30.040 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-09-30T22:01:30.073 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.I41kzsZuvd 2023-09-30T22:01:30.073 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.I41kzsZuvd 2023-09-30T22:01:30.147 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.I41kzsZuvd 2023-09-30T22:01:30.305 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.I41kzsZuvd is 6KB 2023-09-30T22:01:30.353 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.I41kzsZuvd 2023-09-30T22:01:30.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo /bin/ls /boot/loader/entries || true 2023-09-30T22:01:30.451 INFO:teuthology.orchestra.run.smithi089.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-09-30T22:01:30.451 INFO:teuthology.orchestra.run.smithi089.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-09-30T22:01:30.451 INFO:teuthology.orchestra.run.smithi089.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-09-30T22:01:30.451 INFO:teuthology.orchestra.run.smithi089.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-09-30T22:01:30.452 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-09-30T22:01:30.593 DEBUG:teuthology.orchestra.run.smithi089:> sudo shutdown -r now 2023-09-30T22:01:55.751 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-09-30T22:01:55.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-09-30T22:01:55.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:02:00.598 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-30T22:02:00.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-09-30T22:02:00.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:02:14.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2023-09-30T22:02:19.130 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-09-30T22:02:23.274 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-09-30T22:02:23.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:02:26.330 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2023-09-30T22:02:28.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-09-30T22:02:28.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:02:38.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-09-30T22:02:38.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:03:10.544 DEBUG:teuthology.orchestra.run.smithi134:> true 2023-09-30T22:03:10.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-09-30T22:03:10.833 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-514.el8.x86_64"... 2023-09-30T22:03:10.833 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2023-09-30T22:03:10.890 INFO:teuthology.orchestra.run.smithi134.stdout:4.18.0-514.el8.x86_64 2023-09-30T22:03:10.891 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-09-30T22:03:10.891 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T22:03:10.891 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-30T22:03:11.892 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2023-09-30T22:03:11.892 DEBUG:teuthology.orchestra.run.smithi134:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T22:03:11.979 INFO:teuthology.orchestra.run.smithi134.stdout:ttyS1 2023-09-30T22:03:11.989 DEBUG:teuthology.parallel:result is None 2023-09-30T22:03:28.137 DEBUG:teuthology.orchestra.remote:timed out 2023-09-30T22:03:40.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-09-30T22:03:40.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:03:40.552 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-09-30T22:03:40.826 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-09-30T22:03:40.826 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-09-30T22:03:40.826 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-09-30T22:03:40.846 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-514.el8.x86_64 2023-09-30T22:03:40.846 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-09-30T22:03:40.847 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T22:03:40.847 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-30T22:03:41.848 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-30T22:03:41.848 DEBUG:teuthology.orchestra.run.smithi089:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T22:03:41.922 INFO:teuthology.orchestra.run.smithi089.stdout:ttyS1 2023-09-30T22:03:41.935 DEBUG:teuthology.parallel:result is None 2023-09-30T22:03:41.935 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-30T22:03:41.942 INFO:teuthology.task.internal:Creating test directory... 2023-09-30T22:03:41.942 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T22:03:41.944 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T22:03:41.966 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-30T22:03:41.977 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-30T22:03:41.991 INFO:teuthology.task.internal:Creating archive directory... 2023-09-30T22:03:41.992 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T22:03:42.006 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T22:03:42.041 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-30T22:03:42.052 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-30T22:03:42.053 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T22:03:42.078 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T22:03:42.113 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:03:42.120 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:03:42.124 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:03:42.130 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:03:42.132 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-30T22:03:42.142 INFO:teuthology.task.internal:Configuring sudo... 2023-09-30T22:03:42.143 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T22:03:42.174 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T22:03:42.205 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-30T22:03:42.216 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-30T22:03:42.216 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:03:42.245 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:03:42.264 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:03:42.343 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:03:42.396 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:03:42.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:03:42.523 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:03:42.523 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T22:03:42.590 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:03:42.637 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:03:42.683 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:03:42.769 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:03:42.812 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:03:42.813 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T22:03:42.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2023-09-30T22:03:42.883 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2023-09-30T22:03:42.942 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:03:42.986 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:03:43.332 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-30T22:03:43.342 INFO:teuthology.task.internal:Starting timer... 2023-09-30T22:03:43.342 INFO:teuthology.run_tasks:Running task pcp... 2023-09-30T22:03:43.359 INFO:teuthology.run_tasks:Running task selinux... 2023-09-30T22:03:43.376 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-30T22:03:43.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2023-09-30T22:03:43.488 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2023-09-30T22:03:43.490 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2023-09-30T22:03:43.574 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2023-09-30T22:03:43.576 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-30T22:03:43.576 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2023-09-30T22:03:43.617 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2023-09-30T22:03:43.617 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2023-09-30T22:03:43.644 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2023-09-30T22:03:43.644 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2023-09-30T22:03:43.644 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:03:43.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:03:43.685 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:03:43.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:03:43.713 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-30T22:03:43.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2023-09-30T22:03:43.752 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2023-09-30T22:03:43.781 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-30T22:03:43.791 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-09-30T22:03:43.791 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-30T22:03:43.807 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-30T22:03:43.808 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-30T22:11:01.690 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2023-09-30T22:11:01.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-09-30T22:11:01.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:11:01.761 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-09-30T22:11:01.829 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-09-30T22:11:01.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-09-30T22:11:01.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:11:01.897 DEBUG:teuthology.orchestra.run.smithi134:> true 2023-09-30T22:11:01.966 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-09-30T22:11:01.967 INFO:teuthology.run_tasks:Running task clock... 2023-09-30T22:11:01.980 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-30T22:11:01.981 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T22:11:01.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:11:01.984 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T22:11:01.984 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:11:02.014 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T22:11:02.029 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T22:11:02.043 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T22:11:02.057 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T22:11:02.101 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2023-09-30T22:11:02.112 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2023-09-30T22:11:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T22:11:02.127 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2023-09-30T22:11:02.138 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2023-09-30T22:11:02.139 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T22:11:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T22:11:02.167 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T22:11:02.209 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-09-30T22:11:02.212 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:11:02.212 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-09-30T22:11:02.213 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:11:02.213 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:11:02.213 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:11:02.213 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:11:02.230 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-09-30T22:11:02.234 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:11:02.234 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-09-30T22:11:02.234 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:11:02.234 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:11:02.234 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:11:02.234 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:11:02.235 INFO:teuthology.run_tasks:Running task install... 2023-09-30T22:11:02.243 DEBUG:teuthology.task.install:project ceph 2023-09-30T22:11:02.243 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}} 2023-09-30T22:11:02.243 DEBUG:teuthology.task.install:config {'branch': 'pacific', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'} 2023-09-30T22:11:02.244 INFO:teuthology.task.install:Using flavor: default 2023-09-30T22:11:02.250 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T22:11:02.250 INFO:teuthology.task.install:extra packages: ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'] 2023-09-30T22:11:02.251 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'tag': None, 'wait_for_package': False} 2023-09-30T22:11:02.251 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:11:02.251 INFO:teuthology.packaging:ref: None 2023-09-30T22:11:02.251 INFO:teuthology.packaging:tag: None 2023-09-30T22:11:02.251 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:11:02.252 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:11:02.252 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:11:02.254 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'tag': None, 'wait_for_package': False} 2023-09-30T22:11:02.254 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:11:02.254 INFO:teuthology.packaging:ref: None 2023-09-30T22:11:02.254 INFO:teuthology.packaging:tag: None 2023-09-30T22:11:02.254 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:11:02.254 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:11:02.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:11:02.460 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/ 2023-09-30T22:11:02.460 INFO:teuthology.task.install.rpm:Package version is 16.2.14-44.geeb13ac9 2023-09-30T22:11:02.579 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/ 2023-09-30T22:11:02.580 INFO:teuthology.task.install.rpm:Package version is 16.2.14-44.geeb13ac9 2023-09-30T22:11:02.633 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:11:02.633 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:11:02.634 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:11:02.662 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2023-09-30T22:11:02.663 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:11:02.663 INFO:teuthology.packaging:ref: None 2023-09-30T22:11:02.663 INFO:teuthology.packaging:tag: None 2023-09-30T22:11:02.663 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:11:02.663 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:11:02.663 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:11:02.689 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:11:02.690 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:11:02.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:11:02.719 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2023-09-30T22:11:02.719 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:11:02.719 INFO:teuthology.packaging:ref: None 2023-09-30T22:11:02.719 INFO:teuthology.packaging:tag: None 2023-09-30T22:11:02.720 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:11:02.720 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:11:02.720 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:11:02.732 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:11:02.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:11:02.809 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:11:02.866 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:11:02.875 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2023-09-30T22:11:02.877 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2023-09-30T22:11:02.933 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2023-09-30T22:11:02.934 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2023-09-30T22:11:03.150 INFO:teuthology.orchestra.run.smithi134.stdout:76 files removed 2023-09-30T22:11:03.179 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2023-09-30T22:11:03.204 INFO:teuthology.orchestra.run.smithi089.stdout:76 files removed 2023-09-30T22:11:03.232 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2023-09-30T22:11:03.884 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 255 kB/s | 76 kB 00:00 2023-09-30T22:11:03.947 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 243 kB/s | 76 kB 00:00 2023-09-30T22:11:04.286 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 42 kB/s | 15 kB 00:00 2023-09-30T22:11:04.397 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 37 kB/s | 15 kB 00:00 2023-09-30T22:11:04.611 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-09-30T22:11:04.715 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 5.0 kB/s | 1.4 kB 00:00 2023-09-30T22:11:06.786 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - AppStream 16 MB/s | 33 MB 00:02 2023-09-30T22:11:06.824 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - AppStream 16 MB/s | 33 MB 00:02 2023-09-30T22:11:17.127 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - BaseOS 11 MB/s | 48 MB 00:04 2023-09-30T22:11:17.365 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - BaseOS 11 MB/s | 48 MB 00:04 2023-09-30T22:11:24.161 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - Extras 101 kB/s | 18 kB 00:00 2023-09-30T22:11:24.750 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - Extras 50 kB/s | 18 kB 00:00 2023-09-30T22:11:24.870 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - Extras common packages 13 kB/s | 6.9 kB 00:00 2023-09-30T22:11:25.334 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2023-09-30T22:11:26.146 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 8 - PowerTools 5.8 MB/s | 6.5 MB 00:01 2023-09-30T22:11:26.443 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - PowerTools 6.8 MB/s | 6.5 MB 00:00 2023-09-30T22:11:27.548 INFO:teuthology.orchestra.run.smithi134.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-09-30T22:11:27.853 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-09-30T22:11:28.699 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-09-30T22:11:28.712 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-09-30T22:11:32.524 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 390 kB/s | 24 kB 00:00 2023-09-30T22:11:32.554 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 485 kB/s | 24 kB 00:00 2023-09-30T22:11:36.726 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:11:36.727 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:11:36.752 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-09-30T22:11:36.752 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-09-30T22:11:36.752 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-09-30T22:11:36.754 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-09-30T22:11:36.849 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:11:36.850 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:11:36.876 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-09-30T22:11:36.876 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-09-30T22:11:36.877 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-09-30T22:11:36.878 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-09-30T22:11:36.959 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-09-30T22:11:36.971 INFO:teuthology.orchestra.run.smithi134.stdout:==================================================================================================== 2023-09-30T22:11:36.971 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-09-30T22:11:36.971 INFO:teuthology.orchestra.run.smithi134.stdout:==================================================================================================== 2023-09-30T22:11:36.971 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2023-09-30T22:11:36.971 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.4 k 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.1 M 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 910 k 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 1.6 M 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 3.1 M 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 8.6 M 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 27 k 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 12 M 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 52 M 2023-09-30T22:11:36.972 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 25 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 824 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 131 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 215 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 390 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 369 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 114 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 88 k 2023-09-30T22:11:36.973 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 173 k 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-09-30T22:11:36.974 INFO:teuthology.orchestra.run.smithi134.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 4.2 M 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 24 M 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 24 k 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 2.4 M 2023-09-30T22:11:36.975 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 244 k 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 5.0 M 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 18 M 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 15 k 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 26 k 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-09-30T22:11:36.976 INFO:teuthology.orchestra.run.smithi134.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 174 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 493 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.8 M 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-09-30T22:11:36.977 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-09-30T22:11:36.978 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 45 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 99 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T22:11:36.979 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-09-30T22:11:36.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T22:11:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T22:11:36.982 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout:==================================================================================================== 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout:Install 108 Packages 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 14 Packages 2023-09-30T22:11:36.983 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:11:36.984 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 200 M 2023-09-30T22:11:36.984 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-09-30T22:11:37.087 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-09-30T22:11:37.098 INFO:teuthology.orchestra.run.smithi089.stdout:==================================================================================================== 2023-09-30T22:11:37.099 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-09-30T22:11:37.099 INFO:teuthology.orchestra.run.smithi089.stdout:==================================================================================================== 2023-09-30T22:11:37.099 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-09-30T22:11:37.099 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-09-30T22:11:37.099 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.4 k 2023-09-30T22:11:37.100 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.1 M 2023-09-30T22:11:37.100 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 910 k 2023-09-30T22:11:37.100 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 1.6 M 2023-09-30T22:11:37.100 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 3.1 M 2023-09-30T22:11:37.100 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 8.6 M 2023-09-30T22:11:37.101 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 27 k 2023-09-30T22:11:37.101 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 12 M 2023-09-30T22:11:37.101 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 52 M 2023-09-30T22:11:37.101 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-09-30T22:11:37.101 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-09-30T22:11:37.102 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 25 k 2023-09-30T22:11:37.102 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 824 k 2023-09-30T22:11:37.102 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 131 k 2023-09-30T22:11:37.102 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-09-30T22:11:37.102 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 215 k 2023-09-30T22:11:37.103 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 390 k 2023-09-30T22:11:37.103 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 369 k 2023-09-30T22:11:37.103 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 114 k 2023-09-30T22:11:37.103 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 88 k 2023-09-30T22:11:37.103 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:11:37.103 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 173 k 2023-09-30T22:11:37.104 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2023-09-30T22:11:37.104 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-09-30T22:11:37.104 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-09-30T22:11:37.104 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-09-30T22:11:37.104 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-09-30T22:11:37.105 INFO:teuthology.orchestra.run.smithi089.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-09-30T22:11:37.105 INFO:teuthology.orchestra.run.smithi089.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-09-30T22:11:37.105 INFO:teuthology.orchestra.run.smithi089.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-09-30T22:11:37.105 INFO:teuthology.orchestra.run.smithi089.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-09-30T22:11:37.105 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:11:37.106 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 4.2 M 2023-09-30T22:11:37.106 INFO:teuthology.orchestra.run.smithi089.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-09-30T22:11:37.106 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-09-30T22:11:37.106 INFO:teuthology.orchestra.run.smithi089.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-09-30T22:11:37.106 INFO:teuthology.orchestra.run.smithi089.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-09-30T22:11:37.106 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-09-30T22:11:37.106 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 24 M 2023-09-30T22:11:37.107 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 24 k 2023-09-30T22:11:37.107 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 2.4 M 2023-09-30T22:11:37.107 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 244 k 2023-09-30T22:11:37.107 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 5.0 M 2023-09-30T22:11:37.107 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 18 M 2023-09-30T22:11:37.107 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 15 k 2023-09-30T22:11:37.108 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 26 k 2023-09-30T22:11:37.108 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-09-30T22:11:37.108 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T22:11:37.108 INFO:teuthology.orchestra.run.smithi089.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-09-30T22:11:37.108 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-09-30T22:11:37.108 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-09-30T22:11:37.108 INFO:teuthology.orchestra.run.smithi089.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-09-30T22:11:37.109 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 174 k 2023-09-30T22:11:37.109 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-09-30T22:11:37.109 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-09-30T22:11:37.109 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T22:11:37.110 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-09-30T22:11:37.110 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-09-30T22:11:37.110 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 493 k 2023-09-30T22:11:37.110 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-09-30T22:11:37.110 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.8 M 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-09-30T22:11:37.111 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 45 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 99 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T22:11:37.112 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-09-30T22:11:37.113 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-09-30T22:11:37.114 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T22:11:37.115 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:11:37.116 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-09-30T22:11:37.117 INFO:teuthology.orchestra.run.smithi089.stdout:==================================================================================================== 2023-09-30T22:11:37.117 INFO:teuthology.orchestra.run.smithi089.stdout:Install 108 Packages 2023-09-30T22:11:37.117 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 14 Packages 2023-09-30T22:11:37.117 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:11:37.118 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 200 M 2023-09-30T22:11:37.118 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-09-30T22:11:37.508 INFO:teuthology.orchestra.run.smithi134.stdout:(1/122): ceph-16.2.14-44.geeb13ac9.el8.x86_64.r 29 kB/s | 6.4 kB 00:00 2023-09-30T22:11:37.764 INFO:teuthology.orchestra.run.smithi089.stdout:(1/122): ceph-16.2.14-44.geeb13ac9.el8.x86_64.r 28 kB/s | 6.4 kB 00:00 2023-09-30T22:11:38.092 INFO:teuthology.orchestra.run.smithi134.stdout:(2/122): ceph-fuse-16.2.14-44.geeb13ac9.el8.x86 1.5 MB/s | 910 kB 00:00 2023-09-30T22:11:38.423 INFO:teuthology.orchestra.run.smithi089.stdout:(2/122): ceph-fuse-16.2.14-44.geeb13ac9.el8.x86 1.4 MB/s | 910 kB 00:00 2023-09-30T22:11:39.123 INFO:teuthology.orchestra.run.smithi089.stdout:(3/122): ceph-base-16.2.14-44.geeb13ac9.el8.x86 3.8 MB/s | 6.1 MB 00:01 2023-09-30T22:11:39.615 INFO:teuthology.orchestra.run.smithi089.stdout:(4/122): ceph-mgr-16.2.14-44.geeb13ac9.el8.x86_ 3.3 MB/s | 1.6 MB 00:00 2023-09-30T22:11:39.858 INFO:teuthology.orchestra.run.smithi134.stdout:(3/122): ceph-base-16.2.14-44.geeb13ac9.el8.x86 2.4 MB/s | 6.1 MB 00:02 2023-09-30T22:11:40.074 INFO:teuthology.orchestra.run.smithi089.stdout:(5/122): ceph-mds-16.2.14-44.geeb13ac9.el8.x86_ 1.5 MB/s | 2.4 MB 00:01 2023-09-30T22:11:40.501 INFO:teuthology.orchestra.run.smithi134.stdout:(4/122): ceph-mgr-16.2.14-44.geeb13ac9.el8.x86_ 2.5 MB/s | 1.6 MB 00:00 2023-09-30T22:11:40.967 INFO:teuthology.orchestra.run.smithi134.stdout:(5/122): ceph-mds-16.2.14-44.geeb13ac9.el8.x86_ 871 kB/s | 2.4 MB 00:02 2023-09-30T22:11:41.141 INFO:teuthology.orchestra.run.smithi089.stdout:(6/122): ceph-mon-16.2.14-44.geeb13ac9.el8.x86_ 3.3 MB/s | 5.0 MB 00:01 2023-09-30T22:11:42.234 INFO:teuthology.orchestra.run.smithi134.stdout:(6/122): ceph-mon-16.2.14-44.geeb13ac9.el8.x86_ 2.9 MB/s | 5.0 MB 00:01 2023-09-30T22:11:42.391 INFO:teuthology.orchestra.run.smithi089.stdout:(7/122): ceph-common-16.2.14-44.geeb13ac9.el8.x 5.0 MB/s | 24 MB 00:04 2023-09-30T22:11:42.491 INFO:teuthology.orchestra.run.smithi089.stdout:(8/122): ceph-selinux-16.2.14-44.geeb13ac9.el8. 256 kB/s | 26 kB 00:00 2023-09-30T22:11:44.325 INFO:teuthology.orchestra.run.smithi089.stdout:(9/122): ceph-radosgw-16.2.14-44.geeb13ac9.el8. 3.6 MB/s | 12 MB 00:03 2023-09-30T22:11:44.408 INFO:teuthology.orchestra.run.smithi089.stdout:(10/122): libcephfs-devel-16.2.14-44.geeb13ac9. 307 kB/s | 25 kB 00:00 2023-09-30T22:11:44.667 INFO:teuthology.orchestra.run.smithi089.stdout:(11/122): libcephfs2-16.2.14-44.geeb13ac9.el8.x 3.1 MB/s | 824 kB 00:00 2023-09-30T22:11:44.775 INFO:teuthology.orchestra.run.smithi089.stdout:(12/122): libcephsqlite-16.2.14-44.geeb13ac9.el 1.6 MB/s | 174 kB 00:00 2023-09-30T22:11:44.867 INFO:teuthology.orchestra.run.smithi089.stdout:(13/122): librados-devel-16.2.14-44.geeb13ac9.e 1.4 MB/s | 131 kB 00:00 2023-09-30T22:11:45.069 INFO:teuthology.orchestra.run.smithi089.stdout:(14/122): libradosstriper1-16.2.14-44.geeb13ac9 2.4 MB/s | 493 kB 00:00 2023-09-30T22:11:46.076 INFO:teuthology.orchestra.run.smithi089.stdout:(15/122): librgw2-16.2.14-44.geeb13ac9.el8.x86_ 3.7 MB/s | 3.8 MB 00:01 2023-09-30T22:11:46.185 INFO:teuthology.orchestra.run.smithi089.stdout:(16/122): python3-ceph-argparse-16.2.14-44.geeb 416 kB/s | 45 kB 00:00 2023-09-30T22:11:46.250 INFO:teuthology.orchestra.run.smithi134.stdout:(7/122): ceph-radosgw-16.2.14-44.geeb13ac9.el8. 2.9 MB/s | 12 MB 00:04 2023-09-30T22:11:46.269 INFO:teuthology.orchestra.run.smithi089.stdout:(17/122): python3-ceph-common-16.2.14-44.geeb13 1.2 MB/s | 99 kB 00:00 2023-09-30T22:11:46.309 INFO:teuthology.orchestra.run.smithi134.stdout:(8/122): ceph-selinux-16.2.14-44.geeb13ac9.el8. 443 kB/s | 26 kB 00:00 2023-09-30T22:11:46.369 INFO:teuthology.orchestra.run.smithi089.stdout:(18/122): python3-cephfs-16.2.14-44.geeb13ac9.e 2.1 MB/s | 215 kB 00:00 2023-09-30T22:11:46.545 INFO:teuthology.orchestra.run.smithi089.stdout:(19/122): python3-rados-16.2.14-44.geeb13ac9.el 2.2 MB/s | 390 kB 00:00 2023-09-30T22:11:46.695 INFO:teuthology.orchestra.run.smithi089.stdout:(20/122): python3-rbd-16.2.14-44.geeb13ac9.el8. 2.4 MB/s | 369 kB 00:00 2023-09-30T22:11:46.789 INFO:teuthology.orchestra.run.smithi089.stdout:(21/122): python3-rgw-16.2.14-44.geeb13ac9.el8. 1.2 MB/s | 114 kB 00:00 2023-09-30T22:11:46.879 INFO:teuthology.orchestra.run.smithi089.stdout:(22/122): rbd-fuse-16.2.14-44.geeb13ac9.el8.x86 965 kB/s | 88 kB 00:00 2023-09-30T22:11:47.996 INFO:teuthology.orchestra.run.smithi089.stdout:(23/122): rbd-mirror-16.2.14-44.geeb13ac9.el8.x 3.3 MB/s | 3.7 MB 00:01 2023-09-30T22:11:48.263 INFO:teuthology.orchestra.run.smithi089.stdout:(24/122): ceph-osd-16.2.14-44.geeb13ac9.el8.x86 2.2 MB/s | 18 MB 00:08 2023-09-30T22:11:48.288 INFO:teuthology.orchestra.run.smithi089.stdout:(25/122): rbd-nbd-16.2.14-44.geeb13ac9.el8.x86_ 594 kB/s | 173 kB 00:00 2023-09-30T22:11:48.355 INFO:teuthology.orchestra.run.smithi089.stdout:(26/122): ceph-grafana-dashboards-16.2.14-44.ge 266 kB/s | 24 kB 00:00 2023-09-30T22:11:49.405 INFO:teuthology.orchestra.run.smithi089.stdout:(27/122): ceph-mgr-dashboard-16.2.14-44.geeb13a 2.8 MB/s | 3.1 MB 00:01 2023-09-30T22:11:49.564 INFO:teuthology.orchestra.run.smithi089.stdout:(28/122): ceph-mgr-modules-core-16.2.14-44.geeb 1.5 MB/s | 244 kB 00:00 2023-09-30T22:11:49.639 INFO:teuthology.orchestra.run.smithi089.stdout:(29/122): ceph-mgr-rook-16.2.14-44.geeb13ac9.el 366 kB/s | 27 kB 00:00 2023-09-30T22:11:49.723 INFO:teuthology.orchestra.run.smithi089.stdout:(30/122): ceph-prometheus-alerts-16.2.14-44.gee 178 kB/s | 15 kB 00:00 2023-09-30T22:11:50.284 INFO:teuthology.orchestra.run.smithi134.stdout:(9/122): ceph-common-16.2.14-44.geeb13ac9.el8.x 1.9 MB/s | 24 MB 00:12 2023-09-30T22:11:50.390 INFO:teuthology.orchestra.run.smithi089.stdout:(31/122): bison-3.0.4-10.el8.x86_64.rpm 1.0 MB/s | 688 kB 00:00 2023-09-30T22:11:50.467 INFO:teuthology.orchestra.run.smithi134.stdout:(10/122): libcephfs-devel-16.2.14-44.geeb13ac9. 140 kB/s | 25 kB 00:00 2023-09-30T22:11:50.524 INFO:teuthology.orchestra.run.smithi089.stdout:(32/122): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2023-09-30T22:11:50.633 INFO:teuthology.orchestra.run.smithi089.stdout:(33/122): ceph-mgr-diskprediction-local-16.2.14 3.8 MB/s | 8.6 MB 00:02 2023-09-30T22:11:50.658 INFO:teuthology.orchestra.run.smithi089.stdout:(34/122): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2023-09-30T22:11:50.854 INFO:teuthology.orchestra.run.smithi089.stdout:(35/122): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2023-09-30T22:11:50.934 INFO:teuthology.orchestra.run.smithi134.stdout:(11/122): libcephfs2-16.2.14-44.geeb13ac9.el8.x 1.7 MB/s | 824 kB 00:00 2023-09-30T22:11:50.975 INFO:teuthology.orchestra.run.smithi089.stdout:(36/122): oniguruma-6.8.2-2.el8.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2023-09-30T22:11:51.001 INFO:teuthology.orchestra.run.smithi134.stdout:(12/122): libcephsqlite-16.2.14-44.geeb13ac9.el 2.6 MB/s | 174 kB 00:00 2023-09-30T22:11:51.068 INFO:teuthology.orchestra.run.smithi134.stdout:(13/122): librados-devel-16.2.14-44.geeb13ac9.e 1.9 MB/s | 131 kB 00:00 2023-09-30T22:11:51.192 INFO:teuthology.orchestra.run.smithi089.stdout:(37/122): librdkafka-0.11.4-3.el8.x86_64.rpm 633 kB/s | 354 kB 00:00 2023-09-30T22:11:51.285 INFO:teuthology.orchestra.run.smithi134.stdout:(14/122): libradosstriper1-16.2.14-44.geeb13ac9 2.2 MB/s | 493 kB 00:00 2023-09-30T22:11:51.376 INFO:teuthology.orchestra.run.smithi089.stdout:(38/122): openblas-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-09-30T22:11:51.484 INFO:teuthology.orchestra.run.smithi089.stdout:(39/122): python3-jsonpatch-1.21-2.el8.noarch.r 245 kB/s | 27 kB 00:00 2023-09-30T22:11:51.585 INFO:teuthology.orchestra.run.smithi089.stdout:(40/122): python3-jsonpointer-1.10-11.el8.noarc 200 kB/s | 20 kB 00:00 2023-09-30T22:11:51.735 INFO:teuthology.orchestra.run.smithi089.stdout:(41/122): openblas-threads-0.3.15-4.el8.x86_64. 9.0 MB/s | 4.9 MB 00:00 2023-09-30T22:11:51.760 INFO:teuthology.orchestra.run.smithi089.stdout:(42/122): python3-mako-1.0.6-14.el8.noarch.rpm 897 kB/s | 157 kB 00:00 2023-09-30T22:11:51.836 INFO:teuthology.orchestra.run.smithi089.stdout:(43/122): python3-markupsafe-0.23-19.el8.x86_64 387 kB/s | 39 kB 00:00 2023-09-30T22:11:51.936 INFO:teuthology.orchestra.run.smithi089.stdout:(44/122): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-09-30T22:11:52.045 INFO:teuthology.orchestra.run.smithi089.stdout:(45/122): python3-prettytable-0.7.2-14.el8.noar 405 kB/s | 44 kB 00:00 2023-09-30T22:11:52.104 INFO:teuthology.orchestra.run.smithi089.stdout:(46/122): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-09-30T22:11:52.145 INFO:teuthology.orchestra.run.smithi089.stdout:(47/122): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2023-09-30T22:11:52.169 INFO:teuthology.orchestra.run.smithi134.stdout:(15/122): ceph-osd-16.2.14-44.geeb13ac9.el8.x86 1.6 MB/s | 18 MB 00:11 2023-09-30T22:11:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:(48/122): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2023-09-30T22:11:52.235 INFO:teuthology.orchestra.run.smithi134.stdout:(16/122): python3-ceph-argparse-16.2.14-44.geeb 680 kB/s | 45 kB 00:00 2023-09-30T22:11:52.246 INFO:teuthology.orchestra.run.smithi089.stdout:(49/122): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-09-30T22:11:52.313 INFO:teuthology.orchestra.run.smithi089.stdout:(50/122): python3-pytz-2017.2-11.el8.noarch.rpm 524 kB/s | 57 kB 00:00 2023-09-30T22:11:52.319 INFO:teuthology.orchestra.run.smithi134.stdout:(17/122): python3-ceph-common-16.2.14-44.geeb13 1.2 MB/s | 99 kB 00:00 2023-09-30T22:11:52.422 INFO:teuthology.orchestra.run.smithi089.stdout:(51/122): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2023-09-30T22:11:52.453 INFO:teuthology.orchestra.run.smithi134.stdout:(18/122): python3-cephfs-16.2.14-44.geeb13ac9.e 1.6 MB/s | 215 kB 00:00 2023-09-30T22:11:52.531 INFO:teuthology.orchestra.run.smithi089.stdout:(52/122): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-09-30T22:11:52.639 INFO:teuthology.orchestra.run.smithi089.stdout:(53/122): xmlstarlet-1.6.1-20.el8.x86_64.rpm 653 kB/s | 71 kB 00:00 2023-09-30T22:11:52.670 INFO:teuthology.orchestra.run.smithi134.stdout:(19/122): python3-rados-16.2.14-44.geeb13ac9.el 1.8 MB/s | 390 kB 00:00 2023-09-30T22:11:52.773 INFO:teuthology.orchestra.run.smithi089.stdout:(54/122): elfutils-libelf-devel-0.189-3.el8.x86 463 kB/s | 62 kB 00:00 2023-09-30T22:11:52.920 INFO:teuthology.orchestra.run.smithi134.stdout:(20/122): librgw2-16.2.14-44.geeb13ac9.el8.x86_ 2.3 MB/s | 3.8 MB 00:01 2023-09-30T22:11:52.941 INFO:teuthology.orchestra.run.smithi089.stdout:(55/122): python3-scipy-1.0.0-21.module_el8.5.0 20 MB/s | 14 MB 00:00 2023-09-30T22:11:52.945 INFO:teuthology.orchestra.run.smithi134.stdout:(21/122): python3-rbd-16.2.14-44.geeb13ac9.el8. 1.3 MB/s | 369 kB 00:00 2023-09-30T22:11:52.965 INFO:teuthology.orchestra.run.smithi089.stdout:(56/122): fuse-2.9.7-17.el8.x86_64.rpm 432 kB/s | 83 kB 00:00 2023-09-30T22:11:52.988 INFO:teuthology.orchestra.run.smithi134.stdout:(22/122): python3-rgw-16.2.14-44.geeb13ac9.el8. 1.7 MB/s | 114 kB 00:00 2023-09-30T22:11:53.021 INFO:teuthology.orchestra.run.smithi134.stdout:(23/122): rbd-fuse-16.2.14-44.geeb13ac9.el8.x86 1.1 MB/s | 88 kB 00:00 2023-09-30T22:11:53.091 INFO:teuthology.orchestra.run.smithi089.stdout:(57/122): keyutils-libs-devel-1.5.10-9.el8.x86_ 320 kB/s | 48 kB 00:00 2023-09-30T22:11:53.124 INFO:teuthology.orchestra.run.smithi089.stdout:(58/122): krb5-devel-1.18.2-22.el8.x86_64.rpm 3.4 MB/s | 561 kB 00:00 2023-09-30T22:11:53.146 INFO:teuthology.orchestra.run.smithi134.stdout:(24/122): rbd-nbd-16.2.14-44.geeb13ac9.el8.x86_ 1.4 MB/s | 173 kB 00:00 2023-09-30T22:11:53.149 INFO:teuthology.orchestra.run.smithi089.stdout:(59/122): libcom_err-devel-1.45.6-5.el8.x86_64. 662 kB/s | 39 kB 00:00 2023-09-30T22:11:53.197 INFO:teuthology.orchestra.run.smithi134.stdout:(25/122): ceph-grafana-dashboards-16.2.14-44.ge 488 kB/s | 24 kB 00:00 2023-09-30T22:11:53.242 INFO:teuthology.orchestra.run.smithi089.stdout:(60/122): libquadmath-8.5.0-20.el8.x86_64.rpm 1.8 MB/s | 172 kB 00:00 2023-09-30T22:11:53.267 INFO:teuthology.orchestra.run.smithi089.stdout:(61/122): libgfortran-8.5.0-20.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2023-09-30T22:11:53.300 INFO:teuthology.orchestra.run.smithi089.stdout:(62/122): librabbitmq-0.9.0-4.el8.x86_64.rpm 802 kB/s | 47 kB 00:00 2023-09-30T22:11:53.342 INFO:teuthology.orchestra.run.smithi089.stdout:(63/122): libselinux-devel-2.9-8.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2023-09-30T22:11:53.368 INFO:teuthology.orchestra.run.smithi089.stdout:(64/122): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-09-30T22:11:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:(65/122): libverto-devel-0.3.2-2.el8.x86_64.rpm 359 kB/s | 18 kB 00:00 2023-09-30T22:11:53.427 INFO:teuthology.orchestra.run.smithi089.stdout:(66/122): libzstd-devel-1.4.4-1.el8.x86_64.rpm 746 kB/s | 44 kB 00:00 2023-09-30T22:11:53.585 INFO:teuthology.orchestra.run.smithi089.stdout:(67/122): pcre2-devel-10.32-3.el8.x86_64.rpm 3.7 MB/s | 605 kB 00:00 2023-09-30T22:11:54.069 INFO:teuthology.orchestra.run.smithi089.stdout:(68/122): ceph-test-16.2.14-44.geeb13ac9.el8.x8 4.5 MB/s | 52 MB 00:11 2023-09-30T22:11:54.094 INFO:teuthology.orchestra.run.smithi089.stdout:(69/122): pcre2-utf16-10.32-3.el8.x86_64.rpm 450 kB/s | 229 kB 00:00 2023-09-30T22:11:54.130 INFO:teuthology.orchestra.run.smithi134.stdout:(26/122): rbd-mirror-16.2.14-44.geeb13ac9.el8.x 3.2 MB/s | 3.7 MB 00:01 2023-09-30T22:11:54.219 INFO:teuthology.orchestra.run.smithi089.stdout:(70/122): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 249 kB 00:00 2023-09-30T22:11:54.222 INFO:teuthology.orchestra.run.smithi134.stdout:(27/122): ceph-mgr-dashboard-16.2.14-44.geeb13a 3.1 MB/s | 3.1 MB 00:01 2023-09-30T22:11:54.303 INFO:teuthology.orchestra.run.smithi089.stdout:(71/122): openssl-devel-1.1.1k-9.el8.x86_64.rpm 3.5 MB/s | 3.2 MB 00:00 2023-09-30T22:11:54.305 INFO:teuthology.orchestra.run.smithi134.stdout:(28/122): ceph-mgr-modules-core-16.2.14-44.geeb 2.9 MB/s | 244 kB 00:00 2023-09-30T22:11:54.328 INFO:teuthology.orchestra.run.smithi089.stdout:(72/122): pcre2-utf32-10.32-3.el8.x86_64.rpm 852 kB/s | 220 kB 00:00 2023-09-30T22:11:54.356 INFO:teuthology.orchestra.run.smithi134.stdout:(29/122): ceph-mgr-rook-16.2.14-44.geeb13ac9.el 550 kB/s | 27 kB 00:00 2023-09-30T22:11:54.362 INFO:teuthology.orchestra.run.smithi089.stdout:(73/122): python3-jwt-1.6.1-2.el8.noarch.rpm 728 kB/s | 43 kB 00:00 2023-09-30T22:11:54.412 INFO:teuthology.orchestra.run.smithi089.stdout:(74/122): python3-cryptography-3.2.1-6.el8.x86_ 3.3 MB/s | 641 kB 00:00 2023-09-30T22:11:54.414 INFO:teuthology.orchestra.run.smithi134.stdout:(30/122): ceph-prometheus-alerts-16.2.14-44.gee 255 kB/s | 15 kB 00:00 2023-09-30T22:11:54.446 INFO:teuthology.orchestra.run.smithi089.stdout:(75/122): python3-oauthlib-2.1.0-1.el8.noarch.r 1.3 MB/s | 155 kB 00:00 2023-09-30T22:11:54.471 INFO:teuthology.orchestra.run.smithi089.stdout:(76/122): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2023-09-30T22:11:54.496 INFO:teuthology.orchestra.run.smithi089.stdout:(77/122): python3-requests-oauthlib-1.0.0-1.el8 509 kB/s | 43 kB 00:00 2023-09-30T22:11:54.522 INFO:teuthology.orchestra.run.smithi089.stdout:(78/122): zlib-devel-1.2.11-21.el8.x86_64.rpm 768 kB/s | 58 kB 00:00 2023-09-30T22:11:54.872 INFO:teuthology.orchestra.run.smithi089.stdout:(79/122): liboath-2.6.2-3.el8.x86_64.rpm 168 kB/s | 59 kB 00:00 2023-09-30T22:11:54.897 INFO:teuthology.orchestra.run.smithi089.stdout:(80/122): leveldb-1.22-1.el8.x86_64.rpm 452 kB/s | 181 kB 00:00 2023-09-30T22:11:54.989 INFO:teuthology.orchestra.run.smithi089.stdout:(81/122): gperftools-libs-2.7-9.el8.x86_64.rpm 591 kB/s | 306 kB 00:00 2023-09-30T22:11:55.014 INFO:teuthology.orchestra.run.smithi089.stdout:(82/122): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2023-09-30T22:11:55.090 INFO:teuthology.orchestra.run.smithi089.stdout:(83/122): libunwind-1.3.1-3.el8.x86_64.rpm 345 kB/s | 75 kB 00:00 2023-09-30T22:11:55.106 INFO:teuthology.orchestra.run.smithi134.stdout:(31/122): bison-3.0.4-10.el8.x86_64.rpm 995 kB/s | 688 kB 00:00 2023-09-30T22:11:55.138 INFO:teuthology.orchestra.run.smithi089.stdout:(84/122): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2023-09-30T22:11:55.166 INFO:teuthology.orchestra.run.smithi089.stdout:(85/122): python3-cachetools-3.1.1-4.el8.noarch 218 kB/s | 33 kB 00:00 2023-09-30T22:11:55.208 INFO:teuthology.orchestra.run.smithi089.stdout:(86/122): python3-certifi-2018.10.15-7.el8.noar 139 kB/s | 16 kB 00:00 2023-09-30T22:11:55.275 INFO:teuthology.orchestra.run.smithi089.stdout:(87/122): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-09-30T22:11:55.290 INFO:teuthology.orchestra.run.smithi134.stdout:(32/122): flex-2.6.1-9.el8.x86_64.rpm 1.7 MB/s | 320 kB 00:00 2023-09-30T22:11:55.342 INFO:teuthology.orchestra.run.smithi089.stdout:(88/122): python3-google-auth-1.1.1-10.el8.noar 609 kB/s | 81 kB 00:00 2023-09-30T22:11:55.392 INFO:teuthology.orchestra.run.smithi089.stdout:(89/122): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-09-30T22:11:55.415 INFO:teuthology.orchestra.run.smithi134.stdout:(33/122): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2023-09-30T22:11:55.451 INFO:teuthology.orchestra.run.smithi089.stdout:(90/122): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-09-30T22:11:55.476 INFO:teuthology.orchestra.run.smithi089.stdout:(91/122): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-09-30T22:11:55.568 INFO:teuthology.orchestra.run.smithi089.stdout:(92/122): python3-logutils-0.3.5-11.el8.noarch. 417 kB/s | 49 kB 00:00 2023-09-30T22:11:55.590 INFO:teuthology.orchestra.run.smithi134.stdout:(34/122): ceph-mgr-diskprediction-local-16.2.14 5.9 MB/s | 8.6 MB 00:01 2023-09-30T22:11:55.593 INFO:teuthology.orchestra.run.smithi089.stdout:(93/122): python3-more-itertools-7.2.0-3.el8.no 502 kB/s | 59 kB 00:00 2023-09-30T22:11:55.616 INFO:teuthology.orchestra.run.smithi134.stdout:(35/122): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2023-09-30T22:11:55.743 INFO:teuthology.orchestra.run.smithi089.stdout:(94/122): python3-portend-2.6-1.el8.noarch.rpm 104 kB/s | 16 kB 00:00 2023-09-30T22:11:55.844 INFO:teuthology.orchestra.run.smithi089.stdout:(95/122): python3-repoze-lru-0.7-6.el8.noarch.r 337 kB/s | 34 kB 00:00 2023-09-30T22:11:55.849 INFO:teuthology.orchestra.run.smithi134.stdout:(36/122): oniguruma-6.8.2-2.el8.x86_64.rpm 803 kB/s | 187 kB 00:00 2023-09-30T22:11:56.037 INFO:teuthology.orchestra.run.smithi089.stdout:(96/122): python3-kubernetes-11.0.0-6.el8.noarc 1.8 MB/s | 1.2 MB 00:00 2023-09-30T22:11:56.228 INFO:teuthology.orchestra.run.smithi089.stdout:(97/122): python3-pecan-1.3.2-9.el8.noarch.rpm 429 kB/s | 283 kB 00:00 2023-09-30T22:11:56.233 INFO:teuthology.orchestra.run.smithi134.stdout:(37/122): lttng-ust-2.8.1-11.el8.x86_64.rpm 404 kB/s | 259 kB 00:00 2023-09-30T22:11:56.245 INFO:teuthology.orchestra.run.smithi089.stdout:(98/122): python3-rsa-4.9-2.el8.noarch.rpm 310 kB/s | 64 kB 00:00 2023-09-30T22:11:56.270 INFO:teuthology.orchestra.run.smithi089.stdout:(99/122): python3-routes-2.4.1-12.el8.noarch.rp 461 kB/s | 196 kB 00:00 2023-09-30T22:11:56.309 INFO:teuthology.orchestra.run.smithi134.stdout:(38/122): openblas-0.3.15-4.el8.x86_64.rpm 10 MB/s | 4.7 MB 00:00 2023-09-30T22:11:56.320 INFO:teuthology.orchestra.run.smithi089.stdout:(100/122): python3-simplegeneric-0.8.1-17.el8.n 203 kB/s | 19 kB 00:00 2023-09-30T22:11:56.368 INFO:teuthology.orchestra.run.smithi089.stdout:(101/122): python3-singledispatch-3.4.0.3-18.el 195 kB/s | 24 kB 00:00 2023-09-30T22:11:56.396 INFO:teuthology.orchestra.run.smithi089.stdout:(102/122): python3-tempora-1.14.1-5.el8.noarch. 233 kB/s | 29 kB 00:00 2023-09-30T22:11:56.400 INFO:teuthology.orchestra.run.smithi134.stdout:(39/122): python3-jsonpatch-1.21-2.el8.noarch.r 290 kB/s | 27 kB 00:00 2023-09-30T22:11:56.413 INFO:teuthology.orchestra.run.smithi089.stdout:(103/122): python3-trustme-0.6.0-4.el8.noarch.r 298 kB/s | 27 kB 00:00 2023-09-30T22:11:56.463 INFO:teuthology.orchestra.run.smithi089.stdout:(104/122): python3-waitress-1.4.3-1.el8.noarch. 2.6 MB/s | 255 kB 00:00 2023-09-30T22:11:56.501 INFO:teuthology.orchestra.run.smithi134.stdout:(40/122): python3-jsonpointer-1.10-11.el8.noarc 200 kB/s | 20 kB 00:00 2023-09-30T22:11:56.505 INFO:teuthology.orchestra.run.smithi089.stdout:(105/122): python3-websocket-client-0.56.0-5.el 665 kB/s | 61 kB 00:00 2023-09-30T22:11:56.556 INFO:teuthology.orchestra.run.smithi089.stdout:(106/122): python3-webtest-2.0.33-1.el8.noarch. 927 kB/s | 85 kB 00:00 2023-09-30T22:11:56.598 INFO:teuthology.orchestra.run.smithi089.stdout:(107/122): python3-zc-lockfile-2.0-2.el8.noarch 249 kB/s | 23 kB 00:00 2023-09-30T22:11:56.601 INFO:teuthology.orchestra.run.smithi134.stdout:(41/122): python3-mako-1.0.6-14.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2023-09-30T22:11:56.673 INFO:teuthology.orchestra.run.smithi089.stdout:(108/122): python3-webob-1.8.5-1.el8.1.noarch.r 909 kB/s | 251 kB 00:00 2023-09-30T22:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout:(42/122): python3-markupsafe-0.23-19.el8.x86_64 422 kB/s | 39 kB 00:00 2023-09-30T22:11:56.769 INFO:teuthology.orchestra.run.smithi134.stdout:(43/122): openblas-threads-0.3.15-4.el8.x86_64. 9.1 MB/s | 4.9 MB 00:00 2023-09-30T22:11:56.815 INFO:teuthology.orchestra.run.smithi089.stdout:(109/122): NetworkManager-initscripts-updown-1. 1.0 MB/s | 143 kB 00:00 2023-09-30T22:11:56.882 INFO:teuthology.orchestra.run.smithi089.stdout:(110/122): elfutils-debuginfod-client-0.189-3.e 1.1 MB/s | 76 kB 00:00 2023-09-30T22:11:56.966 INFO:teuthology.orchestra.run.smithi089.stdout:(111/122): elfutils-libelf-0.189-3.el8.x86_64.r 2.7 MB/s | 232 kB 00:00 2023-09-30T22:11:57.024 INFO:teuthology.orchestra.run.smithi089.stdout:(112/122): librados2-16.2.14-44.geeb13ac9.el8.x 8.0 MB/s | 3.7 MB 00:00 2023-09-30T22:11:57.083 INFO:teuthology.orchestra.run.smithi089.stdout:(113/122): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2023-09-30T22:11:57.142 INFO:teuthology.orchestra.run.smithi089.stdout:(114/122): libblkid-2.32.1-43.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-09-30T22:11:57.259 INFO:teuthology.orchestra.run.smithi089.stdout:(115/122): libfdisk-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 254 kB 00:00 2023-09-30T22:11:57.294 INFO:teuthology.orchestra.run.smithi134.stdout:(44/122): ceph-test-16.2.14-44.geeb13ac9.el8.x8 4.7 MB/s | 52 MB 00:10 2023-09-30T22:11:57.319 INFO:teuthology.orchestra.run.smithi134.stdout:(45/122): python3-numpy-f2py-1.14.3-10.el8.x86_ 409 kB/s | 225 kB 00:00 2023-09-30T22:11:57.326 INFO:teuthology.orchestra.run.smithi089.stdout:(116/122): iproute-6.2.0-2.el8.x86_64.rpm 2.9 MB/s | 880 kB 00:00 2023-09-30T22:11:57.351 INFO:teuthology.orchestra.run.smithi089.stdout:(117/122): libmount-2.32.1-43.el8.x86_64.rpm 2.5 MB/s | 236 kB 00:00 2023-09-30T22:11:57.376 INFO:teuthology.orchestra.run.smithi089.stdout:(118/122): libsmartcols-2.32.1-43.el8.x86_64.rp 3.5 MB/s | 179 kB 00:00 2023-09-30T22:11:57.394 INFO:teuthology.orchestra.run.smithi134.stdout:(46/122): python3-numpy-1.14.3-10.el8.x86_64.rp 5.3 MB/s | 3.7 MB 00:00 2023-09-30T22:11:57.411 INFO:teuthology.orchestra.run.smithi134.stdout:(47/122): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.1 MB/s | 103 kB 00:00 2023-09-30T22:11:57.435 INFO:teuthology.orchestra.run.smithi089.stdout:(119/122): librbd1-16.2.14-44.geeb13ac9.el8.x86 5.0 MB/s | 4.2 MB 00:00 2023-09-30T22:11:57.460 INFO:teuthology.orchestra.run.smithi089.stdout:(120/122): libuuid-2.32.1-43.el8.x86_64.rpm 907 kB/s | 99 kB 00:00 2023-09-30T22:11:57.503 INFO:teuthology.orchestra.run.smithi134.stdout:(48/122): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-09-30T22:11:57.577 INFO:teuthology.orchestra.run.smithi089.stdout:(121/122): util-linux-user-2.32.1-43.el8.x86_64 724 kB/s | 103 kB 00:00 2023-09-30T22:11:57.603 INFO:teuthology.orchestra.run.smithi134.stdout:(49/122): python3-pyasn1-modules-0.3.7-6.el8.no 573 kB/s | 110 kB 00:00 2023-09-30T22:11:57.629 INFO:teuthology.orchestra.run.smithi134.stdout:(50/122): python3-pytz-2017.2-11.el8.noarch.rpm 455 kB/s | 57 kB 00:00 2023-09-30T22:11:57.654 INFO:teuthology.orchestra.run.smithi134.stdout:(51/122): python3-prettytable-0.7.2-14.el8.noar 122 kB/s | 44 kB 00:00 2023-09-30T22:11:57.754 INFO:teuthology.orchestra.run.smithi134.stdout:(52/122): python3-werkzeug-0.12.2-4.el8.noarch. 3.6 MB/s | 457 kB 00:00 2023-09-30T22:11:57.836 INFO:teuthology.orchestra.run.smithi089.stdout:(122/122): util-linux-2.32.1-43.el8.x86_64.rpm 5.4 MB/s | 2.5 MB 00:00 2023-09-30T22:11:57.839 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:11:57.839 INFO:teuthology.orchestra.run.smithi089.stdout:Total 9.6 MB/s | 200 MB 00:20 2023-09-30T22:11:57.855 INFO:teuthology.orchestra.run.smithi134.stdout:(53/122): xmlstarlet-1.6.1-20.el8.x86_64.rpm 707 kB/s | 71 kB 00:00 2023-09-30T22:11:58.013 INFO:teuthology.orchestra.run.smithi134.stdout:(54/122): elfutils-libelf-devel-0.189-3.el8.x86 390 kB/s | 62 kB 00:00 2023-09-30T22:11:58.055 INFO:teuthology.orchestra.run.smithi134.stdout:(55/122): socat-1.7.4.1-1.el8.x86_64.rpm 806 kB/s | 323 kB 00:00 2023-09-30T22:11:58.089 INFO:teuthology.orchestra.run.smithi134.stdout:(56/122): fuse-2.9.7-17.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-09-30T22:11:58.264 INFO:teuthology.orchestra.run.smithi134.stdout:(57/122): python3-scipy-1.0.0-21.module_el8.5.0 21 MB/s | 14 MB 00:00 2023-09-30T22:11:58.289 INFO:teuthology.orchestra.run.smithi134.stdout:(58/122): keyutils-libs-devel-1.5.10-9.el8.x86_ 206 kB/s | 48 kB 00:00 2023-09-30T22:11:58.315 INFO:teuthology.orchestra.run.smithi134.stdout:(59/122): krb5-devel-1.18.2-22.el8.x86_64.rpm 2.4 MB/s | 561 kB 00:00 2023-09-30T22:11:58.381 INFO:teuthology.orchestra.run.smithi134.stdout:(60/122): libquadmath-8.5.0-20.el8.x86_64.rpm 2.5 MB/s | 172 kB 00:00 2023-09-30T22:11:58.407 INFO:teuthology.orchestra.run.smithi134.stdout:(61/122): libcom_err-devel-1.45.6-5.el8.x86_64. 272 kB/s | 39 kB 00:00 2023-09-30T22:11:58.440 INFO:teuthology.orchestra.run.smithi134.stdout:(62/122): librabbitmq-0.9.0-4.el8.x86_64.rpm 804 kB/s | 47 kB 00:00 2023-09-30T22:11:58.465 INFO:teuthology.orchestra.run.smithi134.stdout:(63/122): libgfortran-8.5.0-20.el8.x86_64.rpm 3.6 MB/s | 645 kB 00:00 2023-09-30T22:11:58.499 INFO:teuthology.orchestra.run.smithi134.stdout:(64/122): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2023-09-30T22:11:58.516 INFO:teuthology.orchestra.run.smithi134.stdout:(65/122): libverto-devel-0.3.2-2.el8.x86_64.rpm 359 kB/s | 18 kB 00:00 2023-09-30T22:11:58.541 INFO:teuthology.orchestra.run.smithi134.stdout:(66/122): libselinux-devel-2.9-8.el8.x86_64.rpm 1.5 MB/s | 200 kB 00:00 2023-09-30T22:11:58.558 INFO:teuthology.orchestra.run.smithi134.stdout:(67/122): libzstd-devel-1.4.4-1.el8.x86_64.rpm 747 kB/s | 44 kB 00:00 2023-09-30T22:11:58.575 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-09-30T22:11:58.617 INFO:teuthology.orchestra.run.smithi134.stdout:(68/122): pcre2-utf16-10.32-3.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-09-30T22:11:58.675 INFO:teuthology.orchestra.run.smithi134.stdout:(69/122): pcre2-devel-10.32-3.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-09-30T22:11:58.701 INFO:teuthology.orchestra.run.smithi134.stdout:(70/122): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2023-09-30T22:11:58.751 INFO:teuthology.orchestra.run.smithi134.stdout:(71/122): python3-cffi-1.11.5-6.el8.x86_64.rpm 3.2 MB/s | 249 kB 00:00 2023-09-30T22:11:58.796 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-09-30T22:11:58.797 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-09-30T22:11:58.801 INFO:teuthology.orchestra.run.smithi134.stdout:(72/122): python3-jwt-1.6.1-2.el8.noarch.rpm 855 kB/s | 43 kB 00:00 2023-09-30T22:11:58.835 INFO:teuthology.orchestra.run.smithi134.stdout:(73/122): python3-cryptography-3.2.1-6.el8.x86_ 4.7 MB/s | 641 kB 00:00 2023-09-30T22:11:58.884 INFO:teuthology.orchestra.run.smithi134.stdout:(74/122): python3-oauthlib-2.1.0-1.el8.noarch.r 1.8 MB/s | 155 kB 00:00 2023-09-30T22:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout:(75/122): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-30T22:11:58.936 INFO:teuthology.orchestra.run.smithi134.stdout:(76/122): python3-requests-oauthlib-1.0.0-1.el8 827 kB/s | 43 kB 00:00 2023-09-30T22:11:58.969 INFO:teuthology.orchestra.run.smithi134.stdout:(77/122): zlib-devel-1.2.11-21.el8.x86_64.rpm 992 kB/s | 58 kB 00:00 2023-09-30T22:11:59.086 INFO:teuthology.orchestra.run.smithi134.stdout:(78/122): openssl-devel-1.1.1k-9.el8.x86_64.rpm 5.6 MB/s | 3.2 MB 00:00 2023-09-30T22:11:59.328 INFO:teuthology.orchestra.run.smithi134.stdout:(79/122): leveldb-1.22-1.el8.x86_64.rpm 505 kB/s | 181 kB 00:00 2023-09-30T22:11:59.353 INFO:teuthology.orchestra.run.smithi134.stdout:(80/122): liboath-2.6.2-3.el8.x86_64.rpm 221 kB/s | 59 kB 00:00 2023-09-30T22:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout:(81/122): gperftools-libs-2.7-9.el8.x86_64.rpm 692 kB/s | 306 kB 00:00 2023-09-30T22:11:59.404 INFO:teuthology.orchestra.run.smithi134.stdout:(82/122): libunwind-1.3.1-3.el8.x86_64.rpm 996 kB/s | 75 kB 00:00 2023-09-30T22:11:59.437 INFO:teuthology.orchestra.run.smithi134.stdout:(83/122): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 522 kB/s | 44 kB 00:00 2023-09-30T22:11:59.471 INFO:teuthology.orchestra.run.smithi134.stdout:(84/122): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-09-30T22:11:59.496 INFO:teuthology.orchestra.run.smithi134.stdout:(85/122): python3-cachetools-3.1.1-4.el8.noarch 359 kB/s | 33 kB 00:00 2023-09-30T22:11:59.522 INFO:teuthology.orchestra.run.smithi134.stdout:(86/122): python3-certifi-2018.10.15-7.el8.noar 194 kB/s | 16 kB 00:00 2023-09-30T22:11:59.547 INFO:teuthology.orchestra.run.smithi134.stdout:(87/122): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-09-30T22:11:59.614 INFO:teuthology.orchestra.run.smithi134.stdout:(88/122): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-09-30T22:11:59.639 INFO:teuthology.orchestra.run.smithi134.stdout:(89/122): python3-google-auth-1.1.1-10.el8.noar 695 kB/s | 81 kB 00:00 2023-09-30T22:11:59.664 INFO:teuthology.orchestra.run.smithi134.stdout:(90/122): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2023-09-30T22:11:59.712 INFO:teuthology.orchestra.run.smithi134.stdout:(91/122): python3-jaraco-functools-2.0-4.el8.no 180 kB/s | 18 kB 00:00 2023-09-30T22:11:59.740 INFO:teuthology.orchestra.run.smithi134.stdout:(92/122): python3-logutils-0.3.5-11.el8.noarch. 647 kB/s | 49 kB 00:00 2023-09-30T22:11:59.790 INFO:teuthology.orchestra.run.smithi134.stdout:(93/122): python3-more-itertools-7.2.0-3.el8.no 756 kB/s | 59 kB 00:00 2023-09-30T22:11:59.824 INFO:teuthology.orchestra.run.smithi134.stdout:(94/122): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-09-30T22:11:59.857 INFO:teuthology.orchestra.run.smithi134.stdout:(95/122): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-09-30T22:11:59.891 INFO:teuthology.orchestra.run.smithi134.stdout:(96/122): python3-repoze-lru-0.7-6.el8.noarch.r 504 kB/s | 34 kB 00:00 2023-09-30T22:11:59.926 INFO:teuthology.orchestra.run.smithi134.stdout:(97/122): python3-kubernetes-11.0.0-6.el8.noarc 4.1 MB/s | 1.2 MB 00:00 2023-09-30T22:11:59.949 INFO:teuthology.orchestra.run.smithi134.stdout:(98/122): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-09-30T22:11:59.975 INFO:teuthology.orchestra.run.smithi134.stdout:(99/122): python3-rsa-4.9-2.el8.noarch.rpm 766 kB/s | 64 kB 00:00 2023-09-30T22:12:00.000 INFO:teuthology.orchestra.run.smithi134.stdout:(100/122): python3-simplegeneric-0.8.1-17.el8.n 248 kB/s | 19 kB 00:00 2023-09-30T22:12:00.025 INFO:teuthology.orchestra.run.smithi134.stdout:(101/122): python3-singledispatch-3.4.0.3-18.el 317 kB/s | 24 kB 00:00 2023-09-30T22:12:00.051 INFO:teuthology.orchestra.run.smithi134.stdout:(102/122): python3-tempora-1.14.1-5.el8.noarch. 388 kB/s | 29 kB 00:00 2023-09-30T22:12:00.076 INFO:teuthology.orchestra.run.smithi134.stdout:(103/122): python3-trustme-0.6.0-4.el8.noarch.r 365 kB/s | 27 kB 00:00 2023-09-30T22:12:00.129 INFO:teuthology.orchestra.run.smithi134.stdout:(104/122): python3-waitress-1.4.3-1.el8.noarch. 2.5 MB/s | 255 kB 00:00 2023-09-30T22:12:00.132 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-09-30T22:12:00.135 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-09-30T22:12:00.152 INFO:teuthology.orchestra.run.smithi134.stdout:(105/122): python3-websocket-client-0.56.0-5.el 811 kB/s | 61 kB 00:00 2023-09-30T22:12:00.177 INFO:teuthology.orchestra.run.smithi134.stdout:(106/122): python3-webob-1.8.5-1.el8.1.noarch.r 2.0 MB/s | 251 kB 00:00 2023-09-30T22:12:00.202 INFO:teuthology.orchestra.run.smithi134.stdout:(107/122): python3-webtest-2.0.33-1.el8.noarch. 1.1 MB/s | 85 kB 00:00 2023-09-30T22:12:00.228 INFO:teuthology.orchestra.run.smithi134.stdout:(108/122): python3-zc-lockfile-2.0-2.el8.noarch 304 kB/s | 23 kB 00:00 2023-09-30T22:12:00.361 INFO:teuthology.orchestra.run.smithi134.stdout:(109/122): NetworkManager-initscripts-updown-1. 1.1 MB/s | 143 kB 00:00 2023-09-30T22:12:00.420 INFO:teuthology.orchestra.run.smithi134.stdout:(110/122): elfutils-debuginfod-client-0.189-3.e 1.3 MB/s | 76 kB 00:00 2023-09-30T22:12:00.503 INFO:teuthology.orchestra.run.smithi134.stdout:(111/122): elfutils-libelf-0.189-3.el8.x86_64.r 2.7 MB/s | 232 kB 00:00 2023-09-30T22:12:00.596 INFO:teuthology.orchestra.run.smithi134.stdout:(112/122): elfutils-libs-0.189-3.el8.x86_64.rpm 3.2 MB/s | 303 kB 00:00 2023-09-30T22:12:00.737 INFO:teuthology.orchestra.run.smithi134.stdout:(113/122): librbd1-16.2.14-44.geeb13ac9.el8.x86 7.8 MB/s | 4.2 MB 00:00 2023-09-30T22:12:00.787 INFO:teuthology.orchestra.run.smithi134.stdout:(114/122): iproute-6.2.0-2.el8.x86_64.rpm 4.5 MB/s | 880 kB 00:00 2023-09-30T22:12:00.879 INFO:teuthology.orchestra.run.smithi134.stdout:(115/122): libfdisk-2.32.1-43.el8.x86_64.rpm 2.7 MB/s | 254 kB 00:00 2023-09-30T22:12:00.929 INFO:teuthology.orchestra.run.smithi134.stdout:(116/122): libblkid-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2023-09-30T22:12:00.955 INFO:teuthology.orchestra.run.smithi134.stdout:(117/122): libmount-2.32.1-43.el8.x86_64.rpm 3.1 MB/s | 236 kB 00:00 2023-09-30T22:12:01.022 INFO:teuthology.orchestra.run.smithi134.stdout:(118/122): librados2-16.2.14-44.geeb13ac9.el8.x 4.4 MB/s | 3.7 MB 00:00 2023-09-30T22:12:01.047 INFO:teuthology.orchestra.run.smithi134.stdout:(119/122): libuuid-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 99 kB 00:00 2023-09-30T22:12:01.072 INFO:teuthology.orchestra.run.smithi134.stdout:(120/122): libsmartcols-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 179 kB 00:00 2023-09-30T22:12:01.106 INFO:teuthology.orchestra.run.smithi134.stdout:(121/122): util-linux-user-2.32.1-43.el8.x86_64 1.7 MB/s | 103 kB 00:00 2023-09-30T22:12:01.473 INFO:teuthology.orchestra.run.smithi134.stdout:(122/122): util-linux-2.32.1-43.el8.x86_64.rpm 5.5 MB/s | 2.5 MB 00:00 2023-09-30T22:12:01.477 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:12:01.477 INFO:teuthology.orchestra.run.smithi134.stdout:Total 8.1 MB/s | 200 MB 00:24 2023-09-30T22:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-09-30T22:12:02.424 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-09-30T22:12:02.424 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-09-30T22:12:03.751 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-09-30T22:12:03.754 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-09-30T22:12:05.299 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-09-30T22:12:05.332 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: liboath-2.6.2-3.el8.x86_64 1/1 2023-09-30T22:12:05.525 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/136 2023-09-30T22:12:05.742 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/136 2023-09-30T22:12:05.892 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:12:05.980 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:12:06.150 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:12:06.287 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:12:06.473 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/136 2023-09-30T22:12:06.836 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/136 2023-09-30T22:12:07.008 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:12:07.419 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:12:07.592 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:12:08.030 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:12:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:12:08.355 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:12:08.459 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-09-30T22:12:08.497 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: liboath-2.6.2-3.el8.x86_64 1/1 2023-09-30T22:12:08.558 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:12:08.652 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:12:08.681 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/136 2023-09-30T22:12:08.906 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:12:08.908 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/136 2023-09-30T22:12:09.055 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:12:09.066 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:12:09.133 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:12:09.297 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:12:09.393 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:12:09.466 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:12:09.610 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/136 2023-09-30T22:12:09.633 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:12:09.749 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:12:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/136 2023-09-30T22:12:09.955 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/136 2023-09-30T22:12:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:12:10.198 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/136 2023-09-30T22:12:10.358 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:12:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:12:10.447 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:12:10.573 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:12:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:12:10.739 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:12:10.939 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/136 2023-09-30T22:12:10.968 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:12:11.098 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 19/136 2023-09-30T22:12:11.147 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:12:11.257 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:12:11.274 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:12:11.382 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:12:11.489 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:12:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:12:11.593 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:12:11.803 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:12:11.981 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:12:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:12:12.363 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:12:12.374 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:12:12.497 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:12:12.550 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:12:12.751 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/136 2023-09-30T22:12:12.945 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/136 2023-09-30T22:12:12.991 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:12:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:12:13.199 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:12:13.199 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:12:13.346 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:12:13.463 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:12:13.661 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/136 2023-09-30T22:12:13.879 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 19/136 2023-09-30T22:12:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:12:13.996 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:12:14.132 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:12:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/136 2023-09-30T22:12:14.280 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:12:14.519 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/136 2023-09-30T22:12:14.738 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 26/136 2023-09-30T22:12:14.880 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:12:14.921 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 27/136 2023-09-30T22:12:15.063 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:12:15.137 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 28/136 2023-09-30T22:12:15.332 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:12:15.666 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:12:15.806 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:12:16.494 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:12:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/136 2023-09-30T22:12:17.017 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/136 2023-09-30T22:12:17.109 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:12:17.252 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 26/136 2023-09-30T22:12:17.416 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 27/136 2023-09-30T22:12:17.436 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 30/136 2023-09-30T22:12:17.634 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 28/136 2023-09-30T22:12:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/136 2023-09-30T22:12:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:12:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/136 2023-09-30T22:12:18.192 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 33/136 2023-09-30T22:12:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 34/136 2023-09-30T22:12:18.648 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 35/136 2023-09-30T22:12:18.829 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 36/136 2023-09-30T22:12:19.057 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 37/136 2023-09-30T22:12:19.323 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:12:19.407 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:12:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:12:19.623 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:12:19.695 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:12:19.906 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:12:19.924 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 30/136 2023-09-30T22:12:20.113 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/136 2023-09-30T22:12:20.341 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/136 2023-09-30T22:12:20.440 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:12:20.614 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 33/136 2023-09-30T22:12:20.765 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:12:20.780 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:12:20.783 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 34/136 2023-09-30T22:12:20.959 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 42/136 2023-09-30T22:12:21.018 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 35/136 2023-09-30T22:12:21.211 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 36/136 2023-09-30T22:12:21.232 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 43/136 2023-09-30T22:12:21.429 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 37/136 2023-09-30T22:12:21.504 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 44/136 2023-09-30T22:12:21.629 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:12:21.703 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:12:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 45/136 2023-09-30T22:12:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 46/136 2023-09-30T22:12:21.870 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:12:21.932 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:12:22.089 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 47/136 2023-09-30T22:12:22.229 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:12:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 48/136 2023-09-30T22:12:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/136 2023-09-30T22:12:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/136 2023-09-30T22:12:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/136 2023-09-30T22:12:22.927 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:12:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 52/136 2023-09-30T22:12:23.188 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:12:23.202 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:12:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 53/136 2023-09-30T22:12:23.389 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 42/136 2023-09-30T22:12:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 54/136 2023-09-30T22:12:23.637 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 43/136 2023-09-30T22:12:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 55/136 2023-09-30T22:12:23.935 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 44/136 2023-09-30T22:12:23.990 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 56/136 2023-09-30T22:12:24.116 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 45/136 2023-09-30T22:12:24.216 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 57/136 2023-09-30T22:12:24.272 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 46/136 2023-09-30T22:12:24.447 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 58/136 2023-09-30T22:12:24.478 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 47/136 2023-09-30T22:12:24.687 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/136 2023-09-30T22:12:24.687 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 48/136 2023-09-30T22:12:24.889 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/136 2023-09-30T22:12:25.050 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/136 2023-09-30T22:12:25.067 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/136 2023-09-30T22:12:25.236 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/136 2023-09-30T22:12:25.342 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 61/136 2023-09-30T22:12:25.411 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 52/136 2023-09-30T22:12:25.574 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/136 2023-09-30T22:12:25.671 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 53/136 2023-09-30T22:12:25.834 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 63/136 2023-09-30T22:12:25.902 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 54/136 2023-09-30T22:12:26.043 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 64/136 2023-09-30T22:12:26.098 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 55/136 2023-09-30T22:12:26.346 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 56/136 2023-09-30T22:12:26.580 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 57/136 2023-09-30T22:12:26.811 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 58/136 2023-09-30T22:12:27.043 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/136 2023-09-30T22:12:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 65/136 2023-09-30T22:12:27.437 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/136 2023-09-30T22:12:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 61/136 2023-09-30T22:12:27.731 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 66/136 2023-09-30T22:12:27.921 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/136 2023-09-30T22:12:27.939 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 67/136 2023-09-30T22:12:28.139 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 68/136 2023-09-30T22:12:28.175 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 63/136 2023-09-30T22:12:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 64/136 2023-09-30T22:12:28.425 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/136 2023-09-30T22:12:28.630 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 70/136 2023-09-30T22:12:28.911 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 71/136 2023-09-30T22:12:29.135 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 72/136 2023-09-30T22:12:29.282 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 73/136 2023-09-30T22:12:29.419 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 65/136 2023-09-30T22:12:29.459 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 74/136 2023-09-30T22:12:29.661 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 66/136 2023-09-30T22:12:29.761 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 75/136 2023-09-30T22:12:29.919 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 67/136 2023-09-30T22:12:30.009 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 76/136 2023-09-30T22:12:30.137 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 68/136 2023-09-30T22:12:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 77/136 2023-09-30T22:12:30.380 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/136 2023-09-30T22:12:30.441 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 78/136 2023-09-30T22:12:30.614 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 70/136 2023-09-30T22:12:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 79/136 2023-09-30T22:12:30.843 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.7-17.el8.x86_64 80/136 2023-09-30T22:12:30.915 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 71/136 2023-09-30T22:12:31.124 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 72/136 2023-09-30T22:12:31.144 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 81/136 2023-09-30T22:12:31.263 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 73/136 2023-09-30T22:12:31.388 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 82/136 2023-09-30T22:12:31.448 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 74/136 2023-09-30T22:12:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 83/136 2023-09-30T22:12:31.726 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 75/136 2023-09-30T22:12:31.769 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 84/136 2023-09-30T22:12:31.964 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 76/136 2023-09-30T22:12:32.089 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 77/136 2023-09-30T22:12:32.098 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-2.6-1.el8.noarch 85/136 2023-09-30T22:12:32.307 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 78/136 2023-09-30T22:12:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/136 2023-09-30T22:12:32.494 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 79/136 2023-09-30T22:12:32.538 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 87/136 2023-09-30T22:12:32.687 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.7-17.el8.x86_64 80/136 2023-09-30T22:12:33.034 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 81/136 2023-09-30T22:12:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 82/136 2023-09-30T22:12:33.406 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 83/136 2023-09-30T22:12:33.599 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 84/136 2023-09-30T22:12:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-2.6-1.el8.noarch 85/136 2023-09-30T22:12:34.153 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/136 2023-09-30T22:12:34.302 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 87/136 2023-09-30T22:12:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:12:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:12:36.112 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:12:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:12:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:12:36.745 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:12:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:12:36.993 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:12:37.693 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:12:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:12:38.128 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:12:38.268 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:12:55.246 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:12:55.247 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2023-09-30T22:12:55.247 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2023-09-30T22:12:55.247 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-09-30T22:12:55.247 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /run 2023-09-30T22:12:55.247 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2023-09-30T22:12:55.247 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2023-09-30T22:12:55.247 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2023-09-30T22:12:55.248 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2023-09-30T22:12:55.248 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-09-30T22:12:55.248 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:12:55.689 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:12:56.127 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:12:56.310 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:12:56.538 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /run 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-09-30T22:12:56.539 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:12:56.949 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:12:57.383 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:12:57.573 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:12:57.787 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:12:58.059 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:12:58.094 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:12:58.319 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 94/136 2023-09-30T22:12:58.681 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 95/136 2023-09-30T22:12:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/136 2023-09-30T22:12:59.053 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:12:59.254 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/136 2023-09-30T22:12:59.290 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:12:59.324 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:12:59.540 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 94/136 2023-09-30T22:12:59.704 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/136 2023-09-30T22:12:59.872 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 95/136 2023-09-30T22:12:59.909 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:12:59.984 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:13:00.154 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/136 2023-09-30T22:13:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : jq-1.6-7.el8.x86_64 100/136 2023-09-30T22:13:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 101/136 2023-09-30T22:13:00.380 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/136 2023-09-30T22:13:00.608 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/136 2023-09-30T22:13:00.773 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:13:00.853 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:13:01.059 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.6-7.el8.x86_64 100/136 2023-09-30T22:13:01.179 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 101/136 2023-09-30T22:13:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 102/136 2023-09-30T22:13:01.732 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:13:02.241 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 102/136 2023-09-30T22:13:02.557 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:13:02.769 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:13:02.974 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:13:03.063 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:13:03.457 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 105/136 2023-09-30T22:13:03.593 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:13:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:13:03.725 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:13:03.780 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:13:03.870 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:13:03.899 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:13:03.911 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:13:04.281 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 105/136 2023-09-30T22:13:04.507 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:13:04.532 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:13:04.697 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:13:04.708 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:13:08.577 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 108/136 2023-09-30T22:13:09.364 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 108/136 2023-09-30T22:13:09.841 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:13:10.115 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:13:10.501 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:13:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:13:10.724 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:13:10.818 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:13:10.881 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:13:11.283 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:13:11.479 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:13:11.573 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:13:11.703 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 112/136 2023-09-30T22:13:12.003 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 113/136 2023-09-30T22:13:12.246 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 114/136 2023-09-30T22:13:12.425 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 112/136 2023-09-30T22:13:12.444 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 115/136 2023-09-30T22:13:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 116/136 2023-09-30T22:13:12.692 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 113/136 2023-09-30T22:13:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 117/136 2023-09-30T22:13:12.903 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 114/136 2023-09-30T22:13:13.066 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 115/136 2023-09-30T22:13:13.172 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 118/136 2023-09-30T22:13:13.278 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 116/136 2023-09-30T22:13:13.450 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 119/136 2023-09-30T22:13:13.486 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 117/136 2023-09-30T22:13:13.644 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:13:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:13:13.811 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 118/136 2023-09-30T22:13:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:13:14.060 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:13:14.098 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 119/136 2023-09-30T22:13:14.237 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:13:14.307 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:13:14.311 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:13:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:13:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/136 2023-09-30T22:13:14.512 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:13:14.598 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:13:14.674 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/136 2023-09-30T22:13:14.675 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:13:14.821 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:13:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:13:14.958 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:13:14.964 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:13:15.005 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/136 2023-09-30T22:13:15.109 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:13:15.109 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:13:15.207 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/136 2023-09-30T22:13:15.207 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:13:15.354 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:13:15.447 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:13:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:13:15.567 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/136 2023-09-30T22:13:15.574 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:13:15.574 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:13:15.767 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/136 2023-09-30T22:13:15.860 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:13:16.041 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:13:16.041 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:13:16.042 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/136 2023-09-30T22:13:16.204 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/136 2023-09-30T22:13:16.269 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:13:16.269 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:13:16.447 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:13:16.447 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:13:16.467 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:13:16.467 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:13:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:13:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:13:16.725 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:13:16.725 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:13:16.872 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:13:16.873 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:13:16.941 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:13:16.942 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:13:17.056 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:13:17.056 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:13:17.175 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:13:17.247 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:13:17.248 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:13:17.373 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:13:17.423 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:13:17.621 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:13:17.987 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:13:17.987 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 1/136 2023-09-30T22:13:17.988 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2/136 2023-09-30T22:13:17.988 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 3/136 2023-09-30T22:13:17.988 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 4/136 2023-09-30T22:13:17.988 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 5/136 2023-09-30T22:13:17.988 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 6/136 2023-09-30T22:13:17.988 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 7/136 2023-09-30T22:13:17.988 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:13:17.988 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 12/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 14/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 15/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 16/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 17/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 18/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 19/136 2023-09-30T22:13:17.989 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 20/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 21/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 22/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 23/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 24/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 25/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 26/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 27/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 28/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 29/136 2023-09-30T22:13:17.990 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 30/136 2023-09-30T22:13:18.000 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 31/136 2023-09-30T22:13:18.000 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 32/136 2023-09-30T22:13:18.000 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.0.4-10.el8.x86_64 33/136 2023-09-30T22:13:18.000 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.1-9.el8.x86_64 34/136 2023-09-30T22:13:18.000 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : jq-1.6-7.el8.x86_64 35/136 2023-09-30T22:13:18.000 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 36/136 2023-09-30T22:13:18.000 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 37/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 38/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 39/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 40/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 41/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 42/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 43/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 44/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 45/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 46/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 47/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/136 2023-09-30T22:13:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 49/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 50/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 51/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 52/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 53/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 54/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 55/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 56/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 57/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 59/136 2023-09-30T22:13:18.002 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 60/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 61/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 62/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 63/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 64/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 65/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 66/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 67/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 68/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 69/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 70/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 71/136 2023-09-30T22:13:18.003 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 72/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 73/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 78/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 79/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : leveldb-1.22-1.el8.x86_64 80/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 81/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 82/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/136 2023-09-30T22:13:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 84/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 85/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 86/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 87/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 88/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 89/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 90/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 91/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 92/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 93/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 94/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 95/136 2023-09-30T22:13:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-2.6-1.el8.noarch 96/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 97/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 98/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 99/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 100/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 101/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 102/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 103/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 104/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 105/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 106/136 2023-09-30T22:13:18.006 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 107/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 108/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 110/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 112/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 113/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 114/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 115/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 117/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/136 2023-09-30T22:13:18.007 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 119/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 121/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 123/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 125/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 127/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 129/136 2023-09-30T22:13:18.008 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:13:18.009 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 131/136 2023-09-30T22:13:18.009 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:13:18.009 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 133/136 2023-09-30T22:13:18.009 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:13:18.009 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 135/136 2023-09-30T22:13:18.194 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:13:18.194 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 1/136 2023-09-30T22:13:18.194 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2/136 2023-09-30T22:13:18.194 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 3/136 2023-09-30T22:13:18.194 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 4/136 2023-09-30T22:13:18.194 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 5/136 2023-09-30T22:13:18.194 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 6/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 7/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 12/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 14/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 15/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 16/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 17/136 2023-09-30T22:13:18.195 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 18/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 19/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 20/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 21/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 22/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 23/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 24/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 25/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 26/136 2023-09-30T22:13:18.196 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 27/136 2023-09-30T22:13:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 28/136 2023-09-30T22:13:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 29/136 2023-09-30T22:13:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 30/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 31/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 32/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.0.4-10.el8.x86_64 33/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.1-9.el8.x86_64 34/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.6-7.el8.x86_64 35/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 36/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 37/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 38/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 39/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 40/136 2023-09-30T22:13:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 41/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 42/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 43/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 44/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 45/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 46/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 47/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 49/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 50/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 51/136 2023-09-30T22:13:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 52/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 53/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 54/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 55/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 56/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 57/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 59/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 60/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 61/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 62/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 63/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 64/136 2023-09-30T22:13:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 65/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 66/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 67/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 68/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 69/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 70/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 71/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 72/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 73/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/136 2023-09-30T22:13:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 78/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 79/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : leveldb-1.22-1.el8.x86_64 80/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 81/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 82/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 84/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 85/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 86/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 87/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 88/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 89/136 2023-09-30T22:13:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 90/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 91/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 92/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 93/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 94/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 95/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-2.6-1.el8.noarch 96/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 97/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 98/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 99/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 100/136 2023-09-30T22:13:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 101/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 102/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 103/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 104/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 105/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 106/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 107/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 108/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 110/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 112/136 2023-09-30T22:13:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 113/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 114/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 115/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 117/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 119/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 121/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 123/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/136 2023-09-30T22:13:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 125/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 127/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 129/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 131/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 133/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:13:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 135/136 2023-09-30T22:13:20.035 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 136/136 2023-09-30T22:13:20.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: iproute-6.2.0-2.el8.x86_64 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: libblkid-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: libmount-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: util-linux-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.037 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.038 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.039 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-09-30T22:13:20.039 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T22:13:20.039 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.7-17.el8.x86_64 2023-09-30T22:13:20.039 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T22:13:20.039 INFO:teuthology.orchestra.run.smithi089.stdout: jq-1.6-7.el8.x86_64 2023-09-30T22:13:20.039 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: leveldb-1.22-1.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.040 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.15-4.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-09-30T22:13:20.041 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T22:13:20.042 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T22:13:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.0.6-14.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T22:13:20.044 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2017.2-11.el8.noarch 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-09-30T22:13:20.045 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.046 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.047 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T22:13:20.047 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T22:13:20.047 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-09-30T22:13:20.047 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:20.047 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-09-30T22:13:20.149 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 136/136 2023-09-30T22:13:20.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:20.149 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2023-09-30T22:13:20.149 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: iproute-6.2.0-2.el8.x86_64 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: libblkid-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: libmount-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.150 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.151 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.7-17.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.6-7.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: leveldb-1.22-1.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.153 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.154 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.15-4.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-09-30T22:13:20.155 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T22:13:20.156 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.0.6-14.el8.noarch 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T22:13:20.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T22:13:20.158 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2017.2-11.el8.noarch 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T22:13:20.159 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T22:13:20.160 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T22:13:20.161 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-09-30T22:13:20.161 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:20.161 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-09-30T22:13:20.310 DEBUG:teuthology.parallel:result is None 2023-09-30T22:13:20.353 DEBUG:teuthology.parallel:result is None 2023-09-30T22:13:20.353 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:13:20.354 INFO:teuthology.packaging:ref: None 2023-09-30T22:13:20.354 INFO:teuthology.packaging:tag: None 2023-09-30T22:13:20.354 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:13:20.354 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:13:20.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:13:20.649 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:13:20.674 INFO:teuthology.orchestra.run.smithi089.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:13:20.675 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:13:20.675 INFO:teuthology.task.install:The correct ceph version 16.2.14-44.geeb13ac9 is installed. 2023-09-30T22:13:20.676 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:13:20.676 INFO:teuthology.packaging:ref: None 2023-09-30T22:13:20.676 INFO:teuthology.packaging:tag: None 2023-09-30T22:13:20.676 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:13:20.677 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:13:20.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:13:20.976 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:13:21.000 INFO:teuthology.orchestra.run.smithi134.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:13:21.000 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:13:21.000 INFO:teuthology.task.install:The correct ceph version 16.2.14-44.geeb13ac9 is installed. 2023-09-30T22:13:21.001 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-30T22:13:21.001 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:21.001 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T22:13:21.032 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:13:21.032 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T22:13:21.072 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-30T22:13:21.072 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:21.073 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T22:13:21.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T22:13:21.167 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:13:21.167 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T22:13:21.194 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T22:13:21.261 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-30T22:13:21.262 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:21.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T22:13:21.288 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T22:13:21.356 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:13:21.357 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T22:13:21.384 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T22:13:21.452 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-30T22:13:21.452 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:21.452 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T22:13:21.478 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T22:13:21.545 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:13:21.545 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T22:13:21.573 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T22:13:21.642 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:13:21.652 INFO:teuthology.task.print:**** done installing pacific 2023-09-30T22:13:21.652 INFO:teuthology.run_tasks:Running task ceph... 2023-09-30T22:13:21.719 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-30T22:13:21.719 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /var/log/ceph 2023-09-30T22:13:21.721 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2023-09-30T22:13:21.749 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-30T22:13:21.749 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T22:13:21.788 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T22:13:21.816 INFO:tasks.ceph:Creating extra log directories... 2023-09-30T22:13:21.817 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T22:13:21.855 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T22:13:21.893 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-30T22:13:21.893 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-30T22:13:21.894 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035', 'branch': 'fix-nfs-apply-err-reporting', 'description': 'fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/fsstress}}}', 'email': None, 'first_in_suite': False, 'job_id': '7408035', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'fix-nfs-apply-err-reporting'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62953', 'sha1': '6973b24e3d99fd494b753449d2ea12077599cbd6'}}, 'owner': 'scheduled_dparmar@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0']], 'seed': 2683, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'sleep_before_teardown': 0, 'suite': 'fs:upgrade', 'suite_branch': 'wip-62953', 'suite_path': '/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/dparmar18/ceph', 'suite_sha1': '6973b24e3d99fd494b753449d2ea12077599cbd6', 'targets': {'smithi089.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHT6UO4ioFtlIcI3t7G0ooNN2C7WcTF/HmMzTJpxs4Rz3xSMq8jsklewTvcYjzA7i+VieIsSZql1hajeiQ40DdU=', 'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGySYt+c1ONBPxgHRkiQTLkeNpBlx2n5G1xBpVB2OApOEJ7I2MiR9s06RV4zXlQcYi699ouZk420QaSwHM7Enqc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'pacific', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}}, {'print': '**** done installing pacific'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client pacific']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'mds_pre_upgrade': None}, {'print': '**** done mds pre-upgrade sequence'}, {'install.upgrade': {'mon.a': {'branch': 'reef'}}}, {'print': '**** done install.upgrade the host'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'ceph.stop': ['mds.*']}, {'ceph.restart': {'daemons': ['mds.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph osd dump -f json-pretty', 'ceph versions', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done', 'ceph osd require-osd-release reef']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}, {'ceph-fuse': {'client.0': {'mounted': False}}}, {'print': '**** done unmount client.0'}, {'install.upgrade': {'client.0': None}}, {'print': '**** done install.upgrade on client.0'}, {'ceph-fuse': {'client.0': None}}, {'print': '**** done remount client.0'}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}, 'timeout': '6h'}}], 'teuthology': {'branch': 'pacific', 'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end", 'if not is_fuse() then reject() end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-30_21:53:45', 'tube': 'smithi', 'user': 'dparmar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-30T22:13:21.895 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T22:13:21.923 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T22:13:21.942 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T22:13:21.980 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T22:13:22.009 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:22.009 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T22:13:22.063 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T22:13:22.063 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2023-09-30T22:13:22.122 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T22:13:22.122 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:13:22.122 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 396069 Links: 1 2023-09-30T22:13:22.122 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:13:22.122 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:13:22.122 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-09-30 22:13:17.636390648 +0000 2023-09-30T22:13:22.123 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-09-30 22:10:24.837078466 +0000 2023-09-30T22:13:22.123 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-09-30 22:10:24.837078466 +0000 2023-09-30T22:13:22.123 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-09-30T22:13:22.123 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T22:13:22.188 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-09-30T22:13:22.188 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-09-30T22:13:22.188 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000199154 s, 2.6 MB/s 2023-09-30T22:13:22.189 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T22:13:22.248 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2023-09-30T22:13:22.306 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T22:13:22.307 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:13:22.307 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 388720 Links: 1 2023-09-30T22:13:22.307 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:13:22.307 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:13:22.307 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-09-30 22:13:17.636390648 +0000 2023-09-30T22:13:22.307 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-09-30 22:10:25.340067440 +0000 2023-09-30T22:13:22.307 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-09-30 22:10:25.340067440 +0000 2023-09-30T22:13:22.307 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-09-30T22:13:22.308 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T22:13:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-09-30T22:13:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-09-30T22:13:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000212892 s, 2.4 MB/s 2023-09-30T22:13:22.373 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T22:13:22.432 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2023-09-30T22:13:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T22:13:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:13:22.491 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 392515 Links: 1 2023-09-30T22:13:22.491 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:13:22.491 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:13:22.491 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-09-30 22:13:17.636390648 +0000 2023-09-30T22:13:22.491 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-09-30 22:10:25.852056216 +0000 2023-09-30T22:13:22.491 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-09-30 22:10:25.852056216 +0000 2023-09-30T22:13:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-09-30T22:13:22.491 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T22:13:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-09-30T22:13:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-09-30T22:13:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000208401 s, 2.5 MB/s 2023-09-30T22:13:22.558 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T22:13:22.617 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2023-09-30T22:13:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T22:13:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:13:22.676 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 391768 Links: 1 2023-09-30T22:13:22.676 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:13:22.676 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:13:22.676 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-09-30 22:13:17.636390648 +0000 2023-09-30T22:13:22.676 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-09-30 22:10:26.366044949 +0000 2023-09-30T22:13:22.676 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-09-30 22:10:26.366044949 +0000 2023-09-30T22:13:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-09-30T22:13:22.676 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T22:13:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-09-30T22:13:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-09-30T22:13:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000212355 s, 2.4 MB/s 2023-09-30T22:13:22.742 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T22:13:22.801 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:13:22.801 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-09-30T22:13:22.801 INFO:tasks.ceph:Generating config... 2023-09-30T22:13:22.802 INFO:tasks.ceph:[global] mon warn on pool no app = False 2023-09-30T22:13:22.802 INFO:tasks.ceph:[global] ms bind msgr2 = False 2023-09-30T22:13:22.802 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2023-09-30T22:13:22.802 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2023-09-30T22:13:22.802 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2023-09-30T22:13:22.803 INFO:tasks.ceph:[client] client mount timeout = 600 2023-09-30T22:13:22.803 INFO:tasks.ceph:[client] debug client = 20 2023-09-30T22:13:22.803 INFO:tasks.ceph:[client] debug ms = 1 2023-09-30T22:13:22.803 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-09-30T22:13:22.803 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] debug mds = 20 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] debug ms = 1 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] mds debug frag = True 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-09-30T22:13:22.803 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-09-30T22:13:22.804 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-09-30T22:13:22.804 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-30T22:13:22.804 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-30T22:13:22.804 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-30T22:13:22.804 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-30T22:13:22.804 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-30T22:13:22.804 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-09-30T22:13:22.804 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2023-09-30T22:13:22.804 INFO:tasks.ceph:[osd] bdev async discard = True 2023-09-30T22:13:22.804 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-09-30T22:13:22.804 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-09-30T22:13:22.804 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-30T22:13:22.804 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-30T22:13:22.805 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-09-30T22:13:22.805 INFO:tasks.ceph:Setting up mon.a... 2023-09-30T22:13:22.806 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-30T22:13:22.942 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.keyring 2023-09-30T22:13:22.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-30T22:13:22.986 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:13:23.053 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.89'), ('mon.b', '[v2:172.21.15.89:3301,v1:172.21.15.89:6790]'), ('mon.c', '[v2:172.21.15.89:3302,v1:172.21.15.89:6791]')] 2023-09-30T22:13:23.053 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.89,[v2:172.21.15.89:3301,v1:172.21.15.89:6790],[v2:172.21.15.89:3302,v1:172.21.15.89:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-09-30T22:13:23.054 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:23.054 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T22:13:23.111 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.89 --addv b '[v2:172.21.15.89:3301,v1:172.21.15.89:6790]' --addv c '[v2:172.21.15.89:3302,v1:172.21.15.89:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: generated fsid 389f6e95-ea58-47bd-a145-778ba0724a9e 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 0 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:fsid 389f6e95-ea58-47bd-a145-778ba0724a9e 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:last_changed 2023-09-30T22:13:23.196788+0000 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:created 2023-09-30T22:13:23.196788+0000 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:min_mon_release 0 (unknown) 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:election_strategy: 1 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2023-09-30T22:13:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:1: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2023-09-30T22:13:23.197 INFO:teuthology.orchestra.run.smithi089.stdout:2: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2023-09-30T22:13:23.197 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-30T22:13:23.198 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T22:13:23.255 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 389f6e95-ea58-47bd-a145-778ba0724a9e... 2023-09-30T22:13:23.256 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T22:13:23.298 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T22:13:23.340 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2023-09-30T22:13:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: chdir = "" 2023-09-30T22:13:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T22:13:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: auth supported = cephx 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: filestore xattr use omap = true 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: mon clock drift allowed = 1.000 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: osd crush chooseleaf type = 0 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: auth debug = true 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on old message = true 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on bug = true 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn max object skew = 0 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default size = 2 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow primary affinity = true 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow pg remap = true 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on legacy crush tunables = false 2023-09-30T22:13:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on crush straw calc version zero = false 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on no sortbitwise = false 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = false 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on too few osds = false 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: mon_allow_pool_size_one = true 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: osd default data pool replay window = 5 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: mon allow pool delete = true 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: mon cluster log file level = debug 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: debug asserts on shutdown = true 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: mon health detail to clog = false 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: mon host = "172.21.15.89,[v2:172.21.15.89:3301,v1:172.21.15.89:6790],[v2:172.21.15.89:3302,v1:172.21.15.89:6791]" 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on pool no app = False 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: ms bind msgr2 = False 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore warn on legacy statfs = False 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore warn on no per pool omap = False 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn min per osd = 0 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = 389f6e95-ea58-47bd-a145-778ba0724a9e 2023-09-30T22:13:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout:[osd] 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd journal size = 100 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub load threshold = 5.0 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub max interval = 600 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd mclock profile = high_recovery_ops 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd recover clone overlap = true 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd recovery max chunk = 1048576 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug shutdown = true 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug op order = true 2023-09-30T22:13:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify stray on activate = true 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: osd open classes on start = true 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug pg log writeout = true 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: osd deep scrub update digest min age = 30 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: osd map max advance = 10 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: journal zero on create = true 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: filestore ondisk finisher threads = 3 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: filestore apply finisher threads = 3 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: bdev debug aio = true 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug misdirected ops = true 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: bdev async discard = True 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: bdev enable discard = True 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore allocator = bitmap 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore block size = 96636764160 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore fsck on mount = True 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluefs = 1/20 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluestore = 1/20 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-09-30T22:13:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: debug osd = 20 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: debug rocksdb = 4/10 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd full ratio = 0.9 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd nearfull ratio = 0.8 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: osd failsafe full ratio = 0.95 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: osd objectstore = bluestore 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: osd op complaint time = 180 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr] 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: debug mgr = 20 2023-09-30T22:13:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: mgr/telemetry/nag = false 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout:[mon] 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: debug paxos = 20 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: mon data avail warn = 5 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: mon mgr mkfs grace = 240 2023-09-30T22:13:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd reporter subtree level = osd 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd prime pg temp = true 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T22:13:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: # 1m isn't quite enough 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: mon_down_mkfs_grace = 2m 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_filestore_osds = false 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: mon op complaint time = 120 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = False 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout:[client] 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: rgw cache enabled = true 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable ops log = true 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable usage log = true 2023-09-30T22:13:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: client mount timeout = 600 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: debug client = 20 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: rados mon op timeout = 900 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: rados osd op timeout = 900 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.a] 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.b] 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.c] 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a] 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout:[mds] 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: debug mds = 20 2023-09-30T22:13:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: debug mds balancer = 20 2023-09-30T22:13:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-09-30T22:13:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: mds debug frag = True 2023-09-30T22:13:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: mds debug scatterstat = True 2023-09-30T22:13:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: mds op complaint time = 180 2023-09-30T22:13:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: mds verify scatter = True 2023-09-30T22:13:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: osd op complaint time = 180 2023-09-30T22:13:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: rados mon op timeout = 900 2023-09-30T22:13:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: rados osd op timeout = 900 2023-09-30T22:13:23.354 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2023-09-30T22:13:23.354 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2023-09-30T22:13:23.354 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T22:13:23.354 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2023-09-30T22:13:23.354 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.354 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2023-09-30T22:13:23.354 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.354 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T22:13:23.355 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T22:13:23.356 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.89,[v2:172.21.15.89:3301,v1:172.21.15.89:6790],[v2:172.21.15.89:3302,v1:172.21.15.89:6791]" 2023-09-30T22:13:23.357 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on pool no app = False 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: ms bind msgr2 = False 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore warn on legacy statfs = False 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore warn on no per pool omap = False 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn min per osd = 0 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = 389f6e95-ea58-47bd-a145-778ba0724a9e 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2023-09-30T22:13:23.358 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.359 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2023-09-30T22:13:23.360 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2023-09-30T22:13:23.361 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.362 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.363 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: mon op complaint time = 120 2023-09-30T22:13:23.364 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = False 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: client mount timeout = 600 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2023-09-30T22:13:23.365 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds = 20 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds balancer = 20 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug frag = True 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = True 2023-09-30T22:13:23.366 INFO:teuthology.orchestra.run.smithi134.stdout: mds op complaint time = 180 2023-09-30T22:13:23.367 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2023-09-30T22:13:23.367 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2023-09-30T22:13:23.367 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2023-09-30T22:13:23.367 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2023-09-30T22:13:23.368 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-30T22:13:23.368 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-30T22:13:23.460 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-30T22:13:23.461 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:23.461 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-30T22:13:23.478 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:23.478 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-30T22:13:23.536 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:13:23.536 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:23.536 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T22:13:23.537 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:13:23.612 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:23.612 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:23.670 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2023-09-30T22:13:23.670 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:13:23.670 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T22:13:23.670 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:13:23.714 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:13:23.714 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:23.774 INFO:tasks.ceph:Setting up mon nodes... 2023-09-30T22:13:23.774 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-30T22:13:23.775 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T22:13:23.823 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T22:13:23.825 INFO:tasks.ceph:Setting up mds nodes... 2023-09-30T22:13:23.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-09-30T22:13:23.874 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-09-30T22:13:23.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-09-30T22:13:23.903 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-30T22:13:23.903 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-30T22:13:24.027 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-30T22:13:24.041 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-30T22:13:24.042 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-09-30T22:13:24.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-30T22:13:24.068 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:13:24.068 INFO:tasks.ceph:role: osd.0 2023-09-30T22:13:24.069 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:13:24.069 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-30T22:13:24.142 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:13:24.143 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:13:24.580 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-09-30T22:13:24.581 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-09-30T22:13:24.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-30T22:13:24.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-30T22:13:24.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-30T22:13:24.843 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:13:24.843 INFO:tasks.ceph:role: osd.1 2023-09-30T22:13:24.843 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:13:24.843 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:13:24.918 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:13:24.919 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:13:25.342 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-09-30T22:13:25.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-09-30T22:13:25.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-30T22:13:25.459 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-30T22:13:25.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-30T22:13:25.557 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:13:25.557 INFO:tasks.ceph:role: osd.2 2023-09-30T22:13:25.557 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:13:25.558 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-30T22:13:25.631 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:13:25.632 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:13:26.085 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-09-30T22:13:26.087 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-09-30T22:13:26.087 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-30T22:13:26.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-30T22:13:26.264 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-30T22:13:26.331 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:13:26.331 INFO:tasks.ceph:role: osd.3 2023-09-30T22:13:26.331 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:13:26.331 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:13:26.406 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:13:26.847 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-09-30T22:13:26.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-09-30T22:13:26.849 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-30T22:13:27.033 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-30T22:13:27.067 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:27.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:27.166+0000 7f4ff4d2f200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-30T22:13:27.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:27.167+0000 7f4ff4d2f200 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-30T22:13:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:27.167+0000 7f4ff4d2f200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:13:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:27.167+0000 7f4ff4d2f200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:13:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:27.167+0000 7f4ff4d2f200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:13:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:27.167+0000 7f4ff4d2f200 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-30T22:13:30.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-30T22:13:30.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:30.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:30.129+0000 7f4e351d4200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-30T22:13:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:30.130+0000 7f4e351d4200 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-30T22:13:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:30.130+0000 7f4e351d4200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:13:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:30.130+0000 7f4e351d4200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:13:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:30.130+0000 7f4e351d4200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:13:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:30.130+0000 7f4e351d4200 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-30T22:13:32.971 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-30T22:13:33.001 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:33.087+0000 7f77ca7ea200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-30T22:13:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:33.087+0000 7f77ca7ea200 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-30T22:13:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:33.087+0000 7f77ca7ea200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:13:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:33.087+0000 7f77ca7ea200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:13:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:33.087+0000 7f77ca7ea200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:13:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:33.088+0000 7f77ca7ea200 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-30T22:13:35.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-30T22:13:35.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:36.044+0000 7f02d2c18200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-30T22:13:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:36.044+0000 7f02d2c18200 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-30T22:13:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:36.044+0000 7f02d2c18200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:13:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:36.044+0000 7f02d2c18200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:13:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:36.044+0000 7f02d2c18200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:13:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:36.045+0000 7f02d2c18200 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-30T22:13:38.890 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-30T22:13:38.922 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-30T22:13:38.923 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:38.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-30T22:13:38.993 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:38.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-09-30T22:13:39.063 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:39.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-30T22:13:39.132 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:39.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-30T22:13:39.200 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:39.200 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-30T22:13:39.268 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:39.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-30T22:13:39.337 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:13:39.337 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-30T22:13:39.359 INFO:tasks.ceph:Adding keys to all mons... 2023-09-30T22:13:39.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-30T22:13:39.403 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.x] 2023-09-30T22:13:39.403 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQCDnRhlEV0oMRAAcYNh7a51Ei1qvJ1hymn5SA== 2023-09-30T22:13:39.403 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a] 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQCDnRhlXMc1NBAATu7OFqZkwugKCkLWbP3VAA== 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.0] 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQCHnRhlh674CRAAPpCGFZmX4dzUIO8KdJpiNQ== 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.1] 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQCKnRhlav/CBxAA0NUKTCIvnwK+AzSeIg4pYg== 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.2] 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQCNnRhlM7U/BRAAeu4IbqmtilQve1suWUVPMA== 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.3] 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQCQnRhlFai7AhAAmTYH9RGz2csX2UoXxdsRNg== 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2023-09-30T22:13:39.404 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQCEnRhlzU62ARAAGYvPL5W/CxMtiQatBTd6FQ== 2023-09-30T22:13:39.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T22:13:39.494 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-30T22:13:39.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:13:39.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:13:39.642 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:13:39.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:13:39.740 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T22:13:39.792 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-30T22:13:39.792 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-30T22:13:39.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:13:39.912 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-30T22:13:39.939 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-30T22:13:40.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:13:40.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-30T22:13:40.127 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-30T22:13:40.194 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:13:40.288 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-30T22:13:40.318 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:13:40.376 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-30T22:13:40.376 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-30T22:13:40.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-30T22:13:40.418 INFO:tasks.ceph.mon.a:Started 2023-09-30T22:13:40.419 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-30T22:13:40.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-30T22:13:40.421 INFO:tasks.ceph.mon.b:Started 2023-09-30T22:13:40.421 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-30T22:13:40.421 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-30T22:13:40.423 INFO:tasks.ceph.mon.c:Started 2023-09-30T22:13:40.423 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-30T22:13:40.423 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-30T22:13:40.423 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-30T22:13:40.425 INFO:tasks.ceph.mgr.x:Started 2023-09-30T22:13:40.425 INFO:tasks.ceph:Setting crush tunables to default 2023-09-30T22:13:40.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd crush tunables default 2023-09-30T22:13:40.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.569+0000 7f338cdc6700 1 Processor -- start 2023-09-30T22:13:40.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.569+0000 7f338cdc6700 1 -- start start 2023-09-30T22:13:40.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.569+0000 7f338cdc6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:40.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.569+0000 7f338cdc6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:40.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.569+0000 7f338cdc6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3388066d80 con 0x7f33880613a0 2023-09-30T22:13:40.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.569+0000 7f338cdc6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3388061780 con 0x7f3388130400 2023-09-30T22:13:40.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.569+0000 7f338cdc6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338812ac40 con 0x7f338812b540 2023-09-30T22:13:40.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3385d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:40.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f338659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f33880613a0 0x7f338812ae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41356/0 (socket says 172.21.15.89:41356) 2023-09-30T22:13:40.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3386d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 learned_addr learned my addr 172.21.15.89:0/3913330200 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3386d9d700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3386d9d700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:13:40.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.570+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3386d9d700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3386d9d700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:13:40.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:40.771+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:13:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:13:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3386d9d700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:13:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3386d9d700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:13:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.172+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:13:41.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:41.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:41.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3386d9d700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:13:41.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3386d9d700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:41.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:41.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:13:41.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:41.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:41.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:13:41.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:41.973+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:13:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f3377fff700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 msgr2=0x7f338812ff10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:13:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f3377fff700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f3377fff700 1 -- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 msgr2=0x7f33881327f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:13:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f3377fff700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f33881327f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f3377fff700 1 -- 172.21.15.89:0/3913330200 >> v1:172.21.15.89:6789/0 conn(0x7f33880613a0 legacy=0x7f338812ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f3377fff700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f3377fff700 1 -- 172.21.15.89:0/3913330200 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3388061780 con 0x7f3368003540 2023-09-30T22:13:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:13:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.570+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:13:43.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.771+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:43.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.771+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:13:43.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.771+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:43.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.771+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:43.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:43.771+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:13:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.172+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.172+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:13:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.172+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.172+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.172+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:13:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.973+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.973+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:13:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.973+0000 7f338659c700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:13:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.973+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:13:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:44.973+0000 7f338659c700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:13:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3377fff700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f33680059b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:13:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3377fff700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f33680059b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3377fff700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f336800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3377fff700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f336800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3377fff700 1 -- 172.21.15.89:0/3913330200 --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3388092fb0 con 0x7f33880613a0 2023-09-30T22:13:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3377fff700 1 -- 172.21.15.89:0/3913330200 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3388061780 con 0x7f3388130400 2023-09-30T22:13:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3377fff700 1 -- 172.21.15.89:0/3913330200 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3368005a80 con 0x7f3368003540 2023-09-30T22:13:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> v2:172.21.15.89:3300/0 conn(0x7f3388130400 0x7f336800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.570+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f336800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 msgr2=0x7f336800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f336800bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 >> v1:172.21.15.89:6789/0 conn(0x7f33880613a0 legacy=0x7f3368008220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f3385d9b700 1 -- 172.21.15.89:0/3913330200 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3368005c40 con 0x7f3388130400 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f338559a700 1 -- 172.21.15.89:0/3913330200 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170604527 0 0) 0x7f3388092fb0 con 0x7f33880613a0 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f3386d9d700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f336800bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f3385d9b700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3388130400 0x7f336800b990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f338809a330 tx=0x7f337c008040).ready entity=mon.0 client_cookie=61e02e69028ed1fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f338559a700 1 -- 172.21.15.89:0/3913330200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c011450 con 0x7f3388130400 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f338559a700 1 -- 172.21.15.89:0/3913330200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c013510 con 0x7f3388130400 2023-09-30T22:13:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.571+0000 7f338559a700 1 -- 172.21.15.89:0/3913330200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c013890 con 0x7f3388130400 2023-09-30T22:13:46.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 -- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3388130400 msgr2=0x7f336800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:46.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3388130400 0x7f336800b990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f338809a330 tx=0x7f337c008040).stop 2023-09-30T22:13:46.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 -- 172.21.15.89:0/3913330200 shutdown_connections 2023-09-30T22:13:46.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3388130400 0x7f336800b990 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3368003540 0x7f336800bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 --2- 172.21.15.89:0/3913330200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f338812b540 0x7f338812ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 -- 172.21.15.89:0/3913330200 >> 172.21.15.89:0/3913330200 conn(0x7f338809a3d0 msgr2=0x7f338809a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 -- 172.21.15.89:0/3913330200 shutdown_connections 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.572+0000 7f338cdc6700 1 -- 172.21.15.89:0/3913330200 wait complete. 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f338cdc6700 1 Processor -- start 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f338cdc6700 1 -- start start 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f338cdc6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3368003540 0x7f338812a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f338cdc6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33880613a0 0x7f33881145c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f338cdc6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f338812b540 0x7f3388114ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f338cdc6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3388117d20 con 0x7f3368003540 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f338cdc6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33881177d0 con 0x7f33880613a0 2023-09-30T22:13:46.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f338cdc6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3388099db0 con 0x7f338812b540 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f3385d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33880613a0 0x7f33881145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f3386d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f338812b540 0x7f3388114ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f3385d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33880613a0 0x7f33881145c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56628/0 (socket says 172.21.15.89:56628) 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.573+0000 7f3385d9b700 1 -- 172.21.15.89:0/2823716113 learned_addr learned my addr 172.21.15.89:0/2823716113 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338659c700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3368003540 0x7f338812a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338659c700 1 -- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f338812b540 msgr2=0x7f3388114ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338659c700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f338812b540 0x7f3388114ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338659c700 1 -- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33880613a0 msgr2=0x7f33881145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338659c700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33880613a0 0x7f33881145c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338659c700 1 -- 172.21.15.89:0/2823716113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3380009ae0 con 0x7f3368003540 2023-09-30T22:13:46.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f3385d9b700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33880613a0 0x7f33881145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f3386d9d700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f338812b540 0x7f3388114ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338659c700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3368003540 0x7f338812a5e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f3378000ff0 tx=0x7f3378009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f33767fc700 1 -- 172.21.15.89:0/2823716113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337801e070 con 0x7f3368003540 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f33767fc700 1 -- 172.21.15.89:0/2823716113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33780014e0 con 0x7f3368003540 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f33767fc700 1 -- 172.21.15.89:0/2823716113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378001880 con 0x7f3368003540 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337c00e040 con 0x7f3368003540 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.574+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33881160f0 con 0x7f3368003540 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.575+0000 7f33767fc700 1 -- 172.21.15.89:0/2823716113 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f3378023420 con 0x7f3368003540 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.575+0000 7f33767fc700 1 -- 172.21.15.89:0/2823716113 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3378001c70 con 0x7f3368003540 2023-09-30T22:13:46.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.576+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338805a260 con 0x7f3368003540 2023-09-30T22:13:46.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.578+0000 7f33767fc700 1 -- 172.21.15.89:0/2823716113 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f33780238b0 con 0x7f3368003540 2023-09-30T22:13:46.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.673+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3388093670 con 0x7f3368003540 2023-09-30T22:13:46.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.677+0000 7f33767fc700 1 -- 172.21.15.89:0/2823716113 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3378023a50 con 0x7f3368003540 2023-09-30T22:13:46.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.678+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3368003540 msgr2=0x7f338812a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:46.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.678+0000 7f338cdc6700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3368003540 0x7f338812a5e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f3378000ff0 tx=0x7f3378009040).stop 2023-09-30T22:13:46.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.678+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 shutdown_connections 2023-09-30T22:13:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.678+0000 7f338cdc6700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f338812b540 0x7f3388114ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.678+0000 7f338cdc6700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3368003540 0x7f338812a5e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.678+0000 7f338cdc6700 1 --2- 172.21.15.89:0/2823716113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33880613a0 0x7f33881145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.678+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 >> 172.21.15.89:0/2823716113 conn(0x7f338809a3d0 msgr2=0x7f338812f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.678+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 shutdown_connections 2023-09-30T22:13:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:46.679+0000 7f338cdc6700 1 -- 172.21.15.89:0/2823716113 wait complete. 2023-09-30T22:13:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2023-09-30T22:13:46.690 INFO:tasks.ceph:check_enable_crimson: False 2023-09-30T22:13:46.690 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-30T22:13:46.691 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:46.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-30T22:13:46.694 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:46.693+0000 7f28874a4680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T22:13:46.717 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:46.718 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-30T22:13:46.766 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:46.766+0000 7f28874a4680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T22:13:46.785 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:46.785 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-30T22:13:46.852 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:13:46.853 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-30T22:13:46.855 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:46.854+0000 7f28874a4680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T22:13:46.881 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 96fbca95-1fa5-4074-9fbb-d4f206bb1aa6 0 2023-09-30T22:13:47.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72fc2f700 1 Processor -- start 2023-09-30T22:13:47.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72fc2f700 1 -- start start 2023-09-30T22:13:47.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72fc2f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280613a0 0x7ff72812ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72fc2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff72812b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72fc2f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7280919e0 0x7ff728091e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72fc2f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff728061780 con 0x7ff7280919e0 2023-09-30T22:13:47.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72fc2f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72812ac40 con 0x7ff72812b320 2023-09-30T22:13:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72fc2f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff728066d80 con 0x7ff7280613a0 2023-09-30T22:13:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72d1ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff72812b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72d9cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280613a0 0x7ff72812ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72e1cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7280919e0 0x7ff728091e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72d9cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280613a0 0x7ff72812ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60232/0 (socket says 172.21.15.89:60232) 2023-09-30T22:13:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72d1ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff72812b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56642/0 (socket says 172.21.15.89:56642) 2023-09-30T22:13:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72d9cb700 1 -- 172.21.15.89:0/1283423310 learned_addr learned my addr 172.21.15.89:0/1283423310 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.037+0000 7ff72e1cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7280919e0 0x7ff728091e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40958/0 (socket says 172.21.15.89:40958) 2023-09-30T22:13:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72d9cb700 1 -- 172.21.15.89:0/1283423310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 msgr2=0x7ff72812b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72d9cb700 1 --2- 172.21.15.89:0/1283423310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff72812b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72d9cb700 1 -- 172.21.15.89:0/1283423310 >> v2:172.21.15.89:3300/0 conn(0x7ff7280919e0 msgr2=0x7ff728091e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72d9cb700 1 --2- 172.21.15.89:0/1283423310 >> v2:172.21.15.89:3300/0 conn(0x7ff7280919e0 0x7ff728091e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72d9cb700 1 -- 172.21.15.89:0/1283423310 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff728092fb0 con 0x7ff7280613a0 2023-09-30T22:13:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72e1cc700 1 --2- 172.21.15.89:0/1283423310 >> v2:172.21.15.89:3300/0 conn(0x7ff7280919e0 0x7ff728091e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72d9cb700 1 --2- 172.21.15.89:0/1283423310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280613a0 0x7ff72812ae30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff724006fd0 tx=0x7ff7240090f0).ready entity=mon.2 client_cookie=71e6be4e8016eb45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72c9c9700 1 -- 172.21.15.89:0/1283423310 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400f510 con 0x7ff7280613a0 2023-09-30T22:13:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72c9c9700 1 -- 172.21.15.89:0/1283423310 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72400f670 con 0x7ff7280613a0 2023-09-30T22:13:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72c9c9700 1 -- 172.21.15.89:0/1283423310 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400f9f0 con 0x7ff7280613a0 2023-09-30T22:13:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72fc2f700 1 -- 172.21.15.89:0/1283423310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280613a0 msgr2=0x7ff72812ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72fc2f700 1 --2- 172.21.15.89:0/1283423310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280613a0 0x7ff72812ae30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff724006fd0 tx=0x7ff7240090f0).stop 2023-09-30T22:13:47.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72fc2f700 1 -- 172.21.15.89:0/1283423310 shutdown_connections 2023-09-30T22:13:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72fc2f700 1 --2- 172.21.15.89:0/1283423310 >> v2:172.21.15.89:3300/0 conn(0x7ff7280919e0 0x7ff728091e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72fc2f700 1 --2- 172.21.15.89:0/1283423310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280613a0 0x7ff72812ae30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72fc2f700 1 --2- 172.21.15.89:0/1283423310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff72812b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.038+0000 7ff72fc2f700 1 -- 172.21.15.89:0/1283423310 >> 172.21.15.89:0/1283423310 conn(0x7ff72809a3d0 msgr2=0x7ff72809a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.039+0000 7ff72fc2f700 1 -- 172.21.15.89:0/1283423310 shutdown_connections 2023-09-30T22:13:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.039+0000 7ff72fc2f700 1 -- 172.21.15.89:0/1283423310 wait complete. 2023-09-30T22:13:47.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.039+0000 7ff72fc2f700 1 Processor -- start 2023-09-30T22:13:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.039+0000 7ff72fc2f700 1 -- start start 2023-09-30T22:13:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72fc2f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7280613a0 0x7ff72813a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72fc2f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280919e0 0x7ff72813aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72fc2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff728114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72fc2f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff728066f90 con 0x7ff7280613a0 2023-09-30T22:13:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72fc2f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7281161d0 con 0x7ff72812b320 2023-09-30T22:13:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72fc2f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff72812f330 con 0x7ff7280919e0 2023-09-30T22:13:47.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d9cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7280613a0 0x7ff72813a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d1ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280919e0 0x7ff72813aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72e1cc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff728114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d9cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7280613a0 0x7ff72813a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40974/0 (socket says 172.21.15.89:40974) 2023-09-30T22:13:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d9cb700 1 -- 172.21.15.89:0/3008733750 learned_addr learned my addr 172.21.15.89:0/3008733750 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d1ca700 1 -- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 msgr2=0x7ff728114d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d1ca700 1 --2- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff728114d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d1ca700 1 -- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7280613a0 msgr2=0x7ff72813a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d1ca700 1 --2- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7280613a0 0x7ff72813a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d1ca700 1 -- 172.21.15.89:0/3008733750 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009a20 con 0x7ff7280919e0 2023-09-30T22:13:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d9cb700 1 --2- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7280613a0 0x7ff72813a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72d1ca700 1 --2- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280919e0 0x7ff72813aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7ff71800e900 tx=0x7ff718009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff71dffb700 1 -- 172.21.15.89:0/3008733750 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718009470 con 0x7ff7280919e0 2023-09-30T22:13:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff71dffb700 1 -- 172.21.15.89:0/3008733750 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718012600 con 0x7ff7280919e0 2023-09-30T22:13:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff724009370 con 0x7ff7280919e0 2023-09-30T22:13:47.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.040+0000 7ff71dffb700 1 -- 172.21.15.89:0/3008733750 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718012980 con 0x7ff7280919e0 2023-09-30T22:13:47.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.041+0000 7ff71dffb700 1 -- 172.21.15.89:0/3008733750 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7ff71801b400 con 0x7ff7280919e0 2023-09-30T22:13:47.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.041+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff728118f90 con 0x7ff7280919e0 2023-09-30T22:13:47.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.041+0000 7ff71dffb700 1 -- 172.21.15.89:0/3008733750 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff71801a3b0 con 0x7ff7280919e0 2023-09-30T22:13:47.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.042+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff728118270 con 0x7ff7280919e0 2023-09-30T22:13:47.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.044+0000 7ff71dffb700 1 -- 172.21.15.89:0/3008733750 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7ff71803ae50 con 0x7ff7280919e0 2023-09-30T22:13:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.123+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd new", "uuid": "96fbca95-1fa5-4074-9fbb-d4f206bb1aa6", "id": 0} v 0) v1 -- 0x7ff72805a260 con 0x7ff7280919e0 2023-09-30T22:13:47.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.128+0000 7ff71dffb700 1 -- 172.21.15.89:0/3008733750 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "96fbca95-1fa5-4074-9fbb-d4f206bb1aa6", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff718029030 con 0x7ff7280919e0 2023-09-30T22:13:47.129 INFO:teuthology.orchestra.run.smithi089.stdout:0 2023-09-30T22:13:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280919e0 msgr2=0x7ff72813aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 --2- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280919e0 0x7ff72813aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7ff71800e900 tx=0x7ff718009060).stop 2023-09-30T22:13:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 shutdown_connections 2023-09-30T22:13:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 --2- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72812b320 0x7ff728114d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 --2- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7280613a0 0x7ff72813a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 --2- 172.21.15.89:0/3008733750 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7280919e0 0x7ff72813aa90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 >> 172.21.15.89:0/3008733750 conn(0x7ff72809a3d0 msgr2=0x7ff728068470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 shutdown_connections 2023-09-30T22:13:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.129+0000 7ff72fc2f700 1 -- 172.21.15.89:0/3008733750 wait complete. 2023-09-30T22:13:47.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new bca437d5-1c37-4cda-a2f2-86f7957e0ac0 1 2023-09-30T22:13:47.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.258+0000 7f522c0a1700 1 Processor -- start 2023-09-30T22:13:47.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.258+0000 7f522c0a1700 1 -- start start 2023-09-30T22:13:47.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f522c0a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5224059070 0x7f5224127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f522c0a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f52240953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f522c0a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5224092a50 con 0x7f52240958e0 2023-09-30T22:13:47.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f522c0a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5224091ba0 con 0x7f5224059070 2023-09-30T22:13:47.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f522c0a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5224091e60 con 0x7f5224127d80 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f5229e3d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5224059070 0x7f5224127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f522963c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f52240953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f5229e3d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5224059070 0x7f5224127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40978/0 (socket says 172.21.15.89:40978) 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f5229e3d700 1 -- 172.21.15.89:0/2207765643 learned_addr learned my addr 172.21.15.89:0/2207765643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f522963c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f52240953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56652/0 (socket says 172.21.15.89:56652) 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f5229e3d700 1 -- 172.21.15.89:0/2207765643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 msgr2=0x7f52240953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f5229e3d700 1 --2- 172.21.15.89:0/2207765643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f52240953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f5229e3d700 1 -- 172.21.15.89:0/2207765643 >> v1:172.21.15.89:6789/0 conn(0x7f52240958e0 legacy=0x7f5224131d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f5228e3b700 1 -- 172.21.15.89:0/2207765643 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438992847 0 0) 0x7f5224092a50 con 0x7f52240958e0 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f5229e3d700 1 -- 172.21.15.89:0/2207765643 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224097220 con 0x7f5224059070 2023-09-30T22:13:47.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.259+0000 7f522963c700 1 --2- 172.21.15.89:0/2207765643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f52240953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.260+0000 7f5229e3d700 1 --2- 172.21.15.89:0/2207765643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 0x7f5224127890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5220006fd0 tx=0x7f52200090f0).ready entity=mon.0 client_cookie=539dda65f264afda server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.260+0000 7f5228e3b700 1 -- 172.21.15.89:0/2207765643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000b580 con 0x7f5224059070 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.260+0000 7f5228e3b700 1 -- 172.21.15.89:0/2207765643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522000b6e0 con 0x7f5224059070 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.260+0000 7f5228e3b700 1 -- 172.21.15.89:0/2207765643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000ba60 con 0x7f5224059070 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.260+0000 7f522c0a1700 1 -- 172.21.15.89:0/2207765643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 msgr2=0x7f5224127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.260+0000 7f522c0a1700 1 --2- 172.21.15.89:0/2207765643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 0x7f5224127890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5220006fd0 tx=0x7f52200090f0).stop 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.260+0000 7f522c0a1700 1 -- 172.21.15.89:0/2207765643 shutdown_connections 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.260+0000 7f522c0a1700 1 --2- 172.21.15.89:0/2207765643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 0x7f5224127890 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.261+0000 7f522c0a1700 1 --2- 172.21.15.89:0/2207765643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f52240953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.261+0000 7f522c0a1700 1 -- 172.21.15.89:0/2207765643 >> 172.21.15.89:0/2207765643 conn(0x7f52240974f0 msgr2=0x7f5224097900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.261+0000 7f522c0a1700 1 -- 172.21.15.89:0/2207765643 shutdown_connections 2023-09-30T22:13:47.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.261+0000 7f522c0a1700 1 -- 172.21.15.89:0/2207765643 wait complete. 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.261+0000 7f522c0a1700 1 Processor -- start 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.261+0000 7f522c0a1700 1 -- start start 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522c0a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 0x7f5224117a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522c0a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52240958e0 0x7f5224113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522c0a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f5224113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522c0a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224124b80 con 0x7f5224059070 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522c0a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5224097010 con 0x7f5224127d80 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522c0a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5224116fb0 con 0x7f52240958e0 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522963c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52240958e0 0x7f5224113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f5229e3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 0x7f5224117a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522963c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52240958e0 0x7f5224113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60254/0 (socket says 172.21.15.89:60254) 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522963c700 1 -- 172.21.15.89:0/3743759225 learned_addr learned my addr 172.21.15.89:0/3743759225 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522a63e700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f5224113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f5229e3d700 1 -- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52240958e0 msgr2=0x7f5224113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f5229e3d700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52240958e0 0x7f5224113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f5229e3d700 1 -- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 msgr2=0x7f5224113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f5229e3d700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f5224113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f5229e3d700 1 -- 172.21.15.89:0/3743759225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52140092a0 con 0x7f5224059070 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522a63e700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f5224113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f522963c700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52240958e0 0x7f5224113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.262+0000 7f5229e3d700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 0x7f5224117a50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f5220014000 tx=0x7f5220017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:47.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.263+0000 7f5219ffb700 1 -- 172.21.15.89:0/3743759225 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000b850 con 0x7f5224059070 2023-09-30T22:13:47.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.263+0000 7f5219ffb700 1 -- 172.21.15.89:0/3743759225 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522000b9b0 con 0x7f5224059070 2023-09-30T22:13:47.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.263+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5220009370 con 0x7f5224059070 2023-09-30T22:13:47.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.263+0000 7f5219ffb700 1 -- 172.21.15.89:0/3743759225 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522001b660 con 0x7f5224059070 2023-09-30T22:13:47.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.263+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5224115ea0 con 0x7f5224059070 2023-09-30T22:13:47.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.263+0000 7f5219ffb700 1 -- 172.21.15.89:0/3743759225 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f522001bcf0 con 0x7f5224059070 2023-09-30T22:13:47.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.263+0000 7f5219ffb700 1 -- 172.21.15.89:0/3743759225 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f5220024a20 con 0x7f5224059070 2023-09-30T22:13:47.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.264+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522405a260 con 0x7f5224059070 2023-09-30T22:13:47.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.266+0000 7f5219ffb700 1 -- 172.21.15.89:0/3743759225 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f5220029060 con 0x7f5224059070 2023-09-30T22:13:47.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.346+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bca437d5-1c37-4cda-a2f2-86f7957e0ac0", "id": 1} v 0) v1 -- 0x7f5224046c20 con 0x7f5224059070 2023-09-30T22:13:47.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.354+0000 7f5219ffb700 1 -- 172.21.15.89:0/3743759225 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bca437d5-1c37-4cda-a2f2-86f7957e0ac0", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f522001f070 con 0x7f5224059070 2023-09-30T22:13:47.354 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-09-30T22:13:47.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 msgr2=0x7f5224117a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 0x7f5224117a50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f5220014000 tx=0x7f5220017000).stop 2023-09-30T22:13:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 shutdown_connections 2023-09-30T22:13:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5224127d80 0x7f5224113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5224059070 0x7f5224117a50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 --2- 172.21.15.89:0/3743759225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52240958e0 0x7f5224113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 >> 172.21.15.89:0/3743759225 conn(0x7f52240974f0 msgr2=0x7f5224094430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 shutdown_connections 2023-09-30T22:13:47.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.355+0000 7f522c0a1700 1 -- 172.21.15.89:0/3743759225 wait complete. 2023-09-30T22:13:47.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 0fb339c9-0000-43f3-bcbc-9cc5b8b65359 2 2023-09-30T22:13:47.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.481+0000 7f440ecbc700 1 Processor -- start 2023-09-30T22:13:47.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.481+0000 7f440ecbc700 1 -- start start 2023-09-30T22:13:47.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.481+0000 7f440ecbc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f4408130850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.481+0000 7f440ecbc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408097150 0x7f4408132d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.481+0000 7f440ecbc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4408097db0 con 0x7f4408065fe0 2023-09-30T22:13:47.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.481+0000 7f440ecbc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4408097940 con 0x7f4408096d30 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.481+0000 7f440ecbc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4408064a00 con 0x7f4408097150 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f4407fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f4408130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440d259700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408097150 0x7f4408132d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440ca58700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4408065fe0 0x7f4408096620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41402/0 (socket says 172.21.15.89:41402) 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f4407fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f4408130850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56684/0 (socket says 172.21.15.89:56684) 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440ca58700 1 -- 172.21.15.89:0/1782014076 learned_addr learned my addr 172.21.15.89:0/1782014076 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440d259700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408097150 0x7f4408132d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60264/0 (socket says 172.21.15.89:60264) 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440d259700 1 -- 172.21.15.89:0/1782014076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 msgr2=0x7f4408130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440d259700 1 --2- 172.21.15.89:0/1782014076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f4408130850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440d259700 1 -- 172.21.15.89:0/1782014076 >> v1:172.21.15.89:6789/0 conn(0x7f4408065fe0 legacy=0x7f4408096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f44077fe700 1 -- 172.21.15.89:0/1782014076 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365396210 0 0) 0x7f4408097db0 con 0x7f4408065fe0 2023-09-30T22:13:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440d259700 1 -- 172.21.15.89:0/1782014076 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408092790 con 0x7f4408097150 2023-09-30T22:13:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f4407fff700 1 --2- 172.21.15.89:0/1782014076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f4408130850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f440d259700 1 --2- 172.21.15.89:0/1782014076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408097150 0x7f4408132d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f4400009e90 tx=0x7f4400009780).ready entity=mon.2 client_cookie=24a3635d3313f8c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f44077fe700 1 -- 172.21.15.89:0/1782014076 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000e070 con 0x7f4408097150 2023-09-30T22:13:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f44077fe700 1 -- 172.21.15.89:0/1782014076 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440000bb00 con 0x7f4408097150 2023-09-30T22:13:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.482+0000 7f44077fe700 1 -- 172.21.15.89:0/1782014076 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000be80 con 0x7f4408097150 2023-09-30T22:13:47.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.483+0000 7f440ecbc700 1 -- 172.21.15.89:0/1782014076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408097150 msgr2=0x7f4408132d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.483+0000 7f440ecbc700 1 --2- 172.21.15.89:0/1782014076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408097150 0x7f4408132d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f4400009e90 tx=0x7f4400009780).stop 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.483+0000 7f440ecbc700 1 -- 172.21.15.89:0/1782014076 shutdown_connections 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.483+0000 7f440ecbc700 1 --2- 172.21.15.89:0/1782014076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408097150 0x7f4408132d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.483+0000 7f440ecbc700 1 --2- 172.21.15.89:0/1782014076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f4408130850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.483+0000 7f440ecbc700 1 -- 172.21.15.89:0/1782014076 >> 172.21.15.89:0/1782014076 conn(0x7f4408098720 msgr2=0x7f4408098b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.483+0000 7f440ecbc700 1 -- 172.21.15.89:0/1782014076 shutdown_connections 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.483+0000 7f440ecbc700 1 -- 172.21.15.89:0/1782014076 wait complete. 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ecbc700 1 Processor -- start 2023-09-30T22:13:47.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ecbc700 1 -- start start 2023-09-30T22:13:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ecbc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408065fe0 0x7f440813a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ecbc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f440813a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ecbc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4408097150 0x7f440813aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ecbc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440811de60 con 0x7f4408097150 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ecbc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4408097ab0 con 0x7f4408096d30 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ecbc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f440809c730 con 0x7f4408065fe0 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f4407fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f440813a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440ca58700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408065fe0 0x7f440813a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f440d259700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4408097150 0x7f440813aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f4407fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f440813a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56690/0 (socket says 172.21.15.89:56690) 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.484+0000 7f4407fff700 1 -- 172.21.15.89:0/1541860191 learned_addr learned my addr 172.21.15.89:0/1541860191 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440ca58700 1 -- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 msgr2=0x7f440813a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440ca58700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f440813a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440ca58700 1 -- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4408097150 msgr2=0x7f440813aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440ca58700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4408097150 0x7f440813aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440ca58700 1 -- 172.21.15.89:0/1541860191 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f40092a0 con 0x7f4408065fe0 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f4407fff700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f440813a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440d259700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4408097150 0x7f440813aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440ca58700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408065fe0 0x7f440813a0f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f43fc00c460 tx=0x7f43fc007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f4404ff9700 1 -- 172.21.15.89:0/1541860191 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0070d0 con 0x7f4408065fe0 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f4404ff9700 1 -- 172.21.15.89:0/1541860191 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc007250 con 0x7f4408065fe0 2023-09-30T22:13:47.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4400009a20 con 0x7f4408065fe0 2023-09-30T22:13:47.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f4404ff9700 1 -- 172.21.15.89:0/1541860191 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc007610 con 0x7f4408065fe0 2023-09-30T22:13:47.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.485+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440811e670 con 0x7f4408065fe0 2023-09-30T22:13:47.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.486+0000 7f4404ff9700 1 -- 172.21.15.89:0/1541860191 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f43fc005c10 con 0x7f4408065fe0 2023-09-30T22:13:47.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.486+0000 7f4404ff9700 1 -- 172.21.15.89:0/1541860191 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f43fc013bf0 con 0x7f4408065fe0 2023-09-30T22:13:47.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.486+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440805a260 con 0x7f4408065fe0 2023-09-30T22:13:47.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.488+0000 7f4404ff9700 1 -- 172.21.15.89:0/1541860191 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f440811e670 con 0x7f4408065fe0 2023-09-30T22:13:47.592 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:47.591+0000 7f28874a4680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T22:13:47.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.601+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd new", "uuid": "0fb339c9-0000-43f3-bcbc-9cc5b8b65359", "id": 2} v 0) v1 -- 0x7f440811da50 con 0x7f4408065fe0 2023-09-30T22:13:47.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.609+0000 7f4404ff9700 1 -- 172.21.15.89:0/1541860191 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0fb339c9-0000-43f3-bcbc-9cc5b8b65359", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f43fc0137c0 con 0x7f4408065fe0 2023-09-30T22:13:47.610 INFO:teuthology.orchestra.run.smithi089.stdout:2 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408065fe0 msgr2=0x7f440813a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408065fe0 0x7f440813a0f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f43fc00c460 tx=0x7f43fc007b90).stop 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 shutdown_connections 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4408097150 0x7f440813aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4408065fe0 0x7f440813a0f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 --2- 172.21.15.89:0/1541860191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4408096d30 0x7f440813a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 >> 172.21.15.89:0/1541860191 conn(0x7f4408098720 msgr2=0x7f44080696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 shutdown_connections 2023-09-30T22:13:47.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.610+0000 7f440ecbc700 1 -- 172.21.15.89:0/1541860191 wait complete. 2023-09-30T22:13:47.622 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 4010d897-d6fa-4f8c-ab79-7415116d796d 3 2023-09-30T22:13:47.704 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:47.703+0000 7f28874a4680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T22:13:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.740+0000 7f12d3e32700 1 Processor -- start 2023-09-30T22:13:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d3e32700 1 -- start start 2023-09-30T22:13:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d3e32700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12cc059070 0x7f12cc127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d3e32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc0953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d3e32700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc092a50 con 0x7f12cc0958e0 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d3e32700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc091ba0 con 0x7f12cc059070 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d3e32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc091e60 con 0x7f12cc127d80 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d13cd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc0953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d1bce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12cc059070 0x7f12cc127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d1bce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12cc059070 0x7f12cc127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41006/0 (socket says 172.21.15.89:41006) 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d23cf700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f12cc0958e0 0x7f12cc131d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41408/0 (socket says 172.21.15.89:41408) 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d1bce700 1 -- 172.21.15.89:0/4058608090 learned_addr learned my addr 172.21.15.89:0/4058608090 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.741+0000 7f12d13cd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc0953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60282/0 (socket says 172.21.15.89:60282) 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d1bce700 1 -- 172.21.15.89:0/4058608090 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 msgr2=0x7f12cc0953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d1bce700 1 --2- 172.21.15.89:0/4058608090 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc0953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d1bce700 1 -- 172.21.15.89:0/4058608090 >> v1:172.21.15.89:6789/0 conn(0x7f12cc0958e0 legacy=0x7f12cc131d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d1bce700 1 -- 172.21.15.89:0/4058608090 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc097220 con 0x7f12cc059070 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d0bcc700 1 -- 172.21.15.89:0/4058608090 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748234503 0 0) 0x7f12cc092a50 con 0x7f12cc0958e0 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d13cd700 1 --2- 172.21.15.89:0/4058608090 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc0953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d1bce700 1 --2- 172.21.15.89:0/4058608090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc059070 0x7f12cc127890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f12c8006fd0 tx=0x7f12c80090f0).ready entity=mon.0 client_cookie=59737f5b7590054d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d0bcc700 1 -- 172.21.15.89:0/4058608090 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800b580 con 0x7f12cc059070 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d0bcc700 1 -- 172.21.15.89:0/4058608090 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c800b6e0 con 0x7f12cc059070 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d0bcc700 1 -- 172.21.15.89:0/4058608090 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800ba60 con 0x7f12cc059070 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d3e32700 1 -- 172.21.15.89:0/4058608090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc059070 msgr2=0x7f12cc127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d3e32700 1 --2- 172.21.15.89:0/4058608090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc059070 0x7f12cc127890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f12c8006fd0 tx=0x7f12c80090f0).stop 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d3e32700 1 -- 172.21.15.89:0/4058608090 shutdown_connections 2023-09-30T22:13:47.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d3e32700 1 --2- 172.21.15.89:0/4058608090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc059070 0x7f12cc127890 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d3e32700 1 --2- 172.21.15.89:0/4058608090 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc0953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.742+0000 7f12d3e32700 1 -- 172.21.15.89:0/4058608090 >> 172.21.15.89:0/4058608090 conn(0x7f12cc0974f0 msgr2=0x7f12cc097900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:47.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.743+0000 7f12d3e32700 1 -- 172.21.15.89:0/4058608090 shutdown_connections 2023-09-30T22:13:47.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.743+0000 7f12d3e32700 1 -- 172.21.15.89:0/4058608090 wait complete. 2023-09-30T22:13:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d3e32700 1 Processor -- start 2023-09-30T22:13:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d3e32700 1 -- start start 2023-09-30T22:13:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d3e32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12cc059070 0x7f12cc117a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d3e32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc0958e0 0x7f12cc113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d3e32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d3e32700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc124b80 con 0x7f12cc0958e0 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d3e32700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc097010 con 0x7f12cc059070 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d3e32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc116fb0 con 0x7f12cc127d80 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d1bce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12cc059070 0x7f12cc117a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.744+0000 7f12d13cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc0958e0 0x7f12cc113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d1bce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12cc059070 0x7f12cc117a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56702/0 (socket says 172.21.15.89:56702) 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d13cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc0958e0 0x7f12cc113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41014/0 (socket says 172.21.15.89:41014) 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d1bce700 1 -- 172.21.15.89:0/897934336 learned_addr learned my addr 172.21.15.89:0/897934336 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d23cf700 1 --2- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d13cd700 1 -- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 msgr2=0x7f12cc113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d13cd700 1 --2- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d13cd700 1 -- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12cc059070 msgr2=0x7f12cc117a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d13cd700 1 --2- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12cc059070 0x7f12cc117a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d13cd700 1 -- 172.21.15.89:0/897934336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc0092a0 con 0x7f12cc0958e0 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d13cd700 1 --2- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc0958e0 0x7f12cc113b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f12bc00e950 tx=0x7f12bc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12c1ffb700 1 -- 172.21.15.89:0/897934336 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00c670 con 0x7f12cc0958e0 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12c1ffb700 1 -- 172.21.15.89:0/897934336 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc00c7d0 con 0x7f12cc0958e0 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12c1ffb700 1 -- 172.21.15.89:0/897934336 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00cb50 con 0x7f12cc0958e0 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c8009370 con 0x7f12cc0958e0 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12c1ffb700 1 -- 172.21.15.89:0/897934336 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f12bc01b660 con 0x7f12cc0958e0 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc115640 con 0x7f12cc0958e0 2023-09-30T22:13:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.745+0000 7f12c1ffb700 1 -- 172.21.15.89:0/897934336 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f12bc01bb40 con 0x7f12cc0958e0 2023-09-30T22:13:47.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.746+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc098a30 con 0x7f12cc0958e0 2023-09-30T22:13:47.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.748+0000 7f12c1ffb700 1 -- 172.21.15.89:0/897934336 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f12bc03ae50 con 0x7f12cc0958e0 2023-09-30T22:13:47.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.829+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4010d897-d6fa-4f8c-ab79-7415116d796d", "id": 3} v 0) v1 -- 0x7f12cc05a260 con 0x7f12cc0958e0 2023-09-30T22:13:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.837+0000 7f12c1ffb700 1 -- 172.21.15.89:0/897934336 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4010d897-d6fa-4f8c-ab79-7415116d796d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f12bc029030 con 0x7f12cc0958e0 2023-09-30T22:13:47.837 INFO:teuthology.orchestra.run.smithi089.stdout:3 2023-09-30T22:13:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc0958e0 msgr2=0x7f12cc113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 --2- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc0958e0 0x7f12cc113b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f12bc00e950 tx=0x7f12bc009060).stop 2023-09-30T22:13:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 shutdown_connections 2023-09-30T22:13:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 --2- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12cc127d80 0x7f12cc113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 --2- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12cc059070 0x7f12cc117a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 --2- 172.21.15.89:0/897934336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12cc0958e0 0x7f12cc113b00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 >> 172.21.15.89:0/897934336 conn(0x7f12cc0974f0 msgr2=0x7f12cc094430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 shutdown_connections 2023-09-30T22:13:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:47.838+0000 7f12d3e32700 1 -- 172.21.15.89:0/897934336 wait complete. 2023-09-30T22:13:47.844 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:47.844+0000 7f28874a4680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T22:13:47.850 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-30T22:13:47.850 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-30T22:13:47.852 INFO:tasks.ceph.osd.0:Started 2023-09-30T22:13:47.852 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-30T22:13:47.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-30T22:13:47.854 INFO:tasks.ceph.osd.1:Started 2023-09-30T22:13:47.854 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-30T22:13:47.854 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-30T22:13:47.856 INFO:tasks.ceph.osd.2:Started 2023-09-30T22:13:47.856 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-30T22:13:47.856 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-30T22:13:47.858 INFO:tasks.ceph.osd.3:Started 2023-09-30T22:13:47.859 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:13:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f29b7c700 1 Processor -- start 2023-09-30T22:13:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f29b7c700 1 -- start start 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f29b7c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f240654a0 0x7f5f24098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f29b7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f24098fa0 0x7f5f24132ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f29b7c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f24065ed0 con 0x7f5f24098b30 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f29b7c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f24095880 con 0x7f5f240654a0 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f29b7c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f24096120 con 0x7f5f24098fa0 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f23fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f24098fa0 0x7f5f24132ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f237fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f240654a0 0x7f5f24098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f237fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f240654a0 0x7f5f24098640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41056/0 (socket says 172.21.15.89:41056) 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f237fe700 1 -- 172.21.15.89:0/2519199030 learned_addr learned my addr 172.21.15.89:0/2519199030 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f23fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f24098fa0 0x7f5f24132ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60326/0 (socket says 172.21.15.89:60326) 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.012+0000 7f5f22ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5f24098b30 0x7f5f2412f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41424/0 (socket says 172.21.15.89:41424) 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f237fe700 1 -- 172.21.15.89:0/2519199030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f24098fa0 msgr2=0x7f5f24132ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f237fe700 1 --2- 172.21.15.89:0/2519199030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f24098fa0 0x7f5f24132ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f237fe700 1 -- 172.21.15.89:0/2519199030 >> v1:172.21.15.89:6789/0 conn(0x7f5f24098b30 legacy=0x7f5f2412f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f227fc700 1 -- 172.21.15.89:0/2519199030 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077127747 0 0) 0x7f5f24065ed0 con 0x7f5f24098b30 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f237fe700 1 -- 172.21.15.89:0/2519199030 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f24066280 con 0x7f5f240654a0 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f23fff700 1 --2- 172.21.15.89:0/2519199030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f24098fa0 0x7f5f24132ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f237fe700 1 --2- 172.21.15.89:0/2519199030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f240654a0 0x7f5f24098640 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f5f1c006fd0 tx=0x7f5f1c0090f0).ready entity=mon.0 client_cookie=10cf330262ff591e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f227fc700 1 -- 172.21.15.89:0/2519199030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00b580 con 0x7f5f240654a0 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f227fc700 1 -- 172.21.15.89:0/2519199030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00b6e0 con 0x7f5f240654a0 2023-09-30T22:13:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f227fc700 1 -- 172.21.15.89:0/2519199030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00ba60 con 0x7f5f240654a0 2023-09-30T22:13:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f29b7c700 1 -- 172.21.15.89:0/2519199030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f240654a0 msgr2=0x7f5f24098640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f29b7c700 1 --2- 172.21.15.89:0/2519199030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f240654a0 0x7f5f24098640 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f5f1c006fd0 tx=0x7f5f1c0090f0).stop 2023-09-30T22:13:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f29b7c700 1 -- 172.21.15.89:0/2519199030 shutdown_connections 2023-09-30T22:13:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f29b7c700 1 --2- 172.21.15.89:0/2519199030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f240654a0 0x7f5f24098640 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.013+0000 7f5f29b7c700 1 --2- 172.21.15.89:0/2519199030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f24098fa0 0x7f5f24132ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.014+0000 7f5f29b7c700 1 -- 172.21.15.89:0/2519199030 >> 172.21.15.89:0/2519199030 conn(0x7f5f2409a740 msgr2=0x7f5f2409ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.014+0000 7f5f29b7c700 1 -- 172.21.15.89:0/2519199030 shutdown_connections 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.014+0000 7f5f29b7c700 1 -- 172.21.15.89:0/2519199030 wait complete. 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f29b7c700 1 Processor -- start 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f29b7c700 1 -- start start 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f29b7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f240654a0 0x7f5f2413a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f29b7c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f24098b30 0x7f5f2413add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f29b7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f24098fa0 0x7f5f2411a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f29b7c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2411dd70 con 0x7f5f24098fa0 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f29b7c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2412f6a0 con 0x7f5f24098b30 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f29b7c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2411d810 con 0x7f5f240654a0 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f237fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f240654a0 0x7f5f2413a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f237fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f240654a0 0x7f5f2413a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60334/0 (socket says 172.21.15.89:60334) 2023-09-30T22:13:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f237fe700 1 -- 172.21.15.89:0/3584797273 learned_addr learned my addr 172.21.15.89:0/3584797273 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f22ffd700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f24098b30 0x7f5f2413add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f23fff700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f24098fa0 0x7f5f2411a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f237fe700 1 -- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f24098b30 msgr2=0x7f5f2413add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f237fe700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f24098b30 0x7f5f2413add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f237fe700 1 -- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f24098fa0 msgr2=0x7f5f2411a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f237fe700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f24098fa0 0x7f5f2411a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f237fe700 1 -- 172.21.15.89:0/3584797273 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14009a20 con 0x7f5f240654a0 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.015+0000 7f5f23fff700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f24098fa0 0x7f5f2411a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.016+0000 7f5f237fe700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f240654a0 0x7f5f2413a8e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5f1c009130 tx=0x7f5f1c017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.016+0000 7f5f1b7fe700 1 -- 172.21.15.89:0/3584797273 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c000bc0 con 0x7f5f240654a0 2023-09-30T22:13:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.016+0000 7f5f1b7fe700 1 -- 172.21.15.89:0/3584797273 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00b850 con 0x7f5f240654a0 2023-09-30T22:13:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.016+0000 7f5f1b7fe700 1 -- 172.21.15.89:0/3584797273 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c000d20 con 0x7f5f240654a0 2023-09-30T22:13:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.016+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c009370 con 0x7f5f240654a0 2023-09-30T22:13:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.016+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2411c8c0 con 0x7f5f240654a0 2023-09-30T22:13:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.016+0000 7f5f1b7fe700 1 -- 172.21.15.89:0/3584797273 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f5f1c025450 con 0x7f5f240654a0 2023-09-30T22:13:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.016+0000 7f5f1b7fe700 1 -- 172.21.15.89:0/3584797273 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f5f1c00ba00 con 0x7f5f240654a0 2023-09-30T22:13:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.017+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2411deb0 con 0x7f5f240654a0 2023-09-30T22:13:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.019+0000 7f5f1b7fe700 1 -- 172.21.15.89:0/3584797273 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f5f1c044e50 con 0x7f5f240654a0 2023-09-30T22:13:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.100+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f2409e8e0 con 0x7f5f240654a0 2023-09-30T22:13:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.100+0000 7f5f1b7fe700 1 -- 172.21.15.89:0/3584797273 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7f5f1c033070 con 0x7f5f240654a0 2023-09-30T22:13:48.101 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:48.101 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":6,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:13:47.831690+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:13:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.101+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f240654a0 msgr2=0x7f5f2413a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.102+0000 7f5f29b7c700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f240654a0 0x7f5f2413a8e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5f1c009130 tx=0x7f5f1c017000).stop 2023-09-30T22:13:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.102+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 shutdown_connections 2023-09-30T22:13:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.102+0000 7f5f29b7c700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f24098fa0 0x7f5f2411a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.102+0000 7f5f29b7c700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f240654a0 0x7f5f2413a8e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.102+0000 7f5f29b7c700 1 --2- 172.21.15.89:0/3584797273 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f24098b30 0x7f5f2413add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.102+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 >> 172.21.15.89:0/3584797273 conn(0x7f5f2409a740 msgr2=0x7f5f241142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.102+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 shutdown_connections 2023-09-30T22:13:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:48.102+0000 7f5f29b7c700 1 -- 172.21.15.89:0/3584797273 wait complete. 2023-09-30T22:13:48.118 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-30T22:13:48.119 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-30T22:13:48.241 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-09-30T22:13:48.240+0000 7ff09cba3200 -1 Falling back to public interface 2023-09-30T22:13:48.325 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:48.325+0000 7f28874a4680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T22:13:48.462 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-09-30T22:13:48.461+0000 7fdd07e85200 -1 Falling back to public interface 2023-09-30T22:13:48.491 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-09-30T22:13:48.490+0000 7f7849d4d200 -1 Falling back to public interface 2023-09-30T22:13:48.491 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-09-30T22:13:48.490+0000 7ff1743a0200 -1 Falling back to public interface 2023-09-30T22:13:48.491 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-09-30T22:13:48.490+0000 7f7843ef5700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6808/0: (98) Address already in use 2023-09-30T22:13:48.498 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:48.498+0000 7f28874a4680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:13:48.522 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T22:13:48.566 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:48.566+0000 7f28874a4680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T22:13:48.630 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.630+0000 7f14b408a700 1 Processor -- start 2023-09-30T22:13:48.631 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.630+0000 7f14b408a700 1 -- start start 2023-09-30T22:13:48.632 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.630+0000 7f14b408a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac1260b0 0x7f14ac1253f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.630+0000 7f14b408a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac130100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.630+0000 7f14b408a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac0634d0 con 0x7f14ac0629b0 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.630+0000 7f14b408a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac062d90 con 0x7f14ac12dd10 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.630+0000 7f14b408a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac06aa10 con 0x7f14ac1260b0 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1625700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac1260b0 0x7f14ac1253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b2627700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac130100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1e26700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f14ac0629b0 0x7f14ac1259a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41428/0 (socket says 172.21.15.89:41428) 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1e26700 1 -- 172.21.15.89:0/3699142112 learned_addr learned my addr 172.21.15.89:0/3699142112 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b2627700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac130100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56768/0 (socket says 172.21.15.89:56768) 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1625700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac1260b0 0x7f14ac1253f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60340/0 (socket says 172.21.15.89:60340) 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1625700 1 -- 172.21.15.89:0/3699142112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 msgr2=0x7f14ac130100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:48.633 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1625700 1 --2- 172.21.15.89:0/3699142112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac130100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1625700 1 -- 172.21.15.89:0/3699142112 >> v1:172.21.15.89:6789/0 conn(0x7f14ac0629b0 legacy=0x7f14ac1259a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b0e24700 1 -- 172.21.15.89:0/3699142112 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230292669 0 0) 0x7f14ac0634d0 con 0x7f14ac0629b0 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1625700 1 -- 172.21.15.89:0/3699142112 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac097330 con 0x7f14ac1260b0 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b2627700 1 --2- 172.21.15.89:0/3699142112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac130100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b1625700 1 --2- 172.21.15.89:0/3699142112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac1260b0 0x7f14ac1253f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f149c009790 tx=0x7f149c009060).ready entity=mon.2 client_cookie=6fad936d9a9f60f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b0e24700 1 -- 172.21.15.89:0/3699142112 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c0090d0 con 0x7f14ac1260b0 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b0e24700 1 -- 172.21.15.89:0/3699142112 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00f510 con 0x7f14ac1260b0 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.631+0000 7f14b0e24700 1 -- 172.21.15.89:0/3699142112 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00f890 con 0x7f14ac1260b0 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.632+0000 7f14b408a700 1 -- 172.21.15.89:0/3699142112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac1260b0 msgr2=0x7f14ac1253f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.632+0000 7f14b408a700 1 --2- 172.21.15.89:0/3699142112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac1260b0 0x7f14ac1253f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f149c009790 tx=0x7f149c009060).stop 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.632+0000 7f14b408a700 1 -- 172.21.15.89:0/3699142112 shutdown_connections 2023-09-30T22:13:48.634 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.632+0000 7f14b408a700 1 --2- 172.21.15.89:0/3699142112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac130100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.635 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.632+0000 7f14b408a700 1 --2- 172.21.15.89:0/3699142112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac1260b0 0x7f14ac1253f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.635 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.632+0000 7f14b408a700 1 -- 172.21.15.89:0/3699142112 >> 172.21.15.89:0/3699142112 conn(0x7f14ac09ab90 msgr2=0x7f14ac09afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:48.635 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.632+0000 7f14b408a700 1 -- 172.21.15.89:0/3699142112 shutdown_connections 2023-09-30T22:13:48.635 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.632+0000 7f14b408a700 1 -- 172.21.15.89:0/3699142112 wait complete. 2023-09-30T22:13:48.637 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b408a700 1 Processor -- start 2023-09-30T22:13:48.637 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b408a700 1 -- start start 2023-09-30T22:13:48.637 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b408a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac0629b0 0x7f14ac0919e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.637 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b408a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ac1260b0 0x7f14ac091ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.637 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b408a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac0923c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b408a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac092b70 con 0x7f14ac1260b0 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b408a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac095e40 con 0x7f14ac12dd10 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b408a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac12a4e0 con 0x7f14ac0629b0 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b2627700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac0923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b2627700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac0923c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56784/0 (socket says 172.21.15.89:56784) 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b1e26700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac0629b0 0x7f14ac0919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b2627700 1 -- 172.21.15.89:0/2873618987 learned_addr learned my addr 172.21.15.89:0/2873618987 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b1625700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ac1260b0 0x7f14ac091ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b2627700 1 -- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac0629b0 msgr2=0x7f14ac0919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b2627700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac0629b0 0x7f14ac0919e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.638 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b2627700 1 -- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ac1260b0 msgr2=0x7f14ac091ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b2627700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ac1260b0 0x7f14ac091ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b2627700 1 -- 172.21.15.89:0/2873618987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4009a20 con 0x7f14ac12dd10 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b1e26700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac0629b0 0x7f14ac0919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.633+0000 7f14b1625700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ac1260b0 0x7f14ac091ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.634+0000 7f14b2627700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac0923c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f14a4009f30 tx=0x7f14a400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.634+0000 7f14a1ffb700 1 -- 172.21.15.89:0/2873618987 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4009d00 con 0x7f14ac12dd10 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.634+0000 7f14a1ffb700 1 -- 172.21.15.89:0/2873618987 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a400f040 con 0x7f14ac12dd10 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.634+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c0092a0 con 0x7f14ac12dd10 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.634+0000 7f14a1ffb700 1 -- 172.21.15.89:0/2873618987 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400cda0 con 0x7f14ac12dd10 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.634+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac129e70 con 0x7f14ac12dd10 2023-09-30T22:13:48.639 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.634+0000 7f14a1ffb700 1 -- 172.21.15.89:0/2873618987 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f14a4013870 con 0x7f14ac12dd10 2023-09-30T22:13:48.640 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.634+0000 7f14a1ffb700 1 -- 172.21.15.89:0/2873618987 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f14a401da60 con 0x7f14ac12dd10 2023-09-30T22:13:48.640 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.635+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac12a330 con 0x7f14ac12dd10 2023-09-30T22:13:48.640 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.637+0000 7f14a1ffb700 1 -- 172.21.15.89:0/2873618987 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f14a401de40 con 0x7f14ac12dd10 2023-09-30T22:13:48.646 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:48.646+0000 7f28874a4680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T22:13:48.704 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.703+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14ac05a260 con 0x7f14ac12dd10 2023-09-30T22:13:48.704 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.703+0000 7f14a1ffb700 1 -- 172.21.15.89:0/2873618987 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7f14a4013dd0 con 0x7f14ac12dd10 2023-09-30T22:13:48.704 INFO:teuthology.misc.health.smithi089.stdout: 2023-09-30T22:13:48.705 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":6,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:13:47.831690+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:13:48.705 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.704+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 msgr2=0x7f14ac0923c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:48.705 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.704+0000 7f14b408a700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac0923c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f14a4009f30 tx=0x7f14a400b040).stop 2023-09-30T22:13:48.706 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.704+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 shutdown_connections 2023-09-30T22:13:48.706 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.704+0000 7f14b408a700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14ac12dd10 0x7f14ac0923c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.706 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.704+0000 7f14b408a700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14ac0629b0 0x7f14ac0919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.706 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.704+0000 7f14b408a700 1 --2- 172.21.15.89:0/2873618987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ac1260b0 0x7f14ac091ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:48.706 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.704+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 >> 172.21.15.89:0/2873618987 conn(0x7f14ac09ab90 msgr2=0x7f14ac1287c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:48.706 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.705+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 shutdown_connections 2023-09-30T22:13:48.706 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:48.705+0000 7f14b408a700 1 -- 172.21.15.89:0/2873618987 wait complete. 2023-09-30T22:13:48.715 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-09-30T22:13:48.717 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:48.717+0000 7f28874a4680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T22:13:49.064 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:49.064+0000 7f28874a4680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T22:13:49.148 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:49.148+0000 7f28874a4680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T22:13:49.774 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:49.774+0000 7f28874a4680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T22:13:49.843 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:49.843+0000 7f28874a4680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T22:13:49.922 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:49.922+0000 7f28874a4680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T22:13:50.208 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:50.207+0000 7f28874a4680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T22:13:50.277 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:50.276+0000 7f28874a4680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T22:13:50.392 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:50.392+0000 7f28874a4680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T22:13:50.499 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:50.499+0000 7f28874a4680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:13:50.648 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:50.647+0000 7f28874a4680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T22:13:50.719 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:50.719+0000 7f28874a4680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T22:13:51.722 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-09-30T22:13:51.721+0000 7ff09cba3200 -1 osd.2 0 log_to_monitors {default=true} 2023-09-30T22:13:51.744 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:51.743+0000 7f283dee4700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-30T22:13:51.746 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:13:51.745+0000 7f283dee4700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-30T22:13:51.919 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-09-30T22:13:51.918+0000 7fdd07e85200 -1 osd.0 0 log_to_monitors {default=true} 2023-09-30T22:13:51.968 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-09-30T22:13:51.967+0000 7ff1743a0200 -1 osd.3 0 log_to_monitors {default=true} 2023-09-30T22:13:52.012 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-09-30T22:13:52.012+0000 7f7849d4d200 -1 osd.1 0 log_to_monitors {default=true} 2023-09-30T22:13:53.523 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-09-30T22:13:53.522+0000 7fdcfe826700 -1 osd.0 0 waiting for initial osdmap 2023-09-30T22:13:53.523 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-09-30T22:13:53.522+0000 7f78406ee700 -1 osd.1 0 waiting for initial osdmap 2023-09-30T22:13:53.528 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-09-30T22:13:53.527+0000 7ff093544700 -1 osd.2 0 waiting for initial osdmap 2023-09-30T22:13:53.528 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-09-30T22:13:53.528+0000 7fdcfafa2700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:13:53.528 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-09-30T22:13:53.528+0000 7ff16ad41700 -1 osd.3 0 waiting for initial osdmap 2023-09-30T22:13:53.530 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-09-30T22:13:53.530+0000 7f783ce6a700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:13:53.531 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-09-30T22:13:53.531+0000 7ff08fcc0700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:13:53.532 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-09-30T22:13:53.531+0000 7ff1674bd700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:13:55.119 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T22:13:55.239 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6292fb700 1 Processor -- start 2023-09-30T22:13:55.239 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6292fb700 1 -- start start 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6292fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624097cc0 0x7fc624127ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6292fb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc624127fc0 0x7fc62412b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6292fb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc624092a10 con 0x7fc624127590 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6292fb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6240980a0 con 0x7fc624127fc0 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6292fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62409a4c0 con 0x7fc624097cc0 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc622ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624097cc0 0x7fc624127ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6237fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc624127fc0 0x7fc62412b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6237fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc624127fc0 0x7fc62412b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42622/0 (socket says 172.21.15.89:42622) 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6237fe700 1 -- 172.21.15.89:0/2921424385 learned_addr learned my addr 172.21.15.89:0/2921424385 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc622ffd700 1 --2- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624097cc0 0x7fc624127ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55880/0 (socket says 172.21.15.89:55880) 2023-09-30T22:13:55.240 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.237+0000 7fc6227fc700 1 --1- 172.21.15.89:0/2921424385 >> v1:172.21.15.89:6789/0 conn(0x7fc624127590 0x7fc62412aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40220/0 (socket says 172.21.15.89:40220) 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6237fe700 1 -- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624097cc0 msgr2=0x7fc624127ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6237fe700 1 --2- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624097cc0 0x7fc624127ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6237fe700 1 -- 172.21.15.89:0/2921424385 >> v1:172.21.15.89:6789/0 conn(0x7fc624127590 legacy=0x7fc62412aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6237fe700 1 -- 172.21.15.89:0/2921424385 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc624128430 con 0x7fc624127fc0 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc621ffb700 1 -- 172.21.15.89:0/2921424385 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166385633 0 0) 0x7fc624092a10 con 0x7fc624127590 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc622ffd700 1 --2- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624097cc0 0x7fc624127ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6237fe700 1 --2- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127fc0 0x7fc62412b180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fc61c009f20 tx=0x7fc61c009840).ready entity=mon.0 client_cookie=56e4596877c3b16f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc621ffb700 1 -- 172.21.15.89:0/2921424385 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00e070 con 0x7fc624127fc0 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc621ffb700 1 -- 172.21.15.89:0/2921424385 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c00b9d0 con 0x7fc624127fc0 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc621ffb700 1 -- 172.21.15.89:0/2921424385 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00bd50 con 0x7fc624127fc0 2023-09-30T22:13:55.241 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6292fb700 1 -- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127fc0 msgr2=0x7fc62412b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.242 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6292fb700 1 --2- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127fc0 0x7fc62412b180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fc61c009f20 tx=0x7fc61c009840).stop 2023-09-30T22:13:55.242 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6292fb700 1 -- 172.21.15.89:0/2921424385 shutdown_connections 2023-09-30T22:13:55.242 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6292fb700 1 --2- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127fc0 0x7fc62412b180 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.242 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6292fb700 1 --2- 172.21.15.89:0/2921424385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624097cc0 0x7fc624127ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.242 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6292fb700 1 -- 172.21.15.89:0/2921424385 >> 172.21.15.89:0/2921424385 conn(0x7fc62409dbf0 msgr2=0x7fc62409e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:55.242 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.238+0000 7fc6292fb700 1 -- 172.21.15.89:0/2921424385 shutdown_connections 2023-09-30T22:13:55.242 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.239+0000 7fc6292fb700 1 -- 172.21.15.89:0/2921424385 wait complete. 2023-09-30T22:13:55.244 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.239+0000 7fc6292fb700 1 Processor -- start 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6292fb700 1 -- start start 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6292fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc624097cc0 0x7fc624097200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6292fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127590 0x7fc62413a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6292fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624127fc0 0x7fc62413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6292fb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc624116d80 con 0x7fc624127590 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6292fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc624115530 con 0x7fc624127fc0 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6292fb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc624099e90 con 0x7fc624097cc0 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6237fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624127fc0 0x7fc62413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6227fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127590 0x7fc62413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6237fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624127fc0 0x7fc62413a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55892/0 (socket says 172.21.15.89:55892) 2023-09-30T22:13:55.245 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6237fe700 1 -- 172.21.15.89:0/3069904819 learned_addr learned my addr 172.21.15.89:0/3069904819 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc622ffd700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc624097cc0 0x7fc624097200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6227fc700 1 -- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc624097cc0 msgr2=0x7fc624097200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6227fc700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc624097cc0 0x7fc624097200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6227fc700 1 -- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624127fc0 msgr2=0x7fc62413a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6227fc700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624127fc0 0x7fc62413a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6227fc700 1 -- 172.21.15.89:0/3069904819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6140092a0 con 0x7fc624127590 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.240+0000 7fc6237fe700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624127fc0 0x7fc62413a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.241+0000 7fc6227fc700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127590 0x7fc62413a100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc618000c00 tx=0x7fc618007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.241+0000 7fc60affd700 1 -- 172.21.15.89:0/3069904819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180070d0 con 0x7fc624127590 2023-09-30T22:13:55.246 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.241+0000 7fc60affd700 1 -- 172.21.15.89:0/3069904819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc618007230 con 0x7fc624127590 2023-09-30T22:13:55.247 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.241+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61c009ae0 con 0x7fc624127590 2023-09-30T22:13:55.247 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.241+0000 7fc60affd700 1 -- 172.21.15.89:0/3069904819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6180056e0 con 0x7fc624127590 2023-09-30T22:13:55.247 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.241+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62413aae0 con 0x7fc624127590 2023-09-30T22:13:55.247 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.241+0000 7fc60affd700 1 -- 172.21.15.89:0/3069904819 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc618005a90 con 0x7fc624127590 2023-09-30T22:13:55.247 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.241+0000 7fc60affd700 1 -- 172.21.15.89:0/3069904819 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2069+0+0 (secure 0 0 0) 0x7fc618039ed0 con 0x7fc624127590 2023-09-30T22:13:55.247 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.242+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62405a260 con 0x7fc624127590 2023-09-30T22:13:55.247 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.246+0000 7fc60affd700 1 -- 172.21.15.89:0/3069904819 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc61801c350 con 0x7fc624127590 2023-09-30T22:13:55.406 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.406+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc624046c20 con 0x7fc624127590 2023-09-30T22:13:55.407 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.406+0000 7fc60affd700 1 -- 172.21.15.89:0/3069904819 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) v1 ==== 74+0+4936 (secure 0 0 0) 0x7fc618010af0 con 0x7fc624127590 2023-09-30T22:13:55.407 INFO:teuthology.misc.health.smithi089.stdout: 2023-09-30T22:13:55.408 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":9,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:13:54.525618+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 >> v1:172.21.15.89:6816/118161 conn(0x7fc61002bb90 legacy=0x7fc61002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127590 msgr2=0x7fc62413a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127590 0x7fc62413a100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc618000c00 tx=0x7fc618007bb0).stop 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 shutdown_connections 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc624127fc0 0x7fc62413a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc624097cc0 0x7fc624097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 --2- 172.21.15.89:0/3069904819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc624127590 0x7fc62413a100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 >> 172.21.15.89:0/3069904819 conn(0x7fc62409dbf0 msgr2=0x7fc62412a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:55.409 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 shutdown_connections 2023-09-30T22:13:55.410 INFO:teuthology.misc.health.smithi089.stderr:2023-09-30T22:13:55.408+0000 7fc6292fb700 1 -- 172.21.15.89:0/3069904819 wait complete. 2023-09-30T22:13:55.419 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-09-30T22:13:55.419 INFO:tasks.ceph:Creating RBD pool 2023-09-30T22:13:55.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-30T22:13:55.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.537+0000 7f5a8cbd4700 1 Processor -- start 2023-09-30T22:13:55.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.537+0000 7f5a8cbd4700 1 -- start start 2023-09-30T22:13:55.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8cbd4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8cbd4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a88098c80 0x7f5a88099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8cbd4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a8812c150 0x7f5a8812e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8cbd4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a88093c20 con 0x7f5a8812c150 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8cbd4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88092b90 con 0x7f5a88093840 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8cbd4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88092d80 con 0x7f5a88098c80 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a86d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a8812c150 0x7f5a8812e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a85d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a88098c80 0x7f5a88099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55896/0 (socket says 172.21.15.89:55896) 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8659c700 1 -- 172.21.15.89:0/1232845679 learned_addr learned my addr 172.21.15.89:0/1232845679 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:55.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8659c700 1 -- 172.21.15.89:0/1232845679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a88098c80 msgr2=0x7f5a88099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8659c700 1 --2- 172.21.15.89:0/1232845679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a88098c80 0x7f5a88099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8659c700 1 -- 172.21.15.89:0/1232845679 >> v2:172.21.15.89:3300/0 conn(0x7f5a8812c150 msgr2=0x7f5a8812e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8659c700 1 --2- 172.21.15.89:0/1232845679 >> v2:172.21.15.89:3300/0 conn(0x7f5a8812c150 0x7f5a8812e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a8659c700 1 -- 172.21.15.89:0/1232845679 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88099b70 con 0x7f5a88093840 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.538+0000 7f5a86d9d700 1 --2- 172.21.15.89:0/1232845679 >> v2:172.21.15.89:3300/0 conn(0x7f5a8812c150 0x7f5a8812e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8659c700 1 --2- 172.21.15.89:0/1232845679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88098790 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5a78006fd0 tx=0x7f5a780090f0).ready entity=mon.1 client_cookie=c1f2baf3c5d7fc51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8559a700 1 -- 172.21.15.89:0/1232845679 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800f510 con 0x7f5a88093840 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8559a700 1 -- 172.21.15.89:0/1232845679 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7800f670 con 0x7f5a88093840 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8559a700 1 -- 172.21.15.89:0/1232845679 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800f9f0 con 0x7f5a88093840 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/1232845679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 msgr2=0x7f5a88098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 --2- 172.21.15.89:0/1232845679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88098790 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5a78006fd0 tx=0x7f5a780090f0).stop 2023-09-30T22:13:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/1232845679 shutdown_connections 2023-09-30T22:13:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 --2- 172.21.15.89:0/1232845679 >> v2:172.21.15.89:3300/0 conn(0x7f5a8812c150 0x7f5a8812e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 --2- 172.21.15.89:0/1232845679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88098790 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 --2- 172.21.15.89:0/1232845679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a88098c80 0x7f5a88099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/1232845679 >> 172.21.15.89:0/1232845679 conn(0x7f5a88097480 msgr2=0x7f5a88097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/1232845679 shutdown_connections 2023-09-30T22:13:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.539+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/1232845679 wait complete. 2023-09-30T22:13:55.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.540+0000 7f5a8cbd4700 1 Processor -- start 2023-09-30T22:13:55.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8cbd4700 1 -- start start 2023-09-30T22:13:55.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8cbd4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88115e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8cbd4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a88098c80 0x7f5a88116330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8cbd4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a8812c150 0x7f5a88116820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8cbd4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88116d10 con 0x7f5a88098c80 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8cbd4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88117be0 con 0x7f5a88093840 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8cbd4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a881193d0 con 0x7f5a8812c150 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88115e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88115e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55906/0 (socket says 172.21.15.89:55906) 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8659c700 1 -- 172.21.15.89:0/399197814 learned_addr learned my addr 172.21.15.89:0/399197814 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a85d9b700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a88098c80 0x7f5a88116330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a86d9d700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a8812c150 0x7f5a88116820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8659c700 1 -- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a8812c150 msgr2=0x7f5a88116820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8659c700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a8812c150 0x7f5a88116820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.541+0000 7f5a8659c700 1 -- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a88098c80 msgr2=0x7f5a88116330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a8659c700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a88098c80 0x7f5a88116330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a8659c700 1 -- 172.21.15.89:0/399197814 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8000bc30 con 0x7f5a88093840 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a8659c700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88115e40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f5a8809a710 tx=0x7f5a78017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a767fc700 1 -- 172.21.15.89:0/399197814 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800fe30 con 0x7f5a88093840 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a767fc700 1 -- 172.21.15.89:0/399197814 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7800f7e0 con 0x7f5a88093840 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a78009370 con 0x7f5a88093840 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a767fc700 1 -- 172.21.15.89:0/399197814 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7801c670 con 0x7f5a88093840 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a88117240 con 0x7f5a88093840 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.542+0000 7f5a767fc700 1 -- 172.21.15.89:0/399197814 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5a7801ca60 con 0x7f5a88093840 2023-09-30T22:13:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.543+0000 7f5a767fc700 1 -- 172.21.15.89:0/399197814 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2461+0+0 (secure 0 0 0) 0x7f5a7803c6f0 con 0x7f5a88093840 2023-09-30T22:13:55.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.543+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8805a260 con 0x7f5a88093840 2023-09-30T22:13:55.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.547+0000 7f5a767fc700 1 -- 172.21.15.89:0/399197814 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5a78020030 con 0x7f5a88093840 2023-09-30T22:13:55.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:55.707+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f5a88064f20 con 0x7f5a88093840 2023-09-30T22:13:56.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.538+0000 7f5a767fc700 1 -- 172.21.15.89:0/399197814 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v11) v1 ==== 109+0+0 (secure 0 0 0) 0x7f5a780340a0 con 0x7f5a88093840 2023-09-30T22:13:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 >> v1:172.21.15.89:6816/118161 conn(0x7f5a6802bb60 legacy=0x7f5a6802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 msgr2=0x7f5a88115e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88115e40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f5a8809a710 tx=0x7f5a78017000).stop 2023-09-30T22:13:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 shutdown_connections 2023-09-30T22:13:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a8812c150 0x7f5a88116820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a88093840 0x7f5a88115e40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 --2- 172.21.15.89:0/399197814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a88098c80 0x7f5a88116330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 >> 172.21.15.89:0/399197814 conn(0x7f5a88097480 msgr2=0x7f5a88068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 shutdown_connections 2023-09-30T22:13:56.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.541+0000 7f5a8cbd4700 1 -- 172.21.15.89:0/399197814 wait complete. 2023-09-30T22:13:56.543 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'rbd' created 2023-09-30T22:13:56.554 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-30T22:13:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.673+0000 7f9cbea5e700 1 Processor -- start 2023-09-30T22:13:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.673+0000 7f9cbea5e700 1 -- start start 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cbea5e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb8097ea0 0x7f9cb8098780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cbea5e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9cb80990b0 0x7f9cb8132c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cbea5e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb80962c0 con 0x7f9cb8098c70 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cbea5e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb80959c0 con 0x7f9cb80990b0 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cbea5e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb8065890 con 0x7f9cb8097ea0 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cbcffb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9cb80990b0 0x7f9cb8132c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cb7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb8097ea0 0x7f9cb8098780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cb7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb8097ea0 0x7f9cb8098780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49150/0 (socket says 172.21.15.89:49150) 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cbcffb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9cb80990b0 0x7f9cb8132c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42668/0 (socket says 172.21.15.89:42668) 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cb7fff700 1 -- 172.21.15.89:0/2144464694 learned_addr learned my addr 172.21.15.89:0/2144464694 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cb77fe700 1 --1- 172.21.15.89:0/2144464694 >> v1:172.21.15.89:6789/0 conn(0x7f9cb8098c70 0x7f9cb812f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40230/0 (socket says 172.21.15.89:40230) 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cb7fff700 1 -- 172.21.15.89:0/2144464694 >> v2:172.21.15.89:3300/0 conn(0x7f9cb80990b0 msgr2=0x7f9cb8132c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cb7fff700 1 --2- 172.21.15.89:0/2144464694 >> v2:172.21.15.89:3300/0 conn(0x7f9cb80990b0 0x7f9cb8132c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cb7fff700 1 -- 172.21.15.89:0/2144464694 >> v1:172.21.15.89:6789/0 conn(0x7f9cb8098c70 legacy=0x7f9cb812f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cb7fff700 1 -- 172.21.15.89:0/2144464694 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb8065f10 con 0x7f9cb8097ea0 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.674+0000 7f9cbcffb700 1 --2- 172.21.15.89:0/2144464694 >> v2:172.21.15.89:3300/0 conn(0x7f9cb80990b0 0x7f9cb8132c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cb7fff700 1 --2- 172.21.15.89:0/2144464694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb8097ea0 0x7f9cb8098780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f9cac006fd0 tx=0x7f9cac0090f0).ready entity=mon.2 client_cookie=908395931b95d545 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2144464694 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00b580 con 0x7f9cb8097ea0 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2144464694 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac00b6e0 con 0x7f9cb8097ea0 2023-09-30T22:13:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cb6ffd700 1 -- 172.21.15.89:0/2144464694 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00ba60 con 0x7f9cb8097ea0 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cbea5e700 1 -- 172.21.15.89:0/2144464694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb8097ea0 msgr2=0x7f9cb8098780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cbea5e700 1 --2- 172.21.15.89:0/2144464694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb8097ea0 0x7f9cb8098780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f9cac006fd0 tx=0x7f9cac0090f0).stop 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cbea5e700 1 -- 172.21.15.89:0/2144464694 shutdown_connections 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cbea5e700 1 --2- 172.21.15.89:0/2144464694 >> v2:172.21.15.89:3300/0 conn(0x7f9cb80990b0 0x7f9cb8132c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cbea5e700 1 --2- 172.21.15.89:0/2144464694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb8097ea0 0x7f9cb8098780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cbea5e700 1 -- 172.21.15.89:0/2144464694 >> 172.21.15.89:0/2144464694 conn(0x7f9cb809dd60 msgr2=0x7f9cb809e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cbea5e700 1 -- 172.21.15.89:0/2144464694 shutdown_connections 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.675+0000 7f9cbea5e700 1 -- 172.21.15.89:0/2144464694 wait complete. 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.676+0000 7f9cbea5e700 1 Processor -- start 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.676+0000 7f9cbea5e700 1 -- start start 2023-09-30T22:13:56.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cbea5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cb8097ea0 0x7f9cb813a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cbea5e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cb8098c70 0x7f9cb813a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cbea5e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb80990b0 0x7f9cb813ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cbea5e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb811e1d0 con 0x7f9cb8098c70 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cbea5e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb8099cd0 con 0x7f9cb8097ea0 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cbea5e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb809d210 con 0x7f9cb80990b0 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cb8097ea0 0x7f9cb813a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cb8097ea0 0x7f9cb813a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55914/0 (socket says 172.21.15.89:55914) 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 -- 172.21.15.89:0/3777003398 learned_addr learned my addr 172.21.15.89:0/3777003398 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb77fe700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cb8098c70 0x7f9cb813a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cbcffb700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb80990b0 0x7f9cb813ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 -- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb80990b0 msgr2=0x7f9cb813ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:56.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb80990b0 0x7f9cb813ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 -- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cb8098c70 msgr2=0x7f9cb813a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cb8098c70 0x7f9cb813a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 -- 172.21.15.89:0/3777003398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb0009a20 con 0x7f9cb8097ea0 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb77fe700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cb8098c70 0x7f9cb813a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cbcffb700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb80990b0 0x7f9cb813ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.677+0000 7f9cb7fff700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cb8097ea0 0x7f9cb813a460 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f9cb8095b00 tx=0x7f9cac017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.678+0000 7f9c9ffff700 1 -- 172.21.15.89:0/3777003398 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac000bc0 con 0x7f9cb8097ea0 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.678+0000 7f9c9ffff700 1 -- 172.21.15.89:0/3777003398 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac00b850 con 0x7f9cb8097ea0 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.678+0000 7f9c9ffff700 1 -- 172.21.15.89:0/3777003398 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac000d20 con 0x7f9cb8097ea0 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.678+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac009370 con 0x7f9cb8097ea0 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.678+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb811e9e0 con 0x7f9cb8097ea0 2023-09-30T22:13:56.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.679+0000 7f9c9ffff700 1 -- 172.21.15.89:0/3777003398 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9cac01c920 con 0x7f9cb8097ea0 2023-09-30T22:13:56.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.679+0000 7f9c9ffff700 1 -- 172.21.15.89:0/3777003398 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2827+0+0 (secure 0 0 0) 0x7f9cac03c7c0 con 0x7f9cb8097ea0 2023-09-30T22:13:56.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.679+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb811e9e0 con 0x7f9cb8097ea0 2023-09-30T22:13:56.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.683+0000 7f9c9ffff700 1 -- 172.21.15.89:0/3777003398 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9cb811e9e0 con 0x7f9cb8097ea0 2023-09-30T22:13:56.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:56.847+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9cb811e9e0 con 0x7f9cb8097ea0 2023-09-30T22:13:57.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.538+0000 7f9c9ffff700 1 -- 172.21.15.89:0/3777003398 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v12) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9cb811e9e0 con 0x7f9cb8097ea0 2023-09-30T22:13:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.541+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 >> v1:172.21.15.89:6816/118161 conn(0x7f9ca402bbb0 legacy=0x7f9ca402e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.541+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cb8097ea0 msgr2=0x7f9cb813a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.541+0000 7f9cbea5e700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cb8097ea0 0x7f9cb813a460 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f9cb8095b00 tx=0x7f9cac017000).stop 2023-09-30T22:13:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.541+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 shutdown_connections 2023-09-30T22:13:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.541+0000 7f9cbea5e700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cb80990b0 0x7f9cb813ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.541+0000 7f9cbea5e700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cb8097ea0 0x7f9cb813a460 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.541+0000 7f9cbea5e700 1 --2- 172.21.15.89:0/3777003398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cb8098c70 0x7f9cb813a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.541+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 >> 172.21.15.89:0/3777003398 conn(0x7f9cb809dd60 msgr2=0x7f9cb81143c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.542+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 shutdown_connections 2023-09-30T22:13:57.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.542+0000 7f9cbea5e700 1 -- 172.21.15.89:0/3777003398 wait complete. 2023-09-30T22:13:57.544 INFO:teuthology.orchestra.run.smithi089.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-30T22:13:57.559 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-30T22:13:57.560 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-30T22:13:57.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-30T22:13:57.562 INFO:tasks.ceph.mds.a:Started 2023-09-30T22:13:57.562 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-09-30T22:13:57.563 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:13:57.642 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.687+0000 7f3ef11e0700 1 Processor -- start 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.687+0000 7f3ef11e0700 1 -- start start 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ef11e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec096d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ef11e0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eec097280 0x7f3eec066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ef11e0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec067640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ef11e0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec067b30 con 0x7f3eec097280 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ef11e0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec092fe0 con 0x7f3eec0671c0 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ef11e0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec094560 con 0x7f3eec0969b0 2023-09-30T22:13:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec096d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eeb59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec067640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eea59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eec097280 0x7f3eec066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec096d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49186/0 (socket says 172.21.15.89:49186) 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eea59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eec097280 0x7f3eec066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42706/0 (socket says 172.21.15.89:42706) 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eeb59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec067640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55926/0 (socket says 172.21.15.89:55926) 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 -- 172.21.15.89:0/2990560679 learned_addr learned my addr 172.21.15.89:0/2990560679 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 -- 172.21.15.89:0/2990560679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 msgr2=0x7f3eec067640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 --2- 172.21.15.89:0/2990560679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec067640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 -- 172.21.15.89:0/2990560679 >> v2:172.21.15.89:3300/0 conn(0x7f3eec097280 msgr2=0x7f3eec066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 --2- 172.21.15.89:0/2990560679 >> v2:172.21.15.89:3300/0 conn(0x7f3eec097280 0x7f3eec066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 -- 172.21.15.89:0/2990560679 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec093260 con 0x7f3eec0969b0 2023-09-30T22:13:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eea59c700 1 --2- 172.21.15.89:0/2990560679 >> v2:172.21.15.89:3300/0 conn(0x7f3eec097280 0x7f3eec066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3eead9d700 1 --2- 172.21.15.89:0/2990560679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec096d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3ee4009f20 tx=0x7f3ee4009840).ready entity=mon.2 client_cookie=360cfa4cc3d67c00 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ee9d9b700 1 -- 172.21.15.89:0/2990560679 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400e070 con 0x7f3eec0969b0 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ee9d9b700 1 -- 172.21.15.89:0/2990560679 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee400b970 con 0x7f3eec0969b0 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.688+0000 7f3ee9d9b700 1 -- 172.21.15.89:0/2990560679 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400bcf0 con 0x7f3eec0969b0 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 -- 172.21.15.89:0/2990560679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 msgr2=0x7f3eec096d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 --2- 172.21.15.89:0/2990560679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec096d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3ee4009f20 tx=0x7f3ee4009840).stop 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 -- 172.21.15.89:0/2990560679 shutdown_connections 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 --2- 172.21.15.89:0/2990560679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec067640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 --2- 172.21.15.89:0/2990560679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec096d90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 --2- 172.21.15.89:0/2990560679 >> v2:172.21.15.89:3300/0 conn(0x7f3eec097280 0x7f3eec066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 -- 172.21.15.89:0/2990560679 >> 172.21.15.89:0/2990560679 conn(0x7f3eec09a220 msgr2=0x7f3eec09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 -- 172.21.15.89:0/2990560679 shutdown_connections 2023-09-30T22:13:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.689+0000 7f3ef11e0700 1 -- 172.21.15.89:0/2990560679 wait complete. 2023-09-30T22:13:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.690+0000 7f3ef11e0700 1 Processor -- start 2023-09-30T22:13:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.690+0000 7f3ef11e0700 1 -- start start 2023-09-30T22:13:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ef11e0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec11ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ef11e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec0685c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ef11e0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec097280 0x7f3eec068ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ef11e0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec092d60 con 0x7f3eec097280 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ef11e0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec11a380 con 0x7f3eec0671c0 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ef11e0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0682d0 con 0x7f3eec0969b0 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eea59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec0685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eeb59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec097280 0x7f3eec068ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eea59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec0685c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49196/0 (socket says 172.21.15.89:49196) 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eea59c700 1 -- 172.21.15.89:0/3577684731 learned_addr learned my addr 172.21.15.89:0/3577684731 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eead9d700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec11ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eeb59e700 1 -- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 msgr2=0x7f3eec0685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eeb59e700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec0685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eeb59e700 1 -- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 msgr2=0x7f3eec11ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eeb59e700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec11ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eeb59e700 1 -- 172.21.15.89:0/3577684731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee00092a0 con 0x7f3eec097280 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eead9d700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec11ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eea59c700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec0685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3eeb59e700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec097280 0x7f3eec068ab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3edc00e900 tx=0x7f3edc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ed2ffd700 1 -- 172.21.15.89:0/3577684731 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc012600 con 0x7f3eec097280 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ed2ffd700 1 -- 172.21.15.89:0/3577684731 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc012760 con 0x7f3eec097280 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.691+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee4009ae0 con 0x7f3eec097280 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.692+0000 7f3ed2ffd700 1 -- 172.21.15.89:0/3577684731 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc012ae0 con 0x7f3eec097280 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.692+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec068fa0 con 0x7f3eec097280 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.692+0000 7f3ed2ffd700 1 -- 172.21.15.89:0/3577684731 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f3edc01b400 con 0x7f3eec097280 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.692+0000 7f3ed2ffd700 1 -- 172.21.15.89:0/3577684731 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f3edc03aff0 con 0x7f3eec097280 2023-09-30T22:13:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.692+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec092320 con 0x7f3eec097280 2023-09-30T22:13:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.697+0000 7f3ed2ffd700 1 -- 172.21.15.89:0/3577684731 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3edc019310 con 0x7f3eec097280 2023-09-30T22:13:57.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.859+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3eec05a260 con 0x7f3eec097280 2023-09-30T22:13:57.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.859+0000 7f3ed2ffd700 1 -- 172.21.15.89:0/3577684731 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+8137 (secure 0 0 0) 0x7f3edc011b40 con 0x7f3eec097280 2023-09-30T22:13:57.860 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:57.860 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":12,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:13:57.530108+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 >> v1:172.21.15.89:6816/118161 conn(0x7f3ed802bb60 legacy=0x7f3ed802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec097280 msgr2=0x7f3eec068ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec097280 0x7f3eec068ab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3edc00e900 tx=0x7f3edc009060).stop 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 shutdown_connections 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec097280 0x7f3eec068ab0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec0671c0 0x7f3eec11ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 --2- 172.21.15.89:0/3577684731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec0969b0 0x7f3eec0685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 >> 172.21.15.89:0/3577684731 conn(0x7f3eec09a220 msgr2=0x7f3eec12bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 shutdown_connections 2023-09-30T22:13:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.861+0000 7f3ef11e0700 1 -- 172.21.15.89:0/3577684731 wait complete. 2023-09-30T22:13:57.874 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-30T22:13:57.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:13:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.991+0000 7f0ea3e51700 1 Processor -- start 2023-09-30T22:13:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.991+0000 7f0ea3e51700 1 -- start start 2023-09-30T22:13:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea3e51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c06a0b0 0x7f0e9c06a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea3e51700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e9c091830 0x7f0e9c131060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea3e51700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9c0927e0 con 0x7f0e9c06a980 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea3e51700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9c0962a0 con 0x7f0e9c091830 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea3e51700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c092b30 con 0x7f0e9c06a0b0 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea1bed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c06a0b0 0x7f0e9c06a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea23ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e9c091830 0x7f0e9c131060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea1bed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c06a0b0 0x7f0e9c06a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55956/0 (socket says 172.21.15.89:55956) 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea1bed700 1 -- 172.21.15.89:0/3321295309 learned_addr learned my addr 172.21.15.89:0/3321295309 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea13ec700 1 --1- 172.21.15.89:0/3321295309 >> v1:172.21.15.89:6789/0 conn(0x7f0e9c06a980 0x7f0e9c091120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40244/0 (socket says 172.21.15.89:40244) 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea1bed700 1 -- 172.21.15.89:0/3321295309 >> v2:172.21.15.89:3300/0 conn(0x7f0e9c091830 msgr2=0x7f0e9c131060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea1bed700 1 --2- 172.21.15.89:0/3321295309 >> v2:172.21.15.89:3300/0 conn(0x7f0e9c091830 0x7f0e9c131060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea1bed700 1 -- 172.21.15.89:0/3321295309 >> v1:172.21.15.89:6789/0 conn(0x7f0e9c06a980 legacy=0x7f0e9c091120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea0beb700 1 -- 172.21.15.89:0/3321295309 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762394915 0 0) 0x7f0e9c0927e0 con 0x7f0e9c06a980 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea1bed700 1 -- 172.21.15.89:0/3321295309 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c092570 con 0x7f0e9c06a0b0 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.992+0000 7f0ea23ee700 1 --2- 172.21.15.89:0/3321295309 >> v2:172.21.15.89:3300/0 conn(0x7f0e9c091830 0x7f0e9c131060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea1bed700 1 --2- 172.21.15.89:0/3321295309 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c06a0b0 0x7f0e9c06a490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f0e8c00dc30 tx=0x7f0e8c00d500).ready entity=mon.1 client_cookie=40a511e8a7e26f9d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea0beb700 1 -- 172.21.15.89:0/3321295309 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c012070 con 0x7f0e9c06a0b0 2023-09-30T22:13:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea0beb700 1 -- 172.21.15.89:0/3321295309 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c00d2e0 con 0x7f0e9c06a0b0 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea0beb700 1 -- 172.21.15.89:0/3321295309 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c005a90 con 0x7f0e9c06a0b0 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea3e51700 1 -- 172.21.15.89:0/3321295309 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c06a0b0 msgr2=0x7f0e9c06a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea3e51700 1 --2- 172.21.15.89:0/3321295309 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c06a0b0 0x7f0e9c06a490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f0e8c00dc30 tx=0x7f0e8c00d500).stop 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea3e51700 1 -- 172.21.15.89:0/3321295309 shutdown_connections 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea3e51700 1 --2- 172.21.15.89:0/3321295309 >> v2:172.21.15.89:3300/0 conn(0x7f0e9c091830 0x7f0e9c131060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea3e51700 1 --2- 172.21.15.89:0/3321295309 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c06a0b0 0x7f0e9c06a490 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea3e51700 1 -- 172.21.15.89:0/3321295309 >> 172.21.15.89:0/3321295309 conn(0x7f0e9c09a890 msgr2=0x7f0e9c0966d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea3e51700 1 -- 172.21.15.89:0/3321295309 shutdown_connections 2023-09-30T22:13:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.993+0000 7f0ea3e51700 1 -- 172.21.15.89:0/3321295309 wait complete. 2023-09-30T22:13:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.994+0000 7f0ea3e51700 1 Processor -- start 2023-09-30T22:13:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.994+0000 7f0ea3e51700 1 -- start start 2023-09-30T22:13:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.994+0000 7f0ea3e51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e9c06a0b0 0x7f0e9c129fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea3e51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c06a980 0x7f0e9c12a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea3e51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c091830 0x7f0e9c1245d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea3e51700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c128bd0 con 0x7f0e9c06a980 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea3e51700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c126b80 con 0x7f0e9c091830 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea3e51700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c0950d0 con 0x7f0e9c06a0b0 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c06a980 0x7f0e9c12a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea1bed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e9c06a0b0 0x7f0e9c129fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c06a980 0x7f0e9c12a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42732/0 (socket says 172.21.15.89:42732) 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 -- 172.21.15.89:0/1086077444 learned_addr learned my addr 172.21.15.89:0/1086077444 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea1bed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e9c06a0b0 0x7f0e9c129fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49204/0 (socket says 172.21.15.89:49204) 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea23ee700 1 --2- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c091830 0x7f0e9c1245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 -- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e9c06a0b0 msgr2=0x7f0e9c129fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 --2- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e9c06a0b0 0x7f0e9c129fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 -- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c091830 msgr2=0x7f0e9c1245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 --2- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c091830 0x7f0e9c1245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 -- 172.21.15.89:0/1086077444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e94009a20 con 0x7f0e9c06a980 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea13ec700 1 --2- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c06a980 0x7f0e9c12a4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f0e9800c420 tx=0x7f0e98007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0e91ffb700 1 -- 172.21.15.89:0/1086077444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800ce80 con 0x7f0e9c06a980 2023-09-30T22:13:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0e91ffb700 1 -- 172.21.15.89:0/1086077444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e980072e0 con 0x7f0e9c06a980 2023-09-30T22:13:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c00d7a0 con 0x7f0e9c06a980 2023-09-30T22:13:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0e91ffb700 1 -- 172.21.15.89:0/1086077444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e980056e0 con 0x7f0e9c06a980 2023-09-30T22:13:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.995+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9c1286d0 con 0x7f0e9c06a980 2023-09-30T22:13:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.996+0000 7f0e91ffb700 1 -- 172.21.15.89:0/1086077444 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0e98005a90 con 0x7f0e9c06a980 2023-09-30T22:13:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.996+0000 7f0e91ffb700 1 -- 172.21.15.89:0/1086077444 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f0e9803a520 con 0x7f0e9c06a980 2023-09-30T22:13:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:57.996+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9c061e60 con 0x7f0e9c06a980 2023-09-30T22:13:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.001+0000 7f0e91ffb700 1 -- 172.21.15.89:0/1086077444 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0e98018030 con 0x7f0e9c06a980 2023-09-30T22:13:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.158+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0e9c05fe30 con 0x7f0e9c06a980 2023-09-30T22:13:58.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.158+0000 7f0e91ffb700 1 -- 172.21.15.89:0/1086077444 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v12) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0e98013970 con 0x7f0e9c06a980 2023-09-30T22:13:58.385 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 >> v1:172.21.15.89:6816/118161 conn(0x7f0e8002bb60 legacy=0x7f0e8002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c06a980 msgr2=0x7f0e9c12a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 --2- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c06a980 0x7f0e9c12a4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f0e9800c420 tx=0x7f0e98007bb0).stop 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 shutdown_connections 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 --2- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e9c091830 0x7f0e9c1245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 --2- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e9c06a0b0 0x7f0e9c129fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 --2- 172.21.15.89:0/1086077444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c06a980 0x7f0e9c12a4c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 >> 172.21.15.89:0/1086077444 conn(0x7f0e9c09a890 msgr2=0x7f0e9c068cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 shutdown_connections 2023-09-30T22:13:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.160+0000 7f0ea3e51700 1 -- 172.21.15.89:0/1086077444 wait complete. 2023-09-30T22:13:58.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:13:58.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.505+0000 7f1e068b2700 1 Processor -- start 2023-09-30T22:13:58.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.505+0000 7f1e068b2700 1 -- start start 2023-09-30T22:13:58.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.505+0000 7f1e068b2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e000613a0 0x7f1e0012ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1e068b2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e00069430 0x7f1e00132d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1e068b2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e00098210 con 0x7f1e0012b340 2023-09-30T22:13:58.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1e068b2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e00093380 con 0x7f1e00069430 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1e068b2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00099670 con 0x7f1e000613a0 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dff7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1e0012b340 0x7f1e00068d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40254/0 (socket says 172.21.15.89:40254) 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1e04e4f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e00069430 0x7f1e00132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dff7fe700 1 -- 172.21.15.89:0/3790332977 learned_addr learned my addr 172.21.15.89:0/3790332977 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e000613a0 0x7f1e0012ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfeffd700 1 -- 172.21.15.89:0/3790332977 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687651272 0 0) 0x7f1e00098210 con 0x7f1e0012b340 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfeffd700 1 -- 172.21.15.89:0/3790332977 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de80033f0 con 0x7f1e0012b340 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfffff700 1 -- 172.21.15.89:0/3790332977 >> v2:172.21.15.89:3300/0 conn(0x7f1e00069430 msgr2=0x7f1e00132d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfffff700 1 --2- 172.21.15.89:0/3790332977 >> v2:172.21.15.89:3300/0 conn(0x7f1e00069430 0x7f1e00132d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfffff700 1 -- 172.21.15.89:0/3790332977 >> v1:172.21.15.89:6789/0 conn(0x7f1e0012b340 legacy=0x7f1e00068d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfffff700 1 -- 172.21.15.89:0/3790332977 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e00093710 con 0x7f1e000613a0 2023-09-30T22:13:58.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1e04e4f700 1 --2- 172.21.15.89:0/3790332977 >> v2:172.21.15.89:3300/0 conn(0x7f1e00069430 0x7f1e00132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfffff700 1 --2- 172.21.15.89:0/3790332977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e000613a0 0x7f1e0012ae50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f1df4009790 tx=0x7f1df4009060).ready entity=mon.1 client_cookie=4533515cc37927d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfeffd700 1 -- 172.21.15.89:0/3790332977 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4009120 con 0x7f1e000613a0 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfeffd700 1 -- 172.21.15.89:0/3790332977 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df400d4a0 con 0x7f1e000613a0 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.506+0000 7f1dfeffd700 1 -- 172.21.15.89:0/3790332977 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400d820 con 0x7f1e000613a0 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.507+0000 7f1e068b2700 1 -- 172.21.15.89:0/3790332977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e000613a0 msgr2=0x7f1e0012ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.507+0000 7f1e068b2700 1 --2- 172.21.15.89:0/3790332977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e000613a0 0x7f1e0012ae50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f1df4009790 tx=0x7f1df4009060).stop 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.507+0000 7f1e068b2700 1 -- 172.21.15.89:0/3790332977 shutdown_connections 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.507+0000 7f1e068b2700 1 --2- 172.21.15.89:0/3790332977 >> v2:172.21.15.89:3300/0 conn(0x7f1e00069430 0x7f1e00132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.507+0000 7f1e068b2700 1 --2- 172.21.15.89:0/3790332977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e000613a0 0x7f1e0012ae50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.507+0000 7f1e068b2700 1 -- 172.21.15.89:0/3790332977 >> 172.21.15.89:0/3790332977 conn(0x7f1e0009a1c0 msgr2=0x7f1e0009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.507+0000 7f1e068b2700 1 -- 172.21.15.89:0/3790332977 shutdown_connections 2023-09-30T22:13:58.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.507+0000 7f1e068b2700 1 -- 172.21.15.89:0/3790332977 wait complete. 2023-09-30T22:13:58.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e068b2700 1 Processor -- start 2023-09-30T22:13:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e068b2700 1 -- start start 2023-09-30T22:13:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e068b2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e000613a0 0x7f1e0013a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e068b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00069430 0x7f1e0013a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e068b2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e0012b340 0x7f1e00115a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e068b2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00119550 con 0x7f1e00069430 2023-09-30T22:13:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e068b2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0012f190 con 0x7f1e0012b340 2023-09-30T22:13:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e068b2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00092dc0 con 0x7f1e000613a0 2023-09-30T22:13:58.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e000613a0 0x7f1e0013a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1e04e4f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e0012b340 0x7f1e00115a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e000613a0 0x7f1e0013a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49220/0 (socket says 172.21.15.89:49220) 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1dfffff700 1 -- 172.21.15.89:0/526736088 learned_addr learned my addr 172.21.15.89:0/526736088 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.508+0000 7f1dff7fe700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00069430 0x7f1e0013a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1e04e4f700 1 -- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e000613a0 msgr2=0x7f1e0013a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1e04e4f700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e000613a0 0x7f1e0013a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1e04e4f700 1 -- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00069430 msgr2=0x7f1e0013a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1e04e4f700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00069430 0x7f1e0013a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1e04e4f700 1 -- 172.21.15.89:0/526736088 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009a20 con 0x7f1e0012b340 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1dfffff700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e000613a0 0x7f1e0013a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1e04e4f700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e0012b340 0x7f1e00115a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f1df8009f10 tx=0x7f1df800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:58.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1de7fff700 1 -- 172.21.15.89:0/526736088 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df800cb70 con 0x7f1e0012b340 2023-09-30T22:13:58.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1de7fff700 1 -- 172.21.15.89:0/526736088 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df800f040 con 0x7f1e0012b340 2023-09-30T22:13:58.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df40092a0 con 0x7f1e0012b340 2023-09-30T22:13:58.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1de7fff700 1 -- 172.21.15.89:0/526736088 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8013400 con 0x7f1e0012b340 2023-09-30T22:13:58.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.509+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e00117640 con 0x7f1e0012b340 2023-09-30T22:13:58.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.510+0000 7f1de7fff700 1 -- 172.21.15.89:0/526736088 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1df80137f0 con 0x7f1e0012b340 2023-09-30T22:13:58.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.510+0000 7f1de7fff700 1 -- 172.21.15.89:0/526736088 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f1df803b510 con 0x7f1e0012b340 2023-09-30T22:13:58.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.510+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0005a260 con 0x7f1e0012b340 2023-09-30T22:13:58.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.514+0000 7f1de7fff700 1 -- 172.21.15.89:0/526736088 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1df8013aa0 con 0x7f1e0012b340 2023-09-30T22:13:58.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.675+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e00093940 con 0x7f1e0012b340 2023-09-30T22:13:58.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.675+0000 7f1de7fff700 1 -- 172.21.15.89:0/526736088 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v13) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1df801b7e0 con 0x7f1e0012b340 2023-09-30T22:13:58.676 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:13:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 >> v1:172.21.15.89:6816/118161 conn(0x7f1dec02bb60 legacy=0x7f1dec02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e0012b340 msgr2=0x7f1e00115a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e0012b340 0x7f1e00115a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f1df8009f10 tx=0x7f1df800b040).stop 2023-09-30T22:13:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 shutdown_connections 2023-09-30T22:13:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e0012b340 0x7f1e00115a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e000613a0 0x7f1e0013a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 --2- 172.21.15.89:0/526736088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00069430 0x7f1e0013a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 >> 172.21.15.89:0/526736088 conn(0x7f1e0009a1c0 msgr2=0x7f1e0012df70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 shutdown_connections 2023-09-30T22:13:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.677+0000 7f1e068b2700 1 -- 172.21.15.89:0/526736088 wait complete. 2023-09-30T22:13:58.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:13:58.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.804+0000 7f7150ee7700 1 Processor -- start 2023-09-30T22:13:58.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.804+0000 7f7150ee7700 1 -- start start 2023-09-30T22:13:58.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f7150ee7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f7150ee7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f7150ee7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f714c094c20 0x7f714c127af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f7150ee7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714c096760 con 0x7f714c094c20 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f7150ee7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c091ff0 con 0x7f714c096380 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f7150ee7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f714c092c80 con 0x7f714c091b00 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f714c094c20 0x7f714c127af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f7149d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55982/0 (socket says 172.21.15.89:55982) 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f714c094c20 0x7f714c127af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42756/0 (socket says 172.21.15.89:42756) 2023-09-30T22:13:58.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 -- 172.21.15.89:0/3066070498 learned_addr learned my addr 172.21.15.89:0/3066070498 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 -- 172.21.15.89:0/3066070498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 msgr2=0x7f714c127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 --2- 172.21.15.89:0/3066070498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c127600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 -- 172.21.15.89:0/3066070498 >> v2:172.21.15.89:3300/0 conn(0x7f714c094c20 msgr2=0x7f714c127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 --2- 172.21.15.89:0/3066070498 >> v2:172.21.15.89:3300/0 conn(0x7f714c094c20 0x7f714c127af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 -- 172.21.15.89:0/3066070498 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c093900 con 0x7f714c096380 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f7149d9b700 1 --2- 172.21.15.89:0/3066070498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714ad9d700 1 --2- 172.21.15.89:0/3066070498 >> v2:172.21.15.89:3300/0 conn(0x7f714c094c20 0x7f714c127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.805+0000 7f714a59c700 1 --2- 172.21.15.89:0/3066070498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c094730 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f713c009830 tx=0x7f713c0090f0).ready entity=mon.1 client_cookie=df77b8c25c2a656f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f714959a700 1 -- 172.21.15.89:0/3066070498 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00f510 con 0x7f714c096380 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f714959a700 1 -- 172.21.15.89:0/3066070498 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c00f670 con 0x7f714c096380 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f714959a700 1 -- 172.21.15.89:0/3066070498 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00f9f0 con 0x7f714c096380 2023-09-30T22:13:58.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 -- 172.21.15.89:0/3066070498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 msgr2=0x7f714c094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 --2- 172.21.15.89:0/3066070498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c094730 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f713c009830 tx=0x7f713c0090f0).stop 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 -- 172.21.15.89:0/3066070498 shutdown_connections 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 --2- 172.21.15.89:0/3066070498 >> v2:172.21.15.89:3300/0 conn(0x7f714c094c20 0x7f714c127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 --2- 172.21.15.89:0/3066070498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c094730 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 --2- 172.21.15.89:0/3066070498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 -- 172.21.15.89:0/3066070498 >> 172.21.15.89:0/3066070498 conn(0x7f714c097470 msgr2=0x7f714c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 -- 172.21.15.89:0/3066070498 shutdown_connections 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.806+0000 7f7150ee7700 1 -- 172.21.15.89:0/3066070498 wait complete. 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.807+0000 7f7150ee7700 1 Processor -- start 2023-09-30T22:13:58.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.807+0000 7f7150ee7700 1 -- start start 2023-09-30T22:13:58.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.807+0000 7f7150ee7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c116c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f7150ee7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f714c094c20 0x7f714c112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f7150ee7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f7150ee7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c1152e0 con 0x7f714c094c20 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f7150ee7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c124d80 con 0x7f714c096380 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f7150ee7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f714c116790 con 0x7f714c091b00 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c116c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c116c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49228/0 (socket says 172.21.15.89:49228) 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 -- 172.21.15.89:0/530772089 learned_addr learned my addr 172.21.15.89:0/530772089 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714ad9d700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f7149d9b700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f714c094c20 0x7f714c112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 -- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 msgr2=0x7f714c113120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c113120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 -- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f714c094c20 msgr2=0x7f714c112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f714c094c20 0x7f714c112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 -- 172.21.15.89:0/530772089 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713400b540 con 0x7f714c091b00 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f714a59c700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c116c10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f714c092130 tx=0x7f713c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f71427fc700 1 -- 172.21.15.89:0/530772089 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00f7e0 con 0x7f714c091b00 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f71427fc700 1 -- 172.21.15.89:0/530772089 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c00f940 con 0x7f714c091b00 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7144009a20 con 0x7f714c091b00 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f71427fc700 1 -- 172.21.15.89:0/530772089 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c01c670 con 0x7f714c091b00 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.808+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c009370 con 0x7f714c091b00 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.809+0000 7f71427fc700 1 -- 172.21.15.89:0/530772089 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f713c01ca60 con 0x7f714c091b00 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.809+0000 7f71427fc700 1 -- 172.21.15.89:0/530772089 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f713c03c890 con 0x7f714c091b00 2023-09-30T22:13:58.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.809+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714c05a260 con 0x7f714c091b00 2023-09-30T22:13:58.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:58.814+0000 7f71427fc700 1 -- 172.21.15.89:0/530772089 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f713c02a020 con 0x7f714c091b00 2023-09-30T22:13:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.007+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f714c046c20 con 0x7f714c091b00 2023-09-30T22:13:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.008+0000 7f71427fc700 1 -- 172.21.15.89:0/530772089 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1196 (secure 0 0 0) 0x7f713c034090 con 0x7f714c091b00 2023-09-30T22:13:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:59.009 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-09-30T22:13:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.009+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 >> v1:172.21.15.89:6816/118161 conn(0x7f712c02bb60 legacy=0x7f712c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.009+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 msgr2=0x7f714c116c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.009+0000 7f7150ee7700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c116c10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f714c092130 tx=0x7f713c015000).stop 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.010+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 shutdown_connections 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.010+0000 7f7150ee7700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f714c096380 0x7f714c113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.010+0000 7f7150ee7700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f714c091b00 0x7f714c116c10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.010+0000 7f7150ee7700 1 --2- 172.21.15.89:0/530772089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f714c094c20 0x7f714c112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.010+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 >> 172.21.15.89:0/530772089 conn(0x7f714c097470 msgr2=0x7f714c059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.010+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 shutdown_connections 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.010+0000 7f7150ee7700 1 -- 172.21.15.89:0/530772089 wait complete. 2023-09-30T22:13:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 2 2023-09-30T22:13:59.022 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:13:59.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.136+0000 7fc52abf2700 1 Processor -- start 2023-09-30T22:13:59.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.136+0000 7fc52abf2700 1 -- start start 2023-09-30T22:13:59.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.136+0000 7fc52abf2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524096380 0x7fc524094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52abf2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52abf2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc524096760 con 0x7fc524091b00 2023-09-30T22:13:59.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52abf2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc524091ff0 con 0x7fc524094c20 2023-09-30T22:13:59.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52abf2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc524092c80 con 0x7fc524096380 2023-09-30T22:13:59.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52898e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524096380 0x7fc524094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc523fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc524091b00 0x7fc524127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40268/0 (socket says 172.21.15.89:40268) 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc523fff700 1 -- 172.21.15.89:0/3113741451 learned_addr learned my addr 172.21.15.89:0/3113741451 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52918f700 1 --2- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc5237fe700 1 -- 172.21.15.89:0/3113741451 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623492097 0 0) 0x7fc524096760 con 0x7fc524091b00 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52918f700 1 -- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524096380 msgr2=0x7fc524094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52918f700 1 --2- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524096380 0x7fc524094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52918f700 1 -- 172.21.15.89:0/3113741451 >> v1:172.21.15.89:6789/0 conn(0x7fc524091b00 legacy=0x7fc524127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52918f700 1 -- 172.21.15.89:0/3113741451 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524093900 con 0x7fc524094c20 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52898e700 1 --2- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524096380 0x7fc524094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:59.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc52918f700 1 --2- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524127d10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc51c009e90 tx=0x7fc51c009780).ready entity=mon.1 client_cookie=9592e586e7ecd2fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc5237fe700 1 -- 172.21.15.89:0/3113741451 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00e070 con 0x7fc524094c20 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc5237fe700 1 -- 172.21.15.89:0/3113741451 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00bb60 con 0x7fc524094c20 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.137+0000 7fc5237fe700 1 -- 172.21.15.89:0/3113741451 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c012410 con 0x7fc524094c20 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.138+0000 7fc52abf2700 1 -- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 msgr2=0x7fc524127d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.138+0000 7fc52abf2700 1 --2- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524127d10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc51c009e90 tx=0x7fc51c009780).stop 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.138+0000 7fc52abf2700 1 -- 172.21.15.89:0/3113741451 shutdown_connections 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.138+0000 7fc52abf2700 1 --2- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524127d10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.138+0000 7fc52abf2700 1 --2- 172.21.15.89:0/3113741451 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524096380 0x7fc524094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.138+0000 7fc52abf2700 1 -- 172.21.15.89:0/3113741451 >> 172.21.15.89:0/3113741451 conn(0x7fc524097470 msgr2=0x7fc524097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.138+0000 7fc52abf2700 1 -- 172.21.15.89:0/3113741451 shutdown_connections 2023-09-30T22:13:59.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.138+0000 7fc52abf2700 1 -- 172.21.15.89:0/3113741451 wait complete. 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52abf2700 1 Processor -- start 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52abf2700 1 -- start start 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52abf2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524091b00 0x7fc524124890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52abf2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52abf2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc524096380 0x7fc524113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52abf2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52406a520 con 0x7fc524096380 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52abf2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc524116cc0 con 0x7fc524094c20 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52abf2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc524117260 con 0x7fc524091b00 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52918f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc524096380 0x7fc524113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52898e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524091b00 0x7fc524124890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc523fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52918f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc524096380 0x7fc524113d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42786/0 (socket says 172.21.15.89:42786) 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52898e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524091b00 0x7fc524124890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49252/0 (socket says 172.21.15.89:49252) 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.139+0000 7fc52918f700 1 -- 172.21.15.89:0/1677728759 learned_addr learned my addr 172.21.15.89:0/1677728759 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52918f700 1 -- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524091b00 msgr2=0x7fc524124890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52918f700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524091b00 0x7fc524124890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52918f700 1 -- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 msgr2=0x7fc524113810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52918f700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524113810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52918f700 1 -- 172.21.15.89:0/1677728759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180092a0 con 0x7fc524096380 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52898e700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524091b00 0x7fc524124890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:59.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc523fff700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524113810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52918f700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc524096380 0x7fc524113d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc51c00b6c0 tx=0x7fc51c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc520ff9700 1 -- 172.21.15.89:0/1677728759 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c01d630 con 0x7fc524096380 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc520ff9700 1 -- 172.21.15.89:0/1677728759 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c01d790 con 0x7fc524096380 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc520ff9700 1 -- 172.21.15.89:0/1677728759 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c01db50 con 0x7fc524096380 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c009a20 con 0x7fc524096380 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.140+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc524117940 con 0x7fc524096380 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.141+0000 7fc520ff9700 1 -- 172.21.15.89:0/1677728759 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc51c003040 con 0x7fc524096380 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.141+0000 7fc520ff9700 1 -- 172.21.15.89:0/1677728759 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fc51c03c9e0 con 0x7fc524096380 2023-09-30T22:13:59.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.141+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc524099fb0 con 0x7fc524096380 2023-09-30T22:13:59.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.145+0000 7fc520ff9700 1 -- 172.21.15.89:0/1677728759 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc51c01bb00 con 0x7fc524096380 2023-09-30T22:13:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.304+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc52405a260 con 0x7fc524096380 2023-09-30T22:13:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.304+0000 7fc520ff9700 1 -- 172.21.15.89:0/1677728759 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8124 (secure 0 0 0) 0x7fc51c01bb00 con 0x7fc524096380 2023-09-30T22:13:59.305 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:13:59.305 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":13,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:13:58.531420+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:13:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.306+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 >> v1:172.21.15.89:6816/118161 conn(0x7fc50802bb90 legacy=0x7fc50802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.306+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc524096380 msgr2=0x7fc524113d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.306+0000 7fc52abf2700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc524096380 0x7fc524113d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc51c00b6c0 tx=0x7fc51c009780).stop 2023-09-30T22:13:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.307+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 shutdown_connections 2023-09-30T22:13:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.307+0000 7fc52abf2700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc524096380 0x7fc524113d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.307+0000 7fc52abf2700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc524091b00 0x7fc524124890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.307+0000 7fc52abf2700 1 --2- 172.21.15.89:0/1677728759 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc524094c20 0x7fc524113810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.307+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 >> 172.21.15.89:0/1677728759 conn(0x7fc524097470 msgr2=0x7fc524126c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.307+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 shutdown_connections 2023-09-30T22:13:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.307+0000 7fc52abf2700 1 -- 172.21.15.89:0/1677728759 wait complete. 2023-09-30T22:13:59.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-30T22:13:59.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:13:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.435+0000 7f888139a700 1 Processor -- start 2023-09-30T22:13:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.435+0000 7f888139a700 1 -- start start 2023-09-30T22:13:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f888139a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c066180 0x7f887c097480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f888139a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f887c12b1d0 0x7f887c127480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f888139a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c12b610 0x7f887c127970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f888139a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c066560 con 0x7f887c12b1d0 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f888139a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c065980 con 0x7f887c066180 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f888139a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0656c0 con 0x7f887c12b610 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f887c12b1d0 0x7f887c127480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c12b610 0x7f887c127970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c066180 0x7f887c097480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f887c12b1d0 0x7f887c127480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42794/0 (socket says 172.21.15.89:42794) 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c12b610 0x7f887c127970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49264/0 (socket says 172.21.15.89:49264) 2023-09-30T22:13:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887a7fc700 1 -- 172.21.15.89:0/4286576921 learned_addr learned my addr 172.21.15.89:0/4286576921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887b7fe700 1 -- 172.21.15.89:0/4286576921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c066180 msgr2=0x7f887c097480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887b7fe700 1 --2- 172.21.15.89:0/4286576921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c066180 0x7f887c097480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887b7fe700 1 -- 172.21.15.89:0/4286576921 >> v2:172.21.15.89:3300/0 conn(0x7f887c12b1d0 msgr2=0x7f887c127480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887b7fe700 1 --2- 172.21.15.89:0/4286576921 >> v2:172.21.15.89:3300/0 conn(0x7f887c12b1d0 0x7f887c127480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887b7fe700 1 -- 172.21.15.89:0/4286576921 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c098250 con 0x7f887c12b610 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887a7fc700 1 --2- 172.21.15.89:0/4286576921 >> v2:172.21.15.89:3300/0 conn(0x7f887c12b1d0 0x7f887c127480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.436+0000 7f887b7fe700 1 --2- 172.21.15.89:0/4286576921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c12b610 0x7f887c127970 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8874009e90 tx=0x7f8874009780).ready entity=mon.2 client_cookie=b4ddbee340ca5efa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f8879ffb700 1 -- 172.21.15.89:0/4286576921 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400e070 con 0x7f887c12b610 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f8879ffb700 1 -- 172.21.15.89:0/4286576921 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887400ba70 con 0x7f887c12b610 2023-09-30T22:13:59.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f8879ffb700 1 -- 172.21.15.89:0/4286576921 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400bdf0 con 0x7f887c12b610 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 -- 172.21.15.89:0/4286576921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c12b610 msgr2=0x7f887c127970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 --2- 172.21.15.89:0/4286576921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c12b610 0x7f887c127970 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8874009e90 tx=0x7f8874009780).stop 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 -- 172.21.15.89:0/4286576921 shutdown_connections 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 --2- 172.21.15.89:0/4286576921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c12b610 0x7f887c127970 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 --2- 172.21.15.89:0/4286576921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c066180 0x7f887c097480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 --2- 172.21.15.89:0/4286576921 >> v2:172.21.15.89:3300/0 conn(0x7f887c12b1d0 0x7f887c127480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 -- 172.21.15.89:0/4286576921 >> 172.21.15.89:0/4286576921 conn(0x7f887c09ba50 msgr2=0x7f887c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 -- 172.21.15.89:0/4286576921 shutdown_connections 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.437+0000 7f888139a700 1 -- 172.21.15.89:0/4286576921 wait complete. 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.438+0000 7f888139a700 1 Processor -- start 2023-09-30T22:13:59.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.438+0000 7f888139a700 1 -- start start 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f888139a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c066180 0x7f887c13a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f888139a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c12b1d0 0x7f887c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f888139a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12b610 0x7f887c1132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f888139a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1159a0 con 0x7f887c12b610 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f888139a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c065030 con 0x7f887c12b1d0 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f888139a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f887c116870 con 0x7f887c066180 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c12b1d0 0x7f887c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12b610 0x7f887c1132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c12b1d0 0x7f887c13a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56030/0 (socket says 172.21.15.89:56030) 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 -- 172.21.15.89:0/3344986323 learned_addr learned my addr 172.21.15.89:0/3344986323 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:59.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c066180 0x7f887c13a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 -- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c066180 msgr2=0x7f887c13a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c066180 0x7f887c13a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 -- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12b610 msgr2=0x7f887c1132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12b610 0x7f887c1132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 -- 172.21.15.89:0/3344986323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c009370 con 0x7f887c12b1d0 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887affd700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c066180 0x7f887c13a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887b7fe700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12b610 0x7f887c1132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f887a7fc700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c12b1d0 0x7f887c13a730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f886c00ea50 tx=0x7f886c0090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f8872ffd700 1 -- 172.21.15.89:0/3344986323 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00ee90 con 0x7f887c12b1d0 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f8872ffd700 1 -- 172.21.15.89:0/3344986323 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c00c7f0 con 0x7f887c12b1d0 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8874009a20 con 0x7f887c12b1d0 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f8872ffd700 1 -- 172.21.15.89:0/3344986323 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00cb70 con 0x7f887c12b1d0 2023-09-30T22:13:59.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.439+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c1174f0 con 0x7f887c12b1d0 2023-09-30T22:13:59.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.440+0000 7f8872ffd700 1 -- 172.21.15.89:0/3344986323 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f886c01b400 con 0x7f887c12b1d0 2023-09-30T22:13:59.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.440+0000 7f8872ffd700 1 -- 172.21.15.89:0/3344986323 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f886c03b150 con 0x7f887c12b1d0 2023-09-30T22:13:59.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.441+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c1147b0 con 0x7f887c12b1d0 2023-09-30T22:13:59.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.445+0000 7f8872ffd700 1 -- 172.21.15.89:0/3344986323 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f886c0123f0 con 0x7f887c12b1d0 2023-09-30T22:13:59.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.600+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f887c05a260 con 0x7f887c12b1d0 2023-09-30T22:13:59.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.601+0000 7f8872ffd700 1 -- 172.21.15.89:0/3344986323 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v13) v1 ==== 110+0+10 (secure 0 0 0) 0x7f886c011350 con 0x7f887c12b1d0 2023-09-30T22:13:59.601 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:13:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 >> v1:172.21.15.89:6816/118161 conn(0x7f885c02bb60 legacy=0x7f885c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c12b1d0 msgr2=0x7f887c13a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c12b1d0 0x7f887c13a730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f886c00ea50 tx=0x7f886c0090f0).stop 2023-09-30T22:13:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 shutdown_connections 2023-09-30T22:13:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12b610 0x7f887c1132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c066180 0x7f887c13a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 --2- 172.21.15.89:0/3344986323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c12b1d0 0x7f887c13a730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 >> 172.21.15.89:0/3344986323 conn(0x7f887c09ba50 msgr2=0x7f887c069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 shutdown_connections 2023-09-30T22:13:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.602+0000 7f888139a700 1 -- 172.21.15.89:0/3344986323 wait complete. 2023-09-30T22:13:59.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:13:59.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2e5aac700 1 Processor -- start 2023-09-30T22:13:59.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2e5aac700 1 -- start start 2023-09-30T22:13:59.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2e5aac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe2e0098740 0x7fe2e0098b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2e5aac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0099010 0x7fe2e0068d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2e5aac700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e0095270 con 0x7fe2e0069210 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2e5aac700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e0094d20 con 0x7fe2e0098740 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2e5aac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0096520 con 0x7fe2e0099010 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2df7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe2e0098740 0x7fe2e0098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2deffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0099010 0x7fe2e0068d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2df7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe2e0098740 0x7fe2e0098b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42814/0 (socket says 172.21.15.89:42814) 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2df7fe700 1 -- 172.21.15.89:0/3859971464 learned_addr learned my addr 172.21.15.89:0/3859971464 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.734+0000 7fe2dffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe2e0069210 0x7fe2e012fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40282/0 (socket says 172.21.15.89:40282) 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2df7fe700 1 -- 172.21.15.89:0/3859971464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0099010 msgr2=0x7fe2e0068d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2df7fe700 1 --2- 172.21.15.89:0/3859971464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0099010 0x7fe2e0068d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2df7fe700 1 -- 172.21.15.89:0/3859971464 >> v1:172.21.15.89:6789/0 conn(0x7fe2e0069210 legacy=0x7fe2e012fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2df7fe700 1 -- 172.21.15.89:0/3859971464 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0094870 con 0x7fe2e0098740 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2de7fc700 1 -- 172.21.15.89:0/3859971464 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140310256 0 0) 0x7fe2e0095270 con 0x7fe2e0069210 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2deffd700 1 --2- 172.21.15.89:0/3859971464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0099010 0x7fe2e0068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2df7fe700 1 --2- 172.21.15.89:0/3859971464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0098740 0x7fe2e0098b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe2d4006fd0 tx=0x7fe2d40090f0).ready entity=mon.0 client_cookie=cd79ef284aa43e77 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2de7fc700 1 -- 172.21.15.89:0/3859971464 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400b580 con 0x7fe2e0098740 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2de7fc700 1 -- 172.21.15.89:0/3859971464 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d400b6e0 con 0x7fe2e0098740 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2de7fc700 1 -- 172.21.15.89:0/3859971464 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400ba80 con 0x7fe2e0098740 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3859971464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0098740 msgr2=0x7fe2e0098b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2e5aac700 1 --2- 172.21.15.89:0/3859971464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0098740 0x7fe2e0098b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe2d4006fd0 tx=0x7fe2d40090f0).stop 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3859971464 shutdown_connections 2023-09-30T22:13:59.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2e5aac700 1 --2- 172.21.15.89:0/3859971464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0098740 0x7fe2e0098b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2e5aac700 1 --2- 172.21.15.89:0/3859971464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0099010 0x7fe2e0068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.735+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3859971464 >> 172.21.15.89:0/3859971464 conn(0x7fe2e009a1c0 msgr2=0x7fe2e009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:59.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.736+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3859971464 shutdown_connections 2023-09-30T22:13:59.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.736+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3859971464 wait complete. 2023-09-30T22:13:59.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.736+0000 7fe2e5aac700 1 Processor -- start 2023-09-30T22:13:59.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2e5aac700 1 -- start start 2023-09-30T22:13:59.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2e5aac700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2e0069210 0x7fe2e0115650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2e5aac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0098740 0x7fe2e0115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2e5aac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0099010 0x7fe2e0116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:13:59.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2e5aac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0115420 con 0x7fe2e0099010 2023-09-30T22:13:59.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2e5aac700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0118440 con 0x7fe2e0069210 2023-09-30T22:13:59.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2e5aac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0098010 con 0x7fe2e0098740 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0099010 0x7fe2e0116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2df7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2e0069210 0x7fe2e0115650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0099010 0x7fe2e0116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42828/0 (socket says 172.21.15.89:42828) 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2df7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2e0069210 0x7fe2e0115650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56032/0 (socket says 172.21.15.89:56032) 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2dffff700 1 -- 172.21.15.89:0/3499687416 learned_addr learned my addr 172.21.15.89:0/3499687416 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2deffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0098740 0x7fe2e0115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2deffd700 1 -- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2e0069210 msgr2=0x7fe2e0115650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2deffd700 1 --2- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2e0069210 0x7fe2e0115650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2deffd700 1 -- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0099010 msgr2=0x7fe2e0116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2deffd700 1 --2- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0099010 0x7fe2e0116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2deffd700 1 -- 172.21.15.89:0/3499687416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d00092a0 con 0x7fe2e0098740 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2dffff700 1 --2- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0099010 0x7fe2e0116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.737+0000 7fe2deffd700 1 --2- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0098740 0x7fe2e0115b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe2d000e950 tx=0x7fe2d0009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.738+0000 7fe2c77fe700 1 -- 172.21.15.89:0/3499687416 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0009580 con 0x7fe2e0098740 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.738+0000 7fe2c77fe700 1 -- 172.21.15.89:0/3499687416 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d0012600 con 0x7fe2e0098740 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.738+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d4009370 con 0x7fe2e0098740 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.738+0000 7fe2c77fe700 1 -- 172.21.15.89:0/3499687416 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00129a0 con 0x7fe2e0098740 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.738+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e01190a0 con 0x7fe2e0098740 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.738+0000 7fe2c77fe700 1 -- 172.21.15.89:0/3499687416 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fe2d0012d10 con 0x7fe2e0098740 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.738+0000 7fe2c77fe700 1 -- 172.21.15.89:0/3499687416 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fe2d003b110 con 0x7fe2e0098740 2023-09-30T22:13:59.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.739+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e005a260 con 0x7fe2e0098740 2023-09-30T22:13:59.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.743+0000 7fe2c77fe700 1 -- 172.21.15.89:0/3499687416 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fe2d0016030 con 0x7fe2e0098740 2023-09-30T22:13:59.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.900+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe2e0094540 con 0x7fe2e0098740 2023-09-30T22:13:59.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.901+0000 7fe2c77fe700 1 -- 172.21.15.89:0/3499687416 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v13) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe2d0011b40 con 0x7fe2e0098740 2023-09-30T22:13:59.901 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:13:59.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.902+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 >> v1:172.21.15.89:6816/118161 conn(0x7fe2cc02bb60 legacy=0x7fe2cc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.902+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0098740 msgr2=0x7fe2e0115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:13:59.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.902+0000 7fe2e5aac700 1 --2- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0098740 0x7fe2e0115b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe2d000e950 tx=0x7fe2d0009060).stop 2023-09-30T22:13:59.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.903+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 shutdown_connections 2023-09-30T22:13:59.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.903+0000 7fe2e5aac700 1 --2- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0099010 0x7fe2e0116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.903+0000 7fe2e5aac700 1 --2- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2e0069210 0x7fe2e0115650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.903+0000 7fe2e5aac700 1 --2- 172.21.15.89:0/3499687416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2e0098740 0x7fe2e0115b40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:13:59.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.903+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 >> 172.21.15.89:0/3499687416 conn(0x7fe2e009a1c0 msgr2=0x7fe2e0065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:13:59.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.903+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 shutdown_connections 2023-09-30T22:13:59.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:13:59.903+0000 7fe2e5aac700 1 -- 172.21.15.89:0/3499687416 wait complete. 2023-09-30T22:13:59.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-30T22:14:00.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.029+0000 7f8b04227700 1 Processor -- start 2023-09-30T22:14:00.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.029+0000 7f8b04227700 1 -- start start 2023-09-30T22:14:00.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b04227700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc091b00 0x7f8afc12c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b04227700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8afc12ce60 0x7f8afc12f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b04227700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc096760 con 0x7f8afc096380 2023-09-30T22:14:00.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b04227700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc091ff0 con 0x7f8afc12ce60 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b04227700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc092c80 con 0x7f8afc091b00 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b017c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc091b00 0x7f8afc12c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b027c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8afc12ce60 0x7f8afc12f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b017c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc091b00 0x7f8afc12c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56036/0 (socket says 172.21.15.89:56036) 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b027c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8afc12ce60 0x7f8afc12f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42840/0 (socket says 172.21.15.89:42840) 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b017c2700 1 -- 172.21.15.89:0/3332505690 learned_addr learned my addr 172.21.15.89:0/3332505690 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b01fc3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8afc096380 0x7f8afc094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40298/0 (socket says 172.21.15.89:40298) 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b027c4700 1 -- 172.21.15.89:0/3332505690 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc091b00 msgr2=0x7f8afc12c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b027c4700 1 --2- 172.21.15.89:0/3332505690 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc091b00 0x7f8afc12c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b027c4700 1 -- 172.21.15.89:0/3332505690 >> v1:172.21.15.89:6789/0 conn(0x7f8afc096380 legacy=0x7f8afc094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b027c4700 1 -- 172.21.15.89:0/3332505690 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc094e40 con 0x7f8afc12ce60 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b00fc1700 1 -- 172.21.15.89:0/3332505690 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979372993 0 0) 0x7f8afc096760 con 0x7f8afc096380 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b017c2700 1 --2- 172.21.15.89:0/3332505690 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc091b00 0x7f8afc12c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b027c4700 1 --2- 172.21.15.89:0/3332505690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc12ce60 0x7f8afc12f250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8af4009e60 tx=0x7f8af4009780).ready entity=mon.0 client_cookie=b0b150454f578706 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b00fc1700 1 -- 172.21.15.89:0/3332505690 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400e070 con 0x7f8afc12ce60 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b00fc1700 1 -- 172.21.15.89:0/3332505690 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af400bb60 con 0x7f8afc12ce60 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.030+0000 7f8b00fc1700 1 -- 172.21.15.89:0/3332505690 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4012410 con 0x7f8afc12ce60 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.031+0000 7f8b04227700 1 -- 172.21.15.89:0/3332505690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc12ce60 msgr2=0x7f8afc12f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.031+0000 7f8b04227700 1 --2- 172.21.15.89:0/3332505690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc12ce60 0x7f8afc12f250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8af4009e60 tx=0x7f8af4009780).stop 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.031+0000 7f8b04227700 1 -- 172.21.15.89:0/3332505690 shutdown_connections 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.031+0000 7f8b04227700 1 --2- 172.21.15.89:0/3332505690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc12ce60 0x7f8afc12f250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.031+0000 7f8b04227700 1 --2- 172.21.15.89:0/3332505690 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc091b00 0x7f8afc12c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.031+0000 7f8b04227700 1 -- 172.21.15.89:0/3332505690 >> 172.21.15.89:0/3332505690 conn(0x7f8afc097470 msgr2=0x7f8afc097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:00.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.031+0000 7f8b04227700 1 -- 172.21.15.89:0/3332505690 shutdown_connections 2023-09-30T22:14:00.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.031+0000 7f8b04227700 1 -- 172.21.15.89:0/3332505690 wait complete. 2023-09-30T22:14:00.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b04227700 1 Processor -- start 2023-09-30T22:14:00.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b04227700 1 -- start start 2023-09-30T22:14:00.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b04227700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc091b00 0x7f8afc124010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b04227700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc096380 0x7f8afc124500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b04227700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8afc12ce60 0x7f8afc1254e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b04227700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc1167d0 con 0x7f8afc091b00 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b04227700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc115b50 con 0x7f8afc096380 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b04227700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc06a520 con 0x7f8afc12ce60 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b01fc3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc091b00 0x7f8afc124010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b01fc3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc091b00 0x7f8afc124010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42850/0 (socket says 172.21.15.89:42850) 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b027c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8afc12ce60 0x7f8afc1254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b01fc3700 1 -- 172.21.15.89:0/66050186 learned_addr learned my addr 172.21.15.89:0/66050186 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.032+0000 7f8b017c2700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc096380 0x7f8afc124500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b01fc3700 1 -- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8afc12ce60 msgr2=0x7f8afc1254e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b01fc3700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8afc12ce60 0x7f8afc1254e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b01fc3700 1 -- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc096380 msgr2=0x7f8afc124500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b01fc3700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc096380 0x7f8afc124500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b01fc3700 1 -- 172.21.15.89:0/66050186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af8009370 con 0x7f8afc091b00 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b027c4700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8afc12ce60 0x7f8afc1254e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b01fc3700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc091b00 0x7f8afc124010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8aec00c4f0 tx=0x7f8aec007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8af1ffb700 1 -- 172.21.15.89:0/66050186 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec00ce20 con 0x7f8afc091b00 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8af1ffb700 1 -- 172.21.15.89:0/66050186 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec0072e0 con 0x7f8afc091b00 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8af1ffb700 1 -- 172.21.15.89:0/66050186 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0056e0 con 0x7f8afc091b00 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af4009a20 con 0x7f8afc091b00 2023-09-30T22:14:00.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.033+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc126c00 con 0x7f8afc091b00 2023-09-30T22:14:00.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.034+0000 7f8af1ffb700 1 -- 172.21.15.89:0/66050186 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8aec005a90 con 0x7f8afc091b00 2023-09-30T22:14:00.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.034+0000 7f8af1ffb700 1 -- 172.21.15.89:0/66050186 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f8aec03a350 con 0x7f8afc091b00 2023-09-30T22:14:00.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.034+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc05a260 con 0x7f8afc091b00 2023-09-30T22:14:00.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.038+0000 7f8af1ffb700 1 -- 172.21.15.89:0/66050186 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8aec01c350 con 0x7f8afc091b00 2023-09-30T22:14:00.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.199+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8afc1251d0 con 0x7f8afc091b00 2023-09-30T22:14:00.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.200+0000 7f8af1ffb700 1 -- 172.21.15.89:0/66050186 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v13) v1 ==== 84+0+143 (secure 0 0 0) 0x7f8aec010af0 con 0x7f8afc091b00 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": "device_health_metrics" 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 2, 2023-09-30T22:14:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": "rbd" 2023-09-30T22:14:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:14:00.202 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 >> v1:172.21.15.89:6816/118161 conn(0x7f8ae002bb60 legacy=0x7f8ae002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc091b00 msgr2=0x7f8afc124010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc091b00 0x7f8afc124010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8aec00c4f0 tx=0x7f8aec007c10).stop 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 shutdown_connections 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8afc12ce60 0x7f8afc1254e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8afc091b00 0x7f8afc124010 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 --2- 172.21.15.89:0/66050186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8afc096380 0x7f8afc124500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 >> 172.21.15.89:0/66050186 conn(0x7f8afc097470 msgr2=0x7f8afc1276f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 shutdown_connections 2023-09-30T22:14:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.202+0000 7f8b04227700 1 -- 172.21.15.89:0/66050186 wait complete. 2023-09-30T22:14:00.214 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-30T22:14:00.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-30T22:14:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.332+0000 7f558ea64700 1 Processor -- start 2023-09-30T22:14:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558ea64700 1 -- start start 2023-09-30T22:14:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558ea64700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5588069370 0x7f5588096410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558ea64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588096900 0x7f5588096ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558ea64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f558812d2b0 0x7f558812f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558ea64700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5588069750 con 0x7f5588069370 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558ea64700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558806a770 con 0x7f558812d2b0 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558ea64700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5588069a30 con 0x7f5588096900 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f5587fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588096900 0x7f5588096ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558c800700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5588069370 0x7f5588096410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f5587fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588096900 0x7f5588096ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57264/0 (socket says 172.21.15.89:57264) 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558c800700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5588069370 0x7f5588096410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38652/0 (socket says 172.21.15.89:38652) 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f5587fff700 1 -- 172.21.15.89:0/2718053577 learned_addr learned my addr 172.21.15.89:0/2718053577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f558d001700 1 --2- 172.21.15.89:0/2718053577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f558812d2b0 0x7f558812f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f5587fff700 1 -- 172.21.15.89:0/2718053577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f558812d2b0 msgr2=0x7f558812f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.333+0000 7f5587fff700 1 --2- 172.21.15.89:0/2718053577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f558812d2b0 0x7f558812f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f5587fff700 1 -- 172.21.15.89:0/2718053577 >> v2:172.21.15.89:3300/0 conn(0x7f5588069370 msgr2=0x7f5588096410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f5587fff700 1 --2- 172.21.15.89:0/2718053577 >> v2:172.21.15.89:3300/0 conn(0x7f5588069370 0x7f5588096410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f5587fff700 1 -- 172.21.15.89:0/2718053577 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588097810 con 0x7f5588096900 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558c800700 1 --2- 172.21.15.89:0/2718053577 >> v2:172.21.15.89:3300/0 conn(0x7f5588069370 0x7f5588096410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f5587fff700 1 --2- 172.21.15.89:0/2718053577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588096900 0x7f5588096ce0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5574009830 tx=0x7f55740090f0).ready entity=mon.2 client_cookie=a81b75a6a526ef41 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f55877fe700 1 -- 172.21.15.89:0/2718053577 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557400f510 con 0x7f5588096900 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f55877fe700 1 -- 172.21.15.89:0/2718053577 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557400f670 con 0x7f5588096900 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f55877fe700 1 -- 172.21.15.89:0/2718053577 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557400f9f0 con 0x7f5588096900 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558ea64700 1 -- 172.21.15.89:0/2718053577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588096900 msgr2=0x7f5588096ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558ea64700 1 --2- 172.21.15.89:0/2718053577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588096900 0x7f5588096ce0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5574009830 tx=0x7f55740090f0).stop 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558ea64700 1 -- 172.21.15.89:0/2718053577 shutdown_connections 2023-09-30T22:14:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558ea64700 1 --2- 172.21.15.89:0/2718053577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f558812d2b0 0x7f558812f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558ea64700 1 --2- 172.21.15.89:0/2718053577 >> v2:172.21.15.89:3300/0 conn(0x7f5588069370 0x7f5588096410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558ea64700 1 --2- 172.21.15.89:0/2718053577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588096900 0x7f5588096ce0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558ea64700 1 -- 172.21.15.89:0/2718053577 >> 172.21.15.89:0/2718053577 conn(0x7f5588059cd0 msgr2=0x7f558809bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:00.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.334+0000 7f558ea64700 1 -- 172.21.15.89:0/2718053577 shutdown_connections 2023-09-30T22:14:00.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.335+0000 7f558ea64700 1 -- 172.21.15.89:0/2718053577 wait complete. 2023-09-30T22:14:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.335+0000 7f558ea64700 1 Processor -- start 2023-09-30T22:14:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.335+0000 7f558ea64700 1 -- start start 2023-09-30T22:14:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558ea64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588069370 0x7f558812cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558ea64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5588096900 0x7f5588117ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558ea64700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558812d2b0 0x7f55881181d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558ea64700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558811b850 con 0x7f558812d2b0 2023-09-30T22:14:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558ea64700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5588091c10 con 0x7f5588096900 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558ea64700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5588067bb0 con 0x7f5588069370 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558c800700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588069370 0x7f558812cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f5587fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5588096900 0x7f5588117ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558d001700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558812d2b0 0x7f55881181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558c800700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588069370 0x7f558812cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57268/0 (socket says 172.21.15.89:57268) 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558c800700 1 -- 172.21.15.89:0/3228119547 learned_addr learned my addr 172.21.15.89:0/3228119547 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558d001700 1 -- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588069370 msgr2=0x7f558812cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558d001700 1 --2- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588069370 0x7f558812cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558d001700 1 -- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5588096900 msgr2=0x7f5588117ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558d001700 1 --2- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5588096900 0x7f5588117ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558d001700 1 -- 172.21.15.89:0/3228119547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c0092a0 con 0x7f558812d2b0 2023-09-30T22:14:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.336+0000 7f558d001700 1 --2- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558812d2b0 0x7f55881181d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5580009e20 tx=0x7f558000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.337+0000 7f5584ff9700 1 -- 172.21.15.89:0/3228119547 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558000caa0 con 0x7f558812d2b0 2023-09-30T22:14:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.337+0000 7f5584ff9700 1 -- 172.21.15.89:0/3228119547 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558000f040 con 0x7f558812d2b0 2023-09-30T22:14:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.337+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5574009370 con 0x7f558812d2b0 2023-09-30T22:14:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.337+0000 7f5584ff9700 1 -- 172.21.15.89:0/3228119547 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558000ce10 con 0x7f558812d2b0 2023-09-30T22:14:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.337+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55881190c0 con 0x7f558812d2b0 2023-09-30T22:14:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.337+0000 7f5584ff9700 1 -- 172.21.15.89:0/3228119547 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5580013610 con 0x7f558812d2b0 2023-09-30T22:14:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.338+0000 7f5584ff9700 1 -- 172.21.15.89:0/3228119547 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f558003b3d0 con 0x7f558812d2b0 2023-09-30T22:14:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.338+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558811b340 con 0x7f558812d2b0 2023-09-30T22:14:00.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.342+0000 7f5584ff9700 1 -- 172.21.15.89:0/3228119547 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f558811b340 con 0x7f558812d2b0 2023-09-30T22:14:00.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.500+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f558811b340 con 0x7f558812d2b0 2023-09-30T22:14:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.670+0000 7f5584ff9700 1 -- 172.21.15.89:0/3228119547 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v14) v1 ==== 138+0+0 (secure 0 0 0) 0x7f558811b340 con 0x7f558812d2b0 2023-09-30T22:14:00.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 >> v1:172.21.15.89:6816/118161 conn(0x7f556c02bb90 legacy=0x7f556c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558812d2b0 msgr2=0x7f55881181d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 --2- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558812d2b0 0x7f55881181d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5580009e20 tx=0x7f558000b040).stop 2023-09-30T22:14:00.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 shutdown_connections 2023-09-30T22:14:00.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 --2- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558812d2b0 0x7f55881181d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 --2- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5588069370 0x7f558812cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 --2- 172.21.15.89:0/3228119547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5588096900 0x7f5588117ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 >> 172.21.15.89:0/3228119547 conn(0x7f5588059cd0 msgr2=0x7f5588112120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:00.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 shutdown_connections 2023-09-30T22:14:00.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.673+0000 7f558ea64700 1 -- 172.21.15.89:0/3228119547 wait complete. 2023-09-30T22:14:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-09-30T22:14:00.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-30T22:14:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.812+0000 7f16a3566700 1 Processor -- start 2023-09-30T22:14:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.812+0000 7f16a3566700 1 -- start start 2023-09-30T22:14:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a3566700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c12ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a3566700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f169c130400 0x7f169c1327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a3566700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c066d80 con 0x7f169c0613a0 2023-09-30T22:14:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a3566700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169c061780 con 0x7f169c130400 2023-09-30T22:14:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a3566700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f169c12ac40 con 0x7f169c12b540 2023-09-30T22:14:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a0b01700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c12ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a1302700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f169c0613a0 0x7f169c12ae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51770/0 (socket says 172.21.15.89:51770) 2023-09-30T22:14:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a1302700 1 -- 172.21.15.89:0/3740941248 learned_addr learned my addr 172.21.15.89:0/3740941248 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a1b03700 1 --2- 172.21.15.89:0/3740941248 >> v2:172.21.15.89:3300/0 conn(0x7f169c130400 0x7f169c1327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f1693fff700 1 -- 172.21.15.89:0/3740941248 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461286311 0 0) 0x7f169c066d80 con 0x7f169c0613a0 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a1b03700 1 -- 172.21.15.89:0/3740941248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 msgr2=0x7f169c12ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a1b03700 1 --2- 172.21.15.89:0/3740941248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c12ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a1b03700 1 -- 172.21.15.89:0/3740941248 >> v1:172.21.15.89:6789/0 conn(0x7f169c0613a0 legacy=0x7f169c12ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a1b03700 1 -- 172.21.15.89:0/3740941248 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c092fb0 con 0x7f169c130400 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a0b01700 1 --2- 172.21.15.89:0/3740941248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c12ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f16a1b03700 1 --2- 172.21.15.89:0/3740941248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c130400 0x7f169c1327f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1694009e60 tx=0x7f1694009780).ready entity=mon.0 client_cookie=11653aec546a615c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.813+0000 7f1693fff700 1 -- 172.21.15.89:0/3740941248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169400e070 con 0x7f169c130400 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f1693fff700 1 -- 172.21.15.89:0/3740941248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169400bb60 con 0x7f169c130400 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f1693fff700 1 -- 172.21.15.89:0/3740941248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694012410 con 0x7f169c130400 2023-09-30T22:14:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f16a3566700 1 -- 172.21.15.89:0/3740941248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c130400 msgr2=0x7f169c1327f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f16a3566700 1 --2- 172.21.15.89:0/3740941248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c130400 0x7f169c1327f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1694009e60 tx=0x7f1694009780).stop 2023-09-30T22:14:00.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f16a3566700 1 -- 172.21.15.89:0/3740941248 shutdown_connections 2023-09-30T22:14:00.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f16a3566700 1 --2- 172.21.15.89:0/3740941248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c130400 0x7f169c1327f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f16a3566700 1 --2- 172.21.15.89:0/3740941248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c12ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f16a3566700 1 -- 172.21.15.89:0/3740941248 >> 172.21.15.89:0/3740941248 conn(0x7f169c09a3d0 msgr2=0x7f169c09a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:00.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f16a3566700 1 -- 172.21.15.89:0/3740941248 shutdown_connections 2023-09-30T22:14:00.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.814+0000 7f16a3566700 1 -- 172.21.15.89:0/3740941248 wait complete. 2023-09-30T22:14:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.815+0000 7f16a3566700 1 Processor -- start 2023-09-30T22:14:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.815+0000 7f16a3566700 1 -- start start 2023-09-30T22:14:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a3566700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c0613a0 0x7f169c12a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a3566700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c114650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a3566700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130400 0x7f169c114b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a3566700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c066f90 con 0x7f169c0613a0 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a3566700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c1186b0 con 0x7f169c130400 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a3566700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f169c1180f0 con 0x7f169c12b540 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a1302700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c0613a0 0x7f169c12a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a0b01700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c114650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a1302700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c0613a0 0x7f169c12a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38664/0 (socket says 172.21.15.89:38664) 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a1302700 1 -- 172.21.15.89:0/4118969311 learned_addr learned my addr 172.21.15.89:0/4118969311 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.816+0000 7f16a1b03700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130400 0x7f169c114b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a1302700 1 -- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 msgr2=0x7f169c114650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a1302700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c114650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a1302700 1 -- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130400 msgr2=0x7f169c114b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a1302700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130400 0x7f169c114b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a1302700 1 -- 172.21.15.89:0/4118969311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16980092a0 con 0x7f169c0613a0 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a0b01700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c114650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a1b03700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130400 0x7f169c114b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a1302700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c0613a0 0x7f169c12a6d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f168c00c4c0 tx=0x7f168c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16917fa700 1 -- 172.21.15.89:0/4118969311 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c00ce30 con 0x7f169c0613a0 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16917fa700 1 -- 172.21.15.89:0/4118969311 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c0072e0 con 0x7f169c0613a0 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16917fa700 1 -- 172.21.15.89:0/4118969311 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c0057a0 con 0x7f169c0613a0 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1694009a20 con 0x7f169c0613a0 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.817+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c117b50 con 0x7f169c0613a0 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.818+0000 7f16917fa700 1 -- 172.21.15.89:0/4118969311 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f168c005bd0 con 0x7f169c0613a0 2023-09-30T22:14:00.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.818+0000 7f16917fa700 1 -- 172.21.15.89:0/4118969311 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3208+0+0 (secure 0 0 0) 0x7f168c03a500 con 0x7f169c0613a0 2023-09-30T22:14:00.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.818+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c117f30 con 0x7f169c0613a0 2023-09-30T22:14:00.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.822+0000 7f16917fa700 1 -- 172.21.15.89:0/4118969311 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f168c01d3f0 con 0x7f169c0613a0 2023-09-30T22:14:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:00.978+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f169c05a260 con 0x7f169c0613a0 2023-09-30T22:14:01.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.679+0000 7f16917fa700 1 -- 172.21.15.89:0/4118969311 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v15) v1 ==== 170+0+0 (secure 0 0 0) 0x7f168c01c350 con 0x7f169c0613a0 2023-09-30T22:14:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.684+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 >> v1:172.21.15.89:6816/118161 conn(0x7f168002bb30 legacy=0x7f168002dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.684+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c0613a0 msgr2=0x7f169c12a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.684+0000 7f16a3566700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c0613a0 0x7f169c12a6d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f168c00c4c0 tx=0x7f168c007c10).stop 2023-09-30T22:14:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.688+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 shutdown_connections 2023-09-30T22:14:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.688+0000 7f16a3566700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130400 0x7f169c114b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.688+0000 7f16a3566700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c0613a0 0x7f169c12a6d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.688+0000 7f16a3566700 1 --2- 172.21.15.89:0/4118969311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c12b540 0x7f169c114650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.688+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 >> 172.21.15.89:0/4118969311 conn(0x7f169c09a3d0 msgr2=0x7f169c0919e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:01.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.689+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 shutdown_connections 2023-09-30T22:14:01.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.690+0000 7f16a3566700 1 -- 172.21.15.89:0/4118969311 wait complete. 2023-09-30T22:14:01.695 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-09-30T22:14:01.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-30T22:14:01.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.832+0000 7fc39684d700 1 Processor -- start 2023-09-30T22:14:01.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.832+0000 7fc39684d700 1 -- start start 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.832+0000 7fc39684d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900920f0 0x7fc3900924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.832+0000 7fc39684d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc39012cde0 0x7fc39012f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.832+0000 7fc39684d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc390092c00 con 0x7fc3900613a0 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.832+0000 7fc39684d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc390061780 con 0x7fc39012cde0 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.832+0000 7fc39684d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc390092ff0 con 0x7fc3900920f0 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc394dea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc39012cde0 0x7fc39012f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900920f0 0x7fc3900924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc394dea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc39012cde0 0x7fc39012f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38678/0 (socket says 172.21.15.89:38678) 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900920f0 0x7fc3900924d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57292/0 (socket says 172.21.15.89:57292) 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc394dea700 1 -- 172.21.15.89:0/1362850235 learned_addr learned my addr 172.21.15.89:0/1362850235 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:01.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38ffff700 1 --1- 172.21.15.89:0/1362850235 >> v1:172.21.15.89:6789/0 conn(0x7fc3900613a0 0x7fc3900919e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51786/0 (socket says 172.21.15.89:51786) 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38f7fe700 1 -- 172.21.15.89:0/1362850235 >> v2:172.21.15.89:3300/0 conn(0x7fc39012cde0 msgr2=0x7fc39012f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38f7fe700 1 --2- 172.21.15.89:0/1362850235 >> v2:172.21.15.89:3300/0 conn(0x7fc39012cde0 0x7fc39012f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38f7fe700 1 -- 172.21.15.89:0/1362850235 >> v1:172.21.15.89:6789/0 conn(0x7fc3900613a0 legacy=0x7fc3900919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38f7fe700 1 -- 172.21.15.89:0/1362850235 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390093b20 con 0x7fc3900920f0 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38effd700 1 -- 172.21.15.89:0/1362850235 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714975244 0 0) 0x7fc390092c00 con 0x7fc3900613a0 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc394dea700 1 --2- 172.21.15.89:0/1362850235 >> v2:172.21.15.89:3300/0 conn(0x7fc39012cde0 0x7fc39012f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38f7fe700 1 --2- 172.21.15.89:0/1362850235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900920f0 0x7fc3900924d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc380009760 tx=0x7fc380009060).ready entity=mon.2 client_cookie=4012875d80a141d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38effd700 1 -- 172.21.15.89:0/1362850235 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38000f510 con 0x7fc3900920f0 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38effd700 1 -- 172.21.15.89:0/1362850235 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38000f670 con 0x7fc3900920f0 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.833+0000 7fc38effd700 1 -- 172.21.15.89:0/1362850235 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38000f9f0 con 0x7fc3900920f0 2023-09-30T22:14:01.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.834+0000 7fc39684d700 1 -- 172.21.15.89:0/1362850235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900920f0 msgr2=0x7fc3900924d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:01.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.834+0000 7fc39684d700 1 --2- 172.21.15.89:0/1362850235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900920f0 0x7fc3900924d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc380009760 tx=0x7fc380009060).stop 2023-09-30T22:14:01.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.834+0000 7fc39684d700 1 -- 172.21.15.89:0/1362850235 shutdown_connections 2023-09-30T22:14:01.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.834+0000 7fc39684d700 1 --2- 172.21.15.89:0/1362850235 >> v2:172.21.15.89:3300/0 conn(0x7fc39012cde0 0x7fc39012f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:01.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.834+0000 7fc39684d700 1 --2- 172.21.15.89:0/1362850235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900920f0 0x7fc3900924d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:01.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.834+0000 7fc39684d700 1 -- 172.21.15.89:0/1362850235 >> 172.21.15.89:0/1362850235 conn(0x7fc39009a1c0 msgr2=0x7fc39009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:01.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.834+0000 7fc39684d700 1 -- 172.21.15.89:0/1362850235 shutdown_connections 2023-09-30T22:14:01.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.834+0000 7fc39684d700 1 -- 172.21.15.89:0/1362850235 wait complete. 2023-09-30T22:14:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc39684d700 1 Processor -- start 2023-09-30T22:14:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc39684d700 1 -- start start 2023-09-30T22:14:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc39684d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900613a0 0x7fc3901373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc39684d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3900920f0 0x7fc3901378e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc39684d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39012cde0 0x7fc390112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc39684d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3901286c0 con 0x7fc39012cde0 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc39684d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc390114260 con 0x7fc3900920f0 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc39684d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc39012d1c0 con 0x7fc3900613a0 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc38f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3900920f0 0x7fc3901378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc394dea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39012cde0 0x7fc390112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc38f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3900920f0 0x7fc3901378e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52058/0 (socket says 172.21.15.89:52058) 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc394dea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39012cde0 0x7fc390112d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38688/0 (socket says 172.21.15.89:38688) 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc38ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900613a0 0x7fc3901373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.835+0000 7fc38f7fe700 1 -- 172.21.15.89:0/3026966481 learned_addr learned my addr 172.21.15.89:0/3026966481 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc394dea700 1 -- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900613a0 msgr2=0x7fc3901373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc394dea700 1 --2- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900613a0 0x7fc3901373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc394dea700 1 -- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3900920f0 msgr2=0x7fc3901378e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc394dea700 1 --2- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3900920f0 0x7fc3901378e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc394dea700 1 -- 172.21.15.89:0/3026966481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388009ae0 con 0x7fc39012cde0 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc394dea700 1 --2- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39012cde0 0x7fc390112d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc38800e6b0 tx=0x7fc38800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc377fff700 1 -- 172.21.15.89:0/3026966481 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880090d0 con 0x7fc39012cde0 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc377fff700 1 -- 172.21.15.89:0/3026966481 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc388009230 con 0x7fc39012cde0 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3800092a0 con 0x7fc39012cde0 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc377fff700 1 -- 172.21.15.89:0/3026966481 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388012640 con 0x7fc39012cde0 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.836+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc390115ff0 con 0x7fc39012cde0 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.837+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc390068480 con 0x7fc39012cde0 2023-09-30T22:14:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.837+0000 7fc377fff700 1 -- 172.21.15.89:0/3026966481 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc3880129b0 con 0x7fc39012cde0 2023-09-30T22:14:01.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.838+0000 7fc377fff700 1 -- 172.21.15.89:0/3026966481 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3590+0+0 (secure 0 0 0) 0x7fc38803dc30 con 0x7fc39012cde0 2023-09-30T22:14:01.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:01.842+0000 7fc377fff700 1 -- 172.21.15.89:0/3026966481 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc3880245c0 con 0x7fc39012cde0 2023-09-30T22:14:02.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.035+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc39005a260 con 0x7fc39012cde0 2023-09-30T22:14:02.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.045+0000 7fc377fff700 1 -- 172.21.15.89:0/3026966481 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v3) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc388020020 con 0x7fc39012cde0 2023-09-30T22:14:02.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 >> v1:172.21.15.89:6816/118161 conn(0x7fc37c02bb60 legacy=0x7fc37c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39012cde0 msgr2=0x7fc390112d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 --2- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39012cde0 0x7fc390112d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc38800e6b0 tx=0x7fc38800b040).stop 2023-09-30T22:14:02.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 shutdown_connections 2023-09-30T22:14:02.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 --2- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39012cde0 0x7fc390112d80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 --2- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3900613a0 0x7fc3901373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 --2- 172.21.15.89:0/3026966481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3900920f0 0x7fc3901378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 >> 172.21.15.89:0/3026966481 conn(0x7fc39009a1c0 msgr2=0x7fc3900672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:02.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.052+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 shutdown_connections 2023-09-30T22:14:02.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.053+0000 7fc39684d700 1 -- 172.21.15.89:0/3026966481 wait complete. 2023-09-30T22:14:02.055 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 3 and data pool 4 2023-09-30T22:14:02.072 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:02.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.195+0000 7f04febee700 1 Processor -- start 2023-09-30T22:14:02.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.195+0000 7f04febee700 1 -- start start 2023-09-30T22:14:02.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04febee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8096380 0x7f04f8094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04febee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04f8091b00 0x7f04f8127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04febee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8094c20 0x7f04f8127af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04febee700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f8096760 con 0x7f04f8091b00 2023-09-30T22:14:02.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04febee700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f8091ff0 con 0x7f04f8096380 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04febee700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04f8092c80 con 0x7f04f8094c20 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04fc98a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8096380 0x7f04f8094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04fd18b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8094c20 0x7f04f8127af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04f7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04f8091b00 0x7f04f8127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04fd18b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8094c20 0x7f04f8127af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57306/0 (socket says 172.21.15.89:57306) 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04fc98a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8096380 0x7f04f8094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52064/0 (socket says 172.21.15.89:52064) 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04fd18b700 1 -- 172.21.15.89:0/2945655549 learned_addr learned my addr 172.21.15.89:0/2945655549 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04f7fff700 1 -- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8094c20 msgr2=0x7f04f8127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04f7fff700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8094c20 0x7f04f8127af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04f7fff700 1 -- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8096380 msgr2=0x7f04f8094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04f7fff700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8096380 0x7f04f8094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04f7fff700 1 -- 172.21.15.89:0/2945655549 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f8093900 con 0x7f04f8091b00 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04fd18b700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8094c20 0x7f04f8127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04fc98a700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8096380 0x7f04f8094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.196+0000 7f04f7fff700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8091b00 0x7f04f8127600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f04e400ba00 tx=0x7f04e400b300).ready entity=mon.0 client_cookie=7971bf5dcfd02ffd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04f77fe700 1 -- 172.21.15.89:0/2945655549 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400d8c0 con 0x7f04f8091b00 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04f77fe700 1 -- 172.21.15.89:0/2945655549 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e400da20 con 0x7f04f8091b00 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04f77fe700 1 -- 172.21.15.89:0/2945655549 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400dda0 con 0x7f04f8091b00 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 -- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8091b00 msgr2=0x7f04f8127600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8091b00 0x7f04f8127600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f04e400ba00 tx=0x7f04e400b300).stop 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 -- 172.21.15.89:0/2945655549 shutdown_connections 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8094c20 0x7f04f8127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8096380 0x7f04f8094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 --2- 172.21.15.89:0/2945655549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8091b00 0x7f04f8127600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 -- 172.21.15.89:0/2945655549 >> 172.21.15.89:0/2945655549 conn(0x7f04f8097470 msgr2=0x7f04f8097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:02.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 -- 172.21.15.89:0/2945655549 shutdown_connections 2023-09-30T22:14:02.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.197+0000 7f04febee700 1 -- 172.21.15.89:0/2945655549 wait complete. 2023-09-30T22:14:02.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.198+0000 7f04febee700 1 Processor -- start 2023-09-30T22:14:02.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.198+0000 7f04febee700 1 -- start start 2023-09-30T22:14:02.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04febee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8091b00 0x7f04f8115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04febee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8094c20 0x7f04f8112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04febee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8096380 0x7f04f81131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04febee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f806a520 con 0x7f04f8096380 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04febee700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f81161c0 con 0x7f04f8094c20 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04febee700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04f8116c60 con 0x7f04f8091b00 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04f7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8094c20 0x7f04f8112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04f7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8094c20 0x7f04f8112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52076/0 (socket says 172.21.15.89:52076) 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04f7fff700 1 -- 172.21.15.89:0/4007363282 learned_addr learned my addr 172.21.15.89:0/4007363282 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04fc98a700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8091b00 0x7f04f8115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04fd18b700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8096380 0x7f04f81131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04f7fff700 1 -- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8091b00 msgr2=0x7f04f8115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04f7fff700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8091b00 0x7f04f8115c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04f7fff700 1 -- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8096380 msgr2=0x7f04f81131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04f7fff700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8096380 0x7f04f81131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.199+0000 7f04f7fff700 1 -- 172.21.15.89:0/4007363282 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f0009ae0 con 0x7f04f8094c20 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.200+0000 7f04f7fff700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8094c20 0x7f04f8112cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f04e400d2f0 tx=0x7f04e400b300).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.200+0000 7f04f4ff9700 1 -- 172.21.15.89:0/4007363282 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4020070 con 0x7f04f8094c20 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.200+0000 7f04f4ff9700 1 -- 172.21.15.89:0/4007363282 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4010e10 con 0x7f04f8094c20 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.200+0000 7f04f4ff9700 1 -- 172.21.15.89:0/4007363282 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4019890 con 0x7f04f8094c20 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.200+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec0092a0 con 0x7f04f8094c20 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.200+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e400b540 con 0x7f04f8094c20 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.200+0000 7f04f4ff9700 1 -- 172.21.15.89:0/4007363282 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f04e4019c80 con 0x7f04f8094c20 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.201+0000 7f04f4ff9700 1 -- 172.21.15.89:0/4007363282 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f04e403e9b0 con 0x7f04f8094c20 2023-09-30T22:14:02.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.201+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f8098850 con 0x7f04f8094c20 2023-09-30T22:14:02.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.205+0000 7f04f4ff9700 1 -- 172.21.15.89:0/4007363282 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f04e402a020 con 0x7f04f8094c20 2023-09-30T22:14:02.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.363+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04f8065070 con 0x7f04f8094c20 2023-09-30T22:14:02.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.364+0000 7f04f4ff9700 1 -- 172.21.15.89:0/4007363282 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11354 (secure 0 0 0) 0x7f04e4017dc0 con 0x7f04f8094c20 2023-09-30T22:14:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:02.365 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":16,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:02.037273+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 >> v1:172.21.15.89:6816/118161 conn(0x7f04dc02bb60 legacy=0x7f04dc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8094c20 msgr2=0x7f04f8112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8094c20 0x7f04f8112cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f04e400d2f0 tx=0x7f04e400b300).stop 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 shutdown_connections 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f8096380 0x7f04f81131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04f8091b00 0x7f04f8115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 --2- 172.21.15.89:0/4007363282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04f8094c20 0x7f04f8112cd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 >> 172.21.15.89:0/4007363282 conn(0x7f04f8097470 msgr2=0x7f04f8098c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 shutdown_connections 2023-09-30T22:14:02.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.366+0000 7f04febee700 1 -- 172.21.15.89:0/4007363282 wait complete. 2023-09-30T22:14:02.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:02.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.493+0000 7fd33323d700 1 Processor -- start 2023-09-30T22:14:02.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.493+0000 7fd33323d700 1 -- start start 2023-09-30T22:14:02.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.493+0000 7fd33323d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c1256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.493+0000 7fd33323d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c125bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.493+0000 7fd33323d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c095f00 con 0x7fd32c093bf0 2023-09-30T22:14:02.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.493+0000 7fd33323d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c059450 con 0x7fd32c1251c0 2023-09-30T22:14:02.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.493+0000 7fd33323d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c096190 con 0x7fd32c059070 2023-09-30T22:14:02.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd32bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c125bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd330fd9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c1256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd32bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c125bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52080/0 (socket says 172.21.15.89:52080) 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd3317da700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd32c093bf0 0x7fd32c094050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51788/0 (socket says 172.21.15.89:51788) 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd330fd9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c1256c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57318/0 (socket says 172.21.15.89:57318) 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd32bfff700 1 -- 172.21.15.89:0/1066475320 learned_addr learned my addr 172.21.15.89:0/1066475320 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd32b7fe700 1 -- 172.21.15.89:0/1066475320 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011404689 0 0) 0x7fd32c095f00 con 0x7fd32c093bf0 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd330fd9700 1 -- 172.21.15.89:0/1066475320 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 msgr2=0x7fd32c125bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd330fd9700 1 --2- 172.21.15.89:0/1066475320 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c125bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd330fd9700 1 -- 172.21.15.89:0/1066475320 >> v1:172.21.15.89:6789/0 conn(0x7fd32c093bf0 legacy=0x7fd32c094050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd330fd9700 1 -- 172.21.15.89:0/1066475320 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c1260a0 con 0x7fd32c059070 2023-09-30T22:14:02.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd32bfff700 1 --2- 172.21.15.89:0/1066475320 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c125bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd330fd9700 1 --2- 172.21.15.89:0/1066475320 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c1256c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd3200097c0 tx=0x7fd320009060).ready entity=mon.2 client_cookie=1444812a69e66c3a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd32b7fe700 1 -- 172.21.15.89:0/1066475320 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32000f510 con 0x7fd32c059070 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd32b7fe700 1 -- 172.21.15.89:0/1066475320 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32000f670 con 0x7fd32c059070 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.494+0000 7fd32b7fe700 1 -- 172.21.15.89:0/1066475320 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32000f9f0 con 0x7fd32c059070 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.495+0000 7fd33323d700 1 -- 172.21.15.89:0/1066475320 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 msgr2=0x7fd32c1256c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.495+0000 7fd33323d700 1 --2- 172.21.15.89:0/1066475320 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c1256c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd3200097c0 tx=0x7fd320009060).stop 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.495+0000 7fd33323d700 1 -- 172.21.15.89:0/1066475320 shutdown_connections 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.495+0000 7fd33323d700 1 --2- 172.21.15.89:0/1066475320 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c1256c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.495+0000 7fd33323d700 1 --2- 172.21.15.89:0/1066475320 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c125bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.495+0000 7fd33323d700 1 -- 172.21.15.89:0/1066475320 >> 172.21.15.89:0/1066475320 conn(0x7fd32c097470 msgr2=0x7fd32c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:02.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.495+0000 7fd33323d700 1 -- 172.21.15.89:0/1066475320 shutdown_connections 2023-09-30T22:14:02.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.495+0000 7fd33323d700 1 -- 172.21.15.89:0/1066475320 wait complete. 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd33323d700 1 Processor -- start 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd33323d700 1 -- start start 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd33323d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd33323d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd32c093bf0 0x7fd32c091a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd33323d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c091f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd33323d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c092eb0 con 0x7fd32c093bf0 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd33323d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c117180 con 0x7fd32c1251c0 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd33323d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c1239e0 con 0x7fd32c059070 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd330fd9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd32bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd32c093bf0 0x7fd32c091a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd3317da700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c091f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd330fd9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c122c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57326/0 (socket says 172.21.15.89:57326) 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.496+0000 7fd330fd9700 1 -- 172.21.15.89:0/484467904 learned_addr learned my addr 172.21.15.89:0/484467904 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd3317da700 1 -- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 msgr2=0x7fd32c122c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd3317da700 1 --2- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c122c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd3317da700 1 -- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd32c093bf0 msgr2=0x7fd32c091a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd3317da700 1 --2- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd32c093bf0 0x7fd32c091a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd3317da700 1 -- 172.21.15.89:0/484467904 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd318009370 con 0x7fd32c1251c0 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd330fd9700 1 --2- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd3317da700 1 --2- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c091f00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd324000f80 tx=0x7fd324007e90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd328ff9700 1 -- 172.21.15.89:0/484467904 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324005c70 con 0x7fd32c1251c0 2023-09-30T22:14:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd328ff9700 1 -- 172.21.15.89:0/484467904 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324005df0 con 0x7fd32c1251c0 2023-09-30T22:14:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3200092a0 con 0x7fd32c1251c0 2023-09-30T22:14:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd328ff9700 1 -- 172.21.15.89:0/484467904 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324007320 con 0x7fd32c1251c0 2023-09-30T22:14:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.497+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c117e40 con 0x7fd32c1251c0 2023-09-30T22:14:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.498+0000 7fd328ff9700 1 -- 172.21.15.89:0/484467904 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd324007710 con 0x7fd32c1251c0 2023-09-30T22:14:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.498+0000 7fd328ff9700 1 -- 172.21.15.89:0/484467904 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd32403a610 con 0x7fd32c1251c0 2023-09-30T22:14:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.498+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c117e40 con 0x7fd32c1251c0 2023-09-30T22:14:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.503+0000 7fd328ff9700 1 -- 172.21.15.89:0/484467904 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fd32c117e40 con 0x7fd32c1251c0 2023-09-30T22:14:02.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.659+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd32c117e40 con 0x7fd32c1251c0 2023-09-30T22:14:02.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.660+0000 7fd328ff9700 1 -- 172.21.15.89:0/484467904 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11354 (secure 0 0 0) 0x7fd32c117e40 con 0x7fd32c1251c0 2023-09-30T22:14:02.661 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:02.661 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":16,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:02.037273+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 >> v1:172.21.15.89:6816/118161 conn(0x7fd31002bb60 legacy=0x7fd31002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 msgr2=0x7fd32c091f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 --2- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c091f00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd324000f80 tx=0x7fd324007e90).stop 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 shutdown_connections 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 --2- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd32c1251c0 0x7fd32c091f00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 --2- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32c059070 0x7fd32c122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 --2- 172.21.15.89:0/484467904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd32c093bf0 0x7fd32c091a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 >> 172.21.15.89:0/484467904 conn(0x7fd32c097470 msgr2=0x7fd32c128d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 shutdown_connections 2023-09-30T22:14:02.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.662+0000 7fd33323d700 1 -- 172.21.15.89:0/484467904 wait complete. 2023-09-30T22:14:02.675 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-30T22:14:02.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd331f700 1 Processor -- start 2023-09-30T22:14:02.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd331f700 1 -- start start 2023-09-30T22:14:02.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd331f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc098b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd331f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc099010 0x7fefcc068d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd331f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefcc069210 0x7fefcc0696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd331f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc069bb0 con 0x7fefcc069210 2023-09-30T22:14:02.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd331f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc095270 con 0x7fefcc098740 2023-09-30T22:14:02.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd331f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc094d20 con 0x7fefcc099010 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd10bb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd18bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefcc069210 0x7fefcc0696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd08ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc099010 0x7fefcc068d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd10bb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc098b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52100/0 (socket says 172.21.15.89:52100) 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd18bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefcc069210 0x7fefcc0696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38724/0 (socket says 172.21.15.89:38724) 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd10bb700 1 -- 172.21.15.89:0/2657245521 learned_addr learned my addr 172.21.15.89:0/2657245521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd08ba700 1 -- 172.21.15.89:0/2657245521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 msgr2=0x7fefcc098b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd08ba700 1 --2- 172.21.15.89:0/2657245521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc098b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd08ba700 1 -- 172.21.15.89:0/2657245521 >> v2:172.21.15.89:3300/0 conn(0x7fefcc069210 msgr2=0x7fefcc0696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd08ba700 1 --2- 172.21.15.89:0/2657245521 >> v2:172.21.15.89:3300/0 conn(0x7fefcc069210 0x7fefcc0696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd08ba700 1 -- 172.21.15.89:0/2657245521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc096520 con 0x7fefcc099010 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd10bb700 1 --2- 172.21.15.89:0/2657245521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc098b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd18bc700 1 --2- 172.21.15.89:0/2657245521 >> v2:172.21.15.89:3300/0 conn(0x7fefcc069210 0x7fefcc0696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefd08ba700 1 --2- 172.21.15.89:0/2657245521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc099010 0x7fefcc068d20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fefbc009790 tx=0x7fefbc009060).ready entity=mon.2 client_cookie=80c2d5e76bc95964 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefc3fff700 1 -- 172.21.15.89:0/2657245521 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0090d0 con 0x7fefcc099010 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefc3fff700 1 -- 172.21.15.89:0/2657245521 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc00d4a0 con 0x7fefcc099010 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.794+0000 7fefc3fff700 1 -- 172.21.15.89:0/2657245521 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00d820 con 0x7fefcc099010 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 -- 172.21.15.89:0/2657245521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc099010 msgr2=0x7fefcc068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 --2- 172.21.15.89:0/2657245521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc099010 0x7fefcc068d20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fefbc009790 tx=0x7fefbc009060).stop 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 -- 172.21.15.89:0/2657245521 shutdown_connections 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 --2- 172.21.15.89:0/2657245521 >> v2:172.21.15.89:3300/0 conn(0x7fefcc069210 0x7fefcc0696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 --2- 172.21.15.89:0/2657245521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc098b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 --2- 172.21.15.89:0/2657245521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc099010 0x7fefcc068d20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 -- 172.21.15.89:0/2657245521 >> 172.21.15.89:0/2657245521 conn(0x7fefcc09a1c0 msgr2=0x7fefcc09a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 -- 172.21.15.89:0/2657245521 shutdown_connections 2023-09-30T22:14:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 -- 172.21.15.89:0/2657245521 wait complete. 2023-09-30T22:14:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.795+0000 7fefd331f700 1 Processor -- start 2023-09-30T22:14:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd331f700 1 -- start start 2023-09-30T22:14:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd331f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc069210 0x7fefcc092030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd331f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc092520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd331f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefcc099010 0x7fefcc092a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd331f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc091e10 con 0x7fefcc099010 2023-09-30T22:14:02.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd331f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0981e0 con 0x7fefcc098740 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd331f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc094f70 con 0x7fefcc069210 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd08ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc092520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd18bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefcc099010 0x7fefcc092a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd08ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc092520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52112/0 (socket says 172.21.15.89:52112) 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd10bb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc069210 0x7fefcc092030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.796+0000 7fefd08ba700 1 -- 172.21.15.89:0/3821872288 learned_addr learned my addr 172.21.15.89:0/3821872288 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd08ba700 1 -- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc069210 msgr2=0x7fefcc092030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd08ba700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc069210 0x7fefcc092030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd08ba700 1 -- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefcc099010 msgr2=0x7fefcc092a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd08ba700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefcc099010 0x7fefcc092a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:02.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd08ba700 1 -- 172.21.15.89:0/3821872288 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4009a20 con 0x7fefcc098740 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd10bb700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc069210 0x7fefcc092030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd18bc700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefcc099010 0x7fefcc092a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd08ba700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc092520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fefcc094eb0 tx=0x7fefbc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefc17fa700 1 -- 172.21.15.89:0/3821872288 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0128a0 con 0x7fefcc098740 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefc17fa700 1 -- 172.21.15.89:0/3821872288 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc012a00 con 0x7fefcc098740 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefc17fa700 1 -- 172.21.15.89:0/3821872288 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc012dc0 con 0x7fefcc098740 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc8009370 con 0x7fefcc098740 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.797+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc0092a0 con 0x7fefcc098740 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.798+0000 7fefc17fa700 1 -- 172.21.15.89:0/3821872288 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fefbc01c630 con 0x7fefcc098740 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.798+0000 7fefc17fa700 1 -- 172.21.15.89:0/3821872288 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fefbc03c470 con 0x7fefcc098740 2023-09-30T22:14:02.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.798+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc05a260 con 0x7fefcc098740 2023-09-30T22:14:02.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.803+0000 7fefc17fa700 1 -- 172.21.15.89:0/3821872288 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fefbc01c8e0 con 0x7fefcc098740 2023-09-30T22:14:02.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:02.996+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fefcc093e00 con 0x7fefcc098740 2023-09-30T22:14:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.042+0000 7fefc17fa700 1 -- 172.21.15.89:0/3821872288 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v4) v1 ==== 118+0+0 (secure 0 0 0) 0x7fefbc024360 con 0x7fefcc098740 2023-09-30T22:14:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.045+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 >> v1:172.21.15.89:6816/118161 conn(0x7fefb002bb60 legacy=0x7fefb002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.045+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 msgr2=0x7fefcc092520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.045+0000 7fefd331f700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc092520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fefcc094eb0 tx=0x7fefbc016000).stop 2023-09-30T22:14:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.046+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 shutdown_connections 2023-09-30T22:14:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.046+0000 7fefd331f700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefcc099010 0x7fefcc092a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.046+0000 7fefd331f700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefcc069210 0x7fefcc092030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.046+0000 7fefd331f700 1 --2- 172.21.15.89:0/3821872288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefcc098740 0x7fefcc092520 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.046+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 >> 172.21.15.89:0/3821872288 conn(0x7fefcc09a1c0 msgr2=0x7fefcc065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.046+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 shutdown_connections 2023-09-30T22:14:03.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.048+0000 7fefd331f700 1 -- 172.21.15.89:0/3821872288 wait complete. 2023-09-30T22:14:03.060 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-30T22:14:03.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-30T22:14:03.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242c3a1700 1 Processor -- start 2023-09-30T22:14:03.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242c3a1700 1 -- start start 2023-09-30T22:14:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242c3a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2424093840 0x7f2424098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242c3a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f242412f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242c3a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2424092d80 con 0x7f2424098c80 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242c3a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2424093c20 con 0x7f2424093840 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242c3a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2424092b90 con 0x7f242412d160 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242a13d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2424093840 0x7f2424098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242993c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2424098c80 0x7f2424099090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51794/0 (socket says 172.21.15.89:51794) 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242993c700 1 -- 172.21.15.89:0/312341288 learned_addr learned my addr 172.21.15.89:0/312341288 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242a93e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f242412f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242a13d700 1 -- 172.21.15.89:0/312341288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 msgr2=0x7f242412f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242a13d700 1 --2- 172.21.15.89:0/312341288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f242412f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:03.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242a13d700 1 -- 172.21.15.89:0/312341288 >> v1:172.21.15.89:6789/0 conn(0x7f2424098c80 legacy=0x7f2424099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:03.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242a13d700 1 -- 172.21.15.89:0/312341288 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2424099b70 con 0x7f2424093840 2023-09-30T22:14:03.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242913b700 1 -- 172.21.15.89:0/312341288 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563975792 0 0) 0x7f2424092d80 con 0x7f2424098c80 2023-09-30T22:14:03.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242a93e700 1 --2- 172.21.15.89:0/312341288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f242412f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:03.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.180+0000 7f242a13d700 1 --2- 172.21.15.89:0/312341288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424098790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f241400ba00 tx=0x7f241400b300).ready entity=mon.0 client_cookie=12df8866d76a77c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:03.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242913b700 1 -- 172.21.15.89:0/312341288 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400da70 con 0x7f2424093840 2023-09-30T22:14:03.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242913b700 1 -- 172.21.15.89:0/312341288 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400dbd0 con 0x7f2424093840 2023-09-30T22:14:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242913b700 1 -- 172.21.15.89:0/312341288 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414010470 con 0x7f2424093840 2023-09-30T22:14:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242c3a1700 1 -- 172.21.15.89:0/312341288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 msgr2=0x7f2424098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242c3a1700 1 --2- 172.21.15.89:0/312341288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424098790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f241400ba00 tx=0x7f241400b300).stop 2023-09-30T22:14:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242c3a1700 1 -- 172.21.15.89:0/312341288 shutdown_connections 2023-09-30T22:14:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242c3a1700 1 --2- 172.21.15.89:0/312341288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f242412f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242c3a1700 1 --2- 172.21.15.89:0/312341288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424098790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242c3a1700 1 -- 172.21.15.89:0/312341288 >> 172.21.15.89:0/312341288 conn(0x7f2424097480 msgr2=0x7f2424097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242c3a1700 1 -- 172.21.15.89:0/312341288 shutdown_connections 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.181+0000 7f242c3a1700 1 -- 172.21.15.89:0/312341288 wait complete. 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242c3a1700 1 Processor -- start 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242c3a1700 1 -- start start 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242c3a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424119ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242c3a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2424098c80 0x7f2424115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242c3a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f24241160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242c3a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242412cd30 con 0x7f2424093840 2023-09-30T22:14:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242c3a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2424118230 con 0x7f242412d160 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242c3a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f242406a550 con 0x7f2424098c80 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242a13d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424119ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242a93e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f24241160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242a13d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424119ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38748/0 (socket says 172.21.15.89:38748) 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.182+0000 7f242a13d700 1 -- 172.21.15.89:0/2716168277 learned_addr learned my addr 172.21.15.89:0/2716168277 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242993c700 1 --2- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2424098c80 0x7f2424115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242a13d700 1 -- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2424098c80 msgr2=0x7f2424115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242a13d700 1 --2- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2424098c80 0x7f2424115c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242a13d700 1 -- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 msgr2=0x7f24241160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242a13d700 1 --2- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f24241160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242a13d700 1 -- 172.21.15.89:0/2716168277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c009a20 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242a13d700 1 --2- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424119ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f241400d5d0 tx=0x7f241400b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f241a7fc700 1 -- 172.21.15.89:0/2716168277 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414020070 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f241a7fc700 1 -- 172.21.15.89:0/2716168277 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2414019450 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241400b540 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f241a7fc700 1 -- 172.21.15.89:0/2716168277 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140197f0 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.183+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2424117000 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.184+0000 7f241a7fc700 1 -- 172.21.15.89:0/2716168277 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2414019be0 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.184+0000 7f241a7fc700 1 -- 172.21.15.89:0/2716168277 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f241403e940 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.184+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242405a260 con 0x7f2424093840 2023-09-30T22:14:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.189+0000 7f241a7fc700 1 -- 172.21.15.89:0/2716168277 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2414017da0 con 0x7f2424093840 2023-09-30T22:14:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:03.385+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2424116a00 con 0x7f2424093840 2023-09-30T22:14:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.047+0000 7f241a7fc700 1 -- 172.21.15.89:0/2716168277 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v5) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2414017360 con 0x7f2424093840 2023-09-30T22:14:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 >> v1:172.21.15.89:6816/118161 conn(0x7f240802bb90 legacy=0x7f240802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 msgr2=0x7f2424119ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 --2- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424119ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f241400d5d0 tx=0x7f241400b300).stop 2023-09-30T22:14:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 shutdown_connections 2023-09-30T22:14:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 --2- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f242412d160 0x7f24241160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 --2- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2424093840 0x7f2424119ba0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 --2- 172.21.15.89:0/2716168277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2424098c80 0x7f2424115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 >> 172.21.15.89:0/2716168277 conn(0x7f2424097480 msgr2=0x7f24240688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 shutdown_connections 2023-09-30T22:14:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.049+0000 7f242c3a1700 1 -- 172.21.15.89:0/2716168277 wait complete. 2023-09-30T22:14:04.063 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-30T22:14:04.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:14:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.184+0000 7f7d1bdf2700 1 Processor -- start 2023-09-30T22:14:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.184+0000 7f7d1bdf2700 1 -- start start 2023-09-30T22:14:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1bdf2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140965e0 0x7f7d140969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1bdf2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d14096eb0 0x7f7d14067de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1bdf2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140682d0 0x7f7d14068740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1bdf2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d14068c30 con 0x7f7d14096eb0 2023-09-30T22:14:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1bdf2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140933b0 con 0x7f7d140965e0 2023-09-30T22:14:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1bdf2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14092190 con 0x7f7d140682d0 2023-09-30T22:14:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140965e0 0x7f7d140969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1938d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d14096eb0 0x7f7d14067de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1a38f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140682d0 0x7f7d14068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140965e0 0x7f7d140969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52136/0 (socket says 172.21.15.89:52136) 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 -- 172.21.15.89:0/2362213137 learned_addr learned my addr 172.21.15.89:0/2362213137 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 -- 172.21.15.89:0/2362213137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140682d0 msgr2=0x7f7d14068740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 --2- 172.21.15.89:0/2362213137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140682d0 0x7f7d14068740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 -- 172.21.15.89:0/2362213137 >> v2:172.21.15.89:3300/0 conn(0x7f7d14096eb0 msgr2=0x7f7d14067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 --2- 172.21.15.89:0/2362213137 >> v2:172.21.15.89:3300/0 conn(0x7f7d14096eb0 0x7f7d14067de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 -- 172.21.15.89:0/2362213137 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14069a20 con 0x7f7d140965e0 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1a38f700 1 --2- 172.21.15.89:0/2362213137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140682d0 0x7f7d14068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d1938d700 1 --2- 172.21.15.89:0/2362213137 >> v2:172.21.15.89:3300/0 conn(0x7f7d14096eb0 0x7f7d14067de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d19b8e700 1 --2- 172.21.15.89:0/2362213137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140965e0 0x7f7d140969c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7d04006fd0 tx=0x7f7d040090f0).ready entity=mon.1 client_cookie=484422f34195c3f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d18b8c700 1 -- 172.21.15.89:0/2362213137 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400f510 con 0x7f7d140965e0 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d18b8c700 1 -- 172.21.15.89:0/2362213137 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0400f670 con 0x7f7d140965e0 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.185+0000 7f7d18b8c700 1 -- 172.21.15.89:0/2362213137 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400f9f0 con 0x7f7d140965e0 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2362213137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140965e0 msgr2=0x7f7d140969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 --2- 172.21.15.89:0/2362213137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140965e0 0x7f7d140969c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7d04006fd0 tx=0x7f7d040090f0).stop 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2362213137 shutdown_connections 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 --2- 172.21.15.89:0/2362213137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140682d0 0x7f7d14068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 --2- 172.21.15.89:0/2362213137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140965e0 0x7f7d140969c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 --2- 172.21.15.89:0/2362213137 >> v2:172.21.15.89:3300/0 conn(0x7f7d14096eb0 0x7f7d14067de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2362213137 >> 172.21.15.89:0/2362213137 conn(0x7f7d1409a220 msgr2=0x7f7d1409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2362213137 shutdown_connections 2023-09-30T22:14:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.186+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2362213137 wait complete. 2023-09-30T22:14:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1bdf2700 1 Processor -- start 2023-09-30T22:14:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1bdf2700 1 -- start start 2023-09-30T22:14:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1bdf2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140682d0 0x7f7d141177d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1bdf2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140965e0 0x7f7d14117cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1bdf2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14096eb0 0x7f7d141181b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1bdf2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14067b10 con 0x7f7d14096eb0 2023-09-30T22:14:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1bdf2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14091bf0 con 0x7f7d140682d0 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1bdf2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1411df40 con 0x7f7d140965e0 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d19b8e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140682d0 0x7f7d141177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1a38f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14096eb0 0x7f7d141181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d19b8e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140682d0 0x7f7d141177d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52152/0 (socket says 172.21.15.89:52152) 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1a38f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14096eb0 0x7f7d141181b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38774/0 (socket says 172.21.15.89:38774) 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d19b8e700 1 -- 172.21.15.89:0/2597186488 learned_addr learned my addr 172.21.15.89:0/2597186488 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.187+0000 7f7d1938d700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140965e0 0x7f7d14117cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d1a38f700 1 -- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140965e0 msgr2=0x7f7d14117cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d1a38f700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140965e0 0x7f7d14117cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d1a38f700 1 -- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140682d0 msgr2=0x7f7d141177d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d1a38f700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140682d0 0x7f7d141177d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d1a38f700 1 -- 172.21.15.89:0/2597186488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c009a20 con 0x7f7d14096eb0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d19b8e700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140682d0 0x7f7d141177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d1a38f700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14096eb0 0x7f7d141181b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f7d0c009e70 tx=0x7f7d0c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d09ffb700 1 -- 172.21.15.89:0/2597186488 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c009bd0 con 0x7f7d14096eb0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d09ffb700 1 -- 172.21.15.89:0/2597186488 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c00f040 con 0x7f7d14096eb0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d100092a0 con 0x7f7d14096eb0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d09ffb700 1 -- 172.21.15.89:0/2597186488 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00ccb0 con 0x7f7d14096eb0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.188+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d04009370 con 0x7f7d14096eb0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.189+0000 7f7d09ffb700 1 -- 172.21.15.89:0/2597186488 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7d0c013610 con 0x7f7d14096eb0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.189+0000 7f7d09ffb700 1 -- 172.21.15.89:0/2597186488 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7d0c03b6a0 con 0x7f7d14096eb0 2023-09-30T22:14:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.189+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d14067910 con 0x7f7d14096eb0 2023-09-30T22:14:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.194+0000 7f7d09ffb700 1 -- 172.21.15.89:0/2597186488 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7d0c01a900 con 0x7f7d14096eb0 2023-09-30T22:14:04.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.385+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d1405a260 con 0x7f7d14096eb0 2023-09-30T22:14:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.385+0000 7f7d09ffb700 1 -- 172.21.15.89:0/2597186488 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f7d0c02b090 con 0x7f7d14096eb0 2023-09-30T22:14:04.385 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:04.385 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:04.044604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 >> v1:172.21.15.89:6816/118161 conn(0x7f7cf802bb90 legacy=0x7f7cf802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14096eb0 msgr2=0x7f7d141181b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14096eb0 0x7f7d141181b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f7d0c009e70 tx=0x7f7d0c00b040).stop 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 shutdown_connections 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14096eb0 0x7f7d141181b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d140682d0 0x7f7d141177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 --2- 172.21.15.89:0/2597186488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d140965e0 0x7f7d14117cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 >> 172.21.15.89:0/2597186488 conn(0x7f7d1409a220 msgr2=0x7f7d14094f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 shutdown_connections 2023-09-30T22:14:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.387+0000 7f7d1bdf2700 1 -- 172.21.15.89:0/2597186488 wait complete. 2023-09-30T22:14:04.388 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 5 2023-09-30T22:14:04.398 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.512+0000 7f1f39a2a700 1 Processor -- start 2023-09-30T22:14:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.512+0000 7f1f39a2a700 1 -- start start 2023-09-30T22:14:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f39a2a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f340650f0 0x7f1f340986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f39a2a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f34097c30 0x7f1f34098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f39a2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f34099080 0x7f1f34131470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f39a2a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f340654d0 con 0x7f1f340650f0 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f39a2a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34099530 con 0x7f1f34099080 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f39a2a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34096160 con 0x7f1f34097c30 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f34097c30 0x7f1f34098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f32ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f340650f0 0x7f1f340986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f337fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f34099080 0x7f1f34131470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f34097c30 0x7f1f34098b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57382/0 (socket says 172.21.15.89:57382) 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f32ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f340650f0 0x7f1f340986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38788/0 (socket says 172.21.15.89:38788) 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 -- 172.21.15.89:0/3045305330 learned_addr learned my addr 172.21.15.89:0/3045305330 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 -- 172.21.15.89:0/3045305330 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f34099080 msgr2=0x7f1f34131470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 --2- 172.21.15.89:0/3045305330 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f34099080 0x7f1f34131470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 -- 172.21.15.89:0/3045305330 >> v2:172.21.15.89:3300/0 conn(0x7f1f340650f0 msgr2=0x7f1f340986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 --2- 172.21.15.89:0/3045305330 >> v2:172.21.15.89:3300/0 conn(0x7f1f340650f0 0x7f1f340986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 -- 172.21.15.89:0/3045305330 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34099790 con 0x7f1f34097c30 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f337fe700 1 --2- 172.21.15.89:0/3045305330 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f34099080 0x7f1f34131470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.513+0000 7f1f327fc700 1 --2- 172.21.15.89:0/3045305330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f34097c30 0x7f1f34098b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1f24009830 tx=0x7f1f240090f0).ready entity=mon.2 client_cookie=9fc4f78f79a06d68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f31ffb700 1 -- 172.21.15.89:0/3045305330 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400f510 con 0x7f1f34097c30 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f31ffb700 1 -- 172.21.15.89:0/3045305330 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2400f670 con 0x7f1f34097c30 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f31ffb700 1 -- 172.21.15.89:0/3045305330 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400f9f0 con 0x7f1f34097c30 2023-09-30T22:14:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 -- 172.21.15.89:0/3045305330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f34097c30 msgr2=0x7f1f34098b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 --2- 172.21.15.89:0/3045305330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f34097c30 0x7f1f34098b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1f24009830 tx=0x7f1f240090f0).stop 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 -- 172.21.15.89:0/3045305330 shutdown_connections 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 --2- 172.21.15.89:0/3045305330 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f34099080 0x7f1f34131470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 --2- 172.21.15.89:0/3045305330 >> v2:172.21.15.89:3300/0 conn(0x7f1f340650f0 0x7f1f340986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 --2- 172.21.15.89:0/3045305330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f34097c30 0x7f1f34098b90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 -- 172.21.15.89:0/3045305330 >> 172.21.15.89:0/3045305330 conn(0x7f1f3409dc70 msgr2=0x7f1f3409e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 -- 172.21.15.89:0/3045305330 shutdown_connections 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.514+0000 7f1f39a2a700 1 -- 172.21.15.89:0/3045305330 wait complete. 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.515+0000 7f1f39a2a700 1 Processor -- start 2023-09-30T22:14:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.515+0000 7f1f39a2a700 1 -- start start 2023-09-30T22:14:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.515+0000 7f1f39a2a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f340650f0 0x7f1f3413a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.515+0000 7f1f39a2a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f34099080 0x7f1f3413adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.515+0000 7f1f39a2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f3413b2b0 0x7f1f3411a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.515+0000 7f1f39a2a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3412f690 con 0x7f1f34099080 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f39a2a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3411b8a0 con 0x7f1f3413b2b0 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f39a2a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34099a80 con 0x7f1f340650f0 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f337fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f3413b2b0 0x7f1f3411a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f327fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f34099080 0x7f1f3413adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f337fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f3413b2b0 0x7f1f3411a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52178/0 (socket says 172.21.15.89:52178) 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f32ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f340650f0 0x7f1f3413a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f337fe700 1 -- 172.21.15.89:0/1890246725 learned_addr learned my addr 172.21.15.89:0/1890246725 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f327fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f34099080 0x7f1f3413adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38802/0 (socket says 172.21.15.89:38802) 2023-09-30T22:14:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f327fc700 1 -- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f340650f0 msgr2=0x7f1f3413a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f327fc700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f340650f0 0x7f1f3413a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f327fc700 1 -- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f3413b2b0 msgr2=0x7f1f3411a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f327fc700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f3413b2b0 0x7f1f3411a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f327fc700 1 -- 172.21.15.89:0/1890246725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c009a20 con 0x7f1f34099080 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f337fe700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f3413b2b0 0x7f1f3411a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f32ffd700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f340650f0 0x7f1f3413a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f327fc700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f34099080 0x7f1f3413adc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f1f34099ea0 tx=0x7f1f24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f2affd700 1 -- 172.21.15.89:0/1890246725 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400fe30 con 0x7f1f34099080 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f2affd700 1 -- 172.21.15.89:0/1890246725 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2400f7e0 con 0x7f1f34099080 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f2affd700 1 -- 172.21.15.89:0/1890246725 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2401c670 con 0x7f1f34099080 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f24009370 con 0x7f1f34099080 2023-09-30T22:14:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.516+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3409cf20 con 0x7f1f34099080 2023-09-30T22:14:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.517+0000 7f1f2affd700 1 -- 172.21.15.89:0/1890246725 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1f2401ca20 con 0x7f1f34099080 2023-09-30T22:14:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.517+0000 7f1f2affd700 1 -- 172.21.15.89:0/1890246725 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f1f2403c4a0 con 0x7f1f34099080 2023-09-30T22:14:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.517+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3405a260 con 0x7f1f34099080 2023-09-30T22:14:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.521+0000 7f1f2affd700 1 -- 172.21.15.89:0/1890246725 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1f24020070 con 0x7f1f34099080 2023-09-30T22:14:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.680+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f340973a0 con 0x7f1f34099080 2023-09-30T22:14:04.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.680+0000 7f1f2affd700 1 -- 172.21.15.89:0/1890246725 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f1f24020030 con 0x7f1f34099080 2023-09-30T22:14:04.680 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:04.680 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:03.041750+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 >> v1:172.21.15.89:6816/118161 conn(0x7f1f1402bb60 legacy=0x7f1f1402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f34099080 msgr2=0x7f1f3413adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f34099080 0x7f1f3413adc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f1f34099ea0 tx=0x7f1f24015000).stop 2023-09-30T22:14:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 shutdown_connections 2023-09-30T22:14:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f3413b2b0 0x7f1f3411a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f340650f0 0x7f1f3413a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 --2- 172.21.15.89:0/1890246725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f34099080 0x7f1f3413adc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 >> 172.21.15.89:0/1890246725 conn(0x7f1f3409dc70 msgr2=0x7f1f341142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 shutdown_connections 2023-09-30T22:14:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.682+0000 7f1f39a2a700 1 -- 172.21.15.89:0/1890246725 wait complete. 2023-09-30T22:14:04.694 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-30T22:14:04.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:14:04.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f69819c4700 1 Processor -- start 2023-09-30T22:14:04.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f69819c4700 1 -- start start 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f69819c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c0986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f69819c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c097c30 0x7f697c098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f69819c4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c096160 con 0x7f697c099080 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f69819c4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0654d0 con 0x7f697c0650f0 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f69819c4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f697c099510 con 0x7f697c097c30 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f697a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c097c30 0x7f697c098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f697affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c0986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f697b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f697c099080 0x7f697c131470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51804/0 (socket says 172.21.15.89:51804) 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.811+0000 7f697b7fe700 1 -- 172.21.15.89:0/334833866 learned_addr learned my addr 172.21.15.89:0/334833866 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f6979ffb700 1 -- 172.21.15.89:0/334833866 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786111551 0 0) 0x7f697c096160 con 0x7f697c099080 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f697affd700 1 -- 172.21.15.89:0/334833866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c097c30 msgr2=0x7f697c098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f697affd700 1 --2- 172.21.15.89:0/334833866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c097c30 0x7f697c098b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f697affd700 1 -- 172.21.15.89:0/334833866 >> v1:172.21.15.89:6789/0 conn(0x7f697c099080 legacy=0x7f697c131470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f697affd700 1 -- 172.21.15.89:0/334833866 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c099790 con 0x7f697c0650f0 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f697a7fc700 1 --2- 172.21.15.89:0/334833866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c097c30 0x7f697c098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f697affd700 1 --2- 172.21.15.89:0/334833866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c0986a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f696c0097c0 tx=0x7f696c009060).ready entity=mon.1 client_cookie=f29841c389add7fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f6979ffb700 1 -- 172.21.15.89:0/334833866 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00f510 con 0x7f697c0650f0 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f6979ffb700 1 -- 172.21.15.89:0/334833866 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c00f670 con 0x7f697c0650f0 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f6979ffb700 1 -- 172.21.15.89:0/334833866 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00f9f0 con 0x7f697c0650f0 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f69819c4700 1 -- 172.21.15.89:0/334833866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 msgr2=0x7f697c0986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.812+0000 7f69819c4700 1 --2- 172.21.15.89:0/334833866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c0986a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f696c0097c0 tx=0x7f696c009060).stop 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.813+0000 7f69819c4700 1 -- 172.21.15.89:0/334833866 shutdown_connections 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.813+0000 7f69819c4700 1 --2- 172.21.15.89:0/334833866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c0986a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.813+0000 7f69819c4700 1 --2- 172.21.15.89:0/334833866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c097c30 0x7f697c098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.813+0000 7f69819c4700 1 -- 172.21.15.89:0/334833866 >> 172.21.15.89:0/334833866 conn(0x7f697c09dc70 msgr2=0x7f697c09e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.813+0000 7f69819c4700 1 -- 172.21.15.89:0/334833866 shutdown_connections 2023-09-30T22:14:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.813+0000 7f69819c4700 1 -- 172.21.15.89:0/334833866 wait complete. 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.813+0000 7f69819c4700 1 Processor -- start 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f69819c4700 1 -- start start 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f69819c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c11af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f69819c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c099080 0x7f697c11b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f69819c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f697c11b940 0x7f697c13a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f69819c4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c11ad70 con 0x7f697c11b940 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f69819c4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c11bda0 con 0x7f697c0650f0 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f69819c4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f697c09cce0 con 0x7f697c099080 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c099080 0x7f697c11b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f697c11b940 0x7f697c13a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c11af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c099080 0x7f697c11b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57412/0 (socket says 172.21.15.89:57412) 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697a7fc700 1 -- 172.21.15.89:0/1336301918 learned_addr learned my addr 172.21.15.89:0/1336301918 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697a7fc700 1 -- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 msgr2=0x7f697c11af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697a7fc700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c11af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697a7fc700 1 -- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f697c11b940 msgr2=0x7f697c13a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697a7fc700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f697c11b940 0x7f697c13a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697a7fc700 1 -- 172.21.15.89:0/1336301918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700092a0 con 0x7f697c099080 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697b7fe700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f697c11b940 0x7f697c13a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.814+0000 7f697affd700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c11af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.815+0000 7f697a7fc700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c099080 0x7f697c11b450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f697000e990 tx=0x7f6970009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.815+0000 7f6962ffd700 1 -- 172.21.15.89:0/1336301918 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697000c670 con 0x7f697c099080 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.815+0000 7f6962ffd700 1 -- 172.21.15.89:0/1336301918 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697000c7d0 con 0x7f697c099080 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.815+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c0092a0 con 0x7f697c099080 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.815+0000 7f6962ffd700 1 -- 172.21.15.89:0/1336301918 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697000cb50 con 0x7f697c099080 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.815+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c11ea10 con 0x7f697c099080 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.815+0000 7f6962ffd700 1 -- 172.21.15.89:0/1336301918 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f697001b400 con 0x7f697c099080 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.815+0000 7f6962ffd700 1 -- 172.21.15.89:0/1336301918 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f697003b450 con 0x7f697c099080 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.816+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c11e2b0 con 0x7f697c099080 2023-09-30T22:14:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.820+0000 7f6962ffd700 1 -- 172.21.15.89:0/1336301918 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6970016070 con 0x7f697c099080 2023-09-30T22:14:04.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.973+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 --> v1:172.21.15.89:6816/118161 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f697c05a260 con 0x7f696802bb60 2023-09-30T22:14:04.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.979+0000 7f6962ffd700 1 -- 172.21.15.89:0/1336301918 <== mgr.4102 v1:172.21.15.89:6816/118161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242457 (unknown 2965378022 0 1817777937) 0x7f697c05a260 con 0x7f696802bb60 2023-09-30T22:14:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:04.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 >> v1:172.21.15.89:6816/118161 conn(0x7f696802bb60 legacy=0x7f696802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c099080 msgr2=0x7f697c11b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c099080 0x7f697c11b450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f697000e990 tx=0x7f6970009060).stop 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 shutdown_connections 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f697c11b940 0x7f697c13a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f697c0650f0 0x7f697c11af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 --2- 172.21.15.89:0/1336301918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f697c099080 0x7f697c11b450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 >> 172.21.15.89:0/1336301918 conn(0x7f697c09dc70 msgr2=0x7f697c1142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 shutdown_connections 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:04.982+0000 7f69819c4700 1 -- 172.21.15.89:0/1336301918 wait complete. 2023-09-30T22:14:04.984 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:14:04.993 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-09-30T22:14:04.745502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188832,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298528,"statfs":{"total":386547056640,"available":385329692672,"internally_reserved":0,"allocated":458752,"data_stored":120816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.551315+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:13:57.551315+0000","last_peered":"2023-09-30T22:13:57.551315+0000","last_clean":"2023-09-30T22:13:57.551315+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:13:57.551315+0000","last_undegraded":"2023-09-30T22:13:57.551315+0000","last_fullsized":"2023-09-30T22:13:57.551315+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553773+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:13:57.553773+0000","last_peered":"2023-09-30T22:13:57.553773+0000","last_clean":"2023-09-30T22:13:57.553773+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:13:57.553773+0000","last_undegraded":"2023-09-30T22:13:57.553773+0000","last_fullsized":"2023-09-30T22:13:57.553773+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553720+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:13:57.553720+0000","last_peered":"2023-09-30T22:13:57.553720+0000","last_clean":"2023-09-30T22:13:57.553720+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:13:57.553720+0000","last_undegraded":"2023-09-30T22:13:57.553720+0000","last_fullsized":"2023-09-30T22:13:57.553720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553739+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:13:57.553739+0000","last_peered":"2023-09-30T22:13:57.553739+0000","last_clean":"2023-09-30T22:13:57.553739+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:13:57.553739+0000","last_undegraded":"2023-09-30T22:13:57.553739+0000","last_fullsized":"2023-09-30T22:13:57.553739+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.541758+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:13:57.541758+0000","last_peered":"2023-09-30T22:13:57.541758+0000","last_clean":"2023-09-30T22:13:57.541758+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:13:57.541758+0000","last_undegraded":"2023-09-30T22:13:57.541758+0000","last_fullsized":"2023-09-30T22:13:57.541758+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553778+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:13:57.553778+0000","last_peered":"2023-09-30T22:13:57.553778+0000","last_clean":"2023-09-30T22:13:57.553778+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:13:57.553778+0000","last_undegraded":"2023-09-30T22:13:57.553778+0000","last_fullsized":"2023-09-30T22:13:57.553778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.555769+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:13:57.555769+0000","last_peered":"2023-09-30T22:13:57.555769+0000","last_clean":"2023-09-30T22:13:57.555769+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:13:57.555769+0000","last_undegraded":"2023-09-30T22:13:57.555769+0000","last_fullsized":"2023-09-30T22:13:57.555769+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.551277+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:13:57.551277+0000","last_peered":"2023-09-30T22:13:57.551277+0000","last_clean":"2023-09-30T22:13:57.551277+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:13:57.551277+0000","last_undegraded":"2023-09-30T22:13:57.551277+0000","last_fullsized":"2023-09-30T22:13:57.551277+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.555720+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:13:57.555720+0000","last_peered":"2023-09-30T22:13:57.555720+0000","last_clean":"2023-09-30T22:13:57.555720+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:13:57.555720+0000","last_undegraded":"2023-09-30T22:13:57.555720+0000","last_fullsized":"2023-09-30T22:13:57.555720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]}]},{"osd":0,"up_from":9,"seq":38654705666,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:14:05.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:14:05.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.149+0000 7fe344617700 1 Processor -- start 2023-09-30T22:14:05.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.149+0000 7fe344617700 1 -- start start 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe344617700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe33c091b00 0x7fe33c12c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe344617700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c12ce60 0x7fe33c12f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe344617700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33c096760 con 0x7fe33c096380 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe344617700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33c091ff0 con 0x7fe33c091b00 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe344617700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c092c80 con 0x7fe33c12ce60 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe341bb2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe33c091b00 0x7fe33c12c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe342bb4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c12ce60 0x7fe33c12f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe3423b3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe33c096380 0x7fe33c094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51818/0 (socket says 172.21.15.89:51818) 2023-09-30T22:14:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe3423b3700 1 -- 172.21.15.89:0/2978433427 learned_addr learned my addr 172.21.15.89:0/2978433427 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe341bb2700 1 -- 172.21.15.89:0/2978433427 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c12ce60 msgr2=0x7fe33c12f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.150+0000 7fe341bb2700 1 --2- 172.21.15.89:0/2978433427 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c12ce60 0x7fe33c12f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe341bb2700 1 -- 172.21.15.89:0/2978433427 >> v1:172.21.15.89:6789/0 conn(0x7fe33c096380 legacy=0x7fe33c094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe3413b1700 1 -- 172.21.15.89:0/2978433427 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704495834 0 0) 0x7fe33c096760 con 0x7fe33c096380 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe341bb2700 1 -- 172.21.15.89:0/2978433427 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c094e40 con 0x7fe33c091b00 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe342bb4700 1 --2- 172.21.15.89:0/2978433427 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c12ce60 0x7fe33c12f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe341bb2700 1 --2- 172.21.15.89:0/2978433427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c12c970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fe338009760 tx=0x7fe338009060).ready entity=mon.0 client_cookie=c32bdf7f5e0ff065 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe3413b1700 1 -- 172.21.15.89:0/2978433427 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33800f510 con 0x7fe33c091b00 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe3413b1700 1 -- 172.21.15.89:0/2978433427 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33800f670 con 0x7fe33c091b00 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe3413b1700 1 -- 172.21.15.89:0/2978433427 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33800f9f0 con 0x7fe33c091b00 2023-09-30T22:14:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe344617700 1 -- 172.21.15.89:0/2978433427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 msgr2=0x7fe33c12c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe344617700 1 --2- 172.21.15.89:0/2978433427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c12c970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fe338009760 tx=0x7fe338009060).stop 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe344617700 1 -- 172.21.15.89:0/2978433427 shutdown_connections 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe344617700 1 --2- 172.21.15.89:0/2978433427 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c12ce60 0x7fe33c12f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe344617700 1 --2- 172.21.15.89:0/2978433427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c12c970 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe344617700 1 -- 172.21.15.89:0/2978433427 >> 172.21.15.89:0/2978433427 conn(0x7fe33c097470 msgr2=0x7fe33c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe344617700 1 -- 172.21.15.89:0/2978433427 shutdown_connections 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.151+0000 7fe344617700 1 -- 172.21.15.89:0/2978433427 wait complete. 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.152+0000 7fe344617700 1 Processor -- start 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.152+0000 7fe344617700 1 -- start start 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe344617700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c1296d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe344617700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c096380 0x7fe33c129bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe344617700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe33c12ce60 0x7fe33c12a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe344617700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c12a5a0 con 0x7fe33c091b00 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe344617700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c12a750 con 0x7fe33c12ce60 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe344617700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c06a520 con 0x7fe33c096380 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c1296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c1296d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38826/0 (socket says 172.21.15.89:38826) 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 -- 172.21.15.89:0/258875016 learned_addr learned my addr 172.21.15.89:0/258875016 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe342bb4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe33c12ce60 0x7fe33c12a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe341bb2700 1 --2- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c096380 0x7fe33c129bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 -- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c096380 msgr2=0x7fe33c129bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 --2- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c096380 0x7fe33c129bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 -- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe33c12ce60 msgr2=0x7fe33c12a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 --2- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe33c12ce60 0x7fe33c12a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 -- 172.21.15.89:0/258875016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe334009a20 con 0x7fe33c091b00 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3423b3700 1 --2- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c1296d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe32c00c4c0 tx=0x7fe32c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3327fc700 1 -- 172.21.15.89:0/258875016 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00ce30 con 0x7fe33c091b00 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3327fc700 1 -- 172.21.15.89:0/258875016 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c0072e0 con 0x7fe33c091b00 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3380092a0 con 0x7fe33c091b00 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe3327fc700 1 -- 172.21.15.89:0/258875016 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0057a0 con 0x7fe33c091b00 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.153+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c12c560 con 0x7fe33c091b00 2023-09-30T22:14:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.154+0000 7fe3327fc700 1 -- 172.21.15.89:0/258875016 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fe32c005b90 con 0x7fe33c091b00 2023-09-30T22:14:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.154+0000 7fe3327fc700 1 -- 172.21.15.89:0/258875016 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fe32c03a610 con 0x7fe33c091b00 2023-09-30T22:14:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.154+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33c05a260 con 0x7fe33c091b00 2023-09-30T22:14:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.159+0000 7fe3327fc700 1 -- 172.21.15.89:0/258875016 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fe32c005e40 con 0x7fe33c091b00 2023-09-30T22:14:05.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.311+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 --> v1:172.21.15.89:6816/118161 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe33c0939b0 con 0x7fe32002bb60 2023-09-30T22:14:05.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.317+0000 7fe3327fc700 1 -- 172.21.15.89:0/258875016 <== mgr.4102 v1:172.21.15.89:6816/118161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242457 (unknown 2965378022 0 1817777937) 0x7fe33c0939b0 con 0x7fe32002bb60 2023-09-30T22:14:05.320 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:05.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 >> v1:172.21.15.89:6816/118161 conn(0x7fe32002bb60 legacy=0x7fe32002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 msgr2=0x7fe33c1296d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 --2- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c1296d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe32c00c4c0 tx=0x7fe32c007c10).stop 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 shutdown_connections 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 --2- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe33c12ce60 0x7fe33c12a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 --2- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe33c091b00 0x7fe33c1296d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 --2- 172.21.15.89:0/258875016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe33c096380 0x7fe33c129bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 >> 172.21.15.89:0/258875016 conn(0x7fe33c097470 msgr2=0x7fe33c1242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 shutdown_connections 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.320+0000 7fe344617700 1 -- 172.21.15.89:0/258875016 wait complete. 2023-09-30T22:14:05.322 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:14:05.331 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-09-30T22:14:04.745502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188832,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298528,"statfs":{"total":386547056640,"available":385329692672,"internally_reserved":0,"allocated":458752,"data_stored":120816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.551315+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:13:57.551315+0000","last_peered":"2023-09-30T22:13:57.551315+0000","last_clean":"2023-09-30T22:13:57.551315+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:13:57.551315+0000","last_undegraded":"2023-09-30T22:13:57.551315+0000","last_fullsized":"2023-09-30T22:13:57.551315+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553773+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:13:57.553773+0000","last_peered":"2023-09-30T22:13:57.553773+0000","last_clean":"2023-09-30T22:13:57.553773+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:13:57.553773+0000","last_undegraded":"2023-09-30T22:13:57.553773+0000","last_fullsized":"2023-09-30T22:13:57.553773+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553720+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:13:57.553720+0000","last_peered":"2023-09-30T22:13:57.553720+0000","last_clean":"2023-09-30T22:13:57.553720+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:13:57.553720+0000","last_undegraded":"2023-09-30T22:13:57.553720+0000","last_fullsized":"2023-09-30T22:13:57.553720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553739+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:13:57.553739+0000","last_peered":"2023-09-30T22:13:57.553739+0000","last_clean":"2023-09-30T22:13:57.553739+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:13:57.553739+0000","last_undegraded":"2023-09-30T22:13:57.553739+0000","last_fullsized":"2023-09-30T22:13:57.553739+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.541758+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:13:57.541758+0000","last_peered":"2023-09-30T22:13:57.541758+0000","last_clean":"2023-09-30T22:13:57.541758+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:13:57.541758+0000","last_undegraded":"2023-09-30T22:13:57.541758+0000","last_fullsized":"2023-09-30T22:13:57.541758+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553778+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:13:57.553778+0000","last_peered":"2023-09-30T22:13:57.553778+0000","last_clean":"2023-09-30T22:13:57.553778+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:13:57.553778+0000","last_undegraded":"2023-09-30T22:13:57.553778+0000","last_fullsized":"2023-09-30T22:13:57.553778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.555769+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:13:57.555769+0000","last_peered":"2023-09-30T22:13:57.555769+0000","last_clean":"2023-09-30T22:13:57.555769+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:13:57.555769+0000","last_undegraded":"2023-09-30T22:13:57.555769+0000","last_fullsized":"2023-09-30T22:13:57.555769+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.551277+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:13:57.551277+0000","last_peered":"2023-09-30T22:13:57.551277+0000","last_clean":"2023-09-30T22:13:57.551277+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:13:57.551277+0000","last_undegraded":"2023-09-30T22:13:57.551277+0000","last_fullsized":"2023-09-30T22:13:57.551277+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.555720+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:13:57.555720+0000","last_peered":"2023-09-30T22:13:57.555720+0000","last_clean":"2023-09-30T22:13:57.555720+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:13:57.555720+0000","last_undegraded":"2023-09-30T22:13:57.555720+0000","last_fullsized":"2023-09-30T22:13:57.555720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]}]},{"osd":0,"up_from":9,"seq":38654705666,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:14:05.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:14:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b985ea700 1 Processor -- start 2023-09-30T22:14:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b985ea700 1 -- start start 2023-09-30T22:14:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b985ea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90096380 0x7f0b90094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b985ea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b90094c20 0x7f0b90127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b985ea700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b90096760 con 0x7f0b90091b00 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b985ea700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b90091ff0 con 0x7f0b90094c20 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b985ea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90092c80 con 0x7f0b90096380 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b96386700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90096380 0x7f0b90094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b96b87700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b90094c20 0x7f0b90127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b96386700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90096380 0x7f0b90094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57436/0 (socket says 172.21.15.89:57436) 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b96b87700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b90094c20 0x7f0b90127d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38832/0 (socket says 172.21.15.89:38832) 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b96386700 1 -- 172.21.15.89:0/1764154930 learned_addr learned my addr 172.21.15.89:0/1764154930 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.487+0000 7f0b95b85700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0b90091b00 0x7f0b90127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51834/0 (socket says 172.21.15.89:51834) 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b96b87700 1 -- 172.21.15.89:0/1764154930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90096380 msgr2=0x7f0b90094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b96b87700 1 --2- 172.21.15.89:0/1764154930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90096380 0x7f0b90094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b96b87700 1 -- 172.21.15.89:0/1764154930 >> v1:172.21.15.89:6789/0 conn(0x7f0b90091b00 legacy=0x7f0b90127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b95384700 1 -- 172.21.15.89:0/1764154930 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998476378 0 0) 0x7f0b90096760 con 0x7f0b90091b00 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b96b87700 1 -- 172.21.15.89:0/1764154930 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90093900 con 0x7f0b90094c20 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b96b87700 1 --2- 172.21.15.89:0/1764154930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b90127d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f0b88009e90 tx=0x7f0b88009780).ready entity=mon.0 client_cookie=458b5e126b6df6c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b95384700 1 -- 172.21.15.89:0/1764154930 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800e070 con 0x7f0b90094c20 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b95384700 1 -- 172.21.15.89:0/1764154930 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8800bb20 con 0x7f0b90094c20 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b95384700 1 -- 172.21.15.89:0/1764154930 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800bea0 con 0x7f0b90094c20 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b985ea700 1 -- 172.21.15.89:0/1764154930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 msgr2=0x7f0b90127d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b985ea700 1 --2- 172.21.15.89:0/1764154930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b90127d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f0b88009e90 tx=0x7f0b88009780).stop 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b985ea700 1 -- 172.21.15.89:0/1764154930 shutdown_connections 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b985ea700 1 --2- 172.21.15.89:0/1764154930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b90127d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.488+0000 7f0b985ea700 1 --2- 172.21.15.89:0/1764154930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90096380 0x7f0b90094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.489+0000 7f0b985ea700 1 -- 172.21.15.89:0/1764154930 >> 172.21.15.89:0/1764154930 conn(0x7f0b90097470 msgr2=0x7f0b90097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.489+0000 7f0b985ea700 1 -- 172.21.15.89:0/1764154930 shutdown_connections 2023-09-30T22:14:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.489+0000 7f0b985ea700 1 -- 172.21.15.89:0/1764154930 wait complete. 2023-09-30T22:14:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b985ea700 1 Processor -- start 2023-09-30T22:14:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b985ea700 1 -- start start 2023-09-30T22:14:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b985ea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90091b00 0x7f0b90111550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b985ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b9013a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b985ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b90096380 0x7f0b9013a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b985ea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900989f0 con 0x7f0b90094c20 2023-09-30T22:14:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b985ea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9006a520 con 0x7f0b90096380 2023-09-30T22:14:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b985ea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90124ec0 con 0x7f0b90091b00 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b95b85700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b9013a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b95b85700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b9013a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38838/0 (socket says 172.21.15.89:38838) 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b95b85700 1 -- 172.21.15.89:0/1039722234 learned_addr learned my addr 172.21.15.89:0/1039722234 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b96b87700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b90096380 0x7f0b9013a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b96386700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90091b00 0x7f0b90111550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b95b85700 1 -- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90091b00 msgr2=0x7f0b90111550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b95b85700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90091b00 0x7f0b90111550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b95b85700 1 -- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b90096380 msgr2=0x7f0b9013a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b95b85700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b90096380 0x7f0b9013a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.490+0000 7f0b95b85700 1 -- 172.21.15.89:0/1039722234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b88009a20 con 0x7f0b90094c20 2023-09-30T22:14:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.491+0000 7f0b95b85700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b9013a400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0b8000c4c0 tx=0x7f0b80007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.491+0000 7f0b867fc700 1 -- 172.21.15.89:0/1039722234 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000ce30 con 0x7f0b90094c20 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.491+0000 7f0b867fc700 1 -- 172.21.15.89:0/1039722234 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b800072e0 con 0x7f0b90094c20 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.491+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b90111290 con 0x7f0b90094c20 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.491+0000 7f0b867fc700 1 -- 172.21.15.89:0/1039722234 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b800056e0 con 0x7f0b90094c20 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.491+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b90137e10 con 0x7f0b90094c20 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.491+0000 7f0b867fc700 1 -- 172.21.15.89:0/1039722234 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0b80005a90 con 0x7f0b90094c20 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.492+0000 7f0b867fc700 1 -- 172.21.15.89:0/1039722234 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0b8003a760 con 0x7f0b90094c20 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.492+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b90123650 con 0x7f0b90094c20 2023-09-30T22:14:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.496+0000 7f0b867fc700 1 -- 172.21.15.89:0/1039722234 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0b80018030 con 0x7f0b90094c20 2023-09-30T22:14:05.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.653+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 --> v1:172.21.15.89:6816/118161 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b9005a260 con 0x7f0b7402bb90 2023-09-30T22:14:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.659+0000 7f0b867fc700 1 -- 172.21.15.89:0/1039722234 <== mgr.4102 v1:172.21.15.89:6816/118161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242457 (unknown 2965378022 0 1817777937) 0x7f0b9005a260 con 0x7f0b7402bb90 2023-09-30T22:14:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 >> v1:172.21.15.89:6816/118161 conn(0x7f0b7402bb90 legacy=0x7f0b7402e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 msgr2=0x7f0b9013a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b9013a400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0b8000c4c0 tx=0x7f0b80007c10).stop 2023-09-30T22:14:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 shutdown_connections 2023-09-30T22:14:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b90096380 0x7f0b9013a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b90091b00 0x7f0b90111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 --2- 172.21.15.89:0/1039722234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b90094c20 0x7f0b9013a400 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 >> 172.21.15.89:0/1039722234 conn(0x7f0b90097470 msgr2=0x7f0b90059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 shutdown_connections 2023-09-30T22:14:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.662+0000 7f0b985ea700 1 -- 172.21.15.89:0/1039722234 wait complete. 2023-09-30T22:14:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:14:05.673 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-09-30T22:14:04.745502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188832,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298528,"statfs":{"total":386547056640,"available":385329692672,"internally_reserved":0,"allocated":458752,"data_stored":120816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.551315+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:13:57.551315+0000","last_peered":"2023-09-30T22:13:57.551315+0000","last_clean":"2023-09-30T22:13:57.551315+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:13:57.551315+0000","last_undegraded":"2023-09-30T22:13:57.551315+0000","last_fullsized":"2023-09-30T22:13:57.551315+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553773+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:13:57.553773+0000","last_peered":"2023-09-30T22:13:57.553773+0000","last_clean":"2023-09-30T22:13:57.553773+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:13:57.553773+0000","last_undegraded":"2023-09-30T22:13:57.553773+0000","last_fullsized":"2023-09-30T22:13:57.553773+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553720+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:13:57.553720+0000","last_peered":"2023-09-30T22:13:57.553720+0000","last_clean":"2023-09-30T22:13:57.553720+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:13:57.553720+0000","last_undegraded":"2023-09-30T22:13:57.553720+0000","last_fullsized":"2023-09-30T22:13:57.553720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553739+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:13:57.553739+0000","last_peered":"2023-09-30T22:13:57.553739+0000","last_clean":"2023-09-30T22:13:57.553739+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:13:57.553739+0000","last_undegraded":"2023-09-30T22:13:57.553739+0000","last_fullsized":"2023-09-30T22:13:57.553739+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.541758+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:13:57.541758+0000","last_peered":"2023-09-30T22:13:57.541758+0000","last_clean":"2023-09-30T22:13:57.541758+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:13:57.541758+0000","last_undegraded":"2023-09-30T22:13:57.541758+0000","last_fullsized":"2023-09-30T22:13:57.541758+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553778+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:13:57.553778+0000","last_peered":"2023-09-30T22:13:57.553778+0000","last_clean":"2023-09-30T22:13:57.553778+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:13:57.553778+0000","last_undegraded":"2023-09-30T22:13:57.553778+0000","last_fullsized":"2023-09-30T22:13:57.553778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.555769+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:13:57.555769+0000","last_peered":"2023-09-30T22:13:57.555769+0000","last_clean":"2023-09-30T22:13:57.555769+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:13:57.555769+0000","last_undegraded":"2023-09-30T22:13:57.555769+0000","last_fullsized":"2023-09-30T22:13:57.555769+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.551277+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:13:57.551277+0000","last_peered":"2023-09-30T22:13:57.551277+0000","last_clean":"2023-09-30T22:13:57.551277+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:13:57.551277+0000","last_undegraded":"2023-09-30T22:13:57.551277+0000","last_fullsized":"2023-09-30T22:13:57.551277+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.555720+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:13:57.555720+0000","last_peered":"2023-09-30T22:13:57.555720+0000","last_clean":"2023-09-30T22:13:57.555720+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:13:57.555720+0000","last_undegraded":"2023-09-30T22:13:57.555720+0000","last_fullsized":"2023-09-30T22:13:57.555720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]}]},{"osd":0,"up_from":9,"seq":38654705666,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:14:05.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3f is not active+clean 2023-09-30T22:14:05.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3e is not active+clean 2023-09-30T22:14:05.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3d is not active+clean 2023-09-30T22:14:05.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3c is not active+clean 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3b is not active+clean 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3a is not active+clean 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.39 is not active+clean 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.38 is not active+clean 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.37 is not active+clean 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.36 is not active+clean 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.35 is not active+clean 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.34 is not active+clean 2023-09-30T22:14:05.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.33 is not active+clean 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.32 is not active+clean 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.31 is not active+clean 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.30 is not active+clean 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2f is not active+clean 2023-09-30T22:14:05.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2e is not active+clean 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2d is not active+clean 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2c is not active+clean 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2b is not active+clean 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2a is not active+clean 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.29 is not active+clean 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.28 is not active+clean 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.27 is not active+clean 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.26 is not active+clean 2023-09-30T22:14:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.25 is not active+clean 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-09-30T22:14:05.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-09-30T22:14:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-09-30T22:14:05.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-09-30T22:14:05.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-09-30T22:14:05.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-09-30T22:14:05.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-09-30T22:14:05.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:00.666720+0000', 'last_change': '2023-09-30T22:14:00.666720+0000', 'last_active': '2023-09-30T22:14:00.666720+0000', 'last_peered': '2023-09-30T22:14:00.666720+0000', 'last_clean': '2023-09-30T22:14:00.666720+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:00.666720+0000', 'last_undegraded': '2023-09-30T22:14:00.666720+0000', 'last_fullsized': '2023-09-30T22:14:00.666720+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:00.666720+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-09-30T22:14:05.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.20 is not active+clean 2023-09-30T22:14:05.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.21 is not active+clean 2023-09-30T22:14:05.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.22 is not active+clean 2023-09-30T22:14:05.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.23 is not active+clean 2023-09-30T22:14:05.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.24 is not active+clean 2023-09-30T22:14:05.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-30T22:14:01.670302+0000', 'last_change': '2023-09-30T22:14:01.670302+0000', 'last_active': '2023-09-30T22:14:01.670302+0000', 'last_peered': '2023-09-30T22:14:01.670302+0000', 'last_clean': '2023-09-30T22:14:01.670302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-30T22:14:01.670302+0000', 'last_undegraded': '2023-09-30T22:14:01.670302+0000', 'last_fullsized': '2023-09-30T22:14:01.670302+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'last_clean_scrub_stamp': '2023-09-30T22:14:01.670302+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-30T22:14:05.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-30T22:14:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dd2261700 1 Processor -- start 2023-09-30T22:14:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dd2261700 1 -- start start 2023-09-30T22:14:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dd2261700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9dcc059070 0x7f9dcc12ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dd2261700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc127a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dd2261700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dcc091ed0 con 0x7f9dcc12b130 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dd2261700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc092b00 con 0x7f9dcc059070 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dd2261700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc091c10 con 0x7f9dcc12b530 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dcb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9dcc059070 0x7f9dcc12ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dcbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc127a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dcb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9dcc059070 0x7f9dcc12ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38846/0 (socket says 172.21.15.89:38846) 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dcb7fe700 1 -- 172.21.15.89:0/805045506 learned_addr learned my addr 172.21.15.89:0/805045506 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dcaffd700 1 --1- 172.21.15.89:0/805045506 >> v1:172.21.15.89:6789/0 conn(0x7f9dcc12b130 0x7f9dcc127380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51850/0 (socket says 172.21.15.89:51850) 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dcbfff700 1 -- 172.21.15.89:0/805045506 >> v2:172.21.15.89:3300/0 conn(0x7f9dcc059070 msgr2=0x7f9dcc12ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dcbfff700 1 --2- 172.21.15.89:0/805045506 >> v2:172.21.15.89:3300/0 conn(0x7f9dcc059070 0x7f9dcc12ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.830+0000 7f9dcbfff700 1 -- 172.21.15.89:0/805045506 >> v1:172.21.15.89:6789/0 conn(0x7f9dcc12b130 legacy=0x7f9dcc127380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dcbfff700 1 -- 172.21.15.89:0/805045506 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc096d00 con 0x7f9dcc12b530 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dca7fc700 1 -- 172.21.15.89:0/805045506 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033683762 0 0) 0x7f9dcc091ed0 con 0x7f9dcc12b130 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dcb7fe700 1 --2- 172.21.15.89:0/805045506 >> v2:172.21.15.89:3300/0 conn(0x7f9dcc059070 0x7f9dcc12ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dcbfff700 1 --2- 172.21.15.89:0/805045506 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc127a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9dc4009e60 tx=0x7f9dc4009780).ready entity=mon.1 client_cookie=462eb2882edc81e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dca7fc700 1 -- 172.21.15.89:0/805045506 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400e070 con 0x7f9dcc12b530 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dca7fc700 1 -- 172.21.15.89:0/805045506 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc400bb60 con 0x7f9dcc12b530 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dca7fc700 1 -- 172.21.15.89:0/805045506 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4012410 con 0x7f9dcc12b530 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dd2261700 1 -- 172.21.15.89:0/805045506 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 msgr2=0x7f9dcc127a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dd2261700 1 --2- 172.21.15.89:0/805045506 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc127a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9dc4009e60 tx=0x7f9dc4009780).stop 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dd2261700 1 -- 172.21.15.89:0/805045506 shutdown_connections 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dd2261700 1 --2- 172.21.15.89:0/805045506 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc127a90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dd2261700 1 --2- 172.21.15.89:0/805045506 >> v2:172.21.15.89:3300/0 conn(0x7f9dcc059070 0x7f9dcc12ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dd2261700 1 -- 172.21.15.89:0/805045506 >> 172.21.15.89:0/805045506 conn(0x7f9dcc09a8e0 msgr2=0x7f9dcc09acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dd2261700 1 -- 172.21.15.89:0/805045506 shutdown_connections 2023-09-30T22:14:05.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.831+0000 7f9dd2261700 1 -- 172.21.15.89:0/805045506 wait complete. 2023-09-30T22:14:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dd2261700 1 Processor -- start 2023-09-30T22:14:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dd2261700 1 -- start start 2023-09-30T22:14:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dd2261700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dcc059070 0x7f9dcc115f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dd2261700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9dcc12b130 0x7f9dcc116420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dd2261700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc116910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dd2261700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc093da0 con 0x7f9dcc059070 2023-09-30T22:14:05.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dd2261700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc096f30 con 0x7f9dcc12b530 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dd2261700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc1189c0 con 0x7f9dcc12b130 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.833+0000 7f9dcaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9dcc12b130 0x7f9dcc116420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9dcc12b130 0x7f9dcc116420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57440/0 (socket says 172.21.15.89:57440) 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcaffd700 1 -- 172.21.15.89:0/3788076496 learned_addr learned my addr 172.21.15.89:0/3788076496 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcbfff700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcb7fe700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dcc059070 0x7f9dcc115f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcaffd700 1 -- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 msgr2=0x7f9dcc116910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcaffd700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc116910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcaffd700 1 -- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dcc059070 msgr2=0x7f9dcc115f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcaffd700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dcc059070 0x7f9dcc115f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:05.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcaffd700 1 -- 172.21.15.89:0/3788076496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc009370 con 0x7f9dcc12b130 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcbfff700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcb7fe700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dcc059070 0x7f9dcc115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dcaffd700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9dcc12b130 0x7f9dcc116420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f9db400c4c0 tx=0x7f9db4007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dc37fe700 1 -- 172.21.15.89:0/3788076496 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400ce30 con 0x7f9dcc12b130 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dc37fe700 1 -- 172.21.15.89:0/3788076496 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db40072e0 con 0x7f9dcc12b130 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc4009a20 con 0x7f9dcc12b130 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dc37fe700 1 -- 172.21.15.89:0/3788076496 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db40056e0 con 0x7f9dcc12b130 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.834+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc119ab0 con 0x7f9dcc12b130 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.835+0000 7f9dc37fe700 1 -- 172.21.15.89:0/3788076496 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9db4005a90 con 0x7f9dcc12b130 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.835+0000 7f9dc37fe700 1 -- 172.21.15.89:0/3788076496 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f9db403a4b0 con 0x7f9dcc12b130 2023-09-30T22:14:05.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.835+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dcc119ab0 con 0x7f9dcc12b130 2023-09-30T22:14:05.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:05.840+0000 7f9dc37fe700 1 -- 172.21.15.89:0/3788076496 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9dcc119ab0 con 0x7f9dcc12b130 2023-09-30T22:14:06.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.059+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9dcc119ab0 con 0x7f9dcc12b130 2023-09-30T22:14:06.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.059+0000 7f9dc37fe700 1 -- 172.21.15.89:0/3788076496 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1020 (secure 0 0 0) 0x7f9dcc119ab0 con 0x7f9dcc12b130 2023-09-30T22:14:06.059 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:06.059 INFO:teuthology.orchestra.run.smithi089.stdout:{"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":20,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1696112034,"num_in_osds":4,"osd_in_since":1696112027,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":0,"data_bytes":0,"bytes_used":1217363968,"bytes_avail":385329692672,"bytes_total":386547056640,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":5,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4190}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-09-30T22:13:45.748970+0000","services":{}},"progress_events":{}} 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 >> v1:172.21.15.89:6816/118161 conn(0x7f9dac02bb90 legacy=0x7f9dac02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9dcc12b130 msgr2=0x7f9dcc116420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9dcc12b130 0x7f9dcc116420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f9db400c4c0 tx=0x7f9db4007c10).stop 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 shutdown_connections 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9dcc12b530 0x7f9dcc116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dcc059070 0x7f9dcc115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 --2- 172.21.15.89:0/3788076496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9dcc12b130 0x7f9dcc116420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 >> 172.21.15.89:0/3788076496 conn(0x7f9dcc09a8e0 msgr2=0x7f9dcc1273b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:06.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 shutdown_connections 2023-09-30T22:14:06.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.061+0000 7f9dd2261700 1 -- 172.21.15.89:0/3788076496 wait complete. 2023-09-30T22:14:06.072 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-30T22:14:06.072 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:14:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c70ba700 1 Processor -- start 2023-09-30T22:14:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c70ba700 1 -- start start 2023-09-30T22:14:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c70ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0096d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c70ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c0066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c70ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f50c00671c0 0x7f50c0067640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c70ba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c0067b30 con 0x7f50c00671c0 2023-09-30T22:14:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c70ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0092fe0 con 0x7f50c00969b0 2023-09-30T22:14:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c70ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0094560 con 0x7f50c0097280 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50bffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c0066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c4e56700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0096d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50bffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c0066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57452/0 (socket says 172.21.15.89:57452) 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c4e56700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0096d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52254/0 (socket says 172.21.15.89:52254) 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50bffff700 1 -- 172.21.15.89:0/712431897 learned_addr learned my addr 172.21.15.89:0/712431897 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.187+0000 7f50c5657700 1 --2- 172.21.15.89:0/712431897 >> v2:172.21.15.89:3300/0 conn(0x7f50c00671c0 0x7f50c0067640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50c4e56700 1 -- 172.21.15.89:0/712431897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 msgr2=0x7f50c0066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50c4e56700 1 --2- 172.21.15.89:0/712431897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c0066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50c4e56700 1 -- 172.21.15.89:0/712431897 >> v2:172.21.15.89:3300/0 conn(0x7f50c00671c0 msgr2=0x7f50c0067640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50c4e56700 1 --2- 172.21.15.89:0/712431897 >> v2:172.21.15.89:3300/0 conn(0x7f50c00671c0 0x7f50c0067640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50c4e56700 1 -- 172.21.15.89:0/712431897 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c0093260 con 0x7f50c00969b0 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50bffff700 1 --2- 172.21.15.89:0/712431897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c0066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50c5657700 1 --2- 172.21.15.89:0/712431897 >> v2:172.21.15.89:3300/0 conn(0x7f50c00671c0 0x7f50c0067640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50c4e56700 1 --2- 172.21.15.89:0/712431897 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0096d90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f50b4009760 tx=0x7f50b4009060).ready entity=mon.1 client_cookie=d24b45494e3a0bf2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50bf7fe700 1 -- 172.21.15.89:0/712431897 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400d4a0 con 0x7f50c00969b0 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50bf7fe700 1 -- 172.21.15.89:0/712431897 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b400d600 con 0x7f50c00969b0 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.188+0000 7f50bf7fe700 1 -- 172.21.15.89:0/712431897 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400d980 con 0x7f50c00969b0 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 -- 172.21.15.89:0/712431897 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 msgr2=0x7f50c0096d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 --2- 172.21.15.89:0/712431897 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0096d90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f50b4009760 tx=0x7f50b4009060).stop 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 -- 172.21.15.89:0/712431897 shutdown_connections 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 --2- 172.21.15.89:0/712431897 >> v2:172.21.15.89:3300/0 conn(0x7f50c00671c0 0x7f50c0067640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 --2- 172.21.15.89:0/712431897 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0096d90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 --2- 172.21.15.89:0/712431897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c0066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 -- 172.21.15.89:0/712431897 >> 172.21.15.89:0/712431897 conn(0x7f50c009a220 msgr2=0x7f50c009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 -- 172.21.15.89:0/712431897 shutdown_connections 2023-09-30T22:14:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 -- 172.21.15.89:0/712431897 wait complete. 2023-09-30T22:14:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.189+0000 7f50c70ba700 1 Processor -- start 2023-09-30T22:14:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c70ba700 1 -- start start 2023-09-30T22:14:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c70ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50c00671c0 0x7f50c011d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c70ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0068170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c70ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c00687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c70ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0069c30 con 0x7f50c00671c0 2023-09-30T22:14:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c70ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c011cc40 con 0x7f50c00969b0 2023-09-30T22:14:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c70ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50c011c6a0 con 0x7f50c0097280 2023-09-30T22:14:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c4e56700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50c00671c0 0x7f50c011d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50bffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0068170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c5657700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c00687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c4e56700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50c00671c0 0x7f50c011d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38866/0 (socket says 172.21.15.89:38866) 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50bffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0068170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52264/0 (socket says 172.21.15.89:52264) 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c4e56700 1 -- 172.21.15.89:0/1161942833 learned_addr learned my addr 172.21.15.89:0/1161942833 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c4e56700 1 -- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 msgr2=0x7f50c00687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c4e56700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c00687b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c4e56700 1 -- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 msgr2=0x7f50c0068170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c4e56700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0068170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c4e56700 1 -- 172.21.15.89:0/1161942833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8009a20 con 0x7f50c00671c0 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50bffff700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0068170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.190+0000 7f50c5657700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c00687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.191+0000 7f50c4e56700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50c00671c0 0x7f50c011d140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f50b4009d50 tx=0x7f50b4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.191+0000 7f50bcff9700 1 -- 172.21.15.89:0/1161942833 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400dd30 con 0x7f50c00671c0 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.191+0000 7f50bcff9700 1 -- 172.21.15.89:0/1161942833 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b400de90 con 0x7f50c00671c0 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.191+0000 7f50bcff9700 1 -- 172.21.15.89:0/1161942833 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400d980 con 0x7f50c00671c0 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.191+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac009370 con 0x7f50c00671c0 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.191+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b40092a0 con 0x7f50c00671c0 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.191+0000 7f50bcff9700 1 -- 172.21.15.89:0/1161942833 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f50b401d640 con 0x7f50c00671c0 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.192+0000 7f50bcff9700 1 -- 172.21.15.89:0/1161942833 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f50b403c7e0 con 0x7f50c00671c0 2023-09-30T22:14:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.192+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c0092320 con 0x7f50c00671c0 2023-09-30T22:14:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.196+0000 7f50bcff9700 1 -- 172.21.15.89:0/1161942833 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f50b401d8f0 con 0x7f50c00671c0 2023-09-30T22:14:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.349+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 --> v1:172.21.15.89:6816/118161 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50c005a260 con 0x7f50a402bb60 2023-09-30T22:14:06.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.355+0000 7f50bcff9700 1 -- 172.21.15.89:0/1161942833 <== mgr.4102 v1:172.21.15.89:6816/118161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242457 (unknown 2965378022 0 1817777937) 0x7f50c005a260 con 0x7f50a402bb60 2023-09-30T22:14:06.357 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 >> v1:172.21.15.89:6816/118161 conn(0x7f50a402bb60 legacy=0x7f50a402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50c00671c0 msgr2=0x7f50c011d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50c00671c0 0x7f50c011d140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f50b4009d50 tx=0x7f50b4016000).stop 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 shutdown_connections 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50c0097280 0x7f50c00687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50c00671c0 0x7f50c011d140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 --2- 172.21.15.89:0/1161942833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50c00969b0 0x7f50c0068170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 >> 172.21.15.89:0/1161942833 conn(0x7f50c009a220 msgr2=0x7f50c00955f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 shutdown_connections 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:06.358+0000 7f50c70ba700 1 -- 172.21.15.89:0/1161942833 wait complete. 2023-09-30T22:14:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:14:06.369 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-09-30T22:14:04.745502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188832,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298528,"statfs":{"total":386547056640,"available":385329692672,"internally_reserved":0,"allocated":458752,"data_stored":120816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.551315+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:13:57.551315+0000","last_peered":"2023-09-30T22:13:57.551315+0000","last_clean":"2023-09-30T22:13:57.551315+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:13:57.551315+0000","last_undegraded":"2023-09-30T22:13:57.551315+0000","last_fullsized":"2023-09-30T22:13:57.551315+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553773+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:13:57.553773+0000","last_peered":"2023-09-30T22:13:57.553773+0000","last_clean":"2023-09-30T22:13:57.553773+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:13:57.553773+0000","last_undegraded":"2023-09-30T22:13:57.553773+0000","last_fullsized":"2023-09-30T22:13:57.553773+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553720+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:13:57.553720+0000","last_peered":"2023-09-30T22:13:57.553720+0000","last_clean":"2023-09-30T22:13:57.553720+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:13:57.553720+0000","last_undegraded":"2023-09-30T22:13:57.553720+0000","last_fullsized":"2023-09-30T22:13:57.553720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553739+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:13:57.553739+0000","last_peered":"2023-09-30T22:13:57.553739+0000","last_clean":"2023-09-30T22:13:57.553739+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:13:57.553739+0000","last_undegraded":"2023-09-30T22:13:57.553739+0000","last_fullsized":"2023-09-30T22:13:57.553739+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.541758+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:13:57.541758+0000","last_peered":"2023-09-30T22:13:57.541758+0000","last_clean":"2023-09-30T22:13:57.541758+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:13:57.541758+0000","last_undegraded":"2023-09-30T22:13:57.541758+0000","last_fullsized":"2023-09-30T22:13:57.541758+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.553778+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:13:57.553778+0000","last_peered":"2023-09-30T22:13:57.553778+0000","last_clean":"2023-09-30T22:13:57.553778+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:13:57.553778+0000","last_undegraded":"2023-09-30T22:13:57.553778+0000","last_fullsized":"2023-09-30T22:13:57.553778+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.555769+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:13:57.555769+0000","last_peered":"2023-09-30T22:13:57.555769+0000","last_clean":"2023-09-30T22:13:57.555769+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:13:57.555769+0000","last_undegraded":"2023-09-30T22:13:57.555769+0000","last_fullsized":"2023-09-30T22:13:57.555769+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.551277+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:13:57.551277+0000","last_peered":"2023-09-30T22:13:57.551277+0000","last_clean":"2023-09-30T22:13:57.551277+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:13:57.551277+0000","last_undegraded":"2023-09-30T22:13:57.551277+0000","last_fullsized":"2023-09-30T22:13:57.551277+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:13:57.555720+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:13:57.555720+0000","last_peered":"2023-09-30T22:13:57.555720+0000","last_clean":"2023-09-30T22:13:57.555720+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:13:57.555720+0000","last_undegraded":"2023-09-30T22:13:57.555720+0000","last_fullsized":"2023-09-30T22:13:57.555720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:00.666720+0000","last_change":"2023-09-30T22:14:00.666720+0000","last_active":"2023-09-30T22:14:00.666720+0000","last_peered":"2023-09-30T22:14:00.666720+0000","last_clean":"2023-09-30T22:14:00.666720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:00.666720+0000","last_undegraded":"2023-09-30T22:14:00.666720+0000","last_fullsized":"2023-09-30T22:14:00.666720+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:14:01.670302+0000","last_change":"2023-09-30T22:14:01.670302+0000","last_active":"2023-09-30T22:14:01.670302+0000","last_peered":"2023-09-30T22:14:01.670302+0000","last_clean":"2023-09-30T22:14:01.670302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:14:01.670302+0000","last_undegraded":"2023-09-30T22:14:01.670302+0000","last_fullsized":"2023-09-30T22:14:01.670302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]}]},{"osd":0,"up_from":9,"seq":38654705666,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:14:07.665 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-09-30T22:14:07.665+0000 7fb1443e5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-30T22:14:09.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:14:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.505+0000 7f9ccc967700 1 Processor -- start 2023-09-30T22:14:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.505+0000 7f9ccc967700 1 -- start start 2023-09-30T22:14:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.505+0000 7f9ccc967700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc8094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.505+0000 7f9ccc967700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.505+0000 7f9ccc967700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc8096760 con 0x7f9cc8091b00 2023-09-30T22:14:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.505+0000 7f9ccc967700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc8091ff0 con 0x7f9cc8096380 2023-09-30T22:14:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.505+0000 7f9ccc967700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc8092c80 con 0x7f9cc8094c20 2023-09-30T22:14:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc8094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc5d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9cc8091b00 0x7f9cc8127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51852/0 (socket says 172.21.15.89:51852) 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc5d9b700 1 -- 172.21.15.89:0/3052773798 learned_addr learned my addr 172.21.15.89:0/3052773798 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc659c700 1 -- 172.21.15.89:0/3052773798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 msgr2=0x7f9cc8127d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc659c700 1 --2- 172.21.15.89:0/3052773798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8127d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc659c700 1 -- 172.21.15.89:0/3052773798 >> v1:172.21.15.89:6789/0 conn(0x7f9cc8091b00 legacy=0x7f9cc8127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc559a700 1 -- 172.21.15.89:0/3052773798 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047152404 0 0) 0x7f9cc8096760 con 0x7f9cc8091b00 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc659c700 1 -- 172.21.15.89:0/3052773798 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8093900 con 0x7f9cc8096380 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc6d9d700 1 --2- 172.21.15.89:0/3052773798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc659c700 1 --2- 172.21.15.89:0/3052773798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc8094730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9cb8009790 tx=0x7f9cb8009060).ready entity=mon.1 client_cookie=1b1b3ad844fc298a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc559a700 1 -- 172.21.15.89:0/3052773798 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800f510 con 0x7f9cc8096380 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc559a700 1 -- 172.21.15.89:0/3052773798 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb800f670 con 0x7f9cc8096380 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.506+0000 7f9cc559a700 1 -- 172.21.15.89:0/3052773798 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800f9f0 con 0x7f9cc8096380 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.507+0000 7f9ccc967700 1 -- 172.21.15.89:0/3052773798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 msgr2=0x7f9cc8094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.507+0000 7f9ccc967700 1 --2- 172.21.15.89:0/3052773798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc8094730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9cb8009790 tx=0x7f9cb8009060).stop 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.507+0000 7f9ccc967700 1 -- 172.21.15.89:0/3052773798 shutdown_connections 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.507+0000 7f9ccc967700 1 --2- 172.21.15.89:0/3052773798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc8094730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.507+0000 7f9ccc967700 1 --2- 172.21.15.89:0/3052773798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.507+0000 7f9ccc967700 1 -- 172.21.15.89:0/3052773798 >> 172.21.15.89:0/3052773798 conn(0x7f9cc8097470 msgr2=0x7f9cc8097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.507+0000 7f9ccc967700 1 -- 172.21.15.89:0/3052773798 shutdown_connections 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.507+0000 7f9ccc967700 1 -- 172.21.15.89:0/3052773798 wait complete. 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9ccc967700 1 Processor -- start 2023-09-30T22:14:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9ccc967700 1 -- start start 2023-09-30T22:14:09.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9ccc967700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc8091b00 0x7f9cc811ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9ccc967700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8116cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9ccc967700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc81171a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9ccc967700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc8119370 con 0x7f9cc8091b00 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9ccc967700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc806a520 con 0x7f9cc8096380 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9ccc967700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc811a240 con 0x7f9cc8094c20 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc5d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8116cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc81171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc8091b00 0x7f9cc811ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc5d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8116cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57480/0 (socket says 172.21.15.89:57480) 2023-09-30T22:14:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc5d9b700 1 -- 172.21.15.89:0/443270289 learned_addr learned my addr 172.21.15.89:0/443270289 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc81171a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52284/0 (socket says 172.21.15.89:52284) 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc5d9b700 1 -- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 msgr2=0x7f9cc81171a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc5d9b700 1 --2- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc81171a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc5d9b700 1 -- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc8091b00 msgr2=0x7f9cc811ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc5d9b700 1 --2- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc8091b00 0x7f9cc811ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc5d9b700 1 -- 172.21.15.89:0/443270289 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc0009a20 con 0x7f9cc8094c20 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.508+0000 7f9cc6d9d700 1 --2- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc81171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.509+0000 7f9cc5d9b700 1 --2- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8116cb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9cbc00c4f0 tx=0x7f9cbc007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.509+0000 7f9cb67fc700 1 -- 172.21.15.89:0/443270289 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0072e0 con 0x7f9cc8094c20 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.509+0000 7f9cb67fc700 1 -- 172.21.15.89:0/443270289 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc007440 con 0x7f9cc8094c20 2023-09-30T22:14:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.509+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb80092a0 con 0x7f9cc8094c20 2023-09-30T22:14:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.509+0000 7f9cb67fc700 1 -- 172.21.15.89:0/443270289 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0057a0 con 0x7f9cc8094c20 2023-09-30T22:14:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.509+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc81188d0 con 0x7f9cc8094c20 2023-09-30T22:14:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.509+0000 7f9cb67fc700 1 -- 172.21.15.89:0/443270289 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9cbc005b90 con 0x7f9cc8094c20 2023-09-30T22:14:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.510+0000 7f9cb67fc700 1 -- 172.21.15.89:0/443270289 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f9cbc03a820 con 0x7f9cc8094c20 2023-09-30T22:14:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.510+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc80990c0 con 0x7f9cc8094c20 2023-09-30T22:14:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.514+0000 7f9cb67fc700 1 -- 172.21.15.89:0/443270289 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9cbc013970 con 0x7f9cc8094c20 2023-09-30T22:14:09.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.666+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 --> v1:172.21.15.89:6816/118161 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9cc805a260 con 0x7f9ca802bb60 2023-09-30T22:14:09.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.673+0000 7f9cb67fc700 1 -- 172.21.15.89:0/443270289 <== mgr.4102 v1:172.21.15.89:6816/118161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249996 (unknown 2965378022 0 3814114769) 0x7f9cc805a260 con 0x7f9ca802bb60 2023-09-30T22:14:09.675 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 >> v1:172.21.15.89:6816/118161 conn(0x7f9ca802bb60 legacy=0x7f9ca802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 msgr2=0x7f9cc8116cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 --2- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8116cb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9cbc00c4f0 tx=0x7f9cbc007c10).stop 2023-09-30T22:14:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 shutdown_connections 2023-09-30T22:14:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 --2- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cc8096380 0x7f9cc81171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 --2- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cc8091b00 0x7f9cc811ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 --2- 172.21.15.89:0/443270289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cc8094c20 0x7f9cc8116cb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 >> 172.21.15.89:0/443270289 conn(0x7f9cc8097470 msgr2=0x7f9cc8126d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 shutdown_connections 2023-09-30T22:14:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.675+0000 7f9ccc967700 1 -- 172.21.15.89:0/443270289 wait complete. 2023-09-30T22:14:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:14:09.686 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-09-30T22:14:08.746226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000429"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090297+0000","last_change":"2023-09-30T22:14:02.086445+0000","last_active":"2023-09-30T22:14:02.090297+0000","last_peered":"2023-09-30T22:14:02.090297+0000","last_clean":"2023-09-30T22:14:02.090297+0000","last_became_active":"2023-09-30T22:14:02.086159+0000","last_became_peered":"2023-09-30T22:14:02.086159+0000","last_unstale":"2023-09-30T22:14:02.090297+0000","last_undegraded":"2023-09-30T22:14:02.090297+0000","last_fullsized":"2023-09-30T22:14:02.090297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088940+0000","last_change":"2023-09-30T22:14:02.082560+0000","last_active":"2023-09-30T22:14:02.088940+0000","last_peered":"2023-09-30T22:14:02.088940+0000","last_clean":"2023-09-30T22:14:02.088940+0000","last_became_active":"2023-09-30T22:14:02.082315+0000","last_became_peered":"2023-09-30T22:14:02.082315+0000","last_unstale":"2023-09-30T22:14:02.088940+0000","last_undegraded":"2023-09-30T22:14:02.088940+0000","last_fullsized":"2023-09-30T22:14:02.088940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091544+0000","last_change":"2023-09-30T22:14:02.082673+0000","last_active":"2023-09-30T22:14:02.091544+0000","last_peered":"2023-09-30T22:14:02.091544+0000","last_clean":"2023-09-30T22:14:02.091544+0000","last_became_active":"2023-09-30T22:14:02.082437+0000","last_became_peered":"2023-09-30T22:14:02.082437+0000","last_unstale":"2023-09-30T22:14:02.091544+0000","last_undegraded":"2023-09-30T22:14:02.091544+0000","last_fullsized":"2023-09-30T22:14:02.091544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091649+0000","last_change":"2023-09-30T22:14:02.080886+0000","last_active":"2023-09-30T22:14:02.091649+0000","last_peered":"2023-09-30T22:14:02.091649+0000","last_clean":"2023-09-30T22:14:02.091649+0000","last_became_active":"2023-09-30T22:14:02.078093+0000","last_became_peered":"2023-09-30T22:14:02.078093+0000","last_unstale":"2023-09-30T22:14:02.091649+0000","last_undegraded":"2023-09-30T22:14:02.091649+0000","last_fullsized":"2023-09-30T22:14:02.091649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087971+0000","last_change":"2023-09-30T22:14:02.074967+0000","last_active":"2023-09-30T22:14:02.087971+0000","last_peered":"2023-09-30T22:14:02.087971+0000","last_clean":"2023-09-30T22:14:02.087971+0000","last_became_active":"2023-09-30T22:14:02.074701+0000","last_became_peered":"2023-09-30T22:14:02.074701+0000","last_unstale":"2023-09-30T22:14:02.087971+0000","last_undegraded":"2023-09-30T22:14:02.087971+0000","last_fullsized":"2023-09-30T22:14:02.087971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087513+0000","last_change":"2023-09-30T22:14:02.081443+0000","last_active":"2023-09-30T22:14:02.087513+0000","last_peered":"2023-09-30T22:14:02.087513+0000","last_clean":"2023-09-30T22:14:02.087513+0000","last_became_active":"2023-09-30T22:14:02.080408+0000","last_became_peered":"2023-09-30T22:14:02.080408+0000","last_unstale":"2023-09-30T22:14:02.087513+0000","last_undegraded":"2023-09-30T22:14:02.087513+0000","last_fullsized":"2023-09-30T22:14:02.087513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093729+0000","last_change":"2023-09-30T22:14:02.085310+0000","last_active":"2023-09-30T22:14:02.093729+0000","last_peered":"2023-09-30T22:14:02.093729+0000","last_clean":"2023-09-30T22:14:02.093729+0000","last_became_active":"2023-09-30T22:14:02.084647+0000","last_became_peered":"2023-09-30T22:14:02.084647+0000","last_unstale":"2023-09-30T22:14:02.093729+0000","last_undegraded":"2023-09-30T22:14:02.093729+0000","last_fullsized":"2023-09-30T22:14:02.093729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093827+0000","last_change":"2023-09-30T22:14:02.085988+0000","last_active":"2023-09-30T22:14:02.093827+0000","last_peered":"2023-09-30T22:14:02.093827+0000","last_clean":"2023-09-30T22:14:02.093827+0000","last_became_active":"2023-09-30T22:14:02.083245+0000","last_became_peered":"2023-09-30T22:14:02.083245+0000","last_unstale":"2023-09-30T22:14:02.093827+0000","last_undegraded":"2023-09-30T22:14:02.093827+0000","last_fullsized":"2023-09-30T22:14:02.093827+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091844+0000","last_change":"2023-09-30T22:14:02.085522+0000","last_active":"2023-09-30T22:14:02.091844+0000","last_peered":"2023-09-30T22:14:02.091844+0000","last_clean":"2023-09-30T22:14:02.091844+0000","last_became_active":"2023-09-30T22:14:02.084959+0000","last_became_peered":"2023-09-30T22:14:02.084959+0000","last_unstale":"2023-09-30T22:14:02.091844+0000","last_undegraded":"2023-09-30T22:14:02.091844+0000","last_fullsized":"2023-09-30T22:14:02.091844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.078675+0000","last_change":"2023-09-30T22:14:02.071672+0000","last_active":"2023-09-30T22:14:02.078675+0000","last_peered":"2023-09-30T22:14:02.078675+0000","last_clean":"2023-09-30T22:14:02.078675+0000","last_became_active":"2023-09-30T22:14:02.071361+0000","last_became_peered":"2023-09-30T22:14:02.071361+0000","last_unstale":"2023-09-30T22:14:02.078675+0000","last_undegraded":"2023-09-30T22:14:02.078675+0000","last_fullsized":"2023-09-30T22:14:02.078675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.077649+0000","last_change":"2023-09-30T22:14:02.074689+0000","last_active":"2023-09-30T22:14:02.077649+0000","last_peered":"2023-09-30T22:14:02.077649+0000","last_clean":"2023-09-30T22:14:02.077649+0000","last_became_active":"2023-09-30T22:14:02.074350+0000","last_became_peered":"2023-09-30T22:14:02.074350+0000","last_unstale":"2023-09-30T22:14:02.077649+0000","last_undegraded":"2023-09-30T22:14:02.077649+0000","last_fullsized":"2023-09-30T22:14:02.077649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090695+0000","last_change":"2023-09-30T22:14:02.086959+0000","last_active":"2023-09-30T22:14:02.090695+0000","last_peered":"2023-09-30T22:14:02.090695+0000","last_clean":"2023-09-30T22:14:02.090695+0000","last_became_active":"2023-09-30T22:14:02.086719+0000","last_became_peered":"2023-09-30T22:14:02.086719+0000","last_unstale":"2023-09-30T22:14:02.090695+0000","last_undegraded":"2023-09-30T22:14:02.090695+0000","last_fullsized":"2023-09-30T22:14:02.090695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090392+0000","last_change":"2023-09-30T22:14:02.087347+0000","last_active":"2023-09-30T22:14:02.090392+0000","last_peered":"2023-09-30T22:14:02.090392+0000","last_clean":"2023-09-30T22:14:02.090392+0000","last_became_active":"2023-09-30T22:14:02.086637+0000","last_became_peered":"2023-09-30T22:14:02.086637+0000","last_unstale":"2023-09-30T22:14:02.090392+0000","last_undegraded":"2023-09-30T22:14:02.090392+0000","last_fullsized":"2023-09-30T22:14:02.090392+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.073518+0000","last_change":"2023-09-30T22:14:02.066399+0000","last_active":"2023-09-30T22:14:02.073518+0000","last_peered":"2023-09-30T22:14:02.073518+0000","last_clean":"2023-09-30T22:14:02.073518+0000","last_became_active":"2023-09-30T22:14:02.066052+0000","last_became_peered":"2023-09-30T22:14:02.066052+0000","last_unstale":"2023-09-30T22:14:02.073518+0000","last_undegraded":"2023-09-30T22:14:02.073518+0000","last_fullsized":"2023-09-30T22:14:02.073518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083856+0000","last_change":"2023-09-30T22:14:02.073165+0000","last_active":"2023-09-30T22:14:02.083856+0000","last_peered":"2023-09-30T22:14:02.083856+0000","last_clean":"2023-09-30T22:14:02.083856+0000","last_became_active":"2023-09-30T22:14:02.073000+0000","last_became_peered":"2023-09-30T22:14:02.073000+0000","last_unstale":"2023-09-30T22:14:02.083856+0000","last_undegraded":"2023-09-30T22:14:02.083856+0000","last_fullsized":"2023-09-30T22:14:02.083856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083597+0000","last_change":"2023-09-30T22:14:02.072772+0000","last_active":"2023-09-30T22:14:02.083597+0000","last_peered":"2023-09-30T22:14:02.083597+0000","last_clean":"2023-09-30T22:14:02.083597+0000","last_became_active":"2023-09-30T22:14:02.072569+0000","last_became_peered":"2023-09-30T22:14:02.072569+0000","last_unstale":"2023-09-30T22:14:02.083597+0000","last_undegraded":"2023-09-30T22:14:02.083597+0000","last_fullsized":"2023-09-30T22:14:02.083597+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091862+0000","last_change":"2023-09-30T22:14:02.085679+0000","last_active":"2023-09-30T22:14:02.091862+0000","last_peered":"2023-09-30T22:14:02.091862+0000","last_clean":"2023-09-30T22:14:02.091862+0000","last_became_active":"2023-09-30T22:14:02.085267+0000","last_became_peered":"2023-09-30T22:14:02.085267+0000","last_unstale":"2023-09-30T22:14:02.091862+0000","last_undegraded":"2023-09-30T22:14:02.091862+0000","last_fullsized":"2023-09-30T22:14:02.091862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082225+0000","last_change":"2023-09-30T22:14:02.070008+0000","last_active":"2023-09-30T22:14:02.082225+0000","last_peered":"2023-09-30T22:14:02.082225+0000","last_clean":"2023-09-30T22:14:02.082225+0000","last_became_active":"2023-09-30T22:14:02.069809+0000","last_became_peered":"2023-09-30T22:14:02.069809+0000","last_unstale":"2023-09-30T22:14:02.082225+0000","last_undegraded":"2023-09-30T22:14:02.082225+0000","last_fullsized":"2023-09-30T22:14:02.082225+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090720+0000","last_change":"2023-09-30T22:14:02.087939+0000","last_active":"2023-09-30T22:14:02.090720+0000","last_peered":"2023-09-30T22:14:02.090720+0000","last_clean":"2023-09-30T22:14:02.090720+0000","last_became_active":"2023-09-30T22:14:02.087733+0000","last_became_peered":"2023-09-30T22:14:02.087733+0000","last_unstale":"2023-09-30T22:14:02.090720+0000","last_undegraded":"2023-09-30T22:14:02.090720+0000","last_fullsized":"2023-09-30T22:14:02.090720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093991+0000","last_change":"2023-09-30T22:14:02.090627+0000","last_active":"2023-09-30T22:14:02.093991+0000","last_peered":"2023-09-30T22:14:02.093991+0000","last_clean":"2023-09-30T22:14:02.093991+0000","last_became_active":"2023-09-30T22:14:02.090255+0000","last_became_peered":"2023-09-30T22:14:02.090255+0000","last_unstale":"2023-09-30T22:14:02.093991+0000","last_undegraded":"2023-09-30T22:14:02.093991+0000","last_fullsized":"2023-09-30T22:14:02.093991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.076392+0000","last_change":"2023-09-30T22:14:02.071417+0000","last_active":"2023-09-30T22:14:02.076392+0000","last_peered":"2023-09-30T22:14:02.076392+0000","last_clean":"2023-09-30T22:14:02.076392+0000","last_became_active":"2023-09-30T22:14:02.071046+0000","last_became_peered":"2023-09-30T22:14:02.071046+0000","last_unstale":"2023-09-30T22:14:02.076392+0000","last_undegraded":"2023-09-30T22:14:02.076392+0000","last_fullsized":"2023-09-30T22:14:02.076392+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088095+0000","last_change":"2023-09-30T22:14:02.076131+0000","last_active":"2023-09-30T22:14:02.088095+0000","last_peered":"2023-09-30T22:14:02.088095+0000","last_clean":"2023-09-30T22:14:02.088095+0000","last_became_active":"2023-09-30T22:14:02.075850+0000","last_became_peered":"2023-09-30T22:14:02.075850+0000","last_unstale":"2023-09-30T22:14:02.088095+0000","last_undegraded":"2023-09-30T22:14:02.088095+0000","last_fullsized":"2023-09-30T22:14:02.088095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093707+0000","last_change":"2023-09-30T22:14:02.085210+0000","last_active":"2023-09-30T22:14:02.093707+0000","last_peered":"2023-09-30T22:14:02.093707+0000","last_clean":"2023-09-30T22:14:02.093707+0000","last_became_active":"2023-09-30T22:14:02.084320+0000","last_became_peered":"2023-09-30T22:14:02.084320+0000","last_unstale":"2023-09-30T22:14:02.093707+0000","last_undegraded":"2023-09-30T22:14:02.093707+0000","last_fullsized":"2023-09-30T22:14:02.093707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093806+0000","last_change":"2023-09-30T22:14:02.084642+0000","last_active":"2023-09-30T22:14:02.093806+0000","last_peered":"2023-09-30T22:14:02.093806+0000","last_clean":"2023-09-30T22:14:02.093806+0000","last_became_active":"2023-09-30T22:14:02.084358+0000","last_became_peered":"2023-09-30T22:14:02.084358+0000","last_unstale":"2023-09-30T22:14:02.093806+0000","last_undegraded":"2023-09-30T22:14:02.093806+0000","last_fullsized":"2023-09-30T22:14:02.093806+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.098581+0000","last_change":"2023-09-30T22:14:02.093959+0000","last_active":"2023-09-30T22:14:02.098581+0000","last_peered":"2023-09-30T22:14:02.098581+0000","last_clean":"2023-09-30T22:14:02.098581+0000","last_became_active":"2023-09-30T22:14:02.093149+0000","last_became_peered":"2023-09-30T22:14:02.093149+0000","last_unstale":"2023-09-30T22:14:02.098581+0000","last_undegraded":"2023-09-30T22:14:02.098581+0000","last_fullsized":"2023-09-30T22:14:02.098581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079587+0000","last_change":"2023-09-30T22:14:02.077989+0000","last_active":"2023-09-30T22:14:02.079587+0000","last_peered":"2023-09-30T22:14:02.079587+0000","last_clean":"2023-09-30T22:14:02.079587+0000","last_became_active":"2023-09-30T22:14:02.071401+0000","last_became_peered":"2023-09-30T22:14:02.071401+0000","last_unstale":"2023-09-30T22:14:02.079587+0000","last_undegraded":"2023-09-30T22:14:02.079587+0000","last_fullsized":"2023-09-30T22:14:02.079587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085651+0000","last_change":"2023-09-30T22:14:02.075617+0000","last_active":"2023-09-30T22:14:02.085651+0000","last_peered":"2023-09-30T22:14:02.085651+0000","last_clean":"2023-09-30T22:14:02.085651+0000","last_became_active":"2023-09-30T22:14:02.074584+0000","last_became_peered":"2023-09-30T22:14:02.074584+0000","last_unstale":"2023-09-30T22:14:02.085651+0000","last_undegraded":"2023-09-30T22:14:02.085651+0000","last_fullsized":"2023-09-30T22:14:02.085651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.073438+0000","last_change":"2023-09-30T22:14:01.697398+0000","last_active":"2023-09-30T22:14:02.073438+0000","last_peered":"2023-09-30T22:14:02.073438+0000","last_clean":"2023-09-30T22:14:02.073438+0000","last_became_active":"2023-09-30T22:14:01.694824+0000","last_became_peered":"2023-09-30T22:14:01.694824+0000","last_unstale":"2023-09-30T22:14:02.073438+0000","last_undegraded":"2023-09-30T22:14:02.073438+0000","last_fullsized":"2023-09-30T22:14:02.073438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085672+0000","last_change":"2023-09-30T22:14:02.075771+0000","last_active":"2023-09-30T22:14:02.085672+0000","last_peered":"2023-09-30T22:14:02.085672+0000","last_clean":"2023-09-30T22:14:02.085672+0000","last_became_active":"2023-09-30T22:14:02.074872+0000","last_became_peered":"2023-09-30T22:14:02.074872+0000","last_unstale":"2023-09-30T22:14:02.085672+0000","last_undegraded":"2023-09-30T22:14:02.085672+0000","last_fullsized":"2023-09-30T22:14:02.085672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053010+0000","last_change":"2023-09-30T22:14:01.702319+0000","last_active":"2023-09-30T22:14:02.053010+0000","last_peered":"2023-09-30T22:14:02.053010+0000","last_clean":"2023-09-30T22:14:02.053010+0000","last_became_active":"2023-09-30T22:14:01.702020+0000","last_became_peered":"2023-09-30T22:14:01.702020+0000","last_unstale":"2023-09-30T22:14:02.053010+0000","last_undegraded":"2023-09-30T22:14:02.053010+0000","last_fullsized":"2023-09-30T22:14:02.053010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091650+0000","last_change":"2023-09-30T22:14:02.079713+0000","last_active":"2023-09-30T22:14:02.091650+0000","last_peered":"2023-09-30T22:14:02.091650+0000","last_clean":"2023-09-30T22:14:02.091650+0000","last_became_active":"2023-09-30T22:14:02.079318+0000","last_became_peered":"2023-09-30T22:14:02.079318+0000","last_unstale":"2023-09-30T22:14:02.091650+0000","last_undegraded":"2023-09-30T22:14:02.091650+0000","last_fullsized":"2023-09-30T22:14:02.091650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053665+0000","last_change":"2023-09-30T22:14:01.707097+0000","last_active":"2023-09-30T22:14:02.053665+0000","last_peered":"2023-09-30T22:14:02.053665+0000","last_clean":"2023-09-30T22:14:02.053665+0000","last_became_active":"2023-09-30T22:14:01.706849+0000","last_became_peered":"2023-09-30T22:14:01.706849+0000","last_unstale":"2023-09-30T22:14:02.053665+0000","last_undegraded":"2023-09-30T22:14:02.053665+0000","last_fullsized":"2023-09-30T22:14:02.053665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087679+0000","last_change":"2023-09-30T22:14:02.084631+0000","last_active":"2023-09-30T22:14:02.087679+0000","last_peered":"2023-09-30T22:14:02.087679+0000","last_clean":"2023-09-30T22:14:02.087679+0000","last_became_active":"2023-09-30T22:14:02.084356+0000","last_became_peered":"2023-09-30T22:14:02.084356+0000","last_unstale":"2023-09-30T22:14:02.087679+0000","last_undegraded":"2023-09-30T22:14:02.087679+0000","last_fullsized":"2023-09-30T22:14:02.087679+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057794+0000","last_change":"2023-09-30T22:14:01.698007+0000","last_active":"2023-09-30T22:14:02.057794+0000","last_peered":"2023-09-30T22:14:02.057794+0000","last_clean":"2023-09-30T22:14:02.057794+0000","last_became_active":"2023-09-30T22:14:01.697731+0000","last_became_peered":"2023-09-30T22:14:01.697731+0000","last_unstale":"2023-09-30T22:14:02.057794+0000","last_undegraded":"2023-09-30T22:14:02.057794+0000","last_fullsized":"2023-09-30T22:14:02.057794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087841+0000","last_change":"2023-09-30T22:14:02.082853+0000","last_active":"2023-09-30T22:14:02.087841+0000","last_peered":"2023-09-30T22:14:02.087841+0000","last_clean":"2023-09-30T22:14:02.087841+0000","last_became_active":"2023-09-30T22:14:02.082597+0000","last_became_peered":"2023-09-30T22:14:02.082597+0000","last_unstale":"2023-09-30T22:14:02.087841+0000","last_undegraded":"2023-09-30T22:14:02.087841+0000","last_fullsized":"2023-09-30T22:14:02.087841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056502+0000","last_change":"2023-09-30T22:14:01.700657+0000","last_active":"2023-09-30T22:14:02.056502+0000","last_peered":"2023-09-30T22:14:02.056502+0000","last_clean":"2023-09-30T22:14:02.056502+0000","last_became_active":"2023-09-30T22:14:01.699633+0000","last_became_peered":"2023-09-30T22:14:01.699633+0000","last_unstale":"2023-09-30T22:14:02.056502+0000","last_undegraded":"2023-09-30T22:14:02.056502+0000","last_fullsized":"2023-09-30T22:14:02.056502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093773+0000","last_change":"2023-09-30T22:14:02.085416+0000","last_active":"2023-09-30T22:14:02.093773+0000","last_peered":"2023-09-30T22:14:02.093773+0000","last_clean":"2023-09-30T22:14:02.093773+0000","last_became_active":"2023-09-30T22:14:02.084970+0000","last_became_peered":"2023-09-30T22:14:02.084970+0000","last_unstale":"2023-09-30T22:14:02.093773+0000","last_undegraded":"2023-09-30T22:14:02.093773+0000","last_fullsized":"2023-09-30T22:14:02.093773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083245+0000","last_change":"2023-09-30T22:14:01.701075+0000","last_active":"2023-09-30T22:14:02.083245+0000","last_peered":"2023-09-30T22:14:02.083245+0000","last_clean":"2023-09-30T22:14:02.083245+0000","last_became_active":"2023-09-30T22:14:01.700806+0000","last_became_peered":"2023-09-30T22:14:01.700806+0000","last_unstale":"2023-09-30T22:14:02.083245+0000","last_undegraded":"2023-09-30T22:14:02.083245+0000","last_fullsized":"2023-09-30T22:14:02.083245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.077417+0000","last_change":"2023-09-30T22:14:02.074078+0000","last_active":"2023-09-30T22:14:02.077417+0000","last_peered":"2023-09-30T22:14:02.077417+0000","last_clean":"2023-09-30T22:14:02.077417+0000","last_became_active":"2023-09-30T22:14:02.073804+0000","last_became_peered":"2023-09-30T22:14:02.073804+0000","last_unstale":"2023-09-30T22:14:02.077417+0000","last_undegraded":"2023-09-30T22:14:02.077417+0000","last_fullsized":"2023-09-30T22:14:02.077417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.096081+0000","last_change":"2023-09-30T22:14:01.711172+0000","last_active":"2023-09-30T22:14:02.096081+0000","last_peered":"2023-09-30T22:14:02.096081+0000","last_clean":"2023-09-30T22:14:02.096081+0000","last_became_active":"2023-09-30T22:14:01.710926+0000","last_became_peered":"2023-09-30T22:14:01.710926+0000","last_unstale":"2023-09-30T22:14:02.096081+0000","last_undegraded":"2023-09-30T22:14:02.096081+0000","last_fullsized":"2023-09-30T22:14:02.096081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090366+0000","last_change":"2023-09-30T22:14:02.087193+0000","last_active":"2023-09-30T22:14:02.090366+0000","last_peered":"2023-09-30T22:14:02.090366+0000","last_clean":"2023-09-30T22:14:02.090366+0000","last_became_active":"2023-09-30T22:14:02.086273+0000","last_became_peered":"2023-09-30T22:14:02.086273+0000","last_unstale":"2023-09-30T22:14:02.090366+0000","last_undegraded":"2023-09-30T22:14:02.090366+0000","last_fullsized":"2023-09-30T22:14:02.090366+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082985+0000","last_change":"2023-09-30T22:14:01.706560+0000","last_active":"2023-09-30T22:14:02.082985+0000","last_peered":"2023-09-30T22:14:02.082985+0000","last_clean":"2023-09-30T22:14:02.082985+0000","last_became_active":"2023-09-30T22:14:01.706330+0000","last_became_peered":"2023-09-30T22:14:01.706330+0000","last_unstale":"2023-09-30T22:14:02.082985+0000","last_undegraded":"2023-09-30T22:14:02.082985+0000","last_fullsized":"2023-09-30T22:14:02.082985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087699+0000","last_change":"2023-09-30T22:14:02.084139+0000","last_active":"2023-09-30T22:14:02.087699+0000","last_peered":"2023-09-30T22:14:02.087699+0000","last_clean":"2023-09-30T22:14:02.087699+0000","last_became_active":"2023-09-30T22:14:02.080986+0000","last_became_peered":"2023-09-30T22:14:02.080986+0000","last_unstale":"2023-09-30T22:14:02.087699+0000","last_undegraded":"2023-09-30T22:14:02.087699+0000","last_fullsized":"2023-09-30T22:14:02.087699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.055029+0000","last_change":"2023-09-30T22:14:01.698141+0000","last_active":"2023-09-30T22:14:02.055029+0000","last_peered":"2023-09-30T22:14:02.055029+0000","last_clean":"2023-09-30T22:14:02.055029+0000","last_became_active":"2023-09-30T22:14:01.697885+0000","last_became_peered":"2023-09-30T22:14:01.697885+0000","last_unstale":"2023-09-30T22:14:02.055029+0000","last_undegraded":"2023-09-30T22:14:02.055029+0000","last_fullsized":"2023-09-30T22:14:02.055029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085690+0000","last_change":"2023-09-30T22:14:02.075925+0000","last_active":"2023-09-30T22:14:02.085690+0000","last_peered":"2023-09-30T22:14:02.085690+0000","last_clean":"2023-09-30T22:14:02.085690+0000","last_became_active":"2023-09-30T22:14:02.075137+0000","last_became_peered":"2023-09-30T22:14:02.075137+0000","last_unstale":"2023-09-30T22:14:02.085690+0000","last_undegraded":"2023-09-30T22:14:02.085690+0000","last_fullsized":"2023-09-30T22:14:02.085690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.064991+0000","last_change":"2023-09-30T22:14:01.712299+0000","last_active":"2023-09-30T22:14:02.064991+0000","last_peered":"2023-09-30T22:14:02.064991+0000","last_clean":"2023-09-30T22:14:02.064991+0000","last_became_active":"2023-09-30T22:14:01.711791+0000","last_became_peered":"2023-09-30T22:14:01.711791+0000","last_unstale":"2023-09-30T22:14:02.064991+0000","last_undegraded":"2023-09-30T22:14:02.064991+0000","last_fullsized":"2023-09-30T22:14:02.064991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091669+0000","last_change":"2023-09-30T22:14:02.084033+0000","last_active":"2023-09-30T22:14:02.091669+0000","last_peered":"2023-09-30T22:14:02.091669+0000","last_clean":"2023-09-30T22:14:02.091669+0000","last_became_active":"2023-09-30T22:14:02.083764+0000","last_became_peered":"2023-09-30T22:14:02.083764+0000","last_unstale":"2023-09-30T22:14:02.091669+0000","last_undegraded":"2023-09-30T22:14:02.091669+0000","last_fullsized":"2023-09-30T22:14:02.091669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.092132+0000","last_change":"2023-09-30T22:14:01.706777+0000","last_active":"2023-09-30T22:14:02.092132+0000","last_peered":"2023-09-30T22:14:02.092132+0000","last_clean":"2023-09-30T22:14:02.092132+0000","last_became_active":"2023-09-30T22:14:01.706479+0000","last_became_peered":"2023-09-30T22:14:01.706479+0000","last_unstale":"2023-09-30T22:14:02.092132+0000","last_undegraded":"2023-09-30T22:14:02.092132+0000","last_fullsized":"2023-09-30T22:14:02.092132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087631+0000","last_change":"2023-09-30T22:14:02.084399+0000","last_active":"2023-09-30T22:14:02.087631+0000","last_peered":"2023-09-30T22:14:02.087631+0000","last_clean":"2023-09-30T22:14:02.087631+0000","last_became_active":"2023-09-30T22:14:02.084137+0000","last_became_peered":"2023-09-30T22:14:02.084137+0000","last_unstale":"2023-09-30T22:14:02.087631+0000","last_undegraded":"2023-09-30T22:14:02.087631+0000","last_fullsized":"2023-09-30T22:14:02.087631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093442+0000","last_change":"2023-09-30T22:14:01.702685+0000","last_active":"2023-09-30T22:14:02.093442+0000","last_peered":"2023-09-30T22:14:02.093442+0000","last_clean":"2023-09-30T22:14:02.093442+0000","last_became_active":"2023-09-30T22:14:01.702083+0000","last_became_peered":"2023-09-30T22:14:01.702083+0000","last_unstale":"2023-09-30T22:14:02.093442+0000","last_undegraded":"2023-09-30T22:14:02.093442+0000","last_fullsized":"2023-09-30T22:14:02.093442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087823+0000","last_change":"2023-09-30T22:14:02.082069+0000","last_active":"2023-09-30T22:14:02.087823+0000","last_peered":"2023-09-30T22:14:02.087823+0000","last_clean":"2023-09-30T22:14:02.087823+0000","last_became_active":"2023-09-30T22:14:02.081702+0000","last_became_peered":"2023-09-30T22:14:02.081702+0000","last_unstale":"2023-09-30T22:14:02.087823+0000","last_undegraded":"2023-09-30T22:14:02.087823+0000","last_fullsized":"2023-09-30T22:14:02.087823+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.069430+0000","last_change":"2023-09-30T22:14:01.706748+0000","last_active":"2023-09-30T22:14:02.069430+0000","last_peered":"2023-09-30T22:14:02.069430+0000","last_clean":"2023-09-30T22:14:02.069430+0000","last_became_active":"2023-09-30T22:14:01.706472+0000","last_became_peered":"2023-09-30T22:14:01.706472+0000","last_unstale":"2023-09-30T22:14:02.069430+0000","last_undegraded":"2023-09-30T22:14:02.069430+0000","last_fullsized":"2023-09-30T22:14:02.069430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.076358+0000","last_change":"2023-09-30T22:14:02.070587+0000","last_active":"2023-09-30T22:14:02.076358+0000","last_peered":"2023-09-30T22:14:02.076358+0000","last_clean":"2023-09-30T22:14:02.076358+0000","last_became_active":"2023-09-30T22:14:02.070259+0000","last_became_peered":"2023-09-30T22:14:02.070259+0000","last_unstale":"2023-09-30T22:14:02.076358+0000","last_undegraded":"2023-09-30T22:14:02.076358+0000","last_fullsized":"2023-09-30T22:14:02.076358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080959+0000","last_change":"2023-09-30T22:14:01.704603+0000","last_active":"2023-09-30T22:14:02.080959+0000","last_peered":"2023-09-30T22:14:02.080959+0000","last_clean":"2023-09-30T22:14:02.080959+0000","last_became_active":"2023-09-30T22:14:01.704006+0000","last_became_peered":"2023-09-30T22:14:01.704006+0000","last_unstale":"2023-09-30T22:14:02.080959+0000","last_undegraded":"2023-09-30T22:14:02.080959+0000","last_fullsized":"2023-09-30T22:14:02.080959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080942+0000","last_change":"2023-09-30T22:14:02.073527+0000","last_active":"2023-09-30T22:14:02.080942+0000","last_peered":"2023-09-30T22:14:02.080942+0000","last_clean":"2023-09-30T22:14:02.080942+0000","last_became_active":"2023-09-30T22:14:02.072950+0000","last_became_peered":"2023-09-30T22:14:02.072950+0000","last_unstale":"2023-09-30T22:14:02.080942+0000","last_undegraded":"2023-09-30T22:14:02.080942+0000","last_fullsized":"2023-09-30T22:14:02.080942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057420+0000","last_change":"2023-09-30T22:14:01.701736+0000","last_active":"2023-09-30T22:14:02.057420+0000","last_peered":"2023-09-30T22:14:02.057420+0000","last_clean":"2023-09-30T22:14:02.057420+0000","last_became_active":"2023-09-30T22:14:01.701471+0000","last_became_peered":"2023-09-30T22:14:01.701471+0000","last_unstale":"2023-09-30T22:14:02.057420+0000","last_undegraded":"2023-09-30T22:14:02.057420+0000","last_fullsized":"2023-09-30T22:14:02.057420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082228+0000","last_change":"2023-09-30T22:14:02.072624+0000","last_active":"2023-09-30T22:14:02.082228+0000","last_peered":"2023-09-30T22:14:02.082228+0000","last_clean":"2023-09-30T22:14:02.082228+0000","last_became_active":"2023-09-30T22:14:02.072363+0000","last_became_peered":"2023-09-30T22:14:02.072363+0000","last_unstale":"2023-09-30T22:14:02.082228+0000","last_undegraded":"2023-09-30T22:14:02.082228+0000","last_fullsized":"2023-09-30T22:14:02.082228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082837+0000","last_change":"2023-09-30T22:14:01.709398+0000","last_active":"2023-09-30T22:14:02.082837+0000","last_peered":"2023-09-30T22:14:02.082837+0000","last_clean":"2023-09-30T22:14:02.082837+0000","last_became_active":"2023-09-30T22:14:01.709110+0000","last_became_peered":"2023-09-30T22:14:01.709110+0000","last_unstale":"2023-09-30T22:14:02.082837+0000","last_undegraded":"2023-09-30T22:14:02.082837+0000","last_fullsized":"2023-09-30T22:14:02.082837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087990+0000","last_change":"2023-09-30T22:14:02.074887+0000","last_active":"2023-09-30T22:14:02.087990+0000","last_peered":"2023-09-30T22:14:02.087990+0000","last_clean":"2023-09-30T22:14:02.087990+0000","last_became_active":"2023-09-30T22:14:02.074631+0000","last_became_peered":"2023-09-30T22:14:02.074631+0000","last_unstale":"2023-09-30T22:14:02.087990+0000","last_undegraded":"2023-09-30T22:14:02.087990+0000","last_fullsized":"2023-09-30T22:14:02.087990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054479+0000","last_change":"2023-09-30T22:14:01.707879+0000","last_active":"2023-09-30T22:14:02.054479+0000","last_peered":"2023-09-30T22:14:02.054479+0000","last_clean":"2023-09-30T22:14:02.054479+0000","last_became_active":"2023-09-30T22:14:01.707594+0000","last_became_peered":"2023-09-30T22:14:01.707594+0000","last_unstale":"2023-09-30T22:14:02.054479+0000","last_undegraded":"2023-09-30T22:14:02.054479+0000","last_fullsized":"2023-09-30T22:14:02.054479+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085707+0000","last_change":"2023-09-30T22:14:02.076079+0000","last_active":"2023-09-30T22:14:02.085707+0000","last_peered":"2023-09-30T22:14:02.085707+0000","last_clean":"2023-09-30T22:14:02.085707+0000","last_became_active":"2023-09-30T22:14:02.075408+0000","last_became_peered":"2023-09-30T22:14:02.075408+0000","last_unstale":"2023-09-30T22:14:02.085707+0000","last_undegraded":"2023-09-30T22:14:02.085707+0000","last_fullsized":"2023-09-30T22:14:02.085707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.065836+0000","last_change":"2023-09-30T22:14:01.711605+0000","last_active":"2023-09-30T22:14:02.065836+0000","last_peered":"2023-09-30T22:14:02.065836+0000","last_clean":"2023-09-30T22:14:02.065836+0000","last_became_active":"2023-09-30T22:14:01.711375+0000","last_became_peered":"2023-09-30T22:14:01.711375+0000","last_unstale":"2023-09-30T22:14:02.065836+0000","last_undegraded":"2023-09-30T22:14:02.065836+0000","last_fullsized":"2023-09-30T22:14:02.065836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090979+0000","last_change":"2023-09-30T22:14:02.088858+0000","last_active":"2023-09-30T22:14:02.090979+0000","last_peered":"2023-09-30T22:14:02.090979+0000","last_clean":"2023-09-30T22:14:02.090979+0000","last_became_active":"2023-09-30T22:14:02.088583+0000","last_became_peered":"2023-09-30T22:14:02.088583+0000","last_unstale":"2023-09-30T22:14:02.090979+0000","last_undegraded":"2023-09-30T22:14:02.090979+0000","last_fullsized":"2023-09-30T22:14:02.090979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053157+0000","last_change":"2023-09-30T22:14:01.699847+0000","last_active":"2023-09-30T22:14:02.053157+0000","last_peered":"2023-09-30T22:14:02.053157+0000","last_clean":"2023-09-30T22:14:02.053157+0000","last_became_active":"2023-09-30T22:14:01.699587+0000","last_became_peered":"2023-09-30T22:14:01.699587+0000","last_unstale":"2023-09-30T22:14:02.053157+0000","last_undegraded":"2023-09-30T22:14:02.053157+0000","last_fullsized":"2023-09-30T22:14:02.053157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.098596+0000","last_change":"2023-09-30T22:14:02.094005+0000","last_active":"2023-09-30T22:14:02.098596+0000","last_peered":"2023-09-30T22:14:02.098596+0000","last_clean":"2023-09-30T22:14:02.098596+0000","last_became_active":"2023-09-30T22:14:02.093632+0000","last_became_peered":"2023-09-30T22:14:02.093632+0000","last_unstale":"2023-09-30T22:14:02.098596+0000","last_undegraded":"2023-09-30T22:14:02.098596+0000","last_fullsized":"2023-09-30T22:14:02.098596+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.052854+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:14:02.052854+0000","last_peered":"2023-09-30T22:14:02.052854+0000","last_clean":"2023-09-30T22:14:02.052854+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:14:02.052854+0000","last_undegraded":"2023-09-30T22:14:02.052854+0000","last_fullsized":"2023-09-30T22:14:02.052854+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056318+0000","last_change":"2023-09-30T22:14:01.702822+0000","last_active":"2023-09-30T22:14:02.056318+0000","last_peered":"2023-09-30T22:14:02.056318+0000","last_clean":"2023-09-30T22:14:02.056318+0000","last_became_active":"2023-09-30T22:14:01.702530+0000","last_became_peered":"2023-09-30T22:14:01.702530+0000","last_unstale":"2023-09-30T22:14:02.056318+0000","last_undegraded":"2023-09-30T22:14:02.056318+0000","last_fullsized":"2023-09-30T22:14:02.056318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093176+0000","last_change":"2023-09-30T22:14:02.084698+0000","last_active":"2023-09-30T22:14:02.093176+0000","last_peered":"2023-09-30T22:14:02.093176+0000","last_clean":"2023-09-30T22:14:02.093176+0000","last_became_active":"2023-09-30T22:14:02.084413+0000","last_became_peered":"2023-09-30T22:14:02.084413+0000","last_unstale":"2023-09-30T22:14:02.093176+0000","last_undegraded":"2023-09-30T22:14:02.093176+0000","last_fullsized":"2023-09-30T22:14:02.093176+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054630+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:14:02.054630+0000","last_peered":"2023-09-30T22:14:02.054630+0000","last_clean":"2023-09-30T22:14:02.054630+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:14:02.054630+0000","last_undegraded":"2023-09-30T22:14:02.054630+0000","last_fullsized":"2023-09-30T22:14:02.054630+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057152+0000","last_change":"2023-09-30T22:14:01.704198+0000","last_active":"2023-09-30T22:14:02.057152+0000","last_peered":"2023-09-30T22:14:02.057152+0000","last_clean":"2023-09-30T22:14:02.057152+0000","last_became_active":"2023-09-30T22:14:01.701691+0000","last_became_peered":"2023-09-30T22:14:01.701691+0000","last_unstale":"2023-09-30T22:14:02.057152+0000","last_undegraded":"2023-09-30T22:14:02.057152+0000","last_fullsized":"2023-09-30T22:14:02.057152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079574+0000","last_change":"2023-09-30T22:14:02.076717+0000","last_active":"2023-09-30T22:14:02.079574+0000","last_peered":"2023-09-30T22:14:02.079574+0000","last_clean":"2023-09-30T22:14:02.079574+0000","last_became_active":"2023-09-30T22:14:02.076119+0000","last_became_peered":"2023-09-30T22:14:02.076119+0000","last_unstale":"2023-09-30T22:14:02.079574+0000","last_undegraded":"2023-09-30T22:14:02.079574+0000","last_fullsized":"2023-09-30T22:14:02.079574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056513+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:14:02.056513+0000","last_peered":"2023-09-30T22:14:02.056513+0000","last_clean":"2023-09-30T22:14:02.056513+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:14:02.056513+0000","last_undegraded":"2023-09-30T22:14:02.056513+0000","last_fullsized":"2023-09-30T22:14:02.056513+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062825+0000","last_change":"2023-09-30T22:14:01.701431+0000","last_active":"2023-09-30T22:14:02.062825+0000","last_peered":"2023-09-30T22:14:02.062825+0000","last_clean":"2023-09-30T22:14:02.062825+0000","last_became_active":"2023-09-30T22:14:01.698771+0000","last_became_peered":"2023-09-30T22:14:01.698771+0000","last_unstale":"2023-09-30T22:14:02.062825+0000","last_undegraded":"2023-09-30T22:14:02.062825+0000","last_fullsized":"2023-09-30T22:14:02.062825+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.094124+0000","last_change":"2023-09-30T22:14:02.085435+0000","last_active":"2023-09-30T22:14:02.094124+0000","last_peered":"2023-09-30T22:14:02.094124+0000","last_clean":"2023-09-30T22:14:02.094124+0000","last_became_active":"2023-09-30T22:14:02.085124+0000","last_became_peered":"2023-09-30T22:14:02.085124+0000","last_unstale":"2023-09-30T22:14:02.094124+0000","last_undegraded":"2023-09-30T22:14:02.094124+0000","last_fullsized":"2023-09-30T22:14:02.094124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062504+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:14:02.062504+0000","last_peered":"2023-09-30T22:14:02.062504+0000","last_clean":"2023-09-30T22:14:02.062504+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:14:02.062504+0000","last_undegraded":"2023-09-30T22:14:02.062504+0000","last_fullsized":"2023-09-30T22:14:02.062504+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062714+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:14:02.062714+0000","last_peered":"2023-09-30T22:14:02.062714+0000","last_clean":"2023-09-30T22:14:02.062714+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:14:02.062714+0000","last_undegraded":"2023-09-30T22:14:02.062714+0000","last_fullsized":"2023-09-30T22:14:02.062714+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054372+0000","last_change":"2023-09-30T22:14:01.712014+0000","last_active":"2023-09-30T22:14:02.054372+0000","last_peered":"2023-09-30T22:14:02.054372+0000","last_clean":"2023-09-30T22:14:02.054372+0000","last_became_active":"2023-09-30T22:14:01.711770+0000","last_became_peered":"2023-09-30T22:14:01.711770+0000","last_unstale":"2023-09-30T22:14:02.054372+0000","last_undegraded":"2023-09-30T22:14:02.054372+0000","last_fullsized":"2023-09-30T22:14:02.054372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087711+0000","last_change":"2023-09-30T22:14:02.080800+0000","last_active":"2023-09-30T22:14:02.087711+0000","last_peered":"2023-09-30T22:14:02.087711+0000","last_clean":"2023-09-30T22:14:02.087711+0000","last_became_active":"2023-09-30T22:14:02.080440+0000","last_became_peered":"2023-09-30T22:14:02.080440+0000","last_unstale":"2023-09-30T22:14:02.087711+0000","last_undegraded":"2023-09-30T22:14:02.087711+0000","last_fullsized":"2023-09-30T22:14:02.087711+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054814+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:14:02.054814+0000","last_peered":"2023-09-30T22:14:02.054814+0000","last_clean":"2023-09-30T22:14:02.054814+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:14:02.054814+0000","last_undegraded":"2023-09-30T22:14:02.054814+0000","last_fullsized":"2023-09-30T22:14:02.054814+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.055456+0000","last_change":"2023-09-30T22:14:01.694407+0000","last_active":"2023-09-30T22:14:02.055456+0000","last_peered":"2023-09-30T22:14:02.055456+0000","last_clean":"2023-09-30T22:14:02.055456+0000","last_became_active":"2023-09-30T22:14:01.694022+0000","last_became_peered":"2023-09-30T22:14:01.694022+0000","last_unstale":"2023-09-30T22:14:02.055456+0000","last_undegraded":"2023-09-30T22:14:02.055456+0000","last_fullsized":"2023-09-30T22:14:02.055456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091628+0000","last_change":"2023-09-30T22:14:02.079540+0000","last_active":"2023-09-30T22:14:02.091628+0000","last_peered":"2023-09-30T22:14:02.091628+0000","last_clean":"2023-09-30T22:14:02.091628+0000","last_became_active":"2023-09-30T22:14:02.079016+0000","last_became_peered":"2023-09-30T22:14:02.079016+0000","last_unstale":"2023-09-30T22:14:02.091628+0000","last_undegraded":"2023-09-30T22:14:02.091628+0000","last_fullsized":"2023-09-30T22:14:02.091628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.059163+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:14:02.059163+0000","last_peered":"2023-09-30T22:14:02.059163+0000","last_clean":"2023-09-30T22:14:02.059163+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:14:02.059163+0000","last_undegraded":"2023-09-30T22:14:02.059163+0000","last_fullsized":"2023-09-30T22:14:02.059163+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053400+0000","last_change":"2023-09-30T22:14:01.695717+0000","last_active":"2023-09-30T22:14:02.053400+0000","last_peered":"2023-09-30T22:14:02.053400+0000","last_clean":"2023-09-30T22:14:02.053400+0000","last_became_active":"2023-09-30T22:14:01.694990+0000","last_became_peered":"2023-09-30T22:14:01.694990+0000","last_unstale":"2023-09-30T22:14:02.053400+0000","last_undegraded":"2023-09-30T22:14:02.053400+0000","last_fullsized":"2023-09-30T22:14:02.053400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080394+0000","last_change":"2023-09-30T22:14:02.068852+0000","last_active":"2023-09-30T22:14:02.080394+0000","last_peered":"2023-09-30T22:14:02.080394+0000","last_clean":"2023-09-30T22:14:02.080394+0000","last_became_active":"2023-09-30T22:14:02.068531+0000","last_became_peered":"2023-09-30T22:14:02.068531+0000","last_unstale":"2023-09-30T22:14:02.080394+0000","last_undegraded":"2023-09-30T22:14:02.080394+0000","last_fullsized":"2023-09-30T22:14:02.080394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.084595+0000","last_change":"2023-09-30T22:14:01.700959+0000","last_active":"2023-09-30T22:14:02.084595+0000","last_peered":"2023-09-30T22:14:02.084595+0000","last_clean":"2023-09-30T22:14:02.084595+0000","last_became_active":"2023-09-30T22:14:01.700568+0000","last_became_peered":"2023-09-30T22:14:01.700568+0000","last_unstale":"2023-09-30T22:14:02.084595+0000","last_undegraded":"2023-09-30T22:14:02.084595+0000","last_fullsized":"2023-09-30T22:14:02.084595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.048709+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:14:02.048709+0000","last_peered":"2023-09-30T22:14:02.048709+0000","last_clean":"2023-09-30T22:14:02.048709+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:14:02.048709+0000","last_undegraded":"2023-09-30T22:14:02.048709+0000","last_fullsized":"2023-09-30T22:14:02.048709+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087805+0000","last_change":"2023-09-30T22:14:02.081943+0000","last_active":"2023-09-30T22:14:02.087805+0000","last_peered":"2023-09-30T22:14:02.087805+0000","last_clean":"2023-09-30T22:14:02.087805+0000","last_became_active":"2023-09-30T22:14:02.080899+0000","last_became_peered":"2023-09-30T22:14:02.080899+0000","last_unstale":"2023-09-30T22:14:02.087805+0000","last_undegraded":"2023-09-30T22:14:02.087805+0000","last_fullsized":"2023-09-30T22:14:02.087805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054595+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:14:02.054595+0000","last_peered":"2023-09-30T22:14:02.054595+0000","last_clean":"2023-09-30T22:14:02.054595+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:14:02.054595+0000","last_undegraded":"2023-09-30T22:14:02.054595+0000","last_fullsized":"2023-09-30T22:14:02.054595+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057898+0000","last_change":"2023-09-30T22:14:01.706704+0000","last_active":"2023-09-30T22:14:02.057898+0000","last_peered":"2023-09-30T22:14:02.057898+0000","last_clean":"2023-09-30T22:14:02.057898+0000","last_became_active":"2023-09-30T22:14:01.706387+0000","last_became_peered":"2023-09-30T22:14:01.706387+0000","last_unstale":"2023-09-30T22:14:02.057898+0000","last_undegraded":"2023-09-30T22:14:02.057898+0000","last_fullsized":"2023-09-30T22:14:02.057898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090754+0000","last_change":"2023-09-30T22:14:02.088343+0000","last_active":"2023-09-30T22:14:02.090754+0000","last_peered":"2023-09-30T22:14:02.090754+0000","last_clean":"2023-09-30T22:14:02.090754+0000","last_became_active":"2023-09-30T22:14:02.088142+0000","last_became_peered":"2023-09-30T22:14:02.088142+0000","last_unstale":"2023-09-30T22:14:02.090754+0000","last_undegraded":"2023-09-30T22:14:02.090754+0000","last_fullsized":"2023-09-30T22:14:02.090754+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083333+0000","last_change":"2023-09-30T22:14:01.702235+0000","last_active":"2023-09-30T22:14:02.083333+0000","last_peered":"2023-09-30T22:14:02.083333+0000","last_clean":"2023-09-30T22:14:02.083333+0000","last_became_active":"2023-09-30T22:14:01.701972+0000","last_became_peered":"2023-09-30T22:14:01.701972+0000","last_unstale":"2023-09-30T22:14:02.083333+0000","last_undegraded":"2023-09-30T22:14:02.083333+0000","last_fullsized":"2023-09-30T22:14:02.083333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080923+0000","last_change":"2023-09-30T22:14:02.078694+0000","last_active":"2023-09-30T22:14:02.080923+0000","last_peered":"2023-09-30T22:14:02.080923+0000","last_clean":"2023-09-30T22:14:02.080923+0000","last_became_active":"2023-09-30T22:14:02.078435+0000","last_became_peered":"2023-09-30T22:14:02.078435+0000","last_unstale":"2023-09-30T22:14:02.080923+0000","last_undegraded":"2023-09-30T22:14:02.080923+0000","last_fullsized":"2023-09-30T22:14:02.080923+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090913+0000","last_change":"2023-09-30T22:14:01.709428+0000","last_active":"2023-09-30T22:14:02.090913+0000","last_peered":"2023-09-30T22:14:02.090913+0000","last_clean":"2023-09-30T22:14:02.090913+0000","last_became_active":"2023-09-30T22:14:01.709089+0000","last_became_peered":"2023-09-30T22:14:01.709089+0000","last_unstale":"2023-09-30T22:14:02.090913+0000","last_undegraded":"2023-09-30T22:14:02.090913+0000","last_fullsized":"2023-09-30T22:14:02.090913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090409+0000","last_change":"2023-09-30T22:14:02.087499+0000","last_active":"2023-09-30T22:14:02.090409+0000","last_peered":"2023-09-30T22:14:02.090409+0000","last_clean":"2023-09-30T22:14:02.090409+0000","last_became_active":"2023-09-30T22:14:02.086941+0000","last_became_peered":"2023-09-30T22:14:02.086941+0000","last_unstale":"2023-09-30T22:14:02.090409+0000","last_undegraded":"2023-09-30T22:14:02.090409+0000","last_fullsized":"2023-09-30T22:14:02.090409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083525+0000","last_change":"2023-09-30T22:14:01.709677+0000","last_active":"2023-09-30T22:14:02.083525+0000","last_peered":"2023-09-30T22:14:02.083525+0000","last_clean":"2023-09-30T22:14:02.083525+0000","last_became_active":"2023-09-30T22:14:01.709115+0000","last_became_peered":"2023-09-30T22:14:01.709115+0000","last_unstale":"2023-09-30T22:14:02.083525+0000","last_undegraded":"2023-09-30T22:14:02.083525+0000","last_fullsized":"2023-09-30T22:14:02.083525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080964+0000","last_change":"2023-09-30T22:14:02.073683+0000","last_active":"2023-09-30T22:14:02.080964+0000","last_peered":"2023-09-30T22:14:02.080964+0000","last_clean":"2023-09-30T22:14:02.080964+0000","last_became_active":"2023-09-30T22:14:02.073269+0000","last_became_peered":"2023-09-30T22:14:02.073269+0000","last_unstale":"2023-09-30T22:14:02.080964+0000","last_undegraded":"2023-09-30T22:14:02.080964+0000","last_fullsized":"2023-09-30T22:14:02.080964+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083125+0000","last_change":"2023-09-30T22:14:01.711766+0000","last_active":"2023-09-30T22:14:02.083125+0000","last_peered":"2023-09-30T22:14:02.083125+0000","last_clean":"2023-09-30T22:14:02.083125+0000","last_became_active":"2023-09-30T22:14:01.711524+0000","last_became_peered":"2023-09-30T22:14:01.711524+0000","last_unstale":"2023-09-30T22:14:02.083125+0000","last_undegraded":"2023-09-30T22:14:02.083125+0000","last_fullsized":"2023-09-30T22:14:02.083125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088038+0000","last_change":"2023-09-30T22:14:02.082106+0000","last_active":"2023-09-30T22:14:02.088038+0000","last_peered":"2023-09-30T22:14:02.088038+0000","last_clean":"2023-09-30T22:14:02.088038+0000","last_became_active":"2023-09-30T22:14:02.081789+0000","last_became_peered":"2023-09-30T22:14:02.081789+0000","last_unstale":"2023-09-30T22:14:02.088038+0000","last_undegraded":"2023-09-30T22:14:02.088038+0000","last_fullsized":"2023-09-30T22:14:02.088038+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091206+0000","last_change":"2023-09-30T22:14:01.702868+0000","last_active":"2023-09-30T22:14:02.091206+0000","last_peered":"2023-09-30T22:14:02.091206+0000","last_clean":"2023-09-30T22:14:02.091206+0000","last_became_active":"2023-09-30T22:14:01.702431+0000","last_became_peered":"2023-09-30T22:14:01.702431+0000","last_unstale":"2023-09-30T22:14:02.091206+0000","last_undegraded":"2023-09-30T22:14:02.091206+0000","last_fullsized":"2023-09-30T22:14:02.091206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.084635+0000","last_change":"2023-09-30T22:14:02.081561+0000","last_active":"2023-09-30T22:14:02.084635+0000","last_peered":"2023-09-30T22:14:02.084635+0000","last_clean":"2023-09-30T22:14:02.084635+0000","last_became_active":"2023-09-30T22:14:02.081199+0000","last_became_peered":"2023-09-30T22:14:02.081199+0000","last_unstale":"2023-09-30T22:14:02.084635+0000","last_undegraded":"2023-09-30T22:14:02.084635+0000","last_fullsized":"2023-09-30T22:14:02.084635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.086077+0000","last_change":"2023-09-30T22:14:02.079397+0000","last_active":"2023-09-30T22:14:02.086077+0000","last_peered":"2023-09-30T22:14:02.086077+0000","last_clean":"2023-09-30T22:14:02.086077+0000","last_became_active":"2023-09-30T22:14:02.079036+0000","last_became_peered":"2023-09-30T22:14:02.079036+0000","last_unstale":"2023-09-30T22:14:02.086077+0000","last_undegraded":"2023-09-30T22:14:02.086077+0000","last_fullsized":"2023-09-30T22:14:02.086077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088018+0000","last_change":"2023-09-30T22:14:02.081974+0000","last_active":"2023-09-30T22:14:02.088018+0000","last_peered":"2023-09-30T22:14:02.088018+0000","last_clean":"2023-09-30T22:14:02.088018+0000","last_became_active":"2023-09-30T22:14:02.081491+0000","last_became_peered":"2023-09-30T22:14:02.081491+0000","last_unstale":"2023-09-30T22:14:02.088018+0000","last_undegraded":"2023-09-30T22:14:02.088018+0000","last_fullsized":"2023-09-30T22:14:02.088018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080473+0000","last_change":"2023-09-30T22:14:02.067554+0000","last_active":"2023-09-30T22:14:02.080473+0000","last_peered":"2023-09-30T22:14:02.080473+0000","last_clean":"2023-09-30T22:14:02.080473+0000","last_became_active":"2023-09-30T22:14:02.067283+0000","last_became_peered":"2023-09-30T22:14:02.067283+0000","last_unstale":"2023-09-30T22:14:02.080473+0000","last_undegraded":"2023-09-30T22:14:02.080473+0000","last_fullsized":"2023-09-30T22:14:02.080473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087735+0000","last_change":"2023-09-30T22:14:02.082875+0000","last_active":"2023-09-30T22:14:02.087735+0000","last_peered":"2023-09-30T22:14:02.087735+0000","last_clean":"2023-09-30T22:14:02.087735+0000","last_became_active":"2023-09-30T22:14:02.082617+0000","last_became_peered":"2023-09-30T22:14:02.082617+0000","last_unstale":"2023-09-30T22:14:02.087735+0000","last_undegraded":"2023-09-30T22:14:02.087735+0000","last_fullsized":"2023-09-30T22:14:02.087735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079595+0000","last_change":"2023-09-30T22:14:02.076951+0000","last_active":"2023-09-30T22:14:02.079595+0000","last_peered":"2023-09-30T22:14:02.079595+0000","last_clean":"2023-09-30T22:14:02.079595+0000","last_became_active":"2023-09-30T22:14:02.076448+0000","last_became_peered":"2023-09-30T22:14:02.076448+0000","last_unstale":"2023-09-30T22:14:02.079595+0000","last_undegraded":"2023-09-30T22:14:02.079595+0000","last_fullsized":"2023-09-30T22:14:02.079595+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]}]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:14:09.691 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-30T22:14:09.692 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:14:09.692 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:14:09.692 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:14:09.692 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-30T22:14:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.842+0000 7fa55eec1700 1 Processor -- start 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.842+0000 7fa55eec1700 1 -- start start 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.842+0000 7fa55eec1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa558097cb0 0x7fa55812b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.842+0000 7fa55eec1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55812fe50 0x7fa558132240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.842+0000 7fa55eec1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa558067530 con 0x7fa5580613a0 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.842+0000 7fa55eec1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa558061780 con 0x7fa558097cb0 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.842+0000 7fa55eec1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa558066ee0 con 0x7fa55812fe50 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa55d45e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55812fe50 0x7fa558132240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa55cc5d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa5580613a0 0x7fa55812ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51860/0 (socket says 172.21.15.89:51860) 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa55cc5d700 1 -- 172.21.15.89:0/693717382 learned_addr learned my addr 172.21.15.89:0/693717382 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa557fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa558097cb0 0x7fa55812b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa55d45e700 1 -- 172.21.15.89:0/693717382 >> v2:172.21.15.89:3300/0 conn(0x7fa558097cb0 msgr2=0x7fa55812b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa55d45e700 1 --2- 172.21.15.89:0/693717382 >> v2:172.21.15.89:3300/0 conn(0x7fa558097cb0 0x7fa55812b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa55d45e700 1 -- 172.21.15.89:0/693717382 >> v1:172.21.15.89:6789/0 conn(0x7fa5580613a0 legacy=0x7fa55812ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa55d45e700 1 -- 172.21.15.89:0/693717382 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55812b880 con 0x7fa55812fe50 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa5577fe700 1 -- 172.21.15.89:0/693717382 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753764790 0 0) 0x7fa558067530 con 0x7fa5580613a0 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa557fff700 1 --2- 172.21.15.89:0/693717382 >> v2:172.21.15.89:3300/0 conn(0x7fa558097cb0 0x7fa55812b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa55d45e700 1 --2- 172.21.15.89:0/693717382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55812fe50 0x7fa558132240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fa550009e60 tx=0x7fa550009780).ready entity=mon.1 client_cookie=16bb2d1a5a2ff221 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa5577fe700 1 -- 172.21.15.89:0/693717382 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55000e070 con 0x7fa55812fe50 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa5577fe700 1 -- 172.21.15.89:0/693717382 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55000bb60 con 0x7fa55812fe50 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.843+0000 7fa5577fe700 1 -- 172.21.15.89:0/693717382 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550012410 con 0x7fa55812fe50 2023-09-30T22:14:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.844+0000 7fa55eec1700 1 -- 172.21.15.89:0/693717382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55812fe50 msgr2=0x7fa558132240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.844+0000 7fa55eec1700 1 --2- 172.21.15.89:0/693717382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55812fe50 0x7fa558132240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fa550009e60 tx=0x7fa550009780).stop 2023-09-30T22:14:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.844+0000 7fa55eec1700 1 -- 172.21.15.89:0/693717382 shutdown_connections 2023-09-30T22:14:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.844+0000 7fa55eec1700 1 --2- 172.21.15.89:0/693717382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55812fe50 0x7fa558132240 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.844+0000 7fa55eec1700 1 --2- 172.21.15.89:0/693717382 >> v2:172.21.15.89:3300/0 conn(0x7fa558097cb0 0x7fa55812b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.844+0000 7fa55eec1700 1 -- 172.21.15.89:0/693717382 >> 172.21.15.89:0/693717382 conn(0x7fa55809a220 msgr2=0x7fa55809a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.844+0000 7fa55eec1700 1 -- 172.21.15.89:0/693717382 shutdown_connections 2023-09-30T22:14:09.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.844+0000 7fa55eec1700 1 -- 172.21.15.89:0/693717382 wait complete. 2023-09-30T22:14:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55eec1700 1 Processor -- start 2023-09-30T22:14:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55eec1700 1 -- start start 2023-09-30T22:14:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55eec1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5580613a0 0x7fa558112460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55eec1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa558097cb0 0x7fa558112950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55eec1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55812fe50 0x7fa558112e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55eec1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5581164c0 con 0x7fa558097cb0 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55eec1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580982a0 con 0x7fa5580613a0 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55eec1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa558067110 con 0x7fa55812fe50 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55cc5d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5580613a0 0x7fa558112460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa557fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa558097cb0 0x7fa558112950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55cc5d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5580613a0 0x7fa558112460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52308/0 (socket says 172.21.15.89:52308) 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55cc5d700 1 -- 172.21.15.89:0/772944721 learned_addr learned my addr 172.21.15.89:0/772944721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.845+0000 7fa55d45e700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55812fe50 0x7fa558112e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa557fff700 1 -- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55812fe50 msgr2=0x7fa558112e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa557fff700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55812fe50 0x7fa558112e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa557fff700 1 -- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5580613a0 msgr2=0x7fa558112460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa557fff700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5580613a0 0x7fa558112460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa557fff700 1 -- 172.21.15.89:0/772944721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5440092a0 con 0x7fa558097cb0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa55cc5d700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5580613a0 0x7fa558112460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa557fff700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa558097cb0 0x7fa558112950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa54400e980 tx=0x7fa544009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa554ff9700 1 -- 172.21.15.89:0/772944721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544009580 con 0x7fa558097cb0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa554ff9700 1 -- 172.21.15.89:0/772944721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa544012600 con 0x7fa558097cb0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa550009a20 con 0x7fa558097cb0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa554ff9700 1 -- 172.21.15.89:0/772944721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544012980 con 0x7fa558097cb0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55812a880 con 0x7fa558097cb0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.846+0000 7fa554ff9700 1 -- 172.21.15.89:0/772944721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fa544012cf0 con 0x7fa558097cb0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.847+0000 7fa554ff9700 1 -- 172.21.15.89:0/772944721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fa54403b1f0 con 0x7fa558097cb0 2023-09-30T22:14:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.847+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55812a880 con 0x7fa558097cb0 2023-09-30T22:14:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:09.851+0000 7fa554ff9700 1 -- 172.21.15.89:0/772944721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fa55812a880 con 0x7fa558097cb0 2023-09-30T22:14:10.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.046+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa55812a880 con 0x7fa558097cb0 2023-09-30T22:14:10.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.047+0000 7fa554ff9700 1 -- 172.21.15.89:0/772944721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+64130 (secure 0 0 0) 0x7fa55812a880 con 0x7fa558097cb0 2023-09-30T22:14:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:10.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 >> v1:172.21.15.89:6816/118161 conn(0x7fa53c02bb90 legacy=0x7fa53c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa558097cb0 msgr2=0x7fa558112950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa558097cb0 0x7fa558112950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa54400e980 tx=0x7fa544009060).stop 2023-09-30T22:14:10.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 shutdown_connections 2023-09-30T22:14:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55812fe50 0x7fa558112e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5580613a0 0x7fa558112460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 --2- 172.21.15.89:0/772944721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa558097cb0 0x7fa558112950 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 >> 172.21.15.89:0/772944721 conn(0x7fa55809a220 msgr2=0x7fa558095df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 shutdown_connections 2023-09-30T22:14:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.049+0000 7fa55eec1700 1 -- 172.21.15.89:0/772944721 wait complete. 2023-09-30T22:14:10.061 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6816","nonce":118161}]},"active_addr":"172.21.15.89:6816/118161","active_change":"2023-09-30T22:13:50.734934+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1269574253}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2579157783}]}]} 2023-09-30T22:14:10.063 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:14:10.063 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:14:10.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:10.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2ab22f700 1 Processor -- start 2023-09-30T22:14:10.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2ab22f700 1 -- start start 2023-09-30T22:14:10.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2ab22f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe2a4123a90 0x7fe2a4123e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2ab22f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4127680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2ab22f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a4068880 con 0x7fe2a4067ce0 2023-09-30T22:14:10.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2ab22f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a40680c0 con 0x7fe2a4123a90 2023-09-30T22:14:10.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2ab22f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4068c90 con 0x7fe2a4092340 2023-09-30T22:14:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2a8fcb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe2a4123a90 0x7fe2a4123e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2a97cc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2a3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe2a4067ce0 0x7fe2a4126f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51874/0 (socket says 172.21.15.89:51874) 2023-09-30T22:14:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.180+0000 7fe2a3fff700 1 -- 172.21.15.89:0/780164732 learned_addr learned my addr 172.21.15.89:0/780164732 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a8fcb700 1 -- 172.21.15.89:0/780164732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 msgr2=0x7fe2a4127680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a8fcb700 1 --2- 172.21.15.89:0/780164732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4127680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a8fcb700 1 -- 172.21.15.89:0/780164732 >> v1:172.21.15.89:6789/0 conn(0x7fe2a4067ce0 legacy=0x7fe2a4126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a8fcb700 1 -- 172.21.15.89:0/780164732 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4124430 con 0x7fe2a4123a90 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a37fe700 1 -- 172.21.15.89:0/780164732 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617530571 0 0) 0x7fe2a4068880 con 0x7fe2a4067ce0 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a8fcb700 1 --2- 172.21.15.89:0/780164732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 0x7fe2a4123e70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fe29800a900 tx=0x7fe29800a200).ready entity=mon.0 client_cookie=7aa824a1ff5b42e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a37fe700 1 -- 172.21.15.89:0/780164732 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29800c790 con 0x7fe2a4123a90 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a37fe700 1 -- 172.21.15.89:0/780164732 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29800c8f0 con 0x7fe2a4123a90 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2a37fe700 1 -- 172.21.15.89:0/780164732 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29800cc70 con 0x7fe2a4123a90 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2ab22f700 1 -- 172.21.15.89:0/780164732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 msgr2=0x7fe2a4123e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2ab22f700 1 --2- 172.21.15.89:0/780164732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 0x7fe2a4123e70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fe29800a900 tx=0x7fe29800a200).stop 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2ab22f700 1 -- 172.21.15.89:0/780164732 shutdown_connections 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2ab22f700 1 --2- 172.21.15.89:0/780164732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4127680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2ab22f700 1 --2- 172.21.15.89:0/780164732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 0x7fe2a4123e70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.181+0000 7fe2ab22f700 1 -- 172.21.15.89:0/780164732 >> 172.21.15.89:0/780164732 conn(0x7fe2a409a220 msgr2=0x7fe2a409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.182+0000 7fe2ab22f700 1 -- 172.21.15.89:0/780164732 shutdown_connections 2023-09-30T22:14:10.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.182+0000 7fe2ab22f700 1 -- 172.21.15.89:0/780164732 wait complete. 2023-09-30T22:14:10.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.182+0000 7fe2ab22f700 1 Processor -- start 2023-09-30T22:14:10.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2ab22f700 1 -- start start 2023-09-30T22:14:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2ab22f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2a4067ce0 0x7fe2a41227f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2ab22f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4122ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2ab22f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 0x7fe2a4113c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2ab22f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40697c0 con 0x7fe2a4123a90 2023-09-30T22:14:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2ab22f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4118360 con 0x7fe2a4092340 2023-09-30T22:14:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2ab22f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4115060 con 0x7fe2a4067ce0 2023-09-30T22:14:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4122ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a97cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 0x7fe2a4113c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a8fcb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2a4067ce0 0x7fe2a41227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4122ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52336/0 (socket says 172.21.15.89:52336) 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a3fff700 1 -- 172.21.15.89:0/2527268510 learned_addr learned my addr 172.21.15.89:0/2527268510 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a8fcb700 1 -- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 msgr2=0x7fe2a4122ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a8fcb700 1 --2- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4122ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a8fcb700 1 -- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 msgr2=0x7fe2a4113c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.183+0000 7fe2a8fcb700 1 --2- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 0x7fe2a4113c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2a8fcb700 1 -- 172.21.15.89:0/2527268510 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29800a440 con 0x7fe2a4067ce0 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2a3fff700 1 --2- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4122ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2a8fcb700 1 --2- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2a4067ce0 0x7fe2a41227f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe298015000 tx=0x7fe2a409b1f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2a0ff9700 1 -- 172.21.15.89:0/2527268510 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298000d10 con 0x7fe2a4067ce0 2023-09-30T22:14:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2a0ff9700 1 -- 172.21.15.89:0/2527268510 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29800ca60 con 0x7fe2a4067ce0 2023-09-30T22:14:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a4117400 con 0x7fe2a4067ce0 2023-09-30T22:14:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2a0ff9700 1 -- 172.21.15.89:0/2527268510 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2980196a0 con 0x7fe2a4067ce0 2023-09-30T22:14:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a4117dc0 con 0x7fe2a4067ce0 2023-09-30T22:14:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.184+0000 7fe2a0ff9700 1 -- 172.21.15.89:0/2527268510 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fe298019a50 con 0x7fe2a4067ce0 2023-09-30T22:14:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.185+0000 7fe2a0ff9700 1 -- 172.21.15.89:0/2527268510 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fe29803beb0 con 0x7fe2a4067ce0 2023-09-30T22:14:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.185+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a405a260 con 0x7fe2a4067ce0 2023-09-30T22:14:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.189+0000 7fe2a0ff9700 1 -- 172.21.15.89:0/2527268510 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fe298014070 con 0x7fe2a4067ce0 2023-09-30T22:14:10.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.347+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2a4046c20 con 0x7fe2a4067ce0 2023-09-30T22:14:10.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.348+0000 7fe2a0ff9700 1 -- 172.21.15.89:0/2527268510 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7fe298014030 con 0x7fe2a4067ce0 2023-09-30T22:14:10.348 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:10.348 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:03.041750+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 >> v1:172.21.15.89:6816/118161 conn(0x7fe28802bb60 legacy=0x7fe28802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2a4067ce0 msgr2=0x7fe2a41227f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 --2- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2a4067ce0 0x7fe2a41227f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe298015000 tx=0x7fe2a409b1f0).stop 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 shutdown_connections 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 --2- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2a4123a90 0x7fe2a4113c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 --2- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2a4067ce0 0x7fe2a41227f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 --2- 172.21.15.89:0/2527268510 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2a4092340 0x7fe2a4122ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 >> 172.21.15.89:0/2527268510 conn(0x7fe2a409a220 msgr2=0x7fe2a4066b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 shutdown_connections 2023-09-30T22:14:10.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.350+0000 7fe2ab22f700 1 -- 172.21.15.89:0/2527268510 wait complete. 2023-09-30T22:14:10.361 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:14:10.362 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:10.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.476+0000 7f0ffe3de700 1 Processor -- start 2023-09-30T22:14:10.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.476+0000 7f0ffe3de700 1 -- start start 2023-09-30T22:14:10.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ffe3de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ffe3de700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ff8094c20 0x7f0ff8127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ffe3de700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff8096760 con 0x7f0ff8091b00 2023-09-30T22:14:10.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ffe3de700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff8091ff0 con 0x7f0ff8094c20 2023-09-30T22:14:10.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ffe3de700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8092c80 con 0x7f0ff8096380 2023-09-30T22:14:10.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff77fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0ff8091b00 0x7f0ff8127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54142/0 (socket says 172.21.15.89:54142) 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ffc97b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ff8094c20 0x7f0ff8127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff77fe700 1 -- 172.21.15.89:0/1125406157 learned_addr learned my addr 172.21.15.89:0/1125406157 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff6ffd700 1 -- 172.21.15.89:0/1125406157 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211584097 0 0) 0x7f0ff8096760 con 0x7f0ff8091b00 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff7fff700 1 -- 172.21.15.89:0/1125406157 >> v2:172.21.15.89:3300/0 conn(0x7f0ff8094c20 msgr2=0x7f0ff8127d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff7fff700 1 --2- 172.21.15.89:0/1125406157 >> v2:172.21.15.89:3300/0 conn(0x7f0ff8094c20 0x7f0ff8127d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff7fff700 1 -- 172.21.15.89:0/1125406157 >> v1:172.21.15.89:6789/0 conn(0x7f0ff8091b00 legacy=0x7f0ff8127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff7fff700 1 -- 172.21.15.89:0/1125406157 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8093900 con 0x7f0ff8096380 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ffc97b700 1 --2- 172.21.15.89:0/1125406157 >> v2:172.21.15.89:3300/0 conn(0x7f0ff8094c20 0x7f0ff8127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff7fff700 1 --2- 172.21.15.89:0/1125406157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8094730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f0fe0009790 tx=0x7f0fe0009060).ready entity=mon.1 client_cookie=758746edb2680b99 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.477+0000 7f0ff6ffd700 1 -- 172.21.15.89:0/1125406157 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000f510 con 0x7f0ff8096380 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ff6ffd700 1 -- 172.21.15.89:0/1125406157 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe000f670 con 0x7f0ff8096380 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ff6ffd700 1 -- 172.21.15.89:0/1125406157 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000f9f0 con 0x7f0ff8096380 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ffe3de700 1 -- 172.21.15.89:0/1125406157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 msgr2=0x7f0ff8094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ffe3de700 1 --2- 172.21.15.89:0/1125406157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8094730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f0fe0009790 tx=0x7f0fe0009060).stop 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ffe3de700 1 -- 172.21.15.89:0/1125406157 shutdown_connections 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ffe3de700 1 --2- 172.21.15.89:0/1125406157 >> v2:172.21.15.89:3300/0 conn(0x7f0ff8094c20 0x7f0ff8127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ffe3de700 1 --2- 172.21.15.89:0/1125406157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8094730 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ffe3de700 1 -- 172.21.15.89:0/1125406157 >> 172.21.15.89:0/1125406157 conn(0x7f0ff8097470 msgr2=0x7f0ff8097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ffe3de700 1 -- 172.21.15.89:0/1125406157 shutdown_connections 2023-09-30T22:14:10.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.478+0000 7f0ffe3de700 1 -- 172.21.15.89:0/1125406157 wait complete. 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.479+0000 7f0ffe3de700 1 Processor -- start 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.479+0000 7f0ffe3de700 1 -- start start 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.479+0000 7f0ffe3de700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8091b00 0x7f0ff811aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.479+0000 7f0ffe3de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff8094c20 0x7f0ff8116f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffe3de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8117410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffe3de700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff806a520 con 0x7f0ff8094c20 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffe3de700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8124fb0 con 0x7f0ff8096380 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffe3de700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff811a9b0 con 0x7f0ff8091b00 2023-09-30T22:14:10.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8117410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ff77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff8094c20 0x7f0ff8116f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8117410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51954/0 (socket says 172.21.15.89:51954) 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 -- 172.21.15.89:0/2275927329 learned_addr learned my addr 172.21.15.89:0/2275927329 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ff7fff700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8091b00 0x7f0ff811aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 -- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8091b00 msgr2=0x7f0ff811aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8091b00 0x7f0ff811aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 -- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff8094c20 msgr2=0x7f0ff8116f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff8094c20 0x7f0ff8116f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 -- 172.21.15.89:0/2275927329 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009a20 con 0x7f0ff8096380 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ff77fe700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff8094c20 0x7f0ff8116f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:10.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffc97b700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8117410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0ff0009fc0 tx=0x7f0ff000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0feffff700 1 -- 172.21.15.89:0/2275927329 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000cbd0 con 0x7f0ff8096380 2023-09-30T22:14:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0feffff700 1 -- 172.21.15.89:0/2275927329 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff000f040 con 0x7f0ff8096380 2023-09-30T22:14:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0feffff700 1 -- 172.21.15.89:0/2275927329 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0013400 con 0x7f0ff8096380 2023-09-30T22:14:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe00092a0 con 0x7f0ff8096380 2023-09-30T22:14:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.480+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff811a410 con 0x7f0ff8096380 2023-09-30T22:14:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.481+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff811a5d0 con 0x7f0ff8096380 2023-09-30T22:14:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.482+0000 7f0feffff700 1 -- 172.21.15.89:0/2275927329 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0ff00137f0 con 0x7f0ff8096380 2023-09-30T22:14:10.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.482+0000 7f0feffff700 1 -- 172.21.15.89:0/2275927329 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0ff003c3b0 con 0x7f0ff8096380 2023-09-30T22:14:10.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.486+0000 7f0feffff700 1 -- 172.21.15.89:0/2275927329 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0ff001cd90 con 0x7f0ff8096380 2023-09-30T22:14:10.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.646+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff8117d20 con 0x7f0ff8096380 2023-09-30T22:14:10.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.647+0000 7f0feffff700 1 -- 172.21.15.89:0/2275927329 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f0ff002b090 con 0x7f0ff8096380 2023-09-30T22:14:10.647 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:10.647 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:03.041750+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:10.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.648+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 >> v1:172.21.15.89:6816/118161 conn(0x7f0fd802bb60 legacy=0x7f0fd802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.648+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 msgr2=0x7f0ff8117410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.648+0000 7f0ffe3de700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8117410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0ff0009fc0 tx=0x7f0ff000b040).stop 2023-09-30T22:14:10.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.649+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 shutdown_connections 2023-09-30T22:14:10.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.649+0000 7f0ffe3de700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff8096380 0x7f0ff8117410 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.649+0000 7f0ffe3de700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8091b00 0x7f0ff811aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.649+0000 7f0ffe3de700 1 --2- 172.21.15.89:0/2275927329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff8094c20 0x7f0ff8116f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.649+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 >> 172.21.15.89:0/2275927329 conn(0x7f0ff8097470 msgr2=0x7f0ff8126b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.649+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 shutdown_connections 2023-09-30T22:14:10.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.649+0000 7f0ffe3de700 1 -- 172.21.15.89:0/2275927329 wait complete. 2023-09-30T22:14:10.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:14:10.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:14:10.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:14:10.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:14:10.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.787+0000 7f59f816d700 1 Processor -- start 2023-09-30T22:14:10.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.787+0000 7f79021bf700 1 Processor -- start 2023-09-30T22:14:10.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.787+0000 7f79021bf700 1 -- start start 2023-09-30T22:14:10.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.787+0000 7f59f816d700 1 -- start start 2023-09-30T22:14:10.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f79021bf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc12ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f79021bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78fc12b2f0 0x7f78fc12b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f79021bf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78fc095df0 0x7f78fc096250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f79021bf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc061780 con 0x7f78fc095df0 2023-09-30T22:14:10.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f79021bf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc12abd0 con 0x7f78fc12b2f0 2023-09-30T22:14:10.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f79021bf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc067530 con 0x7f78fc0613a0 2023-09-30T22:14:10.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f59f816d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 0x7f59f0094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f59f816d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f0127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f59f816d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f0096760 con 0x7f59f0091b00 2023-09-30T22:14:10.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f59f816d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0091ff0 con 0x7f59f0096380 2023-09-30T22:14:10.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f59f816d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0092c80 con 0x7f59f0094c20 2023-09-30T22:14:10.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc12ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc12ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33892/0 (socket says 172.21.15.89:33892) 2023-09-30T22:14:10.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78fc095df0 0x7f78fc096250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 -- 172.21.15.89:0/3604629317 learned_addr learned my addr 172.21.15.89:0/3604629317 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:10.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f59f5708700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f59f0091b00 0x7f59f0127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54148/0 (socket says 172.21.15.89:54148) 2023-09-30T22:14:10.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f59f5708700 1 -- 172.21.15.89:0/1058623597 learned_addr learned my addr 172.21.15.89:0/1058623597 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:10.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 -- 172.21.15.89:0/3604629317 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78fc12b2f0 msgr2=0x7f78fc12b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:10.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 --2- 172.21.15.89:0/3604629317 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78fc12b2f0 0x7f78fc12b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 -- 172.21.15.89:0/3604629317 >> v2:172.21.15.89:3300/0 conn(0x7f78fc095df0 msgr2=0x7f78fc096250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:10.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 --2- 172.21.15.89:0/3604629317 >> v2:172.21.15.89:3300/0 conn(0x7f78fc095df0 0x7f78fc096250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 -- 172.21.15.89:0/3604629317 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc0683c0 con 0x7f78fc0613a0 2023-09-30T22:14:10.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fbfff700 1 --2- 172.21.15.89:0/3604629317 >> v2:172.21.15.89:3300/0 conn(0x7f78fc095df0 0x7f78fc096250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:10.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.788+0000 7f78fb7fe700 1 --2- 172.21.15.89:0/3604629317 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc12ae00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f78ec009830 tx=0x7f78ec0090f0).ready entity=mon.2 client_cookie=6c2fa888a906482c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f78fa7fc700 1 -- 172.21.15.89:0/3604629317 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00b580 con 0x7f78fc0613a0 2023-09-30T22:14:10.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f670a700 1 --2- 172.21.15.89:0/1058623597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f0127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f78fa7fc700 1 -- 172.21.15.89:0/3604629317 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00b6e0 con 0x7f78fc0613a0 2023-09-30T22:14:10.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f78fa7fc700 1 -- 172.21.15.89:0/3604629317 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00ba60 con 0x7f78fc0613a0 2023-09-30T22:14:10.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f670a700 1 -- 172.21.15.89:0/1058623597 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 msgr2=0x7f59f0094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:10.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f670a700 1 --2- 172.21.15.89:0/1058623597 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 0x7f59f0094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f670a700 1 -- 172.21.15.89:0/1058623597 >> v1:172.21.15.89:6789/0 conn(0x7f59f0091b00 legacy=0x7f59f0127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f670a700 1 -- 172.21.15.89:0/1058623597 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f0093900 con 0x7f59f0094c20 2023-09-30T22:14:10.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f670a700 1 --2- 172.21.15.89:0/1058623597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f0127d10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f59e8009e90 tx=0x7f59e8009780).ready entity=mon.2 client_cookie=c7c0061a80a4b08b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f4f07700 1 -- 172.21.15.89:0/1058623597 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800e070 con 0x7f59f0094c20 2023-09-30T22:14:10.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f4f07700 1 -- 172.21.15.89:0/1058623597 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e800bc30 con 0x7f59f0094c20 2023-09-30T22:14:10.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f59f4f07700 1 -- 172.21.15.89:0/1058623597 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8012410 con 0x7f59f0094c20 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 -- 172.21.15.89:0/3604629317 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 msgr2=0x7f78fc12ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 --2- 172.21.15.89:0/3604629317 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc12ae00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f78ec009830 tx=0x7f78ec0090f0).stop 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 -- 172.21.15.89:0/3604629317 shutdown_connections 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 --2- 172.21.15.89:0/3604629317 >> v2:172.21.15.89:3300/0 conn(0x7f78fc095df0 0x7f78fc096250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 --2- 172.21.15.89:0/3604629317 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc12ae00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 --2- 172.21.15.89:0/3604629317 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78fc12b2f0 0x7f78fc12b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 -- 172.21.15.89:0/3604629317 >> 172.21.15.89:0/3604629317 conn(0x7f78fc09a220 msgr2=0x7f78fc09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 -- 172.21.15.89:0/3604629317 shutdown_connections 2023-09-30T22:14:10.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.789+0000 7f79021bf700 1 -- 172.21.15.89:0/3604629317 wait complete. 2023-09-30T22:14:10.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.790+0000 7f59f816d700 1 -- 172.21.15.89:0/1058623597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 msgr2=0x7f59f0127d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.790+0000 7f59f816d700 1 --2- 172.21.15.89:0/1058623597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f0127d10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f59e8009e90 tx=0x7f59e8009780).stop 2023-09-30T22:14:10.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.790+0000 7f59f816d700 1 -- 172.21.15.89:0/1058623597 shutdown_connections 2023-09-30T22:14:10.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.790+0000 7f59f816d700 1 --2- 172.21.15.89:0/1058623597 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 0x7f59f0094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.790+0000 7f59f816d700 1 --2- 172.21.15.89:0/1058623597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f0127d10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.790+0000 7f59f816d700 1 -- 172.21.15.89:0/1058623597 >> 172.21.15.89:0/1058623597 conn(0x7f59f0097470 msgr2=0x7f59f0097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.790+0000 7f59f816d700 1 -- 172.21.15.89:0/1058623597 shutdown_connections 2023-09-30T22:14:10.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f59f816d700 1 -- 172.21.15.89:0/1058623597 wait complete. 2023-09-30T22:14:10.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f79021bf700 1 Processor -- start 2023-09-30T22:14:10.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f79021bf700 1 -- start start 2023-09-30T22:14:10.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f79021bf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc13a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f79021bf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78fc095df0 0x7f78fc13ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f79021bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78fc12b2f0 0x7f78fc113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f79021bf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc067130 con 0x7f78fc095df0 2023-09-30T22:14:10.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f79021bf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc098cc0 con 0x7f78fc12b2f0 2023-09-30T22:14:10.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f79021bf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc12f3f0 con 0x7f78fc0613a0 2023-09-30T22:14:10.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc13a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f78faffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78fc095df0 0x7f78fc13ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc13a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33898/0 (socket says 172.21.15.89:33898) 2023-09-30T22:14:10.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f78fb7fe700 1 -- 172.21.15.89:0/2429593708 learned_addr learned my addr 172.21.15.89:0/2429593708 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:10.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f59f816d700 1 Processor -- start 2023-09-30T22:14:10.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.791+0000 7f78fbfff700 1 --2- 172.21.15.89:0/2429593708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78fc12b2f0 0x7f78fc113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78fb7fe700 1 -- 172.21.15.89:0/2429593708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78fc12b2f0 msgr2=0x7f78fc113cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78fb7fe700 1 --2- 172.21.15.89:0/2429593708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78fc12b2f0 0x7f78fc113cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78fb7fe700 1 -- 172.21.15.89:0/2429593708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78fc095df0 msgr2=0x7f78fc13ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78fb7fe700 1 --2- 172.21.15.89:0/2429593708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78fc095df0 0x7f78fc13ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78fb7fe700 1 -- 172.21.15.89:0/2429593708 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4009a20 con 0x7f78fc0613a0 2023-09-30T22:14:10.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78fb7fe700 1 --2- 172.21.15.89:0/2429593708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc13a680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78ec009ed0 tx=0x7f78ec017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec000bc0 con 0x7f78fc0613a0 2023-09-30T22:14:10.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00b850 con 0x7f78fc0613a0 2023-09-30T22:14:10.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec000d20 con 0x7f78fc0613a0 2023-09-30T22:14:10.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f816d700 1 -- start start 2023-09-30T22:14:10.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f816d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f0091b00 0x7f59f011e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f816d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f011e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f816d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 0x7f59f011ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f816d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0122b00 con 0x7f59f0091b00 2023-09-30T22:14:10.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f816d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0098ca0 con 0x7f59f0096380 2023-09-30T22:14:10.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f816d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0121e80 con 0x7f59f0094c20 2023-09-30T22:14:10.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f670a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 0x7f59f011ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f670a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 0x7f59f011ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51990/0 (socket says 172.21.15.89:51990) 2023-09-30T22:14:10.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f670a700 1 -- 172.21.15.89:0/3255532607 learned_addr learned my addr 172.21.15.89:0/3255532607 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:10.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.792+0000 7f59f5f09700 1 --2- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f0091b00 0x7f59f011e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59f5708700 1 --2- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f011e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ec009370 con 0x7f78fc0613a0 2023-09-30T22:14:10.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc117440 con 0x7f78fc0613a0 2023-09-30T22:14:10.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59f5708700 1 -- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 msgr2=0x7f59f011ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59f5708700 1 --2- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f0096380 0x7f59f011ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59f5708700 1 -- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f0091b00 msgr2=0x7f59f011e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59f5708700 1 --2- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f0091b00 0x7f59f011e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59f5708700 1 -- 172.21.15.89:0/3255532607 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e8009a20 con 0x7f59f0094c20 2023-09-30T22:14:10.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59f5708700 1 --2- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f011e970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f59e000c440 tx=0x7f59e0007d30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e00070d0 con 0x7f59f0094c20 2023-09-30T22:14:10.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e0007230 con 0x7f59f0094c20 2023-09-30T22:14:10.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f01218e0 con 0x7f59f0094c20 2023-09-30T22:14:10.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.793+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e00075b0 con 0x7f59f0094c20 2023-09-30T22:14:10.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.794+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78fc05a260 con 0x7f78fc0613a0 2023-09-30T22:14:10.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.794+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f78ec029020 con 0x7f78fc0613a0 2023-09-30T22:14:10.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.794+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f0122560 con 0x7f59f0094c20 2023-09-30T22:14:10.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.794+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f78ec044e20 con 0x7f78fc0613a0 2023-09-30T22:14:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.794+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 --> v1:172.21.15.89:6800/118778 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78e80384e0 con 0x7f78e80347f0 2023-09-30T22:14:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.794+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78ec047490 con 0x7f78fc0613a0 2023-09-30T22:14:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.794+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f59e00059b0 con 0x7f59f0094c20 2023-09-30T22:14:10.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.795+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59f0122750 con 0x7f59f0094c20 2023-09-30T22:14:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.795+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f59e00198b0 con 0x7f59f0094c20 2023-09-30T22:14:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.795+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 --> v1:172.21.15.89:6809/118777 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59d4038530 con 0x7f59d4034840 2023-09-30T22:14:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.795+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59e0013970 con 0x7f59f0094c20 2023-09-30T22:14:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.795+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 <== osd.2 v1:172.21.15.89:6800/118778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f78e80384e0 con 0x7f78e80347f0 2023-09-30T22:14:10.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.796+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 <== osd.1 v1:172.21.15.89:6809/118777 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f59d4038530 con 0x7f59d4034840 2023-09-30T22:14:10.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c2a4b4700 1 Processor -- start 2023-09-30T22:14:10.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c2a4b4700 1 -- start start 2023-09-30T22:14:10.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c2a4b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c240613a0 0x7f5c2412ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c2a4b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5c2412fe50 0x7f5c24132240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c2a4b4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c24067530 con 0x7f5c24097cb0 2023-09-30T22:14:10.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c2a4b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c24061780 con 0x7f5c2412fe50 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c2a4b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24066ee0 con 0x7f5c240613a0 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c237fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5c24097cb0 0x7f5c2412b170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54156/0 (socket says 172.21.15.89:54156) 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c237fe700 1 -- 172.21.15.89:0/2279400204 learned_addr learned my addr 172.21.15.89:0/2279400204 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.797+0000 7f5c23fff700 1 --2- 172.21.15.89:0/2279400204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c240613a0 0x7f5c2412ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c23fff700 1 -- 172.21.15.89:0/2279400204 >> v2:172.21.15.89:3300/0 conn(0x7f5c2412fe50 msgr2=0x7f5c24132240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c23fff700 1 --2- 172.21.15.89:0/2279400204 >> v2:172.21.15.89:3300/0 conn(0x7f5c2412fe50 0x7f5c24132240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c22ffd700 1 -- 172.21.15.89:0/2279400204 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75380745 0 0) 0x7f5c24067530 con 0x7f5c24097cb0 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c23fff700 1 -- 172.21.15.89:0/2279400204 >> v1:172.21.15.89:6789/0 conn(0x7f5c24097cb0 legacy=0x7f5c2412b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c23fff700 1 -- 172.21.15.89:0/2279400204 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2412b880 con 0x7f5c240613a0 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c23fff700 1 --2- 172.21.15.89:0/2279400204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c240613a0 0x7f5c2412ac80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f5c14009790 tx=0x7f5c14009060).ready entity=mon.1 client_cookie=e671829f32aadd79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c22ffd700 1 -- 172.21.15.89:0/2279400204 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400b580 con 0x7f5c240613a0 2023-09-30T22:14:10.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c22ffd700 1 -- 172.21.15.89:0/2279400204 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1400b6e0 con 0x7f5c240613a0 2023-09-30T22:14:10.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.798+0000 7f5c22ffd700 1 -- 172.21.15.89:0/2279400204 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400ba60 con 0x7f5c240613a0 2023-09-30T22:14:10.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb8320c7700 1 Processor -- start 2023-09-30T22:14:10.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb8320c7700 1 -- start start 2023-09-30T22:14:10.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2279400204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c240613a0 msgr2=0x7f5c2412ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7f5c2a4b4700 1 --2- 172.21.15.89:0/2279400204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c240613a0 0x7f5c2412ac80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f5c14009790 tx=0x7f5c14009060).stop 2023-09-30T22:14:10.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb8320c7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 0x7fb82c098bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb8320c7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb82c1297d0 0x7fb82c131320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb8320c7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82c092fb0 con 0x7fb82c0990b0 2023-09-30T22:14:10.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb8320c7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82c093f20 con 0x7fb82c1297d0 2023-09-30T22:14:10.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb8320c7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c093cd0 con 0x7fb82c0987e0 2023-09-30T22:14:10.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2279400204 shutdown_connections 2023-09-30T22:14:10.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7f5c2a4b4700 1 --2- 172.21.15.89:0/2279400204 >> v2:172.21.15.89:3300/0 conn(0x7f5c2412fe50 0x7f5c24132240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7f5c2a4b4700 1 --2- 172.21.15.89:0/2279400204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c240613a0 0x7f5c2412ac80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2279400204 >> 172.21.15.89:0/2279400204 conn(0x7f5c2409a220 msgr2=0x7f5c2409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb82affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb82c0990b0 0x7fb82c1290c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54162/0 (socket says 172.21.15.89:54162) 2023-09-30T22:14:10.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb82affd700 1 -- 172.21.15.89:0/2593854174 learned_addr learned my addr 172.21.15.89:0/2593854174 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:10.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.799+0000 7fb82bfff700 1 --2- 172.21.15.89:0/2593854174 >> v2:172.21.15.89:3300/0 conn(0x7fb82c1297d0 0x7fb82c131320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2279400204 shutdown_connections 2023-09-30T22:14:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7fb82bfff700 1 -- 172.21.15.89:0/2593854174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 msgr2=0x7fb82c098bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7fb82bfff700 1 --2- 172.21.15.89:0/2593854174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 0x7fb82c098bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7fb82bfff700 1 -- 172.21.15.89:0/2593854174 >> v1:172.21.15.89:6789/0 conn(0x7fb82c0990b0 legacy=0x7fb82c1290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7fb82bfff700 1 -- 172.21.15.89:0/2593854174 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c0675e0 con 0x7fb82c1297d0 2023-09-30T22:14:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7fb82bfff700 1 --2- 172.21.15.89:0/2593854174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb82c1297d0 0x7fb82c131320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb824009e60 tx=0x7fb824009780).ready entity=mon.0 client_cookie=23181bf77b486357 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7fb82a7fc700 1 -- 172.21.15.89:0/2593854174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82400e070 con 0x7fb82c1297d0 2023-09-30T22:14:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7fb82a7fc700 1 -- 172.21.15.89:0/2593854174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82400bc30 con 0x7fb82c1297d0 2023-09-30T22:14:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7fb82a7fc700 1 -- 172.21.15.89:0/2593854174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824012410 con 0x7fb82c1297d0 2023-09-30T22:14:10.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.800+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2279400204 wait complete. 2023-09-30T22:14:10.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c2a4b4700 1 Processor -- start 2023-09-30T22:14:10.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7fb8320c7700 1 -- 172.21.15.89:0/2593854174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb82c1297d0 msgr2=0x7fb82c131320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7fb8320c7700 1 --2- 172.21.15.89:0/2593854174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb82c1297d0 0x7fb82c131320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb824009e60 tx=0x7fb824009780).stop 2023-09-30T22:14:10.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c2a4b4700 1 -- start start 2023-09-30T22:14:10.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7fb8320c7700 1 -- 172.21.15.89:0/2593854174 shutdown_connections 2023-09-30T22:14:10.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7fb8320c7700 1 --2- 172.21.15.89:0/2593854174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb82c1297d0 0x7fb82c131320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7fb8320c7700 1 --2- 172.21.15.89:0/2593854174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 0x7fb82c098bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7fb8320c7700 1 -- 172.21.15.89:0/2593854174 >> 172.21.15.89:0/2593854174 conn(0x7fb82c09a3a0 msgr2=0x7fb82c0963f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7fb8320c7700 1 -- 172.21.15.89:0/2593854174 shutdown_connections 2023-09-30T22:14:10.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7fb8320c7700 1 -- 172.21.15.89:0/2593854174 wait complete. 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c2a4b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c240613a0 0x7f5c2412a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c2a4b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c24097cb0 0x7f5c24094330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c2a4b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2412fe50 0x7f5c24094820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c2a4b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24067110 con 0x7f5c240613a0 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c2a4b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24122720 con 0x7f5c2412fe50 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c2a4b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24122cc0 con 0x7f5c24097cb0 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.801+0000 7f5c23fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c240613a0 0x7f5c2412a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c28a51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2412fe50 0x7f5c24094820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c23fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c240613a0 0x7f5c2412a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36562/0 (socket says 172.21.15.89:36562) 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c23fff700 1 -- 172.21.15.89:0/2241186572 learned_addr learned my addr 172.21.15.89:0/2241186572 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c28a51700 1 -- 172.21.15.89:0/2241186572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c24097cb0 msgr2=0x7f5c24094330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c28a51700 1 --2- 172.21.15.89:0/2241186572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c24097cb0 0x7f5c24094330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c28a51700 1 -- 172.21.15.89:0/2241186572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c240613a0 msgr2=0x7f5c2412a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c28a51700 1 --2- 172.21.15.89:0/2241186572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c240613a0 0x7f5c2412a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c28a51700 1 -- 172.21.15.89:0/2241186572 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c140092a0 con 0x7f5c2412fe50 2023-09-30T22:14:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7f5c28a51700 1 --2- 172.21.15.89:0/2241186572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2412fe50 0x7f5c24094820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f5c24061b60 tx=0x7f5c1c009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7fb8320c7700 1 Processor -- start 2023-09-30T22:14:10.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.802+0000 7fb8320c7700 1 -- start start 2023-09-30T22:14:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb8320c7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 0x7fb82c138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb8320c7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb82c0990b0 0x7fb82c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb8320c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb82c1297d0 0x7fb82c114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb8320c7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c092da0 con 0x7fb82c0990b0 2023-09-30T22:14:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb8320c7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c1158a0 con 0x7fb82c1297d0 2023-09-30T22:14:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb8320c7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c12d5c0 con 0x7fb82c0987e0 2023-09-30T22:14:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7f5c0bfff700 1 -- 172.21.15.89:0/2241186572 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00c7c0 con 0x7f5c2412fe50 2023-09-30T22:14:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb82c1297d0 0x7fb82c114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 0x7fb82c138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb82c1297d0 0x7fb82c114430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52016/0 (socket says 172.21.15.89:52016) 2023-09-30T22:14:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82bfff700 1 -- 172.21.15.89:0/3351241084 learned_addr learned my addr 172.21.15.89:0/3351241084 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82b7fe700 1 --2- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 0x7fb82c138820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33926/0 (socket says 172.21.15.89:33926) 2023-09-30T22:14:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82affd700 1 --2- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb82c0990b0 0x7fb82c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7f5c0bfff700 1 -- 172.21.15.89:0/2241186572 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c00c920 con 0x7f5c2412fe50 2023-09-30T22:14:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7f5c0bfff700 1 -- 172.21.15.89:0/2241186572 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00cab0 con 0x7f5c2412fe50 2023-09-30T22:14:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c24095780 con 0x7f5c2412fe50 2023-09-30T22:14:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c241233a0 con 0x7f5c2412fe50 2023-09-30T22:14:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82bfff700 1 -- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 msgr2=0x7fb82c138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82bfff700 1 --2- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb82c0987e0 0x7fb82c138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82bfff700 1 -- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb82c0990b0 msgr2=0x7fb82c138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82bfff700 1 --2- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb82c0990b0 0x7fb82c138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:10.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.803+0000 7fb82bfff700 1 -- 172.21.15.89:0/3351241084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb824009a20 con 0x7fb82c1297d0 2023-09-30T22:14:10.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.804+0000 7fb82bfff700 1 --2- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb82c1297d0 0x7fb82c114430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb82400b6c0 tx=0x7fb82c093bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:10.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.804+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824009440 con 0x7fb82c1297d0 2023-09-30T22:14:10.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.804+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c117e80 con 0x7fb82c1297d0 2023-09-30T22:14:10.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.804+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c118700 con 0x7fb82c1297d0 2023-09-30T22:14:10.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.804+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82401c870 con 0x7fb82c1297d0 2023-09-30T22:14:10.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.804+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82401c9e0 con 0x7fb82c1297d0 2023-09-30T22:14:10.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.804+0000 7f5c0bfff700 1 -- 172.21.15.89:0/2241186572 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5c1c012020 con 0x7f5c2412fe50 2023-09-30T22:14:10.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.804+0000 7f5c0bfff700 1 -- 172.21.15.89:0/2241186572 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5c1c03af70 con 0x7f5c2412fe50 2023-09-30T22:14:10.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.805+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb80c000fc0 con 0x7fb82c1297d0 2023-09-30T22:14:10.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.805+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb824003040 con 0x7fb82c1297d0 2023-09-30T22:14:10.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.805+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb82403cb30 con 0x7fb82c1297d0 2023-09-30T22:14:10.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.805+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 --> v1:172.21.15.89:6804/118768 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb818038460 con 0x7fb818034770 2023-09-30T22:14:10.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.805+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8240269b0 con 0x7fb82c1297d0 2023-09-30T22:14:10.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.806+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 <== osd.0 v1:172.21.15.89:6804/118768 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7fb818038460 con 0x7fb818034770 2023-09-30T22:14:10.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.807+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 --> v1:172.21.15.89:6808/118781 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c24132730 con 0x7f5c2405e090 2023-09-30T22:14:10.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.809+0000 7f5c0bfff700 1 -- 172.21.15.89:0/2241186572 <== osd.3 v1:172.21.15.89:6808/118781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f5c24132730 con 0x7f5c2405e090 2023-09-30T22:14:10.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.814+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 --> v1:172.21.15.89:6800/118778 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78fc097220 con 0x7f78e80347f0 2023-09-30T22:14:10.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.814+0000 7f78e37fe700 1 -- 172.21.15.89:0/2429593708 <== osd.2 v1:172.21.15.89:6800/118778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f78fc097220 con 0x7f78e80347f0 2023-09-30T22:14:10.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.814+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 >> v1:172.21.15.89:6800/118778 conn(0x7f78e80347f0 legacy=0x7f78e8036c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.814+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 >> v1:172.21.15.89:6816/118161 conn(0x7f78e802bb60 legacy=0x7f78e802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.814+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 msgr2=0x7f78fc13a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.814+0000 7f79021bf700 1 --2- 172.21.15.89:0/2429593708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78fc0613a0 0x7f78fc13a680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78ec009ed0 tx=0x7f78ec017000).stop 2023-09-30T22:14:10.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.814+0000 7f78fbfff700 1 -- 172.21.15.89:0/2429593708 reap_dead start 2023-09-30T22:14:10.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.816+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 shutdown_connections 2023-09-30T22:14:10.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.816+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 >> 172.21.15.89:0/2429593708 conn(0x7f78fc09a220 msgr2=0x7f78fc065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.818+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 shutdown_connections 2023-09-30T22:14:10.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.818+0000 7f79021bf700 1 -- 172.21.15.89:0/2429593708 wait complete. 2023-09-30T22:14:10.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.821+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 --> v1:172.21.15.89:6809/118777 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f59f0121b00 con 0x7f59d4034840 2023-09-30T22:14:10.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.821+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 --> v1:172.21.15.89:6804/118768 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb80c002b10 con 0x7fb818034770 2023-09-30T22:14:10.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.821+0000 7f59e5ffb700 1 -- 172.21.15.89:0/3255532607 <== osd.1 v1:172.21.15.89:6809/118777 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1129848789) 0x7f59f0121b00 con 0x7f59d4034840 2023-09-30T22:14:10.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.821+0000 7fb8137fe700 1 -- 172.21.15.89:0/3351241084 <== osd.0 v1:172.21.15.89:6804/118768 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7fb80c002b10 con 0x7fb818034770 2023-09-30T22:14:10.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.821+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 >> v1:172.21.15.89:6809/118777 conn(0x7f59d4034840 legacy=0x7f59d4036cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 >> v1:172.21.15.89:6804/118768 conn(0x7fb818034770 legacy=0x7fb818036bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 >> v1:172.21.15.89:6816/118161 conn(0x7f59d402bbb0 legacy=0x7f59d402e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 msgr2=0x7f59f011e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7f59f816d700 1 --2- 172.21.15.89:0/3255532607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59f0094c20 0x7f59f011e970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f59e000c440 tx=0x7f59e0007d30).stop 2023-09-30T22:14:10.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7f59f670a700 1 -- 172.21.15.89:0/3255532607 reap_dead start 2023-09-30T22:14:10.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 shutdown_connections 2023-09-30T22:14:10.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 >> 172.21.15.89:0/3255532607 conn(0x7f59f0097470 msgr2=0x7f59f0059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 >> v1:172.21.15.89:6816/118161 conn(0x7fb81802bb60 legacy=0x7fb81802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb82c1297d0 msgr2=0x7fb82c114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb8320c7700 1 --2- 172.21.15.89:0/3351241084 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb82c1297d0 0x7fb82c114430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb82400b6c0 tx=0x7fb82c093bf0).stop 2023-09-30T22:14:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb82bfff700 1 -- 172.21.15.89:0/3351241084 reap_dead start 2023-09-30T22:14:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 shutdown_connections 2023-09-30T22:14:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 shutdown_connections 2023-09-30T22:14:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 >> 172.21.15.89:0/3351241084 conn(0x7fb82c09a3a0 msgr2=0x7fb82c068820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7f59f816d700 1 -- 172.21.15.89:0/3255532607 wait complete. 2023-09-30T22:14:10.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 shutdown_connections 2023-09-30T22:14:10.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.822+0000 7fb8320c7700 1 -- 172.21.15.89:0/3351241084 wait complete. 2023-09-30T22:14:10.846 INFO:teuthology.orchestra.run.smithi089.stdout:38654705669 2023-09-30T22:14:10.846 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:14:10.847 INFO:teuthology.orchestra.run.smithi089.stdout:38654705669 2023-09-30T22:14:10.847 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:14:10.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.836+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 --> v1:172.21.15.89:6808/118781 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5c240696c0 con 0x7f5c2405e090 2023-09-30T22:14:10.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.837+0000 7f5c0bfff700 1 -- 172.21.15.89:0/2241186572 <== osd.3 v1:172.21.15.89:6808/118781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1129848789) 0x7f5c240696c0 con 0x7f5c2405e090 2023-09-30T22:14:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.837+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 >> v1:172.21.15.89:6808/118781 conn(0x7f5c2405e090 legacy=0x7f5c2405e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.837+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 >> v1:172.21.15.89:6816/118161 conn(0x7f5c1002bb60 legacy=0x7f5c1002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.837+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2412fe50 msgr2=0x7f5c24094820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.837+0000 7f5c2a4b4700 1 --2- 172.21.15.89:0/2241186572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2412fe50 0x7f5c24094820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f5c24061b60 tx=0x7f5c1c009780).stop 2023-09-30T22:14:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.837+0000 7f5c28a51700 1 -- 172.21.15.89:0/2241186572 reap_dead start 2023-09-30T22:14:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.837+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 shutdown_connections 2023-09-30T22:14:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.837+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 >> 172.21.15.89:0/2241186572 conn(0x7f5c2409a220 msgr2=0x7f5c24131730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:10.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.838+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 shutdown_connections 2023-09-30T22:14:10.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:10.838+0000 7f5c2a4b4700 1 -- 172.21.15.89:0/2241186572 wait complete. 2023-09-30T22:14:10.850 INFO:teuthology.orchestra.run.smithi089.stdout:38654705670 2023-09-30T22:14:10.851 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:14:10.852 INFO:teuthology.orchestra.run.smithi089.stdout:38654705670 2023-09-30T22:14:10.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:14:11.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.014+0000 7f69b4616700 1 Processor -- start 2023-09-30T22:14:11.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.014+0000 7f69b4616700 1 -- start start 2023-09-30T22:14:11.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.014+0000 7f69b4616700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f69ac091b00 0x7f69ac12c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.014+0000 7f69b4616700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac12ce60 0x7f69ac12f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.014+0000 7f69b4616700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ac096760 con 0x7f69ac096380 2023-09-30T22:14:11.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.014+0000 7f69b4616700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ac091ff0 con 0x7f69ac091b00 2023-09-30T22:14:11.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.014+0000 7f69b4616700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac092c80 con 0x7f69ac12ce60 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b1bb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f69ac091b00 0x7f69ac12c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b1bb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f69ac091b00 0x7f69ac12c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36588/0 (socket says 172.21.15.89:36588) 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b1bb1700 1 -- 172.21.15.89:0/1788694126 learned_addr learned my addr 172.21.15.89:0/1788694126 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b2bb3700 1 --2- 172.21.15.89:0/1788694126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac12ce60 0x7f69ac12f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b1bb1700 1 -- 172.21.15.89:0/1788694126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac12ce60 msgr2=0x7f69ac12f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b1bb1700 1 --2- 172.21.15.89:0/1788694126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac12ce60 0x7f69ac12f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b1bb1700 1 -- 172.21.15.89:0/1788694126 >> v1:172.21.15.89:6789/0 conn(0x7f69ac096380 legacy=0x7f69ac094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b1bb1700 1 -- 172.21.15.89:0/1788694126 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac094e40 con 0x7f69ac091b00 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b1bb1700 1 --2- 172.21.15.89:0/1788694126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ac091b00 0x7f69ac12c970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f699c006fd0 tx=0x7f699c0090f0).ready entity=mon.0 client_cookie=ccc0ca06d71f5d8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b13b0700 1 -- 172.21.15.89:0/1788694126 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c00b600 con 0x7f69ac091b00 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b13b0700 1 -- 172.21.15.89:0/1788694126 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c00b760 con 0x7f69ac091b00 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.015+0000 7f69b13b0700 1 -- 172.21.15.89:0/1788694126 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c00bae0 con 0x7f69ac091b00 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.016+0000 7f69b4616700 1 -- 172.21.15.89:0/1788694126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ac091b00 msgr2=0x7f69ac12c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.016+0000 7f69b4616700 1 --2- 172.21.15.89:0/1788694126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ac091b00 0x7f69ac12c970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f699c006fd0 tx=0x7f699c0090f0).stop 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.016+0000 7f69b4616700 1 -- 172.21.15.89:0/1788694126 shutdown_connections 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.016+0000 7f69b4616700 1 --2- 172.21.15.89:0/1788694126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac12ce60 0x7f69ac12f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.016+0000 7f69b4616700 1 --2- 172.21.15.89:0/1788694126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ac091b00 0x7f69ac12c970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.016+0000 7f69b4616700 1 -- 172.21.15.89:0/1788694126 >> 172.21.15.89:0/1788694126 conn(0x7f69ac097470 msgr2=0x7f69ac097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.016+0000 7f69b4616700 1 -- 172.21.15.89:0/1788694126 shutdown_connections 2023-09-30T22:14:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.016+0000 7f69b4616700 1 -- 172.21.15.89:0/1788694126 wait complete. 2023-09-30T22:14:11.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.017+0000 7f349609e700 1 Processor -- start 2023-09-30T22:14:11.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.017+0000 7f69b4616700 1 Processor -- start 2023-09-30T22:14:11.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.017+0000 7f69b4616700 1 -- start start 2023-09-30T22:14:11.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.017+0000 7f349609e700 1 -- start start 2023-09-30T22:14:11.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7fed69928700 1 Processor -- start 2023-09-30T22:14:11.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b4616700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ac091b00 0x7f69ac12c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b4616700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac096380 0x7f69ac082060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b4616700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ac12ce60 0x7f69ac082550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b4616700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac12a8f0 con 0x7f69ac091b00 2023-09-30T22:14:11.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b4616700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac12e0b0 con 0x7f69ac096380 2023-09-30T22:14:11.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f349609e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3490066cd0 0x7f34900670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f349609e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f34900675a0 0x7f349012f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f349609e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3490098000 con 0x7f349012f6e0 2023-09-30T22:14:11.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f349609e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3490068b90 con 0x7f34900675a0 2023-09-30T22:14:11.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f349609e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34900921c0 con 0x7f3490066cd0 2023-09-30T22:14:11.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b4616700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac06a520 con 0x7f69ac12ce60 2023-09-30T22:14:11.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7fed69928700 1 -- start start 2023-09-30T22:14:11.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f348effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f34900675a0 0x7f349012f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b2bb3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ac12ce60 0x7f69ac082550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b1bb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac096380 0x7f69ac082060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b2bb3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ac12ce60 0x7f69ac082550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33930/0 (socket says 172.21.15.89:33930) 2023-09-30T22:14:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b2bb3700 1 -- 172.21.15.89:0/252925029 learned_addr learned my addr 172.21.15.89:0/252925029 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7fed69928700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64092340 0x7fed64123a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7fed69928700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed64123f10 0x7fed641282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7fed69928700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed64068880 con 0x7fed64067ce0 2023-09-30T22:14:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed62ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fed64067ce0 0x7fed640613a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54192/0 (socket says 172.21.15.89:54192) 2023-09-30T22:14:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed62ffd700 1 -- 172.21.15.89:0/3838973817 learned_addr learned my addr 172.21.15.89:0/3838973817 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed637fe700 1 --2- 172.21.15.89:0/3838973817 >> v2:172.21.15.89:3300/0 conn(0x7fed64123f10 0x7fed641282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed69928700 1 -- 172.21.15.89:0/3838973817 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed640680c0 con 0x7fed64123f10 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed627fc700 1 --2- 172.21.15.89:0/3838973817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64092340 0x7fed64123a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed69928700 1 -- 172.21.15.89:0/3838973817 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed64092720 con 0x7fed64092340 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed637fe700 1 -- 172.21.15.89:0/3838973817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64092340 msgr2=0x7fed64123a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed637fe700 1 --2- 172.21.15.89:0/3838973817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64092340 0x7fed64123a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed637fe700 1 -- 172.21.15.89:0/3838973817 >> v1:172.21.15.89:6789/0 conn(0x7fed64067ce0 legacy=0x7fed640613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed637fe700 1 -- 172.21.15.89:0/3838973817 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64068c90 con 0x7fed64123f10 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed637fe700 1 --2- 172.21.15.89:0/3838973817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64123f10 0x7fed641282a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fed5c009ec0 tx=0x7fed5c009780).ready entity=mon.0 client_cookie=a36ceab0f9971efd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed61ffb700 1 -- 172.21.15.89:0/3838973817 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00e070 con 0x7fed64123f10 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f348effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f34900675a0 0x7f349012f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36612/0 (socket says 172.21.15.89:36612) 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f348effd700 1 -- 172.21.15.89:0/1842439422 learned_addr learned my addr 172.21.15.89:0/1842439422 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f348e7fc700 1 -- 172.21.15.89:0/1842439422 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140599177 0 0) 0x7f3490098000 con 0x7f349012f6e0 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348e7fc700 1 -- 172.21.15.89:0/1842439422 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34780033f0 con 0x7f349012f6e0 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348f7fe700 1 --2- 172.21.15.89:0/1842439422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3490066cd0 0x7f34900670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348e7fc700 1 -- 172.21.15.89:0/1842439422 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 546388580 0 0) 0x7f34780033f0 con 0x7f349012f6e0 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348f7fe700 1 -- 172.21.15.89:0/1842439422 >> v2:172.21.15.89:3300/0 conn(0x7f34900675a0 msgr2=0x7f349012f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348f7fe700 1 --2- 172.21.15.89:0/1842439422 >> v2:172.21.15.89:3300/0 conn(0x7f34900675a0 0x7f349012f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348f7fe700 1 -- 172.21.15.89:0/1842439422 >> v1:172.21.15.89:6789/0 conn(0x7f349012f6e0 legacy=0x7f3490136440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348f7fe700 1 -- 172.21.15.89:0/1842439422 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34900697b0 con 0x7f3490066cd0 2023-09-30T22:14:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348f7fe700 1 --2- 172.21.15.89:0/1842439422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3490066cd0 0x7f34900670b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3480009760 tx=0x7f3480009060).ready entity=mon.2 client_cookie=9efd1f88fc50a73a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b1bb1700 1 -- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ac12ce60 msgr2=0x7f69ac082550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b1bb1700 1 --2- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ac12ce60 0x7f69ac082550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b1bb1700 1 -- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ac091b00 msgr2=0x7f69ac12c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b1bb1700 1 --2- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ac091b00 0x7f69ac12c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.018+0000 7f69b1bb1700 1 -- 172.21.15.89:0/252925029 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c009370 con 0x7f69ac096380 2023-09-30T22:14:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f69b1bb1700 1 --2- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac096380 0x7f69ac082060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f69ac0982c0 tx=0x7f699c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f69a27fc700 1 -- 172.21.15.89:0/252925029 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c00bea0 con 0x7f69ac096380 2023-09-30T22:14:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f69a27fc700 1 -- 172.21.15.89:0/252925029 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c00b8d0 con 0x7f69ac096380 2023-09-30T22:14:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f69a27fc700 1 -- 172.21.15.89:0/252925029 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c01c670 con 0x7f69ac096380 2023-09-30T22:14:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed61ffb700 1 -- 172.21.15.89:0/3838973817 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c00bbb0 con 0x7fed64123f10 2023-09-30T22:14:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7fed61ffb700 1 -- 172.21.15.89:0/3838973817 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c012410 con 0x7fed64123f10 2023-09-30T22:14:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348e7fc700 1 -- 172.21.15.89:0/1842439422 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34880017f0 con 0x7f3490066cd0 2023-09-30T22:14:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348e7fc700 1 -- 172.21.15.89:0/1842439422 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3480011400 con 0x7f3490066cd0 2023-09-30T22:14:11.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.019+0000 7f348e7fc700 1 -- 172.21.15.89:0/1842439422 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480011780 con 0x7f3490066cd0 2023-09-30T22:14:11.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ac12afe0 con 0x7f69ac096380 2023-09-30T22:14:11.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ac12ba10 con 0x7f69ac096380 2023-09-30T22:14:11.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7fed69928700 1 -- 172.21.15.89:0/3838973817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64123f10 msgr2=0x7fed641282a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7fed69928700 1 --2- 172.21.15.89:0/3838973817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64123f10 0x7fed641282a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fed5c009ec0 tx=0x7fed5c009780).stop 2023-09-30T22:14:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7f349609e700 1 -- 172.21.15.89:0/1842439422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3490066cd0 msgr2=0x7f34900670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7f349609e700 1 --2- 172.21.15.89:0/1842439422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3490066cd0 0x7f34900670b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3480009760 tx=0x7f3480009060).stop 2023-09-30T22:14:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7fed69928700 1 -- 172.21.15.89:0/3838973817 shutdown_connections 2023-09-30T22:14:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7fed69928700 1 --2- 172.21.15.89:0/3838973817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64123f10 0x7fed641282a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7fed69928700 1 --2- 172.21.15.89:0/3838973817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64092340 0x7fed64123a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7fed69928700 1 -- 172.21.15.89:0/3838973817 >> 172.21.15.89:0/3838973817 conn(0x7fed6409a220 msgr2=0x7fed6409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7fed69928700 1 -- 172.21.15.89:0/3838973817 shutdown_connections 2023-09-30T22:14:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.020+0000 7fed69928700 1 -- 172.21.15.89:0/3838973817 wait complete. 2023-09-30T22:14:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f69a27fc700 1 -- 172.21.15.89:0/252925029 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f699c01c9e0 con 0x7f69ac096380 2023-09-30T22:14:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f349609e700 1 -- 172.21.15.89:0/1842439422 shutdown_connections 2023-09-30T22:14:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f349609e700 1 --2- 172.21.15.89:0/1842439422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3490066cd0 0x7f34900670b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f349609e700 1 --2- 172.21.15.89:0/1842439422 >> v2:172.21.15.89:3300/0 conn(0x7f34900675a0 0x7f349012f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f349609e700 1 -- 172.21.15.89:0/1842439422 >> 172.21.15.89:0/1842439422 conn(0x7f349009a220 msgr2=0x7f349009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f349609e700 1 -- 172.21.15.89:0/1842439422 shutdown_connections 2023-09-30T22:14:11.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ac05a260 con 0x7f69ac096380 2023-09-30T22:14:11.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f69a27fc700 1 -- 172.21.15.89:0/252925029 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f699c03c810 con 0x7f69ac096380 2023-09-30T22:14:11.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed69928700 1 Processor -- start 2023-09-30T22:14:11.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed69928700 1 -- start start 2023-09-30T22:14:11.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed69928700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64067ce0 0x7fed64130ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed69928700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed64092340 0x7fed64128f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed69928700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64123f10 0x7fed64129400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.021+0000 7f349609e700 1 -- 172.21.15.89:0/1842439422 wait complete. 2023-09-30T22:14:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed69928700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6412c4a0 con 0x7fed64123f10 2023-09-30T22:14:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed69928700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed641279c0 con 0x7fed64067ce0 2023-09-30T22:14:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7f349609e700 1 Processor -- start 2023-09-30T22:14:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed69928700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed640697c0 con 0x7fed64092340 2023-09-30T22:14:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed62ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64067ce0 0x7fed64130ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed62ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64067ce0 0x7fed64130ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52036/0 (socket says 172.21.15.89:52036) 2023-09-30T22:14:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed62ffd700 1 -- 172.21.15.89:0/722689037 learned_addr learned my addr 172.21.15.89:0/722689037 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed627fc700 1 --2- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed64092340 0x7fed64128f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed62ffd700 1 -- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed64092340 msgr2=0x7fed64128f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed62ffd700 1 --2- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed64092340 0x7fed64128f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed62ffd700 1 -- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64123f10 msgr2=0x7fed64129400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed62ffd700 1 --2- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64123f10 0x7fed64129400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7fed62ffd700 1 -- 172.21.15.89:0/722689037 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c009a20 con 0x7fed64067ce0 2023-09-30T22:14:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7fed62ffd700 1 --2- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64067ce0 0x7fed64130ee0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fed54001eb0 tx=0x7fed54007e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.022+0000 7f349609e700 1 -- start start 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f349609e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3490066cd0 0x7f349011d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f349609e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34900675a0 0x7f349011dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f349609e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349012f6e0 0x7f349011e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f349609e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349012eac0 con 0x7f349012f6e0 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f349609e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3490069270 con 0x7f3490066cd0 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3490066cd0 0x7f349011d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34900675a0 0x7f349011dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3490066cd0 0x7f349011d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52040/0 (socket says 172.21.15.89:52040) 2023-09-30T22:14:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348f7fe700 1 -- 172.21.15.89:0/1184356265 learned_addr learned my addr 172.21.15.89:0/1184356265 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3490120ef0 con 0x7f34900675a0 2023-09-30T22:14:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348f7fe700 1 -- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34900675a0 msgr2=0x7f349011dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348f7fe700 1 --2- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34900675a0 0x7f349011dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348f7fe700 1 -- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349012f6e0 msgr2=0x7f349011e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348f7fe700 1 --2- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349012f6e0 0x7f349011e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348f7fe700 1 -- 172.21.15.89:0/1184356265 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800092a0 con 0x7f3490066cd0 2023-09-30T22:14:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.023+0000 7f348effd700 1 --2- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34900675a0 0x7f349011dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.024+0000 7f348f7fe700 1 --2- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3490066cd0 0x7f349011d9f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3480009d50 tx=0x7f3480016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7fed4affd700 1 -- 172.21.15.89:0/722689037 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5400ce40 con 0x7fed64067ce0 2023-09-30T22:14:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7fed4affd700 1 -- 172.21.15.89:0/722689037 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed540072e0 con 0x7fed64067ce0 2023-09-30T22:14:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7fed4affd700 1 -- 172.21.15.89:0/722689037 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed540057a0 con 0x7fed64067ce0 2023-09-30T22:14:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6412a310 con 0x7fed64067ce0 2023-09-30T22:14:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7f34777fe700 1 -- 172.21.15.89:0/1184356265 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480010740 con 0x7f3490066cd0 2023-09-30T22:14:11.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7f34777fe700 1 -- 172.21.15.89:0/1184356265 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34800108a0 con 0x7f3490066cd0 2023-09-30T22:14:11.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7f34777fe700 1 -- 172.21.15.89:0/1184356265 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480010c60 con 0x7f3490066cd0 2023-09-30T22:14:11.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349012f290 con 0x7f3490066cd0 2023-09-30T22:14:11.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7f69a27fc700 1 -- 172.21.15.89:0/252925029 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f699c020070 con 0x7f69ac096380 2023-09-30T22:14:11.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3490121b70 con 0x7f3490066cd0 2023-09-30T22:14:11.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.026+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6412ab30 con 0x7fed64067ce0 2023-09-30T22:14:11.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.027+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3470005120 con 0x7f3490066cd0 2023-09-30T22:14:11.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.028+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6405a260 con 0x7fed64067ce0 2023-09-30T22:14:11.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.028+0000 7fed4affd700 1 -- 172.21.15.89:0/722689037 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fed54005b50 con 0x7fed64067ce0 2023-09-30T22:14:11.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.028+0000 7fed4affd700 1 -- 172.21.15.89:0/722689037 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fed5403a720 con 0x7fed64067ce0 2023-09-30T22:14:11.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.032+0000 7f34777fe700 1 -- 172.21.15.89:0/1184356265 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f34800263b0 con 0x7f3490066cd0 2023-09-30T22:14:11.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.032+0000 7f34777fe700 1 -- 172.21.15.89:0/1184356265 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f348003be30 con 0x7f3490066cd0 2023-09-30T22:14:11.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.036+0000 7fed4affd700 1 -- 172.21.15.89:0/722689037 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fed54005e00 con 0x7fed64067ce0 2023-09-30T22:14:11.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.044+0000 7f34777fe700 1 -- 172.21.15.89:0/1184356265 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3480010e40 con 0x7f3490066cd0 2023-09-30T22:14:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.072+0000 7f1c88fbc700 1 Processor -- start 2023-09-30T22:14:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.072+0000 7f1c88fbc700 1 -- start start 2023-09-30T22:14:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c88fbc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c84067ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c88fbc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1c84092340 0x7f1c84127680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c88fbc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c84068880 con 0x7f1c84123a90 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c88fbc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c84124250 con 0x7f1c84092340 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c88fbc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c84068c90 con 0x7f1c84123e70 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c82d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1c84092340 0x7f1c84127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c8259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1c84123a90 0x7f1c84126f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54208/0 (socket says 172.21.15.89:54208) 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c8259c700 1 -- 172.21.15.89:0/3825294175 learned_addr learned my addr 172.21.15.89:0/3825294175 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c82d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1c84092340 0x7f1c84127680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36648/0 (socket says 172.21.15.89:36648) 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.073+0000 7f1c81d9b700 1 --2- 172.21.15.89:0/3825294175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c84067ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.074+0000 7f1c82d9d700 1 -- 172.21.15.89:0/3825294175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 msgr2=0x7f1c84067ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.074+0000 7f1c82d9d700 1 --2- 172.21.15.89:0/3825294175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c84067ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.074+0000 7f1c82d9d700 1 -- 172.21.15.89:0/3825294175 >> v1:172.21.15.89:6789/0 conn(0x7f1c84123a90 legacy=0x7f1c84126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.074+0000 7f1c82d9d700 1 -- 172.21.15.89:0/3825294175 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84124430 con 0x7f1c84092340 2023-09-30T22:14:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.074+0000 7f1c82d9d700 1 --2- 172.21.15.89:0/3825294175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c84127680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1c7c009e60 tx=0x7f1c7c009780).ready entity=mon.0 client_cookie=7326fa17bb40514d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.074+0000 7f1c8159a700 1 -- 172.21.15.89:0/3825294175 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c00e070 con 0x7f1c84092340 2023-09-30T22:14:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.074+0000 7f1c8159a700 1 -- 172.21.15.89:0/3825294175 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c00bb60 con 0x7f1c84092340 2023-09-30T22:14:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.074+0000 7f1c8159a700 1 -- 172.21.15.89:0/3825294175 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c012410 con 0x7f1c84092340 2023-09-30T22:14:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.075+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3825294175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 msgr2=0x7f1c84127680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.075+0000 7f1c88fbc700 1 --2- 172.21.15.89:0/3825294175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c84127680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1c7c009e60 tx=0x7f1c7c009780).stop 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.075+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3825294175 shutdown_connections 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.075+0000 7f1c88fbc700 1 --2- 172.21.15.89:0/3825294175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c84127680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.075+0000 7f1c88fbc700 1 --2- 172.21.15.89:0/3825294175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c84067ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.075+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3825294175 >> 172.21.15.89:0/3825294175 conn(0x7f1c8409a220 msgr2=0x7f1c8409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.075+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3825294175 shutdown_connections 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.075+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3825294175 wait complete. 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c88fbc700 1 Processor -- start 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c88fbc700 1 -- start start 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c88fbc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c8412d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c88fbc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c84123a90 0x7f1c8412d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c88fbc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c8412dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c88fbc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8412e180 con 0x7f1c84092340 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c88fbc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c84110ea0 con 0x7f1c84123e70 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c88fbc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c840697c0 con 0x7f1c84123a90 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c8259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c8412d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c8259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c8412d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36660/0 (socket says 172.21.15.89:36660) 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c8259c700 1 -- 172.21.15.89:0/3806509359 learned_addr learned my addr 172.21.15.89:0/3806509359 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c82d9d700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c8412dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.076+0000 7f1c81d9b700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c84123a90 0x7f1c8412d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c8259c700 1 -- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c84123a90 msgr2=0x7f1c8412d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c8259c700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c84123a90 0x7f1c8412d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c8259c700 1 -- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 msgr2=0x7f1c8412dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c8259c700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c8412dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c8259c700 1 -- 172.21.15.89:0/3806509359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c009a20 con 0x7f1c84092340 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c82d9d700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c8412dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c8259c700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c8412d2b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f1c7400c490 tx=0x7f1c74007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c727fc700 1 -- 172.21.15.89:0/3806509359 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740072e0 con 0x7f1c84092340 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c727fc700 1 -- 172.21.15.89:0/3806509359 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c74007440 con 0x7f1c84092340 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c84130840 con 0x7f1c84092340 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c727fc700 1 -- 172.21.15.89:0/3806509359 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740057a0 con 0x7f1c84092340 2023-09-30T22:14:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.077+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8412eed0 con 0x7f1c84092340 2023-09-30T22:14:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.078+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8405a260 con 0x7f1c84092340 2023-09-30T22:14:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.079+0000 7f1c727fc700 1 -- 172.21.15.89:0/3806509359 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1c74005bd0 con 0x7f1c84092340 2023-09-30T22:14:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.079+0000 7f1c727fc700 1 -- 172.21.15.89:0/3806509359 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f1c7403a910 con 0x7f1c84092340 2023-09-30T22:14:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.083+0000 7f1c727fc700 1 -- 172.21.15.89:0/3806509359 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1c7401d3f0 con 0x7f1c84092340 2023-09-30T22:14:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.204+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fed64046c20 con 0x7fed64067ce0 2023-09-30T22:14:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.204+0000 7fed4affd700 1 -- 172.21.15.89:0/722689037 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed54022020 con 0x7fed64067ce0 2023-09-30T22:14:11.204 INFO:teuthology.orchestra.run.smithi089.stdout:38654705667 2023-09-30T22:14:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 >> v1:172.21.15.89:6816/118161 conn(0x7fed5002bb60 legacy=0x7fed5002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64067ce0 msgr2=0x7fed64130ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 --2- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64067ce0 0x7fed64130ee0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fed54001eb0 tx=0x7fed54007e50).stop 2023-09-30T22:14:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 shutdown_connections 2023-09-30T22:14:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 --2- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64123f10 0x7fed64129400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 --2- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed64067ce0 0x7fed64130ee0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 --2- 172.21.15.89:0/722689037 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed64092340 0x7fed64128f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 >> 172.21.15.89:0/722689037 conn(0x7fed6409a220 msgr2=0x7fed64127150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 shutdown_connections 2023-09-30T22:14:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.206+0000 7fed69928700 1 -- 172.21.15.89:0/722689037 wait complete. 2023-09-30T22:14:11.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705667 for osd.0 2023-09-30T22:14:11.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.231+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f69ac093980 con 0x7f69ac096380 2023-09-30T22:14:11.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.231+0000 7f69a27fc700 1 -- 172.21.15.89:0/252925029 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f699c020030 con 0x7f69ac096380 2023-09-30T22:14:11.231 INFO:teuthology.orchestra.run.smithi089.stdout:38654705667 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 >> v1:172.21.15.89:6816/118161 conn(0x7f699002bbb0 legacy=0x7f699002e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac096380 msgr2=0x7f69ac082060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 --2- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac096380 0x7f69ac082060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f69ac0982c0 tx=0x7f699c017000).stop 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 shutdown_connections 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 --2- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ac12ce60 0x7f69ac082550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 --2- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ac091b00 0x7f69ac12c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 --2- 172.21.15.89:0/252925029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ac096380 0x7f69ac082060 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 >> 172.21.15.89:0/252925029 conn(0x7f69ac097470 msgr2=0x7f69ac127e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 shutdown_connections 2023-09-30T22:14:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.234+0000 7f69b4616700 1 -- 172.21.15.89:0/252925029 wait complete. 2023-09-30T22:14:11.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705667 for osd.2 2023-09-30T22:14:11.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.246+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f34700057f0 con 0x7f3490066cd0 2023-09-30T22:14:11.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.246+0000 7f34777fe700 1 -- 172.21.15.89:0/1184356265 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3480022030 con 0x7f3490066cd0 2023-09-30T22:14:11.247 INFO:teuthology.orchestra.run.smithi089.stdout:38654705668 2023-09-30T22:14:11.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 >> v1:172.21.15.89:6816/118161 conn(0x7f347c02bb60 legacy=0x7f347c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3490066cd0 msgr2=0x7f349011d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 --2- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3490066cd0 0x7f349011d9f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3480009d50 tx=0x7f3480016000).stop 2023-09-30T22:14:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 shutdown_connections 2023-09-30T22:14:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 --2- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f349012f6e0 0x7f349011e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 --2- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3490066cd0 0x7f349011d9f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 --2- 172.21.15.89:0/1184356265 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34900675a0 0x7f349011dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 >> 172.21.15.89:0/1184356265 conn(0x7f349009a220 msgr2=0x7f3490096870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 shutdown_connections 2023-09-30T22:14:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.248+0000 7f349609e700 1 -- 172.21.15.89:0/1184356265 wait complete. 2023-09-30T22:14:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.251+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1c84046c20 con 0x7f1c84092340 2023-09-30T22:14:11.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.251+0000 7f1c727fc700 1 -- 172.21.15.89:0/3806509359 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c74022070 con 0x7f1c84092340 2023-09-30T22:14:11.251 INFO:teuthology.orchestra.run.smithi089.stdout:38654705668 2023-09-30T22:14:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 >> v1:172.21.15.89:6816/118161 conn(0x7f1c6402bb60 legacy=0x7f1c6402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 msgr2=0x7f1c8412d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c8412d2b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f1c7400c490 tx=0x7f1c74007df0).stop 2023-09-30T22:14:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 shutdown_connections 2023-09-30T22:14:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c84123e70 0x7f1c8412dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c84092340 0x7f1c8412d2b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 --2- 172.21.15.89:0/3806509359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c84123a90 0x7f1c8412d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 >> 172.21.15.89:0/3806509359 conn(0x7f1c8409a220 msgr2=0x7f1c84092a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 shutdown_connections 2023-09-30T22:14:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:11.253+0000 7f1c88fbc700 1 -- 172.21.15.89:0/3806509359 wait complete. 2023-09-30T22:14:11.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705668 for osd.1 2023-09-30T22:14:11.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705668 for osd.3 2023-09-30T22:14:12.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:14:12.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:14:12.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:14:12.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:14:12.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.368+0000 7fe0542e1700 1 Processor -- start 2023-09-30T22:14:12.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.368+0000 7fe0542e1700 1 -- start start 2023-09-30T22:14:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe0542e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c097280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe0542e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe04c097770 0x7fe04c136a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe0542e1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04c065d20 con 0x7fe04c06a220 2023-09-30T22:14:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe0542e1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c06a600 con 0x7fe04c097770 2023-09-30T22:14:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe0542e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c064630 con 0x7fe04c096e60 2023-09-30T22:14:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05287e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe04c097770 0x7fe04c136a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05187c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c097280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05287e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe04c097770 0x7fe04c136a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36662/0 (socket says 172.21.15.89:36662) 2023-09-30T22:14:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05187c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c097280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33978/0 (socket says 172.21.15.89:33978) 2023-09-30T22:14:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05287e700 1 -- 172.21.15.89:0/3016840106 learned_addr learned my addr 172.21.15.89:0/3016840106 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05107b700 1 -- 172.21.15.89:0/3016840106 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183182512 0 0) 0x7fe04c065d20 con 0x7fe04c06a220 2023-09-30T22:14:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05187c700 1 -- 172.21.15.89:0/3016840106 >> v2:172.21.15.89:3300/0 conn(0x7fe04c097770 msgr2=0x7fe04c136a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05187c700 1 --2- 172.21.15.89:0/3016840106 >> v2:172.21.15.89:3300/0 conn(0x7fe04c097770 0x7fe04c136a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05187c700 1 -- 172.21.15.89:0/3016840106 >> v1:172.21.15.89:6789/0 conn(0x7fe04c06a220 legacy=0x7fe04c096750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05187c700 1 -- 172.21.15.89:0/3016840106 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c098260 con 0x7fe04c096e60 2023-09-30T22:14:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.369+0000 7fe05287e700 1 --2- 172.21.15.89:0/3016840106 >> v2:172.21.15.89:3300/0 conn(0x7fe04c097770 0x7fe04c136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.370+0000 7fe05187c700 1 --2- 172.21.15.89:0/3016840106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c097280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe03c009760 tx=0x7fe03c009060).ready entity=mon.2 client_cookie=3d9a4664c09ecaaf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.370+0000 7fe05107b700 1 -- 172.21.15.89:0/3016840106 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0090d0 con 0x7fe04c096e60 2023-09-30T22:14:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.370+0000 7fe05107b700 1 -- 172.21.15.89:0/3016840106 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c00f510 con 0x7fe04c096e60 2023-09-30T22:14:12.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.370+0000 7fe05107b700 1 -- 172.21.15.89:0/3016840106 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00f890 con 0x7fe04c096e60 2023-09-30T22:14:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.371+0000 7fe0542e1700 1 -- 172.21.15.89:0/3016840106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 msgr2=0x7fe04c097280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.371+0000 7fe0542e1700 1 --2- 172.21.15.89:0/3016840106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c097280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe03c009760 tx=0x7fe03c009060).stop 2023-09-30T22:14:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.371+0000 7fe0542e1700 1 -- 172.21.15.89:0/3016840106 shutdown_connections 2023-09-30T22:14:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.371+0000 7fe0542e1700 1 --2- 172.21.15.89:0/3016840106 >> v2:172.21.15.89:3300/0 conn(0x7fe04c097770 0x7fe04c136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.371+0000 7fe0542e1700 1 --2- 172.21.15.89:0/3016840106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c097280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.371+0000 7fe0542e1700 1 -- 172.21.15.89:0/3016840106 >> 172.21.15.89:0/3016840106 conn(0x7fe04c09ba50 msgr2=0x7fe04c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:12.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.371+0000 7fe0542e1700 1 -- 172.21.15.89:0/3016840106 shutdown_connections 2023-09-30T22:14:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.371+0000 7fe0542e1700 1 -- 172.21.15.89:0/3016840106 wait complete. 2023-09-30T22:14:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe0542e1700 1 Processor -- start 2023-09-30T22:14:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe0542e1700 1 -- start start 2023-09-30T22:14:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe0542e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe04c06a220 0x7fe04c13e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe0542e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c13ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe0542e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe04c097770 0x7fe04c13f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe0542e1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c13f660 con 0x7fe04c097770 2023-09-30T22:14:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe0542e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c122500 con 0x7fe04c06a220 2023-09-30T22:14:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe0542e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c099880 con 0x7fe04c096e60 2023-09-30T22:14:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.372+0000 7fe05207d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe04c06a220 0x7fe04c13e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05207d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe04c06a220 0x7fe04c13e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52074/0 (socket says 172.21.15.89:52074) 2023-09-30T22:14:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05207d700 1 -- 172.21.15.89:0/3565216681 learned_addr learned my addr 172.21.15.89:0/3565216681 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05287e700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe04c097770 0x7fe04c13f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05187c700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c13ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05207d700 1 -- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 msgr2=0x7fe04c13ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05207d700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c13ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05207d700 1 -- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe04c097770 msgr2=0x7fe04c13f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05207d700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe04c097770 0x7fe04c13f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05207d700 1 -- 172.21.15.89:0/3565216681 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044009ae0 con 0x7fe04c06a220 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05287e700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe04c097770 0x7fe04c13f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe05207d700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe04c06a220 0x7fe04c13e790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe04800c490 tx=0x7fe048007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe0427fc700 1 -- 172.21.15.89:0/3565216681 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800f470 con 0x7fe04c06a220 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe0427fc700 1 -- 172.21.15.89:0/3565216681 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04800f5f0 con 0x7fe04c06a220 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c0092a0 con 0x7fe04c06a220 2023-09-30T22:14:12.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe0427fc700 1 -- 172.21.15.89:0/3565216681 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800f9b0 con 0x7fe04c06a220 2023-09-30T22:14:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.373+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c122d10 con 0x7fe04c06a220 2023-09-30T22:14:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.375+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c05a260 con 0x7fe04c06a220 2023-09-30T22:14:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.375+0000 7fe0427fc700 1 -- 172.21.15.89:0/3565216681 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fe048012400 con 0x7fe04c06a220 2023-09-30T22:14:12.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.375+0000 7fe0427fc700 1 -- 172.21.15.89:0/3565216681 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fe04803a480 con 0x7fe04c06a220 2023-09-30T22:14:12.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.383+0000 7fe0427fc700 1 -- 172.21.15.89:0/3565216681 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fe048010c90 con 0x7fe04c06a220 2023-09-30T22:14:12.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.396+0000 7f01f8418700 1 Processor -- start 2023-09-30T22:14:12.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f8418700 1 -- start start 2023-09-30T22:14:12.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f8418700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f012c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f8418700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01f012ce60 0x7f01f012f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f8418700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f0096760 con 0x7f01f0096380 2023-09-30T22:14:12.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f8418700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f0091ff0 con 0x7f01f012ce60 2023-09-30T22:14:12.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f8418700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0092c80 con 0x7f01f0091b00 2023-09-30T22:14:12.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f59b3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f012c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f59b3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f012c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52078/0 (socket says 172.21.15.89:52078) 2023-09-30T22:14:12.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.397+0000 7f01f59b3700 1 -- 172.21.15.89:0/2854223000 learned_addr learned my addr 172.21.15.89:0/2854223000 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:12.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f69b5700 1 --2- 172.21.15.89:0/2854223000 >> v2:172.21.15.89:3300/0 conn(0x7f01f012ce60 0x7f01f012f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f59b3700 1 -- 172.21.15.89:0/2854223000 >> v2:172.21.15.89:3300/0 conn(0x7f01f012ce60 msgr2=0x7f01f012f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f59b3700 1 --2- 172.21.15.89:0/2854223000 >> v2:172.21.15.89:3300/0 conn(0x7f01f012ce60 0x7f01f012f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f59b3700 1 -- 172.21.15.89:0/2854223000 >> v1:172.21.15.89:6789/0 conn(0x7f01f0096380 legacy=0x7f01f0094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f59b3700 1 -- 172.21.15.89:0/2854223000 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0094e40 con 0x7f01f0091b00 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f59b3700 1 --2- 172.21.15.89:0/2854223000 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f012c970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f01e0006fd0 tx=0x7f01e00090f0).ready entity=mon.1 client_cookie=5860ef1fa663e081 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f51b2700 1 -- 172.21.15.89:0/2854223000 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000b580 con 0x7f01f0091b00 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f51b2700 1 -- 172.21.15.89:0/2854223000 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e000b6e0 con 0x7f01f0091b00 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.398+0000 7f01f51b2700 1 -- 172.21.15.89:0/2854223000 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000ba60 con 0x7f01f0091b00 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.399+0000 7f01f8418700 1 -- 172.21.15.89:0/2854223000 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 msgr2=0x7f01f012c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.399+0000 7f01f8418700 1 --2- 172.21.15.89:0/2854223000 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f012c970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f01e0006fd0 tx=0x7f01e00090f0).stop 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.399+0000 7f01f8418700 1 -- 172.21.15.89:0/2854223000 shutdown_connections 2023-09-30T22:14:12.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.399+0000 7f01f8418700 1 --2- 172.21.15.89:0/2854223000 >> v2:172.21.15.89:3300/0 conn(0x7f01f012ce60 0x7f01f012f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.399+0000 7f01f8418700 1 --2- 172.21.15.89:0/2854223000 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f012c970 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.399+0000 7f01f8418700 1 -- 172.21.15.89:0/2854223000 >> 172.21.15.89:0/2854223000 conn(0x7f01f0097470 msgr2=0x7f01f0097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:12.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.399+0000 7f01f8418700 1 -- 172.21.15.89:0/2854223000 shutdown_connections 2023-09-30T22:14:12.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.399+0000 7f01f8418700 1 -- 172.21.15.89:0/2854223000 wait complete. 2023-09-30T22:14:12.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.400+0000 7f01f8418700 1 Processor -- start 2023-09-30T22:14:12.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.400+0000 7f01f8418700 1 -- start start 2023-09-30T22:14:12.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.400+0000 7f01f8418700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f0123cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.400+0000 7f01f8418700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01f0096380 0x7f01f01241e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.400+0000 7f01f8418700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f012ce60 0x7f01f01246d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.400+0000 7f01f8418700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0112a30 con 0x7f01f012ce60 2023-09-30T22:14:12.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.400+0000 7f01f8418700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f012c190 con 0x7f01f0091b00 2023-09-30T22:14:12.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.400+0000 7f01f8418700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0098f30 con 0x7f01f0096380 2023-09-30T22:14:12.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f0123cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f0123cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52090/0 (socket says 172.21.15.89:52090) 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 -- 172.21.15.89:0/4290333877 learned_addr learned my addr 172.21.15.89:0/4290333877 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f59b3700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01f0096380 0x7f01f01241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f69b5700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f012ce60 0x7f01f01246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 -- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01f0096380 msgr2=0x7f01f01241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01f0096380 0x7f01f01241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 -- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f012ce60 msgr2=0x7f01f01246d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f012ce60 0x7f01f01246d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 -- 172.21.15.89:0/4290333877 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0009370 con 0x7f01f0091b00 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f59b3700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01f0096380 0x7f01f01241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f61b4700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f0123cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f01ec00c490 tx=0x7f01ec007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01e67fc700 1 -- 172.21.15.89:0/4290333877 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00ce40 con 0x7f01f0091b00 2023-09-30T22:14:12.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01e67fc700 1 -- 172.21.15.89:0/4290333877 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec007020 con 0x7f01f0091b00 2023-09-30T22:14:12.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f0111810 con 0x7f01f0091b00 2023-09-30T22:14:12.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.401+0000 7f01e67fc700 1 -- 172.21.15.89:0/4290333877 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0071d0 con 0x7f01f0091b00 2023-09-30T22:14:12.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.402+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f0112650 con 0x7f01f0091b00 2023-09-30T22:14:12.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.402+0000 7f01e67fc700 1 -- 172.21.15.89:0/4290333877 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f01ec017030 con 0x7f01f0091b00 2023-09-30T22:14:12.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.403+0000 7f01e67fc700 1 -- 172.21.15.89:0/4290333877 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f01ec03de90 con 0x7f01f0091b00 2023-09-30T22:14:12.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.403+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f0112650 con 0x7f01f0091b00 2023-09-30T22:14:12.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.407+0000 7f01e67fc700 1 -- 172.21.15.89:0/4290333877 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f01f0112650 con 0x7f01f0091b00 2023-09-30T22:14:12.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.411+0000 7ff0cb5fe700 1 Processor -- start 2023-09-30T22:14:12.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0cb5fe700 1 -- start start 2023-09-30T22:14:12.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0cb5fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c40970d0 0x7ff0c412cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0cb5fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c412d120 0x7ff0c412f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0cb5fe700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c40919e0 con 0x7ff0c40965e0 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0cb5fe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40933b0 con 0x7ff0c412d120 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0cb5fe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4092190 con 0x7ff0c40970d0 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0c8b99700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c40970d0 0x7ff0c412cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0c8b99700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c40970d0 0x7ff0c412cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33996/0 (socket says 172.21.15.89:33996) 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0c8b99700 1 -- 172.21.15.89:0/1024101952 learned_addr learned my addr 172.21.15.89:0/1024101952 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.412+0000 7ff0c9b9b700 1 --2- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c412d120 0x7ff0c412f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0c8b99700 1 -- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c412d120 msgr2=0x7ff0c412f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0c8b99700 1 --2- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c412d120 0x7ff0c412f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0c8b99700 1 -- 172.21.15.89:0/1024101952 >> v1:172.21.15.89:6789/0 conn(0x7ff0c40965e0 legacy=0x7ff0c40969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0c8b99700 1 -- 172.21.15.89:0/1024101952 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4069a20 con 0x7ff0c40970d0 2023-09-30T22:14:12.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0c9b9b700 1 --2- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c412d120 0x7ff0c412f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:12.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0c8b99700 1 --2- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c40970d0 0x7ff0c412cc30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff0b4009830 tx=0x7ff0b40090f0).ready entity=mon.2 client_cookie=bbc51bc424326219 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:12.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0bbfff700 1 -- 172.21.15.89:0/1024101952 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400b580 con 0x7ff0c40970d0 2023-09-30T22:14:12.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0bbfff700 1 -- 172.21.15.89:0/1024101952 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b400b6e0 con 0x7ff0c40970d0 2023-09-30T22:14:12.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.413+0000 7ff0bbfff700 1 -- 172.21.15.89:0/1024101952 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400ba60 con 0x7ff0c40970d0 2023-09-30T22:14:12.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.414+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c40970d0 msgr2=0x7ff0c412cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.414+0000 7ff0cb5fe700 1 --2- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c40970d0 0x7ff0c412cc30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff0b4009830 tx=0x7ff0b40090f0).stop 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.414+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/1024101952 shutdown_connections 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.414+0000 7ff0cb5fe700 1 --2- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c412d120 0x7ff0c412f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.414+0000 7ff0cb5fe700 1 --2- 172.21.15.89:0/1024101952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c40970d0 0x7ff0c412cc30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.414+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/1024101952 >> 172.21.15.89:0/1024101952 conn(0x7ff0c409a220 msgr2=0x7ff0c409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.414+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/1024101952 shutdown_connections 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.414+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/1024101952 wait complete. 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.415+0000 7ff0cb5fe700 1 Processor -- start 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.415+0000 7ff0cb5fe700 1 -- start start 2023-09-30T22:14:12.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.415+0000 7ff0cb5fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c40965e0 0x7ff0c4136720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.415+0000 7ff0cb5fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c40970d0 0x7ff0c4136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.415+0000 7ff0cb5fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c412d120 0x7ff0c4115e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.415+0000 7ff0cb5fe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4119550 con 0x7ff0c40965e0 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.415+0000 7ff0cb5fe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c412d500 con 0x7ff0c40970d0 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.415+0000 7ff0cb5fe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c412cae0 con 0x7ff0c412d120 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c40965e0 0x7ff0c4136720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c40965e0 0x7ff0c4136720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36684/0 (socket says 172.21.15.89:36684) 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 -- 172.21.15.89:0/3963115002 learned_addr learned my addr 172.21.15.89:0/3963115002 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c9b9b700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c412d120 0x7ff0c4115e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c8b99700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c40970d0 0x7ff0c4136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 -- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c412d120 msgr2=0x7ff0c4115e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c412d120 0x7ff0c4115e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 -- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c40970d0 msgr2=0x7ff0c4136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c40970d0 0x7ff0c4136c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 -- 172.21.15.89:0/3963115002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc009a20 con 0x7ff0c40965e0 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c8b99700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c40970d0 0x7ff0c4136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.416+0000 7ff0c939a700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c40965e0 0x7ff0c4136720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7ff0c000c4f0 tx=0x7ff0c0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.417+0000 7ff0b97fa700 1 -- 172.21.15.89:0/3963115002 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0007020 con 0x7ff0c40965e0 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.417+0000 7ff0b97fa700 1 -- 172.21.15.89:0/3963115002 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0001e90 con 0x7ff0c40965e0 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.417+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b4009370 con 0x7ff0c40965e0 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.417+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c4118120 con 0x7ff0c40965e0 2023-09-30T22:14:12.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.417+0000 7ff0b97fa700 1 -- 172.21.15.89:0/3963115002 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00073b0 con 0x7ff0c40965e0 2023-09-30T22:14:12.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.418+0000 7ff0b97fa700 1 -- 172.21.15.89:0/3963115002 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ff0c00068b0 con 0x7ff0c40965e0 2023-09-30T22:14:12.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.418+0000 7ff0b97fa700 1 -- 172.21.15.89:0/3963115002 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ff0c003aa30 con 0x7ff0c40965e0 2023-09-30T22:14:12.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.418+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a4005120 con 0x7ff0c40965e0 2023-09-30T22:14:12.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.422+0000 7ff0b97fa700 1 -- 172.21.15.89:0/3963115002 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7ff0c0007510 con 0x7ff0c40965e0 2023-09-30T22:14:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.436+0000 7f2b9333a700 1 Processor -- start 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b9333a700 1 -- start start 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b9333a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c059070 0x7f2b8c12f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b9333a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c12ecd0 0x7f2b8c12f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b9333a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c059ba0 con 0x7f2b8c095370 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b9333a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c0969b0 con 0x7f2b8c12ecd0 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b9333a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c092320 con 0x7f2b8c059070 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b910d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c059070 0x7f2b8c12f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b908d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c12ecd0 0x7f2b8c12f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b910d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c059070 0x7f2b8c12f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34022/0 (socket says 172.21.15.89:34022) 2023-09-30T22:14:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b910d6700 1 -- 172.21.15.89:0/1846156413 learned_addr learned my addr 172.21.15.89:0/1846156413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.437+0000 7f2b908d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c12ecd0 0x7f2b8c12f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52120/0 (socket says 172.21.15.89:52120) 2023-09-30T22:14:12.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b908d5700 1 -- 172.21.15.89:0/1846156413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c059070 msgr2=0x7f2b8c12f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:12.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b908d5700 1 --2- 172.21.15.89:0/1846156413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c059070 0x7f2b8c12f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b908d5700 1 -- 172.21.15.89:0/1846156413 >> v1:172.21.15.89:6789/0 conn(0x7f2b8c095370 legacy=0x7f2b8c095820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b83fff700 1 -- 172.21.15.89:0/1846156413 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885109807 0 0) 0x7f2b8c059ba0 con 0x7f2b8c095370 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b908d5700 1 -- 172.21.15.89:0/1846156413 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c091c10 con 0x7f2b8c12ecd0 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b908d5700 1 --2- 172.21.15.89:0/1846156413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c12ecd0 0x7f2b8c12f650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2b7c009790 tx=0x7f2b7c009060).ready entity=mon.1 client_cookie=d8bbdbcdc7cb625a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b83fff700 1 -- 172.21.15.89:0/1846156413 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00f510 con 0x7f2b8c12ecd0 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b83fff700 1 -- 172.21.15.89:0/1846156413 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c00f670 con 0x7f2b8c12ecd0 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.438+0000 7f2b83fff700 1 -- 172.21.15.89:0/1846156413 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00f9f0 con 0x7f2b8c12ecd0 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.439+0000 7f2b9333a700 1 -- 172.21.15.89:0/1846156413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c12ecd0 msgr2=0x7f2b8c12f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.439+0000 7f2b9333a700 1 --2- 172.21.15.89:0/1846156413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c12ecd0 0x7f2b8c12f650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2b7c009790 tx=0x7f2b7c009060).stop 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.439+0000 7f2b9333a700 1 -- 172.21.15.89:0/1846156413 shutdown_connections 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.439+0000 7f2b9333a700 1 --2- 172.21.15.89:0/1846156413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c059070 0x7f2b8c12f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.439+0000 7f2b9333a700 1 --2- 172.21.15.89:0/1846156413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c12ecd0 0x7f2b8c12f650 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.439+0000 7f2b9333a700 1 -- 172.21.15.89:0/1846156413 >> 172.21.15.89:0/1846156413 conn(0x7f2b8c097470 msgr2=0x7f2b8c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:12.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.439+0000 7f2b9333a700 1 -- 172.21.15.89:0/1846156413 shutdown_connections 2023-09-30T22:14:12.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.439+0000 7f2b9333a700 1 -- 172.21.15.89:0/1846156413 wait complete. 2023-09-30T22:14:12.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.440+0000 7f2b9333a700 1 Processor -- start 2023-09-30T22:14:12.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.440+0000 7f2b9333a700 1 -- start start 2023-09-30T22:14:12.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.440+0000 7f2b9333a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b8c059070 0x7f2b8c132800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.440+0000 7f2b9333a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c095370 0x7f2b8c135e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.440+0000 7f2b9333a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c136360 0x7f2b8c1367a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.440+0000 7f2b9333a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c134fb0 con 0x7f2b8c059070 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.440+0000 7f2b9333a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c136c90 con 0x7f2b8c136360 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.440+0000 7f2b9333a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c137cf0 con 0x7f2b8c095370 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b908d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c095370 0x7f2b8c135e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b908d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c095370 0x7f2b8c135e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34036/0 (socket says 172.21.15.89:34036) 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b908d5700 1 -- 172.21.15.89:0/1575522639 learned_addr learned my addr 172.21.15.89:0/1575522639 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b918d7700 1 --2- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c136360 0x7f2b8c1367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b918d7700 1 -- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c095370 msgr2=0x7f2b8c135e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b918d7700 1 --2- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c095370 0x7f2b8c135e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b918d7700 1 -- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b8c059070 msgr2=0x7f2b8c132800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b918d7700 1 --2- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b8c059070 0x7f2b8c132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:12.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b918d7700 1 -- 172.21.15.89:0/1575522639 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0092a0 con 0x7f2b8c136360 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b918d7700 1 --2- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c136360 0x7f2b8c1367a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2b8400c730 tx=0x7f2b84007ee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b817fa700 1 -- 172.21.15.89:0/1575522639 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84001630 con 0x7f2b8c136360 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b817fa700 1 -- 172.21.15.89:0/1575522639 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b84010510 con 0x7f2b8c136360 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c133490 con 0x7f2b8c136360 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b817fa700 1 -- 172.21.15.89:0/1575522639 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840108d0 con 0x7f2b8c136360 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.441+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c06a5a0 con 0x7f2b8c136360 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.443+0000 7f2b817fa700 1 -- 172.21.15.89:0/1575522639 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2b84010c80 con 0x7f2b8c136360 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.443+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c05a260 con 0x7f2b8c136360 2023-09-30T22:14:12.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.443+0000 7f2b817fa700 1 -- 172.21.15.89:0/1575522639 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2b8403ad50 con 0x7f2b8c136360 2023-09-30T22:14:12.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.447+0000 7f2b817fa700 1 -- 172.21.15.89:0/1575522639 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2b840186a0 con 0x7f2b8c136360 2023-09-30T22:14:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.559+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe04c122d10 con 0x7fe04c06a220 2023-09-30T22:14:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.560+0000 7fe0427fc700 1 -- 172.21.15.89:0/3565216681 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe04c122d10 con 0x7fe04c06a220 2023-09-30T22:14:12.560 INFO:teuthology.orchestra.run.smithi089.stdout:38654705668 2023-09-30T22:14:12.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.561+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 >> v1:172.21.15.89:6816/118161 conn(0x7fe03002bb60 legacy=0x7fe03002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.561+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe04c06a220 msgr2=0x7fe04c13e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.561+0000 7fe0542e1700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe04c06a220 0x7fe04c13e790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe04800c490 tx=0x7fe048007c20).stop 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.562+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 shutdown_connections 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.562+0000 7fe0542e1700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe04c097770 0x7fe04c13f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.562+0000 7fe0542e1700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe04c06a220 0x7fe04c13e790 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.562+0000 7fe0542e1700 1 --2- 172.21.15.89:0/3565216681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe04c096e60 0x7fe04c13ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.562+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 >> 172.21.15.89:0/3565216681 conn(0x7fe04c09ba50 msgr2=0x7fe04c067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.562+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 shutdown_connections 2023-09-30T22:14:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.562+0000 7fe0542e1700 1 -- 172.21.15.89:0/3565216681 wait complete. 2023-09-30T22:14:12.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2023-09-30T22:14:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.581+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f01f0112650 con 0x7f01f0091b00 2023-09-30T22:14:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.582+0000 7f01e67fc700 1 -- 172.21.15.89:0/4290333877 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01f0112650 con 0x7f01f0091b00 2023-09-30T22:14:12.581 INFO:teuthology.orchestra.run.smithi089.stdout:38654705669 2023-09-30T22:14:12.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.583+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 >> v1:172.21.15.89:6816/118161 conn(0x7f01d402bb60 legacy=0x7f01d402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.583+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 msgr2=0x7f01f0123cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.583+0000 7f01f8418700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f0123cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f01ec00c490 tx=0x7f01ec007e00).stop 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.584+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 shutdown_connections 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.584+0000 7f01f8418700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01f012ce60 0x7f01f01246d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.584+0000 7f01f8418700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01f0091b00 0x7f01f0123cf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.584+0000 7f01f8418700 1 --2- 172.21.15.89:0/4290333877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01f0096380 0x7f01f01241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.584+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 >> 172.21.15.89:0/4290333877 conn(0x7f01f0097470 msgr2=0x7f01f0059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.584+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 shutdown_connections 2023-09-30T22:14:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.584+0000 7f01f8418700 1 -- 172.21.15.89:0/4290333877 wait complete. 2023-09-30T22:14:12.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.1 2023-09-30T22:14:12.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.609+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff0a40057f0 con 0x7ff0c40965e0 2023-09-30T22:14:12.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.609+0000 7ff0b97fa700 1 -- 172.21.15.89:0/3963115002 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0c001ab80 con 0x7ff0c40965e0 2023-09-30T22:14:12.609 INFO:teuthology.orchestra.run.smithi089.stdout:38654705668 2023-09-30T22:14:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 >> v1:172.21.15.89:6816/118161 conn(0x7ff0a802bb60 legacy=0x7ff0a802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c40965e0 msgr2=0x7ff0c4136720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c40965e0 0x7ff0c4136720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7ff0c000c4f0 tx=0x7ff0c0007e00).stop 2023-09-30T22:14:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 shutdown_connections 2023-09-30T22:14:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c412d120 0x7ff0c4115e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c40965e0 0x7ff0c4136720 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 --2- 172.21.15.89:0/3963115002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c40970d0 0x7ff0c4136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 >> 172.21.15.89:0/3963115002 conn(0x7ff0c409a220 msgr2=0x7ff0c4093580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 shutdown_connections 2023-09-30T22:14:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.611+0000 7ff0cb5fe700 1 -- 172.21.15.89:0/3963115002 wait complete. 2023-09-30T22:14:12.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.2 2023-09-30T22:14:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.633+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2b8c1350f0 con 0x7f2b8c136360 2023-09-30T22:14:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.634+0000 7f2b817fa700 1 -- 172.21.15.89:0/1575522639 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b8c1350f0 con 0x7f2b8c136360 2023-09-30T22:14:12.634 INFO:teuthology.orchestra.run.smithi089.stdout:38654705669 2023-09-30T22:14:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 >> v1:172.21.15.89:6816/118161 conn(0x7f2b7002bbe0 legacy=0x7f2b7002e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c136360 msgr2=0x7f2b8c1367a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 --2- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c136360 0x7f2b8c1367a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2b8400c730 tx=0x7f2b84007ee0).stop 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 shutdown_connections 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 --2- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b8c136360 0x7f2b8c1367a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 --2- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b8c059070 0x7f2b8c132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 --2- 172.21.15.89:0/1575522639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b8c095370 0x7f2b8c135e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 >> 172.21.15.89:0/1575522639 conn(0x7f2b8c097470 msgr2=0x7f2b8c0942f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 shutdown_connections 2023-09-30T22:14:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:12.636+0000 7f2b9333a700 1 -- 172.21.15.89:0/1575522639 wait complete. 2023-09-30T22:14:12.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.3 2023-09-30T22:14:13.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:14:13.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:14:13.623 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:14:13.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:14:13.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.719+0000 7f97be502700 1 Processor -- start 2023-09-30T22:14:13.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.719+0000 7f97be502700 1 -- start start 2023-09-30T22:14:13.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97be502700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b812e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97be502700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8130980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97be502700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b8092fe0 con 0x7f97b80969b0 2023-09-30T22:14:13.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97be502700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8094560 con 0x7f97b812e590 2023-09-30T22:14:13.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97be502700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8093260 con 0x7f97b812bcb0 2023-09-30T22:14:13.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97b7fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f97b80969b0 0x7f97b8096d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54244/0 (socket says 172.21.15.89:54244) 2023-09-30T22:14:13.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97b7fff700 1 -- 172.21.15.89:0/2505756988 learned_addr learned my addr 172.21.15.89:0/2505756988 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:13.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97bca9f700 1 --2- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8130980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97b77fe700 1 --2- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b812e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97b6ffd700 1 -- 172.21.15.89:0/2505756988 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620830226 0 0) 0x7f97b8092fe0 con 0x7f97b80969b0 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97b6ffd700 1 -- 172.21.15.89:0/2505756988 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a00033f0 con 0x7f97b80969b0 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97bca9f700 1 -- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 msgr2=0x7f97b812e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97bca9f700 1 --2- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b812e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97bca9f700 1 -- 172.21.15.89:0/2505756988 >> v1:172.21.15.89:6789/0 conn(0x7f97b80969b0 legacy=0x7f97b8096d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97bca9f700 1 -- 172.21.15.89:0/2505756988 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8092b30 con 0x7f97b812e590 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.720+0000 7f97b77fe700 1 --2- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b812e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:13.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.721+0000 7f97bca9f700 1 --2- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8130980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f97b0009e90 tx=0x7f97b0009780).ready entity=mon.1 client_cookie=1b60616ebbb854d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:13.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.722+0000 7f97b6ffd700 1 -- 172.21.15.89:0/2505756988 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000e070 con 0x7f97b812e590 2023-09-30T22:14:13.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.722+0000 7f97b6ffd700 1 -- 172.21.15.89:0/2505756988 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b000ba70 con 0x7f97b812e590 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.722+0000 7f97b6ffd700 1 -- 172.21.15.89:0/2505756988 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000bdf0 con 0x7f97b812e590 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.723+0000 7f97be502700 1 -- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 msgr2=0x7f97b8130980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.723+0000 7f97be502700 1 --2- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8130980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f97b0009e90 tx=0x7f97b0009780).stop 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.723+0000 7f97be502700 1 -- 172.21.15.89:0/2505756988 shutdown_connections 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.723+0000 7f97be502700 1 --2- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8130980 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.723+0000 7f97be502700 1 --2- 172.21.15.89:0/2505756988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b812e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.723+0000 7f97be502700 1 -- 172.21.15.89:0/2505756988 >> 172.21.15.89:0/2505756988 conn(0x7f97b809a220 msgr2=0x7f97b809a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.723+0000 7f97be502700 1 -- 172.21.15.89:0/2505756988 shutdown_connections 2023-09-30T22:14:13.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.723+0000 7f97be502700 1 -- 172.21.15.89:0/2505756988 wait complete. 2023-09-30T22:14:13.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97be502700 1 Processor -- start 2023-09-30T22:14:13.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97be502700 1 -- start start 2023-09-30T22:14:13.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97be502700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97b80969b0 0x7f97b8115100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97be502700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b8128f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97be502700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8129440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97be502700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8067de0 con 0x7f97b80969b0 2023-09-30T22:14:13.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97be502700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8092d60 con 0x7f97b812e590 2023-09-30T22:14:13.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97be502700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8114660 con 0x7f97b812bcb0 2023-09-30T22:14:13.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97b77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b8128f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97b77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b8128f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34052/0 (socket says 172.21.15.89:34052) 2023-09-30T22:14:13.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.724+0000 7f97b77fe700 1 -- 172.21.15.89:0/1988819355 learned_addr learned my addr 172.21.15.89:0/1988819355 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:13.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97bca9f700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8129440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97b7fff700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97b80969b0 0x7f97b8115100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97b77fe700 1 -- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 msgr2=0x7f97b8129440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97b77fe700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8129440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97b77fe700 1 -- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97b80969b0 msgr2=0x7f97b8115100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97b77fe700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97b80969b0 0x7f97b8115100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97b77fe700 1 -- 172.21.15.89:0/1988819355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0092a0 con 0x7f97b812bcb0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97b77fe700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b8128f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f97ac00ea20 tx=0x7f97ac009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f979ffff700 1 -- 172.21.15.89:0/1988819355 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00ee60 con 0x7f97b812bcb0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f979ffff700 1 -- 172.21.15.89:0/1988819355 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac00c850 con 0x7f97b812bcb0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f979ffff700 1 -- 172.21.15.89:0/1988819355 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00cbd0 con 0x7f97b812bcb0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.725+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b0009a20 con 0x7f97b812bcb0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.726+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b80698c0 con 0x7f97b812bcb0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.727+0000 7f979ffff700 1 -- 172.21.15.89:0/1988819355 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f97ac01b400 con 0x7f97b812bcb0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.727+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b8114880 con 0x7f97b812bcb0 2023-09-30T22:14:13.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.727+0000 7f979ffff700 1 -- 172.21.15.89:0/1988819355 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f97ac03b560 con 0x7f97b812bcb0 2023-09-30T22:14:13.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.735+0000 7f979ffff700 1 -- 172.21.15.89:0/1988819355 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f97b8114880 con 0x7f97b812bcb0 2023-09-30T22:14:13.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.750+0000 7ff89f609700 1 Processor -- start 2023-09-30T22:14:13.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89f609700 1 -- start start 2023-09-30T22:14:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89f609700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff898096380 0x7ff898094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89f609700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898091b00 0x7ff898127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89f609700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898094c20 0x7ff898127af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89f609700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff898096760 con 0x7ff898096380 2023-09-30T22:14:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89f609700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff898091ff0 con 0x7ff898091b00 2023-09-30T22:14:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89f609700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff898092c80 con 0x7ff898094c20 2023-09-30T22:14:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89cba4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898091b00 0x7ff898127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89cba4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898091b00 0x7ff898127600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52150/0 (socket says 172.21.15.89:52150) 2023-09-30T22:14:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89cba4700 1 -- 172.21.15.89:0/3576909306 learned_addr learned my addr 172.21.15.89:0/3576909306 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89dba6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898094c20 0x7ff898127af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.751+0000 7ff89d3a5700 1 --2- 172.21.15.89:0/3576909306 >> v2:172.21.15.89:3300/0 conn(0x7ff898096380 0x7ff898094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff89cba4700 1 -- 172.21.15.89:0/3576909306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898094c20 msgr2=0x7ff898127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff89cba4700 1 --2- 172.21.15.89:0/3576909306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898094c20 0x7ff898127af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff89cba4700 1 -- 172.21.15.89:0/3576909306 >> v2:172.21.15.89:3300/0 conn(0x7ff898096380 msgr2=0x7ff898094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff89cba4700 1 --2- 172.21.15.89:0/3576909306 >> v2:172.21.15.89:3300/0 conn(0x7ff898096380 0x7ff898094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff89cba4700 1 -- 172.21.15.89:0/3576909306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff898093900 con 0x7ff898091b00 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff89cba4700 1 --2- 172.21.15.89:0/3576909306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898091b00 0x7ff898127600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff888006fd0 tx=0x7ff8880090f0).ready entity=mon.1 client_cookie=753523c592ac280f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff88ffff700 1 -- 172.21.15.89:0/3576909306 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880108a0 con 0x7ff898091b00 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff88ffff700 1 -- 172.21.15.89:0/3576909306 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff888010a00 con 0x7ff898091b00 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.752+0000 7ff88ffff700 1 -- 172.21.15.89:0/3576909306 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888010d80 con 0x7ff898091b00 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 -- 172.21.15.89:0/3576909306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898091b00 msgr2=0x7ff898127600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 --2- 172.21.15.89:0/3576909306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898091b00 0x7ff898127600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff888006fd0 tx=0x7ff8880090f0).stop 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 -- 172.21.15.89:0/3576909306 shutdown_connections 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 --2- 172.21.15.89:0/3576909306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898094c20 0x7ff898127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 --2- 172.21.15.89:0/3576909306 >> v2:172.21.15.89:3300/0 conn(0x7ff898096380 0x7ff898094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 --2- 172.21.15.89:0/3576909306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898091b00 0x7ff898127600 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 -- 172.21.15.89:0/3576909306 >> 172.21.15.89:0/3576909306 conn(0x7ff898097470 msgr2=0x7ff898097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 -- 172.21.15.89:0/3576909306 shutdown_connections 2023-09-30T22:14:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.753+0000 7ff89f609700 1 -- 172.21.15.89:0/3576909306 wait complete. 2023-09-30T22:14:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.754+0000 7ff89f609700 1 Processor -- start 2023-09-30T22:14:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.754+0000 7ff89f609700 1 -- start start 2023-09-30T22:14:13.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89f609700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898091b00 0x7ff898113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89f609700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff898094c20 0x7ff898113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89f609700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898096380 0x7ff898113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89f609700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8981140e0 con 0x7ff898094c20 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89f609700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff898127d60 con 0x7ff898096380 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89f609700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff89806a520 con 0x7ff898091b00 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898091b00 0x7ff898113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898091b00 0x7ff898113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34074/0 (socket says 172.21.15.89:34074) 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 -- 172.21.15.89:0/667618644 learned_addr learned my addr 172.21.15.89:0/667618644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89dba6700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898096380 0x7ff898113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89cba4700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff898094c20 0x7ff898113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 -- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898096380 msgr2=0x7ff898113bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898096380 0x7ff898113bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 -- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff898094c20 msgr2=0x7ff898113700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff898094c20 0x7ff898113700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 -- 172.21.15.89:0/667618644 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff888009370 con 0x7ff898091b00 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89cba4700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff898094c20 0x7ff898113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89d3a5700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898091b00 0x7ff898113210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff89400b9b0 tx=0x7ff89400a040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff88d7fa700 1 -- 172.21.15.89:0/667618644 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894005880 con 0x7ff898091b00 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff88d7fa700 1 -- 172.21.15.89:0/667618644 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8940059e0 con 0x7ff898091b00 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff898114fb0 con 0x7ff898091b00 2023-09-30T22:14:13.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.755+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff898117420 con 0x7ff898091b00 2023-09-30T22:14:13.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.756+0000 7ff88d7fa700 1 -- 172.21.15.89:0/667618644 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894005d60 con 0x7ff898091b00 2023-09-30T22:14:13.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.757+0000 7ff88d7fa700 1 -- 172.21.15.89:0/667618644 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ff894007320 con 0x7ff898091b00 2023-09-30T22:14:13.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.757+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89805a260 con 0x7ff898091b00 2023-09-30T22:14:13.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.757+0000 7ff88d7fa700 1 -- 172.21.15.89:0/667618644 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ff89403abd0 con 0x7ff898091b00 2023-09-30T22:14:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.762+0000 7ff88d7fa700 1 -- 172.21.15.89:0/667618644 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7ff89401c310 con 0x7ff898091b00 2023-09-30T22:14:13.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.797+0000 7f9a818f2700 1 Processor -- start 2023-09-30T22:14:13.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.797+0000 7f9a818f2700 1 -- start start 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a818f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c12da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a818f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c12df30 0x7f9a7c130320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a818f2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c125540 con 0x7f9a7c096bc0 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a818f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c096120 con 0x7f9a7c12df30 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a818f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0956a0 con 0x7f9a7c125160 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c12df30 0x7f9a7c130320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c12da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c12da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34080/0 (socket says 172.21.15.89:34080) 2023-09-30T22:14:13.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c12df30 0x7f9a7c130320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52180/0 (socket says 172.21.15.89:52180) 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7a7fc700 1 -- 172.21.15.89:0/1350468461 learned_addr learned my addr 172.21.15.89:0/1350468461 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7a7fc700 1 -- 172.21.15.89:0/1350468461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c12df30 msgr2=0x7f9a7c130320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7a7fc700 1 --2- 172.21.15.89:0/1350468461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c12df30 0x7f9a7c130320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7a7fc700 1 -- 172.21.15.89:0/1350468461 >> v1:172.21.15.89:6789/0 conn(0x7f9a7c096bc0 legacy=0x7f9a7c125780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7a7fc700 1 -- 172.21.15.89:0/1350468461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c096fa0 con 0x7f9a7c125160 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7b7fe700 1 --2- 172.21.15.89:0/1350468461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c12df30 0x7f9a7c130320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.798+0000 7f9a7a7fc700 1 --2- 172.21.15.89:0/1350468461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c12da40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f9a70009830 tx=0x7f9a700090f0).ready entity=mon.2 client_cookie=a5f8444f84bb24ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a79ffb700 1 -- 172.21.15.89:0/1350468461 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7000b580 con 0x7f9a7c125160 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a79ffb700 1 -- 172.21.15.89:0/1350468461 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7000b6e0 con 0x7f9a7c125160 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a79ffb700 1 -- 172.21.15.89:0/1350468461 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7000ba60 con 0x7f9a7c125160 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a818f2700 1 -- 172.21.15.89:0/1350468461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 msgr2=0x7f9a7c12da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a818f2700 1 --2- 172.21.15.89:0/1350468461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c12da40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f9a70009830 tx=0x7f9a700090f0).stop 2023-09-30T22:14:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a818f2700 1 -- 172.21.15.89:0/1350468461 shutdown_connections 2023-09-30T22:14:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a818f2700 1 --2- 172.21.15.89:0/1350468461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c12df30 0x7f9a7c130320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a818f2700 1 --2- 172.21.15.89:0/1350468461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c12da40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a818f2700 1 -- 172.21.15.89:0/1350468461 >> 172.21.15.89:0/1350468461 conn(0x7f9a7c097470 msgr2=0x7f9a7c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a818f2700 1 -- 172.21.15.89:0/1350468461 shutdown_connections 2023-09-30T22:14:13.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.799+0000 7f9a818f2700 1 -- 172.21.15.89:0/1350468461 wait complete. 2023-09-30T22:14:13.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a818f2700 1 Processor -- start 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a818f2700 1 -- start start 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a818f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c096bc0 0x7f9a7c123af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a818f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c123fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a818f2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c12df30 0x7f9a7c12bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a818f2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0958d0 con 0x7f9a7c12df30 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a818f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c099af0 con 0x7f9a7c096bc0 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a818f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c092ea0 con 0x7f9a7c125160 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a7affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c096bc0 0x7f9a7c123af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a7affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c096bc0 0x7f9a7c123af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52184/0 (socket says 172.21.15.89:52184) 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.801+0000 7f9a7affd700 1 -- 172.21.15.89:0/20747183 learned_addr learned my addr 172.21.15.89:0/20747183 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a7a7fc700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c123fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a7b7fe700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c12df30 0x7f9a7c12bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a7affd700 1 -- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 msgr2=0x7f9a7c123fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a7affd700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c123fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a7affd700 1 -- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c12df30 msgr2=0x7f9a7c12bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a7affd700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c12df30 0x7f9a7c12bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a7affd700 1 -- 172.21.15.89:0/20747183 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74009a20 con 0x7f9a7c096bc0 2023-09-30T22:14:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a7affd700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c096bc0 0x7f9a7c123af0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9a6c00c4c0 tx=0x7f9a6c007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:13.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f228d3bb700 1 Processor -- start 2023-09-30T22:14:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a62ffd700 1 -- 172.21.15.89:0/20747183 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00ce70 con 0x7f9a7c096bc0 2023-09-30T22:14:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f228d3bb700 1 -- start start 2023-09-30T22:14:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a62ffd700 1 -- 172.21.15.89:0/20747183 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c001e90 con 0x7f9a7c096bc0 2023-09-30T22:14:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a70009370 con 0x7f9a7c096bc0 2023-09-30T22:14:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a62ffd700 1 -- 172.21.15.89:0/20747183 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c007270 con 0x7f9a7c096bc0 2023-09-30T22:14:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.802+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c092b10 con 0x7f9a7c096bc0 2023-09-30T22:14:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.803+0000 7f228d3bb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22880986c0 0x7f2288130110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.803+0000 7f228d3bb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288130600 0x7f22881329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.803+0000 7f228d3bb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228806a250 con 0x7f2288097bd0 2023-09-30T22:14:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.803+0000 7f228d3bb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2288092530 con 0x7f22880986c0 2023-09-30T22:14:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.803+0000 7f228d3bb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22880921e0 con 0x7f2288130600 2023-09-30T22:14:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.803+0000 7f2286ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2288097bd0 0x7f2288097fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54276/0 (socket says 172.21.15.89:54276) 2023-09-30T22:14:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.803+0000 7f2286ffd700 1 -- 172.21.15.89:0/1198715136 learned_addr learned my addr 172.21.15.89:0/1198715136 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.803+0000 7f22877fe700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288130600 0x7f22881329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f9a62ffd700 1 -- 172.21.15.89:0/20747183 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9a6c0066a0 con 0x7f9a7c096bc0 2023-09-30T22:14:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c0930c0 con 0x7f9a7c096bc0 2023-09-30T22:14:13.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86247648 0 0) 0x7f228806a250 con 0x7f2288097bd0 2023-09-30T22:14:13.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22700033f0 con 0x7f2288097bd0 2023-09-30T22:14:13.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f22867fc700 1 --2- 172.21.15.89:0/1198715136 >> v2:172.21.15.89:3300/0 conn(0x7f22880986c0 0x7f2288130110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1719199677 0 0) 0x7f22700033f0 con 0x7f2288097bd0 2023-09-30T22:14:13.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288130600 msgr2=0x7f22881329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:13.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288130600 0x7f22881329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f9a62ffd700 1 -- 172.21.15.89:0/20747183 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f9a6c03a7e0 con 0x7f9a7c096bc0 2023-09-30T22:14:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 >> v2:172.21.15.89:3300/0 conn(0x7f22880986c0 msgr2=0x7f2288130110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 --2- 172.21.15.89:0/1198715136 >> v2:172.21.15.89:3300/0 conn(0x7f22880986c0 0x7f2288130110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880996b0 con 0x7f2288097bd0 2023-09-30T22:14:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1128284152 0 0) 0x7f2280001470 con 0x7f2288097bd0 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 >> v1:172.21.15.89:6789/0 conn(0x7f2288097bd0 legacy=0x7f2288097fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2270003960 0x7f2270003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2270004690 0x7f2270004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f227000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2288092530 con 0x7f2270005000 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880921e0 con 0x7f2270003960 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f227000bdd0 con 0x7f2270004690 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.804+0000 7f22877fe700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288130600 0x7f22881329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22877fe700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f227000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f2286ffd700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2270003960 0x7f2270003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22867fc700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2270004690 0x7f2270004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22877fe700 1 -- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2270004690 msgr2=0x7f2270004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22877fe700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2270004690 0x7f2270004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22877fe700 1 -- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2270003960 msgr2=0x7f2270003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22877fe700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2270003960 0x7f2270003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22877fe700 1 -- 172.21.15.89:0/1198715136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880996b0 con 0x7f2270005000 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22867fc700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2270004690 0x7f2270004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f22877fe700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f227000b8e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f22880995e0 tx=0x7f227800b040).ready entity=mon.0 client_cookie=97ed9c6281b8035b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800ea70 con 0x7f2270005000 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227800f040 con 0x7f2270005000 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.805+0000 7f2285ffb700 1 -- 172.21.15.89:0/1198715136 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800ee00 con 0x7f2270005000 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.806+0000 7f228d3bb700 1 -- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 msgr2=0x7f227000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.806+0000 7f228d3bb700 1 --2- 172.21.15.89:0/1198715136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f227000b8e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f22880995e0 tx=0x7f227800b040).stop 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.806+0000 7f22877fe700 1 -- 172.21.15.89:0/1198715136 reap_dead start 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.806+0000 7f228d3bb700 1 -- 172.21.15.89:0/1198715136 shutdown_connections 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.806+0000 7f228d3bb700 1 -- 172.21.15.89:0/1198715136 >> 172.21.15.89:0/1198715136 conn(0x7f22880998d0 msgr2=0x7f2288099ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.806+0000 7f228d3bb700 1 -- 172.21.15.89:0/1198715136 shutdown_connections 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.806+0000 7f228d3bb700 1 -- 172.21.15.89:0/1198715136 wait complete. 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f228d3bb700 1 Processor -- start 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f228d3bb700 1 -- start start 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f228d3bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f228811b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f228d3bb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2288117700 0x7f2288117b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f228d3bb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288118070 0x7f2288140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f228d3bb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2288118b90 con 0x7f2270005000 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f228d3bb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228812d8b0 con 0x7f2288117700 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f228d3bb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2288119dc0 con 0x7f2288118070 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f2286ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f228811b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f2286ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f228811b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36784/0 (socket says 172.21.15.89:36784) 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f2286ffd700 1 -- 172.21.15.89:0/3772903658 learned_addr learned my addr 172.21.15.89:0/3772903658 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.808+0000 7f22877fe700 1 --2- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288118070 0x7f2288140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f2286ffd700 1 -- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288118070 msgr2=0x7f2288140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f2286ffd700 1 --2- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288118070 0x7f2288140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f2286ffd700 1 -- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2288117700 msgr2=0x7f2288117b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f2286ffd700 1 --2- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2288117700 0x7f2288117b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f2286ffd700 1 -- 172.21.15.89:0/3772903658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288099320 con 0x7f2270005000 2023-09-30T22:14:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f2286ffd700 1 --2- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f228811b730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f2280004a60 tx=0x7f2280001430).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f226effd700 1 -- 172.21.15.89:0/3772903658 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000f870 con 0x7f2270005000 2023-09-30T22:14:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f226effd700 1 -- 172.21.15.89:0/3772903658 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228000f9d0 con 0x7f2270005000 2023-09-30T22:14:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228811ac90 con 0x7f2270005000 2023-09-30T22:14:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22881184f0 con 0x7f2270005000 2023-09-30T22:14:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.809+0000 7f226effd700 1 -- 172.21.15.89:0/3772903658 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000fd50 con 0x7f2270005000 2023-09-30T22:14:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.810+0000 7f9a62ffd700 1 -- 172.21.15.89:0/20747183 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9a7c0930c0 con 0x7f9a7c096bc0 2023-09-30T22:14:13.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.810+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2268005120 con 0x7f2270005000 2023-09-30T22:14:13.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.810+0000 7f226effd700 1 -- 172.21.15.89:0/3772903658 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2280015620 con 0x7f2270005000 2023-09-30T22:14:13.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.811+0000 7f226effd700 1 -- 172.21.15.89:0/3772903658 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f228003b3e0 con 0x7f2270005000 2023-09-30T22:14:13.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.815+0000 7f226effd700 1 -- 172.21.15.89:0/3772903658 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f228001b6a0 con 0x7f2270005000 2023-09-30T22:14:13.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.910+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f97b8114880 con 0x7f97b812bcb0 2023-09-30T22:14:13.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.911+0000 7f979ffff700 1 -- 172.21.15.89:0/1988819355 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97b8114880 con 0x7f97b812bcb0 2023-09-30T22:14:13.910 INFO:teuthology.orchestra.run.smithi089.stdout:38654705669 2023-09-30T22:14:13.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 >> v1:172.21.15.89:6816/118161 conn(0x7f97a402bb60 legacy=0x7f97a402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 msgr2=0x7f97b8128f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b8128f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f97ac00ea20 tx=0x7f97ac009060).stop 2023-09-30T22:14:13.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 shutdown_connections 2023-09-30T22:14:13.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97b812e590 0x7f97b8129440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97b80969b0 0x7f97b8115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 --2- 172.21.15.89:0/1988819355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97b812bcb0 0x7f97b8128f50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 >> 172.21.15.89:0/1988819355 conn(0x7f97b809a220 msgr2=0x7f97b8067490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:13.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 shutdown_connections 2023-09-30T22:14:13.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.913+0000 7f97be502700 1 -- 172.21.15.89:0/1988819355 wait complete. 2023-09-30T22:14:13.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2023-09-30T22:14:13.924 DEBUG:teuthology.parallel:result is None 2023-09-30T22:14:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.935+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff8980942e0 con 0x7ff898091b00 2023-09-30T22:14:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.935+0000 7ff88d7fa700 1 -- 172.21.15.89:0/667618644 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff894013b30 con 0x7ff898091b00 2023-09-30T22:14:13.935 INFO:teuthology.orchestra.run.smithi089.stdout:38654705670 2023-09-30T22:14:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 >> v1:172.21.15.89:6816/118161 conn(0x7ff87c02bb60 legacy=0x7ff87c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898091b00 msgr2=0x7ff898113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898091b00 0x7ff898113210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff89400b9b0 tx=0x7ff89400a040).stop 2023-09-30T22:14:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 shutdown_connections 2023-09-30T22:14:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff898096380 0x7ff898113bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff898091b00 0x7ff898113210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 --2- 172.21.15.89:0/667618644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff898094c20 0x7ff898113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 >> 172.21.15.89:0/667618644 conn(0x7ff898097470 msgr2=0x7ff898059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 shutdown_connections 2023-09-30T22:14:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.937+0000 7ff89f609700 1 -- 172.21.15.89:0/667618644 wait complete. 2023-09-30T22:14:13.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2023-09-30T22:14:13.948 DEBUG:teuthology.parallel:result is None 2023-09-30T22:14:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.994+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9a7c125540 con 0x7f9a7c096bc0 2023-09-30T22:14:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.994+0000 7f9a62ffd700 1 -- 172.21.15.89:0/20747183 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a7c125540 con 0x7f9a7c096bc0 2023-09-30T22:14:13.994 INFO:teuthology.orchestra.run.smithi089.stdout:38654705669 2023-09-30T22:14:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 >> v1:172.21.15.89:6816/118161 conn(0x7f9a6802bb60 legacy=0x7f9a6802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c096bc0 msgr2=0x7f9a7c123af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c096bc0 0x7f9a7c123af0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9a6c00c4c0 tx=0x7f9a6c007e00).stop 2023-09-30T22:14:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 shutdown_connections 2023-09-30T22:14:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a7c12df30 0x7f9a7c12bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a7c096bc0 0x7f9a7c123af0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 --2- 172.21.15.89:0/20747183 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a7c125160 0x7f9a7c123fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 >> 172.21.15.89:0/20747183 conn(0x7f9a7c097470 msgr2=0x7f9a7c0664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 shutdown_connections 2023-09-30T22:14:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.996+0000 7f9a818f2700 1 -- 172.21.15.89:0/20747183 wait complete. 2023-09-30T22:14:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:13.999+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f22680057f0 con 0x7f2270005000 2023-09-30T22:14:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.000+0000 7f226effd700 1 -- 172.21.15.89:0/3772903658 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f228001b2b0 con 0x7f2270005000 2023-09-30T22:14:13.999 INFO:teuthology.orchestra.run.smithi089.stdout:38654705670 2023-09-30T22:14:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 >> v1:172.21.15.89:6816/118161 conn(0x7f227402bbb0 legacy=0x7f227402e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 msgr2=0x7f228811b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 --2- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f228811b730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f2280004a60 tx=0x7f2280001430).stop 2023-09-30T22:14:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 shutdown_connections 2023-09-30T22:14:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 --2- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2288118070 0x7f2288140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 --2- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2270005000 0x7f228811b730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 --2- 172.21.15.89:0/3772903658 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2288117700 0x7f2288117b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:14:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 >> 172.21.15.89:0/3772903658 conn(0x7f22880998d0 msgr2=0x7f2288111880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 shutdown_connections 2023-09-30T22:14:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.003+0000 7f228d3bb700 1 -- 172.21.15.89:0/3772903658 wait complete. 2023-09-30T22:14:14.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2023-09-30T22:14:14.011 DEBUG:teuthology.parallel:result is None 2023-09-30T22:14:14.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.3 2023-09-30T22:14:14.017 DEBUG:teuthology.parallel:result is None 2023-09-30T22:14:14.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:14:14.017 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:14:14.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9ed6c9700 1 Processor -- start 2023-09-30T22:14:14.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9ed6c9700 1 -- start start 2023-09-30T22:14:14.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9ed6c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e80986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9ed6c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8097c30 0x7fb9e8098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9ed6c9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e8096160 con 0x7fb9e8099080 2023-09-30T22:14:14.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9ed6c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80654d0 con 0x7fb9e8097c30 2023-09-30T22:14:14.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9ed6c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8099510 con 0x7fb9e80650f0 2023-09-30T22:14:14.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9e6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e80986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9e67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8097c30 0x7fb9e8098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9e77fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb9e8099080 0x7fb9e8131470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54280/0 (socket says 172.21.15.89:54280) 2023-09-30T22:14:14.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.173+0000 7fb9e77fe700 1 -- 172.21.15.89:0/916501670 learned_addr learned my addr 172.21.15.89:0/916501670 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:14.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e67fc700 1 -- 172.21.15.89:0/916501670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 msgr2=0x7fb9e80986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:14.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e67fc700 1 --2- 172.21.15.89:0/916501670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e80986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e5ffb700 1 -- 172.21.15.89:0/916501670 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972532369 0 0) 0x7fb9e8096160 con 0x7fb9e8099080 2023-09-30T22:14:14.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e67fc700 1 -- 172.21.15.89:0/916501670 >> v1:172.21.15.89:6789/0 conn(0x7fb9e8099080 legacy=0x7fb9e8131470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e67fc700 1 -- 172.21.15.89:0/916501670 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e8099790 con 0x7fb9e8097c30 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e67fc700 1 --2- 172.21.15.89:0/916501670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8097c30 0x7fb9e8098b90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb9dc0097c0 tx=0x7fb9dc009060).ready entity=mon.1 client_cookie=1d249240a369e956 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e5ffb700 1 -- 172.21.15.89:0/916501670 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00f510 con 0x7fb9e8097c30 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e5ffb700 1 -- 172.21.15.89:0/916501670 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc00f670 con 0x7fb9e8097c30 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9e5ffb700 1 -- 172.21.15.89:0/916501670 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00f9f0 con 0x7fb9e8097c30 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/916501670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8097c30 msgr2=0x7fb9e8098b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.174+0000 7fb9ed6c9700 1 --2- 172.21.15.89:0/916501670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8097c30 0x7fb9e8098b90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb9dc0097c0 tx=0x7fb9dc009060).stop 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.175+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/916501670 shutdown_connections 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.175+0000 7fb9ed6c9700 1 --2- 172.21.15.89:0/916501670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e80986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.175+0000 7fb9ed6c9700 1 --2- 172.21.15.89:0/916501670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8097c30 0x7fb9e8098b90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.175+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/916501670 >> 172.21.15.89:0/916501670 conn(0x7fb9e809dc70 msgr2=0x7fb9e809e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.175+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/916501670 shutdown_connections 2023-09-30T22:14:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.175+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/916501670 wait complete. 2023-09-30T22:14:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.175+0000 7fb9ed6c9700 1 Processor -- start 2023-09-30T22:14:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9ed6c9700 1 -- start start 2023-09-30T22:14:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9ed6c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e813a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9ed6c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8099080 0x7fb9e813a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9ed6c9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e813adb0 0x7fb9e811ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9ed6c9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8099a80 con 0x7fb9e813adb0 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9ed6c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e809cb50 con 0x7fb9e8099080 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9ed6c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e813b1d0 con 0x7fb9e80650f0 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8099080 0x7fb9e813a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8099080 0x7fb9e813a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52214/0 (socket says 172.21.15.89:52214) 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e67fc700 1 -- 172.21.15.89:0/172161419 learned_addr learned my addr 172.21.15.89:0/172161419 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e6ffd700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e813a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e77fe700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e813adb0 0x7fb9e811ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e6ffd700 1 -- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8099080 msgr2=0x7fb9e813a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e6ffd700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8099080 0x7fb9e813a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e6ffd700 1 -- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e813adb0 msgr2=0x7fb9e811ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e6ffd700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e813adb0 0x7fb9e811ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e6ffd700 1 -- 172.21.15.89:0/172161419 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc0092a0 con 0x7fb9e80650f0 2023-09-30T22:14:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.176+0000 7fb9e6ffd700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e813a3d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb9d800e930 tx=0x7fb9d8009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.177+0000 7fb9ceffd700 1 -- 172.21.15.89:0/172161419 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8009450 con 0x7fb9e80650f0 2023-09-30T22:14:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.177+0000 7fb9ceffd700 1 -- 172.21.15.89:0/172161419 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d8012600 con 0x7fb9e80650f0 2023-09-30T22:14:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.177+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e811cbd0 con 0x7fb9e80650f0 2023-09-30T22:14:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.177+0000 7fb9ceffd700 1 -- 172.21.15.89:0/172161419 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8012980 con 0x7fb9e80650f0 2023-09-30T22:14:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.177+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e811c1e0 con 0x7fb9e80650f0 2023-09-30T22:14:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.177+0000 7fb9ceffd700 1 -- 172.21.15.89:0/172161419 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb9d801b400 con 0x7fb9e80650f0 2023-09-30T22:14:14.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.177+0000 7fb9ceffd700 1 -- 172.21.15.89:0/172161419 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb9d803b390 con 0x7fb9e80650f0 2023-09-30T22:14:14.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.178+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e805a260 con 0x7fb9e80650f0 2023-09-30T22:14:14.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.182+0000 7fb9ceffd700 1 -- 172.21.15.89:0/172161419 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fb9d8019310 con 0x7fb9e80650f0 2023-09-30T22:14:14.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.337+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 --> v1:172.21.15.89:6816/118161 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9e80973a0 con 0x7fb9d402bb60 2023-09-30T22:14:14.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.343+0000 7fb9ceffd700 1 -- 172.21.15.89:0/172161419 <== mgr.4102 v1:172.21.15.89:6816/118161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249947 (unknown 2965378022 0 82204692) 0x7fb9e80973a0 con 0x7fb9d402bb60 2023-09-30T22:14:14.345 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:14.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 >> v1:172.21.15.89:6816/118161 conn(0x7fb9d402bb60 legacy=0x7fb9d402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 msgr2=0x7fb9e813a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e813a3d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb9d800e930 tx=0x7fb9d8009060).stop 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 shutdown_connections 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e813adb0 0x7fb9e811ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e80650f0 0x7fb9e813a3d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 --2- 172.21.15.89:0/172161419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e8099080 0x7fb9e813a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 >> 172.21.15.89:0/172161419 conn(0x7fb9e809dc70 msgr2=0x7fb9e81142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 shutdown_connections 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.346+0000 7fb9ed6c9700 1 -- 172.21.15.89:0/172161419 wait complete. 2023-09-30T22:14:14.348 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:14:14.357 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-30T22:14:12.747317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001520"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090297+0000","last_change":"2023-09-30T22:14:02.086445+0000","last_active":"2023-09-30T22:14:02.090297+0000","last_peered":"2023-09-30T22:14:02.090297+0000","last_clean":"2023-09-30T22:14:02.090297+0000","last_became_active":"2023-09-30T22:14:02.086159+0000","last_became_peered":"2023-09-30T22:14:02.086159+0000","last_unstale":"2023-09-30T22:14:02.090297+0000","last_undegraded":"2023-09-30T22:14:02.090297+0000","last_fullsized":"2023-09-30T22:14:02.090297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088940+0000","last_change":"2023-09-30T22:14:02.082560+0000","last_active":"2023-09-30T22:14:02.088940+0000","last_peered":"2023-09-30T22:14:02.088940+0000","last_clean":"2023-09-30T22:14:02.088940+0000","last_became_active":"2023-09-30T22:14:02.082315+0000","last_became_peered":"2023-09-30T22:14:02.082315+0000","last_unstale":"2023-09-30T22:14:02.088940+0000","last_undegraded":"2023-09-30T22:14:02.088940+0000","last_fullsized":"2023-09-30T22:14:02.088940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091544+0000","last_change":"2023-09-30T22:14:02.082673+0000","last_active":"2023-09-30T22:14:02.091544+0000","last_peered":"2023-09-30T22:14:02.091544+0000","last_clean":"2023-09-30T22:14:02.091544+0000","last_became_active":"2023-09-30T22:14:02.082437+0000","last_became_peered":"2023-09-30T22:14:02.082437+0000","last_unstale":"2023-09-30T22:14:02.091544+0000","last_undegraded":"2023-09-30T22:14:02.091544+0000","last_fullsized":"2023-09-30T22:14:02.091544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091649+0000","last_change":"2023-09-30T22:14:02.080886+0000","last_active":"2023-09-30T22:14:02.091649+0000","last_peered":"2023-09-30T22:14:02.091649+0000","last_clean":"2023-09-30T22:14:02.091649+0000","last_became_active":"2023-09-30T22:14:02.078093+0000","last_became_peered":"2023-09-30T22:14:02.078093+0000","last_unstale":"2023-09-30T22:14:02.091649+0000","last_undegraded":"2023-09-30T22:14:02.091649+0000","last_fullsized":"2023-09-30T22:14:02.091649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087971+0000","last_change":"2023-09-30T22:14:02.074967+0000","last_active":"2023-09-30T22:14:02.087971+0000","last_peered":"2023-09-30T22:14:02.087971+0000","last_clean":"2023-09-30T22:14:02.087971+0000","last_became_active":"2023-09-30T22:14:02.074701+0000","last_became_peered":"2023-09-30T22:14:02.074701+0000","last_unstale":"2023-09-30T22:14:02.087971+0000","last_undegraded":"2023-09-30T22:14:02.087971+0000","last_fullsized":"2023-09-30T22:14:02.087971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087513+0000","last_change":"2023-09-30T22:14:02.081443+0000","last_active":"2023-09-30T22:14:02.087513+0000","last_peered":"2023-09-30T22:14:02.087513+0000","last_clean":"2023-09-30T22:14:02.087513+0000","last_became_active":"2023-09-30T22:14:02.080408+0000","last_became_peered":"2023-09-30T22:14:02.080408+0000","last_unstale":"2023-09-30T22:14:02.087513+0000","last_undegraded":"2023-09-30T22:14:02.087513+0000","last_fullsized":"2023-09-30T22:14:02.087513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093729+0000","last_change":"2023-09-30T22:14:02.085310+0000","last_active":"2023-09-30T22:14:02.093729+0000","last_peered":"2023-09-30T22:14:02.093729+0000","last_clean":"2023-09-30T22:14:02.093729+0000","last_became_active":"2023-09-30T22:14:02.084647+0000","last_became_peered":"2023-09-30T22:14:02.084647+0000","last_unstale":"2023-09-30T22:14:02.093729+0000","last_undegraded":"2023-09-30T22:14:02.093729+0000","last_fullsized":"2023-09-30T22:14:02.093729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093827+0000","last_change":"2023-09-30T22:14:02.085988+0000","last_active":"2023-09-30T22:14:02.093827+0000","last_peered":"2023-09-30T22:14:02.093827+0000","last_clean":"2023-09-30T22:14:02.093827+0000","last_became_active":"2023-09-30T22:14:02.083245+0000","last_became_peered":"2023-09-30T22:14:02.083245+0000","last_unstale":"2023-09-30T22:14:02.093827+0000","last_undegraded":"2023-09-30T22:14:02.093827+0000","last_fullsized":"2023-09-30T22:14:02.093827+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091844+0000","last_change":"2023-09-30T22:14:02.085522+0000","last_active":"2023-09-30T22:14:02.091844+0000","last_peered":"2023-09-30T22:14:02.091844+0000","last_clean":"2023-09-30T22:14:02.091844+0000","last_became_active":"2023-09-30T22:14:02.084959+0000","last_became_peered":"2023-09-30T22:14:02.084959+0000","last_unstale":"2023-09-30T22:14:02.091844+0000","last_undegraded":"2023-09-30T22:14:02.091844+0000","last_fullsized":"2023-09-30T22:14:02.091844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.078675+0000","last_change":"2023-09-30T22:14:02.071672+0000","last_active":"2023-09-30T22:14:02.078675+0000","last_peered":"2023-09-30T22:14:02.078675+0000","last_clean":"2023-09-30T22:14:02.078675+0000","last_became_active":"2023-09-30T22:14:02.071361+0000","last_became_peered":"2023-09-30T22:14:02.071361+0000","last_unstale":"2023-09-30T22:14:02.078675+0000","last_undegraded":"2023-09-30T22:14:02.078675+0000","last_fullsized":"2023-09-30T22:14:02.078675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.077649+0000","last_change":"2023-09-30T22:14:02.074689+0000","last_active":"2023-09-30T22:14:02.077649+0000","last_peered":"2023-09-30T22:14:02.077649+0000","last_clean":"2023-09-30T22:14:02.077649+0000","last_became_active":"2023-09-30T22:14:02.074350+0000","last_became_peered":"2023-09-30T22:14:02.074350+0000","last_unstale":"2023-09-30T22:14:02.077649+0000","last_undegraded":"2023-09-30T22:14:02.077649+0000","last_fullsized":"2023-09-30T22:14:02.077649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090695+0000","last_change":"2023-09-30T22:14:02.086959+0000","last_active":"2023-09-30T22:14:02.090695+0000","last_peered":"2023-09-30T22:14:02.090695+0000","last_clean":"2023-09-30T22:14:02.090695+0000","last_became_active":"2023-09-30T22:14:02.086719+0000","last_became_peered":"2023-09-30T22:14:02.086719+0000","last_unstale":"2023-09-30T22:14:02.090695+0000","last_undegraded":"2023-09-30T22:14:02.090695+0000","last_fullsized":"2023-09-30T22:14:02.090695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090392+0000","last_change":"2023-09-30T22:14:02.087347+0000","last_active":"2023-09-30T22:14:02.090392+0000","last_peered":"2023-09-30T22:14:02.090392+0000","last_clean":"2023-09-30T22:14:02.090392+0000","last_became_active":"2023-09-30T22:14:02.086637+0000","last_became_peered":"2023-09-30T22:14:02.086637+0000","last_unstale":"2023-09-30T22:14:02.090392+0000","last_undegraded":"2023-09-30T22:14:02.090392+0000","last_fullsized":"2023-09-30T22:14:02.090392+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.073518+0000","last_change":"2023-09-30T22:14:02.066399+0000","last_active":"2023-09-30T22:14:02.073518+0000","last_peered":"2023-09-30T22:14:02.073518+0000","last_clean":"2023-09-30T22:14:02.073518+0000","last_became_active":"2023-09-30T22:14:02.066052+0000","last_became_peered":"2023-09-30T22:14:02.066052+0000","last_unstale":"2023-09-30T22:14:02.073518+0000","last_undegraded":"2023-09-30T22:14:02.073518+0000","last_fullsized":"2023-09-30T22:14:02.073518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083856+0000","last_change":"2023-09-30T22:14:02.073165+0000","last_active":"2023-09-30T22:14:02.083856+0000","last_peered":"2023-09-30T22:14:02.083856+0000","last_clean":"2023-09-30T22:14:02.083856+0000","last_became_active":"2023-09-30T22:14:02.073000+0000","last_became_peered":"2023-09-30T22:14:02.073000+0000","last_unstale":"2023-09-30T22:14:02.083856+0000","last_undegraded":"2023-09-30T22:14:02.083856+0000","last_fullsized":"2023-09-30T22:14:02.083856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083597+0000","last_change":"2023-09-30T22:14:02.072772+0000","last_active":"2023-09-30T22:14:02.083597+0000","last_peered":"2023-09-30T22:14:02.083597+0000","last_clean":"2023-09-30T22:14:02.083597+0000","last_became_active":"2023-09-30T22:14:02.072569+0000","last_became_peered":"2023-09-30T22:14:02.072569+0000","last_unstale":"2023-09-30T22:14:02.083597+0000","last_undegraded":"2023-09-30T22:14:02.083597+0000","last_fullsized":"2023-09-30T22:14:02.083597+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091862+0000","last_change":"2023-09-30T22:14:02.085679+0000","last_active":"2023-09-30T22:14:02.091862+0000","last_peered":"2023-09-30T22:14:02.091862+0000","last_clean":"2023-09-30T22:14:02.091862+0000","last_became_active":"2023-09-30T22:14:02.085267+0000","last_became_peered":"2023-09-30T22:14:02.085267+0000","last_unstale":"2023-09-30T22:14:02.091862+0000","last_undegraded":"2023-09-30T22:14:02.091862+0000","last_fullsized":"2023-09-30T22:14:02.091862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082225+0000","last_change":"2023-09-30T22:14:02.070008+0000","last_active":"2023-09-30T22:14:02.082225+0000","last_peered":"2023-09-30T22:14:02.082225+0000","last_clean":"2023-09-30T22:14:02.082225+0000","last_became_active":"2023-09-30T22:14:02.069809+0000","last_became_peered":"2023-09-30T22:14:02.069809+0000","last_unstale":"2023-09-30T22:14:02.082225+0000","last_undegraded":"2023-09-30T22:14:02.082225+0000","last_fullsized":"2023-09-30T22:14:02.082225+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090720+0000","last_change":"2023-09-30T22:14:02.087939+0000","last_active":"2023-09-30T22:14:02.090720+0000","last_peered":"2023-09-30T22:14:02.090720+0000","last_clean":"2023-09-30T22:14:02.090720+0000","last_became_active":"2023-09-30T22:14:02.087733+0000","last_became_peered":"2023-09-30T22:14:02.087733+0000","last_unstale":"2023-09-30T22:14:02.090720+0000","last_undegraded":"2023-09-30T22:14:02.090720+0000","last_fullsized":"2023-09-30T22:14:02.090720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093991+0000","last_change":"2023-09-30T22:14:02.090627+0000","last_active":"2023-09-30T22:14:02.093991+0000","last_peered":"2023-09-30T22:14:02.093991+0000","last_clean":"2023-09-30T22:14:02.093991+0000","last_became_active":"2023-09-30T22:14:02.090255+0000","last_became_peered":"2023-09-30T22:14:02.090255+0000","last_unstale":"2023-09-30T22:14:02.093991+0000","last_undegraded":"2023-09-30T22:14:02.093991+0000","last_fullsized":"2023-09-30T22:14:02.093991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.076392+0000","last_change":"2023-09-30T22:14:02.071417+0000","last_active":"2023-09-30T22:14:02.076392+0000","last_peered":"2023-09-30T22:14:02.076392+0000","last_clean":"2023-09-30T22:14:02.076392+0000","last_became_active":"2023-09-30T22:14:02.071046+0000","last_became_peered":"2023-09-30T22:14:02.071046+0000","last_unstale":"2023-09-30T22:14:02.076392+0000","last_undegraded":"2023-09-30T22:14:02.076392+0000","last_fullsized":"2023-09-30T22:14:02.076392+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088095+0000","last_change":"2023-09-30T22:14:02.076131+0000","last_active":"2023-09-30T22:14:02.088095+0000","last_peered":"2023-09-30T22:14:02.088095+0000","last_clean":"2023-09-30T22:14:02.088095+0000","last_became_active":"2023-09-30T22:14:02.075850+0000","last_became_peered":"2023-09-30T22:14:02.075850+0000","last_unstale":"2023-09-30T22:14:02.088095+0000","last_undegraded":"2023-09-30T22:14:02.088095+0000","last_fullsized":"2023-09-30T22:14:02.088095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093707+0000","last_change":"2023-09-30T22:14:02.085210+0000","last_active":"2023-09-30T22:14:02.093707+0000","last_peered":"2023-09-30T22:14:02.093707+0000","last_clean":"2023-09-30T22:14:02.093707+0000","last_became_active":"2023-09-30T22:14:02.084320+0000","last_became_peered":"2023-09-30T22:14:02.084320+0000","last_unstale":"2023-09-30T22:14:02.093707+0000","last_undegraded":"2023-09-30T22:14:02.093707+0000","last_fullsized":"2023-09-30T22:14:02.093707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093806+0000","last_change":"2023-09-30T22:14:02.084642+0000","last_active":"2023-09-30T22:14:02.093806+0000","last_peered":"2023-09-30T22:14:02.093806+0000","last_clean":"2023-09-30T22:14:02.093806+0000","last_became_active":"2023-09-30T22:14:02.084358+0000","last_became_peered":"2023-09-30T22:14:02.084358+0000","last_unstale":"2023-09-30T22:14:02.093806+0000","last_undegraded":"2023-09-30T22:14:02.093806+0000","last_fullsized":"2023-09-30T22:14:02.093806+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.098581+0000","last_change":"2023-09-30T22:14:02.093959+0000","last_active":"2023-09-30T22:14:02.098581+0000","last_peered":"2023-09-30T22:14:02.098581+0000","last_clean":"2023-09-30T22:14:02.098581+0000","last_became_active":"2023-09-30T22:14:02.093149+0000","last_became_peered":"2023-09-30T22:14:02.093149+0000","last_unstale":"2023-09-30T22:14:02.098581+0000","last_undegraded":"2023-09-30T22:14:02.098581+0000","last_fullsized":"2023-09-30T22:14:02.098581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079587+0000","last_change":"2023-09-30T22:14:02.077989+0000","last_active":"2023-09-30T22:14:02.079587+0000","last_peered":"2023-09-30T22:14:02.079587+0000","last_clean":"2023-09-30T22:14:02.079587+0000","last_became_active":"2023-09-30T22:14:02.071401+0000","last_became_peered":"2023-09-30T22:14:02.071401+0000","last_unstale":"2023-09-30T22:14:02.079587+0000","last_undegraded":"2023-09-30T22:14:02.079587+0000","last_fullsized":"2023-09-30T22:14:02.079587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085651+0000","last_change":"2023-09-30T22:14:02.075617+0000","last_active":"2023-09-30T22:14:02.085651+0000","last_peered":"2023-09-30T22:14:02.085651+0000","last_clean":"2023-09-30T22:14:02.085651+0000","last_became_active":"2023-09-30T22:14:02.074584+0000","last_became_peered":"2023-09-30T22:14:02.074584+0000","last_unstale":"2023-09-30T22:14:02.085651+0000","last_undegraded":"2023-09-30T22:14:02.085651+0000","last_fullsized":"2023-09-30T22:14:02.085651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.073438+0000","last_change":"2023-09-30T22:14:01.697398+0000","last_active":"2023-09-30T22:14:02.073438+0000","last_peered":"2023-09-30T22:14:02.073438+0000","last_clean":"2023-09-30T22:14:02.073438+0000","last_became_active":"2023-09-30T22:14:01.694824+0000","last_became_peered":"2023-09-30T22:14:01.694824+0000","last_unstale":"2023-09-30T22:14:02.073438+0000","last_undegraded":"2023-09-30T22:14:02.073438+0000","last_fullsized":"2023-09-30T22:14:02.073438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085672+0000","last_change":"2023-09-30T22:14:02.075771+0000","last_active":"2023-09-30T22:14:02.085672+0000","last_peered":"2023-09-30T22:14:02.085672+0000","last_clean":"2023-09-30T22:14:02.085672+0000","last_became_active":"2023-09-30T22:14:02.074872+0000","last_became_peered":"2023-09-30T22:14:02.074872+0000","last_unstale":"2023-09-30T22:14:02.085672+0000","last_undegraded":"2023-09-30T22:14:02.085672+0000","last_fullsized":"2023-09-30T22:14:02.085672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053010+0000","last_change":"2023-09-30T22:14:01.702319+0000","last_active":"2023-09-30T22:14:02.053010+0000","last_peered":"2023-09-30T22:14:02.053010+0000","last_clean":"2023-09-30T22:14:02.053010+0000","last_became_active":"2023-09-30T22:14:01.702020+0000","last_became_peered":"2023-09-30T22:14:01.702020+0000","last_unstale":"2023-09-30T22:14:02.053010+0000","last_undegraded":"2023-09-30T22:14:02.053010+0000","last_fullsized":"2023-09-30T22:14:02.053010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091650+0000","last_change":"2023-09-30T22:14:02.079713+0000","last_active":"2023-09-30T22:14:02.091650+0000","last_peered":"2023-09-30T22:14:02.091650+0000","last_clean":"2023-09-30T22:14:02.091650+0000","last_became_active":"2023-09-30T22:14:02.079318+0000","last_became_peered":"2023-09-30T22:14:02.079318+0000","last_unstale":"2023-09-30T22:14:02.091650+0000","last_undegraded":"2023-09-30T22:14:02.091650+0000","last_fullsized":"2023-09-30T22:14:02.091650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053665+0000","last_change":"2023-09-30T22:14:01.707097+0000","last_active":"2023-09-30T22:14:02.053665+0000","last_peered":"2023-09-30T22:14:02.053665+0000","last_clean":"2023-09-30T22:14:02.053665+0000","last_became_active":"2023-09-30T22:14:01.706849+0000","last_became_peered":"2023-09-30T22:14:01.706849+0000","last_unstale":"2023-09-30T22:14:02.053665+0000","last_undegraded":"2023-09-30T22:14:02.053665+0000","last_fullsized":"2023-09-30T22:14:02.053665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087679+0000","last_change":"2023-09-30T22:14:02.084631+0000","last_active":"2023-09-30T22:14:02.087679+0000","last_peered":"2023-09-30T22:14:02.087679+0000","last_clean":"2023-09-30T22:14:02.087679+0000","last_became_active":"2023-09-30T22:14:02.084356+0000","last_became_peered":"2023-09-30T22:14:02.084356+0000","last_unstale":"2023-09-30T22:14:02.087679+0000","last_undegraded":"2023-09-30T22:14:02.087679+0000","last_fullsized":"2023-09-30T22:14:02.087679+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057794+0000","last_change":"2023-09-30T22:14:01.698007+0000","last_active":"2023-09-30T22:14:02.057794+0000","last_peered":"2023-09-30T22:14:02.057794+0000","last_clean":"2023-09-30T22:14:02.057794+0000","last_became_active":"2023-09-30T22:14:01.697731+0000","last_became_peered":"2023-09-30T22:14:01.697731+0000","last_unstale":"2023-09-30T22:14:02.057794+0000","last_undegraded":"2023-09-30T22:14:02.057794+0000","last_fullsized":"2023-09-30T22:14:02.057794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087841+0000","last_change":"2023-09-30T22:14:02.082853+0000","last_active":"2023-09-30T22:14:02.087841+0000","last_peered":"2023-09-30T22:14:02.087841+0000","last_clean":"2023-09-30T22:14:02.087841+0000","last_became_active":"2023-09-30T22:14:02.082597+0000","last_became_peered":"2023-09-30T22:14:02.082597+0000","last_unstale":"2023-09-30T22:14:02.087841+0000","last_undegraded":"2023-09-30T22:14:02.087841+0000","last_fullsized":"2023-09-30T22:14:02.087841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056502+0000","last_change":"2023-09-30T22:14:01.700657+0000","last_active":"2023-09-30T22:14:02.056502+0000","last_peered":"2023-09-30T22:14:02.056502+0000","last_clean":"2023-09-30T22:14:02.056502+0000","last_became_active":"2023-09-30T22:14:01.699633+0000","last_became_peered":"2023-09-30T22:14:01.699633+0000","last_unstale":"2023-09-30T22:14:02.056502+0000","last_undegraded":"2023-09-30T22:14:02.056502+0000","last_fullsized":"2023-09-30T22:14:02.056502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093773+0000","last_change":"2023-09-30T22:14:02.085416+0000","last_active":"2023-09-30T22:14:02.093773+0000","last_peered":"2023-09-30T22:14:02.093773+0000","last_clean":"2023-09-30T22:14:02.093773+0000","last_became_active":"2023-09-30T22:14:02.084970+0000","last_became_peered":"2023-09-30T22:14:02.084970+0000","last_unstale":"2023-09-30T22:14:02.093773+0000","last_undegraded":"2023-09-30T22:14:02.093773+0000","last_fullsized":"2023-09-30T22:14:02.093773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083245+0000","last_change":"2023-09-30T22:14:01.701075+0000","last_active":"2023-09-30T22:14:02.083245+0000","last_peered":"2023-09-30T22:14:02.083245+0000","last_clean":"2023-09-30T22:14:02.083245+0000","last_became_active":"2023-09-30T22:14:01.700806+0000","last_became_peered":"2023-09-30T22:14:01.700806+0000","last_unstale":"2023-09-30T22:14:02.083245+0000","last_undegraded":"2023-09-30T22:14:02.083245+0000","last_fullsized":"2023-09-30T22:14:02.083245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.077417+0000","last_change":"2023-09-30T22:14:02.074078+0000","last_active":"2023-09-30T22:14:02.077417+0000","last_peered":"2023-09-30T22:14:02.077417+0000","last_clean":"2023-09-30T22:14:02.077417+0000","last_became_active":"2023-09-30T22:14:02.073804+0000","last_became_peered":"2023-09-30T22:14:02.073804+0000","last_unstale":"2023-09-30T22:14:02.077417+0000","last_undegraded":"2023-09-30T22:14:02.077417+0000","last_fullsized":"2023-09-30T22:14:02.077417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.096081+0000","last_change":"2023-09-30T22:14:01.711172+0000","last_active":"2023-09-30T22:14:02.096081+0000","last_peered":"2023-09-30T22:14:02.096081+0000","last_clean":"2023-09-30T22:14:02.096081+0000","last_became_active":"2023-09-30T22:14:01.710926+0000","last_became_peered":"2023-09-30T22:14:01.710926+0000","last_unstale":"2023-09-30T22:14:02.096081+0000","last_undegraded":"2023-09-30T22:14:02.096081+0000","last_fullsized":"2023-09-30T22:14:02.096081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090366+0000","last_change":"2023-09-30T22:14:02.087193+0000","last_active":"2023-09-30T22:14:02.090366+0000","last_peered":"2023-09-30T22:14:02.090366+0000","last_clean":"2023-09-30T22:14:02.090366+0000","last_became_active":"2023-09-30T22:14:02.086273+0000","last_became_peered":"2023-09-30T22:14:02.086273+0000","last_unstale":"2023-09-30T22:14:02.090366+0000","last_undegraded":"2023-09-30T22:14:02.090366+0000","last_fullsized":"2023-09-30T22:14:02.090366+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082985+0000","last_change":"2023-09-30T22:14:01.706560+0000","last_active":"2023-09-30T22:14:02.082985+0000","last_peered":"2023-09-30T22:14:02.082985+0000","last_clean":"2023-09-30T22:14:02.082985+0000","last_became_active":"2023-09-30T22:14:01.706330+0000","last_became_peered":"2023-09-30T22:14:01.706330+0000","last_unstale":"2023-09-30T22:14:02.082985+0000","last_undegraded":"2023-09-30T22:14:02.082985+0000","last_fullsized":"2023-09-30T22:14:02.082985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087699+0000","last_change":"2023-09-30T22:14:02.084139+0000","last_active":"2023-09-30T22:14:02.087699+0000","last_peered":"2023-09-30T22:14:02.087699+0000","last_clean":"2023-09-30T22:14:02.087699+0000","last_became_active":"2023-09-30T22:14:02.080986+0000","last_became_peered":"2023-09-30T22:14:02.080986+0000","last_unstale":"2023-09-30T22:14:02.087699+0000","last_undegraded":"2023-09-30T22:14:02.087699+0000","last_fullsized":"2023-09-30T22:14:02.087699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.055029+0000","last_change":"2023-09-30T22:14:01.698141+0000","last_active":"2023-09-30T22:14:02.055029+0000","last_peered":"2023-09-30T22:14:02.055029+0000","last_clean":"2023-09-30T22:14:02.055029+0000","last_became_active":"2023-09-30T22:14:01.697885+0000","last_became_peered":"2023-09-30T22:14:01.697885+0000","last_unstale":"2023-09-30T22:14:02.055029+0000","last_undegraded":"2023-09-30T22:14:02.055029+0000","last_fullsized":"2023-09-30T22:14:02.055029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085690+0000","last_change":"2023-09-30T22:14:02.075925+0000","last_active":"2023-09-30T22:14:02.085690+0000","last_peered":"2023-09-30T22:14:02.085690+0000","last_clean":"2023-09-30T22:14:02.085690+0000","last_became_active":"2023-09-30T22:14:02.075137+0000","last_became_peered":"2023-09-30T22:14:02.075137+0000","last_unstale":"2023-09-30T22:14:02.085690+0000","last_undegraded":"2023-09-30T22:14:02.085690+0000","last_fullsized":"2023-09-30T22:14:02.085690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.064991+0000","last_change":"2023-09-30T22:14:01.712299+0000","last_active":"2023-09-30T22:14:02.064991+0000","last_peered":"2023-09-30T22:14:02.064991+0000","last_clean":"2023-09-30T22:14:02.064991+0000","last_became_active":"2023-09-30T22:14:01.711791+0000","last_became_peered":"2023-09-30T22:14:01.711791+0000","last_unstale":"2023-09-30T22:14:02.064991+0000","last_undegraded":"2023-09-30T22:14:02.064991+0000","last_fullsized":"2023-09-30T22:14:02.064991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091669+0000","last_change":"2023-09-30T22:14:02.084033+0000","last_active":"2023-09-30T22:14:02.091669+0000","last_peered":"2023-09-30T22:14:02.091669+0000","last_clean":"2023-09-30T22:14:02.091669+0000","last_became_active":"2023-09-30T22:14:02.083764+0000","last_became_peered":"2023-09-30T22:14:02.083764+0000","last_unstale":"2023-09-30T22:14:02.091669+0000","last_undegraded":"2023-09-30T22:14:02.091669+0000","last_fullsized":"2023-09-30T22:14:02.091669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.092132+0000","last_change":"2023-09-30T22:14:01.706777+0000","last_active":"2023-09-30T22:14:02.092132+0000","last_peered":"2023-09-30T22:14:02.092132+0000","last_clean":"2023-09-30T22:14:02.092132+0000","last_became_active":"2023-09-30T22:14:01.706479+0000","last_became_peered":"2023-09-30T22:14:01.706479+0000","last_unstale":"2023-09-30T22:14:02.092132+0000","last_undegraded":"2023-09-30T22:14:02.092132+0000","last_fullsized":"2023-09-30T22:14:02.092132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087631+0000","last_change":"2023-09-30T22:14:02.084399+0000","last_active":"2023-09-30T22:14:02.087631+0000","last_peered":"2023-09-30T22:14:02.087631+0000","last_clean":"2023-09-30T22:14:02.087631+0000","last_became_active":"2023-09-30T22:14:02.084137+0000","last_became_peered":"2023-09-30T22:14:02.084137+0000","last_unstale":"2023-09-30T22:14:02.087631+0000","last_undegraded":"2023-09-30T22:14:02.087631+0000","last_fullsized":"2023-09-30T22:14:02.087631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093442+0000","last_change":"2023-09-30T22:14:01.702685+0000","last_active":"2023-09-30T22:14:02.093442+0000","last_peered":"2023-09-30T22:14:02.093442+0000","last_clean":"2023-09-30T22:14:02.093442+0000","last_became_active":"2023-09-30T22:14:01.702083+0000","last_became_peered":"2023-09-30T22:14:01.702083+0000","last_unstale":"2023-09-30T22:14:02.093442+0000","last_undegraded":"2023-09-30T22:14:02.093442+0000","last_fullsized":"2023-09-30T22:14:02.093442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087823+0000","last_change":"2023-09-30T22:14:02.082069+0000","last_active":"2023-09-30T22:14:02.087823+0000","last_peered":"2023-09-30T22:14:02.087823+0000","last_clean":"2023-09-30T22:14:02.087823+0000","last_became_active":"2023-09-30T22:14:02.081702+0000","last_became_peered":"2023-09-30T22:14:02.081702+0000","last_unstale":"2023-09-30T22:14:02.087823+0000","last_undegraded":"2023-09-30T22:14:02.087823+0000","last_fullsized":"2023-09-30T22:14:02.087823+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.069430+0000","last_change":"2023-09-30T22:14:01.706748+0000","last_active":"2023-09-30T22:14:02.069430+0000","last_peered":"2023-09-30T22:14:02.069430+0000","last_clean":"2023-09-30T22:14:02.069430+0000","last_became_active":"2023-09-30T22:14:01.706472+0000","last_became_peered":"2023-09-30T22:14:01.706472+0000","last_unstale":"2023-09-30T22:14:02.069430+0000","last_undegraded":"2023-09-30T22:14:02.069430+0000","last_fullsized":"2023-09-30T22:14:02.069430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.076358+0000","last_change":"2023-09-30T22:14:02.070587+0000","last_active":"2023-09-30T22:14:02.076358+0000","last_peered":"2023-09-30T22:14:02.076358+0000","last_clean":"2023-09-30T22:14:02.076358+0000","last_became_active":"2023-09-30T22:14:02.070259+0000","last_became_peered":"2023-09-30T22:14:02.070259+0000","last_unstale":"2023-09-30T22:14:02.076358+0000","last_undegraded":"2023-09-30T22:14:02.076358+0000","last_fullsized":"2023-09-30T22:14:02.076358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080959+0000","last_change":"2023-09-30T22:14:01.704603+0000","last_active":"2023-09-30T22:14:02.080959+0000","last_peered":"2023-09-30T22:14:02.080959+0000","last_clean":"2023-09-30T22:14:02.080959+0000","last_became_active":"2023-09-30T22:14:01.704006+0000","last_became_peered":"2023-09-30T22:14:01.704006+0000","last_unstale":"2023-09-30T22:14:02.080959+0000","last_undegraded":"2023-09-30T22:14:02.080959+0000","last_fullsized":"2023-09-30T22:14:02.080959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080942+0000","last_change":"2023-09-30T22:14:02.073527+0000","last_active":"2023-09-30T22:14:02.080942+0000","last_peered":"2023-09-30T22:14:02.080942+0000","last_clean":"2023-09-30T22:14:02.080942+0000","last_became_active":"2023-09-30T22:14:02.072950+0000","last_became_peered":"2023-09-30T22:14:02.072950+0000","last_unstale":"2023-09-30T22:14:02.080942+0000","last_undegraded":"2023-09-30T22:14:02.080942+0000","last_fullsized":"2023-09-30T22:14:02.080942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057420+0000","last_change":"2023-09-30T22:14:01.701736+0000","last_active":"2023-09-30T22:14:02.057420+0000","last_peered":"2023-09-30T22:14:02.057420+0000","last_clean":"2023-09-30T22:14:02.057420+0000","last_became_active":"2023-09-30T22:14:01.701471+0000","last_became_peered":"2023-09-30T22:14:01.701471+0000","last_unstale":"2023-09-30T22:14:02.057420+0000","last_undegraded":"2023-09-30T22:14:02.057420+0000","last_fullsized":"2023-09-30T22:14:02.057420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082228+0000","last_change":"2023-09-30T22:14:02.072624+0000","last_active":"2023-09-30T22:14:02.082228+0000","last_peered":"2023-09-30T22:14:02.082228+0000","last_clean":"2023-09-30T22:14:02.082228+0000","last_became_active":"2023-09-30T22:14:02.072363+0000","last_became_peered":"2023-09-30T22:14:02.072363+0000","last_unstale":"2023-09-30T22:14:02.082228+0000","last_undegraded":"2023-09-30T22:14:02.082228+0000","last_fullsized":"2023-09-30T22:14:02.082228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082837+0000","last_change":"2023-09-30T22:14:01.709398+0000","last_active":"2023-09-30T22:14:02.082837+0000","last_peered":"2023-09-30T22:14:02.082837+0000","last_clean":"2023-09-30T22:14:02.082837+0000","last_became_active":"2023-09-30T22:14:01.709110+0000","last_became_peered":"2023-09-30T22:14:01.709110+0000","last_unstale":"2023-09-30T22:14:02.082837+0000","last_undegraded":"2023-09-30T22:14:02.082837+0000","last_fullsized":"2023-09-30T22:14:02.082837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087990+0000","last_change":"2023-09-30T22:14:02.074887+0000","last_active":"2023-09-30T22:14:02.087990+0000","last_peered":"2023-09-30T22:14:02.087990+0000","last_clean":"2023-09-30T22:14:02.087990+0000","last_became_active":"2023-09-30T22:14:02.074631+0000","last_became_peered":"2023-09-30T22:14:02.074631+0000","last_unstale":"2023-09-30T22:14:02.087990+0000","last_undegraded":"2023-09-30T22:14:02.087990+0000","last_fullsized":"2023-09-30T22:14:02.087990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054479+0000","last_change":"2023-09-30T22:14:01.707879+0000","last_active":"2023-09-30T22:14:02.054479+0000","last_peered":"2023-09-30T22:14:02.054479+0000","last_clean":"2023-09-30T22:14:02.054479+0000","last_became_active":"2023-09-30T22:14:01.707594+0000","last_became_peered":"2023-09-30T22:14:01.707594+0000","last_unstale":"2023-09-30T22:14:02.054479+0000","last_undegraded":"2023-09-30T22:14:02.054479+0000","last_fullsized":"2023-09-30T22:14:02.054479+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085707+0000","last_change":"2023-09-30T22:14:02.076079+0000","last_active":"2023-09-30T22:14:02.085707+0000","last_peered":"2023-09-30T22:14:02.085707+0000","last_clean":"2023-09-30T22:14:02.085707+0000","last_became_active":"2023-09-30T22:14:02.075408+0000","last_became_peered":"2023-09-30T22:14:02.075408+0000","last_unstale":"2023-09-30T22:14:02.085707+0000","last_undegraded":"2023-09-30T22:14:02.085707+0000","last_fullsized":"2023-09-30T22:14:02.085707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.065836+0000","last_change":"2023-09-30T22:14:01.711605+0000","last_active":"2023-09-30T22:14:02.065836+0000","last_peered":"2023-09-30T22:14:02.065836+0000","last_clean":"2023-09-30T22:14:02.065836+0000","last_became_active":"2023-09-30T22:14:01.711375+0000","last_became_peered":"2023-09-30T22:14:01.711375+0000","last_unstale":"2023-09-30T22:14:02.065836+0000","last_undegraded":"2023-09-30T22:14:02.065836+0000","last_fullsized":"2023-09-30T22:14:02.065836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090979+0000","last_change":"2023-09-30T22:14:02.088858+0000","last_active":"2023-09-30T22:14:02.090979+0000","last_peered":"2023-09-30T22:14:02.090979+0000","last_clean":"2023-09-30T22:14:02.090979+0000","last_became_active":"2023-09-30T22:14:02.088583+0000","last_became_peered":"2023-09-30T22:14:02.088583+0000","last_unstale":"2023-09-30T22:14:02.090979+0000","last_undegraded":"2023-09-30T22:14:02.090979+0000","last_fullsized":"2023-09-30T22:14:02.090979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053157+0000","last_change":"2023-09-30T22:14:01.699847+0000","last_active":"2023-09-30T22:14:02.053157+0000","last_peered":"2023-09-30T22:14:02.053157+0000","last_clean":"2023-09-30T22:14:02.053157+0000","last_became_active":"2023-09-30T22:14:01.699587+0000","last_became_peered":"2023-09-30T22:14:01.699587+0000","last_unstale":"2023-09-30T22:14:02.053157+0000","last_undegraded":"2023-09-30T22:14:02.053157+0000","last_fullsized":"2023-09-30T22:14:02.053157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.098596+0000","last_change":"2023-09-30T22:14:02.094005+0000","last_active":"2023-09-30T22:14:02.098596+0000","last_peered":"2023-09-30T22:14:02.098596+0000","last_clean":"2023-09-30T22:14:02.098596+0000","last_became_active":"2023-09-30T22:14:02.093632+0000","last_became_peered":"2023-09-30T22:14:02.093632+0000","last_unstale":"2023-09-30T22:14:02.098596+0000","last_undegraded":"2023-09-30T22:14:02.098596+0000","last_fullsized":"2023-09-30T22:14:02.098596+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.052854+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:14:02.052854+0000","last_peered":"2023-09-30T22:14:02.052854+0000","last_clean":"2023-09-30T22:14:02.052854+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:14:02.052854+0000","last_undegraded":"2023-09-30T22:14:02.052854+0000","last_fullsized":"2023-09-30T22:14:02.052854+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056318+0000","last_change":"2023-09-30T22:14:01.702822+0000","last_active":"2023-09-30T22:14:02.056318+0000","last_peered":"2023-09-30T22:14:02.056318+0000","last_clean":"2023-09-30T22:14:02.056318+0000","last_became_active":"2023-09-30T22:14:01.702530+0000","last_became_peered":"2023-09-30T22:14:01.702530+0000","last_unstale":"2023-09-30T22:14:02.056318+0000","last_undegraded":"2023-09-30T22:14:02.056318+0000","last_fullsized":"2023-09-30T22:14:02.056318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093176+0000","last_change":"2023-09-30T22:14:02.084698+0000","last_active":"2023-09-30T22:14:02.093176+0000","last_peered":"2023-09-30T22:14:02.093176+0000","last_clean":"2023-09-30T22:14:02.093176+0000","last_became_active":"2023-09-30T22:14:02.084413+0000","last_became_peered":"2023-09-30T22:14:02.084413+0000","last_unstale":"2023-09-30T22:14:02.093176+0000","last_undegraded":"2023-09-30T22:14:02.093176+0000","last_fullsized":"2023-09-30T22:14:02.093176+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054630+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:14:02.054630+0000","last_peered":"2023-09-30T22:14:02.054630+0000","last_clean":"2023-09-30T22:14:02.054630+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:14:02.054630+0000","last_undegraded":"2023-09-30T22:14:02.054630+0000","last_fullsized":"2023-09-30T22:14:02.054630+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057152+0000","last_change":"2023-09-30T22:14:01.704198+0000","last_active":"2023-09-30T22:14:02.057152+0000","last_peered":"2023-09-30T22:14:02.057152+0000","last_clean":"2023-09-30T22:14:02.057152+0000","last_became_active":"2023-09-30T22:14:01.701691+0000","last_became_peered":"2023-09-30T22:14:01.701691+0000","last_unstale":"2023-09-30T22:14:02.057152+0000","last_undegraded":"2023-09-30T22:14:02.057152+0000","last_fullsized":"2023-09-30T22:14:02.057152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079574+0000","last_change":"2023-09-30T22:14:02.076717+0000","last_active":"2023-09-30T22:14:02.079574+0000","last_peered":"2023-09-30T22:14:02.079574+0000","last_clean":"2023-09-30T22:14:02.079574+0000","last_became_active":"2023-09-30T22:14:02.076119+0000","last_became_peered":"2023-09-30T22:14:02.076119+0000","last_unstale":"2023-09-30T22:14:02.079574+0000","last_undegraded":"2023-09-30T22:14:02.079574+0000","last_fullsized":"2023-09-30T22:14:02.079574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056513+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:14:02.056513+0000","last_peered":"2023-09-30T22:14:02.056513+0000","last_clean":"2023-09-30T22:14:02.056513+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:14:02.056513+0000","last_undegraded":"2023-09-30T22:14:02.056513+0000","last_fullsized":"2023-09-30T22:14:02.056513+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062825+0000","last_change":"2023-09-30T22:14:01.701431+0000","last_active":"2023-09-30T22:14:02.062825+0000","last_peered":"2023-09-30T22:14:02.062825+0000","last_clean":"2023-09-30T22:14:02.062825+0000","last_became_active":"2023-09-30T22:14:01.698771+0000","last_became_peered":"2023-09-30T22:14:01.698771+0000","last_unstale":"2023-09-30T22:14:02.062825+0000","last_undegraded":"2023-09-30T22:14:02.062825+0000","last_fullsized":"2023-09-30T22:14:02.062825+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.094124+0000","last_change":"2023-09-30T22:14:02.085435+0000","last_active":"2023-09-30T22:14:02.094124+0000","last_peered":"2023-09-30T22:14:02.094124+0000","last_clean":"2023-09-30T22:14:02.094124+0000","last_became_active":"2023-09-30T22:14:02.085124+0000","last_became_peered":"2023-09-30T22:14:02.085124+0000","last_unstale":"2023-09-30T22:14:02.094124+0000","last_undegraded":"2023-09-30T22:14:02.094124+0000","last_fullsized":"2023-09-30T22:14:02.094124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062504+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:14:02.062504+0000","last_peered":"2023-09-30T22:14:02.062504+0000","last_clean":"2023-09-30T22:14:02.062504+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:14:02.062504+0000","last_undegraded":"2023-09-30T22:14:02.062504+0000","last_fullsized":"2023-09-30T22:14:02.062504+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062714+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:14:02.062714+0000","last_peered":"2023-09-30T22:14:02.062714+0000","last_clean":"2023-09-30T22:14:02.062714+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:14:02.062714+0000","last_undegraded":"2023-09-30T22:14:02.062714+0000","last_fullsized":"2023-09-30T22:14:02.062714+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054372+0000","last_change":"2023-09-30T22:14:01.712014+0000","last_active":"2023-09-30T22:14:02.054372+0000","last_peered":"2023-09-30T22:14:02.054372+0000","last_clean":"2023-09-30T22:14:02.054372+0000","last_became_active":"2023-09-30T22:14:01.711770+0000","last_became_peered":"2023-09-30T22:14:01.711770+0000","last_unstale":"2023-09-30T22:14:02.054372+0000","last_undegraded":"2023-09-30T22:14:02.054372+0000","last_fullsized":"2023-09-30T22:14:02.054372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087711+0000","last_change":"2023-09-30T22:14:02.080800+0000","last_active":"2023-09-30T22:14:02.087711+0000","last_peered":"2023-09-30T22:14:02.087711+0000","last_clean":"2023-09-30T22:14:02.087711+0000","last_became_active":"2023-09-30T22:14:02.080440+0000","last_became_peered":"2023-09-30T22:14:02.080440+0000","last_unstale":"2023-09-30T22:14:02.087711+0000","last_undegraded":"2023-09-30T22:14:02.087711+0000","last_fullsized":"2023-09-30T22:14:02.087711+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054814+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:14:02.054814+0000","last_peered":"2023-09-30T22:14:02.054814+0000","last_clean":"2023-09-30T22:14:02.054814+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:14:02.054814+0000","last_undegraded":"2023-09-30T22:14:02.054814+0000","last_fullsized":"2023-09-30T22:14:02.054814+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.055456+0000","last_change":"2023-09-30T22:14:01.694407+0000","last_active":"2023-09-30T22:14:02.055456+0000","last_peered":"2023-09-30T22:14:02.055456+0000","last_clean":"2023-09-30T22:14:02.055456+0000","last_became_active":"2023-09-30T22:14:01.694022+0000","last_became_peered":"2023-09-30T22:14:01.694022+0000","last_unstale":"2023-09-30T22:14:02.055456+0000","last_undegraded":"2023-09-30T22:14:02.055456+0000","last_fullsized":"2023-09-30T22:14:02.055456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091628+0000","last_change":"2023-09-30T22:14:02.079540+0000","last_active":"2023-09-30T22:14:02.091628+0000","last_peered":"2023-09-30T22:14:02.091628+0000","last_clean":"2023-09-30T22:14:02.091628+0000","last_became_active":"2023-09-30T22:14:02.079016+0000","last_became_peered":"2023-09-30T22:14:02.079016+0000","last_unstale":"2023-09-30T22:14:02.091628+0000","last_undegraded":"2023-09-30T22:14:02.091628+0000","last_fullsized":"2023-09-30T22:14:02.091628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.059163+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:14:02.059163+0000","last_peered":"2023-09-30T22:14:02.059163+0000","last_clean":"2023-09-30T22:14:02.059163+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:14:02.059163+0000","last_undegraded":"2023-09-30T22:14:02.059163+0000","last_fullsized":"2023-09-30T22:14:02.059163+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053400+0000","last_change":"2023-09-30T22:14:01.695717+0000","last_active":"2023-09-30T22:14:02.053400+0000","last_peered":"2023-09-30T22:14:02.053400+0000","last_clean":"2023-09-30T22:14:02.053400+0000","last_became_active":"2023-09-30T22:14:01.694990+0000","last_became_peered":"2023-09-30T22:14:01.694990+0000","last_unstale":"2023-09-30T22:14:02.053400+0000","last_undegraded":"2023-09-30T22:14:02.053400+0000","last_fullsized":"2023-09-30T22:14:02.053400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080394+0000","last_change":"2023-09-30T22:14:02.068852+0000","last_active":"2023-09-30T22:14:02.080394+0000","last_peered":"2023-09-30T22:14:02.080394+0000","last_clean":"2023-09-30T22:14:02.080394+0000","last_became_active":"2023-09-30T22:14:02.068531+0000","last_became_peered":"2023-09-30T22:14:02.068531+0000","last_unstale":"2023-09-30T22:14:02.080394+0000","last_undegraded":"2023-09-30T22:14:02.080394+0000","last_fullsized":"2023-09-30T22:14:02.080394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.084595+0000","last_change":"2023-09-30T22:14:01.700959+0000","last_active":"2023-09-30T22:14:02.084595+0000","last_peered":"2023-09-30T22:14:02.084595+0000","last_clean":"2023-09-30T22:14:02.084595+0000","last_became_active":"2023-09-30T22:14:01.700568+0000","last_became_peered":"2023-09-30T22:14:01.700568+0000","last_unstale":"2023-09-30T22:14:02.084595+0000","last_undegraded":"2023-09-30T22:14:02.084595+0000","last_fullsized":"2023-09-30T22:14:02.084595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.048709+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:14:02.048709+0000","last_peered":"2023-09-30T22:14:02.048709+0000","last_clean":"2023-09-30T22:14:02.048709+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:14:02.048709+0000","last_undegraded":"2023-09-30T22:14:02.048709+0000","last_fullsized":"2023-09-30T22:14:02.048709+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087805+0000","last_change":"2023-09-30T22:14:02.081943+0000","last_active":"2023-09-30T22:14:02.087805+0000","last_peered":"2023-09-30T22:14:02.087805+0000","last_clean":"2023-09-30T22:14:02.087805+0000","last_became_active":"2023-09-30T22:14:02.080899+0000","last_became_peered":"2023-09-30T22:14:02.080899+0000","last_unstale":"2023-09-30T22:14:02.087805+0000","last_undegraded":"2023-09-30T22:14:02.087805+0000","last_fullsized":"2023-09-30T22:14:02.087805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054595+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:14:02.054595+0000","last_peered":"2023-09-30T22:14:02.054595+0000","last_clean":"2023-09-30T22:14:02.054595+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:14:02.054595+0000","last_undegraded":"2023-09-30T22:14:02.054595+0000","last_fullsized":"2023-09-30T22:14:02.054595+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T22:14:03.047055+0000","last_change":"2023-09-30T22:14:01.706704+0000","last_active":"2023-09-30T22:14:03.047055+0000","last_peered":"2023-09-30T22:14:03.047055+0000","last_clean":"2023-09-30T22:14:03.047055+0000","last_became_active":"2023-09-30T22:14:01.706387+0000","last_became_peered":"2023-09-30T22:14:01.706387+0000","last_unstale":"2023-09-30T22:14:03.047055+0000","last_undegraded":"2023-09-30T22:14:03.047055+0000","last_fullsized":"2023-09-30T22:14:03.047055+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090754+0000","last_change":"2023-09-30T22:14:02.088343+0000","last_active":"2023-09-30T22:14:02.090754+0000","last_peered":"2023-09-30T22:14:02.090754+0000","last_clean":"2023-09-30T22:14:02.090754+0000","last_became_active":"2023-09-30T22:14:02.088142+0000","last_became_peered":"2023-09-30T22:14:02.088142+0000","last_unstale":"2023-09-30T22:14:02.090754+0000","last_undegraded":"2023-09-30T22:14:02.090754+0000","last_fullsized":"2023-09-30T22:14:02.090754+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083333+0000","last_change":"2023-09-30T22:14:01.702235+0000","last_active":"2023-09-30T22:14:02.083333+0000","last_peered":"2023-09-30T22:14:02.083333+0000","last_clean":"2023-09-30T22:14:02.083333+0000","last_became_active":"2023-09-30T22:14:01.701972+0000","last_became_peered":"2023-09-30T22:14:01.701972+0000","last_unstale":"2023-09-30T22:14:02.083333+0000","last_undegraded":"2023-09-30T22:14:02.083333+0000","last_fullsized":"2023-09-30T22:14:02.083333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080923+0000","last_change":"2023-09-30T22:14:02.078694+0000","last_active":"2023-09-30T22:14:02.080923+0000","last_peered":"2023-09-30T22:14:02.080923+0000","last_clean":"2023-09-30T22:14:02.080923+0000","last_became_active":"2023-09-30T22:14:02.078435+0000","last_became_peered":"2023-09-30T22:14:02.078435+0000","last_unstale":"2023-09-30T22:14:02.080923+0000","last_undegraded":"2023-09-30T22:14:02.080923+0000","last_fullsized":"2023-09-30T22:14:02.080923+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090913+0000","last_change":"2023-09-30T22:14:01.709428+0000","last_active":"2023-09-30T22:14:02.090913+0000","last_peered":"2023-09-30T22:14:02.090913+0000","last_clean":"2023-09-30T22:14:02.090913+0000","last_became_active":"2023-09-30T22:14:01.709089+0000","last_became_peered":"2023-09-30T22:14:01.709089+0000","last_unstale":"2023-09-30T22:14:02.090913+0000","last_undegraded":"2023-09-30T22:14:02.090913+0000","last_fullsized":"2023-09-30T22:14:02.090913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090409+0000","last_change":"2023-09-30T22:14:02.087499+0000","last_active":"2023-09-30T22:14:02.090409+0000","last_peered":"2023-09-30T22:14:02.090409+0000","last_clean":"2023-09-30T22:14:02.090409+0000","last_became_active":"2023-09-30T22:14:02.086941+0000","last_became_peered":"2023-09-30T22:14:02.086941+0000","last_unstale":"2023-09-30T22:14:02.090409+0000","last_undegraded":"2023-09-30T22:14:02.090409+0000","last_fullsized":"2023-09-30T22:14:02.090409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083525+0000","last_change":"2023-09-30T22:14:01.709677+0000","last_active":"2023-09-30T22:14:02.083525+0000","last_peered":"2023-09-30T22:14:02.083525+0000","last_clean":"2023-09-30T22:14:02.083525+0000","last_became_active":"2023-09-30T22:14:01.709115+0000","last_became_peered":"2023-09-30T22:14:01.709115+0000","last_unstale":"2023-09-30T22:14:02.083525+0000","last_undegraded":"2023-09-30T22:14:02.083525+0000","last_fullsized":"2023-09-30T22:14:02.083525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080964+0000","last_change":"2023-09-30T22:14:02.073683+0000","last_active":"2023-09-30T22:14:02.080964+0000","last_peered":"2023-09-30T22:14:02.080964+0000","last_clean":"2023-09-30T22:14:02.080964+0000","last_became_active":"2023-09-30T22:14:02.073269+0000","last_became_peered":"2023-09-30T22:14:02.073269+0000","last_unstale":"2023-09-30T22:14:02.080964+0000","last_undegraded":"2023-09-30T22:14:02.080964+0000","last_fullsized":"2023-09-30T22:14:02.080964+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083125+0000","last_change":"2023-09-30T22:14:01.711766+0000","last_active":"2023-09-30T22:14:02.083125+0000","last_peered":"2023-09-30T22:14:02.083125+0000","last_clean":"2023-09-30T22:14:02.083125+0000","last_became_active":"2023-09-30T22:14:01.711524+0000","last_became_peered":"2023-09-30T22:14:01.711524+0000","last_unstale":"2023-09-30T22:14:02.083125+0000","last_undegraded":"2023-09-30T22:14:02.083125+0000","last_fullsized":"2023-09-30T22:14:02.083125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088038+0000","last_change":"2023-09-30T22:14:02.082106+0000","last_active":"2023-09-30T22:14:02.088038+0000","last_peered":"2023-09-30T22:14:02.088038+0000","last_clean":"2023-09-30T22:14:02.088038+0000","last_became_active":"2023-09-30T22:14:02.081789+0000","last_became_peered":"2023-09-30T22:14:02.081789+0000","last_unstale":"2023-09-30T22:14:02.088038+0000","last_undegraded":"2023-09-30T22:14:02.088038+0000","last_fullsized":"2023-09-30T22:14:02.088038+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091206+0000","last_change":"2023-09-30T22:14:01.702868+0000","last_active":"2023-09-30T22:14:02.091206+0000","last_peered":"2023-09-30T22:14:02.091206+0000","last_clean":"2023-09-30T22:14:02.091206+0000","last_became_active":"2023-09-30T22:14:01.702431+0000","last_became_peered":"2023-09-30T22:14:01.702431+0000","last_unstale":"2023-09-30T22:14:02.091206+0000","last_undegraded":"2023-09-30T22:14:02.091206+0000","last_fullsized":"2023-09-30T22:14:02.091206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.084635+0000","last_change":"2023-09-30T22:14:02.081561+0000","last_active":"2023-09-30T22:14:02.084635+0000","last_peered":"2023-09-30T22:14:02.084635+0000","last_clean":"2023-09-30T22:14:02.084635+0000","last_became_active":"2023-09-30T22:14:02.081199+0000","last_became_peered":"2023-09-30T22:14:02.081199+0000","last_unstale":"2023-09-30T22:14:02.084635+0000","last_undegraded":"2023-09-30T22:14:02.084635+0000","last_fullsized":"2023-09-30T22:14:02.084635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.086077+0000","last_change":"2023-09-30T22:14:02.079397+0000","last_active":"2023-09-30T22:14:02.086077+0000","last_peered":"2023-09-30T22:14:02.086077+0000","last_clean":"2023-09-30T22:14:02.086077+0000","last_became_active":"2023-09-30T22:14:02.079036+0000","last_became_peered":"2023-09-30T22:14:02.079036+0000","last_unstale":"2023-09-30T22:14:02.086077+0000","last_undegraded":"2023-09-30T22:14:02.086077+0000","last_fullsized":"2023-09-30T22:14:02.086077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088018+0000","last_change":"2023-09-30T22:14:02.081974+0000","last_active":"2023-09-30T22:14:02.088018+0000","last_peered":"2023-09-30T22:14:02.088018+0000","last_clean":"2023-09-30T22:14:02.088018+0000","last_became_active":"2023-09-30T22:14:02.081491+0000","last_became_peered":"2023-09-30T22:14:02.081491+0000","last_unstale":"2023-09-30T22:14:02.088018+0000","last_undegraded":"2023-09-30T22:14:02.088018+0000","last_fullsized":"2023-09-30T22:14:02.088018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080473+0000","last_change":"2023-09-30T22:14:02.067554+0000","last_active":"2023-09-30T22:14:02.080473+0000","last_peered":"2023-09-30T22:14:02.080473+0000","last_clean":"2023-09-30T22:14:02.080473+0000","last_became_active":"2023-09-30T22:14:02.067283+0000","last_became_peered":"2023-09-30T22:14:02.067283+0000","last_unstale":"2023-09-30T22:14:02.080473+0000","last_undegraded":"2023-09-30T22:14:02.080473+0000","last_fullsized":"2023-09-30T22:14:02.080473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087735+0000","last_change":"2023-09-30T22:14:02.082875+0000","last_active":"2023-09-30T22:14:02.087735+0000","last_peered":"2023-09-30T22:14:02.087735+0000","last_clean":"2023-09-30T22:14:02.087735+0000","last_became_active":"2023-09-30T22:14:02.082617+0000","last_became_peered":"2023-09-30T22:14:02.082617+0000","last_unstale":"2023-09-30T22:14:02.087735+0000","last_undegraded":"2023-09-30T22:14:02.087735+0000","last_fullsized":"2023-09-30T22:14:02.087735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079595+0000","last_change":"2023-09-30T22:14:02.076951+0000","last_active":"2023-09-30T22:14:02.079595+0000","last_peered":"2023-09-30T22:14:02.079595+0000","last_clean":"2023-09-30T22:14:02.079595+0000","last_became_active":"2023-09-30T22:14:02.076448+0000","last_became_peered":"2023-09-30T22:14:02.076448+0000","last_unstale":"2023-09-30T22:14:02.079595+0000","last_undegraded":"2023-09-30T22:14:02.079595+0000","last_fullsized":"2023-09-30T22:14:02.079595+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.214}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:14:14.362 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:14:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.512+0000 7fbae5c0e700 1 Processor -- start 2023-09-30T22:14:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.512+0000 7fbae5c0e700 1 -- start start 2023-09-30T22:14:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbae5c0e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbae0096380 0x7fbae0094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbae5c0e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0094c20 0x7fbae0127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbae5c0e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae0096760 con 0x7fbae0091b00 2023-09-30T22:14:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbae5c0e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae0091ff0 con 0x7fbae0096380 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbae5c0e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0092c80 con 0x7fbae0094c20 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbae0096380 0x7fbae0094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadeffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbae0091b00 0x7fbae0127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54282/0 (socket says 172.21.15.89:54282) 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0094c20 0x7fbae0127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbae0096380 0x7fbae0094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36810/0 (socket says 172.21.15.89:36810) 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadeffd700 1 -- 172.21.15.89:0/1200284499 learned_addr learned my addr 172.21.15.89:0/1200284499 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadffff700 1 -- 172.21.15.89:0/1200284499 >> v2:172.21.15.89:3300/0 conn(0x7fbae0096380 msgr2=0x7fbae0094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadffff700 1 --2- 172.21.15.89:0/1200284499 >> v2:172.21.15.89:3300/0 conn(0x7fbae0096380 0x7fbae0094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadffff700 1 -- 172.21.15.89:0/1200284499 >> v1:172.21.15.89:6789/0 conn(0x7fbae0091b00 legacy=0x7fbae0127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadffff700 1 -- 172.21.15.89:0/1200284499 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae0093900 con 0x7fbae0094c20 2023-09-30T22:14:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbade7fc700 1 -- 172.21.15.89:0/1200284499 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616364858 0 0) 0x7fbae0096760 con 0x7fbae0091b00 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadf7fe700 1 --2- 172.21.15.89:0/1200284499 >> v2:172.21.15.89:3300/0 conn(0x7fbae0096380 0x7fbae0094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.513+0000 7fbadffff700 1 --2- 172.21.15.89:0/1200284499 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0094c20 0x7fbae0127d10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fbad8009e60 tx=0x7fbad8009780).ready entity=mon.1 client_cookie=64b35d30e48a710e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbade7fc700 1 -- 172.21.15.89:0/1200284499 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad800e070 con 0x7fbae0094c20 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbade7fc700 1 -- 172.21.15.89:0/1200284499 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad800bb60 con 0x7fbae0094c20 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbade7fc700 1 -- 172.21.15.89:0/1200284499 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8012410 con 0x7fbae0094c20 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbae5c0e700 1 -- 172.21.15.89:0/1200284499 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0094c20 msgr2=0x7fbae0127d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbae5c0e700 1 --2- 172.21.15.89:0/1200284499 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0094c20 0x7fbae0127d10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fbad8009e60 tx=0x7fbad8009780).stop 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbae5c0e700 1 -- 172.21.15.89:0/1200284499 shutdown_connections 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbae5c0e700 1 --2- 172.21.15.89:0/1200284499 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0094c20 0x7fbae0127d10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbae5c0e700 1 --2- 172.21.15.89:0/1200284499 >> v2:172.21.15.89:3300/0 conn(0x7fbae0096380 0x7fbae0094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbae5c0e700 1 -- 172.21.15.89:0/1200284499 >> 172.21.15.89:0/1200284499 conn(0x7fbae0097470 msgr2=0x7fbae0097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbae5c0e700 1 -- 172.21.15.89:0/1200284499 shutdown_connections 2023-09-30T22:14:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.514+0000 7fbae5c0e700 1 -- 172.21.15.89:0/1200284499 wait complete. 2023-09-30T22:14:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.515+0000 7fbae5c0e700 1 Processor -- start 2023-09-30T22:14:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.515+0000 7fbae5c0e700 1 -- start start 2023-09-30T22:14:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbae5c0e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0091b00 0x7fbae0112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbae5c0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbae0094c20 0x7fbae01132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbae5c0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbae0096380 0x7fbae01137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbae5c0e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0125020 con 0x7fbae0094c20 2023-09-30T22:14:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbae5c0e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0098cb0 con 0x7fbae0091b00 2023-09-30T22:14:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbae5c0e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbae006a520 con 0x7fbae0096380 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0091b00 0x7fbae0112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbae0094c20 0x7fbae01132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbae0096380 0x7fbae01137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0091b00 0x7fbae0112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52224/0 (socket says 172.21.15.89:52224) 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadf7fe700 1 -- 172.21.15.89:0/78481554 learned_addr learned my addr 172.21.15.89:0/78481554 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadeffd700 1 -- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbae0096380 msgr2=0x7fbae01137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadeffd700 1 --2- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbae0096380 0x7fbae01137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadeffd700 1 -- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0091b00 msgr2=0x7fbae0112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadeffd700 1 --2- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0091b00 0x7fbae0112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadeffd700 1 -- 172.21.15.89:0/78481554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad00092a0 con 0x7fbae0094c20 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.516+0000 7fbadf7fe700 1 --2- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0091b00 0x7fbae0112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.517+0000 7fbadeffd700 1 --2- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbae0094c20 0x7fbae01132d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fbac8000c00 tx=0x7fbac8007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.517+0000 7fbad77fe700 1 -- 172.21.15.89:0/78481554 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac80070d0 con 0x7fbae0094c20 2023-09-30T22:14:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.517+0000 7fbad77fe700 1 -- 172.21.15.89:0/78481554 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac8007250 con 0x7fbae0094c20 2023-09-30T22:14:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.517+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad8009a20 con 0x7fbae0094c20 2023-09-30T22:14:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.517+0000 7fbad77fe700 1 -- 172.21.15.89:0/78481554 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8007610 con 0x7fbae0094c20 2023-09-30T22:14:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.517+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae0116b00 con 0x7fbae0094c20 2023-09-30T22:14:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.518+0000 7fbad77fe700 1 -- 172.21.15.89:0/78481554 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fbac80059b0 con 0x7fbae0094c20 2023-09-30T22:14:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.518+0000 7fbad77fe700 1 -- 172.21.15.89:0/78481554 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fbac803a3e0 con 0x7fbae0094c20 2023-09-30T22:14:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.518+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae0116b00 con 0x7fbae0094c20 2023-09-30T22:14:14.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.522+0000 7fbad77fe700 1 -- 172.21.15.89:0/78481554 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fbae0116b00 con 0x7fbae0094c20 2023-09-30T22:14:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.676+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 --> v1:172.21.15.89:6816/118161 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbae0116b00 con 0x7fbac002bb90 2023-09-30T22:14:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.682+0000 7fbad77fe700 1 -- 172.21.15.89:0/78481554 <== mgr.4102 v1:172.21.15.89:6816/118161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249947 (unknown 2965378022 0 82204692) 0x7fbae0116b00 con 0x7fbac002bb90 2023-09-30T22:14:14.685 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.684+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 >> v1:172.21.15.89:6816/118161 conn(0x7fbac002bb90 legacy=0x7fbac002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbae0094c20 msgr2=0x7fbae01132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 --2- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbae0094c20 0x7fbae01132d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fbac8000c00 tx=0x7fbac8007b90).stop 2023-09-30T22:14:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 shutdown_connections 2023-09-30T22:14:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 --2- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbae0096380 0x7fbae01137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 --2- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbae0091b00 0x7fbae0112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 --2- 172.21.15.89:0/78481554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbae0094c20 0x7fbae01132d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 >> 172.21.15.89:0/78481554 conn(0x7fbae0097470 msgr2=0x7fbae0127640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 shutdown_connections 2023-09-30T22:14:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.685+0000 7fbae5c0e700 1 -- 172.21.15.89:0/78481554 wait complete. 2023-09-30T22:14:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:14:14.696 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-30T22:14:12.747317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001520"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090297+0000","last_change":"2023-09-30T22:14:02.086445+0000","last_active":"2023-09-30T22:14:02.090297+0000","last_peered":"2023-09-30T22:14:02.090297+0000","last_clean":"2023-09-30T22:14:02.090297+0000","last_became_active":"2023-09-30T22:14:02.086159+0000","last_became_peered":"2023-09-30T22:14:02.086159+0000","last_unstale":"2023-09-30T22:14:02.090297+0000","last_undegraded":"2023-09-30T22:14:02.090297+0000","last_fullsized":"2023-09-30T22:14:02.090297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088940+0000","last_change":"2023-09-30T22:14:02.082560+0000","last_active":"2023-09-30T22:14:02.088940+0000","last_peered":"2023-09-30T22:14:02.088940+0000","last_clean":"2023-09-30T22:14:02.088940+0000","last_became_active":"2023-09-30T22:14:02.082315+0000","last_became_peered":"2023-09-30T22:14:02.082315+0000","last_unstale":"2023-09-30T22:14:02.088940+0000","last_undegraded":"2023-09-30T22:14:02.088940+0000","last_fullsized":"2023-09-30T22:14:02.088940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091544+0000","last_change":"2023-09-30T22:14:02.082673+0000","last_active":"2023-09-30T22:14:02.091544+0000","last_peered":"2023-09-30T22:14:02.091544+0000","last_clean":"2023-09-30T22:14:02.091544+0000","last_became_active":"2023-09-30T22:14:02.082437+0000","last_became_peered":"2023-09-30T22:14:02.082437+0000","last_unstale":"2023-09-30T22:14:02.091544+0000","last_undegraded":"2023-09-30T22:14:02.091544+0000","last_fullsized":"2023-09-30T22:14:02.091544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091649+0000","last_change":"2023-09-30T22:14:02.080886+0000","last_active":"2023-09-30T22:14:02.091649+0000","last_peered":"2023-09-30T22:14:02.091649+0000","last_clean":"2023-09-30T22:14:02.091649+0000","last_became_active":"2023-09-30T22:14:02.078093+0000","last_became_peered":"2023-09-30T22:14:02.078093+0000","last_unstale":"2023-09-30T22:14:02.091649+0000","last_undegraded":"2023-09-30T22:14:02.091649+0000","last_fullsized":"2023-09-30T22:14:02.091649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087971+0000","last_change":"2023-09-30T22:14:02.074967+0000","last_active":"2023-09-30T22:14:02.087971+0000","last_peered":"2023-09-30T22:14:02.087971+0000","last_clean":"2023-09-30T22:14:02.087971+0000","last_became_active":"2023-09-30T22:14:02.074701+0000","last_became_peered":"2023-09-30T22:14:02.074701+0000","last_unstale":"2023-09-30T22:14:02.087971+0000","last_undegraded":"2023-09-30T22:14:02.087971+0000","last_fullsized":"2023-09-30T22:14:02.087971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087513+0000","last_change":"2023-09-30T22:14:02.081443+0000","last_active":"2023-09-30T22:14:02.087513+0000","last_peered":"2023-09-30T22:14:02.087513+0000","last_clean":"2023-09-30T22:14:02.087513+0000","last_became_active":"2023-09-30T22:14:02.080408+0000","last_became_peered":"2023-09-30T22:14:02.080408+0000","last_unstale":"2023-09-30T22:14:02.087513+0000","last_undegraded":"2023-09-30T22:14:02.087513+0000","last_fullsized":"2023-09-30T22:14:02.087513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093729+0000","last_change":"2023-09-30T22:14:02.085310+0000","last_active":"2023-09-30T22:14:02.093729+0000","last_peered":"2023-09-30T22:14:02.093729+0000","last_clean":"2023-09-30T22:14:02.093729+0000","last_became_active":"2023-09-30T22:14:02.084647+0000","last_became_peered":"2023-09-30T22:14:02.084647+0000","last_unstale":"2023-09-30T22:14:02.093729+0000","last_undegraded":"2023-09-30T22:14:02.093729+0000","last_fullsized":"2023-09-30T22:14:02.093729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093827+0000","last_change":"2023-09-30T22:14:02.085988+0000","last_active":"2023-09-30T22:14:02.093827+0000","last_peered":"2023-09-30T22:14:02.093827+0000","last_clean":"2023-09-30T22:14:02.093827+0000","last_became_active":"2023-09-30T22:14:02.083245+0000","last_became_peered":"2023-09-30T22:14:02.083245+0000","last_unstale":"2023-09-30T22:14:02.093827+0000","last_undegraded":"2023-09-30T22:14:02.093827+0000","last_fullsized":"2023-09-30T22:14:02.093827+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091844+0000","last_change":"2023-09-30T22:14:02.085522+0000","last_active":"2023-09-30T22:14:02.091844+0000","last_peered":"2023-09-30T22:14:02.091844+0000","last_clean":"2023-09-30T22:14:02.091844+0000","last_became_active":"2023-09-30T22:14:02.084959+0000","last_became_peered":"2023-09-30T22:14:02.084959+0000","last_unstale":"2023-09-30T22:14:02.091844+0000","last_undegraded":"2023-09-30T22:14:02.091844+0000","last_fullsized":"2023-09-30T22:14:02.091844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.078675+0000","last_change":"2023-09-30T22:14:02.071672+0000","last_active":"2023-09-30T22:14:02.078675+0000","last_peered":"2023-09-30T22:14:02.078675+0000","last_clean":"2023-09-30T22:14:02.078675+0000","last_became_active":"2023-09-30T22:14:02.071361+0000","last_became_peered":"2023-09-30T22:14:02.071361+0000","last_unstale":"2023-09-30T22:14:02.078675+0000","last_undegraded":"2023-09-30T22:14:02.078675+0000","last_fullsized":"2023-09-30T22:14:02.078675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.077649+0000","last_change":"2023-09-30T22:14:02.074689+0000","last_active":"2023-09-30T22:14:02.077649+0000","last_peered":"2023-09-30T22:14:02.077649+0000","last_clean":"2023-09-30T22:14:02.077649+0000","last_became_active":"2023-09-30T22:14:02.074350+0000","last_became_peered":"2023-09-30T22:14:02.074350+0000","last_unstale":"2023-09-30T22:14:02.077649+0000","last_undegraded":"2023-09-30T22:14:02.077649+0000","last_fullsized":"2023-09-30T22:14:02.077649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090695+0000","last_change":"2023-09-30T22:14:02.086959+0000","last_active":"2023-09-30T22:14:02.090695+0000","last_peered":"2023-09-30T22:14:02.090695+0000","last_clean":"2023-09-30T22:14:02.090695+0000","last_became_active":"2023-09-30T22:14:02.086719+0000","last_became_peered":"2023-09-30T22:14:02.086719+0000","last_unstale":"2023-09-30T22:14:02.090695+0000","last_undegraded":"2023-09-30T22:14:02.090695+0000","last_fullsized":"2023-09-30T22:14:02.090695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090392+0000","last_change":"2023-09-30T22:14:02.087347+0000","last_active":"2023-09-30T22:14:02.090392+0000","last_peered":"2023-09-30T22:14:02.090392+0000","last_clean":"2023-09-30T22:14:02.090392+0000","last_became_active":"2023-09-30T22:14:02.086637+0000","last_became_peered":"2023-09-30T22:14:02.086637+0000","last_unstale":"2023-09-30T22:14:02.090392+0000","last_undegraded":"2023-09-30T22:14:02.090392+0000","last_fullsized":"2023-09-30T22:14:02.090392+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.073518+0000","last_change":"2023-09-30T22:14:02.066399+0000","last_active":"2023-09-30T22:14:02.073518+0000","last_peered":"2023-09-30T22:14:02.073518+0000","last_clean":"2023-09-30T22:14:02.073518+0000","last_became_active":"2023-09-30T22:14:02.066052+0000","last_became_peered":"2023-09-30T22:14:02.066052+0000","last_unstale":"2023-09-30T22:14:02.073518+0000","last_undegraded":"2023-09-30T22:14:02.073518+0000","last_fullsized":"2023-09-30T22:14:02.073518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083856+0000","last_change":"2023-09-30T22:14:02.073165+0000","last_active":"2023-09-30T22:14:02.083856+0000","last_peered":"2023-09-30T22:14:02.083856+0000","last_clean":"2023-09-30T22:14:02.083856+0000","last_became_active":"2023-09-30T22:14:02.073000+0000","last_became_peered":"2023-09-30T22:14:02.073000+0000","last_unstale":"2023-09-30T22:14:02.083856+0000","last_undegraded":"2023-09-30T22:14:02.083856+0000","last_fullsized":"2023-09-30T22:14:02.083856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083597+0000","last_change":"2023-09-30T22:14:02.072772+0000","last_active":"2023-09-30T22:14:02.083597+0000","last_peered":"2023-09-30T22:14:02.083597+0000","last_clean":"2023-09-30T22:14:02.083597+0000","last_became_active":"2023-09-30T22:14:02.072569+0000","last_became_peered":"2023-09-30T22:14:02.072569+0000","last_unstale":"2023-09-30T22:14:02.083597+0000","last_undegraded":"2023-09-30T22:14:02.083597+0000","last_fullsized":"2023-09-30T22:14:02.083597+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091862+0000","last_change":"2023-09-30T22:14:02.085679+0000","last_active":"2023-09-30T22:14:02.091862+0000","last_peered":"2023-09-30T22:14:02.091862+0000","last_clean":"2023-09-30T22:14:02.091862+0000","last_became_active":"2023-09-30T22:14:02.085267+0000","last_became_peered":"2023-09-30T22:14:02.085267+0000","last_unstale":"2023-09-30T22:14:02.091862+0000","last_undegraded":"2023-09-30T22:14:02.091862+0000","last_fullsized":"2023-09-30T22:14:02.091862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082225+0000","last_change":"2023-09-30T22:14:02.070008+0000","last_active":"2023-09-30T22:14:02.082225+0000","last_peered":"2023-09-30T22:14:02.082225+0000","last_clean":"2023-09-30T22:14:02.082225+0000","last_became_active":"2023-09-30T22:14:02.069809+0000","last_became_peered":"2023-09-30T22:14:02.069809+0000","last_unstale":"2023-09-30T22:14:02.082225+0000","last_undegraded":"2023-09-30T22:14:02.082225+0000","last_fullsized":"2023-09-30T22:14:02.082225+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090720+0000","last_change":"2023-09-30T22:14:02.087939+0000","last_active":"2023-09-30T22:14:02.090720+0000","last_peered":"2023-09-30T22:14:02.090720+0000","last_clean":"2023-09-30T22:14:02.090720+0000","last_became_active":"2023-09-30T22:14:02.087733+0000","last_became_peered":"2023-09-30T22:14:02.087733+0000","last_unstale":"2023-09-30T22:14:02.090720+0000","last_undegraded":"2023-09-30T22:14:02.090720+0000","last_fullsized":"2023-09-30T22:14:02.090720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093991+0000","last_change":"2023-09-30T22:14:02.090627+0000","last_active":"2023-09-30T22:14:02.093991+0000","last_peered":"2023-09-30T22:14:02.093991+0000","last_clean":"2023-09-30T22:14:02.093991+0000","last_became_active":"2023-09-30T22:14:02.090255+0000","last_became_peered":"2023-09-30T22:14:02.090255+0000","last_unstale":"2023-09-30T22:14:02.093991+0000","last_undegraded":"2023-09-30T22:14:02.093991+0000","last_fullsized":"2023-09-30T22:14:02.093991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.076392+0000","last_change":"2023-09-30T22:14:02.071417+0000","last_active":"2023-09-30T22:14:02.076392+0000","last_peered":"2023-09-30T22:14:02.076392+0000","last_clean":"2023-09-30T22:14:02.076392+0000","last_became_active":"2023-09-30T22:14:02.071046+0000","last_became_peered":"2023-09-30T22:14:02.071046+0000","last_unstale":"2023-09-30T22:14:02.076392+0000","last_undegraded":"2023-09-30T22:14:02.076392+0000","last_fullsized":"2023-09-30T22:14:02.076392+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088095+0000","last_change":"2023-09-30T22:14:02.076131+0000","last_active":"2023-09-30T22:14:02.088095+0000","last_peered":"2023-09-30T22:14:02.088095+0000","last_clean":"2023-09-30T22:14:02.088095+0000","last_became_active":"2023-09-30T22:14:02.075850+0000","last_became_peered":"2023-09-30T22:14:02.075850+0000","last_unstale":"2023-09-30T22:14:02.088095+0000","last_undegraded":"2023-09-30T22:14:02.088095+0000","last_fullsized":"2023-09-30T22:14:02.088095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093707+0000","last_change":"2023-09-30T22:14:02.085210+0000","last_active":"2023-09-30T22:14:02.093707+0000","last_peered":"2023-09-30T22:14:02.093707+0000","last_clean":"2023-09-30T22:14:02.093707+0000","last_became_active":"2023-09-30T22:14:02.084320+0000","last_became_peered":"2023-09-30T22:14:02.084320+0000","last_unstale":"2023-09-30T22:14:02.093707+0000","last_undegraded":"2023-09-30T22:14:02.093707+0000","last_fullsized":"2023-09-30T22:14:02.093707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093806+0000","last_change":"2023-09-30T22:14:02.084642+0000","last_active":"2023-09-30T22:14:02.093806+0000","last_peered":"2023-09-30T22:14:02.093806+0000","last_clean":"2023-09-30T22:14:02.093806+0000","last_became_active":"2023-09-30T22:14:02.084358+0000","last_became_peered":"2023-09-30T22:14:02.084358+0000","last_unstale":"2023-09-30T22:14:02.093806+0000","last_undegraded":"2023-09-30T22:14:02.093806+0000","last_fullsized":"2023-09-30T22:14:02.093806+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.098581+0000","last_change":"2023-09-30T22:14:02.093959+0000","last_active":"2023-09-30T22:14:02.098581+0000","last_peered":"2023-09-30T22:14:02.098581+0000","last_clean":"2023-09-30T22:14:02.098581+0000","last_became_active":"2023-09-30T22:14:02.093149+0000","last_became_peered":"2023-09-30T22:14:02.093149+0000","last_unstale":"2023-09-30T22:14:02.098581+0000","last_undegraded":"2023-09-30T22:14:02.098581+0000","last_fullsized":"2023-09-30T22:14:02.098581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079587+0000","last_change":"2023-09-30T22:14:02.077989+0000","last_active":"2023-09-30T22:14:02.079587+0000","last_peered":"2023-09-30T22:14:02.079587+0000","last_clean":"2023-09-30T22:14:02.079587+0000","last_became_active":"2023-09-30T22:14:02.071401+0000","last_became_peered":"2023-09-30T22:14:02.071401+0000","last_unstale":"2023-09-30T22:14:02.079587+0000","last_undegraded":"2023-09-30T22:14:02.079587+0000","last_fullsized":"2023-09-30T22:14:02.079587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085651+0000","last_change":"2023-09-30T22:14:02.075617+0000","last_active":"2023-09-30T22:14:02.085651+0000","last_peered":"2023-09-30T22:14:02.085651+0000","last_clean":"2023-09-30T22:14:02.085651+0000","last_became_active":"2023-09-30T22:14:02.074584+0000","last_became_peered":"2023-09-30T22:14:02.074584+0000","last_unstale":"2023-09-30T22:14:02.085651+0000","last_undegraded":"2023-09-30T22:14:02.085651+0000","last_fullsized":"2023-09-30T22:14:02.085651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.073438+0000","last_change":"2023-09-30T22:14:01.697398+0000","last_active":"2023-09-30T22:14:02.073438+0000","last_peered":"2023-09-30T22:14:02.073438+0000","last_clean":"2023-09-30T22:14:02.073438+0000","last_became_active":"2023-09-30T22:14:01.694824+0000","last_became_peered":"2023-09-30T22:14:01.694824+0000","last_unstale":"2023-09-30T22:14:02.073438+0000","last_undegraded":"2023-09-30T22:14:02.073438+0000","last_fullsized":"2023-09-30T22:14:02.073438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085672+0000","last_change":"2023-09-30T22:14:02.075771+0000","last_active":"2023-09-30T22:14:02.085672+0000","last_peered":"2023-09-30T22:14:02.085672+0000","last_clean":"2023-09-30T22:14:02.085672+0000","last_became_active":"2023-09-30T22:14:02.074872+0000","last_became_peered":"2023-09-30T22:14:02.074872+0000","last_unstale":"2023-09-30T22:14:02.085672+0000","last_undegraded":"2023-09-30T22:14:02.085672+0000","last_fullsized":"2023-09-30T22:14:02.085672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053010+0000","last_change":"2023-09-30T22:14:01.702319+0000","last_active":"2023-09-30T22:14:02.053010+0000","last_peered":"2023-09-30T22:14:02.053010+0000","last_clean":"2023-09-30T22:14:02.053010+0000","last_became_active":"2023-09-30T22:14:01.702020+0000","last_became_peered":"2023-09-30T22:14:01.702020+0000","last_unstale":"2023-09-30T22:14:02.053010+0000","last_undegraded":"2023-09-30T22:14:02.053010+0000","last_fullsized":"2023-09-30T22:14:02.053010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091650+0000","last_change":"2023-09-30T22:14:02.079713+0000","last_active":"2023-09-30T22:14:02.091650+0000","last_peered":"2023-09-30T22:14:02.091650+0000","last_clean":"2023-09-30T22:14:02.091650+0000","last_became_active":"2023-09-30T22:14:02.079318+0000","last_became_peered":"2023-09-30T22:14:02.079318+0000","last_unstale":"2023-09-30T22:14:02.091650+0000","last_undegraded":"2023-09-30T22:14:02.091650+0000","last_fullsized":"2023-09-30T22:14:02.091650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053665+0000","last_change":"2023-09-30T22:14:01.707097+0000","last_active":"2023-09-30T22:14:02.053665+0000","last_peered":"2023-09-30T22:14:02.053665+0000","last_clean":"2023-09-30T22:14:02.053665+0000","last_became_active":"2023-09-30T22:14:01.706849+0000","last_became_peered":"2023-09-30T22:14:01.706849+0000","last_unstale":"2023-09-30T22:14:02.053665+0000","last_undegraded":"2023-09-30T22:14:02.053665+0000","last_fullsized":"2023-09-30T22:14:02.053665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087679+0000","last_change":"2023-09-30T22:14:02.084631+0000","last_active":"2023-09-30T22:14:02.087679+0000","last_peered":"2023-09-30T22:14:02.087679+0000","last_clean":"2023-09-30T22:14:02.087679+0000","last_became_active":"2023-09-30T22:14:02.084356+0000","last_became_peered":"2023-09-30T22:14:02.084356+0000","last_unstale":"2023-09-30T22:14:02.087679+0000","last_undegraded":"2023-09-30T22:14:02.087679+0000","last_fullsized":"2023-09-30T22:14:02.087679+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057794+0000","last_change":"2023-09-30T22:14:01.698007+0000","last_active":"2023-09-30T22:14:02.057794+0000","last_peered":"2023-09-30T22:14:02.057794+0000","last_clean":"2023-09-30T22:14:02.057794+0000","last_became_active":"2023-09-30T22:14:01.697731+0000","last_became_peered":"2023-09-30T22:14:01.697731+0000","last_unstale":"2023-09-30T22:14:02.057794+0000","last_undegraded":"2023-09-30T22:14:02.057794+0000","last_fullsized":"2023-09-30T22:14:02.057794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087841+0000","last_change":"2023-09-30T22:14:02.082853+0000","last_active":"2023-09-30T22:14:02.087841+0000","last_peered":"2023-09-30T22:14:02.087841+0000","last_clean":"2023-09-30T22:14:02.087841+0000","last_became_active":"2023-09-30T22:14:02.082597+0000","last_became_peered":"2023-09-30T22:14:02.082597+0000","last_unstale":"2023-09-30T22:14:02.087841+0000","last_undegraded":"2023-09-30T22:14:02.087841+0000","last_fullsized":"2023-09-30T22:14:02.087841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056502+0000","last_change":"2023-09-30T22:14:01.700657+0000","last_active":"2023-09-30T22:14:02.056502+0000","last_peered":"2023-09-30T22:14:02.056502+0000","last_clean":"2023-09-30T22:14:02.056502+0000","last_became_active":"2023-09-30T22:14:01.699633+0000","last_became_peered":"2023-09-30T22:14:01.699633+0000","last_unstale":"2023-09-30T22:14:02.056502+0000","last_undegraded":"2023-09-30T22:14:02.056502+0000","last_fullsized":"2023-09-30T22:14:02.056502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093773+0000","last_change":"2023-09-30T22:14:02.085416+0000","last_active":"2023-09-30T22:14:02.093773+0000","last_peered":"2023-09-30T22:14:02.093773+0000","last_clean":"2023-09-30T22:14:02.093773+0000","last_became_active":"2023-09-30T22:14:02.084970+0000","last_became_peered":"2023-09-30T22:14:02.084970+0000","last_unstale":"2023-09-30T22:14:02.093773+0000","last_undegraded":"2023-09-30T22:14:02.093773+0000","last_fullsized":"2023-09-30T22:14:02.093773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083245+0000","last_change":"2023-09-30T22:14:01.701075+0000","last_active":"2023-09-30T22:14:02.083245+0000","last_peered":"2023-09-30T22:14:02.083245+0000","last_clean":"2023-09-30T22:14:02.083245+0000","last_became_active":"2023-09-30T22:14:01.700806+0000","last_became_peered":"2023-09-30T22:14:01.700806+0000","last_unstale":"2023-09-30T22:14:02.083245+0000","last_undegraded":"2023-09-30T22:14:02.083245+0000","last_fullsized":"2023-09-30T22:14:02.083245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.077417+0000","last_change":"2023-09-30T22:14:02.074078+0000","last_active":"2023-09-30T22:14:02.077417+0000","last_peered":"2023-09-30T22:14:02.077417+0000","last_clean":"2023-09-30T22:14:02.077417+0000","last_became_active":"2023-09-30T22:14:02.073804+0000","last_became_peered":"2023-09-30T22:14:02.073804+0000","last_unstale":"2023-09-30T22:14:02.077417+0000","last_undegraded":"2023-09-30T22:14:02.077417+0000","last_fullsized":"2023-09-30T22:14:02.077417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.096081+0000","last_change":"2023-09-30T22:14:01.711172+0000","last_active":"2023-09-30T22:14:02.096081+0000","last_peered":"2023-09-30T22:14:02.096081+0000","last_clean":"2023-09-30T22:14:02.096081+0000","last_became_active":"2023-09-30T22:14:01.710926+0000","last_became_peered":"2023-09-30T22:14:01.710926+0000","last_unstale":"2023-09-30T22:14:02.096081+0000","last_undegraded":"2023-09-30T22:14:02.096081+0000","last_fullsized":"2023-09-30T22:14:02.096081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090366+0000","last_change":"2023-09-30T22:14:02.087193+0000","last_active":"2023-09-30T22:14:02.090366+0000","last_peered":"2023-09-30T22:14:02.090366+0000","last_clean":"2023-09-30T22:14:02.090366+0000","last_became_active":"2023-09-30T22:14:02.086273+0000","last_became_peered":"2023-09-30T22:14:02.086273+0000","last_unstale":"2023-09-30T22:14:02.090366+0000","last_undegraded":"2023-09-30T22:14:02.090366+0000","last_fullsized":"2023-09-30T22:14:02.090366+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082985+0000","last_change":"2023-09-30T22:14:01.706560+0000","last_active":"2023-09-30T22:14:02.082985+0000","last_peered":"2023-09-30T22:14:02.082985+0000","last_clean":"2023-09-30T22:14:02.082985+0000","last_became_active":"2023-09-30T22:14:01.706330+0000","last_became_peered":"2023-09-30T22:14:01.706330+0000","last_unstale":"2023-09-30T22:14:02.082985+0000","last_undegraded":"2023-09-30T22:14:02.082985+0000","last_fullsized":"2023-09-30T22:14:02.082985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087699+0000","last_change":"2023-09-30T22:14:02.084139+0000","last_active":"2023-09-30T22:14:02.087699+0000","last_peered":"2023-09-30T22:14:02.087699+0000","last_clean":"2023-09-30T22:14:02.087699+0000","last_became_active":"2023-09-30T22:14:02.080986+0000","last_became_peered":"2023-09-30T22:14:02.080986+0000","last_unstale":"2023-09-30T22:14:02.087699+0000","last_undegraded":"2023-09-30T22:14:02.087699+0000","last_fullsized":"2023-09-30T22:14:02.087699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.055029+0000","last_change":"2023-09-30T22:14:01.698141+0000","last_active":"2023-09-30T22:14:02.055029+0000","last_peered":"2023-09-30T22:14:02.055029+0000","last_clean":"2023-09-30T22:14:02.055029+0000","last_became_active":"2023-09-30T22:14:01.697885+0000","last_became_peered":"2023-09-30T22:14:01.697885+0000","last_unstale":"2023-09-30T22:14:02.055029+0000","last_undegraded":"2023-09-30T22:14:02.055029+0000","last_fullsized":"2023-09-30T22:14:02.055029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085690+0000","last_change":"2023-09-30T22:14:02.075925+0000","last_active":"2023-09-30T22:14:02.085690+0000","last_peered":"2023-09-30T22:14:02.085690+0000","last_clean":"2023-09-30T22:14:02.085690+0000","last_became_active":"2023-09-30T22:14:02.075137+0000","last_became_peered":"2023-09-30T22:14:02.075137+0000","last_unstale":"2023-09-30T22:14:02.085690+0000","last_undegraded":"2023-09-30T22:14:02.085690+0000","last_fullsized":"2023-09-30T22:14:02.085690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.064991+0000","last_change":"2023-09-30T22:14:01.712299+0000","last_active":"2023-09-30T22:14:02.064991+0000","last_peered":"2023-09-30T22:14:02.064991+0000","last_clean":"2023-09-30T22:14:02.064991+0000","last_became_active":"2023-09-30T22:14:01.711791+0000","last_became_peered":"2023-09-30T22:14:01.711791+0000","last_unstale":"2023-09-30T22:14:02.064991+0000","last_undegraded":"2023-09-30T22:14:02.064991+0000","last_fullsized":"2023-09-30T22:14:02.064991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091669+0000","last_change":"2023-09-30T22:14:02.084033+0000","last_active":"2023-09-30T22:14:02.091669+0000","last_peered":"2023-09-30T22:14:02.091669+0000","last_clean":"2023-09-30T22:14:02.091669+0000","last_became_active":"2023-09-30T22:14:02.083764+0000","last_became_peered":"2023-09-30T22:14:02.083764+0000","last_unstale":"2023-09-30T22:14:02.091669+0000","last_undegraded":"2023-09-30T22:14:02.091669+0000","last_fullsized":"2023-09-30T22:14:02.091669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.092132+0000","last_change":"2023-09-30T22:14:01.706777+0000","last_active":"2023-09-30T22:14:02.092132+0000","last_peered":"2023-09-30T22:14:02.092132+0000","last_clean":"2023-09-30T22:14:02.092132+0000","last_became_active":"2023-09-30T22:14:01.706479+0000","last_became_peered":"2023-09-30T22:14:01.706479+0000","last_unstale":"2023-09-30T22:14:02.092132+0000","last_undegraded":"2023-09-30T22:14:02.092132+0000","last_fullsized":"2023-09-30T22:14:02.092132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087631+0000","last_change":"2023-09-30T22:14:02.084399+0000","last_active":"2023-09-30T22:14:02.087631+0000","last_peered":"2023-09-30T22:14:02.087631+0000","last_clean":"2023-09-30T22:14:02.087631+0000","last_became_active":"2023-09-30T22:14:02.084137+0000","last_became_peered":"2023-09-30T22:14:02.084137+0000","last_unstale":"2023-09-30T22:14:02.087631+0000","last_undegraded":"2023-09-30T22:14:02.087631+0000","last_fullsized":"2023-09-30T22:14:02.087631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093442+0000","last_change":"2023-09-30T22:14:01.702685+0000","last_active":"2023-09-30T22:14:02.093442+0000","last_peered":"2023-09-30T22:14:02.093442+0000","last_clean":"2023-09-30T22:14:02.093442+0000","last_became_active":"2023-09-30T22:14:01.702083+0000","last_became_peered":"2023-09-30T22:14:01.702083+0000","last_unstale":"2023-09-30T22:14:02.093442+0000","last_undegraded":"2023-09-30T22:14:02.093442+0000","last_fullsized":"2023-09-30T22:14:02.093442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087823+0000","last_change":"2023-09-30T22:14:02.082069+0000","last_active":"2023-09-30T22:14:02.087823+0000","last_peered":"2023-09-30T22:14:02.087823+0000","last_clean":"2023-09-30T22:14:02.087823+0000","last_became_active":"2023-09-30T22:14:02.081702+0000","last_became_peered":"2023-09-30T22:14:02.081702+0000","last_unstale":"2023-09-30T22:14:02.087823+0000","last_undegraded":"2023-09-30T22:14:02.087823+0000","last_fullsized":"2023-09-30T22:14:02.087823+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.069430+0000","last_change":"2023-09-30T22:14:01.706748+0000","last_active":"2023-09-30T22:14:02.069430+0000","last_peered":"2023-09-30T22:14:02.069430+0000","last_clean":"2023-09-30T22:14:02.069430+0000","last_became_active":"2023-09-30T22:14:01.706472+0000","last_became_peered":"2023-09-30T22:14:01.706472+0000","last_unstale":"2023-09-30T22:14:02.069430+0000","last_undegraded":"2023-09-30T22:14:02.069430+0000","last_fullsized":"2023-09-30T22:14:02.069430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.076358+0000","last_change":"2023-09-30T22:14:02.070587+0000","last_active":"2023-09-30T22:14:02.076358+0000","last_peered":"2023-09-30T22:14:02.076358+0000","last_clean":"2023-09-30T22:14:02.076358+0000","last_became_active":"2023-09-30T22:14:02.070259+0000","last_became_peered":"2023-09-30T22:14:02.070259+0000","last_unstale":"2023-09-30T22:14:02.076358+0000","last_undegraded":"2023-09-30T22:14:02.076358+0000","last_fullsized":"2023-09-30T22:14:02.076358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080959+0000","last_change":"2023-09-30T22:14:01.704603+0000","last_active":"2023-09-30T22:14:02.080959+0000","last_peered":"2023-09-30T22:14:02.080959+0000","last_clean":"2023-09-30T22:14:02.080959+0000","last_became_active":"2023-09-30T22:14:01.704006+0000","last_became_peered":"2023-09-30T22:14:01.704006+0000","last_unstale":"2023-09-30T22:14:02.080959+0000","last_undegraded":"2023-09-30T22:14:02.080959+0000","last_fullsized":"2023-09-30T22:14:02.080959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080942+0000","last_change":"2023-09-30T22:14:02.073527+0000","last_active":"2023-09-30T22:14:02.080942+0000","last_peered":"2023-09-30T22:14:02.080942+0000","last_clean":"2023-09-30T22:14:02.080942+0000","last_became_active":"2023-09-30T22:14:02.072950+0000","last_became_peered":"2023-09-30T22:14:02.072950+0000","last_unstale":"2023-09-30T22:14:02.080942+0000","last_undegraded":"2023-09-30T22:14:02.080942+0000","last_fullsized":"2023-09-30T22:14:02.080942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057420+0000","last_change":"2023-09-30T22:14:01.701736+0000","last_active":"2023-09-30T22:14:02.057420+0000","last_peered":"2023-09-30T22:14:02.057420+0000","last_clean":"2023-09-30T22:14:02.057420+0000","last_became_active":"2023-09-30T22:14:01.701471+0000","last_became_peered":"2023-09-30T22:14:01.701471+0000","last_unstale":"2023-09-30T22:14:02.057420+0000","last_undegraded":"2023-09-30T22:14:02.057420+0000","last_fullsized":"2023-09-30T22:14:02.057420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082228+0000","last_change":"2023-09-30T22:14:02.072624+0000","last_active":"2023-09-30T22:14:02.082228+0000","last_peered":"2023-09-30T22:14:02.082228+0000","last_clean":"2023-09-30T22:14:02.082228+0000","last_became_active":"2023-09-30T22:14:02.072363+0000","last_became_peered":"2023-09-30T22:14:02.072363+0000","last_unstale":"2023-09-30T22:14:02.082228+0000","last_undegraded":"2023-09-30T22:14:02.082228+0000","last_fullsized":"2023-09-30T22:14:02.082228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.082837+0000","last_change":"2023-09-30T22:14:01.709398+0000","last_active":"2023-09-30T22:14:02.082837+0000","last_peered":"2023-09-30T22:14:02.082837+0000","last_clean":"2023-09-30T22:14:02.082837+0000","last_became_active":"2023-09-30T22:14:01.709110+0000","last_became_peered":"2023-09-30T22:14:01.709110+0000","last_unstale":"2023-09-30T22:14:02.082837+0000","last_undegraded":"2023-09-30T22:14:02.082837+0000","last_fullsized":"2023-09-30T22:14:02.082837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087990+0000","last_change":"2023-09-30T22:14:02.074887+0000","last_active":"2023-09-30T22:14:02.087990+0000","last_peered":"2023-09-30T22:14:02.087990+0000","last_clean":"2023-09-30T22:14:02.087990+0000","last_became_active":"2023-09-30T22:14:02.074631+0000","last_became_peered":"2023-09-30T22:14:02.074631+0000","last_unstale":"2023-09-30T22:14:02.087990+0000","last_undegraded":"2023-09-30T22:14:02.087990+0000","last_fullsized":"2023-09-30T22:14:02.087990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054479+0000","last_change":"2023-09-30T22:14:01.707879+0000","last_active":"2023-09-30T22:14:02.054479+0000","last_peered":"2023-09-30T22:14:02.054479+0000","last_clean":"2023-09-30T22:14:02.054479+0000","last_became_active":"2023-09-30T22:14:01.707594+0000","last_became_peered":"2023-09-30T22:14:01.707594+0000","last_unstale":"2023-09-30T22:14:02.054479+0000","last_undegraded":"2023-09-30T22:14:02.054479+0000","last_fullsized":"2023-09-30T22:14:02.054479+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.085707+0000","last_change":"2023-09-30T22:14:02.076079+0000","last_active":"2023-09-30T22:14:02.085707+0000","last_peered":"2023-09-30T22:14:02.085707+0000","last_clean":"2023-09-30T22:14:02.085707+0000","last_became_active":"2023-09-30T22:14:02.075408+0000","last_became_peered":"2023-09-30T22:14:02.075408+0000","last_unstale":"2023-09-30T22:14:02.085707+0000","last_undegraded":"2023-09-30T22:14:02.085707+0000","last_fullsized":"2023-09-30T22:14:02.085707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.065836+0000","last_change":"2023-09-30T22:14:01.711605+0000","last_active":"2023-09-30T22:14:02.065836+0000","last_peered":"2023-09-30T22:14:02.065836+0000","last_clean":"2023-09-30T22:14:02.065836+0000","last_became_active":"2023-09-30T22:14:01.711375+0000","last_became_peered":"2023-09-30T22:14:01.711375+0000","last_unstale":"2023-09-30T22:14:02.065836+0000","last_undegraded":"2023-09-30T22:14:02.065836+0000","last_fullsized":"2023-09-30T22:14:02.065836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090979+0000","last_change":"2023-09-30T22:14:02.088858+0000","last_active":"2023-09-30T22:14:02.090979+0000","last_peered":"2023-09-30T22:14:02.090979+0000","last_clean":"2023-09-30T22:14:02.090979+0000","last_became_active":"2023-09-30T22:14:02.088583+0000","last_became_peered":"2023-09-30T22:14:02.088583+0000","last_unstale":"2023-09-30T22:14:02.090979+0000","last_undegraded":"2023-09-30T22:14:02.090979+0000","last_fullsized":"2023-09-30T22:14:02.090979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053157+0000","last_change":"2023-09-30T22:14:01.699847+0000","last_active":"2023-09-30T22:14:02.053157+0000","last_peered":"2023-09-30T22:14:02.053157+0000","last_clean":"2023-09-30T22:14:02.053157+0000","last_became_active":"2023-09-30T22:14:01.699587+0000","last_became_peered":"2023-09-30T22:14:01.699587+0000","last_unstale":"2023-09-30T22:14:02.053157+0000","last_undegraded":"2023-09-30T22:14:02.053157+0000","last_fullsized":"2023-09-30T22:14:02.053157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.098596+0000","last_change":"2023-09-30T22:14:02.094005+0000","last_active":"2023-09-30T22:14:02.098596+0000","last_peered":"2023-09-30T22:14:02.098596+0000","last_clean":"2023-09-30T22:14:02.098596+0000","last_became_active":"2023-09-30T22:14:02.093632+0000","last_became_peered":"2023-09-30T22:14:02.093632+0000","last_unstale":"2023-09-30T22:14:02.098596+0000","last_undegraded":"2023-09-30T22:14:02.098596+0000","last_fullsized":"2023-09-30T22:14:02.098596+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.052854+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:14:02.052854+0000","last_peered":"2023-09-30T22:14:02.052854+0000","last_clean":"2023-09-30T22:14:02.052854+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:14:02.052854+0000","last_undegraded":"2023-09-30T22:14:02.052854+0000","last_fullsized":"2023-09-30T22:14:02.052854+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056318+0000","last_change":"2023-09-30T22:14:01.702822+0000","last_active":"2023-09-30T22:14:02.056318+0000","last_peered":"2023-09-30T22:14:02.056318+0000","last_clean":"2023-09-30T22:14:02.056318+0000","last_became_active":"2023-09-30T22:14:01.702530+0000","last_became_peered":"2023-09-30T22:14:01.702530+0000","last_unstale":"2023-09-30T22:14:02.056318+0000","last_undegraded":"2023-09-30T22:14:02.056318+0000","last_fullsized":"2023-09-30T22:14:02.056318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.093176+0000","last_change":"2023-09-30T22:14:02.084698+0000","last_active":"2023-09-30T22:14:02.093176+0000","last_peered":"2023-09-30T22:14:02.093176+0000","last_clean":"2023-09-30T22:14:02.093176+0000","last_became_active":"2023-09-30T22:14:02.084413+0000","last_became_peered":"2023-09-30T22:14:02.084413+0000","last_unstale":"2023-09-30T22:14:02.093176+0000","last_undegraded":"2023-09-30T22:14:02.093176+0000","last_fullsized":"2023-09-30T22:14:02.093176+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054630+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:14:02.054630+0000","last_peered":"2023-09-30T22:14:02.054630+0000","last_clean":"2023-09-30T22:14:02.054630+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:14:02.054630+0000","last_undegraded":"2023-09-30T22:14:02.054630+0000","last_fullsized":"2023-09-30T22:14:02.054630+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.057152+0000","last_change":"2023-09-30T22:14:01.704198+0000","last_active":"2023-09-30T22:14:02.057152+0000","last_peered":"2023-09-30T22:14:02.057152+0000","last_clean":"2023-09-30T22:14:02.057152+0000","last_became_active":"2023-09-30T22:14:01.701691+0000","last_became_peered":"2023-09-30T22:14:01.701691+0000","last_unstale":"2023-09-30T22:14:02.057152+0000","last_undegraded":"2023-09-30T22:14:02.057152+0000","last_fullsized":"2023-09-30T22:14:02.057152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079574+0000","last_change":"2023-09-30T22:14:02.076717+0000","last_active":"2023-09-30T22:14:02.079574+0000","last_peered":"2023-09-30T22:14:02.079574+0000","last_clean":"2023-09-30T22:14:02.079574+0000","last_became_active":"2023-09-30T22:14:02.076119+0000","last_became_peered":"2023-09-30T22:14:02.076119+0000","last_unstale":"2023-09-30T22:14:02.079574+0000","last_undegraded":"2023-09-30T22:14:02.079574+0000","last_fullsized":"2023-09-30T22:14:02.079574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.056513+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:14:02.056513+0000","last_peered":"2023-09-30T22:14:02.056513+0000","last_clean":"2023-09-30T22:14:02.056513+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:14:02.056513+0000","last_undegraded":"2023-09-30T22:14:02.056513+0000","last_fullsized":"2023-09-30T22:14:02.056513+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062825+0000","last_change":"2023-09-30T22:14:01.701431+0000","last_active":"2023-09-30T22:14:02.062825+0000","last_peered":"2023-09-30T22:14:02.062825+0000","last_clean":"2023-09-30T22:14:02.062825+0000","last_became_active":"2023-09-30T22:14:01.698771+0000","last_became_peered":"2023-09-30T22:14:01.698771+0000","last_unstale":"2023-09-30T22:14:02.062825+0000","last_undegraded":"2023-09-30T22:14:02.062825+0000","last_fullsized":"2023-09-30T22:14:02.062825+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.094124+0000","last_change":"2023-09-30T22:14:02.085435+0000","last_active":"2023-09-30T22:14:02.094124+0000","last_peered":"2023-09-30T22:14:02.094124+0000","last_clean":"2023-09-30T22:14:02.094124+0000","last_became_active":"2023-09-30T22:14:02.085124+0000","last_became_peered":"2023-09-30T22:14:02.085124+0000","last_unstale":"2023-09-30T22:14:02.094124+0000","last_undegraded":"2023-09-30T22:14:02.094124+0000","last_fullsized":"2023-09-30T22:14:02.094124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062504+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:14:02.062504+0000","last_peered":"2023-09-30T22:14:02.062504+0000","last_clean":"2023-09-30T22:14:02.062504+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:14:02.062504+0000","last_undegraded":"2023-09-30T22:14:02.062504+0000","last_fullsized":"2023-09-30T22:14:02.062504+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.062714+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:14:02.062714+0000","last_peered":"2023-09-30T22:14:02.062714+0000","last_clean":"2023-09-30T22:14:02.062714+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:14:02.062714+0000","last_undegraded":"2023-09-30T22:14:02.062714+0000","last_fullsized":"2023-09-30T22:14:02.062714+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054372+0000","last_change":"2023-09-30T22:14:01.712014+0000","last_active":"2023-09-30T22:14:02.054372+0000","last_peered":"2023-09-30T22:14:02.054372+0000","last_clean":"2023-09-30T22:14:02.054372+0000","last_became_active":"2023-09-30T22:14:01.711770+0000","last_became_peered":"2023-09-30T22:14:01.711770+0000","last_unstale":"2023-09-30T22:14:02.054372+0000","last_undegraded":"2023-09-30T22:14:02.054372+0000","last_fullsized":"2023-09-30T22:14:02.054372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087711+0000","last_change":"2023-09-30T22:14:02.080800+0000","last_active":"2023-09-30T22:14:02.087711+0000","last_peered":"2023-09-30T22:14:02.087711+0000","last_clean":"2023-09-30T22:14:02.087711+0000","last_became_active":"2023-09-30T22:14:02.080440+0000","last_became_peered":"2023-09-30T22:14:02.080440+0000","last_unstale":"2023-09-30T22:14:02.087711+0000","last_undegraded":"2023-09-30T22:14:02.087711+0000","last_fullsized":"2023-09-30T22:14:02.087711+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054814+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:14:02.054814+0000","last_peered":"2023-09-30T22:14:02.054814+0000","last_clean":"2023-09-30T22:14:02.054814+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:14:02.054814+0000","last_undegraded":"2023-09-30T22:14:02.054814+0000","last_fullsized":"2023-09-30T22:14:02.054814+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.055456+0000","last_change":"2023-09-30T22:14:01.694407+0000","last_active":"2023-09-30T22:14:02.055456+0000","last_peered":"2023-09-30T22:14:02.055456+0000","last_clean":"2023-09-30T22:14:02.055456+0000","last_became_active":"2023-09-30T22:14:01.694022+0000","last_became_peered":"2023-09-30T22:14:01.694022+0000","last_unstale":"2023-09-30T22:14:02.055456+0000","last_undegraded":"2023-09-30T22:14:02.055456+0000","last_fullsized":"2023-09-30T22:14:02.055456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091628+0000","last_change":"2023-09-30T22:14:02.079540+0000","last_active":"2023-09-30T22:14:02.091628+0000","last_peered":"2023-09-30T22:14:02.091628+0000","last_clean":"2023-09-30T22:14:02.091628+0000","last_became_active":"2023-09-30T22:14:02.079016+0000","last_became_peered":"2023-09-30T22:14:02.079016+0000","last_unstale":"2023-09-30T22:14:02.091628+0000","last_undegraded":"2023-09-30T22:14:02.091628+0000","last_fullsized":"2023-09-30T22:14:02.091628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.059163+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:14:02.059163+0000","last_peered":"2023-09-30T22:14:02.059163+0000","last_clean":"2023-09-30T22:14:02.059163+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:14:02.059163+0000","last_undegraded":"2023-09-30T22:14:02.059163+0000","last_fullsized":"2023-09-30T22:14:02.059163+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.053400+0000","last_change":"2023-09-30T22:14:01.695717+0000","last_active":"2023-09-30T22:14:02.053400+0000","last_peered":"2023-09-30T22:14:02.053400+0000","last_clean":"2023-09-30T22:14:02.053400+0000","last_became_active":"2023-09-30T22:14:01.694990+0000","last_became_peered":"2023-09-30T22:14:01.694990+0000","last_unstale":"2023-09-30T22:14:02.053400+0000","last_undegraded":"2023-09-30T22:14:02.053400+0000","last_fullsized":"2023-09-30T22:14:02.053400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080394+0000","last_change":"2023-09-30T22:14:02.068852+0000","last_active":"2023-09-30T22:14:02.080394+0000","last_peered":"2023-09-30T22:14:02.080394+0000","last_clean":"2023-09-30T22:14:02.080394+0000","last_became_active":"2023-09-30T22:14:02.068531+0000","last_became_peered":"2023-09-30T22:14:02.068531+0000","last_unstale":"2023-09-30T22:14:02.080394+0000","last_undegraded":"2023-09-30T22:14:02.080394+0000","last_fullsized":"2023-09-30T22:14:02.080394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.084595+0000","last_change":"2023-09-30T22:14:01.700959+0000","last_active":"2023-09-30T22:14:02.084595+0000","last_peered":"2023-09-30T22:14:02.084595+0000","last_clean":"2023-09-30T22:14:02.084595+0000","last_became_active":"2023-09-30T22:14:01.700568+0000","last_became_peered":"2023-09-30T22:14:01.700568+0000","last_unstale":"2023-09-30T22:14:02.084595+0000","last_undegraded":"2023-09-30T22:14:02.084595+0000","last_fullsized":"2023-09-30T22:14:02.084595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.048709+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:14:02.048709+0000","last_peered":"2023-09-30T22:14:02.048709+0000","last_clean":"2023-09-30T22:14:02.048709+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:14:02.048709+0000","last_undegraded":"2023-09-30T22:14:02.048709+0000","last_fullsized":"2023-09-30T22:14:02.048709+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087805+0000","last_change":"2023-09-30T22:14:02.081943+0000","last_active":"2023-09-30T22:14:02.087805+0000","last_peered":"2023-09-30T22:14:02.087805+0000","last_clean":"2023-09-30T22:14:02.087805+0000","last_became_active":"2023-09-30T22:14:02.080899+0000","last_became_peered":"2023-09-30T22:14:02.080899+0000","last_unstale":"2023-09-30T22:14:02.087805+0000","last_undegraded":"2023-09-30T22:14:02.087805+0000","last_fullsized":"2023-09-30T22:14:02.087805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.054595+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:14:02.054595+0000","last_peered":"2023-09-30T22:14:02.054595+0000","last_clean":"2023-09-30T22:14:02.054595+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:14:02.054595+0000","last_undegraded":"2023-09-30T22:14:02.054595+0000","last_fullsized":"2023-09-30T22:14:02.054595+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T22:14:03.047055+0000","last_change":"2023-09-30T22:14:01.706704+0000","last_active":"2023-09-30T22:14:03.047055+0000","last_peered":"2023-09-30T22:14:03.047055+0000","last_clean":"2023-09-30T22:14:03.047055+0000","last_became_active":"2023-09-30T22:14:01.706387+0000","last_became_peered":"2023-09-30T22:14:01.706387+0000","last_unstale":"2023-09-30T22:14:03.047055+0000","last_undegraded":"2023-09-30T22:14:03.047055+0000","last_fullsized":"2023-09-30T22:14:03.047055+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090754+0000","last_change":"2023-09-30T22:14:02.088343+0000","last_active":"2023-09-30T22:14:02.090754+0000","last_peered":"2023-09-30T22:14:02.090754+0000","last_clean":"2023-09-30T22:14:02.090754+0000","last_became_active":"2023-09-30T22:14:02.088142+0000","last_became_peered":"2023-09-30T22:14:02.088142+0000","last_unstale":"2023-09-30T22:14:02.090754+0000","last_undegraded":"2023-09-30T22:14:02.090754+0000","last_fullsized":"2023-09-30T22:14:02.090754+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083333+0000","last_change":"2023-09-30T22:14:01.702235+0000","last_active":"2023-09-30T22:14:02.083333+0000","last_peered":"2023-09-30T22:14:02.083333+0000","last_clean":"2023-09-30T22:14:02.083333+0000","last_became_active":"2023-09-30T22:14:01.701972+0000","last_became_peered":"2023-09-30T22:14:01.701972+0000","last_unstale":"2023-09-30T22:14:02.083333+0000","last_undegraded":"2023-09-30T22:14:02.083333+0000","last_fullsized":"2023-09-30T22:14:02.083333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080923+0000","last_change":"2023-09-30T22:14:02.078694+0000","last_active":"2023-09-30T22:14:02.080923+0000","last_peered":"2023-09-30T22:14:02.080923+0000","last_clean":"2023-09-30T22:14:02.080923+0000","last_became_active":"2023-09-30T22:14:02.078435+0000","last_became_peered":"2023-09-30T22:14:02.078435+0000","last_unstale":"2023-09-30T22:14:02.080923+0000","last_undegraded":"2023-09-30T22:14:02.080923+0000","last_fullsized":"2023-09-30T22:14:02.080923+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090913+0000","last_change":"2023-09-30T22:14:01.709428+0000","last_active":"2023-09-30T22:14:02.090913+0000","last_peered":"2023-09-30T22:14:02.090913+0000","last_clean":"2023-09-30T22:14:02.090913+0000","last_became_active":"2023-09-30T22:14:01.709089+0000","last_became_peered":"2023-09-30T22:14:01.709089+0000","last_unstale":"2023-09-30T22:14:02.090913+0000","last_undegraded":"2023-09-30T22:14:02.090913+0000","last_fullsized":"2023-09-30T22:14:02.090913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.090409+0000","last_change":"2023-09-30T22:14:02.087499+0000","last_active":"2023-09-30T22:14:02.090409+0000","last_peered":"2023-09-30T22:14:02.090409+0000","last_clean":"2023-09-30T22:14:02.090409+0000","last_became_active":"2023-09-30T22:14:02.086941+0000","last_became_peered":"2023-09-30T22:14:02.086941+0000","last_unstale":"2023-09-30T22:14:02.090409+0000","last_undegraded":"2023-09-30T22:14:02.090409+0000","last_fullsized":"2023-09-30T22:14:02.090409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083525+0000","last_change":"2023-09-30T22:14:01.709677+0000","last_active":"2023-09-30T22:14:02.083525+0000","last_peered":"2023-09-30T22:14:02.083525+0000","last_clean":"2023-09-30T22:14:02.083525+0000","last_became_active":"2023-09-30T22:14:01.709115+0000","last_became_peered":"2023-09-30T22:14:01.709115+0000","last_unstale":"2023-09-30T22:14:02.083525+0000","last_undegraded":"2023-09-30T22:14:02.083525+0000","last_fullsized":"2023-09-30T22:14:02.083525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080964+0000","last_change":"2023-09-30T22:14:02.073683+0000","last_active":"2023-09-30T22:14:02.080964+0000","last_peered":"2023-09-30T22:14:02.080964+0000","last_clean":"2023-09-30T22:14:02.080964+0000","last_became_active":"2023-09-30T22:14:02.073269+0000","last_became_peered":"2023-09-30T22:14:02.073269+0000","last_unstale":"2023-09-30T22:14:02.080964+0000","last_undegraded":"2023-09-30T22:14:02.080964+0000","last_fullsized":"2023-09-30T22:14:02.080964+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.083125+0000","last_change":"2023-09-30T22:14:01.711766+0000","last_active":"2023-09-30T22:14:02.083125+0000","last_peered":"2023-09-30T22:14:02.083125+0000","last_clean":"2023-09-30T22:14:02.083125+0000","last_became_active":"2023-09-30T22:14:01.711524+0000","last_became_peered":"2023-09-30T22:14:01.711524+0000","last_unstale":"2023-09-30T22:14:02.083125+0000","last_undegraded":"2023-09-30T22:14:02.083125+0000","last_fullsized":"2023-09-30T22:14:02.083125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088038+0000","last_change":"2023-09-30T22:14:02.082106+0000","last_active":"2023-09-30T22:14:02.088038+0000","last_peered":"2023-09-30T22:14:02.088038+0000","last_clean":"2023-09-30T22:14:02.088038+0000","last_became_active":"2023-09-30T22:14:02.081789+0000","last_became_peered":"2023-09-30T22:14:02.081789+0000","last_unstale":"2023-09-30T22:14:02.088038+0000","last_undegraded":"2023-09-30T22:14:02.088038+0000","last_fullsized":"2023-09-30T22:14:02.088038+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.091206+0000","last_change":"2023-09-30T22:14:01.702868+0000","last_active":"2023-09-30T22:14:02.091206+0000","last_peered":"2023-09-30T22:14:02.091206+0000","last_clean":"2023-09-30T22:14:02.091206+0000","last_became_active":"2023-09-30T22:14:01.702431+0000","last_became_peered":"2023-09-30T22:14:01.702431+0000","last_unstale":"2023-09-30T22:14:02.091206+0000","last_undegraded":"2023-09-30T22:14:02.091206+0000","last_fullsized":"2023-09-30T22:14:02.091206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.084635+0000","last_change":"2023-09-30T22:14:02.081561+0000","last_active":"2023-09-30T22:14:02.084635+0000","last_peered":"2023-09-30T22:14:02.084635+0000","last_clean":"2023-09-30T22:14:02.084635+0000","last_became_active":"2023-09-30T22:14:02.081199+0000","last_became_peered":"2023-09-30T22:14:02.081199+0000","last_unstale":"2023-09-30T22:14:02.084635+0000","last_undegraded":"2023-09-30T22:14:02.084635+0000","last_fullsized":"2023-09-30T22:14:02.084635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.086077+0000","last_change":"2023-09-30T22:14:02.079397+0000","last_active":"2023-09-30T22:14:02.086077+0000","last_peered":"2023-09-30T22:14:02.086077+0000","last_clean":"2023-09-30T22:14:02.086077+0000","last_became_active":"2023-09-30T22:14:02.079036+0000","last_became_peered":"2023-09-30T22:14:02.079036+0000","last_unstale":"2023-09-30T22:14:02.086077+0000","last_undegraded":"2023-09-30T22:14:02.086077+0000","last_fullsized":"2023-09-30T22:14:02.086077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.088018+0000","last_change":"2023-09-30T22:14:02.081974+0000","last_active":"2023-09-30T22:14:02.088018+0000","last_peered":"2023-09-30T22:14:02.088018+0000","last_clean":"2023-09-30T22:14:02.088018+0000","last_became_active":"2023-09-30T22:14:02.081491+0000","last_became_peered":"2023-09-30T22:14:02.081491+0000","last_unstale":"2023-09-30T22:14:02.088018+0000","last_undegraded":"2023-09-30T22:14:02.088018+0000","last_fullsized":"2023-09-30T22:14:02.088018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.080473+0000","last_change":"2023-09-30T22:14:02.067554+0000","last_active":"2023-09-30T22:14:02.080473+0000","last_peered":"2023-09-30T22:14:02.080473+0000","last_clean":"2023-09-30T22:14:02.080473+0000","last_became_active":"2023-09-30T22:14:02.067283+0000","last_became_peered":"2023-09-30T22:14:02.067283+0000","last_unstale":"2023-09-30T22:14:02.080473+0000","last_undegraded":"2023-09-30T22:14:02.080473+0000","last_fullsized":"2023-09-30T22:14:02.080473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.087735+0000","last_change":"2023-09-30T22:14:02.082875+0000","last_active":"2023-09-30T22:14:02.087735+0000","last_peered":"2023-09-30T22:14:02.087735+0000","last_clean":"2023-09-30T22:14:02.087735+0000","last_became_active":"2023-09-30T22:14:02.082617+0000","last_became_peered":"2023-09-30T22:14:02.082617+0000","last_unstale":"2023-09-30T22:14:02.087735+0000","last_undegraded":"2023-09-30T22:14:02.087735+0000","last_fullsized":"2023-09-30T22:14:02.087735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:14:02.079595+0000","last_change":"2023-09-30T22:14:02.076951+0000","last_active":"2023-09-30T22:14:02.079595+0000","last_peered":"2023-09-30T22:14:02.079595+0000","last_clean":"2023-09-30T22:14:02.079595+0000","last_became_active":"2023-09-30T22:14:02.076448+0000","last_became_peered":"2023-09-30T22:14:02.076448+0000","last_unstale":"2023-09-30T22:14:02.079595+0000","last_undegraded":"2023-09-30T22:14:02.079595+0000","last_fullsized":"2023-09-30T22:14:02.079595+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.214}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:14:14.702 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T22:14:14.702 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T22:14:14.702 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-30T22:14:14.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T22:14:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.850+0000 7f94c7b21700 1 Processor -- start 2023-09-30T22:14:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.850+0000 7f94c7b21700 1 -- start start 2023-09-30T22:14:14.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c7b21700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012bcb0 0x7f94c012e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c7b21700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c012e590 0x7f94c0130980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c7b21700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c0092fe0 con 0x7f94c00969b0 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c7b21700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0094560 con 0x7f94c012e590 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c7b21700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0093260 con 0x7f94c012bcb0 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c50bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012bcb0 0x7f94c012e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c60be700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c012e590 0x7f94c0130980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c58bd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f94c00969b0 0x7f94c0096d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54288/0 (socket says 172.21.15.89:54288) 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c58bd700 1 -- 172.21.15.89:0/3428605629 learned_addr learned my addr 172.21.15.89:0/3428605629 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c48bb700 1 -- 172.21.15.89:0/3428605629 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636404022 0 0) 0x7f94c0092fe0 con 0x7f94c00969b0 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c48bb700 1 -- 172.21.15.89:0/3428605629 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94ac0033f0 con 0x7f94c00969b0 2023-09-30T22:14:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c50bc700 1 -- 172.21.15.89:0/3428605629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c012e590 msgr2=0x7f94c0130980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c50bc700 1 --2- 172.21.15.89:0/3428605629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c012e590 0x7f94c0130980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c50bc700 1 -- 172.21.15.89:0/3428605629 >> v1:172.21.15.89:6789/0 conn(0x7f94c00969b0 legacy=0x7f94c0096d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c50bc700 1 -- 172.21.15.89:0/3428605629 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0092b30 con 0x7f94c012bcb0 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.851+0000 7f94c60be700 1 --2- 172.21.15.89:0/3428605629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c012e590 0x7f94c0130980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c50bc700 1 --2- 172.21.15.89:0/3428605629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012bcb0 0x7f94c012e0a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f94bc009790 tx=0x7f94bc009060).ready entity=mon.2 client_cookie=38befb30a1bf6cac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c48bb700 1 -- 172.21.15.89:0/3428605629 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00f510 con 0x7f94c012bcb0 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c48bb700 1 -- 172.21.15.89:0/3428605629 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc00f670 con 0x7f94c012bcb0 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c48bb700 1 -- 172.21.15.89:0/3428605629 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00f9f0 con 0x7f94c012bcb0 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c7b21700 1 -- 172.21.15.89:0/3428605629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012bcb0 msgr2=0x7f94c012e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c7b21700 1 --2- 172.21.15.89:0/3428605629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012bcb0 0x7f94c012e0a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f94bc009790 tx=0x7f94bc009060).stop 2023-09-30T22:14:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c7b21700 1 -- 172.21.15.89:0/3428605629 shutdown_connections 2023-09-30T22:14:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c7b21700 1 --2- 172.21.15.89:0/3428605629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c012e590 0x7f94c0130980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c7b21700 1 --2- 172.21.15.89:0/3428605629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012bcb0 0x7f94c012e0a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c7b21700 1 -- 172.21.15.89:0/3428605629 >> 172.21.15.89:0/3428605629 conn(0x7f94c009a220 msgr2=0x7f94c009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c7b21700 1 -- 172.21.15.89:0/3428605629 shutdown_connections 2023-09-30T22:14:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.852+0000 7f94c7b21700 1 -- 172.21.15.89:0/3428605629 wait complete. 2023-09-30T22:14:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.853+0000 7f94c7b21700 1 Processor -- start 2023-09-30T22:14:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.853+0000 7f94c7b21700 1 -- start start 2023-09-30T22:14:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c7b21700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c00969b0 0x7f94c01175a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c7b21700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c012bcb0 0x7f94c0113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c7b21700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012e590 0x7f94c0113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c7b21700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0129b60 con 0x7f94c012bcb0 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c7b21700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0129750 con 0x7f94c00969b0 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c7b21700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0092d60 con 0x7f94c012e590 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c58bd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c00969b0 0x7f94c01175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c50bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c012bcb0 0x7f94c0113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c50bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c012bcb0 0x7f94c0113570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36820/0 (socket says 172.21.15.89:36820) 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c58bd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c00969b0 0x7f94c01175a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52250/0 (socket says 172.21.15.89:52250) 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c50bc700 1 -- 172.21.15.89:0/862805351 learned_addr learned my addr 172.21.15.89:0/862805351 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c60be700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012e590 0x7f94c0113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c50bc700 1 -- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012e590 msgr2=0x7f94c0113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c50bc700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012e590 0x7f94c0113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c50bc700 1 -- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c00969b0 msgr2=0x7f94c01175a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c50bc700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c00969b0 0x7f94c01175a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c50bc700 1 -- 172.21.15.89:0/862805351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009a20 con 0x7f94c012bcb0 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.854+0000 7f94c58bd700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c00969b0 0x7f94c01175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.855+0000 7f94c50bc700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c012bcb0 0x7f94c0113570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f94bc009d80 tx=0x7f94bc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.855+0000 7f94b5ffb700 1 -- 172.21.15.89:0/862805351 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00f7e0 con 0x7f94c012bcb0 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.855+0000 7f94b5ffb700 1 -- 172.21.15.89:0/862805351 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc00f940 con 0x7f94c012bcb0 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.855+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc0092a0 con 0x7f94c012bcb0 2023-09-30T22:14:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.855+0000 7f94b5ffb700 1 -- 172.21.15.89:0/862805351 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc01b660 con 0x7f94c012bcb0 2023-09-30T22:14:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.855+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c0114970 con 0x7f94c012bcb0 2023-09-30T22:14:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.856+0000 7f94b5ffb700 1 -- 172.21.15.89:0/862805351 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f94bc01b9d0 con 0x7f94c012bcb0 2023-09-30T22:14:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.856+0000 7f94b5ffb700 1 -- 172.21.15.89:0/862805351 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f94bc03cb60 con 0x7f94c012bcb0 2023-09-30T22:14:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.856+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c012e1d0 con 0x7f94c012bcb0 2023-09-30T22:14:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:14.860+0000 7f94b5ffb700 1 -- 172.21.15.89:0/862805351 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f94bc019380 con 0x7f94c012bcb0 2023-09-30T22:14:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.051+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f94c0092320 con 0x7f94c012bcb0 2023-09-30T22:14:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.051+0000 7f94b5ffb700 1 -- 172.21.15.89:0/862805351 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f94bc010b90 con 0x7f94c012bcb0 2023-09-30T22:14:15.051 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 >> v1:172.21.15.89:6816/118161 conn(0x7f94a402bb60 legacy=0x7f94a402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c012bcb0 msgr2=0x7f94c0113570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c012bcb0 0x7f94c0113570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f94bc009d80 tx=0x7f94bc017000).stop 2023-09-30T22:14:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 shutdown_connections 2023-09-30T22:14:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c012e590 0x7f94c0113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c00969b0 0x7f94c01175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 --2- 172.21.15.89:0/862805351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c012bcb0 0x7f94c0113570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 >> 172.21.15.89:0/862805351 conn(0x7f94c009a220 msgr2=0x7f94c0095a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 shutdown_connections 2023-09-30T22:14:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.053+0000 7f94c7b21700 1 -- 172.21.15.89:0/862805351 wait complete. 2023-09-30T22:14:15.064 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-30T22:14:15.064 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-30T22:14:15.065 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.181+0000 7f345b1a4700 1 Processor -- start 2023-09-30T22:14:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f345b1a4700 1 -- start start 2023-09-30T22:14:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f345b1a4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3454096380 0x7f3454094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f345b1a4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f345b1a4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f3454127af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f345b1a4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3454096760 con 0x7f3454096380 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f345b1a4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3454091ff0 con 0x7f3454091b00 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f345b1a4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3454092c80 con 0x7f3454094c20 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3459741700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f3454127af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3458f40700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3454096380 0x7f3454094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3453fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3459741700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f3454127af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34184/0 (socket says 172.21.15.89:34184) 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3458f40700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3454096380 0x7f3454094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36834/0 (socket says 172.21.15.89:36834) 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3459741700 1 -- 172.21.15.89:0/4005257158 learned_addr learned my addr 172.21.15.89:0/4005257158 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3453fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454127600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52262/0 (socket says 172.21.15.89:52262) 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3459741700 1 -- 172.21.15.89:0/4005257158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 msgr2=0x7f3454127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3459741700 1 --2- 172.21.15.89:0/4005257158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454127600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3459741700 1 -- 172.21.15.89:0/4005257158 >> v2:172.21.15.89:3300/0 conn(0x7f3454096380 msgr2=0x7f3454094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3459741700 1 --2- 172.21.15.89:0/4005257158 >> v2:172.21.15.89:3300/0 conn(0x7f3454096380 0x7f3454094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3459741700 1 -- 172.21.15.89:0/4005257158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454093900 con 0x7f3454094c20 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3453fff700 1 --2- 172.21.15.89:0/4005257158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.182+0000 7f3458f40700 1 --2- 172.21.15.89:0/4005257158 >> v2:172.21.15.89:3300/0 conn(0x7f3454096380 0x7f3454094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f3459741700 1 --2- 172.21.15.89:0/4005257158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f3454127af0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f344c009f20 tx=0x7f344c009840).ready entity=mon.2 client_cookie=220adf60677357f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f34537fe700 1 -- 172.21.15.89:0/4005257158 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c00e070 con 0x7f3454094c20 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f34537fe700 1 -- 172.21.15.89:0/4005257158 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c00b970 con 0x7f3454094c20 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f34537fe700 1 -- 172.21.15.89:0/4005257158 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c00bcf0 con 0x7f3454094c20 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 -- 172.21.15.89:0/4005257158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 msgr2=0x7f3454127af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 --2- 172.21.15.89:0/4005257158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f3454127af0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f344c009f20 tx=0x7f344c009840).stop 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 -- 172.21.15.89:0/4005257158 shutdown_connections 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 --2- 172.21.15.89:0/4005257158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f3454127af0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 --2- 172.21.15.89:0/4005257158 >> v2:172.21.15.89:3300/0 conn(0x7f3454096380 0x7f3454094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 --2- 172.21.15.89:0/4005257158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 -- 172.21.15.89:0/4005257158 >> 172.21.15.89:0/4005257158 conn(0x7f3454097470 msgr2=0x7f3454097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 -- 172.21.15.89:0/4005257158 shutdown_connections 2023-09-30T22:14:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.183+0000 7f345b1a4700 1 -- 172.21.15.89:0/4005257158 wait complete. 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.184+0000 7f345b1a4700 1 Processor -- start 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f345b1a4700 1 -- start start 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f345b1a4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454137df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f345b1a4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f34541382e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f345b1a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3454096380 0x7f34541387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f345b1a4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345406a490 con 0x7f3454096380 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f345b1a4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3454069e80 con 0x7f3454091b00 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f345b1a4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f345406a140 con 0x7f3454094c20 2023-09-30T22:14:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f3458f40700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f3458f40700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454137df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52270/0 (socket says 172.21.15.89:52270) 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f3458f40700 1 -- 172.21.15.89:0/1717491547 learned_addr learned my addr 172.21.15.89:0/1717491547 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f3459741700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3454096380 0x7f34541387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.185+0000 7f3453fff700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f34541382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3458f40700 1 -- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 msgr2=0x7f34541382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3458f40700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f34541382e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3458f40700 1 -- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3454096380 msgr2=0x7f34541387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3458f40700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3454096380 0x7f34541387d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3458f40700 1 -- 172.21.15.89:0/1717491547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400092a0 con 0x7f3454091b00 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3458f40700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454137df0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f344800ea20 tx=0x7f3448009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3450ff9700 1 -- 172.21.15.89:0/1717491547 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800ee60 con 0x7f3454091b00 2023-09-30T22:14:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3450ff9700 1 -- 172.21.15.89:0/1717491547 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344800c850 con 0x7f3454091b00 2023-09-30T22:14:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34480092a0 con 0x7f3454091b00 2023-09-30T22:14:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f3450ff9700 1 -- 172.21.15.89:0/1717491547 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800cbd0 con 0x7f3454091b00 2023-09-30T22:14:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.186+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c009ae0 con 0x7f3454091b00 2023-09-30T22:14:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.187+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345405a260 con 0x7f3454091b00 2023-09-30T22:14:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.187+0000 7f3450ff9700 1 -- 172.21.15.89:0/1717491547 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f344801b400 con 0x7f3454091b00 2023-09-30T22:14:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.187+0000 7f3450ff9700 1 -- 172.21.15.89:0/1717491547 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f344803b530 con 0x7f3454091b00 2023-09-30T22:14:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.191+0000 7f3450ff9700 1 -- 172.21.15.89:0/1717491547 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3448019900 con 0x7f3454091b00 2023-09-30T22:14:15.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.352+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34540942e0 con 0x7f3454091b00 2023-09-30T22:14:15.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.353+0000 7f3450ff9700 1 -- 172.21.15.89:0/1717491547 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f3448019310 con 0x7f3454091b00 2023-09-30T22:14:15.352 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:15.352 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:03.041750+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:15.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 >> v1:172.21.15.89:6816/118161 conn(0x7f343802bb90 legacy=0x7f343802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 msgr2=0x7f3454137df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454137df0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f344800ea20 tx=0x7f3448009060).stop 2023-09-30T22:14:15.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 shutdown_connections 2023-09-30T22:14:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3454096380 0x7f34541387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3454091b00 0x7f3454137df0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 --2- 172.21.15.89:0/1717491547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3454094c20 0x7f34541382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 >> 172.21.15.89:0/1717491547 conn(0x7f3454097470 msgr2=0x7f3454098de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 shutdown_connections 2023-09-30T22:14:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.354+0000 7f345b1a4700 1 -- 172.21.15.89:0/1717491547 wait complete. 2023-09-30T22:14:15.366 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:14:00.533280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:14:01.011104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:14:15.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:14:15.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.484+0000 7fa5ead9f700 1 Processor -- start 2023-09-30T22:14:15.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5ead9f700 1 -- start start 2023-09-30T22:14:15.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5ead9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e409a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5ead9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4127540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5ead9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5e412b730 0x7fa5e4127a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5ead9f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e4066560 con 0x7fa5e412b730 2023-09-30T22:14:15.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5ead9f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40656c0 con 0x7fa5e412b2b0 2023-09-30T22:14:15.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5ead9f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4065980 con 0x7fa5e4066180 2023-09-30T22:14:15.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5e9d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e409a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5e9d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e409a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34198/0 (socket says 172.21.15.89:34198) 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5e9d9d700 1 -- 172.21.15.89:0/3265554538 learned_addr learned my addr 172.21.15.89:0/3265554538 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5ea59e700 1 --2- 172.21.15.89:0/3265554538 >> v2:172.21.15.89:3300/0 conn(0x7fa5e412b730 0x7fa5e4127a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.485+0000 7fa5e959c700 1 --2- 172.21.15.89:0/3265554538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4127540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e9d9d700 1 -- 172.21.15.89:0/3265554538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 msgr2=0x7fa5e4127540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e9d9d700 1 --2- 172.21.15.89:0/3265554538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4127540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e9d9d700 1 -- 172.21.15.89:0/3265554538 >> v2:172.21.15.89:3300/0 conn(0x7fa5e412b730 msgr2=0x7fa5e4127a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e9d9d700 1 --2- 172.21.15.89:0/3265554538 >> v2:172.21.15.89:3300/0 conn(0x7fa5e412b730 0x7fa5e4127a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e9d9d700 1 -- 172.21.15.89:0/3265554538 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e412af40 con 0x7fa5e4066180 2023-09-30T22:14:15.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ea59e700 1 --2- 172.21.15.89:0/3265554538 >> v2:172.21.15.89:3300/0 conn(0x7fa5e412b730 0x7fa5e4127a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e9d9d700 1 --2- 172.21.15.89:0/3265554538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e409a730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa5d8006fd0 tx=0x7fa5d80090f0).ready entity=mon.2 client_cookie=babb20cd3be370a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e8d9b700 1 -- 172.21.15.89:0/3265554538 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800f510 con 0x7fa5e4066180 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e8d9b700 1 -- 172.21.15.89:0/3265554538 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d800f670 con 0x7fa5e4066180 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5e8d9b700 1 -- 172.21.15.89:0/3265554538 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800fa10 con 0x7fa5e4066180 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 -- 172.21.15.89:0/3265554538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 msgr2=0x7fa5e409a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 --2- 172.21.15.89:0/3265554538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e409a730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa5d8006fd0 tx=0x7fa5d80090f0).stop 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 -- 172.21.15.89:0/3265554538 shutdown_connections 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 --2- 172.21.15.89:0/3265554538 >> v2:172.21.15.89:3300/0 conn(0x7fa5e412b730 0x7fa5e4127a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 --2- 172.21.15.89:0/3265554538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e409a730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 --2- 172.21.15.89:0/3265554538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4127540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 -- 172.21.15.89:0/3265554538 >> 172.21.15.89:0/3265554538 conn(0x7fa5e409b8f0 msgr2=0x7fa5e409bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 -- 172.21.15.89:0/3265554538 shutdown_connections 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.486+0000 7fa5ead9f700 1 -- 172.21.15.89:0/3265554538 wait complete. 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.487+0000 7fa5ead9f700 1 Processor -- start 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.487+0000 7fa5ead9f700 1 -- start start 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ead9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e4117930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ead9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ead9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e412b730 0x7fa5e4113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ead9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4115df0 con 0x7fa5e412b730 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ead9f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4124a60 con 0x7fa5e412b2b0 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ead9f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4098be0 con 0x7fa5e4066180 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e4117930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ea59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e412b730 0x7fa5e4113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e4117930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34200/0 (socket says 172.21.15.89:34200) 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 -- 172.21.15.89:0/795194413 learned_addr learned my addr 172.21.15.89:0/795194413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e959c700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 -- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 msgr2=0x7fa5e4113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 -- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e412b730 msgr2=0x7fa5e4113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e412b730 0x7fa5e4113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 -- 172.21.15.89:0/795194413 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc009a20 con 0x7fa5e4066180 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ea59e700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e412b730 0x7fa5e4113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e959c700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5e9d9d700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e4117930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa5d8009df0 tx=0x7fa5d8017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5d5ffb700 1 -- 172.21.15.89:0/795194413 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8000bc0 con 0x7fa5e4066180 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5d5ffb700 1 -- 172.21.15.89:0/795194413 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d800f7e0 con 0x7fa5e4066180 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d8009370 con 0x7fa5e4066180 2023-09-30T22:14:15.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.488+0000 7fa5d5ffb700 1 -- 172.21.15.89:0/795194413 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8000d20 con 0x7fa5e4066180 2023-09-30T22:14:15.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.489+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e4116e90 con 0x7fa5e4066180 2023-09-30T22:14:15.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.489+0000 7fa5d5ffb700 1 -- 172.21.15.89:0/795194413 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fa5d8021070 con 0x7fa5e4066180 2023-09-30T22:14:15.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.489+0000 7fa5d5ffb700 1 -- 172.21.15.89:0/795194413 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fa5d803c870 con 0x7fa5e4066180 2023-09-30T22:14:15.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.490+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e405a260 con 0x7fa5e4066180 2023-09-30T22:14:15.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.494+0000 7fa5d5ffb700 1 -- 172.21.15.89:0/795194413 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fa5d8024680 con 0x7fa5e4066180 2023-09-30T22:14:15.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.652+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa5e4065c60 con 0x7fa5e4066180 2023-09-30T22:14:15.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.652+0000 7fa5d5ffb700 1 -- 172.21.15.89:0/795194413 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa5d8019390 con 0x7fa5e4066180 2023-09-30T22:14:15.652 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:14:15.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 >> v1:172.21.15.89:6816/118161 conn(0x7fa5c802bb60 legacy=0x7fa5c802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 msgr2=0x7fa5e4117930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e4117930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa5d8009df0 tx=0x7fa5d8017000).stop 2023-09-30T22:14:15.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 shutdown_connections 2023-09-30T22:14:15.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e412b730 0x7fa5e4113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5e4066180 0x7fa5e4117930 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 --2- 172.21.15.89:0/795194413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5e412b2b0 0x7fa5e4113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 >> 172.21.15.89:0/795194413 conn(0x7fa5e409b8f0 msgr2=0x7fa5e4127890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:15.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 shutdown_connections 2023-09-30T22:14:15.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.654+0000 7fa5ead9f700 1 -- 172.21.15.89:0/795194413 wait complete. 2023-09-30T22:14:15.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:14:15.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.783+0000 7fa8c8243700 1 Processor -- start 2023-09-30T22:14:15.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c8243700 1 -- start start 2023-09-30T22:14:15.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c8243700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c00613a0 0x7fa8c00919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c8243700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c00922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c8243700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8c0127990 0x7fa8c0127df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c8243700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c0061780 con 0x7fa8c0127990 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c8243700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0092c00 con 0x7fa8c0091ed0 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c8243700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0092ff0 con 0x7fa8c00613a0 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c67e0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8c0127990 0x7fa8c0127df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c57de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c00922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c5fdf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c00613a0 0x7fa8c00919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c67e0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8c0127990 0x7fa8c0127df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36870/0 (socket says 172.21.15.89:36870) 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c5fdf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c00613a0 0x7fa8c00919e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34202/0 (socket says 172.21.15.89:34202) 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c57de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c00922b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52296/0 (socket says 172.21.15.89:52296) 2023-09-30T22:14:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.784+0000 7fa8c67e0700 1 -- 172.21.15.89:0/261595166 learned_addr learned my addr 172.21.15.89:0/261595166 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c67e0700 1 -- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c00613a0 msgr2=0x7fa8c00919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c67e0700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c00613a0 0x7fa8c00919e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c67e0700 1 -- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 msgr2=0x7fa8c00922b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c67e0700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c00922b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c67e0700 1 -- 172.21.15.89:0/261595166 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0093b20 con 0x7fa8c0127990 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c57de700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c00922b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c5fdf700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c00613a0 0x7fa8c00919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c67e0700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c0127990 0x7fa8c0127df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fa8b800f490 tx=0x7fa8b800dc50).ready entity=mon.0 client_cookie=6c5ca772bc79b5fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c4fdd700 1 -- 172.21.15.89:0/261595166 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8012070 con 0x7fa8c0127990 2023-09-30T22:14:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c4fdd700 1 -- 172.21.15.89:0/261595166 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b800d0d0 con 0x7fa8c0127990 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c4fdd700 1 -- 172.21.15.89:0/261595166 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800d450 con 0x7fa8c0127990 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c8243700 1 -- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c0127990 msgr2=0x7fa8c0127df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c8243700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c0127990 0x7fa8c0127df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fa8b800f490 tx=0x7fa8b800dc50).stop 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c8243700 1 -- 172.21.15.89:0/261595166 shutdown_connections 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c8243700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c0127990 0x7fa8c0127df0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c8243700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c00613a0 0x7fa8c00919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c8243700 1 --2- 172.21.15.89:0/261595166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c00922b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.785+0000 7fa8c8243700 1 -- 172.21.15.89:0/261595166 >> 172.21.15.89:0/261595166 conn(0x7fa8c009a1c0 msgr2=0x7fa8c009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.786+0000 7fa8c8243700 1 -- 172.21.15.89:0/261595166 shutdown_connections 2023-09-30T22:14:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.786+0000 7fa8c8243700 1 -- 172.21.15.89:0/261595166 wait complete. 2023-09-30T22:14:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.786+0000 7fa8c8243700 1 Processor -- start 2023-09-30T22:14:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.786+0000 7fa8c8243700 1 -- start start 2023-09-30T22:14:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c8243700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c00613a0 0x7fa8c0115340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c8243700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c0115830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c8243700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c0127990 0x7fa8c0115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c8243700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0097490 con 0x7fa8c00613a0 2023-09-30T22:14:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c8243700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0118230 con 0x7fa8c0091ed0 2023-09-30T22:14:15.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c8243700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0093d70 con 0x7fa8c0127990 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c5fdf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c00613a0 0x7fa8c0115340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c67e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c0127990 0x7fa8c0115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c57de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c0115830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c5fdf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c00613a0 0x7fa8c0115340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36876/0 (socket says 172.21.15.89:36876) 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c5fdf700 1 -- 172.21.15.89:0/247589154 learned_addr learned my addr 172.21.15.89:0/247589154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c57de700 1 -- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c0127990 msgr2=0x7fa8c0115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c57de700 1 --2- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c0127990 0x7fa8c0115d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c57de700 1 -- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c00613a0 msgr2=0x7fa8c0115340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c57de700 1 --2- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c00613a0 0x7fa8c0115340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c57de700 1 -- 172.21.15.89:0/247589154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b40092a0 con 0x7fa8c0091ed0 2023-09-30T22:14:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c5fdf700 1 --2- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c00613a0 0x7fa8c0115340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c57de700 1 --2- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c0115830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fa8bc00e950 tx=0x7fa8bc009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8b1ffb700 1 -- 172.21.15.89:0/247589154 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc012600 con 0x7fa8c0091ed0 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8b1ffb700 1 -- 172.21.15.89:0/247589154 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc012760 con 0x7fa8c0091ed0 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc0092a0 con 0x7fa8c0091ed0 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.787+0000 7fa8b1ffb700 1 -- 172.21.15.89:0/247589154 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc012b00 con 0x7fa8c0091ed0 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.788+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b800f050 con 0x7fa8c0091ed0 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.788+0000 7fa8b1ffb700 1 -- 172.21.15.89:0/247589154 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fa8bc01b400 con 0x7fa8c0091ed0 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.788+0000 7fa8b1ffb700 1 -- 172.21.15.89:0/247589154 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fa8bc03b470 con 0x7fa8c0091ed0 2023-09-30T22:14:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.789+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c005a260 con 0x7fa8c0091ed0 2023-09-30T22:14:15.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.793+0000 7fa8b1ffb700 1 -- 172.21.15.89:0/247589154 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fa8bc019900 con 0x7fa8c0091ed0 2023-09-30T22:14:15.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.951+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8c0046c20 con 0x7fa8c0091ed0 2023-09-30T22:14:15.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.951+0000 7fa8b1ffb700 1 -- 172.21.15.89:0/247589154 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa8bc019310 con 0x7fa8c0091ed0 2023-09-30T22:14:15.951 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:14:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 >> v1:172.21.15.89:6816/118161 conn(0x7fa8a402bb60 legacy=0x7fa8a402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 msgr2=0x7fa8c0115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 --2- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c0115830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fa8bc00e950 tx=0x7fa8bc009060).stop 2023-09-30T22:14:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 shutdown_connections 2023-09-30T22:14:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 --2- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c0127990 0x7fa8c0115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 --2- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c00613a0 0x7fa8c0115340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 --2- 172.21.15.89:0/247589154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c0091ed0 0x7fa8c0115830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 >> 172.21.15.89:0/247589154 conn(0x7fa8c009a1c0 msgr2=0x7fa8c00963c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 shutdown_connections 2023-09-30T22:14:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:15.953+0000 7fa8c8243700 1 -- 172.21.15.89:0/247589154 wait complete. 2023-09-30T22:14:15.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:14:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c2883700 1 Processor -- start 2023-09-30T22:14:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c2883700 1 -- start start 2023-09-30T22:14:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c2883700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f93bc12b450 0x7f93bc12fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c2883700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc130250 0x7f93bc132640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c2883700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93bc0985c0 con 0x7f93bc0613a0 2023-09-30T22:14:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c2883700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc061780 con 0x7f93bc12b450 2023-09-30T22:14:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c2883700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc091dd0 con 0x7f93bc130250 2023-09-30T22:14:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c0e20700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc130250 0x7f93bc132640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c0e20700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc130250 0x7f93bc132640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34212/0 (socket says 172.21.15.89:34212) 2023-09-30T22:14:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93c0e20700 1 -- 172.21.15.89:0/4144489519 learned_addr learned my addr 172.21.15.89:0/4144489519 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93bb7fe700 1 --2- 172.21.15.89:0/4144489519 >> v2:172.21.15.89:3300/0 conn(0x7f93bc12b450 0x7f93bc12fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.083+0000 7f93bbfff700 1 --1- 172.21.15.89:0/4144489519 >> v1:172.21.15.89:6789/0 conn(0x7f93bc0613a0 0x7f93bc12ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54298/0 (socket says 172.21.15.89:54298) 2023-09-30T22:14:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.084+0000 7f93c0e20700 1 -- 172.21.15.89:0/4144489519 >> v2:172.21.15.89:3300/0 conn(0x7f93bc12b450 msgr2=0x7f93bc12fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.084+0000 7f93c0e20700 1 --2- 172.21.15.89:0/4144489519 >> v2:172.21.15.89:3300/0 conn(0x7f93bc12b450 0x7f93bc12fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.084+0000 7f93c0e20700 1 -- 172.21.15.89:0/4144489519 >> v1:172.21.15.89:6789/0 conn(0x7f93bc0613a0 legacy=0x7f93bc12ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.084+0000 7f93c0e20700 1 -- 172.21.15.89:0/4144489519 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc0996a0 con 0x7f93bc130250 2023-09-30T22:14:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.084+0000 7f93c0e20700 1 --2- 172.21.15.89:0/4144489519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc130250 0x7f93bc132640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f93b4009f50 tx=0x7f93b4009840).ready entity=mon.2 client_cookie=552ef717a805d144 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.085+0000 7f93baffd700 1 -- 172.21.15.89:0/4144489519 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b400e070 con 0x7f93bc130250 2023-09-30T22:14:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.085+0000 7f93baffd700 1 -- 172.21.15.89:0/4144489519 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b400b9c0 con 0x7f93bc130250 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.085+0000 7f93baffd700 1 -- 172.21.15.89:0/4144489519 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b400bd40 con 0x7f93bc130250 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.085+0000 7f93c2883700 1 -- 172.21.15.89:0/4144489519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc130250 msgr2=0x7f93bc132640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.085+0000 7f93c2883700 1 --2- 172.21.15.89:0/4144489519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc130250 0x7f93bc132640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f93b4009f50 tx=0x7f93b4009840).stop 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.085+0000 7f93c2883700 1 -- 172.21.15.89:0/4144489519 shutdown_connections 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.085+0000 7f93c2883700 1 --2- 172.21.15.89:0/4144489519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc130250 0x7f93bc132640 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.085+0000 7f93c2883700 1 --2- 172.21.15.89:0/4144489519 >> v2:172.21.15.89:3300/0 conn(0x7f93bc12b450 0x7f93bc12fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.086+0000 7f93c2883700 1 -- 172.21.15.89:0/4144489519 >> 172.21.15.89:0/4144489519 conn(0x7f93bc09a1c0 msgr2=0x7f93bc09a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.086+0000 7f93c2883700 1 -- 172.21.15.89:0/4144489519 shutdown_connections 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.086+0000 7f93c2883700 1 -- 172.21.15.89:0/4144489519 wait complete. 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.086+0000 7f93c2883700 1 Processor -- start 2023-09-30T22:14:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c2883700 1 -- start start 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c2883700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc0613a0 0x7f93bc118470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c2883700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93bc12b450 0x7f93bc114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c2883700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93bc130250 0x7f93bc114930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c2883700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1158d0 con 0x7f93bc130250 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c2883700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc12a5f0 con 0x7f93bc12b450 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c2883700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc116b00 con 0x7f93bc0613a0 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93bb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93bc12b450 0x7f93bc114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c0e20700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93bc130250 0x7f93bc114930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93bb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93bc12b450 0x7f93bc114440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52310/0 (socket says 172.21.15.89:52310) 2023-09-30T22:14:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93bb7fe700 1 -- 172.21.15.89:0/533108811 learned_addr learned my addr 172.21.15.89:0/533108811 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93bbfff700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc0613a0 0x7f93bc118470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c0e20700 1 -- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc0613a0 msgr2=0x7f93bc118470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c0e20700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc0613a0 0x7f93bc118470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c0e20700 1 -- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93bc12b450 msgr2=0x7f93bc114440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c0e20700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93bc12b450 0x7f93bc114440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c0e20700 1 -- 172.21.15.89:0/533108811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4009ae0 con 0x7f93bc130250 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93bb7fe700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93bc12b450 0x7f93bc114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.087+0000 7f93c0e20700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93bc130250 0x7f93bc114930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f93b4013000 tx=0x7f93bc099180).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.088+0000 7f93a3fff700 1 -- 172.21.15.89:0/533108811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40092e0 con 0x7f93bc130250 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.088+0000 7f93a3fff700 1 -- 172.21.15.89:0/533108811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4013d70 con 0x7f93bc130250 2023-09-30T22:14:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.088+0000 7f93a3fff700 1 -- 172.21.15.89:0/533108811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b401e6b0 con 0x7f93bc130250 2023-09-30T22:14:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.088+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc1179d0 con 0x7f93bc130250 2023-09-30T22:14:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.088+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc116060 con 0x7f93bc130250 2023-09-30T22:14:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.088+0000 7f93a3fff700 1 -- 172.21.15.89:0/533108811 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f93b401eae0 con 0x7f93bc130250 2023-09-30T22:14:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.088+0000 7f93a3fff700 1 -- 172.21.15.89:0/533108811 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f93b403cf60 con 0x7f93bc130250 2023-09-30T22:14:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.089+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc116250 con 0x7f93bc130250 2023-09-30T22:14:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.093+0000 7f93a3fff700 1 -- 172.21.15.89:0/533108811 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f93b401ed90 con 0x7f93bc130250 2023-09-30T22:14:16.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.249+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93bc0938f0 con 0x7f93bc130250 2023-09-30T22:14:16.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.250+0000 7f93a3fff700 1 -- 172.21.15.89:0/533108811 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93b401caf0 con 0x7f93bc130250 2023-09-30T22:14:16.249 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-09-30T22:14:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 >> v1:172.21.15.89:6816/118161 conn(0x7f93a802bb90 legacy=0x7f93a802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93bc130250 msgr2=0x7f93bc114930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93bc130250 0x7f93bc114930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f93b4013000 tx=0x7f93bc099180).stop 2023-09-30T22:14:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 shutdown_connections 2023-09-30T22:14:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93bc130250 0x7f93bc114930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93bc0613a0 0x7f93bc118470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 --2- 172.21.15.89:0/533108811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93bc12b450 0x7f93bc114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 >> 172.21.15.89:0/533108811 conn(0x7f93bc09a1c0 msgr2=0x7f93bc066b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:16.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 shutdown_connections 2023-09-30T22:14:16.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.251+0000 7f93c2883700 1 -- 172.21.15.89:0/533108811 wait complete. 2023-09-30T22:14:16.263 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:14:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.382+0000 7f05346aa700 1 Processor -- start 2023-09-30T22:14:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.382+0000 7f05346aa700 1 -- start start 2023-09-30T22:14:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.382+0000 7f05346aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c098740 0x7f052c098b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.382+0000 7f05346aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c069430 0x7f052c130c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.382+0000 7f05346aa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c095270 con 0x7f052c099010 2023-09-30T22:14:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.382+0000 7f05346aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c094d20 con 0x7f052c098740 2023-09-30T22:14:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.382+0000 7f05346aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f052c096520 con 0x7f052c069430 2023-09-30T22:14:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0532446700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c098740 0x7f052c098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0532c47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c069430 0x7f052c130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0531c45700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f052c099010 0x7f052c068d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54308/0 (socket says 172.21.15.89:54308) 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0531c45700 1 -- 172.21.15.89:0/4043290182 learned_addr learned my addr 172.21.15.89:0/4043290182 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0532c47700 1 -- 172.21.15.89:0/4043290182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c098740 msgr2=0x7f052c098b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0532c47700 1 --2- 172.21.15.89:0/4043290182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c098740 0x7f052c098b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0532c47700 1 -- 172.21.15.89:0/4043290182 >> v1:172.21.15.89:6789/0 conn(0x7f052c099010 legacy=0x7f052c068d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0531444700 1 -- 172.21.15.89:0/4043290182 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555243400 0 0) 0x7f052c095270 con 0x7f052c099010 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0532c47700 1 -- 172.21.15.89:0/4043290182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c094870 con 0x7f052c069430 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0532446700 1 --2- 172.21.15.89:0/4043290182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c098740 0x7f052c098b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0532c47700 1 --2- 172.21.15.89:0/4043290182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c069430 0x7f052c130c00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0524009e90 tx=0x7f0524009780).ready entity=mon.2 client_cookie=d218b5cee030e2d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0531444700 1 -- 172.21.15.89:0/4043290182 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052400e070 con 0x7f052c069430 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0531444700 1 -- 172.21.15.89:0/4043290182 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052400bb60 con 0x7f052c069430 2023-09-30T22:14:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.383+0000 7f0531444700 1 -- 172.21.15.89:0/4043290182 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524012410 con 0x7f052c069430 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.384+0000 7f05346aa700 1 -- 172.21.15.89:0/4043290182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c069430 msgr2=0x7f052c130c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.384+0000 7f05346aa700 1 --2- 172.21.15.89:0/4043290182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c069430 0x7f052c130c00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0524009e90 tx=0x7f0524009780).stop 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.384+0000 7f05346aa700 1 -- 172.21.15.89:0/4043290182 shutdown_connections 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.384+0000 7f05346aa700 1 --2- 172.21.15.89:0/4043290182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c098740 0x7f052c098b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.384+0000 7f05346aa700 1 --2- 172.21.15.89:0/4043290182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c069430 0x7f052c130c00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.384+0000 7f05346aa700 1 -- 172.21.15.89:0/4043290182 >> 172.21.15.89:0/4043290182 conn(0x7f052c09a1c0 msgr2=0x7f052c09a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.384+0000 7f05346aa700 1 -- 172.21.15.89:0/4043290182 shutdown_connections 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.384+0000 7f05346aa700 1 -- 172.21.15.89:0/4043290182 wait complete. 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f05346aa700 1 Processor -- start 2023-09-30T22:14:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f05346aa700 1 -- start start 2023-09-30T22:14:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f05346aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c069430 0x7f052c1380b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f05346aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c098740 0x7f052c1385a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f05346aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c099010 0x7f052c117da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f05346aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c097fe0 con 0x7f052c069430 2023-09-30T22:14:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f05346aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c12bdf0 con 0x7f052c099010 2023-09-30T22:14:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f05346aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f052c119a60 con 0x7f052c098740 2023-09-30T22:14:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0531c45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c098740 0x7f052c1385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0532446700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c069430 0x7f052c1380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0532c47700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c099010 0x7f052c117da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0532446700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c069430 0x7f052c1380b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36916/0 (socket says 172.21.15.89:36916) 2023-09-30T22:14:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0532c47700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c099010 0x7f052c117da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52326/0 (socket says 172.21.15.89:52326) 2023-09-30T22:14:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0532446700 1 -- 172.21.15.89:0/1451061686 learned_addr learned my addr 172.21.15.89:0/1451061686 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0531c45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c098740 0x7f052c1385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34230/0 (socket says 172.21.15.89:34230) 2023-09-30T22:14:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0531c45700 1 -- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c099010 msgr2=0x7f052c117da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0531c45700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c099010 0x7f052c117da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0531c45700 1 -- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c069430 msgr2=0x7f052c1380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0531c45700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c069430 0x7f052c1380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0531c45700 1 -- 172.21.15.89:0/1451061686 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05280092a0 con 0x7f052c098740 2023-09-30T22:14:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0532c47700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c099010 0x7f052c117da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.385+0000 7f0532446700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c069430 0x7f052c1380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.386+0000 7f0531c45700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c098740 0x7f052c1385a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f051c000c00 tx=0x7f051c007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.386+0000 7f05227fc700 1 -- 172.21.15.89:0/1451061686 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0070d0 con 0x7f052c098740 2023-09-30T22:14:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.386+0000 7f05227fc700 1 -- 172.21.15.89:0/1451061686 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c007250 con 0x7f052c098740 2023-09-30T22:14:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.386+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0524009a20 con 0x7f052c098740 2023-09-30T22:14:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.386+0000 7f05227fc700 1 -- 172.21.15.89:0/1451061686 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c007610 con 0x7f052c098740 2023-09-30T22:14:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.386+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c11b6a0 con 0x7f052c098740 2023-09-30T22:14:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.386+0000 7f05227fc700 1 -- 172.21.15.89:0/1451061686 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f051c0059b0 con 0x7f052c098740 2023-09-30T22:14:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.387+0000 7f05227fc700 1 -- 172.21.15.89:0/1451061686 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f051c03a6c0 con 0x7f052c098740 2023-09-30T22:14:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.387+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c05a260 con 0x7f052c098740 2023-09-30T22:14:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.391+0000 7f05227fc700 1 -- 172.21.15.89:0/1451061686 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f051c013970 con 0x7f052c098740 2023-09-30T22:14:16.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.546+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f052c094540 con 0x7f052c098740 2023-09-30T22:14:16.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.547+0000 7f05227fc700 1 -- 172.21.15.89:0/1451061686 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f051c010300 con 0x7f052c098740 2023-09-30T22:14:16.547 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-09-30T22:14:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 >> v1:172.21.15.89:6816/118161 conn(0x7f051002bb60 legacy=0x7f051002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c098740 msgr2=0x7f052c1385a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c098740 0x7f052c1385a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f051c000c00 tx=0x7f051c007b90).stop 2023-09-30T22:14:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 shutdown_connections 2023-09-30T22:14:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f052c099010 0x7f052c117da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c069430 0x7f052c1380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 --2- 172.21.15.89:0/1451061686 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f052c098740 0x7f052c1385a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 >> 172.21.15.89:0/1451061686 conn(0x7f052c09a1c0 msgr2=0x7f052c0681f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 shutdown_connections 2023-09-30T22:14:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.548+0000 7f05346aa700 1 -- 172.21.15.89:0/1451061686 wait complete. 2023-09-30T22:14:16.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:14:16.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.674+0000 7f7053c5d700 1 Processor -- start 2023-09-30T22:14:16.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.674+0000 7f7053c5d700 1 -- start start 2023-09-30T22:14:16.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f7053c5d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f704c0654a0 0x7f704c098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f7053c5d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c098f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f7053c5d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c065ed0 con 0x7f704c12f470 2023-09-30T22:14:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f7053c5d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704c095880 con 0x7f704c0654a0 2023-09-30T22:14:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f7053c5d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f704c096120 con 0x7f704c098b30 2023-09-30T22:14:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70519f9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f704c0654a0 0x7f704c098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70511f8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c098f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70511f8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c098f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34236/0 (socket says 172.21.15.89:34236) 2023-09-30T22:14:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70521fa700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f704c12f470 0x7f704c131860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54320/0 (socket says 172.21.15.89:54320) 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70511f8700 1 -- 172.21.15.89:0/4105267077 learned_addr learned my addr 172.21.15.89:0/4105267077 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70519f9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f704c0654a0 0x7f704c098640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36918/0 (socket says 172.21.15.89:36918) 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70511f8700 1 -- 172.21.15.89:0/4105267077 >> v2:172.21.15.89:3300/0 conn(0x7f704c0654a0 msgr2=0x7f704c098640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70511f8700 1 --2- 172.21.15.89:0/4105267077 >> v2:172.21.15.89:3300/0 conn(0x7f704c0654a0 0x7f704c098640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70511f8700 1 -- 172.21.15.89:0/4105267077 >> v1:172.21.15.89:6789/0 conn(0x7f704c12f470 legacy=0x7f704c131860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70509f7700 1 -- 172.21.15.89:0/4105267077 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319629682 0 0) 0x7f704c065ed0 con 0x7f704c12f470 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70511f8700 1 -- 172.21.15.89:0/4105267077 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c099460 con 0x7f704c098b30 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70519f9700 1 --2- 172.21.15.89:0/4105267077 >> v2:172.21.15.89:3300/0 conn(0x7f704c0654a0 0x7f704c098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.675+0000 7f70511f8700 1 --2- 172.21.15.89:0/4105267077 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c098f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7048006fd0 tx=0x7f70480090f0).ready entity=mon.2 client_cookie=835d604a414605ae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f70509f7700 1 -- 172.21.15.89:0/4105267077 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800b580 con 0x7f704c098b30 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f70509f7700 1 -- 172.21.15.89:0/4105267077 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704800b6e0 con 0x7f704c098b30 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f70509f7700 1 -- 172.21.15.89:0/4105267077 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800ba60 con 0x7f704c098b30 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f7053c5d700 1 -- 172.21.15.89:0/4105267077 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 msgr2=0x7f704c098f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f7053c5d700 1 --2- 172.21.15.89:0/4105267077 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c098f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7048006fd0 tx=0x7f70480090f0).stop 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f7053c5d700 1 -- 172.21.15.89:0/4105267077 shutdown_connections 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f7053c5d700 1 --2- 172.21.15.89:0/4105267077 >> v2:172.21.15.89:3300/0 conn(0x7f704c0654a0 0x7f704c098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f7053c5d700 1 --2- 172.21.15.89:0/4105267077 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c098f70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f7053c5d700 1 -- 172.21.15.89:0/4105267077 >> 172.21.15.89:0/4105267077 conn(0x7f704c09a740 msgr2=0x7f704c09ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f7053c5d700 1 -- 172.21.15.89:0/4105267077 shutdown_connections 2023-09-30T22:14:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.676+0000 7f7053c5d700 1 -- 172.21.15.89:0/4105267077 wait complete. 2023-09-30T22:14:16.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.677+0000 7f7053c5d700 1 Processor -- start 2023-09-30T22:14:16.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.677+0000 7f7053c5d700 1 -- start start 2023-09-30T22:14:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7053c5d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704c0654a0 0x7f704c11a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7053c5d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c11a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7053c5d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704c12f470 0x7f704c11ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7053c5d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c11b070 con 0x7f704c12f470 2023-09-30T22:14:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7053c5d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c11d730 con 0x7f704c0654a0 2023-09-30T22:14:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7053c5d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f704c11bf40 con 0x7f704c098b30 2023-09-30T22:14:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70519f9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704c0654a0 0x7f704c11a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70521fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704c12f470 0x7f704c11ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70519f9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704c0654a0 0x7f704c11a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52342/0 (socket says 172.21.15.89:52342) 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70519f9700 1 -- 172.21.15.89:0/742993726 learned_addr learned my addr 172.21.15.89:0/742993726 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70511f8700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c11a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70511f8700 1 -- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704c0654a0 msgr2=0x7f704c11a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70511f8700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704c0654a0 0x7f704c11a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70511f8700 1 -- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704c12f470 msgr2=0x7f704c11ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70511f8700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704c12f470 0x7f704c11ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70511f8700 1 -- 172.21.15.89:0/742993726 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c0092a0 con 0x7f704c098b30 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70521fa700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704c12f470 0x7f704c11ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70519f9700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704c0654a0 0x7f704c11a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f70511f8700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c11a690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7048014000 tx=0x7f7048017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7041ffb700 1 -- 172.21.15.89:0/742993726 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048000bc0 con 0x7f704c098b30 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7041ffb700 1 -- 172.21.15.89:0/742993726 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704800b850 con 0x7f704c098b30 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7048009370 con 0x7f704c098b30 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7041ffb700 1 -- 172.21.15.89:0/742993726 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048000d20 con 0x7f704c098b30 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.678+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c099b00 con 0x7f704c098b30 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.679+0000 7f7041ffb700 1 -- 172.21.15.89:0/742993726 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f704801c880 con 0x7f704c098b30 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.679+0000 7f7041ffb700 1 -- 172.21.15.89:0/742993726 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f704803cc60 con 0x7f704c098b30 2023-09-30T22:14:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.679+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c05a260 con 0x7f704c098b30 2023-09-30T22:14:16.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.684+0000 7f7041ffb700 1 -- 172.21.15.89:0/742993726 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f704802a080 con 0x7f704c098b30 2023-09-30T22:14:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.881+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f704c098070 con 0x7f704c098b30 2023-09-30T22:14:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.882+0000 7f7041ffb700 1 -- 172.21.15.89:0/742993726 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f7048020020 con 0x7f704c098b30 2023-09-30T22:14:16.881 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:16.882 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:04.044604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:14:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.883+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 >> v1:172.21.15.89:6816/118161 conn(0x7f703002bb60 legacy=0x7f703002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.883+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 msgr2=0x7f704c11a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.883+0000 7f7053c5d700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c11a690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7048014000 tx=0x7f7048017000).stop 2023-09-30T22:14:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.883+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 shutdown_connections 2023-09-30T22:14:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.883+0000 7f7053c5d700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704c12f470 0x7f704c11ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.883+0000 7f7053c5d700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704c0654a0 0x7f704c11a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.883+0000 7f7053c5d700 1 --2- 172.21.15.89:0/742993726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704c098b30 0x7f704c11a690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.883+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 >> 172.21.15.89:0/742993726 conn(0x7f704c09a740 msgr2=0x7f704c09bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.884+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 shutdown_connections 2023-09-30T22:14:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:16.884+0000 7f7053c5d700 1 -- 172.21.15.89:0/742993726 wait complete. 2023-09-30T22:14:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 5 2023-09-30T22:14:16.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.010+0000 7f242fe56700 1 Processor -- start 2023-09-30T22:14:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242fe56700 1 -- start start 2023-09-30T22:14:17.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242fe56700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428091830 0x7f2428091c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242fe56700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428127510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242fe56700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2428127a00 0x7f242812e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242fe56700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2428092e00 con 0x7f2428127a00 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242fe56700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428092b40 con 0x7f2428091830 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242fe56700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2428093e70 con 0x7f24281270f0 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242e3f3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2428127a00 0x7f242812e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242d3f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428127510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242dbf2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428091830 0x7f2428091c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242e3f3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2428127a00 0x7f242812e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36942/0 (socket says 172.21.15.89:36942) 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242d3f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428127510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34254/0 (socket says 172.21.15.89:34254) 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242dbf2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428091830 0x7f2428091c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52348/0 (socket says 172.21.15.89:52348) 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.011+0000 7f242e3f3700 1 -- 172.21.15.89:0/2538719790 learned_addr learned my addr 172.21.15.89:0/2538719790 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:17.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242e3f3700 1 -- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 msgr2=0x7f2428127510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242e3f3700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428127510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242e3f3700 1 -- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428091830 msgr2=0x7f2428091c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242e3f3700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428091830 0x7f2428091c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242e3f3700 1 -- 172.21.15.89:0/2538719790 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428091610 con 0x7f2428127a00 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242dbf2700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428091830 0x7f2428091c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242d3f1700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428127510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242e3f3700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428127a00 0x7f242812e470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f242000c0d0 tx=0x7f242000a950).ready entity=mon.0 client_cookie=f5f289a1348d3b9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242cbf0700 1 -- 172.21.15.89:0/2538719790 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000f070 con 0x7f2428127a00 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242cbf0700 1 -- 172.21.15.89:0/2538719790 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242000a2e0 con 0x7f2428127a00 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242cbf0700 1 -- 172.21.15.89:0/2538719790 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000cc90 con 0x7f2428127a00 2023-09-30T22:14:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242fe56700 1 -- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428127a00 msgr2=0x7f242812e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242fe56700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428127a00 0x7f242812e470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f242000c0d0 tx=0x7f242000a950).stop 2023-09-30T22:14:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242fe56700 1 -- 172.21.15.89:0/2538719790 shutdown_connections 2023-09-30T22:14:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242fe56700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428127a00 0x7f242812e470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242fe56700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428091830 0x7f2428091c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242fe56700 1 --2- 172.21.15.89:0/2538719790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428127510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.012+0000 7f242fe56700 1 -- 172.21.15.89:0/2538719790 >> 172.21.15.89:0/2538719790 conn(0x7f242809a750 msgr2=0x7f24280967d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.013+0000 7f242fe56700 1 -- 172.21.15.89:0/2538719790 shutdown_connections 2023-09-30T22:14:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.013+0000 7f242fe56700 1 -- 172.21.15.89:0/2538719790 wait complete. 2023-09-30T22:14:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.013+0000 7f242fe56700 1 Processor -- start 2023-09-30T22:14:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.013+0000 7f242fe56700 1 -- start start 2023-09-30T22:14:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242fe56700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428091830 0x7f2428083170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242fe56700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428121250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242fe56700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428127a00 0x7f2428121740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242fe56700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280838f0 con 0x7f2428091830 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242fe56700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428126310 con 0x7f2428127a00 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242fe56700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2428062970 con 0x7f24281270f0 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428091830 0x7f2428083170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428091830 0x7f2428083170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36958/0 (socket says 172.21.15.89:36958) 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 -- 172.21.15.89:0/2182633474 learned_addr learned my addr 172.21.15.89:0/2182633474 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242e3f3700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428127a00 0x7f2428121740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242d3f1700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428121250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 -- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 msgr2=0x7f2428121250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428121250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 -- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428127a00 msgr2=0x7f2428121740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428127a00 0x7f2428121740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 -- 172.21.15.89:0/2182633474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418009250 con 0x7f2428091830 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242e3f3700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428127a00 0x7f2428121740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f242dbf2700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428091830 0x7f2428083170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f241800ea20 tx=0x7f2418006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f241dffb700 1 -- 172.21.15.89:0/2182633474 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800ee60 con 0x7f2428091830 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.014+0000 7f241dffb700 1 -- 172.21.15.89:0/2182633474 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418009600 con 0x7f2428091830 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.015+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24240092a0 con 0x7f2428091830 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.015+0000 7f241dffb700 1 -- 172.21.15.89:0/2182633474 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418005a90 con 0x7f2428091830 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.015+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242000abf0 con 0x7f2428091830 2023-09-30T22:14:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.015+0000 7f241dffb700 1 -- 172.21.15.89:0/2182633474 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f241800c850 con 0x7f2428091830 2023-09-30T22:14:17.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.015+0000 7f241dffb700 1 -- 172.21.15.89:0/2182633474 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f241803b120 con 0x7f2428091830 2023-09-30T22:14:17.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.016+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242805a260 con 0x7f2428091830 2023-09-30T22:14:17.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.020+0000 7f241dffb700 1 -- 172.21.15.89:0/2182633474 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2418016070 con 0x7f2428091830 2023-09-30T22:14:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.181+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2428046c20 con 0x7f2428091830 2023-09-30T22:14:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.182+0000 7f241dffb700 1 -- 172.21.15.89:0/2182633474 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f2418016020 con 0x7f2428091830 2023-09-30T22:14:17.182 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:17.182 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:03.041750+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:17.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 >> v1:172.21.15.89:6816/118161 conn(0x7f240c02bb60 legacy=0x7f240c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428091830 msgr2=0x7f2428083170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428091830 0x7f2428083170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f241800ea20 tx=0x7f2418006f40).stop 2023-09-30T22:14:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 shutdown_connections 2023-09-30T22:14:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2428127a00 0x7f2428121740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2428091830 0x7f2428083170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 --2- 172.21.15.89:0/2182633474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24281270f0 0x7f2428121250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 >> 172.21.15.89:0/2182633474 conn(0x7f242809a750 msgr2=0x7f2428098710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 shutdown_connections 2023-09-30T22:14:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.184+0000 7f242fe56700 1 -- 172.21.15.89:0/2182633474 wait complete. 2023-09-30T22:14:17.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:14:00.533280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:14:01.011104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:14:17.196 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:14:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5aa5779700 1 Processor -- start 2023-09-30T22:14:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5aa5779700 1 -- start start 2023-09-30T22:14:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5aa5779700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa0066180 0x7f5aa0097480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5aa5779700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa012b610 0x7f5aa0127b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5aa5779700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa0065980 con 0x7f5aa012b1d0 2023-09-30T22:14:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5aa5779700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa0066560 con 0x7f5aa0066180 2023-09-30T22:14:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5aa5779700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00656c0 con 0x7f5aa012b610 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5a9effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa0066180 0x7f5aa0097480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5a9f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa012b610 0x7f5aa0127b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5a9effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa0066180 0x7f5aa0097480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36960/0 (socket says 172.21.15.89:36960) 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5a9e7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5aa012b1d0 0x7f5aa0127480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54332/0 (socket says 172.21.15.89:54332) 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.314+0000 7f5a9effd700 1 -- 172.21.15.89:0/2327199362 learned_addr learned my addr 172.21.15.89:0/2327199362 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9f7fe700 1 -- 172.21.15.89:0/2327199362 >> v2:172.21.15.89:3300/0 conn(0x7f5aa0066180 msgr2=0x7f5aa0097480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9f7fe700 1 --2- 172.21.15.89:0/2327199362 >> v2:172.21.15.89:3300/0 conn(0x7f5aa0066180 0x7f5aa0097480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9f7fe700 1 -- 172.21.15.89:0/2327199362 >> v1:172.21.15.89:6789/0 conn(0x7f5aa012b1d0 legacy=0x7f5aa0127480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9dffb700 1 -- 172.21.15.89:0/2327199362 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403405928 0 0) 0x7f5aa0065980 con 0x7f5aa012b1d0 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9f7fe700 1 -- 172.21.15.89:0/2327199362 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa0098250 con 0x7f5aa012b610 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9effd700 1 --2- 172.21.15.89:0/2327199362 >> v2:172.21.15.89:3300/0 conn(0x7f5aa0066180 0x7f5aa0097480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9f7fe700 1 --2- 172.21.15.89:0/2327199362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa012b610 0x7f5aa0127b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5a98009e90 tx=0x7f5a98009780).ready entity=mon.1 client_cookie=6bf0e711f9e2b82b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9dffb700 1 -- 172.21.15.89:0/2327199362 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800e070 con 0x7f5aa012b610 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9dffb700 1 -- 172.21.15.89:0/2327199362 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9800bb20 con 0x7f5aa012b610 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5a9dffb700 1 -- 172.21.15.89:0/2327199362 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800bea0 con 0x7f5aa012b610 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5aa5779700 1 -- 172.21.15.89:0/2327199362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa012b610 msgr2=0x7f5aa0127b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.315+0000 7f5aa5779700 1 --2- 172.21.15.89:0/2327199362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa012b610 0x7f5aa0127b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5a98009e90 tx=0x7f5a98009780).stop 2023-09-30T22:14:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.316+0000 7f5aa5779700 1 -- 172.21.15.89:0/2327199362 shutdown_connections 2023-09-30T22:14:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.316+0000 7f5aa5779700 1 --2- 172.21.15.89:0/2327199362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa012b610 0x7f5aa0127b90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.316+0000 7f5aa5779700 1 --2- 172.21.15.89:0/2327199362 >> v2:172.21.15.89:3300/0 conn(0x7f5aa0066180 0x7f5aa0097480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.316+0000 7f5aa5779700 1 -- 172.21.15.89:0/2327199362 >> 172.21.15.89:0/2327199362 conn(0x7f5aa009ba50 msgr2=0x7f5aa009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.316+0000 7f5aa5779700 1 -- 172.21.15.89:0/2327199362 shutdown_connections 2023-09-30T22:14:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.316+0000 7f5aa5779700 1 -- 172.21.15.89:0/2327199362 wait complete. 2023-09-30T22:14:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.316+0000 7f5aa5779700 1 Processor -- start 2023-09-30T22:14:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5aa5779700 1 -- start start 2023-09-30T22:14:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5aa5779700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa0066180 0x7f5aa013a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5aa5779700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5aa012b1d0 0x7f5aa013a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5aa5779700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa012b610 0x7f5aa01132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5aa5779700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa012f000 con 0x7f5aa012b610 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5aa5779700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0115960 con 0x7f5aa0066180 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5aa5779700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0065030 con 0x7f5aa012b1d0 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa0066180 0x7f5aa013a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa0066180 0x7f5aa013a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52372/0 (socket says 172.21.15.89:52372) 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 -- 172.21.15.89:0/3310048440 learned_addr learned my addr 172.21.15.89:0/3310048440 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa012b610 0x7f5aa01132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9e7fc700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5aa012b1d0 0x7f5aa013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 -- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5aa012b1d0 msgr2=0x7f5aa013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5aa012b1d0 0x7f5aa013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 -- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa012b610 msgr2=0x7f5aa01132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa012b610 0x7f5aa01132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 -- 172.21.15.89:0/3310048440 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a90009370 con 0x7f5aa0066180 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9f7fe700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa012b610 0x7f5aa01132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.317+0000 7f5a9effd700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa0066180 0x7f5aa013a240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f5a9000ea50 tx=0x7f5a900090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.318+0000 7f5a86ffd700 1 -- 172.21.15.89:0/3310048440 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000ee90 con 0x7f5aa0066180 2023-09-30T22:14:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.318+0000 7f5a86ffd700 1 -- 172.21.15.89:0/3310048440 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9000c850 con 0x7f5aa0066180 2023-09-30T22:14:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.318+0000 7f5a86ffd700 1 -- 172.21.15.89:0/3310048440 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000cbd0 con 0x7f5aa0066180 2023-09-30T22:14:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.318+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a98009a20 con 0x7f5aa0066180 2023-09-30T22:14:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.318+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa01174b0 con 0x7f5aa0066180 2023-09-30T22:14:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.318+0000 7f5a86ffd700 1 -- 172.21.15.89:0/3310048440 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5a9001b400 con 0x7f5aa0066180 2023-09-30T22:14:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.318+0000 7f5a86ffd700 1 -- 172.21.15.89:0/3310048440 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5a9003b510 con 0x7f5aa0066180 2023-09-30T22:14:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.319+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa00998f0 con 0x7f5aa0066180 2023-09-30T22:14:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.323+0000 7f5a86ffd700 1 -- 172.21.15.89:0/3310048440 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5a90016020 con 0x7f5aa0066180 2023-09-30T22:14:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.478+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5aa005a260 con 0x7f5aa0066180 2023-09-30T22:14:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.478+0000 7f5a86ffd700 1 -- 172.21.15.89:0/3310048440 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5a9001be60 con 0x7f5aa0066180 2023-09-30T22:14:17.478 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:14:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 >> v1:172.21.15.89:6816/118161 conn(0x7f5a8c02bb60 legacy=0x7f5a8c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa0066180 msgr2=0x7f5aa013a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa0066180 0x7f5aa013a240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f5a9000ea50 tx=0x7f5a900090f0).stop 2023-09-30T22:14:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 shutdown_connections 2023-09-30T22:14:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa012b610 0x7f5aa01132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5aa0066180 0x7f5aa013a240 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 --2- 172.21.15.89:0/3310048440 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5aa012b1d0 0x7f5aa013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 >> 172.21.15.89:0/3310048440 conn(0x7f5aa009ba50 msgr2=0x7f5aa0069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:17.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 shutdown_connections 2023-09-30T22:14:17.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.480+0000 7f5aa5779700 1 -- 172.21.15.89:0/3310048440 wait complete. 2023-09-30T22:14:17.491 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:14:17.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.609+0000 7f2db5174700 1 Processor -- start 2023-09-30T22:14:17.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.609+0000 7f2db5174700 1 -- start start 2023-09-30T22:14:17.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2db5174700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2db0093840 0x7f2db0098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2db5174700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db0098c80 0x7f2db0099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2db5174700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db012c150 0x7f2db012e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2db5174700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db0093c20 con 0x7f2db0093840 2023-09-30T22:14:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2db5174700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db0092b90 con 0x7f2db0098c80 2023-09-30T22:14:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2db5174700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2db0092d80 con 0x7f2db012c150 2023-09-30T22:14:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2db0093840 0x7f2db0098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db012c150 0x7f2db012e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2dae59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db0098c80 0x7f2db0099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db012c150 0x7f2db012e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34270/0 (socket says 172.21.15.89:34270) 2023-09-30T22:14:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2db0093840 0x7f2db0098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36980/0 (socket says 172.21.15.89:36980) 2023-09-30T22:14:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 -- 172.21.15.89:0/2542843099 learned_addr learned my addr 172.21.15.89:0/2542843099 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2dae59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db0098c80 0x7f2db0099060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52374/0 (socket says 172.21.15.89:52374) 2023-09-30T22:14:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 -- 172.21.15.89:0/2542843099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db0098c80 msgr2=0x7f2db0099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 --2- 172.21.15.89:0/2542843099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db0098c80 0x7f2db0099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 -- 172.21.15.89:0/2542843099 >> v2:172.21.15.89:3300/0 conn(0x7f2db0093840 msgr2=0x7f2db0098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 --2- 172.21.15.89:0/2542843099 >> v2:172.21.15.89:3300/0 conn(0x7f2db0093840 0x7f2db0098790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 -- 172.21.15.89:0/2542843099 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db0099b70 con 0x7f2db012c150 2023-09-30T22:14:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2dae59c700 1 --2- 172.21.15.89:0/2542843099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db0098c80 0x7f2db0099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daed9d700 1 --2- 172.21.15.89:0/2542843099 >> v2:172.21.15.89:3300/0 conn(0x7f2db0093840 0x7f2db0098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2daf59e700 1 --2- 172.21.15.89:0/2542843099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db012c150 0x7f2db012e540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2da800d1a0 tx=0x7f2da800ba50).ready entity=mon.2 client_cookie=d5a6e35a2898b50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2dadd9b700 1 -- 172.21.15.89:0/2542843099 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8010070 con 0x7f2db012c150 2023-09-30T22:14:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2dadd9b700 1 -- 172.21.15.89:0/2542843099 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da800dd60 con 0x7f2db012c150 2023-09-30T22:14:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.610+0000 7f2dadd9b700 1 -- 172.21.15.89:0/2542843099 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da800b2e0 con 0x7f2db012c150 2023-09-30T22:14:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 -- 172.21.15.89:0/2542843099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db012c150 msgr2=0x7f2db012e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 --2- 172.21.15.89:0/2542843099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db012c150 0x7f2db012e540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2da800d1a0 tx=0x7f2da800ba50).stop 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 -- 172.21.15.89:0/2542843099 shutdown_connections 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 --2- 172.21.15.89:0/2542843099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db012c150 0x7f2db012e540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 --2- 172.21.15.89:0/2542843099 >> v2:172.21.15.89:3300/0 conn(0x7f2db0093840 0x7f2db0098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 --2- 172.21.15.89:0/2542843099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db0098c80 0x7f2db0099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 -- 172.21.15.89:0/2542843099 >> 172.21.15.89:0/2542843099 conn(0x7f2db0097480 msgr2=0x7f2db0097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 -- 172.21.15.89:0/2542843099 shutdown_connections 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.611+0000 7f2db5174700 1 -- 172.21.15.89:0/2542843099 wait complete. 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2db5174700 1 Processor -- start 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2db5174700 1 -- start start 2023-09-30T22:14:17.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2db5174700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db0093840 0x7f2db011de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2db5174700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db0098c80 0x7f2db0119de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2db5174700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db012c150 0x7f2db011a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2db5174700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db011d980 con 0x7f2db0093840 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2db5174700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db0064c40 con 0x7f2db012c150 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2db5174700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2db006a520 con 0x7f2db0098c80 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2daed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db0093840 0x7f2db011de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2daf59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db012c150 0x7f2db011a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.612+0000 7f2dae59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db0098c80 0x7f2db0119de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2daed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db0093840 0x7f2db011de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36996/0 (socket says 172.21.15.89:36996) 2023-09-30T22:14:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2daf59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db012c150 0x7f2db011a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52376/0 (socket says 172.21.15.89:52376) 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2daed9d700 1 -- 172.21.15.89:0/2338374517 learned_addr learned my addr 172.21.15.89:0/2338374517 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2dae59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db0098c80 0x7f2db0119de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34280/0 (socket says 172.21.15.89:34280) 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2dae59c700 1 -- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db012c150 msgr2=0x7f2db011a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2dae59c700 1 --2- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db012c150 0x7f2db011a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2dae59c700 1 -- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db0093840 msgr2=0x7f2db011de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2dae59c700 1 --2- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db0093840 0x7f2db011de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2dae59c700 1 -- 172.21.15.89:0/2338374517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da40092a0 con 0x7f2db0098c80 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2daed9d700 1 --2- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db0093840 0x7f2db011de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2dae59c700 1 --2- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db0098c80 0x7f2db0119de0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2da000e980 tx=0x7f2da0009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2d96ffd700 1 -- 172.21.15.89:0/2338374517 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0012600 con 0x7f2db0098c80 2023-09-30T22:14:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2d96ffd700 1 -- 172.21.15.89:0/2338374517 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da0012760 con 0x7f2db0098c80 2023-09-30T22:14:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da00092a0 con 0x7f2db0098c80 2023-09-30T22:14:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2d96ffd700 1 -- 172.21.15.89:0/2338374517 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0012ae0 con 0x7f2db0098c80 2023-09-30T22:14:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.613+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da800bcf0 con 0x7f2db0098c80 2023-09-30T22:14:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.614+0000 7f2d96ffd700 1 -- 172.21.15.89:0/2338374517 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2da001b400 con 0x7f2db0098c80 2023-09-30T22:14:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.614+0000 7f2d96ffd700 1 -- 172.21.15.89:0/2338374517 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2da003b450 con 0x7f2db0098c80 2023-09-30T22:14:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.614+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db011d7c0 con 0x7f2db0098c80 2023-09-30T22:14:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.618+0000 7f2d96ffd700 1 -- 172.21.15.89:0/2338374517 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2da0016030 con 0x7f2db0098c80 2023-09-30T22:14:17.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.780+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2db011d590 con 0x7f2db0098c80 2023-09-30T22:14:17.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.780+0000 7f2d96ffd700 1 -- 172.21.15.89:0/2338374517 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2da0019310 con 0x7f2db0098c80 2023-09-30T22:14:17.780 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:14:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 >> v1:172.21.15.89:6816/118161 conn(0x7f2d9c02bb60 legacy=0x7f2d9c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db0098c80 msgr2=0x7f2db0119de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 --2- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db0098c80 0x7f2db0119de0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2da000e980 tx=0x7f2da0009060).stop 2023-09-30T22:14:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 shutdown_connections 2023-09-30T22:14:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 --2- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2db012c150 0x7f2db011a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 --2- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2db0093840 0x7f2db011de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 --2- 172.21.15.89:0/2338374517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2db0098c80 0x7f2db0119de0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 >> 172.21.15.89:0/2338374517 conn(0x7f2db0097480 msgr2=0x7f2db00690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 shutdown_connections 2023-09-30T22:14:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.782+0000 7f2db5174700 1 -- 172.21.15.89:0/2338374517 wait complete. 2023-09-30T22:14:17.793 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:14:17.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.911+0000 7fcda8431700 1 Processor -- start 2023-09-30T22:14:17.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.911+0000 7fcda8431700 1 -- start start 2023-09-30T22:14:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda8431700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcda00926d0 0x7fcda01275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda8431700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda012b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda8431700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda0091c80 con 0x7fcda0127be0 2023-09-30T22:14:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda8431700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda0099dd0 con 0x7fcda00926d0 2023-09-30T22:14:17.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda8431700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00999d0 con 0x7fcda0128000 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda69ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda012b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda61cd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcda00926d0 0x7fcda01275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda69ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda012b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34284/0 (socket says 172.21.15.89:34284) 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda59cc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcda0127be0 0x7fcda012a9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54348/0 (socket says 172.21.15.89:54348) 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda69ce700 1 -- 172.21.15.89:0/1300343961 learned_addr learned my addr 172.21.15.89:0/1300343961 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.912+0000 7fcda61cd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcda00926d0 0x7fcda01275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37000/0 (socket says 172.21.15.89:37000) 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda69ce700 1 -- 172.21.15.89:0/1300343961 >> v2:172.21.15.89:3300/0 conn(0x7fcda00926d0 msgr2=0x7fcda01275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda69ce700 1 --2- 172.21.15.89:0/1300343961 >> v2:172.21.15.89:3300/0 conn(0x7fcda00926d0 0x7fcda01275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda69ce700 1 -- 172.21.15.89:0/1300343961 >> v1:172.21.15.89:6789/0 conn(0x7fcda0127be0 legacy=0x7fcda012a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda69ce700 1 -- 172.21.15.89:0/1300343961 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda009a280 con 0x7fcda0128000 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda51cb700 1 -- 172.21.15.89:0/1300343961 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177109769 0 0) 0x7fcda0091c80 con 0x7fcda0127be0 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda61cd700 1 --2- 172.21.15.89:0/1300343961 >> v2:172.21.15.89:3300/0 conn(0x7fcda00926d0 0x7fcda01275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda69ce700 1 --2- 172.21.15.89:0/1300343961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda012b0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fcd98009f20 tx=0x7fcd98009840).ready entity=mon.2 client_cookie=7b68b89c42bf3992 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda51cb700 1 -- 172.21.15.89:0/1300343961 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9800e070 con 0x7fcda0128000 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda51cb700 1 -- 172.21.15.89:0/1300343961 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9800b9d0 con 0x7fcda0128000 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda51cb700 1 -- 172.21.15.89:0/1300343961 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9800bd50 con 0x7fcda0128000 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda8431700 1 -- 172.21.15.89:0/1300343961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 msgr2=0x7fcda012b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda8431700 1 --2- 172.21.15.89:0/1300343961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda012b0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fcd98009f20 tx=0x7fcd98009840).stop 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda8431700 1 -- 172.21.15.89:0/1300343961 shutdown_connections 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda8431700 1 --2- 172.21.15.89:0/1300343961 >> v2:172.21.15.89:3300/0 conn(0x7fcda00926d0 0x7fcda01275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda8431700 1 --2- 172.21.15.89:0/1300343961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda012b0b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda8431700 1 -- 172.21.15.89:0/1300343961 >> 172.21.15.89:0/1300343961 conn(0x7fcda0097480 msgr2=0x7fcda0097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda8431700 1 -- 172.21.15.89:0/1300343961 shutdown_connections 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.913+0000 7fcda8431700 1 -- 172.21.15.89:0/1300343961 wait complete. 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.914+0000 7fcda8431700 1 Processor -- start 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.914+0000 7fcda8431700 1 -- start start 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda8431700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda00926d0 0x7fcda0117b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda8431700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda0127be0 0x7fcda0113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda8431700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda0114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda8431700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda0117670 con 0x7fcda00926d0 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda8431700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda0094570 con 0x7fcda0127be0 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda8431700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcda009a4b0 con 0x7fcda0128000 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda00926d0 0x7fcda0117b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda00926d0 0x7fcda0117b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37012/0 (socket says 172.21.15.89:37012) 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 -- 172.21.15.89:0/3898376089 learned_addr learned my addr 172.21.15.89:0/3898376089 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda69ce700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda0114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda59cc700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda0127be0 0x7fcda0113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:17.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 -- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 msgr2=0x7fcda0114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda0114050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 -- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda0127be0 msgr2=0x7fcda0113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda0127be0 0x7fcda0113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 -- 172.21.15.89:0/3898376089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd900092a0 con 0x7fcda00926d0 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda69ce700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda0114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda61cd700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda00926d0 0x7fcda0117b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fcd9000e9b0 tx=0x7fcd90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcd967fc700 1 -- 172.21.15.89:0/3898376089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90009560 con 0x7fcda00926d0 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcd967fc700 1 -- 172.21.15.89:0/3898376089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9000c670 con 0x7fcda00926d0 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd98009ae0 con 0x7fcda00926d0 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcd967fc700 1 -- 172.21.15.89:0/3898376089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000c9f0 con 0x7fcda00926d0 2023-09-30T22:14:17.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.915+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda0117060 con 0x7fcda00926d0 2023-09-30T22:14:17.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.916+0000 7fcd967fc700 1 -- 172.21.15.89:0/3898376089 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fcd9000cd60 con 0x7fcda00926d0 2023-09-30T22:14:17.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.916+0000 7fcd967fc700 1 -- 172.21.15.89:0/3898376089 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fcd9003b3e0 con 0x7fcda00926d0 2023-09-30T22:14:17.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.916+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda0091e30 con 0x7fcda00926d0 2023-09-30T22:14:17.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:17.921+0000 7fcd967fc700 1 -- 172.21.15.89:0/3898376089 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fcd90016030 con 0x7fcda00926d0 2023-09-30T22:14:18.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.081+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcda0117390 con 0x7fcda00926d0 2023-09-30T22:14:18.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.081+0000 7fcd967fc700 1 -- 172.21.15.89:0/3898376089 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd90019310 con 0x7fcda00926d0 2023-09-30T22:14:18.081 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-09-30T22:14:18.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.083+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 >> v1:172.21.15.89:6816/118161 conn(0x7fcd8402bb60 legacy=0x7fcd8402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.083+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda00926d0 msgr2=0x7fcda0117b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.083+0000 7fcda8431700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda00926d0 0x7fcda0117b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fcd9000e9b0 tx=0x7fcd90009060).stop 2023-09-30T22:14:18.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.084+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 shutdown_connections 2023-09-30T22:14:18.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.084+0000 7fcda8431700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda0128000 0x7fcda0114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.084+0000 7fcda8431700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda00926d0 0x7fcda0117b00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.084+0000 7fcda8431700 1 --2- 172.21.15.89:0/3898376089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda0127be0 0x7fcda0113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.084+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 >> 172.21.15.89:0/3898376089 conn(0x7fcda0097480 msgr2=0x7fcda0126a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:18.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.084+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 shutdown_connections 2023-09-30T22:14:18.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.084+0000 7fcda8431700 1 -- 172.21.15.89:0/3898376089 wait complete. 2023-09-30T22:14:18.097 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:14:18.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.216+0000 7f8881543700 1 Processor -- start 2023-09-30T22:14:18.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f8881543700 1 -- start start 2023-09-30T22:14:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f8881543700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f8881543700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c098f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f8881543700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0663b0 con 0x7f887c12f430 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f8881543700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c065e80 con 0x7f887c0653a0 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f8881543700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f887c065780 con 0x7f887c098b30 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c098f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c098640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52386/0 (socket says 172.21.15.89:52386) 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c098f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34296/0 (socket says 172.21.15.89:34296) 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887affd700 1 -- 172.21.15.89:0/1392202946 learned_addr learned my addr 172.21.15.89:0/1392202946 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887b7fe700 1 --1- 172.21.15.89:0/1392202946 >> v1:172.21.15.89:6789/0 conn(0x7f887c12f430 0x7f887c131820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54362/0 (socket says 172.21.15.89:54362) 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887a7fc700 1 -- 172.21.15.89:0/1392202946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 msgr2=0x7f887c098640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887a7fc700 1 --2- 172.21.15.89:0/1392202946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c098640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f8879ffb700 1 -- 172.21.15.89:0/1392202946 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609469203 0 0) 0x7f887c0663b0 con 0x7f887c12f430 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887a7fc700 1 -- 172.21.15.89:0/1392202946 >> v1:172.21.15.89:6789/0 conn(0x7f887c12f430 legacy=0x7f887c131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887a7fc700 1 -- 172.21.15.89:0/1392202946 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c096180 con 0x7f887c098b30 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.217+0000 7f887affd700 1 --2- 172.21.15.89:0/1392202946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f887a7fc700 1 --2- 172.21.15.89:0/1392202946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c098f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f886c0097c0 tx=0x7f886c009060).ready entity=mon.2 client_cookie=bea369d6ba686de3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8879ffb700 1 -- 172.21.15.89:0/1392202946 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00f510 con 0x7f887c098b30 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8879ffb700 1 -- 172.21.15.89:0/1392202946 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c00f670 con 0x7f887c098b30 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8879ffb700 1 -- 172.21.15.89:0/1392202946 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00f9f0 con 0x7f887c098b30 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8881543700 1 -- 172.21.15.89:0/1392202946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 msgr2=0x7f887c098f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8881543700 1 --2- 172.21.15.89:0/1392202946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c098f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f886c0097c0 tx=0x7f886c009060).stop 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8881543700 1 -- 172.21.15.89:0/1392202946 shutdown_connections 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8881543700 1 --2- 172.21.15.89:0/1392202946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8881543700 1 --2- 172.21.15.89:0/1392202946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c098f10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8881543700 1 -- 172.21.15.89:0/1392202946 >> 172.21.15.89:0/1392202946 conn(0x7f887c09dc20 msgr2=0x7f887c09e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8881543700 1 -- 172.21.15.89:0/1392202946 shutdown_connections 2023-09-30T22:14:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.218+0000 7f8881543700 1 -- 172.21.15.89:0/1392202946 wait complete. 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.219+0000 7f8881543700 1 Processor -- start 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.219+0000 7f8881543700 1 -- start start 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8881543700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c11e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8881543700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c11a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8881543700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12f430 0x7f887c11a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8881543700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c11c9c0 con 0x7f887c12f430 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8881543700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c099a90 con 0x7f887c0653a0 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8881543700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f887c11d890 con 0x7f887c098b30 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c11e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c11a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c11e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52402/0 (socket says 172.21.15.89:52402) 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887affd700 1 -- 172.21.15.89:0/133130687 learned_addr learned my addr 172.21.15.89:0/133130687 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887b7fe700 1 --2- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12f430 0x7f887c11a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887a7fc700 1 -- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 msgr2=0x7f887c11e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887a7fc700 1 --2- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c11e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887a7fc700 1 -- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12f430 msgr2=0x7f887c11a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887a7fc700 1 --2- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12f430 0x7f887c11a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887a7fc700 1 -- 172.21.15.89:0/133130687 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870009370 con 0x7f887c098b30 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f887a7fc700 1 --2- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c11a300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f887c063be0 tx=0x7f886c017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8862ffd700 1 -- 172.21.15.89:0/133130687 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00f7e0 con 0x7f887c098b30 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8862ffd700 1 -- 172.21.15.89:0/133130687 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c00f940 con 0x7f887c098b30 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8862ffd700 1 -- 172.21.15.89:0/133130687 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c01b660 con 0x7f887c098b30 2023-09-30T22:14:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886c0092a0 con 0x7f887c098b30 2023-09-30T22:14:18.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.220+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c11bf20 con 0x7f887c098b30 2023-09-30T22:14:18.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.221+0000 7f8862ffd700 1 -- 172.21.15.89:0/133130687 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f886c01ba50 con 0x7f887c098b30 2023-09-30T22:14:18.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.221+0000 7f8862ffd700 1 -- 172.21.15.89:0/133130687 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f886c03c860 con 0x7f887c098b30 2023-09-30T22:14:18.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.222+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c11c110 con 0x7f887c098b30 2023-09-30T22:14:18.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.226+0000 7f8862ffd700 1 -- 172.21.15.89:0/133130687 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f886c029080 con 0x7f887c098b30 2023-09-30T22:14:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.381+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f887c11b100 con 0x7f887c098b30 2023-09-30T22:14:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.382+0000 7f8862ffd700 1 -- 172.21.15.89:0/133130687 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f886c01f030 con 0x7f887c098b30 2023-09-30T22:14:18.381 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-09-30T22:14:18.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 >> v1:172.21.15.89:6816/118161 conn(0x7f886802bb60 legacy=0x7f886802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 msgr2=0x7f887c11a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 --2- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c11a300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f887c063be0 tx=0x7f886c017000).stop 2023-09-30T22:14:18.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 shutdown_connections 2023-09-30T22:14:18.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 --2- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c12f430 0x7f887c11a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 --2- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887c0653a0 0x7f887c11e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 --2- 172.21.15.89:0/133130687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887c098b30 0x7f887c11a300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 >> 172.21.15.89:0/133130687 conn(0x7f887c09dc20 msgr2=0x7f887c09bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 shutdown_connections 2023-09-30T22:14:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.383+0000 7f8881543700 1 -- 172.21.15.89:0/133130687 wait complete. 2023-09-30T22:14:18.395 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:14:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d5aa84700 1 Processor -- start 2023-09-30T22:14:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d5aa84700 1 -- start start 2023-09-30T22:14:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d5aa84700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d54096610 0x7f9d540969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d5aa84700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d540684f0 0x7f9d5412fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d5aa84700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d54069ad0 con 0x7f9d54096ee0 2023-09-30T22:14:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d5aa84700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d54093420 con 0x7f9d54096610 2023-09-30T22:14:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d5aa84700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54091ef0 con 0x7f9d540684f0 2023-09-30T22:14:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d59021700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d540684f0 0x7f9d5412fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d58820700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d54096610 0x7f9d540969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d59021700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d540684f0 0x7f9d5412fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52410/0 (socket says 172.21.15.89:52410) 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d59021700 1 -- 172.21.15.89:0/2898260297 learned_addr learned my addr 172.21.15.89:0/2898260297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d58820700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d54096610 0x7f9d540969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37028/0 (socket says 172.21.15.89:37028) 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.510+0000 7f9d53fff700 1 --1- 172.21.15.89:0/2898260297 >> v1:172.21.15.89:6789/0 conn(0x7f9d54096ee0 0x7f9d54067de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54366/0 (socket says 172.21.15.89:54366) 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d59021700 1 -- 172.21.15.89:0/2898260297 >> v2:172.21.15.89:3300/0 conn(0x7f9d54096610 msgr2=0x7f9d540969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d59021700 1 --2- 172.21.15.89:0/2898260297 >> v2:172.21.15.89:3300/0 conn(0x7f9d54096610 0x7f9d540969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d59021700 1 -- 172.21.15.89:0/2898260297 >> v1:172.21.15.89:6789/0 conn(0x7f9d54096ee0 legacy=0x7f9d54067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d59021700 1 -- 172.21.15.89:0/2898260297 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d54092450 con 0x7f9d540684f0 2023-09-30T22:14:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d537fe700 1 -- 172.21.15.89:0/2898260297 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258232064 0 0) 0x7f9d54069ad0 con 0x7f9d54096ee0 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d58820700 1 --2- 172.21.15.89:0/2898260297 >> v2:172.21.15.89:3300/0 conn(0x7f9d54096610 0x7f9d540969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d59021700 1 --2- 172.21.15.89:0/2898260297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d540684f0 0x7f9d5412fc50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9d4c00d1a0 tx=0x7f9d4c00ba50).ready entity=mon.1 client_cookie=813c26fe23598f87 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d537fe700 1 -- 172.21.15.89:0/2898260297 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c010070 con 0x7f9d540684f0 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d537fe700 1 -- 172.21.15.89:0/2898260297 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c00de10 con 0x7f9d540684f0 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d537fe700 1 -- 172.21.15.89:0/2898260297 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00b2e0 con 0x7f9d540684f0 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2898260297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d540684f0 msgr2=0x7f9d5412fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.511+0000 7f9d5aa84700 1 --2- 172.21.15.89:0/2898260297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d540684f0 0x7f9d5412fc50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9d4c00d1a0 tx=0x7f9d4c00ba50).stop 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.512+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2898260297 shutdown_connections 2023-09-30T22:14:18.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.512+0000 7f9d5aa84700 1 --2- 172.21.15.89:0/2898260297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d540684f0 0x7f9d5412fc50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.512+0000 7f9d5aa84700 1 --2- 172.21.15.89:0/2898260297 >> v2:172.21.15.89:3300/0 conn(0x7f9d54096610 0x7f9d540969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.512+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2898260297 >> 172.21.15.89:0/2898260297 conn(0x7f9d5409a220 msgr2=0x7f9d5409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.512+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2898260297 shutdown_connections 2023-09-30T22:14:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.512+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2898260297 wait complete. 2023-09-30T22:14:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.512+0000 7f9d5aa84700 1 Processor -- start 2023-09-30T22:14:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d5aa84700 1 -- start start 2023-09-30T22:14:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d5aa84700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d540684f0 0x7f9d541358f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d5aa84700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54096610 0x7f9d54135de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d5aa84700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d54096ee0 0x7f9d541362d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d5aa84700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54119660 con 0x7f9d540684f0 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d5aa84700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54091bf0 con 0x7f9d54096ee0 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d5aa84700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54066320 con 0x7f9d54096610 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d540684f0 0x7f9d541358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d540684f0 0x7f9d541358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37038/0 (socket says 172.21.15.89:37038) 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 -- 172.21.15.89:0/2594373169 learned_addr learned my addr 172.21.15.89:0/2594373169 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54096610 0x7f9d54135de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d59021700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d54096ee0 0x7f9d541362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 -- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54096610 msgr2=0x7f9d54135de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54096610 0x7f9d54135de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 -- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d54096ee0 msgr2=0x7f9d541362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d54096ee0 0x7f9d541362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 -- 172.21.15.89:0/2594373169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480092a0 con 0x7f9d540684f0 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d53fff700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54096610 0x7f9d54135de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.513+0000 7f9d58820700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d540684f0 0x7f9d541358f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f9d4800ea50 tx=0x7f9d48009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.514+0000 7f9d50ff9700 1 -- 172.21.15.89:0/2594373169 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800ee90 con 0x7f9d540684f0 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.514+0000 7f9d50ff9700 1 -- 172.21.15.89:0/2594373169 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4800c850 con 0x7f9d540684f0 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.514+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c00bcf0 con 0x7f9d540684f0 2023-09-30T22:14:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.514+0000 7f9d50ff9700 1 -- 172.21.15.89:0/2594373169 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800cbd0 con 0x7f9d540684f0 2023-09-30T22:14:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.514+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d54119e70 con 0x7f9d540684f0 2023-09-30T22:14:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.514+0000 7f9d50ff9700 1 -- 172.21.15.89:0/2594373169 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9d4801b400 con 0x7f9d540684f0 2023-09-30T22:14:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.515+0000 7f9d50ff9700 1 -- 172.21.15.89:0/2594373169 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f9d4803b310 con 0x7f9d540684f0 2023-09-30T22:14:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.515+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d54119e70 con 0x7f9d540684f0 2023-09-30T22:14:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.519+0000 7f9d50ff9700 1 -- 172.21.15.89:0/2594373169 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9d54119e70 con 0x7f9d540684f0 2023-09-30T22:14:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.713+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d54119e70 con 0x7f9d540684f0 2023-09-30T22:14:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.713+0000 7f9d50ff9700 1 -- 172.21.15.89:0/2594373169 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f9d54119e70 con 0x7f9d540684f0 2023-09-30T22:14:18.713 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:18.713 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:04.044604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:14:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 >> v1:172.21.15.89:6816/118161 conn(0x7f9d3802bb60 legacy=0x7f9d3802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d540684f0 msgr2=0x7f9d541358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d540684f0 0x7f9d541358f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f9d4800ea50 tx=0x7f9d48009060).stop 2023-09-30T22:14:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 shutdown_connections 2023-09-30T22:14:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d54096ee0 0x7f9d541362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d540684f0 0x7f9d541358f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 --2- 172.21.15.89:0/2594373169 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54096610 0x7f9d54135de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 >> 172.21.15.89:0/2594373169 conn(0x7f9d5409a220 msgr2=0x7f9d54099020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:18.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 shutdown_connections 2023-09-30T22:14:18.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.715+0000 7f9d5aa84700 1 -- 172.21.15.89:0/2594373169 wait complete. 2023-09-30T22:14:18.716 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 5 2023-09-30T22:14:18.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a2d9f700 1 Processor -- start 2023-09-30T22:14:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a2d9f700 1 -- start start 2023-09-30T22:14:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a2d9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0965e0 0x7f129c0969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a2d9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f129c096eb0 0x7f129c067de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a2d9f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c0933b0 con 0x7f129c0682d0 2023-09-30T22:14:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a2d9f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c092190 con 0x7f129c096eb0 2023-09-30T22:14:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a2d9f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f129c069a20 con 0x7f129c0965e0 2023-09-30T22:14:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f129bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f129c096eb0 0x7f129c067de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a0b3b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0965e0 0x7f129c0969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f129bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f129c096eb0 0x7f129c067de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37046/0 (socket says 172.21.15.89:37046) 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a133c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f129c0682d0 0x7f129c12ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54382/0 (socket says 172.21.15.89:54382) 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f12a0b3b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0965e0 0x7f129c0969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34324/0 (socket says 172.21.15.89:34324) 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.842+0000 7f129bfff700 1 -- 172.21.15.89:0/856496158 learned_addr learned my addr 172.21.15.89:0/856496158 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a0b3b700 1 -- 172.21.15.89:0/856496158 >> v2:172.21.15.89:3300/0 conn(0x7f129c096eb0 msgr2=0x7f129c067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a0b3b700 1 --2- 172.21.15.89:0/856496158 >> v2:172.21.15.89:3300/0 conn(0x7f129c096eb0 0x7f129c067de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a0b3b700 1 -- 172.21.15.89:0/856496158 >> v1:172.21.15.89:6789/0 conn(0x7f129c0682d0 legacy=0x7f129c12ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a0b3b700 1 -- 172.21.15.89:0/856496158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c069ca0 con 0x7f129c0965e0 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f129b7fe700 1 -- 172.21.15.89:0/856496158 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686610402 0 0) 0x7f129c0933b0 con 0x7f129c0682d0 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f129bfff700 1 --2- 172.21.15.89:0/856496158 >> v2:172.21.15.89:3300/0 conn(0x7f129c096eb0 0x7f129c067de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a0b3b700 1 --2- 172.21.15.89:0/856496158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0965e0 0x7f129c0969c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1290009760 tx=0x7f1290009060).ready entity=mon.2 client_cookie=faaf8f26cd04298d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f129b7fe700 1 -- 172.21.15.89:0/856496158 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000f510 con 0x7f129c0965e0 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f129b7fe700 1 -- 172.21.15.89:0/856496158 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129000f670 con 0x7f129c0965e0 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f129b7fe700 1 -- 172.21.15.89:0/856496158 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000f9f0 con 0x7f129c0965e0 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a2d9f700 1 -- 172.21.15.89:0/856496158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0965e0 msgr2=0x7f129c0969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a2d9f700 1 --2- 172.21.15.89:0/856496158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0965e0 0x7f129c0969c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1290009760 tx=0x7f1290009060).stop 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a2d9f700 1 -- 172.21.15.89:0/856496158 shutdown_connections 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a2d9f700 1 --2- 172.21.15.89:0/856496158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0965e0 0x7f129c0969c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a2d9f700 1 --2- 172.21.15.89:0/856496158 >> v2:172.21.15.89:3300/0 conn(0x7f129c096eb0 0x7f129c067de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.843+0000 7f12a2d9f700 1 -- 172.21.15.89:0/856496158 >> 172.21.15.89:0/856496158 conn(0x7f129c09a220 msgr2=0x7f129c09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.844+0000 7f12a2d9f700 1 -- 172.21.15.89:0/856496158 shutdown_connections 2023-09-30T22:14:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.844+0000 7f12a2d9f700 1 -- 172.21.15.89:0/856496158 wait complete. 2023-09-30T22:14:18.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.844+0000 7f12a2d9f700 1 Processor -- start 2023-09-30T22:14:18.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a2d9f700 1 -- start start 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a2d9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0682d0 0x7f129c115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a2d9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f129c0965e0 0x7f129c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a2d9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f129c096eb0 0x7f129c1165b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a2d9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c116aa0 con 0x7f129c0965e0 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a2d9f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c066740 con 0x7f129c096eb0 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a2d9f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f129c091bf0 con 0x7f129c0682d0 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a0b3b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0682d0 0x7f129c115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f129bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f129c0965e0 0x7f129c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a0b3b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0682d0 0x7f129c115bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34326/0 (socket says 172.21.15.89:34326) 2023-09-30T22:14:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a0b3b700 1 -- 172.21.15.89:0/3047342783 learned_addr learned my addr 172.21.15.89:0/3047342783 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a133c700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f129c096eb0 0x7f129c1165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f129bfff700 1 -- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0682d0 msgr2=0x7f129c115bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f129bfff700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0682d0 0x7f129c115bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f129bfff700 1 -- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f129c096eb0 msgr2=0x7f129c1165b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f129bfff700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f129c096eb0 0x7f129c1165b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f129bfff700 1 -- 172.21.15.89:0/3047342783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1288009370 con 0x7f129c0965e0 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f12a0b3b700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0682d0 0x7f129c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.845+0000 7f129bfff700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f129c0965e0 0x7f129c1160c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f128800ea30 tx=0x7f12880090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.846+0000 7f1298ff9700 1 -- 172.21.15.89:0/3047342783 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128800ee70 con 0x7f129c0965e0 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.846+0000 7f1298ff9700 1 -- 172.21.15.89:0/3047342783 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128800c850 con 0x7f129c0965e0 2023-09-30T22:14:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.846+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12900092a0 con 0x7f129c0965e0 2023-09-30T22:14:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.846+0000 7f1298ff9700 1 -- 172.21.15.89:0/3047342783 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128800cbd0 con 0x7f129c0965e0 2023-09-30T22:14:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.846+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c117870 con 0x7f129c0965e0 2023-09-30T22:14:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.846+0000 7f1298ff9700 1 -- 172.21.15.89:0/3047342783 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f128801b400 con 0x7f129c0965e0 2023-09-30T22:14:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.846+0000 7f1298ff9700 1 -- 172.21.15.89:0/3047342783 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f128803b540 con 0x7f129c0965e0 2023-09-30T22:14:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.847+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129c05a260 con 0x7f129c0965e0 2023-09-30T22:14:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:18.851+0000 7f1298ff9700 1 -- 172.21.15.89:0/3047342783 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1288019900 con 0x7f129c0965e0 2023-09-30T22:14:19.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.009+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f129c069210 con 0x7f129c0965e0 2023-09-30T22:14:19.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.010+0000 7f1298ff9700 1 -- 172.21.15.89:0/3047342783 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f1288019310 con 0x7f129c0965e0 2023-09-30T22:14:19.010 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:19.010 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:03.041750+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:19.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 >> v1:172.21.15.89:6816/118161 conn(0x7f128002bb60 legacy=0x7f128002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f129c0965e0 msgr2=0x7f129c1160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f129c0965e0 0x7f129c1160c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f128800ea30 tx=0x7f12880090f0).stop 2023-09-30T22:14:19.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 shutdown_connections 2023-09-30T22:14:19.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f129c096eb0 0x7f129c1165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f129c0682d0 0x7f129c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 --2- 172.21.15.89:0/3047342783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f129c0965e0 0x7f129c1160c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 >> 172.21.15.89:0/3047342783 conn(0x7f129c09a220 msgr2=0x7f129c099000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 shutdown_connections 2023-09-30T22:14:19.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.012+0000 7f12a2d9f700 1 -- 172.21.15.89:0/3047342783 wait complete. 2023-09-30T22:14:19.023 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:14:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.140+0000 7fdcab9e6700 1 Processor -- start 2023-09-30T22:14:19.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.140+0000 7fdcab9e6700 1 -- start start 2023-09-30T22:14:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdcab9e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca40613a0 0x7fdca412ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdcab9e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca412fe50 0x7fdca4132240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdcab9e6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca4067530 con 0x7fdca4097cb0 2023-09-30T22:14:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdcab9e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4061780 con 0x7fdca412fe50 2023-09-30T22:14:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdcab9e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4066ee0 con 0x7fdca40613a0 2023-09-30T22:14:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca9f83700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca412fe50 0x7fdca4132240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca9782700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca40613a0 0x7fdca412ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca8f81700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdca4097cb0 0x7fdca412b170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54384/0 (socket says 172.21.15.89:54384) 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca8f81700 1 -- 172.21.15.89:0/92253838 learned_addr learned my addr 172.21.15.89:0/92253838 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca9f83700 1 -- 172.21.15.89:0/92253838 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca40613a0 msgr2=0x7fdca412ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca9f83700 1 --2- 172.21.15.89:0/92253838 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca40613a0 0x7fdca412ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdc9bfff700 1 -- 172.21.15.89:0/92253838 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236550382 0 0) 0x7fdca4067530 con 0x7fdca4097cb0 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca9f83700 1 -- 172.21.15.89:0/92253838 >> v1:172.21.15.89:6789/0 conn(0x7fdca4097cb0 legacy=0x7fdca412b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca9f83700 1 -- 172.21.15.89:0/92253838 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca412b880 con 0x7fdca412fe50 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca9782700 1 --2- 172.21.15.89:0/92253838 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca40613a0 0x7fdca412ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdca9f83700 1 --2- 172.21.15.89:0/92253838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca412fe50 0x7fdca4132240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fdc9c009e90 tx=0x7fdc9c009780).ready entity=mon.1 client_cookie=7063e6239ba772f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdc9bfff700 1 -- 172.21.15.89:0/92253838 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00e070 con 0x7fdca412fe50 2023-09-30T22:14:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdc9bfff700 1 -- 172.21.15.89:0/92253838 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c00bb60 con 0x7fdca412fe50 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.141+0000 7fdc9bfff700 1 -- 172.21.15.89:0/92253838 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c012410 con 0x7fdca412fe50 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.142+0000 7fdcab9e6700 1 -- 172.21.15.89:0/92253838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca412fe50 msgr2=0x7fdca4132240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.142+0000 7fdcab9e6700 1 --2- 172.21.15.89:0/92253838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca412fe50 0x7fdca4132240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fdc9c009e90 tx=0x7fdc9c009780).stop 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.142+0000 7fdcab9e6700 1 -- 172.21.15.89:0/92253838 shutdown_connections 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.142+0000 7fdcab9e6700 1 --2- 172.21.15.89:0/92253838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca412fe50 0x7fdca4132240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.142+0000 7fdcab9e6700 1 --2- 172.21.15.89:0/92253838 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca40613a0 0x7fdca412ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.142+0000 7fdcab9e6700 1 -- 172.21.15.89:0/92253838 >> 172.21.15.89:0/92253838 conn(0x7fdca409a220 msgr2=0x7fdca409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.142+0000 7fdcab9e6700 1 -- 172.21.15.89:0/92253838 shutdown_connections 2023-09-30T22:14:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.142+0000 7fdcab9e6700 1 -- 172.21.15.89:0/92253838 wait complete. 2023-09-30T22:14:19.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.143+0000 7fdcab9e6700 1 Processor -- start 2023-09-30T22:14:19.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.143+0000 7fdcab9e6700 1 -- start start 2023-09-30T22:14:19.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdcab9e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdca40613a0 0x7fdca4112440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdcab9e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca4097cb0 0x7fdca4112930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdcab9e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca412fe50 0x7fdca4112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdcab9e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca412a4f0 con 0x7fdca40613a0 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdcab9e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca41149e0 con 0x7fdca4097cb0 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdcab9e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdca41323f0 con 0x7fdca412fe50 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca9782700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdca40613a0 0x7fdca4112440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca9f83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca412fe50 0x7fdca4112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca8f81700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca4097cb0 0x7fdca4112930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca9f83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca412fe50 0x7fdca4112e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34356/0 (socket says 172.21.15.89:34356) 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca9782700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdca40613a0 0x7fdca4112440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37062/0 (socket says 172.21.15.89:37062) 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca9f83700 1 -- 172.21.15.89:0/494164076 learned_addr learned my addr 172.21.15.89:0/494164076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca8f81700 1 -- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca412fe50 msgr2=0x7fdca4112e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca8f81700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca412fe50 0x7fdca4112e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca8f81700 1 -- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdca40613a0 msgr2=0x7fdca4112440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca8f81700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdca40613a0 0x7fdca4112440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca8f81700 1 -- 172.21.15.89:0/494164076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940092a0 con 0x7fdca4097cb0 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca9782700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdca40613a0 0x7fdca4112440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca9f83700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca412fe50 0x7fdca4112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.144+0000 7fdca8f81700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca4097cb0 0x7fdca4112930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fdca0000c00 tx=0x7fdca0007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.145+0000 7fdc997fa700 1 -- 172.21.15.89:0/494164076 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00070d0 con 0x7fdca4097cb0 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.145+0000 7fdc997fa700 1 -- 172.21.15.89:0/494164076 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0007250 con 0x7fdca4097cb0 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.145+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c009a20 con 0x7fdca4097cb0 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.145+0000 7fdc997fa700 1 -- 172.21.15.89:0/494164076 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0007610 con 0x7fdca4097cb0 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.145+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca41166f0 con 0x7fdca4097cb0 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.145+0000 7fdc997fa700 1 -- 172.21.15.89:0/494164076 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fdca00059b0 con 0x7fdca4097cb0 2023-09-30T22:14:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.145+0000 7fdc997fa700 1 -- 172.21.15.89:0/494164076 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fdca003a2c0 con 0x7fdca4097cb0 2023-09-30T22:14:19.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.146+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca41166f0 con 0x7fdca4097cb0 2023-09-30T22:14:19.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.150+0000 7fdc997fa700 1 -- 172.21.15.89:0/494164076 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fdca41166f0 con 0x7fdca4097cb0 2023-09-30T22:14:19.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.344+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdca41166f0 con 0x7fdca4097cb0 2023-09-30T22:14:19.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.344+0000 7fdc997fa700 1 -- 172.21.15.89:0/494164076 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2156 (secure 0 0 0) 0x7fdca41166f0 con 0x7fdca4097cb0 2023-09-30T22:14:19.344 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:19.344 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:04.044604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:14:19.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 >> v1:172.21.15.89:6816/118161 conn(0x7fdc8802bb90 legacy=0x7fdc8802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca4097cb0 msgr2=0x7fdca4112930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca4097cb0 0x7fdca4112930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fdca0000c00 tx=0x7fdca0007b90).stop 2023-09-30T22:14:19.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 shutdown_connections 2023-09-30T22:14:19.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdca412fe50 0x7fdca4112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdca40613a0 0x7fdca4112440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 --2- 172.21.15.89:0/494164076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdca4097cb0 0x7fdca4112930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 >> 172.21.15.89:0/494164076 conn(0x7fdca409a220 msgr2=0x7fdca4131990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 shutdown_connections 2023-09-30T22:14:19.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.346+0000 7fdcab9e6700 1 -- 172.21.15.89:0/494164076 wait complete. 2023-09-30T22:14:19.346 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 5 2023-09-30T22:14:19.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:14:02.037584+0000', 'modified': '2023-09-30T22:14:04.044604+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4190': {'gid': 4190, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6817/3539463463', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.89:6817', 'nonce': 3539463463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-30T22:14:19.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:14:19.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:14:19.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.574+0000 7f5f340ce700 1 Processor -- start 2023-09-30T22:14:19.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.574+0000 7f5f340ce700 1 -- start start 2023-09-30T22:14:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f340ce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c0970d0 0x7f5f2c12cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f340ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c12d120 0x7f5f2c12f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f340ce700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0919e0 con 0x7f5f2c0965e0 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f340ce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0933b0 con 0x7f5f2c0970d0 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f340ce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c092190 con 0x7f5f2c12d120 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f31669700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c0970d0 0x7f5f2c12cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f3266b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c12d120 0x7f5f2c12f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f31e6a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5f2c0965e0 0x7f5f2c0969c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54386/0 (socket says 172.21.15.89:54386) 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f31e6a700 1 -- 172.21.15.89:0/1262905141 learned_addr learned my addr 172.21.15.89:0/1262905141 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f30e68700 1 -- 172.21.15.89:0/1262905141 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251400214 0 0) 0x7f5f2c0919e0 con 0x7f5f2c0965e0 2023-09-30T22:14:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f3266b700 1 -- 172.21.15.89:0/1262905141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c0970d0 msgr2=0x7f5f2c12cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f3266b700 1 --2- 172.21.15.89:0/1262905141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c0970d0 0x7f5f2c12cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f3266b700 1 -- 172.21.15.89:0/1262905141 >> v1:172.21.15.89:6789/0 conn(0x7f5f2c0965e0 legacy=0x7f5f2c0969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f3266b700 1 -- 172.21.15.89:0/1262905141 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c069a20 con 0x7f5f2c12d120 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f31669700 1 --2- 172.21.15.89:0/1262905141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c0970d0 0x7f5f2c12cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.575+0000 7f5f3266b700 1 --2- 172.21.15.89:0/1262905141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c12d120 0x7f5f2c12f510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5f24009e90 tx=0x7f5f24009780).ready entity=mon.2 client_cookie=9358bc09ffe1db93 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f30e68700 1 -- 172.21.15.89:0/1262905141 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400e070 con 0x7f5f2c12d120 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f30e68700 1 -- 172.21.15.89:0/1262905141 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2400bb60 con 0x7f5f2c12d120 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f30e68700 1 -- 172.21.15.89:0/1262905141 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24012410 con 0x7f5f2c12d120 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f340ce700 1 -- 172.21.15.89:0/1262905141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c12d120 msgr2=0x7f5f2c12f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f340ce700 1 --2- 172.21.15.89:0/1262905141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c12d120 0x7f5f2c12f510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5f24009e90 tx=0x7f5f24009780).stop 2023-09-30T22:14:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f340ce700 1 -- 172.21.15.89:0/1262905141 shutdown_connections 2023-09-30T22:14:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f340ce700 1 --2- 172.21.15.89:0/1262905141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c12d120 0x7f5f2c12f510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f340ce700 1 --2- 172.21.15.89:0/1262905141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c0970d0 0x7f5f2c12cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f340ce700 1 -- 172.21.15.89:0/1262905141 >> 172.21.15.89:0/1262905141 conn(0x7f5f2c09a220 msgr2=0x7f5f2c09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f340ce700 1 -- 172.21.15.89:0/1262905141 shutdown_connections 2023-09-30T22:14:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.576+0000 7f5f340ce700 1 -- 172.21.15.89:0/1262905141 wait complete. 2023-09-30T22:14:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.577+0000 7f5f340ce700 1 Processor -- start 2023-09-30T22:14:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.577+0000 7f5f340ce700 1 -- start start 2023-09-30T22:14:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.577+0000 7f5f340ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2c0965e0 0x7f5f2c066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.577+0000 7f5f340ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c0970d0 0x7f5f2c12b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.577+0000 7f5f340ce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c12d120 0x7f5f2c12bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.577+0000 7f5f340ce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c066590 con 0x7f5f2c0965e0 2023-09-30T22:14:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.577+0000 7f5f340ce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c091bf0 con 0x7f5f2c12d120 2023-09-30T22:14:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.577+0000 7f5f340ce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c117820 con 0x7f5f2c0970d0 2023-09-30T22:14:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2c0965e0 0x7f5f2c066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31669700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c0970d0 0x7f5f2c12b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2c0965e0 0x7f5f2c066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37068/0 (socket says 172.21.15.89:37068) 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31e6a700 1 -- 172.21.15.89:0/930798742 learned_addr learned my addr 172.21.15.89:0/930798742 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31669700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c0970d0 0x7f5f2c12b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34384/0 (socket says 172.21.15.89:34384) 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f3266b700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c12d120 0x7f5f2c12bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31669700 1 -- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c12d120 msgr2=0x7f5f2c12bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31669700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c12d120 0x7f5f2c12bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31669700 1 -- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2c0965e0 msgr2=0x7f5f2c066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31669700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2c0965e0 0x7f5f2c066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31669700 1 -- 172.21.15.89:0/930798742 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280092a0 con 0x7f5f2c0970d0 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31e6a700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2c0965e0 0x7f5f2c066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f31669700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c0970d0 0x7f5f2c12b6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5f2800e980 tx=0x7f5f28009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f21ffb700 1 -- 172.21.15.89:0/930798742 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28009580 con 0x7f5f2c0970d0 2023-09-30T22:14:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f21ffb700 1 -- 172.21.15.89:0/930798742 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28012600 con 0x7f5f2c0970d0 2023-09-30T22:14:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f21ffb700 1 -- 172.21.15.89:0/930798742 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28012980 con 0x7f5f2c0970d0 2023-09-30T22:14:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f340ce700 1 -- 172.21.15.89:0/930798742 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f24009a20 con 0x7f5f2c0970d0 2023-09-30T22:14:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.578+0000 7f5f340ce700 1 -- 172.21.15.89:0/930798742 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c0677a0 con 0x7f5f2c0970d0 2023-09-30T22:14:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.579+0000 7f5f21ffb700 1 -- 172.21.15.89:0/930798742 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5f28012cf0 con 0x7f5f2c0970d0 2023-09-30T22:14:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.579+0000 7f5f21ffb700 1 -- 172.21.15.89:0/930798742 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5f2803b350 con 0x7f5f2c0970d0 2023-09-30T22:14:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f45a81b80 1 Processor -- start 2023-09-30T22:14:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f45a81b80 1 -- start start 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927170 0x556694927550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566948f1610 con 0x556694927170 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x556694867db0 con 0x556694927a40 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55669487d030 con 0x5566949515b0 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927170 0x556694927550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f3266b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927170 0x556694927550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37076/0 (socket says 172.21.15.89:37076) 2023-09-30T22:14:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f31e6a700 1 -- 172.21.15.89:0/3255841419 learned_addr learned my addr 172.21.15.89:0/3255841419 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.582+0000 7f5f31669700 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f31e6a700 1 -- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 msgr2=0x5566949539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f31e6a700 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f31e6a700 1 -- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 msgr2=0x5566949510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f31e6a700 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f31e6a700 1 -- 172.21.15.89:0/3255841419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556694718710 con 0x556694927170 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f3266b700 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f31669700 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f31e6a700 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927170 0x556694927550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5f1c011610 tx=0x7f5f1c007c10).ready entity=mon.0 client_cookie=8e787e0a57562c38 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f0effd700 1 -- 172.21.15.89:0/3255841419 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0134e0 con 0x556694927170 2023-09-30T22:14:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f0effd700 1 -- 172.21.15.89:0/3255841419 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c013640 con 0x556694927170 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.583+0000 7f5f0effd700 1 -- 172.21.15.89:0/3255841419 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0139c0 con 0x556694927170 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 -- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927170 msgr2=0x556694927550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927170 0x556694927550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5f1c011610 tx=0x7f5f1c007c10).stop 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 -- 172.21.15.89:0/3255841419 shutdown_connections 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927170 0x556694927550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/3255841419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 -- 172.21.15.89:0/3255841419 >> 172.21.15.89:0/3255841419 conn(0x556694795940 msgr2=0x556694947cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 -- 172.21.15.89:0/3255841419 shutdown_connections 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 -- 172.21.15.89:0/3255841419 wait complete. 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 -- start start 2023-09-30T22:14:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.584+0000 7f5f45a81b80 1 Processor -- start 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694956ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927a40 0x556694956fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x5566949574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669487d170 con 0x5566949515b0 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55669487d3f0 con 0x556694927170 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x556694848920 con 0x556694927a40 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f45a81b80 10 client.0 ll_register_callbacks cb 0x55669481dce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f3266b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x5566949574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694956ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f31669700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927a40 0x556694956fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f3266b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x5566949574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37092/0 (socket says 172.21.15.89:37092) 2023-09-30T22:14:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694956ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52462/0 (socket says 172.21.15.89:52462) 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f31669700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927a40 0x556694956fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34394/0 (socket says 172.21.15.89:34394) 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f3266b700 1 -- 172.21.15.89:0/708119898 learned_addr learned my addr 172.21.15.89:0/708119898 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f3266b700 1 -- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927a40 msgr2=0x556694956fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.585+0000 7f5f3266b700 1 --2- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927a40 0x556694956fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f3266b700 1 -- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 msgr2=0x556694956ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f3266b700 1 --2- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694956ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f3266b700 1 -- 172.21.15.89:0/708119898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c0111f0 con 0x5566949515b0 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f3266b700 1 --2- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x5566949574a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f5f2c092320 tx=0x7f5f2c069450).ready entity=mon.0 client_cookie=1b5f3c6c25735a98 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f07fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f07fff700 1 -- 172.21.15.89:0/708119898 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2401e400 con 0x5566949515b0 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f07fff700 1 -- 172.21.15.89:0/708119898 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5f2401e560 con 0x5566949515b0 2023-09-30T22:14:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f45a81b80 1 -- 172.21.15.89:0/708119898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55669491ff20 con 0x5566949515b0 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f07fff700 1 -- 172.21.15.89:0/708119898 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=5) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f2401e930 con 0x5566949515b0 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f45a81b80 10 client.4647 fetch_fsmap learned FSMap version 5 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f45a81b80 1 -- 172.21.15.89:0/708119898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=5,monmap=2+}) v3 -- 0x55669484fb00 con 0x5566949515b0 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f07fff700 1 -- 172.21.15.89:0/708119898 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55669491ff20 con 0x5566949515b0 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f07fff700 1 -- 172.21.15.89:0/708119898 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 5) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f5f2401ea90 con 0x5566949515b0 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f45a81b80 10 client.4647 fetch_fsmap finished waiting for FSMap version 5 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f45a81b80 10 client.4647 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f45a81b80 20 client.4647 populate_metadata read hostname 'smithi089' 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f45a81b80 4 client.4647 mds_command: new command op to 4190 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:14:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f45a81b80 1 -- 172.21.15.89:0/708119898 --> v1:172.21.15.89:6817/3539463463 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566948f5250 con 0x55669494d5d0 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.586+0000 7f5f07fff700 10 client.4647 ms_handle_connect on v1:172.21.15.89:6817/3539463463 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f07fff700 1 -- 172.21.15.89:0/708119898 <== mds.0 v1:172.21.15.89:6817/3539463463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x5566948f5250 con 0x55669494d5d0 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f07fff700 10 client.4647 handle_command_reply: tid=0 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 client.4647 shutdown 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 -- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 msgr2=0x5566949574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x5566949574a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f5f2c092320 tx=0x7f5f2c069450).stop 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 -- 172.21.15.89:0/708119898 shutdown_connections 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x5566949574a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f07fff700 0 client.4647 ms_handle_reset on v1:172.21.15.89:6817/3539463463 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694956ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/708119898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927a40 0x556694956fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 -- 172.21.15.89:0/708119898 >> 172.21.15.89:0/708119898 conn(0x556694795940 msgr2=0x556694947c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 -- 172.21.15.89:0/708119898 shutdown_connections 2023-09-30T22:14:19.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.587+0000 7f5f45a81b80 1 -- 172.21.15.89:0/708119898 wait complete. 2023-09-30T22:14:19.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.588+0000 7f5f45a81b80 20 client.4647 trim_cache size 0 max 16384 2023-09-30T22:14:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f45a81b80 1 Processor -- start 2023-09-30T22:14:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f45a81b80 1 -- start start 2023-09-30T22:14:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927170 0x55669494c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949455a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x55669494da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669492bb80 con 0x5566949515b0 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566949509e0 con 0x556694927a40 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x556694942430 con 0x556694927170 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927170 0x55669494c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927170 0x55669494c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34396/0 (socket says 172.21.15.89:34396) 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 -- 172.21.15.89:0/889462178 learned_addr learned my addr 172.21.15.89:0/889462178 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31669700 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f3266b700 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x55669494da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 -- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 msgr2=0x5566949455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 -- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 msgr2=0x55669494da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x55669494da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 -- 172.21.15.89:0/889462178 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556694875f70 con 0x556694927170 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.604+0000 7f5f31e6a700 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927170 0x55669494c470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f5f1c0079f0 tx=0x7f5f1c018000).ready entity=mon.2 client_cookie=1c3a6f1e8fc15a92 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f0f7fe700 1 -- 172.21.15.89:0/889462178 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0240e0 con 0x556694927170 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f0f7fe700 1 -- 172.21.15.89:0/889462178 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c013db0 con 0x556694927170 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f0f7fe700 1 -- 172.21.15.89:0/889462178 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c001790 con 0x556694927170 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 -- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927170 msgr2=0x55669494c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927170 0x55669494c470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f5f1c0079f0 tx=0x7f5f1c018000).stop 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 -- 172.21.15.89:0/889462178 shutdown_connections 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5566949515b0 0x55669494da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x556694927170 0x55669494c470 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/889462178 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927a40 0x5566949455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 -- 172.21.15.89:0/889462178 >> 172.21.15.89:0/889462178 conn(0x556694795940 msgr2=0x55669492b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 -- 172.21.15.89:0/889462178 shutdown_connections 2023-09-30T22:14:19.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 -- 172.21.15.89:0/889462178 wait complete. 2023-09-30T22:14:19.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.605+0000 7f5f45a81b80 1 -- start start 2023-09-30T22:14:19.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f45a81b80 1 Processor -- start 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694954760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927a40 0x556694954c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f45a81b80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566949571f0 con 0x556694927a40 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x556694957790 con 0x556694927170 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f45a81b80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55669494f420 con 0x5566949515b0 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f45a81b80 10 client.0 ll_register_callbacks cb 0x55669481dce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694954760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f31669700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927a40 0x556694954c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f31e6a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694954760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52484/0 (socket says 172.21.15.89:52484) 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f31e6a700 1 -- 172.21.15.89:0/68575114 learned_addr learned my addr 172.21.15.89:0/68575114 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.606+0000 7f5f3266b700 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f3266b700 1 -- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 msgr2=0x556694954760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f3266b700 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694954760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f3266b700 1 -- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927a40 msgr2=0x556694954c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f3266b700 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927a40 0x556694954c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f3266b700 1 -- 172.21.15.89:0/68575114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556694718710 con 0x5566949515b0 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f31669700 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927a40 0x556694954c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f31e6a700 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694954760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f3266b700 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949abd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f5f2400b6c0 tx=0x7f5f24021040).ready entity=mon.2 client_cookie=e549e2d2967105ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f07fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3302/0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f07fff700 1 -- 172.21.15.89:0/68575114 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24011370 con 0x5566949515b0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f07fff700 1 -- 172.21.15.89:0/68575114 <== mon.2 v2:172.21.15.89:3302/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5f240114d0 con 0x5566949515b0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f45a81b80 1 -- 172.21.15.89:0/68575114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f1c013640 con 0x5566949515b0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f07fff700 1 -- 172.21.15.89:0/68575114 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_get_version_reply(handle=1 version=5) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f24012730 con 0x5566949515b0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f45a81b80 10 client.4508 fetch_fsmap learned FSMap version 5 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f45a81b80 1 -- 172.21.15.89:0/68575114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,fsmap=5,monmap=2+}) v3 -- 0x556694871580 con 0x5566949515b0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f07fff700 1 -- 172.21.15.89:0/68575114 <== mon.2 v2:172.21.15.89:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c013640 con 0x5566949515b0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f07fff700 1 -- 172.21.15.89:0/68575114 <== mon.2 v2:172.21.15.89:3302/0 5 ==== fsmap(e 5) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f5f24012890 con 0x5566949515b0 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f45a81b80 10 client.4508 fetch_fsmap finished waiting for FSMap version 5 2023-09-30T22:14:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f45a81b80 10 client.4508 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f45a81b80 20 client.4508 populate_metadata read hostname 'smithi089' 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f45a81b80 4 client.4508 mds_command: new command op to 4190 tid=0[{"prefix": "status"}] 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.607+0000 7f5f45a81b80 1 -- 172.21.15.89:0/68575114 --> v1:172.21.15.89:6817/3539463463 -- command(tid 0: {"prefix": "status"}) v1 -- 0x5566946e5ef0 con 0x5566949b3120 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f07fff700 10 client.4508 ms_handle_connect on v1:172.21.15.89:6817/3539463463 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f07fff700 1 -- 172.21.15.89:0/68575114 <== mds.0 v1:172.21.15.89:6817/3539463463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (unknown 0 0 947887363) 0x5566946e5ef0 con 0x5566949b3120 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f07fff700 10 client.4508 handle_command_reply: tid=0 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 client.4508 shutdown 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 -- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 msgr2=0x5566949abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949abd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f5f2400b6c0 tx=0x7f5f24021040).stop 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 -- 172.21.15.89:0/68575114 shutdown_connections 2023-09-30T22:14:19.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x5566949515b0 0x5566949abd80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x556694927170 0x556694954760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f07fff700 0 client.4508 ms_handle_reset on v1:172.21.15.89:6817/3539463463 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 --2- 172.21.15.89:0/68575114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556694927a40 0x556694954c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 -- 172.21.15.89:0/68575114 >> 172.21.15.89:0/68575114 conn(0x556694795940 msgr2=0x5566949442e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 -- 172.21.15.89:0/68575114 shutdown_connections 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 1 -- 172.21.15.89:0/68575114 wait complete. 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.608+0000 7f5f45a81b80 20 client.4508 trim_cache size 0 max 16384 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-30T22:14:19.615 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "389f6e95-ea58-47bd-a145-778ba0724a9e", 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4190, 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 17.566600446999999, 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 5, 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 17, 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 17, 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 21.964735675 2023-09-30T22:14:19.616 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-30T22:14:19.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 -- 172.21.15.89:0/930798742 >> v1:172.21.15.89:6816/118161 conn(0x7f5f1002bb60 legacy=0x7f5f1002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 -- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c0970d0 msgr2=0x7f5f2c12b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c0970d0 0x7f5f2c12b6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5f2800e980 tx=0x7f5f28009060).stop 2023-09-30T22:14:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 -- 172.21.15.89:0/930798742 shutdown_connections 2023-09-30T22:14:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2c12d120 0x7f5f2c12bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2c0965e0 0x7f5f2c066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 --2- 172.21.15.89:0/930798742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2c0970d0 0x7f5f2c12b6f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 -- 172.21.15.89:0/930798742 >> 172.21.15.89:0/930798742 conn(0x7f5f2c09a220 msgr2=0x7f5f2c098170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 -- 172.21.15.89:0/930798742 shutdown_connections 2023-09-30T22:14:19.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.609+0000 7f5f340ce700 1 -- 172.21.15.89:0/930798742 wait complete. 2023-09-30T22:14:19.620 INFO:teuthology.run_tasks:Running task exec... 2023-09-30T22:14:19.629 INFO:teuthology.task.exec:Executing custom commands... 2023-09-30T22:14:19.629 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:14:19.630 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client pacific' 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.743+0000 7f4c519a7700 1 Processor -- start 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.743+0000 7f4c519a7700 1 -- start start 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c519a7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c4c127570 0x7f4c4c127dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c519a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1282b0 0x7f4c4c124100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c519a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1245f0 0x7f4c4c124a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c519a7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c09a370 con 0x7f4c4c127570 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c519a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c092cc0 con 0x7f4c4c1282b0 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c519a7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c092f80 con 0x7f4c4c1245f0 2023-09-30T22:14:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1282b0 0x7f4c4c124100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c4c127570 0x7f4c4c127dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1245f0 0x7f4c4c124a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1282b0 0x7f4c4c124100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52498/0 (socket says 172.21.15.89:52498) 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1245f0 0x7f4c4c124a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:34418/0 (socket says 172.21.15.89:34418) 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c4c127570 0x7f4c4c127dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37112/0 (socket says 172.21.15.89:37112) 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 -- 172.21.15.89:0/2414763972 learned_addr learned my addr 172.21.15.89:0/2414763972 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 -- 172.21.15.89:0/2414763972 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1245f0 msgr2=0x7f4c4c124a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 --2- 172.21.15.89:0/2414763972 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1245f0 0x7f4c4c124a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 -- 172.21.15.89:0/2414763972 >> v2:172.21.15.89:3300/0 conn(0x7f4c4c127570 msgr2=0x7f4c4c127dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 --2- 172.21.15.89:0/2414763972 >> v2:172.21.15.89:3300/0 conn(0x7f4c4c127570 0x7f4c4c127dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 -- 172.21.15.89:0/2414763972 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c124f30 con 0x7f4c4c1282b0 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4affd700 1 --2- 172.21.15.89:0/2414763972 >> v2:172.21.15.89:3300/0 conn(0x7f4c4c127570 0x7f4c4c127dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4b7fe700 1 --2- 172.21.15.89:0/2414763972 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1245f0 0x7f4c4c124a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c4a7fc700 1 --2- 172.21.15.89:0/2414763972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1282b0 0x7f4c4c124100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4c40006fd0 tx=0x7f4c4000b3b0).ready entity=mon.1 client_cookie=65b170c3b2005391 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c49ffb700 1 -- 172.21.15.89:0/2414763972 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000da70 con 0x7f4c4c1282b0 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c49ffb700 1 -- 172.21.15.89:0/2414763972 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4000dbd0 con 0x7f4c4c1282b0 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.744+0000 7f4c49ffb700 1 -- 172.21.15.89:0/2414763972 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40010470 con 0x7f4c4c1282b0 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 -- 172.21.15.89:0/2414763972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1282b0 msgr2=0x7f4c4c124100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 --2- 172.21.15.89:0/2414763972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1282b0 0x7f4c4c124100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4c40006fd0 tx=0x7f4c4000b3b0).stop 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 -- 172.21.15.89:0/2414763972 shutdown_connections 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 --2- 172.21.15.89:0/2414763972 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1245f0 0x7f4c4c124a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 --2- 172.21.15.89:0/2414763972 >> v2:172.21.15.89:3300/0 conn(0x7f4c4c127570 0x7f4c4c127dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 --2- 172.21.15.89:0/2414763972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1282b0 0x7f4c4c124100 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 -- 172.21.15.89:0/2414763972 >> 172.21.15.89:0/2414763972 conn(0x7f4c4c059b00 msgr2=0x7f4c4c097410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:19.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 -- 172.21.15.89:0/2414763972 shutdown_connections 2023-09-30T22:14:19.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.745+0000 7f4c519a7700 1 -- 172.21.15.89:0/2414763972 wait complete. 2023-09-30T22:14:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.746+0000 7f4c519a7700 1 Processor -- start 2023-09-30T22:14:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.746+0000 7f4c519a7700 1 -- start start 2023-09-30T22:14:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.746+0000 7f4c519a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1245f0 0x7f4c4c1118a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.746+0000 7f4c519a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c127570 0x7f4c4c111d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.746+0000 7f4c519a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1282b0 0x7f4c4c112280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.746+0000 7f4c519a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c123a10 con 0x7f4c4c127570 2023-09-30T22:14:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.746+0000 7f4c519a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c113f60 con 0x7f4c4c1245f0 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.746+0000 7f4c519a7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c06a490 con 0x7f4c4c1282b0 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1245f0 0x7f4c4c1118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1282b0 0x7f4c4c112280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1245f0 0x7f4c4c1118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52514/0 (socket says 172.21.15.89:52514) 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 -- 172.21.15.89:0/1142386084 learned_addr learned my addr 172.21.15.89:0/1142386084 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4a7fc700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c127570 0x7f4c4c111d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 -- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1282b0 msgr2=0x7f4c4c112280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1282b0 0x7f4c4c112280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 -- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c127570 msgr2=0x7f4c4c111d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c127570 0x7f4c4c111d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 -- 172.21.15.89:0/1142386084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c44009a20 con 0x7f4c4c1245f0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4a7fc700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c127570 0x7f4c4c111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c4affd700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1245f0 0x7f4c4c1118a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f4c3c00ea20 tx=0x7f4c3c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c32ffd700 1 -- 172.21.15.89:0/1142386084 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00c850 con 0x7f4c4c1245f0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c32ffd700 1 -- 172.21.15.89:0/1142386084 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00c9b0 con 0x7f4c4c1245f0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c0092a0 con 0x7f4c4c1245f0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c32ffd700 1 -- 172.21.15.89:0/1142386084 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00cd30 con 0x7f4c4c1245f0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.747+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4000b610 con 0x7f4c4c1245f0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.748+0000 7f4c32ffd700 1 -- 172.21.15.89:0/1142386084 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f4c3c012600 con 0x7f4c4c1245f0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.748+0000 7f4c32ffd700 1 -- 172.21.15.89:0/1142386084 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4c3c03b5d0 con 0x7f4c4c1245f0 2023-09-30T22:14:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.749+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c097b20 con 0x7f4c4c1245f0 2023-09-30T22:14:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.753+0000 7f4c32ffd700 1 -- 172.21.15.89:0/1142386084 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f4c3c01b400 con 0x7f4c4c1245f0 2023-09-30T22:14:19.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:19.920+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "pacific"} v 0) v1 -- 0x7f4c4c046c20 con 0x7f4c4c1245f0 2023-09-30T22:14:20.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.093+0000 7f4c32ffd700 1 -- 172.21.15.89:0/1142386084 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "pacific"}]=0 set require_min_compat_client to pacific v18) v1 ==== 143+0+0 (secure 0 0 0) 0x7f4c3c02a0a0 con 0x7f4c4c1245f0 2023-09-30T22:14:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 >> v1:172.21.15.89:6816/118161 conn(0x7f4c3802bb60 legacy=0x7f4c3802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1245f0 msgr2=0x7f4c4c1118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1245f0 0x7f4c4c1118a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f4c3c00ea20 tx=0x7f4c3c009060).stop 2023-09-30T22:14:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 shutdown_connections 2023-09-30T22:14:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c4c1282b0 0x7f4c4c112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c4c1245f0 0x7f4c4c1118a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 --2- 172.21.15.89:0/1142386084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c4c127570 0x7f4c4c111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 >> 172.21.15.89:0/1142386084 conn(0x7f4c4c059b00 msgr2=0x7f4c4c126be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 shutdown_connections 2023-09-30T22:14:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.098+0000 7f4c519a7700 1 -- 172.21.15.89:0/1142386084 wait complete. 2023-09-30T22:14:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:set require_min_compat_client to pacific 2023-09-30T22:14:20.109 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:14:20.118 INFO:teuthology.task.print:**** done ceph 2023-09-30T22:14:20.118 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-09-30T22:14:20.159 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-09-30T22:14:20.159 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-09-30T22:14:20.159 INFO:tasks.ceph_fuse:client.0 config is {} 2023-09-30T22:14:20.160 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-30T22:14:20.160 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-30T22:14:20.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:20.160 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2023-09-30T22:14:20.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:20.184 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2023-09-30T22:14:20.261 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2023-09-30T22:14:20.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:14:20.266 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-09-30T22:14:20.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-30T22:14:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.383+0000 7f10f01bc700 1 Processor -- start 2023-09-30T22:14:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.383+0000 7f10f01bc700 1 -- start start 2023-09-30T22:14:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10f01bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f10e8096380 0x7f10e8094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10f01bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8091b00 0x7f10e8127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10f01bc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e8096760 con 0x7f10e8094c20 2023-09-30T22:14:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10f01bc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e8091ff0 con 0x7f10e8096380 2023-09-30T22:14:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10f01bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e8092c80 con 0x7f10e8091b00 2023-09-30T22:14:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10ed757700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8091b00 0x7f10e8127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10edf58700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f10e8096380 0x7f10e8094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10ee759700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f10e8094c20 0x7f10e8127af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44198/0 (socket says 172.21.15.89:44198) 2023-09-30T22:14:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10ee759700 1 -- 172.21.15.89:0/866306998 learned_addr learned my addr 172.21.15.89:0/866306998 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10edf58700 1 -- 172.21.15.89:0/866306998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8091b00 msgr2=0x7f10e8127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10edf58700 1 --2- 172.21.15.89:0/866306998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8091b00 0x7f10e8127600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10edf58700 1 -- 172.21.15.89:0/866306998 >> v1:172.21.15.89:6789/0 conn(0x7f10e8094c20 legacy=0x7f10e8127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10edf58700 1 -- 172.21.15.89:0/866306998 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8093900 con 0x7f10e8096380 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10ecf56700 1 -- 172.21.15.89:0/866306998 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407855998 0 0) 0x7f10e8096760 con 0x7f10e8094c20 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10ed757700 1 --2- 172.21.15.89:0/866306998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8091b00 0x7f10e8127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.384+0000 7f10edf58700 1 --2- 172.21.15.89:0/866306998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8096380 0x7f10e8094730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f10d8009790 tx=0x7f10d8009060).ready entity=mon.0 client_cookie=256256ec20ea17d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10ecf56700 1 -- 172.21.15.89:0/866306998 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d800f510 con 0x7f10e8096380 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10ecf56700 1 -- 172.21.15.89:0/866306998 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d800f670 con 0x7f10e8096380 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10ecf56700 1 -- 172.21.15.89:0/866306998 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d800f9f0 con 0x7f10e8096380 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10f01bc700 1 -- 172.21.15.89:0/866306998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8096380 msgr2=0x7f10e8094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10f01bc700 1 --2- 172.21.15.89:0/866306998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8096380 0x7f10e8094730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f10d8009790 tx=0x7f10d8009060).stop 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10f01bc700 1 -- 172.21.15.89:0/866306998 shutdown_connections 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10f01bc700 1 --2- 172.21.15.89:0/866306998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8096380 0x7f10e8094730 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10f01bc700 1 --2- 172.21.15.89:0/866306998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8091b00 0x7f10e8127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10f01bc700 1 -- 172.21.15.89:0/866306998 >> 172.21.15.89:0/866306998 conn(0x7f10e8097470 msgr2=0x7f10e8097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10f01bc700 1 -- 172.21.15.89:0/866306998 shutdown_connections 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.385+0000 7f10f01bc700 1 -- 172.21.15.89:0/866306998 wait complete. 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.386+0000 7f10f01bc700 1 Processor -- start 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.386+0000 7f10f01bc700 1 -- start start 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10f01bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10e8091b00 0x7f10e8117c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10f01bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8094c20 0x7f10e8113c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10f01bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8096380 0x7f10e8114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10f01bc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e8124d30 con 0x7f10e8094c20 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10f01bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e806a520 con 0x7f10e8096380 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10f01bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10e8117160 con 0x7f10e8091b00 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10e8091b00 0x7f10e8117c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10ee759700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8096380 0x7f10e8114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10e8091b00 0x7f10e8117c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58866/0 (socket says 172.21.15.89:58866) 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 -- 172.21.15.89:0/116747417 learned_addr learned my addr 172.21.15.89:0/116747417 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10ed757700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8094c20 0x7f10e8113c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 -- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8096380 msgr2=0x7f10e8114150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8096380 0x7f10e8114150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 -- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8094c20 msgr2=0x7f10e8113c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8094c20 0x7f10e8113c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 -- 172.21.15.89:0/116747417 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40092a0 con 0x7f10e8091b00 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10ed757700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8094c20 0x7f10e8113c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.387+0000 7f10edf58700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10e8091b00 0x7f10e8117c00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f10e8092130 tx=0x7f10d8017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.388+0000 7f10ddffb700 1 -- 172.21.15.89:0/116747417 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d800f7e0 con 0x7f10e8091b00 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.388+0000 7f10ddffb700 1 -- 172.21.15.89:0/116747417 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d800f960 con 0x7f10e8091b00 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.388+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d80092a0 con 0x7f10e8091b00 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.388+0000 7f10ddffb700 1 -- 172.21.15.89:0/116747417 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d801b660 con 0x7f10e8091b00 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.388+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e8116050 con 0x7f10e8091b00 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.388+0000 7f10ddffb700 1 -- 172.21.15.89:0/116747417 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f10d801ba10 con 0x7f10e8091b00 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.389+0000 7f10ddffb700 1 -- 172.21.15.89:0/116747417 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f10d803c850 con 0x7f10e8091b00 2023-09-30T22:14:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.389+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e809a0d0 con 0x7f10e8091b00 2023-09-30T22:14:20.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.393+0000 7f10ddffb700 1 -- 172.21.15.89:0/116747417 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f10d8029080 con 0x7f10e8091b00 2023-09-30T22:14:20.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.589+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f10e80651b0 con 0x7f10e8091b00 2023-09-30T22:14:20.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.589+0000 7f10ddffb700 1 -- 172.21.15.89:0/116747417 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v5) v1 ==== 53+0+73 (secure 0 0 0) 0x7f10d801f020 con 0x7f10e8091b00 2023-09-30T22:14:20.590 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-30T22:14:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 >> v1:172.21.15.89:6816/118161 conn(0x7f10cc02bb60 legacy=0x7f10cc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10e8091b00 msgr2=0x7f10e8117c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10e8091b00 0x7f10e8117c00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f10e8092130 tx=0x7f10d8017000).stop 2023-09-30T22:14:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 shutdown_connections 2023-09-30T22:14:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10e8096380 0x7f10e8114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10e8091b00 0x7f10e8117c00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 --2- 172.21.15.89:0/116747417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10e8094c20 0x7f10e8113c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 >> 172.21.15.89:0/116747417 conn(0x7f10e8097470 msgr2=0x7f10e8098780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:20.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 shutdown_connections 2023-09-30T22:14:20.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:20.591+0000 7f10f01bc700 1 -- 172.21.15.89:0/116747417 wait complete. 2023-09-30T22:14:20.603 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-30T22:14:20.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-30T22:14:20.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2023-09-30T22:14:20.604 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-30T22:14:20.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-30T22:14:20.604 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-30T22:14:20.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-30T22:14:20.604 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-30T22:14:20.604 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-30T22:14:20.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:20.605 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:14:20.628 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42520sec preferred_lft 42520sec 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::ec4:7aff:fe88:6de9/64 scope link 2023-09-30T22:14:20.629 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:14:20.630 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-30T22:14:20.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:20.630 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2023-09-30T22:14:20.630 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2023-09-30T22:14:20.630 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add name ceph-brx type bridge 2023-09-30T22:14:20.630 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr flush dev ceph-brx 2023-09-30T22:14:20.630 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set ceph-brx up 2023-09-30T22:14:20.630 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-30T22:14:20.631 DEBUG:teuthology.orchestra.run.smithi134:> ') 2023-09-30T22:14:20.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:20.797 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-30T22:14:20.827 INFO:teuthology.orchestra.run.smithi134.stdout:1 2023-09-30T22:14:20.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:20.829 DEBUG:teuthology.orchestra.run.smithi134:> route 2023-09-30T22:14:20.910 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2023-09-30T22:14:20.910 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-30T22:14:20.910 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-30T22:14:20.910 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-30T22:14:20.910 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-30T22:14:20.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:20.911 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2023-09-30T22:14:20.911 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2023-09-30T22:14:20.911 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-30T22:14:20.911 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-30T22:14:20.911 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-30T22:14:20.911 DEBUG:teuthology.orchestra.run.smithi134:> ') 2023-09-30T22:14:21.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:21.187 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2023-09-30T22:14:21.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:21.206 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list-id 2023-09-30T22:14:21.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:21.263 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2023-09-30T22:14:21.263 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2023-09-30T22:14:21.263 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:14:21.264 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-30T22:14:21.264 DEBUG:teuthology.orchestra.run.smithi134:> ') 2023-09-30T22:14:21.358 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-30T22:14:21.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:21.358 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2023-09-30T22:14:21.358 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2023-09-30T22:14:21.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-30T22:14:21.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-30T22:14:21.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-30T22:14:21.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-30T22:14:21.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-30T22:14:21.359 DEBUG:teuthology.orchestra.run.smithi134:> ') 2023-09-30T22:14:21.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:21.552 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2023-09-30T22:14:21.552 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2023-09-30T22:14:21.552 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.0 up 2023-09-30T22:14:21.552 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set dev brx.0 master ceph-brx 2023-09-30T22:14:21.552 DEBUG:teuthology.orchestra.run.smithi134:> ') 2023-09-30T22:14:21.649 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-30T22:14:21.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-30T22:14:21.650 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-30T22:14:21.668 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-30T22:14:21.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-30T22:14:21.668 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-30T22:14:21.727 DEBUG:teuthology.orchestra.run.smithi134:> sudo modprobe fuse 2023-09-30T22:14:21.796 DEBUG:teuthology.orchestra.run.smithi134:> cat /proc/self/mounts | awk '{print $2}' 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/sys 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/proc 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/dev 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/security 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/shm 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/pts 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/run 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup 2023-09-30T22:14:21.855 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/systemd 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/pstore 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/bpf 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/cpuset 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/pids 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/rdma 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/freezer 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/memory 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/blkio 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/devices 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/perf_event 2023-09-30T22:14:21.856 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/hugetlb 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/tracing 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/config 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/ 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/selinux 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/sys/fs/binfmt_misc 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/debug 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/hugepages 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/mqueue 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/fuse/connections 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/nfs/rpc_pipefs 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/run/user/1000 2023-09-30T22:14:21.857 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph 2023-09-30T22:14:21.858 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns 2023-09-30T22:14:21.858 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:14:21.858 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:14:21.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:21.858 DEBUG:teuthology.orchestra.run.smithi134:> ls /sys/fs/fuse/connections 2023-09-30T22:14:21.914 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-30T22:14:21.914 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-30T22:14:21.956 DEBUG:teuthology.orchestra.run.smithi134:> sudo modprobe fuse 2023-09-30T22:14:21.983 DEBUG:teuthology.orchestra.run.smithi134:> cat /proc/self/mounts | awk '{print $2}' 2023-09-30T22:14:22.043 INFO:teuthology.orchestra.run.smithi134.stdout:/sys 2023-09-30T22:14:22.043 INFO:teuthology.orchestra.run.smithi134.stdout:/proc 2023-09-30T22:14:22.043 INFO:teuthology.orchestra.run.smithi134.stdout:/dev 2023-09-30T22:14:22.043 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/security 2023-09-30T22:14:22.043 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/shm 2023-09-30T22:14:22.043 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/pts 2023-09-30T22:14:22.043 INFO:teuthology.orchestra.run.smithi134.stdout:/run 2023-09-30T22:14:22.043 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/systemd 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/pstore 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/bpf 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/cpuset 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/pids 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/rdma 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/freezer 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/memory 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/blkio 2023-09-30T22:14:22.044 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/devices 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/perf_event 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/cgroup/hugetlb 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/tracing 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/config 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/ 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/selinux 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/proc/sys/fs/binfmt_misc 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/kernel/debug 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/hugepages 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/dev/mqueue 2023-09-30T22:14:22.045 INFO:teuthology.orchestra.run.smithi134.stdout:/sys/fs/fuse/connections 2023-09-30T22:14:22.046 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/nfs/rpc_pipefs 2023-09-30T22:14:22.046 INFO:teuthology.orchestra.run.smithi134.stdout:/run/user/1000 2023-09-30T22:14:22.046 INFO:teuthology.orchestra.run.smithi134.stdout:/var/lib/ceph 2023-09-30T22:14:22.046 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns 2023-09-30T22:14:22.046 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:14:22.046 INFO:teuthology.orchestra.run.smithi134.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:14:22.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:22.046 DEBUG:teuthology.orchestra.run.smithi134:> ls /sys/fs/fuse/connections 2023-09-30T22:14:22.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi134.stderr:2023-09-30T22:14:22.049+0000 7f2ea5e693c0 -1 init, newargv = 0x555d640eb1b0 newargc=15 2023-09-30T22:14:22.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi134.stderr:ceph-fuse[112275]: starting ceph client 2023-09-30T22:14:22.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi134.stderr:ceph-fuse[112275]: starting fuse 2023-09-30T22:14:22.068 INFO:teuthology.orchestra.run.smithi134.stdout:46 2023-09-30T22:14:22.068 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-09-30T22:14:22.068 DEBUG:teuthology.orchestra.run.smithi134:> sudo stdin-killer -- python3 -c ' 2023-09-30T22:14:22.068 DEBUG:teuthology.orchestra.run.smithi134:> import glob 2023-09-30T22:14:22.068 DEBUG:teuthology.orchestra.run.smithi134:> import re 2023-09-30T22:14:22.068 DEBUG:teuthology.orchestra.run.smithi134:> import os 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> import subprocess 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> def _find_admin_socket(client_name): 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> files = glob.glob(asok_path) 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> # Given a non-glob path, it better be there 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> if "*" not in asok_path: 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> assert(len(files) == 1) 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> return files[0] 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> 2023-09-30T22:14:22.069 DEBUG:teuthology.orchestra.run.smithi134:> for f in files: 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> contents = proc_f.read() 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> if mountpoint in contents: 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> return f 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> print(_find_admin_socket("client.0")) 2023-09-30T22:14:22.070 DEBUG:teuthology.orchestra.run.smithi134:> ' 2023-09-30T22:14:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/var/run/ceph/ceph-client.0.112275.asok 2023-09-30T22:14:22.195 INFO:teuthology.orchestra.run.smithi134.stderr:2023-09-30T22:14:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-30T22:14:22.202 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.112275.asok 2023-09-30T22:14:22.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:22.202 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.112275.asok status 2023-09-30T22:14:22.353 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-09-30T22:14:22.353 INFO:teuthology.orchestra.run.smithi134.stdout: "metadata": { 2023-09-30T22:14:22.353 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_sha1": "eeb13ac974775bd0cce96a2c687e8bcc2faca43d", 2023-09-30T22:14:22.353 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 16.2.14-44-geeb13ac9 (eeb13ac974775bd0cce96a2c687e8bcc2faca43d) pacific (stable)", 2023-09-30T22:14:22.353 INFO:teuthology.orchestra.run.smithi134.stdout: "entity_id": "0", 2023-09-30T22:14:22.353 INFO:teuthology.orchestra.run.smithi134.stdout: "hostname": "smithi134", 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "pid": "112275", 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "root": "/" 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "dentry_count": 0, 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "dentry_pinned_count": 0, 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "id": 4523, 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "inst": { 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "name": { 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "client", 2023-09-30T22:14:22.354 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4523 2023-09-30T22:14:22.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-09-30T22:14:22.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": { 2023-09-30T22:14:22.355 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2023-09-30T22:14:22.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:0", 2023-09-30T22:14:22.355 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 521618095 2023-09-30T22:14:22.355 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-09-30T22:14:22.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-09-30T22:14:22.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": { 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:0", 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 521618095 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: "inst_str": "client.4523 172.21.15.134:0/521618095", 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: "addr_str": "172.21.15.134:0/521618095", 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: "inode_count": 1, 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_epoch": 5, 2023-09-30T22:14:22.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_epoch": 18, 2023-09-30T22:14:22.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_epoch_barrier": 0, 2023-09-30T22:14:22.357 INFO:teuthology.orchestra.run.smithi134.stdout: "blocklisted": false, 2023-09-30T22:14:22.357 INFO:teuthology.orchestra.run.smithi134.stdout: "fs_name": "cephfs" 2023-09-30T22:14:22.357 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-09-30T22:14:22.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:22.363 DEBUG:teuthology.orchestra.run.smithi134:> stat --file-system '--printf=%T 2023-09-30T22:14:22.363 DEBUG:teuthology.orchestra.run.smithi134:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-30T22:14:22.381 INFO:teuthology.orchestra.run.smithi134.stdout:fuseblk 2023-09-30T22:14:22.382 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-30T22:14:22.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:14:22.382 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-30T22:14:22.450 INFO:teuthology.run_tasks:Running task mds_pre_upgrade... 2023-09-30T22:14:22.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.588+0000 7fd1ebca9700 1 Processor -- start 2023-09-30T22:14:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.588+0000 7fd1ebca9700 1 -- start start 2023-09-30T22:14:22.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.588+0000 7fd1ebca9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd1e4059070 0x7fd1e4127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.588+0000 7fd1ebca9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.588+0000 7fd1ebca9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4095370 0x7fd1e40957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.588+0000 7fd1ebca9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e4096940 con 0x7fd1e4059070 2023-09-30T22:14:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1ebca9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4059c00 con 0x7fd1e4127390 2023-09-30T22:14:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1ebca9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4091b60 con 0x7fd1e4095370 2023-09-30T22:14:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9a45700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd1e4059070 0x7fd1e4127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1ea246700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4095370 0x7fd1e40957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4127d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41166/0 (socket says 172.21.15.89:41166) 2023-09-30T22:14:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9a45700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd1e4059070 0x7fd1e4127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33536/0 (socket says 172.21.15.89:33536) 2023-09-30T22:14:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 -- 172.21.15.89:0/2112868388 learned_addr learned my addr 172.21.15.89:0/2112868388 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 -- 172.21.15.89:0/2112868388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4095370 msgr2=0x7fd1e40957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 --2- 172.21.15.89:0/2112868388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4095370 0x7fd1e40957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 -- 172.21.15.89:0/2112868388 >> v2:172.21.15.89:3300/0 conn(0x7fd1e4059070 msgr2=0x7fd1e4127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 --2- 172.21.15.89:0/2112868388 >> v2:172.21.15.89:3300/0 conn(0x7fd1e4059070 0x7fd1e4127890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 -- 172.21.15.89:0/2112868388 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e40929d0 con 0x7fd1e4127390 2023-09-30T22:14:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1ea246700 1 --2- 172.21.15.89:0/2112868388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4095370 0x7fd1e40957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e9244700 1 --2- 172.21.15.89:0/2112868388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4127d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd1d4006fd0 tx=0x7fd1d40090f0).ready entity=mon.1 client_cookie=a57f69d78a0095be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e8a43700 1 -- 172.21.15.89:0/2112868388 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400f510 con 0x7fd1e4127390 2023-09-30T22:14:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e8a43700 1 -- 172.21.15.89:0/2112868388 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400f670 con 0x7fd1e4127390 2023-09-30T22:14:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.589+0000 7fd1e8a43700 1 -- 172.21.15.89:0/2112868388 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400fa10 con 0x7fd1e4127390 2023-09-30T22:14:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 -- 172.21.15.89:0/2112868388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 msgr2=0x7fd1e4127d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 --2- 172.21.15.89:0/2112868388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4127d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd1d4006fd0 tx=0x7fd1d40090f0).stop 2023-09-30T22:14:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 -- 172.21.15.89:0/2112868388 shutdown_connections 2023-09-30T22:14:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 --2- 172.21.15.89:0/2112868388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4095370 0x7fd1e40957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 --2- 172.21.15.89:0/2112868388 >> v2:172.21.15.89:3300/0 conn(0x7fd1e4059070 0x7fd1e4127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 --2- 172.21.15.89:0/2112868388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4127d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 -- 172.21.15.89:0/2112868388 >> 172.21.15.89:0/2112868388 conn(0x7fd1e4097470 msgr2=0x7fd1e4097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 -- 172.21.15.89:0/2112868388 shutdown_connections 2023-09-30T22:14:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.590+0000 7fd1ebca9700 1 -- 172.21.15.89:0/2112868388 wait complete. 2023-09-30T22:14:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ebca9700 1 Processor -- start 2023-09-30T22:14:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ebca9700 1 -- start start 2023-09-30T22:14:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ebca9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4059070 0x7fd1e41179f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ebca9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e4095370 0x7fd1e41139c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ebca9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ebca9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4115eb0 con 0x7fd1e4095370 2023-09-30T22:14:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ebca9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4124b50 con 0x7fd1e4127390 2023-09-30T22:14:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ebca9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4096f20 con 0x7fd1e4059070 2023-09-30T22:14:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1e9a45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4059070 0x7fd1e41179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1e9244700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e4095370 0x7fd1e41139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1e9a45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4059070 0x7fd1e41179f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58896/0 (socket says 172.21.15.89:58896) 2023-09-30T22:14:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1e9a45700 1 -- 172.21.15.89:0/775010986 learned_addr learned my addr 172.21.15.89:0/775010986 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.591+0000 7fd1ea246700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1e9244700 1 -- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4059070 msgr2=0x7fd1e41179f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1e9244700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4059070 0x7fd1e41179f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1e9244700 1 -- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 msgr2=0x7fd1e4113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1e9244700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1e9244700 1 -- 172.21.15.89:0/775010986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc009a20 con 0x7fd1e4095370 2023-09-30T22:14:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1ea246700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1e9244700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e4095370 0x7fd1e41139c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd1e4095060 tx=0x7fd1d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1d9ffb700 1 -- 172.21.15.89:0/775010986 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4000bc0 con 0x7fd1e4095370 2023-09-30T22:14:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1d9ffb700 1 -- 172.21.15.89:0/775010986 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400f7e0 con 0x7fd1e4095370 2023-09-30T22:14:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d4009370 con 0x7fd1e4095370 2023-09-30T22:14:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1d9ffb700 1 -- 172.21.15.89:0/775010986 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4000d20 con 0x7fd1e4095370 2023-09-30T22:14:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.592+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e4116f50 con 0x7fd1e4095370 2023-09-30T22:14:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.593+0000 7fd1d9ffb700 1 -- 172.21.15.89:0/775010986 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd1d4022020 con 0x7fd1e4095370 2023-09-30T22:14:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.593+0000 7fd1d9ffb700 1 -- 172.21.15.89:0/775010986 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd1d403c650 con 0x7fd1e4095370 2023-09-30T22:14:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.593+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e405a260 con 0x7fd1e4095370 2023-09-30T22:14:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.598+0000 7fd1d9ffb700 1 -- 172.21.15.89:0/775010986 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fd1d40103a0 con 0x7fd1e4095370 2023-09-30T22:14:22.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.759+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1e4125290 con 0x7fd1e4095370 2023-09-30T22:14:22.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.760+0000 7fd1d9ffb700 1 -- 172.21.15.89:0/775010986 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+11340 (secure 0 0 0) 0x7fd1d401ab90 con 0x7fd1e4095370 2023-09-30T22:14:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:22.760 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:20.090873+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:22.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 >> v1:172.21.15.89:6816/118161 conn(0x7fd1c802bb60 legacy=0x7fd1c802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:22.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e4095370 msgr2=0x7fd1e41139c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:22.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e4095370 0x7fd1e41139c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd1e4095060 tx=0x7fd1d4017000).stop 2023-09-30T22:14:22.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 shutdown_connections 2023-09-30T22:14:22.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e4127390 0x7fd1e4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e4059070 0x7fd1e41179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 --2- 172.21.15.89:0/775010986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e4095370 0x7fd1e41139c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 >> 172.21.15.89:0/775010986 conn(0x7fd1e4097470 msgr2=0x7fd1e40944e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:22.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.761+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 shutdown_connections 2023-09-30T22:14:22.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.762+0000 7fd1ebca9700 1 -- 172.21.15.89:0/775010986 wait complete. 2023-09-30T22:14:22.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:14:00.533280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:14:01.011104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:14:22.774 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:14:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.891+0000 7f20bc427700 1 Processor -- start 2023-09-30T22:14:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.891+0000 7f20bc427700 1 -- start start 2023-09-30T22:14:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20bc427700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20b40943b0 0x7f20b4126e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20bc427700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b4127440 0x7f20b4127820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20bc427700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4123e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20bc427700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b4093e90 con 0x7f20b40943b0 2023-09-30T22:14:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20bc427700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4127d10 con 0x7f20b4127440 2023-09-30T22:14:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20bc427700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4094790 con 0x7f20b4123950 2023-09-30T22:14:22.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20b40943b0 0x7f20b4126e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20b99c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b4127440 0x7f20b4127820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20b40943b0 0x7f20b4126e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33554/0 (socket says 172.21.15.89:33554) 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20b99c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b4127440 0x7f20b4127820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41186/0 (socket says 172.21.15.89:41186) 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 -- 172.21.15.89:0/108014109 learned_addr learned my addr 172.21.15.89:0/108014109 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba9c4700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4123e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 -- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 msgr2=0x7f20b4123e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4123e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 -- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b4127440 msgr2=0x7f20b4127820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b4127440 0x7f20b4127820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 -- 172.21.15.89:0/108014109 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b4093bb0 con 0x7f20b40943b0 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba9c4700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4123e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20b99c2700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b4127440 0x7f20b4127820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.892+0000 7f20ba1c3700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b40943b0 0x7f20b4126e20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f20a4006fd0 tx=0x7f20a40090f0).ready entity=mon.0 client_cookie=298b6212557a3d1d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20b91c1700 1 -- 172.21.15.89:0/108014109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a400f510 con 0x7f20b40943b0 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20b91c1700 1 -- 172.21.15.89:0/108014109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a400f670 con 0x7f20b40943b0 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20b91c1700 1 -- 172.21.15.89:0/108014109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a400f9f0 con 0x7f20b40943b0 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 -- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b40943b0 msgr2=0x7f20b4126e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b40943b0 0x7f20b4126e20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f20a4006fd0 tx=0x7f20a40090f0).stop 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 -- 172.21.15.89:0/108014109 shutdown_connections 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4123e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b40943b0 0x7f20b4126e20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 --2- 172.21.15.89:0/108014109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b4127440 0x7f20b4127820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 -- 172.21.15.89:0/108014109 >> 172.21.15.89:0/108014109 conn(0x7f20b409a1c0 msgr2=0x7f20b409a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 -- 172.21.15.89:0/108014109 shutdown_connections 2023-09-30T22:14:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.893+0000 7f20bc427700 1 -- 172.21.15.89:0/108014109 wait complete. 2023-09-30T22:14:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.894+0000 7f20bc427700 1 Processor -- start 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.894+0000 7f20bc427700 1 -- start start 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20bc427700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b40943b0 0x7f20b41225d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20bc427700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4122ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20bc427700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b4127440 0x7f20b4111980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20bc427700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4112df0 con 0x7f20b4127440 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20bc427700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4116130 con 0x7f20b40943b0 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20bc427700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4125cb0 con 0x7f20b4123950 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20b99c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4122ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba9c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b4127440 0x7f20b4111980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20b99c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4122ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58902/0 (socket says 172.21.15.89:58902) 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20b99c2700 1 -- 172.21.15.89:0/164285113 learned_addr learned my addr 172.21.15.89:0/164285113 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:22.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba1c3700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b40943b0 0x7f20b41225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba9c4700 1 -- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 msgr2=0x7f20b4122ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba9c4700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4122ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba9c4700 1 -- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b40943b0 msgr2=0x7f20b41225d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba9c4700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b40943b0 0x7f20b41225d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba9c4700 1 -- 172.21.15.89:0/164285113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac00aa20 con 0x7f20b4127440 2023-09-30T22:14:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20b99c2700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4122ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba1c3700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b40943b0 0x7f20b41225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20ba9c4700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b4127440 0x7f20b4111980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f20ac00af40 tx=0x7f20ac00c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20aa7fc700 1 -- 172.21.15.89:0/164285113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac00db70 con 0x7f20b4127440 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20aa7fc700 1 -- 172.21.15.89:0/164285113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac010040 con 0x7f20b4127440 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b00092a0 con 0x7f20b4127440 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20aa7fc700 1 -- 172.21.15.89:0/164285113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac00dee0 con 0x7f20b4127440 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.895+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a4009370 con 0x7f20b4127440 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.896+0000 7f20aa7fc700 1 -- 172.21.15.89:0/164285113 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f20ac014610 con 0x7f20b4127440 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.896+0000 7f20aa7fc700 1 -- 172.21.15.89:0/164285113 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f20ac03c7f0 con 0x7f20b4127440 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.896+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b4063930 con 0x7f20b4127440 2023-09-30T22:14:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:22.901+0000 7f20aa7fc700 1 -- 172.21.15.89:0/164285113 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f20ac01d410 con 0x7f20b4127440 2023-09-30T22:14:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.057+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f20b405a260 con 0x7f20b4127440 2023-09-30T22:14:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.057+0000 7f20aa7fc700 1 -- 172.21.15.89:0/164285113 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f20ac02b0b0 con 0x7f20b4127440 2023-09-30T22:14:23.058 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:14:23.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 >> v1:172.21.15.89:6816/118161 conn(0x7f209802bb60 legacy=0x7f209802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b4127440 msgr2=0x7f20b4111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b4127440 0x7f20b4111980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f20ac00af40 tx=0x7f20ac00c040).stop 2023-09-30T22:14:23.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 shutdown_connections 2023-09-30T22:14:23.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b4127440 0x7f20b4111980 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b40943b0 0x7f20b41225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 --2- 172.21.15.89:0/164285113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b4123950 0x7f20b4122ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 >> 172.21.15.89:0/164285113 conn(0x7f20b409a1c0 msgr2=0x7f20b4124c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:23.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 shutdown_connections 2023-09-30T22:14:23.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.059+0000 7f20bc427700 1 -- 172.21.15.89:0/164285113 wait complete. 2023-09-30T22:14:23.071 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:14:23.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.190+0000 7f5e38a8a700 1 Processor -- start 2023-09-30T22:14:23.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.190+0000 7f5e38a8a700 1 -- start start 2023-09-30T22:14:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e38a8a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e340653a0 0x7f5e34098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e38a8a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e34130440 0x7f5e34132830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e38a8a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e340663b0 con 0x7f5e34098b30 2023-09-30T22:14:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e38a8a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34065e80 con 0x7f5e340653a0 2023-09-30T22:14:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e38a8a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34065780 con 0x7f5e34130440 2023-09-30T22:14:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e340653a0 0x7f5e34098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e32d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e34130440 0x7f5e34132830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e31d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5e34098b30 0x7f5e34098f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44210/0 (socket says 172.21.15.89:44210) 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e31d9b700 1 -- 172.21.15.89:0/2606843057 learned_addr learned my addr 172.21.15.89:0/2606843057 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3259c700 1 -- 172.21.15.89:0/2606843057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e34130440 msgr2=0x7f5e34132830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3159a700 1 -- 172.21.15.89:0/2606843057 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976220280 0 0) 0x7f5e340663b0 con 0x7f5e34098b30 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3259c700 1 --2- 172.21.15.89:0/2606843057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e34130440 0x7f5e34132830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3259c700 1 -- 172.21.15.89:0/2606843057 >> v1:172.21.15.89:6789/0 conn(0x7f5e34098b30 legacy=0x7f5e34098f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3259c700 1 -- 172.21.15.89:0/2606843057 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34096180 con 0x7f5e340653a0 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e32d9d700 1 --2- 172.21.15.89:0/2606843057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e34130440 0x7f5e34132830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3259c700 1 --2- 172.21.15.89:0/2606843057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e340653a0 0x7f5e34098640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5e240097c0 tx=0x7f5e24009060).ready entity=mon.1 client_cookie=2f97473d2b2f82bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3159a700 1 -- 172.21.15.89:0/2606843057 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400f510 con 0x7f5e340653a0 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.191+0000 7f5e3159a700 1 -- 172.21.15.89:0/2606843057 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2400f670 con 0x7f5e340653a0 2023-09-30T22:14:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e3159a700 1 -- 172.21.15.89:0/2606843057 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400fa40 con 0x7f5e340653a0 2023-09-30T22:14:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e38a8a700 1 -- 172.21.15.89:0/2606843057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e340653a0 msgr2=0x7f5e34098640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e38a8a700 1 --2- 172.21.15.89:0/2606843057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e340653a0 0x7f5e34098640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5e240097c0 tx=0x7f5e24009060).stop 2023-09-30T22:14:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e38a8a700 1 -- 172.21.15.89:0/2606843057 shutdown_connections 2023-09-30T22:14:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e38a8a700 1 --2- 172.21.15.89:0/2606843057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e340653a0 0x7f5e34098640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e38a8a700 1 --2- 172.21.15.89:0/2606843057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e34130440 0x7f5e34132830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e38a8a700 1 -- 172.21.15.89:0/2606843057 >> 172.21.15.89:0/2606843057 conn(0x7f5e3409dc20 msgr2=0x7f5e3409e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e38a8a700 1 -- 172.21.15.89:0/2606843057 shutdown_connections 2023-09-30T22:14:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.192+0000 7f5e38a8a700 1 -- 172.21.15.89:0/2606843057 wait complete. 2023-09-30T22:14:23.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e38a8a700 1 Processor -- start 2023-09-30T22:14:23.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e38a8a700 1 -- start start 2023-09-30T22:14:23.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e38a8a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e340653a0 0x7f5e3413aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e38a8a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e34098b30 0x7f5e3413af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e38a8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e34130440 0x7f5e34119650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e38a8a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3411cd90 con 0x7f5e34130440 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e38a8a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34130820 con 0x7f5e34098b30 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e38a8a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3411aaa0 con 0x7f5e340653a0 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e3259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e340653a0 0x7f5e3413aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e31d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e34098b30 0x7f5e3413af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e3259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e340653a0 0x7f5e3413aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58912/0 (socket says 172.21.15.89:58912) 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e3259c700 1 -- 172.21.15.89:0/529746138 learned_addr learned my addr 172.21.15.89:0/529746138 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.193+0000 7f5e32d9d700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e34130440 0x7f5e34119650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e31d9b700 1 -- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e340653a0 msgr2=0x7f5e3413aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e31d9b700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e340653a0 0x7f5e3413aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e31d9b700 1 -- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e34130440 msgr2=0x7f5e34119650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e31d9b700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e34130440 0x7f5e34119650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e31d9b700 1 -- 172.21.15.89:0/529746138 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009a20 con 0x7f5e34098b30 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e3259c700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e340653a0 0x7f5e3413aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e32d9d700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e34130440 0x7f5e34119650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e31d9b700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e34098b30 0x7f5e3413af40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5e28001eb0 tx=0x7f5e28007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e227fc700 1 -- 172.21.15.89:0/529746138 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800ce10 con 0x7f5e34098b30 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e227fc700 1 -- 172.21.15.89:0/529746138 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e280072e0 con 0x7f5e34098b30 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e227fc700 1 -- 172.21.15.89:0/529746138 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280057a0 con 0x7f5e34098b30 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e240092a0 con 0x7f5e34098b30 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.194+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3411b960 con 0x7f5e34098b30 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.195+0000 7f5e227fc700 1 -- 172.21.15.89:0/529746138 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5e28005bd0 con 0x7f5e34098b30 2023-09-30T22:14:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.195+0000 7f5e227fc700 1 -- 172.21.15.89:0/529746138 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5e2803a620 con 0x7f5e34098b30 2023-09-30T22:14:23.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.195+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3411ca10 con 0x7f5e34098b30 2023-09-30T22:14:23.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.199+0000 7f5e227fc700 1 -- 172.21.15.89:0/529746138 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5e3411ca10 con 0x7f5e34098b30 2023-09-30T22:14:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.357+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5e3411ca10 con 0x7f5e34098b30 2023-09-30T22:14:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.357+0000 7f5e227fc700 1 -- 172.21.15.89:0/529746138 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5e3411ca10 con 0x7f5e34098b30 2023-09-30T22:14:23.358 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 >> v1:172.21.15.89:6816/118161 conn(0x7f5e1402bb60 legacy=0x7f5e1402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e34098b30 msgr2=0x7f5e3413af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e34098b30 0x7f5e3413af40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5e28001eb0 tx=0x7f5e28007b90).stop 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 shutdown_connections 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e34130440 0x7f5e34119650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e340653a0 0x7f5e3413aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 --2- 172.21.15.89:0/529746138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e34098b30 0x7f5e3413af40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 >> 172.21.15.89:0/529746138 conn(0x7f5e3409dc20 msgr2=0x7f5e3412f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 shutdown_connections 2023-09-30T22:14:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.359+0000 7f5e38a8a700 1 -- 172.21.15.89:0/529746138 wait complete. 2023-09-30T22:14:23.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:14:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.488+0000 7f8e4c6e1700 1 Processor -- start 2023-09-30T22:14:23.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.488+0000 7f8e4c6e1700 1 -- start start 2023-09-30T22:14:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4c6e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e44098ea0 0x7f8e440992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4c6e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e4412f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4c6e1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e44092d80 con 0x7f8e44093840 2023-09-30T22:14:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4c6e1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e44093c20 con 0x7f8e44098ea0 2023-09-30T22:14:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4c6e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e44092b90 con 0x7f8e4412d160 2023-09-30T22:14:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e49c7c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e44098ea0 0x7f8e440992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4ac7e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e4412f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4a47d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8e44093840 0x7f8e44098790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44220/0 (socket says 172.21.15.89:44220) 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4ac7e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e4412f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41210/0 (socket says 172.21.15.89:41210) 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e49c7c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e44098ea0 0x7f8e440992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33572/0 (socket says 172.21.15.89:33572) 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4a47d700 1 -- 172.21.15.89:0/1041836715 learned_addr learned my addr 172.21.15.89:0/1041836715 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4947b700 1 -- 172.21.15.89:0/1041836715 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235724942 0 0) 0x7f8e44092d80 con 0x7f8e44093840 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4ac7e700 1 -- 172.21.15.89:0/1041836715 >> v2:172.21.15.89:3300/0 conn(0x7f8e44098ea0 msgr2=0x7f8e440992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4ac7e700 1 --2- 172.21.15.89:0/1041836715 >> v2:172.21.15.89:3300/0 conn(0x7f8e44098ea0 0x7f8e440992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4ac7e700 1 -- 172.21.15.89:0/1041836715 >> v1:172.21.15.89:6789/0 conn(0x7f8e44093840 legacy=0x7f8e44098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4ac7e700 1 -- 172.21.15.89:0/1041836715 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44099b70 con 0x7f8e4412d160 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e49c7c700 1 --2- 172.21.15.89:0/1041836715 >> v2:172.21.15.89:3300/0 conn(0x7f8e44098ea0 0x7f8e440992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.489+0000 7f8e4ac7e700 1 --2- 172.21.15.89:0/1041836715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e4412f550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8e3c009e60 tx=0x7f8e3c009780).ready entity=mon.1 client_cookie=c013a3579246534e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4947b700 1 -- 172.21.15.89:0/1041836715 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00e070 con 0x7f8e4412d160 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4947b700 1 -- 172.21.15.89:0/1041836715 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c00bb60 con 0x7f8e4412d160 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4947b700 1 -- 172.21.15.89:0/1041836715 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c012410 con 0x7f8e4412d160 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/1041836715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 msgr2=0x7f8e4412f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4c6e1700 1 --2- 172.21.15.89:0/1041836715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e4412f550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8e3c009e60 tx=0x7f8e3c009780).stop 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/1041836715 shutdown_connections 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4c6e1700 1 --2- 172.21.15.89:0/1041836715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e4412f550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4c6e1700 1 --2- 172.21.15.89:0/1041836715 >> v2:172.21.15.89:3300/0 conn(0x7f8e44098ea0 0x7f8e440992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/1041836715 >> 172.21.15.89:0/1041836715 conn(0x7f8e44097480 msgr2=0x7f8e44097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/1041836715 shutdown_connections 2023-09-30T22:14:23.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.490+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/1041836715 wait complete. 2023-09-30T22:14:23.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4c6e1700 1 Processor -- start 2023-09-30T22:14:23.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4c6e1700 1 -- start start 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4c6e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e44093840 0x7f8e4412af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4c6e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e44098ea0 0x7f8e4412b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4c6e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e44137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4c6e1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4406a520 con 0x7f8e44098ea0 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4c6e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e44111b00 con 0x7f8e4412d160 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4c6e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e44112780 con 0x7f8e44093840 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4a47d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e44093840 0x7f8e4412af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4ac7e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e44137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e49c7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e44098ea0 0x7f8e4412b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4ac7e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e44137e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41214/0 (socket says 172.21.15.89:41214) 2023-09-30T22:14:23.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e4a47d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e44093840 0x7f8e4412af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58920/0 (socket says 172.21.15.89:58920) 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4ac7e700 1 -- 172.21.15.89:0/383450550 learned_addr learned my addr 172.21.15.89:0/383450550 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.491+0000 7f8e49c7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e44098ea0 0x7f8e4412b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33582/0 (socket says 172.21.15.89:33582) 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4ac7e700 1 -- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e44093840 msgr2=0x7f8e4412af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4ac7e700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e44093840 0x7f8e4412af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4ac7e700 1 -- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e44098ea0 msgr2=0x7f8e4412b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4ac7e700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e44098ea0 0x7f8e4412b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4ac7e700 1 -- 172.21.15.89:0/383450550 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e400092a0 con 0x7f8e4412d160 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e49c7c700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e44098ea0 0x7f8e4412b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4a47d700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e44093840 0x7f8e4412af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4ac7e700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e44137e60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8e44093740 tx=0x7f8e3c009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/383450550 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c021070 con 0x7f8e4412d160 2023-09-30T22:14:23.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/383450550 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c01b830 con 0x7f8e4412d160 2023-09-30T22:14:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c009a20 con 0x7f8e4412d160 2023-09-30T22:14:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/383450550 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c012410 con 0x7f8e4412d160 2023-09-30T22:14:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.492+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4412c4f0 con 0x7f8e4412d160 2023-09-30T22:14:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.493+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/383450550 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8e3c003040 con 0x7f8e4412d160 2023-09-30T22:14:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.493+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/383450550 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f8e3c040470 con 0x7f8e4412d160 2023-09-30T22:14:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.493+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e441120f0 con 0x7f8e4412d160 2023-09-30T22:14:23.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.497+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/383450550 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8e3c017070 con 0x7f8e4412d160 2023-09-30T22:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.655+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e4405a260 con 0x7f8e4412d160 2023-09-30T22:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.655+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/383450550 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v18) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e3c024970 con 0x7f8e4412d160 2023-09-30T22:14:23.656 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-09-30T22:14:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 >> v1:172.21.15.89:6816/118161 conn(0x7f8e2802bb60 legacy=0x7f8e2802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 msgr2=0x7f8e44137e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e44137e60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8e44093740 tx=0x7f8e3c009780).stop 2023-09-30T22:14:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 shutdown_connections 2023-09-30T22:14:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e4412d160 0x7f8e44137e60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e44093840 0x7f8e4412af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 --2- 172.21.15.89:0/383450550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e44098ea0 0x7f8e4412b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 >> 172.21.15.89:0/383450550 conn(0x7f8e44097480 msgr2=0x7f8e44099280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:23.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 shutdown_connections 2023-09-30T22:14:23.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.657+0000 7f8e4c6e1700 1 -- 172.21.15.89:0/383450550 wait complete. 2023-09-30T22:14:23.670 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:14:23.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.787+0000 7f92352fa700 1 Processor -- start 2023-09-30T22:14:23.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.787+0000 7f92352fa700 1 -- start start 2023-09-30T22:14:23.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f92352fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92300694d0 0x7f9230096410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f92352fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230130630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f92352fa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9230095e60 con 0x7f9230096900 2023-09-30T22:14:23.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f92352fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9230068d40 con 0x7f92300694d0 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f92352fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9230068fb0 con 0x7f923012e240 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92300694d0 0x7f9230096410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230130630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922e7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9230096900 0x7f9230096d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44232/0 (socket says 172.21.15.89:44232) 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922e7fc700 1 -- 172.21.15.89:0/3976381890 learned_addr learned my addr 172.21.15.89:0/3976381890 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922effd700 1 -- 172.21.15.89:0/3976381890 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 msgr2=0x7f9230130630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922dffb700 1 -- 172.21.15.89:0/3976381890 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300700486 0 0) 0x7f9230095e60 con 0x7f9230096900 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922effd700 1 --2- 172.21.15.89:0/3976381890 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230130630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922effd700 1 -- 172.21.15.89:0/3976381890 >> v1:172.21.15.89:6789/0 conn(0x7f9230096900 legacy=0x7f9230096d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.788+0000 7f922effd700 1 -- 172.21.15.89:0/3976381890 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230068a10 con 0x7f92300694d0 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f922f7fe700 1 --2- 172.21.15.89:0/3976381890 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230130630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f922effd700 1 --2- 172.21.15.89:0/3976381890 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92300694d0 0x7f9230096410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9224009790 tx=0x7f9224009060).ready entity=mon.1 client_cookie=5423fbbf3c3b20c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f922dffb700 1 -- 172.21.15.89:0/3976381890 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922400f510 con 0x7f92300694d0 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f922dffb700 1 -- 172.21.15.89:0/3976381890 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922400f670 con 0x7f92300694d0 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f922dffb700 1 -- 172.21.15.89:0/3976381890 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922400f9f0 con 0x7f92300694d0 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f92352fa700 1 -- 172.21.15.89:0/3976381890 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92300694d0 msgr2=0x7f9230096410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f92352fa700 1 --2- 172.21.15.89:0/3976381890 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92300694d0 0x7f9230096410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9224009790 tx=0x7f9224009060).stop 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f92352fa700 1 -- 172.21.15.89:0/3976381890 shutdown_connections 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f92352fa700 1 --2- 172.21.15.89:0/3976381890 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92300694d0 0x7f9230096410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f92352fa700 1 --2- 172.21.15.89:0/3976381890 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230130630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f92352fa700 1 -- 172.21.15.89:0/3976381890 >> 172.21.15.89:0/3976381890 conn(0x7f923009ba60 msgr2=0x7f923009be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:23.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f92352fa700 1 -- 172.21.15.89:0/3976381890 shutdown_connections 2023-09-30T22:14:23.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.789+0000 7f92352fa700 1 -- 172.21.15.89:0/3976381890 wait complete. 2023-09-30T22:14:23.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.790+0000 7f92352fa700 1 Processor -- start 2023-09-30T22:14:23.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.790+0000 7f92352fa700 1 -- start start 2023-09-30T22:14:23.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f92352fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92300694d0 0x7f923011e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f92352fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9230096900 0x7f9230064540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f92352fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230064a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:23.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f92352fa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9230091ce0 con 0x7f92300694d0 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f92352fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923011e810 con 0x7f9230096900 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f92352fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9230066170 con 0x7f923012e240 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92300694d0 0x7f923011e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92300694d0 0x7f923011e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33584/0 (socket says 172.21.15.89:33584) 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9230096900 0x7f9230064540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 -- 172.21.15.89:0/943972821 learned_addr learned my addr 172.21.15.89:0/943972821 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922f7fe700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230064a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 -- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 msgr2=0x7f9230064a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230064a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 -- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9230096900 msgr2=0x7f9230064540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9230096900 0x7f9230064540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 -- 172.21.15.89:0/943972821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9228009a20 con 0x7f92300694d0 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922f7fe700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230064a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922e7fc700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9230096900 0x7f9230064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f922effd700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92300694d0 0x7f923011e130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f9230095ff0 tx=0x7f9224017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f9216ffd700 1 -- 172.21.15.89:0/943972821 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922400f7e0 con 0x7f92300694d0 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f9216ffd700 1 -- 172.21.15.89:0/943972821 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922400f940 con 0x7f92300694d0 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f9216ffd700 1 -- 172.21.15.89:0/943972821 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922401b660 con 0x7f92300694d0 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92240092a0 con 0x7f92300694d0 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.791+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923011c580 con 0x7f92300694d0 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.792+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923011c7a0 con 0x7f92300694d0 2023-09-30T22:14:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.793+0000 7f9216ffd700 1 -- 172.21.15.89:0/943972821 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f922401ba10 con 0x7f92300694d0 2023-09-30T22:14:23.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.793+0000 7f9216ffd700 1 -- 172.21.15.89:0/943972821 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f922403cd80 con 0x7f92300694d0 2023-09-30T22:14:23.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.797+0000 7f9216ffd700 1 -- 172.21.15.89:0/943972821 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f922402d9d0 con 0x7f92300694d0 2023-09-30T22:14:23.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.954+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f923005a260 con 0x7f92300694d0 2023-09-30T22:14:23.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.954+0000 7f9216ffd700 1 -- 172.21.15.89:0/943972821 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v18) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9224029030 con 0x7f92300694d0 2023-09-30T22:14:23.955 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-09-30T22:14:23.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 >> v1:172.21.15.89:6816/118161 conn(0x7f921c02bb90 legacy=0x7f921c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92300694d0 msgr2=0x7f923011e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:23.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92300694d0 0x7f923011e130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f9230095ff0 tx=0x7f9224017000).stop 2023-09-30T22:14:23.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 shutdown_connections 2023-09-30T22:14:23.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923012e240 0x7f9230064a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92300694d0 0x7f923011e130 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 --2- 172.21.15.89:0/943972821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9230096900 0x7f9230064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:23.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 >> 172.21.15.89:0/943972821 conn(0x7f923009ba60 msgr2=0x7f92300957f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:23.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 shutdown_connections 2023-09-30T22:14:23.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:23.956+0000 7f92352fa700 1 -- 172.21.15.89:0/943972821 wait complete. 2023-09-30T22:14:23.969 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:14:24.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.083+0000 7fd501b53700 1 Processor -- start 2023-09-30T22:14:24.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.083+0000 7fd501b53700 1 -- start start 2023-09-30T22:14:24.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.083+0000 7fd501b53700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc1251c0 0x7fd4fc12da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.083+0000 7fd501b53700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd4fc125dd0 0x7fd4fc12ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd501b53700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc095f00 con 0x7fd4fc059070 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd501b53700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc059450 con 0x7fd4fc125dd0 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd501b53700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc096190 con 0x7fd4fc1251c0 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4faffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc1251c0 0x7fd4fc12da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4fbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd4fc125dd0 0x7fd4fc12ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4faffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc1251c0 0x7fd4fc12da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58946/0 (socket says 172.21.15.89:58946) 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4fbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd4fc125dd0 0x7fd4fc12ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33592/0 (socket says 172.21.15.89:33592) 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4fb7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd4fc059070 0x7fd4fc1256c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44238/0 (socket says 172.21.15.89:44238) 2023-09-30T22:14:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4faffd700 1 -- 172.21.15.89:0/3552645161 learned_addr learned my addr 172.21.15.89:0/3552645161 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4faffd700 1 -- 172.21.15.89:0/3552645161 >> v2:172.21.15.89:3300/0 conn(0x7fd4fc125dd0 msgr2=0x7fd4fc12ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4faffd700 1 --2- 172.21.15.89:0/3552645161 >> v2:172.21.15.89:3300/0 conn(0x7fd4fc125dd0 0x7fd4fc12ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4fa7fc700 1 -- 172.21.15.89:0/3552645161 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771163715 0 0) 0x7fd4fc095f00 con 0x7fd4fc059070 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4faffd700 1 -- 172.21.15.89:0/3552645161 >> v1:172.21.15.89:6789/0 conn(0x7fd4fc059070 legacy=0x7fd4fc1256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4faffd700 1 -- 172.21.15.89:0/3552645161 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc059dd0 con 0x7fd4fc1251c0 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4faffd700 1 --2- 172.21.15.89:0/3552645161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc1251c0 0x7fd4fc12da80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd4ec006fd0 tx=0x7fd4ec0090f0).ready entity=mon.2 client_cookie=d12842178f788853 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4fa7fc700 1 -- 172.21.15.89:0/3552645161 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00b580 con 0x7fd4fc1251c0 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4fa7fc700 1 -- 172.21.15.89:0/3552645161 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec00b6e0 con 0x7fd4fc1251c0 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.084+0000 7fd4fa7fc700 1 -- 172.21.15.89:0/3552645161 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00ba60 con 0x7fd4fc1251c0 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.085+0000 7fd501b53700 1 -- 172.21.15.89:0/3552645161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc1251c0 msgr2=0x7fd4fc12da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.085+0000 7fd501b53700 1 --2- 172.21.15.89:0/3552645161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc1251c0 0x7fd4fc12da80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd4ec006fd0 tx=0x7fd4ec0090f0).stop 2023-09-30T22:14:24.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.085+0000 7fd501b53700 1 -- 172.21.15.89:0/3552645161 shutdown_connections 2023-09-30T22:14:24.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.085+0000 7fd501b53700 1 --2- 172.21.15.89:0/3552645161 >> v2:172.21.15.89:3300/0 conn(0x7fd4fc125dd0 0x7fd4fc12ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.085+0000 7fd501b53700 1 --2- 172.21.15.89:0/3552645161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc1251c0 0x7fd4fc12da80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.085+0000 7fd501b53700 1 -- 172.21.15.89:0/3552645161 >> 172.21.15.89:0/3552645161 conn(0x7fd4fc097470 msgr2=0x7fd4fc097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:24.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.085+0000 7fd501b53700 1 -- 172.21.15.89:0/3552645161 shutdown_connections 2023-09-30T22:14:24.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.085+0000 7fd501b53700 1 -- 172.21.15.89:0/3552645161 wait complete. 2023-09-30T22:14:24.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd501b53700 1 Processor -- start 2023-09-30T22:14:24.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd501b53700 1 -- start start 2023-09-30T22:14:24.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd501b53700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4fc059070 0x7fd4fc122c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd501b53700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4fc1251c0 0x7fd4fc12dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd501b53700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc125dd0 0x7fd4fc12e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd501b53700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc093800 con 0x7fd4fc1251c0 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd501b53700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc093020 con 0x7fd4fc059070 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd501b53700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc091e90 con 0x7fd4fc125dd0 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd4fbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc125dd0 0x7fd4fc12e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd4faffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4fc1251c0 0x7fd4fc12dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd4faffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4fc1251c0 0x7fd4fc12dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33594/0 (socket says 172.21.15.89:33594) 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd4fbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc125dd0 0x7fd4fc12e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58952/0 (socket says 172.21.15.89:58952) 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd4faffd700 1 -- 172.21.15.89:0/4173755133 learned_addr learned my addr 172.21.15.89:0/4173755133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.086+0000 7fd4fb7fe700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4fc059070 0x7fd4fc122c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4fbfff700 1 -- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4fc059070 msgr2=0x7fd4fc122c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4fbfff700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4fc059070 0x7fd4fc122c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4fbfff700 1 -- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4fc1251c0 msgr2=0x7fd4fc12dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4fbfff700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4fc1251c0 0x7fd4fc12dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4fbfff700 1 -- 172.21.15.89:0/4173755133 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec009370 con 0x7fd4fc125dd0 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4faffd700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4fc1251c0 0x7fd4fc12dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4fb7fe700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4fc059070 0x7fd4fc122c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4fbfff700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc125dd0 0x7fd4fc12e150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd4f4009e50 tx=0x7fd4f400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4e37fe700 1 -- 172.21.15.89:0/4173755133 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400caa0 con 0x7fd4fc125dd0 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4e37fe700 1 -- 172.21.15.89:0/4173755133 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f400f040 con 0x7fd4fc125dd0 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd4e37fe700 1 -- 172.21.15.89:0/4173755133 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400ce10 con 0x7fd4fc125dd0 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc092d60 con 0x7fd4fc125dd0 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.087+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc12f500 con 0x7fd4fc125dd0 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.088+0000 7fd4e37fe700 1 -- 172.21.15.89:0/4173755133 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd4f4013610 con 0x7fd4fc125dd0 2023-09-30T22:14:24.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.088+0000 7fd4e37fe700 1 -- 172.21.15.89:0/4173755133 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd4f403b780 con 0x7fd4fc125dd0 2023-09-30T22:14:24.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.088+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc111010 con 0x7fd4fc125dd0 2023-09-30T22:14:24.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.092+0000 7fd4e37fe700 1 -- 172.21.15.89:0/4173755133 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fd4f4007720 con 0x7fd4fc125dd0 2023-09-30T22:14:24.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.290+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4fc05a260 con 0x7fd4fc125dd0 2023-09-30T22:14:24.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.291+0000 7fd4e37fe700 1 -- 172.21.15.89:0/4173755133 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2156 (secure 0 0 0) 0x7fd4f402c090 con 0x7fd4fc125dd0 2023-09-30T22:14:24.291 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:24.292 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:04.044604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:14:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 >> v1:172.21.15.89:6816/118161 conn(0x7fd4e802bb60 legacy=0x7fd4e802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc125dd0 msgr2=0x7fd4fc12e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc125dd0 0x7fd4fc12e150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd4f4009e50 tx=0x7fd4f400b040).stop 2023-09-30T22:14:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 shutdown_connections 2023-09-30T22:14:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4fc125dd0 0x7fd4fc12e150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4fc059070 0x7fd4fc122c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 --2- 172.21.15.89:0/4173755133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4fc1251c0 0x7fd4fc12dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 >> 172.21.15.89:0/4173755133 conn(0x7fd4fc097470 msgr2=0x7fd4fc128550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 shutdown_connections 2023-09-30T22:14:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.292+0000 7fd501b53700 1 -- 172.21.15.89:0/4173755133 wait complete. 2023-09-30T22:14:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 5 2023-09-30T22:14:24.305 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:14:24.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4bae954700 1 Processor -- start 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4bae954700 1 -- start start 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4bae954700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4bae954700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba812fe90 0x7f4ba8132280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4bae954700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba812ba30 con 0x7f4ba8064570 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4bae954700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8066590 con 0x7f4ba80661b0 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4bae954700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80656f0 con 0x7f4ba812fe90 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4ba77fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4ba8064570 0x7f4ba812b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44252/0 (socket says 172.21.15.89:44252) 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4bacef1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba812fe90 0x7f4ba8132280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4ba77fe700 1 -- 172.21.15.89:0/1044413472 learned_addr learned my addr 172.21.15.89:0/1044413472 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.420+0000 7f4ba7fff700 1 --2- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba7fff700 1 -- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba812fe90 msgr2=0x7f4ba8132280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba7fff700 1 --2- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba812fe90 0x7f4ba8132280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba6ffd700 1 -- 172.21.15.89:0/1044413472 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226305006 0 0) 0x7f4ba812ba30 con 0x7f4ba8064570 2023-09-30T22:14:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba7fff700 1 -- 172.21.15.89:0/1044413472 >> v1:172.21.15.89:6789/0 conn(0x7f4ba8064570 legacy=0x7f4ba812b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba7fff700 1 -- 172.21.15.89:0/1044413472 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80649f0 con 0x7f4ba80661b0 2023-09-30T22:14:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bacef1700 1 --2- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba812fe90 0x7f4ba8132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba7fff700 1 --2- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812aa90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f4b98009790 tx=0x7f4b98009060).ready entity=mon.1 client_cookie=10934d93ab4739ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba6ffd700 1 -- 172.21.15.89:0/1044413472 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800f510 con 0x7f4ba80661b0 2023-09-30T22:14:24.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba6ffd700 1 -- 172.21.15.89:0/1044413472 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800f670 con 0x7f4ba80661b0 2023-09-30T22:14:24.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4ba6ffd700 1 -- 172.21.15.89:0/1044413472 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800f9f0 con 0x7f4ba80661b0 2023-09-30T22:14:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bae954700 1 -- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 msgr2=0x7f4ba812aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bae954700 1 --2- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812aa90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f4b98009790 tx=0x7f4b98009060).stop 2023-09-30T22:14:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bae954700 1 -- 172.21.15.89:0/1044413472 shutdown_connections 2023-09-30T22:14:24.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bae954700 1 --2- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812aa90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bae954700 1 --2- 172.21.15.89:0/1044413472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba812fe90 0x7f4ba8132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bae954700 1 -- 172.21.15.89:0/1044413472 >> 172.21.15.89:0/1044413472 conn(0x7f4ba809ba50 msgr2=0x7f4ba809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bae954700 1 -- 172.21.15.89:0/1044413472 shutdown_connections 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.421+0000 7f4bae954700 1 -- 172.21.15.89:0/1044413472 wait complete. 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.422+0000 7f4bae954700 1 Processor -- start 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.422+0000 7f4bae954700 1 -- start start 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4bae954700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba8064570 0x7f4ba80696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4bae954700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4bae954700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ba812fe90 0x7f4ba812c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4bae954700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba812d0e0 con 0x7f4ba812fe90 2023-09-30T22:14:24.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4bae954700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80650b0 con 0x7f4ba80661b0 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4bae954700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba806a350 con 0x7f4ba8064570 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba8064570 0x7f4ba80696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba8064570 0x7f4ba80696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58974/0 (socket says 172.21.15.89:58974) 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4bacef1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ba812fe90 0x7f4ba812c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 -- 172.21.15.89:0/3182058337 learned_addr learned my addr 172.21.15.89:0/3182058337 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 -- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 msgr2=0x7f4ba812bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 --2- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 -- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ba812fe90 msgr2=0x7f4ba812c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 --2- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ba812fe90 0x7f4ba812c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 -- 172.21.15.89:0/3182058337 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba0009a20 con 0x7f4ba8064570 2023-09-30T22:14:24.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4bacef1700 1 --2- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ba812fe90 0x7f4ba812c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.423+0000 7f4ba7fff700 1 --2- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba8064570 0x7f4ba80696a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f4b98006d70 tx=0x7f4b98017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.424+0000 7f4b8ffff700 1 -- 172.21.15.89:0/3182058337 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800f7e0 con 0x7f4ba8064570 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.424+0000 7f4b8ffff700 1 -- 172.21.15.89:0/3182058337 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800f940 con 0x7f4ba8064570 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.424+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b980092a0 con 0x7f4ba8064570 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.424+0000 7f4b8ffff700 1 -- 172.21.15.89:0/3182058337 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9801b660 con 0x7f4ba8064570 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.424+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba812e550 con 0x7f4ba8064570 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.424+0000 7f4b8ffff700 1 -- 172.21.15.89:0/3182058337 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f4b9801b9d0 con 0x7f4ba8064570 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.424+0000 7f4b8ffff700 1 -- 172.21.15.89:0/3182058337 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4b9803c890 con 0x7f4ba8064570 2023-09-30T22:14:24.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.425+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba805a260 con 0x7f4ba8064570 2023-09-30T22:14:24.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.429+0000 7f4b8ffff700 1 -- 172.21.15.89:0/3182058337 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f4b98029080 con 0x7f4ba8064570 2023-09-30T22:14:24.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.587+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ba812afc0 con 0x7f4ba8064570 2023-09-30T22:14:24.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.587+0000 7f4b8ffff700 1 -- 172.21.15.89:0/3182058337 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+11340 (secure 0 0 0) 0x7f4b9801f020 con 0x7f4ba8064570 2023-09-30T22:14:24.588 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:24.588 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:20.090873+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:14:24.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 >> v1:172.21.15.89:6816/118161 conn(0x7f4b9402bb60 legacy=0x7f4b9402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba8064570 msgr2=0x7f4ba80696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 --2- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba8064570 0x7f4ba80696a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f4b98006d70 tx=0x7f4b98017000).stop 2023-09-30T22:14:24.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 shutdown_connections 2023-09-30T22:14:24.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 --2- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ba812fe90 0x7f4ba812c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 --2- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ba8064570 0x7f4ba80696a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 --2- 172.21.15.89:0/3182058337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ba80661b0 0x7f4ba812bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 >> 172.21.15.89:0/3182058337 conn(0x7f4ba809ba50 msgr2=0x7f4ba8128930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:24.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 shutdown_connections 2023-09-30T22:14:24.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.589+0000 7f4bae954700 1 -- 172.21.15.89:0/3182058337 wait complete. 2023-09-30T22:14:24.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-30T22:14:24.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.718+0000 7f258c36b700 1 Processor -- start 2023-09-30T22:14:24.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.718+0000 7f258c36b700 1 -- start start 2023-09-30T22:14:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258c36b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2584125f00 0x7f2584125250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258c36b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f258412db40 0x7f258412ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258c36b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2584098790 con 0x7f2584059070 2023-09-30T22:14:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258c36b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2584059450 con 0x7f2584125f00 2023-09-30T22:14:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258c36b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2584059da0 con 0x7f258412db40 2023-09-30T22:14:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258a908700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f258412db40 0x7f258412ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f2589906700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2584125f00 0x7f2584125250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258a908700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f258412db40 0x7f258412ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58990/0 (socket says 172.21.15.89:58990) 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f2589906700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2584125f00 0x7f2584125250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33608/0 (socket says 172.21.15.89:33608) 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258a908700 1 -- 172.21.15.89:0/560049804 learned_addr learned my addr 172.21.15.89:0/560049804 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.719+0000 7f258a107700 1 --1- 172.21.15.89:0/560049804 >> v1:172.21.15.89:6789/0 conn(0x7f2584059070 0x7f25841257f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44258/0 (socket says 172.21.15.89:44258) 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f2589906700 1 -- 172.21.15.89:0/560049804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f258412db40 msgr2=0x7f258412ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f2589906700 1 --2- 172.21.15.89:0/560049804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f258412db40 0x7f258412ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f2589906700 1 -- 172.21.15.89:0/560049804 >> v1:172.21.15.89:6789/0 conn(0x7f2584059070 legacy=0x7f25841257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f2589906700 1 -- 172.21.15.89:0/560049804 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840996f0 con 0x7f2584125f00 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258a908700 1 --2- 172.21.15.89:0/560049804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f258412db40 0x7f258412ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f2589906700 1 --2- 172.21.15.89:0/560049804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584125f00 0x7f2584125250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f2580009760 tx=0x7f2580009060).ready entity=mon.0 client_cookie=5ce768b145787db4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f2589105700 1 -- 172.21.15.89:0/560049804 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25800090d0 con 0x7f2584125f00 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f2589105700 1 -- 172.21.15.89:0/560049804 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258000f510 con 0x7f2584125f00 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f2589105700 1 -- 172.21.15.89:0/560049804 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258000f890 con 0x7f2584125f00 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258c36b700 1 -- 172.21.15.89:0/560049804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584125f00 msgr2=0x7f2584125250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258c36b700 1 --2- 172.21.15.89:0/560049804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584125f00 0x7f2584125250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f2580009760 tx=0x7f2580009060).stop 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258c36b700 1 -- 172.21.15.89:0/560049804 shutdown_connections 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258c36b700 1 --2- 172.21.15.89:0/560049804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f258412db40 0x7f258412ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258c36b700 1 --2- 172.21.15.89:0/560049804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584125f00 0x7f2584125250 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258c36b700 1 -- 172.21.15.89:0/560049804 >> 172.21.15.89:0/560049804 conn(0x7f2584097480 msgr2=0x7f2584097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258c36b700 1 -- 172.21.15.89:0/560049804 shutdown_connections 2023-09-30T22:14:24.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.720+0000 7f258c36b700 1 -- 172.21.15.89:0/560049804 wait complete. 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.721+0000 7f258c36b700 1 Processor -- start 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.721+0000 7f258c36b700 1 -- start start 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258c36b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584059070 0x7f258412bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258c36b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2584125f00 0x7f258412c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258c36b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f258412db40 0x7f258412c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258c36b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2584098ea0 con 0x7f2584059070 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258c36b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2584069a40 con 0x7f258412db40 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258c36b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f258406a200 con 0x7f2584125f00 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258a107700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584059070 0x7f258412bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2584125f00 0x7f258412c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2584125f00 0x7f258412c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59006/0 (socket says 172.21.15.89:59006) 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258a107700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584059070 0x7f258412bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33618/0 (socket says 172.21.15.89:33618) 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 -- 172.21.15.89:0/183555238 learned_addr learned my addr 172.21.15.89:0/183555238 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258a908700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f258412db40 0x7f258412c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 -- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f258412db40 msgr2=0x7f258412c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f258412db40 0x7f258412c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 -- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584059070 msgr2=0x7f258412bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584059070 0x7f258412bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 -- 172.21.15.89:0/183555238 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c009ae0 con 0x7f2584125f00 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258a107700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584059070 0x7f258412bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:24.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f2589906700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2584125f00 0x7f258412c4a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2584098920 tx=0x7f2580017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f25767fc700 1 -- 172.21.15.89:0/183555238 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258000fea0 con 0x7f2584125f00 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f25767fc700 1 -- 172.21.15.89:0/183555238 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258000f680 con 0x7f2584125f00 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f25767fc700 1 -- 172.21.15.89:0/183555238 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258001b410 con 0x7f2584125f00 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25800092a0 con 0x7f2584125f00 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.722+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258412bab0 con 0x7f2584125f00 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.723+0000 7f25767fc700 1 -- 172.21.15.89:0/183555238 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f258001b800 con 0x7f2584125f00 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.723+0000 7f25767fc700 1 -- 172.21.15.89:0/183555238 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f258003c8c0 con 0x7f2584125f00 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.723+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258405a260 con 0x7f2584125f00 2023-09-30T22:14:24.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.727+0000 7f25767fc700 1 -- 172.21.15.89:0/183555238 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f258001f020 con 0x7f2584125f00 2023-09-30T22:14:24.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:24.924+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f258409a3e0 con 0x7f2584125f00 2023-09-30T22:14:25.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.107+0000 7f25767fc700 1 -- 172.21.15.89:0/183555238 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f258002c940 con 0x7f2584125f00 2023-09-30T22:14:25.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.109+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 >> v1:172.21.15.89:6816/118161 conn(0x7f256802bb60 legacy=0x7f256802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:25.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.109+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2584125f00 msgr2=0x7f258412c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:25.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.109+0000 7f258c36b700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2584125f00 0x7f258412c4a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2584098920 tx=0x7f2580017000).stop 2023-09-30T22:14:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.109+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 shutdown_connections 2023-09-30T22:14:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.109+0000 7f258c36b700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f258412db40 0x7f258412c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.109+0000 7f258c36b700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2584059070 0x7f258412bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.109+0000 7f258c36b700 1 --2- 172.21.15.89:0/183555238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2584125f00 0x7f258412c4a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.109+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 >> 172.21.15.89:0/183555238 conn(0x7f2584097480 msgr2=0x7f25841286a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.110+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 shutdown_connections 2023-09-30T22:14:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.110+0000 7f258c36b700 1 -- 172.21.15.89:0/183555238 wait complete. 2023-09-30T22:14:25.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2023-09-30T22:14:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.239+0000 7f74335d5700 1 Processor -- start 2023-09-30T22:14:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.239+0000 7f74335d5700 1 -- start start 2023-09-30T22:14:25.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f74335d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c093840 0x7f742c098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:25.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f74335d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c098c80 0x7f742c099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f74335d5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f742c12c150 0x7f742c12e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f74335d5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742c093c20 con 0x7f742c12c150 2023-09-30T22:14:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f74335d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c092b90 con 0x7f742c093840 2023-09-30T22:14:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f74335d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f742c092d80 con 0x7f742c098c80 2023-09-30T22:14:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431371700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c093840 0x7f742c098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7430b70700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c098c80 0x7f742c099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431b72700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f742c12c150 0x7f742c12e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431371700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c093840 0x7f742c098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41300/0 (socket says 172.21.15.89:41300) 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7430b70700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c098c80 0x7f742c099060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59014/0 (socket says 172.21.15.89:59014) 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431371700 1 -- 172.21.15.89:0/2380172172 learned_addr learned my addr 172.21.15.89:0/2380172172 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431b72700 1 -- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c098c80 msgr2=0x7f742c099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431b72700 1 --2- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c098c80 0x7f742c099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431b72700 1 -- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c093840 msgr2=0x7f742c098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431b72700 1 --2- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c093840 0x7f742c098790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431b72700 1 -- 172.21.15.89:0/2380172172 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c099b70 con 0x7f742c12c150 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431371700 1 --2- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c093840 0x7f742c098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:25.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7431b72700 1 --2- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c12c150 0x7f742c12e540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f7424009e60 tx=0x7f7424009780).ready entity=mon.0 client_cookie=5d667ea3726fac2a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7423fff700 1 -- 172.21.15.89:0/2380172172 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742400e070 con 0x7f742c12c150 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7423fff700 1 -- 172.21.15.89:0/2380172172 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742400bab0 con 0x7f742c12c150 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.240+0000 7f7423fff700 1 -- 172.21.15.89:0/2380172172 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742400be30 con 0x7f742c12c150 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 -- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c12c150 msgr2=0x7f742c12e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 --2- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c12c150 0x7f742c12e540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f7424009e60 tx=0x7f7424009780).stop 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 -- 172.21.15.89:0/2380172172 shutdown_connections 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 --2- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c12c150 0x7f742c12e540 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 --2- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c093840 0x7f742c098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 --2- 172.21.15.89:0/2380172172 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c098c80 0x7f742c099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 -- 172.21.15.89:0/2380172172 >> 172.21.15.89:0/2380172172 conn(0x7f742c097480 msgr2=0x7f742c097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:25.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 -- 172.21.15.89:0/2380172172 shutdown_connections 2023-09-30T22:14:25.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.241+0000 7f74335d5700 1 -- 172.21.15.89:0/2380172172 wait complete. 2023-09-30T22:14:25.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f74335d5700 1 Processor -- start 2023-09-30T22:14:25.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f74335d5700 1 -- start start 2023-09-30T22:14:25.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f74335d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c093840 0x7f742c0679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f74335d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c098c80 0x7f742c067ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f74335d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c12c150 0x7f742c068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f74335d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c06a500 con 0x7f742c098c80 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f74335d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c064c20 con 0x7f742c12c150 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f74335d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f742c126950 con 0x7f742c093840 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.242+0000 7f7431371700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c093840 0x7f742c0679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7430b70700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c098c80 0x7f742c067ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431371700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c093840 0x7f742c0679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59022/0 (socket says 172.21.15.89:59022) 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431371700 1 -- 172.21.15.89:0/2990373998 learned_addr learned my addr 172.21.15.89:0/2990373998 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431b72700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c12c150 0x7f742c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:25.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431371700 1 -- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c12c150 msgr2=0x7f742c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431371700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c12c150 0x7f742c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431371700 1 -- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c098c80 msgr2=0x7f742c067ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431371700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c098c80 0x7f742c067ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431371700 1 -- 172.21.15.89:0/2990373998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c009370 con 0x7f742c093840 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431b72700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c12c150 0x7f742c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7430b70700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c098c80 0x7f742c067ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f7431371700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c093840 0x7f742c0679b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f741c00ea20 tx=0x7f741c0090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f74217fa700 1 -- 172.21.15.89:0/2990373998 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c009650 con 0x7f742c093840 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f74217fa700 1 -- 172.21.15.89:0/2990373998 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c00ee60 con 0x7f742c093840 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7424009a20 con 0x7f742c093840 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f74217fa700 1 -- 172.21.15.89:0/2990373998 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00ca60 con 0x7f742c093840 2023-09-30T22:14:25.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.243+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c116390 con 0x7f742c093840 2023-09-30T22:14:25.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.244+0000 7f74217fa700 1 -- 172.21.15.89:0/2990373998 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f741c01b400 con 0x7f742c093840 2023-09-30T22:14:25.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.244+0000 7f74217fa700 1 -- 172.21.15.89:0/2990373998 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f741c03b460 con 0x7f742c093840 2023-09-30T22:14:25.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.244+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c115f30 con 0x7f742c093840 2023-09-30T22:14:25.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.248+0000 7f74217fa700 1 -- 172.21.15.89:0/2990373998 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f741c016020 con 0x7f742c093840 2023-09-30T22:14:25.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:25.443+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f742c05a260 con 0x7f742c093840 2023-09-30T22:14:26.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.111+0000 7f74217fa700 1 -- 172.21.15.89:0/2990373998 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v7) v1 ==== 105+0+0 (secure 0 0 0) 0x7f741c019310 con 0x7f742c093840 2023-09-30T22:14:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 >> v1:172.21.15.89:6816/118161 conn(0x7f741002bb60 legacy=0x7f741002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c093840 msgr2=0x7f742c0679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c093840 0x7f742c0679b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f741c00ea20 tx=0x7f741c0090f0).stop 2023-09-30T22:14:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 shutdown_connections 2023-09-30T22:14:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f742c12c150 0x7f742c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f742c093840 0x7f742c0679b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 --2- 172.21.15.89:0/2990373998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c098c80 0x7f742c067ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 >> 172.21.15.89:0/2990373998 conn(0x7f742c097480 msgr2=0x7f742c12b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 shutdown_connections 2023-09-30T22:14:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.113+0000 7f74335d5700 1 -- 172.21.15.89:0/2990373998 wait complete. 2023-09-30T22:14:26.127 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:14:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e8ab46700 1 Processor -- start 2023-09-30T22:14:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e8ab46700 1 -- start start 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e8ab46700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e841251c0 0x7f0e8412da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e8ab46700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e8ab46700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e84095f00 con 0x7f0e84059070 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e8ab46700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e84059450 con 0x7f0e841251c0 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e8ab46700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84096190 con 0x7f0e84125dd0 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e83fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e841251c0 0x7f0e8412da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e890e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e83fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e841251c0 0x7f0e8412da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33634/0 (socket says 172.21.15.89:33634) 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e888e2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0e84059070 0x7f0e841256c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44266/0 (socket says 172.21.15.89:44266) 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e83fff700 1 -- 172.21.15.89:0/3369535545 learned_addr learned my addr 172.21.15.89:0/3369535545 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.242+0000 7f0e890e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59034/0 (socket says 172.21.15.89:59034) 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e83fff700 1 -- 172.21.15.89:0/3369535545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 msgr2=0x7f0e8412ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e83fff700 1 --2- 172.21.15.89:0/3369535545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e83fff700 1 -- 172.21.15.89:0/3369535545 >> v1:172.21.15.89:6789/0 conn(0x7f0e84059070 legacy=0x7f0e841256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e83fff700 1 -- 172.21.15.89:0/3369535545 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84059dd0 con 0x7f0e841251c0 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e837fe700 1 -- 172.21.15.89:0/3369535545 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486621148 0 0) 0x7f0e84095f00 con 0x7f0e84059070 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e890e3700 1 --2- 172.21.15.89:0/3369535545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e83fff700 1 --2- 172.21.15.89:0/3369535545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 0x7f0e8412da80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f0e70006fd0 tx=0x7f0e700090f0).ready entity=mon.0 client_cookie=c5dc3318dfcfaa06 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e837fe700 1 -- 172.21.15.89:0/3369535545 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000b580 con 0x7f0e841251c0 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e837fe700 1 -- 172.21.15.89:0/3369535545 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7000b6e0 con 0x7f0e841251c0 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e837fe700 1 -- 172.21.15.89:0/3369535545 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000ba60 con 0x7f0e841251c0 2023-09-30T22:14:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e8ab46700 1 -- 172.21.15.89:0/3369535545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 msgr2=0x7f0e8412da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.243+0000 7f0e8ab46700 1 --2- 172.21.15.89:0/3369535545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 0x7f0e8412da80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f0e70006fd0 tx=0x7f0e700090f0).stop 2023-09-30T22:14:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.244+0000 7f0e8ab46700 1 -- 172.21.15.89:0/3369535545 shutdown_connections 2023-09-30T22:14:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.244+0000 7f0e8ab46700 1 --2- 172.21.15.89:0/3369535545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.244+0000 7f0e8ab46700 1 --2- 172.21.15.89:0/3369535545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 0x7f0e8412da80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.244+0000 7f0e8ab46700 1 -- 172.21.15.89:0/3369535545 >> 172.21.15.89:0/3369535545 conn(0x7f0e84097470 msgr2=0x7f0e84097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.244+0000 7f0e8ab46700 1 -- 172.21.15.89:0/3369535545 shutdown_connections 2023-09-30T22:14:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.244+0000 7f0e8ab46700 1 -- 172.21.15.89:0/3369535545 wait complete. 2023-09-30T22:14:26.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.244+0000 7f0e8ab46700 1 Processor -- start 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e8ab46700 1 -- start start 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e8ab46700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e84059070 0x7f0e8412f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e8ab46700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 0x7f0e8412dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e8ab46700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e8ab46700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8412b1b0 con 0x7f0e841251c0 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e8ab46700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8412b3a0 con 0x7f0e84059070 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e8ab46700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840958d0 con 0x7f0e84125dd0 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e888e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e84059070 0x7f0e8412f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e83fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 0x7f0e8412dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e890e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e888e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e84059070 0x7f0e8412f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41306/0 (socket says 172.21.15.89:41306) 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e888e2700 1 -- 172.21.15.89:0/1274982419 learned_addr learned my addr 172.21.15.89:0/1274982419 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e83fff700 1 -- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 msgr2=0x7f0e8412e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e83fff700 1 --2- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e83fff700 1 -- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e84059070 msgr2=0x7f0e8412f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e83fff700 1 --2- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e84059070 0x7f0e8412f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e83fff700 1 -- 172.21.15.89:0/1274982419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c009a20 con 0x7f0e841251c0 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.245+0000 7f0e890e3700 1 --2- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.246+0000 7f0e83fff700 1 --2- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 0x7f0e8412dc80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0e70014000 tx=0x7f0e70017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.246+0000 7f0e80ff9700 1 -- 172.21.15.89:0/1274982419 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70000bc0 con 0x7f0e841251c0 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.246+0000 7f0e80ff9700 1 -- 172.21.15.89:0/1274982419 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7000b850 con 0x7f0e841251c0 2023-09-30T22:14:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.246+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e70009370 con 0x7f0e841251c0 2023-09-30T22:14:26.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.246+0000 7f0e80ff9700 1 -- 172.21.15.89:0/1274982419 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70000d20 con 0x7f0e841251c0 2023-09-30T22:14:26.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.246+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8412d0a0 con 0x7f0e841251c0 2023-09-30T22:14:26.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.247+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8405a260 con 0x7f0e841251c0 2023-09-30T22:14:26.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.247+0000 7f0e80ff9700 1 -- 172.21.15.89:0/1274982419 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0e7001cb50 con 0x7f0e841251c0 2023-09-30T22:14:26.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.247+0000 7f0e80ff9700 1 -- 172.21.15.89:0/1274982419 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0e7003d330 con 0x7f0e841251c0 2023-09-30T22:14:26.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.251+0000 7f0e80ff9700 1 -- 172.21.15.89:0/1274982419 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0e7001ce00 con 0x7f0e841251c0 2023-09-30T22:14:26.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.447+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e84046c20 con 0x7f0e841251c0 2023-09-30T22:14:26.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.447+0000 7f0e80ff9700 1 -- 172.21.15.89:0/1274982419 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f0e7002a030 con 0x7f0e841251c0 2023-09-30T22:14:26.448 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:14:26.448 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:26.107818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:14:26.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 >> v1:172.21.15.89:6816/118161 conn(0x7f0e6802bb90 legacy=0x7f0e6802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 msgr2=0x7f0e8412dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 --2- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 0x7f0e8412dc80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0e70014000 tx=0x7f0e70017000).stop 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 shutdown_connections 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 --2- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e84125dd0 0x7f0e8412e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 --2- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e84059070 0x7f0e8412f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 --2- 172.21.15.89:0/1274982419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e841251c0 0x7f0e8412dc80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 >> 172.21.15.89:0/1274982419 conn(0x7f0e84097470 msgr2=0x7f0e8409a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 shutdown_connections 2023-09-30T22:14:26.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.449+0000 7f0e8ab46700 1 -- 172.21.15.89:0/1274982419 wait complete. 2023-09-30T22:14:26.451 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 7 2023-09-30T22:14:26.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:14:02.037584+0000', 'modified': '2023-09-30T22:14:26.107818+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4190': {'gid': 4190, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6817/3539463463', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.89:6817', 'nonce': 3539463463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-30T22:14:26.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:14:26.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:14:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.677+0000 7f4da3d97700 1 Processor -- start 2023-09-30T22:14:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.677+0000 7f4da3d97700 1 -- start start 2023-09-30T22:14:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da3d97700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d9c096820 0x7f4d9c096c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da3d97700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c130280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da3d97700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c068df0 con 0x7f4d9c067940 2023-09-30T22:14:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da3d97700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9c097c90 con 0x7f4d9c096820 2023-09-30T22:14:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da3d97700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c06a4c0 con 0x7f4d9c097800 2023-09-30T22:14:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1b33700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d9c096820 0x7f4d9c096c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da2334700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c130280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1b33700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d9c096820 0x7f4d9c096c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33648/0 (socket says 172.21.15.89:33648) 2023-09-30T22:14:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1332700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4d9c067940 0x7f4d9c0970f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44268/0 (socket says 172.21.15.89:44268) 2023-09-30T22:14:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1b33700 1 -- 172.21.15.89:0/695256849 learned_addr learned my addr 172.21.15.89:0/695256849 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1b33700 1 -- 172.21.15.89:0/695256849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 msgr2=0x7f4d9c130280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1b33700 1 --2- 172.21.15.89:0/695256849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c130280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1b33700 1 -- 172.21.15.89:0/695256849 >> v1:172.21.15.89:6789/0 conn(0x7f4d9c067940 legacy=0x7f4d9c0970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1b33700 1 -- 172.21.15.89:0/695256849 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c098260 con 0x7f4d9c096820 2023-09-30T22:14:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da0b31700 1 -- 172.21.15.89:0/695256849 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388090805 0 0) 0x7f4d9c068df0 con 0x7f4d9c067940 2023-09-30T22:14:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da2334700 1 --2- 172.21.15.89:0/695256849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c130280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da1b33700 1 --2- 172.21.15.89:0/695256849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 0x7f4d9c096c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4d8c006fd0 tx=0x7f4d8c0090f0).ready entity=mon.0 client_cookie=e7a7472c8fca181e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.678+0000 7f4da0b31700 1 -- 172.21.15.89:0/695256849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00b580 con 0x7f4d9c096820 2023-09-30T22:14:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da0b31700 1 -- 172.21.15.89:0/695256849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c00b6e0 con 0x7f4d9c096820 2023-09-30T22:14:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da0b31700 1 -- 172.21.15.89:0/695256849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00ba60 con 0x7f4d9c096820 2023-09-30T22:14:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da3d97700 1 -- 172.21.15.89:0/695256849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 msgr2=0x7f4d9c096c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da3d97700 1 --2- 172.21.15.89:0/695256849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 0x7f4d9c096c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4d8c006fd0 tx=0x7f4d8c0090f0).stop 2023-09-30T22:14:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da3d97700 1 -- 172.21.15.89:0/695256849 shutdown_connections 2023-09-30T22:14:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da3d97700 1 --2- 172.21.15.89:0/695256849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 0x7f4d9c096c00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da3d97700 1 --2- 172.21.15.89:0/695256849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c130280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da3d97700 1 -- 172.21.15.89:0/695256849 >> 172.21.15.89:0/695256849 conn(0x7f4d9c09ba50 msgr2=0x7f4d9c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da3d97700 1 -- 172.21.15.89:0/695256849 shutdown_connections 2023-09-30T22:14:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.679+0000 7f4da3d97700 1 -- 172.21.15.89:0/695256849 wait complete. 2023-09-30T22:14:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da3d97700 1 Processor -- start 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da3d97700 1 -- start start 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da3d97700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d9c067940 0x7f4d9c115000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da3d97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 0x7f4d9c1154f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da3d97700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c1159e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da3d97700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c115ed0 con 0x7f4d9c096820 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da3d97700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c1164b0 con 0x7f4d9c067940 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da3d97700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c118500 con 0x7f4d9c097800 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da2334700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c1159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da1b33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d9c067940 0x7f4d9c115000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da2334700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c1159e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59056/0 (socket says 172.21.15.89:59056) 2023-09-30T22:14:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da1332700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 0x7f4d9c1154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.680+0000 7f4da2334700 1 -- 172.21.15.89:0/1280975438 learned_addr learned my addr 172.21.15.89:0/1280975438 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da1b33700 1 -- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 msgr2=0x7f4d9c1159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da1b33700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c1159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da1b33700 1 -- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 msgr2=0x7f4d9c1154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da1b33700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 0x7f4d9c1154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da1b33700 1 -- 172.21.15.89:0/1280975438 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d94009a20 con 0x7f4d9c067940 2023-09-30T22:14:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da2334700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c1159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da1332700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 0x7f4d9c1154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da1b33700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d9c067940 0x7f4d9c115000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4d9c06aa30 tx=0x7f4d8c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4d91ffb700 1 -- 172.21.15.89:0/1280975438 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c000bc0 con 0x7f4d9c067940 2023-09-30T22:14:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4d91ffb700 1 -- 172.21.15.89:0/1280975438 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c00b850 con 0x7f4d9c067940 2023-09-30T22:14:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da3d97700 1 -- 172.21.15.89:0/1280975438 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8c009370 con 0x7f4d9c067940 2023-09-30T22:14:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4d91ffb700 1 -- 172.21.15.89:0/1280975438 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c000d20 con 0x7f4d9c067940 2023-09-30T22:14:26.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.681+0000 7f4da3d97700 1 -- 172.21.15.89:0/1280975438 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c067e30 con 0x7f4d9c067940 2023-09-30T22:14:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.682+0000 7f4d91ffb700 1 -- 172.21.15.89:0/1280975438 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f4d8c01c8c0 con 0x7f4d9c067940 2023-09-30T22:14:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.682+0000 7f4d91ffb700 1 -- 172.21.15.89:0/1280975438 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4d8c03cc10 con 0x7f4d9c067940 2023-09-30T22:14:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4db574ab80 1 Processor -- start 2023-09-30T22:14:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4db574ab80 1 -- start start 2023-09-30T22:14:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b0982610 con 0x55c8b09bbd70 2023-09-30T22:14:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8b08f8db0 con 0x55c8b09d2e60 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c8b090e030 con 0x55c8b09d2a60 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1b33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1b33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33668/0 (socket says 172.21.15.89:33668) 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1b33700 1 -- 172.21.15.89:0/1411793794 learned_addr learned my addr 172.21.15.89:0/1411793794 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1332700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da2334700 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1b33700 1 -- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 msgr2=0x55c8b09b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1b33700 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1b33700 1 -- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 msgr2=0x55c8b09b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1b33700 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1b33700 1 -- 172.21.15.89:0/1411793794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b07a9710 con 0x55c8b09bbd70 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da1332700 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.685+0000 7f4da2334700 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4da1b33700 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09d2570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f4d8c01b6b0 tx=0x7f4d8c01d000).ready entity=mon.0 client_cookie=9aae3493db459ab server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4d86ffd700 1 -- 172.21.15.89:0/1411793794 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c02d710 con 0x55c8b09bbd70 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4d86ffd700 1 -- 172.21.15.89:0/1411793794 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c02d530 con 0x55c8b09bbd70 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4d86ffd700 1 -- 172.21.15.89:0/1411793794 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c02d350 con 0x55c8b09bbd70 2023-09-30T22:14:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 -- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 msgr2=0x55c8b09d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09d2570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f4d8c01b6b0 tx=0x7f4d8c01d000).stop 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 -- 172.21.15.89:0/1411793794 shutdown_connections 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09d2570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 --2- 172.21.15.89:0/1411793794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 -- 172.21.15.89:0/1411793794 >> 172.21.15.89:0/1411793794 conn(0x55c8b0826940 msgr2=0x55c8b09d8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 -- 172.21.15.89:0/1411793794 shutdown_connections 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.686+0000 7f4db574ab80 1 -- 172.21.15.89:0/1411793794 wait complete. 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.687+0000 7f4db574ab80 1 -- start start 2023-09-30T22:14:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.687+0000 7f4db574ab80 1 Processor -- start 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b090e170 con 0x55c8b09bbd70 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8b090e3f0 con 0x55c8b09d2e60 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c8b08d9920 con 0x55c8b09d2a60 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4db574ab80 10 client.0 ll_register_callbacks cb 0x55c8b08aece0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:14:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33678/0 (socket says 172.21.15.89:33678) 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 -- 172.21.15.89:0/2356547099 learned_addr learned my addr 172.21.15.89:0/2356547099 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1332700 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da2334700 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 -- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 msgr2=0x55c8b09c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 -- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 msgr2=0x55c8b09c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 -- 172.21.15.89:0/2356547099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c067e30 con 0x55c8b09bbd70 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da2334700 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:14:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4da1b33700 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09c4070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f4d8c02d8d0 tx=0x7f4d8c031040).ready entity=mon.0 client_cookie=b2e50ea76cb8b79c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.688+0000 7f4d7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4d7bfff700 1 -- 172.21.15.89:0/2356547099 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c023e90 con 0x55c8b09bbd70 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4d7bfff700 1 -- 172.21.15.89:0/2356547099 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4d8c03b0e0 con 0x55c8b09bbd70 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4db574ab80 1 -- 172.21.15.89:0/2356547099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8b09b0f20 con 0x55c8b09bbd70 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4d7bfff700 1 -- 172.21.15.89:0/2356547099 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d8c01a390 con 0x55c8b09bbd70 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4db574ab80 10 client.4761 fetch_fsmap learned FSMap version 7 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4db574ab80 1 -- 172.21.15.89:0/2356547099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55c8b08e0b00 con 0x55c8b09bbd70 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4d7bfff700 1 -- 172.21.15.89:0/2356547099 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8b09b0f20 con 0x55c8b09bbd70 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4d7bfff700 1 -- 172.21.15.89:0/2356547099 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 7) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f4d8c043050 con 0x55c8b09bbd70 2023-09-30T22:14:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4db574ab80 10 client.4761 fetch_fsmap finished waiting for FSMap version 7 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4db574ab80 10 client.4761 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4db574ab80 20 client.4761 populate_metadata read hostname 'smithi089' 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4db574ab80 4 client.4761 mds_command: new command op to 4190 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4db574ab80 1 -- 172.21.15.89:0/2356547099 --> v1:172.21.15.89:6817/3539463463 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8b0986250 con 0x55c8b09f4790 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.689+0000 7f4d7bfff700 10 client.4761 ms_handle_connect on v1:172.21.15.89:6817/3539463463 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4d7bfff700 1 -- 172.21.15.89:0/2356547099 <== mds.0 v1:172.21.15.89:6817/3539463463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x55c8b0986250 con 0x55c8b09f4790 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4d7bfff700 10 client.4761 handle_command_reply: tid=0 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 client.4761 shutdown 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 -- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 msgr2=0x55c8b09c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09c4070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f4d8c02d8d0 tx=0x7f4d8c031040).stop 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 -- 172.21.15.89:0/2356547099 shutdown_connections 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09bbd70 0x55c8b09c4070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4d7bfff700 0 client.4761 ms_handle_reset on v1:172.21.15.89:6817/3539463463 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 --2- 172.21.15.89:0/2356547099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 -- 172.21.15.89:0/2356547099 >> 172.21.15.89:0/2356547099 conn(0x55c8b0826940 msgr2=0x55c8b09d4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 -- 172.21.15.89:0/2356547099 shutdown_connections 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 1 -- 172.21.15.89:0/2356547099 wait complete. 2023-09-30T22:14:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.690+0000 7f4db574ab80 20 client.4761 trim_cache size 0 max 16384 2023-09-30T22:14:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4db574ab80 1 Processor -- start 2023-09-30T22:14:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4db574ab80 1 -- start start 2023-09-30T22:14:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09bbd70 0x55c8b09c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2e60 0x55c8b09f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b09ba3e0 con 0x55c8b09d2e60 2023-09-30T22:14:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8b09c7680 con 0x55c8b09bbd70 2023-09-30T22:14:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c8b09c5a20 con 0x55c8b09d2a60 2023-09-30T22:14:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da1332700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da2334700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2e60 0x55c8b09f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da1332700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59088/0 (socket says 172.21.15.89:59088) 2023-09-30T22:14:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da2334700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2e60 0x55c8b09f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33686/0 (socket says 172.21.15.89:33686) 2023-09-30T22:14:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da1332700 1 -- 172.21.15.89:0/3117314414 learned_addr learned my addr 172.21.15.89:0/3117314414 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da1b33700 1 --2- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09bbd70 0x55c8b09c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da1332700 1 -- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09bbd70 msgr2=0x55c8b09c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da1332700 1 --2- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09bbd70 0x55c8b09c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da1332700 1 -- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2e60 msgr2=0x55c8b09f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.707+0000 7f4da1332700 1 --2- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2e60 0x55c8b09f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4da1332700 1 -- 172.21.15.89:0/3117314414 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b0906f70 con 0x55c8b09d2a60 2023-09-30T22:14:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4da1332700 1 --2- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c3a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f4d9c116700 tx=0x7f4d980073f0).ready entity=mon.2 client_cookie=ce1a24eff478e146 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4d877fe700 1 -- 172.21.15.89:0/3117314414 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9801a0d0 con 0x55c8b09d2a60 2023-09-30T22:14:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4d877fe700 1 -- 172.21.15.89:0/3117314414 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98006dc0 con 0x55c8b09d2a60 2023-09-30T22:14:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4d877fe700 1 -- 172.21.15.89:0/3117314414 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d980046b0 con 0x55c8b09d2a60 2023-09-30T22:14:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 -- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 msgr2=0x55c8b09c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 --2- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c3a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f4d9c116700 tx=0x7f4d980073f0).stop 2023-09-30T22:14:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 -- 172.21.15.89:0/3117314414 shutdown_connections 2023-09-30T22:14:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 --2- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2e60 0x55c8b09f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 --2- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09bbd70 0x55c8b09c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 --2- 172.21.15.89:0/3117314414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09d2a60 0x55c8b09c3a30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 -- 172.21.15.89:0/3117314414 >> 172.21.15.89:0/3117314414 conn(0x55c8b0826940 msgr2=0x55c8b09bb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 -- 172.21.15.89:0/3117314414 shutdown_connections 2023-09-30T22:14:26.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.708+0000 7f4db574ab80 1 -- 172.21.15.89:0/3117314414 wait complete. 2023-09-30T22:14:26.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.709+0000 7f4db574ab80 1 -- start start 2023-09-30T22:14:26.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.709+0000 7f4db574ab80 1 Processor -- start 2023-09-30T22:14:26.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.709+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09bbd70 0x55c8b09f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.709+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2a60 0x55c8b09f7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.709+0000 7f4db574ab80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09f7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:14:26.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.709+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b09d6470 con 0x55c8b09d2a60 2023-09-30T22:14:26.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.709+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8b09f7a50 con 0x55c8b09d2e60 2023-09-30T22:14:26.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.709+0000 7f4db574ab80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c8b09cc520 con 0x55c8b09bbd70 2023-09-30T22:14:26.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4db574ab80 10 client.0 ll_register_callbacks cb 0x55c8b08aece0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:14:26.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41358/0 (socket says 172.21.15.89:41358) 2023-09-30T22:14:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 -- 172.21.15.89:0/324849659 learned_addr learned my addr 172.21.15.89:0/324849659 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:14:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da1b33700 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09bbd70 0x55c8b09f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da1332700 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2a60 0x55c8b09f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:14:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 -- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09bbd70 msgr2=0x55c8b09f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09bbd70 0x55c8b09f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 -- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2a60 msgr2=0x55c8b09f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2a60 0x55c8b09f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 -- 172.21.15.89:0/324849659 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9800f040 con 0x55c8b09d2e60 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da1332700 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2a60 0x55c8b09f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4da2334700 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09f7560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4d94018e20 tx=0x7f4d9400b040).ready entity=mon.1 client_cookie=a64ecb4b64d0dc75 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4d7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3301/0 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4d7bfff700 1 -- 172.21.15.89:0/324849659 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9401d070 con 0x55c8b09d2e60 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4d7bfff700 1 -- 172.21.15.89:0/324849659 <== mon.1 v2:172.21.15.89:3301/0 2 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4d94015c10 con 0x55c8b09d2e60 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4db574ab80 1 -- 172.21.15.89:0/324849659 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d8c02d530 con 0x55c8b09d2e60 2023-09-30T22:14:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4d7bfff700 1 -- 172.21.15.89:0/324849659 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d94022410 con 0x55c8b09d2e60 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4db574ab80 10 client.4591 fetch_fsmap learned FSMap version 7 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.710+0000 7f4db574ab80 1 -- 172.21.15.89:0/324849659 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55c8b0902580 con 0x55c8b09d2e60 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4d7bfff700 1 -- 172.21.15.89:0/324849659 <== mon.1 v2:172.21.15.89:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c02d530 con 0x55c8b09d2e60 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4d7bfff700 1 -- 172.21.15.89:0/324849659 <== mon.1 v2:172.21.15.89:3301/0 5 ==== fsmap(e 7) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f4d94022b80 con 0x55c8b09d2e60 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 10 client.4591 fetch_fsmap finished waiting for FSMap version 7 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 10 client.4591 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 20 client.4591 populate_metadata read hostname 'smithi089' 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 4 client.4591 mds_command: new command op to 4190 tid=0[{"prefix": "status"}] 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 -- 172.21.15.89:0/324849659 --> v1:172.21.15.89:6817/3539463463 -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c8b0776ef0 con 0x55c8b0a47440 2023-09-30T22:14:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4d7bfff700 10 client.4591 ms_handle_connect on v1:172.21.15.89:6817/3539463463 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4d7bfff700 1 -- 172.21.15.89:0/324849659 <== mds.0 v1:172.21.15.89:6817/3539463463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (unknown 0 0 2536609573) 0x55c8b0776ef0 con 0x55c8b0a47440 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4d7bfff700 10 client.4591 handle_command_reply: tid=0 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 client.4591 shutdown 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 -- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 msgr2=0x55c8b09f7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09f7560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4d94018e20 tx=0x7f4d9400b040).stop 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 -- 172.21.15.89:0/324849659 shutdown_connections 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c8b09d2e60 0x55c8b09f7560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c8b09bbd70 0x55c8b09f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4d7bfff700 0 client.4591 ms_handle_reset on v1:172.21.15.89:6817/3539463463 2023-09-30T22:14:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 --2- 172.21.15.89:0/324849659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b09d2a60 0x55c8b09f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.711+0000 7f4db574ab80 1 -- 172.21.15.89:0/324849659 >> 172.21.15.89:0/324849659 conn(0x55c8b0826940 msgr2=0x55c8b09d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4db574ab80 1 -- 172.21.15.89:0/324849659 shutdown_connections 2023-09-30T22:14:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4db574ab80 1 -- 172.21.15.89:0/324849659 wait complete. 2023-09-30T22:14:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4db574ab80 20 client.4591 trim_cache size 0 max 16384 2023-09-30T22:14:26.724 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-30T22:14:26.724 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "389f6e95-ea58-47bd-a145-778ba0724a9e", 2023-09-30T22:14:26.724 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4190, 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 24.671014881000001, 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 7, 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 18, 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 18, 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 29.069149619000001 2023-09-30T22:14:26.725 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-30T22:14:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 -- 172.21.15.89:0/1280975438 >> v1:172.21.15.89:6816/118161 conn(0x7f4d8002bb60 legacy=0x7f4d8002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 -- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d9c067940 msgr2=0x7f4d9c115000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:14:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d9c067940 0x7f4d9c115000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4d9c06aa30 tx=0x7f4d8c017000).stop 2023-09-30T22:14:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 -- 172.21.15.89:0/1280975438 shutdown_connections 2023-09-30T22:14:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d9c097800 0x7f4d9c1159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d9c067940 0x7f4d9c115000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 --2- 172.21.15.89:0/1280975438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c096820 0x7f4d9c1154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:14:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 -- 172.21.15.89:0/1280975438 >> 172.21.15.89:0/1280975438 conn(0x7f4d9c09ba50 msgr2=0x7f4d9c094090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:14:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 -- 172.21.15.89:0/1280975438 shutdown_connections 2023-09-30T22:14:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:14:26.712+0000 7f4da3d97700 1 -- 172.21.15.89:0/1280975438 wait complete. 2023-09-30T22:14:26.728 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:14:26.738 INFO:teuthology.task.print:**** done mds pre-upgrade sequence 2023-09-30T22:14:26.738 INFO:teuthology.run_tasks:Running task install.upgrade... 2023-09-30T22:14:26.750 INFO:teuthology.task.install:project ceph config {'mon.a': {'branch': 'reef'}} overrides {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'} 2023-09-30T22:14:26.750 INFO:teuthology.task.install:config contains sha1|tag|branch, removing those keys from override 2023-09-30T22:14:26.750 INFO:teuthology.task.install:remote ubuntu@smithi089.front.sepia.ceph.com config {'branch': 'reef', 'flavor': 'default'} 2023-09-30T22:14:26.751 INFO:teuthology.task.install:extra packages: [] 2023-09-30T22:14:26.756 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T22:14:26.756 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2023-09-30T22:14:26.757 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:14:26.779 INFO:teuthology.orchestra.run.smithi089.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:14:26.780 INFO:teuthology.packaging:The installed version of ceph-common is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:14:26.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:14:27.002 INFO:teuthology.task.install:Ceph rpm upgrade from 16.2.14-44.geeb13ac9.el8 to 18.2.0-501.gfc1e90a3 2023-09-30T22:14:27.002 INFO:teuthology.task.install.rpm:Host smithi089 is: centos 8 x86_64 2023-09-30T22:14:27.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:14:27.320 INFO:teuthology.task.install.rpm:Repo base URL: https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/ 2023-09-30T22:14:27.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/yum.repos.d/ceph.repo 2023-09-30T22:14:27.458 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:14:27.458 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-30T22:14:27.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:14:27.487 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:14:27.556 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:14:27.625 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:14:27.683 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2023-09-30T22:14:27.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2023-09-30T22:14:27.964 INFO:teuthology.orchestra.run.smithi089.stdout:71 files removed 2023-09-30T22:14:27.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2023-09-30T22:14:28.774 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 221 kB/s | 80 kB 00:00 2023-09-30T22:14:29.125 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-09-30T22:14:29.416 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 5.7 kB/s | 1.5 kB 00:00 2023-09-30T22:14:31.858 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - AppStream 14 MB/s | 33 MB 00:02 2023-09-30T22:14:41.551 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 48 MB 00:03 2023-09-30T22:14:48.676 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - Extras 116 kB/s | 18 kB 00:00 2023-09-30T22:14:49.410 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 6.9 kB 00:00 2023-09-30T22:14:50.512 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 8 - PowerTools 6.8 MB/s | 6.5 MB 00:00 2023-09-30T22:14:51.972 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-09-30T22:14:53.383 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-09-30T22:14:57.190 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 417 kB/s | 24 kB 00:00 2023-09-30T22:15:01.476 INFO:teuthology.orchestra.run.smithi089.stderr:Package cephadm available, but not installed. 2023-09-30T22:15:01.476 INFO:teuthology.orchestra.run.smithi089.stdout:No match for argument: cephadm 2023-09-30T22:15:01.477 INFO:teuthology.orchestra.run.smithi089.stderr:Package ceph-immutable-object-cache available, but not installed. 2023-09-30T22:15:01.477 INFO:teuthology.orchestra.run.smithi089.stdout:No match for argument: ceph-immutable-object-cache 2023-09-30T22:15:01.489 INFO:teuthology.orchestra.run.smithi089.stderr:Package ceph-mgr-cephadm available, but not installed. 2023-09-30T22:15:01.490 INFO:teuthology.orchestra.run.smithi089.stdout:No match for argument: ceph-mgr-cephadm 2023-09-30T22:15:01.493 INFO:teuthology.orchestra.run.smithi089.stderr:Package ceph-volume available, but not installed. 2023-09-30T22:15:01.493 INFO:teuthology.orchestra.run.smithi089.stdout:No match for argument: ceph-volume 2023-09-30T22:15:01.774 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-09-30T22:15:01.779 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2023-09-30T22:15:01.779 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-09-30T22:15:01.779 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2023-09-30T22:15:01.779 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2023-09-30T22:15:01.779 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 6.4 k 2023-09-30T22:15:01.779 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 5.4 M 2023-09-30T22:15:01.779 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 22 M 2023-09-30T22:15:01.779 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 847 k 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 23 k 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 2.3 M 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 1.5 M 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 1.7 M 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 7.4 M 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 241 k 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 47 k 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 4.6 M 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 18 M 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 15 k 2023-09-30T22:15:01.780 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 13 M 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 24 k 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 47 M 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 30 k 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 769 k 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 137 k 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 139 k 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.7 M 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 474 k 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.6 M 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 6.5 M 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 45 k 2023-09-30T22:15:01.781 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 123 k 2023-09-30T22:15:01.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 190 k 2023-09-30T22:15:01.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 371 k 2023-09-30T22:15:01.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 364 k 2023-09-30T22:15:01.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 107 k 2023-09-30T22:15:01.782 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 94 k 2023-09-30T22:15:01.782 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.4 M 2023-09-30T22:15:01.782 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 179 k 2023-09-30T22:15:01.783 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-09-30T22:15:01.783 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-30T22:15:01.783 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:15:01.783 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-09-30T22:15:01.783 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2023-09-30T22:15:01.783 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-09-30T22:15:01.783 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 34 Packages 2023-09-30T22:15:01.783 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:15:01.784 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 145 M 2023-09-30T22:15:01.784 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-09-30T22:15:02.117 INFO:teuthology.orchestra.run.smithi089.stdout:(1/35): ceph-18.2.0-501.gfc1e90a3.el8.x86_64.rp 44 kB/s | 6.4 kB 00:00 2023-09-30T22:15:02.642 INFO:teuthology.orchestra.run.smithi089.stdout:(2/35): ceph-base-18.2.0-501.gfc1e90a3.el8.x86_ 8.0 MB/s | 5.4 MB 00:00 2023-09-30T22:15:02.834 INFO:teuthology.orchestra.run.smithi089.stdout:(3/35): ceph-fuse-18.2.0-501.gfc1e90a3.el8.x86_ 4.3 MB/s | 847 kB 00:00 2023-09-30T22:15:02.901 INFO:teuthology.orchestra.run.smithi089.stdout:(4/35): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-09-30T22:15:03.385 INFO:teuthology.orchestra.run.smithi089.stdout:(5/35): ceph-mds-18.2.0-501.gfc1e90a3.el8.x86_6 4.1 MB/s | 2.3 MB 00:00 2023-09-30T22:15:03.635 INFO:teuthology.orchestra.run.smithi089.stdout:(6/35): ceph-common-18.2.0-501.gfc1e90a3.el8.x8 14 MB/s | 22 MB 00:01 2023-09-30T22:15:03.777 INFO:teuthology.orchestra.run.smithi089.stdout:(7/35): ceph-mon-18.2.0-501.gfc1e90a3.el8.x86_6 12 MB/s | 4.6 MB 00:00 2023-09-30T22:15:03.869 INFO:teuthology.orchestra.run.smithi089.stdout:(8/35): ceph-mgr-18.2.0-501.gfc1e90a3.el8.x86_6 1.6 MB/s | 1.5 MB 00:00 2023-09-30T22:15:03.969 INFO:teuthology.orchestra.run.smithi089.stdout:(9/35): ceph-selinux-18.2.0-501.gfc1e90a3.el8.x 243 kB/s | 24 kB 00:00 2023-09-30T22:15:05.020 INFO:teuthology.orchestra.run.smithi089.stdout:(10/35): ceph-osd-18.2.0-501.gfc1e90a3.el8.x86_ 13 MB/s | 18 MB 00:01 2023-09-30T22:15:05.079 INFO:teuthology.orchestra.run.smithi089.stdout:(11/35): libcephfs-devel-18.2.0-501.gfc1e90a3.e 524 kB/s | 30 kB 00:00 2023-09-30T22:15:05.264 INFO:teuthology.orchestra.run.smithi089.stdout:(12/35): ceph-radosgw-18.2.0-501.gfc1e90a3.el8. 8.8 MB/s | 13 MB 00:01 2023-09-30T22:15:05.304 INFO:teuthology.orchestra.run.smithi089.stdout:(13/35): libcephfs2-18.2.0-501.gfc1e90a3.el8.x8 3.3 MB/s | 769 kB 00:00 2023-09-30T22:15:05.388 INFO:teuthology.orchestra.run.smithi089.stdout:(14/35): libcephsqlite-18.2.0-501.gfc1e90a3.el8 1.1 MB/s | 137 kB 00:00 2023-09-30T22:15:05.413 INFO:teuthology.orchestra.run.smithi089.stdout:(15/35): librados-devel-18.2.0-501.gfc1e90a3.el 1.2 MB/s | 139 kB 00:00 2023-09-30T22:15:05.538 INFO:teuthology.orchestra.run.smithi089.stdout:(16/35): libradosstriper1-18.2.0-501.gfc1e90a3. 3.7 MB/s | 474 kB 00:00 2023-09-30T22:15:05.674 INFO:teuthology.orchestra.run.smithi089.stdout:(17/35): librados2-18.2.0-501.gfc1e90a3.el8.x86 13 MB/s | 3.7 MB 00:00 2023-09-30T22:15:05.814 INFO:teuthology.orchestra.run.smithi089.stdout:(18/35): librbd1-18.2.0-501.gfc1e90a3.el8.x86_6 13 MB/s | 3.6 MB 00:00 2023-09-30T22:15:05.865 INFO:teuthology.orchestra.run.smithi089.stdout:(19/35): python3-ceph-argparse-18.2.0-501.gfc1e 892 kB/s | 45 kB 00:00 2023-09-30T22:15:05.932 INFO:teuthology.orchestra.run.smithi089.stdout:(20/35): python3-ceph-common-18.2.0-501.gfc1e90 1.8 MB/s | 123 kB 00:00 2023-09-30T22:15:05.999 INFO:teuthology.orchestra.run.smithi089.stdout:(21/35): python3-cephfs-18.2.0-501.gfc1e90a3.el 2.8 MB/s | 190 kB 00:00 2023-09-30T22:15:06.166 INFO:teuthology.orchestra.run.smithi089.stdout:(22/35): librgw2-18.2.0-501.gfc1e90a3.el8.x86_6 13 MB/s | 6.5 MB 00:00 2023-09-30T22:15:06.191 INFO:teuthology.orchestra.run.smithi089.stdout:(23/35): python3-rados-18.2.0-501.gfc1e90a3.el8 1.9 MB/s | 371 kB 00:00 2023-09-30T22:15:06.216 INFO:teuthology.orchestra.run.smithi089.stdout:(24/35): python3-rbd-18.2.0-501.gfc1e90a3.el8.x 7.1 MB/s | 364 kB 00:00 2023-09-30T22:15:06.267 INFO:teuthology.orchestra.run.smithi089.stdout:(25/35): python3-rgw-18.2.0-501.gfc1e90a3.el8.x 1.4 MB/s | 107 kB 00:00 2023-09-30T22:15:06.292 INFO:teuthology.orchestra.run.smithi089.stdout:(26/35): rbd-fuse-18.2.0-501.gfc1e90a3.el8.x86_ 1.2 MB/s | 94 kB 00:00 2023-09-30T22:15:06.426 INFO:teuthology.orchestra.run.smithi089.stdout:(27/35): rbd-nbd-18.2.0-501.gfc1e90a3.el8.x86_6 1.3 MB/s | 179 kB 00:00 2023-09-30T22:15:06.495 INFO:teuthology.orchestra.run.smithi089.stdout:(28/35): rbd-mirror-18.2.0-501.gfc1e90a3.el8.x8 15 MB/s | 3.4 MB 00:00 2023-09-30T22:15:06.510 INFO:teuthology.orchestra.run.smithi089.stdout:(29/35): ceph-grafana-dashboards-18.2.0-501.gfc 275 kB/s | 23 kB 00:00 2023-09-30T22:15:06.664 INFO:teuthology.orchestra.run.smithi089.stdout:(30/35): ceph-mgr-dashboard-18.2.0-501.gfc1e90a 10 MB/s | 1.7 MB 00:00 2023-09-30T22:15:06.777 INFO:teuthology.orchestra.run.smithi089.stdout:(31/35): ceph-mgr-modules-core-18.2.0-501.gfc1e 2.1 MB/s | 241 kB 00:00 2023-09-30T22:15:06.844 INFO:teuthology.orchestra.run.smithi089.stdout:(32/35): ceph-mgr-rook-18.2.0-501.gfc1e90a3.el8 713 kB/s | 47 kB 00:00 2023-09-30T22:15:06.903 INFO:teuthology.orchestra.run.smithi089.stdout:(33/35): ceph-prometheus-alerts-18.2.0-501.gfc1 248 kB/s | 15 kB 00:00 2023-09-30T22:15:07.062 INFO:teuthology.orchestra.run.smithi089.stdout:(34/35): ceph-mgr-diskprediction-local-18.2.0-5 13 MB/s | 7.4 MB 00:00 2023-09-30T22:15:12.286 INFO:teuthology.orchestra.run.smithi089.stdout:(35/35): ceph-test-18.2.0-501.gfc1e90a3.el8.x86 5.6 MB/s | 47 MB 00:08 2023-09-30T22:15:12.289 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:15:12.290 INFO:teuthology.orchestra.run.smithi089.stdout:Total 14 MB/s | 145 MB 00:10 2023-09-30T22:15:12.291 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-09-30T22:15:12.361 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-09-30T22:15:12.362 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-09-30T22:15:13.027 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-09-30T22:15:13.030 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-09-30T22:15:15.392 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-09-30T22:15:15.616 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-09-30T22:15:16.180 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/69 2023-09-30T22:15:16.454 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:15:16.848 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:15:17.050 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 3/69 2023-09-30T22:15:17.173 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 3/69 2023-09-30T22:15:17.362 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:15:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:15:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 5/69 2023-09-30T22:15:18.524 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 5/69 2023-09-30T22:15:18.767 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 6/69 2023-09-30T22:15:18.970 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 7/69 2023-09-30T22:15:19.075 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 7/69 2023-09-30T22:15:19.312 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:15:19.381 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:15:19.587 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x 9/69 2023-09-30T22:15:19.857 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 10/69 2023-09-30T22:15:20.100 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 11/69 2023-09-30T22:15:20.302 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 12/69 2023-09-30T22:15:20.514 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 13/69 2023-09-30T22:15:20.651 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8. 14/69 2023-09-30T22:15:21.431 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8 15/69 2023-09-30T22:15:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86 16/69 2023-09-30T22:15:24.022 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:15:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:15:24.966 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:15:25.303 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:15:25.364 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:15:25.552 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 19/69 2023-09-30T22:15:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 19/69 2023-09-30T22:15:43.116 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 20/69 2023-09-30T22:15:44.277 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 20/69 2023-09-30T22:15:44.550 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 21/69 2023-09-30T22:15:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 21/69 2023-09-30T22:15:45.048 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.n 22/69 2023-09-30T22:15:45.291 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 23/69 2023-09-30T22:15:45.314 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 23/69 2023-09-30T22:15:45.513 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 24/69 2023-09-30T22:15:45.720 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 24/69 2023-09-30T22:15:45.950 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 25/69 2023-09-30T22:15:46.553 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 25/69 2023-09-30T22:15:46.767 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:15:48.556 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:15:48.882 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 27/69 2023-09-30T22:15:48.893 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 27/69 2023-09-30T22:15:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 28/69 2023-09-30T22:15:50.632 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 29/69 2023-09-30T22:15:51.034 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 29/69 2023-09-30T22:15:51.308 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:15:56.465 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:15:56.737 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 31/69 2023-09-30T22:15:56.945 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 32/69 2023-09-30T22:15:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 33/69 2023-09-30T22:15:57.457 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 34/69 2023-09-30T22:15:57.733 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 35/69 2023-09-30T22:15:57.868 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 36/69 2023-09-30T22:15:58.133 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 37/69 2023-09-30T22:15:58.141 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:15:58.142 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:15:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:15:58.595 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 39/69 2023-09-30T22:15:58.596 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 40/69 2023-09-30T22:15:58.779 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 40/69 2023-09-30T22:15:58.786 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:15:58.786 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:15:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:15:59.190 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.n 42/69 2023-09-30T22:15:59.415 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 43/69 2023-09-30T22:15:59.422 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:15:59.422 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:15:59.650 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:15:59.815 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 45/69 2023-09-30T22:15:59.816 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 46/69 2023-09-30T22:16:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 46/69 2023-09-30T22:16:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 47/69 2023-09-30T22:16:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:16:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:16:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:16:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 49/69 2023-09-30T22:16:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 49/69 2023-09-30T22:16:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:16:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:16:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:16:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:16:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:16:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:16:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 52/69 2023-09-30T22:16:01.533 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 52/69 2023-09-30T22:16:01.560 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:16:01.561 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:16:01.945 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:16:01.947 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 54/69 2023-09-30T22:16:02.158 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 54/69 2023-09-30T22:16:02.158 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:16:02.416 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:16:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 56/69 2023-09-30T22:16:02.841 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 57/69 2023-09-30T22:16:02.841 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 58/69 2023-09-30T22:16:03.082 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 58/69 2023-09-30T22:16:03.311 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 59/69 2023-09-30T22:16:03.312 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 60/69 2023-09-30T22:16:03.587 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 60/69 2023-09-30T22:16:03.798 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 61/69 2023-09-30T22:16:03.799 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 62/69 2023-09-30T22:16:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 62/69 2023-09-30T22:16:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 63/69 2023-09-30T22:16:04.383 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x 64/69 2023-09-30T22:16:04.548 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86 65/69 2023-09-30T22:16:04.706 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8 66/69 2023-09-30T22:16:04.800 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8. 67/69 2023-09-30T22:16:04.800 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 68/69 2023-09-30T22:16:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 68/69 2023-09-30T22:16:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 69/69 2023-09-30T22:16:05.725 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 69/69 2023-09-30T22:16:05.725 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 1/69 2023-09-30T22:16:05.725 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:16:05.725 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 3/69 2023-09-30T22:16:05.725 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:16:05.725 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 5/69 2023-09-30T22:16:05.725 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 6/69 2023-09-30T22:16:05.725 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 7/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 9/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 10/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 11/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 12/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 13/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 14/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 15/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 16/69 2023-09-30T22:16:05.726 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 17/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 19/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 20/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 21/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 22/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 23/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 24/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 25/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 27/69 2023-09-30T22:16:05.727 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 28/69 2023-09-30T22:16:05.732 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 29/69 2023-09-30T22:16:05.732 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:16:05.732 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 31/69 2023-09-30T22:16:05.732 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 32/69 2023-09-30T22:16:05.732 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 33/69 2023-09-30T22:16:05.732 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 34/69 2023-09-30T22:16:05.732 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 35/69 2023-09-30T22:16:05.732 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 36/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 37/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 38/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 39/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x 40/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x 41/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86 42/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86 43/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 44/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 45/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 46/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 47/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 48/69 2023-09-30T22:16:05.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 49/69 2023-09-30T22:16:05.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 50/69 2023-09-30T22:16:05.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:16:05.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 52/69 2023-09-30T22:16:05.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:16:05.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 54/69 2023-09-30T22:16:05.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:16:05.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 56/69 2023-09-30T22:16:05.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 57/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8 58/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8 59/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 60/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 61/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 62/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 63/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.n 64/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.n 65/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 66/69 2023-09-30T22:16:05.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 67/69 2023-09-30T22:16:05.736 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8. 68/69 2023-09-30T22:16:07.608 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8. 69/69 2023-09-30T22:16:07.608 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.609 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.610 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.611 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-09-30T22:16:07.763 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:16:07.983 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:16:08.007 INFO:teuthology.orchestra.run.smithi089.stdout:18.2.0-501.gfc1e90a3.el8 2023-09-30T22:16:08.007 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-501.gfc1e90a3.el8 2023-09-30T22:16:08.008 INFO:teuthology.task.install:The correct ceph version 18.2.0-501.gfc1e90a3 is installed. 2023-09-30T22:16:08.008 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:16:08.017 INFO:teuthology.task.print:**** done install.upgrade the host 2023-09-30T22:16:08.017 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:16:08.029 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-30T22:16:08.029 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:08.059 INFO:tasks.ceph.mon.a:Stopped 2023-09-30T22:16:08.059 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-30T22:16:08.059 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-30T22:16:08.101 INFO:tasks.ceph.mon.a:Started 2023-09-30T22:16:08.101 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-30T22:16:08.101 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:08.112 INFO:tasks.ceph.mon.b:Stopped 2023-09-30T22:16:08.112 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-30T22:16:08.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-30T22:16:08.155 INFO:tasks.ceph.mon.b:Started 2023-09-30T22:16:08.155 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-30T22:16:08.155 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:08.166 INFO:tasks.ceph.mon.c:Stopped 2023-09-30T22:16:08.166 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-30T22:16:08.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-30T22:16:08.208 INFO:tasks.ceph.mon.c:Started 2023-09-30T22:16:08.209 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-30T22:16:08.209 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:08.235 INFO:tasks.ceph.mgr.x:Stopped 2023-09-30T22:16:08.235 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-30T22:16:08.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-30T22:16:08.277 INFO:tasks.ceph.mgr.x:Started 2023-09-30T22:16:08.278 INFO:teuthology.run_tasks:Running task ceph.healthy... 2023-09-30T22:16:08.288 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:16:08.289 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:16:08.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:16:08.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.414+0000 7fc95fa7e700 1 Processor -- start 2023-09-30T22:16:08.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95fa7e700 1 -- start start 2023-09-30T22:16:08.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95fa7e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:08.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95fa7e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:08.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95fa7e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:08.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95fa7e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95806b460 con 0x7fc95809e000 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95fa7e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95809b120 con 0x7fc95809e410 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95fa7e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc95809a7e0 con 0x7fc95809d6f0 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95e01b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d019700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95e01b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41112/0 (socket says 172.21.15.89:41112) 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d019700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52420/0 (socket says 172.21.15.89:52420) 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 learned_addr learned my addr 172.21.15.89:0/124993444 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:16:08.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d81a700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d81a700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.415+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:16:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d81a700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d81a700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:08.616+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:16:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:16:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d81a700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d81a700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.017+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:16:09.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d81a700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d81a700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:16:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:16:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:09.818+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:16:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:11.416+0000 7fc94f7fe700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc95809dac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:16:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:11.416+0000 7fc94f7fe700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc95809dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:11.416+0000 7fc94f7fe700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc958133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:16:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:11.416+0000 7fc94f7fe700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc958133f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:11.416+0000 7fc94f7fe700 1 -- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 msgr2=0x7fc9581339f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:16:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:11.416+0000 7fc94f7fe700 1 --2- 172.21.15.89:0/124993444 >> v2:172.21.15.89:3300/0 conn(0x7fc95809e000 0x7fc9581339f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:11.416+0000 7fc94f7fe700 1 -- 172.21.15.89:0/124993444 --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95809a500 con 0x7fc944003760 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.443+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162100293 0 0) 0x7fc95809a500 con 0x7fc944003760 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc93c0035b0 con 0x7fc944003760 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3324160849 0 0) 0x7fc93c0035b0 con 0x7fc944003760 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944008410 con 0x7fc944003760 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1128284152 0 0) 0x7fc95400e590 con 0x7fc944003760 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 >> v1:172.21.15.89:6789/0 conn(0x7fc944003760 legacy=0x7fc944005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc93c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc93c0041c0 0x7fc93c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc93c004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0056e0 con 0x7fc93c0041c0 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00ba10 con 0x7fc95809e410 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00bb50 con 0x7fc95809d6f0 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc93c0041c0 0x7fc93c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95d81a700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc93c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.444+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc93c004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.445+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 msgr2=0x7fc93c003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.445+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc95809d6f0 0x7fc93c003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.445+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 msgr2=0x7fc93c004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.445+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc93c004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.445+0000 7fc95d019700 1 -- 172.21.15.89:0/124993444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c0035b0 con 0x7fc93c0041c0 2023-09-30T22:16:13.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.445+0000 7fc95e01b700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc95809e410 0x7fc93c004ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:13.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.445+0000 7fc95d019700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc93c0041c0 0x7fc93c004850 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc9480129e0 tx=0x7fc93c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e175aff56512f538 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:13.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.447+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480147f0 con 0x7fc93c0041c0 2023-09-30T22:16:13.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.447+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800c040 con 0x7fc93c0041c0 2023-09-30T22:16:13.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.447+0000 7fc95c818700 1 -- 172.21.15.89:0/124993444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948014bb0 con 0x7fc93c0041c0 2023-09-30T22:16:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.448+0000 7fc95fa7e700 1 -- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc93c0041c0 msgr2=0x7fc93c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.448+0000 7fc95fa7e700 1 --2- 172.21.15.89:0/124993444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc93c0041c0 0x7fc93c004850 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc9480129e0 tx=0x7fc93c005880 comp rx=0 tx=0).stop 2023-09-30T22:16:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.448+0000 7fc95e01b700 1 -- 172.21.15.89:0/124993444 reap_dead start 2023-09-30T22:16:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.448+0000 7fc95fa7e700 1 -- 172.21.15.89:0/124993444 shutdown_connections 2023-09-30T22:16:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.448+0000 7fc95fa7e700 1 -- 172.21.15.89:0/124993444 >> 172.21.15.89:0/124993444 conn(0x7fc958066f50 msgr2=0x7fc95809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.448+0000 7fc95fa7e700 1 -- 172.21.15.89:0/124993444 shutdown_connections 2023-09-30T22:16:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.448+0000 7fc95fa7e700 1 -- 172.21.15.89:0/124993444 wait complete. 2023-09-30T22:16:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.450+0000 7fc95fa7e700 1 Processor -- start 2023-09-30T22:16:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.450+0000 7fc95fa7e700 1 -- start start 2023-09-30T22:16:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95fa7e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9581a5360 0x7fc9581a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95fa7e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 0x7fc9581c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95fa7e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9581c87c0 0x7fc9581cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95fa7e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580a1670 con 0x7fc9581c87c0 2023-09-30T22:16:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95fa7e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9581b91f0 con 0x7fc9581a5360 2023-09-30T22:16:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95fa7e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9581a4de0 con 0x7fc9581c7e30 2023-09-30T22:16:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d019700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 0x7fc9581c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d019700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 0x7fc9581c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52452/0 (socket says 172.21.15.89:52452) 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d019700 1 -- 172.21.15.89:0/4230185503 learned_addr learned my addr 172.21.15.89:0/4230185503 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95e01b700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9581c87c0 0x7fc9581cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d81a700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9581a5360 0x7fc9581a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d019700 1 -- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 msgr2=0x7fc9581c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d019700 1 -- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 msgr2=0x7fc9581c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d019700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 0x7fc9581c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d019700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 0x7fc9581c8280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95e01b700 1 -- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 msgr2=0x7fc9581c8280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95e01b700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 0x7fc9581c8280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95e01b700 1 -- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9581a5360 msgr2=0x7fc9581a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95e01b700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9581a5360 0x7fc9581a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95e01b700 1 -- 172.21.15.89:0/4230185503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9581a80a0 con 0x7fc9581c87c0 2023-09-30T22:16:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.451+0000 7fc95d81a700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9581a5360 0x7fc9581a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.452+0000 7fc95e01b700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9581c87c0 0x7fc9581cf160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc95809e7e0 tx=0x7fc950008f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.452+0000 7fc94e7fc700 1 -- 172.21.15.89:0/4230185503 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950007920 con 0x7fc9581c87c0 2023-09-30T22:16:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.452+0000 7fc94e7fc700 1 -- 172.21.15.89:0/4230185503 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95000f040 con 0x7fc9581c87c0 2023-09-30T22:16:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.452+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9581a6a20 con 0x7fc9581c87c0 2023-09-30T22:16:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.452+0000 7fc94e7fc700 1 -- 172.21.15.89:0/4230185503 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950007cc0 con 0x7fc9581c87c0 2023-09-30T22:16:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.452+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9581a6c60 con 0x7fc9581c87c0 2023-09-30T22:16:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.453+0000 7fc94e7fc700 1 -- 172.21.15.89:0/4230185503 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fc950022460 con 0x7fc9581c87c0 2023-09-30T22:16:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.453+0000 7fc94e7fc700 1 -- 172.21.15.89:0/4230185503 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fc95003c290 con 0x7fc9581c87c0 2023-09-30T22:16:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.453+0000 7fc95d81a700 1 -- 172.21.15.89:0/4230185503 >> v1:172.21.15.89:6816/118161 conn(0x7fc94402c8d0 legacy=0x7fc94402ed80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.453+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9581a7ee0 con 0x7fc9581c87c0 2023-09-30T22:16:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.458+0000 7fc94e7fc700 1 -- 172.21.15.89:0/4230185503 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fc9500097e0 con 0x7fc9581c87c0 2023-09-30T22:16:13.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.654+0000 7fc95d81a700 1 -- 172.21.15.89:0/4230185503 >> v1:172.21.15.89:6816/118161 conn(0x7fc94402c8d0 legacy=0x7fc94402ed80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.664+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc9581a7d40 con 0x7fc9581c87c0 2023-09-30T22:16:13.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.665+0000 7fc94e7fc700 1 -- 172.21.15.89:0/4230185503 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+64282 (secure 0 0 0) 0x7fc9581a7d40 con 0x7fc9581c87c0 2023-09-30T22:16:13.667 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 >> v1:172.21.15.89:6816/118161 conn(0x7fc94402c8d0 legacy=0x7fc94402ed80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9581c87c0 msgr2=0x7fc9581cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9581c87c0 0x7fc9581cf160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc95809e7e0 tx=0x7fc950008f30 comp rx=0 tx=0).stop 2023-09-30T22:16:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 shutdown_connections 2023-09-30T22:16:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9581c87c0 0x7fc9581cf160 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9581a5360 0x7fc9581a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 --2- 172.21.15.89:0/4230185503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9581c7e30 0x7fc9581c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 >> 172.21.15.89:0/4230185503 conn(0x7fc958066f50 msgr2=0x7fc9580a2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 shutdown_connections 2023-09-30T22:16:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.667+0000 7fc95fa7e700 1 -- 172.21.15.89:0/4230185503 wait complete. 2023-09-30T22:16:13.679 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6816","nonce":118161}]},"active_addr":"172.21.15.89:6816/118161","active_change":"2023-09-30T22:13:50.734934+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1269574253}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2579157783}]}]} 2023-09-30T22:16:13.682 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:16:13.682 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:16:13.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:13.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.802+0000 7f4f3cb21700 1 Processor -- start 2023-09-30T22:16:13.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.802+0000 7f4f3cb21700 1 -- start start 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.802+0000 7f4f3cb21700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4f38133420 0x7f4f381337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.802+0000 7f4f3cb21700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f381306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.802+0000 7f4f3cb21700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3806b990 con 0x7f4f38133e30 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.802+0000 7f4f3cb21700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f380a0590 con 0x7f4f38133420 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.802+0000 7f4f3cb21700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3809fb80 con 0x7f4f38134260 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.802+0000 7f4f36d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f381306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f3659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4f38133420 0x7f4f381337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f35d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4f38133e30 0x7f4f3812ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35220/0 (socket says 172.21.15.89:35220) 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f35d9b700 1 -- 172.21.15.89:0/1939575788 learned_addr learned my addr 172.21.15.89:0/1939575788 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:13.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f36d9d700 1 -- 172.21.15.89:0/1939575788 >> v2:172.21.15.89:3300/0 conn(0x7f4f38133420 msgr2=0x7f4f381337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f36d9d700 1 --2- 172.21.15.89:0/1939575788 >> v2:172.21.15.89:3300/0 conn(0x7f4f38133420 0x7f4f381337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f36d9d700 1 -- 172.21.15.89:0/1939575788 >> v1:172.21.15.89:6789/0 conn(0x7f4f38133e30 legacy=0x7f4f3812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f36d9d700 1 -- 172.21.15.89:0/1939575788 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f380a0030 con 0x7f4f38134260 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f3559a700 1 -- 172.21.15.89:0/1939575788 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227336624 0 0) 0x7f4f3806b990 con 0x7f4f38133e30 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f3659c700 1 --2- 172.21.15.89:0/1939575788 >> v2:172.21.15.89:3300/0 conn(0x7f4f38133420 0x7f4f381337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f36d9d700 1 --2- 172.21.15.89:0/1939575788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f381306a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009d90 tx=0x7f4f3000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c196e8e42f95d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f3559a700 1 -- 172.21.15.89:0/1939575788 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000e070 con 0x7f4f38134260 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f3559a700 1 -- 172.21.15.89:0/1939575788 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3000bb20 con 0x7f4f38134260 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.803+0000 7f4f3559a700 1 -- 172.21.15.89:0/1939575788 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30012430 con 0x7f4f38134260 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.804+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1939575788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 msgr2=0x7f4f381306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:13.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.804+0000 7f4f3cb21700 1 --2- 172.21.15.89:0/1939575788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f381306a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009d90 tx=0x7f4f3000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.804+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1939575788 shutdown_connections 2023-09-30T22:16:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.804+0000 7f4f3cb21700 1 --2- 172.21.15.89:0/1939575788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f381306a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.804+0000 7f4f3cb21700 1 --2- 172.21.15.89:0/1939575788 >> v2:172.21.15.89:3300/0 conn(0x7f4f38133420 0x7f4f381337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.804+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1939575788 >> 172.21.15.89:0/1939575788 conn(0x7f4f380a4210 msgr2=0x7f4f380a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.804+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1939575788 shutdown_connections 2023-09-30T22:16:13.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.804+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1939575788 wait complete. 2023-09-30T22:16:13.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3cb21700 1 Processor -- start 2023-09-30T22:16:13.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3cb21700 1 -- start start 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3cb21700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f38133420 0x7f4f3809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3cb21700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 0x7f4f3809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3cb21700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f3809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3cb21700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f38120600 con 0x7f4f38133420 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3cb21700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3811fb60 con 0x7f4f38134260 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3cb21700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f38131590 con 0x7f4f38133e30 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f38133420 0x7f4f3809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f38133420 0x7f4f3809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56442/0 (socket says 172.21.15.89:56442) 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f3659c700 1 -- 172.21.15.89:0/1411696140 learned_addr learned my addr 172.21.15.89:0/1411696140 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f36d9d700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f3809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.805+0000 7f4f35d9b700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 0x7f4f3809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f35d9b700 1 -- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 msgr2=0x7f4f3809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f35d9b700 1 -- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 msgr2=0x7f4f3809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f35d9b700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 0x7f4f3809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f35d9b700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 0x7f4f3809c9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f3659c700 1 -- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 msgr2=0x7f4f3809c9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f3659c700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 0x7f4f3809c9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f3659c700 1 -- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 msgr2=0x7f4f3809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f3659c700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f3809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f3659c700 1 -- 172.21.15.89:0/1411696140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f280094a0 con 0x7f4f38133420 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f36d9d700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f3809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f3659c700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f38133420 0x7f4f3809c480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800ebe0 tx=0x7f4f2800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f26ffd700 1 -- 172.21.15.89:0/1411696140 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28009800 con 0x7f4f38133420 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f26ffd700 1 -- 172.21.15.89:0/1411696140 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2800c910 con 0x7f4f38133420 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f30009a70 con 0x7f4f38133420 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f26ffd700 1 -- 172.21.15.89:0/1411696140 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800cc80 con 0x7f4f38133420 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.806+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f38120170 con 0x7f4f38133420 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.807+0000 7f4f26ffd700 1 -- 172.21.15.89:0/1411696140 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f4f28013470 con 0x7f4f38133420 2023-09-30T22:16:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.807+0000 7f4f26ffd700 1 -- 172.21.15.89:0/1411696140 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f4f2803bd40 con 0x7f4f38133420 2023-09-30T22:16:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.807+0000 7f4f35d9b700 1 -- 172.21.15.89:0/1411696140 >> v1:172.21.15.89:6816/118161 conn(0x7f4f1802c1b0 legacy=0x7f4f1802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.807+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3809e120 con 0x7f4f38133420 2023-09-30T22:16:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.811+0000 7f4f26ffd700 1 -- 172.21.15.89:0/1411696140 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f4f28017020 con 0x7f4f38133420 2023-09-30T22:16:13.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.972+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f3811fd80 con 0x7f4f38133420 2023-09-30T22:16:13.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.973+0000 7f4f26ffd700 1 -- 172.21.15.89:0/1411696140 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+12690 (secure 0 0 0) 0x7f4f28013720 con 0x7f4f38133420 2023-09-30T22:16:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:13.974 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:52.544823+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:13.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 >> v1:172.21.15.89:6816/118161 conn(0x7f4f1802c1b0 legacy=0x7f4f1802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f38133420 msgr2=0x7f4f3809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f38133420 0x7f4f3809c480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800ebe0 tx=0x7f4f2800eef0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 shutdown_connections 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f38134260 0x7f4f3809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f38133420 0x7f4f3809c480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 --2- 172.21.15.89:0/1411696140 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f38133e30 0x7f4f3809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 >> 172.21.15.89:0/1411696140 conn(0x7f4f380a4210 msgr2=0x7f4f38130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 shutdown_connections 2023-09-30T22:16:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:13.975+0000 7f4f3cb21700 1 -- 172.21.15.89:0/1411696140 wait complete. 2023-09-30T22:16:13.988 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:16:13.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.108+0000 7fcdaca02700 1 Processor -- start 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.108+0000 7fcdaca02700 1 -- start start 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcdaca02700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcda809e2e0 0x7fcda8138aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcdaca02700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda8138fe0 0x7fcda813b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcdaca02700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda8099e10 con 0x7fcda809d800 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcdaca02700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda809f560 con 0x7fcda809e2e0 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcdaca02700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda806b4d0 con 0x7fcda8138fe0 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda759e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda8138fe0 0x7fcda813b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcda809e2e0 0x7fcda8138aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda6d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcda809d800 0x7fcda809dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35232/0 (socket says 172.21.15.89:35232) 2023-09-30T22:16:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda6d9d700 1 -- 172.21.15.89:0/1818015423 learned_addr learned my addr 172.21.15.89:0/1818015423 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda5d9b700 1 -- 172.21.15.89:0/1818015423 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419604348 0 0) 0x7fcda8099e10 con 0x7fcda809d800 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda5d9b700 1 -- 172.21.15.89:0/1818015423 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd900035b0 con 0x7fcda809d800 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda759e700 1 -- 172.21.15.89:0/1818015423 >> v2:172.21.15.89:3300/0 conn(0x7fcda809e2e0 msgr2=0x7fcda8138aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda759e700 1 --2- 172.21.15.89:0/1818015423 >> v2:172.21.15.89:3300/0 conn(0x7fcda809e2e0 0x7fcda8138aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda759e700 1 -- 172.21.15.89:0/1818015423 >> v1:172.21.15.89:6789/0 conn(0x7fcda809d800 legacy=0x7fcda809dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda759e700 1 -- 172.21.15.89:0/1818015423 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda809a060 con 0x7fcda8138fe0 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda659c700 1 --2- 172.21.15.89:0/1818015423 >> v2:172.21.15.89:3300/0 conn(0x7fcda809e2e0 0x7fcda8138aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda759e700 1 --2- 172.21.15.89:0/1818015423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda8138fe0 0x7fcda813b3c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009d90 tx=0x7fcda000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e2b8442fd463a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda5d9b700 1 -- 172.21.15.89:0/1818015423 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000e070 con 0x7fcda8138fe0 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda5d9b700 1 -- 172.21.15.89:0/1818015423 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda000bb20 con 0x7fcda8138fe0 2023-09-30T22:16:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.109+0000 7fcda5d9b700 1 -- 172.21.15.89:0/1818015423 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0012430 con 0x7fcda8138fe0 2023-09-30T22:16:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.110+0000 7fcdaca02700 1 -- 172.21.15.89:0/1818015423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda8138fe0 msgr2=0x7fcda813b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.110+0000 7fcdaca02700 1 --2- 172.21.15.89:0/1818015423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda8138fe0 0x7fcda813b3c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009d90 tx=0x7fcda000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.110+0000 7fcdaca02700 1 -- 172.21.15.89:0/1818015423 shutdown_connections 2023-09-30T22:16:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.110+0000 7fcdaca02700 1 --2- 172.21.15.89:0/1818015423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda8138fe0 0x7fcda813b3c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.110+0000 7fcdaca02700 1 --2- 172.21.15.89:0/1818015423 >> v2:172.21.15.89:3300/0 conn(0x7fcda809e2e0 0x7fcda8138aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.110+0000 7fcdaca02700 1 -- 172.21.15.89:0/1818015423 >> 172.21.15.89:0/1818015423 conn(0x7fcda80a2e20 msgr2=0x7fcda80a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.110+0000 7fcdaca02700 1 -- 172.21.15.89:0/1818015423 shutdown_connections 2023-09-30T22:16:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.110+0000 7fcdaca02700 1 -- 172.21.15.89:0/1818015423 wait complete. 2023-09-30T22:16:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.111+0000 7fcdaca02700 1 Processor -- start 2023-09-30T22:16:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.111+0000 7fcdaca02700 1 -- start start 2023-09-30T22:16:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcdaca02700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 0x7fcda8121440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcdaca02700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda809e2e0 0x7fcda8121980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcdaca02700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda8138fe0 0x7fcda8121ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcdaca02700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda809c510 con 0x7fcda8138fe0 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcdaca02700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80a1fb0 con 0x7fcda809e2e0 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcdaca02700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcda8122830 con 0x7fcda809d800 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda809e2e0 0x7fcda8121980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 0x7fcda8121440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda809e2e0 0x7fcda8121980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42196/0 (socket says 172.21.15.89:42196) 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 -- 172.21.15.89:0/396508505 learned_addr learned my addr 172.21.15.89:0/396508505 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda8138fe0 0x7fcda8121ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda6d9d700 1 -- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 msgr2=0x7fcda8121440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda6d9d700 1 -- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 msgr2=0x7fcda8121440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda6d9d700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 0x7fcda8121440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda6d9d700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 0x7fcda8121440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 -- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 msgr2=0x7fcda8121440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 0x7fcda8121440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 -- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda8138fe0 msgr2=0x7fcda8121ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda8138fe0 0x7fcda8121ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 -- 172.21.15.89:0/396508505 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd98009430 con 0x7fcda809e2e0 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda759e700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda8138fe0 0x7fcda8121ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcda659c700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda809e2e0 0x7fcda8121980 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800ebe0 tx=0x7fcd9800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcd8f7fe700 1 -- 172.21.15.89:0/396508505 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98005a20 con 0x7fcda809e2e0 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcd8f7fe700 1 -- 172.21.15.89:0/396508505 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98005b80 con 0x7fcda809e2e0 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda0009a70 con 0x7fcda809e2e0 2023-09-30T22:16:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcd8f7fe700 1 -- 172.21.15.89:0/396508505 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9800c910 con 0x7fcda809e2e0 2023-09-30T22:16:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.112+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda8124c80 con 0x7fcda809e2e0 2023-09-30T22:16:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.113+0000 7fcd8f7fe700 1 -- 172.21.15.89:0/396508505 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fcd9800ca70 con 0x7fcda809e2e0 2023-09-30T22:16:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.113+0000 7fcd8f7fe700 1 -- 172.21.15.89:0/396508505 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fcd9803bc20 con 0x7fcda809e2e0 2023-09-30T22:16:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.113+0000 7fcda6d9d700 1 -- 172.21.15.89:0/396508505 >> v1:172.21.15.89:6816/118161 conn(0x7fcd9402c1b0 legacy=0x7fcd9402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.113+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda81248d0 con 0x7fcda809e2e0 2023-09-30T22:16:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.117+0000 7fcd8f7fe700 1 -- 172.21.15.89:0/396508505 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fcd98020020 con 0x7fcda809e2e0 2023-09-30T22:16:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.280+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcda804f2a0 con 0x7fcda809e2e0 2023-09-30T22:16:14.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.281+0000 7fcd8f7fe700 1 -- 172.21.15.89:0/396508505 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+12690 (secure 0 0 0) 0x7fcd98022300 con 0x7fcda809e2e0 2023-09-30T22:16:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:14.282 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:14:52.544823+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 >> v1:172.21.15.89:6816/118161 conn(0x7fcd9402c1b0 legacy=0x7fcd9402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda809e2e0 msgr2=0x7fcda8121980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda809e2e0 0x7fcda8121980 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800ebe0 tx=0x7fcd9800eef0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 shutdown_connections 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcda8138fe0 0x7fcda8121ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcda809d800 0x7fcda8121440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 --2- 172.21.15.89:0/396508505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcda809e2e0 0x7fcda8121980 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 >> 172.21.15.89:0/396508505 conn(0x7fcda80a2e20 msgr2=0x7fcda8070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 shutdown_connections 2023-09-30T22:16:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.283+0000 7fcdaca02700 1 -- 172.21.15.89:0/396508505 wait complete. 2023-09-30T22:16:14.296 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:16:14.297 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:16:14.298 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:16:14.299 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:16:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.437+0000 7fb306280700 1 Processor -- start 2023-09-30T22:16:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.437+0000 7fb306280700 1 -- start start 2023-09-30T22:16:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb306280700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 0x7fb30006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb306280700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb300131d10 0x7fb300139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb306280700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30009ece0 con 0x7fb30006b080 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb306280700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30009e760 con 0x7fb300131d10 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb306280700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb30009e040 con 0x7fb30006a770 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb2fffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 0x7fb30006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb2fffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 0x7fb30006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52500/0 (socket says 172.21.15.89:52500) 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb2fffff700 1 -- 172.21.15.89:0/2240485514 learned_addr learned my addr 172.21.15.89:0/2240485514 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb2fffff700 1 -- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 msgr2=0x7fb30006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb30481d700 1 --2- 172.21.15.89:0/2240485514 >> v2:172.21.15.89:3300/0 conn(0x7fb300131d10 0x7fb300139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb2fffff700 1 -- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 msgr2=0x7fb30006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb2fffff700 1 --2- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 0x7fb30006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.438+0000 7fb2fffff700 1 --2- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 0x7fb30006ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb30481d700 1 -- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 msgr2=0x7fb30006ab40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb30481d700 1 --2- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 0x7fb30006ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb30481d700 1 -- 172.21.15.89:0/2240485514 >> v1:172.21.15.89:6789/0 conn(0x7fb30006b080 legacy=0x7fb300131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb30481d700 1 -- 172.21.15.89:0/2240485514 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30009f630 con 0x7fb300131d10 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb2feffd700 1 -- 172.21.15.89:0/2240485514 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953848142 0 0) 0x7fb30009ece0 con 0x7fb30006b080 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb30481d700 1 --2- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb300131d10 0x7fb300139860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009d90 tx=0x7fb2f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d9622d56172a03 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb2feffd700 1 -- 172.21.15.89:0/2240485514 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f800e070 con 0x7fb300131d10 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb2feffd700 1 -- 172.21.15.89:0/2240485514 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f800bbd0 con 0x7fb300131d10 2023-09-30T22:16:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.439+0000 7fb2feffd700 1 -- 172.21.15.89:0/2240485514 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8012430 con 0x7fb300131d10 2023-09-30T22:16:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.440+0000 7fb306280700 1 -- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb300131d10 msgr2=0x7fb300139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.440+0000 7fb306280700 1 --2- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb300131d10 0x7fb300139860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009d90 tx=0x7fb2f800b330 comp rx=0 tx=0).stop 2023-09-30T22:16:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.440+0000 7fb306280700 1 -- 172.21.15.89:0/2240485514 shutdown_connections 2023-09-30T22:16:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.440+0000 7fb306280700 1 --2- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb300131d10 0x7fb300139860 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.440+0000 7fb306280700 1 --2- 172.21.15.89:0/2240485514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb30006a770 0x7fb30006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.440+0000 7fb306280700 1 -- 172.21.15.89:0/2240485514 >> 172.21.15.89:0/2240485514 conn(0x7fb300066f50 msgr2=0x7fb30009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.440+0000 7fb306280700 1 -- 172.21.15.89:0/2240485514 shutdown_connections 2023-09-30T22:16:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.440+0000 7fb306280700 1 -- 172.21.15.89:0/2240485514 wait complete. 2023-09-30T22:16:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.441+0000 7fb306280700 1 Processor -- start 2023-09-30T22:16:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.441+0000 7fb306280700 1 -- start start 2023-09-30T22:16:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb306280700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb30006a770 0x7fb3001204e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb306280700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb30006b080 0x7fb300120a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb306280700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb300131d10 0x7fb300120f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb306280700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb300122900 con 0x7fb30006b080 2023-09-30T22:16:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb306280700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb300071e60 con 0x7fb30006a770 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb306280700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001239d0 con 0x7fb300131d10 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2fffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb30006a770 0x7fb3001204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2fffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb30006a770 0x7fb3001204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42218/0 (socket says 172.21.15.89:42218) 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2fffff700 1 -- 172.21.15.89:0/3033231881 learned_addr learned my addr 172.21.15.89:0/3033231881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb30481d700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb300131d10 0x7fb300120f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb30481d700 1 -- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb300131d10 msgr2=0x7fb300120f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb30481d700 1 -- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb300131d10 msgr2=0x7fb300120f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb30481d700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb300131d10 0x7fb300120f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2ff7fe700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb30006b080 0x7fb300120a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb30481d700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb300131d10 0x7fb300120f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2fffff700 1 -- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb300131d10 msgr2=0x7fb300120f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2fffff700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb300131d10 0x7fb300120f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2fffff700 1 -- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb30006b080 msgr2=0x7fb300120a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2fffff700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb30006b080 0x7fb300120a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2fffff700 1 -- 172.21.15.89:0/3033231881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f8009a70 con 0x7fb30006a770 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.442+0000 7fb2ff7fe700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb30006b080 0x7fb300120a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.443+0000 7fb2fffff700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb30006a770 0x7fb3001204e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000e810 tx=0x7fb2f000ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.443+0000 7fb2fcff9700 1 -- 172.21.15.89:0/3033231881 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0004220 con 0x7fb30006a770 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.443+0000 7fb2fcff9700 1 -- 172.21.15.89:0/3033231881 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f0004380 con 0x7fb30006a770 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.443+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb300122160 con 0x7fb30006a770 2023-09-30T22:16:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.443+0000 7fb2fcff9700 1 -- 172.21.15.89:0/3033231881 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00036a0 con 0x7fb30006a770 2023-09-30T22:16:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.443+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb300130f50 con 0x7fb30006a770 2023-09-30T22:16:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.444+0000 7fb2fcff9700 1 -- 172.21.15.89:0/3033231881 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fb2f0003800 con 0x7fb30006a770 2023-09-30T22:16:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.444+0000 7fb2fcff9700 1 -- 172.21.15.89:0/3033231881 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fb2f003f170 con 0x7fb30006a770 2023-09-30T22:16:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.444+0000 7fb2ff7fe700 1 -- 172.21.15.89:0/3033231881 >> v1:172.21.15.89:6816/118161 conn(0x7fb2ec02c1b0 legacy=0x7fb2ec02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.450+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 --> v1:172.21.15.89:6808/118781 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3000621a0 con 0x7fb3001c5ee0 2023-09-30T22:16:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.451+0000 7fb2fcff9700 1 -- 172.21.15.89:0/3033231881 <== osd.3 v1:172.21.15.89:6808/118781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7fb3000621a0 con 0x7fb3001c5ee0 2023-09-30T22:16:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.456+0000 7f635e0fb700 1 Processor -- start 2023-09-30T22:16:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.456+0000 7f635e0fb700 1 -- start start 2023-09-30T22:16:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.457+0000 7f635e0fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f635806e570 0x7f635806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.457+0000 7f635e0fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f635809eca0 0x7f635806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.457+0000 7f635e0fb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6358134610 0x7f6358134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.457+0000 7f635e0fb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6358070ee0 con 0x7f6358134610 2023-09-30T22:16:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.457+0000 7f635e0fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6358071480 con 0x7f635806e570 2023-09-30T22:16:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.457+0000 7f635e0fb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f635806f3c0 con 0x7f635809eca0 2023-09-30T22:16:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6358134610 0x7f6358134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6358134610 0x7f6358134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56512/0 (socket says 172.21.15.89:56512) 2023-09-30T22:16:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 -- 172.21.15.89:0/276370699 learned_addr learned my addr 172.21.15.89:0/276370699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 -- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f635809eca0 msgr2=0x7f635806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 --2- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f635809eca0 0x7f635806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 -- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f635806e570 msgr2=0x7f635806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 --2- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f635806e570 0x7f635806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 -- 172.21.15.89:0/276370699 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358071b90 con 0x7f6358134610 2023-09-30T22:16:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f6357fff700 1 --2- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6358134610 0x7f6358134ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6350009e50 tx=0x7f635000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d8d02f8f36cf04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f63567fc700 1 -- 172.21.15.89:0/276370699 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000e070 con 0x7f6358134610 2023-09-30T22:16:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f63567fc700 1 -- 172.21.15.89:0/276370699 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635000b9c0 con 0x7f6358134610 2023-09-30T22:16:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.458+0000 7f63567fc700 1 -- 172.21.15.89:0/276370699 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000bd70 con 0x7f6358134610 2023-09-30T22:16:14.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 -- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6358134610 msgr2=0x7f6358134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 --2- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6358134610 0x7f6358134ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6350009e50 tx=0x7f635000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 -- 172.21.15.89:0/276370699 shutdown_connections 2023-09-30T22:16:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 --2- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6358134610 0x7f6358134ab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 --2- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f635806e570 0x7f635806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 --2- 172.21.15.89:0/276370699 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f635809eca0 0x7f635806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 -- 172.21.15.89:0/276370699 >> 172.21.15.89:0/276370699 conn(0x7f63580a2aa0 msgr2=0x7f63580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 -- 172.21.15.89:0/276370699 shutdown_connections 2023-09-30T22:16:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.459+0000 7f635e0fb700 1 -- 172.21.15.89:0/276370699 wait complete. 2023-09-30T22:16:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.460+0000 7f635e0fb700 1 Processor -- start 2023-09-30T22:16:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.460+0000 7f635e0fb700 1 -- start start 2023-09-30T22:16:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f635e0fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f635806e570 0x7f63581214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f635e0fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6358134610 0x7f6358121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f635e0fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6358121f60 0x7f63581d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f635e0fb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358124820 con 0x7f635806e570 2023-09-30T22:16:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f635e0fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6358124a20 con 0x7f6358134610 2023-09-30T22:16:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f635e0fb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6358123910 con 0x7f6358121f60 2023-09-30T22:16:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f635806e570 0x7f63581214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f635806e570 0x7f63581214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56516/0 (socket says 172.21.15.89:56516) 2023-09-30T22:16:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 -- 172.21.15.89:0/1893921828 learned_addr learned my addr 172.21.15.89:0/1893921828 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f6357fff700 1 --2- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6358121f60 0x7f63581d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f6357fff700 1 -- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6358121f60 msgr2=0x7f63581d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-09-30T22:16:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f6357fff700 1 -- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6358121f60 msgr2=0x7f63581d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f6357fff700 1 --2- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6358121f60 0x7f63581d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f6357fff700 1 --2- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6358121f60 0x7f63581d2d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 -- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6358121f60 msgr2=0x7f63581d2d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 --2- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6358121f60 0x7f63581d2d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 -- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6358134610 msgr2=0x7f6358121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 --2- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6358134610 0x7f6358121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 -- 172.21.15.89:0/1893921828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350009b30 con 0x7f635806e570 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.461+0000 7f63577fe700 1 --2- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f635806e570 0x7f63581214e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f634800b530 tx=0x7f634800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.462+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348011830 con 0x7f635806e570 2023-09-30T22:16:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.462+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63480119b0 con 0x7f635806e570 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.462+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348011d60 con 0x7f635806e570 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.462+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63581343b0 con 0x7f635806e570 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.462+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63580a1f30 con 0x7f635806e570 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.463+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f634801a400 con 0x7f635806e570 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.463+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f63580722d0 con 0x7f635806e570 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.464+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f634803b980 con 0x7f635806e570 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.464+0000 7f6356ffd700 1 -- 172.21.15.89:0/1893921828 >> v1:172.21.15.89:6816/118161 conn(0x7f634402c1b0 legacy=0x7f634402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.464+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 --> v1:172.21.15.89:6800/118778 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6344038e10 con 0x7f6344035110 2023-09-30T22:16:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.464+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6348040240 con 0x7f635806e570 2023-09-30T22:16:14.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.465+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 <== osd.2 v1:172.21.15.89:6800/118778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f6344038e10 con 0x7f6344035110 2023-09-30T22:16:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.466+0000 7f71cdd01700 1 Processor -- start 2023-09-30T22:16:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.466+0000 7f71cdd01700 1 -- start start 2023-09-30T22:16:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71cdd01700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 0x7f71c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71cdd01700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 0x7f71c8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71cdd01700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c809f130 con 0x7f71c809e410 2023-09-30T22:16:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71cdd01700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c809f520 con 0x7f71c809e000 2023-09-30T22:16:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71cdd01700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71c809e930 con 0x7f71c809d6f0 2023-09-30T22:16:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c7fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f71c809e410 0x7f71c8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35244/0 (socket says 172.21.15.89:35244) 2023-09-30T22:16:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c7fff700 1 -- 172.21.15.89:0/3261710803 learned_addr learned my addr 172.21.15.89:0/3261710803 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c6ffd700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 0x7f71c8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168216486 0 0) 0x7f71c809f130 con 0x7f71c809e410 2023-09-30T22:16:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c77fe700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 0x7f71c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b00035b0 con 0x7f71c809e410 2023-09-30T22:16:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c77fe700 1 -- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 msgr2=0x7f71c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:16:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c77fe700 1 -- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 msgr2=0x7f71c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c77fe700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 0x7f71c809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.467+0000 7f71c77fe700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 0x7f71c809dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 317148679 0 0) 0x7f71b00035b0 con 0x7f71c809e410 2023-09-30T22:16:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 msgr2=0x7f71c809dac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:16:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 0x7f71c809dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 msgr2=0x7f71c8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 0x7f71c8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c809b2d0 con 0x7f71c809e410 2023-09-30T22:16:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1128284152 0 0) 0x7f71c00019f0 con 0x7f71c809e410 2023-09-30T22:16:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 >> v1:172.21.15.89:6789/0 conn(0x7f71c809e410 legacy=0x7f71c8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 0x7f71b0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b0003fb0 0x7f71b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 0x7f71b0004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c809e930 con 0x7f71b0003fb0 2023-09-30T22:16:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00052d0 con 0x7f71c809e000 2023-09-30T22:16:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0005490 con 0x7f71c809d6f0 2023-09-30T22:16:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c77fe700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 0x7f71b0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c77fe700 1 -- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 msgr2=0x7f71b0004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c77fe700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 0x7f71b0004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c77fe700 1 -- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b0003fb0 msgr2=0x7f71b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c77fe700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b0003fb0 0x7f71b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c77fe700 1 -- 172.21.15.89:0/3261710803 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c809b2d0 con 0x7f71c809e000 2023-09-30T22:16:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c77fe700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 0x7f71b0003a30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009890 tx=0x7f71b0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b63d9b083684ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.468+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800c070 con 0x7f71c809e000 2023-09-30T22:16:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.469+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b8004020 con 0x7f71c809e000 2023-09-30T22:16:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.469+0000 7f71c67fc700 1 -- 172.21.15.89:0/3261710803 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b80043d0 con 0x7f71c809e000 2023-09-30T22:16:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.469+0000 7f71cdd01700 1 -- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 msgr2=0x7f71b0003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.469+0000 7f71cdd01700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 0x7f71b0003a30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009890 tx=0x7f71b0005880 comp rx=0 tx=0).stop 2023-09-30T22:16:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.470+0000 7f71cdd01700 1 -- 172.21.15.89:0/3261710803 shutdown_connections 2023-09-30T22:16:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.470+0000 7f71cdd01700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b0003fb0 0x7f71b0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.470+0000 7f71cdd01700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809e000 0x7f71b0003a30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.470+0000 7f71cdd01700 1 --2- 172.21.15.89:0/3261710803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71c809d6f0 0x7f71b0004d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.470+0000 7f71cdd01700 1 -- 172.21.15.89:0/3261710803 >> 172.21.15.89:0/3261710803 conn(0x7f71c8066f50 msgr2=0x7f71c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.470+0000 7f71cdd01700 1 -- 172.21.15.89:0/3261710803 shutdown_connections 2023-09-30T22:16:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.470+0000 7f71cdd01700 1 -- 172.21.15.89:0/3261710803 wait complete. 2023-09-30T22:16:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.471+0000 7f71cdd01700 1 Processor -- start 2023-09-30T22:16:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.471+0000 7f6d7461f700 1 Processor -- start 2023-09-30T22:16:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.471+0000 7f71cdd01700 1 -- start start 2023-09-30T22:16:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71cdd01700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 0x7f71c8130820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71cdd01700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809d6f0 0x7f71c8130d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71cdd01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71c809e000 0x7f71c81364d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71cdd01700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8139df0 con 0x7f71c809e000 2023-09-30T22:16:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71cdd01700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c813af10 con 0x7f71c809d6f0 2023-09-30T22:16:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71cdd01700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71c813ac70 con 0x7f71b0003fb0 2023-09-30T22:16:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 0x7f71c8130820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 0x7f71c8130820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52548/0 (socket says 172.21.15.89:52548) 2023-09-30T22:16:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c77fe700 1 -- 172.21.15.89:0/2020824182 learned_addr learned my addr 172.21.15.89:0/2020824182 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c6ffd700 1 --2- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809d6f0 0x7f71c8130d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 --> v1:172.21.15.89:6808/118781 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb30009c850 con 0x7fb3001c5ee0 2023-09-30T22:16:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c77fe700 1 -- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 msgr2=0x7f71c8130820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-09-30T22:16:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c77fe700 1 -- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 msgr2=0x7f71c8130820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c77fe700 1 --2- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 0x7f71c8130820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c7fff700 1 --2- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71c809e000 0x7f71c81364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f71c77fe700 1 --2- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 0x7f71c8130820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.471+0000 7f6d7461f700 1 -- start start 2023-09-30T22:16:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f6d7461f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 0x7f6d6c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f6d7461f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f6d7461f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c070ee0 con 0x7f6d6c06e570 2023-09-30T22:16:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f6d723bb700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6d6c06e570 0x7f6d6c06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35254/0 (socket says 172.21.15.89:35254) 2023-09-30T22:16:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f6d723bb700 1 -- 172.21.15.89:0/2556532346 learned_addr learned my addr 172.21.15.89:0/2556532346 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.472+0000 7f6d72bbc700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d7461f700 1 -- 172.21.15.89:0/2556532346 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c071980 con 0x7f6d6c06f050 2023-09-30T22:16:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d7461f700 1 -- 172.21.15.89:0/2556532346 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c071480 con 0x7f6d6c09eca0 2023-09-30T22:16:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d71bba700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 0x7f6d6c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d71bba700 1 -- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 msgr2=0x7f6d6c13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:16:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d71bba700 1 -- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 msgr2=0x7f6d6c13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:16:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d71bba700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 0x7f6d6c13cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d71bba700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 0x7f6d6c13cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f71c7fff700 1 -- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 msgr2=0x7f71c8130820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f71c7fff700 1 --2- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71b0003fb0 0x7f71c8130820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f71c7fff700 1 -- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809d6f0 msgr2=0x7f71c8130d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f71c7fff700 1 --2- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71c809d6f0 0x7f71c8130d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f71c7fff700 1 -- 172.21.15.89:0/2020824182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80a36b0 con 0x7f71c809e000 2023-09-30T22:16:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d72bbc700 1 -- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 msgr2=0x7f6d6c13cea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:16:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d72bbc700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 0x7f6d6c13cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d72bbc700 1 -- 172.21.15.89:0/2556532346 >> v1:172.21.15.89:6789/0 conn(0x7f6d6c06e570 legacy=0x7f6d6c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f71c7fff700 1 --2- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71c809e000 0x7f71c81364d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f71c0002090 tx=0x7f71c000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d713b9700 1 -- 172.21.15.89:0/2556532346 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554395132 0 0) 0x7f6d6c070ee0 con 0x7f6d6c06e570 2023-09-30T22:16:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.473+0000 7f6d72bbc700 1 -- 172.21.15.89:0/2556532346 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c071b90 con 0x7f6d6c06f050 2023-09-30T22:16:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.474+0000 7f6d72bbc700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c13f3f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009dc0 tx=0x7f6d6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f2e126796e29e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.475+0000 7f6d713b9700 1 -- 172.21.15.89:0/2556532346 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400e070 con 0x7f6d6c06f050 2023-09-30T22:16:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.475+0000 7f6d713b9700 1 -- 172.21.15.89:0/2556532346 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400bbd0 con 0x7f6d6c06f050 2023-09-30T22:16:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.475+0000 7f6d713b9700 1 -- 172.21.15.89:0/2556532346 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64012430 con 0x7f6d6c06f050 2023-09-30T22:16:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb2fcff9700 1 -- 172.21.15.89:0/3033231881 <== osd.3 v1:172.21.15.89:6808/118781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1880693312) 0x7fb2f80098f0 con 0x7fb3001c5ee0 2023-09-30T22:16:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7f6d7461f700 1 -- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 msgr2=0x7f6d6c13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7f6d7461f700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c13f3f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009dc0 tx=0x7f6d6400b330 comp rx=0 tx=0).stop 2023-09-30T22:16:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7f6d7461f700 1 -- 172.21.15.89:0/2556532346 shutdown_connections 2023-09-30T22:16:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7f6d7461f700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c13f3f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7f6d7461f700 1 --2- 172.21.15.89:0/2556532346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c09eca0 0x7f6d6c13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7f6d7461f700 1 -- 172.21.15.89:0/2556532346 >> 172.21.15.89:0/2556532346 conn(0x7f6d6c0a2aa0 msgr2=0x7f6d6c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 >> v1:172.21.15.89:6808/118781 conn(0x7fb3001c5ee0 legacy=0x7fb3001c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 >> v1:172.21.15.89:6816/118161 conn(0x7fb2ec02c1b0 legacy=0x7fb2ec02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb30006a770 msgr2=0x7fb3001204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb306280700 1 --2- 172.21.15.89:0/3033231881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb30006a770 0x7fb3001204e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000e810 tx=0x7fb2f000ebd0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb30481d700 1 -- 172.21.15.89:0/3033231881 reap_dead start 2023-09-30T22:16:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 shutdown_connections 2023-09-30T22:16:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 >> 172.21.15.89:0/3033231881 conn(0x7fb300066f50 msgr2=0x7fb300070000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7f6d7461f700 1 -- 172.21.15.89:0/2556532346 shutdown_connections 2023-09-30T22:16:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 shutdown_connections 2023-09-30T22:16:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7fb306280700 1 -- 172.21.15.89:0/3033231881 wait complete. 2023-09-30T22:16:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.476+0000 7f6d7461f700 1 -- 172.21.15.89:0/2556532346 wait complete. 2023-09-30T22:16:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d7461f700 1 Processor -- start 2023-09-30T22:16:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d7461f700 1 -- start start 2023-09-30T22:16:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d7461f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c06e570 0x7f6d6c1372a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d7461f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c1377e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d7461f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d6c1d4d40 0x7f6d6c1d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d7461f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c138a80 con 0x7f6d6c1d4d40 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d7461f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c099fa0 con 0x7f6d6c06f050 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d7461f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c070610 con 0x7f6d6c06e570 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d71bba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c1377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d71bba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c1377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42284/0 (socket says 172.21.15.89:42284) 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d71bba700 1 -- 172.21.15.89:0/3628593732 learned_addr learned my addr 172.21.15.89:0/3628593732 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d71bba700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 msgr2=0x7f6d6c1377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-09-30T22:16:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d71bba700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 msgr2=0x7f6d6c1377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d71bba700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c1377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.478+0000 7f6d71bba700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c1377e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.480+0000 7f6d723bb700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c06e570 0x7f6d6c1372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.480+0000 7f6d723bb700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c06e570 msgr2=0x7f6d6c1372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-09-30T22:16:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.480+0000 7f6d723bb700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c06e570 msgr2=0x7f6d6c1372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.480+0000 7f6d723bb700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c06e570 0x7f6d6c1372a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.480+0000 7f6d723bb700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c06e570 0x7f6d6c1372a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.483+0000 7f6d72bbc700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d6c1d4d40 0x7f6d6c1d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.483+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 --> v1:172.21.15.89:6800/118778 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6358134ff0 con 0x7f6344035110 2023-09-30T22:16:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f6d72bbc700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d6c1d4d40 msgr2=0x7f6d6c1d51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-09-30T22:16:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f6d72bbc700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d6c1d4d40 msgr2=0x7f6d6c1d51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f6d72bbc700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d6c1d4d40 0x7f6d6c1d51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:16:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f6d72bbc700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d6c1d4d40 0x7f6d6c1d51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:16:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f633ffff700 1 -- 172.21.15.89:0/1893921828 <== osd.2 v1:172.21.15.89:6800/118778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2765323947) 0x7f635000bb30 con 0x7f6344035110 2023-09-30T22:16:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 >> v1:172.21.15.89:6800/118778 conn(0x7f6344035110 legacy=0x7f63440375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 >> v1:172.21.15.89:6816/118161 conn(0x7f634402c1b0 legacy=0x7f634402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f635806e570 msgr2=0x7f63581214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f635e0fb700 1 --2- 172.21.15.89:0/1893921828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f635806e570 0x7f63581214e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f634800b530 tx=0x7f634800b8f0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f6357fff700 1 -- 172.21.15.89:0/1893921828 reap_dead start 2023-09-30T22:16:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 shutdown_connections 2023-09-30T22:16:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 >> 172.21.15.89:0/1893921828 conn(0x7f63580a2aa0 msgr2=0x7f63580a0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 shutdown_connections 2023-09-30T22:16:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.484+0000 7f635e0fb700 1 -- 172.21.15.89:0/1893921828 wait complete. 2023-09-30T22:16:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.488+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000ad80 con 0x7f71c809e000 2023-09-30T22:16:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.488+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c00070d0 con 0x7f71c809e000 2023-09-30T22:16:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.488+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c0007480 con 0x7f71c809e000 2023-09-30T22:16:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.489+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c809c700 con 0x7f71c809e000 2023-09-30T22:16:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.489+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c813b430 con 0x7f71c809e000 2023-09-30T22:16:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.490+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71c809c000 con 0x7f71c809e000 2023-09-30T22:16:14.514 INFO:teuthology.orchestra.run.smithi089.stdout:38654705694 2023-09-30T22:16:14.515 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.491+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f71c00075e0 con 0x7f71c809e000 2023-09-30T22:16:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.491+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f71c003bc90 con 0x7f71c809e000 2023-09-30T22:16:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.491+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 --> v1:172.21.15.89:6804/118768 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71b4038d90 con 0x7f71b4035090 2023-09-30T22:16:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.491+0000 7f71c77fe700 1 -- 172.21.15.89:0/2020824182 >> v1:172.21.15.89:6816/118161 conn(0x7f71b402c1b0 legacy=0x7f71b402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.492+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== osd.0 v1:172.21.15.89:6804/118768 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f71b4038d90 con 0x7f71b4035090 2023-09-30T22:16:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.497+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c0007890 con 0x7f71c809e000 2023-09-30T22:16:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.499+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00198c0 con 0x7f71c809e000 2023-09-30T22:16:14.517 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:14.517 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:14.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.511+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 --> v1:172.21.15.89:6804/118768 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f71c809c450 con 0x7f71b4035090 2023-09-30T22:16:14.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.511+0000 7f71affff700 1 -- 172.21.15.89:0/2020824182 <== osd.0 v1:172.21.15.89:6804/118768 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2765323947) 0x7f71bc009640 con 0x7f71b4035090 2023-09-30T22:16:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 >> v1:172.21.15.89:6804/118768 conn(0x7f71b4035090 legacy=0x7f71b4037520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 >> v1:172.21.15.89:6816/118161 conn(0x7f71b402c1b0 legacy=0x7f71b402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71c809e000 msgr2=0x7f71c81364d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71cdd01700 1 --2- 172.21.15.89:0/2020824182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71c809e000 0x7f71c81364d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f71c0002090 tx=0x7f71c000a340 comp rx=0 tx=0).stop 2023-09-30T22:16:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71c7fff700 1 -- 172.21.15.89:0/2020824182 reap_dead start 2023-09-30T22:16:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 shutdown_connections 2023-09-30T22:16:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 >> 172.21.15.89:0/2020824182 conn(0x7f71c8066f50 msgr2=0x7f71c80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 shutdown_connections 2023-09-30T22:16:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.512+0000 7f71cdd01700 1 -- 172.21.15.89:0/2020824182 wait complete. 2023-09-30T22:16:14.526 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:14.527 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:14.529 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:14.529+0000 7fa1847b0240 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T22:16:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.675+0000 7fe2e1a98700 1 Processor -- start 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.675+0000 7fe2e1a98700 1 -- start start 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.675+0000 7fe2e1a98700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe2dc130040 0x7fe2dc12c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.675+0000 7fe2e1a98700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12ccb0 0x7fe2dc137950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.675+0000 7fe2e1a98700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc09be70 con 0x7fe2dc12fc70 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.675+0000 7fe2e1a98700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc09a6e0 con 0x7fe2dc130040 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.675+0000 7fe2e1a98700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc09ac80 con 0x7fe2dc12ccb0 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2daffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe2dc130040 0x7fe2dc12c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2daffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe2dc130040 0x7fe2dc12c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56542/0 (socket says 172.21.15.89:56542) 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2db7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe2dc12fc70 0x7fe2dc12c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35266/0 (socket says 172.21.15.89:35266) 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2daffd700 1 -- 172.21.15.89:0/567061810 learned_addr learned my addr 172.21.15.89:0/567061810 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2dbfff700 1 --2- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12ccb0 0x7fe2dc137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2daffd700 1 -- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12ccb0 msgr2=0x7fe2dc137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.679+0000 7f6d71bba700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c1377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2daffd700 1 --2- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12ccb0 0x7fe2dc137950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2daffd700 1 -- 172.21.15.89:0/567061810 >> v1:172.21.15.89:6789/0 conn(0x7fe2dc12fc70 legacy=0x7fe2dc12c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2daffd700 1 -- 172.21.15.89:0/567061810 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc099b00 con 0x7fe2dc130040 2023-09-30T22:16:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.676+0000 7fe2daffd700 1 --2- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc12c770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00b9a0 tx=0x7fe2cc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab63f37db08057c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.679+0000 7f6d71bba700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c06e570 msgr2=0x7f6d6c1372a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.679+0000 7f6d71bba700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d6c06e570 0x7f6d6c1372a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.679+0000 7f6d71bba700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d6c1d4d40 msgr2=0x7f6d6c1d51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.679+0000 7f6d71bba700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d6c1d4d40 0x7f6d6c1d51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.679+0000 7f6d71bba700 1 -- 172.21.15.89:0/3628593732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d64009a70 con 0x7f6d6c06f050 2023-09-30T22:16:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.679+0000 7fe2dbfff700 1 --2- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12ccb0 0x7fe2dc137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7fe2da7fc700 1 -- 172.21.15.89:0/567061810 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00caa0 con 0x7fe2dc130040 2023-09-30T22:16:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7f6d71bba700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c1377e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c010860 tx=0x7f6d5c010c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00e860 con 0x7f6d6c06f050 2023-09-30T22:16:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00c050 con 0x7f6d6c06f050 2023-09-30T22:16:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7fe2da7fc700 1 -- 172.21.15.89:0/567061810 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc00cc00 con 0x7fe2dc130040 2023-09-30T22:16:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7fe2da7fc700 1 -- 172.21.15.89:0/567061810 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00f460 con 0x7fe2dc130040 2023-09-30T22:16:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.681+0000 7fe2e1a98700 1 -- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 msgr2=0x7fe2dc12c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.681+0000 7fe2e1a98700 1 --2- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc12c770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00b9a0 tx=0x7fe2cc00bcb0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c13a3f0 con 0x7f6d6c06f050 2023-09-30T22:16:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c13e8e0 con 0x7f6d6c06f050 2023-09-30T22:16:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.680+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00ec40 con 0x7f6d6c06f050 2023-09-30T22:16:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.681+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d54000fb0 con 0x7f6d6c06f050 2023-09-30T22:16:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.682+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f6d5c015470 con 0x7f6d6c06f050 2023-09-30T22:16:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.682+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f6d5c03c400 con 0x7f6d6c06f050 2023-09-30T22:16:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.682+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 --> v1:172.21.15.89:6809/118777 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d50038ee0 con 0x7f6d500351e0 2023-09-30T22:16:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.682+0000 7f6d723bb700 1 -- 172.21.15.89:0/3628593732 >> v1:172.21.15.89:6816/118161 conn(0x7f6d5002c200 legacy=0x7f6d5002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.683+0000 7fe2e1a98700 1 -- 172.21.15.89:0/567061810 shutdown_connections 2023-09-30T22:16:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.683+0000 7fe2e1a98700 1 --2- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12ccb0 0x7fe2dc137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.683+0000 7fe2e1a98700 1 --2- 172.21.15.89:0/567061810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc12c770 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.683+0000 7fe2e1a98700 1 -- 172.21.15.89:0/567061810 >> 172.21.15.89:0/567061810 conn(0x7fe2dc09e7e0 msgr2=0x7fe2dc09ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.684+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 <== osd.1 v1:172.21.15.89:6809/118777 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f6d50038ee0 con 0x7f6d500351e0 2023-09-30T22:16:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.684+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d5c015720 con 0x7f6d6c06f050 2023-09-30T22:16:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.690+0000 7fb8777d5700 1 Processor -- start 2023-09-30T22:16:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.690+0000 7fb8777d5700 1 -- start start 2023-09-30T22:16:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb8777d5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb87012cf60 0x7fb87006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb8777d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb87012f8b0 0x7fb87012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb8777d5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87012cd90 con 0x7fb870130260 2023-09-30T22:16:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb8777d5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87012d330 con 0x7fb87012cf60 2023-09-30T22:16:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb8777d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb870066b20 con 0x7fb87012f8b0 2023-09-30T22:16:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb874d70700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb87012f8b0 0x7fb87012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb874d70700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb87012f8b0 0x7fb87012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52580/0 (socket says 172.21.15.89:52580) 2023-09-30T22:16:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb874d70700 1 -- 172.21.15.89:0/494099805 learned_addr learned my addr 172.21.15.89:0/494099805 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.691+0000 7fb875571700 1 --2- 172.21.15.89:0/494099805 >> v2:172.21.15.89:3300/0 conn(0x7fb87012cf60 0x7fb87006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.692+0000 7fb874d70700 1 -- 172.21.15.89:0/494099805 >> v2:172.21.15.89:3300/0 conn(0x7fb87012cf60 msgr2=0x7fb87006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.692+0000 7fb874d70700 1 --2- 172.21.15.89:0/494099805 >> v2:172.21.15.89:3300/0 conn(0x7fb87012cf60 0x7fb87006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.692+0000 7fb874d70700 1 -- 172.21.15.89:0/494099805 >> v1:172.21.15.89:6789/0 conn(0x7fb870130260 legacy=0x7fb870066410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.692+0000 7fb874d70700 1 -- 172.21.15.89:0/494099805 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8701306d0 con 0x7fb87012f8b0 2023-09-30T22:16:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.692+0000 7fb875571700 1 --2- 172.21.15.89:0/494099805 >> v2:172.21.15.89:3300/0 conn(0x7fb87012cf60 0x7fb87006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.692+0000 7fb874d70700 1 --2- 172.21.15.89:0/494099805 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb87012f8b0 0x7fb87012fd20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb86c010e10 tx=0x7fb86c00b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=691585a1e39c3add server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.696 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:14.695+0000 7fa1847b0240 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T22:16:14.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.705+0000 7fb867fff700 1 -- 172.21.15.89:0/494099805 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00be20 con 0x7fb87012f8b0 2023-09-30T22:16:14.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.705+0000 7fb867fff700 1 -- 172.21.15.89:0/494099805 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c004030 con 0x7fb87012f8b0 2023-09-30T22:16:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.705+0000 7fb867fff700 1 -- 172.21.15.89:0/494099805 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0043a0 con 0x7fb87012f8b0 2023-09-30T22:16:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.705+0000 7fb8777d5700 1 -- 172.21.15.89:0/494099805 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb87012f8b0 msgr2=0x7fb87012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.705+0000 7fb8777d5700 1 --2- 172.21.15.89:0/494099805 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb87012f8b0 0x7fb87012fd20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb86c010e10 tx=0x7fb86c00b7c0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.705+0000 7fe2e1a98700 1 -- 172.21.15.89:0/567061810 shutdown_connections 2023-09-30T22:16:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.705+0000 7fe2e1a98700 1 -- 172.21.15.89:0/567061810 wait complete. 2023-09-30T22:16:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.706+0000 7fb8777d5700 1 -- 172.21.15.89:0/494099805 shutdown_connections 2023-09-30T22:16:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.706+0000 7fb8777d5700 1 --2- 172.21.15.89:0/494099805 >> v2:172.21.15.89:3300/0 conn(0x7fb87012cf60 0x7fb87006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.706+0000 7fb8777d5700 1 --2- 172.21.15.89:0/494099805 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb87012f8b0 0x7fb87012fd20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.706+0000 7fb8777d5700 1 -- 172.21.15.89:0/494099805 >> 172.21.15.89:0/494099805 conn(0x7fb8700a2aa0 msgr2=0x7fb8700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.706+0000 7fb8777d5700 1 -- 172.21.15.89:0/494099805 shutdown_connections 2023-09-30T22:16:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.706+0000 7fb8777d5700 1 -- 172.21.15.89:0/494099805 wait complete. 2023-09-30T22:16:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2e1a98700 1 Processor -- start 2023-09-30T22:16:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2e1a98700 1 -- start start 2023-09-30T22:16:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fb8777d5700 1 Processor -- start 2023-09-30T22:16:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2e1a98700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2dc12ccb0 0x7fe2dc06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2e1a98700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12fc70 0x7fe2dc06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2e1a98700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2e1a98700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc08bbb0 con 0x7fe2dc130040 2023-09-30T22:16:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2e1a98700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc072220 con 0x7fe2dc12ccb0 2023-09-30T22:16:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2e1a98700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc129f60 con 0x7fe2dc12fc70 2023-09-30T22:16:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fe2dbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fe2dbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc06fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56556/0 (socket says 172.21.15.89:56556) 2023-09-30T22:16:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fe2dbfff700 1 -- 172.21.15.89:0/1775280774 learned_addr learned my addr 172.21.15.89:0/1775280774 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.707+0000 7fb8777d5700 1 -- start start 2023-09-30T22:16:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb8777d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb87012cf60 0x7fb870128e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb8777d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb87012f8b0 0x7fb870129350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb8777d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb870130260 0x7fb8701c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb8777d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87009d070 con 0x7fb87012cf60 2023-09-30T22:16:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb8777d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87009ce80 con 0x7fb87012f8b0 2023-09-30T22:16:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb8777d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb87009eb20 con 0x7fb870130260 2023-09-30T22:16:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875571700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb87012cf60 0x7fb870128e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875571700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb87012cf60 0x7fb870128e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56564/0 (socket says 172.21.15.89:56564) 2023-09-30T22:16:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875571700 1 -- 172.21.15.89:0/407522455 learned_addr learned my addr 172.21.15.89:0/407522455 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875d72700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb870130260 0x7fb8701c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb874d70700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb87012f8b0 0x7fb870129350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fe2db7fe700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2dc12ccb0 0x7fe2dc06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fe2daffd700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12fc70 0x7fe2dc06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875d72700 1 -- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb87012f8b0 msgr2=0x7fb870129350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875d72700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb87012f8b0 0x7fb870129350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875d72700 1 -- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb87012cf60 msgr2=0x7fb870128e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875d72700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb87012cf60 0x7fb870128e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fb875d72700 1 -- 172.21.15.89:0/407522455 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860013a10 con 0x7fb870130260 2023-09-30T22:16:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fe2dbfff700 1 -- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12fc70 msgr2=0x7fe2dc06f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fe2dbfff700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12fc70 0x7fe2dc06f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fe2dbfff700 1 -- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2dc12ccb0 msgr2=0x7fe2dc06f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.708+0000 7fe2dbfff700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2dc12ccb0 0x7fe2dc06f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.709+0000 7fe2dbfff700 1 -- 172.21.15.89:0/1775280774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d400bc50 con 0x7fe2dc130040 2023-09-30T22:16:14.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.709+0000 7fb875d72700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb870130260 0x7fb8701c40a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb86800c7b0 tx=0x7fb86800cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.709+0000 7fb865ffb700 1 -- 172.21.15.89:0/407522455 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868003e10 con 0x7fb870130260 2023-09-30T22:16:14.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.709+0000 7fe2dbfff700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc06fc10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40117b0 tx=0x7fe2d4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.709+0000 7fb865ffb700 1 -- 172.21.15.89:0/407522455 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8680050f0 con 0x7fb870130260 2023-09-30T22:16:14.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.709+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c010810 con 0x7fb870130260 2023-09-30T22:16:14.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.709+0000 7fb865ffb700 1 -- 172.21.15.89:0/407522455 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8680054a0 con 0x7fb870130260 2023-09-30T22:16:14.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.709+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb870129890 con 0x7fb870130260 2023-09-30T22:16:14.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.710+0000 7fe2c3fff700 1 -- 172.21.15.89:0/1775280774 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4003a20 con 0x7fe2dc130040 2023-09-30T22:16:14.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.710+0000 7fe2c3fff700 1 -- 172.21.15.89:0/1775280774 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d4003b80 con 0x7fe2dc130040 2023-09-30T22:16:14.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.710+0000 7fe2c3fff700 1 -- 172.21.15.89:0/1775280774 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4010650 con 0x7fe2dc130040 2023-09-30T22:16:14.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.710+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87009e750 con 0x7fb870130260 2023-09-30T22:16:14.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.711+0000 7fb865ffb700 1 -- 172.21.15.89:0/407522455 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fb868005600 con 0x7fb870130260 2023-09-30T22:16:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.711+0000 7fb865ffb700 1 -- 172.21.15.89:0/407522455 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fb86803aa90 con 0x7fb870130260 2023-09-30T22:16:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.710+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2cc00b600 con 0x7fe2dc130040 2023-09-30T22:16:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.710+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc08b7e0 con 0x7fe2dc130040 2023-09-30T22:16:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.711+0000 7fb875571700 1 -- 172.21.15.89:0/407522455 >> v1:172.21.15.89:6816/118161 conn(0x7fb85402c1b0 legacy=0x7fb85402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.712+0000 7fe2c3fff700 1 -- 172.21.15.89:0/1775280774 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fe2d40107b0 con 0x7fe2dc130040 2023-09-30T22:16:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.712+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc0725b0 con 0x7fe2dc130040 2023-09-30T22:16:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.712+0000 7fe2c3fff700 1 -- 172.21.15.89:0/1775280774 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fe2d4040c90 con 0x7fe2dc130040 2023-09-30T22:16:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.712+0000 7fe2db7fe700 1 -- 172.21.15.89:0/1775280774 >> v1:172.21.15.89:6816/118161 conn(0x7fe2c802c200 legacy=0x7fe2c802e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53b48c8700 1 Processor -- start 2023-09-30T22:16:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 --> v1:172.21.15.89:6809/118777 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d54002c60 con 0x7f6d500351e0 2023-09-30T22:16:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f6d62ffd700 1 -- 172.21.15.89:0/3628593732 <== osd.1 v1:172.21.15.89:6809/118777 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1880693312) 0x7f6d54002c60 con 0x7f6d500351e0 2023-09-30T22:16:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53b48c8700 1 -- start start 2023-09-30T22:16:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53b48c8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53b012ffc0 0x7f53b0134a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53b48c8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b0134fc0 0x7f53b01373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53b48c8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b0072c10 con 0x7f53b012f4e0 2023-09-30T22:16:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53b48c8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b01303b0 con 0x7f53b012ffc0 2023-09-30T22:16:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53b48c8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0072520 con 0x7f53b0134fc0 2023-09-30T22:16:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53af7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f53b012f4e0 0x7f53b012f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35284/0 (socket says 172.21.15.89:35284) 2023-09-30T22:16:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.714+0000 7f53af7fe700 1 -- 172.21.15.89:0/3610451821 learned_addr learned my addr 172.21.15.89:0/3610451821 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f53aeffd700 1 --2- 172.21.15.89:0/3610451821 >> v2:172.21.15.89:3300/0 conn(0x7f53b012ffc0 0x7f53b0134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f53affff700 1 --2- 172.21.15.89:0/3610451821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b0134fc0 0x7f53b01373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f53aeffd700 1 -- 172.21.15.89:0/3610451821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b0134fc0 msgr2=0x7f53b01373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f53aeffd700 1 --2- 172.21.15.89:0/3610451821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b0134fc0 0x7f53b01373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f53aeffd700 1 -- 172.21.15.89:0/3610451821 >> v1:172.21.15.89:6789/0 conn(0x7f53b012f4e0 legacy=0x7f53b012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f53aeffd700 1 -- 172.21.15.89:0/3610451821 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b009def0 con 0x7f53b012ffc0 2023-09-30T22:16:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f53aeffd700 1 --2- 172.21.15.89:0/3610451821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012ffc0 0x7f53b0134a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006b00 tx=0x7f53a400ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa2cd67d42cf059 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 >> v1:172.21.15.89:6809/118777 conn(0x7f6d500351e0 legacy=0x7f6d50037670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 >> v1:172.21.15.89:6816/118161 conn(0x7f6d5002c200 legacy=0x7f6d5002e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 msgr2=0x7f6d6c1377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f6d7461f700 1 --2- 172.21.15.89:0/3628593732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d6c06f050 0x7f6d6c1377e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c010860 tx=0x7f6d5c010c20 comp rx=0 tx=0).stop 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.715+0000 7f6d72bbc700 1 -- 172.21.15.89:0/3628593732 reap_dead start 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.716+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 shutdown_connections 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.716+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 >> 172.21.15.89:0/3628593732 conn(0x7f6d6c0a2aa0 msgr2=0x7f6d6c09dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.716+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 shutdown_connections 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.716+0000 7f6d7461f700 1 -- 172.21.15.89:0/3628593732 wait complete. 2023-09-30T22:16:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.716+0000 7fb865ffb700 1 -- 172.21.15.89:0/407522455 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fb86800ee40 con 0x7fb870130260 2023-09-30T22:16:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.717+0000 7fe2c3fff700 1 -- 172.21.15.89:0/1775280774 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fe2d401ad10 con 0x7fe2dc130040 2023-09-30T22:16:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.717+0000 7f53ae7fc700 1 -- 172.21.15.89:0/3610451821 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4004230 con 0x7f53b012ffc0 2023-09-30T22:16:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.717+0000 7f53ae7fc700 1 -- 172.21.15.89:0/3610451821 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a400d040 con 0x7f53b012ffc0 2023-09-30T22:16:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.717+0000 7f53ae7fc700 1 -- 172.21.15.89:0/3610451821 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a40036a0 con 0x7f53b012ffc0 2023-09-30T22:16:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.717+0000 7f53b48c8700 1 -- 172.21.15.89:0/3610451821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012ffc0 msgr2=0x7f53b0134a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.717+0000 7f53b48c8700 1 --2- 172.21.15.89:0/3610451821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012ffc0 0x7f53b0134a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006b00 tx=0x7f53a400ab40 comp rx=0 tx=0).stop 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.718+0000 7f53b48c8700 1 -- 172.21.15.89:0/3610451821 shutdown_connections 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.718+0000 7f53b48c8700 1 --2- 172.21.15.89:0/3610451821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b0134fc0 0x7f53b01373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.718+0000 7f53b48c8700 1 --2- 172.21.15.89:0/3610451821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012ffc0 0x7f53b0134a80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.718+0000 7f53b48c8700 1 -- 172.21.15.89:0/3610451821 >> 172.21.15.89:0/3610451821 conn(0x7f53b00a27d0 msgr2=0x7f53b009e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.718+0000 7f53b48c8700 1 -- 172.21.15.89:0/3610451821 shutdown_connections 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.718+0000 7f53b48c8700 1 -- 172.21.15.89:0/3610451821 wait complete. 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.719+0000 7f53b48c8700 1 Processor -- start 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.719+0000 7f53b48c8700 1 -- start start 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53b48c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012f4e0 0x7f53b011b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53b48c8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b012ffc0 0x7f53b011b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53b48c8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53b0134fc0 0x7f53b011bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53b48c8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b011c780 con 0x7f53b012f4e0 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53b48c8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b011cec0 con 0x7f53b012ffc0 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53b48c8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53b011edd0 con 0x7f53b0134fc0 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53af7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012f4e0 0x7f53b011b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53af7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012f4e0 0x7f53b011b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56588/0 (socket says 172.21.15.89:56588) 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.720+0000 7f53af7fe700 1 -- 172.21.15.89:0/449969700 learned_addr learned my addr 172.21.15.89:0/449969700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f53af7fe700 1 -- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53b0134fc0 msgr2=0x7f53b011bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f53af7fe700 1 --2- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53b0134fc0 0x7f53b011bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f53af7fe700 1 -- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b012ffc0 msgr2=0x7f53b011b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f53af7fe700 1 --2- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b012ffc0 0x7f53b011b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f53af7fe700 1 -- 172.21.15.89:0/449969700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a400a4c0 con 0x7f53b012f4e0 2023-09-30T22:16:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f53af7fe700 1 --2- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012f4e0 0x7f53b011b2b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c960 tx=0x7f53a000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f5397fff700 1 -- 172.21.15.89:0/449969700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0007b80 con 0x7f53b012f4e0 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f5397fff700 1 -- 172.21.15.89:0/449969700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a0007ce0 con 0x7f53b012f4e0 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b012e130 con 0x7f53b012f4e0 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f5397fff700 1 -- 172.21.15.89:0/449969700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0016640 con 0x7f53b012f4e0 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.721+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b011d430 con 0x7f53b012f4e0 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.722+0000 7f5397fff700 1 -- 172.21.15.89:0/449969700 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f53a00167a0 con 0x7f53b012f4e0 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.722+0000 7f5397fff700 1 -- 172.21.15.89:0/449969700 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f53a0012070 con 0x7f53b012f4e0 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.722+0000 7f53aeffd700 1 -- 172.21.15.89:0/449969700 >> v1:172.21.15.89:6816/118161 conn(0x7f539c02c1b0 legacy=0x7f539c02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.722+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b004f2a0 con 0x7f53b012f4e0 2023-09-30T22:16:14.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.727+0000 7f5397fff700 1 -- 172.21.15.89:0/449969700 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f53a0007020 con 0x7f53b012f4e0 2023-09-30T22:16:14.734 INFO:teuthology.orchestra.run.smithi089.stdout:38654705694 2023-09-30T22:16:14.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:14.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.869+0000 7f703bcf9700 1 Processor -- start 2023-09-30T22:16:14.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.869+0000 7f703bcf9700 1 -- start start 2023-09-30T22:16:14.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f703bcf9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f703bcf9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70341380f0 0x7f703413fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f703bcf9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7034134980 con 0x7f7034134ec0 2023-09-30T22:16:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f703bcf9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703406c4b0 con 0x7f703406c0e0 2023-09-30T22:16:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f703bcf9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70340a09c0 con 0x7f70341380f0 2023-09-30T22:16:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f7039294700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7034134ec0 0x7f70341379e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35288/0 (socket says 172.21.15.89:35288) 2023-09-30T22:16:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f7039294700 1 -- 172.21.15.89:0/3812306562 learned_addr learned my addr 172.21.15.89:0/3812306562 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f7039a95700 1 --2- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.870+0000 7f703a296700 1 --2- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70341380f0 0x7f703413fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f7038a93700 1 -- 172.21.15.89:0/3812306562 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407645632 0 0) 0x7f7034134980 con 0x7f7034134ec0 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f7038a93700 1 -- 172.21.15.89:0/3812306562 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70200035b0 con 0x7f7034134ec0 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f703a296700 1 -- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 msgr2=0x7f703409ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f703a296700 1 --2- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703409ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f703a296700 1 -- 172.21.15.89:0/3812306562 >> v1:172.21.15.89:6789/0 conn(0x7f7034134ec0 legacy=0x7f70341379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f703a296700 1 -- 172.21.15.89:0/3812306562 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70340a0600 con 0x7f70341380f0 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f703a296700 1 --2- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70341380f0 0x7f703413fd20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f702c009dc0 tx=0x7f702c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403105ac2413f029 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f7038a93700 1 -- 172.21.15.89:0/3812306562 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c00e070 con 0x7f70341380f0 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f7038a93700 1 -- 172.21.15.89:0/3812306562 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c00bb20 con 0x7f70341380f0 2023-09-30T22:16:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.871+0000 7f7038a93700 1 -- 172.21.15.89:0/3812306562 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c012430 con 0x7f70341380f0 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.872+0000 7f703bcf9700 1 -- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70341380f0 msgr2=0x7f703413fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.872+0000 7f703bcf9700 1 --2- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70341380f0 0x7f703413fd20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f702c009dc0 tx=0x7f702c00b330 comp rx=0 tx=0).stop 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.872+0000 7f703bcf9700 1 -- 172.21.15.89:0/3812306562 shutdown_connections 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.872+0000 7f703bcf9700 1 --2- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.872+0000 7f703bcf9700 1 --2- 172.21.15.89:0/3812306562 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70341380f0 0x7f703413fd20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.872+0000 7f703bcf9700 1 -- 172.21.15.89:0/3812306562 >> 172.21.15.89:0/3812306562 conn(0x7f70340a4210 msgr2=0x7f70340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.872+0000 7f703bcf9700 1 -- 172.21.15.89:0/3812306562 shutdown_connections 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.872+0000 7f703bcf9700 1 -- 172.21.15.89:0/3812306562 wait complete. 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.873+0000 7f703bcf9700 1 Processor -- start 2023-09-30T22:16:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.873+0000 7f703bcf9700 1 -- start start 2023-09-30T22:16:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703bcf9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703411a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703bcf9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7034134ec0 0x7f703411abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703bcf9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70341380f0 0x7f703411b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703bcf9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703411d780 con 0x7f70341380f0 2023-09-30T22:16:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703bcf9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703411cca0 con 0x7f703406c0e0 2023-09-30T22:16:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703bcf9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70340a03f0 con 0x7f7034134ec0 2023-09-30T22:16:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703a296700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70341380f0 0x7f703411b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703a296700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70341380f0 0x7f703411b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56602/0 (socket says 172.21.15.89:56602) 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f703a296700 1 -- 172.21.15.89:0/4287589104 learned_addr learned my addr 172.21.15.89:0/4287589104 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f7039294700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7034134ec0 0x7f703411abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.874+0000 7f7039a95700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703411a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f703a296700 1 -- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7034134ec0 msgr2=0x7f703411abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f703a296700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7034134ec0 0x7f703411abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f703a296700 1 -- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 msgr2=0x7f703411a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f703a296700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703411a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f703a296700 1 -- 172.21.15.89:0/4287589104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c009a70 con 0x7f70341380f0 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f7039a95700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703411a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f703a296700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70341380f0 0x7f703411b100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f702c014040 tx=0x7f702c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f702a7fc700 1 -- 172.21.15.89:0/4287589104 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c012430 con 0x7f70341380f0 2023-09-30T22:16:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f702a7fc700 1 -- 172.21.15.89:0/4287589104 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c020070 con 0x7f70341380f0 2023-09-30T22:16:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70340a1610 con 0x7f70341380f0 2023-09-30T22:16:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f702a7fc700 1 -- 172.21.15.89:0/4287589104 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c012ea0 con 0x7f70341380f0 2023-09-30T22:16:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.875+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703411da40 con 0x7f70341380f0 2023-09-30T22:16:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.876+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7034074ea0 con 0x7f70341380f0 2023-09-30T22:16:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.876+0000 7f702a7fc700 1 -- 172.21.15.89:0/4287589104 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f702c004280 con 0x7f70341380f0 2023-09-30T22:16:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.877+0000 7f702a7fc700 1 -- 172.21.15.89:0/4287589104 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f702c022070 con 0x7f70341380f0 2023-09-30T22:16:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.877+0000 7f7039a95700 1 -- 172.21.15.89:0/4287589104 >> v1:172.21.15.89:6816/118161 conn(0x7f701802c1e0 legacy=0x7f701802e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.881+0000 7f702a7fc700 1 -- 172.21.15.89:0/4287589104 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f702c003e30 con 0x7f70341380f0 2023-09-30T22:16:14.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.906+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f53b011e740 con 0x7f53b012f4e0 2023-09-30T22:16:14.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.906+0000 7f5397fff700 1 -- 172.21.15.89:0/449969700 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f53a0014920 con 0x7f53b012f4e0 2023-09-30T22:16:14.907 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:14.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 >> v1:172.21.15.89:6816/118161 conn(0x7f539c02c1b0 legacy=0x7f539c02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012f4e0 msgr2=0x7f53b011b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 --2- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012f4e0 0x7f53b011b2b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c960 tx=0x7f53a000cd20 comp rx=0 tx=0).stop 2023-09-30T22:16:14.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 shutdown_connections 2023-09-30T22:16:14.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 --2- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53b0134fc0 0x7f53b011bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 --2- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53b012f4e0 0x7f53b011b2b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 --2- 172.21.15.89:0/449969700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53b012ffc0 0x7f53b011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 >> 172.21.15.89:0/449969700 conn(0x7f53b00a27d0 msgr2=0x7f53b0099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 shutdown_connections 2023-09-30T22:16:14.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.908+0000 7f53b48c8700 1 -- 172.21.15.89:0/449969700 wait complete. 2023-09-30T22:16:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.912+0000 7fb875571700 1 -- 172.21.15.89:0/407522455 >> v1:172.21.15.89:6816/118161 conn(0x7fb85402c1b0 legacy=0x7fb85402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.913+0000 7fe2db7fe700 1 -- 172.21.15.89:0/1775280774 >> v1:172.21.15.89:6816/118161 conn(0x7fe2c802c200 legacy=0x7fe2c802e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:14.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:16:14.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.923+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb870074ea0 con 0x7fb870130260 2023-09-30T22:16:14.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.923+0000 7fb865ffb700 1 -- 172.21.15.89:0/407522455 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb868017070 con 0x7fb870130260 2023-09-30T22:16:14.924 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:14.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 >> v1:172.21.15.89:6816/118161 conn(0x7fb85402c1b0 legacy=0x7fb85402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:14.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb870130260 msgr2=0x7fb8701c40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:14.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb870130260 0x7fb8701c40a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb86800c7b0 tx=0x7fb86800cac0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 shutdown_connections 2023-09-30T22:16:14.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb870130260 0x7fb8701c40a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb87012cf60 0x7fb870128e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 --2- 172.21.15.89:0/407522455 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb87012f8b0 0x7fb870129350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:14.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 >> 172.21.15.89:0/407522455 conn(0x7fb8700a2aa0 msgr2=0x7fb8701b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:14.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 shutdown_connections 2023-09-30T22:16:14.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:14.925+0000 7fb8777d5700 1 -- 172.21.15.89:0/407522455 wait complete. 2023-09-30T22:16:14.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.2 2023-09-30T22:16:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.005+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe2dc0a1380 con 0x7fe2dc130040 2023-09-30T22:16:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.006+0000 7fe2c3fff700 1 -- 172.21.15.89:0/1775280774 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe2d4024e60 con 0x7fe2dc130040 2023-09-30T22:16:15.007 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.007+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 >> v1:172.21.15.89:6816/118161 conn(0x7fe2c802c200 legacy=0x7fe2c802e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 msgr2=0x7fe2dc06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc06fc10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40117b0 tx=0x7fe2d4011b70 comp rx=0 tx=0).stop 2023-09-30T22:16:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 shutdown_connections 2023-09-30T22:16:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2dc130040 0x7fe2dc06fc10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe2dc12ccb0 0x7fe2dc06f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 --2- 172.21.15.89:0/1775280774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe2dc12fc70 0x7fe2dc06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 >> 172.21.15.89:0/1775280774 conn(0x7fe2dc09e7e0 msgr2=0x7fe2dc129580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 shutdown_connections 2023-09-30T22:16:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.008+0000 7fe2e1a98700 1 -- 172.21.15.89:0/1775280774 wait complete. 2023-09-30T22:16:15.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.0 2023-09-30T22:16:15.059 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:15.058+0000 7fa1847b0240 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T22:16:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.068+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f703406b250 con 0x7f70341380f0 2023-09-30T22:16:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.068+0000 7f702a7fc700 1 -- 172.21.15.89:0/4287589104 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f702c01a380 con 0x7f70341380f0 2023-09-30T22:16:15.069 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 >> v1:172.21.15.89:6816/118161 conn(0x7f701802c1e0 legacy=0x7f701802e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70341380f0 msgr2=0x7f703411b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70341380f0 0x7f703411b100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f702c014040 tx=0x7f702c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 shutdown_connections 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70341380f0 0x7f703411b100 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f703406c0e0 0x7f703411a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 --2- 172.21.15.89:0/4287589104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7034134ec0 0x7f703411abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 >> 172.21.15.89:0/4287589104 conn(0x7f70340a4210 msgr2=0x7f7034137130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 shutdown_connections 2023-09-30T22:16:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:15.070+0000 7f703bcf9700 1 -- 172.21.15.89:0/4287589104 wait complete. 2023-09-30T22:16:15.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:16:15.538 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:15.538+0000 7fa1847b0240 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T22:16:15.654 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:15.654+0000 7fa1847b0240 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-30T22:16:15.922 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:15.940 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:15.974 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:15.974+0000 7fa1847b0240 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T22:16:16.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:16.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.054+0000 7fabdfd9a700 1 Processor -- start 2023-09-30T22:16:16.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.054+0000 7fabdfd9a700 1 -- start start 2023-09-30T22:16:16.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.054+0000 7fabdfd9a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81338c0 0x7fabd8133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabdfd9a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fabd809e560 0x7fabd813b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabdfd9a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd806f720 con 0x7fabd81341d0 2023-09-30T22:16:16.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabdfd9a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd8070c80 con 0x7fabd809e560 2023-09-30T22:16:16.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabdfd9a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabd806fce0 con 0x7fabd81338c0 2023-09-30T22:16:16.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabddb36700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81338c0 0x7fabd8133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabddb36700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81338c0 0x7fabd8133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52632/0 (socket says 172.21.15.89:52632) 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabddb36700 1 -- 172.21.15.89:0/4186107016 learned_addr learned my addr 172.21.15.89:0/4186107016 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabddb36700 1 -- 172.21.15.89:0/4186107016 >> v2:172.21.15.89:3300/0 conn(0x7fabd809e560 msgr2=0x7fabd813b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabddb36700 1 --2- 172.21.15.89:0/4186107016 >> v2:172.21.15.89:3300/0 conn(0x7fabd809e560 0x7fabd813b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabddb36700 1 -- 172.21.15.89:0/4186107016 >> v1:172.21.15.89:6789/0 conn(0x7fabd81341d0 legacy=0x7fabd809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabddb36700 1 -- 172.21.15.89:0/4186107016 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd81336a0 con 0x7fabd81338c0 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabddb36700 1 --2- 172.21.15.89:0/4186107016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81338c0 0x7fabd8133c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fabd40098f0 tx=0x7fabd4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d1a387554b69f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabdcb34700 1 -- 172.21.15.89:0/4186107016 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4004230 con 0x7fabd81338c0 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabdcb34700 1 -- 172.21.15.89:0/4186107016 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd4004390 con 0x7fabd81338c0 2023-09-30T22:16:16.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.055+0000 7fabdcb34700 1 -- 172.21.15.89:0/4186107016 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40036a0 con 0x7fabd81338c0 2023-09-30T22:16:16.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.056+0000 7fabdfd9a700 1 -- 172.21.15.89:0/4186107016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81338c0 msgr2=0x7fabd8133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.056+0000 7fabdfd9a700 1 --2- 172.21.15.89:0/4186107016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81338c0 0x7fabd8133c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fabd40098f0 tx=0x7fabd4009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:16.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.056+0000 7fabdfd9a700 1 -- 172.21.15.89:0/4186107016 shutdown_connections 2023-09-30T22:16:16.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.056+0000 7fabdfd9a700 1 --2- 172.21.15.89:0/4186107016 >> v2:172.21.15.89:3300/0 conn(0x7fabd809e560 0x7fabd813b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.056+0000 7fabdfd9a700 1 --2- 172.21.15.89:0/4186107016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81338c0 0x7fabd8133c90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.056+0000 7fabdfd9a700 1 -- 172.21.15.89:0/4186107016 >> 172.21.15.89:0/4186107016 conn(0x7fabd80a2aa0 msgr2=0x7fabd80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:16.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.057+0000 7fabdfd9a700 1 -- 172.21.15.89:0/4186107016 shutdown_connections 2023-09-30T22:16:16.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.057+0000 7fabdfd9a700 1 -- 172.21.15.89:0/4186107016 wait complete. 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.057+0000 7fabdfd9a700 1 Processor -- start 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.057+0000 7fabdfd9a700 1 -- start start 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabdfd9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabd809e560 0x7fabd811f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabdfd9a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabd81338c0 0x7fabd811fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabdfd9a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81341d0 0x7fabd81203b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabdfd9a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8122e60 con 0x7fabd809e560 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabdfd9a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8121d30 con 0x7fabd81338c0 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabdfd9a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80a1360 con 0x7fabd81341d0 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabde337700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81341d0 0x7fabd81203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabde337700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81341d0 0x7fabd81203b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52648/0 (socket says 172.21.15.89:52648) 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabde337700 1 -- 172.21.15.89:0/774135179 learned_addr learned my addr 172.21.15.89:0/774135179 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabdd335700 1 --2- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabd81338c0 0x7fabd811fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabde337700 1 -- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabd81338c0 msgr2=0x7fabd811fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabde337700 1 --2- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabd81338c0 0x7fabd811fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabde337700 1 -- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabd809e560 msgr2=0x7fabd811f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabde337700 1 --2- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabd809e560 0x7fabd811f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.058+0000 7fabde337700 1 -- 172.21.15.89:0/774135179 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd4009550 con 0x7fabd81341d0 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.059+0000 7fabde337700 1 --2- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81341d0 0x7fabd81203b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fabd000b9a0 tx=0x7fabd000bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:16.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.059+0000 7fabce7fc700 1 -- 172.21.15.89:0/774135179 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000c930 con 0x7fabd81341d0 2023-09-30T22:16:16.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.059+0000 7fabce7fc700 1 -- 172.21.15.89:0/774135179 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd000cab0 con 0x7fabd81341d0 2023-09-30T22:16:16.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.059+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd806f960 con 0x7fabd81341d0 2023-09-30T22:16:16.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.059+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd8123b90 con 0x7fabd81341d0 2023-09-30T22:16:16.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.059+0000 7fabce7fc700 1 -- 172.21.15.89:0/774135179 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000ce60 con 0x7fabd81341d0 2023-09-30T22:16:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.060+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc00052e0 con 0x7fabd81341d0 2023-09-30T22:16:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.060+0000 7fabce7fc700 1 -- 172.21.15.89:0/774135179 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fabd001d3b0 con 0x7fabd81341d0 2023-09-30T22:16:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.061+0000 7fabce7fc700 1 -- 172.21.15.89:0/774135179 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fabd003b900 con 0x7fabd81341d0 2023-09-30T22:16:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.061+0000 7fabddb36700 1 -- 172.21.15.89:0/774135179 >> v1:172.21.15.89:6816/118161 conn(0x7fabbc02c1b0 legacy=0x7fabbc02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.069+0000 7fabce7fc700 1 -- 172.21.15.89:0/774135179 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fabd0007370 con 0x7fabd81341d0 2023-09-30T22:16:16.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:16.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.096+0000 7fe881c1f700 1 Processor -- start 2023-09-30T22:16:16.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.096+0000 7fe881c1f700 1 -- start start 2023-09-30T22:16:16.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe881c1f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09d6f0 0x7fe87c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe881c1f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09e000 0x7fe87c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe881c1f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87c09b120 con 0x7fe87c09e410 2023-09-30T22:16:16.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe881c1f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c06b460 con 0x7fe87c09e000 2023-09-30T22:16:16.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe881c1f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c09a7e0 con 0x7fe87c09d6f0 2023-09-30T22:16:16.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09d6f0 0x7fe87c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09d6f0 0x7fe87c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52650/0 (socket says 172.21.15.89:52650) 2023-09-30T22:16:16.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87b7fe700 1 -- 172.21.15.89:0/720078135 learned_addr learned my addr 172.21.15.89:0/720078135 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87affd700 1 --2- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09e000 0x7fe87c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87b7fe700 1 -- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09e000 msgr2=0x7fe87c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87b7fe700 1 --2- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09e000 0x7fe87c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87b7fe700 1 -- 172.21.15.89:0/720078135 >> v1:172.21.15.89:6789/0 conn(0x7fe87c09e410 legacy=0x7fe87c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87b7fe700 1 -- 172.21.15.89:0/720078135 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c09a500 con 0x7fe87c09d6f0 2023-09-30T22:16:16.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.097+0000 7fe87affd700 1 --2- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09e000 0x7fe87c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.098+0000 7fe87b7fe700 1 --2- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09d6f0 0x7fe87c09dac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009950 tx=0x7fe86c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3d16a2403dcd63a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.098+0000 7fe87a7fc700 1 -- 172.21.15.89:0/720078135 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c004230 con 0x7fe87c09d6f0 2023-09-30T22:16:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.098+0000 7fe87a7fc700 1 -- 172.21.15.89:0/720078135 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c004390 con 0x7fe87c09d6f0 2023-09-30T22:16:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.098+0000 7fe87a7fc700 1 -- 172.21.15.89:0/720078135 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0036a0 con 0x7fe87c09d6f0 2023-09-30T22:16:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.099+0000 7fe881c1f700 1 -- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09d6f0 msgr2=0x7fe87c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.099+0000 7fe881c1f700 1 --2- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09d6f0 0x7fe87c09dac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009950 tx=0x7fe86c009c60 comp rx=0 tx=0).stop 2023-09-30T22:16:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.099+0000 7fe881c1f700 1 -- 172.21.15.89:0/720078135 shutdown_connections 2023-09-30T22:16:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.099+0000 7fe881c1f700 1 --2- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09d6f0 0x7fe87c09dac0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.099+0000 7fe881c1f700 1 --2- 172.21.15.89:0/720078135 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09e000 0x7fe87c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.099+0000 7fe881c1f700 1 -- 172.21.15.89:0/720078135 >> 172.21.15.89:0/720078135 conn(0x7fe87c066f50 msgr2=0x7fe87c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.100+0000 7fe881c1f700 1 -- 172.21.15.89:0/720078135 shutdown_connections 2023-09-30T22:16:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.100+0000 7fe881c1f700 1 -- 172.21.15.89:0/720078135 wait complete. 2023-09-30T22:16:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.101+0000 7fe881c1f700 1 Processor -- start 2023-09-30T22:16:16.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.101+0000 7fe881c1f700 1 -- start start 2023-09-30T22:16:16.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.102+0000 7fe881c1f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09d6f0 0x7fe87c119720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.102+0000 7fe881c1f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c09e000 0x7fe87c119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.102+0000 7fe881c1f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09e410 0x7fe87c11a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.102+0000 7fe881c1f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c11bb60 con 0x7fe87c09e000 2023-09-30T22:16:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.102+0000 7fe881c1f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c0a3140 con 0x7fe87c09d6f0 2023-09-30T22:16:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.102+0000 7fe881c1f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c09c710 con 0x7fe87c09e410 2023-09-30T22:16:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09e410 0x7fe87c11a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09e410 0x7fe87c11a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52664/0 (socket says 172.21.15.89:52664) 2023-09-30T22:16:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 -- 172.21.15.89:0/3583380765 learned_addr learned my addr 172.21.15.89:0/3583380765 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09d6f0 0x7fe87c119720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 -- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09d6f0 msgr2=0x7fe87c119720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 --2- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09d6f0 0x7fe87c119720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 -- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c09e000 msgr2=0x7fe87c119c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 --2- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c09e000 0x7fe87c119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 -- 172.21.15.89:0/3583380765 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700094a0 con 0x7fe87c09e410 2023-09-30T22:16:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87b7fe700 1 --2- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09d6f0 0x7fe87c119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe87bfff700 1 --2- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09e410 0x7fe87c11a1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c7b0 tx=0x7fe87400cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe863fff700 1 -- 172.21.15.89:0/3583380765 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874003e10 con 0x7fe87c09e410 2023-09-30T22:16:16.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe863fff700 1 -- 172.21.15.89:0/3583380765 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe874005070 con 0x7fe87c09e410 2023-09-30T22:16:16.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.103+0000 7fe863fff700 1 -- 172.21.15.89:0/3583380765 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874005420 con 0x7fe87c09e410 2023-09-30T22:16:16.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.104+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c009550 con 0x7fe87c09e410 2023-09-30T22:16:16.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.104+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c1b9420 con 0x7fe87c09e410 2023-09-30T22:16:16.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.105+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87c074ea0 con 0x7fe87c09e410 2023-09-30T22:16:16.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.106+0000 7fe863fff700 1 -- 172.21.15.89:0/3583380765 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fe874005580 con 0x7fe87c09e410 2023-09-30T22:16:16.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.106+0000 7fe863fff700 1 -- 172.21.15.89:0/3583380765 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fe87403ab10 con 0x7fe87c09e410 2023-09-30T22:16:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.106+0000 7fe87b7fe700 1 -- 172.21.15.89:0/3583380765 >> v1:172.21.15.89:6816/118161 conn(0x7fe86802c1b0 legacy=0x7fe86802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.110+0000 7fe863fff700 1 -- 172.21.15.89:0/3583380765 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fe874005830 con 0x7fe87c09e410 2023-09-30T22:16:16.133 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:16.133+0000 7fa1847b0240 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T22:16:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.206+0000 7f2833d42700 1 Processor -- start 2023-09-30T22:16:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.207+0000 7f2833d42700 1 -- start start 2023-09-30T22:16:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.207+0000 7f2833d42700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f282c1338a0 0x7f282c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.207+0000 7f2833d42700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09e930 0x7f282c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.207+0000 7f2833d42700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c06ef30 con 0x7f282c09de50 2023-09-30T22:16:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.207+0000 7f2833d42700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282c06fef0 con 0x7f282c1338a0 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.207+0000 7f2833d42700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1334b0 con 0x7f282c09e930 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.207+0000 7f28312dd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f282c09de50 0x7f282c09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35312/0 (socket says 172.21.15.89:35312) 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f28312dd700 1 -- 172.21.15.89:0/1555452649 learned_addr learned my addr 172.21.15.89:0/1555452649 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f28322df700 1 --2- 172.21.15.89:0/1555452649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09e930 0x7f282c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f2831ade700 1 --2- 172.21.15.89:0/1555452649 >> v2:172.21.15.89:3300/0 conn(0x7f282c1338a0 0x7f282c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f28322df700 1 -- 172.21.15.89:0/1555452649 >> v2:172.21.15.89:3300/0 conn(0x7f282c1338a0 msgr2=0x7f282c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f28322df700 1 --2- 172.21.15.89:0/1555452649 >> v2:172.21.15.89:3300/0 conn(0x7f282c1338a0 0x7f282c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f28322df700 1 -- 172.21.15.89:0/1555452649 >> v1:172.21.15.89:6789/0 conn(0x7f282c09de50 legacy=0x7f282c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f28322df700 1 -- 172.21.15.89:0/1555452649 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c06fc30 con 0x7f282c09e930 2023-09-30T22:16:16.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f2831ade700 1 --2- 172.21.15.89:0/1555452649 >> v2:172.21.15.89:3300/0 conn(0x7f282c1338a0 0x7f282c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:16.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f28322df700 1 --2- 172.21.15.89:0/1555452649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09e930 0x7f282c13b740 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2824009d90 tx=0x7f282400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ce6fbafadc8ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:16.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.208+0000 7f2830adc700 1 -- 172.21.15.89:0/1555452649 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282400e070 con 0x7f282c09e930 2023-09-30T22:16:16.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2830adc700 1 -- 172.21.15.89:0/1555452649 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282400bb20 con 0x7f282c09e930 2023-09-30T22:16:16.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2830adc700 1 -- 172.21.15.89:0/1555452649 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824012430 con 0x7f282c09e930 2023-09-30T22:16:16.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2833d42700 1 -- 172.21.15.89:0/1555452649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09e930 msgr2=0x7f282c13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2833d42700 1 --2- 172.21.15.89:0/1555452649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09e930 0x7f282c13b740 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2824009d90 tx=0x7f282400b330 comp rx=0 tx=0).stop 2023-09-30T22:16:16.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2833d42700 1 -- 172.21.15.89:0/1555452649 shutdown_connections 2023-09-30T22:16:16.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2833d42700 1 --2- 172.21.15.89:0/1555452649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09e930 0x7f282c13b740 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2833d42700 1 --2- 172.21.15.89:0/1555452649 >> v2:172.21.15.89:3300/0 conn(0x7f282c1338a0 0x7f282c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2833d42700 1 -- 172.21.15.89:0/1555452649 >> 172.21.15.89:0/1555452649 conn(0x7f282c0a2aa0 msgr2=0x7f282c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.209+0000 7f2833d42700 1 -- 172.21.15.89:0/1555452649 shutdown_connections 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.210+0000 7f2833d42700 1 -- 172.21.15.89:0/1555452649 wait complete. 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.211+0000 7f2833d42700 1 Processor -- start 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.211+0000 7f2833d42700 1 -- start start 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.211+0000 7f2833d42700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09de50 0x7f282c1ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.211+0000 7f2833d42700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f282c09e930 0x7f282c1cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.211+0000 7f2833d42700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f282c1338a0 0x7f282c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.211+0000 7f2833d42700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c09bed0 con 0x7f282c09e930 2023-09-30T22:16:16.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.211+0000 7f2833d42700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c09b600 con 0x7f282c09de50 2023-09-30T22:16:16.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.211+0000 7f2833d42700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f282c06f640 con 0x7f282c1338a0 2023-09-30T22:16:16.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f2831ade700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09de50 0x7f282c1ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f2831ade700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09de50 0x7f282c1ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42358/0 (socket says 172.21.15.89:42358) 2023-09-30T22:16:16.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f2831ade700 1 -- 172.21.15.89:0/2638691158 learned_addr learned my addr 172.21.15.89:0/2638691158 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:16.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28312dd700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f282c09e930 0x7f282c1cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28312dd700 1 -- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f282c1338a0 msgr2=0x7f282c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28312dd700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f282c1338a0 0x7f282c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28312dd700 1 -- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09de50 msgr2=0x7f282c1ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28312dd700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09de50 0x7f282c1ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28312dd700 1 -- 172.21.15.89:0/2638691158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280094a0 con 0x7f282c09e930 2023-09-30T22:16:16.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f2831ade700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09de50 0x7f282c1ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:16.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28312dd700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f282c09e930 0x7f282c1cd050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f281c00c3f0 tx=0x7f281c00c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:16.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28227fc700 1 -- 172.21.15.89:0/2638691158 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c005d70 con 0x7f282c09e930 2023-09-30T22:16:16.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28227fc700 1 -- 172.21.15.89:0/2638691158 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c00ce70 con 0x7f282c09e930 2023-09-30T22:16:16.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.212+0000 7f28227fc700 1 -- 172.21.15.89:0/2638691158 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c0072d0 con 0x7f282c09e930 2023-09-30T22:16:16.218 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:16.213+0000 7fa1847b0240 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T22:16:16.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.213+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2824009a70 con 0x7f282c09e930 2023-09-30T22:16:16.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.213+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c09d230 con 0x7f282c09e930 2023-09-30T22:16:16.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.214+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282c074ea0 con 0x7f282c09e930 2023-09-30T22:16:16.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.214+0000 7f28227fc700 1 -- 172.21.15.89:0/2638691158 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f281c007430 con 0x7f282c09e930 2023-09-30T22:16:16.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.214+0000 7f28227fc700 1 -- 172.21.15.89:0/2638691158 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f281c03aaf0 con 0x7f282c09e930 2023-09-30T22:16:16.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.216+0000 7f2831ade700 1 -- 172.21.15.89:0/2638691158 >> v1:172.21.15.89:6816/118161 conn(0x7f281002c1b0 legacy=0x7f281002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:16.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.220+0000 7fc03d429700 1 Processor -- start 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.220+0000 7fc03d429700 1 -- start start 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc03d429700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038134760 0x7fc038136b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc03d429700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc038137080 0x7fc038139460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc03d429700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc038096590 con 0x7fc038097fa0 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc03d429700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc038096bc0 con 0x7fc038137080 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc03d429700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc038097b50 con 0x7fc038134760 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc0377fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc038137080 0x7fc038139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc036ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc038097fa0 0x7fc038098370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35318/0 (socket says 172.21.15.89:35318) 2023-09-30T22:16:16.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc0377fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc038137080 0x7fc038139460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56630/0 (socket says 172.21.15.89:56630) 2023-09-30T22:16:16.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.221+0000 7fc036ffd700 1 -- 172.21.15.89:0/2995060433 learned_addr learned my addr 172.21.15.89:0/2995060433 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:16.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7fc0377fe700 1 -- 172.21.15.89:0/2995060433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038134760 msgr2=0x7fc038136b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:16.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7fc0377fe700 1 --2- 172.21.15.89:0/2995060433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038134760 0x7fc038136b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7fc0377fe700 1 -- 172.21.15.89:0/2995060433 >> v1:172.21.15.89:6789/0 conn(0x7fc038097fa0 legacy=0x7fc038098370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7fc0377fe700 1 -- 172.21.15.89:0/2995060433 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380968e0 con 0x7fc038137080 2023-09-30T22:16:16.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7fc0377fe700 1 --2- 172.21.15.89:0/2995060433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038137080 0x7fc038139460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc030009d90 tx=0x7fc03000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d69aa9b9317d58 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:16.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7fc035ffb700 1 -- 172.21.15.89:0/2995060433 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03000e070 con 0x7fc038137080 2023-09-30T22:16:16.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7f28227fc700 1 -- 172.21.15.89:0/2638691158 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f281c01b3b0 con 0x7f282c09e930 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7fc035ffb700 1 -- 172.21.15.89:0/2995060433 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03000bbd0 con 0x7fc038137080 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.222+0000 7fc035ffb700 1 -- 172.21.15.89:0/2995060433 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030012430 con 0x7fc038137080 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.223+0000 7fc03d429700 1 -- 172.21.15.89:0/2995060433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038137080 msgr2=0x7fc038139460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.223+0000 7fc03d429700 1 --2- 172.21.15.89:0/2995060433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038137080 0x7fc038139460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc030009d90 tx=0x7fc03000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.223+0000 7fc03d429700 1 -- 172.21.15.89:0/2995060433 shutdown_connections 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.223+0000 7fc03d429700 1 --2- 172.21.15.89:0/2995060433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038137080 0x7fc038139460 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.223+0000 7fc03d429700 1 --2- 172.21.15.89:0/2995060433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038134760 0x7fc038136b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.223+0000 7fc03d429700 1 -- 172.21.15.89:0/2995060433 >> 172.21.15.89:0/2995060433 conn(0x7fc0380a2360 msgr2=0x7fc0380a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:16.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.223+0000 7fc03d429700 1 -- 172.21.15.89:0/2995060433 shutdown_connections 2023-09-30T22:16:16.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.223+0000 7fc03d429700 1 -- 172.21.15.89:0/2995060433 wait complete. 2023-09-30T22:16:16.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.225+0000 7fc03d429700 1 Processor -- start 2023-09-30T22:16:16.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.225+0000 7fc03d429700 1 -- start start 2023-09-30T22:16:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc03d429700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038097fa0 0x7fc0381c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc03d429700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038134760 0x7fc0381c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc03d429700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038137080 0x7fc0381c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc03d429700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc038136e50 con 0x7fc038134760 2023-09-30T22:16:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc03d429700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc038098b40 con 0x7fc038137080 2023-09-30T22:16:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc03d429700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380719e0 con 0x7fc038097fa0 2023-09-30T22:16:16.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc036ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038097fa0 0x7fc0381c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc0377fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038137080 0x7fc0381c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc036ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038097fa0 0x7fc0381c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52686/0 (socket says 172.21.15.89:52686) 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc036ffd700 1 -- 172.21.15.89:0/3523669341 learned_addr learned my addr 172.21.15.89:0/3523669341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc0377fe700 1 -- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038097fa0 msgr2=0x7fc0381c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038097fa0 0x7fc0381c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc0377fe700 1 -- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038134760 msgr2=0x7fc0381c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038134760 0x7fc0381c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc0377fe700 1 -- 172.21.15.89:0/3523669341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030009a70 con 0x7fc038137080 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.226+0000 7fc036ffd700 1 --2- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038097fa0 0x7fc0381c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.227+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038137080 0x7fc0381c6df0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc030016000 tx=0x7fc030004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:16.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.227+0000 7fc01f7fe700 1 -- 172.21.15.89:0/3523669341 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030004810 con 0x7fc038137080 2023-09-30T22:16:16.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.227+0000 7fc01f7fe700 1 -- 172.21.15.89:0/3523669341 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc030020070 con 0x7fc038137080 2023-09-30T22:16:16.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.227+0000 7fc01f7fe700 1 -- 172.21.15.89:0/3523669341 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300276c0 con 0x7fc038137080 2023-09-30T22:16:16.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.227+0000 7fc03d429700 1 -- 172.21.15.89:0/3523669341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0381286b0 con 0x7fc038137080 2023-09-30T22:16:16.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.227+0000 7fc03d429700 1 -- 172.21.15.89:0/3523669341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc038070220 con 0x7fc038137080 2023-09-30T22:16:16.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.228+0000 7fc01f7fe700 1 -- 172.21.15.89:0/3523669341 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fc030027820 con 0x7fc038137080 2023-09-30T22:16:16.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.228+0000 7fc01d7fa700 1 -- 172.21.15.89:0/3523669341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc038074ea0 con 0x7fc038137080 2023-09-30T22:16:16.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.228+0000 7fc036ffd700 1 -- 172.21.15.89:0/3523669341 >> v1:172.21.15.89:6816/118161 conn(0x7fc02402c1b0 legacy=0x7fc02402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:16.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.228+0000 7fc01f7fe700 1 -- 172.21.15.89:0/3523669341 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fc030022070 con 0x7fc038137080 2023-09-30T22:16:16.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.236+0000 7fc01f7fe700 1 -- 172.21.15.89:0/3523669341 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fc03001a380 con 0x7fc038137080 2023-09-30T22:16:16.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.262+0000 7fabddb36700 1 -- 172.21.15.89:0/774135179 >> v1:172.21.15.89:6816/118161 conn(0x7fabbc02c1b0 legacy=0x7fabbc02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:16.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.295+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe87c06c360 con 0x7fe87c09e410 2023-09-30T22:16:16.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.296+0000 7fe863fff700 1 -- 172.21.15.89:0/3583380765 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe874019e40 con 0x7fe87c09e410 2023-09-30T22:16:16.297 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:16.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.298+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 >> v1:172.21.15.89:6816/118161 conn(0x7fe86802c1b0 legacy=0x7fe86802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:16.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.298+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09e410 msgr2=0x7fe87c11a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.298+0000 7fe881c1f700 1 --2- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09e410 0x7fe87c11a1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c7b0 tx=0x7fe87400cac0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.298+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 shutdown_connections 2023-09-30T22:16:16.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.298+0000 7fe881c1f700 1 --2- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe87c09e410 0x7fe87c11a1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.298+0000 7fe881c1f700 1 --2- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe87c09d6f0 0x7fe87c119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.298+0000 7fe881c1f700 1 --2- 172.21.15.89:0/3583380765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c09e000 0x7fe87c119c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.298+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 >> 172.21.15.89:0/3583380765 conn(0x7fe87c066f50 msgr2=0x7fe87c1bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:16.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.299+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 shutdown_connections 2023-09-30T22:16:16.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.299+0000 7fe881c1f700 1 -- 172.21.15.89:0/3583380765 wait complete. 2023-09-30T22:16:16.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.310+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fabc0005150 con 0x7fabd81341d0 2023-09-30T22:16:16.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.311+0000 7fabce7fc700 1 -- 172.21.15.89:0/774135179 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fabd0019030 con 0x7fabd81341d0 2023-09-30T22:16:16.312 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:16.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.2 2023-09-30T22:16:16.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 >> v1:172.21.15.89:6816/118161 conn(0x7fabbc02c1b0 legacy=0x7fabbc02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81341d0 msgr2=0x7fabd81203b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 --2- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81341d0 0x7fabd81203b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fabd000b9a0 tx=0x7fabd000bcb0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 shutdown_connections 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 --2- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabd81341d0 0x7fabd81203b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 --2- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabd809e560 0x7fabd811f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 --2- 172.21.15.89:0/774135179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabd81338c0 0x7fabd811fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 >> 172.21.15.89:0/774135179 conn(0x7fabd80a2aa0 msgr2=0x7fabd809f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 shutdown_connections 2023-09-30T22:16:16.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.313+0000 7fabdfd9a700 1 -- 172.21.15.89:0/774135179 wait complete. 2023-09-30T22:16:16.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:16:16.365 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:16.364+0000 7fa1847b0240 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T22:16:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.417+0000 7f2831ade700 1 -- 172.21.15.89:0/2638691158 >> v1:172.21.15.89:6816/118161 conn(0x7f281002c1b0 legacy=0x7f281002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.430+0000 7fc036ffd700 1 -- 172.21.15.89:0/3523669341 >> v1:172.21.15.89:6816/118161 conn(0x7fc02402c1b0 legacy=0x7fc02402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:16.437 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:16.436+0000 7fa1847b0240 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T22:16:16.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.457+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f282c09d5d0 con 0x7f282c09e930 2023-09-30T22:16:16.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.457+0000 7f28227fc700 1 -- 172.21.15.89:0/2638691158 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f281c03ae60 con 0x7f282c09e930 2023-09-30T22:16:16.458 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:16.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 >> v1:172.21.15.89:6816/118161 conn(0x7f281002c1b0 legacy=0x7f281002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:16.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f282c09e930 msgr2=0x7f282c1cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f282c09e930 0x7f282c1cd050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f281c00c3f0 tx=0x7f281c00c700 comp rx=0 tx=0).stop 2023-09-30T22:16:16.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 shutdown_connections 2023-09-30T22:16:16.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f282c1338a0 0x7f282c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f282c09de50 0x7f282c1ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 --2- 172.21.15.89:0/2638691158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f282c09e930 0x7f282c1cd050 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 >> 172.21.15.89:0/2638691158 conn(0x7f282c0a2aa0 msgr2=0x7f282c130710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 shutdown_connections 2023-09-30T22:16:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.460+0000 7f2833d42700 1 -- 172.21.15.89:0/2638691158 wait complete. 2023-09-30T22:16:16.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.0 2023-09-30T22:16:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.480+0000 7fc01d7fa700 1 -- 172.21.15.89:0/3523669341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc038072c10 con 0x7fc038137080 2023-09-30T22:16:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.481+0000 7fc01f7fe700 1 -- 172.21.15.89:0/3523669341 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc03002c070 con 0x7fc038137080 2023-09-30T22:16:16.481 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.482+0000 7fc01d7fa700 1 -- 172.21.15.89:0/3523669341 >> v1:172.21.15.89:6816/118161 conn(0x7fc02402c1b0 legacy=0x7fc02402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.482+0000 7fc01d7fa700 1 -- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038137080 msgr2=0x7fc0381c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.482+0000 7fc01d7fa700 1 --2- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038137080 0x7fc0381c6df0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc030016000 tx=0x7fc030004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.483+0000 7fc01d7fa700 1 -- 172.21.15.89:0/3523669341 shutdown_connections 2023-09-30T22:16:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.483+0000 7fc01d7fa700 1 --2- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc038137080 0x7fc0381c6df0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.483+0000 7fc01d7fa700 1 --2- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038097fa0 0x7fc0381c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.483+0000 7fc01d7fa700 1 --2- 172.21.15.89:0/3523669341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038134760 0x7fc0381c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.483+0000 7fc01d7fa700 1 -- 172.21.15.89:0/3523669341 >> 172.21.15.89:0/3523669341 conn(0x7fc0380a2360 msgr2=0x7fc038134b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.483+0000 7fc01d7fa700 1 -- 172.21.15.89:0/3523669341 shutdown_connections 2023-09-30T22:16:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:16.483+0000 7fc01d7fa700 1 -- 172.21.15.89:0/3523669341 wait complete. 2023-09-30T22:16:16.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:16:16.600 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:16.599+0000 7fa1847b0240 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T22:16:16.677 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:16.677+0000 7fa1847b0240 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T22:16:16.754 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:16.754+0000 7fa1847b0240 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T22:16:16.854 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:16.853+0000 7fa1847b0240 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T22:16:17.116 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:17.116+0000 7fa1847b0240 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T22:16:17.198 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:17.197+0000 7fa1847b0240 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T22:16:17.313 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:17.362 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:17.363 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:17.351+0000 7fa1847b0240 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T22:16:17.428 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:17.428+0000 7fa1847b0240 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T22:16:17.479 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:17.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f6d5cb22700 1 Processor -- start 2023-09-30T22:16:17.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f6d5cb22700 1 -- start start 2023-09-30T22:16:17.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f6d5cb22700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809d6f0 0x7f6d5809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f14ac94d700 1 Processor -- start 2023-09-30T22:16:17.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f6d5cb22700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d5809e000 0x7f6d58130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f6d5cb22700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5809f130 con 0x7f6d5809e410 2023-09-30T22:16:17.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f6d5cb22700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5809f520 con 0x7f6d5809e000 2023-09-30T22:16:17.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f6d5cb22700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5809e930 con 0x7f6d5809d6f0 2023-09-30T22:16:17.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809d6f0 0x7f6d5809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809d6f0 0x7f6d5809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42386/0 (socket says 172.21.15.89:42386) 2023-09-30T22:16:17.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5659c700 1 -- 172.21.15.89:0/1217759092 learned_addr learned my addr 172.21.15.89:0/1217759092 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:17.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d55d9b700 1 --2- 172.21.15.89:0/1217759092 >> v2:172.21.15.89:3300/0 conn(0x7f6d5809e000 0x7f6d58130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.490+0000 7f14ac94d700 1 -- start start 2023-09-30T22:16:17.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5559a700 1 -- 172.21.15.89:0/1217759092 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484657018 0 0) 0x7f6d5809f130 con 0x7f6d5809e410 2023-09-30T22:16:17.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5559a700 1 -- 172.21.15.89:0/1217759092 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d400035b0 con 0x7f6d5809e410 2023-09-30T22:16:17.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5659c700 1 -- 172.21.15.89:0/1217759092 >> v2:172.21.15.89:3300/0 conn(0x7f6d5809e000 msgr2=0x7f6d58130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:17.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5659c700 1 --2- 172.21.15.89:0/1217759092 >> v2:172.21.15.89:3300/0 conn(0x7f6d5809e000 0x7f6d58130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5659c700 1 -- 172.21.15.89:0/1217759092 >> v1:172.21.15.89:6789/0 conn(0x7f6d5809e410 legacy=0x7f6d58130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5659c700 1 -- 172.21.15.89:0/1217759092 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5809b2d0 con 0x7f6d5809d6f0 2023-09-30T22:16:17.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f14ac94d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a806e570 0x7f14a806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f14ac94d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a809eca0 0x7f14a806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f14ac94d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14a8134610 0x7f14a8134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f14ac94d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a8070ee0 con 0x7f14a8134610 2023-09-30T22:16:17.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f14ac94d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8071480 con 0x7f14a806e570 2023-09-30T22:16:17.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f14ac94d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14a806f3c0 con 0x7f14a809eca0 2023-09-30T22:16:17.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f6d5659c700 1 --2- 172.21.15.89:0/1217759092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809d6f0 0x7f6d5809dac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800bb90 tx=0x7f6d4800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b787ee431c39f1d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:17.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.491+0000 7f14a6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14a8134610 0x7f14a8134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a806e570 0x7f14a806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14a8134610 0x7f14a8134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56648/0 (socket says 172.21.15.89:56648) 2023-09-30T22:16:17.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a6d9d700 1 -- 172.21.15.89:0/2568407629 learned_addr learned my addr 172.21.15.89:0/2568407629 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:17.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f6d5559a700 1 -- 172.21.15.89:0/1217759092 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800dd30 con 0x7f6d5809d6f0 2023-09-30T22:16:17.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a5d9b700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a809eca0 0x7f14a806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f6d5559a700 1 -- 172.21.15.89:0/1217759092 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4800de90 con 0x7f6d5809d6f0 2023-09-30T22:16:17.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f6d5559a700 1 -- 172.21.15.89:0/1217759092 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48010690 con 0x7f6d5809d6f0 2023-09-30T22:16:17.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:17.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a6d9d700 1 -- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a809eca0 msgr2=0x7f14a806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:17.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a6d9d700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a809eca0 0x7f14a806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a6d9d700 1 -- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a806e570 msgr2=0x7f14a806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:17.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a6d9d700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a806e570 0x7f14a806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a6d9d700 1 -- 172.21.15.89:0/2568407629 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a8071b90 con 0x7f14a8134610 2023-09-30T22:16:17.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a659c700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a806e570 0x7f14a806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:17.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a6d9d700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8134610 0x7f14a8134ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009e50 tx=0x7f14a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ad4c257010e6f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:17.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a559a700 1 -- 172.21.15.89:0/2568407629 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000e070 con 0x7f14a8134610 2023-09-30T22:16:17.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a559a700 1 -- 172.21.15.89:0/2568407629 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a000b910 con 0x7f14a8134610 2023-09-30T22:16:17.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.492+0000 7f14a559a700 1 -- 172.21.15.89:0/2568407629 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000bcc0 con 0x7f14a8134610 2023-09-30T22:16:17.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1217759092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809d6f0 msgr2=0x7f6d5809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f6d5cb22700 1 --2- 172.21.15.89:0/1217759092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809d6f0 0x7f6d5809dac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800bb90 tx=0x7f6d4800bea0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1217759092 shutdown_connections 2023-09-30T22:16:17.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f6d5cb22700 1 --2- 172.21.15.89:0/1217759092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809d6f0 0x7f6d5809dac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f6d5cb22700 1 --2- 172.21.15.89:0/1217759092 >> v2:172.21.15.89:3300/0 conn(0x7f6d5809e000 0x7f6d58130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1217759092 >> 172.21.15.89:0/1217759092 conn(0x7f6d58066f50 msgr2=0x7f6d5809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:17.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1217759092 shutdown_connections 2023-09-30T22:16:17.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1217759092 wait complete. 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 -- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8134610 msgr2=0x7f14a8134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8134610 0x7f14a8134ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009e50 tx=0x7f14a000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 -- 172.21.15.89:0/2568407629 shutdown_connections 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8134610 0x7f14a8134ab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a806e570 0x7f14a806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 --2- 172.21.15.89:0/2568407629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a809eca0 0x7f14a806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 -- 172.21.15.89:0/2568407629 >> 172.21.15.89:0/2568407629 conn(0x7f14a80a2aa0 msgr2=0x7f14a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 -- 172.21.15.89:0/2568407629 shutdown_connections 2023-09-30T22:16:17.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.493+0000 7f14ac94d700 1 -- 172.21.15.89:0/2568407629 wait complete. 2023-09-30T22:16:17.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.494+0000 7f6d5cb22700 1 Processor -- start 2023-09-30T22:16:17.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.494+0000 7f14ac94d700 1 Processor -- start 2023-09-30T22:16:17.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.494+0000 7f6d5cb22700 1 -- start start 2023-09-30T22:16:17.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.494+0000 7f6d5cb22700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d5809d6f0 0x7f6d5811e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d5cb22700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809e000 0x7f6d5811e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d5cb22700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5809e410 0x7f6d5811eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d5cb22700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58120da0 con 0x7f6d5809e410 2023-09-30T22:16:17.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d5cb22700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58120170 con 0x7f6d5809e000 2023-09-30T22:16:17.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d5cb22700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58121820 con 0x7f6d5809d6f0 2023-09-30T22:16:17.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14ac94d700 1 -- start start 2023-09-30T22:16:17.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d56d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5809e410 0x7f6d5811eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d56d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5809e410 0x7f6d5811eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56660/0 (socket says 172.21.15.89:56660) 2023-09-30T22:16:17.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d56d9d700 1 -- 172.21.15.89:0/1669354700 learned_addr learned my addr 172.21.15.89:0/1669354700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d55d9b700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809e000 0x7f6d5811e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d55d9b700 1 -- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d5809d6f0 msgr2=0x7f6d5811e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d55d9b700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d5809d6f0 0x7f6d5811e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d55d9b700 1 -- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5809e410 msgr2=0x7f6d5811eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d55d9b700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5809e410 0x7f6d5811eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d55d9b700 1 -- 172.21.15.89:0/1669354700 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4800b7f0 con 0x7f6d5809e000 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d56d9d700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5809e410 0x7f6d5811eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f6d55d9b700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809e000 0x7f6d5811e570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e8d0 tx=0x7f6d4c00ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f6d46ffd700 1 -- 172.21.15.89:0/1669354700 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c013980 con 0x7f6d5809e000 2023-09-30T22:16:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f6d46ffd700 1 -- 172.21.15.89:0/1669354700 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c013ae0 con 0x7f6d5809e000 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f6d46ffd700 1 -- 172.21.15.89:0/1669354700 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c013e50 con 0x7f6d5809e000 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5809c700 con 0x7f6d5809e000 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14ac94d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a806e570 0x7f14a81214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14ac94d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a8134610 0x7f14a8121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14ac94d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8121f60 0x7f14a81d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14ac94d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8122840 con 0x7f14a8121f60 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14ac94d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8122c10 con 0x7f14a8134610 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14ac94d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8124730 con 0x7f14a806e570 2023-09-30T22:16:17.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14a6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8121f60 0x7f14a81d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5809a120 con 0x7f6d5809e000 2023-09-30T22:16:17.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14a6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8121f60 0x7f14a81d2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56676/0 (socket says 172.21.15.89:56676) 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14a6d9d700 1 -- 172.21.15.89:0/1513216919 learned_addr learned my addr 172.21.15.89:0/1513216919 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14a6d9d700 1 -- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a806e570 msgr2=0x7f14a81214e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14a6d9d700 1 --2- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a806e570 0x7f14a81214e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.495+0000 7f14a6d9d700 1 -- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a8134610 msgr2=0x7f14a8121a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f14a6d9d700 1 --2- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a8134610 0x7f14a8121a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f14a6d9d700 1 -- 172.21.15.89:0/1513216919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980094a0 con 0x7f14a8121f60 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f14a6d9d700 1 --2- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8121f60 0x7f14a81d2e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14a8071020 tx=0x7f14a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f1496ffd700 1 -- 172.21.15.89:0/1513216919 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0004450 con 0x7f14a8121f60 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f1496ffd700 1 -- 172.21.15.89:0/1513216919 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a00045b0 con 0x7f14a8121f60 2023-09-30T22:16:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.496+0000 7f1496ffd700 1 -- 172.21.15.89:0/1513216919 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0004960 con 0x7f14a8121f60 2023-09-30T22:16:17.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.497+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d58074ea0 con 0x7f6d5809e000 2023-09-30T22:16:17.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.497+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a0009b30 con 0x7f14a8121f60 2023-09-30T22:16:17.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.497+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a8070610 con 0x7f14a8121f60 2023-09-30T22:16:17.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.497+0000 7f6d46ffd700 1 -- 172.21.15.89:0/1669354700 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f6d4c025440 con 0x7f6d5809e000 2023-09-30T22:16:17.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.498+0000 7f6d46ffd700 1 -- 172.21.15.89:0/1669354700 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f6d4c03fe40 con 0x7f6d5809e000 2023-09-30T22:16:17.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.498+0000 7f6d5659c700 1 -- 172.21.15.89:0/1669354700 >> v1:172.21.15.89:6816/118161 conn(0x7f6d3802c1b0 legacy=0x7f6d3802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:17.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.498+0000 7f1496ffd700 1 -- 172.21.15.89:0/1513216919 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f14a0036460 con 0x7f14a8121f60 2023-09-30T22:16:17.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.498+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a809b1e0 con 0x7f14a8121f60 2023-09-30T22:16:17.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.498+0000 7f1496ffd700 1 -- 172.21.15.89:0/1513216919 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f14a004e500 con 0x7f14a8121f60 2023-09-30T22:16:17.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.498+0000 7f14a659c700 1 -- 172.21.15.89:0/1513216919 >> v1:172.21.15.89:6816/118161 conn(0x7f148802c1b0 legacy=0x7f148802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:17.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.502+0000 7f6d46ffd700 1 -- 172.21.15.89:0/1669354700 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f6d4c01a990 con 0x7f6d5809e000 2023-09-30T22:16:17.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.503+0000 7f1496ffd700 1 -- 172.21.15.89:0/1513216919 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f14a0036a90 con 0x7f14a8121f60 2023-09-30T22:16:17.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.655+0000 7f22dbad8700 1 Processor -- start 2023-09-30T22:16:17.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.656+0000 7f22dbad8700 1 -- start start 2023-09-30T22:16:17.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.656+0000 7f22dbad8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22d409d6f0 0x7f22d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.656+0000 7f22dbad8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409e000 0x7f22d4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.656+0000 7f22dbad8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d409f130 con 0x7f22d409e410 2023-09-30T22:16:17.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.656+0000 7f22dbad8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d409f520 con 0x7f22d409d6f0 2023-09-30T22:16:17.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.656+0000 7f22dbad8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d409e930 con 0x7f22d409e000 2023-09-30T22:16:17.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.656+0000 7f22d9073700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409e000 0x7f22d4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.656+0000 7f22d9073700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409e000 0x7f22d4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42414/0 (socket says 172.21.15.89:42414) 2023-09-30T22:16:17.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.657+0000 7f22d9073700 1 -- 172.21.15.89:0/1722620578 learned_addr learned my addr 172.21.15.89:0/1722620578 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:17.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.657+0000 7f22d9874700 1 --2- 172.21.15.89:0/1722620578 >> v2:172.21.15.89:3300/0 conn(0x7f22d409d6f0 0x7f22d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.657+0000 7f22d9073700 1 -- 172.21.15.89:0/1722620578 >> v2:172.21.15.89:3300/0 conn(0x7f22d409d6f0 msgr2=0x7f22d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:17.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.657+0000 7f22d9073700 1 --2- 172.21.15.89:0/1722620578 >> v2:172.21.15.89:3300/0 conn(0x7f22d409d6f0 0x7f22d409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.657+0000 7f22d9073700 1 -- 172.21.15.89:0/1722620578 >> v1:172.21.15.89:6789/0 conn(0x7f22d409e410 legacy=0x7f22d4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.657+0000 7f22d9073700 1 -- 172.21.15.89:0/1722620578 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d409b2d0 con 0x7f22d409e000 2023-09-30T22:16:17.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.657+0000 7f22d9073700 1 --2- 172.21.15.89:0/1722620578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409e000 0x7f22d4130660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22c40098f0 tx=0x7f22c4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daaddc46adfd7253 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:17.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22d8872700 1 -- 172.21.15.89:0/1722620578 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c400b8a0 con 0x7f22d409e000 2023-09-30T22:16:17.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22d8872700 1 -- 172.21.15.89:0/1722620578 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c400ba00 con 0x7f22d409e000 2023-09-30T22:16:17.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22d8872700 1 -- 172.21.15.89:0/1722620578 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c400bd90 con 0x7f22d409e000 2023-09-30T22:16:17.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22dbad8700 1 -- 172.21.15.89:0/1722620578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409e000 msgr2=0x7f22d4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22dbad8700 1 --2- 172.21.15.89:0/1722620578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409e000 0x7f22d4130660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22c40098f0 tx=0x7f22c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:17.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22dbad8700 1 -- 172.21.15.89:0/1722620578 shutdown_connections 2023-09-30T22:16:17.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22dbad8700 1 --2- 172.21.15.89:0/1722620578 >> v2:172.21.15.89:3300/0 conn(0x7f22d409d6f0 0x7f22d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22dbad8700 1 --2- 172.21.15.89:0/1722620578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409e000 0x7f22d4130660 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22dbad8700 1 -- 172.21.15.89:0/1722620578 >> 172.21.15.89:0/1722620578 conn(0x7f22d4066f50 msgr2=0x7f22d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:17.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.658+0000 7f22dbad8700 1 -- 172.21.15.89:0/1722620578 shutdown_connections 2023-09-30T22:16:17.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.659+0000 7f22dbad8700 1 -- 172.21.15.89:0/1722620578 wait complete. 2023-09-30T22:16:17.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.660+0000 7f22dbad8700 1 Processor -- start 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.660+0000 7f22dbad8700 1 -- start start 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22dbad8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409d6f0 0x7f22d411f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22dbad8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d409e000 0x7f22d411f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22dbad8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d409e410 0x7f22d411fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22dbad8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4122ce0 con 0x7f22d409e410 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22dbad8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4120dd0 con 0x7f22d409d6f0 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22dbad8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406e8d0 con 0x7f22d409e000 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409d6f0 0x7f22d411f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409d6f0 0x7f22d411f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42420/0 (socket says 172.21.15.89:42420) 2023-09-30T22:16:17.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 -- 172.21.15.89:0/2322412859 learned_addr learned my addr 172.21.15.89:0/2322412859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:17.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22da075700 1 --2- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d409e410 0x7f22d411fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 -- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d409e000 msgr2=0x7f22d411f700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:17.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 --2- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d409e000 0x7f22d411f700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 -- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d409e410 msgr2=0x7f22d411fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 --2- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d409e410 0x7f22d411fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 -- 172.21.15.89:0/2322412859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c4009550 con 0x7f22d409d6f0 2023-09-30T22:16:17.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.661+0000 7f22d9874700 1 --2- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409d6f0 0x7f22d411f1c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22d0010700 tx=0x7f22d0010a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.662+0000 7f22ca7fc700 1 -- 172.21.15.89:0/2322412859 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0011920 con 0x7f22d409d6f0 2023-09-30T22:16:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.662+0000 7f22ca7fc700 1 -- 172.21.15.89:0/2322412859 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0011a80 con 0x7f22d409d6f0 2023-09-30T22:16:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.662+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d409c700 con 0x7f22d409d6f0 2023-09-30T22:16:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.662+0000 7f22ca7fc700 1 -- 172.21.15.89:0/2322412859 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0011e10 con 0x7f22d409d6f0 2023-09-30T22:16:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.662+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d4120590 con 0x7f22d409d6f0 2023-09-30T22:16:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.663+0000 7f22ca7fc700 1 -- 172.21.15.89:0/2322412859 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f22d0019430 con 0x7f22d409d6f0 2023-09-30T22:16:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.663+0000 7f22ca7fc700 1 -- 172.21.15.89:0/2322412859 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f22d003d940 con 0x7f22d409d6f0 2023-09-30T22:16:17.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.663+0000 7f22d9073700 1 -- 172.21.15.89:0/2322412859 >> v1:172.21.15.89:6816/118161 conn(0x7f22b802c1b0 legacy=0x7f22b802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:17.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.667+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d409dcd0 con 0x7f22d409d6f0 2023-09-30T22:16:17.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.669+0000 7f9d214e7700 1 Processor -- start 2023-09-30T22:16:17.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.669+0000 7f9d214e7700 1 -- start start 2023-09-30T22:16:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d214e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d1c1316c0 0x7f9d1c131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d214e7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d1c09d9f0 0x7f9d1c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d214e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c06acb0 0x7f9d1c06b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d214e7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c06b630 con 0x7f9d1c09d9f0 2023-09-30T22:16:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d214e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c09e130 con 0x7f9d1c06acb0 2023-09-30T22:16:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d214e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1312b0 con 0x7f9d1c1316c0 2023-09-30T22:16:17.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c06acb0 0x7f9d1c06b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c06acb0 0x7f9d1c06b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42432/0 (socket says 172.21.15.89:42432) 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 -- 172.21.15.89:0/2603073428 learned_addr learned my addr 172.21.15.89:0/2603073428 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1a7fc700 1 --2- 172.21.15.89:0/2603073428 >> v2:172.21.15.89:3300/0 conn(0x7f9d1c09d9f0 0x7f9d1c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 -- 172.21.15.89:0/2603073428 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d1c1316c0 msgr2=0x7f9d1c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 --2- 172.21.15.89:0/2603073428 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d1c1316c0 0x7f9d1c131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 -- 172.21.15.89:0/2603073428 >> v2:172.21.15.89:3300/0 conn(0x7f9d1c09d9f0 msgr2=0x7f9d1c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 --2- 172.21.15.89:0/2603073428 >> v2:172.21.15.89:3300/0 conn(0x7f9d1c09d9f0 0x7f9d1c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 -- 172.21.15.89:0/2603073428 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c09f6e0 con 0x7f9d1c06acb0 2023-09-30T22:16:17.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.670+0000 7f9d1b7fe700 1 --2- 172.21.15.89:0/2603073428 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c06acb0 0x7f9d1c06b0f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009e80 tx=0x7f9d1400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3108c468a90af54d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.675+0000 7f9d19ffb700 1 -- 172.21.15.89:0/2603073428 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400e070 con 0x7f9d1c06acb0 2023-09-30T22:16:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.675+0000 7f9d19ffb700 1 -- 172.21.15.89:0/2603073428 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400b8d0 con 0x7f9d1c06acb0 2023-09-30T22:16:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.675+0000 7f9d19ffb700 1 -- 172.21.15.89:0/2603073428 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400bc80 con 0x7f9d1c06acb0 2023-09-30T22:16:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.675+0000 7f22ca7fc700 1 -- 172.21.15.89:0/2322412859 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f22d00183e0 con 0x7f22d409d6f0 2023-09-30T22:16:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 -- 172.21.15.89:0/2603073428 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c06acb0 msgr2=0x7f9d1c06b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 --2- 172.21.15.89:0/2603073428 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c06acb0 0x7f9d1c06b0f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009e80 tx=0x7f9d1400b330 comp rx=0 tx=0).stop 2023-09-30T22:16:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 -- 172.21.15.89:0/2603073428 shutdown_connections 2023-09-30T22:16:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 --2- 172.21.15.89:0/2603073428 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c06acb0 0x7f9d1c06b0f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 --2- 172.21.15.89:0/2603073428 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d1c1316c0 0x7f9d1c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 --2- 172.21.15.89:0/2603073428 >> v2:172.21.15.89:3300/0 conn(0x7f9d1c09d9f0 0x7f9d1c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 -- 172.21.15.89:0/2603073428 >> 172.21.15.89:0/2603073428 conn(0x7f9d1c066f50 msgr2=0x7f9d1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 -- 172.21.15.89:0/2603073428 shutdown_connections 2023-09-30T22:16:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.676+0000 7f9d214e7700 1 -- 172.21.15.89:0/2603073428 wait complete. 2023-09-30T22:16:17.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.677+0000 7f9d214e7700 1 Processor -- start 2023-09-30T22:16:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.677+0000 7f9d214e7700 1 -- start start 2023-09-30T22:16:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d214e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c06acb0 0x7f9d1c1cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d214e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d1c09d9f0 0x7f9d1c1ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d214e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c1316c0 0x7f9d1c070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d214e7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c09b7a0 con 0x7f9d1c06acb0 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d214e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c09bcb0 con 0x7f9d1c1316c0 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d214e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1263b0 con 0x7f9d1c09d9f0 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c1316c0 0x7f9d1c070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c1316c0 0x7f9d1c070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42440/0 (socket says 172.21.15.89:42440) 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.678+0000 7f9d1b7fe700 1 -- 172.21.15.89:0/3758223051 learned_addr learned my addr 172.21.15.89:0/3758223051 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d1b7fe700 1 -- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d1c09d9f0 msgr2=0x7f9d1c1ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d1b7fe700 1 --2- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d1c09d9f0 0x7f9d1c1ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d1b7fe700 1 -- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c06acb0 msgr2=0x7f9d1c1cc730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d1b7fe700 1 --2- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c06acb0 0x7f9d1c1cc730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d1b7fe700 1 -- 172.21.15.89:0/3758223051 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14009b30 con 0x7f9d1c1316c0 2023-09-30T22:16:17.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d1b7fe700 1 --2- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c1316c0 0x7f9d1c070ae0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c09e270 tx=0x7f9d14004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d037fe700 1 -- 172.21.15.89:0/3758223051 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14004250 con 0x7f9d1c1316c0 2023-09-30T22:16:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c09b9d0 con 0x7f9d1c1316c0 2023-09-30T22:16:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.679+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c0727e0 con 0x7f9d1c1316c0 2023-09-30T22:16:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.680+0000 7f9d037fe700 1 -- 172.21.15.89:0/3758223051 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d140043b0 con 0x7f9d1c1316c0 2023-09-30T22:16:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.680+0000 7f9d037fe700 1 -- 172.21.15.89:0/3758223051 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14004740 con 0x7f9d1c1316c0 2023-09-30T22:16:17.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.681+0000 7f9d037fe700 1 -- 172.21.15.89:0/3758223051 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f9d1401c440 con 0x7f9d1c1316c0 2023-09-30T22:16:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.681+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc0052e0 con 0x7f9d1c1316c0 2023-09-30T22:16:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.681+0000 7f9d037fe700 1 -- 172.21.15.89:0/3758223051 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f9d1403cd90 con 0x7f9d1c1316c0 2023-09-30T22:16:17.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.681+0000 7f9d1affd700 1 -- 172.21.15.89:0/3758223051 >> v1:172.21.15.89:6816/118161 conn(0x7f9d0802c1b0 legacy=0x7f9d0802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:17.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.689+0000 7f9d037fe700 1 -- 172.21.15.89:0/3758223051 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f9d14011af0 con 0x7f9d1c1316c0 2023-09-30T22:16:17.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.699+0000 7f14a659c700 1 -- 172.21.15.89:0/1513216919 >> v1:172.21.15.89:6816/118161 conn(0x7f148802c1b0 legacy=0x7f148802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:17.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.699+0000 7f6d5659c700 1 -- 172.21.15.89:0/1669354700 >> v1:172.21.15.89:6816/118161 conn(0x7f6d3802c1b0 legacy=0x7f6d3802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:17.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.713+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f14a8125260 con 0x7f14a8121f60 2023-09-30T22:16:17.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.714+0000 7f1496ffd700 1 -- 172.21.15.89:0/1513216919 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14a002de40 con 0x7f14a8121f60 2023-09-30T22:16:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:17.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.716+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 >> v1:172.21.15.89:6816/118161 conn(0x7f148802c1b0 legacy=0x7f148802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:17.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.716+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8121f60 msgr2=0x7f14a81d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.716+0000 7f14ac94d700 1 --2- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8121f60 0x7f14a81d2e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14a8071020 tx=0x7f14a0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.717+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 shutdown_connections 2023-09-30T22:16:17.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.717+0000 7f14ac94d700 1 --2- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8121f60 0x7f14a81d2e00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.717+0000 7f14ac94d700 1 --2- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14a806e570 0x7f14a81214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.717+0000 7f14ac94d700 1 --2- 172.21.15.89:0/1513216919 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14a8134610 0x7f14a8121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.717+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 >> 172.21.15.89:0/1513216919 conn(0x7f14a80a2aa0 msgr2=0x7f14a80a0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:17.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.721+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 shutdown_connections 2023-09-30T22:16:17.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.721+0000 7f14ac94d700 1 -- 172.21.15.89:0/1513216919 wait complete. 2023-09-30T22:16:17.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.2 2023-09-30T22:16:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.790+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d580a20d0 con 0x7f6d5809e000 2023-09-30T22:16:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.791+0000 7f6d46ffd700 1 -- 172.21.15.89:0/1669354700 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6d4c020030 con 0x7f6d5809e000 2023-09-30T22:16:17.792 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:17.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 >> v1:172.21.15.89:6816/118161 conn(0x7f6d3802c1b0 legacy=0x7f6d3802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:17.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809e000 msgr2=0x7f6d5811e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809e000 0x7f6d5811e570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e8d0 tx=0x7f6d4c00ebe0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 shutdown_connections 2023-09-30T22:16:17.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5809e410 0x7f6d5811eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d5809d6f0 0x7f6d5811e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 --2- 172.21.15.89:0/1669354700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d5809e000 0x7f6d5811e570 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 >> 172.21.15.89:0/1669354700 conn(0x7f6d58066f50 msgr2=0x7f6d58070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:17.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 shutdown_connections 2023-09-30T22:16:17.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.794+0000 7f6d5cb22700 1 -- 172.21.15.89:0/1669354700 wait complete. 2023-09-30T22:16:17.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:16:17.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.864+0000 7f22d9073700 1 -- 172.21.15.89:0/2322412859 >> v1:172.21.15.89:6816/118161 conn(0x7f22b802c1b0 legacy=0x7f22b802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:17.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.870+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f22d4074ea0 con 0x7f22d409d6f0 2023-09-30T22:16:17.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.871+0000 7f22ca7fc700 1 -- 172.21.15.89:0/2322412859 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f22d0020310 con 0x7f22d409d6f0 2023-09-30T22:16:17.871 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:17.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 >> v1:172.21.15.89:6816/118161 conn(0x7f22b802c1b0 legacy=0x7f22b802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409d6f0 msgr2=0x7f22d411f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 --2- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409d6f0 0x7f22d411f1c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22d0010700 tx=0x7f22d0010a10 comp rx=0 tx=0).stop 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 shutdown_connections 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 --2- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d409e410 0x7f22d411fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 --2- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d409d6f0 0x7f22d411f1c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 --2- 172.21.15.89:0/2322412859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d409e000 0x7f22d411f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 >> 172.21.15.89:0/2322412859 conn(0x7f22d4066f50 msgr2=0x7f22d4070120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 shutdown_connections 2023-09-30T22:16:17.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.873+0000 7f22dbad8700 1 -- 172.21.15.89:0/2322412859 wait complete. 2023-09-30T22:16:17.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.882+0000 7f9d1affd700 1 -- 172.21.15.89:0/3758223051 >> v1:172.21.15.89:6816/118161 conn(0x7f9d0802c1b0 legacy=0x7f9d0802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:17.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:16:17.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.907+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9cfc005590 con 0x7f9d1c1316c0 2023-09-30T22:16:17.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.907+0000 7f9d037fe700 1 -- 172.21.15.89:0/3758223051 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9d14020020 con 0x7f9d1c1316c0 2023-09-30T22:16:17.908 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 >> v1:172.21.15.89:6816/118161 conn(0x7f9d0802c1b0 legacy=0x7f9d0802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c1316c0 msgr2=0x7f9d1c070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 --2- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c1316c0 0x7f9d1c070ae0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c09e270 tx=0x7f9d14004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 shutdown_connections 2023-09-30T22:16:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 --2- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d1c1316c0 0x7f9d1c070ae0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 --2- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c06acb0 0x7f9d1c1cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 --2- 172.21.15.89:0/3758223051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d1c09d9f0 0x7f9d1c1ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:17.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 >> 172.21.15.89:0/3758223051 conn(0x7f9d1c066f50 msgr2=0x7f9d1c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:17.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 shutdown_connections 2023-09-30T22:16:17.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:17.909+0000 7f9d214e7700 1 -- 172.21.15.89:0/3758223051 wait complete. 2023-09-30T22:16:17.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.0 2023-09-30T22:16:18.122 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:18.122+0000 7fa1847b0240 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T22:16:18.245 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:18.244+0000 7fa1847b0240 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T22:16:18.360 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:18.359+0000 7fa1847b0240 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:16:18.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:18.816 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.862+0000 7f4b40ea6700 1 Processor -- start 2023-09-30T22:16:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.862+0000 7f4b40ea6700 1 -- start start 2023-09-30T22:16:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b40ea6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09d6f0 0x7f4b3c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b40ea6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b3c09e000 0x7f4b3c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b40ea6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3c09f130 con 0x7f4b3c09e410 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b40ea6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3c09f520 con 0x7f4b3c09e000 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b40ea6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c09e930 con 0x7f4b3c09d6f0 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b3a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09d6f0 0x7f4b3c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b3a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09d6f0 0x7f4b3c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42442/0 (socket says 172.21.15.89:42442) 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b3a59c700 1 -- 172.21.15.89:0/3625174082 learned_addr learned my addr 172.21.15.89:0/3625174082 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.863+0000 7f4b39d9b700 1 --2- 172.21.15.89:0/3625174082 >> v2:172.21.15.89:3300/0 conn(0x7f4b3c09e000 0x7f4b3c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.864+0000 7f4b3a59c700 1 -- 172.21.15.89:0/3625174082 >> v2:172.21.15.89:3300/0 conn(0x7f4b3c09e000 msgr2=0x7f4b3c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.864+0000 7f4b3a59c700 1 --2- 172.21.15.89:0/3625174082 >> v2:172.21.15.89:3300/0 conn(0x7f4b3c09e000 0x7f4b3c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.864+0000 7f4b3a59c700 1 -- 172.21.15.89:0/3625174082 >> v1:172.21.15.89:6789/0 conn(0x7f4b3c09e410 legacy=0x7f4b3c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.864+0000 7f4b3a59c700 1 -- 172.21.15.89:0/3625174082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c09b2d0 con 0x7f4b3c09d6f0 2023-09-30T22:16:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.864+0000 7f4b3a59c700 1 --2- 172.21.15.89:0/3625174082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09d6f0 0x7f4b3c09dac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0098f0 tx=0x7f4b2c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d49ff087510556 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.864+0000 7f4b3959a700 1 -- 172.21.15.89:0/3625174082 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c011830 con 0x7f4b3c09d6f0 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.864+0000 7f4b3959a700 1 -- 172.21.15.89:0/3625174082 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c011990 con 0x7f4b3c09d6f0 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.864+0000 7f4b3959a700 1 -- 172.21.15.89:0/3625174082 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c011d20 con 0x7f4b3c09d6f0 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.865+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3625174082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09d6f0 msgr2=0x7f4b3c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.865+0000 7f4b40ea6700 1 --2- 172.21.15.89:0/3625174082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09d6f0 0x7f4b3c09dac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0098f0 tx=0x7f4b2c009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.865+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3625174082 shutdown_connections 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.865+0000 7f4b40ea6700 1 --2- 172.21.15.89:0/3625174082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09d6f0 0x7f4b3c09dac0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.865+0000 7f4b40ea6700 1 --2- 172.21.15.89:0/3625174082 >> v2:172.21.15.89:3300/0 conn(0x7f4b3c09e000 0x7f4b3c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.865+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3625174082 >> 172.21.15.89:0/3625174082 conn(0x7f4b3c066f50 msgr2=0x7f4b3c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.865+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3625174082 shutdown_connections 2023-09-30T22:16:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.865+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3625174082 wait complete. 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.866+0000 7f4b40ea6700 1 Processor -- start 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.866+0000 7f4b40ea6700 1 -- start start 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b40ea6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3c09d6f0 0x7f4b3c135a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b40ea6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b3c09e000 0x7f4b3c135fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b40ea6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09e410 0x7f4b3c136500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b40ea6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c130dd0 con 0x7f4b3c09d6f0 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b40ea6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c137d90 con 0x7f4b3c09e410 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b40ea6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c138070 con 0x7f4b3c09e000 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b39d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b3c09e000 0x7f4b3c135fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b39d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b3c09e000 0x7f4b3c135fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52720/0 (socket says 172.21.15.89:52720) 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b39d9b700 1 -- 172.21.15.89:0/3670182432 learned_addr learned my addr 172.21.15.89:0/3670182432 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b3a59c700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3c09d6f0 0x7f4b3c135a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b3ad9d700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09e410 0x7f4b3c136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b3ad9d700 1 -- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b3c09e000 msgr2=0x7f4b3c135fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b3ad9d700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b3c09e000 0x7f4b3c135fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b3ad9d700 1 -- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3c09d6f0 msgr2=0x7f4b3c135a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b3ad9d700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3c09d6f0 0x7f4b3c135a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.867+0000 7f4b3ad9d700 1 -- 172.21.15.89:0/3670182432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c009550 con 0x7f4b3c09e410 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.868+0000 7f4b3ad9d700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09e410 0x7f4b3c136500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c06dff0 tx=0x7f4b3400ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.868+0000 7f4b2affd700 1 -- 172.21.15.89:0/3670182432 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34003c00 con 0x7f4b3c09e410 2023-09-30T22:16:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.868+0000 7f4b2affd700 1 -- 172.21.15.89:0/3670182432 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b34003d60 con 0x7f4b3c09e410 2023-09-30T22:16:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.868+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c137440 con 0x7f4b3c09e410 2023-09-30T22:16:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.868+0000 7f4b2affd700 1 -- 172.21.15.89:0/3670182432 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3400a9c0 con 0x7f4b3c09e410 2023-09-30T22:16:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.868+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c137ad0 con 0x7f4b3c09e410 2023-09-30T22:16:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.869+0000 7f4b2affd700 1 -- 172.21.15.89:0/3670182432 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f4b3400ab20 con 0x7f4b3c09e410 2023-09-30T22:16:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.869+0000 7f4b2affd700 1 -- 172.21.15.89:0/3670182432 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f4b3403b6a0 con 0x7f4b3c09e410 2023-09-30T22:16:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.869+0000 7f4b3a59c700 1 -- 172.21.15.89:0/3670182432 >> v1:172.21.15.89:6816/118161 conn(0x7f4b1c02c1b0 legacy=0x7f4b1c02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.870+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c137ed0 con 0x7f4b3c09e410 2023-09-30T22:16:18.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.874+0000 7f4b2affd700 1 -- 172.21.15.89:0/3670182432 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f4b3c137ed0 con 0x7f4b3c09e410 2023-09-30T22:16:18.887 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:18.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:18.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.944+0000 7f94cf266700 1 Processor -- start 2023-09-30T22:16:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.944+0000 7f94cf266700 1 -- start start 2023-09-30T22:16:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.944+0000 7f94cf266700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94c80a09c0 0x7f94c80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.944+0000 7f94cf266700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c80a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.944+0000 7f94cf266700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c80a1810 con 0x7f94c8133920 2023-09-30T22:16:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.944+0000 7f94cf266700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c809ea10 con 0x7f94c80a09c0 2023-09-30T22:16:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.944+0000 7f94cf266700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94c809e210 con 0x7f94c809d5c0 2023-09-30T22:16:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.945+0000 7f94cc801700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c80a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.945+0000 7f94cc801700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c80a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52730/0 (socket says 172.21.15.89:52730) 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.945+0000 7f94cc801700 1 -- 172.21.15.89:0/590626227 learned_addr learned my addr 172.21.15.89:0/590626227 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.945+0000 7f94cc801700 1 -- 172.21.15.89:0/590626227 >> v2:172.21.15.89:3300/0 conn(0x7f94c80a09c0 msgr2=0x7f94c80a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.945+0000 7f94cc801700 1 --2- 172.21.15.89:0/590626227 >> v2:172.21.15.89:3300/0 conn(0x7f94c80a09c0 0x7f94c80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.945+0000 7f94cc801700 1 -- 172.21.15.89:0/590626227 >> v1:172.21.15.89:6789/0 conn(0x7f94c8133920 legacy=0x7f94c8133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.945+0000 7f94cc801700 1 -- 172.21.15.89:0/590626227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c80a26f0 con 0x7f94c809d5c0 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.945+0000 7f94cc801700 1 --2- 172.21.15.89:0/590626227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c80a12d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94b80098f0 tx=0x7f94b8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c61f8175648293f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.946+0000 7f94bffff700 1 -- 172.21.15.89:0/590626227 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800b8a0 con 0x7f94c809d5c0 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.946+0000 7f94bffff700 1 -- 172.21.15.89:0/590626227 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b800ba00 con 0x7f94c809d5c0 2023-09-30T22:16:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.946+0000 7f94bffff700 1 -- 172.21.15.89:0/590626227 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800bd90 con 0x7f94c809d5c0 2023-09-30T22:16:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.946+0000 7f94cf266700 1 -- 172.21.15.89:0/590626227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 msgr2=0x7f94c80a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.946+0000 7f94cf266700 1 --2- 172.21.15.89:0/590626227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c80a12d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94b80098f0 tx=0x7f94b8009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.947+0000 7f94cf266700 1 -- 172.21.15.89:0/590626227 shutdown_connections 2023-09-30T22:16:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.947+0000 7f94cf266700 1 --2- 172.21.15.89:0/590626227 >> v2:172.21.15.89:3300/0 conn(0x7f94c80a09c0 0x7f94c80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.947+0000 7f94cf266700 1 --2- 172.21.15.89:0/590626227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c80a12d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.947+0000 7f94cf266700 1 -- 172.21.15.89:0/590626227 >> 172.21.15.89:0/590626227 conn(0x7f94c80a2fe0 msgr2=0x7f94c80a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.947+0000 7f94cf266700 1 -- 172.21.15.89:0/590626227 shutdown_connections 2023-09-30T22:16:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.947+0000 7f94cf266700 1 -- 172.21.15.89:0/590626227 wait complete. 2023-09-30T22:16:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.948+0000 7f94cf266700 1 Processor -- start 2023-09-30T22:16:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.948+0000 7f94cf266700 1 -- start start 2023-09-30T22:16:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.948+0000 7f94cf266700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c81c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.948+0000 7f94cf266700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c80a09c0 0x7f94c81c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cc801700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c80a09c0 0x7f94c81c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cc801700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c80a09c0 0x7f94c81c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56740/0 (socket says 172.21.15.89:56740) 2023-09-30T22:16:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd002700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c81c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd002700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c81c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52744/0 (socket says 172.21.15.89:52744) 2023-09-30T22:16:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cf266700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c8133920 0x7f94c81c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cf266700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c811f450 con 0x7f94c80a09c0 2023-09-30T22:16:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cf266700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c81213f0 con 0x7f94c8133920 2023-09-30T22:16:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cf266700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94c809cec0 con 0x7f94c809d5c0 2023-09-30T22:16:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cc801700 1 -- 172.21.15.89:0/226257436 learned_addr learned my addr 172.21.15.89:0/226257436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd803700 1 --2- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c8133920 0x7f94c81c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd002700 1 -- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c8133920 msgr2=0x7f94c81c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd002700 1 --2- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c8133920 0x7f94c81c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd002700 1 -- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c80a09c0 msgr2=0x7f94c81c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd002700 1 --2- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c80a09c0 0x7f94c81c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd002700 1 -- 172.21.15.89:0/226257436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009550 con 0x7f94c809d5c0 2023-09-30T22:16:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.949+0000 7f94cd002700 1 --2- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c81c8660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f94c400b450 tx=0x7f94c400b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.950+0000 7f94bdffb700 1 -- 172.21.15.89:0/226257436 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4011830 con 0x7f94c809d5c0 2023-09-30T22:16:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.950+0000 7f94bdffb700 1 -- 172.21.15.89:0/226257436 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c4011990 con 0x7f94c809d5c0 2023-09-30T22:16:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.950+0000 7f94bdffb700 1 -- 172.21.15.89:0/226257436 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4011d40 con 0x7f94c809d5c0 2023-09-30T22:16:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.950+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c8120d40 con 0x7f94c809d5c0 2023-09-30T22:16:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.950+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c8121990 con 0x7f94c809d5c0 2023-09-30T22:16:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.951+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c80747f0 con 0x7f94c809d5c0 2023-09-30T22:16:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.951+0000 7f94bdffb700 1 -- 172.21.15.89:0/226257436 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f94c401a400 con 0x7f94c809d5c0 2023-09-30T22:16:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.952+0000 7f94bdffb700 1 -- 172.21.15.89:0/226257436 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f94c403bc60 con 0x7f94c809d5c0 2023-09-30T22:16:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.952+0000 7f94cc801700 1 -- 172.21.15.89:0/226257436 >> v1:172.21.15.89:6816/118161 conn(0x7f94ac02c1b0 legacy=0x7f94ac02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:18.955+0000 7f94bdffb700 1 -- 172.21.15.89:0/226257436 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f94c4009280 con 0x7f94c809d5c0 2023-09-30T22:16:18.960 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:18.959+0000 7fa1847b0240 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T22:16:19.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.024+0000 7f2c07a6b700 1 Processor -- start 2023-09-30T22:16:19.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.025+0000 7f2c07a6b700 1 -- start start 2023-09-30T22:16:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.025+0000 7f2c07a6b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2c0006c0e0 0x7f2c0009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.025+0000 7f2c07a6b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0013fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.025+0000 7f2c07a6b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c00134980 con 0x7f2c00134ec0 2023-09-30T22:16:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.025+0000 7f2c07a6b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0006c4b0 con 0x7f2c0006c0e0 2023-09-30T22:16:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.025+0000 7f2c07a6b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000a09c0 con 0x7f2c001380f0 2023-09-30T22:16:19.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c05807700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2c0006c0e0 0x7f2c0009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c05006700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2c00134ec0 0x7f2c001379e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35376/0 (socket says 172.21.15.89:35376) 2023-09-30T22:16:19.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c05006700 1 -- 172.21.15.89:0/1612683955 learned_addr learned my addr 172.21.15.89:0/1612683955 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:19.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c06008700 1 --2- 172.21.15.89:0/1612683955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0013fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c06008700 1 -- 172.21.15.89:0/1612683955 >> v2:172.21.15.89:3300/0 conn(0x7f2c0006c0e0 msgr2=0x7f2c0009ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:19.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c06008700 1 --2- 172.21.15.89:0/1612683955 >> v2:172.21.15.89:3300/0 conn(0x7f2c0006c0e0 0x7f2c0009ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c06008700 1 -- 172.21.15.89:0/1612683955 >> v1:172.21.15.89:6789/0 conn(0x7f2c00134ec0 legacy=0x7f2c001379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c06008700 1 -- 172.21.15.89:0/1612683955 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c000a0600 con 0x7f2c001380f0 2023-09-30T22:16:19.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c04805700 1 -- 172.21.15.89:0/1612683955 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022955579 0 0) 0x7f2c00134980 con 0x7f2c00134ec0 2023-09-30T22:16:19.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c05807700 1 --2- 172.21.15.89:0/1612683955 >> v2:172.21.15.89:3300/0 conn(0x7f2c0006c0e0 0x7f2c0009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:19.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c06008700 1 --2- 172.21.15.89:0/1612683955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0013fd20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009d90 tx=0x7f2bf800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45ccbfa67342e7d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:19.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c04805700 1 -- 172.21.15.89:0/1612683955 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf800e070 con 0x7f2c001380f0 2023-09-30T22:16:19.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.026+0000 7f2c04805700 1 -- 172.21.15.89:0/1612683955 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf800bb20 con 0x7f2c001380f0 2023-09-30T22:16:19.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c04805700 1 -- 172.21.15.89:0/1612683955 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8012430 con 0x7f2c001380f0 2023-09-30T22:16:19.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1612683955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 msgr2=0x7f2c0013fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c07a6b700 1 --2- 172.21.15.89:0/1612683955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0013fd20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009d90 tx=0x7f2bf800b330 comp rx=0 tx=0).stop 2023-09-30T22:16:19.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1612683955 shutdown_connections 2023-09-30T22:16:19.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c07a6b700 1 --2- 172.21.15.89:0/1612683955 >> v2:172.21.15.89:3300/0 conn(0x7f2c0006c0e0 0x7f2c0009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c07a6b700 1 --2- 172.21.15.89:0/1612683955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0013fd20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1612683955 >> 172.21.15.89:0/1612683955 conn(0x7f2c000a4210 msgr2=0x7f2c000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:19.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1612683955 shutdown_connections 2023-09-30T22:16:19.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.027+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1612683955 wait complete. 2023-09-30T22:16:19.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.029+0000 7f2c07a6b700 1 Processor -- start 2023-09-30T22:16:19.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.029+0000 7f2c07a6b700 1 -- start start 2023-09-30T22:16:19.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.029+0000 7f2c07a6b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c0006c0e0 0x7f2c0011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.029+0000 7f2c07a6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c00134ec0 0x7f2c0011abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.029+0000 7f2c07a6b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0011b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.029+0000 7f2c07a6b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0011d430 con 0x7f2c00134ec0 2023-09-30T22:16:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.029+0000 7f2c07a6b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000a2050 con 0x7f2c0006c0e0 2023-09-30T22:16:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.029+0000 7f2c07a6b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000a03f0 con 0x7f2c001380f0 2023-09-30T22:16:19.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.032+0000 7f2c06008700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0011b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.032+0000 7f2c06008700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0011b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52770/0 (socket says 172.21.15.89:52770) 2023-09-30T22:16:19.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.032+0000 7f2c06008700 1 -- 172.21.15.89:0/1799722207 learned_addr learned my addr 172.21.15.89:0/1799722207 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:19.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.032+0000 7f2c05006700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c00134ec0 0x7f2c0011abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2c05807700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c0006c0e0 0x7f2c0011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2c05006700 1 -- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 msgr2=0x7f2c0011b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2c05006700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0011b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2c05006700 1 -- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c0006c0e0 msgr2=0x7f2c0011a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2c05006700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c0006c0e0 0x7f2c0011a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2c05006700 1 -- 172.21.15.89:0/1799722207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0094a0 con 0x7f2c00134ec0 2023-09-30T22:16:19.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2c05006700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c00134ec0 0x7f2c0011abc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000c4b0 tx=0x7f2bf000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:19.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2bf67fc700 1 -- 172.21.15.89:0/1799722207 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0004c80 con 0x7f2c00134ec0 2023-09-30T22:16:19.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2bf67fc700 1 -- 172.21.15.89:0/1799722207 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf0004e00 con 0x7f2c00134ec0 2023-09-30T22:16:19.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2bf67fc700 1 -- 172.21.15.89:0/1799722207 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0007310 con 0x7f2c00134ec0 2023-09-30T22:16:19.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.033+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf8009a70 con 0x7f2c00134ec0 2023-09-30T22:16:19.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.034+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0011d670 con 0x7f2c00134ec0 2023-09-30T22:16:19.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.035+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00074ea0 con 0x7f2c00134ec0 2023-09-30T22:16:19.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.043+0000 7f2bf67fc700 1 -- 172.21.15.89:0/1799722207 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f2bf0007470 con 0x7f2c00134ec0 2023-09-30T22:16:19.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.043+0000 7f2c05807700 1 -- 172.21.15.89:0/1799722207 >> v1:172.21.15.89:6816/118161 conn(0x7f2be402c1b0 legacy=0x7f2be402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:19.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.043+0000 7f2bf67fc700 1 -- 172.21.15.89:0/1799722207 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f2bf003bc30 con 0x7f2c00134ec0 2023-09-30T22:16:19.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.043+0000 7f2bf67fc700 1 -- 172.21.15.89:0/1799722207 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f2bf001bd10 con 0x7f2c00134ec0 2023-09-30T22:16:19.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.070+0000 7f4b3a59c700 1 -- 172.21.15.89:0/3670182432 >> v1:172.21.15.89:6816/118161 conn(0x7f4b1c02c1b0 legacy=0x7f4b1c02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:19.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.087+0000 7fee1e306700 1 Processor -- start 2023-09-30T22:16:19.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.087+0000 7fee1e306700 1 -- start start 2023-09-30T22:16:19.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1e306700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1806f5d0 0x7fee1812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1e306700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18135c20 0x7fee18138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1e306700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1806fa40 con 0x7fee1812cf60 2023-09-30T22:16:19.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1e306700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1812cd90 con 0x7fee18135c20 2023-09-30T22:16:19.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1e306700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee18071780 con 0x7fee1806f5d0 2023-09-30T22:16:19.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1c8a3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18135c20 0x7fee18138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1c8a3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18135c20 0x7fee18138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42468/0 (socket says 172.21.15.89:42468) 2023-09-30T22:16:19.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1c8a3700 1 -- 172.21.15.89:0/454040586 learned_addr learned my addr 172.21.15.89:0/454040586 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:19.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee177fe700 1 --2- 172.21.15.89:0/454040586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1806f5d0 0x7fee1812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1c8a3700 1 -- 172.21.15.89:0/454040586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1806f5d0 msgr2=0x7fee1812ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1c8a3700 1 --2- 172.21.15.89:0/454040586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1806f5d0 0x7fee1812ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1c8a3700 1 -- 172.21.15.89:0/454040586 >> v1:172.21.15.89:6789/0 conn(0x7fee1812cf60 legacy=0x7fee1812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1c8a3700 1 -- 172.21.15.89:0/454040586 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18071a40 con 0x7fee18135c20 2023-09-30T22:16:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.088+0000 7fee1c8a3700 1 --2- 172.21.15.89:0/454040586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18135c20 0x7fee18138000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fee10009e80 tx=0x7fee1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1c8e40bb10ee3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.089+0000 7fee16ffd700 1 -- 172.21.15.89:0/454040586 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1000e070 con 0x7fee18135c20 2023-09-30T22:16:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.089+0000 7fee16ffd700 1 -- 172.21.15.89:0/454040586 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee10003a20 con 0x7fee18135c20 2023-09-30T22:16:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.089+0000 7fee16ffd700 1 -- 172.21.15.89:0/454040586 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10003dd0 con 0x7fee18135c20 2023-09-30T22:16:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.089+0000 7fee1e306700 1 -- 172.21.15.89:0/454040586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18135c20 msgr2=0x7fee18138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.089+0000 7fee1e306700 1 --2- 172.21.15.89:0/454040586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18135c20 0x7fee18138000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fee10009e80 tx=0x7fee1000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.090+0000 7fee1e306700 1 -- 172.21.15.89:0/454040586 shutdown_connections 2023-09-30T22:16:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.090+0000 7fee1e306700 1 --2- 172.21.15.89:0/454040586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18135c20 0x7fee18138000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.090+0000 7fee1e306700 1 --2- 172.21.15.89:0/454040586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1806f5d0 0x7fee1812ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.090+0000 7fee1e306700 1 -- 172.21.15.89:0/454040586 >> 172.21.15.89:0/454040586 conn(0x7fee180a2aa0 msgr2=0x7fee180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.090+0000 7fee1e306700 1 -- 172.21.15.89:0/454040586 shutdown_connections 2023-09-30T22:16:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.090+0000 7fee1e306700 1 -- 172.21.15.89:0/454040586 wait complete. 2023-09-30T22:16:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.091+0000 7fee1e306700 1 Processor -- start 2023-09-30T22:16:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.091+0000 7fee1e306700 1 -- start start 2023-09-30T22:16:19.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee1e306700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1806f5d0 0x7fee18099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee1e306700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1812cf60 0x7fee1809a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1812cf60 0x7fee1809a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1812cf60 0x7fee1809a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56766/0 (socket says 172.21.15.89:56766) 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee1e306700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee18135c20 0x7fee1809a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee1e306700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1809ebd0 con 0x7fee1812cf60 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee1e306700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1812c0a0 con 0x7fee1806f5d0 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee1e306700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee18129fa0 con 0x7fee18135c20 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee17fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1806f5d0 0x7fee18099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee177fe700 1 -- 172.21.15.89:0/3641439030 learned_addr learned my addr 172.21.15.89:0/3641439030 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.092+0000 7fee1c8a3700 1 --2- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee18135c20 0x7fee1809a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee177fe700 1 -- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee18135c20 msgr2=0x7fee1809a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee177fe700 1 --2- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee18135c20 0x7fee1809a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee177fe700 1 -- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1806f5d0 msgr2=0x7fee18099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee177fe700 1 --2- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1806f5d0 0x7fee18099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee177fe700 1 -- 172.21.15.89:0/3641439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee10009b30 con 0x7fee1812cf60 2023-09-30T22:16:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee177fe700 1 --2- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1812cf60 0x7fee1809a290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00dba0 tx=0x7fee0c00deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee14ff9700 1 -- 172.21.15.89:0/3641439030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c008930 con 0x7fee1812cf60 2023-09-30T22:16:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee14ff9700 1 -- 172.21.15.89:0/3641439030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c008a90 con 0x7fee1812cf60 2023-09-30T22:16:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.093+0000 7fee14ff9700 1 -- 172.21.15.89:0/3641439030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c008e00 con 0x7fee1812cf60 2023-09-30T22:16:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.094+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1809bf30 con 0x7fee1812cf60 2023-09-30T22:16:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.094+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee181348a0 con 0x7fee1812cf60 2023-09-30T22:16:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.095+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1809b6f0 con 0x7fee1812cf60 2023-09-30T22:16:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.096+0000 7fee14ff9700 1 -- 172.21.15.89:0/3641439030 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fee0c0173f0 con 0x7fee1812cf60 2023-09-30T22:16:19.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.096+0000 7fee14ff9700 1 -- 172.21.15.89:0/3641439030 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fee0c03bf60 con 0x7fee1812cf60 2023-09-30T22:16:19.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.102+0000 7fee17fff700 1 -- 172.21.15.89:0/3641439030 >> v1:172.21.15.89:6816/118161 conn(0x7fee0402c200 legacy=0x7fee0402e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:19.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.103+0000 7fee14ff9700 1 -- 172.21.15.89:0/3641439030 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fee0c062fe0 con 0x7fee1812cf60 2023-09-30T22:16:19.123 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.123+0000 7fa1847b0240 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:16:19.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.132+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4b3c137ed0 con 0x7f4b3c09e410 2023-09-30T22:16:19.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.133+0000 7f4b2affd700 1 -- 172.21.15.89:0/3670182432 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b3c137ed0 con 0x7f4b3c09e410 2023-09-30T22:16:19.134 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:19.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 >> v1:172.21.15.89:6816/118161 conn(0x7f4b1c02c1b0 legacy=0x7f4b1c02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09e410 msgr2=0x7f4b3c136500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09e410 0x7f4b3c136500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c06dff0 tx=0x7f4b3400ce70 comp rx=0 tx=0).stop 2023-09-30T22:16:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 shutdown_connections 2023-09-30T22:16:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b3c09e410 0x7f4b3c136500 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3c09d6f0 0x7f4b3c135a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 --2- 172.21.15.89:0/3670182432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b3c09e000 0x7f4b3c135fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 >> 172.21.15.89:0/3670182432 conn(0x7f4b3c066f50 msgr2=0x7f4b3c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 shutdown_connections 2023-09-30T22:16:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.135+0000 7f4b40ea6700 1 -- 172.21.15.89:0/3670182432 wait complete. 2023-09-30T22:16:19.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.2 2023-09-30T22:16:19.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.153+0000 7f94cc801700 1 -- 172.21.15.89:0/226257436 >> v1:172.21.15.89:6816/118161 conn(0x7f94ac02c1b0 legacy=0x7f94ac02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:19.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.180+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f94c8072980 con 0x7f94c809d5c0 2023-09-30T22:16:19.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.180+0000 7f94bdffb700 1 -- 172.21.15.89:0/226257436 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f94c4028020 con 0x7f94c809d5c0 2023-09-30T22:16:19.182 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:19.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 >> v1:172.21.15.89:6816/118161 conn(0x7f94ac02c1b0 legacy=0x7f94ac02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:19.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 msgr2=0x7f94c81c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 --2- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c81c8660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f94c400b450 tx=0x7f94c400b760 comp rx=0 tx=0).stop 2023-09-30T22:16:19.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 shutdown_connections 2023-09-30T22:16:19.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 --2- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94c8133920 0x7f94c81c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 --2- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94c809d5c0 0x7f94c81c8660 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 --2- 172.21.15.89:0/226257436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94c80a09c0 0x7f94c81c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 >> 172.21.15.89:0/226257436 conn(0x7f94c80a2fe0 msgr2=0x7f94c8133070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 shutdown_connections 2023-09-30T22:16:19.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.183+0000 7f94cf266700 1 -- 172.21.15.89:0/226257436 wait complete. 2023-09-30T22:16:19.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:16:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.232+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2c0006b250 con 0x7f2c00134ec0 2023-09-30T22:16:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.232+0000 7f2bf67fc700 1 -- 172.21.15.89:0/1799722207 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bf001bb30 con 0x7f2c00134ec0 2023-09-30T22:16:19.234 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:19.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 >> v1:172.21.15.89:6816/118161 conn(0x7f2be402c1b0 legacy=0x7f2be402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c00134ec0 msgr2=0x7f2c0011abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c00134ec0 0x7f2c0011abc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000c4b0 tx=0x7f2bf000c7c0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 shutdown_connections 2023-09-30T22:16:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c001380f0 0x7f2c0011b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c0006c0e0 0x7f2c0011a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 --2- 172.21.15.89:0/1799722207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c00134ec0 0x7f2c0011abc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 >> 172.21.15.89:0/1799722207 conn(0x7f2c000a4210 msgr2=0x7f2c00099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:19.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 shutdown_connections 2023-09-30T22:16:19.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.234+0000 7f2c07a6b700 1 -- 172.21.15.89:0/1799722207 wait complete. 2023-09-30T22:16:19.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:16:19.285 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.285+0000 7fa1847b0240 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T22:16:19.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.303+0000 7fee17fff700 1 -- 172.21.15.89:0/3641439030 >> v1:172.21.15.89:6816/118161 conn(0x7fee0402c200 legacy=0x7fee0402e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/118161 2023-09-30T22:16:19.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.325+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fee18074ea0 con 0x7fee1812cf60 2023-09-30T22:16:19.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.325+0000 7fee14ff9700 1 -- 172.21.15.89:0/3641439030 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fee0c0163a0 con 0x7fee1812cf60 2023-09-30T22:16:19.325 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:19.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 >> v1:172.21.15.89:6816/118161 conn(0x7fee0402c200 legacy=0x7fee0402e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:19.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1812cf60 msgr2=0x7fee1809a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 --2- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1812cf60 0x7fee1809a290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00dba0 tx=0x7fee0c00deb0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 shutdown_connections 2023-09-30T22:16:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 --2- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee18135c20 0x7fee1809a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 --2- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1806f5d0 0x7fee18099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 --2- 172.21.15.89:0/3641439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1812cf60 0x7fee1809a290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 >> 172.21.15.89:0/3641439030 conn(0x7fee180a2aa0 msgr2=0x7fee1806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:19.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 shutdown_connections 2023-09-30T22:16:19.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:19.327+0000 7fee1e306700 1 -- 172.21.15.89:0/3641439030 wait complete. 2023-09-30T22:16:19.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.0 2023-09-30T22:16:19.390 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.391+0000 7fa1847b0240 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T22:16:19.469 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa102e08700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.470 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa102e08700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.470 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa102e08700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.470 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa102e08700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.470 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa102e08700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.471 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa0fee00700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.471 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa0fee00700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.471 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa0fee00700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.471 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa0fee00700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:19.471 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:19.470+0000 7fa0fee00700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:16:20.149 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:20.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:20.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:20.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.285+0000 7f7b63627700 1 Processor -- start 2023-09-30T22:16:20.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.285+0000 7f7b63627700 1 -- start start 2023-09-30T22:16:20.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b63627700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c131810 0x7f7b5c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b63627700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b5c099a40 0x7f7b5c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b63627700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c09a3b0 0x7f7b5c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b63627700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5c1313c0 con 0x7f7b5c099a40 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b63627700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c06e8c0 con 0x7f7b5c131810 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b63627700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c09a820 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b60bc2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b5c099a40 0x7f7b5c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b60bc2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b5c099a40 0x7f7b5c099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60736/0 (socket says 172.21.15.89:60736) 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b60bc2700 1 -- 172.21.15.89:0/3455075252 learned_addr learned my addr 172.21.15.89:0/3455075252 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b613c3700 1 --2- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c131810 0x7f7b5c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b61bc4700 1 --2- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c09a3b0 0x7f7b5c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b613c3700 1 -- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c09a3b0 msgr2=0x7f7b5c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:20.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b613c3700 1 --2- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c09a3b0 0x7f7b5c138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b613c3700 1 -- 172.21.15.89:0/3455075252 >> v2:172.21.15.89:3300/0 conn(0x7f7b5c099a40 msgr2=0x7f7b5c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b613c3700 1 --2- 172.21.15.89:0/3455075252 >> v2:172.21.15.89:3300/0 conn(0x7f7b5c099a40 0x7f7b5c099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b613c3700 1 -- 172.21.15.89:0/3455075252 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c071d20 con 0x7f7b5c131810 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b61bc4700 1 --2- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c09a3b0 0x7f7b5c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.286+0000 7f7b60bc2700 1 --2- 172.21.15.89:0/3455075252 >> v2:172.21.15.89:3300/0 conn(0x7f7b5c099a40 0x7f7b5c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b613c3700 1 --2- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c131810 0x7f7b5c131be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009840 tx=0x7f7b58009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd208d7bd3d22908 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b53fff700 1 -- 172.21.15.89:0/3455075252 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011830 con 0x7f7b5c131810 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b53fff700 1 -- 172.21.15.89:0/3455075252 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b58011990 con 0x7f7b5c131810 2023-09-30T22:16:20.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b53fff700 1 -- 172.21.15.89:0/3455075252 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011d20 con 0x7f7b5c131810 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b63627700 1 -- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c131810 msgr2=0x7f7b5c131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b63627700 1 --2- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c131810 0x7f7b5c131be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009840 tx=0x7f7b58009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b63627700 1 -- 172.21.15.89:0/3455075252 shutdown_connections 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b63627700 1 --2- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c09a3b0 0x7f7b5c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b63627700 1 --2- 172.21.15.89:0/3455075252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c131810 0x7f7b5c131be0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b63627700 1 --2- 172.21.15.89:0/3455075252 >> v2:172.21.15.89:3300/0 conn(0x7f7b5c099a40 0x7f7b5c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.287+0000 7f7b63627700 1 -- 172.21.15.89:0/3455075252 >> 172.21.15.89:0/3455075252 conn(0x7f7b5c0a2aa0 msgr2=0x7f7b5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.288+0000 7f7b63627700 1 -- 172.21.15.89:0/3455075252 shutdown_connections 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.288+0000 7f7b63627700 1 -- 172.21.15.89:0/3455075252 wait complete. 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.288+0000 7f7b63627700 1 Processor -- start 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b63627700 1 -- start start 2023-09-30T22:16:20.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b63627700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c099a40 0x7f7b5c11b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b63627700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c09a3b0 0x7f7b5c11bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b63627700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b5c131810 0x7f7b5c11c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b63627700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c11d840 con 0x7f7b5c131810 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b63627700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c11d210 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b63627700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c06eb00 con 0x7f7b5c099a40 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b613c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c099a40 0x7f7b5c11b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b613c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c099a40 0x7f7b5c11b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45932/0 (socket says 172.21.15.89:45932) 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b613c3700 1 -- 172.21.15.89:0/3616119972 learned_addr learned my addr 172.21.15.89:0/3616119972 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b60bc2700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c09a3b0 0x7f7b5c11bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b61bc4700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b5c131810 0x7f7b5c11c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b60bc2700 1 -- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c099a40 msgr2=0x7f7b5c11b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b60bc2700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c099a40 0x7f7b5c11b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b60bc2700 1 -- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b5c131810 msgr2=0x7f7b5c11c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b60bc2700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b5c131810 0x7f7b5c11c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.289+0000 7f7b60bc2700 1 -- 172.21.15.89:0/3616119972 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c009550 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.290+0000 7f7b60bc2700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c09a3b0 0x7f7b5c11bbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00e9a0 tx=0x7f7b4c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.290+0000 7f7b51ffb700 1 -- 172.21.15.89:0/3616119972 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c004030 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.290+0000 7f7b51ffb700 1 -- 172.21.15.89:0/3616119972 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c004190 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.290+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b54009a70 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.290+0000 7f7b51ffb700 1 -- 172.21.15.89:0/3616119972 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c004520 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.290+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b580094a0 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.291+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5c11e4c0 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.292+0000 7f7b51ffb700 1 -- 172.21.15.89:0/3616119972 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7f7b4c0036a0 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.292+0000 7f7b51ffb700 1 -- 172.21.15.89:0/3616119972 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4026+0+0 (secure 0 0 0) 0x7f7b4c03fc40 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.296+0000 7f7b51ffb700 1 -- 172.21.15.89:0/3616119972 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f7b4c012c00 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.332+0000 7fc9e507c700 1 Processor -- start 2023-09-30T22:16:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.332+0000 7fc9e507c700 1 -- start start 2023-09-30T22:16:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9e507c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9e507c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9e00a1340 0x7fc9e0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9e507c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e009edd0 con 0x7fc9e0133fa0 2023-09-30T22:16:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9e507c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e009e1f0 con 0x7fc9e00a1340 2023-09-30T22:16:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9e507c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e009dc50 con 0x7fc9e00a0a30 2023-09-30T22:16:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9de59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9e00a1340 0x7fc9e0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9de59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9e00a1340 0x7fc9e0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60744/0 (socket says 172.21.15.89:60744) 2023-09-30T22:16:20.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9df59e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc9e0133fa0 0x7fc9e013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52840/0 (socket says 172.21.15.89:52840) 2023-09-30T22:16:20.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9de59c700 1 -- 172.21.15.89:0/2771662522 learned_addr learned my addr 172.21.15.89:0/2771662522 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:20.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9ded9d700 1 --2- 172.21.15.89:0/2771662522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9de59c700 1 -- 172.21.15.89:0/2771662522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 msgr2=0x7fc9e00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9de59c700 1 --2- 172.21.15.89:0/2771662522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9de59c700 1 -- 172.21.15.89:0/2771662522 >> v1:172.21.15.89:6789/0 conn(0x7fc9e0133fa0 legacy=0x7fc9e013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9ddd9b700 1 -- 172.21.15.89:0/2771662522 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221414912 0 0) 0x7fc9e009edd0 con 0x7fc9e0133fa0 2023-09-30T22:16:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.333+0000 7fc9de59c700 1 -- 172.21.15.89:0/2771662522 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e00a1fe0 con 0x7fc9e00a1340 2023-09-30T22:16:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9de59c700 1 --2- 172.21.15.89:0/2771662522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0133a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40098f0 tx=0x7fc9d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d89208235468d2c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9ddd9b700 1 -- 172.21.15.89:0/2771662522 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4011830 con 0x7fc9e00a1340 2023-09-30T22:16:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9ddd9b700 1 -- 172.21.15.89:0/2771662522 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4011990 con 0x7fc9e00a1340 2023-09-30T22:16:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9ddd9b700 1 -- 172.21.15.89:0/2771662522 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4011d20 con 0x7fc9e00a1340 2023-09-30T22:16:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9e507c700 1 -- 172.21.15.89:0/2771662522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 msgr2=0x7fc9e0133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9e507c700 1 --2- 172.21.15.89:0/2771662522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0133a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40098f0 tx=0x7fc9d4009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9e507c700 1 -- 172.21.15.89:0/2771662522 shutdown_connections 2023-09-30T22:16:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9e507c700 1 --2- 172.21.15.89:0/2771662522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9e507c700 1 --2- 172.21.15.89:0/2771662522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0133a60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9e507c700 1 -- 172.21.15.89:0/2771662522 >> 172.21.15.89:0/2771662522 conn(0x7fc9e00a6550 msgr2=0x7fc9e00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.334+0000 7fc9e507c700 1 -- 172.21.15.89:0/2771662522 shutdown_connections 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.335+0000 7fc9e507c700 1 -- 172.21.15.89:0/2771662522 wait complete. 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.336+0000 7fc9e507c700 1 Processor -- start 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.336+0000 7fc9e507c700 1 -- start start 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9e507c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e01227c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9e507c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0122d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9e507c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e0133fa0 0x7fc9e0123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9e507c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e0123c90 con 0x7fc9e00a1340 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9e507c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e01332e0 con 0x7fc9e0133fa0 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9e507c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e01244e0 con 0x7fc9e00a0a30 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0122d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60758/0 (socket says 172.21.15.89:60758) 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9de59c700 1 -- 172.21.15.89:0/1374496549 learned_addr learned my addr 172.21.15.89:0/1374496549 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9df59e700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e0133fa0 0x7fc9e0123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9ded9d700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e01227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9ded9d700 1 -- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e0133fa0 msgr2=0x7fc9e0123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9ded9d700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e0133fa0 0x7fc9e0123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9ded9d700 1 -- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 msgr2=0x7fc9e0122d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9ded9d700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0122d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9ded9d700 1 -- 172.21.15.89:0/1374496549 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4009550 con 0x7fc9e00a0a30 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.337+0000 7fc9de59c700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0122d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.338+0000 7fc9ded9d700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e01227c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000e930 tx=0x7fc9d000ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.338+0000 7fc9c77fe700 1 -- 172.21.15.89:0/1374496549 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0013680 con 0x7fc9e00a0a30 2023-09-30T22:16:20.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.338+0000 7fc9c77fe700 1 -- 172.21.15.89:0/1374496549 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d00137e0 con 0x7fc9e00a0a30 2023-09-30T22:16:20.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.338+0000 7fc9c77fe700 1 -- 172.21.15.89:0/1374496549 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0013b50 con 0x7fc9e00a0a30 2023-09-30T22:16:20.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.338+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e0133710 con 0x7fc9e00a0a30 2023-09-30T22:16:20.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.338+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e0125d90 con 0x7fc9e00a0a30 2023-09-30T22:16:20.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.339+0000 7fc9c77fe700 1 -- 172.21.15.89:0/1374496549 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7fc9d0013cb0 con 0x7fc9e00a0a30 2023-09-30T22:16:20.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.339+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e00a1d40 con 0x7fc9e00a0a30 2023-09-30T22:16:20.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.339+0000 7fc9c77fe700 1 -- 172.21.15.89:0/1374496549 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4026+0+0 (secure 0 0 0) 0x7fc9d003f830 con 0x7fc9e00a0a30 2023-09-30T22:16:20.343 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:20.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.343+0000 7fc9c77fe700 1 -- 172.21.15.89:0/1374496549 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fc9d001adc0 con 0x7fc9e00a0a30 2023-09-30T22:16:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.379+0000 7f91488a7700 1 Processor -- start 2023-09-30T22:16:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.379+0000 7f91488a7700 1 -- start start 2023-09-30T22:16:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.379+0000 7f91488a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91441338a0 0x7f9144133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.379+0000 7f91488a7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f914409e930 0x7f914413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.379+0000 7f91488a7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914406ef30 con 0x7f914409de50 2023-09-30T22:16:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.379+0000 7f91488a7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914406fef0 con 0x7f914409e930 2023-09-30T22:16:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.379+0000 7f91488a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91441334b0 con 0x7f91441338a0 2023-09-30T22:16:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f9141d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f914409de50 0x7f914409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52842/0 (socket says 172.21.15.89:52842) 2023-09-30T22:16:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f9141d9b700 1 -- 172.21.15.89:0/2011585083 learned_addr learned my addr 172.21.15.89:0/2011585083 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914259c700 1 --2- 172.21.15.89:0/2011585083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91441338a0 0x7f9144133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914159a700 1 -- 172.21.15.89:0/2011585083 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779119419 0 0) 0x7f914406ef30 con 0x7f914409de50 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914159a700 1 -- 172.21.15.89:0/2011585083 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f912c0035b0 con 0x7f914409de50 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914259c700 1 -- 172.21.15.89:0/2011585083 >> v2:172.21.15.89:3300/0 conn(0x7f914409e930 msgr2=0x7f914413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914259c700 1 --2- 172.21.15.89:0/2011585083 >> v2:172.21.15.89:3300/0 conn(0x7f914409e930 0x7f914413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914259c700 1 -- 172.21.15.89:0/2011585083 >> v1:172.21.15.89:6789/0 conn(0x7f914409de50 legacy=0x7f914409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914259c700 1 -- 172.21.15.89:0/2011585083 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914406fc30 con 0x7f91441338a0 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914259c700 1 --2- 172.21.15.89:0/2011585083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91441338a0 0x7f9144133c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9134009840 tx=0x7f9134009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ac1d4c5abe992ce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914159a700 1 -- 172.21.15.89:0/2011585083 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134011830 con 0x7f91441338a0 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914159a700 1 -- 172.21.15.89:0/2011585083 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9134011990 con 0x7f91441338a0 2023-09-30T22:16:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.380+0000 7f914159a700 1 -- 172.21.15.89:0/2011585083 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134011d20 con 0x7f91441338a0 2023-09-30T22:16:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.381+0000 7f91488a7700 1 -- 172.21.15.89:0/2011585083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91441338a0 msgr2=0x7f9144133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.381+0000 7f91488a7700 1 --2- 172.21.15.89:0/2011585083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91441338a0 0x7f9144133c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9134009840 tx=0x7f9134009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.381+0000 7f91488a7700 1 -- 172.21.15.89:0/2011585083 shutdown_connections 2023-09-30T22:16:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.381+0000 7f91488a7700 1 --2- 172.21.15.89:0/2011585083 >> v2:172.21.15.89:3300/0 conn(0x7f914409e930 0x7f914413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.381+0000 7f91488a7700 1 --2- 172.21.15.89:0/2011585083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91441338a0 0x7f9144133c70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.381+0000 7f91488a7700 1 -- 172.21.15.89:0/2011585083 >> 172.21.15.89:0/2011585083 conn(0x7f91440a2aa0 msgr2=0x7f91440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.381+0000 7f91488a7700 1 -- 172.21.15.89:0/2011585083 shutdown_connections 2023-09-30T22:16:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.381+0000 7f91488a7700 1 -- 172.21.15.89:0/2011585083 wait complete. 2023-09-30T22:16:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.382+0000 7f91488a7700 1 Processor -- start 2023-09-30T22:16:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.382+0000 7f91488a7700 1 -- start start 2023-09-30T22:16:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f91488a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f914409de50 0x7f914411b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f91488a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f914409e930 0x7f914411ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f91488a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91441338a0 0x7f914411bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f91488a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914411c9e0 con 0x7f914409e930 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f91488a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914411f050 con 0x7f914409de50 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f91488a7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f914411e400 con 0x7f91441338a0 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f914259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f914409de50 0x7f914411b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f9141d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f914409e930 0x7f914411ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f914259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f914409de50 0x7f914411b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41782/0 (socket says 172.21.15.89:41782) 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f914259c700 1 -- 172.21.15.89:0/969114515 learned_addr learned my addr 172.21.15.89:0/969114515 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f9142d9d700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91441338a0 0x7f914411bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f914259c700 1 -- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91441338a0 msgr2=0x7f914411bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f914259c700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91441338a0 0x7f914411bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f914259c700 1 -- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f914409e930 msgr2=0x7f914411ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f914259c700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f914409e930 0x7f914411ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.383+0000 7f914259c700 1 -- 172.21.15.89:0/969114515 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340094a0 con 0x7f914409de50 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.384+0000 7f9141d9b700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f914409e930 0x7f914411ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.384+0000 7f914259c700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f914409de50 0x7f914411b530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9134000c00 tx=0x7f9134004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.384+0000 7f9132ffd700 1 -- 172.21.15.89:0/969114515 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91340036a0 con 0x7f914409de50 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.384+0000 7f9132ffd700 1 -- 172.21.15.89:0/969114515 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913401f070 con 0x7f914409de50 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.384+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914411d240 con 0x7f914409de50 2023-09-30T22:16:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.384+0000 7f9132ffd700 1 -- 172.21.15.89:0/969114515 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134003a60 con 0x7f914409de50 2023-09-30T22:16:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.385+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914411df40 con 0x7f914409de50 2023-09-30T22:16:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.385+0000 7f9132ffd700 1 -- 172.21.15.89:0/969114515 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7f9134003bc0 con 0x7f914409de50 2023-09-30T22:16:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.385+0000 7f9132ffd700 1 -- 172.21.15.89:0/969114515 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4026+0+0 (secure 0 0 0) 0x7f9134021070 con 0x7f914409de50 2023-09-30T22:16:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.386+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9144074ea0 con 0x7f914409de50 2023-09-30T22:16:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.394+0000 7f9132ffd700 1 -- 172.21.15.89:0/969114515 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f9134010380 con 0x7f914409de50 2023-09-30T22:16:20.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.422+0000 7f7b51ffb700 1 -- 172.21.15.89:0/3616119972 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f7b4c025d50 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.422+0000 7fc9c77fe700 1 -- 172.21.15.89:0/1374496549 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7fc9d00128f0 con 0x7fc9e00a0a30 2023-09-30T22:16:20.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.420+0000 7f9132ffd700 1 -- 172.21.15.89:0/969114515 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f91340269a0 con 0x7f914409de50 2023-09-30T22:16:20.425 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:20.426+0000 7fa145bcb700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-30T22:16:20.430 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:20.429+0000 7fa145bcb700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-09-30T22:16:20.430 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-09-30T22:16:20.430+0000 7fa145bcb700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-30T22:16:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.486+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7b5c074ea0 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.487+0000 7f7b51ffb700 1 -- 172.21.15.89:0/3616119972 <== mon.1 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b4c01cce0 con 0x7f7b5c09a3b0 2023-09-30T22:16:20.487 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.488+0000 7f062793f700 1 Processor -- start 2023-09-30T22:16:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.488+0000 7f062793f700 1 -- start start 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.488+0000 7f062793f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062009eca0 0x7f062013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.488+0000 7f062793f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062006f050 0x7f062013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.488+0000 7f062793f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0620070ee0 con 0x7f062006e570 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.488+0000 7f062793f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0620071980 con 0x7f062006f050 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.488+0000 7f062793f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0620071480 con 0x7f062009eca0 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0625edc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062006f050 0x7f062013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f06256db700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f062006e570 0x7f062006e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52852/0 (socket says 172.21.15.89:52852) 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f06256db700 1 -- 172.21.15.89:0/531735988 learned_addr learned my addr 172.21.15.89:0/531735988 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0625edc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062006f050 0x7f062013f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41784/0 (socket says 172.21.15.89:41784) 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0624eda700 1 --2- 172.21.15.89:0/531735988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062009eca0 0x7f062013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0625edc700 1 -- 172.21.15.89:0/531735988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062009eca0 msgr2=0x7f062013cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0625edc700 1 --2- 172.21.15.89:0/531735988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062009eca0 0x7f062013cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0625edc700 1 -- 172.21.15.89:0/531735988 >> v1:172.21.15.89:6789/0 conn(0x7f062006e570 legacy=0x7f062006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0625edc700 1 -- 172.21.15.89:0/531735988 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620071b90 con 0x7f062006f050 2023-09-30T22:16:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0625edc700 1 --2- 172.21.15.89:0/531735988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062006f050 0x7f062013f3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0618009dc0 tx=0x7f061800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47919d9533bc1c14 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0617fff700 1 -- 172.21.15.89:0/531735988 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061800e070 con 0x7f062006f050 2023-09-30T22:16:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0617fff700 1 -- 172.21.15.89:0/531735988 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061800bbd0 con 0x7f062006f050 2023-09-30T22:16:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.489+0000 7f0617fff700 1 -- 172.21.15.89:0/531735988 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0618012430 con 0x7f062006f050 2023-09-30T22:16:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 >> v1:172.21.15.89:6816/126953 conn(0x7f7b40034640 legacy=0x7f7b40036a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c09a3b0 msgr2=0x7f7b5c11bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f7b63627700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c09a3b0 0x7f7b5c11bbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00e9a0 tx=0x7f7b4c00ecb0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f062793f700 1 -- 172.21.15.89:0/531735988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062006f050 msgr2=0x7f062013f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f062793f700 1 --2- 172.21.15.89:0/531735988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062006f050 0x7f062013f3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0618009dc0 tx=0x7f061800b330 comp rx=0 tx=0).stop 2023-09-30T22:16:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f062793f700 1 -- 172.21.15.89:0/531735988 shutdown_connections 2023-09-30T22:16:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f062793f700 1 --2- 172.21.15.89:0/531735988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062006f050 0x7f062013f3f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f062793f700 1 --2- 172.21.15.89:0/531735988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062009eca0 0x7f062013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f062793f700 1 -- 172.21.15.89:0/531735988 >> 172.21.15.89:0/531735988 conn(0x7f06200a2aa0 msgr2=0x7f06200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f062793f700 1 -- 172.21.15.89:0/531735988 shutdown_connections 2023-09-30T22:16:20.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.490+0000 7f062793f700 1 -- 172.21.15.89:0/531735988 wait complete. 2023-09-30T22:16:20.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.491+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 shutdown_connections 2023-09-30T22:16:20.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.491+0000 7f7b63627700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b5c131810 0x7f7b5c11c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.491+0000 7f7b63627700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b5c099a40 0x7f7b5c11b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.491+0000 7f7b63627700 1 --2- 172.21.15.89:0/3616119972 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b5c09a3b0 0x7f7b5c11bbc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.491+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 >> 172.21.15.89:0/3616119972 conn(0x7f7b5c0a2aa0 msgr2=0x7f7b5c1b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.491+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 shutdown_connections 2023-09-30T22:16:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.491+0000 7f7b63627700 1 -- 172.21.15.89:0/3616119972 wait complete. 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.491+0000 7f062793f700 1 Processor -- start 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f062793f700 1 -- start start 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f062793f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f062006e570 0x7f062013cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f062793f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062006f050 0x7f062013d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f062793f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062012e180 0x7f062013d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f062793f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062013e020 con 0x7f062006e570 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f062793f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062012dd20 con 0x7f062012e180 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f062793f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f062012d770 con 0x7f062006f050 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f06256db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f062006e570 0x7f062013cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f0624eda700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062006f050 0x7f062013d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f06256db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f062006e570 0x7f062013cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60774/0 (socket says 172.21.15.89:60774) 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f06256db700 1 -- 172.21.15.89:0/3774004328 learned_addr learned my addr 172.21.15.89:0/3774004328 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f06256db700 1 -- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062006f050 msgr2=0x7f062013d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f06256db700 1 --2- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062006f050 0x7f062013d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f06256db700 1 -- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062012e180 msgr2=0x7f062013d5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.492+0000 7f06256db700 1 --2- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062012e180 0x7f062013d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.493+0000 7f06256db700 1 -- 172.21.15.89:0/3774004328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0618009a70 con 0x7f062006e570 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.493+0000 7f0624eda700 1 --2- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062006f050 0x7f062013d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.493+0000 7f06256db700 1 --2- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f062006e570 0x7f062013cb70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f061c00c520 tx=0x7f061c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.493+0000 7f0615ffb700 1 -- 172.21.15.89:0/3774004328 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c005d70 con 0x7f062006e570 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.493+0000 7f0615ffb700 1 -- 172.21.15.89:0/3774004328 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c0070d0 con 0x7f062006e570 2023-09-30T22:16:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.493+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062013ec60 con 0x7f062006e570 2023-09-30T22:16:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.493+0000 7f0615ffb700 1 -- 172.21.15.89:0/3774004328 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c007440 con 0x7f062006e570 2023-09-30T22:16:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.493+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062012d9a0 con 0x7f062006e570 2023-09-30T22:16:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.494+0000 7f0615ffb700 1 -- 172.21.15.89:0/3774004328 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f061c0075a0 con 0x7f062006e570 2023-09-30T22:16:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.494+0000 7f0615ffb700 1 -- 172.21.15.89:0/3774004328 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4009+0+0 (secure 0 0 0) 0x7f061c03ec20 con 0x7f062006e570 2023-09-30T22:16:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.494+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0620074ea0 con 0x7f062006e570 2023-09-30T22:16:20.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.2 2023-09-30T22:16:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.504+0000 7f0615ffb700 1 -- 172.21.15.89:0/3774004328 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f061c017080 con 0x7f062006e570 2023-09-30T22:16:20.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.535+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc9e0074ea0 con 0x7fc9e00a0a30 2023-09-30T22:16:20.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.535+0000 7fc9c77fe700 1 -- 172.21.15.89:0/1374496549 <== mon.2 v2:172.21.15.89:3302/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc9d0017090 con 0x7fc9e00a0a30 2023-09-30T22:16:20.535 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:20.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.537+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 >> v1:172.21.15.89:6816/126953 conn(0x7fc9cc0345f0 legacy=0x7fc9cc0369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 msgr2=0x7fc9e01227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e01227c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000e930 tx=0x7fc9d000ec40 comp rx=0 tx=0).stop 2023-09-30T22:16:20.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 shutdown_connections 2023-09-30T22:16:20.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e0133fa0 0x7fc9e0123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9e00a0a30 0x7fc9e01227c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 --2- 172.21.15.89:0/1374496549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9e00a1340 0x7fc9e0122d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 >> 172.21.15.89:0/1374496549 conn(0x7fc9e00a6550 msgr2=0x7fc9e011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:20.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 shutdown_connections 2023-09-30T22:16:20.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.538+0000 7fc9e507c700 1 -- 172.21.15.89:0/1374496549 wait complete. 2023-09-30T22:16:20.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:16:20.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.581+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f914411ecb0 con 0x7f914409de50 2023-09-30T22:16:20.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.581+0000 7f9132ffd700 1 -- 172.21.15.89:0/969114515 <== mon.1 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f914411ecb0 con 0x7f914409de50 2023-09-30T22:16:20.581 INFO:teuthology.orchestra.run.smithi089.stdout:38654705693 2023-09-30T22:16:20.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 >> v1:172.21.15.89:6816/126953 conn(0x7f9124034690 legacy=0x7f9124036a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f914409de50 msgr2=0x7f914411b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f914409de50 0x7f914411b530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9134000c00 tx=0x7f9134004750 comp rx=0 tx=0).stop 2023-09-30T22:16:20.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 shutdown_connections 2023-09-30T22:16:20.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91441338a0 0x7f914411bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f914409de50 0x7f914411b530 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 --2- 172.21.15.89:0/969114515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f914409e930 0x7f914411ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 >> 172.21.15.89:0/969114515 conn(0x7f91440a2aa0 msgr2=0x7f9144130580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:20.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 shutdown_connections 2023-09-30T22:16:20.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.583+0000 7f91488a7700 1 -- 172.21.15.89:0/969114515 wait complete. 2023-09-30T22:16:20.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:16:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.704+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f062006b9e0 con 0x7f062006e570 2023-09-30T22:16:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.704+0000 7f0615ffb700 1 -- 172.21.15.89:0/3774004328 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f061c019b40 con 0x7f062006e570 2023-09-30T22:16:20.704 INFO:teuthology.orchestra.run.smithi089.stdout:38654705692 2023-09-30T22:16:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.706+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 >> v1:172.21.15.89:6816/126953 conn(0x7f060402f240 legacy=0x7f06040316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.706+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f062006e570 msgr2=0x7f062013cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.706+0000 7f062793f700 1 --2- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f062006e570 0x7f062013cb70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f061c00c520 tx=0x7f061c00c830 comp rx=0 tx=0).stop 2023-09-30T22:16:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.706+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 shutdown_connections 2023-09-30T22:16:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.706+0000 7f062793f700 1 --2- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f062012e180 0x7f062013d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.706+0000 7f062793f700 1 --2- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f062006e570 0x7f062013cb70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.706+0000 7f062793f700 1 --2- 172.21.15.89:0/3774004328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f062006f050 0x7f062013d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.706+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 >> 172.21.15.89:0/3774004328 conn(0x7f06200a2aa0 msgr2=0x7f062009cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.707+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 shutdown_connections 2023-09-30T22:16:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:20.707+0000 7f062793f700 1 -- 172.21.15.89:0/3774004328 wait complete. 2023-09-30T22:16:20.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705692 for osd.0 2023-09-30T22:16:21.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:21.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:21.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:21.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.651+0000 7f0209b9e700 1 Processor -- start 2023-09-30T22:16:21.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.651+0000 7f0209b9e700 1 -- start start 2023-09-30T22:16:21.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0209b9e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02041337f0 0x7f0204138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0209b9e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0209b9e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020406b990 con 0x7f0204133420 2023-09-30T22:16:21.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0209b9e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020409b0e0 con 0x7f02041337f0 2023-09-30T22:16:21.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0209b9e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0204133c20 con 0x7f0204138dc0 2023-09-30T22:16:21.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f02037fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0204133420 0x7f0204133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52866/0 (socket says 172.21.15.89:52866) 2023-09-30T22:16:21.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f02037fe700 1 -- 172.21.15.89:0/1996870095 learned_addr learned my addr 172.21.15.89:0/1996870095 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:21.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0203fff700 1 --2- 172.21.15.89:0/1996870095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0203fff700 1 -- 172.21.15.89:0/1996870095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02041337f0 msgr2=0x7f0204138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:21.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0203fff700 1 --2- 172.21.15.89:0/1996870095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02041337f0 0x7f0204138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0203fff700 1 -- 172.21.15.89:0/1996870095 >> v1:172.21.15.89:6789/0 conn(0x7f0204133420 legacy=0x7f0204133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:21.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.652+0000 7f0203fff700 1 -- 172.21.15.89:0/1996870095 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040a0590 con 0x7f0204138dc0 2023-09-30T22:16:21.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.653+0000 7f0203fff700 1 --2- 172.21.15.89:0/1996870095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020413b1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009870 tx=0x7f01f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb38ac733ea4132f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:21.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.653+0000 7f02027fc700 1 -- 172.21.15.89:0/1996870095 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4004230 con 0x7f0204138dc0 2023-09-30T22:16:21.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.653+0000 7f02027fc700 1 -- 172.21.15.89:0/1996870095 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f4004390 con 0x7f0204138dc0 2023-09-30T22:16:21.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.653+0000 7f02027fc700 1 -- 172.21.15.89:0/1996870095 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40036a0 con 0x7f0204138dc0 2023-09-30T22:16:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.653+0000 7f0209b9e700 1 -- 172.21.15.89:0/1996870095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 msgr2=0x7f020413b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.653+0000 7f0209b9e700 1 --2- 172.21.15.89:0/1996870095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020413b1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009870 tx=0x7f01f4009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.654+0000 7f0209b9e700 1 -- 172.21.15.89:0/1996870095 shutdown_connections 2023-09-30T22:16:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.654+0000 7f0209b9e700 1 --2- 172.21.15.89:0/1996870095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020413b1a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.654+0000 7f0209b9e700 1 --2- 172.21.15.89:0/1996870095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02041337f0 0x7f0204138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.654+0000 7f0209b9e700 1 -- 172.21.15.89:0/1996870095 >> 172.21.15.89:0/1996870095 conn(0x7f02040a4210 msgr2=0x7f02040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.654+0000 7f0209b9e700 1 -- 172.21.15.89:0/1996870095 shutdown_connections 2023-09-30T22:16:21.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.654+0000 7f0209b9e700 1 -- 172.21.15.89:0/1996870095 wait complete. 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.654+0000 7f0209b9e700 1 Processor -- start 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0209b9e700 1 -- start start 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0209b9e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0204133420 0x7f020409c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0209b9e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02041337f0 0x7f020409ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0209b9e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020409d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0209b9e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204134630 con 0x7f02041337f0 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0209b9e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0204130b60 con 0x7f0204133420 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0209b9e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f020412f420 con 0x7f0204138dc0 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0203fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020409d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0203fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020409d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46006/0 (socket says 172.21.15.89:46006) 2023-09-30T22:16:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0203fff700 1 -- 172.21.15.89:0/531041887 learned_addr learned my addr 172.21.15.89:0/531041887 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.655+0000 7f0202ffd700 1 --2- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02041337f0 0x7f020409ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f0203fff700 1 -- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0204133420 msgr2=0x7f020409c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f0203fff700 1 --2- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0204133420 0x7f020409c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f0203fff700 1 -- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02041337f0 msgr2=0x7f020409ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f0203fff700 1 --2- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02041337f0 0x7f020409ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f0203fff700 1 -- 172.21.15.89:0/531041887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40094a0 con 0x7f0204138dc0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f0203fff700 1 --2- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020409d340 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01f40090b0 tx=0x7f01f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f01fbfff700 1 -- 172.21.15.89:0/531041887 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40045d0 con 0x7f0204138dc0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f01fbfff700 1 -- 172.21.15.89:0/531041887 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f401f070 con 0x7f0204138dc0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f01fbfff700 1 -- 172.21.15.89:0/531041887 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011870 con 0x7f0204138dc0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020412f6e0 con 0x7f0204138dc0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.656+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0204130130 con 0x7f0204138dc0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.657+0000 7f01fbfff700 1 -- 172.21.15.89:0/531041887 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f01f40119d0 con 0x7f0204138dc0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.657+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020412fae0 con 0x7f0204138dc0 2023-09-30T22:16:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.657+0000 7f01fbfff700 1 -- 172.21.15.89:0/531041887 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f01f4021070 con 0x7f0204138dc0 2023-09-30T22:16:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.666+0000 7f01fbfff700 1 -- 172.21.15.89:0/531041887 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f01f4019d10 con 0x7f0204138dc0 2023-09-30T22:16:21.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.699+0000 7fa883f68700 1 Processor -- start 2023-09-30T22:16:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.699+0000 7fa883f68700 1 -- start start 2023-09-30T22:16:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa883f68700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12cf60 0x7fa87c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa883f68700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa87c130430 0x7fa87c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa883f68700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c12cd90 con 0x7fa87c12f8b0 2023-09-30T22:16:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa883f68700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87c12d330 con 0x7fa87c130430 2023-09-30T22:16:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa883f68700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c071780 con 0x7fa87c12cf60 2023-09-30T22:16:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa881503700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa87c12f8b0 0x7fa87c12fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52882/0 (socket says 172.21.15.89:52882) 2023-09-30T22:16:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa881503700 1 -- 172.21.15.89:0/1353299595 learned_addr learned my addr 172.21.15.89:0/1353299595 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa882505700 1 --2- 172.21.15.89:0/1353299595 >> v2:172.21.15.89:3300/0 conn(0x7fa87c130430 0x7fa87c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa881d04700 1 --2- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12cf60 0x7fa87c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa882505700 1 -- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12cf60 msgr2=0x7fa87c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa882505700 1 --2- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12cf60 0x7fa87c06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa882505700 1 -- 172.21.15.89:0/1353299595 >> v1:172.21.15.89:6789/0 conn(0x7fa87c12f8b0 legacy=0x7fa87c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa880d02700 1 -- 172.21.15.89:0/1353299595 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308825389 0 0) 0x7fa87c12cd90 con 0x7fa87c12f8b0 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa882505700 1 -- 172.21.15.89:0/1353299595 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c071a40 con 0x7fa87c130430 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.700+0000 7fa881d04700 1 --2- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12cf60 0x7fa87c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa882505700 1 --2- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 0x7fa87c137c30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa874009d90 tx=0x7fa87400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df9a9cf6c10a4f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa880d02700 1 -- 172.21.15.89:0/1353299595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400e070 con 0x7fa87c130430 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa880d02700 1 -- 172.21.15.89:0/1353299595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87400bb20 con 0x7fa87c130430 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa880d02700 1 -- 172.21.15.89:0/1353299595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874012430 con 0x7fa87c130430 2023-09-30T22:16:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa883f68700 1 -- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 msgr2=0x7fa87c137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa883f68700 1 --2- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 0x7fa87c137c30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa874009d90 tx=0x7fa87400b330 comp rx=0 tx=0).stop 2023-09-30T22:16:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa883f68700 1 -- 172.21.15.89:0/1353299595 shutdown_connections 2023-09-30T22:16:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa883f68700 1 --2- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 0x7fa87c137c30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa883f68700 1 --2- 172.21.15.89:0/1353299595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12cf60 0x7fa87c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.701+0000 7fa883f68700 1 -- 172.21.15.89:0/1353299595 >> 172.21.15.89:0/1353299595 conn(0x7fa87c0a2aa0 msgr2=0x7fa87c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.702+0000 7fa883f68700 1 -- 172.21.15.89:0/1353299595 shutdown_connections 2023-09-30T22:16:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.702+0000 7fa883f68700 1 -- 172.21.15.89:0/1353299595 wait complete. 2023-09-30T22:16:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa883f68700 1 Processor -- start 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa883f68700 1 -- start start 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa883f68700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa87c12cf60 0x7fa87c1c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa883f68700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12f8b0 0x7fa87c1c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa883f68700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 0x7fa87c1c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa883f68700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c09ce90 con 0x7fa87c130430 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa883f68700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c12d8d0 con 0x7fa87c12f8b0 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa883f68700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c09eac0 con 0x7fa87c12cf60 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa881503700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12f8b0 0x7fa87c1c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa881503700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12f8b0 0x7fa87c1c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41834/0 (socket says 172.21.15.89:41834) 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa881503700 1 -- 172.21.15.89:0/370018199 learned_addr learned my addr 172.21.15.89:0/370018199 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.703+0000 7fa882505700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 0x7fa87c1c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa881d04700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa87c12cf60 0x7fa87c1c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa881503700 1 -- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa87c12cf60 msgr2=0x7fa87c1c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa881503700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa87c12cf60 0x7fa87c1c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa881503700 1 -- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 msgr2=0x7fa87c1c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa881503700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 0x7fa87c1c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa881503700 1 -- 172.21.15.89:0/370018199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87800a640 con 0x7fa87c12f8b0 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa881503700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12f8b0 0x7fa87c1c4600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa86c001c40 tx=0x7fa86c014330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa8727fc700 1 -- 172.21.15.89:0/370018199 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c017070 con 0x7fa87c12f8b0 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa8727fc700 1 -- 172.21.15.89:0/370018199 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c001d30 con 0x7fa87c12f8b0 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa8727fc700 1 -- 172.21.15.89:0/370018199 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c014df0 con 0x7fa87c12f8b0 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa874009a70 con 0x7fa87c12f8b0 2023-09-30T22:16:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.704+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c09e800 con 0x7fa87c12f8b0 2023-09-30T22:16:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.705+0000 7fa8727fc700 1 -- 172.21.15.89:0/370018199 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7fa86c004870 con 0x7fa87c12f8b0 2023-09-30T22:16:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.705+0000 7fa8727fc700 1 -- 172.21.15.89:0/370018199 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fa86c047300 con 0x7fa87c12f8b0 2023-09-30T22:16:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.705+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c074ea0 con 0x7fa87c12f8b0 2023-09-30T22:16:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.715+0000 7fa8727fc700 1 -- 172.21.15.89:0/370018199 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa86c022070 con 0x7fa87c12f8b0 2023-09-30T22:16:21.719 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:21.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.746+0000 7f65306bf700 1 Processor -- start 2023-09-30T22:16:21.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.746+0000 7f65306bf700 1 -- start start 2023-09-30T22:16:21.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.747+0000 7f65306bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 0x7f6528135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.747+0000 7f65306bf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6528135e50 0x7f6528138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.747+0000 7f65306bf700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652809f130 con 0x7f652809d6f0 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.747+0000 7f65306bf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652809f520 con 0x7f6528135e50 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.747+0000 7f65306bf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652809e930 con 0x7f652809e1d0 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.747+0000 7f652ec5c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6528135e50 0x7f6528138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.747+0000 7f652e45b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f652809d6f0 0x7f652809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52890/0 (socket says 172.21.15.89:52890) 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.747+0000 7f652e45b700 1 -- 172.21.15.89:0/2290547623 learned_addr learned my addr 172.21.15.89:0/2290547623 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652d459700 1 -- 172.21.15.89:0/2290547623 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959998237 0 0) 0x7f652809f130 con 0x7f652809d6f0 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652d459700 1 -- 172.21.15.89:0/2290547623 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65140035b0 con 0x7f652809d6f0 2023-09-30T22:16:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652ec5c700 1 -- 172.21.15.89:0/2290547623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 msgr2=0x7f6528135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652ec5c700 1 --2- 172.21.15.89:0/2290547623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 0x7f6528135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652ec5c700 1 -- 172.21.15.89:0/2290547623 >> v1:172.21.15.89:6789/0 conn(0x7f652809d6f0 legacy=0x7f652809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652ec5c700 1 -- 172.21.15.89:0/2290547623 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652809b2d0 con 0x7f6528135e50 2023-09-30T22:16:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652ec5c700 1 --2- 172.21.15.89:0/2290547623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f6528138230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6520009df0 tx=0x7f652000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53a5d7380b48918 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652d459700 1 -- 172.21.15.89:0/2290547623 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000e070 con 0x7f6528135e50 2023-09-30T22:16:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652d459700 1 -- 172.21.15.89:0/2290547623 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652000baa0 con 0x7f6528135e50 2023-09-30T22:16:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.748+0000 7f652d459700 1 -- 172.21.15.89:0/2290547623 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000be50 con 0x7f6528135e50 2023-09-30T22:16:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.749+0000 7f65306bf700 1 -- 172.21.15.89:0/2290547623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 msgr2=0x7f6528138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.749+0000 7f65306bf700 1 --2- 172.21.15.89:0/2290547623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f6528138230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6520009df0 tx=0x7f652000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.749+0000 7f65306bf700 1 -- 172.21.15.89:0/2290547623 shutdown_connections 2023-09-30T22:16:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.749+0000 7f65306bf700 1 --2- 172.21.15.89:0/2290547623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f6528138230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.749+0000 7f65306bf700 1 --2- 172.21.15.89:0/2290547623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 0x7f6528135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.749+0000 7f65306bf700 1 -- 172.21.15.89:0/2290547623 >> 172.21.15.89:0/2290547623 conn(0x7f6528066f50 msgr2=0x7f652809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.749+0000 7f65306bf700 1 -- 172.21.15.89:0/2290547623 shutdown_connections 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.749+0000 7f65306bf700 1 -- 172.21.15.89:0/2290547623 wait complete. 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.750+0000 7f65306bf700 1 Processor -- start 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.750+0000 7f65306bf700 1 -- start start 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f65306bf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652809d6f0 0x7f652811b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f65306bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 0x7f652811bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f65306bf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f652811c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f65306bf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652811d1a0 con 0x7f6528135e50 2023-09-30T22:16:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f65306bf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652811e3f0 con 0x7f652809e1d0 2023-09-30T22:16:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f65306bf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f652811da00 con 0x7f652809d6f0 2023-09-30T22:16:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652ec5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f652811c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652ec5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f652811c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60822/0 (socket says 172.21.15.89:60822) 2023-09-30T22:16:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652ec5c700 1 -- 172.21.15.89:0/742820329 learned_addr learned my addr 172.21.15.89:0/742820329 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652dc5a700 1 --2- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 0x7f652811bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652ec5c700 1 -- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652809d6f0 msgr2=0x7f652811b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652ec5c700 1 --2- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652809d6f0 0x7f652811b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652ec5c700 1 -- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 msgr2=0x7f652811bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652ec5c700 1 --2- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 0x7f652811bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.751+0000 7f652ec5c700 1 -- 172.21.15.89:0/742820329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6520009a70 con 0x7f6528135e50 2023-09-30T22:16:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.752+0000 7f652ec5c700 1 --2- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f652811c010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6520013000 tx=0x7f6520004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.752+0000 7f651effd700 1 -- 172.21.15.89:0/742820329 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200048e0 con 0x7f6528135e50 2023-09-30T22:16:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.752+0000 7f651effd700 1 -- 172.21.15.89:0/742820329 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520003e10 con 0x7f6528135e50 2023-09-30T22:16:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.752+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652811edd0 con 0x7f6528135e50 2023-09-30T22:16:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.752+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652809c700 con 0x7f6528135e50 2023-09-30T22:16:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.752+0000 7f651effd700 1 -- 172.21.15.89:0/742820329 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520027670 con 0x7f6528135e50 2023-09-30T22:16:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.753+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65100052e0 con 0x7f6528135e50 2023-09-30T22:16:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.753+0000 7f651effd700 1 -- 172.21.15.89:0/742820329 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f6520030430 con 0x7f6528135e50 2023-09-30T22:16:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.754+0000 7f651effd700 1 -- 172.21.15.89:0/742820329 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f6520022030 con 0x7f6528135e50 2023-09-30T22:16:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.758+0000 7f651effd700 1 -- 172.21.15.89:0/742820329 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65200306e0 con 0x7f6528135e50 2023-09-30T22:16:21.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.781+0000 7f651effd700 1 -- 172.21.15.89:0/742820329 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f652001bd20 con 0x7f6528135e50 2023-09-30T22:16:21.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.781+0000 7fa8727fc700 1 -- 172.21.15.89:0/370018199 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7fa86c0075e0 con 0x7fa87c12f8b0 2023-09-30T22:16:21.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.789+0000 7f01fbfff700 1 -- 172.21.15.89:0/531041887 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f01f40269a0 con 0x7f0204138dc0 2023-09-30T22:16:21.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.858+0000 7fa5bf46b700 1 Processor -- start 2023-09-30T22:16:21.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.858+0000 7fa5bf46b700 1 -- start start 2023-09-30T22:16:21.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.859+0000 7fa5bf46b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5b806a770 0x7fa5b809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.859+0000 7fa5bf46b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.859+0000 7fa5bf46b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b809a810 con 0x7fa5b809dc30 2023-09-30T22:16:21.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.859+0000 7fa5bf46b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b806ab40 con 0x7fa5b806a770 2023-09-30T22:16:21.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.859+0000 7fa5bf46b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b809ebc0 con 0x7fa5b813d1f0 2023-09-30T22:16:21.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5bda08700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5bda08700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b813f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46022/0 (socket says 172.21.15.89:46022) 2023-09-30T22:16:21.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5bda08700 1 -- 172.21.15.89:0/2790142671 learned_addr learned my addr 172.21.15.89:0/2790142671 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:21.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5bda08700 1 -- 172.21.15.89:0/2790142671 >> v2:172.21.15.89:3300/0 conn(0x7fa5b806a770 msgr2=0x7fa5b809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:21.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5bda08700 1 --2- 172.21.15.89:0/2790142671 >> v2:172.21.15.89:3300/0 conn(0x7fa5b806a770 0x7fa5b809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5bda08700 1 -- 172.21.15.89:0/2790142671 >> v1:172.21.15.89:6789/0 conn(0x7fa5b809dc30 legacy=0x7fa5b809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:21.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5bda08700 1 -- 172.21.15.89:0/2790142671 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b809e920 con 0x7fa5b813d1f0 2023-09-30T22:16:21.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5bda08700 1 --2- 172.21.15.89:0/2790142671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b813f5d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009e50 tx=0x7fa5b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927ccf223614e947 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:21.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5affff700 1 -- 172.21.15.89:0/2790142671 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000e070 con 0x7fa5b813d1f0 2023-09-30T22:16:21.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5affff700 1 -- 172.21.15.89:0/2790142671 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b0003a20 con 0x7fa5b813d1f0 2023-09-30T22:16:21.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.860+0000 7fa5affff700 1 -- 172.21.15.89:0/2790142671 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0003dd0 con 0x7fa5b813d1f0 2023-09-30T22:16:21.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.861+0000 7fa5bf46b700 1 -- 172.21.15.89:0/2790142671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 msgr2=0x7fa5b813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.861+0000 7fa5bf46b700 1 --2- 172.21.15.89:0/2790142671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b813f5d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009e50 tx=0x7fa5b000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:21.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.861+0000 7fa5bf46b700 1 -- 172.21.15.89:0/2790142671 shutdown_connections 2023-09-30T22:16:21.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.861+0000 7fa5bf46b700 1 --2- 172.21.15.89:0/2790142671 >> v2:172.21.15.89:3300/0 conn(0x7fa5b806a770 0x7fa5b809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.861+0000 7fa5bf46b700 1 --2- 172.21.15.89:0/2790142671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b813f5d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.861+0000 7fa5bf46b700 1 -- 172.21.15.89:0/2790142671 >> 172.21.15.89:0/2790142671 conn(0x7fa5b8066f50 msgr2=0x7fa5b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:21.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.861+0000 7fa5bf46b700 1 -- 172.21.15.89:0/2790142671 shutdown_connections 2023-09-30T22:16:21.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.861+0000 7fa5bf46b700 1 -- 172.21.15.89:0/2790142671 wait complete. 2023-09-30T22:16:21.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.862+0000 7fa5bf46b700 1 Processor -- start 2023-09-30T22:16:21.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.862+0000 7fa5bf46b700 1 -- start start 2023-09-30T22:16:21.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bf46b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b806a770 0x7fa5b81260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bf46b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b809dc30 0x7fa5b81265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bf46b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b8126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bf46b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8128790 con 0x7fa5b806a770 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bf46b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8127560 con 0x7fa5b809dc30 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bf46b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b809f1a0 con 0x7fa5b813d1f0 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bca06700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b809dc30 0x7fa5b81265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bda08700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b8126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bca06700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b809dc30 0x7fa5b81265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41864/0 (socket says 172.21.15.89:41864) 2023-09-30T22:16:21.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.863+0000 7fa5bca06700 1 -- 172.21.15.89:0/1720951387 learned_addr learned my addr 172.21.15.89:0/1720951387 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5bda08700 1 -- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b809dc30 msgr2=0x7fa5b81265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5bda08700 1 --2- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b809dc30 0x7fa5b81265e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5bda08700 1 -- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b806a770 msgr2=0x7fa5b81260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5bda08700 1 --2- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b806a770 0x7fa5b81260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5bda08700 1 -- 172.21.15.89:0/1720951387 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0009b30 con 0x7fa5b813d1f0 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5bda08700 1 --2- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b8126b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0014000 tx=0x7fa5b0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5adffb700 1 -- 172.21.15.89:0/1720951387 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0004440 con 0x7fa5b813d1f0 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b8129100 con 0x7fa5b813d1f0 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5adffb700 1 -- 172.21.15.89:0/1720951387 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b00045a0 con 0x7fa5b813d1f0 2023-09-30T22:16:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5adffb700 1 -- 172.21.15.89:0/1720951387 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0004950 con 0x7fa5b813d1f0 2023-09-30T22:16:21.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.864+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b8127270 con 0x7fa5b813d1f0 2023-09-30T22:16:21.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.865+0000 7fa5adffb700 1 -- 172.21.15.89:0/1720951387 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7fa5b0012430 con 0x7fa5b813d1f0 2023-09-30T22:16:21.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.865+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b81277a0 con 0x7fa5b813d1f0 2023-09-30T22:16:21.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.866+0000 7fa5adffb700 1 -- 172.21.15.89:0/1720951387 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fa5b0041340 con 0x7fa5b813d1f0 2023-09-30T22:16:21.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.875+0000 7fa5adffb700 1 -- 172.21.15.89:0/1720951387 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5b0025ba0 con 0x7fa5b813d1f0 2023-09-30T22:16:21.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.992+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6510005860 con 0x7f6528135e50 2023-09-30T22:16:21.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.993+0000 7f651effd700 1 -- 172.21.15.89:0/742820329 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f652000ca20 con 0x7f6528135e50 2023-09-30T22:16:21.993 INFO:teuthology.orchestra.run.smithi089.stdout:38654705695 2023-09-30T22:16:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 >> v1:172.21.15.89:6816/126953 conn(0x7f650c02f270 legacy=0x7f650c031720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 msgr2=0x7f652811c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 --2- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f652811c010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6520013000 tx=0x7f6520004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 shutdown_connections 2023-09-30T22:16:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 --2- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6528135e50 0x7f652811c010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 --2- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652809d6f0 0x7f652811b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 --2- 172.21.15.89:0/742820329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652809e1d0 0x7f652811bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 >> 172.21.15.89:0/742820329 conn(0x7f6528066f50 msgr2=0x7f652806e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 shutdown_connections 2023-09-30T22:16:21.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:21.995+0000 7f65306bf700 1 -- 172.21.15.89:0/742820329 wait complete. 2023-09-30T22:16:22.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705695 for osd.1 2023-09-30T22:16:22.006 DEBUG:teuthology.parallel:result is None 2023-09-30T22:16:22.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.026+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0204074ea0 con 0x7f0204138dc0 2023-09-30T22:16:22.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.026+0000 7f01fbfff700 1 -- 172.21.15.89:0/531041887 <== mon.2 v2:172.21.15.89:3302/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01f402b020 con 0x7f0204138dc0 2023-09-30T22:16:22.027 INFO:teuthology.orchestra.run.smithi089.stdout:38654705694 2023-09-30T22:16:22.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 >> v1:172.21.15.89:6816/126953 conn(0x7f01e402f240 legacy=0x7f01e40316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 msgr2=0x7f020409d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 --2- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020409d340 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01f40090b0 tx=0x7f01f4004750 comp rx=0 tx=0).stop 2023-09-30T22:16:22.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 shutdown_connections 2023-09-30T22:16:22.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 --2- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0204138dc0 0x7f020409d340 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 --2- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0204133420 0x7f020409c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 --2- 172.21.15.89:0/531041887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02041337f0 0x7f020409ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 >> 172.21.15.89:0/531041887 conn(0x7f02040a4210 msgr2=0x7f0204131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:22.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 shutdown_connections 2023-09-30T22:16:22.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.029+0000 7f0209b9e700 1 -- 172.21.15.89:0/531041887 wait complete. 2023-09-30T22:16:22.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705694 for osd.2 2023-09-30T22:16:22.047 DEBUG:teuthology.parallel:result is None 2023-09-30T22:16:22.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.076+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa5b8074ea0 con 0x7fa5b813d1f0 2023-09-30T22:16:22.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.078+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa87c12e2d0 con 0x7fa87c12f8b0 2023-09-30T22:16:22.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.078+0000 7fa8727fc700 1 -- 172.21.15.89:0/370018199 <== mon.1 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa86c01b560 con 0x7fa87c12f8b0 2023-09-30T22:16:22.078 INFO:teuthology.orchestra.run.smithi089.stdout:38654705695 2023-09-30T22:16:22.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 >> v1:172.21.15.89:6816/126953 conn(0x7fa86002f240 legacy=0x7fa8600316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12f8b0 msgr2=0x7fa87c1c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12f8b0 0x7fa87c1c4600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa86c001c40 tx=0x7fa86c014330 comp rx=0 tx=0).stop 2023-09-30T22:16:22.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 shutdown_connections 2023-09-30T22:16:22.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa87c130430 0x7fa87c1c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa87c12cf60 0x7fa87c1c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 --2- 172.21.15.89:0/370018199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa87c12f8b0 0x7fa87c1c4600 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 >> 172.21.15.89:0/370018199 conn(0x7fa87c0a2aa0 msgr2=0x7fa87c1b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:22.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 shutdown_connections 2023-09-30T22:16:22.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.080+0000 7fa883f68700 1 -- 172.21.15.89:0/370018199 wait complete. 2023-09-30T22:16:22.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705695 for osd.3 2023-09-30T22:16:22.092 DEBUG:teuthology.parallel:result is None 2023-09-30T22:16:22.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.105+0000 7fa5adffb700 1 -- 172.21.15.89:0/1720951387 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5b0017020 con 0x7fa5b813d1f0 2023-09-30T22:16:22.105 INFO:teuthology.orchestra.run.smithi089.stdout:38654705694 2023-09-30T22:16:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 >> v1:172.21.15.89:6816/126953 conn(0x7fa59c02f480 legacy=0x7fa59c031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 msgr2=0x7fa5b8126b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 --2- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b8126b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0014000 tx=0x7fa5b0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 shutdown_connections 2023-09-30T22:16:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 --2- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b813d1f0 0x7fa5b8126b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 --2- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b806a770 0x7fa5b81260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 --2- 172.21.15.89:0/1720951387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b809dc30 0x7fa5b81265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 >> 172.21.15.89:0/1720951387 conn(0x7fa5b8066f50 msgr2=0x7fa5b80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 shutdown_connections 2023-09-30T22:16:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.108+0000 7fa5bf46b700 1 -- 172.21.15.89:0/1720951387 wait complete. 2023-09-30T22:16:22.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705693 got 38654705694 for osd.0 2023-09-30T22:16:22.122 DEBUG:teuthology.parallel:result is None 2023-09-30T22:16:22.122 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:16:22.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:16:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.282+0000 7f570261c700 1 Processor -- start 2023-09-30T22:16:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.282+0000 7f570261c700 1 -- start start 2023-09-30T22:16:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.282+0000 7f570261c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56fc09e930 0x7f56fc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.282+0000 7f570261c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.282+0000 7f570261c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56fc070940 con 0x7f56fc09f650 2023-09-30T22:16:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.282+0000 7f570261c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56fc06fd60 con 0x7f56fc09e930 2023-09-30T22:16:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.282+0000 7f570261c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc06fa80 con 0x7f56fc09f240 2023-09-30T22:16:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56fc09e930 0x7f56fc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f5700bb9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f56fc09f650 0x7f56fc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52896/0 (socket says 172.21.15.89:52896) 2023-09-30T22:16:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f5700bb9700 1 -- 172.21.15.89:0/2742718607 learned_addr learned my addr 172.21.15.89:0/2742718607 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:22.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46042/0 (socket says 172.21.15.89:46042) 2023-09-30T22:16:22.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fb7fe700 1 -- 172.21.15.89:0/2742718607 >> v2:172.21.15.89:3300/0 conn(0x7f56fc09e930 msgr2=0x7f56fc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:22.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fb7fe700 1 --2- 172.21.15.89:0/2742718607 >> v2:172.21.15.89:3300/0 conn(0x7f56fc09e930 0x7f56fc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fb7fe700 1 -- 172.21.15.89:0/2742718607 >> v1:172.21.15.89:6789/0 conn(0x7f56fc09f650 legacy=0x7f56fc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fb7fe700 1 -- 172.21.15.89:0/2742718607 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc09fb60 con 0x7f56fc09f240 2023-09-30T22:16:22.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56faffd700 1 -- 172.21.15.89:0/2742718607 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408692438 0 0) 0x7f56fc070940 con 0x7f56fc09f650 2023-09-30T22:16:22.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fbfff700 1 --2- 172.21.15.89:0/2742718607 >> v2:172.21.15.89:3300/0 conn(0x7f56fc09e930 0x7f56fc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:22.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56fb7fe700 1 --2- 172.21.15.89:0/2742718607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1317f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009840 tx=0x7f56ec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da515a263ca9330f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56faffd700 1 -- 172.21.15.89:0/2742718607 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec011830 con 0x7f56fc09f240 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56faffd700 1 -- 172.21.15.89:0/2742718607 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ec011990 con 0x7f56fc09f240 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.283+0000 7f56faffd700 1 -- 172.21.15.89:0/2742718607 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec011d20 con 0x7f56fc09f240 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.284+0000 7f570261c700 1 -- 172.21.15.89:0/2742718607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 msgr2=0x7f56fc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.284+0000 7f570261c700 1 --2- 172.21.15.89:0/2742718607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1317f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009840 tx=0x7f56ec009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.284+0000 7f570261c700 1 -- 172.21.15.89:0/2742718607 shutdown_connections 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.284+0000 7f570261c700 1 --2- 172.21.15.89:0/2742718607 >> v2:172.21.15.89:3300/0 conn(0x7f56fc09e930 0x7f56fc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.284+0000 7f570261c700 1 --2- 172.21.15.89:0/2742718607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1317f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.284+0000 7f570261c700 1 -- 172.21.15.89:0/2742718607 >> 172.21.15.89:0/2742718607 conn(0x7f56fc0a4210 msgr2=0x7f56fc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:22.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.284+0000 7f570261c700 1 -- 172.21.15.89:0/2742718607 shutdown_connections 2023-09-30T22:16:22.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.284+0000 7f570261c700 1 -- 172.21.15.89:0/2742718607 wait complete. 2023-09-30T22:16:22.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f570261c700 1 Processor -- start 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f570261c700 1 -- start start 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f570261c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56fc09e930 0x7f56fc1c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f570261c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f570261c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56fc09f650 0x7f56fc1c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f570261c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1a7740 con 0x7f56fc09f650 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f570261c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1a4f50 con 0x7f56fc09e930 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f570261c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc0a3260 con 0x7f56fc09f240 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f56fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f5700bb9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56fc09f650 0x7f56fc1c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f56fbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56fc09e930 0x7f56fc1c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f56fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46052/0 (socket says 172.21.15.89:46052) 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.285+0000 7f56fb7fe700 1 -- 172.21.15.89:0/2622801389 learned_addr learned my addr 172.21.15.89:0/2622801389 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f5700bb9700 1 -- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 msgr2=0x7f56fc1c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f5700bb9700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f5700bb9700 1 -- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56fc09e930 msgr2=0x7f56fc1c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f5700bb9700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56fc09e930 0x7f56fc1c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f5700bb9700 1 -- 172.21.15.89:0/2622801389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00094a0 con 0x7f56fc09f650 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f56fbfff700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56fc09e930 0x7f56fc1c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f56fb7fe700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f5700bb9700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56fc09f650 0x7f56fc1c27d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56f400cc10 tx=0x7f56f400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f56f8ff9700 1 -- 172.21.15.89:0/2622801389 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f400aa00 con 0x7f56fc09f650 2023-09-30T22:16:22.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f56f8ff9700 1 -- 172.21.15.89:0/2622801389 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f400ab80 con 0x7f56fc09f650 2023-09-30T22:16:22.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec0094a0 con 0x7f56fc09f650 2023-09-30T22:16:22.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f56f8ff9700 1 -- 172.21.15.89:0/2622801389 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f40070d0 con 0x7f56fc09f650 2023-09-30T22:16:22.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.286+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc0a0190 con 0x7f56fc09f650 2023-09-30T22:16:22.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.287+0000 7f56f8ff9700 1 -- 172.21.15.89:0/2622801389 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f56f4007230 con 0x7f56fc09f650 2023-09-30T22:16:22.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.287+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc1c6c90 con 0x7f56fc09f650 2023-09-30T22:16:22.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.287+0000 7f56f8ff9700 1 -- 172.21.15.89:0/2622801389 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f56f403f840 con 0x7f56fc09f650 2023-09-30T22:16:22.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.292+0000 7f56f8ff9700 1 -- 172.21.15.89:0/2622801389 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56f40074e0 con 0x7f56fc09f650 2023-09-30T22:16:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.471+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 --> v1:172.21.15.89:6816/126953 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56fc0704f0 con 0x7f56e802f480 2023-09-30T22:16:22.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.479+0000 7f56f8ff9700 1 -- 172.21.15.89:0/2622801389 <== mgr.14124 v1:172.21.15.89:6816/126953 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+270369 (unknown 2965378022 0 2887483882) 0x7f56f0009cc0 con 0x7f56e802f480 2023-09-30T22:16:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:22.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 >> v1:172.21.15.89:6816/126953 conn(0x7f56e802f480 legacy=0x7f56e8031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56fc09f650 msgr2=0x7f56fc1c27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56fc09f650 0x7f56fc1c27d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56f400cc10 tx=0x7f56f400cf20 comp rx=0 tx=0).stop 2023-09-30T22:16:22.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 shutdown_connections 2023-09-30T22:16:22.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56fc09f650 0x7f56fc1c27d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56fc09e930 0x7f56fc1c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 --2- 172.21.15.89:0/2622801389 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56fc09f240 0x7f56fc1c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 >> 172.21.15.89:0/2622801389 conn(0x7f56fc0a4210 msgr2=0x7f56fc1b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:22.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 shutdown_connections 2023-09-30T22:16:22.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.482+0000 7f570261c700 1 -- 172.21.15.89:0/2622801389 wait complete. 2023-09-30T22:16:22.494 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:16:22.495 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2023-09-30T22:16:21.424349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189132,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298228,"statfs":{"total":386547056640,"available":385329385472,"internally_reserved":0,"allocated":765952,"data_stored":258524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.992274"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551652+0000","last_change":"2023-09-30T22:14:02.086445+0000","last_active":"2023-09-30T22:14:52.551652+0000","last_peered":"2023-09-30T22:14:52.551652+0000","last_clean":"2023-09-30T22:14:52.551652+0000","last_became_active":"2023-09-30T22:14:02.086159+0000","last_became_peered":"2023-09-30T22:14:02.086159+0000","last_unstale":"2023-09-30T22:14:52.551652+0000","last_undegraded":"2023-09-30T22:14:52.551652+0000","last_fullsized":"2023-09-30T22:14:52.551652+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552688+0000","last_change":"2023-09-30T22:14:02.082560+0000","last_active":"2023-09-30T22:14:52.552688+0000","last_peered":"2023-09-30T22:14:52.552688+0000","last_clean":"2023-09-30T22:14:52.552688+0000","last_became_active":"2023-09-30T22:14:02.082315+0000","last_became_peered":"2023-09-30T22:14:02.082315+0000","last_unstale":"2023-09-30T22:14:52.552688+0000","last_undegraded":"2023-09-30T22:14:52.552688+0000","last_fullsized":"2023-09-30T22:14:52.552688+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549798+0000","last_change":"2023-09-30T22:14:02.082673+0000","last_active":"2023-09-30T22:14:52.549798+0000","last_peered":"2023-09-30T22:14:52.549798+0000","last_clean":"2023-09-30T22:14:52.549798+0000","last_became_active":"2023-09-30T22:14:02.082437+0000","last_became_peered":"2023-09-30T22:14:02.082437+0000","last_unstale":"2023-09-30T22:14:52.549798+0000","last_undegraded":"2023-09-30T22:14:52.549798+0000","last_fullsized":"2023-09-30T22:14:52.549798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550210+0000","last_change":"2023-09-30T22:14:02.080886+0000","last_active":"2023-09-30T22:14:52.550210+0000","last_peered":"2023-09-30T22:14:52.550210+0000","last_clean":"2023-09-30T22:14:52.550210+0000","last_became_active":"2023-09-30T22:14:02.078093+0000","last_became_peered":"2023-09-30T22:14:02.078093+0000","last_unstale":"2023-09-30T22:14:52.550210+0000","last_undegraded":"2023-09-30T22:14:52.550210+0000","last_fullsized":"2023-09-30T22:14:52.550210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549844+0000","last_change":"2023-09-30T22:14:02.074967+0000","last_active":"2023-09-30T22:14:52.549844+0000","last_peered":"2023-09-30T22:14:52.549844+0000","last_clean":"2023-09-30T22:14:52.549844+0000","last_became_active":"2023-09-30T22:14:02.074701+0000","last_became_peered":"2023-09-30T22:14:02.074701+0000","last_unstale":"2023-09-30T22:14:52.549844+0000","last_undegraded":"2023-09-30T22:14:52.549844+0000","last_fullsized":"2023-09-30T22:14:52.549844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733701+0000","last_change":"2023-09-30T22:14:02.081443+0000","last_active":"2023-09-30T22:14:52.733701+0000","last_peered":"2023-09-30T22:14:52.733701+0000","last_clean":"2023-09-30T22:14:52.733701+0000","last_became_active":"2023-09-30T22:14:02.080408+0000","last_became_peered":"2023-09-30T22:14:02.080408+0000","last_unstale":"2023-09-30T22:14:52.733701+0000","last_undegraded":"2023-09-30T22:14:52.733701+0000","last_fullsized":"2023-09-30T22:14:52.733701+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733177+0000","last_change":"2023-09-30T22:14:02.085310+0000","last_active":"2023-09-30T22:14:52.733177+0000","last_peered":"2023-09-30T22:14:52.733177+0000","last_clean":"2023-09-30T22:14:52.733177+0000","last_became_active":"2023-09-30T22:14:02.084647+0000","last_became_peered":"2023-09-30T22:14:02.084647+0000","last_unstale":"2023-09-30T22:14:52.733177+0000","last_undegraded":"2023-09-30T22:14:52.733177+0000","last_fullsized":"2023-09-30T22:14:52.733177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550773+0000","last_change":"2023-09-30T22:14:02.085988+0000","last_active":"2023-09-30T22:14:52.550773+0000","last_peered":"2023-09-30T22:14:52.550773+0000","last_clean":"2023-09-30T22:14:52.550773+0000","last_became_active":"2023-09-30T22:14:02.083245+0000","last_became_peered":"2023-09-30T22:14:02.083245+0000","last_unstale":"2023-09-30T22:14:52.550773+0000","last_undegraded":"2023-09-30T22:14:52.550773+0000","last_fullsized":"2023-09-30T22:14:52.550773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552926+0000","last_change":"2023-09-30T22:14:02.085522+0000","last_active":"2023-09-30T22:14:52.552926+0000","last_peered":"2023-09-30T22:14:52.552926+0000","last_clean":"2023-09-30T22:14:52.552926+0000","last_became_active":"2023-09-30T22:14:02.084959+0000","last_became_peered":"2023-09-30T22:14:02.084959+0000","last_unstale":"2023-09-30T22:14:52.552926+0000","last_undegraded":"2023-09-30T22:14:52.552926+0000","last_fullsized":"2023-09-30T22:14:52.552926+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552859+0000","last_change":"2023-09-30T22:14:02.071672+0000","last_active":"2023-09-30T22:14:52.552859+0000","last_peered":"2023-09-30T22:14:52.552859+0000","last_clean":"2023-09-30T22:14:52.552859+0000","last_became_active":"2023-09-30T22:14:02.071361+0000","last_became_peered":"2023-09-30T22:14:02.071361+0000","last_unstale":"2023-09-30T22:14:52.552859+0000","last_undegraded":"2023-09-30T22:14:52.552859+0000","last_fullsized":"2023-09-30T22:14:52.552859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552384+0000","last_change":"2023-09-30T22:14:02.074689+0000","last_active":"2023-09-30T22:14:52.552384+0000","last_peered":"2023-09-30T22:14:52.552384+0000","last_clean":"2023-09-30T22:14:52.552384+0000","last_became_active":"2023-09-30T22:14:02.074350+0000","last_became_peered":"2023-09-30T22:14:02.074350+0000","last_unstale":"2023-09-30T22:14:52.552384+0000","last_undegraded":"2023-09-30T22:14:52.552384+0000","last_fullsized":"2023-09-30T22:14:52.552384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.556558+0000","last_change":"2023-09-30T22:14:52.554121+0000","last_active":"2023-09-30T22:14:52.556558+0000","last_peered":"2023-09-30T22:14:52.556558+0000","last_clean":"2023-09-30T22:14:52.556558+0000","last_became_active":"2023-09-30T22:14:52.553793+0000","last_became_peered":"2023-09-30T22:14:52.553793+0000","last_unstale":"2023-09-30T22:14:52.556558+0000","last_undegraded":"2023-09-30T22:14:52.556558+0000","last_fullsized":"2023-09-30T22:14:52.556558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458275+0000","last_change":"2023-09-30T22:14:02.087347+0000","last_active":"2023-09-30T22:16:19.458275+0000","last_peered":"2023-09-30T22:16:19.458275+0000","last_clean":"2023-09-30T22:16:19.458275+0000","last_became_active":"2023-09-30T22:14:02.086637+0000","last_became_peered":"2023-09-30T22:14:02.086637+0000","last_unstale":"2023-09-30T22:16:19.458275+0000","last_undegraded":"2023-09-30T22:16:19.458275+0000","last_fullsized":"2023-09-30T22:16:19.458275+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552168+0000","last_change":"2023-09-30T22:14:02.066399+0000","last_active":"2023-09-30T22:14:52.552168+0000","last_peered":"2023-09-30T22:14:52.552168+0000","last_clean":"2023-09-30T22:14:52.552168+0000","last_became_active":"2023-09-30T22:14:02.066052+0000","last_became_peered":"2023-09-30T22:14:02.066052+0000","last_unstale":"2023-09-30T22:14:52.552168+0000","last_undegraded":"2023-09-30T22:14:52.552168+0000","last_fullsized":"2023-09-30T22:14:52.552168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549533+0000","last_change":"2023-09-30T22:14:02.073165+0000","last_active":"2023-09-30T22:14:52.549533+0000","last_peered":"2023-09-30T22:14:52.549533+0000","last_clean":"2023-09-30T22:14:52.549533+0000","last_became_active":"2023-09-30T22:14:02.073000+0000","last_became_peered":"2023-09-30T22:14:02.073000+0000","last_unstale":"2023-09-30T22:14:52.549533+0000","last_undegraded":"2023-09-30T22:14:52.549533+0000","last_fullsized":"2023-09-30T22:14:52.549533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733099+0000","last_change":"2023-09-30T22:14:02.072772+0000","last_active":"2023-09-30T22:14:52.733099+0000","last_peered":"2023-09-30T22:14:52.733099+0000","last_clean":"2023-09-30T22:14:52.733099+0000","last_became_active":"2023-09-30T22:14:02.072569+0000","last_became_peered":"2023-09-30T22:14:02.072569+0000","last_unstale":"2023-09-30T22:14:52.733099+0000","last_undegraded":"2023-09-30T22:14:52.733099+0000","last_fullsized":"2023-09-30T22:14:52.733099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.557223+0000","last_change":"2023-09-30T22:14:52.556259+0000","last_active":"2023-09-30T22:14:52.557223+0000","last_peered":"2023-09-30T22:14:52.557223+0000","last_clean":"2023-09-30T22:14:52.557223+0000","last_became_active":"2023-09-30T22:14:52.556005+0000","last_became_peered":"2023-09-30T22:14:52.556005+0000","last_unstale":"2023-09-30T22:14:52.557223+0000","last_undegraded":"2023-09-30T22:14:52.557223+0000","last_fullsized":"2023-09-30T22:14:52.557223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551000+0000","last_change":"2023-09-30T22:14:02.070008+0000","last_active":"2023-09-30T22:14:52.551000+0000","last_peered":"2023-09-30T22:14:52.551000+0000","last_clean":"2023-09-30T22:14:52.551000+0000","last_became_active":"2023-09-30T22:14:02.069809+0000","last_became_peered":"2023-09-30T22:14:02.069809+0000","last_unstale":"2023-09-30T22:14:52.551000+0000","last_undegraded":"2023-09-30T22:14:52.551000+0000","last_fullsized":"2023-09-30T22:14:52.551000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549880+0000","last_change":"2023-09-30T22:14:02.087939+0000","last_active":"2023-09-30T22:14:52.549880+0000","last_peered":"2023-09-30T22:14:52.549880+0000","last_clean":"2023-09-30T22:14:52.549880+0000","last_became_active":"2023-09-30T22:14:02.087733+0000","last_became_peered":"2023-09-30T22:14:02.087733+0000","last_unstale":"2023-09-30T22:14:52.549880+0000","last_undegraded":"2023-09-30T22:14:52.549880+0000","last_fullsized":"2023-09-30T22:14:52.549880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552401+0000","last_change":"2023-09-30T22:14:02.090627+0000","last_active":"2023-09-30T22:14:52.552401+0000","last_peered":"2023-09-30T22:14:52.552401+0000","last_clean":"2023-09-30T22:14:52.552401+0000","last_became_active":"2023-09-30T22:14:02.090255+0000","last_became_peered":"2023-09-30T22:14:02.090255+0000","last_unstale":"2023-09-30T22:14:52.552401+0000","last_undegraded":"2023-09-30T22:14:52.552401+0000","last_fullsized":"2023-09-30T22:14:52.552401+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551837+0000","last_change":"2023-09-30T22:14:02.071417+0000","last_active":"2023-09-30T22:14:52.551837+0000","last_peered":"2023-09-30T22:14:52.551837+0000","last_clean":"2023-09-30T22:14:52.551837+0000","last_became_active":"2023-09-30T22:14:02.071046+0000","last_became_peered":"2023-09-30T22:14:02.071046+0000","last_unstale":"2023-09-30T22:14:52.551837+0000","last_undegraded":"2023-09-30T22:14:52.551837+0000","last_fullsized":"2023-09-30T22:14:52.551837+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549622+0000","last_change":"2023-09-30T22:14:02.076131+0000","last_active":"2023-09-30T22:14:52.549622+0000","last_peered":"2023-09-30T22:14:52.549622+0000","last_clean":"2023-09-30T22:14:52.549622+0000","last_became_active":"2023-09-30T22:14:02.075850+0000","last_became_peered":"2023-09-30T22:14:02.075850+0000","last_unstale":"2023-09-30T22:14:52.549622+0000","last_undegraded":"2023-09-30T22:14:52.549622+0000","last_fullsized":"2023-09-30T22:14:52.549622+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733087+0000","last_change":"2023-09-30T22:14:02.085210+0000","last_active":"2023-09-30T22:14:52.733087+0000","last_peered":"2023-09-30T22:14:52.733087+0000","last_clean":"2023-09-30T22:14:52.733087+0000","last_became_active":"2023-09-30T22:14:02.084320+0000","last_became_peered":"2023-09-30T22:14:02.084320+0000","last_unstale":"2023-09-30T22:14:52.733087+0000","last_undegraded":"2023-09-30T22:14:52.733087+0000","last_fullsized":"2023-09-30T22:14:52.733087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.557709+0000","last_change":"2023-09-30T22:14:52.555620+0000","last_active":"2023-09-30T22:14:52.557709+0000","last_peered":"2023-09-30T22:14:52.557709+0000","last_clean":"2023-09-30T22:14:52.557709+0000","last_became_active":"2023-09-30T22:14:52.555260+0000","last_became_peered":"2023-09-30T22:14:52.555260+0000","last_unstale":"2023-09-30T22:14:52.557709+0000","last_undegraded":"2023-09-30T22:14:52.557709+0000","last_fullsized":"2023-09-30T22:14:52.557709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.736927+0000","last_change":"2023-09-30T22:14:52.735020+0000","last_active":"2023-09-30T22:14:52.736927+0000","last_peered":"2023-09-30T22:14:52.736927+0000","last_clean":"2023-09-30T22:14:52.736927+0000","last_became_active":"2023-09-30T22:14:52.734697+0000","last_became_peered":"2023-09-30T22:14:52.734697+0000","last_unstale":"2023-09-30T22:14:52.736927+0000","last_undegraded":"2023-09-30T22:14:52.736927+0000","last_fullsized":"2023-09-30T22:14:52.736927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552819+0000","last_change":"2023-09-30T22:14:02.077989+0000","last_active":"2023-09-30T22:14:52.552819+0000","last_peered":"2023-09-30T22:14:52.552819+0000","last_clean":"2023-09-30T22:14:52.552819+0000","last_became_active":"2023-09-30T22:14:02.071401+0000","last_became_peered":"2023-09-30T22:14:02.071401+0000","last_unstale":"2023-09-30T22:14:52.552819+0000","last_undegraded":"2023-09-30T22:14:52.552819+0000","last_fullsized":"2023-09-30T22:14:52.552819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.556563+0000","last_change":"2023-09-30T22:14:52.554267+0000","last_active":"2023-09-30T22:14:52.556563+0000","last_peered":"2023-09-30T22:14:52.556563+0000","last_clean":"2023-09-30T22:14:52.556563+0000","last_became_active":"2023-09-30T22:14:52.553993+0000","last_became_peered":"2023-09-30T22:14:52.553993+0000","last_unstale":"2023-09-30T22:14:52.556563+0000","last_undegraded":"2023-09-30T22:14:52.556563+0000","last_fullsized":"2023-09-30T22:14:52.556563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552916+0000","last_change":"2023-09-30T22:14:01.697398+0000","last_active":"2023-09-30T22:14:52.552916+0000","last_peered":"2023-09-30T22:14:52.552916+0000","last_clean":"2023-09-30T22:14:52.552916+0000","last_became_active":"2023-09-30T22:14:01.694824+0000","last_became_peered":"2023-09-30T22:14:01.694824+0000","last_unstale":"2023-09-30T22:14:52.552916+0000","last_undegraded":"2023-09-30T22:14:52.552916+0000","last_fullsized":"2023-09-30T22:14:52.552916+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733491+0000","last_change":"2023-09-30T22:14:02.075771+0000","last_active":"2023-09-30T22:14:52.733491+0000","last_peered":"2023-09-30T22:14:52.733491+0000","last_clean":"2023-09-30T22:14:52.733491+0000","last_became_active":"2023-09-30T22:14:02.074872+0000","last_became_peered":"2023-09-30T22:14:02.074872+0000","last_unstale":"2023-09-30T22:14:52.733491+0000","last_undegraded":"2023-09-30T22:14:52.733491+0000","last_fullsized":"2023-09-30T22:14:52.733491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549456+0000","last_change":"2023-09-30T22:14:01.702319+0000","last_active":"2023-09-30T22:14:52.549456+0000","last_peered":"2023-09-30T22:14:52.549456+0000","last_clean":"2023-09-30T22:14:52.549456+0000","last_became_active":"2023-09-30T22:14:01.702020+0000","last_became_peered":"2023-09-30T22:14:01.702020+0000","last_unstale":"2023-09-30T22:14:52.549456+0000","last_undegraded":"2023-09-30T22:14:52.549456+0000","last_fullsized":"2023-09-30T22:14:52.549456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551168+0000","last_change":"2023-09-30T22:14:02.079713+0000","last_active":"2023-09-30T22:14:52.551168+0000","last_peered":"2023-09-30T22:14:52.551168+0000","last_clean":"2023-09-30T22:14:52.551168+0000","last_became_active":"2023-09-30T22:14:02.079318+0000","last_became_peered":"2023-09-30T22:14:02.079318+0000","last_unstale":"2023-09-30T22:14:52.551168+0000","last_undegraded":"2023-09-30T22:14:52.551168+0000","last_fullsized":"2023-09-30T22:14:52.551168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733884+0000","last_change":"2023-09-30T22:14:01.707097+0000","last_active":"2023-09-30T22:14:52.733884+0000","last_peered":"2023-09-30T22:14:52.733884+0000","last_clean":"2023-09-30T22:14:52.733884+0000","last_became_active":"2023-09-30T22:14:01.706849+0000","last_became_peered":"2023-09-30T22:14:01.706849+0000","last_unstale":"2023-09-30T22:14:52.733884+0000","last_undegraded":"2023-09-30T22:14:52.733884+0000","last_fullsized":"2023-09-30T22:14:52.733884+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.556893+0000","last_change":"2023-09-30T22:14:52.554552+0000","last_active":"2023-09-30T22:14:52.556893+0000","last_peered":"2023-09-30T22:14:52.556893+0000","last_clean":"2023-09-30T22:14:52.556893+0000","last_became_active":"2023-09-30T22:14:52.554144+0000","last_became_peered":"2023-09-30T22:14:52.554144+0000","last_unstale":"2023-09-30T22:14:52.556893+0000","last_undegraded":"2023-09-30T22:14:52.556893+0000","last_fullsized":"2023-09-30T22:14:52.556893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551593+0000","last_change":"2023-09-30T22:14:01.698007+0000","last_active":"2023-09-30T22:14:52.551593+0000","last_peered":"2023-09-30T22:14:52.551593+0000","last_clean":"2023-09-30T22:14:52.551593+0000","last_became_active":"2023-09-30T22:14:01.697731+0000","last_became_peered":"2023-09-30T22:14:01.697731+0000","last_unstale":"2023-09-30T22:14:52.551593+0000","last_undegraded":"2023-09-30T22:14:52.551593+0000","last_fullsized":"2023-09-30T22:14:52.551593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733979+0000","last_change":"2023-09-30T22:14:02.082853+0000","last_active":"2023-09-30T22:14:52.733979+0000","last_peered":"2023-09-30T22:14:52.733979+0000","last_clean":"2023-09-30T22:14:52.733979+0000","last_became_active":"2023-09-30T22:14:02.082597+0000","last_became_peered":"2023-09-30T22:14:02.082597+0000","last_unstale":"2023-09-30T22:14:52.733979+0000","last_undegraded":"2023-09-30T22:14:52.733979+0000","last_fullsized":"2023-09-30T22:14:52.733979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552549+0000","last_change":"2023-09-30T22:14:01.700657+0000","last_active":"2023-09-30T22:14:52.552549+0000","last_peered":"2023-09-30T22:14:52.552549+0000","last_clean":"2023-09-30T22:14:52.552549+0000","last_became_active":"2023-09-30T22:14:01.699633+0000","last_became_peered":"2023-09-30T22:14:01.699633+0000","last_unstale":"2023-09-30T22:14:52.552549+0000","last_undegraded":"2023-09-30T22:14:52.552549+0000","last_fullsized":"2023-09-30T22:14:52.552549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734087+0000","last_change":"2023-09-30T22:14:02.085416+0000","last_active":"2023-09-30T22:14:52.734087+0000","last_peered":"2023-09-30T22:14:52.734087+0000","last_clean":"2023-09-30T22:14:52.734087+0000","last_became_active":"2023-09-30T22:14:02.084970+0000","last_became_peered":"2023-09-30T22:14:02.084970+0000","last_unstale":"2023-09-30T22:14:52.734087+0000","last_undegraded":"2023-09-30T22:14:52.734087+0000","last_fullsized":"2023-09-30T22:14:52.734087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549708+0000","last_change":"2023-09-30T22:14:01.701075+0000","last_active":"2023-09-30T22:14:52.549708+0000","last_peered":"2023-09-30T22:14:52.549708+0000","last_clean":"2023-09-30T22:14:52.549708+0000","last_became_active":"2023-09-30T22:14:01.700806+0000","last_became_peered":"2023-09-30T22:14:01.700806+0000","last_unstale":"2023-09-30T22:14:52.549708+0000","last_undegraded":"2023-09-30T22:14:52.549708+0000","last_fullsized":"2023-09-30T22:14:52.549708+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552081+0000","last_change":"2023-09-30T22:14:02.074078+0000","last_active":"2023-09-30T22:14:52.552081+0000","last_peered":"2023-09-30T22:14:52.552081+0000","last_clean":"2023-09-30T22:14:52.552081+0000","last_became_active":"2023-09-30T22:14:02.073804+0000","last_became_peered":"2023-09-30T22:14:02.073804+0000","last_unstale":"2023-09-30T22:14:52.552081+0000","last_undegraded":"2023-09-30T22:14:52.552081+0000","last_fullsized":"2023-09-30T22:14:52.552081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'12","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.476400+0000","last_change":"2023-09-30T22:14:01.711172+0000","last_active":"2023-09-30T22:16:19.476400+0000","last_peered":"2023-09-30T22:16:19.476400+0000","last_clean":"2023-09-30T22:16:19.476400+0000","last_became_active":"2023-09-30T22:14:01.710926+0000","last_became_peered":"2023-09-30T22:14:01.710926+0000","last_unstale":"2023-09-30T22:16:19.476400+0000","last_undegraded":"2023-09-30T22:16:19.476400+0000","last_fullsized":"2023-09-30T22:16:19.476400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458362+0000","last_change":"2023-09-30T22:14:02.087193+0000","last_active":"2023-09-30T22:16:19.458362+0000","last_peered":"2023-09-30T22:16:19.458362+0000","last_clean":"2023-09-30T22:16:19.458362+0000","last_became_active":"2023-09-30T22:14:02.086273+0000","last_became_peered":"2023-09-30T22:14:02.086273+0000","last_unstale":"2023-09-30T22:16:19.458362+0000","last_undegraded":"2023-09-30T22:16:19.458362+0000","last_fullsized":"2023-09-30T22:16:19.458362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734112+0000","last_change":"2023-09-30T22:14:01.706560+0000","last_active":"2023-09-30T22:14:52.734112+0000","last_peered":"2023-09-30T22:14:52.734112+0000","last_clean":"2023-09-30T22:14:52.734112+0000","last_became_active":"2023-09-30T22:14:01.706330+0000","last_became_peered":"2023-09-30T22:14:01.706330+0000","last_unstale":"2023-09-30T22:14:52.734112+0000","last_undegraded":"2023-09-30T22:14:52.734112+0000","last_fullsized":"2023-09-30T22:14:52.734112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733307+0000","last_change":"2023-09-30T22:14:02.084139+0000","last_active":"2023-09-30T22:14:52.733307+0000","last_peered":"2023-09-30T22:14:52.733307+0000","last_clean":"2023-09-30T22:14:52.733307+0000","last_became_active":"2023-09-30T22:14:02.080986+0000","last_became_peered":"2023-09-30T22:14:02.080986+0000","last_unstale":"2023-09-30T22:14:52.733307+0000","last_undegraded":"2023-09-30T22:14:52.733307+0000","last_fullsized":"2023-09-30T22:14:52.733307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549677+0000","last_change":"2023-09-30T22:14:01.698141+0000","last_active":"2023-09-30T22:14:52.549677+0000","last_peered":"2023-09-30T22:14:52.549677+0000","last_clean":"2023-09-30T22:14:52.549677+0000","last_became_active":"2023-09-30T22:14:01.697885+0000","last_became_peered":"2023-09-30T22:14:01.697885+0000","last_unstale":"2023-09-30T22:14:52.549677+0000","last_undegraded":"2023-09-30T22:14:52.549677+0000","last_fullsized":"2023-09-30T22:14:52.549677+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733580+0000","last_change":"2023-09-30T22:14:02.075925+0000","last_active":"2023-09-30T22:14:52.733580+0000","last_peered":"2023-09-30T22:14:52.733580+0000","last_clean":"2023-09-30T22:14:52.733580+0000","last_became_active":"2023-09-30T22:14:02.075137+0000","last_became_peered":"2023-09-30T22:14:02.075137+0000","last_unstale":"2023-09-30T22:14:52.733580+0000","last_undegraded":"2023-09-30T22:14:52.733580+0000","last_fullsized":"2023-09-30T22:14:52.733580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734069+0000","last_change":"2023-09-30T22:14:01.712299+0000","last_active":"2023-09-30T22:14:52.734069+0000","last_peered":"2023-09-30T22:14:52.734069+0000","last_clean":"2023-09-30T22:14:52.734069+0000","last_became_active":"2023-09-30T22:14:01.711791+0000","last_became_peered":"2023-09-30T22:14:01.711791+0000","last_unstale":"2023-09-30T22:14:52.734069+0000","last_undegraded":"2023-09-30T22:14:52.734069+0000","last_fullsized":"2023-09-30T22:14:52.734069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551293+0000","last_change":"2023-09-30T22:14:02.084033+0000","last_active":"2023-09-30T22:14:52.551293+0000","last_peered":"2023-09-30T22:14:52.551293+0000","last_clean":"2023-09-30T22:14:52.551293+0000","last_became_active":"2023-09-30T22:14:02.083764+0000","last_became_peered":"2023-09-30T22:14:02.083764+0000","last_unstale":"2023-09-30T22:14:52.551293+0000","last_undegraded":"2023-09-30T22:14:52.551293+0000","last_fullsized":"2023-09-30T22:14:52.551293+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551632+0000","last_change":"2023-09-30T22:14:01.706777+0000","last_active":"2023-09-30T22:14:52.551632+0000","last_peered":"2023-09-30T22:14:52.551632+0000","last_clean":"2023-09-30T22:14:52.551632+0000","last_became_active":"2023-09-30T22:14:01.706479+0000","last_became_peered":"2023-09-30T22:14:01.706479+0000","last_unstale":"2023-09-30T22:14:52.551632+0000","last_undegraded":"2023-09-30T22:14:52.551632+0000","last_fullsized":"2023-09-30T22:14:52.551632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733886+0000","last_change":"2023-09-30T22:14:02.084399+0000","last_active":"2023-09-30T22:14:52.733886+0000","last_peered":"2023-09-30T22:14:52.733886+0000","last_clean":"2023-09-30T22:14:52.733886+0000","last_became_active":"2023-09-30T22:14:02.084137+0000","last_became_peered":"2023-09-30T22:14:02.084137+0000","last_unstale":"2023-09-30T22:14:52.733886+0000","last_undegraded":"2023-09-30T22:14:52.733886+0000","last_fullsized":"2023-09-30T22:14:52.733886+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549339+0000","last_change":"2023-09-30T22:14:01.702685+0000","last_active":"2023-09-30T22:14:52.549339+0000","last_peered":"2023-09-30T22:14:52.549339+0000","last_clean":"2023-09-30T22:14:52.549339+0000","last_became_active":"2023-09-30T22:14:01.702083+0000","last_became_peered":"2023-09-30T22:14:01.702083+0000","last_unstale":"2023-09-30T22:14:52.549339+0000","last_undegraded":"2023-09-30T22:14:52.549339+0000","last_fullsized":"2023-09-30T22:14:52.549339+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458027+0000","last_change":"2023-09-30T22:14:52.553897+0000","last_active":"2023-09-30T22:16:19.458027+0000","last_peered":"2023-09-30T22:16:19.458027+0000","last_clean":"2023-09-30T22:16:19.458027+0000","last_became_active":"2023-09-30T22:14:52.553612+0000","last_became_peered":"2023-09-30T22:14:52.553612+0000","last_unstale":"2023-09-30T22:16:19.458027+0000","last_undegraded":"2023-09-30T22:16:19.458027+0000","last_fullsized":"2023-09-30T22:16:19.458027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552617+0000","last_change":"2023-09-30T22:14:01.706748+0000","last_active":"2023-09-30T22:14:52.552617+0000","last_peered":"2023-09-30T22:14:52.552617+0000","last_clean":"2023-09-30T22:14:52.552617+0000","last_became_active":"2023-09-30T22:14:01.706472+0000","last_became_peered":"2023-09-30T22:14:01.706472+0000","last_unstale":"2023-09-30T22:14:52.552617+0000","last_undegraded":"2023-09-30T22:14:52.552617+0000","last_fullsized":"2023-09-30T22:14:52.552617+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549372+0000","last_change":"2023-09-30T22:14:02.070587+0000","last_active":"2023-09-30T22:14:52.549372+0000","last_peered":"2023-09-30T22:14:52.549372+0000","last_clean":"2023-09-30T22:14:52.549372+0000","last_became_active":"2023-09-30T22:14:02.070259+0000","last_became_peered":"2023-09-30T22:14:02.070259+0000","last_unstale":"2023-09-30T22:14:52.549372+0000","last_undegraded":"2023-09-30T22:14:52.549372+0000","last_fullsized":"2023-09-30T22:14:52.549372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733668+0000","last_change":"2023-09-30T22:14:01.704603+0000","last_active":"2023-09-30T22:14:52.733668+0000","last_peered":"2023-09-30T22:14:52.733668+0000","last_clean":"2023-09-30T22:14:52.733668+0000","last_became_active":"2023-09-30T22:14:01.704006+0000","last_became_peered":"2023-09-30T22:14:01.704006+0000","last_unstale":"2023-09-30T22:14:52.733668+0000","last_undegraded":"2023-09-30T22:14:52.733668+0000","last_fullsized":"2023-09-30T22:14:52.733668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458369+0000","last_change":"2023-09-30T22:14:02.073527+0000","last_active":"2023-09-30T22:16:19.458369+0000","last_peered":"2023-09-30T22:16:19.458369+0000","last_clean":"2023-09-30T22:16:19.458369+0000","last_became_active":"2023-09-30T22:14:02.072950+0000","last_became_peered":"2023-09-30T22:14:02.072950+0000","last_unstale":"2023-09-30T22:16:19.458369+0000","last_undegraded":"2023-09-30T22:16:19.458369+0000","last_fullsized":"2023-09-30T22:16:19.458369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550333+0000","last_change":"2023-09-30T22:14:01.701736+0000","last_active":"2023-09-30T22:14:52.550333+0000","last_peered":"2023-09-30T22:14:52.550333+0000","last_clean":"2023-09-30T22:14:52.550333+0000","last_became_active":"2023-09-30T22:14:01.701471+0000","last_became_peered":"2023-09-30T22:14:01.701471+0000","last_unstale":"2023-09-30T22:14:52.550333+0000","last_undegraded":"2023-09-30T22:14:52.550333+0000","last_fullsized":"2023-09-30T22:14:52.550333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550088+0000","last_change":"2023-09-30T22:14:02.072624+0000","last_active":"2023-09-30T22:14:52.550088+0000","last_peered":"2023-09-30T22:14:52.550088+0000","last_clean":"2023-09-30T22:14:52.550088+0000","last_became_active":"2023-09-30T22:14:02.072363+0000","last_became_peered":"2023-09-30T22:14:02.072363+0000","last_unstale":"2023-09-30T22:14:52.550088+0000","last_undegraded":"2023-09-30T22:14:52.550088+0000","last_fullsized":"2023-09-30T22:14:52.550088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733914+0000","last_change":"2023-09-30T22:14:01.709398+0000","last_active":"2023-09-30T22:14:52.733914+0000","last_peered":"2023-09-30T22:14:52.733914+0000","last_clean":"2023-09-30T22:14:52.733914+0000","last_became_active":"2023-09-30T22:14:01.709110+0000","last_became_peered":"2023-09-30T22:14:01.709110+0000","last_unstale":"2023-09-30T22:14:52.733914+0000","last_undegraded":"2023-09-30T22:14:52.733914+0000","last_fullsized":"2023-09-30T22:14:52.733914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550287+0000","last_change":"2023-09-30T22:14:02.074887+0000","last_active":"2023-09-30T22:14:52.550287+0000","last_peered":"2023-09-30T22:14:52.550287+0000","last_clean":"2023-09-30T22:14:52.550287+0000","last_became_active":"2023-09-30T22:14:02.074631+0000","last_became_peered":"2023-09-30T22:14:02.074631+0000","last_unstale":"2023-09-30T22:14:52.550287+0000","last_undegraded":"2023-09-30T22:14:52.550287+0000","last_fullsized":"2023-09-30T22:14:52.550287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733461+0000","last_change":"2023-09-30T22:14:01.707879+0000","last_active":"2023-09-30T22:14:52.733461+0000","last_peered":"2023-09-30T22:14:52.733461+0000","last_clean":"2023-09-30T22:14:52.733461+0000","last_became_active":"2023-09-30T22:14:01.707594+0000","last_became_peered":"2023-09-30T22:14:01.707594+0000","last_unstale":"2023-09-30T22:14:52.733461+0000","last_undegraded":"2023-09-30T22:14:52.733461+0000","last_fullsized":"2023-09-30T22:14:52.733461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733614+0000","last_change":"2023-09-30T22:14:02.076079+0000","last_active":"2023-09-30T22:14:52.733614+0000","last_peered":"2023-09-30T22:14:52.733614+0000","last_clean":"2023-09-30T22:14:52.733614+0000","last_became_active":"2023-09-30T22:14:02.075408+0000","last_became_peered":"2023-09-30T22:14:02.075408+0000","last_unstale":"2023-09-30T22:14:52.733614+0000","last_undegraded":"2023-09-30T22:14:52.733614+0000","last_fullsized":"2023-09-30T22:14:52.733614+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733270+0000","last_change":"2023-09-30T22:14:01.711605+0000","last_active":"2023-09-30T22:14:52.733270+0000","last_peered":"2023-09-30T22:14:52.733270+0000","last_clean":"2023-09-30T22:14:52.733270+0000","last_became_active":"2023-09-30T22:14:01.711375+0000","last_became_peered":"2023-09-30T22:14:01.711375+0000","last_unstale":"2023-09-30T22:14:52.733270+0000","last_undegraded":"2023-09-30T22:14:52.733270+0000","last_fullsized":"2023-09-30T22:14:52.733270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552671+0000","last_change":"2023-09-30T22:14:02.088858+0000","last_active":"2023-09-30T22:14:52.552671+0000","last_peered":"2023-09-30T22:14:52.552671+0000","last_clean":"2023-09-30T22:14:52.552671+0000","last_became_active":"2023-09-30T22:14:02.088583+0000","last_became_peered":"2023-09-30T22:14:02.088583+0000","last_unstale":"2023-09-30T22:14:52.552671+0000","last_undegraded":"2023-09-30T22:14:52.552671+0000","last_fullsized":"2023-09-30T22:14:52.552671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733195+0000","last_change":"2023-09-30T22:14:01.699847+0000","last_active":"2023-09-30T22:14:52.733195+0000","last_peered":"2023-09-30T22:14:52.733195+0000","last_clean":"2023-09-30T22:14:52.733195+0000","last_became_active":"2023-09-30T22:14:01.699587+0000","last_became_peered":"2023-09-30T22:14:01.699587+0000","last_unstale":"2023-09-30T22:14:52.733195+0000","last_undegraded":"2023-09-30T22:14:52.733195+0000","last_fullsized":"2023-09-30T22:14:52.733195+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552253+0000","last_change":"2023-09-30T22:14:02.094005+0000","last_active":"2023-09-30T22:14:52.552253+0000","last_peered":"2023-09-30T22:14:52.552253+0000","last_clean":"2023-09-30T22:14:52.552253+0000","last_became_active":"2023-09-30T22:14:02.093632+0000","last_became_peered":"2023-09-30T22:14:02.093632+0000","last_unstale":"2023-09-30T22:14:52.552253+0000","last_undegraded":"2023-09-30T22:14:52.552253+0000","last_fullsized":"2023-09-30T22:14:52.552253+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552302+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:14:52.552302+0000","last_peered":"2023-09-30T22:14:52.552302+0000","last_clean":"2023-09-30T22:14:52.552302+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:14:52.552302+0000","last_undegraded":"2023-09-30T22:14:52.552302+0000","last_fullsized":"2023-09-30T22:14:52.552302+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549808+0000","last_change":"2023-09-30T22:14:01.702822+0000","last_active":"2023-09-30T22:14:52.549808+0000","last_peered":"2023-09-30T22:14:52.549808+0000","last_clean":"2023-09-30T22:14:52.549808+0000","last_became_active":"2023-09-30T22:14:01.702530+0000","last_became_peered":"2023-09-30T22:14:01.702530+0000","last_unstale":"2023-09-30T22:14:52.549808+0000","last_undegraded":"2023-09-30T22:14:52.549808+0000","last_fullsized":"2023-09-30T22:14:52.549808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551989+0000","last_change":"2023-09-30T22:14:02.084698+0000","last_active":"2023-09-30T22:14:52.551989+0000","last_peered":"2023-09-30T22:14:52.551989+0000","last_clean":"2023-09-30T22:14:52.551989+0000","last_became_active":"2023-09-30T22:14:02.084413+0000","last_became_peered":"2023-09-30T22:14:02.084413+0000","last_unstale":"2023-09-30T22:14:52.551989+0000","last_undegraded":"2023-09-30T22:14:52.551989+0000","last_fullsized":"2023-09-30T22:14:52.551989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733559+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:14:52.733559+0000","last_peered":"2023-09-30T22:14:52.733559+0000","last_clean":"2023-09-30T22:14:52.733559+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:14:52.733559+0000","last_undegraded":"2023-09-30T22:14:52.733559+0000","last_fullsized":"2023-09-30T22:14:52.733559+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549896+0000","last_change":"2023-09-30T22:14:01.704198+0000","last_active":"2023-09-30T22:14:52.549896+0000","last_peered":"2023-09-30T22:14:52.549896+0000","last_clean":"2023-09-30T22:14:52.549896+0000","last_became_active":"2023-09-30T22:14:01.701691+0000","last_became_peered":"2023-09-30T22:14:01.701691+0000","last_unstale":"2023-09-30T22:14:52.549896+0000","last_undegraded":"2023-09-30T22:14:52.549896+0000","last_fullsized":"2023-09-30T22:14:52.549896+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552005+0000","last_change":"2023-09-30T22:14:02.076717+0000","last_active":"2023-09-30T22:14:52.552005+0000","last_peered":"2023-09-30T22:14:52.552005+0000","last_clean":"2023-09-30T22:14:52.552005+0000","last_became_active":"2023-09-30T22:14:02.076119+0000","last_became_peered":"2023-09-30T22:14:02.076119+0000","last_unstale":"2023-09-30T22:14:52.552005+0000","last_undegraded":"2023-09-30T22:14:52.552005+0000","last_fullsized":"2023-09-30T22:14:52.552005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458314+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:16:19.458314+0000","last_peered":"2023-09-30T22:16:19.458314+0000","last_clean":"2023-09-30T22:16:19.458314+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:16:19.458314+0000","last_undegraded":"2023-09-30T22:16:19.458314+0000","last_fullsized":"2023-09-30T22:16:19.458314+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551062+0000","last_change":"2023-09-30T22:14:01.701431+0000","last_active":"2023-09-30T22:14:52.551062+0000","last_peered":"2023-09-30T22:14:52.551062+0000","last_clean":"2023-09-30T22:14:52.551062+0000","last_became_active":"2023-09-30T22:14:01.698771+0000","last_became_peered":"2023-09-30T22:14:01.698771+0000","last_unstale":"2023-09-30T22:14:52.551062+0000","last_undegraded":"2023-09-30T22:14:52.551062+0000","last_fullsized":"2023-09-30T22:14:52.551062+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552255+0000","last_change":"2023-09-30T22:14:02.085435+0000","last_active":"2023-09-30T22:14:52.552255+0000","last_peered":"2023-09-30T22:14:52.552255+0000","last_clean":"2023-09-30T22:14:52.552255+0000","last_became_active":"2023-09-30T22:14:02.085124+0000","last_became_peered":"2023-09-30T22:14:02.085124+0000","last_unstale":"2023-09-30T22:14:52.552255+0000","last_undegraded":"2023-09-30T22:14:52.552255+0000","last_fullsized":"2023-09-30T22:14:52.552255+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734015+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:14:52.734015+0000","last_peered":"2023-09-30T22:14:52.734015+0000","last_clean":"2023-09-30T22:14:52.734015+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:14:52.734015+0000","last_undegraded":"2023-09-30T22:14:52.734015+0000","last_fullsized":"2023-09-30T22:14:52.734015+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550904+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:14:52.550904+0000","last_peered":"2023-09-30T22:14:52.550904+0000","last_clean":"2023-09-30T22:14:52.550904+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:14:52.550904+0000","last_undegraded":"2023-09-30T22:14:52.550904+0000","last_fullsized":"2023-09-30T22:14:52.550904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733334+0000","last_change":"2023-09-30T22:14:01.712014+0000","last_active":"2023-09-30T22:14:52.733334+0000","last_peered":"2023-09-30T22:14:52.733334+0000","last_clean":"2023-09-30T22:14:52.733334+0000","last_became_active":"2023-09-30T22:14:01.711770+0000","last_became_peered":"2023-09-30T22:14:01.711770+0000","last_unstale":"2023-09-30T22:14:52.733334+0000","last_undegraded":"2023-09-30T22:14:52.733334+0000","last_fullsized":"2023-09-30T22:14:52.733334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734010+0000","last_change":"2023-09-30T22:14:02.080800+0000","last_active":"2023-09-30T22:14:52.734010+0000","last_peered":"2023-09-30T22:14:52.734010+0000","last_clean":"2023-09-30T22:14:52.734010+0000","last_became_active":"2023-09-30T22:14:02.080440+0000","last_became_peered":"2023-09-30T22:14:02.080440+0000","last_unstale":"2023-09-30T22:14:52.734010+0000","last_undegraded":"2023-09-30T22:14:52.734010+0000","last_fullsized":"2023-09-30T22:14:52.734010+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733775+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:14:52.733775+0000","last_peered":"2023-09-30T22:14:52.733775+0000","last_clean":"2023-09-30T22:14:52.733775+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:14:52.733775+0000","last_undegraded":"2023-09-30T22:14:52.733775+0000","last_fullsized":"2023-09-30T22:14:52.733775+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549535+0000","last_change":"2023-09-30T22:14:01.694407+0000","last_active":"2023-09-30T22:14:52.549535+0000","last_peered":"2023-09-30T22:14:52.549535+0000","last_clean":"2023-09-30T22:14:52.549535+0000","last_became_active":"2023-09-30T22:14:01.694022+0000","last_became_peered":"2023-09-30T22:14:01.694022+0000","last_unstale":"2023-09-30T22:14:52.549535+0000","last_undegraded":"2023-09-30T22:14:52.549535+0000","last_fullsized":"2023-09-30T22:14:52.549535+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551372+0000","last_change":"2023-09-30T22:14:02.079540+0000","last_active":"2023-09-30T22:14:52.551372+0000","last_peered":"2023-09-30T22:14:52.551372+0000","last_clean":"2023-09-30T22:14:52.551372+0000","last_became_active":"2023-09-30T22:14:02.079016+0000","last_became_peered":"2023-09-30T22:14:02.079016+0000","last_unstale":"2023-09-30T22:14:52.551372+0000","last_undegraded":"2023-09-30T22:14:52.551372+0000","last_fullsized":"2023-09-30T22:14:52.551372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733752+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:14:52.733752+0000","last_peered":"2023-09-30T22:14:52.733752+0000","last_clean":"2023-09-30T22:14:52.733752+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:14:52.733752+0000","last_undegraded":"2023-09-30T22:14:52.733752+0000","last_fullsized":"2023-09-30T22:14:52.733752+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551911+0000","last_change":"2023-09-30T22:14:01.695717+0000","last_active":"2023-09-30T22:14:52.551911+0000","last_peered":"2023-09-30T22:14:52.551911+0000","last_clean":"2023-09-30T22:14:52.551911+0000","last_became_active":"2023-09-30T22:14:01.694990+0000","last_became_peered":"2023-09-30T22:14:01.694990+0000","last_unstale":"2023-09-30T22:14:52.551911+0000","last_undegraded":"2023-09-30T22:14:52.551911+0000","last_fullsized":"2023-09-30T22:14:52.551911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550011+0000","last_change":"2023-09-30T22:14:02.068852+0000","last_active":"2023-09-30T22:14:52.550011+0000","last_peered":"2023-09-30T22:14:52.550011+0000","last_clean":"2023-09-30T22:14:52.550011+0000","last_became_active":"2023-09-30T22:14:02.068531+0000","last_became_peered":"2023-09-30T22:14:02.068531+0000","last_unstale":"2023-09-30T22:14:52.550011+0000","last_undegraded":"2023-09-30T22:14:52.550011+0000","last_fullsized":"2023-09-30T22:14:52.550011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733800+0000","last_change":"2023-09-30T22:14:01.700959+0000","last_active":"2023-09-30T22:14:52.733800+0000","last_peered":"2023-09-30T22:14:52.733800+0000","last_clean":"2023-09-30T22:14:52.733800+0000","last_became_active":"2023-09-30T22:14:01.700568+0000","last_became_peered":"2023-09-30T22:14:01.700568+0000","last_unstale":"2023-09-30T22:14:52.733800+0000","last_undegraded":"2023-09-30T22:14:52.733800+0000","last_fullsized":"2023-09-30T22:14:52.733800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552770+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:14:52.552770+0000","last_peered":"2023-09-30T22:14:52.552770+0000","last_clean":"2023-09-30T22:14:52.552770+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:14:52.552770+0000","last_undegraded":"2023-09-30T22:14:52.552770+0000","last_fullsized":"2023-09-30T22:14:52.552770+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733297+0000","last_change":"2023-09-30T22:14:02.081943+0000","last_active":"2023-09-30T22:14:52.733297+0000","last_peered":"2023-09-30T22:14:52.733297+0000","last_clean":"2023-09-30T22:14:52.733297+0000","last_became_active":"2023-09-30T22:14:02.080899+0000","last_became_peered":"2023-09-30T22:14:02.080899+0000","last_unstale":"2023-09-30T22:14:52.733297+0000","last_undegraded":"2023-09-30T22:14:52.733297+0000","last_fullsized":"2023-09-30T22:14:52.733297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733844+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:14:52.733844+0000","last_peered":"2023-09-30T22:14:52.733844+0000","last_clean":"2023-09-30T22:14:52.733844+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:14:52.733844+0000","last_undegraded":"2023-09-30T22:14:52.733844+0000","last_fullsized":"2023-09-30T22:14:52.733844+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551086+0000","last_change":"2023-09-30T22:14:01.706704+0000","last_active":"2023-09-30T22:14:52.551086+0000","last_peered":"2023-09-30T22:14:52.551086+0000","last_clean":"2023-09-30T22:14:52.551086+0000","last_became_active":"2023-09-30T22:14:01.706387+0000","last_became_peered":"2023-09-30T22:14:01.706387+0000","last_unstale":"2023-09-30T22:14:52.551086+0000","last_undegraded":"2023-09-30T22:14:52.551086+0000","last_fullsized":"2023-09-30T22:14:52.551086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458167+0000","last_change":"2023-09-30T22:14:02.088343+0000","last_active":"2023-09-30T22:16:19.458167+0000","last_peered":"2023-09-30T22:16:19.458167+0000","last_clean":"2023-09-30T22:16:19.458167+0000","last_became_active":"2023-09-30T22:14:02.088142+0000","last_became_peered":"2023-09-30T22:14:02.088142+0000","last_unstale":"2023-09-30T22:16:19.458167+0000","last_undegraded":"2023-09-30T22:16:19.458167+0000","last_fullsized":"2023-09-30T22:16:19.458167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549939+0000","last_change":"2023-09-30T22:14:01.702235+0000","last_active":"2023-09-30T22:14:52.549939+0000","last_peered":"2023-09-30T22:14:52.549939+0000","last_clean":"2023-09-30T22:14:52.549939+0000","last_became_active":"2023-09-30T22:14:01.701972+0000","last_became_peered":"2023-09-30T22:14:01.701972+0000","last_unstale":"2023-09-30T22:14:52.549939+0000","last_undegraded":"2023-09-30T22:14:52.549939+0000","last_fullsized":"2023-09-30T22:14:52.549939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550032+0000","last_change":"2023-09-30T22:14:02.078694+0000","last_active":"2023-09-30T22:14:52.550032+0000","last_peered":"2023-09-30T22:14:52.550032+0000","last_clean":"2023-09-30T22:14:52.550032+0000","last_became_active":"2023-09-30T22:14:02.078435+0000","last_became_peered":"2023-09-30T22:14:02.078435+0000","last_unstale":"2023-09-30T22:14:52.550032+0000","last_undegraded":"2023-09-30T22:14:52.550032+0000","last_fullsized":"2023-09-30T22:14:52.550032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552103+0000","last_change":"2023-09-30T22:14:01.709428+0000","last_active":"2023-09-30T22:14:52.552103+0000","last_peered":"2023-09-30T22:14:52.552103+0000","last_clean":"2023-09-30T22:14:52.552103+0000","last_became_active":"2023-09-30T22:14:01.709089+0000","last_became_peered":"2023-09-30T22:14:01.709089+0000","last_unstale":"2023-09-30T22:14:52.552103+0000","last_undegraded":"2023-09-30T22:14:52.552103+0000","last_fullsized":"2023-09-30T22:14:52.552103+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458369+0000","last_change":"2023-09-30T22:14:02.087499+0000","last_active":"2023-09-30T22:16:19.458369+0000","last_peered":"2023-09-30T22:16:19.458369+0000","last_clean":"2023-09-30T22:16:19.458369+0000","last_became_active":"2023-09-30T22:14:02.086941+0000","last_became_peered":"2023-09-30T22:14:02.086941+0000","last_unstale":"2023-09-30T22:16:19.458369+0000","last_undegraded":"2023-09-30T22:16:19.458369+0000","last_fullsized":"2023-09-30T22:16:19.458369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733854+0000","last_change":"2023-09-30T22:14:01.709677+0000","last_active":"2023-09-30T22:14:52.733854+0000","last_peered":"2023-09-30T22:14:52.733854+0000","last_clean":"2023-09-30T22:14:52.733854+0000","last_became_active":"2023-09-30T22:14:01.709115+0000","last_became_peered":"2023-09-30T22:14:01.709115+0000","last_unstale":"2023-09-30T22:14:52.733854+0000","last_undegraded":"2023-09-30T22:14:52.733854+0000","last_fullsized":"2023-09-30T22:14:52.733854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458278+0000","last_change":"2023-09-30T22:14:02.073683+0000","last_active":"2023-09-30T22:16:19.458278+0000","last_peered":"2023-09-30T22:16:19.458278+0000","last_clean":"2023-09-30T22:16:19.458278+0000","last_became_active":"2023-09-30T22:14:02.073269+0000","last_became_peered":"2023-09-30T22:14:02.073269+0000","last_unstale":"2023-09-30T22:16:19.458278+0000","last_undegraded":"2023-09-30T22:16:19.458278+0000","last_fullsized":"2023-09-30T22:16:19.458278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734275+0000","last_change":"2023-09-30T22:14:01.711766+0000","last_active":"2023-09-30T22:14:52.734275+0000","last_peered":"2023-09-30T22:14:52.734275+0000","last_clean":"2023-09-30T22:14:52.734275+0000","last_became_active":"2023-09-30T22:14:01.711524+0000","last_became_peered":"2023-09-30T22:14:01.711524+0000","last_unstale":"2023-09-30T22:14:52.734275+0000","last_undegraded":"2023-09-30T22:14:52.734275+0000","last_fullsized":"2023-09-30T22:14:52.734275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734147+0000","last_change":"2023-09-30T22:14:02.082106+0000","last_active":"2023-09-30T22:14:52.734147+0000","last_peered":"2023-09-30T22:14:52.734147+0000","last_clean":"2023-09-30T22:14:52.734147+0000","last_became_active":"2023-09-30T22:14:02.081789+0000","last_became_peered":"2023-09-30T22:14:02.081789+0000","last_unstale":"2023-09-30T22:14:52.734147+0000","last_undegraded":"2023-09-30T22:14:52.734147+0000","last_fullsized":"2023-09-30T22:14:52.734147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552476+0000","last_change":"2023-09-30T22:14:01.702868+0000","last_active":"2023-09-30T22:14:52.552476+0000","last_peered":"2023-09-30T22:14:52.552476+0000","last_clean":"2023-09-30T22:14:52.552476+0000","last_became_active":"2023-09-30T22:14:01.702431+0000","last_became_peered":"2023-09-30T22:14:01.702431+0000","last_unstale":"2023-09-30T22:14:52.552476+0000","last_undegraded":"2023-09-30T22:14:52.552476+0000","last_fullsized":"2023-09-30T22:14:52.552476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458160+0000","last_change":"2023-09-30T22:14:02.081561+0000","last_active":"2023-09-30T22:16:19.458160+0000","last_peered":"2023-09-30T22:16:19.458160+0000","last_clean":"2023-09-30T22:16:19.458160+0000","last_became_active":"2023-09-30T22:14:02.081199+0000","last_became_peered":"2023-09-30T22:14:02.081199+0000","last_unstale":"2023-09-30T22:16:19.458160+0000","last_undegraded":"2023-09-30T22:16:19.458160+0000","last_fullsized":"2023-09-30T22:16:19.458160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551152+0000","last_change":"2023-09-30T22:14:02.079397+0000","last_active":"2023-09-30T22:14:52.551152+0000","last_peered":"2023-09-30T22:14:52.551152+0000","last_clean":"2023-09-30T22:14:52.551152+0000","last_became_active":"2023-09-30T22:14:02.079036+0000","last_became_peered":"2023-09-30T22:14:02.079036+0000","last_unstale":"2023-09-30T22:14:52.551152+0000","last_undegraded":"2023-09-30T22:14:52.551152+0000","last_fullsized":"2023-09-30T22:14:52.551152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733356+0000","last_change":"2023-09-30T22:14:02.081974+0000","last_active":"2023-09-30T22:14:52.733356+0000","last_peered":"2023-09-30T22:14:52.733356+0000","last_clean":"2023-09-30T22:14:52.733356+0000","last_became_active":"2023-09-30T22:14:02.081491+0000","last_became_peered":"2023-09-30T22:14:02.081491+0000","last_unstale":"2023-09-30T22:14:52.733356+0000","last_undegraded":"2023-09-30T22:14:52.733356+0000","last_fullsized":"2023-09-30T22:14:52.733356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549720+0000","last_change":"2023-09-30T22:14:02.067554+0000","last_active":"2023-09-30T22:14:52.549720+0000","last_peered":"2023-09-30T22:14:52.549720+0000","last_clean":"2023-09-30T22:14:52.549720+0000","last_became_active":"2023-09-30T22:14:02.067283+0000","last_became_peered":"2023-09-30T22:14:02.067283+0000","last_unstale":"2023-09-30T22:14:52.549720+0000","last_undegraded":"2023-09-30T22:14:52.549720+0000","last_fullsized":"2023-09-30T22:14:52.549720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734107+0000","last_change":"2023-09-30T22:14:02.082875+0000","last_active":"2023-09-30T22:14:52.734107+0000","last_peered":"2023-09-30T22:14:52.734107+0000","last_clean":"2023-09-30T22:14:52.734107+0000","last_became_active":"2023-09-30T22:14:02.082617+0000","last_became_peered":"2023-09-30T22:14:02.082617+0000","last_unstale":"2023-09-30T22:14:52.734107+0000","last_undegraded":"2023-09-30T22:14:52.734107+0000","last_fullsized":"2023-09-30T22:14:52.734107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552207+0000","last_change":"2023-09-30T22:14:02.076951+0000","last_active":"2023-09-30T22:14:52.552207+0000","last_peered":"2023-09-30T22:14:52.552207+0000","last_clean":"2023-09-30T22:14:52.552207+0000","last_became_active":"2023-09-30T22:14:02.076448+0000","last_became_peered":"2023-09-30T22:14:02.076448+0000","last_unstale":"2023-09-30T22:14:52.552207+0000","last_undegraded":"2023-09-30T22:14:52.552207+0000","last_fullsized":"2023-09-30T22:14:52.552207+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297292,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074548,"statfs":{"total":96636764160,"available":96332337152,"internally_reserved":0,"allocated":200704,"data_stored":67217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.28499999999999998},"min":{"1min":0.20000000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.46999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.224},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.312,"15min":0.311},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.28000000000000003,"15min":0.27800000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.47499999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.29499999999999998,"15min":0.29699999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44600000000000001,"5min":0.497,"15min":0.497},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.30099999999999999,"15min":0.29899999999999999},"min":{"1min":0.23100000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48899999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.33300000000000002,"15min":0.33900000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.498,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.315}]}]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":67023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"min":{"1min":0.158,"5min":0.158,"15min":0.158},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.379}]},{"osd":1,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.22500000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29699999999999999},"min":{"1min":0.19900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.36199999999999999}]},{"osd":3,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.34599999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705694,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":60178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 22:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.29199999999999998,"15min":0.28899999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.318},{"interface":"front","average":{"1min":0.316,"5min":0.30399999999999999,"15min":0.30199999999999999},"min":{"1min":0.22700000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Sep 30 22:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.309,"15min":0.30499999999999999},"min":{"1min":0.22800000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30199999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26900000000000002}]},{"osd":3,"last update":"Sat Sep 30 22:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.251},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33200000000000002,"15min":0.33000000000000002},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.42399999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705695,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":64106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.28199999999999997,"5min":0.30199999999999999,"15min":0.30599999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.307,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.22600000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.41999999999999998,"5min":0.434,"15min":0.434},"last":0.32300000000000001}]},{"osd":2,"last update":"Sat Sep 30 22:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.30499999999999999,"15min":0.30599999999999999},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.39700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.309,"5min":0.33000000000000002,"15min":0.33400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.42699999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.30599999999999999}]},{"osd":3,"last update":"Sat Sep 30 22:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.311,"15min":0.313},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.33800000000000002,"15min":0.34000000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.442,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.315}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:16:22.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:16:22.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.639+0000 7f44527ee700 1 Processor -- start 2023-09-30T22:16:22.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.639+0000 7f44527ee700 1 -- start start 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.639+0000 7f44527ee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.639+0000 7f44527ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f444c09e000 0x7f444c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.639+0000 7f44527ee700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c09f130 con 0x7f444c09e410 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.639+0000 7f44527ee700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c09f520 con 0x7f444c09e000 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.639+0000 7f44527ee700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f444c09e930 con 0x7f444c09d6f0 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f444c09e000 0x7f444c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f4450d8b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f444c09e410 0x7f444c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52910/0 (socket says 172.21.15.89:52910) 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f4450d8b700 1 -- 172.21.15.89:0/413553127 learned_addr learned my addr 172.21.15.89:0/413553127 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444b7fe700 1 -- 172.21.15.89:0/413553127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 msgr2=0x7f444c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444b7fe700 1 --2- 172.21.15.89:0/413553127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444b7fe700 1 -- 172.21.15.89:0/413553127 >> v1:172.21.15.89:6789/0 conn(0x7f444c09e410 legacy=0x7f444c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444affd700 1 -- 172.21.15.89:0/413553127 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156166260 0 0) 0x7f444c09f130 con 0x7f444c09e410 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444b7fe700 1 -- 172.21.15.89:0/413553127 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c09b2d0 con 0x7f444c09e000 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444bfff700 1 --2- 172.21.15.89:0/413553127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444b7fe700 1 --2- 172.21.15.89:0/413553127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c130660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f443c009840 tx=0x7f443c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0504b86cddf5031 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444affd700 1 -- 172.21.15.89:0/413553127 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c011830 con 0x7f444c09e000 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444affd700 1 -- 172.21.15.89:0/413553127 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c011990 con 0x7f444c09e000 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.640+0000 7f444affd700 1 -- 172.21.15.89:0/413553127 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c011d20 con 0x7f444c09e000 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.641+0000 7f44527ee700 1 -- 172.21.15.89:0/413553127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 msgr2=0x7f444c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.641+0000 7f44527ee700 1 --2- 172.21.15.89:0/413553127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c130660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f443c009840 tx=0x7f443c009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.641+0000 7f44527ee700 1 -- 172.21.15.89:0/413553127 shutdown_connections 2023-09-30T22:16:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.641+0000 7f44527ee700 1 --2- 172.21.15.89:0/413553127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.641+0000 7f44527ee700 1 --2- 172.21.15.89:0/413553127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c130660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.641+0000 7f44527ee700 1 -- 172.21.15.89:0/413553127 >> 172.21.15.89:0/413553127 conn(0x7f444c066f50 msgr2=0x7f444c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.641+0000 7f44527ee700 1 -- 172.21.15.89:0/413553127 shutdown_connections 2023-09-30T22:16:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.641+0000 7f44527ee700 1 -- 172.21.15.89:0/413553127 wait complete. 2023-09-30T22:16:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f44527ee700 1 Processor -- start 2023-09-30T22:16:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f44527ee700 1 -- start start 2023-09-30T22:16:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f44527ee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c121080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f44527ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c1215c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f44527ee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f444c09e410 0x7f444c121b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f44527ee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1244f0 con 0x7f444c09e000 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f44527ee700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c122240 con 0x7f444c09e410 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f44527ee700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f444c06ec60 con 0x7f444c09d6f0 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f444bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c121080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f444b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c1215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f444bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c121080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46074/0 (socket says 172.21.15.89:46074) 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f4450d8b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f444c09e410 0x7f444c121b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f444bfff700 1 -- 172.21.15.89:0/1995913175 learned_addr learned my addr 172.21.15.89:0/1995913175 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.642+0000 7f444b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c1215c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60898/0 (socket says 172.21.15.89:60898) 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f444bfff700 1 -- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f444c09e410 msgr2=0x7f444c121b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f444bfff700 1 --2- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f444c09e410 0x7f444c121b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f444bfff700 1 -- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 msgr2=0x7f444c1215c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f444bfff700 1 --2- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c1215c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f444bfff700 1 -- 172.21.15.89:0/1995913175 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400094a0 con 0x7f444c09d6f0 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f444b7fe700 1 --2- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c1215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f444bfff700 1 --2- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c121080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f444000ebe0 tx=0x7f444000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f4448ff9700 1 -- 172.21.15.89:0/1995913175 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000c850 con 0x7f444c09d6f0 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f4448ff9700 1 -- 172.21.15.89:0/1995913175 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000c9b0 con 0x7f444c09d6f0 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c0094a0 con 0x7f444c09d6f0 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f4448ff9700 1 -- 172.21.15.89:0/1995913175 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000cd40 con 0x7f444c09d6f0 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.643+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c123c20 con 0x7f444c09d6f0 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.644+0000 7f4448ff9700 1 -- 172.21.15.89:0/1995913175 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f4440013470 con 0x7f444c09d6f0 2023-09-30T22:16:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.644+0000 7f4448ff9700 1 -- 172.21.15.89:0/1995913175 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f444003fb30 con 0x7f444c09d6f0 2023-09-30T22:16:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.644+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c074ea0 con 0x7f444c09d6f0 2023-09-30T22:16:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.649+0000 7f4448ff9700 1 -- 172.21.15.89:0/1995913175 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4440021080 con 0x7f444c09d6f0 2023-09-30T22:16:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.831+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 --> v1:172.21.15.89:6816/126953 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f444c09c000 con 0x7f443802f480 2023-09-30T22:16:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.838+0000 7f4448ff9700 1 -- 172.21.15.89:0/1995913175 <== mgr.14124 v1:172.21.15.89:6816/126953 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+270369 (unknown 2965378022 0 2887483882) 0x7f443c011630 con 0x7f443802f480 2023-09-30T22:16:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:22.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 >> v1:172.21.15.89:6816/126953 conn(0x7f443802f480 legacy=0x7f4438031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 msgr2=0x7f444c121080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:22.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 --2- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c121080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f444000ebe0 tx=0x7f444000eef0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 shutdown_connections 2023-09-30T22:16:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 --2- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f444c09e410 0x7f444c121b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 --2- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f444c09d6f0 0x7f444c121080 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 --2- 172.21.15.89:0/1995913175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444c09e000 0x7f444c1215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 >> 172.21.15.89:0/1995913175 conn(0x7f444c066f50 msgr2=0x7f444c119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 shutdown_connections 2023-09-30T22:16:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:22.841+0000 7f44527ee700 1 -- 172.21.15.89:0/1995913175 wait complete. 2023-09-30T22:16:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-09-30T22:16:22.852 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2023-09-30T22:16:21.424349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189132,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298228,"statfs":{"total":386547056640,"available":385329385472,"internally_reserved":0,"allocated":765952,"data_stored":258524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.992274"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551652+0000","last_change":"2023-09-30T22:14:02.086445+0000","last_active":"2023-09-30T22:14:52.551652+0000","last_peered":"2023-09-30T22:14:52.551652+0000","last_clean":"2023-09-30T22:14:52.551652+0000","last_became_active":"2023-09-30T22:14:02.086159+0000","last_became_peered":"2023-09-30T22:14:02.086159+0000","last_unstale":"2023-09-30T22:14:52.551652+0000","last_undegraded":"2023-09-30T22:14:52.551652+0000","last_fullsized":"2023-09-30T22:14:52.551652+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552688+0000","last_change":"2023-09-30T22:14:02.082560+0000","last_active":"2023-09-30T22:14:52.552688+0000","last_peered":"2023-09-30T22:14:52.552688+0000","last_clean":"2023-09-30T22:14:52.552688+0000","last_became_active":"2023-09-30T22:14:02.082315+0000","last_became_peered":"2023-09-30T22:14:02.082315+0000","last_unstale":"2023-09-30T22:14:52.552688+0000","last_undegraded":"2023-09-30T22:14:52.552688+0000","last_fullsized":"2023-09-30T22:14:52.552688+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549798+0000","last_change":"2023-09-30T22:14:02.082673+0000","last_active":"2023-09-30T22:14:52.549798+0000","last_peered":"2023-09-30T22:14:52.549798+0000","last_clean":"2023-09-30T22:14:52.549798+0000","last_became_active":"2023-09-30T22:14:02.082437+0000","last_became_peered":"2023-09-30T22:14:02.082437+0000","last_unstale":"2023-09-30T22:14:52.549798+0000","last_undegraded":"2023-09-30T22:14:52.549798+0000","last_fullsized":"2023-09-30T22:14:52.549798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550210+0000","last_change":"2023-09-30T22:14:02.080886+0000","last_active":"2023-09-30T22:14:52.550210+0000","last_peered":"2023-09-30T22:14:52.550210+0000","last_clean":"2023-09-30T22:14:52.550210+0000","last_became_active":"2023-09-30T22:14:02.078093+0000","last_became_peered":"2023-09-30T22:14:02.078093+0000","last_unstale":"2023-09-30T22:14:52.550210+0000","last_undegraded":"2023-09-30T22:14:52.550210+0000","last_fullsized":"2023-09-30T22:14:52.550210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549844+0000","last_change":"2023-09-30T22:14:02.074967+0000","last_active":"2023-09-30T22:14:52.549844+0000","last_peered":"2023-09-30T22:14:52.549844+0000","last_clean":"2023-09-30T22:14:52.549844+0000","last_became_active":"2023-09-30T22:14:02.074701+0000","last_became_peered":"2023-09-30T22:14:02.074701+0000","last_unstale":"2023-09-30T22:14:52.549844+0000","last_undegraded":"2023-09-30T22:14:52.549844+0000","last_fullsized":"2023-09-30T22:14:52.549844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733701+0000","last_change":"2023-09-30T22:14:02.081443+0000","last_active":"2023-09-30T22:14:52.733701+0000","last_peered":"2023-09-30T22:14:52.733701+0000","last_clean":"2023-09-30T22:14:52.733701+0000","last_became_active":"2023-09-30T22:14:02.080408+0000","last_became_peered":"2023-09-30T22:14:02.080408+0000","last_unstale":"2023-09-30T22:14:52.733701+0000","last_undegraded":"2023-09-30T22:14:52.733701+0000","last_fullsized":"2023-09-30T22:14:52.733701+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733177+0000","last_change":"2023-09-30T22:14:02.085310+0000","last_active":"2023-09-30T22:14:52.733177+0000","last_peered":"2023-09-30T22:14:52.733177+0000","last_clean":"2023-09-30T22:14:52.733177+0000","last_became_active":"2023-09-30T22:14:02.084647+0000","last_became_peered":"2023-09-30T22:14:02.084647+0000","last_unstale":"2023-09-30T22:14:52.733177+0000","last_undegraded":"2023-09-30T22:14:52.733177+0000","last_fullsized":"2023-09-30T22:14:52.733177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550773+0000","last_change":"2023-09-30T22:14:02.085988+0000","last_active":"2023-09-30T22:14:52.550773+0000","last_peered":"2023-09-30T22:14:52.550773+0000","last_clean":"2023-09-30T22:14:52.550773+0000","last_became_active":"2023-09-30T22:14:02.083245+0000","last_became_peered":"2023-09-30T22:14:02.083245+0000","last_unstale":"2023-09-30T22:14:52.550773+0000","last_undegraded":"2023-09-30T22:14:52.550773+0000","last_fullsized":"2023-09-30T22:14:52.550773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552926+0000","last_change":"2023-09-30T22:14:02.085522+0000","last_active":"2023-09-30T22:14:52.552926+0000","last_peered":"2023-09-30T22:14:52.552926+0000","last_clean":"2023-09-30T22:14:52.552926+0000","last_became_active":"2023-09-30T22:14:02.084959+0000","last_became_peered":"2023-09-30T22:14:02.084959+0000","last_unstale":"2023-09-30T22:14:52.552926+0000","last_undegraded":"2023-09-30T22:14:52.552926+0000","last_fullsized":"2023-09-30T22:14:52.552926+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552859+0000","last_change":"2023-09-30T22:14:02.071672+0000","last_active":"2023-09-30T22:14:52.552859+0000","last_peered":"2023-09-30T22:14:52.552859+0000","last_clean":"2023-09-30T22:14:52.552859+0000","last_became_active":"2023-09-30T22:14:02.071361+0000","last_became_peered":"2023-09-30T22:14:02.071361+0000","last_unstale":"2023-09-30T22:14:52.552859+0000","last_undegraded":"2023-09-30T22:14:52.552859+0000","last_fullsized":"2023-09-30T22:14:52.552859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552384+0000","last_change":"2023-09-30T22:14:02.074689+0000","last_active":"2023-09-30T22:14:52.552384+0000","last_peered":"2023-09-30T22:14:52.552384+0000","last_clean":"2023-09-30T22:14:52.552384+0000","last_became_active":"2023-09-30T22:14:02.074350+0000","last_became_peered":"2023-09-30T22:14:02.074350+0000","last_unstale":"2023-09-30T22:14:52.552384+0000","last_undegraded":"2023-09-30T22:14:52.552384+0000","last_fullsized":"2023-09-30T22:14:52.552384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.556558+0000","last_change":"2023-09-30T22:14:52.554121+0000","last_active":"2023-09-30T22:14:52.556558+0000","last_peered":"2023-09-30T22:14:52.556558+0000","last_clean":"2023-09-30T22:14:52.556558+0000","last_became_active":"2023-09-30T22:14:52.553793+0000","last_became_peered":"2023-09-30T22:14:52.553793+0000","last_unstale":"2023-09-30T22:14:52.556558+0000","last_undegraded":"2023-09-30T22:14:52.556558+0000","last_fullsized":"2023-09-30T22:14:52.556558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458275+0000","last_change":"2023-09-30T22:14:02.087347+0000","last_active":"2023-09-30T22:16:19.458275+0000","last_peered":"2023-09-30T22:16:19.458275+0000","last_clean":"2023-09-30T22:16:19.458275+0000","last_became_active":"2023-09-30T22:14:02.086637+0000","last_became_peered":"2023-09-30T22:14:02.086637+0000","last_unstale":"2023-09-30T22:16:19.458275+0000","last_undegraded":"2023-09-30T22:16:19.458275+0000","last_fullsized":"2023-09-30T22:16:19.458275+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552168+0000","last_change":"2023-09-30T22:14:02.066399+0000","last_active":"2023-09-30T22:14:52.552168+0000","last_peered":"2023-09-30T22:14:52.552168+0000","last_clean":"2023-09-30T22:14:52.552168+0000","last_became_active":"2023-09-30T22:14:02.066052+0000","last_became_peered":"2023-09-30T22:14:02.066052+0000","last_unstale":"2023-09-30T22:14:52.552168+0000","last_undegraded":"2023-09-30T22:14:52.552168+0000","last_fullsized":"2023-09-30T22:14:52.552168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549533+0000","last_change":"2023-09-30T22:14:02.073165+0000","last_active":"2023-09-30T22:14:52.549533+0000","last_peered":"2023-09-30T22:14:52.549533+0000","last_clean":"2023-09-30T22:14:52.549533+0000","last_became_active":"2023-09-30T22:14:02.073000+0000","last_became_peered":"2023-09-30T22:14:02.073000+0000","last_unstale":"2023-09-30T22:14:52.549533+0000","last_undegraded":"2023-09-30T22:14:52.549533+0000","last_fullsized":"2023-09-30T22:14:52.549533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733099+0000","last_change":"2023-09-30T22:14:02.072772+0000","last_active":"2023-09-30T22:14:52.733099+0000","last_peered":"2023-09-30T22:14:52.733099+0000","last_clean":"2023-09-30T22:14:52.733099+0000","last_became_active":"2023-09-30T22:14:02.072569+0000","last_became_peered":"2023-09-30T22:14:02.072569+0000","last_unstale":"2023-09-30T22:14:52.733099+0000","last_undegraded":"2023-09-30T22:14:52.733099+0000","last_fullsized":"2023-09-30T22:14:52.733099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.557223+0000","last_change":"2023-09-30T22:14:52.556259+0000","last_active":"2023-09-30T22:14:52.557223+0000","last_peered":"2023-09-30T22:14:52.557223+0000","last_clean":"2023-09-30T22:14:52.557223+0000","last_became_active":"2023-09-30T22:14:52.556005+0000","last_became_peered":"2023-09-30T22:14:52.556005+0000","last_unstale":"2023-09-30T22:14:52.557223+0000","last_undegraded":"2023-09-30T22:14:52.557223+0000","last_fullsized":"2023-09-30T22:14:52.557223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551000+0000","last_change":"2023-09-30T22:14:02.070008+0000","last_active":"2023-09-30T22:14:52.551000+0000","last_peered":"2023-09-30T22:14:52.551000+0000","last_clean":"2023-09-30T22:14:52.551000+0000","last_became_active":"2023-09-30T22:14:02.069809+0000","last_became_peered":"2023-09-30T22:14:02.069809+0000","last_unstale":"2023-09-30T22:14:52.551000+0000","last_undegraded":"2023-09-30T22:14:52.551000+0000","last_fullsized":"2023-09-30T22:14:52.551000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549880+0000","last_change":"2023-09-30T22:14:02.087939+0000","last_active":"2023-09-30T22:14:52.549880+0000","last_peered":"2023-09-30T22:14:52.549880+0000","last_clean":"2023-09-30T22:14:52.549880+0000","last_became_active":"2023-09-30T22:14:02.087733+0000","last_became_peered":"2023-09-30T22:14:02.087733+0000","last_unstale":"2023-09-30T22:14:52.549880+0000","last_undegraded":"2023-09-30T22:14:52.549880+0000","last_fullsized":"2023-09-30T22:14:52.549880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552401+0000","last_change":"2023-09-30T22:14:02.090627+0000","last_active":"2023-09-30T22:14:52.552401+0000","last_peered":"2023-09-30T22:14:52.552401+0000","last_clean":"2023-09-30T22:14:52.552401+0000","last_became_active":"2023-09-30T22:14:02.090255+0000","last_became_peered":"2023-09-30T22:14:02.090255+0000","last_unstale":"2023-09-30T22:14:52.552401+0000","last_undegraded":"2023-09-30T22:14:52.552401+0000","last_fullsized":"2023-09-30T22:14:52.552401+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551837+0000","last_change":"2023-09-30T22:14:02.071417+0000","last_active":"2023-09-30T22:14:52.551837+0000","last_peered":"2023-09-30T22:14:52.551837+0000","last_clean":"2023-09-30T22:14:52.551837+0000","last_became_active":"2023-09-30T22:14:02.071046+0000","last_became_peered":"2023-09-30T22:14:02.071046+0000","last_unstale":"2023-09-30T22:14:52.551837+0000","last_undegraded":"2023-09-30T22:14:52.551837+0000","last_fullsized":"2023-09-30T22:14:52.551837+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549622+0000","last_change":"2023-09-30T22:14:02.076131+0000","last_active":"2023-09-30T22:14:52.549622+0000","last_peered":"2023-09-30T22:14:52.549622+0000","last_clean":"2023-09-30T22:14:52.549622+0000","last_became_active":"2023-09-30T22:14:02.075850+0000","last_became_peered":"2023-09-30T22:14:02.075850+0000","last_unstale":"2023-09-30T22:14:52.549622+0000","last_undegraded":"2023-09-30T22:14:52.549622+0000","last_fullsized":"2023-09-30T22:14:52.549622+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733087+0000","last_change":"2023-09-30T22:14:02.085210+0000","last_active":"2023-09-30T22:14:52.733087+0000","last_peered":"2023-09-30T22:14:52.733087+0000","last_clean":"2023-09-30T22:14:52.733087+0000","last_became_active":"2023-09-30T22:14:02.084320+0000","last_became_peered":"2023-09-30T22:14:02.084320+0000","last_unstale":"2023-09-30T22:14:52.733087+0000","last_undegraded":"2023-09-30T22:14:52.733087+0000","last_fullsized":"2023-09-30T22:14:52.733087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.557709+0000","last_change":"2023-09-30T22:14:52.555620+0000","last_active":"2023-09-30T22:14:52.557709+0000","last_peered":"2023-09-30T22:14:52.557709+0000","last_clean":"2023-09-30T22:14:52.557709+0000","last_became_active":"2023-09-30T22:14:52.555260+0000","last_became_peered":"2023-09-30T22:14:52.555260+0000","last_unstale":"2023-09-30T22:14:52.557709+0000","last_undegraded":"2023-09-30T22:14:52.557709+0000","last_fullsized":"2023-09-30T22:14:52.557709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.736927+0000","last_change":"2023-09-30T22:14:52.735020+0000","last_active":"2023-09-30T22:14:52.736927+0000","last_peered":"2023-09-30T22:14:52.736927+0000","last_clean":"2023-09-30T22:14:52.736927+0000","last_became_active":"2023-09-30T22:14:52.734697+0000","last_became_peered":"2023-09-30T22:14:52.734697+0000","last_unstale":"2023-09-30T22:14:52.736927+0000","last_undegraded":"2023-09-30T22:14:52.736927+0000","last_fullsized":"2023-09-30T22:14:52.736927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552819+0000","last_change":"2023-09-30T22:14:02.077989+0000","last_active":"2023-09-30T22:14:52.552819+0000","last_peered":"2023-09-30T22:14:52.552819+0000","last_clean":"2023-09-30T22:14:52.552819+0000","last_became_active":"2023-09-30T22:14:02.071401+0000","last_became_peered":"2023-09-30T22:14:02.071401+0000","last_unstale":"2023-09-30T22:14:52.552819+0000","last_undegraded":"2023-09-30T22:14:52.552819+0000","last_fullsized":"2023-09-30T22:14:52.552819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.556563+0000","last_change":"2023-09-30T22:14:52.554267+0000","last_active":"2023-09-30T22:14:52.556563+0000","last_peered":"2023-09-30T22:14:52.556563+0000","last_clean":"2023-09-30T22:14:52.556563+0000","last_became_active":"2023-09-30T22:14:52.553993+0000","last_became_peered":"2023-09-30T22:14:52.553993+0000","last_unstale":"2023-09-30T22:14:52.556563+0000","last_undegraded":"2023-09-30T22:14:52.556563+0000","last_fullsized":"2023-09-30T22:14:52.556563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552916+0000","last_change":"2023-09-30T22:14:01.697398+0000","last_active":"2023-09-30T22:14:52.552916+0000","last_peered":"2023-09-30T22:14:52.552916+0000","last_clean":"2023-09-30T22:14:52.552916+0000","last_became_active":"2023-09-30T22:14:01.694824+0000","last_became_peered":"2023-09-30T22:14:01.694824+0000","last_unstale":"2023-09-30T22:14:52.552916+0000","last_undegraded":"2023-09-30T22:14:52.552916+0000","last_fullsized":"2023-09-30T22:14:52.552916+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733491+0000","last_change":"2023-09-30T22:14:02.075771+0000","last_active":"2023-09-30T22:14:52.733491+0000","last_peered":"2023-09-30T22:14:52.733491+0000","last_clean":"2023-09-30T22:14:52.733491+0000","last_became_active":"2023-09-30T22:14:02.074872+0000","last_became_peered":"2023-09-30T22:14:02.074872+0000","last_unstale":"2023-09-30T22:14:52.733491+0000","last_undegraded":"2023-09-30T22:14:52.733491+0000","last_fullsized":"2023-09-30T22:14:52.733491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549456+0000","last_change":"2023-09-30T22:14:01.702319+0000","last_active":"2023-09-30T22:14:52.549456+0000","last_peered":"2023-09-30T22:14:52.549456+0000","last_clean":"2023-09-30T22:14:52.549456+0000","last_became_active":"2023-09-30T22:14:01.702020+0000","last_became_peered":"2023-09-30T22:14:01.702020+0000","last_unstale":"2023-09-30T22:14:52.549456+0000","last_undegraded":"2023-09-30T22:14:52.549456+0000","last_fullsized":"2023-09-30T22:14:52.549456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551168+0000","last_change":"2023-09-30T22:14:02.079713+0000","last_active":"2023-09-30T22:14:52.551168+0000","last_peered":"2023-09-30T22:14:52.551168+0000","last_clean":"2023-09-30T22:14:52.551168+0000","last_became_active":"2023-09-30T22:14:02.079318+0000","last_became_peered":"2023-09-30T22:14:02.079318+0000","last_unstale":"2023-09-30T22:14:52.551168+0000","last_undegraded":"2023-09-30T22:14:52.551168+0000","last_fullsized":"2023-09-30T22:14:52.551168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733884+0000","last_change":"2023-09-30T22:14:01.707097+0000","last_active":"2023-09-30T22:14:52.733884+0000","last_peered":"2023-09-30T22:14:52.733884+0000","last_clean":"2023-09-30T22:14:52.733884+0000","last_became_active":"2023-09-30T22:14:01.706849+0000","last_became_peered":"2023-09-30T22:14:01.706849+0000","last_unstale":"2023-09-30T22:14:52.733884+0000","last_undegraded":"2023-09-30T22:14:52.733884+0000","last_fullsized":"2023-09-30T22:14:52.733884+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.556893+0000","last_change":"2023-09-30T22:14:52.554552+0000","last_active":"2023-09-30T22:14:52.556893+0000","last_peered":"2023-09-30T22:14:52.556893+0000","last_clean":"2023-09-30T22:14:52.556893+0000","last_became_active":"2023-09-30T22:14:52.554144+0000","last_became_peered":"2023-09-30T22:14:52.554144+0000","last_unstale":"2023-09-30T22:14:52.556893+0000","last_undegraded":"2023-09-30T22:14:52.556893+0000","last_fullsized":"2023-09-30T22:14:52.556893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551593+0000","last_change":"2023-09-30T22:14:01.698007+0000","last_active":"2023-09-30T22:14:52.551593+0000","last_peered":"2023-09-30T22:14:52.551593+0000","last_clean":"2023-09-30T22:14:52.551593+0000","last_became_active":"2023-09-30T22:14:01.697731+0000","last_became_peered":"2023-09-30T22:14:01.697731+0000","last_unstale":"2023-09-30T22:14:52.551593+0000","last_undegraded":"2023-09-30T22:14:52.551593+0000","last_fullsized":"2023-09-30T22:14:52.551593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733979+0000","last_change":"2023-09-30T22:14:02.082853+0000","last_active":"2023-09-30T22:14:52.733979+0000","last_peered":"2023-09-30T22:14:52.733979+0000","last_clean":"2023-09-30T22:14:52.733979+0000","last_became_active":"2023-09-30T22:14:02.082597+0000","last_became_peered":"2023-09-30T22:14:02.082597+0000","last_unstale":"2023-09-30T22:14:52.733979+0000","last_undegraded":"2023-09-30T22:14:52.733979+0000","last_fullsized":"2023-09-30T22:14:52.733979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552549+0000","last_change":"2023-09-30T22:14:01.700657+0000","last_active":"2023-09-30T22:14:52.552549+0000","last_peered":"2023-09-30T22:14:52.552549+0000","last_clean":"2023-09-30T22:14:52.552549+0000","last_became_active":"2023-09-30T22:14:01.699633+0000","last_became_peered":"2023-09-30T22:14:01.699633+0000","last_unstale":"2023-09-30T22:14:52.552549+0000","last_undegraded":"2023-09-30T22:14:52.552549+0000","last_fullsized":"2023-09-30T22:14:52.552549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734087+0000","last_change":"2023-09-30T22:14:02.085416+0000","last_active":"2023-09-30T22:14:52.734087+0000","last_peered":"2023-09-30T22:14:52.734087+0000","last_clean":"2023-09-30T22:14:52.734087+0000","last_became_active":"2023-09-30T22:14:02.084970+0000","last_became_peered":"2023-09-30T22:14:02.084970+0000","last_unstale":"2023-09-30T22:14:52.734087+0000","last_undegraded":"2023-09-30T22:14:52.734087+0000","last_fullsized":"2023-09-30T22:14:52.734087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549708+0000","last_change":"2023-09-30T22:14:01.701075+0000","last_active":"2023-09-30T22:14:52.549708+0000","last_peered":"2023-09-30T22:14:52.549708+0000","last_clean":"2023-09-30T22:14:52.549708+0000","last_became_active":"2023-09-30T22:14:01.700806+0000","last_became_peered":"2023-09-30T22:14:01.700806+0000","last_unstale":"2023-09-30T22:14:52.549708+0000","last_undegraded":"2023-09-30T22:14:52.549708+0000","last_fullsized":"2023-09-30T22:14:52.549708+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552081+0000","last_change":"2023-09-30T22:14:02.074078+0000","last_active":"2023-09-30T22:14:52.552081+0000","last_peered":"2023-09-30T22:14:52.552081+0000","last_clean":"2023-09-30T22:14:52.552081+0000","last_became_active":"2023-09-30T22:14:02.073804+0000","last_became_peered":"2023-09-30T22:14:02.073804+0000","last_unstale":"2023-09-30T22:14:52.552081+0000","last_undegraded":"2023-09-30T22:14:52.552081+0000","last_fullsized":"2023-09-30T22:14:52.552081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'12","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.476400+0000","last_change":"2023-09-30T22:14:01.711172+0000","last_active":"2023-09-30T22:16:19.476400+0000","last_peered":"2023-09-30T22:16:19.476400+0000","last_clean":"2023-09-30T22:16:19.476400+0000","last_became_active":"2023-09-30T22:14:01.710926+0000","last_became_peered":"2023-09-30T22:14:01.710926+0000","last_unstale":"2023-09-30T22:16:19.476400+0000","last_undegraded":"2023-09-30T22:16:19.476400+0000","last_fullsized":"2023-09-30T22:16:19.476400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458362+0000","last_change":"2023-09-30T22:14:02.087193+0000","last_active":"2023-09-30T22:16:19.458362+0000","last_peered":"2023-09-30T22:16:19.458362+0000","last_clean":"2023-09-30T22:16:19.458362+0000","last_became_active":"2023-09-30T22:14:02.086273+0000","last_became_peered":"2023-09-30T22:14:02.086273+0000","last_unstale":"2023-09-30T22:16:19.458362+0000","last_undegraded":"2023-09-30T22:16:19.458362+0000","last_fullsized":"2023-09-30T22:16:19.458362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734112+0000","last_change":"2023-09-30T22:14:01.706560+0000","last_active":"2023-09-30T22:14:52.734112+0000","last_peered":"2023-09-30T22:14:52.734112+0000","last_clean":"2023-09-30T22:14:52.734112+0000","last_became_active":"2023-09-30T22:14:01.706330+0000","last_became_peered":"2023-09-30T22:14:01.706330+0000","last_unstale":"2023-09-30T22:14:52.734112+0000","last_undegraded":"2023-09-30T22:14:52.734112+0000","last_fullsized":"2023-09-30T22:14:52.734112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733307+0000","last_change":"2023-09-30T22:14:02.084139+0000","last_active":"2023-09-30T22:14:52.733307+0000","last_peered":"2023-09-30T22:14:52.733307+0000","last_clean":"2023-09-30T22:14:52.733307+0000","last_became_active":"2023-09-30T22:14:02.080986+0000","last_became_peered":"2023-09-30T22:14:02.080986+0000","last_unstale":"2023-09-30T22:14:52.733307+0000","last_undegraded":"2023-09-30T22:14:52.733307+0000","last_fullsized":"2023-09-30T22:14:52.733307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549677+0000","last_change":"2023-09-30T22:14:01.698141+0000","last_active":"2023-09-30T22:14:52.549677+0000","last_peered":"2023-09-30T22:14:52.549677+0000","last_clean":"2023-09-30T22:14:52.549677+0000","last_became_active":"2023-09-30T22:14:01.697885+0000","last_became_peered":"2023-09-30T22:14:01.697885+0000","last_unstale":"2023-09-30T22:14:52.549677+0000","last_undegraded":"2023-09-30T22:14:52.549677+0000","last_fullsized":"2023-09-30T22:14:52.549677+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733580+0000","last_change":"2023-09-30T22:14:02.075925+0000","last_active":"2023-09-30T22:14:52.733580+0000","last_peered":"2023-09-30T22:14:52.733580+0000","last_clean":"2023-09-30T22:14:52.733580+0000","last_became_active":"2023-09-30T22:14:02.075137+0000","last_became_peered":"2023-09-30T22:14:02.075137+0000","last_unstale":"2023-09-30T22:14:52.733580+0000","last_undegraded":"2023-09-30T22:14:52.733580+0000","last_fullsized":"2023-09-30T22:14:52.733580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734069+0000","last_change":"2023-09-30T22:14:01.712299+0000","last_active":"2023-09-30T22:14:52.734069+0000","last_peered":"2023-09-30T22:14:52.734069+0000","last_clean":"2023-09-30T22:14:52.734069+0000","last_became_active":"2023-09-30T22:14:01.711791+0000","last_became_peered":"2023-09-30T22:14:01.711791+0000","last_unstale":"2023-09-30T22:14:52.734069+0000","last_undegraded":"2023-09-30T22:14:52.734069+0000","last_fullsized":"2023-09-30T22:14:52.734069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551293+0000","last_change":"2023-09-30T22:14:02.084033+0000","last_active":"2023-09-30T22:14:52.551293+0000","last_peered":"2023-09-30T22:14:52.551293+0000","last_clean":"2023-09-30T22:14:52.551293+0000","last_became_active":"2023-09-30T22:14:02.083764+0000","last_became_peered":"2023-09-30T22:14:02.083764+0000","last_unstale":"2023-09-30T22:14:52.551293+0000","last_undegraded":"2023-09-30T22:14:52.551293+0000","last_fullsized":"2023-09-30T22:14:52.551293+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551632+0000","last_change":"2023-09-30T22:14:01.706777+0000","last_active":"2023-09-30T22:14:52.551632+0000","last_peered":"2023-09-30T22:14:52.551632+0000","last_clean":"2023-09-30T22:14:52.551632+0000","last_became_active":"2023-09-30T22:14:01.706479+0000","last_became_peered":"2023-09-30T22:14:01.706479+0000","last_unstale":"2023-09-30T22:14:52.551632+0000","last_undegraded":"2023-09-30T22:14:52.551632+0000","last_fullsized":"2023-09-30T22:14:52.551632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733886+0000","last_change":"2023-09-30T22:14:02.084399+0000","last_active":"2023-09-30T22:14:52.733886+0000","last_peered":"2023-09-30T22:14:52.733886+0000","last_clean":"2023-09-30T22:14:52.733886+0000","last_became_active":"2023-09-30T22:14:02.084137+0000","last_became_peered":"2023-09-30T22:14:02.084137+0000","last_unstale":"2023-09-30T22:14:52.733886+0000","last_undegraded":"2023-09-30T22:14:52.733886+0000","last_fullsized":"2023-09-30T22:14:52.733886+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549339+0000","last_change":"2023-09-30T22:14:01.702685+0000","last_active":"2023-09-30T22:14:52.549339+0000","last_peered":"2023-09-30T22:14:52.549339+0000","last_clean":"2023-09-30T22:14:52.549339+0000","last_became_active":"2023-09-30T22:14:01.702083+0000","last_became_peered":"2023-09-30T22:14:01.702083+0000","last_unstale":"2023-09-30T22:14:52.549339+0000","last_undegraded":"2023-09-30T22:14:52.549339+0000","last_fullsized":"2023-09-30T22:14:52.549339+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458027+0000","last_change":"2023-09-30T22:14:52.553897+0000","last_active":"2023-09-30T22:16:19.458027+0000","last_peered":"2023-09-30T22:16:19.458027+0000","last_clean":"2023-09-30T22:16:19.458027+0000","last_became_active":"2023-09-30T22:14:52.553612+0000","last_became_peered":"2023-09-30T22:14:52.553612+0000","last_unstale":"2023-09-30T22:16:19.458027+0000","last_undegraded":"2023-09-30T22:16:19.458027+0000","last_fullsized":"2023-09-30T22:16:19.458027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552617+0000","last_change":"2023-09-30T22:14:01.706748+0000","last_active":"2023-09-30T22:14:52.552617+0000","last_peered":"2023-09-30T22:14:52.552617+0000","last_clean":"2023-09-30T22:14:52.552617+0000","last_became_active":"2023-09-30T22:14:01.706472+0000","last_became_peered":"2023-09-30T22:14:01.706472+0000","last_unstale":"2023-09-30T22:14:52.552617+0000","last_undegraded":"2023-09-30T22:14:52.552617+0000","last_fullsized":"2023-09-30T22:14:52.552617+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549372+0000","last_change":"2023-09-30T22:14:02.070587+0000","last_active":"2023-09-30T22:14:52.549372+0000","last_peered":"2023-09-30T22:14:52.549372+0000","last_clean":"2023-09-30T22:14:52.549372+0000","last_became_active":"2023-09-30T22:14:02.070259+0000","last_became_peered":"2023-09-30T22:14:02.070259+0000","last_unstale":"2023-09-30T22:14:52.549372+0000","last_undegraded":"2023-09-30T22:14:52.549372+0000","last_fullsized":"2023-09-30T22:14:52.549372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733668+0000","last_change":"2023-09-30T22:14:01.704603+0000","last_active":"2023-09-30T22:14:52.733668+0000","last_peered":"2023-09-30T22:14:52.733668+0000","last_clean":"2023-09-30T22:14:52.733668+0000","last_became_active":"2023-09-30T22:14:01.704006+0000","last_became_peered":"2023-09-30T22:14:01.704006+0000","last_unstale":"2023-09-30T22:14:52.733668+0000","last_undegraded":"2023-09-30T22:14:52.733668+0000","last_fullsized":"2023-09-30T22:14:52.733668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458369+0000","last_change":"2023-09-30T22:14:02.073527+0000","last_active":"2023-09-30T22:16:19.458369+0000","last_peered":"2023-09-30T22:16:19.458369+0000","last_clean":"2023-09-30T22:16:19.458369+0000","last_became_active":"2023-09-30T22:14:02.072950+0000","last_became_peered":"2023-09-30T22:14:02.072950+0000","last_unstale":"2023-09-30T22:16:19.458369+0000","last_undegraded":"2023-09-30T22:16:19.458369+0000","last_fullsized":"2023-09-30T22:16:19.458369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550333+0000","last_change":"2023-09-30T22:14:01.701736+0000","last_active":"2023-09-30T22:14:52.550333+0000","last_peered":"2023-09-30T22:14:52.550333+0000","last_clean":"2023-09-30T22:14:52.550333+0000","last_became_active":"2023-09-30T22:14:01.701471+0000","last_became_peered":"2023-09-30T22:14:01.701471+0000","last_unstale":"2023-09-30T22:14:52.550333+0000","last_undegraded":"2023-09-30T22:14:52.550333+0000","last_fullsized":"2023-09-30T22:14:52.550333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550088+0000","last_change":"2023-09-30T22:14:02.072624+0000","last_active":"2023-09-30T22:14:52.550088+0000","last_peered":"2023-09-30T22:14:52.550088+0000","last_clean":"2023-09-30T22:14:52.550088+0000","last_became_active":"2023-09-30T22:14:02.072363+0000","last_became_peered":"2023-09-30T22:14:02.072363+0000","last_unstale":"2023-09-30T22:14:52.550088+0000","last_undegraded":"2023-09-30T22:14:52.550088+0000","last_fullsized":"2023-09-30T22:14:52.550088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733914+0000","last_change":"2023-09-30T22:14:01.709398+0000","last_active":"2023-09-30T22:14:52.733914+0000","last_peered":"2023-09-30T22:14:52.733914+0000","last_clean":"2023-09-30T22:14:52.733914+0000","last_became_active":"2023-09-30T22:14:01.709110+0000","last_became_peered":"2023-09-30T22:14:01.709110+0000","last_unstale":"2023-09-30T22:14:52.733914+0000","last_undegraded":"2023-09-30T22:14:52.733914+0000","last_fullsized":"2023-09-30T22:14:52.733914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550287+0000","last_change":"2023-09-30T22:14:02.074887+0000","last_active":"2023-09-30T22:14:52.550287+0000","last_peered":"2023-09-30T22:14:52.550287+0000","last_clean":"2023-09-30T22:14:52.550287+0000","last_became_active":"2023-09-30T22:14:02.074631+0000","last_became_peered":"2023-09-30T22:14:02.074631+0000","last_unstale":"2023-09-30T22:14:52.550287+0000","last_undegraded":"2023-09-30T22:14:52.550287+0000","last_fullsized":"2023-09-30T22:14:52.550287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733461+0000","last_change":"2023-09-30T22:14:01.707879+0000","last_active":"2023-09-30T22:14:52.733461+0000","last_peered":"2023-09-30T22:14:52.733461+0000","last_clean":"2023-09-30T22:14:52.733461+0000","last_became_active":"2023-09-30T22:14:01.707594+0000","last_became_peered":"2023-09-30T22:14:01.707594+0000","last_unstale":"2023-09-30T22:14:52.733461+0000","last_undegraded":"2023-09-30T22:14:52.733461+0000","last_fullsized":"2023-09-30T22:14:52.733461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733614+0000","last_change":"2023-09-30T22:14:02.076079+0000","last_active":"2023-09-30T22:14:52.733614+0000","last_peered":"2023-09-30T22:14:52.733614+0000","last_clean":"2023-09-30T22:14:52.733614+0000","last_became_active":"2023-09-30T22:14:02.075408+0000","last_became_peered":"2023-09-30T22:14:02.075408+0000","last_unstale":"2023-09-30T22:14:52.733614+0000","last_undegraded":"2023-09-30T22:14:52.733614+0000","last_fullsized":"2023-09-30T22:14:52.733614+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733270+0000","last_change":"2023-09-30T22:14:01.711605+0000","last_active":"2023-09-30T22:14:52.733270+0000","last_peered":"2023-09-30T22:14:52.733270+0000","last_clean":"2023-09-30T22:14:52.733270+0000","last_became_active":"2023-09-30T22:14:01.711375+0000","last_became_peered":"2023-09-30T22:14:01.711375+0000","last_unstale":"2023-09-30T22:14:52.733270+0000","last_undegraded":"2023-09-30T22:14:52.733270+0000","last_fullsized":"2023-09-30T22:14:52.733270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552671+0000","last_change":"2023-09-30T22:14:02.088858+0000","last_active":"2023-09-30T22:14:52.552671+0000","last_peered":"2023-09-30T22:14:52.552671+0000","last_clean":"2023-09-30T22:14:52.552671+0000","last_became_active":"2023-09-30T22:14:02.088583+0000","last_became_peered":"2023-09-30T22:14:02.088583+0000","last_unstale":"2023-09-30T22:14:52.552671+0000","last_undegraded":"2023-09-30T22:14:52.552671+0000","last_fullsized":"2023-09-30T22:14:52.552671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733195+0000","last_change":"2023-09-30T22:14:01.699847+0000","last_active":"2023-09-30T22:14:52.733195+0000","last_peered":"2023-09-30T22:14:52.733195+0000","last_clean":"2023-09-30T22:14:52.733195+0000","last_became_active":"2023-09-30T22:14:01.699587+0000","last_became_peered":"2023-09-30T22:14:01.699587+0000","last_unstale":"2023-09-30T22:14:52.733195+0000","last_undegraded":"2023-09-30T22:14:52.733195+0000","last_fullsized":"2023-09-30T22:14:52.733195+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552253+0000","last_change":"2023-09-30T22:14:02.094005+0000","last_active":"2023-09-30T22:14:52.552253+0000","last_peered":"2023-09-30T22:14:52.552253+0000","last_clean":"2023-09-30T22:14:52.552253+0000","last_became_active":"2023-09-30T22:14:02.093632+0000","last_became_peered":"2023-09-30T22:14:02.093632+0000","last_unstale":"2023-09-30T22:14:52.552253+0000","last_undegraded":"2023-09-30T22:14:52.552253+0000","last_fullsized":"2023-09-30T22:14:52.552253+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552302+0000","last_change":"2023-09-30T22:13:57.547923+0000","last_active":"2023-09-30T22:14:52.552302+0000","last_peered":"2023-09-30T22:14:52.552302+0000","last_clean":"2023-09-30T22:14:52.552302+0000","last_became_active":"2023-09-30T22:13:57.547569+0000","last_became_peered":"2023-09-30T22:13:57.547569+0000","last_unstale":"2023-09-30T22:14:52.552302+0000","last_undegraded":"2023-09-30T22:14:52.552302+0000","last_fullsized":"2023-09-30T22:14:52.552302+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549808+0000","last_change":"2023-09-30T22:14:01.702822+0000","last_active":"2023-09-30T22:14:52.549808+0000","last_peered":"2023-09-30T22:14:52.549808+0000","last_clean":"2023-09-30T22:14:52.549808+0000","last_became_active":"2023-09-30T22:14:01.702530+0000","last_became_peered":"2023-09-30T22:14:01.702530+0000","last_unstale":"2023-09-30T22:14:52.549808+0000","last_undegraded":"2023-09-30T22:14:52.549808+0000","last_fullsized":"2023-09-30T22:14:52.549808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551989+0000","last_change":"2023-09-30T22:14:02.084698+0000","last_active":"2023-09-30T22:14:52.551989+0000","last_peered":"2023-09-30T22:14:52.551989+0000","last_clean":"2023-09-30T22:14:52.551989+0000","last_became_active":"2023-09-30T22:14:02.084413+0000","last_became_peered":"2023-09-30T22:14:02.084413+0000","last_unstale":"2023-09-30T22:14:52.551989+0000","last_undegraded":"2023-09-30T22:14:52.551989+0000","last_fullsized":"2023-09-30T22:14:52.551989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733559+0000","last_change":"2023-09-30T22:13:57.550214+0000","last_active":"2023-09-30T22:14:52.733559+0000","last_peered":"2023-09-30T22:14:52.733559+0000","last_clean":"2023-09-30T22:14:52.733559+0000","last_became_active":"2023-09-30T22:13:57.549853+0000","last_became_peered":"2023-09-30T22:13:57.549853+0000","last_unstale":"2023-09-30T22:14:52.733559+0000","last_undegraded":"2023-09-30T22:14:52.733559+0000","last_fullsized":"2023-09-30T22:14:52.733559+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549896+0000","last_change":"2023-09-30T22:14:01.704198+0000","last_active":"2023-09-30T22:14:52.549896+0000","last_peered":"2023-09-30T22:14:52.549896+0000","last_clean":"2023-09-30T22:14:52.549896+0000","last_became_active":"2023-09-30T22:14:01.701691+0000","last_became_peered":"2023-09-30T22:14:01.701691+0000","last_unstale":"2023-09-30T22:14:52.549896+0000","last_undegraded":"2023-09-30T22:14:52.549896+0000","last_fullsized":"2023-09-30T22:14:52.549896+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552005+0000","last_change":"2023-09-30T22:14:02.076717+0000","last_active":"2023-09-30T22:14:52.552005+0000","last_peered":"2023-09-30T22:14:52.552005+0000","last_clean":"2023-09-30T22:14:52.552005+0000","last_became_active":"2023-09-30T22:14:02.076119+0000","last_became_peered":"2023-09-30T22:14:02.076119+0000","last_unstale":"2023-09-30T22:14:52.552005+0000","last_undegraded":"2023-09-30T22:14:52.552005+0000","last_fullsized":"2023-09-30T22:14:52.552005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458314+0000","last_change":"2023-09-30T22:13:57.550890+0000","last_active":"2023-09-30T22:16:19.458314+0000","last_peered":"2023-09-30T22:16:19.458314+0000","last_clean":"2023-09-30T22:16:19.458314+0000","last_became_active":"2023-09-30T22:13:57.550528+0000","last_became_peered":"2023-09-30T22:13:57.550528+0000","last_unstale":"2023-09-30T22:16:19.458314+0000","last_undegraded":"2023-09-30T22:16:19.458314+0000","last_fullsized":"2023-09-30T22:16:19.458314+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551062+0000","last_change":"2023-09-30T22:14:01.701431+0000","last_active":"2023-09-30T22:14:52.551062+0000","last_peered":"2023-09-30T22:14:52.551062+0000","last_clean":"2023-09-30T22:14:52.551062+0000","last_became_active":"2023-09-30T22:14:01.698771+0000","last_became_peered":"2023-09-30T22:14:01.698771+0000","last_unstale":"2023-09-30T22:14:52.551062+0000","last_undegraded":"2023-09-30T22:14:52.551062+0000","last_fullsized":"2023-09-30T22:14:52.551062+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552255+0000","last_change":"2023-09-30T22:14:02.085435+0000","last_active":"2023-09-30T22:14:52.552255+0000","last_peered":"2023-09-30T22:14:52.552255+0000","last_clean":"2023-09-30T22:14:52.552255+0000","last_became_active":"2023-09-30T22:14:02.085124+0000","last_became_peered":"2023-09-30T22:14:02.085124+0000","last_unstale":"2023-09-30T22:14:52.552255+0000","last_undegraded":"2023-09-30T22:14:52.552255+0000","last_fullsized":"2023-09-30T22:14:52.552255+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734015+0000","last_change":"2023-09-30T22:13:57.549984+0000","last_active":"2023-09-30T22:14:52.734015+0000","last_peered":"2023-09-30T22:14:52.734015+0000","last_clean":"2023-09-30T22:14:52.734015+0000","last_became_active":"2023-09-30T22:13:57.549645+0000","last_became_peered":"2023-09-30T22:13:57.549645+0000","last_unstale":"2023-09-30T22:14:52.734015+0000","last_undegraded":"2023-09-30T22:14:52.734015+0000","last_fullsized":"2023-09-30T22:14:52.734015+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550904+0000","last_change":"2023-09-30T22:13:56.543142+0000","last_active":"2023-09-30T22:14:52.550904+0000","last_peered":"2023-09-30T22:14:52.550904+0000","last_clean":"2023-09-30T22:14:52.550904+0000","last_became_active":"2023-09-30T22:13:56.542783+0000","last_became_peered":"2023-09-30T22:13:56.542783+0000","last_unstale":"2023-09-30T22:14:52.550904+0000","last_undegraded":"2023-09-30T22:14:52.550904+0000","last_fullsized":"2023-09-30T22:14:52.550904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:55.527769+0000","last_clean_scrub_stamp":"2023-09-30T22:13:55.527769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733334+0000","last_change":"2023-09-30T22:14:01.712014+0000","last_active":"2023-09-30T22:14:52.733334+0000","last_peered":"2023-09-30T22:14:52.733334+0000","last_clean":"2023-09-30T22:14:52.733334+0000","last_became_active":"2023-09-30T22:14:01.711770+0000","last_became_peered":"2023-09-30T22:14:01.711770+0000","last_unstale":"2023-09-30T22:14:52.733334+0000","last_undegraded":"2023-09-30T22:14:52.733334+0000","last_fullsized":"2023-09-30T22:14:52.733334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734010+0000","last_change":"2023-09-30T22:14:02.080800+0000","last_active":"2023-09-30T22:14:52.734010+0000","last_peered":"2023-09-30T22:14:52.734010+0000","last_clean":"2023-09-30T22:14:52.734010+0000","last_became_active":"2023-09-30T22:14:02.080440+0000","last_became_peered":"2023-09-30T22:14:02.080440+0000","last_unstale":"2023-09-30T22:14:52.734010+0000","last_undegraded":"2023-09-30T22:14:52.734010+0000","last_fullsized":"2023-09-30T22:14:52.734010+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733775+0000","last_change":"2023-09-30T22:13:57.550034+0000","last_active":"2023-09-30T22:14:52.733775+0000","last_peered":"2023-09-30T22:14:52.733775+0000","last_clean":"2023-09-30T22:14:52.733775+0000","last_became_active":"2023-09-30T22:13:57.549719+0000","last_became_peered":"2023-09-30T22:13:57.549719+0000","last_unstale":"2023-09-30T22:14:52.733775+0000","last_undegraded":"2023-09-30T22:14:52.733775+0000","last_fullsized":"2023-09-30T22:14:52.733775+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549535+0000","last_change":"2023-09-30T22:14:01.694407+0000","last_active":"2023-09-30T22:14:52.549535+0000","last_peered":"2023-09-30T22:14:52.549535+0000","last_clean":"2023-09-30T22:14:52.549535+0000","last_became_active":"2023-09-30T22:14:01.694022+0000","last_became_peered":"2023-09-30T22:14:01.694022+0000","last_unstale":"2023-09-30T22:14:52.549535+0000","last_undegraded":"2023-09-30T22:14:52.549535+0000","last_fullsized":"2023-09-30T22:14:52.549535+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551372+0000","last_change":"2023-09-30T22:14:02.079540+0000","last_active":"2023-09-30T22:14:52.551372+0000","last_peered":"2023-09-30T22:14:52.551372+0000","last_clean":"2023-09-30T22:14:52.551372+0000","last_became_active":"2023-09-30T22:14:02.079016+0000","last_became_peered":"2023-09-30T22:14:02.079016+0000","last_unstale":"2023-09-30T22:14:52.551372+0000","last_undegraded":"2023-09-30T22:14:52.551372+0000","last_fullsized":"2023-09-30T22:14:52.551372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733752+0000","last_change":"2023-09-30T22:13:57.552702+0000","last_active":"2023-09-30T22:14:52.733752+0000","last_peered":"2023-09-30T22:14:52.733752+0000","last_clean":"2023-09-30T22:14:52.733752+0000","last_became_active":"2023-09-30T22:13:57.549736+0000","last_became_peered":"2023-09-30T22:13:57.549736+0000","last_unstale":"2023-09-30T22:14:52.733752+0000","last_undegraded":"2023-09-30T22:14:52.733752+0000","last_fullsized":"2023-09-30T22:14:52.733752+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551911+0000","last_change":"2023-09-30T22:14:01.695717+0000","last_active":"2023-09-30T22:14:52.551911+0000","last_peered":"2023-09-30T22:14:52.551911+0000","last_clean":"2023-09-30T22:14:52.551911+0000","last_became_active":"2023-09-30T22:14:01.694990+0000","last_became_peered":"2023-09-30T22:14:01.694990+0000","last_unstale":"2023-09-30T22:14:52.551911+0000","last_undegraded":"2023-09-30T22:14:52.551911+0000","last_fullsized":"2023-09-30T22:14:52.551911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550011+0000","last_change":"2023-09-30T22:14:02.068852+0000","last_active":"2023-09-30T22:14:52.550011+0000","last_peered":"2023-09-30T22:14:52.550011+0000","last_clean":"2023-09-30T22:14:52.550011+0000","last_became_active":"2023-09-30T22:14:02.068531+0000","last_became_peered":"2023-09-30T22:14:02.068531+0000","last_unstale":"2023-09-30T22:14:52.550011+0000","last_undegraded":"2023-09-30T22:14:52.550011+0000","last_fullsized":"2023-09-30T22:14:52.550011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733800+0000","last_change":"2023-09-30T22:14:01.700959+0000","last_active":"2023-09-30T22:14:52.733800+0000","last_peered":"2023-09-30T22:14:52.733800+0000","last_clean":"2023-09-30T22:14:52.733800+0000","last_became_active":"2023-09-30T22:14:01.700568+0000","last_became_peered":"2023-09-30T22:14:01.700568+0000","last_unstale":"2023-09-30T22:14:52.733800+0000","last_undegraded":"2023-09-30T22:14:52.733800+0000","last_fullsized":"2023-09-30T22:14:52.733800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552770+0000","last_change":"2023-09-30T22:13:57.547843+0000","last_active":"2023-09-30T22:14:52.552770+0000","last_peered":"2023-09-30T22:14:52.552770+0000","last_clean":"2023-09-30T22:14:52.552770+0000","last_became_active":"2023-09-30T22:13:57.547587+0000","last_became_peered":"2023-09-30T22:13:57.547587+0000","last_unstale":"2023-09-30T22:14:52.552770+0000","last_undegraded":"2023-09-30T22:14:52.552770+0000","last_fullsized":"2023-09-30T22:14:52.552770+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733297+0000","last_change":"2023-09-30T22:14:02.081943+0000","last_active":"2023-09-30T22:14:52.733297+0000","last_peered":"2023-09-30T22:14:52.733297+0000","last_clean":"2023-09-30T22:14:52.733297+0000","last_became_active":"2023-09-30T22:14:02.080899+0000","last_became_peered":"2023-09-30T22:14:02.080899+0000","last_unstale":"2023-09-30T22:14:52.733297+0000","last_undegraded":"2023-09-30T22:14:52.733297+0000","last_fullsized":"2023-09-30T22:14:52.733297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733844+0000","last_change":"2023-09-30T22:13:57.551269+0000","last_active":"2023-09-30T22:14:52.733844+0000","last_peered":"2023-09-30T22:14:52.733844+0000","last_clean":"2023-09-30T22:14:52.733844+0000","last_became_active":"2023-09-30T22:13:57.550734+0000","last_became_peered":"2023-09-30T22:13:57.550734+0000","last_unstale":"2023-09-30T22:14:52.733844+0000","last_undegraded":"2023-09-30T22:14:52.733844+0000","last_fullsized":"2023-09-30T22:14:52.733844+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:13:56.528787+0000","last_clean_scrub_stamp":"2023-09-30T22:13:56.528787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551086+0000","last_change":"2023-09-30T22:14:01.706704+0000","last_active":"2023-09-30T22:14:52.551086+0000","last_peered":"2023-09-30T22:14:52.551086+0000","last_clean":"2023-09-30T22:14:52.551086+0000","last_became_active":"2023-09-30T22:14:01.706387+0000","last_became_peered":"2023-09-30T22:14:01.706387+0000","last_unstale":"2023-09-30T22:14:52.551086+0000","last_undegraded":"2023-09-30T22:14:52.551086+0000","last_fullsized":"2023-09-30T22:14:52.551086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458167+0000","last_change":"2023-09-30T22:14:02.088343+0000","last_active":"2023-09-30T22:16:19.458167+0000","last_peered":"2023-09-30T22:16:19.458167+0000","last_clean":"2023-09-30T22:16:19.458167+0000","last_became_active":"2023-09-30T22:14:02.088142+0000","last_became_peered":"2023-09-30T22:14:02.088142+0000","last_unstale":"2023-09-30T22:16:19.458167+0000","last_undegraded":"2023-09-30T22:16:19.458167+0000","last_fullsized":"2023-09-30T22:16:19.458167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549939+0000","last_change":"2023-09-30T22:14:01.702235+0000","last_active":"2023-09-30T22:14:52.549939+0000","last_peered":"2023-09-30T22:14:52.549939+0000","last_clean":"2023-09-30T22:14:52.549939+0000","last_became_active":"2023-09-30T22:14:01.701972+0000","last_became_peered":"2023-09-30T22:14:01.701972+0000","last_unstale":"2023-09-30T22:14:52.549939+0000","last_undegraded":"2023-09-30T22:14:52.549939+0000","last_fullsized":"2023-09-30T22:14:52.549939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.550032+0000","last_change":"2023-09-30T22:14:02.078694+0000","last_active":"2023-09-30T22:14:52.550032+0000","last_peered":"2023-09-30T22:14:52.550032+0000","last_clean":"2023-09-30T22:14:52.550032+0000","last_became_active":"2023-09-30T22:14:02.078435+0000","last_became_peered":"2023-09-30T22:14:02.078435+0000","last_unstale":"2023-09-30T22:14:52.550032+0000","last_undegraded":"2023-09-30T22:14:52.550032+0000","last_fullsized":"2023-09-30T22:14:52.550032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552103+0000","last_change":"2023-09-30T22:14:01.709428+0000","last_active":"2023-09-30T22:14:52.552103+0000","last_peered":"2023-09-30T22:14:52.552103+0000","last_clean":"2023-09-30T22:14:52.552103+0000","last_became_active":"2023-09-30T22:14:01.709089+0000","last_became_peered":"2023-09-30T22:14:01.709089+0000","last_unstale":"2023-09-30T22:14:52.552103+0000","last_undegraded":"2023-09-30T22:14:52.552103+0000","last_fullsized":"2023-09-30T22:14:52.552103+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458369+0000","last_change":"2023-09-30T22:14:02.087499+0000","last_active":"2023-09-30T22:16:19.458369+0000","last_peered":"2023-09-30T22:16:19.458369+0000","last_clean":"2023-09-30T22:16:19.458369+0000","last_became_active":"2023-09-30T22:14:02.086941+0000","last_became_peered":"2023-09-30T22:14:02.086941+0000","last_unstale":"2023-09-30T22:16:19.458369+0000","last_undegraded":"2023-09-30T22:16:19.458369+0000","last_fullsized":"2023-09-30T22:16:19.458369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733854+0000","last_change":"2023-09-30T22:14:01.709677+0000","last_active":"2023-09-30T22:14:52.733854+0000","last_peered":"2023-09-30T22:14:52.733854+0000","last_clean":"2023-09-30T22:14:52.733854+0000","last_became_active":"2023-09-30T22:14:01.709115+0000","last_became_peered":"2023-09-30T22:14:01.709115+0000","last_unstale":"2023-09-30T22:14:52.733854+0000","last_undegraded":"2023-09-30T22:14:52.733854+0000","last_fullsized":"2023-09-30T22:14:52.733854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458278+0000","last_change":"2023-09-30T22:14:02.073683+0000","last_active":"2023-09-30T22:16:19.458278+0000","last_peered":"2023-09-30T22:16:19.458278+0000","last_clean":"2023-09-30T22:16:19.458278+0000","last_became_active":"2023-09-30T22:14:02.073269+0000","last_became_peered":"2023-09-30T22:14:02.073269+0000","last_unstale":"2023-09-30T22:16:19.458278+0000","last_undegraded":"2023-09-30T22:16:19.458278+0000","last_fullsized":"2023-09-30T22:16:19.458278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734275+0000","last_change":"2023-09-30T22:14:01.711766+0000","last_active":"2023-09-30T22:14:52.734275+0000","last_peered":"2023-09-30T22:14:52.734275+0000","last_clean":"2023-09-30T22:14:52.734275+0000","last_became_active":"2023-09-30T22:14:01.711524+0000","last_became_peered":"2023-09-30T22:14:01.711524+0000","last_unstale":"2023-09-30T22:14:52.734275+0000","last_undegraded":"2023-09-30T22:14:52.734275+0000","last_fullsized":"2023-09-30T22:14:52.734275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734147+0000","last_change":"2023-09-30T22:14:02.082106+0000","last_active":"2023-09-30T22:14:52.734147+0000","last_peered":"2023-09-30T22:14:52.734147+0000","last_clean":"2023-09-30T22:14:52.734147+0000","last_became_active":"2023-09-30T22:14:02.081789+0000","last_became_peered":"2023-09-30T22:14:02.081789+0000","last_unstale":"2023-09-30T22:14:52.734147+0000","last_undegraded":"2023-09-30T22:14:52.734147+0000","last_fullsized":"2023-09-30T22:14:52.734147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552476+0000","last_change":"2023-09-30T22:14:01.702868+0000","last_active":"2023-09-30T22:14:52.552476+0000","last_peered":"2023-09-30T22:14:52.552476+0000","last_clean":"2023-09-30T22:14:52.552476+0000","last_became_active":"2023-09-30T22:14:01.702431+0000","last_became_peered":"2023-09-30T22:14:01.702431+0000","last_unstale":"2023-09-30T22:14:52.552476+0000","last_undegraded":"2023-09-30T22:14:52.552476+0000","last_fullsized":"2023-09-30T22:14:52.552476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:00.666720+0000","last_clean_scrub_stamp":"2023-09-30T22:14:00.666720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:16:19.458160+0000","last_change":"2023-09-30T22:14:02.081561+0000","last_active":"2023-09-30T22:16:19.458160+0000","last_peered":"2023-09-30T22:16:19.458160+0000","last_clean":"2023-09-30T22:16:19.458160+0000","last_became_active":"2023-09-30T22:14:02.081199+0000","last_became_peered":"2023-09-30T22:14:02.081199+0000","last_unstale":"2023-09-30T22:16:19.458160+0000","last_undegraded":"2023-09-30T22:16:19.458160+0000","last_fullsized":"2023-09-30T22:16:19.458160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.551152+0000","last_change":"2023-09-30T22:14:02.079397+0000","last_active":"2023-09-30T22:14:52.551152+0000","last_peered":"2023-09-30T22:14:52.551152+0000","last_clean":"2023-09-30T22:14:52.551152+0000","last_became_active":"2023-09-30T22:14:02.079036+0000","last_became_peered":"2023-09-30T22:14:02.079036+0000","last_unstale":"2023-09-30T22:14:52.551152+0000","last_undegraded":"2023-09-30T22:14:52.551152+0000","last_fullsized":"2023-09-30T22:14:52.551152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.733356+0000","last_change":"2023-09-30T22:14:02.081974+0000","last_active":"2023-09-30T22:14:52.733356+0000","last_peered":"2023-09-30T22:14:52.733356+0000","last_clean":"2023-09-30T22:14:52.733356+0000","last_became_active":"2023-09-30T22:14:02.081491+0000","last_became_peered":"2023-09-30T22:14:02.081491+0000","last_unstale":"2023-09-30T22:14:52.733356+0000","last_undegraded":"2023-09-30T22:14:52.733356+0000","last_fullsized":"2023-09-30T22:14:52.733356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.549720+0000","last_change":"2023-09-30T22:14:02.067554+0000","last_active":"2023-09-30T22:14:52.549720+0000","last_peered":"2023-09-30T22:14:52.549720+0000","last_clean":"2023-09-30T22:14:52.549720+0000","last_became_active":"2023-09-30T22:14:02.067283+0000","last_became_peered":"2023-09-30T22:14:02.067283+0000","last_unstale":"2023-09-30T22:14:52.549720+0000","last_undegraded":"2023-09-30T22:14:52.549720+0000","last_fullsized":"2023-09-30T22:14:52.549720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.734107+0000","last_change":"2023-09-30T22:14:02.082875+0000","last_active":"2023-09-30T22:14:52.734107+0000","last_peered":"2023-09-30T22:14:52.734107+0000","last_clean":"2023-09-30T22:14:52.734107+0000","last_became_active":"2023-09-30T22:14:02.082617+0000","last_became_peered":"2023-09-30T22:14:02.082617+0000","last_unstale":"2023-09-30T22:14:52.734107+0000","last_undegraded":"2023-09-30T22:14:52.734107+0000","last_fullsized":"2023-09-30T22:14:52.734107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:14:52.552207+0000","last_change":"2023-09-30T22:14:02.076951+0000","last_active":"2023-09-30T22:14:52.552207+0000","last_peered":"2023-09-30T22:14:52.552207+0000","last_clean":"2023-09-30T22:14:52.552207+0000","last_became_active":"2023-09-30T22:14:02.076448+0000","last_became_peered":"2023-09-30T22:14:02.076448+0000","last_unstale":"2023-09-30T22:14:52.552207+0000","last_undegraded":"2023-09-30T22:14:52.552207+0000","last_fullsized":"2023-09-30T22:14:52.552207+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:14:01.670302+0000","last_clean_scrub_stamp":"2023-09-30T22:14:01.670302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297292,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074548,"statfs":{"total":96636764160,"available":96332337152,"internally_reserved":0,"allocated":200704,"data_stored":67217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.28499999999999998},"min":{"1min":0.20000000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.46999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.224},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.312,"15min":0.311},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.28000000000000003,"15min":0.27800000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.47499999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.29499999999999998,"15min":0.29699999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44600000000000001,"5min":0.497,"15min":0.497},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.30099999999999999,"15min":0.29899999999999999},"min":{"1min":0.23100000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48899999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.33300000000000002,"15min":0.33900000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.498,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.315}]}]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":67023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"min":{"1min":0.158,"5min":0.158,"15min":0.158},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.379}]},{"osd":1,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.22500000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29699999999999999},"min":{"1min":0.19900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.36199999999999999}]},{"osd":3,"last update":"Sat Sep 30 22:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.34599999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705694,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":60178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 22:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.29199999999999998,"15min":0.28899999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.318},{"interface":"front","average":{"1min":0.316,"5min":0.30399999999999999,"15min":0.30199999999999999},"min":{"1min":0.22700000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Sep 30 22:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.309,"15min":0.30499999999999999},"min":{"1min":0.22800000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30199999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26900000000000002}]},{"osd":3,"last update":"Sat Sep 30 22:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.251},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33200000000000002,"15min":0.33000000000000002},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.42399999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705695,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":64106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.28199999999999997,"5min":0.30199999999999999,"15min":0.30599999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.307,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.22600000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.41999999999999998,"5min":0.434,"15min":0.434},"last":0.32300000000000001}]},{"osd":2,"last update":"Sat Sep 30 22:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.30499999999999999,"15min":0.30599999999999999},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.39700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.309,"5min":0.33000000000000002,"15min":0.33400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.42699999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.30599999999999999}]},{"osd":3,"last update":"Sat Sep 30 22:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.311,"15min":0.313},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.33800000000000002,"15min":0.34000000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.442,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.315}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:16:22.859 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T22:16:22.859 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T22:16:22.859 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-30T22:16:22.859 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T22:16:23.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.013+0000 7f4e7f653700 1 Processor -- start 2023-09-30T22:16:23.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.013+0000 7f4e7f653700 1 -- start start 2023-09-30T22:16:23.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.013+0000 7f4e7f653700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e78130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.013+0000 7f4e7f653700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e78130e50 0x7f4e7806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.013+0000 7f4e7f653700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e780716a0 con 0x7f4e7806fb20 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.013+0000 7f4e7f653700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7809a390 con 0x7f4e78130e50 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.013+0000 7f4e7f653700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78071290 con 0x7f4e78130540 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.013+0000 7f4e7d3ef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e78130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7cbee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e78130e50 0x7f4e7806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7dbf0700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4e7806fb20 0x7f4e78137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52920/0 (socket says 172.21.15.89:52920) 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7dbf0700 1 -- 172.21.15.89:0/1003583333 learned_addr learned my addr 172.21.15.89:0/1003583333 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7d3ef700 1 --2- 172.21.15.89:0/1003583333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e78130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46076/0 (socket says 172.21.15.89:46076) 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7d3ef700 1 -- 172.21.15.89:0/1003583333 >> v2:172.21.15.89:3300/0 conn(0x7f4e78130e50 msgr2=0x7f4e7806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7d3ef700 1 --2- 172.21.15.89:0/1003583333 >> v2:172.21.15.89:3300/0 conn(0x7f4e78130e50 0x7f4e7806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7d3ef700 1 -- 172.21.15.89:0/1003583333 >> v1:172.21.15.89:6789/0 conn(0x7f4e7806fb20 legacy=0x7f4e78137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7d3ef700 1 -- 172.21.15.89:0/1003583333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e78130340 con 0x7f4e78130540 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e6ffff700 1 -- 172.21.15.89:0/1003583333 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191819828 0 0) 0x7f4e780716a0 con 0x7f4e7806fb20 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7cbee700 1 --2- 172.21.15.89:0/1003583333 >> v2:172.21.15.89:3300/0 conn(0x7f4e78130e50 0x7f4e7806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e7d3ef700 1 --2- 172.21.15.89:0/1003583333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e78130910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009870 tx=0x7f4e74009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b6e65898bdf6618 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e6ffff700 1 -- 172.21.15.89:0/1003583333 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74011830 con 0x7f4e78130540 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e6ffff700 1 -- 172.21.15.89:0/1003583333 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e74011990 con 0x7f4e78130540 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.014+0000 7f4e6ffff700 1 -- 172.21.15.89:0/1003583333 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74011d20 con 0x7f4e78130540 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.015+0000 7f4e7f653700 1 -- 172.21.15.89:0/1003583333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 msgr2=0x7f4e78130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.015+0000 7f4e7f653700 1 --2- 172.21.15.89:0/1003583333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e78130910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009870 tx=0x7f4e74009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.015+0000 7f4e7f653700 1 -- 172.21.15.89:0/1003583333 shutdown_connections 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.015+0000 7f4e7f653700 1 --2- 172.21.15.89:0/1003583333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e78130910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.015+0000 7f4e7f653700 1 --2- 172.21.15.89:0/1003583333 >> v2:172.21.15.89:3300/0 conn(0x7f4e78130e50 0x7f4e7806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.015+0000 7f4e7f653700 1 -- 172.21.15.89:0/1003583333 >> 172.21.15.89:0/1003583333 conn(0x7f4e780a2aa0 msgr2=0x7f4e780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:23.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.015+0000 7f4e7f653700 1 -- 172.21.15.89:0/1003583333 shutdown_connections 2023-09-30T22:16:23.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.015+0000 7f4e7f653700 1 -- 172.21.15.89:0/1003583333 wait complete. 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7f653700 1 Processor -- start 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7f653700 1 -- start start 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7f653700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7806fb20 0x7f4e7811ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7f653700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e7811f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7f653700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e78130e50 0x7f4e7811f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7f653700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780a0810 con 0x7f4e78130e50 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7f653700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780718e0 con 0x7f4e7806fb20 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7f653700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78122000 con 0x7f4e78130540 2023-09-30T22:16:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7dbf0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e78130e50 0x7f4e7811f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7cbee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e7811f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7d3ef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7806fb20 0x7f4e7811ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7dbf0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e78130e50 0x7f4e7811f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60914/0 (socket says 172.21.15.89:60914) 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7cbee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e7811f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46082/0 (socket says 172.21.15.89:46082) 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.016+0000 7f4e7dbf0700 1 -- 172.21.15.89:0/364684986 learned_addr learned my addr 172.21.15.89:0/364684986 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e7dbf0700 1 -- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 msgr2=0x7f4e7811f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e7dbf0700 1 --2- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e7811f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e7dbf0700 1 -- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7806fb20 msgr2=0x7f4e7811ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e7dbf0700 1 --2- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7806fb20 0x7f4e7811ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e7dbf0700 1 -- 172.21.15.89:0/364684986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6800b740 con 0x7f4e78130e50 2023-09-30T22:16:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e7dbf0700 1 --2- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e78130e50 0x7f4e7811f7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e70002230 tx=0x7f4e7000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e6dffb700 1 -- 172.21.15.89:0/364684986 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7000aac0 con 0x7f4e78130e50 2023-09-30T22:16:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e6dffb700 1 -- 172.21.15.89:0/364684986 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7000ac40 con 0x7f4e78130e50 2023-09-30T22:16:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e740094a0 con 0x7f4e78130e50 2023-09-30T22:16:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e6dffb700 1 -- 172.21.15.89:0/364684986 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e700070d0 con 0x7f4e78130e50 2023-09-30T22:16:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.017+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e78121110 con 0x7f4e78130e50 2023-09-30T22:16:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.018+0000 7f4e6dffb700 1 -- 172.21.15.89:0/364684986 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f4e70007250 con 0x7f4e78130e50 2023-09-30T22:16:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.018+0000 7f4e6dffb700 1 -- 172.21.15.89:0/364684986 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f4e7003f880 con 0x7f4e78130e50 2023-09-30T22:16:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.018+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e78074ea0 con 0x7f4e78130e50 2023-09-30T22:16:23.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.023+0000 7f4e6dffb700 1 -- 172.21.15.89:0/364684986 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4e70018020 con 0x7f4e78130e50 2023-09-30T22:16:23.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.248+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4e7806b9e0 con 0x7f4e78130e50 2023-09-30T22:16:23.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.248+0000 7f4e6dffb700 1 -- 172.21.15.89:0/364684986 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4e70007500 con 0x7f4e78130e50 2023-09-30T22:16:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:23.251 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-30T22:16:23.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 >> v1:172.21.15.89:6816/126953 conn(0x7f4e5c02f480 legacy=0x7f4e5c031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e78130e50 msgr2=0x7f4e7811f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 --2- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e78130e50 0x7f4e7811f7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e70002230 tx=0x7f4e7000cf50 comp rx=0 tx=0).stop 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 shutdown_connections 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 --2- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e78130e50 0x7f4e7811f7d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 --2- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7806fb20 0x7f4e7811ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 --2- 172.21.15.89:0/364684986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e78130540 0x7f4e7811f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 >> 172.21.15.89:0/364684986 conn(0x7f4e780a2aa0 msgr2=0x7f4e7806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 shutdown_connections 2023-09-30T22:16:23.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.250+0000 7f4e7f653700 1 -- 172.21.15.89:0/364684986 wait complete. 2023-09-30T22:16:23.261 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-30T22:16:23.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:23.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.383+0000 7f570b80c700 1 Processor -- start 2023-09-30T22:16:23.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.383+0000 7f570b80c700 1 -- start start 2023-09-30T22:16:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.383+0000 7f570b80c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f57040a0a30 0x7f57040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.383+0000 7f570b80c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5704138d80 0x7f570413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.383+0000 7f570b80c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57040a2a70 con 0x7f570409d630 2023-09-30T22:16:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.383+0000 7f570b80c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57040a01d0 con 0x7f57040a0a30 2023-09-30T22:16:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.383+0000 7f570b80c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570409e2b0 con 0x7f5704138d80 2023-09-30T22:16:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f5709da9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5704138d80 0x7f570413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f57095a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f57040a0a30 0x7f57040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f5708da7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f570409d630 0x7f57040a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52924/0 (socket says 172.21.15.89:52924) 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f5708da7700 1 -- 172.21.15.89:0/1430604886 learned_addr learned my addr 172.21.15.89:0/1430604886 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f5709da9700 1 -- 172.21.15.89:0/1430604886 >> v2:172.21.15.89:3300/0 conn(0x7f57040a0a30 msgr2=0x7f57040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f5709da9700 1 --2- 172.21.15.89:0/1430604886 >> v2:172.21.15.89:3300/0 conn(0x7f57040a0a30 0x7f57040a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f5709da9700 1 -- 172.21.15.89:0/1430604886 >> v1:172.21.15.89:6789/0 conn(0x7f570409d630 legacy=0x7f57040a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f5709da9700 1 -- 172.21.15.89:0/1430604886 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570409dc50 con 0x7f5704138d80 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f57095a8700 1 --2- 172.21.15.89:0/1430604886 >> v2:172.21.15.89:3300/0 conn(0x7f57040a0a30 0x7f57040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f5709da9700 1 --2- 172.21.15.89:0/1430604886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5704138d80 0x7f570413b160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009d90 tx=0x7f56fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e994837b8439f5ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f56fbfff700 1 -- 172.21.15.89:0/1430604886 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00e070 con 0x7f5704138d80 2023-09-30T22:16:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f56fbfff700 1 -- 172.21.15.89:0/1430604886 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc00bb20 con 0x7f5704138d80 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.384+0000 7f56fbfff700 1 -- 172.21.15.89:0/1430604886 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc012430 con 0x7f5704138d80 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.385+0000 7f570b80c700 1 -- 172.21.15.89:0/1430604886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5704138d80 msgr2=0x7f570413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.385+0000 7f570b80c700 1 --2- 172.21.15.89:0/1430604886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5704138d80 0x7f570413b160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009d90 tx=0x7f56fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.385+0000 7f570b80c700 1 -- 172.21.15.89:0/1430604886 shutdown_connections 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.385+0000 7f570b80c700 1 --2- 172.21.15.89:0/1430604886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5704138d80 0x7f570413b160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.385+0000 7f570b80c700 1 --2- 172.21.15.89:0/1430604886 >> v2:172.21.15.89:3300/0 conn(0x7f57040a0a30 0x7f57040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.385+0000 7f570b80c700 1 -- 172.21.15.89:0/1430604886 >> 172.21.15.89:0/1430604886 conn(0x7f57040a3050 msgr2=0x7f57040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.385+0000 7f570b80c700 1 -- 172.21.15.89:0/1430604886 shutdown_connections 2023-09-30T22:16:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.385+0000 7f570b80c700 1 -- 172.21.15.89:0/1430604886 wait complete. 2023-09-30T22:16:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.386+0000 7f570b80c700 1 Processor -- start 2023-09-30T22:16:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.386+0000 7f570b80c700 1 -- start start 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f570b80c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f570409d630 0x7f5704133a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f570b80c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57040a0a30 0x7f5704133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f570b80c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5704138d80 0x7f57041344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f570b80c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5704136b90 con 0x7f57040a0a30 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f570b80c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5704138090 con 0x7f570409d630 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f570b80c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57041374b0 con 0x7f5704138d80 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f570409d630 0x7f5704133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f5708da7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57040a0a30 0x7f5704133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f570409d630 0x7f5704133a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41918/0 (socket says 172.21.15.89:41918) 2023-09-30T22:16:23.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 -- 172.21.15.89:0/662730933 learned_addr learned my addr 172.21.15.89:0/662730933 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f5709da9700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5704138d80 0x7f57041344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 -- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5704138d80 msgr2=0x7f57041344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5704138d80 0x7f57041344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 -- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57040a0a30 msgr2=0x7f5704133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57040a0a30 0x7f5704133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 -- 172.21.15.89:0/662730933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57000094a0 con 0x7f570409d630 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f5708da7700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57040a0a30 0x7f5704133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f57095a8700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f570409d630 0x7f5704133a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f570000ebe0 tx=0x7f570000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f56f9ffb700 1 -- 172.21.15.89:0/662730933 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570000c850 con 0x7f570409d630 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f56f9ffb700 1 -- 172.21.15.89:0/662730933 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570000c9b0 con 0x7f570409d630 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc009a70 con 0x7f570409d630 2023-09-30T22:16:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.387+0000 7f56f9ffb700 1 -- 172.21.15.89:0/662730933 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570000cd40 con 0x7f570409d630 2023-09-30T22:16:23.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.388+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57041371d0 con 0x7f570409d630 2023-09-30T22:16:23.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.388+0000 7f56f9ffb700 1 -- 172.21.15.89:0/662730933 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f5700013470 con 0x7f570409d630 2023-09-30T22:16:23.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.388+0000 7f56f9ffb700 1 -- 172.21.15.89:0/662730933 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f570003fef0 con 0x7f570409d630 2023-09-30T22:16:23.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.389+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5704137030 con 0x7f570409d630 2023-09-30T22:16:23.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.393+0000 7f56f9ffb700 1 -- 172.21.15.89:0/662730933 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5704137030 con 0x7f570409d630 2023-09-30T22:16:23.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.580+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57040747f0 con 0x7f570409d630 2023-09-30T22:16:23.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.581+0000 7f56f9ffb700 1 -- 172.21.15.89:0/662730933 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12866 (secure 0 0 0) 0x7f570002b090 con 0x7f570409d630 2023-09-30T22:16:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:23.581 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":23,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:21.418378+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:23.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 >> v1:172.21.15.89:6816/126953 conn(0x7f56e802f480 legacy=0x7f56e8031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f570409d630 msgr2=0x7f5704133a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f570409d630 0x7f5704133a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f570000ebe0 tx=0x7f570000eef0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 shutdown_connections 2023-09-30T22:16:23.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5704138d80 0x7f57041344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f570409d630 0x7f5704133a30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 --2- 172.21.15.89:0/662730933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57040a0a30 0x7f5704133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 >> 172.21.15.89:0/662730933 conn(0x7f57040a3050 msgr2=0x7f570409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:23.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 shutdown_connections 2023-09-30T22:16:23.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.583+0000 7f570b80c700 1 -- 172.21.15.89:0/662730933 wait complete. 2023-09-30T22:16:23.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:14:00.533280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:14:01.011104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:16:23.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:16:23.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.719+0000 7f713a451700 1 Processor -- start 2023-09-30T22:16:23.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.719+0000 7f713a451700 1 -- start start 2023-09-30T22:16:23.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.719+0000 7f713a451700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f7134130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.719+0000 7f713a451700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f713406b290 0x7f713413b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.719+0000 7f713a451700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713409f620 con 0x7f713406a770 2023-09-30T22:16:23.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.719+0000 7f713a451700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7134130280 con 0x7f713406b290 2023-09-30T22:16:23.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.719+0000 7f713a451700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713409e1b0 con 0x7f71341303c0 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f7133fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f7134130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f71389ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f713406b290 0x7f713413b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f7133fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f7134130790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41926/0 (socket says 172.21.15.89:41926) 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f71389ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f713406b290 0x7f713413b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60954/0 (socket says 172.21.15.89:60954) 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f71337fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f713406a770 0x7f713406ab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52932/0 (socket says 172.21.15.89:52932) 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f7133fff700 1 -- 172.21.15.89:0/3213784517 learned_addr learned my addr 172.21.15.89:0/3213784517 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f71389ee700 1 -- 172.21.15.89:0/3213784517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 msgr2=0x7f7134130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f71389ee700 1 --2- 172.21.15.89:0/3213784517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f7134130790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f71389ee700 1 -- 172.21.15.89:0/3213784517 >> v1:172.21.15.89:6789/0 conn(0x7f713406a770 legacy=0x7f713406ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f71389ee700 1 -- 172.21.15.89:0/3213784517 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7134130cd0 con 0x7f713406b290 2023-09-30T22:16:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f7132ffd700 1 -- 172.21.15.89:0/3213784517 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559887661 0 0) 0x7f713409f620 con 0x7f713406a770 2023-09-30T22:16:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f7133fff700 1 --2- 172.21.15.89:0/3213784517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f7134130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f71389ee700 1 --2- 172.21.15.89:0/3213784517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713413b7b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f712c009d90 tx=0x7f712c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe8a123a3b51753 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f7132ffd700 1 -- 172.21.15.89:0/3213784517 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00e070 con 0x7f713406b290 2023-09-30T22:16:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f7132ffd700 1 -- 172.21.15.89:0/3213784517 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c00bb20 con 0x7f713406b290 2023-09-30T22:16:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.720+0000 7f7132ffd700 1 -- 172.21.15.89:0/3213784517 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c012430 con 0x7f713406b290 2023-09-30T22:16:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.721+0000 7f713a451700 1 -- 172.21.15.89:0/3213784517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 msgr2=0x7f713413b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.721+0000 7f713a451700 1 --2- 172.21.15.89:0/3213784517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713413b7b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f712c009d90 tx=0x7f712c00b330 comp rx=0 tx=0).stop 2023-09-30T22:16:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.721+0000 7f713a451700 1 -- 172.21.15.89:0/3213784517 shutdown_connections 2023-09-30T22:16:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.721+0000 7f713a451700 1 --2- 172.21.15.89:0/3213784517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713413b7b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.721+0000 7f713a451700 1 --2- 172.21.15.89:0/3213784517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f7134130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.721+0000 7f713a451700 1 -- 172.21.15.89:0/3213784517 >> 172.21.15.89:0/3213784517 conn(0x7f7134066d70 msgr2=0x7f71340a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.721+0000 7f713a451700 1 -- 172.21.15.89:0/3213784517 shutdown_connections 2023-09-30T22:16:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.721+0000 7f713a451700 1 -- 172.21.15.89:0/3213784517 wait complete. 2023-09-30T22:16:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f713a451700 1 Processor -- start 2023-09-30T22:16:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f713a451700 1 -- start start 2023-09-30T22:16:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f713a451700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f713406a770 0x7f713411d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f713a451700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713411de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f713a451700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f713411e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f713a451700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713412db20 con 0x7f713406b290 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f713a451700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713411fc50 con 0x7f71341303c0 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f713a451700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f713412d620 con 0x7f713406a770 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f71337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713411de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f7133fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f713406a770 0x7f713411d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f71337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713411de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60964/0 (socket says 172.21.15.89:60964) 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.722+0000 7f71337fe700 1 -- 172.21.15.89:0/2360330226 learned_addr learned my addr 172.21.15.89:0/2360330226 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f71389ee700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f713411e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7133fff700 1 -- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 msgr2=0x7f713411e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7133fff700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f713411e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7133fff700 1 -- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 msgr2=0x7f713411de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7133fff700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713411de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7133fff700 1 -- 172.21.15.89:0/2360330226 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128009550 con 0x7f713406a770 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f71337fe700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713411de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7133fff700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f713406a770 0x7f713411d930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f712800e9a0 tx=0x7f712800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7130ff9700 1 -- 172.21.15.89:0/2360330226 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128004030 con 0x7f713406a770 2023-09-30T22:16:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7130ff9700 1 -- 172.21.15.89:0/2360330226 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128004190 con 0x7f713406a770 2023-09-30T22:16:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f7130ff9700 1 -- 172.21.15.89:0/2360330226 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128004520 con 0x7f713406a770 2023-09-30T22:16:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c009a70 con 0x7f713406a770 2023-09-30T22:16:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.723+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713409a0c0 con 0x7f713406a770 2023-09-30T22:16:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.724+0000 7f7130ff9700 1 -- 172.21.15.89:0/2360330226 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f71280036a0 con 0x7f713406a770 2023-09-30T22:16:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.724+0000 7f7130ff9700 1 -- 172.21.15.89:0/2360330226 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f712803fc50 con 0x7f713406a770 2023-09-30T22:16:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.724+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7134071970 con 0x7f713406a770 2023-09-30T22:16:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.729+0000 7f7130ff9700 1 -- 172.21.15.89:0/2360330226 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7128020080 con 0x7f713406a770 2023-09-30T22:16:23.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.911+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f713404f2a0 con 0x7f713406a770 2023-09-30T22:16:23.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.911+0000 7f7130ff9700 1 -- 172.21.15.89:0/2360330226 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7128011b80 con 0x7f713406a770 2023-09-30T22:16:23.912 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:16:23.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 >> v1:172.21.15.89:6816/126953 conn(0x7f712002f480 legacy=0x7f7120031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f713406a770 msgr2=0x7f713411d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:23.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f713406a770 0x7f713411d930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f712800e9a0 tx=0x7f712800ecb0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 shutdown_connections 2023-09-30T22:16:23.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71341303c0 0x7f713411e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f713406a770 0x7f713411d930 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 --2- 172.21.15.89:0/2360330226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f713406b290 0x7f713411de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:23.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 >> 172.21.15.89:0/2360330226 conn(0x7f7134066d70 msgr2=0x7f713412eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:23.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 shutdown_connections 2023-09-30T22:16:23.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:23.913+0000 7f713a451700 1 -- 172.21.15.89:0/2360330226 wait complete. 2023-09-30T22:16:23.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:16:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.049+0000 7ff78e9e2700 1 Processor -- start 2023-09-30T22:16:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff78e9e2700 1 -- start start 2023-09-30T22:16:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff78e9e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff788071cd0 0x7ff7880720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff78e9e2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff78809e850 0x7ff78809ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff78e9e2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78806bcc0 con 0x7ff78809f180 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff78e9e2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78806ba00 con 0x7ff78809e850 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff78e9e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880a0680 con 0x7ff788071cd0 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff787fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff788071cd0 0x7ff7880720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff7877fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff78809e850 0x7ff78809ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff78cf7f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff78809f180 0x7ff78813aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52944/0 (socket says 172.21.15.89:52944) 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff787fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff788071cd0 0x7ff7880720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46112/0 (socket says 172.21.15.89:46112) 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff78cf7f700 1 -- 172.21.15.89:0/384877044 learned_addr learned my addr 172.21.15.89:0/384877044 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff7877fe700 1 -- 172.21.15.89:0/384877044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff788071cd0 msgr2=0x7ff7880720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff7877fe700 1 --2- 172.21.15.89:0/384877044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff788071cd0 0x7ff7880720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff7877fe700 1 -- 172.21.15.89:0/384877044 >> v1:172.21.15.89:6789/0 conn(0x7ff78809f180 legacy=0x7ff78813aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff7877fe700 1 -- 172.21.15.89:0/384877044 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78806c0f0 con 0x7ff78809e850 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.050+0000 7ff787fff700 1 --2- 172.21.15.89:0/384877044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff788071cd0 0x7ff7880720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff7877fe700 1 --2- 172.21.15.89:0/384877044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff78809e850 0x7ff78809ec40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff778009840 tx=0x7ff778009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92017db077c207d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff786ffd700 1 -- 172.21.15.89:0/384877044 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778011830 con 0x7ff78809e850 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff786ffd700 1 -- 172.21.15.89:0/384877044 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff778011990 con 0x7ff78809e850 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff786ffd700 1 -- 172.21.15.89:0/384877044 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778011d20 con 0x7ff78809e850 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff78e9e2700 1 -- 172.21.15.89:0/384877044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff78809e850 msgr2=0x7ff78809ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff78e9e2700 1 --2- 172.21.15.89:0/384877044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff78809e850 0x7ff78809ec40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff778009840 tx=0x7ff778009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff78e9e2700 1 -- 172.21.15.89:0/384877044 shutdown_connections 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff78e9e2700 1 --2- 172.21.15.89:0/384877044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff788071cd0 0x7ff7880720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff78e9e2700 1 --2- 172.21.15.89:0/384877044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff78809e850 0x7ff78809ec40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff78e9e2700 1 -- 172.21.15.89:0/384877044 >> 172.21.15.89:0/384877044 conn(0x7ff7880a4130 msgr2=0x7ff7880a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff78e9e2700 1 -- 172.21.15.89:0/384877044 shutdown_connections 2023-09-30T22:16:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.051+0000 7ff78e9e2700 1 -- 172.21.15.89:0/384877044 wait complete. 2023-09-30T22:16:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.052+0000 7ff78e9e2700 1 Processor -- start 2023-09-30T22:16:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.052+0000 7ff78e9e2700 1 -- start start 2023-09-30T22:16:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff78e9e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff788071cd0 0x7ff7881c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff78e9e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff78809e850 0x7ff7881c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff78e9e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff78809f180 0x7ff7881c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff78e9e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7881aaf00 con 0x7ff788071cd0 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff78e9e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880a1560 con 0x7ff78809f180 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff78e9e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff78806cda0 con 0x7ff78809e850 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff7877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff78809e850 0x7ff7881c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff7877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff78809e850 0x7ff7881c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46116/0 (socket says 172.21.15.89:46116) 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff787fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff788071cd0 0x7ff7881c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff78cf7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff78809f180 0x7ff7881c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff7877fe700 1 -- 172.21.15.89:0/2847308500 learned_addr learned my addr 172.21.15.89:0/2847308500 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff787fff700 1 -- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff78809e850 msgr2=0x7ff7881c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff787fff700 1 --2- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff78809e850 0x7ff7881c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff787fff700 1 -- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff78809f180 msgr2=0x7ff7881c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff787fff700 1 --2- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff78809f180 0x7ff7881c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff787fff700 1 -- 172.21.15.89:0/2847308500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0094a0 con 0x7ff788071cd0 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff7877fe700 1 --2- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff78809e850 0x7ff7881c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.053+0000 7ff787fff700 1 --2- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff788071cd0 0x7ff7881c8c00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00e9b0 tx=0x7ff77c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.054+0000 7ff784ff9700 1 -- 172.21.15.89:0/2847308500 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00ee50 con 0x7ff788071cd0 2023-09-30T22:16:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.054+0000 7ff784ff9700 1 -- 172.21.15.89:0/2847308500 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00c850 con 0x7ff788071cd0 2023-09-30T22:16:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.054+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7780094a0 con 0x7ff788071cd0 2023-09-30T22:16:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.054+0000 7ff784ff9700 1 -- 172.21.15.89:0/2847308500 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00cc00 con 0x7ff788071cd0 2023-09-30T22:16:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.054+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7881aa2c0 con 0x7ff788071cd0 2023-09-30T22:16:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.054+0000 7ff784ff9700 1 -- 172.21.15.89:0/2847308500 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7ff77c013470 con 0x7ff788071cd0 2023-09-30T22:16:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.054+0000 7ff784ff9700 1 -- 172.21.15.89:0/2847308500 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7ff77c03f930 con 0x7ff788071cd0 2023-09-30T22:16:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.055+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7881aa2c0 con 0x7ff788071cd0 2023-09-30T22:16:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.059+0000 7ff784ff9700 1 -- 172.21.15.89:0/2847308500 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7881aa2c0 con 0x7ff788071cd0 2023-09-30T22:16:24.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.225+0000 7ff784ff9700 1 -- 172.21.15.89:0/2847308500 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff77c01acf0 con 0x7ff788071cd0 2023-09-30T22:16:24.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.243+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff7881aa2c0 con 0x7ff788071cd0 2023-09-30T22:16:24.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.243+0000 7ff784ff9700 1 -- 172.21.15.89:0/2847308500 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff7881aa2c0 con 0x7ff788071cd0 2023-09-30T22:16:24.243 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:16:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 >> v1:172.21.15.89:6816/126953 conn(0x7ff77402f4b0 legacy=0x7ff774031960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff788071cd0 msgr2=0x7ff7881c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 --2- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff788071cd0 0x7ff7881c8c00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00e9b0 tx=0x7ff77c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 shutdown_connections 2023-09-30T22:16:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 --2- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff78809f180 0x7ff7881c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 --2- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff788071cd0 0x7ff7881c8c00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 --2- 172.21.15.89:0/2847308500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff78809e850 0x7ff7881c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 >> 172.21.15.89:0/2847308500 conn(0x7ff7880a4130 msgr2=0x7ff7880a2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.245+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 shutdown_connections 2023-09-30T22:16:24.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.246+0000 7ff78e9e2700 1 -- 172.21.15.89:0/2847308500 wait complete. 2023-09-30T22:16:24.257 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:16:24.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.381+0000 7f5b37145700 1 Processor -- start 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.381+0000 7f5b37145700 1 -- start start 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.381+0000 7f5b37145700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30136b80 0x7f5b30138f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.381+0000 7f5b37145700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b301394a0 0x7f5b3013b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.381+0000 7f5b37145700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b300a0280 con 0x7f5b30071d60 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.381+0000 7f5b37145700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b300a0020 con 0x7f5b301394a0 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.381+0000 7f5b37145700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3009f860 con 0x7f5b30136b80 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30136b80 0x7f5b30138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30136b80 0x7f5b30138f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41952/0 (socket says 172.21.15.89:41952) 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b34ee1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5b30071d60 0x7f5b30072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52958/0 (socket says 172.21.15.89:52958) 2023-09-30T22:16:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2ffff700 1 -- 172.21.15.89:0/3018866154 learned_addr learned my addr 172.21.15.89:0/3018866154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b356e2700 1 --2- 172.21.15.89:0/3018866154 >> v2:172.21.15.89:3300/0 conn(0x7f5b301394a0 0x7f5b3013b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2ffff700 1 -- 172.21.15.89:0/3018866154 >> v2:172.21.15.89:3300/0 conn(0x7f5b301394a0 msgr2=0x7f5b3013b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2ffff700 1 --2- 172.21.15.89:0/3018866154 >> v2:172.21.15.89:3300/0 conn(0x7f5b301394a0 0x7f5b3013b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2f7fe700 1 -- 172.21.15.89:0/3018866154 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301223324 0 0) 0x7f5b300a0280 con 0x7f5b30071d60 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2ffff700 1 -- 172.21.15.89:0/3018866154 >> v1:172.21.15.89:6789/0 conn(0x7f5b30071d60 legacy=0x7f5b30072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2ffff700 1 -- 172.21.15.89:0/3018866154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3006ba00 con 0x7f5b30136b80 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2ffff700 1 --2- 172.21.15.89:0/3018866154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30136b80 0x7f5b30138f60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0098f0 tx=0x7f5b1c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd2cc75618f5b1be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2f7fe700 1 -- 172.21.15.89:0/3018866154 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c011830 con 0x7f5b30136b80 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2f7fe700 1 -- 172.21.15.89:0/3018866154 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c011990 con 0x7f5b30136b80 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.382+0000 7f5b2f7fe700 1 -- 172.21.15.89:0/3018866154 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c011d20 con 0x7f5b30136b80 2023-09-30T22:16:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.383+0000 7f5b37145700 1 -- 172.21.15.89:0/3018866154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30136b80 msgr2=0x7f5b30138f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.383+0000 7f5b37145700 1 --2- 172.21.15.89:0/3018866154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30136b80 0x7f5b30138f60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0098f0 tx=0x7f5b1c009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.383+0000 7f5b37145700 1 -- 172.21.15.89:0/3018866154 shutdown_connections 2023-09-30T22:16:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.383+0000 7f5b37145700 1 --2- 172.21.15.89:0/3018866154 >> v2:172.21.15.89:3300/0 conn(0x7f5b301394a0 0x7f5b3013b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.383+0000 7f5b37145700 1 --2- 172.21.15.89:0/3018866154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30136b80 0x7f5b30138f60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.383+0000 7f5b37145700 1 -- 172.21.15.89:0/3018866154 >> 172.21.15.89:0/3018866154 conn(0x7f5b300a4130 msgr2=0x7f5b300a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.383+0000 7f5b37145700 1 -- 172.21.15.89:0/3018866154 shutdown_connections 2023-09-30T22:16:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.383+0000 7f5b37145700 1 -- 172.21.15.89:0/3018866154 wait complete. 2023-09-30T22:16:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b37145700 1 Processor -- start 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b37145700 1 -- start start 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b37145700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30071d60 0x7f5b3011ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b37145700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b30136b80 0x7f5b3011ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b37145700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b301394a0 0x7f5b3011f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b37145700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b30121c60 con 0x7f5b301394a0 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b37145700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b30120000 con 0x7f5b30071d60 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b37145700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301211e0 con 0x7f5b30136b80 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b34ee1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30071d60 0x7f5b3011ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b34ee1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30071d60 0x7f5b3011ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41960/0 (socket says 172.21.15.89:41960) 2023-09-30T22:16:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b34ee1700 1 -- 172.21.15.89:0/860706251 learned_addr learned my addr 172.21.15.89:0/860706251 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b356e2700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b301394a0 0x7f5b3011f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.384+0000 7f5b2ffff700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b30136b80 0x7f5b3011ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b34ee1700 1 -- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b30136b80 msgr2=0x7f5b3011ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b34ee1700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b30136b80 0x7f5b3011ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b34ee1700 1 -- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b301394a0 msgr2=0x7f5b3011f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b34ee1700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b301394a0 0x7f5b3011f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b34ee1700 1 -- 172.21.15.89:0/860706251 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c009550 con 0x7f5b30071d60 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b356e2700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b301394a0 0x7f5b3011f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b34ee1700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30071d60 0x7f5b3011ea10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c520 tx=0x7f5b2400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b2d7fa700 1 -- 172.21.15.89:0/860706251 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24004c80 con 0x7f5b30071d60 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b2d7fa700 1 -- 172.21.15.89:0/860706251 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b24004de0 con 0x7f5b30071d60 2023-09-30T22:16:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b301207f0 con 0x7f5b30071d60 2023-09-30T22:16:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b2d7fa700 1 -- 172.21.15.89:0/860706251 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24007310 con 0x7f5b30071d60 2023-09-30T22:16:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.385+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3006cda0 con 0x7f5b30071d60 2023-09-30T22:16:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.386+0000 7f5b2d7fa700 1 -- 172.21.15.89:0/860706251 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5b24007470 con 0x7f5b30071d60 2023-09-30T22:16:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.386+0000 7f5b2d7fa700 1 -- 172.21.15.89:0/860706251 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f5b2403f710 con 0x7f5b30071d60 2023-09-30T22:16:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.386+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3004f2a0 con 0x7f5b30071d60 2023-09-30T22:16:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.390+0000 7f5b2d7fa700 1 -- 172.21.15.89:0/860706251 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b240123b0 con 0x7f5b30071d60 2023-09-30T22:16:24.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.576+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b300728b0 con 0x7f5b30071d60 2023-09-30T22:16:24.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.576+0000 7f5b2d7fa700 1 -- 172.21.15.89:0/860706251 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b240102c0 con 0x7f5b30071d60 2023-09-30T22:16:24.576 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-09-30T22:16:24.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 >> v1:172.21.15.89:6816/126953 conn(0x7f5b1402f510 legacy=0x7f5b140319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30071d60 msgr2=0x7f5b3011ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30071d60 0x7f5b3011ea10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c520 tx=0x7f5b2400c830 comp rx=0 tx=0).stop 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 shutdown_connections 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b301394a0 0x7f5b3011f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b30071d60 0x7f5b3011ea10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 --2- 172.21.15.89:0/860706251 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b30136b80 0x7f5b3011ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 >> 172.21.15.89:0/860706251 conn(0x7f5b300a4130 msgr2=0x7f5b30070760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 shutdown_connections 2023-09-30T22:16:24.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.578+0000 7f5b37145700 1 -- 172.21.15.89:0/860706251 wait complete. 2023-09-30T22:16:24.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:16:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.713+0000 7f3659471700 1 Processor -- start 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.713+0000 7f3659471700 1 -- start start 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.713+0000 7f3659471700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a09c0 0x7f36540a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.713+0000 7f3659471700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36540a12d0 0x7f3654134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3659471700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365409ea10 con 0x7f365409d5c0 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3659471700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365409e210 con 0x7f36540a12d0 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3659471700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36540a26f0 con 0x7f36540a09c0 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3652ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a09c0 0x7f36540a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f36537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36540a12d0 0x7f3654134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f36527fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f365409d5c0 0x7f3654133920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52964/0 (socket says 172.21.15.89:52964) 2023-09-30T22:16:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3652ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a09c0 0x7f36540a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46132/0 (socket says 172.21.15.89:46132) 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f36527fc700 1 -- 172.21.15.89:0/3892851930 learned_addr learned my addr 172.21.15.89:0/3892851930 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f36537fe700 1 -- 172.21.15.89:0/3892851930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a09c0 msgr2=0x7f36540a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f36537fe700 1 --2- 172.21.15.89:0/3892851930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a09c0 0x7f36540a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f36537fe700 1 -- 172.21.15.89:0/3892851930 >> v1:172.21.15.89:6789/0 conn(0x7f365409d5c0 legacy=0x7f3654133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f36537fe700 1 -- 172.21.15.89:0/3892851930 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365409db70 con 0x7f36540a12d0 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3651ffb700 1 -- 172.21.15.89:0/3892851930 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120219884 0 0) 0x7f365409ea10 con 0x7f365409d5c0 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3652ffd700 1 --2- 172.21.15.89:0/3892851930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a09c0 0x7f36540a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f36537fe700 1 --2- 172.21.15.89:0/3892851930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a12d0 0x7f3654134030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f364c009d90 tx=0x7f364c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ae7dd730d6c08e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3651ffb700 1 -- 172.21.15.89:0/3892851930 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c00e070 con 0x7f36540a12d0 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3651ffb700 1 -- 172.21.15.89:0/3892851930 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c00bb20 con 0x7f36540a12d0 2023-09-30T22:16:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.714+0000 7f3651ffb700 1 -- 172.21.15.89:0/3892851930 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c012430 con 0x7f36540a12d0 2023-09-30T22:16:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.715+0000 7f3659471700 1 -- 172.21.15.89:0/3892851930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a12d0 msgr2=0x7f3654134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.715+0000 7f3659471700 1 --2- 172.21.15.89:0/3892851930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a12d0 0x7f3654134030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f364c009d90 tx=0x7f364c00b330 comp rx=0 tx=0).stop 2023-09-30T22:16:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.715+0000 7f3659471700 1 -- 172.21.15.89:0/3892851930 shutdown_connections 2023-09-30T22:16:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.715+0000 7f3659471700 1 --2- 172.21.15.89:0/3892851930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a12d0 0x7f3654134030 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.715+0000 7f3659471700 1 --2- 172.21.15.89:0/3892851930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a09c0 0x7f36540a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.715+0000 7f3659471700 1 -- 172.21.15.89:0/3892851930 >> 172.21.15.89:0/3892851930 conn(0x7f36540a2fe0 msgr2=0x7f36540a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.715+0000 7f3659471700 1 -- 172.21.15.89:0/3892851930 shutdown_connections 2023-09-30T22:16:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.715+0000 7f3659471700 1 -- 172.21.15.89:0/3892851930 wait complete. 2023-09-30T22:16:24.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.716+0000 7f3659471700 1 Processor -- start 2023-09-30T22:16:24.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.716+0000 7f3659471700 1 -- start start 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.716+0000 7f3659471700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f365409d5c0 0x7f36541221f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.716+0000 7f3659471700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a09c0 0x7f3654122730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.716+0000 7f3659471700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a12d0 0x7f3654122c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.716+0000 7f3659471700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3654123d20 con 0x7f36540a09c0 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.716+0000 7f3659471700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3654124bb0 con 0x7f365409d5c0 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.716+0000 7f3659471700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3654125690 con 0x7f36540a12d0 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36527fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a09c0 0x7f3654122730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a12d0 0x7f3654122c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a12d0 0x7f3654122c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46142/0 (socket says 172.21.15.89:46142) 2023-09-30T22:16:24.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36527fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a09c0 0x7f3654122730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32774/0 (socket says 172.21.15.89:32774) 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 -- 172.21.15.89:0/1675819703 learned_addr learned my addr 172.21.15.89:0/1675819703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f3652ffd700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f365409d5c0 0x7f36541221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 -- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f365409d5c0 msgr2=0x7f36541221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f365409d5c0 0x7f36541221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 -- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a09c0 msgr2=0x7f3654122730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a09c0 0x7f3654122730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 -- 172.21.15.89:0/1675819703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480094a0 con 0x7f36540a12d0 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36527fc700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a09c0 0x7f3654122730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f36537fe700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a12d0 0x7f3654122c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f364c014040 tx=0x7f364c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f363b7fe700 1 -- 172.21.15.89:0/1675819703 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c012430 con 0x7f36540a12d0 2023-09-30T22:16:24.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f363b7fe700 1 -- 172.21.15.89:0/1675819703 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c020070 con 0x7f36540a12d0 2023-09-30T22:16:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c009a70 con 0x7f36540a12d0 2023-09-30T22:16:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f363b7fe700 1 -- 172.21.15.89:0/1675819703 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c004280 con 0x7f36540a12d0 2023-09-30T22:16:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.717+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365409cd10 con 0x7f36540a12d0 2023-09-30T22:16:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.718+0000 7f363b7fe700 1 -- 172.21.15.89:0/1675819703 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f364c0043e0 con 0x7f36540a12d0 2023-09-30T22:16:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.718+0000 7f363b7fe700 1 -- 172.21.15.89:0/1675819703 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f364c022070 con 0x7f36540a12d0 2023-09-30T22:16:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.718+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365409cf20 con 0x7f36540a12d0 2023-09-30T22:16:24.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.723+0000 7f363b7fe700 1 -- 172.21.15.89:0/1675819703 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f364c004870 con 0x7f36540a12d0 2023-09-30T22:16:24.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.903+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f365411f670 con 0x7f36540a12d0 2023-09-30T22:16:24.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.904+0000 7f363b7fe700 1 -- 172.21.15.89:0/1675819703 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f364c01ab70 con 0x7f36540a12d0 2023-09-30T22:16:24.904 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-09-30T22:16:24.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 >> v1:172.21.15.89:6816/126953 conn(0x7f364002f540 legacy=0x7f36400319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a12d0 msgr2=0x7f3654122c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a12d0 0x7f3654122c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f364c014040 tx=0x7f364c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 shutdown_connections 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36540a12d0 0x7f3654122c70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f365409d5c0 0x7f36541221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 --2- 172.21.15.89:0/1675819703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36540a09c0 0x7f3654122730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 >> 172.21.15.89:0/1675819703 conn(0x7f36540a2fe0 msgr2=0x7f365411c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 shutdown_connections 2023-09-30T22:16:24.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:24.906+0000 7f3659471700 1 -- 172.21.15.89:0/1675819703 wait complete. 2023-09-30T22:16:24.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:16:25.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.037+0000 7ff20a388700 1 Processor -- start 2023-09-30T22:16:25.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.037+0000 7ff20a388700 1 -- start start 2023-09-30T22:16:25.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff20a388700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2041316c0 0x7ff204131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff20a388700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff20409d9f0 0x7ff20406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff20a388700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20409e130 con 0x7ff20406acb0 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff20a388700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2041312b0 con 0x7ff20409d9f0 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff20a388700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20409f6e0 con 0x7ff2041316c0 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff203fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2041316c0 0x7ff204131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff2037fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff20409d9f0 0x7ff20406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff208925700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff20406acb0 0x7ff204138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52974/0 (socket says 172.21.15.89:52974) 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff208925700 1 -- 172.21.15.89:0/3879109896 learned_addr learned my addr 172.21.15.89:0/3879109896 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff2037fe700 1 -- 172.21.15.89:0/3879109896 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2041316c0 msgr2=0x7ff204131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff2037fe700 1 --2- 172.21.15.89:0/3879109896 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2041316c0 0x7ff204131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff2037fe700 1 -- 172.21.15.89:0/3879109896 >> v1:172.21.15.89:6789/0 conn(0x7ff20406acb0 legacy=0x7ff204138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff2037fe700 1 -- 172.21.15.89:0/3879109896 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20409e760 con 0x7ff20409d9f0 2023-09-30T22:16:25.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff202ffd700 1 -- 172.21.15.89:0/3879109896 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574170682 0 0) 0x7ff1fc001490 con 0x7ff20406acb0 2023-09-30T22:16:25.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff203fff700 1 --2- 172.21.15.89:0/3879109896 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2041316c0 0x7ff204131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:25.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.038+0000 7ff2037fe700 1 --2- 172.21.15.89:0/3879109896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff20409d9f0 0x7ff20406a770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009840 tx=0x7ff1f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421319ec5fca8c97 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:25.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff202ffd700 1 -- 172.21.15.89:0/3879109896 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8011830 con 0x7ff20409d9f0 2023-09-30T22:16:25.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff202ffd700 1 -- 172.21.15.89:0/3879109896 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f8011990 con 0x7ff20409d9f0 2023-09-30T22:16:25.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff202ffd700 1 -- 172.21.15.89:0/3879109896 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8011d20 con 0x7ff20409d9f0 2023-09-30T22:16:25.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff20a388700 1 -- 172.21.15.89:0/3879109896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff20409d9f0 msgr2=0x7ff20406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff20a388700 1 --2- 172.21.15.89:0/3879109896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff20409d9f0 0x7ff20406a770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009840 tx=0x7ff1f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff20a388700 1 -- 172.21.15.89:0/3879109896 shutdown_connections 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff20a388700 1 --2- 172.21.15.89:0/3879109896 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2041316c0 0x7ff204131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff20a388700 1 --2- 172.21.15.89:0/3879109896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff20409d9f0 0x7ff20406a770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff20a388700 1 -- 172.21.15.89:0/3879109896 >> 172.21.15.89:0/3879109896 conn(0x7ff204066f50 msgr2=0x7ff20409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff20a388700 1 -- 172.21.15.89:0/3879109896 shutdown_connections 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.039+0000 7ff20a388700 1 -- 172.21.15.89:0/3879109896 wait complete. 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.040+0000 7ff20a388700 1 Processor -- start 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.040+0000 7ff20a388700 1 -- start start 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff20a388700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff20406acb0 0x7ff204070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff20a388700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff20409d9f0 0x7ff204070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff20a388700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2041316c0 0x7ff2040713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff20a388700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20411a800 con 0x7ff2041316c0 2023-09-30T22:16:25.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff20a388700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040a36d0 con 0x7ff20406acb0 2023-09-30T22:16:25.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff20a388700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff204119680 con 0x7ff20409d9f0 2023-09-30T22:16:25.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff20406acb0 0x7ff204070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff2037fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff20409d9f0 0x7ff204070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff20406acb0 0x7ff204070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41982/0 (socket says 172.21.15.89:41982) 2023-09-30T22:16:25.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff208925700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2041316c0 0x7ff2040713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 -- 172.21.15.89:0/3286092087 learned_addr learned my addr 172.21.15.89:0/3286092087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:25.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 -- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff20409d9f0 msgr2=0x7ff204070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff20409d9f0 0x7ff204070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 -- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2041316c0 msgr2=0x7ff2040713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2041316c0 0x7ff2040713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 -- 172.21.15.89:0/3286092087 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f40094a0 con 0x7ff20406acb0 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff208925700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2041316c0 0x7ff2040713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff2037fe700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff20409d9f0 0x7ff204070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff203fff700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff20406acb0 0x7ff204070960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400ec10 tx=0x7ff1f400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff200ff9700 1 -- 172.21.15.89:0/3286092087 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f400c850 con 0x7ff20406acb0 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff200ff9700 1 -- 172.21.15.89:0/3286092087 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f400c9b0 con 0x7ff20406acb0 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f80094a0 con 0x7ff20406acb0 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff200ff9700 1 -- 172.21.15.89:0/3286092087 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f400cd20 con 0x7ff20406acb0 2023-09-30T22:16:25.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.041+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff204119be0 con 0x7ff20406acb0 2023-09-30T22:16:25.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.042+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff204074ea0 con 0x7ff20406acb0 2023-09-30T22:16:25.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.043+0000 7ff200ff9700 1 -- 172.21.15.89:0/3286092087 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff1f4013470 con 0x7ff20406acb0 2023-09-30T22:16:25.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.043+0000 7ff200ff9700 1 -- 172.21.15.89:0/3286092087 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7ff1f4040460 con 0x7ff20406acb0 2023-09-30T22:16:25.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.047+0000 7ff200ff9700 1 -- 172.21.15.89:0/3286092087 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1f4013720 con 0x7ff20406acb0 2023-09-30T22:16:25.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.269+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff20406c360 con 0x7ff20406acb0 2023-09-30T22:16:25.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.269+0000 7ff200ff9700 1 -- 172.21.15.89:0/3286092087 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2315 (secure 0 0 0) 0x7ff1f4021020 con 0x7ff20406acb0 2023-09-30T22:16:25.269 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:25.269 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:26.107818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:16:25.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.271+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 >> v1:172.21.15.89:6816/126953 conn(0x7ff1f002f510 legacy=0x7ff1f00319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.271+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff20406acb0 msgr2=0x7ff204070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.271+0000 7ff20a388700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff20406acb0 0x7ff204070960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400ec10 tx=0x7ff1f400ef20 comp rx=0 tx=0).stop 2023-09-30T22:16:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.271+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 shutdown_connections 2023-09-30T22:16:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.271+0000 7ff20a388700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2041316c0 0x7ff2040713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.271+0000 7ff20a388700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff20406acb0 0x7ff204070960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.271+0000 7ff20a388700 1 --2- 172.21.15.89:0/3286092087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff20409d9f0 0x7ff204070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.271+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 >> 172.21.15.89:0/3286092087 conn(0x7ff204066f50 msgr2=0x7ff2040a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.272+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 shutdown_connections 2023-09-30T22:16:25.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.272+0000 7ff20a388700 1 -- 172.21.15.89:0/3286092087 wait complete. 2023-09-30T22:16:25.273 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 7 2023-09-30T22:16:25.283 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.404+0000 7f96f19e6700 1 Processor -- start 2023-09-30T22:16:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.404+0000 7f96f19e6700 1 -- start start 2023-09-30T22:16:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96f19e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09e850 0x7f96ec09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96f19e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec134370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96f19e6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec06bec0 con 0x7f96ec09f160 2023-09-30T22:16:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96f19e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0a0030 con 0x7f96ec09e850 2023-09-30T22:16:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96f19e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0a0830 con 0x7f96ec09f5b0 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96eb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96ea7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f96ec09f160 0x7f96ec133c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52976/0 (socket says 172.21.15.89:52976) 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96ea7fc700 1 -- 172.21.15.89:0/195661523 learned_addr learned my addr 172.21.15.89:0/195661523 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96eaffd700 1 --2- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09e850 0x7f96ec09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96e9ffb700 1 -- 172.21.15.89:0/195661523 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341773563 0 0) 0x7f96ec06bec0 con 0x7f96ec09f160 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96e9ffb700 1 -- 172.21.15.89:0/195661523 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d40035b0 con 0x7f96ec09f160 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96eb7fe700 1 -- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09e850 msgr2=0x7f96ec09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96eb7fe700 1 --2- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09e850 0x7f96ec09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96eb7fe700 1 -- 172.21.15.89:0/195661523 >> v1:172.21.15.89:6789/0 conn(0x7f96ec09f160 legacy=0x7f96ec133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96eb7fe700 1 -- 172.21.15.89:0/195661523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec06caa0 con 0x7f96ec09f5b0 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96eaffd700 1 --2- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09e850 0x7f96ec09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.405+0000 7f96eb7fe700 1 --2- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec134370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009dc0 tx=0x7f96e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1889e4b1a80d590 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96e9ffb700 1 -- 172.21.15.89:0/195661523 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400e070 con 0x7f96ec09f5b0 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96e9ffb700 1 -- 172.21.15.89:0/195661523 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e400bb20 con 0x7f96ec09f5b0 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96e9ffb700 1 -- 172.21.15.89:0/195661523 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4012430 con 0x7f96ec09f5b0 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96f19e6700 1 -- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 msgr2=0x7f96ec134370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96f19e6700 1 --2- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec134370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009dc0 tx=0x7f96e400b330 comp rx=0 tx=0).stop 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96f19e6700 1 -- 172.21.15.89:0/195661523 shutdown_connections 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96f19e6700 1 --2- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09e850 0x7f96ec09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96f19e6700 1 --2- 172.21.15.89:0/195661523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec134370 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96f19e6700 1 -- 172.21.15.89:0/195661523 >> 172.21.15.89:0/195661523 conn(0x7f96ec0a3f70 msgr2=0x7f96ec0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:25.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96f19e6700 1 -- 172.21.15.89:0/195661523 shutdown_connections 2023-09-30T22:16:25.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.406+0000 7f96f19e6700 1 -- 172.21.15.89:0/195661523 wait complete. 2023-09-30T22:16:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.407+0000 7f96f19e6700 1 Processor -- start 2023-09-30T22:16:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.407+0000 7f96f19e6700 1 -- start start 2023-09-30T22:16:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96f19e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ec09e850 0x7f96ec120310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96f19e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09f160 0x7f96ec120850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96f19e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec120d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96f19e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec122500 con 0x7f96ec09e850 2023-09-30T22:16:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96f19e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec1235c0 con 0x7f96ec09f160 2023-09-30T22:16:25.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96f19e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec06ccc0 con 0x7f96ec09f5b0 2023-09-30T22:16:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ec09e850 0x7f96ec120310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96ea7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09f160 0x7f96ec120850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec120d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ec09e850 0x7f96ec120310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32808/0 (socket says 172.21.15.89:32808) 2023-09-30T22:16:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 -- 172.21.15.89:0/1227357099 learned_addr learned my addr 172.21.15.89:0/1227357099 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96ea7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09f160 0x7f96ec120850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42004/0 (socket says 172.21.15.89:42004) 2023-09-30T22:16:25.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 -- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 msgr2=0x7f96ec120d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 --2- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec120d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 -- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09f160 msgr2=0x7f96ec120850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 --2- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09f160 0x7f96ec120850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 -- 172.21.15.89:0/1227357099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e00094a0 con 0x7f96ec09e850 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eb7fe700 1 --2- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec120d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96eaffd700 1 --2- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ec09e850 0x7f96ec120310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96e000ebe0 tx=0x7f96e000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96d3fff700 1 -- 172.21.15.89:0/1227357099 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0009760 con 0x7f96ec09e850 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e4009a70 con 0x7f96ec09e850 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96d3fff700 1 -- 172.21.15.89:0/1227357099 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e000c850 con 0x7f96ec09e850 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96d3fff700 1 -- 172.21.15.89:0/1227357099 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000cbe0 con 0x7f96ec09e850 2023-09-30T22:16:25.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.408+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec121f20 con 0x7f96ec09e850 2023-09-30T22:16:25.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.409+0000 7f96d3fff700 1 -- 172.21.15.89:0/1227357099 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f96e000cd40 con 0x7f96ec09e850 2023-09-30T22:16:25.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.409+0000 7f96d3fff700 1 -- 172.21.15.89:0/1227357099 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f96e003fc00 con 0x7f96ec09e850 2023-09-30T22:16:25.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.409+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec09caf0 con 0x7f96ec09e850 2023-09-30T22:16:25.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.414+0000 7f96d3fff700 1 -- 172.21.15.89:0/1227357099 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96e001be10 con 0x7f96ec09e850 2023-09-30T22:16:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.604+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96ec1239b0 con 0x7f96ec09e850 2023-09-30T22:16:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.605+0000 7f96d3fff700 1 -- 172.21.15.89:0/1227357099 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12866 (secure 0 0 0) 0x7f96ec1239b0 con 0x7f96ec09e850 2023-09-30T22:16:25.604 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:25.605 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":23,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:21.418378+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 >> v1:172.21.15.89:6816/126953 conn(0x7f96d802f510 legacy=0x7f96d80319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ec09e850 msgr2=0x7f96ec120310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 --2- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ec09e850 0x7f96ec120310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96e000ebe0 tx=0x7f96e000eef0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 shutdown_connections 2023-09-30T22:16:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 --2- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ec09f5b0 0x7f96ec120d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 --2- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ec09e850 0x7f96ec120310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 --2- 172.21.15.89:0/1227357099 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ec09f160 0x7f96ec120850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 >> 172.21.15.89:0/1227357099 conn(0x7f96ec0a3f70 msgr2=0x7f96ec1bb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 shutdown_connections 2023-09-30T22:16:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.607+0000 7f96f19e6700 1 -- 172.21.15.89:0/1227357099 wait complete. 2023-09-30T22:16:25.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:14:00.533280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:14:01.011104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:16:25.619 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:16:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7f8b4700 1 Processor -- start 2023-09-30T22:16:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7f8b4700 1 -- start start 2023-09-30T22:16:25.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7f8b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809e2b0 0x7f0f78135a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7f8b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f78135f60 0x7f0f78138340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7f8b4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7809b800 con 0x7f0f7809d7d0 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7f8b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7809b620 con 0x7f0f78135f60 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7f8b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7809f370 con 0x7f0f7809e2b0 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7ce4f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809e2b0 0x7f0f78135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7de51700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f78135f60 0x7f0f78138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7d650700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0f7809d7d0 0x7f0f7809dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52986/0 (socket says 172.21.15.89:52986) 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7ce4f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809e2b0 0x7f0f78135a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46186/0 (socket says 172.21.15.89:46186) 2023-09-30T22:16:25.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7de51700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f78135f60 0x7f0f78138340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32820/0 (socket says 172.21.15.89:32820) 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.745+0000 7f0f7d650700 1 -- 172.21.15.89:0/1061055670 learned_addr learned my addr 172.21.15.89:0/1061055670 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f7ce4f700 1 -- 172.21.15.89:0/1061055670 >> v2:172.21.15.89:3300/0 conn(0x7f0f78135f60 msgr2=0x7f0f78138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f7ce4f700 1 --2- 172.21.15.89:0/1061055670 >> v2:172.21.15.89:3300/0 conn(0x7f0f78135f60 0x7f0f78138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f7ce4f700 1 -- 172.21.15.89:0/1061055670 >> v1:172.21.15.89:6789/0 conn(0x7f0f7809d7d0 legacy=0x7f0f7809dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f7ce4f700 1 -- 172.21.15.89:0/1061055670 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7809ef80 con 0x7f0f7809e2b0 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f7de51700 1 --2- 172.21.15.89:0/1061055670 >> v2:172.21.15.89:3300/0 conn(0x7f0f78135f60 0x7f0f78138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f7ce4f700 1 --2- 172.21.15.89:0/1061055670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809e2b0 0x7f0f78135a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009840 tx=0x7f0f68009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e54df96b2f3476a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f6ffff700 1 -- 172.21.15.89:0/1061055670 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68011830 con 0x7f0f7809e2b0 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f6ffff700 1 -- 172.21.15.89:0/1061055670 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f68011990 con 0x7f0f7809e2b0 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.746+0000 7f0f6ffff700 1 -- 172.21.15.89:0/1061055670 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68011d20 con 0x7f0f7809e2b0 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1061055670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809e2b0 msgr2=0x7f0f78135a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 --2- 172.21.15.89:0/1061055670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809e2b0 0x7f0f78135a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009840 tx=0x7f0f68009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1061055670 shutdown_connections 2023-09-30T22:16:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 --2- 172.21.15.89:0/1061055670 >> v2:172.21.15.89:3300/0 conn(0x7f0f78135f60 0x7f0f78138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 --2- 172.21.15.89:0/1061055670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809e2b0 0x7f0f78135a20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1061055670 >> 172.21.15.89:0/1061055670 conn(0x7f0f78066d70 msgr2=0x7f0f780a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1061055670 shutdown_connections 2023-09-30T22:16:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1061055670 wait complete. 2023-09-30T22:16:25.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.747+0000 7f0f7f8b4700 1 Processor -- start 2023-09-30T22:16:25.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7f8b4700 1 -- start start 2023-09-30T22:16:25.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7f8b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809d7d0 0x7f0f781c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7f8b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7809e2b0 0x7f0f781c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7f8b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f78135f60 0x7f0f781c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7f8b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7806b230 con 0x7f0f7809e2b0 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7f8b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7811fd30 con 0x7f0f78135f60 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7f8b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f781255b0 con 0x7f0f7809d7d0 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7809e2b0 0x7f0f781c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7d650700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809d7d0 0x7f0f781c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7809e2b0 0x7f0f781c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32836/0 (socket says 172.21.15.89:32836) 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 -- 172.21.15.89:0/1902683844 learned_addr learned my addr 172.21.15.89:0/1902683844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7de51700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f78135f60 0x7f0f781c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 -- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809d7d0 msgr2=0x7f0f781c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809d7d0 0x7f0f781c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 -- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f78135f60 msgr2=0x7f0f781c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f78135f60 0x7f0f781c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 -- 172.21.15.89:0/1902683844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f70009a70 con 0x7f0f7809e2b0 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7d650700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809d7d0 0x7f0f781c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.748+0000 7f0f7ce4f700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7809e2b0 0x7f0f781c8ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009fd0 tx=0x7f0f68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.749+0000 7f0f6dffb700 1 -- 172.21.15.89:0/1902683844 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680045d0 con 0x7f0f7809e2b0 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.749+0000 7f0f6dffb700 1 -- 172.21.15.89:0/1902683844 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6801e070 con 0x7f0f7809e2b0 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.749+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f680094a0 con 0x7f0f7809e2b0 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.749+0000 7f0f6dffb700 1 -- 172.21.15.89:0/1902683844 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680038e0 con 0x7f0f7809e2b0 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.749+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f78120210 con 0x7f0f7809e2b0 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.750+0000 7f0f6dffb700 1 -- 172.21.15.89:0/1902683844 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0f68003a40 con 0x7f0f7809e2b0 2023-09-30T22:16:25.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.750+0000 7f0f6dffb700 1 -- 172.21.15.89:0/1902683844 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f0f68040a90 con 0x7f0f7809e2b0 2023-09-30T22:16:25.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.750+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7804f2a0 con 0x7f0f7809e2b0 2023-09-30T22:16:25.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.754+0000 7f0f6dffb700 1 -- 172.21.15.89:0/1902683844 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f68015020 con 0x7f0f7809e2b0 2023-09-30T22:16:25.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.935+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f78125330 con 0x7f0f7809e2b0 2023-09-30T22:16:25.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.936+0000 7f0f6dffb700 1 -- 172.21.15.89:0/1902683844 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f6802d450 con 0x7f0f7809e2b0 2023-09-30T22:16:25.936 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:16:25.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 >> v1:172.21.15.89:6816/126953 conn(0x7f0f5c02f510 legacy=0x7f0f5c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7809e2b0 msgr2=0x7f0f781c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:25.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7809e2b0 0x7f0f781c8ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009fd0 tx=0x7f0f68004750 comp rx=0 tx=0).stop 2023-09-30T22:16:25.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 shutdown_connections 2023-09-30T22:16:25.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f78135f60 0x7f0f781c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7809d7d0 0x7f0f781c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 --2- 172.21.15.89:0/1902683844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7809e2b0 0x7f0f781c8ab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:25.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 >> 172.21.15.89:0/1902683844 conn(0x7f0f78066d70 msgr2=0x7f0f781348e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:25.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 shutdown_connections 2023-09-30T22:16:25.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:25.938+0000 7f0f7f8b4700 1 -- 172.21.15.89:0/1902683844 wait complete. 2023-09-30T22:16:25.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:16:26.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.073+0000 7f209d6c8700 1 Processor -- start 2023-09-30T22:16:26.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.073+0000 7f209d6c8700 1 -- start start 2023-09-30T22:16:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f209d6c8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f2098136e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f209d6c8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20981373b0 0x7f2098139790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f209d6c8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209809ec00 con 0x7f2098131c20 2023-09-30T22:16:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f209d6c8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209806ee30 con 0x7f20981373b0 2023-09-30T22:16:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f209d6c8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2098131780 con 0x7f2098134a90 2023-09-30T22:16:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20967fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f2098136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20977fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20981373b0 0x7f2098139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20967fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f2098136e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46210/0 (socket says 172.21.15.89:46210) 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20977fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20981373b0 0x7f2098139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32842/0 (socket says 172.21.15.89:32842) 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20967fc700 1 -- 172.21.15.89:0/987613618 learned_addr learned my addr 172.21.15.89:0/987613618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20977fe700 1 -- 172.21.15.89:0/987613618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 msgr2=0x7f2098136e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20977fe700 1 --2- 172.21.15.89:0/987613618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f2098136e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20977fe700 1 -- 172.21.15.89:0/987613618 >> v1:172.21.15.89:6789/0 conn(0x7f2098131c20 legacy=0x7f2098131ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20977fe700 1 -- 172.21.15.89:0/987613618 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098131a20 con 0x7f20981373b0 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f2095ffb700 1 -- 172.21.15.89:0/987613618 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032491316 0 0) 0x7f209809ec00 con 0x7f2098131c20 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20967fc700 1 --2- 172.21.15.89:0/987613618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f2098136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.074+0000 7f20977fe700 1 --2- 172.21.15.89:0/987613618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 0x7f2098139790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2090009d90 tx=0x7f209000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f418370ef34298f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f2095ffb700 1 -- 172.21.15.89:0/987613618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209000e070 con 0x7f20981373b0 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f2095ffb700 1 -- 172.21.15.89:0/987613618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209000bb20 con 0x7f20981373b0 2023-09-30T22:16:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f2095ffb700 1 -- 172.21.15.89:0/987613618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090012430 con 0x7f20981373b0 2023-09-30T22:16:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f209d6c8700 1 -- 172.21.15.89:0/987613618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 msgr2=0x7f2098139790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f209d6c8700 1 --2- 172.21.15.89:0/987613618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 0x7f2098139790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2090009d90 tx=0x7f209000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f209d6c8700 1 -- 172.21.15.89:0/987613618 shutdown_connections 2023-09-30T22:16:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f209d6c8700 1 --2- 172.21.15.89:0/987613618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 0x7f2098139790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f209d6c8700 1 --2- 172.21.15.89:0/987613618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f2098136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f209d6c8700 1 -- 172.21.15.89:0/987613618 >> 172.21.15.89:0/987613618 conn(0x7f20980a2c10 msgr2=0x7f20980a3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f209d6c8700 1 -- 172.21.15.89:0/987613618 shutdown_connections 2023-09-30T22:16:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.075+0000 7f209d6c8700 1 -- 172.21.15.89:0/987613618 wait complete. 2023-09-30T22:16:26.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.076+0000 7f209d6c8700 1 Processor -- start 2023-09-30T22:16:26.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f209d6c8700 1 -- start start 2023-09-30T22:16:26.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f209d6c8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2098131c20 0x7f209806f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f209d6c8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f209806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f209d6c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 0x7f209806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f209d6c8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20980709d0 con 0x7f20981373b0 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f209d6c8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2098070f80 con 0x7f2098131c20 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f209d6c8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20980717f0 con 0x7f2098134a90 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f20967fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f209806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f20967fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f209806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46226/0 (socket says 172.21.15.89:46226) 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f20967fc700 1 -- 172.21.15.89:0/976284641 learned_addr learned my addr 172.21.15.89:0/976284641 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f20977fe700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 0x7f209806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.077+0000 7f2096ffd700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2098131c20 0x7f209806f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f20967fc700 1 -- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2098131c20 msgr2=0x7f209806f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f20967fc700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2098131c20 0x7f209806f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f20967fc700 1 -- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 msgr2=0x7f209806fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f20967fc700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 0x7f209806fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f20967fc700 1 -- 172.21.15.89:0/976284641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c009550 con 0x7f2098134a90 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f20967fc700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f209806f5e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f208c00ebe0 tx=0x7f208c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f207f7fe700 1 -- 172.21.15.89:0/976284641 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00c850 con 0x7f2098134a90 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f207f7fe700 1 -- 172.21.15.89:0/976284641 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c00c9b0 con 0x7f2098134a90 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2090009a70 con 0x7f2098134a90 2023-09-30T22:16:26.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f207f7fe700 1 -- 172.21.15.89:0/976284641 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00cd20 con 0x7f2098134a90 2023-09-30T22:16:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.078+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2098119cd0 con 0x7f2098134a90 2023-09-30T22:16:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.079+0000 7f207f7fe700 1 -- 172.21.15.89:0/976284641 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f208c013470 con 0x7f2098134a90 2023-09-30T22:16:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.079+0000 7f207f7fe700 1 -- 172.21.15.89:0/976284641 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f208c03fd20 con 0x7f2098134a90 2023-09-30T22:16:26.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.079+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209804f2a0 con 0x7f2098134a90 2023-09-30T22:16:26.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.084+0000 7f207f7fe700 1 -- 172.21.15.89:0/976284641 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f208c0232f0 con 0x7f2098134a90 2023-09-30T22:16:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.268+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f209806e180 con 0x7f2098134a90 2023-09-30T22:16:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.268+0000 7f207f7fe700 1 -- 172.21.15.89:0/976284641 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f208c021070 con 0x7f2098134a90 2023-09-30T22:16:26.268 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:16:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 >> v1:172.21.15.89:6816/126953 conn(0x7f208402f510 legacy=0x7f20840319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 msgr2=0x7f209806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f209806f5e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f208c00ebe0 tx=0x7f208c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 shutdown_connections 2023-09-30T22:16:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20981373b0 0x7f209806fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2098131c20 0x7f209806f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 --2- 172.21.15.89:0/976284641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2098134a90 0x7f209806f5e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 >> 172.21.15.89:0/976284641 conn(0x7f20980a2c10 msgr2=0x7f209812e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 shutdown_connections 2023-09-30T22:16:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.270+0000 7f209d6c8700 1 -- 172.21.15.89:0/976284641 wait complete. 2023-09-30T22:16:26.281 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:16:26.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17bd870700 1 Processor -- start 2023-09-30T22:16:26.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17bd870700 1 -- start start 2023-09-30T22:16:26.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17bd870700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b809db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17bd870700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17b809e070 0x7f17b809e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17bd870700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b8133a20 0x7f17b8133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17bd870700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b809a570 con 0x7f17b809e070 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17bd870700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b809b230 con 0x7f17b809d760 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17bd870700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17b809ecf0 con 0x7f17b8133a20 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17b67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17b809e070 0x7f17b809e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17b77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b8133a20 0x7f17b8133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17b67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17b809e070 0x7f17b809e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32872/0 (socket says 172.21.15.89:32872) 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b809db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42044/0 (socket says 172.21.15.89:42044) 2023-09-30T22:16:26.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17b77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b8133a20 0x7f17b8133ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46230/0 (socket says 172.21.15.89:46230) 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.405+0000 7f17b67fc700 1 -- 172.21.15.89:0/2979050268 learned_addr learned my addr 172.21.15.89:0/2979050268 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b67fc700 1 -- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b8133a20 msgr2=0x7f17b8133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b67fc700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b8133a20 0x7f17b8133ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b67fc700 1 -- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 msgr2=0x7f17b809db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b67fc700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b809db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b67fc700 1 -- 172.21.15.89:0/2979050268 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b809f770 con 0x7f17b809e070 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b6ffd700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b809db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b77fe700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b8133a20 0x7f17b8133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b67fc700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b809e070 0x7f17b809e4a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17ac006fd0 tx=0x7f17ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab9de15966033e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:26.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b5ffb700 1 -- 172.21.15.89:0/2979050268 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac005e30 con 0x7f17b809e070 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b5ffb700 1 -- 172.21.15.89:0/2979050268 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac004030 con 0x7f17b809e070 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.406+0000 7f17b5ffb700 1 -- 172.21.15.89:0/2979050268 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0043e0 con 0x7f17b809e070 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 -- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b809e070 msgr2=0x7f17b809e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b809e070 0x7f17b809e4a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17ac006fd0 tx=0x7f17ac009bd0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 -- 172.21.15.89:0/2979050268 shutdown_connections 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b8133a20 0x7f17b8133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b809db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 --2- 172.21.15.89:0/2979050268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b809e070 0x7f17b809e4a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 -- 172.21.15.89:0/2979050268 >> 172.21.15.89:0/2979050268 conn(0x7f17b8066f50 msgr2=0x7f17b80a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 -- 172.21.15.89:0/2979050268 shutdown_connections 2023-09-30T22:16:26.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 -- 172.21.15.89:0/2979050268 wait complete. 2023-09-30T22:16:26.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.407+0000 7f17bd870700 1 Processor -- start 2023-09-30T22:16:26.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17bd870700 1 -- start start 2023-09-30T22:16:26.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17bd870700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b81c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17bd870700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b809e070 0x7f17b81c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17bd870700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b8133a20 0x7f17b81c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17bd870700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b809c7d0 con 0x7f17b8133a20 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17bd870700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8124770 con 0x7f17b809d760 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17bd870700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8124e50 con 0x7f17b809e070 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b81c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b67fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b809e070 0x7f17b81c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b81c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42060/0 (socket says 172.21.15.89:42060) 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b8133a20 0x7f17b81c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b6ffd700 1 -- 172.21.15.89:0/688320303 learned_addr learned my addr 172.21.15.89:0/688320303 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b6ffd700 1 -- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b809e070 msgr2=0x7f17b81c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b6ffd700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b809e070 0x7f17b81c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b6ffd700 1 -- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b8133a20 msgr2=0x7f17b81c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.408+0000 7f17b6ffd700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b8133a20 0x7f17b81c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f17b6ffd700 1 -- 172.21.15.89:0/688320303 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a80094a0 con 0x7f17b809d760 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f17b67fc700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b809e070 0x7f17b81c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f17b77fe700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b8133a20 0x7f17b81c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f17b6ffd700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b81c8300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f17a800ec10 tx=0x7f17a800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f179ffff700 1 -- 172.21.15.89:0/688320303 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8009760 con 0x7f17b809d760 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f179ffff700 1 -- 172.21.15.89:0/688320303 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a800c850 con 0x7f17b809d760 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b0009a70 con 0x7f17b809d760 2023-09-30T22:16:26.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f179ffff700 1 -- 172.21.15.89:0/688320303 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800cbe0 con 0x7f17b809d760 2023-09-30T22:16:26.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac009500 con 0x7f17b809d760 2023-09-30T22:16:26.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.409+0000 7f179ffff700 1 -- 172.21.15.89:0/688320303 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f17a800cd40 con 0x7f17b809d760 2023-09-30T22:16:26.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.410+0000 7f179ffff700 1 -- 172.21.15.89:0/688320303 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f17a803ff10 con 0x7f17b809d760 2023-09-30T22:16:26.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.410+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b8071ce0 con 0x7f17b809d760 2023-09-30T22:16:26.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.415+0000 7f179ffff700 1 -- 172.21.15.89:0/688320303 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17a8021020 con 0x7f17b809d760 2023-09-30T22:16:26.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.597+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17b804f2a0 con 0x7f17b809d760 2023-09-30T22:16:26.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.597+0000 7f179ffff700 1 -- 172.21.15.89:0/688320303 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f17a8011b40 con 0x7f17b809d760 2023-09-30T22:16:26.597 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-09-30T22:16:26.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 >> v1:172.21.15.89:6816/126953 conn(0x7f17a402f510 legacy=0x7f17a40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 msgr2=0x7f17b81c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b81c8300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f17a800ec10 tx=0x7f17a800ef20 comp rx=0 tx=0).stop 2023-09-30T22:16:26.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 shutdown_connections 2023-09-30T22:16:26.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b8133a20 0x7f17b81c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17b809d760 0x7f17b81c8300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 --2- 172.21.15.89:0/688320303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17b809e070 0x7f17b81c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 >> 172.21.15.89:0/688320303 conn(0x7f17b8066f50 msgr2=0x7f17b8070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:26.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 shutdown_connections 2023-09-30T22:16:26.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.599+0000 7f17bd870700 1 -- 172.21.15.89:0/688320303 wait complete. 2023-09-30T22:16:26.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:16:26.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f006762e700 1 Processor -- start 2023-09-30T22:16:26.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f006762e700 1 -- start start 2023-09-30T22:16:26.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f006762e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131c20 0x7f0060131ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f006762e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f006009bc50 0x7f006009c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f006762e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009c5a0 0x7f0060138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f006762e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006009ec00 con 0x7f006009bc50 2023-09-30T22:16:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f006762e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006006ee30 con 0x7f006009c5a0 2023-09-30T22:16:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f006762e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0060131780 con 0x7f0060131c20 2023-09-30T22:16:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f0064bc9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f006009bc50 0x7f006009c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f00653ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131c20 0x7f0060131ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f0065bcb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009c5a0 0x7f0060138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f00653ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131c20 0x7f0060131ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46258/0 (socket says 172.21.15.89:46258) 2023-09-30T22:16:26.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f0064bc9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f006009bc50 0x7f006009c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32890/0 (socket says 172.21.15.89:32890) 2023-09-30T22:16:26.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f0065bcb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009c5a0 0x7f0060138ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42068/0 (socket says 172.21.15.89:42068) 2023-09-30T22:16:26.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.735+0000 7f00653ca700 1 -- 172.21.15.89:0/2804960453 learned_addr learned my addr 172.21.15.89:0/2804960453 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:26.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0064bc9700 1 -- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131c20 msgr2=0x7f0060131ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:26.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0064bc9700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131c20 0x7f0060131ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0064bc9700 1 -- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009c5a0 msgr2=0x7f0060138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:26.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0064bc9700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009c5a0 0x7f0060138ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0064bc9700 1 -- 172.21.15.89:0/2804960453 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060131a20 con 0x7f006009bc50 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f00653ca700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131c20 0x7f0060131ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0065bcb700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009c5a0 0x7f0060138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0064bc9700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006009bc50 0x7f006009c060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f005c009840 tx=0x7f005c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5f271a42864c2e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0057fff700 1 -- 172.21.15.89:0/2804960453 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011830 con 0x7f006009bc50 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0057fff700 1 -- 172.21.15.89:0/2804960453 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c011990 con 0x7f006009bc50 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f0057fff700 1 -- 172.21.15.89:0/2804960453 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011d20 con 0x7f006009bc50 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f006762e700 1 -- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006009bc50 msgr2=0x7f006009c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.736+0000 7f006762e700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006009bc50 0x7f006009c060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f005c009840 tx=0x7f005c009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.737+0000 7f006762e700 1 -- 172.21.15.89:0/2804960453 shutdown_connections 2023-09-30T22:16:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.737+0000 7f006762e700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009c5a0 0x7f0060138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.737+0000 7f006762e700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131c20 0x7f0060131ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.737+0000 7f006762e700 1 --2- 172.21.15.89:0/2804960453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006009bc50 0x7f006009c060 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.737+0000 7f006762e700 1 -- 172.21.15.89:0/2804960453 >> 172.21.15.89:0/2804960453 conn(0x7f00600a2c10 msgr2=0x7f00600a3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:26.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.737+0000 7f006762e700 1 -- 172.21.15.89:0/2804960453 shutdown_connections 2023-09-30T22:16:26.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.737+0000 7f006762e700 1 -- 172.21.15.89:0/2804960453 wait complete. 2023-09-30T22:16:26.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.737+0000 7f006762e700 1 Processor -- start 2023-09-30T22:16:26.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f006762e700 1 -- start start 2023-09-30T22:16:26.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f006762e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009bc50 0x7f0060118e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f006762e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006009c5a0 0x7f0060119380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f006762e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060131c20 0x7f00601198c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:26.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f006762e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006011c110 con 0x7f0060131c20 2023-09-30T22:16:26.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f006762e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006011c860 con 0x7f006009bc50 2023-09-30T22:16:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f006762e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f006011b6a0 con 0x7f006009c5a0 2023-09-30T22:16:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0064bc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006009c5a0 0x7f0060119380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060131c20 0x7f00601198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f00653ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009bc50 0x7f0060118e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060131c20 0x7f00601198c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32904/0 (socket says 172.21.15.89:32904) 2023-09-30T22:16:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0064bc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006009c5a0 0x7f0060119380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46270/0 (socket says 172.21.15.89:46270) 2023-09-30T22:16:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 -- 172.21.15.89:0/3219746988 learned_addr learned my addr 172.21.15.89:0/3219746988 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 -- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006009c5a0 msgr2=0x7f0060119380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 --2- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006009c5a0 0x7f0060119380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 -- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009bc50 msgr2=0x7f0060118e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 --2- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009bc50 0x7f0060118e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 -- 172.21.15.89:0/3219746988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0058009a70 con 0x7f0060131c20 2023-09-30T22:16:26.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.738+0000 7f0065bcb700 1 --2- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060131c20 0x7f00601198c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0058009f00 tx=0x7f005800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:26.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.739+0000 7f0055ffb700 1 -- 172.21.15.89:0/3219746988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058009d70 con 0x7f0060131c20 2023-09-30T22:16:26.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.739+0000 7f0055ffb700 1 -- 172.21.15.89:0/3219746988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005800f040 con 0x7f0060131c20 2023-09-30T22:16:26.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.739+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0050009550 con 0x7f0060131c20 2023-09-30T22:16:26.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.739+0000 7f0055ffb700 1 -- 172.21.15.89:0/3219746988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005800cd70 con 0x7f0060131c20 2023-09-30T22:16:26.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.739+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c0094a0 con 0x7f0060131c20 2023-09-30T22:16:26.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.739+0000 7f0055ffb700 1 -- 172.21.15.89:0/3219746988 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0058014480 con 0x7f0060131c20 2023-09-30T22:16:26.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.740+0000 7f0055ffb700 1 -- 172.21.15.89:0/3219746988 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f0058040360 con 0x7f0060131c20 2023-09-30T22:16:26.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.740+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0060130e20 con 0x7f0060131c20 2023-09-30T22:16:26.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.744+0000 7f0055ffb700 1 -- 172.21.15.89:0/3219746988 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0058012930 con 0x7f0060131c20 2023-09-30T22:16:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.926+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f006011bd20 con 0x7f0060131c20 2023-09-30T22:16:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.926+0000 7f0055ffb700 1 -- 172.21.15.89:0/3219746988 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0058012340 con 0x7f0060131c20 2023-09-30T22:16:26.927 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-09-30T22:16:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 >> v1:172.21.15.89:6816/126953 conn(0x7f004402f510 legacy=0x7f00440319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060131c20 msgr2=0x7f00601198c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 --2- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060131c20 0x7f00601198c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0058009f00 tx=0x7f005800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 shutdown_connections 2023-09-30T22:16:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 --2- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060131c20 0x7f00601198c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 --2- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006009bc50 0x7f0060118e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 --2- 172.21.15.89:0/3219746988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006009c5a0 0x7f0060119380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 >> 172.21.15.89:0/3219746988 conn(0x7f00600a2c10 msgr2=0x7f00601b8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.928+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 shutdown_connections 2023-09-30T22:16:26.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:26.929+0000 7f006762e700 1 -- 172.21.15.89:0/3219746988 wait complete. 2023-09-30T22:16:26.940 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:16:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.061+0000 7f0be77b4700 1 Processor -- start 2023-09-30T22:16:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.061+0000 7f0be77b4700 1 -- start start 2023-09-30T22:16:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.061+0000 7f0be77b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0131810 0x7f0be0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.061+0000 7f0be77b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0be0099a40 0x7f0be0099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.061+0000 7f0be77b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be0138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.061+0000 7f0be77b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be01313c0 con 0x7f0be0099a40 2023-09-30T22:16:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.061+0000 7f0be77b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be006e8c0 con 0x7f0be0131810 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.061+0000 7f0be77b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0be009a820 con 0x7f0be009a3b0 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5d51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be0138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5550700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0131810 0x7f0be0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be4d4f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0be0099a40 0x7f0be0099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be4d4f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0be0099a40 0x7f0be0099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32906/0 (socket says 172.21.15.89:32906) 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5d51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be0138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46278/0 (socket says 172.21.15.89:46278) 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5550700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0131810 0x7f0be0131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42082/0 (socket says 172.21.15.89:42082) 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be4d4f700 1 -- 172.21.15.89:0/3860565943 learned_addr learned my addr 172.21.15.89:0/3860565943 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5550700 1 -- 172.21.15.89:0/3860565943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 msgr2=0x7f0be0138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5550700 1 --2- 172.21.15.89:0/3860565943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be0138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5550700 1 -- 172.21.15.89:0/3860565943 >> v2:172.21.15.89:3300/0 conn(0x7f0be0099a40 msgr2=0x7f0be0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5550700 1 --2- 172.21.15.89:0/3860565943 >> v2:172.21.15.89:3300/0 conn(0x7f0be0099a40 0x7f0be0099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5550700 1 -- 172.21.15.89:0/3860565943 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0071d20 con 0x7f0be0131810 2023-09-30T22:16:27.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be4d4f700 1 --2- 172.21.15.89:0/3860565943 >> v2:172.21.15.89:3300/0 conn(0x7f0be0099a40 0x7f0be0099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:27.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0be5550700 1 --2- 172.21.15.89:0/3860565943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0131810 0x7f0be0131be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009840 tx=0x7f0bdc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0f7cf3056aa262d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:27.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0bd7fff700 1 -- 172.21.15.89:0/3860565943 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc011830 con 0x7f0be0131810 2023-09-30T22:16:27.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0bd7fff700 1 -- 172.21.15.89:0/3860565943 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc011990 con 0x7f0be0131810 2023-09-30T22:16:27.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.062+0000 7f0bd7fff700 1 -- 172.21.15.89:0/3860565943 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc011d20 con 0x7f0be0131810 2023-09-30T22:16:27.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 -- 172.21.15.89:0/3860565943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0131810 msgr2=0x7f0be0131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 --2- 172.21.15.89:0/3860565943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0131810 0x7f0be0131be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009840 tx=0x7f0bdc009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:27.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 -- 172.21.15.89:0/3860565943 shutdown_connections 2023-09-30T22:16:27.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 --2- 172.21.15.89:0/3860565943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be0138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 --2- 172.21.15.89:0/3860565943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0131810 0x7f0be0131be0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 --2- 172.21.15.89:0/3860565943 >> v2:172.21.15.89:3300/0 conn(0x7f0be0099a40 0x7f0be0099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 -- 172.21.15.89:0/3860565943 >> 172.21.15.89:0/3860565943 conn(0x7f0be00a2aa0 msgr2=0x7f0be00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 -- 172.21.15.89:0/3860565943 shutdown_connections 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.063+0000 7f0be77b4700 1 -- 172.21.15.89:0/3860565943 wait complete. 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be77b4700 1 Processor -- start 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be77b4700 1 -- start start 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be77b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0099a40 0x7f0be011b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be77b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be011bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be77b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be0131810 0x7f0be011c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be77b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be011d2d0 con 0x7f0be0131810 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be77b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be011e520 con 0x7f0be0099a40 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be77b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0be011d8b0 con 0x7f0be009a3b0 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be5550700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0099a40 0x7f0be011b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be4d4f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be011bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be5550700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0099a40 0x7f0be011b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42098/0 (socket says 172.21.15.89:42098) 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be5550700 1 -- 172.21.15.89:0/2562519207 learned_addr learned my addr 172.21.15.89:0/2562519207 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.064+0000 7f0be5d51700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be0131810 0x7f0be011c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be5550700 1 -- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 msgr2=0x7f0be011bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be5550700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be011bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be5550700 1 -- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be0131810 msgr2=0x7f0be011c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be5550700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be0131810 0x7f0be011c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be5550700 1 -- 172.21.15.89:0/2562519207 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd0009550 con 0x7f0be0099a40 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be5d51700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be0131810 0x7f0be011c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be5550700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0099a40 0x7f0be011b6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0090b0 tx=0x7f0bdc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0bd5ffb700 1 -- 172.21.15.89:0/2562519207 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc004450 con 0x7f0be0099a40 2023-09-30T22:16:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0bd5ffb700 1 -- 172.21.15.89:0/2562519207 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc0045b0 con 0x7f0be0099a40 2023-09-30T22:16:27.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bdc0094a0 con 0x7f0be0099a40 2023-09-30T22:16:27.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0bd5ffb700 1 -- 172.21.15.89:0/2562519207 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc0038e0 con 0x7f0be0099a40 2023-09-30T22:16:27.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.065+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be00a1cf0 con 0x7f0be0099a40 2023-09-30T22:16:27.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.066+0000 7f0bd5ffb700 1 -- 172.21.15.89:0/2562519207 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0bdc003a40 con 0x7f0be0099a40 2023-09-30T22:16:27.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.066+0000 7f0bd5ffb700 1 -- 172.21.15.89:0/2562519207 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f0bdc040d30 con 0x7f0be0099a40 2023-09-30T22:16:27.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.066+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be00717f0 con 0x7f0be0099a40 2023-09-30T22:16:27.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.071+0000 7f0bd5ffb700 1 -- 172.21.15.89:0/2562519207 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0bdc015020 con 0x7f0be0099a40 2023-09-30T22:16:27.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.294+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0be0074ea0 con 0x7f0be0099a40 2023-09-30T22:16:27.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.294+0000 7f0bd5ffb700 1 -- 172.21.15.89:0/2562519207 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2315 (secure 0 0 0) 0x7f0bdc01a420 con 0x7f0be0099a40 2023-09-30T22:16:27.294 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:27.294 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:26.107818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:16:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 >> v1:172.21.15.89:6816/126953 conn(0x7f0bc402f540 legacy=0x7f0bc40319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0099a40 msgr2=0x7f0be011b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0099a40 0x7f0be011b6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0090b0 tx=0x7f0bdc004750 comp rx=0 tx=0).stop 2023-09-30T22:16:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 shutdown_connections 2023-09-30T22:16:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be0131810 0x7f0be011c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0be0099a40 0x7f0be011b6c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 --2- 172.21.15.89:0/2562519207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0be009a3b0 0x7f0be011bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 >> 172.21.15.89:0/2562519207 conn(0x7f0be00a2aa0 msgr2=0x7f0be00a0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 shutdown_connections 2023-09-30T22:16:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.296+0000 7f0be77b4700 1 -- 172.21.15.89:0/2562519207 wait complete. 2023-09-30T22:16:27.297 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 7 2023-09-30T22:16:27.308 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:27.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.428+0000 7f5c71ecd700 1 Processor -- start 2023-09-30T22:16:27.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.428+0000 7f5c71ecd700 1 -- start start 2023-09-30T22:16:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c71ecd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c71ecd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e410 0x7f5c6c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c71ecd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c09f130 con 0x7f5c6c09e000 2023-09-30T22:16:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c71ecd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c09f520 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c71ecd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c09e930 con 0x7f5c6c09e410 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5c6c09e000 0x7f5c6c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53006/0 (socket says 172.21.15.89:53006) 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6affd700 1 -- 172.21.15.89:0/4161375473 learned_addr learned my addr 172.21.15.89:0/4161375473 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6bfff700 1 --2- 172.21.15.89:0/4161375473 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e410 0x7f5c6c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6b7fe700 1 -- 172.21.15.89:0/4161375473 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e410 msgr2=0x7f5c6c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6b7fe700 1 --2- 172.21.15.89:0/4161375473 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e410 0x7f5c6c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6b7fe700 1 -- 172.21.15.89:0/4161375473 >> v1:172.21.15.89:6789/0 conn(0x7f5c6c09e000 legacy=0x7f5c6c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6a7fc700 1 -- 172.21.15.89:0/4161375473 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824166229 0 0) 0x7f5c6c09f130 con 0x7f5c6c09e000 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.429+0000 7f5c6b7fe700 1 -- 172.21.15.89:0/4161375473 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c09b2d0 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c6b7fe700 1 --2- 172.21.15.89:0/4161375473 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c09dac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009870 tx=0x7f5c60009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10265428c825a112 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c6a7fc700 1 -- 172.21.15.89:0/4161375473 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60011830 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c6a7fc700 1 -- 172.21.15.89:0/4161375473 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c60011990 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c6a7fc700 1 -- 172.21.15.89:0/4161375473 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60011d20 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c71ecd700 1 -- 172.21.15.89:0/4161375473 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 msgr2=0x7f5c6c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c71ecd700 1 --2- 172.21.15.89:0/4161375473 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c09dac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009870 tx=0x7f5c60009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c71ecd700 1 -- 172.21.15.89:0/4161375473 shutdown_connections 2023-09-30T22:16:27.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c71ecd700 1 --2- 172.21.15.89:0/4161375473 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c09dac0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c71ecd700 1 --2- 172.21.15.89:0/4161375473 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e410 0x7f5c6c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c71ecd700 1 -- 172.21.15.89:0/4161375473 >> 172.21.15.89:0/4161375473 conn(0x7f5c6c066f50 msgr2=0x7f5c6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:27.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c71ecd700 1 -- 172.21.15.89:0/4161375473 shutdown_connections 2023-09-30T22:16:27.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.430+0000 7f5c71ecd700 1 -- 172.21.15.89:0/4161375473 wait complete. 2023-09-30T22:16:27.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c71ecd700 1 Processor -- start 2023-09-30T22:16:27.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c71ecd700 1 -- start start 2023-09-30T22:16:27.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c71ecd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c11ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c71ecd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e000 0x7f5c6c11f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c71ecd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c6c09e410 0x7f5c6c11f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c71ecd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1302e0 con 0x7f5c6c09e410 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c71ecd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c120370 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c71ecd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1219b0 con 0x7f5c6c09e000 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c6b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c11ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c6affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e000 0x7f5c6c11f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.432+0000 7f5c6b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c11ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42108/0 (socket says 172.21.15.89:42108) 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c6b7fe700 1 -- 172.21.15.89:0/1538873820 learned_addr learned my addr 172.21.15.89:0/1538873820 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c6bfff700 1 --2- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c6c09e410 0x7f5c6c11f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c6b7fe700 1 -- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e000 msgr2=0x7f5c6c11f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c6b7fe700 1 --2- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e000 0x7f5c6c11f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c6b7fe700 1 -- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c6c09e410 msgr2=0x7f5c6c11f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c6b7fe700 1 --2- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c6c09e410 0x7f5c6c11f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c6b7fe700 1 -- 172.21.15.89:0/1538873820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c600094a0 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c6b7fe700 1 --2- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c11ef10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c09d470 tx=0x7f5c60004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c53fff700 1 -- 172.21.15.89:0/1538873820 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60004450 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c53fff700 1 -- 172.21.15.89:0/1538873820 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6001f070 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c122490 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c53fff700 1 -- 172.21.15.89:0/1538873820 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c600038e0 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.433+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c09c700 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.434+0000 7f5c53fff700 1 -- 172.21.15.89:0/1538873820 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5c60003a40 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.434+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c074ea0 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.434+0000 7f5c53fff700 1 -- 172.21.15.89:0/1538873820 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f5c60021070 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.438+0000 7f5c53fff700 1 -- 172.21.15.89:0/1538873820 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c60015020 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.624+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c6c122620 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.625+0000 7f5c53fff700 1 -- 172.21.15.89:0/1538873820 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12866 (secure 0 0 0) 0x7f5c60003cf0 con 0x7f5c6c09d6f0 2023-09-30T22:16:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:27.625 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":23,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:21.418378+0000","last_up_change":"2023-09-30T22:13:54.525618+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 >> v1:172.21.15.89:6816/126953 conn(0x7f5c5802f510 legacy=0x7f5c580319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 msgr2=0x7f5c6c11ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 --2- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c11ef10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c09d470 tx=0x7f5c60004750 comp rx=0 tx=0).stop 2023-09-30T22:16:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 shutdown_connections 2023-09-30T22:16:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 --2- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c6c09e410 0x7f5c6c11f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 --2- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c6c09d6f0 0x7f5c6c11ef10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 --2- 172.21.15.89:0/1538873820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c6c09e000 0x7f5c6c11f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 >> 172.21.15.89:0/1538873820 conn(0x7f5c6c066f50 msgr2=0x7f5c6c119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 shutdown_connections 2023-09-30T22:16:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.627+0000 7f5c71ecd700 1 -- 172.21.15.89:0/1538873820 wait complete. 2023-09-30T22:16:27.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:16:27.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.759+0000 7f948be19700 1 Processor -- start 2023-09-30T22:16:27.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f948be19700 1 -- start start 2023-09-30T22:16:27.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f948be19700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409d6f0 0x7f948409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f948be19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f948409e000 0x7f94841339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f948be19700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f9484133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f948be19700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948406b460 con 0x7f948409e000 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f948be19700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948409b120 con 0x7f948409e410 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f948be19700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f948409a7e0 con 0x7f948409d6f0 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f9489bb5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409d6f0 0x7f948409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f94893b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f948409e000 0x7f94841339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f948a3b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f9484133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f9489bb5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409d6f0 0x7f948409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46306/0 (socket says 172.21.15.89:46306) 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f94893b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f948409e000 0x7f94841339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32946/0 (socket says 172.21.15.89:32946) 2023-09-30T22:16:27.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.760+0000 7f9489bb5700 1 -- 172.21.15.89:0/858873862 learned_addr learned my addr 172.21.15.89:0/858873862 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f94893b4700 1 -- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409d6f0 msgr2=0x7f948409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f94893b4700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409d6f0 0x7f948409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f94893b4700 1 -- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 msgr2=0x7f9484133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f94893b4700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f9484133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f94893b4700 1 -- 172.21.15.89:0/858873862 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948409a500 con 0x7f948409e000 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f9489bb5700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409d6f0 0x7f948409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f948a3b6700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f9484133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f94893b4700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409e000 0x7f94841339f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9480009840 tx=0x7f9480009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9392876723a73bb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f9488bb3700 1 -- 172.21.15.89:0/858873862 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480011830 con 0x7f948409e000 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f9488bb3700 1 -- 172.21.15.89:0/858873862 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9480011990 con 0x7f948409e000 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f9488bb3700 1 -- 172.21.15.89:0/858873862 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480011d20 con 0x7f948409e000 2023-09-30T22:16:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f948be19700 1 -- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409e000 msgr2=0x7f94841339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f948be19700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409e000 0x7f94841339f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9480009840 tx=0x7f9480009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:27.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f948be19700 1 -- 172.21.15.89:0/858873862 shutdown_connections 2023-09-30T22:16:27.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f948be19700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f9484133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f948be19700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409d6f0 0x7f948409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f948be19700 1 --2- 172.21.15.89:0/858873862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409e000 0x7f94841339f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.761+0000 7f948be19700 1 -- 172.21.15.89:0/858873862 >> 172.21.15.89:0/858873862 conn(0x7f9484066f50 msgr2=0x7f948409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:27.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.762+0000 7f948be19700 1 -- 172.21.15.89:0/858873862 shutdown_connections 2023-09-30T22:16:27.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.762+0000 7f948be19700 1 -- 172.21.15.89:0/858873862 wait complete. 2023-09-30T22:16:27.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.762+0000 7f948be19700 1 Processor -- start 2023-09-30T22:16:27.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.762+0000 7f948be19700 1 -- start start 2023-09-30T22:16:27.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948be19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409d6f0 0x7f94841a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948be19700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409e000 0x7f94841a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948be19700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f94841a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:27.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948be19700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94841a9a40 con 0x7f948409d6f0 2023-09-30T22:16:27.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948be19700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94841aa280 con 0x7f948409e410 2023-09-30T22:16:27.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948be19700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94841a79b0 con 0x7f948409e000 2023-09-30T22:16:27.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f94893b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409e000 0x7f94841a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948a3b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f94841a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f94893b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409e000 0x7f94841a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46318/0 (socket says 172.21.15.89:46318) 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f94893b4700 1 -- 172.21.15.89:0/1476976070 learned_addr learned my addr 172.21.15.89:0/1476976070 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f9489bb5700 1 --2- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409d6f0 0x7f94841a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948a3b6700 1 -- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409e000 msgr2=0x7f94841a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948a3b6700 1 --2- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409e000 0x7f94841a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948a3b6700 1 -- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409d6f0 msgr2=0x7f94841a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948a3b6700 1 --2- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409d6f0 0x7f94841a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948a3b6700 1 -- 172.21.15.89:0/1476976070 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009550 con 0x7f948409e410 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.763+0000 7f948a3b6700 1 --2- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f94841a6820 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f947c00efc0 tx=0x7f947c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.764+0000 7f947a7fc700 1 -- 172.21.15.89:0/1476976070 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c00ca90 con 0x7f948409e410 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.764+0000 7f947a7fc700 1 -- 172.21.15.89:0/1476976070 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c00f040 con 0x7f948409e410 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.764+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c009a70 con 0x7f948409e410 2023-09-30T22:16:27.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.764+0000 7f947a7fc700 1 -- 172.21.15.89:0/1476976070 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c00ce30 con 0x7f948409e410 2023-09-30T22:16:27.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.764+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94800094a0 con 0x7f948409e410 2023-09-30T22:16:27.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.764+0000 7f947a7fc700 1 -- 172.21.15.89:0/1476976070 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f947c014480 con 0x7f948409e410 2023-09-30T22:16:27.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.764+0000 7f947a7fc700 1 -- 172.21.15.89:0/1476976070 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f947c040210 con 0x7f948409e410 2023-09-30T22:16:27.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.764+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948409c770 con 0x7f948409e410 2023-09-30T22:16:27.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.769+0000 7f947a7fc700 1 -- 172.21.15.89:0/1476976070 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f947c018070 con 0x7f948409e410 2023-09-30T22:16:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.990+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f948406c360 con 0x7f948409e410 2023-09-30T22:16:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.990+0000 7f947a7fc700 1 -- 172.21.15.89:0/1476976070 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2315 (secure 0 0 0) 0x7f947c018070 con 0x7f948409e410 2023-09-30T22:16:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:27.991 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:14:02.037584+0000","modified":"2023-09-30T22:14:26.107818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4190":{"gid":4190,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6817/3539463463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6817","nonce":3539463463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:16:27.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 >> v1:172.21.15.89:6816/126953 conn(0x7f946802f510 legacy=0x7f94680319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 msgr2=0x7f94841a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:27.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 --2- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f94841a6820 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f947c00efc0 tx=0x7f947c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 shutdown_connections 2023-09-30T22:16:27.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 --2- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948409e410 0x7f94841a6820 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 --2- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948409d6f0 0x7f94841a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 --2- 172.21.15.89:0/1476976070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948409e000 0x7f94841a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:27.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 >> 172.21.15.89:0/1476976070 conn(0x7f9484066f50 msgr2=0x7f94841bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:27.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 shutdown_connections 2023-09-30T22:16:27.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:27.993+0000 7f948be19700 1 -- 172.21.15.89:0/1476976070 wait complete. 2023-09-30T22:16:27.993 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 7 2023-09-30T22:16:28.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:14:02.037584+0000', 'modified': '2023-09-30T22:14:26.107818+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4190': {'gid': 4190, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6817/3539463463', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.89:6817', 'nonce': 3539463463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-30T22:16:28.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:16:28.105 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e59655700 1 Processor -- start 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e59655700 1 -- start start 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e59655700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e5409e850 0x7f6e5409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e59655700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f160 0x7f6e54133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e59655700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5406bec0 con 0x7f6e5409f5b0 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e59655700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e540a0030 con 0x7f6e5409e850 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e59655700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540a0830 con 0x7f6e5409f160 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f160 0x7f6e54133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e52ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e5409e850 0x7f6e5409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e52ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e5409e850 0x7f6e5409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32968/0 (socket says 172.21.15.89:32968) 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f160 0x7f6e54133c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42138/0 (socket says 172.21.15.89:42138) 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e52ffd700 1 -- 172.21.15.89:0/3265829033 learned_addr learned my addr 172.21.15.89:0/3265829033 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.228+0000 7f6e537fe700 1 --1- 172.21.15.89:0/3265829033 >> v1:172.21.15.89:6789/0 conn(0x7f6e5409f5b0 0x7f6e541341a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53022/0 (socket says 172.21.15.89:53022) 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e527fc700 1 -- 172.21.15.89:0/3265829033 >> v2:172.21.15.89:3300/0 conn(0x7f6e5409e850 msgr2=0x7f6e5409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e527fc700 1 --2- 172.21.15.89:0/3265829033 >> v2:172.21.15.89:3300/0 conn(0x7f6e5409e850 0x7f6e5409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e527fc700 1 -- 172.21.15.89:0/3265829033 >> v1:172.21.15.89:6789/0 conn(0x7f6e5409f5b0 legacy=0x7f6e541341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e527fc700 1 -- 172.21.15.89:0/3265829033 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5406caa0 con 0x7f6e5409f160 2023-09-30T22:16:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/3265829033 >> v2:172.21.15.89:3300/0 conn(0x7f6e5409e850 0x7f6e5409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e527fc700 1 --2- 172.21.15.89:0/3265829033 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f160 0x7f6e54133c60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009840 tx=0x7f6e48009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df131887c116249f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e51ffb700 1 -- 172.21.15.89:0/3265829033 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48011830 con 0x7f6e5409f160 2023-09-30T22:16:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e51ffb700 1 -- 172.21.15.89:0/3265829033 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e48011990 con 0x7f6e5409f160 2023-09-30T22:16:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.229+0000 7f6e51ffb700 1 -- 172.21.15.89:0/3265829033 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48011d20 con 0x7f6e5409f160 2023-09-30T22:16:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.230+0000 7f6e59655700 1 -- 172.21.15.89:0/3265829033 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f160 msgr2=0x7f6e54133c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.230+0000 7f6e59655700 1 --2- 172.21.15.89:0/3265829033 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f160 0x7f6e54133c60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009840 tx=0x7f6e48009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.230+0000 7f6e59655700 1 -- 172.21.15.89:0/3265829033 shutdown_connections 2023-09-30T22:16:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.230+0000 7f6e59655700 1 --2- 172.21.15.89:0/3265829033 >> v2:172.21.15.89:3300/0 conn(0x7f6e5409e850 0x7f6e5409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.230+0000 7f6e59655700 1 --2- 172.21.15.89:0/3265829033 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f160 0x7f6e54133c60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.230+0000 7f6e59655700 1 -- 172.21.15.89:0/3265829033 >> 172.21.15.89:0/3265829033 conn(0x7f6e540a3f70 msgr2=0x7f6e540a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.230+0000 7f6e59655700 1 -- 172.21.15.89:0/3265829033 shutdown_connections 2023-09-30T22:16:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.230+0000 7f6e59655700 1 -- 172.21.15.89:0/3265829033 wait complete. 2023-09-30T22:16:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.231+0000 7f6e59655700 1 Processor -- start 2023-09-30T22:16:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.231+0000 7f6e59655700 1 -- start start 2023-09-30T22:16:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.231+0000 7f6e59655700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e5409e850 0x7f6e54120400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.231+0000 7f6e59655700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5409f160 0x7f6e54120940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.231+0000 7f6e59655700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f5b0 0x7f6e54120e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e59655700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e54123f20 con 0x7f6e5409f160 2023-09-30T22:16:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e59655700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541232d0 con 0x7f6e5409f5b0 2023-09-30T22:16:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e59655700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541220b0 con 0x7f6e5409e850 2023-09-30T22:16:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5409f160 0x7f6e54120940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f5b0 0x7f6e54120e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5409f160 0x7f6e54120940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32972/0 (socket says 172.21.15.89:32972) 2023-09-30T22:16:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 -- 172.21.15.89:0/2963052775 learned_addr learned my addr 172.21.15.89:0/2963052775 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 -- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e5409e850 msgr2=0x7f6e54120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 --2- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e5409e850 0x7f6e54120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 -- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f5b0 msgr2=0x7f6e54120e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 --2- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f5b0 0x7f6e54120e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 -- 172.21.15.89:0/2963052775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e44009550 con 0x7f6e5409f160 2023-09-30T22:16:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.232+0000 7f6e527fc700 1 --2- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5409f160 0x7f6e54120940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009fd0 tx=0x7f6e48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.233+0000 7f6e3b7fe700 1 -- 172.21.15.89:0/2963052775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e480036a0 con 0x7f6e5409f160 2023-09-30T22:16:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.234+0000 7f6e3b7fe700 1 -- 172.21.15.89:0/2963052775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4801e070 con 0x7f6e5409f160 2023-09-30T22:16:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.234+0000 7f6e3b7fe700 1 -- 172.21.15.89:0/2963052775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48003a60 con 0x7f6e5409f160 2023-09-30T22:16:28.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.234+0000 7f6e59655700 1 -- 172.21.15.89:0/2963052775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e480094a0 con 0x7f6e5409f160 2023-09-30T22:16:28.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.234+0000 7f6e59655700 1 -- 172.21.15.89:0/2963052775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e540a3600 con 0x7f6e5409f160 2023-09-30T22:16:28.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.234+0000 7f6e3b7fe700 1 -- 172.21.15.89:0/2963052775 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6e48003bc0 con 0x7f6e5409f160 2023-09-30T22:16:28.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.234+0000 7f6e3b7fe700 1 -- 172.21.15.89:0/2963052775 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f6e48040ad0 con 0x7f6e5409f160 2023-09-30T22:16:28.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.237+0000 7f6e657f8b80 1 Processor -- start 2023-09-30T22:16:28.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.237+0000 7f6e657f8b80 1 -- start start 2023-09-30T22:16:28.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699dac50 0x55c3699db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c3699db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f18f0 0x55c3699f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36993dc60 con 0x55c3699f18f0 2023-09-30T22:16:28.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c369938a10 con 0x55c3699dac50 2023-09-30T22:16:28.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c369936210 con 0x55c3699f7100 2023-09-30T22:16:28.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699dac50 0x55c3699db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f18f0 0x55c3699f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699dac50 0x55c3699db020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42158/0 (socket says 172.21.15.89:42158) 2023-09-30T22:16:28.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f18f0 0x55c3699f1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32978/0 (socket says 172.21.15.89:32978) 2023-09-30T22:16:28.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e52ffd700 1 -- 172.21.15.89:0/3782484548 learned_addr learned my addr 172.21.15.89:0/3782484548 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e527fc700 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c3699db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e537fe700 1 -- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 msgr2=0x55c3699db560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e537fe700 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c3699db560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e537fe700 1 -- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699dac50 msgr2=0x55c3699db020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e537fe700 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699dac50 0x55c3699db020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e537fe700 1 -- 172.21.15.89:0/3782484548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c369725790 con 0x55c3699f18f0 2023-09-30T22:16:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699dac50 0x55c3699db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e527fc700 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c3699db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e537fe700 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f18f0 0x55c3699f1d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0021c0 tx=0x7f6e4c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c428cdbb9515c466 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:28.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e38ff9700 1 -- 172.21.15.89:0/3782484548 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c017070 con 0x55c3699f18f0 2023-09-30T22:16:28.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e38ff9700 1 -- 172.21.15.89:0/3782484548 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c0133f0 con 0x55c3699f18f0 2023-09-30T22:16:28.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.238+0000 7f6e38ff9700 1 -- 172.21.15.89:0/3782484548 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c013560 con 0x55c3699f18f0 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f18f0 msgr2=0x55c3699f1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f18f0 0x55c3699f1d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0021c0 tx=0x7f6e4c0021f0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3782484548 shutdown_connections 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f18f0 0x55c3699f1d60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699dac50 0x55c3699db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/3782484548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c3699db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3782484548 >> 172.21.15.89:0/3782484548 conn(0x55c3699df0f0 msgr2=0x55c3699de690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3782484548 shutdown_connections 2023-09-30T22:16:28.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.239+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3782484548 wait complete. 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 1 -- start start 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 1 Processor -- start 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699fff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c369a00480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c369a009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c369931b00 con 0x55c3699dac50 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3698cd770 con 0x55c3699f18f0 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c3698f06f0 con 0x55c3699f7100 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e657f8b80 10 client.0 _ll_register_callbacks cb 0x55c3697395a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:16:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c369a00480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c369a00480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42168/0 (socket says 172.21.15.89:42168) 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32994/0 (socket says 172.21.15.89:32994) 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e527fc700 1 -- 172.21.15.89:0/119491771 learned_addr learned my addr 172.21.15.89:0/119491771 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.240+0000 7f6e537fe700 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c369a009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.241+0000 7f6e52ffd700 1 -- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 msgr2=0x55c369a009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.241+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c369a009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.241+0000 7f6e52ffd700 1 -- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 msgr2=0x55c369a00480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.241+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c369a00480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.241+0000 7f6e52ffd700 1 -- 172.21.15.89:0/119491771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e540a3600 con 0x55c3699dac50 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.241+0000 7f6e537fe700 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c369a009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.241+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699fff40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6e44013ed0 tx=0x7f6e440115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7902428726110 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.241+0000 7f6e31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-30T22:16:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e31ffb700 1 -- 172.21.15.89:0/119491771 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44011b00 con 0x55c3699dac50 2023-09-30T22:16:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e31ffb700 1 -- 172.21.15.89:0/119491771 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f6e44011c60 con 0x55c3699dac50 2023-09-30T22:16:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e657f8b80 1 -- 172.21.15.89:0/119491771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3699d32b0 con 0x55c3699dac50 2023-09-30T22:16:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e31ffb700 1 -- 172.21.15.89:0/119491771 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e44011e20 con 0x55c3699dac50 2023-09-30T22:16:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e657f8b80 10 client.14463 fetch_fsmap learned FSMap version 7 2023-09-30T22:16:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e657f8b80 1 -- 172.21.15.89:0/119491771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=3+}) v3 -- 0x7f6e5406bec0 con 0x55c3699dac50 2023-09-30T22:16:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e31ffb700 1 -- 172.21.15.89:0/119491771 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3699d32b0 con 0x55c3699dac50 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e31ffb700 1 -- 172.21.15.89:0/119491771 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 7) v1 ==== 1436+0+0 (secure 0 0 0) 0x7f6e4401b020 con 0x55c3699dac50 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e657f8b80 10 client.14463 fetch_fsmap finished waiting for FSMap version 7 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e657f8b80 10 client.14463 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e657f8b80 20 client.14463 populate_metadata read hostname 'smithi089' 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e657f8b80 4 client.14463 mds_command: new command op to 4190 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e657f8b80 1 -- 172.21.15.89:0/119491771 --> v1:172.21.15.89:6817/3539463463 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3699aff40 con 0x55c369a0bc10 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.243+0000 7f6e31ffb700 10 client.14463 ms_handle_connect on v1:172.21.15.89:6817/3539463463 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e31ffb700 1 -- 172.21.15.89:0/119491771 <== mds.0 v1:172.21.15.89:6817/3539463463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x55c3699aff40 con 0x55c369a0bc10 2023-09-30T22:16:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e31ffb700 10 client.14463 handle_command_reply: tid=0 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 client.14463 shutdown 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 -- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 msgr2=0x55c3699fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699fff40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6e44013ed0 tx=0x7f6e440115a0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 -- 172.21.15.89:0/119491771 shutdown_connections 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f7100 0x55c369a009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e31ffb700 0 client.14463 ms_handle_reset on v1:172.21.15.89:6817/3539463463 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699fff40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/119491771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c369a00480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 -- 172.21.15.89:0/119491771 >> 172.21.15.89:0/119491771 conn(0x55c3699df0f0 msgr2=0x55c3699db230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 -- 172.21.15.89:0/119491771 shutdown_connections 2023-09-30T22:16:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.244+0000 7f6e657f8b80 1 -- 172.21.15.89:0/119491771 wait complete. 2023-09-30T22:16:28.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.245+0000 7f6e657f8b80 20 client.14463 trim_cache size 0 max 16384 2023-09-30T22:16:28.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.261+0000 7f6e657f8b80 1 Processor -- start 2023-09-30T22:16:28.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.261+0000 7f6e657f8b80 1 -- start start 2023-09-30T22:16:28.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.261+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.261+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f18f0 0x55c3699f2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.261+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f7100 0x55c3699d9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.261+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c369726390 con 0x55c3699dac50 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.261+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3699fa660 con 0x55c3699f7100 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.261+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c3699f8730 con 0x55c3699f18f0 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699f1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32996/0 (socket says 172.21.15.89:32996) 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e52ffd700 1 -- 172.21.15.89:0/1320486594 learned_addr learned my addr 172.21.15.89:0/1320486594 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e527fc700 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f18f0 0x55c3699f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e537fe700 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f7100 0x55c3699d9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e527fc700 1 -- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f7100 msgr2=0x55c3699d9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e527fc700 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f7100 0x55c3699d9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e527fc700 1 -- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 msgr2=0x55c3699f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e527fc700 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e527fc700 1 -- 172.21.15.89:0/1320486594 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3698120a0 con 0x55c3699f18f0 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e527fc700 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f18f0 0x55c3699f2200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009b90 tx=0x7f6e4801c070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6408bbd162a3ec44 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e397fa700 1 -- 172.21.15.89:0/1320486594 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48010b20 con 0x55c3699f18f0 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e397fa700 1 -- 172.21.15.89:0/1320486594 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e480233b0 con 0x55c3699f18f0 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.262+0000 7f6e397fa700 1 -- 172.21.15.89:0/1320486594 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4801aca0 con 0x55c3699f18f0 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 -- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f18f0 msgr2=0x55c3699f2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f18f0 0x55c3699f2200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009b90 tx=0x7f6e4801c070 comp rx=0 tx=0).stop 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 -- 172.21.15.89:0/1320486594 shutdown_connections 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f7100 0x55c3699d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699dac50 0x55c3699f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/1320486594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699f18f0 0x55c3699f2200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 -- 172.21.15.89:0/1320486594 >> 172.21.15.89:0/1320486594 conn(0x55c3699df0f0 msgr2=0x55c3699d7210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 -- 172.21.15.89:0/1320486594 shutdown_connections 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 -- 172.21.15.89:0/1320486594 wait complete. 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 -- start start 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.263+0000 7f6e657f8b80 1 Processor -- start 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699dac50 0x55c3699feb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c3699ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e657f8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f7100 0x55c3699ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3699f74d0 con 0x55c3699f7100 2023-09-30T22:16:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c369a0f690 con 0x55c3699f18f0 2023-09-30T22:16:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e657f8b80 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c369a1b0c0 con 0x55c3699dac50 2023-09-30T22:16:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e657f8b80 10 client.0 _ll_register_callbacks cb 0x7f6e440136e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:16:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c3699ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699dac50 0x55c3699feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c3699ff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42192/0 (socket says 172.21.15.89:42192) 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e527fc700 1 -- 172.21.15.89:0/3249452281 learned_addr learned my addr 172.21.15.89:0/3249452281 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e537fe700 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f7100 0x55c3699ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e52ffd700 1 -- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 msgr2=0x55c3699ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c3699ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e52ffd700 1 -- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f7100 msgr2=0x55c3699ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f7100 0x55c3699ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e52ffd700 1 -- 172.21.15.89:0/3249452281 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5406bec0 con 0x55c3699dac50 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e527fc700 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c3699ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e52ffd700 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699dac50 0x55c3699feb90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009dc0 tx=0x7f6e44016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5714339b8240dd5e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3302/0 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e31ffb700 1 -- 172.21.15.89:0/3249452281 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4402e040 con 0x55c3699dac50 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e31ffb700 1 -- 172.21.15.89:0/3249452281 <== mon.2 v2:172.21.15.89:3302/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f6e44015420 con 0x55c3699dac50 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.264+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3249452281 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3699dc820 con 0x55c3699dac50 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e31ffb700 1 -- 172.21.15.89:0/3249452281 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e440118f0 con 0x55c3699dac50 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 10 client.14327 fetch_fsmap learned FSMap version 7 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3249452281 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,fsmap=7,monmap=3+}) v3 -- 0x7f6e480094a0 con 0x55c3699dac50 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e31ffb700 1 -- 172.21.15.89:0/3249452281 <== mon.2 v2:172.21.15.89:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3699dc820 con 0x55c3699dac50 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e31ffb700 1 -- 172.21.15.89:0/3249452281 <== mon.2 v2:172.21.15.89:3302/0 5 ==== fsmap(e 7) v1 ==== 1436+0+0 (secure 0 0 0) 0x7f6e4401b070 con 0x55c3699dac50 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 10 client.14327 fetch_fsmap finished waiting for FSMap version 7 2023-09-30T22:16:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 10 client.14327 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:16:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 20 client.14327 populate_metadata read hostname 'smithi089' 2023-09-30T22:16:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 4 client.14327 mds_command: new command op to 4190 tid=0[{"prefix": "status"}] 2023-09-30T22:16:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3249452281 --> v1:172.21.15.89:6817/3539463463 -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f6e4c0133f0 con 0x55c369a5d430 2023-09-30T22:16:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e31ffb700 10 client.14327 ms_handle_connect on v1:172.21.15.89:6817/3539463463 2023-09-30T22:16:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e31ffb700 1 -- 172.21.15.89:0/3249452281 <== mds.0 v1:172.21.15.89:6817/3539463463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (unknown 0 0 3557561587) 0x7f6e4c0133f0 con 0x55c369a5d430 2023-09-30T22:16:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e31ffb700 10 client.14327 handle_command_reply: tid=0 2023-09-30T22:16:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 1 client.14327 shutdown 2023-09-30T22:16:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699dac50 msgr2=0x55c3699feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699dac50 0x55c3699feb90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009dc0 tx=0x7f6e44016000 comp rx=0 tx=0).stop 2023-09-30T22:16:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.265+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3249452281 shutdown_connections 2023-09-30T22:16:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3699f7100 0x55c3699ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x55c3699dac50 0x55c3699feb90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e31ffb700 0 client.14327 ms_handle_reset on v1:172.21.15.89:6817/3539463463 2023-09-30T22:16:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e657f8b80 1 --2- 172.21.15.89:0/3249452281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x55c3699f18f0 0x55c3699ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3249452281 >> 172.21.15.89:0/3249452281 conn(0x55c3699df0f0 msgr2=0x55c3699f1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3249452281 shutdown_connections 2023-09-30T22:16:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e657f8b80 1 -- 172.21.15.89:0/3249452281 wait complete. 2023-09-30T22:16:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e657f8b80 20 client.14327 trim_cache size 0 max 16384 2023-09-30T22:16:28.276 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-30T22:16:28.276 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "389f6e95-ea58-47bd-a145-778ba0724a9e", 2023-09-30T22:16:28.276 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-09-30T22:16:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4190, 2023-09-30T22:16:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-09-30T22:16:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-09-30T22:16:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-09-30T22:16:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 146.22405853999999, 2023-09-30T22:16:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 7, 2023-09-30T22:16:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 23, 2023-09-30T22:16:28.278 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 18, 2023-09-30T22:16:28.278 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 150.62219172499999 2023-09-30T22:16:28.278 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-30T22:16:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 -- 172.21.15.89:0/2963052775 >> v1:172.21.15.89:6816/126953 conn(0x7f6e4002f510 legacy=0x7f6e400319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 -- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5409f160 msgr2=0x7f6e54120940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 --2- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5409f160 0x7f6e54120940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009fd0 tx=0x7f6e48004750 comp rx=0 tx=0).stop 2023-09-30T22:16:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 -- 172.21.15.89:0/2963052775 shutdown_connections 2023-09-30T22:16:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 --2- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e5409f5b0 0x7f6e54120e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 --2- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e5409e850 0x7f6e54120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 --2- 172.21.15.89:0/2963052775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5409f160 0x7f6e54120940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 -- 172.21.15.89:0/2963052775 >> 172.21.15.89:0/2963052775 conn(0x7f6e540a3f70 msgr2=0x7f6e541bb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 -- 172.21.15.89:0/2963052775 shutdown_connections 2023-09-30T22:16:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.266+0000 7f6e59655700 1 -- 172.21.15.89:0/2963052775 wait complete. 2023-09-30T22:16:28.281 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:16:28.292 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-30T22:16:28.292 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:28.316 INFO:tasks.ceph.osd.0:Stopped 2023-09-30T22:16:28.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2023-09-30T22:16:28.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634ed5e700 1 Processor -- start 2023-09-30T22:16:28.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634ed5e700 1 -- start start 2023-09-30T22:16:28.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634ed5e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f634809d6f0 0x7f634809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634ed5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634ed5e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f6348133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634ed5e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634806b460 con 0x7f634809d6f0 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634ed5e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634809b120 con 0x7f634809e000 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634ed5e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f634809a7e0 con 0x7f634809e410 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f6347fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634cafa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f634809d6f0 0x7f634809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634d2fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f6348133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f6347fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42194/0 (socket says 172.21.15.89:42194) 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634cafa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f634809d6f0 0x7f634809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33018/0 (socket says 172.21.15.89:33018) 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f634d2fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f6348133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46372/0 (socket says 172.21.15.89:46372) 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f6347fff700 1 -- 172.21.15.89:0/130649069 learned_addr learned my addr 172.21.15.89:0/130649069 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:28.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f6347fff700 1 -- 172.21.15.89:0/130649069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 msgr2=0x7f6348133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f6347fff700 1 --2- 172.21.15.89:0/130649069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f6348133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f6347fff700 1 -- 172.21.15.89:0/130649069 >> v2:172.21.15.89:3300/0 conn(0x7f634809d6f0 msgr2=0x7f634809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:28.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f6347fff700 1 --2- 172.21.15.89:0/130649069 >> v2:172.21.15.89:3300/0 conn(0x7f634809d6f0 0x7f634809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.478+0000 7f6347fff700 1 -- 172.21.15.89:0/130649069 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634809a500 con 0x7f634809e000 2023-09-30T22:16:28.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634d2fb700 1 --2- 172.21.15.89:0/130649069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f6348133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634cafa700 1 --2- 172.21.15.89:0/130649069 >> v2:172.21.15.89:3300/0 conn(0x7f634809d6f0 0x7f634809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f6347fff700 1 --2- 172.21.15.89:0/130649069 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481339f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6334009920 tx=0x7f6334009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d5f54ce2626801 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f63477fe700 1 -- 172.21.15.89:0/130649069 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334011830 con 0x7f634809e000 2023-09-30T22:16:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f63477fe700 1 -- 172.21.15.89:0/130649069 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6334011990 con 0x7f634809e000 2023-09-30T22:16:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f63477fe700 1 -- 172.21.15.89:0/130649069 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334011d20 con 0x7f634809e000 2023-09-30T22:16:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 -- 172.21.15.89:0/130649069 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 msgr2=0x7f63481339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 --2- 172.21.15.89:0/130649069 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481339f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6334009920 tx=0x7f6334009c30 comp rx=0 tx=0).stop 2023-09-30T22:16:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 -- 172.21.15.89:0/130649069 shutdown_connections 2023-09-30T22:16:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 --2- 172.21.15.89:0/130649069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f6348133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 --2- 172.21.15.89:0/130649069 >> v2:172.21.15.89:3300/0 conn(0x7f634809d6f0 0x7f634809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 --2- 172.21.15.89:0/130649069 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481339f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 -- 172.21.15.89:0/130649069 >> 172.21.15.89:0/130649069 conn(0x7f6348066f50 msgr2=0x7f634809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 -- 172.21.15.89:0/130649069 shutdown_connections 2023-09-30T22:16:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.479+0000 7f634ed5e700 1 -- 172.21.15.89:0/130649069 wait complete. 2023-09-30T22:16:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.480+0000 7f634ed5e700 1 Processor -- start 2023-09-30T22:16:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.480+0000 7f634ed5e700 1 -- start start 2023-09-30T22:16:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634ed5e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f634809d6f0 0x7f63481a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634ed5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634ed5e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f63481a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634ed5e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63481a98e0 con 0x7f634809d6f0 2023-09-30T22:16:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634ed5e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63481a7ff0 con 0x7f634809e000 2023-09-30T22:16:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634ed5e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63481aa280 con 0x7f634809e410 2023-09-30T22:16:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634d2fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f63481a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42210/0 (socket says 172.21.15.89:42210) 2023-09-30T22:16:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 -- 172.21.15.89:0/783828219 learned_addr learned my addr 172.21.15.89:0/783828219 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634cafa700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f634809d6f0 0x7f63481a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 -- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 msgr2=0x7f63481a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f63481a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 -- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f634809d6f0 msgr2=0x7f63481a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f634809d6f0 0x7f63481a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 -- 172.21.15.89:0/783828219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0094a0 con 0x7f634809e000 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634cafa700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f634809d6f0 0x7f63481a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f634d2fb700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f63481a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.481+0000 7f6347fff700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63340098f0 tx=0x7f6334004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.482+0000 7f63457fa700 1 -- 172.21.15.89:0/783828219 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334004240 con 0x7f634809e000 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.482+0000 7f63457fa700 1 -- 172.21.15.89:0/783828219 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633401f070 con 0x7f634809e000 2023-09-30T22:16:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.482+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6340009a70 con 0x7f634809e000 2023-09-30T22:16:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.482+0000 7f63457fa700 1 -- 172.21.15.89:0/783828219 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340036a0 con 0x7f634809e000 2023-09-30T22:16:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.482+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6334009550 con 0x7f634809e000 2023-09-30T22:16:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.482+0000 7f63457fa700 1 -- 172.21.15.89:0/783828219 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6334021030 con 0x7f634809e000 2023-09-30T22:16:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.483+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6348074ea0 con 0x7f634809e000 2023-09-30T22:16:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.483+0000 7f63457fa700 1 -- 172.21.15.89:0/783828219 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f6334040cd0 con 0x7f634809e000 2023-09-30T22:16:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.487+0000 7f63457fa700 1 -- 172.21.15.89:0/783828219 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6334015020 con 0x7f634809e000 2023-09-30T22:16:28.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:28.683+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd down", "ids": ["0"]} v 0) v1 -- 0x7f63481a79d0 con 0x7f634809e000 2023-09-30T22:16:28.683 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-09-30T22:16:28.683+0000 7f2d91337700 -1 mon.a@0(leader).osd e23 definitely_dead 0 2023-09-30T22:16:29.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.249+0000 7f63457fa700 1 -- 172.21.15.89:0/783828219 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["0"]}]=0 marked down osd.0. v24) v1 ==== 89+0+0 (secure 0 0 0) 0x7f6334032070 con 0x7f634809e000 2023-09-30T22:16:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.252+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 >> v1:172.21.15.89:6816/126953 conn(0x7f632c02f510 legacy=0x7f632c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.252+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 msgr2=0x7f63481a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.252+0000 7f634ed5e700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63340098f0 tx=0x7f6334004750 comp rx=0 tx=0).stop 2023-09-30T22:16:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.252+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 shutdown_connections 2023-09-30T22:16:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.252+0000 7f634ed5e700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f634809e410 0x7f63481a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.252+0000 7f634ed5e700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f634809d6f0 0x7f63481a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.252+0000 7f634ed5e700 1 --2- 172.21.15.89:0/783828219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f634809e000 0x7f63481a6300 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63340098f0 tx=0x7f6334004750 comp rx=0 tx=0).stop 2023-09-30T22:16:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.252+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 >> 172.21.15.89:0/783828219 conn(0x7f6348066f50 msgr2=0x7f63481bab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.253+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 shutdown_connections 2023-09-30T22:16:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.254+0000 7f634ed5e700 1 -- 172.21.15.89:0/783828219 wait complete. 2023-09-30T22:16:29.256 INFO:teuthology.orchestra.run.smithi089.stderr:marked down osd.0. 2023-09-30T22:16:29.270 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-30T22:16:29.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-30T22:16:29.273 INFO:tasks.ceph.osd.0:Started 2023-09-30T22:16:29.273 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-30T22:16:29.273 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:29.307 INFO:tasks.ceph.osd.1:Stopped 2023-09-30T22:16:29.307 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2023-09-30T22:16:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f568589d700 1 Processor -- start 2023-09-30T22:16:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f568589d700 1 -- start start 2023-09-30T22:16:29.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f568589d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568006f5d0 0x7f568012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:29.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f568589d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:29.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f568589d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568006fa40 con 0x7f568012cf60 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f568589d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568012cd90 con 0x7f5680135c20 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f568589d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5680071780 con 0x7f568006f5d0 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f567e7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568006f5d0 0x7f568012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f567f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f567effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f568012cf60 0x7f568012f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53052/0 (socket says 172.21.15.89:53052) 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f567effd700 1 -- 172.21.15.89:0/2355383194 learned_addr learned my addr 172.21.15.89:0/2355383194 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.471+0000 7f567e7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568006f5d0 0x7f568012ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46394/0 (socket says 172.21.15.89:46394) 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567dffb700 1 -- 172.21.15.89:0/2355383194 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258644838 0 0) 0x7f568006fa40 con 0x7f568012cf60 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567e7fc700 1 -- 172.21.15.89:0/2355383194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 msgr2=0x7f5680138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567e7fc700 1 --2- 172.21.15.89:0/2355383194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567e7fc700 1 -- 172.21.15.89:0/2355383194 >> v1:172.21.15.89:6789/0 conn(0x7f568012cf60 legacy=0x7f568012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567e7fc700 1 -- 172.21.15.89:0/2355383194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5680071a40 con 0x7f568006f5d0 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567f7fe700 1 --2- 172.21.15.89:0/2355383194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567e7fc700 1 --2- 172.21.15.89:0/2355383194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568006f5d0 0x7f568012ffc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f56740098a0 tx=0x7f5674009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d864a1ae0bef565 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567dffb700 1 -- 172.21.15.89:0/2355383194 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674011830 con 0x7f568006f5d0 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567dffb700 1 -- 172.21.15.89:0/2355383194 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5674011990 con 0x7f568006f5d0 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f567dffb700 1 -- 172.21.15.89:0/2355383194 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674011d20 con 0x7f568006f5d0 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f568589d700 1 -- 172.21.15.89:0/2355383194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568006f5d0 msgr2=0x7f568012ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f568589d700 1 --2- 172.21.15.89:0/2355383194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568006f5d0 0x7f568012ffc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f56740098a0 tx=0x7f5674009bb0 comp rx=0 tx=0).stop 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f568589d700 1 -- 172.21.15.89:0/2355383194 shutdown_connections 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f568589d700 1 --2- 172.21.15.89:0/2355383194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f568589d700 1 --2- 172.21.15.89:0/2355383194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568006f5d0 0x7f568012ffc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:29.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.472+0000 7f568589d700 1 -- 172.21.15.89:0/2355383194 >> 172.21.15.89:0/2355383194 conn(0x7f56800a2aa0 msgr2=0x7f56800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:29.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.473+0000 7f568589d700 1 -- 172.21.15.89:0/2355383194 shutdown_connections 2023-09-30T22:16:29.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.473+0000 7f568589d700 1 -- 172.21.15.89:0/2355383194 wait complete. 2023-09-30T22:16:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.473+0000 7f568589d700 1 Processor -- start 2023-09-30T22:16:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f568589d700 1 -- start start 2023-09-30T22:16:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f568589d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f568006f5d0 0x7f5680132ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f568589d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568012cf60 0x7f5680133530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f568589d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680133a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f568589d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5680134680 con 0x7f568006f5d0 2023-09-30T22:16:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f568589d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5680119a10 con 0x7f5680135c20 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f568589d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5680119050 con 0x7f568012cf60 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f568006f5d0 0x7f5680132ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680133a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f568006f5d0 0x7f5680132ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33032/0 (socket says 172.21.15.89:33032) 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567effd700 1 -- 172.21.15.89:0/3616203154 learned_addr learned my addr 172.21.15.89:0/3616203154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567e7fc700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568012cf60 0x7f5680133530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567f7fe700 1 -- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568012cf60 msgr2=0x7f5680133530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567f7fe700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568012cf60 0x7f5680133530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567f7fe700 1 -- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f568006f5d0 msgr2=0x7f5680132ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567f7fe700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f568006f5d0 0x7f5680132ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567f7fe700 1 -- 172.21.15.89:0/3616203154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567800bc80 con 0x7f5680135c20 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.474+0000 7f567e7fc700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568012cf60 0x7f5680133530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.475+0000 7f567f7fe700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680133a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f567800e340 tx=0x7f567800e650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.475+0000 7f5667fff700 1 -- 172.21.15.89:0/3616203154 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678010040 con 0x7f5680135c20 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.475+0000 7f5667fff700 1 -- 172.21.15.89:0/3616203154 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567800b400 con 0x7f5680135c20 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.475+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56740094a0 con 0x7f5680135c20 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.475+0000 7f5667fff700 1 -- 172.21.15.89:0/3616203154 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678014640 con 0x7f5680135c20 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.475+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5680134bd0 con 0x7f5680135c20 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.476+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5680074ea0 con 0x7f5680135c20 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.476+0000 7f5667fff700 1 -- 172.21.15.89:0/3616203154 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f56780147a0 con 0x7f5680135c20 2023-09-30T22:16:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.476+0000 7f5667fff700 1 -- 172.21.15.89:0/3616203154 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f5678041180 con 0x7f5680135c20 2023-09-30T22:16:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.481+0000 7f5667fff700 1 -- 172.21.15.89:0/3616203154 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5678014c30 con 0x7f5680135c20 2023-09-30T22:16:29.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:29.679+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd down", "ids": ["1"]} v 0) v1 -- 0x7f5680135a00 con 0x7f5680135c20 2023-09-30T22:16:29.679 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-09-30T22:16:29.679+0000 7f2d91337700 -1 mon.a@0(leader).osd e24 definitely_dead 0 2023-09-30T22:16:29.869 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-09-30T22:16:29.869+0000 7f7701c28680 -1 Falling back to public interface 2023-09-30T22:16:30.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.252+0000 7f5667fff700 1 -- 172.21.15.89:0/3616203154 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["1"]}]=0 marked down osd.1. v25) v1 ==== 89+0+0 (secure 0 0 0) 0x7f567800ab70 con 0x7f5680135c20 2023-09-30T22:16:30.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.256+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 >> v1:172.21.15.89:6816/126953 conn(0x7f566c02f510 legacy=0x7f566c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:30.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.256+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 msgr2=0x7f5680133a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:30.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.256+0000 7f568589d700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680133a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f567800e340 tx=0x7f567800e650 comp rx=0 tx=0).stop 2023-09-30T22:16:30.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.257+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 shutdown_connections 2023-09-30T22:16:30.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.257+0000 7f568589d700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5680135c20 0x7f5680133a70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:30.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.257+0000 7f568589d700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f568006f5d0 0x7f5680132ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:30.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.257+0000 7f568589d700 1 --2- 172.21.15.89:0/3616203154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f568012cf60 0x7f5680133530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:30.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.257+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 >> 172.21.15.89:0/3616203154 conn(0x7f56800a2aa0 msgr2=0x7f568006eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:30.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.260+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 shutdown_connections 2023-09-30T22:16:30.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.260+0000 7f568589d700 1 -- 172.21.15.89:0/3616203154 wait complete. 2023-09-30T22:16:30.262 INFO:teuthology.orchestra.run.smithi089.stderr:marked down osd.1. 2023-09-30T22:16:30.275 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-30T22:16:30.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-30T22:16:30.278 INFO:tasks.ceph.osd.1:Started 2023-09-30T22:16:30.278 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-30T22:16:30.279 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:30.310 INFO:tasks.ceph.osd.2:Stopped 2023-09-30T22:16:30.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2023-09-30T22:16:30.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6fe2f7700 1 Processor -- start 2023-09-30T22:16:30.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6fe2f7700 1 -- start start 2023-09-30T22:16:30.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6fe2f7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:30.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6fe2f7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:30.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6fe2f7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f812cd90 con 0x7ff6f8130260 2023-09-30T22:16:30.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6fe2f7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f812d330 con 0x7ff6f812f8b0 2023-09-30T22:16:30.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6fe2f7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f81306d0 con 0x7ff6f812cf60 2023-09-30T22:16:30.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6f7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6f7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48852/0 (socket says 172.21.15.89:48852) 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48254/0 (socket says 172.21.15.89:48254) 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.473+0000 7ff6f7fff700 1 -- 172.21.15.89:0/1845539043 learned_addr learned my addr 172.21.15.89:0/1845539043 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f77fe700 1 -- 172.21.15.89:0/1845539043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 msgr2=0x7ff6f806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f77fe700 1 --2- 172.21.15.89:0/1845539043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f77fe700 1 -- 172.21.15.89:0/1845539043 >> v1:172.21.15.89:6789/0 conn(0x7ff6f8130260 legacy=0x7ff6f8136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f77fe700 1 -- 172.21.15.89:0/1845539043 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8071780 con 0x7ff6f812f8b0 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f7fff700 1 --2- 172.21.15.89:0/1845539043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f6ffd700 1 -- 172.21.15.89:0/1845539043 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745910215 0 0) 0x7ff6f812cd90 con 0x7ff6f8130260 2023-09-30T22:16:30.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f77fe700 1 --2- 172.21.15.89:0/1845539043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f812fd20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009870 tx=0x7ff6ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8659339efb359e50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:30.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f6ffd700 1 -- 172.21.15.89:0/1845539043 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec011830 con 0x7ff6f812f8b0 2023-09-30T22:16:30.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f6ffd700 1 -- 172.21.15.89:0/1845539043 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec011990 con 0x7ff6f812f8b0 2023-09-30T22:16:30.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.474+0000 7ff6f6ffd700 1 -- 172.21.15.89:0/1845539043 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec011d20 con 0x7ff6f812f8b0 2023-09-30T22:16:30.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/1845539043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 msgr2=0x7ff6f812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 --2- 172.21.15.89:0/1845539043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f812fd20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009870 tx=0x7ff6ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/1845539043 shutdown_connections 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 --2- 172.21.15.89:0/1845539043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 --2- 172.21.15.89:0/1845539043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f812fd20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/1845539043 >> 172.21.15.89:0/1845539043 conn(0x7ff6f80a2aa0 msgr2=0x7ff6f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/1845539043 shutdown_connections 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/1845539043 wait complete. 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.475+0000 7ff6fe2f7700 1 Processor -- start 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6fe2f7700 1 -- start start 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6fe2f7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f8129220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:30.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6fe2f7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f8129760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6fe2f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f8130260 0x7ff6f809dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6fe2f7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8138380 con 0x7ff6f8130260 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6fe2f7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8138560 con 0x7ff6f812f8b0 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6fe2f7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8131380 con 0x7ff6f812cf60 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f8129760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f8129220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6fc894700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f8130260 0x7ff6f809dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f8129760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48270/0 (socket says 172.21.15.89:48270) 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f77fe700 1 -- 172.21.15.89:0/3749587884 learned_addr learned my addr 172.21.15.89:0/3749587884 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f77fe700 1 -- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 msgr2=0x7ff6f8129220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f77fe700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f8129220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:30.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f77fe700 1 -- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f8130260 msgr2=0x7ff6f809dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f77fe700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f8130260 0x7ff6f809dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.476+0000 7ff6f77fe700 1 -- 172.21.15.89:0/3749587884 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8009550 con 0x7ff6f812f8b0 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6f7fff700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f8129220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6fc894700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f8130260 0x7ff6f809dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6f77fe700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f8129760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009bc0 tx=0x7ff6ec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6f4ff9700 1 -- 172.21.15.89:0/3749587884 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec004510 con 0x7ff6f812f8b0 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6f4ff9700 1 -- 172.21.15.89:0/3749587884 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec01f070 con 0x7ff6f812f8b0 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6f4ff9700 1 -- 172.21.15.89:0/3749587884 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec0038e0 con 0x7ff6f812f8b0 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec0094a0 con 0x7ff6f812f8b0 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f81329e0 con 0x7ff6f812f8b0 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.477+0000 7ff6f4ff9700 1 -- 172.21.15.89:0/3749587884 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff6ec003a40 con 0x7ff6f812f8b0 2023-09-30T22:16:30.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.478+0000 7ff6f4ff9700 1 -- 172.21.15.89:0/3749587884 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4020+0+0 (secure 0 0 0) 0x7ff6ec021070 con 0x7ff6f812f8b0 2023-09-30T22:16:30.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.478+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f81386a0 con 0x7ff6f812f8b0 2023-09-30T22:16:30.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.482+0000 7ff6f4ff9700 1 -- 172.21.15.89:0/3749587884 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6f81386a0 con 0x7ff6f812f8b0 2023-09-30T22:16:30.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:30.679+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd down", "ids": ["2"]} v 0) v1 -- 0x7ff6f81386a0 con 0x7ff6f812f8b0 2023-09-30T22:16:30.679 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-09-30T22:16:30.679+0000 7f2d91337700 -1 mon.a@0(leader).osd e25 definitely_dead 0 2023-09-30T22:16:30.868 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-09-30T22:16:30.868+0000 7fdf3a35c680 -1 Falling back to public interface 2023-09-30T22:16:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.253+0000 7ff6f4ff9700 1 -- 172.21.15.89:0/3749587884 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["2"]}]=0 marked down osd.2. v26) v1 ==== 89+0+0 (secure 0 0 0) 0x7ff6f81386a0 con 0x7ff6f812f8b0 2023-09-30T22:16:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.256+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 >> v1:172.21.15.89:6816/126953 conn(0x7ff6e402f540 legacy=0x7ff6e40319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 msgr2=0x7ff6f8129760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f8129760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009bc0 tx=0x7ff6ec004750 comp rx=0 tx=0).stop 2023-09-30T22:16:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 shutdown_connections 2023-09-30T22:16:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f8130260 0x7ff6f809dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6f812cf60 0x7ff6f8129220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 --2- 172.21.15.89:0/3749587884 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6f812f8b0 0x7ff6f8129760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 >> 172.21.15.89:0/3749587884 conn(0x7ff6f80a2aa0 msgr2=0x7ff6f806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 shutdown_connections 2023-09-30T22:16:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.257+0000 7ff6fe2f7700 1 -- 172.21.15.89:0/3749587884 wait complete. 2023-09-30T22:16:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:marked down osd.2. 2023-09-30T22:16:31.274 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-30T22:16:31.274 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-30T22:16:31.277 INFO:tasks.ceph.osd.2:Started 2023-09-30T22:16:31.277 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-30T22:16:31.277 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:31.314 INFO:tasks.ceph.osd.3:Stopped 2023-09-30T22:16:31.314 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 3 2023-09-30T22:16:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e3e32e700 1 Processor -- start 2023-09-30T22:16:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e3e32e700 1 -- start start 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e3e32e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7e3809f410 0x7f7e38136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e3e32e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e381393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e3e32e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e38070940 con 0x7f7e3809e930 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e3e32e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3806fd60 con 0x7f7e3809f410 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e3e32e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3806fa80 con 0x7f7e38136fe0 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e377fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7e3809f410 0x7f7e38136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e37fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7e3809e930 0x7f7e3809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39366/0 (socket says 172.21.15.89:39366) 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e37fff700 1 -- 172.21.15.89:0/564273559 learned_addr learned my addr 172.21.15.89:0/564273559 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.477+0000 7f7e3c8cb700 1 --2- 172.21.15.89:0/564273559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e381393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:31.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e377fe700 1 -- 172.21.15.89:0/564273559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 msgr2=0x7f7e381393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e377fe700 1 --2- 172.21.15.89:0/564273559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e381393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e377fe700 1 -- 172.21.15.89:0/564273559 >> v1:172.21.15.89:6789/0 conn(0x7f7e3809e930 legacy=0x7f7e3809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e36ffd700 1 -- 172.21.15.89:0/564273559 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046726045 0 0) 0x7f7e38070940 con 0x7f7e3809e930 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e377fe700 1 -- 172.21.15.89:0/564273559 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3809fb60 con 0x7f7e3809f410 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e377fe700 1 --2- 172.21.15.89:0/564273559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e38136aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009840 tx=0x7f7e2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4544d24a40689b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e36ffd700 1 -- 172.21.15.89:0/564273559 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011830 con 0x7f7e3809f410 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e36ffd700 1 -- 172.21.15.89:0/564273559 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c011990 con 0x7f7e3809f410 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e36ffd700 1 -- 172.21.15.89:0/564273559 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011d20 con 0x7f7e3809f410 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e3e32e700 1 -- 172.21.15.89:0/564273559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 msgr2=0x7f7e38136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e3e32e700 1 --2- 172.21.15.89:0/564273559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e38136aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009840 tx=0x7f7e2c009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:31.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e3e32e700 1 -- 172.21.15.89:0/564273559 shutdown_connections 2023-09-30T22:16:31.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e3e32e700 1 --2- 172.21.15.89:0/564273559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e381393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:31.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e3e32e700 1 --2- 172.21.15.89:0/564273559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e38136aa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:31.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.478+0000 7f7e3e32e700 1 -- 172.21.15.89:0/564273559 >> 172.21.15.89:0/564273559 conn(0x7f7e380a4210 msgr2=0x7f7e380a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:31.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.479+0000 7f7e3e32e700 1 -- 172.21.15.89:0/564273559 shutdown_connections 2023-09-30T22:16:31.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.479+0000 7f7e3e32e700 1 -- 172.21.15.89:0/564273559 wait complete. 2023-09-30T22:16:31.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.479+0000 7f7e3e32e700 1 Processor -- start 2023-09-30T22:16:31.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e3e32e700 1 -- start start 2023-09-30T22:16:31.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e3e32e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e3809e930 0x7f7e3811c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e3e32e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e3811ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e3e32e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e3811cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e3e32e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3811e160 con 0x7f7e3809f410 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e3e32e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e381323a0 con 0x7f7e38136fe0 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e3e32e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3811e8e0 con 0x7f7e3809e930 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e3811ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e37fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e3809e930 0x7f7e3811c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e3c8cb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e3811cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e377fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e3811ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43548/0 (socket says 172.21.15.89:43548) 2023-09-30T22:16:31.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.480+0000 7f7e377fe700 1 -- 172.21.15.89:0/2959785535 learned_addr learned my addr 172.21.15.89:0/2959785535 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e377fe700 1 -- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e3809e930 msgr2=0x7f7e3811c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e377fe700 1 --2- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e3809e930 0x7f7e3811c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e377fe700 1 -- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 msgr2=0x7f7e3811cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e377fe700 1 --2- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e3811cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e377fe700 1 -- 172.21.15.89:0/2959785535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0094a0 con 0x7f7e3809f410 2023-09-30T22:16:31.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e3c8cb700 1 --2- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e3811cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e377fe700 1 --2- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e3811ca90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009fd0 tx=0x7f7e2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e34ff9700 1 -- 172.21.15.89:0/2959785535 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c004550 con 0x7f7e3809f410 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e34ff9700 1 -- 172.21.15.89:0/2959785535 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c01f070 con 0x7f7e3809f410 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e38071d20 con 0x7f7e3809f410 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e34ff9700 1 -- 172.21.15.89:0/2959785535 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0038e0 con 0x7f7e3809f410 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.481+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3811f6c0 con 0x7f7e3809f410 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.482+0000 7f7e34ff9700 1 -- 172.21.15.89:0/2959785535 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7e2c003a40 con 0x7f7e3809f410 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.482+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e38074ea0 con 0x7f7e3809f410 2023-09-30T22:16:31.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.482+0000 7f7e34ff9700 1 -- 172.21.15.89:0/2959785535 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f7e2c021070 con 0x7f7e3809f410 2023-09-30T22:16:31.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.487+0000 7f7e34ff9700 1 -- 172.21.15.89:0/2959785535 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e2c01a430 con 0x7f7e3809f410 2023-09-30T22:16:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:31.682+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd down", "ids": ["3"]} v 0) v1 -- 0x7f7e381325e0 con 0x7f7e3809f410 2023-09-30T22:16:31.682 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-09-30T22:16:31.682+0000 7f2d91337700 -1 mon.a@0(leader).osd e26 definitely_dead 0 2023-09-30T22:16:31.869 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-09-30T22:16:31.869+0000 7fc28126a680 -1 Falling back to public interface 2023-09-30T22:16:32.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.260+0000 7f7e34ff9700 1 -- 172.21.15.89:0/2959785535 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["3"]}]=0 marked down osd.3. v27) v1 ==== 89+0+0 (secure 0 0 0) 0x7f7e2c010b70 con 0x7f7e3809f410 2023-09-30T22:16:32.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.262+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 >> v1:172.21.15.89:6816/126953 conn(0x7f7e2402f510 legacy=0x7f7e240319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:32.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.262+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 msgr2=0x7f7e3811ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:32.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.262+0000 7f7e3e32e700 1 --2- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e3811ca90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009fd0 tx=0x7f7e2c004750 comp rx=0 tx=0).stop 2023-09-30T22:16:32.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.262+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 shutdown_connections 2023-09-30T22:16:32.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.262+0000 7f7e3e32e700 1 --2- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e38136fe0 0x7f7e3811cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.262+0000 7f7e3e32e700 1 --2- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e3809e930 0x7f7e3811c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.262+0000 7f7e3e32e700 1 --2- 172.21.15.89:0/2959785535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e3809f410 0x7f7e3811ca90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.262+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 >> 172.21.15.89:0/2959785535 conn(0x7f7e380a4210 msgr2=0x7f7e38134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:32.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.263+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 shutdown_connections 2023-09-30T22:16:32.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.263+0000 7f7e3e32e700 1 -- 172.21.15.89:0/2959785535 wait complete. 2023-09-30T22:16:32.265 INFO:teuthology.orchestra.run.smithi089.stderr:marked down osd.3. 2023-09-30T22:16:32.275 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-30T22:16:32.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-30T22:16:32.277 INFO:tasks.ceph.osd.3:Started 2023-09-30T22:16:32.277 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:16:32.277 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.401+0000 7f97d0951700 1 Processor -- start 2023-09-30T22:16:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.401+0000 7f97d0951700 1 -- start start 2023-09-30T22:16:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97d0951700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc09e1d0 0x7f97cc135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97d0951700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97cc135e50 0x7f97cc138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97d0951700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97cc09f130 con 0x7f97cc09d6f0 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97d0951700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc09f520 con 0x7f97cc135e50 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97d0951700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc09e930 con 0x7f97cc09e1d0 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc09e1d0 0x7f97cc135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97cad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97cc135e50 0x7f97cc138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc09e1d0 0x7f97cc135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48904/0 (socket says 172.21.15.89:48904) 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97cad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97cc135e50 0x7f97cc138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43560/0 (socket says 172.21.15.89:43560) 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97c9d9b700 1 -- 172.21.15.89:0/1365010773 learned_addr learned my addr 172.21.15.89:0/1365010773 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97ca59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f97cc09d6f0 0x7f97cc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39372/0 (socket says 172.21.15.89:39372) 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97c9d9b700 1 -- 172.21.15.89:0/1365010773 >> v2:172.21.15.89:3300/0 conn(0x7f97cc135e50 msgr2=0x7f97cc138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97c9d9b700 1 --2- 172.21.15.89:0/1365010773 >> v2:172.21.15.89:3300/0 conn(0x7f97cc135e50 0x7f97cc138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97c9d9b700 1 -- 172.21.15.89:0/1365010773 >> v1:172.21.15.89:6789/0 conn(0x7f97cc09d6f0 legacy=0x7f97cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97c9d9b700 1 -- 172.21.15.89:0/1365010773 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc09b2d0 con 0x7f97cc09e1d0 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.402+0000 7f97cad9d700 1 --2- 172.21.15.89:0/1365010773 >> v2:172.21.15.89:3300/0 conn(0x7f97cc135e50 0x7f97cc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97c9d9b700 1 --2- 172.21.15.89:0/1365010773 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc09e1d0 0x7f97cc135910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97c000bbc0 tx=0x7f97c000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1041180e0220705f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97c959a700 1 -- 172.21.15.89:0/1365010773 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c000dcf0 con 0x7f97cc09e1d0 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97c959a700 1 -- 172.21.15.89:0/1365010773 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c000de50 con 0x7f97cc09e1d0 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97c959a700 1 -- 172.21.15.89:0/1365010773 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0010690 con 0x7f97cc09e1d0 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97d0951700 1 -- 172.21.15.89:0/1365010773 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc09e1d0 msgr2=0x7f97cc135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97d0951700 1 --2- 172.21.15.89:0/1365010773 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc09e1d0 0x7f97cc135910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97c000bbc0 tx=0x7f97c000bed0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97d0951700 1 -- 172.21.15.89:0/1365010773 shutdown_connections 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97d0951700 1 --2- 172.21.15.89:0/1365010773 >> v2:172.21.15.89:3300/0 conn(0x7f97cc135e50 0x7f97cc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97d0951700 1 --2- 172.21.15.89:0/1365010773 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc09e1d0 0x7f97cc135910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97d0951700 1 -- 172.21.15.89:0/1365010773 >> 172.21.15.89:0/1365010773 conn(0x7f97cc066f50 msgr2=0x7f97cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97d0951700 1 -- 172.21.15.89:0/1365010773 shutdown_connections 2023-09-30T22:16:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.403+0000 7f97d0951700 1 -- 172.21.15.89:0/1365010773 wait complete. 2023-09-30T22:16:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.404+0000 7f97d0951700 1 Processor -- start 2023-09-30T22:16:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.404+0000 7f97d0951700 1 -- start start 2023-09-30T22:16:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97d0951700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97cc09d6f0 0x7f97cc1c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97d0951700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97cc09e1d0 0x7f97cc1c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97d0951700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc135e50 0x7f97cc1c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97d0951700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc11b420 con 0x7f97cc09d6f0 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97d0951700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc11b230 con 0x7f97cc09e1d0 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97d0951700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc11d120 con 0x7f97cc135e50 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97cc09d6f0 0x7f97cc1c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97c9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97cc09e1d0 0x7f97cc1c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97cad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc135e50 0x7f97cc1c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97cc09d6f0 0x7f97cc1c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43568/0 (socket says 172.21.15.89:43568) 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 -- 172.21.15.89:0/379569981 learned_addr learned my addr 172.21.15.89:0/379569981 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 -- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc135e50 msgr2=0x7f97cc1c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 --2- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc135e50 0x7f97cc1c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 -- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97cc09e1d0 msgr2=0x7f97cc1c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 --2- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97cc09e1d0 0x7f97cc1c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 -- 172.21.15.89:0/379569981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0094a0 con 0x7f97cc09d6f0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97ca59c700 1 --2- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97cc09d6f0 0x7f97cc1c4130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97c40102e0 tx=0x7f97c40105f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97baffd700 1 -- 172.21.15.89:0/379569981 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4013070 con 0x7f97cc09d6f0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97baffd700 1 -- 172.21.15.89:0/379569981 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c400e2e0 con 0x7f97cc09d6f0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.405+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c000b7f0 con 0x7f97cc09d6f0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.406+0000 7f97baffd700 1 -- 172.21.15.89:0/379569981 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4010d90 con 0x7f97cc09d6f0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.406+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc09c700 con 0x7f97cc09d6f0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.406+0000 7f97baffd700 1 -- 172.21.15.89:0/379569981 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f97c400c450 con 0x7f97cc09d6f0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.406+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc11cbd0 con 0x7f97cc09d6f0 2023-09-30T22:16:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.406+0000 7f97baffd700 1 -- 172.21.15.89:0/379569981 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f97c4043d40 con 0x7f97cc09d6f0 2023-09-30T22:16:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.411+0000 7f97baffd700 1 -- 172.21.15.89:0/379569981 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97c400c700 con 0x7f97cc09d6f0 2023-09-30T22:16:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.602+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97cc04ea50 con 0x7f97cc09d6f0 2023-09-30T22:16:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.603+0000 7f97baffd700 1 -- 172.21.15.89:0/379569981 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+11902 (secure 0 0 0) 0x7f97c40203a0 con 0x7f97cc09d6f0 2023-09-30T22:16:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:32.603 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:32.255166+0000","last_up_change":"2023-09-30T22:16:32.255166+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":118768}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":118768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":118768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":118768}]},"public_addr":"172.21.15.89:6804/118768","cluster_addr":"172.21.15.89:6805/118768","heartbeat_back_addr":"172.21.15.89:6807/118768","heartbeat_front_addr":"172.21.15.89:6806/118768","state":["exists"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:16:32.255166+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:32.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 >> v1:172.21.15.89:6816/126953 conn(0x7f97ac02f540 legacy=0x7f97ac0319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:32.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97cc09d6f0 msgr2=0x7f97cc1c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:32.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 --2- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97cc09d6f0 0x7f97cc1c4130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97c40102e0 tx=0x7f97c40105f0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 shutdown_connections 2023-09-30T22:16:32.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 --2- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97cc135e50 0x7f97cc1c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 --2- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97cc09d6f0 0x7f97cc1c4130 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 --2- 172.21.15.89:0/379569981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97cc09e1d0 0x7f97cc1c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 >> 172.21.15.89:0/379569981 conn(0x7f97cc066f50 msgr2=0x7f97cc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 shutdown_connections 2023-09-30T22:16:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:32.605+0000 7f97d0951700 1 -- 172.21.15.89:0/379569981 wait complete. 2023-09-30T22:16:32.869 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-09-30T22:16:32.869+0000 7f09f1cc5680 -1 Falling back to public interface 2023-09-30T22:16:33.407 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-09-30T22:16:33.408+0000 7f7701c28680 -1 osd.0 23 log_to_monitors true 2023-09-30T22:16:34.468 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-09-30T22:16:34.469+0000 7fdf3a35c680 -1 osd.1 23 log_to_monitors true 2023-09-30T22:16:35.180 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-09-30T22:16:35.181+0000 7f76edba3700 -1 osd.0 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:16:35.619 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:35.691 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-09-30T22:16:35.692+0000 7fc28126a680 -1 osd.2 25 log_to_monitors true 2023-09-30T22:16:35.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.779+0000 7f39714ca700 1 Processor -- start 2023-09-30T22:16:35.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.779+0000 7f39714ca700 1 -- start start 2023-09-30T22:16:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f39714ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c0a1510 0x7f396c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f39714ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f396c139250 0x7f396c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f39714ca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396c09edd0 con 0x7f396c0a0a30 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f39714ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396c0a18e0 con 0x7f396c139250 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f39714ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f396c09e1f0 con 0x7f396c0a1510 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f396b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f396c139250 0x7f396c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f396a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c0a1510 0x7f396c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f396b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f396c139250 0x7f396c13b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43584/0 (socket says 172.21.15.89:43584) 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f396affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f396c0a0a30 0x7f396c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39422/0 (socket says 172.21.15.89:39422) 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f396b7fe700 1 -- 172.21.15.89:0/1172098847 learned_addr learned my addr 172.21.15.89:0/1172098847 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f396a7fc700 1 -- 172.21.15.89:0/1172098847 >> v2:172.21.15.89:3300/0 conn(0x7f396c139250 msgr2=0x7f396c13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f396a7fc700 1 --2- 172.21.15.89:0/1172098847 >> v2:172.21.15.89:3300/0 conn(0x7f396c139250 0x7f396c13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f396a7fc700 1 -- 172.21.15.89:0/1172098847 >> v1:172.21.15.89:6789/0 conn(0x7f396c0a0a30 legacy=0x7f396c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.781+0000 7f396a7fc700 1 -- 172.21.15.89:0/1172098847 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c09dc50 con 0x7f396c0a1510 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.780+0000 7f3969ffb700 1 -- 172.21.15.89:0/1172098847 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861405491 0 0) 0x7f396c09edd0 con 0x7f396c0a0a30 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.781+0000 7f396b7fe700 1 --2- 172.21.15.89:0/1172098847 >> v2:172.21.15.89:3300/0 conn(0x7f396c139250 0x7f396c13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.781+0000 7f396a7fc700 1 --2- 172.21.15.89:0/1172098847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c0a1510 0x7f396c138d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3960009870 tx=0x7f3960009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732f68750e29cf37 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.781+0000 7f3969ffb700 1 -- 172.21.15.89:0/1172098847 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960011830 con 0x7f396c0a1510 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.781+0000 7f3969ffb700 1 -- 172.21.15.89:0/1172098847 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3960011990 con 0x7f396c0a1510 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.781+0000 7f3969ffb700 1 -- 172.21.15.89:0/1172098847 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960011d20 con 0x7f396c0a1510 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.782+0000 7f39714ca700 1 -- 172.21.15.89:0/1172098847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c0a1510 msgr2=0x7f396c138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.782+0000 7f39714ca700 1 --2- 172.21.15.89:0/1172098847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c0a1510 0x7f396c138d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3960009870 tx=0x7f3960009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.782+0000 7f39714ca700 1 -- 172.21.15.89:0/1172098847 shutdown_connections 2023-09-30T22:16:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.782+0000 7f39714ca700 1 --2- 172.21.15.89:0/1172098847 >> v2:172.21.15.89:3300/0 conn(0x7f396c139250 0x7f396c13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.782+0000 7f39714ca700 1 --2- 172.21.15.89:0/1172098847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c0a1510 0x7f396c138d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.782+0000 7f39714ca700 1 -- 172.21.15.89:0/1172098847 >> 172.21.15.89:0/1172098847 conn(0x7f396c0a6550 msgr2=0x7f396c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.782+0000 7f39714ca700 1 -- 172.21.15.89:0/1172098847 shutdown_connections 2023-09-30T22:16:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.782+0000 7f39714ca700 1 -- 172.21.15.89:0/1172098847 wait complete. 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.783+0000 7f39714ca700 1 Processor -- start 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.783+0000 7f39714ca700 1 -- start start 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f39714ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f396c0a0a30 0x7f396c122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f39714ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f396c0a1510 0x7f396c122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f39714ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c139250 0x7f396c123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f39714ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c1389b0 con 0x7f396c0a1510 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f39714ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396c124390 con 0x7f396c0a0a30 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f39714ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f396c124c60 con 0x7f396c139250 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f396c0a0a30 0x7f396c122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:35.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f396c0a0a30 0x7f396c122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48306/0 (socket says 172.21.15.89:48306) 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 -- 172.21.15.89:0/807043121 learned_addr learned my addr 172.21.15.89:0/807043121 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396b7fe700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c139250 0x7f396c123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396a7fc700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f396c0a1510 0x7f396c122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 -- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c139250 msgr2=0x7f396c123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c139250 0x7f396c123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 -- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f396c0a1510 msgr2=0x7f396c122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f396c0a1510 0x7f396c122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 -- 172.21.15.89:0/807043121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3964009b30 con 0x7f396c0a0a30 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.784+0000 7f396affd700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f396c0a0a30 0x7f396c122780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c550 tx=0x7f395c00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:35.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.785+0000 7f39537fe700 1 -- 172.21.15.89:0/807043121 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c004c80 con 0x7f396c0a0a30 2023-09-30T22:16:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.785+0000 7f39537fe700 1 -- 172.21.15.89:0/807043121 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c004e00 con 0x7f396c0a0a30 2023-09-30T22:16:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.785+0000 7f39537fe700 1 -- 172.21.15.89:0/807043121 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c007310 con 0x7f396c0a0a30 2023-09-30T22:16:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.786+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39600094a0 con 0x7f396c0a0a30 2023-09-30T22:16:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.786+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c1386d0 con 0x7f396c0a0a30 2023-09-30T22:16:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.787+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c074ea0 con 0x7f396c0a0a30 2023-09-30T22:16:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.787+0000 7f39537fe700 1 -- 172.21.15.89:0/807043121 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f395c007470 con 0x7f396c0a0a30 2023-09-30T22:16:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.787+0000 7f39537fe700 1 -- 172.21.15.89:0/807043121 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f395c03f680 con 0x7f396c0a0a30 2023-09-30T22:16:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:35.792+0000 7f39537fe700 1 -- 172.21.15.89:0/807043121 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f395c0102c0 con 0x7f396c0a0a30 2023-09-30T22:16:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.028+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f396c1386d0 con 0x7f396c0a0a30 2023-09-30T22:16:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.030+0000 7f39537fe700 1 -- 172.21.15.89:0/807043121 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12690 (secure 0 0 0) 0x7f395c011e80 con 0x7f396c0a0a30 2023-09-30T22:16:36.029 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:36.030 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:35.266115+0000","last_up_change":"2023-09-30T22:16:35.266115+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.25,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"error_message":"pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.0), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"error_message":"pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.0), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"error_message":"pool 4 has some PGs where all OSDs are with primary_affinity 0 (at least pg 4.0), read balance score may not be reliable","score_acting":0,"score_stable":0}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":19,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":129669}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":129669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":129669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":129669}]},"public_addr":"172.21.15.89:6804/129669","cluster_addr":"172.21.15.89:6805/129669","heartbeat_back_addr":"172.21.15.89:6807/129669","heartbeat_front_addr":"172.21.15.89:6806/129669","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":118777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":118777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":118777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":118777}]},"public_addr":"172.21.15.89:6809/118777","cluster_addr":"172.21.15.89:6811/118777","heartbeat_back_addr":"172.21.15.89:6815/118777","heartbeat_front_addr":"172.21.15.89:6813/118777","state":["exists"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":118778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":118778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":118778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":118778}]},"public_addr":"172.21.15.89:6800/118778","cluster_addr":"172.21.15.89:6801/118778","heartbeat_back_addr":"172.21.15.89:6803/118778","heartbeat_front_addr":"172.21.15.89:6802/118778","state":["exists"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":118781}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":118781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":118781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":118781}]},"public_addr":"172.21.15.89:6808/118781","cluster_addr":"172.21.15.89:6810/118781","heartbeat_back_addr":"172.21.15.89:6814/118781","heartbeat_front_addr":"172.21.15.89:6812/118781","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:16:32.255166+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 >> v1:172.21.15.89:6816/126953 conn(0x7f395802f510 legacy=0x7f39580319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f396c0a0a30 msgr2=0x7f396c122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f396c0a0a30 0x7f396c122780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c550 tx=0x7f395c00c860 comp rx=0 tx=0).stop 2023-09-30T22:16:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 shutdown_connections 2023-09-30T22:16:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f396c139250 0x7f396c123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f396c0a0a30 0x7f396c122780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 --2- 172.21.15.89:0/807043121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f396c0a1510 0x7f396c122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 >> 172.21.15.89:0/807043121 conn(0x7f396c0a6550 msgr2=0x7f396c0a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 shutdown_connections 2023-09-30T22:16:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:36.032+0000 7f39714ca700 1 -- 172.21.15.89:0/807043121 wait complete. 2023-09-30T22:16:36.279 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-09-30T22:16:36.280+0000 7fdf262d7700 -1 osd.1 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:16:36.718 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-09-30T22:16:36.718+0000 7f09f1cc5680 -1 osd.3 26 log_to_monitors true 2023-09-30T22:16:37.147 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-09-30T22:16:37.147+0000 7fc26d1e5700 -1 osd.2 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:16:38.191 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-09-30T22:16:38.191+0000 7f09ddc40700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:16:39.046 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:39.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.179+0000 7f180e84d700 1 Processor -- start 2023-09-30T22:16:39.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.179+0000 7f180e84d700 1 -- start start 2023-09-30T22:16:39.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.179+0000 7f180e84d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18081337f0 0x7f1808138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.179+0000 7f180e84d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1808138dc0 0x7f180813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.179+0000 7f180e84d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180806b990 con 0x7f1808133420 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.179+0000 7f180e84d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180809b0e0 con 0x7f18081337f0 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.179+0000 7f180e84d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1808133c20 con 0x7f1808138dc0 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f180cdea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1808138dc0 0x7f180813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f18077fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18081337f0 0x7f1808138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f180cdea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1808138dc0 0x7f180813b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48318/0 (socket says 172.21.15.89:48318) 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f180cdea700 1 -- 172.21.15.89:0/3653295231 learned_addr learned my addr 172.21.15.89:0/3653295231 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:39.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f180cdea700 1 -- 172.21.15.89:0/3653295231 >> v2:172.21.15.89:3300/0 conn(0x7f18081337f0 msgr2=0x7f1808138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f180cdea700 1 --2- 172.21.15.89:0/3653295231 >> v2:172.21.15.89:3300/0 conn(0x7f18081337f0 0x7f1808138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f180cdea700 1 -- 172.21.15.89:0/3653295231 >> v1:172.21.15.89:6789/0 conn(0x7f1808133420 legacy=0x7f1808133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f180cdea700 1 -- 172.21.15.89:0/3653295231 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18080a0590 con 0x7f1808138dc0 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f18077fe700 1 --2- 172.21.15.89:0/3653295231 >> v2:172.21.15.89:3300/0 conn(0x7f18081337f0 0x7f1808138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f180cdea700 1 --2- 172.21.15.89:0/3653295231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1808138dc0 0x7f180813b1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1800009e50 tx=0x7f180000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00994ee89fe7e53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f1806ffd700 1 -- 172.21.15.89:0/3653295231 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180000e070 con 0x7f1808138dc0 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f1806ffd700 1 -- 172.21.15.89:0/3653295231 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180000b910 con 0x7f1808138dc0 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.180+0000 7f1806ffd700 1 -- 172.21.15.89:0/3653295231 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180000bcc0 con 0x7f1808138dc0 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.181+0000 7f180e84d700 1 -- 172.21.15.89:0/3653295231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1808138dc0 msgr2=0x7f180813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.181+0000 7f180e84d700 1 --2- 172.21.15.89:0/3653295231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1808138dc0 0x7f180813b1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1800009e50 tx=0x7f180000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:39.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.181+0000 7f180e84d700 1 -- 172.21.15.89:0/3653295231 shutdown_connections 2023-09-30T22:16:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.181+0000 7f180e84d700 1 --2- 172.21.15.89:0/3653295231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1808138dc0 0x7f180813b1a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.181+0000 7f180e84d700 1 --2- 172.21.15.89:0/3653295231 >> v2:172.21.15.89:3300/0 conn(0x7f18081337f0 0x7f1808138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.181+0000 7f180e84d700 1 -- 172.21.15.89:0/3653295231 >> 172.21.15.89:0/3653295231 conn(0x7f18080a4210 msgr2=0x7f18080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.181+0000 7f180e84d700 1 -- 172.21.15.89:0/3653295231 shutdown_connections 2023-09-30T22:16:39.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.181+0000 7f180e84d700 1 -- 172.21.15.89:0/3653295231 wait complete. 2023-09-30T22:16:39.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180e84d700 1 Processor -- start 2023-09-30T22:16:39.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180e84d700 1 -- start start 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180e84d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1808133420 0x7f180812f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180e84d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18081337f0 0x7f180812f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180e84d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1808138dc0 0x7f180812fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180e84d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18081304f0 con 0x7f1808138dc0 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180e84d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180809c510 con 0x7f18081337f0 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180e84d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f180809c720 con 0x7f1808133420 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f18077fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18081337f0 0x7f180812f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180cdea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1808138dc0 0x7f180812fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f18077fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18081337f0 0x7f180812f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48326/0 (socket says 172.21.15.89:48326) 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f18077fe700 1 -- 172.21.15.89:0/1678633848 learned_addr learned my addr 172.21.15.89:0/1678633848 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:39.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f180cdea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1808138dc0 0x7f180812fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43626/0 (socket says 172.21.15.89:43626) 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.182+0000 7f1807fff700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1808133420 0x7f180812f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f18077fe700 1 -- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1808133420 msgr2=0x7f180812f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f18077fe700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1808133420 0x7f180812f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f18077fe700 1 -- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1808138dc0 msgr2=0x7f180812fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f18077fe700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1808138dc0 0x7f180812fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f18077fe700 1 -- 172.21.15.89:0/1678633848 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f80094a0 con 0x7f18081337f0 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f180cdea700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1808138dc0 0x7f180812fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f18077fe700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18081337f0 0x7f180812f5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17f800ebe0 tx=0x7f17f800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f1804ff9700 1 -- 172.21.15.89:0/1678633848 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f800c850 con 0x7f18081337f0 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f1804ff9700 1 -- 172.21.15.89:0/1678633848 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f800c9b0 con 0x7f18081337f0 2023-09-30T22:16:39.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1800009b30 con 0x7f18081337f0 2023-09-30T22:16:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f1804ff9700 1 -- 172.21.15.89:0/1678633848 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f800cd40 con 0x7f18081337f0 2023-09-30T22:16:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.183+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180809e130 con 0x7f18081337f0 2023-09-30T22:16:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.184+0000 7f1804ff9700 1 -- 172.21.15.89:0/1678633848 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f17f8013470 con 0x7f18081337f0 2023-09-30T22:16:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.184+0000 7f1804ff9700 1 -- 172.21.15.89:0/1678633848 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f17f803fcd0 con 0x7f18081337f0 2023-09-30T22:16:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.184+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1808074ea0 con 0x7f18081337f0 2023-09-30T22:16:39.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.189+0000 7f1804ff9700 1 -- 172.21.15.89:0/1678633848 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17f8013720 con 0x7f18081337f0 2023-09-30T22:16:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.393+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f180806b250 con 0x7f18081337f0 2023-09-30T22:16:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.394+0000 7f1804ff9700 1 -- 172.21.15.89:0/1678633848 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+12904 (secure 0 0 0) 0x7f17f801b3b0 con 0x7f18081337f0 2023-09-30T22:16:39.394 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:39.394 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":33,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:39.309318+0000","last_up_change":"2023-09-30T22:16:38.305512+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":129669}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":129669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":129669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":129669}]},"public_addr":"172.21.15.89:6804/129669","cluster_addr":"172.21.15.89:6805/129669","heartbeat_back_addr":"172.21.15.89:6807/129669","heartbeat_front_addr":"172.21.15.89:6806/129669","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":129767}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":129767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":129767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":129767}]},"public_addr":"172.21.15.89:6800/129767","cluster_addr":"172.21.15.89:6801/129767","heartbeat_back_addr":"172.21.15.89:6803/129767","heartbeat_front_addr":"172.21.15.89:6802/129767","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":129877}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":129877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":129877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":129877}]},"public_addr":"172.21.15.89:6808/129877","cluster_addr":"172.21.15.89:6809/129877","heartbeat_back_addr":"172.21.15.89:6811/129877","heartbeat_front_addr":"172.21.15.89:6810/129877","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":130217}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":130217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":130217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":130217}]},"public_addr":"172.21.15.89:6812/130217","cluster_addr":"172.21.15.89:6813/130217","heartbeat_back_addr":"172.21.15.89:6815/130217","heartbeat_front_addr":"172.21.15.89:6814/130217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:16:32.255166+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 >> v1:172.21.15.89:6816/126953 conn(0x7f17f402f510 legacy=0x7f17f40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18081337f0 msgr2=0x7f180812f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18081337f0 0x7f180812f5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17f800ebe0 tx=0x7f17f800eef0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 shutdown_connections 2023-09-30T22:16:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1808138dc0 0x7f180812fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1808133420 0x7f180812f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 --2- 172.21.15.89:0/1678633848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18081337f0 0x7f180812f5f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 >> 172.21.15.89:0/1678633848 conn(0x7f18080a4210 msgr2=0x7f18080a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 shutdown_connections 2023-09-30T22:16:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:39.396+0000 7f180e84d700 1 -- 172.21.15.89:0/1678633848 wait complete. 2023-09-30T22:16:39.408 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:16:39.408 INFO:teuthology.run_tasks:Running task ceph.stop... 2023-09-30T22:16:39.419 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-30T22:16:39.420 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:16:39.430 INFO:tasks.ceph.mds.a:Stopped 2023-09-30T22:16:43.173 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog finished 2023-09-30T22:16:43.174 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:16:43.187 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2023-09-30T22:16:43.187 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-30T22:16:43.188 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-30T22:16:43.191 INFO:tasks.ceph.mds.a:Started 2023-09-30T22:16:43.191 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:16:43.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:43.282 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-09-30T22:16:43.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-09-30T22:16:43.297+0000 7f2d91337700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-30T22:16:43.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.337+0000 7f594fb71700 1 Processor -- start 2023-09-30T22:16:43.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.337+0000 7f594fb71700 1 -- start start 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594fb71700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f59480a0a30 0x7f59480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594fb71700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f594813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594fb71700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59480a2a70 con 0x7f594809d630 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594fb71700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59480a01d0 con 0x7f59480a0a30 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594fb71700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f594809e2b0 con 0x7f5948138d80 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594d90d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f59480a0a30 0x7f59480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594e10e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f594813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594d90d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f59480a0a30 0x7f59480a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50748/0 (socket says 172.21.15.89:50748) 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594d10c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f594809d630 0x7f59480a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53198/0 (socket says 172.21.15.89:53198) 2023-09-30T22:16:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594d90d700 1 -- 172.21.15.89:0/390774857 learned_addr learned my addr 172.21.15.89:0/390774857 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594e10e700 1 -- 172.21.15.89:0/390774857 >> v2:172.21.15.89:3300/0 conn(0x7f59480a0a30 msgr2=0x7f59480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594e10e700 1 --2- 172.21.15.89:0/390774857 >> v2:172.21.15.89:3300/0 conn(0x7f59480a0a30 0x7f59480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594e10e700 1 -- 172.21.15.89:0/390774857 >> v1:172.21.15.89:6789/0 conn(0x7f594809d630 legacy=0x7f59480a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594e10e700 1 -- 172.21.15.89:0/390774857 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594809dc50 con 0x7f5948138d80 2023-09-30T22:16:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594c90b700 1 -- 172.21.15.89:0/390774857 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918139471 0 0) 0x7f59480a2a70 con 0x7f594809d630 2023-09-30T22:16:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594d90d700 1 --2- 172.21.15.89:0/390774857 >> v2:172.21.15.89:3300/0 conn(0x7f59480a0a30 0x7f59480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594e10e700 1 --2- 172.21.15.89:0/390774857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f594813b160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5938009d90 tx=0x7f593800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d90f7f17b037aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594c90b700 1 -- 172.21.15.89:0/390774857 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800e070 con 0x7f5948138d80 2023-09-30T22:16:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594c90b700 1 -- 172.21.15.89:0/390774857 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593800bb20 con 0x7f5948138d80 2023-09-30T22:16:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.338+0000 7f594c90b700 1 -- 172.21.15.89:0/390774857 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938012430 con 0x7f5948138d80 2023-09-30T22:16:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.339+0000 7f594fb71700 1 -- 172.21.15.89:0/390774857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 msgr2=0x7f594813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.339+0000 7f594fb71700 1 --2- 172.21.15.89:0/390774857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f594813b160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5938009d90 tx=0x7f593800b330 comp rx=0 tx=0).stop 2023-09-30T22:16:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.339+0000 7f594fb71700 1 -- 172.21.15.89:0/390774857 shutdown_connections 2023-09-30T22:16:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.339+0000 7f594fb71700 1 --2- 172.21.15.89:0/390774857 >> v2:172.21.15.89:3300/0 conn(0x7f59480a0a30 0x7f59480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.339+0000 7f594fb71700 1 --2- 172.21.15.89:0/390774857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f594813b160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.339+0000 7f594fb71700 1 -- 172.21.15.89:0/390774857 >> 172.21.15.89:0/390774857 conn(0x7f59480a3050 msgr2=0x7f59480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.339+0000 7f594fb71700 1 -- 172.21.15.89:0/390774857 shutdown_connections 2023-09-30T22:16:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.339+0000 7f594fb71700 1 -- 172.21.15.89:0/390774857 wait complete. 2023-09-30T22:16:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594fb71700 1 Processor -- start 2023-09-30T22:16:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594fb71700 1 -- start start 2023-09-30T22:16:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594fb71700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f594809d630 0x7f5948133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594fb71700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59480a0a30 0x7f5948133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594fb71700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f59481340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594fb71700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5948137170 con 0x7f59480a0a30 2023-09-30T22:16:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594fb71700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5948136060 con 0x7f594809d630 2023-09-30T22:16:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594fb71700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5948136520 con 0x7f5948138d80 2023-09-30T22:16:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594e10e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f59481340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594d10c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59480a0a30 0x7f5948133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594d90d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f594809d630 0x7f5948133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594e10e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f59481340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40166/0 (socket says 172.21.15.89:40166) 2023-09-30T22:16:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594d10c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59480a0a30 0x7f5948133b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50762/0 (socket says 172.21.15.89:50762) 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.340+0000 7f594e10e700 1 -- 172.21.15.89:0/4093820769 learned_addr learned my addr 172.21.15.89:0/4093820769 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594d10c700 1 -- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 msgr2=0x7f59481340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594d10c700 1 --2- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f59481340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594d10c700 1 -- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f594809d630 msgr2=0x7f5948133650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594d10c700 1 --2- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f594809d630 0x7f5948133650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594d10c700 1 -- 172.21.15.89:0/4093820769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940009550 con 0x7f59480a0a30 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594e10e700 1 --2- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f59481340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594d10c700 1 --2- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59480a0a30 0x7f5948133b90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c470 tx=0x7f593c00c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f59467fc700 1 -- 172.21.15.89:0/4093820769 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c004c80 con 0x7f59480a0a30 2023-09-30T22:16:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f59467fc700 1 -- 172.21.15.89:0/4093820769 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c004de0 con 0x7f59480a0a30 2023-09-30T22:16:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5938009a70 con 0x7f59480a0a30 2023-09-30T22:16:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f59467fc700 1 -- 172.21.15.89:0/4093820769 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c007310 con 0x7f59480a0a30 2023-09-30T22:16:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.341+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5948136bd0 con 0x7f59480a0a30 2023-09-30T22:16:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.342+0000 7f59467fc700 1 -- 172.21.15.89:0/4093820769 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f593c007470 con 0x7f59480a0a30 2023-09-30T22:16:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.342+0000 7f59467fc700 1 -- 172.21.15.89:0/4093820769 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f593c03f6e0 con 0x7f59480a0a30 2023-09-30T22:16:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.342+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59480747f0 con 0x7f59480a0a30 2023-09-30T22:16:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.347+0000 7f59467fc700 1 -- 172.21.15.89:0/4093820769 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f593c007720 con 0x7f59480a0a30 2023-09-30T22:16:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.534+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5948072980 con 0x7f59480a0a30 2023-09-30T22:16:43.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.535+0000 7f59467fc700 1 -- 172.21.15.89:0/4093820769 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+12969 (secure 0 0 0) 0x7f593c011360 con 0x7f59480a0a30 2023-09-30T22:16:43.536 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:43.536 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":34,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:43.298064+0000","last_up_change":"2023-09-30T22:16:38.305512+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":129669}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":129669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":129669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":129669}]},"public_addr":"172.21.15.89:6804/129669","cluster_addr":"172.21.15.89:6805/129669","heartbeat_back_addr":"172.21.15.89:6807/129669","heartbeat_front_addr":"172.21.15.89:6806/129669","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":129767}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":129767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":129767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":129767}]},"public_addr":"172.21.15.89:6800/129767","cluster_addr":"172.21.15.89:6801/129767","heartbeat_back_addr":"172.21.15.89:6803/129767","heartbeat_front_addr":"172.21.15.89:6802/129767","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":129877}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":129877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":129877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":129877}]},"public_addr":"172.21.15.89:6808/129877","cluster_addr":"172.21.15.89:6809/129877","heartbeat_back_addr":"172.21.15.89:6811/129877","heartbeat_front_addr":"172.21.15.89:6810/129877","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":130217}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":130217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":130217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":130217}]},"public_addr":"172.21.15.89:6812/130217","cluster_addr":"172.21.15.89:6813/130217","heartbeat_back_addr":"172.21.15.89:6815/130217","heartbeat_front_addr":"172.21.15.89:6814/130217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:16:32.255166+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6817/3539463463":"2023-10-01T22:16:43.298028+0000","172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 >> v1:172.21.15.89:6816/126953 conn(0x7f593402f510 legacy=0x7f59340319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59480a0a30 msgr2=0x7f5948133b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 --2- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59480a0a30 0x7f5948133b90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c470 tx=0x7f593c00c780 comp rx=0 tx=0).stop 2023-09-30T22:16:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 shutdown_connections 2023-09-30T22:16:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 --2- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5948138d80 0x7f59481340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 --2- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f594809d630 0x7f5948133650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 --2- 172.21.15.89:0/4093820769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59480a0a30 0x7f5948133b90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 >> 172.21.15.89:0/4093820769 conn(0x7f59480a3050 msgr2=0x7f594809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 shutdown_connections 2023-09-30T22:16:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.537+0000 7f594fb71700 1 -- 172.21.15.89:0/4093820769 wait complete. 2023-09-30T22:16:43.550 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:16:43.550 INFO:teuthology.run_tasks:Running task exec... 2023-09-30T22:16:43.564 INFO:teuthology.task.exec:Executing custom commands... 2023-09-30T22:16:43.564 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:16:43.567 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd dump -f json-pretty' 2023-09-30T22:16:43.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.684+0000 7f1587807700 1 Processor -- start 2023-09-30T22:16:43.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.684+0000 7f1587807700 1 -- start start 2023-09-30T22:16:43.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1587807700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1580133b00 0x7f1580133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1587807700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158009e060 0x7f158013ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1587807700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158009f640 con 0x7f1580134410 2023-09-30T22:16:43.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1587807700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158009a790 con 0x7f1580133b00 2023-09-30T22:16:43.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1587807700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158009a4f0 con 0x7f158009e060 2023-09-30T22:16:43.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f15855a3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1580133b00 0x7f1580133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1585da4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158009e060 0x7f158013ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1584da2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1580134410 0x7f158009d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53208/0 (socket says 172.21.15.89:53208) 2023-09-30T22:16:43.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1584da2700 1 -- 172.21.15.89:0/2922249419 learned_addr learned my addr 172.21.15.89:0/2922249419 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:43.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1585da4700 1 -- 172.21.15.89:0/2922249419 >> v2:172.21.15.89:3300/0 conn(0x7f1580133b00 msgr2=0x7f1580133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:43.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1585da4700 1 --2- 172.21.15.89:0/2922249419 >> v2:172.21.15.89:3300/0 conn(0x7f1580133b00 0x7f1580133ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1585da4700 1 -- 172.21.15.89:0/2922249419 >> v1:172.21.15.89:6789/0 conn(0x7f1580134410 legacy=0x7f158009d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1585da4700 1 -- 172.21.15.89:0/2922249419 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158009eb20 con 0x7f158009e060 2023-09-30T22:16:43.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f15855a3700 1 --2- 172.21.15.89:0/2922249419 >> v2:172.21.15.89:3300/0 conn(0x7f1580133b00 0x7f1580133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:43.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1585da4700 1 --2- 172.21.15.89:0/2922249419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158009e060 0x7f158013ba80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1578009d90 tx=0x7f157800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12686d2b420d259c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:43.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1577fff700 1 -- 172.21.15.89:0/2922249419 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157800e070 con 0x7f158009e060 2023-09-30T22:16:43.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1577fff700 1 -- 172.21.15.89:0/2922249419 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157800bb20 con 0x7f158009e060 2023-09-30T22:16:43.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.685+0000 7f1577fff700 1 -- 172.21.15.89:0/2922249419 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578012430 con 0x7f158009e060 2023-09-30T22:16:43.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.686+0000 7f1587807700 1 -- 172.21.15.89:0/2922249419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158009e060 msgr2=0x7f158013ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.686+0000 7f1587807700 1 --2- 172.21.15.89:0/2922249419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158009e060 0x7f158013ba80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1578009d90 tx=0x7f157800b330 comp rx=0 tx=0).stop 2023-09-30T22:16:43.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.686+0000 7f1587807700 1 -- 172.21.15.89:0/2922249419 shutdown_connections 2023-09-30T22:16:43.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.686+0000 7f1587807700 1 --2- 172.21.15.89:0/2922249419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158009e060 0x7f158013ba80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.686+0000 7f1587807700 1 --2- 172.21.15.89:0/2922249419 >> v2:172.21.15.89:3300/0 conn(0x7f1580133b00 0x7f1580133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.686+0000 7f1587807700 1 -- 172.21.15.89:0/2922249419 >> 172.21.15.89:0/2922249419 conn(0x7f15800a2f50 msgr2=0x7f15800a3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.686+0000 7f1587807700 1 -- 172.21.15.89:0/2922249419 shutdown_connections 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.686+0000 7f1587807700 1 -- 172.21.15.89:0/2922249419 wait complete. 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.687+0000 7f1587807700 1 Processor -- start 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.687+0000 7f1587807700 1 -- start start 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1587807700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158009e060 0x7f158011dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1587807700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1580133b00 0x7f158011e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1587807700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1580134410 0x7f158011e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:43.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1587807700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1580122240 con 0x7f1580133b00 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1587807700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158011f970 con 0x7f1580134410 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1587807700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f158011ff90 con 0x7f158009e060 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1584da2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1580133b00 0x7f158011e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f15855a3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158009e060 0x7f158011dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1584da2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1580133b00 0x7f158011e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50776/0 (socket says 172.21.15.89:50776) 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f15855a3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158009e060 0x7f158011dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40178/0 (socket says 172.21.15.89:40178) 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1584da2700 1 -- 172.21.15.89:0/2671709580 learned_addr learned my addr 172.21.15.89:0/2671709580 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1585da4700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1580134410 0x7f158011e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1584da2700 1 -- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158009e060 msgr2=0x7f158011dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1584da2700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158009e060 0x7f158011dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1584da2700 1 -- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1580134410 msgr2=0x7f158011e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1584da2700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1580134410 0x7f158011e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1584da2700 1 -- 172.21.15.89:0/2671709580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15700094a0 con 0x7f1580133b00 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f15855a3700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158009e060 0x7f158011dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.688+0000 7f1585da4700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1580134410 0x7f158011e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.689+0000 7f1584da2700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1580133b00 0x7f158011e2a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f157c00c730 tx=0x7f157c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.689+0000 7f1575ffb700 1 -- 172.21.15.89:0/2671709580 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0074e0 con 0x7f1580133b00 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.689+0000 7f1575ffb700 1 -- 172.21.15.89:0/2671709580 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c007640 con 0x7f1580133b00 2023-09-30T22:16:43.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.689+0000 7f1575ffb700 1 -- 172.21.15.89:0/2671709580 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c005a30 con 0x7f1580133b00 2023-09-30T22:16:43.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.689+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1578009a70 con 0x7f1580133b00 2023-09-30T22:16:43.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.689+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158009bfa0 con 0x7f1580133b00 2023-09-30T22:16:43.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.689+0000 7f1575ffb700 1 -- 172.21.15.89:0/2671709580 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f157c005b90 con 0x7f1580133b00 2023-09-30T22:16:43.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.690+0000 7f1575ffb700 1 -- 172.21.15.89:0/2671709580 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f157c03eed0 con 0x7f1580133b00 2023-09-30T22:16:43.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.690+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158009ccc0 con 0x7f1580133b00 2023-09-30T22:16:43.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.695+0000 7f1575ffb700 1 -- 172.21.15.89:0/2671709580 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f157c017020 con 0x7f1580133b00 2023-09-30T22:16:43.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.878+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f158004f2a0 con 0x7f1580133b00 2023-09-30T22:16:43.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.879+0000 7f1575ffb700 1 -- 172.21.15.89:0/2671709580 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v34) v1 ==== 81+0+24159 (secure 0 0 0) 0x7f157c024070 con 0x7f1580133b00 2023-09-30T22:16:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:43.881 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-30T22:16:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 34, 2023-09-30T22:16:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "389f6e95-ea58-47bd-a145-778ba0724a9e", 2023-09-30T22:16:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2023-09-30T22:13:45.747992+0000", 2023-09-30T22:16:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2023-09-30T22:16:43.298064+0000", 2023-09-30T22:16:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "last_up_change": "2023-09-30T22:16:38.305512+0000", 2023-09-30T22:16:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "last_in_change": "2023-09-30T22:13:47.831690+0000", 2023-09-30T22:16:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-09-30T22:16:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "flags_num": 5799936, 2023-09-30T22:16:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "flags_set": [ 2023-09-30T22:16:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "pglog_hardlimit", 2023-09-30T22:16:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "purged_snapdirs", 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_deletes", 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "sortbitwise" 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_version": 8, 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "full_ratio": 0.94999998807907104, 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "nearfull_ratio": 0.85000002384185791, 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_snapshot": "", 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_max": 4, 2023-09-30T22:16:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd": 4, 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "require_min_compat_client": "pacific", 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "min_compat_client": "luminous", 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "require_osd_release": "pacific", 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_crimson": false, 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "pools": [ 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "pool": 1, 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": ".mgr", 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "create_time": "2023-09-30T22:13:54.564869+0000", 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "flags_names": "hashpspool", 2023-09-30T22:16:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "type": 1, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 2, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": 1, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_rule": 0, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "object_hash": 2, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": 1, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_placement_num": 1, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_placement_num_target": 1, 2023-09-30T22:16:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_target": 1, 2023-09-30T22:16:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_pending": 1, 2023-09-30T22:16:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "last_pg_merge_meta": { 2023-09-30T22:16:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "source_pgid": "0.0", 2023-09-30T22:16:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "ready_epoch": 0, 2023-09-30T22:16:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "last_epoch_started": 0, 2023-09-30T22:16:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "last_epoch_clean": 0, 2023-09-30T22:16:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "source_version": "0'0", 2023-09-30T22:16:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "target_version": "0'0" 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "last_change": "23", 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend": "0", 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "auid": 0, 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_seq": 0, 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_epoch": 0, 2023-09-30T22:16:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_snaps": [], 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_snaps": "[]", 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "quota_max_bytes": 0, 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "quota_max_objects": 0, 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "tiers": [], 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "tier_of": -1, 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "read_tier": -1, 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "write_tier": -1, 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_mode": "none", 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "target_max_bytes": 0, 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "target_max_objects": 0, 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:16:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_min_flush_age": 0, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_min_evict_age": 0, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "erasure_code_profile": "", 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_params": { 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "none" 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_period": 0, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_count": 0, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "use_gmt_hitset": true, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:16:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "grade_table": [], 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "stripe_width": 0, 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "expected_num_objects": 0, 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "fast_read": false, 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "options": { 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_max": 32, 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_min": 1 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "application_metadata": { 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": {}, 2023-09-30T22:16:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr_devicehealth": {} 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "read_balance": { 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "score_acting": 4, 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "score_stable": 4, 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "optimal_score": 0.5, 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "raw_score_acting": 2, 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "raw_score_stable": 2, 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "average_primary_affinity": 1, 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "pool": 2, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": "rbd", 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "create_time": "2023-09-30T22:13:55.740519+0000", 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "flags_names": "hashpspool", 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type": 1, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 2, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": 1, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_rule": 0, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:16:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "object_hash": 2, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": 8, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_placement_num": 8, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_placement_num_target": 8, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_target": 8, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_pending": 8, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "last_pg_merge_meta": { 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "source_pgid": "0.0", 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "ready_epoch": 0, 2023-09-30T22:16:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "last_epoch_started": 0, 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "last_epoch_clean": 0, 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "source_version": "0'0", 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "target_version": "0'0" 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "last_change": "13", 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend": "0", 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "auid": 0, 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_seq": 0, 2023-09-30T22:16:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_epoch": 0, 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_snaps": [], 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_snaps": "[]", 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "quota_max_bytes": 0, 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "quota_max_objects": 0, 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "tiers": [], 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "tier_of": -1, 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "read_tier": -1, 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "write_tier": -1, 2023-09-30T22:16:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_mode": "none", 2023-09-30T22:16:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "target_max_bytes": 0, 2023-09-30T22:16:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "target_max_objects": 0, 2023-09-30T22:16:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:16:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:16:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:16:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_min_flush_age": 0, 2023-09-30T22:16:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_min_evict_age": 0, 2023-09-30T22:16:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "erasure_code_profile": "", 2023-09-30T22:16:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_params": { 2023-09-30T22:16:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "none" 2023-09-30T22:16:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_period": 0, 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_count": 0, 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "use_gmt_hitset": true, 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "grade_table": [], 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "stripe_width": 0, 2023-09-30T22:16:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "expected_num_objects": 0, 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "fast_read": false, 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "options": {}, 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "application_metadata": { 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd": {} 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "read_balance": { 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "score_acting": 2.5, 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "score_stable": 2.5, 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "optimal_score": 1, 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "raw_score_acting": 2.5, 2023-09-30T22:16:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "raw_score_stable": 2.5, 2023-09-30T22:16:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:16:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "average_primary_affinity": 1, 2023-09-30T22:16:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:16:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "pool": 3, 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": "cephfs_metadata", 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "create_time": "2023-09-30T22:14:00.533280+0000", 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "flags_names": "hashpspool", 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type": 1, 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 2, 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": 1, 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_rule": 0, 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:16:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "object_hash": 2, 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": 32, 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_placement_num": 32, 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_placement_num_target": 32, 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_target": 32, 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_pending": 32, 2023-09-30T22:16:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "last_pg_merge_meta": { 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "source_pgid": "0.0", 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "ready_epoch": 0, 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "last_epoch_started": 0, 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "last_epoch_clean": 0, 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "source_version": "0'0", 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "target_version": "0'0" 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "last_change": "16", 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend": "0", 2023-09-30T22:16:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "auid": 0, 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_seq": 0, 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_epoch": 0, 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_snaps": [], 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_snaps": "[]", 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "quota_max_bytes": 0, 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "quota_max_objects": 0, 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "tiers": [], 2023-09-30T22:16:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "tier_of": -1, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "read_tier": -1, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "write_tier": -1, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_mode": "none", 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "target_max_bytes": 0, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "target_max_objects": 0, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_min_flush_age": 0, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_min_evict_age": 0, 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "erasure_code_profile": "", 2023-09-30T22:16:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_params": { 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "none" 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_period": 0, 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_count": 0, 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "use_gmt_hitset": true, 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:16:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:16:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "grade_table": [], 2023-09-30T22:16:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "stripe_width": 0, 2023-09-30T22:16:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "expected_num_objects": 0, 2023-09-30T22:16:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "fast_read": false, 2023-09-30T22:16:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "options": { 2023-09-30T22:16:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscale_bias": 4, 2023-09-30T22:16:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_min": 16, 2023-09-30T22:16:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_priority": 5 2023-09-30T22:16:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "application_metadata": { 2023-09-30T22:16:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "cephfs": { 2023-09-30T22:16:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": "cephfs" 2023-09-30T22:16:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "read_balance": { 2023-09-30T22:16:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "score_acting": 1.6299999952316284, 2023-09-30T22:16:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "score_stable": 1.6299999952316284, 2023-09-30T22:16:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "optimal_score": 1, 2023-09-30T22:16:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "raw_score_acting": 1.6299999952316284, 2023-09-30T22:16:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "raw_score_stable": 1.6299999952316284, 2023-09-30T22:16:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:16:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "average_primary_affinity": 1, 2023-09-30T22:16:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:16:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "pool": 4, 2023-09-30T22:16:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": "cephfs_data", 2023-09-30T22:16:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "create_time": "2023-09-30T22:14:01.011104+0000", 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "flags_names": "hashpspool", 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "type": 1, 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 2, 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": 1, 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_rule": 0, 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:16:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "object_hash": 2, 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": 64, 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_placement_num": 64, 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_placement_num_target": 64, 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_target": 64, 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_pending": 64, 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "last_pg_merge_meta": { 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "source_pgid": "0.0", 2023-09-30T22:16:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "ready_epoch": 0, 2023-09-30T22:16:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "last_epoch_started": 0, 2023-09-30T22:16:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "last_epoch_clean": 0, 2023-09-30T22:16:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "source_version": "0'0", 2023-09-30T22:16:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "target_version": "0'0" 2023-09-30T22:16:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "last_change": "17", 2023-09-30T22:16:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend": "0", 2023-09-30T22:16:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "auid": 0, 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_seq": 0, 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_epoch": 0, 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_snaps": [], 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_snaps": "[]", 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "quota_max_bytes": 0, 2023-09-30T22:16:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "quota_max_objects": 0, 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "tiers": [], 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "tier_of": -1, 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "read_tier": -1, 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "write_tier": -1, 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_mode": "none", 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "target_max_bytes": 0, 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "target_max_objects": 0, 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:16:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_min_flush_age": 0, 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "cache_min_evict_age": 0, 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "erasure_code_profile": "", 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_params": { 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "none" 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_period": 0, 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_count": 0, 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "use_gmt_hitset": true, 2023-09-30T22:16:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "grade_table": [], 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "stripe_width": 0, 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "expected_num_objects": 0, 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "fast_read": false, 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "options": { 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num_min": 64, 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "target_size_ratio": 0.89999997615814209 2023-09-30T22:16:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "application_metadata": { 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "cephfs": { 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "data": "cephfs" 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "read_balance": { 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "score_acting": 1.059999942779541, 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "score_stable": 1.059999942779541, 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "optimal_score": 1, 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "raw_score_acting": 1.059999942779541, 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "raw_score_stable": 1.059999942779541, 2023-09-30T22:16:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:16:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "average_primary_affinity": 1, 2023-09-30T22:16:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:16:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-09-30T22:16:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "osds": [ 2023-09-30T22:16:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": 0, 2023-09-30T22:16:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "uuid": "96fbca95-1fa5-4074-9fbb-d4f206bb1aa6", 2023-09-30T22:16:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "up": 1, 2023-09-30T22:16:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "in": 1, 2023-09-30T22:16:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 1, 2023-09-30T22:16:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_affinity": 1, 2023-09-30T22:16:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "last_clean_begin": 9, 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "last_clean_end": 23, 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "up_from": 29, 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "up_thru": 32, 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "down_at": 24, 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "lost_at": 0, 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6804", 2023-09-30T22:16:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129669 2023-09-30T22:16:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_addrs": { 2023-09-30T22:16:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6805", 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129669 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_back_addrs": { 2023-09-30T22:16:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6807", 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129669 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_front_addrs": { 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6806", 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129669 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6804/129669", 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_addr": "172.21.15.89:6805/129669", 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_back_addr": "172.21.15.89:6807/129669", 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_front_addr": "172.21.15.89:6806/129669", 2023-09-30T22:16:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "state": [ 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "exists", 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "up" 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": 1, 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "uuid": "bca437d5-1c37-4cda-a2f2-86f7957e0ac0", 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "up": 1, 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "in": 1, 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 1, 2023-09-30T22:16:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_affinity": 1, 2023-09-30T22:16:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "last_clean_begin": 9, 2023-09-30T22:16:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "last_clean_end": 23, 2023-09-30T22:16:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "up_from": 31, 2023-09-30T22:16:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "up_thru": 32, 2023-09-30T22:16:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "down_at": 25, 2023-09-30T22:16:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "lost_at": 0, 2023-09-30T22:16:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2023-09-30T22:16:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6800", 2023-09-30T22:16:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129767 2023-09-30T22:16:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_addrs": { 2023-09-30T22:16:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6801", 2023-09-30T22:16:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129767 2023-09-30T22:16:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_back_addrs": { 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6803", 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129767 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_front_addrs": { 2023-09-30T22:16:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6802", 2023-09-30T22:16:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129767 2023-09-30T22:16:43.934 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.934 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.934 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6800/129767", 2023-09-30T22:16:43.934 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_addr": "172.21.15.89:6801/129767", 2023-09-30T22:16:43.934 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_back_addr": "172.21.15.89:6803/129767", 2023-09-30T22:16:43.934 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_front_addr": "172.21.15.89:6802/129767", 2023-09-30T22:16:43.935 INFO:teuthology.orchestra.run.smithi089.stdout: "state": [ 2023-09-30T22:16:43.935 INFO:teuthology.orchestra.run.smithi089.stdout: "exists", 2023-09-30T22:16:43.935 INFO:teuthology.orchestra.run.smithi089.stdout: "up" 2023-09-30T22:16:43.935 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.935 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": 2, 2023-09-30T22:16:43.935 INFO:teuthology.orchestra.run.smithi089.stdout: "uuid": "0fb339c9-0000-43f3-bcbc-9cc5b8b65359", 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "up": 1, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "in": 1, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 1, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_affinity": 1, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "last_clean_begin": 9, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "last_clean_end": 25, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "up_from": 31, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "up_thru": 32, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "down_at": 26, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "lost_at": 0, 2023-09-30T22:16:43.936 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6808", 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129877 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_addrs": { 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6809", 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129877 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_back_addrs": { 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.938 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6811", 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129877 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_front_addrs": { 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6810", 2023-09-30T22:16:43.939 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 129877 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6808/129877", 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_addr": "172.21.15.89:6809/129877", 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_back_addr": "172.21.15.89:6811/129877", 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_front_addr": "172.21.15.89:6810/129877", 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: "state": [ 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: "exists", 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: "up" 2023-09-30T22:16:43.940 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": 3, 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: "uuid": "4010d897-d6fa-4f8c-ab79-7415116d796d", 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: "up": 1, 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: "in": 1, 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 1, 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_affinity": 1, 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: "last_clean_begin": 9, 2023-09-30T22:16:43.941 INFO:teuthology.orchestra.run.smithi089.stdout: "last_clean_end": 26, 2023-09-30T22:16:43.942 INFO:teuthology.orchestra.run.smithi089.stdout: "up_from": 32, 2023-09-30T22:16:43.942 INFO:teuthology.orchestra.run.smithi089.stdout: "up_thru": 32, 2023-09-30T22:16:43.942 INFO:teuthology.orchestra.run.smithi089.stdout: "down_at": 27, 2023-09-30T22:16:43.942 INFO:teuthology.orchestra.run.smithi089.stdout: "lost_at": 0, 2023-09-30T22:16:43.942 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2023-09-30T22:16:43.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6812", 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 130217 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_addrs": { 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6813", 2023-09-30T22:16:43.944 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 130217 2023-09-30T22:16:43.944 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.944 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.944 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_back_addrs": { 2023-09-30T22:16:43.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6815", 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 130217 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_front_addrs": { 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-09-30T22:16:43.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6814", 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 130217 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6812/130217", 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_addr": "172.21.15.89:6813/130217", 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_back_addr": "172.21.15.89:6815/130217", 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: "heartbeat_front_addr": "172.21.15.89:6814/130217", 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: "state": [ 2023-09-30T22:16:43.946 INFO:teuthology.orchestra.run.smithi089.stdout: "exists", 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: "up" 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_xinfo": [ 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": 0, 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: "down_stamp": "0.000000", 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: "laggy_probability": 0, 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: "laggy_interval": 0, 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: "features": 4540138322906710015, 2023-09-30T22:16:43.947 INFO:teuthology.orchestra.run.smithi089.stdout: "old_weight": 0, 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: "last_purged_snaps_scrub": "2023-09-30T22:13:52.964504+0000", 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: "dead_epoch": 24 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": 1, 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: "down_stamp": "0.000000", 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: "laggy_probability": 0, 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: "laggy_interval": 0, 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: "features": 4540138322906710015, 2023-09-30T22:16:43.948 INFO:teuthology.orchestra.run.smithi089.stdout: "old_weight": 0, 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "last_purged_snaps_scrub": "2023-09-30T22:13:53.039111+0000", 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "dead_epoch": 25 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": 2, 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "down_stamp": "0.000000", 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "laggy_probability": 0, 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "laggy_interval": 0, 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "features": 4540138322906710015, 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "old_weight": 0, 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "last_purged_snaps_scrub": "2023-09-30T22:13:52.738991+0000", 2023-09-30T22:16:43.949 INFO:teuthology.orchestra.run.smithi089.stdout: "dead_epoch": 26 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": 3, 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: "down_stamp": "2023-09-30T22:16:32.255166+0000", 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: "laggy_probability": 0, 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: "laggy_interval": 0, 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: "features": 4540138322906710015, 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: "old_weight": 0, 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: "last_purged_snaps_scrub": "2023-09-30T22:13:52.958026+0000", 2023-09-30T22:16:43.950 INFO:teuthology.orchestra.run.smithi089.stdout: "dead_epoch": 0 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_upmap": [], 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_upmap_items": [ 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: "pgid": "4.8", 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: "mappings": [ 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: "from": 3, 2023-09-30T22:16:43.951 INFO:teuthology.orchestra.run.smithi089.stdout: "to": 2 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: "pgid": "4.1f", 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: "mappings": [ 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: "from": 3, 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: "to": 2 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.952 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.953 INFO:teuthology.orchestra.run.smithi089.stdout: "pgid": "4.25", 2023-09-30T22:16:43.953 INFO:teuthology.orchestra.run.smithi089.stdout: "mappings": [ 2023-09-30T22:16:43.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.953 INFO:teuthology.orchestra.run.smithi089.stdout: "from": 3, 2023-09-30T22:16:43.953 INFO:teuthology.orchestra.run.smithi089.stdout: "to": 2 2023-09-30T22:16:43.954 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.954 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.954 INFO:teuthology.orchestra.run.smithi089.stdout: "pgid": "4.27", 2023-09-30T22:16:43.954 INFO:teuthology.orchestra.run.smithi089.stdout: "mappings": [ 2023-09-30T22:16:43.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.954 INFO:teuthology.orchestra.run.smithi089.stdout: "from": 1, 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: "to": 2 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: "pgid": "4.28", 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: "mappings": [ 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: "from": 3, 2023-09-30T22:16:43.955 INFO:teuthology.orchestra.run.smithi089.stdout: "to": 0 2023-09-30T22:16:43.956 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.956 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.956 INFO:teuthology.orchestra.run.smithi089.stdout: "pgid": "4.2f", 2023-09-30T22:16:43.956 INFO:teuthology.orchestra.run.smithi089.stdout: "mappings": [ 2023-09-30T22:16:43.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: "from": 3, 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: "to": 2 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: "pgid": "4.34", 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: "mappings": [ 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-09-30T22:16:43.957 INFO:teuthology.orchestra.run.smithi089.stdout: "from": 3, 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: "to": 0 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_upmap_primaries": [], 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_temp": [], 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: "primary_temp": [], 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: "blocklist": { 2023-09-30T22:16:43.958 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:6817/3539463463": "2023-10-01T22:16:43.298028+0000", 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:6816/118161": "2023-10-01T22:16:19.404121+0000", 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:0/2579157783": "2023-10-01T22:16:19.404121+0000", 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "172.21.15.89:0/1269574253": "2023-10-01T22:16:19.404121+0000" 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "range_blocklist": {}, 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "erasure_code_profiles": { 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "default": { 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-failure-domain": "osd", 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "k": "2", 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "m": "1", 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "plugin": "jerasure", 2023-09-30T22:16:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: "technique": "reed_sol_van" 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_snaps_queue": [], 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "new_removed_snaps": [], 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "new_purged_snaps": [], 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_node_flags": {}, 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "device_class_flags": {}, 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": { 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode_enabled": false, 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_bucket_count": 0, 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "degraded_stretch_mode": 0, 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "recovering_stretch_mode": 0, 2023-09-30T22:16:43.960 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode_bucket": 0 2023-09-30T22:16:43.961 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:43.961 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-30T22:16:43.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.881+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 >> v1:172.21.15.89:6816/126953 conn(0x7f156402f510 legacy=0x7f15640319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.881+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1580133b00 msgr2=0x7f158011e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:43.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.881+0000 7f1587807700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1580133b00 0x7f158011e2a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f157c00c730 tx=0x7f157c00ca40 comp rx=0 tx=0).stop 2023-09-30T22:16:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.881+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 shutdown_connections 2023-09-30T22:16:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.881+0000 7f1587807700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1580134410 0x7f158011e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.881+0000 7f1587807700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158009e060 0x7f158011dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.881+0000 7f1587807700 1 --2- 172.21.15.89:0/2671709580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1580133b00 0x7f158011e2a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.881+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 >> 172.21.15.89:0/2671709580 conn(0x7f15800a2f50 msgr2=0x7f1580132c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.882+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 shutdown_connections 2023-09-30T22:16:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:43.882+0000 7f1587807700 1 -- 172.21.15.89:0/2671709580 wait complete. 2023-09-30T22:16:43.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph versions' 2023-09-30T22:16:44.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.079+0000 7ffa82284700 1 Processor -- start 2023-09-30T22:16:44.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.079+0000 7ffa82284700 1 -- start start 2023-09-30T22:16:44.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa82284700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffa7c09e9d0 0x7ffa7c09eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa82284700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09f2e0 0x7ffa7c09f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa82284700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7c09ab50 con 0x7ffa7c1317b0 2023-09-30T22:16:44.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa82284700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c0a0140 con 0x7ffa7c09e9d0 2023-09-30T22:16:44.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa82284700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0a0330 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09f2e0 0x7ffa7c09f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09f2e0 0x7ffa7c09f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40194/0 (socket says 172.21.15.89:40194) 2023-09-30T22:16:44.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7b7fe700 1 -- 172.21.15.89:0/3941815504 learned_addr learned my addr 172.21.15.89:0/3941815504 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:44.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7bfff700 1 --2- 172.21.15.89:0/3941815504 >> v2:172.21.15.89:3300/0 conn(0x7ffa7c09e9d0 0x7ffa7c09eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7b7fe700 1 -- 172.21.15.89:0/3941815504 >> v2:172.21.15.89:3300/0 conn(0x7ffa7c09e9d0 msgr2=0x7ffa7c09eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:44.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7b7fe700 1 --2- 172.21.15.89:0/3941815504 >> v2:172.21.15.89:3300/0 conn(0x7ffa7c09e9d0 0x7ffa7c09eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7b7fe700 1 -- 172.21.15.89:0/3941815504 >> v1:172.21.15.89:6789/0 conn(0x7ffa7c1317b0 legacy=0x7ffa7c131bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7b7fe700 1 -- 172.21.15.89:0/3941815504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c06ef60 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7affd700 1 -- 172.21.15.89:0/3941815504 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445844633 0 0) 0x7ffa7c09ab50 con 0x7ffa7c1317b0 2023-09-30T22:16:44.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7bfff700 1 --2- 172.21.15.89:0/3941815504 >> v2:172.21.15.89:3300/0 conn(0x7ffa7c09e9d0 0x7ffa7c09eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:44.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.080+0000 7ffa7b7fe700 1 --2- 172.21.15.89:0/3941815504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09f2e0 0x7ffa7c09f6b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa640098f0 tx=0x7ffa64009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1dc13b967b91849 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:44.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa7affd700 1 -- 172.21.15.89:0/3941815504 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64011830 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa7affd700 1 -- 172.21.15.89:0/3941815504 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64011990 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa7affd700 1 -- 172.21.15.89:0/3941815504 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64011d20 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa82284700 1 -- 172.21.15.89:0/3941815504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09f2e0 msgr2=0x7ffa7c09f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa82284700 1 --2- 172.21.15.89:0/3941815504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09f2e0 0x7ffa7c09f6b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa640098f0 tx=0x7ffa64009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa82284700 1 -- 172.21.15.89:0/3941815504 shutdown_connections 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa82284700 1 --2- 172.21.15.89:0/3941815504 >> v2:172.21.15.89:3300/0 conn(0x7ffa7c09e9d0 0x7ffa7c09eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa82284700 1 --2- 172.21.15.89:0/3941815504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09f2e0 0x7ffa7c09f6b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa82284700 1 -- 172.21.15.89:0/3941815504 >> 172.21.15.89:0/3941815504 conn(0x7ffa7c0a43f0 msgr2=0x7ffa7c0a47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa82284700 1 -- 172.21.15.89:0/3941815504 shutdown_connections 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.081+0000 7ffa82284700 1 -- 172.21.15.89:0/3941815504 wait complete. 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.082+0000 7ffa82284700 1 Processor -- start 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.082+0000 7ffa82284700 1 -- start start 2023-09-30T22:16:44.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa82284700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09e9d0 0x7ffa7c11d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa82284700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa7c09f2e0 0x7ffa7c11d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa82284700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa7c1317b0 0x7ffa7c1c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa82284700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c120050 con 0x7ffa7c1317b0 2023-09-30T22:16:44.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa82284700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c11f120 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa82284700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c1311a0 con 0x7ffa7c09e9d0 2023-09-30T22:16:44.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09e9d0 0x7ffa7c11d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09e9d0 0x7ffa7c11d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40206/0 (socket says 172.21.15.89:40206) 2023-09-30T22:16:44.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7bfff700 1 -- 172.21.15.89:0/1859792622 learned_addr learned my addr 172.21.15.89:0/1859792622 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7b7fe700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa7c09f2e0 0x7ffa7c11d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa80821700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa7c1317b0 0x7ffa7c1c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7b7fe700 1 -- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09e9d0 msgr2=0x7ffa7c11d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7b7fe700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09e9d0 0x7ffa7c11d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7b7fe700 1 -- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa7c1317b0 msgr2=0x7ffa7c1c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7b7fe700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa7c1317b0 0x7ffa7c1c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7b7fe700 1 -- 172.21.15.89:0/1859792622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c0094a0 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa80821700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa7c1317b0 0x7ffa7c1c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa7b7fe700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa7c09f2e0 0x7ffa7c11d850 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c09e640 tx=0x7ffa64004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:44.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa78ff9700 1 -- 172.21.15.89:0/1859792622 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004300 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa78ff9700 1 -- 172.21.15.89:0/1859792622 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6401f070 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa78ff9700 1 -- 172.21.15.89:0/1859792622 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa640036a0 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa64009550 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.083+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c11fde0 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.084+0000 7ffa78ff9700 1 -- 172.21.15.89:0/1859792622 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ffa64021030 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.084+0000 7ffa78ff9700 1 -- 172.21.15.89:0/1859792622 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7ffa64040d90 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.084+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c11ce90 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.089+0000 7ffa78ff9700 1 -- 172.21.15.89:0/1859792622 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa640239a0 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.352+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffa7c04f2a0 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.353+0000 7ffa78ff9700 1 -- 172.21.15.89:0/1859792622 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+627 (secure 0 0 0) 0x7ffa640233d0 con 0x7ffa7c09f2e0 2023-09-30T22:16:44.354 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-30T22:16:44.354 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": { 2023-09-30T22:16:44.355 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 3 2023-09-30T22:16:44.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:44.355 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": { 2023-09-30T22:16:44.355 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 1 2023-09-30T22:16:44.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:44.355 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": { 2023-09-30T22:16:44.356 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 4 2023-09-30T22:16:44.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:44.356 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2023-09-30T22:16:44.356 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 1 2023-09-30T22:16:44.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-30T22:16:44.356 INFO:teuthology.orchestra.run.smithi089.stdout: "overall": { 2023-09-30T22:16:44.356 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 9 2023-09-30T22:16:44.356 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-30T22:16:44.357 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-30T22:16:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 >> v1:172.21.15.89:6816/126953 conn(0x7ffa5c02f510 legacy=0x7ffa5c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa7c09f2e0 msgr2=0x7ffa7c11d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa7c09f2e0 0x7ffa7c11d850 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c09e640 tx=0x7ffa64004750 comp rx=0 tx=0).stop 2023-09-30T22:16:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 shutdown_connections 2023-09-30T22:16:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa7c1317b0 0x7ffa7c1c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa7c09e9d0 0x7ffa7c11d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 --2- 172.21.15.89:0/1859792622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa7c09f2e0 0x7ffa7c11d850 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 >> 172.21.15.89:0/1859792622 conn(0x7ffa7c0a43f0 msgr2=0x7ffa7c1b99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 shutdown_connections 2023-09-30T22:16:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.356+0000 7ffa82284700 1 -- 172.21.15.89:0/1859792622 wait complete. 2023-09-30T22:16:44.369 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done' 2023-09-30T22:16:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.489+0000 7f734a06e700 1 Processor -- start 2023-09-30T22:16:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.489+0000 7f734a06e700 1 -- start start 2023-09-30T22:16:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f734a06e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409c910 0x7f734409cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f734a06e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f734409d850 0x7f734409dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f734a06e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409e1c0 0x7f734413ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f734a06e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734409ed90 con 0x7f734409d850 2023-09-30T22:16:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f734a06e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73440a0880 con 0x7f734409e1c0 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f734a06e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f734409f200 con 0x7f734409c910 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f7342ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f734409d850 0x7f734409dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409c910 0x7f734409cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f7343fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409e1c0 0x7f734413ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f7342ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f734409d850 0x7f734409dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50808/0 (socket says 172.21.15.89:50808) 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f7343fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409e1c0 0x7f734413ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36336/0 (socket says 172.21.15.89:36336) 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409c910 0x7f734409cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40212/0 (socket says 172.21.15.89:40212) 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f7342ffd700 1 -- 172.21.15.89:0/2308421214 learned_addr learned my addr 172.21.15.89:0/2308421214 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73437fe700 1 -- 172.21.15.89:0/2308421214 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409e1c0 msgr2=0x7f734413ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73437fe700 1 --2- 172.21.15.89:0/2308421214 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409e1c0 0x7f734413ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73437fe700 1 -- 172.21.15.89:0/2308421214 >> v2:172.21.15.89:3300/0 conn(0x7f734409d850 msgr2=0x7f734409dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73437fe700 1 --2- 172.21.15.89:0/2308421214 >> v2:172.21.15.89:3300/0 conn(0x7f734409d850 0x7f734409dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73437fe700 1 -- 172.21.15.89:0/2308421214 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73440a05e0 con 0x7f734409c910 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f7343fff700 1 --2- 172.21.15.89:0/2308421214 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409e1c0 0x7f734413ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f7342ffd700 1 --2- 172.21.15.89:0/2308421214 >> v2:172.21.15.89:3300/0 conn(0x7f734409d850 0x7f734409dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73437fe700 1 --2- 172.21.15.89:0/2308421214 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409c910 0x7f734409cce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7338009820 tx=0x7f7338009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be64e4d72a473d48 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73427fc700 1 -- 172.21.15.89:0/2308421214 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338005e30 con 0x7f734409c910 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73427fc700 1 -- 172.21.15.89:0/2308421214 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7338006030 con 0x7f734409c910 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.490+0000 7f73427fc700 1 -- 172.21.15.89:0/2308421214 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380063c0 con 0x7f734409c910 2023-09-30T22:16:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 -- 172.21.15.89:0/2308421214 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409c910 msgr2=0x7f734409cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 --2- 172.21.15.89:0/2308421214 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409c910 0x7f734409cce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7338009820 tx=0x7f7338009b30 comp rx=0 tx=0).stop 2023-09-30T22:16:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 -- 172.21.15.89:0/2308421214 shutdown_connections 2023-09-30T22:16:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 --2- 172.21.15.89:0/2308421214 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409e1c0 0x7f734413ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 --2- 172.21.15.89:0/2308421214 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409c910 0x7f734409cce0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 --2- 172.21.15.89:0/2308421214 >> v2:172.21.15.89:3300/0 conn(0x7f734409d850 0x7f734409dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 -- 172.21.15.89:0/2308421214 >> 172.21.15.89:0/2308421214 conn(0x7f73440a2e50 msgr2=0x7f73440a3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 -- 172.21.15.89:0/2308421214 shutdown_connections 2023-09-30T22:16:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.491+0000 7f734a06e700 1 -- 172.21.15.89:0/2308421214 wait complete. 2023-09-30T22:16:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.492+0000 7f734a06e700 1 Processor -- start 2023-09-30T22:16:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.492+0000 7f734a06e700 1 -- start start 2023-09-30T22:16:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f734a06e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409c910 0x7f734412b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f734a06e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734409d850 0x7f734412b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f734a06e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409e1c0 0x7f734412be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f734a06e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734412c9f0 con 0x7f734409d850 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f734a06e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734412e510 con 0x7f734409c910 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f734a06e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73440a4a60 con 0x7f734409e1c0 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734409d850 0x7f734412b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7343fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409e1c0 0x7f734412be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734409d850 0x7f734412b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50810/0 (socket says 172.21.15.89:50810) 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7343fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409e1c0 0x7f734412be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40218/0 (socket says 172.21.15.89:40218) 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 -- 172.21.15.89:0/1406836100 learned_addr learned my addr 172.21.15.89:0/1406836100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f73437fe700 1 --2- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409c910 0x7f734412b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 -- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409e1c0 msgr2=0x7f734412be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 --2- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409e1c0 0x7f734412be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 -- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409c910 msgr2=0x7f734412b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 --2- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409c910 0x7f734412b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 -- 172.21.15.89:0/1406836100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7334009550 con 0x7f734409d850 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f7342ffd700 1 --2- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734409d850 0x7f734412b950 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f733400ebb0 tx=0x7f733400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f732bfff700 1 -- 172.21.15.89:0/1406836100 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334009e60 con 0x7f734409d850 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f732bfff700 1 -- 172.21.15.89:0/1406836100 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334004240 con 0x7f734409d850 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c009a70 con 0x7f734409d850 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f732bfff700 1 -- 172.21.15.89:0/1406836100 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340036a0 con 0x7f734409d850 2023-09-30T22:16:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.493+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7338009430 con 0x7f734409d850 2023-09-30T22:16:44.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.494+0000 7f732bfff700 1 -- 172.21.15.89:0/1406836100 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7334003800 con 0x7f734409d850 2023-09-30T22:16:44.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.494+0000 7f732bfff700 1 -- 172.21.15.89:0/1406836100 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f733403fdf0 con 0x7f734409d850 2023-09-30T22:16:44.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.494+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734406e560 con 0x7f734409d850 2023-09-30T22:16:44.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.499+0000 7f732bfff700 1 -- 172.21.15.89:0/1406836100 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7334011760 con 0x7f734409d850 2023-09-30T22:16:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.682+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7f734412e1b0 con 0x7f734409d850 2023-09-30T22:16:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.683+0000 7f732bfff700 1 -- 172.21.15.89:0/1406836100 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v34) v1 ==== 59+0+37 (secure 0 0 0) 0x7f734412e1b0 con 0x7f734409d850 2023-09-30T22:16:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 >> v1:172.21.15.89:6816/126953 conn(0x7f733002f510 legacy=0x7f73300319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734409d850 msgr2=0x7f734412b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 --2- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734409d850 0x7f734412b950 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f733400ebb0 tx=0x7f733400eec0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 shutdown_connections 2023-09-30T22:16:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 --2- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734409e1c0 0x7f734412be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 --2- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734409c910 0x7f734412b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 --2- 172.21.15.89:0/1406836100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734409d850 0x7f734412b950 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 >> 172.21.15.89:0/1406836100 conn(0x7f73440a2e50 msgr2=0x7f73441bb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 shutdown_connections 2023-09-30T22:16:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.685+0000 7f734a06e700 1 -- 172.21.15.89:0/1406836100 wait complete. 2023-09-30T22:16:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.791+0000 7f6627960700 1 Processor -- start 2023-09-30T22:16:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.791+0000 7f6627960700 1 -- start start 2023-09-30T22:16:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6627960700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662012d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6627960700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f662012dc60 0x7f6620099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6627960700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662009a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6627960700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662009c230 con 0x7f662012dc60 2023-09-30T22:16:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6627960700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662009b4a0 con 0x7f6620099f90 2023-09-30T22:16:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6627960700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f662009ce10 con 0x7f662012d350 2023-09-30T22:16:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6625efd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662009a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6624efb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f662012dc60 0x7f6620099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f66256fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662012d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6625efd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662009a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36346/0 (socket says 172.21.15.89:36346) 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6624efb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f662012dc60 0x7f6620099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50818/0 (socket says 172.21.15.89:50818) 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6625efd700 1 -- 172.21.15.89:0/3382148429 learned_addr learned my addr 172.21.15.89:0/3382148429 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f66256fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662012d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40222/0 (socket says 172.21.15.89:40222) 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f66256fc700 1 -- 172.21.15.89:0/3382148429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 msgr2=0x7f662009a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f66256fc700 1 --2- 172.21.15.89:0/3382148429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662009a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f66256fc700 1 -- 172.21.15.89:0/3382148429 >> v2:172.21.15.89:3300/0 conn(0x7f662012dc60 msgr2=0x7f6620099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f66256fc700 1 --2- 172.21.15.89:0/3382148429 >> v2:172.21.15.89:3300/0 conn(0x7f662012dc60 0x7f6620099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f66256fc700 1 -- 172.21.15.89:0/3382148429 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662009bc90 con 0x7f662012d350 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6624efb700 1 --2- 172.21.15.89:0/3382148429 >> v2:172.21.15.89:3300/0 conn(0x7f662012dc60 0x7f6620099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6625efd700 1 --2- 172.21.15.89:0/3382148429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662009a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f66256fc700 1 --2- 172.21.15.89:0/3382148429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662012d720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6610009840 tx=0x7f6610009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71698111c5c049e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6617fff700 1 -- 172.21.15.89:0/3382148429 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610011830 con 0x7f662012d350 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6617fff700 1 -- 172.21.15.89:0/3382148429 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610011990 con 0x7f662012d350 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.792+0000 7f6617fff700 1 -- 172.21.15.89:0/3382148429 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610011d20 con 0x7f662012d350 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 -- 172.21.15.89:0/3382148429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 msgr2=0x7f662012d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 --2- 172.21.15.89:0/3382148429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662012d720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6610009840 tx=0x7f6610009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 -- 172.21.15.89:0/3382148429 shutdown_connections 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 --2- 172.21.15.89:0/3382148429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662009a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 --2- 172.21.15.89:0/3382148429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662012d720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 --2- 172.21.15.89:0/3382148429 >> v2:172.21.15.89:3300/0 conn(0x7f662012dc60 0x7f6620099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 -- 172.21.15.89:0/3382148429 >> 172.21.15.89:0/3382148429 conn(0x7f66200a29e0 msgr2=0x7f66200a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 -- 172.21.15.89:0/3382148429 shutdown_connections 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.793+0000 7f6627960700 1 -- 172.21.15.89:0/3382148429 wait complete. 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6627960700 1 Processor -- start 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6627960700 1 -- start start 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6627960700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662011da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6627960700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662011df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6627960700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f662012dc60 0x7f662011e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6627960700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662011fea0 con 0x7f662012dc60 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6627960700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6620120ce0 con 0x7f6620099f90 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6627960700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6620130e80 con 0x7f662012d350 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f66256fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662011da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6624efb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662011df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f6625efd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f662012dc60 0x7f662011e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f66256fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662011da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36358/0 (socket says 172.21.15.89:36358) 2023-09-30T22:16:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.794+0000 7f66256fc700 1 -- 172.21.15.89:0/2398166308 learned_addr learned my addr 172.21.15.89:0/2398166308 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6624efb700 1 -- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 msgr2=0x7f662011da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6624efb700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662011da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6624efb700 1 -- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f662012dc60 msgr2=0x7f662011e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6624efb700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f662012dc60 0x7f662011e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6624efb700 1 -- 172.21.15.89:0/2398166308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618009b30 con 0x7f662012d350 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f66256fc700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662011da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6625efd700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f662012dc60 0x7f662011e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6624efb700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662011df90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6620074d00 tx=0x7f661c00d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:44.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6615ffb700 1 -- 172.21.15.89:0/2398166308 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c00dd90 con 0x7f662012d350 2023-09-30T22:16:44.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6615ffb700 1 -- 172.21.15.89:0/2398166308 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c010440 con 0x7f662012d350 2023-09-30T22:16:44.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c00b740 con 0x7f662012d350 2023-09-30T22:16:44.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6615ffb700 1 -- 172.21.15.89:0/2398166308 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0107f0 con 0x7f662012d350 2023-09-30T22:16:44.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.795+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66100094a0 con 0x7f662012d350 2023-09-30T22:16:44.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.796+0000 7f6615ffb700 1 -- 172.21.15.89:0/2398166308 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f661c010950 con 0x7f662012d350 2023-09-30T22:16:44.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.796+0000 7f6615ffb700 1 -- 172.21.15.89:0/2398166308 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f661c040190 con 0x7f662012d350 2023-09-30T22:16:44.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.796+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662006ecb0 con 0x7f662012d350 2023-09-30T22:16:44.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.801+0000 7f6615ffb700 1 -- 172.21.15.89:0/2398166308 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f661c018400 con 0x7f662012d350 2023-09-30T22:16:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:44.983+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f662004f2a0 con 0x7f662012d350 2023-09-30T22:16:45.312 INFO:tasks.ceph.mgr.x.smithi089.stderr:terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of_buffer' 2023-09-30T22:16:45.312 INFO:tasks.ceph.mgr.x.smithi089.stderr:terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of_buffer' 2023-09-30T22:16:45.312 INFO:tasks.ceph.mgr.x.smithi089.stderr: what(): End of buffer [buffer:2] 2023-09-30T22:16:45.313 INFO:tasks.ceph.mgr.x.smithi089.stderr: what(): End of buffer [buffer:2] 2023-09-30T22:16:45.313 INFO:tasks.ceph.mgr.x.smithi089.stderr:*** Caught signal (Aborted) ** 2023-09-30T22:16:45.313 INFO:tasks.ceph.mgr.x.smithi089.stderr: in thread 7fa0f7df2700 thread_name:ms_dispatch 2023-09-30T22:16:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.363+0000 7f6615ffb700 1 -- 172.21.15.89:0/2398166308 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 1 pg_autoscale_mode to off v35) v1 ==== 153+0+0 (secure 0 0 0) 0x7f661c016300 con 0x7f662012d350 2023-09-30T22:16:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 >> v1:172.21.15.89:6816/126953 conn(0x7f660402f510 legacy=0x7f66040319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 msgr2=0x7f662011df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662011df90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6620074d00 tx=0x7f661c00d630 comp rx=0 tx=0).stop 2023-09-30T22:16:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 shutdown_connections 2023-09-30T22:16:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f662012dc60 0x7f662011e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6620099f90 0x7f662011da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 --2- 172.21.15.89:0/2398166308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f662012d350 0x7f662011df90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 >> 172.21.15.89:0/2398166308 conn(0x7f66200a29e0 msgr2=0x7f662012fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 shutdown_connections 2023-09-30T22:16:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.368+0000 7f6627960700 1 -- 172.21.15.89:0/2398166308 wait complete. 2023-09-30T22:16:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:set pool 1 pg_autoscale_mode to off 2023-09-30T22:16:45.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.483+0000 7fae932e2700 1 Processor -- start 2023-09-30T22:16:45.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.483+0000 7fae932e2700 1 -- start start 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.483+0000 7fae932e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c09dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.483+0000 7fae932e2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae8c130e20 0x7fae8c138b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.483+0000 7fae932e2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8c09b9b0 con 0x7fae8c09e260 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.483+0000 7fae932e2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8c09b320 con 0x7fae8c130e20 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.483+0000 7fae932e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c09ed20 con 0x7fae8c09d950 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9187f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae8c130e20 0x7fae8c138b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9107e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c09dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9187f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae8c130e20 0x7fae8c138b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50842/0 (socket says 172.21.15.89:50842) 2023-09-30T22:16:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9087d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fae8c09e260 0x7fae8c130710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53230/0 (socket says 172.21.15.89:53230) 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9187f700 1 -- 172.21.15.89:0/2879932576 learned_addr learned my addr 172.21.15.89:0/2879932576 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9107e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c09dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40252/0 (socket says 172.21.15.89:40252) 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9107e700 1 -- 172.21.15.89:0/2879932576 >> v2:172.21.15.89:3300/0 conn(0x7fae8c130e20 msgr2=0x7fae8c138b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9107e700 1 --2- 172.21.15.89:0/2879932576 >> v2:172.21.15.89:3300/0 conn(0x7fae8c130e20 0x7fae8c138b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9107e700 1 -- 172.21.15.89:0/2879932576 >> v1:172.21.15.89:6789/0 conn(0x7fae8c09e260 legacy=0x7fae8c130710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae83fff700 1 -- 172.21.15.89:0/2879932576 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748465834 0 0) 0x7fae8c09b9b0 con 0x7fae8c09e260 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9107e700 1 -- 172.21.15.89:0/2879932576 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c09a350 con 0x7fae8c09d950 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9187f700 1 --2- 172.21.15.89:0/2879932576 >> v2:172.21.15.89:3300/0 conn(0x7fae8c130e20 0x7fae8c138b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae9107e700 1 --2- 172.21.15.89:0/2879932576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c09dd20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae88009840 tx=0x7fae88009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d281f42cd987046d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae83fff700 1 -- 172.21.15.89:0/2879932576 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88011830 con 0x7fae8c09d950 2023-09-30T22:16:45.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae83fff700 1 -- 172.21.15.89:0/2879932576 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88011990 con 0x7fae8c09d950 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.484+0000 7fae83fff700 1 -- 172.21.15.89:0/2879932576 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88011d20 con 0x7fae8c09d950 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.485+0000 7fae932e2700 1 -- 172.21.15.89:0/2879932576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 msgr2=0x7fae8c09dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.485+0000 7fae932e2700 1 --2- 172.21.15.89:0/2879932576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c09dd20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae88009840 tx=0x7fae88009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.485+0000 7fae932e2700 1 -- 172.21.15.89:0/2879932576 shutdown_connections 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.485+0000 7fae932e2700 1 --2- 172.21.15.89:0/2879932576 >> v2:172.21.15.89:3300/0 conn(0x7fae8c130e20 0x7fae8c138b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.485+0000 7fae932e2700 1 --2- 172.21.15.89:0/2879932576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c09dd20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.485+0000 7fae932e2700 1 -- 172.21.15.89:0/2879932576 >> 172.21.15.89:0/2879932576 conn(0x7fae8c0a2f50 msgr2=0x7fae8c0a3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.485+0000 7fae932e2700 1 -- 172.21.15.89:0/2879932576 shutdown_connections 2023-09-30T22:16:45.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.485+0000 7fae932e2700 1 -- 172.21.15.89:0/2879932576 wait complete. 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.486+0000 7fae932e2700 1 Processor -- start 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.486+0000 7fae932e2700 1 -- start start 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.486+0000 7fae932e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c1c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.486+0000 7fae932e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae8c09e260 0x7fae8c1c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.486+0000 7fae932e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c130e20 0x7fae8c11dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.486+0000 7fae932e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0a2c20 con 0x7fae8c130e20 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.486+0000 7fae932e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c11fec0 con 0x7fae8c09e260 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.486+0000 7fae932e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c11f440 con 0x7fae8c09d950 2023-09-30T22:16:45.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9187f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c130e20 0x7fae8c11dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9107e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c1c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9107e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c1c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40260/0 (socket says 172.21.15.89:40260) 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9187f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c130e20 0x7fae8c11dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50850/0 (socket says 172.21.15.89:50850) 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9107e700 1 -- 172.21.15.89:0/3027903643 learned_addr learned my addr 172.21.15.89:0/3027903643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9087d700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae8c09e260 0x7fae8c1c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9187f700 1 -- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 msgr2=0x7fae8c1c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9187f700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c1c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9187f700 1 -- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae8c09e260 msgr2=0x7fae8c1c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9187f700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae8c09e260 0x7fae8c1c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:45.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9187f700 1 -- 172.21.15.89:0/3027903643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84009b30 con 0x7fae8c130e20 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9107e700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c1c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9087d700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae8c09e260 0x7fae8c1c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae9187f700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c130e20 0x7fae8c11dab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae8400efc0 tx=0x7fae8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae81ffb700 1 -- 172.21.15.89:0/3027903643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84018070 con 0x7fae8c130e20 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae81ffb700 1 -- 172.21.15.89:0/3027903643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8400f040 con 0x7fae8c130e20 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae880094a0 con 0x7fae8c130e20 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.487+0000 7fae81ffb700 1 -- 172.21.15.89:0/3027903643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84003c60 con 0x7fae8c130e20 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.488+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c130470 con 0x7fae8c130e20 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.488+0000 7fae81ffb700 1 -- 172.21.15.89:0/3027903643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fae8400c880 con 0x7fae8c130e20 2023-09-30T22:16:45.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.488+0000 7fae81ffb700 1 -- 172.21.15.89:0/3027903643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fae8403fe50 con 0x7fae8c130e20 2023-09-30T22:16:45.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.489+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c120000 con 0x7fae8c130e20 2023-09-30T22:16:45.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.493+0000 7fae81ffb700 1 -- 172.21.15.89:0/3027903643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae84003dc0 con 0x7fae8c130e20 2023-09-30T22:16:45.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.565+0000 7fae9107e700 1 -- 172.21.15.89:0/3027903643 >> v1:172.21.15.89:6816/126953 conn(0x7fae7002f510 legacy=0x7fae700319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk reading from fd=16 : -104 Unknown error -104 2023-09-30T22:16:45.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.565+0000 7fae9107e700 1 -- 172.21.15.89:0/3027903643 >> v1:172.21.15.89:6816/126953 conn(0x7fae7002f510 legacy=0x7fae700319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:16:45.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.565+0000 7fae9107e700 1 --1- 172.21.15.89:0/3027903643 >> v1:172.21.15.89:6816/126953 conn(0x7fae7002f510 0x7fae700319c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify read banner and identify addresses failed 2023-09-30T22:16:45.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.565+0000 7fae9107e700 1 --1- 172.21.15.89:0/3027903643 >> v1:172.21.15.89:6816/126953 conn(0x7fae7002f510 0x7fae700319c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).fault on lossy channel, failing 2023-09-30T22:16:45.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.565+0000 7fae81ffb700 1 -- 172.21.15.89:0/3027903643 >> v1:172.21.15.89:6816/126953 conn(0x7fae7002f510 legacy=0x7fae700319c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-09-30T22:16:45.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:45.680+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7fae640051e0 con 0x7fae8c130e20 2023-09-30T22:16:46.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.367+0000 7fae81ffb700 1 -- 172.21.15.89:0/3027903643 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 2 pg_autoscale_mode to off v36) v1 ==== 152+0+0 (secure 0 0 0) 0x7fae8401cbb0 con 0x7fae8c130e20 2023-09-30T22:16:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.370+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c130e20 msgr2=0x7fae8c11dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.370+0000 7fae932e2700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c130e20 0x7fae8c11dab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae8400efc0 tx=0x7fae8400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.370+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 shutdown_connections 2023-09-30T22:16:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.370+0000 7fae932e2700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c130e20 0x7fae8c11dab0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.370+0000 7fae932e2700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae8c09d950 0x7fae8c1c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.370+0000 7fae932e2700 1 --2- 172.21.15.89:0/3027903643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae8c09e260 0x7fae8c1c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.370+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 >> 172.21.15.89:0/3027903643 conn(0x7fae8c0a2f50 msgr2=0x7fae8c1b89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.371+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 shutdown_connections 2023-09-30T22:16:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.371+0000 7fae932e2700 1 -- 172.21.15.89:0/3027903643 wait complete. 2023-09-30T22:16:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:set pool 2 pg_autoscale_mode to off 2023-09-30T22:16:46.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f612940d700 1 Processor -- start 2023-09-30T22:16:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f612940d700 1 -- start start 2023-09-30T22:16:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f612940d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f612412d350 0x7f612412d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f612940d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612409a160 0x7f61241387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f612940d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612409b4a0 con 0x7f612412dc60 2023-09-30T22:16:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f612940d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612409c230 con 0x7f612412d350 2023-09-30T22:16:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f612940d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612409ce10 con 0x7f612409a160 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f6122ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f612412d350 0x7f612412d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f61237fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612409a160 0x7f61241387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f6122ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f612412d350 0x7f612412d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50860/0 (socket says 172.21.15.89:50860) 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.482+0000 7f6122ffd700 1 -- 172.21.15.89:0/2150836125 learned_addr learned my addr 172.21.15.89:0/2150836125 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f6122ffd700 1 -- 172.21.15.89:0/2150836125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612409a160 msgr2=0x7f61241387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f6122ffd700 1 --2- 172.21.15.89:0/2150836125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612409a160 0x7f61241387d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f6122ffd700 1 -- 172.21.15.89:0/2150836125 >> v1:172.21.15.89:6789/0 conn(0x7f612412dc60 legacy=0x7f6124099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f6122ffd700 1 -- 172.21.15.89:0/2150836125 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612409bc90 con 0x7f612412d350 2023-09-30T22:16:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f61237fe700 1 --2- 172.21.15.89:0/2150836125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612409a160 0x7f61241387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f6122ffd700 1 --2- 172.21.15.89:0/2150836125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612412d350 0x7f612412d720 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f61180098f0 tx=0x7f6118009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7511fbb98b047073 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f6121ffb700 1 -- 172.21.15.89:0/2150836125 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118011830 con 0x7f612412d350 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f6121ffb700 1 -- 172.21.15.89:0/2150836125 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6118011990 con 0x7f612412d350 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.483+0000 7f6121ffb700 1 -- 172.21.15.89:0/2150836125 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118011d20 con 0x7f612412d350 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.484+0000 7f612940d700 1 -- 172.21.15.89:0/2150836125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612412d350 msgr2=0x7f612412d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.484+0000 7f612940d700 1 --2- 172.21.15.89:0/2150836125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612412d350 0x7f612412d720 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f61180098f0 tx=0x7f6118009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.484+0000 7f612940d700 1 -- 172.21.15.89:0/2150836125 shutdown_connections 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.484+0000 7f612940d700 1 --2- 172.21.15.89:0/2150836125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612409a160 0x7f61241387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.484+0000 7f612940d700 1 --2- 172.21.15.89:0/2150836125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612412d350 0x7f612412d720 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.484+0000 7f612940d700 1 -- 172.21.15.89:0/2150836125 >> 172.21.15.89:0/2150836125 conn(0x7f61240a29e0 msgr2=0x7f61240a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.484+0000 7f612940d700 1 -- 172.21.15.89:0/2150836125 shutdown_connections 2023-09-30T22:16:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.484+0000 7f612940d700 1 -- 172.21.15.89:0/2150836125 wait complete. 2023-09-30T22:16:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.485+0000 7f612940d700 1 Processor -- start 2023-09-30T22:16:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.485+0000 7f612940d700 1 -- start start 2023-09-30T22:16:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f612940d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612409a160 0x7f61241c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f612940d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612412d350 0x7f61241c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f612940d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612412dc60 0x7f61241c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f612940d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612411be60 con 0x7f612409a160 2023-09-30T22:16:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f612940d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6124130fe0 con 0x7f612412dc60 2023-09-30T22:16:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f612940d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f612409b670 con 0x7f612412d350 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612409a160 0x7f61241c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612409a160 0x7f61241c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50868/0 (socket says 172.21.15.89:50868) 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 -- 172.21.15.89:0/4193017221 learned_addr learned my addr 172.21.15.89:0/4193017221 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f61227fc700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612412d350 0x7f61241c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f61237fe700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612412dc60 0x7f61241c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 -- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612412d350 msgr2=0x7f61241c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612412d350 0x7f61241c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 -- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612412dc60 msgr2=0x7f61241c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612412dc60 0x7f61241c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 -- 172.21.15.89:0/4193017221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c009a70 con 0x7f612409a160 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f6122ffd700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612409a160 0x7f61241c85c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6118000c00 tx=0x7f6118004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f610b7fe700 1 -- 172.21.15.89:0/4193017221 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180043c0 con 0x7f612409a160 2023-09-30T22:16:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f610b7fe700 1 -- 172.21.15.89:0/4193017221 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611801f070 con 0x7f612409a160 2023-09-30T22:16:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f610b7fe700 1 -- 172.21.15.89:0/4193017221 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180038c0 con 0x7f612409a160 2023-09-30T22:16:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.486+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6118009550 con 0x7f612409a160 2023-09-30T22:16:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.487+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6124121220 con 0x7f612409a160 2023-09-30T22:16:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.487+0000 7f610b7fe700 1 -- 172.21.15.89:0/4193017221 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6118021030 con 0x7f612409a160 2023-09-30T22:16:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.487+0000 7f61227fc700 1 -- 172.21.15.89:0/4193017221 >> v1:172.21.15.89:6816/126953 conn(0x7f611002f540 legacy=0x7f61100319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.487+0000 7f610b7fe700 1 -- 172.21.15.89:0/4193017221 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f6118040f00 con 0x7f612409a160 2023-09-30T22:16:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.487+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612404f2a0 con 0x7f612409a160 2023-09-30T22:16:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.492+0000 7f610b7fe700 1 -- 172.21.15.89:0/4193017221 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6118015080 con 0x7f612409a160 2023-09-30T22:16:46.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.675+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_metadata", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f61240721f0 con 0x7f612409a160 2023-09-30T22:16:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:46.688+0000 7f61227fc700 1 -- 172.21.15.89:0/4193017221 >> v1:172.21.15.89:6816/126953 conn(0x7f611002f540 legacy=0x7f61100319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:47.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.089+0000 7f61227fc700 1 -- 172.21.15.89:0/4193017221 >> v1:172.21.15.89:6816/126953 conn(0x7f611002f540 legacy=0x7f61100319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:47.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.371+0000 7f610b7fe700 1 -- 172.21.15.89:0/4193017221 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_metadata", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 3 pg_autoscale_mode to off v37) v1 ==== 164+0+0 (secure 0 0 0) 0x7f611800b9d0 con 0x7f612409a160 2023-09-30T22:16:47.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 >> v1:172.21.15.89:6816/126953 conn(0x7f611002f540 legacy=0x7f61100319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:47.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612409a160 msgr2=0x7f61241c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612409a160 0x7f61241c85c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6118000c00 tx=0x7f6118004750 comp rx=0 tx=0).stop 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 shutdown_connections 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612412dc60 0x7f61241c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612409a160 0x7f61241c85c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 --2- 172.21.15.89:0/4193017221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612412d350 0x7f61241c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 >> 172.21.15.89:0/4193017221 conn(0x7f61240a29e0 msgr2=0x7f61240a0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 shutdown_connections 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.374+0000 7f612940d700 1 -- 172.21.15.89:0/4193017221 wait complete. 2023-09-30T22:16:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:set pool 3 pg_autoscale_mode to off 2023-09-30T22:16:47.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.490+0000 7fc79745f700 1 Processor -- start 2023-09-30T22:16:47.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.490+0000 7fc79745f700 1 -- start start 2023-09-30T22:16:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.490+0000 7fc79745f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc79009c980 0x7fc79009cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.490+0000 7fc79745f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc790133600 0x7fc790133a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.490+0000 7fc79745f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79013a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.490+0000 7fc79745f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7900a1320 con 0x7fc79009c980 2023-09-30T22:16:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.490+0000 7fc79745f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79009f250 con 0x7fc790133600 2023-09-30T22:16:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.490+0000 7fc79745f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc79009ebd0 con 0x7fc790133f90 2023-09-30T22:16:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc79009c980 0x7fc79009cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:47.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7949fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc790133600 0x7fc790133a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc79009c980 0x7fc79009cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50880/0 (socket says 172.21.15.89:50880) 2023-09-30T22:16:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 -- 172.21.15.89:0/3489866801 learned_addr learned my addr 172.21.15.89:0/3489866801 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7959fc700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79013a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 -- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 msgr2=0x7fc79013a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79013a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 -- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc790133600 msgr2=0x7fc790133a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc790133600 0x7fc790133a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 -- 172.21.15.89:0/3489866801 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7901343e0 con 0x7fc79009c980 2023-09-30T22:16:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7949fa700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc790133600 0x7fc790133a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc7951fb700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79009c980 0x7fc79009cd50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098f0 tx=0x7fc78c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511774e371998e0a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc787fff700 1 -- 172.21.15.89:0/3489866801 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c011830 con 0x7fc79009c980 2023-09-30T22:16:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc787fff700 1 -- 172.21.15.89:0/3489866801 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c011990 con 0x7fc79009c980 2023-09-30T22:16:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.491+0000 7fc787fff700 1 -- 172.21.15.89:0/3489866801 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c011d20 con 0x7fc79009c980 2023-09-30T22:16:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 -- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79009c980 msgr2=0x7fc79009cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79009c980 0x7fc79009cd50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098f0 tx=0x7fc78c009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:47.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 -- 172.21.15.89:0/3489866801 shutdown_connections 2023-09-30T22:16:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79013a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79009c980 0x7fc79009cd50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 --2- 172.21.15.89:0/3489866801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc790133600 0x7fc790133a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 -- 172.21.15.89:0/3489866801 >> 172.21.15.89:0/3489866801 conn(0x7fc7900a64c0 msgr2=0x7fc7900a68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 -- 172.21.15.89:0/3489866801 shutdown_connections 2023-09-30T22:16:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.492+0000 7fc79745f700 1 -- 172.21.15.89:0/3489866801 wait complete. 2023-09-30T22:16:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc79745f700 1 Processor -- start 2023-09-30T22:16:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc79745f700 1 -- start start 2023-09-30T22:16:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc79745f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79009c980 0x7fc79006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc79745f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc790133600 0x7fc79006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc79745f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc79745f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79006ec70 con 0x7fc790133600 2023-09-30T22:16:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc79745f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900712c0 con 0x7fc79009c980 2023-09-30T22:16:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc79745f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc79006f9d0 con 0x7fc790133f90 2023-09-30T22:16:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc7951fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79009c980 0x7fc79006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc7959fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc7951fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79009c980 0x7fc79006d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36392/0 (socket says 172.21.15.89:36392) 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc7951fb700 1 -- 172.21.15.89:0/99800430 learned_addr learned my addr 172.21.15.89:0/99800430 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.493+0000 7fc7949fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc790133600 0x7fc79006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc7951fb700 1 -- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 msgr2=0x7fc79006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc7951fb700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79006e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc7951fb700 1 -- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc790133600 msgr2=0x7fc79006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc7951fb700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc790133600 0x7fc79006dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc7951fb700 1 -- 172.21.15.89:0/99800430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800094a0 con 0x7fc79009c980 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc7959fc700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc7949fa700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc790133600 0x7fc79006dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc7951fb700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79009c980 0x7fc79006d7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000c00 tx=0x7fc78c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:47.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc785ffb700 1 -- 172.21.15.89:0/99800430 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c004240 con 0x7fc79009c980 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc785ffb700 1 -- 172.21.15.89:0/99800430 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c01f070 con 0x7fc79009c980 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc785ffb700 1 -- 172.21.15.89:0/99800430 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0045e0 con 0x7fc79009c980 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c009550 con 0x7fc79009c980 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.494+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7900a36e0 con 0x7fc79009c980 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.495+0000 7fc785ffb700 1 -- 172.21.15.89:0/99800430 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc78c021030 con 0x7fc79009c980 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.495+0000 7fc785ffb700 1 -- 172.21.15.89:0/99800430 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc78c040d20 con 0x7fc79009c980 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.495+0000 7fc7949fa700 1 -- 172.21.15.89:0/99800430 >> v1:172.21.15.89:6816/126953 conn(0x7fc77402f540 legacy=0x7fc7740319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.495+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79009e770 con 0x7fc79009c980 2023-09-30T22:16:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.500+0000 7fc785ffb700 1 -- 172.21.15.89:0/99800430 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc78c015080 con 0x7fc79009c980 2023-09-30T22:16:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.684+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7fc79004f2a0 con 0x7fc79009c980 2023-09-30T22:16:47.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:47.696+0000 7fc7949fa700 1 -- 172.21.15.89:0/99800430 >> v1:172.21.15.89:6816/126953 conn(0x7fc77402f540 legacy=0x7fc7740319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:47.989 INFO:tasks.ceph.mgr.x.smithi089.stderr:daemon-helper: command crashed with signal 6 2023-09-30T22:16:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.097+0000 7fc7949fa700 1 -- 172.21.15.89:0/99800430 >> v1:172.21.15.89:6816/126953 conn(0x7fc77402f540 legacy=0x7fc7740319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.376+0000 7fc785ffb700 1 -- 172.21.15.89:0/99800430 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 4 pg_autoscale_mode to off v38) v1 ==== 160+0+0 (secure 0 0 0) 0x7fc78c01a450 con 0x7fc79009c980 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 >> v1:172.21.15.89:6816/126953 conn(0x7fc77402f540 legacy=0x7fc7740319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79009c980 msgr2=0x7fc79006d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79009c980 0x7fc79006d7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000c00 tx=0x7fc78c004750 comp rx=0 tx=0).stop 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 shutdown_connections 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc790133f90 0x7fc79006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79009c980 0x7fc79006d7a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 --2- 172.21.15.89:0/99800430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc790133600 0x7fc79006dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 >> 172.21.15.89:0/99800430 conn(0x7fc7900a64c0 msgr2=0x7fc790099bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 shutdown_connections 2023-09-30T22:16:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.381+0000 7fc79745f700 1 -- 172.21.15.89:0/99800430 wait complete. 2023-09-30T22:16:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:set pool 4 pg_autoscale_mode to off 2023-09-30T22:16:48.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release reef' 2023-09-30T22:16:48.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.522+0000 7fb6894c0700 1 Processor -- start 2023-09-30T22:16:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.522+0000 7fb6894c0700 1 -- start start 2023-09-30T22:16:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.522+0000 7fb6894c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6840744d0 0x7fb6841304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.522+0000 7fb6894c0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130a20 0x7fb68409d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.522+0000 7fb6894c0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb684130e10 0x7fb68409da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.522+0000 7fb6894c0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68409a200 con 0x7fb684130e10 2023-09-30T22:16:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.522+0000 7fb6894c0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68409a7c0 con 0x7fb684130a20 2023-09-30T22:16:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.522+0000 7fb6894c0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb68409ef60 con 0x7fb6840744d0 2023-09-30T22:16:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb682ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6840744d0 0x7fb6841304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6827fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130a20 0x7fb68409d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6837fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb684130e10 0x7fb68409da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb682ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6840744d0 0x7fb6841304e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40304/0 (socket says 172.21.15.89:40304) 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6837fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb684130e10 0x7fb68409da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50906/0 (socket says 172.21.15.89:50906) 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb682ffd700 1 -- 172.21.15.89:0/4215979250 learned_addr learned my addr 172.21.15.89:0/4215979250 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6827fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130a20 0x7fb68409d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36402/0 (socket says 172.21.15.89:36402) 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6827fc700 1 -- 172.21.15.89:0/4215979250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6840744d0 msgr2=0x7fb6841304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6827fc700 1 --2- 172.21.15.89:0/4215979250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6840744d0 0x7fb6841304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6827fc700 1 -- 172.21.15.89:0/4215979250 >> v2:172.21.15.89:3300/0 conn(0x7fb684130e10 msgr2=0x7fb68409da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6827fc700 1 --2- 172.21.15.89:0/4215979250 >> v2:172.21.15.89:3300/0 conn(0x7fb684130e10 0x7fb68409da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6827fc700 1 -- 172.21.15.89:0/4215979250 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6841301d0 con 0x7fb684130a20 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6837fe700 1 --2- 172.21.15.89:0/4215979250 >> v2:172.21.15.89:3300/0 conn(0x7fb684130e10 0x7fb68409da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb682ffd700 1 --2- 172.21.15.89:0/4215979250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6840744d0 0x7fb6841304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb6827fc700 1 --2- 172.21.15.89:0/4215979250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130a20 0x7fb68409d500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb674009840 tx=0x7fb674009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41eec384bd129c07 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb681ffb700 1 -- 172.21.15.89:0/4215979250 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674011830 con 0x7fb684130a20 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb681ffb700 1 -- 172.21.15.89:0/4215979250 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb674011990 con 0x7fb684130a20 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.523+0000 7fb681ffb700 1 -- 172.21.15.89:0/4215979250 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674011d20 con 0x7fb684130a20 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 -- 172.21.15.89:0/4215979250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130a20 msgr2=0x7fb68409d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 --2- 172.21.15.89:0/4215979250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130a20 0x7fb68409d500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb674009840 tx=0x7fb674009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 -- 172.21.15.89:0/4215979250 shutdown_connections 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 --2- 172.21.15.89:0/4215979250 >> v2:172.21.15.89:3300/0 conn(0x7fb684130e10 0x7fb68409da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 --2- 172.21.15.89:0/4215979250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6840744d0 0x7fb6841304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 --2- 172.21.15.89:0/4215979250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130a20 0x7fb68409d500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 -- 172.21.15.89:0/4215979250 >> 172.21.15.89:0/4215979250 conn(0x7fb6840a29e0 msgr2=0x7fb6840a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 -- 172.21.15.89:0/4215979250 shutdown_connections 2023-09-30T22:16:48.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.524+0000 7fb6894c0700 1 -- 172.21.15.89:0/4215979250 wait complete. 2023-09-30T22:16:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6894c0700 1 Processor -- start 2023-09-30T22:16:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6894c0700 1 -- start start 2023-09-30T22:16:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6894c0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6840744d0 0x7fb68411a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6894c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb684130a20 0x7fb68411adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6894c0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130e10 0x7fb68411b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6894c0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68411e130 con 0x7fb6840744d0 2023-09-30T22:16:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6894c0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68411e310 con 0x7fb684130e10 2023-09-30T22:16:48.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6894c0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb68411e810 con 0x7fb684130a20 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb682ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6840744d0 0x7fb68411a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6837fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130e10 0x7fb68411b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb682ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6840744d0 0x7fb68411a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50918/0 (socket says 172.21.15.89:50918) 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb6837fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130e10 0x7fb68411b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36416/0 (socket says 172.21.15.89:36416) 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.525+0000 7fb682ffd700 1 -- 172.21.15.89:0/883104161 learned_addr learned my addr 172.21.15.89:0/883104161 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb6827fc700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb684130a20 0x7fb68411adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb682ffd700 1 -- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb684130a20 msgr2=0x7fb68411adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb682ffd700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb684130a20 0x7fb68411adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb682ffd700 1 -- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130e10 msgr2=0x7fb68411b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb682ffd700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130e10 0x7fb68411b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb682ffd700 1 -- 172.21.15.89:0/883104161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678009550 con 0x7fb6840744d0 2023-09-30T22:16:48.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb6837fe700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130e10 0x7fb68411b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb682ffd700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6840744d0 0x7fb68411a8b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb67800ec10 tx=0x7fb67800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb66b7fe700 1 -- 172.21.15.89:0/883104161 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800c850 con 0x7fb6840744d0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb66b7fe700 1 -- 172.21.15.89:0/883104161 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67800c9b0 con 0x7fb6840744d0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c009a70 con 0x7fb6840744d0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb66b7fe700 1 -- 172.21.15.89:0/883104161 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800cd20 con 0x7fb6840744d0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.526+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6740094a0 con 0x7fb6840744d0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.527+0000 7fb66b7fe700 1 -- 172.21.15.89:0/883104161 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb678013470 con 0x7fb6840744d0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.527+0000 7fb6827fc700 1 -- 172.21.15.89:0/883104161 >> v1:172.21.15.89:6816/126953 conn(0x7fb67002f510 legacy=0x7fb6700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.527+0000 7fb66b7fe700 1 -- 172.21.15.89:0/883104161 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fb67803fc50 con 0x7fb6840744d0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.527+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68404f2a0 con 0x7fb6840744d0 2023-09-30T22:16:48.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.532+0000 7fb66b7fe700 1 -- 172.21.15.89:0/883104161 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb678017020 con 0x7fb6840744d0 2023-09-30T22:16:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.728+0000 7fb6827fc700 1 -- 172.21.15.89:0/883104161 >> v1:172.21.15.89:6816/126953 conn(0x7fb67002f510 legacy=0x7fb6700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:48.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:48.729+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd require-osd-release", "release": "reef"} v 0) v1 -- 0x7fb6840721f0 con 0x7fb6840744d0 2023-09-30T22:16:49.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.129+0000 7fb6827fc700 1 -- 172.21.15.89:0/883104161 >> v1:172.21.15.89:6816/126953 conn(0x7fb67002f510 legacy=0x7fb6700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:49.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.382+0000 7fb66b7fe700 1 -- 172.21.15.89:0/883104161 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd require-osd-release", "release": "reef"}]=0 v39) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb678013720 con 0x7fb6840744d0 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 >> v1:172.21.15.89:6816/126953 conn(0x7fb67002f510 legacy=0x7fb6700319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6840744d0 msgr2=0x7fb68411a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6840744d0 0x7fb68411a8b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb67800ec10 tx=0x7fb67800ef20 comp rx=0 tx=0).stop 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 shutdown_connections 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb684130e10 0x7fb68411b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6840744d0 0x7fb68411a8b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 --2- 172.21.15.89:0/883104161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb684130a20 0x7fb68411adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 >> 172.21.15.89:0/883104161 conn(0x7fb6840a29e0 msgr2=0x7fb68409fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 shutdown_connections 2023-09-30T22:16:49.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.387+0000 7fb6894c0700 1 -- 172.21.15.89:0/883104161 wait complete. 2023-09-30T22:16:49.403 INFO:teuthology.run_tasks:Running task ceph.healthy... 2023-09-30T22:16:49.414 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:16:49.414 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:16:49.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:16:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.541+0000 7fecf1633700 1 Processor -- start 2023-09-30T22:16:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.541+0000 7fecf1633700 1 -- start start 2023-09-30T22:16:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7fecf1633700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec0a09c0 0x7fecec0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7fecf1633700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fecec0a12d0 0x7fecec134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7fecf1633700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec09ea10 con 0x7fecec09d5c0 2023-09-30T22:16:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7fecf1633700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec09e210 con 0x7fecec0a12d0 2023-09-30T22:16:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7fecf1633700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0a26f0 con 0x7fecec0a09c0 2023-09-30T22:16:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec0a09c0 0x7fecec0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fecec0a12d0 0x7fecec134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec0a09c0 0x7fecec0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36426/0 (socket says 172.21.15.89:36426) 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceaffd700 1 -- 172.21.15.89:0/1482007277 learned_addr learned my addr 172.21.15.89:0/1482007277 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fecec0a12d0 0x7fecec134030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50932/0 (socket says 172.21.15.89:50932) 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceaffd700 1 -- 172.21.15.89:0/1482007277 >> v2:172.21.15.89:3300/0 conn(0x7fecec0a12d0 msgr2=0x7fecec134030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceaffd700 1 --2- 172.21.15.89:0/1482007277 >> v2:172.21.15.89:3300/0 conn(0x7fecec0a12d0 0x7fecec134030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceaffd700 1 -- 172.21.15.89:0/1482007277 >> v1:172.21.15.89:6789/0 conn(0x7fecec09d5c0 legacy=0x7fecec133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceaffd700 1 -- 172.21.15.89:0/1482007277 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec09db70 con 0x7fecec0a09c0 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceb7fe700 1 --2- 172.21.15.89:0/1482007277 >> v2:172.21.15.89:3300/0 conn(0x7fecec0a12d0 0x7fecec134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.542+0000 7feceaffd700 1 --2- 172.21.15.89:0/1482007277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec0a09c0 0x7fecec0a0d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fece00098f0 tx=0x7fece0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d7b9f2808630f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fece9ffb700 1 -- 172.21.15.89:0/1482007277 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0011830 con 0x7fecec0a09c0 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fece9ffb700 1 -- 172.21.15.89:0/1482007277 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece0011990 con 0x7fecec0a09c0 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fece9ffb700 1 -- 172.21.15.89:0/1482007277 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0011d20 con 0x7fecec0a09c0 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fecf1633700 1 -- 172.21.15.89:0/1482007277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec0a09c0 msgr2=0x7fecec0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fecf1633700 1 --2- 172.21.15.89:0/1482007277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec0a09c0 0x7fecec0a0d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fece00098f0 tx=0x7fece0009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fecf1633700 1 -- 172.21.15.89:0/1482007277 shutdown_connections 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fecf1633700 1 --2- 172.21.15.89:0/1482007277 >> v2:172.21.15.89:3300/0 conn(0x7fecec0a12d0 0x7fecec134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fecf1633700 1 --2- 172.21.15.89:0/1482007277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec0a09c0 0x7fecec0a0d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fecf1633700 1 -- 172.21.15.89:0/1482007277 >> 172.21.15.89:0/1482007277 conn(0x7fecec0a2fe0 msgr2=0x7fecec0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fecf1633700 1 -- 172.21.15.89:0/1482007277 shutdown_connections 2023-09-30T22:16:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.543+0000 7fecf1633700 1 -- 172.21.15.89:0/1482007277 wait complete. 2023-09-30T22:16:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.544+0000 7fecf1633700 1 Processor -- start 2023-09-30T22:16:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.544+0000 7fecf1633700 1 -- start start 2023-09-30T22:16:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecf1633700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d5c0 0x7fecec122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecf1633700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec0a09c0 0x7fecec122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecf1633700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec0a12d0 0x7fecec122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecf1633700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec125b60 con 0x7fecec0a09c0 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecf1633700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec125930 con 0x7fecec09d5c0 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecf1633700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fecec124dc0 con 0x7fecec0a12d0 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec0a12d0 0x7fecec122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecea7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec0a09c0 0x7fecec122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d5c0 0x7fecec122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec0a12d0 0x7fecec122da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40312/0 (socket says 172.21.15.89:40312) 2023-09-30T22:16:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecea7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec0a09c0 0x7fecec122860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50948/0 (socket says 172.21.15.89:50948) 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceb7fe700 1 -- 172.21.15.89:0/976019176 learned_addr learned my addr 172.21.15.89:0/976019176 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceaffd700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d5c0 0x7fecec122320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36434/0 (socket says 172.21.15.89:36434) 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceaffd700 1 -- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec0a12d0 msgr2=0x7fecec122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceaffd700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec0a12d0 0x7fecec122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceaffd700 1 -- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec0a09c0 msgr2=0x7fecec122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceaffd700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec0a09c0 0x7fecec122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceaffd700 1 -- 172.21.15.89:0/976019176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece4009a70 con 0x7fecec09d5c0 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecea7fc700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec0a09c0 0x7fecec122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceb7fe700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec0a12d0 0x7fecec122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7feceaffd700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d5c0 0x7fecec122320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecec0a7190 tx=0x7fece0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecd37fe700 1 -- 172.21.15.89:0/976019176 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0004400 con 0x7fecec09d5c0 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecd37fe700 1 -- 172.21.15.89:0/976019176 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece001f070 con 0x7fecec09d5c0 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece0009550 con 0x7fecec09d5c0 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecd37fe700 1 -- 172.21.15.89:0/976019176 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00038e0 con 0x7fecec09d5c0 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.545+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec09cd10 con 0x7fecec09d5c0 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.546+0000 7fecd37fe700 1 -- 172.21.15.89:0/976019176 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fece0021030 con 0x7fecec09d5c0 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.546+0000 7fecea7fc700 1 -- 172.21.15.89:0/976019176 >> v1:172.21.15.89:6816/126953 conn(0x7fecd802f510 legacy=0x7fecd80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.546+0000 7fecd37fe700 1 -- 172.21.15.89:0/976019176 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fece0040ee0 con 0x7fecec09d5c0 2023-09-30T22:16:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.546+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec11f670 con 0x7fecec09d5c0 2023-09-30T22:16:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.551+0000 7fecd37fe700 1 -- 172.21.15.89:0/976019176 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fece0015080 con 0x7fecec09d5c0 2023-09-30T22:16:49.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.747+0000 7fecea7fc700 1 -- 172.21.15.89:0/976019176 >> v1:172.21.15.89:6816/126953 conn(0x7fecd802f510 legacy=0x7fecd80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:49.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.784+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fecec0747f0 con 0x7fecec09d5c0 2023-09-30T22:16:49.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.785+0000 7fecd37fe700 1 -- 172.21.15.89:0/976019176 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v9) v1 ==== 74+0+70484 (secure 0 0 0) 0x7fece0004560 con 0x7fecec09d5c0 2023-09-30T22:16:49.786 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:49.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 >> v1:172.21.15.89:6816/126953 conn(0x7fecd802f510 legacy=0x7fecd80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:49.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d5c0 msgr2=0x7fecec122320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d5c0 0x7fecec122320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecec0a7190 tx=0x7fece0004750 comp rx=0 tx=0).stop 2023-09-30T22:16:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 shutdown_connections 2023-09-30T22:16:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec0a12d0 0x7fecec122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d5c0 0x7fecec122320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 --2- 172.21.15.89:0/976019176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec0a09c0 0x7fecec122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 >> 172.21.15.89:0/976019176 conn(0x7fecec0a2fe0 msgr2=0x7fecec11c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 shutdown_connections 2023-09-30T22:16:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.787+0000 7fecf1633700 1 -- 172.21.15.89:0/976019176 wait complete. 2023-09-30T22:16:49.800 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"active_gid":14124,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6816","nonce":126953}]},"active_addr":"172.21.15.89:6816/126953","active_change":"2023-09-30T22:16:19.404337+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":21,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1295992953}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":4100667825}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2736305244}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2456756709}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1785021332}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":4019526010}]}]} 2023-09-30T22:16:49.802 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:16:49.802 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:16:49.803 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.922+0000 7faca37c0700 1 Processor -- start 2023-09-30T22:16:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.922+0000 7faca37c0700 1 -- start start 2023-09-30T22:16:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.922+0000 7faca37c0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac9c09d6f0 0x7fac9c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca37c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e000 0x7fac9c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca37c0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c09b120 con 0x7fac9c09e410 2023-09-30T22:16:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca37c0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9c06b460 con 0x7fac9c09d6f0 2023-09-30T22:16:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca37c0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c09a7e0 con 0x7fac9c09e000 2023-09-30T22:16:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca155c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac9c09d6f0 0x7fac9c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca0d5b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e000 0x7fac9c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca155c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac9c09d6f0 0x7fac9c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50964/0 (socket says 172.21.15.89:50964) 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca0d5b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e000 0x7fac9c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40322/0 (socket says 172.21.15.89:40322) 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca155c700 1 -- 172.21.15.89:0/1898595379 learned_addr learned my addr 172.21.15.89:0/1898595379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca0d5b700 1 -- 172.21.15.89:0/1898595379 >> v2:172.21.15.89:3300/0 conn(0x7fac9c09d6f0 msgr2=0x7fac9c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca0d5b700 1 --2- 172.21.15.89:0/1898595379 >> v2:172.21.15.89:3300/0 conn(0x7fac9c09d6f0 0x7fac9c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca0d5b700 1 -- 172.21.15.89:0/1898595379 >> v1:172.21.15.89:6789/0 conn(0x7fac9c09e410 legacy=0x7fac9c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca0d5b700 1 -- 172.21.15.89:0/1898595379 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c09a500 con 0x7fac9c09e000 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca155c700 1 --2- 172.21.15.89:0/1898595379 >> v2:172.21.15.89:3300/0 conn(0x7fac9c09d6f0 0x7fac9c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7faca0d5b700 1 --2- 172.21.15.89:0/1898595379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e000 0x7fac9c1339f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009870 tx=0x7fac8c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c081db961a3b88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7fac93fff700 1 -- 172.21.15.89:0/1898595379 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c011830 con 0x7fac9c09e000 2023-09-30T22:16:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7fac93fff700 1 -- 172.21.15.89:0/1898595379 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c011990 con 0x7fac9c09e000 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.923+0000 7fac93fff700 1 -- 172.21.15.89:0/1898595379 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c011d20 con 0x7fac9c09e000 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.924+0000 7faca37c0700 1 -- 172.21.15.89:0/1898595379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e000 msgr2=0x7fac9c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.924+0000 7faca37c0700 1 --2- 172.21.15.89:0/1898595379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e000 0x7fac9c1339f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009870 tx=0x7fac8c009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.924+0000 7faca37c0700 1 -- 172.21.15.89:0/1898595379 shutdown_connections 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.924+0000 7faca37c0700 1 --2- 172.21.15.89:0/1898595379 >> v2:172.21.15.89:3300/0 conn(0x7fac9c09d6f0 0x7fac9c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.924+0000 7faca37c0700 1 --2- 172.21.15.89:0/1898595379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e000 0x7fac9c1339f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.924+0000 7faca37c0700 1 -- 172.21.15.89:0/1898595379 >> 172.21.15.89:0/1898595379 conn(0x7fac9c066f50 msgr2=0x7fac9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.924+0000 7faca37c0700 1 -- 172.21.15.89:0/1898595379 shutdown_connections 2023-09-30T22:16:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.924+0000 7faca37c0700 1 -- 172.21.15.89:0/1898595379 wait complete. 2023-09-30T22:16:49.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.925+0000 7faca37c0700 1 Processor -- start 2023-09-30T22:16:49.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.925+0000 7faca37c0700 1 -- start start 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca37c0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fac9c09d6f0 0x7fac9c1a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca37c0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09e000 0x7fac9c1a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca37c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e410 0x7fac9c1a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca37c0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1a7a50 con 0x7fac9c09e000 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca37c0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1a8070 con 0x7fac9c09d6f0 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca37c0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1a8ca0 con 0x7fac9c09e410 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca155c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fac9c09d6f0 0x7fac9c1a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca0d5b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09e000 0x7fac9c1a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca1d5d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e410 0x7fac9c1a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca155c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fac9c09d6f0 0x7fac9c1a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36440/0 (socket says 172.21.15.89:36440) 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca1d5d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e410 0x7fac9c1a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:40326/0 (socket says 172.21.15.89:40326) 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca0d5b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09e000 0x7fac9c1a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50980/0 (socket says 172.21.15.89:50980) 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca155c700 1 -- 172.21.15.89:0/3483947597 learned_addr learned my addr 172.21.15.89:0/3483947597 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca1d5d700 1 -- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fac9c09d6f0 msgr2=0x7fac9c1a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca1d5d700 1 --2- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fac9c09d6f0 0x7fac9c1a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca1d5d700 1 -- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09e000 msgr2=0x7fac9c1a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca1d5d700 1 --2- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09e000 0x7fac9c1a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca1d5d700 1 -- 172.21.15.89:0/3483947597 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98009550 con 0x7fac9c09e410 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca0d5b700 1 --2- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09e000 0x7fac9c1a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7faca1d5d700 1 --2- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e410 0x7fac9c1a6910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac9400cb90 tx=0x7fac9400cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7fac91ffb700 1 -- 172.21.15.89:0/3483947597 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94003c00 con 0x7fac9c09e410 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.926+0000 7fac91ffb700 1 -- 172.21.15.89:0/3483947597 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94003d60 con 0x7fac9c09e410 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.927+0000 7fac91ffb700 1 -- 172.21.15.89:0/3483947597 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400aa80 con 0x7fac9c09e410 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.927+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8c0094a0 con 0x7fac9c09e410 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.927+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c1a90f0 con 0x7fac9c09e410 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.927+0000 7fac91ffb700 1 -- 172.21.15.89:0/3483947597 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fac9400abe0 con 0x7fac9c09e410 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.927+0000 7fac91ffb700 1 -- 172.21.15.89:0/3483947597 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fac9403f9f0 con 0x7fac9c09e410 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.927+0000 7faca155c700 1 -- 172.21.15.89:0/3483947597 >> v1:172.21.15.89:6816/126953 conn(0x7fac8002f510 legacy=0x7fac800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.928+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c0a20b0 con 0x7fac9c09e410 2023-09-30T22:16:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:49.932+0000 7fac91ffb700 1 -- 172.21.15.89:0/3483947597 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fac940198f0 con 0x7fac9c09e410 2023-09-30T22:16:50.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.118+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac9c074ea0 con 0x7fac9c09e410 2023-09-30T22:16:50.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.119+0000 7fac91ffb700 1 -- 172.21.15.89:0/3483947597 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+12966 (secure 0 0 0) 0x7fac94019300 con 0x7fac9c09e410 2023-09-30T22:16:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:50.120 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":39,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:49.379224+0000","last_up_change":"2023-09-30T22:16:38.305512+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":129669}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":129669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":129669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":129669}]},"public_addr":"172.21.15.89:6804/129669","cluster_addr":"172.21.15.89:6805/129669","heartbeat_back_addr":"172.21.15.89:6807/129669","heartbeat_front_addr":"172.21.15.89:6806/129669","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":129767}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":129767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":129767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":129767}]},"public_addr":"172.21.15.89:6800/129767","cluster_addr":"172.21.15.89:6801/129767","heartbeat_back_addr":"172.21.15.89:6803/129767","heartbeat_front_addr":"172.21.15.89:6802/129767","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":129877}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":129877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":129877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":129877}]},"public_addr":"172.21.15.89:6808/129877","cluster_addr":"172.21.15.89:6809/129877","heartbeat_back_addr":"172.21.15.89:6811/129877","heartbeat_front_addr":"172.21.15.89:6810/129877","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":130217}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":130217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":130217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":130217}]},"public_addr":"172.21.15.89:6812/130217","cluster_addr":"172.21.15.89:6813/130217","heartbeat_back_addr":"172.21.15.89:6815/130217","heartbeat_front_addr":"172.21.15.89:6814/130217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:16:32.255166+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6817/3539463463":"2023-10-01T22:16:43.298028+0000","172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:50.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 >> v1:172.21.15.89:6816/126953 conn(0x7fac8002f510 legacy=0x7fac800319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:50.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e410 msgr2=0x7fac9c1a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 --2- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e410 0x7fac9c1a6910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac9400cb90 tx=0x7fac9400cea0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 shutdown_connections 2023-09-30T22:16:50.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 --2- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fac9c09e410 0x7fac9c1a6910 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 --2- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fac9c09d6f0 0x7fac9c1a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 --2- 172.21.15.89:0/3483947597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09e000 0x7fac9c1a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 >> 172.21.15.89:0/3483947597 conn(0x7fac9c066f50 msgr2=0x7fac9c1bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 shutdown_connections 2023-09-30T22:16:50.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.121+0000 7faca37c0700 1 -- 172.21.15.89:0/3483947597 wait complete. 2023-09-30T22:16:50.134 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:16:50.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:16:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f7685fd5700 1 Processor -- start 2023-09-30T22:16:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f7685fd5700 1 -- start start 2023-09-30T22:16:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f7685fd5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680133420 0x7f76801337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f7685fd5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133e30 0x7f768012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f7685fd5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7680134260 0x7f76801304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f7685fd5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768006b990 con 0x7f7680134260 2023-09-30T22:16:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f7685fd5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76800a0590 con 0x7f7680133420 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f7685fd5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f768009fb80 con 0x7f7680133e30 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680133420 0x7f76801337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133e30 0x7f768012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7680134260 0x7f76801304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680133420 0x7f76801337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49080/0 (socket says 172.21.15.89:49080) 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767f7fe700 1 -- 172.21.15.89:0/804752923 learned_addr learned my addr 172.21.15.89:0/804752923 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767effd700 1 -- 172.21.15.89:0/804752923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680133420 msgr2=0x7f76801337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767effd700 1 --2- 172.21.15.89:0/804752923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680133420 0x7f76801337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767effd700 1 -- 172.21.15.89:0/804752923 >> v2:172.21.15.89:3300/0 conn(0x7f7680134260 msgr2=0x7f76801304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.255+0000 7f767effd700 1 --2- 172.21.15.89:0/804752923 >> v2:172.21.15.89:3300/0 conn(0x7f7680134260 0x7f76801304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f767effd700 1 -- 172.21.15.89:0/804752923 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76800a0030 con 0x7f7680133e30 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f767f7fe700 1 --2- 172.21.15.89:0/804752923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680133420 0x7f76801337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f767ffff700 1 --2- 172.21.15.89:0/804752923 >> v2:172.21.15.89:3300/0 conn(0x7f7680134260 0x7f76801304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f767effd700 1 --2- 172.21.15.89:0/804752923 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133e30 0x7f768012ff90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7668009840 tx=0x7f7668009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395c13eab2cfc111 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f767e7fc700 1 -- 172.21.15.89:0/804752923 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668011830 con 0x7f7680133e30 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f767e7fc700 1 -- 172.21.15.89:0/804752923 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668011990 con 0x7f7680133e30 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f767e7fc700 1 -- 172.21.15.89:0/804752923 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668011d20 con 0x7f7680133e30 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 -- 172.21.15.89:0/804752923 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133e30 msgr2=0x7f768012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 --2- 172.21.15.89:0/804752923 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133e30 0x7f768012ff90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7668009840 tx=0x7f7668009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 -- 172.21.15.89:0/804752923 shutdown_connections 2023-09-30T22:16:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 --2- 172.21.15.89:0/804752923 >> v2:172.21.15.89:3300/0 conn(0x7f7680134260 0x7f76801304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 --2- 172.21.15.89:0/804752923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680133420 0x7f76801337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 --2- 172.21.15.89:0/804752923 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133e30 0x7f768012ff90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 -- 172.21.15.89:0/804752923 >> 172.21.15.89:0/804752923 conn(0x7f76800a4210 msgr2=0x7f76800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 -- 172.21.15.89:0/804752923 shutdown_connections 2023-09-30T22:16:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.256+0000 7f7685fd5700 1 -- 172.21.15.89:0/804752923 wait complete. 2023-09-30T22:16:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.257+0000 7f7685fd5700 1 Processor -- start 2023-09-30T22:16:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f7685fd5700 1 -- start start 2023-09-30T22:16:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f7685fd5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133420 0x7f768009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f7685fd5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7680133e30 0x7f768009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f7685fd5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680134260 0x7f768009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f7685fd5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768012d5e0 con 0x7f7680133e30 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f7685fd5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7680131490 con 0x7f7680134260 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f7685fd5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f768011fcc0 con 0x7f7680133420 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7680133e30 0x7f768009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680134260 0x7f768009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7680133e30 0x7f768009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38282/0 (socket says 172.21.15.89:38282) 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767effd700 1 -- 172.21.15.89:0/294587162 learned_addr learned my addr 172.21.15.89:0/294587162 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767f7fe700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133420 0x7f768009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767ffff700 1 -- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133420 msgr2=0x7f768009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767ffff700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133420 0x7f768009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767ffff700 1 -- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7680133e30 msgr2=0x7f768009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767ffff700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7680133e30 0x7f768009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767ffff700 1 -- 172.21.15.89:0/294587162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7670009550 con 0x7f7680134260 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767effd700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7680133e30 0x7f768009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767f7fe700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133420 0x7f768009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.258+0000 7f767ffff700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680134260 0x7f768009cf00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f767800efc0 tx=0x7f767800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.259+0000 7f7677fff700 1 -- 172.21.15.89:0/294587162 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767800ca90 con 0x7f7680134260 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.259+0000 7f7677fff700 1 -- 172.21.15.89:0/294587162 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767800f040 con 0x7f7680134260 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.259+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7678009a70 con 0x7f7680134260 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.259+0000 7f7677fff700 1 -- 172.21.15.89:0/294587162 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767800ce30 con 0x7f7680134260 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.259+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76680094a0 con 0x7f7680134260 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.259+0000 7f7677fff700 1 -- 172.21.15.89:0/294587162 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7678014480 con 0x7f7680134260 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.260+0000 7f767f7fe700 1 -- 172.21.15.89:0/294587162 >> v1:172.21.15.89:6816/126953 conn(0x7f766002f510 legacy=0x7f76600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.260+0000 7f7677fff700 1 -- 172.21.15.89:0/294587162 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f7678040300 con 0x7f7680134260 2023-09-30T22:16:50.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.260+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768011fee0 con 0x7f7680134260 2023-09-30T22:16:50.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.264+0000 7f7677fff700 1 -- 172.21.15.89:0/294587162 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7678018070 con 0x7f7680134260 2023-09-30T22:16:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.449+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7680074ea0 con 0x7f7680134260 2023-09-30T22:16:50.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.450+0000 7f7677fff700 1 -- 172.21.15.89:0/294587162 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+12966 (secure 0 0 0) 0x7f7678018070 con 0x7f7680134260 2023-09-30T22:16:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:16:50.451 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":39,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:16:49.379224+0000","last_up_change":"2023-09-30T22:16:38.305512+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":129669}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":129669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":129669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":129669}]},"public_addr":"172.21.15.89:6804/129669","cluster_addr":"172.21.15.89:6805/129669","heartbeat_back_addr":"172.21.15.89:6807/129669","heartbeat_front_addr":"172.21.15.89:6806/129669","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":129767}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":129767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":129767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":129767}]},"public_addr":"172.21.15.89:6800/129767","cluster_addr":"172.21.15.89:6801/129767","heartbeat_back_addr":"172.21.15.89:6803/129767","heartbeat_front_addr":"172.21.15.89:6802/129767","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":129877}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":129877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":129877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":129877}]},"public_addr":"172.21.15.89:6808/129877","cluster_addr":"172.21.15.89:6809/129877","heartbeat_back_addr":"172.21.15.89:6811/129877","heartbeat_front_addr":"172.21.15.89:6810/129877","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":130217}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":130217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":130217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":130217}]},"public_addr":"172.21.15.89:6812/130217","cluster_addr":"172.21.15.89:6813/130217","heartbeat_back_addr":"172.21.15.89:6815/130217","heartbeat_front_addr":"172.21.15.89:6814/130217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:16:32.255166+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6817/3539463463":"2023-10-01T22:16:43.298028+0000","172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:16:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 >> v1:172.21.15.89:6816/126953 conn(0x7f766002f510 legacy=0x7f76600319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680134260 msgr2=0x7f768009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680134260 0x7f768009cf00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f767800efc0 tx=0x7f767800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 shutdown_connections 2023-09-30T22:16:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7680134260 0x7f768009cf00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7680133420 0x7f768009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 --2- 172.21.15.89:0/294587162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7680133e30 0x7f768009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 >> 172.21.15.89:0/294587162 conn(0x7f76800a4210 msgr2=0x7f76800a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 shutdown_connections 2023-09-30T22:16:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.452+0000 7f7685fd5700 1 -- 172.21.15.89:0/294587162 wait complete. 2023-09-30T22:16:50.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:16:50.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:16:50.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:16:50.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:16:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.596+0000 7f27ca937700 1 Processor -- start 2023-09-30T22:16:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.596+0000 7f27ca937700 1 -- start start 2023-09-30T22:16:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f956fc58700 1 Processor -- start 2023-09-30T22:16:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27ca937700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27c409d6f0 0x7f27c409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27ca937700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27c409e000 0x7f27c4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27ca937700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c409f130 con 0x7f27c409e410 2023-09-30T22:16:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27ca937700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c409f520 con 0x7f27c409d6f0 2023-09-30T22:16:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27ca937700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27c409e930 con 0x7f27c409e000 2023-09-30T22:16:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27c8ed4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f27c409e410 0x7f27c4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50420/0 (socket says 172.21.15.89:50420) 2023-09-30T22:16:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27c8ed4700 1 -- 172.21.15.89:0/346296823 learned_addr learned my addr 172.21.15.89:0/346296823 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385746112 0 0) 0x7f27c409f130 con 0x7f27c409e410 2023-09-30T22:16:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ac0035b0 con 0x7f27c409e410 2023-09-30T22:16:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f956fc58700 1 -- start start 2023-09-30T22:16:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f956fc58700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a0a30 0x7f95680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f956fc58700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95680a1340 0x7f9568133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f956fc58700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956809edd0 con 0x7f9568133fa0 2023-09-30T22:16:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f956fc58700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956809e1f0 con 0x7f95680a1340 2023-09-30T22:16:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.597+0000 7f956fc58700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f956809dc50 con 0x7f95680a0a30 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c37fe700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27c409e000 0x7f27c4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1222755073 0 0) 0x7f27ac0035b0 con 0x7f27c409e410 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27c409e000 msgr2=0x7f27c4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27c409e000 0x7f27c4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27c409d6f0 msgr2=0x7f27c409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27c409d6f0 0x7f27c409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c409b2d0 con 0x7f27c409e410 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7f27bc0019f0 con 0x7f27c409e410 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 >> v1:172.21.15.89:6789/0 conn(0x7f27c409e410 legacy=0x7f27c4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27ac003a30 0x7f27ac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27ac004850 0x7f27ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac005350 0x7f27ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f956e1f5700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9568133fa0 0x7f956813ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50432/0 (socket says 172.21.15.89:50432) 2023-09-30T22:16:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f956e1f5700 1 -- 172.21.15.89:0/1351616798 learned_addr learned my addr 172.21.15.89:0/1351616798 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c409e930 con 0x7f27ac004850 2023-09-30T22:16:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac00c190 con 0x7f27ac005350 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac00c350 con 0x7f27ac003a30 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c37fe700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27c409e000 0x7f27c4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c8ed4700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac005350 0x7f27ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c3fff700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27ac003a30 0x7f27ac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c37fe700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27ac004850 0x7f27ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f956d9f4700 1 --2- 172.21.15.89:0/1351616798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a0a30 0x7f95680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f956c9f2700 1 -- 172.21.15.89:0/1351616798 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019110026 0 0) 0x7f956809edd0 con 0x7f9568133fa0 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f956c9f2700 1 -- 172.21.15.89:0/1351616798 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95540035b0 con 0x7f9568133fa0 2023-09-30T22:16:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f956d9f4700 1 -- 172.21.15.89:0/1351616798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95680a1340 msgr2=0x7f9568133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f956d9f4700 1 --2- 172.21.15.89:0/1351616798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95680a1340 0x7f9568133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f956d9f4700 1 -- 172.21.15.89:0/1351616798 >> v1:172.21.15.89:6789/0 conn(0x7f9568133fa0 legacy=0x7f956813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c8ed4700 1 -- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27ac003a30 msgr2=0x7f27ac003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c8ed4700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27ac003a30 0x7f27ac003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c8ed4700 1 -- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27ac004850 msgr2=0x7f27ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c8ed4700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27ac004850 0x7f27ac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.598+0000 7f27c8ed4700 1 -- 172.21.15.89:0/346296823 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0035b0 con 0x7f27ac005350 2023-09-30T22:16:50.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f956d9f4700 1 -- 172.21.15.89:0/1351616798 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680a1fe0 con 0x7f95680a0a30 2023-09-30T22:16:50.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f956d9f4700 1 --2- 172.21.15.89:0/1351616798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a0a30 0x7f95680a0e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9564009870 tx=0x7f9564009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf99b43389a784c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f27c8ed4700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac005350 0x7f27ac00ba10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00a750 tx=0x7f27ac005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6502a3398e37a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00a210 con 0x7f27ac005350 2023-09-30T22:16:50.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc0072f0 con 0x7f27ac005350 2023-09-30T22:16:50.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f956c9f2700 1 -- 172.21.15.89:0/1351616798 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564011830 con 0x7f95680a0a30 2023-09-30T22:16:50.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f956c9f2700 1 -- 172.21.15.89:0/1351616798 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9564011990 con 0x7f95680a0a30 2023-09-30T22:16:50.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f27c2ffd700 1 -- 172.21.15.89:0/346296823 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0076a0 con 0x7f27ac005350 2023-09-30T22:16:50.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.599+0000 7f956c9f2700 1 -- 172.21.15.89:0/1351616798 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564011d20 con 0x7f95680a0a30 2023-09-30T22:16:50.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 -- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac005350 msgr2=0x7f27ac00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac005350 0x7f27ac00ba10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00a750 tx=0x7f27ac005880 comp rx=0 tx=0).stop 2023-09-30T22:16:50.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f956fc58700 1 -- 172.21.15.89:0/1351616798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a0a30 msgr2=0x7f95680a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f956fc58700 1 --2- 172.21.15.89:0/1351616798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a0a30 0x7f95680a0e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9564009870 tx=0x7f9564009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 -- 172.21.15.89:0/346296823 shutdown_connections 2023-09-30T22:16:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac005350 0x7f27ac00ba10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27ac004850 0x7f27ac004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 --2- 172.21.15.89:0/346296823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27ac003a30 0x7f27ac003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 -- 172.21.15.89:0/346296823 >> 172.21.15.89:0/346296823 conn(0x7f27c4066f50 msgr2=0x7f27c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f956fc58700 1 -- 172.21.15.89:0/1351616798 shutdown_connections 2023-09-30T22:16:50.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f956fc58700 1 --2- 172.21.15.89:0/1351616798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a0a30 0x7f95680a0e00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f956fc58700 1 --2- 172.21.15.89:0/1351616798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95680a1340 0x7f9568133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f956fc58700 1 -- 172.21.15.89:0/1351616798 >> 172.21.15.89:0/1351616798 conn(0x7f95680a6550 msgr2=0x7f95680a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 -- 172.21.15.89:0/346296823 shutdown_connections 2023-09-30T22:16:50.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f956fc58700 1 -- 172.21.15.89:0/1351616798 shutdown_connections 2023-09-30T22:16:50.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f956fc58700 1 -- 172.21.15.89:0/1351616798 wait complete. 2023-09-30T22:16:50.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.600+0000 7f27ca937700 1 -- 172.21.15.89:0/346296823 wait complete. 2023-09-30T22:16:50.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f956fc58700 1 Processor -- start 2023-09-30T22:16:50.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f956fc58700 1 -- start start 2023-09-30T22:16:50.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f27ca937700 1 Processor -- start 2023-09-30T22:16:50.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f956fc58700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95680a0a30 0x7f95681c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f956fc58700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a1340 0x7f95681c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f956fc58700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9568133fa0 0x7f95681c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f956fc58700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95681246b0 con 0x7f95680a0a30 2023-09-30T22:16:50.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f956fc58700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95680a0220 con 0x7f9568133fa0 2023-09-30T22:16:50.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f956fc58700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9568124480 con 0x7f95680a1340 2023-09-30T22:16:50.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.601+0000 7f27ca937700 1 -- start start 2023-09-30T22:16:50.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9568133fa0 0x7f95681c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9568133fa0 0x7f95681c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49116/0 (socket says 172.21.15.89:49116) 2023-09-30T22:16:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 -- 172.21.15.89:0/1247593349 learned_addr learned my addr 172.21.15.89:0/1247593349 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27ca937700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac003a30 0x7f27c411dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27ca937700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27ac004850 0x7f27c411e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27ca937700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27ac005350 0x7f27c411e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27ca937700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c41221b0 con 0x7f27ac005350 2023-09-30T22:16:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27ca937700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c41217f0 con 0x7f27ac003a30 2023-09-30T22:16:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27ca937700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27c4120f80 con 0x7f27ac004850 2023-09-30T22:16:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 -- 172.21.15.89:0/1247593349 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a1340 msgr2=0x7f95681c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 --2- 172.21.15.89:0/1247593349 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95680a1340 0x7f95681c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 -- 172.21.15.89:0/1247593349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95680a0a30 msgr2=0x7f95681c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 --2- 172.21.15.89:0/1247593349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95680a0a30 0x7f95681c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 -- 172.21.15.89:0/1247593349 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640094a0 con 0x7f9568133fa0 2023-09-30T22:16:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27c3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac003a30 0x7f27c411dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27c3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac003a30 0x7f27c411dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49126/0 (socket says 172.21.15.89:49126) 2023-09-30T22:16:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27c3fff700 1 -- 172.21.15.89:0/1122598846 learned_addr learned my addr 172.21.15.89:0/1122598846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f956e1f5700 1 --2- 172.21.15.89:0/1247593349 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9568133fa0 0x7f95681c8d00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f956000cba0 tx=0x7f956000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27c3fff700 1 -- 172.21.15.89:0/1122598846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27ac004850 msgr2=0x7f27c411e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27c3fff700 1 --2- 172.21.15.89:0/1122598846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f27ac004850 0x7f27c411e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27c3fff700 1 -- 172.21.15.89:0/1122598846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27ac005350 msgr2=0x7f27c411e7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27c3fff700 1 --2- 172.21.15.89:0/1122598846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27ac005350 0x7f27c411e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f27c3fff700 1 -- 172.21.15.89:0/1122598846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c4120b30 con 0x7f27ac003a30 2023-09-30T22:16:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000ab00 con 0x7f9568133fa0 2023-09-30T22:16:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.602+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000ac60 con 0x7f9568133fa0 2023-09-30T22:16:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95680a25e0 con 0x7f9568133fa0 2023-09-30T22:16:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9568124c70 con 0x7f9568133fa0 2023-09-30T22:16:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f27c3fff700 1 --2- 172.21.15.89:0/1122598846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac003a30 0x7f27c411dd20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27b400e620 tx=0x7f27b400e9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560012400 con 0x7f9568133fa0 2023-09-30T22:16:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4013680 con 0x7f27ac003a30 2023-09-30T22:16:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b40137e0 con 0x7f27ac003a30 2023-09-30T22:16:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4013b90 con 0x7f27ac003a30 2023-09-30T22:16:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9550000fb0 con 0x7f9568133fa0 2023-09-30T22:16:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.603+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f956001b3b0 con 0x7f9568133fa0 2023-09-30T22:16:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.604+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c4072040 con 0x7f27ac003a30 2023-09-30T22:16:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.604+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c411f8e0 con 0x7f27ac003a30 2023-09-30T22:16:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.604+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f956003f9f0 con 0x7f9568133fa0 2023-09-30T22:16:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.604+0000 7f956d9f4700 1 -- 172.21.15.89:0/1247593349 >> v1:172.21.15.89:6816/126953 conn(0x7f954c02f510 legacy=0x7f954c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.604+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 --> v1:172.21.15.89:6800/129767 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f954c03c260 con 0x7f954c038560 2023-09-30T22:16:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.604+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27c409c000 con 0x7f27ac003a30 2023-09-30T22:16:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.604+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f956001be10 con 0x7f9568133fa0 2023-09-30T22:16:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.604+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f27b4013cf0 con 0x7f27ac003a30 2023-09-30T22:16:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.605+0000 7f27c37fe700 1 -- 172.21.15.89:0/1122598846 >> v1:172.21.15.89:6816/126953 conn(0x7f27b002f560 legacy=0x7f27b0031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.605+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f27b403fd10 con 0x7f27ac003a30 2023-09-30T22:16:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.605+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 --> v1:172.21.15.89:6812/130217 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27b003c3b0 con 0x7f27b00386b0 2023-09-30T22:16:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.605+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27b4042320 con 0x7f27ac003a30 2023-09-30T22:16:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.605+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 <== osd.1 v1:172.21.15.89:6800/129767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f954c03c260 con 0x7f954c038560 2023-09-30T22:16:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.607+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 <== osd.3 v1:172.21.15.89:6812/130217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f27b003c3b0 con 0x7f27b00386b0 2023-09-30T22:16:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.608+0000 7fd9e3c46700 1 Processor -- start 2023-09-30T22:16:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.608+0000 7fd9e3c46700 1 -- start start 2023-09-30T22:16:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7f969d8d6700 1 Processor -- start 2023-09-30T22:16:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7f969d8d6700 1 -- start start 2023-09-30T22:16:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7fd9e3c46700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12cf60 0x7fd9dc06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7fd9e3c46700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9dc12f8b0 0x7fd9dc12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7fd9e3c46700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc12cd90 con 0x7fd9dc130260 2023-09-30T22:16:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7fd9e3c46700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc12d330 con 0x7fd9dc12cf60 2023-09-30T22:16:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7fd9e3c46700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1306d0 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7fd9e21e3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd9dc130260 0x7fd9dc136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50440/0 (socket says 172.21.15.89:50440) 2023-09-30T22:16:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7fd9e21e3700 1 -- 172.21.15.89:0/3839722328 learned_addr learned my addr 172.21.15.89:0/3839722328 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7f969d8d6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f969809e1d0 0x7f9698135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7f969d8d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f9698138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7f969d8d6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969809f130 con 0x7f969809d6f0 2023-09-30T22:16:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7fd9e19e2700 1 --2- 172.21.15.89:0/3839722328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12cf60 0x7fd9dc06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7f969d8d6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969809f520 con 0x7f969809e1d0 2023-09-30T22:16:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.609+0000 7f969d8d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969809e930 con 0x7f9698135e50 2023-09-30T22:16:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f9696ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f969809d6f0 0x7f969809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50450/0 (socket says 172.21.15.89:50450) 2023-09-30T22:16:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f9696ffd700 1 -- 172.21.15.89:0/491094517 learned_addr learned my addr 172.21.15.89:0/491094517 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f96977fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f9698138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f96967fc700 1 --2- 172.21.15.89:0/491094517 >> v2:172.21.15.89:3300/0 conn(0x7f969809e1d0 0x7f9698135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f96977fe700 1 -- 172.21.15.89:0/491094517 >> v2:172.21.15.89:3300/0 conn(0x7f969809e1d0 msgr2=0x7f9698135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f96977fe700 1 --2- 172.21.15.89:0/491094517 >> v2:172.21.15.89:3300/0 conn(0x7f969809e1d0 0x7f9698135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f96977fe700 1 -- 172.21.15.89:0/491094517 >> v1:172.21.15.89:6789/0 conn(0x7f969809d6f0 legacy=0x7f969809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f96977fe700 1 -- 172.21.15.89:0/491094517 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969809b2d0 con 0x7f9698135e50 2023-09-30T22:16:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7fd9e19e2700 1 -- 172.21.15.89:0/3839722328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9dc12f8b0 msgr2=0x7fd9dc12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7fd9e19e2700 1 --2- 172.21.15.89:0/3839722328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9dc12f8b0 0x7fd9dc12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7fd9e19e2700 1 -- 172.21.15.89:0/3839722328 >> v1:172.21.15.89:6789/0 conn(0x7fd9dc130260 legacy=0x7fd9dc136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7fd9e19e2700 1 -- 172.21.15.89:0/3839722328 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc071780 con 0x7fd9dc12cf60 2023-09-30T22:16:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7fd9e19e2700 1 --2- 172.21.15.89:0/3839722328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12cf60 0x7fd9dc06f5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009870 tx=0x7fd9d8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9479165ce174302 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7fd9e09e0700 1 -- 172.21.15.89:0/3839722328 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011830 con 0x7fd9dc12cf60 2023-09-30T22:16:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7fd9e09e0700 1 -- 172.21.15.89:0/3839722328 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8011990 con 0x7fd9dc12cf60 2023-09-30T22:16:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7fd9e09e0700 1 -- 172.21.15.89:0/3839722328 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011d20 con 0x7fd9dc12cf60 2023-09-30T22:16:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.610+0000 7f96977fe700 1 --2- 172.21.15.89:0/491094517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f9698138230 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9690009d90 tx=0x7f969000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c36679cbec00b72 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7f9695ffb700 1 -- 172.21.15.89:0/491094517 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000e070 con 0x7f9698135e50 2023-09-30T22:16:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7f9695ffb700 1 -- 172.21.15.89:0/491094517 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969000bbd0 con 0x7f9698135e50 2023-09-30T22:16:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7f9695ffb700 1 -- 172.21.15.89:0/491094517 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690012430 con 0x7f9698135e50 2023-09-30T22:16:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7fd9e3c46700 1 -- 172.21.15.89:0/3839722328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12cf60 msgr2=0x7fd9dc06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7fd9e3c46700 1 --2- 172.21.15.89:0/3839722328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12cf60 0x7fd9dc06f5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009870 tx=0x7fd9d8009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7fd9e3c46700 1 -- 172.21.15.89:0/3839722328 shutdown_connections 2023-09-30T22:16:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7fd9e3c46700 1 --2- 172.21.15.89:0/3839722328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12cf60 0x7fd9dc06f5d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7fd9e3c46700 1 --2- 172.21.15.89:0/3839722328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9dc12f8b0 0x7fd9dc12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7fd9e3c46700 1 -- 172.21.15.89:0/3839722328 >> 172.21.15.89:0/3839722328 conn(0x7fd9dc0a2aa0 msgr2=0x7fd9dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7fd9e3c46700 1 -- 172.21.15.89:0/3839722328 shutdown_connections 2023-09-30T22:16:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.611+0000 7fd9e3c46700 1 -- 172.21.15.89:0/3839722328 wait complete. 2023-09-30T22:16:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7f969d8d6700 1 -- 172.21.15.89:0/491094517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 msgr2=0x7f9698138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7f969d8d6700 1 --2- 172.21.15.89:0/491094517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f9698138230 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9690009d90 tx=0x7f969000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7f969d8d6700 1 -- 172.21.15.89:0/491094517 shutdown_connections 2023-09-30T22:16:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7f969d8d6700 1 --2- 172.21.15.89:0/491094517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f9698138230 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7f969d8d6700 1 --2- 172.21.15.89:0/491094517 >> v2:172.21.15.89:3300/0 conn(0x7f969809e1d0 0x7f9698135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7f969d8d6700 1 -- 172.21.15.89:0/491094517 >> 172.21.15.89:0/491094517 conn(0x7f9698066f50 msgr2=0x7f969809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7fd9e3c46700 1 Processor -- start 2023-09-30T22:16:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7fd9e3c46700 1 -- start start 2023-09-30T22:16:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.612+0000 7fd9e3c46700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9dc12cf60 0x7fd9dc1309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7f969d8d6700 1 -- 172.21.15.89:0/491094517 shutdown_connections 2023-09-30T22:16:50.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e3c46700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12f8b0 0x7fd9dc130fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e3c46700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9dc130260 0x7fd9dc131500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e3c46700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1342d0 con 0x7fd9dc130260 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e3c46700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc129fa0 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e3c46700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc133360 con 0x7fd9dc12cf60 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12f8b0 0x7fd9dc130fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12f8b0 0x7fd9dc130fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49160/0 (socket says 172.21.15.89:49160) 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 -- 172.21.15.89:0/4126990432 learned_addr learned my addr 172.21.15.89:0/4126990432 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e21e3700 1 --2- 172.21.15.89:0/4126990432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9dc130260 0x7fd9dc131500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 -- 172.21.15.89:0/4126990432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9dc12cf60 msgr2=0x7fd9dc1309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 --2- 172.21.15.89:0/4126990432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9dc12cf60 0x7fd9dc1309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 -- 172.21.15.89:0/4126990432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9dc130260 msgr2=0x7fd9dc131500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 --2- 172.21.15.89:0/4126990432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9dc130260 0x7fd9dc131500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 -- 172.21.15.89:0/4126990432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80094a0 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e11e1700 1 --2- 172.21.15.89:0/4126990432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12f8b0 0x7fd9dc130fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc013f40 tx=0x7fd9cc013f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc014d10 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc014e90 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc133620 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc131a40 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.613+0000 7f969d8d6700 1 -- 172.21.15.89:0/491094517 wait complete. 2023-09-30T22:16:50.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.614+0000 7f969d8d6700 1 Processor -- start 2023-09-30T22:16:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.614+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc011310 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.614+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9dc12c620 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.614+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd9cc011470 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fd9cc047800 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7fd9e19e2700 1 -- 172.21.15.89:0/4126990432 >> v1:172.21.15.89:6816/126953 conn(0x7fd9c002f540 legacy=0x7fd9c00319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 --> v1:172.21.15.89:6808/129877 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9c003c290 con 0x7fd9c0038590 2023-09-30T22:16:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9cc04c2c0 con 0x7fd9dc12f8b0 2023-09-30T22:16:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.614+0000 7f969d8d6700 1 -- start start 2023-09-30T22:16:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f969d8d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969809d6f0 0x7f96981d5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f969d8d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f969809e1d0 0x7f96981d5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f969d8d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f969812a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f969d8d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969812dfa0 con 0x7f969809d6f0 2023-09-30T22:16:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f969d8d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96981d4d50 con 0x7f9698135e50 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f969d8d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f969809c700 con 0x7f969809e1d0 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f96967fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f969809e1d0 0x7f96981d5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f96967fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f969809e1d0 0x7f96981d5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38232/0 (socket says 172.21.15.89:38232) 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f96967fc700 1 -- 172.21.15.89:0/2603538942 learned_addr learned my addr 172.21.15.89:0/2603538942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.615+0000 7f96977fe700 1 --2- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f969812a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.616+0000 7f96967fc700 1 -- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 msgr2=0x7f969812a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.616+0000 7f96967fc700 1 --2- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f969812a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.616+0000 7f96967fc700 1 -- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969809d6f0 msgr2=0x7f96981d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.616+0000 7f96967fc700 1 --2- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f969809d6f0 0x7f96981d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.616+0000 7f96967fc700 1 -- 172.21.15.89:0/2603538942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009a70 con 0x7f969809e1d0 2023-09-30T22:16:50.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.616+0000 7f96977fe700 1 --2- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9698135e50 0x7f969812a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:16:50.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.616+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 <== osd.2 v1:172.21.15.89:6808/129877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7fd9c003c290 con 0x7fd9c0038590 2023-09-30T22:16:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.616+0000 7f96967fc700 1 --2- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f969809e1d0 0x7f96981d5670 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f968c00e840 tx=0x7f968c00ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.617+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c013610 con 0x7f969809e1d0 2023-09-30T22:16:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.617+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c013770 con 0x7f969809e1d0 2023-09-30T22:16:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.617+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c013b20 con 0x7f969809e1d0 2023-09-30T22:16:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.617+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96981d5bb0 con 0x7f969809e1d0 2023-09-30T22:16:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.617+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969812c300 con 0x7f969809e1d0 2023-09-30T22:16:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.618+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f969809c000 con 0x7f969809e1d0 2023-09-30T22:16:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.618+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f968c013c80 con 0x7f969809e1d0 2023-09-30T22:16:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.618+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f968c03fa90 con 0x7f969809e1d0 2023-09-30T22:16:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.618+0000 7f9696ffd700 1 -- 172.21.15.89:0/2603538942 >> v1:172.21.15.89:6816/126953 conn(0x7f968402f560 legacy=0x7f9684031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.618+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 --> v1:172.21.15.89:6804/129669 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f968403c330 con 0x7f9684038630 2023-09-30T22:16:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.619+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f968c012370 con 0x7f969809e1d0 2023-09-30T22:16:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.620+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 <== osd.0 v1:172.21.15.89:6804/129669 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f968403c330 con 0x7f9684038630 2023-09-30T22:16:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.626+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 --> v1:172.21.15.89:6800/129767 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9550002d00 con 0x7f954c038560 2023-09-30T22:16:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f955e7fc700 1 -- 172.21.15.89:0/1247593349 <== osd.1 v1:172.21.15.89:6800/129767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3918910179) 0x7f9550002d00 con 0x7f954c038560 2023-09-30T22:16:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 >> v1:172.21.15.89:6800/129767 conn(0x7f954c038560 legacy=0x7f954c03a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 >> v1:172.21.15.89:6816/126953 conn(0x7f954c02f510 legacy=0x7f954c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9568133fa0 msgr2=0x7f95681c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956fc58700 1 --2- 172.21.15.89:0/1247593349 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9568133fa0 0x7f95681c8d00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f956000cba0 tx=0x7f956000cf60 comp rx=0 tx=0).stop 2023-09-30T22:16:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956e1f5700 1 -- 172.21.15.89:0/1247593349 reap_dead start 2023-09-30T22:16:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 shutdown_connections 2023-09-30T22:16:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 >> 172.21.15.89:0/1247593349 conn(0x7f95680a6550 msgr2=0x7f956811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 shutdown_connections 2023-09-30T22:16:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.627+0000 7f956fc58700 1 -- 172.21.15.89:0/1247593349 wait complete. 2023-09-30T22:16:50.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.628+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 --> v1:172.21.15.89:6812/130217 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f27c409c450 con 0x7f27b00386b0 2023-09-30T22:16:50.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27c0ff9700 1 -- 172.21.15.89:0/1122598846 <== osd.3 v1:172.21.15.89:6812/130217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3191560019) 0x7f27bc0070d0 con 0x7f27b00386b0 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 >> v1:172.21.15.89:6812/130217 conn(0x7f27b00386b0 legacy=0x7f27b003ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 >> v1:172.21.15.89:6816/126953 conn(0x7f27b002f560 legacy=0x7f27b0031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac003a30 msgr2=0x7f27c411dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27ca937700 1 --2- 172.21.15.89:0/1122598846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f27ac003a30 0x7f27c411dd20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27b400e620 tx=0x7f27b400e9e0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27c8ed4700 1 -- 172.21.15.89:0/1122598846 reap_dead start 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 shutdown_connections 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 >> 172.21.15.89:0/1122598846 conn(0x7f27c4066f50 msgr2=0x7f27c41309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 shutdown_connections 2023-09-30T22:16:50.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.629+0000 7f27ca937700 1 -- 172.21.15.89:0/1122598846 wait complete. 2023-09-30T22:16:50.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.636+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 --> v1:172.21.15.89:6804/129669 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f969809c490 con 0x7f9684038630 2023-09-30T22:16:50.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.636+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 --> v1:172.21.15.89:6808/129877 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9dc133990 con 0x7fd9c0038590 2023-09-30T22:16:50.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.636+0000 7f967ffff700 1 -- 172.21.15.89:0/2603538942 <== osd.0 v1:172.21.15.89:6804/129669 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 277563570) 0x7f9690003a20 con 0x7f9684038630 2023-09-30T22:16:50.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.636+0000 7fd9d27fc700 1 -- 172.21.15.89:0/4126990432 <== osd.2 v1:172.21.15.89:6808/129877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1985872850) 0x7fd9dc133990 con 0x7fd9c0038590 2023-09-30T22:16:50.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 >> v1:172.21.15.89:6804/129669 conn(0x7f9684038630 legacy=0x7f968403aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 >> v1:172.21.15.89:6816/126953 conn(0x7f968402f560 legacy=0x7f9684031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:50.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 >> v1:172.21.15.89:6808/129877 conn(0x7fd9c0038590 legacy=0x7fd9c003aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f969809e1d0 msgr2=0x7f96981d5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f969d8d6700 1 --2- 172.21.15.89:0/2603538942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f969809e1d0 0x7f96981d5670 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f968c00e840 tx=0x7f968c00ec00 comp rx=0 tx=0).stop 2023-09-30T22:16:50.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f96977fe700 1 -- 172.21.15.89:0/2603538942 reap_dead start 2023-09-30T22:16:50.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 shutdown_connections 2023-09-30T22:16:50.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 >> 172.21.15.89:0/2603538942 conn(0x7f9698066f50 msgr2=0x7f9698071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 shutdown_connections 2023-09-30T22:16:50.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.637+0000 7f969d8d6700 1 -- 172.21.15.89:0/2603538942 wait complete. 2023-09-30T22:16:50.648 INFO:teuthology.orchestra.run.smithi089.stdout:133143986180 2023-09-30T22:16:50.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.641+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 >> v1:172.21.15.89:6816/126953 conn(0x7fd9c002f540 legacy=0x7fd9c00319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.641+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12f8b0 msgr2=0x7fd9dc130fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.641+0000 7fd9e3c46700 1 --2- 172.21.15.89:0/4126990432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9dc12f8b0 0x7fd9dc130fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc013f40 tx=0x7fd9cc013f70 comp rx=0 tx=0).stop 2023-09-30T22:16:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.641+0000 7fd9e21e3700 1 -- 172.21.15.89:0/4126990432 reap_dead start 2023-09-30T22:16:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.641+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 shutdown_connections 2023-09-30T22:16:50.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.641+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 >> 172.21.15.89:0/4126990432 conn(0x7fd9dc0a2aa0 msgr2=0x7fd9dc12e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.641+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 shutdown_connections 2023-09-30T22:16:50.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.641+0000 7fd9e3c46700 1 -- 172.21.15.89:0/4126990432 wait complete. 2023-09-30T22:16:50.651 INFO:teuthology.orchestra.run.smithi089.stdout:137438953475 2023-09-30T22:16:50.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:50.655 INFO:teuthology.orchestra.run.smithi089.stdout:124554051588 2023-09-30T22:16:50.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:50.657 INFO:teuthology.orchestra.run.smithi089.stdout:133143986179 2023-09-30T22:16:50.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.787+0000 7fb4738f7700 1 Processor -- start 2023-09-30T22:16:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.787+0000 7fb4738f7700 1 -- start start 2023-09-30T22:16:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb4738f7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09e930 0x7fb46c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb4738f7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb46c09f240 0x7fb46c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb4738f7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09f650 0x7fb46c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb4738f7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c070940 con 0x7fb46c09f240 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb4738f7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c06fd60 con 0x7fb46c09e930 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb4738f7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c06fa80 con 0x7fb46c09f650 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb471693700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09e930 0x7fb46c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb471693700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09e930 0x7fb46c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49190/0 (socket says 172.21.15.89:49190) 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb471693700 1 -- 172.21.15.89:0/2142048168 learned_addr learned my addr 172.21.15.89:0/2142048168 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb470e92700 1 --2- 172.21.15.89:0/2142048168 >> v2:172.21.15.89:3300/0 conn(0x7fb46c09f240 0x7fb46c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.788+0000 7fb471e94700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09f650 0x7fb46c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb470e92700 1 -- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09f650 msgr2=0x7fb46c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb470e92700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09f650 0x7fb46c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb470e92700 1 -- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09e930 msgr2=0x7fb46c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb470e92700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09e930 0x7fb46c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb470e92700 1 -- 172.21.15.89:0/2142048168 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c09fb60 con 0x7fb46c09f240 2023-09-30T22:16:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb471e94700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09f650 0x7fb46c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb471693700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09e930 0x7fb46c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb470e92700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f240 0x7fb46c1317f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00f9b0 tx=0x7fb45c00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9177293f068f89f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb463fff700 1 -- 172.21.15.89:0/2142048168 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00f460 con 0x7fb46c09f240 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb463fff700 1 -- 172.21.15.89:0/2142048168 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00fe50 con 0x7fb46c09f240 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.789+0000 7fb463fff700 1 -- 172.21.15.89:0/2142048168 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c009b10 con 0x7fb46c09f240 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 -- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f240 msgr2=0x7fb46c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f240 0x7fb46c1317f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00f9b0 tx=0x7fb45c00fcc0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 -- 172.21.15.89:0/2142048168 shutdown_connections 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09f650 0x7fb46c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09e930 0x7fb46c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 --2- 172.21.15.89:0/2142048168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f240 0x7fb46c1317f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 -- 172.21.15.89:0/2142048168 >> 172.21.15.89:0/2142048168 conn(0x7fb46c0a4210 msgr2=0x7fb46c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 -- 172.21.15.89:0/2142048168 shutdown_connections 2023-09-30T22:16:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.790+0000 7fb4738f7700 1 -- 172.21.15.89:0/2142048168 wait complete. 2023-09-30T22:16:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.791+0000 7f3cdb18b700 1 Processor -- start 2023-09-30T22:16:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.791+0000 7fb4738f7700 1 Processor -- start 2023-09-30T22:16:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.791+0000 7f3cdb18b700 1 -- start start 2023-09-30T22:16:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.791+0000 7fb4738f7700 1 -- start start 2023-09-30T22:16:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cdb18b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd4131810 0x7f3cd4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb4738f7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09e930 0x7fb46c11d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb4738f7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09f240 0x7fb46c11db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb4738f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f650 0x7fb46c11e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb4738f7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c11f0e0 con 0x7fb46c09f650 2023-09-30T22:16:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb4738f7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c11f700 con 0x7fb46c09f240 2023-09-30T22:16:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb4738f7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c130440 con 0x7fb46c09e930 2023-09-30T22:16:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cdb18b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 0x7f3cd4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cdb18b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3cd409a3b0 0x7f3cd4138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cdb18b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd409b580 con 0x7f3cd409a3b0 2023-09-30T22:16:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cdb18b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41313c0 con 0x7f3cd4099a40 2023-09-30T22:16:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cdb18b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd406e8c0 con 0x7f3cd4131810 2023-09-30T22:16:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cd9728700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3cd409a3b0 0x7f3cd4138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cd9728700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3cd409a3b0 0x7f3cd4138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38346/0 (socket says 172.21.15.89:38346) 2023-09-30T22:16:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cd9728700 1 -- 172.21.15.89:0/2074111730 learned_addr learned my addr 172.21.15.89:0/2074111730 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cd8f27700 1 --2- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd4131810 0x7f3cd4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7f3cd3fff700 1 --2- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 0x7f3cd4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb471693700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09e930 0x7fb46c11d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb471693700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09e930 0x7fb46c11d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38260/0 (socket says 172.21.15.89:38260) 2023-09-30T22:16:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb471693700 1 -- 172.21.15.89:0/1728382535 learned_addr learned my addr 172.21.15.89:0/1728382535 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.792+0000 7fb471e94700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f650 0x7fb46c11e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb470e92700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09f240 0x7fb46c11db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd8f27700 1 -- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 msgr2=0x7f3cd4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd8f27700 1 --2- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 0x7f3cd4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd8f27700 1 -- 172.21.15.89:0/2074111730 >> v2:172.21.15.89:3300/0 conn(0x7f3cd409a3b0 msgr2=0x7f3cd4138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd8f27700 1 --2- 172.21.15.89:0/2074111730 >> v2:172.21.15.89:3300/0 conn(0x7f3cd409a3b0 0x7f3cd4138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd8f27700 1 -- 172.21.15.89:0/2074111730 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd409a820 con 0x7f3cd4131810 2023-09-30T22:16:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb470e92700 1 -- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09e930 msgr2=0x7fb46c11d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb470e92700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09e930 0x7fb46c11d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb470e92700 1 -- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f650 msgr2=0x7fb46c11e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb470e92700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f650 0x7fb46c11e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd8f27700 1 --2- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd4131810 0x7f3cd4131be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009840 tx=0x7f3cc8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1cdb81b0ae17bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb470e92700 1 -- 172.21.15.89:0/1728382535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468009550 con 0x7fb46c09f240 2023-09-30T22:16:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd37fe700 1 -- 172.21.15.89:0/2074111730 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8011830 con 0x7f3cd4131810 2023-09-30T22:16:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd37fe700 1 -- 172.21.15.89:0/2074111730 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8011990 con 0x7f3cd4131810 2023-09-30T22:16:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7f3cd37fe700 1 -- 172.21.15.89:0/2074111730 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8011d20 con 0x7f3cd4131810 2023-09-30T22:16:50.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb470e92700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09f240 0x7fb46c11db40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb45c019000 tx=0x7fb45c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb461ffb700 1 -- 172.21.15.89:0/1728382535 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c009dc0 con 0x7fb46c09f240 2023-09-30T22:16:50.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb461ffb700 1 -- 172.21.15.89:0/1728382535 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c0036a0 con 0x7fb46c09f240 2023-09-30T22:16:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb464009a70 con 0x7fb46c09f240 2023-09-30T22:16:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.793+0000 7fb461ffb700 1 -- 172.21.15.89:0/1728382535 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c003a10 con 0x7fb46c09f240 2023-09-30T22:16:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.794+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c00f610 con 0x7fb46c09f240 2023-09-30T22:16:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.794+0000 7f3cdb18b700 1 -- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd4131810 msgr2=0x7f3cd4131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.794+0000 7f3cdb18b700 1 --2- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd4131810 0x7f3cd4131be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009840 tx=0x7f3cc8009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.794+0000 7f3cdb18b700 1 -- 172.21.15.89:0/2074111730 shutdown_connections 2023-09-30T22:16:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.794+0000 7f3cdb18b700 1 --2- 172.21.15.89:0/2074111730 >> v2:172.21.15.89:3300/0 conn(0x7f3cd409a3b0 0x7f3cd4138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.794+0000 7f3cdb18b700 1 --2- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd4131810 0x7f3cd4131be0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009840 tx=0x7f3cc8009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.794+0000 7f3cdb18b700 1 --2- 172.21.15.89:0/2074111730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 0x7f3cd4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.794+0000 7f3cdb18b700 1 -- 172.21.15.89:0/2074111730 >> 172.21.15.89:0/2074111730 conn(0x7f3cd40a2aa0 msgr2=0x7f3cd40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.795+0000 7fb461ffb700 1 -- 172.21.15.89:0/1728382535 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb45c01e040 con 0x7fb46c09f240 2023-09-30T22:16:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.795+0000 7fb461ffb700 1 -- 172.21.15.89:0/1728382535 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fb45c044ea0 con 0x7fb46c09f240 2023-09-30T22:16:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.795+0000 7fb471693700 1 -- 172.21.15.89:0/1728382535 >> v1:172.21.15.89:6816/126953 conn(0x7fb45002f540 legacy=0x7fb4500319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.795+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c071d20 con 0x7fb46c09f240 2023-09-30T22:16:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.796+0000 7f3cdb18b700 1 -- 172.21.15.89:0/2074111730 shutdown_connections 2023-09-30T22:16:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.800+0000 7f3cdb18b700 1 -- 172.21.15.89:0/2074111730 wait complete. 2023-09-30T22:16:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.800+0000 7fb461ffb700 1 -- 172.21.15.89:0/1728382535 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb45c01c080 con 0x7fb46c09f240 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.800+0000 7f3cdb18b700 1 Processor -- start 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cdb18b700 1 -- start start 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cdb18b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 0x7f3cd41b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cdb18b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cd409a3b0 0x7f3cd41b8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cdb18b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd412a9b0 0x7f3cd412ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cdb18b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41bae80 con 0x7f3cd409a3b0 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cdb18b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41c7ee0 con 0x7f3cd4099a40 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cdb18b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41b7810 con 0x7f3cd412a9b0 2023-09-30T22:16:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cd9728700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd412a9b0 0x7f3cd412ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cd9728700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd412a9b0 0x7f3cd412ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38270/0 (socket says 172.21.15.89:38270) 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cd9728700 1 -- 172.21.15.89:0/3658215246 learned_addr learned my addr 172.21.15.89:0/3658215246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cd3fff700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cd409a3b0 0x7f3cd41b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.801+0000 7f3cd8f27700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 0x7f3cd41b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd9728700 1 -- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 msgr2=0x7f3cd41b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd9728700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 0x7f3cd41b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd9728700 1 -- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cd409a3b0 msgr2=0x7f3cd41b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd9728700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cd409a3b0 0x7f3cd41b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd9728700 1 -- 172.21.15.89:0/3658215246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80094a0 con 0x7f3cd412a9b0 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd9728700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd412a9b0 0x7f3cd412ae30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00efc0 tx=0x7f3ccc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd17fa700 1 -- 172.21.15.89:0/3658215246 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc003c20 con 0x7f3cd412a9b0 2023-09-30T22:16:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd17fa700 1 -- 172.21.15.89:0/3658215246 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc00f040 con 0x7f3cd412a9b0 2023-09-30T22:16:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd40a1dd0 con 0x7f3cd412a9b0 2023-09-30T22:16:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cd17fa700 1 -- 172.21.15.89:0/3658215246 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc009e60 con 0x7f3cd412a9b0 2023-09-30T22:16:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.802+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd41b94f0 con 0x7f3cd412a9b0 2023-09-30T22:16:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.803+0000 7f3cd17fa700 1 -- 172.21.15.89:0/3658215246 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3ccc00c880 con 0x7f3cd412a9b0 2023-09-30T22:16:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.803+0000 7f3cd17fa700 1 -- 172.21.15.89:0/3658215246 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f3ccc03fe20 con 0x7f3cd412a9b0 2023-09-30T22:16:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.803+0000 7f3cd8f27700 1 -- 172.21.15.89:0/3658215246 >> v1:172.21.15.89:6816/126953 conn(0x7f3cb802f510 legacy=0x7f3cb80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.803+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd41b9030 con 0x7f3cd412a9b0 2023-09-30T22:16:50.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.808+0000 7f3cd17fa700 1 -- 172.21.15.89:0/3658215246 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ccc01c360 con 0x7f3cd412a9b0 2023-09-30T22:16:50.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.827+0000 7f6ebb7bc700 1 Processor -- start 2023-09-30T22:16:50.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6ebb7bc700 1 -- start start 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6ebb7bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6ebb7bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6eb4139170 0x7f6eb413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6ebb7bc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb409b120 con 0x7f6eb409d6f0 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6ebb7bc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb406b460 con 0x7f6eb4139170 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6ebb7bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb409a7e0 con 0x7f6eb409e1d0 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6eb8d57700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6eb9558700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6eb409d6f0 0x7f6eb409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50458/0 (socket says 172.21.15.89:50458) 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.828+0000 7f6eb9558700 1 -- 172.21.15.89:0/1504551290 learned_addr learned my addr 172.21.15.89:0/1504551290 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eb9d59700 1 --2- 172.21.15.89:0/1504551290 >> v2:172.21.15.89:3300/0 conn(0x7f6eb4139170 0x7f6eb413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eb8d57700 1 -- 172.21.15.89:0/1504551290 >> v2:172.21.15.89:3300/0 conn(0x7f6eb4139170 msgr2=0x7f6eb413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eb8d57700 1 --2- 172.21.15.89:0/1504551290 >> v2:172.21.15.89:3300/0 conn(0x7f6eb4139170 0x7f6eb413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eb8d57700 1 -- 172.21.15.89:0/1504551290 >> v1:172.21.15.89:6789/0 conn(0x7f6eb409d6f0 legacy=0x7f6eb409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eb8d57700 1 -- 172.21.15.89:0/1504551290 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb409a500 con 0x7f6eb409e1d0 2023-09-30T22:16:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eb8d57700 1 --2- 172.21.15.89:0/1504551290 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb4138c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009870 tx=0x7f6ea4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed20c93ad277216d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eabfff700 1 -- 172.21.15.89:0/1504551290 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011830 con 0x7f6eb409e1d0 2023-09-30T22:16:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eabfff700 1 -- 172.21.15.89:0/1504551290 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4011990 con 0x7f6eb409e1d0 2023-09-30T22:16:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.829+0000 7f6eabfff700 1 -- 172.21.15.89:0/1504551290 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011d20 con 0x7f6eb409e1d0 2023-09-30T22:16:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1504551290 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 msgr2=0x7f6eb4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6ebb7bc700 1 --2- 172.21.15.89:0/1504551290 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb4138c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009870 tx=0x7f6ea4009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1504551290 shutdown_connections 2023-09-30T22:16:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6ebb7bc700 1 --2- 172.21.15.89:0/1504551290 >> v2:172.21.15.89:3300/0 conn(0x7f6eb4139170 0x7f6eb413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6ebb7bc700 1 --2- 172.21.15.89:0/1504551290 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb4138c30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1504551290 >> 172.21.15.89:0/1504551290 conn(0x7f6eb4066f50 msgr2=0x7f6eb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1504551290 shutdown_connections 2023-09-30T22:16:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1504551290 wait complete. 2023-09-30T22:16:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6a9bc55700 1 Processor -- start 2023-09-30T22:16:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.830+0000 7f6a9bc55700 1 -- start start 2023-09-30T22:16:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a9bc55700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a940954c0 0x7f6a94095890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a9bc55700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6a94099680 0x7f6a94099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6ebb7bc700 1 Processor -- start 2023-09-30T22:16:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6ebb7bc700 1 -- start start 2023-09-30T22:16:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a9bc55700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9409c280 con 0x7f6a94099ff0 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a9bc55700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9409aaf0 con 0x7f6a94099680 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a9bc55700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9409b960 con 0x7f6a940954c0 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a999f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a940954c0 0x7f6a94095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a999f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a940954c0 0x7f6a94095890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49228/0 (socket says 172.21.15.89:49228) 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a999f1700 1 -- 172.21.15.89:0/682336744 learned_addr learned my addr 172.21.15.89:0/682336744 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a999f1700 1 -- 172.21.15.89:0/682336744 >> v2:172.21.15.89:3300/0 conn(0x7f6a94099680 msgr2=0x7f6a94099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a999f1700 1 --2- 172.21.15.89:0/682336744 >> v2:172.21.15.89:3300/0 conn(0x7f6a94099680 0x7f6a94099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a999f1700 1 -- 172.21.15.89:0/682336744 >> v1:172.21.15.89:6789/0 conn(0x7f6a94099ff0 legacy=0x7f6a94137590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a999f1700 1 -- 172.21.15.89:0/682336744 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9409a7b0 con 0x7f6a940954c0 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6a999f1700 1 --2- 172.21.15.89:0/682336744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a940954c0 0x7f6a94095890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a900098f0 tx=0x7f6a90009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8299d9a4fadaaa34 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6ebb7bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb409d6f0 0x7f6eb41c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6ebb7bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb41c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6ebb7bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6eb4139170 0x7f6eb4121860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6ebb7bc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4122d20 con 0x7f6eb409d6f0 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6ebb7bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4123660 con 0x7f6eb4139170 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.831+0000 7f6ebb7bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb409c710 con 0x7f6eb409e1d0 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb8d57700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb41c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb8d57700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb41c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38274/0 (socket says 172.21.15.89:38274) 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb8d57700 1 -- 172.21.15.89:0/1467482208 learned_addr learned my addr 172.21.15.89:0/1467482208 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb9d59700 1 --2- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6eb4139170 0x7f6eb4121860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb9d59700 1 -- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 msgr2=0x7f6eb41c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb9d59700 1 --2- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb41c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb9d59700 1 -- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb409d6f0 msgr2=0x7f6eb41c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb9d59700 1 --2- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb409d6f0 0x7f6eb41c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb9d59700 1 -- 172.21.15.89:0/1467482208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40094a0 con 0x7f6eb4139170 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6a989ef700 1 -- 172.21.15.89:0/682336744 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90004230 con 0x7f6a940954c0 2023-09-30T22:16:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6a989ef700 1 -- 172.21.15.89:0/682336744 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90004390 con 0x7f6a940954c0 2023-09-30T22:16:50.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6a989ef700 1 -- 172.21.15.89:0/682336744 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900036a0 con 0x7f6a940954c0 2023-09-30T22:16:50.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6eb9d59700 1 --2- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6eb4139170 0x7f6eb4121860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6eac007d90 tx=0x7f6eac00e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6ea9ffb700 1 -- 172.21.15.89:0/1467482208 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac018070 con 0x7f6eb4139170 2023-09-30T22:16:50.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6ea9ffb700 1 -- 172.21.15.89:0/1467482208 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac00ebe0 con 0x7f6eb4139170 2023-09-30T22:16:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb4125380 con 0x7f6eb4139170 2023-09-30T22:16:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6ea9ffb700 1 -- 172.21.15.89:0/1467482208 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac003a20 con 0x7f6eb4139170 2023-09-30T22:16:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6a9bc55700 1 -- 172.21.15.89:0/682336744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a940954c0 msgr2=0x7f6a94095890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.832+0000 7f6a9bc55700 1 --2- 172.21.15.89:0/682336744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a940954c0 0x7f6a94095890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a900098f0 tx=0x7f6a90009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.833+0000 7f6a9bc55700 1 -- 172.21.15.89:0/682336744 shutdown_connections 2023-09-30T22:16:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.833+0000 7f6a9bc55700 1 --2- 172.21.15.89:0/682336744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a940954c0 0x7f6a94095890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.833+0000 7f6a9bc55700 1 --2- 172.21.15.89:0/682336744 >> v2:172.21.15.89:3300/0 conn(0x7f6a94099680 0x7f6a94099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.833+0000 7f6a9bc55700 1 -- 172.21.15.89:0/682336744 >> 172.21.15.89:0/682336744 conn(0x7f6a940a2360 msgr2=0x7f6a940a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.833+0000 7f6ea9ffb700 1 -- 172.21.15.89:0/1467482208 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6eac003b80 con 0x7f6eb4139170 2023-09-30T22:16:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.833+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb41250a0 con 0x7f6eb4139170 2023-09-30T22:16:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.833+0000 7f6a9bc55700 1 -- 172.21.15.89:0/682336744 shutdown_connections 2023-09-30T22:16:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.834+0000 7f6a9bc55700 1 -- 172.21.15.89:0/682336744 wait complete. 2023-09-30T22:16:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.833+0000 7f6eb9558700 1 -- 172.21.15.89:0/1467482208 >> v1:172.21.15.89:6816/126953 conn(0x7f6e9802f560 legacy=0x7f6e98031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.834+0000 7f6ea9ffb700 1 -- 172.21.15.89:0/1467482208 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f6eac004020 con 0x7f6eb4139170 2023-09-30T22:16:50.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.834+0000 7f6a9bc55700 1 Processor -- start 2023-09-30T22:16:50.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.834+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb4124cc0 con 0x7f6eb4139170 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.834+0000 7f6a9bc55700 1 -- start start 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a9bc55700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a940954c0 0x7f6a94137340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a9bc55700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a94099680 0x7f6a94137880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a9bc55700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a94099ff0 0x7f6a94137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a9bc55700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94138430 con 0x7f6a94099680 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a9bc55700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941392f0 con 0x7f6a94099ff0 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a9bc55700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9409b000 con 0x7f6a940954c0 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a999f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a940954c0 0x7f6a94137340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a991f0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a94099680 0x7f6a94137880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a999f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a940954c0 0x7f6a94137340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38288/0 (socket says 172.21.15.89:38288) 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a999f1700 1 -- 172.21.15.89:0/3735712487 learned_addr learned my addr 172.21.15.89:0/3735712487 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a991f0700 1 -- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a940954c0 msgr2=0x7f6a94137340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a991f0700 1 --2- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a940954c0 0x7f6a94137340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a991f0700 1 -- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a94099ff0 msgr2=0x7f6a94137dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a991f0700 1 --2- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a94099ff0 0x7f6a94137dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a991f0700 1 -- 172.21.15.89:0/3735712487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a90009550 con 0x7f6a94099680 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.835+0000 7f6a999f1700 1 --2- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a940954c0 0x7f6a94137340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.836+0000 7f6a991f0700 1 --2- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a94099680 0x7f6a94137880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400b450 tx=0x7f6a8400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.836+0000 7f6a8a7fc700 1 -- 172.21.15.89:0/3735712487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84011830 con 0x7f6a94099680 2023-09-30T22:16:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.836+0000 7f6a8a7fc700 1 -- 172.21.15.89:0/3735712487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a84011990 con 0x7f6a94099680 2023-09-30T22:16:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.836+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a941390b0 con 0x7f6a94099680 2023-09-30T22:16:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.836+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a94134dc0 con 0x7f6a94099680 2023-09-30T22:16:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.836+0000 7f6a8a7fc700 1 -- 172.21.15.89:0/3735712487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84011d40 con 0x7f6a94099680 2023-09-30T22:16:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.837+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c0052e0 con 0x7f6a94099680 2023-09-30T22:16:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.837+0000 7f6a8a7fc700 1 -- 172.21.15.89:0/3735712487 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6a8401a400 con 0x7f6a94099680 2023-09-30T22:16:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.838+0000 7f6a999f1700 1 -- 172.21.15.89:0/3735712487 >> v1:172.21.15.89:6816/126953 conn(0x7f6a7802f510 legacy=0x7f6a780319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.838+0000 7f6a8a7fc700 1 -- 172.21.15.89:0/3735712487 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f6a8403fe90 con 0x7f6a94099680 2023-09-30T22:16:50.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.840+0000 7f6ea9ffb700 1 -- 172.21.15.89:0/1467482208 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6eac014420 con 0x7f6eb4139170 2023-09-30T22:16:50.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.842+0000 7f6a8a7fc700 1 -- 172.21.15.89:0/3735712487 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a84022af0 con 0x7f6a94099680 2023-09-30T22:16:50.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:50.996+0000 7fb471693700 1 -- 172.21.15.89:0/1728382535 >> v1:172.21.15.89:6816/126953 conn(0x7fb45002f540 legacy=0x7fb4500319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:51.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.004+0000 7f3cd8f27700 1 -- 172.21.15.89:0/3658215246 >> v1:172.21.15.89:6816/126953 conn(0x7f3cb802f510 legacy=0x7f3cb80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.025+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb46c074ea0 con 0x7fb46c09f240 2023-09-30T22:16:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.026+0000 7fb461ffb700 1 -- 172.21.15.89:0/1728382535 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb45c00e4e0 con 0x7fb46c09f240 2023-09-30T22:16:51.027 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:51.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 >> v1:172.21.15.89:6816/126953 conn(0x7fb45002f540 legacy=0x7fb4500319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09f240 msgr2=0x7fb46c11db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09f240 0x7fb46c11db40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb45c019000 tx=0x7fb45c004750 comp rx=0 tx=0).stop 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 shutdown_connections 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46c09f650 0x7fb46c11e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c09e930 0x7fb46c11d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 --2- 172.21.15.89:0/1728382535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46c09f240 0x7fb46c11db40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 >> 172.21.15.89:0/1728382535 conn(0x7fb46c0a4210 msgr2=0x7fb46c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 shutdown_connections 2023-09-30T22:16:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.028+0000 7fb4738f7700 1 -- 172.21.15.89:0/1728382535 wait complete. 2023-09-30T22:16:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.030+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3cd4071d20 con 0x7f3cd412a9b0 2023-09-30T22:16:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.030+0000 7f3cd17fa700 1 -- 172.21.15.89:0/3658215246 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ccc014e60 con 0x7f3cd412a9b0 2023-09-30T22:16:51.031 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:16:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.032+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 >> v1:172.21.15.89:6816/126953 conn(0x7f3cb802f510 legacy=0x7f3cb80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.032+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd412a9b0 msgr2=0x7f3cd412ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.032+0000 7f3cdb18b700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd412a9b0 0x7f3cd412ae30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00efc0 tx=0x7f3ccc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.032+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 shutdown_connections 2023-09-30T22:16:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.032+0000 7f3cdb18b700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cd412a9b0 0x7f3cd412ae30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.032+0000 7f3cdb18b700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cd4099a40 0x7f3cd41b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.033+0000 7f3cdb18b700 1 --2- 172.21.15.89:0/3658215246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cd409a3b0 0x7f3cd41b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.033+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 >> 172.21.15.89:0/3658215246 conn(0x7f3cd40a2aa0 msgr2=0x7f3cd40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:51.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.033+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 shutdown_connections 2023-09-30T22:16:51.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.033+0000 7f3cdb18b700 1 -- 172.21.15.89:0/3658215246 wait complete. 2023-09-30T22:16:51.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.034+0000 7f6eb9558700 1 -- 172.21.15.89:0/1467482208 >> v1:172.21.15.89:6816/126953 conn(0x7f6e9802f560 legacy=0x7f6e98031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:51.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.034+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6eb4074ea0 con 0x7f6eb4139170 2023-09-30T22:16:51.036 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:16:51.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.034+0000 7f6ea9ffb700 1 -- 172.21.15.89:0/1467482208 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6eac01c370 con 0x7f6eb4139170 2023-09-30T22:16:51.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.036+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6a7c005860 con 0x7f6a94099680 2023-09-30T22:16:51.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.037+0000 7f6a8a7fc700 1 -- 172.21.15.89:0/3735712487 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a84028020 con 0x7f6a94099680 2023-09-30T22:16:51.038 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:51.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 >> v1:172.21.15.89:6816/126953 conn(0x7f6e9802f560 legacy=0x7f6e98031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:51.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6eb4139170 msgr2=0x7f6eb4121860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:51.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 --2- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6eb4139170 0x7f6eb4121860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6eac007d90 tx=0x7f6eac00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 shutdown_connections 2023-09-30T22:16:51.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 --2- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6eb4139170 0x7f6eb4121860 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 --2- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb409d6f0 0x7f6eb41c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 --2- 172.21.15.89:0/1467482208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6eb409e1d0 0x7f6eb41c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 >> 172.21.15.89:0/1467482208 conn(0x7f6eb4066f50 msgr2=0x7f6eb406f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:51.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 shutdown_connections 2023-09-30T22:16:51.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.038+0000 7f6ebb7bc700 1 -- 172.21.15.89:0/1467482208 wait complete. 2023-09-30T22:16:51.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a999f1700 1 -- 172.21.15.89:0/3735712487 >> v1:172.21.15.89:6816/126953 conn(0x7f6a7802f510 legacy=0x7f6a780319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:51.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 >> v1:172.21.15.89:6816/126953 conn(0x7f6a7802f510 legacy=0x7f6a780319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:51.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a94099680 msgr2=0x7f6a94137880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:51.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 --2- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a94099680 0x7f6a94137880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400b450 tx=0x7f6a8400b760 comp rx=0 tx=0).stop 2023-09-30T22:16:51.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 shutdown_connections 2023-09-30T22:16:51.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 --2- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a94099ff0 0x7f6a94137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 --2- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a940954c0 0x7f6a94137340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 --2- 172.21.15.89:0/3735712487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a94099680 0x7f6a94137880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:51.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 >> 172.21.15.89:0/3735712487 conn(0x7f6a940a2360 msgr2=0x7f6a94096af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:51.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 shutdown_connections 2023-09-30T22:16:51.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:51.039+0000 7f6a9bc55700 1 -- 172.21.15.89:0/3735712487 wait complete. 2023-09-30T22:16:51.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:16:51.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:16:51.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:16:51.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:16:52.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:52.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:52.056 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:52.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:52.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.201+0000 7f42d715b700 1 Processor -- start 2023-09-30T22:16:52.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.201+0000 7f42d715b700 1 -- start start 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.202+0000 7f42d715b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42d006e570 0x7f42d006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.202+0000 7f42d715b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d009eca0 0x7f42d006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.202+0000 7f42d715b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d0070ee0 con 0x7f42d0134610 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.202+0000 7f42d715b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d0071480 con 0x7f42d006e570 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.202+0000 7f42d715b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42d006f3c0 con 0x7f42d009eca0 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.202+0000 7f42cffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d009eca0 0x7f42d006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42cffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d009eca0 0x7f42d006ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38292/0 (socket says 172.21.15.89:38292) 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42cffff700 1 -- 172.21.15.89:0/759161710 learned_addr learned my addr 172.21.15.89:0/759161710 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42d4ef7700 1 --2- 172.21.15.89:0/759161710 >> v2:172.21.15.89:3300/0 conn(0x7f42d006e570 0x7f42d006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42d4ef7700 1 -- 172.21.15.89:0/759161710 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d009eca0 msgr2=0x7f42d006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42d4ef7700 1 --2- 172.21.15.89:0/759161710 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d009eca0 0x7f42d006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42d4ef7700 1 -- 172.21.15.89:0/759161710 >> v1:172.21.15.89:6789/0 conn(0x7f42d0134610 legacy=0x7f42d0134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42d4ef7700 1 -- 172.21.15.89:0/759161710 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0071b90 con 0x7f42d006e570 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42cffff700 1 --2- 172.21.15.89:0/759161710 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d009eca0 0x7f42d006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42d4ef7700 1 --2- 172.21.15.89:0/759161710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d006e570 0x7f42d006e940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009840 tx=0x7f42c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd10d3a1432ca6e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42cf7fe700 1 -- 172.21.15.89:0/759161710 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4011830 con 0x7f42d006e570 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42cf7fe700 1 -- 172.21.15.89:0/759161710 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c4011990 con 0x7f42d006e570 2023-09-30T22:16:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.203+0000 7f42cf7fe700 1 -- 172.21.15.89:0/759161710 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4011d20 con 0x7f42d006e570 2023-09-30T22:16:52.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.204+0000 7f42d715b700 1 -- 172.21.15.89:0/759161710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d006e570 msgr2=0x7f42d006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.204+0000 7f42d715b700 1 --2- 172.21.15.89:0/759161710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d006e570 0x7f42d006e940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009840 tx=0x7f42c4009b50 comp rx=0 tx=0).stop 2023-09-30T22:16:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.204+0000 7f42d715b700 1 -- 172.21.15.89:0/759161710 shutdown_connections 2023-09-30T22:16:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.204+0000 7f42d715b700 1 --2- 172.21.15.89:0/759161710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d006e570 0x7f42d006e940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.204+0000 7f42d715b700 1 --2- 172.21.15.89:0/759161710 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d009eca0 0x7f42d006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.204+0000 7f42d715b700 1 -- 172.21.15.89:0/759161710 >> 172.21.15.89:0/759161710 conn(0x7f42d00a2aa0 msgr2=0x7f42d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.204+0000 7f42d715b700 1 -- 172.21.15.89:0/759161710 shutdown_connections 2023-09-30T22:16:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.204+0000 7f42d715b700 1 -- 172.21.15.89:0/759161710 wait complete. 2023-09-30T22:16:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.205+0000 7f42d715b700 1 Processor -- start 2023-09-30T22:16:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.206+0000 7f42d715b700 1 -- start start 2023-09-30T22:16:52.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.206+0000 7f42d715b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d006e570 0x7f42d0123280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.206+0000 7f42d715b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d0134610 0x7f42d01237c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.206+0000 7f42d715b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42d0123d00 0x7f42d01cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.206+0000 7f42d715b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d01267c0 con 0x7f42d0134610 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.206+0000 7f42d715b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d0134200 con 0x7f42d0123d00 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.206+0000 7f42d715b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42d0126e60 con 0x7f42d006e570 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d56f8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42d0123d00 0x7f42d01cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d4ef7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d006e570 0x7f42d0123280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d56f8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42d0123d00 0x7f42d01cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49248/0 (socket says 172.21.15.89:49248) 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d4ef7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d006e570 0x7f42d0123280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38298/0 (socket says 172.21.15.89:38298) 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d56f8700 1 -- 172.21.15.89:0/4089255260 learned_addr learned my addr 172.21.15.89:0/4089255260 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d4ef7700 1 -- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42d0123d00 msgr2=0x7f42d01cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d4ef7700 1 --2- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42d0123d00 0x7f42d01cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d4ef7700 1 -- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d0134610 msgr2=0x7f42d01237c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d4ef7700 1 --2- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d0134610 0x7f42d01237c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d4ef7700 1 -- 172.21.15.89:0/4089255260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc009550 con 0x7f42d006e570 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d56f8700 1 --2- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42d0123d00 0x7f42d01cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d4ef7700 1 --2- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d006e570 0x7f42d0123280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42c4016000 tx=0x7f42c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42cd7fa700 1 -- 172.21.15.89:0/4089255260 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40036a0 con 0x7f42d006e570 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42cd7fa700 1 -- 172.21.15.89:0/4089255260 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c401e070 con 0x7f42d006e570 2023-09-30T22:16:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42cd7fa700 1 -- 172.21.15.89:0/4089255260 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4003a20 con 0x7f42d006e570 2023-09-30T22:16:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.207+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c40094a0 con 0x7f42d006e570 2023-09-30T22:16:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.208+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d0126440 con 0x7f42d006e570 2023-09-30T22:16:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.208+0000 7f42cd7fa700 1 -- 172.21.15.89:0/4089255260 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f42c4003b80 con 0x7f42d006e570 2023-09-30T22:16:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.209+0000 7f42cd7fa700 1 -- 172.21.15.89:0/4089255260 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f42c4040bd0 con 0x7f42d006e570 2023-09-30T22:16:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.209+0000 7f42cffff700 1 -- 172.21.15.89:0/4089255260 >> v1:172.21.15.89:6816/126953 conn(0x7f42b402f510 legacy=0x7f42b40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.209+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d0126ca0 con 0x7f42d006e570 2023-09-30T22:16:52.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.217+0000 7f42cd7fa700 1 -- 172.21.15.89:0/4089255260 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f42d0126ca0 con 0x7f42d006e570 2023-09-30T22:16:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.227+0000 7fcc9b13e700 1 Processor -- start 2023-09-30T22:16:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.227+0000 7fcc9b13e700 1 -- start start 2023-09-30T22:16:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.227+0000 7fcc9b13e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9409eca0 0x7fcc9413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.227+0000 7fcc9b13e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc9406f050 0x7fcc9413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc9b13e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc94070ee0 con 0x7fcc9406e570 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc9b13e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc94071980 con 0x7fcc9406f050 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc9b13e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94071480 con 0x7fcc9409eca0 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc93fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9409eca0 0x7fcc9413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc93fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9409eca0 0x7fcc9413cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38300/0 (socket says 172.21.15.89:38300) 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc93fff700 1 -- 172.21.15.89:0/2266343810 learned_addr learned my addr 172.21.15.89:0/2266343810 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc93fff700 1 -- 172.21.15.89:0/2266343810 >> v2:172.21.15.89:3300/0 conn(0x7fcc9406f050 msgr2=0x7fcc9413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc93fff700 1 --2- 172.21.15.89:0/2266343810 >> v2:172.21.15.89:3300/0 conn(0x7fcc9406f050 0x7fcc9413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc93fff700 1 -- 172.21.15.89:0/2266343810 >> v1:172.21.15.89:6789/0 conn(0x7fcc9406e570 legacy=0x7fcc9406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc93fff700 1 -- 172.21.15.89:0/2266343810 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc94071b90 con 0x7fcc9409eca0 2023-09-30T22:16:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc93fff700 1 --2- 172.21.15.89:0/2266343810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9409eca0 0x7fcc9413cea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc800098f0 tx=0x7fcc80009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42461cad88f25d3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc937fe700 1 -- 172.21.15.89:0/2266343810 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80004230 con 0x7fcc9409eca0 2023-09-30T22:16:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc937fe700 1 -- 172.21.15.89:0/2266343810 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc80004390 con 0x7fcc9409eca0 2023-09-30T22:16:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.228+0000 7fcc937fe700 1 -- 172.21.15.89:0/2266343810 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800036a0 con 0x7fcc9409eca0 2023-09-30T22:16:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.229+0000 7fcc9b13e700 1 -- 172.21.15.89:0/2266343810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9409eca0 msgr2=0x7fcc9413cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.229+0000 7fcc9b13e700 1 --2- 172.21.15.89:0/2266343810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9409eca0 0x7fcc9413cea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc800098f0 tx=0x7fcc80009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.229+0000 7f286608b700 1 Processor -- start 2023-09-30T22:16:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.229+0000 7fcc9b13e700 1 -- 172.21.15.89:0/2266343810 shutdown_connections 2023-09-30T22:16:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.229+0000 7fcc9b13e700 1 --2- 172.21.15.89:0/2266343810 >> v2:172.21.15.89:3300/0 conn(0x7fcc9406f050 0x7fcc9413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.229+0000 7fcc9b13e700 1 --2- 172.21.15.89:0/2266343810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9409eca0 0x7fcc9413cea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.229+0000 7fcc9b13e700 1 -- 172.21.15.89:0/2266343810 >> 172.21.15.89:0/2266343810 conn(0x7fcc940a2aa0 msgr2=0x7fcc940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.229+0000 7f286608b700 1 -- start start 2023-09-30T22:16:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7fcc9b13e700 1 -- 172.21.15.89:0/2266343810 shutdown_connections 2023-09-30T22:16:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7fcc9b13e700 1 -- 172.21.15.89:0/2266343810 wait complete. 2023-09-30T22:16:52.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f286608b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f286009d6f0 0x7f286009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f286608b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e410 0x7f2860130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f286608b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286009f130 con 0x7f286009e000 2023-09-30T22:16:52.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f286608b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286009f520 con 0x7f286009e410 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f286608b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f286009e930 con 0x7f286009d6f0 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f286009d6f0 0x7f286009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f285effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f286009e000 0x7f2860130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50480/0 (socket says 172.21.15.89:50480) 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f285effd700 1 -- 172.21.15.89:0/1250670337 learned_addr learned my addr 172.21.15.89:0/1250670337 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850304556 0 0) 0x7f286009f130 con 0x7f286009e000 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.230+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28480035b0 con 0x7f286009e000 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285ffff700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e410 0x7f2860130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 316482033 0 0) 0x7f28480035b0 con 0x7f286009e000 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f286009d6f0 msgr2=0x7f286009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f286009d6f0 0x7f286009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e410 msgr2=0x7f2860130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e410 0x7f2860130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286009b2d0 con 0x7f286009e000 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7f2854001670 con 0x7f286009e000 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 >> v1:172.21.15.89:6789/0 conn(0x7f286009e000 legacy=0x7f2860130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848003a00 0x7f2848003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848004850 0x7f2848004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e410 0x7f2848005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286009e930 con 0x7f2848003a00 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284800bb20 con 0x7f286009e410 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f284800bce0 con 0x7f2848004850 2023-09-30T22:16:52.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285f7fe700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848003a00 0x7f2848003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285effd700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848004850 0x7f2848004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7fcc9b13e700 1 Processor -- start 2023-09-30T22:16:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285f7fe700 1 -- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848004850 msgr2=0x7f2848004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285f7fe700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848004850 0x7f2848004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7fcc9b13e700 1 -- start start 2023-09-30T22:16:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285f7fe700 1 -- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e410 msgr2=0x7f2848005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285f7fe700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e410 0x7f2848005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285f7fe700 1 -- 172.21.15.89:0/1250670337 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480035b0 con 0x7f2848003a00 2023-09-30T22:16:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.231+0000 7f285f7fe700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848003a00 0x7f2848003e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28600a0670 tx=0x7f2848005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c894c10919cac5d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:52.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850004230 con 0x7f2848003a00 2023-09-30T22:16:52.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2850004390 con 0x7f2848003a00 2023-09-30T22:16:52.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7f285e7fc700 1 -- 172.21.15.89:0/1250670337 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28500036a0 con 0x7f2848003a00 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc9b13e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9406e570 0x7fcc9413bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc9b13e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc9406f050 0x7fcc9413c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc9b13e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc9413c850 0x7fcc94129de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc9b13e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9413e180 con 0x7fcc9413c850 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc9b13e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9412cc80 con 0x7fcc9406f050 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc9b13e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9413e8a0 con 0x7fcc9406e570 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc996db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc9413c850 0x7fcc94129de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc996db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc9413c850 0x7fcc94129de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38412/0 (socket says 172.21.15.89:38412) 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc996db700 1 -- 172.21.15.89:0/914160885 learned_addr learned my addr 172.21.15.89:0/914160885 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:52.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7fcc93fff700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc9406f050 0x7fcc9413c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7f286608b700 1 -- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848003a00 msgr2=0x7f2848003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.232+0000 7f286608b700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848003a00 0x7f2848003e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28600a0670 tx=0x7f2848005880 comp rx=0 tx=0).stop 2023-09-30T22:16:52.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7f286608b700 1 -- 172.21.15.89:0/1250670337 shutdown_connections 2023-09-30T22:16:52.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7f286608b700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e410 0x7f2848005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7f286608b700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848003a00 0x7f2848003e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7f286608b700 1 --2- 172.21.15.89:0/1250670337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848004850 0x7f2848004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7f286608b700 1 -- 172.21.15.89:0/1250670337 >> 172.21.15.89:0/1250670337 conn(0x7f2860066f50 msgr2=0x7f286009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:52.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc98eda700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9406e570 0x7fcc9413bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc996db700 1 -- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9406e570 msgr2=0x7fcc9413bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc996db700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9406e570 0x7fcc9413bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7f286608b700 1 -- 172.21.15.89:0/1250670337 shutdown_connections 2023-09-30T22:16:52.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc996db700 1 -- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc9406f050 msgr2=0x7fcc9413c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc996db700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc9406f050 0x7fcc9413c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc996db700 1 -- 172.21.15.89:0/914160885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80009550 con 0x7fcc9413c850 2023-09-30T22:16:52.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc996db700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc9413c850 0x7fcc94129de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00b9a0 tx=0x7fcc8c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:52.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc917fa700 1 -- 172.21.15.89:0/914160885 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00d8e0 con 0x7fcc9413c850 2023-09-30T22:16:52.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc917fa700 1 -- 172.21.15.89:0/914160885 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c00da60 con 0x7fcc9413c850 2023-09-30T22:16:52.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc917fa700 1 -- 172.21.15.89:0/914160885 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00de10 con 0x7fcc9413c850 2023-09-30T22:16:52.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc94138780 con 0x7fcc9413c850 2023-09-30T22:16:52.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9412d6f0 con 0x7fcc9413c850 2023-09-30T22:16:52.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.233+0000 7f286608b700 1 -- 172.21.15.89:0/1250670337 wait complete. 2023-09-30T22:16:52.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.234+0000 7f286608b700 1 Processor -- start 2023-09-30T22:16:52.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.234+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc940a24f0 con 0x7fcc9413c850 2023-09-30T22:16:52.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.234+0000 7f286608b700 1 -- start start 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f286608b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848003a00 0x7f28601d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f286608b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848004850 0x7f28601d5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f286608b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e000 0x7f28601d5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f286608b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2860130420 con 0x7f2848004850 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f286608b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28601b6960 con 0x7f286009e000 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f286608b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f286009c700 con 0x7f2848003a00 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848004850 0x7f28601d5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848004850 0x7f28601d5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38422/0 (socket says 172.21.15.89:38422) 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285effd700 1 -- 172.21.15.89:0/1464239868 learned_addr learned my addr 172.21.15.89:0/1464239868 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:52.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285ffff700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e000 0x7f28601d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285f7fe700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848003a00 0x7f28601d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285f7fe700 1 -- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e000 msgr2=0x7f28601d5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285f7fe700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e000 0x7f28601d5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285f7fe700 1 -- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848004850 msgr2=0x7f28601d5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285f7fe700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848004850 0x7f28601d5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285f7fe700 1 -- 172.21.15.89:0/1464239868 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28600a3bd0 con 0x7f2848003a00 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7fcc917fa700 1 -- 172.21.15.89:0/914160885 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fcc8c015420 con 0x7fcc9413c850 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7fcc917fa700 1 -- 172.21.15.89:0/914160885 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fcc8c011070 con 0x7fcc9413c850 2023-09-30T22:16:52.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7fcc98eda700 1 -- 172.21.15.89:0/914160885 >> v1:172.21.15.89:6816/126953 conn(0x7fcc7802f540 legacy=0x7fcc780319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:52.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.235+0000 7f285f7fe700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848003a00 0x7f28601d4ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f286009f6f0 tx=0x7f2850004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:52.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.236+0000 7f2847fff700 1 -- 172.21.15.89:0/1464239868 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285001e040 con 0x7f2848003a00 2023-09-30T22:16:52.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.236+0000 7f2847fff700 1 -- 172.21.15.89:0/1464239868 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2850019610 con 0x7f2848003a00 2023-09-30T22:16:52.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.236+0000 7f2847fff700 1 -- 172.21.15.89:0/1464239868 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28500199c0 con 0x7f2848003a00 2023-09-30T22:16:52.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.236+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28601b7f70 con 0x7f2848003a00 2023-09-30T22:16:52.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.236+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28601b6fb0 con 0x7f2848003a00 2023-09-30T22:16:52.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.237+0000 7f2847fff700 1 -- 172.21.15.89:0/1464239868 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2850019b20 con 0x7f2848003a00 2023-09-30T22:16:52.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.237+0000 7f2847fff700 1 -- 172.21.15.89:0/1464239868 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f2850040df0 con 0x7f2848003a00 2023-09-30T22:16:52.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.237+0000 7f285effd700 1 -- 172.21.15.89:0/1464239868 >> v1:172.21.15.89:6816/126953 conn(0x7f284c02f560 legacy=0x7f284c031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:52.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.237+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286009c700 con 0x7f2848003a00 2023-09-30T22:16:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.237+0000 7f512afbc700 1 Processor -- start 2023-09-30T22:16:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f512afbc700 1 -- start start 2023-09-30T22:16:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f512afbc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 0x7f5124138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f512afbc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f512413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f512afbc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512406ef30 con 0x7f51241338a0 2023-09-30T22:16:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f512afbc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512406fef0 con 0x7f5124139030 2023-09-30T22:16:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f512afbc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51241334b0 con 0x7f5124136710 2023-09-30T22:16:52.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f5128d58700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f51241338a0 0x7f5124133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50496/0 (socket says 172.21.15.89:50496) 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f5129559700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f512413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.238+0000 7f5128d58700 1 -- 172.21.15.89:0/3328436477 learned_addr learned my addr 172.21.15.89:0/3328436477 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.239+0000 7f5129559700 1 -- 172.21.15.89:0/3328436477 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 msgr2=0x7f5124138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.239+0000 7f5129559700 1 --2- 172.21.15.89:0/3328436477 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 0x7f5124138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.239+0000 7f5129559700 1 -- 172.21.15.89:0/3328436477 >> v1:172.21.15.89:6789/0 conn(0x7f51241338a0 legacy=0x7f5124133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.239+0000 7f5129559700 1 -- 172.21.15.89:0/3328436477 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512406fc30 con 0x7f5124139030 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.239+0000 7f5129559700 1 --2- 172.21.15.89:0/3328436477 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f512413b410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f511c009dc0 tx=0x7f511c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64fcece11e554bdf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.239+0000 7f51237fe700 1 -- 172.21.15.89:0/3328436477 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c00e070 con 0x7f5124139030 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.239+0000 7f51237fe700 1 -- 172.21.15.89:0/3328436477 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c00bbd0 con 0x7f5124139030 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.239+0000 7f51237fe700 1 -- 172.21.15.89:0/3328436477 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c012430 con 0x7f5124139030 2023-09-30T22:16:52.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.240+0000 7f512afbc700 1 -- 172.21.15.89:0/3328436477 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 msgr2=0x7f512413b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.240+0000 7f512afbc700 1 --2- 172.21.15.89:0/3328436477 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f512413b410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f511c009dc0 tx=0x7f511c00b330 comp rx=0 tx=0).stop 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.240+0000 7f512afbc700 1 -- 172.21.15.89:0/3328436477 shutdown_connections 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.240+0000 7f512afbc700 1 --2- 172.21.15.89:0/3328436477 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f512413b410 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.240+0000 7f512afbc700 1 --2- 172.21.15.89:0/3328436477 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 0x7f5124138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.240+0000 7f512afbc700 1 -- 172.21.15.89:0/3328436477 >> 172.21.15.89:0/3328436477 conn(0x7f51240a2aa0 msgr2=0x7f51240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.240+0000 7f512afbc700 1 -- 172.21.15.89:0/3328436477 shutdown_connections 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.240+0000 7f512afbc700 1 -- 172.21.15.89:0/3328436477 wait complete. 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.242+0000 7f512afbc700 1 Processor -- start 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.242+0000 7f512afbc700 1 -- start start 2023-09-30T22:16:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.242+0000 7f2847fff700 1 -- 172.21.15.89:0/1464239868 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28500113f0 con 0x7f2848003a00 2023-09-30T22:16:52.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f512afbc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51241338a0 0x7f512411b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f512afbc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 0x7f512411bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f512afbc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f51241c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:52.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f512afbc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512411e190 con 0x7f51241338a0 2023-09-30T22:16:52.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f512afbc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512411e3a0 con 0x7f5124139030 2023-09-30T22:16:52.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f512afbc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5124138b00 con 0x7f5124136710 2023-09-30T22:16:52.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f51241c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5123fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 0x7f512411bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f51241c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49282/0 (socket says 172.21.15.89:49282) 2023-09-30T22:16:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 -- 172.21.15.89:0/1115834976 learned_addr learned my addr 172.21.15.89:0/1115834976 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5128d58700 1 --2- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51241338a0 0x7f512411b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7fcc917fa700 1 -- 172.21.15.89:0/914160885 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc8c013920 con 0x7fcc9413c850 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 -- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 msgr2=0x7f512411bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 --2- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 0x7f512411bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 -- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51241338a0 msgr2=0x7f512411b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 --2- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51241338a0 0x7f512411b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 -- 172.21.15.89:0/1115834976 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c009a70 con 0x7f5124139030 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.243+0000 7f5129559700 1 --2- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f51241c8220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f511c014040 tx=0x7f511c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.244+0000 7f51217fa700 1 -- 172.21.15.89:0/1115834976 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c004800 con 0x7f5124139030 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.244+0000 7f51217fa700 1 -- 172.21.15.89:0/1115834976 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c020070 con 0x7f5124139030 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.244+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512406f640 con 0x7f5124139030 2023-09-30T22:16:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.244+0000 7f51217fa700 1 -- 172.21.15.89:0/1115834976 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c0276c0 con 0x7f5124139030 2023-09-30T22:16:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.244+0000 7f51217fa700 1 -- 172.21.15.89:0/1115834976 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f511c012430 con 0x7f5124139030 2023-09-30T22:16:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.245+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512411b3a0 con 0x7f5124139030 2023-09-30T22:16:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.245+0000 7f5128d58700 1 -- 172.21.15.89:0/1115834976 >> v1:172.21.15.89:6816/126953 conn(0x7f510802f540 legacy=0x7f51080319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.245+0000 7f51217fa700 1 -- 172.21.15.89:0/1115834976 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f511c022070 con 0x7f5124139030 2023-09-30T22:16:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.246+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512411e550 con 0x7f5124139030 2023-09-30T22:16:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.250+0000 7f51217fa700 1 -- 172.21.15.89:0/1115834976 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f512411e550 con 0x7f5124139030 2023-09-30T22:16:52.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.410+0000 7f42cffff700 1 -- 172.21.15.89:0/4089255260 >> v1:172.21.15.89:6816/126953 conn(0x7f42b402f510 legacy=0x7f42b40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:52.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.436+0000 7fcc98eda700 1 -- 172.21.15.89:0/914160885 >> v1:172.21.15.89:6816/126953 conn(0x7fcc7802f540 legacy=0x7fcc780319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:52.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.438+0000 7f285effd700 1 -- 172.21.15.89:0/1464239868 >> v1:172.21.15.89:6816/126953 conn(0x7f284c02f560 legacy=0x7f284c031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.441+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f42d0126ca0 con 0x7f42d006e570 2023-09-30T22:16:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.442+0000 7f42cd7fa700 1 -- 172.21.15.89:0/4089255260 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f42d0126ca0 con 0x7f42d006e570 2023-09-30T22:16:52.443 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 >> v1:172.21.15.89:6816/126953 conn(0x7f42b402f510 legacy=0x7f42b40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d006e570 msgr2=0x7f42d0123280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 --2- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d006e570 0x7f42d0123280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42c4016000 tx=0x7f42c4004750 comp rx=0 tx=0).stop 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 shutdown_connections 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 --2- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42d0123d00 0x7f42d01cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 --2- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42d006e570 0x7f42d0123280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 --2- 172.21.15.89:0/4089255260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d0134610 0x7f42d01237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 >> 172.21.15.89:0/4089255260 conn(0x7f42d00a2aa0 msgr2=0x7f42d00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 shutdown_connections 2023-09-30T22:16:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.444+0000 7f42d715b700 1 -- 172.21.15.89:0/4089255260 wait complete. 2023-09-30T22:16:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.446+0000 7f5128d58700 1 -- 172.21.15.89:0/1115834976 >> v1:172.21.15.89:6816/126953 conn(0x7f510802f540 legacy=0x7f51080319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:52.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.452+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2860074ea0 con 0x7f2848003a00 2023-09-30T22:16:52.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.452+0000 7f2847fff700 1 -- 172.21.15.89:0/1464239868 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2850023710 con 0x7f2848003a00 2023-09-30T22:16:52.453 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.454+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 >> v1:172.21.15.89:6816/126953 conn(0x7f284c02f560 legacy=0x7f284c031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.454+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848003a00 msgr2=0x7f28601d4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.454+0000 7f286608b700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848003a00 0x7f28601d4ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f286009f6f0 tx=0x7f2850004750 comp rx=0 tx=0).stop 2023-09-30T22:16:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.454+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 shutdown_connections 2023-09-30T22:16:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.454+0000 7f286608b700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f286009e000 0x7f28601d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.454+0000 7f286608b700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2848003a00 0x7f28601d4ad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.454+0000 7f286608b700 1 --2- 172.21.15.89:0/1464239868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2848004850 0x7f28601d5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.454+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 >> 172.21.15.89:0/1464239868 conn(0x7f2860066f50 msgr2=0x7f28600a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.455+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 shutdown_connections 2023-09-30T22:16:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.455+0000 7f286608b700 1 -- 172.21.15.89:0/1464239868 wait complete. 2023-09-30T22:16:52.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:16:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.457+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f512411e550 con 0x7f5124139030 2023-09-30T22:16:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.457+0000 7f51217fa700 1 -- 172.21.15.89:0/1115834976 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f512411e550 con 0x7f5124139030 2023-09-30T22:16:52.458 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:16:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 >> v1:172.21.15.89:6816/126953 conn(0x7f510802f540 legacy=0x7f51080319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 msgr2=0x7f51241c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 --2- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f51241c8220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f511c014040 tx=0x7f511c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 shutdown_connections 2023-09-30T22:16:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 --2- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5124139030 0x7f51241c8220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 --2- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51241338a0 0x7f512411b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 --2- 172.21.15.89:0/1115834976 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5124136710 0x7f512411bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 >> 172.21.15.89:0/1115834976 conn(0x7f51240a2aa0 msgr2=0x7f51241344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 shutdown_connections 2023-09-30T22:16:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.461+0000 7f512afbc700 1 -- 172.21.15.89:0/1115834976 wait complete. 2023-09-30T22:16:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.464+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcc94070610 con 0x7fcc9413c850 2023-09-30T22:16:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.464+0000 7fcc917fa700 1 -- 172.21.15.89:0/914160885 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc8c013330 con 0x7fcc9413c850 2023-09-30T22:16:52.465 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:16:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.466+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 >> v1:172.21.15.89:6816/126953 conn(0x7fcc7802f540 legacy=0x7fcc780319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.466+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc9413c850 msgr2=0x7fcc94129de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.466+0000 7fcc9b13e700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc9413c850 0x7fcc94129de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00b9a0 tx=0x7fcc8c00bcb0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.466+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 shutdown_connections 2023-09-30T22:16:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.466+0000 7fcc9b13e700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc9413c850 0x7fcc94129de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.466+0000 7fcc9b13e700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc9406e570 0x7fcc9413bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.466+0000 7fcc9b13e700 1 --2- 172.21.15.89:0/914160885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc9406f050 0x7fcc9413c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.466+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 >> 172.21.15.89:0/914160885 conn(0x7fcc940a2aa0 msgr2=0x7fcc9412aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.467+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 shutdown_connections 2023-09-30T22:16:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:52.467+0000 7fcc9b13e700 1 -- 172.21.15.89:0/914160885 wait complete. 2023-09-30T22:16:52.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:16:52.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:16:52.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:16:53.459 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:53.471 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:53.478 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:53.482 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:53.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.613+0000 7fbb4d2d1700 1 Processor -- start 2023-09-30T22:16:53.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.613+0000 7fbb4d2d1700 1 -- start start 2023-09-30T22:16:53.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb4d2d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb4809d6f0 0x7fbb4809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb4d2d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb4d2d1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4809d2f0 con 0x7fbb48133f30 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb4d2d1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4809a810 con 0x7fbb4809d6f0 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb4d2d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4809a470 con 0x7fbb4809e000 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb467fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb467fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38356/0 (socket says 172.21.15.89:38356) 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb467fc700 1 -- 172.21.15.89:0/3619497579 learned_addr learned my addr 172.21.15.89:0/3619497579 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb46ffd700 1 --2- 172.21.15.89:0/3619497579 >> v2:172.21.15.89:3300/0 conn(0x7fbb4809d6f0 0x7fbb4809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.614+0000 7fbb467fc700 1 -- 172.21.15.89:0/3619497579 >> v2:172.21.15.89:3300/0 conn(0x7fbb4809d6f0 msgr2=0x7fbb4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:53.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.615+0000 7fbb467fc700 1 --2- 172.21.15.89:0/3619497579 >> v2:172.21.15.89:3300/0 conn(0x7fbb4809d6f0 0x7fbb4809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.615+0000 7fbb467fc700 1 -- 172.21.15.89:0/3619497579 >> v1:172.21.15.89:6789/0 conn(0x7fbb48133f30 legacy=0x7fbb4813ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.615+0000 7fbb467fc700 1 -- 172.21.15.89:0/3619497579 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4809ec40 con 0x7fbb4809e000 2023-09-30T22:16:53.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.615+0000 7fbb467fc700 1 --2- 172.21.15.89:0/3619497579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481339f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0098f0 tx=0x7fbb3c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a743089fa6df8dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:53.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.615+0000 7fbb45ffb700 1 -- 172.21.15.89:0/3619497579 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c011830 con 0x7fbb4809e000 2023-09-30T22:16:53.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.615+0000 7fbb45ffb700 1 -- 172.21.15.89:0/3619497579 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c011990 con 0x7fbb4809e000 2023-09-30T22:16:53.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.615+0000 7fbb45ffb700 1 -- 172.21.15.89:0/3619497579 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c011d20 con 0x7fbb4809e000 2023-09-30T22:16:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.616+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/3619497579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 msgr2=0x7fbb481339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.616+0000 7fbb4d2d1700 1 --2- 172.21.15.89:0/3619497579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481339f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0098f0 tx=0x7fbb3c009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.616+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/3619497579 shutdown_connections 2023-09-30T22:16:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.616+0000 7fbb4d2d1700 1 --2- 172.21.15.89:0/3619497579 >> v2:172.21.15.89:3300/0 conn(0x7fbb4809d6f0 0x7fbb4809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.616+0000 7fbb4d2d1700 1 --2- 172.21.15.89:0/3619497579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481339f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.616+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/3619497579 >> 172.21.15.89:0/3619497579 conn(0x7fbb48066f50 msgr2=0x7fbb4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:53.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.616+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/3619497579 shutdown_connections 2023-09-30T22:16:53.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.616+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/3619497579 wait complete. 2023-09-30T22:16:53.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.618+0000 7fbb4d2d1700 1 Processor -- start 2023-09-30T22:16:53.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.618+0000 7fbb4d2d1700 1 -- start start 2023-09-30T22:16:53.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.618+0000 7fbb4d2d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4809d6f0 0x7fbb481194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.618+0000 7fbb4d2d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.618+0000 7fbb4d2d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb48133f30 0x7fbb48119f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.618+0000 7fbb4d2d1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4811cfd0 con 0x7fbb4809d6f0 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.618+0000 7fbb4d2d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481339f0 con 0x7fbb48133f30 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.618+0000 7fbb4d2d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4809c640 con 0x7fbb4809e000 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb48133f30 0x7fbb48119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb48133f30 0x7fbb48119f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49290/0 (socket says 172.21.15.89:49290) 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb46ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4809d6f0 0x7fbb481194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 -- 172.21.15.89:0/65193104 learned_addr learned my addr 172.21.15.89:0/65193104 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb467fc700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 -- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 msgr2=0x7fbb481199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 -- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4809d6f0 msgr2=0x7fbb481194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4809d6f0 0x7fbb481194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 -- 172.21.15.89:0/65193104 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c009550 con 0x7fbb48133f30 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb46ffd700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4809d6f0 0x7fbb481194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb477fe700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb48133f30 0x7fbb48119f30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000cc80 tx=0x7fbb4000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb2f7fe700 1 -- 172.21.15.89:0/65193104 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40003c00 con 0x7fbb48133f30 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb2f7fe700 1 -- 172.21.15.89:0/65193104 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb40003d60 con 0x7fbb48133f30 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4811b0c0 con 0x7fbb48133f30 2023-09-30T22:16:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.619+0000 7fbb2f7fe700 1 -- 172.21.15.89:0/65193104 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000aaa0 con 0x7fbb48133f30 2023-09-30T22:16:53.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.620+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481cc660 con 0x7fbb48133f30 2023-09-30T22:16:53.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.620+0000 7fbb2f7fe700 1 -- 172.21.15.89:0/65193104 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbb4000ac00 con 0x7fbb48133f30 2023-09-30T22:16:53.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.621+0000 7fbb46ffd700 1 -- 172.21.15.89:0/65193104 >> v1:172.21.15.89:6816/126953 conn(0x7fbb3402f510 legacy=0x7fbb340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:53.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.621+0000 7fbb2f7fe700 1 -- 172.21.15.89:0/65193104 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fbb4003fb00 con 0x7fbb48133f30 2023-09-30T22:16:53.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.622+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb48074ea0 con 0x7fbb48133f30 2023-09-30T22:16:53.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.626+0000 7fbb2f7fe700 1 -- 172.21.15.89:0/65193104 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb400198f0 con 0x7fbb48133f30 2023-09-30T22:16:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7f2711987700 1 Processor -- start 2023-09-30T22:16:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7f2711987700 1 -- start start 2023-09-30T22:16:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7fc7941b5700 1 Processor -- start 2023-09-30T22:16:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7fc7941b5700 1 -- start start 2023-09-30T22:16:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7f2711987700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f270c09d6f0 0x7f270c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7f2711987700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e000 0x7f270c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7f2711987700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270c09f130 con 0x7f270c09e410 2023-09-30T22:16:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7f2711987700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c09f520 con 0x7f270c09d6f0 2023-09-30T22:16:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.638+0000 7f2711987700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f270c09e930 con 0x7f270c09e000 2023-09-30T22:16:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc7941b5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 0x7fc78c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7f270affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f270c09d6f0 0x7f270c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7f270affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f270c09d6f0 0x7f270c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38458/0 (socket says 172.21.15.89:38458) 2023-09-30T22:16:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7f270affd700 1 -- 172.21.15.89:0/2838439010 learned_addr learned my addr 172.21.15.89:0/2838439010 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc7941b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e410 0x7fc78c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc7941b5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78c09f130 con 0x7fc78c09e000 2023-09-30T22:16:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc7941b5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c09f520 con 0x7fc78c09e410 2023-09-30T22:16:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc7941b5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c09e930 con 0x7fc78c09d6f0 2023-09-30T22:16:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc791750700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc78c09e000 0x7fc78c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50520/0 (socket says 172.21.15.89:50520) 2023-09-30T22:16:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc791750700 1 -- 172.21.15.89:0/2057774092 learned_addr learned my addr 172.21.15.89:0/2057774092 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc792752700 1 --2- 172.21.15.89:0/2057774092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e410 0x7fc78c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7f270a7fc700 1 --2- 172.21.15.89:0/2838439010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e000 0x7f270c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7f270a7fc700 1 -- 172.21.15.89:0/2838439010 >> v2:172.21.15.89:3300/0 conn(0x7f270c09d6f0 msgr2=0x7f270c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7f270a7fc700 1 --2- 172.21.15.89:0/2838439010 >> v2:172.21.15.89:3300/0 conn(0x7f270c09d6f0 0x7f270c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7f270a7fc700 1 -- 172.21.15.89:0/2838439010 >> v1:172.21.15.89:6789/0 conn(0x7f270c09e410 legacy=0x7f270c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7f270a7fc700 1 -- 172.21.15.89:0/2838439010 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c09b2d0 con 0x7f270c09e000 2023-09-30T22:16:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc792752700 1 -- 172.21.15.89:0/2057774092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 msgr2=0x7fc78c09dac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:16:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc792752700 1 --2- 172.21.15.89:0/2057774092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 0x7fc78c09dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc792752700 1 -- 172.21.15.89:0/2057774092 >> v1:172.21.15.89:6789/0 conn(0x7fc78c09e000 legacy=0x7fc78c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.639+0000 7fc792752700 1 -- 172.21.15.89:0/2057774092 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c09b2d0 con 0x7fc78c09e410 2023-09-30T22:16:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7f270a7fc700 1 --2- 172.21.15.89:0/2838439010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e000 0x7f270c130660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f270000bae0 tx=0x7f270000bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4eb7238279d00d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7fc792752700 1 --2- 172.21.15.89:0/2057774092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e410 0x7fc78c130d70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc784009dc0 tx=0x7fc78400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82fee2664bbe876 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7fc790f4f700 1 -- 172.21.15.89:0/2057774092 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78400e070 con 0x7fc78c09e410 2023-09-30T22:16:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7f2709ffb700 1 -- 172.21.15.89:0/2838439010 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270000dc70 con 0x7f270c09e000 2023-09-30T22:16:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7f2709ffb700 1 -- 172.21.15.89:0/2838439010 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270000ddd0 con 0x7f270c09e000 2023-09-30T22:16:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7f2709ffb700 1 -- 172.21.15.89:0/2838439010 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700010670 con 0x7f270c09e000 2023-09-30T22:16:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7fc790f4f700 1 -- 172.21.15.89:0/2057774092 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78400bb90 con 0x7fc78c09e410 2023-09-30T22:16:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7fc790f4f700 1 -- 172.21.15.89:0/2057774092 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784003a20 con 0x7fc78c09e410 2023-09-30T22:16:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7fc7941b5700 1 -- 172.21.15.89:0/2057774092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e410 msgr2=0x7fc78c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7fc7941b5700 1 --2- 172.21.15.89:0/2057774092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e410 0x7fc78c130d70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc784009dc0 tx=0x7fc78400b330 comp rx=0 tx=0).stop 2023-09-30T22:16:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7f2711987700 1 -- 172.21.15.89:0/2838439010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e000 msgr2=0x7f270c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.640+0000 7f2711987700 1 --2- 172.21.15.89:0/2838439010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e000 0x7f270c130660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f270000bae0 tx=0x7f270000bdf0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7fc7941b5700 1 -- 172.21.15.89:0/2057774092 shutdown_connections 2023-09-30T22:16:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7fc7941b5700 1 --2- 172.21.15.89:0/2057774092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e410 0x7fc78c130d70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7fc7941b5700 1 --2- 172.21.15.89:0/2057774092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 0x7fc78c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7fc7941b5700 1 -- 172.21.15.89:0/2057774092 >> 172.21.15.89:0/2057774092 conn(0x7fc78c066f50 msgr2=0x7fc78c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7f2711987700 1 -- 172.21.15.89:0/2838439010 shutdown_connections 2023-09-30T22:16:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7f2711987700 1 --2- 172.21.15.89:0/2838439010 >> v2:172.21.15.89:3300/0 conn(0x7f270c09d6f0 0x7f270c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7f2711987700 1 --2- 172.21.15.89:0/2838439010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e000 0x7f270c130660 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7f2711987700 1 -- 172.21.15.89:0/2838439010 >> 172.21.15.89:0/2838439010 conn(0x7f270c066f50 msgr2=0x7f270c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7f2711987700 1 -- 172.21.15.89:0/2838439010 shutdown_connections 2023-09-30T22:16:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7f2711987700 1 -- 172.21.15.89:0/2838439010 wait complete. 2023-09-30T22:16:53.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7fc7941b5700 1 -- 172.21.15.89:0/2057774092 shutdown_connections 2023-09-30T22:16:53.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.641+0000 7fc7941b5700 1 -- 172.21.15.89:0/2057774092 wait complete. 2023-09-30T22:16:53.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7f2711987700 1 Processor -- start 2023-09-30T22:16:53.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7fc7941b5700 1 Processor -- start 2023-09-30T22:16:53.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7fc7941b5700 1 -- start start 2023-09-30T22:16:53.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc7941b5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 0x7fc78c12bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc7941b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e000 0x7fc78c12c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc7941b5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc78c09e410 0x7fc78c12c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc7941b5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c12f430 con 0x7fc78c09e410 2023-09-30T22:16:53.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc7941b5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c131130 con 0x7fc78c09e000 2023-09-30T22:16:53.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc7941b5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c12f610 con 0x7fc78c09d6f0 2023-09-30T22:16:53.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc791f51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 0x7fc78c12bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc791f51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 0x7fc78c12bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38390/0 (socket says 172.21.15.89:38390) 2023-09-30T22:16:53.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc791f51700 1 -- 172.21.15.89:0/3271553860 learned_addr learned my addr 172.21.15.89:0/3271553860 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:53.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc791750700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e000 0x7fc78c12c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7fc792752700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc78c09e410 0x7fc78c12c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7f2711987700 1 -- start start 2023-09-30T22:16:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7f2711987700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f270c09d6f0 0x7f270c11e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7f2711987700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f270c09e000 0x7f270c11e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7f2711987700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e410 0x7f270c11eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7f2711987700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0a1d00 con 0x7f270c09d6f0 2023-09-30T22:16:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7f2711987700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c09c700 con 0x7f270c09e000 2023-09-30T22:16:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.642+0000 7f2711987700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f270c121590 con 0x7f270c09e410 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f270c09d6f0 0x7f270c11e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f270c09d6f0 0x7f270c11e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38460/0 (socket says 172.21.15.89:38460) 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270affd700 1 -- 172.21.15.89:0/366769808 learned_addr learned my addr 172.21.15.89:0/366769808 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270b7fe700 1 --2- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e410 0x7f270c11eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270affd700 1 -- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e410 msgr2=0x7f270c11eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270affd700 1 --2- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e410 0x7f270c11eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270affd700 1 -- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f270c09e000 msgr2=0x7f270c11e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270affd700 1 --2- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f270c09e000 0x7f270c11e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc791750700 1 -- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 msgr2=0x7fc78c12bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc791750700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 0x7fc78c12bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc791750700 1 -- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc78c09e410 msgr2=0x7fc78c12c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.643+0000 7f270affd700 1 -- 172.21.15.89:0/366769808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270000b740 con 0x7f270c09d6f0 2023-09-30T22:16:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc791750700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc78c09e410 0x7fc78c12c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc791750700 1 -- 172.21.15.89:0/3271553860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784009a70 con 0x7fc78c09e000 2023-09-30T22:16:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc791750700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e000 0x7fc78c12c210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c430 tx=0x7fc77c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc7827fc700 1 -- 172.21.15.89:0/3271553860 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c005b70 con 0x7fc78c09e000 2023-09-30T22:16:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7f270affd700 1 --2- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f270c09d6f0 0x7f270c11e090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00e910 tx=0x7f26fc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc7827fc700 1 -- 172.21.15.89:0/3271553860 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c005cd0 con 0x7fc78c09e000 2023-09-30T22:16:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c130290 con 0x7fc78c09e000 2023-09-30T22:16:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc7827fc700 1 -- 172.21.15.89:0/3271553860 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c007310 con 0x7fc78c09e000 2023-09-30T22:16:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.644+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c12e7a0 con 0x7fc78c09e000 2023-09-30T22:16:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.645+0000 7f26f3fff700 1 -- 172.21.15.89:0/366769808 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0096a0 con 0x7f270c09d6f0 2023-09-30T22:16:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.645+0000 7f26f3fff700 1 -- 172.21.15.89:0/366769808 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26fc00ee60 con 0x7f270c09d6f0 2023-09-30T22:16:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.645+0000 7f26f3fff700 1 -- 172.21.15.89:0/366769808 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc00ca90 con 0x7f270c09d6f0 2023-09-30T22:16:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.645+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78c074ea0 con 0x7fc78c09e000 2023-09-30T22:16:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.647+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270c11fcc0 con 0x7f270c09d6f0 2023-09-30T22:16:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.647+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270c1cc5d0 con 0x7f270c09d6f0 2023-09-30T22:16:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.648+0000 7f26f3fff700 1 -- 172.21.15.89:0/366769808 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f26fc00cbf0 con 0x7f270c09d6f0 2023-09-30T22:16:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.648+0000 7f26f3fff700 1 -- 172.21.15.89:0/366769808 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f26fc061bd0 con 0x7f270c09d6f0 2023-09-30T22:16:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.648+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270c074ea0 con 0x7f270c09d6f0 2023-09-30T22:16:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.646+0000 7fc7827fc700 1 -- 172.21.15.89:0/3271553860 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc77c007470 con 0x7fc78c09e000 2023-09-30T22:16:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.646+0000 7fc7827fc700 1 -- 172.21.15.89:0/3271553860 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc77c03f4a0 con 0x7fc78c09e000 2023-09-30T22:16:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.646+0000 7fc791f51700 1 -- 172.21.15.89:0/3271553860 >> v1:172.21.15.89:6816/126953 conn(0x7fc77002f510 legacy=0x7fc7700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.651+0000 7f270a7fc700 1 -- 172.21.15.89:0/366769808 >> v1:172.21.15.89:6816/126953 conn(0x7f26f802f510 legacy=0x7f26f80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.651+0000 7fc7827fc700 1 -- 172.21.15.89:0/3271553860 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc77c01b3b0 con 0x7fc78c09e000 2023-09-30T22:16:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.653+0000 7f26f3fff700 1 -- 172.21.15.89:0/366769808 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f26fc039080 con 0x7f270c09d6f0 2023-09-30T22:16:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.658+0000 7fbc62534700 1 Processor -- start 2023-09-30T22:16:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.658+0000 7fbc62534700 1 -- start start 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.658+0000 7fbc62534700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09d6f0 0x7fbc5c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.658+0000 7fbc62534700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc5c09e410 0x7fbc5c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.658+0000 7fbc62534700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c09b120 con 0x7fbc5c09e000 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc62534700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5c06b460 con 0x7fbc5c09e410 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc62534700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c09a7e0 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbc5c09e000 0x7fbc5c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50528/0 (socket says 172.21.15.89:50528) 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5b7fe700 1 -- 172.21.15.89:0/2912784501 learned_addr learned my addr 172.21.15.89:0/2912784501 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5bfff700 1 --2- 172.21.15.89:0/2912784501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09d6f0 0x7fbc5c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc60ad1700 1 --2- 172.21.15.89:0/2912784501 >> v2:172.21.15.89:3300/0 conn(0x7fbc5c09e410 0x7fbc5c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5bfff700 1 -- 172.21.15.89:0/2912784501 >> v2:172.21.15.89:3300/0 conn(0x7fbc5c09e410 msgr2=0x7fbc5c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5bfff700 1 --2- 172.21.15.89:0/2912784501 >> v2:172.21.15.89:3300/0 conn(0x7fbc5c09e410 0x7fbc5c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5bfff700 1 -- 172.21.15.89:0/2912784501 >> v1:172.21.15.89:6789/0 conn(0x7fbc5c09e000 legacy=0x7fbc5c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5affd700 1 -- 172.21.15.89:0/2912784501 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072943428 0 0) 0x7fbc5c09b120 con 0x7fbc5c09e000 2023-09-30T22:16:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5bfff700 1 -- 172.21.15.89:0/2912784501 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c09a500 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5bfff700 1 --2- 172.21.15.89:0/2912784501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09d6f0 0x7fbc5c09dac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009870 tx=0x7fbc4c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=941304d488fa865b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.659+0000 7fbc5affd700 1 -- 172.21.15.89:0/2912784501 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c011830 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.660+0000 7fbc5affd700 1 -- 172.21.15.89:0/2912784501 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c011990 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.660+0000 7fbc5affd700 1 -- 172.21.15.89:0/2912784501 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c011d20 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.660+0000 7fbc62534700 1 -- 172.21.15.89:0/2912784501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09d6f0 msgr2=0x7fbc5c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.660+0000 7fbc62534700 1 --2- 172.21.15.89:0/2912784501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09d6f0 0x7fbc5c09dac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009870 tx=0x7fbc4c009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.661+0000 7fbc62534700 1 -- 172.21.15.89:0/2912784501 shutdown_connections 2023-09-30T22:16:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.661+0000 7fbc62534700 1 --2- 172.21.15.89:0/2912784501 >> v2:172.21.15.89:3300/0 conn(0x7fbc5c09e410 0x7fbc5c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.661+0000 7fbc62534700 1 --2- 172.21.15.89:0/2912784501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09d6f0 0x7fbc5c09dac0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.661+0000 7fbc62534700 1 -- 172.21.15.89:0/2912784501 >> 172.21.15.89:0/2912784501 conn(0x7fbc5c066f50 msgr2=0x7fbc5c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.661+0000 7fbc62534700 1 -- 172.21.15.89:0/2912784501 shutdown_connections 2023-09-30T22:16:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.661+0000 7fbc62534700 1 -- 172.21.15.89:0/2912784501 wait complete. 2023-09-30T22:16:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc62534700 1 Processor -- start 2023-09-30T22:16:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc62534700 1 -- start start 2023-09-30T22:16:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc62534700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc5c09d6f0 0x7fbc5c1c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc62534700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc5c09e000 0x7fbc5c1c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc62534700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09e410 0x7fbc5c1c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc62534700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1a6950 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc62534700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0a2ed0 con 0x7fbc5c09e000 2023-09-30T22:16:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc62534700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1a64b0 con 0x7fbc5c09e410 2023-09-30T22:16:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc5bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc5c09d6f0 0x7fbc5c1c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc5bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc5c09d6f0 0x7fbc5c1c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38482/0 (socket says 172.21.15.89:38482) 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc5bfff700 1 -- 172.21.15.89:0/97459670 learned_addr learned my addr 172.21.15.89:0/97459670 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.662+0000 7fbc60ad1700 1 --2- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09e410 0x7fbc5c1c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc5bfff700 1 -- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09e410 msgr2=0x7fbc5c1c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc5bfff700 1 --2- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09e410 0x7fbc5c1c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc5bfff700 1 -- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc5c09e000 msgr2=0x7fbc5c1c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc5bfff700 1 --2- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc5c09e000 0x7fbc5c1c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc5bfff700 1 -- 172.21.15.89:0/97459670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0094a0 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc5bfff700 1 --2- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc5c09d6f0 0x7fbc5c1c8020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0090b0 tx=0x7fbc4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc58ff9700 1 -- 172.21.15.89:0/97459670 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c01a960 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc58ff9700 1 -- 172.21.15.89:0/97459670 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c01aac0 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c1a6b90 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc58ff9700 1 -- 172.21.15.89:0/97459670 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c01ae50 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.663+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c1a82b0 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.664+0000 7fbc58ff9700 1 -- 172.21.15.89:0/97459670 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbc4c02d450 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.664+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c1a7c70 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.664+0000 7fbc5b7fe700 1 -- 172.21.15.89:0/97459670 >> v1:172.21.15.89:6816/126953 conn(0x7fbc4802f510 legacy=0x7fbc480319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.664+0000 7fbc58ff9700 1 -- 172.21.15.89:0/97459670 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fbc4c041090 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.670+0000 7fbc58ff9700 1 -- 172.21.15.89:0/97459670 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc4c02d700 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.822+0000 7fbb46ffd700 1 -- 172.21.15.89:0/65193104 >> v1:172.21.15.89:6816/126953 conn(0x7fbb3402f510 legacy=0x7fbb340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.834+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb4811cc50 con 0x7fbb48133f30 2023-09-30T22:16:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.835+0000 7fbb2f7fe700 1 -- 172.21.15.89:0/65193104 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb40019300 con 0x7fbb48133f30 2023-09-30T22:16:53.835 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 >> v1:172.21.15.89:6816/126953 conn(0x7fbb3402f510 legacy=0x7fbb340319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb48133f30 msgr2=0x7fbb48119f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb48133f30 0x7fbb48119f30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000cc80 tx=0x7fbb4000cf90 comp rx=0 tx=0).stop 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 shutdown_connections 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb48133f30 0x7fbb48119f30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4809d6f0 0x7fbb481194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 --2- 172.21.15.89:0/65193104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb4809e000 0x7fbb481199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 >> 172.21.15.89:0/65193104 conn(0x7fbb48066f50 msgr2=0x7fbb480a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 shutdown_connections 2023-09-30T22:16:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.837+0000 7fbb4d2d1700 1 -- 172.21.15.89:0/65193104 wait complete. 2023-09-30T22:16:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.846+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc78c0008d0 con 0x7fc78c09e000 2023-09-30T22:16:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.847+0000 7fc791f51700 1 -- 172.21.15.89:0/3271553860 >> v1:172.21.15.89:6816/126953 conn(0x7fc77002f510 legacy=0x7fc7700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.847+0000 7fc7827fc700 1 -- 172.21.15.89:0/3271553860 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc77c017030 con 0x7fc78c09e000 2023-09-30T22:16:53.848 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:16:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.849+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 >> v1:172.21.15.89:6816/126953 conn(0x7fc77002f510 legacy=0x7fc7700319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.849+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e000 msgr2=0x7fc78c12c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.849+0000 7fc7941b5700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e000 0x7fc78c12c210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c430 tx=0x7fc77c00c740 comp rx=0 tx=0).stop 2023-09-30T22:16:53.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:16:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.850+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 shutdown_connections 2023-09-30T22:16:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.850+0000 7fc7941b5700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc78c09e410 0x7fc78c12c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.850+0000 7fc7941b5700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc78c09d6f0 0x7fc78c12bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.850+0000 7fc7941b5700 1 --2- 172.21.15.89:0/3271553860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc78c09e000 0x7fc78c12c210 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.850+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 >> 172.21.15.89:0/3271553860 conn(0x7fc78c066f50 msgr2=0x7fc78c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.850+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 shutdown_connections 2023-09-30T22:16:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.850+0000 7fc7941b5700 1 -- 172.21.15.89:0/3271553860 wait complete. 2023-09-30T22:16:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.851+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f270c06c360 con 0x7f270c09d6f0 2023-09-30T22:16:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.851+0000 7f26f3fff700 1 -- 172.21.15.89:0/366769808 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f26fc043030 con 0x7f270c09d6f0 2023-09-30T22:16:53.853 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.852+0000 7f270a7fc700 1 -- 172.21.15.89:0/366769808 >> v1:172.21.15.89:6816/126953 conn(0x7f26f802f510 legacy=0x7f26f80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.853+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 >> v1:172.21.15.89:6816/126953 conn(0x7f26f802f510 legacy=0x7f26f80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.853+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f270c09d6f0 msgr2=0x7f270c11e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.853+0000 7f2711987700 1 --2- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f270c09d6f0 0x7f270c11e090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00e910 tx=0x7f26fc00ec20 comp rx=0 tx=0).stop 2023-09-30T22:16:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.854+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 shutdown_connections 2023-09-30T22:16:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.854+0000 7f2711987700 1 --2- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f270c09e410 0x7f270c11eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.854+0000 7f2711987700 1 --2- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f270c09d6f0 0x7f270c11e090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.854+0000 7f2711987700 1 --2- 172.21.15.89:0/366769808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f270c09e000 0x7f270c11e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.854+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 >> 172.21.15.89:0/366769808 conn(0x7f270c066f50 msgr2=0x7f270c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.854+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 shutdown_connections 2023-09-30T22:16:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.854+0000 7f2711987700 1 -- 172.21.15.89:0/366769808 wait complete. 2023-09-30T22:16:53.863 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:16:53.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.864+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbc5c074ea0 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.865+0000 7fbc5b7fe700 1 -- 172.21.15.89:0/97459670 >> v1:172.21.15.89:6816/126953 conn(0x7fbc4802f510 legacy=0x7fbc480319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:53.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.865+0000 7fbc58ff9700 1 -- 172.21.15.89:0/97459670 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc4c02c3b0 con 0x7fbc5c09d6f0 2023-09-30T22:16:53.865 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:16:53.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 >> v1:172.21.15.89:6816/126953 conn(0x7fbc4802f510 legacy=0x7fbc480319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc5c09d6f0 msgr2=0x7fbc5c1c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 --2- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc5c09d6f0 0x7fbc5c1c8020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0090b0 tx=0x7fbc4c004750 comp rx=0 tx=0).stop 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 shutdown_connections 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 --2- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc5c09e410 0x7fbc5c1c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 --2- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc5c09d6f0 0x7fbc5c1c8020 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 --2- 172.21.15.89:0/97459670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc5c09e000 0x7fbc5c1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 >> 172.21.15.89:0/97459670 conn(0x7fbc5c066f50 msgr2=0x7fbc5c1bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 shutdown_connections 2023-09-30T22:16:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:53.867+0000 7fbc62534700 1 -- 172.21.15.89:0/97459670 wait complete. 2023-09-30T22:16:53.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:16:54.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:54.864 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:54.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:54.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:54.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.993+0000 7faff29b1700 1 Processor -- start 2023-09-30T22:16:54.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.993+0000 7faff29b1700 1 -- start start 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.993+0000 7faff29b1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09d6f0 0x7fafec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.993+0000 7faff29b1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09e410 0x7fafec130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.993+0000 7faff29b1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec09f130 con 0x7fafec09e000 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.993+0000 7faff29b1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec09f520 con 0x7fafec09d6f0 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.993+0000 7faff29b1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafec09e930 con 0x7fafec09e410 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.993+0000 7fafeb7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fafec09e000 0x7fafec130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50544/0 (socket says 172.21.15.89:50544) 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafeb7fe700 1 -- 172.21.15.89:0/4000223661 learned_addr learned my addr 172.21.15.89:0/4000223661 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafebfff700 1 --2- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09d6f0 0x7fafec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7faff0f4e700 1 --2- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09e410 0x7fafec130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafeaffd700 1 -- 172.21.15.89:0/4000223661 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940265024 0 0) 0x7fafec09f130 con 0x7fafec09e000 2023-09-30T22:16:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafeaffd700 1 -- 172.21.15.89:0/4000223661 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafd40035b0 con 0x7fafec09e000 2023-09-30T22:16:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafebfff700 1 -- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09e410 msgr2=0x7fafec130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafebfff700 1 --2- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09e410 0x7fafec130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafebfff700 1 -- 172.21.15.89:0/4000223661 >> v1:172.21.15.89:6789/0 conn(0x7fafec09e000 legacy=0x7fafec130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafebfff700 1 -- 172.21.15.89:0/4000223661 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec09b2d0 con 0x7fafec09d6f0 2023-09-30T22:16:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafebfff700 1 --2- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09d6f0 0x7fafec09dac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009870 tx=0x7fafdc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ffddd71e7def13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.994+0000 7fafeaffd700 1 -- 172.21.15.89:0/4000223661 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc011830 con 0x7fafec09d6f0 2023-09-30T22:16:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7fafeaffd700 1 -- 172.21.15.89:0/4000223661 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc011990 con 0x7fafec09d6f0 2023-09-30T22:16:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7fafeaffd700 1 -- 172.21.15.89:0/4000223661 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc011d20 con 0x7fafec09d6f0 2023-09-30T22:16:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7faff29b1700 1 -- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09d6f0 msgr2=0x7fafec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7faff29b1700 1 --2- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09d6f0 0x7fafec09dac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009870 tx=0x7fafdc009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7faff29b1700 1 -- 172.21.15.89:0/4000223661 shutdown_connections 2023-09-30T22:16:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7faff29b1700 1 --2- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09d6f0 0x7fafec09dac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7faff29b1700 1 --2- 172.21.15.89:0/4000223661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09e410 0x7fafec130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7faff29b1700 1 -- 172.21.15.89:0/4000223661 >> 172.21.15.89:0/4000223661 conn(0x7fafec066f50 msgr2=0x7fafec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7faff29b1700 1 -- 172.21.15.89:0/4000223661 shutdown_connections 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.995+0000 7faff29b1700 1 -- 172.21.15.89:0/4000223661 wait complete. 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.996+0000 7faff29b1700 1 Processor -- start 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7faff29b1700 1 -- start start 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7faff29b1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09d6f0 0x7fafec11dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7faff29b1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09e000 0x7fafec11e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7faff29b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec09e410 0x7fafec11ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7faff29b1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec11f4a0 con 0x7fafec09e410 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7faff29b1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec121af0 con 0x7fafec09e000 2023-09-30T22:16:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7faff29b1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafec120e30 con 0x7fafec09d6f0 2023-09-30T22:16:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7fafeb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09e000 0x7fafec11e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7fafebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09d6f0 0x7fafec11dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7fafeb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09e000 0x7fafec11e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49350/0 (socket says 172.21.15.89:49350) 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7fafeb7fe700 1 -- 172.21.15.89:0/2003308294 learned_addr learned my addr 172.21.15.89:0/2003308294 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.997+0000 7faff0f4e700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec09e410 0x7fafec11ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7faff0f4e700 1 -- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09d6f0 msgr2=0x7fafec11dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7faff0f4e700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09d6f0 0x7fafec11dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7faff0f4e700 1 -- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09e000 msgr2=0x7fafec11e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7faff0f4e700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09e000 0x7fafec11e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7faff0f4e700 1 -- 172.21.15.89:0/2003308294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0094a0 con 0x7fafec09e410 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7fafeb7fe700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09e000 0x7fafec11e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7faff0f4e700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec09e410 0x7fafec11ea50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009fb0 tx=0x7fafe400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7fafe8ff9700 1 -- 172.21.15.89:0/2003308294 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400fe10 con 0x7fafec09e410 2023-09-30T22:16:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7fafe8ff9700 1 -- 172.21.15.89:0/2003308294 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe400e040 con 0x7fafec09e410 2023-09-30T22:16:55.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec1303b0 con 0x7fafec09e410 2023-09-30T22:16:55.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7fafe8ff9700 1 -- 172.21.15.89:0/2003308294 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4003c60 con 0x7fafec09e410 2023-09-30T22:16:55.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.998+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec09a0c0 con 0x7fafec09e410 2023-09-30T22:16:55.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.999+0000 7fafe8ff9700 1 -- 172.21.15.89:0/2003308294 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fafe4015440 con 0x7fafec09e410 2023-09-30T22:16:55.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.999+0000 7fafebfff700 1 -- 172.21.15.89:0/2003308294 >> v1:172.21.15.89:6816/126953 conn(0x7fafd802f510 legacy=0x7fafd80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:55.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.999+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec09c760 con 0x7fafec09e410 2023-09-30T22:16:55.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:54.999+0000 7fafe8ff9700 1 -- 172.21.15.89:0/2003308294 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fafe4013e40 con 0x7fafec09e410 2023-09-30T22:16:55.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.008+0000 7fafe8ff9700 1 -- 172.21.15.89:0/2003308294 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fafe4047060 con 0x7fafec09e410 2023-09-30T22:16:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.033+0000 7f40da0fa700 1 Processor -- start 2023-09-30T22:16:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.033+0000 7f40da0fa700 1 -- start start 2023-09-30T22:16:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40da0fa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40d4133420 0x7f40d41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40da0fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4133e30 0x7f40d412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40da0fa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d406b990 con 0x7f40d4134260 2023-09-30T22:16:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40da0fa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d40a0590 con 0x7f40d4133420 2023-09-30T22:16:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40da0fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40d409fb80 con 0x7f40d4133e30 2023-09-30T22:16:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40d4133420 0x7f40d41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40d4133420 0x7f40d41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38508/0 (socket says 172.21.15.89:38508) 2023-09-30T22:16:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f40d4134260 0x7f40d41304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50548/0 (socket says 172.21.15.89:50548) 2023-09-30T22:16:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d37fe700 1 -- 172.21.15.89:0/3001730013 learned_addr learned my addr 172.21.15.89:0/3001730013 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40cbfff700 1 --2- 172.21.15.89:0/3001730013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4133e30 0x7f40d412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d37fe700 1 -- 172.21.15.89:0/3001730013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4133e30 msgr2=0x7f40d412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d37fe700 1 --2- 172.21.15.89:0/3001730013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4133e30 0x7f40d412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d37fe700 1 -- 172.21.15.89:0/3001730013 >> v1:172.21.15.89:6789/0 conn(0x7f40d4134260 legacy=0x7f40d41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d37fe700 1 -- 172.21.15.89:0/3001730013 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40a0030 con 0x7f40d4133420 2023-09-30T22:16:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d37fe700 1 --2- 172.21.15.89:0/3001730013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133420 0x7f40d41337f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40c40098f0 tx=0x7f40c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387b615ff67f02fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.034+0000 7f40d2ffd700 1 -- 172.21.15.89:0/3001730013 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4004230 con 0x7f40d4133420 2023-09-30T22:16:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.035+0000 7f40d2ffd700 1 -- 172.21.15.89:0/3001730013 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4004390 con 0x7f40d4133420 2023-09-30T22:16:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.035+0000 7f40d2ffd700 1 -- 172.21.15.89:0/3001730013 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c40036a0 con 0x7f40d4133420 2023-09-30T22:16:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.035+0000 7f40da0fa700 1 -- 172.21.15.89:0/3001730013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133420 msgr2=0x7f40d41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.035+0000 7f40da0fa700 1 --2- 172.21.15.89:0/3001730013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133420 0x7f40d41337f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40c40098f0 tx=0x7f40c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7f40da0fa700 1 -- 172.21.15.89:0/3001730013 shutdown_connections 2023-09-30T22:16:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7f40da0fa700 1 --2- 172.21.15.89:0/3001730013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133420 0x7f40d41337f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7f40da0fa700 1 --2- 172.21.15.89:0/3001730013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4133e30 0x7f40d412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7f40da0fa700 1 -- 172.21.15.89:0/3001730013 >> 172.21.15.89:0/3001730013 conn(0x7f40d40a4210 msgr2=0x7f40d40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:55.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7f40da0fa700 1 -- 172.21.15.89:0/3001730013 shutdown_connections 2023-09-30T22:16:55.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7f40da0fa700 1 -- 172.21.15.89:0/3001730013 wait complete. 2023-09-30T22:16:55.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7fc97a779700 1 Processor -- start 2023-09-30T22:16:55.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7fc97a779700 1 -- start start 2023-09-30T22:16:55.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.036+0000 7f40da0fa700 1 Processor -- start 2023-09-30T22:16:55.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40da0fa700 1 -- start start 2023-09-30T22:16:55.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40da0fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40d4133420 0x7f40d409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40da0fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133e30 0x7f40d409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40da0fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4134260 0x7f40d409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7fc97a779700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741338c0 0x7fc974133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7fc97a779700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9741341d0 0x7fc97409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7fc97a779700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97406f720 con 0x7fc97409e390 2023-09-30T22:16:55.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7fc97a779700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc974070c80 con 0x7fc9741338c0 2023-09-30T22:16:55.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7fc97a779700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc97406fce0 con 0x7fc9741341d0 2023-09-30T22:16:55.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40da0fa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d409d850 con 0x7f40d4133e30 2023-09-30T22:16:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40da0fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d409df30 con 0x7f40d4133420 2023-09-30T22:16:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40da0fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40d411fc80 con 0x7f40d4134260 2023-09-30T22:16:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133e30 0x7f40d409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133e30 0x7f40d409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38524/0 (socket says 172.21.15.89:38524) 2023-09-30T22:16:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7f40cbfff700 1 -- 172.21.15.89:0/700356752 learned_addr learned my addr 172.21.15.89:0/700356752 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:55.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7fc9737fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9741341d0 0x7fc97409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7fc9737fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9741341d0 0x7fc97409de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38430/0 (socket says 172.21.15.89:38430) 2023-09-30T22:16:55.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.037+0000 7fc9737fe700 1 -- 172.21.15.89:0/2017652567 learned_addr learned my addr 172.21.15.89:0/2017652567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:55.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40cbfff700 1 -- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4134260 msgr2=0x7f40d409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40cbfff700 1 --2- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4134260 0x7f40d409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40cbfff700 1 -- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40d4133420 msgr2=0x7f40d409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40cbfff700 1 --2- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40d4133420 0x7f40d409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40cbfff700 1 -- 172.21.15.89:0/700356752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c4009550 con 0x7f40d4133e30 2023-09-30T22:16:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc972ffd700 1 -- 172.21.15.89:0/2017652567 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791761480 0 0) 0x7fc97406f720 con 0x7fc97409e390 2023-09-30T22:16:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc972ffd700 1 -- 172.21.15.89:0/2017652567 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc95c0035b0 con 0x7fc97409e390 2023-09-30T22:16:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc9737fe700 1 -- 172.21.15.89:0/2017652567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741338c0 msgr2=0x7fc974133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc9737fe700 1 --2- 172.21.15.89:0/2017652567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741338c0 0x7fc974133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc9737fe700 1 -- 172.21.15.89:0/2017652567 >> v1:172.21.15.89:6789/0 conn(0x7fc97409e390 legacy=0x7fc97413a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc9737fe700 1 -- 172.21.15.89:0/2017652567 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9741336a0 con 0x7fc9741341d0 2023-09-30T22:16:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc9737fe700 1 --2- 172.21.15.89:0/2017652567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9741341d0 0x7fc97409de50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc968009920 tx=0x7fc968009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b466c9173e9fed88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40cbfff700 1 --2- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133e30 0x7f40d409c9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e9b0 tx=0x7f40bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40d0ff9700 1 -- 172.21.15.89:0/700356752 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc009710 con 0x7f40d4133e30 2023-09-30T22:16:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc972ffd700 1 -- 172.21.15.89:0/2017652567 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968011830 con 0x7fc9741341d0 2023-09-30T22:16:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc972ffd700 1 -- 172.21.15.89:0/2017652567 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968011990 con 0x7fc9741341d0 2023-09-30T22:16:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7fc972ffd700 1 -- 172.21.15.89:0/2017652567 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968011d20 con 0x7fc9741341d0 2023-09-30T22:16:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40d0ff9700 1 -- 172.21.15.89:0/700356752 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00c850 con 0x7f40d4133e30 2023-09-30T22:16:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d412d650 con 0x7f40d4133e30 2023-09-30T22:16:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d411ea90 con 0x7f40d4133e30 2023-09-30T22:16:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7fc97a779700 1 -- 172.21.15.89:0/2017652567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9741341d0 msgr2=0x7fc97409de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7fc97a779700 1 --2- 172.21.15.89:0/2017652567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9741341d0 0x7fc97409de50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc968009920 tx=0x7fc968009c30 comp rx=0 tx=0).stop 2023-09-30T22:16:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.038+0000 7f40d0ff9700 1 -- 172.21.15.89:0/700356752 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00cbc0 con 0x7f40d4133e30 2023-09-30T22:16:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7fc97a779700 1 -- 172.21.15.89:0/2017652567 shutdown_connections 2023-09-30T22:16:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7fc97a779700 1 --2- 172.21.15.89:0/2017652567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741338c0 0x7fc974133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7fc97a779700 1 --2- 172.21.15.89:0/2017652567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9741341d0 0x7fc97409de50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7f40d0ff9700 1 -- 172.21.15.89:0/700356752 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f40bc00cd20 con 0x7f40d4133e30 2023-09-30T22:16:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7fc97a779700 1 -- 172.21.15.89:0/2017652567 >> 172.21.15.89:0/2017652567 conn(0x7fc9740a2aa0 msgr2=0x7fc9740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7fc97a779700 1 -- 172.21.15.89:0/2017652567 shutdown_connections 2023-09-30T22:16:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7fc97a779700 1 -- 172.21.15.89:0/2017652567 wait complete. 2023-09-30T22:16:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b80052e0 con 0x7f40d4133e30 2023-09-30T22:16:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.039+0000 7f40d37fe700 1 -- 172.21.15.89:0/700356752 >> v1:172.21.15.89:6816/126953 conn(0x7f40b402f510 legacy=0x7f40b40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.040+0000 7f40d0ff9700 1 -- 172.21.15.89:0/700356752 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f40bc03fd00 con 0x7f40d4133e30 2023-09-30T22:16:55.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.040+0000 7fc97a779700 1 Processor -- start 2023-09-30T22:16:55.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc97a779700 1 -- start start 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc97a779700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409e390 0x7fc97411f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc97a779700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9741338c0 0x7fc97411ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc97a779700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741341d0 0x7fc974120450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc97a779700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc974122ec0 con 0x7fc9741338c0 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc97a779700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc974120e10 con 0x7fc9741341d0 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc97a779700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc974121550 con 0x7fc97409e390 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc973fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409e390 0x7fc97411f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc9737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9741338c0 0x7fc97411ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc973fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409e390 0x7fc97411f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38442/0 (socket says 172.21.15.89:38442) 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc973fff700 1 -- 172.21.15.89:0/400834489 learned_addr learned my addr 172.21.15.89:0/400834489 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc978d16700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741341d0 0x7fc974120450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc973fff700 1 -- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741341d0 msgr2=0x7fc974120450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc973fff700 1 --2- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741341d0 0x7fc974120450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc973fff700 1 -- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9741338c0 msgr2=0x7fc97411ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.041+0000 7fc973fff700 1 --2- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9741338c0 0x7fc97411ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.042+0000 7fc973fff700 1 -- 172.21.15.89:0/400834489 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968009550 con 0x7fc97409e390 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.042+0000 7fc9737fe700 1 --2- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9741338c0 0x7fc97411ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.042+0000 7fc973fff700 1 --2- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409e390 0x7fc97411f9d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc96400b530 tx=0x7fc96400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.042+0000 7fc970ff9700 1 -- 172.21.15.89:0/400834489 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964011830 con 0x7fc97409e390 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.042+0000 7fc970ff9700 1 -- 172.21.15.89:0/400834489 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964011990 con 0x7fc97409e390 2023-09-30T22:16:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.042+0000 7fc970ff9700 1 -- 172.21.15.89:0/400834489 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964011d40 con 0x7fc97409e390 2023-09-30T22:16:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.042+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97406f960 con 0x7fc97409e390 2023-09-30T22:16:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.042+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc974130d30 con 0x7fc97409e390 2023-09-30T22:16:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.043+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc974074ea0 con 0x7fc97409e390 2023-09-30T22:16:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.043+0000 7fc970ff9700 1 -- 172.21.15.89:0/400834489 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc96401a400 con 0x7fc97409e390 2023-09-30T22:16:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.044+0000 7fc970ff9700 1 -- 172.21.15.89:0/400834489 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc964040250 con 0x7fc97409e390 2023-09-30T22:16:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.044+0000 7fc9737fe700 1 -- 172.21.15.89:0/400834489 >> v1:172.21.15.89:6816/126953 conn(0x7fc96002f510 legacy=0x7fc9600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.045+0000 7f40d0ff9700 1 -- 172.21.15.89:0/700356752 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f40bc06e750 con 0x7f40d4133e30 2023-09-30T22:16:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.047+0000 7fd5c3a62700 1 Processor -- start 2023-09-30T22:16:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c3a62700 1 -- start start 2023-09-30T22:16:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c3a62700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5bc1338c0 0x7fd5bc133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c3a62700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc13b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c3a62700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc06f720 con 0x7fd5bc1341d0 2023-09-30T22:16:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c3a62700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc070c80 con 0x7fd5bc1338c0 2023-09-30T22:16:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c3a62700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc06fce0 con 0x7fd5bc09e560 2023-09-30T22:16:55.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fc970ff9700 1 -- 172.21.15.89:0/400834489 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc964021710 con 0x7fc97409e390 2023-09-30T22:16:55.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c1fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc13b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c0ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd5bc1341d0 0x7fd5bc09de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50568/0 (socket says 172.21.15.89:50568) 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.049+0000 7fd5c0ffd700 1 -- 172.21.15.89:0/3840076943 learned_addr learned my addr 172.21.15.89:0/3840076943 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.048+0000 7fd5c1fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc13b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38446/0 (socket says 172.21.15.89:38446) 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.049+0000 7fd5c17fe700 1 --2- 172.21.15.89:0/3840076943 >> v2:172.21.15.89:3300/0 conn(0x7fd5bc1338c0 0x7fd5bc133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.049+0000 7fd5c1fff700 1 -- 172.21.15.89:0/3840076943 >> v2:172.21.15.89:3300/0 conn(0x7fd5bc1338c0 msgr2=0x7fd5bc133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.049+0000 7fd5c1fff700 1 --2- 172.21.15.89:0/3840076943 >> v2:172.21.15.89:3300/0 conn(0x7fd5bc1338c0 0x7fd5bc133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.049+0000 7fd5c1fff700 1 -- 172.21.15.89:0/3840076943 >> v1:172.21.15.89:6789/0 conn(0x7fd5bc1341d0 legacy=0x7fd5bc09de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.049+0000 7fd5c1fff700 1 -- 172.21.15.89:0/3840076943 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc1336a0 con 0x7fd5bc09e560 2023-09-30T22:16:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.049+0000 7fd5c1fff700 1 --2- 172.21.15.89:0/3840076943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc13b860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009d90 tx=0x7fd5b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf96cd1f8ccbf4d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5b3fff700 1 -- 172.21.15.89:0/3840076943 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b400e070 con 0x7fd5bc09e560 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5b3fff700 1 -- 172.21.15.89:0/3840076943 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b400bb20 con 0x7fd5bc09e560 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5b3fff700 1 -- 172.21.15.89:0/3840076943 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4012430 con 0x7fd5bc09e560 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3840076943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 msgr2=0x7fd5bc13b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5c3a62700 1 --2- 172.21.15.89:0/3840076943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc13b860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009d90 tx=0x7fd5b400b330 comp rx=0 tx=0).stop 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3840076943 shutdown_connections 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5c3a62700 1 --2- 172.21.15.89:0/3840076943 >> v2:172.21.15.89:3300/0 conn(0x7fd5bc1338c0 0x7fd5bc133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5c3a62700 1 --2- 172.21.15.89:0/3840076943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc13b860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.052+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3840076943 >> 172.21.15.89:0/3840076943 conn(0x7fd5bc0a2aa0 msgr2=0x7fd5bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.053+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3840076943 shutdown_connections 2023-09-30T22:16:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.053+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3840076943 wait complete. 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.054+0000 7fd5c3a62700 1 Processor -- start 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.054+0000 7fd5c3a62700 1 -- start start 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c3a62700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc11eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c3a62700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5bc1338c0 0x7fd5bc11f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c3a62700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5bc1341d0 0x7fd5bc11f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c3a62700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc12fb50 con 0x7fd5bc1341d0 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c3a62700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc06f960 con 0x7fd5bc1338c0 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c3a62700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc122110 con 0x7fd5bc09e560 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc11eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc11eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38462/0 (socket says 172.21.15.89:38462) 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 -- 172.21.15.89:0/3130117846 learned_addr learned my addr 172.21.15.89:0/3130117846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c0ffd700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5bc1338c0 0x7fd5bc11f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c1fff700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5bc1341d0 0x7fd5bc11f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 -- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5bc1338c0 msgr2=0x7fd5bc11f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5bc1338c0 0x7fd5bc11f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 -- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5bc1341d0 msgr2=0x7fd5bc11f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5bc1341d0 0x7fd5bc11f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 -- 172.21.15.89:0/3130117846 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4009a70 con 0x7fd5bc09e560 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c17fe700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc11eb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9b0 tx=0x7fd5b800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5b1ffb700 1 -- 172.21.15.89:0/3130117846 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8009710 con 0x7fd5bc09e560 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5b1ffb700 1 -- 172.21.15.89:0/3130117846 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b800ee50 con 0x7fd5bc09e560 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5b1ffb700 1 -- 172.21.15.89:0/3130117846 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800ca50 con 0x7fd5bc09e560 2023-09-30T22:16:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.055+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc12f690 con 0x7fd5bc09e560 2023-09-30T22:16:55.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.056+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc122e40 con 0x7fd5bc09e560 2023-09-30T22:16:55.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.056+0000 7fd5b1ffb700 1 -- 172.21.15.89:0/3130117846 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd5b800cbb0 con 0x7fd5bc09e560 2023-09-30T22:16:55.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.057+0000 7fd5b1ffb700 1 -- 172.21.15.89:0/3130117846 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fd5b803fc50 con 0x7fd5bc09e560 2023-09-30T22:16:55.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.057+0000 7fd5c0ffd700 1 -- 172.21.15.89:0/3130117846 >> v1:172.21.15.89:6816/126953 conn(0x7fd5a002f510 legacy=0x7fd5a00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:55.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.060+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc074ea0 con 0x7fd5bc09e560 2023-09-30T22:16:55.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.065+0000 7fd5b1ffb700 1 -- 172.21.15.89:0/3130117846 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5b801be10 con 0x7fd5bc09e560 2023-09-30T22:16:55.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.200+0000 7fafebfff700 1 -- 172.21.15.89:0/2003308294 >> v1:172.21.15.89:6816/126953 conn(0x7fafd802f510 legacy=0x7fafd80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.218+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fafec074ea0 con 0x7fafec09e410 2023-09-30T22:16:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.219+0000 7fafe8ff9700 1 -- 172.21.15.89:0/2003308294 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafe401c360 con 0x7fafec09e410 2023-09-30T22:16:55.220 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 >> v1:172.21.15.89:6816/126953 conn(0x7fafd802f510 legacy=0x7fafd80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec09e410 msgr2=0x7fafec11ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec09e410 0x7fafec11ea50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009fb0 tx=0x7fafe400f4a0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 shutdown_connections 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec09e410 0x7fafec11ea50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafec09d6f0 0x7fafec11dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 --2- 172.21.15.89:0/2003308294 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafec09e000 0x7fafec11e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 >> 172.21.15.89:0/2003308294 conn(0x7fafec066f50 msgr2=0x7fafec070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 shutdown_connections 2023-09-30T22:16:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.222+0000 7faff29b1700 1 -- 172.21.15.89:0/2003308294 wait complete. 2023-09-30T22:16:55.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:16:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.241+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f40b8005860 con 0x7f40d4133e30 2023-09-30T22:16:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.241+0000 7f40d37fe700 1 -- 172.21.15.89:0/700356752 >> v1:172.21.15.89:6816/126953 conn(0x7f40b402f510 legacy=0x7f40b40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.241+0000 7f40d0ff9700 1 -- 172.21.15.89:0/700356752 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40bc021030 con 0x7f40d4133e30 2023-09-30T22:16:55.242 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 >> v1:172.21.15.89:6816/126953 conn(0x7f40b402f510 legacy=0x7f40b40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133e30 msgr2=0x7f40d409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 --2- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133e30 0x7f40d409c9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e9b0 tx=0x7f40bc00ed70 comp rx=0 tx=0).stop 2023-09-30T22:16:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 shutdown_connections 2023-09-30T22:16:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 --2- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40d4134260 0x7f40d409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 --2- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40d4133420 0x7f40d409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 --2- 172.21.15.89:0/700356752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40d4133e30 0x7f40d409c9c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 >> 172.21.15.89:0/700356752 conn(0x7f40d40a4210 msgr2=0x7f40d40a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 shutdown_connections 2023-09-30T22:16:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.243+0000 7f40da0fa700 1 -- 172.21.15.89:0/700356752 wait complete. 2023-09-30T22:16:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.245+0000 7fc9737fe700 1 -- 172.21.15.89:0/400834489 >> v1:172.21.15.89:6816/126953 conn(0x7fc96002f510 legacy=0x7fc9600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:55.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:16:55.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.258+0000 7fd5c0ffd700 1 -- 172.21.15.89:0/3130117846 >> v1:172.21.15.89:6816/126953 conn(0x7fd5a002f510 legacy=0x7fd5a00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:55.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.284+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc974123c50 con 0x7fc97409e390 2023-09-30T22:16:55.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.284+0000 7fc970ff9700 1 -- 172.21.15.89:0/400834489 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc964028090 con 0x7fc97409e390 2023-09-30T22:16:55.285 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:16:55.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.286+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 >> v1:172.21.15.89:6816/126953 conn(0x7fc96002f510 legacy=0x7fc9600319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:55.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409e390 msgr2=0x7fc97411f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 --2- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409e390 0x7fc97411f9d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc96400b530 tx=0x7fc96400b840 comp rx=0 tx=0).stop 2023-09-30T22:16:55.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 shutdown_connections 2023-09-30T22:16:55.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 --2- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9741341d0 0x7fc974120450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 --2- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409e390 0x7fc97411f9d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 --2- 172.21.15.89:0/400834489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9741338c0 0x7fc97411ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 >> 172.21.15.89:0/400834489 conn(0x7fc9740a2aa0 msgr2=0x7fc97409f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:55.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 shutdown_connections 2023-09-30T22:16:55.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.287+0000 7fc97a779700 1 -- 172.21.15.89:0/400834489 wait complete. 2023-09-30T22:16:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.292+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd5bc0a0d20 con 0x7fd5bc09e560 2023-09-30T22:16:55.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.292+0000 7fd5b1ffb700 1 -- 172.21.15.89:0/3130117846 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5b801b3b0 con 0x7fd5bc09e560 2023-09-30T22:16:55.293 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:16:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 >> v1:172.21.15.89:6816/126953 conn(0x7fd5a002f510 legacy=0x7fd5a00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 msgr2=0x7fd5bc11eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc11eb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9b0 tx=0x7fd5b800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 shutdown_connections 2023-09-30T22:16:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5bc1341d0 0x7fd5bc11f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5bc09e560 0x7fd5bc11eb90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 --2- 172.21.15.89:0/3130117846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5bc1338c0 0x7fd5bc11f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 >> 172.21.15.89:0/3130117846 conn(0x7fd5bc0a2aa0 msgr2=0x7fd5bc1306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 shutdown_connections 2023-09-30T22:16:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:55.294+0000 7fd5c3a62700 1 -- 172.21.15.89:0/3130117846 wait complete. 2023-09-30T22:16:55.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:16:55.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:16:56.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:56.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:56.301 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:56.309 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.384+0000 7f8add324700 1 Processor -- start 2023-09-30T22:16:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.384+0000 7f8add324700 1 -- start start 2023-09-30T22:16:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.384+0000 7f8add324700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ad809d6f0 0x7f8ad809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.384+0000 7f8add324700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809e440 0x7f8ad8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.384+0000 7f8add324700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad809b120 con 0x7f8ad809e000 2023-09-30T22:16:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.384+0000 7f8add324700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad806b460 con 0x7f8ad809d6f0 2023-09-30T22:16:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8add324700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad809a7e0 con 0x7f8ad809e440 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ad809d6f0 0x7f8ad809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ad809d6f0 0x7f8ad809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38566/0 (socket says 172.21.15.89:38566) 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad6ffd700 1 -- 172.21.15.89:0/3042324690 learned_addr learned my addr 172.21.15.89:0/3042324690 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad6ffd700 1 -- 172.21.15.89:0/3042324690 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809e440 msgr2=0x7f8ad8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad6ffd700 1 --2- 172.21.15.89:0/3042324690 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809e440 0x7f8ad8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad6ffd700 1 -- 172.21.15.89:0/3042324690 >> v1:172.21.15.89:6789/0 conn(0x7f8ad809e000 legacy=0x7f8ad81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad6ffd700 1 -- 172.21.15.89:0/3042324690 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad809a500 con 0x7f8ad809d6f0 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad6ffd700 1 --2- 172.21.15.89:0/3042324690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809d6f0 0x7f8ad809dac0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8013210 tx=0x7f8ac8013520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565b88174d1abc79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad5ffb700 1 -- 172.21.15.89:0/3042324690 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8016070 con 0x7f8ad809d6f0 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad5ffb700 1 -- 172.21.15.89:0/3042324690 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac8013e20 con 0x7f8ad809d6f0 2023-09-30T22:16:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.385+0000 7f8ad5ffb700 1 -- 172.21.15.89:0/3042324690 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80112f0 con 0x7f8ad809d6f0 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.386+0000 7f8add324700 1 -- 172.21.15.89:0/3042324690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809d6f0 msgr2=0x7f8ad809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.386+0000 7f8add324700 1 --2- 172.21.15.89:0/3042324690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809d6f0 0x7f8ad809dac0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8013210 tx=0x7f8ac8013520 comp rx=0 tx=0).stop 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.386+0000 7f8add324700 1 -- 172.21.15.89:0/3042324690 shutdown_connections 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.386+0000 7f8add324700 1 --2- 172.21.15.89:0/3042324690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809d6f0 0x7f8ad809dac0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.386+0000 7f8add324700 1 --2- 172.21.15.89:0/3042324690 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809e440 0x7f8ad8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.386+0000 7f8add324700 1 -- 172.21.15.89:0/3042324690 >> 172.21.15.89:0/3042324690 conn(0x7f8ad8066f50 msgr2=0x7f8ad809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.386+0000 7f8add324700 1 -- 172.21.15.89:0/3042324690 shutdown_connections 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.386+0000 7f8add324700 1 -- 172.21.15.89:0/3042324690 wait complete. 2023-09-30T22:16:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.387+0000 7f8add324700 1 Processor -- start 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.387+0000 7f8add324700 1 -- start start 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8add324700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809d6f0 0x7f8ad81a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8add324700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad809e000 0x7f8ad81a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8add324700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809e440 0x7f8ad81a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8add324700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81a5650 con 0x7f8ad809e440 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8add324700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81a4a00 con 0x7f8ad809e000 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8add324700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81a37e0 con 0x7f8ad809d6f0 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad809e000 0x7f8ad81a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809e440 0x7f8ad81a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad809e000 0x7f8ad81a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49396/0 (socket says 172.21.15.89:49396) 2023-09-30T22:16:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809e440 0x7f8ad81a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38572/0 (socket says 172.21.15.89:38572) 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad67fc700 1 -- 172.21.15.89:0/2738631686 learned_addr learned my addr 172.21.15.89:0/2738631686 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad6ffd700 1 --2- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809d6f0 0x7f8ad81a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad67fc700 1 -- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809d6f0 msgr2=0x7f8ad81a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad67fc700 1 --2- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809d6f0 0x7f8ad81a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad67fc700 1 -- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809e440 msgr2=0x7f8ad81a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad67fc700 1 --2- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809e440 0x7f8ad81a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.388+0000 7f8ad67fc700 1 -- 172.21.15.89:0/2738631686 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8011df0 con 0x7f8ad809e000 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.389+0000 7f8ad67fc700 1 --2- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad809e000 0x7f8ad81a2070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00c4e0 tx=0x7f8acc00c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.389+0000 7f8abf7fe700 1 -- 172.21.15.89:0/2738631686 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0052d0 con 0x7f8ad809e000 2023-09-30T22:16:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.389+0000 7f8abf7fe700 1 -- 172.21.15.89:0/2738631686 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc005430 con 0x7f8ad809e000 2023-09-30T22:16:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.389+0000 7f8abf7fe700 1 -- 172.21.15.89:0/2738631686 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0038c0 con 0x7f8ad809e000 2023-09-30T22:16:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.389+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad80a2200 con 0x7f8ad809e000 2023-09-30T22:16:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.389+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81a5b50 con 0x7f8ad809e000 2023-09-30T22:16:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.390+0000 7f8abf7fe700 1 -- 172.21.15.89:0/2738631686 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8acc003a20 con 0x7f8ad809e000 2023-09-30T22:16:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.390+0000 7f8abf7fe700 1 -- 172.21.15.89:0/2738631686 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f8acc03efe0 con 0x7f8ad809e000 2023-09-30T22:16:56.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.390+0000 7f8ad6ffd700 1 -- 172.21.15.89:0/2738631686 >> v1:172.21.15.89:6816/126953 conn(0x7f8ac402f510 legacy=0x7f8ac40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:56.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.390+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad8074ea0 con 0x7f8ad809e000 2023-09-30T22:16:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.399+0000 7f8abf7fe700 1 -- 172.21.15.89:0/2738631686 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8acc003cd0 con 0x7f8ad809e000 2023-09-30T22:16:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.420+0000 7f8780c83700 1 Processor -- start 2023-09-30T22:16:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.420+0000 7f8780c83700 1 -- start start 2023-09-30T22:16:56.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8780c83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8780c83700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f877c09e000 0x7f877c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8780c83700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877c09b120 con 0x7f877c09e410 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8780c83700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c06b460 con 0x7f877c09e000 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8780c83700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f877c09a7e0 con 0x7f877c09d6f0 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8779d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f877c09e000 0x7f877c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8779d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f877c09e000 0x7f877c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38580/0 (socket says 172.21.15.89:38580) 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8779d9b700 1 -- 172.21.15.89:0/974973234 learned_addr learned my addr 172.21.15.89:0/974973234 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f877a59c700 1 --2- 172.21.15.89:0/974973234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8779d9b700 1 -- 172.21.15.89:0/974973234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 msgr2=0x7f877c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.421+0000 7f8779d9b700 1 --2- 172.21.15.89:0/974973234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.422+0000 7f8779d9b700 1 -- 172.21.15.89:0/974973234 >> v1:172.21.15.89:6789/0 conn(0x7f877c09e410 legacy=0x7f877c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.422+0000 7f8779d9b700 1 -- 172.21.15.89:0/974973234 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c09a500 con 0x7f877c09e000 2023-09-30T22:16:56.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.422+0000 7f8779d9b700 1 --2- 172.21.15.89:0/974973234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 0x7f877c1339f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87700098f0 tx=0x7f8770009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896bd4e4671ed00e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:56.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.422+0000 7f877959a700 1 -- 172.21.15.89:0/974973234 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770011830 con 0x7f877c09e000 2023-09-30T22:16:56.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.422+0000 7f877959a700 1 -- 172.21.15.89:0/974973234 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8770011990 con 0x7f877c09e000 2023-09-30T22:16:56.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.422+0000 7f877959a700 1 -- 172.21.15.89:0/974973234 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770011d20 con 0x7f877c09e000 2023-09-30T22:16:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.423+0000 7f8780c83700 1 -- 172.21.15.89:0/974973234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 msgr2=0x7f877c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.423+0000 7f8780c83700 1 --2- 172.21.15.89:0/974973234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 0x7f877c1339f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87700098f0 tx=0x7f8770009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.423+0000 7f8780c83700 1 -- 172.21.15.89:0/974973234 shutdown_connections 2023-09-30T22:16:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.423+0000 7f8780c83700 1 --2- 172.21.15.89:0/974973234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.423+0000 7f8780c83700 1 --2- 172.21.15.89:0/974973234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 0x7f877c1339f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.423+0000 7f8780c83700 1 -- 172.21.15.89:0/974973234 >> 172.21.15.89:0/974973234 conn(0x7f877c066f50 msgr2=0x7f877c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.423+0000 7f8780c83700 1 -- 172.21.15.89:0/974973234 shutdown_connections 2023-09-30T22:16:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.423+0000 7f8780c83700 1 -- 172.21.15.89:0/974973234 wait complete. 2023-09-30T22:16:56.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.424+0000 7f8780c83700 1 Processor -- start 2023-09-30T22:16:56.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.424+0000 7f8780c83700 1 -- start start 2023-09-30T22:16:56.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f8780c83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c1a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f8780c83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 0x7f877c1a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f8780c83700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c09e410 0x7f877c1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f8780c83700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c1a93e0 con 0x7f877c09e000 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f8780c83700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c1a7b80 con 0x7f877c09e410 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f8780c83700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f877c09c710 con 0x7f877c09d6f0 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f877a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c1a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f877a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c1a5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38510/0 (socket says 172.21.15.89:38510) 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f877a59c700 1 -- 172.21.15.89:0/3881910904 learned_addr learned my addr 172.21.15.89:0/3881910904 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f8779d9b700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 0x7f877c1a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.425+0000 7f877ad9d700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c09e410 0x7f877c1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f877a59c700 1 -- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c09e410 msgr2=0x7f877c1a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f877a59c700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c09e410 0x7f877c1a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f877a59c700 1 -- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 msgr2=0x7f877c1a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f877a59c700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 0x7f877c1a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f877a59c700 1 -- 172.21.15.89:0/3881910904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770009550 con 0x7f877c09d6f0 2023-09-30T22:16:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f877a59c700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c1a5ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f876c00e9b0 tx=0x7f876c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f876affd700 1 -- 172.21.15.89:0/3881910904 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00ee50 con 0x7f877c09d6f0 2023-09-30T22:16:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f876affd700 1 -- 172.21.15.89:0/3881910904 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c00c850 con 0x7f877c09d6f0 2023-09-30T22:16:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f876affd700 1 -- 172.21.15.89:0/3881910904 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00cbc0 con 0x7f877c09d6f0 2023-09-30T22:16:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c1a8d30 con 0x7f877c09d6f0 2023-09-30T22:16:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.426+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c1a9980 con 0x7f877c09d6f0 2023-09-30T22:16:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.427+0000 7f876affd700 1 -- 172.21.15.89:0/3881910904 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f876c00cd20 con 0x7f877c09d6f0 2023-09-30T22:16:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.427+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c074ea0 con 0x7f877c09d6f0 2023-09-30T22:16:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.427+0000 7f876affd700 1 -- 172.21.15.89:0/3881910904 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f876c03ff80 con 0x7f877c09d6f0 2023-09-30T22:16:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.427+0000 7f8779d9b700 1 -- 172.21.15.89:0/3881910904 >> v1:172.21.15.89:6816/126953 conn(0x7f875c02f510 legacy=0x7f875c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:56.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.432+0000 7f876affd700 1 -- 172.21.15.89:0/3881910904 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f876c013470 con 0x7f877c09d6f0 2023-09-30T22:16:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.461+0000 7fe169a79700 1 Processor -- start 2023-09-30T22:16:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.461+0000 7fe169a79700 1 -- start start 2023-09-30T22:16:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.461+0000 7fe169a79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe16409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.461+0000 7fe169a79700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e000 0x7fe164130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.461+0000 7fe169a79700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe16409e410 0x7fe164130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.461+0000 7fe169a79700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16409f520 con 0x7fe16409e410 2023-09-30T22:16:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.461+0000 7fe169a79700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16409f130 con 0x7fe16409e000 2023-09-30T22:16:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.461+0000 7fe169a79700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe16409e930 con 0x7fe16409d6f0 2023-09-30T22:16:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe1637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe16409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe1637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe16409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38526/0 (socket says 172.21.15.89:38526) 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe1637fe700 1 -- 172.21.15.89:0/4115446003 learned_addr learned my addr 172.21.15.89:0/4115446003 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe163fff700 1 --2- 172.21.15.89:0/4115446003 >> v2:172.21.15.89:3300/0 conn(0x7fe16409e410 0x7fe164130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe162ffd700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e000 0x7fe164130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe163fff700 1 -- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 msgr2=0x7fe16409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe163fff700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe16409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe163fff700 1 -- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e000 msgr2=0x7fe164130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe163fff700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e000 0x7fe164130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe163fff700 1 -- 172.21.15.89:0/4115446003 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16409b2d0 con 0x7fe16409e410 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe162ffd700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e000 0x7fe164130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe163fff700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e410 0x7fe164130ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009d90 tx=0x7fe15c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b48cd73642f4497 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe1627fc700 1 -- 172.21.15.89:0/4115446003 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00e070 con 0x7fe16409e410 2023-09-30T22:16:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe1627fc700 1 -- 172.21.15.89:0/4115446003 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c00bb20 con 0x7fe16409e410 2023-09-30T22:16:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.462+0000 7fe1627fc700 1 -- 172.21.15.89:0/4115446003 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c012430 con 0x7fe16409e410 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 -- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e410 msgr2=0x7fe164130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e410 0x7fe164130ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009d90 tx=0x7fe15c00b330 comp rx=0 tx=0).stop 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 -- 172.21.15.89:0/4115446003 shutdown_connections 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e410 0x7fe164130ba0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe16409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 --2- 172.21.15.89:0/4115446003 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e000 0x7fe164130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 -- 172.21.15.89:0/4115446003 >> 172.21.15.89:0/4115446003 conn(0x7fe164066f50 msgr2=0x7fe16409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 -- 172.21.15.89:0/4115446003 shutdown_connections 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.463+0000 7fe169a79700 1 -- 172.21.15.89:0/4115446003 wait complete. 2023-09-30T22:16:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.464+0000 7fe169a79700 1 Processor -- start 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.464+0000 7fe169a79700 1 -- start start 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe169a79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe164122470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe169a79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e000 0x7fe1641229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe169a79700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e410 0x7fe164122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe169a79700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164125cb0 con 0x7fe16409e000 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe169a79700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe164124190 con 0x7fe16409e410 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe169a79700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640a3a60 con 0x7fe16409d6f0 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e410 0x7fe164122ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e410 0x7fe164122ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49422/0 (socket says 172.21.15.89:49422) 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 -- 172.21.15.89:0/1731431702 learned_addr learned my addr 172.21.15.89:0/1731431702 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe162ffd700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e000 0x7fe1641229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe1637fe700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe164122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 -- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 msgr2=0x7fe164122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe164122470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 -- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e000 msgr2=0x7fe1641229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e000 0x7fe1641229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fbb92b4f700 1 Processor -- start 2023-09-30T22:16:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 -- 172.21.15.89:0/1731431702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580094a0 con 0x7fe16409e410 2023-09-30T22:16:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe1637fe700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe164122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:16:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fbb92b4f700 1 -- start start 2023-09-30T22:16:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.465+0000 7fe163fff700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e410 0x7fe164122ef0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe15c004270 tx=0x7fe15c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fe14bfff700 1 -- 172.21.15.89:0/1731431702 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c003e10 con 0x7fe16409e410 2023-09-30T22:16:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c009a70 con 0x7fe16409e410 2023-09-30T22:16:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16409c750 con 0x7fe16409e410 2023-09-30T22:16:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb92b4f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133420 0x7fbb8c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb92b4f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133e30 0x7fbb8c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb92b4f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb8c134260 0x7fbb8c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb92b4f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c06b990 con 0x7fbb8c134260 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb92b4f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0a0590 con 0x7fbb8c133420 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb92b4f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c09fb80 con 0x7fbb8c133e30 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb908eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133420 0x7fbb8c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb908eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133420 0x7fbb8c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49426/0 (socket says 172.21.15.89:49426) 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb908eb700 1 -- 172.21.15.89:0/4273895392 learned_addr learned my addr 172.21.15.89:0/4273895392 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb908eb700 1 -- 172.21.15.89:0/4273895392 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133e30 msgr2=0x7fbb8c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb908eb700 1 --2- 172.21.15.89:0/4273895392 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133e30 0x7fbb8c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb908eb700 1 -- 172.21.15.89:0/4273895392 >> v2:172.21.15.89:3300/0 conn(0x7fbb8c134260 msgr2=0x7fbb8c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb908eb700 1 --2- 172.21.15.89:0/4273895392 >> v2:172.21.15.89:3300/0 conn(0x7fbb8c134260 0x7fbb8c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fbb908eb700 1 -- 172.21.15.89:0/4273895392 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c0a0030 con 0x7fbb8c133420 2023-09-30T22:16:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fe14bfff700 1 -- 172.21.15.89:0/1731431702 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c020070 con 0x7fe16409e410 2023-09-30T22:16:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.466+0000 7fe14bfff700 1 -- 172.21.15.89:0/1731431702 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c027680 con 0x7fe16409e410 2023-09-30T22:16:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.467+0000 7fbb908eb700 1 --2- 172.21.15.89:0/4273895392 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133420 0x7fbb8c1337f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0098f0 tx=0x7fbb7c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b78e940905af3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.467+0000 7fe162ffd700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e000 0x7fe1641229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:16:56.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.467+0000 7fbb8b7fe700 1 -- 172.21.15.89:0/4273895392 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00b8a0 con 0x7fbb8c133420 2023-09-30T22:16:56.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.467+0000 7fbb8b7fe700 1 -- 172.21.15.89:0/4273895392 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c00ba00 con 0x7fbb8c133420 2023-09-30T22:16:56.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.467+0000 7fbb8b7fe700 1 -- 172.21.15.89:0/4273895392 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00bd90 con 0x7fbb8c133420 2023-09-30T22:16:56.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.468+0000 7fe14bfff700 1 -- 172.21.15.89:0/1731431702 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe15c012430 con 0x7fe16409e410 2023-09-30T22:16:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.468+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1440052e0 con 0x7fe16409e410 2023-09-30T22:16:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.468+0000 7fe1637fe700 1 -- 172.21.15.89:0/1731431702 >> v1:172.21.15.89:6816/126953 conn(0x7fe15002f510 legacy=0x7fe1500319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.468+0000 7fbb92b4f700 1 -- 172.21.15.89:0/4273895392 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133420 msgr2=0x7fbb8c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.468+0000 7fbb92b4f700 1 --2- 172.21.15.89:0/4273895392 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133420 0x7fbb8c1337f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0098f0 tx=0x7fbb7c009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.468+0000 7fe14bfff700 1 -- 172.21.15.89:0/1731431702 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fe15c022070 con 0x7fe16409e410 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.469+0000 7fbb92b4f700 1 -- 172.21.15.89:0/4273895392 shutdown_connections 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.469+0000 7fbb92b4f700 1 --2- 172.21.15.89:0/4273895392 >> v2:172.21.15.89:3300/0 conn(0x7fbb8c134260 0x7fbb8c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.469+0000 7fbb92b4f700 1 --2- 172.21.15.89:0/4273895392 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133420 0x7fbb8c1337f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.469+0000 7fbb92b4f700 1 --2- 172.21.15.89:0/4273895392 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133e30 0x7fbb8c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.469+0000 7fbb92b4f700 1 -- 172.21.15.89:0/4273895392 >> 172.21.15.89:0/4273895392 conn(0x7fbb8c0a4210 msgr2=0x7fbb8c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.469+0000 7fbb92b4f700 1 -- 172.21.15.89:0/4273895392 shutdown_connections 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.469+0000 7fbb92b4f700 1 -- 172.21.15.89:0/4273895392 wait complete. 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.470+0000 7fbb92b4f700 1 Processor -- start 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.470+0000 7fbb92b4f700 1 -- start start 2023-09-30T22:16:56.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb92b4f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133420 0x7fbb8c09c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb92b4f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133e30 0x7fbb8c09ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb92b4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c134260 0x7fbb8c09d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb92b4f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c135850 con 0x7fbb8c134260 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb92b4f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c1347e0 con 0x7fbb8c133e30 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb92b4f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c135c30 con 0x7fbb8c133420 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb8bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133e30 0x7fbb8c09ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb8bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133e30 0x7fbb8c09ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49440/0 (socket says 172.21.15.89:49440) 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb8bfff700 1 -- 172.21.15.89:0/3506333932 learned_addr learned my addr 172.21.15.89:0/3506333932 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb910ec700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c134260 0x7fbb8c09d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb908eb700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133420 0x7fbb8c09c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb8bfff700 1 -- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133420 msgr2=0x7fbb8c09c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb8bfff700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133420 0x7fbb8c09c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb8bfff700 1 -- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c134260 msgr2=0x7fbb8c09d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb8bfff700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c134260 0x7fbb8c09d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb8bfff700 1 -- 172.21.15.89:0/3506333932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c009550 con 0x7fbb8c133e30 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.471+0000 7fbb910ec700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c134260 0x7fbb8c09d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.472+0000 7fbb8bfff700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133e30 0x7fbb8c09ce00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000b500 tx=0x7fbb8000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.474+0000 7fbb897fa700 1 -- 172.21.15.89:0/3506333932 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80011830 con 0x7fbb8c133e30 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.474+0000 7fbb897fa700 1 -- 172.21.15.89:0/3506333932 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb800119b0 con 0x7fbb8c133e30 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.474+0000 7fbb897fa700 1 -- 172.21.15.89:0/3506333932 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80011d60 con 0x7fbb8c133e30 2023-09-30T22:16:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.474+0000 7fe14bfff700 1 -- 172.21.15.89:0/1731431702 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe15c011750 con 0x7fe16409e410 2023-09-30T22:16:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.475+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c136960 con 0x7fbb8c133e30 2023-09-30T22:16:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.475+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c09e230 con 0x7fbb8c133e30 2023-09-30T22:16:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.476+0000 7fbb897fa700 1 -- 172.21.15.89:0/3506333932 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbb8001a400 con 0x7fbb8c133e30 2023-09-30T22:16:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.476+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c074ea0 con 0x7fbb8c133e30 2023-09-30T22:16:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.476+0000 7fbb908eb700 1 -- 172.21.15.89:0/3506333932 >> v1:172.21.15.89:6816/126953 conn(0x7fbb7802f560 legacy=0x7fbb78031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.476+0000 7fbb897fa700 1 -- 172.21.15.89:0/3506333932 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fbb8003fc10 con 0x7fbb8c133e30 2023-09-30T22:16:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.481+0000 7fbb897fa700 1 -- 172.21.15.89:0/3506333932 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb8000f8d0 con 0x7fbb8c133e30 2023-09-30T22:16:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.591+0000 7f8ad6ffd700 1 -- 172.21.15.89:0/2738631686 >> v1:172.21.15.89:6816/126953 conn(0x7f8ac402f510 legacy=0x7f8ac40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.602+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8ad809f420 con 0x7f8ad809e000 2023-09-30T22:16:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.602+0000 7f8abf7fe700 1 -- 172.21.15.89:0/2738631686 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8acc0192c0 con 0x7f8ad809e000 2023-09-30T22:16:56.603 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 >> v1:172.21.15.89:6816/126953 conn(0x7f8ac402f510 legacy=0x7f8ac40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad809e000 msgr2=0x7f8ad81a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 --2- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad809e000 0x7f8ad81a2070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00c4e0 tx=0x7f8acc00c7f0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 shutdown_connections 2023-09-30T22:16:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 --2- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad809e440 0x7f8ad81a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 --2- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad809d6f0 0x7f8ad81a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 --2- 172.21.15.89:0/2738631686 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad809e000 0x7f8ad81a2070 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 >> 172.21.15.89:0/2738631686 conn(0x7f8ad8066f50 msgr2=0x7f8ad81b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:56.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 shutdown_connections 2023-09-30T22:16:56.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.604+0000 7f8add324700 1 -- 172.21.15.89:0/2738631686 wait complete. 2023-09-30T22:16:56.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:16:56.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.629+0000 7f8779d9b700 1 -- 172.21.15.89:0/3881910904 >> v1:172.21.15.89:6816/126953 conn(0x7f875c02f510 legacy=0x7f875c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:56.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.631+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f877c06c360 con 0x7f877c09d6f0 2023-09-30T22:16:56.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.631+0000 7f876affd700 1 -- 172.21.15.89:0/3881910904 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f876c021070 con 0x7f877c09d6f0 2023-09-30T22:16:56.632 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:56.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 >> v1:172.21.15.89:6816/126953 conn(0x7f875c02f510 legacy=0x7f875c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:56.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 msgr2=0x7f877c1a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c1a5ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f876c00e9b0 tx=0x7f876c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 shutdown_connections 2023-09-30T22:16:56.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c09e410 0x7f877c1a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f877c09d6f0 0x7f877c1a5ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 --2- 172.21.15.89:0/3881910904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f877c09e000 0x7f877c1a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 >> 172.21.15.89:0/3881910904 conn(0x7f877c066f50 msgr2=0x7f877c1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:56.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 shutdown_connections 2023-09-30T22:16:56.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.633+0000 7f8780c83700 1 -- 172.21.15.89:0/3881910904 wait complete. 2023-09-30T22:16:56.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:16:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.669+0000 7fe1637fe700 1 -- 172.21.15.89:0/1731431702 >> v1:172.21.15.89:6816/126953 conn(0x7fe15002f510 legacy=0x7fe1500319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.672+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe144005860 con 0x7fe16409e410 2023-09-30T22:16:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.672+0000 7fe14bfff700 1 -- 172.21.15.89:0/1731431702 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe15c011390 con 0x7fe16409e410 2023-09-30T22:16:56.673 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:16:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 >> v1:172.21.15.89:6816/126953 conn(0x7fe15002f510 legacy=0x7fe1500319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e410 msgr2=0x7fe164122ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e410 0x7fe164122ef0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe15c004270 tx=0x7fe15c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 shutdown_connections 2023-09-30T22:16:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16409e410 0x7fe164122ef0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16409d6f0 0x7fe164122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 --2- 172.21.15.89:0/1731431702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16409e000 0x7fe1641229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 >> 172.21.15.89:0/1731431702 conn(0x7fe164066f50 msgr2=0x7fe1641309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 shutdown_connections 2023-09-30T22:16:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fe169a79700 1 -- 172.21.15.89:0/1731431702 wait complete. 2023-09-30T22:16:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.674+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbb8c06b250 con 0x7fbb8c133e30 2023-09-30T22:16:56.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.675+0000 7fbb897fa700 1 -- 172.21.15.89:0/3506333932 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb80015020 con 0x7fbb8c133e30 2023-09-30T22:16:56.677 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:16:56.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb908eb700 1 -- 172.21.15.89:0/3506333932 >> v1:172.21.15.89:6816/126953 conn(0x7fbb7802f560 legacy=0x7fbb78031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:56.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 >> v1:172.21.15.89:6816/126953 conn(0x7fbb7802f560 legacy=0x7fbb78031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:56.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133e30 msgr2=0x7fbb8c09ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:56.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133e30 0x7fbb8c09ce00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000b500 tx=0x7fbb8000b8c0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 shutdown_connections 2023-09-30T22:16:56.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c134260 0x7fbb8c09d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c133420 0x7fbb8c09c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 --2- 172.21.15.89:0/3506333932 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c133e30 0x7fbb8c09ce00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:56.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 >> 172.21.15.89:0/3506333932 conn(0x7fbb8c0a4210 msgr2=0x7fbb8c0a3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:56.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 shutdown_connections 2023-09-30T22:16:56.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:56.677+0000 7fbb92b4f700 1 -- 172.21.15.89:0/3506333932 wait complete. 2023-09-30T22:16:56.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:16:56.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:16:57.617 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:57.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:57.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:57.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:57.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.769+0000 7f993adcc700 1 Processor -- start 2023-09-30T22:16:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.769+0000 7f993adcc700 1 -- start start 2023-09-30T22:16:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f993adcc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f9934130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f993adcc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9934130e50 0x7f993406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f993adcc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340716a0 con 0x7f993406fb20 2023-09-30T22:16:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f993adcc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993409a390 con 0x7f9934130e50 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f993adcc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9934071290 con 0x7f9934130540 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9933fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9934130e50 0x7f993406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9938b68700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f9934130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9933fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9934130e50 0x7f993406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38642/0 (socket says 172.21.15.89:38642) 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9933fff700 1 -- 172.21.15.89:0/3672445341 learned_addr learned my addr 172.21.15.89:0/3672445341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9933fff700 1 -- 172.21.15.89:0/3672445341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 msgr2=0x7f9934130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9933fff700 1 --2- 172.21.15.89:0/3672445341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f9934130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9933fff700 1 -- 172.21.15.89:0/3672445341 >> v1:172.21.15.89:6789/0 conn(0x7f993406fb20 legacy=0x7f9934137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9933fff700 1 -- 172.21.15.89:0/3672445341 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934130340 con 0x7f9934130e50 2023-09-30T22:16:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f99337fe700 1 -- 172.21.15.89:0/3672445341 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712892875 0 0) 0x7f99340716a0 con 0x7f993406fb20 2023-09-30T22:16:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9938b68700 1 --2- 172.21.15.89:0/3672445341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f9934130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.770+0000 7f9933fff700 1 --2- 172.21.15.89:0/3672445341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9934130e50 0x7f993406f5e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f992000bb90 tx=0x7f992000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a886f21744e679b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.771+0000 7f99337fe700 1 -- 172.21.15.89:0/3672445341 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000dc70 con 0x7f9934130e50 2023-09-30T22:16:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.771+0000 7f99337fe700 1 -- 172.21.15.89:0/3672445341 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992000ddd0 con 0x7f9934130e50 2023-09-30T22:16:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.771+0000 7f99337fe700 1 -- 172.21.15.89:0/3672445341 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920012990 con 0x7f9934130e50 2023-09-30T22:16:57.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.772+0000 7f993adcc700 1 -- 172.21.15.89:0/3672445341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9934130e50 msgr2=0x7f993406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.772+0000 7f993adcc700 1 --2- 172.21.15.89:0/3672445341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9934130e50 0x7f993406f5e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f992000bb90 tx=0x7f992000bea0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.772+0000 7f993adcc700 1 -- 172.21.15.89:0/3672445341 shutdown_connections 2023-09-30T22:16:57.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.772+0000 7f993adcc700 1 --2- 172.21.15.89:0/3672445341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f9934130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.772+0000 7f993adcc700 1 --2- 172.21.15.89:0/3672445341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9934130e50 0x7f993406f5e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.772+0000 7f993adcc700 1 -- 172.21.15.89:0/3672445341 >> 172.21.15.89:0/3672445341 conn(0x7f99340a2aa0 msgr2=0x7f99340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:57.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.772+0000 7f993adcc700 1 -- 172.21.15.89:0/3672445341 shutdown_connections 2023-09-30T22:16:57.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.772+0000 7f993adcc700 1 -- 172.21.15.89:0/3672445341 wait complete. 2023-09-30T22:16:57.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.773+0000 7f993adcc700 1 Processor -- start 2023-09-30T22:16:57.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.773+0000 7f993adcc700 1 -- start start 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f993adcc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f993406fb20 0x7f993411a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f993adcc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f993411ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f993adcc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9934130e50 0x7f993411b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f993adcc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993411c240 con 0x7f993406fb20 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f993adcc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993411d490 con 0x7f9934130e50 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f993adcc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f993411c820 con 0x7f9934130540 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9933fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f993411ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9933fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f993411ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38558/0 (socket says 172.21.15.89:38558) 2023-09-30T22:16:57.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9933fff700 1 -- 172.21.15.89:0/3331776131 learned_addr learned my addr 172.21.15.89:0/3331776131 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9939369700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9934130e50 0x7f993411b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9938b68700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f993406fb20 0x7f993411a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9939369700 1 -- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 msgr2=0x7f993411ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9939369700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f993411ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9939369700 1 -- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f993406fb20 msgr2=0x7f993411a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.774+0000 7f9939369700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f993406fb20 0x7f993411a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f9939369700 1 -- 172.21.15.89:0/3331776131 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99280094a0 con 0x7f9934130e50 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f9933fff700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f993411ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f9938b68700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f993406fb20 0x7f993411a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:57.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f9939369700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9934130e50 0x7f993411b0b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c760 tx=0x7f992c00ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f99317fa700 1 -- 172.21.15.89:0/3331776131 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c003e30 con 0x7f9934130e50 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f99317fa700 1 -- 172.21.15.89:0/3331776131 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c005070 con 0x7f9934130e50 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992000b7f0 con 0x7f9934130e50 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f99317fa700 1 -- 172.21.15.89:0/3331776131 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c005420 con 0x7f9934130e50 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.775+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99340a0810 con 0x7f9934130e50 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.776+0000 7f99317fa700 1 -- 172.21.15.89:0/3331776131 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f992c005580 con 0x7f9934130e50 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.776+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9934074ea0 con 0x7f9934130e50 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.776+0000 7f9938b68700 1 -- 172.21.15.89:0/3331776131 >> v1:172.21.15.89:6816/126953 conn(0x7f991802f540 legacy=0x7f99180319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.776+0000 7f99317fa700 1 -- 172.21.15.89:0/3331776131 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f992c03f7f0 con 0x7f9934130e50 2023-09-30T22:16:57.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.785+0000 7f99317fa700 1 -- 172.21.15.89:0/3331776131 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f992c01b400 con 0x7f9934130e50 2023-09-30T22:16:57.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.799+0000 7f20bf363700 1 Processor -- start 2023-09-30T22:16:57.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.799+0000 7f20bf363700 1 -- start start 2023-09-30T22:16:57.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bf363700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20b809d6f0 0x7f20b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bf363700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e000 0x7f20b81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bf363700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e410 0x7f20b8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bf363700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b806b460 con 0x7f20b809d6f0 2023-09-30T22:16:57.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bf363700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b809b120 con 0x7f20b809e000 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bf363700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20b809a7e0 con 0x7f20b809e410 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bd900700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e410 0x7f20b8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bd900700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e410 0x7f20b8133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38572/0 (socket says 172.21.15.89:38572) 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bd900700 1 -- 172.21.15.89:0/756636406 learned_addr learned my addr 172.21.15.89:0/756636406 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.800+0000 7f20bc8fe700 1 --2- 172.21.15.89:0/756636406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e000 0x7f20b81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20bd900700 1 -- 172.21.15.89:0/756636406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e000 msgr2=0x7f20b81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20bd900700 1 --2- 172.21.15.89:0/756636406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e000 0x7f20b81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20bd900700 1 -- 172.21.15.89:0/756636406 >> v2:172.21.15.89:3300/0 conn(0x7f20b809d6f0 msgr2=0x7f20b809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20bd900700 1 --2- 172.21.15.89:0/756636406 >> v2:172.21.15.89:3300/0 conn(0x7f20b809d6f0 0x7f20b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20bd900700 1 -- 172.21.15.89:0/756636406 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b809a500 con 0x7f20b809e410 2023-09-30T22:16:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20bd900700 1 --2- 172.21.15.89:0/756636406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e410 0x7f20b8133f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009e80 tx=0x7f20b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd0f66618812f82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20affff700 1 -- 172.21.15.89:0/756636406 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b000e070 con 0x7f20b809e410 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20affff700 1 -- 172.21.15.89:0/756636406 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b000b8d0 con 0x7f20b809e410 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.801+0000 7f20affff700 1 -- 172.21.15.89:0/756636406 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b000bc80 con 0x7f20b809e410 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 -- 172.21.15.89:0/756636406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e410 msgr2=0x7f20b8133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 --2- 172.21.15.89:0/756636406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e410 0x7f20b8133f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009e80 tx=0x7f20b000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 -- 172.21.15.89:0/756636406 shutdown_connections 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 --2- 172.21.15.89:0/756636406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e410 0x7f20b8133f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 --2- 172.21.15.89:0/756636406 >> v2:172.21.15.89:3300/0 conn(0x7f20b809d6f0 0x7f20b809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 --2- 172.21.15.89:0/756636406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e000 0x7f20b81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 -- 172.21.15.89:0/756636406 >> 172.21.15.89:0/756636406 conn(0x7f20b8066f50 msgr2=0x7f20b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 -- 172.21.15.89:0/756636406 shutdown_connections 2023-09-30T22:16:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.802+0000 7f20bf363700 1 -- 172.21.15.89:0/756636406 wait complete. 2023-09-30T22:16:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.803+0000 7f20bf363700 1 Processor -- start 2023-09-30T22:16:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.803+0000 7f20bf363700 1 -- start start 2023-09-30T22:16:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bf363700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b809d6f0 0x7f20b81a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bf363700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e000 0x7f20b81a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bf363700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e410 0x7f20b81a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bf363700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81aa250 con 0x7f20b809d6f0 2023-09-30T22:16:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bf363700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81a9890 con 0x7f20b809e410 2023-09-30T22:16:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bf363700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81a8c60 con 0x7f20b809e000 2023-09-30T22:16:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bd0ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b809d6f0 0x7f20b81a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bd0ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b809d6f0 0x7f20b81a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38678/0 (socket says 172.21.15.89:38678) 2023-09-30T22:16:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bd0ff700 1 -- 172.21.15.89:0/3879407831 learned_addr learned my addr 172.21.15.89:0/3879407831 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bc8fe700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e000 0x7f20b81a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.804+0000 7f20bd900700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e410 0x7f20b81a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bc8fe700 1 -- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e410 msgr2=0x7f20b81a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bc8fe700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e410 0x7f20b81a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bc8fe700 1 -- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b809d6f0 msgr2=0x7f20b81a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bc8fe700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b809d6f0 0x7f20b81a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bc8fe700 1 -- 172.21.15.89:0/3879407831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0009b30 con 0x7f20b809e000 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bd0ff700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b809d6f0 0x7f20b81a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bc8fe700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e000 0x7f20b81a62b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e930 tx=0x7f20a800ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20adffb700 1 -- 172.21.15.89:0/3879407831 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8013680 con 0x7f20b809e000 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20adffb700 1 -- 172.21.15.89:0/3879407831 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a80137e0 con 0x7f20b809e000 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b81a9020 con 0x7f20b809e000 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20adffb700 1 -- 172.21.15.89:0/3879407831 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8013b50 con 0x7f20b809e000 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.805+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b809c710 con 0x7f20b809e000 2023-09-30T22:16:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.806+0000 7f20adffb700 1 -- 172.21.15.89:0/3879407831 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f20a8013cb0 con 0x7f20b809e000 2023-09-30T22:16:57.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.806+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b8074ea0 con 0x7f20b809e000 2023-09-30T22:16:57.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.807+0000 7f20adffb700 1 -- 172.21.15.89:0/3879407831 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f20a8040000 con 0x7f20b809e000 2023-09-30T22:16:57.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.812+0000 7f20bd0ff700 1 -- 172.21.15.89:0/3879407831 >> v1:172.21.15.89:6816/126953 conn(0x7f209c02f510 legacy=0x7f209c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:57.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.815+0000 7f20adffb700 1 -- 172.21.15.89:0/3879407831 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20a801adc0 con 0x7f20b809e000 2023-09-30T22:16:57.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f9b8d431700 1 Processor -- start 2023-09-30T22:16:57.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f9b8d431700 1 -- start start 2023-09-30T22:16:57.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f03893d0700 1 Processor -- start 2023-09-30T22:16:57.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f03893d0700 1 -- start start 2023-09-30T22:16:57.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f9b8d431700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b88133420 0x7f9b881337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f9b8d431700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88133e30 0x7f9b8812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f9b8d431700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8806b990 con 0x7f9b88134260 2023-09-30T22:16:57.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f9b8d431700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b880a0590 con 0x7f9b88133420 2023-09-30T22:16:57.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f9b8d431700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8809fb80 con 0x7f9b88133e30 2023-09-30T22:16:57.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f03893d0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f038412cf60 0x7f038406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f03893d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066410 0x7f038412f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f03893d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038412cd90 con 0x7f0384066880 2023-09-30T22:16:57.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f03893d0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038412d330 con 0x7f038412cf60 2023-09-30T22:16:57.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.840+0000 7f03893d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0384071780 con 0x7f0384066410 2023-09-30T22:16:57.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b867fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88133e30 0x7f9b8812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b867fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88133e30 0x7f9b8812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38590/0 (socket says 172.21.15.89:38590) 2023-09-30T22:16:57.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b867fc700 1 -- 172.21.15.89:0/2395354787 learned_addr learned my addr 172.21.15.89:0/2395354787 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:57.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f03827fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066410 0x7f038412f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f03827fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066410 0x7f038412f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38598/0 (socket says 172.21.15.89:38598) 2023-09-30T22:16:57.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f03827fc700 1 -- 172.21.15.89:0/2370716734 learned_addr learned my addr 172.21.15.89:0/2370716734 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:57.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b867fc700 1 -- 172.21.15.89:0/2395354787 >> v2:172.21.15.89:3300/0 conn(0x7f9b88133420 msgr2=0x7f9b881337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:57.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b867fc700 1 --2- 172.21.15.89:0/2395354787 >> v2:172.21.15.89:3300/0 conn(0x7f9b88133420 0x7f9b881337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b867fc700 1 -- 172.21.15.89:0/2395354787 >> v1:172.21.15.89:6789/0 conn(0x7f9b88134260 legacy=0x7f9b881304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b867fc700 1 -- 172.21.15.89:0/2395354787 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880a0030 con 0x7f9b88133e30 2023-09-30T22:16:57.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f03827fc700 1 -- 172.21.15.89:0/2370716734 >> v2:172.21.15.89:3300/0 conn(0x7f038412cf60 msgr2=0x7f038406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:57.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f03827fc700 1 --2- 172.21.15.89:0/2370716734 >> v2:172.21.15.89:3300/0 conn(0x7f038412cf60 0x7f038406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f03827fc700 1 -- 172.21.15.89:0/2370716734 >> v1:172.21.15.89:6789/0 conn(0x7f0384066880 legacy=0x7f038412fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f03827fc700 1 -- 172.21.15.89:0/2370716734 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384071a40 con 0x7f0384066410 2023-09-30T22:16:57.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f03827fc700 1 --2- 172.21.15.89:0/2370716734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066410 0x7f038412f8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f037800bb90 tx=0x7f037800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc60d5fa01809d47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:57.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b867fc700 1 --2- 172.21.15.89:0/2395354787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88133e30 0x7f9b8812ff90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0098f0 tx=0x7f9b7c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c5b13912d582fd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:57.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f0381ffb700 1 -- 172.21.15.89:0/2370716734 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037800dd50 con 0x7f0384066410 2023-09-30T22:16:57.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f9b85ffb700 1 -- 172.21.15.89:0/2395354787 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00b8a0 con 0x7f9b88133e30 2023-09-30T22:16:57.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b85ffb700 1 -- 172.21.15.89:0/2395354787 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c00ba00 con 0x7f9b88133e30 2023-09-30T22:16:57.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b85ffb700 1 -- 172.21.15.89:0/2395354787 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00bd90 con 0x7f9b88133e30 2023-09-30T22:16:57.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.841+0000 7f0381ffb700 1 -- 172.21.15.89:0/2370716734 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378010470 con 0x7f0384066410 2023-09-30T22:16:57.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f0381ffb700 1 -- 172.21.15.89:0/2370716734 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378010800 con 0x7f0384066410 2023-09-30T22:16:57.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b8d431700 1 -- 172.21.15.89:0/2395354787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88133e30 msgr2=0x7f9b8812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b8d431700 1 --2- 172.21.15.89:0/2395354787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88133e30 0x7f9b8812ff90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0098f0 tx=0x7f9b7c009c00 comp rx=0 tx=0).stop 2023-09-30T22:16:57.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f03893d0700 1 -- 172.21.15.89:0/2370716734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066410 msgr2=0x7f038412f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f03893d0700 1 --2- 172.21.15.89:0/2370716734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066410 0x7f038412f8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f037800bb90 tx=0x7f037800bea0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b8d431700 1 -- 172.21.15.89:0/2395354787 shutdown_connections 2023-09-30T22:16:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b8d431700 1 --2- 172.21.15.89:0/2395354787 >> v2:172.21.15.89:3300/0 conn(0x7f9b88133420 0x7f9b881337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b8d431700 1 --2- 172.21.15.89:0/2395354787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88133e30 0x7f9b8812ff90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b8d431700 1 -- 172.21.15.89:0/2395354787 >> 172.21.15.89:0/2395354787 conn(0x7f9b880a4210 msgr2=0x7f9b880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f03893d0700 1 -- 172.21.15.89:0/2370716734 shutdown_connections 2023-09-30T22:16:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f03893d0700 1 --2- 172.21.15.89:0/2370716734 >> v2:172.21.15.89:3300/0 conn(0x7f038412cf60 0x7f038406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f03893d0700 1 --2- 172.21.15.89:0/2370716734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066410 0x7f038412f8b0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f037800bb90 tx=0x7f037800bea0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f03893d0700 1 -- 172.21.15.89:0/2370716734 >> 172.21.15.89:0/2370716734 conn(0x7f03840a2aa0 msgr2=0x7f03840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b8d431700 1 -- 172.21.15.89:0/2395354787 shutdown_connections 2023-09-30T22:16:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f03893d0700 1 -- 172.21.15.89:0/2370716734 shutdown_connections 2023-09-30T22:16:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f03893d0700 1 -- 172.21.15.89:0/2370716734 wait complete. 2023-09-30T22:16:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.842+0000 7f9b8d431700 1 -- 172.21.15.89:0/2395354787 wait complete. 2023-09-30T22:16:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.843+0000 7f03893d0700 1 Processor -- start 2023-09-30T22:16:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.843+0000 7f03893d0700 1 -- start start 2023-09-30T22:16:57.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f9b8d431700 1 Processor -- start 2023-09-30T22:16:57.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f9b8d431700 1 -- start start 2023-09-30T22:16:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03893d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066880 0x7f0384118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03893d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038412cf60 0x7f0384119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03893d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0384119750 0x7f03841caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03893d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038411c110 con 0x7f038412cf60 2023-09-30T22:16:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03893d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03840a1590 con 0x7f0384119750 2023-09-30T22:16:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03893d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f038411b8a0 con 0x7f0384066880 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03827fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038412cf60 0x7f0384119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03827fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038412cf60 0x7f0384119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38710/0 (socket says 172.21.15.89:38710) 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03827fc700 1 -- 172.21.15.89:0/2396626633 learned_addr learned my addr 172.21.15.89:0/2396626633 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f0382ffd700 1 --2- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066880 0x7f0384118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03827fc700 1 -- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066880 msgr2=0x7f0384118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03827fc700 1 --2- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066880 0x7f0384118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03827fc700 1 -- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0384119750 msgr2=0x7f03841caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03827fc700 1 --2- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0384119750 0x7f03841caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.844+0000 7f03827fc700 1 -- 172.21.15.89:0/2396626633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037800b7f0 con 0x7f038412cf60 2023-09-30T22:16:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f03827fc700 1 --2- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038412cf60 0x7f0384119210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0378015000 tx=0x7f0378004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:57.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b8d431700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b88133420 0x7f9b8809c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b8d431700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b88133e30 0x7f9b8809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b8d431700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88134260 0x7f9b8809d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b8d431700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88120510 con 0x7f9b88133e30 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b8d431700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8812d580 con 0x7f9b88133420 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b8d431700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8811ff70 con 0x7f9b88134260 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b88133e30 0x7f9b8809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b88133e30 0x7f9b8809cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38722/0 (socket says 172.21.15.89:38722) 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 -- 172.21.15.89:0/576535700 learned_addr learned my addr 172.21.15.89:0/576535700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b877fe700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88134260 0x7f9b8809d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b86ffd700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b88133420 0x7f9b8809c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 -- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88134260 msgr2=0x7f9b8809d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88134260 0x7f9b8809d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 -- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b88133420 msgr2=0x7f9b8809c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b88133420 0x7f9b8809c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 -- 172.21.15.89:0/576535700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c009550 con 0x7f9b88133e30 2023-09-30T22:16:57.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.845+0000 7f9b867fc700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b88133e30 0x7f9b8809cc20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:57.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f036b7fe700 1 -- 172.21.15.89:0/2396626633 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378003ab0 con 0x7f038412cf60 2023-09-30T22:16:57.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f036b7fe700 1 -- 172.21.15.89:0/2396626633 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378003c30 con 0x7f038412cf60 2023-09-30T22:16:57.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f036b7fe700 1 -- 172.21.15.89:0/2396626633 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03780105e0 con 0x7f038412cf60 2023-09-30T22:16:57.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f9b6f7fe700 1 -- 172.21.15.89:0/576535700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c004550 con 0x7f9b88133e30 2023-09-30T22:16:57.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f9b6f7fe700 1 -- 172.21.15.89:0/576535700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c01f070 con 0x7f9b88133e30 2023-09-30T22:16:57.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f9b6f7fe700 1 -- 172.21.15.89:0/576535700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0038e0 con 0x7f9b88133e30 2023-09-30T22:16:57.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b88131940 con 0x7f9b88133e30 2023-09-30T22:16:57.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8812db80 con 0x7f9b88133e30 2023-09-30T22:16:57.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.846+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038411bb70 con 0x7f038412cf60 2023-09-30T22:16:57.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.847+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038411a200 con 0x7f038412cf60 2023-09-30T22:16:57.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.848+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038409b4f0 con 0x7f038412cf60 2023-09-30T22:16:57.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.848+0000 7f036b7fe700 1 -- 172.21.15.89:0/2396626633 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f03780214d0 con 0x7f038412cf60 2023-09-30T22:16:57.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.847+0000 7f9b6f7fe700 1 -- 172.21.15.89:0/576535700 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9b7c003a40 con 0x7f9b88133e30 2023-09-30T22:16:57.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.847+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b88131400 con 0x7f9b88133e30 2023-09-30T22:16:57.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.847+0000 7f9b6f7fe700 1 -- 172.21.15.89:0/576535700 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f9b7c021070 con 0x7f9b88133e30 2023-09-30T22:16:57.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.848+0000 7f9b86ffd700 1 -- 172.21.15.89:0/576535700 >> v1:172.21.15.89:6816/126953 conn(0x7f9b7402f510 legacy=0x7f9b740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:57.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.848+0000 7f0382ffd700 1 -- 172.21.15.89:0/2396626633 >> v1:172.21.15.89:6816/126953 conn(0x7f037002f510 legacy=0x7f03700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.848+0000 7f036b7fe700 1 -- 172.21.15.89:0/2396626633 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f0378045420 con 0x7f038412cf60 2023-09-30T22:16:57.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.857+0000 7f9b6f7fe700 1 -- 172.21.15.89:0/576535700 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b7c0193b0 con 0x7f9b88133e30 2023-09-30T22:16:57.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.865+0000 7f036b7fe700 1 -- 172.21.15.89:0/2396626633 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f037800e950 con 0x7f038412cf60 2023-09-30T22:16:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.977+0000 7f9938b68700 1 -- 172.21.15.89:0/3331776131 >> v1:172.21.15.89:6816/126953 conn(0x7f991802f540 legacy=0x7f99180319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:57.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.989+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f993406b9e0 con 0x7f9934130e50 2023-09-30T22:16:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.989+0000 7f99317fa700 1 -- 172.21.15.89:0/3331776131 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f992c019af0 con 0x7f9934130e50 2023-09-30T22:16:57.990 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 >> v1:172.21.15.89:6816/126953 conn(0x7f991802f540 legacy=0x7f99180319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9934130e50 msgr2=0x7f993411b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9934130e50 0x7f993411b0b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c760 tx=0x7f992c00ca70 comp rx=0 tx=0).stop 2023-09-30T22:16:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 shutdown_connections 2023-09-30T22:16:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9934130e50 0x7f993411b0b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f993406fb20 0x7f993411a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 --2- 172.21.15.89:0/3331776131 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9934130540 0x7f993411ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 >> 172.21.15.89:0/3331776131 conn(0x7f99340a2aa0 msgr2=0x7f993412d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 shutdown_connections 2023-09-30T22:16:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:57.992+0000 7f993adcc700 1 -- 172.21.15.89:0/3331776131 wait complete. 2023-09-30T22:16:58.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:16:58.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.013+0000 7f20bd0ff700 1 -- 172.21.15.89:0/3879407831 >> v1:172.21.15.89:6816/126953 conn(0x7f209c02f510 legacy=0x7f209c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:58.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.021+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20b806c360 con 0x7f20b809e000 2023-09-30T22:16:58.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.022+0000 7f20adffb700 1 -- 172.21.15.89:0/3879407831 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20a8017020 con 0x7f20b809e000 2023-09-30T22:16:58.022 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 >> v1:172.21.15.89:6816/126953 conn(0x7f209c02f510 legacy=0x7f209c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e000 msgr2=0x7f20b81a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e000 0x7f20b81a62b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e930 tx=0x7f20a800ec40 comp rx=0 tx=0).stop 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 shutdown_connections 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20b809e410 0x7f20b81a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20b809d6f0 0x7f20b81a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 --2- 172.21.15.89:0/3879407831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20b809e000 0x7f20b81a62b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 >> 172.21.15.89:0/3879407831 conn(0x7f20b8066f50 msgr2=0x7f20b81bac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 shutdown_connections 2023-09-30T22:16:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.024+0000 7f20bf363700 1 -- 172.21.15.89:0/3879407831 wait complete. 2023-09-30T22:16:58.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:16:58.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.049+0000 7f9b86ffd700 1 -- 172.21.15.89:0/576535700 >> v1:172.21.15.89:6816/126953 conn(0x7f9b7402f510 legacy=0x7f9b740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:58.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.049+0000 7f0382ffd700 1 -- 172.21.15.89:0/2396626633 >> v1:172.21.15.89:6816/126953 conn(0x7f037002f510 legacy=0x7f03700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.060+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0384074ea0 con 0x7f038412cf60 2023-09-30T22:16:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.060+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9b88074ea0 con 0x7f9b88133e30 2023-09-30T22:16:58.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.060+0000 7f036b7fe700 1 -- 172.21.15.89:0/2396626633 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f037800e360 con 0x7f038412cf60 2023-09-30T22:16:58.061 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:16:58.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.060+0000 7f9b6f7fe700 1 -- 172.21.15.89:0/576535700 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b7c02b020 con 0x7f9b88133e30 2023-09-30T22:16:58.062 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:16:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 >> v1:172.21.15.89:6816/126953 conn(0x7f037002f510 legacy=0x7f03700319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038412cf60 msgr2=0x7f0384119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 --2- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038412cf60 0x7f0384119210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0378015000 tx=0x7f0378004750 comp rx=0 tx=0).stop 2023-09-30T22:16:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 shutdown_connections 2023-09-30T22:16:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 --2- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0384119750 0x7f03841caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 --2- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0384066880 0x7f0384118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 --2- 172.21.15.89:0/2396626633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038412cf60 0x7f0384119210 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 >> 172.21.15.89:0/2396626633 conn(0x7f03840a2aa0 msgr2=0x7f0384130130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:58.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 shutdown_connections 2023-09-30T22:16:58.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f03893d0700 1 -- 172.21.15.89:0/2396626633 wait complete. 2023-09-30T22:16:58.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 >> v1:172.21.15.89:6816/126953 conn(0x7f9b7402f510 legacy=0x7f9b740319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b88133e30 msgr2=0x7f9b8809cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b88133e30 0x7f9b8809cc20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c004750 comp rx=0 tx=0).stop 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 shutdown_connections 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b88134260 0x7f9b8809d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b88133420 0x7f9b8809c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 --2- 172.21.15.89:0/576535700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b88133e30 0x7f9b8809cc20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 >> 172.21.15.89:0/576535700 conn(0x7f9b880a4210 msgr2=0x7f9b880a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 shutdown_connections 2023-09-30T22:16:58.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:58.062+0000 7f9b8d431700 1 -- 172.21.15.89:0/576535700 wait complete. 2023-09-30T22:16:58.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:16:58.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:16:59.011 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:16:59.038 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:16:59.077 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:16:59.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:16:59.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.164+0000 7faf43beb700 1 Processor -- start 2023-09-30T22:16:59.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.164+0000 7faf43beb700 1 -- start start 2023-09-30T22:16:59.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf43beb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf43beb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf43beb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c12cd90 con 0x7faf3c12f8b0 2023-09-30T22:16:59.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf43beb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c12d330 con 0x7faf3c12cf60 2023-09-30T22:16:59.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf43beb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c071780 con 0x7faf3c130430 2023-09-30T22:16:59.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf41186700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faf3c12f8b0 0x7faf3c12fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50616/0 (socket says 172.21.15.89:50616) 2023-09-30T22:16:59.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf41987700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf41186700 1 -- 172.21.15.89:0/4194984078 learned_addr learned my addr 172.21.15.89:0/4194984078 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:16:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.165+0000 7faf42188700 1 --2- 172.21.15.89:0/4194984078 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf40985700 1 -- 172.21.15.89:0/4194984078 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753556339 0 0) 0x7faf3c12cd90 con 0x7faf3c12f8b0 2023-09-30T22:16:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf40985700 1 -- 172.21.15.89:0/4194984078 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf280035b0 con 0x7faf3c12f8b0 2023-09-30T22:16:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf41987700 1 -- 172.21.15.89:0/4194984078 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 msgr2=0x7faf3c137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf41987700 1 --2- 172.21.15.89:0/4194984078 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf41987700 1 -- 172.21.15.89:0/4194984078 >> v1:172.21.15.89:6789/0 conn(0x7faf3c12f8b0 legacy=0x7faf3c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf41987700 1 -- 172.21.15.89:0/4194984078 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c071a40 con 0x7faf3c12cf60 2023-09-30T22:16:59.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf41987700 1 --2- 172.21.15.89:0/4194984078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c06f5d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faf38009870 tx=0x7faf38009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2ecc0ecbf35c02 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:59.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf40985700 1 -- 172.21.15.89:0/4194984078 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38004230 con 0x7faf3c12cf60 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf40985700 1 -- 172.21.15.89:0/4194984078 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf38004390 con 0x7faf3c12cf60 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.166+0000 7faf40985700 1 -- 172.21.15.89:0/4194984078 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf380036a0 con 0x7faf3c12cf60 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.167+0000 7faf43beb700 1 -- 172.21.15.89:0/4194984078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 msgr2=0x7faf3c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.167+0000 7faf43beb700 1 --2- 172.21.15.89:0/4194984078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c06f5d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faf38009870 tx=0x7faf38009b80 comp rx=0 tx=0).stop 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.167+0000 7faf43beb700 1 -- 172.21.15.89:0/4194984078 shutdown_connections 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.167+0000 7faf43beb700 1 --2- 172.21.15.89:0/4194984078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c06f5d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.167+0000 7faf43beb700 1 --2- 172.21.15.89:0/4194984078 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.167+0000 7faf43beb700 1 -- 172.21.15.89:0/4194984078 >> 172.21.15.89:0/4194984078 conn(0x7faf3c0a2aa0 msgr2=0x7faf3c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:59.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.167+0000 7faf43beb700 1 -- 172.21.15.89:0/4194984078 shutdown_connections 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.167+0000 7faf43beb700 1 -- 172.21.15.89:0/4194984078 wait complete. 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.168+0000 7faf43beb700 1 Processor -- start 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.168+0000 7faf43beb700 1 -- start start 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf43beb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c1c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf43beb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c12f8b0 0x7faf3c1c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf43beb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c1c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf43beb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c09ecc0 con 0x7faf3c12f8b0 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf43beb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1c5c20 con 0x7faf3c12cf60 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf43beb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c12d8d0 con 0x7faf3c130430 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf41987700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c1c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf41987700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c1c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49494/0 (socket says 172.21.15.89:49494) 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf41987700 1 -- 172.21.15.89:0/1178036204 learned_addr learned my addr 172.21.15.89:0/1178036204 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf42188700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c1c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.169+0000 7faf41186700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c12f8b0 0x7faf3c1c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf42188700 1 -- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 msgr2=0x7faf3c1c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf42188700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c1c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf42188700 1 -- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c12f8b0 msgr2=0x7faf3c1c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf42188700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c12f8b0 0x7faf3c1c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf42188700 1 -- 172.21.15.89:0/1178036204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380094a0 con 0x7faf3c130430 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf42188700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c1c81e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf3400efc0 tx=0x7faf3400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf327fc700 1 -- 172.21.15.89:0/1178036204 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3400cc90 con 0x7faf3c130430 2023-09-30T22:16:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf327fc700 1 -- 172.21.15.89:0/1178036204 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3400f040 con 0x7faf3c130430 2023-09-30T22:16:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf327fc700 1 -- 172.21.15.89:0/1178036204 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf340146c0 con 0x7faf3c130430 2023-09-30T22:16:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c09e970 con 0x7faf3c130430 2023-09-30T22:16:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.170+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c09dee0 con 0x7faf3c130430 2023-09-30T22:16:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.171+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c09e610 con 0x7faf3c130430 2023-09-30T22:16:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.172+0000 7faf327fc700 1 -- 172.21.15.89:0/1178036204 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7faf34014820 con 0x7faf3c130430 2023-09-30T22:16:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.172+0000 7faf327fc700 1 -- 172.21.15.89:0/1178036204 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7faf34048350 con 0x7faf3c130430 2023-09-30T22:16:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.172+0000 7faf41987700 1 -- 172.21.15.89:0/1178036204 >> v1:172.21.15.89:6816/126953 conn(0x7faf2002f510 legacy=0x7faf200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:59.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.177+0000 7faf327fc700 1 -- 172.21.15.89:0/1178036204 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf34045060 con 0x7faf3c130430 2023-09-30T22:16:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.205+0000 7f32dfcdb700 1 Processor -- start 2023-09-30T22:16:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32dfcdb700 1 -- start start 2023-09-30T22:16:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32dfcdb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8133540 0x7f32d8135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32dfcdb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32d8135e60 0x7f32d8138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32dfcdb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d80716a0 con 0x7f32d8130540 2023-09-30T22:16:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32dfcdb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d8071290 con 0x7f32d8135e60 2023-09-30T22:16:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32dfcdb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d8130340 con 0x7f32d8133540 2023-09-30T22:16:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32de278700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32d8135e60 0x7f32d8138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32de278700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32d8135e60 0x7f32d8138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38740/0 (socket says 172.21.15.89:38740) 2023-09-30T22:16:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32de278700 1 -- 172.21.15.89:0/2254186012 learned_addr learned my addr 172.21.15.89:0/2254186012 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.206+0000 7f32dd276700 1 --2- 172.21.15.89:0/2254186012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8133540 0x7f32d8135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.207+0000 7f32de278700 1 -- 172.21.15.89:0/2254186012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8133540 msgr2=0x7f32d8135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.207+0000 7f32de278700 1 --2- 172.21.15.89:0/2254186012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8133540 0x7f32d8135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.207+0000 7f32de278700 1 -- 172.21.15.89:0/2254186012 >> v1:172.21.15.89:6789/0 conn(0x7f32d8130540 legacy=0x7f32d8130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.207+0000 7f32de278700 1 -- 172.21.15.89:0/2254186012 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8070b80 con 0x7f32d8135e60 2023-09-30T22:16:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.207+0000 7f32de278700 1 --2- 172.21.15.89:0/2254186012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8135e60 0x7f32d8138240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009e50 tx=0x7f32d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9a7caf1c763940 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.207+0000 7f32dca75700 1 -- 172.21.15.89:0/2254186012 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d000e070 con 0x7f32d8135e60 2023-09-30T22:16:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.207+0000 7f32dca75700 1 -- 172.21.15.89:0/2254186012 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d0003a20 con 0x7f32d8135e60 2023-09-30T22:16:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.207+0000 7f32dca75700 1 -- 172.21.15.89:0/2254186012 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0003dd0 con 0x7f32d8135e60 2023-09-30T22:16:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.208+0000 7f32dfcdb700 1 -- 172.21.15.89:0/2254186012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8135e60 msgr2=0x7f32d8138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.208+0000 7f32dfcdb700 1 --2- 172.21.15.89:0/2254186012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8135e60 0x7f32d8138240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009e50 tx=0x7f32d000b330 comp rx=0 tx=0).stop 2023-09-30T22:16:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.208+0000 7f32dfcdb700 1 -- 172.21.15.89:0/2254186012 shutdown_connections 2023-09-30T22:16:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.208+0000 7f32dfcdb700 1 --2- 172.21.15.89:0/2254186012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8135e60 0x7f32d8138240 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.208+0000 7f32dfcdb700 1 --2- 172.21.15.89:0/2254186012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8133540 0x7f32d8135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.208+0000 7f32dfcdb700 1 -- 172.21.15.89:0/2254186012 >> 172.21.15.89:0/2254186012 conn(0x7f32d80a2aa0 msgr2=0x7f32d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.208+0000 7f32dfcdb700 1 -- 172.21.15.89:0/2254186012 shutdown_connections 2023-09-30T22:16:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.208+0000 7f32dfcdb700 1 -- 172.21.15.89:0/2254186012 wait complete. 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.209+0000 7f32dfcdb700 1 Processor -- start 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.209+0000 7f32dfcdb700 1 -- start start 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.209+0000 7f32dfcdb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32d8130540 0x7f32d809dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.209+0000 7f32dfcdb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8133540 0x7f32d8139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32dfcdb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8135e60 0x7f32d8139cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32dfcdb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d8138d60 con 0x7f32d8133540 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32dfcdb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d8138b70 con 0x7f32d8135e60 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32dfcdb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80718e0 con 0x7f32d8130540 2023-09-30T22:16:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32dd276700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8133540 0x7f32d8139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32dd276700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8133540 0x7f32d8139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38748/0 (socket says 172.21.15.89:38748) 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32dd276700 1 -- 172.21.15.89:0/3496303584 learned_addr learned my addr 172.21.15.89:0/3496303584 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32dda77700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32d8130540 0x7f32d809dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32de278700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8135e60 0x7f32d8139cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32de278700 1 -- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32d8130540 msgr2=0x7f32d809dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32de278700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32d8130540 0x7f32d809dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32de278700 1 -- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8133540 msgr2=0x7f32d8139790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32de278700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8133540 0x7f32d8139790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.210+0000 7f32de278700 1 -- 172.21.15.89:0/3496303584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0009b30 con 0x7f32d8135e60 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.211+0000 7f32de278700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8135e60 0x7f32d8139cd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009e50 tx=0x7f32d0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.211+0000 7f32ce7fc700 1 -- 172.21.15.89:0/3496303584 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0004440 con 0x7f32d8135e60 2023-09-30T22:16:59.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.211+0000 7f32ce7fc700 1 -- 172.21.15.89:0/3496303584 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d00092e0 con 0x7f32d8135e60 2023-09-30T22:16:59.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.211+0000 7f32ce7fc700 1 -- 172.21.15.89:0/3496303584 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d00047e0 con 0x7f32d8135e60 2023-09-30T22:16:59.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.211+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d8070ec0 con 0x7f32d8135e60 2023-09-30T22:16:59.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.211+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d809a630 con 0x7f32d8135e60 2023-09-30T22:16:59.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.212+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d8074ea0 con 0x7f32d8135e60 2023-09-30T22:16:59.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.212+0000 7f32ce7fc700 1 -- 172.21.15.89:0/3496303584 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f32d0012430 con 0x7f32d8135e60 2023-09-30T22:16:59.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.212+0000 7f32ce7fc700 1 -- 172.21.15.89:0/3496303584 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f32d0040e50 con 0x7f32d8135e60 2023-09-30T22:16:59.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.212+0000 7f32dda77700 1 -- 172.21.15.89:0/3496303584 >> v1:172.21.15.89:6816/126953 conn(0x7f32bc02f540 legacy=0x7f32bc0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:59.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.217+0000 7f32ce7fc700 1 -- 172.21.15.89:0/3496303584 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32d0024d80 con 0x7f32d8135e60 2023-09-30T22:16:59.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.249+0000 7f1f86bf5700 1 Processor -- start 2023-09-30T22:16:59.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.249+0000 7f1f86bf5700 1 -- start start 2023-09-30T22:16:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.249+0000 7f1f86bf5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009d6f0 0x7f1f8009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.249+0000 7f1f86bf5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e000 0x7f1f801339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f86bf5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8009b120 con 0x7f1f8009e410 2023-09-30T22:16:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f86bf5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8006b460 con 0x7f1f8009d6f0 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f86bf5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8009a7e0 con 0x7f1f8009e000 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e000 0x7f1f801339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f84991700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009d6f0 0x7f1f8009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e000 0x7f1f801339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38654/0 (socket says 172.21.15.89:38654) 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7ffff700 1 -- 172.21.15.89:0/1753007748 learned_addr learned my addr 172.21.15.89:0/1753007748 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7ffff700 1 -- 172.21.15.89:0/1753007748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009d6f0 msgr2=0x7f1f8009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7ffff700 1 --2- 172.21.15.89:0/1753007748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009d6f0 0x7f1f8009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7ffff700 1 -- 172.21.15.89:0/1753007748 >> v1:172.21.15.89:6789/0 conn(0x7f1f8009e410 legacy=0x7f1f80133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7ffff700 1 -- 172.21.15.89:0/1753007748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8009a500 con 0x7f1f8009e000 2023-09-30T22:16:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f84991700 1 --2- 172.21.15.89:0/1753007748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009d6f0 0x7f1f8009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:16:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7ffff700 1 --2- 172.21.15.89:0/1753007748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e000 0x7f1f801339f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009950 tx=0x7f1f74009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80217148a6a5276e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.250+0000 7f1f7f7fe700 1 -- 172.21.15.89:0/1753007748 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74011830 con 0x7f1f8009e000 2023-09-30T22:16:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.251+0000 7f1f7f7fe700 1 -- 172.21.15.89:0/1753007748 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f74011990 con 0x7f1f8009e000 2023-09-30T22:16:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.251+0000 7f1f7f7fe700 1 -- 172.21.15.89:0/1753007748 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74011d40 con 0x7f1f8009e000 2023-09-30T22:16:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.251+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1753007748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e000 msgr2=0x7f1f801339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.251+0000 7f1f86bf5700 1 --2- 172.21.15.89:0/1753007748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e000 0x7f1f801339f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009950 tx=0x7f1f74009c60 comp rx=0 tx=0).stop 2023-09-30T22:16:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.251+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1753007748 shutdown_connections 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.251+0000 7f1f86bf5700 1 --2- 172.21.15.89:0/1753007748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009d6f0 0x7f1f8009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.251+0000 7f1f86bf5700 1 --2- 172.21.15.89:0/1753007748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e000 0x7f1f801339f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.251+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1753007748 >> 172.21.15.89:0/1753007748 conn(0x7f1f80066f50 msgr2=0x7f1f8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.252+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1753007748 shutdown_connections 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.252+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1753007748 wait complete. 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.252+0000 7f1f86bf5700 1 Processor -- start 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.252+0000 7f1f86bf5700 1 -- start start 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f86bf5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8009d6f0 0x7f1f801a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f86bf5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009e000 0x7f1f801a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f86bf5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e410 0x7f1f801a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f86bf5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f801a7380 con 0x7f1f8009d6f0 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f86bf5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f801a7ae0 con 0x7f1f8009e000 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f86bf5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f801a8d30 con 0x7f1f8009e410 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f84991700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8009d6f0 0x7f1f801a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f85192700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e410 0x7f1f801a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f84991700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8009d6f0 0x7f1f801a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38760/0 (socket says 172.21.15.89:38760) 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f84991700 1 -- 172.21.15.89:0/1941002639 learned_addr learned my addr 172.21.15.89:0/1941002639 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.253+0000 7f1f7ffff700 1 --2- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009e000 0x7f1f801a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f85192700 1 -- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009e000 msgr2=0x7f1f801a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f85192700 1 --2- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009e000 0x7f1f801a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f85192700 1 -- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8009d6f0 msgr2=0x7f1f801a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f85192700 1 --2- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8009d6f0 0x7f1f801a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f85192700 1 -- 172.21.15.89:0/1941002639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f700094a0 con 0x7f1f8009e410 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f85192700 1 --2- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e410 0x7f1f801a6950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800cb90 tx=0x7f1f7800cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f7d7fa700 1 -- 172.21.15.89:0/1941002639 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78003c00 con 0x7f1f8009e410 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f74009550 con 0x7f1f8009e410 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f7d7fa700 1 -- 172.21.15.89:0/1941002639 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f78003d60 con 0x7f1f8009e410 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f7d7fa700 1 -- 172.21.15.89:0/1941002639 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800aa60 con 0x7f1f8009e410 2023-09-30T22:16:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.254+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f801a9770 con 0x7f1f8009e410 2023-09-30T22:16:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.255+0000 7f1f7d7fa700 1 -- 172.21.15.89:0/1941002639 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1f7800abc0 con 0x7f1f8009e410 2023-09-30T22:16:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.255+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8009c710 con 0x7f1f8009e410 2023-09-30T22:16:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.255+0000 7f1f7d7fa700 1 -- 172.21.15.89:0/1941002639 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f1f7803fb00 con 0x7f1f8009e410 2023-09-30T22:16:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.255+0000 7f1f84991700 1 -- 172.21.15.89:0/1941002639 >> v1:172.21.15.89:6816/126953 conn(0x7f1f6c02f510 legacy=0x7f1f6c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.260+0000 7f1f7d7fa700 1 -- 172.21.15.89:0/1941002639 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f7803fe60 con 0x7f1f8009e410 2023-09-30T22:16:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.283+0000 7fa1ff02d700 1 Processor -- start 2023-09-30T22:16:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.283+0000 7fa1ff02d700 1 -- start start 2023-09-30T22:16:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.283+0000 7fa1ff02d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809d6f0 0x7fa1f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.283+0000 7fa1ff02d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa1f809e000 0x7fa1f81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.283+0000 7fa1ff02d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f809b120 con 0x7fa1f809e410 2023-09-30T22:16:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.283+0000 7fa1ff02d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f806b460 con 0x7fa1f809e000 2023-09-30T22:16:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.283+0000 7fa1ff02d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f809a7e0 con 0x7fa1f809d6f0 2023-09-30T22:16:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.284+0000 7fa1fcdc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809d6f0 0x7fa1f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.284+0000 7fa1fcdc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809d6f0 0x7fa1f809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38672/0 (socket says 172.21.15.89:38672) 2023-09-30T22:16:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.284+0000 7fa1fcdc9700 1 -- 172.21.15.89:0/2885361982 learned_addr learned my addr 172.21.15.89:0/2885361982 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1f7fff700 1 --2- 172.21.15.89:0/2885361982 >> v2:172.21.15.89:3300/0 conn(0x7fa1f809e000 0x7fa1f81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1fcdc9700 1 -- 172.21.15.89:0/2885361982 >> v2:172.21.15.89:3300/0 conn(0x7fa1f809e000 msgr2=0x7fa1f81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1fcdc9700 1 --2- 172.21.15.89:0/2885361982 >> v2:172.21.15.89:3300/0 conn(0x7fa1f809e000 0x7fa1f81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1fcdc9700 1 -- 172.21.15.89:0/2885361982 >> v1:172.21.15.89:6789/0 conn(0x7fa1f809e410 legacy=0x7fa1f8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1fcdc9700 1 -- 172.21.15.89:0/2885361982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f809a500 con 0x7fa1f809d6f0 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1fcdc9700 1 --2- 172.21.15.89:0/2885361982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809d6f0 0x7fa1f809dac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009920 tx=0x7fa1ec009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781037fc0157e63e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1f77fe700 1 -- 172.21.15.89:0/2885361982 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec004230 con 0x7fa1f809d6f0 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1f77fe700 1 -- 172.21.15.89:0/2885361982 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec004390 con 0x7fa1f809d6f0 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.285+0000 7fa1f77fe700 1 -- 172.21.15.89:0/2885361982 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0036a0 con 0x7fa1f809d6f0 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.286+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2885361982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809d6f0 msgr2=0x7fa1f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.286+0000 7fa1ff02d700 1 --2- 172.21.15.89:0/2885361982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809d6f0 0x7fa1f809dac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009920 tx=0x7fa1ec009c30 comp rx=0 tx=0).stop 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.286+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2885361982 shutdown_connections 2023-09-30T22:16:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.286+0000 7fa1ff02d700 1 --2- 172.21.15.89:0/2885361982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809d6f0 0x7fa1f809dac0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.286+0000 7fa1ff02d700 1 --2- 172.21.15.89:0/2885361982 >> v2:172.21.15.89:3300/0 conn(0x7fa1f809e000 0x7fa1f81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.286+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2885361982 >> 172.21.15.89:0/2885361982 conn(0x7fa1f8066f50 msgr2=0x7fa1f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.286+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2885361982 shutdown_connections 2023-09-30T22:16:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.286+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2885361982 wait complete. 2023-09-30T22:16:59.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.287+0000 7fa1ff02d700 1 Processor -- start 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.288+0000 7fa1ff02d700 1 -- start start 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.288+0000 7fa1ff02d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f809d6f0 0x7fa1f81a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.288+0000 7fa1ff02d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1f809e000 0x7fa1f81a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.288+0000 7fa1ff02d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809e410 0x7fa1f81a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.288+0000 7fa1ff02d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f80a3290 con 0x7fa1f809d6f0 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.288+0000 7fa1ff02d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f81a8480 con 0x7fa1f809e000 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.288+0000 7fa1ff02d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f81a6c90 con 0x7fa1f809e410 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809e410 0x7fa1f81a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809e410 0x7fa1f81a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38684/0 (socket says 172.21.15.89:38684) 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 -- 172.21.15.89:0/2187655904 learned_addr learned my addr 172.21.15.89:0/2187655904 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fcdc9700 1 --2- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f809d6f0 0x7fa1f81a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 -- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1f809e000 msgr2=0x7fa1f81a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 --2- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1f809e000 0x7fa1f81a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 -- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f809d6f0 msgr2=0x7fa1f81a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 --2- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f809d6f0 0x7fa1f81a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 -- 172.21.15.89:0/2187655904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009550 con 0x7fa1f809e410 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1fd5ca700 1 --2- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809e410 0x7fa1f81a5af0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000cc10 tx=0x7fa1f000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1f57fa700 1 -- 172.21.15.89:0/2187655904 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000ab10 con 0x7fa1f809e410 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1f57fa700 1 -- 172.21.15.89:0/2187655904 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f000ac70 con 0x7fa1f809e410 2023-09-30T22:16:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.289+0000 7fa1f57fa700 1 -- 172.21.15.89:0/2187655904 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0012420 con 0x7fa1f809e410 2023-09-30T22:16:59.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.290+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f809c710 con 0x7fa1f809e410 2023-09-30T22:16:59.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.290+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f81a7dc0 con 0x7fa1f809e410 2023-09-30T22:16:59.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.291+0000 7fa1f57fa700 1 -- 172.21.15.89:0/2187655904 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa1f0012580 con 0x7fa1f809e410 2023-09-30T22:16:59.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.291+0000 7fa1f57fa700 1 -- 172.21.15.89:0/2187655904 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fa1f003ff90 con 0x7fa1f809e410 2023-09-30T22:16:59.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.291+0000 7fa1fcdc9700 1 -- 172.21.15.89:0/2187655904 >> v1:172.21.15.89:6816/126953 conn(0x7fa1dc02f510 legacy=0x7fa1dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:59.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.293+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f8074ea0 con 0x7fa1f809e410 2023-09-30T22:16:59.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.297+0000 7fa1f57fa700 1 -- 172.21.15.89:0/2187655904 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa1f0019900 con 0x7fa1f809e410 2023-09-30T22:16:59.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.373+0000 7faf41987700 1 -- 172.21.15.89:0/1178036204 >> v1:172.21.15.89:6816/126953 conn(0x7faf2002f510 legacy=0x7faf200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.413+0000 7f32dda77700 1 -- 172.21.15.89:0/3496303584 >> v1:172.21.15.89:6816/126953 conn(0x7f32bc02f540 legacy=0x7f32bc0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:59.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.420+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faf3c129fa0 con 0x7faf3c130430 2023-09-30T22:16:59.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.420+0000 7faf327fc700 1 -- 172.21.15.89:0/1178036204 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf34018020 con 0x7faf3c130430 2023-09-30T22:16:59.421 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.423+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 >> v1:172.21.15.89:6816/126953 conn(0x7faf2002f510 legacy=0x7faf200319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.423+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 msgr2=0x7faf3c1c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.423+0000 7faf43beb700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c1c81e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf3400efc0 tx=0x7faf3400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.423+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 shutdown_connections 2023-09-30T22:16:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.423+0000 7faf43beb700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c130430 0x7faf3c1c81e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.423+0000 7faf43beb700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c12cf60 0x7faf3c1c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.423+0000 7faf43beb700 1 --2- 172.21.15.89:0/1178036204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c12f8b0 0x7faf3c1c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.423+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 >> 172.21.15.89:0/1178036204 conn(0x7faf3c0a2aa0 msgr2=0x7faf3c12a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.424+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 shutdown_connections 2023-09-30T22:16:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.424+0000 7faf43beb700 1 -- 172.21.15.89:0/1178036204 wait complete. 2023-09-30T22:16:59.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:16:59.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.446+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f32d812db80 con 0x7f32d8135e60 2023-09-30T22:16:59.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.447+0000 7f32ce7fc700 1 -- 172.21.15.89:0/3496303584 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32d0017020 con 0x7f32d8135e60 2023-09-30T22:16:59.447 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:16:59.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 >> v1:172.21.15.89:6816/126953 conn(0x7f32bc02f540 legacy=0x7f32bc0319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:59.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8135e60 msgr2=0x7f32d8139cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8135e60 0x7f32d8139cd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009e50 tx=0x7f32d0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 shutdown_connections 2023-09-30T22:16:59.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32d8135e60 0x7f32d8139cd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32d8130540 0x7f32d809dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 --2- 172.21.15.89:0/3496303584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d8133540 0x7f32d8139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 >> 172.21.15.89:0/3496303584 conn(0x7f32d80a2aa0 msgr2=0x7f32d8131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:59.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 shutdown_connections 2023-09-30T22:16:59.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.450+0000 7f32dfcdb700 1 -- 172.21.15.89:0/3496303584 wait complete. 2023-09-30T22:16:59.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.456+0000 7f1f84991700 1 -- 172.21.15.89:0/1941002639 >> v1:172.21.15.89:6816/126953 conn(0x7f1f6c02f510 legacy=0x7f1f6c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:59.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.459+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1f80074ea0 con 0x7f1f8009e410 2023-09-30T22:16:59.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.459+0000 7f1f7d7fa700 1 -- 172.21.15.89:0/1941002639 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f780198f0 con 0x7f1f8009e410 2023-09-30T22:16:59.460 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:16:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.461+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 >> v1:172.21.15.89:6816/126953 conn(0x7f1f6c02f510 legacy=0x7f1f6c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.461+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e410 msgr2=0x7f1f801a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.461+0000 7f1f86bf5700 1 --2- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e410 0x7f1f801a6950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800cb90 tx=0x7f1f7800cea0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.462+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 shutdown_connections 2023-09-30T22:16:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.462+0000 7f1f86bf5700 1 --2- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f8009e410 0x7f1f801a6950 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.462+0000 7f1f86bf5700 1 --2- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8009d6f0 0x7f1f801a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.462+0000 7f1f86bf5700 1 --2- 172.21.15.89:0/1941002639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f8009e000 0x7f1f801a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.462+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 >> 172.21.15.89:0/1941002639 conn(0x7f1f80066f50 msgr2=0x7f1f801baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.462+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 shutdown_connections 2023-09-30T22:16:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.462+0000 7f1f86bf5700 1 -- 172.21.15.89:0/1941002639 wait complete. 2023-09-30T22:16:59.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:16:59.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:16:59.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.492+0000 7fa1fcdc9700 1 -- 172.21.15.89:0/2187655904 >> v1:172.21.15.89:6816/126953 conn(0x7fa1dc02f510 legacy=0x7fa1dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:16:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.523+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa1f81a8ae0 con 0x7fa1f809e410 2023-09-30T22:16:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.523+0000 7fa1f57fa700 1 -- 172.21.15.89:0/2187655904 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa1f0019310 con 0x7fa1f809e410 2023-09-30T22:16:59.524 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:16:59.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.525+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 >> v1:172.21.15.89:6816/126953 conn(0x7fa1dc02f510 legacy=0x7fa1dc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:16:59.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.525+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809e410 msgr2=0x7fa1f81a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:16:59.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.525+0000 7fa1ff02d700 1 --2- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809e410 0x7fa1f81a5af0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000cc10 tx=0x7fa1f000cf20 comp rx=0 tx=0).stop 2023-09-30T22:16:59.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.526+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 shutdown_connections 2023-09-30T22:16:59.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.526+0000 7fa1ff02d700 1 --2- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1f809e410 0x7fa1f81a5af0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.526+0000 7fa1ff02d700 1 --2- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f809d6f0 0x7fa1f81a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.526+0000 7fa1ff02d700 1 --2- 172.21.15.89:0/2187655904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1f809e000 0x7fa1f81a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:16:59.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.526+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 >> 172.21.15.89:0/2187655904 conn(0x7fa1f8066f50 msgr2=0x7fa1f806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:16:59.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.526+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 shutdown_connections 2023-09-30T22:16:59.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:16:59.526+0000 7fa1ff02d700 1 -- 172.21.15.89:0/2187655904 wait complete. 2023-09-30T22:16:59.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:00.438 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:00.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:00.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:00.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:00.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.578+0000 7f1ca2c44700 1 Processor -- start 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca2c44700 1 -- start start 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca2c44700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1c9c06f5d0 0x7f1c9c12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca2c44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c135c20 0x7f1c9c138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca2c44700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9c06fa40 con 0x7f1c9c12cf60 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca2c44700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9c12cd90 con 0x7f1c9c06f5d0 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca2c44700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c071780 con 0x7f1c9c135c20 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca11e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c135c20 0x7f1c9c138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca11e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c135c20 0x7f1c9c138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42478/0 (socket says 172.21.15.89:42478) 2023-09-30T22:17:00.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1ca11e1700 1 -- 172.21.15.89:0/1480390446 learned_addr learned my addr 172.21.15.89:0/1480390446 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.579+0000 7f1c9bfff700 1 --2- 172.21.15.89:0/1480390446 >> v2:172.21.15.89:3300/0 conn(0x7f1c9c06f5d0 0x7f1c9c12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.580+0000 7f1ca11e1700 1 -- 172.21.15.89:0/1480390446 >> v2:172.21.15.89:3300/0 conn(0x7f1c9c06f5d0 msgr2=0x7f1c9c12ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.580+0000 7f1ca11e1700 1 --2- 172.21.15.89:0/1480390446 >> v2:172.21.15.89:3300/0 conn(0x7f1c9c06f5d0 0x7f1c9c12ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.580+0000 7f1ca11e1700 1 -- 172.21.15.89:0/1480390446 >> v1:172.21.15.89:6789/0 conn(0x7f1c9c12cf60 legacy=0x7f1c9c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.580+0000 7f1ca11e1700 1 -- 172.21.15.89:0/1480390446 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c071a40 con 0x7f1c9c135c20 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.580+0000 7f1ca11e1700 1 --2- 172.21.15.89:0/1480390446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c135c20 0x7f1c9c138000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400d150 tx=0x7f1c9400d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31652cde8d3589c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.580+0000 7f1c9b7fe700 1 -- 172.21.15.89:0/1480390446 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94010070 con 0x7f1c9c135c20 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.580+0000 7f1c9b7fe700 1 -- 172.21.15.89:0/1480390446 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9400db00 con 0x7f1c9c135c20 2023-09-30T22:17:00.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.580+0000 7f1c9b7fe700 1 -- 172.21.15.89:0/1480390446 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94016400 con 0x7f1c9c135c20 2023-09-30T22:17:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.581+0000 7f1ca2c44700 1 -- 172.21.15.89:0/1480390446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c135c20 msgr2=0x7f1c9c138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.581+0000 7f1ca2c44700 1 --2- 172.21.15.89:0/1480390446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c135c20 0x7f1c9c138000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400d150 tx=0x7f1c9400d460 comp rx=0 tx=0).stop 2023-09-30T22:17:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.581+0000 7f1ca2c44700 1 -- 172.21.15.89:0/1480390446 shutdown_connections 2023-09-30T22:17:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.581+0000 7f1ca2c44700 1 --2- 172.21.15.89:0/1480390446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c135c20 0x7f1c9c138000 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.581+0000 7f1ca2c44700 1 --2- 172.21.15.89:0/1480390446 >> v2:172.21.15.89:3300/0 conn(0x7f1c9c06f5d0 0x7f1c9c12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.581+0000 7f1ca2c44700 1 -- 172.21.15.89:0/1480390446 >> 172.21.15.89:0/1480390446 conn(0x7f1c9c0a2aa0 msgr2=0x7f1c9c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.581+0000 7f1ca2c44700 1 -- 172.21.15.89:0/1480390446 shutdown_connections 2023-09-30T22:17:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.581+0000 7f1ca2c44700 1 -- 172.21.15.89:0/1480390446 wait complete. 2023-09-30T22:17:00.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.582+0000 7f1ca2c44700 1 Processor -- start 2023-09-30T22:17:00.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.582+0000 7f1ca2c44700 1 -- start start 2023-09-30T22:17:00.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.582+0000 7f1ca2c44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c06f5d0 0x7f1c9c127170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.582+0000 7f1ca2c44700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c12cf60 0x7f1c9c1276b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.582+0000 7f1ca2c44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c135c20 0x7f1c9c1c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.582+0000 7f1ca2c44700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c09a9e0 con 0x7f1c9c135c20 2023-09-30T22:17:00.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.582+0000 7f1ca2c44700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c09a620 con 0x7f1c9c12cf60 2023-09-30T22:17:00.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.582+0000 7f1ca2c44700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c09abc0 con 0x7f1c9c06f5d0 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c12cf60 0x7f1c9c1276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c12cf60 0x7f1c9c1276b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45966/0 (socket says 172.21.15.89:45966) 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 -- 172.21.15.89:0/2750529266 learned_addr learned my addr 172.21.15.89:0/2750529266 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1ca09e0700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c06f5d0 0x7f1c9c127170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1ca11e1700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c135c20 0x7f1c9c1c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 -- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c06f5d0 msgr2=0x7f1c9c127170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c06f5d0 0x7f1c9c127170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 -- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c135c20 msgr2=0x7f1c9c1c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c135c20 0x7f1c9c1c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 -- 172.21.15.89:0/2750529266 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9400bd40 con 0x7f1c9c12cf60 2023-09-30T22:17:00.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c9bfff700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c12cf60 0x7f1c9c1276b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800e9b0 tx=0x7f1c8800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:00.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c997fa700 1 -- 172.21.15.89:0/2750529266 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88009710 con 0x7f1c9c12cf60 2023-09-30T22:17:00.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c997fa700 1 -- 172.21.15.89:0/2750529266 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8800ee50 con 0x7f1c9c12cf60 2023-09-30T22:17:00.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.583+0000 7f1c997fa700 1 -- 172.21.15.89:0/2750529266 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88013a60 con 0x7f1c9c12cf60 2023-09-30T22:17:00.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.584+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c127bf0 con 0x7f1c9c12cf60 2023-09-30T22:17:00.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.584+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c129fa0 con 0x7f1c9c12cf60 2023-09-30T22:17:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.585+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9c1332e0 con 0x7f1c9c12cf60 2023-09-30T22:17:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.585+0000 7f1c997fa700 1 -- 172.21.15.89:0/2750529266 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1c88013bc0 con 0x7f1c9c12cf60 2023-09-30T22:17:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.585+0000 7f1c997fa700 1 -- 172.21.15.89:0/2750529266 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f1c88043ae0 con 0x7f1c9c12cf60 2023-09-30T22:17:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.585+0000 7f1ca09e0700 1 -- 172.21.15.89:0/2750529266 >> v1:172.21.15.89:6816/126953 conn(0x7f1c8002f510 legacy=0x7f1c800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:00.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.594+0000 7f1c997fa700 1 -- 172.21.15.89:0/2750529266 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c8801a7b0 con 0x7f1c9c12cf60 2023-09-30T22:17:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.616+0000 7f81a798d700 1 Processor -- start 2023-09-30T22:17:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.616+0000 7fabb56cd700 1 Processor -- start 2023-09-30T22:17:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.616+0000 7fabb56cd700 1 -- start start 2023-09-30T22:17:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.616+0000 7f81a798d700 1 -- start start 2023-09-30T22:17:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabb56cd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb006f5d0 0x7fabb012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabb56cd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb0135c20 0x7fabb0138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabb56cd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb006fa40 con 0x7fabb012cf60 2023-09-30T22:17:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabb56cd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb012cd90 con 0x7fabb0135c20 2023-09-30T22:17:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabb56cd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabb0071780 con 0x7fabb006f5d0 2023-09-30T22:17:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabaf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb0135c20 0x7fabb0138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabaf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb0135c20 0x7fabb0138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45974/0 (socket says 172.21.15.89:45974) 2023-09-30T22:17:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabaeffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fabb012cf60 0x7fabb012f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59816/0 (socket says 172.21.15.89:59816) 2023-09-30T22:17:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabaf7fe700 1 -- 172.21.15.89:0/2628104646 learned_addr learned my addr 172.21.15.89:0/2628104646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7f81a798d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7f81a798d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81a0130e50 0x7f81a0135750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7f81a5729700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7f81a4f28700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81a0130e50 0x7f81a0135750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7fabae7fc700 1 --2- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb006f5d0 0x7fabb012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.617+0000 7f81a4f28700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81a0130e50 0x7f81a0135750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33140/0 (socket says 172.21.15.89:33140) 2023-09-30T22:17:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a4f28700 1 -- 172.21.15.89:0/4168328732 learned_addr learned my addr 172.21.15.89:0/4168328732 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a798d700 1 -- 172.21.15.89:0/4168328732 --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a00716a0 con 0x7f81a0135c90 2023-09-30T22:17:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a798d700 1 -- 172.21.15.89:0/4168328732 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a009a390 con 0x7f81a0130e50 2023-09-30T22:17:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a798d700 1 -- 172.21.15.89:0/4168328732 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0071290 con 0x7f81a0130540 2023-09-30T22:17:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabae7fc700 1 -- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb0135c20 msgr2=0x7fabb0138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabae7fc700 1 --2- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb0135c20 0x7fabb0138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabadffb700 1 -- 172.21.15.89:0/2628104646 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958358665 0 0) 0x7fabb006fa40 con 0x7fabb012cf60 2023-09-30T22:17:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabae7fc700 1 -- 172.21.15.89:0/2628104646 >> v1:172.21.15.89:6789/0 conn(0x7fabb012cf60 legacy=0x7fabb012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabae7fc700 1 -- 172.21.15.89:0/2628104646 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0071a40 con 0x7fabb006f5d0 2023-09-30T22:17:00.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabaf7fe700 1 --2- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb0135c20 0x7fabb0138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:00.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a5729700 1 -- 172.21.15.89:0/4168328732 >> v2:172.21.15.89:3300/0 conn(0x7f81a0130e50 msgr2=0x7f81a0135750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:00.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f8197fff700 1 -- 172.21.15.89:0/4168328732 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846726840 0 0) 0x7f81a00716a0 con 0x7f81a0135c90 2023-09-30T22:17:00.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a5729700 1 --2- 172.21.15.89:0/4168328732 >> v2:172.21.15.89:3300/0 conn(0x7f81a0130e50 0x7f81a0135750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a5729700 1 -- 172.21.15.89:0/4168328732 >> v1:172.21.15.89:6789/0 conn(0x7f81a0135c90 legacy=0x7f81a0138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabae7fc700 1 --2- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb006f5d0 0x7fabb012ffc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faba4009870 tx=0x7faba4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe8a7c2bf1bc33f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:00.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a5729700 1 -- 172.21.15.89:0/4168328732 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0130340 con 0x7f81a0130540 2023-09-30T22:17:00.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a4f28700 1 --2- 172.21.15.89:0/4168328732 >> v2:172.21.15.89:3300/0 conn(0x7f81a0130e50 0x7f81a0135750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:00.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f81a5729700 1 --2- 172.21.15.89:0/4168328732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0130910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f819c009840 tx=0x7f819c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d615a226f7d555 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:00.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabadffb700 1 -- 172.21.15.89:0/2628104646 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4011830 con 0x7fabb006f5d0 2023-09-30T22:17:00.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabadffb700 1 -- 172.21.15.89:0/2628104646 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4011990 con 0x7fabb006f5d0 2023-09-30T22:17:00.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7f8197fff700 1 -- 172.21.15.89:0/4168328732 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c011830 con 0x7f81a0130540 2023-09-30T22:17:00.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.618+0000 7fabadffb700 1 -- 172.21.15.89:0/2628104646 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4011d20 con 0x7fabb006f5d0 2023-09-30T22:17:00.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7f8197fff700 1 -- 172.21.15.89:0/4168328732 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c011990 con 0x7f81a0130540 2023-09-30T22:17:00.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7f8197fff700 1 -- 172.21.15.89:0/4168328732 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c011d20 con 0x7f81a0130540 2023-09-30T22:17:00.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7f81a798d700 1 -- 172.21.15.89:0/4168328732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 msgr2=0x7f81a0130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7f81a798d700 1 --2- 172.21.15.89:0/4168328732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0130910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f819c009840 tx=0x7f819c009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:00.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7fabb56cd700 1 -- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb006f5d0 msgr2=0x7fabb012ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7fabb56cd700 1 --2- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb006f5d0 0x7fabb012ffc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faba4009870 tx=0x7faba4009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:00.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7f81a798d700 1 -- 172.21.15.89:0/4168328732 shutdown_connections 2023-09-30T22:17:00.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7f81a798d700 1 --2- 172.21.15.89:0/4168328732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0130910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7f81a798d700 1 --2- 172.21.15.89:0/4168328732 >> v2:172.21.15.89:3300/0 conn(0x7f81a0130e50 0x7f81a0135750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7f81a798d700 1 -- 172.21.15.89:0/4168328732 >> 172.21.15.89:0/4168328732 conn(0x7f81a00a2aa0 msgr2=0x7f81a00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7fabb56cd700 1 -- 172.21.15.89:0/2628104646 shutdown_connections 2023-09-30T22:17:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7fabb56cd700 1 --2- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb0135c20 0x7fabb0138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.619+0000 7fabb56cd700 1 --2- 172.21.15.89:0/2628104646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb006f5d0 0x7fabb012ffc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.620+0000 7fabb56cd700 1 -- 172.21.15.89:0/2628104646 >> 172.21.15.89:0/2628104646 conn(0x7fabb00a2aa0 msgr2=0x7fabb00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.620+0000 7fabb56cd700 1 -- 172.21.15.89:0/2628104646 shutdown_connections 2023-09-30T22:17:00.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.620+0000 7fabb56cd700 1 -- 172.21.15.89:0/2628104646 wait complete. 2023-09-30T22:17:00.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.620+0000 7f81a798d700 1 -- 172.21.15.89:0/4168328732 shutdown_connections 2023-09-30T22:17:00.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.620+0000 7f81a798d700 1 -- 172.21.15.89:0/4168328732 wait complete. 2023-09-30T22:17:00.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.620+0000 7f81a798d700 1 Processor -- start 2023-09-30T22:17:00.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.620+0000 7f81a798d700 1 -- start start 2023-09-30T22:17:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a798d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0134270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a798d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a0130e50 0x7f81a01c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a798d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81a0135c90 0x7f81a01c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a798d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a012d8c0 con 0x7f81a0130e50 2023-09-30T22:17:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a798d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00718e0 con 0x7f81a0135c90 2023-09-30T22:17:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a798d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81a011e4b0 con 0x7f81a0130540 2023-09-30T22:17:00.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7fabb56cd700 1 Processor -- start 2023-09-30T22:17:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5729700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0134270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5729700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0134270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42516/0 (socket says 172.21.15.89:42516) 2023-09-30T22:17:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5729700 1 -- 172.21.15.89:0/3152482707 learned_addr learned my addr 172.21.15.89:0/3152482707 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a4f28700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a0130e50 0x7f81a01c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5f2a700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81a0135c90 0x7f81a01c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5729700 1 -- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81a0135c90 msgr2=0x7f81a01c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5729700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81a0135c90 0x7f81a01c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5729700 1 -- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a0130e50 msgr2=0x7f81a01c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5729700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a0130e50 0x7f81a01c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5729700 1 -- 172.21.15.89:0/3152482707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c0094a0 con 0x7f81a0130540 2023-09-30T22:17:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7f81a5f2a700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81a0135c90 0x7f81a01c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:17:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7f81a5729700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0134270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f819c0090b0 tx=0x7f819c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7f8195ffb700 1 -- 172.21.15.89:0/3152482707 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0036a0 con 0x7f81a0130540 2023-09-30T22:17:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.621+0000 7fabb56cd700 1 -- start start 2023-09-30T22:17:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabb56cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb006f5d0 0x7fabb0134e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabb56cd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb012cf60 0x7fabb01353c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabb56cd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb0135c20 0x7fabb013a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabb56cd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb013bf60 con 0x7fabb006f5d0 2023-09-30T22:17:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabb56cd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb0131d00 con 0x7fabb012cf60 2023-09-30T22:17:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabb56cd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabb0131f70 con 0x7fabb0135c20 2023-09-30T22:17:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabaeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb006f5d0 0x7fabb0134e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabaeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb006f5d0 0x7fabb0134e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33158/0 (socket says 172.21.15.89:33158) 2023-09-30T22:17:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabaeffd700 1 -- 172.21.15.89:0/3250982058 learned_addr learned my addr 172.21.15.89:0/3250982058 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabaf7fe700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb0135c20 0x7fabb013a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7f8195ffb700 1 -- 172.21.15.89:0/3152482707 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c01f070 con 0x7f81a0130540 2023-09-30T22:17:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7f8195ffb700 1 -- 172.21.15.89:0/3152482707 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c003a40 con 0x7f81a0130540 2023-09-30T22:17:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a0133af0 con 0x7f81a0130540 2023-09-30T22:17:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a011f1e0 con 0x7f81a0130540 2023-09-30T22:17:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.622+0000 7fabae7fc700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb012cf60 0x7fabb01353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.623+0000 7fabaf7fe700 1 -- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb012cf60 msgr2=0x7fabb01353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.623+0000 7fabaf7fe700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb012cf60 0x7fabb01353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.623+0000 7fabaf7fe700 1 -- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb006f5d0 msgr2=0x7fabb0134e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.623+0000 7fabaf7fe700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb006f5d0 0x7fabb0134e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.623+0000 7fabaf7fe700 1 -- 172.21.15.89:0/3250982058 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8009b30 con 0x7fabb0135c20 2023-09-30T22:17:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.623+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0074ea0 con 0x7f81a0130540 2023-09-30T22:17:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.624+0000 7fabaf7fe700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb0135c20 0x7fabb013a660 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fabb0071fd0 tx=0x7faba800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.628+0000 7f8195ffb700 1 -- 172.21.15.89:0/3152482707 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f819c003ba0 con 0x7f81a0130540 2023-09-30T22:17:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.628+0000 7f8195ffb700 1 -- 172.21.15.89:0/3152482707 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f819c021070 con 0x7f81a0130540 2023-09-30T22:17:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.628+0000 7f81a4f28700 1 -- 172.21.15.89:0/3152482707 >> v1:172.21.15.89:6816/126953 conn(0x7f818402f540 legacy=0x7f81840319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.628+0000 7fab977fe700 1 -- 172.21.15.89:0/3250982058 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8018070 con 0x7fabb0135c20 2023-09-30T22:17:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.628+0000 7fab977fe700 1 -- 172.21.15.89:0/3250982058 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba800f040 con 0x7fabb0135c20 2023-09-30T22:17:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.628+0000 7f8195ffb700 1 -- 172.21.15.89:0/3152482707 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f819c019b40 con 0x7f81a0130540 2023-09-30T22:17:00.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.628+0000 7fab977fe700 1 -- 172.21.15.89:0/3250982058 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8003c60 con 0x7fabb0135c20 2023-09-30T22:17:00.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.628+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba40094a0 con 0x7fabb0135c20 2023-09-30T22:17:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.629+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb00a1000 con 0x7fabb0135c20 2023-09-30T22:17:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.630+0000 7fab977fe700 1 -- 172.21.15.89:0/3250982058 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7faba800c7e0 con 0x7fabb0135c20 2023-09-30T22:17:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.630+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb0131b60 con 0x7fabb0135c20 2023-09-30T22:17:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.630+0000 7fabaeffd700 1 -- 172.21.15.89:0/3250982058 >> v1:172.21.15.89:6816/126953 conn(0x7fab9c02f560 legacy=0x7fab9c031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.633+0000 7fab977fe700 1 -- 172.21.15.89:0/3250982058 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7faba80133e0 con 0x7fabb0135c20 2023-09-30T22:17:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.635+0000 7fab977fe700 1 -- 172.21.15.89:0/3250982058 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fabb0131b60 con 0x7fabb0135c20 2023-09-30T22:17:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.679+0000 7fbf6c58a700 1 Processor -- start 2023-09-30T22:17:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.679+0000 7fbf6c58a700 1 -- start start 2023-09-30T22:17:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.679+0000 7fbf6c58a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf640a0a30 0x7fbf640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6c58a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6c58a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf640a2a70 con 0x7fbf6409d630 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6c58a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640a01d0 con 0x7fbf64138d80 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6c58a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6409e2b0 con 0x7fbf640a0a30 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf69b25700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbf6409d630 0x7fbf640a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59834/0 (socket says 172.21.15.89:59834) 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf69b25700 1 -- 172.21.15.89:0/1357592889 learned_addr learned my addr 172.21.15.89:0/1357592889 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6a326700 1 --2- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf640a0a30 0x7fbf640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf69324700 1 -- 172.21.15.89:0/1357592889 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064650861 0 0) 0x7fbf640a2a70 con 0x7fbf6409d630 2023-09-30T22:17:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6ab27700 1 --2- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf69324700 1 -- 172.21.15.89:0/1357592889 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf500035b0 con 0x7fbf6409d630 2023-09-30T22:17:00.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6a326700 1 -- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 msgr2=0x7fbf6413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:00.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6a326700 1 --2- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6a326700 1 -- 172.21.15.89:0/1357592889 >> v1:172.21.15.89:6789/0 conn(0x7fbf6409d630 legacy=0x7fbf640a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6a326700 1 -- 172.21.15.89:0/1357592889 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6409dc50 con 0x7fbf640a0a30 2023-09-30T22:17:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.680+0000 7fbf6a326700 1 --2- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf640a0a30 0x7fbf640a0e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009870 tx=0x7fbf60009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d31dd179c8d3644f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf69324700 1 -- 172.21.15.89:0/1357592889 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60004230 con 0x7fbf640a0a30 2023-09-30T22:17:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf69324700 1 -- 172.21.15.89:0/1357592889 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60004390 con 0x7fbf640a0a30 2023-09-30T22:17:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf69324700 1 -- 172.21.15.89:0/1357592889 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600036a0 con 0x7fbf640a0a30 2023-09-30T22:17:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf6c58a700 1 -- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf640a0a30 msgr2=0x7fbf640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf6c58a700 1 --2- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf640a0a30 0x7fbf640a0e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009870 tx=0x7fbf60009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf6c58a700 1 -- 172.21.15.89:0/1357592889 shutdown_connections 2023-09-30T22:17:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf6c58a700 1 --2- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf6c58a700 1 --2- 172.21.15.89:0/1357592889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf640a0a30 0x7fbf640a0e00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf6c58a700 1 -- 172.21.15.89:0/1357592889 >> 172.21.15.89:0/1357592889 conn(0x7fbf640a3050 msgr2=0x7fbf640a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf6c58a700 1 -- 172.21.15.89:0/1357592889 shutdown_connections 2023-09-30T22:17:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.681+0000 7fbf6c58a700 1 -- 172.21.15.89:0/1357592889 wait complete. 2023-09-30T22:17:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6c58a700 1 Processor -- start 2023-09-30T22:17:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6c58a700 1 -- start start 2023-09-30T22:17:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6c58a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf6409d630 0x7fbf641cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6c58a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640a0a30 0x7fbf641cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6c58a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6c58a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf64133840 con 0x7fbf640a0a30 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6c58a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf64071270 con 0x7fbf64138d80 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6c58a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf64071d00 con 0x7fbf6409d630 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf69b25700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640a0a30 0x7fbf641cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf69b25700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640a0a30 0x7fbf641cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33162/0 (socket says 172.21.15.89:33162) 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf69b25700 1 -- 172.21.15.89:0/2176031718 learned_addr learned my addr 172.21.15.89:0/2176031718 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.683+0000 7fbf6ab27700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.684+0000 7fbf6a326700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf6409d630 0x7fbf641cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.684+0000 7fbf6ab27700 1 -- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf6409d630 msgr2=0x7fbf641cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.684+0000 7fbf6ab27700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf6409d630 0x7fbf641cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.684+0000 7fbf6ab27700 1 -- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640a0a30 msgr2=0x7fbf641cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.684+0000 7fbf6ab27700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640a0a30 0x7fbf641cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.684+0000 7fbf6ab27700 1 -- 172.21.15.89:0/2176031718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf600094a0 con 0x7fbf64138d80 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.684+0000 7fbf69b25700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640a0a30 0x7fbf641cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.684+0000 7fbf6ab27700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6406dac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbf640a23a0 tx=0x7fbf5c00cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.685+0000 7fbf5affd700 1 -- 172.21.15.89:0/2176031718 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00ad10 con 0x7fbf64138d80 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.685+0000 7fbf5affd700 1 -- 172.21.15.89:0/2176031718 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c00ae90 con 0x7fbf64138d80 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.685+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6406fbf0 con 0x7fbf64138d80 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.685+0000 7fbf5affd700 1 -- 172.21.15.89:0/2176031718 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003c60 con 0x7fbf64138d80 2023-09-30T22:17:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.685+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf64071550 con 0x7fbf64138d80 2023-09-30T22:17:00.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.685+0000 7fbf5affd700 1 -- 172.21.15.89:0/2176031718 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbf5c01f440 con 0x7fbf64138d80 2023-09-30T22:17:00.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.686+0000 7fbf6a326700 1 -- 172.21.15.89:0/2176031718 >> v1:172.21.15.89:6816/126953 conn(0x7fbf4802f510 legacy=0x7fbf480319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:00.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.686+0000 7fbf5affd700 1 -- 172.21.15.89:0/2176031718 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fbf5c012070 con 0x7fbf64138d80 2023-09-30T22:17:00.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.686+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf640713b0 con 0x7fbf64138d80 2023-09-30T22:17:00.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.691+0000 7fbf5affd700 1 -- 172.21.15.89:0/2176031718 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf640713b0 con 0x7fbf64138d80 2023-09-30T22:17:00.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.786+0000 7f1ca09e0700 1 -- 172.21.15.89:0/2750529266 >> v1:172.21.15.89:6816/126953 conn(0x7f1c8002f510 legacy=0x7f1c800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:00.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.793+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1c9c09a840 con 0x7f1c9c12cf60 2023-09-30T22:17:00.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.793+0000 7f1c997fa700 1 -- 172.21.15.89:0/2750529266 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c9c09a840 con 0x7f1c9c12cf60 2023-09-30T22:17:00.794 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 >> v1:172.21.15.89:6816/126953 conn(0x7f1c8002f510 legacy=0x7f1c800319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c12cf60 msgr2=0x7f1c9c1276b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c12cf60 0x7f1c9c1276b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800e9b0 tx=0x7f1c8800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 shutdown_connections 2023-09-30T22:17:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c135c20 0x7f1c9c1c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c06f5d0 0x7f1c9c127170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 --2- 172.21.15.89:0/2750529266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c12cf60 0x7f1c9c1276b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 >> 172.21.15.89:0/2750529266 conn(0x7f1c9c0a2aa0 msgr2=0x7f1c9c12f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 shutdown_connections 2023-09-30T22:17:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.796+0000 7f1ca2c44700 1 -- 172.21.15.89:0/2750529266 wait complete. 2023-09-30T22:17:00.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.829+0000 7f81a4f28700 1 -- 172.21.15.89:0/3152482707 >> v1:172.21.15.89:6816/126953 conn(0x7f818402f540 legacy=0x7f81840319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:00.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.831+0000 7fabaeffd700 1 -- 172.21.15.89:0/3250982058 >> v1:172.21.15.89:6816/126953 conn(0x7fab9c02f560 legacy=0x7fab9c031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:00.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.832+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fabb0131b60 con 0x7fabb0135c20 2023-09-30T22:17:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.832+0000 7fab977fe700 1 -- 172.21.15.89:0/3250982058 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fabb0131b60 con 0x7fabb0135c20 2023-09-30T22:17:00.833 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 >> v1:172.21.15.89:6816/126953 conn(0x7fab9c02f560 legacy=0x7fab9c031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb0135c20 msgr2=0x7fabb013a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb0135c20 0x7fabb013a660 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fabb0071fd0 tx=0x7faba800ef90 comp rx=0 tx=0).stop 2023-09-30T22:17:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 shutdown_connections 2023-09-30T22:17:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb0135c20 0x7fabb013a660 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb006f5d0 0x7fabb0134e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 --2- 172.21.15.89:0/3250982058 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb012cf60 0x7fabb01353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 >> 172.21.15.89:0/3250982058 conn(0x7fabb00a2aa0 msgr2=0x7fabb012f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 shutdown_connections 2023-09-30T22:17:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.834+0000 7fabb56cd700 1 -- 172.21.15.89:0/3250982058 wait complete. 2023-09-30T22:17:00.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:00.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.846+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f81a0132c00 con 0x7f81a0130540 2023-09-30T22:17:00.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.847+0000 7f8195ffb700 1 -- 172.21.15.89:0/3152482707 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f819c019d10 con 0x7f81a0130540 2023-09-30T22:17:00.848 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:00.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 >> v1:172.21.15.89:6816/126953 conn(0x7f818402f540 legacy=0x7f81840319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 msgr2=0x7f81a0134270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0134270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f819c0090b0 tx=0x7f819c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 shutdown_connections 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81a0135c90 0x7f81a01c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81a0130540 0x7f81a0134270 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 --2- 172.21.15.89:0/3152482707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81a0130e50 0x7f81a01c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 >> 172.21.15.89:0/3152482707 conn(0x7f81a00a2aa0 msgr2=0x7f81a00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 shutdown_connections 2023-09-30T22:17:00.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.849+0000 7f81a798d700 1 -- 172.21.15.89:0/3152482707 wait complete. 2023-09-30T22:17:00.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:00.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.887+0000 7fbf6a326700 1 -- 172.21.15.89:0/2176031718 >> v1:172.21.15.89:6816/126953 conn(0x7fbf4802f510 legacy=0x7fbf480319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:00.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.907+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbf64070ef0 con 0x7fbf64138d80 2023-09-30T22:17:00.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.908+0000 7fbf5affd700 1 -- 172.21.15.89:0/2176031718 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf64070ef0 con 0x7fbf64138d80 2023-09-30T22:17:00.908 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:00.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.909+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 >> v1:172.21.15.89:6816/126953 conn(0x7fbf4802f510 legacy=0x7fbf480319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.909+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 msgr2=0x7fbf6406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.909+0000 7fbf6c58a700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6406dac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbf640a23a0 tx=0x7fbf5c00cf80 comp rx=0 tx=0).stop 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.910+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 shutdown_connections 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.910+0000 7fbf6c58a700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf64138d80 0x7fbf6406dac0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.910+0000 7fbf6c58a700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf6409d630 0x7fbf641cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.910+0000 7fbf6c58a700 1 --2- 172.21.15.89:0/2176031718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf640a0a30 0x7fbf641cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.910+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 >> 172.21.15.89:0/2176031718 conn(0x7fbf640a3050 msgr2=0x7fbf6409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.910+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 shutdown_connections 2023-09-30T22:17:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:00.910+0000 7fbf6c58a700 1 -- 172.21.15.89:0/2176031718 wait complete. 2023-09-30T22:17:00.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:01.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:01.848 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:01.862 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:01.922 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:01.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.952+0000 7f8e2a99b700 1 Processor -- start 2023-09-30T22:17:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.952+0000 7f8e2a99b700 1 -- start start 2023-09-30T22:17:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.952+0000 7f8e2a99b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e241338c0 0x7f8e24133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.952+0000 7f8e2a99b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.952+0000 7f8e2a99b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2406f720 con 0x7f8e241341d0 2023-09-30T22:17:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e2a99b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e24070c80 con 0x7f8e241338c0 2023-09-30T22:17:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e2a99b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2406fce0 con 0x7f8e2409e560 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e23fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e241338c0 0x7f8e24133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e23fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e241338c0 0x7f8e24133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33176/0 (socket says 172.21.15.89:33176) 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e23fff700 1 -- 172.21.15.89:0/1097551340 learned_addr learned my addr 172.21.15.89:0/1097551340 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e28f38700 1 --2- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e23fff700 1 -- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 msgr2=0x7f8e2413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e23fff700 1 --2- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e23fff700 1 -- 172.21.15.89:0/1097551340 >> v1:172.21.15.89:6789/0 conn(0x7f8e241341d0 legacy=0x7f8e2409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e23fff700 1 -- 172.21.15.89:0/1097551340 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e241336a0 con 0x7f8e241338c0 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e28f38700 1 --2- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e23fff700 1 --2- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 0x7f8e24133c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e140098f0 tx=0x7f8e14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2103cc527ca7ec83 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.953+0000 7f8e22ffd700 1 -- 172.21.15.89:0/1097551340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14004230 con 0x7f8e241338c0 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.954+0000 7f8e22ffd700 1 -- 172.21.15.89:0/1097551340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e14004390 con 0x7f8e241338c0 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.954+0000 7f8e22ffd700 1 -- 172.21.15.89:0/1097551340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140036a0 con 0x7f8e241338c0 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.954+0000 7f8e2a99b700 1 -- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 msgr2=0x7f8e24133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.954+0000 7f8e2a99b700 1 --2- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 0x7f8e24133c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e140098f0 tx=0x7f8e14009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.954+0000 7f8e2a99b700 1 -- 172.21.15.89:0/1097551340 shutdown_connections 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.955+0000 7f8e2a99b700 1 --2- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.955+0000 7f8e2a99b700 1 --2- 172.21.15.89:0/1097551340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 0x7f8e24133c90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.955+0000 7f8e2a99b700 1 -- 172.21.15.89:0/1097551340 >> 172.21.15.89:0/1097551340 conn(0x7f8e240a2aa0 msgr2=0x7f8e240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.955+0000 7f8e2a99b700 1 -- 172.21.15.89:0/1097551340 shutdown_connections 2023-09-30T22:17:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.955+0000 7f8e2a99b700 1 -- 172.21.15.89:0/1097551340 wait complete. 2023-09-30T22:17:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.956+0000 7f8e2a99b700 1 Processor -- start 2023-09-30T22:17:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.956+0000 7f8e2a99b700 1 -- start start 2023-09-30T22:17:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e2a99b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2411e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e2a99b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 0x7f8e2411edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e2a99b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e241341d0 0x7f8e2411f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e2a99b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2411fca0 con 0x7f8e241338c0 2023-09-30T22:17:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e2a99b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e241222f0 con 0x7f8e2409e560 2023-09-30T22:17:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e2a99b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2412fc60 con 0x7f8e241341d0 2023-09-30T22:17:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e23fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2411e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e23fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2411e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46032/0 (socket says 172.21.15.89:46032) 2023-09-30T22:17:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e23fff700 1 -- 172.21.15.89:0/2808513379 learned_addr learned my addr 172.21.15.89:0/2808513379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e28f38700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e241341d0 0x7f8e2411f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.957+0000 7f8e237fe700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 0x7f8e2411edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e28f38700 1 -- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 msgr2=0x7f8e2411e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e28f38700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2411e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e28f38700 1 -- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 msgr2=0x7f8e2411edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e28f38700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 0x7f8e2411edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e28f38700 1 -- 172.21.15.89:0/2808513379 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c009a70 con 0x7f8e241341d0 2023-09-30T22:17:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e237fe700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 0x7f8e2411edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e28f38700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e241341d0 0x7f8e2411f330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00efc0 tx=0x7f8e1c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e20ff9700 1 -- 172.21.15.89:0/2808513379 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00ca90 con 0x7f8e241341d0 2023-09-30T22:17:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e20ff9700 1 -- 172.21.15.89:0/2808513379 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c00f040 con 0x7f8e241341d0 2023-09-30T22:17:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e14009550 con 0x7f8e241341d0 2023-09-30T22:17:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e20ff9700 1 -- 172.21.15.89:0/2808513379 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00ce30 con 0x7f8e241341d0 2023-09-30T22:17:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.958+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e240a12b0 con 0x7f8e241341d0 2023-09-30T22:17:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.959+0000 7f8e20ff9700 1 -- 172.21.15.89:0/2808513379 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8e1c014480 con 0x7f8e241341d0 2023-09-30T22:17:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.959+0000 7f8e20ff9700 1 -- 172.21.15.89:0/2808513379 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f8e1c0401c0 con 0x7f8e241341d0 2023-09-30T22:17:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.959+0000 7f8e23fff700 1 -- 172.21.15.89:0/2808513379 >> v1:172.21.15.89:6816/126953 conn(0x7f8e1002f510 legacy=0x7f8e100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.959+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e040052e0 con 0x7f8e241341d0 2023-09-30T22:17:01.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.968+0000 7f8e20ff9700 1 -- 172.21.15.89:0/2808513379 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e1c018070 con 0x7f8e241341d0 2023-09-30T22:17:01.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.995+0000 7fafcf525700 1 Processor -- start 2023-09-30T22:17:01.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.995+0000 7fafcf525700 1 -- start start 2023-09-30T22:17:01.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.995+0000 7fafcf525700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc809e1d0 0x7fafc8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:01.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.995+0000 7fafcf525700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc8135e50 0x7fafc8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:01.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcf525700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc809f130 con 0x7fafc809d6f0 2023-09-30T22:17:01.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcf525700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc809f520 con 0x7fafc809e1d0 2023-09-30T22:17:01.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcf525700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafc809e930 con 0x7fafc8135e50 2023-09-30T22:17:01.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcd2c1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fafc809d6f0 0x7fafc809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59862/0 (socket says 172.21.15.89:59862) 2023-09-30T22:17:01.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcd2c1700 1 -- 172.21.15.89:0/4037015898 learned_addr learned my addr 172.21.15.89:0/4037015898 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:01.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcdac2700 1 --2- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc8135e50 0x7fafc8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:01.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafccac0700 1 --2- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc809e1d0 0x7fafc8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:01.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcdac2700 1 -- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc809e1d0 msgr2=0x7fafc8135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:01.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcdac2700 1 --2- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc809e1d0 0x7fafc8135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcdac2700 1 -- 172.21.15.89:0/4037015898 >> v1:172.21.15.89:6789/0 conn(0x7fafc809d6f0 legacy=0x7fafc809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcdac2700 1 -- 172.21.15.89:0/4037015898 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc809b2d0 con 0x7fafc8135e50 2023-09-30T22:17:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafbffff700 1 -- 172.21.15.89:0/4037015898 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700585326 0 0) 0x7fafc809f130 con 0x7fafc809d6f0 2023-09-30T22:17:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafcdac2700 1 --2- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc8135e50 0x7fafc8138230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009dc0 tx=0x7fafc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c7574d634be9cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.996+0000 7fafbffff700 1 -- 172.21.15.89:0/4037015898 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000e070 con 0x7fafc8135e50 2023-09-30T22:17:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafbffff700 1 -- 172.21.15.89:0/4037015898 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000bb20 con 0x7fafc8135e50 2023-09-30T22:17:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafbffff700 1 -- 172.21.15.89:0/4037015898 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0012430 con 0x7fafc8135e50 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafcf525700 1 -- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc8135e50 msgr2=0x7fafc8138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafcf525700 1 --2- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc8135e50 0x7fafc8138230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009dc0 tx=0x7fafc000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafcf525700 1 -- 172.21.15.89:0/4037015898 shutdown_connections 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafcf525700 1 --2- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc8135e50 0x7fafc8138230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafcf525700 1 --2- 172.21.15.89:0/4037015898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc809e1d0 0x7fafc8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafcf525700 1 -- 172.21.15.89:0/4037015898 >> 172.21.15.89:0/4037015898 conn(0x7fafc8066f50 msgr2=0x7fafc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafcf525700 1 -- 172.21.15.89:0/4037015898 shutdown_connections 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.997+0000 7fafcf525700 1 -- 172.21.15.89:0/4037015898 wait complete. 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.998+0000 7fafcf525700 1 Processor -- start 2023-09-30T22:17:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.998+0000 7fafcf525700 1 -- start start 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcf525700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafc809d6f0 0x7fafc8119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcf525700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc809e1d0 0x7fafc81199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcf525700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc8135e50 0x7fafc8119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcf525700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc811c980 con 0x7fafc809d6f0 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcf525700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc809a0c0 con 0x7fafc8135e50 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcf525700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafc809c700 con 0x7fafc809e1d0 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcdac2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc8135e50 0x7fafc8119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcdac2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc8135e50 0x7fafc8119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46052/0 (socket says 172.21.15.89:46052) 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcdac2700 1 -- 172.21.15.89:0/3824946395 learned_addr learned my addr 172.21.15.89:0/3824946395 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafccac0700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc809e1d0 0x7fafc81199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:01.999+0000 7fafcd2c1700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafc809d6f0 0x7fafc8119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcdac2700 1 -- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc809e1d0 msgr2=0x7fafc81199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcdac2700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc809e1d0 0x7fafc81199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcdac2700 1 -- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafc809d6f0 msgr2=0x7fafc8119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcdac2700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafc809d6f0 0x7fafc8119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcdac2700 1 -- 172.21.15.89:0/3824946395 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0009a70 con 0x7fafc8135e50 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcd2c1700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafc809d6f0 0x7fafc8119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafccac0700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc809e1d0 0x7fafc81199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcdac2700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc8135e50 0x7fafc8119f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fafc0013000 tx=0x7fafc0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafbdffb700 1 -- 172.21.15.89:0/3824946395 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0012430 con 0x7fafc8135e50 2023-09-30T22:17:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafbdffb700 1 -- 172.21.15.89:0/3824946395 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc0020070 con 0x7fafc8135e50 2023-09-30T22:17:02.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc811b040 con 0x7fafc8135e50 2023-09-30T22:17:02.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafbdffb700 1 -- 172.21.15.89:0/3824946395 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0004280 con 0x7fafc8135e50 2023-09-30T22:17:02.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.000+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc81301b0 con 0x7fafc8135e50 2023-09-30T22:17:02.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.001+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc8074ea0 con 0x7fafc8135e50 2023-09-30T22:17:02.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.002+0000 7fafbdffb700 1 -- 172.21.15.89:0/3824946395 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fafc00043e0 con 0x7fafc8135e50 2023-09-30T22:17:02.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.002+0000 7fafbdffb700 1 -- 172.21.15.89:0/3824946395 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fafc0022070 con 0x7fafc8135e50 2023-09-30T22:17:02.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.002+0000 7fafcd2c1700 1 -- 172.21.15.89:0/3824946395 >> v1:172.21.15.89:6816/126953 conn(0x7fafac02f510 legacy=0x7fafac0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:02.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.010+0000 7fafbdffb700 1 -- 172.21.15.89:0/3824946395 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fafc001b400 con 0x7fafc8135e50 2023-09-30T22:17:02.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.041+0000 7fb50ed35700 1 Processor -- start 2023-09-30T22:17:02.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.041+0000 7fb50ed35700 1 -- start start 2023-09-30T22:17:02.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb50ed35700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb508133a80 0x7fb508133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb50ed35700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb50812ff20 0x7fb508130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb50ed35700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb508130870 0x7fb50813a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb50ed35700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50806c570 con 0x7fb508130870 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb50ed35700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50806bc00 con 0x7fb50812ff20 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb50ed35700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080a0040 con 0x7fb508133a80 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb507fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb50812ff20 0x7fb508130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb50cad1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb508133a80 0x7fb508133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb507fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb50812ff20 0x7fb508130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46060/0 (socket says 172.21.15.89:46060) 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb507fff700 1 -- 172.21.15.89:0/1196432322 learned_addr learned my addr 172.21.15.89:0/1196432322 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb507fff700 1 -- 172.21.15.89:0/1196432322 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb508133a80 msgr2=0x7fb508133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb507fff700 1 --2- 172.21.15.89:0/1196432322 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb508133a80 0x7fb508133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb507fff700 1 -- 172.21.15.89:0/1196432322 >> v2:172.21.15.89:3300/0 conn(0x7fb508130870 msgr2=0x7fb50813a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:02.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb507fff700 1 --2- 172.21.15.89:0/1196432322 >> v2:172.21.15.89:3300/0 conn(0x7fb508130870 0x7fb50813a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.042+0000 7fb507fff700 1 -- 172.21.15.89:0/1196432322 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb508133860 con 0x7fb50812ff20 2023-09-30T22:17:02.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.043+0000 7fb507fff700 1 --2- 172.21.15.89:0/1196432322 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb50812ff20 0x7fb508130330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40098f0 tx=0x7fb4f4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41733517df79a906 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:02.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.043+0000 7fb5077fe700 1 -- 172.21.15.89:0/1196432322 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4011830 con 0x7fb50812ff20 2023-09-30T22:17:02.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.043+0000 7fb5077fe700 1 -- 172.21.15.89:0/1196432322 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f4011990 con 0x7fb50812ff20 2023-09-30T22:17:02.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.043+0000 7fb5077fe700 1 -- 172.21.15.89:0/1196432322 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4011d20 con 0x7fb50812ff20 2023-09-30T22:17:02.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.043+0000 7fb50ed35700 1 -- 172.21.15.89:0/1196432322 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb50812ff20 msgr2=0x7fb508130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.043+0000 7fb50ed35700 1 --2- 172.21.15.89:0/1196432322 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb50812ff20 0x7fb508130330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40098f0 tx=0x7fb4f4009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.044+0000 7fb50ed35700 1 -- 172.21.15.89:0/1196432322 shutdown_connections 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.044+0000 7fb50ed35700 1 --2- 172.21.15.89:0/1196432322 >> v2:172.21.15.89:3300/0 conn(0x7fb508130870 0x7fb50813a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.044+0000 7fb50ed35700 1 --2- 172.21.15.89:0/1196432322 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb508133a80 0x7fb508133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.044+0000 7fb50ed35700 1 --2- 172.21.15.89:0/1196432322 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb50812ff20 0x7fb508130330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.044+0000 7fb50ed35700 1 -- 172.21.15.89:0/1196432322 >> 172.21.15.89:0/1196432322 conn(0x7fb5080a4210 msgr2=0x7fb5080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.044+0000 7fb50ed35700 1 -- 172.21.15.89:0/1196432322 shutdown_connections 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.044+0000 7fb50ed35700 1 -- 172.21.15.89:0/1196432322 wait complete. 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50ed35700 1 Processor -- start 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50ed35700 1 -- start start 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50ed35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb508130870 0x7fb50809d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50ed35700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb508133a80 0x7fb50809d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50ed35700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5081c87f0 0x7fb5081c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50ed35700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50812dc20 con 0x7fb508130870 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50ed35700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50811f270 con 0x7fb508133a80 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50ed35700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb50809cd00 con 0x7fb5081c87f0 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50cad1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb508130870 0x7fb50809d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50cad1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb508130870 0x7fb50809d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33218/0 (socket says 172.21.15.89:33218) 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50cad1700 1 -- 172.21.15.89:0/3994586930 learned_addr learned my addr 172.21.15.89:0/3994586930 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb50d2d2700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5081c87f0 0x7fb5081c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.045+0000 7fb507fff700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb508133a80 0x7fb50809d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb50cad1700 1 -- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5081c87f0 msgr2=0x7fb5081c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb50cad1700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5081c87f0 0x7fb5081c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb50cad1700 1 -- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb508133a80 msgr2=0x7fb50809d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb50cad1700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb508133a80 0x7fb50809d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb50cad1700 1 -- 172.21.15.89:0/3994586930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009550 con 0x7fb508130870 2023-09-30T22:17:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb50cad1700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb508130870 0x7fb50809d0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00e9b0 tx=0x7fb4fc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb5057fa700 1 -- 172.21.15.89:0/3994586930 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00ee50 con 0x7fb508130870 2023-09-30T22:17:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb5057fa700 1 -- 172.21.15.89:0/3994586930 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc00c7f0 con 0x7fb508130870 2023-09-30T22:17:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb5057fa700 1 -- 172.21.15.89:0/3994586930 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00cb60 con 0x7fb508130870 2023-09-30T22:17:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5081205e0 con 0x7fb508130870 2023-09-30T22:17:02.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.046+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5081208a0 con 0x7fb508130870 2023-09-30T22:17:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.047+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5080a21a0 con 0x7fb508130870 2023-09-30T22:17:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.047+0000 7fb5057fa700 1 -- 172.21.15.89:0/3994586930 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb4fc00ccc0 con 0x7fb508130870 2023-09-30T22:17:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.048+0000 7fb5057fa700 1 -- 172.21.15.89:0/3994586930 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fb4fc03fa20 con 0x7fb508130870 2023-09-30T22:17:02.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.048+0000 7fb507fff700 1 -- 172.21.15.89:0/3994586930 >> v1:172.21.15.89:6816/126953 conn(0x7fb4ec02f560 legacy=0x7fb4ec031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:02.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.056+0000 7fb5057fa700 1 -- 172.21.15.89:0/3994586930 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4fc013470 con 0x7fb508130870 2023-09-30T22:17:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.082+0000 7f4c99aed700 1 Processor -- start 2023-09-30T22:17:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.082+0000 7f4c99aed700 1 -- start start 2023-09-30T22:17:02.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c99aed700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c941337f0 0x7f4c94138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c99aed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94138dc0 0x7f4c9413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c99aed700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9406b990 con 0x7f4c94133420 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c99aed700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9409b0e0 con 0x7f4c941337f0 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c99aed700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94133c20 con 0x7f4c94138dc0 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c937fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4c94133420 0x7f4c94133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59878/0 (socket says 172.21.15.89:59878) 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c92ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c941337f0 0x7f4c94138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c937fe700 1 -- 172.21.15.89:0/1381037842 learned_addr learned my addr 172.21.15.89:0/1381037842 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c93fff700 1 --2- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94138dc0 0x7f4c9413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c92ffd700 1 -- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94138dc0 msgr2=0x7f4c9413b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c92ffd700 1 --2- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94138dc0 0x7f4c9413b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c92ffd700 1 -- 172.21.15.89:0/1381037842 >> v1:172.21.15.89:6789/0 conn(0x7f4c94133420 legacy=0x7f4c94133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c92ffd700 1 -- 172.21.15.89:0/1381037842 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940a0590 con 0x7f4c941337f0 2023-09-30T22:17:02.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.083+0000 7f4c93fff700 1 --2- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94138dc0 0x7f4c9413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:02.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.084+0000 7f4c92ffd700 1 --2- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 0x7f4c94138880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009840 tx=0x7f4c88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b4257dc9430cdc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:02.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.084+0000 7f4c927fc700 1 -- 172.21.15.89:0/1381037842 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88011830 con 0x7f4c941337f0 2023-09-30T22:17:02.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.084+0000 7f4c927fc700 1 -- 172.21.15.89:0/1381037842 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c88011990 con 0x7f4c941337f0 2023-09-30T22:17:02.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.084+0000 7f4c927fc700 1 -- 172.21.15.89:0/1381037842 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88011d20 con 0x7f4c941337f0 2023-09-30T22:17:02.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.085+0000 7f4c99aed700 1 -- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 msgr2=0x7f4c94138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.085+0000 7f4c99aed700 1 --2- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 0x7f4c94138880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009840 tx=0x7f4c88009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.085+0000 7f4c99aed700 1 -- 172.21.15.89:0/1381037842 shutdown_connections 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.085+0000 7f4c99aed700 1 --2- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94138dc0 0x7f4c9413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.085+0000 7f4c99aed700 1 --2- 172.21.15.89:0/1381037842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 0x7f4c94138880 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.085+0000 7f4c99aed700 1 -- 172.21.15.89:0/1381037842 >> 172.21.15.89:0/1381037842 conn(0x7f4c940a4210 msgr2=0x7f4c940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.085+0000 7f4c99aed700 1 -- 172.21.15.89:0/1381037842 shutdown_connections 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.085+0000 7f4c99aed700 1 -- 172.21.15.89:0/1381037842 wait complete. 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.086+0000 7f4c99aed700 1 Processor -- start 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.086+0000 7f4c99aed700 1 -- start start 2023-09-30T22:17:02.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.086+0000 7f4c99aed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94133420 0x7f4c9409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.086+0000 7f4c99aed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 0x7f4c9409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.086+0000 7f4c99aed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c94138dc0 0x7f4c9409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.086+0000 7f4c99aed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9412df60 con 0x7f4c941337f0 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.086+0000 7f4c99aed700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9412f950 con 0x7f4c94133420 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.086+0000 7f4c99aed700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94130780 con 0x7f4c94138dc0 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94133420 0x7f4c9409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c93fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c94138dc0 0x7f4c9409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94133420 0x7f4c9409c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46078/0 (socket says 172.21.15.89:46078) 2023-09-30T22:17:02.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 -- 172.21.15.89:0/502218176 learned_addr learned my addr 172.21.15.89:0/502218176 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:02.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c92ffd700 1 --2- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 0x7f4c9409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:02.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 -- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c94138dc0 msgr2=0x7f4c9409d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 --2- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c94138dc0 0x7f4c9409d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 -- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 msgr2=0x7f4c9409cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 --2- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 0x7f4c9409cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 -- 172.21.15.89:0/502218176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009a70 con 0x7f4c94133420 2023-09-30T22:17:02.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c937fe700 1 --2- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94133420 0x7f4c9409c9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c550 tx=0x7f4c8400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:02.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c7bfff700 1 -- 172.21.15.89:0/502218176 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84005d70 con 0x7f4c94133420 2023-09-30T22:17:02.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c7bfff700 1 -- 172.21.15.89:0/502218176 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c840070d0 con 0x7f4c94133420 2023-09-30T22:17:02.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c880094a0 con 0x7f4c94133420 2023-09-30T22:17:02.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c7bfff700 1 -- 172.21.15.89:0/502218176 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84007440 con 0x7f4c94133420 2023-09-30T22:17:02.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.087+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c94130d40 con 0x7f4c94133420 2023-09-30T22:17:02.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.088+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94074ea0 con 0x7f4c94133420 2023-09-30T22:17:02.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.089+0000 7f4c7bfff700 1 -- 172.21.15.89:0/502218176 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4c840075a0 con 0x7f4c94133420 2023-09-30T22:17:02.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.089+0000 7f4c7bfff700 1 -- 172.21.15.89:0/502218176 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f4c8403f510 con 0x7f4c94133420 2023-09-30T22:17:02.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.089+0000 7f4c92ffd700 1 -- 172.21.15.89:0/502218176 >> v1:172.21.15.89:6816/126953 conn(0x7f4c8002f510 legacy=0x7f4c800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:02.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.093+0000 7f4c7bfff700 1 -- 172.21.15.89:0/502218176 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4c84019d20 con 0x7f4c94133420 2023-09-30T22:17:02.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.160+0000 7f8e23fff700 1 -- 172.21.15.89:0/2808513379 >> v1:172.21.15.89:6816/126953 conn(0x7f8e1002f510 legacy=0x7f8e100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:02.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.168+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8e04005150 con 0x7f8e241341d0 2023-09-30T22:17:02.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.168+0000 7f8e20ff9700 1 -- 172.21.15.89:0/2808513379 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e1c012930 con 0x7f8e241341d0 2023-09-30T22:17:02.170 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:02.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.170+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 >> v1:172.21.15.89:6816/126953 conn(0x7f8e1002f510 legacy=0x7f8e100319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:02.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.170+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e241341d0 msgr2=0x7f8e2411f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.170+0000 7f8e2a99b700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e241341d0 0x7f8e2411f330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00efc0 tx=0x7f8e1c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.170+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 shutdown_connections 2023-09-30T22:17:02.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.171+0000 7f8e2a99b700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e241341d0 0x7f8e2411f330 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.171+0000 7f8e2a99b700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e2409e560 0x7f8e2411e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.171+0000 7f8e2a99b700 1 --2- 172.21.15.89:0/2808513379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e241338c0 0x7f8e2411edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.171+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 >> 172.21.15.89:0/2808513379 conn(0x7f8e240a2aa0 msgr2=0x7f8e24130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:02.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.171+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 shutdown_connections 2023-09-30T22:17:02.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.171+0000 7f8e2a99b700 1 -- 172.21.15.89:0/2808513379 wait complete. 2023-09-30T22:17:02.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:02.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.203+0000 7fafcd2c1700 1 -- 172.21.15.89:0/3824946395 >> v1:172.21.15.89:6816/126953 conn(0x7fafac02f510 legacy=0x7fafac0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:02.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.207+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fafc80a20d0 con 0x7fafc8135e50 2023-09-30T22:17:02.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.207+0000 7fafbdffb700 1 -- 172.21.15.89:0/3824946395 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafc0025b70 con 0x7fafc8135e50 2023-09-30T22:17:02.208 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 >> v1:172.21.15.89:6816/126953 conn(0x7fafac02f510 legacy=0x7fafac0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc8135e50 msgr2=0x7fafc8119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc8135e50 0x7fafc8119f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fafc0013000 tx=0x7fafc0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 shutdown_connections 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafc8135e50 0x7fafc8119f00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafc809d6f0 0x7fafc8119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 --2- 172.21.15.89:0/3824946395 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafc809e1d0 0x7fafc81199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 >> 172.21.15.89:0/3824946395 conn(0x7fafc8066f50 msgr2=0x7fafc8132ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 shutdown_connections 2023-09-30T22:17:02.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.210+0000 7fafcf525700 1 -- 172.21.15.89:0/3824946395 wait complete. 2023-09-30T22:17:02.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:02.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.249+0000 7fb507fff700 1 -- 172.21.15.89:0/3994586930 >> v1:172.21.15.89:6816/126953 conn(0x7fb4ec02f560 legacy=0x7fb4ec031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:02.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.254+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb5081208a0 con 0x7fb508130870 2023-09-30T22:17:02.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.254+0000 7fb5057fa700 1 -- 172.21.15.89:0/3994586930 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5081208a0 con 0x7fb508130870 2023-09-30T22:17:02.255 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:02.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 >> v1:172.21.15.89:6816/126953 conn(0x7fb4ec02f560 legacy=0x7fb4ec031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:02.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb508130870 msgr2=0x7fb50809d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb508130870 0x7fb50809d0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00e9b0 tx=0x7fb4fc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 shutdown_connections 2023-09-30T22:17:02.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5081c87f0 0x7fb5081c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb508130870 0x7fb50809d0d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 --2- 172.21.15.89:0/3994586930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb508133a80 0x7fb50809d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 >> 172.21.15.89:0/3994586930 conn(0x7fb5080a4210 msgr2=0x7fb50809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:02.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 shutdown_connections 2023-09-30T22:17:02.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.256+0000 7fb50ed35700 1 -- 172.21.15.89:0/3994586930 wait complete. 2023-09-30T22:17:02.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:02.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.289+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4c9406b250 con 0x7f4c94133420 2023-09-30T22:17:02.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.289+0000 7f4c7bfff700 1 -- 172.21.15.89:0/502218176 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c84017030 con 0x7f4c94133420 2023-09-30T22:17:02.290 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:02.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.290+0000 7f4c92ffd700 1 -- 172.21.15.89:0/502218176 >> v1:172.21.15.89:6816/126953 conn(0x7f4c8002f510 legacy=0x7f4c800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:02.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 >> v1:172.21.15.89:6816/126953 conn(0x7f4c8002f510 legacy=0x7f4c800319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:02.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94133420 msgr2=0x7f4c9409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:02.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 --2- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94133420 0x7f4c9409c9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c550 tx=0x7f4c8400c860 comp rx=0 tx=0).stop 2023-09-30T22:17:02.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 shutdown_connections 2023-09-30T22:17:02.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 --2- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c94138dc0 0x7f4c9409d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 --2- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c94133420 0x7f4c9409c9e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 --2- 172.21.15.89:0/502218176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c941337f0 0x7f4c9409cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:02.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 >> 172.21.15.89:0/502218176 conn(0x7f4c940a4210 msgr2=0x7f4c940a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:02.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 shutdown_connections 2023-09-30T22:17:02.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:02.291+0000 7f4c99aed700 1 -- 172.21.15.89:0/502218176 wait complete. 2023-09-30T22:17:02.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:03.185 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:03.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:03.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:03.305 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:03.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.334+0000 7f721180a700 1 Processor -- start 2023-09-30T22:17:03.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.334+0000 7f721180a700 1 -- start start 2023-09-30T22:17:03.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f721180a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09d6f0 0x7f720c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f721180a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e410 0x7f720c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f721180a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720c09b120 con 0x7f720c09e000 2023-09-30T22:17:03.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f721180a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c06b460 con 0x7f720c09d6f0 2023-09-30T22:17:03.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f721180a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f720c09a7e0 con 0x7f720c09e410 2023-09-30T22:17:03.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f720affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09d6f0 0x7f720c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f720affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09d6f0 0x7f720c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46092/0 (socket says 172.21.15.89:46092) 2023-09-30T22:17:03.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f720affd700 1 -- 172.21.15.89:0/38123399 learned_addr learned my addr 172.21.15.89:0/38123399 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:03.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f720affd700 1 -- 172.21.15.89:0/38123399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e410 msgr2=0x7f720c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:03.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f720affd700 1 --2- 172.21.15.89:0/38123399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e410 0x7f720c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f720affd700 1 -- 172.21.15.89:0/38123399 >> v1:172.21.15.89:6789/0 conn(0x7f720c09e000 legacy=0x7f720c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:03.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f720affd700 1 -- 172.21.15.89:0/38123399 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c09a500 con 0x7f720c09d6f0 2023-09-30T22:17:03.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.335+0000 7f720affd700 1 --2- 172.21.15.89:0/38123399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09d6f0 0x7f720c09dac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009920 tx=0x7f71fc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aaf05b8c687bcb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:03.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.336+0000 7f7209ffb700 1 -- 172.21.15.89:0/38123399 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004230 con 0x7f720c09d6f0 2023-09-30T22:17:03.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.337+0000 7f7209ffb700 1 -- 172.21.15.89:0/38123399 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc004390 con 0x7f720c09d6f0 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.337+0000 7f7209ffb700 1 -- 172.21.15.89:0/38123399 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc009e70 con 0x7f720c09d6f0 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.337+0000 7f721180a700 1 -- 172.21.15.89:0/38123399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09d6f0 msgr2=0x7f720c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.337+0000 7f721180a700 1 --2- 172.21.15.89:0/38123399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09d6f0 0x7f720c09dac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009920 tx=0x7f71fc009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.337+0000 7f721180a700 1 -- 172.21.15.89:0/38123399 shutdown_connections 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.337+0000 7f721180a700 1 --2- 172.21.15.89:0/38123399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09d6f0 0x7f720c09dac0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009920 tx=0x7f71fc009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.337+0000 7f721180a700 1 --2- 172.21.15.89:0/38123399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e410 0x7f720c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.337+0000 7f721180a700 1 -- 172.21.15.89:0/38123399 >> 172.21.15.89:0/38123399 conn(0x7f720c066f50 msgr2=0x7f720c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.338+0000 7f721180a700 1 -- 172.21.15.89:0/38123399 shutdown_connections 2023-09-30T22:17:03.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.338+0000 7f721180a700 1 -- 172.21.15.89:0/38123399 wait complete. 2023-09-30T22:17:03.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.339+0000 7f721180a700 1 Processor -- start 2023-09-30T22:17:03.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.339+0000 7f721180a700 1 -- start start 2023-09-30T22:17:03.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f721180a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e000 0x7f720c122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f721180a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09e410 0x7f720c122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f721180a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720c1231f0 0x7f720c1c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f721180a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1263b0 con 0x7f720c1231f0 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f721180a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c125a00 con 0x7f720c09e410 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f721180a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f720c124420 con 0x7f720c09e000 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f720a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09e410 0x7f720c122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f720b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720c1231f0 0x7f720c1c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f720a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09e410 0x7f720c122cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46104/0 (socket says 172.21.15.89:46104) 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f720a7fc700 1 -- 172.21.15.89:0/3497202005 learned_addr learned my addr 172.21.15.89:0/3497202005 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f720b7fe700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720c1231f0 0x7f720c1c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33260/0 (socket says 172.21.15.89:33260) 2023-09-30T22:17:03.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.340+0000 7f720affd700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e000 0x7f720c122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f720a7fc700 1 -- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e000 msgr2=0x7f720c122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f720a7fc700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e000 0x7f720c122770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f720a7fc700 1 -- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720c1231f0 msgr2=0x7f720c1c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f720a7fc700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720c1231f0 0x7f720c1c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f720a7fc700 1 -- 172.21.15.89:0/3497202005 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009550 con 0x7f720c09e410 2023-09-30T22:17:03.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f720b7fe700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720c1231f0 0x7f720c1c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:03.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f720a7fc700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09e410 0x7f720c122cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f720000c4a0 tx=0x7f720000c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:03.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f71f3fff700 1 -- 172.21.15.89:0/3497202005 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000054e0 con 0x7f720c09e410 2023-09-30T22:17:03.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f71f3fff700 1 -- 172.21.15.89:0/3497202005 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7200003680 con 0x7f720c09e410 2023-09-30T22:17:03.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f71f3fff700 1 -- 172.21.15.89:0/3497202005 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200003a10 con 0x7f720c09e410 2023-09-30T22:17:03.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c125cf0 con 0x7f720c09e410 2023-09-30T22:17:03.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.341+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c133730 con 0x7f720c09e410 2023-09-30T22:17:03.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.342+0000 7f71f3fff700 1 -- 172.21.15.89:0/3497202005 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7200003b70 con 0x7f720c09e410 2023-09-30T22:17:03.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.342+0000 7f720affd700 1 -- 172.21.15.89:0/3497202005 >> v1:172.21.15.89:6816/126953 conn(0x7f71f802f560 legacy=0x7f71f8031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:03.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.342+0000 7f71f3fff700 1 -- 172.21.15.89:0/3497202005 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f720003efe0 con 0x7f720c09e410 2023-09-30T22:17:03.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.343+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c1260d0 con 0x7f720c09e410 2023-09-30T22:17:03.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.352+0000 7f71f3fff700 1 -- 172.21.15.89:0/3497202005 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7200017020 con 0x7f720c09e410 2023-09-30T22:17:03.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.378+0000 7f92c1e2b700 1 Processor -- start 2023-09-30T22:17:03.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.378+0000 7f92c1e2b700 1 -- start start 2023-09-30T22:17:03.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92c1e2b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92bc06a770 0x7f92bc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92c1e2b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92c1e2b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc131b40 0x7f92bc131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92c1e2b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc09ece0 con 0x7f92bc06a770 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92c1e2b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc09e760 con 0x7f92bc06b080 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92c1e2b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc09e040 con 0x7f92bc131b40 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92baffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92bb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92bc06a770 0x7f92bc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92baffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46120/0 (socket says 172.21.15.89:46120) 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92baffd700 1 -- 172.21.15.89:0/3077702108 learned_addr learned my addr 172.21.15.89:0/3077702108 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.379+0000 7f92bbfff700 1 --2- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc131b40 0x7f92bc131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92baffd700 1 -- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc131b40 msgr2=0x7f92bc131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92baffd700 1 --2- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc131b40 0x7f92bc131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92baffd700 1 -- 172.21.15.89:0/3077702108 >> v2:172.21.15.89:3300/0 conn(0x7f92bc06a770 msgr2=0x7f92bc06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92baffd700 1 --2- 172.21.15.89:0/3077702108 >> v2:172.21.15.89:3300/0 conn(0x7f92bc06a770 0x7f92bc06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92baffd700 1 -- 172.21.15.89:0/3077702108 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc09f630 con 0x7f92bc06b080 2023-09-30T22:17:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92bb7fe700 1 --2- 172.21.15.89:0/3077702108 >> v2:172.21.15.89:3300/0 conn(0x7f92bc06a770 0x7f92bc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92bbfff700 1 --2- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc131b40 0x7f92bc131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:03.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92baffd700 1 --2- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc131600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f92b00098f0 tx=0x7f92b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bbeb8b5dda84dc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92ba7fc700 1 -- 172.21.15.89:0/3077702108 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0011830 con 0x7f92bc06b080 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92ba7fc700 1 -- 172.21.15.89:0/3077702108 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b0011990 con 0x7f92bc06b080 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.380+0000 7f92ba7fc700 1 -- 172.21.15.89:0/3077702108 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0011d20 con 0x7f92bc06b080 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 msgr2=0x7f92bc131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 --2- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc131600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f92b00098f0 tx=0x7f92b0009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3077702108 shutdown_connections 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 --2- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc131b40 0x7f92bc131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 --2- 172.21.15.89:0/3077702108 >> v2:172.21.15.89:3300/0 conn(0x7f92bc06a770 0x7f92bc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 --2- 172.21.15.89:0/3077702108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc131600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3077702108 >> 172.21.15.89:0/3077702108 conn(0x7f92bc066f50 msgr2=0x7f92bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3077702108 shutdown_connections 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.381+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3077702108 wait complete. 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.382+0000 7f92c1e2b700 1 Processor -- start 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.382+0000 7f92c1e2b700 1 -- start start 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92c1e2b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc06a770 0x7f92bc11f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92c1e2b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc11f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92c1e2b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92bc131b40 0x7f92bc1c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92c1e2b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc130ec0 con 0x7f92bc131b40 2023-09-30T22:17:03.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92c1e2b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc11ed00 con 0x7f92bc06b080 2023-09-30T22:17:03.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92c1e2b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc11fc70 con 0x7f92bc06a770 2023-09-30T22:17:03.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92bb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc06a770 0x7f92bc11f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92bb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc06a770 0x7f92bc11f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42610/0 (socket says 172.21.15.89:42610) 2023-09-30T22:17:03.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92baffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc11f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.383+0000 7f92bb7fe700 1 -- 172.21.15.89:0/3487544383 learned_addr learned my addr 172.21.15.89:0/3487544383 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:03.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92bb7fe700 1 -- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 msgr2=0x7f92bc11f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92bb7fe700 1 --2- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc11f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92bb7fe700 1 -- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92bc131b40 msgr2=0x7f92bc1c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92bb7fe700 1 --2- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92bc131b40 0x7f92bc1c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92bb7fe700 1 -- 172.21.15.89:0/3487544383 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac00d940 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92bb7fe700 1 --2- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc06a770 0x7f92bc11f1f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00bef0 tx=0x7f92ac00df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92a3fff700 1 -- 172.21.15.89:0/3487544383 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac008980 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92a3fff700 1 -- 172.21.15.89:0/3487544383 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac008ae0 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b4014660 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92a3fff700 1 -- 172.21.15.89:0/3487544383 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac008e50 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.384+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b0009550 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.385+0000 7f92a3fff700 1 -- 172.21.15.89:0/3487544383 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f92ac016480 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.385+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc071e60 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.385+0000 7f92a3fff700 1 -- 172.21.15.89:0/3487544383 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f92ac016920 con 0x7f92bc06a770 2023-09-30T22:17:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.385+0000 7f92baffd700 1 -- 172.21.15.89:0/3487544383 >> v1:172.21.15.89:6816/126953 conn(0x7f92a802f560 legacy=0x7f92a8031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.390+0000 7f92a3fff700 1 -- 172.21.15.89:0/3487544383 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92ac01b070 con 0x7f92bc06a770 2023-09-30T22:17:03.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.411+0000 7f41f6e98700 1 Processor -- start 2023-09-30T22:17:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.411+0000 7f41f6e98700 1 -- start start 2023-09-30T22:17:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f6e98700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f41f006e570 0x7f41f006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f6e98700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f009eca0 0x7f41f006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f6e98700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f0070ee0 con 0x7f41f0134610 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f6e98700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f0071480 con 0x7f41f006e570 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f6e98700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41f006f3c0 con 0x7f41f009eca0 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f4c34700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f41f006e570 0x7f41f006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f4c34700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f41f006e570 0x7f41f006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33306/0 (socket says 172.21.15.89:33306) 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f4c34700 1 -- 172.21.15.89:0/2343593739 learned_addr learned my addr 172.21.15.89:0/2343593739 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41effff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f009eca0 0x7f41f006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f4c34700 1 -- 172.21.15.89:0/2343593739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f009eca0 msgr2=0x7f41f006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f4c34700 1 --2- 172.21.15.89:0/2343593739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f009eca0 0x7f41f006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f4c34700 1 -- 172.21.15.89:0/2343593739 >> v1:172.21.15.89:6789/0 conn(0x7f41f0134610 legacy=0x7f41f0134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f4c34700 1 -- 172.21.15.89:0/2343593739 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0071b90 con 0x7f41f006e570 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41effff700 1 --2- 172.21.15.89:0/2343593739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f009eca0 0x7f41f006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.412+0000 7f41f4c34700 1 --2- 172.21.15.89:0/2343593739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f006e940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41e40098f0 tx=0x7f41e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b07855b696c2564 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41ef7fe700 1 -- 172.21.15.89:0/2343593739 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4011830 con 0x7f41f006e570 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41ef7fe700 1 -- 172.21.15.89:0/2343593739 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4011990 con 0x7f41f006e570 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41ef7fe700 1 -- 172.21.15.89:0/2343593739 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4011d20 con 0x7f41f006e570 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41f6e98700 1 -- 172.21.15.89:0/2343593739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 msgr2=0x7f41f006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41f6e98700 1 --2- 172.21.15.89:0/2343593739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f006e940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41e40098f0 tx=0x7f41e4009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41f6e98700 1 -- 172.21.15.89:0/2343593739 shutdown_connections 2023-09-30T22:17:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41f6e98700 1 --2- 172.21.15.89:0/2343593739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f006e940 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41f6e98700 1 --2- 172.21.15.89:0/2343593739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f009eca0 0x7f41f006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41f6e98700 1 -- 172.21.15.89:0/2343593739 >> 172.21.15.89:0/2343593739 conn(0x7f41f00a2aa0 msgr2=0x7f41f00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41f6e98700 1 -- 172.21.15.89:0/2343593739 shutdown_connections 2023-09-30T22:17:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.413+0000 7f41f6e98700 1 -- 172.21.15.89:0/2343593739 wait complete. 2023-09-30T22:17:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.415+0000 7f41f6e98700 1 Processor -- start 2023-09-30T22:17:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.415+0000 7f41f6e98700 1 -- start start 2023-09-30T22:17:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f6e98700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f0123280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f6e98700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f0134610 0x7f41f01237c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f6e98700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41f0123d00 0x7f41f01cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f6e98700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f01267c0 con 0x7f41f006e570 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f6e98700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0134200 con 0x7f41f0123d00 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f6e98700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0126e60 con 0x7f41f0134610 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f4c34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f0123280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f4c34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f0123280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33312/0 (socket says 172.21.15.89:33312) 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f4c34700 1 -- 172.21.15.89:0/459809943 learned_addr learned my addr 172.21.15.89:0/459809943 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f5435700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41f0123d00 0x7f41f01cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41effff700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f0134610 0x7f41f01237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f5435700 1 -- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f0134610 msgr2=0x7f41f01237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f5435700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f0134610 0x7f41f01237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f5435700 1 -- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 msgr2=0x7f41f0123280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f5435700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f0123280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f5435700 1 -- 172.21.15.89:0/459809943 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0094a0 con 0x7f41f0123d00 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f4c34700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f0123280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.416+0000 7f41f5435700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41f0123d00 0x7f41f01cca00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f41e800cbc0 tx=0x7f41e800ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.417+0000 7f41ed7fa700 1 -- 172.21.15.89:0/459809943 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8003c00 con 0x7f41f0123d00 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.417+0000 7f41ed7fa700 1 -- 172.21.15.89:0/459809943 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e8003d60 con 0x7f41f0123d00 2023-09-30T22:17:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.417+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e4009550 con 0x7f41f0123d00 2023-09-30T22:17:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.417+0000 7f41ed7fa700 1 -- 172.21.15.89:0/459809943 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800aa40 con 0x7f41f0123d00 2023-09-30T22:17:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.417+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f0126440 con 0x7f41f0123d00 2023-09-30T22:17:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.418+0000 7f41ed7fa700 1 -- 172.21.15.89:0/459809943 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f41e800aba0 con 0x7f41f0123d00 2023-09-30T22:17:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.418+0000 7f41ed7fa700 1 -- 172.21.15.89:0/459809943 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f41e803fa20 con 0x7f41f0123d00 2023-09-30T22:17:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.418+0000 7f41f4c34700 1 -- 172.21.15.89:0/459809943 >> v1:172.21.15.89:6816/126953 conn(0x7f41d402f560 legacy=0x7f41d4031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.418+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f0074ea0 con 0x7f41f0123d00 2023-09-30T22:17:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.423+0000 7f41ed7fa700 1 -- 172.21.15.89:0/459809943 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41e8012420 con 0x7f41f0123d00 2023-09-30T22:17:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.452+0000 7f56a7124700 1 Processor -- start 2023-09-30T22:17:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.452+0000 7f56a7124700 1 -- start start 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a7124700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a006c0e0 0x7f56a009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a7124700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56a0134ec0 0x7f56a01379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a7124700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a01383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a7124700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a006c4b0 con 0x7f56a0134ec0 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a7124700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00a09c0 con 0x7f56a0137f20 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a7124700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56a0134980 con 0x7f56a006c0e0 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a56c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a01383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a56c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a01383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46138/0 (socket says 172.21.15.89:46138) 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a56c1700 1 -- 172.21.15.89:0/1615082753 learned_addr learned my addr 172.21.15.89:0/1615082753 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a4ec0700 1 --2- 172.21.15.89:0/1615082753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a006c0e0 0x7f56a009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a56c1700 1 -- 172.21.15.89:0/1615082753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a006c0e0 msgr2=0x7f56a009ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a56c1700 1 --2- 172.21.15.89:0/1615082753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a006c0e0 0x7f56a009ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a56c1700 1 -- 172.21.15.89:0/1615082753 >> v2:172.21.15.89:3300/0 conn(0x7f56a0134ec0 msgr2=0x7f56a01379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.453+0000 7f56a56c1700 1 --2- 172.21.15.89:0/1615082753 >> v2:172.21.15.89:3300/0 conn(0x7f56a0134ec0 0x7f56a01379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.454+0000 7f56a56c1700 1 -- 172.21.15.89:0/1615082753 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a00a0600 con 0x7f56a0137f20 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.454+0000 7f56a56c1700 1 --2- 172.21.15.89:0/1615082753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a01383c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5698009e50 tx=0x7f569800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=529b8582f647fc33 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.454+0000 7f569f7fe700 1 -- 172.21.15.89:0/1615082753 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800e070 con 0x7f56a0137f20 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.454+0000 7f569f7fe700 1 -- 172.21.15.89:0/1615082753 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569800b910 con 0x7f56a0137f20 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.454+0000 7f569f7fe700 1 -- 172.21.15.89:0/1615082753 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800bcc0 con 0x7f56a0137f20 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 -- 172.21.15.89:0/1615082753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 msgr2=0x7f56a01383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 --2- 172.21.15.89:0/1615082753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a01383c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5698009e50 tx=0x7f569800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 -- 172.21.15.89:0/1615082753 shutdown_connections 2023-09-30T22:17:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 --2- 172.21.15.89:0/1615082753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a01383c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 --2- 172.21.15.89:0/1615082753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a006c0e0 0x7f56a009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 --2- 172.21.15.89:0/1615082753 >> v2:172.21.15.89:3300/0 conn(0x7f56a0134ec0 0x7f56a01379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 -- 172.21.15.89:0/1615082753 >> 172.21.15.89:0/1615082753 conn(0x7f56a00a4210 msgr2=0x7f56a00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 -- 172.21.15.89:0/1615082753 shutdown_connections 2023-09-30T22:17:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.455+0000 7f56a7124700 1 -- 172.21.15.89:0/1615082753 wait complete. 2023-09-30T22:17:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a7124700 1 Processor -- start 2023-09-30T22:17:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a7124700 1 -- start start 2023-09-30T22:17:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a7124700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a006c0e0 0x7f56a011d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a7124700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a0134ec0 0x7f56a011a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a7124700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a011ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a7124700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a011c250 con 0x7f56a006c0e0 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a7124700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a011c880 con 0x7f56a0137f20 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a7124700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56a011d990 con 0x7f56a0134ec0 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a56c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a011ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a56c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a011ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46146/0 (socket says 172.21.15.89:46146) 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f56a56c1700 1 -- 172.21.15.89:0/947030895 learned_addr learned my addr 172.21.15.89:0/947030895 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.456+0000 7f569ffff700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a0134ec0 0x7f56a011a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a4ec0700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a006c0e0 0x7f56a011d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a56c1700 1 -- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a0134ec0 msgr2=0x7f56a011a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a56c1700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a0134ec0 0x7f56a011a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a56c1700 1 -- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a006c0e0 msgr2=0x7f56a011d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a56c1700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a006c0e0 0x7f56a011d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a56c1700 1 -- 172.21.15.89:0/947030895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5698009b30 con 0x7f56a0137f20 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a56c1700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a011ac10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f569800b680 tx=0x7f5698004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f569d7fa700 1 -- 172.21.15.89:0/947030895 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698004450 con 0x7f56a0137f20 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f569d7fa700 1 -- 172.21.15.89:0/947030895 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56980045f0 con 0x7f56a0137f20 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f569d7fa700 1 -- 172.21.15.89:0/947030895 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56980092e0 con 0x7f56a0137f20 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a013e910 con 0x7f56a0137f20 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.457+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a00a0440 con 0x7f56a0137f20 2023-09-30T22:17:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.458+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a0074ea0 con 0x7f56a0137f20 2023-09-30T22:17:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.459+0000 7f569d7fa700 1 -- 172.21.15.89:0/947030895 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f569801c410 con 0x7f56a0137f20 2023-09-30T22:17:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.459+0000 7f569d7fa700 1 -- 172.21.15.89:0/947030895 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f5698041930 con 0x7f56a0137f20 2023-09-30T22:17:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.459+0000 7f56a4ec0700 1 -- 172.21.15.89:0/947030895 >> v1:172.21.15.89:6816/126953 conn(0x7f568402f510 legacy=0x7f56840319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.463+0000 7f569d7fa700 1 -- 172.21.15.89:0/947030895 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5698023770 con 0x7f56a0137f20 2023-09-30T22:17:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.543+0000 7f720affd700 1 -- 172.21.15.89:0/3497202005 >> v1:172.21.15.89:6816/126953 conn(0x7f71f802f560 legacy=0x7f71f8031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.563+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f720c074ea0 con 0x7f720c09e410 2023-09-30T22:17:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.563+0000 7f71f3fff700 1 -- 172.21.15.89:0/3497202005 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7200012400 con 0x7f720c09e410 2023-09-30T22:17:03.566 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 >> v1:172.21.15.89:6816/126953 conn(0x7f71f802f560 legacy=0x7f71f8031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09e410 msgr2=0x7f720c122cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09e410 0x7f720c122cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f720000c4a0 tx=0x7f720000c7b0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 shutdown_connections 2023-09-30T22:17:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720c1231f0 0x7f720c1c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f720c09e000 0x7f720c122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 --2- 172.21.15.89:0/3497202005 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f720c09e410 0x7f720c122cb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 >> 172.21.15.89:0/3497202005 conn(0x7f720c066f50 msgr2=0x7f720c06f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 shutdown_connections 2023-09-30T22:17:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.566+0000 7f721180a700 1 -- 172.21.15.89:0/3497202005 wait complete. 2023-09-30T22:17:03.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:03.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.586+0000 7f92baffd700 1 -- 172.21.15.89:0/3487544383 >> v1:172.21.15.89:6816/126953 conn(0x7f92a802f560 legacy=0x7f92a8031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:03.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.591+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f92bc074ea0 con 0x7f92bc06a770 2023-09-30T22:17:03.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.592+0000 7f92a3fff700 1 -- 172.21.15.89:0/3487544383 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92ac01b030 con 0x7f92bc06a770 2023-09-30T22:17:03.593 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 >> v1:172.21.15.89:6816/126953 conn(0x7f92a802f560 legacy=0x7f92a8031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc06a770 msgr2=0x7f92bc11f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 --2- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc06a770 0x7f92bc11f1f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00bef0 tx=0x7f92ac00df80 comp rx=0 tx=0).stop 2023-09-30T22:17:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 shutdown_connections 2023-09-30T22:17:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 --2- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92bc131b40 0x7f92bc1c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 --2- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92bc06a770 0x7f92bc11f1f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 --2- 172.21.15.89:0/3487544383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92bc06b080 0x7f92bc11f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 >> 172.21.15.89:0/3487544383 conn(0x7f92bc066f50 msgr2=0x7f92bc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 shutdown_connections 2023-09-30T22:17:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.594+0000 7f92c1e2b700 1 -- 172.21.15.89:0/3487544383 wait complete. 2023-09-30T22:17:03.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:03.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.619+0000 7f41f4c34700 1 -- 172.21.15.89:0/459809943 >> v1:172.21.15.89:6816/126953 conn(0x7f41d402f560 legacy=0x7f41d4031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.621+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f41f0071100 con 0x7f41f0123d00 2023-09-30T22:17:03.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.622+0000 7f41ed7fa700 1 -- 172.21.15.89:0/459809943 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f41e801a360 con 0x7f41f0123d00 2023-09-30T22:17:03.622 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:03.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 >> v1:172.21.15.89:6816/126953 conn(0x7f41d402f560 legacy=0x7f41d4031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41f0123d00 msgr2=0x7f41f01cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41f0123d00 0x7f41f01cca00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f41e800cbc0 tx=0x7f41e800ced0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 shutdown_connections 2023-09-30T22:17:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41f0123d00 0x7f41f01cca00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41f006e570 0x7f41f0123280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 --2- 172.21.15.89:0/459809943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41f0134610 0x7f41f01237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 >> 172.21.15.89:0/459809943 conn(0x7f41f00a2aa0 msgr2=0x7f41f00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:03.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 shutdown_connections 2023-09-30T22:17:03.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.624+0000 7f41f6e98700 1 -- 172.21.15.89:0/459809943 wait complete. 2023-09-30T22:17:03.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.659+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f56a011dad0 con 0x7f56a0137f20 2023-09-30T22:17:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.659+0000 7f569d7fa700 1 -- 172.21.15.89:0/947030895 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f569802a090 con 0x7f56a0137f20 2023-09-30T22:17:03.661 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.660+0000 7f56a4ec0700 1 -- 172.21.15.89:0/947030895 >> v1:172.21.15.89:6816/126953 conn(0x7f568402f510 legacy=0x7f56840319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 >> v1:172.21.15.89:6816/126953 conn(0x7f568402f510 legacy=0x7f56840319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 msgr2=0x7f56a011ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a011ac10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f569800b680 tx=0x7f5698004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 shutdown_connections 2023-09-30T22:17:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0137f20 0x7f56a011ac10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a006c0e0 0x7f56a011d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 --2- 172.21.15.89:0/947030895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a0134ec0 0x7f56a011a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 >> 172.21.15.89:0/947030895 conn(0x7f56a00a4210 msgr2=0x7f56a0099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 shutdown_connections 2023-09-30T22:17:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:03.662+0000 7f56a7124700 1 -- 172.21.15.89:0/947030895 wait complete. 2023-09-30T22:17:03.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:04.580 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:04.608 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:04.638 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:04.675 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.720+0000 7f0e13ede700 1 Processor -- start 2023-09-30T22:17:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.721+0000 7f0e13ede700 1 -- start start 2023-09-30T22:17:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.721+0000 7f0e13ede700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.721+0000 7f0e13ede700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.721+0000 7f0e13ede700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0c099b50 con 0x7f0e0c12f320 2023-09-30T22:17:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.721+0000 7f0e13ede700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0667e0 con 0x7f0e0c06eb80 2023-09-30T22:17:04.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.721+0000 7f0e13ede700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c099f60 con 0x7f0e0c066410 2023-09-30T22:17:04.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11479700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11479700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46162/0 (socket says 172.21.15.89:46162) 2023-09-30T22:17:04.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11479700 1 -- 172.21.15.89:0/2432047316 learned_addr learned my addr 172.21.15.89:0/2432047316 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:04.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11c7a700 1 --2- 172.21.15.89:0/2432047316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e10c78700 1 -- 172.21.15.89:0/2432047316 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992487103 0 0) 0x7f0e0c099b50 con 0x7f0e0c12f320 2023-09-30T22:17:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e10c78700 1 -- 172.21.15.89:0/2432047316 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df80035b0 con 0x7f0e0c12f320 2023-09-30T22:17:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11479700 1 -- 172.21.15.89:0/2432047316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 msgr2=0x7f0e0c06e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11479700 1 --2- 172.21.15.89:0/2432047316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11479700 1 -- 172.21.15.89:0/2432047316 >> v1:172.21.15.89:6789/0 conn(0x7f0e0c12f320 legacy=0x7f0e0c12f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11479700 1 -- 172.21.15.89:0/2432047316 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c09a920 con 0x7f0e0c06eb80 2023-09-30T22:17:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e11479700 1 --2- 172.21.15.89:0/2432047316 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06ef70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009950 tx=0x7f0dfc009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6f9f7600dfd6bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.722+0000 7f0e10c78700 1 -- 172.21.15.89:0/2432047316 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc011830 con 0x7f0e0c06eb80 2023-09-30T22:17:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.723+0000 7f0e10c78700 1 -- 172.21.15.89:0/2432047316 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc011990 con 0x7f0e0c06eb80 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.723+0000 7f0e10c78700 1 -- 172.21.15.89:0/2432047316 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc011d40 con 0x7f0e0c06eb80 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.723+0000 7f0e13ede700 1 -- 172.21.15.89:0/2432047316 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 msgr2=0x7f0e0c06ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.723+0000 7f0e13ede700 1 --2- 172.21.15.89:0/2432047316 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06ef70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009950 tx=0x7f0dfc009c60 comp rx=0 tx=0).stop 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.724+0000 7f0e13ede700 1 -- 172.21.15.89:0/2432047316 shutdown_connections 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.724+0000 7f0e13ede700 1 --2- 172.21.15.89:0/2432047316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.724+0000 7f0e13ede700 1 --2- 172.21.15.89:0/2432047316 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06ef70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.724+0000 7f0e13ede700 1 -- 172.21.15.89:0/2432047316 >> 172.21.15.89:0/2432047316 conn(0x7f0e0c0a2aa0 msgr2=0x7f0e0c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.724+0000 7f0e13ede700 1 -- 172.21.15.89:0/2432047316 shutdown_connections 2023-09-30T22:17:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.724+0000 7f0e13ede700 1 -- 172.21.15.89:0/2432047316 wait complete. 2023-09-30T22:17:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.725+0000 7f0e13ede700 1 Processor -- start 2023-09-30T22:17:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.725+0000 7f0e13ede700 1 -- start start 2023-09-30T22:17:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.725+0000 7f0e13ede700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.725+0000 7f0e13ede700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.725+0000 7f0e13ede700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e0c12f320 0x7f0e0c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.725+0000 7f0e13ede700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c12eb70 con 0x7f0e0c12f320 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.725+0000 7f0e13ede700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c120d20 con 0x7f0e0c06eb80 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.725+0000 7f0e13ede700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c120240 con 0x7f0e0c066410 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11c7a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11c7a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42658/0 (socket says 172.21.15.89:42658) 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11c7a700 1 -- 172.21.15.89:0/3195643031 learned_addr learned my addr 172.21.15.89:0/3195643031 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11479700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e1247b700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e0c12f320 0x7f0e0c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11479700 1 -- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 msgr2=0x7f0e0c06f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11479700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11479700 1 -- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e0c12f320 msgr2=0x7f0e0c0700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11479700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e0c12f320 0x7f0e0c0700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11479700 1 -- 172.21.15.89:0/3195643031 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc009550 con 0x7f0e0c06eb80 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11c7a700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e11479700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06fbb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc016000 tx=0x7f0dfc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e027fc700 1 -- 172.21.15.89:0/3195643031 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc004240 con 0x7f0e0c06eb80 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e027fc700 1 -- 172.21.15.89:0/3195643031 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc01f070 con 0x7f0e0c06eb80 2023-09-30T22:17:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e027fc700 1 -- 172.21.15.89:0/3195643031 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc0045e0 con 0x7f0e0c06eb80 2023-09-30T22:17:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.726+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c120a50 con 0x7f0e0c06eb80 2023-09-30T22:17:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.727+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c120fe0 con 0x7f0e0c06eb80 2023-09-30T22:17:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.728+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c074ea0 con 0x7f0e0c06eb80 2023-09-30T22:17:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.728+0000 7f0e027fc700 1 -- 172.21.15.89:0/3195643031 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0dfc021030 con 0x7f0e0c06eb80 2023-09-30T22:17:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.728+0000 7f0e027fc700 1 -- 172.21.15.89:0/3195643031 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f0dfc040d30 con 0x7f0e0c06eb80 2023-09-30T22:17:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.728+0000 7f0e11c7a700 1 -- 172.21.15.89:0/3195643031 >> v1:172.21.15.89:6816/126953 conn(0x7f0df002f510 legacy=0x7f0df00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.737+0000 7f0e027fc700 1 -- 172.21.15.89:0/3195643031 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0dfc0197c0 con 0x7f0e0c06eb80 2023-09-30T22:17:04.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.766+0000 7f72321d8700 1 Processor -- start 2023-09-30T22:17:04.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.766+0000 7f72321d8700 1 -- start start 2023-09-30T22:17:04.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f72321d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f72321d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f722c06fcf0 0x7f722c138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f72321d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c0716a0 con 0x7f722c130e50 2023-09-30T22:17:04.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f72321d8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c09a390 con 0x7f722c06fcf0 2023-09-30T22:17:04.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f72321d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c071290 con 0x7f722c130540 2023-09-30T22:17:04.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f722b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f722b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46186/0 (socket says 172.21.15.89:46186) 2023-09-30T22:17:04.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f722b7fe700 1 -- 172.21.15.89:0/3961961902 learned_addr learned my addr 172.21.15.89:0/3961961902 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:04.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f722bfff700 1 --2- 172.21.15.89:0/3961961902 >> v2:172.21.15.89:3300/0 conn(0x7f722c06fcf0 0x7f722c138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f722b7fe700 1 -- 172.21.15.89:0/3961961902 >> v2:172.21.15.89:3300/0 conn(0x7f722c06fcf0 msgr2=0x7f722c138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:04.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f722b7fe700 1 --2- 172.21.15.89:0/3961961902 >> v2:172.21.15.89:3300/0 conn(0x7f722c06fcf0 0x7f722c138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f722b7fe700 1 -- 172.21.15.89:0/3961961902 >> v1:172.21.15.89:6789/0 conn(0x7f722c130e50 legacy=0x7f722c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.767+0000 7f722a7fc700 1 -- 172.21.15.89:0/3961961902 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072633577 0 0) 0x7f722c0716a0 con 0x7f722c130e50 2023-09-30T22:17:04.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.768+0000 7f722b7fe700 1 -- 172.21.15.89:0/3961961902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c130340 con 0x7f722c130540 2023-09-30T22:17:04.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.768+0000 7f722bfff700 1 --2- 172.21.15.89:0/3961961902 >> v2:172.21.15.89:3300/0 conn(0x7f722c06fcf0 0x7f722c138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:17:04.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.768+0000 7f722b7fe700 1 --2- 172.21.15.89:0/3961961902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c130910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f721c009920 tx=0x7f721c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4c3437ccc3430f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:04.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.768+0000 7f722a7fc700 1 -- 172.21.15.89:0/3961961902 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c00ba20 con 0x7f722c130540 2023-09-30T22:17:04.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.768+0000 7f722a7fc700 1 -- 172.21.15.89:0/3961961902 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c00bb80 con 0x7f722c130540 2023-09-30T22:17:04.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.768+0000 7f722a7fc700 1 -- 172.21.15.89:0/3961961902 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c009e70 con 0x7f722c130540 2023-09-30T22:17:04.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.769+0000 7f72321d8700 1 -- 172.21.15.89:0/3961961902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 msgr2=0x7f722c130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.769+0000 7f72321d8700 1 --2- 172.21.15.89:0/3961961902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c130910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f721c009920 tx=0x7f721c009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:04.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.769+0000 7f72321d8700 1 -- 172.21.15.89:0/3961961902 shutdown_connections 2023-09-30T22:17:04.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.769+0000 7f72321d8700 1 --2- 172.21.15.89:0/3961961902 >> v2:172.21.15.89:3300/0 conn(0x7f722c06fcf0 0x7f722c138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.769+0000 7f72321d8700 1 --2- 172.21.15.89:0/3961961902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c130910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.769+0000 7f72321d8700 1 -- 172.21.15.89:0/3961961902 >> 172.21.15.89:0/3961961902 conn(0x7f722c0a2aa0 msgr2=0x7f722c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.769+0000 7f72321d8700 1 -- 172.21.15.89:0/3961961902 shutdown_connections 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.769+0000 7f72321d8700 1 -- 172.21.15.89:0/3961961902 wait complete. 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.770+0000 7f72321d8700 1 Processor -- start 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f72321d8700 1 -- start start 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f72321d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f722c06fcf0 0x7f722c12ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f72321d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c12b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f72321d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c130e50 0x7f722c1c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f72321d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c139250 con 0x7f722c130e50 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f72321d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0a0580 con 0x7f722c130540 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f72321d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f722c12a860 con 0x7f722c06fcf0 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f722affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c12b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f722affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c12b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46200/0 (socket says 172.21.15.89:46200) 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f722affd700 1 -- 172.21.15.89:0/324435024 learned_addr learned my addr 172.21.15.89:0/324435024 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f722b7fe700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f722c06fcf0 0x7f722c12ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.771+0000 7f722bfff700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c130e50 0x7f722c1c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f722affd700 1 -- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f722c06fcf0 msgr2=0x7f722c12ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f722affd700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f722c06fcf0 0x7f722c12ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f722affd700 1 -- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c130e50 msgr2=0x7f722c1c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f722affd700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c130e50 0x7f722c1c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f722affd700 1 -- 172.21.15.89:0/324435024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009550 con 0x7f722c130540 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f722b7fe700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f722c06fcf0 0x7f722c12ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f722bfff700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c130e50 0x7f722c1c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f722affd700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c12b190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f722000c520 tx=0x7f722000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f7213fff700 1 -- 172.21.15.89:0/324435024 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72200054e0 con 0x7f722c130540 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c12dbc0 con 0x7f722c130540 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c12bff0 con 0x7f722c130540 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.772+0000 7f7213fff700 1 -- 172.21.15.89:0/324435024 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7220003680 con 0x7f722c130540 2023-09-30T22:17:04.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.773+0000 7f7213fff700 1 -- 172.21.15.89:0/324435024 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220003a30 con 0x7f722c130540 2023-09-30T22:17:04.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.773+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c0052e0 con 0x7f722c130540 2023-09-30T22:17:04.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.774+0000 7f7213fff700 1 -- 172.21.15.89:0/324435024 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7220003b90 con 0x7f722c130540 2023-09-30T22:17:04.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.774+0000 7f7213fff700 1 -- 172.21.15.89:0/324435024 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f722003ef10 con 0x7f722c130540 2023-09-30T22:17:04.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.774+0000 7f722b7fe700 1 -- 172.21.15.89:0/324435024 >> v1:172.21.15.89:6816/126953 conn(0x7f721802f540 legacy=0x7f72180319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:04.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.778+0000 7f7213fff700 1 -- 172.21.15.89:0/324435024 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f722001b3b0 con 0x7f722c130540 2023-09-30T22:17:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.800+0000 7fe7292f9700 1 Processor -- start 2023-09-30T22:17:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.800+0000 7fe7292f9700 1 -- start start 2023-09-30T22:17:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe7292f9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe724131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe7292f9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe724139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe7292f9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe724071a00 con 0x7fe724131f40 2023-09-30T22:17:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe7292f9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe724131430 con 0x7fe72406eba0 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe7292f9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe72409a510 con 0x7fe724131630 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe722ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe724131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe722ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe724131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42684/0 (socket says 172.21.15.89:42684) 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe7227fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe724131f40 0x7fe72406e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59930/0 (socket says 172.21.15.89:59930) 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe722ffd700 1 -- 172.21.15.89:0/1748913760 learned_addr learned my addr 172.21.15.89:0/1748913760 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.801+0000 7fe7237fe700 1 --2- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe724139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe722ffd700 1 -- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 msgr2=0x7fe724139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe722ffd700 1 --2- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe724139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe722ffd700 1 -- 172.21.15.89:0/1748913760 >> v1:172.21.15.89:6789/0 conn(0x7fe724131f40 legacy=0x7fe72406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe722ffd700 1 -- 172.21.15.89:0/1748913760 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240721c0 con 0x7fe724131630 2023-09-30T22:17:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe7237fe700 1 --2- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe724139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe722ffd700 1 --2- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe724131a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe714009920 tx=0x7fe714009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8706f1c395f32a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe721ffb700 1 -- 172.21.15.89:0/1748913760 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714011830 con 0x7fe724131630 2023-09-30T22:17:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe721ffb700 1 -- 172.21.15.89:0/1748913760 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe714011990 con 0x7fe724131630 2023-09-30T22:17:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.802+0000 7fe721ffb700 1 -- 172.21.15.89:0/1748913760 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714011d20 con 0x7fe724131630 2023-09-30T22:17:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.803+0000 7fe7292f9700 1 -- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 msgr2=0x7fe724131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.803+0000 7fe7292f9700 1 --2- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe724131a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe714009920 tx=0x7fe714009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.803+0000 7fe7292f9700 1 -- 172.21.15.89:0/1748913760 shutdown_connections 2023-09-30T22:17:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.803+0000 7fe7292f9700 1 --2- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe724139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.803+0000 7fe7292f9700 1 --2- 172.21.15.89:0/1748913760 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe724131a00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.803+0000 7fe7292f9700 1 -- 172.21.15.89:0/1748913760 >> 172.21.15.89:0/1748913760 conn(0x7fe7240a2aa0 msgr2=0x7fe7240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.803+0000 7fe7292f9700 1 -- 172.21.15.89:0/1748913760 shutdown_connections 2023-09-30T22:17:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.803+0000 7fe7292f9700 1 -- 172.21.15.89:0/1748913760 wait complete. 2023-09-30T22:17:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.804+0000 7fe7292f9700 1 Processor -- start 2023-09-30T22:17:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.804+0000 7fe7292f9700 1 -- start start 2023-09-30T22:17:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe7292f9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe72411b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe7292f9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe72411bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe7292f9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe724131f40 0x7fe72411c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe7292f9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72411f420 con 0x7fe724131f40 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe7292f9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72409f960 con 0x7fe72406eba0 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe7292f9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe724099f20 con 0x7fe724131630 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe722ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe72411b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe722ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe72411b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46222/0 (socket says 172.21.15.89:46222) 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe722ffd700 1 -- 172.21.15.89:0/1154599966 learned_addr learned my addr 172.21.15.89:0/1154599966 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe7237fe700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe724131f40 0x7fe72411c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe7227fc700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe72411bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.805+0000 7fe722ffd700 1 -- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 msgr2=0x7fe72411bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe722ffd700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe72411bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe722ffd700 1 -- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe724131f40 msgr2=0x7fe72411c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe722ffd700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe724131f40 0x7fe72411c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe722ffd700 1 -- 172.21.15.89:0/1154599966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c009a70 con 0x7fe72406eba0 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe722ffd700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe72411b990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe714000c00 tx=0x7fe714004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe70b7fe700 1 -- 172.21.15.89:0/1154599966 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714004450 con 0x7fe72406eba0 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe70b7fe700 1 -- 172.21.15.89:0/1154599966 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7140045b0 con 0x7fe72406eba0 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe714009550 con 0x7fe72406eba0 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe70b7fe700 1 -- 172.21.15.89:0/1154599966 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7140038a0 con 0x7fe72406eba0 2023-09-30T22:17:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.806+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72411ebb0 con 0x7fe72406eba0 2023-09-30T22:17:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.807+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe724074ea0 con 0x7fe72406eba0 2023-09-30T22:17:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.808+0000 7fe70b7fe700 1 -- 172.21.15.89:0/1154599966 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe714003a00 con 0x7fe72406eba0 2023-09-30T22:17:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.808+0000 7fe70b7fe700 1 -- 172.21.15.89:0/1154599966 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fe714021030 con 0x7fe72406eba0 2023-09-30T22:17:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.808+0000 7fe7227fc700 1 -- 172.21.15.89:0/1154599966 >> v1:172.21.15.89:6816/126953 conn(0x7fe71002f510 legacy=0x7fe7100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:04.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.812+0000 7fe70b7fe700 1 -- 172.21.15.89:0/1154599966 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe714003cb0 con 0x7fe72406eba0 2023-09-30T22:17:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.852+0000 7f2a58da2700 1 Processor -- start 2023-09-30T22:17:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.852+0000 7f2a58da2700 1 -- start start 2023-09-30T22:17:04.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.852+0000 7f2a58da2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a540a0870 0x7f2a540a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.852+0000 7f2a58da2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a54133ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.852+0000 7f2a58da2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5409db90 con 0x7f2a540a1180 2023-09-30T22:17:04.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.852+0000 7f2a58da2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a540a1ce0 con 0x7f2a540a0870 2023-09-30T22:17:04.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.852+0000 7f2a58da2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540a24f0 con 0x7f2a540a15e0 2023-09-30T22:17:04.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a540a0870 0x7f2a540a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a540a0870 0x7f2a540a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33366/0 (socket says 172.21.15.89:33366) 2023-09-30T22:17:04.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a537fe700 1 -- 172.21.15.89:0/2437828839 learned_addr learned my addr 172.21.15.89:0/2437828839 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:04.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a53fff700 1 --2- 172.21.15.89:0/2437828839 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a54133ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a537fe700 1 -- 172.21.15.89:0/2437828839 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 msgr2=0x7f2a54133ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:04.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a537fe700 1 --2- 172.21.15.89:0/2437828839 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a54133ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a537fe700 1 -- 172.21.15.89:0/2437828839 >> v1:172.21.15.89:6789/0 conn(0x7f2a540a1180 legacy=0x7f2a54133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a537fe700 1 -- 172.21.15.89:0/2437828839 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540a1aa0 con 0x7f2a540a0870 2023-09-30T22:17:04.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a537fe700 1 --2- 172.21.15.89:0/2437828839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a0870 0x7f2a540a0c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2a440098f0 tx=0x7f2a44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2056af11b4a53407 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a527fc700 1 -- 172.21.15.89:0/2437828839 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44011830 con 0x7f2a540a0870 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a527fc700 1 -- 172.21.15.89:0/2437828839 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a44011990 con 0x7f2a540a0870 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.853+0000 7f2a527fc700 1 -- 172.21.15.89:0/2437828839 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44011d20 con 0x7f2a540a0870 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.854+0000 7f2a58da2700 1 -- 172.21.15.89:0/2437828839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a0870 msgr2=0x7f2a540a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.854+0000 7f2a58da2700 1 --2- 172.21.15.89:0/2437828839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a0870 0x7f2a540a0c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2a440098f0 tx=0x7f2a44009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.854+0000 7f2a58da2700 1 -- 172.21.15.89:0/2437828839 shutdown_connections 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.854+0000 7f2a58da2700 1 --2- 172.21.15.89:0/2437828839 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a54133ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.854+0000 7f2a58da2700 1 --2- 172.21.15.89:0/2437828839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a0870 0x7f2a540a0c40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.854+0000 7f2a58da2700 1 -- 172.21.15.89:0/2437828839 >> 172.21.15.89:0/2437828839 conn(0x7f2a540a6390 msgr2=0x7f2a540a6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.854+0000 7f2a58da2700 1 -- 172.21.15.89:0/2437828839 shutdown_connections 2023-09-30T22:17:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.854+0000 7f2a58da2700 1 -- 172.21.15.89:0/2437828839 wait complete. 2023-09-30T22:17:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.856+0000 7f2a58da2700 1 Processor -- start 2023-09-30T22:17:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.856+0000 7f2a58da2700 1 -- start start 2023-09-30T22:17:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a58da2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a0870 0x7f2a54121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a58da2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a1180 0x7f2a541224a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a58da2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a541229e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a58da2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54123b20 con 0x7f2a540a1180 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a58da2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a541249b0 con 0x7f2a540a15e0 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a58da2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54125490 con 0x7f2a540a0870 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a52ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a1180 0x7f2a541224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a53fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a541229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a52ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a1180 0x7f2a541224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33372/0 (socket says 172.21.15.89:33372) 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a52ffd700 1 -- 172.21.15.89:0/80058718 learned_addr learned my addr 172.21.15.89:0/80058718 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a53fff700 1 -- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a0870 msgr2=0x7f2a54121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a53fff700 1 --2- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a0870 0x7f2a54121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a53fff700 1 -- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a1180 msgr2=0x7f2a541224a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a53fff700 1 --2- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a1180 0x7f2a541224a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a53fff700 1 -- 172.21.15.89:0/80058718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44009550 con 0x7f2a540a15e0 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a52ffd700 1 --2- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a1180 0x7f2a541224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a53fff700 1 --2- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a541229e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009d90 tx=0x7f2a4c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.857+0000 7f2a3bfff700 1 -- 172.21.15.89:0/80058718 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c009c00 con 0x7f2a540a15e0 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.858+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a54125750 con 0x7f2a540a15e0 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.858+0000 7f2a3bfff700 1 -- 172.21.15.89:0/80058718 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c00f040 con 0x7f2a540a15e0 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.858+0000 7f2a3bfff700 1 -- 172.21.15.89:0/80058718 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c014480 con 0x7f2a540a15e0 2023-09-30T22:17:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.858+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5409c010 con 0x7f2a540a15e0 2023-09-30T22:17:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.858+0000 7f2a3bfff700 1 -- 172.21.15.89:0/80058718 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2a4c0145e0 con 0x7f2a540a15e0 2023-09-30T22:17:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.859+0000 7f2a537fe700 1 -- 172.21.15.89:0/80058718 >> v1:172.21.15.89:6816/126953 conn(0x7f2a4002f540 legacy=0x7f2a400319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.859+0000 7f2a3bfff700 1 -- 172.21.15.89:0/80058718 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f2a4c040920 con 0x7f2a540a15e0 2023-09-30T22:17:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.859+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a541252e0 con 0x7f2a540a15e0 2023-09-30T22:17:04.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.868+0000 7f2a3bfff700 1 -- 172.21.15.89:0/80058718 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2a4c018020 con 0x7f2a540a15e0 2023-09-30T22:17:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.929+0000 7f0e11c7a700 1 -- 172.21.15.89:0/3195643031 >> v1:172.21.15.89:6816/126953 conn(0x7f0df002f510 legacy=0x7f0df00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:04.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.963+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0e0c06b9e0 con 0x7f0e0c06eb80 2023-09-30T22:17:04.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.964+0000 7f0e027fc700 1 -- 172.21.15.89:0/3195643031 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0dfc00b420 con 0x7f0e0c06eb80 2023-09-30T22:17:04.964 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:04.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 >> v1:172.21.15.89:6816/126953 conn(0x7f0df002f510 legacy=0x7f0df00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 msgr2=0x7f0e0c06fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06fbb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc016000 tx=0x7f0dfc004750 comp rx=0 tx=0).stop 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 shutdown_connections 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e0c12f320 0x7f0e0c0700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e0c066410 0x7f0e0c06f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 --2- 172.21.15.89:0/3195643031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e0c06eb80 0x7f0e0c06fbb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 >> 172.21.15.89:0/3195643031 conn(0x7f0e0c0a2aa0 msgr2=0x7f0e0c1b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 shutdown_connections 2023-09-30T22:17:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.966+0000 7f0e13ede700 1 -- 172.21.15.89:0/3195643031 wait complete. 2023-09-30T22:17:04.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.975+0000 7f722b7fe700 1 -- 172.21.15.89:0/324435024 >> v1:172.21.15.89:6816/126953 conn(0x7f721802f540 legacy=0x7f72180319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:04.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:04.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.979+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f720c005860 con 0x7f722c130540 2023-09-30T22:17:04.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.979+0000 7f7213fff700 1 -- 172.21.15.89:0/324435024 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7220017030 con 0x7f722c130540 2023-09-30T22:17:04.979 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:04.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.982+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 >> v1:172.21.15.89:6816/126953 conn(0x7f721802f540 legacy=0x7f72180319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:04.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.982+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 msgr2=0x7f722c12b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:04.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.982+0000 7f72321d8700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c12b190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f722000c520 tx=0x7f722000c830 comp rx=0 tx=0).stop 2023-09-30T22:17:04.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.983+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 shutdown_connections 2023-09-30T22:17:04.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.983+0000 7f72321d8700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c130e50 0x7f722c1c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.983+0000 7f72321d8700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f722c06fcf0 0x7f722c12ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.983+0000 7f72321d8700 1 --2- 172.21.15.89:0/324435024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f722c130540 0x7f722c12b190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:04.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.983+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 >> 172.21.15.89:0/324435024 conn(0x7f722c0a2aa0 msgr2=0x7f722c06e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:04.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.983+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 shutdown_connections 2023-09-30T22:17:04.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:04.983+0000 7f72321d8700 1 -- 172.21.15.89:0/324435024 wait complete. 2023-09-30T22:17:04.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:05.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.009+0000 7fe7227fc700 1 -- 172.21.15.89:0/1154599966 >> v1:172.21.15.89:6816/126953 conn(0x7fe71002f510 legacy=0x7fe7100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:05.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.009+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe72411da60 con 0x7fe72406eba0 2023-09-30T22:17:05.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.009+0000 7fe70b7fe700 1 -- 172.21.15.89:0/1154599966 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe71402b020 con 0x7fe72406eba0 2023-09-30T22:17:05.009 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:05.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 >> v1:172.21.15.89:6816/126953 conn(0x7fe71002f510 legacy=0x7fe7100319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:05.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 msgr2=0x7fe72411b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:05.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe72411b990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe714000c00 tx=0x7fe714004750 comp rx=0 tx=0).stop 2023-09-30T22:17:05.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 shutdown_connections 2023-09-30T22:17:05.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe724131f40 0x7fe72411c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe72406eba0 0x7fe72411b990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 --2- 172.21.15.89:0/1154599966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe724131630 0x7fe72411bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 >> 172.21.15.89:0/1154599966 conn(0x7fe7240a2aa0 msgr2=0x7fe7240a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 shutdown_connections 2023-09-30T22:17:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.011+0000 7fe7292f9700 1 -- 172.21.15.89:0/1154599966 wait complete. 2023-09-30T22:17:05.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:05.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.060+0000 7f2a537fe700 1 -- 172.21.15.89:0/80058718 >> v1:172.21.15.89:6816/126953 conn(0x7f2a4002f540 legacy=0x7f2a400319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:05.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.075+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2a5411edd0 con 0x7f2a540a15e0 2023-09-30T22:17:05.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.076+0000 7f2a3bfff700 1 -- 172.21.15.89:0/80058718 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a4c02b090 con 0x7f2a540a15e0 2023-09-30T22:17:05.076 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 >> v1:172.21.15.89:6816/126953 conn(0x7f2a4002f540 legacy=0x7f2a400319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 msgr2=0x7f2a541229e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 --2- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a541229e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009d90 tx=0x7f2a4c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:17:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 shutdown_connections 2023-09-30T22:17:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 --2- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a15e0 0x7f2a541229e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:05.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 --2- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a0870 0x7f2a54121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 --2- 172.21.15.89:0/80058718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a540a1180 0x7f2a541224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 >> 172.21.15.89:0/80058718 conn(0x7f2a540a6390 msgr2=0x7f2a541339a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 shutdown_connections 2023-09-30T22:17:05.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:05.078+0000 7f2a58da2700 1 -- 172.21.15.89:0/80058718 wait complete. 2023-09-30T22:17:05.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:05.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:05.995 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:06.023 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:06.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:06.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.118+0000 7f92d4ef4700 1 Processor -- start 2023-09-30T22:17:06.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.118+0000 7f92d4ef4700 1 -- start start 2023-09-30T22:17:06.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92d4ef4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92d009e930 0x7f92d009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92d4ef4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009f240 0x7f92d01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92d4ef4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d0070940 con 0x7f92d009f650 2023-09-30T22:17:06.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92d4ef4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d006fd60 con 0x7f92d009f240 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92d4ef4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92d006fa80 con 0x7f92d009e930 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92ced9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f92d009f650 0x7f92d0131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59960/0 (socket says 172.21.15.89:59960) 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92ced9d700 1 -- 172.21.15.89:0/1590039478 learned_addr learned my addr 172.21.15.89:0/1590039478 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92cdd9b700 1 --2- 172.21.15.89:0/1590039478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009f240 0x7f92d01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92cd59a700 1 -- 172.21.15.89:0/1590039478 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232364973 0 0) 0x7f92d0070940 con 0x7f92d009f650 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92cd59a700 1 -- 172.21.15.89:0/1590039478 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92b80035b0 con 0x7f92d009f650 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92cdd9b700 1 -- 172.21.15.89:0/1590039478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92d009e930 msgr2=0x7f92d009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92cdd9b700 1 --2- 172.21.15.89:0/1590039478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92d009e930 0x7f92d009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.119+0000 7f92cdd9b700 1 -- 172.21.15.89:0/1590039478 >> v1:172.21.15.89:6789/0 conn(0x7f92d009f650 legacy=0x7f92d0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.120+0000 7f92cdd9b700 1 -- 172.21.15.89:0/1590039478 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d009fb60 con 0x7f92d009f240 2023-09-30T22:17:06.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.120+0000 7f92cdd9b700 1 --2- 172.21.15.89:0/1590039478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009f240 0x7f92d01317f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009870 tx=0x7f92c4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d926100e3f3b7eec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:06.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.120+0000 7f92cd59a700 1 -- 172.21.15.89:0/1590039478 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4011830 con 0x7f92d009f240 2023-09-30T22:17:06.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.120+0000 7f92cd59a700 1 -- 172.21.15.89:0/1590039478 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4011990 con 0x7f92d009f240 2023-09-30T22:17:06.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.120+0000 7f92cd59a700 1 -- 172.21.15.89:0/1590039478 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4011d20 con 0x7f92d009f240 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.121+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1590039478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009f240 msgr2=0x7f92d01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.121+0000 7f92d4ef4700 1 --2- 172.21.15.89:0/1590039478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009f240 0x7f92d01317f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009870 tx=0x7f92c4009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.121+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1590039478 shutdown_connections 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.121+0000 7f92d4ef4700 1 --2- 172.21.15.89:0/1590039478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92d009e930 0x7f92d009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.121+0000 7f92d4ef4700 1 --2- 172.21.15.89:0/1590039478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009f240 0x7f92d01317f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.121+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1590039478 >> 172.21.15.89:0/1590039478 conn(0x7f92d00a4210 msgr2=0x7f92d00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.121+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1590039478 shutdown_connections 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.121+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1590039478 wait complete. 2023-09-30T22:17:06.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.123+0000 7f92d4ef4700 1 Processor -- start 2023-09-30T22:17:06.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.123+0000 7f92d4ef4700 1 -- start start 2023-09-30T22:17:06.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.123+0000 7f92d4ef4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009e930 0x7f92d006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.123+0000 7f92d4ef4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92d009f240 0x7f92d006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.123+0000 7f92d4ef4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d009f650 0x7f92d006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.123+0000 7f92d4ef4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d01b7220 con 0x7f92d009f650 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.123+0000 7f92d4ef4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d01a93d0 con 0x7f92d009e930 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.123+0000 7f92d4ef4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92d01a9730 con 0x7f92d009f240 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d009f650 0x7f92d006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d009f650 0x7f92d006c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33376/0 (socket says 172.21.15.89:33376) 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 -- 172.21.15.89:0/1166900781 learned_addr learned my addr 172.21.15.89:0/1166900781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:06.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 -- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92d009f240 msgr2=0x7f92d006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 --2- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92d009f240 0x7f92d006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 -- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009e930 msgr2=0x7f92d006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 --2- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009e930 0x7f92d006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 -- 172.21.15.89:0/1166900781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40094a0 con 0x7f92d009f650 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92ced9d700 1 --2- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d009f650 0x7f92d006c300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f92c800cbb0 tx=0x7f92c800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92beffd700 1 -- 172.21.15.89:0/1166900781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c800aaf0 con 0x7f92d009f650 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92beffd700 1 -- 172.21.15.89:0/1166900781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c800ac50 con 0x7f92d009f650 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92beffd700 1 -- 172.21.15.89:0/1166900781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8012400 con 0x7f92d009f650 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.124+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d00a0190 con 0x7f92d009f650 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.125+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d00a3510 con 0x7f92d009f650 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.126+0000 7f92beffd700 1 -- 172.21.15.89:0/1166900781 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f92c8012560 con 0x7f92d009f650 2023-09-30T22:17:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.126+0000 7f92beffd700 1 -- 172.21.15.89:0/1166900781 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f92c803f8a0 con 0x7f92d009f650 2023-09-30T22:17:06.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.126+0000 7f92ce59c700 1 -- 172.21.15.89:0/1166900781 >> v1:172.21.15.89:6816/126953 conn(0x7f92b002f510 legacy=0x7f92b00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:06.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.130+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d01a0b30 con 0x7f92d009f650 2023-09-30T22:17:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.139+0000 7f92beffd700 1 -- 172.21.15.89:0/1166900781 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92c8017080 con 0x7f92d009f650 2023-09-30T22:17:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.162+0000 7f1fea9c4700 1 Processor -- start 2023-09-30T22:17:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fea9c4700 1 -- start start 2023-09-30T22:17:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fea9c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe40a0a30 0x7f1fe40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fea9c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1fe409d630 0x7f1fe40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fea9c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe4133930 0x7f1fe4133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fea9c4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe40a2a70 con 0x7f1fe409d630 2023-09-30T22:17:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fea9c4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40a01d0 con 0x7f1fe4133930 2023-09-30T22:17:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fea9c4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe409e2b0 con 0x7f1fe40a0a30 2023-09-30T22:17:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fe37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1fe409d630 0x7f1fe40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe40a0a30 0x7f1fe40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fe8f61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe4133930 0x7f1fe4133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fe37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1fe409d630 0x7f1fe40a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33392/0 (socket says 172.21.15.89:33392) 2023-09-30T22:17:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe40a0a30 0x7f1fe40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42718/0 (socket says 172.21.15.89:42718) 2023-09-30T22:17:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.163+0000 7f1fe37fe700 1 -- 172.21.15.89:0/3221120359 learned_addr learned my addr 172.21.15.89:0/3221120359 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe3fff700 1 -- 172.21.15.89:0/3221120359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe4133930 msgr2=0x7f1fe4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe3fff700 1 --2- 172.21.15.89:0/3221120359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe4133930 0x7f1fe4133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe3fff700 1 -- 172.21.15.89:0/3221120359 >> v2:172.21.15.89:3300/0 conn(0x7f1fe409d630 msgr2=0x7f1fe40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe3fff700 1 --2- 172.21.15.89:0/3221120359 >> v2:172.21.15.89:3300/0 conn(0x7f1fe409d630 0x7f1fe40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe3fff700 1 -- 172.21.15.89:0/3221120359 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe409dc50 con 0x7f1fe40a0a30 2023-09-30T22:17:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe37fe700 1 --2- 172.21.15.89:0/3221120359 >> v2:172.21.15.89:3300/0 conn(0x7f1fe409d630 0x7f1fe40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f36ff886700 1 Processor -- start 2023-09-30T22:17:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f36ff886700 1 -- start start 2023-09-30T22:17:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe3fff700 1 --2- 172.21.15.89:0/3221120359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe40a0a30 0x7f1fe40a0e00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009840 tx=0x7f1fd4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6da356bb469a7bc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe2ffd700 1 -- 172.21.15.89:0/3221120359 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4011830 con 0x7f1fe40a0a30 2023-09-30T22:17:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe2ffd700 1 -- 172.21.15.89:0/3221120359 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd4011990 con 0x7f1fe40a0a30 2023-09-30T22:17:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.164+0000 7f1fe2ffd700 1 -- 172.21.15.89:0/3221120359 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4011d20 con 0x7f1fe40a0a30 2023-09-30T22:17:06.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36ff886700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36f8071b40 0x7f36f8130420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36ff886700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8130960 0x7f36f8133570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36ff886700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130d50 0x7f36f8133ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36ff886700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f8099ef0 con 0x7f36f8071b40 2023-09-30T22:17:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36ff886700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f8099500 con 0x7f36f8130960 2023-09-30T22:17:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36ff886700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36f8099c10 con 0x7f36f8130d50 2023-09-30T22:17:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fce21700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8130960 0x7f36f8133570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fce21700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8130960 0x7f36f8133570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46256/0 (socket says 172.21.15.89:46256) 2023-09-30T22:17:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fce21700 1 -- 172.21.15.89:0/4034737334 learned_addr learned my addr 172.21.15.89:0/4034737334 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 -- 172.21.15.89:0/3221120359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe40a0a30 msgr2=0x7f1fe40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 --2- 172.21.15.89:0/3221120359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe40a0a30 0x7f1fe40a0e00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009840 tx=0x7f1fd4009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fde23700 1 --2- 172.21.15.89:0/4034737334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130d50 0x7f36f8133ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 -- 172.21.15.89:0/3221120359 shutdown_connections 2023-09-30T22:17:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 --2- 172.21.15.89:0/3221120359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe4133930 0x7f1fe4133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 --2- 172.21.15.89:0/3221120359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe40a0a30 0x7f1fe40a0e00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 --2- 172.21.15.89:0/3221120359 >> v2:172.21.15.89:3300/0 conn(0x7f1fe409d630 0x7f1fe40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 -- 172.21.15.89:0/3221120359 >> 172.21.15.89:0/3221120359 conn(0x7f1fe40a3050 msgr2=0x7f1fe40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 -- 172.21.15.89:0/3221120359 shutdown_connections 2023-09-30T22:17:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f1fea9c4700 1 -- 172.21.15.89:0/3221120359 wait complete. 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fce21700 1 -- 172.21.15.89:0/4034737334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130d50 msgr2=0x7f36f8133ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fce21700 1 --2- 172.21.15.89:0/4034737334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130d50 0x7f36f8133ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fce21700 1 -- 172.21.15.89:0/4034737334 >> v2:172.21.15.89:3300/0 conn(0x7f36f8071b40 msgr2=0x7f36f8130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fce21700 1 --2- 172.21.15.89:0/4034737334 >> v2:172.21.15.89:3300/0 conn(0x7f36f8071b40 0x7f36f8130420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.165+0000 7f36fce21700 1 -- 172.21.15.89:0/4034737334 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f809e370 con 0x7f36f8130960 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.166+0000 7f36fce21700 1 --2- 172.21.15.89:0/4034737334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8130960 0x7f36f8133570 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36e80098f0 tx=0x7f36e8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df63ca1d2348faa9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.166+0000 7f36effff700 1 -- 172.21.15.89:0/4034737334 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e800b8a0 con 0x7f36f8130960 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.166+0000 7f36effff700 1 -- 172.21.15.89:0/4034737334 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e800ba00 con 0x7f36f8130960 2023-09-30T22:17:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.166+0000 7f36effff700 1 -- 172.21.15.89:0/4034737334 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e800bd90 con 0x7f36f8130960 2023-09-30T22:17:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f1fea9c4700 1 Processor -- start 2023-09-30T22:17:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 -- 172.21.15.89:0/4034737334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8130960 msgr2=0x7f36f8133570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 --2- 172.21.15.89:0/4034737334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8130960 0x7f36f8133570 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36e80098f0 tx=0x7f36e8009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 -- 172.21.15.89:0/4034737334 shutdown_connections 2023-09-30T22:17:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 --2- 172.21.15.89:0/4034737334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130d50 0x7f36f8133ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 --2- 172.21.15.89:0/4034737334 >> v2:172.21.15.89:3300/0 conn(0x7f36f8071b40 0x7f36f8130420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 --2- 172.21.15.89:0/4034737334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8130960 0x7f36f8133570 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 -- 172.21.15.89:0/4034737334 >> 172.21.15.89:0/4034737334 conn(0x7f36f809e7e0 msgr2=0x7f36f809ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 -- 172.21.15.89:0/4034737334 shutdown_connections 2023-09-30T22:17:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f1fea9c4700 1 -- start start 2023-09-30T22:17:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f1fea9c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe409d630 0x7f1fe406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f1fea9c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fe40a0a30 0x7f1fe406e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f1fea9c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe4133930 0x7f1fe41cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fea9c4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe406fee0 con 0x7f1fe40a0a30 2023-09-30T22:17:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fea9c4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4071a20 con 0x7f1fe409d630 2023-09-30T22:17:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fea9c4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe406ee90 con 0x7f1fe4133930 2023-09-30T22:17:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe409d630 0x7f1fe406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.167+0000 7f36ff886700 1 -- 172.21.15.89:0/4034737334 wait complete. 2023-09-30T22:17:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe409d630 0x7f1fe406dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46260/0 (socket says 172.21.15.89:46260) 2023-09-30T22:17:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe3fff700 1 -- 172.21.15.89:0/4141827929 learned_addr learned my addr 172.21.15.89:0/4141827929 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f36ff886700 1 Processor -- start 2023-09-30T22:17:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe8f61700 1 --2- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe4133930 0x7f1fe41cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe8f61700 1 -- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe409d630 msgr2=0x7f1fe406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe8f61700 1 --2- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe409d630 0x7f1fe406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe8f61700 1 -- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fe40a0a30 msgr2=0x7f1fe406e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe8f61700 1 --2- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fe40a0a30 0x7f1fe406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe8f61700 1 -- 172.21.15.89:0/4141827929 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8009550 con 0x7f1fe4133930 2023-09-30T22:17:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f1fe8f61700 1 --2- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe4133930 0x7f1fe41cc3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00efc0 tx=0x7f1fdc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f1fe0ff9700 1 -- 172.21.15.89:0/4141827929 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00cd40 con 0x7f1fe4133930 2023-09-30T22:17:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f1fe0ff9700 1 -- 172.21.15.89:0/4141827929 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc00f040 con 0x7f1fe4133930 2023-09-30T22:17:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f1fe0ff9700 1 -- 172.21.15.89:0/4141827929 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0146c0 con 0x7f1fe4133930 2023-09-30T22:17:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.168+0000 7f36ff886700 1 -- start start 2023-09-30T22:17:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36ff886700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8071b40 0x7f36f8098910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36ff886700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130960 0x7f36f80972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36ff886700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f8130d50 0x7f36f80977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36ff886700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f812a510 con 0x7f36f8130d50 2023-09-30T22:17:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36ff886700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f812b860 con 0x7f36f8071b40 2023-09-30T22:17:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36ff886700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36f812bb20 con 0x7f36f8130960 2023-09-30T22:17:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd40094a0 con 0x7f1fe4133930 2023-09-30T22:17:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe4070280 con 0x7f1fe4133930 2023-09-30T22:17:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36fce21700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130960 0x7f36f80972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36fce21700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130960 0x7f36f80972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42738/0 (socket says 172.21.15.89:42738) 2023-09-30T22:17:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36fce21700 1 -- 172.21.15.89:0/444482096 learned_addr learned my addr 172.21.15.89:0/444482096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36fce21700 1 -- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8071b40 msgr2=0x7f36f8098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36fce21700 1 --2- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8071b40 0x7f36f8098910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36fce21700 1 -- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f8130d50 msgr2=0x7f36f80977e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36fce21700 1 --2- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f8130d50 0x7f36f80977e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.169+0000 7f36fce21700 1 -- 172.21.15.89:0/444482096 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8009550 con 0x7f36f8130960 2023-09-30T22:17:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.170+0000 7f1fe0ff9700 1 -- 172.21.15.89:0/4141827929 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1fdc014820 con 0x7f1fe4133930 2023-09-30T22:17:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.170+0000 7f36fce21700 1 --2- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130960 0x7f36f80972a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36e8000c00 tx=0x7f36e8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.170+0000 7f1fe3fff700 1 -- 172.21.15.89:0/4141827929 >> v1:172.21.15.89:6816/126953 conn(0x7f1fd002f510 legacy=0x7f1fd00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.170+0000 7f1fe0ff9700 1 -- 172.21.15.89:0/4141827929 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f1fdc0408b0 con 0x7f1fe4133930 2023-09-30T22:17:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.170+0000 7f36edffb700 1 -- 172.21.15.89:0/444482096 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8004400 con 0x7f36f8130960 2023-09-30T22:17:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.170+0000 7f36edffb700 1 -- 172.21.15.89:0/444482096 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e801f070 con 0x7f36f8130960 2023-09-30T22:17:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.170+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f812a070 con 0x7f36f8130960 2023-09-30T22:17:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.170+0000 7f36edffb700 1 -- 172.21.15.89:0/444482096 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e80038c0 con 0x7f36f8130960 2023-09-30T22:17:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.171+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f812ade0 con 0x7f36f8130960 2023-09-30T22:17:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.172+0000 7f36edffb700 1 -- 172.21.15.89:0/444482096 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f36e8021030 con 0x7f36f8130960 2023-09-30T22:17:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.172+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f806fee0 con 0x7f36f8130960 2023-09-30T22:17:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.172+0000 7f36edffb700 1 -- 172.21.15.89:0/444482096 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f36e8040eb0 con 0x7f36f8130960 2023-09-30T22:17:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.172+0000 7f36fd622700 1 -- 172.21.15.89:0/444482096 >> v1:172.21.15.89:6816/126953 conn(0x7f36dc02f510 legacy=0x7f36dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.173+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe40747f0 con 0x7f1fe4133930 2023-09-30T22:17:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.177+0000 7f36edffb700 1 -- 172.21.15.89:0/444482096 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f36e80239c0 con 0x7f36f8130960 2023-09-30T22:17:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.181+0000 7f1fe0ff9700 1 -- 172.21.15.89:0/4141827929 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1fdc018020 con 0x7f1fe4133930 2023-09-30T22:17:06.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.226+0000 7f08d4e7b700 1 Processor -- start 2023-09-30T22:17:06.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.226+0000 7f08d4e7b700 1 -- start start 2023-09-30T22:17:06.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08d4e7b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d00a0a30 0x7f08d00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08d4e7b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d0134170 0x7f08d013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08d4e7b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d009edd0 con 0x7f08d00a1340 2023-09-30T22:17:06.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08d4e7b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d009e1f0 con 0x7f08d0134170 2023-09-30T22:17:06.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08d4e7b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08d009dc50 con 0x7f08d00a0a30 2023-09-30T22:17:06.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ce59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d00a0a30 0x7f08d00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ce59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d00a0a30 0x7f08d00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42744/0 (socket says 172.21.15.89:42744) 2023-09-30T22:17:06.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ce59c700 1 -- 172.21.15.89:0/1420550438 learned_addr learned my addr 172.21.15.89:0/1420550438 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:06.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ced9d700 1 --2- 172.21.15.89:0/1420550438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d0134170 0x7f08d013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ce59c700 1 -- 172.21.15.89:0/1420550438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d0134170 msgr2=0x7f08d013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:06.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ce59c700 1 --2- 172.21.15.89:0/1420550438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d0134170 0x7f08d013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ce59c700 1 -- 172.21.15.89:0/1420550438 >> v1:172.21.15.89:6789/0 conn(0x7f08d00a1340 legacy=0x7f08d0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:06.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ce59c700 1 -- 172.21.15.89:0/1420550438 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d00a1fe0 con 0x7f08d00a0a30 2023-09-30T22:17:06.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08ce59c700 1 --2- 172.21.15.89:0/1420550438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d00a0a30 0x7f08d00a0e00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009920 tx=0x7f08c0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ab44628d8059cac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:06.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08cd59a700 1 -- 172.21.15.89:0/1420550438 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0004230 con 0x7f08d00a0a30 2023-09-30T22:17:06.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08cd59a700 1 -- 172.21.15.89:0/1420550438 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c0004390 con 0x7f08d00a0a30 2023-09-30T22:17:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.227+0000 7f08cd59a700 1 -- 172.21.15.89:0/1420550438 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0009e70 con 0x7f08d00a0a30 2023-09-30T22:17:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.228+0000 7f08d4e7b700 1 -- 172.21.15.89:0/1420550438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d00a0a30 msgr2=0x7f08d00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.228+0000 7f08d4e7b700 1 --2- 172.21.15.89:0/1420550438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d00a0a30 0x7f08d00a0e00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009920 tx=0x7f08c0009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.228+0000 7f08d4e7b700 1 -- 172.21.15.89:0/1420550438 shutdown_connections 2023-09-30T22:17:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.228+0000 7f08d4e7b700 1 --2- 172.21.15.89:0/1420550438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d0134170 0x7f08d013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.228+0000 7f08d4e7b700 1 --2- 172.21.15.89:0/1420550438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d00a0a30 0x7f08d00a0e00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.228+0000 7f08d4e7b700 1 -- 172.21.15.89:0/1420550438 >> 172.21.15.89:0/1420550438 conn(0x7f08d00a6550 msgr2=0x7f08d00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.228+0000 7f08d4e7b700 1 -- 172.21.15.89:0/1420550438 shutdown_connections 2023-09-30T22:17:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.228+0000 7f08d4e7b700 1 -- 172.21.15.89:0/1420550438 wait complete. 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.229+0000 7f08d4e7b700 1 Processor -- start 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.229+0000 7f08d4e7b700 1 -- start start 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08d4e7b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08d00a0a30 0x7f08d01233b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08d4e7b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d00a1340 0x7f08d01238f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08d4e7b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d0134170 0x7f08d01c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08d4e7b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0122f10 con 0x7f08d00a0a30 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08d4e7b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d00a0220 con 0x7f08d00a1340 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08d4e7b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0125f40 con 0x7f08d0134170 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d00a1340 0x7f08d01238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d00a1340 0x7f08d01238f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46300/0 (socket says 172.21.15.89:46300) 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 -- 172.21.15.89:0/2287511912 learned_addr learned my addr 172.21.15.89:0/2287511912 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08ced9d700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d0134170 0x7f08d01c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08ce59c700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08d00a0a30 0x7f08d01233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 -- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d0134170 msgr2=0x7f08d01c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d0134170 0x7f08d01c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 -- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08d00a0a30 msgr2=0x7f08d01233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08d00a0a30 0x7f08d01233b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 -- 172.21.15.89:0/2287511912 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c0009550 con 0x7f08d00a1340 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.230+0000 7f08cdd9b700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d00a1340 0x7f08d01238f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c4d0 tx=0x7f08c400c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.231+0000 7f08beffd700 1 -- 172.21.15.89:0/2287511912 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4011790 con 0x7f08d00a1340 2023-09-30T22:17:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.231+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d00a25e0 con 0x7f08d00a1340 2023-09-30T22:17:06.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.231+0000 7f08beffd700 1 -- 172.21.15.89:0/2287511912 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c40118f0 con 0x7f08d00a1340 2023-09-30T22:17:06.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.231+0000 7f08beffd700 1 -- 172.21.15.89:0/2287511912 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4011ca0 con 0x7f08d00a1340 2023-09-30T22:17:06.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.231+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01245d0 con 0x7f08d00a1340 2023-09-30T22:17:06.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.232+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d0074ea0 con 0x7f08d00a1340 2023-09-30T22:17:06.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.232+0000 7f08beffd700 1 -- 172.21.15.89:0/2287511912 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f08c401c460 con 0x7f08d00a1340 2023-09-30T22:17:06.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.232+0000 7f08beffd700 1 -- 172.21.15.89:0/2287511912 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f08c40403a0 con 0x7f08d00a1340 2023-09-30T22:17:06.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.232+0000 7f08ce59c700 1 -- 172.21.15.89:0/2287511912 >> v1:172.21.15.89:6816/126953 conn(0x7f08b002f510 legacy=0x7f08b00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:06.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.236+0000 7f08beffd700 1 -- 172.21.15.89:0/2287511912 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08c401a950 con 0x7f08d00a1340 2023-09-30T22:17:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.327+0000 7f92ce59c700 1 -- 172.21.15.89:0/1166900781 >> v1:172.21.15.89:6816/126953 conn(0x7f92b002f510 legacy=0x7f92b00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:06.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.337+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f92d009ffe0 con 0x7f92d009f650 2023-09-30T22:17:06.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.337+0000 7f92beffd700 1 -- 172.21.15.89:0/1166900781 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92c801b3b0 con 0x7f92d009f650 2023-09-30T22:17:06.337 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:06.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 >> v1:172.21.15.89:6816/126953 conn(0x7f92b002f510 legacy=0x7f92b00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:06.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d009f650 msgr2=0x7f92d006c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 --2- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d009f650 0x7f92d006c300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f92c800cbb0 tx=0x7f92c800cf70 comp rx=0 tx=0).stop 2023-09-30T22:17:06.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 shutdown_connections 2023-09-30T22:17:06.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 --2- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d009f650 0x7f92d006c300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 --2- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92d009e930 0x7f92d006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 --2- 172.21.15.89:0/1166900781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92d009f240 0x7f92d006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 >> 172.21.15.89:0/1166900781 conn(0x7f92d00a4210 msgr2=0x7f92d01b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:06.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 shutdown_connections 2023-09-30T22:17:06.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.339+0000 7f92d4ef4700 1 -- 172.21.15.89:0/1166900781 wait complete. 2023-09-30T22:17:06.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.371+0000 7f1fe3fff700 1 -- 172.21.15.89:0/4141827929 >> v1:172.21.15.89:6816/126953 conn(0x7f1fd002f510 legacy=0x7f1fd00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.373+0000 7f36fd622700 1 -- 172.21.15.89:0/444482096 >> v1:172.21.15.89:6816/126953 conn(0x7f36dc02f510 legacy=0x7f36dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:06.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.431+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f08d00a28e0 con 0x7f08d00a1340 2023-09-30T22:17:06.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.431+0000 7f08beffd700 1 -- 172.21.15.89:0/2287511912 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08c402a090 con 0x7f08d00a1340 2023-09-30T22:17:06.431 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:06.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08ce59c700 1 -- 172.21.15.89:0/2287511912 >> v1:172.21.15.89:6816/126953 conn(0x7f08b002f510 legacy=0x7f08b00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 >> v1:172.21.15.89:6816/126953 conn(0x7f08b002f510 legacy=0x7f08b00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d00a1340 msgr2=0x7f08d01238f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d00a1340 0x7f08d01238f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c4d0 tx=0x7f08c400c7e0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 shutdown_connections 2023-09-30T22:17:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08d0134170 0x7f08d01c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08d00a0a30 0x7f08d01233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 --2- 172.21.15.89:0/2287511912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08d00a1340 0x7f08d01238f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 >> 172.21.15.89:0/2287511912 conn(0x7f08d00a6550 msgr2=0x7f08d011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 shutdown_connections 2023-09-30T22:17:06.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.433+0000 7f08d4e7b700 1 -- 172.21.15.89:0/2287511912 wait complete. 2023-09-30T22:17:06.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.436+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f36f80747f0 con 0x7f36f8130960 2023-09-30T22:17:06.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.436+0000 7f36edffb700 1 -- 172.21.15.89:0/444482096 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36e8015020 con 0x7f36f8130960 2023-09-30T22:17:06.436 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:06.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.438+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 >> v1:172.21.15.89:6816/126953 conn(0x7f36dc02f510 legacy=0x7f36dc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130960 msgr2=0x7f36f80972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 --2- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130960 0x7f36f80972a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36e8000c00 tx=0x7f36e8004750 comp rx=0 tx=0).stop 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 shutdown_connections 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 --2- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f8130d50 0x7f36f80977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 --2- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36f8071b40 0x7f36f8098910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 --2- 172.21.15.89:0/444482096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36f8130960 0x7f36f80972a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 >> 172.21.15.89:0/444482096 conn(0x7f36f809e7e0 msgr2=0x7f36f8133480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 shutdown_connections 2023-09-30T22:17:06.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.439+0000 7f36ff886700 1 -- 172.21.15.89:0/444482096 wait complete. 2023-09-30T22:17:06.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.442+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1fe4071300 con 0x7f1fe4133930 2023-09-30T22:17:06.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.442+0000 7f1fe0ff9700 1 -- 172.21.15.89:0/4141827929 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1fdc007900 con 0x7f1fe4133930 2023-09-30T22:17:06.442 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 >> v1:172.21.15.89:6816/126953 conn(0x7f1fd002f510 legacy=0x7f1fd00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe4133930 msgr2=0x7f1fe41cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 --2- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe4133930 0x7f1fe41cc3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00efc0 tx=0x7f1fdc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 shutdown_connections 2023-09-30T22:17:06.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 --2- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fe4133930 0x7f1fe41cc3c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 --2- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fe409d630 0x7f1fe406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 --2- 172.21.15.89:0/4141827929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fe40a0a30 0x7f1fe406e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 >> 172.21.15.89:0/4141827929 conn(0x7f1fe40a3050 msgr2=0x7f1fe41bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 shutdown_connections 2023-09-30T22:17:06.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:06.444+0000 7f1fea9c4700 1 -- 172.21.15.89:0/4141827929 wait complete. 2023-09-30T22:17:06.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:06.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:06.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:07.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:07.448 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:07.453 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:07.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:07.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.490+0000 7fb099cb0700 1 Processor -- start 2023-09-30T22:17:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.490+0000 7fb099cb0700 1 -- start start 2023-09-30T22:17:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb099cb0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb09412cf60 0x7fb09406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb099cb0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb09412f8b0 0x7fb09412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb099cb0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09412cd90 con 0x7fb094130260 2023-09-30T22:17:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb099cb0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09412d330 con 0x7fb09412cf60 2023-09-30T22:17:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb099cb0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941306d0 con 0x7fb09412f8b0 2023-09-30T22:17:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb092ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb09412f8b0 0x7fb09412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb0937fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb09412cf60 0x7fb09406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb092ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb09412f8b0 0x7fb09412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46302/0 (socket says 172.21.15.89:46302) 2023-09-30T22:17:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb092ffd700 1 -- 172.21.15.89:0/3229535081 learned_addr learned my addr 172.21.15.89:0/3229535081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb092ffd700 1 -- 172.21.15.89:0/3229535081 >> v2:172.21.15.89:3300/0 conn(0x7fb09412cf60 msgr2=0x7fb09406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb092ffd700 1 --2- 172.21.15.89:0/3229535081 >> v2:172.21.15.89:3300/0 conn(0x7fb09412cf60 0x7fb09406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb092ffd700 1 -- 172.21.15.89:0/3229535081 >> v1:172.21.15.89:6789/0 conn(0x7fb094130260 legacy=0x7fb094136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.491+0000 7fb092ffd700 1 -- 172.21.15.89:0/3229535081 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094071780 con 0x7fb09412f8b0 2023-09-30T22:17:07.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb092ffd700 1 --2- 172.21.15.89:0/3229535081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb09412f8b0 0x7fb09412fd20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb08800bb90 tx=0x7fb08800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a427416389d7f01 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:07.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb0927fc700 1 -- 172.21.15.89:0/3229535081 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800dc20 con 0x7fb09412f8b0 2023-09-30T22:17:07.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb0927fc700 1 -- 172.21.15.89:0/3229535081 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08800dd80 con 0x7fb09412f8b0 2023-09-30T22:17:07.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb0927fc700 1 -- 172.21.15.89:0/3229535081 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088017630 con 0x7fb09412f8b0 2023-09-30T22:17:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb099cb0700 1 -- 172.21.15.89:0/3229535081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb09412f8b0 msgr2=0x7fb09412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb099cb0700 1 --2- 172.21.15.89:0/3229535081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb09412f8b0 0x7fb09412fd20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb08800bb90 tx=0x7fb08800bea0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb099cb0700 1 -- 172.21.15.89:0/3229535081 shutdown_connections 2023-09-30T22:17:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb099cb0700 1 --2- 172.21.15.89:0/3229535081 >> v2:172.21.15.89:3300/0 conn(0x7fb09412cf60 0x7fb09406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb099cb0700 1 --2- 172.21.15.89:0/3229535081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb09412f8b0 0x7fb09412fd20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb099cb0700 1 -- 172.21.15.89:0/3229535081 >> 172.21.15.89:0/3229535081 conn(0x7fb0940a2aa0 msgr2=0x7fb0940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:07.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb099cb0700 1 -- 172.21.15.89:0/3229535081 shutdown_connections 2023-09-30T22:17:07.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.492+0000 7fb099cb0700 1 -- 172.21.15.89:0/3229535081 wait complete. 2023-09-30T22:17:07.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.493+0000 7fb099cb0700 1 Processor -- start 2023-09-30T22:17:07.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb099cb0700 1 -- start start 2023-09-30T22:17:07.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb099cb0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb09412cf60 0x7fb09409bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb099cb0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb09412f8b0 0x7fb09409c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb099cb0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb094130260 0x7fb0941c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb099cb0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09409e6e0 con 0x7fb09412f8b0 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb099cb0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09409ce70 con 0x7fb094130260 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb099cb0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb094129fa0 con 0x7fb09412cf60 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb0937fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb09412cf60 0x7fb09409bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb0937fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb09412cf60 0x7fb09409bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42760/0 (socket says 172.21.15.89:42760) 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb0937fe700 1 -- 172.21.15.89:0/327172924 learned_addr learned my addr 172.21.15.89:0/327172924 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.494+0000 7fb093fff700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb094130260 0x7fb0941c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb0937fe700 1 -- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb094130260 msgr2=0x7fb0941c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb0937fe700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb094130260 0x7fb0941c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb0937fe700 1 -- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb09412f8b0 msgr2=0x7fb09409c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb0937fe700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb09412f8b0 0x7fb09409c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb0937fe700 1 -- 172.21.15.89:0/327172924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08800b7f0 con 0x7fb09412cf60 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb093fff700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb094130260 0x7fb0941c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb0937fe700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb09412cf60 0x7fb09409bd90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb08400efc0 tx=0x7fb08400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb07bfff700 1 -- 172.21.15.89:0/327172924 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08400cdc0 con 0x7fb09412cf60 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb07bfff700 1 -- 172.21.15.89:0/327172924 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08400f040 con 0x7fb09412cf60 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb07bfff700 1 -- 172.21.15.89:0/327172924 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0840146a0 con 0x7fb09412cf60 2023-09-30T22:17:07.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09409bae0 con 0x7fb09412cf60 2023-09-30T22:17:07.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.495+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09409b6e0 con 0x7fb09412cf60 2023-09-30T22:17:07.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.496+0000 7fb07bfff700 1 -- 172.21.15.89:0/327172924 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb084014800 con 0x7fb09412cf60 2023-09-30T22:17:07.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.496+0000 7fb092ffd700 1 -- 172.21.15.89:0/327172924 >> v1:172.21.15.89:6816/126953 conn(0x7fb08002f540 legacy=0x7fb0800319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:07.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.496+0000 7fb07bfff700 1 -- 172.21.15.89:0/327172924 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fb084040760 con 0x7fb09412cf60 2023-09-30T22:17:07.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.496+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09409ebe0 con 0x7fb09412cf60 2023-09-30T22:17:07.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.501+0000 7fb07bfff700 1 -- 172.21.15.89:0/327172924 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb08401a8b0 con 0x7fb09412cf60 2023-09-30T22:17:07.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.585+0000 7fae9f378700 1 Processor -- start 2023-09-30T22:17:07.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.585+0000 7fae9f378700 1 -- start start 2023-09-30T22:17:07.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9f378700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae98130750 0x7fae98130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9f378700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae9809e250 0x7fae981384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9f378700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9809c190 con 0x7fae9809d6f0 2023-09-30T22:17:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9f378700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9809b090 con 0x7fae98130750 2023-09-30T22:17:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9f378700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae9809a6c0 con 0x7fae9809e250 2023-09-30T22:17:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9d915700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae9809e250 0x7fae981384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9c913700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fae9809d6f0 0x7fae9809db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59990/0 (socket says 172.21.15.89:59990) 2023-09-30T22:17:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9c913700 1 -- 172.21.15.89:0/763161916 learned_addr learned my addr 172.21.15.89:0/763161916 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9d114700 1 --2- 172.21.15.89:0/763161916 >> v2:172.21.15.89:3300/0 conn(0x7fae98130750 0x7fae98130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9d915700 1 -- 172.21.15.89:0/763161916 >> v2:172.21.15.89:3300/0 conn(0x7fae98130750 msgr2=0x7fae98130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.586+0000 7fae9d915700 1 --2- 172.21.15.89:0/763161916 >> v2:172.21.15.89:3300/0 conn(0x7fae98130750 0x7fae98130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.587+0000 7fae9d915700 1 -- 172.21.15.89:0/763161916 >> v1:172.21.15.89:6789/0 conn(0x7fae9809d6f0 legacy=0x7fae9809db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.587+0000 7fae9d915700 1 -- 172.21.15.89:0/763161916 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae98131060 con 0x7fae9809e250 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.587+0000 7fae9d114700 1 --2- 172.21.15.89:0/763161916 >> v2:172.21.15.89:3300/0 conn(0x7fae98130750 0x7fae98130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.587+0000 7fae9d915700 1 --2- 172.21.15.89:0/763161916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae9809e250 0x7fae981384c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae90009d90 tx=0x7fae9000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b54ee589c86377b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.587+0000 7fae8ffff700 1 -- 172.21.15.89:0/763161916 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000e070 con 0x7fae9809e250 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.587+0000 7fae8ffff700 1 -- 172.21.15.89:0/763161916 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9000bb20 con 0x7fae9809e250 2023-09-30T22:17:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.587+0000 7fae8ffff700 1 -- 172.21.15.89:0/763161916 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90012430 con 0x7fae9809e250 2023-09-30T22:17:07.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.588+0000 7fae9f378700 1 -- 172.21.15.89:0/763161916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae9809e250 msgr2=0x7fae981384c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.588+0000 7fae9f378700 1 --2- 172.21.15.89:0/763161916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae9809e250 0x7fae981384c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae90009d90 tx=0x7fae9000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.588+0000 7fae9f378700 1 -- 172.21.15.89:0/763161916 shutdown_connections 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.588+0000 7fae9f378700 1 --2- 172.21.15.89:0/763161916 >> v2:172.21.15.89:3300/0 conn(0x7fae98130750 0x7fae98130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.588+0000 7fae9f378700 1 --2- 172.21.15.89:0/763161916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae9809e250 0x7fae981384c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.588+0000 7fae9f378700 1 -- 172.21.15.89:0/763161916 >> 172.21.15.89:0/763161916 conn(0x7fae9809f470 msgr2=0x7fae980a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.588+0000 7fae9f378700 1 -- 172.21.15.89:0/763161916 shutdown_connections 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.588+0000 7fae9f378700 1 -- 172.21.15.89:0/763161916 wait complete. 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.589+0000 7fae9f378700 1 Processor -- start 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.589+0000 7fae9f378700 1 -- start start 2023-09-30T22:17:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9f378700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae9809d6f0 0x7fae98135e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9f378700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae9809e250 0x7fae98136370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9f378700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae98130750 0x7fae981368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9f378700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae98137220 con 0x7fae9809d6f0 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9f378700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9809b2b0 con 0x7fae9809e250 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9f378700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae98136f50 con 0x7fae98130750 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9c913700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae9809e250 0x7fae98136370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9d915700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae98130750 0x7fae981368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9c913700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae9809e250 0x7fae98136370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46320/0 (socket says 172.21.15.89:46320) 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9c913700 1 -- 172.21.15.89:0/1912533500 learned_addr learned my addr 172.21.15.89:0/1912533500 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.590+0000 7fae9d114700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae9809d6f0 0x7fae98135e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9d114700 1 -- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae98130750 msgr2=0x7fae981368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9d114700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae98130750 0x7fae981368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9d114700 1 -- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae9809e250 msgr2=0x7fae98136370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9d114700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae9809e250 0x7fae98136370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9d114700 1 -- 172.21.15.89:0/1912533500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940094a0 con 0x7fae9809d6f0 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9d915700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae98130750 0x7fae981368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9c913700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae9809e250 0x7fae98136370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9d114700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae9809d6f0 0x7fae98135e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae9400e9b0 tx=0x7fae9400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae8dffb700 1 -- 172.21.15.89:0/1912533500 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400c850 con 0x7fae9809d6f0 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae8dffb700 1 -- 172.21.15.89:0/1912533500 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9400c9b0 con 0x7fae9809d6f0 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae8dffb700 1 -- 172.21.15.89:0/1912533500 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400cd60 con 0x7fae9809d6f0 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae90009a70 con 0x7fae9809d6f0 2023-09-30T22:17:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.591+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae98138750 con 0x7fae9809d6f0 2023-09-30T22:17:07.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.592+0000 7fae8dffb700 1 -- 172.21.15.89:0/1912533500 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fae94013470 con 0x7fae9809d6f0 2023-09-30T22:17:07.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.593+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae98132a10 con 0x7fae9809d6f0 2023-09-30T22:17:07.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.593+0000 7fae9c913700 1 -- 172.21.15.89:0/1912533500 >> v1:172.21.15.89:6816/126953 conn(0x7fae7c02f510 legacy=0x7fae7c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:07.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.593+0000 7fae8dffb700 1 -- 172.21.15.89:0/1912533500 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fae9403fff0 con 0x7fae9809d6f0 2023-09-30T22:17:07.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.597+0000 7fae8dffb700 1 -- 172.21.15.89:0/1912533500 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae9401b3b0 con 0x7fae9809d6f0 2023-09-30T22:17:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.622+0000 7fd08fda0700 1 Processor -- start 2023-09-30T22:17:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.623+0000 7fd08fda0700 1 -- start start 2023-09-30T22:17:07.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.623+0000 7fd08fda0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd08809d6f0 0x7fd08809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.623+0000 7fd08fda0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809e000 0x7fd088130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.623+0000 7fd08fda0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08809f130 con 0x7fd08809e410 2023-09-30T22:17:07.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.623+0000 7fd08fda0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08809f520 con 0x7fd08809d6f0 2023-09-30T22:17:07.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.623+0000 7fd08fda0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd08809e930 con 0x7fd08809e000 2023-09-30T22:17:07.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.623+0000 7fd08d33b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809e000 0x7fd088130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08d33b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809e000 0x7fd088130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42786/0 (socket says 172.21.15.89:42786) 2023-09-30T22:17:07.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08d33b700 1 -- 172.21.15.89:0/3170365672 learned_addr learned my addr 172.21.15.89:0/3170365672 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:07.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb9f7f700 1 Processor -- start 2023-09-30T22:17:07.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08db3c700 1 --2- 172.21.15.89:0/3170365672 >> v2:172.21.15.89:3300/0 conn(0x7fd08809d6f0 0x7fd08809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb9f7f700 1 -- start start 2023-09-30T22:17:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08cb3a700 1 -- 172.21.15.89:0/3170365672 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825086206 0 0) 0x7fd08809f130 con 0x7fd08809e410 2023-09-30T22:17:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08cb3a700 1 -- 172.21.15.89:0/3170365672 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0740057c0 con 0x7fd08809e410 2023-09-30T22:17:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08db3c700 1 -- 172.21.15.89:0/3170365672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809e000 msgr2=0x7fd088130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08db3c700 1 --2- 172.21.15.89:0/3170365672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809e000 0x7fd088130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08db3c700 1 -- 172.21.15.89:0/3170365672 >> v1:172.21.15.89:6789/0 conn(0x7fd08809e410 legacy=0x7fd088130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08db3c700 1 -- 172.21.15.89:0/3170365672 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08809b2d0 con 0x7fd08809d6f0 2023-09-30T22:17:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7fd08db3c700 1 --2- 172.21.15.89:0/3170365672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809d6f0 0x7fd08809dac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd084009840 tx=0x7fd084009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d2b48ecb4f7287 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb9f7f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4fb4136830 0x7f4fb4138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb9f7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb4139150 0x7f4fb413b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb9f7f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb406f720 con 0x7f4fb41338c0 2023-09-30T22:17:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb9f7f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb4070c80 con 0x7f4fb4136830 2023-09-30T22:17:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb9f7f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb406fce0 con 0x7f4fb4139150 2023-09-30T22:17:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4fb41338c0 0x7f4fb4133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60010/0 (socket says 172.21.15.89:60010) 2023-09-30T22:17:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.624+0000 7f4fb37fe700 1 -- 172.21.15.89:0/3129527430 learned_addr learned my addr 172.21.15.89:0/3129527430 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7fd08cb3a700 1 -- 172.21.15.89:0/3170365672 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084011830 con 0x7fd08809d6f0 2023-09-30T22:17:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7fd08cb3a700 1 -- 172.21.15.89:0/3170365672 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084011990 con 0x7fd08809d6f0 2023-09-30T22:17:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7fd08cb3a700 1 -- 172.21.15.89:0/3170365672 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084011d00 con 0x7fd08809d6f0 2023-09-30T22:17:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb2ffd700 1 --2- 172.21.15.89:0/3129527430 >> v2:172.21.15.89:3300/0 conn(0x7f4fb4136830 0x7f4fb4138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb3fff700 1 --2- 172.21.15.89:0/3129527430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb4139150 0x7f4fb413b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb27fc700 1 -- 172.21.15.89:0/3129527430 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562915313 0 0) 0x7f4fb406f720 con 0x7f4fb41338c0 2023-09-30T22:17:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb27fc700 1 -- 172.21.15.89:0/3129527430 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f9c0035b0 con 0x7f4fb41338c0 2023-09-30T22:17:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb2ffd700 1 -- 172.21.15.89:0/3129527430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb4139150 msgr2=0x7f4fb413b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb2ffd700 1 --2- 172.21.15.89:0/3129527430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb4139150 0x7f4fb413b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb2ffd700 1 -- 172.21.15.89:0/3129527430 >> v1:172.21.15.89:6789/0 conn(0x7f4fb41338c0 legacy=0x7f4fb4133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb2ffd700 1 -- 172.21.15.89:0/3129527430 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb41336a0 con 0x7f4fb4136830 2023-09-30T22:17:07.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb2ffd700 1 --2- 172.21.15.89:0/3129527430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fb4136830 0x7f4fb4138c10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009840 tx=0x7f4fa4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ced066786e60c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:07.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb27fc700 1 -- 172.21.15.89:0/3129527430 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4011830 con 0x7f4fb4136830 2023-09-30T22:17:07.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb27fc700 1 -- 172.21.15.89:0/3129527430 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4011990 con 0x7f4fb4136830 2023-09-30T22:17:07.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7f4fb27fc700 1 -- 172.21.15.89:0/3129527430 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4011d00 con 0x7f4fb4136830 2023-09-30T22:17:07.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7fd08fda0700 1 -- 172.21.15.89:0/3170365672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809d6f0 msgr2=0x7fd08809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.625+0000 7fd08fda0700 1 --2- 172.21.15.89:0/3170365672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809d6f0 0x7fd08809dac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd084009840 tx=0x7fd084009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:07.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7fd08fda0700 1 -- 172.21.15.89:0/3170365672 shutdown_connections 2023-09-30T22:17:07.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7fd08fda0700 1 --2- 172.21.15.89:0/3170365672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809d6f0 0x7fd08809dac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7fd08fda0700 1 --2- 172.21.15.89:0/3170365672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809e000 0x7fd088130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7fd08fda0700 1 -- 172.21.15.89:0/3170365672 >> 172.21.15.89:0/3170365672 conn(0x7fd088066f50 msgr2=0x7fd08809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7fd08fda0700 1 -- 172.21.15.89:0/3170365672 shutdown_connections 2023-09-30T22:17:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7fd08fda0700 1 -- 172.21.15.89:0/3170365672 wait complete. 2023-09-30T22:17:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/3129527430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fb4136830 msgr2=0x7f4fb4138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7f4fb9f7f700 1 --2- 172.21.15.89:0/3129527430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fb4136830 0x7f4fb4138c10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009840 tx=0x7f4fa4009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:07.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.626+0000 7fd08fda0700 1 Processor -- start 2023-09-30T22:17:07.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08fda0700 1 -- start start 2023-09-30T22:17:07.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08fda0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809d6f0 0x7fd0881cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08fda0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd08809e000 0x7fd0881ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08fda0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809e410 0x7fd08811dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08fda0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08811ff10 con 0x7fd08809e410 2023-09-30T22:17:07.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08fda0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880a1fe0 con 0x7fd08809e000 2023-09-30T22:17:07.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08fda0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd08811fa00 con 0x7fd08809d6f0 2023-09-30T22:17:07.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08e33d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809e410 0x7fd08811dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08e33d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809e410 0x7fd08811dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33494/0 (socket says 172.21.15.89:33494) 2023-09-30T22:17:07.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08e33d700 1 -- 172.21.15.89:0/97091494 learned_addr learned my addr 172.21.15.89:0/97091494 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08db3c700 1 --2- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809d6f0 0x7fd0881cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08e33d700 1 -- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809d6f0 msgr2=0x7fd0881cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08e33d700 1 --2- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809d6f0 0x7fd0881cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08e33d700 1 -- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd08809e000 msgr2=0x7fd0881ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08e33d700 1 --2- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd08809e000 0x7fd0881ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.627+0000 7fd08e33d700 1 -- 172.21.15.89:0/97091494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840094a0 con 0x7fd08809e410 2023-09-30T22:17:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.628+0000 7fd08e33d700 1 --2- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809e410 0x7fd08811dda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd08000cc20 tx=0x7fd08000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.628+0000 7fd07e7fc700 1 -- 172.21.15.89:0/97091494 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080003c00 con 0x7fd08809e410 2023-09-30T22:17:07.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.628+0000 7fd07e7fc700 1 -- 172.21.15.89:0/97091494 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd080003d60 con 0x7fd08809e410 2023-09-30T22:17:07.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.628+0000 7fd07e7fc700 1 -- 172.21.15.89:0/97091494 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000ab90 con 0x7fd08809e410 2023-09-30T22:17:07.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/3129527430 shutdown_connections 2023-09-30T22:17:07.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7f4fb9f7f700 1 --2- 172.21.15.89:0/3129527430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb4139150 0x7f4fb413b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7f4fb9f7f700 1 --2- 172.21.15.89:0/3129527430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fb4136830 0x7f4fb4138c10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/3129527430 >> 172.21.15.89:0/3129527430 conn(0x7f4fb40a2aa0 msgr2=0x7f4fb40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:07.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.628+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd088120150 con 0x7fd08809e410 2023-09-30T22:17:07.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/3129527430 shutdown_connections 2023-09-30T22:17:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.628+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08811e2e0 con 0x7fd08809e410 2023-09-30T22:17:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7fd07e7fc700 1 -- 172.21.15.89:0/97091494 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd08000acf0 con 0x7fd08809e410 2023-09-30T22:17:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/3129527430 wait complete. 2023-09-30T22:17:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd088074ea0 con 0x7fd08809e410 2023-09-30T22:17:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7fd08db3c700 1 -- 172.21.15.89:0/97091494 >> v1:172.21.15.89:6816/126953 conn(0x7fd06c02f510 legacy=0x7fd06c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.629+0000 7fd07e7fc700 1 -- 172.21.15.89:0/97091494 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fd08003fc60 con 0x7fd08809e410 2023-09-30T22:17:07.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.630+0000 7f4fb9f7f700 1 Processor -- start 2023-09-30T22:17:07.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb9f7f700 1 -- start start 2023-09-30T22:17:07.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb9f7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb41338c0 0x7f4fb409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb9f7f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fb4136830 0x7f4fb409af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb9f7f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fb4139150 0x7f4fb409b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:07.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb9f7f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41192a0 con 0x7f4fb4136830 2023-09-30T22:17:07.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb9f7f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb409bdf0 con 0x7f4fb41338c0 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb9f7f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb409de50 con 0x7f4fb4139150 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb41338c0 0x7f4fb409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb41338c0 0x7f4fb409a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46352/0 (socket says 172.21.15.89:46352) 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.631+0000 7f4fb37fe700 1 -- 172.21.15.89:0/1662382209 learned_addr learned my addr 172.21.15.89:0/1662382209 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4fb37fe700 1 -- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fb4139150 msgr2=0x7f4fb409b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4fb37fe700 1 --2- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fb4139150 0x7f4fb409b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4fb37fe700 1 -- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fb4136830 msgr2=0x7f4fb409af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4fb37fe700 1 --2- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fb4136830 0x7f4fb409af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4fb37fe700 1 -- 172.21.15.89:0/1662382209 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa40094a0 con 0x7f4fb41338c0 2023-09-30T22:17:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4fb37fe700 1 --2- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb41338c0 0x7f4fb409a9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800c700 tx=0x7f4fa800cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4f9bfff700 1 -- 172.21.15.89:0/1662382209 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8005d70 con 0x7f4fb41338c0 2023-09-30T22:17:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4f9bfff700 1 -- 172.21.15.89:0/1662382209 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa80070d0 con 0x7f4fb41338c0 2023-09-30T22:17:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.632+0000 7f4f9bfff700 1 -- 172.21.15.89:0/1662382209 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8007440 con 0x7f4fb41338c0 2023-09-30T22:17:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.633+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb409cc20 con 0x7f4fb41338c0 2023-09-30T22:17:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.633+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb4119d10 con 0x7f4fb41338c0 2023-09-30T22:17:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.634+0000 7f4f9bfff700 1 -- 172.21.15.89:0/1662382209 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4fa80075a0 con 0x7f4fb41338c0 2023-09-30T22:17:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.634+0000 7f4f9bfff700 1 -- 172.21.15.89:0/1662382209 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f4fa803f190 con 0x7f4fb41338c0 2023-09-30T22:17:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.634+0000 7f4fb2ffd700 1 -- 172.21.15.89:0/1662382209 >> v1:172.21.15.89:6816/126953 conn(0x7f4fa002f510 legacy=0x7f4fa00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.634+0000 7fd07e7fc700 1 -- 172.21.15.89:0/97091494 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd0800198f0 con 0x7fd08809e410 2023-09-30T22:17:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.634+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb4074ea0 con 0x7f4fb41338c0 2023-09-30T22:17:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.639+0000 7f4f9bfff700 1 -- 172.21.15.89:0/1662382209 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4fa8011d70 con 0x7f4fb41338c0 2023-09-30T22:17:07.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.697+0000 7fb092ffd700 1 -- 172.21.15.89:0/327172924 >> v1:172.21.15.89:6816/126953 conn(0x7fb08002f540 legacy=0x7fb0800319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:07.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.708+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb094074ea0 con 0x7fb09412cf60 2023-09-30T22:17:07.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.708+0000 7fb07bfff700 1 -- 172.21.15.89:0/327172924 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb08401a2c0 con 0x7fb09412cf60 2023-09-30T22:17:07.708 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:07.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.711+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 >> v1:172.21.15.89:6816/126953 conn(0x7fb08002f540 legacy=0x7fb0800319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:07.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.711+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb09412cf60 msgr2=0x7fb09409bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.711+0000 7fb099cb0700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb09412cf60 0x7fb09409bd90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb08400efc0 tx=0x7fb08400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.711+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 shutdown_connections 2023-09-30T22:17:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.711+0000 7fb099cb0700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb094130260 0x7fb0941c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.711+0000 7fb099cb0700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb09412cf60 0x7fb09409bd90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.711+0000 7fb099cb0700 1 --2- 172.21.15.89:0/327172924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb09412f8b0 0x7fb09409c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.711+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 >> 172.21.15.89:0/327172924 conn(0x7fb0940a2aa0 msgr2=0x7fb0941b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.712+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 shutdown_connections 2023-09-30T22:17:07.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.712+0000 7fb099cb0700 1 -- 172.21.15.89:0/327172924 wait complete. 2023-09-30T22:17:07.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:07.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.794+0000 7fae9c913700 1 -- 172.21.15.89:0/1912533500 >> v1:172.21.15.89:6816/126953 conn(0x7fae7c02f510 legacy=0x7fae7c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:07.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.813+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae980747f0 con 0x7fae9809d6f0 2023-09-30T22:17:07.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.813+0000 7fae8dffb700 1 -- 172.21.15.89:0/1912533500 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae94017070 con 0x7fae9809d6f0 2023-09-30T22:17:07.813 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 >> v1:172.21.15.89:6816/126953 conn(0x7fae7c02f510 legacy=0x7fae7c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae9809d6f0 msgr2=0x7fae98135e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae9809d6f0 0x7fae98135e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae9400e9b0 tx=0x7fae9400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 shutdown_connections 2023-09-30T22:17:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae98130750 0x7fae981368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae9809d6f0 0x7fae98135e30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 --2- 172.21.15.89:0/1912533500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae9809e250 0x7fae98136370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 >> 172.21.15.89:0/1912533500 conn(0x7fae9809f470 msgr2=0x7fae980715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 shutdown_connections 2023-09-30T22:17:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.816+0000 7fae9f378700 1 -- 172.21.15.89:0/1912533500 wait complete. 2023-09-30T22:17:07.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.830+0000 7fd08db3c700 1 -- 172.21.15.89:0/97091494 >> v1:172.21.15.89:6816/126953 conn(0x7fd06c02f510 legacy=0x7fd06c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:07.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:07.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.835+0000 7f4fb2ffd700 1 -- 172.21.15.89:0/1662382209 >> v1:172.21.15.89:6816/126953 conn(0x7f4fa002f510 legacy=0x7f4fa00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:07.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.842+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4fb406b9e0 con 0x7f4fb41338c0 2023-09-30T22:17:07.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.842+0000 7f4f9bfff700 1 -- 172.21.15.89:0/1662382209 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4fa801ae80 con 0x7f4fb41338c0 2023-09-30T22:17:07.843 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:07.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.844+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 >> v1:172.21.15.89:6816/126953 conn(0x7f4fa002f510 legacy=0x7f4fa00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:07.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb41338c0 msgr2=0x7f4fb409a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 --2- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb41338c0 0x7f4fb409a9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800c700 tx=0x7f4fa800cac0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 shutdown_connections 2023-09-30T22:17:07.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 --2- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fb4139150 0x7f4fb409b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 --2- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fb41338c0 0x7f4fb409a9f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 --2- 172.21.15.89:0/1662382209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fb4136830 0x7f4fb409af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 >> 172.21.15.89:0/1662382209 conn(0x7f4fb40a2aa0 msgr2=0x7f4fb4134d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:07.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 shutdown_connections 2023-09-30T22:17:07.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.845+0000 7f4fb9f7f700 1 -- 172.21.15.89:0/1662382209 wait complete. 2023-09-30T22:17:07.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.856+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd08806c360 con 0x7fd08809e410 2023-09-30T22:17:07.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:07.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.857+0000 7fd07e7fc700 1 -- 172.21.15.89:0/97091494 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd080019300 con 0x7fd08809e410 2023-09-30T22:17:07.858 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 >> v1:172.21.15.89:6816/126953 conn(0x7fd06c02f510 legacy=0x7fd06c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809e410 msgr2=0x7fd08811dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 --2- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809e410 0x7fd08811dda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd08000cc20 tx=0x7fd08000cf30 comp rx=0 tx=0).stop 2023-09-30T22:17:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 shutdown_connections 2023-09-30T22:17:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 --2- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd08809e410 0x7fd08811dda0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 --2- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd08809d6f0 0x7fd0881cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 --2- 172.21.15.89:0/97091494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd08809e000 0x7fd0881ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 >> 172.21.15.89:0/97091494 conn(0x7fd088066f50 msgr2=0x7fd08809fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:07.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 shutdown_connections 2023-09-30T22:17:07.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:07.859+0000 7fd08fda0700 1 -- 172.21.15.89:0/97091494 wait complete. 2023-09-30T22:17:07.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:08.730 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:08.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:08.858 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb162bf9700 1 Processor -- start 2023-09-30T22:17:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb162bf9700 1 -- start start 2023-09-30T22:17:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb162bf9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12cc20 0x7fb15c12f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb162bf9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb15c12c2f0 0x7fb15c12fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb162bf9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c1303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb162bf9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15c09a510 con 0x7fb15c12c2f0 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb162bf9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c12c7d0 con 0x7fb15c12cff0 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb162bf9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c09c190 con 0x7fb15c12cc20 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb160995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12cc20 0x7fb15c12f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb161196700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c1303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb15bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb15c12c2f0 0x7fb15c12fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb160995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12cc20 0x7fb15c12f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42818/0 (socket says 172.21.15.89:42818) 2023-09-30T22:17:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.866+0000 7fb160995700 1 -- 172.21.15.89:0/2710053256 learned_addr learned my addr 172.21.15.89:0/2710053256 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb161196700 1 -- 172.21.15.89:0/2710053256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12cc20 msgr2=0x7fb15c12f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb161196700 1 --2- 172.21.15.89:0/2710053256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12cc20 0x7fb15c12f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb161196700 1 -- 172.21.15.89:0/2710053256 >> v2:172.21.15.89:3300/0 conn(0x7fb15c12c2f0 msgr2=0x7fb15c12fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb161196700 1 --2- 172.21.15.89:0/2710053256 >> v2:172.21.15.89:3300/0 conn(0x7fb15c12c2f0 0x7fb15c12fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb161196700 1 -- 172.21.15.89:0/2710053256 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c09b540 con 0x7fb15c12cff0 2023-09-30T22:17:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb161196700 1 --2- 172.21.15.89:0/2710053256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c1303b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb15400f370 tx=0x7fb15400f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee59f584eea7ce12 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb15b7fe700 1 -- 172.21.15.89:0/2710053256 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154012070 con 0x7fb15c12cff0 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb15b7fe700 1 -- 172.21.15.89:0/2710053256 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15400fe70 con 0x7fb15c12cff0 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.867+0000 7fb15b7fe700 1 -- 172.21.15.89:0/2710053256 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15400d2f0 con 0x7fb15c12cff0 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 -- 172.21.15.89:0/2710053256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 msgr2=0x7fb15c1303b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 --2- 172.21.15.89:0/2710053256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c1303b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb15400f370 tx=0x7fb15400f680 comp rx=0 tx=0).stop 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 -- 172.21.15.89:0/2710053256 shutdown_connections 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 --2- 172.21.15.89:0/2710053256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c1303b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 --2- 172.21.15.89:0/2710053256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12cc20 0x7fb15c12f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 --2- 172.21.15.89:0/2710053256 >> v2:172.21.15.89:3300/0 conn(0x7fb15c12c2f0 0x7fb15c12fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 -- 172.21.15.89:0/2710053256 >> 172.21.15.89:0/2710053256 conn(0x7fb15c09f470 msgr2=0x7fb15c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 -- 172.21.15.89:0/2710053256 shutdown_connections 2023-09-30T22:17:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.868+0000 7fb162bf9700 1 -- 172.21.15.89:0/2710053256 wait complete. 2023-09-30T22:17:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.869+0000 7fb162bf9700 1 Processor -- start 2023-09-30T22:17:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.869+0000 7fb162bf9700 1 -- start start 2023-09-30T22:17:08.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:08.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.870+0000 7fb162bf9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12c2f0 0x7fb15c128290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.870+0000 7fb162bf9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb15c12cc20 0x7fb15c1287d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.870+0000 7fb162bf9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c128d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.870+0000 7fb162bf9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c129f10 con 0x7fb15c12cc20 2023-09-30T22:17:08.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.870+0000 7fb162bf9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c06e0b0 con 0x7fb15c12cff0 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.870+0000 7fb162bf9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c06ee20 con 0x7fb15c12c2f0 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.870+0000 7fb15bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb15c12cc20 0x7fb15c1287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.870+0000 7fb160995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12c2f0 0x7fb15c128290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb15bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb15c12cc20 0x7fb15c1287d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33520/0 (socket says 172.21.15.89:33520) 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb160995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12c2f0 0x7fb15c128290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42834/0 (socket says 172.21.15.89:42834) 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb15bfff700 1 -- 172.21.15.89:0/2695627321 learned_addr learned my addr 172.21.15.89:0/2695627321 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb161196700 1 --2- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c128d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb15bfff700 1 -- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12c2f0 msgr2=0x7fb15c128290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb15bfff700 1 --2- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12c2f0 0x7fb15c128290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb15bfff700 1 -- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 msgr2=0x7fb15c128d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb15bfff700 1 --2- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c128d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb15bfff700 1 -- 172.21.15.89:0/2695627321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15400f050 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb15bfff700 1 --2- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb15c12cc20 0x7fb15c1287d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb14800e980 tx=0x7fb14800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb1597fa700 1 -- 172.21.15.89:0/2695627321 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148009710 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb1597fa700 1 -- 172.21.15.89:0/2695627321 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800c690 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb1597fa700 1 -- 172.21.15.89:0/2695627321 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800ca20 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c070120 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.871+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c129c30 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.872+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c0747f0 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.872+0000 7fb1597fa700 1 -- 172.21.15.89:0/2695627321 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb14800cb80 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.873+0000 7fb1597fa700 1 -- 172.21.15.89:0/2695627321 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fb14803fc40 con 0x7fb15c12cc20 2023-09-30T22:17:08.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.873+0000 7fb160995700 1 -- 172.21.15.89:0/2695627321 >> v1:172.21.15.89:6816/126953 conn(0x7fb14002f510 legacy=0x7fb1400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.879+0000 7fb1597fa700 1 -- 172.21.15.89:0/2695627321 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb1480232f0 con 0x7fb15c12cc20 2023-09-30T22:17:08.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.973+0000 7f22b094b700 1 Processor -- start 2023-09-30T22:17:08.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.973+0000 7f22b094b700 1 -- start start 2023-09-30T22:17:08.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22b094b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22b094b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22ac09e000 0x7f22ac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22b094b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac09b120 con 0x7f22ac09e410 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22b094b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac06b460 con 0x7f22ac09e000 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22b094b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac09a7e0 con 0x7f22ac09d6f0 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22aa59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22aa59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42846/0 (socket says 172.21.15.89:42846) 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22a9d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22ac09e000 0x7f22ac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22aad9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f22ac09e410 0x7f22ac133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60012/0 (socket says 172.21.15.89:60012) 2023-09-30T22:17:08.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22aa59c700 1 -- 172.21.15.89:0/411042909 learned_addr learned my addr 172.21.15.89:0/411042909 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22a9d9b700 1 -- 172.21.15.89:0/411042909 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09d6f0 msgr2=0x7f22ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22a9d9b700 1 --2- 172.21.15.89:0/411042909 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22a9d9b700 1 -- 172.21.15.89:0/411042909 >> v1:172.21.15.89:6789/0 conn(0x7f22ac09e410 legacy=0x7f22ac133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22a9d9b700 1 -- 172.21.15.89:0/411042909 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac09a500 con 0x7f22ac09e000 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.974+0000 7f22aa59c700 1 --2- 172.21.15.89:0/411042909 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.975+0000 7f22a9d9b700 1 --2- 172.21.15.89:0/411042909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 0x7f22ac1339f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009840 tx=0x7f22a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb36da060f6fdb24 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.975+0000 7f22a959a700 1 -- 172.21.15.89:0/411042909 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011830 con 0x7f22ac09e000 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.975+0000 7f22a959a700 1 -- 172.21.15.89:0/411042909 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a0011990 con 0x7f22ac09e000 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.975+0000 7f22a959a700 1 -- 172.21.15.89:0/411042909 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011d20 con 0x7f22ac09e000 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.976+0000 7f22b094b700 1 -- 172.21.15.89:0/411042909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 msgr2=0x7f22ac1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:08.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.976+0000 7f22b094b700 1 --2- 172.21.15.89:0/411042909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 0x7f22ac1339f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009840 tx=0x7f22a0009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:08.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.976+0000 7f22b094b700 1 -- 172.21.15.89:0/411042909 shutdown_connections 2023-09-30T22:17:08.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.976+0000 7f22b094b700 1 --2- 172.21.15.89:0/411042909 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09d6f0 0x7f22ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.976+0000 7f22b094b700 1 --2- 172.21.15.89:0/411042909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 0x7f22ac1339f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.976+0000 7f22b094b700 1 -- 172.21.15.89:0/411042909 >> 172.21.15.89:0/411042909 conn(0x7f22ac066f50 msgr2=0x7f22ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:08.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.976+0000 7f22b094b700 1 -- 172.21.15.89:0/411042909 shutdown_connections 2023-09-30T22:17:08.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.976+0000 7f22b094b700 1 -- 172.21.15.89:0/411042909 wait complete. 2023-09-30T22:17:08.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.977+0000 7f22b094b700 1 Processor -- start 2023-09-30T22:17:08.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.977+0000 7f22b094b700 1 -- start start 2023-09-30T22:17:08.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22b094b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22ac09d6f0 0x7f22ac1a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22b094b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 0x7f22ac1a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22b094b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09e410 0x7f22ac1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22b094b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1a93e0 con 0x7f22ac09e000 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22b094b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1a7b80 con 0x7f22ac09d6f0 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22b094b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac09c710 con 0x7f22ac09e410 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22aa59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22ac09d6f0 0x7f22ac1a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22aa59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22ac09d6f0 0x7f22ac1a5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46390/0 (socket says 172.21.15.89:46390) 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22aa59c700 1 -- 172.21.15.89:0/862625864 learned_addr learned my addr 172.21.15.89:0/862625864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22aad9d700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09e410 0x7f22ac1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.978+0000 7f22a9d9b700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 0x7f22ac1a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22aa59c700 1 -- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09e410 msgr2=0x7f22ac1a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:08.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22aa59c700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09e410 0x7f22ac1a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22aa59c700 1 -- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 msgr2=0x7f22ac1a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22aa59c700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 0x7f22ac1a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22aa59c700 1 -- 172.21.15.89:0/862625864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c009550 con 0x7f22ac09d6f0 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22aad9d700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09e410 0x7f22ac1a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22aa59c700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22ac09d6f0 0x7f22ac1a5ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f229c00ebe0 tx=0x7f229c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f229affd700 1 -- 172.21.15.89:0/862625864 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00c850 con 0x7f22ac09d6f0 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f229affd700 1 -- 172.21.15.89:0/862625864 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c00c9b0 con 0x7f22ac09d6f0 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a00094a0 con 0x7f22ac09d6f0 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f229affd700 1 -- 172.21.15.89:0/862625864 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00cd40 con 0x7f22ac09d6f0 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.979+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac1a8870 con 0x7f22ac09d6f0 2023-09-30T22:17:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.980+0000 7f229affd700 1 -- 172.21.15.89:0/862625864 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f229c013470 con 0x7f22ac09d6f0 2023-09-30T22:17:08.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.980+0000 7f229affd700 1 -- 172.21.15.89:0/862625864 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f229c0400e0 con 0x7f22ac09d6f0 2023-09-30T22:17:08.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.980+0000 7f22a9d9b700 1 -- 172.21.15.89:0/862625864 >> v1:172.21.15.89:6816/126953 conn(0x7f228c02f510 legacy=0x7f228c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:08.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.981+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac074ea0 con 0x7f22ac09d6f0 2023-09-30T22:17:08.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:08.990+0000 7f229affd700 1 -- 172.21.15.89:0/862625864 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f229c01b3b0 con 0x7f22ac09d6f0 2023-09-30T22:17:09.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.000+0000 7f275eda2700 1 Processor -- start 2023-09-30T22:17:09.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275eda2700 1 -- start start 2023-09-30T22:17:09.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275eda2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758132c00 0x7f2758134fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275eda2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2758135520 0x7f2758137900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275eda2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275809d670 con 0x7f275809dba0 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275eda2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275809e680 con 0x7f2758135520 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275eda2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2758099d80 con 0x7f2758132c00 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275d59f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758132c00 0x7f2758134fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275d59f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758132c00 0x7f2758134fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46392/0 (socket says 172.21.15.89:46392) 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.001+0000 7f275d59f700 1 -- 172.21.15.89:0/680879744 learned_addr learned my addr 172.21.15.89:0/680879744 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275e5a1700 1 --2- 172.21.15.89:0/680879744 >> v2:172.21.15.89:3300/0 conn(0x7f2758135520 0x7f2758137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275d59f700 1 -- 172.21.15.89:0/680879744 >> v2:172.21.15.89:3300/0 conn(0x7f2758135520 msgr2=0x7f2758137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275d59f700 1 --2- 172.21.15.89:0/680879744 >> v2:172.21.15.89:3300/0 conn(0x7f2758135520 0x7f2758137900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275d59f700 1 -- 172.21.15.89:0/680879744 >> v1:172.21.15.89:6789/0 conn(0x7f275809dba0 legacy=0x7f275809df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275d59f700 1 -- 172.21.15.89:0/680879744 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275809b020 con 0x7f2758132c00 2023-09-30T22:17:09.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275e5a1700 1 --2- 172.21.15.89:0/680879744 >> v2:172.21.15.89:3300/0 conn(0x7f2758135520 0x7f2758137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:09.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275d59f700 1 --2- 172.21.15.89:0/680879744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758132c00 0x7f2758134fe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f27480098f0 tx=0x7f2748009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0e833d1d5ac8c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275cd9e700 1 -- 172.21.15.89:0/680879744 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274800b8a0 con 0x7f2758132c00 2023-09-30T22:17:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275cd9e700 1 -- 172.21.15.89:0/680879744 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274800ba00 con 0x7f2758132c00 2023-09-30T22:17:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.002+0000 7f275cd9e700 1 -- 172.21.15.89:0/680879744 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274800bd90 con 0x7f2758132c00 2023-09-30T22:17:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.003+0000 7f275eda2700 1 -- 172.21.15.89:0/680879744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758132c00 msgr2=0x7f2758134fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.003+0000 7f275eda2700 1 --2- 172.21.15.89:0/680879744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758132c00 0x7f2758134fe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f27480098f0 tx=0x7f2748009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.003+0000 7f275eda2700 1 -- 172.21.15.89:0/680879744 shutdown_connections 2023-09-30T22:17:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.003+0000 7f275eda2700 1 --2- 172.21.15.89:0/680879744 >> v2:172.21.15.89:3300/0 conn(0x7f2758135520 0x7f2758137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.003+0000 7f275eda2700 1 --2- 172.21.15.89:0/680879744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758132c00 0x7f2758134fe0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.003+0000 7f275eda2700 1 -- 172.21.15.89:0/680879744 >> 172.21.15.89:0/680879744 conn(0x7f27580a2540 msgr2=0x7f27580a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.003+0000 7f275eda2700 1 -- 172.21.15.89:0/680879744 shutdown_connections 2023-09-30T22:17:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.003+0000 7f275eda2700 1 -- 172.21.15.89:0/680879744 wait complete. 2023-09-30T22:17:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.004+0000 7f275eda2700 1 Processor -- start 2023-09-30T22:17:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.004+0000 7f275eda2700 1 -- start start 2023-09-30T22:17:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275eda2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f275809dba0 0x7f275812efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275eda2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2758132c00 0x7f275812f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275eda2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758135520 0x7f275812fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275eda2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758134f90 con 0x7f2758132c00 2023-09-30T22:17:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275eda2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2758134030 con 0x7f2758135520 2023-09-30T22:17:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275eda2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27581342f0 con 0x7f275809dba0 2023-09-30T22:17:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275e5a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758135520 0x7f275812fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275e5a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758135520 0x7f275812fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46398/0 (socket says 172.21.15.89:46398) 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275e5a1700 1 -- 172.21.15.89:0/1782370844 learned_addr learned my addr 172.21.15.89:0/1782370844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275d59f700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2758132c00 0x7f275812f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275dda0700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f275809dba0 0x7f275812efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275e5a1700 1 -- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f275809dba0 msgr2=0x7f275812efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275e5a1700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f275809dba0 0x7f275812efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275e5a1700 1 -- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2758132c00 msgr2=0x7f275812f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275e5a1700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2758132c00 0x7f275812f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.005+0000 7f275e5a1700 1 -- 172.21.15.89:0/1782370844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748009550 con 0x7f2758135520 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.006+0000 7f275e5a1700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758135520 0x7f275812fa30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2750009fa0 tx=0x7f275000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.006+0000 7f274e7fc700 1 -- 172.21.15.89:0/1782370844 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275000ddb0 con 0x7f2758135520 2023-09-30T22:17:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.006+0000 7f274e7fc700 1 -- 172.21.15.89:0/1782370844 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275000f040 con 0x7f2758135520 2023-09-30T22:17:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.006+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27581330b0 con 0x7f2758135520 2023-09-30T22:17:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.006+0000 7f274e7fc700 1 -- 172.21.15.89:0/1782370844 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750003c60 con 0x7f2758135520 2023-09-30T22:17:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.006+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2758072120 con 0x7f2758135520 2023-09-30T22:17:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.007+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2758097900 con 0x7f2758135520 2023-09-30T22:17:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.007+0000 7f274e7fc700 1 -- 172.21.15.89:0/1782370844 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2750004030 con 0x7f2758135520 2023-09-30T22:17:09.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.007+0000 7f274e7fc700 1 -- 172.21.15.89:0/1782370844 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f275003fe20 con 0x7f2758135520 2023-09-30T22:17:09.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.007+0000 7f275dda0700 1 -- 172.21.15.89:0/1782370844 >> v1:172.21.15.89:6816/126953 conn(0x7f273c02f510 legacy=0x7f273c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:09.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.008+0000 7f6fb8c52700 1 Processor -- start 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.008+0000 7f6fb8c52700 1 -- start start 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.009+0000 7f6fb8c52700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.009+0000 7f6fb8c52700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6fb4135e50 0x7f6fb4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.009+0000 7f6fb8c52700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb409f130 con 0x7f6fb409d6f0 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.009+0000 7f6fb8c52700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb409f520 con 0x7f6fb4135e50 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.009+0000 7f6fb8c52700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb409e930 con 0x7f6fb409e1d0 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.010+0000 7f6fb1d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.010+0000 7f6fb1d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46406/0 (socket says 172.21.15.89:46406) 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.010+0000 7f6fb1d9b700 1 -- 172.21.15.89:0/3276618645 learned_addr learned my addr 172.21.15.89:0/3276618645 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.010+0000 7f6fb1d9b700 1 -- 172.21.15.89:0/3276618645 >> v2:172.21.15.89:3300/0 conn(0x7f6fb4135e50 msgr2=0x7f6fb4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.010+0000 7f6fb1d9b700 1 --2- 172.21.15.89:0/3276618645 >> v2:172.21.15.89:3300/0 conn(0x7f6fb4135e50 0x7f6fb4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.010+0000 7f6fb1d9b700 1 -- 172.21.15.89:0/3276618645 >> v1:172.21.15.89:6789/0 conn(0x7f6fb409d6f0 legacy=0x7f6fb409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.010+0000 7f6fb1d9b700 1 -- 172.21.15.89:0/3276618645 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb409b2d0 con 0x7f6fb409e1d0 2023-09-30T22:17:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.010+0000 7f6fb1d9b700 1 --2- 172.21.15.89:0/3276618645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4135910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009920 tx=0x7f6fa8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a7805f471caf50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.012+0000 7f6fb159a700 1 -- 172.21.15.89:0/3276618645 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8004230 con 0x7f6fb409e1d0 2023-09-30T22:17:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.012+0000 7f6fb159a700 1 -- 172.21.15.89:0/3276618645 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa8004390 con 0x7f6fb409e1d0 2023-09-30T22:17:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.012+0000 7f6fb159a700 1 -- 172.21.15.89:0/3276618645 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8009e70 con 0x7f6fb409e1d0 2023-09-30T22:17:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.013+0000 7f274e7fc700 1 -- 172.21.15.89:0/1782370844 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f275001e3b0 con 0x7f2758135520 2023-09-30T22:17:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.014+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3276618645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 msgr2=0x7f6fb4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.014+0000 7f6fb8c52700 1 --2- 172.21.15.89:0/3276618645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4135910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009920 tx=0x7f6fa8009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.014+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3276618645 shutdown_connections 2023-09-30T22:17:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.014+0000 7f6fb8c52700 1 --2- 172.21.15.89:0/3276618645 >> v2:172.21.15.89:3300/0 conn(0x7f6fb4135e50 0x7f6fb4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.014+0000 7f6fb8c52700 1 --2- 172.21.15.89:0/3276618645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4135910 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.014+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3276618645 >> 172.21.15.89:0/3276618645 conn(0x7f6fb4066f50 msgr2=0x7f6fb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.014+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3276618645 shutdown_connections 2023-09-30T22:17:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.014+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3276618645 wait complete. 2023-09-30T22:17:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.015+0000 7f6fb8c52700 1 Processor -- start 2023-09-30T22:17:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb8c52700 1 -- start start 2023-09-30T22:17:09.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb8c52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb409d6f0 0x7f6fb4132b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb8c52700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4133090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb8c52700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4135e50 0x7f6fb41335d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb8c52700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4134800 con 0x7f6fb4135e50 2023-09-30T22:17:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb8c52700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41199c0 con 0x7f6fb409e1d0 2023-09-30T22:17:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb8c52700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4119720 con 0x7f6fb409d6f0 2023-09-30T22:17:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb1d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4133090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb1d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4133090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46410/0 (socket says 172.21.15.89:46410) 2023-09-30T22:17:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb1d9b700 1 -- 172.21.15.89:0/3026717096 learned_addr learned my addr 172.21.15.89:0/3026717096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb1d9b700 1 -- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb409d6f0 msgr2=0x7f6fb4132b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb1d9b700 1 --2- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb409d6f0 0x7f6fb4132b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb1d9b700 1 -- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4135e50 msgr2=0x7f6fb41335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.016+0000 7f6fb1d9b700 1 --2- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4135e50 0x7f6fb41335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.017+0000 7f6fb1d9b700 1 -- 172.21.15.89:0/3026717096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8009550 con 0x7f6fb409e1d0 2023-09-30T22:17:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.017+0000 7f6fb1d9b700 1 --2- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4133090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000c00 tx=0x7f6fa8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.017+0000 7f6fa2ffd700 1 -- 172.21.15.89:0/3026717096 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80036a0 con 0x7f6fb409e1d0 2023-09-30T22:17:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.017+0000 7f6fa2ffd700 1 -- 172.21.15.89:0/3026717096 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa8003800 con 0x7f6fb409e1d0 2023-09-30T22:17:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.017+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb4135150 con 0x7f6fb409e1d0 2023-09-30T22:17:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.017+0000 7f6fa2ffd700 1 -- 172.21.15.89:0/3026717096 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8003b90 con 0x7f6fb409e1d0 2023-09-30T22:17:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.017+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb4119ca0 con 0x7f6fb409e1d0 2023-09-30T22:17:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.018+0000 7f6fa2ffd700 1 -- 172.21.15.89:0/3026717096 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6fa8022070 con 0x7f6fb409e1d0 2023-09-30T22:17:09.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.018+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4071700 con 0x7f6fb409e1d0 2023-09-30T22:17:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.018+0000 7f6fa2ffd700 1 -- 172.21.15.89:0/3026717096 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f6fa80417f0 con 0x7f6fb409e1d0 2023-09-30T22:17:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.018+0000 7f6fb259c700 1 -- 172.21.15.89:0/3026717096 >> v1:172.21.15.89:6816/126953 conn(0x7f6f94033920 legacy=0x7f6f94035dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:09.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.027+0000 7f6fa2ffd700 1 -- 172.21.15.89:0/3026717096 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fa801a3d0 con 0x7f6fb409e1d0 2023-09-30T22:17:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.074+0000 7fb160995700 1 -- 172.21.15.89:0/2695627321 >> v1:172.21.15.89:6816/126953 conn(0x7fb14002f510 legacy=0x7fb1400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.085+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb15c06f320 con 0x7fb15c12cc20 2023-09-30T22:17:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.086+0000 7fb1597fa700 1 -- 172.21.15.89:0/2695627321 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb14801bd00 con 0x7fb15c12cc20 2023-09-30T22:17:09.085 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:09.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 >> v1:172.21.15.89:6816/126953 conn(0x7fb14002f510 legacy=0x7fb1400319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb15c12cc20 msgr2=0x7fb15c1287d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 --2- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb15c12cc20 0x7fb15c1287d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb14800e980 tx=0x7fb14800ec90 comp rx=0 tx=0).stop 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 shutdown_connections 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 --2- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb15c12cff0 0x7fb15c128d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 --2- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb15c12c2f0 0x7fb15c128290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 --2- 172.21.15.89:0/2695627321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb15c12cc20 0x7fb15c1287d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 >> 172.21.15.89:0/2695627321 conn(0x7fb15c09f470 msgr2=0x7fb15c12a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 shutdown_connections 2023-09-30T22:17:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.089+0000 7fb162bf9700 1 -- 172.21.15.89:0/2695627321 wait complete. 2023-09-30T22:17:09.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.181+0000 7f22a9d9b700 1 -- 172.21.15.89:0/862625864 >> v1:172.21.15.89:6816/126953 conn(0x7f228c02f510 legacy=0x7f228c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:09.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.190+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f22ac1a9550 con 0x7f22ac09d6f0 2023-09-30T22:17:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.190+0000 7f229affd700 1 -- 172.21.15.89:0/862625864 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f229c021070 con 0x7f22ac09d6f0 2023-09-30T22:17:09.190 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 >> v1:172.21.15.89:6816/126953 conn(0x7f228c02f510 legacy=0x7f228c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22ac09d6f0 msgr2=0x7f22ac1a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22ac09d6f0 0x7f22ac1a5ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f229c00ebe0 tx=0x7f229c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 shutdown_connections 2023-09-30T22:17:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22ac09e410 0x7f22ac1a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22ac09d6f0 0x7f22ac1a5ef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 --2- 172.21.15.89:0/862625864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22ac09e000 0x7f22ac1a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 >> 172.21.15.89:0/862625864 conn(0x7f22ac066f50 msgr2=0x7f22ac1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 shutdown_connections 2023-09-30T22:17:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.192+0000 7f22b094b700 1 -- 172.21.15.89:0/862625864 wait complete. 2023-09-30T22:17:09.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:09.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.208+0000 7f275dda0700 1 -- 172.21.15.89:0/1782370844 >> v1:172.21.15.89:6816/126953 conn(0x7f273c02f510 legacy=0x7f273c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:09.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.217+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f275809a8f0 con 0x7f2758135520 2023-09-30T22:17:09.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.218+0000 7f274e7fc700 1 -- 172.21.15.89:0/1782370844 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f275001a030 con 0x7f2758135520 2023-09-30T22:17:09.217 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:09.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.219+0000 7f6fb259c700 1 -- 172.21.15.89:0/3026717096 >> v1:172.21.15.89:6816/126953 conn(0x7f6f94033920 legacy=0x7f6f94035dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 >> v1:172.21.15.89:6816/126953 conn(0x7f273c02f510 legacy=0x7f273c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758135520 msgr2=0x7f275812fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758135520 0x7f275812fa30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2750009fa0 tx=0x7f275000d3b0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 shutdown_connections 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2758135520 0x7f275812fa30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f275809dba0 0x7f275812efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 --2- 172.21.15.89:0/1782370844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2758132c00 0x7f275812f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 >> 172.21.15.89:0/1782370844 conn(0x7f27580a2540 msgr2=0x7f2758095270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 shutdown_connections 2023-09-30T22:17:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f275eda2700 1 -- 172.21.15.89:0/1782370844 wait complete. 2023-09-30T22:17:09.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6fb4074ea0 con 0x7f6fb409e1d0 2023-09-30T22:17:09.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.220+0000 7f6fa2ffd700 1 -- 172.21.15.89:0/3026717096 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fa802a020 con 0x7f6fb409e1d0 2023-09-30T22:17:09.221 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:09.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 >> v1:172.21.15.89:6816/126953 conn(0x7f6f94033920 legacy=0x7f6f94035dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 msgr2=0x7f6fb4133090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 --2- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4133090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000c00 tx=0x7f6fa8004750 comp rx=0 tx=0).stop 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 shutdown_connections 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 --2- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4135e50 0x7f6fb41335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 --2- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb409d6f0 0x7f6fb4132b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 --2- 172.21.15.89:0/3026717096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb409e1d0 0x7f6fb4133090 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 >> 172.21.15.89:0/3026717096 conn(0x7f6fb4066f50 msgr2=0x7f6fb406e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 shutdown_connections 2023-09-30T22:17:09.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:09.223+0000 7f6fb8c52700 1 -- 172.21.15.89:0/3026717096 wait complete. 2023-09-30T22:17:09.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:09.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:10.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:10.205 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:10.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:10.236 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:10.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.242+0000 7f7947548700 1 Processor -- start 2023-09-30T22:17:10.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.242+0000 7f7947548700 1 -- start start 2023-09-30T22:17:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7947548700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79401337f0 0x7f7940138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7947548700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7940138dc0 0x7f794013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7947548700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794006b990 con 0x7f7940133420 2023-09-30T22:17:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7947548700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794009b0e0 con 0x7f7940138dc0 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7947548700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7940133c20 con 0x7f79401337f0 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7945ae5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7940138dc0 0x7f794013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7945ae5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7940138dc0 0x7f794013b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43224/0 (socket says 172.21.15.89:43224) 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7945ae5700 1 -- 172.21.15.89:0/2636441705 learned_addr learned my addr 172.21.15.89:0/2636441705 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7944ae3700 1 --2- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79401337f0 0x7f7940138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7945ae5700 1 -- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79401337f0 msgr2=0x7f7940138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7945ae5700 1 --2- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79401337f0 0x7f7940138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7945ae5700 1 -- 172.21.15.89:0/2636441705 >> v1:172.21.15.89:6789/0 conn(0x7f7940133420 legacy=0x7f7940133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7945ae5700 1 -- 172.21.15.89:0/2636441705 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400a0590 con 0x7f7940138dc0 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7937fff700 1 -- 172.21.15.89:0/2636441705 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882851358 0 0) 0x7f794006b990 con 0x7f7940133420 2023-09-30T22:17:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7944ae3700 1 --2- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79401337f0 0x7f7940138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:10.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7945ae5700 1 --2- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 0x7f794013b1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7938009e50 tx=0x7f793800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79eb38169be852f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:10.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.243+0000 7f7937fff700 1 -- 172.21.15.89:0/2636441705 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800e070 con 0x7f7940138dc0 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.244+0000 7f7937fff700 1 -- 172.21.15.89:0/2636441705 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793800b9c0 con 0x7f7940138dc0 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.244+0000 7f7937fff700 1 -- 172.21.15.89:0/2636441705 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800bd70 con 0x7f7940138dc0 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.244+0000 7f7947548700 1 -- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 msgr2=0x7f794013b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.244+0000 7f7947548700 1 --2- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 0x7f794013b1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7938009e50 tx=0x7f793800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.245+0000 7f7947548700 1 -- 172.21.15.89:0/2636441705 shutdown_connections 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.245+0000 7f7947548700 1 --2- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 0x7f794013b1a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.245+0000 7f7947548700 1 --2- 172.21.15.89:0/2636441705 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79401337f0 0x7f7940138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.245+0000 7f7947548700 1 -- 172.21.15.89:0/2636441705 >> 172.21.15.89:0/2636441705 conn(0x7f79400a4210 msgr2=0x7f79400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.245+0000 7f7947548700 1 -- 172.21.15.89:0/2636441705 shutdown_connections 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.245+0000 7f7947548700 1 -- 172.21.15.89:0/2636441705 wait complete. 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.246+0000 7f7947548700 1 Processor -- start 2023-09-30T22:17:10.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.246+0000 7f7947548700 1 -- start start 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7947548700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7940133420 0x7f794012f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7947548700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79401337f0 0x7f794012f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7947548700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 0x7f794012fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7947548700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794009b750 con 0x7f7940138dc0 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7947548700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794009c2c0 con 0x7f79401337f0 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7947548700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79401309e0 con 0x7f7940133420 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f79452e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7940133420 0x7f794012f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f79452e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7940133420 0x7f794012f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57362/0 (socket says 172.21.15.89:57362) 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f79452e4700 1 -- 172.21.15.89:0/904768829 learned_addr learned my addr 172.21.15.89:0/904768829 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7945ae5700 1 --2- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 0x7f794012fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7945ae5700 1 -- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7940133420 msgr2=0x7f794012f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7945ae5700 1 --2- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7940133420 0x7f794012f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7945ae5700 1 -- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79401337f0 msgr2=0x7f794012f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7945ae5700 1 --2- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79401337f0 0x7f794012f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.247+0000 7f7945ae5700 1 -- 172.21.15.89:0/904768829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938009b30 con 0x7f7940138dc0 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.248+0000 7f7945ae5700 1 --2- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 0x7f794012fb30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7938015000 tx=0x7f7938004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.248+0000 7f7935ffb700 1 -- 172.21.15.89:0/904768829 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938004930 con 0x7f7940138dc0 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.248+0000 7f7935ffb700 1 -- 172.21.15.89:0/904768829 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380092e0 con 0x7f7940138dc0 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.248+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794009c7f0 con 0x7f7940138dc0 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.248+0000 7f7935ffb700 1 -- 172.21.15.89:0/904768829 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793801c660 con 0x7f7940138dc0 2023-09-30T22:17:10.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.248+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79401388b0 con 0x7f7940138dc0 2023-09-30T22:17:10.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.249+0000 7f7935ffb700 1 -- 172.21.15.89:0/904768829 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f793801c7c0 con 0x7f7940138dc0 2023-09-30T22:17:10.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.249+0000 7f7935ffb700 1 -- 172.21.15.89:0/904768829 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f7938041480 con 0x7f7940138dc0 2023-09-30T22:17:10.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.249+0000 7f79452e4700 1 -- 172.21.15.89:0/904768829 >> v1:172.21.15.89:6816/126953 conn(0x7f792402f510 legacy=0x7f79240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:10.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.249+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7940074ea0 con 0x7f7940138dc0 2023-09-30T22:17:10.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.258+0000 7f7935ffb700 1 -- 172.21.15.89:0/904768829 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79380249a0 con 0x7f7940138dc0 2023-09-30T22:17:10.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.343+0000 7f0fca1a0700 1 Processor -- start 2023-09-30T22:17:10.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.343+0000 7f0fca1a0700 1 -- start start 2023-09-30T22:17:10.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fca1a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4133870 0x7f0fc4133c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fca1a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc4134180 0x7f0fc409d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fca1a0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0fc409d640 0x7f0fc409dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fca1a0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc409aa80 con 0x7f0fc409d640 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fca1a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4099780 con 0x7f0fc4133870 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fca1a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4133650 con 0x7f0fc4134180 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fc919e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4133870 0x7f0fc4133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fc919e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4133870 0x7f0fc4133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56192/0 (socket says 172.21.15.89:56192) 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fc919e700 1 -- 172.21.15.89:0/1146506007 learned_addr learned my addr 172.21.15.89:0/1146506007 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fc919e700 1 -- 172.21.15.89:0/1146506007 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc4134180 msgr2=0x7f0fc409d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fc919e700 1 --2- 172.21.15.89:0/1146506007 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc4134180 0x7f0fc409d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fc919e700 1 -- 172.21.15.89:0/1146506007 >> v2:172.21.15.89:3300/0 conn(0x7f0fc409d640 msgr2=0x7f0fc409dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:10.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fc919e700 1 --2- 172.21.15.89:0/1146506007 >> v2:172.21.15.89:3300/0 conn(0x7f0fc409d640 0x7f0fc409dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.344+0000 7f0fc919e700 1 -- 172.21.15.89:0/1146506007 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc409ebc0 con 0x7f0fc4133870 2023-09-30T22:17:10.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.345+0000 7f0fc919e700 1 --2- 172.21.15.89:0/1146506007 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4133870 0x7f0fc4133c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006fd0 tx=0x7f0fc0009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a727146350bd4643 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:10.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.345+0000 7f0fbbfff700 1 -- 172.21.15.89:0/1146506007 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0005e30 con 0x7f0fc4133870 2023-09-30T22:17:10.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.345+0000 7f0fbbfff700 1 -- 172.21.15.89:0/1146506007 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0004030 con 0x7f0fc4133870 2023-09-30T22:17:10.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.345+0000 7f0fbbfff700 1 -- 172.21.15.89:0/1146506007 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00043e0 con 0x7f0fc4133870 2023-09-30T22:17:10.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.345+0000 7f0fca1a0700 1 -- 172.21.15.89:0/1146506007 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4133870 msgr2=0x7f0fc4133c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.345+0000 7f0fca1a0700 1 --2- 172.21.15.89:0/1146506007 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4133870 0x7f0fc4133c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006fd0 tx=0x7f0fc0009bd0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.346+0000 7f0fca1a0700 1 -- 172.21.15.89:0/1146506007 shutdown_connections 2023-09-30T22:17:10.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.346+0000 7f0fca1a0700 1 --2- 172.21.15.89:0/1146506007 >> v2:172.21.15.89:3300/0 conn(0x7f0fc409d640 0x7f0fc409dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.346+0000 7f0fca1a0700 1 --2- 172.21.15.89:0/1146506007 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4133870 0x7f0fc4133c40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.346+0000 7f0fca1a0700 1 --2- 172.21.15.89:0/1146506007 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc4134180 0x7f0fc409d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.346+0000 7f0fca1a0700 1 -- 172.21.15.89:0/1146506007 >> 172.21.15.89:0/1146506007 conn(0x7f0fc40a2540 msgr2=0x7f0fc40a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.346+0000 7f0fca1a0700 1 -- 172.21.15.89:0/1146506007 shutdown_connections 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.346+0000 7f0fca1a0700 1 -- 172.21.15.89:0/1146506007 wait complete. 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.347+0000 7f0fca1a0700 1 Processor -- start 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.347+0000 7f0fca1a0700 1 -- start start 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fca1a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc409d640 0x7f0fc412eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fca1a0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4133870 0x7f0fc412f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fca1a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4134180 0x7f0fc412f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fca1a0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc408b4f0 con 0x7f0fc4133870 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fca1a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc409bfd0 con 0x7f0fc4134180 2023-09-30T22:17:10.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fca1a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc41312d0 con 0x7f0fc409d640 2023-09-30T22:17:10.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc409d640 0x7f0fc412eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc409d640 0x7f0fc412eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57384/0 (socket says 172.21.15.89:57384) 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 -- 172.21.15.89:0/2661379729 learned_addr learned my addr 172.21.15.89:0/2661379729 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc899d700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4133870 0x7f0fc412f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc999f700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4134180 0x7f0fc412f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 -- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4134180 msgr2=0x7f0fc412f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4134180 0x7f0fc412f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 -- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4133870 msgr2=0x7f0fc412f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4133870 0x7f0fc412f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 -- 172.21.15.89:0/2661379729 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0009500 con 0x7f0fc409d640 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.348+0000 7f0fc919e700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc409d640 0x7f0fc412eae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009130 tx=0x7f0fc0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.349+0000 7f0fb9ffb700 1 -- 172.21.15.89:0/2661379729 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00045c0 con 0x7f0fc409d640 2023-09-30T22:17:10.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.349+0000 7f0fb9ffb700 1 -- 172.21.15.89:0/2661379729 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc001d430 con 0x7f0fc409d640 2023-09-30T22:17:10.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.349+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc4130f10 con 0x7f0fc409d640 2023-09-30T22:17:10.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.349+0000 7f0fb9ffb700 1 -- 172.21.15.89:0/2661379729 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc001d7e0 con 0x7f0fc409d640 2023-09-30T22:17:10.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.349+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc408bed0 con 0x7f0fc409d640 2023-09-30T22:17:10.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.350+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc404f2a0 con 0x7f0fc409d640 2023-09-30T22:17:10.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.351+0000 7f0fb9ffb700 1 -- 172.21.15.89:0/2661379729 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0fc001d940 con 0x7f0fc409d640 2023-09-30T22:17:10.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.351+0000 7f0fb9ffb700 1 -- 172.21.15.89:0/2661379729 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f0fc0041520 con 0x7f0fc409d640 2023-09-30T22:17:10.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.351+0000 7f0fc899d700 1 -- 172.21.15.89:0/2661379729 >> v1:172.21.15.89:6816/126953 conn(0x7f0fa802f510 legacy=0x7f0fa80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:10.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.359+0000 7f0fb9ffb700 1 -- 172.21.15.89:0/2661379729 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0fc001dbf0 con 0x7f0fc409d640 2023-09-30T22:17:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.375+0000 7fddfb5e3700 1 Processor -- start 2023-09-30T22:17:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddfb5e3700 1 -- start start 2023-09-30T22:17:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddfb5e3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fddf40a0a30 0x7fddf40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddfb5e3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf40a1340 0x7fddf4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddfb5e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf4133fa0 0x7fddf4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddfb5e3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf409edd0 con 0x7fddf40a0a30 2023-09-30T22:17:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddfb5e3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf409e1f0 con 0x7fddf40a1340 2023-09-30T22:17:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddfb5e3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddf409dc50 con 0x7fddf4133fa0 2023-09-30T22:17:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddf8b7e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf40a1340 0x7fddf4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddf8b7e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf40a1340 0x7fddf4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56216/0 (socket says 172.21.15.89:56216) 2023-09-30T22:17:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddf8b7e700 1 -- 172.21.15.89:0/2244305807 learned_addr learned my addr 172.21.15.89:0/2244305807 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.376+0000 7fddf937f700 1 --2- 172.21.15.89:0/2244305807 >> v2:172.21.15.89:3300/0 conn(0x7fddf40a0a30 0x7fddf40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddf8b7e700 1 -- 172.21.15.89:0/2244305807 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf4133fa0 msgr2=0x7fddf4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddf8b7e700 1 --2- 172.21.15.89:0/2244305807 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf4133fa0 0x7fddf4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddf8b7e700 1 -- 172.21.15.89:0/2244305807 >> v2:172.21.15.89:3300/0 conn(0x7fddf40a0a30 msgr2=0x7fddf40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddf8b7e700 1 --2- 172.21.15.89:0/2244305807 >> v2:172.21.15.89:3300/0 conn(0x7fddf40a0a30 0x7fddf40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddf8b7e700 1 -- 172.21.15.89:0/2244305807 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40a1fe0 con 0x7fddf40a1340 2023-09-30T22:17:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddf8b7e700 1 --2- 172.21.15.89:0/2244305807 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf40a1340 0x7fddf4133a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009920 tx=0x7fdde4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fd3655c5ffa0489 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddebfff700 1 -- 172.21.15.89:0/2244305807 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4011830 con 0x7fddf40a1340 2023-09-30T22:17:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddebfff700 1 -- 172.21.15.89:0/2244305807 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde4011990 con 0x7fddf40a1340 2023-09-30T22:17:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7fddebfff700 1 -- 172.21.15.89:0/2244305807 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4011d20 con 0x7fddf40a1340 2023-09-30T22:17:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7f36d26dc700 1 Processor -- start 2023-09-30T22:17:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 -- 172.21.15.89:0/2244305807 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf40a1340 msgr2=0x7fddf4133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 --2- 172.21.15.89:0/2244305807 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf40a1340 0x7fddf4133a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009920 tx=0x7fdde4009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 -- 172.21.15.89:0/2244305807 shutdown_connections 2023-09-30T22:17:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 --2- 172.21.15.89:0/2244305807 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf4133fa0 0x7fddf4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 --2- 172.21.15.89:0/2244305807 >> v2:172.21.15.89:3300/0 conn(0x7fddf40a0a30 0x7fddf40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 --2- 172.21.15.89:0/2244305807 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf40a1340 0x7fddf4133a60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 -- 172.21.15.89:0/2244305807 >> 172.21.15.89:0/2244305807 conn(0x7fddf40a6550 msgr2=0x7fddf40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 -- 172.21.15.89:0/2244305807 shutdown_connections 2023-09-30T22:17:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.377+0000 7f36d26dc700 1 -- start start 2023-09-30T22:17:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36d26dc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc0a2360 0x7f36cc0a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36d26dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc13d130 0x7f36cc13f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36d26dc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36cc0a27e0 con 0x7f36cc06ccc0 2023-09-30T22:17:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36d26dc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc0a1dc0 con 0x7f36cc13d130 2023-09-30T22:17:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36d26dc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc09dde0 con 0x7f36cc0a2360 2023-09-30T22:17:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7fddfb5e3700 1 -- 172.21.15.89:0/2244305807 wait complete. 2023-09-30T22:17:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36cbfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f36cc06ccc0 0x7f36cc0a0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52740/0 (socket says 172.21.15.89:52740) 2023-09-30T22:17:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36cb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc0a2360 0x7f36cc0a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36cbfff700 1 -- 172.21.15.89:0/742128663 learned_addr learned my addr 172.21.15.89:0/742128663 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.378+0000 7f36d0c79700 1 --2- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc13d130 0x7f36cc13f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36cb7fe700 1 -- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc13d130 msgr2=0x7f36cc13f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36cb7fe700 1 --2- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc13d130 0x7f36cc13f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36cb7fe700 1 -- 172.21.15.89:0/742128663 >> v1:172.21.15.89:6789/0 conn(0x7f36cc06ccc0 legacy=0x7f36cc0a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36cb7fe700 1 -- 172.21.15.89:0/742128663 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc071b00 con 0x7f36cc0a2360 2023-09-30T22:17:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36d0c79700 1 --2- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc13d130 0x7f36cc13f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36cb7fe700 1 --2- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc0a2360 0x7f36cc0a1140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f36c00098a0 tx=0x7f36c0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd2d671aaae6122b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36caffd700 1 -- 172.21.15.89:0/742128663 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0004230 con 0x7f36cc0a2360 2023-09-30T22:17:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36caffd700 1 -- 172.21.15.89:0/742128663 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0004390 con 0x7f36cc0a2360 2023-09-30T22:17:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7f36caffd700 1 -- 172.21.15.89:0/742128663 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00036a0 con 0x7f36cc0a2360 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7fddfb5e3700 1 Processor -- start 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.379+0000 7fddfb5e3700 1 -- start start 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddfb5e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf40a0a30 0x7fddf4121530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddfb5e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf40a1340 0x7fddf4121a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddfb5e3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf4133fa0 0x7fddf4121fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddfb5e3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4132560 con 0x7fddf40a1340 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddfb5e3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4124d70 con 0x7fddf4133fa0 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddfb5e3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4123250 con 0x7fddf40a0a30 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf8b7e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf40a1340 0x7fddf4121a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf8b7e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf40a1340 0x7fddf4121a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43260/0 (socket says 172.21.15.89:43260) 2023-09-30T22:17:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf8b7e700 1 -- 172.21.15.89:0/1769407456 learned_addr learned my addr 172.21.15.89:0/1769407456 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf937f700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf40a0a30 0x7fddf4121530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf9b80700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf4133fa0 0x7fddf4121fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7f36d26dc700 1 -- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc0a2360 msgr2=0x7f36cc0a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7f36d26dc700 1 --2- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc0a2360 0x7f36cc0a1140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f36c00098a0 tx=0x7f36c0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf8b7e700 1 -- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf40a0a30 msgr2=0x7fddf4121530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf8b7e700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf40a0a30 0x7fddf4121530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf8b7e700 1 -- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf4133fa0 msgr2=0x7fddf4121fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf8b7e700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf4133fa0 0x7fddf4121fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf8b7e700 1 -- 172.21.15.89:0/1769407456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4009550 con 0x7fddf40a1340 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.380+0000 7fddf9b80700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf4133fa0 0x7fddf4121fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7fddf8b7e700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf40a1340 0x7fddf4121a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009130 tx=0x7fdde4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7fdde9ffb700 1 -- 172.21.15.89:0/1769407456 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde40044d0 con 0x7fddf40a1340 2023-09-30T22:17:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf40a0220 con 0x7fddf40a1340 2023-09-30T22:17:10.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7f36d26dc700 1 -- 172.21.15.89:0/742128663 shutdown_connections 2023-09-30T22:17:10.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7f36d26dc700 1 --2- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc13d130 0x7f36cc13f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7f36d26dc700 1 --2- 172.21.15.89:0/742128663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc0a2360 0x7f36cc0a1140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7f36d26dc700 1 -- 172.21.15.89:0/742128663 >> 172.21.15.89:0/742128663 conn(0x7f36cc0a6550 msgr2=0x7f36cc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:10.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7f36d26dc700 1 -- 172.21.15.89:0/742128663 shutdown_connections 2023-09-30T22:17:10.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf41257e0 con 0x7fddf40a1340 2023-09-30T22:17:10.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7fdde9ffb700 1 -- 172.21.15.89:0/1769407456 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde401e070 con 0x7fddf40a1340 2023-09-30T22:17:10.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7fdde9ffb700 1 -- 172.21.15.89:0/1769407456 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde40038c0 con 0x7fddf40a1340 2023-09-30T22:17:10.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.382+0000 7fdde9ffb700 1 -- 172.21.15.89:0/1769407456 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fdde4003a20 con 0x7fddf40a1340 2023-09-30T22:17:10.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.381+0000 7f36d26dc700 1 -- 172.21.15.89:0/742128663 wait complete. 2023-09-30T22:17:10.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.382+0000 7f36d26dc700 1 Processor -- start 2023-09-30T22:17:10.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.382+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd40052e0 con 0x7fddf40a1340 2023-09-30T22:17:10.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.382+0000 7fdde9ffb700 1 -- 172.21.15.89:0/1769407456 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fdde4020070 con 0x7fddf40a1340 2023-09-30T22:17:10.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.382+0000 7fddf937f700 1 -- 172.21.15.89:0/1769407456 >> v1:172.21.15.89:6816/126953 conn(0x7fddd802f510 legacy=0x7fddd80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:10.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36d26dc700 1 -- start start 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36d26dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36cc06ccc0 0x7f36cc13c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36d26dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc0a2360 0x7f36cc13c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36d26dc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc13d130 0x7f36cc13cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36d26dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc0a86e0 con 0x7f36cc06ccc0 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36d26dc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc0a9220 con 0x7f36cc0a2360 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36d26dc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc09fc40 con 0x7f36cc13d130 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36cb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc0a2360 0x7f36cc13c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36cb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc0a2360 0x7f36cc13c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56242/0 (socket says 172.21.15.89:56242) 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.383+0000 7f36cb7fe700 1 -- 172.21.15.89:0/4209564047 learned_addr learned my addr 172.21.15.89:0/4209564047 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36d0c79700 1 --2- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc13d130 0x7f36cc13cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36cb7fe700 1 -- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc13d130 msgr2=0x7f36cc13cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36cb7fe700 1 --2- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc13d130 0x7f36cc13cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36cb7fe700 1 -- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36cc06ccc0 msgr2=0x7f36cc13c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36cb7fe700 1 --2- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36cc06ccc0 0x7f36cc13c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36cb7fe700 1 -- 172.21.15.89:0/4209564047 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c4009a70 con 0x7f36cc0a2360 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36cb7fe700 1 --2- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc0a2360 0x7f36cc13c680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009870 tx=0x7f36c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36c8ff9700 1 -- 172.21.15.89:0/4209564047 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0003e90 con 0x7f36cc0a2360 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36c8ff9700 1 -- 172.21.15.89:0/4209564047 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0011630 con 0x7f36cc0a2360 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36c8ff9700 1 -- 172.21.15.89:0/4209564047 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00119e0 con 0x7f36cc0a2360 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c00094a0 con 0x7f36cc0a2360 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.384+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc09d9f0 con 0x7f36cc0a2360 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.386+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac0052e0 con 0x7f36cc0a2360 2023-09-30T22:17:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.386+0000 7f36c8ff9700 1 -- 172.21.15.89:0/4209564047 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f36c0011b40 con 0x7f36cc0a2360 2023-09-30T22:17:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.386+0000 7f36c8ff9700 1 -- 172.21.15.89:0/4209564047 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f36c0020030 con 0x7f36cc0a2360 2023-09-30T22:17:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.386+0000 7f36cbfff700 1 -- 172.21.15.89:0/4209564047 >> v1:172.21.15.89:6816/126953 conn(0x7f36b402f620 legacy=0x7f36b4031ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.387+0000 7fdde9ffb700 1 -- 172.21.15.89:0/1769407456 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdde4015070 con 0x7fddf40a1340 2023-09-30T22:17:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.391+0000 7f36c8ff9700 1 -- 172.21.15.89:0/4209564047 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f36c002e3e0 con 0x7f36cc0a2360 2023-09-30T22:17:10.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.450+0000 7f79452e4700 1 -- 172.21.15.89:0/904768829 >> v1:172.21.15.89:6816/126953 conn(0x7f792402f510 legacy=0x7f79240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.463+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f794006b250 con 0x7f7940138dc0 2023-09-30T22:17:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.464+0000 7f7935ffb700 1 -- 172.21.15.89:0/904768829 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7938013e80 con 0x7f7940138dc0 2023-09-30T22:17:10.464 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 >> v1:172.21.15.89:6816/126953 conn(0x7f792402f510 legacy=0x7f79240319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 msgr2=0x7f794012fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 --2- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 0x7f794012fb30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7938015000 tx=0x7f7938004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 shutdown_connections 2023-09-30T22:17:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 --2- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7940138dc0 0x7f794012fb30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 --2- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7940133420 0x7f794012f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 --2- 172.21.15.89:0/904768829 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79401337f0 0x7f794012f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 >> 172.21.15.89:0/904768829 conn(0x7f79400a4210 msgr2=0x7f79400a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 shutdown_connections 2023-09-30T22:17:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.466+0000 7f7947548700 1 -- 172.21.15.89:0/904768829 wait complete. 2023-09-30T22:17:10.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:10.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.552+0000 7f0fc899d700 1 -- 172.21.15.89:0/2661379729 >> v1:172.21.15.89:6816/126953 conn(0x7f0fa802f510 legacy=0x7f0fa80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.563+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0fc4074740 con 0x7f0fc409d640 2023-09-30T22:17:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.563+0000 7f0fb9ffb700 1 -- 172.21.15.89:0/2661379729 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0fc002b070 con 0x7f0fc409d640 2023-09-30T22:17:10.563 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:10.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 >> v1:172.21.15.89:6816/126953 conn(0x7f0fa802f510 legacy=0x7f0fa80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:10.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc409d640 msgr2=0x7f0fc412eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc409d640 0x7f0fc412eae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009130 tx=0x7f0fc0004750 comp rx=0 tx=0).stop 2023-09-30T22:17:10.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 shutdown_connections 2023-09-30T22:17:10.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc4134180 0x7f0fc412f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc409d640 0x7f0fc412eae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 --2- 172.21.15.89:0/2661379729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4133870 0x7f0fc412f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 >> 172.21.15.89:0/2661379729 conn(0x7f0fc40a2540 msgr2=0x7f0fc40a0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:10.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 shutdown_connections 2023-09-30T22:17:10.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.565+0000 7f0fca1a0700 1 -- 172.21.15.89:0/2661379729 wait complete. 2023-09-30T22:17:10.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:10.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.583+0000 7fddf937f700 1 -- 172.21.15.89:0/1769407456 >> v1:172.21.15.89:6816/126953 conn(0x7fddd802f510 legacy=0x7fddd80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.586+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f36ac005860 con 0x7f36cc0a2360 2023-09-30T22:17:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.587+0000 7f36cbfff700 1 -- 172.21.15.89:0/4209564047 >> v1:172.21.15.89:6816/126953 conn(0x7f36b402f620 legacy=0x7f36b4031ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.587+0000 7f36c8ff9700 1 -- 172.21.15.89:0/4209564047 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36c002a020 con 0x7f36cc0a2360 2023-09-30T22:17:10.587 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 >> v1:172.21.15.89:6816/126953 conn(0x7f36b402f620 legacy=0x7f36b4031ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc0a2360 msgr2=0x7f36cc13c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 --2- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc0a2360 0x7f36cc13c680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009870 tx=0x7f36c0004750 comp rx=0 tx=0).stop 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 shutdown_connections 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 --2- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36cc13d130 0x7f36cc13cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 --2- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36cc06ccc0 0x7f36cc13c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 --2- 172.21.15.89:0/4209564047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36cc0a2360 0x7f36cc13c680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 >> 172.21.15.89:0/4209564047 conn(0x7f36cc0a6550 msgr2=0x7f36cc0a2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 shutdown_connections 2023-09-30T22:17:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.589+0000 7f36d26dc700 1 -- 172.21.15.89:0/4209564047 wait complete. 2023-09-30T22:17:10.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.592+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fddd4005860 con 0x7fddf40a1340 2023-09-30T22:17:10.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.592+0000 7fdde9ffb700 1 -- 172.21.15.89:0/1769407456 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdde4010380 con 0x7fddf40a1340 2023-09-30T22:17:10.592 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 >> v1:172.21.15.89:6816/126953 conn(0x7fddd802f510 legacy=0x7fddd80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf40a1340 msgr2=0x7fddf4121a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf40a1340 0x7fddf4121a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009130 tx=0x7fdde4004750 comp rx=0 tx=0).stop 2023-09-30T22:17:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 shutdown_connections 2023-09-30T22:17:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddf4133fa0 0x7fddf4121fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddf40a0a30 0x7fddf4121530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 --2- 172.21.15.89:0/1769407456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf40a1340 0x7fddf4121a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 >> 172.21.15.89:0/1769407456 conn(0x7fddf40a6550 msgr2=0x7fddf40a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 shutdown_connections 2023-09-30T22:17:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:10.594+0000 7fddfb5e3700 1 -- 172.21.15.89:0/1769407456 wait complete. 2023-09-30T22:17:10.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:10.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:11.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:11.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:11.602 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:11.607 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:11.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.623+0000 7f8fb727b700 1 Processor -- start 2023-09-30T22:17:11.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.623+0000 7f8fb727b700 1 -- start start 2023-09-30T22:17:11.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb727b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e930 0x7f8fb009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb727b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8fb009f240 0x7f8fb01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb727b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb0070940 con 0x7f8fb009f650 2023-09-30T22:17:11.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb727b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb006fd60 con 0x7f8fb009f240 2023-09-30T22:17:11.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb727b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb006fa80 con 0x7f8fb009e930 2023-09-30T22:17:11.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb4816700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8fb009f240 0x7f8fb01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb5017700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e930 0x7f8fb009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb4816700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8fb009f240 0x7f8fb01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43292/0 (socket says 172.21.15.89:43292) 2023-09-30T22:17:11.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb4816700 1 -- 172.21.15.89:0/323351425 learned_addr learned my addr 172.21.15.89:0/323351425 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb5017700 1 -- 172.21.15.89:0/323351425 >> v2:172.21.15.89:3300/0 conn(0x7f8fb009f240 msgr2=0x7f8fb01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb5017700 1 --2- 172.21.15.89:0/323351425 >> v2:172.21.15.89:3300/0 conn(0x7f8fb009f240 0x7f8fb01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb5017700 1 -- 172.21.15.89:0/323351425 >> v1:172.21.15.89:6789/0 conn(0x7f8fb009f650 legacy=0x7f8fb0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb5017700 1 -- 172.21.15.89:0/323351425 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb009fb60 con 0x7f8fb009e930 2023-09-30T22:17:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb4816700 1 --2- 172.21.15.89:0/323351425 >> v2:172.21.15.89:3300/0 conn(0x7f8fb009f240 0x7f8fb01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.624+0000 7f8fb5017700 1 --2- 172.21.15.89:0/323351425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e930 0x7f8fb009ed00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009840 tx=0x7f8fac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf6b085d1a70ada server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fa7fff700 1 -- 172.21.15.89:0/323351425 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac011830 con 0x7f8fb009e930 2023-09-30T22:17:11.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fa7fff700 1 -- 172.21.15.89:0/323351425 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac011990 con 0x7f8fb009e930 2023-09-30T22:17:11.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fa7fff700 1 -- 172.21.15.89:0/323351425 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac011d20 con 0x7f8fb009e930 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fb727b700 1 -- 172.21.15.89:0/323351425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e930 msgr2=0x7f8fb009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fb727b700 1 --2- 172.21.15.89:0/323351425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e930 0x7f8fb009ed00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009840 tx=0x7f8fac009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fb727b700 1 -- 172.21.15.89:0/323351425 shutdown_connections 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fb727b700 1 --2- 172.21.15.89:0/323351425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e930 0x7f8fb009ed00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fb727b700 1 --2- 172.21.15.89:0/323351425 >> v2:172.21.15.89:3300/0 conn(0x7f8fb009f240 0x7f8fb01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fb727b700 1 -- 172.21.15.89:0/323351425 >> 172.21.15.89:0/323351425 conn(0x7f8fb00a4210 msgr2=0x7f8fb00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fb727b700 1 -- 172.21.15.89:0/323351425 shutdown_connections 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.625+0000 7f8fb727b700 1 -- 172.21.15.89:0/323351425 wait complete. 2023-09-30T22:17:11.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.627+0000 7f8fb727b700 1 Processor -- start 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.627+0000 7f8fb727b700 1 -- start start 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb727b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fb009e930 0x7f8fb01a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb727b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009f240 0x7f8fb01a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb727b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009f650 0x7f8fb01a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb727b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a51b0 con 0x7f8fb009f650 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb727b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a5910 con 0x7f8fb009e930 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb727b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a6b60 con 0x7f8fb009f240 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb4816700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009f240 0x7f8fb01a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb4816700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009f240 0x7f8fb01a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57426/0 (socket says 172.21.15.89:57426) 2023-09-30T22:17:11.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb4816700 1 -- 172.21.15.89:0/195149596 learned_addr learned my addr 172.21.15.89:0/195149596 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb5818700 1 --2- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009f650 0x7f8fb01a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb4816700 1 -- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fb009e930 msgr2=0x7f8fb01a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb4816700 1 --2- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fb009e930 0x7f8fb01a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb4816700 1 -- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009f650 msgr2=0x7f8fb01a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb4816700 1 --2- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009f650 0x7f8fb01a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.628+0000 7f8fb4816700 1 -- 172.21.15.89:0/195149596 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0009550 con 0x7f8fb009f240 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.629+0000 7f8fb4816700 1 --2- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009f240 0x7f8fb01a4240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000ec10 tx=0x7f8fa000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.629+0000 7f8fa5ffb700 1 -- 172.21.15.89:0/195149596 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000c850 con 0x7f8fb009f240 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.629+0000 7f8fa5ffb700 1 -- 172.21.15.89:0/195149596 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa000c9b0 con 0x7f8fb009f240 2023-09-30T22:17:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.629+0000 7f8fa5ffb700 1 -- 172.21.15.89:0/195149596 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000cd60 con 0x7f8fb009f240 2023-09-30T22:17:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.630+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac0094a0 con 0x7f8fb009f240 2023-09-30T22:17:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.630+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb00a0190 con 0x7f8fb009f240 2023-09-30T22:17:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.631+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb01a0b30 con 0x7f8fb009f240 2023-09-30T22:17:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.631+0000 7f8fa5ffb700 1 -- 172.21.15.89:0/195149596 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8fa0013470 con 0x7f8fb009f240 2023-09-30T22:17:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.631+0000 7f8fa5ffb700 1 -- 172.21.15.89:0/195149596 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f8fa003fc70 con 0x7f8fb009f240 2023-09-30T22:17:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.631+0000 7f8fb5017700 1 -- 172.21.15.89:0/195149596 >> v1:172.21.15.89:6816/126953 conn(0x7f8f9402f510 legacy=0x7f8f940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:11.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.640+0000 7f8fa5ffb700 1 -- 172.21.15.89:0/195149596 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8fa001bba0 con 0x7f8fb009f240 2023-09-30T22:17:11.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.723+0000 7f838d799700 1 Processor -- start 2023-09-30T22:17:11.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.724+0000 7f838d799700 1 -- start start 2023-09-30T22:17:11.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.724+0000 7f838d799700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f838809f410 0x7f8388136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.724+0000 7f838d799700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8388136fe0 0x7f83881393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.724+0000 7f838d799700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8388070940 con 0x7f838809e930 2023-09-30T22:17:11.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.724+0000 7f838d799700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838806fd60 con 0x7f838809f410 2023-09-30T22:17:11.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.724+0000 7f838d799700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f838806fa80 con 0x7f8388136fe0 2023-09-30T22:17:11.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.724+0000 7f83867fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f838809f410 0x7f8388136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f83877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8388136fe0 0x7f83881393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f83867fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f838809f410 0x7f8388136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43302/0 (socket says 172.21.15.89:43302) 2023-09-30T22:17:11.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f83867fc700 1 -- 172.21.15.89:0/3344294424 learned_addr learned my addr 172.21.15.89:0/3344294424 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:11.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f83877fe700 1 -- 172.21.15.89:0/3344294424 >> v2:172.21.15.89:3300/0 conn(0x7f838809f410 msgr2=0x7f8388136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:11.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f83877fe700 1 --2- 172.21.15.89:0/3344294424 >> v2:172.21.15.89:3300/0 conn(0x7f838809f410 0x7f8388136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f83877fe700 1 -- 172.21.15.89:0/3344294424 >> v1:172.21.15.89:6789/0 conn(0x7f838809e930 legacy=0x7f838809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f83877fe700 1 -- 172.21.15.89:0/3344294424 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838809fb60 con 0x7f8388136fe0 2023-09-30T22:17:11.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f83877fe700 1 --2- 172.21.15.89:0/3344294424 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8388136fe0 0x7f83881393c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8380009d90 tx=0x7f838000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689dc03af6f9759c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:11.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f8385ffb700 1 -- 172.21.15.89:0/3344294424 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838000e070 con 0x7f8388136fe0 2023-09-30T22:17:11.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f8385ffb700 1 -- 172.21.15.89:0/3344294424 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838000bb20 con 0x7f8388136fe0 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.725+0000 7f8385ffb700 1 -- 172.21.15.89:0/3344294424 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380012430 con 0x7f8388136fe0 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.726+0000 7f838d799700 1 -- 172.21.15.89:0/3344294424 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8388136fe0 msgr2=0x7f83881393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.726+0000 7f838d799700 1 --2- 172.21.15.89:0/3344294424 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8388136fe0 0x7f83881393c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8380009d90 tx=0x7f838000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.726+0000 7f838d799700 1 -- 172.21.15.89:0/3344294424 shutdown_connections 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.726+0000 7f838d799700 1 --2- 172.21.15.89:0/3344294424 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8388136fe0 0x7f83881393c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.726+0000 7f838d799700 1 --2- 172.21.15.89:0/3344294424 >> v2:172.21.15.89:3300/0 conn(0x7f838809f410 0x7f8388136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.726+0000 7f838d799700 1 -- 172.21.15.89:0/3344294424 >> 172.21.15.89:0/3344294424 conn(0x7f83880a4210 msgr2=0x7f83880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.726+0000 7f838d799700 1 -- 172.21.15.89:0/3344294424 shutdown_connections 2023-09-30T22:17:11.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.726+0000 7f838d799700 1 -- 172.21.15.89:0/3344294424 wait complete. 2023-09-30T22:17:11.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.727+0000 7f838d799700 1 Processor -- start 2023-09-30T22:17:11.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f838d799700 1 -- start start 2023-09-30T22:17:11.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f838d799700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f838809e930 0x7f838811c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f838d799700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f838809f410 0x7f838811ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f838d799700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8388136fe0 0x7f838811cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f838d799700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8388071db0 con 0x7f838809e930 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f838d799700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880a33f0 con 0x7f8388136fe0 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f838d799700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f838811f820 con 0x7f838809f410 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f83877fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8388136fe0 0x7f838811cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f83877fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8388136fe0 0x7f838811cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56252/0 (socket says 172.21.15.89:56252) 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f83877fe700 1 -- 172.21.15.89:0/1062482778 learned_addr learned my addr 172.21.15.89:0/1062482778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f83867fc700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f838809f410 0x7f838811ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.728+0000 7f8386ffd700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f838809e930 0x7f838811c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f83877fe700 1 -- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f838809f410 msgr2=0x7f838811ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f83877fe700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f838809f410 0x7f838811ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f83877fe700 1 -- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f838809e930 msgr2=0x7f838811c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f83877fe700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f838809e930 0x7f838811c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f83877fe700 1 -- 172.21.15.89:0/1062482778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8380009a70 con 0x7f8388136fe0 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f83867fc700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f838809f410 0x7f838811ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f8386ffd700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f838809e930 0x7f838811c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f83877fe700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8388136fe0 0x7f838811cfd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f838806fbc0 tx=0x7f8380004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f836f7fe700 1 -- 172.21.15.89:0/1062482778 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380012430 con 0x7f8388136fe0 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f836f7fe700 1 -- 172.21.15.89:0/1062482778 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8380020070 con 0x7f8388136fe0 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f836f7fe700 1 -- 172.21.15.89:0/1062482778 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380003ea0 con 0x7f8388136fe0 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838811e9c0 con 0x7f8388136fe0 2023-09-30T22:17:11.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.729+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838811fd90 con 0x7f8388136fe0 2023-09-30T22:17:11.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.730+0000 7f836f7fe700 1 -- 172.21.15.89:0/1062482778 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8380004280 con 0x7f8388136fe0 2023-09-30T22:17:11.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.731+0000 7f836f7fe700 1 -- 172.21.15.89:0/1062482778 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f8380022070 con 0x7f8388136fe0 2023-09-30T22:17:11.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.731+0000 7f8386ffd700 1 -- 172.21.15.89:0/1062482778 >> v1:172.21.15.89:6816/126953 conn(0x7f837402f510 legacy=0x7f83740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:11.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.731+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838811fbd0 con 0x7f8388136fe0 2023-09-30T22:17:11.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.736+0000 7f836f7fe700 1 -- 172.21.15.89:0/1062482778 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f838002c080 con 0x7f8388136fe0 2023-09-30T22:17:11.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.752+0000 7fc08a4c7700 1 Processor -- start 2023-09-30T22:17:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.752+0000 7f254adc4700 1 Processor -- start 2023-09-30T22:17:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.752+0000 7f254adc4700 1 -- start start 2023-09-30T22:17:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f254adc4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409d6f0 0x7f254409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f254adc4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f254409e410 0x7f2544134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f254adc4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254409b120 con 0x7f254409e000 2023-09-30T22:17:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f254adc4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254406b460 con 0x7f254409e410 2023-09-30T22:17:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f254adc4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254409a7e0 con 0x7f254409d6f0 2023-09-30T22:17:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f2548b60700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409d6f0 0x7f254409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f2548b60700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409d6f0 0x7f254409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56266/0 (socket says 172.21.15.89:56266) 2023-09-30T22:17:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f2548b60700 1 -- 172.21.15.89:0/335578930 learned_addr learned my addr 172.21.15.89:0/335578930 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.752+0000 7fc08a4c7700 1 -- start start 2023-09-30T22:17:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc08a4c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc08a4c7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc08409e000 0x7fc084130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc08a4c7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc084130ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc08a4c7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08409b420 con 0x7fc08409e000 2023-09-30T22:17:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc08a4c7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc084131530 con 0x7fc08409d6f0 2023-09-30T22:17:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc08a4c7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc08409a850 con 0x7fc084130ba0 2023-09-30T22:17:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc088a64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc084130ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc0837fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc08409e000 0x7fc084130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc088a64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc084130ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57460/0 (socket says 172.21.15.89:57460) 2023-09-30T22:17:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc088a64700 1 -- 172.21.15.89:0/2626429654 learned_addr learned my addr 172.21.15.89:0/2626429654 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc083fff700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f2548b60700 1 -- 172.21.15.89:0/335578930 >> v2:172.21.15.89:3300/0 conn(0x7f254409e410 msgr2=0x7f2544134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f2548b60700 1 --2- 172.21.15.89:0/335578930 >> v2:172.21.15.89:3300/0 conn(0x7f254409e410 0x7f2544134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f2548b60700 1 -- 172.21.15.89:0/335578930 >> v1:172.21.15.89:6789/0 conn(0x7f254409e000 legacy=0x7f25441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f2548b60700 1 -- 172.21.15.89:0/335578930 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254409a500 con 0x7f254409d6f0 2023-09-30T22:17:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7f2548b60700 1 --2- 172.21.15.89:0/335578930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409d6f0 0x7f254409dac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2538009920 tx=0x7f2538009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6addd00a65953866 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.753+0000 7fc0837fe700 1 -- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 msgr2=0x7fc084130ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7f25437fe700 1 -- 172.21.15.89:0/335578930 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538004230 con 0x7f254409d6f0 2023-09-30T22:17:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7f25437fe700 1 -- 172.21.15.89:0/335578930 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538004390 con 0x7f254409d6f0 2023-09-30T22:17:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7f25437fe700 1 -- 172.21.15.89:0/335578930 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538009e70 con 0x7f254409d6f0 2023-09-30T22:17:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc0837fe700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc084130ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc0837fe700 1 -- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 msgr2=0x7fc08409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc0837fe700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc0837fe700 1 -- 172.21.15.89:0/2626429654 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08409b8e0 con 0x7fc08409e000 2023-09-30T22:17:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc088a64700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc084130ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc083fff700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc0837fe700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc08409e000 0x7fc084130660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc07800dc60 tx=0x7fc07800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb1ebae970c82ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc082ffd700 1 -- 172.21.15.89:0/2626429654 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800b070 con 0x7fc08409e000 2023-09-30T22:17:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc082ffd700 1 -- 172.21.15.89:0/2626429654 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc078008a30 con 0x7fc08409e000 2023-09-30T22:17:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.754+0000 7fc082ffd700 1 -- 172.21.15.89:0/2626429654 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078008de0 con 0x7fc08409e000 2023-09-30T22:17:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7f254adc4700 1 -- 172.21.15.89:0/335578930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409d6f0 msgr2=0x7f254409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7f254adc4700 1 --2- 172.21.15.89:0/335578930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409d6f0 0x7f254409dac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2538009920 tx=0x7f2538009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7f254adc4700 1 -- 172.21.15.89:0/335578930 shutdown_connections 2023-09-30T22:17:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7f254adc4700 1 --2- 172.21.15.89:0/335578930 >> v2:172.21.15.89:3300/0 conn(0x7f254409e410 0x7f2544134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7f254adc4700 1 --2- 172.21.15.89:0/335578930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409d6f0 0x7f254409dac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7f254adc4700 1 -- 172.21.15.89:0/335578930 >> 172.21.15.89:0/335578930 conn(0x7f2544066f50 msgr2=0x7f254409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7fc08a4c7700 1 -- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc08409e000 msgr2=0x7fc084130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7fc08a4c7700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc08409e000 0x7fc084130660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc07800dc60 tx=0x7fc07800df70 comp rx=0 tx=0).stop 2023-09-30T22:17:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7f254adc4700 1 -- 172.21.15.89:0/335578930 shutdown_connections 2023-09-30T22:17:11.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7fc08a4c7700 1 -- 172.21.15.89:0/2626429654 shutdown_connections 2023-09-30T22:17:11.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7fc08a4c7700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc084130ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7fc08a4c7700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7fc08a4c7700 1 --2- 172.21.15.89:0/2626429654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc08409e000 0x7fc084130660 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7fc08a4c7700 1 -- 172.21.15.89:0/2626429654 >> 172.21.15.89:0/2626429654 conn(0x7fc084066f50 msgr2=0x7fc08409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:11.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7f254adc4700 1 -- 172.21.15.89:0/335578930 wait complete. 2023-09-30T22:17:11.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.755+0000 7fc08a4c7700 1 -- 172.21.15.89:0/2626429654 shutdown_connections 2023-09-30T22:17:11.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.756+0000 7fc08a4c7700 1 -- 172.21.15.89:0/2626429654 wait complete. 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.756+0000 7f254adc4700 1 Processor -- start 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.756+0000 7f254adc4700 1 -- start start 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f254adc4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f254409d6f0 0x7f25441abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f254adc4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254409e000 0x7f25441ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f254adc4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409e410 0x7f25441ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f254adc4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25441af4d0 con 0x7f254409e000 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f254adc4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25441ad7c0 con 0x7f254409e410 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f254adc4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f254409c710 con 0x7f254409d6f0 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2549361700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409e410 0x7f25441ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2549361700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409e410 0x7f25441ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56278/0 (socket says 172.21.15.89:56278) 2023-09-30T22:17:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2549361700 1 -- 172.21.15.89:0/1506771846 learned_addr learned my addr 172.21.15.89:0/1506771846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2543fff700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254409e000 0x7f25441ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7fc08a4c7700 1 Processor -- start 2023-09-30T22:17:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2548b60700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f254409d6f0 0x7f25441abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2549361700 1 -- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f254409d6f0 msgr2=0x7f25441abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2549361700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f254409d6f0 0x7f25441abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2549361700 1 -- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254409e000 msgr2=0x7f25441ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2549361700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254409e000 0x7f25441ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2549361700 1 -- 172.21.15.89:0/1506771846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2538009550 con 0x7f254409e410 2023-09-30T22:17:11.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7f2543fff700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254409e000 0x7f25441ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:11.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.757+0000 7fc08a4c7700 1 -- start start 2023-09-30T22:17:11.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7f2549361700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409e410 0x7f25441ac630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b9a0 tx=0x7f253c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:11.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7f25417fa700 1 -- 172.21.15.89:0/1506771846 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c00d850 con 0x7f254409e410 2023-09-30T22:17:11.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc08a4c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08411bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc08a4c7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc08409e000 0x7fc08411c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc08a4c7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc08411c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc08a4c7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08411dab0 con 0x7fc08409e000 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc08a4c7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08411ed00 con 0x7fc08409d6f0 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc08a4c7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc08411f780 con 0x7fc084130ba0 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc083fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08411bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc083fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08411bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56290/0 (socket says 172.21.15.89:56290) 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc083fff700 1 -- 172.21.15.89:0/1431176895 learned_addr learned my addr 172.21.15.89:0/1431176895 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc088a64700 1 --2- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc08411c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7f25417fa700 1 -- 172.21.15.89:0/1506771846 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c00d9d0 con 0x7f254409e410 2023-09-30T22:17:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7f25417fa700 1 -- 172.21.15.89:0/1506771846 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c00dd80 con 0x7f254409e410 2023-09-30T22:17:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25441af130 con 0x7f254409e410 2023-09-30T22:17:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25441afe60 con 0x7f254409e410 2023-09-30T22:17:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc088a64700 1 -- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 msgr2=0x7fc08411bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc088a64700 1 --2- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08411bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc088a64700 1 -- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc08409e000 msgr2=0x7fc08411c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc088a64700 1 --2- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc08409e000 0x7fc08411c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.758+0000 7fc088a64700 1 -- 172.21.15.89:0/1431176895 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc074011d40 con 0x7fc084130ba0 2023-09-30T22:17:11.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.759+0000 7fc088a64700 1 --2- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc08411c920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009f90 tx=0x7fc07c0084e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:11.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.759+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2544074ea0 con 0x7f254409e410 2023-09-30T22:17:11.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.759+0000 7fc080ff9700 1 -- 172.21.15.89:0/1431176895 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c010040 con 0x7fc084130ba0 2023-09-30T22:17:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.759+0000 7fc080ff9700 1 -- 172.21.15.89:0/1431176895 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c003a20 con 0x7fc084130ba0 2023-09-30T22:17:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.759+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c00bb50 con 0x7fc084130ba0 2023-09-30T22:17:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.759+0000 7fc080ff9700 1 -- 172.21.15.89:0/1431176895 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c003db0 con 0x7fc084130ba0 2023-09-30T22:17:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.759+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07800d940 con 0x7fc084130ba0 2023-09-30T22:17:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.760+0000 7fc080ff9700 1 -- 172.21.15.89:0/1431176895 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc07c004020 con 0x7fc084130ba0 2023-09-30T22:17:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.760+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08409c6e0 con 0x7fc084130ba0 2023-09-30T22:17:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.760+0000 7fc080ff9700 1 -- 172.21.15.89:0/1431176895 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc07c040380 con 0x7fc084130ba0 2023-09-30T22:17:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.761+0000 7f25417fa700 1 -- 172.21.15.89:0/1506771846 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f253c00e420 con 0x7f254409e410 2023-09-30T22:17:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.761+0000 7f25417fa700 1 -- 172.21.15.89:0/1506771846 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f253c006e40 con 0x7f254409e410 2023-09-30T22:17:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.761+0000 7f2548b60700 1 -- 172.21.15.89:0/1506771846 >> v1:172.21.15.89:6816/126953 conn(0x7f252802f510 legacy=0x7f25280319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.763+0000 7fc083fff700 1 -- 172.21.15.89:0/1431176895 >> v1:172.21.15.89:6816/126953 conn(0x7fc07002f510 legacy=0x7fc0700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.764+0000 7f25417fa700 1 -- 172.21.15.89:0/1506771846 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f253c0158f0 con 0x7f254409e410 2023-09-30T22:17:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.765+0000 7fc080ff9700 1 -- 172.21.15.89:0/1431176895 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc07c013700 con 0x7fc084130ba0 2023-09-30T22:17:11.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.832+0000 7f8fb5017700 1 -- 172.21.15.89:0/195149596 >> v1:172.21.15.89:6816/126953 conn(0x7f8f9402f510 legacy=0x7f8f940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.882+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8fb00a34c0 con 0x7f8fb009f240 2023-09-30T22:17:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.882+0000 7f8fa5ffb700 1 -- 172.21.15.89:0/195149596 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8fa0021070 con 0x7f8fb009f240 2023-09-30T22:17:11.882 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.884+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 >> v1:172.21.15.89:6816/126953 conn(0x7f8f9402f510 legacy=0x7f8f940319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.884+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009f240 msgr2=0x7f8fb01a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.884+0000 7f8fb727b700 1 --2- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009f240 0x7f8fb01a4240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000ec10 tx=0x7f8fa000ef20 comp rx=0 tx=0).stop 2023-09-30T22:17:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.884+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 shutdown_connections 2023-09-30T22:17:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.884+0000 7f8fb727b700 1 --2- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009f650 0x7f8fb01a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.884+0000 7f8fb727b700 1 --2- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fb009e930 0x7f8fb01a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.884+0000 7f8fb727b700 1 --2- 172.21.15.89:0/195149596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009f240 0x7f8fb01a4240 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.884+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 >> 172.21.15.89:0/195149596 conn(0x7f8fb00a4210 msgr2=0x7f8fb01b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.885+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 shutdown_connections 2023-09-30T22:17:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.885+0000 7f8fb727b700 1 -- 172.21.15.89:0/195149596 wait complete. 2023-09-30T22:17:11.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:11.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.932+0000 7f8386ffd700 1 -- 172.21.15.89:0/1062482778 >> v1:172.21.15.89:6816/126953 conn(0x7f837402f510 legacy=0x7f83740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:11.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.960+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f25441bad20 con 0x7f254409e410 2023-09-30T22:17:11.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.961+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc0840747f0 con 0x7fc084130ba0 2023-09-30T22:17:11.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.961+0000 7f25417fa700 1 -- 172.21.15.89:0/1506771846 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f253c01c020 con 0x7f254409e410 2023-09-30T22:17:11.961 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:11.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.961+0000 7fc080ff9700 1 -- 172.21.15.89:0/1431176895 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc07c00b540 con 0x7fc084130ba0 2023-09-30T22:17:11.961 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:11.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.962+0000 7f2548b60700 1 -- 172.21.15.89:0/1506771846 >> v1:172.21.15.89:6816/126953 conn(0x7f252802f510 legacy=0x7f25280319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:11.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.963+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 >> v1:172.21.15.89:6816/126953 conn(0x7f252802f510 legacy=0x7f25280319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:11.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.963+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f83881208c0 con 0x7f8388136fe0 2023-09-30T22:17:11.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.963+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409e410 msgr2=0x7f25441ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.963+0000 7f254adc4700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409e410 0x7f25441ac630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b9a0 tx=0x7f253c00bcb0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc083fff700 1 -- 172.21.15.89:0/1431176895 >> v1:172.21.15.89:6816/126953 conn(0x7fc07002f510 legacy=0x7fc0700319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:11.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 shutdown_connections 2023-09-30T22:17:11.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7f254adc4700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f254409e410 0x7f25441ac630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7f254adc4700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f254409d6f0 0x7f25441abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7f254adc4700 1 --2- 172.21.15.89:0/1506771846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f254409e000 0x7f25441ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 >> 172.21.15.89:0/1506771846 conn(0x7f2544066f50 msgr2=0x7f25440714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:11.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 shutdown_connections 2023-09-30T22:17:11.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7f254adc4700 1 -- 172.21.15.89:0/1506771846 wait complete. 2023-09-30T22:17:11.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7f836f7fe700 1 -- 172.21.15.89:0/1062482778 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f838001c400 con 0x7f8388136fe0 2023-09-30T22:17:11.965 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:11.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 >> v1:172.21.15.89:6816/126953 conn(0x7fc07002f510 legacy=0x7fc0700319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:11.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 msgr2=0x7fc08411c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 --2- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc08411c920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009f90 tx=0x7fc07c0084e0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 shutdown_connections 2023-09-30T22:17:11.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 --2- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc084130ba0 0x7fc08411c920 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 --2- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc08409d6f0 0x7fc08411bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 --2- 172.21.15.89:0/1431176895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc08409e000 0x7fc08411c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 >> 172.21.15.89:0/1431176895 conn(0x7fc084066f50 msgr2=0x7fc08412fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:11.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 shutdown_connections 2023-09-30T22:17:11.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.964+0000 7fc08a4c7700 1 -- 172.21.15.89:0/1431176895 wait complete. 2023-09-30T22:17:11.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 >> v1:172.21.15.89:6816/126953 conn(0x7f837402f510 legacy=0x7f83740319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:11.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8388136fe0 msgr2=0x7f838811cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:11.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8388136fe0 0x7f838811cfd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f838806fbc0 tx=0x7f8380004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 shutdown_connections 2023-09-30T22:17:11.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8388136fe0 0x7f838811cfd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f838809e930 0x7f838811c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 --2- 172.21.15.89:0/1062482778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f838809f410 0x7f838811ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:11.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 >> 172.21.15.89:0/1062482778 conn(0x7f83880a4210 msgr2=0x7f8388135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:11.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 shutdown_connections 2023-09-30T22:17:11.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:11.966+0000 7f838d799700 1 -- 172.21.15.89:0/1062482778 wait complete. 2023-09-30T22:17:11.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:11.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:11.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:12.898 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:12.980 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:12.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:12.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:13.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.049+0000 7f6bb5bda700 1 Processor -- start 2023-09-30T22:17:13.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.049+0000 7f6bb5bda700 1 -- start start 2023-09-30T22:17:13.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6bb5bda700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6bb01338a0 0x7f6bb0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6bb5bda700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009e930 0x7f6bb013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6bb5bda700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb006ef30 con 0x7f6bb009de50 2023-09-30T22:17:13.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6bb5bda700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb006fef0 con 0x7f6bb01338a0 2023-09-30T22:17:13.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6bb5bda700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01334b0 con 0x7f6bb009e930 2023-09-30T22:17:13.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6baffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009e930 0x7f6bb013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6baf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6bb01338a0 0x7f6bb0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6baffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009e930 0x7f6bb013b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57478/0 (socket says 172.21.15.89:57478) 2023-09-30T22:17:13.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6baffff700 1 -- 172.21.15.89:0/2331714118 learned_addr learned my addr 172.21.15.89:0/2331714118 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:13.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.050+0000 7f6baf7fe700 1 --2- 172.21.15.89:0/2331714118 >> v2:172.21.15.89:3300/0 conn(0x7f6bb01338a0 0x7f6bb0133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43352/0 (socket says 172.21.15.89:43352) 2023-09-30T22:17:13.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6baffff700 1 -- 172.21.15.89:0/2331714118 >> v2:172.21.15.89:3300/0 conn(0x7f6bb01338a0 msgr2=0x7f6bb0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6baffff700 1 --2- 172.21.15.89:0/2331714118 >> v2:172.21.15.89:3300/0 conn(0x7f6bb01338a0 0x7f6bb0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6baffff700 1 -- 172.21.15.89:0/2331714118 >> v1:172.21.15.89:6789/0 conn(0x7f6bb009de50 legacy=0x7f6bb009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6baffff700 1 -- 172.21.15.89:0/2331714118 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb006fc30 con 0x7f6bb009e930 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6baf7fe700 1 --2- 172.21.15.89:0/2331714118 >> v2:172.21.15.89:3300/0 conn(0x7f6bb01338a0 0x7f6bb0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6baffff700 1 --2- 172.21.15.89:0/2331714118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009e930 0x7f6bb013b740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009e50 tx=0x7f6ba800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=686114ddd9bf13d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2331714118 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800e070 con 0x7f6bb009e930 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2331714118 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba800b910 con 0x7f6bb009e930 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.051+0000 7f6bae7fc700 1 -- 172.21.15.89:0/2331714118 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800bcc0 con 0x7f6bb009e930 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.052+0000 7f6bb5bda700 1 -- 172.21.15.89:0/2331714118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009e930 msgr2=0x7f6bb013b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.052+0000 7f6bb5bda700 1 --2- 172.21.15.89:0/2331714118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009e930 0x7f6bb013b740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009e50 tx=0x7f6ba800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.052+0000 7f6bb5bda700 1 -- 172.21.15.89:0/2331714118 shutdown_connections 2023-09-30T22:17:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.052+0000 7f6bb5bda700 1 --2- 172.21.15.89:0/2331714118 >> v2:172.21.15.89:3300/0 conn(0x7f6bb01338a0 0x7f6bb0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.052+0000 7f6bb5bda700 1 --2- 172.21.15.89:0/2331714118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009e930 0x7f6bb013b740 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.052+0000 7f6bb5bda700 1 -- 172.21.15.89:0/2331714118 >> 172.21.15.89:0/2331714118 conn(0x7f6bb00a2aa0 msgr2=0x7f6bb00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.052+0000 7f6bb5bda700 1 -- 172.21.15.89:0/2331714118 shutdown_connections 2023-09-30T22:17:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.052+0000 7f6bb5bda700 1 -- 172.21.15.89:0/2331714118 wait complete. 2023-09-30T22:17:13.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.054+0000 7f6bb5bda700 1 Processor -- start 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.054+0000 7f6bb5bda700 1 -- start start 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6bb5bda700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009de50 0x7f6bb011d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6bb5bda700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bb009e930 0x7f6bb011dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6bb5bda700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bb01338a0 0x7f6bb011e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6bb5bda700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb0121260 con 0x7f6bb01338a0 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6bb5bda700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb011f930 con 0x7f6bb009e930 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6bb5bda700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01209f0 con 0x7f6bb009de50 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baeffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bb009e930 0x7f6bb011dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009de50 0x7f6bb011d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baeffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bb009e930 0x7f6bb011dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56296/0 (socket says 172.21.15.89:56296) 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baeffd700 1 -- 172.21.15.89:0/1856426163 learned_addr learned my addr 172.21.15.89:0/1856426163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baffff700 1 --2- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bb01338a0 0x7f6bb011e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baf7fe700 1 -- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bb009e930 msgr2=0x7f6bb011dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baf7fe700 1 --2- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bb009e930 0x7f6bb011dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baf7fe700 1 -- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bb01338a0 msgr2=0x7f6bb011e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baf7fe700 1 --2- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bb01338a0 0x7f6bb011e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.055+0000 7f6baf7fe700 1 -- 172.21.15.89:0/1856426163 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba00094a0 con 0x7f6bb009de50 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.056+0000 7f6baf7fe700 1 --2- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009de50 0x7f6bb011d790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000e9e0 tx=0x7f6ba000ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.056+0000 7f6b97fff700 1 -- 172.21.15.89:0/1856426163 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000ee80 con 0x7f6bb009de50 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.056+0000 7f6b97fff700 1 -- 172.21.15.89:0/1856426163 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba000c850 con 0x7f6bb009de50 2023-09-30T22:17:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.056+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba8009b30 con 0x7f6bb009de50 2023-09-30T22:17:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.056+0000 7f6b97fff700 1 -- 172.21.15.89:0/1856426163 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000cc00 con 0x7f6bb009de50 2023-09-30T22:17:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.056+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb006f690 con 0x7f6bb009de50 2023-09-30T22:17:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.057+0000 7f6b97fff700 1 -- 172.21.15.89:0/1856426163 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6ba000cd60 con 0x7f6bb009de50 2023-09-30T22:17:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.057+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb0120de0 con 0x7f6bb009de50 2023-09-30T22:17:13.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.057+0000 7f6baeffd700 1 -- 172.21.15.89:0/1856426163 >> v1:172.21.15.89:6816/126953 conn(0x7f6b9c02f510 legacy=0x7f6b9c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:13.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.057+0000 7f6b97fff700 1 -- 172.21.15.89:0/1856426163 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f6ba003ff50 con 0x7f6bb009de50 2023-09-30T22:17:13.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.062+0000 7f6b97fff700 1 -- 172.21.15.89:0/1856426163 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bb0120de0 con 0x7f6bb009de50 2023-09-30T22:17:13.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.114+0000 7f2e2cb48700 1 Processor -- start 2023-09-30T22:17:13.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.114+0000 7f2e2cb48700 1 -- start start 2023-09-30T22:17:13.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2cb48700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e2809d6f0 0x7f2e2809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2cb48700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 0x7f2e28130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2cb48700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2809f130 con 0x7f2e2809e000 2023-09-30T22:17:13.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2cb48700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2809f520 con 0x7f2e2809d6f0 2023-09-30T22:17:13.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2cb48700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2809e930 con 0x7f2e2809e410 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e2809d6f0 0x7f2e2809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e25d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2e2809e000 0x7f2e28130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52778/0 (socket says 172.21.15.89:52778) 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e25d9b700 1 -- 172.21.15.89:0/963410991 learned_addr learned my addr 172.21.15.89:0/963410991 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e26d9d700 1 --2- 172.21.15.89:0/963410991 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 0x7f2e28130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2659c700 1 -- 172.21.15.89:0/963410991 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 msgr2=0x7f2e28130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2659c700 1 --2- 172.21.15.89:0/963410991 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 0x7f2e28130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2659c700 1 -- 172.21.15.89:0/963410991 >> v1:172.21.15.89:6789/0 conn(0x7f2e2809e000 legacy=0x7f2e28130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.115+0000 7f2e2659c700 1 -- 172.21.15.89:0/963410991 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2809b2d0 con 0x7f2e2809d6f0 2023-09-30T22:17:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.116+0000 7f2e2659c700 1 --2- 172.21.15.89:0/963410991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809d6f0 0x7f2e2809dac0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009840 tx=0x7f2e18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96262808edb99b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:13.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.116+0000 7fbfdb257700 1 Processor -- start 2023-09-30T22:17:13.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.116+0000 7fbfdb257700 1 -- start start 2023-09-30T22:17:13.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfdb257700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbfd40a0a30 0x7fbfd40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfdb257700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a1340 0x7fbfd4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfdb257700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 0x7fbfd4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfdb257700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd409edd0 con 0x7fbfd40a0a30 2023-09-30T22:17:13.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfdb257700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd409e1f0 con 0x7fbfd4133fa0 2023-09-30T22:17:13.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfdb257700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd409dc50 con 0x7fbfd40a1340 2023-09-30T22:17:13.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7f2e2559a700 1 -- 172.21.15.89:0/963410991 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18011830 con 0x7f2e2809d6f0 2023-09-30T22:17:13.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbfd40a0a30 0x7fbfd40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbfd40a0a30 0x7fbfd40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43382/0 (socket says 172.21.15.89:43382) 2023-09-30T22:17:13.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 -- 172.21.15.89:0/2998974913 learned_addr learned my addr 172.21.15.89:0/2998974913 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:13.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7f2e2559a700 1 -- 172.21.15.89:0/963410991 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e18011990 con 0x7f2e2809d6f0 2023-09-30T22:17:13.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7f2e2559a700 1 -- 172.21.15.89:0/963410991 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18011d20 con 0x7f2e2809d6f0 2023-09-30T22:17:13.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd97f4700 1 --2- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 0x7fbfd4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 -- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a1340 msgr2=0x7fbfd4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 --2- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a1340 0x7fbfd4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 -- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 msgr2=0x7fbfd4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 --2- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 0x7fbfd4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 -- 172.21.15.89:0/2998974913 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40a1fe0 con 0x7fbfd40a0a30 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd8ff3700 1 --2- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd40a0e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80098f0 tx=0x7fbfc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa33c542d1016a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd37fe700 1 -- 172.21.15.89:0/2998974913 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800b8a0 con 0x7fbfd40a0a30 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd37fe700 1 -- 172.21.15.89:0/2998974913 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc800ba00 con 0x7fbfd40a0a30 2023-09-30T22:17:13.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.117+0000 7fbfd37fe700 1 -- 172.21.15.89:0/2998974913 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800bd90 con 0x7fbfd40a0a30 2023-09-30T22:17:13.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7f2e2cb48700 1 -- 172.21.15.89:0/963410991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809d6f0 msgr2=0x7f2e2809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7f2e2cb48700 1 --2- 172.21.15.89:0/963410991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809d6f0 0x7f2e2809dac0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009840 tx=0x7f2e18009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:13.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7f2e2cb48700 1 -- 172.21.15.89:0/963410991 shutdown_connections 2023-09-30T22:17:13.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7f2e2cb48700 1 --2- 172.21.15.89:0/963410991 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 0x7f2e28130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7f2e2cb48700 1 --2- 172.21.15.89:0/963410991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809d6f0 0x7f2e2809dac0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7f2e2cb48700 1 -- 172.21.15.89:0/963410991 >> 172.21.15.89:0/963410991 conn(0x7f2e28066f50 msgr2=0x7f2e2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:13.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7f2e2cb48700 1 -- 172.21.15.89:0/963410991 shutdown_connections 2023-09-30T22:17:13.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7f2e2cb48700 1 -- 172.21.15.89:0/963410991 wait complete. 2023-09-30T22:17:13.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7fbfdb257700 1 -- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 msgr2=0x7fbfd40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.118+0000 7fbfdb257700 1 --2- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd40a0e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80098f0 tx=0x7fbfc8009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:13.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.119+0000 7fbfdb257700 1 -- 172.21.15.89:0/2998974913 shutdown_connections 2023-09-30T22:17:13.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.119+0000 7fbfdb257700 1 --2- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 0x7fbfd4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.119+0000 7fbfdb257700 1 --2- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd40a0e00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.119+0000 7fbfdb257700 1 --2- 172.21.15.89:0/2998974913 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a1340 0x7fbfd4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.119+0000 7fbfdb257700 1 -- 172.21.15.89:0/2998974913 >> 172.21.15.89:0/2998974913 conn(0x7fbfd40a6550 msgr2=0x7fbfd40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.119+0000 7f2e2cb48700 1 Processor -- start 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.119+0000 7f2e2cb48700 1 -- start start 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2cb48700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2809d6f0 0x7f2e281c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2cb48700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809e000 0x7f2e281c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2cb48700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 0x7f2e281c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2cb48700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28132460 con 0x7f2e2809e000 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2cb48700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28072350 con 0x7f2e2809e410 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2cb48700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2809c700 con 0x7f2e2809d6f0 2023-09-30T22:17:13.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2809d6f0 0x7f2e281c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2809d6f0 0x7f2e281c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57506/0 (socket says 172.21.15.89:57506) 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e2659c700 1 -- 172.21.15.89:0/1561562767 learned_addr learned my addr 172.21.15.89:0/1561562767 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e26d9d700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 0x7f2e281c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.120+0000 7f2e25d9b700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809e000 0x7f2e281c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e2659c700 1 -- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 msgr2=0x7f2e281c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e2659c700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 0x7f2e281c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e2659c700 1 -- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809e000 msgr2=0x7f2e281c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e2659c700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809e000 0x7f2e281c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e2659c700 1 -- 172.21.15.89:0/1561562767 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e180094a0 con 0x7f2e2809d6f0 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e2659c700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2809d6f0 0x7f2e281c50c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009fd0 tx=0x7f2e18004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e16ffd700 1 -- 172.21.15.89:0/1561562767 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e180036a0 con 0x7f2e2809d6f0 2023-09-30T22:17:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e16ffd700 1 -- 172.21.15.89:0/1561562767 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1801f070 con 0x7f2e2809d6f0 2023-09-30T22:17:13.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e28131c60 con 0x7f2e2809d6f0 2023-09-30T22:17:13.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e16ffd700 1 -- 172.21.15.89:0/1561562767 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18003a60 con 0x7f2e2809d6f0 2023-09-30T22:17:13.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.121+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e28130730 con 0x7f2e2809d6f0 2023-09-30T22:17:13.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.122+0000 7f0204b78700 1 Processor -- start 2023-09-30T22:17:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.122+0000 7f2e16ffd700 1 -- 172.21.15.89:0/1561562767 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2e18003bc0 con 0x7f2e2809d6f0 2023-09-30T22:17:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.122+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28074ea0 con 0x7f2e2809d6f0 2023-09-30T22:17:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.122+0000 7f2e25d9b700 1 -- 172.21.15.89:0/1561562767 >> v1:172.21.15.89:6816/126953 conn(0x7f2e0802f510 legacy=0x7f2e080319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.122+0000 7f2e16ffd700 1 -- 172.21.15.89:0/1561562767 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f2e18021070 con 0x7f2e2809d6f0 2023-09-30T22:17:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.122+0000 7f0204b78700 1 -- start start 2023-09-30T22:17:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f0204b78700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f020006e570 0x7f020006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f0204b78700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020009eca0 0x7f020006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f0204b78700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0200134610 0x7f0200134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f0204b78700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0200070ee0 con 0x7f0200134610 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f0204b78700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0200071480 con 0x7f020006e570 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f0204b78700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f020006f3c0 con 0x7f020009eca0 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f020006e570 0x7f020006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f020006e570 0x7f020006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56352/0 (socket says 172.21.15.89:56352) 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 -- 172.21.15.89:0/306910527 learned_addr learned my addr 172.21.15.89:0/306910527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fed9d700 1 --2- 172.21.15.89:0/306910527 >> v2:172.21.15.89:3300/0 conn(0x7f0200134610 0x7f0200134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 -- 172.21.15.89:0/306910527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020009eca0 msgr2=0x7f020006ee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 --2- 172.21.15.89:0/306910527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020009eca0 0x7f020006ee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 -- 172.21.15.89:0/306910527 >> v2:172.21.15.89:3300/0 conn(0x7f0200134610 msgr2=0x7f0200134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 --2- 172.21.15.89:0/306910527 >> v2:172.21.15.89:3300/0 conn(0x7f0200134610 0x7f0200134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 -- 172.21.15.89:0/306910527 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0200071b90 con 0x7f020006e570 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fe59c700 1 --2- 172.21.15.89:0/306910527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f020006e570 0x7f020006e940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f01f40098f0 tx=0x7f01f4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5decc665cff8f26d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fd59a700 1 -- 172.21.15.89:0/306910527 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400b8a0 con 0x7f020006e570 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fd59a700 1 -- 172.21.15.89:0/306910527 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f400ba00 con 0x7f020006e570 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.123+0000 7f01fd59a700 1 -- 172.21.15.89:0/306910527 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400bd90 con 0x7f020006e570 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 -- 172.21.15.89:0/306910527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f020006e570 msgr2=0x7f020006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 --2- 172.21.15.89:0/306910527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f020006e570 0x7f020006e940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f01f40098f0 tx=0x7f01f4009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 -- 172.21.15.89:0/306910527 shutdown_connections 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 --2- 172.21.15.89:0/306910527 >> v2:172.21.15.89:3300/0 conn(0x7f0200134610 0x7f0200134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 --2- 172.21.15.89:0/306910527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f020006e570 0x7f020006e940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 --2- 172.21.15.89:0/306910527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020009eca0 0x7f020006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 -- 172.21.15.89:0/306910527 >> 172.21.15.89:0/306910527 conn(0x7f02000a2aa0 msgr2=0x7f02000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 -- 172.21.15.89:0/306910527 shutdown_connections 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.124+0000 7f0204b78700 1 -- 172.21.15.89:0/306910527 wait complete. 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.125+0000 7f0204b78700 1 Processor -- start 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.125+0000 7f0204b78700 1 -- start start 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f0204b78700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020006e570 0x7f02001214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f0204b78700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0200134610 0x7f0200121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f0204b78700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200124ee0 0x7f0200121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f0204b78700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02001247a0 con 0x7f0200124ee0 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f0204b78700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02001342b0 con 0x7f0200134610 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f0204b78700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02001230f0 con 0x7f020006e570 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fe59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020006e570 0x7f02001214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fe59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020006e570 0x7f02001214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57518/0 (socket says 172.21.15.89:57518) 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fe59c700 1 -- 172.21.15.89:0/1752375125 learned_addr learned my addr 172.21.15.89:0/1752375125 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fed9d700 1 --2- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200124ee0 0x7f0200121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fe59c700 1 -- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0200134610 msgr2=0x7f0200121a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fe59c700 1 --2- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0200134610 0x7f0200121a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fe59c700 1 -- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200124ee0 msgr2=0x7f0200121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fe59c700 1 --2- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200124ee0 0x7f0200121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.126+0000 7f01fe59c700 1 -- 172.21.15.89:0/1752375125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4009550 con 0x7f020006e570 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.125+0000 7fbfdb257700 1 -- 172.21.15.89:0/2998974913 shutdown_connections 2023-09-30T22:17:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.127+0000 7fbfdb257700 1 -- 172.21.15.89:0/2998974913 wait complete. 2023-09-30T22:17:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.127+0000 7f01fe59c700 1 --2- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020006e570 0x7f02001214e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009c40 tx=0x7f01f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.127+0000 7f01eeffd700 1 -- 172.21.15.89:0/1752375125 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4004380 con 0x7f020006e570 2023-09-30T22:17:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.127+0000 7f01eeffd700 1 -- 172.21.15.89:0/1752375125 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f401f070 con 0x7f020006e570 2023-09-30T22:17:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.127+0000 7f01eeffd700 1 -- 172.21.15.89:0/1752375125 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40036a0 con 0x7f020006e570 2023-09-30T22:17:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.127+0000 7f2e16ffd700 1 -- 172.21.15.89:0/1561562767 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e18006bd0 con 0x7f2e2809d6f0 2023-09-30T22:17:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.127+0000 7fbfdb257700 1 Processor -- start 2023-09-30T22:17:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.127+0000 7fbfdb257700 1 -- start start 2023-09-30T22:17:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.128+0000 7f0204b78700 1 -- 172.21.15.89:0/1752375125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0200123d20 con 0x7f020006e570 2023-09-30T22:17:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.128+0000 7f0204b78700 1 -- 172.21.15.89:0/1752375125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0200070610 con 0x7f020006e570 2023-09-30T22:17:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.128+0000 7fbfdb257700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd41200f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.128+0000 7fbfdb257700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a1340 0x7fbfd4120630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.128+0000 7fbfdb257700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 0x7fbfd4120b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.128+0000 7fbfdb257700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4123620 con 0x7fbfd40a0a30 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.128+0000 7fbfdb257700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4122f70 con 0x7fbfd4133fa0 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.128+0000 7fbfdb257700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40a0220 con 0x7fbfd40a1340 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd41200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd41200f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43420/0 (socket says 172.21.15.89:43420) 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 -- 172.21.15.89:0/3991216068 learned_addr learned my addr 172.21.15.89:0/3991216068 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd97f4700 1 --2- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 0x7fbfd4120b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 -- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a1340 msgr2=0x7fbfd4120630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 --2- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a1340 0x7fbfd4120630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 -- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 msgr2=0x7fbfd4120b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 --2- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 0x7fbfd4120b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 -- 172.21.15.89:0/3991216068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8009550 con 0x7fbfd40a0a30 2023-09-30T22:17:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.129+0000 7fbfd8ff3700 1 --2- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd41200f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009c40 tx=0x7fbfc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7f01eeffd700 1 -- 172.21.15.89:0/1752375125 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f01f4021030 con 0x7f020006e570 2023-09-30T22:17:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7f01ecff9700 1 -- 172.21.15.89:0/1752375125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d80052e0 con 0x7f020006e570 2023-09-30T22:17:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7f01eeffd700 1 -- 172.21.15.89:0/1752375125 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f01f4041110 con 0x7f020006e570 2023-09-30T22:17:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7f01fdd9b700 1 -- 172.21.15.89:0/1752375125 >> v1:172.21.15.89:6816/126953 conn(0x7f01e002f560 legacy=0x7f01e0031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7fbfd17fa700 1 -- 172.21.15.89:0/3991216068 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8004300 con 0x7fbfd40a0a30 2023-09-30T22:17:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd4121dc0 con 0x7fbfd40a0a30 2023-09-30T22:17:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7fbfd17fa700 1 -- 172.21.15.89:0/3991216068 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc801f070 con 0x7fbfd40a0a30 2023-09-30T22:17:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7fbfd17fa700 1 -- 172.21.15.89:0/3991216068 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80036a0 con 0x7fbfd40a0a30 2023-09-30T22:17:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.130+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4123bc0 con 0x7fbfd40a0a30 2023-09-30T22:17:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.131+0000 7fbfd17fa700 1 -- 172.21.15.89:0/3991216068 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbfc8021030 con 0x7fbfd40a0a30 2023-09-30T22:17:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.131+0000 7fbfd17fa700 1 -- 172.21.15.89:0/3991216068 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fbfc8040fd0 con 0x7fbfd40a0a30 2023-09-30T22:17:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.131+0000 7fbfd3fff700 1 -- 172.21.15.89:0/3991216068 >> v1:172.21.15.89:6816/126953 conn(0x7fbfb802f510 legacy=0x7fbfb80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.132+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd4123790 con 0x7fbfd40a0a30 2023-09-30T22:17:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.137+0000 7fbfd17fa700 1 -- 172.21.15.89:0/3991216068 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbfc8015020 con 0x7fbfd40a0a30 2023-09-30T22:17:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.139+0000 7f01eeffd700 1 -- 172.21.15.89:0/1752375125 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f01f40199a0 con 0x7f020006e570 2023-09-30T22:17:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.258+0000 7f6baeffd700 1 -- 172.21.15.89:0/1856426163 >> v1:172.21.15.89:6816/126953 conn(0x7f6b9c02f510 legacy=0x7f6b9c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:13.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.281+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6bb0120de0 con 0x7f6bb009de50 2023-09-30T22:17:13.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.281+0000 7f6b97fff700 1 -- 172.21.15.89:0/1856426163 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bb0120de0 con 0x7f6bb009de50 2023-09-30T22:17:13.281 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 >> v1:172.21.15.89:6816/126953 conn(0x7f6b9c02f510 legacy=0x7f6b9c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009de50 msgr2=0x7f6bb011d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 --2- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009de50 0x7f6bb011d790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000e9e0 tx=0x7f6ba000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 shutdown_connections 2023-09-30T22:17:13.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 --2- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bb01338a0 0x7f6bb011e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 --2- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009de50 0x7f6bb011d790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 --2- 172.21.15.89:0/1856426163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bb009e930 0x7f6bb011dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 >> 172.21.15.89:0/1856426163 conn(0x7f6bb00a2aa0 msgr2=0x7f6bb01b9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 shutdown_connections 2023-09-30T22:17:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.283+0000 7f6bb5bda700 1 -- 172.21.15.89:0/1856426163 wait complete. 2023-09-30T22:17:13.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:13.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.323+0000 7f2e25d9b700 1 -- 172.21.15.89:0/1561562767 >> v1:172.21.15.89:6816/126953 conn(0x7f2e0802f510 legacy=0x7f2e080319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:13.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.327+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2e28132680 con 0x7f2e2809d6f0 2023-09-30T22:17:13.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.327+0000 7f2e16ffd700 1 -- 172.21.15.89:0/1561562767 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e1802b090 con 0x7f2e2809d6f0 2023-09-30T22:17:13.327 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:13.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.330+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 >> v1:172.21.15.89:6816/126953 conn(0x7f2e0802f510 legacy=0x7f2e080319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:13.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2809d6f0 msgr2=0x7f2e281c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2809d6f0 0x7f2e281c50c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009fd0 tx=0x7f2e18004750 comp rx=0 tx=0).stop 2023-09-30T22:17:13.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f01fdd9b700 1 -- 172.21.15.89:0/1752375125 >> v1:172.21.15.89:6816/126953 conn(0x7f01e002f560 legacy=0x7f01e0031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:13.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 shutdown_connections 2023-09-30T22:17:13.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2809e410 0x7f2e281c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2809d6f0 0x7f2e281c50c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 --2- 172.21.15.89:0/1561562767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2809e000 0x7f2e281c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 >> 172.21.15.89:0/1561562767 conn(0x7f2e28066f50 msgr2=0x7f2e280a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:13.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 shutdown_connections 2023-09-30T22:17:13.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.331+0000 7f2e2cb48700 1 -- 172.21.15.89:0/1561562767 wait complete. 2023-09-30T22:17:13.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.332+0000 7fbfd3fff700 1 -- 172.21.15.89:0/3991216068 >> v1:172.21.15.89:6816/126953 conn(0x7fbfb802f510 legacy=0x7fbfb80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:13.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.334+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbfd4074ea0 con 0x7fbfd40a0a30 2023-09-30T22:17:13.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.334+0000 7fbfd17fa700 1 -- 172.21.15.89:0/3991216068 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbfc8023990 con 0x7fbfd40a0a30 2023-09-30T22:17:13.334 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 >> v1:172.21.15.89:6816/126953 conn(0x7fbfb802f510 legacy=0x7fbfb80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 msgr2=0x7fbfd41200f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 --2- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd41200f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009c40 tx=0x7fbfc8004750 comp rx=0 tx=0).stop 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 shutdown_connections 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 --2- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd4133fa0 0x7fbfd4120b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 --2- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0a30 0x7fbfd41200f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 --2- 172.21.15.89:0/3991216068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a1340 0x7fbfd4120630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 >> 172.21.15.89:0/3991216068 conn(0x7fbfd40a6550 msgr2=0x7fbfd41bb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 shutdown_connections 2023-09-30T22:17:13.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.338+0000 7fbfdb257700 1 -- 172.21.15.89:0/3991216068 wait complete. 2023-09-30T22:17:13.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.348+0000 7f01ecff9700 1 -- 172.21.15.89:0/1752375125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f01d8005860 con 0x7f020006e570 2023-09-30T22:17:13.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.348+0000 7f01eeffd700 1 -- 172.21.15.89:0/1752375125 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f01f4015070 con 0x7f020006e570 2023-09-30T22:17:13.349 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.350+0000 7f01ecff9700 1 -- 172.21.15.89:0/1752375125 >> v1:172.21.15.89:6816/126953 conn(0x7f01e002f560 legacy=0x7f01e0031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.350+0000 7f01ecff9700 1 -- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020006e570 msgr2=0x7f02001214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.350+0000 7f01ecff9700 1 --2- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020006e570 0x7f02001214e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009c40 tx=0x7f01f4004750 comp rx=0 tx=0).stop 2023-09-30T22:17:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.350+0000 7f01ecff9700 1 -- 172.21.15.89:0/1752375125 shutdown_connections 2023-09-30T22:17:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.350+0000 7f01ecff9700 1 --2- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0200124ee0 0x7f0200121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.350+0000 7f01ecff9700 1 --2- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020006e570 0x7f02001214e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.350+0000 7f01ecff9700 1 --2- 172.21.15.89:0/1752375125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0200134610 0x7f0200121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.351+0000 7f01ecff9700 1 -- 172.21.15.89:0/1752375125 >> 172.21.15.89:0/1752375125 conn(0x7f02000a2aa0 msgr2=0x7f02001bf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.351+0000 7f01ecff9700 1 -- 172.21.15.89:0/1752375125 shutdown_connections 2023-09-30T22:17:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:13.351+0000 7f01ecff9700 1 -- 172.21.15.89:0/1752375125 wait complete. 2023-09-30T22:17:13.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:13.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:14.296 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:14.350 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:14.354 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:14.363 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.447+0000 7f595811e700 1 Processor -- start 2023-09-30T22:17:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.447+0000 7f595811e700 1 -- start start 2023-09-30T22:17:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.448+0000 7f595811e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130540 0x7f5950130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.448+0000 7f595811e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f595006fcf0 0x7f5950138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.448+0000 7f595811e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500716a0 con 0x7f5950130e50 2023-09-30T22:17:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.448+0000 7f595811e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595009a390 con 0x7f5950130540 2023-09-30T22:17:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.448+0000 7f595811e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5950071290 con 0x7f595006fcf0 2023-09-30T22:17:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.448+0000 7f5955eba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130540 0x7f5950130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.448+0000 7f5955eba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130540 0x7f5950130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56362/0 (socket says 172.21.15.89:56362) 2023-09-30T22:17:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.448+0000 7f5955eba700 1 -- 172.21.15.89:0/2540624319 learned_addr learned my addr 172.21.15.89:0/2540624319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f59566bb700 1 --2- 172.21.15.89:0/2540624319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f595006fcf0 0x7f5950138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f5955eba700 1 -- 172.21.15.89:0/2540624319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f595006fcf0 msgr2=0x7f5950138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f5955eba700 1 --2- 172.21.15.89:0/2540624319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f595006fcf0 0x7f5950138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f5955eba700 1 -- 172.21.15.89:0/2540624319 >> v1:172.21.15.89:6789/0 conn(0x7f5950130e50 legacy=0x7f595006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f5955eba700 1 -- 172.21.15.89:0/2540624319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950130340 con 0x7f5950130540 2023-09-30T22:17:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f5955eba700 1 --2- 172.21.15.89:0/2540624319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130540 0x7f5950130910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f594c009920 tx=0x7f594c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564901504ccac41 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f5954eb8700 1 -- 172.21.15.89:0/2540624319 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c012b80 con 0x7f5950130540 2023-09-30T22:17:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f5954eb8700 1 -- 172.21.15.89:0/2540624319 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c012ce0 con 0x7f5950130540 2023-09-30T22:17:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.449+0000 7f5954eb8700 1 -- 172.21.15.89:0/2540624319 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c01b620 con 0x7f5950130540 2023-09-30T22:17:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.450+0000 7f595811e700 1 -- 172.21.15.89:0/2540624319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130540 msgr2=0x7f5950130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.450+0000 7f595811e700 1 --2- 172.21.15.89:0/2540624319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130540 0x7f5950130910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f594c009920 tx=0x7f594c009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.450+0000 7f595811e700 1 -- 172.21.15.89:0/2540624319 shutdown_connections 2023-09-30T22:17:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.450+0000 7f595811e700 1 --2- 172.21.15.89:0/2540624319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130540 0x7f5950130910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.450+0000 7f595811e700 1 --2- 172.21.15.89:0/2540624319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f595006fcf0 0x7f5950138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.450+0000 7f595811e700 1 -- 172.21.15.89:0/2540624319 >> 172.21.15.89:0/2540624319 conn(0x7f59500a2aa0 msgr2=0x7f59500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.450+0000 7f595811e700 1 -- 172.21.15.89:0/2540624319 shutdown_connections 2023-09-30T22:17:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.450+0000 7f595811e700 1 -- 172.21.15.89:0/2540624319 wait complete. 2023-09-30T22:17:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.452+0000 7f595811e700 1 Processor -- start 2023-09-30T22:17:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.452+0000 7f595811e700 1 -- start start 2023-09-30T22:17:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.452+0000 7f595811e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f595006fcf0 0x7f595009ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.452+0000 7f595811e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5950130540 0x7f595009d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.452+0000 7f595811e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130e50 0x7f595009d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.452+0000 7f595811e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500718e0 con 0x7f595006fcf0 2023-09-30T22:17:14.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.452+0000 7f595811e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500a0e20 con 0x7f5950130e50 2023-09-30T22:17:14.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.452+0000 7f595811e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f595011a980 con 0x7f5950130540 2023-09-30T22:17:14.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f595006fcf0 0x7f595009ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f59566bb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130e50 0x7f595009d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f595006fcf0 0x7f595009ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43430/0 (socket says 172.21.15.89:43430) 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 -- 172.21.15.89:0/3437906468 learned_addr learned my addr 172.21.15.89:0/3437906468 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f59556b9700 1 --2- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5950130540 0x7f595009d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 -- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5950130540 msgr2=0x7f595009d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 --2- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5950130540 0x7f595009d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 -- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130e50 msgr2=0x7f595009d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 --2- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130e50 0x7f595009d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 -- 172.21.15.89:0/3437906468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c009550 con 0x7f595006fcf0 2023-09-30T22:17:14.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f5955eba700 1 --2- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f595006fcf0 0x7f595009ce50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f594c000c00 tx=0x7f594c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f59467fc700 1 -- 172.21.15.89:0/3437906468 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c020070 con 0x7f595006fcf0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f59467fc700 1 -- 172.21.15.89:0/3437906468 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c0038b0 con 0x7f595006fcf0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5950119120 con 0x7f595006fcf0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.453+0000 7f59467fc700 1 -- 172.21.15.89:0/3437906468 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c003c20 con 0x7f595006fcf0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.454+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595011af20 con 0x7f595006fcf0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.455+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950074ea0 con 0x7f595006fcf0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.455+0000 7f59467fc700 1 -- 172.21.15.89:0/3437906468 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f594c01b400 con 0x7f595006fcf0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.455+0000 7f59467fc700 1 -- 172.21.15.89:0/3437906468 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f594c044880 con 0x7f595006fcf0 2023-09-30T22:17:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.456+0000 7f59556b9700 1 -- 172.21.15.89:0/3437906468 >> v1:172.21.15.89:6816/126953 conn(0x7f593402f510 legacy=0x7f59340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.464+0000 7f59467fc700 1 -- 172.21.15.89:0/3437906468 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f594c01b6b0 con 0x7f595006fcf0 2023-09-30T22:17:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.492+0000 7fdc77d33700 1 Processor -- start 2023-09-30T22:17:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.492+0000 7fdc77d33700 1 -- start start 2023-09-30T22:17:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc77d33700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70136710 0x7fdc70138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc77d33700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc70139030 0x7fdc7013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc77d33700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7006ef30 con 0x7fdc701338a0 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc77d33700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7006fef0 con 0x7fdc70139030 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc77d33700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701334b0 con 0x7fdc70136710 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc75acf700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdc701338a0 0x7fdc70133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52800/0 (socket says 172.21.15.89:52800) 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc75acf700 1 -- 172.21.15.89:0/2318223695 learned_addr learned my addr 172.21.15.89:0/2318223695 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc762d0700 1 --2- 172.21.15.89:0/2318223695 >> v2:172.21.15.89:3300/0 conn(0x7fdc70139030 0x7fdc7013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc752ce700 1 --2- 172.21.15.89:0/2318223695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70136710 0x7fdc70138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc74acd700 1 -- 172.21.15.89:0/2318223695 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265258539 0 0) 0x7fdc7006ef30 con 0x7fdc701338a0 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.493+0000 7fdc74acd700 1 -- 172.21.15.89:0/2318223695 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc5c0035b0 con 0x7fdc701338a0 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.494+0000 7fdc762d0700 1 -- 172.21.15.89:0/2318223695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70136710 msgr2=0x7fdc70138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.494+0000 7fdc762d0700 1 --2- 172.21.15.89:0/2318223695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70136710 0x7fdc70138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.494+0000 7fdc762d0700 1 -- 172.21.15.89:0/2318223695 >> v1:172.21.15.89:6789/0 conn(0x7fdc701338a0 legacy=0x7fdc70133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.494+0000 7fdc762d0700 1 -- 172.21.15.89:0/2318223695 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7006fc30 con 0x7fdc70139030 2023-09-30T22:17:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.494+0000 7fdc762d0700 1 --2- 172.21.15.89:0/2318223695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70139030 0x7fdc7013b410 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009d90 tx=0x7fdc6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cb80efc3d1e5b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.494+0000 7fdc74acd700 1 -- 172.21.15.89:0/2318223695 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6800e070 con 0x7fdc70139030 2023-09-30T22:17:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.494+0000 7fdc74acd700 1 -- 172.21.15.89:0/2318223695 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6800bb20 con 0x7fdc70139030 2023-09-30T22:17:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.494+0000 7fdc74acd700 1 -- 172.21.15.89:0/2318223695 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6800be90 con 0x7fdc70139030 2023-09-30T22:17:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.495+0000 7fdc77d33700 1 -- 172.21.15.89:0/2318223695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70139030 msgr2=0x7fdc7013b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.495+0000 7fdc77d33700 1 --2- 172.21.15.89:0/2318223695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70139030 0x7fdc7013b410 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009d90 tx=0x7fdc6800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.495+0000 7fdc77d33700 1 -- 172.21.15.89:0/2318223695 shutdown_connections 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.495+0000 7fdc77d33700 1 --2- 172.21.15.89:0/2318223695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70139030 0x7fdc7013b410 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.495+0000 7fdc77d33700 1 --2- 172.21.15.89:0/2318223695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70136710 0x7fdc70138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.495+0000 7fdc77d33700 1 -- 172.21.15.89:0/2318223695 >> 172.21.15.89:0/2318223695 conn(0x7fdc700a2aa0 msgr2=0x7fdc700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.495+0000 7fdc77d33700 1 -- 172.21.15.89:0/2318223695 shutdown_connections 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.495+0000 7fdc77d33700 1 -- 172.21.15.89:0/2318223695 wait complete. 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.496+0000 7fdc77d33700 1 Processor -- start 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.496+0000 7fdc77d33700 1 -- start start 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc77d33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc701338a0 0x7fdc7011a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc77d33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70136710 0x7fdc7011a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc77d33700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70139030 0x7fdc7011aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc77d33700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7011bb30 con 0x7fdc70136710 2023-09-30T22:17:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc77d33700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7011dae0 con 0x7fdc701338a0 2023-09-30T22:17:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc77d33700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7011b4f0 con 0x7fdc70139030 2023-09-30T22:17:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc75acf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc701338a0 0x7fdc7011a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc75acf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc701338a0 0x7fdc7011a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56374/0 (socket says 172.21.15.89:56374) 2023-09-30T22:17:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc75acf700 1 -- 172.21.15.89:0/1436616944 learned_addr learned my addr 172.21.15.89:0/1436616944 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc762d0700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70139030 0x7fdc7011aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc752ce700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70136710 0x7fdc7011a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.497+0000 7fdc75acf700 1 -- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70139030 msgr2=0x7fdc7011aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc75acf700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70139030 0x7fdc7011aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc75acf700 1 -- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70136710 msgr2=0x7fdc7011a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc75acf700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70136710 0x7fdc7011a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc75acf700 1 -- 172.21.15.89:0/1436616944 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68009a70 con 0x7fdc701338a0 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc75acf700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc701338a0 0x7fdc7011a000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c700 tx=0x7fdc6c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc667fc700 1 -- 172.21.15.89:0/1436616944 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c005d70 con 0x7fdc701338a0 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc667fc700 1 -- 172.21.15.89:0/1436616944 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c0070d0 con 0x7fdc701338a0 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7011d050 con 0x7fdc701338a0 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc667fc700 1 -- 172.21.15.89:0/1436616944 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c007460 con 0x7fdc701338a0 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.498+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7011c160 con 0x7fdc701338a0 2023-09-30T22:17:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.499+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7009ffb0 con 0x7fdc701338a0 2023-09-30T22:17:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.499+0000 7fdc667fc700 1 -- 172.21.15.89:0/1436616944 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fdc6c0075c0 con 0x7fdc701338a0 2023-09-30T22:17:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.500+0000 7fdc752ce700 1 -- 172.21.15.89:0/1436616944 >> v1:172.21.15.89:6816/126953 conn(0x7fdc5402f510 legacy=0x7fdc540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.500+0000 7fdc667fc700 1 -- 172.21.15.89:0/1436616944 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fdc6c03efd0 con 0x7fdc701338a0 2023-09-30T22:17:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.508+0000 7fdc667fc700 1 -- 172.21.15.89:0/1436616944 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc6c070750 con 0x7fdc701338a0 2023-09-30T22:17:14.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.539+0000 7f0d3dfb1700 1 Processor -- start 2023-09-30T22:17:14.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.539+0000 7f0d3dfb1700 1 -- start start 2023-09-30T22:17:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d3dfb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d3dfb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e000 0x7f0d38130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d3dfb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d3809e410 0x7f0d38130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d3dfb1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3809f520 con 0x7f0d3809e410 2023-09-30T22:17:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d3dfb1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3809f130 con 0x7f0d3809e000 2023-09-30T22:17:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d3dfb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3809e930 con 0x7f0d3809d6f0 2023-09-30T22:17:14.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d36ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e000 0x7f0d38130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d36ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e000 0x7f0d38130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56380/0 (socket says 172.21.15.89:56380) 2023-09-30T22:17:14.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d36ffd700 1 -- 172.21.15.89:0/2950389838 learned_addr learned my addr 172.21.15.89:0/2950389838 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:14.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d37fff700 1 --2- 172.21.15.89:0/2950389838 >> v2:172.21.15.89:3300/0 conn(0x7f0d3809e410 0x7f0d38130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d377fe700 1 --2- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d37fff700 1 -- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 msgr2=0x7f0d3809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:14.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d37fff700 1 --2- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d37fff700 1 -- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e000 msgr2=0x7f0d38130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d37fff700 1 --2- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e000 0x7f0d38130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d37fff700 1 -- 172.21.15.89:0/2950389838 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3809b2d0 con 0x7f0d3809e410 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.540+0000 7f0d37fff700 1 --2- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e410 0x7f0d38130ba0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009d90 tx=0x7f0d3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281e928bbdd46b46 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.541+0000 7f0d367fc700 1 -- 172.21.15.89:0/2950389838 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000bd20 con 0x7f0d3809e410 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.541+0000 7f0d367fc700 1 -- 172.21.15.89:0/2950389838 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3000be80 con 0x7f0d3809e410 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.541+0000 7f0d367fc700 1 -- 172.21.15.89:0/2950389838 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30010710 con 0x7f0d3809e410 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e410 msgr2=0x7f0d38130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 --2- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e410 0x7f0d38130ba0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009d90 tx=0x7f0d3000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:14.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/2950389838 shutdown_connections 2023-09-30T22:17:14.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 --2- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e410 0x7f0d38130ba0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 --2- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 --2- 172.21.15.89:0/2950389838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e000 0x7f0d38130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/2950389838 >> 172.21.15.89:0/2950389838 conn(0x7f0d38066f50 msgr2=0x7f0d3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:14.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/2950389838 shutdown_connections 2023-09-30T22:17:14.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.542+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/2950389838 wait complete. 2023-09-30T22:17:14.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.543+0000 7f0d3dfb1700 1 Processor -- start 2023-09-30T22:17:14.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.543+0000 7f0d3dfb1700 1 -- start start 2023-09-30T22:17:14.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d3dfb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3811df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d3dfb1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e000 0x7f0d3811e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d3dfb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e410 0x7f0d3811e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d3dfb1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3811f400 con 0x7f0d3809e000 2023-09-30T22:17:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d3dfb1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38130310 con 0x7f0d3809e410 2023-09-30T22:17:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d3dfb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38120d90 con 0x7f0d3809d6f0 2023-09-30T22:17:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d36ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e000 0x7f0d3811e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d36ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e000 0x7f0d3811e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43468/0 (socket says 172.21.15.89:43468) 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d377fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3811df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d36ffd700 1 -- 172.21.15.89:0/3932651036 learned_addr learned my addr 172.21.15.89:0/3932651036 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d37fff700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e410 0x7f0d3811e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d377fe700 1 -- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e410 msgr2=0x7f0d3811e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.544+0000 7f0d377fe700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e410 0x7f0d3811e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.545+0000 7f0d377fe700 1 -- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e000 msgr2=0x7f0d3811e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.545+0000 7f0d377fe700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e000 0x7f0d3811e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.545+0000 7f0d377fe700 1 -- 172.21.15.89:0/3932651036 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30009a70 con 0x7f0d3809d6f0 2023-09-30T22:17:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.545+0000 7f0d36ffd700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e000 0x7f0d3811e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.545+0000 7f0d377fe700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3811df30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800eab0 tx=0x7f0d2800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.545+0000 7f0d1ffff700 1 -- 172.21.15.89:0/3932651036 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800dc50 con 0x7f0d3809d6f0 2023-09-30T22:17:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.545+0000 7f0d1ffff700 1 -- 172.21.15.89:0/3932651036 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800ddb0 con 0x7f0d3809d6f0 2023-09-30T22:17:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.545+0000 7f0d1ffff700 1 -- 172.21.15.89:0/3932651036 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28010640 con 0x7f0d3809d6f0 2023-09-30T22:17:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.546+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d38121770 con 0x7f0d3809d6f0 2023-09-30T22:17:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.546+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d380a1640 con 0x7f0d3809d6f0 2023-09-30T22:17:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.547+0000 7f0d1ffff700 1 -- 172.21.15.89:0/3932651036 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0d280107a0 con 0x7f0d3809d6f0 2023-09-30T22:17:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.547+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3811ac20 con 0x7f0d3809d6f0 2023-09-30T22:17:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.547+0000 7f0d36ffd700 1 -- 172.21.15.89:0/3932651036 >> v1:172.21.15.89:6816/126953 conn(0x7f0d2402f510 legacy=0x7f0d240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.547+0000 7f0d1ffff700 1 -- 172.21.15.89:0/3932651036 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f0d28014030 con 0x7f0d3809d6f0 2023-09-30T22:17:14.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.548+0000 7f5dec211700 1 Processor -- start 2023-09-30T22:17:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.548+0000 7f5dec211700 1 -- start start 2023-09-30T22:17:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dec211700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de41338a0 0x7f5de4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dec211700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5de409e930 0x7f5de413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dec211700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de406ef30 con 0x7f5de409de50 2023-09-30T22:17:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dec211700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de406fef0 con 0x7f5de409e930 2023-09-30T22:17:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dec211700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de41334b0 con 0x7f5de41338a0 2023-09-30T22:17:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5de97ac700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5de409de50 0x7f5de409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52810/0 (socket says 172.21.15.89:52810) 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5de97ac700 1 -- 172.21.15.89:0/1792094544 learned_addr learned my addr 172.21.15.89:0/1792094544 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dea7ae700 1 --2- 172.21.15.89:0/1792094544 >> v2:172.21.15.89:3300/0 conn(0x7f5de409e930 0x7f5de413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5de8fab700 1 -- 172.21.15.89:0/1792094544 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904535881 0 0) 0x7f5de406ef30 con 0x7f5de409de50 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5de8fab700 1 -- 172.21.15.89:0/1792094544 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd00035b0 con 0x7f5de409de50 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dea7ae700 1 -- 172.21.15.89:0/1792094544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de41338a0 msgr2=0x7f5de4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dea7ae700 1 --2- 172.21.15.89:0/1792094544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de41338a0 0x7f5de4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dea7ae700 1 -- 172.21.15.89:0/1792094544 >> v1:172.21.15.89:6789/0 conn(0x7f5de409de50 legacy=0x7f5de409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dea7ae700 1 -- 172.21.15.89:0/1792094544 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de406fc30 con 0x7f5de409e930 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5dea7ae700 1 --2- 172.21.15.89:0/1792094544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409e930 0x7f5de413b740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009d90 tx=0x7f5ddc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd351b2c9e455567 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.549+0000 7f5de8fab700 1 -- 172.21.15.89:0/1792094544 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc00e070 con 0x7f5de409e930 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.550+0000 7f5de8fab700 1 -- 172.21.15.89:0/1792094544 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ddc00bb20 con 0x7f5de409e930 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.550+0000 7f5de8fab700 1 -- 172.21.15.89:0/1792094544 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc012430 con 0x7f5de409e930 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.550+0000 7f5dec211700 1 -- 172.21.15.89:0/1792094544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409e930 msgr2=0x7f5de413b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.550+0000 7f5dec211700 1 --2- 172.21.15.89:0/1792094544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409e930 0x7f5de413b740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009d90 tx=0x7f5ddc00b330 comp rx=0 tx=0).stop 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.550+0000 7f5dec211700 1 -- 172.21.15.89:0/1792094544 shutdown_connections 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.550+0000 7f5dec211700 1 --2- 172.21.15.89:0/1792094544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409e930 0x7f5de413b740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.550+0000 7f5dec211700 1 --2- 172.21.15.89:0/1792094544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de41338a0 0x7f5de4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.550+0000 7f5dec211700 1 -- 172.21.15.89:0/1792094544 >> 172.21.15.89:0/1792094544 conn(0x7f5de40a2aa0 msgr2=0x7f5de40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.551+0000 7f5dec211700 1 -- 172.21.15.89:0/1792094544 shutdown_connections 2023-09-30T22:17:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.551+0000 7f5dec211700 1 -- 172.21.15.89:0/1792094544 wait complete. 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.552+0000 7f5dec211700 1 Processor -- start 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.552+0000 7f5dec211700 1 -- start start 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dec211700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409de50 0x7f5de411b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dec211700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de409e930 0x7f5de411bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dec211700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5de41338a0 0x7f5de411c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dec211700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de411e0f0 con 0x7f5de409de50 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dec211700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de411ec30 con 0x7f5de409e930 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dec211700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40a0de0 con 0x7f5de41338a0 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409de50 0x7f5de411b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409de50 0x7f5de411b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43474/0 (socket says 172.21.15.89:43474) 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 -- 172.21.15.89:0/120828563 learned_addr learned my addr 172.21.15.89:0/120828563 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dea7ae700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5de41338a0 0x7f5de411c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de97ac700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de409e930 0x7f5de411bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 -- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5de41338a0 msgr2=0x7f5de411c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5de41338a0 0x7f5de411c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 -- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de409e930 msgr2=0x7f5de411bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de409e930 0x7f5de411bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 -- 172.21.15.89:0/120828563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc009a70 con 0x7f5de409de50 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5de9fad700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409de50 0x7f5de411b730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5de000b560 tx=0x7f5de000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dda7fc700 1 -- 172.21.15.89:0/120828563 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0011830 con 0x7f5de409de50 2023-09-30T22:17:14.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dda7fc700 1 -- 172.21.15.89:0/120828563 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de00119b0 con 0x7f5de409de50 2023-09-30T22:17:14.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de411dab0 con 0x7f5de409de50 2023-09-30T22:17:14.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.553+0000 7f5dda7fc700 1 -- 172.21.15.89:0/120828563 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0011d60 con 0x7f5de409de50 2023-09-30T22:17:14.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.554+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de411f1d0 con 0x7f5de409de50 2023-09-30T22:17:14.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.554+0000 7f0d1ffff700 1 -- 172.21.15.89:0/3932651036 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d28045060 con 0x7f0d3809d6f0 2023-09-30T22:17:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.555+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de4074ea0 con 0x7f5de409de50 2023-09-30T22:17:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.555+0000 7f5dda7fc700 1 -- 172.21.15.89:0/120828563 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5de001a400 con 0x7f5de409de50 2023-09-30T22:17:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.556+0000 7f5dda7fc700 1 -- 172.21.15.89:0/120828563 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f5de0040190 con 0x7f5de409de50 2023-09-30T22:17:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.556+0000 7f5de97ac700 1 -- 172.21.15.89:0/120828563 >> v1:172.21.15.89:6816/126953 conn(0x7f5dc802f510 legacy=0x7f5dc80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.560+0000 7f5dda7fc700 1 -- 172.21.15.89:0/120828563 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5de0021710 con 0x7f5de409de50 2023-09-30T22:17:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.657+0000 7f59556b9700 1 -- 172.21.15.89:0/3437906468 >> v1:172.21.15.89:6816/126953 conn(0x7f593402f510 legacy=0x7f59340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.666+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f595011aac0 con 0x7f595006fcf0 2023-09-30T22:17:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.666+0000 7f59467fc700 1 -- 172.21.15.89:0/3437906468 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f594c029020 con 0x7f595006fcf0 2023-09-30T22:17:14.671 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 >> v1:172.21.15.89:6816/126953 conn(0x7f593402f510 legacy=0x7f59340319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f595006fcf0 msgr2=0x7f595009ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 --2- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f595006fcf0 0x7f595009ce50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f594c000c00 tx=0x7f594c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 shutdown_connections 2023-09-30T22:17:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 --2- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5950130e50 0x7f595009d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 --2- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f595006fcf0 0x7f595009ce50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 --2- 172.21.15.89:0/3437906468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5950130540 0x7f595009d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 >> 172.21.15.89:0/3437906468 conn(0x7f59500a2aa0 msgr2=0x7f595012d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.668+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 shutdown_connections 2023-09-30T22:17:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.669+0000 7f595811e700 1 -- 172.21.15.89:0/3437906468 wait complete. 2023-09-30T22:17:14.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.701+0000 7fdc752ce700 1 -- 172.21.15.89:0/1436616944 >> v1:172.21.15.89:6816/126953 conn(0x7fdc5402f510 legacy=0x7fdc540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:14.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.748+0000 7f0d36ffd700 1 -- 172.21.15.89:0/3932651036 >> v1:172.21.15.89:6816/126953 conn(0x7f0d2402f510 legacy=0x7f0d240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:14.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.748+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdc7006ea40 con 0x7fdc701338a0 2023-09-30T22:17:14.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.748+0000 7fdc667fc700 1 -- 172.21.15.89:0/1436616944 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc6c017070 con 0x7fdc701338a0 2023-09-30T22:17:14.748 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:14.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.748+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0d3809c700 con 0x7f0d3809d6f0 2023-09-30T22:17:14.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.749+0000 7f0d1ffff700 1 -- 172.21.15.89:0/3932651036 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d28045070 con 0x7f0d3809d6f0 2023-09-30T22:17:14.749 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 >> v1:172.21.15.89:6816/126953 conn(0x7fdc5402f510 legacy=0x7fdc540319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc701338a0 msgr2=0x7fdc7011a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc701338a0 0x7fdc7011a000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c700 tx=0x7fdc6c00ca10 comp rx=0 tx=0).stop 2023-09-30T22:17:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 shutdown_connections 2023-09-30T22:17:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc70139030 0x7fdc7011aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc701338a0 0x7fdc7011a000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 --2- 172.21.15.89:0/1436616944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc70136710 0x7fdc7011a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 >> 172.21.15.89:0/1436616944 conn(0x7fdc700a2aa0 msgr2=0x7fdc70130050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:14.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 shutdown_connections 2023-09-30T22:17:14.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7fdc77d33700 1 -- 172.21.15.89:0/1436616944 wait complete. 2023-09-30T22:17:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 >> v1:172.21.15.89:6816/126953 conn(0x7f0d2402f510 legacy=0x7f0d240319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 msgr2=0x7f0d3811df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3811df30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800eab0 tx=0x7f0d2800edc0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 shutdown_connections 2023-09-30T22:17:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d3809e410 0x7f0d3811e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d3809d6f0 0x7f0d3811df30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 --2- 172.21.15.89:0/3932651036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d3809e000 0x7f0d3811e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 >> 172.21.15.89:0/3932651036 conn(0x7f0d38066f50 msgr2=0x7f0d38130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:14.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 shutdown_connections 2023-09-30T22:17:14.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.751+0000 7f0d3dfb1700 1 -- 172.21.15.89:0/3932651036 wait complete. 2023-09-30T22:17:14.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.757+0000 7f5de97ac700 1 -- 172.21.15.89:0/120828563 >> v1:172.21.15.89:6816/126953 conn(0x7f5dc802f510 legacy=0x7f5dc80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:14.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:14.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.797+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5de411ed70 con 0x7f5de409de50 2023-09-30T22:17:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.798+0000 7f5dda7fc700 1 -- 172.21.15.89:0/120828563 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5de0015090 con 0x7f5de409de50 2023-09-30T22:17:14.797 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 >> v1:172.21.15.89:6816/126953 conn(0x7f5dc802f510 legacy=0x7f5dc80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409de50 msgr2=0x7f5de411b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409de50 0x7f5de411b730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5de000b560 tx=0x7f5de000b870 comp rx=0 tx=0).stop 2023-09-30T22:17:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 shutdown_connections 2023-09-30T22:17:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5de41338a0 0x7f5de411c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5de409de50 0x7f5de411b730 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 --2- 172.21.15.89:0/120828563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5de409e930 0x7f5de411bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 >> 172.21.15.89:0/120828563 conn(0x7f5de40a2aa0 msgr2=0x7f5de4130630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 shutdown_connections 2023-09-30T22:17:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:14.800+0000 7f5dec211700 1 -- 172.21.15.89:0/120828563 wait complete. 2023-09-30T22:17:14.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:15.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:15.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:15.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:15.812 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.822+0000 7f1cd6fb6700 1 Processor -- start 2023-09-30T22:17:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.822+0000 7f1cd6fb6700 1 -- start start 2023-09-30T22:17:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1cd6fb6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cd009e1d0 0x7f1cd0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1cd6fb6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1cd6fb6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd009f130 con 0x7f1cd009d6f0 2023-09-30T22:17:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1cd6fb6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd009f520 con 0x7f1cd009e1d0 2023-09-30T22:17:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1cd6fb6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd009e930 con 0x7f1cd0135e50 2023-09-30T22:17:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1ccffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cd009e1d0 0x7f1cd0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1ccffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cd009e1d0 0x7f1cd0135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43486/0 (socket says 172.21.15.89:43486) 2023-09-30T22:17:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1ccffff700 1 -- 172.21.15.89:0/1408443246 learned_addr learned my addr 172.21.15.89:0/1408443246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.823+0000 7f1cd5553700 1 --2- 172.21.15.89:0/1408443246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccffff700 1 -- 172.21.15.89:0/1408443246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 msgr2=0x7f1cd0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccffff700 1 --2- 172.21.15.89:0/1408443246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccf7fe700 1 -- 172.21.15.89:0/1408443246 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157306813 0 0) 0x7f1cd009f130 con 0x7f1cd009d6f0 2023-09-30T22:17:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccffff700 1 -- 172.21.15.89:0/1408443246 >> v1:172.21.15.89:6789/0 conn(0x7f1cd009d6f0 legacy=0x7f1cd009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccffff700 1 -- 172.21.15.89:0/1408443246 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd009b2d0 con 0x7f1cd009e1d0 2023-09-30T22:17:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccffff700 1 --2- 172.21.15.89:0/1408443246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009e1d0 0x7f1cd0135910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00bb90 tx=0x7f1cbc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b67a599d5b50076 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccf7fe700 1 -- 172.21.15.89:0/1408443246 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00dd30 con 0x7f1cd009e1d0 2023-09-30T22:17:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccf7fe700 1 -- 172.21.15.89:0/1408443246 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00de90 con 0x7f1cd009e1d0 2023-09-30T22:17:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.824+0000 7f1ccf7fe700 1 -- 172.21.15.89:0/1408443246 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc010690 con 0x7f1cd009e1d0 2023-09-30T22:17:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.825+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/1408443246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009e1d0 msgr2=0x7f1cd0135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.825+0000 7f1cd6fb6700 1 --2- 172.21.15.89:0/1408443246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009e1d0 0x7f1cd0135910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00bb90 tx=0x7f1cbc00bea0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.825+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/1408443246 shutdown_connections 2023-09-30T22:17:15.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.825+0000 7f1cd6fb6700 1 --2- 172.21.15.89:0/1408443246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.825+0000 7f1cd6fb6700 1 --2- 172.21.15.89:0/1408443246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009e1d0 0x7f1cd0135910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.825+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/1408443246 >> 172.21.15.89:0/1408443246 conn(0x7f1cd0066f50 msgr2=0x7f1cd009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:15.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.825+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/1408443246 shutdown_connections 2023-09-30T22:17:15.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.825+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/1408443246 wait complete. 2023-09-30T22:17:15.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.826+0000 7f1cd6fb6700 1 Processor -- start 2023-09-30T22:17:15.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.826+0000 7f1cd6fb6700 1 -- start start 2023-09-30T22:17:15.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1cd6fb6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009d6f0 0x7f1cd011e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1cd6fb6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009e1d0 0x7f1cd011e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1cd6fb6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd01c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1cd6fb6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd011f320 con 0x7f1cd009d6f0 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1cd6fb6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd011f8e0 con 0x7f1cd0135e50 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1cd6fb6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0130010 con 0x7f1cd009e1d0 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1ccffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009e1d0 0x7f1cd011e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1ccffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009e1d0 0x7f1cd011e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57572/0 (socket says 172.21.15.89:57572) 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1cd5553700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd01c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1ccffff700 1 -- 172.21.15.89:0/2525521949 learned_addr learned my addr 172.21.15.89:0/2525521949 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.827+0000 7f1cd4d52700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009d6f0 0x7f1cd011e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1cd5553700 1 -- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009e1d0 msgr2=0x7f1cd011e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1cd5553700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009e1d0 0x7f1cd011e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1cd5553700 1 -- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009d6f0 msgr2=0x7f1cd011e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1cd5553700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009d6f0 0x7f1cd011e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1cd5553700 1 -- 172.21.15.89:0/2525521949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc00b7f0 con 0x7f1cd0135e50 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1ccffff700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009e1d0 0x7f1cd011e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1cd5553700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd01c80e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009d90 tx=0x7f1cc800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1ccd7fa700 1 -- 172.21.15.89:0/2525521949 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8009c00 con 0x7f1cd0135e50 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1ccd7fa700 1 -- 172.21.15.89:0/2525521949 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc800f040 con 0x7f1cd0135e50 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd00a1d00 con 0x7f1cd0135e50 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1ccd7fa700 1 -- 172.21.15.89:0/2525521949 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800ce90 con 0x7f1cd0135e50 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.828+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd011ff70 con 0x7f1cd0135e50 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.829+0000 7f1ccd7fa700 1 -- 172.21.15.89:0/2525521949 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1cc8014480 con 0x7f1cd0135e50 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.829+0000 7f1cd4d52700 1 -- 172.21.15.89:0/2525521949 >> v1:172.21.15.89:6816/126953 conn(0x7f1cb402f510 legacy=0x7f1cb40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.829+0000 7f1ccd7fa700 1 -- 172.21.15.89:0/2525521949 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f1cc80406d0 con 0x7f1cd0135e50 2023-09-30T22:17:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.829+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd011ff70 con 0x7f1cd0135e50 2023-09-30T22:17:15.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.834+0000 7f1ccd7fa700 1 -- 172.21.15.89:0/2525521949 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1cd011ff70 con 0x7f1cd0135e50 2023-09-30T22:17:15.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.899+0000 7f3644459700 1 Processor -- start 2023-09-30T22:17:15.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f3644459700 1 -- start start 2023-09-30T22:17:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f3644459700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f3644459700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f363c09ace0 0x7f363c09e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f3644459700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c09b150 con 0x7f363c1339f0 2023-09-30T22:17:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f3644459700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c09a500 con 0x7f363c09ace0 2023-09-30T22:17:15.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f3644459700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f363c09e540 con 0x7f363c09d6f0 2023-09-30T22:17:15.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f36421f5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f36421f5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57586/0 (socket says 172.21.15.89:57586) 2023-09-30T22:17:15.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.900+0000 7f36421f5700 1 -- 172.21.15.89:0/950060220 learned_addr learned my addr 172.21.15.89:0/950060220 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:15.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36419f4700 1 --2- 172.21.15.89:0/950060220 >> v2:172.21.15.89:3300/0 conn(0x7f363c09ace0 0x7f363c09e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36421f5700 1 -- 172.21.15.89:0/950060220 >> v2:172.21.15.89:3300/0 conn(0x7f363c09ace0 msgr2=0x7f363c09e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:15.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36421f5700 1 --2- 172.21.15.89:0/950060220 >> v2:172.21.15.89:3300/0 conn(0x7f363c09ace0 0x7f363c09e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36421f5700 1 -- 172.21.15.89:0/950060220 >> v1:172.21.15.89:6789/0 conn(0x7f363c1339f0 legacy=0x7f363c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36421f5700 1 -- 172.21.15.89:0/950060220 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c09f720 con 0x7f363c09d6f0 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36419f4700 1 --2- 172.21.15.89:0/950060220 >> v2:172.21.15.89:3300/0 conn(0x7f363c09ace0 0x7f363c09e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36421f5700 1 --2- 172.21.15.89:0/950060220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c09dac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36380098f0 tx=0x7f3638009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ca511103f72bf4e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36411f3700 1 -- 172.21.15.89:0/950060220 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638004230 con 0x7f363c09d6f0 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36411f3700 1 -- 172.21.15.89:0/950060220 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638004390 con 0x7f363c09d6f0 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.901+0000 7f36411f3700 1 -- 172.21.15.89:0/950060220 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380036a0 con 0x7f363c09d6f0 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.902+0000 7f3644459700 1 -- 172.21.15.89:0/950060220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 msgr2=0x7f363c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.902+0000 7f3644459700 1 --2- 172.21.15.89:0/950060220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c09dac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36380098f0 tx=0x7f3638009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.902+0000 7f3644459700 1 -- 172.21.15.89:0/950060220 shutdown_connections 2023-09-30T22:17:15.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.902+0000 7f3644459700 1 --2- 172.21.15.89:0/950060220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c09dac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.902+0000 7f3644459700 1 --2- 172.21.15.89:0/950060220 >> v2:172.21.15.89:3300/0 conn(0x7f363c09ace0 0x7f363c09e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.902+0000 7f3644459700 1 -- 172.21.15.89:0/950060220 >> 172.21.15.89:0/950060220 conn(0x7f363c066f50 msgr2=0x7f363c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.902+0000 7f3644459700 1 -- 172.21.15.89:0/950060220 shutdown_connections 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.902+0000 7f3644459700 1 -- 172.21.15.89:0/950060220 wait complete. 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.903+0000 7f3644459700 1 Processor -- start 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.903+0000 7f3644459700 1 -- start start 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3644459700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f363c09ace0 0x7f363c11e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3644459700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c11e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3644459700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f363c1339f0 0x7f363c11ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3644459700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c11fe50 con 0x7f363c1339f0 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3644459700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c120470 con 0x7f363c09ace0 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3644459700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f363c1210a0 con 0x7f363c09d6f0 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c11e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c11e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57594/0 (socket says 172.21.15.89:57594) 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 -- 172.21.15.89:0/700349414 learned_addr learned my addr 172.21.15.89:0/700349414 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36429f6700 1 --2- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f363c1339f0 0x7f363c11ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 -- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f363c09ace0 msgr2=0x7f363c11e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 --2- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f363c09ace0 0x7f363c11e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 -- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f363c1339f0 msgr2=0x7f363c11ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 --2- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f363c1339f0 0x7f363c11ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 -- 172.21.15.89:0/700349414 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009550 con 0x7f363c09d6f0 2023-09-30T22:17:15.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f36419f4700 1 --2- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c11e7d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e7e0 tx=0x7f362c00eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:15.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3632ffd700 1 -- 172.21.15.89:0/700349414 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c013610 con 0x7f363c09d6f0 2023-09-30T22:17:15.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3632ffd700 1 -- 172.21.15.89:0/700349414 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c013770 con 0x7f363c09d6f0 2023-09-30T22:17:15.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.904+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c121a80 con 0x7f363c09d6f0 2023-09-30T22:17:15.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.905+0000 7f3632ffd700 1 -- 172.21.15.89:0/700349414 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c013ae0 con 0x7f363c09d6f0 2023-09-30T22:17:15.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.905+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c09c6f0 con 0x7f363c09d6f0 2023-09-30T22:17:15.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.905+0000 7f3632ffd700 1 -- 172.21.15.89:0/700349414 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f362c013c40 con 0x7f363c09d6f0 2023-09-30T22:17:15.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.906+0000 7f3632ffd700 1 -- 172.21.15.89:0/700349414 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f362c03f810 con 0x7f363c09d6f0 2023-09-30T22:17:15.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.906+0000 7f36421f5700 1 -- 172.21.15.89:0/700349414 >> v1:172.21.15.89:6816/126953 conn(0x7f362002f510 legacy=0x7f36200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:15.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.906+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36240052e0 con 0x7f363c09d6f0 2023-09-30T22:17:15.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.914+0000 7f3632ffd700 1 -- 172.21.15.89:0/700349414 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f362c021080 con 0x7f363c09d6f0 2023-09-30T22:17:15.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.915+0000 7fc9f63d9700 1 Processor -- start 2023-09-30T22:17:15.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.915+0000 7fc9f63d9700 1 -- start start 2023-09-30T22:17:15.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f63d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 0x7fc9f0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f63d9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f63d9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f009f130 con 0x7fc9f009d6f0 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f63d9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f009f520 con 0x7fc9f009e1d0 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f63d9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f009e930 con 0x7fc9f0135e50 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9effff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc9f009d6f0 0x7fc9f009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52838/0 (socket says 172.21.15.89:52838) 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9effff700 1 -- 172.21.15.89:0/2951366749 learned_addr learned my addr 172.21.15.89:0/2951366749 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f4976700 1 --2- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9ef7fe700 1 --2- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 0x7fc9f0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9eeffd700 1 -- 172.21.15.89:0/2951366749 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380881605 0 0) 0x7fc9f009f130 con 0x7fc9f009d6f0 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f4976700 1 -- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 msgr2=0x7fc9f0135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f4976700 1 --2- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 0x7fc9f0135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f4976700 1 -- 172.21.15.89:0/2951366749 >> v1:172.21.15.89:6789/0 conn(0x7fc9f009d6f0 legacy=0x7fc9f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.916+0000 7fc9f4976700 1 -- 172.21.15.89:0/2951366749 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f009b2d0 con 0x7fc9f0135e50 2023-09-30T22:17:15.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9f4976700 1 --2- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f0138230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009dc0 tx=0x7fc9e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf139578c128f12a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:15.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9eeffd700 1 -- 172.21.15.89:0/2951366749 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800e070 con 0x7fc9f0135e50 2023-09-30T22:17:15.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9eeffd700 1 -- 172.21.15.89:0/2951366749 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e800bb20 con 0x7fc9f0135e50 2023-09-30T22:17:15.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9eeffd700 1 -- 172.21.15.89:0/2951366749 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8012430 con 0x7fc9f0135e50 2023-09-30T22:17:15.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 msgr2=0x7fc9f0138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9f63d9700 1 --2- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f0138230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009dc0 tx=0x7fc9e800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:15.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2951366749 shutdown_connections 2023-09-30T22:17:15.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9f63d9700 1 --2- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f0138230 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9f63d9700 1 --2- 172.21.15.89:0/2951366749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 0x7fc9f0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.917+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2951366749 >> 172.21.15.89:0/2951366749 conn(0x7fc9f0066f50 msgr2=0x7fc9f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:15.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.918+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2951366749 shutdown_connections 2023-09-30T22:17:15.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.918+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2951366749 wait complete. 2023-09-30T22:17:15.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.918+0000 7fc9f63d9700 1 Processor -- start 2023-09-30T22:17:15.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f63d9700 1 -- start start 2023-09-30T22:17:15.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f63d9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9f009d6f0 0x7fc9f01397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f63d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 0x7fc9f0139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f63d9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f013a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f63d9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f01212e0 con 0x7fc9f009d6f0 2023-09-30T22:17:15.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f63d9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f0135b10 con 0x7fc9f009e1d0 2023-09-30T22:17:15.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f63d9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f0120530 con 0x7fc9f0135e50 2023-09-30T22:17:15.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f4976700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f013a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9effff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9f009d6f0 0x7fc9f01397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f4976700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f013a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57614/0 (socket says 172.21.15.89:57614) 2023-09-30T22:17:15.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9f4976700 1 -- 172.21.15.89:0/2157395377 learned_addr learned my addr 172.21.15.89:0/2157395377 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:15.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.919+0000 7fc9ef7fe700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 0x7fc9f0139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9f4976700 1 -- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 msgr2=0x7fc9f0139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9f4976700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 0x7fc9f0139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9f4976700 1 -- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9f009d6f0 msgr2=0x7fc9f01397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9f4976700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9f009d6f0 0x7fc9f01397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9f4976700 1 -- 172.21.15.89:0/2157395377 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8009a70 con 0x7fc9f0135e50 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9effff700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9f009d6f0 0x7fc9f01397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9f4976700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f013a230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8013000 tx=0x7fc9e8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9ecff9700 1 -- 172.21.15.89:0/2157395377 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8012430 con 0x7fc9f0135e50 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9ecff9700 1 -- 172.21.15.89:0/2157395377 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e8020070 con 0x7fc9f0135e50 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9ecff9700 1 -- 172.21.15.89:0/2157395377 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8012ea0 con 0x7fc9f0135e50 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f0120e80 con 0x7fc9f0135e50 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.920+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f00a2090 con 0x7fc9f0135e50 2023-09-30T22:17:15.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.921+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f011fec0 con 0x7fc9f0135e50 2023-09-30T22:17:15.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.922+0000 7fc9ecff9700 1 -- 172.21.15.89:0/2157395377 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc9e8004280 con 0x7fc9f0135e50 2023-09-30T22:17:15.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.922+0000 7fc9ecff9700 1 -- 172.21.15.89:0/2157395377 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc9e8022070 con 0x7fc9f0135e50 2023-09-30T22:17:15.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.922+0000 7fc9effff700 1 -- 172.21.15.89:0/2157395377 >> v1:172.21.15.89:6816/126953 conn(0x7fc9dc02f510 legacy=0x7fc9dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:15.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.926+0000 7fc9ecff9700 1 -- 172.21.15.89:0/2157395377 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9e8011980 con 0x7fc9f0135e50 2023-09-30T22:17:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.957+0000 7fc22e458700 1 Processor -- start 2023-09-30T22:17:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.957+0000 7fc22e458700 1 -- start start 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc22e458700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2280a0a30 0x7fc2280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc22e458700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc22809d630 0x7fc2280a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc22e458700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 0x7fc228133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc22e458700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2280a2a70 con 0x7fc22809d630 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc22e458700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280a01d0 con 0x7fc2280a0a30 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc22e458700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc22809e2b0 con 0x7fc228133930 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc227fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2280a0a30 0x7fc2280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc227fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2280a0a30 0x7fc2280a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56464/0 (socket says 172.21.15.89:56464) 2023-09-30T22:17:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc2277fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc22809d630 0x7fc2280a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc227fff700 1 -- 172.21.15.89:0/1969701868 learned_addr learned my addr 172.21.15.89:0/1969701868 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.958+0000 7fc22c9f5700 1 --2- 172.21.15.89:0/1969701868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 0x7fc228133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc227fff700 1 -- 172.21.15.89:0/1969701868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 msgr2=0x7fc228133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc227fff700 1 --2- 172.21.15.89:0/1969701868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 0x7fc228133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc227fff700 1 -- 172.21.15.89:0/1969701868 >> v2:172.21.15.89:3300/0 conn(0x7fc22809d630 msgr2=0x7fc2280a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc227fff700 1 --2- 172.21.15.89:0/1969701868 >> v2:172.21.15.89:3300/0 conn(0x7fc22809d630 0x7fc2280a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc227fff700 1 -- 172.21.15.89:0/1969701868 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22809dc50 con 0x7fc2280a0a30 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc2277fe700 1 --2- 172.21.15.89:0/1969701868 >> v2:172.21.15.89:3300/0 conn(0x7fc22809d630 0x7fc2280a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc227fff700 1 --2- 172.21.15.89:0/1969701868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2280a0a30 0x7fc2280a0e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098f0 tx=0x7fc218009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840537b855079626 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc226ffd700 1 -- 172.21.15.89:0/1969701868 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218011830 con 0x7fc2280a0a30 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc226ffd700 1 -- 172.21.15.89:0/1969701868 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218011990 con 0x7fc2280a0a30 2023-09-30T22:17:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.959+0000 7fc226ffd700 1 -- 172.21.15.89:0/1969701868 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218011d20 con 0x7fc2280a0a30 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 -- 172.21.15.89:0/1969701868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2280a0a30 msgr2=0x7fc2280a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 --2- 172.21.15.89:0/1969701868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2280a0a30 0x7fc2280a0e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098f0 tx=0x7fc218009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 -- 172.21.15.89:0/1969701868 shutdown_connections 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 --2- 172.21.15.89:0/1969701868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 0x7fc228133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 --2- 172.21.15.89:0/1969701868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2280a0a30 0x7fc2280a0e00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 --2- 172.21.15.89:0/1969701868 >> v2:172.21.15.89:3300/0 conn(0x7fc22809d630 0x7fc2280a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 -- 172.21.15.89:0/1969701868 >> 172.21.15.89:0/1969701868 conn(0x7fc2280a3050 msgr2=0x7fc2280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 -- 172.21.15.89:0/1969701868 shutdown_connections 2023-09-30T22:17:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.960+0000 7fc22e458700 1 -- 172.21.15.89:0/1969701868 wait complete. 2023-09-30T22:17:15.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.961+0000 7fc22e458700 1 Processor -- start 2023-09-30T22:17:15.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.961+0000 7fc22e458700 1 -- start start 2023-09-30T22:17:15.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc22e458700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc22809d630 0x7fc228121190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc22e458700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2280a0a30 0x7fc2281216d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc22e458700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 0x7fc228121c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:15.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc22e458700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228124440 con 0x7fc2280a0a30 2023-09-30T22:17:15.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc22e458700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281255e0 con 0x7fc22809d630 2023-09-30T22:17:15.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc22e458700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280a0410 con 0x7fc228133930 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc227fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc22809d630 0x7fc228121190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc227fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc22809d630 0x7fc228121190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56474/0 (socket says 172.21.15.89:56474) 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc227fff700 1 -- 172.21.15.89:0/2190056008 learned_addr learned my addr 172.21.15.89:0/2190056008 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc22c9f5700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 0x7fc228121c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc2277fe700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2280a0a30 0x7fc2281216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc227fff700 1 -- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 msgr2=0x7fc228121c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc227fff700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 0x7fc228121c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc227fff700 1 -- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2280a0a30 msgr2=0x7fc2281216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc227fff700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2280a0a30 0x7fc2281216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.962+0000 7fc227fff700 1 -- 172.21.15.89:0/2190056008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0094a0 con 0x7fc22809d630 2023-09-30T22:17:15.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.963+0000 7fc227fff700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc22809d630 0x7fc228121190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc218004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:15.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.963+0000 7fc224ff9700 1 -- 172.21.15.89:0/2190056008 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218004450 con 0x7fc22809d630 2023-09-30T22:17:15.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.963+0000 7fc224ff9700 1 -- 172.21.15.89:0/2190056008 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2180045b0 con 0x7fc22809d630 2023-09-30T22:17:15.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.963+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc218009550 con 0x7fc22809d630 2023-09-30T22:17:15.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.963+0000 7fc224ff9700 1 -- 172.21.15.89:0/2190056008 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180038c0 con 0x7fc22809d630 2023-09-30T22:17:15.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.963+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc228124940 con 0x7fc22809d630 2023-09-30T22:17:15.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.964+0000 7fc224ff9700 1 -- 172.21.15.89:0/2190056008 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc218021070 con 0x7fc22809d630 2023-09-30T22:17:15.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.964+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22811df60 con 0x7fc22809d630 2023-09-30T22:17:15.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.964+0000 7fc224ff9700 1 -- 172.21.15.89:0/2190056008 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc218040ee0 con 0x7fc22809d630 2023-09-30T22:17:15.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.965+0000 7fc2277fe700 1 -- 172.21.15.89:0/2190056008 >> v1:172.21.15.89:6816/126953 conn(0x7fc21402f510 legacy=0x7fc2140319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:15.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:15.973+0000 7fc224ff9700 1 -- 172.21.15.89:0/2190056008 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc218019770 con 0x7fc22809d630 2023-09-30T22:17:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.030+0000 7f1cd4d52700 1 -- 172.21.15.89:0/2525521949 >> v1:172.21.15.89:6816/126953 conn(0x7f1cb402f510 legacy=0x7f1cb40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.048+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1cd011ff70 con 0x7f1cd0135e50 2023-09-30T22:17:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.049+0000 7f1ccd7fa700 1 -- 172.21.15.89:0/2525521949 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1cd011ff70 con 0x7f1cd0135e50 2023-09-30T22:17:16.048 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 >> v1:172.21.15.89:6816/126953 conn(0x7f1cb402f510 legacy=0x7f1cb40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 msgr2=0x7f1cd01c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd01c80e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009d90 tx=0x7f1cc800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 shutdown_connections 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0135e50 0x7f1cd01c80e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd009d6f0 0x7f1cd011e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 --2- 172.21.15.89:0/2525521949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009e1d0 0x7f1cd011e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 >> 172.21.15.89:0/2525521949 conn(0x7f1cd0066f50 msgr2=0x7f1cd00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 shutdown_connections 2023-09-30T22:17:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.051+0000 7f1cd6fb6700 1 -- 172.21.15.89:0/2525521949 wait complete. 2023-09-30T22:17:16.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:16.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.107+0000 7f36421f5700 1 -- 172.21.15.89:0/700349414 >> v1:172.21.15.89:6816/126953 conn(0x7f362002f510 legacy=0x7f36200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:16.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.123+0000 7fc9effff700 1 -- 172.21.15.89:0/2157395377 >> v1:172.21.15.89:6816/126953 conn(0x7fc9dc02f510 legacy=0x7fc9dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:16.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.125+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc9f0071700 con 0x7fc9f0135e50 2023-09-30T22:17:16.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.125+0000 7fc9ecff9700 1 -- 172.21.15.89:0/2157395377 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9e802c070 con 0x7fc9f0135e50 2023-09-30T22:17:16.126 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:16.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 >> v1:172.21.15.89:6816/126953 conn(0x7fc9dc02f510 legacy=0x7fc9dc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:16.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 msgr2=0x7fc9f013a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:16.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f013a230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8013000 tx=0x7fc9e8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 shutdown_connections 2023-09-30T22:17:16.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9f0135e50 0x7fc9f013a230 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9f009d6f0 0x7fc9f01397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 --2- 172.21.15.89:0/2157395377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9f009e1d0 0x7fc9f0139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 >> 172.21.15.89:0/2157395377 conn(0x7fc9f0066f50 msgr2=0x7fc9f006e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:16.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 shutdown_connections 2023-09-30T22:17:16.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.127+0000 7fc9f63d9700 1 -- 172.21.15.89:0/2157395377 wait complete. 2023-09-30T22:17:16.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:16.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.141+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3624005860 con 0x7f363c09d6f0 2023-09-30T22:17:16.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.142+0000 7f3632ffd700 1 -- 172.21.15.89:0/700349414 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f362c012370 con 0x7f363c09d6f0 2023-09-30T22:17:16.142 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:16.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 >> v1:172.21.15.89:6816/126953 conn(0x7f362002f510 legacy=0x7f36200319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:16.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 msgr2=0x7f363c11e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:16.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 --2- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c11e7d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e7e0 tx=0x7f362c00eaf0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 shutdown_connections 2023-09-30T22:17:16.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 --2- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f363c1339f0 0x7f363c11ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 --2- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f363c09ace0 0x7f363c11e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 --2- 172.21.15.89:0/700349414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f363c09d6f0 0x7f363c11e7d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 >> 172.21.15.89:0/700349414 conn(0x7f363c066f50 msgr2=0x7f363c133020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:16.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 shutdown_connections 2023-09-30T22:17:16.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.144+0000 7f3644459700 1 -- 172.21.15.89:0/700349414 wait complete. 2023-09-30T22:17:16.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:16.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.166+0000 7fc2277fe700 1 -- 172.21.15.89:0/2190056008 >> v1:172.21.15.89:6816/126953 conn(0x7fc21402f510 legacy=0x7fc2140319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.181+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc2280747f0 con 0x7fc22809d630 2023-09-30T22:17:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.182+0000 7fc224ff9700 1 -- 172.21.15.89:0/2190056008 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc218029030 con 0x7fc22809d630 2023-09-30T22:17:16.182 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 >> v1:172.21.15.89:6816/126953 conn(0x7fc21402f510 legacy=0x7fc2140319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc22809d630 msgr2=0x7fc228121190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc22809d630 0x7fc228121190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc218004750 comp rx=0 tx=0).stop 2023-09-30T22:17:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 shutdown_connections 2023-09-30T22:17:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc228133930 0x7fc228121c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc22809d630 0x7fc228121190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 --2- 172.21.15.89:0/2190056008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2280a0a30 0x7fc2281216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 >> 172.21.15.89:0/2190056008 conn(0x7fc2280a3050 msgr2=0x7fc2281331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 shutdown_connections 2023-09-30T22:17:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:16.184+0000 7fc22e458700 1 -- 172.21.15.89:0/2190056008 wait complete. 2023-09-30T22:17:16.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:17.065 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:17.140 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:17.157 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:17.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:17.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.203+0000 7f28ff7c1700 1 Processor -- start 2023-09-30T22:17:17.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.203+0000 7f28ff7c1700 1 -- start start 2023-09-30T22:17:17.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28ff7c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809d6f0 0x7f28f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28ff7c1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28f809e000 0x7f28f81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28ff7c1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f809b120 con 0x7f28f809e410 2023-09-30T22:17:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28ff7c1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f806b460 con 0x7f28f809e000 2023-09-30T22:17:17.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28ff7c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f809a7e0 con 0x7f28f809d6f0 2023-09-30T22:17:17.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fcd5c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28f809e000 0x7f28f81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fdd5e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f28f809e410 0x7f28f8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52854/0 (socket says 172.21.15.89:52854) 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fcd5c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28f809e000 0x7f28f81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43550/0 (socket says 172.21.15.89:43550) 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fdd5e700 1 -- 172.21.15.89:0/1164149192 learned_addr learned my addr 172.21.15.89:0/1164149192 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fd55d700 1 --2- 172.21.15.89:0/1164149192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809d6f0 0x7f28f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fd55d700 1 -- 172.21.15.89:0/1164149192 >> v2:172.21.15.89:3300/0 conn(0x7f28f809e000 msgr2=0x7f28f81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fd55d700 1 --2- 172.21.15.89:0/1164149192 >> v2:172.21.15.89:3300/0 conn(0x7f28f809e000 0x7f28f81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fd55d700 1 -- 172.21.15.89:0/1164149192 >> v1:172.21.15.89:6789/0 conn(0x7f28f809e410 legacy=0x7f28f8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fd55d700 1 -- 172.21.15.89:0/1164149192 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f809a500 con 0x7f28f809d6f0 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.204+0000 7f28fcd5c700 1 --2- 172.21.15.89:0/1164149192 >> v2:172.21.15.89:3300/0 conn(0x7f28f809e000 0x7f28f81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:17.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.205+0000 7f28fd55d700 1 --2- 172.21.15.89:0/1164149192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809d6f0 0x7f28f809dac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009870 tx=0x7f28f4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bbb3ea1552d1d48 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.205+0000 7f28effff700 1 -- 172.21.15.89:0/1164149192 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4004230 con 0x7f28f809d6f0 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.205+0000 7f28effff700 1 -- 172.21.15.89:0/1164149192 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4004390 con 0x7f28f809d6f0 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.205+0000 7f28effff700 1 -- 172.21.15.89:0/1164149192 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40036a0 con 0x7f28f809d6f0 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.206+0000 7f28ff7c1700 1 -- 172.21.15.89:0/1164149192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809d6f0 msgr2=0x7f28f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.206+0000 7f28ff7c1700 1 --2- 172.21.15.89:0/1164149192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809d6f0 0x7f28f809dac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009870 tx=0x7f28f4009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.206+0000 7f28ff7c1700 1 -- 172.21.15.89:0/1164149192 shutdown_connections 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.206+0000 7f28ff7c1700 1 --2- 172.21.15.89:0/1164149192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809d6f0 0x7f28f809dac0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.206+0000 7f28ff7c1700 1 --2- 172.21.15.89:0/1164149192 >> v2:172.21.15.89:3300/0 conn(0x7f28f809e000 0x7f28f81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.206+0000 7f28ff7c1700 1 -- 172.21.15.89:0/1164149192 >> 172.21.15.89:0/1164149192 conn(0x7f28f8066f50 msgr2=0x7f28f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.206+0000 7f28ff7c1700 1 -- 172.21.15.89:0/1164149192 shutdown_connections 2023-09-30T22:17:17.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.206+0000 7f28ff7c1700 1 -- 172.21.15.89:0/1164149192 wait complete. 2023-09-30T22:17:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.207+0000 7f28ff7c1700 1 Processor -- start 2023-09-30T22:17:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.207+0000 7f28ff7c1700 1 -- start start 2023-09-30T22:17:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.207+0000 7f28ff7c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28f809d6f0 0x7f28f81a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28ff7c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28f809e000 0x7f28f81a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28ff7c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809e410 0x7f28f81a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28ff7c1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80a3290 con 0x7f28f809e000 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28ff7c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f81a8480 con 0x7f28f809e410 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28ff7c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28f81a6c90 con 0x7f28f809d6f0 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fd55d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28f809d6f0 0x7f28f81a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fdd5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809e410 0x7f28f81a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fd55d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28f809d6f0 0x7f28f81a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57634/0 (socket says 172.21.15.89:57634) 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fd55d700 1 -- 172.21.15.89:0/2131568768 learned_addr learned my addr 172.21.15.89:0/2131568768 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fcd5c700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28f809e000 0x7f28f81a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fdd5e700 1 -- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28f809d6f0 msgr2=0x7f28f81a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fdd5e700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28f809d6f0 0x7f28f81a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fdd5e700 1 -- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28f809e000 msgr2=0x7f28f81a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fdd5e700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28f809e000 0x7f28f81a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fdd5e700 1 -- 172.21.15.89:0/2131568768 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80094a0 con 0x7f28f809e410 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fcd5c700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28f809e000 0x7f28f81a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.208+0000 7f28fdd5e700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809e410 0x7f28f81a5af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28f000cb20 tx=0x7f28f000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.209+0000 7f28edffb700 1 -- 172.21.15.89:0/2131568768 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000a9a0 con 0x7f28f809e410 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.209+0000 7f28edffb700 1 -- 172.21.15.89:0/2131568768 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f000ab20 con 0x7f28f809e410 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.209+0000 7f28edffb700 1 -- 172.21.15.89:0/2131568768 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0012400 con 0x7f28f809e410 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.210+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f40094a0 con 0x7f28f809e410 2023-09-30T22:17:17.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.210+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f81a8740 con 0x7f28f809e410 2023-09-30T22:17:17.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.211+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f8074ea0 con 0x7f28f809e410 2023-09-30T22:17:17.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.211+0000 7f28edffb700 1 -- 172.21.15.89:0/2131568768 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f28f0012560 con 0x7f28f809e410 2023-09-30T22:17:17.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.212+0000 7f28edffb700 1 -- 172.21.15.89:0/2131568768 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f28f003fdc0 con 0x7f28f809e410 2023-09-30T22:17:17.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.212+0000 7f28fd55d700 1 -- 172.21.15.89:0/2131568768 >> v1:172.21.15.89:6816/126953 conn(0x7f28dc02f510 legacy=0x7f28dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:17.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.217+0000 7f28edffb700 1 -- 172.21.15.89:0/2131568768 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28f0012810 con 0x7f28f809e410 2023-09-30T22:17:17.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.276+0000 7f401729e700 1 Processor -- start 2023-09-30T22:17:17.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.276+0000 7f401729e700 1 -- start start 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.276+0000 7f401729e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010131810 0x7f4010131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.276+0000 7f401729e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4010099a40 0x7f4010099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.276+0000 7f401729e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401009a850 con 0x7f401009a3b0 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.276+0000 7f401729e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401009b580 con 0x7f4010099a40 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.276+0000 7f401729e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40101313c0 con 0x7f4010131810 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f401503a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010131810 0x7f4010131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f401503a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010131810 0x7f4010131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56490/0 (socket says 172.21.15.89:56490) 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f401503a700 1 -- 172.21.15.89:0/4010318938 learned_addr learned my addr 172.21.15.89:0/4010318938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:17.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f4014839700 1 --2- 172.21.15.89:0/4010318938 >> v2:172.21.15.89:3300/0 conn(0x7f4010099a40 0x7f4010099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f401503a700 1 -- 172.21.15.89:0/4010318938 >> v2:172.21.15.89:3300/0 conn(0x7f4010099a40 msgr2=0x7f4010099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:17.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f401503a700 1 --2- 172.21.15.89:0/4010318938 >> v2:172.21.15.89:3300/0 conn(0x7f4010099a40 0x7f4010099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f401503a700 1 -- 172.21.15.89:0/4010318938 >> v1:172.21.15.89:6789/0 conn(0x7f401009a3b0 legacy=0x7f4010138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f401503a700 1 -- 172.21.15.89:0/4010318938 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401006e8c0 con 0x7f4010131810 2023-09-30T22:17:17.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f401503a700 1 --2- 172.21.15.89:0/4010318938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010131810 0x7f4010131be0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f400c009920 tx=0x7f400c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde93be3bf14d27e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:17.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f4007fff700 1 -- 172.21.15.89:0/4010318938 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c00ba20 con 0x7f4010131810 2023-09-30T22:17:17.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f4007fff700 1 -- 172.21.15.89:0/4010318938 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c00bb80 con 0x7f4010131810 2023-09-30T22:17:17.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.277+0000 7f4007fff700 1 -- 172.21.15.89:0/4010318938 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c009e70 con 0x7f4010131810 2023-09-30T22:17:17.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.278+0000 7f401729e700 1 -- 172.21.15.89:0/4010318938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010131810 msgr2=0x7f4010131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.278+0000 7f401729e700 1 --2- 172.21.15.89:0/4010318938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010131810 0x7f4010131be0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f400c009920 tx=0x7f400c009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.278+0000 7f401729e700 1 -- 172.21.15.89:0/4010318938 shutdown_connections 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.278+0000 7f401729e700 1 --2- 172.21.15.89:0/4010318938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010131810 0x7f4010131be0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.278+0000 7f401729e700 1 --2- 172.21.15.89:0/4010318938 >> v2:172.21.15.89:3300/0 conn(0x7f4010099a40 0x7f4010099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.278+0000 7f401729e700 1 -- 172.21.15.89:0/4010318938 >> 172.21.15.89:0/4010318938 conn(0x7f40100a2aa0 msgr2=0x7f40100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.278+0000 7f401729e700 1 -- 172.21.15.89:0/4010318938 shutdown_connections 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.278+0000 7f401729e700 1 -- 172.21.15.89:0/4010318938 wait complete. 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.279+0000 7f401729e700 1 Processor -- start 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.279+0000 7f401729e700 1 -- start start 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401729e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010099a40 0x7f40101196e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401729e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f401009a3b0 0x7f4010119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401729e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4010131810 0x7f401011a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401729e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401011d200 con 0x7f401009a3b0 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401729e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401011b390 con 0x7f4010099a40 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401729e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40100a1c20 con 0x7f4010131810 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401583b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4010131810 0x7f401011a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f4014839700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f401009a3b0 0x7f4010119c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401583b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4010131810 0x7f401011a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57646/0 (socket says 172.21.15.89:57646) 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401583b700 1 -- 172.21.15.89:0/1971934694 learned_addr learned my addr 172.21.15.89:0/1971934694 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:17.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401503a700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010099a40 0x7f40101196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401503a700 1 -- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4010131810 msgr2=0x7f401011a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401503a700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4010131810 0x7f401011a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401503a700 1 -- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f401009a3b0 msgr2=0x7f4010119c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401503a700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f401009a3b0 0x7f4010119c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f401503a700 1 -- 172.21.15.89:0/1971934694 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c009550 con 0x7f4010099a40 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.280+0000 7f4014839700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f401009a3b0 0x7f4010119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.281+0000 7f401503a700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010099a40 0x7f40101196e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f400c016000 tx=0x7f400c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.281+0000 7f4005ffb700 1 -- 172.21.15.89:0/1971934694 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c004450 con 0x7f4010099a40 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.281+0000 7f4005ffb700 1 -- 172.21.15.89:0/1971934694 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c0045b0 con 0x7f4010099a40 2023-09-30T22:17:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.281+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401006eb00 con 0x7f4010099a40 2023-09-30T22:17:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.281+0000 7f4005ffb700 1 -- 172.21.15.89:0/1971934694 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0038e0 con 0x7f4010099a40 2023-09-30T22:17:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.281+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401011d700 con 0x7f4010099a40 2023-09-30T22:17:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.282+0000 7f4005ffb700 1 -- 172.21.15.89:0/1971934694 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f400c021070 con 0x7f4010099a40 2023-09-30T22:17:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.282+0000 7f4005ffb700 1 -- 172.21.15.89:0/1971934694 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f400c040f40 con 0x7f4010099a40 2023-09-30T22:17:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.282+0000 7f4014839700 1 -- 172.21.15.89:0/1971934694 >> v1:172.21.15.89:6816/126953 conn(0x7f3ff402f510 legacy=0x7f3ff40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.282+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4010074ea0 con 0x7f4010099a40 2023-09-30T22:17:17.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.287+0000 7f4005ffb700 1 -- 172.21.15.89:0/1971934694 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f400c015020 con 0x7f4010099a40 2023-09-30T22:17:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.326+0000 7f4371bd1700 1 Processor -- start 2023-09-30T22:17:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.326+0000 7f4371bd1700 1 -- start start 2023-09-30T22:17:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f4371bd1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09d6f0 0x7f436c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f4371bd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09e000 0x7f436c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f4371bd1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c09b120 con 0x7f436c09e410 2023-09-30T22:17:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f4371bd1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c06b460 con 0x7f436c09e000 2023-09-30T22:17:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f4371bd1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f436c09a7e0 con 0x7f436c09d6f0 2023-09-30T22:17:17.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f436affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09e000 0x7f436c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f436affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09e000 0x7f436c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56516/0 (socket says 172.21.15.89:56516) 2023-09-30T22:17:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f436affd700 1 -- 172.21.15.89:0/2746464220 learned_addr learned my addr 172.21.15.89:0/2746464220 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f436b7fe700 1 --2- 172.21.15.89:0/2746464220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09d6f0 0x7f436c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f436affd700 1 -- 172.21.15.89:0/2746464220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09d6f0 msgr2=0x7f436c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f436affd700 1 --2- 172.21.15.89:0/2746464220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09d6f0 0x7f436c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f436affd700 1 -- 172.21.15.89:0/2746464220 >> v1:172.21.15.89:6789/0 conn(0x7f436c09e410 legacy=0x7f436c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.327+0000 7f436affd700 1 -- 172.21.15.89:0/2746464220 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c09a500 con 0x7f436c09e000 2023-09-30T22:17:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.328+0000 7f436affd700 1 --2- 172.21.15.89:0/2746464220 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09e000 0x7f436c1339f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4360009920 tx=0x7f4360009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d1ca14430ce21d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.328+0000 7f436a7fc700 1 -- 172.21.15.89:0/2746464220 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360011830 con 0x7f436c09e000 2023-09-30T22:17:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.328+0000 7f436a7fc700 1 -- 172.21.15.89:0/2746464220 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360011990 con 0x7f436c09e000 2023-09-30T22:17:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.328+0000 7f436a7fc700 1 -- 172.21.15.89:0/2746464220 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360011d00 con 0x7f436c09e000 2023-09-30T22:17:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.329+0000 7f4371bd1700 1 -- 172.21.15.89:0/2746464220 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09e000 msgr2=0x7f436c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.329+0000 7f4371bd1700 1 --2- 172.21.15.89:0/2746464220 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09e000 0x7f436c1339f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4360009920 tx=0x7f4360009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.330+0000 7f4371bd1700 1 -- 172.21.15.89:0/2746464220 shutdown_connections 2023-09-30T22:17:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.330+0000 7f4371bd1700 1 --2- 172.21.15.89:0/2746464220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09d6f0 0x7f436c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.330+0000 7f4371bd1700 1 --2- 172.21.15.89:0/2746464220 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09e000 0x7f436c1339f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.330+0000 7f4371bd1700 1 -- 172.21.15.89:0/2746464220 >> 172.21.15.89:0/2746464220 conn(0x7f436c066f50 msgr2=0x7f436c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.330+0000 7f4371bd1700 1 -- 172.21.15.89:0/2746464220 shutdown_connections 2023-09-30T22:17:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.330+0000 7f4371bd1700 1 -- 172.21.15.89:0/2746464220 wait complete. 2023-09-30T22:17:17.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.331+0000 7f4371bd1700 1 Processor -- start 2023-09-30T22:17:17.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.331+0000 7f4371bd1700 1 -- start start 2023-09-30T22:17:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f4371bd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09d6f0 0x7f436c1225f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f4371bd1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09e000 0x7f436c122b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f4371bd1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f436c09e410 0x7f436c123070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f4371bd1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c0a2db0 con 0x7f436c09e410 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f4371bd1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c09c710 con 0x7f436c09d6f0 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f4371bd1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f436c124fd0 con 0x7f436c09e000 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09d6f0 0x7f436c1225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09d6f0 0x7f436c1225f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56520/0 (socket says 172.21.15.89:56520) 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 -- 172.21.15.89:0/3877041304 learned_addr learned my addr 172.21.15.89:0/3877041304 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436affd700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09e000 0x7f436c122b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436bfff700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f436c09e410 0x7f436c123070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 -- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09e000 msgr2=0x7f436c122b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09e000 0x7f436c122b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 -- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f436c09e410 msgr2=0x7f436c123070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f436c09e410 0x7f436c123070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 -- 172.21.15.89:0/3877041304 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009550 con 0x7f436c09d6f0 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.332+0000 7f436b7fe700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09d6f0 0x7f436c1225f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f435c00e9b0 tx=0x7f435c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.333+0000 7f4353fff700 1 -- 172.21.15.89:0/3877041304 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c009710 con 0x7f436c09d6f0 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.333+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c125850 con 0x7f436c09d6f0 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.333+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c126850 con 0x7f436c09d6f0 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.333+0000 7f4353fff700 1 -- 172.21.15.89:0/3877041304 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c00c850 con 0x7f436c09d6f0 2023-09-30T22:17:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.333+0000 7f4353fff700 1 -- 172.21.15.89:0/3877041304 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c00cbc0 con 0x7f436c09d6f0 2023-09-30T22:17:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.333+0000 7f4353fff700 1 -- 172.21.15.89:0/3877041304 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f435c00cd20 con 0x7f436c09d6f0 2023-09-30T22:17:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.334+0000 7f4353fff700 1 -- 172.21.15.89:0/3877041304 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f435c03fde0 con 0x7f436c09d6f0 2023-09-30T22:17:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.334+0000 7f436affd700 1 -- 172.21.15.89:0/3877041304 >> v1:172.21.15.89:6816/126953 conn(0x7f435802f510 legacy=0x7f43580319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.334+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434c0052e0 con 0x7f436c09d6f0 2023-09-30T22:17:17.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.338+0000 7f6d3d52c700 1 Processor -- start 2023-09-30T22:17:17.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.338+0000 7f6d3d52c700 1 -- start start 2023-09-30T22:17:17.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d3d52c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809d6f0 0x7f6d3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d3d52c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809e000 0x7f6d38130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f4353fff700 1 -- 172.21.15.89:0/3877041304 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f435c0232f0 con 0x7f436c09d6f0 2023-09-30T22:17:17.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d3d52c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d3809e410 0x7f6d38130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d3d52c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3809f520 con 0x7f6d3809e410 2023-09-30T22:17:17.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d3d52c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3809f130 con 0x7f6d3809d6f0 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d3d52c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3809e930 con 0x7f6d3809e000 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d367fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809e000 0x7f6d38130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d36ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809d6f0 0x7f6d3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d367fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809e000 0x7f6d38130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57690/0 (socket says 172.21.15.89:57690) 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d367fc700 1 -- 172.21.15.89:0/1950601159 learned_addr learned my addr 172.21.15.89:0/1950601159 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d377fe700 1 --2- 172.21.15.89:0/1950601159 >> v2:172.21.15.89:3300/0 conn(0x7f6d3809e410 0x7f6d38130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d36ffd700 1 -- 172.21.15.89:0/1950601159 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809e000 msgr2=0x7f6d38130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d36ffd700 1 --2- 172.21.15.89:0/1950601159 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809e000 0x7f6d38130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d36ffd700 1 -- 172.21.15.89:0/1950601159 >> v2:172.21.15.89:3300/0 conn(0x7f6d3809e410 msgr2=0x7f6d38130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d36ffd700 1 --2- 172.21.15.89:0/1950601159 >> v2:172.21.15.89:3300/0 conn(0x7f6d3809e410 0x7f6d38130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d36ffd700 1 -- 172.21.15.89:0/1950601159 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3809b2d0 con 0x7f6d3809d6f0 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.339+0000 7f6d367fc700 1 --2- 172.21.15.89:0/1950601159 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809e000 0x7f6d38130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.340+0000 7f6d377fe700 1 --2- 172.21.15.89:0/1950601159 >> v2:172.21.15.89:3300/0 conn(0x7f6d3809e410 0x7f6d38130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.340+0000 7f6d36ffd700 1 --2- 172.21.15.89:0/1950601159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809d6f0 0x7f6d3809dac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009870 tx=0x7f6d28009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3d14fd9f667c81 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.340+0000 7f6d35ffb700 1 -- 172.21.15.89:0/1950601159 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28011830 con 0x7f6d3809d6f0 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.340+0000 7f6d35ffb700 1 -- 172.21.15.89:0/1950601159 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d28011990 con 0x7f6d3809d6f0 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.340+0000 7f6d35ffb700 1 -- 172.21.15.89:0/1950601159 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28011d20 con 0x7f6d3809d6f0 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1950601159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809d6f0 msgr2=0x7f6d3809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 --2- 172.21.15.89:0/1950601159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809d6f0 0x7f6d3809dac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009870 tx=0x7f6d28009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1950601159 shutdown_connections 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 --2- 172.21.15.89:0/1950601159 >> v2:172.21.15.89:3300/0 conn(0x7f6d3809e410 0x7f6d38130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 --2- 172.21.15.89:0/1950601159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809d6f0 0x7f6d3809dac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 --2- 172.21.15.89:0/1950601159 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809e000 0x7f6d38130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1950601159 >> 172.21.15.89:0/1950601159 conn(0x7f6d38066f50 msgr2=0x7f6d3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:17.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1950601159 shutdown_connections 2023-09-30T22:17:17.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.341+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1950601159 wait complete. 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.342+0000 7f6d3d52c700 1 Processor -- start 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.342+0000 7f6d3d52c700 1 -- start start 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d3d52c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809d6f0 0x7f6d381222a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d3d52c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3809e000 0x7f6d381227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d3d52c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809e410 0x7f6d38122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d3d52c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38123700 con 0x7f6d3809e000 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d3d52c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380a3af0 con 0x7f6d3809e410 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d3d52c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38123fc0 con 0x7f6d3809d6f0 2023-09-30T22:17:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d36ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809d6f0 0x7f6d381222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d36ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809d6f0 0x7f6d381222a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57692/0 (socket says 172.21.15.89:57692) 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d36ffd700 1 -- 172.21.15.89:0/1902929083 learned_addr learned my addr 172.21.15.89:0/1902929083 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d367fc700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3809e000 0x7f6d381227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.343+0000 7f6d377fe700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809e410 0x7f6d38122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d36ffd700 1 -- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809e410 msgr2=0x7f6d38122d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d36ffd700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809e410 0x7f6d38122d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d36ffd700 1 -- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3809e000 msgr2=0x7f6d381227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d36ffd700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3809e000 0x7f6d381227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d36ffd700 1 -- 172.21.15.89:0/1902929083 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c009550 con 0x7f6d3809d6f0 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d377fe700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809e410 0x7f6d38122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:17:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d36ffd700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809d6f0 0x7f6d381222a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d28000c00 tx=0x7f6d28004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d1f7fe700 1 -- 172.21.15.89:0/1902929083 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d280036a0 con 0x7f6d3809d6f0 2023-09-30T22:17:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d1f7fe700 1 -- 172.21.15.89:0/1902929083 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2801f070 con 0x7f6d3809d6f0 2023-09-30T22:17:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d280094a0 con 0x7f6d3809d6f0 2023-09-30T22:17:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d1f7fe700 1 -- 172.21.15.89:0/1902929083 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28003a20 con 0x7f6d3809d6f0 2023-09-30T22:17:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.344+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d38124670 con 0x7f6d3809d6f0 2023-09-30T22:17:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.345+0000 7f6d1f7fe700 1 -- 172.21.15.89:0/1902929083 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6d28003b80 con 0x7f6d3809d6f0 2023-09-30T22:17:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.345+0000 7f6d1f7fe700 1 -- 172.21.15.89:0/1902929083 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f6d28021070 con 0x7f6d3809d6f0 2023-09-30T22:17:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.345+0000 7f6d367fc700 1 -- 172.21.15.89:0/1902929083 >> v1:172.21.15.89:6816/126953 conn(0x7f6d2402f510 legacy=0x7f6d240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.345+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d38126610 con 0x7f6d3809d6f0 2023-09-30T22:17:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.350+0000 7f6d1f7fe700 1 -- 172.21.15.89:0/1902929083 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d28015080 con 0x7f6d3809d6f0 2023-09-30T22:17:17.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.413+0000 7f28fd55d700 1 -- 172.21.15.89:0/2131568768 >> v1:172.21.15.89:6816/126953 conn(0x7f28dc02f510 legacy=0x7f28dc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:17.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.417+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28f806c360 con 0x7f28f809e410 2023-09-30T22:17:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.417+0000 7f28edffb700 1 -- 172.21.15.89:0/2131568768 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28f0019770 con 0x7f28f809e410 2023-09-30T22:17:17.417 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:17.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 >> v1:172.21.15.89:6816/126953 conn(0x7f28dc02f510 legacy=0x7f28dc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809e410 msgr2=0x7f28f81a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809e410 0x7f28f81a5af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28f000cb20 tx=0x7f28f000ce30 comp rx=0 tx=0).stop 2023-09-30T22:17:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 shutdown_connections 2023-09-30T22:17:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28f809e410 0x7f28f81a5af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28f809d6f0 0x7f28f81a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 --2- 172.21.15.89:0/2131568768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28f809e000 0x7f28f81a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 >> 172.21.15.89:0/2131568768 conn(0x7f28f8066f50 msgr2=0x7f28f806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 shutdown_connections 2023-09-30T22:17:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.419+0000 7f28ff7c1700 1 -- 172.21.15.89:0/2131568768 wait complete. 2023-09-30T22:17:17.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:17.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.483+0000 7f4014839700 1 -- 172.21.15.89:0/1971934694 >> v1:172.21.15.89:6816/126953 conn(0x7f3ff402f510 legacy=0x7f3ff40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.515+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4010071db0 con 0x7f4010099a40 2023-09-30T22:17:17.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.516+0000 7f4005ffb700 1 -- 172.21.15.89:0/1971934694 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f400c003a40 con 0x7f4010099a40 2023-09-30T22:17:17.516 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 >> v1:172.21.15.89:6816/126953 conn(0x7f3ff402f510 legacy=0x7f3ff40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010099a40 msgr2=0x7f40101196e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010099a40 0x7f40101196e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f400c016000 tx=0x7f400c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 shutdown_connections 2023-09-30T22:17:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4010131810 0x7f401011a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4010099a40 0x7f40101196e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 --2- 172.21.15.89:0/1971934694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f401009a3b0 0x7f4010119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 >> 172.21.15.89:0/1971934694 conn(0x7f40100a2aa0 msgr2=0x7f401012f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:17.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 shutdown_connections 2023-09-30T22:17:17.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.518+0000 7f401729e700 1 -- 172.21.15.89:0/1971934694 wait complete. 2023-09-30T22:17:17.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:17.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.535+0000 7f436affd700 1 -- 172.21.15.89:0/3877041304 >> v1:172.21.15.89:6816/126953 conn(0x7f435802f510 legacy=0x7f43580319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:17.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.536+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f434c005860 con 0x7f436c09d6f0 2023-09-30T22:17:17.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.537+0000 7f4353fff700 1 -- 172.21.15.89:0/3877041304 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f435c01ae80 con 0x7f436c09d6f0 2023-09-30T22:17:17.536 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:17.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.538+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 >> v1:172.21.15.89:6816/126953 conn(0x7f435802f510 legacy=0x7f43580319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.538+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09d6f0 msgr2=0x7f436c1225f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.538+0000 7f4371bd1700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09d6f0 0x7f436c1225f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f435c00e9b0 tx=0x7f435c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.539+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 shutdown_connections 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.539+0000 7f4371bd1700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f436c09e410 0x7f436c123070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.539+0000 7f4371bd1700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f436c09d6f0 0x7f436c1225f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.539+0000 7f4371bd1700 1 --2- 172.21.15.89:0/3877041304 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f436c09e000 0x7f436c122b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.539+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 >> 172.21.15.89:0/3877041304 conn(0x7f436c066f50 msgr2=0x7f436c06f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.539+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 shutdown_connections 2023-09-30T22:17:17.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.539+0000 7f4371bd1700 1 -- 172.21.15.89:0/3877041304 wait complete. 2023-09-30T22:17:17.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.546+0000 7f6d367fc700 1 -- 172.21.15.89:0/1902929083 >> v1:172.21.15.89:6816/126953 conn(0x7f6d2402f510 legacy=0x7f6d240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.89:6816/126953 2023-09-30T22:17:17.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:17.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.562+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6d38074ea0 con 0x7f6d3809d6f0 2023-09-30T22:17:17.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.563+0000 7f6d1f7fe700 1 -- 172.21.15.89:0/1902929083 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d2802b020 con 0x7f6d3809d6f0 2023-09-30T22:17:17.562 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:17.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 >> v1:172.21.15.89:6816/126953 conn(0x7f6d2402f510 legacy=0x7f6d240319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:17.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809d6f0 msgr2=0x7f6d381222a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:17.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809d6f0 0x7f6d381222a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d28000c00 tx=0x7f6d28004750 comp rx=0 tx=0).stop 2023-09-30T22:17:17.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 shutdown_connections 2023-09-30T22:17:17.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d3809e410 0x7f6d38122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d3809d6f0 0x7f6d381222a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 --2- 172.21.15.89:0/1902929083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3809e000 0x7f6d381227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:17.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 >> 172.21.15.89:0/1902929083 conn(0x7f6d38066f50 msgr2=0x7f6d38070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:17.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 shutdown_connections 2023-09-30T22:17:17.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:17.565+0000 7f6d3d52c700 1 -- 172.21.15.89:0/1902929083 wait complete. 2023-09-30T22:17:17.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:18.433 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:18.531 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:18.551 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.566+0000 7f80142b4700 1 Processor -- start 2023-09-30T22:17:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.566+0000 7f80142b4700 1 -- start start 2023-09-30T22:17:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f80142b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f80142b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09e000 0x7f800c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f80142b4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c09b120 con 0x7f800c09e410 2023-09-30T22:17:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f80142b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c06b460 con 0x7f800c09d6f0 2023-09-30T22:17:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f80142b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f800c09a7e0 con 0x7f800c09e000 2023-09-30T22:17:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f8012050700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801184f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09e000 0x7f800c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801184f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09e000 0x7f800c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57694/0 (socket says 172.21.15.89:57694) 2023-09-30T22:17:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f8012050700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56558/0 (socket says 172.21.15.89:56558) 2023-09-30T22:17:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801184f700 1 -- 172.21.15.89:0/3931745717 learned_addr learned my addr 172.21.15.89:0/3931745717 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801184f700 1 -- 172.21.15.89:0/3931745717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09d6f0 msgr2=0x7f800c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801184f700 1 --2- 172.21.15.89:0/3931745717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801184f700 1 -- 172.21.15.89:0/3931745717 >> v1:172.21.15.89:6789/0 conn(0x7f800c09e410 legacy=0x7f800c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801184f700 1 -- 172.21.15.89:0/3931745717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c09a500 con 0x7f800c09e000 2023-09-30T22:17:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801104e700 1 -- 172.21.15.89:0/3931745717 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914513225 0 0) 0x7f800c09b120 con 0x7f800c09e410 2023-09-30T22:17:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f8012050700 1 --2- 172.21.15.89:0/3931745717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.567+0000 7f801184f700 1 --2- 172.21.15.89:0/3931745717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09e000 0x7f800c1339f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009950 tx=0x7f7ffc009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad50ff048bebc3c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f801104e700 1 -- 172.21.15.89:0/3931745717 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc011830 con 0x7f800c09e000 2023-09-30T22:17:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f801104e700 1 -- 172.21.15.89:0/3931745717 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc011990 con 0x7f800c09e000 2023-09-30T22:17:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f801104e700 1 -- 172.21.15.89:0/3931745717 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc011d40 con 0x7f800c09e000 2023-09-30T22:17:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f80142b4700 1 -- 172.21.15.89:0/3931745717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09e000 msgr2=0x7f800c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f80142b4700 1 --2- 172.21.15.89:0/3931745717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09e000 0x7f800c1339f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009950 tx=0x7f7ffc009c60 comp rx=0 tx=0).stop 2023-09-30T22:17:18.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f80142b4700 1 -- 172.21.15.89:0/3931745717 shutdown_connections 2023-09-30T22:17:18.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f80142b4700 1 --2- 172.21.15.89:0/3931745717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f80142b4700 1 --2- 172.21.15.89:0/3931745717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09e000 0x7f800c1339f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f80142b4700 1 -- 172.21.15.89:0/3931745717 >> 172.21.15.89:0/3931745717 conn(0x7f800c066f50 msgr2=0x7f800c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:18.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f80142b4700 1 -- 172.21.15.89:0/3931745717 shutdown_connections 2023-09-30T22:17:18.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.568+0000 7f80142b4700 1 -- 172.21.15.89:0/3931745717 wait complete. 2023-09-30T22:17:18.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.569+0000 7f80142b4700 1 Processor -- start 2023-09-30T22:17:18.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.569+0000 7f80142b4700 1 -- start start 2023-09-30T22:17:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f80142b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09d6f0 0x7f800c1a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f80142b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09e000 0x7f800c1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f80142b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f800c09e410 0x7f800c1c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f80142b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1b91f0 con 0x7f800c09e410 2023-09-30T22:17:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f80142b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1a4d40 con 0x7f800c09e000 2023-09-30T22:17:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f80142b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1a6470 con 0x7f800c09d6f0 2023-09-30T22:17:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09d6f0 0x7f800c1a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09d6f0 0x7f800c1a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57702/0 (socket says 172.21.15.89:57702) 2023-09-30T22:17:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 -- 172.21.15.89:0/3746670029 learned_addr learned my addr 172.21.15.89:0/3746670029 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f801184f700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09e000 0x7f800c1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012851700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f800c09e410 0x7f800c1c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 -- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09e000 msgr2=0x7f800c1a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09e000 0x7f800c1a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 -- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f800c09e410 msgr2=0x7f800c1c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f800c09e410 0x7f800c1c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 -- 172.21.15.89:0/3746670029 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080094a0 con 0x7f800c09d6f0 2023-09-30T22:17:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8012050700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09d6f0 0x7f800c1a5130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f800800ebe0 tx=0x7f800800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8002ffd700 1 -- 172.21.15.89:0/3746670029 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800c850 con 0x7f800c09d6f0 2023-09-30T22:17:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8002ffd700 1 -- 172.21.15.89:0/3746670029 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800800c9b0 con 0x7f800c09d6f0 2023-09-30T22:17:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f8002ffd700 1 -- 172.21.15.89:0/3746670029 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800cd40 con 0x7f800c09d6f0 2023-09-30T22:17:18.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.570+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc009550 con 0x7f800c09d6f0 2023-09-30T22:17:18.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.571+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c09c710 con 0x7f800c09d6f0 2023-09-30T22:17:18.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.571+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c074ea0 con 0x7f800c09d6f0 2023-09-30T22:17:18.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.572+0000 7f8002ffd700 1 -- 172.21.15.89:0/3746670029 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8008013470 con 0x7f800c09d6f0 2023-09-30T22:17:18.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.572+0000 7f8002ffd700 1 -- 172.21.15.89:0/3746670029 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f800803ffc0 con 0x7f800c09d6f0 2023-09-30T22:17:18.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:18.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.576+0000 7f8002ffd700 1 -- 172.21.15.89:0/3746670029 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8008013720 con 0x7f800c09d6f0 2023-09-30T22:17:18.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.664+0000 7f4742e3a700 1 Processor -- start 2023-09-30T22:17:18.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.664+0000 7f4742e3a700 1 -- start start 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f4742e3a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c1338a0 0x7f473c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f4742e3a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c09de50 0x7f473c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f4742e3a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f473c09e760 0x7f473c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f4742e3a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c06fef0 con 0x7f473c09e760 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f4742e3a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c06ef30 con 0x7f473c1338a0 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f4742e3a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f473c1334b0 con 0x7f473c09de50 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c09de50 0x7f473c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f4740bd6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c1338a0 0x7f473c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c09de50 0x7f473c09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57718/0 (socket says 172.21.15.89:57718) 2023-09-30T22:17:18.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 -- 172.21.15.89:0/4076520181 learned_addr learned my addr 172.21.15.89:0/4076520181 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f47413d7700 1 --2- 172.21.15.89:0/4076520181 >> v2:172.21.15.89:3300/0 conn(0x7f473c09e760 0x7f473c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 -- 172.21.15.89:0/4076520181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c1338a0 msgr2=0x7f473c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 --2- 172.21.15.89:0/4076520181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c1338a0 0x7f473c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 -- 172.21.15.89:0/4076520181 >> v2:172.21.15.89:3300/0 conn(0x7f473c09e760 msgr2=0x7f473c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 --2- 172.21.15.89:0/4076520181 >> v2:172.21.15.89:3300/0 conn(0x7f473c09e760 0x7f473c13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 -- 172.21.15.89:0/4076520181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c06fc30 con 0x7f473c09de50 2023-09-30T22:17:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.665+0000 7f473bfff700 1 --2- 172.21.15.89:0/4076520181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c09de50 0x7f473c09e220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4728009920 tx=0x7f4728009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c6405b9640d49d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f473b7fe700 1 -- 172.21.15.89:0/4076520181 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728011830 con 0x7f473c09de50 2023-09-30T22:17:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f473b7fe700 1 -- 172.21.15.89:0/4076520181 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4728011990 con 0x7f473c09de50 2023-09-30T22:17:18.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f473b7fe700 1 -- 172.21.15.89:0/4076520181 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728011d20 con 0x7f473c09de50 2023-09-30T22:17:18.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f4742e3a700 1 -- 172.21.15.89:0/4076520181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c09de50 msgr2=0x7f473c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f4742e3a700 1 --2- 172.21.15.89:0/4076520181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c09de50 0x7f473c09e220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4728009920 tx=0x7f4728009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:18.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f4742e3a700 1 -- 172.21.15.89:0/4076520181 shutdown_connections 2023-09-30T22:17:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f4742e3a700 1 --2- 172.21.15.89:0/4076520181 >> v2:172.21.15.89:3300/0 conn(0x7f473c09e760 0x7f473c13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f4742e3a700 1 --2- 172.21.15.89:0/4076520181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c1338a0 0x7f473c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f4742e3a700 1 --2- 172.21.15.89:0/4076520181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c09de50 0x7f473c09e220 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.666+0000 7f4742e3a700 1 -- 172.21.15.89:0/4076520181 >> 172.21.15.89:0/4076520181 conn(0x7f473c0a2aa0 msgr2=0x7f473c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:18.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.667+0000 7f4742e3a700 1 -- 172.21.15.89:0/4076520181 shutdown_connections 2023-09-30T22:17:18.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.667+0000 7f4742e3a700 1 -- 172.21.15.89:0/4076520181 wait complete. 2023-09-30T22:17:18.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.667+0000 7f4742e3a700 1 Processor -- start 2023-09-30T22:17:18.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f4742e3a700 1 -- start start 2023-09-30T22:17:18.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f4742e3a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c09de50 0x7f473c11ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f4742e3a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f473c09e760 0x7f473c11ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f4742e3a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c1338a0 0x7f473c11f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f4742e3a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c122280 con 0x7f473c09e760 2023-09-30T22:17:18.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f4742e3a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c120760 con 0x7f473c09de50 2023-09-30T22:17:18.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f4742e3a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f473c120e90 con 0x7f473c1338a0 2023-09-30T22:17:18.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f47413d7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c1338a0 0x7f473c11f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f47413d7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c1338a0 0x7f473c11f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57726/0 (socket says 172.21.15.89:57726) 2023-09-30T22:17:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f47413d7700 1 -- 172.21.15.89:0/1523371242 learned_addr learned my addr 172.21.15.89:0/1523371242 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f473bfff700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f473c09e760 0x7f473c11ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.668+0000 7f4740bd6700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c09de50 0x7f473c11ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47413d7700 1 -- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c09de50 msgr2=0x7f473c11ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47413d7700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c09de50 0x7f473c11ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47413d7700 1 -- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f473c09e760 msgr2=0x7f473c11ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47413d7700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f473c09e760 0x7f473c11ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47413d7700 1 -- 172.21.15.89:0/1523371242 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728009550 con 0x7f473c1338a0 2023-09-30T22:17:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f473bfff700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f473c09e760 0x7f473c11ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:18.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47413d7700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c1338a0 0x7f473c11f4c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4734007e70 tx=0x7f473400e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47397fa700 1 -- 172.21.15.89:0/1523371242 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400ede0 con 0x7f473c1338a0 2023-09-30T22:17:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47397fa700 1 -- 172.21.15.89:0/1523371242 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4734014420 con 0x7f473c1338a0 2023-09-30T22:17:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f47397fa700 1 -- 172.21.15.89:0/1523371242 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340147d0 con 0x7f473c1338a0 2023-09-30T22:17:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c0a1160 con 0x7f473c1338a0 2023-09-30T22:17:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.669+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c122cf0 con 0x7f473c1338a0 2023-09-30T22:17:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.670+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c074ea0 con 0x7f473c1338a0 2023-09-30T22:17:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.671+0000 7f47397fa700 1 -- 172.21.15.89:0/1523371242 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4734014930 con 0x7f473c1338a0 2023-09-30T22:17:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.671+0000 7f47397fa700 1 -- 172.21.15.89:0/1523371242 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4734040d20 con 0x7f473c1338a0 2023-09-30T22:17:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.675+0000 7f47397fa700 1 -- 172.21.15.89:0/1523371242 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4734013d50 con 0x7f473c1338a0 2023-09-30T22:17:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.698+0000 7f8cef4a8700 1 Processor -- start 2023-09-30T22:17:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.698+0000 7f8cef4a8700 1 -- start start 2023-09-30T22:17:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8cef4a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce81338a0 0x7f8ce8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8cef4a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ce809de50 0x7f8ce809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8cef4a8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce806ef30 con 0x7f8ce809e760 2023-09-30T22:17:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8cef4a8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce806fef0 con 0x7f8ce809de50 2023-09-30T22:17:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8cef4a8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81334b0 con 0x7f8ce81338a0 2023-09-30T22:17:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8ceca43700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ce809de50 0x7f8ce809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8ceca43700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ce809de50 0x7f8ce809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43646/0 (socket says 172.21.15.89:43646) 2023-09-30T22:17:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8ced244700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce81338a0 0x7f8ce8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8ceca43700 1 -- 172.21.15.89:0/2995536250 learned_addr learned my addr 172.21.15.89:0/2995536250 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8ceca43700 1 -- 172.21.15.89:0/2995536250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce81338a0 msgr2=0x7f8ce8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8ceca43700 1 --2- 172.21.15.89:0/2995536250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce81338a0 0x7f8ce8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8ceca43700 1 -- 172.21.15.89:0/2995536250 >> v1:172.21.15.89:6789/0 conn(0x7f8ce809e760 legacy=0x7f8ce813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.699+0000 7f8ceca43700 1 -- 172.21.15.89:0/2995536250 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce806fc30 con 0x7f8ce809de50 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.700+0000 7f8ceca43700 1 --2- 172.21.15.89:0/2995536250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809de50 0x7f8ce809e220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80098f0 tx=0x7f8cd8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d18c5a07675a5f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.700+0000 7f8cdffff700 1 -- 172.21.15.89:0/2995536250 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8011830 con 0x7f8ce809de50 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.700+0000 7f8cdffff700 1 -- 172.21.15.89:0/2995536250 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd8011990 con 0x7f8ce809de50 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.700+0000 7f8cdffff700 1 -- 172.21.15.89:0/2995536250 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8011d20 con 0x7f8ce809de50 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.700+0000 7f8cef4a8700 1 -- 172.21.15.89:0/2995536250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809de50 msgr2=0x7f8ce809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.700+0000 7f8cef4a8700 1 --2- 172.21.15.89:0/2995536250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809de50 0x7f8ce809e220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80098f0 tx=0x7f8cd8009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.701+0000 7f8cef4a8700 1 -- 172.21.15.89:0/2995536250 shutdown_connections 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.701+0000 7f8cef4a8700 1 --2- 172.21.15.89:0/2995536250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce81338a0 0x7f8ce8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.701+0000 7f8cef4a8700 1 --2- 172.21.15.89:0/2995536250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809de50 0x7f8ce809e220 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.701+0000 7f8cef4a8700 1 -- 172.21.15.89:0/2995536250 >> 172.21.15.89:0/2995536250 conn(0x7f8ce80a2aa0 msgr2=0x7f8ce80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.701+0000 7f8cef4a8700 1 -- 172.21.15.89:0/2995536250 shutdown_connections 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.701+0000 7f8cef4a8700 1 -- 172.21.15.89:0/2995536250 wait complete. 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8cef4a8700 1 Processor -- start 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8cef4a8700 1 -- start start 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8cef4a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce809de50 0x7f8ce811d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8cef4a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809e760 0x7f8ce811dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8cef4a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ce81338a0 0x7f8ce811e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8cef4a8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81211b0 con 0x7f8ce809e760 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8cef4a8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce811f340 con 0x7f8ce809de50 2023-09-30T22:17:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8cef4a8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80a1040 con 0x7f8ce81338a0 2023-09-30T22:17:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8ced244700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce809de50 0x7f8ce811d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8ced244700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce809de50 0x7f8ce811d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56608/0 (socket says 172.21.15.89:56608) 2023-09-30T22:17:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8ced244700 1 -- 172.21.15.89:0/891151213 learned_addr learned my addr 172.21.15.89:0/891151213 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.702+0000 7f8ceca43700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809e760 0x7f8ce811dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ceda45700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ce81338a0 0x7f8ce811e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ced244700 1 -- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ce81338a0 msgr2=0x7f8ce811e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ced244700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ce81338a0 0x7f8ce811e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ced244700 1 -- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809e760 msgr2=0x7f8ce811dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ced244700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809e760 0x7f8ce811dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ced244700 1 -- 172.21.15.89:0/891151213 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd8009550 con 0x7f8ce809de50 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ceda45700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ce81338a0 0x7f8ce811e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ceca43700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809e760 0x7f8ce811dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8ced244700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce809de50 0x7f8ce811d690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400e9b0 tx=0x7f8ce400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8cddffb700 1 -- 172.21.15.89:0/891151213 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4009710 con 0x7f8ce809de50 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8cddffb700 1 -- 172.21.15.89:0/891151213 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce400c850 con 0x7f8ce809de50 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8cddffb700 1 -- 172.21.15.89:0/891151213 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400cbc0 con 0x7f8ce809de50 2023-09-30T22:17:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce806f640 con 0x7f8ce809de50 2023-09-30T22:17:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.703+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce81205c0 con 0x7f8ce809de50 2023-09-30T22:17:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.705+0000 7f8cddffb700 1 -- 172.21.15.89:0/891151213 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ce400cd20 con 0x7f8ce809de50 2023-09-30T22:17:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.705+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce8074ea0 con 0x7f8ce809de50 2023-09-30T22:17:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.705+0000 7f8cddffb700 1 -- 172.21.15.89:0/891151213 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8ce403f810 con 0x7f8ce809de50 2023-09-30T22:17:18.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.709+0000 7f8cddffb700 1 -- 172.21.15.89:0/891151213 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ce40232f0 con 0x7f8ce809de50 2023-09-30T22:17:18.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.717+0000 7f90ab0d8700 1 Processor -- start 2023-09-30T22:17:18.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.717+0000 7f90ab0d8700 1 -- start start 2023-09-30T22:17:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90ab0d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90a412cf60 0x7f90a406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90ab0d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90ab0d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a412cd90 con 0x7f90a4130260 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90ab0d8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a412d330 con 0x7f90a412cf60 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90ab0d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a41306d0 con 0x7f90a412f8b0 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90a8e74700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90a412cf60 0x7f90a406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90a8e74700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90a412cf60 0x7f90a406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43660/0 (socket says 172.21.15.89:43660) 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90a8e74700 1 -- 172.21.15.89:0/2568714429 learned_addr learned my addr 172.21.15.89:0/2568714429 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.718+0000 7f90a3fff700 1 --2- 172.21.15.89:0/2568714429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a3fff700 1 -- 172.21.15.89:0/2568714429 >> v2:172.21.15.89:3300/0 conn(0x7f90a412cf60 msgr2=0x7f90a406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a3fff700 1 --2- 172.21.15.89:0/2568714429 >> v2:172.21.15.89:3300/0 conn(0x7f90a412cf60 0x7f90a406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a3fff700 1 -- 172.21.15.89:0/2568714429 >> v1:172.21.15.89:6789/0 conn(0x7f90a4130260 legacy=0x7f90a4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a3fff700 1 -- 172.21.15.89:0/2568714429 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a4071780 con 0x7f90a412f8b0 2023-09-30T22:17:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a37fe700 1 -- 172.21.15.89:0/2568714429 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985018130 0 0) 0x7f90a412cd90 con 0x7f90a4130260 2023-09-30T22:17:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a8e74700 1 --2- 172.21.15.89:0/2568714429 >> v2:172.21.15.89:3300/0 conn(0x7f90a412cf60 0x7f90a406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a3fff700 1 --2- 172.21.15.89:0/2568714429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a412fd20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9090009840 tx=0x7f9090009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c887e3f500f1a201 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a37fe700 1 -- 172.21.15.89:0/2568714429 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090010800 con 0x7f90a412f8b0 2023-09-30T22:17:18.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a37fe700 1 -- 172.21.15.89:0/2568714429 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090010960 con 0x7f90a412f8b0 2023-09-30T22:17:18.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.719+0000 7f90a37fe700 1 -- 172.21.15.89:0/2568714429 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090010cf0 con 0x7f90a412f8b0 2023-09-30T22:17:18.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.720+0000 7f90ab0d8700 1 -- 172.21.15.89:0/2568714429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 msgr2=0x7f90a412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.720+0000 7f90ab0d8700 1 --2- 172.21.15.89:0/2568714429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a412fd20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9090009840 tx=0x7f9090009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:18.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.720+0000 7f90ab0d8700 1 -- 172.21.15.89:0/2568714429 shutdown_connections 2023-09-30T22:17:18.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.720+0000 7f90ab0d8700 1 --2- 172.21.15.89:0/2568714429 >> v2:172.21.15.89:3300/0 conn(0x7f90a412cf60 0x7f90a406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.720+0000 7f90ab0d8700 1 --2- 172.21.15.89:0/2568714429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a412fd20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.720+0000 7f90ab0d8700 1 -- 172.21.15.89:0/2568714429 >> 172.21.15.89:0/2568714429 conn(0x7f90a40a2aa0 msgr2=0x7f90a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:18.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.720+0000 7f90ab0d8700 1 -- 172.21.15.89:0/2568714429 shutdown_connections 2023-09-30T22:17:18.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.720+0000 7f90ab0d8700 1 -- 172.21.15.89:0/2568714429 wait complete. 2023-09-30T22:17:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.721+0000 7f90ab0d8700 1 Processor -- start 2023-09-30T22:17:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.721+0000 7f90ab0d8700 1 -- start start 2023-09-30T22:17:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90ab0d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90a412cf60 0x7f90a409daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90ab0d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a409e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90ab0d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90a4130260 0x7f90a41c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:18.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90ab0d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4119700 con 0x7f90a412cf60 2023-09-30T22:17:18.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90ab0d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4118fb0 con 0x7f90a412f8b0 2023-09-30T22:17:18.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90ab0d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90a411ab70 con 0x7f90a4130260 2023-09-30T22:17:18.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90a9675700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90a4130260 0x7f90a41c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90a9675700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90a4130260 0x7f90a41c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57740/0 (socket says 172.21.15.89:57740) 2023-09-30T22:17:18.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90a9675700 1 -- 172.21.15.89:0/383686797 learned_addr learned my addr 172.21.15.89:0/383686797 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:18.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90a8e74700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90a412cf60 0x7f90a409daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90a3fff700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a409e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:18.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90a9675700 1 -- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 msgr2=0x7f90a409e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.722+0000 7f90a9675700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a409e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90a9675700 1 -- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90a412cf60 msgr2=0x7f90a409daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90a9675700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90a412cf60 0x7f90a409daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90a9675700 1 -- 172.21.15.89:0/383686797 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9098009550 con 0x7f90a4130260 2023-09-30T22:17:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90a9675700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90a4130260 0x7f90a41c5f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f909c00c810 tx=0x7f909c00cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90a17fa700 1 -- 172.21.15.89:0/383686797 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c003e30 con 0x7f90a4130260 2023-09-30T22:17:18.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90a17fa700 1 -- 172.21.15.89:0/383686797 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c005070 con 0x7f90a4130260 2023-09-30T22:17:18.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90a17fa700 1 -- 172.21.15.89:0/383686797 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c005420 con 0x7f90a4130260 2023-09-30T22:17:18.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90900094a0 con 0x7f90a4130260 2023-09-30T22:17:18.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.723+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a4129fa0 con 0x7f90a4130260 2023-09-30T22:17:18.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.724+0000 7f90a17fa700 1 -- 172.21.15.89:0/383686797 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f909c005580 con 0x7f90a4130260 2023-09-30T22:17:18.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.724+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a4074ea0 con 0x7f90a4130260 2023-09-30T22:17:18.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.724+0000 7f90a17fa700 1 -- 172.21.15.89:0/383686797 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f909c03f230 con 0x7f90a4130260 2023-09-30T22:17:18.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.729+0000 7f90a17fa700 1 -- 172.21.15.89:0/383686797 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f909c005830 con 0x7f90a4130260 2023-09-30T22:17:18.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.783+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f800c1a82e0 con 0x7f800c09d6f0 2023-09-30T22:17:18.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.784+0000 7f8002ffd700 1 -- 172.21.15.89:0/3746670029 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f800c1a82e0 con 0x7f800c09d6f0 2023-09-30T22:17:18.784 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:18.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.786+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09d6f0 msgr2=0x7f800c1a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.786+0000 7f80142b4700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09d6f0 0x7f800c1a5130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f800800ebe0 tx=0x7f800800eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.787+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 shutdown_connections 2023-09-30T22:17:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.787+0000 7f80142b4700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f800c09e410 0x7f800c1c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.787+0000 7f80142b4700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f800c09d6f0 0x7f800c1a5130 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.787+0000 7f80142b4700 1 --2- 172.21.15.89:0/3746670029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f800c09e000 0x7f800c1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.787+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 >> 172.21.15.89:0/3746670029 conn(0x7f800c066f50 msgr2=0x7f800c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.787+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 shutdown_connections 2023-09-30T22:17:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.787+0000 7f80142b4700 1 -- 172.21.15.89:0/3746670029 wait complete. 2023-09-30T22:17:18.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:18.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.874+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f473c06b9e0 con 0x7f473c1338a0 2023-09-30T22:17:18.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.875+0000 7f47397fa700 1 -- 172.21.15.89:0/1523371242 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4734013b70 con 0x7f473c1338a0 2023-09-30T22:17:18.875 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.876+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c1338a0 msgr2=0x7f473c11f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.877+0000 7f4742e3a700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c1338a0 0x7f473c11f4c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4734007e70 tx=0x7f473400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.877+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 shutdown_connections 2023-09-30T22:17:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.877+0000 7f4742e3a700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f473c1338a0 0x7f473c11f4c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.877+0000 7f4742e3a700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f473c09de50 0x7f473c11ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.877+0000 7f4742e3a700 1 --2- 172.21.15.89:0/1523371242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f473c09e760 0x7f473c11ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.877+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 >> 172.21.15.89:0/1523371242 conn(0x7f473c0a2aa0 msgr2=0x7f473c09ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.877+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 shutdown_connections 2023-09-30T22:17:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.877+0000 7f4742e3a700 1 -- 172.21.15.89:0/1523371242 wait complete. 2023-09-30T22:17:18.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:18.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.905+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8ce806b9e0 con 0x7f8ce809de50 2023-09-30T22:17:18.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.906+0000 7f8cddffb700 1 -- 172.21.15.89:0/891151213 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ce401ae80 con 0x7f8ce809de50 2023-09-30T22:17:18.906 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:18.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce809de50 msgr2=0x7f8ce811d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce809de50 0x7f8ce811d690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400e9b0 tx=0x7f8ce400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 shutdown_connections 2023-09-30T22:17:18.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ce81338a0 0x7f8ce811e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ce809de50 0x7f8ce811d690 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 --2- 172.21.15.89:0/891151213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ce809e760 0x7f8ce811dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 >> 172.21.15.89:0/891151213 conn(0x7f8ce80a2aa0 msgr2=0x7f8ce81b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:18.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 shutdown_connections 2023-09-30T22:17:18.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.908+0000 7f8cef4a8700 1 -- 172.21.15.89:0/891151213 wait complete. 2023-09-30T22:17:18.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:18.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.925+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f90a40713a0 con 0x7f90a4130260 2023-09-30T22:17:18.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.925+0000 7f90a17fa700 1 -- 172.21.15.89:0/383686797 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f909c01a360 con 0x7f90a4130260 2023-09-30T22:17:18.925 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90a4130260 msgr2=0x7f90a41c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90a4130260 0x7f90a41c5f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f909c00c810 tx=0x7f909c00cb20 comp rx=0 tx=0).stop 2023-09-30T22:17:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 shutdown_connections 2023-09-30T22:17:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90a4130260 0x7f90a41c5f80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90a412cf60 0x7f90a409daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 --2- 172.21.15.89:0/383686797 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90a412f8b0 0x7f90a409e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 >> 172.21.15.89:0/383686797 conn(0x7f90a40a2aa0 msgr2=0x7f90a41b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 shutdown_connections 2023-09-30T22:17:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:18.927+0000 7f90ab0d8700 1 -- 172.21.15.89:0/383686797 wait complete. 2023-09-30T22:17:18.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:19.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:19.890 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:19.921 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:19.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.930+0000 7f02d0adc700 1 Processor -- start 2023-09-30T22:17:19.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.930+0000 7f02d0adc700 1 -- start start 2023-09-30T22:17:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.930+0000 7f02d0adc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc133420 0x7f02cc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.930+0000 7f02d0adc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.930+0000 7f02d0adc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f02cc134260 0x7f02cc1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.930+0000 7f02d0adc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02cc06b990 con 0x7f02cc134260 2023-09-30T22:17:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.930+0000 7f02d0adc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc0a0590 con 0x7f02cc133420 2023-09-30T22:17:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.930+0000 7f02d0adc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc09fb80 con 0x7f02cc133e30 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02ca59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc133420 0x7f02cc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57748/0 (socket says 172.21.15.89:57748) 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02cad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f02cc134260 0x7f02cc1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02c9d9b700 1 -- 172.21.15.89:0/2210591782 learned_addr learned my addr 172.21.15.89:0/2210591782 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02ca59c700 1 -- 172.21.15.89:0/2210591782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 msgr2=0x7f02cc12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02ca59c700 1 --2- 172.21.15.89:0/2210591782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02ca59c700 1 -- 172.21.15.89:0/2210591782 >> v2:172.21.15.89:3300/0 conn(0x7f02cc134260 msgr2=0x7f02cc1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02ca59c700 1 --2- 172.21.15.89:0/2210591782 >> v2:172.21.15.89:3300/0 conn(0x7f02cc134260 0x7f02cc1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02ca59c700 1 -- 172.21.15.89:0/2210591782 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc0a0030 con 0x7f02cc133420 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02c9d9b700 1 --2- 172.21.15.89:0/2210591782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02cad9d700 1 --2- 172.21.15.89:0/2210591782 >> v2:172.21.15.89:3300/0 conn(0x7f02cc134260 0x7f02cc1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02ca59c700 1 --2- 172.21.15.89:0/2210591782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc133420 0x7f02cc1337f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009840 tx=0x7f02bc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=464f69a37604827b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02c959a700 1 -- 172.21.15.89:0/2210591782 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc011830 con 0x7f02cc133420 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02c959a700 1 -- 172.21.15.89:0/2210591782 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc011990 con 0x7f02cc133420 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.931+0000 7f02c959a700 1 -- 172.21.15.89:0/2210591782 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc011d20 con 0x7f02cc133420 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 -- 172.21.15.89:0/2210591782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc133420 msgr2=0x7f02cc1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 --2- 172.21.15.89:0/2210591782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc133420 0x7f02cc1337f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009840 tx=0x7f02bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 -- 172.21.15.89:0/2210591782 shutdown_connections 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 --2- 172.21.15.89:0/2210591782 >> v2:172.21.15.89:3300/0 conn(0x7f02cc134260 0x7f02cc1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 --2- 172.21.15.89:0/2210591782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc133420 0x7f02cc1337f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 --2- 172.21.15.89:0/2210591782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 -- 172.21.15.89:0/2210591782 >> 172.21.15.89:0/2210591782 conn(0x7f02cc0a4210 msgr2=0x7f02cc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 -- 172.21.15.89:0/2210591782 shutdown_connections 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.932+0000 7f02d0adc700 1 -- 172.21.15.89:0/2210591782 wait complete. 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.933+0000 7f02d0adc700 1 Processor -- start 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.933+0000 7f02d0adc700 1 -- start start 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02d0adc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02cc133420 0x7f02cc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02d0adc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02d0adc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc134260 0x7f02cc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02d0adc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc120230 con 0x7f02cc133420 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02d0adc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc11fde0 con 0x7f02cc134260 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02d0adc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc11ee20 con 0x7f02cc133e30 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57758/0 (socket says 172.21.15.89:57758) 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02c9d9b700 1 -- 172.21.15.89:0/2938067108 learned_addr learned my addr 172.21.15.89:0/2938067108 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02ca59c700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02cc133420 0x7f02cc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02cad9d700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc134260 0x7f02cc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02ca59c700 1 -- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 msgr2=0x7f02cc09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02ca59c700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02ca59c700 1 -- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc134260 msgr2=0x7f02cc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02ca59c700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc134260 0x7f02cc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02ca59c700 1 -- 172.21.15.89:0/2938067108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0009550 con 0x7f02cc133420 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02c9d9b700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02cad9d700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc134260 0x7f02cc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.934+0000 7f02ca59c700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02cc133420 0x7f02cc09c480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f02bc016000 tx=0x7f02bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.935+0000 7f02baffd700 1 -- 172.21.15.89:0/2938067108 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc004240 con 0x7f02cc133420 2023-09-30T22:17:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.935+0000 7f02baffd700 1 -- 172.21.15.89:0/2938067108 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc01f070 con 0x7f02cc133420 2023-09-30T22:17:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.935+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c4009a70 con 0x7f02cc133420 2023-09-30T22:17:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.935+0000 7f02baffd700 1 -- 172.21.15.89:0/2938067108 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0045e0 con 0x7f02cc133420 2023-09-30T22:17:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.935+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc0094a0 con 0x7f02cc133420 2023-09-30T22:17:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.935+0000 7f02baffd700 1 -- 172.21.15.89:0/2938067108 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02bc021030 con 0x7f02cc133420 2023-09-30T22:17:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.936+0000 7f02baffd700 1 -- 172.21.15.89:0/2938067108 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f02bc040950 con 0x7f02cc133420 2023-09-30T22:17:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.936+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc120020 con 0x7f02cc133420 2023-09-30T22:17:19.939 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:19.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:19.941+0000 7f02baffd700 1 -- 172.21.15.89:0/2938067108 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02bc015080 con 0x7f02cc133420 2023-09-30T22:17:20.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.031+0000 7f118cecb700 1 Processor -- start 2023-09-30T22:17:20.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.031+0000 7f118cecb700 1 -- start start 2023-09-30T22:17:20.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f118cecb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f11880a1510 0x7f1188138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f118cecb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f118813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f118cecb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118809edd0 con 0x7f11880a0a30 2023-09-30T22:17:20.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f118cecb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880a18e0 con 0x7f11880a1510 2023-09-30T22:17:20.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f118cecb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118809e1f0 con 0x7f1188139250 2023-09-30T22:17:20.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f118659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f11880a0a30 0x7f11880a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52914/0 (socket says 172.21.15.89:52914) 2023-09-30T22:17:20.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f118659c700 1 -- 172.21.15.89:0/910586992 learned_addr learned my addr 172.21.15.89:0/910586992 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f1185d9b700 1 --2- 172.21.15.89:0/910586992 >> v2:172.21.15.89:3300/0 conn(0x7f11880a1510 0x7f1188138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f1186d9d700 1 --2- 172.21.15.89:0/910586992 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f118813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.032+0000 7f118559a700 1 -- 172.21.15.89:0/910586992 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259094346 0 0) 0x7f118809edd0 con 0x7f11880a0a30 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f118559a700 1 -- 172.21.15.89:0/910586992 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11700035b0 con 0x7f11880a0a30 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f1186d9d700 1 -- 172.21.15.89:0/910586992 >> v2:172.21.15.89:3300/0 conn(0x7f11880a1510 msgr2=0x7f1188138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f1186d9d700 1 --2- 172.21.15.89:0/910586992 >> v2:172.21.15.89:3300/0 conn(0x7f11880a1510 0x7f1188138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f1186d9d700 1 -- 172.21.15.89:0/910586992 >> v1:172.21.15.89:6789/0 conn(0x7f11880a0a30 legacy=0x7f11880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f1186d9d700 1 -- 172.21.15.89:0/910586992 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118809dc50 con 0x7f1188139250 2023-09-30T22:17:20.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f1186d9d700 1 --2- 172.21.15.89:0/910586992 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f118813b630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1180009d90 tx=0x7f118000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e8855d3b754bcaa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:20.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f118559a700 1 -- 172.21.15.89:0/910586992 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000e070 con 0x7f1188139250 2023-09-30T22:17:20.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f118559a700 1 -- 172.21.15.89:0/910586992 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118000bb20 con 0x7f1188139250 2023-09-30T22:17:20.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.033+0000 7f118559a700 1 -- 172.21.15.89:0/910586992 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180012430 con 0x7f1188139250 2023-09-30T22:17:20.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.034+0000 7f118cecb700 1 -- 172.21.15.89:0/910586992 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 msgr2=0x7f118813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.034+0000 7f118cecb700 1 --2- 172.21.15.89:0/910586992 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f118813b630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1180009d90 tx=0x7f118000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.034+0000 7f118cecb700 1 -- 172.21.15.89:0/910586992 shutdown_connections 2023-09-30T22:17:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.034+0000 7f118cecb700 1 --2- 172.21.15.89:0/910586992 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f118813b630 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.034+0000 7f118cecb700 1 --2- 172.21.15.89:0/910586992 >> v2:172.21.15.89:3300/0 conn(0x7f11880a1510 0x7f1188138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.034+0000 7f118cecb700 1 -- 172.21.15.89:0/910586992 >> 172.21.15.89:0/910586992 conn(0x7f11880a6550 msgr2=0x7f11880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.034+0000 7f118cecb700 1 -- 172.21.15.89:0/910586992 shutdown_connections 2023-09-30T22:17:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.034+0000 7f118cecb700 1 -- 172.21.15.89:0/910586992 wait complete. 2023-09-30T22:17:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f118cecb700 1 Processor -- start 2023-09-30T22:17:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f118cecb700 1 -- start start 2023-09-30T22:17:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f118cecb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11880a0a30 0x7f1188122740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f118cecb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11880a1510 0x7f1188122c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f118cecb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f11881231c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f118cecb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880a0220 con 0x7f11880a0a30 2023-09-30T22:17:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f118cecb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11881259d0 con 0x7f1188139250 2023-09-30T22:17:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f118cecb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11881385a0 con 0x7f11880a1510 2023-09-30T22:17:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f1185d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11880a1510 0x7f1188122c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f1185d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11880a1510 0x7f1188122c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57766/0 (socket says 172.21.15.89:57766) 2023-09-30T22:17:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.036+0000 7f1185d9b700 1 -- 172.21.15.89:0/4076550246 learned_addr learned my addr 172.21.15.89:0/4076550246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f118659c700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11880a0a30 0x7f1188122740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1186d9d700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f11881231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1185d9b700 1 -- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 msgr2=0x7f11881231c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1185d9b700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f11881231c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1185d9b700 1 -- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11880a0a30 msgr2=0x7f1188122740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1185d9b700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11880a0a30 0x7f1188122740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1185d9b700 1 -- 172.21.15.89:0/4076550246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180009a70 con 0x7f11880a1510 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1185d9b700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11880a1510 0x7f1188122c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f117c00e9b0 tx=0x7f117c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1176ffd700 1 -- 172.21.15.89:0/4076550246 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00ee50 con 0x7f11880a1510 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1176ffd700 1 -- 172.21.15.89:0/4076550246 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00c850 con 0x7f11880a1510 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1188124ae0 con 0x7f11880a1510 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f1176ffd700 1 -- 172.21.15.89:0/4076550246 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00cbc0 con 0x7f11880a1510 2023-09-30T22:17:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.037+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1188126960 con 0x7f11880a1510 2023-09-30T22:17:20.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.038+0000 7f1176ffd700 1 -- 172.21.15.89:0/4076550246 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f117c00cd20 con 0x7f11880a1510 2023-09-30T22:17:20.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.038+0000 7f1176ffd700 1 -- 172.21.15.89:0/4076550246 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f117c03fa90 con 0x7f11880a1510 2023-09-30T22:17:20.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.039+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1188074ea0 con 0x7f11880a1510 2023-09-30T22:17:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.047+0000 7f1176ffd700 1 -- 172.21.15.89:0/4076550246 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f117c01b3b0 con 0x7f11880a1510 2023-09-30T22:17:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.059+0000 7fa28de11700 1 Processor -- start 2023-09-30T22:17:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.059+0000 7fa28de11700 1 -- start start 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa28de11700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa28809e1d0 0x7fa288138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa28de11700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa288139170 0x7fa28813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa28de11700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28809b120 con 0x7fa28809d6f0 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa28de11700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28806b460 con 0x7fa288139170 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa28de11700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28809a7e0 con 0x7fa28809e1d0 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa287fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa288139170 0x7fa28813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa286ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa28809e1d0 0x7fa288138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa287fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa288139170 0x7fa28813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43692/0 (socket says 172.21.15.89:43692) 2023-09-30T22:17:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa287fff700 1 -- 172.21.15.89:0/3145129204 learned_addr learned my addr 172.21.15.89:0/3145129204 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa287fff700 1 -- 172.21.15.89:0/3145129204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa28809e1d0 msgr2=0x7fa288138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa287fff700 1 --2- 172.21.15.89:0/3145129204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa28809e1d0 0x7fa288138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa287fff700 1 -- 172.21.15.89:0/3145129204 >> v1:172.21.15.89:6789/0 conn(0x7fa28809d6f0 legacy=0x7fa28809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa287fff700 1 -- 172.21.15.89:0/3145129204 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28809a500 con 0x7fa288139170 2023-09-30T22:17:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa287fff700 1 --2- 172.21.15.89:0/3145129204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa288139170 0x7fa28813b550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa280009e50 tx=0x7fa28000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234886d2cff557d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:20.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.060+0000 7fa2867fc700 1 -- 172.21.15.89:0/3145129204 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28000e070 con 0x7fa288139170 2023-09-30T22:17:20.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa2867fc700 1 -- 172.21.15.89:0/3145129204 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa280003a20 con 0x7fa288139170 2023-09-30T22:17:20.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa2867fc700 1 -- 172.21.15.89:0/3145129204 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280003db0 con 0x7fa288139170 2023-09-30T22:17:20.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa28de11700 1 -- 172.21.15.89:0/3145129204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa288139170 msgr2=0x7fa28813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa28de11700 1 --2- 172.21.15.89:0/3145129204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa288139170 0x7fa28813b550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa280009e50 tx=0x7fa28000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:20.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa28de11700 1 -- 172.21.15.89:0/3145129204 shutdown_connections 2023-09-30T22:17:20.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa28de11700 1 --2- 172.21.15.89:0/3145129204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa288139170 0x7fa28813b550 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa28de11700 1 --2- 172.21.15.89:0/3145129204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa28809e1d0 0x7fa288138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa28de11700 1 -- 172.21.15.89:0/3145129204 >> 172.21.15.89:0/3145129204 conn(0x7fa288066f50 msgr2=0x7fa28809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa28de11700 1 -- 172.21.15.89:0/3145129204 shutdown_connections 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.061+0000 7fa28de11700 1 -- 172.21.15.89:0/3145129204 wait complete. 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.062+0000 7fa28de11700 1 Processor -- start 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.062+0000 7fa28de11700 1 -- start start 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa28de11700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa28809d6f0 0x7fa288119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa28de11700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa28809e1d0 0x7fa2881199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa28de11700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa288139170 0x7fa288119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa28de11700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28811ace0 con 0x7fa28809d6f0 2023-09-30T22:17:20.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa28de11700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28809c710 con 0x7fa288139170 2023-09-30T22:17:20.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa28de11700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa288138cd0 con 0x7fa28809e1d0 2023-09-30T22:17:20.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa28809d6f0 0x7fa288119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa287fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa288139170 0x7fa288119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa28809d6f0 0x7fa288119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43698/0 (socket says 172.21.15.89:43698) 2023-09-30T22:17:20.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 -- 172.21.15.89:0/1097379388 learned_addr learned my addr 172.21.15.89:0/1097379388 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:20.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 -- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa28809e1d0 msgr2=0x7fa2881199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 --2- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa28809e1d0 0x7fa2881199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 -- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa288139170 msgr2=0x7fa288119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 --2- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa288139170 0x7fa288119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 -- 172.21.15.89:0/1097379388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa280009b30 con 0x7fa28809d6f0 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa287fff700 1 --2- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa288139170 0x7fa288119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa2877fe700 1 --2- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa28809d6f0 0x7fa288119480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c4d0 tx=0x7fa27800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa26ffff700 1 -- 172.21.15.89:0/1097379388 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278011790 con 0x7fa28809d6f0 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.063+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2880a3040 con 0x7fa28809d6f0 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.064+0000 7fa26ffff700 1 -- 172.21.15.89:0/1097379388 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2780118f0 con 0x7fa28809d6f0 2023-09-30T22:17:20.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.064+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28811a7b0 con 0x7fa28809d6f0 2023-09-30T22:17:20.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.064+0000 7fa26ffff700 1 -- 172.21.15.89:0/1097379388 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278011ca0 con 0x7fa28809d6f0 2023-09-30T22:17:20.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.064+0000 7fa26ffff700 1 -- 172.21.15.89:0/1097379388 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa27801c460 con 0x7fa28809d6f0 2023-09-30T22:17:20.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.065+0000 7fa26ffff700 1 -- 172.21.15.89:0/1097379388 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa278040df0 con 0x7fa28809d6f0 2023-09-30T22:17:20.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.065+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2680052e0 con 0x7fa28809d6f0 2023-09-30T22:17:20.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.074+0000 7fa26ffff700 1 -- 172.21.15.89:0/1097379388 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa278044060 con 0x7fa28809d6f0 2023-09-30T22:17:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.097+0000 7f86609a8700 1 Processor -- start 2023-09-30T22:17:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.097+0000 7f86609a8700 1 -- start start 2023-09-30T22:17:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f86609a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09e1d0 0x7f865c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f86609a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f865c135e50 0x7f865c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f86609a8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c09f130 con 0x7f865c09d6f0 2023-09-30T22:17:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f86609a8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c09f520 con 0x7f865c135e50 2023-09-30T22:17:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f86609a8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c09e930 con 0x7f865c09e1d0 2023-09-30T22:17:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f8659d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09e1d0 0x7f865c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f8659d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09e1d0 0x7f865c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56710/0 (socket says 172.21.15.89:56710) 2023-09-30T22:17:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f8659d9b700 1 -- 172.21.15.89:0/1769475789 learned_addr learned my addr 172.21.15.89:0/1769475789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f865ad9d700 1 --2- 172.21.15.89:0/1769475789 >> v2:172.21.15.89:3300/0 conn(0x7f865c135e50 0x7f865c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f865ad9d700 1 -- 172.21.15.89:0/1769475789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09e1d0 msgr2=0x7f865c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f865ad9d700 1 --2- 172.21.15.89:0/1769475789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09e1d0 0x7f865c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f865ad9d700 1 -- 172.21.15.89:0/1769475789 >> v1:172.21.15.89:6789/0 conn(0x7f865c09d6f0 legacy=0x7f865c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f865ad9d700 1 -- 172.21.15.89:0/1769475789 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c09b2d0 con 0x7f865c135e50 2023-09-30T22:17:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f8659d9b700 1 --2- 172.21.15.89:0/1769475789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09e1d0 0x7f865c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f865ad9d700 1 --2- 172.21.15.89:0/1769475789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865c135e50 0x7f865c138230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8654009dc0 tx=0x7f865400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1df48608c980738 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.098+0000 7f865959a700 1 -- 172.21.15.89:0/1769475789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400e070 con 0x7f865c135e50 2023-09-30T22:17:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f865959a700 1 -- 172.21.15.89:0/1769475789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865400bae0 con 0x7f865c135e50 2023-09-30T22:17:20.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f865959a700 1 -- 172.21.15.89:0/1769475789 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400be90 con 0x7f865c135e50 2023-09-30T22:17:20.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f86609a8700 1 -- 172.21.15.89:0/1769475789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865c135e50 msgr2=0x7f865c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f86609a8700 1 --2- 172.21.15.89:0/1769475789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865c135e50 0x7f865c138230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8654009dc0 tx=0x7f865400b330 comp rx=0 tx=0).stop 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f86609a8700 1 -- 172.21.15.89:0/1769475789 shutdown_connections 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f86609a8700 1 --2- 172.21.15.89:0/1769475789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865c135e50 0x7f865c138230 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f86609a8700 1 --2- 172.21.15.89:0/1769475789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09e1d0 0x7f865c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f86609a8700 1 -- 172.21.15.89:0/1769475789 >> 172.21.15.89:0/1769475789 conn(0x7f865c066f50 msgr2=0x7f865c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f86609a8700 1 -- 172.21.15.89:0/1769475789 shutdown_connections 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.099+0000 7f86609a8700 1 -- 172.21.15.89:0/1769475789 wait complete. 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.100+0000 7f86609a8700 1 Processor -- start 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.100+0000 7f86609a8700 1 -- start start 2023-09-30T22:17:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f86609a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09d6f0 0x7f865c1c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f86609a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865c09e1d0 0x7f865c1c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f86609a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f865c135e50 0x7f865c119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:20.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f86609a8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c11c820 con 0x7f865c09e1d0 2023-09-30T22:17:20.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f86609a8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1baf30 con 0x7f865c09d6f0 2023-09-30T22:17:20.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f86609a8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f865c11d240 con 0x7f865c135e50 2023-09-30T22:17:20.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09d6f0 0x7f865c1c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09d6f0 0x7f865c1c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56716/0 (socket says 172.21.15.89:56716) 2023-09-30T22:17:20.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 -- 172.21.15.89:0/182925069 learned_addr learned my addr 172.21.15.89:0/182925069 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865ad9d700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f865c135e50 0x7f865c119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 -- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f865c135e50 msgr2=0x7f865c119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f865c135e50 0x7f865c119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 -- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865c09e1d0 msgr2=0x7f865c1c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865c09e1d0 0x7f865c1c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 -- 172.21.15.89:0/182925069 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865000b7f0 con 0x7f865c09d6f0 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865ad9d700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f865c135e50 0x7f865c119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.101+0000 7f865a59c700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09d6f0 0x7f865c1c4380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c510 tx=0x7f864c00c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.102+0000 7f864affd700 1 -- 172.21.15.89:0/182925069 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0052d0 con 0x7f865c09d6f0 2023-09-30T22:17:20.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.102+0000 7f864affd700 1 -- 172.21.15.89:0/182925069 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c005430 con 0x7f865c09d6f0 2023-09-30T22:17:20.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.102+0000 7f864affd700 1 -- 172.21.15.89:0/182925069 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0038c0 con 0x7f865c09d6f0 2023-09-30T22:17:20.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.102+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8654009a70 con 0x7f865c09d6f0 2023-09-30T22:17:20.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.102+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c09c700 con 0x7f865c09d6f0 2023-09-30T22:17:20.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.103+0000 7f864affd700 1 -- 172.21.15.89:0/182925069 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f864c003a20 con 0x7f865c09d6f0 2023-09-30T22:17:20.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.103+0000 7f864affd700 1 -- 172.21.15.89:0/182925069 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f864c03ec20 con 0x7f865c09d6f0 2023-09-30T22:17:20.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.105+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c130320 con 0x7f865c09d6f0 2023-09-30T22:17:20.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.113+0000 7f864affd700 1 -- 172.21.15.89:0/182925069 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f864c017080 con 0x7f865c09d6f0 2023-09-30T22:17:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.151+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f02cc074ea0 con 0x7f02cc133420 2023-09-30T22:17:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.152+0000 7f02baffd700 1 -- 172.21.15.89:0/2938067108 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02bc01a450 con 0x7f02cc133420 2023-09-30T22:17:20.152 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:20.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02cc133420 msgr2=0x7f02cc09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02cc133420 0x7f02cc09c480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f02bc016000 tx=0x7f02bc004750 comp rx=0 tx=0).stop 2023-09-30T22:17:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 shutdown_connections 2023-09-30T22:17:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02cc134260 0x7f02cc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02cc133420 0x7f02cc09c480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 --2- 172.21.15.89:0/2938067108 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02cc133e30 0x7f02cc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 >> 172.21.15.89:0/2938067108 conn(0x7f02cc0a4210 msgr2=0x7f02cc0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 shutdown_connections 2023-09-30T22:17:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.154+0000 7f02d0adc700 1 -- 172.21.15.89:0/2938067108 wait complete. 2023-09-30T22:17:20.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:20.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.267+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa268005150 con 0x7fa28809d6f0 2023-09-30T22:17:20.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.268+0000 7fa26ffff700 1 -- 172.21.15.89:0/1097379388 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa27803b090 con 0x7fa28809d6f0 2023-09-30T22:17:20.268 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa28809d6f0 msgr2=0x7fa288119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 --2- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa28809d6f0 0x7fa288119480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c4d0 tx=0x7fa27800c7e0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 shutdown_connections 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 --2- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa288139170 0x7fa288119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 --2- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa28809d6f0 0x7fa288119480 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 --2- 172.21.15.89:0/1097379388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa28809e1d0 0x7fa2881199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 >> 172.21.15.89:0/1097379388 conn(0x7fa288066f50 msgr2=0x7fa288070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 shutdown_connections 2023-09-30T22:17:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.270+0000 7fa28de11700 1 -- 172.21.15.89:0/1097379388 wait complete. 2023-09-30T22:17:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.272+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f11880a28e0 con 0x7f11880a1510 2023-09-30T22:17:20.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.273+0000 7f1176ffd700 1 -- 172.21.15.89:0/4076550246 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f117c011350 con 0x7f11880a1510 2023-09-30T22:17:20.272 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11880a1510 msgr2=0x7f1188122c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11880a1510 0x7f1188122c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f117c00e9b0 tx=0x7f117c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 shutdown_connections 2023-09-30T22:17:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1188139250 0x7f11881231c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11880a0a30 0x7f1188122740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 --2- 172.21.15.89:0/4076550246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11880a1510 0x7f1188122c80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 >> 172.21.15.89:0/4076550246 conn(0x7f11880a6550 msgr2=0x7f118811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:20.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 shutdown_connections 2023-09-30T22:17:20.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.275+0000 7f118cecb700 1 -- 172.21.15.89:0/4076550246 wait complete. 2023-09-30T22:17:20.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:20.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:20.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.316+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f865c09ef50 con 0x7f865c09d6f0 2023-09-30T22:17:20.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.316+0000 7f864affd700 1 -- 172.21.15.89:0/182925069 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f864c018260 con 0x7f865c09d6f0 2023-09-30T22:17:20.316 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09d6f0 msgr2=0x7f865c1c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09d6f0 0x7f865c1c4380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c510 tx=0x7f864c00c820 comp rx=0 tx=0).stop 2023-09-30T22:17:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 shutdown_connections 2023-09-30T22:17:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f865c135e50 0x7f865c119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f865c09d6f0 0x7f865c1c4380 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 --2- 172.21.15.89:0/182925069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865c09e1d0 0x7f865c1c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:20.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 >> 172.21.15.89:0/182925069 conn(0x7f865c066f50 msgr2=0x7f865c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:20.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 shutdown_connections 2023-09-30T22:17:20.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:20.318+0000 7f86609a8700 1 -- 172.21.15.89:0/182925069 wait complete. 2023-09-30T22:17:20.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:21.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:21.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:21.288 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e74cc7700 1 Processor -- start 2023-09-30T22:17:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e74cc7700 1 -- start start 2023-09-30T22:17:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e74cc7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e7009e1d0 0x7f6e70138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e74cc7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e74cc7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7009b120 con 0x7f6e7009d6f0 2023-09-30T22:17:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e74cc7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7006b460 con 0x7f6e7009e1d0 2023-09-30T22:17:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e74cc7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7009a7e0 con 0x7f6e70139170 2023-09-30T22:17:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e6ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e6dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e7009e1d0 0x7f6e70138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e6ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44562/0 (socket says 172.21.15.89:44562) 2023-09-30T22:17:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e6dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e7009e1d0 0x7f6e70138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60944/0 (socket says 172.21.15.89:60944) 2023-09-30T22:17:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.296+0000 7f6e6ed9d700 1 -- 172.21.15.89:0/2570054250 learned_addr learned my addr 172.21.15.89:0/2570054250 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:21.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.297+0000 7f6e6ed9d700 1 -- 172.21.15.89:0/2570054250 >> v2:172.21.15.89:3300/0 conn(0x7f6e7009e1d0 msgr2=0x7f6e70138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:21.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.297+0000 7f6e6ed9d700 1 --2- 172.21.15.89:0/2570054250 >> v2:172.21.15.89:3300/0 conn(0x7f6e7009e1d0 0x7f6e70138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.297+0000 7f6e6ed9d700 1 -- 172.21.15.89:0/2570054250 >> v1:172.21.15.89:6789/0 conn(0x7f6e7009d6f0 legacy=0x7f6e7009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.297+0000 7f6e6ed9d700 1 -- 172.21.15.89:0/2570054250 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7009a500 con 0x7f6e70139170 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.297+0000 7f6e6ed9d700 1 --2- 172.21.15.89:0/2570054250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7013b550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009e80 tx=0x7f6e6800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eeb6f8eea26dae4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.297+0000 7f6e6d59a700 1 -- 172.21.15.89:0/2570054250 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800e070 con 0x7f6e70139170 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.297+0000 7f6e6d59a700 1 -- 172.21.15.89:0/2570054250 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6800b8d0 con 0x7f6e70139170 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.297+0000 7f6e6d59a700 1 -- 172.21.15.89:0/2570054250 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800bc80 con 0x7f6e70139170 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.298+0000 7f6e74cc7700 1 -- 172.21.15.89:0/2570054250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 msgr2=0x7f6e7013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.298+0000 7f6e74cc7700 1 --2- 172.21.15.89:0/2570054250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7013b550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009e80 tx=0x7f6e6800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.298+0000 7f6e74cc7700 1 -- 172.21.15.89:0/2570054250 shutdown_connections 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.298+0000 7f6e74cc7700 1 --2- 172.21.15.89:0/2570054250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7013b550 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.298+0000 7f6e74cc7700 1 --2- 172.21.15.89:0/2570054250 >> v2:172.21.15.89:3300/0 conn(0x7f6e7009e1d0 0x7f6e70138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.298+0000 7f6e74cc7700 1 -- 172.21.15.89:0/2570054250 >> 172.21.15.89:0/2570054250 conn(0x7f6e70066f50 msgr2=0x7f6e7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.298+0000 7f6e74cc7700 1 -- 172.21.15.89:0/2570054250 shutdown_connections 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.298+0000 7f6e74cc7700 1 -- 172.21.15.89:0/2570054250 wait complete. 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.299+0000 7f6e74cc7700 1 Processor -- start 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.299+0000 7f6e74cc7700 1 -- start start 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.299+0000 7f6e74cc7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e7009d6f0 0x7f6e701c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.299+0000 7f6e74cc7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e7009e1d0 0x7f6e701c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.299+0000 7f6e74cc7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7011e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.299+0000 7f6e74cc7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70122110 con 0x7f6e7009e1d0 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.299+0000 7f6e74cc7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70120400 con 0x7f6e7009d6f0 2023-09-30T22:17:21.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.299+0000 7f6e74cc7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70121d70 con 0x7f6e70139170 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e7009e1d0 0x7f6e701c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6e59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e7009d6f0 0x7f6e701c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e7009e1d0 0x7f6e701c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60952/0 (socket says 172.21.15.89:60952) 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6dd9b700 1 -- 172.21.15.89:0/3452172713 learned_addr learned my addr 172.21.15.89:0/3452172713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6ed9d700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7011e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6e59c700 1 -- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 msgr2=0x7f6e7011e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6e59c700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7011e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6e59c700 1 -- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e7009e1d0 msgr2=0x7f6e701c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6e59c700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e7009e1d0 0x7f6e701c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6e59c700 1 -- 172.21.15.89:0/3452172713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009b30 con 0x7f6e7009d6f0 2023-09-30T22:17:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6dd9b700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e7009e1d0 0x7f6e701c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:21.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e6e59c700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e7009d6f0 0x7f6e701c84d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e60000f60 tx=0x7f6e6000c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:21.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e5effd700 1 -- 172.21.15.89:0/3452172713 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e600052d0 con 0x7f6e7009d6f0 2023-09-30T22:17:21.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e5effd700 1 -- 172.21.15.89:0/3452172713 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60005430 con 0x7f6e7009d6f0 2023-09-30T22:17:21.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7009c710 con 0x7f6e7009d6f0 2023-09-30T22:17:21.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e5effd700 1 -- 172.21.15.89:0/3452172713 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e600038c0 con 0x7f6e7009d6f0 2023-09-30T22:17:21.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.300+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e70122310 con 0x7f6e7009d6f0 2023-09-30T22:17:21.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.301+0000 7f6e5effd700 1 -- 172.21.15.89:0/3452172713 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e60003a20 con 0x7f6e7009d6f0 2023-09-30T22:17:21.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.302+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e70074ea0 con 0x7f6e7009d6f0 2023-09-30T22:17:21.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.302+0000 7f6e5effd700 1 -- 172.21.15.89:0/3452172713 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6e6003e9a0 con 0x7f6e7009d6f0 2023-09-30T22:17:21.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.307+0000 7f6e5effd700 1 -- 172.21.15.89:0/3452172713 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e60011590 con 0x7f6e7009d6f0 2023-09-30T22:17:21.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:21.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.414+0000 7f47ab898700 1 Processor -- start 2023-09-30T22:17:21.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.414+0000 7f47ab898700 1 -- start start 2023-09-30T22:17:21.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47ab898700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47a412cf60 0x7f47a406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47ab898700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47ab898700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a4130260 0x7f47a4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47ab898700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a412d330 con 0x7f47a412cf60 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47ab898700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a412cd90 con 0x7f47a412f8b0 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47ab898700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41306d0 con 0x7f47a4130260 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43100/0 (socket says 172.21.15.89:43100) 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 -- 172.21.15.89:0/366415849 learned_addr learned my addr 172.21.15.89:0/366415849 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a9e35700 1 --2- 172.21.15.89:0/366415849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a4130260 0x7f47a4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a9634700 1 --2- 172.21.15.89:0/366415849 >> v2:172.21.15.89:3300/0 conn(0x7f47a412cf60 0x7f47a406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 -- 172.21.15.89:0/366415849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a4130260 msgr2=0x7f47a4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 --2- 172.21.15.89:0/366415849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a4130260 0x7f47a4136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 -- 172.21.15.89:0/366415849 >> v2:172.21.15.89:3300/0 conn(0x7f47a412cf60 msgr2=0x7f47a406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 --2- 172.21.15.89:0/366415849 >> v2:172.21.15.89:3300/0 conn(0x7f47a412cf60 0x7f47a406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 -- 172.21.15.89:0/366415849 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a4071780 con 0x7f47a412f8b0 2023-09-30T22:17:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.415+0000 7f47a8e33700 1 --2- 172.21.15.89:0/366415849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a412fd20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f47940098f0 tx=0x7f4794009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c650ec52bedd66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f479bfff700 1 -- 172.21.15.89:0/366415849 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794011830 con 0x7f47a412f8b0 2023-09-30T22:17:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f479bfff700 1 -- 172.21.15.89:0/366415849 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4794011990 con 0x7f47a412f8b0 2023-09-30T22:17:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f479bfff700 1 -- 172.21.15.89:0/366415849 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794011d20 con 0x7f47a412f8b0 2023-09-30T22:17:21.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f47ab898700 1 -- 172.21.15.89:0/366415849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 msgr2=0x7f47a412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f47ab898700 1 --2- 172.21.15.89:0/366415849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a412fd20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f47940098f0 tx=0x7f4794009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:21.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f47ab898700 1 -- 172.21.15.89:0/366415849 shutdown_connections 2023-09-30T22:17:21.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f47ab898700 1 --2- 172.21.15.89:0/366415849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a4130260 0x7f47a4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f47ab898700 1 --2- 172.21.15.89:0/366415849 >> v2:172.21.15.89:3300/0 conn(0x7f47a412cf60 0x7f47a406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f47ab898700 1 --2- 172.21.15.89:0/366415849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a412fd20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.416+0000 7f47ab898700 1 -- 172.21.15.89:0/366415849 >> 172.21.15.89:0/366415849 conn(0x7f47a40a2aa0 msgr2=0x7f47a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.417+0000 7f47ab898700 1 -- 172.21.15.89:0/366415849 shutdown_connections 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.417+0000 7f47ab898700 1 -- 172.21.15.89:0/366415849 wait complete. 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.418+0000 7f47ab898700 1 Processor -- start 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.418+0000 7f47ab898700 1 -- start start 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.418+0000 7f47ab898700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a412cf60 0x7f47a41309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.418+0000 7f47ab898700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a4130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.418+0000 7f47ab898700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4130260 0x7f47a4131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47ab898700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a409e8c0 con 0x7f47a4130260 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47ab898700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41329f0 con 0x7f47a412f8b0 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47ab898700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41330a0 con 0x7f47a412cf60 2023-09-30T22:17:21.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4130260 0x7f47a4131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4130260 0x7f47a4131460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60966/0 (socket says 172.21.15.89:60966) 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 -- 172.21.15.89:0/4048879251 learned_addr learned my addr 172.21.15.89:0/4048879251 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a8e33700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a4130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9634700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a412cf60 0x7f47a41309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 -- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a412cf60 msgr2=0x7f47a41309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a412cf60 0x7f47a41309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 -- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 msgr2=0x7f47a4130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a4130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 -- 172.21.15.89:0/4048879251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4794009550 con 0x7f47a4130260 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47a9e35700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4130260 0x7f47a4131460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f479c00efc0 tx=0x7f479c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:21.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f4799ffb700 1 -- 172.21.15.89:0/4048879251 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c00cd40 con 0x7f47a4130260 2023-09-30T22:17:21.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f4799ffb700 1 -- 172.21.15.89:0/4048879251 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c00f040 con 0x7f47a4130260 2023-09-30T22:17:21.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f4799ffb700 1 -- 172.21.15.89:0/4048879251 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0146c0 con 0x7f47a4130260 2023-09-30T22:17:21.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.419+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a4133310 con 0x7f47a4130260 2023-09-30T22:17:21.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.420+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a4129fa0 con 0x7f47a4130260 2023-09-30T22:17:21.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.421+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a4131c60 con 0x7f47a4130260 2023-09-30T22:17:21.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.421+0000 7f4799ffb700 1 -- 172.21.15.89:0/4048879251 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f479c014820 con 0x7f47a4130260 2023-09-30T22:17:21.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.421+0000 7f4799ffb700 1 -- 172.21.15.89:0/4048879251 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f479c040200 con 0x7f47a4130260 2023-09-30T22:17:21.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.425+0000 7fea80521700 1 Processor -- start 2023-09-30T22:17:21.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.425+0000 7fea80521700 1 -- start start 2023-09-30T22:17:21.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea80521700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78131810 0x7fea78131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea80521700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78099a40 0x7fea78099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea80521700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea7809a3b0 0x7fea78138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea80521700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7809b580 con 0x7fea7809a3b0 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea80521700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea781313c0 con 0x7fea78131810 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea80521700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea7806e8c0 con 0x7fea78099a40 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea7dabc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78099a40 0x7fea78099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea7e2bd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78131810 0x7fea78131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea7e2bd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78131810 0x7fea78131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43118/0 (socket says 172.21.15.89:43118) 2023-09-30T22:17:21.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea7e2bd700 1 -- 172.21.15.89:0/4252627624 learned_addr learned my addr 172.21.15.89:0/4252627624 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea7eabe700 1 --2- 172.21.15.89:0/4252627624 >> v2:172.21.15.89:3300/0 conn(0x7fea7809a3b0 0x7fea78138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.426+0000 7fea7dabc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78099a40 0x7fea78099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44606/0 (socket says 172.21.15.89:44606) 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.427+0000 7fea7eabe700 1 -- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78099a40 msgr2=0x7fea78099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.427+0000 7fea7eabe700 1 --2- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78099a40 0x7fea78099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.427+0000 7fea7eabe700 1 -- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78131810 msgr2=0x7fea78131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.427+0000 7fea7eabe700 1 --2- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78131810 0x7fea78131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.427+0000 7fea7eabe700 1 -- 172.21.15.89:0/4252627624 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7809a820 con 0x7fea7809a3b0 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.427+0000 7fea7e2bd700 1 --2- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78131810 0x7fea78131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:21.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.427+0000 7fea7eabe700 1 --2- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 0x7fea78138650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea70009d90 tx=0x7fea7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6662241d50daf8a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:21.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.428+0000 7fea7d2bb700 1 -- 172.21.15.89:0/4252627624 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7000e070 con 0x7fea7809a3b0 2023-09-30T22:17:21.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea7d2bb700 1 -- 172.21.15.89:0/4252627624 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7000bb20 con 0x7fea7809a3b0 2023-09-30T22:17:21.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea7d2bb700 1 -- 172.21.15.89:0/4252627624 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70012430 con 0x7fea7809a3b0 2023-09-30T22:17:21.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7f4799ffb700 1 -- 172.21.15.89:0/4048879251 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f479c007d40 con 0x7f47a4130260 2023-09-30T22:17:21.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea80521700 1 -- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 msgr2=0x7fea78138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea80521700 1 --2- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 0x7fea78138650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea70009d90 tx=0x7fea7000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:21.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea80521700 1 -- 172.21.15.89:0/4252627624 shutdown_connections 2023-09-30T22:17:21.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea80521700 1 --2- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 0x7fea78138650 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea80521700 1 --2- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78131810 0x7fea78131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea80521700 1 --2- 172.21.15.89:0/4252627624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78099a40 0x7fea78099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.429+0000 7fea80521700 1 -- 172.21.15.89:0/4252627624 >> 172.21.15.89:0/4252627624 conn(0x7fea780a2aa0 msgr2=0x7fea780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:21.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.430+0000 7fea80521700 1 -- 172.21.15.89:0/4252627624 shutdown_connections 2023-09-30T22:17:21.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.430+0000 7fea80521700 1 -- 172.21.15.89:0/4252627624 wait complete. 2023-09-30T22:17:21.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.431+0000 7fea80521700 1 Processor -- start 2023-09-30T22:17:21.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.431+0000 7fea80521700 1 -- start start 2023-09-30T22:17:21.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.431+0000 7fea80521700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78099a40 0x7fea7811b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.431+0000 7fea80521700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 0x7fea7811bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.431+0000 7fea80521700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78131810 0x7fea7811c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.431+0000 7fea80521700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7811cb50 con 0x7fea7809a3b0 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.431+0000 7fea80521700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7811d220 con 0x7fea78099a40 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.431+0000 7fea80521700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea7811e470 con 0x7fea78131810 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7eabe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78131810 0x7fea7811c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7eabe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78131810 0x7fea7811c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44618/0 (socket says 172.21.15.89:44618) 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7eabe700 1 -- 172.21.15.89:0/3642905527 learned_addr learned my addr 172.21.15.89:0/3642905527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7e2bd700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78099a40 0x7fea7811b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7dabc700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 0x7fea7811bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7dabc700 1 -- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78131810 msgr2=0x7fea7811c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7dabc700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78131810 0x7fea7811c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7dabc700 1 -- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78099a40 msgr2=0x7fea7811b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7dabc700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78099a40 0x7fea7811b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7dabc700 1 -- 172.21.15.89:0/3642905527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea74009550 con 0x7fea7809a3b0 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.432+0000 7fea7dabc700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 0x7fea7811bbe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e900 tx=0x7fea6800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.433+0000 7fea6effd700 1 -- 172.21.15.89:0/3642905527 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68013680 con 0x7fea7809a3b0 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.433+0000 7fea6effd700 1 -- 172.21.15.89:0/3642905527 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea680137e0 con 0x7fea7809a3b0 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.433+0000 7fea6effd700 1 -- 172.21.15.89:0/3642905527 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68013b50 con 0x7fea7809a3b0 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.433+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea70009a70 con 0x7fea7809a3b0 2023-09-30T22:17:21.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.433+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7811eeb0 con 0x7fea7809a3b0 2023-09-30T22:17:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.434+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7806eb00 con 0x7fea7809a3b0 2023-09-30T22:17:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.434+0000 7fea6effd700 1 -- 172.21.15.89:0/3642905527 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea68013cb0 con 0x7fea7809a3b0 2023-09-30T22:17:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.434+0000 7fea6effd700 1 -- 172.21.15.89:0/3642905527 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea6803f5f0 con 0x7fea7809a3b0 2023-09-30T22:17:21.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.442+0000 7fea6effd700 1 -- 172.21.15.89:0/3642905527 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea6801bde0 con 0x7fea7809a3b0 2023-09-30T22:17:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.468+0000 7f029d16b700 1 Processor -- start 2023-09-30T22:17:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.468+0000 7f029d16b700 1 -- start start 2023-09-30T22:17:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029d16b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809d6f0 0x7f029809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029d16b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f029809e000 0x7f0298130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029d16b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809e410 0x7f0298130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029d16b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029809f520 con 0x7f029809e000 2023-09-30T22:17:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029d16b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029809f130 con 0x7f029809e410 2023-09-30T22:17:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029d16b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f029809e930 con 0x7f029809d6f0 2023-09-30T22:17:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029759e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809e410 0x7f0298130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f029809e000 0x7f0298130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029759e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809e410 0x7f0298130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43134/0 (socket says 172.21.15.89:43134) 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029759e700 1 -- 172.21.15.89:0/1136371085 learned_addr learned my addr 172.21.15.89:0/1136371085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f0296d9d700 1 --2- 172.21.15.89:0/1136371085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809d6f0 0x7f029809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f0296d9d700 1 -- 172.21.15.89:0/1136371085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809e410 msgr2=0x7f0298130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f0296d9d700 1 --2- 172.21.15.89:0/1136371085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809e410 0x7f0298130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f0296d9d700 1 -- 172.21.15.89:0/1136371085 >> v2:172.21.15.89:3300/0 conn(0x7f029809e000 msgr2=0x7f0298130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f0296d9d700 1 --2- 172.21.15.89:0/1136371085 >> v2:172.21.15.89:3300/0 conn(0x7f029809e000 0x7f0298130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f0296d9d700 1 -- 172.21.15.89:0/1136371085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029809b2d0 con 0x7f029809d6f0 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f029659c700 1 --2- 172.21.15.89:0/1136371085 >> v2:172.21.15.89:3300/0 conn(0x7f029809e000 0x7f0298130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.469+0000 7f0296d9d700 1 --2- 172.21.15.89:0/1136371085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809d6f0 0x7f029809dac0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0288009870 tx=0x7f0288009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c6bf4b2158082f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.470+0000 7f0295d9b700 1 -- 172.21.15.89:0/1136371085 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288011830 con 0x7f029809d6f0 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.470+0000 7f0295d9b700 1 -- 172.21.15.89:0/1136371085 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0288011990 con 0x7f029809d6f0 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.470+0000 7f0295d9b700 1 -- 172.21.15.89:0/1136371085 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288011d20 con 0x7f029809d6f0 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.470+0000 7f029d16b700 1 -- 172.21.15.89:0/1136371085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809d6f0 msgr2=0x7f029809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.470+0000 7f029d16b700 1 --2- 172.21.15.89:0/1136371085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809d6f0 0x7f029809dac0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0288009870 tx=0x7f0288009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.470+0000 7f029d16b700 1 -- 172.21.15.89:0/1136371085 shutdown_connections 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.470+0000 7f029d16b700 1 --2- 172.21.15.89:0/1136371085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809e410 0x7f0298130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.470+0000 7f029d16b700 1 --2- 172.21.15.89:0/1136371085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809d6f0 0x7f029809dac0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.471+0000 7f029d16b700 1 --2- 172.21.15.89:0/1136371085 >> v2:172.21.15.89:3300/0 conn(0x7f029809e000 0x7f0298130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.471+0000 7f029d16b700 1 -- 172.21.15.89:0/1136371085 >> 172.21.15.89:0/1136371085 conn(0x7f0298066f50 msgr2=0x7f029809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.471+0000 7f029d16b700 1 -- 172.21.15.89:0/1136371085 shutdown_connections 2023-09-30T22:17:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.471+0000 7f029d16b700 1 -- 172.21.15.89:0/1136371085 wait complete. 2023-09-30T22:17:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.472+0000 7f029d16b700 1 Processor -- start 2023-09-30T22:17:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.472+0000 7f029d16b700 1 -- start start 2023-09-30T22:17:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029d16b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809d6f0 0x7f029811e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029d16b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f029809e000 0x7f029811e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029d16b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809e410 0x7f029811ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029d16b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0298122880 con 0x7f029809e000 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029d16b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980a16b0 con 0x7f029809d6f0 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029d16b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f029809c700 con 0x7f029809e410 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f029809e000 0x7f029811e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f029809e000 0x7f029811e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60994/0 (socket says 172.21.15.89:60994) 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029659c700 1 -- 172.21.15.89:0/78421553 learned_addr learned my addr 172.21.15.89:0/78421553 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f029759e700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809e410 0x7f029811ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.473+0000 7f0296d9d700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809d6f0 0x7f029811e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f029659c700 1 -- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809e410 msgr2=0x7f029811ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f029659c700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809e410 0x7f029811ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f029659c700 1 -- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809d6f0 msgr2=0x7f029811e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f029659c700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809d6f0 0x7f029811e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f029659c700 1 -- 172.21.15.89:0/78421553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c00d940 con 0x7f029809e000 2023-09-30T22:17:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f029659c700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f029809e000 0x7f029811e740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f028c012ea0 tx=0x7f028c00f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f027f7fe700 1 -- 172.21.15.89:0/78421553 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c014040 con 0x7f029809e000 2023-09-30T22:17:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f027f7fe700 1 -- 172.21.15.89:0/78421553 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c00faf0 con 0x7f029809e000 2023-09-30T22:17:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02880094a0 con 0x7f029809e000 2023-09-30T22:17:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f027f7fe700 1 -- 172.21.15.89:0/78421553 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00fe60 con 0x7f029809e000 2023-09-30T22:17:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.474+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029811fd20 con 0x7f029809e000 2023-09-30T22:17:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.475+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02780052e0 con 0x7f029809e000 2023-09-30T22:17:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.476+0000 7f027f7fe700 1 -- 172.21.15.89:0/78421553 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f028c00b450 con 0x7f029809e000 2023-09-30T22:17:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.476+0000 7f027f7fe700 1 -- 172.21.15.89:0/78421553 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f028c043290 con 0x7f029809e000 2023-09-30T22:17:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.485+0000 7f027f7fe700 1 -- 172.21.15.89:0/78421553 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f028c01f900 con 0x7f029809e000 2023-09-30T22:17:21.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.511+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6e7009f420 con 0x7f6e7009d6f0 2023-09-30T22:17:21.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.511+0000 7f6e5effd700 1 -- 172.21.15.89:0/3452172713 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e60017030 con 0x7f6e7009d6f0 2023-09-30T22:17:21.511 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:21.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.513+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e7009d6f0 msgr2=0x7f6e701c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.513+0000 7f6e74cc7700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e7009d6f0 0x7f6e701c84d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e60000f60 tx=0x7f6e6000c730 comp rx=0 tx=0).stop 2023-09-30T22:17:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.514+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 shutdown_connections 2023-09-30T22:17:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.514+0000 7f6e74cc7700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e70139170 0x7f6e7011e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.514+0000 7f6e74cc7700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e7009d6f0 0x7f6e701c84d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.514+0000 7f6e74cc7700 1 --2- 172.21.15.89:0/3452172713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e7009e1d0 0x7f6e701c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.514+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 >> 172.21.15.89:0/3452172713 conn(0x7f6e70066f50 msgr2=0x7f6e700a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.514+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 shutdown_connections 2023-09-30T22:17:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.514+0000 7f6e74cc7700 1 -- 172.21.15.89:0/3452172713 wait complete. 2023-09-30T22:17:21.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.636+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f47a4074ea0 con 0x7f47a4130260 2023-09-30T22:17:21.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.636+0000 7f4799ffb700 1 -- 172.21.15.89:0/4048879251 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f479c018020 con 0x7f47a4130260 2023-09-30T22:17:21.636 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4130260 msgr2=0x7f47a4131460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4130260 0x7f47a4131460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f479c00efc0 tx=0x7f479c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 shutdown_connections 2023-09-30T22:17:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4130260 0x7f47a4131460 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47a412cf60 0x7f47a41309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 --2- 172.21.15.89:0/4048879251 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47a412f8b0 0x7f47a4130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 >> 172.21.15.89:0/4048879251 conn(0x7f47a40a2aa0 msgr2=0x7f47a412a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 shutdown_connections 2023-09-30T22:17:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.638+0000 7f47ab898700 1 -- 172.21.15.89:0/4048879251 wait complete. 2023-09-30T22:17:21.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:21.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.677+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fea78071d80 con 0x7fea7809a3b0 2023-09-30T22:17:21.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.678+0000 7fea6effd700 1 -- 172.21.15.89:0/3642905527 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea68021020 con 0x7fea7809a3b0 2023-09-30T22:17:21.678 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 msgr2=0x7fea7811bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 0x7fea7811bbe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e900 tx=0x7fea6800ec10 comp rx=0 tx=0).stop 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 shutdown_connections 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea78131810 0x7fea7811c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea78099a40 0x7fea7811b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 --2- 172.21.15.89:0/3642905527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7809a3b0 0x7fea7811bbe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 >> 172.21.15.89:0/3642905527 conn(0x7fea780a2aa0 msgr2=0x7fea781b8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 shutdown_connections 2023-09-30T22:17:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.680+0000 7fea80521700 1 -- 172.21.15.89:0/3642905527 wait complete. 2023-09-30T22:17:21.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.713+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0278005590 con 0x7f029809e000 2023-09-30T22:17:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.714+0000 7f027f7fe700 1 -- 172.21.15.89:0/78421553 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f028c01f310 con 0x7f029809e000 2023-09-30T22:17:21.713 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.715+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f029809e000 msgr2=0x7f029811e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.715+0000 7f029d16b700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f029809e000 0x7f029811e740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f028c012ea0 tx=0x7f028c00f390 comp rx=0 tx=0).stop 2023-09-30T22:17:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.715+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 shutdown_connections 2023-09-30T22:17:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.715+0000 7f029d16b700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f029809e410 0x7f029811ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.715+0000 7f029d16b700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f029809d6f0 0x7f029811e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.715+0000 7f029d16b700 1 --2- 172.21.15.89:0/78421553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f029809e000 0x7f029811e740 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.715+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 >> 172.21.15.89:0/78421553 conn(0x7f0298066f50 msgr2=0x7f029806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.715+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 shutdown_connections 2023-09-30T22:17:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:21.716+0000 7f029d16b700 1 -- 172.21.15.89:0/78421553 wait complete. 2023-09-30T22:17:21.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:22.527 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:22.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.656+0000 7f69f03f4700 1 Processor -- start 2023-09-30T22:17:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.656+0000 7f69f03f4700 1 -- start start 2023-09-30T22:17:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69f03f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f69e809d6f0 0x7f69e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69f03f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69f03f4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e809b120 con 0x7f69e809e410 2023-09-30T22:17:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69f03f4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e806b460 con 0x7f69e809d6f0 2023-09-30T22:17:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69f03f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e809a7e0 con 0x7f69e809e000 2023-09-30T22:17:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ed98f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ee190700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f69e809d6f0 0x7f69e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ed98f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43162/0 (socket says 172.21.15.89:43162) 2023-09-30T22:17:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ed98f700 1 -- 172.21.15.89:0/1468863346 learned_addr learned my addr 172.21.15.89:0/1468863346 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ee190700 1 -- 172.21.15.89:0/1468863346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 msgr2=0x7f69e81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ee190700 1 --2- 172.21.15.89:0/1468863346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ee190700 1 -- 172.21.15.89:0/1468863346 >> v1:172.21.15.89:6789/0 conn(0x7f69e809e410 legacy=0x7f69e8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ee190700 1 -- 172.21.15.89:0/1468863346 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e809a500 con 0x7f69e809d6f0 2023-09-30T22:17:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ed98f700 1 --2- 172.21.15.89:0/1468863346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ee190700 1 --2- 172.21.15.89:0/1468863346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809d6f0 0x7f69e809dac0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009840 tx=0x7f69e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd55cf4ce3fe66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.657+0000 7f69ed18e700 1 -- 172.21.15.89:0/1468863346 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4011830 con 0x7f69e809d6f0 2023-09-30T22:17:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69ed18e700 1 -- 172.21.15.89:0/1468863346 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e4011990 con 0x7f69e809d6f0 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69ed18e700 1 -- 172.21.15.89:0/1468863346 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4011d20 con 0x7f69e809d6f0 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69f03f4700 1 -- 172.21.15.89:0/1468863346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809d6f0 msgr2=0x7f69e809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69f03f4700 1 --2- 172.21.15.89:0/1468863346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809d6f0 0x7f69e809dac0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009840 tx=0x7f69e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69f03f4700 1 -- 172.21.15.89:0/1468863346 shutdown_connections 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69f03f4700 1 --2- 172.21.15.89:0/1468863346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809d6f0 0x7f69e809dac0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69f03f4700 1 --2- 172.21.15.89:0/1468863346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69f03f4700 1 -- 172.21.15.89:0/1468863346 >> 172.21.15.89:0/1468863346 conn(0x7f69e8066f50 msgr2=0x7f69e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69f03f4700 1 -- 172.21.15.89:0/1468863346 shutdown_connections 2023-09-30T22:17:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.658+0000 7f69f03f4700 1 -- 172.21.15.89:0/1468863346 wait complete. 2023-09-30T22:17:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.659+0000 7f69f03f4700 1 Processor -- start 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.659+0000 7f69f03f4700 1 -- start start 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69f03f4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69e809d6f0 0x7f69e81a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69f03f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69f03f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809e410 0x7f69e81a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69f03f4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e81a7340 con 0x7f69e809e410 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69f03f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e81a7ac0 con 0x7f69e809e000 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69f03f4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69e81a80e0 con 0x7f69e809d6f0 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ed98f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ee190700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69e809d6f0 0x7f69e81a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ed98f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81a63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43170/0 (socket says 172.21.15.89:43170) 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ed98f700 1 -- 172.21.15.89:0/3179832174 learned_addr learned my addr 172.21.15.89:0/3179832174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ee991700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809e410 0x7f69e81a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ee190700 1 -- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 msgr2=0x7f69e81a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ee190700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ee190700 1 -- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809e410 msgr2=0x7f69e81a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ee190700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809e410 0x7f69e81a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ee190700 1 -- 172.21.15.89:0/3179832174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d8009550 con 0x7f69e809d6f0 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ed98f700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69ee190700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69e809d6f0 0x7f69e81a5eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f69e4016000 tx=0x7f69e4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69deffd700 1 -- 172.21.15.89:0/3179832174 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e40036a0 con 0x7f69e809d6f0 2023-09-30T22:17:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69deffd700 1 -- 172.21.15.89:0/3179832174 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e401e070 con 0x7f69e809d6f0 2023-09-30T22:17:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e40094a0 con 0x7f69e809d6f0 2023-09-30T22:17:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.660+0000 7f69deffd700 1 -- 172.21.15.89:0/3179832174 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4003a20 con 0x7f69e809d6f0 2023-09-30T22:17:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.661+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e80a3220 con 0x7f69e809d6f0 2023-09-30T22:17:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.661+0000 7f69deffd700 1 -- 172.21.15.89:0/3179832174 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f69e4003b80 con 0x7f69e809d6f0 2023-09-30T22:17:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.661+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e809c710 con 0x7f69e809d6f0 2023-09-30T22:17:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.661+0000 7f69deffd700 1 -- 172.21.15.89:0/3179832174 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f69e4040560 con 0x7f69e809d6f0 2023-09-30T22:17:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.666+0000 7f69deffd700 1 -- 172.21.15.89:0/3179832174 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f69e4015070 con 0x7f69e809d6f0 2023-09-30T22:17:22.693 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:22.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:22.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.788+0000 7fcaa14bb700 1 Processor -- start 2023-09-30T22:17:22.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.788+0000 7fcaa14bb700 1 -- start start 2023-09-30T22:17:22.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.788+0000 7fcaa14bb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09d6f0 0x7fca9c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.788+0000 7fcaa14bb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 0x7fca9c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.788+0000 7fcaa14bb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c09f130 con 0x7fca9c09e410 2023-09-30T22:17:22.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.788+0000 7fcaa14bb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c09f520 con 0x7fca9c09d6f0 2023-09-30T22:17:22.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.788+0000 7fcaa14bb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c09e930 con 0x7fca9c09e000 2023-09-30T22:17:22.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.788+0000 7fca9affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09d6f0 0x7fca9c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca9affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09d6f0 0x7fca9c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43178/0 (socket says 172.21.15.89:43178) 2023-09-30T22:17:22.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca9b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fca9c09e410 0x7fca9c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49848/0 (socket says 172.21.15.89:49848) 2023-09-30T22:17:22.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca9affd700 1 -- 172.21.15.89:0/2125988923 learned_addr learned my addr 172.21.15.89:0/2125988923 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:22.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca99ffb700 1 -- 172.21.15.89:0/2125988923 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864780770 0 0) 0x7fca9c09f130 con 0x7fca9c09e410 2023-09-30T22:17:22.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca99ffb700 1 -- 172.21.15.89:0/2125988923 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca840035b0 con 0x7fca9c09e410 2023-09-30T22:17:22.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca9affd700 1 -- 172.21.15.89:0/2125988923 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 msgr2=0x7fca9c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:22.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca9affd700 1 --2- 172.21.15.89:0/2125988923 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 0x7fca9c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca9affd700 1 -- 172.21.15.89:0/2125988923 >> v1:172.21.15.89:6789/0 conn(0x7fca9c09e410 legacy=0x7fca9c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca9affd700 1 -- 172.21.15.89:0/2125988923 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c09b2d0 con 0x7fca9c09d6f0 2023-09-30T22:17:22.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca9affd700 1 --2- 172.21.15.89:0/2125988923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09d6f0 0x7fca9c09dac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009920 tx=0x7fca8c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c0ce2bbf3f345b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:22.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca99ffb700 1 -- 172.21.15.89:0/2125988923 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c011830 con 0x7fca9c09d6f0 2023-09-30T22:17:22.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca99ffb700 1 -- 172.21.15.89:0/2125988923 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c011990 con 0x7fca9c09d6f0 2023-09-30T22:17:22.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.789+0000 7fca99ffb700 1 -- 172.21.15.89:0/2125988923 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c011d20 con 0x7fca9c09d6f0 2023-09-30T22:17:22.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.790+0000 7fcaa14bb700 1 -- 172.21.15.89:0/2125988923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09d6f0 msgr2=0x7fca9c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.790+0000 7fcaa14bb700 1 --2- 172.21.15.89:0/2125988923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09d6f0 0x7fca9c09dac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009920 tx=0x7fca8c009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:22.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.790+0000 7fcaa14bb700 1 -- 172.21.15.89:0/2125988923 shutdown_connections 2023-09-30T22:17:22.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.790+0000 7fcaa14bb700 1 --2- 172.21.15.89:0/2125988923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09d6f0 0x7fca9c09dac0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.790+0000 7fcaa14bb700 1 --2- 172.21.15.89:0/2125988923 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 0x7fca9c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.790+0000 7fcaa14bb700 1 -- 172.21.15.89:0/2125988923 >> 172.21.15.89:0/2125988923 conn(0x7fca9c066f50 msgr2=0x7fca9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.790+0000 7fcaa14bb700 1 -- 172.21.15.89:0/2125988923 shutdown_connections 2023-09-30T22:17:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.791+0000 7fcaa14bb700 1 -- 172.21.15.89:0/2125988923 wait complete. 2023-09-30T22:17:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fcaa14bb700 1 Processor -- start 2023-09-30T22:17:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fcaa14bb700 1 -- start start 2023-09-30T22:17:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fcaa14bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca9c09d6f0 0x7fca9c1b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fcaa14bb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 0x7fca9c1b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fcaa14bb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09e410 0x7fca9c1b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fcaa14bb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c1bad10 con 0x7fca9c09d6f0 2023-09-30T22:17:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fcaa14bb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c1b8ac0 con 0x7fca9c09e410 2023-09-30T22:17:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fcaa14bb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c1b8f90 con 0x7fca9c09e000 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fca9affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca9c09d6f0 0x7fca9c1b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fca9affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca9c09d6f0 0x7fca9c1b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32792/0 (socket says 172.21.15.89:32792) 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fca9affd700 1 -- 172.21.15.89:0/1267946212 learned_addr learned my addr 172.21.15.89:0/1267946212 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fca9b7fe700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09e410 0x7fca9c1b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.792+0000 7fca9a7fc700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 0x7fca9c1b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca9affd700 1 -- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 msgr2=0x7fca9c1b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca9affd700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 0x7fca9c1b7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca9affd700 1 -- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09e410 msgr2=0x7fca9c1b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca9affd700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09e410 0x7fca9c1b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca9affd700 1 -- 172.21.15.89:0/1267946212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c009550 con 0x7fca9c09d6f0 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca9affd700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca9c09d6f0 0x7fca9c1b7660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00a010 tx=0x7fca8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca837fe700 1 -- 172.21.15.89:0/1267946212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004300 con 0x7fca9c09d6f0 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca837fe700 1 -- 172.21.15.89:0/1267946212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c01f070 con 0x7fca9c09d6f0 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c06eba0 con 0x7fca9c09d6f0 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c1bb6a0 con 0x7fca9c09d6f0 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.793+0000 7fca837fe700 1 -- 172.21.15.89:0/1267946212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0036a0 con 0x7fca9c09d6f0 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.794+0000 7fca837fe700 1 -- 172.21.15.89:0/1267946212 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca8c021030 con 0x7fca9c09d6f0 2023-09-30T22:17:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.794+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c0052e0 con 0x7fca9c09d6f0 2023-09-30T22:17:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.794+0000 7fca837fe700 1 -- 172.21.15.89:0/1267946212 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fca8c040ab0 con 0x7fca9c09d6f0 2023-09-30T22:17:22.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.803+0000 7fca837fe700 1 -- 172.21.15.89:0/1267946212 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca8c0199a0 con 0x7fca9c09d6f0 2023-09-30T22:17:22.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.827+0000 7fb7847ce700 1 Processor -- start 2023-09-30T22:17:22.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.827+0000 7fb7847ce700 1 -- start start 2023-09-30T22:17:22.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb7847ce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09d6f0 0x7fb77c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb7847ce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb77c09e000 0x7fb77c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb7847ce700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77c09f130 con 0x7fb77c09e410 2023-09-30T22:17:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb7847ce700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77c09f520 con 0x7fb77c09e000 2023-09-30T22:17:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb7847ce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c09e930 con 0x7fb77c09d6f0 2023-09-30T22:17:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb781d69700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb77c09e000 0x7fb77c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb781d69700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb77c09e000 0x7fb77c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32804/0 (socket says 172.21.15.89:32804) 2023-09-30T22:17:22.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb781d69700 1 -- 172.21.15.89:0/2466011357 learned_addr learned my addr 172.21.15.89:0/2466011357 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:22.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb78256a700 1 --2- 172.21.15.89:0/2466011357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09d6f0 0x7fb77c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb78256a700 1 -- 172.21.15.89:0/2466011357 >> v2:172.21.15.89:3300/0 conn(0x7fb77c09e000 msgr2=0x7fb77c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:22.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb78256a700 1 --2- 172.21.15.89:0/2466011357 >> v2:172.21.15.89:3300/0 conn(0x7fb77c09e000 0x7fb77c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb78256a700 1 -- 172.21.15.89:0/2466011357 >> v1:172.21.15.89:6789/0 conn(0x7fb77c09e410 legacy=0x7fb77c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.828+0000 7fb78256a700 1 -- 172.21.15.89:0/2466011357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c09b2d0 con 0x7fb77c09d6f0 2023-09-30T22:17:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.829+0000 7fb781d69700 1 --2- 172.21.15.89:0/2466011357 >> v2:172.21.15.89:3300/0 conn(0x7fb77c09e000 0x7fb77c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.829+0000 7fb78256a700 1 --2- 172.21.15.89:0/2466011357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09d6f0 0x7fb77c09dac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb778009840 tx=0x7fb778009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0bd8bcb758de32 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.829+0000 7fb781568700 1 -- 172.21.15.89:0/2466011357 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778011830 con 0x7fb77c09d6f0 2023-09-30T22:17:22.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.829+0000 7fb781568700 1 -- 172.21.15.89:0/2466011357 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb778011990 con 0x7fb77c09d6f0 2023-09-30T22:17:22.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.829+0000 7fb781568700 1 -- 172.21.15.89:0/2466011357 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778011d20 con 0x7fb77c09d6f0 2023-09-30T22:17:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.830+0000 7fb7847ce700 1 -- 172.21.15.89:0/2466011357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09d6f0 msgr2=0x7fb77c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.830+0000 7fb7847ce700 1 --2- 172.21.15.89:0/2466011357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09d6f0 0x7fb77c09dac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb778009840 tx=0x7fb778009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.830+0000 7fb7847ce700 1 -- 172.21.15.89:0/2466011357 shutdown_connections 2023-09-30T22:17:22.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.830+0000 7fb7847ce700 1 --2- 172.21.15.89:0/2466011357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09d6f0 0x7fb77c09dac0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.830+0000 7fb7847ce700 1 --2- 172.21.15.89:0/2466011357 >> v2:172.21.15.89:3300/0 conn(0x7fb77c09e000 0x7fb77c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.830+0000 7fb7847ce700 1 -- 172.21.15.89:0/2466011357 >> 172.21.15.89:0/2466011357 conn(0x7fb77c066f50 msgr2=0x7fb77c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.830+0000 7fb7847ce700 1 -- 172.21.15.89:0/2466011357 shutdown_connections 2023-09-30T22:17:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.830+0000 7fb7847ce700 1 -- 172.21.15.89:0/2466011357 wait complete. 2023-09-30T22:17:22.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.831+0000 7fb7847ce700 1 Processor -- start 2023-09-30T22:17:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.831+0000 7fb7847ce700 1 -- start start 2023-09-30T22:17:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb7847ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb77c09d6f0 0x7fb77c122430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb7847ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb77c09e000 0x7fb77c122970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb7847ce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09e410 0x7fb77c122eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb7847ce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c124640 con 0x7fb77c09e000 2023-09-30T22:17:22.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb7847ce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c125f00 con 0x7fb77c09e410 2023-09-30T22:17:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb7847ce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c0a1d90 con 0x7fb77c09d6f0 2023-09-30T22:17:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb78256a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb77c09d6f0 0x7fb77c122430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb782d6b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09e410 0x7fb77c122eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb78256a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb77c09d6f0 0x7fb77c122430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44674/0 (socket says 172.21.15.89:44674) 2023-09-30T22:17:22.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb782d6b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09e410 0x7fb77c122eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43198/0 (socket says 172.21.15.89:43198) 2023-09-30T22:17:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb78256a700 1 -- 172.21.15.89:0/2977578182 learned_addr learned my addr 172.21.15.89:0/2977578182 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.832+0000 7fb781d69700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb77c09e000 0x7fb77c122970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb78256a700 1 -- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09e410 msgr2=0x7fb77c122eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb78256a700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09e410 0x7fb77c122eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb78256a700 1 -- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb77c09e000 msgr2=0x7fb77c122970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb78256a700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb77c09e000 0x7fb77c122970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb78256a700 1 -- 172.21.15.89:0/2977578182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c00b7f0 con 0x7fb77c09d6f0 2023-09-30T22:17:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb781d69700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb77c09e000 0x7fb77c122970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb78256a700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb77c09d6f0 0x7fb77c122430 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb778009fd0 tx=0x7fb778004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb772ffd700 1 -- 172.21.15.89:0/2977578182 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7780036a0 con 0x7fb77c09d6f0 2023-09-30T22:17:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb772ffd700 1 -- 172.21.15.89:0/2977578182 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77801e070 con 0x7fb77c09d6f0 2023-09-30T22:17:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7780094a0 con 0x7fb77c09d6f0 2023-09-30T22:17:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb772ffd700 1 -- 172.21.15.89:0/2977578182 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778003a40 con 0x7fb77c09d6f0 2023-09-30T22:17:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.833+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77c130300 con 0x7fb77c09d6f0 2023-09-30T22:17:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.834+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77c1258c0 con 0x7fb77c09d6f0 2023-09-30T22:17:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.835+0000 7fb772ffd700 1 -- 172.21.15.89:0/2977578182 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb778003ba0 con 0x7fb77c09d6f0 2023-09-30T22:17:22.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.835+0000 7fb772ffd700 1 -- 172.21.15.89:0/2977578182 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb778040810 con 0x7fb77c09d6f0 2023-09-30T22:17:22.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.839+0000 7fb772ffd700 1 -- 172.21.15.89:0/2977578182 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb778010740 con 0x7fb77c09d6f0 2023-09-30T22:17:22.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.870+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f69e8074ea0 con 0x7f69e809d6f0 2023-09-30T22:17:22.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.871+0000 7f69deffd700 1 -- 172.21.15.89:0/3179832174 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f69e4003e30 con 0x7f69e809d6f0 2023-09-30T22:17:22.872 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:22.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69e809d6f0 msgr2=0x7f69e81a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69e809d6f0 0x7f69e81a5eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f69e4016000 tx=0x7f69e4004750 comp rx=0 tx=0).stop 2023-09-30T22:17:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 shutdown_connections 2023-09-30T22:17:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e809e410 0x7f69e81a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69e809d6f0 0x7f69e81a5eb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 --2- 172.21.15.89:0/3179832174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69e809e000 0x7f69e81a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 >> 172.21.15.89:0/3179832174 conn(0x7f69e8066f50 msgr2=0x7f69e81badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 shutdown_connections 2023-09-30T22:17:22.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.873+0000 7f69f03f4700 1 -- 172.21.15.89:0/3179832174 wait complete. 2023-09-30T22:17:22.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.875+0000 7f603343d700 1 Processor -- start 2023-09-30T22:17:22.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.875+0000 7f603343d700 1 -- start start 2023-09-30T22:17:22.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f603343d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f603343d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e000 0x7f602c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f603343d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f602c09e410 0x7f602c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f603343d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602c09f520 con 0x7f602c09e410 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f603343d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c09f130 con 0x7f602c09d6f0 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f603343d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f602c09e930 con 0x7f602c09e000 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43206/0 (socket says 172.21.15.89:43206) 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 -- 172.21.15.89:0/512752313 learned_addr learned my addr 172.21.15.89:0/512752313 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 -- 172.21.15.89:0/512752313 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e000 msgr2=0x7f602c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 --2- 172.21.15.89:0/512752313 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e000 0x7f602c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 -- 172.21.15.89:0/512752313 >> v2:172.21.15.89:3300/0 conn(0x7f602c09e410 msgr2=0x7f602c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 --2- 172.21.15.89:0/512752313 >> v2:172.21.15.89:3300/0 conn(0x7f602c09e410 0x7f602c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 -- 172.21.15.89:0/512752313 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c09b2d0 con 0x7f602c09d6f0 2023-09-30T22:17:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.876+0000 7f60311d9700 1 --2- 172.21.15.89:0/512752313 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c09dac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6028009920 tx=0x7f6028009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86faabb44a551407 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.877+0000 7f6023fff700 1 -- 172.21.15.89:0/512752313 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800ba20 con 0x7f602c09d6f0 2023-09-30T22:17:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.877+0000 7f6023fff700 1 -- 172.21.15.89:0/512752313 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602800bb80 con 0x7f602c09d6f0 2023-09-30T22:17:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.877+0000 7f6023fff700 1 -- 172.21.15.89:0/512752313 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028009e70 con 0x7f602c09d6f0 2023-09-30T22:17:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 -- 172.21.15.89:0/512752313 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 msgr2=0x7f602c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 --2- 172.21.15.89:0/512752313 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c09dac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6028009920 tx=0x7f6028009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 -- 172.21.15.89:0/512752313 shutdown_connections 2023-09-30T22:17:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 --2- 172.21.15.89:0/512752313 >> v2:172.21.15.89:3300/0 conn(0x7f602c09e410 0x7f602c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 --2- 172.21.15.89:0/512752313 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c09dac0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 --2- 172.21.15.89:0/512752313 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e000 0x7f602c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 -- 172.21.15.89:0/512752313 >> 172.21.15.89:0/512752313 conn(0x7f602c066f50 msgr2=0x7f602c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 -- 172.21.15.89:0/512752313 shutdown_connections 2023-09-30T22:17:22.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.878+0000 7f603343d700 1 -- 172.21.15.89:0/512752313 wait complete. 2023-09-30T22:17:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.879+0000 7f603343d700 1 Processor -- start 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f603343d700 1 -- start start 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f603343d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c1c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f603343d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c09e000 0x7f602c1c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f603343d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e410 0x7f602c1c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f603343d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c09c700 con 0x7f602c09e000 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f603343d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c120fc0 con 0x7f602c09d6f0 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f603343d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f602c125860 con 0x7f602c09e410 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f60311d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c1c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f60311d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c1c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43208/0 (socket says 172.21.15.89:43208) 2023-09-30T22:17:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f60311d9700 1 -- 172.21.15.89:0/2966482956 learned_addr learned my addr 172.21.15.89:0/2966482956 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.880+0000 7f60319da700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e410 0x7f602c1c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f60309d8700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c09e000 0x7f602c1c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f60311d9700 1 -- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e410 msgr2=0x7f602c1c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f60311d9700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e410 0x7f602c1c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f60311d9700 1 -- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c09e000 msgr2=0x7f602c1c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f60311d9700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c09e000 0x7f602c1c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f60311d9700 1 -- 172.21.15.89:0/2966482956 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028009550 con 0x7f602c09d6f0 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f60311d9700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c1c8480 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f602800b3c0 tx=0x7f6028004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f60319da700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e410 0x7f602c1c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f6021ffb700 1 -- 172.21.15.89:0/2966482956 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028004380 con 0x7f602c09d6f0 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c0a2a80 con 0x7f602c09d6f0 2023-09-30T22:17:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f6021ffb700 1 -- 172.21.15.89:0/2966482956 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602801f070 con 0x7f602c09d6f0 2023-09-30T22:17:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f6021ffb700 1 -- 172.21.15.89:0/2966482956 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280036a0 con 0x7f602c09d6f0 2023-09-30T22:17:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.881+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602c125bd0 con 0x7f602c09d6f0 2023-09-30T22:17:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.883+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602c074ea0 con 0x7f602c09d6f0 2023-09-30T22:17:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.883+0000 7f6021ffb700 1 -- 172.21.15.89:0/2966482956 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6028021030 con 0x7f602c09d6f0 2023-09-30T22:17:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.883+0000 7f6021ffb700 1 -- 172.21.15.89:0/2966482956 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f60280409a0 con 0x7f602c09d6f0 2023-09-30T22:17:22.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:22.888+0000 7f6021ffb700 1 -- 172.21.15.89:0/2966482956 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60280239c0 con 0x7f602c09d6f0 2023-09-30T22:17:23.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.004+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fca7c005590 con 0x7fca9c09d6f0 2023-09-30T22:17:23.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.005+0000 7fca837fe700 1 -- 172.21.15.89:0/1267946212 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca8c015020 con 0x7fca9c09d6f0 2023-09-30T22:17:23.006 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca9c09d6f0 msgr2=0x7fca9c1b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca9c09d6f0 0x7fca9c1b7660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00a010 tx=0x7fca8c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 shutdown_connections 2023-09-30T22:17:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca9c09e410 0x7fca9c1b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca9c09d6f0 0x7fca9c1b7660 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 --2- 172.21.15.89:0/1267946212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca9c09e000 0x7fca9c1b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 >> 172.21.15.89:0/1267946212 conn(0x7fca9c066f50 msgr2=0x7fca9c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:23.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 shutdown_connections 2023-09-30T22:17:23.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.007+0000 7fcaa14bb700 1 -- 172.21.15.89:0/1267946212 wait complete. 2023-09-30T22:17:23.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:23.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.036+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb77c1258c0 con 0x7fb77c09d6f0 2023-09-30T22:17:23.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.037+0000 7fb772ffd700 1 -- 172.21.15.89:0/2977578182 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb77c1258c0 con 0x7fb77c09d6f0 2023-09-30T22:17:23.038 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:23.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb77c09d6f0 msgr2=0x7fb77c122430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:23.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb77c09d6f0 0x7fb77c122430 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb778009fd0 tx=0x7fb778004750 comp rx=0 tx=0).stop 2023-09-30T22:17:23.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 shutdown_connections 2023-09-30T22:17:23.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb77c09e410 0x7fb77c122eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb77c09d6f0 0x7fb77c122430 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 --2- 172.21.15.89:0/2977578182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb77c09e000 0x7fb77c122970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 >> 172.21.15.89:0/2977578182 conn(0x7fb77c066f50 msgr2=0x7fb77c0710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:23.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 shutdown_connections 2023-09-30T22:17:23.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.039+0000 7fb7847ce700 1 -- 172.21.15.89:0/2977578182 wait complete. 2023-09-30T22:17:23.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.081+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f602c06c360 con 0x7f602c09d6f0 2023-09-30T22:17:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.082+0000 7f6021ffb700 1 -- 172.21.15.89:0/2966482956 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60280233d0 con 0x7f602c09d6f0 2023-09-30T22:17:23.083 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:23.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.083+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 msgr2=0x7f602c1c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:23.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.083+0000 7f603343d700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c1c8480 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f602800b3c0 tx=0x7f6028004750 comp rx=0 tx=0).stop 2023-09-30T22:17:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.084+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 shutdown_connections 2023-09-30T22:17:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.084+0000 7f603343d700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f602c09e410 0x7f602c1c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.084+0000 7f603343d700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f602c09d6f0 0x7f602c1c8480 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.084+0000 7f603343d700 1 --2- 172.21.15.89:0/2966482956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602c09e000 0x7f602c1c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.084+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 >> 172.21.15.89:0/2966482956 conn(0x7f602c066f50 msgr2=0x7f602c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.084+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 shutdown_connections 2023-09-30T22:17:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:23.084+0000 7f603343d700 1 -- 172.21.15.89:0/2966482956 wait complete. 2023-09-30T22:17:23.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:23.889 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:24.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:24.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.023+0000 7f658ce95700 1 Processor -- start 2023-09-30T22:17:24.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.023+0000 7f658ce95700 1 -- start start 2023-09-30T22:17:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f658ce95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f6588138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f658ce95700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6588139170 0x7f658813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f658ce95700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658809b120 con 0x7f658809d6f0 2023-09-30T22:17:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f658ce95700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658806b460 con 0x7f6588139170 2023-09-30T22:17:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f658ce95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f658809a7e0 con 0x7f658809e1d0 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6586d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6588139170 0x7f658813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6585d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f6588138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6586d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6588139170 0x7f658813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32856/0 (socket says 172.21.15.89:32856) 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6585d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f6588138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44700/0 (socket says 172.21.15.89:44700) 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6586d9d700 1 -- 172.21.15.89:0/559414153 learned_addr learned my addr 172.21.15.89:0/559414153 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6585d9b700 1 -- 172.21.15.89:0/559414153 >> v2:172.21.15.89:3300/0 conn(0x7f6588139170 msgr2=0x7f658813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6585d9b700 1 --2- 172.21.15.89:0/559414153 >> v2:172.21.15.89:3300/0 conn(0x7f6588139170 0x7f658813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6585d9b700 1 -- 172.21.15.89:0/559414153 >> v1:172.21.15.89:6789/0 conn(0x7f658809d6f0 legacy=0x7f658809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6585d9b700 1 -- 172.21.15.89:0/559414153 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658809a500 con 0x7f658809e1d0 2023-09-30T22:17:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f6585d9b700 1 --2- 172.21.15.89:0/559414153 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f6588138c30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6570009870 tx=0x7f6570009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb297dffd57b392c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f658559a700 1 -- 172.21.15.89:0/559414153 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570011830 con 0x7f658809e1d0 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f658559a700 1 -- 172.21.15.89:0/559414153 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570011990 con 0x7f658809e1d0 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.024+0000 7f658559a700 1 -- 172.21.15.89:0/559414153 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570011d20 con 0x7f658809e1d0 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.025+0000 7f658ce95700 1 -- 172.21.15.89:0/559414153 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 msgr2=0x7f6588138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.025+0000 7f658ce95700 1 --2- 172.21.15.89:0/559414153 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f6588138c30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6570009870 tx=0x7f6570009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.025+0000 7f658ce95700 1 -- 172.21.15.89:0/559414153 shutdown_connections 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.025+0000 7f658ce95700 1 --2- 172.21.15.89:0/559414153 >> v2:172.21.15.89:3300/0 conn(0x7f6588139170 0x7f658813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.025+0000 7f658ce95700 1 --2- 172.21.15.89:0/559414153 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f6588138c30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.025+0000 7f658ce95700 1 -- 172.21.15.89:0/559414153 >> 172.21.15.89:0/559414153 conn(0x7f6588066f50 msgr2=0x7f658809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.025+0000 7f658ce95700 1 -- 172.21.15.89:0/559414153 shutdown_connections 2023-09-30T22:17:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.025+0000 7f658ce95700 1 -- 172.21.15.89:0/559414153 wait complete. 2023-09-30T22:17:24.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.026+0000 7f658ce95700 1 Processor -- start 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.026+0000 7f658ce95700 1 -- start start 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f658ce95700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658809d6f0 0x7f658811f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f658ce95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f658811f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f658ce95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6588139170 0x7f65881c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f658ce95700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658811efb0 con 0x7f658809d6f0 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f658ce95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658811fe80 con 0x7f6588139170 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f658ce95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6588122240 con 0x7f658809e1d0 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f658811f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f658811f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44702/0 (socket says 172.21.15.89:44702) 2023-09-30T22:17:24.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 -- 172.21.15.89:0/3258975481 learned_addr learned my addr 172.21.15.89:0/3258975481 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f658659c700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658809d6f0 0x7f658811f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6586d9d700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6588139170 0x7f65881c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 -- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6588139170 msgr2=0x7f65881c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6588139170 0x7f65881c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 -- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658809d6f0 msgr2=0x7f658811f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658809d6f0 0x7f658811f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 -- 172.21.15.89:0/3258975481 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700094a0 con 0x7f658809e1d0 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.027+0000 7f6585d9b700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f658811f940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6570000c00 tx=0x7f6570004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.028+0000 7f657effd700 1 -- 172.21.15.89:0/3258975481 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700036a0 con 0x7f658809e1d0 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.028+0000 7f657effd700 1 -- 172.21.15.89:0/3258975481 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657001f070 con 0x7f658809e1d0 2023-09-30T22:17:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.028+0000 7f657effd700 1 -- 172.21.15.89:0/3258975481 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570003a20 con 0x7f658809e1d0 2023-09-30T22:17:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.028+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658809c710 con 0x7f658809e1d0 2023-09-30T22:17:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.028+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65881212d0 con 0x7f658809e1d0 2023-09-30T22:17:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.029+0000 7f657effd700 1 -- 172.21.15.89:0/3258975481 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6570003b80 con 0x7f658809e1d0 2023-09-30T22:17:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.029+0000 7f657effd700 1 -- 172.21.15.89:0/3258975481 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6570021070 con 0x7f658809e1d0 2023-09-30T22:17:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.029+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658804ea50 con 0x7f658809e1d0 2023-09-30T22:17:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.034+0000 7f657effd700 1 -- 172.21.15.89:0/3258975481 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f657002b080 con 0x7f658809e1d0 2023-09-30T22:17:24.052 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:24.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:24.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.152+0000 7f37ad5ad700 1 Processor -- start 2023-09-30T22:17:24.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.152+0000 7f37ad5ad700 1 -- start start 2023-09-30T22:17:24.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.152+0000 7f37ad5ad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.152+0000 7f37ad5ad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37a8130430 0x7f37a8137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.152+0000 7f37ad5ad700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a812cd90 con 0x7f37a812f8b0 2023-09-30T22:17:24.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.152+0000 7f37ad5ad700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a812d330 con 0x7f37a8130430 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.152+0000 7f37ad5ad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8071780 con 0x7f37a812cf60 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37a8130430 0x7f37a8137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37a8130430 0x7f37a8137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32866/0 (socket says 172.21.15.89:32866) 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a77fe700 1 -- 172.21.15.89:0/2172126949 learned_addr learned my addr 172.21.15.89:0/2172126949 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a6ffd700 1 --2- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a77fe700 1 -- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 msgr2=0x7f37a806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a77fe700 1 --2- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a77fe700 1 -- 172.21.15.89:0/2172126949 >> v1:172.21.15.89:6789/0 conn(0x7f37a812f8b0 legacy=0x7f37a812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a77fe700 1 -- 172.21.15.89:0/2172126949 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a8071a40 con 0x7f37a8130430 2023-09-30T22:17:24.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a6ffd700 1 --2- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:24.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a77fe700 1 --2- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 0x7f37a8137c30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f37a000bef0 tx=0x7f37a000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5f21bbebc3b698 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:24.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.153+0000 7f37a5ffb700 1 -- 172.21.15.89:0/2172126949 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a000a2e0 con 0x7f37a8130430 2023-09-30T22:17:24.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37a5ffb700 1 -- 172.21.15.89:0/2172126949 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a000cb50 con 0x7f37a8130430 2023-09-30T22:17:24.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37a5ffb700 1 -- 172.21.15.89:0/2172126949 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0014480 con 0x7f37a8130430 2023-09-30T22:17:24.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37ad5ad700 1 -- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 msgr2=0x7f37a8137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37ad5ad700 1 --2- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 0x7f37a8137c30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f37a000bef0 tx=0x7f37a000a4c0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37ad5ad700 1 -- 172.21.15.89:0/2172126949 shutdown_connections 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37ad5ad700 1 --2- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 0x7f37a8137c30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37ad5ad700 1 --2- 172.21.15.89:0/2172126949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37ad5ad700 1 -- 172.21.15.89:0/2172126949 >> 172.21.15.89:0/2172126949 conn(0x7f37a80a2aa0 msgr2=0x7f37a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37ad5ad700 1 -- 172.21.15.89:0/2172126949 shutdown_connections 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.154+0000 7f37ad5ad700 1 -- 172.21.15.89:0/2172126949 wait complete. 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.156+0000 7f37ad5ad700 1 Processor -- start 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.156+0000 7f37ad5ad700 1 -- start start 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37ad5ad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a81c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37ad5ad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37a812f8b0 0x7f37a81c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37ad5ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 0x7f37a81c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37ad5ad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a812a190 con 0x7f37a8130430 2023-09-30T22:17:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37ad5ad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8129fa0 con 0x7f37a812cf60 2023-09-30T22:17:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37ad5ad700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37a809d570 con 0x7f37a812f8b0 2023-09-30T22:17:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a81c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a81c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43246/0 (socket says 172.21.15.89:43246) 2023-09-30T22:17:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 -- 172.21.15.89:0/1891597100 learned_addr learned my addr 172.21.15.89:0/1891597100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a67fc700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37a812f8b0 0x7f37a81c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a77fe700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 0x7f37a81c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 -- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37a812f8b0 msgr2=0x7f37a81c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37a812f8b0 0x7f37a81c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 -- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 msgr2=0x7f37a81c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 0x7f37a81c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 -- 172.21.15.89:0/1891597100 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379800d940 con 0x7f37a812cf60 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a77fe700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 0x7f37a81c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37a6ffd700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a81c3f90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f379800de30 tx=0x7f37980103a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f378f7fe700 1 -- 172.21.15.89:0/1891597100 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798010b50 con 0x7f37a812cf60 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f378f7fe700 1 -- 172.21.15.89:0/1891597100 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3798013040 con 0x7f37a812cf60 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a000bb50 con 0x7f37a812cf60 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.157+0000 7f378f7fe700 1 -- 172.21.15.89:0/1891597100 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798010ed0 con 0x7f37a812cf60 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.158+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a812f550 con 0x7f37a812cf60 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.159+0000 7f378f7fe700 1 -- 172.21.15.89:0/1891597100 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3798018480 con 0x7f37a812cf60 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.159+0000 7f378f7fe700 1 -- 172.21.15.89:0/1891597100 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3798047290 con 0x7f37a812cf60 2023-09-30T22:17:24.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.159+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a8074ea0 con 0x7f37a812cf60 2023-09-30T22:17:24.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.167+0000 7f378f7fe700 1 -- 172.21.15.89:0/1891597100 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3798016930 con 0x7f37a812cf60 2023-09-30T22:17:24.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.182+0000 7fd26a5dd700 1 Processor -- start 2023-09-30T22:17:24.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd26a5dd700 1 -- start start 2023-09-30T22:17:24.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd26a5dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409de00 0x7fd26409e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd26a5dd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd26413d1f0 0x7fd26413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd26a5dd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26409a810 con 0x7fd26406a770 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd26a5dd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26406ab40 con 0x7fd26413d1f0 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd26a5dd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd26409ebc0 con 0x7fd26409de00 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409de00 0x7fd26409e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409de00 0x7fd26409e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44720/0 (socket says 172.21.15.89:44720) 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd2637fe700 1 -- 172.21.15.89:0/2021847341 learned_addr learned my addr 172.21.15.89:0/2021847341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.183+0000 7fd268b7a700 1 --2- 172.21.15.89:0/2021847341 >> v2:172.21.15.89:3300/0 conn(0x7fd26413d1f0 0x7fd26413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.184+0000 7fd2637fe700 1 -- 172.21.15.89:0/2021847341 >> v2:172.21.15.89:3300/0 conn(0x7fd26413d1f0 msgr2=0x7fd26413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:24.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.184+0000 7fd2637fe700 1 --2- 172.21.15.89:0/2021847341 >> v2:172.21.15.89:3300/0 conn(0x7fd26413d1f0 0x7fd26413f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.184+0000 7fd2637fe700 1 -- 172.21.15.89:0/2021847341 >> v1:172.21.15.89:6789/0 conn(0x7fd26406a770 legacy=0x7fd26409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.184+0000 7fd2637fe700 1 -- 172.21.15.89:0/2021847341 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26409e920 con 0x7fd26409de00 2023-09-30T22:17:24.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.184+0000 7fd2637fe700 1 --2- 172.21.15.89:0/2021847341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409de00 0x7fd26409e250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2580098f0 tx=0x7fd258009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d4a834f98be5203 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:24.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.184+0000 7fd262ffd700 1 -- 172.21.15.89:0/2021847341 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258011830 con 0x7fd26409de00 2023-09-30T22:17:24.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.184+0000 7fd262ffd700 1 -- 172.21.15.89:0/2021847341 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd258011990 con 0x7fd26409de00 2023-09-30T22:17:24.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.184+0000 7fd262ffd700 1 -- 172.21.15.89:0/2021847341 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258011d20 con 0x7fd26409de00 2023-09-30T22:17:24.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.185+0000 7fd26a5dd700 1 -- 172.21.15.89:0/2021847341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409de00 msgr2=0x7fd26409e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.185+0000 7fd26a5dd700 1 --2- 172.21.15.89:0/2021847341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409de00 0x7fd26409e250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2580098f0 tx=0x7fd258009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:24.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.185+0000 7fd26a5dd700 1 -- 172.21.15.89:0/2021847341 shutdown_connections 2023-09-30T22:17:24.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.185+0000 7fd26a5dd700 1 --2- 172.21.15.89:0/2021847341 >> v2:172.21.15.89:3300/0 conn(0x7fd26413d1f0 0x7fd26413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.185+0000 7fd26a5dd700 1 --2- 172.21.15.89:0/2021847341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409de00 0x7fd26409e250 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.185+0000 7fd26a5dd700 1 -- 172.21.15.89:0/2021847341 >> 172.21.15.89:0/2021847341 conn(0x7fd264066f50 msgr2=0x7fd26409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:24.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.185+0000 7fd26a5dd700 1 -- 172.21.15.89:0/2021847341 shutdown_connections 2023-09-30T22:17:24.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.185+0000 7fd26a5dd700 1 -- 172.21.15.89:0/2021847341 wait complete. 2023-09-30T22:17:24.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.186+0000 7fd26a5dd700 1 Processor -- start 2023-09-30T22:17:24.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.186+0000 7fd26a5dd700 1 -- start start 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd26a5dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26406a770 0x7fd2641260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd26a5dd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409de00 0x7fd2641265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd26a5dd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26413d1f0 0x7fd264126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd26a5dd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264072790 con 0x7fd26413d1f0 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd26a5dd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640a3990 con 0x7fd26409de00 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd26a5dd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd26409f1a0 con 0x7fd26406a770 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409de00 0x7fd2641265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409de00 0x7fd2641265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43262/0 (socket says 172.21.15.89:43262) 2023-09-30T22:17:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd263fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26406a770 0x7fd2641260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd2637fe700 1 -- 172.21.15.89:0/30635090 learned_addr learned my addr 172.21.15.89:0/30635090 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.187+0000 7fd268b7a700 1 --2- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26413d1f0 0x7fd264126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd2637fe700 1 -- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26406a770 msgr2=0x7fd2641260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd2637fe700 1 --2- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26406a770 0x7fd2641260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd2637fe700 1 -- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26413d1f0 msgr2=0x7fd264126b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd2637fe700 1 --2- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26413d1f0 0x7fd264126b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd2637fe700 1 -- 172.21.15.89:0/30635090 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258009550 con 0x7fd26409de00 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd2637fe700 1 --2- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409de00 0x7fd2641265e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd258000c00 tx=0x7fd258004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd260ff9700 1 -- 172.21.15.89:0/30635090 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258004400 con 0x7fd26409de00 2023-09-30T22:17:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd260ff9700 1 -- 172.21.15.89:0/30635090 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25801f070 con 0x7fd26409de00 2023-09-30T22:17:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.188+0000 7fd260ff9700 1 -- 172.21.15.89:0/30635090 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2580038e0 con 0x7fd26409de00 2023-09-30T22:17:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.189+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd264127f40 con 0x7fd26409de00 2023-09-30T22:17:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.189+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2641284d0 con 0x7fd26409de00 2023-09-30T22:17:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.190+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2641280d0 con 0x7fd26409de00 2023-09-30T22:17:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.190+0000 7fd260ff9700 1 -- 172.21.15.89:0/30635090 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd258021030 con 0x7fd26409de00 2023-09-30T22:17:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.190+0000 7fd260ff9700 1 -- 172.21.15.89:0/30635090 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd258040990 con 0x7fd26409de00 2023-09-30T22:17:24.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.195+0000 7fd260ff9700 1 -- 172.21.15.89:0/30635090 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd2580199a0 con 0x7fd26409de00 2023-09-30T22:17:24.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.227+0000 7faf1882f700 1 Processor -- start 2023-09-30T22:17:24.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.227+0000 7faf1882f700 1 -- start start 2023-09-30T22:17:24.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.227+0000 7faf1882f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.227+0000 7faf1882f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf14099a40 0x7faf14099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.227+0000 7faf1882f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf141313c0 con 0x7faf1409a3b0 2023-09-30T22:17:24.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.227+0000 7faf1882f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1409b580 con 0x7faf14099a40 2023-09-30T22:17:24.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.227+0000 7faf1882f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf1406e8c0 con 0x7faf14131810 2023-09-30T22:17:24.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44730/0 (socket says 172.21.15.89:44730) 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1259c700 1 -- 172.21.15.89:0/3393780592 learned_addr learned my addr 172.21.15.89:0/3393780592 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf11d9b700 1 --2- 172.21.15.89:0/3393780592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf14099a40 0x7faf14099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1259c700 1 -- 172.21.15.89:0/3393780592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf14099a40 msgr2=0x7faf14099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1259c700 1 --2- 172.21.15.89:0/3393780592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf14099a40 0x7faf14099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1259c700 1 -- 172.21.15.89:0/3393780592 >> v1:172.21.15.89:6789/0 conn(0x7faf1409a3b0 legacy=0x7faf14138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1259c700 1 -- 172.21.15.89:0/3393780592 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1409a820 con 0x7faf14131810 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1259c700 1 --2- 172.21.15.89:0/3393780592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14131be0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf04009920 tx=0x7faf04009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42336f8b74f3a9ae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1159a700 1 -- 172.21.15.89:0/3393780592 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04004230 con 0x7faf14131810 2023-09-30T22:17:24.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1159a700 1 -- 172.21.15.89:0/3393780592 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf04004390 con 0x7faf14131810 2023-09-30T22:17:24.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.228+0000 7faf1159a700 1 -- 172.21.15.89:0/3393780592 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04009e70 con 0x7faf14131810 2023-09-30T22:17:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.229+0000 7faf1882f700 1 -- 172.21.15.89:0/3393780592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 msgr2=0x7faf14131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.229+0000 7faf1882f700 1 --2- 172.21.15.89:0/3393780592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14131be0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf04009920 tx=0x7faf04009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.229+0000 7faf1882f700 1 -- 172.21.15.89:0/3393780592 shutdown_connections 2023-09-30T22:17:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.229+0000 7faf1882f700 1 --2- 172.21.15.89:0/3393780592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14131be0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.229+0000 7faf1882f700 1 --2- 172.21.15.89:0/3393780592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf14099a40 0x7faf14099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.229+0000 7faf1882f700 1 -- 172.21.15.89:0/3393780592 >> 172.21.15.89:0/3393780592 conn(0x7faf140a2aa0 msgr2=0x7faf140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.229+0000 7faf1882f700 1 -- 172.21.15.89:0/3393780592 shutdown_connections 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.229+0000 7faf1882f700 1 -- 172.21.15.89:0/3393780592 wait complete. 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.230+0000 7faf1882f700 1 Processor -- start 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.230+0000 7faf1882f700 1 -- start start 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1882f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf14099a40 0x7faf1406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1882f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf1409a3b0 0x7faf1406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1882f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1882f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14070ae0 con 0x7faf14099a40 2023-09-30T22:17:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1882f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf14121140 con 0x7faf1409a3b0 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1882f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf1406eb00 con 0x7faf14131810 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf14099a40 0x7faf1406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf14099a40 0x7faf1406f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32892/0 (socket says 172.21.15.89:32892) 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf12d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1259c700 1 -- 172.21.15.89:0/1262130795 learned_addr learned my addr 172.21.15.89:0/1262130795 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf11d9b700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf1409a3b0 0x7faf1406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf11d9b700 1 -- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 msgr2=0x7faf14070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf11d9b700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf11d9b700 1 -- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf14099a40 msgr2=0x7faf1406f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf11d9b700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf14099a40 0x7faf1406f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf11d9b700 1 -- 172.21.15.89:0/1262130795 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04009550 con 0x7faf1409a3b0 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf1259c700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf14099a40 0x7faf1406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.231+0000 7faf11d9b700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf1409a3b0 0x7faf1406fb50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e7e0 tx=0x7faf0800eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.232+0000 7faf02ffd700 1 -- 172.21.15.89:0/1262130795 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800c070 con 0x7faf1409a3b0 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.232+0000 7faf02ffd700 1 -- 172.21.15.89:0/1262130795 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf08004020 con 0x7faf1409a3b0 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.232+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf14120e80 con 0x7faf1409a3b0 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.232+0000 7faf02ffd700 1 -- 172.21.15.89:0/1262130795 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf080043b0 con 0x7faf1409a3b0 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.232+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf141203c0 con 0x7faf1409a3b0 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.233+0000 7faf02ffd700 1 -- 172.21.15.89:0/1262130795 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf080036a0 con 0x7faf1409a3b0 2023-09-30T22:17:24.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.233+0000 7faf02ffd700 1 -- 172.21.15.89:0/1262130795 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faf0803efe0 con 0x7faf1409a3b0 2023-09-30T22:17:24.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.233+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf14121280 con 0x7faf1409a3b0 2023-09-30T22:17:24.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.240+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f658809f420 con 0x7f658809e1d0 2023-09-30T22:17:24.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.241+0000 7f657effd700 1 -- 172.21.15.89:0/3258975481 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6570015020 con 0x7f658809e1d0 2023-09-30T22:17:24.241 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:24.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.242+0000 7faf02ffd700 1 -- 172.21.15.89:0/1262130795 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf0801c410 con 0x7faf1409a3b0 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 msgr2=0x7f658811f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f658811f940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6570000c00 tx=0x7f6570004750 comp rx=0 tx=0).stop 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 shutdown_connections 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6588139170 0x7f65881c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658809d6f0 0x7f658811f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 --2- 172.21.15.89:0/3258975481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658809e1d0 0x7f658811f940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 >> 172.21.15.89:0/3258975481 conn(0x7f6588066f50 msgr2=0x7f65880a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 shutdown_connections 2023-09-30T22:17:24.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.243+0000 7f658ce95700 1 -- 172.21.15.89:0/3258975481 wait complete. 2023-09-30T22:17:24.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:24.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.368+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f37a80713a0 con 0x7f37a812cf60 2023-09-30T22:17:24.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.369+0000 7f378f7fe700 1 -- 172.21.15.89:0/1891597100 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3798016340 con 0x7f37a812cf60 2023-09-30T22:17:24.370 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 msgr2=0x7f37a81c3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a81c3f90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f379800de30 tx=0x7f37980103a0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 shutdown_connections 2023-09-30T22:17:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37a8130430 0x7f37a81c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37a812cf60 0x7f37a81c3f90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 --2- 172.21.15.89:0/1891597100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37a812f8b0 0x7f37a81c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 >> 172.21.15.89:0/1891597100 conn(0x7f37a80a2aa0 msgr2=0x7f37a806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 shutdown_connections 2023-09-30T22:17:24.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.371+0000 7f37ad5ad700 1 -- 172.21.15.89:0/1891597100 wait complete. 2023-09-30T22:17:24.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.396+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd26411cc30 con 0x7fd26409de00 2023-09-30T22:17:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.396+0000 7fd260ff9700 1 -- 172.21.15.89:0/30635090 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd25801b430 con 0x7fd26409de00 2023-09-30T22:17:24.397 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409de00 msgr2=0x7fd2641265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 --2- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409de00 0x7fd2641265e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd258000c00 tx=0x7fd258004750 comp rx=0 tx=0).stop 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 shutdown_connections 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 --2- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26413d1f0 0x7fd264126b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 --2- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26406a770 0x7fd2641260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 --2- 172.21.15.89:0/30635090 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409de00 0x7fd2641265e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 >> 172.21.15.89:0/30635090 conn(0x7fd264066f50 msgr2=0x7fd2640a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 shutdown_connections 2023-09-30T22:17:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.398+0000 7fd26a5dd700 1 -- 172.21.15.89:0/30635090 wait complete. 2023-09-30T22:17:24.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:24.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.438+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faf14120180 con 0x7faf1409a3b0 2023-09-30T22:17:24.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.438+0000 7faf02ffd700 1 -- 172.21.15.89:0/1262130795 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf0802b090 con 0x7faf1409a3b0 2023-09-30T22:17:24.439 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:24.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf1409a3b0 msgr2=0x7faf1406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:24.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf1409a3b0 0x7faf1406fb50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e7e0 tx=0x7faf0800eaf0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 shutdown_connections 2023-09-30T22:17:24.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf14131810 0x7faf14070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf14099a40 0x7faf1406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 --2- 172.21.15.89:0/1262130795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf1409a3b0 0x7faf1406fb50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:24.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 >> 172.21.15.89:0/1262130795 conn(0x7faf140a2aa0 msgr2=0x7faf141b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:24.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 shutdown_connections 2023-09-30T22:17:24.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:24.440+0000 7faf1882f700 1 -- 172.21.15.89:0/1262130795 wait complete. 2023-09-30T22:17:24.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:25.259 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:25.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:25.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.385+0000 7effae0ea700 1 Processor -- start 2023-09-30T22:17:25.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effae0ea700 1 -- start start 2023-09-30T22:17:25.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effae0ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effae0ea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7effa809e000 0x7effa81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effae0ea700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa809b120 con 0x7effa809e410 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effae0ea700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa806b460 con 0x7effa809e000 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effae0ea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa809a7e0 con 0x7effa809d6f0 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7effa809e000 0x7effa81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7effa809e000 0x7effa81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32894/0 (socket says 172.21.15.89:32894) 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43274/0 (socket says 172.21.15.89:43274) 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa6ffd700 1 -- 172.21.15.89:0/681867273 learned_addr learned my addr 172.21.15.89:0/681867273 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa6ffd700 1 -- 172.21.15.89:0/681867273 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 msgr2=0x7effa809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa6ffd700 1 --2- 172.21.15.89:0/681867273 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa6ffd700 1 -- 172.21.15.89:0/681867273 >> v1:172.21.15.89:6789/0 conn(0x7effa809e410 legacy=0x7effa8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa6ffd700 1 -- 172.21.15.89:0/681867273 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa809a500 con 0x7effa809e000 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.386+0000 7effa67fc700 1 -- 172.21.15.89:0/681867273 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791046729 0 0) 0x7effa809b120 con 0x7effa809e410 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effa6ffd700 1 --2- 172.21.15.89:0/681867273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81339f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0098f0 tx=0x7eff9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de842c2274cdde35 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effa67fc700 1 -- 172.21.15.89:0/681867273 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c011830 con 0x7effa809e000 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effa67fc700 1 -- 172.21.15.89:0/681867273 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c011990 con 0x7effa809e000 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effa67fc700 1 -- 172.21.15.89:0/681867273 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c011d20 con 0x7effa809e000 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effae0ea700 1 -- 172.21.15.89:0/681867273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 msgr2=0x7effa81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effae0ea700 1 --2- 172.21.15.89:0/681867273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81339f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0098f0 tx=0x7eff9c009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effae0ea700 1 -- 172.21.15.89:0/681867273 shutdown_connections 2023-09-30T22:17:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effae0ea700 1 --2- 172.21.15.89:0/681867273 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effae0ea700 1 --2- 172.21.15.89:0/681867273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81339f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.387+0000 7effae0ea700 1 -- 172.21.15.89:0/681867273 >> 172.21.15.89:0/681867273 conn(0x7effa8066f50 msgr2=0x7effa809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.388+0000 7effae0ea700 1 -- 172.21.15.89:0/681867273 shutdown_connections 2023-09-30T22:17:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.388+0000 7effae0ea700 1 -- 172.21.15.89:0/681867273 wait complete. 2023-09-30T22:17:25.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.388+0000 7effae0ea700 1 Processor -- start 2023-09-30T22:17:25.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effae0ea700 1 -- start start 2023-09-30T22:17:25.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effae0ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa81c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effae0ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effae0ea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7effa809e410 0x7effa81c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effae0ea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa81a7fb0 con 0x7effa809e000 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effae0ea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa81b9320 con 0x7effa809d6f0 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effae0ea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effa81a6500 con 0x7effa809e410 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effa6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effa6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32902/0 (socket says 172.21.15.89:32902) 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effa6ffd700 1 -- 172.21.15.89:0/1522492431 learned_addr learned my addr 172.21.15.89:0/1522492431 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effa77fe700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa81c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.389+0000 7effa7fff700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7effa809e410 0x7effa81c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effa6ffd700 1 -- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7effa809e410 msgr2=0x7effa81c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effa6ffd700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7effa809e410 0x7effa81c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effa6ffd700 1 -- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 msgr2=0x7effa81c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effa6ffd700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa81c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effa6ffd700 1 -- 172.21.15.89:0/1522492431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009550 con 0x7effa809e000 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effa77fe700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa81c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effa6ffd700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81c8510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff9c013000 tx=0x7eff9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7eff8ffff700 1 -- 172.21.15.89:0/1522492431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c01a900 con 0x7effa809e000 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7eff8ffff700 1 -- 172.21.15.89:0/1522492431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c01f070 con 0x7effa809e000 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7eff8ffff700 1 -- 172.21.15.89:0/1522492431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c01aca0 con 0x7effa809e000 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa809c710 con 0x7effa809e000 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.390+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa81a8a20 con 0x7effa809e000 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.391+0000 7eff8ffff700 1 -- 172.21.15.89:0/1522492431 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff9c026480 con 0x7effa809e000 2023-09-30T22:17:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.391+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa81a8100 con 0x7effa809e000 2023-09-30T22:17:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.391+0000 7eff8ffff700 1 -- 172.21.15.89:0/1522492431 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7eff9c021070 con 0x7effa809e000 2023-09-30T22:17:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.396+0000 7eff8ffff700 1 -- 172.21.15.89:0/1522492431 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff9c026730 con 0x7effa809e000 2023-09-30T22:17:25.412 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:25.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:25.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.513+0000 7f82c9d51700 1 Processor -- start 2023-09-30T22:17:25.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.513+0000 7f82c9d51700 1 -- start start 2023-09-30T22:17:25.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.513+0000 7f82c9d51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c406a770 0x7f82c409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.513+0000 7f82c9d51700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f82c409dc30 0x7f82c409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.513+0000 7f82c9d51700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c409a810 con 0x7f82c413c1e0 2023-09-30T22:17:25.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.513+0000 7f82c9d51700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c406ab40 con 0x7f82c409dc30 2023-09-30T22:17:25.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.513+0000 7f82c9d51700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c409ebc0 con 0x7f82c406a770 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c406a770 0x7f82c409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c406a770 0x7f82c409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43294/0 (socket says 172.21.15.89:43294) 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c37fe700 1 -- 172.21.15.89:0/1368894345 learned_addr learned my addr 172.21.15.89:0/1368894345 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c2ffd700 1 --2- 172.21.15.89:0/1368894345 >> v2:172.21.15.89:3300/0 conn(0x7f82c409dc30 0x7f82c409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c37fe700 1 -- 172.21.15.89:0/1368894345 >> v2:172.21.15.89:3300/0 conn(0x7f82c409dc30 msgr2=0x7f82c409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c37fe700 1 --2- 172.21.15.89:0/1368894345 >> v2:172.21.15.89:3300/0 conn(0x7f82c409dc30 0x7f82c409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c37fe700 1 -- 172.21.15.89:0/1368894345 >> v1:172.21.15.89:6789/0 conn(0x7f82c413c1e0 legacy=0x7f82c413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c37fe700 1 -- 172.21.15.89:0/1368894345 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c409e920 con 0x7f82c406a770 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c37fe700 1 --2- 172.21.15.89:0/1368894345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c406a770 0x7f82c409d6f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009920 tx=0x7f82b4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bb81dcbba8ff6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c27fc700 1 -- 172.21.15.89:0/1368894345 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4011830 con 0x7f82c406a770 2023-09-30T22:17:25.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c27fc700 1 -- 172.21.15.89:0/1368894345 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4011990 con 0x7f82c406a770 2023-09-30T22:17:25.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.514+0000 7f82c27fc700 1 -- 172.21.15.89:0/1368894345 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4011d20 con 0x7f82c406a770 2023-09-30T22:17:25.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.515+0000 7f82c9d51700 1 -- 172.21.15.89:0/1368894345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c406a770 msgr2=0x7f82c409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.515+0000 7f82c9d51700 1 --2- 172.21.15.89:0/1368894345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c406a770 0x7f82c409d6f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009920 tx=0x7f82b4009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.515+0000 7f82c9d51700 1 -- 172.21.15.89:0/1368894345 shutdown_connections 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.515+0000 7f82c9d51700 1 --2- 172.21.15.89:0/1368894345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c406a770 0x7f82c409d6f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.515+0000 7f82c9d51700 1 --2- 172.21.15.89:0/1368894345 >> v2:172.21.15.89:3300/0 conn(0x7f82c409dc30 0x7f82c409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.515+0000 7f82c9d51700 1 -- 172.21.15.89:0/1368894345 >> 172.21.15.89:0/1368894345 conn(0x7f82c4066f50 msgr2=0x7f82c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.515+0000 7f82c9d51700 1 -- 172.21.15.89:0/1368894345 shutdown_connections 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.515+0000 7f82c9d51700 1 -- 172.21.15.89:0/1368894345 wait complete. 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.517+0000 7f82c9d51700 1 Processor -- start 2023-09-30T22:17:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.517+0000 7f82c9d51700 1 -- start start 2023-09-30T22:17:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c9d51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82c406a770 0x7f82c41cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c9d51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c409dc30 0x7f82c41cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c9d51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82c413c1e0 0x7f82c41cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c9d51700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c409f1a0 con 0x7f82c413c1e0 2023-09-30T22:17:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c9d51700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4128f00 con 0x7f82c409dc30 2023-09-30T22:17:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c9d51700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82c412a6f0 con 0x7f82c406a770 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82c413c1e0 0x7f82c41cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82c406a770 0x7f82c41cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82c413c1e0 0x7f82c41cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32914/0 (socket says 172.21.15.89:32914) 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c2ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c409dc30 0x7f82c41cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c3fff700 1 -- 172.21.15.89:0/152583093 learned_addr learned my addr 172.21.15.89:0/152583093 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c3fff700 1 -- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82c406a770 msgr2=0x7f82c41cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c3fff700 1 --2- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82c406a770 0x7f82c41cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c3fff700 1 -- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c409dc30 msgr2=0x7f82c41cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c3fff700 1 --2- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c409dc30 0x7f82c41cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.518+0000 7f82c3fff700 1 -- 172.21.15.89:0/152583093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4009550 con 0x7f82c413c1e0 2023-09-30T22:17:25.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.519+0000 7f82c3fff700 1 --2- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82c413c1e0 0x7f82c41cced0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00cbc0 tx=0x7f82bc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.519+0000 7f82abfff700 1 -- 172.21.15.89:0/152583093 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc003c00 con 0x7f82c413c1e0 2023-09-30T22:17:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.519+0000 7f82abfff700 1 -- 172.21.15.89:0/152583093 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc003d60 con 0x7f82c413c1e0 2023-09-30T22:17:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.519+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c413bc20 con 0x7f82c413c1e0 2023-09-30T22:17:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.519+0000 7f82abfff700 1 -- 172.21.15.89:0/152583093 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00aae0 con 0x7f82c413c1e0 2023-09-30T22:17:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.519+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c412b420 con 0x7f82c413c1e0 2023-09-30T22:17:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.520+0000 7f82abfff700 1 -- 172.21.15.89:0/152583093 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f82bc00ac40 con 0x7f82c413c1e0 2023-09-30T22:17:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.520+0000 7f82abfff700 1 -- 172.21.15.89:0/152583093 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f82bc03f550 con 0x7f82c413c1e0 2023-09-30T22:17:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.520+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c412a8b0 con 0x7f82c413c1e0 2023-09-30T22:17:25.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.525+0000 7f82abfff700 1 -- 172.21.15.89:0/152583093 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f82bc0198f0 con 0x7f82c413c1e0 2023-09-30T22:17:25.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.546+0000 7fe000aab700 1 Processor -- start 2023-09-30T22:17:25.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.546+0000 7fe000aab700 1 -- start start 2023-09-30T22:17:25.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.547+0000 7fe000aab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.547+0000 7fe000aab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdffc09f240 0x7fdffc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.547+0000 7fe000aab700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdffc070940 con 0x7fdffc09f650 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.547+0000 7fe000aab700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc06fd60 con 0x7fdffc09f240 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.547+0000 7fe000aab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc06fa80 con 0x7fdffc09e930 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff9d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdffc09f240 0x7fdffc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdffad9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdffc09f650 0x7fdffc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49924/0 (socket says 172.21.15.89:49924) 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdffad9d700 1 -- 172.21.15.89:0/1619565601 learned_addr learned my addr 172.21.15.89:0/1619565601 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdffa59c700 1 --2- 172.21.15.89:0/1619565601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff959a700 1 -- 172.21.15.89:0/1619565601 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282464548 0 0) 0x7fdffc070940 con 0x7fdffc09f650 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff959a700 1 -- 172.21.15.89:0/1619565601 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe40035b0 con 0x7fdffc09f650 2023-09-30T22:17:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff9d9b700 1 -- 172.21.15.89:0/1619565601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 msgr2=0x7fdffc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff9d9b700 1 --2- 172.21.15.89:0/1619565601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff9d9b700 1 -- 172.21.15.89:0/1619565601 >> v1:172.21.15.89:6789/0 conn(0x7fdffc09f650 legacy=0x7fdffc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff9d9b700 1 -- 172.21.15.89:0/1619565601 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc09fb60 con 0x7fdffc09f240 2023-09-30T22:17:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff9d9b700 1 --2- 172.21.15.89:0/1619565601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f240 0x7fdffc1317f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009840 tx=0x7fdff0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd3c00b2e7de5b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff959a700 1 -- 172.21.15.89:0/1619565601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0011830 con 0x7fdffc09f240 2023-09-30T22:17:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff959a700 1 -- 172.21.15.89:0/1619565601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff0011990 con 0x7fdffc09f240 2023-09-30T22:17:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.548+0000 7fdff959a700 1 -- 172.21.15.89:0/1619565601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0011d20 con 0x7fdffc09f240 2023-09-30T22:17:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.549+0000 7fe000aab700 1 -- 172.21.15.89:0/1619565601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f240 msgr2=0x7fdffc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.549+0000 7fe000aab700 1 --2- 172.21.15.89:0/1619565601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f240 0x7fdffc1317f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009840 tx=0x7fdff0009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.550+0000 7fe000aab700 1 -- 172.21.15.89:0/1619565601 shutdown_connections 2023-09-30T22:17:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.550+0000 7fe000aab700 1 --2- 172.21.15.89:0/1619565601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.550+0000 7fe000aab700 1 --2- 172.21.15.89:0/1619565601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f240 0x7fdffc1317f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.550+0000 7fe000aab700 1 -- 172.21.15.89:0/1619565601 >> 172.21.15.89:0/1619565601 conn(0x7fdffc0a4210 msgr2=0x7fdffc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.550+0000 7fe000aab700 1 -- 172.21.15.89:0/1619565601 shutdown_connections 2023-09-30T22:17:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.550+0000 7fe000aab700 1 -- 172.21.15.89:0/1619565601 wait complete. 2023-09-30T22:17:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.551+0000 7fe000aab700 1 Processor -- start 2023-09-30T22:17:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.551+0000 7fe000aab700 1 -- start start 2023-09-30T22:17:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fe000aab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc1c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fe000aab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdffc09f240 0x7fdffc1c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fe000aab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f650 0x7fdffc1c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fe000aab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc1364c0 con 0x7fdffc09f650 2023-09-30T22:17:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fe000aab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc138830 con 0x7fdffc09f240 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fe000aab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc136ba0 con 0x7fdffc09e930 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f650 0x7fdffc1c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffa59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc1c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffa59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc1c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44776/0 (socket says 172.21.15.89:44776) 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f650 0x7fdffc1c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32926/0 (socket says 172.21.15.89:32926) 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffa59c700 1 -- 172.21.15.89:0/2107907709 learned_addr learned my addr 172.21.15.89:0/2107907709 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdff9d9b700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdffc09f240 0x7fdffc1c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffad9d700 1 -- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 msgr2=0x7fdffc1c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffad9d700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc1c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffad9d700 1 -- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdffc09f240 msgr2=0x7fdffc1c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffad9d700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdffc09f240 0x7fdffc1c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffad9d700 1 -- 172.21.15.89:0/2107907709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00094a0 con 0x7fdffc09f650 2023-09-30T22:17:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.552+0000 7fdffa59c700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc1c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.553+0000 7fdffad9d700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f650 0x7fdffc1c8aa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdff400ca80 tx=0x7fdff400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.553+0000 7fdfeaffd700 1 -- 172.21.15.89:0/2107907709 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400a9e0 con 0x7fdffc09f650 2023-09-30T22:17:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.553+0000 7fdfeaffd700 1 -- 172.21.15.89:0/2107907709 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff400ab40 con 0x7fdffc09f650 2023-09-30T22:17:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.553+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc0a0190 con 0x7fdffc09f650 2023-09-30T22:17:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.553+0000 7fdfeaffd700 1 -- 172.21.15.89:0/2107907709 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4012400 con 0x7fdffc09f650 2023-09-30T22:17:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.553+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc137b80 con 0x7fdffc09f650 2023-09-30T22:17:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.554+0000 7fdfeaffd700 1 -- 172.21.15.89:0/2107907709 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdff401b3b0 con 0x7fdffc09f650 2023-09-30T22:17:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.554+0000 7fdfeaffd700 1 -- 172.21.15.89:0/2107907709 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdff403f320 con 0x7fdffc09f650 2023-09-30T22:17:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.555+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdffc074ea0 con 0x7fdffc09f650 2023-09-30T22:17:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.560+0000 7fdfeaffd700 1 -- 172.21.15.89:0/2107907709 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdff4017080 con 0x7fdffc09f650 2023-09-30T22:17:25.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.594+0000 7f15918eb700 1 Processor -- start 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.594+0000 7f15918eb700 1 -- start start 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f15918eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f158c131810 0x7f158c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f15918eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c099a40 0x7f158c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f15918eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c09a3b0 0x7f158c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f15918eb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c1313c0 con 0x7f158c131810 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f15918eb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c06e8c0 con 0x7f158c099a40 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f15918eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f158c071d20 con 0x7f158c09a3b0 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c099a40 0x7f158c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f158c131810 0x7f158c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f158c131810 0x7f158c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32940/0 (socket says 172.21.15.89:32940) 2023-09-30T22:17:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c099a40 0x7f158c099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43332/0 (socket says 172.21.15.89:43332) 2023-09-30T22:17:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158affd700 1 -- 172.21.15.89:0/935108900 learned_addr learned my addr 172.21.15.89:0/935108900 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158b7fe700 1 --2- 172.21.15.89:0/935108900 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c09a3b0 0x7f158c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158a7fc700 1 -- 172.21.15.89:0/935108900 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c09a3b0 msgr2=0x7f158c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158a7fc700 1 --2- 172.21.15.89:0/935108900 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c09a3b0 0x7f158c138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158a7fc700 1 -- 172.21.15.89:0/935108900 >> v2:172.21.15.89:3300/0 conn(0x7f158c131810 msgr2=0x7f158c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158a7fc700 1 --2- 172.21.15.89:0/935108900 >> v2:172.21.15.89:3300/0 conn(0x7f158c131810 0x7f158c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158a7fc700 1 -- 172.21.15.89:0/935108900 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0721a0 con 0x7f158c099a40 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158affd700 1 --2- 172.21.15.89:0/935108900 >> v2:172.21.15.89:3300/0 conn(0x7f158c131810 0x7f158c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.595+0000 7f158a7fc700 1 --2- 172.21.15.89:0/935108900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c099a40 0x7f158c099e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f158000dc60 tx=0x7f158000df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61775a233fa1a356 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f1589ffb700 1 -- 172.21.15.89:0/935108900 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580012070 con 0x7f158c099a40 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f1589ffb700 1 -- 172.21.15.89:0/935108900 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158000d2e0 con 0x7f158c099a40 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f1589ffb700 1 -- 172.21.15.89:0/935108900 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158000f900 con 0x7f158c099a40 2023-09-30T22:17:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 -- 172.21.15.89:0/935108900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c099a40 msgr2=0x7f158c099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 --2- 172.21.15.89:0/935108900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c099a40 0x7f158c099e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f158000dc60 tx=0x7f158000df70 comp rx=0 tx=0).stop 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 -- 172.21.15.89:0/935108900 shutdown_connections 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 --2- 172.21.15.89:0/935108900 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c09a3b0 0x7f158c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 --2- 172.21.15.89:0/935108900 >> v2:172.21.15.89:3300/0 conn(0x7f158c131810 0x7f158c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 --2- 172.21.15.89:0/935108900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c099a40 0x7f158c099e70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 -- 172.21.15.89:0/935108900 >> 172.21.15.89:0/935108900 conn(0x7f158c0a2aa0 msgr2=0x7f158c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 -- 172.21.15.89:0/935108900 shutdown_connections 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.596+0000 7f15918eb700 1 -- 172.21.15.89:0/935108900 wait complete. 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.597+0000 7f15918eb700 1 Processor -- start 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.597+0000 7f15918eb700 1 -- start start 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f15918eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c099a40 0x7f158c119640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f15918eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c09a3b0 0x7f158c119b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f15918eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f158c131810 0x7f158c11a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f15918eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c11c850 con 0x7f158c131810 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f15918eb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c06eb00 con 0x7f158c09a3b0 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f15918eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f158c1308e0 con 0x7f158c099a40 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f158c131810 0x7f158c11a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f158c131810 0x7f158c11a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32954/0 (socket says 172.21.15.89:32954) 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158b7fe700 1 -- 172.21.15.89:0/345170340 learned_addr learned my addr 172.21.15.89:0/345170340 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158affd700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c099a40 0x7f158c119640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158a7fc700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c09a3b0 0x7f158c119b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158b7fe700 1 -- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c099a40 msgr2=0x7f158c119640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158b7fe700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c099a40 0x7f158c119640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158b7fe700 1 -- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c09a3b0 msgr2=0x7f158c119b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158b7fe700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c09a3b0 0x7f158c119b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.598+0000 7f158b7fe700 1 -- 172.21.15.89:0/345170340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c009550 con 0x7f158c131810 2023-09-30T22:17:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.599+0000 7f158b7fe700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f158c131810 0x7f158c11a0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1584007e40 tx=0x7f158400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.599+0000 7f1573fff700 1 -- 172.21.15.89:0/345170340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400ed30 con 0x7f158c131810 2023-09-30T22:17:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.599+0000 7f1573fff700 1 -- 172.21.15.89:0/345170340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158400ee90 con 0x7f158c131810 2023-09-30T22:17:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.599+0000 7f1573fff700 1 -- 172.21.15.89:0/345170340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584014660 con 0x7f158c131810 2023-09-30T22:17:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.599+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158000d940 con 0x7f158c131810 2023-09-30T22:17:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.599+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c11c400 con 0x7f158c131810 2023-09-30T22:17:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.600+0000 7f1573fff700 1 -- 172.21.15.89:0/345170340 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f15840147c0 con 0x7f158c131810 2023-09-30T22:17:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.600+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c11cc60 con 0x7f158c131810 2023-09-30T22:17:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.600+0000 7f1573fff700 1 -- 172.21.15.89:0/345170340 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f158403fd40 con 0x7f158c131810 2023-09-30T22:17:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.605+0000 7f1573fff700 1 -- 172.21.15.89:0/345170340 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f158401bce0 con 0x7f158c131810 2023-09-30T22:17:25.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.622+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7effa8074ea0 con 0x7effa809e000 2023-09-30T22:17:25.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.623+0000 7eff8ffff700 1 -- 172.21.15.89:0/1522492431 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff9c02b020 con 0x7effa809e000 2023-09-30T22:17:25.623 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:25.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 msgr2=0x7effa81c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81c8510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff9c013000 tx=0x7eff9c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:25.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 shutdown_connections 2023-09-30T22:17:25.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7effa809e410 0x7effa81c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7effa809d6f0 0x7effa81c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 --2- 172.21.15.89:0/1522492431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effa809e000 0x7effa81c8510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 >> 172.21.15.89:0/1522492431 conn(0x7effa8066f50 msgr2=0x7effa806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:25.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 shutdown_connections 2023-09-30T22:17:25.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.626+0000 7effae0ea700 1 -- 172.21.15.89:0/1522492431 wait complete. 2023-09-30T22:17:25.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:25.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.733+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f82c4074ea0 con 0x7f82c413c1e0 2023-09-30T22:17:25.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.733+0000 7f82abfff700 1 -- 172.21.15.89:0/152583093 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82bc029090 con 0x7f82c413c1e0 2023-09-30T22:17:25.734 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.735+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82c413c1e0 msgr2=0x7f82c41cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.735+0000 7f82c9d51700 1 --2- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82c413c1e0 0x7f82c41cced0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00cbc0 tx=0x7f82bc00ced0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.735+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 shutdown_connections 2023-09-30T22:17:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.735+0000 7f82c9d51700 1 --2- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82c413c1e0 0x7f82c41cced0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.735+0000 7f82c9d51700 1 --2- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82c406a770 0x7f82c41cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.735+0000 7f82c9d51700 1 --2- 172.21.15.89:0/152583093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c409dc30 0x7f82c41cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.735+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 >> 172.21.15.89:0/152583093 conn(0x7f82c4066f50 msgr2=0x7f82c4070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.735+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 shutdown_connections 2023-09-30T22:17:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.736+0000 7f82c9d51700 1 -- 172.21.15.89:0/152583093 wait complete. 2023-09-30T22:17:25.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:25.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.767+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdffc138180 con 0x7fdffc09f650 2023-09-30T22:17:25.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.767+0000 7fdfeaffd700 1 -- 172.21.15.89:0/2107907709 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdff40192c0 con 0x7fdffc09f650 2023-09-30T22:17:25.768 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:25.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f650 msgr2=0x7fdffc1c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f650 0x7fdffc1c8aa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdff400ca80 tx=0x7fdff400cd90 comp rx=0 tx=0).stop 2023-09-30T22:17:25.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 shutdown_connections 2023-09-30T22:17:25.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdffc09f650 0x7fdffc1c8aa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdffc09e930 0x7fdffc1c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 --2- 172.21.15.89:0/2107907709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdffc09f240 0x7fdffc1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 >> 172.21.15.89:0/2107907709 conn(0x7fdffc0a4210 msgr2=0x7fdffc06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 shutdown_connections 2023-09-30T22:17:25.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.769+0000 7fe000aab700 1 -- 172.21.15.89:0/2107907709 wait complete. 2023-09-30T22:17:25.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.800+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f158c074ea0 con 0x7f158c131810 2023-09-30T22:17:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.800+0000 7f1573fff700 1 -- 172.21.15.89:0/345170340 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f158401bb00 con 0x7f158c131810 2023-09-30T22:17:25.801 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.802+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f158c131810 msgr2=0x7f158c11a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.802+0000 7f15918eb700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f158c131810 0x7f158c11a0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1584007e40 tx=0x7f158400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.802+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 shutdown_connections 2023-09-30T22:17:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.802+0000 7f15918eb700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f158c131810 0x7f158c11a0c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.802+0000 7f15918eb700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f158c099a40 0x7f158c119640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.802+0000 7f15918eb700 1 --2- 172.21.15.89:0/345170340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f158c09a3b0 0x7f158c119b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.802+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 >> 172.21.15.89:0/345170340 conn(0x7f158c0a2aa0 msgr2=0x7f158c12e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.802+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 shutdown_connections 2023-09-30T22:17:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:25.803+0000 7f15918eb700 1 -- 172.21.15.89:0/345170340 wait complete. 2023-09-30T22:17:25.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:26.640 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:26.749 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.775+0000 7f2f549c2700 1 Processor -- start 2023-09-30T22:17:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f549c2700 1 -- start start 2023-09-30T22:17:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f549c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f549c2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f5009e930 0x7f2f5013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f549c2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5006ef30 con 0x7f2f5009de50 2023-09-30T22:17:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f549c2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5006fef0 con 0x7f2f5009e930 2023-09-30T22:17:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f549c2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f501334b0 con 0x7f2f501338a0 2023-09-30T22:17:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f4ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f5009e930 0x7f2f5013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f4e59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f4e59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43344/0 (socket says 172.21.15.89:43344) 2023-09-30T22:17:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f4ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f5009e930 0x7f2f5013b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32956/0 (socket says 172.21.15.89:32956) 2023-09-30T22:17:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.776+0000 7f2f4e59c700 1 -- 172.21.15.89:0/3525779696 learned_addr learned my addr 172.21.15.89:0/3525779696 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4e59c700 1 -- 172.21.15.89:0/3525779696 >> v2:172.21.15.89:3300/0 conn(0x7f2f5009e930 msgr2=0x7f2f5013b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4e59c700 1 --2- 172.21.15.89:0/3525779696 >> v2:172.21.15.89:3300/0 conn(0x7f2f5009e930 0x7f2f5013b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4e59c700 1 -- 172.21.15.89:0/3525779696 >> v1:172.21.15.89:6789/0 conn(0x7f2f5009de50 legacy=0x7f2f5009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4e59c700 1 -- 172.21.15.89:0/3525779696 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5006fc30 con 0x7f2f501338a0 2023-09-30T22:17:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4ed9d700 1 --2- 172.21.15.89:0/3525779696 >> v2:172.21.15.89:3300/0 conn(0x7f2f5009e930 0x7f2f5013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4e59c700 1 --2- 172.21.15.89:0/3525779696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50133c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f400098f0 tx=0x7f2f40009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d639326d320d49 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4d59a700 1 -- 172.21.15.89:0/3525779696 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40011830 con 0x7f2f501338a0 2023-09-30T22:17:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4d59a700 1 -- 172.21.15.89:0/3525779696 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f40011990 con 0x7f2f501338a0 2023-09-30T22:17:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.777+0000 7f2f4d59a700 1 -- 172.21.15.89:0/3525779696 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40011d20 con 0x7f2f501338a0 2023-09-30T22:17:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.778+0000 7f2f549c2700 1 -- 172.21.15.89:0/3525779696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 msgr2=0x7f2f50133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.778+0000 7f2f549c2700 1 --2- 172.21.15.89:0/3525779696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50133c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f400098f0 tx=0x7f2f40009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.778+0000 7f2f549c2700 1 -- 172.21.15.89:0/3525779696 shutdown_connections 2023-09-30T22:17:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.778+0000 7f2f549c2700 1 --2- 172.21.15.89:0/3525779696 >> v2:172.21.15.89:3300/0 conn(0x7f2f5009e930 0x7f2f5013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.778+0000 7f2f549c2700 1 --2- 172.21.15.89:0/3525779696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50133c70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.778+0000 7f2f549c2700 1 -- 172.21.15.89:0/3525779696 >> 172.21.15.89:0/3525779696 conn(0x7f2f500a2aa0 msgr2=0x7f2f500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.778+0000 7f2f549c2700 1 -- 172.21.15.89:0/3525779696 shutdown_connections 2023-09-30T22:17:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.778+0000 7f2f549c2700 1 -- 172.21.15.89:0/3525779696 wait complete. 2023-09-30T22:17:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.779+0000 7f2f549c2700 1 Processor -- start 2023-09-30T22:17:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.779+0000 7f2f549c2700 1 -- start start 2023-09-30T22:17:26.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f549c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f5009de50 0x7f2f501ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f549c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f5009e930 0x7f2f501cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f549c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f549c2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5009ccc0 con 0x7f2f5009e930 2023-09-30T22:17:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f549c2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5009d1d0 con 0x7f2f501338a0 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f549c2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5009dc90 con 0x7f2f5009de50 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f5009e930 0x7f2f501cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f5009de50 0x7f2f501ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f5009de50 0x7f2f501ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44810/0 (socket says 172.21.15.89:44810) 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f5009e930 0x7f2f501cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32968/0 (socket says 172.21.15.89:32968) 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 -- 172.21.15.89:0/4118809942 learned_addr learned my addr 172.21.15.89:0/4118809942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4ed9d700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 -- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 msgr2=0x7f2f50099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 -- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f5009e930 msgr2=0x7f2f501cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f5009e930 0x7f2f501cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 -- 172.21.15.89:0/4118809942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f48009a70 con 0x7f2f5009de50 2023-09-30T22:17:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4ed9d700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4dd9b700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f5009e930 0x7f2f501cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.780+0000 7f2f4e59c700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f5009de50 0x7f2f501ccb80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009c40 tx=0x7f2f40004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.781+0000 7f2f3effd700 1 -- 172.21.15.89:0/4118809942 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400043c0 con 0x7f2f5009de50 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.781+0000 7f2f3effd700 1 -- 172.21.15.89:0/4118809942 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4001f070 con 0x7f2f5009de50 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.781+0000 7f2f3effd700 1 -- 172.21.15.89:0/4118809942 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400036a0 con 0x7f2f5009de50 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.781+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f40009550 con 0x7f2f5009de50 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.781+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5009bd90 con 0x7f2f5009de50 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.781+0000 7f2f3effd700 1 -- 172.21.15.89:0/4118809942 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f40021030 con 0x7f2f5009de50 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.782+0000 7f2f3effd700 1 -- 172.21.15.89:0/4118809942 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2f40040aa0 con 0x7f2f5009de50 2023-09-30T22:17:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.782+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5009d310 con 0x7f2f5009de50 2023-09-30T22:17:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.787+0000 7f2f3effd700 1 -- 172.21.15.89:0/4118809942 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f5009d310 con 0x7f2f5009de50 2023-09-30T22:17:26.816 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:26.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.885+0000 7fd953577700 1 Processor -- start 2023-09-30T22:17:26.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.885+0000 7fd953577700 1 -- start start 2023-09-30T22:17:26.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd953577700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c1338a0 0x7fd94c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd953577700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd953577700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd94c09e760 0x7fd94c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd953577700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd94c06fef0 con 0x7fd94c09e760 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd953577700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c06ef30 con 0x7fd94c09de50 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd953577700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c1334b0 con 0x7fd94c1338a0 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd951313700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c1338a0 0x7fd94c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd951313700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c1338a0 0x7fd94c133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44818/0 (socket says 172.21.15.89:44818) 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd950b12700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd951313700 1 -- 172.21.15.89:0/246996514 learned_addr learned my addr 172.21.15.89:0/246996514 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd951b14700 1 --2- 172.21.15.89:0/246996514 >> v2:172.21.15.89:3300/0 conn(0x7fd94c09e760 0x7fd94c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd950b12700 1 -- 172.21.15.89:0/246996514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c1338a0 msgr2=0x7fd94c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd950b12700 1 --2- 172.21.15.89:0/246996514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c1338a0 0x7fd94c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd950b12700 1 -- 172.21.15.89:0/246996514 >> v2:172.21.15.89:3300/0 conn(0x7fd94c09e760 msgr2=0x7fd94c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd950b12700 1 --2- 172.21.15.89:0/246996514 >> v2:172.21.15.89:3300/0 conn(0x7fd94c09e760 0x7fd94c13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.886+0000 7fd950b12700 1 -- 172.21.15.89:0/246996514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c06fc30 con 0x7fd94c09de50 2023-09-30T22:17:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.887+0000 7fd950b12700 1 --2- 172.21.15.89:0/246996514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c09e220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009870 tx=0x7fd93c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345d54f0e57299ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd943fff700 1 -- 172.21.15.89:0/246996514 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c011830 con 0x7fd94c09de50 2023-09-30T22:17:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd943fff700 1 -- 172.21.15.89:0/246996514 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c011990 con 0x7fd94c09de50 2023-09-30T22:17:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd943fff700 1 -- 172.21.15.89:0/246996514 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c011d20 con 0x7fd94c09de50 2023-09-30T22:17:26.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 -- 172.21.15.89:0/246996514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 msgr2=0x7fd94c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 --2- 172.21.15.89:0/246996514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c09e220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009870 tx=0x7fd93c009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:26.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 -- 172.21.15.89:0/246996514 shutdown_connections 2023-09-30T22:17:26.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 --2- 172.21.15.89:0/246996514 >> v2:172.21.15.89:3300/0 conn(0x7fd94c09e760 0x7fd94c13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 --2- 172.21.15.89:0/246996514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c1338a0 0x7fd94c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 --2- 172.21.15.89:0/246996514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c09e220 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 -- 172.21.15.89:0/246996514 >> 172.21.15.89:0/246996514 conn(0x7fd94c0a2aa0 msgr2=0x7fd94c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:26.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 -- 172.21.15.89:0/246996514 shutdown_connections 2023-09-30T22:17:26.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.889+0000 7fd953577700 1 -- 172.21.15.89:0/246996514 wait complete. 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.890+0000 7fd953577700 1 Processor -- start 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.890+0000 7fd953577700 1 -- start start 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd953577700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c11d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd953577700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c09e760 0x7fd94c11dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd953577700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd94c1338a0 0x7fd94c11e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd953577700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c11eb20 con 0x7fd94c1338a0 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd953577700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c11f2a0 con 0x7fd94c09de50 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd953577700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c11f8c0 con 0x7fd94c09e760 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951b14700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd94c1338a0 0x7fd94c11e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951b14700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd94c1338a0 0x7fd94c11e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32986/0 (socket says 172.21.15.89:32986) 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951313700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c11d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951b14700 1 -- 172.21.15.89:0/4264831270 learned_addr learned my addr 172.21.15.89:0/4264831270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd950b12700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c09e760 0x7fd94c11dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951313700 1 -- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c09e760 msgr2=0x7fd94c11dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951313700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c09e760 0x7fd94c11dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951313700 1 -- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd94c1338a0 msgr2=0x7fd94c11e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951313700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd94c1338a0 0x7fd94c11e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951313700 1 -- 172.21.15.89:0/4264831270 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c0094a0 con 0x7fd94c09de50 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd950b12700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c09e760 0x7fd94c11dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.891+0000 7fd951313700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c11d690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd94800e940 tx=0x7fd94800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.892+0000 7fd941ffb700 1 -- 172.21.15.89:0/4264831270 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94800ee90 con 0x7fd94c09de50 2023-09-30T22:17:26.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.892+0000 7fd941ffb700 1 -- 172.21.15.89:0/4264831270 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94800c850 con 0x7fd94c09de50 2023-09-30T22:17:26.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.892+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c120ed0 con 0x7fd94c09de50 2023-09-30T22:17:26.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.892+0000 7fd941ffb700 1 -- 172.21.15.89:0/4264831270 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94800cbe0 con 0x7fd94c09de50 2023-09-30T22:17:26.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.892+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c06f640 con 0x7fd94c09de50 2023-09-30T22:17:26.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.893+0000 7fd941ffb700 1 -- 172.21.15.89:0/4264831270 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd94800cd40 con 0x7fd94c09de50 2023-09-30T22:17:26.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.893+0000 7fd941ffb700 1 -- 172.21.15.89:0/4264831270 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd94803f680 con 0x7fd94c09de50 2023-09-30T22:17:26.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.893+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c11a470 con 0x7fd94c09de50 2023-09-30T22:17:26.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.902+0000 7fd941ffb700 1 -- 172.21.15.89:0/4264831270 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9480232f0 con 0x7fd94c09de50 2023-09-30T22:17:26.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.921+0000 7f068021b700 1 Processor -- start 2023-09-30T22:17:26.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.921+0000 7f068021b700 1 -- start start 2023-09-30T22:17:26.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f068021b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678130540 0x7f0678130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f068021b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f067806fcf0 0x7f0678138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f068021b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780716a0 con 0x7f0678130e50 2023-09-30T22:17:26.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f068021b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067809a390 con 0x7f067806fcf0 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f068021b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0678071290 con 0x7f0678130540 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067d7b6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0678130e50 0x7f067806f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49934/0 (socket says 172.21.15.89:49934) 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067d7b6700 1 -- 172.21.15.89:0/2173739656 learned_addr learned my addr 172.21.15.89:0/2173739656 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067dfb7700 1 --2- 172.21.15.89:0/2173739656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678130540 0x7f0678130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067e7b8700 1 --2- 172.21.15.89:0/2173739656 >> v2:172.21.15.89:3300/0 conn(0x7f067806fcf0 0x7f0678138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067dfb7700 1 -- 172.21.15.89:0/2173739656 >> v2:172.21.15.89:3300/0 conn(0x7f067806fcf0 msgr2=0x7f0678138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067dfb7700 1 --2- 172.21.15.89:0/2173739656 >> v2:172.21.15.89:3300/0 conn(0x7f067806fcf0 0x7f0678138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067dfb7700 1 -- 172.21.15.89:0/2173739656 >> v1:172.21.15.89:6789/0 conn(0x7f0678130e50 legacy=0x7f067806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067dfb7700 1 -- 172.21.15.89:0/2173739656 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678130340 con 0x7f0678130540 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.922+0000 7f067dfb7700 1 --2- 172.21.15.89:0/2173739656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678130540 0x7f0678130910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f067400bb10 tx=0x7f067400be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d4ec8358960fcc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:26.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f067cfb5700 1 -- 172.21.15.89:0/2173739656 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067400dcf0 con 0x7f0678130540 2023-09-30T22:17:26.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f067cfb5700 1 -- 172.21.15.89:0/2173739656 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067400de50 con 0x7f0678130540 2023-09-30T22:17:26.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f067cfb5700 1 -- 172.21.15.89:0/2173739656 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674010690 con 0x7f0678130540 2023-09-30T22:17:26.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f068021b700 1 -- 172.21.15.89:0/2173739656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678130540 msgr2=0x7f0678130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f068021b700 1 --2- 172.21.15.89:0/2173739656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678130540 0x7f0678130910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f067400bb10 tx=0x7f067400be20 comp rx=0 tx=0).stop 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f068021b700 1 -- 172.21.15.89:0/2173739656 shutdown_connections 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f068021b700 1 --2- 172.21.15.89:0/2173739656 >> v2:172.21.15.89:3300/0 conn(0x7f067806fcf0 0x7f0678138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f068021b700 1 --2- 172.21.15.89:0/2173739656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678130540 0x7f0678130910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.923+0000 7f068021b700 1 -- 172.21.15.89:0/2173739656 >> 172.21.15.89:0/2173739656 conn(0x7f06780a2aa0 msgr2=0x7f06780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.924+0000 7f068021b700 1 -- 172.21.15.89:0/2173739656 shutdown_connections 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.924+0000 7f068021b700 1 -- 172.21.15.89:0/2173739656 wait complete. 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.925+0000 7f068021b700 1 Processor -- start 2023-09-30T22:17:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.925+0000 7f068021b700 1 -- start start 2023-09-30T22:17:26.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f068021b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f067806fcf0 0x7f067811a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f068021b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678130540 0x7f067811ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f068021b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678130e50 0x7f067811b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f068021b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067811e0d0 con 0x7f0678130540 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f068021b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067811c8d0 con 0x7f067806fcf0 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f068021b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f067811e320 con 0x7f0678130e50 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067dfb7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f067806fcf0 0x7f067811a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067dfb7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f067806fcf0 0x7f067811a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43402/0 (socket says 172.21.15.89:43402) 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067dfb7700 1 -- 172.21.15.89:0/3401501297 learned_addr learned my addr 172.21.15.89:0/3401501297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067e7b8700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678130e50 0x7f067811b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067d7b6700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678130540 0x7f067811ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067e7b8700 1 -- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f067806fcf0 msgr2=0x7f067811a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067e7b8700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f067806fcf0 0x7f067811a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067e7b8700 1 -- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678130540 msgr2=0x7f067811ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067e7b8700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678130540 0x7f067811ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.926+0000 7f067e7b8700 1 -- 172.21.15.89:0/3401501297 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067400b740 con 0x7f0678130e50 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.927+0000 7f067e7b8700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678130e50 0x7f067811b350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0670007dc0 tx=0x7f067000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.927+0000 7f066e7fc700 1 -- 172.21.15.89:0/3401501297 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670018070 con 0x7f0678130e50 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.927+0000 7f066e7fc700 1 -- 172.21.15.89:0/3401501297 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067000ebe0 con 0x7f0678130e50 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.927+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067811d210 con 0x7f0678130e50 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.927+0000 7f066e7fc700 1 -- 172.21.15.89:0/3401501297 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670003a20 con 0x7f0678130e50 2023-09-30T22:17:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.927+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067812dcb0 con 0x7f0678130e50 2023-09-30T22:17:26.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.928+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0678074ea0 con 0x7f0678130e50 2023-09-30T22:17:26.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.929+0000 7f066e7fc700 1 -- 172.21.15.89:0/3401501297 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0670003b80 con 0x7f0678130e50 2023-09-30T22:17:26.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.929+0000 7f066e7fc700 1 -- 172.21.15.89:0/3401501297 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0670040230 con 0x7f0678130e50 2023-09-30T22:17:26.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.937+0000 7f066e7fc700 1 -- 172.21.15.89:0/3401501297 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0670009950 con 0x7f0678130e50 2023-09-30T22:17:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.971+0000 7f23cb71c700 1 Processor -- start 2023-09-30T22:17:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.972+0000 7f23cb71c700 1 -- start start 2023-09-30T22:17:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.972+0000 7f23cb71c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23c409e930 0x7f23c409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.972+0000 7f23cb71c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409f650 0x7f23c4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.972+0000 7f23cb71c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c4070940 con 0x7f23c409f240 2023-09-30T22:17:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.972+0000 7f23cb71c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c406fd60 con 0x7f23c409e930 2023-09-30T22:17:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.972+0000 7f23cb71c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c406fa80 con 0x7f23c409f650 2023-09-30T22:17:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.972+0000 7f23c94b8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23c409e930 0x7f23c409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23c94b8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23c409e930 0x7f23c409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33008/0 (socket says 172.21.15.89:33008) 2023-09-30T22:17:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23c94b8700 1 -- 172.21.15.89:0/4191208421 learned_addr learned my addr 172.21.15.89:0/4191208421 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23c9cb9700 1 --2- 172.21.15.89:0/4191208421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409f650 0x7f23c4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23c94b8700 1 -- 172.21.15.89:0/4191208421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409f650 msgr2=0x7f23c4131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23c94b8700 1 --2- 172.21.15.89:0/4191208421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409f650 0x7f23c4131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23c94b8700 1 -- 172.21.15.89:0/4191208421 >> v1:172.21.15.89:6789/0 conn(0x7f23c409f240 legacy=0x7f23c41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23c94b8700 1 -- 172.21.15.89:0/4191208421 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c409fb60 con 0x7f23c409e930 2023-09-30T22:17:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23c94b8700 1 --2- 172.21.15.89:0/4191208421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409e930 0x7f23c409ed00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f23c00098f0 tx=0x7f23c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6121ae44cfeb192d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23bbfff700 1 -- 172.21.15.89:0/4191208421 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0011830 con 0x7f23c409e930 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23bbfff700 1 -- 172.21.15.89:0/4191208421 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0011990 con 0x7f23c409e930 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.973+0000 7f23bbfff700 1 -- 172.21.15.89:0/4191208421 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0011d20 con 0x7f23c409e930 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.974+0000 7f23cb71c700 1 -- 172.21.15.89:0/4191208421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409e930 msgr2=0x7f23c409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.974+0000 7f23cb71c700 1 --2- 172.21.15.89:0/4191208421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409e930 0x7f23c409ed00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f23c00098f0 tx=0x7f23c0009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.974+0000 7f23cb71c700 1 -- 172.21.15.89:0/4191208421 shutdown_connections 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.974+0000 7f23cb71c700 1 --2- 172.21.15.89:0/4191208421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409f650 0x7f23c4131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.974+0000 7f23cb71c700 1 --2- 172.21.15.89:0/4191208421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409e930 0x7f23c409ed00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.974+0000 7f23cb71c700 1 -- 172.21.15.89:0/4191208421 >> 172.21.15.89:0/4191208421 conn(0x7f23c40a4210 msgr2=0x7f23c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.974+0000 7f23cb71c700 1 -- 172.21.15.89:0/4191208421 shutdown_connections 2023-09-30T22:17:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.974+0000 7f23cb71c700 1 -- 172.21.15.89:0/4191208421 wait complete. 2023-09-30T22:17:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.975+0000 7f23cb71c700 1 Processor -- start 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.976+0000 7f23cb71c700 1 -- start start 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.976+0000 7f23cb71c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409e930 0x7f23c41a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.976+0000 7f23cb71c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409f240 0x7f23c41a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.976+0000 7f23cb71c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23c409f650 0x7f23c41a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.976+0000 7f23cb71c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c41a7040 con 0x7f23c409f240 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.976+0000 7f23cb71c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4071d20 con 0x7f23c409e930 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.976+0000 7f23cb71c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40a34c0 con 0x7f23c409f650 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c9cb9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23c409f650 0x7f23c41a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c9cb9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23c409f650 0x7f23c41a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44844/0 (socket says 172.21.15.89:44844) 2023-09-30T22:17:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c9cb9700 1 -- 172.21.15.89:0/30816987 learned_addr learned my addr 172.21.15.89:0/30816987 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c94b8700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409e930 0x7f23c41a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c8cb7700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409f240 0x7f23c41a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c8cb7700 1 -- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23c409f650 msgr2=0x7f23c41a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c8cb7700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23c409f650 0x7f23c41a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c8cb7700 1 -- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409e930 msgr2=0x7f23c41a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c8cb7700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409e930 0x7f23c41a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c8cb7700 1 -- 172.21.15.89:0/30816987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c0009550 con 0x7f23c409f240 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c9cb9700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23c409f650 0x7f23c41a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c94b8700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409e930 0x7f23c41a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23c8cb7700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409f240 0x7f23c41a42d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000f60 tx=0x7f23b400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23b9ffb700 1 -- 172.21.15.89:0/30816987 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b40052d0 con 0x7f23c409f240 2023-09-30T22:17:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23b9ffb700 1 -- 172.21.15.89:0/30816987 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b4005430 con 0x7f23c409f240 2023-09-30T22:17:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23b9ffb700 1 -- 172.21.15.89:0/30816987 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b40038c0 con 0x7f23c409f240 2023-09-30T22:17:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.977+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c40a0190 con 0x7f23c409f240 2023-09-30T22:17:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.978+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c41a6bf0 con 0x7f23c409f240 2023-09-30T22:17:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.978+0000 7f23b9ffb700 1 -- 172.21.15.89:0/30816987 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f23b4003a20 con 0x7f23c409f240 2023-09-30T22:17:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.979+0000 7f23b9ffb700 1 -- 172.21.15.89:0/30816987 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f23b4040a80 con 0x7f23c409f240 2023-09-30T22:17:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.979+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c41a7450 con 0x7f23c409f240 2023-09-30T22:17:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.984+0000 7f23b9ffb700 1 -- 172.21.15.89:0/30816987 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23b4017070 con 0x7f23c409f240 2023-09-30T22:17:26.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.991+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2f5009d310 con 0x7f2f5009de50 2023-09-30T22:17:26.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.991+0000 7f2f3effd700 1 -- 172.21.15.89:0/4118809942 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f5009d310 con 0x7f2f5009de50 2023-09-30T22:17:26.994 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.993+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f5009de50 msgr2=0x7f2f501ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.993+0000 7f2f549c2700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f5009de50 0x7f2f501ccb80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009c40 tx=0x7f2f40004750 comp rx=0 tx=0).stop 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.994+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 shutdown_connections 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.994+0000 7f2f549c2700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f501338a0 0x7f2f50099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.994+0000 7f2f549c2700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f5009de50 0x7f2f501ccb80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.994+0000 7f2f549c2700 1 --2- 172.21.15.89:0/4118809942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f5009e930 0x7f2f501cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.994+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 >> 172.21.15.89:0/4118809942 conn(0x7f2f500a2aa0 msgr2=0x7f2f501badd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.994+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 shutdown_connections 2023-09-30T22:17:26.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:26.994+0000 7f2f549c2700 1 -- 172.21.15.89:0/4118809942 wait complete. 2023-09-30T22:17:27.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:27.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.123+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd94c074ea0 con 0x7fd94c09de50 2023-09-30T22:17:27.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.124+0000 7fd941ffb700 1 -- 172.21.15.89:0/4264831270 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd948017020 con 0x7fd94c09de50 2023-09-30T22:17:27.124 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:27.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 msgr2=0x7fd94c11d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c11d690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd94800e940 tx=0x7fd94800ec50 comp rx=0 tx=0).stop 2023-09-30T22:17:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 shutdown_connections 2023-09-30T22:17:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd94c1338a0 0x7fd94c11e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd94c09de50 0x7fd94c11d690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 --2- 172.21.15.89:0/4264831270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd94c09e760 0x7fd94c11dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 >> 172.21.15.89:0/4264831270 conn(0x7fd94c0a2aa0 msgr2=0x7fd94c1baea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 shutdown_connections 2023-09-30T22:17:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.126+0000 7fd953577700 1 -- 172.21.15.89:0/4264831270 wait complete. 2023-09-30T22:17:27.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.131+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f067811dce0 con 0x7f0678130e50 2023-09-30T22:17:27.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.131+0000 7f066e7fc700 1 -- 172.21.15.89:0/3401501297 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f067001d410 con 0x7f0678130e50 2023-09-30T22:17:27.133 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:27.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678130e50 msgr2=0x7f067811b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:27.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678130e50 0x7f067811b350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0670007dc0 tx=0x7f067000e3c0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 shutdown_connections 2023-09-30T22:17:27.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678130e50 0x7f067811b350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f067806fcf0 0x7f067811a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 --2- 172.21.15.89:0/3401501297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678130540 0x7f067811ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 >> 172.21.15.89:0/3401501297 conn(0x7f06780a2aa0 msgr2=0x7f067812f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:27.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 shutdown_connections 2023-09-30T22:17:27.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.134+0000 7f068021b700 1 -- 172.21.15.89:0/3401501297 wait complete. 2023-09-30T22:17:27.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:27.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.189+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f23c4074ea0 con 0x7f23c409f240 2023-09-30T22:17:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.189+0000 7f23b9ffb700 1 -- 172.21.15.89:0/30816987 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23b4017020 con 0x7f23c409f240 2023-09-30T22:17:27.190 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.191+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409f240 msgr2=0x7f23c41a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.191+0000 7f23cb71c700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409f240 0x7f23c41a42d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000f60 tx=0x7f23b400c730 comp rx=0 tx=0).stop 2023-09-30T22:17:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.192+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 shutdown_connections 2023-09-30T22:17:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.192+0000 7f23cb71c700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23c409f650 0x7f23c41a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.192+0000 7f23cb71c700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23c409e930 0x7f23c41a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.192+0000 7f23cb71c700 1 --2- 172.21.15.89:0/30816987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c409f240 0x7f23c41a42d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.192+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 >> 172.21.15.89:0/30816987 conn(0x7f23c40a4210 msgr2=0x7f23c41b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.192+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 shutdown_connections 2023-09-30T22:17:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:27.192+0000 7f23cb71c700 1 -- 172.21.15.89:0/30816987 wait complete. 2023-09-30T22:17:27.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:28.007 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdfff528700 1 Processor -- start 2023-09-30T22:17:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdfff528700 1 -- start start 2023-09-30T22:17:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdfff528700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdff809d6f0 0x7fdff809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdfff528700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e000 0x7fdff8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdfff528700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff809f130 con 0x7fdff809e410 2023-09-30T22:17:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdfff528700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff809f520 con 0x7fdff809d6f0 2023-09-30T22:17:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdfff528700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff809e930 con 0x7fdff809e000 2023-09-30T22:17:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdffd2c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdff809d6f0 0x7fdff809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdffcac3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e000 0x7fdff8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdffcac3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e000 0x7fdff8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43436/0 (socket says 172.21.15.89:43436) 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdffd2c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdff809d6f0 0x7fdff809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33032/0 (socket says 172.21.15.89:33032) 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdffcac3700 1 -- 172.21.15.89:0/3242854039 learned_addr learned my addr 172.21.15.89:0/3242854039 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.131+0000 7fdffdac5700 1 --1- 172.21.15.89:0/3242854039 >> v1:172.21.15.89:6789/0 conn(0x7fdff809e410 0x7fdff8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49962/0 (socket says 172.21.15.89:49962) 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdffd2c4700 1 -- 172.21.15.89:0/3242854039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e000 msgr2=0x7fdff8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdffd2c4700 1 --2- 172.21.15.89:0/3242854039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e000 0x7fdff8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdffd2c4700 1 -- 172.21.15.89:0/3242854039 >> v1:172.21.15.89:6789/0 conn(0x7fdff809e410 legacy=0x7fdff8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdffd2c4700 1 -- 172.21.15.89:0/3242854039 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff809b2d0 con 0x7fdff809d6f0 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdffcac3700 1 --2- 172.21.15.89:0/3242854039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e000 0x7fdff8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdffd2c4700 1 --2- 172.21.15.89:0/3242854039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809d6f0 0x7fdff809dac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009840 tx=0x7fdff4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30525a369bf8ac1e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdfeffff700 1 -- 172.21.15.89:0/3242854039 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4011830 con 0x7fdff809d6f0 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdfeffff700 1 -- 172.21.15.89:0/3242854039 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff4011990 con 0x7fdff809d6f0 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.132+0000 7fdfeffff700 1 -- 172.21.15.89:0/3242854039 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4011d20 con 0x7fdff809d6f0 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.133+0000 7fdfff528700 1 -- 172.21.15.89:0/3242854039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809d6f0 msgr2=0x7fdff809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.133+0000 7fdfff528700 1 --2- 172.21.15.89:0/3242854039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809d6f0 0x7fdff809dac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009840 tx=0x7fdff4009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.133+0000 7fdfff528700 1 -- 172.21.15.89:0/3242854039 shutdown_connections 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.133+0000 7fdfff528700 1 --2- 172.21.15.89:0/3242854039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809d6f0 0x7fdff809dac0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.133+0000 7fdfff528700 1 --2- 172.21.15.89:0/3242854039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e000 0x7fdff8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.133+0000 7fdfff528700 1 -- 172.21.15.89:0/3242854039 >> 172.21.15.89:0/3242854039 conn(0x7fdff8066f50 msgr2=0x7fdff809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.133+0000 7fdfff528700 1 -- 172.21.15.89:0/3242854039 shutdown_connections 2023-09-30T22:17:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.133+0000 7fdfff528700 1 -- 172.21.15.89:0/3242854039 wait complete. 2023-09-30T22:17:28.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdfff528700 1 Processor -- start 2023-09-30T22:17:28.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdfff528700 1 -- start start 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdfff528700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff809d6f0 0x7fdff81c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdfff528700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809e000 0x7fdff81c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdfff528700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e410 0x7fdff81c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdfff528700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff8123b70 con 0x7fdff809e000 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdfff528700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff8124950 con 0x7fdff809e410 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdfff528700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdff809c700 con 0x7fdff809d6f0 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffcac3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809e000 0x7fdff81c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffdac5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e410 0x7fdff81c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffcac3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809e000 0x7fdff81c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33044/0 (socket says 172.21.15.89:33044) 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffcac3700 1 -- 172.21.15.89:0/916518043 learned_addr learned my addr 172.21.15.89:0/916518043 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffd2c4700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff809d6f0 0x7fdff81c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffcac3700 1 -- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff809d6f0 msgr2=0x7fdff81c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffcac3700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff809d6f0 0x7fdff81c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffcac3700 1 -- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e410 msgr2=0x7fdff81c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffcac3700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e410 0x7fdff81c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffcac3700 1 -- 172.21.15.89:0/916518043 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8009550 con 0x7fdff809e000 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.134+0000 7fdffdac5700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e410 0x7fdff81c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.135+0000 7fdffcac3700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809e000 0x7fdff81c6ad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800ebe0 tx=0x7fdfe800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.135+0000 7fdfedffb700 1 -- 172.21.15.89:0/916518043 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800c850 con 0x7fdff809e000 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.135+0000 7fdfedffb700 1 -- 172.21.15.89:0/916518043 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe800c9b0 con 0x7fdff809e000 2023-09-30T22:17:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.135+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff40094a0 con 0x7fdff809e000 2023-09-30T22:17:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.135+0000 7fdfedffb700 1 -- 172.21.15.89:0/916518043 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800cd40 con 0x7fdff809e000 2023-09-30T22:17:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.135+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff81237d0 con 0x7fdff809e000 2023-09-30T22:17:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.136+0000 7fdfedffb700 1 -- 172.21.15.89:0/916518043 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdfe8013470 con 0x7fdff809e000 2023-09-30T22:17:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.136+0000 7fdfedffb700 1 -- 172.21.15.89:0/916518043 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdfe803f830 con 0x7fdff809e000 2023-09-30T22:17:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.136+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff8074ea0 con 0x7fdff809e000 2023-09-30T22:17:28.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.141+0000 7fdfedffb700 1 -- 172.21.15.89:0/916518043 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdfe8021080 con 0x7fdff809e000 2023-09-30T22:17:28.148 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:28.205 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:28.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.277+0000 7f73460fa700 1 Processor -- start 2023-09-30T22:17:28.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.277+0000 7f73460fa700 1 -- start start 2023-09-30T22:17:28.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f73460fa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f734009d6f0 0x7f734009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f73460fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f7340130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f73460fa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734009f130 con 0x7f734009e410 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f73460fa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734009f520 con 0x7f734009d6f0 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f73460fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734009e930 con 0x7f734009e000 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f734009d6f0 0x7f734009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f734009d6f0 0x7f734009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33060/0 (socket says 172.21.15.89:33060) 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733f7fe700 1 -- 172.21.15.89:0/4051996813 learned_addr learned my addr 172.21.15.89:0/4051996813 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733effd700 1 --2- 172.21.15.89:0/4051996813 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f7340130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733effd700 1 -- 172.21.15.89:0/4051996813 >> v2:172.21.15.89:3300/0 conn(0x7f734009d6f0 msgr2=0x7f734009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733effd700 1 --2- 172.21.15.89:0/4051996813 >> v2:172.21.15.89:3300/0 conn(0x7f734009d6f0 0x7f734009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733effd700 1 -- 172.21.15.89:0/4051996813 >> v1:172.21.15.89:6789/0 conn(0x7f734009e410 legacy=0x7f7340130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733effd700 1 -- 172.21.15.89:0/4051996813 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734009b2d0 con 0x7f734009e000 2023-09-30T22:17:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.278+0000 7f733f7fe700 1 --2- 172.21.15.89:0/4051996813 >> v2:172.21.15.89:3300/0 conn(0x7f734009d6f0 0x7f734009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.279+0000 7f733effd700 1 --2- 172.21.15.89:0/4051996813 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f7340130660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7334009840 tx=0x7f7334009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff1decf40ef5b5f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.279+0000 7f733e7fc700 1 -- 172.21.15.89:0/4051996813 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334011830 con 0x7f734009e000 2023-09-30T22:17:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.279+0000 7f733e7fc700 1 -- 172.21.15.89:0/4051996813 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334011990 con 0x7f734009e000 2023-09-30T22:17:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.279+0000 7f733e7fc700 1 -- 172.21.15.89:0/4051996813 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334011d20 con 0x7f734009e000 2023-09-30T22:17:28.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.279+0000 7f73460fa700 1 -- 172.21.15.89:0/4051996813 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 msgr2=0x7f7340130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.279+0000 7f73460fa700 1 --2- 172.21.15.89:0/4051996813 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f7340130660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7334009840 tx=0x7f7334009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:28.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.280+0000 7f73460fa700 1 -- 172.21.15.89:0/4051996813 shutdown_connections 2023-09-30T22:17:28.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.280+0000 7f73460fa700 1 --2- 172.21.15.89:0/4051996813 >> v2:172.21.15.89:3300/0 conn(0x7f734009d6f0 0x7f734009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.280+0000 7f73460fa700 1 --2- 172.21.15.89:0/4051996813 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f7340130660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.280+0000 7f73460fa700 1 -- 172.21.15.89:0/4051996813 >> 172.21.15.89:0/4051996813 conn(0x7f7340066f50 msgr2=0x7f734009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.280+0000 7f73460fa700 1 -- 172.21.15.89:0/4051996813 shutdown_connections 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.280+0000 7f73460fa700 1 -- 172.21.15.89:0/4051996813 wait complete. 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f73460fa700 1 Processor -- start 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f73460fa700 1 -- start start 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f73460fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734009d6f0 0x7f73401c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f73460fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f73401c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f73460fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734009e410 0x7f73401c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f73460fa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734006b280 con 0x7f734009d6f0 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f73460fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73401204c0 con 0x7f734009e000 2023-09-30T22:17:28.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f73460fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7340125980 con 0x7f734009e410 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734009d6f0 0x7f73401c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734009d6f0 0x7f73401c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33068/0 (socket says 172.21.15.89:33068) 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f733f7fe700 1 -- 172.21.15.89:0/4051186821 learned_addr learned my addr 172.21.15.89:0/4051186821 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.281+0000 7f733ffff700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734009e410 0x7f73401c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f733effd700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f73401c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f733ffff700 1 -- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 msgr2=0x7f73401c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f733ffff700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f73401c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f733ffff700 1 -- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734009d6f0 msgr2=0x7f73401c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f733ffff700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734009d6f0 0x7f73401c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f733ffff700 1 -- 172.21.15.89:0/4051186821 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330009550 con 0x7f734009e410 2023-09-30T22:17:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f733f7fe700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734009d6f0 0x7f73401c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:28.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f733ffff700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734009e410 0x7f73401c8f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f733800cb90 tx=0x7f733800cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:28.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f7327fff700 1 -- 172.21.15.89:0/4051186821 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338003c00 con 0x7f734009e410 2023-09-30T22:17:28.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f7327fff700 1 -- 172.21.15.89:0/4051186821 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7338003d60 con 0x7f734009e410 2023-09-30T22:17:28.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73340094a0 con 0x7f734009e410 2023-09-30T22:17:28.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.282+0000 7f7327fff700 1 -- 172.21.15.89:0/4051186821 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800aa80 con 0x7f734009e410 2023-09-30T22:17:28.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.283+0000 7f7327fff700 1 -- 172.21.15.89:0/4051186821 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f733800abe0 con 0x7f734009e410 2023-09-30T22:17:28.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.283+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734009c700 con 0x7f734009e410 2023-09-30T22:17:28.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.283+0000 7f7327fff700 1 -- 172.21.15.89:0/4051186821 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7338011540 con 0x7f734009e410 2023-09-30T22:17:28.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.284+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734009dcd0 con 0x7f734009e410 2023-09-30T22:17:28.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.293+0000 7f7327fff700 1 -- 172.21.15.89:0/4051186821 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73380198f0 con 0x7f734009e410 2023-09-30T22:17:28.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.333+0000 7f0fd038d700 1 Processor -- start 2023-09-30T22:17:28.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.333+0000 7f0fd038d700 1 -- start start 2023-09-30T22:17:28.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fd038d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc8131810 0x7f0fc8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fd038d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8099a40 0x7f0fc8099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fd038d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0fc809a3b0 0x7f0fc8138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fd038d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc809b580 con 0x7f0fc809a3b0 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fd038d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81313c0 con 0x7f0fc8099a40 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fd038d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc806e8c0 con 0x7f0fc8131810 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fce92a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0fc809a3b0 0x7f0fc8138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fce92a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0fc809a3b0 0x7f0fc8138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33074/0 (socket says 172.21.15.89:33074) 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fce92a700 1 -- 172.21.15.89:0/1678273457 learned_addr learned my addr 172.21.15.89:0/1678273457 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fce129700 1 --2- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc8131810 0x7f0fc8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fcd928700 1 --2- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8099a40 0x7f0fc8099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fcd928700 1 -- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc8131810 msgr2=0x7f0fc8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fcd928700 1 --2- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc8131810 0x7f0fc8131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.334+0000 7f0fcd928700 1 -- 172.21.15.89:0/1678273457 >> v2:172.21.15.89:3300/0 conn(0x7f0fc809a3b0 msgr2=0x7f0fc8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.335+0000 7f0fcd928700 1 --2- 172.21.15.89:0/1678273457 >> v2:172.21.15.89:3300/0 conn(0x7f0fc809a3b0 0x7f0fc8138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.335+0000 7f0fcd928700 1 -- 172.21.15.89:0/1678273457 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc809a820 con 0x7f0fc8099a40 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.335+0000 7f0fce129700 1 --2- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc8131810 0x7f0fc8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.335+0000 7f0fcd928700 1 --2- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8099a40 0x7f0fc8099e70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009840 tx=0x7f0fb8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a551caeea499d50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.335+0000 7f0fcd127700 1 -- 172.21.15.89:0/1678273457 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8011830 con 0x7f0fc8099a40 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.335+0000 7f0fcd127700 1 -- 172.21.15.89:0/1678273457 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb8011990 con 0x7f0fc8099a40 2023-09-30T22:17:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.335+0000 7f0fcd127700 1 -- 172.21.15.89:0/1678273457 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8011d20 con 0x7f0fc8099a40 2023-09-30T22:17:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 -- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8099a40 msgr2=0x7f0fc8099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 --2- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8099a40 0x7f0fc8099e70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009840 tx=0x7f0fb8009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 -- 172.21.15.89:0/1678273457 shutdown_connections 2023-09-30T22:17:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 --2- 172.21.15.89:0/1678273457 >> v2:172.21.15.89:3300/0 conn(0x7f0fc809a3b0 0x7f0fc8138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 --2- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc8131810 0x7f0fc8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 --2- 172.21.15.89:0/1678273457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8099a40 0x7f0fc8099e70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 -- 172.21.15.89:0/1678273457 >> 172.21.15.89:0/1678273457 conn(0x7f0fc80a2aa0 msgr2=0x7f0fc80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 -- 172.21.15.89:0/1678273457 shutdown_connections 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.336+0000 7f0fd038d700 1 -- 172.21.15.89:0/1678273457 wait complete. 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.337+0000 7f0fd038d700 1 Processor -- start 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.337+0000 7f0fd038d700 1 -- start start 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fd038d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc8099a40 0x7f0fc81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fd038d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc809a3b0 0x7f0fc81c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fd038d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8131810 0x7f0fc81c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fd038d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc812e140 con 0x7f0fc8099a40 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fd038d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc812df50 con 0x7f0fc8131810 2023-09-30T22:17:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fd038d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81333b0 con 0x7f0fc809a3b0 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fce129700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc8099a40 0x7f0fc81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fce129700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc8099a40 0x7f0fc81c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33080/0 (socket says 172.21.15.89:33080) 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fce129700 1 -- 172.21.15.89:0/3595371477 learned_addr learned my addr 172.21.15.89:0/3595371477 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fce92a700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8131810 0x7f0fc81c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.338+0000 7f0fcd928700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc809a3b0 0x7f0fc81c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fce129700 1 -- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc809a3b0 msgr2=0x7f0fc81c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fce129700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc809a3b0 0x7f0fc81c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fce129700 1 -- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8131810 msgr2=0x7f0fc81c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fce129700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8131810 0x7f0fc81c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fce129700 1 -- 172.21.15.89:0/3595371477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40094a0 con 0x7f0fc8099a40 2023-09-30T22:17:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fce129700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc8099a40 0x7f0fc81c8090 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400ebe0 tx=0x7f0fc400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:28.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fbeffd700 1 -- 172.21.15.89:0/3595371477 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4009800 con 0x7f0fc8099a40 2023-09-30T22:17:28.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fbeffd700 1 -- 172.21.15.89:0/3595371477 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc400c910 con 0x7f0fc8099a40 2023-09-30T22:17:28.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fbeffd700 1 -- 172.21.15.89:0/3595371477 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400cc80 con 0x7f0fc8099a40 2023-09-30T22:17:28.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb80094a0 con 0x7f0fc8099a40 2023-09-30T22:17:28.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.339+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc81330d0 con 0x7f0fc8099a40 2023-09-30T22:17:28.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.340+0000 7f0fbeffd700 1 -- 172.21.15.89:0/3595371477 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0fc4013470 con 0x7f0fc8099a40 2023-09-30T22:17:28.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.340+0000 7f0fbeffd700 1 -- 172.21.15.89:0/3595371477 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0fc403fc00 con 0x7f0fc8099a40 2023-09-30T22:17:28.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.340+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc812e7a0 con 0x7f0fc8099a40 2023-09-30T22:17:28.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.346+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdff81303d0 con 0x7fdff809e000 2023-09-30T22:17:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.349+0000 7fdfedffb700 1 -- 172.21.15.89:0/916518043 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdfe8017020 con 0x7fdff809e000 2023-09-30T22:17:28.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.349+0000 7f0fbeffd700 1 -- 172.21.15.89:0/3595371477 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0fc401b3b0 con 0x7f0fc8099a40 2023-09-30T22:17:28.351 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.352+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809e000 msgr2=0x7fdff81c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.352+0000 7fdfff528700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809e000 0x7fdff81c6ad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800ebe0 tx=0x7fdfe800eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.353+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 shutdown_connections 2023-09-30T22:17:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.353+0000 7fdfff528700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff809e410 0x7fdff81c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.353+0000 7fdfff528700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff809d6f0 0x7fdff81c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.353+0000 7fdfff528700 1 --2- 172.21.15.89:0/916518043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff809e000 0x7fdff81c6ad0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.353+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 >> 172.21.15.89:0/916518043 conn(0x7fdff8066f50 msgr2=0x7fdff8070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:28.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.355+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 shutdown_connections 2023-09-30T22:17:28.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.355+0000 7fdfff528700 1 -- 172.21.15.89:0/916518043 wait complete. 2023-09-30T22:17:28.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc3959fc700 1 Processor -- start 2023-09-30T22:17:28.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc3959fc700 1 -- start start 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc3959fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39009de00 0x7fc39009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc3959fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc39013d1f0 0x7fc39013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc3959fc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc39009a810 con 0x7fc39006a770 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc3959fc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39006ab40 con 0x7fc39013d1f0 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc3959fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc39009ebc0 con 0x7fc39009de00 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc38effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc39006a770 0x7fc39009d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49982/0 (socket says 172.21.15.89:49982) 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc38effd700 1 -- 172.21.15.89:0/1292983835 learned_addr learned my addr 172.21.15.89:0/1292983835 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc38e7fc700 1 --2- 172.21.15.89:0/1292983835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39009de00 0x7fc39009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.365+0000 7fc38f7fe700 1 --2- 172.21.15.89:0/1292983835 >> v2:172.21.15.89:3300/0 conn(0x7fc39013d1f0 0x7fc39013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.366+0000 7fc38e7fc700 1 -- 172.21.15.89:0/1292983835 >> v2:172.21.15.89:3300/0 conn(0x7fc39013d1f0 msgr2=0x7fc39013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:28.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.366+0000 7fc38e7fc700 1 --2- 172.21.15.89:0/1292983835 >> v2:172.21.15.89:3300/0 conn(0x7fc39013d1f0 0x7fc39013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.366+0000 7fc38e7fc700 1 -- 172.21.15.89:0/1292983835 >> v1:172.21.15.89:6789/0 conn(0x7fc39006a770 legacy=0x7fc39009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.366+0000 7fc38e7fc700 1 -- 172.21.15.89:0/1292983835 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39009e920 con 0x7fc39009de00 2023-09-30T22:17:28.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.366+0000 7fc38e7fc700 1 --2- 172.21.15.89:0/1292983835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39009de00 0x7fc39009e250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc380009840 tx=0x7fc380009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5434cf595d94ab5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:28.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.366+0000 7fc38dffb700 1 -- 172.21.15.89:0/1292983835 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380011830 con 0x7fc39009de00 2023-09-30T22:17:28.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.366+0000 7fc38dffb700 1 -- 172.21.15.89:0/1292983835 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380011990 con 0x7fc39009de00 2023-09-30T22:17:28.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.366+0000 7fc38dffb700 1 -- 172.21.15.89:0/1292983835 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380011d20 con 0x7fc39009de00 2023-09-30T22:17:28.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.367+0000 7fc3959fc700 1 -- 172.21.15.89:0/1292983835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39009de00 msgr2=0x7fc39009e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.367+0000 7fc3959fc700 1 --2- 172.21.15.89:0/1292983835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39009de00 0x7fc39009e250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc380009840 tx=0x7fc380009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:28.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.367+0000 7fc3959fc700 1 -- 172.21.15.89:0/1292983835 shutdown_connections 2023-09-30T22:17:28.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.367+0000 7fc3959fc700 1 --2- 172.21.15.89:0/1292983835 >> v2:172.21.15.89:3300/0 conn(0x7fc39013d1f0 0x7fc39013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.367+0000 7fc3959fc700 1 --2- 172.21.15.89:0/1292983835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39009de00 0x7fc39009e250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.367+0000 7fc3959fc700 1 -- 172.21.15.89:0/1292983835 >> 172.21.15.89:0/1292983835 conn(0x7fc390066f50 msgr2=0x7fc39009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:28.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.367+0000 7fc3959fc700 1 -- 172.21.15.89:0/1292983835 shutdown_connections 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.367+0000 7fc3959fc700 1 -- 172.21.15.89:0/1292983835 wait complete. 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.368+0000 7fc3959fc700 1 Processor -- start 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.368+0000 7fc3959fc700 1 -- start start 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.368+0000 7fc3959fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39006a770 0x7fc3901260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.368+0000 7fc3959fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc39009de00 0x7fc3901265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.368+0000 7fc3959fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39013d1f0 0x7fc390126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.368+0000 7fc3959fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900727f0 con 0x7fc39006a770 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.368+0000 7fc3959fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3901282c0 con 0x7fc39009de00 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.368+0000 7fc3959fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900a3990 con 0x7fc39013d1f0 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc39009de00 0x7fc3901265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39013d1f0 0x7fc390126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc39009de00 0x7fc3901265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43502/0 (socket says 172.21.15.89:43502) 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 -- 172.21.15.89:0/1416630902 learned_addr learned my addr 172.21.15.89:0/1416630902 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39013d1f0 0x7fc390126b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44908/0 (socket says 172.21.15.89:44908) 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38effd700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39006a770 0x7fc3901260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 -- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39013d1f0 msgr2=0x7fc390126b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39013d1f0 0x7fc390126b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 -- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39006a770 msgr2=0x7fc3901260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39006a770 0x7fc3901260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 -- 172.21.15.89:0/1416630902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800094a0 con 0x7fc39009de00 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38f7fe700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39013d1f0 0x7fc390126b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc38e7fc700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc39009de00 0x7fc3901265e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc380009fd0 tx=0x7fc380004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc377fff700 1 -- 172.21.15.89:0/1416630902 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800036a0 con 0x7fc39009de00 2023-09-30T22:17:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc377fff700 1 -- 172.21.15.89:0/1416630902 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38001f070 con 0x7fc39009de00 2023-09-30T22:17:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc390127f60 con 0x7fc39009de00 2023-09-30T22:17:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.369+0000 7fc377fff700 1 -- 172.21.15.89:0/1416630902 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380003a60 con 0x7fc39009de00 2023-09-30T22:17:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.370+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc390127060 con 0x7fc39009de00 2023-09-30T22:17:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.371+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39011cc30 con 0x7fc39009de00 2023-09-30T22:17:28.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.371+0000 7fc377fff700 1 -- 172.21.15.89:0/1416630902 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc380003bc0 con 0x7fc39009de00 2023-09-30T22:17:28.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.371+0000 7fc377fff700 1 -- 172.21.15.89:0/1416630902 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc380021070 con 0x7fc39009de00 2023-09-30T22:17:28.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.380+0000 7fc377fff700 1 -- 172.21.15.89:0/1416630902 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3800193b0 con 0x7fc39009de00 2023-09-30T22:17:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.508+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7340074ea0 con 0x7f734009e410 2023-09-30T22:17:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.508+0000 7f7327fff700 1 -- 172.21.15.89:0/4051186821 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7338011380 con 0x7f734009e410 2023-09-30T22:17:28.509 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:28.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.510+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734009e410 msgr2=0x7f73401c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.510+0000 7f73460fa700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734009e410 0x7f73401c8f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f733800cb90 tx=0x7f733800cea0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.510+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 shutdown_connections 2023-09-30T22:17:28.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.510+0000 7f73460fa700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f734009e410 0x7f73401c8f50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.510+0000 7f73460fa700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f734009d6f0 0x7f73401c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.510+0000 7f73460fa700 1 --2- 172.21.15.89:0/4051186821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f734009e000 0x7f73401c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.510+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 >> 172.21.15.89:0/4051186821 conn(0x7f7340066f50 msgr2=0x7f7340070120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:28.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.510+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 shutdown_connections 2023-09-30T22:17:28.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.511+0000 7f73460fa700 1 -- 172.21.15.89:0/4051186821 wait complete. 2023-09-30T22:17:28.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.552+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0fc8074ea0 con 0x7f0fc8099a40 2023-09-30T22:17:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.552+0000 7f0fbeffd700 1 -- 172.21.15.89:0/3595371477 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0fc4017020 con 0x7f0fc8099a40 2023-09-30T22:17:28.553 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc8099a40 msgr2=0x7f0fc81c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc8099a40 0x7f0fc81c8090 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400ebe0 tx=0x7f0fc400eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 shutdown_connections 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0fc8131810 0x7f0fc81c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc8099a40 0x7f0fc81c8090 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 --2- 172.21.15.89:0/3595371477 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0fc809a3b0 0x7f0fc81c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 >> 172.21.15.89:0/3595371477 conn(0x7f0fc80a2aa0 msgr2=0x7f0fc812f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 shutdown_connections 2023-09-30T22:17:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.554+0000 7f0fd038d700 1 -- 172.21.15.89:0/3595371477 wait complete. 2023-09-30T22:17:28.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:28.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.573+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc390074ea0 con 0x7fc39009de00 2023-09-30T22:17:28.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.573+0000 7fc377fff700 1 -- 172.21.15.89:0/1416630902 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc380034090 con 0x7fc39009de00 2023-09-30T22:17:28.574 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:28.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.575+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc39009de00 msgr2=0x7fc3901265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:28.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.575+0000 7fc3959fc700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc39009de00 0x7fc3901265e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc380009fd0 tx=0x7fc380004750 comp rx=0 tx=0).stop 2023-09-30T22:17:28.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.575+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 shutdown_connections 2023-09-30T22:17:28.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.575+0000 7fc3959fc700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc39013d1f0 0x7fc390126b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.575+0000 7fc3959fc700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc39006a770 0x7fc3901260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.575+0000 7fc3959fc700 1 --2- 172.21.15.89:0/1416630902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc39009de00 0x7fc3901265e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:28.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.575+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 >> 172.21.15.89:0/1416630902 conn(0x7fc390066f50 msgr2=0x7fc3900a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:28.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.576+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 shutdown_connections 2023-09-30T22:17:28.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:28.576+0000 7fc3959fc700 1 -- 172.21.15.89:0/1416630902 wait complete. 2023-09-30T22:17:28.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:29.377 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:29.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.511+0000 7fdd4c3ae700 1 Processor -- start 2023-09-30T22:17:29.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.511+0000 7fdd4c3ae700 1 -- start start 2023-09-30T22:17:29.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4c3ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd44133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4c3ae700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd4409e930 0x7fdd4413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4c3ae700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4406ef30 con 0x7fdd4409de50 2023-09-30T22:17:29.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4c3ae700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4406fef0 con 0x7fdd4409e930 2023-09-30T22:17:29.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4c3ae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd441334b0 con 0x7fdd441338a0 2023-09-30T22:17:29.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a94b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd4409e930 0x7fdd4413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a14a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd44133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a14a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd44133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44918/0 (socket says 172.21.15.89:44918) 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a94b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd4409e930 0x7fdd4413b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33106/0 (socket says 172.21.15.89:33106) 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a14a700 1 -- 172.21.15.89:0/3970173612 learned_addr learned my addr 172.21.15.89:0/3970173612 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd49949700 1 --1- 172.21.15.89:0/3970173612 >> v1:172.21.15.89:6789/0 conn(0x7fdd4409de50 0x7fdd4409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49986/0 (socket says 172.21.15.89:49986) 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a14a700 1 -- 172.21.15.89:0/3970173612 >> v2:172.21.15.89:3300/0 conn(0x7fdd4409e930 msgr2=0x7fdd4413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a14a700 1 --2- 172.21.15.89:0/3970173612 >> v2:172.21.15.89:3300/0 conn(0x7fdd4409e930 0x7fdd4413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a14a700 1 -- 172.21.15.89:0/3970173612 >> v1:172.21.15.89:6789/0 conn(0x7fdd4409de50 legacy=0x7fdd4409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd49148700 1 -- 172.21.15.89:0/3970173612 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493176987 0 0) 0x7fdd4406ef30 con 0x7fdd4409de50 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a14a700 1 -- 172.21.15.89:0/3970173612 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4406fc30 con 0x7fdd441338a0 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a94b700 1 --2- 172.21.15.89:0/3970173612 >> v2:172.21.15.89:3300/0 conn(0x7fdd4409e930 0x7fdd4413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:29.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd4a14a700 1 --2- 172.21.15.89:0/3970173612 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd44133c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd340098f0 tx=0x7fdd34009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0fdc5f2e5aa0884 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd49148700 1 -- 172.21.15.89:0/3970173612 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34011830 con 0x7fdd441338a0 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd49148700 1 -- 172.21.15.89:0/3970173612 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd34011990 con 0x7fdd441338a0 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.512+0000 7fdd49148700 1 -- 172.21.15.89:0/3970173612 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34011d20 con 0x7fdd441338a0 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.513+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/3970173612 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 msgr2=0x7fdd44133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.513+0000 7fdd4c3ae700 1 --2- 172.21.15.89:0/3970173612 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd44133c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd340098f0 tx=0x7fdd34009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.513+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/3970173612 shutdown_connections 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.513+0000 7fdd4c3ae700 1 --2- 172.21.15.89:0/3970173612 >> v2:172.21.15.89:3300/0 conn(0x7fdd4409e930 0x7fdd4413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.513+0000 7fdd4c3ae700 1 --2- 172.21.15.89:0/3970173612 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd44133c70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.513+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/3970173612 >> 172.21.15.89:0/3970173612 conn(0x7fdd440a2aa0 msgr2=0x7fdd440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.513+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/3970173612 shutdown_connections 2023-09-30T22:17:29.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.513+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/3970173612 wait complete. 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4c3ae700 1 Processor -- start 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4c3ae700 1 -- start start 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4c3ae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4409de50 0x7fdd4411d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4c3ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4409e930 0x7fdd4411dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4c3ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd4411e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4c3ae700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4411eaf0 con 0x7fdd4409e930 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4c3ae700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4411f270 con 0x7fdd4409de50 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4c3ae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4411f890 con 0x7fdd441338a0 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4a94b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd4411e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd4a14a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4409de50 0x7fdd4411d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a94b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd4411e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44930/0 (socket says 172.21.15.89:44930) 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a94b700 1 -- 172.21.15.89:0/2048313490 learned_addr learned my addr 172.21.15.89:0/2048313490 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.514+0000 7fdd49949700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4409e930 0x7fdd4411dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a14a700 1 -- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 msgr2=0x7fdd4411e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a14a700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd4411e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a14a700 1 -- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4409e930 msgr2=0x7fdd4411dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a14a700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4409e930 0x7fdd4411dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a14a700 1 -- 172.21.15.89:0/2048313490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009a70 con 0x7fdd4409de50 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd49949700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4409e930 0x7fdd4411dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a94b700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd4411e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4a14a700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4409de50 0x7fdd4411d660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd4406dc90 tx=0x7fdd34004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:29.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd3affd700 1 -- 172.21.15.89:0/2048313490 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34004300 con 0x7fdd4409de50 2023-09-30T22:17:29.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd3affd700 1 -- 172.21.15.89:0/2048313490 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3401f070 con 0x7fdd4409de50 2023-09-30T22:17:29.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd34009550 con 0x7fdd4409de50 2023-09-30T22:17:29.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd3affd700 1 -- 172.21.15.89:0/2048313490 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340036a0 con 0x7fdd4409de50 2023-09-30T22:17:29.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.515+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd440a0eb0 con 0x7fdd4409de50 2023-09-30T22:17:29.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.516+0000 7fdd3affd700 1 -- 172.21.15.89:0/2048313490 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdd34021030 con 0x7fdd4409de50 2023-09-30T22:17:29.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.516+0000 7fdd3affd700 1 -- 172.21.15.89:0/2048313490 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdd340408b0 con 0x7fdd4409de50 2023-09-30T22:17:29.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.516+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4406f6a0 con 0x7fdd4409de50 2023-09-30T22:17:29.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.521+0000 7fdd3affd700 1 -- 172.21.15.89:0/2048313490 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdd340239c0 con 0x7fdd4409de50 2023-09-30T22:17:29.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:29.567 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:29.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:29.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.662+0000 7f6c0dffa700 1 Processor -- start 2023-09-30T22:17:29.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.662+0000 7f6c0dffa700 1 -- start start 2023-09-30T22:17:29.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.662+0000 7f6c0dffa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0809de00 0x7f6c0809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.662+0000 7f6c0dffa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0813d1f0 0x7f6c0813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c0dffa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0809a810 con 0x7f6c0806a770 2023-09-30T22:17:29.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c0dffa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0806ab40 con 0x7f6c0813d1f0 2023-09-30T22:17:29.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c0dffa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0809ebc0 con 0x7f6c0809de00 2023-09-30T22:17:29.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c07fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0813d1f0 0x7f6c0813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c06ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0809de00 0x7f6c0809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c077fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6c0806a770 0x7f6c0809d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49990/0 (socket says 172.21.15.89:49990) 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c06ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0809de00 0x7f6c0809e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44946/0 (socket says 172.21.15.89:44946) 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c07fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0813d1f0 0x7f6c0813f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43534/0 (socket says 172.21.15.89:43534) 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c077fe700 1 -- 172.21.15.89:0/4135504650 learned_addr learned my addr 172.21.15.89:0/4135504650 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c067fc700 1 -- 172.21.15.89:0/4135504650 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228351590 0 0) 0x7f6c0809a810 con 0x7f6c0806a770 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c06ffd700 1 -- 172.21.15.89:0/4135504650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0813d1f0 msgr2=0x7f6c0813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c06ffd700 1 --2- 172.21.15.89:0/4135504650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0813d1f0 0x7f6c0813f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c06ffd700 1 -- 172.21.15.89:0/4135504650 >> v1:172.21.15.89:6789/0 conn(0x7f6c0806a770 legacy=0x7f6c0809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c06ffd700 1 -- 172.21.15.89:0/4135504650 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0809e920 con 0x7f6c0809de00 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c06ffd700 1 --2- 172.21.15.89:0/4135504650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0809de00 0x7f6c0809e250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009870 tx=0x7f6bfc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e90cc602e7138df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:29.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.663+0000 7f6c067fc700 1 -- 172.21.15.89:0/4135504650 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc011830 con 0x7f6c0809de00 2023-09-30T22:17:29.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.664+0000 7f6c067fc700 1 -- 172.21.15.89:0/4135504650 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc011990 con 0x7f6c0809de00 2023-09-30T22:17:29.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.664+0000 7f6c067fc700 1 -- 172.21.15.89:0/4135504650 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc011d20 con 0x7f6c0809de00 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.664+0000 7f6c0dffa700 1 -- 172.21.15.89:0/4135504650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0809de00 msgr2=0x7f6c0809e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.664+0000 7f6c0dffa700 1 --2- 172.21.15.89:0/4135504650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0809de00 0x7f6c0809e250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009870 tx=0x7f6bfc009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.665+0000 7f6c0dffa700 1 -- 172.21.15.89:0/4135504650 shutdown_connections 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.665+0000 7f6c0dffa700 1 --2- 172.21.15.89:0/4135504650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0813d1f0 0x7f6c0813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.665+0000 7f6c0dffa700 1 --2- 172.21.15.89:0/4135504650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0809de00 0x7f6c0809e250 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.665+0000 7f6c0dffa700 1 -- 172.21.15.89:0/4135504650 >> 172.21.15.89:0/4135504650 conn(0x7f6c08066f50 msgr2=0x7f6c0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.665+0000 7f6c0dffa700 1 -- 172.21.15.89:0/4135504650 shutdown_connections 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.665+0000 7f6c0dffa700 1 -- 172.21.15.89:0/4135504650 wait complete. 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.666+0000 7f6c0dffa700 1 Processor -- start 2023-09-30T22:17:29.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.666+0000 7f6c0dffa700 1 -- start start 2023-09-30T22:17:29.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c0dffa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0806a770 0x7f6c081cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c0dffa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0809de00 0x7f6c081ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c0dffa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0813d1f0 0x7f6c081cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c0dffa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0806e840 con 0x7f6c0806a770 2023-09-30T22:17:29.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c0dffa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0809f1a0 con 0x7f6c0809de00 2023-09-30T22:17:29.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c0dffa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0806eda0 con 0x7f6c0813d1f0 2023-09-30T22:17:29.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c07fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0813d1f0 0x7f6c081cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c07fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0813d1f0 0x7f6c081cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44956/0 (socket says 172.21.15.89:44956) 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c07fff700 1 -- 172.21.15.89:0/886101067 learned_addr learned my addr 172.21.15.89:0/886101067 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c06ffd700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0809de00 0x7f6c081ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c077fe700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0806a770 0x7f6c081cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c07fff700 1 -- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0809de00 msgr2=0x7f6c081ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c07fff700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0809de00 0x7f6c081ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c07fff700 1 -- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0806a770 msgr2=0x7f6c081cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c07fff700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0806a770 0x7f6c081cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.667+0000 7f6c07fff700 1 -- 172.21.15.89:0/886101067 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0094a0 con 0x7f6c0813d1f0 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.668+0000 7f6c07fff700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0813d1f0 0x7f6c081cd440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000efc0 tx=0x7f6c0000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:29.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.668+0000 7f6beffff700 1 -- 172.21.15.89:0/886101067 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00009ea0 con 0x7f6c0813d1f0 2023-09-30T22:17:29.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.668+0000 7f6beffff700 1 -- 172.21.15.89:0/886101067 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0000f040 con 0x7f6c0813d1f0 2023-09-30T22:17:29.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.668+0000 7f6beffff700 1 -- 172.21.15.89:0/886101067 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0000ced0 con 0x7f6c0813d1f0 2023-09-30T22:17:29.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.668+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0806fd60 con 0x7f6c0813d1f0 2023-09-30T22:17:29.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.668+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c080707d0 con 0x7f6c0813d1f0 2023-09-30T22:17:29.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.669+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0806f960 con 0x7f6c0813d1f0 2023-09-30T22:17:29.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.670+0000 7f6beffff700 1 -- 172.21.15.89:0/886101067 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c00014480 con 0x7f6c0813d1f0 2023-09-30T22:17:29.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.670+0000 7f6beffff700 1 -- 172.21.15.89:0/886101067 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6c00040820 con 0x7f6c0813d1f0 2023-09-30T22:17:29.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.674+0000 7f6beffff700 1 -- 172.21.15.89:0/886101067 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c00012910 con 0x7f6c0813d1f0 2023-09-30T22:17:29.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.710+0000 7f91e7e94700 1 Processor -- start 2023-09-30T22:17:29.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.710+0000 7f91e7e94700 1 -- start start 2023-09-30T22:17:29.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e7e94700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e7e94700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e000 0x7f91e0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e7e94700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f91e009e410 0x7f91e0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e7e94700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e009f520 con 0x7f91e009e410 2023-09-30T22:17:29.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e7e94700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e009f130 con 0x7f91e009d6f0 2023-09-30T22:17:29.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e7e94700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91e009e930 con 0x7f91e009e000 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43544/0 (socket says 172.21.15.89:43544) 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 -- 172.21.15.89:0/1299100023 learned_addr learned my addr 172.21.15.89:0/1299100023 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e542f700 1 --2- 172.21.15.89:0/1299100023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e000 0x7f91e0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e6431700 1 --2- 172.21.15.89:0/1299100023 >> v2:172.21.15.89:3300/0 conn(0x7f91e009e410 0x7f91e0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 -- 172.21.15.89:0/1299100023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e000 msgr2=0x7f91e0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 --2- 172.21.15.89:0/1299100023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e000 0x7f91e0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 -- 172.21.15.89:0/1299100023 >> v2:172.21.15.89:3300/0 conn(0x7f91e009e410 msgr2=0x7f91e0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 --2- 172.21.15.89:0/1299100023 >> v2:172.21.15.89:3300/0 conn(0x7f91e009e410 0x7f91e0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 -- 172.21.15.89:0/1299100023 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e009b2d0 con 0x7f91e009d6f0 2023-09-30T22:17:29.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e5c30700 1 --2- 172.21.15.89:0/1299100023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e009dac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009920 tx=0x7f91dc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33489742786378a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:29.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.711+0000 7f91e4c2e700 1 -- 172.21.15.89:0/1299100023 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc011830 con 0x7f91e009d6f0 2023-09-30T22:17:29.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e4c2e700 1 -- 172.21.15.89:0/1299100023 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc011990 con 0x7f91e009d6f0 2023-09-30T22:17:29.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e4c2e700 1 -- 172.21.15.89:0/1299100023 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc011d20 con 0x7f91e009d6f0 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 -- 172.21.15.89:0/1299100023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 msgr2=0x7f91e009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 --2- 172.21.15.89:0/1299100023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e009dac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009920 tx=0x7f91dc009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 -- 172.21.15.89:0/1299100023 shutdown_connections 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 --2- 172.21.15.89:0/1299100023 >> v2:172.21.15.89:3300/0 conn(0x7f91e009e410 0x7f91e0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 --2- 172.21.15.89:0/1299100023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e009dac0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 --2- 172.21.15.89:0/1299100023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e000 0x7f91e0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 -- 172.21.15.89:0/1299100023 >> 172.21.15.89:0/1299100023 conn(0x7f91e0066f50 msgr2=0x7f91e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 -- 172.21.15.89:0/1299100023 shutdown_connections 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.712+0000 7f91e7e94700 1 -- 172.21.15.89:0/1299100023 wait complete. 2023-09-30T22:17:29.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.713+0000 7f91e7e94700 1 Processor -- start 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.713+0000 7f91e7e94700 1 -- start start 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e7e94700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e0122400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e7e94700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e009e000 0x7f91e0122940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e7e94700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e410 0x7f91e0122e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e7e94700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0123fd0 con 0x7f91e009e000 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e7e94700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01255c0 con 0x7f91e009d6f0 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e7e94700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00a3ac0 con 0x7f91e009e410 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e6431700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e410 0x7f91e0122e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e5c30700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e0122400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e5c30700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e0122400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43560/0 (socket says 172.21.15.89:43560) 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e6431700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e410 0x7f91e0122e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44970/0 (socket says 172.21.15.89:44970) 2023-09-30T22:17:29.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e5c30700 1 -- 172.21.15.89:0/2476591587 learned_addr learned my addr 172.21.15.89:0/2476591587 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e542f700 1 --2- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e009e000 0x7f91e0122940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e6431700 1 -- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 msgr2=0x7f91e0122400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e6431700 1 --2- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e0122400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e6431700 1 -- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e009e000 msgr2=0x7f91e0122940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e6431700 1 --2- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e009e000 0x7f91e0122940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e6431700 1 -- 172.21.15.89:0/2476591587 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc009550 con 0x7f91e009e410 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.714+0000 7f91e6431700 1 --2- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e410 0x7f91e0122e80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009b60 tx=0x7f91d8009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.715+0000 7f91d67fc700 1 -- 172.21.15.89:0/2476591587 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8010070 con 0x7f91e009e410 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.715+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e009c700 con 0x7f91e009e410 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.715+0000 7f91d67fc700 1 -- 172.21.15.89:0/2476591587 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8003a20 con 0x7f91e009e410 2023-09-30T22:17:29.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.715+0000 7f91d67fc700 1 -- 172.21.15.89:0/2476591587 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8003db0 con 0x7f91e009e410 2023-09-30T22:17:29.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.715+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0125940 con 0x7f91e009e410 2023-09-30T22:17:29.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.716+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e0074ea0 con 0x7f91e009e410 2023-09-30T22:17:29.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.716+0000 7f91d67fc700 1 -- 172.21.15.89:0/2476591587 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f91d8014070 con 0x7f91e009e410 2023-09-30T22:17:29.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.716+0000 7f91d67fc700 1 -- 172.21.15.89:0/2476591587 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f91d803ee60 con 0x7f91e009e410 2023-09-30T22:17:29.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.720+0000 7f91d67fc700 1 -- 172.21.15.89:0/2476591587 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91d8003f10 con 0x7f91e009e410 2023-09-30T22:17:29.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.734+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdd4411a470 con 0x7fdd4409de50 2023-09-30T22:17:29.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.734+0000 7fdd3affd700 1 -- 172.21.15.89:0/2048313490 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd340233d0 con 0x7fdd4409de50 2023-09-30T22:17:29.735 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4409de50 msgr2=0x7fdd4411d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4409de50 0x7fdd4411d660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd4406dc90 tx=0x7fdd34004750 comp rx=0 tx=0).stop 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 shutdown_connections 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd441338a0 0x7fdd4411e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4409de50 0x7fdd4411d660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 --2- 172.21.15.89:0/2048313490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4409e930 0x7fdd4411dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 >> 172.21.15.89:0/2048313490 conn(0x7fdd440a2aa0 msgr2=0x7fdd441bad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 shutdown_connections 2023-09-30T22:17:29.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.737+0000 7fdd4c3ae700 1 -- 172.21.15.89:0/2048313490 wait complete. 2023-09-30T22:17:29.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.743+0000 7f923b5c6700 1 Processor -- start 2023-09-30T22:17:29.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.744+0000 7f923b5c6700 1 -- start start 2023-09-30T22:17:29.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.744+0000 7f923b5c6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f9234131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.744+0000 7f923b5c6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923409d9f0 0x7f923406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f923b5c6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923409e130 con 0x7f923406acb0 2023-09-30T22:17:29.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f923b5c6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92341312b0 con 0x7f92341316c0 2023-09-30T22:17:29.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f923b5c6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f923409f6e0 con 0x7f923409d9f0 2023-09-30T22:17:29.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9239362700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f9234131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9239362700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f9234131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43574/0 (socket says 172.21.15.89:43574) 2023-09-30T22:17:29.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9239362700 1 -- 172.21.15.89:0/690383271 learned_addr learned my addr 172.21.15.89:0/690383271 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:29.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9238b61700 1 --2- 172.21.15.89:0/690383271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923409d9f0 0x7f923406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9239362700 1 -- 172.21.15.89:0/690383271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923409d9f0 msgr2=0x7f923406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:29.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9239362700 1 --2- 172.21.15.89:0/690383271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923409d9f0 0x7f923406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9239362700 1 -- 172.21.15.89:0/690383271 >> v1:172.21.15.89:6789/0 conn(0x7f923406acb0 legacy=0x7f9234138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9239362700 1 -- 172.21.15.89:0/690383271 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923409e760 con 0x7f92341316c0 2023-09-30T22:17:29.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f9239362700 1 --2- 172.21.15.89:0/690383271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f9234131a90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9230009920 tx=0x7f9230009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22504285fd3e733b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:29.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f922bfff700 1 -- 172.21.15.89:0/690383271 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230011830 con 0x7f92341316c0 2023-09-30T22:17:29.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.745+0000 7f922bfff700 1 -- 172.21.15.89:0/690383271 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9230011990 con 0x7f92341316c0 2023-09-30T22:17:29.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.746+0000 7f922bfff700 1 -- 172.21.15.89:0/690383271 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230011d20 con 0x7f92341316c0 2023-09-30T22:17:29.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.746+0000 7f923b5c6700 1 -- 172.21.15.89:0/690383271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 msgr2=0x7f9234131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.746+0000 7f923b5c6700 1 --2- 172.21.15.89:0/690383271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f9234131a90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9230009920 tx=0x7f9230009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:29.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.746+0000 7f923b5c6700 1 -- 172.21.15.89:0/690383271 shutdown_connections 2023-09-30T22:17:29.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.746+0000 7f923b5c6700 1 --2- 172.21.15.89:0/690383271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f9234131a90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.746+0000 7f923b5c6700 1 --2- 172.21.15.89:0/690383271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923409d9f0 0x7f923406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.747+0000 7f923b5c6700 1 -- 172.21.15.89:0/690383271 >> 172.21.15.89:0/690383271 conn(0x7f9234066f50 msgr2=0x7f923409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.747+0000 7f923b5c6700 1 -- 172.21.15.89:0/690383271 shutdown_connections 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.747+0000 7f923b5c6700 1 -- 172.21.15.89:0/690383271 wait complete. 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f923b5c6700 1 Processor -- start 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f923b5c6700 1 -- start start 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f923b5c6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923406acb0 0x7f9234070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f923b5c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f923409d9f0 0x7f9234070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f923b5c6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f92340714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f923b5c6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92341283d0 con 0x7f923409d9f0 2023-09-30T22:17:29.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f923b5c6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923411b030 con 0x7f92341316c0 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f923b5c6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f923412b900 con 0x7f923406acb0 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f9239362700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923406acb0 0x7f9234070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f9239362700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923406acb0 0x7f9234070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44992/0 (socket says 172.21.15.89:44992) 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f9239362700 1 -- 172.21.15.89:0/685964965 learned_addr learned my addr 172.21.15.89:0/685964965 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f9239b63700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f92340714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.748+0000 7f9238b61700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f923409d9f0 0x7f9234070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9239362700 1 -- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 msgr2=0x7f92340714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9239362700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f92340714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9239362700 1 -- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f923409d9f0 msgr2=0x7f9234070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9239362700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f923409d9f0 0x7f9234070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9239362700 1 -- 172.21.15.89:0/685964965 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230009550 con 0x7f923406acb0 2023-09-30T22:17:29.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9239362700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923406acb0 0x7f9234070a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92300098f0 tx=0x7f9230004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:29.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9229ffb700 1 -- 172.21.15.89:0/685964965 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230004450 con 0x7f923406acb0 2023-09-30T22:17:29.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9229ffb700 1 -- 172.21.15.89:0/685964965 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92300045b0 con 0x7f923406acb0 2023-09-30T22:17:29.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f9229ffb700 1 -- 172.21.15.89:0/685964965 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92300038e0 con 0x7f923406acb0 2023-09-30T22:17:29.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9234072610 con 0x7f923406acb0 2023-09-30T22:17:29.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.749+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9234119480 con 0x7f923406acb0 2023-09-30T22:17:29.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.750+0000 7f9229ffb700 1 -- 172.21.15.89:0/685964965 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9230003a40 con 0x7f923406acb0 2023-09-30T22:17:29.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.750+0000 7f9229ffb700 1 -- 172.21.15.89:0/685964965 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9230040af0 con 0x7f923406acb0 2023-09-30T22:17:29.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.750+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9234074ea0 con 0x7f923406acb0 2023-09-30T22:17:29.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:29.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.755+0000 7f9229ffb700 1 -- 172.21.15.89:0/685964965 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9230015080 con 0x7f923406acb0 2023-09-30T22:17:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.886+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6c08074ea0 con 0x7f6c0813d1f0 2023-09-30T22:17:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.887+0000 7f6beffff700 1 -- 172.21.15.89:0/886101067 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c0002a070 con 0x7f6c0813d1f0 2023-09-30T22:17:29.887 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0813d1f0 msgr2=0x7f6c081cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0813d1f0 0x7f6c081cd440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000efc0 tx=0x7f6c0000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 shutdown_connections 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c0813d1f0 0x7f6c081cd440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0806a770 0x7f6c081cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 --2- 172.21.15.89:0/886101067 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c0809de00 0x7f6c081ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 >> 172.21.15.89:0/886101067 conn(0x7f6c08066f50 msgr2=0x7f6c0813c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 shutdown_connections 2023-09-30T22:17:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.889+0000 7f6c0dffa700 1 -- 172.21.15.89:0/886101067 wait complete. 2023-09-30T22:17:29.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:29.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.916+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f91e006c360 con 0x7f91e009e410 2023-09-30T22:17:29.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.917+0000 7f91d67fc700 1 -- 172.21.15.89:0/2476591587 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f91d8027020 con 0x7f91e009e410 2023-09-30T22:17:29.917 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e410 msgr2=0x7f91e0122e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 --2- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e410 0x7f91e0122e80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009b60 tx=0x7f91d8009e70 comp rx=0 tx=0).stop 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 shutdown_connections 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 --2- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91e009e410 0x7f91e0122e80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 --2- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91e009d6f0 0x7f91e0122400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 --2- 172.21.15.89:0/2476591587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91e009e000 0x7f91e0122940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 >> 172.21.15.89:0/2476591587 conn(0x7f91e0066f50 msgr2=0x7f91e0130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 shutdown_connections 2023-09-30T22:17:29.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.919+0000 7f91e7e94700 1 -- 172.21.15.89:0/2476591587 wait complete. 2023-09-30T22:17:29.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:29.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.950+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f923406c360 con 0x7f923406acb0 2023-09-30T22:17:29.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.950+0000 7f9229ffb700 1 -- 172.21.15.89:0/685964965 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92300243a0 con 0x7f923406acb0 2023-09-30T22:17:29.951 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923406acb0 msgr2=0x7f9234070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923406acb0 0x7f9234070a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92300098f0 tx=0x7f9230004750 comp rx=0 tx=0).stop 2023-09-30T22:17:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 shutdown_connections 2023-09-30T22:17:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92341316c0 0x7f92340714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f923406acb0 0x7f9234070a50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 --2- 172.21.15.89:0/685964965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f923409d9f0 0x7f9234070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 >> 172.21.15.89:0/685964965 conn(0x7f9234066f50 msgr2=0x7f92340a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 shutdown_connections 2023-09-30T22:17:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:29.952+0000 7f923b5c6700 1 -- 172.21.15.89:0/685964965 wait complete. 2023-09-30T22:17:29.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:30.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:30.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c388f3700 1 Processor -- start 2023-09-30T22:17:30.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c388f3700 1 -- start start 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c388f3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c34133420 0x7f3c341337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c388f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c388f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c341304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c388f3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3406b990 con 0x7f3c34133420 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c388f3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c340a0590 con 0x7f3c34134260 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c388f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3409fb80 con 0x7f3c34133e30 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c3259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c34133420 0x7f3c341337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c32d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c341304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c31d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:30.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c3259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c34133420 0x7f3c341337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53842/0 (socket says 172.21.15.89:53842) 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.889+0000 7f3c3259c700 1 -- 172.21.15.89:0/4255284817 learned_addr learned my addr 172.21.15.89:0/4255284817 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c31d9b700 1 -- 172.21.15.89:0/4255284817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 msgr2=0x7f3c341304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c31d9b700 1 --2- 172.21.15.89:0/4255284817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c341304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c31d9b700 1 -- 172.21.15.89:0/4255284817 >> v2:172.21.15.89:3300/0 conn(0x7f3c34133420 msgr2=0x7f3c341337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c31d9b700 1 --2- 172.21.15.89:0/4255284817 >> v2:172.21.15.89:3300/0 conn(0x7f3c34133420 0x7f3c341337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c31d9b700 1 -- 172.21.15.89:0/4255284817 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c340a0030 con 0x7f3c34133e30 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c32d9d700 1 --2- 172.21.15.89:0/4255284817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c341304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c3259c700 1 --2- 172.21.15.89:0/4255284817 >> v2:172.21.15.89:3300/0 conn(0x7f3c34133420 0x7f3c341337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c31d9b700 1 --2- 172.21.15.89:0/4255284817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3412ff90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009840 tx=0x7f3c24009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24809ae65c712139 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c3159a700 1 -- 172.21.15.89:0/4255284817 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24011830 con 0x7f3c34133e30 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c3159a700 1 -- 172.21.15.89:0/4255284817 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c24011990 con 0x7f3c34133e30 2023-09-30T22:17:30.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.890+0000 7f3c3159a700 1 -- 172.21.15.89:0/4255284817 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24011d20 con 0x7f3c34133e30 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 -- 172.21.15.89:0/4255284817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 msgr2=0x7f3c3412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 --2- 172.21.15.89:0/4255284817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3412ff90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009840 tx=0x7f3c24009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 -- 172.21.15.89:0/4255284817 shutdown_connections 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 --2- 172.21.15.89:0/4255284817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c341304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 --2- 172.21.15.89:0/4255284817 >> v2:172.21.15.89:3300/0 conn(0x7f3c34133420 0x7f3c341337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 --2- 172.21.15.89:0/4255284817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3412ff90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 -- 172.21.15.89:0/4255284817 >> 172.21.15.89:0/4255284817 conn(0x7f3c340a4210 msgr2=0x7f3c340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 -- 172.21.15.89:0/4255284817 shutdown_connections 2023-09-30T22:17:30.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.891+0000 7f3c388f3700 1 -- 172.21.15.89:0/4255284817 wait complete. 2023-09-30T22:17:30.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c388f3700 1 Processor -- start 2023-09-30T22:17:30.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c388f3700 1 -- start start 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c388f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c34133420 0x7f3c3409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c388f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c388f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c3409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c388f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3412d950 con 0x7f3c34133420 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c388f3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3411fb90 con 0x7f3c34134260 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c388f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3411f450 con 0x7f3c34133e30 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c3259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c34133420 0x7f3c3409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c3259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c34133420 0x7f3c3409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53844/0 (socket says 172.21.15.89:53844) 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c3259c700 1 -- 172.21.15.89:0/4062353489 learned_addr learned my addr 172.21.15.89:0/4062353489 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:30.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c32d9d700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c3409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.892+0000 7f3c31d9b700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c3259c700 1 -- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 msgr2=0x7f3c3409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c3259c700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c3259c700 1 -- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 msgr2=0x7f3c3409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c3259c700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c3409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c3259c700 1 -- 172.21.15.89:0/4062353489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28009550 con 0x7f3c34133420 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c32d9d700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c3409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c3259c700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c34133420 0x7f3c3409c480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800ebe0 tx=0x7f3c2800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c22ffd700 1 -- 172.21.15.89:0/4062353489 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800c850 con 0x7f3c34133420 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c22ffd700 1 -- 172.21.15.89:0/4062353489 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2800c9b0 con 0x7f3c34133420 2023-09-30T22:17:30.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c22ffd700 1 -- 172.21.15.89:0/4062353489 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800cd40 con 0x7f3c34133420 2023-09-30T22:17:30.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c009a70 con 0x7f3c34133420 2023-09-30T22:17:30.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.893+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c240094a0 con 0x7f3c34133420 2023-09-30T22:17:30.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.894+0000 7f3c22ffd700 1 -- 172.21.15.89:0/4062353489 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c28013470 con 0x7f3c34133420 2023-09-30T22:17:30.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.894+0000 7f3c22ffd700 1 -- 172.21.15.89:0/4062353489 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3c2803f860 con 0x7f3c34133420 2023-09-30T22:17:30.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.894+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c34074ea0 con 0x7f3c34133420 2023-09-30T22:17:30.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:30.899+0000 7f3c22ffd700 1 -- 172.21.15.89:0/4062353489 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c28017080 con 0x7f3c34133420 2023-09-30T22:17:30.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:30.932 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:30.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.033+0000 7fcb87372700 1 Processor -- start 2023-09-30T22:17:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.033+0000 7fcb87372700 1 -- start start 2023-09-30T22:17:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb87372700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009e7e0 0x7fcb8009ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb87372700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb8009fa10 0x7fcb8009f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb87372700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8009b610 con 0x7fcb801316a0 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb87372700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8009f630 con 0x7fcb8009fa10 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb87372700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8009bbb0 con 0x7fcb8009e7e0 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb8490d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb8009fa10 0x7fcb8009f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb8490d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb8009fa10 0x7fcb8009f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53856/0 (socket says 172.21.15.89:53856) 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb8490d700 1 -- 172.21.15.89:0/1942888428 learned_addr learned my addr 172.21.15.89:0/1942888428 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.034+0000 7fcb8510e700 1 --2- 172.21.15.89:0/1942888428 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009e7e0 0x7fcb8009ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.035+0000 7fcb8490d700 1 -- 172.21.15.89:0/1942888428 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009e7e0 msgr2=0x7fcb8009ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.035+0000 7fcb8490d700 1 --2- 172.21.15.89:0/1942888428 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009e7e0 0x7fcb8009ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.035+0000 7fcb8490d700 1 -- 172.21.15.89:0/1942888428 >> v1:172.21.15.89:6789/0 conn(0x7fcb801316a0 legacy=0x7fcb80131b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.035+0000 7fcb8490d700 1 -- 172.21.15.89:0/1942888428 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb800a0010 con 0x7fcb8009fa10 2023-09-30T22:17:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.035+0000 7fcb8490d700 1 --2- 172.21.15.89:0/1942888428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009fa10 0x7fcb8009f0f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcb700098f0 tx=0x7fcb70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd43517cc84bb4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.036+0000 7fcb77fff700 1 -- 172.21.15.89:0/1942888428 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70011830 con 0x7fcb8009fa10 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.036+0000 7fcb77fff700 1 -- 172.21.15.89:0/1942888428 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb70011990 con 0x7fcb8009fa10 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.036+0000 7fcb77fff700 1 -- 172.21.15.89:0/1942888428 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70011d20 con 0x7fcb8009fa10 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.037+0000 7fcb87372700 1 -- 172.21.15.89:0/1942888428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009fa10 msgr2=0x7fcb8009f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.037+0000 7fcb87372700 1 --2- 172.21.15.89:0/1942888428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009fa10 0x7fcb8009f0f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcb700098f0 tx=0x7fcb70009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.037+0000 7fcb87372700 1 -- 172.21.15.89:0/1942888428 shutdown_connections 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.037+0000 7fcb87372700 1 --2- 172.21.15.89:0/1942888428 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009e7e0 0x7fcb8009ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.037+0000 7fcb87372700 1 --2- 172.21.15.89:0/1942888428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009fa10 0x7fcb8009f0f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.037+0000 7fcb87372700 1 -- 172.21.15.89:0/1942888428 >> 172.21.15.89:0/1942888428 conn(0x7fcb800a4300 msgr2=0x7fcb800a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.037+0000 7fcb87372700 1 -- 172.21.15.89:0/1942888428 shutdown_connections 2023-09-30T22:17:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.037+0000 7fcb87372700 1 -- 172.21.15.89:0/1942888428 wait complete. 2023-09-30T22:17:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb87372700 1 Processor -- start 2023-09-30T22:17:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb87372700 1 -- start start 2023-09-30T22:17:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb87372700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009e7e0 0x7fcb8011bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb87372700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009fa10 0x7fcb8011c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb87372700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb801316a0 0x7fcb8011c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb87372700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8011d350 con 0x7fcb8009e7e0 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb87372700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8011dab0 con 0x7fcb801316a0 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb87372700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8011ed00 con 0x7fcb8009fa10 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb8510e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009e7e0 0x7fcb8011bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb8510e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009e7e0 0x7fcb8011bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53868/0 (socket says 172.21.15.89:53868) 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb8510e700 1 -- 172.21.15.89:0/2350012155 learned_addr learned my addr 172.21.15.89:0/2350012155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb8590f700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb801316a0 0x7fcb8011c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.038+0000 7fcb8490d700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009fa10 0x7fcb8011c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb8510e700 1 -- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009fa10 msgr2=0x7fcb8011c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb8510e700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009fa10 0x7fcb8011c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb8510e700 1 -- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb801316a0 msgr2=0x7fcb8011c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb8510e700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb801316a0 0x7fcb8011c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb8510e700 1 -- 172.21.15.89:0/2350012155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb70009550 con 0x7fcb8009e7e0 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb8590f700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb801316a0 0x7fcb8011c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb8510e700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009e7e0 0x7fcb8011bea0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00e9e0 tx=0x7fcb7c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb75ffb700 1 -- 172.21.15.89:0/2350012155 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c009710 con 0x7fcb8009e7e0 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb75ffb700 1 -- 172.21.15.89:0/2350012155 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c00c850 con 0x7fcb8009e7e0 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb75ffb700 1 -- 172.21.15.89:0/2350012155 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00cbc0 con 0x7fcb8009e7e0 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.039+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8011f6e0 con 0x7fcb8009e7e0 2023-09-30T22:17:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.040+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8011f1b0 con 0x7fcb8009e7e0 2023-09-30T22:17:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.041+0000 7fcb75ffb700 1 -- 172.21.15.89:0/2350012155 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb7c00cd20 con 0x7fcb8009e7e0 2023-09-30T22:17:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.041+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb80070f50 con 0x7fcb8009e7e0 2023-09-30T22:17:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.041+0000 7fcb75ffb700 1 -- 172.21.15.89:0/2350012155 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb7c03f720 con 0x7fcb8009e7e0 2023-09-30T22:17:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.049+0000 7fcb75ffb700 1 -- 172.21.15.89:0/2350012155 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb7c013470 con 0x7fcb8009e7e0 2023-09-30T22:17:31.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.085+0000 7f8626084700 1 Processor -- start 2023-09-30T22:17:31.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.085+0000 7f8626084700 1 -- start start 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.085+0000 7f8626084700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862006a770 0x7f862009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.085+0000 7f8626084700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009dc30 0x7f862009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.085+0000 7f8626084700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f862013c1e0 0x7f862013e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.085+0000 7f8626084700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862006ab40 con 0x7f862013c1e0 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.085+0000 7f8626084700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862009ebc0 con 0x7f862006a770 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.085+0000 7f8626084700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f862009a810 con 0x7f862009dc30 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f862013c1e0 0x7f862013e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009dc30 0x7f862009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862006a770 0x7f862009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f862013c1e0 0x7f862013e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53872/0 (socket says 172.21.15.89:53872) 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 -- 172.21.15.89:0/3038702600 learned_addr learned my addr 172.21.15.89:0/3038702600 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 -- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009dc30 msgr2=0x7f862009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 --2- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009dc30 0x7f862009e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 -- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862006a770 msgr2=0x7f862009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 --2- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862006a770 0x7f862009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 -- 172.21.15.89:0/3038702600 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862009e920 con 0x7f862013c1e0 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861effd700 1 --2- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009dc30 0x7f862009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861ffff700 1 --2- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f862013e5c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8618009e50 tx=0x7f861800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc582a360f19ec3c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861e7fc700 1 -- 172.21.15.89:0/3038702600 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800e070 con 0x7f862013c1e0 2023-09-30T22:17:31.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861e7fc700 1 -- 172.21.15.89:0/3038702600 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861800b910 con 0x7f862013c1e0 2023-09-30T22:17:31.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.086+0000 7f861e7fc700 1 -- 172.21.15.89:0/3038702600 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800bcc0 con 0x7f862013c1e0 2023-09-30T22:17:31.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 -- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 msgr2=0x7f862013e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 --2- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f862013e5c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8618009e50 tx=0x7f861800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 -- 172.21.15.89:0/3038702600 shutdown_connections 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 --2- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f862013e5c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 --2- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862006a770 0x7f862009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 --2- 172.21.15.89:0/3038702600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009dc30 0x7f862009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 -- 172.21.15.89:0/3038702600 >> 172.21.15.89:0/3038702600 conn(0x7f8620066f50 msgr2=0x7f862009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 -- 172.21.15.89:0/3038702600 shutdown_connections 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.087+0000 7f8626084700 1 -- 172.21.15.89:0/3038702600 wait complete. 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.089+0000 7f8626084700 1 Processor -- start 2023-09-30T22:17:31.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.089+0000 7f8626084700 1 -- start start 2023-09-30T22:17:31.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.089+0000 7f8626084700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862006a770 0x7f86201cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.089+0000 7f8626084700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009dc30 0x7f86201ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.089+0000 7f8626084700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f8620126380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.089+0000 7f8626084700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8620129aa0 con 0x7f862013c1e0 2023-09-30T22:17:31.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f8626084700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86201cd290 con 0x7f862009dc30 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f8626084700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f862013ad20 con 0x7f862006a770 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f8620126380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f8620126380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53884/0 (socket says 172.21.15.89:53884) 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862006a770 0x7f86201cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 -- 172.21.15.89:0/1227999770 learned_addr learned my addr 172.21.15.89:0/1227999770 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861effd700 1 --2- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009dc30 0x7f86201ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 -- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862006a770 msgr2=0x7f86201cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 --2- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862006a770 0x7f86201cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 -- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009dc30 msgr2=0x7f86201ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 --2- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009dc30 0x7f86201ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 -- 172.21.15.89:0/1227999770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86140094a0 con 0x7f862013c1e0 2023-09-30T22:17:31.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f861ffff700 1 --2- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f8620126380 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8618003f50 tx=0x7f8618004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f8607fff700 1 -- 172.21.15.89:0/1227999770 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618004450 con 0x7f862013c1e0 2023-09-30T22:17:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f8607fff700 1 -- 172.21.15.89:0/1227999770 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86180045b0 con 0x7f862013c1e0 2023-09-30T22:17:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f8607fff700 1 -- 172.21.15.89:0/1227999770 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618004960 con 0x7f862013c1e0 2023-09-30T22:17:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.090+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8618009b30 con 0x7f862013c1e0 2023-09-30T22:17:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.091+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862009f1a0 con 0x7f862013c1e0 2023-09-30T22:17:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.091+0000 7f8607fff700 1 -- 172.21.15.89:0/1227999770 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f861801c410 con 0x7f862013c1e0 2023-09-30T22:17:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.092+0000 7f8607fff700 1 -- 172.21.15.89:0/1227999770 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8618040ff0 con 0x7f862013c1e0 2023-09-30T22:17:31.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.092+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8620074ea0 con 0x7f862013c1e0 2023-09-30T22:17:31.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.097+0000 7f8607fff700 1 -- 172.21.15.89:0/1227999770 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8618029070 con 0x7f862013c1e0 2023-09-30T22:17:31.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.101+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3c3411f030 con 0x7f3c34133420 2023-09-30T22:17:31.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.102+0000 7f3c22ffd700 1 -- 172.21.15.89:0/4062353489 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c28021020 con 0x7f3c34133420 2023-09-30T22:17:31.102 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:31.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c34133420 msgr2=0x7f3c3409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c34133420 0x7f3c3409c480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800ebe0 tx=0x7f3c2800eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 shutdown_connections 2023-09-30T22:17:31.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c34134260 0x7f3c3409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c34133420 0x7f3c3409c480 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 --2- 172.21.15.89:0/4062353489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c34133e30 0x7f3c3409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 >> 172.21.15.89:0/4062353489 conn(0x7f3c340a4210 msgr2=0x7f3c341302c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:31.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 shutdown_connections 2023-09-30T22:17:31.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.104+0000 7f3c388f3700 1 -- 172.21.15.89:0/4062353489 wait complete. 2023-09-30T22:17:31.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.142+0000 7f6952eb1700 1 Processor -- start 2023-09-30T22:17:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.142+0000 7f6952eb1700 1 -- start start 2023-09-30T22:17:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6952eb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c0a0a30 0x7f694c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6952eb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f694c138d80 0x7f694c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6952eb1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694c0a2a70 con 0x7f694c09d630 2023-09-30T22:17:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6952eb1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c0a01d0 con 0x7f694c138d80 2023-09-30T22:17:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6952eb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f694c09e2b0 con 0x7f694c0a0a30 2023-09-30T22:17:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6950c4d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c0a0a30 0x7f694c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6950c4d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c0a0a30 0x7f694c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48446/0 (socket says 172.21.15.89:48446) 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f694bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f694c09d630 0x7f694c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57364/0 (socket says 172.21.15.89:57364) 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6950c4d700 1 -- 172.21.15.89:0/4048161412 learned_addr learned my addr 172.21.15.89:0/4048161412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f695144e700 1 --2- 172.21.15.89:0/4048161412 >> v2:172.21.15.89:3300/0 conn(0x7f694c138d80 0x7f694c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f695144e700 1 -- 172.21.15.89:0/4048161412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c0a0a30 msgr2=0x7f694c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f695144e700 1 --2- 172.21.15.89:0/4048161412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c0a0a30 0x7f694c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f695144e700 1 -- 172.21.15.89:0/4048161412 >> v1:172.21.15.89:6789/0 conn(0x7f694c09d630 legacy=0x7f694c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f695144e700 1 -- 172.21.15.89:0/4048161412 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c09dc50 con 0x7f694c138d80 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.143+0000 7f6950c4d700 1 --2- 172.21.15.89:0/4048161412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c0a0a30 0x7f694c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f695144e700 1 --2- 172.21.15.89:0/4048161412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 0x7f694c13b160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6944009d90 tx=0x7f694400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc4d851299c148d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f694b7fe700 1 -- 172.21.15.89:0/4048161412 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694400e070 con 0x7f694c138d80 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f694b7fe700 1 -- 172.21.15.89:0/4048161412 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694400bb20 con 0x7f694c138d80 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f694b7fe700 1 -- 172.21.15.89:0/4048161412 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6944012430 con 0x7f694c138d80 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f6952eb1700 1 -- 172.21.15.89:0/4048161412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 msgr2=0x7f694c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f6952eb1700 1 --2- 172.21.15.89:0/4048161412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 0x7f694c13b160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6944009d90 tx=0x7f694400b330 comp rx=0 tx=0).stop 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f6952eb1700 1 -- 172.21.15.89:0/4048161412 shutdown_connections 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f6952eb1700 1 --2- 172.21.15.89:0/4048161412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 0x7f694c13b160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f6952eb1700 1 --2- 172.21.15.89:0/4048161412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c0a0a30 0x7f694c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f6952eb1700 1 -- 172.21.15.89:0/4048161412 >> 172.21.15.89:0/4048161412 conn(0x7f694c0a3050 msgr2=0x7f694c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f6952eb1700 1 -- 172.21.15.89:0/4048161412 shutdown_connections 2023-09-30T22:17:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.144+0000 7f6952eb1700 1 -- 172.21.15.89:0/4048161412 wait complete. 2023-09-30T22:17:31.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.145+0000 7f6952eb1700 1 Processor -- start 2023-09-30T22:17:31.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.145+0000 7f6952eb1700 1 -- start start 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6952eb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c09d630 0x7f694c133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6952eb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f694c0a0a30 0x7f694c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6952eb1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 0x7f694c134310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6952eb1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c1371b0 con 0x7f694c138d80 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6952eb1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c138030 con 0x7f694c0a0a30 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6952eb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f694c1373b0 con 0x7f694c09d630 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6950c4d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c09d630 0x7f694c133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6950c4d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c09d630 0x7f694c133890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48452/0 (socket says 172.21.15.89:48452) 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6950c4d700 1 -- 172.21.15.89:0/3650941479 learned_addr learned my addr 172.21.15.89:0/3650941479 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f695144e700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 0x7f694c134310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f694bfff700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f694c0a0a30 0x7f694c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6950c4d700 1 -- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f694c0a0a30 msgr2=0x7f694c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6950c4d700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f694c0a0a30 0x7f694c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6950c4d700 1 -- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 msgr2=0x7f694c134310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6950c4d700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 0x7f694c134310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f6950c4d700 1 -- 172.21.15.89:0/3650941479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940009550 con 0x7f694c09d630 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.146+0000 7f695144e700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 0x7f694c134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.147+0000 7f6950c4d700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c09d630 0x7f694c133890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f694000ec10 tx=0x7f694000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.147+0000 7f69497fa700 1 -- 172.21.15.89:0/3650941479 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694000c850 con 0x7f694c09d630 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.147+0000 7f69497fa700 1 -- 172.21.15.89:0/3650941479 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694000c9b0 con 0x7f694c09d630 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.147+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6944009a70 con 0x7f694c09d630 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.147+0000 7f69497fa700 1 -- 172.21.15.89:0/3650941479 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694000cd40 con 0x7f694c09d630 2023-09-30T22:17:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.147+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c0a4eb0 con 0x7f694c09d630 2023-09-30T22:17:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.148+0000 7f69497fa700 1 -- 172.21.15.89:0/3650941479 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6940013470 con 0x7f694c09d630 2023-09-30T22:17:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.148+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c0747f0 con 0x7f694c09d630 2023-09-30T22:17:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.148+0000 7f69497fa700 1 -- 172.21.15.89:0/3650941479 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f694003f830 con 0x7f694c09d630 2023-09-30T22:17:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.157+0000 7f69497fa700 1 -- 172.21.15.89:0/3650941479 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6940013720 con 0x7f694c09d630 2023-09-30T22:17:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.256+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcb8011f150 con 0x7fcb8009e7e0 2023-09-30T22:17:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.256+0000 7fcb75ffb700 1 -- 172.21.15.89:0/2350012155 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb7c021070 con 0x7fcb8009e7e0 2023-09-30T22:17:31.257 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009e7e0 msgr2=0x7fcb8011bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009e7e0 0x7fcb8011bea0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00e9e0 tx=0x7fcb7c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 shutdown_connections 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb801316a0 0x7fcb8011c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8009e7e0 0x7fcb8011bea0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 --2- 172.21.15.89:0/2350012155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8009fa10 0x7fcb8011c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 >> 172.21.15.89:0/2350012155 conn(0x7fcb800a4300 msgr2=0x7fcb80130e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 shutdown_connections 2023-09-30T22:17:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.259+0000 7fcb87372700 1 -- 172.21.15.89:0/2350012155 wait complete. 2023-09-30T22:17:31.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:31.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.313+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86201c1440 con 0x7f862013c1e0 2023-09-30T22:17:31.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.313+0000 7f8607fff700 1 -- 172.21.15.89:0/1227999770 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f861801c6c0 con 0x7f862013c1e0 2023-09-30T22:17:31.314 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:31.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 msgr2=0x7f8620126380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 --2- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f8620126380 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8618003f50 tx=0x7f8618004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 shutdown_connections 2023-09-30T22:17:31.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 --2- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862013c1e0 0x7f8620126380 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 --2- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862006a770 0x7f86201cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 --2- 172.21.15.89:0/1227999770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009dc30 0x7f86201ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 >> 172.21.15.89:0/1227999770 conn(0x7f8620066f50 msgr2=0x7f862013b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:31.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 shutdown_connections 2023-09-30T22:17:31.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.315+0000 7f8626084700 1 -- 172.21.15.89:0/1227999770 wait complete. 2023-09-30T22:17:31.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.368+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f694c136bc0 con 0x7f694c09d630 2023-09-30T22:17:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.368+0000 7f69497fa700 1 -- 172.21.15.89:0/3650941479 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6940021070 con 0x7f694c09d630 2023-09-30T22:17:31.369 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.370+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c09d630 msgr2=0x7f694c133890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.370+0000 7f6952eb1700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c09d630 0x7f694c133890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f694000ec10 tx=0x7f694000ef20 comp rx=0 tx=0).stop 2023-09-30T22:17:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.371+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 shutdown_connections 2023-09-30T22:17:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.371+0000 7f6952eb1700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f694c138d80 0x7f694c134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.371+0000 7f6952eb1700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694c09d630 0x7f694c133890 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.371+0000 7f6952eb1700 1 --2- 172.21.15.89:0/3650941479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f694c0a0a30 0x7f694c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.371+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 >> 172.21.15.89:0/3650941479 conn(0x7f694c0a3050 msgr2=0x7f694c09f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.371+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 shutdown_connections 2023-09-30T22:17:31.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:31.371+0000 7f6952eb1700 1 -- 172.21.15.89:0/3650941479 wait complete. 2023-09-30T22:17:31.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:32.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.243+0000 7f856a64d700 1 Processor -- start 2023-09-30T22:17:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.243+0000 7f856a64d700 1 -- start start 2023-09-30T22:17:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f856a64d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f856409e1d0 0x7f8564135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f856a64d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8564135e50 0x7f8564138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f856a64d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f856409f130 con 0x7f856409d6f0 2023-09-30T22:17:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f856a64d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f856409f520 con 0x7f856409e1d0 2023-09-30T22:17:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f856a64d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f856409e930 con 0x7f8564135e50 2023-09-30T22:17:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f85637fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f856409e1d0 0x7f8564135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f8568bea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8564135e50 0x7f8564138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f85637fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f856409e1d0 0x7f8564135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53920/0 (socket says 172.21.15.89:53920) 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f8568bea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8564135e50 0x7f8564138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48456/0 (socket says 172.21.15.89:48456) 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f85637fe700 1 -- 172.21.15.89:0/1797674089 learned_addr learned my addr 172.21.15.89:0/1797674089 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f85637fe700 1 -- 172.21.15.89:0/1797674089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8564135e50 msgr2=0x7f8564138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f85637fe700 1 --2- 172.21.15.89:0/1797674089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8564135e50 0x7f8564138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f85637fe700 1 -- 172.21.15.89:0/1797674089 >> v1:172.21.15.89:6789/0 conn(0x7f856409d6f0 legacy=0x7f856409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f85637fe700 1 -- 172.21.15.89:0/1797674089 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856409b2d0 con 0x7f856409e1d0 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f8562ffd700 1 -- 172.21.15.89:0/1797674089 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589260424 0 0) 0x7f856409f130 con 0x7f856409d6f0 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f8568bea700 1 --2- 172.21.15.89:0/1797674089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8564135e50 0x7f8564138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f85637fe700 1 --2- 172.21.15.89:0/1797674089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f8564135910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f855400bb90 tx=0x7f855400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fe7327067e0bb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f8562ffd700 1 -- 172.21.15.89:0/1797674089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400dd30 con 0x7f856409e1d0 2023-09-30T22:17:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f8562ffd700 1 -- 172.21.15.89:0/1797674089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855400de90 con 0x7f856409e1d0 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.244+0000 7f8562ffd700 1 -- 172.21.15.89:0/1797674089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554010690 con 0x7f856409e1d0 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.245+0000 7f856a64d700 1 -- 172.21.15.89:0/1797674089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 msgr2=0x7f8564135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.245+0000 7f856a64d700 1 --2- 172.21.15.89:0/1797674089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f8564135910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f855400bb90 tx=0x7f855400bea0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.245+0000 7f856a64d700 1 -- 172.21.15.89:0/1797674089 shutdown_connections 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.245+0000 7f856a64d700 1 --2- 172.21.15.89:0/1797674089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8564135e50 0x7f8564138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.245+0000 7f856a64d700 1 --2- 172.21.15.89:0/1797674089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f8564135910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.245+0000 7f856a64d700 1 -- 172.21.15.89:0/1797674089 >> 172.21.15.89:0/1797674089 conn(0x7f8564066f50 msgr2=0x7f856409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.245+0000 7f856a64d700 1 -- 172.21.15.89:0/1797674089 shutdown_connections 2023-09-30T22:17:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.245+0000 7f856a64d700 1 -- 172.21.15.89:0/1797674089 wait complete. 2023-09-30T22:17:32.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f856a64d700 1 Processor -- start 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f856a64d700 1 -- start start 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f856a64d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f856409d6f0 0x7f85641c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f856a64d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f85641c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f856a64d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8564135e50 0x7f85641196e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f856a64d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856411ce40 con 0x7f856409e1d0 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f856a64d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856411b6a0 con 0x7f8564135e50 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f856a64d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f856411b0c0 con 0x7f856409d6f0 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f8563fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f856409d6f0 0x7f85641c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.246+0000 7f85637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f85641c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f85637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f85641c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53922/0 (socket says 172.21.15.89:53922) 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f8563fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f856409d6f0 0x7f85641c4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48472/0 (socket says 172.21.15.89:48472) 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f85637fe700 1 -- 172.21.15.89:0/3622538430 learned_addr learned my addr 172.21.15.89:0/3622538430 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f8568bea700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8564135e50 0x7f85641196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f85637fe700 1 -- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f856409d6f0 msgr2=0x7f85641c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f85637fe700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f856409d6f0 0x7f85641c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f85637fe700 1 -- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8564135e50 msgr2=0x7f85641196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f85637fe700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8564135e50 0x7f85641196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f85637fe700 1 -- 172.21.15.89:0/3622538430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c009a70 con 0x7f856409e1d0 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f8563fff700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f856409d6f0 0x7f85641c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f85637fe700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f85641c4770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f856409b9e0 tx=0x7f8554004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:32.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f8560ff9700 1 -- 172.21.15.89:0/3622538430 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855401f070 con 0x7f856409e1d0 2023-09-30T22:17:32.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f8560ff9700 1 -- 172.21.15.89:0/3622538430 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85540038b0 con 0x7f856409e1d0 2023-09-30T22:17:32.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855400b7f0 con 0x7f856409e1d0 2023-09-30T22:17:32.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f8560ff9700 1 -- 172.21.15.89:0/3622538430 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554003c20 con 0x7f856409e1d0 2023-09-30T22:17:32.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.247+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856411d7d0 con 0x7f856409e1d0 2023-09-30T22:17:32.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.248+0000 7f8560ff9700 1 -- 172.21.15.89:0/3622538430 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8554019400 con 0x7f856409e1d0 2023-09-30T22:17:32.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.248+0000 7f8560ff9700 1 -- 172.21.15.89:0/3622538430 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8554042a30 con 0x7f856409e1d0 2023-09-30T22:17:32.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.248+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85640a2a60 con 0x7f856409e1d0 2023-09-30T22:17:32.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.253+0000 7f8560ff9700 1 -- 172.21.15.89:0/3622538430 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f855400e360 con 0x7f856409e1d0 2023-09-30T22:17:32.279 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:32.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:32.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe49da3700 1 Processor -- start 2023-09-30T22:17:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe49da3700 1 -- start start 2023-09-30T22:17:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe49da3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe44130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe49da3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe44130e50 0x7efe4406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe49da3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe4406fb20 0x7efe4406ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe49da3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe440704b0 con 0x7efe44130e50 2023-09-30T22:17:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe49da3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4409a390 con 0x7efe44130540 2023-09-30T22:17:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe49da3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe440716a0 con 0x7efe4406fb20 2023-09-30T22:17:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe42ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe44130e50 0x7efe4406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe42ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe44130e50 0x7efe4406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53930/0 (socket says 172.21.15.89:53930) 2023-09-30T22:17:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe42ffd700 1 -- 172.21.15.89:0/1555517601 learned_addr learned my addr 172.21.15.89:0/1555517601 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.405+0000 7efe437fe700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe44130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe43fff700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe4406fb20 0x7efe4406ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe42ffd700 1 -- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe4406fb20 msgr2=0x7efe4406ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe42ffd700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe4406fb20 0x7efe4406ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe42ffd700 1 -- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 msgr2=0x7efe44130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe42ffd700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe44130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe42ffd700 1 -- 172.21.15.89:0/1555517601 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe44071290 con 0x7efe44130e50 2023-09-30T22:17:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe437fe700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe44130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe42ffd700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe44130e50 0x7efe4406f5e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe38009920 tx=0x7efe38009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d85c3cbfbdd637f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe427fc700 1 -- 172.21.15.89:0/1555517601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38011830 con 0x7efe44130e50 2023-09-30T22:17:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe427fc700 1 -- 172.21.15.89:0/1555517601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38011990 con 0x7efe44130e50 2023-09-30T22:17:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.406+0000 7efe427fc700 1 -- 172.21.15.89:0/1555517601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38011d20 con 0x7efe44130e50 2023-09-30T22:17:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.407+0000 7efe49da3700 1 -- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe44130e50 msgr2=0x7efe4406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.407+0000 7efe49da3700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe44130e50 0x7efe4406f5e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe38009920 tx=0x7efe38009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.409+0000 7efe49da3700 1 -- 172.21.15.89:0/1555517601 shutdown_connections 2023-09-30T22:17:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.409+0000 7efe49da3700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe4406fb20 0x7efe4406ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.409+0000 7efe49da3700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe44130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.409+0000 7efe49da3700 1 --2- 172.21.15.89:0/1555517601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe44130e50 0x7efe4406f5e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.409+0000 7efe49da3700 1 -- 172.21.15.89:0/1555517601 >> 172.21.15.89:0/1555517601 conn(0x7efe440a2aa0 msgr2=0x7efe440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.409+0000 7efe49da3700 1 -- 172.21.15.89:0/1555517601 shutdown_connections 2023-09-30T22:17:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.409+0000 7efe49da3700 1 -- 172.21.15.89:0/1555517601 wait complete. 2023-09-30T22:17:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.410+0000 7efe49da3700 1 Processor -- start 2023-09-30T22:17:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.410+0000 7efe49da3700 1 -- start start 2023-09-30T22:17:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe49da3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4406fb20 0x7efe4406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe49da3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe4406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe49da3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe44130e50 0x7efe4406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe49da3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe44128390 con 0x7efe4406fb20 2023-09-30T22:17:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe49da3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440a1ef0 con 0x7efe44130540 2023-09-30T22:17:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe49da3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe440718e0 con 0x7efe44130e50 2023-09-30T22:17:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe42ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe4406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe42ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe4406ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43774/0 (socket says 172.21.15.89:43774) 2023-09-30T22:17:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe42ffd700 1 -- 172.21.15.89:0/3901621687 learned_addr learned my addr 172.21.15.89:0/3901621687 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe437fe700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4406fb20 0x7efe4406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.411+0000 7efe43fff700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe44130e50 0x7efe4406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe42ffd700 1 -- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe44130e50 msgr2=0x7efe4406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe42ffd700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe44130e50 0x7efe4406ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe42ffd700 1 -- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4406fb20 msgr2=0x7efe4406e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe42ffd700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4406fb20 0x7efe4406e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe42ffd700 1 -- 172.21.15.89:0/3901621687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340094a0 con 0x7efe44130540 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe437fe700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4406fb20 0x7efe4406e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe42ffd700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe4406ea10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe38000c00 tx=0x7efe38004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe2bfff700 1 -- 172.21.15.89:0/3901621687 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38004590 con 0x7efe44130540 2023-09-30T22:17:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe2bfff700 1 -- 172.21.15.89:0/3901621687 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe380036a0 con 0x7efe44130540 2023-09-30T22:17:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe38009550 con 0x7efe44130540 2023-09-30T22:17:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe2bfff700 1 -- 172.21.15.89:0/3901621687 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38003a50 con 0x7efe44130540 2023-09-30T22:17:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.412+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4409e110 con 0x7efe44130540 2023-09-30T22:17:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.413+0000 7efe2bfff700 1 -- 172.21.15.89:0/3901621687 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efe38003bb0 con 0x7efe44130540 2023-09-30T22:17:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.413+0000 7efe2bfff700 1 -- 172.21.15.89:0/3901621687 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efe38040b20 con 0x7efe44130540 2023-09-30T22:17:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.413+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe44074ea0 con 0x7efe44130540 2023-09-30T22:17:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.418+0000 7efe2bfff700 1 -- 172.21.15.89:0/3901621687 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efe38003e60 con 0x7efe44130540 2023-09-30T22:17:32.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.458+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8564074ea0 con 0x7f856409e1d0 2023-09-30T22:17:32.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.459+0000 7f8560ff9700 1 -- 172.21.15.89:0/3622538430 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8554029030 con 0x7f856409e1d0 2023-09-30T22:17:32.460 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:32.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 msgr2=0x7f85641c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f85641c4770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f856409b9e0 tx=0x7f8554004750 comp rx=0 tx=0).stop 2023-09-30T22:17:32.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 shutdown_connections 2023-09-30T22:17:32.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8564135e50 0x7f85641196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f856409d6f0 0x7f85641c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 --2- 172.21.15.89:0/3622538430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f856409e1d0 0x7f85641c4770 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 >> 172.21.15.89:0/3622538430 conn(0x7f8564066f50 msgr2=0x7f85641327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:32.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 shutdown_connections 2023-09-30T22:17:32.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.461+0000 7f856a64d700 1 -- 172.21.15.89:0/3622538430 wait complete. 2023-09-30T22:17:32.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.472+0000 7fcd8360c700 1 Processor -- start 2023-09-30T22:17:32.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd8360c700 1 -- start start 2023-09-30T22:17:32.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd8360c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd8360c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 0x7fcd7c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd8360c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c09f130 con 0x7fcd7c09e000 2023-09-30T22:17:32.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd8360c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c09f520 con 0x7fcd7c09e410 2023-09-30T22:17:32.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd8360c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c09e930 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd813a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd80ba7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcd7c09e000 0x7fcd7c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57382/0 (socket says 172.21.15.89:57382) 2023-09-30T22:17:32.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd80ba7700 1 -- 172.21.15.89:0/2664601334 learned_addr learned my addr 172.21.15.89:0/2664601334 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.473+0000 7fcd813a8700 1 --2- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48504/0 (socket says 172.21.15.89:48504) 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd81ba9700 1 --2- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 0x7fcd7c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd813a8700 1 -- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 msgr2=0x7fcd7c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd813a8700 1 --2- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 0x7fcd7c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd813a8700 1 -- 172.21.15.89:0/2664601334 >> v1:172.21.15.89:6789/0 conn(0x7fcd7c09e000 legacy=0x7fcd7c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd813a8700 1 -- 172.21.15.89:0/2664601334 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c09b2d0 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd813a8700 1 --2- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800bb10 tx=0x7fcd7800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba94fadcab726776 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd73fff700 1 -- 172.21.15.89:0/2664601334 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7800dc70 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd73fff700 1 -- 172.21.15.89:0/2664601334 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7800ddd0 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.474+0000 7fcd73fff700 1 -- 172.21.15.89:0/2664601334 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78010690 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.475+0000 7fcd8360c700 1 -- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 msgr2=0x7fcd7c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.475+0000 7fcd8360c700 1 --2- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800bb10 tx=0x7fcd7800be20 comp rx=0 tx=0).stop 2023-09-30T22:17:32.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.475+0000 7fcd8360c700 1 -- 172.21.15.89:0/2664601334 shutdown_connections 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.475+0000 7fcd8360c700 1 --2- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 0x7fcd7c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.475+0000 7fcd8360c700 1 --2- 172.21.15.89:0/2664601334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c09dac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.475+0000 7fcd8360c700 1 -- 172.21.15.89:0/2664601334 >> 172.21.15.89:0/2664601334 conn(0x7fcd7c066f50 msgr2=0x7fcd7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.475+0000 7fcd8360c700 1 -- 172.21.15.89:0/2664601334 shutdown_connections 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.475+0000 7fcd8360c700 1 -- 172.21.15.89:0/2664601334 wait complete. 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.476+0000 7fcd8360c700 1 Processor -- start 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.476+0000 7fcd8360c700 1 -- start start 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.476+0000 7fcd8360c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c11df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.476+0000 7fcd8360c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c09e000 0x7fcd7c11e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.476+0000 7fcd8360c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 0x7fcd7c11e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.476+0000 7fcd8360c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c11f3f0 con 0x7fcd7c09e000 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.476+0000 7fcd8360c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c130300 con 0x7fcd7c09e410 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.476+0000 7fcd8360c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c120d80 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd80ba7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c09e000 0x7fcd7c11e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd80ba7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c09e000 0x7fcd7c11e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53950/0 (socket says 172.21.15.89:53950) 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd80ba7700 1 -- 172.21.15.89:0/497291456 learned_addr learned my addr 172.21.15.89:0/497291456 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd813a8700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c11df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd81ba9700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 0x7fcd7c11e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd813a8700 1 -- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 msgr2=0x7fcd7c11e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd813a8700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 0x7fcd7c11e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd813a8700 1 -- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c09e000 msgr2=0x7fcd7c11e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd813a8700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c09e000 0x7fcd7c11e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd813a8700 1 -- 172.21.15.89:0/497291456 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7800b740 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd813a8700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c11df20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd78018000 tx=0x7fcd78004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd71ffb700 1 -- 172.21.15.89:0/497291456 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7801d070 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd71ffb700 1 -- 172.21.15.89:0/497291456 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7801e040 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.477+0000 7fcd71ffb700 1 -- 172.21.15.89:0/497291456 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd780106b0 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.478+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c121760 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.478+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c120150 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.479+0000 7fcd71ffb700 1 -- 172.21.15.89:0/497291456 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd78023460 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.479+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c0a1d60 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.479+0000 7fcd71ffb700 1 -- 172.21.15.89:0/497291456 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcd780433f0 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.484+0000 7fcd71ffb700 1 -- 172.21.15.89:0/497291456 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd7802b730 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.515+0000 7fc2cb341700 1 Processor -- start 2023-09-30T22:17:32.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2cb341700 1 -- start start 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2cb341700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409e930 0x7fc2c409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2cb341700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2cb341700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c4070940 con 0x7fc2c409f650 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2cb341700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c406fd60 con 0x7fc2c409f240 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2cb341700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c406fa80 con 0x7fc2c409e930 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2c90dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409e930 0x7fc2c409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2c88dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2c90dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409e930 0x7fc2c409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48526/0 (socket says 172.21.15.89:48526) 2023-09-30T22:17:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2c88dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43810/0 (socket says 172.21.15.89:43810) 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.516+0000 7fc2c90dd700 1 -- 172.21.15.89:0/670478784 learned_addr learned my addr 172.21.15.89:0/670478784 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2c90dd700 1 -- 172.21.15.89:0/670478784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 msgr2=0x7fc2c41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2c90dd700 1 --2- 172.21.15.89:0/670478784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2c90dd700 1 -- 172.21.15.89:0/670478784 >> v1:172.21.15.89:6789/0 conn(0x7fc2c409f650 legacy=0x7fc2c4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2c90dd700 1 -- 172.21.15.89:0/670478784 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c409fb60 con 0x7fc2c409e930 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2bbfff700 1 -- 172.21.15.89:0/670478784 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530778275 0 0) 0x7fc2c4070940 con 0x7fc2c409f650 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2c88dc700 1 --2- 172.21.15.89:0/670478784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2c90dd700 1 --2- 172.21.15.89:0/670478784 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409e930 0x7fc2c409ed00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009920 tx=0x7fc2c0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0d2a11d855bda1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2bbfff700 1 -- 172.21.15.89:0/670478784 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011830 con 0x7fc2c409e930 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2bbfff700 1 -- 172.21.15.89:0/670478784 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0011990 con 0x7fc2c409e930 2023-09-30T22:17:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.517+0000 7fc2bbfff700 1 -- 172.21.15.89:0/670478784 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011d20 con 0x7fc2c409e930 2023-09-30T22:17:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.518+0000 7fc2cb341700 1 -- 172.21.15.89:0/670478784 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409e930 msgr2=0x7fc2c409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.518+0000 7fc2cb341700 1 --2- 172.21.15.89:0/670478784 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409e930 0x7fc2c409ed00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009920 tx=0x7fc2c0009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.518+0000 7fc2cb341700 1 -- 172.21.15.89:0/670478784 shutdown_connections 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.518+0000 7fc2cb341700 1 --2- 172.21.15.89:0/670478784 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409e930 0x7fc2c409ed00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.518+0000 7fc2cb341700 1 --2- 172.21.15.89:0/670478784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.518+0000 7fc2cb341700 1 -- 172.21.15.89:0/670478784 >> 172.21.15.89:0/670478784 conn(0x7fc2c40a4210 msgr2=0x7fc2c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.518+0000 7fc2cb341700 1 -- 172.21.15.89:0/670478784 shutdown_connections 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.518+0000 7fc2cb341700 1 -- 172.21.15.89:0/670478784 wait complete. 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2cb341700 1 Processor -- start 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2cb341700 1 -- start start 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2cb341700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2c409e930 0x7fc2c41a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2cb341700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2cb341700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409f650 0x7fc2c41a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2cb341700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41a51b0 con 0x7fc2c409e930 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2cb341700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41a5870 con 0x7fc2c409f240 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2cb341700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41a6ac0 con 0x7fc2c409f650 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2c88dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2c90dd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2c409e930 0x7fc2c41a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2c88dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43822/0 (socket says 172.21.15.89:43822) 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2c88dc700 1 -- 172.21.15.89:0/210546707 learned_addr learned my addr 172.21.15.89:0/210546707 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.519+0000 7fc2c90dd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2c409e930 0x7fc2c41a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53962/0 (socket says 172.21.15.89:53962) 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2c88dc700 1 -- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409f650 msgr2=0x7fc2c41a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2c88dc700 1 --2- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409f650 0x7fc2c41a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2c88dc700 1 -- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2c409e930 msgr2=0x7fc2c41a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2c88dc700 1 --2- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2c409e930 0x7fc2c41a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2c88dc700 1 -- 172.21.15.89:0/210546707 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40094a0 con 0x7fc2c409f240 2023-09-30T22:17:32.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2c88dc700 1 --2- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41a4240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400ebb0 tx=0x7fc2b400eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:32.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2b9ffb700 1 -- 172.21.15.89:0/210546707 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400c850 con 0x7fc2c409f240 2023-09-30T22:17:32.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2b9ffb700 1 -- 172.21.15.89:0/210546707 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b400c9b0 con 0x7fc2c409f240 2023-09-30T22:17:32.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2b9ffb700 1 -- 172.21.15.89:0/210546707 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400cd40 con 0x7fc2c409f240 2023-09-30T22:17:32.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c0009550 con 0x7fc2c409f240 2023-09-30T22:17:32.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.520+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c40a0190 con 0x7fc2c409f240 2023-09-30T22:17:32.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.521+0000 7fc2b9ffb700 1 -- 172.21.15.89:0/210546707 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc2b4013470 con 0x7fc2c409f240 2023-09-30T22:17:32.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.521+0000 7fc2b9ffb700 1 -- 172.21.15.89:0/210546707 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc2b403f890 con 0x7fc2c409f240 2023-09-30T22:17:32.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.522+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c41a0b30 con 0x7fc2c409f240 2023-09-30T22:17:32.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.526+0000 7fc2b9ffb700 1 -- 172.21.15.89:0/210546707 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2b4017080 con 0x7fc2c409f240 2023-09-30T22:17:32.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.661+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efe4406b9e0 con 0x7efe44130540 2023-09-30T22:17:32.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.661+0000 7efe2bfff700 1 -- 172.21.15.89:0/3901621687 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efe38015030 con 0x7efe44130540 2023-09-30T22:17:32.662 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:32.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 msgr2=0x7efe4406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe4406ea10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe38000c00 tx=0x7efe38004750 comp rx=0 tx=0).stop 2023-09-30T22:17:32.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 shutdown_connections 2023-09-30T22:17:32.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe44130e50 0x7efe4406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4406fb20 0x7efe4406e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 --2- 172.21.15.89:0/3901621687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe44130540 0x7efe4406ea10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 >> 172.21.15.89:0/3901621687 conn(0x7efe440a2aa0 msgr2=0x7efe4412d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:32.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 shutdown_connections 2023-09-30T22:17:32.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.663+0000 7efe49da3700 1 -- 172.21.15.89:0/3901621687 wait complete. 2023-09-30T22:17:32.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:32.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.679+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcd7c11ac20 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.679+0000 7fcd71ffb700 1 -- 172.21.15.89:0/497291456 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd7800f3d0 con 0x7fcd7c09d6f0 2023-09-30T22:17:32.680 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:32.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 msgr2=0x7fcd7c11df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c11df20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd78018000 tx=0x7fcd78004750 comp rx=0 tx=0).stop 2023-09-30T22:17:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 shutdown_connections 2023-09-30T22:17:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd7c09e410 0x7fcd7c11e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd7c09d6f0 0x7fcd7c11df20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 --2- 172.21.15.89:0/497291456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd7c09e000 0x7fcd7c11e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 >> 172.21.15.89:0/497291456 conn(0x7fcd7c066f50 msgr2=0x7fcd7c130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 shutdown_connections 2023-09-30T22:17:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.681+0000 7fcd8360c700 1 -- 172.21.15.89:0/497291456 wait complete. 2023-09-30T22:17:32.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:32.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.736+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc2c40a34a0 con 0x7fc2c409f240 2023-09-30T22:17:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.737+0000 7fc2b9ffb700 1 -- 172.21.15.89:0/210546707 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2b401b3b0 con 0x7fc2c409f240 2023-09-30T22:17:32.738 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 msgr2=0x7fc2c41a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 --2- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41a4240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400ebb0 tx=0x7fc2b400eec0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 shutdown_connections 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 --2- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2c409f650 0x7fc2c41a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 --2- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2c409e930 0x7fc2c41a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 --2- 172.21.15.89:0/210546707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2c409f240 0x7fc2c41a4240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 >> 172.21.15.89:0/210546707 conn(0x7fc2c40a4210 msgr2=0x7fc2c41b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 shutdown_connections 2023-09-30T22:17:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:32.739+0000 7fc2cb341700 1 -- 172.21.15.89:0/210546707 wait complete. 2023-09-30T22:17:32.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:33.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.618+0000 7f4286e21700 1 Processor -- start 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.618+0000 7f4286e21700 1 -- start start 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.618+0000 7f4286e21700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f428009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.618+0000 7f4286e21700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f428009e410 0x7f4280134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.618+0000 7f4286e21700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f428009b120 con 0x7f428009e000 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.618+0000 7f4286e21700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f428006b460 con 0x7f428009e410 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.618+0000 7f4286e21700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f428009a7e0 con 0x7f428009d6f0 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f4284bbd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f428009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f42853be700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f428009e410 0x7f4280134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f4284bbd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f428009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48548/0 (socket says 172.21.15.89:48548) 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f42853be700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f428009e410 0x7f4280134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53976/0 (socket says 172.21.15.89:53976) 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f4284bbd700 1 -- 172.21.15.89:0/2386716956 learned_addr learned my addr 172.21.15.89:0/2386716956 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f4284bbd700 1 -- 172.21.15.89:0/2386716956 >> v2:172.21.15.89:3300/0 conn(0x7f428009e410 msgr2=0x7f4280134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f4284bbd700 1 --2- 172.21.15.89:0/2386716956 >> v2:172.21.15.89:3300/0 conn(0x7f428009e410 0x7f4280134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f4284bbd700 1 -- 172.21.15.89:0/2386716956 >> v1:172.21.15.89:6789/0 conn(0x7f428009e000 legacy=0x7f42801339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f4284bbd700 1 -- 172.21.15.89:0/2386716956 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428009a500 con 0x7f428009d6f0 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f42853be700 1 --2- 172.21.15.89:0/2386716956 >> v2:172.21.15.89:3300/0 conn(0x7f428009e410 0x7f4280134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f4284bbd700 1 --2- 172.21.15.89:0/2386716956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f428009dac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4274009920 tx=0x7f4274009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d546d10ee37fc1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f427f7fe700 1 -- 172.21.15.89:0/2386716956 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274011830 con 0x7f428009d6f0 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f427f7fe700 1 -- 172.21.15.89:0/2386716956 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4274011990 con 0x7f428009d6f0 2023-09-30T22:17:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.619+0000 7f427f7fe700 1 -- 172.21.15.89:0/2386716956 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274011d20 con 0x7f428009d6f0 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.620+0000 7f4286e21700 1 -- 172.21.15.89:0/2386716956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 msgr2=0x7f428009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.620+0000 7f4286e21700 1 --2- 172.21.15.89:0/2386716956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f428009dac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4274009920 tx=0x7f4274009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.620+0000 7f4286e21700 1 -- 172.21.15.89:0/2386716956 shutdown_connections 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.620+0000 7f4286e21700 1 --2- 172.21.15.89:0/2386716956 >> v2:172.21.15.89:3300/0 conn(0x7f428009e410 0x7f4280134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.620+0000 7f4286e21700 1 --2- 172.21.15.89:0/2386716956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f428009dac0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.620+0000 7f4286e21700 1 -- 172.21.15.89:0/2386716956 >> 172.21.15.89:0/2386716956 conn(0x7f4280066f50 msgr2=0x7f428009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.620+0000 7f4286e21700 1 -- 172.21.15.89:0/2386716956 shutdown_connections 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.620+0000 7f4286e21700 1 -- 172.21.15.89:0/2386716956 wait complete. 2023-09-30T22:17:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4286e21700 1 Processor -- start 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4286e21700 1 -- start start 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4286e21700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f42801a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4286e21700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f428009e000 0x7f42801a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4286e21700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f428009e410 0x7f42801a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4286e21700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42800a3480 con 0x7f428009e000 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4286e21700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42801a6b90 con 0x7f428009e410 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4286e21700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42801a7d40 con 0x7f428009d6f0 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4284bbd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f42801a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f427ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f428009e000 0x7f42801a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4284bbd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f42801a4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48560/0 (socket says 172.21.15.89:48560) 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f4284bbd700 1 -- 172.21.15.89:0/2132659601 learned_addr learned my addr 172.21.15.89:0/2132659601 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.621+0000 7f42853be700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f428009e410 0x7f42801a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427ffff700 1 -- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 msgr2=0x7f42801a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427ffff700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f42801a4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427ffff700 1 -- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f428009e410 msgr2=0x7f42801a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427ffff700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f428009e410 0x7f42801a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427ffff700 1 -- 172.21.15.89:0/2132659601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4278009a70 con 0x7f428009e000 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f4284bbd700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f42801a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f42853be700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f428009e410 0x7f42801a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427ffff700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f428009e000 0x7f42801a5440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f426c000f60 tx=0x7f426c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427d7fa700 1 -- 172.21.15.89:0/2132659601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0052d0 con 0x7f428009e000 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427d7fa700 1 -- 172.21.15.89:0/2132659601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c005430 con 0x7f428009e000 2023-09-30T22:17:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4274009550 con 0x7f428009e000 2023-09-30T22:17:33.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f427d7fa700 1 -- 172.21.15.89:0/2132659601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0038c0 con 0x7f428009e000 2023-09-30T22:17:33.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.622+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42801a8990 con 0x7f428009e000 2023-09-30T22:17:33.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.623+0000 7f427d7fa700 1 -- 172.21.15.89:0/2132659601 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f426c003a20 con 0x7f428009e000 2023-09-30T22:17:33.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.623+0000 7f427d7fa700 1 -- 172.21.15.89:0/2132659601 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f426c03e980 con 0x7f428009e000 2023-09-30T22:17:33.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.623+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42801a8520 con 0x7f428009e000 2023-09-30T22:17:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.628+0000 7f427d7fa700 1 -- 172.21.15.89:0/2132659601 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f426c017020 con 0x7f428009e000 2023-09-30T22:17:33.678 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:33.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:33.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:33.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.806+0000 7f03c1f88700 1 Processor -- start 2023-09-30T22:17:33.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.806+0000 7f03c1f88700 1 -- start start 2023-09-30T22:17:33.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03c1f88700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03c1f88700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03bc06eba0 0x7f03bc139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03c1f88700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc071a00 con 0x7f03bc131f40 2023-09-30T22:17:33.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03c1f88700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03bc131430 con 0x7f03bc06eba0 2023-09-30T22:17:33.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03c1f88700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc09a510 con 0x7f03bc131630 2023-09-30T22:17:33.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03bb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03bb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48564/0 (socket says 172.21.15.89:48564) 2023-09-30T22:17:33.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03bbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03bc06eba0 0x7f03bc139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.807+0000 7f03bb7fe700 1 -- 172.21.15.89:0/1285259785 learned_addr learned my addr 172.21.15.89:0/1285259785 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:33.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03bb7fe700 1 -- 172.21.15.89:0/1285259785 >> v2:172.21.15.89:3300/0 conn(0x7f03bc06eba0 msgr2=0x7f03bc139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:33.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03bb7fe700 1 --2- 172.21.15.89:0/1285259785 >> v2:172.21.15.89:3300/0 conn(0x7f03bc06eba0 0x7f03bc139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03bb7fe700 1 -- 172.21.15.89:0/1285259785 >> v1:172.21.15.89:6789/0 conn(0x7f03bc131f40 legacy=0x7f03bc06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03bb7fe700 1 -- 172.21.15.89:0/1285259785 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc0721c0 con 0x7f03bc131630 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03bbfff700 1 --2- 172.21.15.89:0/1285259785 >> v2:172.21.15.89:3300/0 conn(0x7f03bc06eba0 0x7f03bc139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03bb7fe700 1 --2- 172.21.15.89:0/1285259785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc131a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0098f0 tx=0x7f03ac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8215745694be357b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03ba7fc700 1 -- 172.21.15.89:0/1285259785 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac011830 con 0x7f03bc131630 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03ba7fc700 1 -- 172.21.15.89:0/1285259785 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac011990 con 0x7f03bc131630 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.808+0000 7f03ba7fc700 1 -- 172.21.15.89:0/1285259785 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac011d20 con 0x7f03bc131630 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.809+0000 7f03c1f88700 1 -- 172.21.15.89:0/1285259785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 msgr2=0x7f03bc131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.809+0000 7f03c1f88700 1 --2- 172.21.15.89:0/1285259785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc131a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0098f0 tx=0x7f03ac009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:33.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.809+0000 7f03c1f88700 1 -- 172.21.15.89:0/1285259785 shutdown_connections 2023-09-30T22:17:33.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.809+0000 7f03c1f88700 1 --2- 172.21.15.89:0/1285259785 >> v2:172.21.15.89:3300/0 conn(0x7f03bc06eba0 0x7f03bc139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.809+0000 7f03c1f88700 1 --2- 172.21.15.89:0/1285259785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc131a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.809+0000 7f03c1f88700 1 -- 172.21.15.89:0/1285259785 >> 172.21.15.89:0/1285259785 conn(0x7f03bc0a2aa0 msgr2=0x7f03bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:33.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.809+0000 7f03c1f88700 1 -- 172.21.15.89:0/1285259785 shutdown_connections 2023-09-30T22:17:33.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.809+0000 7f03c1f88700 1 -- 172.21.15.89:0/1285259785 wait complete. 2023-09-30T22:17:33.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.810+0000 7f03c1f88700 1 Processor -- start 2023-09-30T22:17:33.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.810+0000 7f03c1f88700 1 -- start start 2023-09-30T22:17:33.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03c1f88700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03bc06eba0 0x7f03bc129950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03c1f88700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc129e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03c1f88700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03bc131f40 0x7f03bc12a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03c1f88700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc12d130 con 0x7f03bc06eba0 2023-09-30T22:17:33.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03c1f88700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc12dcf0 con 0x7f03bc131f40 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03c1f88700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc12df10 con 0x7f03bc131630 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc129e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03bb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03bc06eba0 0x7f03bc129950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03bbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03bc131f40 0x7f03bc12a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc129e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48574/0 (socket says 172.21.15.89:48574) 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 -- 172.21.15.89:0/796221717 learned_addr learned my addr 172.21.15.89:0/796221717 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 -- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03bc131f40 msgr2=0x7f03bc12a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 --2- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03bc131f40 0x7f03bc12a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 -- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03bc06eba0 msgr2=0x7f03bc129950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 --2- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03bc06eba0 0x7f03bc129950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 -- 172.21.15.89:0/796221717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4009a70 con 0x7f03bc131630 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03bb7fe700 1 --2- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03bc06eba0 0x7f03bc129950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.811+0000 7f03baffd700 1 --2- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc129e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c4b0 tx=0x7f03b000c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.812+0000 7f03a3fff700 1 -- 172.21.15.89:0/796221717 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00052d0 con 0x7f03bc131630 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.812+0000 7f03a3fff700 1 -- 172.21.15.89:0/796221717 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0005430 con 0x7f03bc131630 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.812+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac009550 con 0x7f03bc131630 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.812+0000 7f03a3fff700 1 -- 172.21.15.89:0/796221717 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00038c0 con 0x7f03bc131630 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.812+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc12b4a0 con 0x7f03bc131630 2023-09-30T22:17:33.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.813+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc099f80 con 0x7f03bc131630 2023-09-30T22:17:33.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.813+0000 7f03a3fff700 1 -- 172.21.15.89:0/796221717 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03b0003a20 con 0x7f03bc131630 2023-09-30T22:17:33.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.814+0000 7f03a3fff700 1 -- 172.21.15.89:0/796221717 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f03b003ead0 con 0x7f03bc131630 2023-09-30T22:17:33.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.818+0000 7f03a3fff700 1 -- 172.21.15.89:0/796221717 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03b0018260 con 0x7f03bc131630 2023-09-30T22:17:33.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.826+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f42801a82d0 con 0x7f428009e000 2023-09-30T22:17:33.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.826+0000 7f427d7fa700 1 -- 172.21.15.89:0/2132659601 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f426c019b40 con 0x7f428009e000 2023-09-30T22:17:33.827 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.829+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f428009e000 msgr2=0x7f42801a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.829+0000 7f4286e21700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f428009e000 0x7f42801a5440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f426c000f60 tx=0x7f426c00c730 comp rx=0 tx=0).stop 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.829+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 shutdown_connections 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.829+0000 7f4286e21700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f428009e410 0x7f42801a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.829+0000 7f4286e21700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f428009d6f0 0x7f42801a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.829+0000 7f4286e21700 1 --2- 172.21.15.89:0/2132659601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f428009e000 0x7f42801a5440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.829+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 >> 172.21.15.89:0/2132659601 conn(0x7f4280066f50 msgr2=0x7f42801bb020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.830+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 shutdown_connections 2023-09-30T22:17:33.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.830+0000 7f4286e21700 1 -- 172.21.15.89:0/2132659601 wait complete. 2023-09-30T22:17:33.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:33.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.858+0000 7f17e7be8700 1 Processor -- start 2023-09-30T22:17:33.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.858+0000 7f17e7be8700 1 -- start start 2023-09-30T22:17:33.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e7be8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17e009d6f0 0x7f17e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e7be8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009e000 0x7f17e0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e7be8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e009f130 con 0x7f17e009e410 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e7be8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e009f520 con 0x7f17e009d6f0 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e7be8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e009e930 con 0x7f17e009e000 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e5984700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17e009d6f0 0x7f17e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e5984700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17e009d6f0 0x7f17e009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54012/0 (socket says 172.21.15.89:54012) 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e5984700 1 -- 172.21.15.89:0/1160297864 learned_addr learned my addr 172.21.15.89:0/1160297864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e5183700 1 --2- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009e000 0x7f17e0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e5984700 1 -- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009e000 msgr2=0x7f17e0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e5984700 1 --2- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009e000 0x7f17e0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e5984700 1 -- 172.21.15.89:0/1160297864 >> v1:172.21.15.89:6789/0 conn(0x7f17e009e410 legacy=0x7f17e0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.859+0000 7f17e5984700 1 -- 172.21.15.89:0/1160297864 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e009b2d0 con 0x7f17e009d6f0 2023-09-30T22:17:33.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.860+0000 7f17e5183700 1 --2- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009e000 0x7f17e0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.860+0000 7f17e5984700 1 --2- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009d6f0 0x7f17e009dac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0098f0 tx=0x7f17dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2268a1997cf0e977 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:33.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.860+0000 7f17e4982700 1 -- 172.21.15.89:0/1160297864 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc011830 con 0x7f17e009d6f0 2023-09-30T22:17:33.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.860+0000 7f17e4982700 1 -- 172.21.15.89:0/1160297864 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc011990 con 0x7f17e009d6f0 2023-09-30T22:17:33.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.860+0000 7f17e4982700 1 -- 172.21.15.89:0/1160297864 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc011d20 con 0x7f17e009d6f0 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.861+0000 7f17e7be8700 1 -- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009d6f0 msgr2=0x7f17e009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.861+0000 7f17e7be8700 1 --2- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009d6f0 0x7f17e009dac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0098f0 tx=0x7f17dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.861+0000 7f17e7be8700 1 -- 172.21.15.89:0/1160297864 shutdown_connections 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.861+0000 7f17e7be8700 1 --2- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009d6f0 0x7f17e009dac0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.861+0000 7f17e7be8700 1 --2- 172.21.15.89:0/1160297864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009e000 0x7f17e0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.861+0000 7f17e7be8700 1 -- 172.21.15.89:0/1160297864 >> 172.21.15.89:0/1160297864 conn(0x7f17e0066f50 msgr2=0x7f17e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.861+0000 7f17e7be8700 1 -- 172.21.15.89:0/1160297864 shutdown_connections 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.861+0000 7f17e7be8700 1 -- 172.21.15.89:0/1160297864 wait complete. 2023-09-30T22:17:33.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e7be8700 1 Processor -- start 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e7be8700 1 -- start start 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e7be8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009d6f0 0x7f17e0131630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e7be8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e000 0x7f17e0131b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e7be8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009e410 0x7f17e01320b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e7be8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e0132a00 con 0x7f17e009e410 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e7be8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e009c700 con 0x7f17e009d6f0 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e7be8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17e0132e30 con 0x7f17e009e000 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e5984700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009d6f0 0x7f17e0131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e5984700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009d6f0 0x7f17e0131630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43864/0 (socket says 172.21.15.89:43864) 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e5984700 1 -- 172.21.15.89:0/1101629825 learned_addr learned my addr 172.21.15.89:0/1101629825 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e5183700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e000 0x7f17e0131b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.862+0000 7f17e6185700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009e410 0x7f17e01320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17e5984700 1 -- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e000 msgr2=0x7f17e0131b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17e5984700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e000 0x7f17e0131b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17e5984700 1 -- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009e410 msgr2=0x7f17e01320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17e5984700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009e410 0x7f17e01320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17e5984700 1 -- 172.21.15.89:0/1101629825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00094a0 con 0x7f17e009d6f0 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17e5984700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009d6f0 0x7f17e0131630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17dc000c00 tx=0x7f17dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17d67fc700 1 -- 172.21.15.89:0/1101629825 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0036a0 con 0x7f17e009d6f0 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc009550 con 0x7f17e009d6f0 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e01334a0 con 0x7f17e009d6f0 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17d67fc700 1 -- 172.21.15.89:0/1101629825 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc006ce0 con 0x7f17e009d6f0 2023-09-30T22:17:33.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.863+0000 7f17d67fc700 1 -- 172.21.15.89:0/1101629825 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc003a00 con 0x7f17e009d6f0 2023-09-30T22:17:33.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.864+0000 7f17d67fc700 1 -- 172.21.15.89:0/1101629825 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f17dc021070 con 0x7f17e009d6f0 2023-09-30T22:17:33.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.864+0000 7f17d67fc700 1 -- 172.21.15.89:0/1101629825 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f17dc040b90 con 0x7f17e009d6f0 2023-09-30T22:17:33.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.864+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c80052e0 con 0x7f17e009d6f0 2023-09-30T22:17:33.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.869+0000 7f17d67fc700 1 -- 172.21.15.89:0/1101629825 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17dc015070 con 0x7f17e009d6f0 2023-09-30T22:17:33.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.896+0000 7f89ff941700 1 Processor -- start 2023-09-30T22:17:33.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.896+0000 7f89ff941700 1 -- start start 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.896+0000 7f89ff941700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f80a0a30 0x7f89f80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.896+0000 7f89ff941700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f89f809d630 0x7f89f80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.896+0000 7f89ff941700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f80a2a70 con 0x7f89f8133930 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.896+0000 7f89ff941700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f80a01d0 con 0x7f89f809d630 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.896+0000 7f89ff941700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89f809e2b0 con 0x7f89f80a0a30 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fcedc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f89f809d630 0x7f89f80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fd6dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f80a0a30 0x7f89f80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fcedc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f89f809d630 0x7f89f80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54038/0 (socket says 172.21.15.89:54038) 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fd6dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f80a0a30 0x7f89f80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48592/0 (socket says 172.21.15.89:48592) 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fcedc700 1 -- 172.21.15.89:0/3903547013 learned_addr learned my addr 172.21.15.89:0/3903547013 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fd6dd700 1 -- 172.21.15.89:0/3903547013 >> v2:172.21.15.89:3300/0 conn(0x7f89f809d630 msgr2=0x7f89f80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fd6dd700 1 --2- 172.21.15.89:0/3903547013 >> v2:172.21.15.89:3300/0 conn(0x7f89f809d630 0x7f89f80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fd6dd700 1 -- 172.21.15.89:0/3903547013 >> v1:172.21.15.89:6789/0 conn(0x7f89f8133930 legacy=0x7f89f8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fd6dd700 1 -- 172.21.15.89:0/3903547013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f809dc50 con 0x7f89f80a0a30 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fcedc700 1 --2- 172.21.15.89:0/3903547013 >> v2:172.21.15.89:3300/0 conn(0x7f89f809d630 0x7f89f80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89fd6dd700 1 --2- 172.21.15.89:0/3903547013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f80a0a30 0x7f89f80a0e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009840 tx=0x7f89f4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45090e96171646c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89effff700 1 -- 172.21.15.89:0/3903547013 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4011830 con 0x7f89f80a0a30 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89effff700 1 -- 172.21.15.89:0/3903547013 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f4011990 con 0x7f89f80a0a30 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.897+0000 7f89effff700 1 -- 172.21.15.89:0/3903547013 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4011d20 con 0x7f89f80a0a30 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.898+0000 7f89ff941700 1 -- 172.21.15.89:0/3903547013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f80a0a30 msgr2=0x7f89f80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.898+0000 7f89ff941700 1 --2- 172.21.15.89:0/3903547013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f80a0a30 0x7f89f80a0e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009840 tx=0x7f89f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.898+0000 7f89ff941700 1 -- 172.21.15.89:0/3903547013 shutdown_connections 2023-09-30T22:17:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.898+0000 7f89ff941700 1 --2- 172.21.15.89:0/3903547013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f80a0a30 0x7f89f80a0e00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.898+0000 7f89ff941700 1 --2- 172.21.15.89:0/3903547013 >> v2:172.21.15.89:3300/0 conn(0x7f89f809d630 0x7f89f80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.898+0000 7f89ff941700 1 -- 172.21.15.89:0/3903547013 >> 172.21.15.89:0/3903547013 conn(0x7f89f80a3050 msgr2=0x7f89f80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:33.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.898+0000 7f89ff941700 1 -- 172.21.15.89:0/3903547013 shutdown_connections 2023-09-30T22:17:33.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.898+0000 7f89ff941700 1 -- 172.21.15.89:0/3903547013 wait complete. 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89ff941700 1 Processor -- start 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89ff941700 1 -- start start 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89ff941700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f809d630 0x7f89f81c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89ff941700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f80a0a30 0x7f89f81c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89ff941700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f8133930 0x7f89f81c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89ff941700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f8120670 con 0x7f89f80a0a30 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89ff941700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f8121e60 con 0x7f89f809d630 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89ff941700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80a0410 con 0x7f89f8133930 2023-09-30T22:17:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89fd6dd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f809d630 0x7f89f81c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89fdede700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f8133930 0x7f89f81c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89fcedc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f80a0a30 0x7f89f81c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89fdede700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f8133930 0x7f89f81c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48606/0 (socket says 172.21.15.89:48606) 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89fd6dd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f809d630 0x7f89f81c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43876/0 (socket says 172.21.15.89:43876) 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.899+0000 7f89fdede700 1 -- 172.21.15.89:0/798089540 learned_addr learned my addr 172.21.15.89:0/798089540 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89fdede700 1 -- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f809d630 msgr2=0x7f89f81c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89fdede700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f809d630 0x7f89f81c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89fdede700 1 -- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f80a0a30 msgr2=0x7f89f81c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89fdede700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f80a0a30 0x7f89f81c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89fdede700 1 -- 172.21.15.89:0/798089540 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e8009550 con 0x7f89f8133930 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89fcedc700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f80a0a30 0x7f89f81c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89fd6dd700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f809d630 0x7f89f81c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89fdede700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f8133930 0x7f89f81c8bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f89f000cc50 tx=0x7f89f000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89edffb700 1 -- 172.21.15.89:0/798089540 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0003c00 con 0x7f89f8133930 2023-09-30T22:17:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89edffb700 1 -- 172.21.15.89:0/798089540 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f0003d60 con 0x7f89f8133930 2023-09-30T22:17:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f40094a0 con 0x7f89f8133930 2023-09-30T22:17:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89edffb700 1 -- 172.21.15.89:0/798089540 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f000aab0 con 0x7f89f8133930 2023-09-30T22:17:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.900+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f8121160 con 0x7f89f8133930 2023-09-30T22:17:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.901+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f80747f0 con 0x7f89f8133930 2023-09-30T22:17:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.902+0000 7f89edffb700 1 -- 172.21.15.89:0/798089540 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89f000ac10 con 0x7f89f8133930 2023-09-30T22:17:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.902+0000 7f89edffb700 1 -- 172.21.15.89:0/798089540 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f89f0047350 con 0x7f89f8133930 2023-09-30T22:17:33.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:33.906+0000 7f89edffb700 1 -- 172.21.15.89:0/798089540 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89f0044060 con 0x7f89f8133930 2023-09-30T22:17:34.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.069+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f03bc074ea0 con 0x7f03bc131630 2023-09-30T22:17:34.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.069+0000 7f03a3fff700 1 -- 172.21.15.89:0/796221717 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03b0017070 con 0x7f03bc131630 2023-09-30T22:17:34.070 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:34.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 msgr2=0x7f03bc129e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:34.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 --2- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc129e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c4b0 tx=0x7f03b000c7c0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 shutdown_connections 2023-09-30T22:17:34.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 --2- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03bc131f40 0x7f03bc12a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 --2- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03bc06eba0 0x7f03bc129950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 --2- 172.21.15.89:0/796221717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03bc131630 0x7f03bc129e90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 >> 172.21.15.89:0/796221717 conn(0x7f03bc0a2aa0 msgr2=0x7f03bc130120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:34.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 shutdown_connections 2023-09-30T22:17:34.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.071+0000 7f03c1f88700 1 -- 172.21.15.89:0/796221717 wait complete. 2023-09-30T22:17:34.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:34.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.101+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f89f8072980 con 0x7f89f8133930 2023-09-30T22:17:34.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.101+0000 7f89edffb700 1 -- 172.21.15.89:0/798089540 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89f0044030 con 0x7f89f8133930 2023-09-30T22:17:34.102 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:34.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f8133930 msgr2=0x7f89f81c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:34.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f8133930 0x7f89f81c8bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f89f000cc50 tx=0x7f89f000cf60 comp rx=0 tx=0).stop 2023-09-30T22:17:34.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 shutdown_connections 2023-09-30T22:17:34.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f8133930 0x7f89f81c8bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f809d630 0x7f89f81c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 --2- 172.21.15.89:0/798089540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f80a0a30 0x7f89f81c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 >> 172.21.15.89:0/798089540 conn(0x7f89f80a3050 msgr2=0x7f89f80a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:34.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 shutdown_connections 2023-09-30T22:17:34.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.103+0000 7f89ff941700 1 -- 172.21.15.89:0/798089540 wait complete. 2023-09-30T22:17:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.104+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f17c8005590 con 0x7f17e009d6f0 2023-09-30T22:17:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.104+0000 7f17d67fc700 1 -- 172.21.15.89:0/1101629825 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17dc024470 con 0x7f17e009d6f0 2023-09-30T22:17:34.106 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009d6f0 msgr2=0x7f17e0131630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009d6f0 0x7f17e0131630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17dc000c00 tx=0x7f17dc004750 comp rx=0 tx=0).stop 2023-09-30T22:17:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 shutdown_connections 2023-09-30T22:17:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e009e410 0x7f17e01320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009d6f0 0x7f17e0131630 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 --2- 172.21.15.89:0/1101629825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e000 0x7f17e0131b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 >> 172.21.15.89:0/1101629825 conn(0x7f17e0066f50 msgr2=0x7f17e00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:34.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 shutdown_connections 2023-09-30T22:17:34.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.106+0000 7f17e7be8700 1 -- 172.21.15.89:0/1101629825 wait complete. 2023-09-30T22:17:34.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:34.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:34.845 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58bbf86700 1 Processor -- start 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58bbf86700 1 -- start start 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58bbf86700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f58b4136710 0x7f58b4138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58bbf86700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4139030 0x7f58b413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58bbf86700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b406ef30 con 0x7f58b41338a0 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58bbf86700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b406fef0 con 0x7f58b4136710 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58bbf86700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41334b0 con 0x7f58b4139030 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58ba523700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4139030 0x7f58b413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58b9521700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f58b4136710 0x7f58b4138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58b9521700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f58b4136710 0x7f58b4138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54044/0 (socket says 172.21.15.89:54044) 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58ba523700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4139030 0x7f58b413b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48612/0 (socket says 172.21.15.89:48612) 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.970+0000 7f58b9521700 1 -- 172.21.15.89:0/2127660739 learned_addr learned my addr 172.21.15.89:0/2127660739 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58b9521700 1 -- 172.21.15.89:0/2127660739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4139030 msgr2=0x7f58b413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58b9521700 1 --2- 172.21.15.89:0/2127660739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4139030 0x7f58b413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58b9521700 1 -- 172.21.15.89:0/2127660739 >> v1:172.21.15.89:6789/0 conn(0x7f58b41338a0 legacy=0x7f58b4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58b9521700 1 -- 172.21.15.89:0/2127660739 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b406fc30 con 0x7f58b4136710 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58ba523700 1 --2- 172.21.15.89:0/2127660739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4139030 0x7f58b413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58b9521700 1 --2- 172.21.15.89:0/2127660739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4136710 0x7f58b4138af0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f58a40098f0 tx=0x7f58a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5464acc9e986ad8e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58b8d20700 1 -- 172.21.15.89:0/2127660739 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4011830 con 0x7f58b4136710 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58b8d20700 1 -- 172.21.15.89:0/2127660739 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a4011990 con 0x7f58b4136710 2023-09-30T22:17:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.971+0000 7f58b8d20700 1 -- 172.21.15.89:0/2127660739 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4011d20 con 0x7f58b4136710 2023-09-30T22:17:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.972+0000 7f58bbf86700 1 -- 172.21.15.89:0/2127660739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4136710 msgr2=0x7f58b4138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.972+0000 7f58bbf86700 1 --2- 172.21.15.89:0/2127660739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4136710 0x7f58b4138af0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f58a40098f0 tx=0x7f58a4009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.972+0000 7f58bbf86700 1 -- 172.21.15.89:0/2127660739 shutdown_connections 2023-09-30T22:17:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.972+0000 7f58bbf86700 1 --2- 172.21.15.89:0/2127660739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4139030 0x7f58b413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.972+0000 7f58bbf86700 1 --2- 172.21.15.89:0/2127660739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4136710 0x7f58b4138af0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.972+0000 7f58bbf86700 1 -- 172.21.15.89:0/2127660739 >> 172.21.15.89:0/2127660739 conn(0x7f58b40a2aa0 msgr2=0x7f58b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.972+0000 7f58bbf86700 1 -- 172.21.15.89:0/2127660739 shutdown_connections 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.972+0000 7f58bbf86700 1 -- 172.21.15.89:0/2127660739 wait complete. 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58bbf86700 1 Processor -- start 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58bbf86700 1 -- start start 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58bbf86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b41338a0 0x7f58b411ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58bbf86700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4136710 0x7f58b411b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58bbf86700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4139030 0x7f58b411b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58bbf86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b411e5d0 con 0x7f58b41338a0 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58bbf86700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b411cab0 con 0x7f58b4139030 2023-09-30T22:17:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58bbf86700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4138b00 con 0x7f58b4136710 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9d22700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b41338a0 0x7f58b411ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9d22700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b41338a0 0x7f58b411ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54048/0 (socket says 172.21.15.89:54048) 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58ba523700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4139030 0x7f58b411b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9d22700 1 -- 172.21.15.89:0/3511872334 learned_addr learned my addr 172.21.15.89:0/3511872334 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9521700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4136710 0x7f58b411b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9d22700 1 -- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4136710 msgr2=0x7f58b411b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9d22700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4136710 0x7f58b411b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9d22700 1 -- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4139030 msgr2=0x7f58b411b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9d22700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4139030 0x7f58b411b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58b9d22700 1 -- 172.21.15.89:0/3511872334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac009a70 con 0x7f58b41338a0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.973+0000 7f58ba523700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4139030 0x7f58b411b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.974+0000 7f58b9d22700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b41338a0 0x7f58b411ae30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c4e0 tx=0x7f58b000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.974+0000 7f58aa7fc700 1 -- 172.21.15.89:0/3511872334 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00052d0 con 0x7f58b41338a0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.974+0000 7f58aa7fc700 1 -- 172.21.15.89:0/3511872334 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b0005430 con 0x7f58b41338a0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.974+0000 7f58aa7fc700 1 -- 172.21.15.89:0/3511872334 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00038c0 con 0x7f58b41338a0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.974+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a4009550 con 0x7f58b41338a0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.974+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b406f690 con 0x7f58b41338a0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.974+0000 7f58aa7fc700 1 -- 172.21.15.89:0/3511872334 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f58b0003a20 con 0x7f58b41338a0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.975+0000 7f58aa7fc700 1 -- 172.21.15.89:0/3511872334 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f58b003eb70 con 0x7f58b41338a0 2023-09-30T22:17:34.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.975+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b411f100 con 0x7f58b41338a0 2023-09-30T22:17:34.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:34.979+0000 7f58aa7fc700 1 -- 172.21.15.89:0/3511872334 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58b0017080 con 0x7f58b41338a0 2023-09-30T22:17:35.085 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:35.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:35.121 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:35.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.171+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f58b406ea40 con 0x7f58b41338a0 2023-09-30T22:17:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.172+0000 7f58aa7fc700 1 -- 172.21.15.89:0/3511872334 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58b0018260 con 0x7f58b41338a0 2023-09-30T22:17:35.174 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b41338a0 msgr2=0x7f58b411ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b41338a0 0x7f58b411ae30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c4e0 tx=0x7f58b000c7f0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 shutdown_connections 2023-09-30T22:17:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4139030 0x7f58b411b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b41338a0 0x7f58b411ae30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 --2- 172.21.15.89:0/3511872334 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b4136710 0x7f58b411b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 >> 172.21.15.89:0/3511872334 conn(0x7f58b40a2aa0 msgr2=0x7f58b41344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 shutdown_connections 2023-09-30T22:17:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.174+0000 7f58bbf86700 1 -- 172.21.15.89:0/3511872334 wait complete. 2023-09-30T22:17:35.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:35.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.217+0000 7f6be01fc700 1 Processor -- start 2023-09-30T22:17:35.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.217+0000 7f6be01fc700 1 -- start start 2023-09-30T22:17:35.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6be01fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6bd809e930 0x7f6bd809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6be01fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809f240 0x7f6bd81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6be01fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f650 0x7f6bd8131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6be01fc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd8070940 con 0x7f6bd809e930 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6be01fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd806fd60 con 0x7f6bd809f650 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6be01fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd806fa80 con 0x7f6bd809f240 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bdd797700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809f240 0x7f6bd81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bddf98700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6bd809e930 0x7f6bd809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bdd797700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809f240 0x7f6bd81317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48624/0 (socket says 172.21.15.89:48624) 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bdd797700 1 -- 172.21.15.89:0/3962415588 learned_addr learned my addr 172.21.15.89:0/3962415588 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bde799700 1 --2- 172.21.15.89:0/3962415588 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f650 0x7f6bd8131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bdd797700 1 -- 172.21.15.89:0/3962415588 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f650 msgr2=0x7f6bd8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bdd797700 1 --2- 172.21.15.89:0/3962415588 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f650 0x7f6bd8131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bdd797700 1 -- 172.21.15.89:0/3962415588 >> v2:172.21.15.89:3300/0 conn(0x7f6bd809e930 msgr2=0x7f6bd809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:35.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bdd797700 1 --2- 172.21.15.89:0/3962415588 >> v2:172.21.15.89:3300/0 conn(0x7f6bd809e930 0x7f6bd809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.218+0000 7f6bdd797700 1 -- 172.21.15.89:0/3962415588 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd809fb60 con 0x7f6bd809f240 2023-09-30T22:17:35.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.219+0000 7f6bdd797700 1 --2- 172.21.15.89:0/3962415588 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809f240 0x7f6bd81317f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80098f0 tx=0x7f6bc8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed252c74929ce80 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:35.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.219+0000 7f6bdcf96700 1 -- 172.21.15.89:0/3962415588 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8011830 con 0x7f6bd809f240 2023-09-30T22:17:35.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.219+0000 7f6bdcf96700 1 -- 172.21.15.89:0/3962415588 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8011990 con 0x7f6bd809f240 2023-09-30T22:17:35.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.219+0000 7f6bdcf96700 1 -- 172.21.15.89:0/3962415588 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8011d20 con 0x7f6bd809f240 2023-09-30T22:17:35.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 -- 172.21.15.89:0/3962415588 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809f240 msgr2=0x7f6bd81317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 --2- 172.21.15.89:0/3962415588 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809f240 0x7f6bd81317f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80098f0 tx=0x7f6bc8009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 -- 172.21.15.89:0/3962415588 shutdown_connections 2023-09-30T22:17:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 --2- 172.21.15.89:0/3962415588 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f650 0x7f6bd8131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 --2- 172.21.15.89:0/3962415588 >> v2:172.21.15.89:3300/0 conn(0x7f6bd809e930 0x7f6bd809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 --2- 172.21.15.89:0/3962415588 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809f240 0x7f6bd81317f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 -- 172.21.15.89:0/3962415588 >> 172.21.15.89:0/3962415588 conn(0x7f6bd80a4210 msgr2=0x7f6bd80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:35.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 -- 172.21.15.89:0/3962415588 shutdown_connections 2023-09-30T22:17:35.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.220+0000 7f6be01fc700 1 -- 172.21.15.89:0/3962415588 wait complete. 2023-09-30T22:17:35.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.221+0000 7f6be01fc700 1 Processor -- start 2023-09-30T22:17:35.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.221+0000 7f6be01fc700 1 -- start start 2023-09-30T22:17:35.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6be01fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809e930 0x7f6bd81a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6be01fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f240 0x7f6bd81a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6be01fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd809f650 0x7f6bd81a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6be01fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd81a5e30 con 0x7f6bd809f650 2023-09-30T22:17:35.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6be01fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd81a6ef0 con 0x7f6bd809f240 2023-09-30T22:17:35.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6be01fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd8071d20 con 0x7f6bd809e930 2023-09-30T22:17:35.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bdd797700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f240 0x7f6bd81a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bdd797700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f240 0x7f6bd81a4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43894/0 (socket says 172.21.15.89:43894) 2023-09-30T22:17:35.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bdd797700 1 -- 172.21.15.89:0/1615175668 learned_addr learned my addr 172.21.15.89:0/1615175668 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:35.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bddf98700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809e930 0x7f6bd81a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bde799700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd809f650 0x7f6bd81a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bdd797700 1 -- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809e930 msgr2=0x7f6bd81a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bdd797700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809e930 0x7f6bd81a3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bdd797700 1 -- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd809f650 msgr2=0x7f6bd81a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bdd797700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd809f650 0x7f6bd81a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.222+0000 7f6bdd797700 1 -- 172.21.15.89:0/1615175668 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc8009550 con 0x7f6bd809f240 2023-09-30T22:17:35.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.223+0000 7f6bdd797700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f240 0x7f6bd81a4180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8000c00 tx=0x7f6bc8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:35.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.223+0000 7f6bce7fc700 1 -- 172.21.15.89:0/1615175668 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8004240 con 0x7f6bd809f240 2023-09-30T22:17:35.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.223+0000 7f6bce7fc700 1 -- 172.21.15.89:0/1615175668 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8023070 con 0x7f6bd809f240 2023-09-30T22:17:35.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.223+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd80a0190 con 0x7f6bd809f240 2023-09-30T22:17:35.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.223+0000 7f6bce7fc700 1 -- 172.21.15.89:0/1615175668 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80045c0 con 0x7f6bd809f240 2023-09-30T22:17:35.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.223+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd81a5850 con 0x7f6bd809f240 2023-09-30T22:17:35.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.224+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd8074ea0 con 0x7f6bd809f240 2023-09-30T22:17:35.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.224+0000 7f6bce7fc700 1 -- 172.21.15.89:0/1615175668 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6bc8025030 con 0x7f6bd809f240 2023-09-30T22:17:35.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.225+0000 7f6bce7fc700 1 -- 172.21.15.89:0/1615175668 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6bc8045080 con 0x7f6bd809f240 2023-09-30T22:17:35.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.233+0000 7f6bce7fc700 1 -- 172.21.15.89:0/1615175668 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bc800b3e0 con 0x7f6bd809f240 2023-09-30T22:17:35.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.260+0000 7f5d5de2a700 1 Processor -- start 2023-09-30T22:17:35.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.260+0000 7f5d5de2a700 1 -- start start 2023-09-30T22:17:35.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.260+0000 7f5d5de2a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809d6f0 0x7f5d5809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.260+0000 7f5d5de2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e000 0x7f5d581339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.260+0000 7f5d5de2a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d5809e410 0x7f5d58133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.260+0000 7f5d5de2a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5806b460 con 0x7f5d5809e410 2023-09-30T22:17:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.260+0000 7f5d5de2a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5809b120 con 0x7f5d5809e000 2023-09-30T22:17:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.260+0000 7f5d5de2a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5809a7e0 con 0x7f5d5809d6f0 2023-09-30T22:17:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d57fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d5809e410 0x7f5d58133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d577fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809d6f0 0x7f5d5809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d577fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809d6f0 0x7f5d5809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48656/0 (socket says 172.21.15.89:48656) 2023-09-30T22:17:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d57fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d5809e410 0x7f5d58133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54086/0 (socket says 172.21.15.89:54086) 2023-09-30T22:17:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d577fe700 1 -- 172.21.15.89:0/3302044308 learned_addr learned my addr 172.21.15.89:0/3302044308 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d56ffd700 1 --2- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e000 0x7f5d581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d56ffd700 1 -- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809d6f0 msgr2=0x7f5d5809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d56ffd700 1 --2- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809d6f0 0x7f5d5809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d56ffd700 1 -- 172.21.15.89:0/3302044308 >> v2:172.21.15.89:3300/0 conn(0x7f5d5809e410 msgr2=0x7f5d58133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d56ffd700 1 --2- 172.21.15.89:0/3302044308 >> v2:172.21.15.89:3300/0 conn(0x7f5d5809e410 0x7f5d58133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d56ffd700 1 -- 172.21.15.89:0/3302044308 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5809a500 con 0x7f5d5809e000 2023-09-30T22:17:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d577fe700 1 --2- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809d6f0 0x7f5d5809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d57fff700 1 --2- 172.21.15.89:0/3302044308 >> v2:172.21.15.89:3300/0 conn(0x7f5d5809e410 0x7f5d58133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d56ffd700 1 --2- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e000 0x7f5d581339f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009870 tx=0x7f5d4c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f390379b37d48d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d567fc700 1 -- 172.21.15.89:0/3302044308 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c011830 con 0x7f5d5809e000 2023-09-30T22:17:35.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d567fc700 1 -- 172.21.15.89:0/3302044308 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c011990 con 0x7f5d5809e000 2023-09-30T22:17:35.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.261+0000 7f5d567fc700 1 -- 172.21.15.89:0/3302044308 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c011d20 con 0x7f5d5809e000 2023-09-30T22:17:35.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 -- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e000 msgr2=0x7f5d581339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 --2- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e000 0x7f5d581339f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009870 tx=0x7f5d4c009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:35.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 -- 172.21.15.89:0/3302044308 shutdown_connections 2023-09-30T22:17:35.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 --2- 172.21.15.89:0/3302044308 >> v2:172.21.15.89:3300/0 conn(0x7f5d5809e410 0x7f5d58133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 --2- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809d6f0 0x7f5d5809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 --2- 172.21.15.89:0/3302044308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e000 0x7f5d581339f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 -- 172.21.15.89:0/3302044308 >> 172.21.15.89:0/3302044308 conn(0x7f5d58066f50 msgr2=0x7f5d5809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 -- 172.21.15.89:0/3302044308 shutdown_connections 2023-09-30T22:17:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.262+0000 7f5d5de2a700 1 -- 172.21.15.89:0/3302044308 wait complete. 2023-09-30T22:17:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.263+0000 7f5d5de2a700 1 Processor -- start 2023-09-30T22:17:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.263+0000 7f5d5de2a700 1 -- start start 2023-09-30T22:17:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.263+0000 7f5d5de2a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5809d6f0 0x7f5d581a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.263+0000 7f5d5de2a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809e000 0x7f5d581a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.263+0000 7f5d5de2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e410 0x7f5d581a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.263+0000 7f5d5de2a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d581aa270 con 0x7f5d5809d6f0 2023-09-30T22:17:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.263+0000 7f5d5de2a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d581a79a0 con 0x7f5d5809e410 2023-09-30T22:17:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.263+0000 7f5d5de2a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d581a7fc0 con 0x7f5d5809e000 2023-09-30T22:17:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5809d6f0 0x7f5d581a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5809d6f0 0x7f5d581a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54096/0 (socket says 172.21.15.89:54096) 2023-09-30T22:17:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 -- 172.21.15.89:0/437599094 learned_addr learned my addr 172.21.15.89:0/437599094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 -- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809e000 msgr2=0x7f5d581a62d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:17:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 --2- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809e000 0x7f5d581a62d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 -- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e410 msgr2=0x7f5d581a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 --2- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e410 0x7f5d581a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 -- 172.21.15.89:0/437599094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48009550 con 0x7f5d5809d6f0 2023-09-30T22:17:35.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d577fe700 1 --2- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5809d6f0 0x7f5d581a5d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800ec10 tx=0x7f5d48009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:35.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d3ffff700 1 -- 172.21.15.89:0/437599094 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4800c860 con 0x7f5d5809d6f0 2023-09-30T22:17:35.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d3ffff700 1 -- 172.21.15.89:0/437599094 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4800c9c0 con 0x7f5d5809d6f0 2023-09-30T22:17:35.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d50009a70 con 0x7f5d5809d6f0 2023-09-30T22:17:35.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d3ffff700 1 -- 172.21.15.89:0/437599094 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4800cd30 con 0x7f5d5809d6f0 2023-09-30T22:17:35.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c0094a0 con 0x7f5d5809d6f0 2023-09-30T22:17:35.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5d3ffff700 1 -- 172.21.15.89:0/437599094 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d48013470 con 0x7f5d5809d6f0 2023-09-30T22:17:35.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5d3ffff700 1 -- 172.21.15.89:0/437599094 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5d4803f820 con 0x7f5d5809d6f0 2023-09-30T22:17:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f596dd07700 1 Processor -- start 2023-09-30T22:17:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.264+0000 7f596dd07700 1 -- start start 2023-09-30T22:17:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f596dd07700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 0x7f596806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f596dd07700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5968130430 0x7f5968137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f596dd07700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596812cd90 con 0x7f596812f8b0 2023-09-30T22:17:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f596dd07700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596812d330 con 0x7f5968130430 2023-09-30T22:17:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f596dd07700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5968071780 con 0x7f596812cf60 2023-09-30T22:17:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5967fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5968130430 0x7f5968137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5967fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5968130430 0x7f5968137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54102/0 (socket says 172.21.15.89:54102) 2023-09-30T22:17:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5967fff700 1 -- 172.21.15.89:0/1805808223 learned_addr learned my addr 172.21.15.89:0/1805808223 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5967fff700 1 -- 172.21.15.89:0/1805808223 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 msgr2=0x7f596806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:35.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5967fff700 1 --2- 172.21.15.89:0/1805808223 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 0x7f596806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f59667fc700 1 -- 172.21.15.89:0/1805808223 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413018786 0 0) 0x7f596812cd90 con 0x7f596812f8b0 2023-09-30T22:17:35.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5967fff700 1 -- 172.21.15.89:0/1805808223 >> v1:172.21.15.89:6789/0 conn(0x7f596812f8b0 legacy=0x7f596812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5967fff700 1 -- 172.21.15.89:0/1805808223 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5968071a40 con 0x7f5968130430 2023-09-30T22:17:35.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.265+0000 7f5967fff700 1 --2- 172.21.15.89:0/1805808223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5968130430 0x7f5968137c30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5960009e50 tx=0x7f596000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b2bbcc6fc7ae6a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:35.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f59667fc700 1 -- 172.21.15.89:0/1805808223 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000e070 con 0x7f5968130430 2023-09-30T22:17:35.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f59667fc700 1 -- 172.21.15.89:0/1805808223 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596000b9c0 con 0x7f5968130430 2023-09-30T22:17:35.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f59667fc700 1 -- 172.21.15.89:0/1805808223 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000bd70 con 0x7f5968130430 2023-09-30T22:17:35.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d580a2fc0 con 0x7f5d5809d6f0 2023-09-30T22:17:35.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f596dd07700 1 -- 172.21.15.89:0/1805808223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5968130430 msgr2=0x7f5968137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f596dd07700 1 --2- 172.21.15.89:0/1805808223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5968130430 0x7f5968137c30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5960009e50 tx=0x7f596000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f596dd07700 1 -- 172.21.15.89:0/1805808223 shutdown_connections 2023-09-30T22:17:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f596dd07700 1 --2- 172.21.15.89:0/1805808223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5968130430 0x7f5968137c30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f596dd07700 1 --2- 172.21.15.89:0/1805808223 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 0x7f596806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f596dd07700 1 -- 172.21.15.89:0/1805808223 >> 172.21.15.89:0/1805808223 conn(0x7f59680a2aa0 msgr2=0x7f59680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f596dd07700 1 -- 172.21.15.89:0/1805808223 shutdown_connections 2023-09-30T22:17:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.266+0000 7f596dd07700 1 -- 172.21.15.89:0/1805808223 wait complete. 2023-09-30T22:17:35.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.267+0000 7f596dd07700 1 Processor -- start 2023-09-30T22:17:35.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.267+0000 7f596dd07700 1 -- start start 2023-09-30T22:17:35.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f596dd07700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 0x7f596809bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f596dd07700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f596812f8b0 0x7f596809c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f596dd07700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5968130430 0x7f59681c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:35.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f596dd07700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596809da70 con 0x7f596812f8b0 2023-09-30T22:17:35.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f596dd07700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5968129840 con 0x7f596812cf60 2023-09-30T22:17:35.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f596dd07700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f596809e670 con 0x7f5968130430 2023-09-30T22:17:35.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5966ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f596812f8b0 0x7f596809c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5966ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f596812f8b0 0x7f596809c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54104/0 (socket says 172.21.15.89:54104) 2023-09-30T22:17:35.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5966ffd700 1 -- 172.21.15.89:0/4022490616 learned_addr learned my addr 172.21.15.89:0/4022490616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:35.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5967fff700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5968130430 0x7f59681c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f59677fe700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 0x7f596809bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:35.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5966ffd700 1 -- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5968130430 msgr2=0x7f59681c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5966ffd700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5968130430 0x7f59681c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5966ffd700 1 -- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 msgr2=0x7f596809bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5966ffd700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 0x7f596809bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.268+0000 7f5966ffd700 1 -- 172.21.15.89:0/4022490616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5960009b30 con 0x7f596812f8b0 2023-09-30T22:17:35.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.269+0000 7f5966ffd700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f596812f8b0 0x7f596809c110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f595000c550 tx=0x7f595000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:35.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.271+0000 7f595ffff700 1 -- 172.21.15.89:0/4022490616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500054e0 con 0x7f596812f8b0 2023-09-30T22:17:35.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.271+0000 7f5d3ffff700 1 -- 172.21.15.89:0/437599094 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d48013720 con 0x7f5d5809d6f0 2023-09-30T22:17:35.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.271+0000 7f595ffff700 1 -- 172.21.15.89:0/4022490616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5950003680 con 0x7f596812f8b0 2023-09-30T22:17:35.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.271+0000 7f595ffff700 1 -- 172.21.15.89:0/4022490616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950003a30 con 0x7f596812f8b0 2023-09-30T22:17:35.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.271+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596809b710 con 0x7f596812f8b0 2023-09-30T22:17:35.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.271+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596812a000 con 0x7f596812f8b0 2023-09-30T22:17:35.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.272+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59680705f0 con 0x7f596812f8b0 2023-09-30T22:17:35.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.275+0000 7f595ffff700 1 -- 172.21.15.89:0/4022490616 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5950003b90 con 0x7f596812f8b0 2023-09-30T22:17:35.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.275+0000 7f595ffff700 1 -- 172.21.15.89:0/4022490616 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f595003eec0 con 0x7f596812f8b0 2023-09-30T22:17:35.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.279+0000 7f595ffff700 1 -- 172.21.15.89:0/4022490616 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5950003e40 con 0x7f596812f8b0 2023-09-30T22:17:35.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.433+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6bd81a6aa0 con 0x7f6bd809f240 2023-09-30T22:17:35.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.434+0000 7f6bce7fc700 1 -- 172.21.15.89:0/1615175668 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bc8015070 con 0x7f6bd809f240 2023-09-30T22:17:35.435 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:35.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f240 msgr2=0x7f6bd81a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f240 0x7f6bd81a4180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8000c00 tx=0x7f6bc8004750 comp rx=0 tx=0).stop 2023-09-30T22:17:35.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 shutdown_connections 2023-09-30T22:17:35.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bd809f650 0x7f6bd81a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bd809e930 0x7f6bd81a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 --2- 172.21.15.89:0/1615175668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bd809f240 0x7f6bd81a4180 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 >> 172.21.15.89:0/1615175668 conn(0x7f6bd80a4210 msgr2=0x7f6bd81b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:35.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 shutdown_connections 2023-09-30T22:17:35.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.437+0000 7f6be01fc700 1 -- 172.21.15.89:0/1615175668 wait complete. 2023-09-30T22:17:35.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:35.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.520+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5d5809c710 con 0x7f5d5809d6f0 2023-09-30T22:17:35.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.520+0000 7f5d3ffff700 1 -- 172.21.15.89:0/437599094 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d48021020 con 0x7f5d5809d6f0 2023-09-30T22:17:35.524 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:35.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5809d6f0 msgr2=0x7f5d581a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 --2- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5809d6f0 0x7f5d581a5d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800ec10 tx=0x7f5d48009e70 comp rx=0 tx=0).stop 2023-09-30T22:17:35.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 shutdown_connections 2023-09-30T22:17:35.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 --2- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5809e410 0x7f5d581a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 --2- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5809d6f0 0x7f5d581a5d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 --2- 172.21.15.89:0/437599094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5809e000 0x7f5d581a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 >> 172.21.15.89:0/437599094 conn(0x7f5d58066f50 msgr2=0x7f5d581bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:35.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 shutdown_connections 2023-09-30T22:17:35.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.523+0000 7f5d5de2a700 1 -- 172.21.15.89:0/437599094 wait complete. 2023-09-30T22:17:35.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:35.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.535+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f596809ec70 con 0x7f596812f8b0 2023-09-30T22:17:35.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.535+0000 7f595ffff700 1 -- 172.21.15.89:0/4022490616 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f596809ec70 con 0x7f596812f8b0 2023-09-30T22:17:35.541 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:35.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.537+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f596812f8b0 msgr2=0x7f596809c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:35.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.537+0000 7f596dd07700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f596812f8b0 0x7f596809c110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f595000c550 tx=0x7f595000c860 comp rx=0 tx=0).stop 2023-09-30T22:17:35.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.537+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 shutdown_connections 2023-09-30T22:17:35.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.537+0000 7f596dd07700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5968130430 0x7f59681c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.537+0000 7f596dd07700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f596812cf60 0x7f596809bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.537+0000 7f596dd07700 1 --2- 172.21.15.89:0/4022490616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f596812f8b0 0x7f596809c110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:35.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.537+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 >> 172.21.15.89:0/4022490616 conn(0x7f59680a2aa0 msgr2=0x7f59681b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:35.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.538+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 shutdown_connections 2023-09-30T22:17:35.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:35.538+0000 7f596dd07700 1 -- 172.21.15.89:0/4022490616 wait complete. 2023-09-30T22:17:35.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:36.193 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:36.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab894f1700 1 Processor -- start 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab894f1700 1 -- start start 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab894f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab84133960 0x7fab84133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab894f1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab8409de50 0x7fab8409e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab894f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab8409e7c0 0x7fab8413a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab894f1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab8406fab0 con 0x7fab8409de50 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab894f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab84133550 con 0x7fab8409e7c0 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab894f1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab84071e20 con 0x7fab84133960 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab82ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab84133960 0x7fab84133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab827fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab8409de50 0x7fab8409e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab837fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab8409e7c0 0x7fab8413a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab827fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab8409de50 0x7fab8409e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54108/0 (socket says 172.21.15.89:54108) 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab82ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab84133960 0x7fab84133d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48666/0 (socket says 172.21.15.89:48666) 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.324+0000 7fab827fc700 1 -- 172.21.15.89:0/1685380224 learned_addr learned my addr 172.21.15.89:0/1685380224 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab837fe700 1 -- 172.21.15.89:0/1685380224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab84133960 msgr2=0x7fab84133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab837fe700 1 --2- 172.21.15.89:0/1685380224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab84133960 0x7fab84133d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab837fe700 1 -- 172.21.15.89:0/1685380224 >> v2:172.21.15.89:3300/0 conn(0x7fab8409de50 msgr2=0x7fab8409e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab837fe700 1 --2- 172.21.15.89:0/1685380224 >> v2:172.21.15.89:3300/0 conn(0x7fab8409de50 0x7fab8409e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab837fe700 1 -- 172.21.15.89:0/1685380224 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8409ec60 con 0x7fab8409e7c0 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab827fc700 1 --2- 172.21.15.89:0/1685380224 >> v2:172.21.15.89:3300/0 conn(0x7fab8409de50 0x7fab8409e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab837fe700 1 --2- 172.21.15.89:0/1685380224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab8409e7c0 0x7fab8413a7e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e44249a9e7118b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab81ffb700 1 -- 172.21.15.89:0/1685380224 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00e070 con 0x7fab8409e7c0 2023-09-30T22:17:36.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab81ffb700 1 -- 172.21.15.89:0/1685380224 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c00bb20 con 0x7fab8409e7c0 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab81ffb700 1 -- 172.21.15.89:0/1685380224 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c012430 con 0x7fab8409e7c0 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab894f1700 1 -- 172.21.15.89:0/1685380224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab8409e7c0 msgr2=0x7fab8413a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab894f1700 1 --2- 172.21.15.89:0/1685380224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab8409e7c0 0x7fab8413a7e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab894f1700 1 -- 172.21.15.89:0/1685380224 shutdown_connections 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab894f1700 1 --2- 172.21.15.89:0/1685380224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab8409e7c0 0x7fab8413a7e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab894f1700 1 --2- 172.21.15.89:0/1685380224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab84133960 0x7fab84133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab894f1700 1 --2- 172.21.15.89:0/1685380224 >> v2:172.21.15.89:3300/0 conn(0x7fab8409de50 0x7fab8409e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.325+0000 7fab894f1700 1 -- 172.21.15.89:0/1685380224 >> 172.21.15.89:0/1685380224 conn(0x7fab840a2610 msgr2=0x7fab840a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.326+0000 7fab894f1700 1 -- 172.21.15.89:0/1685380224 shutdown_connections 2023-09-30T22:17:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.326+0000 7fab894f1700 1 -- 172.21.15.89:0/1685380224 wait complete. 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab894f1700 1 Processor -- start 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab894f1700 1 -- start start 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab894f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab8409de50 0x7fab8411c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab894f1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8409e7c0 0x7fab8411cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab894f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab84133960 0x7fab8411d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab894f1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8411fe80 con 0x7fab8409e7c0 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab894f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8411e1b0 con 0x7fab84133960 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab894f1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab8411f400 con 0x7fab8409de50 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab827fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8409e7c0 0x7fab8411cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab827fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8409e7c0 0x7fab8411cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54114/0 (socket says 172.21.15.89:54114) 2023-09-30T22:17:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab837fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab84133960 0x7fab8411d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab827fc700 1 -- 172.21.15.89:0/4005054274 learned_addr learned my addr 172.21.15.89:0/4005054274 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab82ffd700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab8409de50 0x7fab8411c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab827fc700 1 -- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab8409de50 msgr2=0x7fab8411c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab827fc700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab8409de50 0x7fab8411c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab827fc700 1 -- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab84133960 msgr2=0x7fab8411d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab827fc700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab84133960 0x7fab8411d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab827fc700 1 -- 172.21.15.89:0/4005054274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab78009550 con 0x7fab8409e7c0 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.327+0000 7fab837fe700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab84133960 0x7fab8411d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.328+0000 7fab827fc700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8409e7c0 0x7fab8411cae0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fab7800e9c0 tx=0x7fab7800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.328+0000 7fab6b7fe700 1 -- 172.21.15.89:0/4005054274 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7800ee60 con 0x7fab8409e7c0 2023-09-30T22:17:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.328+0000 7fab6b7fe700 1 -- 172.21.15.89:0/4005054274 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7800c850 con 0x7fab8409e7c0 2023-09-30T22:17:36.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.328+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c009a70 con 0x7fab8409e7c0 2023-09-30T22:17:36.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.328+0000 7fab6b7fe700 1 -- 172.21.15.89:0/4005054274 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7800cc00 con 0x7fab8409e7c0 2023-09-30T22:17:36.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.328+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8406ea50 con 0x7fab8409e7c0 2023-09-30T22:17:36.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.328+0000 7fab6b7fe700 1 -- 172.21.15.89:0/4005054274 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab7800cd60 con 0x7fab8409e7c0 2023-09-30T22:17:36.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.329+0000 7fab6b7fe700 1 -- 172.21.15.89:0/4005054274 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab7803f700 con 0x7fab8409e7c0 2023-09-30T22:17:36.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.329+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8406f420 con 0x7fab8409e7c0 2023-09-30T22:17:36.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.333+0000 7fab6b7fe700 1 -- 172.21.15.89:0/4005054274 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab7801bdf0 con 0x7fab8409e7c0 2023-09-30T22:17:36.457 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.523+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fab840747f0 con 0x7fab8409e7c0 2023-09-30T22:17:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.523+0000 7fab6b7fe700 1 -- 172.21.15.89:0/4005054274 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab7802a090 con 0x7fab8409e7c0 2023-09-30T22:17:36.525 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8409e7c0 msgr2=0x7fab8411cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8409e7c0 0x7fab8411cae0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fab7800e9c0 tx=0x7fab7800ecd0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 shutdown_connections 2023-09-30T22:17:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab84133960 0x7fab8411d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab8409de50 0x7fab8411c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 --2- 172.21.15.89:0/4005054274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8409e7c0 0x7fab8411cae0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 >> 172.21.15.89:0/4005054274 conn(0x7fab840a2610 msgr2=0x7fab841bac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 shutdown_connections 2023-09-30T22:17:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.525+0000 7fab894f1700 1 -- 172.21.15.89:0/4005054274 wait complete. 2023-09-30T22:17:36.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:36.542 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:36.551 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:36.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.583+0000 7fa0dd731700 1 Processor -- start 2023-09-30T22:17:36.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.583+0000 7fa0dd731700 1 -- start start 2023-09-30T22:17:36.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0dd731700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a0a30 0x7fa0d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0dd731700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa0d80a1340 0x7fa0d8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0dd731700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d809edd0 con 0x7fa0d8133fa0 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0dd731700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d809e1f0 con 0x7fa0d80a1340 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0dd731700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d809dc50 con 0x7fa0d80a0a30 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a0a30 0x7fa0d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa0d80a1340 0x7fa0d8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a0a30 0x7fa0d80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43956/0 (socket says 172.21.15.89:43956) 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d6ffd700 1 -- 172.21.15.89:0/18946699 learned_addr learned my addr 172.21.15.89:0/18946699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d6ffd700 1 -- 172.21.15.89:0/18946699 >> v2:172.21.15.89:3300/0 conn(0x7fa0d80a1340 msgr2=0x7fa0d8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d6ffd700 1 --2- 172.21.15.89:0/18946699 >> v2:172.21.15.89:3300/0 conn(0x7fa0d80a1340 0x7fa0d8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d6ffd700 1 -- 172.21.15.89:0/18946699 >> v1:172.21.15.89:6789/0 conn(0x7fa0d8133fa0 legacy=0x7fa0d813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d6ffd700 1 -- 172.21.15.89:0/18946699 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d80a1fe0 con 0x7fa0d80a0a30 2023-09-30T22:17:36.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d67fc700 1 --2- 172.21.15.89:0/18946699 >> v2:172.21.15.89:3300/0 conn(0x7fa0d80a1340 0x7fa0d8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:36.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d6ffd700 1 --2- 172.21.15.89:0/18946699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a0a30 0x7fa0d80a0e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80098f0 tx=0x7fa0c8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc9533697c8fb25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:36.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d5ffb700 1 -- 172.21.15.89:0/18946699 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8011830 con 0x7fa0d80a0a30 2023-09-30T22:17:36.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d5ffb700 1 -- 172.21.15.89:0/18946699 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c8011990 con 0x7fa0d80a0a30 2023-09-30T22:17:36.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.584+0000 7fa0d5ffb700 1 -- 172.21.15.89:0/18946699 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8011d20 con 0x7fa0d80a0a30 2023-09-30T22:17:36.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.585+0000 7fa0dd731700 1 -- 172.21.15.89:0/18946699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a0a30 msgr2=0x7fa0d80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.585+0000 7fa0dd731700 1 --2- 172.21.15.89:0/18946699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a0a30 0x7fa0d80a0e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80098f0 tx=0x7fa0c8009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.585+0000 7fa0dd731700 1 -- 172.21.15.89:0/18946699 shutdown_connections 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.585+0000 7fa0dd731700 1 --2- 172.21.15.89:0/18946699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a0a30 0x7fa0d80a0e00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.585+0000 7fa0dd731700 1 --2- 172.21.15.89:0/18946699 >> v2:172.21.15.89:3300/0 conn(0x7fa0d80a1340 0x7fa0d8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.585+0000 7fa0dd731700 1 -- 172.21.15.89:0/18946699 >> 172.21.15.89:0/18946699 conn(0x7fa0d80a6550 msgr2=0x7fa0d80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.585+0000 7fa0dd731700 1 -- 172.21.15.89:0/18946699 shutdown_connections 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.585+0000 7fa0dd731700 1 -- 172.21.15.89:0/18946699 wait complete. 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0dd731700 1 Processor -- start 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0dd731700 1 -- start start 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0dd731700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0d80a0a30 0x7fa0d811e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0dd731700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a1340 0x7fa0d811ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0dd731700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0d8133fa0 0x7fa0d811f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0dd731700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d8121f50 con 0x7fa0d8133fa0 2023-09-30T22:17:36.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0dd731700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d8120430 con 0x7fa0d80a1340 2023-09-30T22:17:36.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0dd731700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d80a37e0 con 0x7fa0d80a0a30 2023-09-30T22:17:36.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0d67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a1340 0x7fa0d811ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0d67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a1340 0x7fa0d811ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43972/0 (socket says 172.21.15.89:43972) 2023-09-30T22:17:36.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0d67fc700 1 -- 172.21.15.89:0/2765328621 learned_addr learned my addr 172.21.15.89:0/2765328621 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:36.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.587+0000 7fa0d77fe700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0d8133fa0 0x7fa0d811f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0d67fc700 1 -- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0d80a0a30 msgr2=0x7fa0d811e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0d67fc700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0d80a0a30 0x7fa0d811e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0d67fc700 1 -- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0d8133fa0 msgr2=0x7fa0d811f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0d67fc700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0d8133fa0 0x7fa0d811f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0d67fc700 1 -- 172.21.15.89:0/2765328621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0094a0 con 0x7fa0d80a1340 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0d77fe700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0d8133fa0 0x7fa0d811f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0d67fc700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a1340 0x7fa0d811ece0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00ec10 tx=0x7fa0cc009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0bf7fe700 1 -- 172.21.15.89:0/2765328621 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00c940 con 0x7fa0d80a1340 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0bf7fe700 1 -- 172.21.15.89:0/2765328621 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc00caa0 con 0x7fa0d80a1340 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0bf7fe700 1 -- 172.21.15.89:0/2765328621 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00ce10 con 0x7fa0d80a1340 2023-09-30T22:17:36.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c8009550 con 0x7fa0d80a1340 2023-09-30T22:17:36.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.588+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d8120ad0 con 0x7fa0d80a1340 2023-09-30T22:17:36.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.589+0000 7fa0bf7fe700 1 -- 172.21.15.89:0/2765328621 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa0cc013470 con 0x7fa0d80a1340 2023-09-30T22:17:36.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.589+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d8074ea0 con 0x7fa0d80a1340 2023-09-30T22:17:36.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.589+0000 7fa0bf7fe700 1 -- 172.21.15.89:0/2765328621 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa0cc03fcb0 con 0x7fa0d80a1340 2023-09-30T22:17:36.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.598+0000 7fa0bf7fe700 1 -- 172.21.15.89:0/2765328621 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa0cc01b3b0 con 0x7fa0d80a1340 2023-09-30T22:17:36.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.673+0000 7f5d11167700 1 Processor -- start 2023-09-30T22:17:36.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.673+0000 7f5d11167700 1 -- start start 2023-09-30T22:17:36.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d11167700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c133a80 0x7f5d0c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d11167700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d0c12ff20 0x7f5d0c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d11167700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d11167700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c06c570 con 0x7f5d0c12ff20 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d11167700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c06bc00 con 0x7f5d0c133a80 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d11167700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0a0040 con 0x7f5d0c130870 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c133a80 0x7f5d0c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0b59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c133a80 0x7f5d0c133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43976/0 (socket says 172.21.15.89:43976) 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0ad9d700 1 -- 172.21.15.89:0/758345100 learned_addr learned my addr 172.21.15.89:0/758345100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0a59c700 1 --2- 172.21.15.89:0/758345100 >> v2:172.21.15.89:3300/0 conn(0x7f5d0c12ff20 0x7f5d0c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0ad9d700 1 -- 172.21.15.89:0/758345100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 msgr2=0x7f5d0c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:36.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0ad9d700 1 --2- 172.21.15.89:0/758345100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0ad9d700 1 -- 172.21.15.89:0/758345100 >> v2:172.21.15.89:3300/0 conn(0x7f5d0c12ff20 msgr2=0x7f5d0c130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:36.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0ad9d700 1 --2- 172.21.15.89:0/758345100 >> v2:172.21.15.89:3300/0 conn(0x7f5d0c12ff20 0x7f5d0c130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0ad9d700 1 -- 172.21.15.89:0/758345100 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c133860 con 0x7f5d0c133a80 2023-09-30T22:17:36.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0b59e700 1 --2- 172.21.15.89:0/758345100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:36.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.674+0000 7f5d0a59c700 1 --2- 172.21.15.89:0/758345100 >> v2:172.21.15.89:3300/0 conn(0x7f5d0c12ff20 0x7f5d0c130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:36.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d0ad9d700 1 --2- 172.21.15.89:0/758345100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c133a80 0x7f5d0c133e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0098f0 tx=0x7f5cfc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a86b3fd990c64e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:36.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d09d9b700 1 -- 172.21.15.89:0/758345100 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc011830 con 0x7f5d0c133a80 2023-09-30T22:17:36.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d09d9b700 1 -- 172.21.15.89:0/758345100 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc011990 con 0x7f5d0c133a80 2023-09-30T22:17:36.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d09d9b700 1 -- 172.21.15.89:0/758345100 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc011d20 con 0x7f5d0c133a80 2023-09-30T22:17:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d11167700 1 -- 172.21.15.89:0/758345100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c133a80 msgr2=0x7f5d0c133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d11167700 1 --2- 172.21.15.89:0/758345100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c133a80 0x7f5d0c133e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0098f0 tx=0x7f5cfc009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d11167700 1 -- 172.21.15.89:0/758345100 shutdown_connections 2023-09-30T22:17:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d11167700 1 --2- 172.21.15.89:0/758345100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d11167700 1 --2- 172.21.15.89:0/758345100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c133a80 0x7f5d0c133e50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d11167700 1 --2- 172.21.15.89:0/758345100 >> v2:172.21.15.89:3300/0 conn(0x7f5d0c12ff20 0x7f5d0c130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.675+0000 7f5d11167700 1 -- 172.21.15.89:0/758345100 >> 172.21.15.89:0/758345100 conn(0x7f5d0c0a4210 msgr2=0x7f5d0c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.676+0000 7f5d11167700 1 -- 172.21.15.89:0/758345100 shutdown_connections 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.676+0000 7f5d11167700 1 -- 172.21.15.89:0/758345100 wait complete. 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.677+0000 7f5d11167700 1 Processor -- start 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.677+0000 7f5d11167700 1 -- start start 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d11167700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c12ff20 0x7f5d0c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d11167700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d11167700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c133a80 0x7f5d0c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d11167700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0a21e0 con 0x7f5d0c133a80 2023-09-30T22:17:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d11167700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c09df60 con 0x7f5d0c12ff20 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d11167700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c11fcc0 con 0x7f5d0c130870 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48706/0 (socket says 172.21.15.89:48706) 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 -- 172.21.15.89:0/2382233788 learned_addr learned my addr 172.21.15.89:0/2382233788 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0b59e700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c133a80 0x7f5d0c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0ad9d700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c12ff20 0x7f5d0c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 -- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c12ff20 msgr2=0x7f5d0c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c12ff20 0x7f5d0c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 -- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c133a80 msgr2=0x7f5d0c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c133a80 0x7f5d0c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 -- 172.21.15.89:0/2382233788 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d000094a0 con 0x7f5d0c130870 2023-09-30T22:17:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0b59e700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c133a80 0x7f5d0c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.678+0000 7f5d0a59c700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c09c9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000ebb0 tx=0x7f5d0000eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.679+0000 7f5cf37fe700 1 -- 172.21.15.89:0/2382233788 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000c850 con 0x7f5d0c130870 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.679+0000 7f5cf37fe700 1 -- 172.21.15.89:0/2382233788 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0000c9b0 con 0x7f5d0c130870 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.679+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d04009a70 con 0x7f5d0c130870 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.679+0000 7f5cf37fe700 1 -- 172.21.15.89:0/2382233788 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000cd20 con 0x7f5d0c130870 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.679+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc009550 con 0x7f5d0c130870 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.680+0000 7f5cf37fe700 1 -- 172.21.15.89:0/2382233788 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d00013470 con 0x7f5d0c130870 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.680+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c074ea0 con 0x7f5d0c130870 2023-09-30T22:17:36.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.680+0000 7f5cf37fe700 1 -- 172.21.15.89:0/2382233788 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5d0003f780 con 0x7f5d0c130870 2023-09-30T22:17:36.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.688+0000 7f5cf37fe700 1 -- 172.21.15.89:0/2382233788 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d000232f0 con 0x7f5d0c130870 2023-09-30T22:17:36.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.712+0000 7fddf351f700 1 Processor -- start 2023-09-30T22:17:36.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.712+0000 7fddf351f700 1 -- start start 2023-09-30T22:17:36.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf351f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fddec09d6f0 0x7fddec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf351f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf351f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec09f130 con 0x7fddec09e000 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf351f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddec09f520 con 0x7fddec09d6f0 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf351f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddec09e930 con 0x7fddec09e410 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf12bb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fddec09d6f0 0x7fddec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf1abc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf1abc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48722/0 (socket says 172.21.15.89:48722) 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf12bb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fddec09d6f0 0x7fddec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54166/0 (socket says 172.21.15.89:54166) 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.713+0000 7fddf1abc700 1 -- 172.21.15.89:0/3109127412 learned_addr learned my addr 172.21.15.89:0/3109127412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:36.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf1abc700 1 -- 172.21.15.89:0/3109127412 >> v2:172.21.15.89:3300/0 conn(0x7fddec09d6f0 msgr2=0x7fddec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf1abc700 1 --2- 172.21.15.89:0/3109127412 >> v2:172.21.15.89:3300/0 conn(0x7fddec09d6f0 0x7fddec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf1abc700 1 -- 172.21.15.89:0/3109127412 >> v1:172.21.15.89:6789/0 conn(0x7fddec09e000 legacy=0x7fddec130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf1abc700 1 -- 172.21.15.89:0/3109127412 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec09b2d0 con 0x7fddec09e410 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf12bb700 1 --2- 172.21.15.89:0/3109127412 >> v2:172.21.15.89:3300/0 conn(0x7fddec09d6f0 0x7fddec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf1abc700 1 --2- 172.21.15.89:0/3109127412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec130d70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009e50 tx=0x7fdde400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23a282e2f5f4a678 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fdde3fff700 1 -- 172.21.15.89:0/3109127412 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde400e070 con 0x7fddec09e410 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fdde3fff700 1 -- 172.21.15.89:0/3109127412 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde400b910 con 0x7fddec09e410 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fdde3fff700 1 -- 172.21.15.89:0/3109127412 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde400bcc0 con 0x7fddec09e410 2023-09-30T22:17:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf351f700 1 -- 172.21.15.89:0/3109127412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 msgr2=0x7fddec130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf351f700 1 --2- 172.21.15.89:0/3109127412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec130d70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009e50 tx=0x7fdde400b330 comp rx=0 tx=0).stop 2023-09-30T22:17:36.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.714+0000 7fddf351f700 1 -- 172.21.15.89:0/3109127412 shutdown_connections 2023-09-30T22:17:36.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.715+0000 7fddf351f700 1 --2- 172.21.15.89:0/3109127412 >> v2:172.21.15.89:3300/0 conn(0x7fddec09d6f0 0x7fddec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.715+0000 7fddf351f700 1 --2- 172.21.15.89:0/3109127412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec130d70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.715+0000 7fddf351f700 1 -- 172.21.15.89:0/3109127412 >> 172.21.15.89:0/3109127412 conn(0x7fddec066f50 msgr2=0x7fddec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:36.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.715+0000 7fddf351f700 1 -- 172.21.15.89:0/3109127412 shutdown_connections 2023-09-30T22:17:36.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.715+0000 7fddf351f700 1 -- 172.21.15.89:0/3109127412 wait complete. 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.715+0000 7fddf351f700 1 Processor -- start 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf351f700 1 -- start start 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf351f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddec09d6f0 0x7fddec11de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf351f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddec09e000 0x7fddec11e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf351f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec11e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf351f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec121940 con 0x7fddec09d6f0 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf351f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec120050 con 0x7fddec09e000 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf351f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddec120d10 con 0x7fddec09e410 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf12bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddec09d6f0 0x7fddec11de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf12bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddec09d6f0 0x7fddec11de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54180/0 (socket says 172.21.15.89:54180) 2023-09-30T22:17:36.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf12bb700 1 -- 172.21.15.89:0/1241269771 learned_addr learned my addr 172.21.15.89:0/1241269771 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:36.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.716+0000 7fddf1abc700 1 --2- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec11e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:36.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fddf12bb700 1 -- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 msgr2=0x7fddec11e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fddf12bb700 1 --2- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec11e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fddf12bb700 1 -- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddec09e000 msgr2=0x7fddec11e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fddf12bb700 1 --2- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddec09e000 0x7fddec11e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fddf12bb700 1 -- 172.21.15.89:0/1241269771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde80094a0 con 0x7fddec09d6f0 2023-09-30T22:17:36.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fddf12bb700 1 --2- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddec09d6f0 0x7fddec11de20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdde800ec10 tx=0x7fdde800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:36.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fdde1ffb700 1 -- 172.21.15.89:0/1241269771 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800c910 con 0x7fddec09d6f0 2023-09-30T22:17:36.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fdde1ffb700 1 -- 172.21.15.89:0/1241269771 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde800ca70 con 0x7fddec09d6f0 2023-09-30T22:17:36.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde4009b30 con 0x7fddec09d6f0 2023-09-30T22:17:36.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fdde1ffb700 1 -- 172.21.15.89:0/1241269771 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800cde0 con 0x7fddec09d6f0 2023-09-30T22:17:36.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.717+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec072500 con 0x7fddec09d6f0 2023-09-30T22:17:36.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.718+0000 7fdde1ffb700 1 -- 172.21.15.89:0/1241269771 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdde8013470 con 0x7fddec09d6f0 2023-09-30T22:17:36.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.718+0000 7fdde1ffb700 1 -- 172.21.15.89:0/1241269771 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdde803fc30 con 0x7fddec09d6f0 2023-09-30T22:17:36.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.718+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddec074ea0 con 0x7fddec09d6f0 2023-09-30T22:17:36.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.723+0000 7fdde1ffb700 1 -- 172.21.15.89:0/1241269771 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdde801b3b0 con 0x7fddec09d6f0 2023-09-30T22:17:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.795+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa0d80a28e0 con 0x7fa0d80a1340 2023-09-30T22:17:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.796+0000 7fa0bf7fe700 1 -- 172.21.15.89:0/2765328621 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa0cc011b00 con 0x7fa0d80a1340 2023-09-30T22:17:36.797 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a1340 msgr2=0x7fa0d811ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a1340 0x7fa0d811ece0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00ec10 tx=0x7fa0cc009ec0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 shutdown_connections 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0d8133fa0 0x7fa0d811f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0d80a0a30 0x7fa0d811e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 --2- 172.21.15.89:0/2765328621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0d80a1340 0x7fa0d811ece0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 >> 172.21.15.89:0/2765328621 conn(0x7fa0d80a6550 msgr2=0x7fa0d81b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 shutdown_connections 2023-09-30T22:17:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.798+0000 7fa0dd731700 1 -- 172.21.15.89:0/2765328621 wait complete. 2023-09-30T22:17:36.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.883+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5d0c120320 con 0x7f5d0c130870 2023-09-30T22:17:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.883+0000 7f5cf37fe700 1 -- 172.21.15.89:0/2382233788 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d00021070 con 0x7f5d0c130870 2023-09-30T22:17:36.884 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 msgr2=0x7f5d0c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c09c9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000ebb0 tx=0x7f5d0000eec0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 shutdown_connections 2023-09-30T22:17:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c133a80 0x7f5d0c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d0c12ff20 0x7f5d0c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 --2- 172.21.15.89:0/2382233788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d0c130870 0x7f5d0c09c9c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 >> 172.21.15.89:0/2382233788 conn(0x7f5d0c0a4210 msgr2=0x7f5d0c0a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 shutdown_connections 2023-09-30T22:17:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.885+0000 7f5d11167700 1 -- 172.21.15.89:0/2382233788 wait complete. 2023-09-30T22:17:36.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:36.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.917+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fddec11fa30 con 0x7fddec09d6f0 2023-09-30T22:17:36.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.918+0000 7fdde1ffb700 1 -- 172.21.15.89:0/1241269771 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdde8017030 con 0x7fddec09d6f0 2023-09-30T22:17:36.919 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddec09d6f0 msgr2=0x7fddec11de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 --2- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddec09d6f0 0x7fddec11de20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdde800ec10 tx=0x7fdde800ef20 comp rx=0 tx=0).stop 2023-09-30T22:17:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 shutdown_connections 2023-09-30T22:17:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 --2- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fddec09e410 0x7fddec11e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 --2- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddec09d6f0 0x7fddec11de20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 --2- 172.21.15.89:0/1241269771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fddec09e000 0x7fddec11e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 >> 172.21.15.89:0/1241269771 conn(0x7fddec066f50 msgr2=0x7fddec070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 shutdown_connections 2023-09-30T22:17:36.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:36.920+0000 7fddf351f700 1 -- 172.21.15.89:0/1241269771 wait complete. 2023-09-30T22:17:36.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:37.539 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:37.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.665+0000 7f6be0d15700 1 Processor -- start 2023-09-30T22:17:37.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.665+0000 7f6be0d15700 1 -- start start 2023-09-30T22:17:37.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6be0d15700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6bdc1305f0 0x7f6bdc1309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6be0d15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc070190 0x7f6bdc138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6be0d15700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bdc1301c0 con 0x7f6bdc06f660 2023-09-30T22:17:37.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6be0d15700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc09a2e0 con 0x7f6bdc1305f0 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6be0d15700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc071ff0 con 0x7f6bdc070190 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bdad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc070190 0x7f6bdc138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bda59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6bdc1305f0 0x7f6bdc1309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bdad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc070190 0x7f6bdc138540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48730/0 (socket says 172.21.15.89:48730) 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bda59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6bdc1305f0 0x7f6bdc1309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54188/0 (socket says 172.21.15.89:54188) 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bdad9d700 1 -- 172.21.15.89:0/3356335294 learned_addr learned my addr 172.21.15.89:0/3356335294 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bda59c700 1 -- 172.21.15.89:0/3356335294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc070190 msgr2=0x7f6bdc138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bda59c700 1 --2- 172.21.15.89:0/3356335294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc070190 0x7f6bdc138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bda59c700 1 -- 172.21.15.89:0/3356335294 >> v1:172.21.15.89:6789/0 conn(0x7f6bdc06f660 legacy=0x7f6bdc06fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bda59c700 1 -- 172.21.15.89:0/3356335294 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc070e70 con 0x7f6bdc1305f0 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bdad9d700 1 --2- 172.21.15.89:0/3356335294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc070190 0x7f6bdc138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bda59c700 1 --2- 172.21.15.89:0/3356335294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc1305f0 0x7f6bdc1309c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009840 tx=0x7f6bd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77bffa97daf737d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bd959a700 1 -- 172.21.15.89:0/3356335294 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0011830 con 0x7f6bdc1305f0 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bd959a700 1 -- 172.21.15.89:0/3356335294 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd0011990 con 0x7f6bdc1305f0 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.666+0000 7f6bd959a700 1 -- 172.21.15.89:0/3356335294 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0011d20 con 0x7f6bdc1305f0 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.667+0000 7f6be0d15700 1 -- 172.21.15.89:0/3356335294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc1305f0 msgr2=0x7f6bdc1309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.667+0000 7f6be0d15700 1 --2- 172.21.15.89:0/3356335294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc1305f0 0x7f6bdc1309c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009840 tx=0x7f6bd0009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.667+0000 7f6be0d15700 1 -- 172.21.15.89:0/3356335294 shutdown_connections 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.667+0000 7f6be0d15700 1 --2- 172.21.15.89:0/3356335294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc1305f0 0x7f6bdc1309c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.667+0000 7f6be0d15700 1 --2- 172.21.15.89:0/3356335294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc070190 0x7f6bdc138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.667+0000 7f6be0d15700 1 -- 172.21.15.89:0/3356335294 >> 172.21.15.89:0/3356335294 conn(0x7f6bdc0a2aa0 msgr2=0x7f6bdc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.667+0000 7f6be0d15700 1 -- 172.21.15.89:0/3356335294 shutdown_connections 2023-09-30T22:17:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.667+0000 7f6be0d15700 1 -- 172.21.15.89:0/3356335294 wait complete. 2023-09-30T22:17:37.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.668+0000 7f6be0d15700 1 Processor -- start 2023-09-30T22:17:37.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.668+0000 7f6be0d15700 1 -- start start 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.668+0000 7f6be0d15700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc06f660 0x7f6bdc09ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6be0d15700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc070190 0x7f6bdc09b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6be0d15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1305f0 0x7f6bdc1c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6be0d15700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc09d260 con 0x7f6bdc06f660 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6be0d15700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc071820 con 0x7f6bdc070190 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6be0d15700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc09a8c0 con 0x7f6bdc1305f0 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1305f0 0x7f6bdc1c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bd9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc070190 0x7f6bdc09b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1305f0 0x7f6bdc1c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48740/0 (socket says 172.21.15.89:48740) 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bd9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc070190 0x7f6bdc09b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44018/0 (socket says 172.21.15.89:44018) 2023-09-30T22:17:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 -- 172.21.15.89:0/905877794 learned_addr learned my addr 172.21.15.89:0/905877794 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bda59c700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc06f660 0x7f6bdc09ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 -- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc070190 msgr2=0x7f6bdc09b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc070190 0x7f6bdc09b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 -- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc06f660 msgr2=0x7f6bdc09ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc06f660 0x7f6bdc09ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 -- 172.21.15.89:0/905877794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd4009b30 con 0x7f6bdc1305f0 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bda59c700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc06f660 0x7f6bdc09ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bdad9d700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1305f0 0x7f6bdc1c8410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009f10 tx=0x7f6bd400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bcaffd700 1 -- 172.21.15.89:0/905877794 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4018070 con 0x7f6bdc1305f0 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bcaffd700 1 -- 172.21.15.89:0/905877794 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd400f040 con 0x7f6bdc1305f0 2023-09-30T22:17:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd00094a0 con 0x7f6bdc1305f0 2023-09-30T22:17:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6bcaffd700 1 -- 172.21.15.89:0/905877794 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4003c40 con 0x7f6bdc1305f0 2023-09-30T22:17:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.669+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc09b8b0 con 0x7f6bdc1305f0 2023-09-30T22:17:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.670+0000 7f6bcaffd700 1 -- 172.21.15.89:0/905877794 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6bd400c940 con 0x7f6bdc1305f0 2023-09-30T22:17:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.670+0000 7f6bcaffd700 1 -- 172.21.15.89:0/905877794 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6bd403f7d0 con 0x7f6bdc1305f0 2023-09-30T22:17:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.670+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc074ea0 con 0x7f6bdc1305f0 2023-09-30T22:17:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.675+0000 7f6bcaffd700 1 -- 172.21.15.89:0/905877794 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bd4014480 con 0x7f6bdc1305f0 2023-09-30T22:17:37.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:37.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.862+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6bdc09cde0 con 0x7f6bdc1305f0 2023-09-30T22:17:37.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.863+0000 7f6bcaffd700 1 -- 172.21.15.89:0/905877794 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bd401cbb0 con 0x7f6bdc1305f0 2023-09-30T22:17:37.863 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1305f0 msgr2=0x7f6bdc1c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1305f0 0x7f6bdc1c8410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009f10 tx=0x7f6bd400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 shutdown_connections 2023-09-30T22:17:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1305f0 0x7f6bdc1c8410 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc06f660 0x7f6bdc09ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 --2- 172.21.15.89:0/905877794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc070190 0x7f6bdc09b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 >> 172.21.15.89:0/905877794 conn(0x7f6bdc0a2aa0 msgr2=0x7f6bdc09e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 shutdown_connections 2023-09-30T22:17:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.865+0000 7f6be0d15700 1 -- 172.21.15.89:0/905877794 wait complete. 2023-09-30T22:17:37.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:37.899 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:37.934 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:37.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18ab2c1700 1 Processor -- start 2023-09-30T22:17:37.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18ab2c1700 1 -- start start 2023-09-30T22:17:37.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18ab2c1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18a40a0a30 0x7f18a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18ab2c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18ab2c1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a409edd0 con 0x7f18a4133fa0 2023-09-30T22:17:37.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18ab2c1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a409e1f0 con 0x7f18a40a0a30 2023-09-30T22:17:37.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18ab2c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a409dc50 con 0x7f18a40a1340 2023-09-30T22:17:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18a885c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18a905d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18a40a0a30 0x7f18a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18a885c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44030/0 (socket says 172.21.15.89:44030) 2023-09-30T22:17:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.941+0000 7f18a885c700 1 -- 172.21.15.89:0/2483048681 learned_addr learned my addr 172.21.15.89:0/2483048681 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f18a885c700 1 -- 172.21.15.89:0/2483048681 >> v2:172.21.15.89:3300/0 conn(0x7f18a40a0a30 msgr2=0x7f18a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f18a885c700 1 --2- 172.21.15.89:0/2483048681 >> v2:172.21.15.89:3300/0 conn(0x7f18a40a0a30 0x7f18a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f18a885c700 1 -- 172.21.15.89:0/2483048681 >> v1:172.21.15.89:6789/0 conn(0x7f18a4133fa0 legacy=0x7f18a413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f18a885c700 1 -- 172.21.15.89:0/2483048681 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40a1fe0 con 0x7f18a40a1340 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f18a905d700 1 --2- 172.21.15.89:0/2483048681 >> v2:172.21.15.89:3300/0 conn(0x7f18a40a0a30 0x7f18a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f18a885c700 1 --2- 172.21.15.89:0/2483048681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a4133a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18940098f0 tx=0x7f1894009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f093ddf336b6ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f189bfff700 1 -- 172.21.15.89:0/2483048681 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894011830 con 0x7f18a40a1340 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f189bfff700 1 -- 172.21.15.89:0/2483048681 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1894011990 con 0x7f18a40a1340 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.942+0000 7f189bfff700 1 -- 172.21.15.89:0/2483048681 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894011d20 con 0x7f18a40a1340 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.943+0000 7f18ab2c1700 1 -- 172.21.15.89:0/2483048681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 msgr2=0x7f18a4133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.943+0000 7f18ab2c1700 1 --2- 172.21.15.89:0/2483048681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a4133a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18940098f0 tx=0x7f1894009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.943+0000 7f18ab2c1700 1 -- 172.21.15.89:0/2483048681 shutdown_connections 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.943+0000 7f18ab2c1700 1 --2- 172.21.15.89:0/2483048681 >> v2:172.21.15.89:3300/0 conn(0x7f18a40a0a30 0x7f18a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.943+0000 7f18ab2c1700 1 --2- 172.21.15.89:0/2483048681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a4133a60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.943+0000 7f18ab2c1700 1 -- 172.21.15.89:0/2483048681 >> 172.21.15.89:0/2483048681 conn(0x7f18a40a6550 msgr2=0x7f18a40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.943+0000 7f18ab2c1700 1 -- 172.21.15.89:0/2483048681 shutdown_connections 2023-09-30T22:17:37.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.943+0000 7f18ab2c1700 1 -- 172.21.15.89:0/2483048681 wait complete. 2023-09-30T22:17:37.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.944+0000 7f18ab2c1700 1 Processor -- start 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.944+0000 7f18ab2c1700 1 -- start start 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18ab2c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a40a0a30 0x7f18a411e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18ab2c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a411ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18ab2c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a4133fa0 0x7f18a411f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18ab2c1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4120bd0 con 0x7f18a40a0a30 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18ab2c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4120480 con 0x7f18a40a1340 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18ab2c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40a3850 con 0x7f18a4133fa0 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a885c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a411ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a885c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a411ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44044/0 (socket says 172.21.15.89:44044) 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a885c700 1 -- 172.21.15.89:0/786282483 learned_addr learned my addr 172.21.15.89:0/786282483 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a905d700 1 --2- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a40a0a30 0x7f18a411e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a885c700 1 -- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a4133fa0 msgr2=0x7f18a411f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a885c700 1 --2- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a4133fa0 0x7f18a411f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a885c700 1 -- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a40a0a30 msgr2=0x7f18a411e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a885c700 1 --2- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a40a0a30 0x7f18a411e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.945+0000 7f18a885c700 1 -- 172.21.15.89:0/786282483 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a00094a0 con 0x7f18a40a1340 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.946+0000 7f18a885c700 1 --2- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a411ed30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.946+0000 7f1899ffb700 1 -- 172.21.15.89:0/786282483 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894004480 con 0x7f18a40a1340 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.946+0000 7f1899ffb700 1 -- 172.21.15.89:0/786282483 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189401f070 con 0x7f18a40a1340 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.946+0000 7f1899ffb700 1 -- 172.21.15.89:0/786282483 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940038c0 con 0x7f18a40a1340 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.946+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1894009550 con 0x7f18a40a1340 2023-09-30T22:17:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.946+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a4121fa0 con 0x7f18a40a1340 2023-09-30T22:17:37.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.947+0000 7f1899ffb700 1 -- 172.21.15.89:0/786282483 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1894021030 con 0x7f18a40a1340 2023-09-30T22:17:37.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.947+0000 7f1899ffb700 1 -- 172.21.15.89:0/786282483 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1894040ae0 con 0x7f18a40a1340 2023-09-30T22:17:37.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.947+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a4074ea0 con 0x7f18a40a1340 2023-09-30T22:17:37.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:37.953+0000 7f1899ffb700 1 -- 172.21.15.89:0/786282483 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1894015020 con 0x7f18a40a1340 2023-09-30T22:17:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.026+0000 7fa2b1e78700 1 Processor -- start 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2b1e78700 1 -- start start 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2b1e78700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09e850 0x7fa2ac09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2b1e78700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f160 0x7fa2ac133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2b1e78700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac06bf30 con 0x7fa2ac09f5b0 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2b1e78700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac09ffb0 con 0x7fa2ac09f160 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2b1e78700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac06cb10 con 0x7fa2ac09e850 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f160 0x7fa2ac133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f160 0x7fa2ac133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44058/0 (socket says 172.21.15.89:44058) 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2aaffd700 1 -- 172.21.15.89:0/3150387479 learned_addr learned my addr 172.21.15.89:0/3150387479 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2ab7fe700 1 --2- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09e850 0x7fa2ac09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2aaffd700 1 -- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09e850 msgr2=0x7fa2ac09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2aaffd700 1 --2- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09e850 0x7fa2ac09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2aaffd700 1 -- 172.21.15.89:0/3150387479 >> v1:172.21.15.89:6789/0 conn(0x7fa2ac09f5b0 legacy=0x7fa2ac134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2aaffd700 1 -- 172.21.15.89:0/3150387479 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac0a0270 con 0x7fa2ac09f160 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2aa7fc700 1 -- 172.21.15.89:0/3150387479 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681332722 0 0) 0x7fa2ac06bf30 con 0x7fa2ac09f5b0 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.027+0000 7fa2ab7fe700 1 --2- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09e850 0x7fa2ac09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2aaffd700 1 --2- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f160 0x7fa2ac133bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009920 tx=0x7fa2a0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3574fd135152dc45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2aa7fc700 1 -- 172.21.15.89:0/3150387479 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000ba60 con 0x7fa2ac09f160 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2aa7fc700 1 -- 172.21.15.89:0/3150387479 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a000bbc0 con 0x7fa2ac09f160 2023-09-30T22:17:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2aa7fc700 1 -- 172.21.15.89:0/3150387479 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0009e70 con 0x7fa2ac09f160 2023-09-30T22:17:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2b1e78700 1 -- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f160 msgr2=0x7fa2ac133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2b1e78700 1 --2- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f160 0x7fa2ac133bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009920 tx=0x7fa2a0009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2b1e78700 1 -- 172.21.15.89:0/3150387479 shutdown_connections 2023-09-30T22:17:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2b1e78700 1 --2- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09e850 0x7fa2ac09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2b1e78700 1 --2- 172.21.15.89:0/3150387479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f160 0x7fa2ac133bf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.028+0000 7fa2b1e78700 1 -- 172.21.15.89:0/3150387479 >> 172.21.15.89:0/3150387479 conn(0x7fa2ac0a3f70 msgr2=0x7fa2ac0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.029+0000 7fa2b1e78700 1 -- 172.21.15.89:0/3150387479 shutdown_connections 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.029+0000 7fa2b1e78700 1 -- 172.21.15.89:0/3150387479 wait complete. 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.030+0000 7fa2b1e78700 1 Processor -- start 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.030+0000 7fa2b1e78700 1 -- start start 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2b1e78700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac09e850 0x7fa2ac122580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2b1e78700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09f160 0x7fa2ac122ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2b1e78700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f5b0 0x7fa2ac123000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2b1e78700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac125ea0 con 0x7fa2ac09e850 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2b1e78700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac125030 con 0x7fa2ac09f5b0 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2b1e78700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1260a0 con 0x7fa2ac09f160 2023-09-30T22:17:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac09e850 0x7fa2ac122580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09f160 0x7fa2ac122ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09f160 0x7fa2ac122ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48764/0 (socket says 172.21.15.89:48764) 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 -- 172.21.15.89:0/3286764462 learned_addr learned my addr 172.21.15.89:0/3286764462 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac09e850 0x7fa2ac122580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54228/0 (socket says 172.21.15.89:54228) 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2abfff700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f5b0 0x7fa2ac123000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 -- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f5b0 msgr2=0x7fa2ac123000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f5b0 0x7fa2ac123000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 -- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac09e850 msgr2=0x7fa2ac122580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac09e850 0x7fa2ac122580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 -- 172.21.15.89:0/3286764462 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0009550 con 0x7fa2ac09f160 2023-09-30T22:17:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2abfff700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f5b0 0x7fa2ac123000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa2aaffd700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09f160 0x7fa2ac122ac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0013000 tx=0x7fa2a0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa293fff700 1 -- 172.21.15.89:0/3286764462 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004300 con 0x7fa2ac09f160 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa293fff700 1 -- 172.21.15.89:0/3286764462 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a001f070 con 0x7fa2ac09f160 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.031+0000 7fa293fff700 1 -- 172.21.15.89:0/3286764462 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00036a0 con 0x7fa2ac09f160 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.032+0000 7fa2b1e78700 1 -- 172.21.15.89:0/3286764462 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac0720f0 con 0x7fa2ac09f160 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.032+0000 7fa2b1e78700 1 -- 172.21.15.89:0/3286764462 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac06cd30 con 0x7fa2ac09f160 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.032+0000 7fa293fff700 1 -- 172.21.15.89:0/3286764462 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2a0021030 con 0x7fa2ac09f160 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.033+0000 7fa293fff700 1 -- 172.21.15.89:0/3286764462 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa2a00409f0 con 0x7fa2ac09f160 2023-09-30T22:17:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.033+0000 7fa291ffb700 1 -- 172.21.15.89:0/3286764462 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2840052e0 con 0x7fa2ac09f160 2023-09-30T22:17:38.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.037+0000 7fa293fff700 1 -- 172.21.15.89:0/3286764462 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa2a0015020 con 0x7fa2ac09f160 2023-09-30T22:17:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.066+0000 7f53e8a83700 1 Processor -- start 2023-09-30T22:17:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.066+0000 7f53e8a83700 1 -- start start 2023-09-30T22:17:38.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e8a83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e4133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e8a83700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53e4130a40 0x7f53e413b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e8a83700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e406bc00 con 0x7f53e412ff20 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e8a83700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e406c570 con 0x7f53e4130a40 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e8a83700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40a0040 con 0x7f53e4133a80 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e1d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f53e412ff20 0x7f53e4130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57490/0 (socket says 172.21.15.89:57490) 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e1d9b700 1 -- 172.21.15.89:0/150483444 learned_addr learned my addr 172.21.15.89:0/150483444 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e2d9d700 1 --2- 172.21.15.89:0/150483444 >> v2:172.21.15.89:3300/0 conn(0x7f53e4130a40 0x7f53e413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e259c700 1 --2- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e4133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e2d9d700 1 -- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 msgr2=0x7f53e4133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e2d9d700 1 --2- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e4133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e2d9d700 1 -- 172.21.15.89:0/150483444 >> v1:172.21.15.89:6789/0 conn(0x7f53e412ff20 legacy=0x7f53e4130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e159a700 1 -- 172.21.15.89:0/150483444 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586276427 0 0) 0x7f53e406bc00 con 0x7f53e412ff20 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e2d9d700 1 -- 172.21.15.89:0/150483444 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e4133860 con 0x7f53e4130a40 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.067+0000 7f53e259c700 1 --2- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e4133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e2d9d700 1 --2- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e413b850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa11865a20fa9a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e159a700 1 -- 172.21.15.89:0/150483444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00e070 con 0x7f53e4130a40 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e159a700 1 -- 172.21.15.89:0/150483444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc00bb20 con 0x7f53e4130a40 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e159a700 1 -- 172.21.15.89:0/150483444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc012430 con 0x7f53e4130a40 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e8a83700 1 -- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 msgr2=0x7f53e413b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e8a83700 1 --2- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e413b850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc00b330 comp rx=0 tx=0).stop 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e8a83700 1 -- 172.21.15.89:0/150483444 shutdown_connections 2023-09-30T22:17:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e8a83700 1 --2- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e413b850 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e8a83700 1 --2- 172.21.15.89:0/150483444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e4133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e8a83700 1 -- 172.21.15.89:0/150483444 >> 172.21.15.89:0/150483444 conn(0x7f53e40a4210 msgr2=0x7f53e40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:38.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e8a83700 1 -- 172.21.15.89:0/150483444 shutdown_connections 2023-09-30T22:17:38.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.068+0000 7f53e8a83700 1 -- 172.21.15.89:0/150483444 wait complete. 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.069+0000 7f53e8a83700 1 Processor -- start 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.069+0000 7f53e8a83700 1 -- start start 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e8a83700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53e412ff20 0x7f53e409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e8a83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e409cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e8a83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e409d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e8a83700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e411fe10 con 0x7f53e4130a40 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e8a83700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e409dc40 con 0x7f53e412ff20 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e8a83700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4120840 con 0x7f53e4133a80 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e409cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e409cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54244/0 (socket says 172.21.15.89:54244) 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e1d9b700 1 -- 172.21.15.89:0/833632421 learned_addr learned my addr 172.21.15.89:0/833632421 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e259c700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53e412ff20 0x7f53e409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e2d9d700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e409d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e259c700 1 -- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 msgr2=0x7f53e409d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e259c700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e409d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e259c700 1 -- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 msgr2=0x7f53e409cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e259c700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e409cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e259c700 1 -- 172.21.15.89:0/833632421 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40094a0 con 0x7f53e412ff20 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e1d9b700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.070+0000 7f53e2d9d700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.071+0000 7f53e259c700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53e412ff20 0x7f53e409c6e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53d400e9e0 tx=0x7f53d400ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.071+0000 7f53d2ffd700 1 -- 172.21.15.89:0/833632421 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400ee80 con 0x7f53e412ff20 2023-09-30T22:17:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.071+0000 7f53d2ffd700 1 -- 172.21.15.89:0/833632421 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400c850 con 0x7f53e412ff20 2023-09-30T22:17:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.071+0000 7f53d2ffd700 1 -- 172.21.15.89:0/833632421 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400cc00 con 0x7f53e412ff20 2023-09-30T22:17:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.071+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53dc009a70 con 0x7f53e412ff20 2023-09-30T22:17:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.071+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e412d770 con 0x7f53e412ff20 2023-09-30T22:17:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.072+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e4074ea0 con 0x7f53e412ff20 2023-09-30T22:17:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.072+0000 7f53d2ffd700 1 -- 172.21.15.89:0/833632421 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53d400cd60 con 0x7f53e412ff20 2023-09-30T22:17:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.072+0000 7f53d2ffd700 1 -- 172.21.15.89:0/833632421 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f53d403f710 con 0x7f53e412ff20 2023-09-30T22:17:38.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.081+0000 7f53d2ffd700 1 -- 172.21.15.89:0/833632421 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53d401be10 con 0x7f53e412ff20 2023-09-30T22:17:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.151+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f18a40a28e0 con 0x7f18a40a1340 2023-09-30T22:17:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.152+0000 7f1899ffb700 1 -- 172.21.15.89:0/786282483 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1894003cb0 con 0x7f18a40a1340 2023-09-30T22:17:38.152 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:38.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 msgr2=0x7f18a411ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 --2- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a411ed30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894004750 comp rx=0 tx=0).stop 2023-09-30T22:17:38.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 shutdown_connections 2023-09-30T22:17:38.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 --2- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a4133fa0 0x7f18a411f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 --2- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a40a0a30 0x7f18a411e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 --2- 172.21.15.89:0/786282483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a40a1340 0x7f18a411ed30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 >> 172.21.15.89:0/786282483 conn(0x7f18a40a6550 msgr2=0x7f18a409a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:38.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 shutdown_connections 2023-09-30T22:17:38.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.154+0000 7f18ab2c1700 1 -- 172.21.15.89:0/786282483 wait complete. 2023-09-30T22:17:38.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:38.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.235+0000 7fa291ffb700 1 -- 172.21.15.89:0/3286764462 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa284005590 con 0x7fa2ac09f160 2023-09-30T22:17:38.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.235+0000 7fa293fff700 1 -- 172.21.15.89:0/3286764462 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2a00199a0 con 0x7fa2ac09f160 2023-09-30T22:17:38.236 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:38.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.238+0000 7fa291ffb700 1 -- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09f160 msgr2=0x7fa2ac122ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.238+0000 7fa291ffb700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09f160 0x7fa2ac122ac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0013000 tx=0x7fa2a0004750 comp rx=0 tx=0).stop 2023-09-30T22:17:38.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.238+0000 7fa291ffb700 1 -- 172.21.15.89:0/3286764462 shutdown_connections 2023-09-30T22:17:38.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.238+0000 7fa291ffb700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2ac09f5b0 0x7fa2ac123000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.238+0000 7fa291ffb700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac09e850 0x7fa2ac122580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.238+0000 7fa291ffb700 1 --2- 172.21.15.89:0/3286764462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2ac09f160 0x7fa2ac122ac0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.238+0000 7fa291ffb700 1 -- 172.21.15.89:0/3286764462 >> 172.21.15.89:0/3286764462 conn(0x7fa2ac0a3f70 msgr2=0x7fa2ac09d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:38.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.239+0000 7fa291ffb700 1 -- 172.21.15.89:0/3286764462 shutdown_connections 2023-09-30T22:17:38.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.239+0000 7fa291ffb700 1 -- 172.21.15.89:0/3286764462 wait complete. 2023-09-30T22:17:38.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:38.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.277+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f53e406d740 con 0x7f53e412ff20 2023-09-30T22:17:38.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.278+0000 7f53d2ffd700 1 -- 172.21.15.89:0/833632421 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53d401b3b0 con 0x7f53e412ff20 2023-09-30T22:17:38.279 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.279+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53e412ff20 msgr2=0x7f53e409c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.279+0000 7f53e8a83700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53e412ff20 0x7f53e409c6e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53d400e9e0 tx=0x7f53d400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.280+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 shutdown_connections 2023-09-30T22:17:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.280+0000 7f53e8a83700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53e4133a80 0x7f53e409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.280+0000 7f53e8a83700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53e412ff20 0x7f53e409c6e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.280+0000 7f53e8a83700 1 --2- 172.21.15.89:0/833632421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53e4130a40 0x7f53e409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:38.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.280+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 >> 172.21.15.89:0/833632421 conn(0x7f53e40a4210 msgr2=0x7f53e409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:38.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.280+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 shutdown_connections 2023-09-30T22:17:38.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:38.280+0000 7f53e8a83700 1 -- 172.21.15.89:0/833632421 wait complete. 2023-09-30T22:17:38.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:38.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.003+0000 7fa495cb7700 1 Processor -- start 2023-09-30T22:17:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.003+0000 7fa495cb7700 1 -- start start 2023-09-30T22:17:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.003+0000 7fa495cb7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.003+0000 7fa495cb7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa495cb7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49009f130 con 0x7fa49009e410 2023-09-30T22:17:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa495cb7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49009f520 con 0x7fa49009e000 2023-09-30T22:17:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa495cb7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa49009e930 con 0x7fa49009d6f0 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54256/0 (socket says 172.21.15.89:54256) 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48effd700 1 -- 172.21.15.89:0/4176683755 learned_addr learned my addr 172.21.15.89:0/4176683755 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48effd700 1 -- 172.21.15.89:0/4176683755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009d6f0 msgr2=0x7fa49009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48effd700 1 --2- 172.21.15.89:0/4176683755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48effd700 1 -- 172.21.15.89:0/4176683755 >> v1:172.21.15.89:6789/0 conn(0x7fa49009e410 legacy=0x7fa490130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48effd700 1 -- 172.21.15.89:0/4176683755 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49009b2d0 con 0x7fa49009e000 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48f7fe700 1 --2- 172.21.15.89:0/4176683755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48effd700 1 --2- 172.21.15.89:0/4176683755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e000 0x7fa490130660 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa4800098f0 tx=0x7fa480009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd62c0b8872d9268 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48e7fc700 1 -- 172.21.15.89:0/4176683755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480011830 con 0x7fa49009e000 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48e7fc700 1 -- 172.21.15.89:0/4176683755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa480011990 con 0x7fa49009e000 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.004+0000 7fa48e7fc700 1 -- 172.21.15.89:0/4176683755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480011d20 con 0x7fa49009e000 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.005+0000 7fa495cb7700 1 -- 172.21.15.89:0/4176683755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e000 msgr2=0x7fa490130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.005+0000 7fa495cb7700 1 --2- 172.21.15.89:0/4176683755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e000 0x7fa490130660 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa4800098f0 tx=0x7fa480009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.005+0000 7fa495cb7700 1 -- 172.21.15.89:0/4176683755 shutdown_connections 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.005+0000 7fa495cb7700 1 --2- 172.21.15.89:0/4176683755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.005+0000 7fa495cb7700 1 --2- 172.21.15.89:0/4176683755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.005+0000 7fa495cb7700 1 -- 172.21.15.89:0/4176683755 >> 172.21.15.89:0/4176683755 conn(0x7fa490066f50 msgr2=0x7fa49009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.005+0000 7fa495cb7700 1 -- 172.21.15.89:0/4176683755 shutdown_connections 2023-09-30T22:17:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.005+0000 7fa495cb7700 1 -- 172.21.15.89:0/4176683755 wait complete. 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa495cb7700 1 Processor -- start 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa495cb7700 1 -- start start 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa495cb7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa490120180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa495cb7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa4901206c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa495cb7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa490120c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa495cb7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490121d90 con 0x7fa49009e410 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa495cb7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901223b0 con 0x7fa49009d6f0 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa495cb7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901239c0 con 0x7fa49009e000 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa48f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa490120180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa48ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa490120c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa48f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa490120180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44090/0 (socket says 172.21.15.89:44090) 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa48effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa4901206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.006+0000 7fa48f7fe700 1 -- 172.21.15.89:0/2965921736 learned_addr learned my addr 172.21.15.89:0/2965921736 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa48ffff700 1 -- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 msgr2=0x7fa4901206c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa48ffff700 1 --2- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa4901206c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa48ffff700 1 -- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 msgr2=0x7fa490120180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa48ffff700 1 --2- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa490120180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa48ffff700 1 -- 172.21.15.89:0/2965921736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840094a0 con 0x7fa49009e410 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa48ffff700 1 --2- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa490120c00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa49006dff0 tx=0x7fa48800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa477fff700 1 -- 172.21.15.89:0/2965921736 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488003c00 con 0x7fa49009e410 2023-09-30T22:17:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa477fff700 1 -- 172.21.15.89:0/2965921736 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa488003d60 con 0x7fa49009e410 2023-09-30T22:17:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa480009550 con 0x7fa49009e410 2023-09-30T22:17:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa477fff700 1 -- 172.21.15.89:0/2965921736 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800a9c0 con 0x7fa49009e410 2023-09-30T22:17:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.007+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa490122580 con 0x7fa49009e410 2023-09-30T22:17:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.008+0000 7fa477fff700 1 -- 172.21.15.89:0/2965921736 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa48800ab20 con 0x7fa49009e410 2023-09-30T22:17:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.008+0000 7fa477fff700 1 -- 172.21.15.89:0/2965921736 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa48803f670 con 0x7fa49009e410 2023-09-30T22:17:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.008+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa490074ea0 con 0x7fa49009e410 2023-09-30T22:17:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.013+0000 7fa477fff700 1 -- 172.21.15.89:0/2965921736 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa488012420 con 0x7fa49009e410 2023-09-30T22:17:39.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.198+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa49006c360 con 0x7fa49009e410 2023-09-30T22:17:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.198+0000 7fa477fff700 1 -- 172.21.15.89:0/2965921736 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa488011380 con 0x7fa49009e410 2023-09-30T22:17:39.199 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:39.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.200+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 msgr2=0x7fa490120c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.200+0000 7fa495cb7700 1 --2- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa490120c00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa49006dff0 tx=0x7fa48800ce70 comp rx=0 tx=0).stop 2023-09-30T22:17:39.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.200+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 shutdown_connections 2023-09-30T22:17:39.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.200+0000 7fa495cb7700 1 --2- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa490120c00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.200+0000 7fa495cb7700 1 --2- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa490120180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.200+0000 7fa495cb7700 1 --2- 172.21.15.89:0/2965921736 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa4901206c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.200+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 >> 172.21.15.89:0/2965921736 conn(0x7fa490066f50 msgr2=0x7fa49006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:39.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.201+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 shutdown_connections 2023-09-30T22:17:39.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.201+0000 7fa495cb7700 1 -- 172.21.15.89:0/2965921736 wait complete. 2023-09-30T22:17:39.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:39.253 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:39.293 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.294+0000 7fb0b522a700 1 Processor -- start 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.294+0000 7fb0b522a700 1 -- start start 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0b522a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0b522a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0b522a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b0070940 con 0x7fb0b009f240 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0b522a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b006fd60 con 0x7fb0b009e930 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0b522a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b006fa80 con 0x7fb0b009f650 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0af59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0aed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0ae59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb0b009f240 0x7fb0b01317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57500/0 (socket says 172.21.15.89:57500) 2023-09-30T22:17:39.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0af59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b0131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48800/0 (socket says 172.21.15.89:48800) 2023-09-30T22:17:39.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0ae59c700 1 -- 172.21.15.89:0/3651250436 learned_addr learned my addr 172.21.15.89:0/3651250436 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:39.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0add9b700 1 -- 172.21.15.89:0/3651250436 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261507909 0 0) 0x7fb0b0070940 con 0x7fb0b009f240 2023-09-30T22:17:39.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0af59e700 1 -- 172.21.15.89:0/3651250436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 msgr2=0x7fb0b009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0af59e700 1 --2- 172.21.15.89:0/3651250436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0af59e700 1 -- 172.21.15.89:0/3651250436 >> v1:172.21.15.89:6789/0 conn(0x7fb0b009f240 legacy=0x7fb0b01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0af59e700 1 -- 172.21.15.89:0/3651250436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b009fb60 con 0x7fb0b009f650 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0aed9d700 1 --2- 172.21.15.89:0/3651250436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.295+0000 7fb0af59e700 1 --2- 172.21.15.89:0/3651250436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b0131f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009dc0 tx=0x7fb0a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a124327e1c02b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0add9b700 1 -- 172.21.15.89:0/3651250436 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800e070 con 0x7fb0b009f650 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0add9b700 1 -- 172.21.15.89:0/3651250436 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a800bb20 con 0x7fb0b009f650 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0add9b700 1 -- 172.21.15.89:0/3651250436 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8012430 con 0x7fb0b009f650 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0b522a700 1 -- 172.21.15.89:0/3651250436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 msgr2=0x7fb0b0131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0b522a700 1 --2- 172.21.15.89:0/3651250436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b0131f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009dc0 tx=0x7fb0a800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0b522a700 1 -- 172.21.15.89:0/3651250436 shutdown_connections 2023-09-30T22:17:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0b522a700 1 --2- 172.21.15.89:0/3651250436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0b522a700 1 --2- 172.21.15.89:0/3651250436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b0131f00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.296+0000 7fb0b522a700 1 -- 172.21.15.89:0/3651250436 >> 172.21.15.89:0/3651250436 conn(0x7fb0b00a4210 msgr2=0x7fb0b00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.297+0000 7fb0b522a700 1 -- 172.21.15.89:0/3651250436 shutdown_connections 2023-09-30T22:17:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.297+0000 7fb0b522a700 1 -- 172.21.15.89:0/3651250436 wait complete. 2023-09-30T22:17:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0b522a700 1 Processor -- start 2023-09-30T22:17:39.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0b522a700 1 -- start start 2023-09-30T22:17:39.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0b522a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0b522a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009f240 0x7fb0b006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0b522a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0b522a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b006cd60 con 0x7fb0b009f240 2023-09-30T22:17:39.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0b522a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b01a9800 con 0x7fb0b009e930 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0b522a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b01b7140 con 0x7fb0b009f650 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0ae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009f240 0x7fb0b006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0aed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0ae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009f240 0x7fb0b006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54268/0 (socket says 172.21.15.89:54268) 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0aed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44102/0 (socket says 172.21.15.89:44102) 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.298+0000 7fb0ae59c700 1 -- 172.21.15.89:0/1765583545 learned_addr learned my addr 172.21.15.89:0/1765583545 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0af59e700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0af59e700 1 -- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 msgr2=0x7fb0b006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0af59e700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0af59e700 1 -- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009f240 msgr2=0x7fb0b006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0af59e700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009f240 0x7fb0b006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0af59e700 1 -- 172.21.15.89:0/1765583545 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00094a0 con 0x7fb0b009f650 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0ae59c700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009f240 0x7fb0b006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0af59e700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b006c330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8016000 tx=0x7fb0a8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0977fe700 1 -- 172.21.15.89:0/1765583545 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004880 con 0x7fb0b009f650 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0977fe700 1 -- 172.21.15.89:0/1765583545 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8020070 con 0x7fb0b009f650 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0977fe700 1 -- 172.21.15.89:0/1765583545 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80276c0 con 0x7fb0b009f650 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a8009a70 con 0x7fb0b009f650 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.299+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b00a0190 con 0x7fb0b009f650 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.300+0000 7fb0977fe700 1 -- 172.21.15.89:0/1765583545 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0a8012430 con 0x7fb0b009f650 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.300+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b01a9480 con 0x7fb0b009f650 2023-09-30T22:17:39.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.301+0000 7fb0977fe700 1 -- 172.21.15.89:0/1765583545 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb0a8022070 con 0x7fb0b009f650 2023-09-30T22:17:39.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.305+0000 7fb0977fe700 1 -- 172.21.15.89:0/1765583545 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0b01a9480 con 0x7fb0b009f650 2023-09-30T22:17:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1597253700 1 Processor -- start 2023-09-30T22:17:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1597253700 1 -- start start 2023-09-30T22:17:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1597253700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f15900a0a30 0x7f15900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1597253700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f159013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1597253700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15900a2a70 con 0x7f159009d630 2023-09-30T22:17:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1597253700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15900a01d0 con 0x7f15900a0a30 2023-09-30T22:17:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1597253700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159009e2b0 con 0x7f1590138d80 2023-09-30T22:17:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f15957f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f159013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1594fef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f15900a0a30 0x7f15900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f15957f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f159013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44108/0 (socket says 172.21.15.89:44108) 2023-09-30T22:17:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f1594fef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f15900a0a30 0x7f15900a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54278/0 (socket says 172.21.15.89:54278) 2023-09-30T22:17:39.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.384+0000 7f15957f0700 1 -- 172.21.15.89:0/124864645 learned_addr learned my addr 172.21.15.89:0/124864645 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f15957f0700 1 -- 172.21.15.89:0/124864645 >> v2:172.21.15.89:3300/0 conn(0x7f15900a0a30 msgr2=0x7f15900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f15957f0700 1 --2- 172.21.15.89:0/124864645 >> v2:172.21.15.89:3300/0 conn(0x7f15900a0a30 0x7f15900a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f15957f0700 1 -- 172.21.15.89:0/124864645 >> v1:172.21.15.89:6789/0 conn(0x7f159009d630 legacy=0x7f15900a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f15957f0700 1 -- 172.21.15.89:0/124864645 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159009dc50 con 0x7f1590138d80 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f1594fef700 1 --2- 172.21.15.89:0/124864645 >> v2:172.21.15.89:3300/0 conn(0x7f15900a0a30 0x7f15900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f15957f0700 1 --2- 172.21.15.89:0/124864645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f159013b160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e44e3d66080efcb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f158f7fe700 1 -- 172.21.15.89:0/124864645 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800e070 con 0x7f1590138d80 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f158f7fe700 1 -- 172.21.15.89:0/124864645 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158800b910 con 0x7f1590138d80 2023-09-30T22:17:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.385+0000 7f158f7fe700 1 -- 172.21.15.89:0/124864645 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800bcc0 con 0x7f1590138d80 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.386+0000 7f1597253700 1 -- 172.21.15.89:0/124864645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 msgr2=0x7f159013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.386+0000 7f1597253700 1 --2- 172.21.15.89:0/124864645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f159013b160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.386+0000 7f1597253700 1 -- 172.21.15.89:0/124864645 shutdown_connections 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.386+0000 7f1597253700 1 --2- 172.21.15.89:0/124864645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f159013b160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.386+0000 7f1597253700 1 --2- 172.21.15.89:0/124864645 >> v2:172.21.15.89:3300/0 conn(0x7f15900a0a30 0x7f15900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.386+0000 7f1597253700 1 -- 172.21.15.89:0/124864645 >> 172.21.15.89:0/124864645 conn(0x7f15900a3050 msgr2=0x7f15900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.386+0000 7f1597253700 1 -- 172.21.15.89:0/124864645 shutdown_connections 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.386+0000 7f1597253700 1 -- 172.21.15.89:0/124864645 wait complete. 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.387+0000 7f1597253700 1 Processor -- start 2023-09-30T22:17:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.387+0000 7f1597253700 1 -- start start 2023-09-30T22:17:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f1597253700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f159009d630 0x7f1590133a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f1597253700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15900a0a30 0x7f1590133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f1597253700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f15901344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f1597253700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1590136b90 con 0x7f15900a0a30 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f1597253700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1590138090 con 0x7f1590138d80 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f1597253700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15901374b0 con 0x7f159009d630 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f15957f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f15901344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f15957f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f15901344b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44112/0 (socket says 172.21.15.89:44112) 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f15957f0700 1 -- 172.21.15.89:0/1058976286 learned_addr learned my addr 172.21.15.89:0/1058976286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f158ffff700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15900a0a30 0x7f1590133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f1594fef700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f159009d630 0x7f1590133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f15957f0700 1 -- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f159009d630 msgr2=0x7f1590133a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f15957f0700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f159009d630 0x7f1590133a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f15957f0700 1 -- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15900a0a30 msgr2=0x7f1590133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f15957f0700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15900a0a30 0x7f1590133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.388+0000 7f15957f0700 1 -- 172.21.15.89:0/1058976286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840094a0 con 0x7f1590138d80 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.389+0000 7f15957f0700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f15901344b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1588012000 tx=0x7f1588004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.389+0000 7f158d7fa700 1 -- 172.21.15.89:0/1058976286 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588004800 con 0x7f1590138d80 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.389+0000 7f158d7fa700 1 -- 172.21.15.89:0/1058976286 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1588004960 con 0x7f1590138d80 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.389+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1588009b30 con 0x7f1590138d80 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.389+0000 7f158d7fa700 1 -- 172.21.15.89:0/1058976286 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158801c650 con 0x7f1590138d80 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.389+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15901371d0 con 0x7f1590138d80 2023-09-30T22:17:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.390+0000 7f158d7fa700 1 -- 172.21.15.89:0/1058976286 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f158801c7b0 con 0x7f1590138d80 2023-09-30T22:17:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.390+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1590136e70 con 0x7f1590138d80 2023-09-30T22:17:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.390+0000 7f158d7fa700 1 -- 172.21.15.89:0/1058976286 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f15880413e0 con 0x7f1590138d80 2023-09-30T22:17:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.394+0000 7f158d7fa700 1 -- 172.21.15.89:0/1058976286 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1590136e70 con 0x7f1590138d80 2023-09-30T22:17:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.421+0000 7fbd50923700 1 Processor -- start 2023-09-30T22:17:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.421+0000 7fbd50923700 1 -- start start 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.421+0000 7fbd50923700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c09d630 0x7fbd4c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd50923700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c138d80 0x7fbd4c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd50923700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c0a2a70 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd50923700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0a01d0 con 0x7fbd4c138d80 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd50923700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c09e2b0 con 0x7fbd4c09d630 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbd4c0a0a30 0x7fbd4c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57520/0 (socket says 172.21.15.89:57520) 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4a59c700 1 -- 172.21.15.89:0/1403861128 learned_addr learned my addr 172.21.15.89:0/1403861128 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd49d9b700 1 --2- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c09d630 0x7fbd4c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4ad9d700 1 --2- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c138d80 0x7fbd4c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4959a700 1 -- 172.21.15.89:0/1403861128 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510811479 0 0) 0x7fbd4c0a2a70 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4959a700 1 -- 172.21.15.89:0/1403861128 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd340035b0 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4ad9d700 1 -- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c09d630 msgr2=0x7fbd4c0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4ad9d700 1 --2- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c09d630 0x7fbd4c0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4ad9d700 1 -- 172.21.15.89:0/1403861128 >> v1:172.21.15.89:6789/0 conn(0x7fbd4c0a0a30 legacy=0x7fbd4c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4ad9d700 1 -- 172.21.15.89:0/1403861128 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c09dc50 con 0x7fbd4c138d80 2023-09-30T22:17:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd49d9b700 1 --2- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c09d630 0x7fbd4c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4ad9d700 1 --2- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c138d80 0x7fbd4c13b160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009dc0 tx=0x7fbd4400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b42ac42ff6ec0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.422+0000 7fbd4959a700 1 -- 172.21.15.89:0/1403861128 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400e070 con 0x7fbd4c138d80 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd4959a700 1 -- 172.21.15.89:0/1403861128 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4400bb20 con 0x7fbd4c138d80 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd4959a700 1 -- 172.21.15.89:0/1403861128 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44012430 con 0x7fbd4c138d80 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd50923700 1 -- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c138d80 msgr2=0x7fbd4c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd50923700 1 --2- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c138d80 0x7fbd4c13b160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009dc0 tx=0x7fbd4400b330 comp rx=0 tx=0).stop 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd50923700 1 -- 172.21.15.89:0/1403861128 shutdown_connections 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd50923700 1 --2- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c138d80 0x7fbd4c13b160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd50923700 1 --2- 172.21.15.89:0/1403861128 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c09d630 0x7fbd4c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd50923700 1 -- 172.21.15.89:0/1403861128 >> 172.21.15.89:0/1403861128 conn(0x7fbd4c0a3050 msgr2=0x7fbd4c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd50923700 1 -- 172.21.15.89:0/1403861128 shutdown_connections 2023-09-30T22:17:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.423+0000 7fbd50923700 1 -- 172.21.15.89:0/1403861128 wait complete. 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.424+0000 7fbd50923700 1 Processor -- start 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.424+0000 7fbd50923700 1 -- start start 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd50923700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd4c09d630 0x7fbd4c1336b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd50923700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c0a0a30 0x7fbd4c133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd50923700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c138d80 0x7fbd4c134130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd50923700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c1364a0 con 0x7fbd4c09d630 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd50923700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c136e10 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd50923700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c13ab60 con 0x7fbd4c138d80 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c0a0a30 0x7fbd4c133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c0a0a30 0x7fbd4c133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44126/0 (socket says 172.21.15.89:44126) 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd4a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd4c09d630 0x7fbd4c1336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 -- 172.21.15.89:0/2922828918 learned_addr learned my addr 172.21.15.89:0/2922828918 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd4ad9d700 1 --2- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c138d80 0x7fbd4c134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 -- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c138d80 msgr2=0x7fbd4c134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 --2- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c138d80 0x7fbd4c134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 -- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd4c09d630 msgr2=0x7fbd4c1336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 --2- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd4c09d630 0x7fbd4c1336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 -- 172.21.15.89:0/2922828918 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd400094a0 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd49d9b700 1 --2- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c0a0a30 0x7fbd4c133bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000ebe0 tx=0x7fbd4000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd3affd700 1 -- 172.21.15.89:0/2922828918 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000c850 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd3affd700 1 -- 172.21.15.89:0/2922828918 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4000c9b0 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.425+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd44009a70 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.426+0000 7fbd3affd700 1 -- 172.21.15.89:0/2922828918 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000cd40 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.426+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c134a80 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.426+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c0747f0 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.427+0000 7fbd3affd700 1 -- 172.21.15.89:0/2922828918 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd40013470 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.427+0000 7fbd3affd700 1 -- 172.21.15.89:0/2922828918 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbd4003f770 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.431+0000 7fbd3affd700 1 -- 172.21.15.89:0/2922828918 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd40013720 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.506+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb0b01a9480 con 0x7fb0b009f650 2023-09-30T22:17:39.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.506+0000 7fb0977fe700 1 -- 172.21.15.89:0/1765583545 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0b01a9480 con 0x7fb0b009f650 2023-09-30T22:17:39.507 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:39.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 msgr2=0x7fb0b006c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b006c330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8016000 tx=0x7fb0a8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 shutdown_connections 2023-09-30T22:17:39.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b009f650 0x7fb0b006c330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009e930 0x7fb0b006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 --2- 172.21.15.89:0/1765583545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009f240 0x7fb0b006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 >> 172.21.15.89:0/1765583545 conn(0x7fb0b00a4210 msgr2=0x7fb0b01b8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 shutdown_connections 2023-09-30T22:17:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.508+0000 7fb0b522a700 1 -- 172.21.15.89:0/1765583545 wait complete. 2023-09-30T22:17:39.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.588+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1590136e70 con 0x7f1590138d80 2023-09-30T22:17:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.589+0000 7f158d7fa700 1 -- 172.21.15.89:0/1058976286 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1590136e70 con 0x7f1590138d80 2023-09-30T22:17:39.590 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 msgr2=0x7f15901344b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f15901344b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1588012000 tx=0x7f1588004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 shutdown_connections 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1590138d80 0x7f15901344b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f159009d630 0x7f1590133a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 --2- 172.21.15.89:0/1058976286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15900a0a30 0x7f1590133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 >> 172.21.15.89:0/1058976286 conn(0x7f15900a3050 msgr2=0x7f159009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 shutdown_connections 2023-09-30T22:17:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.591+0000 7f1597253700 1 -- 172.21.15.89:0/1058976286 wait complete. 2023-09-30T22:17:39.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.625+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbd4c072980 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.625+0000 7fbd3affd700 1 -- 172.21.15.89:0/2922828918 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd40021070 con 0x7fbd4c0a0a30 2023-09-30T22:17:39.626 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c0a0a30 msgr2=0x7fbd4c133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 --2- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c0a0a30 0x7fbd4c133bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000ebe0 tx=0x7fbd4000eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 shutdown_connections 2023-09-30T22:17:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 --2- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd4c138d80 0x7fbd4c134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 --2- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd4c09d630 0x7fbd4c1336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 --2- 172.21.15.89:0/2922828918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd4c0a0a30 0x7fbd4c133bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 >> 172.21.15.89:0/2922828918 conn(0x7fbd4c0a3050 msgr2=0x7fbd4c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 shutdown_connections 2023-09-30T22:17:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:39.627+0000 7fbd50923700 1 -- 172.21.15.89:0/2922828918 wait complete. 2023-09-30T22:17:39.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:40.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.343+0000 7f976e4d5700 1 Processor -- start 2023-09-30T22:17:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.343+0000 7f976e4d5700 1 -- start start 2023-09-30T22:17:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f976e4d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809d6f0 0x7f976809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f976e4d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f976e4d5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976809f130 con 0x7f976809e000 2023-09-30T22:17:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f976e4d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976809f520 con 0x7f976809d6f0 2023-09-30T22:17:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f976e4d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f976809e930 con 0x7f976809e410 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f97677fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f976809e000 0x7f9768130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42612/0 (socket says 172.21.15.89:42612) 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f97677fe700 1 -- 172.21.15.89:0/1770872155 learned_addr learned my addr 172.21.15.89:0/1770872155 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f976ca72700 1 --2- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9767fff700 1 --2- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809d6f0 0x7f976809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9766ffd700 1 -- 172.21.15.89:0/1770872155 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834676693 0 0) 0x7f976809f130 con 0x7f976809e000 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9766ffd700 1 -- 172.21.15.89:0/1770872155 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97500035b0 con 0x7f976809e000 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9767fff700 1 -- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 msgr2=0x7f9768130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9767fff700 1 --2- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9767fff700 1 -- 172.21.15.89:0/1770872155 >> v1:172.21.15.89:6789/0 conn(0x7f976809e000 legacy=0x7f9768130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9767fff700 1 -- 172.21.15.89:0/1770872155 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976809b2d0 con 0x7f976809d6f0 2023-09-30T22:17:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f976ca72700 1 --2- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9767fff700 1 --2- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809d6f0 0x7f976809dac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f975c009870 tx=0x7f975c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87e209eff26f064d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9766ffd700 1 -- 172.21.15.89:0/1770872155 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c011830 con 0x7f976809d6f0 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9766ffd700 1 -- 172.21.15.89:0/1770872155 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c011990 con 0x7f976809d6f0 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.344+0000 7f9766ffd700 1 -- 172.21.15.89:0/1770872155 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c011d20 con 0x7f976809d6f0 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.345+0000 7f976e4d5700 1 -- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809d6f0 msgr2=0x7f976809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.345+0000 7f976e4d5700 1 --2- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809d6f0 0x7f976809dac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f975c009870 tx=0x7f975c009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.345+0000 7f976e4d5700 1 -- 172.21.15.89:0/1770872155 shutdown_connections 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.345+0000 7f976e4d5700 1 --2- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809d6f0 0x7f976809dac0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.345+0000 7f976e4d5700 1 --2- 172.21.15.89:0/1770872155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.345+0000 7f976e4d5700 1 -- 172.21.15.89:0/1770872155 >> 172.21.15.89:0/1770872155 conn(0x7f9768066f50 msgr2=0x7f976809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.345+0000 7f976e4d5700 1 -- 172.21.15.89:0/1770872155 shutdown_connections 2023-09-30T22:17:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.345+0000 7f976e4d5700 1 -- 172.21.15.89:0/1770872155 wait complete. 2023-09-30T22:17:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.346+0000 7f976e4d5700 1 Processor -- start 2023-09-30T22:17:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.346+0000 7f976e4d5700 1 -- start start 2023-09-30T22:17:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.346+0000 7f976e4d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f976809d6f0 0x7f976811fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.346+0000 7f976e4d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809e000 0x7f9768120530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.346+0000 7f976e4d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768120a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.346+0000 7f976e4d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768123830 con 0x7f976809d6f0 2023-09-30T22:17:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.346+0000 7f976e4d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9768122460 con 0x7f976809e000 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.346+0000 7f976e4d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97681232a0 con 0x7f976809e410 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f976809d6f0 0x7f976811fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f97677fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809e000 0x7f9768120530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f976ca72700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768120a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f976809d6f0 0x7f976811fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33422/0 (socket says 172.21.15.89:33422) 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 -- 172.21.15.89:0/3723258145 learned_addr learned my addr 172.21.15.89:0/3723258145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 -- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 msgr2=0x7f9768120a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 --2- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768120a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 -- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809e000 msgr2=0x7f9768120530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 --2- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809e000 0x7f9768120530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 -- 172.21.15.89:0/3723258145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9760009a70 con 0x7f976809d6f0 2023-09-30T22:17:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f97677fe700 1 --2- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809e000 0x7f9768120530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9767fff700 1 --2- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f976809d6f0 0x7f976811fff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f975c000c00 tx=0x7f975c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9764ff9700 1 -- 172.21.15.89:0/3723258145 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c004450 con 0x7f976809d6f0 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9764ff9700 1 -- 172.21.15.89:0/3723258145 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c01f070 con 0x7f976809d6f0 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c0094a0 con 0x7f976809d6f0 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f9764ff9700 1 -- 172.21.15.89:0/3723258145 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c0038e0 con 0x7f976809d6f0 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.347+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9768123b10 con 0x7f976809d6f0 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.348+0000 7f9764ff9700 1 -- 172.21.15.89:0/3723258145 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f975c003a40 con 0x7f976809d6f0 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.348+0000 7f9764ff9700 1 -- 172.21.15.89:0/3723258145 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f975c021070 con 0x7f976809d6f0 2023-09-30T22:17:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.348+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9768074ea0 con 0x7f976809d6f0 2023-09-30T22:17:40.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.353+0000 7f9764ff9700 1 -- 172.21.15.89:0/3723258145 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f975c024380 con 0x7f976809d6f0 2023-09-30T22:17:40.522 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.536+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f976806c360 con 0x7f976809d6f0 2023-09-30T22:17:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.536+0000 7f9764ff9700 1 -- 172.21.15.89:0/3723258145 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f975c02eb90 con 0x7f976809d6f0 2023-09-30T22:17:40.537 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f976809d6f0 msgr2=0x7f976811fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 --2- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f976809d6f0 0x7f976811fff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f975c000c00 tx=0x7f975c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 shutdown_connections 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 --2- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f976809e410 0x7f9768120a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 --2- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f976809d6f0 0x7f976811fff0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 --2- 172.21.15.89:0/3723258145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f976809e000 0x7f9768120530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 >> 172.21.15.89:0/3723258145 conn(0x7f9768066f50 msgr2=0x7f976806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 shutdown_connections 2023-09-30T22:17:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.538+0000 7f976e4d5700 1 -- 172.21.15.89:0/3723258145 wait complete. 2023-09-30T22:17:40.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:40.605 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:40.641 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.646+0000 7f5acc228700 1 Processor -- start 2023-09-30T22:17:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.646+0000 7f5acc228700 1 -- start start 2023-09-30T22:17:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5acc228700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409e930 0x7f5ac409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5acc228700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ac409f240 0x7f5ac41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5acc228700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409f650 0x7f5ac4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5acc228700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac4070940 con 0x7f5ac409f240 2023-09-30T22:17:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5acc228700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac406fd60 con 0x7f5ac409e930 2023-09-30T22:17:40.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5acc228700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac406fa80 con 0x7f5ac409f650 2023-09-30T22:17:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac97c3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ac409f240 0x7f5ac41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5aca7c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409f650 0x7f5ac4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac97c3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ac409f240 0x7f5ac41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33434/0 (socket says 172.21.15.89:33434) 2023-09-30T22:17:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5aca7c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409f650 0x7f5ac4131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52154/0 (socket says 172.21.15.89:52154) 2023-09-30T22:17:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac97c3700 1 -- 172.21.15.89:0/1163235472 learned_addr learned my addr 172.21.15.89:0/1163235472 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac9fc4700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409e930 0x7f5ac409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac97c3700 1 -- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409f650 msgr2=0x7f5ac4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac97c3700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409f650 0x7f5ac4131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac97c3700 1 -- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409e930 msgr2=0x7f5ac409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac97c3700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409e930 0x7f5ac409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac97c3700 1 -- 172.21.15.89:0/1163235472 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac409fb60 con 0x7f5ac409f240 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5ac9fc4700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409e930 0x7f5ac409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.647+0000 7f5aca7c5700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409f650 0x7f5ac4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.648+0000 7f5ac97c3700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 0x7f5ac41317f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40098f0 tx=0x7f5ab4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7c0eff978b7000 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.648+0000 7f5ac8fc2700 1 -- 172.21.15.89:0/1163235472 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4011830 con 0x7f5ac409f240 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.648+0000 7f5ac8fc2700 1 -- 172.21.15.89:0/1163235472 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4011990 con 0x7f5ac409f240 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.648+0000 7f5ac8fc2700 1 -- 172.21.15.89:0/1163235472 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4011d20 con 0x7f5ac409f240 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.648+0000 7f5acc228700 1 -- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 msgr2=0x7f5ac41317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.648+0000 7f5acc228700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 0x7f5ac41317f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40098f0 tx=0x7f5ab4009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.649+0000 7f5acc228700 1 -- 172.21.15.89:0/1163235472 shutdown_connections 2023-09-30T22:17:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.649+0000 7f5acc228700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409f650 0x7f5ac4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.649+0000 7f5acc228700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409e930 0x7f5ac409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.649+0000 7f5acc228700 1 --2- 172.21.15.89:0/1163235472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 0x7f5ac41317f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.649+0000 7f5acc228700 1 -- 172.21.15.89:0/1163235472 >> 172.21.15.89:0/1163235472 conn(0x7f5ac40a4210 msgr2=0x7f5ac40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.649+0000 7f5acc228700 1 -- 172.21.15.89:0/1163235472 shutdown_connections 2023-09-30T22:17:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.649+0000 7f5acc228700 1 -- 172.21.15.89:0/1163235472 wait complete. 2023-09-30T22:17:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5acc228700 1 Processor -- start 2023-09-30T22:17:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5acc228700 1 -- start start 2023-09-30T22:17:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5acc228700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409e930 0x7f5ac406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5acc228700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 0x7f5ac406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5acc228700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409f650 0x7f5ac406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5acc228700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac406cd80 con 0x7f5ac409f240 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5acc228700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac41a99b0 con 0x7f5ac409f650 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5acc228700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac41b72f0 con 0x7f5ac409e930 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.650+0000 7f5ac97c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 0x7f5ac406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac97c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 0x7f5ac406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33444/0 (socket says 172.21.15.89:33444) 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac97c3700 1 -- 172.21.15.89:0/162391901 learned_addr learned my addr 172.21.15.89:0/162391901 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac9fc4700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409e930 0x7f5ac406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5aca7c5700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409f650 0x7f5ac406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac9fc4700 1 -- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409f650 msgr2=0x7f5ac406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac9fc4700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409f650 0x7f5ac406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac9fc4700 1 -- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 msgr2=0x7f5ac406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac9fc4700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 0x7f5ac406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac9fc4700 1 -- 172.21.15.89:0/162391901 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac00094a0 con 0x7f5ac409e930 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5ac9fc4700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409e930 0x7f5ac406b8b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000e9b0 tx=0x7f5ac000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5aba7fc700 1 -- 172.21.15.89:0/162391901 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000ee50 con 0x7f5ac409e930 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5aba7fc700 1 -- 172.21.15.89:0/162391901 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac0004030 con 0x7f5ac409e930 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5aba7fc700 1 -- 172.21.15.89:0/162391901 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00043c0 con 0x7f5ac409e930 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc009a70 con 0x7f5ac409e930 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.651+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab4009550 con 0x7f5ac409e930 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.652+0000 7f5aba7fc700 1 -- 172.21.15.89:0/162391901 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ac000c7f0 con 0x7f5ac409e930 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.652+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac40a3530 con 0x7f5ac409e930 2023-09-30T22:17:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.653+0000 7f5aba7fc700 1 -- 172.21.15.89:0/162391901 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ac003f830 con 0x7f5ac409e930 2023-09-30T22:17:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.657+0000 7f5aba7fc700 1 -- 172.21.15.89:0/162391901 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ac0011db0 con 0x7f5ac409e930 2023-09-30T22:17:40.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.731+0000 7f1aeccd8700 1 Processor -- start 2023-09-30T22:17:40.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.731+0000 7f1aeccd8700 1 -- start start 2023-09-30T22:17:40.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1aeccd8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1aeccd8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1ae809e410 0x7f1ae8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1aeccd8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae809b120 con 0x7f1ae809e000 2023-09-30T22:17:40.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1aeccd8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae806b460 con 0x7f1ae809e410 2023-09-30T22:17:40.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1aeccd8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae809a7e0 con 0x7f1ae809d6f0 2023-09-30T22:17:40.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52178/0 (socket says 172.21.15.89:52178) 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae659c700 1 -- 172.21.15.89:0/4016261122 learned_addr learned my addr 172.21.15.89:0/4016261122 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae6d9d700 1 --2- 172.21.15.89:0/4016261122 >> v2:172.21.15.89:3300/0 conn(0x7f1ae809e410 0x7f1ae8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae659c700 1 -- 172.21.15.89:0/4016261122 >> v2:172.21.15.89:3300/0 conn(0x7f1ae809e410 msgr2=0x7f1ae8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae659c700 1 --2- 172.21.15.89:0/4016261122 >> v2:172.21.15.89:3300/0 conn(0x7f1ae809e410 0x7f1ae8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae659c700 1 -- 172.21.15.89:0/4016261122 >> v1:172.21.15.89:6789/0 conn(0x7f1ae809e000 legacy=0x7f1ae81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae659c700 1 -- 172.21.15.89:0/4016261122 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae809a500 con 0x7f1ae809d6f0 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.732+0000 7f1ae659c700 1 --2- 172.21.15.89:0/4016261122 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae809dac0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009920 tx=0x7f1ad8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79ebc27062a78b53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1ae559a700 1 -- 172.21.15.89:0/4016261122 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011830 con 0x7f1ae809d6f0 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1ae559a700 1 -- 172.21.15.89:0/4016261122 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad8011990 con 0x7f1ae809d6f0 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1ae559a700 1 -- 172.21.15.89:0/4016261122 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011d20 con 0x7f1ae809d6f0 2023-09-30T22:17:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1aeccd8700 1 -- 172.21.15.89:0/4016261122 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 msgr2=0x7f1ae809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1aeccd8700 1 --2- 172.21.15.89:0/4016261122 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae809dac0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009920 tx=0x7f1ad8009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1aeccd8700 1 -- 172.21.15.89:0/4016261122 shutdown_connections 2023-09-30T22:17:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1aeccd8700 1 --2- 172.21.15.89:0/4016261122 >> v2:172.21.15.89:3300/0 conn(0x7f1ae809e410 0x7f1ae8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1aeccd8700 1 --2- 172.21.15.89:0/4016261122 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae809dac0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1aeccd8700 1 -- 172.21.15.89:0/4016261122 >> 172.21.15.89:0/4016261122 conn(0x7f1ae8066f50 msgr2=0x7f1ae809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1aeccd8700 1 -- 172.21.15.89:0/4016261122 shutdown_connections 2023-09-30T22:17:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.733+0000 7f1aeccd8700 1 -- 172.21.15.89:0/4016261122 wait complete. 2023-09-30T22:17:40.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.734+0000 7f1aeccd8700 1 Processor -- start 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.734+0000 7f1aeccd8700 1 -- start start 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1aeccd8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae81a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1aeccd8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ae809e000 0x7f1ae81a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1aeccd8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ae809e410 0x7f1ae81a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1aeccd8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81a4e60 con 0x7f1ae809e000 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1aeccd8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae809c710 con 0x7f1ae809e410 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1aeccd8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80a32d0 con 0x7f1ae809d6f0 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ae809e000 0x7f1ae81a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ae809e000 0x7f1ae81a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33462/0 (socket says 172.21.15.89:33462) 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae5d9b700 1 -- 172.21.15.89:0/136049701 learned_addr learned my addr 172.21.15.89:0/136049701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:40.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae659c700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae81a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae6d9d700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ae809e410 0x7f1ae81a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae5d9b700 1 -- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 msgr2=0x7f1ae81a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae5d9b700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae81a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae5d9b700 1 -- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ae809e410 msgr2=0x7f1ae81a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae5d9b700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ae809e410 0x7f1ae81a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae5d9b700 1 -- 172.21.15.89:0/136049701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad8009550 con 0x7f1ae809e000 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae6d9d700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ae809e410 0x7f1ae81a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.735+0000 7f1ae659c700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae81a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.736+0000 7f1ae5d9b700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ae809e000 0x7f1ae81a2180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00c4b0 tx=0x7f1adc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.736+0000 7f1ad6ffd700 1 -- 172.21.15.89:0/136049701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0052d0 con 0x7f1ae809e000 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.736+0000 7f1ad6ffd700 1 -- 172.21.15.89:0/136049701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc005430 con 0x7f1ae809e000 2023-09-30T22:17:40.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.736+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae81a37c0 con 0x7f1ae809e000 2023-09-30T22:17:40.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.736+0000 7f1ad6ffd700 1 -- 172.21.15.89:0/136049701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0038c0 con 0x7f1ae809e000 2023-09-30T22:17:40.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.736+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae81a5bd0 con 0x7f1ae809e000 2023-09-30T22:17:40.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.737+0000 7f1ad6ffd700 1 -- 172.21.15.89:0/136049701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1adc003a20 con 0x7f1ae809e000 2023-09-30T22:17:40.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.737+0000 7f1ad6ffd700 1 -- 172.21.15.89:0/136049701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1adc03ea10 con 0x7f1ae809e000 2023-09-30T22:17:40.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.737+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae8074ea0 con 0x7f1ae809e000 2023-09-30T22:17:40.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.742+0000 7f1ad6ffd700 1 -- 172.21.15.89:0/136049701 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1adc017080 con 0x7f1ae809e000 2023-09-30T22:17:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.771+0000 7f317a071700 1 Processor -- start 2023-09-30T22:17:40.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.771+0000 7f317a071700 1 -- start start 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f317a071700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f317a071700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f3174133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f317a071700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317409edd0 con 0x7f3174133fa0 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f317a071700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317409e1f0 con 0x7f31740a0a30 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f317a071700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f317409dc50 con 0x7f31740a1340 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f31737fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f3172ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f3174133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f31737fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31740a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40330/0 (socket says 172.21.15.89:40330) 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.772+0000 7f31737fe700 1 -- 172.21.15.89:0/4117512333 learned_addr learned my addr 172.21.15.89:0/4117512333 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31737fe700 1 -- 172.21.15.89:0/4117512333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 msgr2=0x7f3174133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31737fe700 1 --2- 172.21.15.89:0/4117512333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f3174133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31737fe700 1 -- 172.21.15.89:0/4117512333 >> v1:172.21.15.89:6789/0 conn(0x7f3174133fa0 legacy=0x7f317413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31727fc700 1 -- 172.21.15.89:0/4117512333 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763118161 0 0) 0x7f317409edd0 con 0x7f3174133fa0 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31737fe700 1 -- 172.21.15.89:0/4117512333 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740a1fe0 con 0x7f31740a0a30 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f3172ffd700 1 --2- 172.21.15.89:0/4117512333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f3174133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31737fe700 1 --2- 172.21.15.89:0/4117512333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31740a0e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3164009920 tx=0x7f3164009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ca8da7cb09e84c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31727fc700 1 -- 172.21.15.89:0/4117512333 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164011830 con 0x7f31740a0a30 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31727fc700 1 -- 172.21.15.89:0/4117512333 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3164011990 con 0x7f31740a0a30 2023-09-30T22:17:40.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.773+0000 7f31727fc700 1 -- 172.21.15.89:0/4117512333 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164011d20 con 0x7f31740a0a30 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.774+0000 7f317a071700 1 -- 172.21.15.89:0/4117512333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 msgr2=0x7f31740a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.774+0000 7f317a071700 1 --2- 172.21.15.89:0/4117512333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31740a0e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3164009920 tx=0x7f3164009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.774+0000 7f317a071700 1 -- 172.21.15.89:0/4117512333 shutdown_connections 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.774+0000 7f317a071700 1 --2- 172.21.15.89:0/4117512333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31740a0e00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.774+0000 7f317a071700 1 --2- 172.21.15.89:0/4117512333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f3174133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.774+0000 7f317a071700 1 -- 172.21.15.89:0/4117512333 >> 172.21.15.89:0/4117512333 conn(0x7f31740a6550 msgr2=0x7f31740a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.774+0000 7f317a071700 1 -- 172.21.15.89:0/4117512333 shutdown_connections 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.774+0000 7f317a071700 1 -- 172.21.15.89:0/4117512333 wait complete. 2023-09-30T22:17:40.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f317a071700 1 Processor -- start 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f317a071700 1 -- start start 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f317a071700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31741c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f317a071700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f31741c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f317a071700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3174133fa0 0x7f31741c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f317a071700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31740a0220 con 0x7f3174133fa0 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f317a071700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3174125170 con 0x7f31740a0a30 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f317a071700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3174124ab0 con 0x7f31740a1340 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f3173fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3174133fa0 0x7f31741c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f3172ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f31741c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f3173fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3174133fa0 0x7f31741c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33472/0 (socket says 172.21.15.89:33472) 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.775+0000 7f3173fff700 1 -- 172.21.15.89:0/2726743579 learned_addr learned my addr 172.21.15.89:0/2726743579 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:40.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f31737fe700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31741c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:40.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f3173fff700 1 -- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 msgr2=0x7f31741c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f3173fff700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f31741c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f3173fff700 1 -- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 msgr2=0x7f31741c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f3173fff700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31741c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f3173fff700 1 -- 172.21.15.89:0/2726743579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680094a0 con 0x7f3174133fa0 2023-09-30T22:17:40.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f3172ffd700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f31741c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:40.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f3173fff700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3174133fa0 0x7f31741c8d00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f316c001560 tx=0x7f316c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:40.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f315bfff700 1 -- 172.21.15.89:0/2726743579 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c003e30 con 0x7f3174133fa0 2023-09-30T22:17:40.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f315bfff700 1 -- 172.21.15.89:0/2726743579 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c005070 con 0x7f3174133fa0 2023-09-30T22:17:40.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3164009550 con 0x7f3174133fa0 2023-09-30T22:17:40.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f315bfff700 1 -- 172.21.15.89:0/2726743579 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c005420 con 0x7f3174133fa0 2023-09-30T22:17:40.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.776+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31740a25e0 con 0x7f3174133fa0 2023-09-30T22:17:40.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.777+0000 7f315bfff700 1 -- 172.21.15.89:0/2726743579 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f316c005580 con 0x7f3174133fa0 2023-09-30T22:17:40.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.777+0000 7f315bfff700 1 -- 172.21.15.89:0/2726743579 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f316c03ed90 con 0x7f3174133fa0 2023-09-30T22:17:40.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.778+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3174125dd0 con 0x7f3174133fa0 2023-09-30T22:17:40.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.782+0000 7f315bfff700 1 -- 172.21.15.89:0/2726743579 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f316c010d60 con 0x7f3174133fa0 2023-09-30T22:17:40.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.853+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5ac4071d20 con 0x7f5ac409e930 2023-09-30T22:17:40.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.854+0000 7f5aba7fc700 1 -- 172.21.15.89:0/162391901 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ac0020030 con 0x7f5ac409e930 2023-09-30T22:17:40.855 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409e930 msgr2=0x7f5ac406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409e930 0x7f5ac406b8b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000e9b0 tx=0x7f5ac000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 shutdown_connections 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ac409f650 0x7f5ac406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ac409e930 0x7f5ac406b8b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 --2- 172.21.15.89:0/162391901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac409f240 0x7f5ac406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 >> 172.21.15.89:0/162391901 conn(0x7f5ac40a4210 msgr2=0x7f5ac41b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 shutdown_connections 2023-09-30T22:17:40.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.856+0000 7f5acc228700 1 -- 172.21.15.89:0/162391901 wait complete. 2023-09-30T22:17:40.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:40.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.936+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1ae81a5250 con 0x7f1ae809e000 2023-09-30T22:17:40.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.937+0000 7f1ad6ffd700 1 -- 172.21.15.89:0/136049701 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1adc018260 con 0x7f1ae809e000 2023-09-30T22:17:40.938 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:40.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ae809e000 msgr2=0x7f1ae81a2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ae809e000 0x7f1ae81a2180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00c4b0 tx=0x7f1adc00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 shutdown_connections 2023-09-30T22:17:40.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ae809e410 0x7f1ae81a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ae809d6f0 0x7f1ae81a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 --2- 172.21.15.89:0/136049701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ae809e000 0x7f1ae81a2180 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 >> 172.21.15.89:0/136049701 conn(0x7f1ae8066f50 msgr2=0x7f1ae81b6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:40.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 shutdown_connections 2023-09-30T22:17:40.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.939+0000 7f1aeccd8700 1 -- 172.21.15.89:0/136049701 wait complete. 2023-09-30T22:17:40.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.974+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3174074ea0 con 0x7f3174133fa0 2023-09-30T22:17:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.974+0000 7f315bfff700 1 -- 172.21.15.89:0/2726743579 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f316c03e020 con 0x7f3174133fa0 2023-09-30T22:17:40.975 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3174133fa0 msgr2=0x7f31741c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3174133fa0 0x7f31741c8d00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f316c001560 tx=0x7f316c00cb50 comp rx=0 tx=0).stop 2023-09-30T22:17:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 shutdown_connections 2023-09-30T22:17:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3174133fa0 0x7f31741c8d00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31740a0a30 0x7f31741c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 --2- 172.21.15.89:0/2726743579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31740a1340 0x7f31741c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:40.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 >> 172.21.15.89:0/2726743579 conn(0x7f31740a6550 msgr2=0x7f317411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:40.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 shutdown_connections 2023-09-30T22:17:40.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:40.976+0000 7f317a071700 1 -- 172.21.15.89:0/2726743579 wait complete. 2023-09-30T22:17:40.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:41.553 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.676+0000 7fc77ae9c700 1 Processor -- start 2023-09-30T22:17:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.676+0000 7fc77ae9c700 1 -- start start 2023-09-30T22:17:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc77ae9c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc77ae9c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc774130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc77ae9c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77409f130 con 0x7fc77409e410 2023-09-30T22:17:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc77ae9c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77409f520 con 0x7fc77409e000 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc77ae9c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc77409e930 con 0x7fc77409d6f0 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc773fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc774130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc779439700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc77409e410 0x7fc774130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42638/0 (socket says 172.21.15.89:42638) 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc779439700 1 -- 172.21.15.89:0/1782991792 learned_addr learned my addr 172.21.15.89:0/1782991792 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc778c38700 1 --2- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc7737fe700 1 -- 172.21.15.89:0/1782991792 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714228052 0 0) 0x7fc77409f130 con 0x7fc77409e410 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc778c38700 1 -- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 msgr2=0x7fc774130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc778c38700 1 --2- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc774130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc778c38700 1 -- 172.21.15.89:0/1782991792 >> v1:172.21.15.89:6789/0 conn(0x7fc77409e410 legacy=0x7fc774130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc778c38700 1 -- 172.21.15.89:0/1782991792 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77409b2d0 con 0x7fc77409d6f0 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc773fff700 1 --2- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc774130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:41.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc778c38700 1 --2- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77409dac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc7680098a0 tx=0x7fc768009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ef524342496df0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc7737fe700 1 -- 172.21.15.89:0/1782991792 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768011830 con 0x7fc77409d6f0 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc7737fe700 1 -- 172.21.15.89:0/1782991792 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc768011990 con 0x7fc77409d6f0 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.677+0000 7fc7737fe700 1 -- 172.21.15.89:0/1782991792 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768011d20 con 0x7fc77409d6f0 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.678+0000 7fc77ae9c700 1 -- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 msgr2=0x7fc77409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.678+0000 7fc77ae9c700 1 --2- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77409dac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc7680098a0 tx=0x7fc768009bb0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.678+0000 7fc77ae9c700 1 -- 172.21.15.89:0/1782991792 shutdown_connections 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.678+0000 7fc77ae9c700 1 --2- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77409dac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.678+0000 7fc77ae9c700 1 --2- 172.21.15.89:0/1782991792 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc774130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.678+0000 7fc77ae9c700 1 -- 172.21.15.89:0/1782991792 >> 172.21.15.89:0/1782991792 conn(0x7fc774066f50 msgr2=0x7fc77409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.678+0000 7fc77ae9c700 1 -- 172.21.15.89:0/1782991792 shutdown_connections 2023-09-30T22:17:41.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.678+0000 7fc77ae9c700 1 -- 172.21.15.89:0/1782991792 wait complete. 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.679+0000 7fc77ae9c700 1 Processor -- start 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.679+0000 7fc77ae9c700 1 -- start start 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.679+0000 7fc77ae9c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77411f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.679+0000 7fc77ae9c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc77411f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.679+0000 7fc77ae9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc77409e410 0x7fc77411fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc77ae9c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc774121200 con 0x7fc77409e410 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc77ae9c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc774130000 con 0x7fc77409e000 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc77ae9c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc774122f90 con 0x7fc77409d6f0 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc77411f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:41.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc778c38700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77411f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc779439700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc77409e410 0x7fc77411fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc77411f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40358/0 (socket says 172.21.15.89:40358) 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 -- 172.21.15.89:0/4196288333 learned_addr learned my addr 172.21.15.89:0/4196288333 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 -- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 msgr2=0x7fc77411f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 --2- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77411f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 -- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc77409e410 msgr2=0x7fc77411fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 --2- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc77409e410 0x7fc77411fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 -- 172.21.15.89:0/4196288333 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600094a0 con 0x7fc77409e000 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc773fff700 1 --2- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc77411f9b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc76000ebb0 tx=0x7fc76000eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc7717fa700 1 -- 172.21.15.89:0/4196288333 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000c850 con 0x7fc77409e000 2023-09-30T22:17:41.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc7717fa700 1 -- 172.21.15.89:0/4196288333 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76000c9b0 con 0x7fc77409e000 2023-09-30T22:17:41.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7680094a0 con 0x7fc77409e000 2023-09-30T22:17:41.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc7717fa700 1 -- 172.21.15.89:0/4196288333 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000cd40 con 0x7fc77409e000 2023-09-30T22:17:41.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.680+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77409c700 con 0x7fc77409e000 2023-09-30T22:17:41.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.681+0000 7fc7717fa700 1 -- 172.21.15.89:0/4196288333 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc760013470 con 0x7fc77409e000 2023-09-30T22:17:41.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.681+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77409dcd0 con 0x7fc77409e000 2023-09-30T22:17:41.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.681+0000 7fc7717fa700 1 -- 172.21.15.89:0/4196288333 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc76003faf0 con 0x7fc77409e000 2023-09-30T22:17:41.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.686+0000 7fc7717fa700 1 -- 172.21.15.89:0/4196288333 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc76001b3b0 con 0x7fc77409e000 2023-09-30T22:17:41.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.867+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc7740a3540 con 0x7fc77409e000 2023-09-30T22:17:41.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.867+0000 7fc7717fa700 1 -- 172.21.15.89:0/4196288333 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc760024350 con 0x7fc77409e000 2023-09-30T22:17:41.869 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:41.870 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:41.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.869+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 msgr2=0x7fc77411f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:41.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.869+0000 7fc77ae9c700 1 --2- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc77411f9b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc76000ebb0 tx=0x7fc76000eec0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.869+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 shutdown_connections 2023-09-30T22:17:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.869+0000 7fc77ae9c700 1 --2- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc77409e410 0x7fc77411fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.869+0000 7fc77ae9c700 1 --2- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc77409d6f0 0x7fc77411f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.869+0000 7fc77ae9c700 1 --2- 172.21.15.89:0/4196288333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc77409e000 0x7fc77411f9b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.869+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 >> 172.21.15.89:0/4196288333 conn(0x7fc774066f50 msgr2=0x7fc774130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.870+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 shutdown_connections 2023-09-30T22:17:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.870+0000 7fc77ae9c700 1 -- 172.21.15.89:0/4196288333 wait complete. 2023-09-30T22:17:41.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:41.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:41.990 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:41.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.994+0000 7fc2176f0700 1 Processor -- start 2023-09-30T22:17:41.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc2176f0700 1 -- start start 2023-09-30T22:17:41.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc2176f0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc21009e930 0x7fc21009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:41.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc2176f0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc210131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:41.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc2176f0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc210070940 con 0x7fc21009f240 2023-09-30T22:17:41.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc2176f0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21006fd60 con 0x7fc21009e930 2023-09-30T22:17:41.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc2176f0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21006fa80 con 0x7fc21009f650 2023-09-30T22:17:41.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc21548c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc21009e930 0x7fc21009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:41.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc21548c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc21009e930 0x7fc21009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33484/0 (socket says 172.21.15.89:33484) 2023-09-30T22:17:41.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc21548c700 1 -- 172.21.15.89:0/3450902161 learned_addr learned my addr 172.21.15.89:0/3450902161 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:41.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.995+0000 7fc215c8d700 1 --2- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc210131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc21548c700 1 -- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 msgr2=0x7fc210131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc21548c700 1 --2- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc210131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc21548c700 1 -- 172.21.15.89:0/3450902161 >> v1:172.21.15.89:6789/0 conn(0x7fc21009f240 legacy=0x7fc2101317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc21548c700 1 -- 172.21.15.89:0/3450902161 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21009fb60 con 0x7fc21009e930 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc215c8d700 1 --2- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc210131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc21548c700 1 --2- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009e930 0x7fc21009ed00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0098f0 tx=0x7fc20c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ba47b4108c9c9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc207fff700 1 -- 172.21.15.89:0/3450902161 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c011830 con 0x7fc21009e930 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc207fff700 1 -- 172.21.15.89:0/3450902161 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c011990 con 0x7fc21009e930 2023-09-30T22:17:41.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.996+0000 7fc207fff700 1 -- 172.21.15.89:0/3450902161 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c011d20 con 0x7fc21009e930 2023-09-30T22:17:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.997+0000 7fc2176f0700 1 -- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009e930 msgr2=0x7fc21009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.997+0000 7fc2176f0700 1 --2- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009e930 0x7fc21009ed00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0098f0 tx=0x7fc20c009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.997+0000 7fc2176f0700 1 -- 172.21.15.89:0/3450902161 shutdown_connections 2023-09-30T22:17:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.997+0000 7fc2176f0700 1 --2- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009e930 0x7fc21009ed00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.997+0000 7fc2176f0700 1 --2- 172.21.15.89:0/3450902161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc210131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.997+0000 7fc2176f0700 1 -- 172.21.15.89:0/3450902161 >> 172.21.15.89:0/3450902161 conn(0x7fc2100a4210 msgr2=0x7fc2100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.997+0000 7fc2176f0700 1 -- 172.21.15.89:0/3450902161 shutdown_connections 2023-09-30T22:17:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.997+0000 7fc2176f0700 1 -- 172.21.15.89:0/3450902161 wait complete. 2023-09-30T22:17:42.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc2176f0700 1 Processor -- start 2023-09-30T22:17:42.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc2176f0700 1 -- start start 2023-09-30T22:17:42.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc2176f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc21009e930 0x7fc21006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc2176f0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009f240 0x7fc21006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc2176f0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc21006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc2176f0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21006cd80 con 0x7fc21009f240 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc2176f0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21006d4c0 con 0x7fc21009e930 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc2176f0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101a9330 con 0x7fc21009f650 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc214c8b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009f240 0x7fc21006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc214c8b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009f240 0x7fc21006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33496/0 (socket says 172.21.15.89:33496) 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc214c8b700 1 -- 172.21.15.89:0/1915258199 learned_addr learned my addr 172.21.15.89:0/1915258199 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc215c8d700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc21006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.998+0000 7fc21548c700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc21009e930 0x7fc21006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc214c8b700 1 -- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 msgr2=0x7fc21006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc214c8b700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc21006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc214c8b700 1 -- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc21009e930 msgr2=0x7fc21006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc214c8b700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc21009e930 0x7fc21006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc214c8b700 1 -- 172.21.15.89:0/1915258199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc208009a70 con 0x7fc21009f240 2023-09-30T22:17:42.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc214c8b700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009f240 0x7fc21006bdf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc20000c510 tx=0x7fc20000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:42.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc205ffb700 1 -- 172.21.15.89:0/1915258199 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000052d0 con 0x7fc21009f240 2023-09-30T22:17:42.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc205ffb700 1 -- 172.21.15.89:0/1915258199 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc200005430 con 0x7fc21009f240 2023-09-30T22:17:42.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c009550 con 0x7fc21009f240 2023-09-30T22:17:42.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc205ffb700 1 -- 172.21.15.89:0/1915258199 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000038c0 con 0x7fc21009f240 2023-09-30T22:17:42.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:41.999+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2100a0190 con 0x7fc21009f240 2023-09-30T22:17:42.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.000+0000 7fc205ffb700 1 -- 172.21.15.89:0/1915258199 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc200003a20 con 0x7fc21009f240 2023-09-30T22:17:42.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.000+0000 7fc205ffb700 1 -- 172.21.15.89:0/1915258199 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc20003eaa0 con 0x7fc21009f240 2023-09-30T22:17:42.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.000+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2101a8c10 con 0x7fc21009f240 2023-09-30T22:17:42.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.005+0000 7fc205ffb700 1 -- 172.21.15.89:0/1915258199 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc200019e60 con 0x7fc21009f240 2023-09-30T22:17:42.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0ceaa7700 1 Processor -- start 2023-09-30T22:17:42.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0ceaa7700 1 -- start start 2023-09-30T22:17:42.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0ceaa7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809d6f0 0x7ff0c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0ceaa7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e000 0x7ff0c81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0ceaa7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c809b120 con 0x7ff0c809e410 2023-09-30T22:17:42.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0ceaa7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c806b460 con 0x7ff0c809d6f0 2023-09-30T22:17:42.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0ceaa7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c809a7e0 con 0x7ff0c809e000 2023-09-30T22:17:42.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0c7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e000 0x7ff0c81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0cd044700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff0c809e410 0x7ff0c8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42656/0 (socket says 172.21.15.89:42656) 2023-09-30T22:17:42.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0cd044700 1 -- 172.21.15.89:0/2276487536 learned_addr learned my addr 172.21.15.89:0/2276487536 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:42.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.080+0000 7ff0cc843700 1 --2- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809d6f0 0x7ff0c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0c7fff700 1 -- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809d6f0 msgr2=0x7ff0c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:42.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0c7fff700 1 --2- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809d6f0 0x7ff0c809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0c7fff700 1 -- 172.21.15.89:0/2276487536 >> v1:172.21.15.89:6789/0 conn(0x7ff0c809e410 legacy=0x7ff0c8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0c7fff700 1 -- 172.21.15.89:0/2276487536 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c809a500 con 0x7ff0c809e000 2023-09-30T22:17:42.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0cc843700 1 --2- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809d6f0 0x7ff0c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0c7fff700 1 --2- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e000 0x7ff0c81339f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40098a0 tx=0x7ff0b4009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a95cd31e74e05d3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0c77fe700 1 -- 172.21.15.89:0/2276487536 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4011830 con 0x7ff0c809e000 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0c77fe700 1 -- 172.21.15.89:0/2276487536 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b4011990 con 0x7ff0c809e000 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0c77fe700 1 -- 172.21.15.89:0/2276487536 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4011d20 con 0x7ff0c809e000 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e000 msgr2=0x7ff0c81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.081+0000 7ff0ceaa7700 1 --2- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e000 0x7ff0c81339f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40098a0 tx=0x7ff0b4009bb0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.082+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/2276487536 shutdown_connections 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.082+0000 7ff0ceaa7700 1 --2- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809d6f0 0x7ff0c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.082+0000 7ff0ceaa7700 1 --2- 172.21.15.89:0/2276487536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e000 0x7ff0c81339f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.082+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/2276487536 >> 172.21.15.89:0/2276487536 conn(0x7ff0c8066f50 msgr2=0x7ff0c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.082+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/2276487536 shutdown_connections 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.082+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/2276487536 wait complete. 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0ceaa7700 1 Processor -- start 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0ceaa7700 1 -- start start 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0ceaa7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c809d6f0 0x7ff0c81a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0ceaa7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809e000 0x7ff0c81a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0ceaa7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e410 0x7ff0c81a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0ceaa7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80a2090 con 0x7ff0c809d6f0 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0ceaa7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c81a43e0 con 0x7ff0c809e000 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0ceaa7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c81a4c30 con 0x7ff0c809e410 2023-09-30T22:17:42.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0cc843700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c809d6f0 0x7ff0c81a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0cc843700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c809d6f0 0x7ff0c81a2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33502/0 (socket says 172.21.15.89:33502) 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0cc843700 1 -- 172.21.15.89:0/3963055106 learned_addr learned my addr 172.21.15.89:0/3963055106 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0c7fff700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809e000 0x7ff0c81a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.083+0000 7ff0cd044700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e410 0x7ff0c81a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0cc843700 1 -- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e410 msgr2=0x7ff0c81a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0cc843700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e410 0x7ff0c81a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0cc843700 1 -- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809e000 msgr2=0x7ff0c81a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0cc843700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809e000 0x7ff0c81a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0cc843700 1 -- 172.21.15.89:0/3963055106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc0094a0 con 0x7ff0c809d6f0 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0cc843700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c809d6f0 0x7ff0c81a2f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00ebe0 tx=0x7ff0bc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:42.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0c57fa700 1 -- 172.21.15.89:0/3963055106 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00c850 con 0x7ff0c809d6f0 2023-09-30T22:17:42.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0c57fa700 1 -- 172.21.15.89:0/3963055106 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc00c9b0 con 0x7ff0c809d6f0 2023-09-30T22:17:42.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b40094a0 con 0x7ff0c809d6f0 2023-09-30T22:17:42.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0c57fa700 1 -- 172.21.15.89:0/3963055106 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00cd40 con 0x7ff0c809d6f0 2023-09-30T22:17:42.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.084+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c81a6a10 con 0x7ff0c809d6f0 2023-09-30T22:17:42.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.085+0000 7ff0c57fa700 1 -- 172.21.15.89:0/3963055106 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0bc013470 con 0x7ff0c809d6f0 2023-09-30T22:17:42.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.085+0000 7ff0c57fa700 1 -- 172.21.15.89:0/3963055106 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff0bc03f5d0 con 0x7ff0c809d6f0 2023-09-30T22:17:42.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.085+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c81a6a10 con 0x7ff0c809d6f0 2023-09-30T22:17:42.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.090+0000 7ff0c57fa700 1 -- 172.21.15.89:0/3963055106 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0c81a6a10 con 0x7ff0c809d6f0 2023-09-30T22:17:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.119+0000 7f9cf9a37700 1 Processor -- start 2023-09-30T22:17:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.119+0000 7f9cf9a37700 1 -- start start 2023-09-30T22:17:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf9a37700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4133420 0x7f9cf41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf9a37700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9cf4134260 0x7f9cf41306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf9a37700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf406b990 con 0x7f9cf4133e30 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf9a37700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf40a0590 con 0x7f9cf4134260 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf9a37700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf409fb80 con 0x7f9cf4133420 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9cf4134260 0x7f9cf41306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9cf4134260 0x7f9cf41306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33516/0 (socket says 172.21.15.89:33516) 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4133420 0x7f9cf41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf37fe700 1 -- 172.21.15.89:0/3640041158 learned_addr learned my addr 172.21.15.89:0/3640041158 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf27fc700 1 --1- 172.21.15.89:0/3640041158 >> v1:172.21.15.89:6789/0 conn(0x7f9cf4133e30 0x7f9cf412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42666/0 (socket says 172.21.15.89:42666) 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf37fe700 1 -- 172.21.15.89:0/3640041158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4133420 msgr2=0x7f9cf41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf37fe700 1 --2- 172.21.15.89:0/3640041158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4133420 0x7f9cf41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf37fe700 1 -- 172.21.15.89:0/3640041158 >> v1:172.21.15.89:6789/0 conn(0x7f9cf4133e30 legacy=0x7f9cf412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf37fe700 1 -- 172.21.15.89:0/3640041158 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf40a0030 con 0x7f9cf4134260 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf1ffb700 1 -- 172.21.15.89:0/3640041158 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667719565 0 0) 0x7f9cf406b990 con 0x7f9cf4133e30 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.120+0000 7f9cf2ffd700 1 --2- 172.21.15.89:0/3640041158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4133420 0x7f9cf41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.121+0000 7f9cf37fe700 1 --2- 172.21.15.89:0/3640041158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4134260 0x7f9cf41306a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009e50 tx=0x7f9cec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741a9808ab7e0b4b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.121+0000 7f9cf1ffb700 1 -- 172.21.15.89:0/3640041158 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00e070 con 0x7f9cf4134260 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.121+0000 7f9cf1ffb700 1 -- 172.21.15.89:0/3640041158 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec00b910 con 0x7f9cf4134260 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.121+0000 7f9cf1ffb700 1 -- 172.21.15.89:0/3640041158 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00bcc0 con 0x7f9cf4134260 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.121+0000 7f9cf9a37700 1 -- 172.21.15.89:0/3640041158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4134260 msgr2=0x7f9cf41306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.121+0000 7f9cf9a37700 1 --2- 172.21.15.89:0/3640041158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4134260 0x7f9cf41306a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009e50 tx=0x7f9cec00b330 comp rx=0 tx=0).stop 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.122+0000 7f9cf9a37700 1 -- 172.21.15.89:0/3640041158 shutdown_connections 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.122+0000 7f9cf9a37700 1 --2- 172.21.15.89:0/3640041158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4134260 0x7f9cf41306a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.122+0000 7f9cf9a37700 1 --2- 172.21.15.89:0/3640041158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4133420 0x7f9cf41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.122+0000 7f9cf9a37700 1 -- 172.21.15.89:0/3640041158 >> 172.21.15.89:0/3640041158 conn(0x7f9cf40a4210 msgr2=0x7f9cf40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.122+0000 7f9cf9a37700 1 -- 172.21.15.89:0/3640041158 shutdown_connections 2023-09-30T22:17:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.122+0000 7f9cf9a37700 1 -- 172.21.15.89:0/3640041158 wait complete. 2023-09-30T22:17:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.122+0000 7f9cf9a37700 1 Processor -- start 2023-09-30T22:17:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf9a37700 1 -- start start 2023-09-30T22:17:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf9a37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4133420 0x7f9cf409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf9a37700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cf4133e30 0x7f9cf409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf9a37700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4134260 0x7f9cf409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf9a37700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf411fdb0 con 0x7f9cf4133420 2023-09-30T22:17:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf9a37700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf411faf0 con 0x7f9cf4133e30 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf9a37700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf41314e0 con 0x7f9cf4134260 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4134260 0x7f9cf409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf27fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cf4133e30 0x7f9cf409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4134260 0x7f9cf409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52260/0 (socket says 172.21.15.89:52260) 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf37fe700 1 -- 172.21.15.89:0/1416677825 learned_addr learned my addr 172.21.15.89:0/1416677825 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.123+0000 7f9cf2ffd700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4133420 0x7f9cf409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf37fe700 1 -- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cf4133e30 msgr2=0x7f9cf409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf37fe700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cf4133e30 0x7f9cf409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf37fe700 1 -- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4133420 msgr2=0x7f9cf409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf37fe700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4133420 0x7f9cf409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf37fe700 1 -- 172.21.15.89:0/1416677825 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce40094a0 con 0x7f9cf4134260 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf27fc700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cf4133e30 0x7f9cf409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf2ffd700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4133420 0x7f9cf409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf37fe700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4134260 0x7f9cf409cf00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40a06d0 tx=0x7f9cec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cdbfff700 1 -- 172.21.15.89:0/1416677825 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec004880 con 0x7f9cf4134260 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cdbfff700 1 -- 172.21.15.89:0/1416677825 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec0099b0 con 0x7f9cf4134260 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cdbfff700 1 -- 172.21.15.89:0/1416677825 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec01c650 con 0x7f9cf4134260 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec009b30 con 0x7f9cf4134260 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.124+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf409dd10 con 0x7f9cf4134260 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.125+0000 7f9cdbfff700 1 -- 172.21.15.89:0/1416677825 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9cec01c7b0 con 0x7f9cf4134260 2023-09-30T22:17:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.125+0000 7f9cdbfff700 1 -- 172.21.15.89:0/1416677825 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9cec040e00 con 0x7f9cf4134260 2023-09-30T22:17:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.125+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf41200f0 con 0x7f9cf4134260 2023-09-30T22:17:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.130+0000 7f9cdbfff700 1 -- 172.21.15.89:0/1416677825 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9cec029080 con 0x7f9cf4134260 2023-09-30T22:17:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.202+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc2101a90c0 con 0x7fc21009f240 2023-09-30T22:17:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.203+0000 7fc205ffb700 1 -- 172.21.15.89:0/1915258199 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2101a90c0 con 0x7fc21009f240 2023-09-30T22:17:42.203 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009f240 msgr2=0x7fc21006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009f240 0x7fc21006bdf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc20000c510 tx=0x7fc20000c8d0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 shutdown_connections 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc21009f650 0x7fc21006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc21009e930 0x7fc21006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 --2- 172.21.15.89:0/1915258199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc21009f240 0x7fc21006bdf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 >> 172.21.15.89:0/1915258199 conn(0x7fc2100a4210 msgr2=0x7fc2101b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 shutdown_connections 2023-09-30T22:17:42.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.205+0000 7fc2176f0700 1 -- 172.21.15.89:0/1915258199 wait complete. 2023-09-30T22:17:42.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:42.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.288+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff0c81a6a10 con 0x7ff0c809d6f0 2023-09-30T22:17:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.288+0000 7ff0c57fa700 1 -- 172.21.15.89:0/3963055106 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0c81a6a10 con 0x7ff0c809d6f0 2023-09-30T22:17:42.289 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c809d6f0 msgr2=0x7ff0c81a2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c809d6f0 0x7ff0c81a2f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00ebe0 tx=0x7ff0bc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 shutdown_connections 2023-09-30T22:17:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c809e410 0x7ff0c81a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c809d6f0 0x7ff0c81a2f30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 --2- 172.21.15.89:0/3963055106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0c809e000 0x7ff0c81a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 >> 172.21.15.89:0/3963055106 conn(0x7ff0c8066f50 msgr2=0x7ff0c81b9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 shutdown_connections 2023-09-30T22:17:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.290+0000 7ff0ceaa7700 1 -- 172.21.15.89:0/3963055106 wait complete. 2023-09-30T22:17:42.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:42.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.324+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9cf4074ea0 con 0x7f9cf4134260 2023-09-30T22:17:42.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.324+0000 7f9cdbfff700 1 -- 172.21.15.89:0/1416677825 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9cec01ca60 con 0x7f9cf4134260 2023-09-30T22:17:42.325 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:42.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.326+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4134260 msgr2=0x7f9cf409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:42.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.326+0000 7f9cf9a37700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4134260 0x7f9cf409cf00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40a06d0 tx=0x7f9cec004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.326+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 shutdown_connections 2023-09-30T22:17:42.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.326+0000 7f9cf9a37700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9cf4134260 0x7f9cf409cf00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.326+0000 7f9cf9a37700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cf4133420 0x7f9cf409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.326+0000 7f9cf9a37700 1 --2- 172.21.15.89:0/1416677825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9cf4133e30 0x7f9cf409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:42.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.326+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 >> 172.21.15.89:0/1416677825 conn(0x7f9cf40a4210 msgr2=0x7f9cf4130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:42.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.327+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 shutdown_connections 2023-09-30T22:17:42.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:42.327+0000 7f9cf9a37700 1 -- 172.21.15.89:0/1416677825 wait complete. 2023-09-30T22:17:42.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:42.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.017+0000 7f32cd781700 1 Processor -- start 2023-09-30T22:17:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.017+0000 7f32cd781700 1 -- start start 2023-09-30T22:17:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.017+0000 7f32cd781700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32c80a0a30 0x7f32c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.017+0000 7f32cd781700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.017+0000 7f32cd781700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.017+0000 7f32cd781700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c80a2a70 con 0x7f32c80a0a30 2023-09-30T22:17:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.017+0000 7f32cd781700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80a01d0 con 0x7f32c8133930 2023-09-30T22:17:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.017+0000 7f32cd781700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32c809e2b0 con 0x7f32c809d630 2023-09-30T22:17:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32c80a0a30 0x7f32c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c67fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32c80a0a30 0x7f32c80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33532/0 (socket says 172.21.15.89:33532) 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c6ffd700 1 -- 172.21.15.89:0/381746256 learned_addr learned my addr 172.21.15.89:0/381746256 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c77fe700 1 -- 172.21.15.89:0/381746256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 msgr2=0x7f32c80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c77fe700 1 --2- 172.21.15.89:0/381746256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c77fe700 1 -- 172.21.15.89:0/381746256 >> v2:172.21.15.89:3300/0 conn(0x7f32c80a0a30 msgr2=0x7f32c80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c77fe700 1 --2- 172.21.15.89:0/381746256 >> v2:172.21.15.89:3300/0 conn(0x7f32c80a0a30 0x7f32c80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c77fe700 1 -- 172.21.15.89:0/381746256 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c809dc50 con 0x7f32c8133930 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c67fc700 1 --2- 172.21.15.89:0/381746256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c77fe700 1 --2- 172.21.15.89:0/381746256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c8133db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009d90 tx=0x7f32c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a62523bb0b027 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:43.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c5ffb700 1 -- 172.21.15.89:0/381746256 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000e070 con 0x7f32c8133930 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c5ffb700 1 -- 172.21.15.89:0/381746256 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000bb20 con 0x7f32c8133930 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.018+0000 7f32c5ffb700 1 -- 172.21.15.89:0/381746256 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0012430 con 0x7f32c8133930 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 -- 172.21.15.89:0/381746256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 msgr2=0x7f32c8133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 --2- 172.21.15.89:0/381746256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c8133db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009d90 tx=0x7f32c000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 -- 172.21.15.89:0/381746256 shutdown_connections 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 --2- 172.21.15.89:0/381746256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c8133db0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 --2- 172.21.15.89:0/381746256 >> v2:172.21.15.89:3300/0 conn(0x7f32c80a0a30 0x7f32c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 --2- 172.21.15.89:0/381746256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 -- 172.21.15.89:0/381746256 >> 172.21.15.89:0/381746256 conn(0x7f32c80a3050 msgr2=0x7f32c80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 -- 172.21.15.89:0/381746256 shutdown_connections 2023-09-30T22:17:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.019+0000 7f32cd781700 1 -- 172.21.15.89:0/381746256 wait complete. 2023-09-30T22:17:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32cd781700 1 Processor -- start 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32cd781700 1 -- start start 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32cd781700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c81c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32cd781700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c80a0a30 0x7f32c81c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32cd781700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c81c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32cd781700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80a0410 con 0x7f32c80a0a30 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32cd781700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8120630 con 0x7f32c8133930 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32cd781700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80a4030 con 0x7f32c809d630 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.020+0000 7f32c67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c80a0a30 0x7f32c81c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c81c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c81c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c80a0a30 0x7f32c81c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33536/0 (socket says 172.21.15.89:33536) 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c67fc700 1 -- 172.21.15.89:0/2015968418 learned_addr learned my addr 172.21.15.89:0/2015968418 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c77fe700 1 -- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 msgr2=0x7f32c81c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c77fe700 1 --2- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c81c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c77fe700 1 -- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c80a0a30 msgr2=0x7f32c81c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c77fe700 1 --2- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c80a0a30 0x7f32c81c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c77fe700 1 -- 172.21.15.89:0/2015968418 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80094a0 con 0x7f32c8133930 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c67fc700 1 --2- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c80a0a30 0x7f32c81c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32c77fe700 1 --2- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c81c8900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f32c0016000 tx=0x7f32c0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32af7fe700 1 -- 172.21.15.89:0/2015968418 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0012430 con 0x7f32c8133930 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32af7fe700 1 -- 172.21.15.89:0/2015968418 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0020070 con 0x7f32c8133930 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32af7fe700 1 -- 172.21.15.89:0/2015968418 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0004280 con 0x7f32c8133930 2023-09-30T22:17:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c0009a70 con 0x7f32c8133930 2023-09-30T22:17:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.021+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c8120ca0 con 0x7f32c8133930 2023-09-30T22:17:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.022+0000 7f32af7fe700 1 -- 172.21.15.89:0/2015968418 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f32c00043e0 con 0x7f32c8133930 2023-09-30T22:17:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.022+0000 7f32af7fe700 1 -- 172.21.15.89:0/2015968418 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f32c0022070 con 0x7f32c8133930 2023-09-30T22:17:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.022+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8121d80 con 0x7f32c8133930 2023-09-30T22:17:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.027+0000 7f32af7fe700 1 -- 172.21.15.89:0/2015968418 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32c8121d80 con 0x7f32c8133930 2023-09-30T22:17:43.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.208+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f32c8121d80 con 0x7f32c8133930 2023-09-30T22:17:43.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.208+0000 7f32af7fe700 1 -- 172.21.15.89:0/2015968418 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32c8121d80 con 0x7f32c8133930 2023-09-30T22:17:43.209 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 msgr2=0x7f32c81c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 --2- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c81c8900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f32c0016000 tx=0x7f32c0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 shutdown_connections 2023-09-30T22:17:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 --2- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c8133930 0x7f32c81c8900 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 --2- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809d630 0x7f32c81c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 --2- 172.21.15.89:0/2015968418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c80a0a30 0x7f32c81c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 >> 172.21.15.89:0/2015968418 conn(0x7f32c80a3050 msgr2=0x7f32c809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 shutdown_connections 2023-09-30T22:17:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.210+0000 7f32cd781700 1 -- 172.21.15.89:0/2015968418 wait complete. 2023-09-30T22:17:43.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:43.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:43.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:43.341 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.344+0000 7fbeb3d96700 1 Processor -- start 2023-09-30T22:17:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.344+0000 7fbeb3d96700 1 -- start start 2023-09-30T22:17:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb3d96700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbeac09e930 0x7fbeac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb3d96700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb3d96700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeac070940 con 0x7fbeac09f650 2023-09-30T22:17:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb3d96700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac06fd60 con 0x7fbeac09e930 2023-09-30T22:17:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb3d96700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac06fa80 con 0x7fbeac09f240 2023-09-30T22:17:43.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb2333700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbeac09f650 0x7fbeac131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42680/0 (socket says 172.21.15.89:42680) 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb2333700 1 -- 172.21.15.89:0/2647459461 learned_addr learned my addr 172.21.15.89:0/2647459461 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb1331700 1 --2- 172.21.15.89:0/2647459461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb1b32700 1 --2- 172.21.15.89:0/2647459461 >> v2:172.21.15.89:3300/0 conn(0x7fbeac09e930 0x7fbeac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb0b30700 1 -- 172.21.15.89:0/2647459461 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764366907 0 0) 0x7fbeac070940 con 0x7fbeac09f650 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb0b30700 1 -- 172.21.15.89:0/2647459461 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe980035b0 con 0x7fbeac09f650 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb1331700 1 -- 172.21.15.89:0/2647459461 >> v2:172.21.15.89:3300/0 conn(0x7fbeac09e930 msgr2=0x7fbeac09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb1331700 1 --2- 172.21.15.89:0/2647459461 >> v2:172.21.15.89:3300/0 conn(0x7fbeac09e930 0x7fbeac09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb1331700 1 -- 172.21.15.89:0/2647459461 >> v1:172.21.15.89:6789/0 conn(0x7fbeac09f650 legacy=0x7fbeac131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb1331700 1 -- 172.21.15.89:0/2647459461 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac09fb60 con 0x7fbeac09f240 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb1b32700 1 --2- 172.21.15.89:0/2647459461 >> v2:172.21.15.89:3300/0 conn(0x7fbeac09e930 0x7fbeac09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb1331700 1 --2- 172.21.15.89:0/2647459461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac1317f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009840 tx=0x7fbe9c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6fd4285de434a29 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.345+0000 7fbeb0b30700 1 -- 172.21.15.89:0/2647459461 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c011830 con 0x7fbeac09f240 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb0b30700 1 -- 172.21.15.89:0/2647459461 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c011990 con 0x7fbeac09f240 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb0b30700 1 -- 172.21.15.89:0/2647459461 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c011d20 con 0x7fbeac09f240 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb3d96700 1 -- 172.21.15.89:0/2647459461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 msgr2=0x7fbeac1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb3d96700 1 --2- 172.21.15.89:0/2647459461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac1317f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009840 tx=0x7fbe9c009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb3d96700 1 -- 172.21.15.89:0/2647459461 shutdown_connections 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb3d96700 1 --2- 172.21.15.89:0/2647459461 >> v2:172.21.15.89:3300/0 conn(0x7fbeac09e930 0x7fbeac09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb3d96700 1 --2- 172.21.15.89:0/2647459461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac1317f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb3d96700 1 -- 172.21.15.89:0/2647459461 >> 172.21.15.89:0/2647459461 conn(0x7fbeac0a4210 msgr2=0x7fbeac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb3d96700 1 -- 172.21.15.89:0/2647459461 shutdown_connections 2023-09-30T22:17:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.346+0000 7fbeb3d96700 1 -- 172.21.15.89:0/2647459461 wait complete. 2023-09-30T22:17:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb3d96700 1 Processor -- start 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb3d96700 1 -- start start 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb3d96700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeac09e930 0x7fbeac06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb3d96700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb3d96700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeac09f650 0x7fbeac06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb3d96700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1a9150 con 0x7fbeac09f650 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb3d96700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0a31f0 con 0x7fbeac09f240 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb3d96700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1b72a0 con 0x7fbeac09e930 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb1b32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeac09e930 0x7fbeac06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb2333700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeac09f650 0x7fbeac06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb1b32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeac09e930 0x7fbeac06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52296/0 (socket says 172.21.15.89:52296) 2023-09-30T22:17:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb1b32700 1 -- 172.21.15.89:0/4160768577 learned_addr learned my addr 172.21.15.89:0/4160768577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.348+0000 7fbeb1331700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb1b32700 1 -- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 msgr2=0x7fbeac06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb1b32700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb1b32700 1 -- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeac09f650 msgr2=0x7fbeac06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb1b32700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeac09f650 0x7fbeac06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb1b32700 1 -- 172.21.15.89:0/4160768577 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80094a0 con 0x7fbeac09e930 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb2333700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeac09f650 0x7fbeac06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb1b32700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeac09e930 0x7fbeac06b8b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbea800ebe0 tx=0x7fbea800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbea27fc700 1 -- 172.21.15.89:0/4160768577 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800c850 con 0x7fbeac09e930 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbea27fc700 1 -- 172.21.15.89:0/4160768577 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea800c9b0 con 0x7fbeac09e930 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbea27fc700 1 -- 172.21.15.89:0/4160768577 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800cd40 con 0x7fbeac09e930 2023-09-30T22:17:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c0094a0 con 0x7fbeac09e930 2023-09-30T22:17:43.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.349+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac0a0190 con 0x7fbeac09e930 2023-09-30T22:17:43.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.350+0000 7fbea27fc700 1 -- 172.21.15.89:0/4160768577 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbea8013470 con 0x7fbeac09e930 2023-09-30T22:17:43.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.350+0000 7fbea27fc700 1 -- 172.21.15.89:0/4160768577 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbea803fb10 con 0x7fbeac09e930 2023-09-30T22:17:43.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.350+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac09fe10 con 0x7fbeac09e930 2023-09-30T22:17:43.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.355+0000 7fbea27fc700 1 -- 172.21.15.89:0/4160768577 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbea8013720 con 0x7fbeac09e930 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.433+0000 7f03a27c0700 1 Processor -- start 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f03a27c0700 1 -- start start 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f03a27c0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09e1d0 0x7f039c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f03a27c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c139170 0x7f039c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f03a27c0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c09b120 con 0x7f039c09d6f0 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f03a27c0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c06b460 con 0x7f039c09e1d0 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f03a27c0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f039c09a7e0 con 0x7f039c139170 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f039b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09e1d0 0x7f039c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f03a0d5d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c139170 0x7f039c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f039b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09e1d0 0x7f039c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40440/0 (socket says 172.21.15.89:40440) 2023-09-30T22:17:43.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f039b7fe700 1 -- 172.21.15.89:0/2512497755 learned_addr learned my addr 172.21.15.89:0/2512497755 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:43.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f039b7fe700 1 -- 172.21.15.89:0/2512497755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c139170 msgr2=0x7f039c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:43.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f039b7fe700 1 --2- 172.21.15.89:0/2512497755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c139170 0x7f039c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f039affd700 1 -- 172.21.15.89:0/2512497755 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201699397 0 0) 0x7f039c09b120 con 0x7f039c09d6f0 2023-09-30T22:17:43.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f039b7fe700 1 -- 172.21.15.89:0/2512497755 >> v1:172.21.15.89:6789/0 conn(0x7f039c09d6f0 legacy=0x7f039c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f039b7fe700 1 -- 172.21.15.89:0/2512497755 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c09a500 con 0x7f039c09e1d0 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.434+0000 7f03a0d5d700 1 --2- 172.21.15.89:0/2512497755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c139170 0x7f039c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.435+0000 7f039b7fe700 1 --2- 172.21.15.89:0/2512497755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09e1d0 0x7f039c138c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0390009920 tx=0x7f0390009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93a624fab32ca70 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.435+0000 7f039affd700 1 -- 172.21.15.89:0/2512497755 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011830 con 0x7f039c09e1d0 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.435+0000 7f039affd700 1 -- 172.21.15.89:0/2512497755 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390011990 con 0x7f039c09e1d0 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.435+0000 7f039affd700 1 -- 172.21.15.89:0/2512497755 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011d20 con 0x7f039c09e1d0 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.436+0000 7f03a27c0700 1 -- 172.21.15.89:0/2512497755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09e1d0 msgr2=0x7f039c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.436+0000 7f03a27c0700 1 --2- 172.21.15.89:0/2512497755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09e1d0 0x7f039c138c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0390009920 tx=0x7f0390009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.436+0000 7f03a27c0700 1 -- 172.21.15.89:0/2512497755 shutdown_connections 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.436+0000 7f03a27c0700 1 --2- 172.21.15.89:0/2512497755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c139170 0x7f039c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.436+0000 7f03a27c0700 1 --2- 172.21.15.89:0/2512497755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09e1d0 0x7f039c138c30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.436+0000 7f03a27c0700 1 -- 172.21.15.89:0/2512497755 >> 172.21.15.89:0/2512497755 conn(0x7f039c066f50 msgr2=0x7f039c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.436+0000 7f03a27c0700 1 -- 172.21.15.89:0/2512497755 shutdown_connections 2023-09-30T22:17:43.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.436+0000 7f03a27c0700 1 -- 172.21.15.89:0/2512497755 wait complete. 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.437+0000 7f03a27c0700 1 Processor -- start 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.437+0000 7f03a27c0700 1 -- start start 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f03a27c0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09d6f0 0x7f039c120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f03a27c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09e1d0 0x7f039c120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f03a27c0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c139170 0x7f039c1211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f03a27c0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c122ba0 con 0x7f039c139170 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f03a27c0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c136610 con 0x7f039c09d6f0 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f03a27c0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f039c09c710 con 0x7f039c09e1d0 2023-09-30T22:17:43.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09d6f0 0x7f039c120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09d6f0 0x7f039c120770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40448/0 (socket says 172.21.15.89:40448) 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039bfff700 1 -- 172.21.15.89:0/2308947715 learned_addr learned my addr 172.21.15.89:0/2308947715 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f03a0d5d700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c139170 0x7f039c1211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039b7fe700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09e1d0 0x7f039c120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039bfff700 1 -- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09e1d0 msgr2=0x7f039c120cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039bfff700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09e1d0 0x7f039c120cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039bfff700 1 -- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c139170 msgr2=0x7f039c1211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039bfff700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c139170 0x7f039c1211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.438+0000 7f039bfff700 1 -- 172.21.15.89:0/2308947715 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394009a70 con 0x7f039c09d6f0 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.439+0000 7f03a0d5d700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c139170 0x7f039c1211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:43.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.439+0000 7f039bfff700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09d6f0 0x7f039c120770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f038c001eb0 tx=0x7f038c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.439+0000 7f0398ff9700 1 -- 172.21.15.89:0/2308947715 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0054e0 con 0x7f039c09d6f0 2023-09-30T22:17:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.439+0000 7f0398ff9700 1 -- 172.21.15.89:0/2308947715 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c003680 con 0x7f039c09d6f0 2023-09-30T22:17:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.439+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0390009550 con 0x7f039c09d6f0 2023-09-30T22:17:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.439+0000 7f0398ff9700 1 -- 172.21.15.89:0/2308947715 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0039f0 con 0x7f039c09d6f0 2023-09-30T22:17:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.439+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c072150 con 0x7f039c09d6f0 2023-09-30T22:17:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.440+0000 7f0398ff9700 1 -- 172.21.15.89:0/2308947715 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f038c003b50 con 0x7f039c09d6f0 2023-09-30T22:17:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.440+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c123e80 con 0x7f039c09d6f0 2023-09-30T22:17:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.440+0000 7f0398ff9700 1 -- 172.21.15.89:0/2308947715 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f038c03eb00 con 0x7f039c09d6f0 2023-09-30T22:17:43.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.444+0000 7f0398ff9700 1 -- 172.21.15.89:0/2308947715 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f038c01b3b0 con 0x7f039c09d6f0 2023-09-30T22:17:43.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f5656d0a700 1 Processor -- start 2023-09-30T22:17:43.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f5656d0a700 1 -- start start 2023-09-30T22:17:43.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f5656d0a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f565009d9f0 0x7f5650136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f5656d0a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f5656d0a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565009e130 con 0x7f56501316c0 2023-09-30T22:17:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f5656d0a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56501321a0 con 0x7f565009d9f0 2023-09-30T22:17:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f5656d0a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56501312b0 con 0x7f5650136a70 2023-09-30T22:17:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f564ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f565009d9f0 0x7f5650136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.473+0000 7f56552a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f56552a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52338/0 (socket says 172.21.15.89:52338) 2023-09-30T22:17:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f564ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f565009d9f0 0x7f5650136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33572/0 (socket says 172.21.15.89:33572) 2023-09-30T22:17:43.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f56552a7700 1 -- 172.21.15.89:0/131863835 learned_addr learned my addr 172.21.15.89:0/131863835 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:43.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f56552a7700 1 -- 172.21.15.89:0/131863835 >> v2:172.21.15.89:3300/0 conn(0x7f565009d9f0 msgr2=0x7f5650136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:43.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f56552a7700 1 --2- 172.21.15.89:0/131863835 >> v2:172.21.15.89:3300/0 conn(0x7f565009d9f0 0x7f5650136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f56552a7700 1 -- 172.21.15.89:0/131863835 >> v1:172.21.15.89:6789/0 conn(0x7f56501316c0 legacy=0x7f5650131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f56552a7700 1 -- 172.21.15.89:0/131863835 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565009f6e0 con 0x7f5650136a70 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f564ffff700 1 --2- 172.21.15.89:0/131863835 >> v2:172.21.15.89:3300/0 conn(0x7f565009d9f0 0x7f5650136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f56552a7700 1 --2- 172.21.15.89:0/131863835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650138e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5648009e80 tx=0x7f564800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e0764dc56c153e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f564f7fe700 1 -- 172.21.15.89:0/131863835 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800e070 con 0x7f5650136a70 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f564f7fe700 1 -- 172.21.15.89:0/131863835 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564800b8d0 con 0x7f5650136a70 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.474+0000 7f564f7fe700 1 -- 172.21.15.89:0/131863835 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800bc80 con 0x7f5650136a70 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.475+0000 7f5656d0a700 1 -- 172.21.15.89:0/131863835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 msgr2=0x7f5650138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.475+0000 7f5656d0a700 1 --2- 172.21.15.89:0/131863835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650138e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5648009e80 tx=0x7f564800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.475+0000 7f5656d0a700 1 -- 172.21.15.89:0/131863835 shutdown_connections 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.475+0000 7f5656d0a700 1 --2- 172.21.15.89:0/131863835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650138e50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.475+0000 7f5656d0a700 1 --2- 172.21.15.89:0/131863835 >> v2:172.21.15.89:3300/0 conn(0x7f565009d9f0 0x7f5650136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.475+0000 7f5656d0a700 1 -- 172.21.15.89:0/131863835 >> 172.21.15.89:0/131863835 conn(0x7f5650066f50 msgr2=0x7f565009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:43.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.475+0000 7f5656d0a700 1 -- 172.21.15.89:0/131863835 shutdown_connections 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.475+0000 7f5656d0a700 1 -- 172.21.15.89:0/131863835 wait complete. 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5656d0a700 1 Processor -- start 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5656d0a700 1 -- start start 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5656d0a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f565009d9f0 0x7f5650070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5656d0a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56501316c0 0x7f5650071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5656d0a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5656d0a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650135570 con 0x7f565009d9f0 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5656d0a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565006fc60 con 0x7f56501316c0 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5656d0a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5650070200 con 0x7f5650136a70 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5654aa6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f565009d9f0 0x7f5650070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5654aa6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f565009d9f0 0x7f5650070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33584/0 (socket says 172.21.15.89:33584) 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f5654aa6700 1 -- 172.21.15.89:0/3716434784 learned_addr learned my addr 172.21.15.89:0/3716434784 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.476+0000 7f56552a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f564ffff700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56501316c0 0x7f5650071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f56552a7700 1 -- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56501316c0 msgr2=0x7f5650071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f56552a7700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56501316c0 0x7f5650071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f56552a7700 1 -- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f565009d9f0 msgr2=0x7f5650070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f56552a7700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f565009d9f0 0x7f5650070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f56552a7700 1 -- 172.21.15.89:0/3716434784 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0094a0 con 0x7f5650136a70 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f564ffff700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56501316c0 0x7f5650071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f56552a7700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650071860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5648015000 tx=0x7f5648004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f564d7fa700 1 -- 172.21.15.89:0/3716434784 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56480048b0 con 0x7f5650136a70 2023-09-30T22:17:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f564d7fa700 1 -- 172.21.15.89:0/3716434784 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56480092e0 con 0x7f5650136a70 2023-09-30T22:17:43.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5648009b30 con 0x7f5650136a70 2023-09-30T22:17:43.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f564d7fa700 1 -- 172.21.15.89:0/3716434784 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564801c680 con 0x7f5650136a70 2023-09-30T22:17:43.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.477+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5650071df0 con 0x7f5650136a70 2023-09-30T22:17:43.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.478+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5650070a50 con 0x7f5650136a70 2023-09-30T22:17:43.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.479+0000 7f564d7fa700 1 -- 172.21.15.89:0/3716434784 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f564801c7e0 con 0x7f5650136a70 2023-09-30T22:17:43.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.479+0000 7f564d7fa700 1 -- 172.21.15.89:0/3716434784 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5648041640 con 0x7f5650136a70 2023-09-30T22:17:43.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.487+0000 7f564d7fa700 1 -- 172.21.15.89:0/3716434784 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5648023950 con 0x7f5650136a70 2023-09-30T22:17:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.553+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbeac074ea0 con 0x7fbeac09e930 2023-09-30T22:17:43.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.553+0000 7fbea27fc700 1 -- 172.21.15.89:0/4160768577 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbea8021020 con 0x7fbeac09e930 2023-09-30T22:17:43.554 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeac09e930 msgr2=0x7fbeac06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeac09e930 0x7fbeac06b8b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbea800ebe0 tx=0x7fbea800eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 shutdown_connections 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeac09f650 0x7fbeac06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeac09e930 0x7fbeac06b8b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 --2- 172.21.15.89:0/4160768577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeac09f240 0x7fbeac06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 >> 172.21.15.89:0/4160768577 conn(0x7fbeac0a4210 msgr2=0x7fbeac0a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 shutdown_connections 2023-09-30T22:17:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.555+0000 7fbeb3d96700 1 -- 172.21.15.89:0/4160768577 wait complete. 2023-09-30T22:17:43.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:43.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.644+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f039c074ea0 con 0x7f039c09d6f0 2023-09-30T22:17:43.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.645+0000 7f0398ff9700 1 -- 172.21.15.89:0/2308947715 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f038c017020 con 0x7f039c09d6f0 2023-09-30T22:17:43.646 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:43.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09d6f0 msgr2=0x7f039c120770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09d6f0 0x7f039c120770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f038c001eb0 tx=0x7f038c00c800 comp rx=0 tx=0).stop 2023-09-30T22:17:43.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 shutdown_connections 2023-09-30T22:17:43.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c139170 0x7f039c1211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c09d6f0 0x7f039c120770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 --2- 172.21.15.89:0/2308947715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09e1d0 0x7f039c120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 >> 172.21.15.89:0/2308947715 conn(0x7f039c066f50 msgr2=0x7f039c136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:43.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 shutdown_connections 2023-09-30T22:17:43.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.647+0000 7f03a27c0700 1 -- 172.21.15.89:0/2308947715 wait complete. 2023-09-30T22:17:43.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.682+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5650074ea0 con 0x7f5650136a70 2023-09-30T22:17:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.682+0000 7f564d7fa700 1 -- 172.21.15.89:0/3716434784 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5648033090 con 0x7f5650136a70 2023-09-30T22:17:43.683 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:43.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 msgr2=0x7f5650071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:43.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650071860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5648015000 tx=0x7f5648004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 shutdown_connections 2023-09-30T22:17:43.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5650136a70 0x7f5650071860 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f565009d9f0 0x7f5650070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 --2- 172.21.15.89:0/3716434784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56501316c0 0x7f5650071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:43.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 >> 172.21.15.89:0/3716434784 conn(0x7f5650066f50 msgr2=0x7f56500a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:43.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 shutdown_connections 2023-09-30T22:17:43.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:43.684+0000 7f5656d0a700 1 -- 172.21.15.89:0/3716434784 wait complete. 2023-09-30T22:17:43.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:44.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd4a11e4700 1 Processor -- start 2023-09-30T22:17:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd4a11e4700 1 -- start start 2023-09-30T22:17:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd4a11e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd49c133a80 0x7fd49c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd4a11e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c12ff20 0x7fd49c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd4a11e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd4a11e4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49c06c570 con 0x7fd49c133a80 2023-09-30T22:17:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd4a11e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c06bc00 con 0x7fd49c130870 2023-09-30T22:17:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd4a11e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0a0040 con 0x7fd49c12ff20 2023-09-30T22:17:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c12ff20 0x7fd49c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd49c133a80 0x7fd49c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49b59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd49c133a80 0x7fd49c133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33600/0 (socket says 172.21.15.89:33600) 2023-09-30T22:17:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c12ff20 0x7fd49c130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52370/0 (socket says 172.21.15.89:52370) 2023-09-30T22:17:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49ad9d700 1 -- 172.21.15.89:0/3542904603 learned_addr learned my addr 172.21.15.89:0/3542904603 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49b59e700 1 -- 172.21.15.89:0/3542904603 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c12ff20 msgr2=0x7fd49c130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49b59e700 1 --2- 172.21.15.89:0/3542904603 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c12ff20 0x7fd49c130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49b59e700 1 -- 172.21.15.89:0/3542904603 >> v2:172.21.15.89:3300/0 conn(0x7fd49c133a80 msgr2=0x7fd49c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:44.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49b59e700 1 --2- 172.21.15.89:0/3542904603 >> v2:172.21.15.89:3300/0 conn(0x7fd49c133a80 0x7fd49c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49b59e700 1 -- 172.21.15.89:0/3542904603 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c133860 con 0x7fd49c130870 2023-09-30T22:17:44.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.351+0000 7fd49ad9d700 1 --2- 172.21.15.89:0/3542904603 >> v2:172.21.15.89:3300/0 conn(0x7fd49c133a80 0x7fd49c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd49b59e700 1 --2- 172.21.15.89:0/3542904603 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c13a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd494009d90 tx=0x7fd49400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec19de897c7588f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:44.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd499d9b700 1 -- 172.21.15.89:0/3542904603 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49400e070 con 0x7fd49c130870 2023-09-30T22:17:44.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd499d9b700 1 -- 172.21.15.89:0/3542904603 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49400bb20 con 0x7fd49c130870 2023-09-30T22:17:44.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd499d9b700 1 -- 172.21.15.89:0/3542904603 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd494012430 con 0x7fd49c130870 2023-09-30T22:17:44.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3542904603 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 msgr2=0x7fd49c13a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 --2- 172.21.15.89:0/3542904603 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c13a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd494009d90 tx=0x7fd49400b330 comp rx=0 tx=0).stop 2023-09-30T22:17:44.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3542904603 shutdown_connections 2023-09-30T22:17:44.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 --2- 172.21.15.89:0/3542904603 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c13a840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 --2- 172.21.15.89:0/3542904603 >> v2:172.21.15.89:3300/0 conn(0x7fd49c133a80 0x7fd49c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 --2- 172.21.15.89:0/3542904603 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c12ff20 0x7fd49c130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3542904603 >> 172.21.15.89:0/3542904603 conn(0x7fd49c0a4210 msgr2=0x7fd49c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:44.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3542904603 shutdown_connections 2023-09-30T22:17:44.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.352+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3542904603 wait complete. 2023-09-30T22:17:44.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd4a11e4700 1 Processor -- start 2023-09-30T22:17:44.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd4a11e4700 1 -- start start 2023-09-30T22:17:44.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd4a11e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c12ff20 0x7fd49c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd4a11e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd4a11e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c133a80 0x7fd49c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd4a11e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c12d940 con 0x7fd49c12ff20 2023-09-30T22:17:44.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd4a11e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c11ff20 con 0x7fd49c130870 2023-09-30T22:17:44.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd4a11e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c09dd50 con 0x7fd49c133a80 2023-09-30T22:17:44.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49b59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c133a80 0x7fd49c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c12ff20 0x7fd49c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49b59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c133a80 0x7fd49c09cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52386/0 (socket says 172.21.15.89:52386) 2023-09-30T22:17:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40478/0 (socket says 172.21.15.89:40478) 2023-09-30T22:17:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49b59e700 1 -- 172.21.15.89:0/3794268674 learned_addr learned my addr 172.21.15.89:0/3794268674 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49a59c700 1 -- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c133a80 msgr2=0x7fd49c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49a59c700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c133a80 0x7fd49c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49a59c700 1 -- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c12ff20 msgr2=0x7fd49c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49a59c700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c12ff20 0x7fd49c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49a59c700 1 -- 172.21.15.89:0/3794268674 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490009550 con 0x7fd49c130870 2023-09-30T22:17:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49ad9d700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c12ff20 0x7fd49c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.354+0000 7fd49b59e700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c133a80 0x7fd49c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.355+0000 7fd49a59c700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c09c9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e930 tx=0x7fd48c00ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.355+0000 7fd4837fe700 1 -- 172.21.15.89:0/3794268674 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c013680 con 0x7fd49c130870 2023-09-30T22:17:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.355+0000 7fd4837fe700 1 -- 172.21.15.89:0/3794268674 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c0137e0 con 0x7fd49c130870 2023-09-30T22:17:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.355+0000 7fd4837fe700 1 -- 172.21.15.89:0/3794268674 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c013b50 con 0x7fd49c130870 2023-09-30T22:17:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.355+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd494009a70 con 0x7fd49c130870 2023-09-30T22:17:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.355+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c11fbd0 con 0x7fd49c130870 2023-09-30T22:17:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.356+0000 7fd4837fe700 1 -- 172.21.15.89:0/3794268674 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd48c013cb0 con 0x7fd49c130870 2023-09-30T22:17:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.356+0000 7fd4837fe700 1 -- 172.21.15.89:0/3794268674 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd48c03f9f0 con 0x7fd49c130870 2023-09-30T22:17:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.356+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c11f460 con 0x7fd49c130870 2023-09-30T22:17:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.360+0000 7fd4837fe700 1 -- 172.21.15.89:0/3794268674 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd48c01adc0 con 0x7fd49c130870 2023-09-30T22:17:44.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.545+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd49c074ea0 con 0x7fd49c130870 2023-09-30T22:17:44.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.546+0000 7fd4837fe700 1 -- 172.21.15.89:0/3794268674 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd48c021030 con 0x7fd49c130870 2023-09-30T22:17:44.547 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 msgr2=0x7fd49c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c09c9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e930 tx=0x7fd48c00ec40 comp rx=0 tx=0).stop 2023-09-30T22:17:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 shutdown_connections 2023-09-30T22:17:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c133a80 0x7fd49c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c12ff20 0x7fd49c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 --2- 172.21.15.89:0/3794268674 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c130870 0x7fd49c09c9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 >> 172.21.15.89:0/3794268674 conn(0x7fd49c0a4210 msgr2=0x7fd49c09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 shutdown_connections 2023-09-30T22:17:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.548+0000 7fd4a11e4700 1 -- 172.21.15.89:0/3794268674 wait complete. 2023-09-30T22:17:44.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:44.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:44.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:44.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.691+0000 7fb4fe9e2700 1 Processor -- start 2023-09-30T22:17:44.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.691+0000 7fb4fe9e2700 1 -- start start 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fe9e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809d6f0 0x7fb4f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fe9e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fe9e2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f809b120 con 0x7fb4f809e000 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fe9e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f806b460 con 0x7fb4f809d6f0 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fe9e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f809a7e0 con 0x7fb4f809e410 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fcf7f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4f7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809d6f0 0x7fb4f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4f77fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb4f809e000 0x7fb4f81339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42706/0 (socket says 172.21.15.89:42706) 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4f77fe700 1 -- 172.21.15.89:0/1351200661 learned_addr learned my addr 172.21.15.89:0/1351200661 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4f6ffd700 1 -- 172.21.15.89:0/1351200661 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365199258 0 0) 0x7fb4f809b120 con 0x7fb4f809e000 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fcf7f700 1 -- 172.21.15.89:0/1351200661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809d6f0 msgr2=0x7fb4f809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fcf7f700 1 --2- 172.21.15.89:0/1351200661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809d6f0 0x7fb4f809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fcf7f700 1 -- 172.21.15.89:0/1351200661 >> v1:172.21.15.89:6789/0 conn(0x7fb4f809e000 legacy=0x7fb4f81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4fcf7f700 1 -- 172.21.15.89:0/1351200661 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f809a500 con 0x7fb4f809e410 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.692+0000 7fb4f7fff700 1 --2- 172.21.15.89:0/1351200661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809d6f0 0x7fb4f809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fcf7f700 1 --2- 172.21.15.89:0/1351200661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8134100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009df0 tx=0x7fb4f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db0e6f7fb5815d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4f6ffd700 1 -- 172.21.15.89:0/1351200661 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000e070 con 0x7fb4f809e410 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4f6ffd700 1 -- 172.21.15.89:0/1351200661 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f000bae0 con 0x7fb4f809e410 2023-09-30T22:17:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4f6ffd700 1 -- 172.21.15.89:0/1351200661 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000be90 con 0x7fb4f809e410 2023-09-30T22:17:44.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/1351200661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 msgr2=0x7fb4f8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fe9e2700 1 --2- 172.21.15.89:0/1351200661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8134100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009df0 tx=0x7fb4f000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:44.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/1351200661 shutdown_connections 2023-09-30T22:17:44.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fe9e2700 1 --2- 172.21.15.89:0/1351200661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809d6f0 0x7fb4f809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fe9e2700 1 --2- 172.21.15.89:0/1351200661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8134100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/1351200661 >> 172.21.15.89:0/1351200661 conn(0x7fb4f8066f50 msgr2=0x7fb4f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/1351200661 shutdown_connections 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.693+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/1351200661 wait complete. 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.694+0000 7fb4fe9e2700 1 Processor -- start 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.694+0000 7fb4fe9e2700 1 -- start start 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fe9e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f809d6f0 0x7fb4f8122300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fe9e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809e000 0x7fb4f8122840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fe9e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8122d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fe9e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81237d0 con 0x7fb4f809d6f0 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fe9e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81336d0 con 0x7fb4f809e000 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fe9e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8124da0 con 0x7fb4f809e410 2023-09-30T22:17:44.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809e000 0x7fb4f8122840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809e000 0x7fb4f8122840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40490/0 (socket says 172.21.15.89:40490) 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4f77fe700 1 -- 172.21.15.89:0/582859148 learned_addr learned my addr 172.21.15.89:0/582859148 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fcf7f700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8122d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4f7fff700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f809d6f0 0x7fb4f8122300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fcf7f700 1 -- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809e000 msgr2=0x7fb4f8122840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fcf7f700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809e000 0x7fb4f8122840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fcf7f700 1 -- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f809d6f0 msgr2=0x7fb4f8122300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fcf7f700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f809d6f0 0x7fb4f8122300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4fcf7f700 1 -- 172.21.15.89:0/582859148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e80094a0 con 0x7fb4f809e410 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.695+0000 7fb4f7fff700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f809d6f0 0x7fb4f8122300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.696+0000 7fb4fcf7f700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8122d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0016000 tx=0x7fb4f0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:44.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.696+0000 7fb4f4ff9700 1 -- 172.21.15.89:0/582859148 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00045d0 con 0x7fb4f809e410 2023-09-30T22:17:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.696+0000 7fb4f4ff9700 1 -- 172.21.15.89:0/582859148 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0020070 con 0x7fb4f809e410 2023-09-30T22:17:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.696+0000 7fb4f4ff9700 1 -- 172.21.15.89:0/582859148 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0027480 con 0x7fb4f809e410 2023-09-30T22:17:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.696+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f0009a70 con 0x7fb4f809e410 2023-09-30T22:17:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.696+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f80a2020 con 0x7fb4f809e410 2023-09-30T22:17:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.697+0000 7fb4f4ff9700 1 -- 172.21.15.89:0/582859148 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb4f00275e0 con 0x7fb4f809e410 2023-09-30T22:17:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.697+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f811f030 con 0x7fb4f809e410 2023-09-30T22:17:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.697+0000 7fb4f4ff9700 1 -- 172.21.15.89:0/582859148 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb4f0022070 con 0x7fb4f809e410 2023-09-30T22:17:44.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:44.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.701+0000 7fb4f4ff9700 1 -- 172.21.15.89:0/582859148 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4f0027890 con 0x7fb4f809e410 2023-09-30T22:17:44.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa96069e700 1 Processor -- start 2023-09-30T22:17:44.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa96069e700 1 -- start start 2023-09-30T22:17:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa96069e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806a770 0x7fa95806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa96069e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806b080 0x7fa958131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa96069e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95809ece0 con 0x7fa958131b40 2023-09-30T22:17:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa96069e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95809e760 con 0x7fa95806a770 2023-09-30T22:17:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa96069e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa95809e040 con 0x7fa95806b080 2023-09-30T22:17:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa95dc39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806b080 0x7fa958131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa95ec3b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa958131b40 0x7fa958138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42718/0 (socket says 172.21.15.89:42718) 2023-09-30T22:17:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa95e43a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806a770 0x7fa95806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.789+0000 7fa95ec3b700 1 -- 172.21.15.89:0/1635138626 learned_addr learned my addr 172.21.15.89:0/1635138626 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95d438700 1 -- 172.21.15.89:0/1635138626 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246389315 0 0) 0x7fa95809ece0 con 0x7fa958131b40 2023-09-30T22:17:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95d438700 1 -- 172.21.15.89:0/1635138626 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9440035b0 con 0x7fa958131b40 2023-09-30T22:17:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95e43a700 1 -- 172.21.15.89:0/1635138626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806b080 msgr2=0x7fa958131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:44.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95e43a700 1 --2- 172.21.15.89:0/1635138626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806b080 0x7fa958131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95e43a700 1 -- 172.21.15.89:0/1635138626 >> v1:172.21.15.89:6789/0 conn(0x7fa958131b40 legacy=0x7fa958138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95e43a700 1 -- 172.21.15.89:0/1635138626 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95809f630 con 0x7fa95806a770 2023-09-30T22:17:44.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95dc39700 1 --2- 172.21.15.89:0/1635138626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806b080 0x7fa958131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95e43a700 1 --2- 172.21.15.89:0/1635138626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806a770 0x7fa95806ab40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa954009870 tx=0x7fa954009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e9a8f2c6ff39698 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:44.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95d438700 1 -- 172.21.15.89:0/1635138626 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954011830 con 0x7fa95806a770 2023-09-30T22:17:44.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95d438700 1 -- 172.21.15.89:0/1635138626 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954011990 con 0x7fa95806a770 2023-09-30T22:17:44.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.790+0000 7fa95d438700 1 -- 172.21.15.89:0/1635138626 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954011d00 con 0x7fa95806a770 2023-09-30T22:17:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.791+0000 7fa96069e700 1 -- 172.21.15.89:0/1635138626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806a770 msgr2=0x7fa95806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.791+0000 7fa96069e700 1 --2- 172.21.15.89:0/1635138626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806a770 0x7fa95806ab40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa954009870 tx=0x7fa954009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.791+0000 7fa96069e700 1 -- 172.21.15.89:0/1635138626 shutdown_connections 2023-09-30T22:17:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.791+0000 7fa96069e700 1 --2- 172.21.15.89:0/1635138626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806a770 0x7fa95806ab40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.791+0000 7fa96069e700 1 --2- 172.21.15.89:0/1635138626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806b080 0x7fa958131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.791+0000 7fa96069e700 1 -- 172.21.15.89:0/1635138626 >> 172.21.15.89:0/1635138626 conn(0x7fa958066f50 msgr2=0x7fa95809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.791+0000 7fa96069e700 1 -- 172.21.15.89:0/1635138626 shutdown_connections 2023-09-30T22:17:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.791+0000 7fa96069e700 1 -- 172.21.15.89:0/1635138626 wait complete. 2023-09-30T22:17:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa96069e700 1 Processor -- start 2023-09-30T22:17:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa96069e700 1 -- start start 2023-09-30T22:17:44.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa96069e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806a770 0x7fa95809acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa96069e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806b080 0x7fa95809b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa96069e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa958131b40 0x7fa9581c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa96069e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9580a3be0 con 0x7fa958131b40 2023-09-30T22:17:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa96069e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95809a8c0 con 0x7fa95806b080 2023-09-30T22:17:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa96069e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa958126160 con 0x7fa95806a770 2023-09-30T22:17:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa95dc39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806b080 0x7fa95809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa95dc39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806b080 0x7fa95809b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40516/0 (socket says 172.21.15.89:40516) 2023-09-30T22:17:44.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.793+0000 7fa95dc39700 1 -- 172.21.15.89:0/4291089161 learned_addr learned my addr 172.21.15.89:0/4291089161 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95ec3b700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa958131b40 0x7fa9581c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95e43a700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806a770 0x7fa95809acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95ec3b700 1 -- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806a770 msgr2=0x7fa95809acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95ec3b700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806a770 0x7fa95809acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95ec3b700 1 -- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806b080 msgr2=0x7fa95809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95ec3b700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806b080 0x7fa95809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95ec3b700 1 -- 172.21.15.89:0/4291089161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480094a0 con 0x7fa958131b40 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95dc39700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806b080 0x7fa95809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa95ec3b700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa958131b40 0x7fa9581c8180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa95000ca80 tx=0x7fa95000cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:44.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa94effd700 1 -- 172.21.15.89:0/4291089161 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000a9e0 con 0x7fa958131b40 2023-09-30T22:17:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa94effd700 1 -- 172.21.15.89:0/4291089161 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95000ab40 con 0x7fa958131b40 2023-09-30T22:17:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9540094a0 con 0x7fa958131b40 2023-09-30T22:17:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa94effd700 1 -- 172.21.15.89:0/4291089161 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950012400 con 0x7fa958131b40 2023-09-30T22:17:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.794+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9581254b0 con 0x7fa958131b40 2023-09-30T22:17:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.795+0000 7fa94effd700 1 -- 172.21.15.89:0/4291089161 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa95001b3b0 con 0x7fa958131b40 2023-09-30T22:17:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.795+0000 7fa94effd700 1 -- 172.21.15.89:0/4291089161 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa95003f280 con 0x7fa958131b40 2023-09-30T22:17:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.796+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958074ea0 con 0x7fa958131b40 2023-09-30T22:17:44.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.805+0000 7fa94effd700 1 -- 172.21.15.89:0/4291089161 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9500192c0 con 0x7fa958131b40 2023-09-30T22:17:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cf9c9700 1 Processor -- start 2023-09-30T22:17:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cf9c9700 1 -- start start 2023-09-30T22:17:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cf9c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809e930 0x7f51c809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cf9c9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51c809f650 0x7f51c8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cf9c9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c8070940 con 0x7f51c809f240 2023-09-30T22:17:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cf9c9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c806fd60 con 0x7f51c809f650 2023-09-30T22:17:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cf9c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51c806fa80 con 0x7f51c809e930 2023-09-30T22:17:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cd765700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809e930 0x7f51c809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cdf66700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51c809f650 0x7f51c8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cd765700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809e930 0x7f51c809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52438/0 (socket says 172.21.15.89:52438) 2023-09-30T22:17:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.833+0000 7f51cd765700 1 -- 172.21.15.89:0/3493081320 learned_addr learned my addr 172.21.15.89:0/3493081320 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51cdf66700 1 -- 172.21.15.89:0/3493081320 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809e930 msgr2=0x7f51c809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51cdf66700 1 --2- 172.21.15.89:0/3493081320 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809e930 0x7f51c809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51cdf66700 1 -- 172.21.15.89:0/3493081320 >> v1:172.21.15.89:6789/0 conn(0x7f51c809f240 legacy=0x7f51c81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51cdf66700 1 -- 172.21.15.89:0/3493081320 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c809fb60 con 0x7f51c809f650 2023-09-30T22:17:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51cd765700 1 --2- 172.21.15.89:0/3493081320 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809e930 0x7f51c809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51cdf66700 1 --2- 172.21.15.89:0/3493081320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809f650 0x7f51c8131f00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51c000f370 tx=0x7f51c000f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689d1ad922288147 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51bffff700 1 -- 172.21.15.89:0/3493081320 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0012070 con 0x7f51c809f650 2023-09-30T22:17:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51bffff700 1 -- 172.21.15.89:0/3493081320 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c000fe70 con 0x7f51c809f650 2023-09-30T22:17:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.834+0000 7f51bffff700 1 -- 172.21.15.89:0/3493081320 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000d2f0 con 0x7f51c809f650 2023-09-30T22:17:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.835+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3493081320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809f650 msgr2=0x7f51c8131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.835+0000 7f51cf9c9700 1 --2- 172.21.15.89:0/3493081320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809f650 0x7f51c8131f00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51c000f370 tx=0x7f51c000f680 comp rx=0 tx=0).stop 2023-09-30T22:17:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.835+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3493081320 shutdown_connections 2023-09-30T22:17:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.835+0000 7f51cf9c9700 1 --2- 172.21.15.89:0/3493081320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809f650 0x7f51c8131f00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.835+0000 7f51cf9c9700 1 --2- 172.21.15.89:0/3493081320 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809e930 0x7f51c809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.835+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3493081320 >> 172.21.15.89:0/3493081320 conn(0x7f51c80a4210 msgr2=0x7f51c80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.835+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3493081320 shutdown_connections 2023-09-30T22:17:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.835+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3493081320 wait complete. 2023-09-30T22:17:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cf9c9700 1 Processor -- start 2023-09-30T22:17:44.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cf9c9700 1 -- start start 2023-09-30T22:17:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cf9c9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809e930 0x7f51c81a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cf9c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51c809f240 0x7f51c81a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cf9c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809f650 0x7f51c81c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cf9c9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c81a42c0 con 0x7f51c809e930 2023-09-30T22:17:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cf9c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c81a5710 con 0x7f51c809f240 2023-09-30T22:17:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cf9c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51c81b5330 con 0x7f51c809f650 2023-09-30T22:17:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cdf66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809f650 0x7f51c81c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cd765700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809e930 0x7f51c81a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cdf66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809f650 0x7f51c81c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52450/0 (socket says 172.21.15.89:52450) 2023-09-30T22:17:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51cdf66700 1 -- 172.21.15.89:0/3690278927 learned_addr learned my addr 172.21.15.89:0/3690278927 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.836+0000 7f51ccf64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51c809f240 0x7f51c81a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:44.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51cd765700 1 -- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809f650 msgr2=0x7f51c81c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51cd765700 1 --2- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809f650 0x7f51c81c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51cd765700 1 -- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51c809f240 msgr2=0x7f51c81a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51cd765700 1 --2- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51c809f240 0x7f51c81a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51cd765700 1 -- 172.21.15.89:0/3690278927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c4009550 con 0x7f51c809e930 2023-09-30T22:17:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51ccf64700 1 --2- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51c809f240 0x7f51c81a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51cd765700 1 --2- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809e930 0x7f51c81a2470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51c400e9a0 tx=0x7f51c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:44.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51bdffb700 1 -- 172.21.15.89:0/3690278927 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4004030 con 0x7f51c809e930 2023-09-30T22:17:44.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51bdffb700 1 -- 172.21.15.89:0/3690278927 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c4004190 con 0x7f51c809e930 2023-09-30T22:17:44.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c000f050 con 0x7f51c809e930 2023-09-30T22:17:44.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51bdffb700 1 -- 172.21.15.89:0/3690278927 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4004520 con 0x7f51c809e930 2023-09-30T22:17:44.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.837+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c80a0190 con 0x7f51c809e930 2023-09-30T22:17:44.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.838+0000 7f51bdffb700 1 -- 172.21.15.89:0/3690278927 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f51c40036a0 con 0x7f51c809e930 2023-09-30T22:17:44.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.838+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c8074ea0 con 0x7f51c809e930 2023-09-30T22:17:44.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.838+0000 7f51bdffb700 1 -- 172.21.15.89:0/3690278927 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f51c403f4b0 con 0x7f51c809e930 2023-09-30T22:17:44.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.847+0000 7f51bdffb700 1 -- 172.21.15.89:0/3690278927 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f51c4019300 con 0x7f51c809e930 2023-09-30T22:17:44.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.899+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb4f809c710 con 0x7fb4f809e410 2023-09-30T22:17:44.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.900+0000 7fb4f4ff9700 1 -- 172.21.15.89:0/582859148 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4f002c030 con 0x7fb4f809e410 2023-09-30T22:17:44.901 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:44.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 msgr2=0x7fb4f8122d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:44.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8122d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0016000 tx=0x7fb4f0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 shutdown_connections 2023-09-30T22:17:44.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f809e410 0x7fb4f8122d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f809d6f0 0x7fb4f8122300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 --2- 172.21.15.89:0/582859148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4f809e000 0x7fb4f8122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:44.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 >> 172.21.15.89:0/582859148 conn(0x7fb4f8066f50 msgr2=0x7fb4f8133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:44.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 shutdown_connections 2023-09-30T22:17:44.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:44.903+0000 7fb4fe9e2700 1 -- 172.21.15.89:0/582859148 wait complete. 2023-09-30T22:17:44.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:45.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.004+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa958125e00 con 0x7fa958131b40 2023-09-30T22:17:45.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.005+0000 7fa94effd700 1 -- 172.21.15.89:0/4291089161 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa958125e00 con 0x7fa958131b40 2023-09-30T22:17:45.006 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa958131b40 msgr2=0x7fa9581c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa958131b40 0x7fa9581c8180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa95000ca80 tx=0x7fa95000cd90 comp rx=0 tx=0).stop 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 shutdown_connections 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa958131b40 0x7fa9581c8180 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa95806a770 0x7fa95809acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 --2- 172.21.15.89:0/4291089161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa95806b080 0x7fa95809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 >> 172.21.15.89:0/4291089161 conn(0x7fa958066f50 msgr2=0x7fa95809fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 shutdown_connections 2023-09-30T22:17:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.007+0000 7fa96069e700 1 -- 172.21.15.89:0/4291089161 wait complete. 2023-09-30T22:17:45.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:45.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.041+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f51c806b250 con 0x7f51c809e930 2023-09-30T22:17:45.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.041+0000 7f51bdffb700 1 -- 172.21.15.89:0/3690278927 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51c4011b40 con 0x7f51c809e930 2023-09-30T22:17:45.042 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:45.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809e930 msgr2=0x7f51c81a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:45.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 --2- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809e930 0x7f51c81a2470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51c400e9a0 tx=0x7f51c400ecb0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 shutdown_connections 2023-09-30T22:17:45.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 --2- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f51c809f650 0x7f51c81c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 --2- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c809e930 0x7f51c81a2470 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 --2- 172.21.15.89:0/3690278927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51c809f240 0x7f51c81a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 >> 172.21.15.89:0/3690278927 conn(0x7f51c80a4210 msgr2=0x7f51c81b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:45.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 shutdown_connections 2023-09-30T22:17:45.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.043+0000 7f51cf9c9700 1 -- 172.21.15.89:0/3690278927 wait complete. 2023-09-30T22:17:45.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:45.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:45.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.687+0000 7f7cdbc65700 1 Processor -- start 2023-09-30T22:17:45.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.688+0000 7f7cdbc65700 1 -- start start 2023-09-30T22:17:45.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.688+0000 7f7cdbc65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409d6f0 0x7f7cd409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:45.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.688+0000 7f7cdbc65700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7cd409e410 0x7f7cd4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:45.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.688+0000 7f7cdbc65700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd409b120 con 0x7f7cd409e000 2023-09-30T22:17:45.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.688+0000 7f7cdbc65700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd406b460 con 0x7f7cd409e410 2023-09-30T22:17:45.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.688+0000 7f7cdbc65700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd409a7e0 con 0x7f7cd409d6f0 2023-09-30T22:17:45.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.688+0000 7f7cd9200700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7cd409e000 0x7f7cd41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42728/0 (socket says 172.21.15.89:42728) 2023-09-30T22:17:45.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.688+0000 7f7cd9200700 1 -- 172.21.15.89:0/3144226102 learned_addr learned my addr 172.21.15.89:0/3144226102 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:45.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cd9a01700 1 --2- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409d6f0 0x7f7cd409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cda202700 1 --2- 172.21.15.89:0/3144226102 >> v2:172.21.15.89:3300/0 conn(0x7f7cd409e410 0x7f7cd4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cd89ff700 1 -- 172.21.15.89:0/3144226102 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126624025 0 0) 0x7f7cd409b120 con 0x7f7cd409e000 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cd89ff700 1 -- 172.21.15.89:0/3144226102 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cc00035b0 con 0x7f7cd409e000 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cda202700 1 -- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409d6f0 msgr2=0x7f7cd409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cda202700 1 --2- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409d6f0 0x7f7cd409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cda202700 1 -- 172.21.15.89:0/3144226102 >> v1:172.21.15.89:6789/0 conn(0x7f7cd409e000 legacy=0x7f7cd41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cda202700 1 -- 172.21.15.89:0/3144226102 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd409a500 con 0x7f7cd409e410 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cd9a01700 1 --2- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409d6f0 0x7f7cd409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cda202700 1 --2- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409e410 0x7f7cd4134100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009d90 tx=0x7f7ccc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb591e6aa277e0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:45.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cd89ff700 1 -- 172.21.15.89:0/3144226102 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00e070 con 0x7f7cd409e410 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cd89ff700 1 -- 172.21.15.89:0/3144226102 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc00bb20 con 0x7f7cd409e410 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.689+0000 7f7cd89ff700 1 -- 172.21.15.89:0/3144226102 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00be90 con 0x7f7cd409e410 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.690+0000 7f7cdbc65700 1 -- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409e410 msgr2=0x7f7cd4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.690+0000 7f7cdbc65700 1 --2- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409e410 0x7f7cd4134100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009d90 tx=0x7f7ccc00b330 comp rx=0 tx=0).stop 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.690+0000 7f7cdbc65700 1 -- 172.21.15.89:0/3144226102 shutdown_connections 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.690+0000 7f7cdbc65700 1 --2- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409e410 0x7f7cd4134100 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.690+0000 7f7cdbc65700 1 --2- 172.21.15.89:0/3144226102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409d6f0 0x7f7cd409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.690+0000 7f7cdbc65700 1 -- 172.21.15.89:0/3144226102 >> 172.21.15.89:0/3144226102 conn(0x7f7cd4066f50 msgr2=0x7f7cd409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:45.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.690+0000 7f7cdbc65700 1 -- 172.21.15.89:0/3144226102 shutdown_connections 2023-09-30T22:17:45.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.690+0000 7f7cdbc65700 1 -- 172.21.15.89:0/3144226102 wait complete. 2023-09-30T22:17:45.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.691+0000 7f7cdbc65700 1 Processor -- start 2023-09-30T22:17:45.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.691+0000 7f7cdbc65700 1 -- start start 2023-09-30T22:17:45.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.691+0000 7f7cdbc65700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409d6f0 0x7f7cd41c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:45.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.691+0000 7f7cdbc65700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cd409e000 0x7f7cd41c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:45.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.691+0000 7f7cdbc65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409e410 0x7f7cd41c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:45.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.691+0000 7f7cdbc65700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41a6310 con 0x7f7cd409d6f0 2023-09-30T22:17:45.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.691+0000 7f7cdbc65700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41b9090 con 0x7f7cd409e000 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.691+0000 7f7cdbc65700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40a2cb0 con 0x7f7cd409e410 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409e410 0x7f7cd41c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cd9200700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cd409e000 0x7f7cd41c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cd9a01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409d6f0 0x7f7cd41c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409e410 0x7f7cd41c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52480/0 (socket says 172.21.15.89:52480) 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cd9200700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cd409e000 0x7f7cd41c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40530/0 (socket says 172.21.15.89:40530) 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 -- 172.21.15.89:0/739878490 learned_addr learned my addr 172.21.15.89:0/739878490 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 -- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cd409e000 msgr2=0x7f7cd41c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cd409e000 0x7f7cd41c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 -- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409d6f0 msgr2=0x7f7cd41c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409d6f0 0x7f7cd41c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 -- 172.21.15.89:0/739878490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00094a0 con 0x7f7cd409e410 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cd9200700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cd409e000 0x7f7cd41c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cd9a01700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409d6f0 0x7f7cd41c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cda202700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409e410 0x7f7cd41c8a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00b680 tx=0x7f7ccc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cca7fc700 1 -- 172.21.15.89:0/739878490 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc007000 con 0x7f7cd409e410 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cca7fc700 1 -- 172.21.15.89:0/739878490 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc012650 con 0x7f7cd409e410 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc009a70 con 0x7f7cd409e410 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cca7fc700 1 -- 172.21.15.89:0/739878490 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc012a00 con 0x7f7cd409e410 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.692+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd41a8290 con 0x7f7cd409e410 2023-09-30T22:17:45.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.693+0000 7f7cca7fc700 1 -- 172.21.15.89:0/739878490 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ccc012b60 con 0x7f7cd409e410 2023-09-30T22:17:45.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.693+0000 7f7cca7fc700 1 -- 172.21.15.89:0/739878490 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7ccc041420 con 0x7f7cd409e410 2023-09-30T22:17:45.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.693+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd41a6fe0 con 0x7f7cd409e410 2023-09-30T22:17:45.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.698+0000 7f7cca7fc700 1 -- 172.21.15.89:0/739878490 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ccc003e10 con 0x7f7cd409e410 2023-09-30T22:17:45.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.879+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7cd4074ea0 con 0x7f7cd409e410 2023-09-30T22:17:45.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.879+0000 7f7cca7fc700 1 -- 172.21.15.89:0/739878490 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ccc02b030 con 0x7f7cd409e410 2023-09-30T22:17:45.880 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:45.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.881+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409e410 msgr2=0x7f7cd41c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:45.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.881+0000 7f7cdbc65700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409e410 0x7f7cd41c8a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00b680 tx=0x7f7ccc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.881+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 shutdown_connections 2023-09-30T22:17:45.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.881+0000 7f7cdbc65700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cd409e410 0x7f7cd41c8a30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.881+0000 7f7cdbc65700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd409d6f0 0x7f7cd41c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.881+0000 7f7cdbc65700 1 --2- 172.21.15.89:0/739878490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cd409e000 0x7f7cd41c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:45.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.881+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 >> 172.21.15.89:0/739878490 conn(0x7f7cd4066f50 msgr2=0x7f7cd41bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:45.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.882+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 shutdown_connections 2023-09-30T22:17:45.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:45.882+0000 7f7cdbc65700 1 -- 172.21.15.89:0/739878490 wait complete. 2023-09-30T22:17:45.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:45.919 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:46.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:46.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.040+0000 7f01b0cb7700 1 Processor -- start 2023-09-30T22:17:46.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.040+0000 7f01b0cb7700 1 -- start start 2023-09-30T22:17:46.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01b0cb7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01ac09e1d0 0x7f01ac135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01b0cb7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac135e50 0x7f01ac138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01b0cb7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac09f130 con 0x7f01ac09d6f0 2023-09-30T22:17:46.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01b0cb7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac09f520 con 0x7f01ac09e1d0 2023-09-30T22:17:46.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01b0cb7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac09e930 con 0x7f01ac135e50 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a9d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01ac09e1d0 0x7f01ac135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a9d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01ac09e1d0 0x7f01ac135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33662/0 (socket says 172.21.15.89:33662) 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a9d9b700 1 -- 172.21.15.89:0/2564114394 learned_addr learned my addr 172.21.15.89:0/2564114394 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01aad9d700 1 --2- 172.21.15.89:0/2564114394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac135e50 0x7f01ac138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a9d9b700 1 -- 172.21.15.89:0/2564114394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac135e50 msgr2=0x7f01ac138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a9d9b700 1 --2- 172.21.15.89:0/2564114394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac135e50 0x7f01ac138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a9d9b700 1 -- 172.21.15.89:0/2564114394 >> v1:172.21.15.89:6789/0 conn(0x7f01ac09d6f0 legacy=0x7f01ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a9d9b700 1 -- 172.21.15.89:0/2564114394 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac09b2d0 con 0x7f01ac09e1d0 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a9d9b700 1 --2- 172.21.15.89:0/2564114394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac09e1d0 0x7f01ac135910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f01a000bb90 tx=0x7f01a000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfd31f51bc2266a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a959a700 1 -- 172.21.15.89:0/2564114394 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000dd30 con 0x7f01ac09e1d0 2023-09-30T22:17:46.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a959a700 1 -- 172.21.15.89:0/2564114394 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a000de90 con 0x7f01ac09e1d0 2023-09-30T22:17:46.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.041+0000 7f01a959a700 1 -- 172.21.15.89:0/2564114394 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0010690 con 0x7f01ac09e1d0 2023-09-30T22:17:46.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.042+0000 7f01b0cb7700 1 -- 172.21.15.89:0/2564114394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac09e1d0 msgr2=0x7f01ac135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.042+0000 7f01b0cb7700 1 --2- 172.21.15.89:0/2564114394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac09e1d0 0x7f01ac135910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f01a000bb90 tx=0x7f01a000bea0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.042+0000 7f01b0cb7700 1 -- 172.21.15.89:0/2564114394 shutdown_connections 2023-09-30T22:17:46.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.042+0000 7f01b0cb7700 1 --2- 172.21.15.89:0/2564114394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac135e50 0x7f01ac138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.042+0000 7f01b0cb7700 1 --2- 172.21.15.89:0/2564114394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac09e1d0 0x7f01ac135910 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.042+0000 7f01b0cb7700 1 -- 172.21.15.89:0/2564114394 >> 172.21.15.89:0/2564114394 conn(0x7f01ac066f50 msgr2=0x7f01ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.042+0000 7f01b0cb7700 1 -- 172.21.15.89:0/2564114394 shutdown_connections 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.042+0000 7f01b0cb7700 1 -- 172.21.15.89:0/2564114394 wait complete. 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01b0cb7700 1 Processor -- start 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01b0cb7700 1 -- start start 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01b0cb7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01ac09d6f0 0x7f01ac1c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01b0cb7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac09e1d0 0x7f01ac1c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01b0cb7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac135e50 0x7f01ac1c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01b0cb7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac11bbc0 con 0x7f01ac135e50 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01b0cb7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac11f030 con 0x7f01ac09e1d0 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01b0cb7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac09c700 con 0x7f01ac09d6f0 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01ac09d6f0 0x7f01ac1c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01a9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac09e1d0 0x7f01ac1c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01ac09d6f0 0x7f01ac1c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52496/0 (socket says 172.21.15.89:52496) 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01aa59c700 1 -- 172.21.15.89:0/4058959763 learned_addr learned my addr 172.21.15.89:0/4058959763 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.044+0000 7f01aad9d700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac135e50 0x7f01ac1c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01a9d9b700 1 -- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01ac09d6f0 msgr2=0x7f01ac1c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01a9d9b700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01ac09d6f0 0x7f01ac1c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01a9d9b700 1 -- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac135e50 msgr2=0x7f01ac1c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01a9d9b700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac135e50 0x7f01ac1c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01a9d9b700 1 -- 172.21.15.89:0/4058959763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a000b7f0 con 0x7f01ac09e1d0 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01aa59c700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01ac09d6f0 0x7f01ac1c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01aad9d700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac135e50 0x7f01ac1c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01a9d9b700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac09e1d0 0x7f01ac1c46e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f01ac09f6f0 tx=0x7f01a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f019affd700 1 -- 172.21.15.89:0/4058959763 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a001f070 con 0x7f01ac09e1d0 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f019affd700 1 -- 172.21.15.89:0/4058959763 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0003970 con 0x7f01ac09e1d0 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac11d320 con 0x7f01ac09e1d0 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f019affd700 1 -- 172.21.15.89:0/4058959763 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0003d20 con 0x7f01ac09e1d0 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.045+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac11f9c0 con 0x7f01ac09e1d0 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.046+0000 7f019affd700 1 -- 172.21.15.89:0/4058959763 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f01a0023410 con 0x7f01ac09e1d0 2023-09-30T22:17:46.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.046+0000 7f019affd700 1 -- 172.21.15.89:0/4058959763 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f01a0042c80 con 0x7f01ac09e1d0 2023-09-30T22:17:46.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.046+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac074ea0 con 0x7f01ac09e1d0 2023-09-30T22:17:46.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.051+0000 7f019affd700 1 -- 172.21.15.89:0/4058959763 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f01a00236c0 con 0x7f01ac09e1d0 2023-09-30T22:17:46.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:46.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a55d55700 1 Processor -- start 2023-09-30T22:17:46.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a55d55700 1 -- start start 2023-09-30T22:17:46.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f687e1b1700 1 Processor -- start 2023-09-30T22:17:46.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f687e1b1700 1 -- start start 2023-09-30T22:17:46.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a55d55700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a5009e1d0 0x7f5a50135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a55d55700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a50135e50 0x7f5a50138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a55d55700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5009f130 con 0x7f5a5009d6f0 2023-09-30T22:17:46.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a55d55700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5009f520 con 0x7f5a50135e50 2023-09-30T22:17:46.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a55d55700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5009e930 con 0x7f5a5009e1d0 2023-09-30T22:17:46.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a4effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a5009e1d0 0x7f5a50135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a4effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a5009e1d0 0x7f5a50135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52506/0 (socket says 172.21.15.89:52506) 2023-09-30T22:17:46.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a4effd700 1 -- 172.21.15.89:0/2042577685 learned_addr learned my addr 172.21.15.89:0/2042577685 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:46.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.181+0000 7f5a4ffff700 1 --2- 172.21.15.89:0/2042577685 >> v2:172.21.15.89:3300/0 conn(0x7f5a50135e50 0x7f5a50138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f5a4ffff700 1 -- 172.21.15.89:0/2042577685 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a5009e1d0 msgr2=0x7f5a50135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:46.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f5a4ffff700 1 --2- 172.21.15.89:0/2042577685 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a5009e1d0 0x7f5a50135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f5a4ffff700 1 -- 172.21.15.89:0/2042577685 >> v1:172.21.15.89:6789/0 conn(0x7f5a5009d6f0 legacy=0x7f5a5009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f5a4ffff700 1 -- 172.21.15.89:0/2042577685 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5009b2d0 con 0x7f5a50135e50 2023-09-30T22:17:46.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f687e1b1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f6878136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f687e1b1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6878136f50 0x7f6878139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f687e1b1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687809b4f0 con 0x7f6878131810 2023-09-30T22:17:46.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f687e1b1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68781313c0 con 0x7f6878136f50 2023-09-30T22:17:46.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f687e1b1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f687806e8c0 con 0x7f6878134630 2023-09-30T22:17:46.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f5a4effd700 1 --2- 172.21.15.89:0/2042577685 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a5009e1d0 0x7f5a50135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:46.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6877fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6878136f50 0x7f6878139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6876ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f6878136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f68777fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6878131810 0x7f6878131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42748/0 (socket says 172.21.15.89:42748) 2023-09-30T22:17:46.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6877fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6878136f50 0x7f6878139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33700/0 (socket says 172.21.15.89:33700) 2023-09-30T22:17:46.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6876ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f6878136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52508/0 (socket says 172.21.15.89:52508) 2023-09-30T22:17:46.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f68777fe700 1 -- 172.21.15.89:0/423814966 learned_addr learned my addr 172.21.15.89:0/423814966 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:46.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f5a4ffff700 1 --2- 172.21.15.89:0/2042577685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a50135e50 0x7f5a50138230 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009df0 tx=0x7f5a4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda96a28444edd11 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:46.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6876ffd700 1 -- 172.21.15.89:0/423814966 >> v2:172.21.15.89:3300/0 conn(0x7f6878136f50 msgr2=0x7f6878139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:46.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6876ffd700 1 --2- 172.21.15.89:0/423814966 >> v2:172.21.15.89:3300/0 conn(0x7f6878136f50 0x7f6878139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6876ffd700 1 -- 172.21.15.89:0/423814966 >> v1:172.21.15.89:6789/0 conn(0x7f6878131810 legacy=0x7f6878131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6876ffd700 1 -- 172.21.15.89:0/423814966 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6878071d20 con 0x7f6878134630 2023-09-30T22:17:46.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.182+0000 7f6876ffd700 1 --2- 172.21.15.89:0/423814966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f6878136a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f686c009840 tx=0x7f686c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b294be08bc404df9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:46.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a4e7fc700 1 -- 172.21.15.89:0/2042577685 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800e070 con 0x7f5a50135e50 2023-09-30T22:17:46.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a4e7fc700 1 -- 172.21.15.89:0/2042577685 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4800baa0 con 0x7f5a50135e50 2023-09-30T22:17:46.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a4e7fc700 1 -- 172.21.15.89:0/2042577685 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800be50 con 0x7f5a50135e50 2023-09-30T22:17:46.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f68767fc700 1 -- 172.21.15.89:0/423814966 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c011830 con 0x7f6878134630 2023-09-30T22:17:46.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f68767fc700 1 -- 172.21.15.89:0/423814966 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c011990 con 0x7f6878134630 2023-09-30T22:17:46.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f68767fc700 1 -- 172.21.15.89:0/423814966 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c011d20 con 0x7f6878134630 2023-09-30T22:17:46.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a55d55700 1 -- 172.21.15.89:0/2042577685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a50135e50 msgr2=0x7f5a50138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a55d55700 1 --2- 172.21.15.89:0/2042577685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a50135e50 0x7f5a50138230 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009df0 tx=0x7f5a4800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:46.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a55d55700 1 -- 172.21.15.89:0/2042577685 shutdown_connections 2023-09-30T22:17:46.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a55d55700 1 --2- 172.21.15.89:0/2042577685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a50135e50 0x7f5a50138230 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a55d55700 1 --2- 172.21.15.89:0/2042577685 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a5009e1d0 0x7f5a50135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a55d55700 1 -- 172.21.15.89:0/2042577685 >> 172.21.15.89:0/2042577685 conn(0x7f5a50066f50 msgr2=0x7f5a5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:46.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f687e1b1700 1 -- 172.21.15.89:0/423814966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 msgr2=0x7f6878136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f687e1b1700 1 --2- 172.21.15.89:0/423814966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f6878136a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f686c009840 tx=0x7f686c009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:46.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a55d55700 1 -- 172.21.15.89:0/2042577685 shutdown_connections 2023-09-30T22:17:46.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f5a55d55700 1 -- 172.21.15.89:0/2042577685 wait complete. 2023-09-30T22:17:46.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f687e1b1700 1 -- 172.21.15.89:0/423814966 shutdown_connections 2023-09-30T22:17:46.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f687e1b1700 1 --2- 172.21.15.89:0/423814966 >> v2:172.21.15.89:3300/0 conn(0x7f6878136f50 0x7f6878139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f687e1b1700 1 --2- 172.21.15.89:0/423814966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f6878136a10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.183+0000 7f687e1b1700 1 -- 172.21.15.89:0/423814966 >> 172.21.15.89:0/423814966 conn(0x7f68780a2aa0 msgr2=0x7f68780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:46.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.184+0000 7f687e1b1700 1 -- 172.21.15.89:0/423814966 shutdown_connections 2023-09-30T22:17:46.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.184+0000 7f687e1b1700 1 -- 172.21.15.89:0/423814966 wait complete. 2023-09-30T22:17:46.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.184+0000 7f5a55d55700 1 Processor -- start 2023-09-30T22:17:46.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.184+0000 7f5a55d55700 1 -- start start 2023-09-30T22:17:46.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f687e1b1700 1 Processor -- start 2023-09-30T22:17:46.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a55d55700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a5009d6f0 0x7f5a501328b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a55d55700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a5009e1d0 0x7f5a50132df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a55d55700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a50135e50 0x7f5a50133330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a55d55700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50119970 con 0x7f5a5009d6f0 2023-09-30T22:17:46.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a55d55700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50135ad0 con 0x7f5a5009e1d0 2023-09-30T22:17:46.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a55d55700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5009c700 con 0x7f5a50135e50 2023-09-30T22:17:46.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a50135e50 0x7f5a50133330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a50135e50 0x7f5a50133330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52520/0 (socket says 172.21.15.89:52520) 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4ffff700 1 -- 172.21.15.89:0/3159273892 learned_addr learned my addr 172.21.15.89:0/3159273892 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4f7fe700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a5009d6f0 0x7f5a501328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4f7fe700 1 -- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a50135e50 msgr2=0x7f5a50133330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4f7fe700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a50135e50 0x7f5a50133330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4f7fe700 1 -- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a5009e1d0 msgr2=0x7f5a50132df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4f7fe700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a5009e1d0 0x7f5a50132df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4f7fe700 1 -- 172.21.15.89:0/3159273892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a44009550 con 0x7f5a5009d6f0 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4ffff700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a50135e50 0x7f5a50133330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:46.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f5a4f7fe700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a5009d6f0 0x7f5a501328b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5a40000f60 tx=0x7f5a4000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:46.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.185+0000 7f687e1b1700 1 -- start start 2023-09-30T22:17:46.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f5a37fff700 1 -- 172.21.15.89:0/3159273892 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400052d0 con 0x7f5a5009d6f0 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f687e1b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878131810 0x7f687809c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f687e1b1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f687809c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f687e1b1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6878136f50 0x7f687809ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f687e1b1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878138600 con 0x7f6878131810 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f687e1b1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687809d690 con 0x7f6878136f50 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f687e1b1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6878137430 con 0x7f6878134630 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f68777fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878131810 0x7f687809c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f68777fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878131810 0x7f687809c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33730/0 (socket says 172.21.15.89:33730) 2023-09-30T22:17:46.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f68777fe700 1 -- 172.21.15.89:0/3756620666 learned_addr learned my addr 172.21.15.89:0/3756620666 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f6876ffd700 1 --2- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f687809c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f68777fe700 1 -- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 msgr2=0x7f687809c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f68777fe700 1 --2- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f687809c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f68777fe700 1 -- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6878136f50 msgr2=0x7f687809ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f68777fe700 1 --2- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6878136f50 0x7f687809ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f68777fe700 1 -- 172.21.15.89:0/3756620666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0094a0 con 0x7f6878131810 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f5a37fff700 1 -- 172.21.15.89:0/3159273892 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4000ce90 con 0x7f5a5009d6f0 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f5a37fff700 1 -- 172.21.15.89:0/3159273892 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40003680 con 0x7f5a5009d6f0 2023-09-30T22:17:46.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a48009a70 con 0x7f5a5009d6f0 2023-09-30T22:17:46.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.186+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50134c20 con 0x7f5a5009d6f0 2023-09-30T22:17:46.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f68777fe700 1 --2- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878131810 0x7f687809c230 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f686800c4b0 tx=0x7f686800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:46.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f685ffff700 1 -- 172.21.15.89:0/3756620666 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868005d70 con 0x7f6878131810 2023-09-30T22:17:46.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f685ffff700 1 -- 172.21.15.89:0/3756620666 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68680070d0 con 0x7f6878131810 2023-09-30T22:17:46.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687806eb00 con 0x7f6878131810 2023-09-30T22:17:46.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f685ffff700 1 -- 172.21.15.89:0/3756620666 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868007440 con 0x7f6878131810 2023-09-30T22:17:46.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6878137690 con 0x7f6878131810 2023-09-30T22:17:46.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f5a37fff700 1 -- 172.21.15.89:0/3159273892 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a400037e0 con 0x7f5a5009d6f0 2023-09-30T22:17:46.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f5a37fff700 1 -- 172.21.15.89:0/3159273892 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5a4003e9a0 con 0x7f5a5009d6f0 2023-09-30T22:17:46.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.187+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a50071700 con 0x7f5a5009d6f0 2023-09-30T22:17:46.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.188+0000 7f685ffff700 1 -- 172.21.15.89:0/3756620666 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68680075a0 con 0x7f6878131810 2023-09-30T22:17:46.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.188+0000 7f685ffff700 1 -- 172.21.15.89:0/3756620666 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f686803e990 con 0x7f6878131810 2023-09-30T22:17:46.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.188+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68781309c0 con 0x7f6878131810 2023-09-30T22:17:46.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.196+0000 7f5a37fff700 1 -- 172.21.15.89:0/3159273892 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a40017030 con 0x7f5a5009d6f0 2023-09-30T22:17:46.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.204+0000 7f685ffff700 1 -- 172.21.15.89:0/3756620666 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6868019b40 con 0x7f6878131810 2023-09-30T22:17:46.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.300+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f01ac072340 con 0x7f01ac09e1d0 2023-09-30T22:17:46.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.301+0000 7f019affd700 1 -- 172.21.15.89:0/4058959763 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f01a002b3a0 con 0x7f01ac09e1d0 2023-09-30T22:17:46.302 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:46.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac09e1d0 msgr2=0x7f01ac1c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac09e1d0 0x7f01ac1c46e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f01ac09f6f0 tx=0x7f01a0004750 comp rx=0 tx=0).stop 2023-09-30T22:17:46.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 shutdown_connections 2023-09-30T22:17:46.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01ac135e50 0x7f01ac1c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01ac09d6f0 0x7f01ac1c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 --2- 172.21.15.89:0/4058959763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01ac09e1d0 0x7f01ac1c46e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 >> 172.21.15.89:0/4058959763 conn(0x7f01ac066f50 msgr2=0x7f01ac0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:46.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 shutdown_connections 2023-09-30T22:17:46.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.304+0000 7f01b0cb7700 1 -- 172.21.15.89:0/4058959763 wait complete. 2023-09-30T22:17:46.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:46.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.393+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5a50074ea0 con 0x7f5a5009d6f0 2023-09-30T22:17:46.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.393+0000 7f5a37fff700 1 -- 172.21.15.89:0/3159273892 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a40011590 con 0x7f5a5009d6f0 2023-09-30T22:17:46.394 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:46.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a5009d6f0 msgr2=0x7f5a501328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a5009d6f0 0x7f5a501328b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5a40000f60 tx=0x7f5a4000c6e0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 shutdown_connections 2023-09-30T22:17:46.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a50135e50 0x7f5a50133330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a5009d6f0 0x7f5a501328b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 --2- 172.21.15.89:0/3159273892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a5009e1d0 0x7f5a50132df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 >> 172.21.15.89:0/3159273892 conn(0x7f5a50066f50 msgr2=0x7f5a5006e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:46.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 shutdown_connections 2023-09-30T22:17:46.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.395+0000 7f5a55d55700 1 -- 172.21.15.89:0/3159273892 wait complete. 2023-09-30T22:17:46.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.408+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6878139190 con 0x7f6878131810 2023-09-30T22:17:46.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.408+0000 7f685ffff700 1 -- 172.21.15.89:0/3756620666 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6868017030 con 0x7f6878131810 2023-09-30T22:17:46.409 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878131810 msgr2=0x7f687809c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 --2- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878131810 0x7f687809c230 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f686800c4b0 tx=0x7f686800c7c0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 shutdown_connections 2023-09-30T22:17:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 --2- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6878136f50 0x7f687809ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 --2- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6878131810 0x7f687809c230 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 --2- 172.21.15.89:0/3756620666 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6878134630 0x7f687809c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 >> 172.21.15.89:0/3756620666 conn(0x7f68780a2aa0 msgr2=0x7f68780a18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 shutdown_connections 2023-09-30T22:17:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:46.410+0000 7f687e1b1700 1 -- 172.21.15.89:0/3756620666 wait complete. 2023-09-30T22:17:46.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:46.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:47.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f693c9700 1 Processor -- start 2023-09-30T22:17:47.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f693c9700 1 -- start start 2023-09-30T22:17:47.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f693c9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f640a09c0 0x7f3f640a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f693c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f640a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f693c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f64133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f693c9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640a1810 con 0x7f3f640a09c0 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f693c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6409ea10 con 0x7f3f64133920 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f693c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6409e210 con 0x7f3f6409d5c0 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f627fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f640a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f62ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f640a09c0 0x7f3f640a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f64133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f62ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f640a09c0 0x7f3f640a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33738/0 (socket says 172.21.15.89:33738) 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f62ffd700 1 -- 172.21.15.89:0/1513750343 learned_addr learned my addr 172.21.15.89:0/1513750343 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f64133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40568/0 (socket says 172.21.15.89:40568) 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f627fc700 1 -- 172.21.15.89:0/1513750343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 msgr2=0x7f3f64133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f627fc700 1 --2- 172.21.15.89:0/1513750343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f64133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f627fc700 1 -- 172.21.15.89:0/1513750343 >> v2:172.21.15.89:3300/0 conn(0x7f3f640a09c0 msgr2=0x7f3f640a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.019+0000 7f3f627fc700 1 --2- 172.21.15.89:0/1513750343 >> v2:172.21.15.89:3300/0 conn(0x7f3f640a09c0 0x7f3f640a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f627fc700 1 -- 172.21.15.89:0/1513750343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640a26f0 con 0x7f3f6409d5c0 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f62ffd700 1 --2- 172.21.15.89:0/1513750343 >> v2:172.21.15.89:3300/0 conn(0x7f3f640a09c0 0x7f3f640a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f637fe700 1 --2- 172.21.15.89:0/1513750343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f64133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f627fc700 1 --2- 172.21.15.89:0/1513750343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f640a12d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009840 tx=0x7f3f54009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f65775a5f7b9e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f61ffb700 1 -- 172.21.15.89:0/1513750343 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54011830 con 0x7f3f6409d5c0 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f61ffb700 1 -- 172.21.15.89:0/1513750343 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f54011990 con 0x7f3f6409d5c0 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f61ffb700 1 -- 172.21.15.89:0/1513750343 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54011d20 con 0x7f3f6409d5c0 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 -- 172.21.15.89:0/1513750343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 msgr2=0x7f3f640a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 --2- 172.21.15.89:0/1513750343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f640a12d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009840 tx=0x7f3f54009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:47.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 -- 172.21.15.89:0/1513750343 shutdown_connections 2023-09-30T22:17:47.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 --2- 172.21.15.89:0/1513750343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f64133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 --2- 172.21.15.89:0/1513750343 >> v2:172.21.15.89:3300/0 conn(0x7f3f640a09c0 0x7f3f640a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 --2- 172.21.15.89:0/1513750343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f640a12d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 -- 172.21.15.89:0/1513750343 >> 172.21.15.89:0/1513750343 conn(0x7f3f640a2fe0 msgr2=0x7f3f640a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:47.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 -- 172.21.15.89:0/1513750343 shutdown_connections 2023-09-30T22:17:47.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.020+0000 7f3f693c9700 1 -- 172.21.15.89:0/1513750343 wait complete. 2023-09-30T22:17:47.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.021+0000 7f3f693c9700 1 Processor -- start 2023-09-30T22:17:47.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f693c9700 1 -- start start 2023-09-30T22:17:47.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f693c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f641c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f693c9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f640a09c0 0x7f3f641c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f693c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f641c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f693c9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f641210d0 con 0x7f3f640a09c0 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f693c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f641207e0 con 0x7f3f64133920 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f693c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f641209d0 con 0x7f3f6409d5c0 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f641c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f627fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f640a09c0 0x7f3f641c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f641c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f641c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52544/0 (socket says 172.21.15.89:52544) 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f627fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f640a09c0 0x7f3f641c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33752/0 (socket says 172.21.15.89:33752) 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 -- 172.21.15.89:0/2435560893 learned_addr learned my addr 172.21.15.89:0/2435560893 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:47.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 -- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 msgr2=0x7f3f641c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 --2- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f641c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 -- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f640a09c0 msgr2=0x7f3f641c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 --2- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f640a09c0 0x7f3f641c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 -- 172.21.15.89:0/2435560893 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c009550 con 0x7f3f6409d5c0 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f637fe700 1 --2- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f641c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.022+0000 7f3f62ffd700 1 --2- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f641c8290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00ec10 tx=0x7f3f4c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.023+0000 7f3f5b7fe700 1 -- 172.21.15.89:0/2435560893 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00c850 con 0x7f3f6409d5c0 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.023+0000 7f3f5b7fe700 1 -- 172.21.15.89:0/2435560893 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c00c9b0 con 0x7f3f6409d5c0 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.023+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c009a70 con 0x7f3f6409d5c0 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.023+0000 7f3f5b7fe700 1 -- 172.21.15.89:0/2435560893 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00cd40 con 0x7f3f6409d5c0 2023-09-30T22:17:47.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.023+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f540094a0 con 0x7f3f6409d5c0 2023-09-30T22:17:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.023+0000 7f3f5b7fe700 1 -- 172.21.15.89:0/2435560893 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f4c013470 con 0x7f3f6409d5c0 2023-09-30T22:17:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.024+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6409cf20 con 0x7f3f6409d5c0 2023-09-30T22:17:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.024+0000 7f3f5b7fe700 1 -- 172.21.15.89:0/2435560893 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3f4c03f760 con 0x7f3f6409d5c0 2023-09-30T22:17:47.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.028+0000 7f3f5b7fe700 1 -- 172.21.15.89:0/2435560893 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f4c013720 con 0x7f3f6409d5c0 2023-09-30T22:17:47.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.211+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3f640747f0 con 0x7f3f6409d5c0 2023-09-30T22:17:47.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.211+0000 7f3f5b7fe700 1 -- 172.21.15.89:0/2435560893 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f4c01b3b0 con 0x7f3f6409d5c0 2023-09-30T22:17:47.212 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 msgr2=0x7f3f641c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 --2- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f641c8290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00ec10 tx=0x7f3f4c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 shutdown_connections 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 --2- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f64133920 0x7f3f641c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 --2- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f6409d5c0 0x7f3f641c8290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 --2- 172.21.15.89:0/2435560893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f640a09c0 0x7f3f641c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 >> 172.21.15.89:0/2435560893 conn(0x7f3f640a2fe0 msgr2=0x7f3f641bbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 shutdown_connections 2023-09-30T22:17:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.214+0000 7f3f693c9700 1 -- 172.21.15.89:0/2435560893 wait complete. 2023-09-30T22:17:47.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:47.322 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:47.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:47.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:47.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.445+0000 7f1e777c7700 1 Processor -- start 2023-09-30T22:17:47.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.445+0000 7f1e777c7700 1 -- start start 2023-09-30T22:17:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e777c7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e7009f410 0x7f1e70136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e777c7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e777c7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e70070940 con 0x7f1e7009e930 2023-09-30T22:17:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e777c7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7006fd60 con 0x7f1e7009f410 2023-09-30T22:17:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e777c7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7006fa80 con 0x7f1e70136fe0 2023-09-30T22:17:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e75d64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e74d62700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e7009f410 0x7f1e70136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e74d62700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e7009f410 0x7f1e70136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33756/0 (socket says 172.21.15.89:33756) 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e75d64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52546/0 (socket says 172.21.15.89:52546) 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e74d62700 1 -- 172.21.15.89:0/1593273589 learned_addr learned my addr 172.21.15.89:0/1593273589 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e74d62700 1 -- 172.21.15.89:0/1593273589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 msgr2=0x7f1e701393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e74d62700 1 --2- 172.21.15.89:0/1593273589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e74d62700 1 -- 172.21.15.89:0/1593273589 >> v1:172.21.15.89:6789/0 conn(0x7f1e7009e930 legacy=0x7f1e7009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e74d62700 1 -- 172.21.15.89:0/1593273589 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7009fb60 con 0x7f1e7009f410 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e75d64700 1 --2- 172.21.15.89:0/1593273589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:47.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e74d62700 1 --2- 172.21.15.89:0/1593273589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70136aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1e600098f0 tx=0x7f1e60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ebf79841e95424 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e67fff700 1 -- 172.21.15.89:0/1593273589 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60011830 con 0x7f1e7009f410 2023-09-30T22:17:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e67fff700 1 -- 172.21.15.89:0/1593273589 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e60011990 con 0x7f1e7009f410 2023-09-30T22:17:47.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.446+0000 7f1e67fff700 1 -- 172.21.15.89:0/1593273589 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60011d20 con 0x7f1e7009f410 2023-09-30T22:17:47.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.447+0000 7f1e777c7700 1 -- 172.21.15.89:0/1593273589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 msgr2=0x7f1e70136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.447+0000 7f1e777c7700 1 --2- 172.21.15.89:0/1593273589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70136aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1e600098f0 tx=0x7f1e60009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:47.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.447+0000 7f1e777c7700 1 -- 172.21.15.89:0/1593273589 shutdown_connections 2023-09-30T22:17:47.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.447+0000 7f1e777c7700 1 --2- 172.21.15.89:0/1593273589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.447+0000 7f1e777c7700 1 --2- 172.21.15.89:0/1593273589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70136aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.447+0000 7f1e777c7700 1 -- 172.21.15.89:0/1593273589 >> 172.21.15.89:0/1593273589 conn(0x7f1e700a4210 msgr2=0x7f1e700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.447+0000 7f1e777c7700 1 -- 172.21.15.89:0/1593273589 shutdown_connections 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.447+0000 7f1e777c7700 1 -- 172.21.15.89:0/1593273589 wait complete. 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.448+0000 7f1e777c7700 1 Processor -- start 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.448+0000 7f1e777c7700 1 -- start start 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e777c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e7009e930 0x7f1e701319f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e777c7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70131f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e777c7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e777c7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70120520 con 0x7f1e7009f410 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e777c7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7011e7a0 con 0x7f1e7009e930 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e777c7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7011f840 con 0x7f1e70136fe0 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70131f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70131f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33758/0 (socket says 172.21.15.89:33758) 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 -- 172.21.15.89:0/583996781 learned_addr learned my addr 172.21.15.89:0/583996781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e75563700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e7009e930 0x7f1e701319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 -- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 msgr2=0x7f1e701c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 -- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e7009e930 msgr2=0x7f1e701319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e7009e930 0x7f1e701319f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 -- 172.21.15.89:0/583996781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009a70 con 0x7f1e7009f410 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e75563700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e7009e930 0x7f1e701319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e74d62700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70131f30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1e60000c00 tx=0x7f1e60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e65ffb700 1 -- 172.21.15.89:0/583996781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600044f0 con 0x7f1e7009f410 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e65ffb700 1 -- 172.21.15.89:0/583996781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6001f070 con 0x7f1e7009f410 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e60009550 con 0x7f1e7009f410 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.449+0000 7f1e65ffb700 1 -- 172.21.15.89:0/583996781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600038c0 con 0x7f1e7009f410 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.450+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7011f410 con 0x7f1e7009f410 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.450+0000 7f1e65ffb700 1 -- 172.21.15.89:0/583996781 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e60021030 con 0x7f1e7009f410 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.451+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e701324d0 con 0x7f1e7009f410 2023-09-30T22:17:47.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.451+0000 7f1e65ffb700 1 -- 172.21.15.89:0/583996781 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e60040af0 con 0x7f1e7009f410 2023-09-30T22:17:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.456+0000 7f1e65ffb700 1 -- 172.21.15.89:0/583996781 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e60003a20 con 0x7f1e7009f410 2023-09-30T22:17:47.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.537+0000 7f546867a700 1 Processor -- start 2023-09-30T22:17:47.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.537+0000 7f546867a700 1 -- start start 2023-09-30T22:17:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f546867a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009e930 0x7f546009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f546867a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f546009f240 0x7f54601317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f546867a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5460070940 con 0x7f546009f650 2023-09-30T22:17:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f546867a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f546006fd60 con 0x7f546009f240 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f546867a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f546006fa80 con 0x7f546009e930 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465c15700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f546009f240 0x7f54601317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5466416700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009e930 0x7f546009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465c15700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f546009f240 0x7f54601317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33774/0 (socket says 172.21.15.89:33774) 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5466416700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009e930 0x7f546009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52562/0 (socket says 172.21.15.89:52562) 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465c15700 1 -- 172.21.15.89:0/876266450 learned_addr learned my addr 172.21.15.89:0/876266450 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5466c17700 1 --1- 172.21.15.89:0/876266450 >> v1:172.21.15.89:6789/0 conn(0x7f546009f650 0x7f5460131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42768/0 (socket says 172.21.15.89:42768) 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465c15700 1 -- 172.21.15.89:0/876266450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009e930 msgr2=0x7f546009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465c15700 1 --2- 172.21.15.89:0/876266450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009e930 0x7f546009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465c15700 1 -- 172.21.15.89:0/876266450 >> v1:172.21.15.89:6789/0 conn(0x7f546009f650 legacy=0x7f5460131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465c15700 1 -- 172.21.15.89:0/876266450 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546009fb60 con 0x7f546009f240 2023-09-30T22:17:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465414700 1 -- 172.21.15.89:0/876266450 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258041841 0 0) 0x7f5460070940 con 0x7f546009f650 2023-09-30T22:17:47.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5466416700 1 --2- 172.21.15.89:0/876266450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009e930 0x7f546009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:47.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.538+0000 7f5465c15700 1 --2- 172.21.15.89:0/876266450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009f240 0x7f54601317f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f54500098f0 tx=0x7f5450009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3de7f6359a9fa4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:47.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f5465414700 1 -- 172.21.15.89:0/876266450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450011830 con 0x7f546009f240 2023-09-30T22:17:47.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f5465414700 1 -- 172.21.15.89:0/876266450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450011990 con 0x7f546009f240 2023-09-30T22:17:47.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f5465414700 1 -- 172.21.15.89:0/876266450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450011d20 con 0x7f546009f240 2023-09-30T22:17:47.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f546867a700 1 -- 172.21.15.89:0/876266450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009f240 msgr2=0x7f54601317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f546867a700 1 --2- 172.21.15.89:0/876266450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009f240 0x7f54601317f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f54500098f0 tx=0x7f5450009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f546867a700 1 -- 172.21.15.89:0/876266450 shutdown_connections 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f546867a700 1 --2- 172.21.15.89:0/876266450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009e930 0x7f546009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f546867a700 1 --2- 172.21.15.89:0/876266450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009f240 0x7f54601317f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f546867a700 1 -- 172.21.15.89:0/876266450 >> 172.21.15.89:0/876266450 conn(0x7f54600a4210 msgr2=0x7f54600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f546867a700 1 -- 172.21.15.89:0/876266450 shutdown_connections 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.539+0000 7f546867a700 1 -- 172.21.15.89:0/876266450 wait complete. 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.540+0000 7f546867a700 1 Processor -- start 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.540+0000 7f546867a700 1 -- start start 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f546867a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009e930 0x7f54601c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f546867a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f546009f240 0x7f54601c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f546867a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009f650 0x7f54601c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f546867a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601a8290 con 0x7f546009e930 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f546867a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54601c9120 con 0x7f546009f240 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f546867a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54600a31d0 con 0x7f546009f650 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5466c17700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009f650 0x7f54601c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5465c15700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f546009f240 0x7f54601c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5466c17700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009f650 0x7f54601c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52578/0 (socket says 172.21.15.89:52578) 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5466c17700 1 -- 172.21.15.89:0/3529873432 learned_addr learned my addr 172.21.15.89:0/3529873432 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5465c15700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f546009f240 0x7f54601c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40590/0 (socket says 172.21.15.89:40590) 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5466416700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009e930 0x7f54601c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5465c15700 1 -- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009f650 msgr2=0x7f54601c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5465c15700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009f650 0x7f54601c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5465c15700 1 -- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009e930 msgr2=0x7f54601c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5465c15700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009e930 0x7f54601c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5465c15700 1 -- 172.21.15.89:0/3529873432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0094a0 con 0x7f546009f240 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5466c17700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009f650 0x7f54601c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5466416700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009e930 0x7f54601c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.541+0000 7f5465c15700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f546009f240 0x7f54601c8be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5450016000 tx=0x7f5450004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.542+0000 7f5456ffd700 1 -- 172.21.15.89:0/3529873432 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500045d0 con 0x7f546009f240 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.542+0000 7f5456ffd700 1 -- 172.21.15.89:0/3529873432 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545001f070 con 0x7f546009f240 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.542+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5450009550 con 0x7f546009f240 2023-09-30T22:17:47.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.542+0000 7f5456ffd700 1 -- 172.21.15.89:0/3529873432 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500038e0 con 0x7f546009f240 2023-09-30T22:17:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.542+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54600a0190 con 0x7f546009f240 2023-09-30T22:17:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.542+0000 7f5456ffd700 1 -- 172.21.15.89:0/3529873432 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5450003a40 con 0x7f546009f240 2023-09-30T22:17:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.543+0000 7f5456ffd700 1 -- 172.21.15.89:0/3529873432 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5450021070 con 0x7f546009f240 2023-09-30T22:17:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.543+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54600a18c0 con 0x7f546009f240 2023-09-30T22:17:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.547+0000 7f5456ffd700 1 -- 172.21.15.89:0/3529873432 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5450015070 con 0x7f546009f240 2023-09-30T22:17:47.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.580+0000 7faa0e3b2700 1 Processor -- start 2023-09-30T22:17:47.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.580+0000 7faa0e3b2700 1 -- start start 2023-09-30T22:17:47.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0e3b2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0e3b2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa08138d80 0x7faa0813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0e3b2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa080a2a70 con 0x7faa0809d630 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0e3b2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa080a01d0 con 0x7faa08138d80 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0e3b2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0809e2b0 con 0x7faa080a0a30 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0c94f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa08138d80 0x7faa0813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa07fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0c94f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa08138d80 0x7faa0813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33788/0 (socket says 172.21.15.89:33788) 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa07fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa080a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40594/0 (socket says 172.21.15.89:40594) 2023-09-30T22:17:47.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0c94f700 1 -- 172.21.15.89:0/551525746 learned_addr learned my addr 172.21.15.89:0/551525746 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa07fff700 1 -- 172.21.15.89:0/551525746 >> v2:172.21.15.89:3300/0 conn(0x7faa08138d80 msgr2=0x7faa0813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa07fff700 1 --2- 172.21.15.89:0/551525746 >> v2:172.21.15.89:3300/0 conn(0x7faa08138d80 0x7faa0813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa07fff700 1 -- 172.21.15.89:0/551525746 >> v1:172.21.15.89:6789/0 conn(0x7faa0809d630 legacy=0x7faa080a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa07fff700 1 -- 172.21.15.89:0/551525746 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0809dc50 con 0x7faa080a0a30 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa0c94f700 1 --2- 172.21.15.89:0/551525746 >> v2:172.21.15.89:3300/0 conn(0x7faa08138d80 0x7faa0813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.581+0000 7faa07fff700 1 --2- 172.21.15.89:0/551525746 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa080a0e00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be8f6f247b5d7e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa06ffd700 1 -- 172.21.15.89:0/551525746 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8011830 con 0x7faa080a0a30 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa06ffd700 1 -- 172.21.15.89:0/551525746 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8011990 con 0x7faa080a0a30 2023-09-30T22:17:47.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa06ffd700 1 -- 172.21.15.89:0/551525746 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8011d20 con 0x7faa080a0a30 2023-09-30T22:17:47.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa0e3b2700 1 -- 172.21.15.89:0/551525746 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 msgr2=0x7faa080a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa0e3b2700 1 --2- 172.21.15.89:0/551525746 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa080a0e00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:47.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa0e3b2700 1 -- 172.21.15.89:0/551525746 shutdown_connections 2023-09-30T22:17:47.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa0e3b2700 1 --2- 172.21.15.89:0/551525746 >> v2:172.21.15.89:3300/0 conn(0x7faa08138d80 0x7faa0813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa0e3b2700 1 --2- 172.21.15.89:0/551525746 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa080a0e00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa0e3b2700 1 -- 172.21.15.89:0/551525746 >> 172.21.15.89:0/551525746 conn(0x7faa080a3050 msgr2=0x7faa080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:47.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa0e3b2700 1 -- 172.21.15.89:0/551525746 shutdown_connections 2023-09-30T22:17:47.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.582+0000 7faa0e3b2700 1 -- 172.21.15.89:0/551525746 wait complete. 2023-09-30T22:17:47.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.583+0000 7faa0e3b2700 1 Processor -- start 2023-09-30T22:17:47.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa0e3b2700 1 -- start start 2023-09-30T22:17:47.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa0e3b2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa0809d630 0x7faa08120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa0e3b2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa08120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa0e3b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa08138d80 0x7faa081211e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:47.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa0e3b2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa08123670 con 0x7faa08138d80 2023-09-30T22:17:47.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa0e3b2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa08121bc0 con 0x7faa080a0a30 2023-09-30T22:17:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa0e3b2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa08124230 con 0x7faa0809d630 2023-09-30T22:17:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa07fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa0809d630 0x7faa08120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa07fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa0809d630 0x7faa08120760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52592/0 (socket says 172.21.15.89:52592) 2023-09-30T22:17:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa07fff700 1 -- 172.21.15.89:0/487821209 learned_addr learned my addr 172.21.15.89:0/487821209 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa077fe700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa08120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa0c94f700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa08138d80 0x7faa081211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa07fff700 1 -- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 msgr2=0x7faa08120ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa07fff700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa08120ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa07fff700 1 -- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa08138d80 msgr2=0x7faa081211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa07fff700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa08138d80 0x7faa081211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.584+0000 7faa07fff700 1 -- 172.21.15.89:0/487821209 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00009b30 con 0x7faa0809d630 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.585+0000 7faa07fff700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa0809d630 0x7faa08120760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80090b0 tx=0x7fa9f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.585+0000 7faa04ff9700 1 -- 172.21.15.89:0/487821209 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80045d0 con 0x7faa0809d630 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.585+0000 7faa04ff9700 1 -- 172.21.15.89:0/487821209 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f801f070 con 0x7faa0809d630 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.585+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f80094a0 con 0x7faa0809d630 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.585+0000 7faa04ff9700 1 -- 172.21.15.89:0/487821209 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80038e0 con 0x7faa0809d630 2023-09-30T22:17:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.585+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa080a0410 con 0x7faa0809d630 2023-09-30T22:17:47.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.586+0000 7faa04ff9700 1 -- 172.21.15.89:0/487821209 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9f8003a40 con 0x7faa0809d630 2023-09-30T22:17:47.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.586+0000 7faa04ff9700 1 -- 172.21.15.89:0/487821209 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa9f8021070 con 0x7faa0809d630 2023-09-30T22:17:47.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.586+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa080747f0 con 0x7faa0809d630 2023-09-30T22:17:47.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.595+0000 7faa04ff9700 1 -- 172.21.15.89:0/487821209 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9f802b080 con 0x7faa0809d630 2023-09-30T22:17:47.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.656+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1e7009e190 con 0x7f1e7009f410 2023-09-30T22:17:47.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.657+0000 7f1e65ffb700 1 -- 172.21.15.89:0/583996781 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e60015070 con 0x7f1e7009f410 2023-09-30T22:17:47.657 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:47.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 msgr2=0x7f1e70131f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70131f30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1e60000c00 tx=0x7f1e60004750 comp rx=0 tx=0).stop 2023-09-30T22:17:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 shutdown_connections 2023-09-30T22:17:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e70136fe0 0x7f1e701c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e7009e930 0x7f1e701319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 --2- 172.21.15.89:0/583996781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e7009f410 0x7f1e70131f30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 >> 172.21.15.89:0/583996781 conn(0x7f1e700a4210 msgr2=0x7f1e70134b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 shutdown_connections 2023-09-30T22:17:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.659+0000 7f1e777c7700 1 -- 172.21.15.89:0/583996781 wait complete. 2023-09-30T22:17:47.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:47.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.765+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5460074ea0 con 0x7f546009f240 2023-09-30T22:17:47.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.765+0000 7f5456ffd700 1 -- 172.21.15.89:0/3529873432 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f545000b3e0 con 0x7f546009f240 2023-09-30T22:17:47.766 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:47.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f546009f240 msgr2=0x7f54601c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f546009f240 0x7f54601c8be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5450016000 tx=0x7f5450004750 comp rx=0 tx=0).stop 2023-09-30T22:17:47.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 shutdown_connections 2023-09-30T22:17:47.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f546009f650 0x7f54601c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546009e930 0x7f54601c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 --2- 172.21.15.89:0/3529873432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f546009f240 0x7f54601c8be0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 >> 172.21.15.89:0/3529873432 conn(0x7f54600a4210 msgr2=0x7f54601b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:47.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 shutdown_connections 2023-09-30T22:17:47.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.767+0000 7f546867a700 1 -- 172.21.15.89:0/3529873432 wait complete. 2023-09-30T22:17:47.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:47.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.792+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faa08072980 con 0x7faa0809d630 2023-09-30T22:17:47.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.792+0000 7faa04ff9700 1 -- 172.21.15.89:0/487821209 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9f8015020 con 0x7faa0809d630 2023-09-30T22:17:47.793 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:47.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa0809d630 msgr2=0x7faa08120760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:47.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa0809d630 0x7faa08120760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80090b0 tx=0x7fa9f8004750 comp rx=0 tx=0).stop 2023-09-30T22:17:47.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 shutdown_connections 2023-09-30T22:17:47.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa08138d80 0x7faa081211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa0809d630 0x7faa08120760 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 --2- 172.21.15.89:0/487821209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa080a0a30 0x7faa08120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:47.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 >> 172.21.15.89:0/487821209 conn(0x7faa080a3050 msgr2=0x7faa08139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:47.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 shutdown_connections 2023-09-30T22:17:47.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:47.794+0000 7faa0e3b2700 1 -- 172.21.15.89:0/487821209 wait complete. 2023-09-30T22:17:47.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:48.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:48.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.354+0000 7f6c9f427700 1 Processor -- start 2023-09-30T22:17:48.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.355+0000 7f6c9f427700 1 -- start start 2023-09-30T22:17:48.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.355+0000 7f6c9f427700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c9812f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.355+0000 7f6c9f427700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c9812fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.355+0000 7f6c9f427700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c9812cff0 0x7f6c981303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.355+0000 7f6c9f427700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9809a510 con 0x7f6c9812cff0 2023-09-30T22:17:48.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.355+0000 7f6c9f427700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9812c7d0 con 0x7f6c9812cc20 2023-09-30T22:17:48.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.355+0000 7f6c9f427700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9809c190 con 0x7f6c9812c2f0 2023-09-30T22:17:48.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9c9c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c9812fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d9c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c9812cff0 0x7f6c981303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d1c3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c9812f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d9c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c9812cff0 0x7f6c981303b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33812/0 (socket says 172.21.15.89:33812) 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9c9c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c9812fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52604/0 (socket says 172.21.15.89:52604) 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d9c4700 1 -- 172.21.15.89:0/2812758845 learned_addr learned my addr 172.21.15.89:0/2812758845 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d1c3700 1 -- 172.21.15.89:0/2812758845 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 msgr2=0x7f6c9812fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d1c3700 1 --2- 172.21.15.89:0/2812758845 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c9812fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d1c3700 1 -- 172.21.15.89:0/2812758845 >> v2:172.21.15.89:3300/0 conn(0x7f6c9812cff0 msgr2=0x7f6c981303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d1c3700 1 --2- 172.21.15.89:0/2812758845 >> v2:172.21.15.89:3300/0 conn(0x7f6c9812cff0 0x7f6c981303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d1c3700 1 -- 172.21.15.89:0/2812758845 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9809b540 con 0x7f6c9812cc20 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9c9c2700 1 --2- 172.21.15.89:0/2812758845 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c9812fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d9c4700 1 --2- 172.21.15.89:0/2812758845 >> v2:172.21.15.89:3300/0 conn(0x7f6c9812cff0 0x7f6c981303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c9d1c3700 1 --2- 172.21.15.89:0/2812758845 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c9812f930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009840 tx=0x7f6c88009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa90169c2945ad4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:48.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c8ffff700 1 -- 172.21.15.89:0/2812758845 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800f8a0 con 0x7f6c9812cc20 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c8ffff700 1 -- 172.21.15.89:0/2812758845 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8800fa00 con 0x7f6c9812cc20 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.356+0000 7f6c8ffff700 1 -- 172.21.15.89:0/2812758845 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800fd90 con 0x7f6c9812cc20 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 -- 172.21.15.89:0/2812758845 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 msgr2=0x7f6c9812f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 --2- 172.21.15.89:0/2812758845 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c9812f930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009840 tx=0x7f6c88009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 -- 172.21.15.89:0/2812758845 shutdown_connections 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 --2- 172.21.15.89:0/2812758845 >> v2:172.21.15.89:3300/0 conn(0x7f6c9812cff0 0x7f6c981303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 --2- 172.21.15.89:0/2812758845 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c9812f930 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 --2- 172.21.15.89:0/2812758845 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c9812fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 -- 172.21.15.89:0/2812758845 >> 172.21.15.89:0/2812758845 conn(0x7f6c9809f470 msgr2=0x7f6c980a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 -- 172.21.15.89:0/2812758845 shutdown_connections 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.357+0000 7f6c9f427700 1 -- 172.21.15.89:0/2812758845 wait complete. 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9f427700 1 Processor -- start 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9f427700 1 -- start start 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9f427700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c981c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9f427700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c981c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9f427700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c9812cff0 0x7f6c981c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9f427700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98098ee0 con 0x7f6c9812cff0 2023-09-30T22:17:48.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9f427700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981c5d20 con 0x7f6c9812cc20 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9f427700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981c4e90 con 0x7f6c9812c2f0 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9d1c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c981c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9c9c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c981c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9d1c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c981c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52616/0 (socket says 172.21.15.89:52616) 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9d1c3700 1 -- 172.21.15.89:0/204345938 learned_addr learned my addr 172.21.15.89:0/204345938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.358+0000 7f6c9d9c4700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c9812cff0 0x7f6c981c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9c9c2700 1 -- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 msgr2=0x7f6c981c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9c9c2700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c981c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9c9c2700 1 -- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c9812cff0 msgr2=0x7f6c981c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9c9c2700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c9812cff0 0x7f6c981c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9c9c2700 1 -- 172.21.15.89:0/204345938 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9400a640 con 0x7f6c9812cc20 2023-09-30T22:17:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9d1c3700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c981c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9c9c2700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c981c57e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e8e0 tx=0x7f6c9400ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c8dffb700 1 -- 172.21.15.89:0/204345938 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94004440 con 0x7f6c9812cc20 2023-09-30T22:17:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c8dffb700 1 -- 172.21.15.89:0/204345938 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c940045a0 con 0x7f6c9812cc20 2023-09-30T22:17:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c90009b30 con 0x7f6c9812cc20 2023-09-30T22:17:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c8dffb700 1 -- 172.21.15.89:0/204345938 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c940038c0 con 0x7f6c9812cc20 2023-09-30T22:17:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.359+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c880094a0 con 0x7f6c9812cc20 2023-09-30T22:17:48.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.360+0000 7f6c8dffb700 1 -- 172.21.15.89:0/204345938 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c94003a20 con 0x7f6c9812cc20 2023-09-30T22:17:48.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.360+0000 7f6c8dffb700 1 -- 172.21.15.89:0/204345938 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6c9403f5d0 con 0x7f6c9812cc20 2023-09-30T22:17:48.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.360+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9808bd70 con 0x7f6c9812cc20 2023-09-30T22:17:48.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.365+0000 7f6c8dffb700 1 -- 172.21.15.89:0/204345938 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c9401f020 con 0x7f6c9812cc20 2023-09-30T22:17:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.546+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6c9808bac0 con 0x7f6c9812cc20 2023-09-30T22:17:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.547+0000 7f6c8dffb700 1 -- 172.21.15.89:0/204345938 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c9808bac0 con 0x7f6c9812cc20 2023-09-30T22:17:48.548 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 msgr2=0x7f6c981c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c981c57e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e8e0 tx=0x7f6c9400ebf0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 shutdown_connections 2023-09-30T22:17:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c9812cff0 0x7f6c981c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c9812c2f0 0x7f6c981c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 --2- 172.21.15.89:0/204345938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c9812cc20 0x7f6c981c57e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 >> 172.21.15.89:0/204345938 conn(0x7f6c9809f470 msgr2=0x7f6c9812f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 shutdown_connections 2023-09-30T22:17:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.549+0000 7f6c9f427700 1 -- 172.21.15.89:0/204345938 wait complete. 2023-09-30T22:17:48.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:48.676 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:48.781 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.799+0000 7f20ecfbb700 1 Processor -- start 2023-09-30T22:17:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.799+0000 7f20ecfbb700 1 -- start start 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20ecfbb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e81316c0 0x7f20e8131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20ecfbb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20e806ae80 0x7f20e8139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20ecfbb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e809e130 con 0x7f20e809d9f0 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20ecfbb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e81312b0 con 0x7f20e806ae80 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20ecfbb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20e809f6e0 con 0x7f20e81316c0 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20e6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20e806ae80 0x7f20e8139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20e659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e81316c0 0x7f20e8131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20e6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20e806ae80 0x7f20e8139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33822/0 (socket says 172.21.15.89:33822) 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20e659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e81316c0 0x7f20e8131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52632/0 (socket says 172.21.15.89:52632) 2023-09-30T22:17:48.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.800+0000 7f20e6d9d700 1 -- 172.21.15.89:0/1925931960 learned_addr learned my addr 172.21.15.89:0/1925931960 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:48.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e659c700 1 -- 172.21.15.89:0/1925931960 >> v2:172.21.15.89:3300/0 conn(0x7f20e806ae80 msgr2=0x7f20e8139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:48.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e659c700 1 --2- 172.21.15.89:0/1925931960 >> v2:172.21.15.89:3300/0 conn(0x7f20e806ae80 0x7f20e8139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e659c700 1 -- 172.21.15.89:0/1925931960 >> v1:172.21.15.89:6789/0 conn(0x7f20e809d9f0 legacy=0x7f20e806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e659c700 1 -- 172.21.15.89:0/1925931960 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e809e760 con 0x7f20e81316c0 2023-09-30T22:17:48.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e6d9d700 1 --2- 172.21.15.89:0/1925931960 >> v2:172.21.15.89:3300/0 conn(0x7f20e806ae80 0x7f20e8139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:48.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e659c700 1 --2- 172.21.15.89:0/1925931960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e81316c0 0x7f20e8131a90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009840 tx=0x7f20d8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c98735b871af2ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:48.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e559a700 1 -- 172.21.15.89:0/1925931960 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8011830 con 0x7f20e81316c0 2023-09-30T22:17:48.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e559a700 1 -- 172.21.15.89:0/1925931960 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8011990 con 0x7f20e81316c0 2023-09-30T22:17:48.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.801+0000 7f20e559a700 1 -- 172.21.15.89:0/1925931960 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8011d20 con 0x7f20e81316c0 2023-09-30T22:17:48.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.802+0000 7f20ecfbb700 1 -- 172.21.15.89:0/1925931960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e81316c0 msgr2=0x7f20e8131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.802+0000 7f20ecfbb700 1 --2- 172.21.15.89:0/1925931960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e81316c0 0x7f20e8131a90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009840 tx=0x7f20d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.802+0000 7f20ecfbb700 1 -- 172.21.15.89:0/1925931960 shutdown_connections 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.802+0000 7f20ecfbb700 1 --2- 172.21.15.89:0/1925931960 >> v2:172.21.15.89:3300/0 conn(0x7f20e806ae80 0x7f20e8139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.802+0000 7f20ecfbb700 1 --2- 172.21.15.89:0/1925931960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e81316c0 0x7f20e8131a90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.802+0000 7f20ecfbb700 1 -- 172.21.15.89:0/1925931960 >> 172.21.15.89:0/1925931960 conn(0x7f20e8066f50 msgr2=0x7f20e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.802+0000 7f20ecfbb700 1 -- 172.21.15.89:0/1925931960 shutdown_connections 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.802+0000 7f20ecfbb700 1 -- 172.21.15.89:0/1925931960 wait complete. 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.803+0000 7f20ecfbb700 1 Processor -- start 2023-09-30T22:17:48.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.803+0000 7f20ecfbb700 1 -- start start 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20ecfbb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e806ae80 0x7f20e8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20ecfbb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e809d9f0 0x7f20e8071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20ecfbb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20e81316c0 0x7f20e8071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20ecfbb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e811d9a0 con 0x7f20e806ae80 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20ecfbb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e812b900 con 0x7f20e81316c0 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20ecfbb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20e809b840 con 0x7f20e809d9f0 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e806ae80 0x7f20e8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e5d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e809d9f0 0x7f20e8071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e806ae80 0x7f20e8070b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33828/0 (socket says 172.21.15.89:33828) 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e659c700 1 -- 172.21.15.89:0/3447696147 learned_addr learned my addr 172.21.15.89:0/3447696147 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:48.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e6d9d700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20e81316c0 0x7f20e8071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e5d9b700 1 -- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20e81316c0 msgr2=0x7f20e8071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e5d9b700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20e81316c0 0x7f20e8071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e5d9b700 1 -- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e806ae80 msgr2=0x7f20e8070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e5d9b700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e806ae80 0x7f20e8070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e5d9b700 1 -- 172.21.15.89:0/3447696147 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0009b30 con 0x7f20e809d9f0 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.804+0000 7f20e659c700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e806ae80 0x7f20e8070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.805+0000 7f20e6d9d700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20e81316c0 0x7f20e8071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.805+0000 7f20e5d9b700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e809d9f0 0x7f20e8071050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000f60 tx=0x7f20dc00c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.805+0000 7f20d6ffd700 1 -- 172.21.15.89:0/3447696147 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0050d0 con 0x7f20e809d9f0 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.805+0000 7f20d6ffd700 1 -- 172.21.15.89:0/3447696147 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc005230 con 0x7f20e809d9f0 2023-09-30T22:17:48.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.805+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d80094a0 con 0x7f20e809d9f0 2023-09-30T22:17:48.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.805+0000 7f20d6ffd700 1 -- 172.21.15.89:0/3447696147 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc003680 con 0x7f20e809d9f0 2023-09-30T22:17:48.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.805+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e809c2e0 con 0x7f20e809d9f0 2023-09-30T22:17:48.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.806+0000 7f20d6ffd700 1 -- 172.21.15.89:0/3447696147 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20dc0037e0 con 0x7f20e809d9f0 2023-09-30T22:17:48.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.806+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e8074ea0 con 0x7f20e809d9f0 2023-09-30T22:17:48.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.806+0000 7f20d6ffd700 1 -- 172.21.15.89:0/3447696147 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f20dc03e900 con 0x7f20e809d9f0 2023-09-30T22:17:48.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:48.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.811+0000 7f20d6ffd700 1 -- 172.21.15.89:0/3447696147 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20dc0196d0 con 0x7f20e809d9f0 2023-09-30T22:17:48.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.907+0000 7f7306518700 1 Processor -- start 2023-09-30T22:17:48.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.907+0000 7f7306518700 1 -- start start 2023-09-30T22:17:48.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f7306518700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f730009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f7306518700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f730009f240 0x7f73001317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f7306518700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7300070940 con 0x7f730009f650 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f7306518700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730006fd60 con 0x7f730009f240 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f7306518700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730006fa80 con 0x7f730009e930 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72fffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f730009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72fffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f730009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40646/0 (socket says 172.21.15.89:40646) 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72fffff700 1 -- 172.21.15.89:0/1120168616 learned_addr learned my addr 172.21.15.89:0/1120168616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72ff7fe700 1 --2- 172.21.15.89:0/1120168616 >> v2:172.21.15.89:3300/0 conn(0x7f730009f240 0x7f73001317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72fffff700 1 -- 172.21.15.89:0/1120168616 >> v2:172.21.15.89:3300/0 conn(0x7f730009f240 msgr2=0x7f73001317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72fffff700 1 --2- 172.21.15.89:0/1120168616 >> v2:172.21.15.89:3300/0 conn(0x7f730009f240 0x7f73001317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72fffff700 1 -- 172.21.15.89:0/1120168616 >> v1:172.21.15.89:6789/0 conn(0x7f730009f650 legacy=0x7f7300131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72fffff700 1 -- 172.21.15.89:0/1120168616 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730009fb60 con 0x7f730009e930 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72ff7fe700 1 --2- 172.21.15.89:0/1120168616 >> v2:172.21.15.89:3300/0 conn(0x7f730009f240 0x7f73001317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.908+0000 7f72fffff700 1 --2- 172.21.15.89:0/1120168616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f730009ed00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72f00098f0 tx=0x7f72f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b41b2b5bf729149 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f72feffd700 1 -- 172.21.15.89:0/1120168616 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0011830 con 0x7f730009e930 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f72feffd700 1 -- 172.21.15.89:0/1120168616 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f0011990 con 0x7f730009e930 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f72feffd700 1 -- 172.21.15.89:0/1120168616 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0011d20 con 0x7f730009e930 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f7306518700 1 -- 172.21.15.89:0/1120168616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 msgr2=0x7f730009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f7306518700 1 --2- 172.21.15.89:0/1120168616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f730009ed00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72f00098f0 tx=0x7f72f0009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f7306518700 1 -- 172.21.15.89:0/1120168616 shutdown_connections 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f7306518700 1 --2- 172.21.15.89:0/1120168616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f730009ed00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f7306518700 1 --2- 172.21.15.89:0/1120168616 >> v2:172.21.15.89:3300/0 conn(0x7f730009f240 0x7f73001317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f7306518700 1 -- 172.21.15.89:0/1120168616 >> 172.21.15.89:0/1120168616 conn(0x7f73000a4210 msgr2=0x7f73000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:48.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f7306518700 1 -- 172.21.15.89:0/1120168616 shutdown_connections 2023-09-30T22:17:48.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.909+0000 7f7306518700 1 -- 172.21.15.89:0/1120168616 wait complete. 2023-09-30T22:17:48.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.910+0000 7f7306518700 1 Processor -- start 2023-09-30T22:17:48.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.911+0000 7f7306518700 1 -- start start 2023-09-30T22:17:48.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.911+0000 7f7306518700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f73001a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.911+0000 7f7306518700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f730009f240 0x7f73001a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.911+0000 7f7306518700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730009f650 0x7f73001a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.911+0000 7f7306518700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73001a51b0 con 0x7f730009f240 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.911+0000 7f7306518700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73001a5910 con 0x7f730009e930 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.911+0000 7f7306518700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73001a6b60 con 0x7f730009f650 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f730009f240 0x7f73001a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f730009f240 0x7f73001a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33848/0 (socket says 172.21.15.89:33848) 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 -- 172.21.15.89:0/916910969 learned_addr learned my addr 172.21.15.89:0/916910969 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f7304ab5700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730009f650 0x7f73001a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72fffff700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f73001a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 -- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730009f650 msgr2=0x7f73001a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730009f650 0x7f73001a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 -- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 msgr2=0x7f73001a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f73001a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 -- 172.21.15.89:0/916910969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40094a0 con 0x7f730009f240 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72ff7fe700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f730009f240 0x7f73001a4240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009ea0 tx=0x7f72f400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72fcff9700 1 -- 172.21.15.89:0/916910969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4013860 con 0x7f730009f240 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72fcff9700 1 -- 172.21.15.89:0/916910969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f40139c0 con 0x7f730009f240 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f0009550 con 0x7f730009f240 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f72fcff9700 1 -- 172.21.15.89:0/916910969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4013d50 con 0x7f730009f240 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.912+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73000a0190 con 0x7f730009f240 2023-09-30T22:17:48.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.914+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73000a3520 con 0x7f730009f240 2023-09-30T22:17:48.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.914+0000 7f72fcff9700 1 -- 172.21.15.89:0/916910969 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72f400c640 con 0x7f730009f240 2023-09-30T22:17:48.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.914+0000 7f72fcff9700 1 -- 172.21.15.89:0/916910969 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f72f4043c70 con 0x7f730009f240 2023-09-30T22:17:48.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.919+0000 7f72fcff9700 1 -- 172.21.15.89:0/916910969 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72f400c8f0 con 0x7f730009f240 2023-09-30T22:17:48.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.937+0000 7f56a825e700 1 Processor -- start 2023-09-30T22:17:48.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.937+0000 7f56a825e700 1 -- start start 2023-09-30T22:17:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a825e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56a00a0a30 0x7f56a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a825e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a009d630 0x7f56a00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a825e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a0133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a825e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a00a2a70 con 0x7f56a00a0a30 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a825e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00a01d0 con 0x7f56a0133930 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a825e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56a009e2b0 con 0x7f56a009d630 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56a00a0a30 0x7f56a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56a00a0a30 0x7f56a00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33852/0 (socket says 172.21.15.89:33852) 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 -- 172.21.15.89:0/1979658239 learned_addr learned my addr 172.21.15.89:0/1979658239 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a57f9700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a009d630 0x7f56a00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a67fb700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a0133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 -- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a009d630 msgr2=0x7f56a00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a009d630 0x7f56a00a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 -- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 msgr2=0x7f56a0133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a0133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 -- 172.21.15.89:0/1979658239 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a009dc50 con 0x7f56a00a0a30 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a5ffa700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a00a0a30 0x7f56a00a0e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f569c0098f0 tx=0x7f569c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb2e9182f047a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.938+0000 7f56a4ff8700 1 -- 172.21.15.89:0/1979658239 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c011830 con 0x7f56a00a0a30 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a4ff8700 1 -- 172.21.15.89:0/1979658239 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c011990 con 0x7f56a00a0a30 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a4ff8700 1 -- 172.21.15.89:0/1979658239 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c011d20 con 0x7f56a00a0a30 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 -- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a00a0a30 msgr2=0x7f56a00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a00a0a30 0x7f56a00a0e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f569c0098f0 tx=0x7f569c009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 -- 172.21.15.89:0/1979658239 shutdown_connections 2023-09-30T22:17:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a0133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a00a0a30 0x7f56a00a0e00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 --2- 172.21.15.89:0/1979658239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a009d630 0x7f56a00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 -- 172.21.15.89:0/1979658239 >> 172.21.15.89:0/1979658239 conn(0x7f56a00a3050 msgr2=0x7f56a00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 -- 172.21.15.89:0/1979658239 shutdown_connections 2023-09-30T22:17:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.939+0000 7f56a825e700 1 -- 172.21.15.89:0/1979658239 wait complete. 2023-09-30T22:17:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.940+0000 7f56a825e700 1 Processor -- start 2023-09-30T22:17:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.940+0000 7f56a825e700 1 -- start start 2023-09-30T22:17:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a825e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a009d630 0x7f56a011f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a825e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a00a0a30 0x7f56a011f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a825e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a01c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a825e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a0121e70 con 0x7f56a009d630 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a825e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a011eea0 con 0x7f56a0133930 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a825e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56a01211e0 con 0x7f56a00a0a30 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a009d630 0x7f56a011f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a67fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a01c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a009d630 0x7f56a011f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33866/0 (socket says 172.21.15.89:33866) 2023-09-30T22:17:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 -- 172.21.15.89:0/44584513 learned_addr learned my addr 172.21.15.89:0/44584513 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a57f9700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a00a0a30 0x7f56a011f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 -- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a00a0a30 msgr2=0x7f56a011f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a00a0a30 0x7f56a011f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 -- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 msgr2=0x7f56a01c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a01c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 -- 172.21.15.89:0/44584513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c009550 con 0x7f56a009d630 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a67fb700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a01c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56a5ffa700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a009d630 0x7f56a011f2d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56967fc700 1 -- 172.21.15.89:0/44584513 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c004450 con 0x7f56a009d630 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.941+0000 7f56967fc700 1 -- 172.21.15.89:0/44584513 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c0045b0 con 0x7f56a009d630 2023-09-30T22:17:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.942+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a0122130 con 0x7f56a009d630 2023-09-30T22:17:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.942+0000 7f56967fc700 1 -- 172.21.15.89:0/44584513 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c0038e0 con 0x7f56a009d630 2023-09-30T22:17:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.942+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a0122410 con 0x7f56a009d630 2023-09-30T22:17:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.942+0000 7f56967fc700 1 -- 172.21.15.89:0/44584513 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f569c021070 con 0x7f56a009d630 2023-09-30T22:17:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.942+0000 7f56967fc700 1 -- 172.21.15.89:0/44584513 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f569c040bc0 con 0x7f56a009d630 2023-09-30T22:17:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.943+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a00747f0 con 0x7f56a009d630 2023-09-30T22:17:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:48.947+0000 7f56967fc700 1 -- 172.21.15.89:0/44584513 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f569c029020 con 0x7f56a009d630 2023-09-30T22:17:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.012+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20e809e130 con 0x7f20e809d9f0 2023-09-30T22:17:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.013+0000 7f20d6ffd700 1 -- 172.21.15.89:0/3447696147 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20e809e130 con 0x7f20e809d9f0 2023-09-30T22:17:49.013 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e809d9f0 msgr2=0x7f20e8071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e809d9f0 0x7f20e8071050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000f60 tx=0x7f20dc00c730 comp rx=0 tx=0).stop 2023-09-30T22:17:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 shutdown_connections 2023-09-30T22:17:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20e81316c0 0x7f20e8071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e806ae80 0x7f20e8070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 --2- 172.21.15.89:0/3447696147 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20e809d9f0 0x7f20e8071050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 >> 172.21.15.89:0/3447696147 conn(0x7f20e8066f50 msgr2=0x7f20e80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 shutdown_connections 2023-09-30T22:17:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.015+0000 7f20ecfbb700 1 -- 172.21.15.89:0/3447696147 wait complete. 2023-09-30T22:17:49.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:49.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.114+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f73001a0b30 con 0x7f730009f240 2023-09-30T22:17:49.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.114+0000 7f72fcff9700 1 -- 172.21.15.89:0/916910969 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72f401a3a0 con 0x7f730009f240 2023-09-30T22:17:49.115 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:49.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f730009f240 msgr2=0x7f73001a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:49.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f730009f240 0x7f73001a4240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009ea0 tx=0x7f72f400eec0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 shutdown_connections 2023-09-30T22:17:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730009f650 0x7f73001a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730009e930 0x7f73001a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 --2- 172.21.15.89:0/916910969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f730009f240 0x7f73001a4240 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 >> 172.21.15.89:0/916910969 conn(0x7f73000a4210 msgr2=0x7f73001b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 shutdown_connections 2023-09-30T22:17:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.117+0000 7f7306518700 1 -- 172.21.15.89:0/916910969 wait complete. 2023-09-30T22:17:49.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:49.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.141+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f56a0072980 con 0x7f56a009d630 2023-09-30T22:17:49.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.142+0000 7f56967fc700 1 -- 172.21.15.89:0/44584513 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f569c0093b0 con 0x7f56a009d630 2023-09-30T22:17:49.143 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a009d630 msgr2=0x7f56a011f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a009d630 0x7f56a011f2d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 shutdown_connections 2023-09-30T22:17:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56a0133930 0x7f56a01c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56a009d630 0x7f56a011f2d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 --2- 172.21.15.89:0/44584513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56a00a0a30 0x7f56a011f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 >> 172.21.15.89:0/44584513 conn(0x7f56a00a3050 msgr2=0x7f56a00a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 shutdown_connections 2023-09-30T22:17:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.145+0000 7f56a825e700 1 -- 172.21.15.89:0/44584513 wait complete. 2023-09-30T22:17:49.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:49.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b2a5ec700 1 Processor -- start 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b2a5ec700 1 -- start start 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b2a5ec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b2409d6f0 0x7f4b2409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b2a5ec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409e440 0x7f4b24134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b2a5ec700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2409b120 con 0x7f4b2409e000 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b2a5ec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2406b460 con 0x7f4b2409d6f0 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b2a5ec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2409a7e0 con 0x7f4b2409e440 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b23fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b2409d6f0 0x7f4b2409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b28b89700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409e440 0x7f4b24134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b28b89700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409e440 0x7f4b24134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52666/0 (socket says 172.21.15.89:52666) 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b23fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b2409d6f0 0x7f4b2409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33868/0 (socket says 172.21.15.89:33868) 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b28b89700 1 -- 172.21.15.89:0/3084699794 learned_addr learned my addr 172.21.15.89:0/3084699794 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b23fff700 1 -- 172.21.15.89:0/3084699794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409e440 msgr2=0x7f4b24134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b23fff700 1 --2- 172.21.15.89:0/3084699794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409e440 0x7f4b24134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b23fff700 1 -- 172.21.15.89:0/3084699794 >> v1:172.21.15.89:6789/0 conn(0x7f4b2409e000 legacy=0x7f4b241339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b23fff700 1 -- 172.21.15.89:0/3084699794 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2409a500 con 0x7f4b2409d6f0 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b28b89700 1 --2- 172.21.15.89:0/3084699794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409e440 0x7f4b24134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b23fff700 1 --2- 172.21.15.89:0/3084699794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409d6f0 0x7f4b2409dac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009870 tx=0x7f4b18009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e813044650b2ce6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b22ffd700 1 -- 172.21.15.89:0/3084699794 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18011830 con 0x7f4b2409d6f0 2023-09-30T22:17:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b22ffd700 1 -- 172.21.15.89:0/3084699794 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18011990 con 0x7f4b2409d6f0 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.691+0000 7f4b22ffd700 1 -- 172.21.15.89:0/3084699794 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18011d20 con 0x7f4b2409d6f0 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.692+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3084699794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409d6f0 msgr2=0x7f4b2409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.692+0000 7f4b2a5ec700 1 --2- 172.21.15.89:0/3084699794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409d6f0 0x7f4b2409dac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009870 tx=0x7f4b18009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.692+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3084699794 shutdown_connections 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.692+0000 7f4b2a5ec700 1 --2- 172.21.15.89:0/3084699794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409d6f0 0x7f4b2409dac0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.692+0000 7f4b2a5ec700 1 --2- 172.21.15.89:0/3084699794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409e440 0x7f4b24134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.692+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3084699794 >> 172.21.15.89:0/3084699794 conn(0x7f4b24066f50 msgr2=0x7f4b2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.692+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3084699794 shutdown_connections 2023-09-30T22:17:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.692+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3084699794 wait complete. 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b2a5ec700 1 Processor -- start 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b2a5ec700 1 -- start start 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b2a5ec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409d6f0 0x7f4b241a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b2a5ec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2409e000 0x7f4b241a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b2a5ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409e440 0x7f4b241a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b2a5ec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241a7a60 con 0x7f4b2409e440 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b2a5ec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2409c710 con 0x7f4b2409e000 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b2a5ec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241a9690 con 0x7f4b2409d6f0 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b28b89700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409e440 0x7f4b241a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b237fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2409e000 0x7f4b241a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b28b89700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409e440 0x7f4b241a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33884/0 (socket says 172.21.15.89:33884) 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b28b89700 1 -- 172.21.15.89:0/3073232438 learned_addr learned my addr 172.21.15.89:0/3073232438 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.693+0000 7f4b23fff700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409d6f0 0x7f4b241a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b23fff700 1 -- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2409e000 msgr2=0x7f4b241a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b23fff700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2409e000 0x7f4b241a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b23fff700 1 -- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409e440 msgr2=0x7f4b241a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b23fff700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409e440 0x7f4b241a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b23fff700 1 -- 172.21.15.89:0/3073232438 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c009b30 con 0x7f4b2409d6f0 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b237fe700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2409e000 0x7f4b241a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b28b89700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409e440 0x7f4b241a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b23fff700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409d6f0 0x7f4b241a5e90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4b18016000 tx=0x7f4b18004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b20ff9700 1 -- 172.21.15.89:0/3073232438 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180045d0 con 0x7f4b2409d6f0 2023-09-30T22:17:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b20ff9700 1 -- 172.21.15.89:0/3073232438 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1801f070 con 0x7f4b2409d6f0 2023-09-30T22:17:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b20ff9700 1 -- 172.21.15.89:0/3073232438 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180038c0 con 0x7f4b2409d6f0 2023-09-30T22:17:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b180094a0 con 0x7f4b2409d6f0 2023-09-30T22:17:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.694+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b241a8cb0 con 0x7f4b2409d6f0 2023-09-30T22:17:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.695+0000 7f4b20ff9700 1 -- 172.21.15.89:0/3073232438 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b18003a20 con 0x7f4b2409d6f0 2023-09-30T22:17:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.695+0000 7f4b20ff9700 1 -- 172.21.15.89:0/3073232438 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4b18021070 con 0x7f4b2409d6f0 2023-09-30T22:17:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.695+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b24074ea0 con 0x7f4b2409d6f0 2023-09-30T22:17:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.700+0000 7f4b20ff9700 1 -- 172.21.15.89:0/3073232438 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b1802b020 con 0x7f4b2409d6f0 2023-09-30T22:17:49.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.883+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4b2409f420 con 0x7f4b2409d6f0 2023-09-30T22:17:49.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.884+0000 7f4b20ff9700 1 -- 172.21.15.89:0/3073232438 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b18015070 con 0x7f4b2409d6f0 2023-09-30T22:17:49.883 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.885+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409d6f0 msgr2=0x7f4b241a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.885+0000 7f4b2a5ec700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409d6f0 0x7f4b241a5e90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4b18016000 tx=0x7f4b18004750 comp rx=0 tx=0).stop 2023-09-30T22:17:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.886+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 shutdown_connections 2023-09-30T22:17:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.886+0000 7f4b2a5ec700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2409e440 0x7f4b241a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.886+0000 7f4b2a5ec700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2409d6f0 0x7f4b241a5e90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.886+0000 7f4b2a5ec700 1 --2- 172.21.15.89:0/3073232438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2409e000 0x7f4b241a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.886+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 >> 172.21.15.89:0/3073232438 conn(0x7f4b24066f50 msgr2=0x7f4b241bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.886+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 shutdown_connections 2023-09-30T22:17:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:49.886+0000 7f4b2a5ec700 1 -- 172.21.15.89:0/3073232438 wait complete. 2023-09-30T22:17:49.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:50.028 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:50.130 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37dc3fa700 1 Processor -- start 2023-09-30T22:17:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37dc3fa700 1 -- start start 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37dc3fa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37d409d6f0 0x7f37d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37dc3fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409e000 0x7f37d4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37dc3fa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d409f130 con 0x7f37d409e410 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37dc3fa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d409f520 con 0x7f37d409d6f0 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37dc3fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37d409e930 con 0x7f37d409e000 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37d9995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409e000 0x7f37d4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37da196700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37d409d6f0 0x7f37d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37d9995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409e000 0x7f37d4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52684/0 (socket says 172.21.15.89:52684) 2023-09-30T22:17:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37da196700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37d409d6f0 0x7f37d409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33888/0 (socket says 172.21.15.89:33888) 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37d9995700 1 -- 172.21.15.89:0/3742783643 learned_addr learned my addr 172.21.15.89:0/3742783643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.152+0000 7f37da997700 1 --1- 172.21.15.89:0/3742783643 >> v1:172.21.15.89:6789/0 conn(0x7f37d409e410 0x7f37d4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42810/0 (socket says 172.21.15.89:42810) 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.153+0000 7f37d9995700 1 -- 172.21.15.89:0/3742783643 >> v2:172.21.15.89:3300/0 conn(0x7f37d409d6f0 msgr2=0x7f37d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.153+0000 7f37d9995700 1 --2- 172.21.15.89:0/3742783643 >> v2:172.21.15.89:3300/0 conn(0x7f37d409d6f0 0x7f37d409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.153+0000 7f37d9995700 1 -- 172.21.15.89:0/3742783643 >> v1:172.21.15.89:6789/0 conn(0x7f37d409e410 legacy=0x7f37d4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.153+0000 7f37d9995700 1 -- 172.21.15.89:0/3742783643 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d409b2d0 con 0x7f37d409e000 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.153+0000 7f37d9995700 1 --2- 172.21.15.89:0/3742783643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409e000 0x7f37d4130660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f37c400bb90 tx=0x7f37c400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8899729c0501cc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.153+0000 7f37d9194700 1 -- 172.21.15.89:0/3742783643 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c400dd30 con 0x7f37d409e000 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.153+0000 7f37d9194700 1 -- 172.21.15.89:0/3742783643 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c400de90 con 0x7f37d409e000 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.153+0000 7f37d9194700 1 -- 172.21.15.89:0/3742783643 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4010690 con 0x7f37d409e000 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.154+0000 7f37dc3fa700 1 -- 172.21.15.89:0/3742783643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409e000 msgr2=0x7f37d4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.154+0000 7f37dc3fa700 1 --2- 172.21.15.89:0/3742783643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409e000 0x7f37d4130660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f37c400bb90 tx=0x7f37c400bea0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.154+0000 7f37dc3fa700 1 -- 172.21.15.89:0/3742783643 shutdown_connections 2023-09-30T22:17:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.154+0000 7f37dc3fa700 1 --2- 172.21.15.89:0/3742783643 >> v2:172.21.15.89:3300/0 conn(0x7f37d409d6f0 0x7f37d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.154+0000 7f37dc3fa700 1 --2- 172.21.15.89:0/3742783643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409e000 0x7f37d4130660 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.154+0000 7f37dc3fa700 1 -- 172.21.15.89:0/3742783643 >> 172.21.15.89:0/3742783643 conn(0x7f37d4066f50 msgr2=0x7f37d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.154+0000 7f37dc3fa700 1 -- 172.21.15.89:0/3742783643 shutdown_connections 2023-09-30T22:17:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.154+0000 7f37dc3fa700 1 -- 172.21.15.89:0/3742783643 wait complete. 2023-09-30T22:17:50.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37dc3fa700 1 Processor -- start 2023-09-30T22:17:50.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37dc3fa700 1 -- start start 2023-09-30T22:17:50.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37dc3fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409d6f0 0x7f37d411e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37dc3fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d409e000 0x7f37d411e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37dc3fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37d409e410 0x7f37d411eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37dc3fa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4121930 con 0x7f37d409e000 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37dc3fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d409c700 con 0x7f37d409e410 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37dc3fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4130390 con 0x7f37d409d6f0 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37d9995700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d409e000 0x7f37d411e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37d9995700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d409e000 0x7f37d411e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33898/0 (socket says 172.21.15.89:33898) 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37da997700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37d409e410 0x7f37d411eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37d9995700 1 -- 172.21.15.89:0/2547083041 learned_addr learned my addr 172.21.15.89:0/2547083041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:50.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.155+0000 7f37da196700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409d6f0 0x7f37d411e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37d9995700 1 -- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409d6f0 msgr2=0x7f37d411e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37d9995700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409d6f0 0x7f37d411e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37d9995700 1 -- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37d409e410 msgr2=0x7f37d411eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37d9995700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37d409e410 0x7f37d411eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37d9995700 1 -- 172.21.15.89:0/2547083041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c400b7f0 con 0x7f37d409e000 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37da196700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409d6f0 0x7f37d411e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37d9995700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d409e000 0x7f37d411e5d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37c400bee0 tx=0x7f37c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37caffd700 1 -- 172.21.15.89:0/2547083041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c401f070 con 0x7f37d409e000 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37caffd700 1 -- 172.21.15.89:0/2547083041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c4003970 con 0x7f37d409e000 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d4121b30 con 0x7f37d409e000 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37caffd700 1 -- 172.21.15.89:0/2547083041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4003d40 con 0x7f37d409e000 2023-09-30T22:17:50.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.156+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d41222c0 con 0x7f37d409e000 2023-09-30T22:17:50.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.157+0000 7f37caffd700 1 -- 172.21.15.89:0/2547083041 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f37c4019400 con 0x7f37d409e000 2023-09-30T22:17:50.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.157+0000 7f37caffd700 1 -- 172.21.15.89:0/2547083041 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f37c4043120 con 0x7f37d409e000 2023-09-30T22:17:50.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.157+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d4074ea0 con 0x7f37d409e000 2023-09-30T22:17:50.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:50.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.162+0000 7f37caffd700 1 -- 172.21.15.89:0/2547083041 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f37c4022960 con 0x7f37d409e000 2023-09-30T22:17:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.267+0000 7f8e3a2aa700 1 Processor -- start 2023-09-30T22:17:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.267+0000 7f8e3a2aa700 1 -- start start 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e3a2aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409d6f0 0x7f8e3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e3a2aa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e3409e000 0x7f8e341339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e3a2aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e410 0x7f8e34133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e3a2aa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3406b460 con 0x7f8e3409e000 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e3a2aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3409b120 con 0x7f8e3409d6f0 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e3a2aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3409a7e0 con 0x7f8e3409e410 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e33fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409d6f0 0x7f8e3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e38847700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e410 0x7f8e34133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e33fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409d6f0 0x7f8e3409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57198/0 (socket says 172.21.15.89:57198) 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e33fff700 1 -- 172.21.15.89:0/2541884529 learned_addr learned my addr 172.21.15.89:0/2541884529 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e337fe700 1 --2- 172.21.15.89:0/2541884529 >> v2:172.21.15.89:3300/0 conn(0x7f8e3409e000 0x7f8e341339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e38847700 1 -- 172.21.15.89:0/2541884529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409d6f0 msgr2=0x7f8e3409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e38847700 1 --2- 172.21.15.89:0/2541884529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409d6f0 0x7f8e3409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e38847700 1 -- 172.21.15.89:0/2541884529 >> v2:172.21.15.89:3300/0 conn(0x7f8e3409e000 msgr2=0x7f8e341339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e38847700 1 --2- 172.21.15.89:0/2541884529 >> v2:172.21.15.89:3300/0 conn(0x7f8e3409e000 0x7f8e341339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e38847700 1 -- 172.21.15.89:0/2541884529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3409a500 con 0x7f8e3409e410 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.268+0000 7f8e33fff700 1 --2- 172.21.15.89:0/2541884529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409d6f0 0x7f8e3409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e38847700 1 --2- 172.21.15.89:0/2541884529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e410 0x7f8e34133f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009dc0 tx=0x7f8e2c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78748b99f521ec51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e32ffd700 1 -- 172.21.15.89:0/2541884529 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00e070 con 0x7f8e3409e410 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e32ffd700 1 -- 172.21.15.89:0/2541884529 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c00bae0 con 0x7f8e3409e410 2023-09-30T22:17:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e32ffd700 1 -- 172.21.15.89:0/2541884529 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00be90 con 0x7f8e3409e410 2023-09-30T22:17:50.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/2541884529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e410 msgr2=0x7f8e34133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 --2- 172.21.15.89:0/2541884529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e410 0x7f8e34133f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009dc0 tx=0x7f8e2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:17:50.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/2541884529 shutdown_connections 2023-09-30T22:17:50.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 --2- 172.21.15.89:0/2541884529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e410 0x7f8e34133f30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 --2- 172.21.15.89:0/2541884529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409d6f0 0x7f8e3409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 --2- 172.21.15.89:0/2541884529 >> v2:172.21.15.89:3300/0 conn(0x7f8e3409e000 0x7f8e341339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/2541884529 >> 172.21.15.89:0/2541884529 conn(0x7f8e34066f50 msgr2=0x7f8e3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/2541884529 shutdown_connections 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.269+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/2541884529 wait complete. 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.270+0000 7f8e3a2aa700 1 Processor -- start 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e3a2aa700 1 -- start start 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e3a2aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3409d6f0 0x7f8e341c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e3a2aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e000 0x7f8e341c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e3a2aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409e410 0x7f8e341c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e3a2aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e341a6420 con 0x7f8e3409d6f0 2023-09-30T22:17:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e3a2aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e341a7c10 con 0x7f8e3409e410 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e3a2aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3409c710 con 0x7f8e3409e000 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e337fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e000 0x7f8e341c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e337fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e000 0x7f8e341c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39138/0 (socket says 172.21.15.89:39138) 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e337fe700 1 -- 172.21.15.89:0/394139375 learned_addr learned my addr 172.21.15.89:0/394139375 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.271+0000 7f8e33fff700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3409d6f0 0x7f8e341c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e38847700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409e410 0x7f8e341c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e337fe700 1 -- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409e410 msgr2=0x7f8e341c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e337fe700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409e410 0x7f8e341c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e337fe700 1 -- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3409d6f0 msgr2=0x7f8e341c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e337fe700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3409d6f0 0x7f8e341c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e337fe700 1 -- 172.21.15.89:0/394139375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e24009550 con 0x7f8e3409e000 2023-09-30T22:17:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e337fe700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e000 0x7f8e341c84f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800e980 tx=0x7f8e2800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e30ff9700 1 -- 172.21.15.89:0/394139375 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28009710 con 0x7f8e3409e000 2023-09-30T22:17:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e30ff9700 1 -- 172.21.15.89:0/394139375 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2800c690 con 0x7f8e3409e000 2023-09-30T22:17:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e30ff9700 1 -- 172.21.15.89:0/394139375 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800ca20 con 0x7f8e3409e000 2023-09-30T22:17:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c009a70 con 0x7f8e3409e000 2023-09-30T22:17:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.272+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e341a6f10 con 0x7f8e3409e000 2023-09-30T22:17:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.273+0000 7f8e30ff9700 1 -- 172.21.15.89:0/394139375 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e2800cb80 con 0x7f8e3409e000 2023-09-30T22:17:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.273+0000 7f8e30ff9700 1 -- 172.21.15.89:0/394139375 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e280421c0 con 0x7f8e3409e000 2023-09-30T22:17:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.273+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e34074ea0 con 0x7f8e3409e000 2023-09-30T22:17:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.278+0000 7f8e30ff9700 1 -- 172.21.15.89:0/394139375 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e28017020 con 0x7f8e3409e000 2023-09-30T22:17:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f2745e34700 1 Processor -- start 2023-09-30T22:17:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f2745e34700 1 -- start start 2023-09-30T22:17:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f2745e34700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f274009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f2745e34700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f274009e000 0x7f2740130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f2745e34700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274009f130 con 0x7f274009e410 2023-09-30T22:17:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f2745e34700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274009f520 con 0x7f274009e000 2023-09-30T22:17:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f2745e34700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f274009e930 con 0x7f274009d6f0 2023-09-30T22:17:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f273effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f274009e000 0x7f2740130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.288+0000 7f273f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f274009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f274009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39152/0 (socket says 172.21.15.89:39152) 2023-09-30T22:17:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f274009e000 0x7f2740130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32876/0 (socket says 172.21.15.89:32876) 2023-09-30T22:17:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273f7fe700 1 -- 172.21.15.89:0/3558822042 learned_addr learned my addr 172.21.15.89:0/3558822042 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273f7fe700 1 -- 172.21.15.89:0/3558822042 >> v2:172.21.15.89:3300/0 conn(0x7f274009e000 msgr2=0x7f2740130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273f7fe700 1 --2- 172.21.15.89:0/3558822042 >> v2:172.21.15.89:3300/0 conn(0x7f274009e000 0x7f2740130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273f7fe700 1 -- 172.21.15.89:0/3558822042 >> v1:172.21.15.89:6789/0 conn(0x7f274009e410 legacy=0x7f2740130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273f7fe700 1 -- 172.21.15.89:0/3558822042 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274009b2d0 con 0x7f274009d6f0 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273f7fe700 1 --2- 172.21.15.89:0/3558822042 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f274009dac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f273000bb90 tx=0x7f273000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ba1b3831bd3ed6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273e7fc700 1 -- 172.21.15.89:0/3558822042 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000dd30 con 0x7f274009d6f0 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273e7fc700 1 -- 172.21.15.89:0/3558822042 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273000de90 con 0x7f274009d6f0 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.289+0000 7f273e7fc700 1 -- 172.21.15.89:0/3558822042 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730010690 con 0x7f274009d6f0 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.290+0000 7f2745e34700 1 -- 172.21.15.89:0/3558822042 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 msgr2=0x7f274009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.290+0000 7f2745e34700 1 --2- 172.21.15.89:0/3558822042 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f274009dac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f273000bb90 tx=0x7f273000bea0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.290+0000 7f2745e34700 1 -- 172.21.15.89:0/3558822042 shutdown_connections 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.290+0000 7f2745e34700 1 --2- 172.21.15.89:0/3558822042 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f274009dac0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.290+0000 7f2745e34700 1 --2- 172.21.15.89:0/3558822042 >> v2:172.21.15.89:3300/0 conn(0x7f274009e000 0x7f2740130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.290+0000 7f2745e34700 1 -- 172.21.15.89:0/3558822042 >> 172.21.15.89:0/3558822042 conn(0x7f2740066f50 msgr2=0x7f274009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.290+0000 7f2745e34700 1 -- 172.21.15.89:0/3558822042 shutdown_connections 2023-09-30T22:17:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.290+0000 7f2745e34700 1 -- 172.21.15.89:0/3558822042 wait complete. 2023-09-30T22:17:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.291+0000 7f2745e34700 1 Processor -- start 2023-09-30T22:17:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.291+0000 7f2745e34700 1 -- start start 2023-09-30T22:17:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f2745e34700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f27401224f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f2745e34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f274009e000 0x7f2740122a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f2745e34700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f274009e410 0x7f2740122f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f2745e34700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2740125ce0 con 0x7f274009e000 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f2745e34700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27401241c0 con 0x7f274009e410 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f2745e34700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27400a1d00 con 0x7f274009d6f0 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f27401224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f27401224f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39166/0 (socket says 172.21.15.89:39166) 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273f7fe700 1 -- 172.21.15.89:0/2938911968 learned_addr learned my addr 172.21.15.89:0/2938911968 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273effd700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f274009e000 0x7f2740122a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273ffff700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f274009e410 0x7f2740122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273ffff700 1 -- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 msgr2=0x7f27401224f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273ffff700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f27401224f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273ffff700 1 -- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f274009e000 msgr2=0x7f2740122a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273ffff700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f274009e000 0x7f2740122a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273ffff700 1 -- 172.21.15.89:0/2938911968 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273000b7f0 con 0x7f274009e410 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.292+0000 7f273f7fe700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f27401224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.293+0000 7f273ffff700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f274009e410 0x7f2740122f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f274006dff0 tx=0x7f273800ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.293+0000 7f2727fff700 1 -- 172.21.15.89:0/2938911968 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738003c00 con 0x7f274009e410 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.293+0000 7f2727fff700 1 -- 172.21.15.89:0/2938911968 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738003d60 con 0x7f274009e410 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.293+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274009c700 con 0x7f274009e410 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.293+0000 7f2727fff700 1 -- 172.21.15.89:0/2938911968 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800a9c0 con 0x7f274009e410 2023-09-30T22:17:50.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.293+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2740126750 con 0x7f274009e410 2023-09-30T22:17:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.294+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2740074ea0 con 0x7f274009e410 2023-09-30T22:17:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.294+0000 7f2727fff700 1 -- 172.21.15.89:0/2938911968 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f273800ab20 con 0x7f274009e410 2023-09-30T22:17:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.294+0000 7f2727fff700 1 -- 172.21.15.89:0/2938911968 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f273803f600 con 0x7f274009e410 2023-09-30T22:17:50.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.303+0000 7f2727fff700 1 -- 172.21.15.89:0/2938911968 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f273800add0 con 0x7f274009e410 2023-09-30T22:17:50.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.375+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f37d406c360 con 0x7f37d409e000 2023-09-30T22:17:50.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.375+0000 7f37caffd700 1 -- 172.21.15.89:0/2547083041 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f37c4031070 con 0x7f37d409e000 2023-09-30T22:17:50.375 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d409e000 msgr2=0x7f37d411e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d409e000 0x7f37d411e5d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37c400bee0 tx=0x7f37c4004750 comp rx=0 tx=0).stop 2023-09-30T22:17:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 shutdown_connections 2023-09-30T22:17:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37d409e410 0x7f37d411eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37d409d6f0 0x7f37d411e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 --2- 172.21.15.89:0/2547083041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37d409e000 0x7f37d411e5d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 >> 172.21.15.89:0/2547083041 conn(0x7f37d4066f50 msgr2=0x7f37d406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 shutdown_connections 2023-09-30T22:17:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.377+0000 7f37dc3fa700 1 -- 172.21.15.89:0/2547083041 wait complete. 2023-09-30T22:17:50.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:50.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.478+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8e3406c360 con 0x7f8e3409e000 2023-09-30T22:17:50.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.478+0000 7f8e30ff9700 1 -- 172.21.15.89:0/394139375 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e28044060 con 0x7f8e3409e000 2023-09-30T22:17:50.478 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e000 msgr2=0x7f8e341c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e000 0x7f8e341c84f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800e980 tx=0x7f8e2800ec90 comp rx=0 tx=0).stop 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 shutdown_connections 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3409e410 0x7f8e341c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3409d6f0 0x7f8e341c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 --2- 172.21.15.89:0/394139375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3409e000 0x7f8e341c84f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 >> 172.21.15.89:0/394139375 conn(0x7f8e34066f50 msgr2=0x7f8e341bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 shutdown_connections 2023-09-30T22:17:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.481+0000 7f8e3a2aa700 1 -- 172.21.15.89:0/394139375 wait complete. 2023-09-30T22:17:50.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.499+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2740126020 con 0x7f274009e410 2023-09-30T22:17:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.499+0000 7f2727fff700 1 -- 172.21.15.89:0/2938911968 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f273801a360 con 0x7f274009e410 2023-09-30T22:17:50.499 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f274009e410 msgr2=0x7f2740122f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f274009e410 0x7f2740122f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f274006dff0 tx=0x7f273800ce70 comp rx=0 tx=0).stop 2023-09-30T22:17:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 shutdown_connections 2023-09-30T22:17:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f274009e410 0x7f2740122f70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f274009d6f0 0x7f27401224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 --2- 172.21.15.89:0/2938911968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f274009e000 0x7f2740122a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 >> 172.21.15.89:0/2938911968 conn(0x7f2740066f50 msgr2=0x7f274006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 shutdown_connections 2023-09-30T22:17:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:50.501+0000 7f2745e34700 1 -- 172.21.15.89:0/2938911968 wait complete. 2023-09-30T22:17:50.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:50.898 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.022+0000 7f88276a7700 1 Processor -- start 2023-09-30T22:17:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.022+0000 7f88276a7700 1 -- start start 2023-09-30T22:17:51.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f88276a7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f882009e1d0 0x7f8820138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f88276a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8820139170 0x7f882013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f88276a7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882009b120 con 0x7f882009d6f0 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f88276a7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882006b460 con 0x7f882009e1d0 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f88276a7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f882009a7e0 con 0x7f8820139170 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8824c42700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f882009e1d0 0x7f8820138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8825c44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8820139170 0x7f882013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8824c42700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f882009e1d0 0x7f8820138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32898/0 (socket says 172.21.15.89:32898) 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8825c44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8820139170 0x7f882013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39182/0 (socket says 172.21.15.89:39182) 2023-09-30T22:17:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8824c42700 1 -- 172.21.15.89:0/4082460943 learned_addr learned my addr 172.21.15.89:0/4082460943 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8825c44700 1 -- 172.21.15.89:0/4082460943 >> v2:172.21.15.89:3300/0 conn(0x7f882009e1d0 msgr2=0x7f8820138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8825c44700 1 --2- 172.21.15.89:0/4082460943 >> v2:172.21.15.89:3300/0 conn(0x7f882009e1d0 0x7f8820138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8825c44700 1 -- 172.21.15.89:0/4082460943 >> v1:172.21.15.89:6789/0 conn(0x7f882009d6f0 legacy=0x7f882009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8825c44700 1 -- 172.21.15.89:0/4082460943 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882009a500 con 0x7f8820139170 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8824c42700 1 --2- 172.21.15.89:0/4082460943 >> v2:172.21.15.89:3300/0 conn(0x7f882009e1d0 0x7f8820138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.023+0000 7f8825c44700 1 --2- 172.21.15.89:0/4082460943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8820139170 0x7f882013b550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8818009dc0 tx=0x7f881800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c9060d183498cc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f8817fff700 1 -- 172.21.15.89:0/4082460943 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800e070 con 0x7f8820139170 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f8817fff700 1 -- 172.21.15.89:0/4082460943 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881800bae0 con 0x7f8820139170 2023-09-30T22:17:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f8817fff700 1 -- 172.21.15.89:0/4082460943 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800be90 con 0x7f8820139170 2023-09-30T22:17:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f88276a7700 1 -- 172.21.15.89:0/4082460943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8820139170 msgr2=0x7f882013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f88276a7700 1 --2- 172.21.15.89:0/4082460943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8820139170 0x7f882013b550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8818009dc0 tx=0x7f881800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f88276a7700 1 -- 172.21.15.89:0/4082460943 shutdown_connections 2023-09-30T22:17:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f88276a7700 1 --2- 172.21.15.89:0/4082460943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8820139170 0x7f882013b550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f88276a7700 1 --2- 172.21.15.89:0/4082460943 >> v2:172.21.15.89:3300/0 conn(0x7f882009e1d0 0x7f8820138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f88276a7700 1 -- 172.21.15.89:0/4082460943 >> 172.21.15.89:0/4082460943 conn(0x7f8820066f50 msgr2=0x7f882009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f88276a7700 1 -- 172.21.15.89:0/4082460943 shutdown_connections 2023-09-30T22:17:51.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.024+0000 7f88276a7700 1 -- 172.21.15.89:0/4082460943 wait complete. 2023-09-30T22:17:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.025+0000 7f88276a7700 1 Processor -- start 2023-09-30T22:17:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.025+0000 7f88276a7700 1 -- start start 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f88276a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f882009d6f0 0x7f88201c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f88276a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f882009e1d0 0x7f88201c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f88276a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8820139170 0x7f882011e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f88276a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8820122110 con 0x7f882009e1d0 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f88276a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8820120400 con 0x7f8820139170 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f88276a7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8820121d70 con 0x7f882009d6f0 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8824c42700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f882009e1d0 0x7f88201c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8824c42700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f882009e1d0 0x7f88201c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32904/0 (socket says 172.21.15.89:32904) 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8824c42700 1 -- 172.21.15.89:0/2350354333 learned_addr learned my addr 172.21.15.89:0/2350354333 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8825c44700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8820139170 0x7f882011e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8825443700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f882009d6f0 0x7f88201c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8824c42700 1 -- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f882009d6f0 msgr2=0x7f88201c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8824c42700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f882009d6f0 0x7f88201c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8824c42700 1 -- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8820139170 msgr2=0x7f882011e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.026+0000 7f8824c42700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8820139170 0x7f882011e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.027+0000 7f8824c42700 1 -- 172.21.15.89:0/2350354333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c009550 con 0x7f882009e1d0 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.027+0000 7f8825c44700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8820139170 0x7f882011e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.027+0000 7f8824c42700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f882009e1d0 0x7f88201c8a10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f881c00ebe0 tx=0x7f881c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.027+0000 7f8815ffb700 1 -- 172.21.15.89:0/2350354333 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c00c850 con 0x7f882009e1d0 2023-09-30T22:17:51.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.027+0000 7f8815ffb700 1 -- 172.21.15.89:0/2350354333 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881c00c9b0 con 0x7f882009e1d0 2023-09-30T22:17:51.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.027+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8818009a70 con 0x7f882009e1d0 2023-09-30T22:17:51.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.027+0000 7f8815ffb700 1 -- 172.21.15.89:0/2350354333 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c00cd40 con 0x7f882009e1d0 2023-09-30T22:17:51.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.027+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8820122aa0 con 0x7f882009e1d0 2023-09-30T22:17:51.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.028+0000 7f8815ffb700 1 -- 172.21.15.89:0/2350354333 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f881c013470 con 0x7f882009e1d0 2023-09-30T22:17:51.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.028+0000 7f8815ffb700 1 -- 172.21.15.89:0/2350354333 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f881c03fc30 con 0x7f882009e1d0 2023-09-30T22:17:51.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.028+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8820074ea0 con 0x7f882009e1d0 2023-09-30T22:17:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.033+0000 7f8815ffb700 1 -- 172.21.15.89:0/2350354333 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f881c01b3b0 con 0x7f882009e1d0 2023-09-30T22:17:51.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.218+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f882009f420 con 0x7f882009e1d0 2023-09-30T22:17:51.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.219+0000 7f8815ffb700 1 -- 172.21.15.89:0/2350354333 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f881c011b00 con 0x7f882009e1d0 2023-09-30T22:17:51.219 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:51.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f882009e1d0 msgr2=0x7f88201c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f882009e1d0 0x7f88201c8a10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f881c00ebe0 tx=0x7f881c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 shutdown_connections 2023-09-30T22:17:51.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8820139170 0x7f882011e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f882009d6f0 0x7f88201c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 --2- 172.21.15.89:0/2350354333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f882009e1d0 0x7f88201c8a10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 >> 172.21.15.89:0/2350354333 conn(0x7f8820066f50 msgr2=0x7f88200a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:51.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 shutdown_connections 2023-09-30T22:17:51.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.221+0000 7f88276a7700 1 -- 172.21.15.89:0/2350354333 wait complete. 2023-09-30T22:17:51.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:51.391 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:51.494 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:51.513 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:51.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.514+0000 7fd5e2be9700 1 Processor -- start 2023-09-30T22:17:51.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.514+0000 7fd5e2be9700 1 -- start start 2023-09-30T22:17:51.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.514+0000 7fd5e2be9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5dc09d6f0 0x7fd5dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e2be9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e2be9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 0x7fd5dc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e2be9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5dc06b460 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e2be9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc09b120 con 0x7fd5dc09e410 2023-09-30T22:17:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e2be9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc09a7e0 con 0x7fd5dc09e000 2023-09-30T22:17:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5dc09d6f0 0x7fd5dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5dbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5dc09d6f0 0x7fd5dc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32920/0 (socket says 172.21.15.89:32920) 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 -- 172.21.15.89:0/991430465 learned_addr learned my addr 172.21.15.89:0/991430465 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e1186700 1 --2- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 0x7fd5dc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 -- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 msgr2=0x7fd5dc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 --2- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 -- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 msgr2=0x7fd5dc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 --2- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 0x7fd5dc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 -- 172.21.15.89:0/991430465 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc09a500 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5e0985700 1 --2- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc09dac0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098f0 tx=0x7fd5d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34e827ee17b28c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5db7fe700 1 -- 172.21.15.89:0/991430465 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0011830 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5db7fe700 1 -- 172.21.15.89:0/991430465 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d0011990 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.515+0000 7fd5db7fe700 1 -- 172.21.15.89:0/991430465 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0011d20 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 -- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 msgr2=0x7fd5dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 --2- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc09dac0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098f0 tx=0x7fd5d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 -- 172.21.15.89:0/991430465 shutdown_connections 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 --2- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 0x7fd5dc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 --2- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc09dac0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 --2- 172.21.15.89:0/991430465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 -- 172.21.15.89:0/991430465 >> 172.21.15.89:0/991430465 conn(0x7fd5dc066f50 msgr2=0x7fd5dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 -- 172.21.15.89:0/991430465 shutdown_connections 2023-09-30T22:17:51.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.516+0000 7fd5e2be9700 1 -- 172.21.15.89:0/991430465 wait complete. 2023-09-30T22:17:51.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.517+0000 7fd5e2be9700 1 Processor -- start 2023-09-30T22:17:51.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.517+0000 7fd5e2be9700 1 -- start start 2023-09-30T22:17:51.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e2be9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc1c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e2be9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e2be9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 0x7fd5dc1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e2be9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc1a95a0 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e2be9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc1a9780 con 0x7fd5dc09e410 2023-09-30T22:17:51.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e2be9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc1c5aa0 con 0x7fd5dc09e000 2023-09-30T22:17:51.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc1c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc1c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32922/0 (socket says 172.21.15.89:32922) 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 -- 172.21.15.89:0/825156408 learned_addr learned my addr 172.21.15.89:0/825156408 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5dbfff700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e1186700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 0x7fd5dc1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 -- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 msgr2=0x7fd5dc1c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 -- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 msgr2=0x7fd5dc1c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 0x7fd5dc1c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 -- 172.21.15.89:0/825156408 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d0009550 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5dbfff700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:51.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.518+0000 7fd5e0985700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc1c5020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0011db0 tx=0x7fd5d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.519+0000 7fd5d97fa700 1 -- 172.21.15.89:0/825156408 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0004240 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.519+0000 7fd5d97fa700 1 -- 172.21.15.89:0/825156408 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d001f070 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.519+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5dc0a2090 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.519+0000 7fd5d97fa700 1 -- 172.21.15.89:0/825156408 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00045a0 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.519+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc1a7870 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.520+0000 7fd5d97fa700 1 -- 172.21.15.89:0/825156408 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5d0021030 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.520+0000 7fd5d97fa700 1 -- 172.21.15.89:0/825156408 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd5d00409b0 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.520+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5dc074ea0 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.525+0000 7fd5d97fa700 1 -- 172.21.15.89:0/825156408 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5d000b9d0 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.622+0000 7f7b13b02700 1 Processor -- start 2023-09-30T22:17:51.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b13b02700 1 -- start start 2023-09-30T22:17:51.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b13b02700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09d6f0 0x7f7b0c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b13b02700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09e000 0x7f7b0c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b13b02700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c09b120 con 0x7f7b0c09e410 2023-09-30T22:17:51.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b13b02700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c06b460 con 0x7f7b0c09e000 2023-09-30T22:17:51.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b13b02700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c09a7e0 con 0x7f7b0c09d6f0 2023-09-30T22:17:51.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1109d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09e000 0x7f7b0c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1109d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09e000 0x7f7b0c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57254/0 (socket says 172.21.15.89:57254) 2023-09-30T22:17:51.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1109d700 1 -- 172.21.15.89:0/762224989 learned_addr learned my addr 172.21.15.89:0/762224989 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:51.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1209f700 1 --1- 172.21.15.89:0/762224989 >> v1:172.21.15.89:6789/0 conn(0x7f7b0c09e410 0x7f7b0c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36066/0 (socket says 172.21.15.89:36066) 2023-09-30T22:17:51.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1189e700 1 --2- 172.21.15.89:0/762224989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09d6f0 0x7f7b0c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1109d700 1 -- 172.21.15.89:0/762224989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09d6f0 msgr2=0x7f7b0c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1109d700 1 --2- 172.21.15.89:0/762224989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09d6f0 0x7f7b0c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1109d700 1 -- 172.21.15.89:0/762224989 >> v1:172.21.15.89:6789/0 conn(0x7f7b0c09e410 legacy=0x7f7b0c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.623+0000 7f7b1089c700 1 -- 172.21.15.89:0/762224989 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382183606 0 0) 0x7f7b0c09b120 con 0x7f7b0c09e410 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b1109d700 1 -- 172.21.15.89:0/762224989 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c09a500 con 0x7f7b0c09e000 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b1109d700 1 --2- 172.21.15.89:0/762224989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09e000 0x7f7b0c1339f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009920 tx=0x7f7afc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e58ddb97f3a031 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b1089c700 1 -- 172.21.15.89:0/762224989 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc011830 con 0x7f7b0c09e000 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b1089c700 1 -- 172.21.15.89:0/762224989 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc011990 con 0x7f7b0c09e000 2023-09-30T22:17:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b1089c700 1 -- 172.21.15.89:0/762224989 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc011d20 con 0x7f7b0c09e000 2023-09-30T22:17:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b13b02700 1 -- 172.21.15.89:0/762224989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09e000 msgr2=0x7f7b0c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b13b02700 1 --2- 172.21.15.89:0/762224989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09e000 0x7f7b0c1339f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009920 tx=0x7f7afc009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b13b02700 1 -- 172.21.15.89:0/762224989 shutdown_connections 2023-09-30T22:17:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b13b02700 1 --2- 172.21.15.89:0/762224989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09d6f0 0x7f7b0c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b13b02700 1 --2- 172.21.15.89:0/762224989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09e000 0x7f7b0c1339f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.624+0000 7f7b13b02700 1 -- 172.21.15.89:0/762224989 >> 172.21.15.89:0/762224989 conn(0x7f7b0c066f50 msgr2=0x7f7b0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.625+0000 7f7b13b02700 1 -- 172.21.15.89:0/762224989 shutdown_connections 2023-09-30T22:17:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.625+0000 7f7b13b02700 1 -- 172.21.15.89:0/762224989 wait complete. 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.625+0000 7f7b13b02700 1 Processor -- start 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b13b02700 1 -- start start 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b13b02700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09d6f0 0x7f7b0c1224c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b13b02700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09e000 0x7f7b0c122a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b13b02700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0c09e410 0x7f7b0c122f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b13b02700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c1337b0 con 0x7f7b0c09e410 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b13b02700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c124590 con 0x7f7b0c09d6f0 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b13b02700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0720b0 con 0x7f7b0c09e000 2023-09-30T22:17:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1109d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09e000 0x7f7b0c122a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1209f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0c09e410 0x7f7b0c122f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1109d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09e000 0x7f7b0c122a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39224/0 (socket says 172.21.15.89:39224) 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1109d700 1 -- 172.21.15.89:0/1029605995 learned_addr learned my addr 172.21.15.89:0/1029605995 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1189e700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09d6f0 0x7f7b0c1224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1109d700 1 -- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09d6f0 msgr2=0x7f7b0c1224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1109d700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09d6f0 0x7f7b0c1224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1109d700 1 -- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0c09e410 msgr2=0x7f7b0c122f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1109d700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0c09e410 0x7f7b0c122f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1109d700 1 -- 172.21.15.89:0/1029605995 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc009550 con 0x7f7b0c09e000 2023-09-30T22:17:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.626+0000 7f7b1209f700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0c09e410 0x7f7b0c122f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.627+0000 7f7b1109d700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09e000 0x7f7b0c122a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009c70 tx=0x7f7afc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.627+0000 7f7b027fc700 1 -- 172.21.15.89:0/1029605995 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc004240 con 0x7f7b0c09e000 2023-09-30T22:17:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.627+0000 7f7b027fc700 1 -- 172.21.15.89:0/1029605995 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc01f070 con 0x7f7b0c09e000 2023-09-30T22:17:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.627+0000 7f7b027fc700 1 -- 172.21.15.89:0/1029605995 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0045a0 con 0x7f7b0c09e000 2023-09-30T22:17:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.627+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c09c710 con 0x7f7b0c09e000 2023-09-30T22:17:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.627+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c126460 con 0x7f7b0c09e000 2023-09-30T22:17:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.627+0000 7f7b027fc700 1 -- 172.21.15.89:0/1029605995 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7afc021030 con 0x7f7b0c09e000 2023-09-30T22:17:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.628+0000 7f7b027fc700 1 -- 172.21.15.89:0/1029605995 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7afc0407c0 con 0x7f7b0c09e000 2023-09-30T22:17:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.628+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c074ea0 con 0x7f7b0c09e000 2023-09-30T22:17:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.632+0000 7f7b027fc700 1 -- 172.21.15.89:0/1029605995 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7afc01a450 con 0x7f7b0c09e000 2023-09-30T22:17:51.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.649+0000 7f92edb52700 1 Processor -- start 2023-09-30T22:17:51.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92edb52700 1 -- start start 2023-09-30T22:17:51.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92edb52700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92e809e930 0x7f92e809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92edb52700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f650 0x7f92e8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92edb52700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e8070940 con 0x7f92e809f240 2023-09-30T22:17:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92edb52700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e806fd60 con 0x7f92e809e930 2023-09-30T22:17:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92edb52700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e806fa80 con 0x7f92e809f650 2023-09-30T22:17:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92e7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f650 0x7f92e8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92e77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92e809e930 0x7f92e809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92e7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f650 0x7f92e8131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57286/0 (socket says 172.21.15.89:57286) 2023-09-30T22:17:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.650+0000 7f92e7fff700 1 -- 172.21.15.89:0/3968636579 learned_addr learned my addr 172.21.15.89:0/3968636579 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.651+0000 7f92e7fff700 1 -- 172.21.15.89:0/3968636579 >> v2:172.21.15.89:3300/0 conn(0x7f92e809e930 msgr2=0x7f92e809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:51.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.651+0000 7f92e7fff700 1 --2- 172.21.15.89:0/3968636579 >> v2:172.21.15.89:3300/0 conn(0x7f92e809e930 0x7f92e809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.651+0000 7f92e7fff700 1 -- 172.21.15.89:0/3968636579 >> v1:172.21.15.89:6789/0 conn(0x7f92e809f240 legacy=0x7f92e81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.651+0000 7f92e7fff700 1 -- 172.21.15.89:0/3968636579 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e809fb60 con 0x7f92e809f650 2023-09-30T22:17:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.651+0000 7f92e7fff700 1 --2- 172.21.15.89:0/3968636579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f650 0x7f92e8131f00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009e50 tx=0x7f92e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3799944fa98586a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.651+0000 7f92e67fc700 1 -- 172.21.15.89:0/3968636579 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e000e070 con 0x7f92e809f650 2023-09-30T22:17:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.651+0000 7f92e67fc700 1 -- 172.21.15.89:0/3968636579 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e000b910 con 0x7f92e809f650 2023-09-30T22:17:51.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.651+0000 7f92e67fc700 1 -- 172.21.15.89:0/3968636579 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e000bcc0 con 0x7f92e809f650 2023-09-30T22:17:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.652+0000 7f92edb52700 1 -- 172.21.15.89:0/3968636579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f650 msgr2=0x7f92e8131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.652+0000 7f92edb52700 1 --2- 172.21.15.89:0/3968636579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f650 0x7f92e8131f00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009e50 tx=0x7f92e000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.652+0000 7f92edb52700 1 -- 172.21.15.89:0/3968636579 shutdown_connections 2023-09-30T22:17:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.652+0000 7f92edb52700 1 --2- 172.21.15.89:0/3968636579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f650 0x7f92e8131f00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.652+0000 7f92edb52700 1 --2- 172.21.15.89:0/3968636579 >> v2:172.21.15.89:3300/0 conn(0x7f92e809e930 0x7f92e809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.652+0000 7f92edb52700 1 -- 172.21.15.89:0/3968636579 >> 172.21.15.89:0/3968636579 conn(0x7f92e80a4210 msgr2=0x7f92e80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.652+0000 7f92edb52700 1 -- 172.21.15.89:0/3968636579 shutdown_connections 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.652+0000 7f92edb52700 1 -- 172.21.15.89:0/3968636579 wait complete. 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.653+0000 7f92edb52700 1 Processor -- start 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.653+0000 7f92edb52700 1 -- start start 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92edb52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92e809e930 0x7f92e81a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92edb52700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f240 0x7f92e81a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92edb52700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e809f650 0x7f92e81a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92edb52700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e81a8040 con 0x7f92e809f650 2023-09-30T22:17:51.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92edb52700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e81a5770 con 0x7f92e809f240 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92edb52700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92e81a5d90 con 0x7f92e809e930 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f240 0x7f92e81a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e809f650 0x7f92e81a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f240 0x7f92e81a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57288/0 (socket says 172.21.15.89:57288) 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 -- 172.21.15.89:0/904472008 learned_addr learned my addr 172.21.15.89:0/904472008 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e77fe700 1 --2- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92e809e930 0x7f92e81a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 -- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92e809e930 msgr2=0x7f92e81a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 --2- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92e809e930 0x7f92e81a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 -- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e809f650 msgr2=0x7f92e81a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 --2- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e809f650 0x7f92e81a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 -- 172.21.15.89:0/904472008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0009b30 con 0x7f92e809f240 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92e6ffd700 1 --2- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f240 0x7f92e81a40a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c510 tx=0x7f92dc00c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92cffff700 1 -- 172.21.15.89:0/904472008 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0052d0 con 0x7f92e809f240 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92cffff700 1 -- 172.21.15.89:0/904472008 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc005430 con 0x7f92e809f240 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.654+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e80a0190 con 0x7f92e809f240 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.655+0000 7f92cffff700 1 -- 172.21.15.89:0/904472008 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0038a0 con 0x7f92e809f240 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.655+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e80a3650 con 0x7f92e809f240 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.655+0000 7f92cffff700 1 -- 172.21.15.89:0/904472008 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f92dc003a00 con 0x7f92e809f240 2023-09-30T22:17:51.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.655+0000 7f92cffff700 1 -- 172.21.15.89:0/904472008 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f92dc03e9c0 con 0x7f92e809f240 2023-09-30T22:17:51.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.656+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e8074ea0 con 0x7f92e809f240 2023-09-30T22:17:51.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.661+0000 7f92cffff700 1 -- 172.21.15.89:0/904472008 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92dc017080 con 0x7f92e809f240 2023-09-30T22:17:51.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.766+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd5dc1a9350 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.767+0000 7fd5d97fa700 1 -- 172.21.15.89:0/825156408 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5d0015030 con 0x7fd5dc09d6f0 2023-09-30T22:17:51.767 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 msgr2=0x7fd5dc1c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc1c5020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0011db0 tx=0x7fd5d0004750 comp rx=0 tx=0).stop 2023-09-30T22:17:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 shutdown_connections 2023-09-30T22:17:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5dc09e410 0x7fd5dc1c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5dc09d6f0 0x7fd5dc1c5020 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 --2- 172.21.15.89:0/825156408 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5dc09e000 0x7fd5dc1c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 >> 172.21.15.89:0/825156408 conn(0x7fd5dc066f50 msgr2=0x7fd5dc1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:51.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 shutdown_connections 2023-09-30T22:17:51.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.769+0000 7fd5e2be9700 1 -- 172.21.15.89:0/825156408 wait complete. 2023-09-30T22:17:51.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:51.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.833+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7b0c126740 con 0x7f7b0c09e000 2023-09-30T22:17:51.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.834+0000 7f7b027fc700 1 -- 172.21.15.89:0/1029605995 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7afc015020 con 0x7f7b0c09e000 2023-09-30T22:17:51.834 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09e000 msgr2=0x7f7b0c122a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09e000 0x7f7b0c122a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009c70 tx=0x7f7afc004750 comp rx=0 tx=0).stop 2023-09-30T22:17:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 shutdown_connections 2023-09-30T22:17:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0c09e410 0x7f7b0c122f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b0c09d6f0 0x7f7b0c1224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 --2- 172.21.15.89:0/1029605995 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7b0c09e000 0x7f7b0c122a00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 >> 172.21.15.89:0/1029605995 conn(0x7f7b0c066f50 msgr2=0x7f7b0c070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 shutdown_connections 2023-09-30T22:17:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.836+0000 7f7b13b02700 1 -- 172.21.15.89:0/1029605995 wait complete. 2023-09-30T22:17:51.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:51.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.890+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f92e806b250 con 0x7f92e809f240 2023-09-30T22:17:51.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.890+0000 7f92cffff700 1 -- 172.21.15.89:0/904472008 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92dc003cb0 con 0x7f92e809f240 2023-09-30T22:17:51.890 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f240 msgr2=0x7f92e81a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 --2- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f240 0x7f92e81a40a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c510 tx=0x7f92dc00c820 comp rx=0 tx=0).stop 2023-09-30T22:17:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 shutdown_connections 2023-09-30T22:17:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 --2- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92e809f650 0x7f92e81a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 --2- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92e809e930 0x7f92e81a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 --2- 172.21.15.89:0/904472008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92e809f240 0x7f92e81a40a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 >> 172.21.15.89:0/904472008 conn(0x7f92e80a4210 msgr2=0x7f92e81b8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 shutdown_connections 2023-09-30T22:17:51.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:51.892+0000 7f92edb52700 1 -- 172.21.15.89:0/904472008 wait complete. 2023-09-30T22:17:51.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:52.234 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f915bb32700 1 Processor -- start 2023-09-30T22:17:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f915bb32700 1 -- start start 2023-09-30T22:17:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f915bb32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f915409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f915bb32700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f915409e000 0x7f91541339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f915bb32700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915409b120 con 0x7f915409e410 2023-09-30T22:17:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f915bb32700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915406b460 con 0x7f915409e000 2023-09-30T22:17:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f915bb32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f915409a7e0 con 0x7f915409d6f0 2023-09-30T22:17:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f91598ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f915409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f91590cd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f915409e000 0x7f91541339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f91590cd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f915409e000 0x7f91541339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32950/0 (socket says 172.21.15.89:32950) 2023-09-30T22:17:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f91598ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f915409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39234/0 (socket says 172.21.15.89:39234) 2023-09-30T22:17:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.359+0000 7f91590cd700 1 -- 172.21.15.89:0/3845302683 learned_addr learned my addr 172.21.15.89:0/3845302683 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91590cd700 1 -- 172.21.15.89:0/3845302683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 msgr2=0x7f915409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91590cd700 1 --2- 172.21.15.89:0/3845302683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f915409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91590cd700 1 -- 172.21.15.89:0/3845302683 >> v1:172.21.15.89:6789/0 conn(0x7f915409e410 legacy=0x7f9154133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91590cd700 1 -- 172.21.15.89:0/3845302683 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915409a500 con 0x7f915409e000 2023-09-30T22:17:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91598ce700 1 --2- 172.21.15.89:0/3845302683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f915409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91590cd700 1 --2- 172.21.15.89:0/3845302683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541339f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f91440098f0 tx=0x7f9144009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f13e3f8896a6da3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91588cc700 1 -- 172.21.15.89:0/3845302683 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144011830 con 0x7f915409e000 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91588cc700 1 -- 172.21.15.89:0/3845302683 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9144011990 con 0x7f915409e000 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f91588cc700 1 -- 172.21.15.89:0/3845302683 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144011d20 con 0x7f915409e000 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f915bb32700 1 -- 172.21.15.89:0/3845302683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 msgr2=0x7f91541339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.360+0000 7f915bb32700 1 --2- 172.21.15.89:0/3845302683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541339f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f91440098f0 tx=0x7f9144009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.361+0000 7f915bb32700 1 -- 172.21.15.89:0/3845302683 shutdown_connections 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.361+0000 7f915bb32700 1 --2- 172.21.15.89:0/3845302683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f915409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.361+0000 7f915bb32700 1 --2- 172.21.15.89:0/3845302683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541339f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.361+0000 7f915bb32700 1 -- 172.21.15.89:0/3845302683 >> 172.21.15.89:0/3845302683 conn(0x7f9154066f50 msgr2=0x7f915409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.361+0000 7f915bb32700 1 -- 172.21.15.89:0/3845302683 shutdown_connections 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.361+0000 7f915bb32700 1 -- 172.21.15.89:0/3845302683 wait complete. 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.361+0000 7f915bb32700 1 Processor -- start 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f915bb32700 1 -- start start 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f915bb32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f91541a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f915bb32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f915bb32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915409e410 0x7f91541a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f915bb32700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91541a91f0 con 0x7f915409e000 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f915bb32700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915409c710 con 0x7f915409e410 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f915bb32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91540a3160 con 0x7f915409d6f0 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91598ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f91541a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32956/0 (socket says 172.21.15.89:32956) 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 -- 172.21.15.89:0/364186388 learned_addr learned my addr 172.21.15.89:0/364186388 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f915a0cf700 1 --2- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915409e410 0x7f91541a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 -- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 msgr2=0x7f91541a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 --2- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f91541a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 -- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915409e410 msgr2=0x7f91541a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 --2- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915409e410 0x7f91541a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 -- 172.21.15.89:0/364186388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500094a0 con 0x7f915409e000 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.362+0000 7f91590cd700 1 --2- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541a6510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9144009c40 tx=0x7f9144004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.363+0000 7f914a7fc700 1 -- 172.21.15.89:0/364186388 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144004300 con 0x7f915409e000 2023-09-30T22:17:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.363+0000 7f914a7fc700 1 -- 172.21.15.89:0/364186388 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914401f070 con 0x7f915409e000 2023-09-30T22:17:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.363+0000 7f914a7fc700 1 -- 172.21.15.89:0/364186388 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91440036a0 con 0x7f915409e000 2023-09-30T22:17:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.363+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9144009550 con 0x7f915409e000 2023-09-30T22:17:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.363+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91541a8da0 con 0x7f915409e000 2023-09-30T22:17:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.363+0000 7f914a7fc700 1 -- 172.21.15.89:0/364186388 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9144021030 con 0x7f915409e000 2023-09-30T22:17:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.363+0000 7f914a7fc700 1 -- 172.21.15.89:0/364186388 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f91440408f0 con 0x7f915409e000 2023-09-30T22:17:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.364+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91541a9600 con 0x7f915409e000 2023-09-30T22:17:52.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.368+0000 7f914a7fc700 1 -- 172.21.15.89:0/364186388 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9144015080 con 0x7f915409e000 2023-09-30T22:17:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.553+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9154074ea0 con 0x7f915409e000 2023-09-30T22:17:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.554+0000 7f914a7fc700 1 -- 172.21.15.89:0/364186388 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9144019770 con 0x7f915409e000 2023-09-30T22:17:52.553 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 msgr2=0x7f91541a6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 --2- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541a6510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9144009c40 tx=0x7f9144004750 comp rx=0 tx=0).stop 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 shutdown_connections 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 --2- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915409e410 0x7f91541a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 --2- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915409d6f0 0x7f91541a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 --2- 172.21.15.89:0/364186388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915409e000 0x7f91541a6510 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 >> 172.21.15.89:0/364186388 conn(0x7f9154066f50 msgr2=0x7f91541baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 shutdown_connections 2023-09-30T22:17:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.556+0000 7f915bb32700 1 -- 172.21.15.89:0/364186388 wait complete. 2023-09-30T22:17:52.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:52.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:52.849 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:52.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.914+0000 7f2ab648c700 1 Processor -- start 2023-09-30T22:17:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.914+0000 7f2ab648c700 1 -- start start 2023-09-30T22:17:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2ab648c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab009e1d0 0x7f2ab0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2ab648c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ab0135e50 0x7f2ab0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2ab648c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab009f130 con 0x7f2ab009d6f0 2023-09-30T22:17:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2ab648c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab009f520 con 0x7f2ab0135e50 2023-09-30T22:17:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2ab648c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab009e930 con 0x7f2ab009e1d0 2023-09-30T22:17:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab009e1d0 0x7f2ab0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab009e1d0 0x7f2ab0135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39240/0 (socket says 172.21.15.89:39240) 2023-09-30T22:17:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaf7fe700 1 -- 172.21.15.89:0/1257420118 learned_addr learned my addr 172.21.15.89:0/1257420118 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2ab4a29700 1 --2- 172.21.15.89:0/1257420118 >> v2:172.21.15.89:3300/0 conn(0x7f2ab0135e50 0x7f2ab0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaf7fe700 1 -- 172.21.15.89:0/1257420118 >> v2:172.21.15.89:3300/0 conn(0x7f2ab0135e50 msgr2=0x7f2ab0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaf7fe700 1 --2- 172.21.15.89:0/1257420118 >> v2:172.21.15.89:3300/0 conn(0x7f2ab0135e50 0x7f2ab0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaf7fe700 1 -- 172.21.15.89:0/1257420118 >> v1:172.21.15.89:6789/0 conn(0x7f2ab009d6f0 legacy=0x7f2ab009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaf7fe700 1 -- 172.21.15.89:0/1257420118 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab009b2d0 con 0x7f2ab009e1d0 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaf7fe700 1 --2- 172.21.15.89:0/1257420118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab009e1d0 0x7f2ab0135910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400b9d0 tx=0x7f2aa400bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33753074d7ac98e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.915+0000 7f2aaeffd700 1 -- 172.21.15.89:0/1257420118 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400cc20 con 0x7f2ab009e1d0 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2aaeffd700 1 -- 172.21.15.89:0/1257420118 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa400cd80 con 0x7f2ab009e1d0 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2aaeffd700 1 -- 172.21.15.89:0/1257420118 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400f660 con 0x7f2ab009e1d0 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2ab648c700 1 -- 172.21.15.89:0/1257420118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab009e1d0 msgr2=0x7f2ab0135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2ab648c700 1 --2- 172.21.15.89:0/1257420118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab009e1d0 0x7f2ab0135910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400b9d0 tx=0x7f2aa400bce0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2ab648c700 1 -- 172.21.15.89:0/1257420118 shutdown_connections 2023-09-30T22:17:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2ab648c700 1 --2- 172.21.15.89:0/1257420118 >> v2:172.21.15.89:3300/0 conn(0x7f2ab0135e50 0x7f2ab0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2ab648c700 1 --2- 172.21.15.89:0/1257420118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab009e1d0 0x7f2ab0135910 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2ab648c700 1 -- 172.21.15.89:0/1257420118 >> 172.21.15.89:0/1257420118 conn(0x7f2ab0066f50 msgr2=0x7f2ab009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2ab648c700 1 -- 172.21.15.89:0/1257420118 shutdown_connections 2023-09-30T22:17:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.916+0000 7f2ab648c700 1 -- 172.21.15.89:0/1257420118 wait complete. 2023-09-30T22:17:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.918+0000 7f2ab648c700 1 Processor -- start 2023-09-30T22:17:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.918+0000 7f2ab648c700 1 -- start start 2023-09-30T22:17:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.918+0000 7f2ab648c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab009d6f0 0x7f2ab011a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.918+0000 7f2ab648c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ab009e1d0 0x7f2ab011a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.918+0000 7f2ab648c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab0135e50 0x7f2ab01c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.918+0000 7f2ab648c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab009c700 con 0x7f2ab009d6f0 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.918+0000 7f2ab648c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab011ce30 con 0x7f2ab009e1d0 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.918+0000 7f2ab648c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0119c00 con 0x7f2ab0135e50 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ab009e1d0 0x7f2ab011a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ab009e1d0 0x7f2ab011a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57300/0 (socket says 172.21.15.89:57300) 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaf7fe700 1 -- 172.21.15.89:0/1851720962 learned_addr learned my addr 172.21.15.89:0/1851720962 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaffff700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab009d6f0 0x7f2ab011a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2ab4a29700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab0135e50 0x7f2ab01c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaffff700 1 -- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab0135e50 msgr2=0x7f2ab01c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaffff700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab0135e50 0x7f2ab01c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaffff700 1 -- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ab009e1d0 msgr2=0x7f2ab011a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaffff700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ab009e1d0 0x7f2ab011a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaffff700 1 -- 172.21.15.89:0/1851720962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa400b600 con 0x7f2ab009d6f0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2ab4a29700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab0135e50 0x7f2ab01c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaf7fe700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ab009e1d0 0x7f2ab011a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aaffff700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab009d6f0 0x7f2ab011a180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000c450 tx=0x7f2aa000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aacff9700 1 -- 172.21.15.89:0/1851720962 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00052d0 con 0x7f2ab009d6f0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aacff9700 1 -- 172.21.15.89:0/1851720962 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa0005430 con 0x7f2ab009d6f0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab00a1e20 con 0x7f2ab009d6f0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.919+0000 7f2aacff9700 1 -- 172.21.15.89:0/1851720962 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00038c0 con 0x7f2ab009d6f0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.920+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab011b220 con 0x7f2ab009d6f0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.920+0000 7f2aacff9700 1 -- 172.21.15.89:0/1851720962 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2aa0003a20 con 0x7f2ab009d6f0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.920+0000 7f2aacff9700 1 -- 172.21.15.89:0/1851720962 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2aa003e990 con 0x7f2ab009d6f0 2023-09-30T22:17:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.921+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab011cf70 con 0x7f2ab009d6f0 2023-09-30T22:17:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.926+0000 7f2aacff9700 1 -- 172.21.15.89:0/1851720962 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ab011cf70 con 0x7f2ab009d6f0 2023-09-30T22:17:52.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.979+0000 7f5114f6d700 1 Processor -- start 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.979+0000 7f5114f6d700 1 -- start start 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f5114f6d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133420 0x7f51101337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f5114f6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110133e30 0x7f511012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f5114f6d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5110134260 0x7f51101304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f5114f6d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511006b990 con 0x7f5110134260 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f5114f6d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51100a0590 con 0x7f5110133420 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f5114f6d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f511009fb80 con 0x7f5110133e30 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510dd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110133e30 0x7f511012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510e59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133420 0x7f51101337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510dd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110133e30 0x7f511012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39260/0 (socket says 172.21.15.89:39260) 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510e59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133420 0x7f51101337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57314/0 (socket says 172.21.15.89:57314) 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510dd9b700 1 -- 172.21.15.89:0/444103616 learned_addr learned my addr 172.21.15.89:0/444103616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510ed9d700 1 --2- 172.21.15.89:0/444103616 >> v2:172.21.15.89:3300/0 conn(0x7f5110134260 0x7f51101304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510e59c700 1 -- 172.21.15.89:0/444103616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110133e30 msgr2=0x7f511012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510e59c700 1 --2- 172.21.15.89:0/444103616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110133e30 0x7f511012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510e59c700 1 -- 172.21.15.89:0/444103616 >> v2:172.21.15.89:3300/0 conn(0x7f5110134260 msgr2=0x7f51101304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510e59c700 1 --2- 172.21.15.89:0/444103616 >> v2:172.21.15.89:3300/0 conn(0x7f5110134260 0x7f51101304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510e59c700 1 -- 172.21.15.89:0/444103616 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100a0030 con 0x7f5110133420 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510dd9b700 1 --2- 172.21.15.89:0/444103616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110133e30 0x7f511012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.980+0000 7f510e59c700 1 --2- 172.21.15.89:0/444103616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133420 0x7f51101337f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5100009840 tx=0x7f5100009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e22ece3f5527ca7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.981+0000 7f510d59a700 1 -- 172.21.15.89:0/444103616 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100011830 con 0x7f5110133420 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.981+0000 7f510d59a700 1 -- 172.21.15.89:0/444103616 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5100011990 con 0x7f5110133420 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.981+0000 7f510d59a700 1 -- 172.21.15.89:0/444103616 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100011d20 con 0x7f5110133420 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.981+0000 7f5114f6d700 1 -- 172.21.15.89:0/444103616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133420 msgr2=0x7f51101337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.981+0000 7f5114f6d700 1 --2- 172.21.15.89:0/444103616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133420 0x7f51101337f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5100009840 tx=0x7f5100009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.982+0000 7f5114f6d700 1 -- 172.21.15.89:0/444103616 shutdown_connections 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.982+0000 7f5114f6d700 1 --2- 172.21.15.89:0/444103616 >> v2:172.21.15.89:3300/0 conn(0x7f5110134260 0x7f51101304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.982+0000 7f5114f6d700 1 --2- 172.21.15.89:0/444103616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133420 0x7f51101337f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.982+0000 7f5114f6d700 1 --2- 172.21.15.89:0/444103616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110133e30 0x7f511012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.982+0000 7f5114f6d700 1 -- 172.21.15.89:0/444103616 >> 172.21.15.89:0/444103616 conn(0x7f51100a4210 msgr2=0x7f51100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.982+0000 7f5114f6d700 1 -- 172.21.15.89:0/444103616 shutdown_connections 2023-09-30T22:17:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.982+0000 7f5114f6d700 1 -- 172.21.15.89:0/444103616 wait complete. 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.983+0000 7f5114f6d700 1 Processor -- start 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.983+0000 7f5114f6d700 1 -- start start 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f5114f6d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5110133420 0x7f5110070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f5114f6d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133e30 0x7f5110071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f5114f6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110134260 0x7f5110071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f5114f6d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511011ea90 con 0x7f5110133420 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f5114f6d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5110120260 con 0x7f5110133e30 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f5114f6d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51100726b0 con 0x7f5110134260 2023-09-30T22:17:52.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f510e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5110133420 0x7f5110070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f510e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5110133420 0x7f5110070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32986/0 (socket says 172.21.15.89:32986) 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f510e59c700 1 -- 172.21.15.89:0/70416262 learned_addr learned my addr 172.21.15.89:0/70416262 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f510ed9d700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110134260 0x7f5110071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f510dd9b700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133e30 0x7f5110071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f510e59c700 1 -- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110134260 msgr2=0x7f5110071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f510e59c700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110134260 0x7f5110071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.984+0000 7f510e59c700 1 -- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133e30 msgr2=0x7f5110071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f510e59c700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133e30 0x7f5110071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f510e59c700 1 -- 172.21.15.89:0/70416262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5104009550 con 0x7f5110133420 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f510ed9d700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110134260 0x7f5110071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:52.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f510e59c700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5110133420 0x7f5110070c00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f51000090b0 tx=0x7f5100004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f50feffd700 1 -- 172.21.15.89:0/70416262 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100004240 con 0x7f5110133420 2023-09-30T22:17:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f50feffd700 1 -- 172.21.15.89:0/70416262 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510001f070 con 0x7f5110133420 2023-09-30T22:17:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f50feffd700 1 -- 172.21.15.89:0/70416262 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000036a0 con 0x7f5110133420 2023-09-30T22:17:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51000094a0 con 0x7f5110133420 2023-09-30T22:17:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.985+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511012d970 con 0x7f5110133420 2023-09-30T22:17:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.986+0000 7f50feffd700 1 -- 172.21.15.89:0/70416262 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5100021030 con 0x7f5110133420 2023-09-30T22:17:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.986+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51100a4940 con 0x7f5110133420 2023-09-30T22:17:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.986+0000 7f50feffd700 1 -- 172.21.15.89:0/70416262 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5100040600 con 0x7f5110133420 2023-09-30T22:17:52.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:52.991+0000 7f50feffd700 1 -- 172.21.15.89:0/70416262 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f51000239c0 con 0x7f5110133420 2023-09-30T22:17:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.033+0000 7f361e32e700 1 Processor -- start 2023-09-30T22:17:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.033+0000 7f361e32e700 1 -- start start 2023-09-30T22:17:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.033+0000 7f361e32e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f361809f410 0x7f3618136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.033+0000 7f361e32e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3618136fe0 0x7f36181393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.033+0000 7f361e32e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3618070940 con 0x7f361809e930 2023-09-30T22:17:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.033+0000 7f361e32e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361806fd60 con 0x7f361809f410 2023-09-30T22:17:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.033+0000 7f361e32e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f361806fa80 con 0x7f3618136fe0 2023-09-30T22:17:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.033+0000 7f361c8cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3618136fe0 0x7f36181393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f36177fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f361809f410 0x7f3618136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f361c8cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3618136fe0 0x7f36181393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39272/0 (socket says 172.21.15.89:39272) 2023-09-30T22:17:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f36177fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f361809f410 0x7f3618136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57344/0 (socket says 172.21.15.89:57344) 2023-09-30T22:17:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f361c8cb700 1 -- 172.21.15.89:0/3812453106 learned_addr learned my addr 172.21.15.89:0/3812453106 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f36177fe700 1 -- 172.21.15.89:0/3812453106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3618136fe0 msgr2=0x7f36181393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f36177fe700 1 --2- 172.21.15.89:0/3812453106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3618136fe0 0x7f36181393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f36177fe700 1 -- 172.21.15.89:0/3812453106 >> v1:172.21.15.89:6789/0 conn(0x7f361809e930 legacy=0x7f361809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f3616ffd700 1 -- 172.21.15.89:0/3812453106 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52146919 0 0) 0x7f3618070940 con 0x7f361809e930 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f36177fe700 1 -- 172.21.15.89:0/3812453106 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361809fb60 con 0x7f361809f410 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f361c8cb700 1 --2- 172.21.15.89:0/3812453106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3618136fe0 0x7f36181393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f36177fe700 1 --2- 172.21.15.89:0/3812453106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f361809f410 0x7f3618136aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f360c009870 tx=0x7f360c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6dcf2c7ac1df2b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f3616ffd700 1 -- 172.21.15.89:0/3812453106 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c011830 con 0x7f361809f410 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f3616ffd700 1 -- 172.21.15.89:0/3812453106 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c011990 con 0x7f361809f410 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.034+0000 7f3616ffd700 1 -- 172.21.15.89:0/3812453106 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c011d20 con 0x7f361809f410 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.035+0000 7f361e32e700 1 -- 172.21.15.89:0/3812453106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f361809f410 msgr2=0x7f3618136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.035+0000 7f361e32e700 1 --2- 172.21.15.89:0/3812453106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f361809f410 0x7f3618136aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f360c009870 tx=0x7f360c009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:53.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.035+0000 7f361e32e700 1 -- 172.21.15.89:0/3812453106 shutdown_connections 2023-09-30T22:17:53.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.035+0000 7f361e32e700 1 --2- 172.21.15.89:0/3812453106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3618136fe0 0x7f36181393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.035+0000 7f361e32e700 1 --2- 172.21.15.89:0/3812453106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f361809f410 0x7f3618136aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.035+0000 7f361e32e700 1 -- 172.21.15.89:0/3812453106 >> 172.21.15.89:0/3812453106 conn(0x7f36180a4210 msgr2=0x7f36180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:53.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.035+0000 7f361e32e700 1 -- 172.21.15.89:0/3812453106 shutdown_connections 2023-09-30T22:17:53.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.035+0000 7f361e32e700 1 -- 172.21.15.89:0/3812453106 wait complete. 2023-09-30T22:17:53.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f361e32e700 1 Processor -- start 2023-09-30T22:17:53.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f361e32e700 1 -- start start 2023-09-30T22:17:53.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f361e32e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f361809e930 0x7f361806c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f361e32e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f361809f410 0x7f361806c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f361e32e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3618136fe0 0x7f36181c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f361e32e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361806cd50 con 0x7f361809f410 2023-09-30T22:17:53.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f361e32e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361806be60 con 0x7f3618136fe0 2023-09-30T22:17:53.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f361e32e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3618121da0 con 0x7f361809e930 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f3617fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f361809e930 0x7f361806c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f3617fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f361809e930 0x7f361806c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39286/0 (socket says 172.21.15.89:39286) 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.036+0000 7f3617fff700 1 -- 172.21.15.89:0/908170869 learned_addr learned my addr 172.21.15.89:0/908170869 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f36177fe700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f361809f410 0x7f361806c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f361c8cb700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3618136fe0 0x7f36181c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3617fff700 1 -- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3618136fe0 msgr2=0x7f36181c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3617fff700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3618136fe0 0x7f36181c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3617fff700 1 -- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f361809f410 msgr2=0x7f361806c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3617fff700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f361809f410 0x7f361806c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3617fff700 1 -- 172.21.15.89:0/908170869 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3610009b30 con 0x7f361809e930 2023-09-30T22:17:53.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f361c8cb700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3618136fe0 0x7f36181c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3617fff700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f361809e930 0x7f361806c2d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f360800c450 tx=0x7f360800c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3614ff9700 1 -- 172.21.15.89:0/908170869 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36080054e0 con 0x7f361809e930 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3614ff9700 1 -- 172.21.15.89:0/908170869 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3608003680 con 0x7f361809e930 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c0094a0 con 0x7f361809e930 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f3614ff9700 1 -- 172.21.15.89:0/908170869 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608003a30 con 0x7f361809e930 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.037+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3618071d70 con 0x7f361809e930 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.038+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3618122180 con 0x7f361809e930 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.039+0000 7f3614ff9700 1 -- 172.21.15.89:0/908170869 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3608003b90 con 0x7f361809e930 2023-09-30T22:17:53.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.039+0000 7f3614ff9700 1 -- 172.21.15.89:0/908170869 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f360803ead0 con 0x7f361809e930 2023-09-30T22:17:53.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.043+0000 7f3614ff9700 1 -- 172.21.15.89:0/908170869 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3608003e40 con 0x7f361809e930 2023-09-30T22:17:53.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.122+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2ab011cf70 con 0x7f2ab009d6f0 2023-09-30T22:17:53.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.122+0000 7f2aacff9700 1 -- 172.21.15.89:0/1851720962 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ab011cf70 con 0x7f2ab009d6f0 2023-09-30T22:17:53.122 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab009d6f0 msgr2=0x7f2ab011a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab009d6f0 0x7f2ab011a180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000c450 tx=0x7f2aa000c760 comp rx=0 tx=0).stop 2023-09-30T22:17:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 shutdown_connections 2023-09-30T22:17:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ab0135e50 0x7f2ab01c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab009d6f0 0x7f2ab011a180 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 --2- 172.21.15.89:0/1851720962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ab009e1d0 0x7f2ab011a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 >> 172.21.15.89:0/1851720962 conn(0x7f2ab0066f50 msgr2=0x7f2ab006f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 shutdown_connections 2023-09-30T22:17:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.124+0000 7f2ab648c700 1 -- 172.21.15.89:0/1851720962 wait complete. 2023-09-30T22:17:53.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:53.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.206+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5110074ea0 con 0x7f5110133420 2023-09-30T22:17:53.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.206+0000 7f50feffd700 1 -- 172.21.15.89:0/70416262 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51000233d0 con 0x7f5110133420 2023-09-30T22:17:53.206 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5110133420 msgr2=0x7f5110070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5110133420 0x7f5110070c00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f51000090b0 tx=0x7f5100004750 comp rx=0 tx=0).stop 2023-09-30T22:17:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 shutdown_connections 2023-09-30T22:17:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5110134260 0x7f5110071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5110133420 0x7f5110070c00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 --2- 172.21.15.89:0/70416262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5110133e30 0x7f5110071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 >> 172.21.15.89:0/70416262 conn(0x7f51100a4210 msgr2=0x7f51100a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:53.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 shutdown_connections 2023-09-30T22:17:53.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.208+0000 7f5114f6d700 1 -- 172.21.15.89:0/70416262 wait complete. 2023-09-30T22:17:53.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:53.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.252+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3618074ea0 con 0x7f361809e930 2023-09-30T22:17:53.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.252+0000 7f3614ff9700 1 -- 172.21.15.89:0/908170869 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3608017030 con 0x7f361809e930 2023-09-30T22:17:53.252 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:53.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f361809e930 msgr2=0x7f361806c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f361809e930 0x7f361806c2d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f360800c450 tx=0x7f360800c760 comp rx=0 tx=0).stop 2023-09-30T22:17:53.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 shutdown_connections 2023-09-30T22:17:53.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3618136fe0 0x7f36181c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f361809e930 0x7f361806c2d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 --2- 172.21.15.89:0/908170869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f361809f410 0x7f361806c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 >> 172.21.15.89:0/908170869 conn(0x7f36180a4210 msgr2=0x7f36180a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:53.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 shutdown_connections 2023-09-30T22:17:53.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.254+0000 7f361e32e700 1 -- 172.21.15.89:0/908170869 wait complete. 2023-09-30T22:17:53.266 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:53.568 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:53.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.690+0000 7fbd3ae8c700 1 Processor -- start 2023-09-30T22:17:53.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.690+0000 7fbd3ae8c700 1 -- start start 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd3ae8c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd3409d6f0 0x7fbd3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd3ae8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd3ae8c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd3ae8c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3409f520 con 0x7fbd3409d6f0 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd3ae8c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3409f130 con 0x7fbd3409e410 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd3ae8c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3409e930 con 0x7fbd3409e000 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd38c28700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd3409d6f0 0x7fbd3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd39429700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd33fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd38c28700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd3409d6f0 0x7fbd3409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33016/0 (socket says 172.21.15.89:33016) 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd38c28700 1 -- 172.21.15.89:0/3167859173 learned_addr learned my addr 172.21.15.89:0/3167859173 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd38c28700 1 -- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 msgr2=0x7fbd34130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd38c28700 1 --2- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd38c28700 1 -- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 msgr2=0x7fbd34130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd38c28700 1 --2- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd38c28700 1 -- 172.21.15.89:0/3167859173 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3409b2d0 con 0x7fbd3409d6f0 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.691+0000 7fbd39429700 1 --2- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd38c28700 1 --2- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd3409dac0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009920 tx=0x7fbd28009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d008bd509cd8c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd337fe700 1 -- 172.21.15.89:0/3167859173 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28011830 con 0x7fbd3409d6f0 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd337fe700 1 -- 172.21.15.89:0/3167859173 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28011990 con 0x7fbd3409d6f0 2023-09-30T22:17:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd337fe700 1 -- 172.21.15.89:0/3167859173 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28011d20 con 0x7fbd3409d6f0 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 msgr2=0x7fbd3409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 --2- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd3409dac0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009920 tx=0x7fbd28009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3167859173 shutdown_connections 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 --2- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 --2- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd3409dac0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 --2- 172.21.15.89:0/3167859173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3167859173 >> 172.21.15.89:0/3167859173 conn(0x7fbd34066f50 msgr2=0x7fbd3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3167859173 shutdown_connections 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.692+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3167859173 wait complete. 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.693+0000 7fbd3ae8c700 1 Processor -- start 2023-09-30T22:17:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.693+0000 7fbd3ae8c700 1 -- start start 2023-09-30T22:17:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd3ae8c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd34132e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd3ae8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd3ae8c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd3ae8c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd341372e0 con 0x7fbd3409d6f0 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd3ae8c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd341352a0 con 0x7fbd3409e410 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd3ae8c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd34136920 con 0x7fbd3409e000 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd38c28700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd34132e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd39429700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34133340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39308/0 (socket says 172.21.15.89:39308) 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 -- 172.21.15.89:0/3852412262 learned_addr learned my addr 172.21.15.89:0/3852412262 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd38c28700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd34132e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33028/0 (socket says 172.21.15.89:33028) 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 -- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 msgr2=0x7fbd34133880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34133880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 -- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 msgr2=0x7fbd34132e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd34132e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 -- 172.21.15.89:0/3852412262 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c009a70 con 0x7fbd3409e000 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd39429700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34133880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd38c28700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd34132e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd33fff700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34133340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e960 tx=0x7fbd2000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.694+0000 7fbd317fa700 1 -- 172.21.15.89:0/3852412262 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000c5e0 con 0x7fbd3409e000 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.695+0000 7fbd317fa700 1 -- 172.21.15.89:0/3852412262 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2000c740 con 0x7fbd3409e000 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.695+0000 7fbd317fa700 1 -- 172.21.15.89:0/3852412262 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000cad0 con 0x7fbd3409e000 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.695+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd28009550 con 0x7fbd3409e000 2023-09-30T22:17:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.695+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3409c700 con 0x7fbd3409e000 2023-09-30T22:17:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.695+0000 7fbd317fa700 1 -- 172.21.15.89:0/3852412262 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd2000cc30 con 0x7fbd3409e000 2023-09-30T22:17:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.696+0000 7fbd317fa700 1 -- 172.21.15.89:0/3852412262 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbd2003f9d0 con 0x7fbd3409e000 2023-09-30T22:17:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.696+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd34074ea0 con 0x7fbd3409e000 2023-09-30T22:17:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.700+0000 7fbd317fa700 1 -- 172.21.15.89:0/3852412262 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd20021020 con 0x7fbd3409e000 2023-09-30T22:17:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.885+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbd341363e0 con 0x7fbd3409e000 2023-09-30T22:17:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.885+0000 7fbd317fa700 1 -- 172.21.15.89:0/3852412262 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd20017070 con 0x7fbd3409e000 2023-09-30T22:17:53.885 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:53.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 msgr2=0x7fbd34133340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:53.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34133340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e960 tx=0x7fbd2000ec70 comp rx=0 tx=0).stop 2023-09-30T22:17:53.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 shutdown_connections 2023-09-30T22:17:53.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd3409e410 0x7fbd34133880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd3409d6f0 0x7fbd34132e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 --2- 172.21.15.89:0/3852412262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd3409e000 0x7fbd34133340 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:53.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 >> 172.21.15.89:0/3852412262 conn(0x7fbd34066f50 msgr2=0x7fbd34070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:53.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 shutdown_connections 2023-09-30T22:17:53.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:53.887+0000 7fbd3ae8c700 1 -- 172.21.15.89:0/3852412262 wait complete. 2023-09-30T22:17:53.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:54.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:54.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:54.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.263+0000 7f704c13d700 1 Processor -- start 2023-09-30T22:17:54.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.263+0000 7f704c13d700 1 -- start start 2023-09-30T22:17:54.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f704c13d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704409e1d0 0x7f7044138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f704c13d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7044139170 0x7f704413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f704c13d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704409b120 con 0x7f704409d6f0 2023-09-30T22:17:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f704c13d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704406b460 con 0x7f704409e1d0 2023-09-30T22:17:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f704c13d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f704409a7e0 con 0x7f7044139170 2023-09-30T22:17:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f704a6da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7044139170 0x7f704413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f70496d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704409e1d0 0x7f7044138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f7049ed9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f704409d6f0 0x7f704409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36124/0 (socket says 172.21.15.89:36124) 2023-09-30T22:17:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f7049ed9700 1 -- 172.21.15.89:0/551241622 learned_addr learned my addr 172.21.15.89:0/551241622 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f7048ed7700 1 -- 172.21.15.89:0/551241622 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120774034 0 0) 0x7f704409b120 con 0x7f704409d6f0 2023-09-30T22:17:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f70496d8700 1 -- 172.21.15.89:0/551241622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7044139170 msgr2=0x7f704413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f70496d8700 1 --2- 172.21.15.89:0/551241622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7044139170 0x7f704413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f70496d8700 1 -- 172.21.15.89:0/551241622 >> v1:172.21.15.89:6789/0 conn(0x7f704409d6f0 legacy=0x7f704409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f70496d8700 1 -- 172.21.15.89:0/551241622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704409a500 con 0x7f704409e1d0 2023-09-30T22:17:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.264+0000 7f70496d8700 1 --2- 172.21.15.89:0/551241622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704409e1d0 0x7f7044138c30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7034009870 tx=0x7f7034009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3541320fc17af9b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f7048ed7700 1 -- 172.21.15.89:0/551241622 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034011830 con 0x7f704409e1d0 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f7048ed7700 1 -- 172.21.15.89:0/551241622 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7034011990 con 0x7f704409e1d0 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f7048ed7700 1 -- 172.21.15.89:0/551241622 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034011d20 con 0x7f704409e1d0 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f704c13d700 1 -- 172.21.15.89:0/551241622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704409e1d0 msgr2=0x7f7044138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f704c13d700 1 --2- 172.21.15.89:0/551241622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704409e1d0 0x7f7044138c30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7034009870 tx=0x7f7034009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f704c13d700 1 -- 172.21.15.89:0/551241622 shutdown_connections 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f704c13d700 1 --2- 172.21.15.89:0/551241622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7044139170 0x7f704413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f704c13d700 1 --2- 172.21.15.89:0/551241622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f704409e1d0 0x7f7044138c30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f704c13d700 1 -- 172.21.15.89:0/551241622 >> 172.21.15.89:0/551241622 conn(0x7f7044066f50 msgr2=0x7f704409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f704c13d700 1 -- 172.21.15.89:0/551241622 shutdown_connections 2023-09-30T22:17:54.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.265+0000 7f704c13d700 1 -- 172.21.15.89:0/551241622 wait complete. 2023-09-30T22:17:54.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.266+0000 7f704c13d700 1 Processor -- start 2023-09-30T22:17:54.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.266+0000 7f704c13d700 1 -- start start 2023-09-30T22:17:54.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:54.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704c13d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704409d6f0 0x7f70441217b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704c13d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704409e1d0 0x7f7044121cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704c13d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7044139170 0x7f7044122230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704c13d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7044124230 con 0x7f704409d6f0 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704c13d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044124f50 con 0x7f7044139170 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704c13d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70440a2090 con 0x7f704409e1d0 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704409e1d0 0x7f7044121cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704a6da700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7044139170 0x7f7044122230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704409e1d0 0x7f7044121cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39322/0 (socket says 172.21.15.89:39322) 2023-09-30T22:17:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 -- 172.21.15.89:0/3908738522 learned_addr learned my addr 172.21.15.89:0/3908738522 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f7049ed9700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704409d6f0 0x7f70441217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 -- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7044139170 msgr2=0x7f7044122230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7044139170 0x7f7044122230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 -- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704409d6f0 msgr2=0x7f70441217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704409d6f0 0x7f70441217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 -- 172.21.15.89:0/3908738522 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70340094a0 con 0x7f704409e1d0 2023-09-30T22:17:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704a6da700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7044139170 0x7f7044122230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f7049ed9700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704409d6f0 0x7f70441217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f70496d8700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704409e1d0 0x7f7044121cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7034000c00 tx=0x7f7034004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f703a7fc700 1 -- 172.21.15.89:0/3908738522 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034024620 con 0x7f704409e1d0 2023-09-30T22:17:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f703a7fc700 1 -- 172.21.15.89:0/3908738522 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7034024780 con 0x7f704409e1d0 2023-09-30T22:17:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704409c710 con 0x7f704409e1d0 2023-09-30T22:17:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f703a7fc700 1 -- 172.21.15.89:0/3908738522 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034024b30 con 0x7f704409e1d0 2023-09-30T22:17:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.267+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70441259c0 con 0x7f704409e1d0 2023-09-30T22:17:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.268+0000 7f703a7fc700 1 -- 172.21.15.89:0/3908738522 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7034022070 con 0x7f704409e1d0 2023-09-30T22:17:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.268+0000 7f703a7fc700 1 -- 172.21.15.89:0/3908738522 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7034040a20 con 0x7f704409e1d0 2023-09-30T22:17:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.268+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7044074ea0 con 0x7f704409e1d0 2023-09-30T22:17:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.277+0000 7f703a7fc700 1 -- 172.21.15.89:0/3908738522 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7034029080 con 0x7f704409e1d0 2023-09-30T22:17:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f948d4af700 1 Processor -- start 2023-09-30T22:17:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f948d4af700 1 -- start start 2023-09-30T22:17:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f948d4af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 0x7f948809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f948d4af700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f948809e000 0x7f9488130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f948d4af700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e410 0x7f9488130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f948d4af700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948809f520 con 0x7f948809e000 2023-09-30T22:17:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f948d4af700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948809f130 con 0x7f948809e410 2023-09-30T22:17:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f948d4af700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f948809e930 con 0x7f948809d6f0 2023-09-30T22:17:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f94877fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e410 0x7f9488130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f94877fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e410 0x7f9488130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57412/0 (socket says 172.21.15.89:57412) 2023-09-30T22:17:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f94877fe700 1 -- 172.21.15.89:0/1898060474 learned_addr learned my addr 172.21.15.89:0/1898060474 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f94867fc700 1 --2- 172.21.15.89:0/1898060474 >> v2:172.21.15.89:3300/0 conn(0x7f948809e000 0x7f9488130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.352+0000 7f9486ffd700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 0x7f948809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f94867fc700 1 -- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 msgr2=0x7f948809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f94867fc700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 0x7f948809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f94867fc700 1 -- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e410 msgr2=0x7f9488130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f94867fc700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e410 0x7f9488130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f94867fc700 1 -- 172.21.15.89:0/1898060474 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948809b2d0 con 0x7f948809e000 2023-09-30T22:17:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f94877fe700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e410 0x7f9488130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f94867fc700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e000 0x7f9488130660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f947c009870 tx=0x7f947c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745d55a6e5d0f6e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f9485ffb700 1 -- 172.21.15.89:0/1898060474 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c011830 con 0x7f948809e000 2023-09-30T22:17:54.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f9485ffb700 1 -- 172.21.15.89:0/1898060474 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c011990 con 0x7f948809e000 2023-09-30T22:17:54.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.353+0000 7f9485ffb700 1 -- 172.21.15.89:0/1898060474 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c011d20 con 0x7f948809e000 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 -- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e000 msgr2=0x7f9488130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e000 0x7f9488130660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f947c009870 tx=0x7f947c009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 -- 172.21.15.89:0/1898060474 shutdown_connections 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e410 0x7f9488130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 0x7f948809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 --2- 172.21.15.89:0/1898060474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e000 0x7f9488130660 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 -- 172.21.15.89:0/1898060474 >> 172.21.15.89:0/1898060474 conn(0x7f9488066f50 msgr2=0x7f948809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 -- 172.21.15.89:0/1898060474 shutdown_connections 2023-09-30T22:17:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.354+0000 7f948d4af700 1 -- 172.21.15.89:0/1898060474 wait complete. 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f948d4af700 1 Processor -- start 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f948d4af700 1 -- start start 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f948d4af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 0x7f9488136910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f948d4af700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e000 0x7f9488136e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f948d4af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e410 0x7f9488137390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f948d4af700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9488119c70 con 0x7f948809e410 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f948d4af700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9488072480 con 0x7f948809e000 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f948d4af700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f948809c700 con 0x7f948809d6f0 2023-09-30T22:17:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f94867fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e000 0x7f9488136e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f94867fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e000 0x7f9488136e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57428/0 (socket says 172.21.15.89:57428) 2023-09-30T22:17:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.355+0000 7f94867fc700 1 -- 172.21.15.89:0/3154214436 learned_addr learned my addr 172.21.15.89:0/3154214436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f9486ffd700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 0x7f9488136910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f94877fe700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e410 0x7f9488137390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f94867fc700 1 -- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 msgr2=0x7f9488136910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f94867fc700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 0x7f9488136910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f94867fc700 1 -- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e410 msgr2=0x7f9488137390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f94867fc700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e410 0x7f9488137390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f94867fc700 1 -- 172.21.15.89:0/3154214436 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480009b30 con 0x7f948809e000 2023-09-30T22:17:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f94867fc700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e000 0x7f9488136e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f947c000c00 tx=0x7f947c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f946f7fe700 1 -- 172.21.15.89:0/3154214436 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c004450 con 0x7f948809e000 2023-09-30T22:17:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f946f7fe700 1 -- 172.21.15.89:0/3154214436 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c01f070 con 0x7f948809e000 2023-09-30T22:17:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c0094a0 con 0x7f948809e000 2023-09-30T22:17:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f946f7fe700 1 -- 172.21.15.89:0/3154214436 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0038a0 con 0x7f948809e000 2023-09-30T22:17:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.356+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948811a280 con 0x7f948809e000 2023-09-30T22:17:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.357+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94881310c0 con 0x7f948809e000 2023-09-30T22:17:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.357+0000 7f946f7fe700 1 -- 172.21.15.89:0/3154214436 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f947c003a00 con 0x7f948809e000 2023-09-30T22:17:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.357+0000 7f946f7fe700 1 -- 172.21.15.89:0/3154214436 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f947c021070 con 0x7f948809e000 2023-09-30T22:17:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.362+0000 7f946f7fe700 1 -- 172.21.15.89:0/3154214436 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94881310c0 con 0x7f948809e000 2023-09-30T22:17:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.407+0000 7f333d973700 1 Processor -- start 2023-09-30T22:17:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.407+0000 7f333d973700 1 -- start start 2023-09-30T22:17:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f333d973700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f33380a1510 0x7f3338138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f333d973700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3338139250 0x7f333813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f333d973700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333809edd0 con 0x7f33380a0a30 2023-09-30T22:17:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f333d973700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33380a18e0 con 0x7f33380a1510 2023-09-30T22:17:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f333d973700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333809e1f0 con 0x7f3338139250 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33377fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3338139250 0x7f333813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33367fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f33380a1510 0x7f3338138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33377fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3338139250 0x7f333813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57432/0 (socket says 172.21.15.89:57432) 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33377fe700 1 -- 172.21.15.89:0/658107417 learned_addr learned my addr 172.21.15.89:0/658107417 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f3336ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f33380a0a30 0x7f33380a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36130/0 (socket says 172.21.15.89:36130) 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33367fc700 1 -- 172.21.15.89:0/658107417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3338139250 msgr2=0x7f333813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33367fc700 1 --2- 172.21.15.89:0/658107417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3338139250 0x7f333813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33367fc700 1 -- 172.21.15.89:0/658107417 >> v1:172.21.15.89:6789/0 conn(0x7f33380a0a30 legacy=0x7f33380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33367fc700 1 -- 172.21.15.89:0/658107417 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333809dc50 con 0x7f33380a1510 2023-09-30T22:17:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.408+0000 7f33367fc700 1 --2- 172.21.15.89:0/658107417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33380a1510 0x7f3338138d10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f332c009840 tx=0x7f332c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487ab837bfbef77c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f3335ffb700 1 -- 172.21.15.89:0/658107417 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c011830 con 0x7f33380a1510 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f3335ffb700 1 -- 172.21.15.89:0/658107417 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c011990 con 0x7f33380a1510 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f3335ffb700 1 -- 172.21.15.89:0/658107417 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c011d20 con 0x7f33380a1510 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f333d973700 1 -- 172.21.15.89:0/658107417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33380a1510 msgr2=0x7f3338138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f333d973700 1 --2- 172.21.15.89:0/658107417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33380a1510 0x7f3338138d10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f332c009840 tx=0x7f332c009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f333d973700 1 -- 172.21.15.89:0/658107417 shutdown_connections 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f333d973700 1 --2- 172.21.15.89:0/658107417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3338139250 0x7f333813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f333d973700 1 --2- 172.21.15.89:0/658107417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33380a1510 0x7f3338138d10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f333d973700 1 -- 172.21.15.89:0/658107417 >> 172.21.15.89:0/658107417 conn(0x7f33380a6550 msgr2=0x7f33380a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f333d973700 1 -- 172.21.15.89:0/658107417 shutdown_connections 2023-09-30T22:17:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.409+0000 7f333d973700 1 -- 172.21.15.89:0/658107417 wait complete. 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.410+0000 7f333d973700 1 Processor -- start 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.410+0000 7f333d973700 1 -- start start 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f333d973700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33380a0a30 0x7f3338123440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f333d973700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33380a1510 0x7f3338123980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f333d973700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338139250 0x7f33381c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f333d973700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338138d40 con 0x7f3338139250 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f333d973700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33380a0220 con 0x7f33380a1510 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f333d973700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33381260f0 con 0x7f33380a0a30 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f3336ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33380a0a30 0x7f3338123440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33367fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33380a1510 0x7f3338123980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f3336ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33380a0a30 0x7f3338123440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39340/0 (socket says 172.21.15.89:39340) 2023-09-30T22:17:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f3336ffd700 1 -- 172.21.15.89:0/2465045109 learned_addr learned my addr 172.21.15.89:0/2465045109 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33377fe700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338139250 0x7f33381c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33367fc700 1 -- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33380a0a30 msgr2=0x7f3338123440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33367fc700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33380a0a30 0x7f3338123440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33367fc700 1 -- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338139250 msgr2=0x7f33381c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33367fc700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338139250 0x7f33381c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33367fc700 1 -- 172.21.15.89:0/2465045109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0094a0 con 0x7f33380a1510 2023-09-30T22:17:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33377fe700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338139250 0x7f33381c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.411+0000 7f33367fc700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33380a1510 0x7f3338123980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f332c016000 tx=0x7f332c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.412+0000 7f331ffff700 1 -- 172.21.15.89:0/2465045109 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0036a0 con 0x7f33380a1510 2023-09-30T22:17:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.412+0000 7f331ffff700 1 -- 172.21.15.89:0/2465045109 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c01f070 con 0x7f33380a1510 2023-09-30T22:17:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.412+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3338126e20 con 0x7f33380a1510 2023-09-30T22:17:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.412+0000 7f331ffff700 1 -- 172.21.15.89:0/2465045109 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c003a60 con 0x7f33380a1510 2023-09-30T22:17:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.412+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3338126690 con 0x7f33380a1510 2023-09-30T22:17:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.413+0000 7f331ffff700 1 -- 172.21.15.89:0/2465045109 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f332c003bc0 con 0x7f33380a1510 2023-09-30T22:17:54.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.413+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3338074ea0 con 0x7f33380a1510 2023-09-30T22:17:54.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.413+0000 7f331ffff700 1 -- 172.21.15.89:0/2465045109 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f332c021070 con 0x7f33380a1510 2023-09-30T22:17:54.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.417+0000 7f331ffff700 1 -- 172.21.15.89:0/2465045109 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f332c019d80 con 0x7f33380a1510 2023-09-30T22:17:54.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.474+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f704406c360 con 0x7f704409e1d0 2023-09-30T22:17:54.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.474+0000 7f703a7fc700 1 -- 172.21.15.89:0/3908738522 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7034015020 con 0x7f704409e1d0 2023-09-30T22:17:54.475 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:54.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704409e1d0 msgr2=0x7f7044121cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704409e1d0 0x7f7044121cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7034000c00 tx=0x7f7034004750 comp rx=0 tx=0).stop 2023-09-30T22:17:54.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 shutdown_connections 2023-09-30T22:17:54.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7044139170 0x7f7044122230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f704409d6f0 0x7f70441217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 --2- 172.21.15.89:0/3908738522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f704409e1d0 0x7f7044121cf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 >> 172.21.15.89:0/3908738522 conn(0x7f7044066f50 msgr2=0x7f7044070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 shutdown_connections 2023-09-30T22:17:54.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.476+0000 7f704c13d700 1 -- 172.21.15.89:0/3908738522 wait complete. 2023-09-30T22:17:54.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:54.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.562+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f94881310c0 con 0x7f948809e000 2023-09-30T22:17:54.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.563+0000 7f946f7fe700 1 -- 172.21.15.89:0/3154214436 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94881310c0 con 0x7f948809e000 2023-09-30T22:17:54.563 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:54.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e000 msgr2=0x7f9488136e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e000 0x7f9488136e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f947c000c00 tx=0x7f947c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:54.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 shutdown_connections 2023-09-30T22:17:54.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f948809e410 0x7f9488137390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948809d6f0 0x7f9488136910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 --2- 172.21.15.89:0/3154214436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f948809e000 0x7f9488136e50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 >> 172.21.15.89:0/3154214436 conn(0x7f9488066f50 msgr2=0x7f948806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:54.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 shutdown_connections 2023-09-30T22:17:54.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.565+0000 7f948d4af700 1 -- 172.21.15.89:0/3154214436 wait complete. 2023-09-30T22:17:54.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:54.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.611+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f33380a1a90 con 0x7f33380a1510 2023-09-30T22:17:54.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.611+0000 7f331ffff700 1 -- 172.21.15.89:0/2465045109 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f332c02b070 con 0x7f33380a1510 2023-09-30T22:17:54.611 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:54.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33380a1510 msgr2=0x7f3338123980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:54.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33380a1510 0x7f3338123980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f332c016000 tx=0x7f332c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:54.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 shutdown_connections 2023-09-30T22:17:54.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338139250 0x7f33381c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33380a0a30 0x7f3338123440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 --2- 172.21.15.89:0/2465045109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33380a1510 0x7f3338123980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:54.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 >> 172.21.15.89:0/2465045109 conn(0x7f33380a6550 msgr2=0x7f33380a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:54.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 shutdown_connections 2023-09-30T22:17:54.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:54.613+0000 7f333d973700 1 -- 172.21.15.89:0/2465045109 wait complete. 2023-09-30T22:17:54.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:54.900 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.024+0000 7f5ea61d3700 1 Processor -- start 2023-09-30T22:17:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.024+0000 7f5ea61d3700 1 -- start start 2023-09-30T22:17:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.024+0000 7f5ea61d3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ea009d6f0 0x7f5ea009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.024+0000 7f5ea61d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.024+0000 7f5ea61d3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea009f130 con 0x7f5ea009e410 2023-09-30T22:17:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.024+0000 7f5ea61d3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea009f520 con 0x7f5ea009d6f0 2023-09-30T22:17:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.024+0000 7f5ea61d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea009e930 con 0x7f5ea009e000 2023-09-30T22:17:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ea009d6f0 0x7f5ea009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ea009d6f0 0x7f5ea009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33094/0 (socket says 172.21.15.89:33094) 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5ea009e410 0x7f5ea0130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36134/0 (socket says 172.21.15.89:36134) 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39356/0 (socket says 172.21.15.89:39356) 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9f7fe700 1 -- 172.21.15.89:0/1253933407 learned_addr learned my addr 172.21.15.89:0/1253933407 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9f7fe700 1 -- 172.21.15.89:0/1253933407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 msgr2=0x7f5ea0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9f7fe700 1 --2- 172.21.15.89:0/1253933407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9f7fe700 1 -- 172.21.15.89:0/1253933407 >> v1:172.21.15.89:6789/0 conn(0x7f5ea009e410 legacy=0x7f5ea0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9f7fe700 1 -- 172.21.15.89:0/1253933407 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea009b2d0 con 0x7f5ea009d6f0 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9e7fc700 1 -- 172.21.15.89:0/1253933407 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36045512 0 0) 0x7f5ea009f130 con 0x7f5ea009e410 2023-09-30T22:17:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9effd700 1 --2- 172.21.15.89:0/1253933407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9f7fe700 1 --2- 172.21.15.89:0/1253933407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009d6f0 0x7f5ea009dac0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5e940098f0 tx=0x7f5e94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c5ee2972198357 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9e7fc700 1 -- 172.21.15.89:0/1253933407 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94011830 con 0x7f5ea009d6f0 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9e7fc700 1 -- 172.21.15.89:0/1253933407 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94011990 con 0x7f5ea009d6f0 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.025+0000 7f5e9e7fc700 1 -- 172.21.15.89:0/1253933407 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94011d20 con 0x7f5ea009d6f0 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.026+0000 7f5ea61d3700 1 -- 172.21.15.89:0/1253933407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009d6f0 msgr2=0x7f5ea009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.026+0000 7f5ea61d3700 1 --2- 172.21.15.89:0/1253933407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009d6f0 0x7f5ea009dac0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5e940098f0 tx=0x7f5e94009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.026+0000 7f5ea61d3700 1 -- 172.21.15.89:0/1253933407 shutdown_connections 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.026+0000 7f5ea61d3700 1 --2- 172.21.15.89:0/1253933407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009d6f0 0x7f5ea009dac0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.026+0000 7f5ea61d3700 1 --2- 172.21.15.89:0/1253933407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.026+0000 7f5ea61d3700 1 -- 172.21.15.89:0/1253933407 >> 172.21.15.89:0/1253933407 conn(0x7f5ea0066f50 msgr2=0x7f5ea009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.026+0000 7f5ea61d3700 1 -- 172.21.15.89:0/1253933407 shutdown_connections 2023-09-30T22:17:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.026+0000 7f5ea61d3700 1 -- 172.21.15.89:0/1253933407 wait complete. 2023-09-30T22:17:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.027+0000 7f5ea61d3700 1 Processor -- start 2023-09-30T22:17:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.027+0000 7f5ea61d3700 1 -- start start 2023-09-30T22:17:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5ea61d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ea009d6f0 0x7f5ea011dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5ea61d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea011e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5ea61d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009e410 0x7f5ea011e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5ea61d3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0122270 con 0x7f5ea009e410 2023-09-30T22:17:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5ea61d3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea01218b0 con 0x7f5ea009d6f0 2023-09-30T22:17:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5ea61d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0120bf0 con 0x7f5ea009e000 2023-09-30T22:17:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ea009d6f0 0x7f5ea011dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009e410 0x7f5ea011e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ea009d6f0 0x7f5ea011dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57462/0 (socket says 172.21.15.89:57462) 2023-09-30T22:17:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 -- 172.21.15.89:0/3593321898 learned_addr learned my addr 172.21.15.89:0/3593321898 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9effd700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea011e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 -- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 msgr2=0x7f5ea011e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea011e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 -- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009e410 msgr2=0x7f5ea011e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009e410 0x7f5ea011e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 -- 172.21.15.89:0/3593321898 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9000b740 con 0x7f5ea009d6f0 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9ffff700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009e410 0x7f5ea011e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.028+0000 7f5e9f7fe700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ea009d6f0 0x7f5ea011dd90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ea009b9e0 tx=0x7f5e94004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.029+0000 7f5e87fff700 1 -- 172.21.15.89:0/3593321898 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94004300 con 0x7f5ea009d6f0 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.029+0000 7f5e87fff700 1 -- 172.21.15.89:0/3593321898 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9401f070 con 0x7f5ea009d6f0 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.029+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e94009550 con 0x7f5ea009d6f0 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.029+0000 7f5e87fff700 1 -- 172.21.15.89:0/3593321898 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940036a0 con 0x7f5ea009d6f0 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.029+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea011f9a0 con 0x7f5ea009d6f0 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.029+0000 7f5e87fff700 1 -- 172.21.15.89:0/3593321898 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e94021030 con 0x7f5ea009d6f0 2023-09-30T22:17:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.029+0000 7f5e87fff700 1 -- 172.21.15.89:0/3593321898 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5e94040980 con 0x7f5ea009d6f0 2023-09-30T22:17:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.030+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea011ac20 con 0x7f5ea009d6f0 2023-09-30T22:17:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.034+0000 7f5e87fff700 1 -- 172.21.15.89:0/3593321898 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e940199a0 con 0x7f5ea009d6f0 2023-09-30T22:17:55.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.217+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5ea0121450 con 0x7f5ea009d6f0 2023-09-30T22:17:55.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.217+0000 7f5e87fff700 1 -- 172.21.15.89:0/3593321898 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e940193b0 con 0x7f5ea009d6f0 2023-09-30T22:17:55.217 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.219+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ea009d6f0 msgr2=0x7f5ea011dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.219+0000 7f5ea61d3700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ea009d6f0 0x7f5ea011dd90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ea009b9e0 tx=0x7f5e94004750 comp rx=0 tx=0).stop 2023-09-30T22:17:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.219+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 shutdown_connections 2023-09-30T22:17:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.219+0000 7f5ea61d3700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ea009e410 0x7f5ea011e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.219+0000 7f5ea61d3700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ea009d6f0 0x7f5ea011dd90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.219+0000 7f5ea61d3700 1 --2- 172.21.15.89:0/3593321898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ea009e000 0x7f5ea011e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.219+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 >> 172.21.15.89:0/3593321898 conn(0x7f5ea0066f50 msgr2=0x7f5ea0130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.219+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 shutdown_connections 2023-09-30T22:17:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.220+0000 7f5ea61d3700 1 -- 172.21.15.89:0/3593321898 wait complete. 2023-09-30T22:17:55.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:55.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:55.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.619+0000 7f2a0286c700 1 Processor -- start 2023-09-30T22:17:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.619+0000 7f2a0286c700 1 -- start start 2023-09-30T22:17:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a0286c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f29fc09e930 0x7f29fc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a0286c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f240 0x7f29fc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a0286c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09f650 0x7f29fc131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a0286c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29fc070940 con 0x7f29fc09e930 2023-09-30T22:17:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a0286c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc06fd60 con 0x7f29fc09f240 2023-09-30T22:17:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a0286c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc06fa80 con 0x7f29fc09f650 2023-09-30T22:17:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f29fb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f240 0x7f29fc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f29fbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f29fc09e930 0x7f29fc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f29fb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f240 0x7f29fc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57472/0 (socket says 172.21.15.89:57472) 2023-09-30T22:17:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f29fb7fe700 1 -- 172.21.15.89:0/393216896 learned_addr learned my addr 172.21.15.89:0/393216896 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a00e09700 1 --2- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09f650 0x7f29fc131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a00e09700 1 -- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f240 msgr2=0x7f29fc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a00e09700 1 --2- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f240 0x7f29fc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a00e09700 1 -- 172.21.15.89:0/393216896 >> v2:172.21.15.89:3300/0 conn(0x7f29fc09e930 msgr2=0x7f29fc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a00e09700 1 --2- 172.21.15.89:0/393216896 >> v2:172.21.15.89:3300/0 conn(0x7f29fc09e930 0x7f29fc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a00e09700 1 -- 172.21.15.89:0/393216896 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc09fb60 con 0x7f29fc09f650 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f29fb7fe700 1 --2- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f240 0x7f29fc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f29fbfff700 1 --2- 172.21.15.89:0/393216896 >> v2:172.21.15.89:3300/0 conn(0x7f29fc09e930 0x7f29fc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.620+0000 7f2a00e09700 1 --2- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09f650 0x7f29fc131d30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009d90 tx=0x7f29f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd1702500efc618 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f29faffd700 1 -- 172.21.15.89:0/393216896 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f400e070 con 0x7f29fc09f650 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f29faffd700 1 -- 172.21.15.89:0/393216896 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f400bb20 con 0x7f29fc09f650 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f29faffd700 1 -- 172.21.15.89:0/393216896 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4012430 con 0x7f29fc09f650 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 -- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09f650 msgr2=0x7f29fc131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 --2- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09f650 0x7f29fc131d30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009d90 tx=0x7f29f400b330 comp rx=0 tx=0).stop 2023-09-30T22:17:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 -- 172.21.15.89:0/393216896 shutdown_connections 2023-09-30T22:17:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 --2- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09f650 0x7f29fc131d30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 --2- 172.21.15.89:0/393216896 >> v2:172.21.15.89:3300/0 conn(0x7f29fc09e930 0x7f29fc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 --2- 172.21.15.89:0/393216896 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f240 0x7f29fc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 -- 172.21.15.89:0/393216896 >> 172.21.15.89:0/393216896 conn(0x7f29fc0a4210 msgr2=0x7f29fc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 -- 172.21.15.89:0/393216896 shutdown_connections 2023-09-30T22:17:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.621+0000 7f2a0286c700 1 -- 172.21.15.89:0/393216896 wait complete. 2023-09-30T22:17:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.622+0000 7f2a0286c700 1 Processor -- start 2023-09-30T22:17:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.622+0000 7f2a0286c700 1 -- start start 2023-09-30T22:17:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a0286c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09e930 0x7f29fc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a0286c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29fc09f240 0x7f29fc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a0286c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f650 0x7f29fc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a0286c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc06d550 con 0x7f29fc09f240 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a0286c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1a94e0 con 0x7f29fc09f650 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a0286c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc06d360 con 0x7f29fc09e930 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f29fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29fc09f240 0x7f29fc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a00e09700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f650 0x7f29fc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f29fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29fc09f240 0x7f29fc06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33112/0 (socket says 172.21.15.89:33112) 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f29fb7fe700 1 -- 172.21.15.89:0/1990646330 learned_addr learned my addr 172.21.15.89:0/1990646330 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f29fbfff700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09e930 0x7f29fc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a00e09700 1 -- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09e930 msgr2=0x7f29fc06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a00e09700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09e930 0x7f29fc06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a00e09700 1 -- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29fc09f240 msgr2=0x7f29fc06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a00e09700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29fc09f240 0x7f29fc06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a00e09700 1 -- 172.21.15.89:0/1990646330 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec0094a0 con 0x7f29fc09f650 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f29fb7fe700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29fc09f240 0x7f29fc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f29fbfff700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09e930 0x7f29fc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.623+0000 7f2a00e09700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f650 0x7f29fc06c330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f29fc06f570 tx=0x7f29f4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.624+0000 7f29f8ff9700 1 -- 172.21.15.89:0/1990646330 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004800 con 0x7f29fc09f650 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.624+0000 7f29f8ff9700 1 -- 172.21.15.89:0/1990646330 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f4020070 con 0x7f29fc09f650 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.624+0000 7f29f8ff9700 1 -- 172.21.15.89:0/1990646330 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40276c0 con 0x7f29fc09f650 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.624+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f0009550 con 0x7f29fc09f650 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.624+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f4009a70 con 0x7f29fc09f650 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.624+0000 7f29f8ff9700 1 -- 172.21.15.89:0/1990646330 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f29f4012430 con 0x7f29fc09f650 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.625+0000 7f29f8ff9700 1 -- 172.21.15.89:0/1990646330 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f29f4022070 con 0x7f29fc09f650 2023-09-30T22:17:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.625+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29fc1a9620 con 0x7f29fc09f650 2023-09-30T22:17:55.629 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.629+0000 7f29f8ff9700 1 -- 172.21.15.89:0/1990646330 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29fc1a9620 con 0x7f29fc09f650 2023-09-30T22:17:55.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.708+0000 7fe59b27e700 1 Processor -- start 2023-09-30T22:17:55.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.708+0000 7fe59b27e700 1 -- start start 2023-09-30T22:17:55.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59b27e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5940a0a30 0x7fe5940a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59b27e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe5940a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59b27e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe594133930 0x7fe594133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59b27e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5940a2a70 con 0x7fe5940a0a30 2023-09-30T22:17:55.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59b27e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5940a01d0 con 0x7fe594133930 2023-09-30T22:17:55.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59b27e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe59409e2b0 con 0x7fe59409d630 2023-09-30T22:17:55.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5940a0a30 0x7fe5940a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5940a0a30 0x7fe5940a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33118/0 (socket says 172.21.15.89:33118) 2023-09-30T22:17:55.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 -- 172.21.15.89:0/1653042469 learned_addr learned my addr 172.21.15.89:0/1653042469 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:55.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe598819700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe5940a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59981b700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe594133930 0x7fe594133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 -- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 msgr2=0x7fe5940a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe5940a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 -- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe594133930 msgr2=0x7fe594133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe594133930 0x7fe594133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 -- 172.21.15.89:0/1653042469 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59409dc50 con 0x7fe5940a0a30 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe598819700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe5940a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.709+0000 7fe59901a700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5940a0a30 0x7fe5940a0e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe5900098f0 tx=0x7fe590009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1c214acf2d6d61 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe58bfff700 1 -- 172.21.15.89:0/1653042469 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011830 con 0x7fe5940a0a30 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe58bfff700 1 -- 172.21.15.89:0/1653042469 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590011990 con 0x7fe5940a0a30 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe58bfff700 1 -- 172.21.15.89:0/1653042469 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011d20 con 0x7fe5940a0a30 2023-09-30T22:17:55.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 -- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5940a0a30 msgr2=0x7fe5940a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5940a0a30 0x7fe5940a0e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe5900098f0 tx=0x7fe590009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:55.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 -- 172.21.15.89:0/1653042469 shutdown_connections 2023-09-30T22:17:55.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe594133930 0x7fe594133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5940a0a30 0x7fe5940a0e00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 --2- 172.21.15.89:0/1653042469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe5940a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 -- 172.21.15.89:0/1653042469 >> 172.21.15.89:0/1653042469 conn(0x7fe5940a3050 msgr2=0x7fe5940a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:55.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 -- 172.21.15.89:0/1653042469 shutdown_connections 2023-09-30T22:17:55.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.710+0000 7fe59b27e700 1 -- 172.21.15.89:0/1653042469 wait complete. 2023-09-30T22:17:55.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.711+0000 7fe59b27e700 1 Processor -- start 2023-09-30T22:17:55.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.711+0000 7fe59b27e700 1 -- start start 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe59b27e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe59411e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe59b27e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5940a0a30 0x7fe59411ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe59b27e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe594133930 0x7fe59411f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe59b27e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5941205c0 con 0x7fe594133930 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe59b27e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe594120a40 con 0x7fe5940a0a30 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe59b27e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe594122350 con 0x7fe59409d630 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5940a0a30 0x7fe59411ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5940a0a30 0x7fe59411ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57504/0 (socket says 172.21.15.89:57504) 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 -- 172.21.15.89:0/3511592971 learned_addr learned my addr 172.21.15.89:0/3511592971 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:55.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe59901a700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe59411e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe59981b700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe594133930 0x7fe59411f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 -- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 msgr2=0x7fe59411e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe59411e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 -- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe594133930 msgr2=0x7fe59411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe594133930 0x7fe59411f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 -- 172.21.15.89:0/3511592971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840094a0 con 0x7fe5940a0a30 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe598819700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5940a0a30 0x7fe59411ee00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe58400ebe0 tx=0x7fe58400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.712+0000 7fe589ffb700 1 -- 172.21.15.89:0/3511592971 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58400c850 con 0x7fe5940a0a30 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.713+0000 7fe589ffb700 1 -- 172.21.15.89:0/3511592971 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58400c9b0 con 0x7fe5940a0a30 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.713+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590009550 con 0x7fe5940a0a30 2023-09-30T22:17:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.713+0000 7fe589ffb700 1 -- 172.21.15.89:0/3511592971 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58400cd40 con 0x7fe5940a0a30 2023-09-30T22:17:55.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.713+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe594122070 con 0x7fe5940a0a30 2023-09-30T22:17:55.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.713+0000 7fe589ffb700 1 -- 172.21.15.89:0/3511592971 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe584013470 con 0x7fe5940a0a30 2023-09-30T22:17:55.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.714+0000 7fe589ffb700 1 -- 172.21.15.89:0/3511592971 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe58403fbf0 con 0x7fe5940a0a30 2023-09-30T22:17:55.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.714+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5940747f0 con 0x7fe5940a0a30 2023-09-30T22:17:55.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.718+0000 7fe589ffb700 1 -- 172.21.15.89:0/3511592971 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe584021080 con 0x7fe5940a0a30 2023-09-30T22:17:55.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.757+0000 7f5fdf4ba700 1 Processor -- start 2023-09-30T22:17:55.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.757+0000 7f5fdf4ba700 1 -- start start 2023-09-30T22:17:55.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdf4ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd8133a80 0x7f5fd8133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdf4ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd812ff20 0x7f5fd8130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdf4ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fd8130870 0x7f5fd813a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdf4ba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd806c570 con 0x7f5fd8130870 2023-09-30T22:17:55.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdf4ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd806bc00 con 0x7f5fd812ff20 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdf4ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80a0040 con 0x7f5fd8133a80 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdca55700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd812ff20 0x7f5fd8130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdda57700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fd8130870 0x7f5fd813a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdca55700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd812ff20 0x7f5fd8130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57512/0 (socket says 172.21.15.89:57512) 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdd256700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd8133a80 0x7f5fd8133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdda57700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5fd8130870 0x7f5fd813a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33128/0 (socket says 172.21.15.89:33128) 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdca55700 1 -- 172.21.15.89:0/230092284 learned_addr learned my addr 172.21.15.89:0/230092284 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdd256700 1 -- 172.21.15.89:0/230092284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd812ff20 msgr2=0x7f5fd8130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:55.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdd256700 1 --2- 172.21.15.89:0/230092284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd812ff20 0x7f5fd8130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdd256700 1 -- 172.21.15.89:0/230092284 >> v2:172.21.15.89:3300/0 conn(0x7f5fd8130870 msgr2=0x7f5fd813a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdd256700 1 --2- 172.21.15.89:0/230092284 >> v2:172.21.15.89:3300/0 conn(0x7f5fd8130870 0x7f5fd813a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdd256700 1 -- 172.21.15.89:0/230092284 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8133860 con 0x7f5fd8133a80 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdca55700 1 --2- 172.21.15.89:0/230092284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd812ff20 0x7f5fd8130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.758+0000 7f5fdd256700 1 --2- 172.21.15.89:0/230092284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd8133a80 0x7f5fd8133e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009840 tx=0x7f5fd4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a104f2be13ac31b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fcffff700 1 -- 172.21.15.89:0/230092284 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4011830 con 0x7f5fd8133a80 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fcffff700 1 -- 172.21.15.89:0/230092284 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd4011990 con 0x7f5fd8133a80 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fcffff700 1 -- 172.21.15.89:0/230092284 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4011d20 con 0x7f5fd8133a80 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/230092284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd8133a80 msgr2=0x7f5fd8133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 --2- 172.21.15.89:0/230092284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd8133a80 0x7f5fd8133e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009840 tx=0x7f5fd4009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:55.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/230092284 shutdown_connections 2023-09-30T22:17:55.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 --2- 172.21.15.89:0/230092284 >> v2:172.21.15.89:3300/0 conn(0x7f5fd8130870 0x7f5fd813a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 --2- 172.21.15.89:0/230092284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd8133a80 0x7f5fd8133e50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 --2- 172.21.15.89:0/230092284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd812ff20 0x7f5fd8130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/230092284 >> 172.21.15.89:0/230092284 conn(0x7f5fd80a4210 msgr2=0x7f5fd80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:55.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/230092284 shutdown_connections 2023-09-30T22:17:55.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.759+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/230092284 wait complete. 2023-09-30T22:17:55.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.760+0000 7f5fdf4ba700 1 Processor -- start 2023-09-30T22:17:55.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.760+0000 7f5fdf4ba700 1 -- start start 2023-09-30T22:17:55.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdf4ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd812ff20 0x7f5fd809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdf4ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd8130870 0x7f5fd809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdf4ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd8133a80 0x7f5fd809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdf4ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd812db50 con 0x7f5fd8133a80 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdf4ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8120060 con 0x7f5fd8130870 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdf4ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd809dee0 con 0x7f5fd812ff20 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd812ff20 0x7f5fd809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd812ff20 0x7f5fd809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39422/0 (socket says 172.21.15.89:39422) 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 -- 172.21.15.89:0/4026962301 learned_addr learned my addr 172.21.15.89:0/4026962301 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdca55700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd8130870 0x7f5fd809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdda57700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd8133a80 0x7f5fd809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 -- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd8130870 msgr2=0x7f5fd809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd8130870 0x7f5fd809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 -- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd8133a80 msgr2=0x7f5fd809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd8133a80 0x7f5fd809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 -- 172.21.15.89:0/4026962301 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc8009550 con 0x7f5fd812ff20 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fdd256700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd812ff20 0x7f5fd809c480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009b90 tx=0x7f5fd4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.761+0000 7f5fcdffb700 1 -- 172.21.15.89:0/4026962301 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4004450 con 0x7f5fd812ff20 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.762+0000 7f5fcdffb700 1 -- 172.21.15.89:0/4026962301 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd40045b0 con 0x7f5fd812ff20 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.762+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd40094a0 con 0x7f5fd812ff20 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.762+0000 7f5fcdffb700 1 -- 172.21.15.89:0/4026962301 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40038e0 con 0x7f5fd812ff20 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.762+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd812d2d0 con 0x7f5fd812ff20 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.762+0000 7f5fcdffb700 1 -- 172.21.15.89:0/4026962301 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5fd4003a40 con 0x7f5fd812ff20 2023-09-30T22:17:55.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.763+0000 7f5fcdffb700 1 -- 172.21.15.89:0/4026962301 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5fd40407a0 con 0x7f5fd812ff20 2023-09-30T22:17:55.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.763+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd812d750 con 0x7f5fd812ff20 2023-09-30T22:17:55.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.768+0000 7f5fcdffb700 1 -- 172.21.15.89:0/4026962301 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5fd402e970 con 0x7f5fd812ff20 2023-09-30T22:17:55.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.831+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f29fc1a9620 con 0x7f29fc09f650 2023-09-30T22:17:55.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.832+0000 7f29f8ff9700 1 -- 172.21.15.89:0/1990646330 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29fc1a9620 con 0x7f29fc09f650 2023-09-30T22:17:55.832 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:55.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f650 msgr2=0x7f29fc06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f650 0x7f29fc06c330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f29fc06f570 tx=0x7f29f4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 shutdown_connections 2023-09-30T22:17:55.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29fc09f650 0x7f29fc06c330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29fc09e930 0x7f29fc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 --2- 172.21.15.89:0/1990646330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29fc09f240 0x7f29fc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 >> 172.21.15.89:0/1990646330 conn(0x7f29fc0a4210 msgr2=0x7f29fc0a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:55.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 shutdown_connections 2023-09-30T22:17:55.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.834+0000 7f2a0286c700 1 -- 172.21.15.89:0/1990646330 wait complete. 2023-09-30T22:17:55.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:55.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.911+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe594072980 con 0x7fe5940a0a30 2023-09-30T22:17:55.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.912+0000 7fe589ffb700 1 -- 172.21.15.89:0/3511592971 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe584017020 con 0x7fe5940a0a30 2023-09-30T22:17:55.913 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:55.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5940a0a30 msgr2=0x7fe59411ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5940a0a30 0x7fe59411ee00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe58400ebe0 tx=0x7fe58400eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 shutdown_connections 2023-09-30T22:17:55.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe594133930 0x7fe59411f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe59409d630 0x7fe59411e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 --2- 172.21.15.89:0/3511592971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5940a0a30 0x7fe59411ee00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 >> 172.21.15.89:0/3511592971 conn(0x7fe5940a3050 msgr2=0x7fe5940a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:55.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 shutdown_connections 2023-09-30T22:17:55.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.914+0000 7fe59b27e700 1 -- 172.21.15.89:0/3511592971 wait complete. 2023-09-30T22:17:55.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:55.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.961+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5fd8074ea0 con 0x7f5fd812ff20 2023-09-30T22:17:55.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.961+0000 7f5fcdffb700 1 -- 172.21.15.89:0/4026962301 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fd4015030 con 0x7f5fd812ff20 2023-09-30T22:17:55.964 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:55.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd812ff20 msgr2=0x7f5fd809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:55.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd812ff20 0x7f5fd809c480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009b90 tx=0x7f5fd4004750 comp rx=0 tx=0).stop 2023-09-30T22:17:55.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 shutdown_connections 2023-09-30T22:17:55.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd8133a80 0x7f5fd809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5fd812ff20 0x7f5fd809c480 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 --2- 172.21.15.89:0/4026962301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fd8130870 0x7f5fd809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:55.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 >> 172.21.15.89:0/4026962301 conn(0x7f5fd80a4210 msgr2=0x7f5fd809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:55.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 shutdown_connections 2023-09-30T22:17:55.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:55.963+0000 7f5fdf4ba700 1 -- 172.21.15.89:0/4026962301 wait complete. 2023-09-30T22:17:55.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:56.232 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.356+0000 7f99af696700 1 Processor -- start 2023-09-30T22:17:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.356+0000 7f99af696700 1 -- start start 2023-09-30T22:17:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99af696700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a8138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99af696700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8139030 0x7f99a813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99af696700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a806ef30 con 0x7f99a81338a0 2023-09-30T22:17:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99af696700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a806fef0 con 0x7f99a8139030 2023-09-30T22:17:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99af696700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99a81334b0 con 0x7f99a8136710 2023-09-30T22:17:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99acc31700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a8138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99ad432700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f99a81338a0 0x7f99a8133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36150/0 (socket says 172.21.15.89:36150) 2023-09-30T22:17:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99adc33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8139030 0x7f99a813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99ad432700 1 -- 172.21.15.89:0/3519028519 learned_addr learned my addr 172.21.15.89:0/3519028519 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99acc31700 1 -- 172.21.15.89:0/3519028519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8139030 msgr2=0x7f99a813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99acc31700 1 --2- 172.21.15.89:0/3519028519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8139030 0x7f99a813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99acc31700 1 -- 172.21.15.89:0/3519028519 >> v1:172.21.15.89:6789/0 conn(0x7f99a81338a0 legacy=0x7f99a8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f999ffff700 1 -- 172.21.15.89:0/3519028519 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50027477 0 0) 0x7f99a806ef30 con 0x7f99a81338a0 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99acc31700 1 -- 172.21.15.89:0/3519028519 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a806fc30 con 0x7f99a8136710 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99adc33700 1 --2- 172.21.15.89:0/3519028519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8139030 0x7f99a813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f99acc31700 1 --2- 172.21.15.89:0/3519028519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a8138af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009870 tx=0x7f99a4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35eb5f1807b37f8b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f999ffff700 1 -- 172.21.15.89:0/3519028519 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4011830 con 0x7f99a8136710 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f999ffff700 1 -- 172.21.15.89:0/3519028519 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a4011990 con 0x7f99a8136710 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.357+0000 7f999ffff700 1 -- 172.21.15.89:0/3519028519 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4011d20 con 0x7f99a8136710 2023-09-30T22:17:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.358+0000 7f99af696700 1 -- 172.21.15.89:0/3519028519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 msgr2=0x7f99a8138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.358+0000 7f99af696700 1 --2- 172.21.15.89:0/3519028519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a8138af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009870 tx=0x7f99a4009b80 comp rx=0 tx=0).stop 2023-09-30T22:17:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.358+0000 7f99af696700 1 -- 172.21.15.89:0/3519028519 shutdown_connections 2023-09-30T22:17:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.358+0000 7f99af696700 1 --2- 172.21.15.89:0/3519028519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8139030 0x7f99a813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.358+0000 7f99af696700 1 --2- 172.21.15.89:0/3519028519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a8138af0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.358+0000 7f99af696700 1 -- 172.21.15.89:0/3519028519 >> 172.21.15.89:0/3519028519 conn(0x7f99a80a2aa0 msgr2=0x7f99a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.358+0000 7f99af696700 1 -- 172.21.15.89:0/3519028519 shutdown_connections 2023-09-30T22:17:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.358+0000 7f99af696700 1 -- 172.21.15.89:0/3519028519 wait complete. 2023-09-30T22:17:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99af696700 1 Processor -- start 2023-09-30T22:17:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99af696700 1 -- start start 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99af696700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a81338a0 0x7f99a811acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99af696700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a811b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99af696700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a8139030 0x7f99a811b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99af696700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a806f640 con 0x7f99a8139030 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99af696700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8138d90 con 0x7f99a81338a0 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99af696700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99a811c9a0 con 0x7f99a8136710 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99ad432700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a81338a0 0x7f99a811acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99adc33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a8139030 0x7f99a811b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99acc31700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a811b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99ad432700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a81338a0 0x7f99a811acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57550/0 (socket says 172.21.15.89:57550) 2023-09-30T22:17:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.359+0000 7f99ad432700 1 -- 172.21.15.89:0/4259589494 learned_addr learned my addr 172.21.15.89:0/4259589494 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99adc33700 1 -- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 msgr2=0x7f99a811b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99adc33700 1 --2- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a811b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99adc33700 1 -- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a81338a0 msgr2=0x7f99a811acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99adc33700 1 --2- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a81338a0 0x7f99a811acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99adc33700 1 -- 172.21.15.89:0/4259589494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0009a70 con 0x7f99a8139030 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99ad432700 1 --2- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a81338a0 0x7f99a811acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99adc33700 1 --2- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a8139030 0x7f99a811b770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f99a806fb00 tx=0x7f99a000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f999dffb700 1 -- 172.21.15.89:0/4259589494 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0003a20 con 0x7f99a8139030 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f999dffb700 1 -- 172.21.15.89:0/4259589494 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a000f040 con 0x7f99a8139030 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a40094a0 con 0x7f99a8139030 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f999dffb700 1 -- 172.21.15.89:0/4259589494 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0003dc0 con 0x7f99a8139030 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.360+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a811e530 con 0x7f99a8139030 2023-09-30T22:17:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.361+0000 7f999dffb700 1 -- 172.21.15.89:0/4259589494 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99a000ca20 con 0x7f99a8139030 2023-09-30T22:17:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.361+0000 7f999dffb700 1 -- 172.21.15.89:0/4259589494 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f99a003fe40 con 0x7f99a8139030 2023-09-30T22:17:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.361+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a811e180 con 0x7f99a8139030 2023-09-30T22:17:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.366+0000 7f999dffb700 1 -- 172.21.15.89:0/4259589494 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99a001cba0 con 0x7f99a8139030 2023-09-30T22:17:56.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.550+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f99a8074ea0 con 0x7f99a8139030 2023-09-30T22:17:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.550+0000 7f999dffb700 1 -- 172.21.15.89:0/4259589494 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99a002b090 con 0x7f99a8139030 2023-09-30T22:17:56.550 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a8139030 msgr2=0x7f99a811b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 --2- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a8139030 0x7f99a811b770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f99a806fb00 tx=0x7f99a000ef90 comp rx=0 tx=0).stop 2023-09-30T22:17:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 shutdown_connections 2023-09-30T22:17:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 --2- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a8139030 0x7f99a811b770 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 --2- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a81338a0 0x7f99a811acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 --2- 172.21.15.89:0/4259589494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a8136710 0x7f99a811b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 >> 172.21.15.89:0/4259589494 conn(0x7f99a80a2aa0 msgr2=0x7f99a80a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 shutdown_connections 2023-09-30T22:17:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.552+0000 7f99af696700 1 -- 172.21.15.89:0/4259589494 wait complete. 2023-09-30T22:17:56.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:56.847 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:56.927 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.971+0000 7fc3f86ce700 1 Processor -- start 2023-09-30T22:17:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.971+0000 7fc3f86ce700 1 -- start start 2023-09-30T22:17:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f86ce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3f009d6f0 0x7fc3f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f86ce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009e440 0x7fc3f0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f86ce700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f009b120 con 0x7fc3f009e000 2023-09-30T22:17:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f86ce700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f006b460 con 0x7fc3f009d6f0 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f86ce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f009a7e0 con 0x7fc3f009e440 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f646a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3f009d6f0 0x7fc3f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f6c6b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009e440 0x7fc3f0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f646a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3f009d6f0 0x7fc3f009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33146/0 (socket says 172.21.15.89:33146) 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f646a700 1 -- 172.21.15.89:0/4244335819 learned_addr learned my addr 172.21.15.89:0/4244335819 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f6c6b700 1 -- 172.21.15.89:0/4244335819 >> v2:172.21.15.89:3300/0 conn(0x7fc3f009d6f0 msgr2=0x7fc3f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f6c6b700 1 --2- 172.21.15.89:0/4244335819 >> v2:172.21.15.89:3300/0 conn(0x7fc3f009d6f0 0x7fc3f009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f6c6b700 1 -- 172.21.15.89:0/4244335819 >> v1:172.21.15.89:6789/0 conn(0x7fc3f009e000 legacy=0x7fc3f01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f6c6b700 1 -- 172.21.15.89:0/4244335819 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f009a500 con 0x7fc3f009e440 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f646a700 1 --2- 172.21.15.89:0/4244335819 >> v2:172.21.15.89:3300/0 conn(0x7fc3f009d6f0 0x7fc3f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.972+0000 7fc3f6c6b700 1 --2- 172.21.15.89:0/4244335819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009e440 0x7fc3f0134100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009dc0 tx=0x7fc3e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0cda776d829e22 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f5468700 1 -- 172.21.15.89:0/4244335819 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800e070 con 0x7fc3f009e440 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f5468700 1 -- 172.21.15.89:0/4244335819 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e800bae0 con 0x7fc3f009e440 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f5468700 1 -- 172.21.15.89:0/4244335819 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800be90 con 0x7fc3f009e440 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f86ce700 1 -- 172.21.15.89:0/4244335819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009e440 msgr2=0x7fc3f0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f86ce700 1 --2- 172.21.15.89:0/4244335819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009e440 0x7fc3f0134100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009dc0 tx=0x7fc3e800b330 comp rx=0 tx=0).stop 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f86ce700 1 -- 172.21.15.89:0/4244335819 shutdown_connections 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f86ce700 1 --2- 172.21.15.89:0/4244335819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009e440 0x7fc3f0134100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f86ce700 1 --2- 172.21.15.89:0/4244335819 >> v2:172.21.15.89:3300/0 conn(0x7fc3f009d6f0 0x7fc3f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f86ce700 1 -- 172.21.15.89:0/4244335819 >> 172.21.15.89:0/4244335819 conn(0x7fc3f0066f50 msgr2=0x7fc3f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f86ce700 1 -- 172.21.15.89:0/4244335819 shutdown_connections 2023-09-30T22:17:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.973+0000 7fc3f86ce700 1 -- 172.21.15.89:0/4244335819 wait complete. 2023-09-30T22:17:56.976 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:56.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.974+0000 7fc3f86ce700 1 Processor -- start 2023-09-30T22:17:56.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.974+0000 7fc3f86ce700 1 -- start start 2023-09-30T22:17:56.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f86ce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009d6f0 0x7fc3f01a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f86ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3f009e000 0x7fc3f01a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f86ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3f009e440 0x7fc3f01a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f86ce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f01aa360 con 0x7fc3f009e440 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f86ce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f01a99a0 con 0x7fc3f009d6f0 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f86ce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f01a8ce0 con 0x7fc3f009e000 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3f009e440 0x7fc3f01a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f646a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009d6f0 0x7fc3f01a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f5c69700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3f009e000 0x7fc3f01a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3f009e440 0x7fc3f01a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33148/0 (socket says 172.21.15.89:33148) 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 -- 172.21.15.89:0/3715528914 learned_addr learned my addr 172.21.15.89:0/3715528914 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:56.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 -- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3f009e000 msgr2=0x7fc3f01a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 --2- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3f009e000 0x7fc3f01a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 -- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009d6f0 msgr2=0x7fc3f01a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 --2- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009d6f0 0x7fc3f01a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 -- 172.21.15.89:0/3715528914 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec009550 con 0x7fc3f009e440 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f646a700 1 --2- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009d6f0 0x7fc3f01a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f6c6b700 1 --2- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3f009e440 0x7fc3f01a6950 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8014040 tx=0x7fc3e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3e6ffd700 1 -- 172.21.15.89:0/3715528914 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e80045d0 con 0x7fc3f009e440 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3e6ffd700 1 -- 172.21.15.89:0/3715528914 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8020070 con 0x7fc3f009e440 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e8009a70 con 0x7fc3f009e440 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.975+0000 7fc3e6ffd700 1 -- 172.21.15.89:0/3715528914 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8027480 con 0x7fc3f009e440 2023-09-30T22:17:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.976+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f01a7a90 con 0x7fc3f009e440 2023-09-30T22:17:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.976+0000 7fc3e6ffd700 1 -- 172.21.15.89:0/3715528914 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc3e8012430 con 0x7fc3f009e440 2023-09-30T22:17:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.976+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f01a9540 con 0x7fc3f009e440 2023-09-30T22:17:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.976+0000 7fc3e6ffd700 1 -- 172.21.15.89:0/3715528914 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc3e8022070 con 0x7fc3f009e440 2023-09-30T22:17:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:56.981+0000 7fc3e6ffd700 1 -- 172.21.15.89:0/3715528914 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3e801a970 con 0x7fc3f009e440 2023-09-30T22:17:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.056+0000 7ff0721a9700 1 Processor -- start 2023-09-30T22:17:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.056+0000 7ff0721a9700 1 -- start start 2023-09-30T22:17:57.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff0721a9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff0721a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c09d630 0x7ff06c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff0721a9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff06c133930 0x7ff06c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff0721a9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0a2a70 con 0x7ff06c133930 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff0721a9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0a01d0 con 0x7ff06c0a0a30 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff0721a9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c09e2b0 con 0x7ff06c09d630 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c09d630 0x7ff06c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c09d630 0x7ff06c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39466/0 (socket says 172.21.15.89:39466) 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06affd700 1 -- 172.21.15.89:0/2149306116 learned_addr learned my addr 172.21.15.89:0/2149306116 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06bfff700 1 --2- 172.21.15.89:0/2149306116 >> v2:172.21.15.89:3300/0 conn(0x7ff06c133930 0x7ff06c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06b7fe700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06bfff700 1 -- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c09d630 msgr2=0x7ff06c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06bfff700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c09d630 0x7ff06c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06bfff700 1 -- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c0a0a30 msgr2=0x7ff06c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06bfff700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06bfff700 1 -- 172.21.15.89:0/2149306116 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c09dc50 con 0x7ff06c133930 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.057+0000 7ff06b7fe700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff06bfff700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c133930 0x7ff06c133dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff064009d90 tx=0x7ff06400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537fc01f823f03da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff06a7fc700 1 -- 172.21.15.89:0/2149306116 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400e070 con 0x7ff06c133930 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff06a7fc700 1 -- 172.21.15.89:0/2149306116 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06400bb20 con 0x7ff06c133930 2023-09-30T22:17:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff06a7fc700 1 -- 172.21.15.89:0/2149306116 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064012430 con 0x7ff06c133930 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff0721a9700 1 -- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c133930 msgr2=0x7ff06c133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff0721a9700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c133930 0x7ff06c133dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff064009d90 tx=0x7ff06400b330 comp rx=0 tx=0).stop 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff0721a9700 1 -- 172.21.15.89:0/2149306116 shutdown_connections 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff0721a9700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c133930 0x7ff06c133dd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff0721a9700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff0721a9700 1 --2- 172.21.15.89:0/2149306116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c09d630 0x7ff06c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.058+0000 7ff0721a9700 1 -- 172.21.15.89:0/2149306116 >> 172.21.15.89:0/2149306116 conn(0x7ff06c0a3050 msgr2=0x7ff06c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.059+0000 7ff0721a9700 1 -- 172.21.15.89:0/2149306116 shutdown_connections 2023-09-30T22:17:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.059+0000 7ff0721a9700 1 -- 172.21.15.89:0/2149306116 wait complete. 2023-09-30T22:17:57.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.060+0000 7ff0721a9700 1 Processor -- start 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.060+0000 7ff0721a9700 1 -- start start 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff0721a9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c09d630 0x7ff06c11f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff0721a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c0a0a30 0x7ff06c11f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff0721a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c133930 0x7ff06c1c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff0721a9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c11ee70 con 0x7ff06c0a0a30 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff0721a9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c133290 con 0x7ff06c09d630 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff0721a9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c122140 con 0x7ff06c133930 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c09d630 0x7ff06c11f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c09d630 0x7ff06c11f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57588/0 (socket says 172.21.15.89:57588) 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06b7fe700 1 -- 172.21.15.89:0/1642234051 learned_addr learned my addr 172.21.15.89:0/1642234051 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06affd700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c0a0a30 0x7ff06c11f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06bfff700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c133930 0x7ff06c1c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06affd700 1 -- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c133930 msgr2=0x7ff06c1c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06affd700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c133930 0x7ff06c1c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06affd700 1 -- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c09d630 msgr2=0x7ff06c11f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06affd700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c09d630 0x7ff06c11f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06affd700 1 -- 172.21.15.89:0/1642234051 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009a70 con 0x7ff06c0a0a30 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06b7fe700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c09d630 0x7ff06c11f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.061+0000 7ff06affd700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c0a0a30 0x7ff06c11f760 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff06000e8b0 tx=0x7ff06000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.062+0000 7ff053fff700 1 -- 172.21.15.89:0/1642234051 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060013680 con 0x7ff06c0a0a30 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.062+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c120510 con 0x7ff06c0a0a30 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.062+0000 7ff053fff700 1 -- 172.21.15.89:0/1642234051 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0600137e0 con 0x7ff06c0a0a30 2023-09-30T22:17:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.062+0000 7ff053fff700 1 -- 172.21.15.89:0/1642234051 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060013b50 con 0x7ff06c0a0a30 2023-09-30T22:17:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.062+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c120d00 con 0x7ff06c0a0a30 2023-09-30T22:17:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.063+0000 7ff053fff700 1 -- 172.21.15.89:0/1642234051 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff060013cb0 con 0x7ff06c0a0a30 2023-09-30T22:17:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.063+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c0747f0 con 0x7ff06c0a0a30 2023-09-30T22:17:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.063+0000 7ff053fff700 1 -- 172.21.15.89:0/1642234051 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff06003f620 con 0x7ff06c0a0a30 2023-09-30T22:17:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.072+0000 7ff053fff700 1 -- 172.21.15.89:0/1642234051 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff060021020 con 0x7ff06c0a0a30 2023-09-30T22:17:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.109+0000 7f518e6a5700 1 Processor -- start 2023-09-30T22:17:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.109+0000 7f518e6a5700 1 -- start start 2023-09-30T22:17:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518e6a5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809d6f0 0x7f518809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518e6a5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f518809e000 0x7f5188130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518e6a5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809e410 0x7f5188130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518e6a5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518809f520 con 0x7f518809e000 2023-09-30T22:17:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518e6a5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518809f130 con 0x7f518809d6f0 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518e6a5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f518809e930 con 0x7f518809e410 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f5187fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809d6f0 0x7f518809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809e410 0x7f5188130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809e410 0x7f5188130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39490/0 (socket says 172.21.15.89:39490) 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 -- 172.21.15.89:0/4067551030 learned_addr learned my addr 172.21.15.89:0/4067551030 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f5187fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809d6f0 0x7f518809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57602/0 (socket says 172.21.15.89:57602) 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f51877fe700 1 --2- 172.21.15.89:0/4067551030 >> v2:172.21.15.89:3300/0 conn(0x7f518809e000 0x7f5188130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 -- 172.21.15.89:0/4067551030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809d6f0 msgr2=0x7f518809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 --2- 172.21.15.89:0/4067551030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809d6f0 0x7f518809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 -- 172.21.15.89:0/4067551030 >> v2:172.21.15.89:3300/0 conn(0x7f518809e000 msgr2=0x7f5188130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 --2- 172.21.15.89:0/4067551030 >> v2:172.21.15.89:3300/0 conn(0x7f518809e000 0x7f5188130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 -- 172.21.15.89:0/4067551030 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518809b2d0 con 0x7f518809e410 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.110+0000 7f518cc42700 1 --2- 172.21.15.89:0/4067551030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809e410 0x7f5188130ba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5180009e80 tx=0x7f518000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbfe74030ceddfe4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f5186ffd700 1 -- 172.21.15.89:0/4067551030 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518000e070 con 0x7f518809e410 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f5186ffd700 1 -- 172.21.15.89:0/4067551030 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518000b8d0 con 0x7f518809e410 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f5186ffd700 1 -- 172.21.15.89:0/4067551030 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518000bc80 con 0x7f518809e410 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 -- 172.21.15.89:0/4067551030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809e410 msgr2=0x7f5188130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 --2- 172.21.15.89:0/4067551030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809e410 0x7f5188130ba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5180009e80 tx=0x7f518000b330 comp rx=0 tx=0).stop 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 -- 172.21.15.89:0/4067551030 shutdown_connections 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 --2- 172.21.15.89:0/4067551030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809e410 0x7f5188130ba0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 --2- 172.21.15.89:0/4067551030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809d6f0 0x7f518809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 --2- 172.21.15.89:0/4067551030 >> v2:172.21.15.89:3300/0 conn(0x7f518809e000 0x7f5188130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 -- 172.21.15.89:0/4067551030 >> 172.21.15.89:0/4067551030 conn(0x7f5188066f50 msgr2=0x7f518809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:57.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 -- 172.21.15.89:0/4067551030 shutdown_connections 2023-09-30T22:17:57.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.111+0000 7f518e6a5700 1 -- 172.21.15.89:0/4067551030 wait complete. 2023-09-30T22:17:57.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.112+0000 7f518e6a5700 1 Processor -- start 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f518e6a5700 1 -- start start 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f518e6a5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809d6f0 0x7f518811dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f518e6a5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f518809e000 0x7f518811e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f518e6a5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809e410 0x7f518811e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f518e6a5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518811f220 con 0x7f518809e000 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f518e6a5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518811f980 con 0x7f518809e410 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f518e6a5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5188120bd0 con 0x7f518809d6f0 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f51877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f518809e000 0x7f518811e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f51877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f518809e000 0x7f518811e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33170/0 (socket says 172.21.15.89:33170) 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f5187fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809d6f0 0x7f518811dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f51877fe700 1 -- 172.21.15.89:0/1123234221 learned_addr learned my addr 172.21.15.89:0/1123234221 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f518cc42700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809e410 0x7f518811e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f5187fff700 1 -- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809e410 msgr2=0x7f518811e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f5187fff700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809e410 0x7f518811e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f5187fff700 1 -- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f518809e000 msgr2=0x7f518811e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f5187fff700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f518809e000 0x7f518811e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.113+0000 7f5187fff700 1 -- 172.21.15.89:0/1123234221 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5180009b30 con 0x7f518809d6f0 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.114+0000 7f51877fe700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f518809e000 0x7f518811e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.114+0000 7f5187fff700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809d6f0 0x7f518811dd70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5178010e20 tx=0x7f517800e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.114+0000 7f5184ff9700 1 -- 172.21.15.89:0/1123234221 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178011450 con 0x7f518809d6f0 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.114+0000 7f5184ff9700 1 -- 172.21.15.89:0/1123234221 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51780115b0 con 0x7f518809d6f0 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.114+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51881215b0 con 0x7f518809d6f0 2023-09-30T22:17:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.114+0000 7f5184ff9700 1 -- 172.21.15.89:0/1123234221 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178011920 con 0x7f518809d6f0 2023-09-30T22:17:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.114+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518809c700 con 0x7f518809d6f0 2023-09-30T22:17:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.115+0000 7f5184ff9700 1 -- 172.21.15.89:0/1123234221 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5178011a80 con 0x7f518809d6f0 2023-09-30T22:17:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.115+0000 7f5184ff9700 1 -- 172.21.15.89:0/1123234221 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5178042170 con 0x7f518809d6f0 2023-09-30T22:17:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.115+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518811ac20 con 0x7f518809d6f0 2023-09-30T22:17:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.120+0000 7f5184ff9700 1 -- 172.21.15.89:0/1123234221 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5178017b90 con 0x7f518809d6f0 2023-09-30T22:17:57.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.186+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc3f0074ea0 con 0x7fc3f009e440 2023-09-30T22:17:57.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.186+0000 7fc3e6ffd700 1 -- 172.21.15.89:0/3715528914 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc3e801a380 con 0x7fc3f009e440 2023-09-30T22:17:57.186 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:57.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3f009e440 msgr2=0x7fc3f01a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 --2- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3f009e440 0x7fc3f01a6950 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8014040 tx=0x7fc3e8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 shutdown_connections 2023-09-30T22:17:57.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 --2- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3f009e440 0x7fc3f01a6950 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 --2- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3f009d6f0 0x7fc3f01a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 --2- 172.21.15.89:0/3715528914 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3f009e000 0x7fc3f01a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 >> 172.21.15.89:0/3715528914 conn(0x7fc3f0066f50 msgr2=0x7fc3f01baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 shutdown_connections 2023-09-30T22:17:57.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.188+0000 7fc3f86ce700 1 -- 172.21.15.89:0/3715528914 wait complete. 2023-09-30T22:17:57.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.266+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff06c072980 con 0x7ff06c0a0a30 2023-09-30T22:17:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.266+0000 7ff053fff700 1 -- 172.21.15.89:0/1642234051 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff06001be10 con 0x7ff06c0a0a30 2023-09-30T22:17:57.266 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:57.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.268+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c0a0a30 msgr2=0x7ff06c11f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.268+0000 7ff0721a9700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c0a0a30 0x7ff06c11f760 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff06000e8b0 tx=0x7ff06000ebc0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.268+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 shutdown_connections 2023-09-30T22:17:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.268+0000 7ff0721a9700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff06c133930 0x7ff06c1c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.268+0000 7ff0721a9700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff06c09d630 0x7ff06c11f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.268+0000 7ff0721a9700 1 --2- 172.21.15.89:0/1642234051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff06c0a0a30 0x7ff06c11f760 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.268+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 >> 172.21.15.89:0/1642234051 conn(0x7ff06c0a3050 msgr2=0x7ff06c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.269+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 shutdown_connections 2023-09-30T22:17:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.269+0000 7ff0721a9700 1 -- 172.21.15.89:0/1642234051 wait complete. 2023-09-30T22:17:57.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.313+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5188074ea0 con 0x7f518809d6f0 2023-09-30T22:17:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.313+0000 7f5184ff9700 1 -- 172.21.15.89:0/1123234221 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51780177b0 con 0x7f518809d6f0 2023-09-30T22:17:57.313 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.315+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809d6f0 msgr2=0x7f518811dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.315+0000 7f518e6a5700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809d6f0 0x7f518811dd70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5178010e20 tx=0x7f517800e640 comp rx=0 tx=0).stop 2023-09-30T22:17:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.315+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 shutdown_connections 2023-09-30T22:17:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.315+0000 7f518e6a5700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f518809e410 0x7f518811e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.315+0000 7f518e6a5700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f518809d6f0 0x7f518811dd70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.315+0000 7f518e6a5700 1 --2- 172.21.15.89:0/1123234221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f518809e000 0x7f518811e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.315+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 >> 172.21.15.89:0/1123234221 conn(0x7f5188066f50 msgr2=0x7f5188130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.316+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 shutdown_connections 2023-09-30T22:17:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.316+0000 7f518e6a5700 1 -- 172.21.15.89:0/1123234221 wait complete. 2023-09-30T22:17:57.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:57.565 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff622db700 1 Processor -- start 2023-09-30T22:17:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff622db700 1 -- start start 2023-09-30T22:17:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff622db700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff622db700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7eff5c09e000 0x7eff5c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff622db700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e410 0x7eff5c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff622db700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c06b460 con 0x7eff5c09e000 2023-09-30T22:17:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff622db700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c09b120 con 0x7eff5c09d6f0 2023-09-30T22:17:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff622db700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c09a7e0 con 0x7eff5c09e410 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff5b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7eff5c09e000 0x7eff5c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.689+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57620/0 (socket says 172.21.15.89:57620) 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7eff5c09e000 0x7eff5c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33182/0 (socket says 172.21.15.89:33182) 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5bfff700 1 -- 172.21.15.89:0/1054459204 learned_addr learned my addr 172.21.15.89:0/1054459204 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff60878700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e410 0x7eff5c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5b7fe700 1 -- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e410 msgr2=0x7eff5c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5b7fe700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e410 0x7eff5c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5b7fe700 1 -- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 msgr2=0x7eff5c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5b7fe700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5b7fe700 1 -- 172.21.15.89:0/1054459204 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c09a500 con 0x7eff5c09e000 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5bfff700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5b7fe700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e000 0x7eff5c1339f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009840 tx=0x7eff4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898cc0674926ab68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5affd700 1 -- 172.21.15.89:0/1054459204 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c011830 con 0x7eff5c09e000 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5affd700 1 -- 172.21.15.89:0/1054459204 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c011990 con 0x7eff5c09e000 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.690+0000 7eff5affd700 1 -- 172.21.15.89:0/1054459204 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c011d20 con 0x7eff5c09e000 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 -- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e000 msgr2=0x7eff5c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e000 0x7eff5c1339f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009840 tx=0x7eff4c009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 -- 172.21.15.89:0/1054459204 shutdown_connections 2023-09-30T22:17:57.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e410 0x7eff5c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 --2- 172.21.15.89:0/1054459204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e000 0x7eff5c1339f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 -- 172.21.15.89:0/1054459204 >> 172.21.15.89:0/1054459204 conn(0x7eff5c066f50 msgr2=0x7eff5c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 -- 172.21.15.89:0/1054459204 shutdown_connections 2023-09-30T22:17:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.691+0000 7eff622db700 1 -- 172.21.15.89:0/1054459204 wait complete. 2023-09-30T22:17:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff622db700 1 Processor -- start 2023-09-30T22:17:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff622db700 1 -- start start 2023-09-30T22:17:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff622db700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c1a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff622db700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e000 0x7eff5c1a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff622db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e410 0x7eff5c1a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff622db700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c1a9cf0 con 0x7eff5c09e410 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff622db700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c0a1fe0 con 0x7eff5c09d6f0 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff622db700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c09c710 con 0x7eff5c09e000 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff5b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e000 0x7eff5c1a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff60878700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e410 0x7eff5c1a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c1a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff5b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e000 0x7eff5c1a6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39516/0 (socket says 172.21.15.89:39516) 2023-09-30T22:17:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.692+0000 7eff5b7fe700 1 -- 172.21.15.89:0/3320326989 learned_addr learned my addr 172.21.15.89:0/3320326989 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff5b7fe700 1 -- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 msgr2=0x7eff5c1a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff5b7fe700 1 --2- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c1a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff5b7fe700 1 -- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e410 msgr2=0x7eff5c1a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff5b7fe700 1 --2- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e410 0x7eff5c1a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff5b7fe700 1 -- 172.21.15.89:0/3320326989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff50009550 con 0x7eff5c09e000 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff60878700 1 --2- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e410 0x7eff5c1a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff5b7fe700 1 --2- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e000 0x7eff5c1a6710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009fd0 tx=0x7eff4c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff58ff9700 1 -- 172.21.15.89:0/3320326989 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c004450 con 0x7eff5c09e000 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff58ff9700 1 -- 172.21.15.89:0/3320326989 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c01f070 con 0x7eff5c09e000 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c0094a0 con 0x7eff5c09e000 2023-09-30T22:17:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff58ff9700 1 -- 172.21.15.89:0/3320326989 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0038e0 con 0x7eff5c09e000 2023-09-30T22:17:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.693+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c1a9480 con 0x7eff5c09e000 2023-09-30T22:17:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.694+0000 7eff58ff9700 1 -- 172.21.15.89:0/3320326989 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff4c003a40 con 0x7eff5c09e000 2023-09-30T22:17:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.694+0000 7eff58ff9700 1 -- 172.21.15.89:0/3320326989 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7eff4c021070 con 0x7eff5c09e000 2023-09-30T22:17:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.694+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c1a83f0 con 0x7eff5c09e000 2023-09-30T22:17:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.699+0000 7eff58ff9700 1 -- 172.21.15.89:0/3320326989 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff5c1a83f0 con 0x7eff5c09e000 2023-09-30T22:17:57.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.884+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7eff5c1a83f0 con 0x7eff5c09e000 2023-09-30T22:17:57.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.884+0000 7eff58ff9700 1 -- 172.21.15.89:0/3320326989 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff5c1a83f0 con 0x7eff5c09e000 2023-09-30T22:17:57.884 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e000 msgr2=0x7eff5c1a6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 --2- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e000 0x7eff5c1a6710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009fd0 tx=0x7eff4c004750 comp rx=0 tx=0).stop 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 shutdown_connections 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 --2- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff5c09e410 0x7eff5c1a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 --2- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff5c09d6f0 0x7eff5c1a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 --2- 172.21.15.89:0/3320326989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff5c09e000 0x7eff5c1a6710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 >> 172.21.15.89:0/3320326989 conn(0x7eff5c066f50 msgr2=0x7eff5c1bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 shutdown_connections 2023-09-30T22:17:57.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:57.886+0000 7eff622db700 1 -- 172.21.15.89:0/3320326989 wait complete. 2023-09-30T22:17:57.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:58.201 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:58.281 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:58.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.328+0000 7fbb5a1fc700 1 Processor -- start 2023-09-30T22:17:58.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.328+0000 7fbb5a1fc700 1 -- start start 2023-09-30T22:17:58.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb5a1fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb540a0a30 0x7fbb540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb5a1fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb5a1fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb54134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb5a1fc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5409edd0 con 0x7fbb540a0a30 2023-09-30T22:17:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb5a1fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5409e1f0 con 0x7fbb54133fa0 2023-09-30T22:17:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb5a1fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5409dc50 con 0x7fbb540a1340 2023-09-30T22:17:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb540a0a30 0x7fbb540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb53fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb54134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39532/0 (socket says 172.21.15.89:39532) 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb540a0a30 0x7fbb540a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33190/0 (socket says 172.21.15.89:33190) 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb52ffd700 1 -- 172.21.15.89:0/599188323 learned_addr learned my addr 172.21.15.89:0/599188323 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb52ffd700 1 -- 172.21.15.89:0/599188323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 msgr2=0x7fbb54134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb52ffd700 1 --2- 172.21.15.89:0/599188323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb54134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb52ffd700 1 -- 172.21.15.89:0/599188323 >> v2:172.21.15.89:3300/0 conn(0x7fbb540a0a30 msgr2=0x7fbb540a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb52ffd700 1 --2- 172.21.15.89:0/599188323 >> v2:172.21.15.89:3300/0 conn(0x7fbb540a0a30 0x7fbb540a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb52ffd700 1 -- 172.21.15.89:0/599188323 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540a1fe0 con 0x7fbb540a1340 2023-09-30T22:17:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb53fff700 1 --2- 172.21.15.89:0/599188323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb54134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.329+0000 7fbb537fe700 1 --2- 172.21.15.89:0/599188323 >> v2:172.21.15.89:3300/0 conn(0x7fbb540a0a30 0x7fbb540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.330+0000 7fbb52ffd700 1 --2- 172.21.15.89:0/599188323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54133a60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009920 tx=0x7fbb3c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91790cf57ea1c03a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.330+0000 7fbb527fc700 1 -- 172.21.15.89:0/599188323 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c011830 con 0x7fbb540a1340 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.330+0000 7fbb527fc700 1 -- 172.21.15.89:0/599188323 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c011990 con 0x7fbb540a1340 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.330+0000 7fbb527fc700 1 -- 172.21.15.89:0/599188323 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c011d20 con 0x7fbb540a1340 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/599188323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 msgr2=0x7fbb54133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 --2- 172.21.15.89:0/599188323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54133a60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009920 tx=0x7fbb3c009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/599188323 shutdown_connections 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 --2- 172.21.15.89:0/599188323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb54134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 --2- 172.21.15.89:0/599188323 >> v2:172.21.15.89:3300/0 conn(0x7fbb540a0a30 0x7fbb540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 --2- 172.21.15.89:0/599188323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54133a60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/599188323 >> 172.21.15.89:0/599188323 conn(0x7fbb540a6550 msgr2=0x7fbb540a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/599188323 shutdown_connections 2023-09-30T22:17:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.331+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/599188323 wait complete. 2023-09-30T22:17:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.332+0000 7fbb5a1fc700 1 Processor -- start 2023-09-30T22:17:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.332+0000 7fbb5a1fc700 1 -- start start 2023-09-30T22:17:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb5a1fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb540a0a30 0x7fbb54120650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb5a1fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54120b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb5a1fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb541210d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb5a1fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54122840 con 0x7fbb540a0a30 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb5a1fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54123900 con 0x7fbb54133fa0 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb5a1fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540a0220 con 0x7fbb540a1340 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb53fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb541210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb53fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb541210d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57644/0 (socket says 172.21.15.89:57644) 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb53fff700 1 -- 172.21.15.89:0/2362900573 learned_addr learned my addr 172.21.15.89:0/2362900573 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb537fe700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb540a0a30 0x7fbb54120650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.333+0000 7fbb52ffd700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54120b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb53fff700 1 -- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 msgr2=0x7fbb54120b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb53fff700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54120b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb53fff700 1 -- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb540a0a30 msgr2=0x7fbb54120650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb53fff700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb540a0a30 0x7fbb54120650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb53fff700 1 -- 172.21.15.89:0/2362900573 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb44009a70 con 0x7fbb54133fa0 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb53fff700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb541210d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00eb60 tx=0x7fbb4c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb4bfff700 1 -- 172.21.15.89:0/2362900573 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0097a0 con 0x7fbb54133fa0 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb4bfff700 1 -- 172.21.15.89:0/2362900573 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c009e60 con 0x7fbb54133fa0 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb4bfff700 1 -- 172.21.15.89:0/2362900573 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c004240 con 0x7fbb54133fa0 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c0094a0 con 0x7fbb54133fa0 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.334+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c009550 con 0x7fbb54133fa0 2023-09-30T22:17:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.335+0000 7fbb4bfff700 1 -- 172.21.15.89:0/2362900573 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbb4c0043a0 con 0x7fbb54133fa0 2023-09-30T22:17:58.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.335+0000 7fbb4bfff700 1 -- 172.21.15.89:0/2362900573 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbb4c03f3f0 con 0x7fbb54133fa0 2023-09-30T22:17:58.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.335+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb54074ea0 con 0x7fbb54133fa0 2023-09-30T22:17:58.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.341+0000 7fbb4bfff700 1 -- 172.21.15.89:0/2362900573 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb4c020070 con 0x7fbb54133fa0 2023-09-30T22:17:58.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.413+0000 7f0c0e405700 1 Processor -- start 2023-09-30T22:17:58.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.413+0000 7f0c0e405700 1 -- start start 2023-09-30T22:17:58.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c0e405700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c08131630 0x7f0c08131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c0e405700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0c08131f40 0x7f0c0806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c0e405700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c0806e9d0 0x7f0c0806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c0e405700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c08071a00 con 0x7f0c08131f40 2023-09-30T22:17:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c0e405700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c08131430 con 0x7f0c0806e9d0 2023-09-30T22:17:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c0e405700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0809a510 con 0x7f0c08131630 2023-09-30T22:17:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0c08131f40 0x7f0c0806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0c08131f40 0x7f0c0806e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33196/0 (socket says 172.21.15.89:33196) 2023-09-30T22:17:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 -- 172.21.15.89:0/2419464327 learned_addr learned my addr 172.21.15.89:0/2419464327 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c07fff700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c08131630 0x7f0c08131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c0c9a2700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c0806e9d0 0x7f0c0806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 -- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c08131630 msgr2=0x7f0c08131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c08131630 0x7f0c08131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 -- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c0806e9d0 msgr2=0x7f0c0806ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c0806e9d0 0x7f0c0806ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 -- 172.21.15.89:0/2419464327 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c080721c0 con 0x7f0c08131f40 2023-09-30T22:17:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c077fe700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 0x7f0c0806e490 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0098f0 tx=0x7f0bfc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5149e1de4e4e97 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c06ffd700 1 -- 172.21.15.89:0/2419464327 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc011830 con 0x7f0c08131f40 2023-09-30T22:17:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c06ffd700 1 -- 172.21.15.89:0/2419464327 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc011990 con 0x7f0c08131f40 2023-09-30T22:17:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.414+0000 7f0c06ffd700 1 -- 172.21.15.89:0/2419464327 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc011d20 con 0x7f0c08131f40 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 -- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 msgr2=0x7f0c0806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 0x7f0c0806e490 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0098f0 tx=0x7f0bfc009c00 comp rx=0 tx=0).stop 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 -- 172.21.15.89:0/2419464327 shutdown_connections 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c0806e9d0 0x7f0c0806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c08131630 0x7f0c08131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 --2- 172.21.15.89:0/2419464327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 0x7f0c0806e490 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 -- 172.21.15.89:0/2419464327 >> 172.21.15.89:0/2419464327 conn(0x7f0c080a2aa0 msgr2=0x7f0c080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 -- 172.21.15.89:0/2419464327 shutdown_connections 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.415+0000 7f0c0e405700 1 -- 172.21.15.89:0/2419464327 wait complete. 2023-09-30T22:17:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.416+0000 7f0c0e405700 1 Processor -- start 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c0e405700 1 -- start start 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c0e405700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c0806e9d0 0x7f0c0811b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c0e405700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c08131630 0x7f0c0811bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c0e405700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 0x7f0c0811c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c0e405700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0811d1c0 con 0x7f0c08131f40 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c0e405700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0811e410 con 0x7f0c08131630 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c0e405700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0811d7b0 con 0x7f0c0806e9d0 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c077fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c08131630 0x7f0c0811bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c077fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c08131630 0x7f0c0811bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57672/0 (socket says 172.21.15.89:57672) 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c077fe700 1 -- 172.21.15.89:0/2261272318 learned_addr learned my addr 172.21.15.89:0/2261272318 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c0c9a2700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 0x7f0c0811c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c07fff700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c0806e9d0 0x7f0c0811b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c077fe700 1 -- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c0806e9d0 msgr2=0x7f0c0811b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c077fe700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c0806e9d0 0x7f0c0811b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c077fe700 1 -- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 msgr2=0x7f0c0811c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c077fe700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 0x7f0c0811c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.417+0000 7f0c077fe700 1 -- 172.21.15.89:0/2261272318 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc009550 con 0x7f0c08131630 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.418+0000 7f0c0c9a2700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 0x7f0c0811c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.418+0000 7f0c077fe700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c08131630 0x7f0c0811bbd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc013000 tx=0x7f0bfc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.418+0000 7f0c04ff9700 1 -- 172.21.15.89:0/2261272318 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc004450 con 0x7f0c08131630 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.418+0000 7f0c04ff9700 1 -- 172.21.15.89:0/2261272318 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc0045d0 con 0x7f0c08131630 2023-09-30T22:17:58.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.418+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0811e860 con 0x7f0c08131630 2023-09-30T22:17:58.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.418+0000 7f0c04ff9700 1 -- 172.21.15.89:0/2261272318 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0038e0 con 0x7f0c08131630 2023-09-30T22:17:58.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.418+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0809f830 con 0x7f0c08131630 2023-09-30T22:17:58.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.419+0000 7f0c04ff9700 1 -- 172.21.15.89:0/2261272318 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0bfc003a40 con 0x7f0c08131630 2023-09-30T22:17:58.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.419+0000 7f0c04ff9700 1 -- 172.21.15.89:0/2261272318 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0bfc0406c0 con 0x7f0c08131630 2023-09-30T22:17:58.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.419+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c08074ea0 con 0x7f0c08131630 2023-09-30T22:17:58.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.427+0000 7f0c04ff9700 1 -- 172.21.15.89:0/2261272318 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0bfc01a420 con 0x7f0c08131630 2023-09-30T22:17:58.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.461+0000 7fc27698b700 1 Processor -- start 2023-09-30T22:17:58.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc27698b700 1 -- start start 2023-09-30T22:17:58.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc27698b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc270136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc27698b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc270136fe0 0x7fc2701393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc27698b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc270070940 con 0x7fc27009e930 2023-09-30T22:17:58.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc27698b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27006fd60 con 0x7fc27009f410 2023-09-30T22:17:58.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc27698b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc27006fa80 con 0x7fc270136fe0 2023-09-30T22:17:58.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc267fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc270136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc267fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc270136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57678/0 (socket says 172.21.15.89:57678) 2023-09-30T22:17:58.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc267fff700 1 -- 172.21.15.89:0/1101578075 learned_addr learned my addr 172.21.15.89:0/1101578075 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:58.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc26ffff700 1 --1- 172.21.15.89:0/1101578075 >> v1:172.21.15.89:6789/0 conn(0x7fc27009e930 0x7fc27009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36170/0 (socket says 172.21.15.89:36170) 2023-09-30T22:17:58.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc274f28700 1 --2- 172.21.15.89:0/1101578075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc270136fe0 0x7fc2701393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.462+0000 7fc267fff700 1 -- 172.21.15.89:0/1101578075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc270136fe0 msgr2=0x7fc2701393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc267fff700 1 --2- 172.21.15.89:0/1101578075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc270136fe0 0x7fc2701393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc267fff700 1 -- 172.21.15.89:0/1101578075 >> v1:172.21.15.89:6789/0 conn(0x7fc27009e930 legacy=0x7fc27009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc267fff700 1 -- 172.21.15.89:0/1101578075 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27009fb60 con 0x7fc27009f410 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc26f7fe700 1 -- 172.21.15.89:0/1101578075 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209944898 0 0) 0x7fc270070940 con 0x7fc27009e930 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc267fff700 1 --2- 172.21.15.89:0/1101578075 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc270136aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc258009920 tx=0x7fc258009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce820f703f83b4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc26f7fe700 1 -- 172.21.15.89:0/1101578075 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258011830 con 0x7fc27009f410 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc26f7fe700 1 -- 172.21.15.89:0/1101578075 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258011990 con 0x7fc27009f410 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc26f7fe700 1 -- 172.21.15.89:0/1101578075 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258011d20 con 0x7fc27009f410 2023-09-30T22:17:58.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc27698b700 1 -- 172.21.15.89:0/1101578075 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 msgr2=0x7fc270136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc27698b700 1 --2- 172.21.15.89:0/1101578075 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc270136aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc258009920 tx=0x7fc258009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.463+0000 7fc27698b700 1 -- 172.21.15.89:0/1101578075 shutdown_connections 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.464+0000 7fc27698b700 1 --2- 172.21.15.89:0/1101578075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc270136fe0 0x7fc2701393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.464+0000 7fc27698b700 1 --2- 172.21.15.89:0/1101578075 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc270136aa0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.464+0000 7fc27698b700 1 -- 172.21.15.89:0/1101578075 >> 172.21.15.89:0/1101578075 conn(0x7fc2700a4210 msgr2=0x7fc2700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.464+0000 7fc27698b700 1 -- 172.21.15.89:0/1101578075 shutdown_connections 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.464+0000 7fc27698b700 1 -- 172.21.15.89:0/1101578075 wait complete. 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.465+0000 7fc27698b700 1 Processor -- start 2023-09-30T22:17:58.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.465+0000 7fc27698b700 1 -- start start 2023-09-30T22:17:58.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc27698b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc27009e930 0x7fc27011b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc27698b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc27011b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc27698b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc270136fe0 0x7fc27011bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:58.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc27698b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27006d880 con 0x7fc270136fe0 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc27698b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc270071d20 con 0x7fc27009f410 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc27698b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc27011e600 con 0x7fc27009e930 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc27009e930 0x7fc27011b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc27009e930 0x7fc27011b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39574/0 (socket says 172.21.15.89:39574) 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 -- 172.21.15.89:0/4234195521 learned_addr learned my addr 172.21.15.89:0/4234195521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc274f28700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc270136fe0 0x7fc27011bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc267fff700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc27011b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 -- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 msgr2=0x7fc27011b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc27011b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 -- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc270136fe0 msgr2=0x7fc27011bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc270136fe0 0x7fc27011bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 -- 172.21.15.89:0/4234195521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc258009550 con 0x7fc27009e930 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc274f28700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc270136fe0 0x7fc27011bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26ffff700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc27009e930 0x7fc27011b350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c520 tx=0x7fc26000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26d7fa700 1 -- 172.21.15.89:0/4234195521 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260004c80 con 0x7fc27009e930 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26d7fa700 1 -- 172.21.15.89:0/4234195521 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260004e00 con 0x7fc27009e930 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27011eb90 con 0x7fc27009e930 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc26d7fa700 1 -- 172.21.15.89:0/4234195521 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260007310 con 0x7fc27009e930 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.466+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27011f600 con 0x7fc27009e930 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.467+0000 7fc26d7fa700 1 -- 172.21.15.89:0/4234195521 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc260007470 con 0x7fc27009e930 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.467+0000 7fc26d7fa700 1 -- 172.21.15.89:0/4234195521 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc26003f360 con 0x7fc27009e930 2023-09-30T22:17:58.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.467+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc270074ea0 con 0x7fc27009e930 2023-09-30T22:17:58.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.476+0000 7fc26d7fa700 1 -- 172.21.15.89:0/4234195521 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2600123b0 con 0x7fc27009e930 2023-09-30T22:17:58.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.544+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbb54122260 con 0x7fbb54133fa0 2023-09-30T22:17:58.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.545+0000 7fbb4bfff700 1 -- 172.21.15.89:0/2362900573 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb4c017030 con 0x7fbb54133fa0 2023-09-30T22:17:58.544 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:58.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 msgr2=0x7fbb541210d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb541210d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00eb60 tx=0x7fbb4c00ee70 comp rx=0 tx=0).stop 2023-09-30T22:17:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 shutdown_connections 2023-09-30T22:17:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb54133fa0 0x7fbb541210d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb540a0a30 0x7fbb54120650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 --2- 172.21.15.89:0/2362900573 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb540a1340 0x7fbb54120b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 >> 172.21.15.89:0/2362900573 conn(0x7fbb540a6550 msgr2=0x7fbb5409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 shutdown_connections 2023-09-30T22:17:58.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.547+0000 7fbb5a1fc700 1 -- 172.21.15.89:0/2362900573 wait complete. 2023-09-30T22:17:58.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:58.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.623+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0c0811eeb0 con 0x7f0c08131630 2023-09-30T22:17:58.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.623+0000 7f0c04ff9700 1 -- 172.21.15.89:0/2261272318 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0bfc028030 con 0x7f0c08131630 2023-09-30T22:17:58.623 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c08131630 msgr2=0x7f0c0811bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c08131630 0x7f0c0811bbd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc013000 tx=0x7f0bfc004750 comp rx=0 tx=0).stop 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 shutdown_connections 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c08131f40 0x7f0c0811c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0c0806e9d0 0x7f0c0811b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 --2- 172.21.15.89:0/2261272318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c08131630 0x7f0c0811bbd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 >> 172.21.15.89:0/2261272318 conn(0x7f0c080a2aa0 msgr2=0x7f0c081b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 shutdown_connections 2023-09-30T22:17:58.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.625+0000 7f0c0e405700 1 -- 172.21.15.89:0/2261272318 wait complete. 2023-09-30T22:17:58.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:17:58.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.669+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc27006b250 con 0x7fc27009e930 2023-09-30T22:17:58.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.670+0000 7fc26d7fa700 1 -- 172.21.15.89:0/4234195521 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc260011360 con 0x7fc27009e930 2023-09-30T22:17:58.669 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc27009e930 msgr2=0x7fc27011b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc27009e930 0x7fc27011b350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c520 tx=0x7fc26000c830 comp rx=0 tx=0).stop 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 shutdown_connections 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc270136fe0 0x7fc27011bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc27009e930 0x7fc27011b350 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 --2- 172.21.15.89:0/4234195521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc27009f410 0x7fc27011b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 >> 172.21.15.89:0/4234195521 conn(0x7fc2700a4210 msgr2=0x7fc2700a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 shutdown_connections 2023-09-30T22:17:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:58.672+0000 7fc27698b700 1 -- 172.21.15.89:0/4234195521 wait complete. 2023-09-30T22:17:58.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:17:58.899 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:17:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec299d1700 1 Processor -- start 2023-09-30T22:17:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec299d1700 1 -- start start 2023-09-30T22:17:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec299d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409ace0 0x7fec24138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec299d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2413b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec299d1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2409b150 con 0x7fec2409d6f0 2023-09-30T22:17:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec299d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2409a500 con 0x7fec2409ace0 2023-09-30T22:17:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec299d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec2409f720 con 0x7fec2409e1d0 2023-09-30T22:17:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec227fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409ace0 0x7fec24138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec237fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2413b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec22ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fec2409d6f0 0x7fec2409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36176/0 (socket says 172.21.15.89:36176) 2023-09-30T22:17:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.022+0000 7fec22ffd700 1 -- 172.21.15.89:0/3251077049 learned_addr learned my addr 172.21.15.89:0/3251077049 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec237fe700 1 -- 172.21.15.89:0/3251077049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409ace0 msgr2=0x7fec24138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec237fe700 1 --2- 172.21.15.89:0/3251077049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409ace0 0x7fec24138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec21ffb700 1 -- 172.21.15.89:0/3251077049 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360682354 0 0) 0x7fec2409b150 con 0x7fec2409d6f0 2023-09-30T22:17:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec237fe700 1 -- 172.21.15.89:0/3251077049 >> v1:172.21.15.89:6789/0 conn(0x7fec2409d6f0 legacy=0x7fec2409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec237fe700 1 -- 172.21.15.89:0/3251077049 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2406b490 con 0x7fec2409e1d0 2023-09-30T22:17:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec227fc700 1 --2- 172.21.15.89:0/3251077049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409ace0 0x7fec24138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec237fe700 1 --2- 172.21.15.89:0/3251077049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2413b180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec1c009dc0 tx=0x7fec1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f79c94ceb43cba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec21ffb700 1 -- 172.21.15.89:0/3251077049 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00e070 con 0x7fec2409e1d0 2023-09-30T22:17:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec21ffb700 1 -- 172.21.15.89:0/3251077049 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c00bb20 con 0x7fec2409e1d0 2023-09-30T22:17:59.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec21ffb700 1 -- 172.21.15.89:0/3251077049 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c012430 con 0x7fec2409e1d0 2023-09-30T22:17:59.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec299d1700 1 -- 172.21.15.89:0/3251077049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 msgr2=0x7fec2413b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec299d1700 1 --2- 172.21.15.89:0/3251077049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2413b180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec1c009dc0 tx=0x7fec1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:17:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec299d1700 1 -- 172.21.15.89:0/3251077049 shutdown_connections 2023-09-30T22:17:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec299d1700 1 --2- 172.21.15.89:0/3251077049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2413b180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec299d1700 1 --2- 172.21.15.89:0/3251077049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409ace0 0x7fec24138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.023+0000 7fec299d1700 1 -- 172.21.15.89:0/3251077049 >> 172.21.15.89:0/3251077049 conn(0x7fec24066f50 msgr2=0x7fec2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.024+0000 7fec299d1700 1 -- 172.21.15.89:0/3251077049 shutdown_connections 2023-09-30T22:17:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.024+0000 7fec299d1700 1 -- 172.21.15.89:0/3251077049 wait complete. 2023-09-30T22:17:59.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.024+0000 7fec299d1700 1 Processor -- start 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec299d1700 1 -- start start 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec299d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec2409ace0 0x7fec2411d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec299d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409d6f0 0x7fec2411ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec299d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2411e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec299d1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240a2990 con 0x7fec2409ace0 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec299d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2411f890 con 0x7fec2409d6f0 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec299d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec2413ad80 con 0x7fec2409e1d0 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec227fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409d6f0 0x7fec2411ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec22ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec2409ace0 0x7fec2411d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec227fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409d6f0 0x7fec2411ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57712/0 (socket says 172.21.15.89:57712) 2023-09-30T22:17:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec227fc700 1 -- 172.21.15.89:0/2149282547 learned_addr learned my addr 172.21.15.89:0/2149282547 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec237fe700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2411e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec227fc700 1 -- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 msgr2=0x7fec2411e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec227fc700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2411e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec227fc700 1 -- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec2409ace0 msgr2=0x7fec2411d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec227fc700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec2409ace0 0x7fec2411d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec227fc700 1 -- 172.21.15.89:0/2149282547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec180094a0 con 0x7fec2409d6f0 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec22ffd700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec2409ace0 0x7fec2411d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.025+0000 7fec237fe700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2411e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.026+0000 7fec227fc700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409d6f0 0x7fec2411ddd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fec1800ebe0 tx=0x7fec1800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.026+0000 7fec0bfff700 1 -- 172.21.15.89:0/2149282547 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800c850 con 0x7fec2409d6f0 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.026+0000 7fec0bfff700 1 -- 172.21.15.89:0/2149282547 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1800c9b0 con 0x7fec2409d6f0 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.026+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c009a70 con 0x7fec2409d6f0 2023-09-30T22:17:59.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.026+0000 7fec0bfff700 1 -- 172.21.15.89:0/2149282547 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800cd40 con 0x7fec2409d6f0 2023-09-30T22:17:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.026+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2413a430 con 0x7fec2409d6f0 2023-09-30T22:17:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.027+0000 7fec0bfff700 1 -- 172.21.15.89:0/2149282547 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fec18013470 con 0x7fec2409d6f0 2023-09-30T22:17:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.027+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec240747f0 con 0x7fec2409d6f0 2023-09-30T22:17:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.027+0000 7fec0bfff700 1 -- 172.21.15.89:0/2149282547 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fec1803fac0 con 0x7fec2409d6f0 2023-09-30T22:17:59.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.031+0000 7fec0bfff700 1 -- 172.21.15.89:0/2149282547 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec18021020 con 0x7fec2409d6f0 2023-09-30T22:17:59.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.213+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fec2411f9d0 con 0x7fec2409d6f0 2023-09-30T22:17:59.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.214+0000 7fec0bfff700 1 -- 172.21.15.89:0/2149282547 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec18017070 con 0x7fec2409d6f0 2023-09-30T22:17:59.214 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:59.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409d6f0 msgr2=0x7fec2411ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409d6f0 0x7fec2411ddd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fec1800ebe0 tx=0x7fec1800eef0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 shutdown_connections 2023-09-30T22:17:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec2409e1d0 0x7fec2411e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec2409ace0 0x7fec2411d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 --2- 172.21.15.89:0/2149282547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec2409d6f0 0x7fec2411ddd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 >> 172.21.15.89:0/2149282547 conn(0x7fec24066f50 msgr2=0x7fec240a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 shutdown_connections 2023-09-30T22:17:59.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.216+0000 7fec299d1700 1 -- 172.21.15.89:0/2149282547 wait complete. 2023-09-30T22:17:59.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:17:59.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:17:59.638 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:17:59.684 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:17:59.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.687+0000 7ff5d761a700 1 Processor -- start 2023-09-30T22:17:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.687+0000 7ff5d761a700 1 -- start start 2023-09-30T22:17:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d761a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009e1d0 0x7ff5d0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d761a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d0139170 0x7ff5d013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d761a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d009b120 con 0x7ff5d009d6f0 2023-09-30T22:17:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d761a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d006b460 con 0x7ff5d0139170 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d761a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d009a7e0 con 0x7ff5d009e1d0 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d53b6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff5d009d6f0 0x7ff5d009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36180/0 (socket says 172.21.15.89:36180) 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d53b6700 1 -- 172.21.15.89:0/1744527228 learned_addr learned my addr 172.21.15.89:0/1744527228 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d4bb5700 1 --2- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009e1d0 0x7ff5d0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d5bb7700 1 --2- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d0139170 0x7ff5d013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5c7fff700 1 -- 172.21.15.89:0/1744527228 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125412710 0 0) 0x7ff5d009b120 con 0x7ff5d009d6f0 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5c7fff700 1 -- 172.21.15.89:0/1744527228 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5bc0035b0 con 0x7ff5d009d6f0 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d4bb5700 1 -- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d0139170 msgr2=0x7ff5d013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d4bb5700 1 --2- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d0139170 0x7ff5d013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d4bb5700 1 -- 172.21.15.89:0/1744527228 >> v1:172.21.15.89:6789/0 conn(0x7ff5d009d6f0 legacy=0x7ff5d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.688+0000 7ff5d4bb5700 1 -- 172.21.15.89:0/1744527228 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d009a500 con 0x7ff5d009e1d0 2023-09-30T22:17:59.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5d4bb5700 1 --2- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009e1d0 0x7ff5d0138c30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00098a0 tx=0x7ff5c0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=366ede43f7c9c6ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:59.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5c7fff700 1 -- 172.21.15.89:0/1744527228 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0004230 con 0x7ff5d009e1d0 2023-09-30T22:17:59.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5c7fff700 1 -- 172.21.15.89:0/1744527228 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c0004390 con 0x7ff5d009e1d0 2023-09-30T22:17:59.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5c7fff700 1 -- 172.21.15.89:0/1744527228 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c00036a0 con 0x7ff5d009e1d0 2023-09-30T22:17:59.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5d761a700 1 -- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009e1d0 msgr2=0x7ff5d0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5d761a700 1 --2- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009e1d0 0x7ff5d0138c30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00098a0 tx=0x7ff5c0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5d761a700 1 -- 172.21.15.89:0/1744527228 shutdown_connections 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5d761a700 1 --2- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d0139170 0x7ff5d013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5d761a700 1 --2- 172.21.15.89:0/1744527228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009e1d0 0x7ff5d0138c30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.689+0000 7ff5d761a700 1 -- 172.21.15.89:0/1744527228 >> 172.21.15.89:0/1744527228 conn(0x7ff5d0066f50 msgr2=0x7ff5d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.690+0000 7ff5d761a700 1 -- 172.21.15.89:0/1744527228 shutdown_connections 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.690+0000 7ff5d761a700 1 -- 172.21.15.89:0/1744527228 wait complete. 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.691+0000 7ff5d761a700 1 Processor -- start 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.691+0000 7ff5d761a700 1 -- start start 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d761a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009d6f0 0x7ff5d01206d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d761a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d009e1d0 0x7ff5d0120c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d761a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d0139170 0x7ff5d0121150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d761a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d01239f0 con 0x7ff5d0139170 2023-09-30T22:17:59.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d761a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d0135fe0 con 0x7ff5d009e1d0 2023-09-30T22:17:59.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d761a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d009c710 con 0x7ff5d009d6f0 2023-09-30T22:17:59.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d009e1d0 0x7ff5d0120c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d53b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009d6f0 0x7ff5d01206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d009e1d0 0x7ff5d0120c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57744/0 (socket says 172.21.15.89:57744) 2023-09-30T22:17:59.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d53b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009d6f0 0x7ff5d01206d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39614/0 (socket says 172.21.15.89:39614) 2023-09-30T22:17:59.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 -- 172.21.15.89:0/3554656762 learned_addr learned my addr 172.21.15.89:0/3554656762 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:59.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d5bb7700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d0139170 0x7ff5d0121150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 -- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009d6f0 msgr2=0x7ff5d01206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009d6f0 0x7ff5d01206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 -- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d0139170 msgr2=0x7ff5d0121150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d0139170 0x7ff5d0121150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 -- 172.21.15.89:0/3554656762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00094a0 con 0x7ff5d009e1d0 2023-09-30T22:17:59.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d53b6700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009d6f0 0x7ff5d01206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:59.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d5bb7700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d0139170 0x7ff5d0121150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.692+0000 7ff5d4bb5700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d009e1d0 0x7ff5d0120c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009870 tx=0x7ff5c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.693+0000 7ff5c5ffb700 1 -- 172.21.15.89:0/3554656762 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0003c70 con 0x7ff5d009e1d0 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.693+0000 7ff5c5ffb700 1 -- 172.21.15.89:0/3554656762 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c0026070 con 0x7ff5d009e1d0 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.693+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d0123f10 con 0x7ff5d009e1d0 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.693+0000 7ff5c5ffb700 1 -- 172.21.15.89:0/3554656762 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0011630 con 0x7ff5d009e1d0 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.693+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d0124980 con 0x7ff5d009e1d0 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.694+0000 7ff5c5ffb700 1 -- 172.21.15.89:0/3554656762 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff5c0011790 con 0x7ff5d009e1d0 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.694+0000 7ff5c5ffb700 1 -- 172.21.15.89:0/3554656762 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff5c00491c0 con 0x7ff5d009e1d0 2023-09-30T22:17:59.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.694+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d0074ea0 con 0x7ff5d009e1d0 2023-09-30T22:17:59.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.699+0000 7ff5c5ffb700 1 -- 172.21.15.89:0/3554656762 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff5c0018940 con 0x7ff5d009e1d0 2023-09-30T22:17:59.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.773+0000 7f76e2aa8700 1 Processor -- start 2023-09-30T22:17:59.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.773+0000 7f76e2aa8700 1 -- start start 2023-09-30T22:17:59.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76e2aa8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc09d9f0 0x7f76dc136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76e2aa8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f76dc136a70 0x7f76dc138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76e2aa8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76dc09e130 con 0x7f76dc1316c0 2023-09-30T22:17:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76e2aa8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc1321a0 con 0x7f76dc136a70 2023-09-30T22:17:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76e2aa8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc1312b0 con 0x7f76dc09d9f0 2023-09-30T22:17:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc09d9f0 0x7f76dc136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76e0844700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f76dc1316c0 0x7f76dc131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36182/0 (socket says 172.21.15.89:36182) 2023-09-30T22:17:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76e0844700 1 -- 172.21.15.89:0/1698281119 learned_addr learned my addr 172.21.15.89:0/1698281119 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:17:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76e1045700 1 --2- 172.21.15.89:0/1698281119 >> v2:172.21.15.89:3300/0 conn(0x7f76dc136a70 0x7f76dc138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76dbfff700 1 -- 172.21.15.89:0/1698281119 >> v2:172.21.15.89:3300/0 conn(0x7f76dc136a70 msgr2=0x7f76dc138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76dbfff700 1 --2- 172.21.15.89:0/1698281119 >> v2:172.21.15.89:3300/0 conn(0x7f76dc136a70 0x7f76dc138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76dbfff700 1 -- 172.21.15.89:0/1698281119 >> v1:172.21.15.89:6789/0 conn(0x7f76dc1316c0 legacy=0x7f76dc131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.774+0000 7f76dbfff700 1 -- 172.21.15.89:0/1698281119 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc09f6e0 con 0x7f76dc09d9f0 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76dbfff700 1 --2- 172.21.15.89:0/1698281119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc09d9f0 0x7f76dc136530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009840 tx=0x7f76c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=366bf95b7bd0a212 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76db7fe700 1 -- 172.21.15.89:0/1698281119 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8004230 con 0x7f76dc09d9f0 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76db7fe700 1 -- 172.21.15.89:0/1698281119 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c8004390 con 0x7f76dc09d9f0 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76db7fe700 1 -- 172.21.15.89:0/1698281119 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c80036a0 con 0x7f76dc09d9f0 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76e2aa8700 1 -- 172.21.15.89:0/1698281119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc09d9f0 msgr2=0x7f76dc136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76e2aa8700 1 --2- 172.21.15.89:0/1698281119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc09d9f0 0x7f76dc136530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009840 tx=0x7f76c8009b50 comp rx=0 tx=0).stop 2023-09-30T22:17:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76e2aa8700 1 -- 172.21.15.89:0/1698281119 shutdown_connections 2023-09-30T22:17:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76e2aa8700 1 --2- 172.21.15.89:0/1698281119 >> v2:172.21.15.89:3300/0 conn(0x7f76dc136a70 0x7f76dc138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76e2aa8700 1 --2- 172.21.15.89:0/1698281119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc09d9f0 0x7f76dc136530 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.775+0000 7f76e2aa8700 1 -- 172.21.15.89:0/1698281119 >> 172.21.15.89:0/1698281119 conn(0x7f76dc066f50 msgr2=0x7f76dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.776+0000 7f76e2aa8700 1 -- 172.21.15.89:0/1698281119 shutdown_connections 2023-09-30T22:17:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.776+0000 7f76e2aa8700 1 -- 172.21.15.89:0/1698281119 wait complete. 2023-09-30T22:17:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.777+0000 7f76e2aa8700 1 Processor -- start 2023-09-30T22:17:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.777+0000 7f76e2aa8700 1 -- start start 2023-09-30T22:17:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e2aa8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76dc09d9f0 0x7f76dc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e2aa8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76dc1316c0 0x7f76dc071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e2aa8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc136a70 0x7f76dc071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e2aa8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc1362e0 con 0x7f76dc09d9f0 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e2aa8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc06ff40 con 0x7f76dc1316c0 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e2aa8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc136550 con 0x7f76dc136a70 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76dc1316c0 0x7f76dc071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e1045700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc136a70 0x7f76dc071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76dc1316c0 0x7f76dc071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57752/0 (socket says 172.21.15.89:57752) 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 -- 172.21.15.89:0/2886499866 learned_addr learned my addr 172.21.15.89:0/2886499866 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e0844700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76dc09d9f0 0x7f76dc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 -- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc136a70 msgr2=0x7f76dc071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc136a70 0x7f76dc071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 -- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76dc09d9f0 msgr2=0x7f76dc0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76dc09d9f0 0x7f76dc0710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 -- 172.21.15.89:0/2886499866 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c80094a0 con 0x7f76dc1316c0 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76e0844700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76dc09d9f0 0x7f76dc0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76dbfff700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76dc1316c0 0x7f76dc071600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f76c80090b0 tx=0x7f76c8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.778+0000 7f76d97fa700 1 -- 172.21.15.89:0/2886499866 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c80045d0 con 0x7f76dc1316c0 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.779+0000 7f76d97fa700 1 -- 172.21.15.89:0/2886499866 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c8011630 con 0x7f76dc1316c0 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.779+0000 7f76d97fa700 1 -- 172.21.15.89:0/2886499866 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c80119c0 con 0x7f76dc1316c0 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.779+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc12b900 con 0x7f76dc1316c0 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.779+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc072080 con 0x7f76dc1316c0 2023-09-30T22:17:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.779+0000 7f76d97fa700 1 -- 172.21.15.89:0/2886499866 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76c8011b20 con 0x7f76dc1316c0 2023-09-30T22:17:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.780+0000 7f76d97fa700 1 -- 172.21.15.89:0/2886499866 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f76c8040b50 con 0x7f76dc1316c0 2023-09-30T22:17:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.780+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc074ea0 con 0x7f76dc1316c0 2023-09-30T22:17:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.784+0000 7f76d97fa700 1 -- 172.21.15.89:0/2886499866 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76c802d080 con 0x7f76dc1316c0 2023-09-30T22:17:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.811+0000 7fcf48d87700 1 Processor -- start 2023-09-30T22:17:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.811+0000 7fcf48d87700 1 -- start start 2023-09-30T22:17:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf48d87700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf48d87700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf4409e000 0x7fcf441339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf48d87700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4409b120 con 0x7fcf4409e410 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf48d87700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4406b460 con 0x7fcf4409e000 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf48d87700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4409a7e0 con 0x7fcf4409d6f0 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf4259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf41d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf4409e000 0x7fcf441339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf4259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf4409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39648/0 (socket says 172.21.15.89:39648) 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf4259c700 1 -- 172.21.15.89:0/2942908744 learned_addr learned my addr 172.21.15.89:0/2942908744 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf4259c700 1 -- 172.21.15.89:0/2942908744 >> v2:172.21.15.89:3300/0 conn(0x7fcf4409e000 msgr2=0x7fcf441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf4259c700 1 --2- 172.21.15.89:0/2942908744 >> v2:172.21.15.89:3300/0 conn(0x7fcf4409e000 0x7fcf441339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf4259c700 1 -- 172.21.15.89:0/2942908744 >> v1:172.21.15.89:6789/0 conn(0x7fcf4409e410 legacy=0x7fcf44133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf4259c700 1 -- 172.21.15.89:0/2942908744 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4409a500 con 0x7fcf4409d6f0 2023-09-30T22:17:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.812+0000 7fcf41d9b700 1 --2- 172.21.15.89:0/2942908744 >> v2:172.21.15.89:3300/0 conn(0x7fcf4409e000 0x7fcf441339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf4259c700 1 --2- 172.21.15.89:0/2942908744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf4409dac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009920 tx=0x7fcf34009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0e136b06018e325 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf4159a700 1 -- 172.21.15.89:0/2942908744 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011830 con 0x7fcf4409d6f0 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf4159a700 1 -- 172.21.15.89:0/2942908744 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf34011990 con 0x7fcf4409d6f0 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf4159a700 1 -- 172.21.15.89:0/2942908744 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011d20 con 0x7fcf4409d6f0 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf48d87700 1 -- 172.21.15.89:0/2942908744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 msgr2=0x7fcf4409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf48d87700 1 --2- 172.21.15.89:0/2942908744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf4409dac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009920 tx=0x7fcf34009c30 comp rx=0 tx=0).stop 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf48d87700 1 -- 172.21.15.89:0/2942908744 shutdown_connections 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf48d87700 1 --2- 172.21.15.89:0/2942908744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf4409dac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf48d87700 1 --2- 172.21.15.89:0/2942908744 >> v2:172.21.15.89:3300/0 conn(0x7fcf4409e000 0x7fcf441339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf48d87700 1 -- 172.21.15.89:0/2942908744 >> 172.21.15.89:0/2942908744 conn(0x7fcf44066f50 msgr2=0x7fcf4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.813+0000 7fcf48d87700 1 -- 172.21.15.89:0/2942908744 shutdown_connections 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.814+0000 7fcf48d87700 1 -- 172.21.15.89:0/2942908744 wait complete. 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf48d87700 1 Processor -- start 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf48d87700 1 -- start start 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf48d87700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf441a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf48d87700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4409e000 0x7fcf441a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf48d87700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4409e410 0x7fcf441a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf48d87700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441a7cb0 con 0x7fcf4409e410 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf48d87700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441a9820 con 0x7fcf4409e000 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf48d87700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440a3380 con 0x7fcf4409d6f0 2023-09-30T22:17:59.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf41d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4409e000 0x7fcf441a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf4259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf441a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf41d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4409e000 0x7fcf441a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57756/0 (socket says 172.21.15.89:57756) 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf41d9b700 1 -- 172.21.15.89:0/4077416512 learned_addr learned my addr 172.21.15.89:0/4077416512 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf42d9d700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4409e410 0x7fcf441a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf4259c700 1 -- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4409e000 msgr2=0x7fcf441a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf4259c700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4409e000 0x7fcf441a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf4259c700 1 -- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4409e410 msgr2=0x7fcf441a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.815+0000 7fcf4259c700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4409e410 0x7fcf441a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.816+0000 7fcf4259c700 1 -- 172.21.15.89:0/4077416512 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf380094a0 con 0x7fcf4409d6f0 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.816+0000 7fcf42d9d700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4409e410 0x7fcf441a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:17:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.816+0000 7fcf4259c700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf441a60f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf4409ac00 tx=0x7fcf34004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:17:59.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.816+0000 7fcf32ffd700 1 -- 172.21.15.89:0/4077416512 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34004450 con 0x7fcf4409d6f0 2023-09-30T22:17:59.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.816+0000 7fcf32ffd700 1 -- 172.21.15.89:0/4077416512 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf340036a0 con 0x7fcf4409d6f0 2023-09-30T22:17:59.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.816+0000 7fcf32ffd700 1 -- 172.21.15.89:0/4077416512 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf340045f0 con 0x7fcf4409d6f0 2023-09-30T22:17:59.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.816+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf34009550 con 0x7fcf4409d6f0 2023-09-30T22:17:59.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.816+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf441a8330 con 0x7fcf4409d6f0 2023-09-30T22:17:59.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.817+0000 7fcf32ffd700 1 -- 172.21.15.89:0/4077416512 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf34003a50 con 0x7fcf4409d6f0 2023-09-30T22:17:59.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.817+0000 7fcf32ffd700 1 -- 172.21.15.89:0/4077416512 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcf34021030 con 0x7fcf4409d6f0 2023-09-30T22:17:59.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.817+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf44074ea0 con 0x7fcf4409d6f0 2023-09-30T22:17:59.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.826+0000 7fcf32ffd700 1 -- 172.21.15.89:0/4077416512 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf34015070 con 0x7fcf4409d6f0 2023-09-30T22:17:59.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.909+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff5d0122ad0 con 0x7ff5d009e1d0 2023-09-30T22:17:59.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.909+0000 7ff5c5ffb700 1 -- 172.21.15.89:0/3554656762 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff5c0018350 con 0x7ff5d009e1d0 2023-09-30T22:17:59.909 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:17:59.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d009e1d0 msgr2=0x7ff5d0120c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d009e1d0 0x7ff5d0120c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009870 tx=0x7ff5c0004750 comp rx=0 tx=0).stop 2023-09-30T22:17:59.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 shutdown_connections 2023-09-30T22:17:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d0139170 0x7ff5d0121150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5d009d6f0 0x7ff5d01206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 --2- 172.21.15.89:0/3554656762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5d009e1d0 0x7ff5d0120c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 >> 172.21.15.89:0/3554656762 conn(0x7ff5d0066f50 msgr2=0x7ff5d0136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 shutdown_connections 2023-09-30T22:17:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.912+0000 7ff5d761a700 1 -- 172.21.15.89:0/3554656762 wait complete. 2023-09-30T22:17:59.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:17:59.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.986+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f76dc072330 con 0x7f76dc1316c0 2023-09-30T22:17:59.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.986+0000 7f76d97fa700 1 -- 172.21.15.89:0/2886499866 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76c8023020 con 0x7f76dc1316c0 2023-09-30T22:17:59.986 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:17:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76dc1316c0 msgr2=0x7f76dc071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:17:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76dc1316c0 0x7f76dc071600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f76c80090b0 tx=0x7f76c8004750 comp rx=0 tx=0).stop 2023-09-30T22:17:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 shutdown_connections 2023-09-30T22:17:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76dc136a70 0x7f76dc071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76dc09d9f0 0x7f76dc0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 --2- 172.21.15.89:0/2886499866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76dc1316c0 0x7f76dc071600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:17:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 >> 172.21.15.89:0/2886499866 conn(0x7f76dc066f50 msgr2=0x7f76dc136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:17:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 shutdown_connections 2023-09-30T22:17:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:17:59.988+0000 7f76e2aa8700 1 -- 172.21.15.89:0/2886499866 wait complete. 2023-09-30T22:18:00.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.018+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcf4406c360 con 0x7fcf4409d6f0 2023-09-30T22:18:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.019+0000 7fcf32ffd700 1 -- 172.21.15.89:0/4077416512 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf3401a430 con 0x7fcf4409d6f0 2023-09-30T22:18:00.019 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 msgr2=0x7fcf441a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf441a60f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf4409ac00 tx=0x7fcf34004750 comp rx=0 tx=0).stop 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 shutdown_connections 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4409e410 0x7fcf441a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4409d6f0 0x7fcf441a60f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 --2- 172.21.15.89:0/4077416512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4409e000 0x7fcf441a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 >> 172.21.15.89:0/4077416512 conn(0x7fcf44066f50 msgr2=0x7fcf441baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 shutdown_connections 2023-09-30T22:18:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.021+0000 7fcf48d87700 1 -- 172.21.15.89:0/4077416512 wait complete. 2023-09-30T22:18:00.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:00.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:00.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.351+0000 7fbc3b33a700 1 Processor -- start 2023-09-30T22:18:00.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.351+0000 7fbc3b33a700 1 -- start start 2023-09-30T22:18:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.351+0000 7fbc3b33a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc34136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.351+0000 7fbc3b33a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc34136fe0 0x7fbc341393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.351+0000 7fbc3b33a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc34070940 con 0x7fbc3409e930 2023-09-30T22:18:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.351+0000 7fbc3b33a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3406fd60 con 0x7fbc34136fe0 2023-09-30T22:18:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.351+0000 7fbc3b33a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3406fa80 con 0x7fbc3409f410 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc388d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc34136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc390d6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbc3409e930 0x7fbc3409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38004/0 (socket says 172.21.15.89:38004) 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc390d6700 1 -- 172.21.15.89:0/4291592478 learned_addr learned my addr 172.21.15.89:0/4291592478 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc398d7700 1 --2- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc34136fe0 0x7fbc341393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc2bfff700 1 -- 172.21.15.89:0/4291592478 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439259640 0 0) 0x7fbc34070940 con 0x7fbc3409e930 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc2bfff700 1 -- 172.21.15.89:0/4291592478 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc200035b0 con 0x7fbc3409e930 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc398d7700 1 -- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 msgr2=0x7fbc34136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc398d7700 1 --2- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc34136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc398d7700 1 -- 172.21.15.89:0/4291592478 >> v1:172.21.15.89:6789/0 conn(0x7fbc3409e930 legacy=0x7fbc3409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc398d7700 1 -- 172.21.15.89:0/4291592478 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3409fb60 con 0x7fbc34136fe0 2023-09-30T22:18:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc388d5700 1 --2- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc34136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc398d7700 1 --2- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc34136fe0 0x7fbc341393c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009dc0 tx=0x7fbc2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d344d1454917db0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc2bfff700 1 -- 172.21.15.89:0/4291592478 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00e070 con 0x7fbc34136fe0 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc2bfff700 1 -- 172.21.15.89:0/4291592478 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c00bb20 con 0x7fbc34136fe0 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.352+0000 7fbc2bfff700 1 -- 172.21.15.89:0/4291592478 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c012430 con 0x7fbc34136fe0 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.353+0000 7fbc3b33a700 1 -- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc34136fe0 msgr2=0x7fbc341393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.353+0000 7fbc3b33a700 1 --2- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc34136fe0 0x7fbc341393c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009dc0 tx=0x7fbc2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.353+0000 7fbc3b33a700 1 -- 172.21.15.89:0/4291592478 shutdown_connections 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.353+0000 7fbc3b33a700 1 --2- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc34136fe0 0x7fbc341393c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.353+0000 7fbc3b33a700 1 --2- 172.21.15.89:0/4291592478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc34136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.353+0000 7fbc3b33a700 1 -- 172.21.15.89:0/4291592478 >> 172.21.15.89:0/4291592478 conn(0x7fbc340a4210 msgr2=0x7fbc340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.353+0000 7fbc3b33a700 1 -- 172.21.15.89:0/4291592478 shutdown_connections 2023-09-30T22:18:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.353+0000 7fbc3b33a700 1 -- 172.21.15.89:0/4291592478 wait complete. 2023-09-30T22:18:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.354+0000 7fbc3b33a700 1 Processor -- start 2023-09-30T22:18:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.354+0000 7fbc3b33a700 1 -- start start 2023-09-30T22:18:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc3b33a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc3409e930 0x7fbc3411e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc3b33a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc3411ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc3b33a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc34136fe0 0x7fbc3411f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc3b33a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc34134540 con 0x7fbc34136fe0 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc3b33a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340a3310 con 0x7fbc3409e930 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc3b33a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc34071d20 con 0x7fbc3409f410 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc3409e930 0x7fbc3411e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc3409e930 0x7fbc3411e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58870/0 (socket says 172.21.15.89:58870) 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 -- 172.21.15.89:0/3460448823 learned_addr learned my addr 172.21.15.89:0/3460448823 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc398d7700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc34136fe0 0x7fbc3411f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc388d5700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc3411ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 -- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 msgr2=0x7fbc3411ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc3411ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 -- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc34136fe0 msgr2=0x7fbc3411f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc34136fe0 0x7fbc3411f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 -- 172.21.15.89:0/3460448823 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240094a0 con 0x7fbc3409e930 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc398d7700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc34136fe0 0x7fbc3411f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc390d6700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc3409e930 0x7fbc3411e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000c730 tx=0x7fbc3000ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc29ffb700 1 -- 172.21.15.89:0/3460448823 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30005d70 con 0x7fbc3409e930 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc29ffb700 1 -- 172.21.15.89:0/3460448823 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc300070d0 con 0x7fbc3409e930 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc29ffb700 1 -- 172.21.15.89:0/3460448823 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30007460 con 0x7fbc3409e930 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.355+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c009a70 con 0x7fbc3409e930 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.356+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc34120370 con 0x7fbc3409e930 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.356+0000 7fbc29ffb700 1 -- 172.21.15.89:0/3460448823 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc300075c0 con 0x7fbc3409e930 2023-09-30T22:18:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.356+0000 7fbc29ffb700 1 -- 172.21.15.89:0/3460448823 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbc3003ebf0 con 0x7fbc3409e930 2023-09-30T22:18:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.356+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc34121b30 con 0x7fbc3409e930 2023-09-30T22:18:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.361+0000 7fbc29ffb700 1 -- 172.21.15.89:0/3460448823 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc3000e280 con 0x7fbc3409e930 2023-09-30T22:18:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.543+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbc34074ea0 con 0x7fbc3409e930 2023-09-30T22:18:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.544+0000 7fbc29ffb700 1 -- 172.21.15.89:0/3460448823 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc30017020 con 0x7fbc3409e930 2023-09-30T22:18:00.544 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:00.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc3409e930 msgr2=0x7fbc3411e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:00.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc3409e930 0x7fbc3411e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000c730 tx=0x7fbc3000ca40 comp rx=0 tx=0).stop 2023-09-30T22:18:00.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 shutdown_connections 2023-09-30T22:18:00.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc34136fe0 0x7fbc3411f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc3409e930 0x7fbc3411e770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 --2- 172.21.15.89:0/3460448823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc3409f410 0x7fbc3411ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:00.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 >> 172.21.15.89:0/3460448823 conn(0x7fbc340a4210 msgr2=0x7fbc34134900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:00.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 shutdown_connections 2023-09-30T22:18:00.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:00.546+0000 7fbc3b33a700 1 -- 172.21.15.89:0/3460448823 wait complete. 2023-09-30T22:18:00.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:00.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:01.002 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:01.033 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:01.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.059+0000 7fc393604700 1 Processor -- start 2023-09-30T22:18:01.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.059+0000 7fc393604700 1 -- start start 2023-09-30T22:18:01.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc393604700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c130560 0x7fc38c130930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc393604700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc38c130e70 0x7fc38c06fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc393604700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38c070a50 con 0x7fc38c09eca0 2023-09-30T22:18:01.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc393604700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c0717d0 con 0x7fc38c130e70 2023-09-30T22:18:01.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc393604700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c130340 con 0x7fc38c130560 2023-09-30T22:18:01.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc390b9f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc38c09eca0 0x7fc38c06f620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38010/0 (socket says 172.21.15.89:38010) 2023-09-30T22:18:01.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc390b9f700 1 -- 172.21.15.89:0/2822105289 learned_addr learned my addr 172.21.15.89:0/2822105289 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:01.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc391ba1700 1 --2- 172.21.15.89:0/2822105289 >> v2:172.21.15.89:3300/0 conn(0x7fc38c130e70 0x7fc38c06fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc3913a0700 1 --2- 172.21.15.89:0/2822105289 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c130560 0x7fc38c130930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc391ba1700 1 -- 172.21.15.89:0/2822105289 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c130560 msgr2=0x7fc38c130930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc391ba1700 1 --2- 172.21.15.89:0/2822105289 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c130560 0x7fc38c130930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc391ba1700 1 -- 172.21.15.89:0/2822105289 >> v1:172.21.15.89:6789/0 conn(0x7fc38c09eca0 legacy=0x7fc38c06f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc383fff700 1 -- 172.21.15.89:0/2822105289 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178606534 0 0) 0x7fc38c070a50 con 0x7fc38c09eca0 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.060+0000 7fc391ba1700 1 -- 172.21.15.89:0/2822105289 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c071060 con 0x7fc38c130e70 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.061+0000 7fc391ba1700 1 --2- 172.21.15.89:0/2822105289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 0x7fc38c06fd30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc384009d90 tx=0x7fc38400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6b28c62adb3503 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.061+0000 7fc383fff700 1 -- 172.21.15.89:0/2822105289 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38400bd20 con 0x7fc38c130e70 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.061+0000 7fc383fff700 1 -- 172.21.15.89:0/2822105289 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38400be80 con 0x7fc38c130e70 2023-09-30T22:18:01.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.061+0000 7fc383fff700 1 -- 172.21.15.89:0/2822105289 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3840116a0 con 0x7fc38c130e70 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.061+0000 7fc393604700 1 -- 172.21.15.89:0/2822105289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 msgr2=0x7fc38c06fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.061+0000 7fc393604700 1 --2- 172.21.15.89:0/2822105289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 0x7fc38c06fd30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc384009d90 tx=0x7fc38400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.062+0000 7fc393604700 1 -- 172.21.15.89:0/2822105289 shutdown_connections 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.062+0000 7fc393604700 1 --2- 172.21.15.89:0/2822105289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 0x7fc38c06fd30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.062+0000 7fc393604700 1 --2- 172.21.15.89:0/2822105289 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c130560 0x7fc38c130930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.062+0000 7fc393604700 1 -- 172.21.15.89:0/2822105289 >> 172.21.15.89:0/2822105289 conn(0x7fc38c0a2aa0 msgr2=0x7fc38c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.062+0000 7fc393604700 1 -- 172.21.15.89:0/2822105289 shutdown_connections 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.062+0000 7fc393604700 1 -- 172.21.15.89:0/2822105289 wait complete. 2023-09-30T22:18:01.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc393604700 1 Processor -- start 2023-09-30T22:18:01.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc393604700 1 -- start start 2023-09-30T22:18:01.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc393604700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c09eca0 0x7fc38c11edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc393604700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc38c130560 0x7fc38c11f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc393604700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 0x7fc38c11f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc393604700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c120250 con 0x7fc38c130e70 2023-09-30T22:18:01.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc393604700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c122810 con 0x7fc38c09eca0 2023-09-30T22:18:01.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc393604700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c12c7c0 con 0x7fc38c130560 2023-09-30T22:18:01.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc3913a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c09eca0 0x7fc38c11edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc391ba1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 0x7fc38c11f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc3913a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c09eca0 0x7fc38c11edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58900/0 (socket says 172.21.15.89:58900) 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc3913a0700 1 -- 172.21.15.89:0/103208295 learned_addr learned my addr 172.21.15.89:0/103208295 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.063+0000 7fc390b9f700 1 --2- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc38c130560 0x7fc38c11f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc3913a0700 1 -- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc38c130560 msgr2=0x7fc38c11f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc3913a0700 1 --2- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc38c130560 0x7fc38c11f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc3913a0700 1 -- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 msgr2=0x7fc38c11f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc3913a0700 1 --2- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 0x7fc38c11f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc3913a0700 1 -- 172.21.15.89:0/103208295 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc384009a70 con 0x7fc38c09eca0 2023-09-30T22:18:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc3913a0700 1 --2- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c09eca0 0x7fc38c11edf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc38800e9b0 tx=0x7fc38800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc381ffb700 1 -- 172.21.15.89:0/103208295 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388009710 con 0x7fc38c09eca0 2023-09-30T22:18:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc381ffb700 1 -- 172.21.15.89:0/103208295 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38800ee50 con 0x7fc38c09eca0 2023-09-30T22:18:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc381ffb700 1 -- 172.21.15.89:0/103208295 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38800ca50 con 0x7fc38c09eca0 2023-09-30T22:18:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38c1209a0 con 0x7fc38c09eca0 2023-09-30T22:18:01.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.064+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38c0719f0 con 0x7fc38c09eca0 2023-09-30T22:18:01.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.065+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38c074ea0 con 0x7fc38c09eca0 2023-09-30T22:18:01.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.065+0000 7fc381ffb700 1 -- 172.21.15.89:0/103208295 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc38800cbb0 con 0x7fc38c09eca0 2023-09-30T22:18:01.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.065+0000 7fc381ffb700 1 -- 172.21.15.89:0/103208295 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc38803faa0 con 0x7fc38c09eca0 2023-09-30T22:18:01.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.070+0000 7fc381ffb700 1 -- 172.21.15.89:0/103208295 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc38801b3b0 con 0x7fc38c09eca0 2023-09-30T22:18:01.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.142+0000 7f6511170700 1 Processor -- start 2023-09-30T22:18:01.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.142+0000 7f6511170700 1 -- start start 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f6511170700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f6511170700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c133e30 0x7f650c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f6511170700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650c06b990 con 0x7f650c134260 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f6511170700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c0a0590 con 0x7f650c133e30 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f6511170700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f650c09fb80 con 0x7f650c133420 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650b59e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f650c134260 0x7f650c1304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38024/0 (socket says 172.21.15.89:38024) 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650b59e700 1 -- 172.21.15.89:0/1366668773 learned_addr learned my addr 172.21.15.89:0/1366668773 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650a59c700 1 --2- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c133e30 0x7f650c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650ad9d700 1 -- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c133e30 msgr2=0x7f650c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650ad9d700 1 --2- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c133e30 0x7f650c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650ad9d700 1 -- 172.21.15.89:0/1366668773 >> v1:172.21.15.89:6789/0 conn(0x7f650c134260 legacy=0x7f650c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650ad9d700 1 -- 172.21.15.89:0/1366668773 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c0a0030 con 0x7f650c133420 2023-09-30T22:18:01.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f6509d9b700 1 -- 172.21.15.89:0/1366668773 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676609747 0 0) 0x7f650c06b990 con 0x7f650c134260 2023-09-30T22:18:01.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.143+0000 7f650a59c700 1 --2- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c133e30 0x7f650c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:01.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f650ad9d700 1 --2- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c1337f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009870 tx=0x7f64fc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def7e667ce934743 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:01.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6509d9b700 1 -- 172.21.15.89:0/1366668773 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc011830 con 0x7f650c133420 2023-09-30T22:18:01.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6509d9b700 1 -- 172.21.15.89:0/1366668773 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc011990 con 0x7f650c133420 2023-09-30T22:18:01.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6509d9b700 1 -- 172.21.15.89:0/1366668773 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc011d20 con 0x7f650c133420 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6511170700 1 -- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 msgr2=0x7f650c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6511170700 1 --2- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c1337f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009870 tx=0x7f64fc009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6511170700 1 -- 172.21.15.89:0/1366668773 shutdown_connections 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6511170700 1 --2- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c1337f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6511170700 1 --2- 172.21.15.89:0/1366668773 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c133e30 0x7f650c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6511170700 1 -- 172.21.15.89:0/1366668773 >> 172.21.15.89:0/1366668773 conn(0x7f650c0a4210 msgr2=0x7f650c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6511170700 1 -- 172.21.15.89:0/1366668773 shutdown_connections 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.144+0000 7f6511170700 1 -- 172.21.15.89:0/1366668773 wait complete. 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.146+0000 7f6511170700 1 Processor -- start 2023-09-30T22:18:01.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.146+0000 7f6511170700 1 -- start start 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f6511170700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f6511170700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650c133e30 0x7f650c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f6511170700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c134260 0x7f650c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f6511170700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c12d900 con 0x7f650c133e30 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f6511170700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c09dc30 con 0x7f650c134260 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f6511170700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f650c11fab0 con 0x7f650c133420 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650c133e30 0x7f650c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650c133e30 0x7f650c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43092/0 (socket says 172.21.15.89:43092) 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650a59c700 1 -- 172.21.15.89:0/1079136 learned_addr learned my addr 172.21.15.89:0/1079136 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650b59e700 1 --2- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c134260 0x7f650c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650ad9d700 1 -- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c134260 msgr2=0x7f650c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650ad9d700 1 --2- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c134260 0x7f650c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650ad9d700 1 -- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650c133e30 msgr2=0x7f650c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650ad9d700 1 --2- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650c133e30 0x7f650c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650ad9d700 1 -- 172.21.15.89:0/1079136 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65000094a0 con 0x7f650c133420 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.147+0000 7f650ad9d700 1 --2- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c09c480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64fc016000 tx=0x7f64fc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.148+0000 7f64f37fe700 1 -- 172.21.15.89:0/1079136 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc004510 con 0x7f650c133420 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.148+0000 7f64f37fe700 1 -- 172.21.15.89:0/1079136 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc01f070 con 0x7f650c133420 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.148+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc0094a0 con 0x7f650c133420 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.148+0000 7f64f37fe700 1 -- 172.21.15.89:0/1079136 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0038e0 con 0x7f650c133420 2023-09-30T22:18:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.148+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c11fd70 con 0x7f650c133420 2023-09-30T22:18:01.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.149+0000 7f64f37fe700 1 -- 172.21.15.89:0/1079136 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64fc003a40 con 0x7f650c133420 2023-09-30T22:18:01.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.149+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c074ea0 con 0x7f650c133420 2023-09-30T22:18:01.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.149+0000 7f64f37fe700 1 -- 172.21.15.89:0/1079136 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f64fc021070 con 0x7f650c133420 2023-09-30T22:18:01.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.153+0000 7f64f37fe700 1 -- 172.21.15.89:0/1079136 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64fc0269a0 con 0x7f650c133420 2023-09-30T22:18:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.186+0000 7f75f150d700 1 Processor -- start 2023-09-30T22:18:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.186+0000 7f75f150d700 1 -- start start 2023-09-30T22:18:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75f150d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09d6f0 0x7f75ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75f150d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f75ec09e000 0x7f75ec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75f150d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec09b120 con 0x7f75ec09e410 2023-09-30T22:18:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75f150d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ec06b460 con 0x7f75ec09e000 2023-09-30T22:18:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75f150d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec09a7e0 con 0x7f75ec09d6f0 2023-09-30T22:18:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75ea7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f75ec09e000 0x7f75ec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75ea7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f75ec09e000 0x7f75ec1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43098/0 (socket says 172.21.15.89:43098) 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75ea7fc700 1 -- 172.21.15.89:0/2084453342 learned_addr learned my addr 172.21.15.89:0/2084453342 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75eaffd700 1 --2- 172.21.15.89:0/2084453342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09d6f0 0x7f75ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75ea7fc700 1 -- 172.21.15.89:0/2084453342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09d6f0 msgr2=0x7f75ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75ea7fc700 1 --2- 172.21.15.89:0/2084453342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09d6f0 0x7f75ec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75ea7fc700 1 -- 172.21.15.89:0/2084453342 >> v1:172.21.15.89:6789/0 conn(0x7f75ec09e410 legacy=0x7f75ec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75ea7fc700 1 -- 172.21.15.89:0/2084453342 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec09a500 con 0x7f75ec09e000 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.187+0000 7f75ea7fc700 1 --2- 172.21.15.89:0/2084453342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1339f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f75d40098f0 tx=0x7f75d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef0b29d7cfc9831 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75e9ffb700 1 -- 172.21.15.89:0/2084453342 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4011830 con 0x7f75ec09e000 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75e9ffb700 1 -- 172.21.15.89:0/2084453342 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d4011990 con 0x7f75ec09e000 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75e9ffb700 1 -- 172.21.15.89:0/2084453342 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4011d20 con 0x7f75ec09e000 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75f150d700 1 -- 172.21.15.89:0/2084453342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 msgr2=0x7f75ec1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75f150d700 1 --2- 172.21.15.89:0/2084453342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1339f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f75d40098f0 tx=0x7f75d4009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75f150d700 1 -- 172.21.15.89:0/2084453342 shutdown_connections 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75f150d700 1 --2- 172.21.15.89:0/2084453342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09d6f0 0x7f75ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75f150d700 1 --2- 172.21.15.89:0/2084453342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1339f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75f150d700 1 -- 172.21.15.89:0/2084453342 >> 172.21.15.89:0/2084453342 conn(0x7f75ec066f50 msgr2=0x7f75ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75f150d700 1 -- 172.21.15.89:0/2084453342 shutdown_connections 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.188+0000 7f75f150d700 1 -- 172.21.15.89:0/2084453342 wait complete. 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.190+0000 7f75f150d700 1 Processor -- start 2023-09-30T22:18:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.190+0000 7f75f150d700 1 -- start start 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.190+0000 7f75f150d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75ec09d6f0 0x7f75ec1a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.190+0000 7f75f150d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.190+0000 7f75f150d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09e410 0x7f75ec1a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.190+0000 7f75f150d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a7b30 con 0x7f75ec09e000 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.190+0000 7f75f150d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a8150 con 0x7f75ec09e410 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.190+0000 7f75f150d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a8d80 con 0x7f75ec09d6f0 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43114/0 (socket says 172.21.15.89:43114) 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 -- 172.21.15.89:0/605538618 learned_addr learned my addr 172.21.15.89:0/605538618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75eb7fe700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09e410 0x7f75ec1a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75eaffd700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75ec09d6f0 0x7f75ec1a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 -- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75ec09d6f0 msgr2=0x7f75ec1a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75ec09d6f0 0x7f75ec1a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 -- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09e410 msgr2=0x7f75ec1a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09e410 0x7f75ec1a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 -- 172.21.15.89:0/605538618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4009550 con 0x7f75ec09e000 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75eb7fe700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09e410 0x7f75ec1a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75ea7fc700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1a64f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f75d4013000 tx=0x7f75d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75e37fe700 1 -- 172.21.15.89:0/605538618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d40036a0 con 0x7f75ec09e000 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75e37fe700 1 -- 172.21.15.89:0/605538618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d401f070 con 0x7f75ec09e000 2023-09-30T22:18:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75e37fe700 1 -- 172.21.15.89:0/605538618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4003a20 con 0x7f75ec09e000 2023-09-30T22:18:01.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec1a91d0 con 0x7f75ec09e000 2023-09-30T22:18:01.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.191+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec1a7410 con 0x7f75ec09e000 2023-09-30T22:18:01.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.192+0000 7f75e37fe700 1 -- 172.21.15.89:0/605538618 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f75d4003b80 con 0x7f75ec09e000 2023-09-30T22:18:01.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.192+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec074ea0 con 0x7f75ec09e000 2023-09-30T22:18:01.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.192+0000 7f75e37fe700 1 -- 172.21.15.89:0/605538618 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f75d4021070 con 0x7f75ec09e000 2023-09-30T22:18:01.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.197+0000 7f75e37fe700 1 -- 172.21.15.89:0/605538618 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f75d4010380 con 0x7f75ec09e000 2023-09-30T22:18:01.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.292+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc38c06b9e0 con 0x7fc38c09eca0 2023-09-30T22:18:01.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.292+0000 7fc381ffb700 1 -- 172.21.15.89:0/103208295 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc388017020 con 0x7fc38c09eca0 2023-09-30T22:18:01.292 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:01.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.294+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c09eca0 msgr2=0x7fc38c11edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.294+0000 7fc393604700 1 --2- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c09eca0 0x7fc38c11edf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc38800e9b0 tx=0x7fc38800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.294+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 shutdown_connections 2023-09-30T22:18:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.294+0000 7fc393604700 1 --2- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc38c130e70 0x7fc38c11f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.294+0000 7fc393604700 1 --2- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc38c09eca0 0x7fc38c11edf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.294+0000 7fc393604700 1 --2- 172.21.15.89:0/103208295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc38c130560 0x7fc38c11f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.294+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 >> 172.21.15.89:0/103208295 conn(0x7fc38c0a2aa0 msgr2=0x7fc38c12d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.295+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 shutdown_connections 2023-09-30T22:18:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.295+0000 7fc393604700 1 -- 172.21.15.89:0/103208295 wait complete. 2023-09-30T22:18:01.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:01.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.351+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f650c11ea90 con 0x7f650c133420 2023-09-30T22:18:01.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.351+0000 7f64f37fe700 1 -- 172.21.15.89:0/1079136 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64fc015020 con 0x7f650c133420 2023-09-30T22:18:01.351 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:01.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 msgr2=0x7f650c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 --2- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c09c480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64fc016000 tx=0x7f64fc004750 comp rx=0 tx=0).stop 2023-09-30T22:18:01.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 shutdown_connections 2023-09-30T22:18:01.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 --2- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f650c134260 0x7f650c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 --2- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f650c133420 0x7f650c09c480 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 --2- 172.21.15.89:0/1079136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f650c133e30 0x7f650c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 >> 172.21.15.89:0/1079136 conn(0x7f650c0a4210 msgr2=0x7f650c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:01.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 shutdown_connections 2023-09-30T22:18:01.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.354+0000 7f6511170700 1 -- 172.21.15.89:0/1079136 wait complete. 2023-09-30T22:18:01.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:01.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.390+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f75ec06c360 con 0x7f75ec09e000 2023-09-30T22:18:01.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.390+0000 7f75e37fe700 1 -- 172.21.15.89:0/605538618 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75d402a020 con 0x7f75ec09e000 2023-09-30T22:18:01.390 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:01.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 msgr2=0x7f75ec1a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1a64f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f75d4013000 tx=0x7f75d4004750 comp rx=0 tx=0).stop 2023-09-30T22:18:01.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 shutdown_connections 2023-09-30T22:18:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75ec09e410 0x7f75ec1a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75ec09d6f0 0x7f75ec1a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 --2- 172.21.15.89:0/605538618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75ec09e000 0x7f75ec1a64f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 >> 172.21.15.89:0/605538618 conn(0x7f75ec066f50 msgr2=0x7f75ec1baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 shutdown_connections 2023-09-30T22:18:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.392+0000 7f75f150d700 1 -- 172.21.15.89:0/605538618 wait complete. 2023-09-30T22:18:01.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:01.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:01.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.681+0000 7f9c7881e700 1 Processor -- start 2023-09-30T22:18:01.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.681+0000 7f9c7881e700 1 -- start start 2023-09-30T22:18:01.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c7881e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c741338a0 0x7f9c74133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c7881e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c7409de50 0x7f9c7409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c7881e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c7409e760 0x7f9c7413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c7881e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c7406fef0 con 0x7f9c7409e760 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c7881e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7406ef30 con 0x7f9c741338a0 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c7881e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c741334b0 con 0x7f9c7409de50 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c71d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c7409de50 0x7f9c7409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c7259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c741338a0 0x7f9c74133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c72d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c7409e760 0x7f9c7413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c71d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c7409de50 0x7f9c7409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36216/0 (socket says 172.21.15.89:36216) 2023-09-30T22:18:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c7259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c741338a0 0x7f9c74133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58922/0 (socket says 172.21.15.89:58922) 2023-09-30T22:18:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c71d9b700 1 -- 172.21.15.89:0/1694070478 learned_addr learned my addr 172.21.15.89:0/1694070478 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c72d9d700 1 -- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c7409de50 msgr2=0x7f9c7409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c72d9d700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c7409de50 0x7f9c7409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c72d9d700 1 -- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c741338a0 msgr2=0x7f9c74133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c72d9d700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c741338a0 0x7f9c74133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.682+0000 7f9c72d9d700 1 -- 172.21.15.89:0/1694070478 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7406fc30 con 0x7f9c7409e760 2023-09-30T22:18:01.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c71d9b700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c7409de50 0x7f9c7409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7259c700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c741338a0 0x7f9c74133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c72d9d700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 0x7f9c7413a730 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009d90 tx=0x7f9c6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41c1a7c51b0131 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7159a700 1 -- 172.21.15.89:0/1694070478 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00e070 con 0x7f9c7409e760 2023-09-30T22:18:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7159a700 1 -- 172.21.15.89:0/1694070478 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c00bb20 con 0x7f9c7409e760 2023-09-30T22:18:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7159a700 1 -- 172.21.15.89:0/1694070478 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c012430 con 0x7f9c7409e760 2023-09-30T22:18:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 -- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 msgr2=0x7f9c7413a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 0x7f9c7413a730 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009d90 tx=0x7f9c6c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:01.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 -- 172.21.15.89:0/1694070478 shutdown_connections 2023-09-30T22:18:01.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 0x7f9c7413a730 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c741338a0 0x7f9c74133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 --2- 172.21.15.89:0/1694070478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c7409de50 0x7f9c7409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 -- 172.21.15.89:0/1694070478 >> 172.21.15.89:0/1694070478 conn(0x7f9c740a2aa0 msgr2=0x7f9c740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:01.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 -- 172.21.15.89:0/1694070478 shutdown_connections 2023-09-30T22:18:01.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.683+0000 7f9c7881e700 1 -- 172.21.15.89:0/1694070478 wait complete. 2023-09-30T22:18:01.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.684+0000 7f9c7881e700 1 Processor -- start 2023-09-30T22:18:01.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7881e700 1 -- start start 2023-09-30T22:18:01.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7881e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c7409de50 0x7f9c7411ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7881e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 0x7f9c7411f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7881e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c741338a0 0x7f9c7411f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:01.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7881e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c741210e0 con 0x7f9c7409e760 2023-09-30T22:18:01.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7881e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c741200b0 con 0x7f9c7409de50 2023-09-30T22:18:01.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7881e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c74120920 con 0x7f9c741338a0 2023-09-30T22:18:01.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c7409de50 0x7f9c7411ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c7409de50 0x7f9c7411ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58932/0 (socket says 172.21.15.89:58932) 2023-09-30T22:18:01.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 -- 172.21.15.89:0/1030874878 learned_addr learned my addr 172.21.15.89:0/1030874878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:01.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c71d9b700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 0x7f9c7411f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c72d9d700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c741338a0 0x7f9c7411f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 -- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c741338a0 msgr2=0x7f9c7411f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c741338a0 0x7f9c7411f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 -- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 msgr2=0x7f9c7411f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 0x7f9c7411f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 -- 172.21.15.89:0/1030874878 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c64009550 con 0x7f9c7409de50 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c72d9d700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c741338a0 0x7f9c7411f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c71d9b700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 0x7f9c7411f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.685+0000 7f9c7259c700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c7409de50 0x7f9c7411ec50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800ebe0 tx=0x7f9c6800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.686+0000 7f9c62ffd700 1 -- 172.21.15.89:0/1030874878 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800c850 con 0x7f9c7409de50 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.686+0000 7f9c62ffd700 1 -- 172.21.15.89:0/1030874878 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6800c9b0 con 0x7f9c7409de50 2023-09-30T22:18:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.686+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c680094a0 con 0x7f9c7409de50 2023-09-30T22:18:01.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.686+0000 7f9c62ffd700 1 -- 172.21.15.89:0/1030874878 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800cd40 con 0x7f9c7409de50 2023-09-30T22:18:01.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.686+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c009a70 con 0x7f9c7409de50 2023-09-30T22:18:01.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.686+0000 7f9c62ffd700 1 -- 172.21.15.89:0/1030874878 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c68013470 con 0x7f9c7409de50 2023-09-30T22:18:01.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.686+0000 7f9c62ffd700 1 -- 172.21.15.89:0/1030874878 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9c6803fb30 con 0x7f9c7409de50 2023-09-30T22:18:01.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.687+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c741344f0 con 0x7f9c7409de50 2023-09-30T22:18:01.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.691+0000 7f9c62ffd700 1 -- 172.21.15.89:0/1030874878 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c68013720 con 0x7f9c7409de50 2023-09-30T22:18:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.874+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9c74074ea0 con 0x7f9c7409de50 2023-09-30T22:18:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.875+0000 7f9c62ffd700 1 -- 172.21.15.89:0/1030874878 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c68017020 con 0x7f9c7409de50 2023-09-30T22:18:01.875 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:01.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c7409de50 msgr2=0x7f9c7411ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c7409de50 0x7f9c7411ec50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800ebe0 tx=0x7f9c6800eef0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 shutdown_connections 2023-09-30T22:18:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c741338a0 0x7f9c7411f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c7409de50 0x7f9c7411ec50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 --2- 172.21.15.89:0/1030874878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c7409e760 0x7f9c7411f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 >> 172.21.15.89:0/1030874878 conn(0x7f9c740a2aa0 msgr2=0x7f9c7409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 shutdown_connections 2023-09-30T22:18:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:01.877+0000 7f9c7881e700 1 -- 172.21.15.89:0/1030874878 wait complete. 2023-09-30T22:18:01.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:02.307 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:02.367 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:02.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:02.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c1ad9e700 1 Processor -- start 2023-09-30T22:18:02.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c1ad9e700 1 -- start start 2023-09-30T22:18:02.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c1ad9e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c140a0a30 0x7f2c140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c1ad9e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2c14138d80 0x7f2c1413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c1ad9e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c140a2a70 con 0x7f2c1409d630 2023-09-30T22:18:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c1ad9e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c140a01d0 con 0x7f2c14138d80 2023-09-30T22:18:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c1ad9e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1409e2b0 con 0x7f2c140a0a30 2023-09-30T22:18:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c18b3a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c140a0a30 0x7f2c140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c18b3a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c140a0a30 0x7f2c140a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58942/0 (socket says 172.21.15.89:58942) 2023-09-30T22:18:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c18b3a700 1 -- 172.21.15.89:0/1712787911 learned_addr learned my addr 172.21.15.89:0/1712787911 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.441+0000 7f2c1933b700 1 --2- 172.21.15.89:0/1712787911 >> v2:172.21.15.89:3300/0 conn(0x7f2c14138d80 0x7f2c1413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c1933b700 1 -- 172.21.15.89:0/1712787911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c140a0a30 msgr2=0x7f2c140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:02.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c1933b700 1 --2- 172.21.15.89:0/1712787911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c140a0a30 0x7f2c140a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c1933b700 1 -- 172.21.15.89:0/1712787911 >> v1:172.21.15.89:6789/0 conn(0x7f2c1409d630 legacy=0x7f2c140a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c1933b700 1 -- 172.21.15.89:0/1712787911 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1409dc50 con 0x7f2c14138d80 2023-09-30T22:18:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c18b3a700 1 --2- 172.21.15.89:0/1712787911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c140a0a30 0x7f2c140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c1933b700 1 --2- 172.21.15.89:0/1712787911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c14138d80 0x7f2c1413b160 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009d90 tx=0x7f2c0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20a7453908fc293 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c137fe700 1 -- 172.21.15.89:0/1712787911 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00e070 con 0x7f2c14138d80 2023-09-30T22:18:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c137fe700 1 -- 172.21.15.89:0/1712787911 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0c00bb20 con 0x7f2c14138d80 2023-09-30T22:18:02.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.442+0000 7f2c137fe700 1 -- 172.21.15.89:0/1712787911 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c012430 con 0x7f2c14138d80 2023-09-30T22:18:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.443+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/1712787911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c14138d80 msgr2=0x7f2c1413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.443+0000 7f2c1ad9e700 1 --2- 172.21.15.89:0/1712787911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c14138d80 0x7f2c1413b160 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009d90 tx=0x7f2c0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.443+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/1712787911 shutdown_connections 2023-09-30T22:18:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.443+0000 7f2c1ad9e700 1 --2- 172.21.15.89:0/1712787911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c14138d80 0x7f2c1413b160 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.443+0000 7f2c1ad9e700 1 --2- 172.21.15.89:0/1712787911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c140a0a30 0x7f2c140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.443+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/1712787911 >> 172.21.15.89:0/1712787911 conn(0x7f2c140a3050 msgr2=0x7f2c140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.443+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/1712787911 shutdown_connections 2023-09-30T22:18:02.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.443+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/1712787911 wait complete. 2023-09-30T22:18:02.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.444+0000 7f2c1ad9e700 1 Processor -- start 2023-09-30T22:18:02.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.444+0000 7f2c1ad9e700 1 -- start start 2023-09-30T22:18:02.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c1ad9e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409d630 0x7f2c14133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c1ad9e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c140a0a30 0x7f2c14133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c1ad9e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14138d80 0x7f2c14134310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c1ad9e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141371b0 con 0x7f2c140a0a30 2023-09-30T22:18:02.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c1ad9e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14138030 con 0x7f2c14138d80 2023-09-30T22:18:02.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c1ad9e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141373b0 con 0x7f2c1409d630 2023-09-30T22:18:02.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c18b3a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409d630 0x7f2c14133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c18b3a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409d630 0x7f2c14133890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36242/0 (socket says 172.21.15.89:36242) 2023-09-30T22:18:02.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c18b3a700 1 -- 172.21.15.89:0/3933550646 learned_addr learned my addr 172.21.15.89:0/3933550646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:02.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c13fff700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c140a0a30 0x7f2c14133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.445+0000 7f2c1933b700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14138d80 0x7f2c14134310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c18b3a700 1 -- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14138d80 msgr2=0x7f2c14134310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c18b3a700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14138d80 0x7f2c14134310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c18b3a700 1 -- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c140a0a30 msgr2=0x7f2c14133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c18b3a700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c140a0a30 0x7f2c14133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c18b3a700 1 -- 172.21.15.89:0/3933550646 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c08009550 con 0x7f2c1409d630 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c18b3a700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409d630 0x7f2c14133890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800ec10 tx=0x7f2c0800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c117fa700 1 -- 172.21.15.89:0/3933550646 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800c850 con 0x7f2c1409d630 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c009a70 con 0x7f2c1409d630 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c117fa700 1 -- 172.21.15.89:0/3933550646 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0800c9b0 con 0x7f2c1409d630 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c117fa700 1 -- 172.21.15.89:0/3933550646 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800cd40 con 0x7f2c1409d630 2023-09-30T22:18:02.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.446+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c140a0410 con 0x7f2c1409d630 2023-09-30T22:18:02.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.447+0000 7f2c117fa700 1 -- 172.21.15.89:0/3933550646 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c08013470 con 0x7f2c1409d630 2023-09-30T22:18:02.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.447+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c140747f0 con 0x7f2c1409d630 2023-09-30T22:18:02.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.447+0000 7f2c117fa700 1 -- 172.21.15.89:0/3933550646 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2c0803f890 con 0x7f2c1409d630 2023-09-30T22:18:02.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.456+0000 7f2c117fa700 1 -- 172.21.15.89:0/3933550646 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c08013720 con 0x7f2c1409d630 2023-09-30T22:18:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.501+0000 7f71d8672700 1 Processor -- start 2023-09-30T22:18:02.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.501+0000 7f71d8672700 1 -- start start 2023-09-30T22:18:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.501+0000 7f71d8672700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f71d00a1510 0x7f71d0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d8672700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d0139250 0x7f71d013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d8672700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d009edd0 con 0x7f71d00a0a30 2023-09-30T22:18:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d8672700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d00a18e0 con 0x7f71d00a1510 2023-09-30T22:18:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d8672700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d009e1f0 con 0x7f71d0139250 2023-09-30T22:18:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d6c0f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d0139250 0x7f71d013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d5c0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f71d00a1510 0x7f71d0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d6c0f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d0139250 0x7f71d013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58954/0 (socket says 172.21.15.89:58954) 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d6c0f700 1 -- 172.21.15.89:0/1912615395 learned_addr learned my addr 172.21.15.89:0/1912615395 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d6c0f700 1 -- 172.21.15.89:0/1912615395 >> v2:172.21.15.89:3300/0 conn(0x7f71d00a1510 msgr2=0x7f71d0138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d6c0f700 1 --2- 172.21.15.89:0/1912615395 >> v2:172.21.15.89:3300/0 conn(0x7f71d00a1510 0x7f71d0138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d6c0f700 1 -- 172.21.15.89:0/1912615395 >> v1:172.21.15.89:6789/0 conn(0x7f71d00a0a30 legacy=0x7f71d00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d6c0f700 1 -- 172.21.15.89:0/1912615395 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d009dc50 con 0x7f71d0139250 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d6c0f700 1 --2- 172.21.15.89:0/1912615395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d0139250 0x7f71d013b630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009e50 tx=0x7f71c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c06487550e52f368 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d540c700 1 -- 172.21.15.89:0/1912615395 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c800e070 con 0x7f71d0139250 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d540c700 1 -- 172.21.15.89:0/1912615395 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c800b910 con 0x7f71d0139250 2023-09-30T22:18:02.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.502+0000 7f71d540c700 1 -- 172.21.15.89:0/1912615395 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c800bcc0 con 0x7f71d0139250 2023-09-30T22:18:02.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.503+0000 7f71d8672700 1 -- 172.21.15.89:0/1912615395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d0139250 msgr2=0x7f71d013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.503+0000 7f71d8672700 1 --2- 172.21.15.89:0/1912615395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d0139250 0x7f71d013b630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009e50 tx=0x7f71c800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:02.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.503+0000 7f71d8672700 1 -- 172.21.15.89:0/1912615395 shutdown_connections 2023-09-30T22:18:02.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.503+0000 7f71d8672700 1 --2- 172.21.15.89:0/1912615395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d0139250 0x7f71d013b630 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.503+0000 7f71d8672700 1 --2- 172.21.15.89:0/1912615395 >> v2:172.21.15.89:3300/0 conn(0x7f71d00a1510 0x7f71d0138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.503+0000 7f71d8672700 1 -- 172.21.15.89:0/1912615395 >> 172.21.15.89:0/1912615395 conn(0x7f71d00a6550 msgr2=0x7f71d00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:02.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.503+0000 7f71d8672700 1 -- 172.21.15.89:0/1912615395 shutdown_connections 2023-09-30T22:18:02.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.503+0000 7f71d8672700 1 -- 172.21.15.89:0/1912615395 wait complete. 2023-09-30T22:18:02.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d8672700 1 Processor -- start 2023-09-30T22:18:02.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d8672700 1 -- start start 2023-09-30T22:18:02.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d8672700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71d00a0a30 0x7f71d01c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d8672700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d00a1510 0x7f71d01c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d8672700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71d0139250 0x7f71d01c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d8672700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d0124820 con 0x7f71d0139250 2023-09-30T22:18:02.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d8672700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d0125d40 con 0x7f71d00a1510 2023-09-30T22:18:02.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d8672700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71d01253a0 con 0x7f71d00a0a30 2023-09-30T22:18:02.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d640e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71d00a0a30 0x7f71d01c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.505+0000 7f71d640e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71d00a0a30 0x7f71d01c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36248/0 (socket says 172.21.15.89:36248) 2023-09-30T22:18:02.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d640e700 1 -- 172.21.15.89:0/993205181 learned_addr learned my addr 172.21.15.89:0/993205181 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:02.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d5c0d700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d00a1510 0x7f71d01c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d6c0f700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71d0139250 0x7f71d01c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d640e700 1 -- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d00a1510 msgr2=0x7f71d01c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d640e700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d00a1510 0x7f71d01c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d640e700 1 -- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71d0139250 msgr2=0x7f71d01c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d640e700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71d0139250 0x7f71d01c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d640e700 1 -- 172.21.15.89:0/993205181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c8009b30 con 0x7f71d00a0a30 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d640e700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71d00a0a30 0x7f71d01c8540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00c4e0 tx=0x7f71cc00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71c6ffd700 1 -- 172.21.15.89:0/993205181 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0052d0 con 0x7f71d00a0a30 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71c6ffd700 1 -- 172.21.15.89:0/993205181 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc005430 con 0x7f71d00a0a30 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d0126010 con 0x7f71d00a0a30 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71c6ffd700 1 -- 172.21.15.89:0/993205181 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0038c0 con 0x7f71d00a0a30 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.506+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d0126d40 con 0x7f71d00a0a30 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.507+0000 7f71c6ffd700 1 -- 172.21.15.89:0/993205181 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71cc003a20 con 0x7f71d00a0a30 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.507+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d0074ea0 con 0x7f71d00a0a30 2023-09-30T22:18:02.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.508+0000 7f71c6ffd700 1 -- 172.21.15.89:0/993205181 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f71cc03ea70 con 0x7f71d00a0a30 2023-09-30T22:18:02.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.512+0000 7f71c6ffd700 1 -- 172.21.15.89:0/993205181 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71cc040e60 con 0x7f71d00a0a30 2023-09-30T22:18:02.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.542+0000 7fba5c461700 1 Processor -- start 2023-09-30T22:18:02.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.542+0000 7fba5c461700 1 -- start start 2023-09-30T22:18:02.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.542+0000 7fba5c461700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fba541338a0 0x7fba54133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.542+0000 7fba5c461700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba5409de50 0x7fba5409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.542+0000 7fba5c461700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5406ef30 con 0x7fba5409e760 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.542+0000 7fba5c461700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5406fef0 con 0x7fba541338a0 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.542+0000 7fba5c461700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba541334b0 con 0x7fba5409de50 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba599fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba5409de50 0x7fba5409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba599fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba5409de50 0x7fba5409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36254/0 (socket says 172.21.15.89:36254) 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba599fc700 1 -- 172.21.15.89:0/4097471096 learned_addr learned my addr 172.21.15.89:0/4097471096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba5a1fd700 1 --2- 172.21.15.89:0/4097471096 >> v2:172.21.15.89:3300/0 conn(0x7fba541338a0 0x7fba54133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba5a1fd700 1 -- 172.21.15.89:0/4097471096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba5409de50 msgr2=0x7fba5409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba5a1fd700 1 --2- 172.21.15.89:0/4097471096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba5409de50 0x7fba5409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba5a1fd700 1 -- 172.21.15.89:0/4097471096 >> v1:172.21.15.89:6789/0 conn(0x7fba5409e760 legacy=0x7fba5413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba5a1fd700 1 -- 172.21.15.89:0/4097471096 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5406fc30 con 0x7fba541338a0 2023-09-30T22:18:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba599fc700 1 --2- 172.21.15.89:0/4097471096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba5409de50 0x7fba5409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba5a1fd700 1 --2- 172.21.15.89:0/4097471096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba541338a0 0x7fba54133c70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fba50009840 tx=0x7fba50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7362c11ae39c2eb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba591fb700 1 -- 172.21.15.89:0/4097471096 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50011830 con 0x7fba541338a0 2023-09-30T22:18:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba591fb700 1 -- 172.21.15.89:0/4097471096 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba50011990 con 0x7fba541338a0 2023-09-30T22:18:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.543+0000 7fba591fb700 1 -- 172.21.15.89:0/4097471096 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50011d20 con 0x7fba541338a0 2023-09-30T22:18:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.544+0000 7fba5c461700 1 -- 172.21.15.89:0/4097471096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba541338a0 msgr2=0x7fba54133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.544+0000 7fba5c461700 1 --2- 172.21.15.89:0/4097471096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba541338a0 0x7fba54133c70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fba50009840 tx=0x7fba50009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.544+0000 7fba5c461700 1 -- 172.21.15.89:0/4097471096 shutdown_connections 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.544+0000 7fba5c461700 1 --2- 172.21.15.89:0/4097471096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba541338a0 0x7fba54133c70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.544+0000 7fba5c461700 1 --2- 172.21.15.89:0/4097471096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba5409de50 0x7fba5409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.544+0000 7fba5c461700 1 -- 172.21.15.89:0/4097471096 >> 172.21.15.89:0/4097471096 conn(0x7fba540a2aa0 msgr2=0x7fba540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.544+0000 7fba5c461700 1 -- 172.21.15.89:0/4097471096 shutdown_connections 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.544+0000 7fba5c461700 1 -- 172.21.15.89:0/4097471096 wait complete. 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5c461700 1 Processor -- start 2023-09-30T22:18:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5c461700 1 -- start start 2023-09-30T22:18:02.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5c461700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba5409de50 0x7fba5411ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5c461700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5409e760 0x7fba5411f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5c461700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba541338a0 0x7fba5411f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:02.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5c461700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba54120830 con 0x7fba5409de50 2023-09-30T22:18:02.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5c461700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540a10f0 con 0x7fba5409e760 2023-09-30T22:18:02.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5c461700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba5406f640 con 0x7fba541338a0 2023-09-30T22:18:02.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5a9fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba541338a0 0x7fba5411f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5a1fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba5409de50 0x7fba5411ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:02.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5a9fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba541338a0 0x7fba5411f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36258/0 (socket says 172.21.15.89:36258) 2023-09-30T22:18:02.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5a1fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba5409de50 0x7fba5411ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43210/0 (socket says 172.21.15.89:43210) 2023-09-30T22:18:02.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.545+0000 7fba5a9fe700 1 -- 172.21.15.89:0/2982164740 learned_addr learned my addr 172.21.15.89:0/2982164740 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:02.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5a1fd700 1 -- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba541338a0 msgr2=0x7fba5411f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5a1fd700 1 --2- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba541338a0 0x7fba5411f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5a1fd700 1 -- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5409e760 msgr2=0x7fba5411f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5a1fd700 1 --2- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5409e760 0x7fba5411f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5a1fd700 1 -- 172.21.15.89:0/2982164740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44009550 con 0x7fba5409de50 2023-09-30T22:18:02.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5a9fe700 1 --2- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba541338a0 0x7fba5411f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:02.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5a1fd700 1 --2- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba5409de50 0x7fba5411ec20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fba50016000 tx=0x7fba50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:02.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba4affd700 1 -- 172.21.15.89:0/2982164740 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500036a0 con 0x7fba5409de50 2023-09-30T22:18:02.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba4affd700 1 -- 172.21.15.89:0/2982164740 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5001f070 con 0x7fba5409de50 2023-09-30T22:18:02.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba4affd700 1 -- 172.21.15.89:0/2982164740 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50003a60 con 0x7fba5409de50 2023-09-30T22:18:02.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba500094a0 con 0x7fba5409de50 2023-09-30T22:18:02.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.546+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5412f740 con 0x7fba5409de50 2023-09-30T22:18:02.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.547+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba54122100 con 0x7fba5409de50 2023-09-30T22:18:02.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.548+0000 7fba4affd700 1 -- 172.21.15.89:0/2982164740 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba50003bc0 con 0x7fba5409de50 2023-09-30T22:18:02.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.548+0000 7fba4affd700 1 -- 172.21.15.89:0/2982164740 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fba50021070 con 0x7fba5409de50 2023-09-30T22:18:02.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.552+0000 7fba4affd700 1 -- 172.21.15.89:0/2982164740 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba5001a430 con 0x7fba5409de50 2023-09-30T22:18:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.651+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2c14072980 con 0x7f2c1409d630 2023-09-30T22:18:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.651+0000 7f2c117fa700 1 -- 172.21.15.89:0/3933550646 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c08021070 con 0x7f2c1409d630 2023-09-30T22:18:02.651 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:02.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409d630 msgr2=0x7f2c14133890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409d630 0x7f2c14133890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800ec10 tx=0x7f2c0800ef20 comp rx=0 tx=0).stop 2023-09-30T22:18:02.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 shutdown_connections 2023-09-30T22:18:02.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14138d80 0x7f2c14134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409d630 0x7f2c14133890 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 --2- 172.21.15.89:0/3933550646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c140a0a30 0x7f2c14133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 >> 172.21.15.89:0/3933550646 conn(0x7f2c140a3050 msgr2=0x7f2c1409f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:02.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 shutdown_connections 2023-09-30T22:18:02.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.653+0000 7f2c1ad9e700 1 -- 172.21.15.89:0/3933550646 wait complete. 2023-09-30T22:18:02.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:02.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.716+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f71d006d540 con 0x7f71d00a0a30 2023-09-30T22:18:02.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.717+0000 7f71c6ffd700 1 -- 172.21.15.89:0/993205181 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71cc017070 con 0x7f71d00a0a30 2023-09-30T22:18:02.717 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71d00a0a30 msgr2=0x7f71d01c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71d00a0a30 0x7f71d01c8540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00c4e0 tx=0x7f71cc00c7f0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 shutdown_connections 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71d0139250 0x7f71d01c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71d00a0a30 0x7f71d01c8540 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 --2- 172.21.15.89:0/993205181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71d00a1510 0x7f71d01c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 >> 172.21.15.89:0/993205181 conn(0x7f71d00a6550 msgr2=0x7f71d00a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 shutdown_connections 2023-09-30T22:18:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.719+0000 7f71d8672700 1 -- 172.21.15.89:0/993205181 wait complete. 2023-09-30T22:18:02.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:02.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.754+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fba54122100 con 0x7fba5409de50 2023-09-30T22:18:02.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.754+0000 7fba4affd700 1 -- 172.21.15.89:0/2982164740 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba54122100 con 0x7fba5409de50 2023-09-30T22:18:02.754 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:02.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.756+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba5409de50 msgr2=0x7fba5411ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:02.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.756+0000 7fba5c461700 1 --2- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba5409de50 0x7fba5411ec20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fba50016000 tx=0x7fba50004750 comp rx=0 tx=0).stop 2023-09-30T22:18:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.756+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 shutdown_connections 2023-09-30T22:18:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.756+0000 7fba5c461700 1 --2- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba541338a0 0x7fba5411f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.756+0000 7fba5c461700 1 --2- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba5409de50 0x7fba5411ec20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.756+0000 7fba5c461700 1 --2- 172.21.15.89:0/2982164740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5409e760 0x7fba5411f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.756+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 >> 172.21.15.89:0/2982164740 conn(0x7fba540a2aa0 msgr2=0x7fba5409ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.757+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 shutdown_connections 2023-09-30T22:18:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:02.757+0000 7fba5c461700 1 -- 172.21.15.89:0/2982164740 wait complete. 2023-09-30T22:18:02.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:02.889 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:03.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5368692700 1 Processor -- start 2023-09-30T22:18:03.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5368692700 1 -- start start 2023-09-30T22:18:03.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5368692700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f536009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:03.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5368692700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f536009e000 0x7f53601339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5368692700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e410 0x7f5360133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5368692700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536006b460 con 0x7f536009e000 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5368692700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536009b120 con 0x7f536009e410 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5368692700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f536009a7e0 con 0x7f536009d6f0 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5366c2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e410 0x7f5360133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5365c2d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f536009e000 0x7f53601339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f536642e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f536009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5366c2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e410 0x7f5360133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58972/0 (socket says 172.21.15.89:58972) 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5365c2d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f536009e000 0x7f53601339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43214/0 (socket says 172.21.15.89:43214) 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.012+0000 7f5366c2f700 1 -- 172.21.15.89:0/2382540060 learned_addr learned my addr 172.21.15.89:0/2382540060 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:03.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5366c2f700 1 -- 172.21.15.89:0/2382540060 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 msgr2=0x7f536009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5366c2f700 1 --2- 172.21.15.89:0/2382540060 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f536009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5366c2f700 1 -- 172.21.15.89:0/2382540060 >> v2:172.21.15.89:3300/0 conn(0x7f536009e000 msgr2=0x7f53601339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5366c2f700 1 --2- 172.21.15.89:0/2382540060 >> v2:172.21.15.89:3300/0 conn(0x7f536009e000 0x7f53601339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5366c2f700 1 -- 172.21.15.89:0/2382540060 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536009a500 con 0x7f536009e410 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5365c2d700 1 --2- 172.21.15.89:0/2382540060 >> v2:172.21.15.89:3300/0 conn(0x7f536009e000 0x7f53601339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5366c2f700 1 --2- 172.21.15.89:0/2382540060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e410 0x7f5360133f30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5358009e80 tx=0x7f535800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ce4e3029a559e2a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f536542c700 1 -- 172.21.15.89:0/2382540060 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535800e070 con 0x7f536009e410 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f536542c700 1 -- 172.21.15.89:0/2382540060 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535800b8d0 con 0x7f536009e410 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f536542c700 1 -- 172.21.15.89:0/2382540060 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535800bc80 con 0x7f536009e410 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5368692700 1 -- 172.21.15.89:0/2382540060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e410 msgr2=0x7f5360133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.013+0000 7f5368692700 1 --2- 172.21.15.89:0/2382540060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e410 0x7f5360133f30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5358009e80 tx=0x7f535800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:03.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.014+0000 7f5368692700 1 -- 172.21.15.89:0/2382540060 shutdown_connections 2023-09-30T22:18:03.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.014+0000 7f5368692700 1 --2- 172.21.15.89:0/2382540060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e410 0x7f5360133f30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.014+0000 7f5368692700 1 --2- 172.21.15.89:0/2382540060 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f536009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.014+0000 7f5368692700 1 --2- 172.21.15.89:0/2382540060 >> v2:172.21.15.89:3300/0 conn(0x7f536009e000 0x7f53601339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.014+0000 7f5368692700 1 -- 172.21.15.89:0/2382540060 >> 172.21.15.89:0/2382540060 conn(0x7f5360066f50 msgr2=0x7f536009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:03.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.014+0000 7f5368692700 1 -- 172.21.15.89:0/2382540060 shutdown_connections 2023-09-30T22:18:03.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.014+0000 7f5368692700 1 -- 172.21.15.89:0/2382540060 wait complete. 2023-09-30T22:18:03.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.014+0000 7f5368692700 1 Processor -- start 2023-09-30T22:18:03.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5368692700 1 -- start start 2023-09-30T22:18:03.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5368692700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f53601a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:03.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5368692700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e000 0x7f53601a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:03.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5368692700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536009e410 0x7f53601c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:03.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5368692700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53601a8250 con 0x7f536009e410 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5368692700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53601a8430 con 0x7f536009e000 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5368692700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53601b92a0 con 0x7f536009d6f0 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f53601a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5365c2d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e000 0x7f53601a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f53601a52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36282/0 (socket says 172.21.15.89:36282) 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 -- 172.21.15.89:0/3032371114 learned_addr learned my addr 172.21.15.89:0/3032371114 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5366c2f700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536009e410 0x7f53601c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 -- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e000 msgr2=0x7f53601a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e000 0x7f53601a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 -- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536009e410 msgr2=0x7f53601c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536009e410 0x7f53601c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 -- 172.21.15.89:0/3032371114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0094a0 con 0x7f536009d6f0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f5365c2d700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e000 0x7f53601a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.015+0000 7f536642e700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f53601a52e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f535400e9e0 tx=0x7f535400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.016+0000 7f5352ffd700 1 -- 172.21.15.89:0/3032371114 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354009710 con 0x7f536009d6f0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.016+0000 7f5352ffd700 1 -- 172.21.15.89:0/3032371114 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400ee80 con 0x7f536009d6f0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.016+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5358009b30 con 0x7f536009d6f0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.016+0000 7f5352ffd700 1 -- 172.21.15.89:0/3032371114 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400ca50 con 0x7f536009d6f0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.016+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536009c710 con 0x7f536009d6f0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.016+0000 7f5352ffd700 1 -- 172.21.15.89:0/3032371114 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f535400cbb0 con 0x7f536009d6f0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.016+0000 7f5352ffd700 1 -- 172.21.15.89:0/3032371114 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f535403f700 con 0x7f536009d6f0 2023-09-30T22:18:03.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.017+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5360074ea0 con 0x7f536009d6f0 2023-09-30T22:18:03.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.021+0000 7f5352ffd700 1 -- 172.21.15.89:0/3032371114 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53540232f0 con 0x7f536009d6f0 2023-09-30T22:18:03.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.203+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53601a6d10 con 0x7f536009d6f0 2023-09-30T22:18:03.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.204+0000 7f5352ffd700 1 -- 172.21.15.89:0/3032371114 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f535401ae80 con 0x7f536009d6f0 2023-09-30T22:18:03.204 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:03.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 msgr2=0x7f53601a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:03.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f53601a52e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f535400e9e0 tx=0x7f535400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 shutdown_connections 2023-09-30T22:18:03.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536009e410 0x7f53601c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f536009d6f0 0x7f53601a52e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 --2- 172.21.15.89:0/3032371114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f536009e000 0x7f53601a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:03.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 >> 172.21.15.89:0/3032371114 conn(0x7f5360066f50 msgr2=0x7f53601bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:03.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 shutdown_connections 2023-09-30T22:18:03.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:03.206+0000 7f5368692700 1 -- 172.21.15.89:0/3032371114 wait complete. 2023-09-30T22:18:03.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:03.666 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:04.053 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:04.053 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.186+0000 7fb9e7830700 1 Processor -- start 2023-09-30T22:18:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.186+0000 7f900b59e700 1 Processor -- start 2023-09-30T22:18:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.186+0000 7f900b59e700 1 -- start start 2023-09-30T22:18:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e7830700 1 -- start start 2023-09-30T22:18:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f34d709a700 1 Processor -- start 2023-09-30T22:18:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f34d709a700 1 -- start start 2023-09-30T22:18:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e7830700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e7830700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9e009f240 0x7fb9e01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e7830700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e009f650 0x7fb9e0131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e7830700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e0070940 con 0x7fb9e009f240 2023-09-30T22:18:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e7830700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e006fd60 con 0x7fb9e009f650 2023-09-30T22:18:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e7830700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e006fa80 con 0x7fb9e009e930 2023-09-30T22:18:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f900b59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 0x7f900c09dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f900b59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c09e110 0x7f900c133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f900b59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f900c133da0 0x7f900c134210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f900b59e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c09f560 con 0x7f900c133da0 2023-09-30T22:18:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f900b59e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c099e10 con 0x7f900c09d800 2023-09-30T22:18:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f900b59e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f900c06b4d0 con 0x7f900c09e110 2023-09-30T22:18:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e4dcb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9e009f240 0x7fb9e01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e4dcb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9e009f240 0x7fb9e01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43238/0 (socket says 172.21.15.89:43238) 2023-09-30T22:18:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7fb9e4dcb700 1 -- 172.21.15.89:0/1647100737 learned_addr learned my addr 172.21.15.89:0/1647100737 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f34d709a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f34d709a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f34d009d9f0 0x7f34d006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.187+0000 7f900ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f900c133da0 0x7f900c134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f900ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f900c133da0 0x7f900c134210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43242/0 (socket says 172.21.15.89:43242) 2023-09-30T22:18:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f900ad9d700 1 -- 172.21.15.89:0/3253970293 learned_addr learned my addr 172.21.15.89:0/3253970293 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d709a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d009e130 con 0x7f34d006acb0 2023-09-30T22:18:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d709a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d01312b0 con 0x7f34d009d9f0 2023-09-30T22:18:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d709a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34d009f6e0 con 0x7f34d01316c0 2023-09-30T22:18:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d4e36700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d4e36700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36308/0 (socket says 172.21.15.89:36308) 2023-09-30T22:18:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d4e36700 1 -- 172.21.15.89:0/3448144436 learned_addr learned my addr 172.21.15.89:0/3448144436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34cffff700 1 --2- 172.21.15.89:0/3448144436 >> v2:172.21.15.89:3300/0 conn(0x7f34d009d9f0 0x7f34d006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f9009d9b700 1 --2- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c09e110 0x7f900c133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d4e36700 1 -- 172.21.15.89:0/3448144436 >> v2:172.21.15.89:3300/0 conn(0x7f34d009d9f0 msgr2=0x7f34d006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d4e36700 1 --2- 172.21.15.89:0/3448144436 >> v2:172.21.15.89:3300/0 conn(0x7f34d009d9f0 0x7f34d006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d4e36700 1 -- 172.21.15.89:0/3448144436 >> v1:172.21.15.89:6789/0 conn(0x7f34d006acb0 legacy=0x7f34d0138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34cf7fe700 1 -- 172.21.15.89:0/3448144436 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173453698 0 0) 0x7f34c8001490 con 0x7f34d006acb0 2023-09-30T22:18:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d4e36700 1 -- 172.21.15.89:0/3448144436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d009e760 con 0x7f34d01316c0 2023-09-30T22:18:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7fb9e55cc700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7fb9e4dcb700 1 -- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 msgr2=0x7fb9e009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7fb9e4dcb700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7fb9e4dcb700 1 -- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e009f650 msgr2=0x7fb9e0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7fb9e4dcb700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e009f650 0x7fb9e0131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7fb9e4dcb700 1 -- 172.21.15.89:0/1647100737 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e009fb60 con 0x7fb9e009f240 2023-09-30T22:18:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7fb9e55cc700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f900ad9d700 1 -- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c09e110 msgr2=0x7f900c133860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f900ad9d700 1 --2- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c09e110 0x7f900c133860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f900ad9d700 1 -- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 msgr2=0x7f900c09dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f900ad9d700 1 --2- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 0x7f900c09dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f900ad9d700 1 -- 172.21.15.89:0/3253970293 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c09a060 con 0x7f900c133da0 2023-09-30T22:18:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34d4e36700 1 --2- 172.21.15.89:0/3448144436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0131a90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f34c40098f0 tx=0x7f34c4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c051ff2f8d4c67fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7fb9e4dcb700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 0x7fb9e01317f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00098f0 tx=0x7fb9d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c872ad169fbf3f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f900ad9d700 1 --2- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c133da0 0x7f900c134210 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9004009e50 tx=0x7f900400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af76f6c398ac202f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.188+0000 7f34cf7fe700 1 -- 172.21.15.89:0/3448144436 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4011830 con 0x7f34d01316c0 2023-09-30T22:18:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34cf7fe700 1 -- 172.21.15.89:0/3448144436 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4011990 con 0x7f34d01316c0 2023-09-30T22:18:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34cf7fe700 1 -- 172.21.15.89:0/3448144436 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4011d20 con 0x7f34d01316c0 2023-09-30T22:18:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9d7fff700 1 -- 172.21.15.89:0/1647100737 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000b8a0 con 0x7fb9e009f240 2023-09-30T22:18:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9d7fff700 1 -- 172.21.15.89:0/1647100737 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d000ba00 con 0x7fb9e009f240 2023-09-30T22:18:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9d7fff700 1 -- 172.21.15.89:0/1647100737 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000bd90 con 0x7fb9e009f240 2023-09-30T22:18:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f900959a700 1 -- 172.21.15.89:0/3253970293 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400e070 con 0x7f900c133da0 2023-09-30T22:18:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f900959a700 1 -- 172.21.15.89:0/3253970293 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400b910 con 0x7f900c133da0 2023-09-30T22:18:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f900959a700 1 -- 172.21.15.89:0/3253970293 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400bcc0 con 0x7f900c133da0 2023-09-30T22:18:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34d709a700 1 -- 172.21.15.89:0/3448144436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 msgr2=0x7f34d0131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34d709a700 1 --2- 172.21.15.89:0/3448144436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0131a90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f34c40098f0 tx=0x7f34c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9e7830700 1 -- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 msgr2=0x7fb9e01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9e7830700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 0x7fb9e01317f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00098f0 tx=0x7fb9d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34d709a700 1 -- 172.21.15.89:0/3448144436 shutdown_connections 2023-09-30T22:18:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34d709a700 1 --2- 172.21.15.89:0/3448144436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0131a90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34d709a700 1 --2- 172.21.15.89:0/3448144436 >> v2:172.21.15.89:3300/0 conn(0x7f34d009d9f0 0x7f34d006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34d709a700 1 -- 172.21.15.89:0/3448144436 >> 172.21.15.89:0/3448144436 conn(0x7f34d0066f50 msgr2=0x7f34d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34d709a700 1 -- 172.21.15.89:0/3448144436 shutdown_connections 2023-09-30T22:18:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9e7830700 1 -- 172.21.15.89:0/1647100737 shutdown_connections 2023-09-30T22:18:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9e7830700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e009f650 0x7fb9e0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9e7830700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7f34d709a700 1 -- 172.21.15.89:0/3448144436 wait complete. 2023-09-30T22:18:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9e7830700 1 --2- 172.21.15.89:0/1647100737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 0x7fb9e01317f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.189+0000 7fb9e7830700 1 -- 172.21.15.89:0/1647100737 >> 172.21.15.89:0/1647100737 conn(0x7fb9e00a4210 msgr2=0x7fb9e00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.190+0000 7fb9e7830700 1 -- 172.21.15.89:0/1647100737 shutdown_connections 2023-09-30T22:18:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.190+0000 7fb9e7830700 1 -- 172.21.15.89:0/1647100737 wait complete. 2023-09-30T22:18:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.190+0000 7f900b59e700 1 -- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c133da0 msgr2=0x7f900c134210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.190+0000 7f900b59e700 1 --2- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c133da0 0x7f900c134210 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9004009e50 tx=0x7f900400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.190+0000 7f34d709a700 1 Processor -- start 2023-09-30T22:18:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.190+0000 7f34d709a700 1 -- start start 2023-09-30T22:18:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.190+0000 7fb9e7830700 1 Processor -- start 2023-09-30T22:18:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d709a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34d006acb0 0x7f34d0070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d709a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34d009d9f0 0x7f34d0071040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d709a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d709a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d011aaa0 con 0x7f34d006acb0 2023-09-30T22:18:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d709a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d011bc20 con 0x7f34d009d9f0 2023-09-30T22:18:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d709a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34d00a37f0 con 0x7f34d01316c0 2023-09-30T22:18:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e7830700 1 -- start start 2023-09-30T22:18:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e7830700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e01a7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e7830700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 0x7fb9e01a7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e7830700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e009f650 0x7fb9e006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e7830700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01a5240 con 0x7fb9e009f240 2023-09-30T22:18:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e7830700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01a4310 con 0x7fb9e009f650 2023-09-30T22:18:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e7830700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01a5520 con 0x7fb9e009e930 2023-09-30T22:18:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e55cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e01a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e55cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e01a7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36320/0 (socket says 172.21.15.89:36320) 2023-09-30T22:18:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e55cc700 1 -- 172.21.15.89:0/3821070121 learned_addr learned my addr 172.21.15.89:0/3821070121 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d4e36700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34d006acb0 0x7f34d0070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d4e36700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34d006acb0 0x7f34d0070b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43276/0 (socket says 172.21.15.89:43276) 2023-09-30T22:18:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d4e36700 1 -- 172.21.15.89:0/59228646 learned_addr learned my addr 172.21.15.89:0/59228646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34d5637700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e4dcb700 1 --2- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 0x7fb9e01a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e55cc700 1 -- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e009f650 msgr2=0x7fb9e006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e55cc700 1 --2- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e009f650 0x7fb9e006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e55cc700 1 -- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 msgr2=0x7fb9e01a7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e55cc700 1 --2- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 0x7fb9e01a7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7fb9e55cc700 1 -- 172.21.15.89:0/3821070121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d0009550 con 0x7fb9e009e930 2023-09-30T22:18:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.191+0000 7f34cffff700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34d009d9f0 0x7f34d0071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7fb9e55cc700 1 --2- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e01a7340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c750 tx=0x7fb9dc00cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34d4e36700 1 -- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 msgr2=0x7f34d0071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34d4e36700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34d4e36700 1 -- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34d009d9f0 msgr2=0x7f34d0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34d4e36700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34d009d9f0 0x7f34d0071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34d4e36700 1 -- 172.21.15.89:0/59228646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4009550 con 0x7f34d006acb0 2023-09-30T22:18:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7fb9d5ffb700 1 -- 172.21.15.89:0/3821070121 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc011800 con 0x7fb9e009e930 2023-09-30T22:18:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7fb9d5ffb700 1 -- 172.21.15.89:0/3821070121 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc011960 con 0x7fb9e009e930 2023-09-30T22:18:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7fb9d5ffb700 1 -- 172.21.15.89:0/3821070121 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc011cd0 con 0x7fb9e009e930 2023-09-30T22:18:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e00a0190 con 0x7fb9e009e930 2023-09-30T22:18:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34d4e36700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34d006acb0 0x7f34d0070b00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f34d009f820 tx=0x7f34c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e0071d20 con 0x7fb9e009e930 2023-09-30T22:18:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f900b59e700 1 -- 172.21.15.89:0/3253970293 shutdown_connections 2023-09-30T22:18:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f900b59e700 1 --2- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c133da0 0x7f900c134210 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f900b59e700 1 --2- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 0x7f900c09dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f900b59e700 1 --2- 172.21.15.89:0/3253970293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c09e110 0x7f900c133860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f900b59e700 1 -- 172.21.15.89:0/3253970293 >> 172.21.15.89:0/3253970293 conn(0x7f900c0a2e20 msgr2=0x7f900c0a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34cd7fa700 1 -- 172.21.15.89:0/59228646 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4004480 con 0x7f34d006acb0 2023-09-30T22:18:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d009ead0 con 0x7f34d006acb0 2023-09-30T22:18:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d00725d0 con 0x7f34d006acb0 2023-09-30T22:18:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7fb9d5ffb700 1 -- 172.21.15.89:0/3821070121 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb9dc021350 con 0x7fb9e009e930 2023-09-30T22:18:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34cd7fa700 1 -- 172.21.15.89:0/59228646 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4023070 con 0x7f34d006acb0 2023-09-30T22:18:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.192+0000 7f34cd7fa700 1 -- 172.21.15.89:0/59228646 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c401fa60 con 0x7f34d006acb0 2023-09-30T22:18:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.193+0000 7fb9d5ffb700 1 -- 172.21.15.89:0/3821070121 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb9dc020e20 con 0x7fb9e009e930 2023-09-30T22:18:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.193+0000 7f34cd7fa700 1 -- 172.21.15.89:0/59228646 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34c4025030 con 0x7f34d006acb0 2023-09-30T22:18:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.193+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b00052e0 con 0x7f34d006acb0 2023-09-30T22:18:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.194+0000 7f34cd7fa700 1 -- 172.21.15.89:0/59228646 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f34c4044e50 con 0x7f34d006acb0 2023-09-30T22:18:04.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.194+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e01a50a0 con 0x7fb9e009e930 2023-09-30T22:18:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.195+0000 7f900b59e700 1 -- 172.21.15.89:0/3253970293 shutdown_connections 2023-09-30T22:18:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.195+0000 7f900b59e700 1 -- 172.21.15.89:0/3253970293 wait complete. 2023-09-30T22:18:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900b59e700 1 Processor -- start 2023-09-30T22:18:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900b59e700 1 -- start start 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900b59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 0x7f900c125f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c09e110 0x7f900c126450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900b59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c133da0 0x7f900c126990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900b59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c129720 con 0x7f900c09e110 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900b59e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0a0420 con 0x7f900c09d800 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900b59e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f900c09c510 con 0x7f900c133da0 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c133da0 0x7f900c126990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c133da0 0x7f900c126990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36344/0 (socket says 172.21.15.89:36344) 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900ad9d700 1 -- 172.21.15.89:0/738032435 learned_addr learned my addr 172.21.15.89:0/738032435 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 0x7f900c125f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900ad9d700 1 -- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 msgr2=0x7f900c125f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900ad9d700 1 --2- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 0x7f900c125f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900ad9d700 1 -- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c09e110 msgr2=0x7f900c126450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900ad9d700 1 --2- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c09e110 0x7f900c126450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.196+0000 7f900ad9d700 1 -- 172.21.15.89:0/738032435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004009b30 con 0x7f900c133da0 2023-09-30T22:18:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.197+0000 7f900ad9d700 1 --2- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c133da0 0x7f900c126990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f900400b680 tx=0x7f9004004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.199+0000 7f8ffaffd700 1 -- 172.21.15.89:0/738032435 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004004450 con 0x7f900c133da0 2023-09-30T22:18:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.199+0000 7fb9d5ffb700 1 -- 172.21.15.89:0/3821070121 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb9e01a50a0 con 0x7fb9e009e930 2023-09-30T22:18:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.199+0000 7f34cd7fa700 1 -- 172.21.15.89:0/59228646 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34c4015080 con 0x7f34d006acb0 2023-09-30T22:18:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.199+0000 7f8ffaffd700 1 -- 172.21.15.89:0/738032435 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90040045b0 con 0x7f900c133da0 2023-09-30T22:18:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.199+0000 7f8ffaffd700 1 -- 172.21.15.89:0/738032435 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004004940 con 0x7f900c133da0 2023-09-30T22:18:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.199+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c132b10 con 0x7f900c133da0 2023-09-30T22:18:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.199+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c128000 con 0x7f900c133da0 2023-09-30T22:18:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.200+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c129070 con 0x7f900c133da0 2023-09-30T22:18:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.200+0000 7f8ffaffd700 1 -- 172.21.15.89:0/738032435 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f900401c410 con 0x7f900c133da0 2023-09-30T22:18:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.201+0000 7f8ffaffd700 1 -- 172.21.15.89:0/738032435 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9004041670 con 0x7f900c133da0 2023-09-30T22:18:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.209+0000 7f8ffaffd700 1 -- 172.21.15.89:0/738032435 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f900401c6c0 con 0x7f900c133da0 2023-09-30T22:18:04.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:04.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.347+0000 7f7ab4c7c700 1 Processor -- start 2023-09-30T22:18:04.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.347+0000 7f7ab4c7c700 1 -- start start 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7ab4c7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab01316c0 0x7f7ab0131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7ab4c7c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ab009d9f0 0x7f7ab006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7ab4c7c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab009e130 con 0x7f7ab006acb0 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7ab4c7c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab01312b0 con 0x7f7ab009d9f0 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7ab4c7c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab009f6e0 con 0x7f7ab01316c0 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7aadd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ab009d9f0 0x7f7ab006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7aae59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab01316c0 0x7f7ab0131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7aadd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ab009d9f0 0x7f7ab006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43294/0 (socket says 172.21.15.89:43294) 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7aadd9b700 1 -- 172.21.15.89:0/2481253019 learned_addr learned my addr 172.21.15.89:0/2481253019 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:04.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7aadd9b700 1 -- 172.21.15.89:0/2481253019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab01316c0 msgr2=0x7f7ab0131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:04.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7aadd9b700 1 --2- 172.21.15.89:0/2481253019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab01316c0 0x7f7ab0131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7aadd9b700 1 -- 172.21.15.89:0/2481253019 >> v1:172.21.15.89:6789/0 conn(0x7f7ab006acb0 legacy=0x7f7ab0138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.348+0000 7f7aadd9b700 1 -- 172.21.15.89:0/2481253019 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab009e760 con 0x7f7ab009d9f0 2023-09-30T22:18:04.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7aadd9b700 1 --2- 172.21.15.89:0/2481253019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab009d9f0 0x7f7ab006a770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40098f0 tx=0x7f7aa4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdc610c735e2367 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:04.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7aad59a700 1 -- 172.21.15.89:0/2481253019 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4011830 con 0x7f7ab009d9f0 2023-09-30T22:18:04.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7aad59a700 1 -- 172.21.15.89:0/2481253019 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4011990 con 0x7f7ab009d9f0 2023-09-30T22:18:04.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7aad59a700 1 -- 172.21.15.89:0/2481253019 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4011d20 con 0x7f7ab009d9f0 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/2481253019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab009d9f0 msgr2=0x7f7ab006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7ab4c7c700 1 --2- 172.21.15.89:0/2481253019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab009d9f0 0x7f7ab006a770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40098f0 tx=0x7f7aa4009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/2481253019 shutdown_connections 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7ab4c7c700 1 --2- 172.21.15.89:0/2481253019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab01316c0 0x7f7ab0131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7ab4c7c700 1 --2- 172.21.15.89:0/2481253019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab009d9f0 0x7f7ab006a770 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.349+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/2481253019 >> 172.21.15.89:0/2481253019 conn(0x7f7ab0066f50 msgr2=0x7f7ab009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.350+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/2481253019 shutdown_connections 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.350+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/2481253019 wait complete. 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7ab4c7c700 1 Processor -- start 2023-09-30T22:18:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7ab4c7c700 1 -- start start 2023-09-30T22:18:04.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7ab4c7c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ab006acb0 0x7f7ab0119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7ab4c7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab009d9f0 0x7f7ab01199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7ab4c7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab01316c0 0x7f7ab0119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:04.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7ab4c7c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab0072250 con 0x7f7ab01316c0 2023-09-30T22:18:04.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7ab4c7c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab011a690 con 0x7f7ab006acb0 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7ab4c7c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00a3540 con 0x7f7ab009d9f0 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7aadd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab009d9f0 0x7f7ab01199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7aaed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab01316c0 0x7f7ab0119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7aadd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab009d9f0 0x7f7ab01199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36356/0 (socket says 172.21.15.89:36356) 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.351+0000 7f7aadd9b700 1 -- 172.21.15.89:0/1292952523 learned_addr learned my addr 172.21.15.89:0/1292952523 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7aadd9b700 1 -- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ab006acb0 msgr2=0x7f7ab0119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7aadd9b700 1 --2- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ab006acb0 0x7f7ab0119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7aadd9b700 1 -- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab01316c0 msgr2=0x7f7ab0119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7aadd9b700 1 --2- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab01316c0 0x7f7ab0119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7aadd9b700 1 -- 172.21.15.89:0/1292952523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa4009550 con 0x7f7ab009d9f0 2023-09-30T22:18:04.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7aaed9d700 1 --2- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab01316c0 0x7f7ab0119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7aadd9b700 1 --2- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab009d9f0 0x7f7ab01199c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009c40 tx=0x7f7aa4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7a9effd700 1 -- 172.21.15.89:0/1292952523 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4004480 con 0x7f7ab009d9f0 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7a9effd700 1 -- 172.21.15.89:0/1292952523 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa401f070 con 0x7f7ab009d9f0 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab011b730 con 0x7f7ab009d9f0 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7a9effd700 1 -- 172.21.15.89:0/1292952523 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40038c0 con 0x7f7ab009d9f0 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.352+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab0071910 con 0x7f7ab009d9f0 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.353+0000 7f7a9effd700 1 -- 172.21.15.89:0/1292952523 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7aa4021030 con 0x7f7ab009d9f0 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.353+0000 7f7a9effd700 1 -- 172.21.15.89:0/1292952523 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7aa4040980 con 0x7f7ab009d9f0 2023-09-30T22:18:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.353+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab009fe70 con 0x7f7ab009d9f0 2023-09-30T22:18:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.358+0000 7f7a9effd700 1 -- 172.21.15.89:0/1292952523 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7aa4015080 con 0x7f7ab009d9f0 2023-09-30T22:18:04.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.397+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f34b0005590 con 0x7f34d006acb0 2023-09-30T22:18:04.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.397+0000 7f34cd7fa700 1 -- 172.21.15.89:0/59228646 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34c40036a0 con 0x7f34d006acb0 2023-09-30T22:18:04.397 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:04.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.399+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34d006acb0 msgr2=0x7f34d0070b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.399+0000 7f34d709a700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34d006acb0 0x7f34d0070b00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f34d009f820 tx=0x7f34c4004750 comp rx=0 tx=0).stop 2023-09-30T22:18:04.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.399+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 shutdown_connections 2023-09-30T22:18:04.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.399+0000 7f34d709a700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34d01316c0 0x7f34d0071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.399+0000 7f34d709a700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34d006acb0 0x7f34d0070b00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.399+0000 7f34d709a700 1 --2- 172.21.15.89:0/59228646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34d009d9f0 0x7f34d0071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.399+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 >> 172.21.15.89:0/59228646 conn(0x7f34d0066f50 msgr2=0x7f34d00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:04.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.400+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 shutdown_connections 2023-09-30T22:18:04.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.400+0000 7f34d709a700 1 -- 172.21.15.89:0/59228646 wait complete. 2023-09-30T22:18:04.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:04.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.509+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb9e01a50a0 con 0x7fb9e009e930 2023-09-30T22:18:04.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.510+0000 7fb9d5ffb700 1 -- 172.21.15.89:0/3821070121 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb9e01a50a0 con 0x7fb9e009e930 2023-09-30T22:18:04.510 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 msgr2=0x7fb9e01a7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 --2- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e01a7340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c750 tx=0x7fb9dc00cb10 comp rx=0 tx=0).stop 2023-09-30T22:18:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 shutdown_connections 2023-09-30T22:18:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 --2- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9e009f650 0x7fb9e006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 --2- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9e009e930 0x7fb9e01a7340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 --2- 172.21.15.89:0/3821070121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9e009f240 0x7fb9e01a7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 >> 172.21.15.89:0/3821070121 conn(0x7fb9e00a4210 msgr2=0x7fb9e01b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:04.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 shutdown_connections 2023-09-30T22:18:04.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.512+0000 7fb9e7830700 1 -- 172.21.15.89:0/3821070121 wait complete. 2023-09-30T22:18:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.515+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f900c04f2a0 con 0x7f900c133da0 2023-09-30T22:18:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.515+0000 7f8ffaffd700 1 -- 172.21.15.89:0/738032435 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f900402c390 con 0x7f900c133da0 2023-09-30T22:18:04.515 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.517+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c133da0 msgr2=0x7f900c126990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.517+0000 7f900b59e700 1 --2- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c133da0 0x7f900c126990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f900400b680 tx=0x7f9004004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.518+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 shutdown_connections 2023-09-30T22:18:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.518+0000 7f900b59e700 1 --2- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f900c133da0 0x7f900c126990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.518+0000 7f900b59e700 1 --2- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f900c09d800 0x7f900c125f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.518+0000 7f900b59e700 1 --2- 172.21.15.89:0/738032435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f900c09e110 0x7f900c126450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.518+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 >> 172.21.15.89:0/738032435 conn(0x7f900c0a2e20 msgr2=0x7f900c071620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.518+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 shutdown_connections 2023-09-30T22:18:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.518+0000 7f900b59e700 1 -- 172.21.15.89:0/738032435 wait complete. 2023-09-30T22:18:04.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:04.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.560+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ab0074ea0 con 0x7f7ab009d9f0 2023-09-30T22:18:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.560+0000 7f7a9effd700 1 -- 172.21.15.89:0/1292952523 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7aa4003cb0 con 0x7f7ab009d9f0 2023-09-30T22:18:04.560 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.562+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab009d9f0 msgr2=0x7f7ab01199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.562+0000 7f7ab4c7c700 1 --2- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab009d9f0 0x7f7ab01199c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009c40 tx=0x7f7aa4004750 comp rx=0 tx=0).stop 2023-09-30T22:18:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.563+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 shutdown_connections 2023-09-30T22:18:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.563+0000 7f7ab4c7c700 1 --2- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ab01316c0 0x7f7ab0119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.563+0000 7f7ab4c7c700 1 --2- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ab006acb0 0x7f7ab0119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.563+0000 7f7ab4c7c700 1 --2- 172.21.15.89:0/1292952523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ab009d9f0 0x7f7ab01199c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.563+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 >> 172.21.15.89:0/1292952523 conn(0x7f7ab0066f50 msgr2=0x7f7ab0130160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.563+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 shutdown_connections 2023-09-30T22:18:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:04.563+0000 7f7ab4c7c700 1 -- 172.21.15.89:0/1292952523 wait complete. 2023-09-30T22:18:04.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:05.413 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:05.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:05.531 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.545+0000 7fa3494f4700 1 Processor -- start 2023-09-30T22:18:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.545+0000 7fa3494f4700 1 -- start start 2023-09-30T22:18:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3494f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409e930 0x7fa34409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3494f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa34409f650 0x7fa344131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3494f4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa344070940 con 0x7fa34409f240 2023-09-30T22:18:05.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3494f4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa34406fd60 con 0x7fa34409f650 2023-09-30T22:18:05.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3494f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34406fa80 con 0x7fa34409e930 2023-09-30T22:18:05.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3427fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa34409f240 0x7fa3441317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38082/0 (socket says 172.21.15.89:38082) 2023-09-30T22:18:05.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3427fc700 1 -- 172.21.15.89:0/2489407543 learned_addr learned my addr 172.21.15.89:0/2489407543 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:05.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa342ffd700 1 --2- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409e930 0x7fa34409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3437fe700 1 --2- 172.21.15.89:0/2489407543 >> v2:172.21.15.89:3300/0 conn(0x7fa34409f650 0x7fa344131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3437fe700 1 -- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409e930 msgr2=0x7fa34409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:05.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3437fe700 1 --2- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409e930 0x7fa34409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3437fe700 1 -- 172.21.15.89:0/2489407543 >> v1:172.21.15.89:6789/0 conn(0x7fa34409f240 legacy=0x7fa3441317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3437fe700 1 -- 172.21.15.89:0/2489407543 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34409fb60 con 0x7fa34409f650 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa342ffd700 1 --2- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409e930 0x7fa34409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.546+0000 7fa3437fe700 1 --2- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409f650 0x7fa344131f00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009d90 tx=0x7fa33c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ed3c49f0710b6d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa341ffb700 1 -- 172.21.15.89:0/2489407543 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c00e070 con 0x7fa34409f650 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa341ffb700 1 -- 172.21.15.89:0/2489407543 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33c00bb20 con 0x7fa34409f650 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa341ffb700 1 -- 172.21.15.89:0/2489407543 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c012430 con 0x7fa34409f650 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa3494f4700 1 -- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409f650 msgr2=0x7fa344131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa3494f4700 1 --2- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409f650 0x7fa344131f00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009d90 tx=0x7fa33c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa3494f4700 1 -- 172.21.15.89:0/2489407543 shutdown_connections 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa3494f4700 1 --2- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409f650 0x7fa344131f00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa3494f4700 1 --2- 172.21.15.89:0/2489407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409e930 0x7fa34409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa3494f4700 1 -- 172.21.15.89:0/2489407543 >> 172.21.15.89:0/2489407543 conn(0x7fa3440a4210 msgr2=0x7fa3440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:05.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa3494f4700 1 -- 172.21.15.89:0/2489407543 shutdown_connections 2023-09-30T22:18:05.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.547+0000 7fa3494f4700 1 -- 172.21.15.89:0/2489407543 wait complete. 2023-09-30T22:18:05.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.549+0000 7fa3494f4700 1 Processor -- start 2023-09-30T22:18:05.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.549+0000 7fa3494f4700 1 -- start start 2023-09-30T22:18:05.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.549+0000 7fa3494f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409e930 0x7fa34406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.549+0000 7fa3494f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409f240 0x7fa34406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.549+0000 7fa3494f4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa34409f650 0x7fa34406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.549+0000 7fa3494f4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34406cd80 con 0x7fa34409e930 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.549+0000 7fa3494f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3441a88e0 con 0x7fa34409f240 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.549+0000 7fa3494f4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3441b71d0 con 0x7fa34409f650 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409e930 0x7fa34406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409e930 0x7fa34406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43316/0 (socket says 172.21.15.89:43316) 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 -- 172.21.15.89:0/1724981803 learned_addr learned my addr 172.21.15.89:0/1724981803 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa3427fc700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409f240 0x7fa34406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa3437fe700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa34409f650 0x7fa34406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 -- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa34409f650 msgr2=0x7fa34406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa34409f650 0x7fa34406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 -- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409f240 msgr2=0x7fa34406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409f240 0x7fa34406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 -- 172.21.15.89:0/1724981803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3340094a0 con 0x7fa34409e930 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa342ffd700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409e930 0x7fa34406b8b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa33400ec10 tx=0x7fa33400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa32b7fe700 1 -- 172.21.15.89:0/1724981803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400c850 con 0x7fa34409e930 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa32b7fe700 1 -- 172.21.15.89:0/1724981803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33400c9b0 con 0x7fa34409e930 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c009a70 con 0x7fa34409e930 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa32b7fe700 1 -- 172.21.15.89:0/1724981803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400cd20 con 0x7fa34409e930 2023-09-30T22:18:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.550+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3440a0190 con 0x7fa34409e930 2023-09-30T22:18:05.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.551+0000 7fa32b7fe700 1 -- 172.21.15.89:0/1724981803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa334013470 con 0x7fa34409e930 2023-09-30T22:18:05.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.551+0000 7fa32b7fe700 1 -- 172.21.15.89:0/1724981803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa33403fb60 con 0x7fa34409e930 2023-09-30T22:18:05.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.551+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3441a8c60 con 0x7fa34409e930 2023-09-30T22:18:05.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.557+0000 7fa32b7fe700 1 -- 172.21.15.89:0/1724981803 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa334013720 con 0x7fa34409e930 2023-09-30T22:18:05.575 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.657+0000 7f9c6301c700 1 Processor -- start 2023-09-30T22:18:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.657+0000 7f9c6301c700 1 -- start start 2023-09-30T22:18:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c6301c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c1338a0 0x7f9c5c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c6301c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c09de50 0x7f9c5c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c6301c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c06ef30 con 0x7f9c5c09e760 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c6301c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c06fef0 con 0x7f9c5c1338a0 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c6301c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1334b0 con 0x7f9c5c09de50 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c60db8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c1338a0 0x7f9c5c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c60db8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c1338a0 0x7f9c5c133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59040/0 (socket says 172.21.15.89:59040) 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c60db8700 1 -- 172.21.15.89:0/880001318 learned_addr learned my addr 172.21.15.89:0/880001318 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c5bfff700 1 --2- 172.21.15.89:0/880001318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c09de50 0x7f9c5c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c60db8700 1 -- 172.21.15.89:0/880001318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c09de50 msgr2=0x7f9c5c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c60db8700 1 --2- 172.21.15.89:0/880001318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c09de50 0x7f9c5c09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c60db8700 1 -- 172.21.15.89:0/880001318 >> v1:172.21.15.89:6789/0 conn(0x7f9c5c09e760 legacy=0x7f9c5c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.658+0000 7f9c60db8700 1 -- 172.21.15.89:0/880001318 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c06fc30 con 0x7f9c5c1338a0 2023-09-30T22:18:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.659+0000 7f9c60db8700 1 --2- 172.21.15.89:0/880001318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c1338a0 0x7f9c5c133c70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009920 tx=0x7f9c50009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b69f4d224eae74f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.659+0000 7f9c5b7fe700 1 -- 172.21.15.89:0/880001318 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50004230 con 0x7f9c5c1338a0 2023-09-30T22:18:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.659+0000 7f9c5b7fe700 1 -- 172.21.15.89:0/880001318 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50004390 con 0x7f9c5c1338a0 2023-09-30T22:18:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.659+0000 7f9c5b7fe700 1 -- 172.21.15.89:0/880001318 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500036a0 con 0x7f9c5c1338a0 2023-09-30T22:18:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.659+0000 7f9c6301c700 1 -- 172.21.15.89:0/880001318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c1338a0 msgr2=0x7f9c5c133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.659+0000 7f9c6301c700 1 --2- 172.21.15.89:0/880001318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c1338a0 0x7f9c5c133c70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009920 tx=0x7f9c50009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.660+0000 7f9c6301c700 1 -- 172.21.15.89:0/880001318 shutdown_connections 2023-09-30T22:18:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.660+0000 7f9c6301c700 1 --2- 172.21.15.89:0/880001318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c1338a0 0x7f9c5c133c70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.660+0000 7f9c6301c700 1 --2- 172.21.15.89:0/880001318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c09de50 0x7f9c5c09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.660+0000 7f9c6301c700 1 -- 172.21.15.89:0/880001318 >> 172.21.15.89:0/880001318 conn(0x7f9c5c0a2aa0 msgr2=0x7f9c5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.660+0000 7f9c6301c700 1 -- 172.21.15.89:0/880001318 shutdown_connections 2023-09-30T22:18:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.660+0000 7f9c6301c700 1 -- 172.21.15.89:0/880001318 wait complete. 2023-09-30T22:18:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.660+0000 7f9c6301c700 1 Processor -- start 2023-09-30T22:18:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c6301c700 1 -- start start 2023-09-30T22:18:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c6301c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c09de50 0x7f9c5c11eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c6301c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c5c09e760 0x7f9c5c11f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c6301c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c1338a0 0x7f9c5c11f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c6301c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c12f710 con 0x7f9c5c09e760 2023-09-30T22:18:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c6301c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c121d30 con 0x7f9c5c09de50 2023-09-30T22:18:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c6301c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c120720 con 0x7f9c5c1338a0 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c60db8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c09de50 0x7f9c5c11eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c60db8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c09de50 0x7f9c5c11eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59050/0 (socket says 172.21.15.89:59050) 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c60db8700 1 -- 172.21.15.89:0/3545896014 learned_addr learned my addr 172.21.15.89:0/3545896014 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c615b9700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c1338a0 0x7f9c5c11f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.661+0000 7f9c5bfff700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c5c09e760 0x7f9c5c11f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c60db8700 1 -- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c1338a0 msgr2=0x7f9c5c11f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c60db8700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c1338a0 0x7f9c5c11f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c60db8700 1 -- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c5c09e760 msgr2=0x7f9c5c11f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c60db8700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c5c09e760 0x7f9c5c11f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c60db8700 1 -- 172.21.15.89:0/3545896014 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50009550 con 0x7f9c5c09de50 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c60db8700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c09de50 0x7f9c5c11eb10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009f80 tx=0x7f9c50004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c597fa700 1 -- 172.21.15.89:0/3545896014 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50003d30 con 0x7f9c5c09de50 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c597fa700 1 -- 172.21.15.89:0/3545896014 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50011630 con 0x7f9c5c09de50 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c122630 con 0x7f9c5c09de50 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c597fa700 1 -- 172.21.15.89:0/3545896014 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500119e0 con 0x7f9c5c09de50 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.662+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c122350 con 0x7f9c5c09de50 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.663+0000 7f9c597fa700 1 -- 172.21.15.89:0/3545896014 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c50021070 con 0x7f9c5c09de50 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.663+0000 7f9c597fa700 1 -- 172.21.15.89:0/3545896014 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9c50040a30 con 0x7f9c5c09de50 2023-09-30T22:18:05.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.663+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c121fb0 con 0x7f9c5c09de50 2023-09-30T22:18:05.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.672+0000 7f9c597fa700 1 -- 172.21.15.89:0/3545896014 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c5c121fb0 con 0x7f9c5c09de50 2023-09-30T22:18:05.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.675+0000 7fa513153700 1 Processor -- start 2023-09-30T22:18:05.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.675+0000 7fa513153700 1 -- start start 2023-09-30T22:18:05.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa513153700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c09c650 0x7fa50c09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa513153700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c09cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa513153700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa50c131970 0x7fa50c131e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa513153700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c09ba70 con 0x7fa50c131970 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa513153700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c09d4a0 con 0x7fa50c09c650 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa513153700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c09e8f0 con 0x7fa50c0a0090 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa50bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c09cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa50bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c09cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36374/0 (socket says 172.21.15.89:36374) 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa50bfff700 1 -- 172.21.15.89:0/2147973043 learned_addr learned my addr 172.21.15.89:0/2147973043 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa510eef700 1 --2- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c09c650 0x7fa50c09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa5116f0700 1 --2- 172.21.15.89:0/2147973043 >> v2:172.21.15.89:3300/0 conn(0x7fa50c131970 0x7fa50c131e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa510eef700 1 -- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 msgr2=0x7fa50c09cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa510eef700 1 --2- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c09cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa510eef700 1 -- 172.21.15.89:0/2147973043 >> v2:172.21.15.89:3300/0 conn(0x7fa50c131970 msgr2=0x7fa50c131e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa510eef700 1 --2- 172.21.15.89:0/2147973043 >> v2:172.21.15.89:3300/0 conn(0x7fa50c131970 0x7fa50c131e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa510eef700 1 -- 172.21.15.89:0/2147973043 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0a0760 con 0x7fa50c09c650 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.676+0000 7fa50bfff700 1 --2- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c09cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa510eef700 1 --2- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c09c650 0x7fa50c09ca20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa500009870 tx=0x7fa500009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=796cd62e4b1179bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa50b7fe700 1 -- 172.21.15.89:0/2147973043 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500011830 con 0x7fa50c09c650 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa50b7fe700 1 -- 172.21.15.89:0/2147973043 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa500011990 con 0x7fa50c09c650 2023-09-30T22:18:05.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa50b7fe700 1 -- 172.21.15.89:0/2147973043 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500011d20 con 0x7fa50c09c650 2023-09-30T22:18:05.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa513153700 1 -- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c09c650 msgr2=0x7fa50c09ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa513153700 1 --2- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c09c650 0x7fa50c09ca20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa500009870 tx=0x7fa500009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:05.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa513153700 1 -- 172.21.15.89:0/2147973043 shutdown_connections 2023-09-30T22:18:05.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa513153700 1 --2- 172.21.15.89:0/2147973043 >> v2:172.21.15.89:3300/0 conn(0x7fa50c131970 0x7fa50c131e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa513153700 1 --2- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c09c650 0x7fa50c09ca20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa513153700 1 --2- 172.21.15.89:0/2147973043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c09cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.677+0000 7fa513153700 1 -- 172.21.15.89:0/2147973043 >> 172.21.15.89:0/2147973043 conn(0x7fa50c0a3f70 msgr2=0x7fa50c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.678+0000 7fa513153700 1 -- 172.21.15.89:0/2147973043 shutdown_connections 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.678+0000 7fa513153700 1 -- 172.21.15.89:0/2147973043 wait complete. 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.678+0000 7fa513153700 1 Processor -- start 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.678+0000 7fa513153700 1 -- start start 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa513153700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa50c09c650 0x7fa50c11a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa513153700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c11ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa513153700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c131970 0x7fa50c11b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa513153700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c11d870 con 0x7fa50c09c650 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa513153700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c12d570 con 0x7fa50c131970 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa513153700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c11b910 con 0x7fa50c0a0090 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa510eef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa50c09c650 0x7fa50c11a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa510eef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa50c09c650 0x7fa50c11a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43332/0 (socket says 172.21.15.89:43332) 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa510eef700 1 -- 172.21.15.89:0/3674452585 learned_addr learned my addr 172.21.15.89:0/3674452585 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa5116f0700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c131970 0x7fa50c11b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.679+0000 7fa50bfff700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c11ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa510eef700 1 -- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 msgr2=0x7fa50c11ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa510eef700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c11ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa510eef700 1 -- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c131970 msgr2=0x7fa50c11b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa510eef700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c131970 0x7fa50c11b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa510eef700 1 -- 172.21.15.89:0/3674452585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8009550 con 0x7fa50c09c650 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa50bfff700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c11ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa5116f0700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c131970 0x7fa50c11b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa510eef700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa50c09c650 0x7fa50c11a620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa500000c00 tx=0x7fa500004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa5097fa700 1 -- 172.21.15.89:0/3674452585 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500004510 con 0x7fa50c09c650 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa5097fa700 1 -- 172.21.15.89:0/3674452585 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50001f070 con 0x7fa50c09c650 2023-09-30T22:18:05.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5000094a0 con 0x7fa50c09c650 2023-09-30T22:18:05.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa5097fa700 1 -- 172.21.15.89:0/3674452585 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000038c0 con 0x7fa50c09c650 2023-09-30T22:18:05.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.680+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c12fd50 con 0x7fa50c09c650 2023-09-30T22:18:05.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.681+0000 7fa5097fa700 1 -- 172.21.15.89:0/3674452585 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa500003a20 con 0x7fa50c09c650 2023-09-30T22:18:05.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.681+0000 7fa5097fa700 1 -- 172.21.15.89:0/3674452585 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa500021070 con 0x7fa50c09c650 2023-09-30T22:18:05.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.681+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c11d3f0 con 0x7fa50c09c650 2023-09-30T22:18:05.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.686+0000 7fa5097fa700 1 -- 172.21.15.89:0/3674452585 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa50c11d3f0 con 0x7fa50c09c650 2023-09-30T22:18:05.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.718+0000 7f3d8cd7f700 1 Processor -- start 2023-09-30T22:18:05.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.718+0000 7f3d8cd7f700 1 -- start start 2023-09-30T22:18:05.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.718+0000 7f3d8cd7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d8809c650 0x7f3d8809ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.718+0000 7f3d8cd7f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d880a0090 0x7f3d8809cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.718+0000 7f3d8cd7f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8809ba70 con 0x7f3d88131970 2023-09-30T22:18:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.718+0000 7f3d8cd7f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8809d4a0 con 0x7f3d8809c650 2023-09-30T22:18:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.718+0000 7f3d8cd7f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8809e8f0 con 0x7f3d880a0090 2023-09-30T22:18:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d8809c650 0x7f3d8809ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d85d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d880a0090 0x7f3d8809cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d86d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3d88131970 0x7f3d88131e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38104/0 (socket says 172.21.15.89:38104) 2023-09-30T22:18:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d8809c650 0x7f3d8809ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59072/0 (socket says 172.21.15.89:59072) 2023-09-30T22:18:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d86d9d700 1 -- 172.21.15.89:0/108827337 learned_addr learned my addr 172.21.15.89:0/108827337 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8659c700 1 -- 172.21.15.89:0/108827337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d880a0090 msgr2=0x7f3d8809cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8659c700 1 --2- 172.21.15.89:0/108827337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d880a0090 0x7f3d8809cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8659c700 1 -- 172.21.15.89:0/108827337 >> v1:172.21.15.89:6789/0 conn(0x7f3d88131970 legacy=0x7f3d88131e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8659c700 1 -- 172.21.15.89:0/108827337 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880a0760 con 0x7f3d8809c650 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d85d9b700 1 --2- 172.21.15.89:0/108827337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d880a0090 0x7f3d8809cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8659c700 1 --2- 172.21.15.89:0/108827337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d8809c650 0x7f3d8809ca20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d780098a0 tx=0x7f3d78009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83db21973fbe5c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8559a700 1 -- 172.21.15.89:0/108827337 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78011830 con 0x7f3d8809c650 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8559a700 1 -- 172.21.15.89:0/108827337 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d78011990 con 0x7f3d8809c650 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.719+0000 7f3d8559a700 1 -- 172.21.15.89:0/108827337 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78011d20 con 0x7f3d8809c650 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.720+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/108827337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d8809c650 msgr2=0x7f3d8809ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.720+0000 7f3d8cd7f700 1 --2- 172.21.15.89:0/108827337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d8809c650 0x7f3d8809ca20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d780098a0 tx=0x7f3d78009bb0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.720+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/108827337 shutdown_connections 2023-09-30T22:18:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.720+0000 7f3d8cd7f700 1 --2- 172.21.15.89:0/108827337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d8809c650 0x7f3d8809ca20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.720+0000 7f3d8cd7f700 1 --2- 172.21.15.89:0/108827337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d880a0090 0x7f3d8809cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.720+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/108827337 >> 172.21.15.89:0/108827337 conn(0x7f3d880a3f70 msgr2=0x7f3d880a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.720+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/108827337 shutdown_connections 2023-09-30T22:18:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.720+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/108827337 wait complete. 2023-09-30T22:18:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.722+0000 7f3d8cd7f700 1 Processor -- start 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.722+0000 7f3d8cd7f700 1 -- start start 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8cd7f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d8809c650 0x7f3d8806b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8cd7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d880a0090 0x7f3d8806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8cd7f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d88131970 0x7f3d8806c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8cd7f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8811f6f0 con 0x7f3d8809c650 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8cd7f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8809d980 con 0x7f3d880a0090 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8cd7f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8811f9b0 con 0x7f3d88131970 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d8809c650 0x7f3d8806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d8809c650 0x7f3d8806b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43338/0 (socket says 172.21.15.89:43338) 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 -- 172.21.15.89:0/2704725270 learned_addr learned my addr 172.21.15.89:0/2704725270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d85d9b700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d880a0090 0x7f3d8806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d86d9d700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d88131970 0x7f3d8806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 -- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d88131970 msgr2=0x7f3d8806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d88131970 0x7f3d8806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 -- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d880a0090 msgr2=0x7f3d8806bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d880a0090 0x7f3d8806bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 -- 172.21.15.89:0/2704725270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0094a0 con 0x7f3d8809c650 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d85d9b700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d880a0090 0x7f3d8806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d8659c700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d8809c650 0x7f3d8806b880 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3d78000c00 tx=0x7f3d78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.723+0000 7f3d76ffd700 1 -- 172.21.15.89:0/2704725270 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d780045d0 con 0x7f3d8809c650 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.724+0000 7f3d76ffd700 1 -- 172.21.15.89:0/2704725270 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7801e070 con 0x7f3d8809c650 2023-09-30T22:18:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.724+0000 7f3d76ffd700 1 -- 172.21.15.89:0/2704725270 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d780038c0 con 0x7f3d8809c650 2023-09-30T22:18:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.724+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d780094a0 con 0x7f3d8809c650 2023-09-30T22:18:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.724+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d88072520 con 0x7f3d8809c650 2023-09-30T22:18:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.724+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d880747f0 con 0x7f3d8809c650 2023-09-30T22:18:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.725+0000 7f3d76ffd700 1 -- 172.21.15.89:0/2704725270 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3d78003a20 con 0x7f3d8809c650 2023-09-30T22:18:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.725+0000 7f3d76ffd700 1 -- 172.21.15.89:0/2704725270 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3d78020070 con 0x7f3d8809c650 2023-09-30T22:18:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.730+0000 7f3d76ffd700 1 -- 172.21.15.89:0/2704725270 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3d78010380 con 0x7f3d8809c650 2023-09-30T22:18:05.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.792+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa34409ffe0 con 0x7fa34409e930 2023-09-30T22:18:05.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.792+0000 7fa32b7fe700 1 -- 172.21.15.89:0/1724981803 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa33402b090 con 0x7fa34409e930 2023-09-30T22:18:05.792 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.794+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409e930 msgr2=0x7fa34406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.794+0000 7fa3494f4700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409e930 0x7fa34406b8b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa33400ec10 tx=0x7fa33400ef20 comp rx=0 tx=0).stop 2023-09-30T22:18:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.794+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 shutdown_connections 2023-09-30T22:18:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.794+0000 7fa3494f4700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa34409f650 0x7fa34406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.794+0000 7fa3494f4700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa34409e930 0x7fa34406b8b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.794+0000 7fa3494f4700 1 --2- 172.21.15.89:0/1724981803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa34409f240 0x7fa34406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.794+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 >> 172.21.15.89:0/1724981803 conn(0x7fa3440a4210 msgr2=0x7fa3441b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.795+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 shutdown_connections 2023-09-30T22:18:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.795+0000 7fa3494f4700 1 -- 172.21.15.89:0/1724981803 wait complete. 2023-09-30T22:18:05.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:05.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.904+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9c5c121fb0 con 0x7f9c5c09de50 2023-09-30T22:18:05.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.904+0000 7f9c597fa700 1 -- 172.21.15.89:0/3545896014 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c5c121fb0 con 0x7f9c5c09de50 2023-09-30T22:18:05.904 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:05.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.906+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c09de50 msgr2=0x7f9c5c11eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.906+0000 7f9c6301c700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c09de50 0x7f9c5c11eb10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009f80 tx=0x7f9c50004750 comp rx=0 tx=0).stop 2023-09-30T22:18:05.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.907+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 shutdown_connections 2023-09-30T22:18:05.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.907+0000 7f9c6301c700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c5c1338a0 0x7f9c5c11f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.907+0000 7f9c6301c700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c5c09de50 0x7f9c5c11eb10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.907+0000 7f9c6301c700 1 --2- 172.21.15.89:0/3545896014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c5c09e760 0x7f9c5c11f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.907+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 >> 172.21.15.89:0/3545896014 conn(0x7f9c5c0a2aa0 msgr2=0x7f9c5c09ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:05.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.907+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 shutdown_connections 2023-09-30T22:18:05.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.907+0000 7f9c6301c700 1 -- 172.21.15.89:0/3545896014 wait complete. 2023-09-30T22:18:05.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:05.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.934+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa50c11d3f0 con 0x7fa50c09c650 2023-09-30T22:18:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.935+0000 7fa5097fa700 1 -- 172.21.15.89:0/3674452585 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa50c11d3f0 con 0x7fa50c09c650 2023-09-30T22:18:05.935 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa50c09c650 msgr2=0x7fa50c11a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa50c09c650 0x7fa50c11a620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa500000c00 tx=0x7fa500004750 comp rx=0 tx=0).stop 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 shutdown_connections 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa50c131970 0x7fa50c11b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa50c09c650 0x7fa50c11a620 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 --2- 172.21.15.89:0/3674452585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa50c0a0090 0x7fa50c11ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 >> 172.21.15.89:0/3674452585 conn(0x7fa50c0a3f70 msgr2=0x7fa50c12ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 shutdown_connections 2023-09-30T22:18:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.937+0000 7fa513153700 1 -- 172.21.15.89:0/3674452585 wait complete. 2023-09-30T22:18:05.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.967+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3d88072840 con 0x7f3d8809c650 2023-09-30T22:18:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.968+0000 7f3d76ffd700 1 -- 172.21.15.89:0/2704725270 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3d7802a020 con 0x7f3d8809c650 2023-09-30T22:18:05.968 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d8809c650 msgr2=0x7f3d8806b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d8809c650 0x7f3d8806b880 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3d78000c00 tx=0x7f3d78004750 comp rx=0 tx=0).stop 2023-09-30T22:18:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 shutdown_connections 2023-09-30T22:18:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3d88131970 0x7f3d8806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d8809c650 0x7f3d8806b880 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 --2- 172.21.15.89:0/2704725270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3d880a0090 0x7f3d8806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 >> 172.21.15.89:0/2704725270 conn(0x7f3d880a3f70 msgr2=0x7f3d8812ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 shutdown_connections 2023-09-30T22:18:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:05.970+0000 7f3d8cd7f700 1 -- 172.21.15.89:0/2704725270 wait complete. 2023-09-30T22:18:05.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:06.807 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:06.919 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:06.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.941+0000 7ff0f73c1700 1 Processor -- start 2023-09-30T22:18:06.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.941+0000 7ff0f73c1700 1 -- start start 2023-09-30T22:18:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f73c1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0f009d6f0 0x7ff0f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f73c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e000 0x7ff0f0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f73c1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f009f130 con 0x7ff0f009e410 2023-09-30T22:18:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f73c1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f009f520 con 0x7ff0f009d6f0 2023-09-30T22:18:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f73c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f009e930 con 0x7ff0f009e000 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f495c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e000 0x7ff0f0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f515d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0f009d6f0 0x7ff0f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f515d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0f009d6f0 0x7ff0f009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43340/0 (socket says 172.21.15.89:43340) 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f495c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e000 0x7ff0f0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36418/0 (socket says 172.21.15.89:36418) 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f515d700 1 -- 172.21.15.89:0/1906636087 learned_addr learned my addr 172.21.15.89:0/1906636087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f495c700 1 -- 172.21.15.89:0/1906636087 >> v2:172.21.15.89:3300/0 conn(0x7ff0f009d6f0 msgr2=0x7ff0f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f495c700 1 --2- 172.21.15.89:0/1906636087 >> v2:172.21.15.89:3300/0 conn(0x7ff0f009d6f0 0x7ff0f009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f495c700 1 -- 172.21.15.89:0/1906636087 >> v1:172.21.15.89:6789/0 conn(0x7ff0f009e410 legacy=0x7ff0f0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f495c700 1 -- 172.21.15.89:0/1906636087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f009b2d0 con 0x7ff0f009e000 2023-09-30T22:18:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0e7fff700 1 -- 172.21.15.89:0/1906636087 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947718523 0 0) 0x7ff0f009f130 con 0x7ff0f009e410 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f515d700 1 --2- 172.21.15.89:0/1906636087 >> v2:172.21.15.89:3300/0 conn(0x7ff0f009d6f0 0x7ff0f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0f495c700 1 --2- 172.21.15.89:0/1906636087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e000 0x7ff0f0130660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009840 tx=0x7ff0e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fbeda3fb6efb2d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0e7fff700 1 -- 172.21.15.89:0/1906636087 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0011830 con 0x7ff0f009e000 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.942+0000 7ff0e7fff700 1 -- 172.21.15.89:0/1906636087 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e0011990 con 0x7ff0f009e000 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0e7fff700 1 -- 172.21.15.89:0/1906636087 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0011d20 con 0x7ff0f009e000 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0f73c1700 1 -- 172.21.15.89:0/1906636087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e000 msgr2=0x7ff0f0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0f73c1700 1 --2- 172.21.15.89:0/1906636087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e000 0x7ff0f0130660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009840 tx=0x7ff0e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0f73c1700 1 -- 172.21.15.89:0/1906636087 shutdown_connections 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0f73c1700 1 --2- 172.21.15.89:0/1906636087 >> v2:172.21.15.89:3300/0 conn(0x7ff0f009d6f0 0x7ff0f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0f73c1700 1 --2- 172.21.15.89:0/1906636087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e000 0x7ff0f0130660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0f73c1700 1 -- 172.21.15.89:0/1906636087 >> 172.21.15.89:0/1906636087 conn(0x7ff0f0066f50 msgr2=0x7ff0f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0f73c1700 1 -- 172.21.15.89:0/1906636087 shutdown_connections 2023-09-30T22:18:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.943+0000 7ff0f73c1700 1 -- 172.21.15.89:0/1906636087 wait complete. 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f73c1700 1 Processor -- start 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f73c1700 1 -- start start 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f73c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009d6f0 0x7ff0f011eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f73c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009e000 0x7ff0f011f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f73c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f009e410 0x7ff0f011f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f73c1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0122780 con 0x7ff0f009e000 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f73c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0130270 con 0x7ff0f009e410 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f73c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0122420 con 0x7ff0f009d6f0 2023-09-30T22:18:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f495c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009e000 0x7ff0f011f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f495c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009e000 0x7ff0f011f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43346/0 (socket says 172.21.15.89:43346) 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f495c700 1 -- 172.21.15.89:0/3489091653 learned_addr learned my addr 172.21.15.89:0/3489091653 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f515d700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009d6f0 0x7ff0f011eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f595e700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f009e410 0x7ff0f011f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f495c700 1 -- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009d6f0 msgr2=0x7ff0f011eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f495c700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009d6f0 0x7ff0f011eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.945+0000 7ff0f495c700 1 -- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f009e410 msgr2=0x7ff0f011f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0f495c700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f009e410 0x7ff0f011f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0f495c700 1 -- 172.21.15.89:0/3489091653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec009550 con 0x7ff0f009e000 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0f595e700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f009e410 0x7ff0f011f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0f515d700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009d6f0 0x7ff0f011eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0f495c700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009e000 0x7ff0f011f3e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00090b0 tx=0x7ff0e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0e5ffb700 1 -- 172.21.15.89:0/3489091653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00036a0 con 0x7ff0f009e000 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0e5ffb700 1 -- 172.21.15.89:0/3489091653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e001f070 con 0x7ff0f009e000 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00094a0 con 0x7ff0f009e000 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0e5ffb700 1 -- 172.21.15.89:0/3489091653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0003a60 con 0x7ff0f009e000 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.946+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f0072670 con 0x7ff0f009e000 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.947+0000 7ff0e5ffb700 1 -- 172.21.15.89:0/3489091653 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0e0003bc0 con 0x7ff0f009e000 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.947+0000 7ff0e5ffb700 1 -- 172.21.15.89:0/3489091653 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff0e0021070 con 0x7ff0f009e000 2023-09-30T22:18:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.947+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f0122270 con 0x7ff0f009e000 2023-09-30T22:18:06.951 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:06.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:06.952+0000 7ff0e5ffb700 1 -- 172.21.15.89:0/3489091653 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0e0010380 con 0x7ff0f009e000 2023-09-30T22:18:06.983 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:07.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.052+0000 7f531fdc8700 1 Processor -- start 2023-09-30T22:18:07.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.052+0000 7f531fdc8700 1 -- start start 2023-09-30T22:18:07.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.052+0000 7f531fdc8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f531806a770 0x7f531806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.052+0000 7f531fdc8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531806b080 0x7f5318131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.052+0000 7f531fdc8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5318131b40 0x7f5318131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.052+0000 7f531fdc8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531809ece0 con 0x7f531806a770 2023-09-30T22:18:07.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.052+0000 7f531fdc8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531809e760 con 0x7f5318131b40 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.052+0000 7f531fdc8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f531809e040 con 0x7f531806b080 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531e365700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5318131b40 0x7f5318131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531db64700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f531806a770 0x7f531806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531e365700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5318131b40 0x7f5318131fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59102/0 (socket says 172.21.15.89:59102) 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531e365700 1 -- 172.21.15.89:0/2574084346 learned_addr learned my addr 172.21.15.89:0/2574084346 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531d363700 1 --2- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531806b080 0x7f5318131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531d363700 1 -- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5318131b40 msgr2=0x7f5318131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531d363700 1 --2- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5318131b40 0x7f5318131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531d363700 1 -- 172.21.15.89:0/2574084346 >> v2:172.21.15.89:3300/0 conn(0x7f531806a770 msgr2=0x7f531806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531d363700 1 --2- 172.21.15.89:0/2574084346 >> v2:172.21.15.89:3300/0 conn(0x7f531806a770 0x7f531806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531d363700 1 -- 172.21.15.89:0/2574084346 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531809f630 con 0x7f531806b080 2023-09-30T22:18:07.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531e365700 1 --2- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5318131b40 0x7f5318131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:07.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531d363700 1 --2- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531806b080 0x7f5318131600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5308009840 tx=0x7f5308009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e189e2c4c10ec2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:07.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531cb62700 1 -- 172.21.15.89:0/2574084346 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308011830 con 0x7f531806b080 2023-09-30T22:18:07.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531cb62700 1 -- 172.21.15.89:0/2574084346 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5308011990 con 0x7f531806b080 2023-09-30T22:18:07.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.053+0000 7f531cb62700 1 -- 172.21.15.89:0/2574084346 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308011d20 con 0x7f531806b080 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 -- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531806b080 msgr2=0x7f5318131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 --2- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531806b080 0x7f5318131600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5308009840 tx=0x7f5308009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 -- 172.21.15.89:0/2574084346 shutdown_connections 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 --2- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5318131b40 0x7f5318131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 --2- 172.21.15.89:0/2574084346 >> v2:172.21.15.89:3300/0 conn(0x7f531806a770 0x7f531806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 --2- 172.21.15.89:0/2574084346 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531806b080 0x7f5318131600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 -- 172.21.15.89:0/2574084346 >> 172.21.15.89:0/2574084346 conn(0x7f5318066f50 msgr2=0x7f531809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 -- 172.21.15.89:0/2574084346 shutdown_connections 2023-09-30T22:18:07.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.054+0000 7f531fdc8700 1 -- 172.21.15.89:0/2574084346 wait complete. 2023-09-30T22:18:07.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.055+0000 7f531fdc8700 1 Processor -- start 2023-09-30T22:18:07.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.055+0000 7f531fdc8700 1 -- start start 2023-09-30T22:18:07.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531fdc8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531806a770 0x7f531811c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531fdc8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531806b080 0x7f531811c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531fdc8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5318131b40 0x7f531811cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531fdc8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531811fd80 con 0x7f531806b080 2023-09-30T22:18:07.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531fdc8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5318131500 con 0x7f531806a770 2023-09-30T22:18:07.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531fdc8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f531811e620 con 0x7f5318131b40 2023-09-30T22:18:07.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531e365700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5318131b40 0x7f531811cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531d363700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531806b080 0x7f531811c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531e365700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5318131b40 0x7f531811cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36436/0 (socket says 172.21.15.89:36436) 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531e365700 1 -- 172.21.15.89:0/654687985 learned_addr learned my addr 172.21.15.89:0/654687985 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531db64700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531806a770 0x7f531811c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531d363700 1 -- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5318131b40 msgr2=0x7f531811cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531d363700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5318131b40 0x7f531811cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531d363700 1 -- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531806a770 msgr2=0x7f531811c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531d363700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531806a770 0x7f531811c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531d363700 1 -- 172.21.15.89:0/654687985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310009b30 con 0x7f531806b080 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531e365700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5318131b40 0x7f531811cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.056+0000 7f531d363700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531806b080 0x7f531811c9a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5308016000 tx=0x7f5308004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.057+0000 7f530e7fc700 1 -- 172.21.15.89:0/654687985 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308004550 con 0x7f531806b080 2023-09-30T22:18:07.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.057+0000 7f530e7fc700 1 -- 172.21.15.89:0/654687985 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530801f070 con 0x7f531806b080 2023-09-30T22:18:07.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.057+0000 7f530e7fc700 1 -- 172.21.15.89:0/654687985 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53080038e0 con 0x7f531806b080 2023-09-30T22:18:07.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.057+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53080094a0 con 0x7f531806b080 2023-09-30T22:18:07.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.057+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5318120680 con 0x7f531806b080 2023-09-30T22:18:07.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.058+0000 7f530e7fc700 1 -- 172.21.15.89:0/654687985 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5308003a40 con 0x7f531806b080 2023-09-30T22:18:07.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.058+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5318074ea0 con 0x7f531806b080 2023-09-30T22:18:07.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.058+0000 7f530e7fc700 1 -- 172.21.15.89:0/654687985 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5308021070 con 0x7f531806b080 2023-09-30T22:18:07.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.066+0000 7f530e7fc700 1 -- 172.21.15.89:0/654687985 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5308045060 con 0x7f531806b080 2023-09-30T22:18:07.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.080+0000 7f345d8fc700 1 Processor -- start 2023-09-30T22:18:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.080+0000 7f345d8fc700 1 -- start start 2023-09-30T22:18:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f345d8fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f345d8fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f345809e000 0x7f3458130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f345d8fc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345809f130 con 0x7f345809e410 2023-09-30T22:18:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f345d8fc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345809f520 con 0x7f345809e000 2023-09-30T22:18:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f345d8fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f345809e930 con 0x7f345809d6f0 2023-09-30T22:18:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3456ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3456ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36450/0 (socket says 172.21.15.89:36450) 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3456ffd700 1 -- 172.21.15.89:0/1688893291 learned_addr learned my addr 172.21.15.89:0/1688893291 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f34567fc700 1 --2- 172.21.15.89:0/1688893291 >> v2:172.21.15.89:3300/0 conn(0x7f345809e000 0x7f3458130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3455ffb700 1 -- 172.21.15.89:0/1688893291 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273050399 0 0) 0x7f345809f130 con 0x7f345809e410 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3455ffb700 1 -- 172.21.15.89:0/1688893291 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34400035b0 con 0x7f345809e410 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3456ffd700 1 -- 172.21.15.89:0/1688893291 >> v2:172.21.15.89:3300/0 conn(0x7f345809e000 msgr2=0x7f3458130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3456ffd700 1 --2- 172.21.15.89:0/1688893291 >> v2:172.21.15.89:3300/0 conn(0x7f345809e000 0x7f3458130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3456ffd700 1 -- 172.21.15.89:0/1688893291 >> v1:172.21.15.89:6789/0 conn(0x7f345809e410 legacy=0x7f3458130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.081+0000 7f3456ffd700 1 -- 172.21.15.89:0/1688893291 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345809b2d0 con 0x7f345809d6f0 2023-09-30T22:18:07.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.082+0000 7f3456ffd700 1 --2- 172.21.15.89:0/1688893291 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345809dac0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34480098f0 tx=0x7f3448009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af914d8f9ab39b42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:07.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.082+0000 7f3455ffb700 1 -- 172.21.15.89:0/1688893291 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448011830 con 0x7f345809d6f0 2023-09-30T22:18:07.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.082+0000 7f3455ffb700 1 -- 172.21.15.89:0/1688893291 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448011990 con 0x7f345809d6f0 2023-09-30T22:18:07.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.082+0000 7f3455ffb700 1 -- 172.21.15.89:0/1688893291 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448011d20 con 0x7f345809d6f0 2023-09-30T22:18:07.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.082+0000 7f345d8fc700 1 -- 172.21.15.89:0/1688893291 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 msgr2=0x7f345809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.082+0000 7f345d8fc700 1 --2- 172.21.15.89:0/1688893291 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345809dac0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34480098f0 tx=0x7f3448009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:07.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.083+0000 7f345d8fc700 1 -- 172.21.15.89:0/1688893291 shutdown_connections 2023-09-30T22:18:07.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.083+0000 7f345d8fc700 1 --2- 172.21.15.89:0/1688893291 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345809dac0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.083+0000 7f345d8fc700 1 --2- 172.21.15.89:0/1688893291 >> v2:172.21.15.89:3300/0 conn(0x7f345809e000 0x7f3458130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.083+0000 7f345d8fc700 1 -- 172.21.15.89:0/1688893291 >> 172.21.15.89:0/1688893291 conn(0x7f3458066f50 msgr2=0x7f345809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:07.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.083+0000 7f345d8fc700 1 -- 172.21.15.89:0/1688893291 shutdown_connections 2023-09-30T22:18:07.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.083+0000 7f345d8fc700 1 -- 172.21.15.89:0/1688893291 wait complete. 2023-09-30T22:18:07.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.083+0000 7f345d8fc700 1 Processor -- start 2023-09-30T22:18:07.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f345d8fc700 1 -- start start 2023-09-30T22:18:07.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f345d8fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345811ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f345d8fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f345809e000 0x7f34581204d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f345d8fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f345809e410 0x7f3458120a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f345d8fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3458123ab0 con 0x7f345809e410 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f345d8fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345809c700 con 0x7f345809e000 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f345d8fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3458123240 con 0x7f345809d6f0 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f3456ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345811ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f3456ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345811ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36464/0 (socket says 172.21.15.89:36464) 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f3456ffd700 1 -- 172.21.15.89:0/1595648622 learned_addr learned my addr 172.21.15.89:0/1595648622 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f34567fc700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f345809e000 0x7f34581204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.084+0000 7f34577fe700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f345809e410 0x7f3458120a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f34567fc700 1 -- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 msgr2=0x7f345811ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f34567fc700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345811ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f34567fc700 1 -- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f345809e410 msgr2=0x7f3458120a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f34567fc700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f345809e410 0x7f3458120a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f34567fc700 1 -- 172.21.15.89:0/1595648622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448009550 con 0x7f345809e000 2023-09-30T22:18:07.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f34567fc700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f345809e000 0x7f34581204d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e8d0 tx=0x7f344c00ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:07.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f343ffff700 1 -- 172.21.15.89:0/1595648622 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c013680 con 0x7f345809e000 2023-09-30T22:18:07.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f343ffff700 1 -- 172.21.15.89:0/1595648622 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c0137e0 con 0x7f345809e000 2023-09-30T22:18:07.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34580a1b80 con 0x7f345809e000 2023-09-30T22:18:07.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f343ffff700 1 -- 172.21.15.89:0/1595648622 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c013b50 con 0x7f345809e000 2023-09-30T22:18:07.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.085+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3458123510 con 0x7f345809e000 2023-09-30T22:18:07.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.086+0000 7f343ffff700 1 -- 172.21.15.89:0/1595648622 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f344c013cb0 con 0x7f345809e000 2023-09-30T22:18:07.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.086+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345811cc30 con 0x7f345809e000 2023-09-30T22:18:07.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.086+0000 7f343ffff700 1 -- 172.21.15.89:0/1595648622 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f344c03f610 con 0x7f345809e000 2023-09-30T22:18:07.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.095+0000 7f343ffff700 1 -- 172.21.15.89:0/1595648622 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f344c021020 con 0x7f345809e000 2023-09-30T22:18:07.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.138+0000 7f1173571700 1 Processor -- start 2023-09-30T22:18:07.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.138+0000 7f1173571700 1 -- start start 2023-09-30T22:18:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f1173571700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f116c09e930 0x7f116c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f1173571700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09f650 0x7f116c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f1173571700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c070940 con 0x7f116c09f240 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f1173571700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c06fd60 con 0x7f116c09e930 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f1173571700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c06fa80 con 0x7f116c09f650 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f117130d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f116c09e930 0x7f116c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f117130d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f116c09e930 0x7f116c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43384/0 (socket says 172.21.15.89:43384) 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f117130d700 1 -- 172.21.15.89:0/902859368 learned_addr learned my addr 172.21.15.89:0/902859368 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f1171b0e700 1 --2- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09f650 0x7f116c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f117130d700 1 -- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09f650 msgr2=0x7f116c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f117130d700 1 --2- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09f650 0x7f116c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f117130d700 1 -- 172.21.15.89:0/902859368 >> v1:172.21.15.89:6789/0 conn(0x7f116c09f240 legacy=0x7f116c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f117130d700 1 -- 172.21.15.89:0/902859368 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c09fb60 con 0x7f116c09e930 2023-09-30T22:18:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.139+0000 7f1171b0e700 1 --2- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09f650 0x7f116c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.140+0000 7f117130d700 1 --2- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09e930 0x7f116c09ed00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f11680098f0 tx=0x7f1168009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155ec0b0a9fe5c17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.140+0000 7f1163fff700 1 -- 172.21.15.89:0/902859368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011830 con 0x7f116c09e930 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.140+0000 7f1163fff700 1 -- 172.21.15.89:0/902859368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168011990 con 0x7f116c09e930 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.140+0000 7f1163fff700 1 -- 172.21.15.89:0/902859368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011d20 con 0x7f116c09e930 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.141+0000 7f1173571700 1 -- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09e930 msgr2=0x7f116c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.141+0000 7f1173571700 1 --2- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09e930 0x7f116c09ed00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f11680098f0 tx=0x7f1168009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.141+0000 7f1173571700 1 -- 172.21.15.89:0/902859368 shutdown_connections 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.141+0000 7f1173571700 1 --2- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09f650 0x7f116c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.141+0000 7f1173571700 1 --2- 172.21.15.89:0/902859368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09e930 0x7f116c09ed00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.141+0000 7f1173571700 1 -- 172.21.15.89:0/902859368 >> 172.21.15.89:0/902859368 conn(0x7f116c0a4210 msgr2=0x7f116c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.141+0000 7f1173571700 1 -- 172.21.15.89:0/902859368 shutdown_connections 2023-09-30T22:18:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.141+0000 7f1173571700 1 -- 172.21.15.89:0/902859368 wait complete. 2023-09-30T22:18:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1173571700 1 Processor -- start 2023-09-30T22:18:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1173571700 1 -- start start 2023-09-30T22:18:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1173571700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09e930 0x7f116c134880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1173571700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f116c09f240 0x7f116c134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1173571700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09f650 0x7f116c135300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1173571700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c0a3630 con 0x7f116c09f650 2023-09-30T22:18:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1173571700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c135c70 con 0x7f116c09e930 2023-09-30T22:18:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1173571700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f116c137c30 con 0x7f116c09f240 2023-09-30T22:18:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1171b0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09f650 0x7f116c135300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1171b0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09f650 0x7f116c135300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43394/0 (socket says 172.21.15.89:43394) 2023-09-30T22:18:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f1171b0e700 1 -- 172.21.15.89:0/2455075097 learned_addr learned my addr 172.21.15.89:0/2455075097 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.142+0000 7f117130d700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09e930 0x7f116c134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1170b0c700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f116c09f240 0x7f116c134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1171b0e700 1 -- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f116c09f240 msgr2=0x7f116c134dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1171b0e700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f116c09f240 0x7f116c134dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1171b0e700 1 -- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09e930 msgr2=0x7f116c134880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1171b0e700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09e930 0x7f116c134880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1171b0e700 1 -- 172.21.15.89:0/2455075097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1164009a70 con 0x7f116c09f650 2023-09-30T22:18:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1171b0e700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09f650 0x7f116c135300 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f116400efc0 tx=0x7f116400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1161ffb700 1 -- 172.21.15.89:0/2455075097 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400cae0 con 0x7f116c09f650 2023-09-30T22:18:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1161ffb700 1 -- 172.21.15.89:0/2455075097 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116400f040 con 0x7f116c09f650 2023-09-30T22:18:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1168009550 con 0x7f116c09f650 2023-09-30T22:18:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1161ffb700 1 -- 172.21.15.89:0/2455075097 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400ce60 con 0x7f116c09f650 2023-09-30T22:18:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.143+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c0a0190 con 0x7f116c09f650 2023-09-30T22:18:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.144+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c074ea0 con 0x7f116c09f650 2023-09-30T22:18:07.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.145+0000 7f1161ffb700 1 -- 172.21.15.89:0/2455075097 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1164014480 con 0x7f116c09f650 2023-09-30T22:18:07.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.149+0000 7f1161ffb700 1 -- 172.21.15.89:0/2455075097 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f116401c7b0 con 0x7f116c09f650 2023-09-30T22:18:07.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.154+0000 7f1161ffb700 1 -- 172.21.15.89:0/2455075097 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1164045060 con 0x7f116c09f650 2023-09-30T22:18:07.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.161+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff0f0121d00 con 0x7ff0f009e000 2023-09-30T22:18:07.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.161+0000 7ff0e5ffb700 1 -- 172.21.15.89:0/3489091653 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0e0015020 con 0x7ff0f009e000 2023-09-30T22:18:07.162 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:07.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.163+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009e000 msgr2=0x7ff0f011f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.163+0000 7ff0f73c1700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009e000 0x7ff0f011f3e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00090b0 tx=0x7ff0e0004750 comp rx=0 tx=0).stop 2023-09-30T22:18:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.164+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 shutdown_connections 2023-09-30T22:18:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.164+0000 7ff0f73c1700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f009e410 0x7ff0f011f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.164+0000 7ff0f73c1700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009d6f0 0x7ff0f011eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.164+0000 7ff0f73c1700 1 --2- 172.21.15.89:0/3489091653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009e000 0x7ff0f011f3e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.164+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 >> 172.21.15.89:0/3489091653 conn(0x7ff0f0066f50 msgr2=0x7ff0f0130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.164+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 shutdown_connections 2023-09-30T22:18:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.164+0000 7ff0f73c1700 1 -- 172.21.15.89:0/3489091653 wait complete. 2023-09-30T22:18:07.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.268+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f531806c360 con 0x7f531806b080 2023-09-30T22:18:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.268+0000 7f530e7fc700 1 -- 172.21.15.89:0/654687985 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5308045020 con 0x7f531806b080 2023-09-30T22:18:07.269 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:07.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531806b080 msgr2=0x7f531811c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531806b080 0x7f531811c9a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5308016000 tx=0x7f5308004750 comp rx=0 tx=0).stop 2023-09-30T22:18:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 shutdown_connections 2023-09-30T22:18:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5318131b40 0x7f531811cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531806a770 0x7f531811c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 --2- 172.21.15.89:0/654687985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531806b080 0x7f531811c9a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 >> 172.21.15.89:0/654687985 conn(0x7f5318066f50 msgr2=0x7f5318070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 shutdown_connections 2023-09-30T22:18:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.270+0000 7f531fdc8700 1 -- 172.21.15.89:0/654687985 wait complete. 2023-09-30T22:18:07.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.310+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3458074ea0 con 0x7f345809e000 2023-09-30T22:18:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.310+0000 7f343ffff700 1 -- 172.21.15.89:0/1595648622 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f344c01be10 con 0x7f345809e000 2023-09-30T22:18:07.311 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:07.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f345809e000 msgr2=0x7f34581204d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f345809e000 0x7f34581204d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e8d0 tx=0x7f344c00ebe0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 shutdown_connections 2023-09-30T22:18:07.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f345809e410 0x7f3458120a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f345809d6f0 0x7f345811ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 --2- 172.21.15.89:0/1595648622 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f345809e000 0x7f34581204d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 >> 172.21.15.89:0/1595648622 conn(0x7f3458066f50 msgr2=0x7f3458130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 shutdown_connections 2023-09-30T22:18:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.312+0000 7f345d8fc700 1 -- 172.21.15.89:0/1595648622 wait complete. 2023-09-30T22:18:07.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:07.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.350+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f116c0a2d90 con 0x7f116c09f650 2023-09-30T22:18:07.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.350+0000 7f1161ffb700 1 -- 172.21.15.89:0/2455075097 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1164045020 con 0x7f116c09f650 2023-09-30T22:18:07.351 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.352+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09f650 msgr2=0x7f116c135300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.352+0000 7f1173571700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09f650 0x7f116c135300 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f116400efc0 tx=0x7f116400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.352+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 shutdown_connections 2023-09-30T22:18:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.352+0000 7f1173571700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f116c09f650 0x7f116c135300 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.352+0000 7f1173571700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f116c09e930 0x7f116c134880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.352+0000 7f1173571700 1 --2- 172.21.15.89:0/2455075097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f116c09f240 0x7f116c134dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.352+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 >> 172.21.15.89:0/2455075097 conn(0x7f116c0a4210 msgr2=0x7f116c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.353+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 shutdown_connections 2023-09-30T22:18:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:07.353+0000 7f1173571700 1 -- 172.21.15.89:0/2455075097 wait complete. 2023-09-30T22:18:07.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:08.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:08.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:08.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.314+0000 7f1a7070c700 1 Processor -- start 2023-09-30T22:18:08.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.314+0000 7f1a7070c700 1 -- start start 2023-09-30T22:18:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a7070c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a6809d6f0 0x7f1a6809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a7070c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a7070c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a7070c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6809f520 con 0x7f1a6809d6f0 2023-09-30T22:18:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a7070c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6809f130 con 0x7f1a6809e410 2023-09-30T22:18:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a7070c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6809e930 con 0x7f1a6809e000 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6e4a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a6809d6f0 0x7f1a6809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6eca9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6e4a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a6809d6f0 0x7f1a6809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43398/0 (socket says 172.21.15.89:43398) 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6e4a8700 1 -- 172.21.15.89:0/4104477814 learned_addr learned my addr 172.21.15.89:0/4104477814 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6dca7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6e4a8700 1 -- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 msgr2=0x7f1a68130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6e4a8700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6e4a8700 1 -- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 msgr2=0x7f1a68130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6e4a8700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6e4a8700 1 -- 172.21.15.89:0/4104477814 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6809b2d0 con 0x7f1a6809d6f0 2023-09-30T22:18:08.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6dca7700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.315+0000 7f1a6eca9700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a6e4a8700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 0x7f1a6809dac0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009920 tx=0x7f1a64009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8992e971f2176c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a6d4a6700 1 -- 172.21.15.89:0/4104477814 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64011830 con 0x7f1a6809d6f0 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a6d4a6700 1 -- 172.21.15.89:0/4104477814 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a64011990 con 0x7f1a6809d6f0 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a6d4a6700 1 -- 172.21.15.89:0/4104477814 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64011d20 con 0x7f1a6809d6f0 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a7070c700 1 -- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 msgr2=0x7f1a6809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a7070c700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 0x7f1a6809dac0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009920 tx=0x7f1a64009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a7070c700 1 -- 172.21.15.89:0/4104477814 shutdown_connections 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a7070c700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a7070c700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 0x7f1a6809dac0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a7070c700 1 --2- 172.21.15.89:0/4104477814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.316+0000 7f1a7070c700 1 -- 172.21.15.89:0/4104477814 >> 172.21.15.89:0/4104477814 conn(0x7f1a68066f50 msgr2=0x7f1a6809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:08.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.317+0000 7f1a7070c700 1 -- 172.21.15.89:0/4104477814 shutdown_connections 2023-09-30T22:18:08.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.317+0000 7f1a7070c700 1 -- 172.21.15.89:0/4104477814 wait complete. 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.318+0000 7f1a7070c700 1 Processor -- start 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.318+0000 7f1a7070c700 1 -- start start 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a7070c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 0x7f1a681227c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a7070c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68122d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a7070c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a7070c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68125d30 con 0x7f1a6809d6f0 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a7070c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681301e0 con 0x7f1a6809e410 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a7070c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6809c700 con 0x7f1a6809e000 2023-09-30T22:18:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6eca9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68122d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36494/0 (socket says 172.21.15.89:36494) 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 -- 172.21.15.89:0/3770456942 learned_addr learned my addr 172.21.15.89:0/3770456942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6e4a8700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 0x7f1a681227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 -- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 msgr2=0x7f1a68123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 -- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 msgr2=0x7f1a681227c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 0x7f1a681227c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 -- 172.21.15.89:0/3770456942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a580094a0 con 0x7f1a6809e000 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6eca9700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6e4a8700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 0x7f1a681227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a6dca7700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68122d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800ec10 tx=0x7f1a5800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a5effd700 1 -- 172.21.15.89:0/3770456942 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58009780 con 0x7f1a6809e000 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a5effd700 1 -- 172.21.15.89:0/3770456942 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800c850 con 0x7f1a6809e000 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a60009a70 con 0x7f1a6809e000 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.319+0000 7f1a5effd700 1 -- 172.21.15.89:0/3770456942 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800cbe0 con 0x7f1a6809e000 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.320+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a64009550 con 0x7f1a6809e000 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.320+0000 7f1a5effd700 1 -- 172.21.15.89:0/3770456942 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a5800cd40 con 0x7f1a6809e000 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.320+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a68074ea0 con 0x7f1a6809e000 2023-09-30T22:18:08.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.321+0000 7f1a5effd700 1 -- 172.21.15.89:0/3770456942 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1a5803fac0 con 0x7f1a6809e000 2023-09-30T22:18:08.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:08.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.325+0000 7f1a5effd700 1 -- 172.21.15.89:0/3770456942 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a5801b3b0 con 0x7f1a6809e000 2023-09-30T22:18:08.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:08.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.423+0000 7f19d27b1700 1 Processor -- start 2023-09-30T22:18:08.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.423+0000 7f19d27b1700 1 -- start start 2023-09-30T22:18:08.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d27b1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19cc09d6f0 0x7f19cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d27b1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09e000 0x7f19cc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d27b1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e410 0x7f19cc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d27b1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19cc09f520 con 0x7f19cc09d6f0 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d27b1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc09f130 con 0x7f19cc09e410 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d27b1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc09e930 con 0x7f19cc09e000 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19cbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19cc09d6f0 0x7f19cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d0d4e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e410 0x7f19cc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19cbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19cc09d6f0 0x7f19cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43422/0 (socket says 172.21.15.89:43422) 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d0d4e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e410 0x7f19cc130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59178/0 (socket says 172.21.15.89:59178) 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19cbfff700 1 -- 172.21.15.89:0/2512245917 learned_addr learned my addr 172.21.15.89:0/2512245917 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19cb7fe700 1 --2- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09e000 0x7f19cc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d0d4e700 1 -- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09e000 msgr2=0x7f19cc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d0d4e700 1 --2- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09e000 0x7f19cc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d0d4e700 1 -- 172.21.15.89:0/2512245917 >> v2:172.21.15.89:3300/0 conn(0x7f19cc09d6f0 msgr2=0x7f19cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d0d4e700 1 --2- 172.21.15.89:0/2512245917 >> v2:172.21.15.89:3300/0 conn(0x7f19cc09d6f0 0x7f19cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19d0d4e700 1 -- 172.21.15.89:0/2512245917 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc09b2d0 con 0x7f19cc09e410 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.424+0000 7f19cb7fe700 1 --2- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09e000 0x7f19cc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.425+0000 7f19d0d4e700 1 --2- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e410 0x7f19cc130ba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f19c4008600 tx=0x7f19c4008910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f432205d3d110026 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.425+0000 7f19caffd700 1 -- 172.21.15.89:0/2512245917 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400d2d0 con 0x7f19cc09e410 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.425+0000 7f19caffd700 1 -- 172.21.15.89:0/2512245917 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c400d430 con 0x7f19cc09e410 2023-09-30T22:18:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.425+0000 7f19caffd700 1 -- 172.21.15.89:0/2512245917 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400b980 con 0x7f19cc09e410 2023-09-30T22:18:08.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 -- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e410 msgr2=0x7f19cc130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 --2- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e410 0x7f19cc130ba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f19c4008600 tx=0x7f19c4008910 comp rx=0 tx=0).stop 2023-09-30T22:18:08.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 -- 172.21.15.89:0/2512245917 shutdown_connections 2023-09-30T22:18:08.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 --2- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e410 0x7f19cc130ba0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 --2- 172.21.15.89:0/2512245917 >> v2:172.21.15.89:3300/0 conn(0x7f19cc09d6f0 0x7f19cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 --2- 172.21.15.89:0/2512245917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09e000 0x7f19cc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 -- 172.21.15.89:0/2512245917 >> 172.21.15.89:0/2512245917 conn(0x7f19cc066f50 msgr2=0x7f19cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:08.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 -- 172.21.15.89:0/2512245917 shutdown_connections 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.426+0000 7f19d27b1700 1 -- 172.21.15.89:0/2512245917 wait complete. 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.427+0000 7f19d27b1700 1 Processor -- start 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.427+0000 7f19d27b1700 1 -- start start 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19d27b1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09d6f0 0x7f19cc11de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19d27b1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e000 0x7f19cc11e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19d27b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19cc09e410 0x7f19cc11e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19d27b1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc122270 con 0x7f19cc09e410 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19d27b1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc1218b0 con 0x7f19cc09e000 2023-09-30T22:18:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19d27b1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc120bf0 con 0x7f19cc09d6f0 2023-09-30T22:18:08.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19cbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09d6f0 0x7f19cc11de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19cbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09d6f0 0x7f19cc11de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36510/0 (socket says 172.21.15.89:36510) 2023-09-30T22:18:08.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19cbfff700 1 -- 172.21.15.89:0/507414232 learned_addr learned my addr 172.21.15.89:0/507414232 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:08.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19d0d4e700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19cc09e410 0x7f19cc11e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.428+0000 7f19cb7fe700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e000 0x7f19cc11e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19d0d4e700 1 -- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09d6f0 msgr2=0x7f19cc11de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19d0d4e700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09d6f0 0x7f19cc11de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19d0d4e700 1 -- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e000 msgr2=0x7f19cc11e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19d0d4e700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e000 0x7f19cc11e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19d0d4e700 1 -- 172.21.15.89:0/507414232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00094a0 con 0x7f19cc09e410 2023-09-30T22:18:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19d0d4e700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19cc09e410 0x7f19cc11e8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f19c4008600 tx=0x7f19c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19c8ff9700 1 -- 172.21.15.89:0/507414232 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4020670 con 0x7f19cc09e410 2023-09-30T22:18:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c4008260 con 0x7f19cc09e410 2023-09-30T22:18:08.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19c8ff9700 1 -- 172.21.15.89:0/507414232 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c401a050 con 0x7f19cc09e410 2023-09-30T22:18:08.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19c8ff9700 1 -- 172.21.15.89:0/507414232 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4020a30 con 0x7f19cc09e410 2023-09-30T22:18:08.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.429+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc0a16a0 con 0x7f19cc09e410 2023-09-30T22:18:08.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.430+0000 7f19c8ff9700 1 -- 172.21.15.89:0/507414232 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19c4020b90 con 0x7f19cc09e410 2023-09-30T22:18:08.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.430+0000 7f19c8ff9700 1 -- 172.21.15.89:0/507414232 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f19c4041960 con 0x7f19cc09e410 2023-09-30T22:18:08.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.430+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc121450 con 0x7f19cc09e410 2023-09-30T22:18:08.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.439+0000 7f19c8ff9700 1 -- 172.21.15.89:0/507414232 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19c4020e40 con 0x7f19cc09e410 2023-09-30T22:18:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.461+0000 7efd16389700 1 Processor -- start 2023-09-30T22:18:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.461+0000 7efd16389700 1 -- start start 2023-09-30T22:18:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd16389700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10133420 0x7efd101337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd16389700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133e30 0x7efd1012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd16389700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1006b990 con 0x7efd10134260 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd16389700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd100a0590 con 0x7efd10133e30 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd16389700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd1009fb80 con 0x7efd10133420 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd0ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10133420 0x7efd101337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd0ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10133420 0x7efd101337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36524/0 (socket says 172.21.15.89:36524) 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd0ffff700 1 -- 172.21.15.89:0/1996074086 learned_addr learned my addr 172.21.15.89:0/1996074086 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.462+0000 7efd0f7fe700 1 --2- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133e30 0x7efd1012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0ffff700 1 -- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133e30 msgr2=0x7efd1012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0ffff700 1 --2- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133e30 0x7efd1012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0ffff700 1 -- 172.21.15.89:0/1996074086 >> v1:172.21.15.89:6789/0 conn(0x7efd10134260 legacy=0x7efd101304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0ffff700 1 -- 172.21.15.89:0/1996074086 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100a0030 con 0x7efd10133420 2023-09-30T22:18:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0f7fe700 1 --2- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133e30 0x7efd1012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0ffff700 1 --2- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10133420 0x7efd101337f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd00009920 tx=0x7efd00009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d13800da22bf697 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0effd700 1 -- 172.21.15.89:0/1996074086 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00004230 con 0x7efd10133420 2023-09-30T22:18:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0effd700 1 -- 172.21.15.89:0/1996074086 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd00004390 con 0x7efd10133420 2023-09-30T22:18:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.463+0000 7efd0effd700 1 -- 172.21.15.89:0/1996074086 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd000036a0 con 0x7efd10133420 2023-09-30T22:18:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.464+0000 7efd16389700 1 -- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10133420 msgr2=0x7efd101337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.464+0000 7efd16389700 1 --2- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10133420 0x7efd101337f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd00009920 tx=0x7efd00009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.464+0000 7efd16389700 1 -- 172.21.15.89:0/1996074086 shutdown_connections 2023-09-30T22:18:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.464+0000 7efd16389700 1 --2- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10133420 0x7efd101337f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.464+0000 7efd16389700 1 --2- 172.21.15.89:0/1996074086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133e30 0x7efd1012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.464+0000 7efd16389700 1 -- 172.21.15.89:0/1996074086 >> 172.21.15.89:0/1996074086 conn(0x7efd100a4210 msgr2=0x7efd100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.464+0000 7efd16389700 1 -- 172.21.15.89:0/1996074086 shutdown_connections 2023-09-30T22:18:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.464+0000 7efd16389700 1 -- 172.21.15.89:0/1996074086 wait complete. 2023-09-30T22:18:08.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.465+0000 7efd16389700 1 Processor -- start 2023-09-30T22:18:08.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.465+0000 7efd16389700 1 -- start start 2023-09-30T22:18:08.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd16389700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133420 0x7efd1009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd16389700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd10133e30 0x7efd1009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd16389700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10134260 0x7efd1009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd16389700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd10120140 con 0x7efd10133e30 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd16389700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1009dea0 con 0x7efd10133420 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd16389700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd1012daf0 con 0x7efd10134260 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd0f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd10133e30 0x7efd1009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd0f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd10133e30 0x7efd1009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43442/0 (socket says 172.21.15.89:43442) 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd0f7fe700 1 -- 172.21.15.89:0/785927087 learned_addr learned my addr 172.21.15.89:0/785927087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd14926700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10134260 0x7efd1009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.466+0000 7efd0ffff700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133420 0x7efd1009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd14926700 1 -- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133420 msgr2=0x7efd1009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd14926700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133420 0x7efd1009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd14926700 1 -- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd10133e30 msgr2=0x7efd1009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd14926700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd10133e30 0x7efd1009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd14926700 1 -- 172.21.15.89:0/785927087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040094a0 con 0x7efd10134260 2023-09-30T22:18:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd14926700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10134260 0x7efd1009cf00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd0800c780 tx=0x7efd0800ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd0cff9700 1 -- 172.21.15.89:0/785927087 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08003e10 con 0x7efd10134260 2023-09-30T22:18:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd0cff9700 1 -- 172.21.15.89:0/785927087 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd080050d0 con 0x7efd10134260 2023-09-30T22:18:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd00009550 con 0x7efd10134260 2023-09-30T22:18:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd0cff9700 1 -- 172.21.15.89:0/785927087 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08005480 con 0x7efd10134260 2023-09-30T22:18:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.467+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd10131890 con 0x7efd10134260 2023-09-30T22:18:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.468+0000 7efd0cff9700 1 -- 172.21.15.89:0/785927087 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efd080055e0 con 0x7efd10134260 2023-09-30T22:18:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.468+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1011ff80 con 0x7efd10134260 2023-09-30T22:18:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.469+0000 7efd0cff9700 1 -- 172.21.15.89:0/785927087 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efd0803e9a0 con 0x7efd10134260 2023-09-30T22:18:08.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.477+0000 7efd0cff9700 1 -- 172.21.15.89:0/785927087 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efd08017080 con 0x7efd10134260 2023-09-30T22:18:08.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.507+0000 7f4f8aa26700 1 Processor -- start 2023-09-30T22:18:08.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.507+0000 7f4f8aa26700 1 -- start start 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f8aa26700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 0x7f4f84135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f8aa26700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f84138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f8aa26700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8409f130 con 0x7f4f8409d6f0 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f8aa26700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8409f520 con 0x7f4f8409e1d0 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f8aa26700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8409e930 con 0x7f4f84135e50 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f88fc3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f84138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f88fc3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f84138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36550/0 (socket says 172.21.15.89:36550) 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f88fc3700 1 -- 172.21.15.89:0/2687970210 learned_addr learned my addr 172.21.15.89:0/2687970210 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f837fe700 1 --2- 172.21.15.89:0/2687970210 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 0x7f4f84135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f88fc3700 1 -- 172.21.15.89:0/2687970210 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 msgr2=0x7f4f84135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f88fc3700 1 --2- 172.21.15.89:0/2687970210 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 0x7f4f84135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f88fc3700 1 -- 172.21.15.89:0/2687970210 >> v1:172.21.15.89:6789/0 conn(0x7f4f8409d6f0 legacy=0x7f4f8409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f82ffd700 1 -- 172.21.15.89:0/2687970210 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967914413 0 0) 0x7f4f8409f130 con 0x7f4f8409d6f0 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f88fc3700 1 -- 172.21.15.89:0/2687970210 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8409b2d0 con 0x7f4f84135e50 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.508+0000 7f4f88fc3700 1 --2- 172.21.15.89:0/2687970210 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f84138230 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009e80 tx=0x7f4f7c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd133465fb7a0d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f82ffd700 1 -- 172.21.15.89:0/2687970210 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c00e070 con 0x7f4f84135e50 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f82ffd700 1 -- 172.21.15.89:0/2687970210 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c00b910 con 0x7f4f84135e50 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f82ffd700 1 -- 172.21.15.89:0/2687970210 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c00bcc0 con 0x7f4f84135e50 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f8aa26700 1 -- 172.21.15.89:0/2687970210 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 msgr2=0x7f4f84138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f8aa26700 1 --2- 172.21.15.89:0/2687970210 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f84138230 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009e80 tx=0x7f4f7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f8aa26700 1 -- 172.21.15.89:0/2687970210 shutdown_connections 2023-09-30T22:18:08.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f8aa26700 1 --2- 172.21.15.89:0/2687970210 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f84138230 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f8aa26700 1 --2- 172.21.15.89:0/2687970210 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 0x7f4f84135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f8aa26700 1 -- 172.21.15.89:0/2687970210 >> 172.21.15.89:0/2687970210 conn(0x7f4f84066f50 msgr2=0x7f4f8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:08.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f8aa26700 1 -- 172.21.15.89:0/2687970210 shutdown_connections 2023-09-30T22:18:08.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.509+0000 7f4f8aa26700 1 -- 172.21.15.89:0/2687970210 wait complete. 2023-09-30T22:18:08.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.510+0000 7f4f8aa26700 1 Processor -- start 2023-09-30T22:18:08.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.510+0000 7f4f8aa26700 1 -- start start 2023-09-30T22:18:08.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f8aa26700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8409d6f0 0x7f4f841c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f8aa26700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 0x7f4f841c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f8aa26700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f841c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f8aa26700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84120ee0 con 0x7f4f8409d6f0 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f8aa26700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8411f3e0 con 0x7f4f8409e1d0 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f8aa26700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840a2020 con 0x7f4f84135e50 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8409d6f0 0x7f4f841c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f88fc3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f841c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8409d6f0 0x7f4f841c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43446/0 (socket says 172.21.15.89:43446) 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 -- 172.21.15.89:0/1096155670 learned_addr learned my addr 172.21.15.89:0/1096155670 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f837fe700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 0x7f4f841c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 -- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 msgr2=0x7f4f841c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f841c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 -- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 msgr2=0x7f4f841c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 0x7f4f841c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 -- 172.21.15.89:0/1096155670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c009b30 con 0x7f4f8409d6f0 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f88fc3700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f841c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.511+0000 7f4f83fff700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8409d6f0 0x7f4f841c8170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400c450 tx=0x7f4f7400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.512+0000 7f4f80ff9700 1 -- 172.21.15.89:0/1096155670 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740054e0 con 0x7f4f8409d6f0 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.512+0000 7f4f80ff9700 1 -- 172.21.15.89:0/1096155670 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f74003680 con 0x7f4f8409d6f0 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.512+0000 7f4f80ff9700 1 -- 172.21.15.89:0/1096155670 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74003a30 con 0x7f4f8409d6f0 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.512+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8411faa0 con 0x7f4f8409d6f0 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.512+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f841211c0 con 0x7f4f8409d6f0 2023-09-30T22:18:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.513+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f84074ea0 con 0x7f4f8409d6f0 2023-09-30T22:18:08.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.521+0000 7f4f80ff9700 1 -- 172.21.15.89:0/1096155670 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f74003b90 con 0x7f4f8409d6f0 2023-09-30T22:18:08.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.522+0000 7f4f80ff9700 1 -- 172.21.15.89:0/1096155670 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4f7403f0b0 con 0x7f4f8409d6f0 2023-09-30T22:18:08.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.522+0000 7f4f80ff9700 1 -- 172.21.15.89:0/1096155670 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f74010df0 con 0x7f4f8409d6f0 2023-09-30T22:18:08.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.543+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1a68125e70 con 0x7f1a6809e000 2023-09-30T22:18:08.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.543+0000 7f1a5effd700 1 -- 172.21.15.89:0/3770456942 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a68125e70 con 0x7f1a6809e000 2023-09-30T22:18:08.544 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:08.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 msgr2=0x7f1a68122d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68122d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800ec10 tx=0x7f1a5800ef20 comp rx=0 tx=0).stop 2023-09-30T22:18:08.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 shutdown_connections 2023-09-30T22:18:08.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a6809e410 0x7f1a68123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a6809d6f0 0x7f1a681227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 --2- 172.21.15.89:0/3770456942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a6809e000 0x7f1a68122d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 >> 172.21.15.89:0/3770456942 conn(0x7f1a68066f50 msgr2=0x7f1a6806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:08.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 shutdown_connections 2023-09-30T22:18:08.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.545+0000 7f1a7070c700 1 -- 172.21.15.89:0/3770456942 wait complete. 2023-09-30T22:18:08.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:08.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.639+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f19cc074ea0 con 0x7f19cc09e410 2023-09-30T22:18:08.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.639+0000 7f19c8ff9700 1 -- 172.21.15.89:0/507414232 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19c400ab30 con 0x7f19cc09e410 2023-09-30T22:18:08.640 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:08.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19cc09e410 msgr2=0x7f19cc11e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19cc09e410 0x7f19cc11e8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f19c4008600 tx=0x7f19c4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 shutdown_connections 2023-09-30T22:18:08.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19cc09e410 0x7f19cc11e8a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19cc09d6f0 0x7f19cc11de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 --2- 172.21.15.89:0/507414232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19cc09e000 0x7f19cc11e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 >> 172.21.15.89:0/507414232 conn(0x7f19cc066f50 msgr2=0x7f19cc070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:08.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 shutdown_connections 2023-09-30T22:18:08.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.641+0000 7f19d27b1700 1 -- 172.21.15.89:0/507414232 wait complete. 2023-09-30T22:18:08.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.679+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efd10074ea0 con 0x7efd10134260 2023-09-30T22:18:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.679+0000 7efd0cff9700 1 -- 172.21.15.89:0/785927087 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd08018270 con 0x7efd10134260 2023-09-30T22:18:08.680 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10134260 msgr2=0x7efd1009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10134260 0x7efd1009cf00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd0800c780 tx=0x7efd0800ca90 comp rx=0 tx=0).stop 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 shutdown_connections 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd10134260 0x7efd1009cf00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd10133420 0x7efd1009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 --2- 172.21.15.89:0/785927087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd10133e30 0x7efd1009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 >> 172.21.15.89:0/785927087 conn(0x7efd100a4210 msgr2=0x7efd100a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 shutdown_connections 2023-09-30T22:18:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.681+0000 7efd16389700 1 -- 172.21.15.89:0/785927087 wait complete. 2023-09-30T22:18:08.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:08.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.718+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4f8406c360 con 0x7f4f8409d6f0 2023-09-30T22:18:08.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.718+0000 7f4f80ff9700 1 -- 172.21.15.89:0/1096155670 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f740192c0 con 0x7f4f8409d6f0 2023-09-30T22:18:08.719 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8409d6f0 msgr2=0x7f4f841c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8409d6f0 0x7f4f841c8170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400c450 tx=0x7f4f7400c760 comp rx=0 tx=0).stop 2023-09-30T22:18:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 shutdown_connections 2023-09-30T22:18:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f84135e50 0x7f4f841c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8409d6f0 0x7f4f841c8170 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 --2- 172.21.15.89:0/1096155670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f8409e1d0 0x7f4f841c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 >> 172.21.15.89:0/1096155670 conn(0x7f4f84066f50 msgr2=0x7f4f841329b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:08.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 shutdown_connections 2023-09-30T22:18:08.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:08.720+0000 7f4f8aa26700 1 -- 172.21.15.89:0/1096155670 wait complete. 2023-09-30T22:18:08.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:09.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:09.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:09.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:09.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.696+0000 7fb0688b4700 1 Processor -- start 2023-09-30T22:18:09.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.696+0000 7fb0688b4700 1 -- start start 2023-09-30T22:18:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb0688b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409e930 0x7fb06409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb0688b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb0688b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb06409f650 0x7fb064131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb0688b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb064070940 con 0x7fb06409f650 2023-09-30T22:18:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb0688b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06406fd60 con 0x7fb06409e930 2023-09-30T22:18:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb0688b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb06406fa80 con 0x7fb06409f240 2023-09-30T22:18:09.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb062d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb06409f650 0x7fb064131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb06259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409e930 0x7fb06409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb061d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb062d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb06409f650 0x7fb064131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43460/0 (socket says 172.21.15.89:43460) 2023-09-30T22:18:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb061d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36554/0 (socket says 172.21.15.89:36554) 2023-09-30T22:18:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb062d9d700 1 -- 172.21.15.89:0/1840155016 learned_addr learned my addr 172.21.15.89:0/1840155016 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:09.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb06259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409e930 0x7fb06409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59228/0 (socket says 172.21.15.89:59228) 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb062d9d700 1 -- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 msgr2=0x7fb0641317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb062d9d700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb062d9d700 1 -- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409e930 msgr2=0x7fb06409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb062d9d700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409e930 0x7fb06409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb062d9d700 1 -- 172.21.15.89:0/1840155016 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06409fb60 con 0x7fb06409f650 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb06259c700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409e930 0x7fb06409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.697+0000 7fb061d9b700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.698+0000 7fb062d9d700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409f650 0x7fb064131d30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00f370 tx=0x7fb05c00f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461c0061184b0d8d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:09.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.698+0000 7fb06159a700 1 -- 172.21.15.89:0/1840155016 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c012070 con 0x7fb06409f650 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.698+0000 7fb06159a700 1 -- 172.21.15.89:0/1840155016 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c00d0d0 con 0x7fb06409f650 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.698+0000 7fb06159a700 1 -- 172.21.15.89:0/1840155016 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c00d480 con 0x7fb06409f650 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.698+0000 7fb0688b4700 1 -- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409f650 msgr2=0x7fb064131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.698+0000 7fb0688b4700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409f650 0x7fb064131d30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00f370 tx=0x7fb05c00f680 comp rx=0 tx=0).stop 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.699+0000 7fb0688b4700 1 -- 172.21.15.89:0/1840155016 shutdown_connections 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.699+0000 7fb0688b4700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409f650 0x7fb064131d30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.699+0000 7fb0688b4700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409e930 0x7fb06409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.699+0000 7fb0688b4700 1 --2- 172.21.15.89:0/1840155016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.699+0000 7fb0688b4700 1 -- 172.21.15.89:0/1840155016 >> 172.21.15.89:0/1840155016 conn(0x7fb0640a4210 msgr2=0x7fb0640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.699+0000 7fb0688b4700 1 -- 172.21.15.89:0/1840155016 shutdown_connections 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.699+0000 7fb0688b4700 1 -- 172.21.15.89:0/1840155016 wait complete. 2023-09-30T22:18:09.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb0688b4700 1 Processor -- start 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb0688b4700 1 -- start start 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb0688b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409e930 0x7fb0641a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb0688b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb0688b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409f650 0x7fb0641a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb0688b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641a57c0 con 0x7fb06409e930 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb0688b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641a6000 con 0x7fb06409f650 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb0688b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641a3730 con 0x7fb06409f240 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb06259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409e930 0x7fb0641a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb06259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409e930 0x7fb0641a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43466/0 (socket says 172.21.15.89:43466) 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb06259c700 1 -- 172.21.15.89:0/3048152672 learned_addr learned my addr 172.21.15.89:0/3048152672 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:09.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.700+0000 7fb062d9d700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409f650 0x7fb0641a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb06259c700 1 -- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 msgr2=0x7fb0641a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb06259c700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb06259c700 1 -- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409f650 msgr2=0x7fb0641a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb06259c700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409f650 0x7fb0641a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb06259c700 1 -- 172.21.15.89:0/3048152672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540094a0 con 0x7fb06409e930 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb062d9d700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409f650 0x7fb0641a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb06259c700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409e930 0x7fb0641a1b20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb05400ec40 tx=0x7fb05400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb052ffd700 1 -- 172.21.15.89:0/3048152672 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054009780 con 0x7fb06409e930 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb052ffd700 1 -- 172.21.15.89:0/3048152672 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05400c910 con 0x7fb06409e930 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0580094a0 con 0x7fb06409e930 2023-09-30T22:18:09.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb052ffd700 1 -- 172.21.15.89:0/3048152672 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05400cca0 con 0x7fb06409e930 2023-09-30T22:18:09.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.701+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c00f050 con 0x7fb06409e930 2023-09-30T22:18:09.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.702+0000 7fb052ffd700 1 -- 172.21.15.89:0/3048152672 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb054013470 con 0x7fb06409e930 2023-09-30T22:18:09.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.702+0000 7fb052ffd700 1 -- 172.21.15.89:0/3048152672 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb05403f800 con 0x7fb06409e930 2023-09-30T22:18:09.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.702+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb064071d80 con 0x7fb06409e930 2023-09-30T22:18:09.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.708+0000 7fb052ffd700 1 -- 172.21.15.89:0/3048152672 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb054013720 con 0x7fb06409e930 2023-09-30T22:18:09.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.794+0000 7f997031b700 1 Processor -- start 2023-09-30T22:18:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.794+0000 7f997031b700 1 -- start start 2023-09-30T22:18:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f997031b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f996809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f997031b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f996809e000 0x7f99681339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f997031b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996809b120 con 0x7f996809e410 2023-09-30T22:18:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f997031b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996806b460 con 0x7f996809e000 2023-09-30T22:18:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f997031b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f996809a7e0 con 0x7f996809d6f0 2023-09-30T22:18:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f996d8b6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f996809e000 0x7f99681339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f996d8b6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f996809e000 0x7f99681339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43480/0 (socket says 172.21.15.89:43480) 2023-09-30T22:18:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f996d8b6700 1 -- 172.21.15.89:0/795224412 learned_addr learned my addr 172.21.15.89:0/795224412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.795+0000 7f996e0b7700 1 --2- 172.21.15.89:0/795224412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f996809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.796+0000 7f996d8b6700 1 -- 172.21.15.89:0/795224412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 msgr2=0x7f996809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.796+0000 7f996d8b6700 1 --2- 172.21.15.89:0/795224412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f996809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.796+0000 7f996d8b6700 1 -- 172.21.15.89:0/795224412 >> v1:172.21.15.89:6789/0 conn(0x7f996809e410 legacy=0x7f9968133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.796+0000 7f996d8b6700 1 -- 172.21.15.89:0/795224412 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996809a500 con 0x7f996809e000 2023-09-30T22:18:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.796+0000 7f996d8b6700 1 --2- 172.21.15.89:0/795224412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 0x7f99681339f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f99580098f0 tx=0x7f9958009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473280f26c96ad3a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.796+0000 7f996d0b5700 1 -- 172.21.15.89:0/795224412 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958011830 con 0x7f996809e000 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.796+0000 7f996d0b5700 1 -- 172.21.15.89:0/795224412 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9958011990 con 0x7f996809e000 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.796+0000 7f996d0b5700 1 -- 172.21.15.89:0/795224412 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958011d20 con 0x7f996809e000 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.797+0000 7f997031b700 1 -- 172.21.15.89:0/795224412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 msgr2=0x7f99681339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.797+0000 7f997031b700 1 --2- 172.21.15.89:0/795224412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 0x7f99681339f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f99580098f0 tx=0x7f9958009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.797+0000 7f997031b700 1 -- 172.21.15.89:0/795224412 shutdown_connections 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.797+0000 7f997031b700 1 --2- 172.21.15.89:0/795224412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f996809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.797+0000 7f997031b700 1 --2- 172.21.15.89:0/795224412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 0x7f99681339f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.797+0000 7f997031b700 1 -- 172.21.15.89:0/795224412 >> 172.21.15.89:0/795224412 conn(0x7f9968066f50 msgr2=0x7f996809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.797+0000 7f997031b700 1 -- 172.21.15.89:0/795224412 shutdown_connections 2023-09-30T22:18:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.797+0000 7f997031b700 1 -- 172.21.15.89:0/795224412 wait complete. 2023-09-30T22:18:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.798+0000 7f997031b700 1 Processor -- start 2023-09-30T22:18:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.798+0000 7f997031b700 1 -- start start 2023-09-30T22:18:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f997031b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f99681a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f997031b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 0x7f99681a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f997031b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f996809e410 0x7f99681a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f997031b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99681a8800 con 0x7f996809e000 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f997031b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99681a6510 con 0x7f996809e410 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f997031b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99680a2fd0 con 0x7f996809d6f0 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f99681a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f99681a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36594/0 (socket says 172.21.15.89:36594) 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 -- 172.21.15.89:0/1053475227 learned_addr learned my addr 172.21.15.89:0/1053475227 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e8b8700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f996809e410 0x7f99681a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996d8b6700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 0x7f99681a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 -- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f996809e410 msgr2=0x7f99681a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f996809e410 0x7f99681a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 -- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 msgr2=0x7f99681a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 0x7f99681a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 -- 172.21.15.89:0/1053475227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958009550 con 0x7f996809d6f0 2023-09-30T22:18:09.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996d8b6700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 0x7f99681a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:09.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.799+0000 7f996e0b7700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f99681a5120 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f996400e9e0 tx=0x7f996400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:09.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.800+0000 7f995effd700 1 -- 172.21.15.89:0/1053475227 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964009710 con 0x7f996809d6f0 2023-09-30T22:18:09.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.800+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99681a6ce0 con 0x7f996809d6f0 2023-09-30T22:18:09.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.800+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99681a7e20 con 0x7f996809d6f0 2023-09-30T22:18:09.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.800+0000 7f995effd700 1 -- 172.21.15.89:0/1053475227 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996400c850 con 0x7f996809d6f0 2023-09-30T22:18:09.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.800+0000 7f995effd700 1 -- 172.21.15.89:0/1053475227 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996400cbc0 con 0x7f996809d6f0 2023-09-30T22:18:09.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.801+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99500052e0 con 0x7f996809d6f0 2023-09-30T22:18:09.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.801+0000 7f995effd700 1 -- 172.21.15.89:0/1053475227 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f996400cd20 con 0x7f996809d6f0 2023-09-30T22:18:09.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.801+0000 7f995effd700 1 -- 172.21.15.89:0/1053475227 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f996403fee0 con 0x7f996809d6f0 2023-09-30T22:18:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.809+0000 7f995effd700 1 -- 172.21.15.89:0/1053475227 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f996401ab50 con 0x7f996809d6f0 2023-09-30T22:18:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.830+0000 7f0b0f81a700 1 Processor -- start 2023-09-30T22:18:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.830+0000 7f0b0f81a700 1 -- start start 2023-09-30T22:18:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0f81a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b0806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0f81a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b08131d10 0x7f0b08139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0f81a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0809ece0 con 0x7f0b0806b080 2023-09-30T22:18:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0f81a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0809e760 con 0x7f0b08131d10 2023-09-30T22:18:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0f81a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0809e040 con 0x7f0b0806a770 2023-09-30T22:18:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0d5b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b0806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0ddb7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b08131d10 0x7f0b08139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0d5b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b0806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36600/0 (socket says 172.21.15.89:36600) 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0d5b6700 1 -- 172.21.15.89:0/1074435809 learned_addr learned my addr 172.21.15.89:0/1074435809 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0d5b6700 1 -- 172.21.15.89:0/1074435809 >> v2:172.21.15.89:3300/0 conn(0x7f0b08131d10 msgr2=0x7f0b08139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0d5b6700 1 --2- 172.21.15.89:0/1074435809 >> v2:172.21.15.89:3300/0 conn(0x7f0b08131d10 0x7f0b08139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0d5b6700 1 -- 172.21.15.89:0/1074435809 >> v1:172.21.15.89:6789/0 conn(0x7f0b0806b080 legacy=0x7f0b08131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0d5b6700 1 -- 172.21.15.89:0/1074435809 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0809f630 con 0x7f0b0806a770 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0b0d5b6700 1 --2- 172.21.15.89:0/1074435809 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b0806ab40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b040098f0 tx=0x7f0b04009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03b5f2f32e4145 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.831+0000 7f0afffff700 1 -- 172.21.15.89:0/1074435809 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04011830 con 0x7f0b0806a770 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.832+0000 7f0afffff700 1 -- 172.21.15.89:0/1074435809 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b04011990 con 0x7f0b0806a770 2023-09-30T22:18:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.832+0000 7f0afffff700 1 -- 172.21.15.89:0/1074435809 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04011d20 con 0x7f0b0806a770 2023-09-30T22:18:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.832+0000 7f0b0f81a700 1 -- 172.21.15.89:0/1074435809 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 msgr2=0x7f0b0806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.832+0000 7f0b0f81a700 1 --2- 172.21.15.89:0/1074435809 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b0806ab40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b040098f0 tx=0x7f0b04009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.832+0000 7f0b0f81a700 1 -- 172.21.15.89:0/1074435809 shutdown_connections 2023-09-30T22:18:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.832+0000 7f0b0f81a700 1 --2- 172.21.15.89:0/1074435809 >> v2:172.21.15.89:3300/0 conn(0x7f0b08131d10 0x7f0b08139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.832+0000 7f0b0f81a700 1 --2- 172.21.15.89:0/1074435809 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b0806ab40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.832+0000 7f0b0f81a700 1 -- 172.21.15.89:0/1074435809 >> 172.21.15.89:0/1074435809 conn(0x7f0b08066f50 msgr2=0x7f0b0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.833+0000 7f0b0f81a700 1 -- 172.21.15.89:0/1074435809 shutdown_connections 2023-09-30T22:18:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.833+0000 7f0b0f81a700 1 -- 172.21.15.89:0/1074435809 wait complete. 2023-09-30T22:18:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.834+0000 7f0b0f81a700 1 Processor -- start 2023-09-30T22:18:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.834+0000 7f0b0f81a700 1 -- start start 2023-09-30T22:18:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.834+0000 7f0b0f81a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b081c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.834+0000 7f0b0f81a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b0806b080 0x7f0b081c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.834+0000 7f0b0f81a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b08131d10 0x7f0b0811d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.834+0000 7f0b0f81a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0811f100 con 0x7f0b0806b080 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.834+0000 7f0b0f81a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08120880 con 0x7f0b08131d10 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.834+0000 7f0b0f81a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0812fe80 con 0x7f0b0806a770 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b081c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b081c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36614/0 (socket says 172.21.15.89:36614) 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 -- 172.21.15.89:0/2322016001 learned_addr learned my addr 172.21.15.89:0/2322016001 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0cdb5700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b0806b080 0x7f0b081c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 -- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b08131d10 msgr2=0x7f0b0811d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b08131d10 0x7f0b0811d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 -- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b0806b080 msgr2=0x7f0b081c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b0806b080 0x7f0b081c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 -- 172.21.15.89:0/2322016001 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04009550 con 0x7f0b0806a770 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0cdb5700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b0806b080 0x7f0b081c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0d5b6700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b081c4310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009c40 tx=0x7f0b04004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0afdffb700 1 -- 172.21.15.89:0/2322016001 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04004400 con 0x7f0b0806a770 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0afdffb700 1 -- 172.21.15.89:0/2322016001 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0401f070 con 0x7f0b0806a770 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b08121010 con 0x7f0b0806a770 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.835+0000 7f0afdffb700 1 -- 172.21.15.89:0/2322016001 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040038c0 con 0x7f0b0806a770 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.836+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0809a3d0 con 0x7f0b0806a770 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.836+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b08071e60 con 0x7f0b0806a770 2023-09-30T22:18:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.837+0000 7f0afdffb700 1 -- 172.21.15.89:0/2322016001 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b04021030 con 0x7f0b0806a770 2023-09-30T22:18:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.837+0000 7f0afdffb700 1 -- 172.21.15.89:0/2322016001 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0b04040f10 con 0x7f0b0806a770 2023-09-30T22:18:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.841+0000 7f0afdffb700 1 -- 172.21.15.89:0/2322016001 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b040199a0 con 0x7f0b0806a770 2023-09-30T22:18:09.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.866+0000 7fa62663a700 1 Processor -- start 2023-09-30T22:18:09.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.866+0000 7fa62663a700 1 -- start start 2023-09-30T22:18:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.866+0000 7fa62663a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa62663a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa620130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa62663a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62009f130 con 0x7fa62009e410 2023-09-30T22:18:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa62663a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62009f520 con 0x7fa62009d6f0 2023-09-30T22:18:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa62663a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa62009e930 con 0x7fa62009e000 2023-09-30T22:18:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59256/0 (socket says 172.21.15.89:59256) 2023-09-30T22:18:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61ffff700 1 -- 172.21.15.89:0/3743171262 learned_addr learned my addr 172.21.15.89:0/3743171262 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61f7fe700 1 --2- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa620130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61f7fe700 1 -- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 msgr2=0x7fa62009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61f7fe700 1 --2- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61f7fe700 1 -- 172.21.15.89:0/3743171262 >> v1:172.21.15.89:6789/0 conn(0x7fa62009e410 legacy=0x7fa620130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61f7fe700 1 -- 172.21.15.89:0/3743171262 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62009b2d0 con 0x7fa62009e000 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61ffff700 1 --2- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61f7fe700 1 --2- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa620130660 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098a0 tx=0x7fa614009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15bca2d24fd9204c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.867+0000 7fa61effd700 1 -- 172.21.15.89:0/3743171262 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614011830 con 0x7fa62009e000 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa61effd700 1 -- 172.21.15.89:0/3743171262 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa614011990 con 0x7fa62009e000 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa61effd700 1 -- 172.21.15.89:0/3743171262 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614011d20 con 0x7fa62009e000 2023-09-30T22:18:09.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa62663a700 1 -- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 msgr2=0x7fa620130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa62663a700 1 --2- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa620130660 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098a0 tx=0x7fa614009bb0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa62663a700 1 -- 172.21.15.89:0/3743171262 shutdown_connections 2023-09-30T22:18:09.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa62663a700 1 --2- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa62663a700 1 --2- 172.21.15.89:0/3743171262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa620130660 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa62663a700 1 -- 172.21.15.89:0/3743171262 >> 172.21.15.89:0/3743171262 conn(0x7fa620066f50 msgr2=0x7fa62009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:09.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa62663a700 1 -- 172.21.15.89:0/3743171262 shutdown_connections 2023-09-30T22:18:09.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.868+0000 7fa62663a700 1 -- 172.21.15.89:0/3743171262 wait complete. 2023-09-30T22:18:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa62663a700 1 Processor -- start 2023-09-30T22:18:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa62663a700 1 -- start start 2023-09-30T22:18:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa62663a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62012dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa62663a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa62012e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa62663a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62009e410 0x7fa62012e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa62663a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa620139370 con 0x7fa62009e410 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa62663a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62009c770 con 0x7fa62009d6f0 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa62663a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa620131840 con 0x7fa62009e000 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa61ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62012dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa61f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa62012e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa624bd7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62009e410 0x7fa62012e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa61ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62012dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59272/0 (socket says 172.21.15.89:59272) 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.870+0000 7fa61ffff700 1 -- 172.21.15.89:0/3027643912 learned_addr learned my addr 172.21.15.89:0/3027643912 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61f7fe700 1 -- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 msgr2=0x7fa62012dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61f7fe700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62012dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61f7fe700 1 -- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62009e410 msgr2=0x7fa62012e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61f7fe700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62009e410 0x7fa62012e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61f7fe700 1 -- 172.21.15.89:0/3027643912 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610009550 con 0x7fa62009e000 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa624bd7700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62009e410 0x7fa62012e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61ffff700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62012dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61f7fe700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa62012e1e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa620072a80 tx=0x7fa614004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61cff9700 1 -- 172.21.15.89:0/3027643912 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6140036a0 con 0x7fa62009e000 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61cff9700 1 -- 172.21.15.89:0/3027643912 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61401f070 con 0x7fa62009e000 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa61cff9700 1 -- 172.21.15.89:0/3027643912 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614003a60 con 0x7fa62009e000 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6140094a0 con 0x7fa62009e000 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.871+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa620072490 con 0x7fa62009e000 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.872+0000 7fa61cff9700 1 -- 172.21.15.89:0/3027643912 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa614003bc0 con 0x7fa62009e000 2023-09-30T22:18:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.872+0000 7fa61cff9700 1 -- 172.21.15.89:0/3027643912 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa614021070 con 0x7fa62009e000 2023-09-30T22:18:09.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.873+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62006f470 con 0x7fa62009e000 2023-09-30T22:18:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.877+0000 7fa61cff9700 1 -- 172.21.15.89:0/3027643912 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa614015080 con 0x7fa62009e000 2023-09-30T22:18:09.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.957+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb06406b980 con 0x7fb06409e930 2023-09-30T22:18:09.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.958+0000 7fb052ffd700 1 -- 172.21.15.89:0/3048152672 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb05401bb60 con 0x7fb06409e930 2023-09-30T22:18:09.959 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:09.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409e930 msgr2=0x7fb0641a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:09.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409e930 0x7fb0641a1b20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb05400ec40 tx=0x7fb05400ef50 comp rx=0 tx=0).stop 2023-09-30T22:18:09.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 shutdown_connections 2023-09-30T22:18:09.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb06409f650 0x7fb0641a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb06409e930 0x7fb0641a1b20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 --2- 172.21.15.89:0/3048152672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb06409f240 0x7fb0641a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:09.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 >> 172.21.15.89:0/3048152672 conn(0x7fb0640a4210 msgr2=0x7fb0641b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:09.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 shutdown_connections 2023-09-30T22:18:09.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:09.961+0000 7fb0688b4700 1 -- 172.21.15.89:0/3048152672 wait complete. 2023-09-30T22:18:09.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.035+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0b08074ea0 con 0x7f0b0806a770 2023-09-30T22:18:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.035+0000 7f0afdffb700 1 -- 172.21.15.89:0/2322016001 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b04015030 con 0x7f0b0806a770 2023-09-30T22:18:10.036 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 msgr2=0x7f0b081c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b081c4310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009c40 tx=0x7f0b04004750 comp rx=0 tx=0).stop 2023-09-30T22:18:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 shutdown_connections 2023-09-30T22:18:10.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b08131d10 0x7f0b0811d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b0806a770 0x7f0b081c4310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 --2- 172.21.15.89:0/2322016001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b0806b080 0x7f0b081c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 >> 172.21.15.89:0/2322016001 conn(0x7f0b08066f50 msgr2=0x7f0b080a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:10.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 shutdown_connections 2023-09-30T22:18:10.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.037+0000 7f0b0f81a700 1 -- 172.21.15.89:0/2322016001 wait complete. 2023-09-30T22:18:10.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.046+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9950005590 con 0x7f996809d6f0 2023-09-30T22:18:10.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.046+0000 7f995effd700 1 -- 172.21.15.89:0/1053475227 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9964021070 con 0x7f996809d6f0 2023-09-30T22:18:10.047 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:10.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.048+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 msgr2=0x7f99681a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:10.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.048+0000 7f997031b700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f99681a5120 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f996400e9e0 tx=0x7f996400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.049+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 shutdown_connections 2023-09-30T22:18:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.049+0000 7f997031b700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f996809e410 0x7f99681a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.049+0000 7f997031b700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f996809d6f0 0x7f99681a5120 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.049+0000 7f997031b700 1 --2- 172.21.15.89:0/1053475227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f996809e000 0x7f99681a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.049+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 >> 172.21.15.89:0/1053475227 conn(0x7f9968066f50 msgr2=0x7f99681bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.049+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 shutdown_connections 2023-09-30T22:18:10.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.049+0000 7f997031b700 1 -- 172.21.15.89:0/1053475227 wait complete. 2023-09-30T22:18:10.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:10.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:10.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.079+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa620074ea0 con 0x7fa62009e000 2023-09-30T22:18:10.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.080+0000 7fa61cff9700 1 -- 172.21.15.89:0/3027643912 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa61402b020 con 0x7fa62009e000 2023-09-30T22:18:10.081 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:10.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 msgr2=0x7fa62012e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:10.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa62012e1e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa620072a80 tx=0x7fa614004750 comp rx=0 tx=0).stop 2023-09-30T22:18:10.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 shutdown_connections 2023-09-30T22:18:10.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62009e410 0x7fa62012e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62009d6f0 0x7fa62012dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 --2- 172.21.15.89:0/3027643912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62009e000 0x7fa62012e1e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:10.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 >> 172.21.15.89:0/3027643912 conn(0x7fa620066f50 msgr2=0x7fa6200a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:10.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 shutdown_connections 2023-09-30T22:18:10.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:10.082+0000 7fa62663a700 1 -- 172.21.15.89:0/3027643912 wait complete. 2023-09-30T22:18:10.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:10.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:11.052 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:11.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:11.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:11.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.120+0000 7f35706d0700 1 Processor -- start 2023-09-30T22:18:11.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.120+0000 7f35706d0700 1 -- start start 2023-09-30T22:18:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.120+0000 7f35706d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809d6f0 0x7f356809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.120+0000 7f35706d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809e410 0x7f3568130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.120+0000 7f35706d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356809f130 con 0x7f356809e000 2023-09-30T22:18:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.120+0000 7f35706d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356809f520 con 0x7f356809d6f0 2023-09-30T22:18:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.120+0000 7f35706d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f356809e930 con 0x7f356809e410 2023-09-30T22:18:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356ec6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809e410 0x7f3568130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356ec6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809e410 0x7f3568130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43024/0 (socket says 172.21.15.89:43024) 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356ec6d700 1 -- 172.21.15.89:0/403149757 learned_addr learned my addr 172.21.15.89:0/403149757 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356e46c700 1 --2- 172.21.15.89:0/403149757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809d6f0 0x7f356809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356ec6d700 1 -- 172.21.15.89:0/403149757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809d6f0 msgr2=0x7f356809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356ec6d700 1 --2- 172.21.15.89:0/403149757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809d6f0 0x7f356809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356ec6d700 1 -- 172.21.15.89:0/403149757 >> v1:172.21.15.89:6789/0 conn(0x7f356809e000 legacy=0x7f3568130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356ec6d700 1 -- 172.21.15.89:0/403149757 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356809b2d0 con 0x7f356809e410 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356ec6d700 1 --2- 172.21.15.89:0/403149757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809e410 0x7f3568130d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3560009e80 tx=0x7f356000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e4c6e7e3ea4c7ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356d46a700 1 -- 172.21.15.89:0/403149757 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000e070 con 0x7f356809e410 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356d46a700 1 -- 172.21.15.89:0/403149757 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356000b9c0 con 0x7f356809e410 2023-09-30T22:18:11.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.121+0000 7f356d46a700 1 -- 172.21.15.89:0/403149757 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000bd70 con 0x7f356809e410 2023-09-30T22:18:11.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.122+0000 7f35706d0700 1 -- 172.21.15.89:0/403149757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809e410 msgr2=0x7f3568130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.122+0000 7f35706d0700 1 --2- 172.21.15.89:0/403149757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809e410 0x7f3568130d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3560009e80 tx=0x7f356000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:11.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.123+0000 7f35706d0700 1 -- 172.21.15.89:0/403149757 shutdown_connections 2023-09-30T22:18:11.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.123+0000 7f35706d0700 1 --2- 172.21.15.89:0/403149757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809d6f0 0x7f356809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.123+0000 7f35706d0700 1 --2- 172.21.15.89:0/403149757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809e410 0x7f3568130d70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.123+0000 7f35706d0700 1 -- 172.21.15.89:0/403149757 >> 172.21.15.89:0/403149757 conn(0x7f3568066f50 msgr2=0x7f356809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:11.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.123+0000 7f35706d0700 1 -- 172.21.15.89:0/403149757 shutdown_connections 2023-09-30T22:18:11.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.123+0000 7f35706d0700 1 -- 172.21.15.89:0/403149757 wait complete. 2023-09-30T22:18:11.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.124+0000 7f35706d0700 1 Processor -- start 2023-09-30T22:18:11.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.124+0000 7f35706d0700 1 -- start start 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f35706d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809d6f0 0x7f3568122390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f35706d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809e000 0x7f35681228d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f35706d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f356809e410 0x7f3568122e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f35706d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568123f10 con 0x7f356809e410 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f35706d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35681247a0 con 0x7f356809e000 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f35706d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3568125e20 con 0x7f356809d6f0 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809d6f0 0x7f3568122390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809d6f0 0x7f3568122390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43028/0 (socket says 172.21.15.89:43028) 2023-09-30T22:18:11.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 -- 172.21.15.89:0/2690384148 learned_addr learned my addr 172.21.15.89:0/2690384148 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356ec6d700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f356809e410 0x7f3568122e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 -- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809e000 msgr2=0x7f35681228d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809e000 0x7f35681228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 -- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f356809e410 msgr2=0x7f3568122e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f356809e410 0x7f3568122e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 -- 172.21.15.89:0/2690384148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3560009b30 con 0x7f356809d6f0 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356ec6d700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f356809e410 0x7f3568122e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.125+0000 7f356e46c700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809d6f0 0x7f3568122390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35640086b0 tx=0x7f3564008a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.126+0000 7f355effd700 1 -- 172.21.15.89:0/2690384148 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564010070 con 0x7f356809d6f0 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.126+0000 7f355effd700 1 -- 172.21.15.89:0/2690384148 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356400d7b0 con 0x7f356809d6f0 2023-09-30T22:18:11.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.126+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3568125880 con 0x7f356809d6f0 2023-09-30T22:18:11.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.126+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3568125b40 con 0x7f356809d6f0 2023-09-30T22:18:11.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.126+0000 7f355effd700 1 -- 172.21.15.89:0/2690384148 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400db40 con 0x7f356809d6f0 2023-09-30T22:18:11.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.127+0000 7f355effd700 1 -- 172.21.15.89:0/2690384148 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f356400dca0 con 0x7f356809d6f0 2023-09-30T22:18:11.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.127+0000 7f355effd700 1 -- 172.21.15.89:0/2690384148 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3564042220 con 0x7f356809d6f0 2023-09-30T22:18:11.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.127+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35500052e0 con 0x7f356809d6f0 2023-09-30T22:18:11.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.132+0000 7f355effd700 1 -- 172.21.15.89:0/2690384148 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3564019030 con 0x7f356809d6f0 2023-09-30T22:18:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.194+0000 7f46b9c33700 1 Processor -- start 2023-09-30T22:18:11.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.194+0000 7f46b9c33700 1 -- start start 2023-09-30T22:18:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.194+0000 7f46b9c33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b40a0a30 0x7f46b40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.194+0000 7f46b9c33700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46b409d630 0x7f46b40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.194+0000 7f46b9c33700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4133930 0x7f46b4133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.194+0000 7f46b9c33700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40a2a70 con 0x7f46b409d630 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.194+0000 7f46b9c33700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40a01d0 con 0x7f46b40a0a30 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.194+0000 7f46b9c33700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46b409e2b0 con 0x7f46b4133930 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b40a0a30 0x7f46b40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b40a0a30 0x7f46b40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36076/0 (socket says 172.21.15.89:36076) 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 -- 172.21.15.89:0/3960075081 learned_addr learned my addr 172.21.15.89:0/3960075081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b3fff700 1 --2- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4133930 0x7f46b4133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b2ffd700 1 --2- 172.21.15.89:0/3960075081 >> v2:172.21.15.89:3300/0 conn(0x7f46b409d630 0x7f46b40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 -- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4133930 msgr2=0x7f46b4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:11.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 --2- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4133930 0x7f46b4133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 -- 172.21.15.89:0/3960075081 >> v2:172.21.15.89:3300/0 conn(0x7f46b409d630 msgr2=0x7f46b40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 --2- 172.21.15.89:0/3960075081 >> v2:172.21.15.89:3300/0 conn(0x7f46b409d630 0x7f46b40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 -- 172.21.15.89:0/3960075081 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b409dc50 con 0x7f46b40a0a30 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b3fff700 1 --2- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4133930 0x7f46b4133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b2ffd700 1 --2- 172.21.15.89:0/3960075081 >> v2:172.21.15.89:3300/0 conn(0x7f46b409d630 0x7f46b40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b37fe700 1 --2- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b40a0a30 0x7f46b40a0e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f46a40098f0 tx=0x7f46a4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3db199dbb7d532 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b27fc700 1 -- 172.21.15.89:0/3960075081 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4011830 con 0x7f46b40a0a30 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b27fc700 1 -- 172.21.15.89:0/3960075081 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a4011990 con 0x7f46b40a0a30 2023-09-30T22:18:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.195+0000 7f46b27fc700 1 -- 172.21.15.89:0/3960075081 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4011d20 con 0x7f46b40a0a30 2023-09-30T22:18:11.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.196+0000 7f46b9c33700 1 -- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b40a0a30 msgr2=0x7f46b40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.196+0000 7f46b9c33700 1 --2- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b40a0a30 0x7f46b40a0e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f46a40098f0 tx=0x7f46a4009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.196+0000 7f46b9c33700 1 -- 172.21.15.89:0/3960075081 shutdown_connections 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.196+0000 7f46b9c33700 1 --2- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4133930 0x7f46b4133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.196+0000 7f46b9c33700 1 --2- 172.21.15.89:0/3960075081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b40a0a30 0x7f46b40a0e00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.196+0000 7f46b9c33700 1 --2- 172.21.15.89:0/3960075081 >> v2:172.21.15.89:3300/0 conn(0x7f46b409d630 0x7f46b40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.196+0000 7f46b9c33700 1 -- 172.21.15.89:0/3960075081 >> 172.21.15.89:0/3960075081 conn(0x7f46b40a3050 msgr2=0x7f46b40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.197+0000 7f46b9c33700 1 -- 172.21.15.89:0/3960075081 shutdown_connections 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.197+0000 7f46b9c33700 1 -- 172.21.15.89:0/3960075081 wait complete. 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.198+0000 7f46b9c33700 1 Processor -- start 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.198+0000 7f46b9c33700 1 -- start start 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b9c33700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b409d630 0x7f46b41c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b9c33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40a0a30 0x7f46b41c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b9c33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b4133930 0x7f46b41c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b9c33700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4120820 con 0x7f46b40a0a30 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b9c33700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4122010 con 0x7f46b4133930 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b9c33700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40a41c0 con 0x7f46b409d630 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b4133930 0x7f46b41c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b409d630 0x7f46b41c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b4133930 0x7f46b41c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36084/0 (socket says 172.21.15.89:36084) 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b3fff700 1 -- 172.21.15.89:0/3689534345 learned_addr learned my addr 172.21.15.89:0/3689534345 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.199+0000 7f46b2ffd700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40a0a30 0x7f46b41c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f46b37fe700 1 -- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b4133930 msgr2=0x7f46b41c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f46b37fe700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b4133930 0x7f46b41c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f46b37fe700 1 -- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40a0a30 msgr2=0x7f46b41c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f46b37fe700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40a0a30 0x7f46b41c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f46b37fe700 1 -- 172.21.15.89:0/3689534345 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80094a0 con 0x7f46b409d630 2023-09-30T22:18:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f46b3fff700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b4133930 0x7f46b41c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f46b2ffd700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40a0a30 0x7f46b41c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f46b37fe700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b409d630 0x7f46b41c8090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46a4016000 tx=0x7f46a4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f469bfff700 1 -- 172.21.15.89:0/3689534345 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4004240 con 0x7f46b409d630 2023-09-30T22:18:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f469bfff700 1 -- 172.21.15.89:0/3689534345 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a401f070 con 0x7f46b409d630 2023-09-30T22:18:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.200+0000 7f469bfff700 1 -- 172.21.15.89:0/3689534345 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a40036a0 con 0x7f46b409d630 2023-09-30T22:18:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.201+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac009a70 con 0x7f46b409d630 2023-09-30T22:18:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.201+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a4009550 con 0x7f46b409d630 2023-09-30T22:18:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.202+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b40747f0 con 0x7f46b409d630 2023-09-30T22:18:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.203+0000 7f469bfff700 1 -- 172.21.15.89:0/3689534345 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46a4021030 con 0x7f46b409d630 2023-09-30T22:18:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.203+0000 7f469bfff700 1 -- 172.21.15.89:0/3689534345 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f46a4040800 con 0x7f46b409d630 2023-09-30T22:18:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.207+0000 7f469bfff700 1 -- 172.21.15.89:0/3689534345 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46a40239c0 con 0x7f46b409d630 2023-09-30T22:18:11.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.247+0000 7fbfdb903700 1 Processor -- start 2023-09-30T22:18:11.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.247+0000 7fbfdb903700 1 -- start start 2023-09-30T22:18:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.247+0000 7fbfdb903700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a09c0 0x7fbfd406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.247+0000 7fbfdb903700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbfd40a0d90 0x7fbfd40a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.247+0000 7fbfdb903700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd41338a0 0x7fbfd4133d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.247+0000 7fbfdb903700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd40a2a20 con 0x7fbfd40a0d90 2023-09-30T22:18:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.247+0000 7fbfdb903700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd409e690 con 0x7fbfd41338a0 2023-09-30T22:18:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.247+0000 7fbfdb903700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40a1cc0 con 0x7fbfd40a09c0 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a09c0 0x7fbfd406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a09c0 0x7fbfd406c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43064/0 (socket says 172.21.15.89:43064) 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 -- 172.21.15.89:0/157044577 learned_addr learned my addr 172.21.15.89:0/157044577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd8e9e700 1 --2- 172.21.15.89:0/157044577 >> v2:172.21.15.89:3300/0 conn(0x7fbfd40a0d90 0x7fbfd40a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd9ea0700 1 --2- 172.21.15.89:0/157044577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd41338a0 0x7fbfd4133d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 -- 172.21.15.89:0/157044577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd41338a0 msgr2=0x7fbfd4133d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 --2- 172.21.15.89:0/157044577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd41338a0 0x7fbfd4133d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 -- 172.21.15.89:0/157044577 >> v2:172.21.15.89:3300/0 conn(0x7fbfd40a0d90 msgr2=0x7fbfd40a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 --2- 172.21.15.89:0/157044577 >> v2:172.21.15.89:3300/0 conn(0x7fbfd40a0d90 0x7fbfd40a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 -- 172.21.15.89:0/157044577 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40a1f40 con 0x7fbfd40a09c0 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfd969f700 1 --2- 172.21.15.89:0/157044577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a09c0 0x7fbfd406c5d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00098f0 tx=0x7fbfd0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6dd183dff8f032 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfcbfff700 1 -- 172.21.15.89:0/157044577 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0011830 con 0x7fbfd40a09c0 2023-09-30T22:18:11.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfcbfff700 1 -- 172.21.15.89:0/157044577 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0011990 con 0x7fbfd40a09c0 2023-09-30T22:18:11.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.248+0000 7fbfcbfff700 1 -- 172.21.15.89:0/157044577 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0011d00 con 0x7fbfd40a09c0 2023-09-30T22:18:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 -- 172.21.15.89:0/157044577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a09c0 msgr2=0x7fbfd406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 --2- 172.21.15.89:0/157044577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a09c0 0x7fbfd406c5d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00098f0 tx=0x7fbfd0009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 -- 172.21.15.89:0/157044577 shutdown_connections 2023-09-30T22:18:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 --2- 172.21.15.89:0/157044577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd41338a0 0x7fbfd4133d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 --2- 172.21.15.89:0/157044577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd40a09c0 0x7fbfd406c5d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 --2- 172.21.15.89:0/157044577 >> v2:172.21.15.89:3300/0 conn(0x7fbfd40a0d90 0x7fbfd40a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 -- 172.21.15.89:0/157044577 >> 172.21.15.89:0/157044577 conn(0x7fbfd40a60c0 msgr2=0x7fbfd40a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 -- 172.21.15.89:0/157044577 shutdown_connections 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.249+0000 7fbfdb903700 1 -- 172.21.15.89:0/157044577 wait complete. 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.250+0000 7fbfdb903700 1 Processor -- start 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.250+0000 7fbfdb903700 1 -- start start 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.250+0000 7fbfdb903700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd40a09c0 0x7fbfd41c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.250+0000 7fbfdb903700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0d90 0x7fbfd41c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.250+0000 7fbfdb903700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd41338a0 0x7fbfd41c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.250+0000 7fbfdb903700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4121ee0 con 0x7fbfd40a0d90 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.250+0000 7fbfdb903700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4120570 con 0x7fbfd40a09c0 2023-09-30T22:18:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.250+0000 7fbfdb903700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4122560 con 0x7fbfd41338a0 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0d90 0x7fbfd41c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0d90 0x7fbfd41c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42704/0 (socket says 172.21.15.89:42704) 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 -- 172.21.15.89:0/3108401811 learned_addr learned my addr 172.21.15.89:0/3108401811 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd9ea0700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd41338a0 0x7fbfd41c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 -- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd41338a0 msgr2=0x7fbfd41c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd41338a0 0x7fbfd41c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 -- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd40a09c0 msgr2=0x7fbfd41c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd40a09c0 0x7fbfd41c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 -- 172.21.15.89:0/3108401811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd0009550 con 0x7fbfd40a0d90 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd9ea0700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd41338a0 0x7fbfd41c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:11.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfd8e9e700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0d90 0x7fbfd41c86b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400e980 tx=0x7fbfc400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:11.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfc9ffb700 1 -- 172.21.15.89:0/3108401811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4009710 con 0x7fbfd40a0d90 2023-09-30T22:18:11.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfc9ffb700 1 -- 172.21.15.89:0/3108401811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc400eed0 con 0x7fbfd40a0d90 2023-09-30T22:18:11.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfc9ffb700 1 -- 172.21.15.89:0/3108401811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400ca70 con 0x7fbfd40a0d90 2023-09-30T22:18:11.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.251+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40a68b0 con 0x7fbfd40a0d90 2023-09-30T22:18:11.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.252+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4133410 con 0x7fbfd40a0d90 2023-09-30T22:18:11.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.253+0000 7fbfc9ffb700 1 -- 172.21.15.89:0/3108401811 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbfc400cbd0 con 0x7fbfd40a0d90 2023-09-30T22:18:11.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.253+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd4122100 con 0x7fbfd40a0d90 2023-09-30T22:18:11.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.253+0000 7fbfc9ffb700 1 -- 172.21.15.89:0/3108401811 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbfc403fbc0 con 0x7fbfd40a0d90 2023-09-30T22:18:11.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.258+0000 7fbfc9ffb700 1 -- 172.21.15.89:0/3108401811 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbfc401b3b0 con 0x7fbfd40a0d90 2023-09-30T22:18:11.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.263+0000 7f8a43142700 1 Processor -- start 2023-09-30T22:18:11.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.263+0000 7f8a43142700 1 -- start start 2023-09-30T22:18:11.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a43142700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a3c133420 0x7f8a3c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a43142700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133e30 0x7f8a3c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a43142700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c06b990 con 0x7f8a3c134260 2023-09-30T22:18:11.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a43142700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3c0a0590 con 0x7f8a3c133420 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a43142700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c09fb80 con 0x7f8a3c133e30 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a3bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133e30 0x7f8a3c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a40ede700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a3c133420 0x7f8a3c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a3bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133e30 0x7f8a3c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36122/0 (socket says 172.21.15.89:36122) 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a3bfff700 1 -- 172.21.15.89:0/4026460483 learned_addr learned my addr 172.21.15.89:0/4026460483 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a3bfff700 1 -- 172.21.15.89:0/4026460483 >> v2:172.21.15.89:3300/0 conn(0x7f8a3c133420 msgr2=0x7f8a3c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a3bfff700 1 --2- 172.21.15.89:0/4026460483 >> v2:172.21.15.89:3300/0 conn(0x7f8a3c133420 0x7f8a3c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a3bfff700 1 -- 172.21.15.89:0/4026460483 >> v1:172.21.15.89:6789/0 conn(0x7f8a3c134260 legacy=0x7f8a3c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.264+0000 7f8a3bfff700 1 -- 172.21.15.89:0/4026460483 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0a0030 con 0x7f8a3c133e30 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.265+0000 7f8a3bfff700 1 --2- 172.21.15.89:0/4026460483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133e30 0x7f8a3c12ff90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a280098f0 tx=0x7f8a28009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87082dd637d417e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.265+0000 7f8a3b7fe700 1 -- 172.21.15.89:0/4026460483 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28011830 con 0x7f8a3c133e30 2023-09-30T22:18:11.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.265+0000 7f8a3b7fe700 1 -- 172.21.15.89:0/4026460483 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a28011990 con 0x7f8a3c133e30 2023-09-30T22:18:11.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.265+0000 7f8a3b7fe700 1 -- 172.21.15.89:0/4026460483 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28011d00 con 0x7f8a3c133e30 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.266+0000 7f8a43142700 1 -- 172.21.15.89:0/4026460483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133e30 msgr2=0x7f8a3c12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.266+0000 7f8a43142700 1 --2- 172.21.15.89:0/4026460483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133e30 0x7f8a3c12ff90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a280098f0 tx=0x7f8a28009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.266+0000 7f8a43142700 1 -- 172.21.15.89:0/4026460483 shutdown_connections 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.266+0000 7f8a43142700 1 --2- 172.21.15.89:0/4026460483 >> v2:172.21.15.89:3300/0 conn(0x7f8a3c133420 0x7f8a3c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.266+0000 7f8a43142700 1 --2- 172.21.15.89:0/4026460483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133e30 0x7f8a3c12ff90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.266+0000 7f8a43142700 1 -- 172.21.15.89:0/4026460483 >> 172.21.15.89:0/4026460483 conn(0x7f8a3c0a4210 msgr2=0x7f8a3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.266+0000 7f8a43142700 1 -- 172.21.15.89:0/4026460483 shutdown_connections 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.266+0000 7f8a43142700 1 -- 172.21.15.89:0/4026460483 wait complete. 2023-09-30T22:18:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.267+0000 7f8a43142700 1 Processor -- start 2023-09-30T22:18:11.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.267+0000 7f8a43142700 1 -- start start 2023-09-30T22:18:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a43142700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133420 0x7f8a3c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a43142700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a3c133e30 0x7f8a3c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a43142700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a3c134260 0x7f8a3c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a43142700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c09e070 con 0x7f8a3c133e30 2023-09-30T22:18:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a43142700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c11ee60 con 0x7f8a3c133420 2023-09-30T22:18:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a43142700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c12d670 con 0x7f8a3c134260 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a3bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a3c133e30 0x7f8a3c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a3bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a3c133e30 0x7f8a3c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42716/0 (socket says 172.21.15.89:42716) 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a3bfff700 1 -- 172.21.15.89:0/2638622340 learned_addr learned my addr 172.21.15.89:0/2638622340 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a40ede700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133420 0x7f8a3c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.268+0000 7f8a416df700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a3c134260 0x7f8a3c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a3bfff700 1 -- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a3c134260 msgr2=0x7f8a3c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a3bfff700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a3c134260 0x7f8a3c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a3bfff700 1 -- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133420 msgr2=0x7f8a3c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a3bfff700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133420 0x7f8a3c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a3bfff700 1 -- 172.21.15.89:0/2638622340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a28009550 con 0x7f8a3c133e30 2023-09-30T22:18:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a416df700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a3c134260 0x7f8a3c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a3bfff700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a3c133e30 0x7f8a3c09c9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c09fcc0 tx=0x7f8a28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a397fa700 1 -- 172.21.15.89:0/2638622340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28004450 con 0x7f8a3c133e30 2023-09-30T22:18:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a397fa700 1 -- 172.21.15.89:0/2638622340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a280045d0 con 0x7f8a3c133e30 2023-09-30T22:18:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c11fd30 con 0x7f8a3c133e30 2023-09-30T22:18:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c1202d0 con 0x7f8a3c133e30 2023-09-30T22:18:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.269+0000 7f8a397fa700 1 -- 172.21.15.89:0/2638622340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a280038e0 con 0x7f8a3c133e30 2023-09-30T22:18:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.270+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c0052e0 con 0x7f8a3c133e30 2023-09-30T22:18:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.271+0000 7f8a397fa700 1 -- 172.21.15.89:0/2638622340 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a28003a40 con 0x7f8a3c133e30 2023-09-30T22:18:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.273+0000 7f8a397fa700 1 -- 172.21.15.89:0/2638622340 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8a28003cf0 con 0x7f8a3c133e30 2023-09-30T22:18:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.280+0000 7f8a397fa700 1 -- 172.21.15.89:0/2638622340 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a2802d440 con 0x7f8a3c133e30 2023-09-30T22:18:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.376+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3550005590 con 0x7f356809d6f0 2023-09-30T22:18:11.378 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.377+0000 7f355effd700 1 -- 172.21.15.89:0/2690384148 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3564012310 con 0x7f356809d6f0 2023-09-30T22:18:11.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809d6f0 msgr2=0x7f3568122390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809d6f0 0x7f3568122390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35640086b0 tx=0x7f3564008a70 comp rx=0 tx=0).stop 2023-09-30T22:18:11.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 shutdown_connections 2023-09-30T22:18:11.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f356809e410 0x7f3568122e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f356809d6f0 0x7f3568122390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 --2- 172.21.15.89:0/2690384148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f356809e000 0x7f35681228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 >> 172.21.15.89:0/2690384148 conn(0x7f3568066f50 msgr2=0x7f35680a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:11.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 shutdown_connections 2023-09-30T22:18:11.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.379+0000 7f35706d0700 1 -- 172.21.15.89:0/2690384148 wait complete. 2023-09-30T22:18:11.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.423+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f46b4072980 con 0x7f46b409d630 2023-09-30T22:18:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.424+0000 7f469bfff700 1 -- 172.21.15.89:0/3689534345 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46a40233d0 con 0x7f46b409d630 2023-09-30T22:18:11.424 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:11.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b409d630 msgr2=0x7f46b41c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b409d630 0x7f46b41c8090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46a4016000 tx=0x7f46a4004750 comp rx=0 tx=0).stop 2023-09-30T22:18:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 shutdown_connections 2023-09-30T22:18:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b4133930 0x7f46b41c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b409d630 0x7f46b41c8090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 --2- 172.21.15.89:0/3689534345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40a0a30 0x7f46b41c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 >> 172.21.15.89:0/3689534345 conn(0x7f46b40a3050 msgr2=0x7f46b41bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 shutdown_connections 2023-09-30T22:18:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.426+0000 7f46b9c33700 1 -- 172.21.15.89:0/3689534345 wait complete. 2023-09-30T22:18:11.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:11.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.456+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbfd404f2a0 con 0x7fbfd40a0d90 2023-09-30T22:18:11.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.456+0000 7fbfc9ffb700 1 -- 172.21.15.89:0/3108401811 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbfc4017020 con 0x7fbfd40a0d90 2023-09-30T22:18:11.457 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:11.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0d90 msgr2=0x7fbfd41c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0d90 0x7fbfd41c86b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400e980 tx=0x7fbfc400ec90 comp rx=0 tx=0).stop 2023-09-30T22:18:11.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 shutdown_connections 2023-09-30T22:18:11.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfd41338a0 0x7fbfd41c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfd40a09c0 0x7fbfd41c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 --2- 172.21.15.89:0/3108401811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd40a0d90 0x7fbfd41c86b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 >> 172.21.15.89:0/3108401811 conn(0x7fbfd40a60c0 msgr2=0x7fbfd41bbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:11.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 shutdown_connections 2023-09-30T22:18:11.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.458+0000 7fbfdb903700 1 -- 172.21.15.89:0/3108401811 wait complete. 2023-09-30T22:18:11.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.477+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8a1c005860 con 0x7f8a3c133e30 2023-09-30T22:18:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.477+0000 7f8a397fa700 1 -- 172.21.15.89:0/2638622340 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a28015070 con 0x7f8a3c133e30 2023-09-30T22:18:11.478 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a3c133e30 msgr2=0x7f8a3c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a3c133e30 0x7f8a3c09c9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c09fcc0 tx=0x7f8a28004750 comp rx=0 tx=0).stop 2023-09-30T22:18:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 shutdown_connections 2023-09-30T22:18:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a3c134260 0x7f8a3c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a3c133420 0x7f8a3c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 --2- 172.21.15.89:0/2638622340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a3c133e30 0x7f8a3c09c9c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:11.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 >> 172.21.15.89:0/2638622340 conn(0x7f8a3c0a4210 msgr2=0x7f8a3c0a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:11.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 shutdown_connections 2023-09-30T22:18:11.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:11.479+0000 7f8a43142700 1 -- 172.21.15.89:0/2638622340 wait complete. 2023-09-30T22:18:11.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:12.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:12.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:12.472 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:12.492 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.523+0000 7fb60b9d7700 1 Processor -- start 2023-09-30T22:18:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb60b9d7700 1 -- start start 2023-09-30T22:18:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb60b9d7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409d6f0 0x7fb60409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb60b9d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb60409e000 0x7fb6041339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb60b9d7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60409b120 con 0x7fb60409e410 2023-09-30T22:18:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb60b9d7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60406b460 con 0x7fb60409e000 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb60b9d7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60409a7e0 con 0x7fb60409d6f0 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb609773700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409d6f0 0x7fb60409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb608f72700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb60409e000 0x7fb6041339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb608f72700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb60409e000 0x7fb6041339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42728/0 (socket says 172.21.15.89:42728) 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb609773700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409d6f0 0x7fb60409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36146/0 (socket says 172.21.15.89:36146) 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.524+0000 7fb608f72700 1 -- 172.21.15.89:0/2013834959 learned_addr learned my addr 172.21.15.89:0/2013834959 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.525+0000 7fb608f72700 1 -- 172.21.15.89:0/2013834959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409d6f0 msgr2=0x7fb60409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.525+0000 7fb608f72700 1 --2- 172.21.15.89:0/2013834959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409d6f0 0x7fb60409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.525+0000 7fb608f72700 1 -- 172.21.15.89:0/2013834959 >> v1:172.21.15.89:6789/0 conn(0x7fb60409e410 legacy=0x7fb604133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.525+0000 7fb608f72700 1 -- 172.21.15.89:0/2013834959 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60409a500 con 0x7fb60409e000 2023-09-30T22:18:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.525+0000 7fb608f72700 1 --2- 172.21.15.89:0/2013834959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e000 0x7fb6041339f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098f0 tx=0x7fb5f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e3e5c7a0772dcc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.525+0000 7fb5fbfff700 1 -- 172.21.15.89:0/2013834959 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4011830 con 0x7fb60409e000 2023-09-30T22:18:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.525+0000 7fb5fbfff700 1 -- 172.21.15.89:0/2013834959 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4011990 con 0x7fb60409e000 2023-09-30T22:18:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.525+0000 7fb5fbfff700 1 -- 172.21.15.89:0/2013834959 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4011d20 con 0x7fb60409e000 2023-09-30T22:18:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.526+0000 7fb60b9d7700 1 -- 172.21.15.89:0/2013834959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e000 msgr2=0x7fb6041339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.526+0000 7fb60b9d7700 1 --2- 172.21.15.89:0/2013834959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e000 0x7fb6041339f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098f0 tx=0x7fb5f4009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.526+0000 7fb60b9d7700 1 -- 172.21.15.89:0/2013834959 shutdown_connections 2023-09-30T22:18:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.526+0000 7fb60b9d7700 1 --2- 172.21.15.89:0/2013834959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409d6f0 0x7fb60409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.526+0000 7fb60b9d7700 1 --2- 172.21.15.89:0/2013834959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e000 0x7fb6041339f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.526+0000 7fb60b9d7700 1 -- 172.21.15.89:0/2013834959 >> 172.21.15.89:0/2013834959 conn(0x7fb604066f50 msgr2=0x7fb60409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.526+0000 7fb60b9d7700 1 -- 172.21.15.89:0/2013834959 shutdown_connections 2023-09-30T22:18:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.526+0000 7fb60b9d7700 1 -- 172.21.15.89:0/2013834959 wait complete. 2023-09-30T22:18:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.527+0000 7fb60b9d7700 1 Processor -- start 2023-09-30T22:18:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.527+0000 7fb60b9d7700 1 -- start start 2023-09-30T22:18:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.527+0000 7fb60b9d7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb60409d6f0 0x7fb6041a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.527+0000 7fb60b9d7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409e000 0x7fb6041a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.527+0000 7fb60b9d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e410 0x7fb6041a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.527+0000 7fb60b9d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6041a7b30 con 0x7fb60409e410 2023-09-30T22:18:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.527+0000 7fb60b9d7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6041a8150 con 0x7fb60409e000 2023-09-30T22:18:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.527+0000 7fb60b9d7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6041a8d80 con 0x7fb60409d6f0 2023-09-30T22:18:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409e000 0x7fb6041a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb609f74700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e410 0x7fb6041a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb609773700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb60409d6f0 0x7fb6041a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409e000 0x7fb6041a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36162/0 (socket says 172.21.15.89:36162) 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 -- 172.21.15.89:0/1618299174 learned_addr learned my addr 172.21.15.89:0/1618299174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 -- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb60409d6f0 msgr2=0x7fb6041a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb60409d6f0 0x7fb6041a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 -- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e410 msgr2=0x7fb6041a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e410 0x7fb6041a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 -- 172.21.15.89:0/1618299174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4009550 con 0x7fb60409e000 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb609773700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb60409d6f0 0x7fb6041a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb609f74700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e410 0x7fb6041a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb608f72700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409e000 0x7fb6041a64f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4013000 tx=0x7fb5f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb5f9ffb700 1 -- 172.21.15.89:0/1618299174 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4004300 con 0x7fb60409e000 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb5f9ffb700 1 -- 172.21.15.89:0/1618299174 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f401f070 con 0x7fb60409e000 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6041a91d0 con 0x7fb60409e000 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.528+0000 7fb5f9ffb700 1 -- 172.21.15.89:0/1618299174 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40036a0 con 0x7fb60409e000 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.529+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6041a7400 con 0x7fb60409e000 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.530+0000 7fb5f9ffb700 1 -- 172.21.15.89:0/1618299174 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb5f4021030 con 0x7fb60409e000 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.530+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb604074ea0 con 0x7fb60409e000 2023-09-30T22:18:12.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.530+0000 7fb5f9ffb700 1 -- 172.21.15.89:0/1618299174 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb5f4040ac0 con 0x7fb60409e000 2023-09-30T22:18:12.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.539+0000 7fb5f9ffb700 1 -- 172.21.15.89:0/1618299174 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5f4015020 con 0x7fb60409e000 2023-09-30T22:18:12.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.575+0000 7f4d2ca53700 1 Processor -- start 2023-09-30T22:18:12.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.575+0000 7f4d2ca53700 1 -- start start 2023-09-30T22:18:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d2ca53700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809f410 0x7f4d28136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d2ca53700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d28136fe0 0x7f4d281393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d2ca53700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d28070940 con 0x7f4d2809e930 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d2ca53700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2806fd60 con 0x7f4d28136fe0 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d2ca53700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2806fa80 con 0x7f4d2809f410 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d25d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809f410 0x7f4d28136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d25d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809f410 0x7f4d28136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43124/0 (socket says 172.21.15.89:43124) 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d25d9b700 1 -- 172.21.15.89:0/3766667875 learned_addr learned my addr 172.21.15.89:0/3766667875 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.576+0000 7f4d26d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d28136fe0 0x7f4d281393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.577+0000 7f4d25d9b700 1 -- 172.21.15.89:0/3766667875 >> v2:172.21.15.89:3300/0 conn(0x7f4d28136fe0 msgr2=0x7f4d281393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.577+0000 7f4d25d9b700 1 --2- 172.21.15.89:0/3766667875 >> v2:172.21.15.89:3300/0 conn(0x7f4d28136fe0 0x7f4d281393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.577+0000 7f4d25d9b700 1 -- 172.21.15.89:0/3766667875 >> v1:172.21.15.89:6789/0 conn(0x7f4d2809e930 legacy=0x7f4d2809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.577+0000 7f4d25d9b700 1 -- 172.21.15.89:0/3766667875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2809fb60 con 0x7f4d2809f410 2023-09-30T22:18:12.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.577+0000 7f4d25d9b700 1 --2- 172.21.15.89:0/3766667875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809f410 0x7f4d28136aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0098f0 tx=0x7f4d1c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968d8600f07da543 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.577+0000 7f4d2559a700 1 -- 172.21.15.89:0/3766667875 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c004230 con 0x7f4d2809f410 2023-09-30T22:18:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.577+0000 7f4d2559a700 1 -- 172.21.15.89:0/3766667875 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c004390 con 0x7f4d2809f410 2023-09-30T22:18:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.577+0000 7f4d2559a700 1 -- 172.21.15.89:0/3766667875 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0036a0 con 0x7f4d2809f410 2023-09-30T22:18:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.578+0000 7f4d2ca53700 1 -- 172.21.15.89:0/3766667875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809f410 msgr2=0x7f4d28136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.578+0000 7f4d2ca53700 1 --2- 172.21.15.89:0/3766667875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809f410 0x7f4d28136aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0098f0 tx=0x7f4d1c009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.578+0000 7f4d2ca53700 1 -- 172.21.15.89:0/3766667875 shutdown_connections 2023-09-30T22:18:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.578+0000 7f4d2ca53700 1 --2- 172.21.15.89:0/3766667875 >> v2:172.21.15.89:3300/0 conn(0x7f4d28136fe0 0x7f4d281393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.578+0000 7f4d2ca53700 1 --2- 172.21.15.89:0/3766667875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809f410 0x7f4d28136aa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.578+0000 7f4d2ca53700 1 -- 172.21.15.89:0/3766667875 >> 172.21.15.89:0/3766667875 conn(0x7f4d280a4210 msgr2=0x7f4d280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.578+0000 7f4d2ca53700 1 -- 172.21.15.89:0/3766667875 shutdown_connections 2023-09-30T22:18:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.578+0000 7f4d2ca53700 1 -- 172.21.15.89:0/3766667875 wait complete. 2023-09-30T22:18:12.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.579+0000 7f4d2ca53700 1 Processor -- start 2023-09-30T22:18:12.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.579+0000 7f4d2ca53700 1 -- start start 2023-09-30T22:18:12.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.579+0000 7f4d2ca53700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809e930 0x7f4d281c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.579+0000 7f4d2ca53700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d2809f410 0x7f4d281c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.579+0000 7f4d2ca53700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28136fe0 0x7f4d281c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.579+0000 7f4d2ca53700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28131600 con 0x7f4d28136fe0 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.579+0000 7f4d2ca53700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2806d6b0 con 0x7f4d2809f410 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.579+0000 7f4d2ca53700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d281214f0 con 0x7f4d2809e930 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d25d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d2809f410 0x7f4d281c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809e930 0x7f4d281c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d25d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d2809f410 0x7f4d281c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36164/0 (socket says 172.21.15.89:36164) 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d25d9b700 1 -- 172.21.15.89:0/2676223256 learned_addr learned my addr 172.21.15.89:0/2676223256 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809e930 0x7f4d281c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43134/0 (socket says 172.21.15.89:43134) 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d26d9d700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28136fe0 0x7f4d281c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2659c700 1 -- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d2809f410 msgr2=0x7f4d281c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2659c700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d2809f410 0x7f4d281c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2659c700 1 -- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28136fe0 msgr2=0x7f4d281c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2659c700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28136fe0 0x7f4d281c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2659c700 1 -- 172.21.15.89:0/2676223256 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c009550 con 0x7f4d2809e930 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d26d9d700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28136fe0 0x7f4d281c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2659c700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809e930 0x7f4d281c8400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800c400 tx=0x7f4d1800c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d16ffd700 1 -- 172.21.15.89:0/2676223256 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18011790 con 0x7f4d2809e930 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d16ffd700 1 -- 172.21.15.89:0/2676223256 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d180118f0 con 0x7f4d2809e930 2023-09-30T22:18:12.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d280a2f40 con 0x7f4d2809e930 2023-09-30T22:18:12.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.580+0000 7f4d16ffd700 1 -- 172.21.15.89:0/2676223256 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18011ca0 con 0x7f4d2809e930 2023-09-30T22:18:12.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.581+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2811fdd0 con 0x7f4d2809e930 2023-09-30T22:18:12.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.582+0000 7f4d16ffd700 1 -- 172.21.15.89:0/2676223256 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d1801c460 con 0x7f4d2809e930 2023-09-30T22:18:12.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.582+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2809e4b0 con 0x7f4d2809e930 2023-09-30T22:18:12.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.582+0000 7f4d16ffd700 1 -- 172.21.15.89:0/2676223256 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4d1801a360 con 0x7f4d2809e930 2023-09-30T22:18:12.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.590+0000 7f4d16ffd700 1 -- 172.21.15.89:0/2676223256 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d18044060 con 0x7f4d2809e930 2023-09-30T22:18:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.625+0000 7ff0de9ca700 1 Processor -- start 2023-09-30T22:18:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.626+0000 7ff0de9ca700 1 -- start start 2023-09-30T22:18:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.626+0000 7ff0de9ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0d8134630 0x7ff0d8136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.626+0000 7ff0de9ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8136f50 0x7ff0d8139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.626+0000 7ff0de9ca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d809b4f0 con 0x7ff0d8131810 2023-09-30T22:18:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.626+0000 7ff0de9ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d81313c0 con 0x7ff0d8134630 2023-09-30T22:18:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.626+0000 7ff0de9ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d806e8c0 con 0x7ff0d8136f50 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.626+0000 7ff0d77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0d8134630 0x7ff0d8136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.626+0000 7ff0dcf67700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8136f50 0x7ff0d8139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0dcf67700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8136f50 0x7ff0d8139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36180/0 (socket says 172.21.15.89:36180) 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0d77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0d8134630 0x7ff0d8136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42754/0 (socket says 172.21.15.89:42754) 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0dcf67700 1 -- 172.21.15.89:0/1645606923 learned_addr learned my addr 172.21.15.89:0/1645606923 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0d6ffd700 1 -- 172.21.15.89:0/1645606923 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663207695 0 0) 0x7ff0d809b4f0 con 0x7ff0d8131810 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0d6ffd700 1 -- 172.21.15.89:0/1645606923 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0c00035b0 con 0x7ff0d8131810 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0dcf67700 1 -- 172.21.15.89:0/1645606923 >> v2:172.21.15.89:3300/0 conn(0x7ff0d8134630 msgr2=0x7ff0d8136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0dcf67700 1 --2- 172.21.15.89:0/1645606923 >> v2:172.21.15.89:3300/0 conn(0x7ff0d8134630 0x7ff0d8136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0dcf67700 1 -- 172.21.15.89:0/1645606923 >> v1:172.21.15.89:6789/0 conn(0x7ff0d8131810 legacy=0x7ff0d8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0dcf67700 1 -- 172.21.15.89:0/1645606923 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d8071d20 con 0x7ff0d8136f50 2023-09-30T22:18:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0dcf67700 1 --2- 172.21.15.89:0/1645606923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8136f50 0x7ff0d8139330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009e50 tx=0x7ff0d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f61700f0ea7f45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0d6ffd700 1 -- 172.21.15.89:0/1645606923 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d000e070 con 0x7ff0d8136f50 2023-09-30T22:18:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0d6ffd700 1 -- 172.21.15.89:0/1645606923 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d0003a20 con 0x7ff0d8136f50 2023-09-30T22:18:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.627+0000 7ff0d6ffd700 1 -- 172.21.15.89:0/1645606923 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0003d90 con 0x7ff0d8136f50 2023-09-30T22:18:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.628+0000 7ff0de9ca700 1 -- 172.21.15.89:0/1645606923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8136f50 msgr2=0x7ff0d8139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.628+0000 7ff0de9ca700 1 --2- 172.21.15.89:0/1645606923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8136f50 0x7ff0d8139330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009e50 tx=0x7ff0d000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.628+0000 7ff0de9ca700 1 -- 172.21.15.89:0/1645606923 shutdown_connections 2023-09-30T22:18:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.628+0000 7ff0de9ca700 1 --2- 172.21.15.89:0/1645606923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8136f50 0x7ff0d8139330 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.628+0000 7ff0de9ca700 1 --2- 172.21.15.89:0/1645606923 >> v2:172.21.15.89:3300/0 conn(0x7ff0d8134630 0x7ff0d8136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.628+0000 7ff0de9ca700 1 -- 172.21.15.89:0/1645606923 >> 172.21.15.89:0/1645606923 conn(0x7ff0d80a2aa0 msgr2=0x7ff0d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.628+0000 7ff0de9ca700 1 -- 172.21.15.89:0/1645606923 shutdown_connections 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.628+0000 7ff0de9ca700 1 -- 172.21.15.89:0/1645606923 wait complete. 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.630+0000 7ff0de9ca700 1 Processor -- start 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.630+0000 7ff0de9ca700 1 -- start start 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.630+0000 7ff0de9ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0d8131810 0x7ff0d806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.630+0000 7ff0de9ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8134630 0x7ff0d806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.630+0000 7ff0de9ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d8136f50 0x7ff0d8070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0de9ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d8070ec0 con 0x7ff0d8136f50 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0de9ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d811e400 con 0x7ff0d8134630 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0de9ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d812e7a0 con 0x7ff0d8131810 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0dcf67700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d8136f50 0x7ff0d8070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0dcf67700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d8136f50 0x7ff0d8070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42770/0 (socket says 172.21.15.89:42770) 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0dcf67700 1 -- 172.21.15.89:0/3658751677 learned_addr learned my addr 172.21.15.89:0/3658751677 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0d77fe700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8134630 0x7ff0d806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0d77fe700 1 -- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0d8131810 msgr2=0x7ff0d806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0d77fe700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0d8131810 0x7ff0d806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0d77fe700 1 -- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d8136f50 msgr2=0x7ff0d8070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0d77fe700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d8136f50 0x7ff0d8070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0d77fe700 1 -- 172.21.15.89:0/3658751677 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0009b30 con 0x7ff0d8134630 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0dcf67700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d8136f50 0x7ff0d8070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.631+0000 7ff0d77fe700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8134630 0x7ff0d806fb20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e8d0 tx=0x7ff0cc00ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:12.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.632+0000 7ff0d4ff9700 1 -- 172.21.15.89:0/3658751677 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00c690 con 0x7ff0d8134630 2023-09-30T22:18:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.632+0000 7ff0d4ff9700 1 -- 172.21.15.89:0/3658751677 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc00c7f0 con 0x7ff0d8134630 2023-09-30T22:18:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.632+0000 7ff0d4ff9700 1 -- 172.21.15.89:0/3658751677 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00cb60 con 0x7ff0d8134630 2023-09-30T22:18:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.632+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d8132f50 con 0x7ff0d8134630 2023-09-30T22:18:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.633+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d811ee70 con 0x7ff0d8134630 2023-09-30T22:18:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.634+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d8074ea0 con 0x7ff0d8134630 2023-09-30T22:18:12.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.634+0000 7ff0d4ff9700 1 -- 172.21.15.89:0/3658751677 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0cc00ccc0 con 0x7ff0d8134630 2023-09-30T22:18:12.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.634+0000 7ff0d4ff9700 1 -- 172.21.15.89:0/3658751677 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff0cc03f6e0 con 0x7ff0d8134630 2023-09-30T22:18:12.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.643+0000 7ff0d4ff9700 1 -- 172.21.15.89:0/3658751677 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0cc0232f0 con 0x7ff0d8134630 2023-09-30T22:18:12.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f6995fbe700 1 Processor -- start 2023-09-30T22:18:12.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f6995fbe700 1 -- start start 2023-09-30T22:18:12.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f6995fbe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f699009eca0 0x7f699013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f6995fbe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006f050 0x7f699013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f6995fbe700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6990070ee0 con 0x7f699006e570 2023-09-30T22:18:12.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f6995fbe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6990071980 con 0x7f699006f050 2023-09-30T22:18:12.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f6995fbe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6990071480 con 0x7f699009eca0 2023-09-30T22:18:12.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f698ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006f050 0x7f699013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f698ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006f050 0x7f699013f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36188/0 (socket says 172.21.15.89:36188) 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.666+0000 7f698ffff700 1 -- 172.21.15.89:0/334266257 learned_addr learned my addr 172.21.15.89:0/334266257 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698effd700 1 --2- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f699009eca0 0x7f699013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698ffff700 1 -- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f699009eca0 msgr2=0x7f699013cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698ffff700 1 --2- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f699009eca0 0x7f699013cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698ffff700 1 -- 172.21.15.89:0/334266257 >> v1:172.21.15.89:6789/0 conn(0x7f699006e570 legacy=0x7f699006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698ffff700 1 -- 172.21.15.89:0/334266257 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6990071b90 con 0x7f699006f050 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698effd700 1 --2- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f699009eca0 0x7f699013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698ffff700 1 --2- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006f050 0x7f699013f3f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6988009e80 tx=0x7f698800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18fabf2d887d091 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698e7fc700 1 -- 172.21.15.89:0/334266257 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698800e070 con 0x7f699006f050 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698e7fc700 1 -- 172.21.15.89:0/334266257 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698800b910 con 0x7f699006f050 2023-09-30T22:18:12.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.667+0000 7f698e7fc700 1 -- 172.21.15.89:0/334266257 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698800bcc0 con 0x7f699006f050 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.668+0000 7f6995fbe700 1 -- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006f050 msgr2=0x7f699013f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.668+0000 7f6995fbe700 1 --2- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006f050 0x7f699013f3f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6988009e80 tx=0x7f698800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.668+0000 7f6995fbe700 1 -- 172.21.15.89:0/334266257 shutdown_connections 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.668+0000 7f6995fbe700 1 --2- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006f050 0x7f699013f3f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.668+0000 7f6995fbe700 1 --2- 172.21.15.89:0/334266257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f699009eca0 0x7f699013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.668+0000 7f6995fbe700 1 -- 172.21.15.89:0/334266257 >> 172.21.15.89:0/334266257 conn(0x7f69900a2aa0 msgr2=0x7f69900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.668+0000 7f6995fbe700 1 -- 172.21.15.89:0/334266257 shutdown_connections 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.668+0000 7f6995fbe700 1 -- 172.21.15.89:0/334266257 wait complete. 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.669+0000 7f6995fbe700 1 Processor -- start 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.669+0000 7f6995fbe700 1 -- start start 2023-09-30T22:18:12.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f6995fbe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006e570 0x7f6990123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f6995fbe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699006f050 0x7f69901237a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f6995fbe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6990123ce0 0x7f69901cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f6995fbe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69901267a0 con 0x7f699006f050 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f6995fbe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69901252f0 con 0x7f699006e570 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f6995fbe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f699009a2d0 con 0x7f6990123ce0 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006e570 0x7f6990123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006e570 0x7f6990123260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36192/0 (socket says 172.21.15.89:36192) 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698f7fe700 1 -- 172.21.15.89:0/565560359 learned_addr learned my addr 172.21.15.89:0/565560359 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698effd700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699006f050 0x7f69901237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698ffff700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6990123ce0 0x7f69901cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698ffff700 1 -- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006e570 msgr2=0x7f6990123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698ffff700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006e570 0x7f6990123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698ffff700 1 -- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699006f050 msgr2=0x7f69901237a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698ffff700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699006f050 0x7f69901237a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698ffff700 1 -- 172.21.15.89:0/565560359 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840094a0 con 0x7f6990123ce0 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.670+0000 7f698ffff700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6990123ce0 0x7f69901cca40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6990070170 tx=0x7f6988004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.671+0000 7f6977fff700 1 -- 172.21.15.89:0/565560359 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69880092e0 con 0x7f6990123ce0 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.671+0000 7f6977fff700 1 -- 172.21.15.89:0/565560359 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69880099b0 con 0x7f6990123ce0 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.671+0000 7f6977fff700 1 -- 172.21.15.89:0/565560359 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988004490 con 0x7f6990123ce0 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.671+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6988009b30 con 0x7f6990123ce0 2023-09-30T22:18:12.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.671+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69901263f0 con 0x7f6990123ce0 2023-09-30T22:18:12.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.672+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69901265e0 con 0x7f6990123ce0 2023-09-30T22:18:12.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.672+0000 7f6977fff700 1 -- 172.21.15.89:0/565560359 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f69880045f0 con 0x7f6990123ce0 2023-09-30T22:18:12.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.673+0000 7f6977fff700 1 -- 172.21.15.89:0/565560359 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f69880409f0 con 0x7f6990123ce0 2023-09-30T22:18:12.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.681+0000 7f6977fff700 1 -- 172.21.15.89:0/565560359 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6988042e60 con 0x7f6990123ce0 2023-09-30T22:18:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.741+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb60406c360 con 0x7fb60409e000 2023-09-30T22:18:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.742+0000 7fb5f9ffb700 1 -- 172.21.15.89:0/1618299174 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5f4019770 con 0x7fb60409e000 2023-09-30T22:18:12.742 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409e000 msgr2=0x7fb6041a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409e000 0x7fb6041a64f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4013000 tx=0x7fb5f4004750 comp rx=0 tx=0).stop 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 shutdown_connections 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60409e410 0x7fb6041a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb60409d6f0 0x7fb6041a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 --2- 172.21.15.89:0/1618299174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb60409e000 0x7fb6041a64f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 >> 172.21.15.89:0/1618299174 conn(0x7fb604066f50 msgr2=0x7fb6041baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 shutdown_connections 2023-09-30T22:18:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.744+0000 7fb60b9d7700 1 -- 172.21.15.89:0/1618299174 wait complete. 2023-09-30T22:18:12.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.801+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d28074ea0 con 0x7f4d2809e930 2023-09-30T22:18:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.802+0000 7f4d16ffd700 1 -- 172.21.15.89:0/2676223256 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d18011e00 con 0x7f4d2809e930 2023-09-30T22:18:12.803 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809e930 msgr2=0x7f4d281c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809e930 0x7f4d281c8400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800c400 tx=0x7f4d1800c710 comp rx=0 tx=0).stop 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 shutdown_connections 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28136fe0 0x7f4d281c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d2809e930 0x7f4d281c8400 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 --2- 172.21.15.89:0/2676223256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d2809f410 0x7f4d281c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 >> 172.21.15.89:0/2676223256 conn(0x7f4d280a4210 msgr2=0x7f4d28136120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 shutdown_connections 2023-09-30T22:18:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.804+0000 7f4d2ca53700 1 -- 172.21.15.89:0/2676223256 wait complete. 2023-09-30T22:18:12.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:12.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.846+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff0d806b9e0 con 0x7ff0d8134630 2023-09-30T22:18:12.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.846+0000 7ff0d4ff9700 1 -- 172.21.15.89:0/3658751677 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0cc021030 con 0x7ff0d8134630 2023-09-30T22:18:12.847 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:12.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8134630 msgr2=0x7ff0d806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8134630 0x7ff0d806fb20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e8d0 tx=0x7ff0cc00ebe0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 shutdown_connections 2023-09-30T22:18:12.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d8136f50 0x7ff0d8070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0d8131810 0x7ff0d806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 --2- 172.21.15.89:0/3658751677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0d8134630 0x7ff0d806fb20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 >> 172.21.15.89:0/3658751677 conn(0x7ff0d80a2aa0 msgr2=0x7ff0d809f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:12.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 shutdown_connections 2023-09-30T22:18:12.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.848+0000 7ff0de9ca700 1 -- 172.21.15.89:0/3658751677 wait complete. 2023-09-30T22:18:12.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.876+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6990125fa0 con 0x7f6990123ce0 2023-09-30T22:18:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.876+0000 7f6977fff700 1 -- 172.21.15.89:0/565560359 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f698801a330 con 0x7f6990123ce0 2023-09-30T22:18:12.877 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.878+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6990123ce0 msgr2=0x7f69901cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.878+0000 7f6995fbe700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6990123ce0 0x7f69901cca40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6990070170 tx=0x7f6988004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.878+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 shutdown_connections 2023-09-30T22:18:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.878+0000 7f6995fbe700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6990123ce0 0x7f69901cca40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.878+0000 7f6995fbe700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f699006e570 0x7f6990123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.878+0000 7f6995fbe700 1 --2- 172.21.15.89:0/565560359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699006f050 0x7f69901237a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:12.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.878+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 >> 172.21.15.89:0/565560359 conn(0x7f69900a2aa0 msgr2=0x7f69900a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:12.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.879+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 shutdown_connections 2023-09-30T22:18:12.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:12.879+0000 7f6995fbe700 1 -- 172.21.15.89:0/565560359 wait complete. 2023-09-30T22:18:12.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:13.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:13.818 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:13.862 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:13.893 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:13.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.893+0000 7f3ed18ec700 1 Processor -- start 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.893+0000 7f3ed18ec700 1 -- start start 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.893+0000 7f3ed18ec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09d6f0 0x7f3ecc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.893+0000 7f3ed18ec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09e410 0x7f3ecc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.893+0000 7f3ed18ec700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc09b120 con 0x7f3ecc09e000 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.893+0000 7f3ed18ec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc06b460 con 0x7f3ecc09e410 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ed18ec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc09a7e0 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09d6f0 0x7f3ecc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09d6f0 0x7f3ecc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43164/0 (socket says 172.21.15.89:43164) 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecaffd700 1 -- 172.21.15.89:0/3316857269 learned_addr learned my addr 172.21.15.89:0/3316857269 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:13.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecb7fe700 1 --2- 172.21.15.89:0/3316857269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09e410 0x7f3ecc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecaffd700 1 -- 172.21.15.89:0/3316857269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09e410 msgr2=0x7f3ecc134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:13.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecaffd700 1 --2- 172.21.15.89:0/3316857269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09e410 0x7f3ecc134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecaffd700 1 -- 172.21.15.89:0/3316857269 >> v1:172.21.15.89:6789/0 conn(0x7f3ecc09e000 legacy=0x7f3ecc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:13.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecaffd700 1 -- 172.21.15.89:0/3316857269 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc09a500 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ecaffd700 1 --2- 172.21.15.89:0/3316857269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09d6f0 0x7f3ecc09dac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009950 tx=0x7f3ebc009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1e77caac1ea40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:13.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ec9ffb700 1 -- 172.21.15.89:0/3316857269 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc004230 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ec9ffb700 1 -- 172.21.15.89:0/3316857269 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc004390 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.894+0000 7f3ec9ffb700 1 -- 172.21.15.89:0/3316857269 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc009ea0 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.895+0000 7f3ed18ec700 1 -- 172.21.15.89:0/3316857269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09d6f0 msgr2=0x7f3ecc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.895+0000 7f3ed18ec700 1 --2- 172.21.15.89:0/3316857269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09d6f0 0x7f3ecc09dac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009950 tx=0x7f3ebc009c60 comp rx=0 tx=0).stop 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.895+0000 7f3ed18ec700 1 -- 172.21.15.89:0/3316857269 shutdown_connections 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.895+0000 7f3ed18ec700 1 --2- 172.21.15.89:0/3316857269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09e410 0x7f3ecc134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.895+0000 7f3ed18ec700 1 --2- 172.21.15.89:0/3316857269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09d6f0 0x7f3ecc09dac0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.895+0000 7f3ed18ec700 1 -- 172.21.15.89:0/3316857269 >> 172.21.15.89:0/3316857269 conn(0x7f3ecc066f50 msgr2=0x7f3ecc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.895+0000 7f3ed18ec700 1 -- 172.21.15.89:0/3316857269 shutdown_connections 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.895+0000 7f3ed18ec700 1 -- 172.21.15.89:0/3316857269 wait complete. 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.896+0000 7f3ed18ec700 1 Processor -- start 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.896+0000 7f3ed18ec700 1 -- start start 2023-09-30T22:18:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ed18ec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09d6f0 0x7f3ecc122690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ed18ec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09e000 0x7f3ecc122bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ed18ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ecc09e410 0x7f3ecc123110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ed18ec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc123af0 con 0x7f3ecc09e410 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ed18ec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc1260b0 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ed18ec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0a1d90 con 0x7f3ecc09e000 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09d6f0 0x7f3ecc122690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3eca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09e000 0x7f3ecc122bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09d6f0 0x7f3ecc122690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36208/0 (socket says 172.21.15.89:36208) 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecaffd700 1 -- 172.21.15.89:0/2996301708 learned_addr learned my addr 172.21.15.89:0/2996301708 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:13.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecb7fe700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ecc09e410 0x7f3ecc123110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecaffd700 1 -- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09e000 msgr2=0x7f3ecc122bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:13.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecaffd700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09e000 0x7f3ecc122bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecaffd700 1 -- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ecc09e410 msgr2=0x7f3ecc123110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecaffd700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ecc09e410 0x7f3ecc123110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3ecaffd700 1 -- 172.21.15.89:0/2996301708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc009550 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.897+0000 7f3eca7fc700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09e000 0x7f3ecc122bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.898+0000 7f3ecaffd700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09d6f0 0x7f3ecc122690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009920 tx=0x7f3ebc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.898+0000 7f3eb3fff700 1 -- 172.21.15.89:0/2996301708 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00bb20 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.898+0000 7f3eb3fff700 1 -- 172.21.15.89:0/2996301708 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc01f070 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.898+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc09c710 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.898+0000 7f3eb3fff700 1 -- 172.21.15.89:0/2996301708 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00bea0 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.898+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc124fc0 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.899+0000 7f3eb3fff700 1 -- 172.21.15.89:0/2996301708 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ebc021030 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.899+0000 7f3eb3fff700 1 -- 172.21.15.89:0/2996301708 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3ebc0409e0 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.899+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc074ea0 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.904+0000 7f3eb3fff700 1 -- 172.21.15.89:0/2996301708 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ebc0199b0 con 0x7f3ecc09d6f0 2023-09-30T22:18:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.950+0000 7fee1cba1700 1 Processor -- start 2023-09-30T22:18:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.950+0000 7fee1cba1700 1 -- start start 2023-09-30T22:18:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.950+0000 7fee1cba1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809dba0 0x7fee1809df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.950+0000 7fee1cba1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1812f8c0 0x7fee1812fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.950+0000 7fee1cba1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee18130210 0x7fee18136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.950+0000 7fee1cba1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1809d670 con 0x7fee18130210 2023-09-30T22:18:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.950+0000 7fee1cba1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee18099d80 con 0x7fee1812f8c0 2023-09-30T22:18:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.950+0000 7fee1cba1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee1809b020 con 0x7fee1809dba0 2023-09-30T22:18:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809dba0 0x7fee1809df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee16ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1812f8c0 0x7fee1812fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809dba0 0x7fee1809df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43182/0 (socket says 172.21.15.89:43182) 2023-09-30T22:18:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 -- 172.21.15.89:0/4140499262 learned_addr learned my addr 172.21.15.89:0/4140499262 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee17fff700 1 --2- 172.21.15.89:0/4140499262 >> v2:172.21.15.89:3300/0 conn(0x7fee18130210 0x7fee18136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 -- 172.21.15.89:0/4140499262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1812f8c0 msgr2=0x7fee1812fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 --2- 172.21.15.89:0/4140499262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1812f8c0 0x7fee1812fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 -- 172.21.15.89:0/4140499262 >> v2:172.21.15.89:3300/0 conn(0x7fee18130210 msgr2=0x7fee18136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 --2- 172.21.15.89:0/4140499262 >> v2:172.21.15.89:3300/0 conn(0x7fee18130210 0x7fee18136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 -- 172.21.15.89:0/4140499262 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1809a440 con 0x7fee1809dba0 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee17fff700 1 --2- 172.21.15.89:0/4140499262 >> v2:172.21.15.89:3300/0 conn(0x7fee18130210 0x7fee18136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee177fe700 1 --2- 172.21.15.89:0/4140499262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809dba0 0x7fee1809df70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee08006fd0 tx=0x7fee08009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d79e610e5f89a65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee167fc700 1 -- 172.21.15.89:0/4140499262 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08005e30 con 0x7fee1809dba0 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee167fc700 1 -- 172.21.15.89:0/4140499262 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee08004030 con 0x7fee1809dba0 2023-09-30T22:18:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.951+0000 7fee167fc700 1 -- 172.21.15.89:0/4140499262 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee080043e0 con 0x7fee1809dba0 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 -- 172.21.15.89:0/4140499262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809dba0 msgr2=0x7fee1809df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 --2- 172.21.15.89:0/4140499262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809dba0 0x7fee1809df70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee08006fd0 tx=0x7fee08009bd0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 -- 172.21.15.89:0/4140499262 shutdown_connections 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 --2- 172.21.15.89:0/4140499262 >> v2:172.21.15.89:3300/0 conn(0x7fee18130210 0x7fee18136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 --2- 172.21.15.89:0/4140499262 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809dba0 0x7fee1809df70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 --2- 172.21.15.89:0/4140499262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1812f8c0 0x7fee1812fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 -- 172.21.15.89:0/4140499262 >> 172.21.15.89:0/4140499262 conn(0x7fee180a2540 msgr2=0x7fee180a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 -- 172.21.15.89:0/4140499262 shutdown_connections 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.952+0000 7fee1cba1700 1 -- 172.21.15.89:0/4140499262 wait complete. 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.953+0000 7fee1cba1700 1 Processor -- start 2023-09-30T22:18:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.953+0000 7fee1cba1700 1 -- start start 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee1cba1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809dba0 0x7fee1812c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee1cba1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1812f8c0 0x7fee18134b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee1cba1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18130210 0x7fee181350a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee1cba1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee18128ff0 con 0x7fee1809dba0 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee1cba1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee18127a30 con 0x7fee18130210 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee1cba1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee18127cf0 con 0x7fee1812f8c0 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809dba0 0x7fee1812c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809dba0 0x7fee1812c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42804/0 (socket says 172.21.15.89:42804) 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 -- 172.21.15.89:0/1385108345 learned_addr learned my addr 172.21.15.89:0/1385108345 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee16ffd700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1812f8c0 0x7fee18134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee17fff700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18130210 0x7fee181350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 -- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1812f8c0 msgr2=0x7fee18134b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1812f8c0 0x7fee18134b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 -- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18130210 msgr2=0x7fee181350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18130210 0x7fee181350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 -- 172.21.15.89:0/1385108345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee08009500 con 0x7fee1809dba0 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.954+0000 7fee177fe700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809dba0 0x7fee1812c600 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee08000c00 tx=0x7fee08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.955+0000 7fedfffff700 1 -- 172.21.15.89:0/1385108345 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0801c420 con 0x7fee1809dba0 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.955+0000 7fedfffff700 1 -- 172.21.15.89:0/1385108345 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0801c5a0 con 0x7fee1809dba0 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.955+0000 7fedfffff700 1 -- 172.21.15.89:0/1385108345 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0801c950 con 0x7fee1809dba0 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.955+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee18137dd0 con 0x7fee1809dba0 2023-09-30T22:18:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.955+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee18127fd0 con 0x7fee1809dba0 2023-09-30T22:18:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.956+0000 7fedfffff700 1 -- 172.21.15.89:0/1385108345 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee0801cab0 con 0x7fee1809dba0 2023-09-30T22:18:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.956+0000 7fedfffff700 1 -- 172.21.15.89:0/1385108345 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fee080404c0 con 0x7fee1809dba0 2023-09-30T22:18:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.956+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee18074740 con 0x7fee1809dba0 2023-09-30T22:18:13.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.965+0000 7fedfffff700 1 -- 172.21.15.89:0/1385108345 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee08016080 con 0x7fee1809dba0 2023-09-30T22:18:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.997+0000 7fa950ae1700 1 Processor -- start 2023-09-30T22:18:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.997+0000 7fa950ae1700 1 -- start start 2023-09-30T22:18:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa950ae1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c1345a0 0x7fa94c136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa950ae1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa94c136ec0 0x7fa94c1392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa950ae1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c099620 con 0x7fa94c0984f0 2023-09-30T22:18:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa950ae1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c09a9e0 con 0x7fa94c136ec0 2023-09-30T22:18:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa950ae1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c09bce0 con 0x7fa94c1345a0 2023-09-30T22:18:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa94ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa94c136ec0 0x7fa94c1392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa94a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa94c0984f0 0x7fa94c0988c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54102/0 (socket says 172.21.15.89:54102) 2023-09-30T22:18:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa94a59c700 1 -- 172.21.15.89:0/1615106021 learned_addr learned my addr 172.21.15.89:0/1615106021 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa949d9b700 1 --2- 172.21.15.89:0/1615106021 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c1345a0 0x7fa94c136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa94ad9d700 1 -- 172.21.15.89:0/1615106021 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c1345a0 msgr2=0x7fa94c136980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa94ad9d700 1 --2- 172.21.15.89:0/1615106021 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c1345a0 0x7fa94c136980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa94ad9d700 1 -- 172.21.15.89:0/1615106021 >> v1:172.21.15.89:6789/0 conn(0x7fa94c0984f0 legacy=0x7fa94c0988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.998+0000 7fa94ad9d700 1 -- 172.21.15.89:0/1615106021 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c09aca0 con 0x7fa94c136ec0 2023-09-30T22:18:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.999+0000 7fa94ad9d700 1 --2- 172.21.15.89:0/1615106021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c136ec0 0x7fa94c1392a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa944009d90 tx=0x7fa94400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6638aea88dc4674e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.999+0000 7fa94959a700 1 -- 172.21.15.89:0/1615106021 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400e070 con 0x7fa94c136ec0 2023-09-30T22:18:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.999+0000 7fa94959a700 1 -- 172.21.15.89:0/1615106021 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94400bb20 con 0x7fa94c136ec0 2023-09-30T22:18:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.999+0000 7fa94959a700 1 -- 172.21.15.89:0/1615106021 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944012430 con 0x7fa94c136ec0 2023-09-30T22:18:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.999+0000 7fa950ae1700 1 -- 172.21.15.89:0/1615106021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c136ec0 msgr2=0x7fa94c1392a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:13.999+0000 7fa950ae1700 1 --2- 172.21.15.89:0/1615106021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c136ec0 0x7fa94c1392a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa944009d90 tx=0x7fa94400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.000+0000 7fa950ae1700 1 -- 172.21.15.89:0/1615106021 shutdown_connections 2023-09-30T22:18:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.000+0000 7fa950ae1700 1 --2- 172.21.15.89:0/1615106021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c136ec0 0x7fa94c1392a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.000+0000 7fa950ae1700 1 --2- 172.21.15.89:0/1615106021 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c1345a0 0x7fa94c136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.000+0000 7fa950ae1700 1 -- 172.21.15.89:0/1615106021 >> 172.21.15.89:0/1615106021 conn(0x7fa94c09f470 msgr2=0x7fa94c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:14.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.000+0000 7fa950ae1700 1 -- 172.21.15.89:0/1615106021 shutdown_connections 2023-09-30T22:18:14.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.000+0000 7fa950ae1700 1 -- 172.21.15.89:0/1615106021 wait complete. 2023-09-30T22:18:14.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.001+0000 7fa950ae1700 1 Processor -- start 2023-09-30T22:18:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.001+0000 7fa950ae1700 1 -- start start 2023-09-30T22:18:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa950ae1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c0984f0 0x7fa94c1c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa950ae1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c1345a0 0x7fa94c1c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa950ae1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94c136ec0 0x7fa94c1c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa950ae1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c129b20 con 0x7fa94c1345a0 2023-09-30T22:18:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa950ae1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c09a2b0 con 0x7fa94c0984f0 2023-09-30T22:18:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa950ae1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c129d20 con 0x7fa94c136ec0 2023-09-30T22:18:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94c136ec0 0x7fa94c1c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94c136ec0 0x7fa94c1c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43212/0 (socket says 172.21.15.89:43212) 2023-09-30T22:18:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c0984f0 0x7fa94c1c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94ad9d700 1 -- 172.21.15.89:0/1714483507 learned_addr learned my addr 172.21.15.89:0/1714483507 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa949d9b700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c1345a0 0x7fa94c1c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94ad9d700 1 -- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c0984f0 msgr2=0x7fa94c1c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94ad9d700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c0984f0 0x7fa94c1c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94ad9d700 1 -- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c1345a0 msgr2=0x7fa94c1c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94ad9d700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c1345a0 0x7fa94c1c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa94ad9d700 1 -- 172.21.15.89:0/1714483507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa944009a70 con 0x7fa94c136ec0 2023-09-30T22:18:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.002+0000 7fa949d9b700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c1345a0 0x7fa94c1c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.003+0000 7fa94ad9d700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94c136ec0 0x7fa94c1c6d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa944014040 tx=0x7fa944004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.003+0000 7fa93affd700 1 -- 172.21.15.89:0/1714483507 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944012430 con 0x7fa94c136ec0 2023-09-30T22:18:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.003+0000 7fa93affd700 1 -- 172.21.15.89:0/1714483507 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa944020070 con 0x7fa94c136ec0 2023-09-30T22:18:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.003+0000 7fa950ae1700 1 -- 172.21.15.89:0/1714483507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c126e00 con 0x7fa94c136ec0 2023-09-30T22:18:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.003+0000 7fa93affd700 1 -- 172.21.15.89:0/1714483507 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944004040 con 0x7fa94c136ec0 2023-09-30T22:18:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.003+0000 7fa950ae1700 1 -- 172.21.15.89:0/1714483507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c128600 con 0x7fa94c136ec0 2023-09-30T22:18:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.004+0000 7fa938ff9700 1 -- 172.21.15.89:0/1714483507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9240052e0 con 0x7fa94c136ec0 2023-09-30T22:18:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.004+0000 7fa93affd700 1 -- 172.21.15.89:0/1714483507 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9440041a0 con 0x7fa94c136ec0 2023-09-30T22:18:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.004+0000 7fa93affd700 1 -- 172.21.15.89:0/1714483507 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa944022070 con 0x7fa94c136ec0 2023-09-30T22:18:14.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.013+0000 7fa93affd700 1 -- 172.21.15.89:0/1714483507 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa944011980 con 0x7fa94c136ec0 2023-09-30T22:18:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.031+0000 7f73ffd3c700 1 Processor -- start 2023-09-30T22:18:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.032+0000 7f73ffd3c700 1 -- start start 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.032+0000 7f73ffd3c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73f809d6f0 0x7f73f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.032+0000 7f73ffd3c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e000 0x7f73f8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.032+0000 7f73ffd3c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809e410 0x7f73f8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.032+0000 7f73ffd3c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f809f520 con 0x7f73f809d6f0 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.032+0000 7f73ffd3c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f809f130 con 0x7f73f809e410 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.032+0000 7f73ffd3c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73f809e930 con 0x7f73f809e000 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.032+0000 7f73fdad8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73f809d6f0 0x7f73f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fdad8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73f809d6f0 0x7f73f809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42822/0 (socket says 172.21.15.89:42822) 2023-09-30T22:18:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fdad8700 1 -- 172.21.15.89:0/1245870062 learned_addr learned my addr 172.21.15.89:0/1245870062 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fe2d9700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809e410 0x7f73f8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fd2d7700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e000 0x7f73f8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fdad8700 1 -- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e000 msgr2=0x7f73f8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fdad8700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e000 0x7f73f8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fdad8700 1 -- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809e410 msgr2=0x7f73f8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fdad8700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809e410 0x7f73f8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fdad8700 1 -- 172.21.15.89:0/1245870062 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f809b2d0 con 0x7f73f809d6f0 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fd2d7700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e000 0x7f73f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fdad8700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809d6f0 0x7f73f809dac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009920 tx=0x7f73f4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98bc81389cd91d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.033+0000 7f73fcad6700 1 -- 172.21.15.89:0/1245870062 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4011830 con 0x7f73f809d6f0 2023-09-30T22:18:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73fcad6700 1 -- 172.21.15.89:0/1245870062 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f4011990 con 0x7f73f809d6f0 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73fcad6700 1 -- 172.21.15.89:0/1245870062 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4011d20 con 0x7f73f809d6f0 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73ffd3c700 1 -- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809d6f0 msgr2=0x7f73f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73ffd3c700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809d6f0 0x7f73f809dac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009920 tx=0x7f73f4009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73ffd3c700 1 -- 172.21.15.89:0/1245870062 shutdown_connections 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73ffd3c700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809e410 0x7f73f8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73ffd3c700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809d6f0 0x7f73f809dac0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73ffd3c700 1 --2- 172.21.15.89:0/1245870062 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e000 0x7f73f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.034+0000 7f73ffd3c700 1 -- 172.21.15.89:0/1245870062 >> 172.21.15.89:0/1245870062 conn(0x7f73f8066f50 msgr2=0x7f73f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.035+0000 7f73ffd3c700 1 -- 172.21.15.89:0/1245870062 shutdown_connections 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.035+0000 7f73ffd3c700 1 -- 172.21.15.89:0/1245870062 wait complete. 2023-09-30T22:18:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.036+0000 7f73ffd3c700 1 Processor -- start 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.036+0000 7f73ffd3c700 1 -- start start 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73ffd3c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809d6f0 0x7f73f811e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73ffd3c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809e000 0x7f73f811e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73ffd3c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e410 0x7f73f811eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73ffd3c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80a3b10 con 0x7f73f809e000 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73ffd3c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f81218f0 con 0x7f73f809d6f0 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73ffd3c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73f811fcc0 con 0x7f73f809e410 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e410 0x7f73f811eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e410 0x7f73f811eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43228/0 (socket says 172.21.15.89:43228) 2023-09-30T22:18:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 -- 172.21.15.89:0/165407980 learned_addr learned my addr 172.21.15.89:0/165407980 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fd2d7700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809e000 0x7f73f811e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fdad8700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809d6f0 0x7f73f811e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 -- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809d6f0 msgr2=0x7f73f811e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809d6f0 0x7f73f811e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 -- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809e000 msgr2=0x7f73f811e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809e000 0x7f73f811e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 -- 172.21.15.89:0/165407980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80094a0 con 0x7f73f809e410 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.037+0000 7f73fe2d9700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e410 0x7f73f811eb30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009dd0 tx=0x7f73f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.038+0000 7f73ee7fc700 1 -- 172.21.15.89:0/165407980 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0009c40 con 0x7f73f809e410 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.038+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f4009550 con 0x7f73f809e410 2023-09-30T22:18:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.038+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f81204b0 con 0x7f73f809e410 2023-09-30T22:18:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.038+0000 7f73ee7fc700 1 -- 172.21.15.89:0/165407980 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f000f040 con 0x7f73f809e410 2023-09-30T22:18:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.038+0000 7f73ee7fc700 1 -- 172.21.15.89:0/165407980 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0014480 con 0x7f73f809e410 2023-09-30T22:18:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.039+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f8074ea0 con 0x7f73f809e410 2023-09-30T22:18:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.039+0000 7f73ee7fc700 1 -- 172.21.15.89:0/165407980 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f73f00145e0 con 0x7f73f809e410 2023-09-30T22:18:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.039+0000 7f73ee7fc700 1 -- 172.21.15.89:0/165407980 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f73f003feb0 con 0x7f73f809e410 2023-09-30T22:18:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.048+0000 7f73ee7fc700 1 -- 172.21.15.89:0/165407980 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73f0023890 con 0x7f73f809e410 2023-09-30T22:18:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.115+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ecc074ea0 con 0x7f3ecc09d6f0 2023-09-30T22:18:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.115+0000 7f3eb3fff700 1 -- 172.21.15.89:0/2996301708 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ebc01b440 con 0x7f3ecc09d6f0 2023-09-30T22:18:14.118 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09d6f0 msgr2=0x7f3ecc122690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09d6f0 0x7f3ecc122690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009920 tx=0x7f3ebc004750 comp rx=0 tx=0).stop 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 shutdown_connections 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ecc09e410 0x7f3ecc123110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ecc09d6f0 0x7f3ecc122690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 --2- 172.21.15.89:0/2996301708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ecc09e000 0x7f3ecc122bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 >> 172.21.15.89:0/2996301708 conn(0x7f3ecc066f50 msgr2=0x7f3ecc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 shutdown_connections 2023-09-30T22:18:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.117+0000 7f3ed18ec700 1 -- 172.21.15.89:0/2996301708 wait complete. 2023-09-30T22:18:14.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:14.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.165+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fee1804f2a0 con 0x7fee1809dba0 2023-09-30T22:18:14.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.166+0000 7fedfffff700 1 -- 172.21.15.89:0/1385108345 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee08020020 con 0x7fee1809dba0 2023-09-30T22:18:14.166 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:14.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.167+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809dba0 msgr2=0x7fee1812c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.167+0000 7fee1cba1700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809dba0 0x7fee1812c600 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee08000c00 tx=0x7fee08004750 comp rx=0 tx=0).stop 2023-09-30T22:18:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.167+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 shutdown_connections 2023-09-30T22:18:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.167+0000 7fee1cba1700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee18130210 0x7fee181350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.167+0000 7fee1cba1700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809dba0 0x7fee1812c600 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.167+0000 7fee1cba1700 1 --2- 172.21.15.89:0/1385108345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1812f8c0 0x7fee18134b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.167+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 >> 172.21.15.89:0/1385108345 conn(0x7fee180a2540 msgr2=0x7fee1812d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.168+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 shutdown_connections 2023-09-30T22:18:14.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.168+0000 7fee1cba1700 1 -- 172.21.15.89:0/1385108345 wait complete. 2023-09-30T22:18:14.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:14.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.212+0000 7fa938ff9700 1 -- 172.21.15.89:0/1714483507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa924005590 con 0x7fa94c136ec0 2023-09-30T22:18:14.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.212+0000 7fa93affd700 1 -- 172.21.15.89:0/1714483507 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa944011390 con 0x7fa94c136ec0 2023-09-30T22:18:14.213 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:14.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 -- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94c136ec0 msgr2=0x7fa94c1c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94c136ec0 0x7fa94c1c6d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa944014040 tx=0x7fa944004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 -- 172.21.15.89:0/1714483507 shutdown_connections 2023-09-30T22:18:14.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94c136ec0 0x7fa94c1c6d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa94c0984f0 0x7fa94c1c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 --2- 172.21.15.89:0/1714483507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94c1345a0 0x7fa94c1c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 -- 172.21.15.89:0/1714483507 >> 172.21.15.89:0/1714483507 conn(0x7fa94c09f470 msgr2=0x7fa94c09c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 -- 172.21.15.89:0/1714483507 shutdown_connections 2023-09-30T22:18:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.214+0000 7fa938ff9700 1 -- 172.21.15.89:0/1714483507 wait complete. 2023-09-30T22:18:14.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.242+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f73f8071e20 con 0x7f73f809e410 2023-09-30T22:18:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.242+0000 7f73ee7fc700 1 -- 172.21.15.89:0/165407980 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73f0012340 con 0x7f73f809e410 2023-09-30T22:18:14.243 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:14.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e410 msgr2=0x7f73f811eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:14.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e410 0x7f73f811eb30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009dd0 tx=0x7f73f000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 shutdown_connections 2023-09-30T22:18:14.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73f809e410 0x7f73f811eb30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f809d6f0 0x7f73f811e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 --2- 172.21.15.89:0/165407980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f809e000 0x7f73f811e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:14.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 >> 172.21.15.89:0/165407980 conn(0x7f73f8066f50 msgr2=0x7f73f8070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:14.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 shutdown_connections 2023-09-30T22:18:14.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:14.244+0000 7f73ffd3c700 1 -- 172.21.15.89:0/165407980 wait complete. 2023-09-30T22:18:14.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:15.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:16.636 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:16.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:16.648 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.790+0000 7f964c361700 1 Processor -- start 2023-09-30T22:18:16.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.790+0000 7f964c361700 1 -- start start 2023-09-30T22:18:16.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f964c361700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409e1d0 0x7f9644135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f964c361700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9644135e50 0x7f9644138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f964c361700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964409f130 con 0x7f964409d6f0 2023-09-30T22:18:16.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f964c361700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964409f520 con 0x7f9644135e50 2023-09-30T22:18:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f964c361700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f964409e930 con 0x7f964409e1d0 2023-09-30T22:18:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f195dccc700 1 Processor -- start 2023-09-30T22:18:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96498fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409e1d0 0x7f9644135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96498fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409e1d0 0x7f9644135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43236/0 (socket says 172.21.15.89:43236) 2023-09-30T22:18:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96498fc700 1 -- 172.21.15.89:0/2373744744 learned_addr learned my addr 172.21.15.89:0/2373744744 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f195dccc700 1 -- start start 2023-09-30T22:18:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96498fc700 1 -- 172.21.15.89:0/2373744744 >> v2:172.21.15.89:3300/0 conn(0x7f9644135e50 msgr2=0x7f9644138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96498fc700 1 --2- 172.21.15.89:0/2373744744 >> v2:172.21.15.89:3300/0 conn(0x7f9644135e50 0x7f9644138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96498fc700 1 -- 172.21.15.89:0/2373744744 >> v1:172.21.15.89:6789/0 conn(0x7f964409d6f0 legacy=0x7f964409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96498fc700 1 -- 172.21.15.89:0/2373744744 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964409b2d0 con 0x7f964409e1d0 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96498fc700 1 --2- 172.21.15.89:0/2373744744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409e1d0 0x7f9644135910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9634009920 tx=0x7f9634009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=472c5a8745268fa0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.791+0000 7f96490fb700 1 -- 172.21.15.89:0/2373744744 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634004230 con 0x7f964409e1d0 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f96490fb700 1 -- 172.21.15.89:0/2373744744 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634004390 con 0x7f964409e1d0 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f96490fb700 1 -- 172.21.15.89:0/2373744744 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634009e70 con 0x7f964409e1d0 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f964c361700 1 -- 172.21.15.89:0/2373744744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409e1d0 msgr2=0x7f9644135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f964c361700 1 --2- 172.21.15.89:0/2373744744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409e1d0 0x7f9644135910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9634009920 tx=0x7f9634009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f195dccc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f195806e570 0x7f195806e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f195dccc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f1958139740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f195dccc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1958130260 con 0x7f1958130670 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f195dccc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195809a350 con 0x7f195806e570 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f195dccc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1958071ff0 con 0x7f195806eec0 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f1957fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f1958139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f1957fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f1958139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43250/0 (socket says 172.21.15.89:43250) 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f1957fff700 1 -- 172.21.15.89:0/1474470079 learned_addr learned my addr 172.21.15.89:0/1474470079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f1957fff700 1 -- 172.21.15.89:0/1474470079 >> v2:172.21.15.89:3300/0 conn(0x7f195806e570 msgr2=0x7f195806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f1957fff700 1 --2- 172.21.15.89:0/1474470079 >> v2:172.21.15.89:3300/0 conn(0x7f195806e570 0x7f195806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f1957fff700 1 -- 172.21.15.89:0/1474470079 >> v1:172.21.15.89:6789/0 conn(0x7f1958130670 legacy=0x7f1958130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f1957fff700 1 -- 172.21.15.89:0/1474470079 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195809ecf0 con 0x7f195806eec0 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f1957fff700 1 --2- 172.21.15.89:0/1474470079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f1958139740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1950014500 tx=0x7f1950014810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d240b728e4725502 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.792+0000 7f19567fc700 1 -- 172.21.15.89:0/1474470079 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195000f220 con 0x7f195806eec0 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f964c361700 1 -- 172.21.15.89:0/2373744744 shutdown_connections 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f964c361700 1 --2- 172.21.15.89:0/2373744744 >> v2:172.21.15.89:3300/0 conn(0x7f9644135e50 0x7f9644138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f964c361700 1 --2- 172.21.15.89:0/2373744744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409e1d0 0x7f9644135910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f964c361700 1 -- 172.21.15.89:0/2373744744 >> 172.21.15.89:0/2373744744 conn(0x7f9644066f50 msgr2=0x7f964409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f19567fc700 1 -- 172.21.15.89:0/1474470079 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195000f380 con 0x7f195806eec0 2023-09-30T22:18:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f19567fc700 1 -- 172.21.15.89:0/1474470079 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950003c40 con 0x7f195806eec0 2023-09-30T22:18:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f964c361700 1 -- 172.21.15.89:0/2373744744 shutdown_connections 2023-09-30T22:18:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f964c361700 1 -- 172.21.15.89:0/2373744744 wait complete. 2023-09-30T22:18:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f195dccc700 1 -- 172.21.15.89:0/1474470079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 msgr2=0x7f1958139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f195dccc700 1 --2- 172.21.15.89:0/1474470079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f1958139740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1950014500 tx=0x7f1950014810 comp rx=0 tx=0).stop 2023-09-30T22:18:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f195dccc700 1 -- 172.21.15.89:0/1474470079 shutdown_connections 2023-09-30T22:18:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f195dccc700 1 --2- 172.21.15.89:0/1474470079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f1958139740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f195dccc700 1 --2- 172.21.15.89:0/1474470079 >> v2:172.21.15.89:3300/0 conn(0x7f195806e570 0x7f195806e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f195dccc700 1 -- 172.21.15.89:0/1474470079 >> 172.21.15.89:0/1474470079 conn(0x7f19580a2aa0 msgr2=0x7f19580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:16.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.793+0000 7f964c361700 1 Processor -- start 2023-09-30T22:18:16.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964c361700 1 -- start start 2023-09-30T22:18:16.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f195dccc700 1 -- 172.21.15.89:0/1474470079 shutdown_connections 2023-09-30T22:18:16.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f195dccc700 1 -- 172.21.15.89:0/1474470079 wait complete. 2023-09-30T22:18:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964c361700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409d6f0 0x7f964411a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964c361700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964409e1d0 0x7f964411a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964c361700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9644135e50 0x7f96441ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964c361700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964411d150 con 0x7f964409e1d0 2023-09-30T22:18:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964c361700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964411bc30 con 0x7f9644135e50 2023-09-30T22:18:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964c361700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9644135be0 con 0x7f964409d6f0 2023-09-30T22:18:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964a0fd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409d6f0 0x7f964411a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964a0fd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409d6f0 0x7f964411a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43256/0 (socket says 172.21.15.89:43256) 2023-09-30T22:18:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964a0fd700 1 -- 172.21.15.89:0/3838043172 learned_addr learned my addr 172.21.15.89:0/3838043172 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.794+0000 7f964a8fe700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9644135e50 0x7f96441ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f964a0fd700 1 -- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9644135e50 msgr2=0x7f96441ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f964a0fd700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9644135e50 0x7f96441ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f964a0fd700 1 -- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964409e1d0 msgr2=0x7f964411a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-30T22:18:16.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f195dccc700 1 Processor -- start 2023-09-30T22:18:16.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f96498fc700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964409e1d0 0x7f964411a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f964a0fd700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964409e1d0 0x7f964411a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f964a0fd700 1 -- 172.21.15.89:0/3838043172 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634009550 con 0x7f964409d6f0 2023-09-30T22:18:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f96498fc700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964409e1d0 0x7f964411a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f195dccc700 1 -- start start 2023-09-30T22:18:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f964a0fd700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409d6f0 0x7f964411a2b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f964000b470 tx=0x7f964000b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:16.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f963affd700 1 -- 172.21.15.89:0/3838043172 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640011830 con 0x7f964409d6f0 2023-09-30T22:18:16.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f963affd700 1 -- 172.21.15.89:0/3838043172 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96400119b0 con 0x7f964409d6f0 2023-09-30T22:18:16.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964411c4c0 con 0x7f964409d6f0 2023-09-30T22:18:16.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f963affd700 1 -- 172.21.15.89:0/3838043172 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640011d60 con 0x7f964409d6f0 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f195dccc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f195806e570 0x7f195811b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f195dccc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f195811be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f195dccc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1958130670 0x7f195811c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f195dccc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195811f510 con 0x7f1958130670 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f195dccc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195809fd70 con 0x7f195806e570 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f195dccc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1958071820 con 0x7f195806eec0 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f195811be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f195811be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43270/0 (socket says 172.21.15.89:43270) 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 -- 172.21.15.89:0/3432796531 learned_addr learned my addr 172.21.15.89:0/3432796531 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1957fff700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1958130670 0x7f195811c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 -- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f195806e570 msgr2=0x7f195811b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f195806e570 0x7f195811b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 -- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1958130670 msgr2=0x7f195811c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1958130670 0x7f195811c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 -- 172.21.15.89:0/3432796531 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1950014160 con 0x7f195806eec0 2023-09-30T22:18:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1957fff700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1958130670 0x7f195811c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.795+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964411ad30 con 0x7f964409d6f0 2023-09-30T22:18:16.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f1956ffd700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f195811be90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f194c00b560 tx=0x7f194c00b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:16.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f193ffff700 1 -- 172.21.15.89:0/3432796531 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c011830 con 0x7f195806eec0 2023-09-30T22:18:16.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f193ffff700 1 -- 172.21.15.89:0/3432796531 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c011990 con 0x7f195806eec0 2023-09-30T22:18:16.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f193ffff700 1 -- 172.21.15.89:0/3432796531 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c011d40 con 0x7f195806eec0 2023-09-30T22:18:16.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195811cd40 con 0x7f195806eec0 2023-09-30T22:18:16.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.796+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964409ef50 con 0x7f964409d6f0 2023-09-30T22:18:16.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.797+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195811dad0 con 0x7f195806eec0 2023-09-30T22:18:16.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.797+0000 7f963affd700 1 -- 172.21.15.89:0/3838043172 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f964001a400 con 0x7f964409d6f0 2023-09-30T22:18:16.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.797+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195811ede0 con 0x7f195806eec0 2023-09-30T22:18:16.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.798+0000 7f193ffff700 1 -- 172.21.15.89:0/3432796531 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f194c01a400 con 0x7f195806eec0 2023-09-30T22:18:16.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.798+0000 7f963affd700 1 -- 172.21.15.89:0/3838043172 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f964003fd90 con 0x7f964409d6f0 2023-09-30T22:18:16.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f7da9700 1 Processor -- start 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f7da9700 1 -- start start 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f7da9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009e1d0 0x7fe5f0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f7da9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f0139170 0x7fe5f013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f7da9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f009b120 con 0x7fe5f009d6f0 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f7da9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f006b460 con 0x7fe5f009e1d0 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f7da9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f009a7e0 con 0x7fe5f0139170 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f6346700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f0139170 0x7fe5f013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.801+0000 7fe5f5344700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009e1d0 0x7fe5f0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f6346700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f0139170 0x7fe5f013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43286/0 (socket says 172.21.15.89:43286) 2023-09-30T22:18:16.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f5344700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009e1d0 0x7fe5f0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36292/0 (socket says 172.21.15.89:36292) 2023-09-30T22:18:16.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f6346700 1 -- 172.21.15.89:0/1870792586 learned_addr learned my addr 172.21.15.89:0/1870792586 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:16.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f6346700 1 -- 172.21.15.89:0/1870792586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009e1d0 msgr2=0x7fe5f0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:16.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f6346700 1 --2- 172.21.15.89:0/1870792586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009e1d0 0x7fe5f0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f6346700 1 -- 172.21.15.89:0/1870792586 >> v1:172.21.15.89:6789/0 conn(0x7fe5f009d6f0 legacy=0x7fe5f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:16.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f6346700 1 -- 172.21.15.89:0/1870792586 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f009a500 con 0x7fe5f0139170 2023-09-30T22:18:16.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f5344700 1 --2- 172.21.15.89:0/1870792586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009e1d0 0x7fe5f0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:16.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.802+0000 7fe5f6346700 1 --2- 172.21.15.89:0/1870792586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f0139170 0x7fe5f013b550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009eb0 tx=0x7fe5e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e894f9cf78be6e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:16.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.803+0000 7fa5dc211700 1 Processor -- start 2023-09-30T22:18:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.803+0000 7fa5dc211700 1 -- start start 2023-09-30T22:18:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.803+0000 7fa5dc211700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5dc211700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5d409e000 0x7fa5d41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5dc211700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d409b120 con 0x7fa5d409e410 2023-09-30T22:18:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5dc211700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d406b460 con 0x7fa5d409e000 2023-09-30T22:18:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5dc211700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d409a7e0 con 0x7fa5d409d6f0 2023-09-30T22:18:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d97ac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5d409e000 0x7fa5d41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d9fad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d97ac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5d409e000 0x7fa5d41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42862/0 (socket says 172.21.15.89:42862) 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d9fad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36306/0 (socket says 172.21.15.89:36306) 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d97ac700 1 -- 172.21.15.89:0/2955889228 learned_addr learned my addr 172.21.15.89:0/2955889228 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d97ac700 1 -- 172.21.15.89:0/2955889228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 msgr2=0x7fa5d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d97ac700 1 --2- 172.21.15.89:0/2955889228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d97ac700 1 -- 172.21.15.89:0/2955889228 >> v1:172.21.15.89:6789/0 conn(0x7fa5d409e410 legacy=0x7fa5d4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d97ac700 1 -- 172.21.15.89:0/2955889228 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d409a500 con 0x7fa5d409e000 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d9fad700 1 --2- 172.21.15.89:0/2955889228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d97ac700 1 --2- 172.21.15.89:0/2955889228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 0x7fa5d41339f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40098f0 tx=0x7fa5c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1fa01d6616c1df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d8fab700 1 -- 172.21.15.89:0/2955889228 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4011830 con 0x7fa5d409e000 2023-09-30T22:18:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d8fab700 1 -- 172.21.15.89:0/2955889228 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c4011990 con 0x7fa5d409e000 2023-09-30T22:18:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fa5d8fab700 1 -- 172.21.15.89:0/2955889228 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4011d20 con 0x7fa5d409e000 2023-09-30T22:18:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7fe5f4b43700 1 -- 172.21.15.89:0/1870792586 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800e070 con 0x7fe5f0139170 2023-09-30T22:18:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.804+0000 7f193ffff700 1 -- 172.21.15.89:0/3432796531 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f194c022750 con 0x7f195806eec0 2023-09-30T22:18:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7f963affd700 1 -- 172.21.15.89:0/3838043172 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96400218d0 con 0x7f964409d6f0 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7fa5dc211700 1 -- 172.21.15.89:0/2955889228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 msgr2=0x7fa5d41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7fa5dc211700 1 --2- 172.21.15.89:0/2955889228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 0x7fa5d41339f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40098f0 tx=0x7fa5c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7fa5dc211700 1 -- 172.21.15.89:0/2955889228 shutdown_connections 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7fa5dc211700 1 --2- 172.21.15.89:0/2955889228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7fa5dc211700 1 --2- 172.21.15.89:0/2955889228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 0x7fa5d41339f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7fa5dc211700 1 -- 172.21.15.89:0/2955889228 >> 172.21.15.89:0/2955889228 conn(0x7fa5d4066f50 msgr2=0x7fa5d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7fa5dc211700 1 -- 172.21.15.89:0/2955889228 shutdown_connections 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.805+0000 7fa5dc211700 1 -- 172.21.15.89:0/2955889228 wait complete. 2023-09-30T22:18:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.806+0000 7fa5dc211700 1 Processor -- start 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.806+0000 7fa5dc211700 1 -- start start 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5dc211700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d41a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5dc211700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 0x7fa5d41a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5dc211700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5d409e410 0x7fa5d41a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5dc211700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d41a91f0 con 0x7fa5d409e000 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5dc211700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d409c710 con 0x7fa5d409d6f0 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5dc211700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40a3160 con 0x7fa5d409e410 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d97ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 0x7fa5d41a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d9fad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d41a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d97ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 0x7fa5d41a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42868/0 (socket says 172.21.15.89:42868) 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d97ac700 1 -- 172.21.15.89:0/2591005121 learned_addr learned my addr 172.21.15.89:0/2591005121 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5da7ae700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5d409e410 0x7fa5d41a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d9fad700 1 -- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5d409e410 msgr2=0x7fa5d41a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d9fad700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5d409e410 0x7fa5d41a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d9fad700 1 -- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 msgr2=0x7fa5d41a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d9fad700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 0x7fa5d41a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d9fad700 1 -- 172.21.15.89:0/2591005121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d00094a0 con 0x7fa5d409d6f0 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5da7ae700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5d409e410 0x7fa5d41a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5d9fad700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d41a5fd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e9b0 tx=0x7fa5d000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.807+0000 7fa5ca7fc700 1 -- 172.21.15.89:0/2591005121 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000ee50 con 0x7fa5d409d6f0 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.808+0000 7fa5ca7fc700 1 -- 172.21.15.89:0/2591005121 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d000c850 con 0x7fa5d409d6f0 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.808+0000 7fa5ca7fc700 1 -- 172.21.15.89:0/2591005121 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000cc00 con 0x7fa5d409d6f0 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.808+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c4009550 con 0x7fa5d409d6f0 2023-09-30T22:18:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.808+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d41a8da0 con 0x7fa5d409d6f0 2023-09-30T22:18:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.808+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d41a9600 con 0x7fa5d409d6f0 2023-09-30T22:18:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.809+0000 7fa5ca7fc700 1 -- 172.21.15.89:0/2591005121 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5d0013470 con 0x7fa5d409d6f0 2023-09-30T22:18:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.809+0000 7fa5ca7fc700 1 -- 172.21.15.89:0/2591005121 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa5d003f760 con 0x7fa5d409d6f0 2023-09-30T22:18:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7fe5f4b43700 1 -- 172.21.15.89:0/1870792586 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e8003a20 con 0x7fe5f0139170 2023-09-30T22:18:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7fe5f4b43700 1 -- 172.21.15.89:0/1870792586 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8003dd0 con 0x7fe5f0139170 2023-09-30T22:18:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7f193ffff700 1 -- 172.21.15.89:0/3432796531 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f194c00f8d0 con 0x7f195806eec0 2023-09-30T22:18:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7fe5f7da9700 1 -- 172.21.15.89:0/1870792586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f0139170 msgr2=0x7fe5f013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7fe5f7da9700 1 --2- 172.21.15.89:0/1870792586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f0139170 0x7fe5f013b550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009eb0 tx=0x7fe5e800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7fe5f7da9700 1 -- 172.21.15.89:0/1870792586 shutdown_connections 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7fe5f7da9700 1 --2- 172.21.15.89:0/1870792586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f0139170 0x7fe5f013b550 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7fe5f7da9700 1 --2- 172.21.15.89:0/1870792586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009e1d0 0x7fe5f0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.810+0000 7fe5f7da9700 1 -- 172.21.15.89:0/1870792586 >> 172.21.15.89:0/1870792586 conn(0x7fe5f0066f50 msgr2=0x7fe5f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.811+0000 7fe5f7da9700 1 -- 172.21.15.89:0/1870792586 shutdown_connections 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.811+0000 7fe5f7da9700 1 -- 172.21.15.89:0/1870792586 wait complete. 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.812+0000 7fe5f7da9700 1 Processor -- start 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.812+0000 7fe5f7da9700 1 -- start start 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f7da9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009d6f0 0x7fe5f0120720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f7da9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f009e1d0 0x7fe5f0120c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f7da9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f0139170 0x7fe5f01211a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f7da9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0122b90 con 0x7fe5f0139170 2023-09-30T22:18:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f7da9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0124240 con 0x7fe5f009d6f0 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f7da9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f01224b0 con 0x7fe5f009e1d0 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009d6f0 0x7fe5f0120720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009d6f0 0x7fe5f0120720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36332/0 (socket says 172.21.15.89:36332) 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 -- 172.21.15.89:0/2991339211 learned_addr learned my addr 172.21.15.89:0/2991339211 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5344700 1 --2- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f009e1d0 0x7fe5f0120c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 -- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f009e1d0 msgr2=0x7fe5f0120c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 --2- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f009e1d0 0x7fe5f0120c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 -- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f0139170 msgr2=0x7fe5f01211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 --2- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f0139170 0x7fe5f01211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 -- 172.21.15.89:0/2991339211 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00094a0 con 0x7fe5f009d6f0 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.813+0000 7fe5f5b45700 1 --2- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009d6f0 0x7fe5f0120720 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00c4a0 tx=0x7fe5ec00c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.817+0000 7fe5e67fc700 1 -- 172.21.15.89:0/2991339211 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0054e0 con 0x7fe5f009d6f0 2023-09-30T22:18:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.817+0000 7fe5e67fc700 1 -- 172.21.15.89:0/2991339211 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec003680 con 0x7fe5f009d6f0 2023-09-30T22:18:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.817+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e8009b30 con 0x7fe5f009d6f0 2023-09-30T22:18:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.818+0000 7fa5ca7fc700 1 -- 172.21.15.89:0/2591005121 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5d0013720 con 0x7fa5d409d6f0 2023-09-30T22:18:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.818+0000 7fe5e67fc700 1 -- 172.21.15.89:0/2991339211 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec003a30 con 0x7fe5f009d6f0 2023-09-30T22:18:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.818+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f009c710 con 0x7fe5f009d6f0 2023-09-30T22:18:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.819+0000 7fe5e67fc700 1 -- 172.21.15.89:0/2991339211 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5ec003b90 con 0x7fe5f009d6f0 2023-09-30T22:18:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.819+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f0074ea0 con 0x7fe5f009d6f0 2023-09-30T22:18:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.819+0000 7fe5e67fc700 1 -- 172.21.15.89:0/2991339211 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe5ec03ec90 con 0x7fe5f009d6f0 2023-09-30T22:18:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:16.827+0000 7fe5e67fc700 1 -- 172.21.15.89:0/2991339211 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5ec003e40 con 0x7fe5f009d6f0 2023-09-30T22:18:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.004+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1958074ea0 con 0x7f195806eec0 2023-09-30T22:18:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.004+0000 7f193ffff700 1 -- 172.21.15.89:0/3432796531 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f194c015020 con 0x7f195806eec0 2023-09-30T22:18:17.005 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.007+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 msgr2=0x7f195811be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.007+0000 7f195dccc700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f195811be90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f194c00b560 tx=0x7f194c00b920 comp rx=0 tx=0).stop 2023-09-30T22:18:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.008+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 shutdown_connections 2023-09-30T22:18:17.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.008+0000 7f195dccc700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1958130670 0x7f195811c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.008+0000 7f195dccc700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f195806e570 0x7f195811b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.008+0000 7f195dccc700 1 --2- 172.21.15.89:0/3432796531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f195806eec0 0x7f195811be90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.008+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 >> 172.21.15.89:0/3432796531 conn(0x7f19580a2aa0 msgr2=0x7f19580a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:17.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.008+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 shutdown_connections 2023-09-30T22:18:17.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.008+0000 7f195dccc700 1 -- 172.21.15.89:0/3432796531 wait complete. 2023-09-30T22:18:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.009+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa5d4074ea0 con 0x7fa5d409d6f0 2023-09-30T22:18:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.010+0000 7fa5ca7fc700 1 -- 172.21.15.89:0/2591005121 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5d0021070 con 0x7fa5d409d6f0 2023-09-30T22:18:17.010 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:17.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.011+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9644074ea0 con 0x7f964409d6f0 2023-09-30T22:18:17.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.011+0000 7f963affd700 1 -- 172.21.15.89:0/3838043172 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9640028070 con 0x7f964409d6f0 2023-09-30T22:18:17.012 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.013+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 msgr2=0x7fa5d41a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.013+0000 7fa5dc211700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d41a5fd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e9b0 tx=0x7fa5d000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.014+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 shutdown_connections 2023-09-30T22:18:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.014+0000 7fa5dc211700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5d409e410 0x7fa5d41a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.014+0000 7fa5dc211700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5d409d6f0 0x7fa5d41a5fd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.014+0000 7fa5dc211700 1 --2- 172.21.15.89:0/2591005121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5d409e000 0x7fa5d41a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.014+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 >> 172.21.15.89:0/2591005121 conn(0x7fa5d4066f50 msgr2=0x7fa5d41baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 shutdown_connections 2023-09-30T22:18:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7fa5dc211700 1 -- 172.21.15.89:0/2591005121 wait complete. 2023-09-30T22:18:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409d6f0 msgr2=0x7f964411a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7f964c361700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409d6f0 0x7f964411a2b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f964000b470 tx=0x7f964000b780 comp rx=0 tx=0).stop 2023-09-30T22:18:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 shutdown_connections 2023-09-30T22:18:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7f964c361700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9644135e50 0x7f96441ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7f964c361700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f964409d6f0 0x7f964411a2b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7f964c361700 1 --2- 172.21.15.89:0/3838043172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964409e1d0 0x7f964411a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.015+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 >> 172.21.15.89:0/3838043172 conn(0x7f9644066f50 msgr2=0x7f9644136a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.016+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 shutdown_connections 2023-09-30T22:18:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.016+0000 7f964c361700 1 -- 172.21.15.89:0/3838043172 wait complete. 2023-09-30T22:18:17.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.031+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe5f01365f0 con 0x7fe5f009d6f0 2023-09-30T22:18:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.031+0000 7fe5e67fc700 1 -- 172.21.15.89:0/2991339211 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5ec017070 con 0x7fe5f009d6f0 2023-09-30T22:18:17.032 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:17.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009d6f0 msgr2=0x7fe5f0120720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 --2- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009d6f0 0x7fe5f0120720 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00c4a0 tx=0x7fe5ec00c7b0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 shutdown_connections 2023-09-30T22:18:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 --2- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5f0139170 0x7fe5f01211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 --2- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5f009d6f0 0x7fe5f0120720 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 --2- 172.21.15.89:0/2991339211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5f009e1d0 0x7fe5f0120c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 >> 172.21.15.89:0/2991339211 conn(0x7fe5f0066f50 msgr2=0x7fe5f0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 shutdown_connections 2023-09-30T22:18:17.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:17.034+0000 7fe5f7da9700 1 -- 172.21.15.89:0/2991339211 wait complete. 2023-09-30T22:18:17.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:18.028 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:18.033 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:18.038 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:18.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.179+0000 7fea90471700 1 Processor -- start 2023-09-30T22:18:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.179+0000 7fea90471700 1 -- start start 2023-09-30T22:18:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.180+0000 7fea90471700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.180+0000 7fea90471700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a1340 0x7fea8813c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.180+0000 7fea90471700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8813c640 0x7fea8813ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.180+0000 7fea90471700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8809d920 con 0x7fea880a0a30 2023-09-30T22:18:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.180+0000 7fea90471700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8809d020 con 0x7fea880a1340 2023-09-30T22:18:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.180+0000 7fea90471700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea880a2010 con 0x7fea8813c640 2023-09-30T22:18:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8813c640 0x7fea8813ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8e20d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8813c640 0x7fea8813ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43324/0 (socket says 172.21.15.89:43324) 2023-09-30T22:18:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8e20d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42890/0 (socket says 172.21.15.89:42890) 2023-09-30T22:18:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 -- 172.21.15.89:0/4091650352 learned_addr learned my addr 172.21.15.89:0/4091650352 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8da0c700 1 --2- 172.21.15.89:0/4091650352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a1340 0x7fea8813c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 -- 172.21.15.89:0/4091650352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a1340 msgr2=0x7fea8813c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 --2- 172.21.15.89:0/4091650352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a1340 0x7fea8813c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 -- 172.21.15.89:0/4091650352 >> v2:172.21.15.89:3300/0 conn(0x7fea880a0a30 msgr2=0x7fea880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 --2- 172.21.15.89:0/4091650352 >> v2:172.21.15.89:3300/0 conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 -- 172.21.15.89:0/4091650352 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8809dbe0 con 0x7fea8813c640 2023-09-30T22:18:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8e20d700 1 --2- 172.21.15.89:0/4091650352 >> v2:172.21.15.89:3300/0 conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8ea0e700 1 --2- 172.21.15.89:0/4091650352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8813c640 0x7fea8813ea20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fea80009e50 tx=0x7fea8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0d28729581a048 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8d20b700 1 -- 172.21.15.89:0/4091650352 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8000e070 con 0x7fea8813c640 2023-09-30T22:18:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8d20b700 1 -- 172.21.15.89:0/4091650352 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8000b910 con 0x7fea8813c640 2023-09-30T22:18:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.181+0000 7fea8d20b700 1 -- 172.21.15.89:0/4091650352 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8000bcc0 con 0x7fea8813c640 2023-09-30T22:18:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.182+0000 7fea90471700 1 -- 172.21.15.89:0/4091650352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8813c640 msgr2=0x7fea8813ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.182+0000 7fea90471700 1 --2- 172.21.15.89:0/4091650352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8813c640 0x7fea8813ea20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fea80009e50 tx=0x7fea8000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.182+0000 7fea90471700 1 -- 172.21.15.89:0/4091650352 shutdown_connections 2023-09-30T22:18:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.182+0000 7fea90471700 1 --2- 172.21.15.89:0/4091650352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8813c640 0x7fea8813ea20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.182+0000 7fea90471700 1 --2- 172.21.15.89:0/4091650352 >> v2:172.21.15.89:3300/0 conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.182+0000 7fea90471700 1 --2- 172.21.15.89:0/4091650352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a1340 0x7fea8813c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.183+0000 7fea90471700 1 -- 172.21.15.89:0/4091650352 >> 172.21.15.89:0/4091650352 conn(0x7fea880a6550 msgr2=0x7fea880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.183+0000 7fea90471700 1 -- 172.21.15.89:0/4091650352 shutdown_connections 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.183+0000 7fea90471700 1 -- 172.21.15.89:0/4091650352 wait complete. 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.184+0000 7fea90471700 1 Processor -- start 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.184+0000 7fea90471700 1 -- start start 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea90471700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a0a30 0x7fea88126ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea90471700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea880a1340 0x7fea88127400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea90471700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8813c640 0x7fea88127940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea90471700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea88128390 con 0x7fea8813c640 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea90471700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8812a9e0 con 0x7fea880a0a30 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea90471700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea88128ad0 con 0x7fea880a1340 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8e20d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a0a30 0x7fea88126ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8e20d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a0a30 0x7fea88126ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36338/0 (socket says 172.21.15.89:36338) 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8e20d700 1 -- 172.21.15.89:0/3242614045 learned_addr learned my addr 172.21.15.89:0/3242614045 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8da0c700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea880a1340 0x7fea88127400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8ea0e700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8813c640 0x7fea88127940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8da0c700 1 -- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a0a30 msgr2=0x7fea88126ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8da0c700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a0a30 0x7fea88126ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8da0c700 1 -- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8813c640 msgr2=0x7fea88127940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8da0c700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8813c640 0x7fea88127940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8da0c700 1 -- 172.21.15.89:0/3242614045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840094a0 con 0x7fea880a1340 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8ea0e700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8813c640 0x7fea88127940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.185+0000 7fea8da0c700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea880a1340 0x7fea88127400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fea7800e930 tx=0x7fea7800ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.186+0000 7fea7effd700 1 -- 172.21.15.89:0/3242614045 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800ee80 con 0x7fea880a1340 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.186+0000 7fea7effd700 1 -- 172.21.15.89:0/3242614045 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800c850 con 0x7fea880a1340 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.186+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea80009b30 con 0x7fea880a1340 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.186+0000 7fea7effd700 1 -- 172.21.15.89:0/3242614045 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800cbc0 con 0x7fea880a1340 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.186+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8812a440 con 0x7fea880a1340 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.187+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea88071e20 con 0x7fea880a1340 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.187+0000 7fea7effd700 1 -- 172.21.15.89:0/3242614045 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea7800cd20 con 0x7fea880a1340 2023-09-30T22:18:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.188+0000 7fea7effd700 1 -- 172.21.15.89:0/3242614045 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea7803fdf0 con 0x7fea880a1340 2023-09-30T22:18:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.195+0000 7fea7effd700 1 -- 172.21.15.89:0/3242614045 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea7801b3b0 con 0x7fea880a1340 2023-09-30T22:18:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.208+0000 7f1e99cd3700 1 Processor -- start 2023-09-30T22:18:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.208+0000 7f1e99cd3700 1 -- start start 2023-09-30T22:18:18.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e99cd3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e9409d6f0 0x7f1e9409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e99cd3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409e000 0x7f1e94130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e99cd3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9409f130 con 0x7f1e9409e410 2023-09-30T22:18:18.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e99cd3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9409f520 con 0x7f1e9409d6f0 2023-09-30T22:18:18.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e99cd3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9409e930 con 0x7f1e9409e000 2023-09-30T22:18:18.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e937fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e9409d6f0 0x7f1e9409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e92ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409e000 0x7f1e94130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e937fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e9409d6f0 0x7f1e9409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42904/0 (socket says 172.21.15.89:42904) 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.209+0000 7f1e937fe700 1 -- 172.21.15.89:0/2355882938 learned_addr learned my addr 172.21.15.89:0/2355882938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e937fe700 1 -- 172.21.15.89:0/2355882938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409e000 msgr2=0x7f1e94130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e937fe700 1 --2- 172.21.15.89:0/2355882938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409e000 0x7f1e94130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e937fe700 1 -- 172.21.15.89:0/2355882938 >> v1:172.21.15.89:6789/0 conn(0x7f1e9409e410 legacy=0x7f1e94130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e927fc700 1 -- 172.21.15.89:0/2355882938 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092122334 0 0) 0x7f1e9409f130 con 0x7f1e9409e410 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e937fe700 1 -- 172.21.15.89:0/2355882938 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9409b2d0 con 0x7f1e9409d6f0 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e92ffd700 1 --2- 172.21.15.89:0/2355882938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409e000 0x7f1e94130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e937fe700 1 --2- 172.21.15.89:0/2355882938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409d6f0 0x7f1e9409dac0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1e840098f0 tx=0x7f1e84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145757cb932cd9d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:18.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e927fc700 1 -- 172.21.15.89:0/2355882938 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84011830 con 0x7f1e9409d6f0 2023-09-30T22:18:18.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e927fc700 1 -- 172.21.15.89:0/2355882938 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e84011990 con 0x7f1e9409d6f0 2023-09-30T22:18:18.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.210+0000 7f1e927fc700 1 -- 172.21.15.89:0/2355882938 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84011d20 con 0x7f1e9409d6f0 2023-09-30T22:18:18.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f1e99cd3700 1 -- 172.21.15.89:0/2355882938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409d6f0 msgr2=0x7f1e9409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f1e99cd3700 1 --2- 172.21.15.89:0/2355882938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409d6f0 0x7f1e9409dac0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1e840098f0 tx=0x7f1e84009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:18.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f1e99cd3700 1 -- 172.21.15.89:0/2355882938 shutdown_connections 2023-09-30T22:18:18.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f1e99cd3700 1 --2- 172.21.15.89:0/2355882938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409d6f0 0x7f1e9409dac0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f1e99cd3700 1 --2- 172.21.15.89:0/2355882938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409e000 0x7f1e94130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f1e99cd3700 1 -- 172.21.15.89:0/2355882938 >> 172.21.15.89:0/2355882938 conn(0x7f1e94066f50 msgr2=0x7f1e9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:18.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f3b4bac2700 1 Processor -- start 2023-09-30T22:18:18.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f1e99cd3700 1 -- 172.21.15.89:0/2355882938 shutdown_connections 2023-09-30T22:18:18.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f1e99cd3700 1 -- 172.21.15.89:0/2355882938 wait complete. 2023-09-30T22:18:18.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.211+0000 7f3b4bac2700 1 -- start start 2023-09-30T22:18:18.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f1e99cd3700 1 Processor -- start 2023-09-30T22:18:18.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f1e99cd3700 1 -- start start 2023-09-30T22:18:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f3b4bac2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4406e570 0x7f3b4406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f3b4bac2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3b4406ee80 0x7f3b44134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f3b4bac2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b44070ee0 con 0x7f3b4409eca0 2023-09-30T22:18:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f3b4bac2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b44071480 con 0x7f3b4406ee80 2023-09-30T22:18:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f3b4bac2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44071b90 con 0x7f3b4406e570 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4a05f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3b4406ee80 0x7f3b44134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4a05f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3b4406ee80 0x7f3b44134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42912/0 (socket says 172.21.15.89:42912) 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4a05f700 1 -- 172.21.15.89:0/1672401705 learned_addr learned my addr 172.21.15.89:0/1672401705 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f1e99cd3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409d6f0 0x7f1e94136380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f1e99cd3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409e000 0x7f1e941368c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.212+0000 7f1e99cd3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e9409e410 0x7f1e94136e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e99cd3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9409c700 con 0x7f1e9409e000 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e99cd3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940a1d00 con 0x7f1e9409e410 2023-09-30T22:18:18.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e99cd3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94139900 con 0x7f1e9409d6f0 2023-09-30T22:18:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409d6f0 0x7f1e94136380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409d6f0 0x7f1e94136380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43356/0 (socket says 172.21.15.89:43356) 2023-09-30T22:18:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 -- 172.21.15.89:0/3271330918 learned_addr learned my addr 172.21.15.89:0/3271330918 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4985e700 1 --2- 172.21.15.89:0/1672401705 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4406e570 0x7f3b4406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e93fff700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e9409e410 0x7f1e94136e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e92ffd700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409e000 0x7f1e941368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4a05f700 1 -- 172.21.15.89:0/1672401705 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4406e570 msgr2=0x7f3b4406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4a05f700 1 --2- 172.21.15.89:0/1672401705 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4406e570 0x7f3b4406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4a05f700 1 -- 172.21.15.89:0/1672401705 >> v1:172.21.15.89:6789/0 conn(0x7f3b4409eca0 legacy=0x7f3b44134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4a05f700 1 -- 172.21.15.89:0/1672401705 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b440711a0 con 0x7f3b4406ee80 2023-09-30T22:18:18.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 -- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e9409e410 msgr2=0x7f1e94136e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e9409e410 0x7f1e94136e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 -- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409e000 msgr2=0x7f1e941368c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409e000 0x7f1e941368c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 -- 172.21.15.89:0/3271330918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880094a0 con 0x7f1e9409d6f0 2023-09-30T22:18:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e937fe700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409d6f0 0x7f1e94136380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1e9409b9e0 tx=0x7f1e84004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f3b4a05f700 1 --2- 172.21.15.89:0/1672401705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406ee80 0x7f3b44134d20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009e50 tx=0x7f3b3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2678d47cf70dd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:18.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e7bfff700 1 -- 172.21.15.89:0/3271330918 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840036a0 con 0x7f1e9409d6f0 2023-09-30T22:18:18.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e7bfff700 1 -- 172.21.15.89:0/3271330918 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8401f070 con 0x7f1e9409d6f0 2023-09-30T22:18:18.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.213+0000 7f1e7bfff700 1 -- 172.21.15.89:0/3271330918 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84003a40 con 0x7f1e9409d6f0 2023-09-30T22:18:18.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.214+0000 7f3b4885c700 1 -- 172.21.15.89:0/1672401705 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00e070 con 0x7f3b4406ee80 2023-09-30T22:18:18.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.214+0000 7f3b4885c700 1 -- 172.21.15.89:0/1672401705 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3c00b910 con 0x7f3b4406ee80 2023-09-30T22:18:18.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.214+0000 7f3b4885c700 1 -- 172.21.15.89:0/1672401705 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00bcc0 con 0x7f3b4406ee80 2023-09-30T22:18:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.214+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e84009550 con 0x7f1e9409d6f0 2023-09-30T22:18:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.214+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e941384e0 con 0x7f1e9409d6f0 2023-09-30T22:18:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.214+0000 7f1e7bfff700 1 -- 172.21.15.89:0/3271330918 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e84003ba0 con 0x7f1e9409d6f0 2023-09-30T22:18:18.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.215+0000 7f1e7bfff700 1 -- 172.21.15.89:0/3271330918 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e84021070 con 0x7f1e9409d6f0 2023-09-30T22:18:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.214+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1672401705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406ee80 msgr2=0x7f3b44134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.214+0000 7f3b4bac2700 1 --2- 172.21.15.89:0/1672401705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406ee80 0x7f3b44134d20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009e50 tx=0x7f3b3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:18.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.215+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e94074ea0 con 0x7f1e9409d6f0 2023-09-30T22:18:18.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.215+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1672401705 shutdown_connections 2023-09-30T22:18:18.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.215+0000 7f3b4bac2700 1 --2- 172.21.15.89:0/1672401705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406ee80 0x7f3b44134d20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.215+0000 7f3b4bac2700 1 --2- 172.21.15.89:0/1672401705 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4406e570 0x7f3b4406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.215+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1672401705 >> 172.21.15.89:0/1672401705 conn(0x7f3b440a2aa0 msgr2=0x7f3b440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:18.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.216+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1672401705 shutdown_connections 2023-09-30T22:18:18.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.218+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1672401705 wait complete. 2023-09-30T22:18:18.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.219+0000 7f3b4bac2700 1 Processor -- start 2023-09-30T22:18:18.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.219+0000 7f3b4bac2700 1 -- start start 2023-09-30T22:18:18.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.219+0000 7f3b4bac2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406e570 0x7f3b44123170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.219+0000 7f3b4bac2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4406ee80 0x7f3b441236b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.219+0000 7f3b4bac2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b44123bf0 0x7f3b441cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4bac2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44124640 con 0x7f3b4406e570 2023-09-30T22:18:18.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4bac2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44126c90 con 0x7f3b44123bf0 2023-09-30T22:18:18.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4bac2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44124e20 con 0x7f3b4406ee80 2023-09-30T22:18:18.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f1e7bfff700 1 -- 172.21.15.89:0/3271330918 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e8402b080 con 0x7f1e9409d6f0 2023-09-30T22:18:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4985e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406e570 0x7f3b44123170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4985e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406e570 0x7f3b44123170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42934/0 (socket says 172.21.15.89:42934) 2023-09-30T22:18:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4985e700 1 -- 172.21.15.89:0/1267003949 learned_addr learned my addr 172.21.15.89:0/1267003949 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4905d700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4406ee80 0x7f3b441236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4a05f700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b44123bf0 0x7f3b441cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4905d700 1 -- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b44123bf0 msgr2=0x7f3b441cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4905d700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b44123bf0 0x7f3b441cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4905d700 1 -- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406e570 msgr2=0x7f3b44123170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4905d700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406e570 0x7f3b44123170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4905d700 1 -- 172.21.15.89:0/1267003949 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c009b30 con 0x7f3b4406ee80 2023-09-30T22:18:18.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4985e700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406e570 0x7f3b44123170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:18.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.220+0000 7f3b4a05f700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b44123bf0 0x7f3b441cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:18.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.221+0000 7f3b4905d700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4406ee80 0x7f3b441236b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b44070e10 tx=0x7f3b3400c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:18.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.221+0000 7f3b3a7fc700 1 -- 172.21.15.89:0/1267003949 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b340052d0 con 0x7f3b4406ee80 2023-09-30T22:18:18.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.221+0000 7f3b3a7fc700 1 -- 172.21.15.89:0/1267003949 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b34005430 con 0x7f3b4406ee80 2023-09-30T22:18:18.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.221+0000 7f3b3a7fc700 1 -- 172.21.15.89:0/1267003949 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b340038c0 con 0x7f3b4406ee80 2023-09-30T22:18:18.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.222+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b44135a90 con 0x7f3b4406ee80 2023-09-30T22:18:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.222+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b441266f0 con 0x7f3b4406ee80 2023-09-30T22:18:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.222+0000 7f3b3a7fc700 1 -- 172.21.15.89:0/1267003949 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3b34003a20 con 0x7f3b4406ee80 2023-09-30T22:18:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.223+0000 7f3b3a7fc700 1 -- 172.21.15.89:0/1267003949 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3b3403e970 con 0x7f3b4406ee80 2023-09-30T22:18:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.224+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b44074ea0 con 0x7f3b4406ee80 2023-09-30T22:18:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7f3b3a7fc700 1 -- 172.21.15.89:0/1267003949 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3b34019d20 con 0x7f3b4406ee80 2023-09-30T22:18:18.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.228+0000 7fbda0093700 1 Processor -- start 2023-09-30T22:18:18.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.228+0000 7fbda0093700 1 -- start start 2023-09-30T22:18:18.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbda0093700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd9809d6f0 0x7fbd9809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbda0093700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e000 0x7fbd98130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbda0093700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e410 0x7fbd98130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbda0093700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9809f520 con 0x7fbd9809d6f0 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbda0093700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9809f130 con 0x7fbd9809e000 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbda0093700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9809e930 con 0x7fbd9809e410 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbd9d62e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e000 0x7fbd98130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbd9d62e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e000 0x7fbd98130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36358/0 (socket says 172.21.15.89:36358) 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbd9d62e700 1 -- 172.21.15.89:0/2104396287 learned_addr learned my addr 172.21.15.89:0/2104396287 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbd9d62e700 1 -- 172.21.15.89:0/2104396287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e410 msgr2=0x7fbd98130ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbd9d62e700 1 --2- 172.21.15.89:0/2104396287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e410 0x7fbd98130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbd9d62e700 1 -- 172.21.15.89:0/2104396287 >> v2:172.21.15.89:3300/0 conn(0x7fbd9809d6f0 msgr2=0x7fbd9809dac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:18:18.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbd9d62e700 1 --2- 172.21.15.89:0/2104396287 >> v2:172.21.15.89:3300/0 conn(0x7fbd9809d6f0 0x7fbd9809dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.229+0000 7fbd9d62e700 1 -- 172.21.15.89:0/2104396287 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9809b2d0 con 0x7fbd9809e000 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.230+0000 7fbd9d62e700 1 --2- 172.21.15.89:0/2104396287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e000 0x7fbd98130660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbd940098f0 tx=0x7fbd94009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d74d8624ebac52 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.230+0000 7fbd9ce2d700 1 -- 172.21.15.89:0/2104396287 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400b8a0 con 0x7fbd9809e000 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.230+0000 7fbd9ce2d700 1 -- 172.21.15.89:0/2104396287 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9400ba00 con 0x7fbd9809e000 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.230+0000 7fbd9ce2d700 1 -- 172.21.15.89:0/2104396287 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400bd70 con 0x7fbd9809e000 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 -- 172.21.15.89:0/2104396287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e000 msgr2=0x7fbd98130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 --2- 172.21.15.89:0/2104396287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e000 0x7fbd98130660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbd940098f0 tx=0x7fbd94009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 -- 172.21.15.89:0/2104396287 shutdown_connections 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 --2- 172.21.15.89:0/2104396287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e410 0x7fbd98130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 --2- 172.21.15.89:0/2104396287 >> v2:172.21.15.89:3300/0 conn(0x7fbd9809d6f0 0x7fbd9809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 --2- 172.21.15.89:0/2104396287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e000 0x7fbd98130660 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 -- 172.21.15.89:0/2104396287 >> 172.21.15.89:0/2104396287 conn(0x7fbd98066f50 msgr2=0x7fbd9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 -- 172.21.15.89:0/2104396287 shutdown_connections 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.231+0000 7fbda0093700 1 -- 172.21.15.89:0/2104396287 wait complete. 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.232+0000 7fbda0093700 1 Processor -- start 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.232+0000 7fbda0093700 1 -- start start 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbda0093700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9809d6f0 0x7fbd9811f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbda0093700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e000 0x7fbd9811f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbda0093700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e410 0x7fbd9811fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbda0093700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9806e840 con 0x7fbd9809d6f0 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbda0093700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9809c700 con 0x7fbd9809e410 2023-09-30T22:18:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbda0093700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd981216f0 con 0x7fbd9809e000 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9d62e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e000 0x7fbd9811f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9d62e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e000 0x7fbd9811f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43380/0 (socket says 172.21.15.89:43380) 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9d62e700 1 -- 172.21.15.89:0/4050566703 learned_addr learned my addr 172.21.15.89:0/4050566703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9e630700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e410 0x7fbd9811fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9d62e700 1 -- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e410 msgr2=0x7fbd9811fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9de2f700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9809d6f0 0x7fbd9811f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9d62e700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e410 0x7fbd9811fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9d62e700 1 -- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9809d6f0 msgr2=0x7fbd9811f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9d62e700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9809d6f0 0x7fbd9811f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9d62e700 1 -- 172.21.15.89:0/4050566703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94009550 con 0x7fbd9809e000 2023-09-30T22:18:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9e630700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e410 0x7fbd9811fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.233+0000 7fbd9de2f700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9809d6f0 0x7fbd9811f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.234+0000 7fbd9d62e700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e000 0x7fbd9811f7c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd94000c00 tx=0x7fbd94004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.234+0000 7fbd8a7fc700 1 -- 172.21.15.89:0/4050566703 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd940045d0 con 0x7fbd9809e000 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.234+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd98120fa0 con 0x7fbd9809e000 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.234+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd98122b20 con 0x7fbd9809e000 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.234+0000 7fbd8a7fc700 1 -- 172.21.15.89:0/4050566703 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd940036a0 con 0x7fbd9809e000 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.234+0000 7fbd8a7fc700 1 -- 172.21.15.89:0/4050566703 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94003a50 con 0x7fbd9809e000 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.235+0000 7fbd8a7fc700 1 -- 172.21.15.89:0/4050566703 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd94003bb0 con 0x7fbd9809e000 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.235+0000 7fbd8a7fc700 1 -- 172.21.15.89:0/4050566703 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbd94042b60 con 0x7fbd9809e000 2023-09-30T22:18:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.235+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd800052e0 con 0x7fbd9809e000 2023-09-30T22:18:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.241+0000 7fbd8a7fc700 1 -- 172.21.15.89:0/4050566703 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd94003e60 con 0x7fbd9809e000 2023-09-30T22:18:18.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.431+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1e94131200 con 0x7f1e9409d6f0 2023-09-30T22:18:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.432+0000 7f1e7bfff700 1 -- 172.21.15.89:0/3271330918 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e84015020 con 0x7f1e9409d6f0 2023-09-30T22:18:18.433 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409d6f0 msgr2=0x7f1e94136380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409d6f0 0x7f1e94136380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1e9409b9e0 tx=0x7f1e84004750 comp rx=0 tx=0).stop 2023-09-30T22:18:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 shutdown_connections 2023-09-30T22:18:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e9409e410 0x7f1e94136e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e9409d6f0 0x7f1e94136380 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 --2- 172.21.15.89:0/3271330918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e9409e000 0x7f1e941368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 >> 172.21.15.89:0/3271330918 conn(0x7f1e94066f50 msgr2=0x7f1e94070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 shutdown_connections 2023-09-30T22:18:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.434+0000 7f1e99cd3700 1 -- 172.21.15.89:0/3271330918 wait complete. 2023-09-30T22:18:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.439+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3b44126880 con 0x7f3b4406ee80 2023-09-30T22:18:18.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.439+0000 7f3b3a7fc700 1 -- 172.21.15.89:0/1267003949 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b3401ae80 con 0x7f3b4406ee80 2023-09-30T22:18:18.440 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:18.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.441+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4406ee80 msgr2=0x7f3b441236b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.441+0000 7f3b4bac2700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4406ee80 0x7f3b441236b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b44070e10 tx=0x7f3b3400c730 comp rx=0 tx=0).stop 2023-09-30T22:18:18.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.441+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 shutdown_connections 2023-09-30T22:18:18.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.441+0000 7f3b4bac2700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b44123bf0 0x7f3b441cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.441+0000 7f3b4bac2700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4406e570 0x7f3b44123170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.441+0000 7f3b4bac2700 1 --2- 172.21.15.89:0/1267003949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4406ee80 0x7f3b441236b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.441+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 >> 172.21.15.89:0/1267003949 conn(0x7f3b440a2aa0 msgr2=0x7f3b440a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:18.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.441+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 shutdown_connections 2023-09-30T22:18:18.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.442+0000 7f3b4bac2700 1 -- 172.21.15.89:0/1267003949 wait complete. 2023-09-30T22:18:18.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:18.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:18.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.505+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fea8809e5a0 con 0x7fea880a1340 2023-09-30T22:18:18.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.505+0000 7fea7effd700 1 -- 172.21.15.89:0/3242614045 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea78021070 con 0x7fea880a1340 2023-09-30T22:18:18.506 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea880a1340 msgr2=0x7fea88127400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea880a1340 0x7fea88127400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fea7800e930 tx=0x7fea7800ec40 comp rx=0 tx=0).stop 2023-09-30T22:18:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 shutdown_connections 2023-09-30T22:18:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8813c640 0x7fea88127940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea880a0a30 0x7fea88126ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 --2- 172.21.15.89:0/3242614045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea880a1340 0x7fea88127400 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 >> 172.21.15.89:0/3242614045 conn(0x7fea880a6550 msgr2=0x7fea8806da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 shutdown_connections 2023-09-30T22:18:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.508+0000 7fea90471700 1 -- 172.21.15.89:0/3242614045 wait complete. 2023-09-30T22:18:18.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:18.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.537+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbd80005590 con 0x7fbd9809e000 2023-09-30T22:18:18.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.537+0000 7fbd8a7fc700 1 -- 172.21.15.89:0/4050566703 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd94037090 con 0x7fbd9809e000 2023-09-30T22:18:18.538 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e000 msgr2=0x7fbd9811f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e000 0x7fbd9811f7c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd94000c00 tx=0x7fbd94004750 comp rx=0 tx=0).stop 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 shutdown_connections 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd9809e410 0x7fbd9811fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9809d6f0 0x7fbd9811f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 --2- 172.21.15.89:0/4050566703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd9809e000 0x7fbd9811f7c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 >> 172.21.15.89:0/4050566703 conn(0x7fbd98066f50 msgr2=0x7fbd98119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 shutdown_connections 2023-09-30T22:18:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:18.539+0000 7fbda0093700 1 -- 172.21.15.89:0/4050566703 wait complete. 2023-09-30T22:18:18.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:19.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:19.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:19.521 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:19.553 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.583+0000 7f94e4bb3700 1 Processor -- start 2023-09-30T22:18:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.583+0000 7f94e4bb3700 1 -- start start 2023-09-30T22:18:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94e4bb3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94e009d6f0 0x7f94e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94e4bb3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94e4bb3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94e4bb3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e006b460 con 0x7f94e009d6f0 2023-09-30T22:18:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94e4bb3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e009b120 con 0x7f94e009e410 2023-09-30T22:18:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94e4bb3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94e009a7e0 con 0x7f94e009e000 2023-09-30T22:18:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94de59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94e009d6f0 0x7f94e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:43384/0 (socket says 172.21.15.89:43384) 2023-09-30T22:18:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 -- 172.21.15.89:0/1148859743 learned_addr learned my addr 172.21.15.89:0/1148859743 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ded9d700 1 --2- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 -- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 msgr2=0x7f94e0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 --2- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 -- 172.21.15.89:0/1148859743 >> v2:172.21.15.89:3300/0 conn(0x7f94e009d6f0 msgr2=0x7f94e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 --2- 172.21.15.89:0/1148859743 >> v2:172.21.15.89:3300/0 conn(0x7f94e009d6f0 0x7f94e009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 -- 172.21.15.89:0/1148859743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e009a500 con 0x7f94e009e000 2023-09-30T22:18:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ded9d700 1 --2- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.584+0000 7f94ddd9b700 1 --2- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01339f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009920 tx=0x7f94d4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c7aaab060d294b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.585+0000 7f94dd59a700 1 -- 172.21.15.89:0/1148859743 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4011830 con 0x7f94e009e000 2023-09-30T22:18:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.585+0000 7f94dd59a700 1 -- 172.21.15.89:0/1148859743 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d4011990 con 0x7f94e009e000 2023-09-30T22:18:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.585+0000 7f94dd59a700 1 -- 172.21.15.89:0/1148859743 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4011d20 con 0x7f94e009e000 2023-09-30T22:18:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.585+0000 7f94e4bb3700 1 -- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 msgr2=0x7f94e01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.585+0000 7f94e4bb3700 1 --2- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01339f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009920 tx=0x7f94d4009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.586+0000 7f94e4bb3700 1 -- 172.21.15.89:0/1148859743 shutdown_connections 2023-09-30T22:18:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.586+0000 7f94e4bb3700 1 --2- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.586+0000 7f94e4bb3700 1 --2- 172.21.15.89:0/1148859743 >> v2:172.21.15.89:3300/0 conn(0x7f94e009d6f0 0x7f94e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.586+0000 7f94e4bb3700 1 --2- 172.21.15.89:0/1148859743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01339f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.586+0000 7f94e4bb3700 1 -- 172.21.15.89:0/1148859743 >> 172.21.15.89:0/1148859743 conn(0x7f94e0066f50 msgr2=0x7f94e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.586+0000 7f94e4bb3700 1 -- 172.21.15.89:0/1148859743 shutdown_connections 2023-09-30T22:18:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.586+0000 7f94e4bb3700 1 -- 172.21.15.89:0/1148859743 wait complete. 2023-09-30T22:18:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.587+0000 7f94e4bb3700 1 Processor -- start 2023-09-30T22:18:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.587+0000 7f94e4bb3700 1 -- start start 2023-09-30T22:18:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.587+0000 7f94e4bb3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e009d6f0 0x7f94e01a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.587+0000 7f94e4bb3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.587+0000 7f94e4bb3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e01a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.587+0000 7f94e4bb3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e01a95b0 con 0x7f94e009d6f0 2023-09-30T22:18:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.587+0000 7f94e4bb3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e00a2190 con 0x7f94e009e410 2023-09-30T22:18:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.587+0000 7f94e4bb3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94e01a7a90 con 0x7f94e009e000 2023-09-30T22:18:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e009d6f0 0x7f94e01a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e009d6f0 0x7f94e01a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42966/0 (socket says 172.21.15.89:42966) 2023-09-30T22:18:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94de59c700 1 -- 172.21.15.89:0/4294112832 learned_addr learned my addr 172.21.15.89:0/4294112832 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ddd9b700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ded9d700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e01a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ddd9b700 1 -- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 msgr2=0x7f94e01a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ddd9b700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e01a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ddd9b700 1 -- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e009d6f0 msgr2=0x7f94e01a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ddd9b700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e009d6f0 0x7f94e01a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ddd9b700 1 -- 172.21.15.89:0/4294112832 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8009a70 con 0x7f94e009e000 2023-09-30T22:18:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ded9d700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e01a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ddd9b700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01a6370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f94e006b640 tx=0x7f94d4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:19.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ceffd700 1 -- 172.21.15.89:0/4294112832 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4004450 con 0x7f94e009e000 2023-09-30T22:18:19.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94ceffd700 1 -- 172.21.15.89:0/4294112832 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d40036a0 con 0x7f94e009e000 2023-09-30T22:18:19.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.588+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d4009550 con 0x7f94e009e000 2023-09-30T22:18:19.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.589+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e01a8bd0 con 0x7f94e009e000 2023-09-30T22:18:19.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.589+0000 7f94ceffd700 1 -- 172.21.15.89:0/4294112832 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d40045f0 con 0x7f94e009e000 2023-09-30T22:18:19.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.589+0000 7f94ceffd700 1 -- 172.21.15.89:0/4294112832 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94d4003a50 con 0x7f94e009e000 2023-09-30T22:18:19.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.589+0000 7f94ceffd700 1 -- 172.21.15.89:0/4294112832 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f94d4021030 con 0x7f94e009e000 2023-09-30T22:18:19.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.590+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c40052e0 con 0x7f94e009e000 2023-09-30T22:18:19.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.599+0000 7f94ceffd700 1 -- 172.21.15.89:0/4294112832 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94d4015020 con 0x7f94e009e000 2023-09-30T22:18:19.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.640+0000 7f79aedbe700 1 Processor -- start 2023-09-30T22:18:19.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.640+0000 7f79aedbe700 1 -- start start 2023-09-30T22:18:19.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79aedbe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a80a0a30 0x7f79a80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79aedbe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79aedbe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79a8133930 0x7f79a8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79aedbe700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a80a2a70 con 0x7f79a8133930 2023-09-30T22:18:19.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79aedbe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80a01d0 con 0x7f79a809d630 2023-09-30T22:18:19.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79aedbe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79a809e2b0 con 0x7f79a80a0a30 2023-09-30T22:18:19.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79a8133930 0x7f79a8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79a8133930 0x7f79a8133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42968/0 (socket says 172.21.15.89:42968) 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 -- 172.21.15.89:0/749648814 learned_addr learned my addr 172.21.15.89:0/749648814 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79a7fff700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79acb5a700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a80a0a30 0x7f79a80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 -- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a80a0a30 msgr2=0x7f79a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a80a0a30 0x7f79a80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 -- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 msgr2=0x7f79a80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 -- 172.21.15.89:0/749648814 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a809dc50 con 0x7f79a8133930 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79a7fff700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:19.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.641+0000 7f79ad35b700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a8133930 0x7f79a8133dd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009e50 tx=0x7f79a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a326176fa7038945 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.642+0000 7f79a77fe700 1 -- 172.21.15.89:0/749648814 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a000e070 con 0x7f79a8133930 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.642+0000 7f79a77fe700 1 -- 172.21.15.89:0/749648814 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a000b910 con 0x7f79a8133930 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.642+0000 7f79a77fe700 1 -- 172.21.15.89:0/749648814 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a000bcc0 con 0x7f79a8133930 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 -- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a8133930 msgr2=0x7f79a8133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a8133930 0x7f79a8133dd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009e50 tx=0x7f79a000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 -- 172.21.15.89:0/749648814 shutdown_connections 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a8133930 0x7f79a8133dd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a80a0a30 0x7f79a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 --2- 172.21.15.89:0/749648814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 -- 172.21.15.89:0/749648814 >> 172.21.15.89:0/749648814 conn(0x7f79a80a3050 msgr2=0x7f79a80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 -- 172.21.15.89:0/749648814 shutdown_connections 2023-09-30T22:18:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.643+0000 7f79aedbe700 1 -- 172.21.15.89:0/749648814 wait complete. 2023-09-30T22:18:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.644+0000 7f79aedbe700 1 Processor -- start 2023-09-30T22:18:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.644+0000 7f79aedbe700 1 -- start start 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79aedbe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a811f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79aedbe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a80a0a30 0x7f79a811fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79aedbe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a8133930 0x7f79a81c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79aedbe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a81223b0 con 0x7f79a80a0a30 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79aedbe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a811ef40 con 0x7f79a809d630 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79aedbe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8133420 con 0x7f79a8133930 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a80a0a30 0x7f79a811fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a80a0a30 0x7f79a811fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42980/0 (socket says 172.21.15.89:42980) 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79a7fff700 1 -- 172.21.15.89:0/2748410851 learned_addr learned my addr 172.21.15.89:0/2748410851 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79acb5a700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a811f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79ad35b700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a8133930 0x7f79a81c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79a7fff700 1 -- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a8133930 msgr2=0x7f79a81c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79a7fff700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a8133930 0x7f79a81c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79a7fff700 1 -- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 msgr2=0x7f79a811f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.645+0000 7f79a7fff700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a811f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.646+0000 7f79a7fff700 1 -- 172.21.15.89:0/2748410851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79940094a0 con 0x7f79a80a0a30 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.646+0000 7f79acb5a700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a811f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.646+0000 7f79a7fff700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a80a0a30 0x7f79a811fa60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f799400ec10 tx=0x7f799400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.646+0000 7f79a57fa700 1 -- 172.21.15.89:0/2748410851 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799400c850 con 0x7f79a80a0a30 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.646+0000 7f79a57fa700 1 -- 172.21.15.89:0/2748410851 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799400c9b0 con 0x7f79a80a0a30 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.646+0000 7f79a57fa700 1 -- 172.21.15.89:0/2748410851 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799400cd20 con 0x7f79a80a0a30 2023-09-30T22:18:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.646+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a0009b30 con 0x7f79a80a0a30 2023-09-30T22:18:19.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.646+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a80a6e40 con 0x7f79a80a0a30 2023-09-30T22:18:19.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.647+0000 7f79a57fa700 1 -- 172.21.15.89:0/2748410851 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7994013470 con 0x7f79a80a0a30 2023-09-30T22:18:19.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.647+0000 7f79a57fa700 1 -- 172.21.15.89:0/2748410851 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f799403fb90 con 0x7f79a80a0a30 2023-09-30T22:18:19.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.647+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a8120ef0 con 0x7f79a80a0a30 2023-09-30T22:18:19.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.652+0000 7f79a57fa700 1 -- 172.21.15.89:0/2748410851 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7994013720 con 0x7f79a80a0a30 2023-09-30T22:18:19.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.674+0000 7f25b25b6700 1 Processor -- start 2023-09-30T22:18:19.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.674+0000 7f25b25b6700 1 -- start start 2023-09-30T22:18:19.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.674+0000 7f25b25b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac09d9f0 0x7f25ac136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b25b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac136a70 0x7f25ac138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b25b6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac09e130 con 0x7f25ac1316c0 2023-09-30T22:18:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b25b6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1321a0 con 0x7f25ac136a70 2023-09-30T22:18:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b25b6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1312b0 con 0x7f25ac09d9f0 2023-09-30T22:18:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b0b53700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac136a70 0x7f25ac138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b0b53700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac136a70 0x7f25ac138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36402/0 (socket says 172.21.15.89:36402) 2023-09-30T22:18:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b0b53700 1 -- 172.21.15.89:0/593764460 learned_addr learned my addr 172.21.15.89:0/593764460 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25ab7fe700 1 --2- 172.21.15.89:0/593764460 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac09d9f0 0x7f25ac136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b0b53700 1 -- 172.21.15.89:0/593764460 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac09d9f0 msgr2=0x7f25ac136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:19.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b0b53700 1 --2- 172.21.15.89:0/593764460 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac09d9f0 0x7f25ac136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b0b53700 1 -- 172.21.15.89:0/593764460 >> v1:172.21.15.89:6789/0 conn(0x7f25ac1316c0 legacy=0x7f25ac131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b0b53700 1 -- 172.21.15.89:0/593764460 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac09f6e0 con 0x7f25ac136a70 2023-09-30T22:18:19.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.675+0000 7f25b0b53700 1 --2- 172.21.15.89:0/593764460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac136a70 0x7f25ac138e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009e80 tx=0x7f25a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ea26b45b810c0d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:19.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.676+0000 7f25aaffd700 1 -- 172.21.15.89:0/593764460 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400bb10 con 0x7f25ac136a70 2023-09-30T22:18:19.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.676+0000 7f25aaffd700 1 -- 172.21.15.89:0/593764460 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400bc70 con 0x7f25ac136a70 2023-09-30T22:18:19.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.676+0000 7f25aaffd700 1 -- 172.21.15.89:0/593764460 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40104d0 con 0x7f25ac136a70 2023-09-30T22:18:19.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.677+0000 7f25b25b6700 1 -- 172.21.15.89:0/593764460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac136a70 msgr2=0x7f25ac138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.677+0000 7f25b25b6700 1 --2- 172.21.15.89:0/593764460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac136a70 0x7f25ac138e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009e80 tx=0x7f25a400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:19.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.677+0000 7f25b25b6700 1 -- 172.21.15.89:0/593764460 shutdown_connections 2023-09-30T22:18:19.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.677+0000 7f25b25b6700 1 --2- 172.21.15.89:0/593764460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac136a70 0x7f25ac138e50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.677+0000 7f25b25b6700 1 --2- 172.21.15.89:0/593764460 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac09d9f0 0x7f25ac136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.677+0000 7f25b25b6700 1 -- 172.21.15.89:0/593764460 >> 172.21.15.89:0/593764460 conn(0x7f25ac066f50 msgr2=0x7f25ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:19.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.677+0000 7f25b25b6700 1 -- 172.21.15.89:0/593764460 shutdown_connections 2023-09-30T22:18:19.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.677+0000 7f25b25b6700 1 -- 172.21.15.89:0/593764460 wait complete. 2023-09-30T22:18:19.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.678+0000 7f25b25b6700 1 Processor -- start 2023-09-30T22:18:19.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.678+0000 7f25b25b6700 1 -- start start 2023-09-30T22:18:19.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25b25b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ac09d9f0 0x7f25ac0719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25b25b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac1316c0 0x7f25ac071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25b25b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac136a70 0x7f25ac1cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25b25b6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac12b900 con 0x7f25ac09d9f0 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25b25b6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac070380 con 0x7f25ac1316c0 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25b25b6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac06f990 con 0x7f25ac136a70 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25ab7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac1316c0 0x7f25ac071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25ab7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac1316c0 0x7f25ac071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36416/0 (socket says 172.21.15.89:36416) 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25ab7fe700 1 -- 172.21.15.89:0/2852359323 learned_addr learned my addr 172.21.15.89:0/2852359323 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25b0b53700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac136a70 0x7f25ac1cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.679+0000 7f25abfff700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ac09d9f0 0x7f25ac0719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25ab7fe700 1 -- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac136a70 msgr2=0x7f25ac1cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25ab7fe700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac136a70 0x7f25ac1cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25ab7fe700 1 -- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ac09d9f0 msgr2=0x7f25ac0719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25ab7fe700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ac09d9f0 0x7f25ac0719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25ab7fe700 1 -- 172.21.15.89:0/2852359323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a4009b30 con 0x7f25ac1316c0 2023-09-30T22:18:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25ab7fe700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac1316c0 0x7f25ac071f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f25a000e9b0 tx=0x7f25a000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25a8ff9700 1 -- 172.21.15.89:0/2852359323 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0009710 con 0x7f25ac1316c0 2023-09-30T22:18:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25a8ff9700 1 -- 172.21.15.89:0/2852359323 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a000c850 con 0x7f25ac1316c0 2023-09-30T22:18:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25a8ff9700 1 -- 172.21.15.89:0/2852359323 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a000cbc0 con 0x7f25ac1316c0 2023-09-30T22:18:19.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac072850 con 0x7f25ac1316c0 2023-09-30T22:18:19.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.680+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac0710b0 con 0x7f25ac1316c0 2023-09-30T22:18:19.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.681+0000 7f25a8ff9700 1 -- 172.21.15.89:0/2852359323 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f25a000cd20 con 0x7f25ac1316c0 2023-09-30T22:18:19.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.681+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac09c210 con 0x7f25ac1316c0 2023-09-30T22:18:19.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.681+0000 7f25a8ff9700 1 -- 172.21.15.89:0/2852359323 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f25a003f850 con 0x7f25ac1316c0 2023-09-30T22:18:19.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.690+0000 7f25a8ff9700 1 -- 172.21.15.89:0/2852359323 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f25a0021020 con 0x7f25ac1316c0 2023-09-30T22:18:19.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.717+0000 7fef0b5bd700 1 Processor -- start 2023-09-30T22:18:19.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.717+0000 7fef0b5bd700 1 -- start start 2023-09-30T22:18:19.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef0b5bd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409e930 0x7fef0409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef0b5bd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409f240 0x7fef041317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef0b5bd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef04070940 con 0x7fef0409f650 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef0b5bd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0406fd60 con 0x7fef0409f240 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef0b5bd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef0406fa80 con 0x7fef0409e930 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef08b58700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409f240 0x7fef041317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef08b58700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409f240 0x7fef041317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36420/0 (socket says 172.21.15.89:36420) 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef09b5a700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fef0409f650 0x7fef04131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54180/0 (socket says 172.21.15.89:54180) 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef08b58700 1 -- 172.21.15.89:0/581021133 learned_addr learned my addr 172.21.15.89:0/581021133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:19.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef09359700 1 --2- 172.21.15.89:0/581021133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409e930 0x7fef0409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef08b58700 1 -- 172.21.15.89:0/581021133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409e930 msgr2=0x7fef0409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:19.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef08b58700 1 --2- 172.21.15.89:0/581021133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409e930 0x7fef0409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef08b58700 1 -- 172.21.15.89:0/581021133 >> v1:172.21.15.89:6789/0 conn(0x7fef0409f650 legacy=0x7fef04131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7feefbfff700 1 -- 172.21.15.89:0/581021133 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369060019 0 0) 0x7fef04070940 con 0x7fef0409f650 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef08b58700 1 -- 172.21.15.89:0/581021133 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0409fb60 con 0x7fef0409f240 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.718+0000 7fef08b58700 1 --2- 172.21.15.89:0/581021133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409f240 0x7fef041317f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feef4009920 tx=0x7feef4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb2e0d5a8c550a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.719+0000 7feefbfff700 1 -- 172.21.15.89:0/581021133 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4011830 con 0x7fef0409f240 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.719+0000 7feefbfff700 1 -- 172.21.15.89:0/581021133 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef4011990 con 0x7fef0409f240 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.719+0000 7feefbfff700 1 -- 172.21.15.89:0/581021133 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4011d20 con 0x7fef0409f240 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.720+0000 7fef0b5bd700 1 -- 172.21.15.89:0/581021133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409f240 msgr2=0x7fef041317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.720+0000 7fef0b5bd700 1 --2- 172.21.15.89:0/581021133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409f240 0x7fef041317f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feef4009920 tx=0x7feef4009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:19.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.720+0000 7fef0b5bd700 1 -- 172.21.15.89:0/581021133 shutdown_connections 2023-09-30T22:18:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.720+0000 7fef0b5bd700 1 --2- 172.21.15.89:0/581021133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409e930 0x7fef0409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.720+0000 7fef0b5bd700 1 --2- 172.21.15.89:0/581021133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409f240 0x7fef041317f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.720+0000 7fef0b5bd700 1 -- 172.21.15.89:0/581021133 >> 172.21.15.89:0/581021133 conn(0x7fef040a4210 msgr2=0x7fef040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.720+0000 7fef0b5bd700 1 -- 172.21.15.89:0/581021133 shutdown_connections 2023-09-30T22:18:19.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.720+0000 7fef0b5bd700 1 -- 172.21.15.89:0/581021133 wait complete. 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.721+0000 7fef0b5bd700 1 Processor -- start 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.721+0000 7fef0b5bd700 1 -- start start 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef0b5bd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409e930 0x7fef041abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef0b5bd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409f240 0x7fef041ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef0b5bd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef0409f650 0x7fef041aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef0b5bd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef041af120 con 0x7fef0409f650 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef0b5bd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef041ade90 con 0x7fef0409e930 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef0b5bd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef041af380 con 0x7fef0409f240 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef09b5a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef0409f650 0x7fef041aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef09b5a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef0409f650 0x7fef041aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43002/0 (socket says 172.21.15.89:43002) 2023-09-30T22:18:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef09b5a700 1 -- 172.21.15.89:0/1817920137 learned_addr learned my addr 172.21.15.89:0/1817920137 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.722+0000 7fef08b58700 1 --2- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409f240 0x7fef041ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7fef09b5a700 1 -- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409f240 msgr2=0x7fef041ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7fef09b5a700 1 --2- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409f240 0x7fef041ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7fef09b5a700 1 -- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409e930 msgr2=0x7fef041abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7fef09b5a700 1 --2- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409e930 0x7fef041abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7fef09b5a700 1 -- 172.21.15.89:0/1817920137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef4009550 con 0x7fef0409f650 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7fef09b5a700 1 --2- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef0409f650 0x7fef041aca30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feefc00cc10 tx=0x7feefc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7feef9ffb700 1 -- 172.21.15.89:0/1817920137 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc00ab10 con 0x7fef0409f650 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7feef9ffb700 1 -- 172.21.15.89:0/1817920137 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc00ac70 con 0x7fef0409f650 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef040a0190 con 0x7fef0409f650 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7feef9ffb700 1 -- 172.21.15.89:0/1817920137 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0070d0 con 0x7fef0409f650 2023-09-30T22:18:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.723+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef041b00b0 con 0x7fef0409f650 2023-09-30T22:18:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.725+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef041ad3f0 con 0x7fef0409f650 2023-09-30T22:18:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.726+0000 7feef9ffb700 1 -- 172.21.15.89:0/1817920137 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feefc007230 con 0x7fef0409f650 2023-09-30T22:18:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.726+0000 7feef9ffb700 1 -- 172.21.15.89:0/1817920137 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feefc03fbc0 con 0x7fef0409f650 2023-09-30T22:18:19.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.734+0000 7feef9ffb700 1 -- 172.21.15.89:0/1817920137 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feefc0074e0 con 0x7fef0409f650 2023-09-30T22:18:19.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.823+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f94c4005590 con 0x7f94e009e000 2023-09-30T22:18:19.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.823+0000 7f94ceffd700 1 -- 172.21.15.89:0/4294112832 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94d402b070 con 0x7f94e009e000 2023-09-30T22:18:19.824 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:19.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 msgr2=0x7f94e01a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01a6370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f94e006b640 tx=0x7f94d4004750 comp rx=0 tx=0).stop 2023-09-30T22:18:19.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 shutdown_connections 2023-09-30T22:18:19.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94e009e410 0x7f94e01a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94e009d6f0 0x7f94e01a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 --2- 172.21.15.89:0/4294112832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94e009e000 0x7f94e01a6370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 >> 172.21.15.89:0/4294112832 conn(0x7f94e0066f50 msgr2=0x7f94e01bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 shutdown_connections 2023-09-30T22:18:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.825+0000 7f94e4bb3700 1 -- 172.21.15.89:0/4294112832 wait complete. 2023-09-30T22:18:19.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.853+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f79a80747f0 con 0x7f79a80a0a30 2023-09-30T22:18:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.853+0000 7f79a57fa700 1 -- 172.21.15.89:0/2748410851 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7994011b00 con 0x7f79a80a0a30 2023-09-30T22:18:19.854 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:19.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a80a0a30 msgr2=0x7f79a811fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a80a0a30 0x7f79a811fa60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f799400ec10 tx=0x7f799400ef20 comp rx=0 tx=0).stop 2023-09-30T22:18:19.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 shutdown_connections 2023-09-30T22:18:19.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a8133930 0x7f79a81c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a809d630 0x7f79a811f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 --2- 172.21.15.89:0/2748410851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a80a0a30 0x7f79a811fa60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 >> 172.21.15.89:0/2748410851 conn(0x7f79a80a3050 msgr2=0x7f79a80a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:19.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 shutdown_connections 2023-09-30T22:18:19.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.855+0000 7f79aedbe700 1 -- 172.21.15.89:0/2748410851 wait complete. 2023-09-30T22:18:19.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:19.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.887+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f25ac074ea0 con 0x7f25ac1316c0 2023-09-30T22:18:19.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.888+0000 7f25a8ff9700 1 -- 172.21.15.89:0/2852359323 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25a0017070 con 0x7f25ac1316c0 2023-09-30T22:18:19.888 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.889+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac1316c0 msgr2=0x7f25ac071f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.889+0000 7f25b25b6700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac1316c0 0x7f25ac071f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f25a000e9b0 tx=0x7f25a000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.889+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 shutdown_connections 2023-09-30T22:18:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.890+0000 7f25b25b6700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25ac136a70 0x7f25ac1cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.890+0000 7f25b25b6700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ac09d9f0 0x7f25ac0719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.890+0000 7f25b25b6700 1 --2- 172.21.15.89:0/2852359323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25ac1316c0 0x7f25ac071f00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.890+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 >> 172.21.15.89:0/2852359323 conn(0x7f25ac066f50 msgr2=0x7f25ac09a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.890+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 shutdown_connections 2023-09-30T22:18:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.890+0000 7f25b25b6700 1 -- 172.21.15.89:0/2852359323 wait complete. 2023-09-30T22:18:19.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:19.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.928+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fef04074ea0 con 0x7fef0409f650 2023-09-30T22:18:19.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.928+0000 7feef9ffb700 1 -- 172.21.15.89:0/1817920137 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feefc018020 con 0x7fef0409f650 2023-09-30T22:18:19.929 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef0409f650 msgr2=0x7fef041aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 --2- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef0409f650 0x7fef041aca30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feefc00cc10 tx=0x7feefc00cf20 comp rx=0 tx=0).stop 2023-09-30T22:18:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 shutdown_connections 2023-09-30T22:18:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 --2- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef0409f650 0x7fef041aca30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 --2- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef0409e930 0x7fef041abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 --2- 172.21.15.89:0/1817920137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef0409f240 0x7fef041ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 >> 172.21.15.89:0/1817920137 conn(0x7fef040a4210 msgr2=0x7fef0406b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 shutdown_connections 2023-09-30T22:18:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:19.930+0000 7fef0b5bd700 1 -- 172.21.15.89:0/1817920137 wait complete. 2023-09-30T22:18:19.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:20.839 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:20.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:20.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:20.943 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:20.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.987+0000 7f56ef837700 1 Processor -- start 2023-09-30T22:18:20.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.987+0000 7f56ef837700 1 -- start start 2023-09-30T22:18:20.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ef837700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e81316c0 0x7f56e8131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ef837700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56e809d9f0 0x7f56e806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ef837700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e806acb0 0x7f56e806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ef837700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e806b630 con 0x7f56e809d9f0 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ef837700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e809e130 con 0x7f56e81316c0 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ef837700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81312b0 con 0x7f56e806acb0 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ed5d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e81316c0 0x7f56e8131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ed5d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e81316c0 0x7f56e8131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48664/0 (socket says 172.21.15.89:48664) 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ed5d3700 1 -- 172.21.15.89:0/3343748732 learned_addr learned my addr 172.21.15.89:0/3343748732 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:20.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.988+0000 7f56ecdd2700 1 --2- 172.21.15.89:0/3343748732 >> v2:172.21.15.89:3300/0 conn(0x7f56e809d9f0 0x7f56e806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56ed5d3700 1 -- 172.21.15.89:0/3343748732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e806acb0 msgr2=0x7f56e806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56ed5d3700 1 --2- 172.21.15.89:0/3343748732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e806acb0 0x7f56e806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56ed5d3700 1 -- 172.21.15.89:0/3343748732 >> v2:172.21.15.89:3300/0 conn(0x7f56e809d9f0 msgr2=0x7f56e806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56ed5d3700 1 --2- 172.21.15.89:0/3343748732 >> v2:172.21.15.89:3300/0 conn(0x7f56e809d9f0 0x7f56e806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56ed5d3700 1 -- 172.21.15.89:0/3343748732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e809f6e0 con 0x7f56e81316c0 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56ecdd2700 1 --2- 172.21.15.89:0/3343748732 >> v2:172.21.15.89:3300/0 conn(0x7f56e809d9f0 0x7f56e806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56ed5d3700 1 --2- 172.21.15.89:0/3343748732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e81316c0 0x7f56e8131a90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009920 tx=0x7f56e4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1543679da3af05a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56dffff700 1 -- 172.21.15.89:0/3343748732 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4011830 con 0x7f56e81316c0 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56dffff700 1 -- 172.21.15.89:0/3343748732 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e4011990 con 0x7f56e81316c0 2023-09-30T22:18:20.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.989+0000 7f56dffff700 1 -- 172.21.15.89:0/3343748732 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4011d20 con 0x7f56e81316c0 2023-09-30T22:18:20.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 -- 172.21.15.89:0/3343748732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e81316c0 msgr2=0x7f56e8131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 --2- 172.21.15.89:0/3343748732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e81316c0 0x7f56e8131a90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009920 tx=0x7f56e4009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 -- 172.21.15.89:0/3343748732 shutdown_connections 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 --2- 172.21.15.89:0/3343748732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e806acb0 0x7f56e806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 --2- 172.21.15.89:0/3343748732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e81316c0 0x7f56e8131a90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 --2- 172.21.15.89:0/3343748732 >> v2:172.21.15.89:3300/0 conn(0x7f56e809d9f0 0x7f56e806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 -- 172.21.15.89:0/3343748732 >> 172.21.15.89:0/3343748732 conn(0x7f56e8066f50 msgr2=0x7f56e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 -- 172.21.15.89:0/3343748732 shutdown_connections 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.990+0000 7f56ef837700 1 -- 172.21.15.89:0/3343748732 wait complete. 2023-09-30T22:18:20.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ef837700 1 Processor -- start 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ef837700 1 -- start start 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ef837700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e806acb0 0x7f56e811a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ef837700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e809d9f0 0x7f56e811a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ef837700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e81316c0 0x7f56e811acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ef837700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e811dd60 con 0x7f56e806acb0 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ef837700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e811b6a0 con 0x7f56e809d9f0 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ef837700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56e811ccf0 con 0x7f56e81316c0 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.991+0000 7f56ecdd2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e809d9f0 0x7f56e811a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ecdd2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e809d9f0 0x7f56e811a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48680/0 (socket says 172.21.15.89:48680) 2023-09-30T22:18:20.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ecdd2700 1 -- 172.21.15.89:0/3539083082 learned_addr learned my addr 172.21.15.89:0/3539083082 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56eddd4700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e81316c0 0x7f56e811acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ecdd2700 1 -- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e81316c0 msgr2=0x7f56e811acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ecdd2700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e81316c0 0x7f56e811acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ecdd2700 1 -- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e806acb0 msgr2=0x7f56e811a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ecdd2700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e806acb0 0x7f56e811a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ecdd2700 1 -- 172.21.15.89:0/3539083082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80094a0 con 0x7f56e809d9f0 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56eddd4700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e81316c0 0x7f56e811acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ecdd2700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e809d9f0 0x7f56e811a780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f56d800ec10 tx=0x7f56d800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ddffb700 1 -- 172.21.15.89:0/3539083082 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d800c850 con 0x7f56e809d9f0 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ddffb700 1 -- 172.21.15.89:0/3539083082 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d800c9b0 con 0x7f56e809d9f0 2023-09-30T22:18:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e4009550 con 0x7f56e809d9f0 2023-09-30T22:18:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.992+0000 7f56ddffb700 1 -- 172.21.15.89:0/3539083082 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d800cd40 con 0x7f56e809d9f0 2023-09-30T22:18:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.993+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e80a36f0 con 0x7f56e809d9f0 2023-09-30T22:18:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.994+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e8074ea0 con 0x7f56e809d9f0 2023-09-30T22:18:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.994+0000 7f56ddffb700 1 -- 172.21.15.89:0/3539083082 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56d8013470 con 0x7f56e809d9f0 2023-09-30T22:18:20.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.994+0000 7f56ddffb700 1 -- 172.21.15.89:0/3539083082 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f56d803fb90 con 0x7f56e809d9f0 2023-09-30T22:18:21.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:20.999+0000 7f56ddffb700 1 -- 172.21.15.89:0/3539083082 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56d8013720 con 0x7f56e809d9f0 2023-09-30T22:18:21.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.026+0000 7f3c0d5a0700 1 Processor -- start 2023-09-30T22:18:21.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0d5a0700 1 -- start start 2023-09-30T22:18:21.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0d5a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c08133870 0x7f3c08133c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0d5a0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c0809d810 0x7f3c0813b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0d5a0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c08099780 con 0x7f3c08134180 2023-09-30T22:18:21.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0d5a0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c0809aa80 con 0x7f3c0809d810 2023-09-30T22:18:21.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0d5a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08133650 con 0x7f3c08133870 2023-09-30T22:18:21.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0cd9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c0809d810 0x7f3c0813b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0cd9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c0809d810 0x7f3c0813b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45118/0 (socket says 172.21.15.89:45118) 2023-09-30T22:18:21.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c077fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3c08134180 0x7f3c0809d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42126/0 (socket says 172.21.15.89:42126) 2023-09-30T22:18:21.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c0cd9f700 1 -- 172.21.15.89:0/3618496802 learned_addr learned my addr 172.21.15.89:0/3618496802 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:21.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.027+0000 7f3c07fff700 1 --2- 172.21.15.89:0/3618496802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c08133870 0x7f3c08133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c0cd9f700 1 -- 172.21.15.89:0/3618496802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c08133870 msgr2=0x7f3c08133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:21.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c0cd9f700 1 --2- 172.21.15.89:0/3618496802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c08133870 0x7f3c08133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c0cd9f700 1 -- 172.21.15.89:0/3618496802 >> v1:172.21.15.89:6789/0 conn(0x7f3c08134180 legacy=0x7f3c0809d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c06ffd700 1 -- 172.21.15.89:0/3618496802 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283339235 0 0) 0x7f3c08099780 con 0x7f3c08134180 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c0cd9f700 1 -- 172.21.15.89:0/3618496802 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0809ebc0 con 0x7f3c0809d810 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c0cd9f700 1 --2- 172.21.15.89:0/3618496802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c0809d810 0x7f3c0813b8f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009e50 tx=0x7f3c0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25e8449ed07e63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c06ffd700 1 -- 172.21.15.89:0/3618496802 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000e070 con 0x7f3c0809d810 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c06ffd700 1 -- 172.21.15.89:0/3618496802 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0000b910 con 0x7f3c0809d810 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.028+0000 7f3c06ffd700 1 -- 172.21.15.89:0/3618496802 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000bcc0 con 0x7f3c0809d810 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.029+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/3618496802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c0809d810 msgr2=0x7f3c0813b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.029+0000 7f3c0d5a0700 1 --2- 172.21.15.89:0/3618496802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c0809d810 0x7f3c0813b8f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009e50 tx=0x7f3c0000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.029+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/3618496802 shutdown_connections 2023-09-30T22:18:21.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.029+0000 7f3c0d5a0700 1 --2- 172.21.15.89:0/3618496802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c0809d810 0x7f3c0813b8f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.029+0000 7f3c0d5a0700 1 --2- 172.21.15.89:0/3618496802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c08133870 0x7f3c08133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.029+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/3618496802 >> 172.21.15.89:0/3618496802 conn(0x7f3c080a2540 msgr2=0x7f3c080a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:21.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.029+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/3618496802 shutdown_connections 2023-09-30T22:18:21.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.029+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/3618496802 wait complete. 2023-09-30T22:18:21.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.030+0000 7f3c0d5a0700 1 Processor -- start 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.030+0000 7f3c0d5a0700 1 -- start start 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.030+0000 7f3c0d5a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c0809d810 0x7f3c0808eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.030+0000 7f3c0d5a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c08133870 0x7f3c0808f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0d5a0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c08134180 0x7f3c0808f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0d5a0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08096020 con 0x7f3c08134180 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0d5a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0809c040 con 0x7f3c0809d810 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0d5a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08090e20 con 0x7f3c08133870 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c08134180 0x7f3c0808f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c08134180 0x7f3c0808f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45124/0 (socket says 172.21.15.89:45124) 2023-09-30T22:18:21.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 -- 172.21.15.89:0/2965361676 learned_addr learned my addr 172.21.15.89:0/2965361676 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c07fff700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c0809d810 0x7f3c0808eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c077fe700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c08133870 0x7f3c0808f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 -- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c08133870 msgr2=0x7f3c0808f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c08133870 0x7f3c0808f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 -- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c0809d810 msgr2=0x7f3c0808eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c0809d810 0x7f3c0808eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 -- 172.21.15.89:0/2965361676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c00009b30 con 0x7f3c08134180 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c077fe700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c08133870 0x7f3c0808f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c0cd9f700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c08134180 0x7f3c0808f610 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3c00004270 tx=0x7f3c00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c04ff9700 1 -- 172.21.15.89:0/2965361676 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c000048a0 con 0x7f3c08134180 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c04ff9700 1 -- 172.21.15.89:0/2965361676 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c000099b0 con 0x7f3c08134180 2023-09-30T22:18:21.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.031+0000 7f3c04ff9700 1 -- 172.21.15.89:0/2965361676 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0001c660 con 0x7f3c08134180 2023-09-30T22:18:21.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.032+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c080917e0 con 0x7f3c08134180 2023-09-30T22:18:21.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.032+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c08091f00 con 0x7f3c08134180 2023-09-30T22:18:21.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.033+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0804f2a0 con 0x7f3c08134180 2023-09-30T22:18:21.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.033+0000 7f3c04ff9700 1 -- 172.21.15.89:0/2965361676 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c0001c7c0 con 0x7f3c08134180 2023-09-30T22:18:21.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.033+0000 7f3c04ff9700 1 -- 172.21.15.89:0/2965361676 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3c00040ff0 con 0x7f3c08134180 2023-09-30T22:18:21.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.041+0000 7f3c04ff9700 1 -- 172.21.15.89:0/2965361676 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c00024980 con 0x7f3c08134180 2023-09-30T22:18:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.068+0000 7ff836fac700 1 Processor -- start 2023-09-30T22:18:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.069+0000 7ff836fac700 1 -- start start 2023-09-30T22:18:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.069+0000 7ff836fac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133420 0x7ff8301337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.069+0000 7ff836fac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff830134260 0x7ff8301306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.069+0000 7ff836fac700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83006b990 con 0x7ff830133e30 2023-09-30T22:18:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.069+0000 7ff836fac700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8300a0590 con 0x7ff830134260 2023-09-30T22:18:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.069+0000 7ff836fac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff83009fb80 con 0x7ff830133420 2023-09-30T22:18:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.069+0000 7ff82ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff830133e30 0x7ff83012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42136/0 (socket says 172.21.15.89:42136) 2023-09-30T22:18:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff82ffff700 1 -- 172.21.15.89:0/1479324024 learned_addr learned my addr 172.21.15.89:0/1479324024 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff834d48700 1 --2- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133420 0x7ff8301337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff835549700 1 --2- 172.21.15.89:0/1479324024 >> v2:172.21.15.89:3300/0 conn(0x7ff830134260 0x7ff8301306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff835549700 1 -- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133420 msgr2=0x7ff8301337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff835549700 1 --2- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133420 0x7ff8301337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff835549700 1 -- 172.21.15.89:0/1479324024 >> v1:172.21.15.89:6789/0 conn(0x7ff830133e30 legacy=0x7ff83012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff835549700 1 -- 172.21.15.89:0/1479324024 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8300a0030 con 0x7ff830134260 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff82f7fe700 1 -- 172.21.15.89:0/1479324024 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763459640 0 0) 0x7ff83006b990 con 0x7ff830133e30 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff834d48700 1 --2- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133420 0x7ff8301337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff835549700 1 --2- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff8301306a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff828009d90 tx=0x7ff82800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17223ab533f03259 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff82f7fe700 1 -- 172.21.15.89:0/1479324024 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800e070 con 0x7ff830134260 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff82f7fe700 1 -- 172.21.15.89:0/1479324024 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82800bb20 con 0x7ff830134260 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.070+0000 7ff82f7fe700 1 -- 172.21.15.89:0/1479324024 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828012430 con 0x7ff830134260 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.071+0000 7ff836fac700 1 -- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 msgr2=0x7ff8301306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.071+0000 7ff836fac700 1 --2- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff8301306a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff828009d90 tx=0x7ff82800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.071+0000 7ff836fac700 1 -- 172.21.15.89:0/1479324024 shutdown_connections 2023-09-30T22:18:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.071+0000 7ff836fac700 1 --2- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff8301306a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.071+0000 7ff836fac700 1 --2- 172.21.15.89:0/1479324024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133420 0x7ff8301337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.071+0000 7ff836fac700 1 -- 172.21.15.89:0/1479324024 >> 172.21.15.89:0/1479324024 conn(0x7ff8300a4210 msgr2=0x7ff8300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.071+0000 7ff836fac700 1 -- 172.21.15.89:0/1479324024 shutdown_connections 2023-09-30T22:18:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.071+0000 7ff836fac700 1 -- 172.21.15.89:0/1479324024 wait complete. 2023-09-30T22:18:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.072+0000 7ff836fac700 1 Processor -- start 2023-09-30T22:18:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.072+0000 7ff836fac700 1 -- start start 2023-09-30T22:18:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff836fac700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff830133420 0x7ff83009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff836fac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133e30 0x7ff83009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff836fac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff83009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff836fac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83011fdb0 con 0x7ff830134260 2023-09-30T22:18:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff836fac700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83011faf0 con 0x7ff830133420 2023-09-30T22:18:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff836fac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8301314e0 con 0x7ff830133e30 2023-09-30T22:18:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff835549700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff83009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff835549700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff83009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45148/0 (socket says 172.21.15.89:45148) 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff835549700 1 -- 172.21.15.89:0/3164777849 learned_addr learned my addr 172.21.15.89:0/3164777849 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff834d48700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff830133420 0x7ff83009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff82ffff700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133e30 0x7ff83009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff835549700 1 -- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133e30 msgr2=0x7ff83009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff835549700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133e30 0x7ff83009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff835549700 1 -- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff830133420 msgr2=0x7ff83009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff835549700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff830133420 0x7ff83009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.073+0000 7ff835549700 1 -- 172.21.15.89:0/3164777849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240094a0 con 0x7ff830134260 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.074+0000 7ff835549700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff83009cf00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff828013000 tx=0x7ff828004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.074+0000 7ff82d7fa700 1 -- 172.21.15.89:0/3164777849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828012430 con 0x7ff830134260 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.074+0000 7ff82d7fa700 1 -- 172.21.15.89:0/3164777849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828020070 con 0x7ff830134260 2023-09-30T22:18:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.074+0000 7ff82d7fa700 1 -- 172.21.15.89:0/3164777849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828004280 con 0x7ff830134260 2023-09-30T22:18:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.074+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff828009a70 con 0x7ff830134260 2023-09-30T22:18:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.074+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83009dd10 con 0x7ff830134260 2023-09-30T22:18:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.075+0000 7ff82d7fa700 1 -- 172.21.15.89:0/3164777849 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8280043e0 con 0x7ff830134260 2023-09-30T22:18:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.075+0000 7ff82d7fa700 1 -- 172.21.15.89:0/3164777849 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff828022070 con 0x7ff830134260 2023-09-30T22:18:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.076+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff830074ea0 con 0x7ff830134260 2023-09-30T22:18:21.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.080+0000 7ff82d7fa700 1 -- 172.21.15.89:0/3164777849 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff82801b400 con 0x7ff830134260 2023-09-30T22:18:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.109+0000 7fb5482fd700 1 Processor -- start 2023-09-30T22:18:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.109+0000 7fb5482fd700 1 -- start start 2023-09-30T22:18:21.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb5482fd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb54009d630 0x7fb5400a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb5482fd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb540138d80 0x7fb54013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb5482fd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5400a2a70 con 0x7fb5400a0a30 2023-09-30T22:18:21.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb5482fd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5400a01d0 con 0x7fb54009d630 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb5482fd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54009e2b0 con 0x7fb540138d80 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb546099700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb5400a0a30 0x7fb5400a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42140/0 (socket says 172.21.15.89:42140) 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb545898700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb54009d630 0x7fb5400a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb546099700 1 -- 172.21.15.89:0/1628456670 learned_addr learned my addr 172.21.15.89:0/1628456670 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb54689a700 1 --2- 172.21.15.89:0/1628456670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb540138d80 0x7fb54013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb545097700 1 -- 172.21.15.89:0/1628456670 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047105265 0 0) 0x7fb5400a2a70 con 0x7fb5400a0a30 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb545097700 1 -- 172.21.15.89:0/1628456670 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb52c0035b0 con 0x7fb5400a0a30 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb545898700 1 -- 172.21.15.89:0/1628456670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb540138d80 msgr2=0x7fb54013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb545898700 1 --2- 172.21.15.89:0/1628456670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb540138d80 0x7fb54013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb545898700 1 -- 172.21.15.89:0/1628456670 >> v1:172.21.15.89:6789/0 conn(0x7fb5400a0a30 legacy=0x7fb5400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.110+0000 7fb545898700 1 -- 172.21.15.89:0/1628456670 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54009dc50 con 0x7fb54009d630 2023-09-30T22:18:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.111+0000 7fb545898700 1 --2- 172.21.15.89:0/1628456670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 0x7fb5400a1510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb530009840 tx=0x7fb530009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2d858cfc807052 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.111+0000 7fb545097700 1 -- 172.21.15.89:0/1628456670 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530011830 con 0x7fb54009d630 2023-09-30T22:18:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.111+0000 7fb545097700 1 -- 172.21.15.89:0/1628456670 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb530011990 con 0x7fb54009d630 2023-09-30T22:18:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.111+0000 7fb545097700 1 -- 172.21.15.89:0/1628456670 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530011d20 con 0x7fb54009d630 2023-09-30T22:18:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.112+0000 7fb5482fd700 1 -- 172.21.15.89:0/1628456670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 msgr2=0x7fb5400a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.112+0000 7fb5482fd700 1 --2- 172.21.15.89:0/1628456670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 0x7fb5400a1510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb530009840 tx=0x7fb530009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.112+0000 7fb5482fd700 1 -- 172.21.15.89:0/1628456670 shutdown_connections 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.112+0000 7fb5482fd700 1 --2- 172.21.15.89:0/1628456670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb540138d80 0x7fb54013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.112+0000 7fb5482fd700 1 --2- 172.21.15.89:0/1628456670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 0x7fb5400a1510 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.112+0000 7fb5482fd700 1 -- 172.21.15.89:0/1628456670 >> 172.21.15.89:0/1628456670 conn(0x7fb5400a3050 msgr2=0x7fb5400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.112+0000 7fb5482fd700 1 -- 172.21.15.89:0/1628456670 shutdown_connections 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.112+0000 7fb5482fd700 1 -- 172.21.15.89:0/1628456670 wait complete. 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb5482fd700 1 Processor -- start 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb5482fd700 1 -- start start 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb5482fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 0x7fb54012c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb5482fd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5400a0a30 0x7fb54012cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb5482fd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb540138d80 0x7fb54012d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb5482fd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54012e8d0 con 0x7fb54009d630 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb5482fd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54012e240 con 0x7fb5400a0a30 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb5482fd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb54012fe60 con 0x7fb540138d80 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb545898700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5400a0a30 0x7fb54012cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb545898700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5400a0a30 0x7fb54012cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48742/0 (socket says 172.21.15.89:48742) 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb545898700 1 -- 172.21.15.89:0/2415311547 learned_addr learned my addr 172.21.15.89:0/2415311547 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.113+0000 7fb54689a700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb540138d80 0x7fb54012d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb546099700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 0x7fb54012c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb545898700 1 -- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb540138d80 msgr2=0x7fb54012d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb545898700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb540138d80 0x7fb54012d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb545898700 1 -- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 msgr2=0x7fb54012c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb545898700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 0x7fb54012c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb545898700 1 -- 172.21.15.89:0/2415311547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5300094a0 con 0x7fb5400a0a30 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb54689a700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb540138d80 0x7fb54012d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb545898700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5400a0a30 0x7fb54012cc70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb5300090b0 tx=0x7fb530004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb536ffd700 1 -- 172.21.15.89:0/2415311547 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530004400 con 0x7fb5400a0a30 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb536ffd700 1 -- 172.21.15.89:0/2415311547 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53001f070 con 0x7fb5400a0a30 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb536ffd700 1 -- 172.21.15.89:0/2415311547 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300038e0 con 0x7fb5400a0a30 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb540138aa0 con 0x7fb5400a0a30 2023-09-30T22:18:21.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.114+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb540130140 con 0x7fb5400a0a30 2023-09-30T22:18:21.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.116+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54012db90 con 0x7fb5400a0a30 2023-09-30T22:18:21.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.116+0000 7fb536ffd700 1 -- 172.21.15.89:0/2415311547 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb530021030 con 0x7fb5400a0a30 2023-09-30T22:18:21.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.116+0000 7fb536ffd700 1 -- 172.21.15.89:0/2415311547 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb5300407b0 con 0x7fb5400a0a30 2023-09-30T22:18:21.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.120+0000 7fb536ffd700 1 -- 172.21.15.89:0/2415311547 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5300239c0 con 0x7fb5400a0a30 2023-09-30T22:18:21.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.228+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f56e809e980 con 0x7f56e809d9f0 2023-09-30T22:18:21.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.229+0000 7f56ddffb700 1 -- 172.21.15.89:0/3539083082 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56d8021070 con 0x7f56e809d9f0 2023-09-30T22:18:21.230 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:21.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e809d9f0 msgr2=0x7f56e811a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e809d9f0 0x7f56e811a780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f56d800ec10 tx=0x7f56d800ef20 comp rx=0 tx=0).stop 2023-09-30T22:18:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 shutdown_connections 2023-09-30T22:18:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56e81316c0 0x7f56e811acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e806acb0 0x7f56e811a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 --2- 172.21.15.89:0/3539083082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56e809d9f0 0x7f56e811a780 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 >> 172.21.15.89:0/3539083082 conn(0x7f56e8066f50 msgr2=0x7f56e80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 shutdown_connections 2023-09-30T22:18:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.231+0000 7f56ef837700 1 -- 172.21.15.89:0/3539083082 wait complete. 2023-09-30T22:18:21.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:21.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.260+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3c08092500 con 0x7f3c08134180 2023-09-30T22:18:21.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.260+0000 7f3c04ff9700 1 -- 172.21.15.89:0/2965361676 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c000243b0 con 0x7f3c08134180 2023-09-30T22:18:21.261 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c08134180 msgr2=0x7f3c0808f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c08134180 0x7f3c0808f610 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3c00004270 tx=0x7f3c00004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 shutdown_connections 2023-09-30T22:18:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c08134180 0x7f3c0808f610 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c0809d810 0x7f3c0808eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 --2- 172.21.15.89:0/2965361676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c08133870 0x7f3c0808f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 >> 172.21.15.89:0/2965361676 conn(0x7f3c080a2540 msgr2=0x7f3c080978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:21.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 shutdown_connections 2023-09-30T22:18:21.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.262+0000 7f3c0d5a0700 1 -- 172.21.15.89:0/2965361676 wait complete. 2023-09-30T22:18:21.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.278+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8301200f0 con 0x7ff830134260 2023-09-30T22:18:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.279+0000 7ff82d7fa700 1 -- 172.21.15.89:0/3164777849 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff82802c020 con 0x7ff830134260 2023-09-30T22:18:21.279 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 msgr2=0x7ff83009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff83009cf00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff828013000 tx=0x7ff828004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 shutdown_connections 2023-09-30T22:18:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff830134260 0x7ff83009cf00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff830133420 0x7ff83009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 --2- 172.21.15.89:0/3164777849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff830133e30 0x7ff83009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 >> 172.21.15.89:0/3164777849 conn(0x7ff8300a4210 msgr2=0x7ff830130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 shutdown_connections 2023-09-30T22:18:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.281+0000 7ff836fac700 1 -- 172.21.15.89:0/3164777849 wait complete. 2023-09-30T22:18:21.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:21.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.345+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb54012ea40 con 0x7fb5400a0a30 2023-09-30T22:18:21.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.346+0000 7fb536ffd700 1 -- 172.21.15.89:0/2415311547 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb530015070 con 0x7fb5400a0a30 2023-09-30T22:18:21.347 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.347+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5400a0a30 msgr2=0x7fb54012cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.347+0000 7fb5482fd700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5400a0a30 0x7fb54012cc70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb5300090b0 tx=0x7fb530004750 comp rx=0 tx=0).stop 2023-09-30T22:18:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.348+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 shutdown_connections 2023-09-30T22:18:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.348+0000 7fb5482fd700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb540138d80 0x7fb54012d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.348+0000 7fb5482fd700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb54009d630 0x7fb54012c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.348+0000 7fb5482fd700 1 --2- 172.21.15.89:0/2415311547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5400a0a30 0x7fb54012cc70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.348+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 >> 172.21.15.89:0/2415311547 conn(0x7fb5400a3050 msgr2=0x7fb540139380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.348+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 shutdown_connections 2023-09-30T22:18:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:21.348+0000 7fb5482fd700 1 -- 172.21.15.89:0/2415311547 wait complete. 2023-09-30T22:18:21.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:22.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:22.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:22.294 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:22.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.388+0000 7f18a97e4700 1 Processor -- start 2023-09-30T22:18:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.388+0000 7f18a97e4700 1 -- start start 2023-09-30T22:18:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.388+0000 7f18a97e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.388+0000 7f18a97e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18a409de50 0x7f18a409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.388+0000 7f18a97e4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a406ef30 con 0x7f18a409e760 2023-09-30T22:18:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.388+0000 7f18a97e4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a406fef0 con 0x7f18a409de50 2023-09-30T22:18:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.388+0000 7f18a97e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18a41334b0 con 0x7f18a41338a0 2023-09-30T22:18:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a27fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18a409de50 0x7f18a409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52400/0 (socket says 172.21.15.89:52400) 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a27fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18a409de50 0x7f18a409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45182/0 (socket says 172.21.15.89:45182) 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a2ffd700 1 -- 172.21.15.89:0/1863134879 learned_addr learned my addr 172.21.15.89:0/1863134879 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a27fc700 1 -- 172.21.15.89:0/1863134879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 msgr2=0x7f18a4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a27fc700 1 --2- 172.21.15.89:0/1863134879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a4133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a27fc700 1 -- 172.21.15.89:0/1863134879 >> v1:172.21.15.89:6789/0 conn(0x7f18a409e760 legacy=0x7f18a413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a27fc700 1 -- 172.21.15.89:0/1863134879 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a406fc30 con 0x7f18a409de50 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a1ffb700 1 -- 172.21.15.89:0/1863134879 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999586734 0 0) 0x7f18a406ef30 con 0x7f18a409e760 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a2ffd700 1 --2- 172.21.15.89:0/1863134879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a27fc700 1 --2- 172.21.15.89:0/1863134879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a409de50 0x7f18a409e220 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1898009840 tx=0x7f1898009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f093fdc9472f8f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a1ffb700 1 -- 172.21.15.89:0/1863134879 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898011830 con 0x7f18a409de50 2023-09-30T22:18:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a1ffb700 1 -- 172.21.15.89:0/1863134879 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898011990 con 0x7f18a409de50 2023-09-30T22:18:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.389+0000 7f18a1ffb700 1 -- 172.21.15.89:0/1863134879 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898011d20 con 0x7f18a409de50 2023-09-30T22:18:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.390+0000 7f18a97e4700 1 -- 172.21.15.89:0/1863134879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a409de50 msgr2=0x7f18a409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.390+0000 7f18a97e4700 1 --2- 172.21.15.89:0/1863134879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a409de50 0x7f18a409e220 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1898009840 tx=0x7f1898009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.390+0000 7f18a97e4700 1 -- 172.21.15.89:0/1863134879 shutdown_connections 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.390+0000 7f18a97e4700 1 --2- 172.21.15.89:0/1863134879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.390+0000 7f18a97e4700 1 --2- 172.21.15.89:0/1863134879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a409de50 0x7f18a409e220 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.390+0000 7f18a97e4700 1 -- 172.21.15.89:0/1863134879 >> 172.21.15.89:0/1863134879 conn(0x7f18a40a2aa0 msgr2=0x7f18a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.390+0000 7f18a97e4700 1 -- 172.21.15.89:0/1863134879 shutdown_connections 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.391+0000 7f18a97e4700 1 -- 172.21.15.89:0/1863134879 wait complete. 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.392+0000 7f18a97e4700 1 Processor -- start 2023-09-30T22:18:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.392+0000 7f18a97e4700 1 -- start start 2023-09-30T22:18:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a97e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a409de50 0x7f18a411a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a97e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a409e760 0x7f18a411ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a97e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a411b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a97e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a411e0c0 con 0x7f18a409de50 2023-09-30T22:18:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a97e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a40a0d40 con 0x7f18a409e760 2023-09-30T22:18:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a97e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18a411d850 con 0x7f18a41338a0 2023-09-30T22:18:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a27fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a409e760 0x7f18a411ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a27fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a409e760 0x7f18a411ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48744/0 (socket says 172.21.15.89:48744) 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a27fc700 1 -- 172.21.15.89:0/4259632654 learned_addr learned my addr 172.21.15.89:0/4259632654 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a37fe700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a411b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a27fc700 1 -- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 msgr2=0x7f18a411b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a27fc700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a411b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a27fc700 1 -- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a409de50 msgr2=0x7f18a411a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a27fc700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a409de50 0x7f18a411a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a27fc700 1 -- 172.21.15.89:0/4259632654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894009550 con 0x7f18a409e760 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.393+0000 7f18a37fe700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a411b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.394+0000 7f18a27fc700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a409e760 0x7f18a411ac10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1898009fd0 tx=0x7f1898004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.394+0000 7f188b7fe700 1 -- 172.21.15.89:0/4259632654 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18980036a0 con 0x7f18a409e760 2023-09-30T22:18:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.394+0000 7f188b7fe700 1 -- 172.21.15.89:0/4259632654 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189801e070 con 0x7f18a409e760 2023-09-30T22:18:22.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.394+0000 7f188b7fe700 1 -- 172.21.15.89:0/4259632654 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898003a20 con 0x7f18a409e760 2023-09-30T22:18:22.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.394+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18980094a0 con 0x7f18a409e760 2023-09-30T22:18:22.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.394+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a411c1b0 con 0x7f18a409e760 2023-09-30T22:18:22.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.395+0000 7f188b7fe700 1 -- 172.21.15.89:0/4259632654 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1898003b80 con 0x7f18a409e760 2023-09-30T22:18:22.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.395+0000 7f188b7fe700 1 -- 172.21.15.89:0/4259632654 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1898040630 con 0x7f18a409e760 2023-09-30T22:18:22.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.395+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a411d460 con 0x7f18a409e760 2023-09-30T22:18:22.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.400+0000 7f188b7fe700 1 -- 172.21.15.89:0/4259632654 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1898015020 con 0x7f18a409e760 2023-09-30T22:18:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.427+0000 7f5f2b5de700 1 Processor -- start 2023-09-30T22:18:22.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.427+0000 7f5f2b5de700 1 -- start start 2023-09-30T22:18:22.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.427+0000 7f5f2b5de700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409d6f0 0x7f5f2409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2b5de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409e000 0x7f5f24130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2b5de700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f2409e410 0x7f5f24130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2b5de700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2409f520 con 0x7f5f2409e410 2023-09-30T22:18:22.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2b5de700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2409f130 con 0x7f5f2409e000 2023-09-30T22:18:22.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2b5de700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2409e930 con 0x7f5f2409d6f0 2023-09-30T22:18:22.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409d6f0 0x7f5f2409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409d6f0 0x7f5f2409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52420/0 (socket says 172.21.15.89:52420) 2023-09-30T22:18:22.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 -- 172.21.15.89:0/2738150008 learned_addr learned my addr 172.21.15.89:0/2738150008 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:22.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f29b7b700 1 --2- 172.21.15.89:0/2738150008 >> v2:172.21.15.89:3300/0 conn(0x7f5f2409e410 0x7f5f24130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f28b79700 1 --2- 172.21.15.89:0/2738150008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409e000 0x7f5f24130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 -- 172.21.15.89:0/2738150008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409e000 msgr2=0x7f5f24130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 --2- 172.21.15.89:0/2738150008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409e000 0x7f5f24130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 -- 172.21.15.89:0/2738150008 >> v2:172.21.15.89:3300/0 conn(0x7f5f2409e410 msgr2=0x7f5f24130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 --2- 172.21.15.89:0/2738150008 >> v2:172.21.15.89:3300/0 conn(0x7f5f2409e410 0x7f5f24130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 -- 172.21.15.89:0/2738150008 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2409b2d0 con 0x7f5f2409d6f0 2023-09-30T22:18:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f2937a700 1 --2- 172.21.15.89:0/2738150008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409d6f0 0x7f5f2409dac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009920 tx=0x7f5f20009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a03883533b9d0d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.428+0000 7f5f1bfff700 1 -- 172.21.15.89:0/2738150008 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20011830 con 0x7f5f2409d6f0 2023-09-30T22:18:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f1bfff700 1 -- 172.21.15.89:0/2738150008 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20011990 con 0x7f5f2409d6f0 2023-09-30T22:18:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f1bfff700 1 -- 172.21.15.89:0/2738150008 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20011d20 con 0x7f5f2409d6f0 2023-09-30T22:18:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 -- 172.21.15.89:0/2738150008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409d6f0 msgr2=0x7f5f2409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 --2- 172.21.15.89:0/2738150008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409d6f0 0x7f5f2409dac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009920 tx=0x7f5f20009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 -- 172.21.15.89:0/2738150008 shutdown_connections 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 --2- 172.21.15.89:0/2738150008 >> v2:172.21.15.89:3300/0 conn(0x7f5f2409e410 0x7f5f24130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 --2- 172.21.15.89:0/2738150008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409d6f0 0x7f5f2409dac0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 --2- 172.21.15.89:0/2738150008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409e000 0x7f5f24130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 -- 172.21.15.89:0/2738150008 >> 172.21.15.89:0/2738150008 conn(0x7f5f24066f50 msgr2=0x7f5f2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 -- 172.21.15.89:0/2738150008 shutdown_connections 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.429+0000 7f5f2b5de700 1 -- 172.21.15.89:0/2738150008 wait complete. 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.431+0000 7f5f2b5de700 1 Processor -- start 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.431+0000 7f5f2b5de700 1 -- start start 2023-09-30T22:18:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.431+0000 7f5f2b5de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409d6f0 0x7f5f24137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.431+0000 7f5f2b5de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2409e000 0x7f5f24137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.431+0000 7f5f2b5de700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409e410 0x7f5f241383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.431+0000 7f5f2b5de700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2411a680 con 0x7f5f2409e000 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.431+0000 7f5f2b5de700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f24119480 con 0x7f5f2409d6f0 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.431+0000 7f5f2b5de700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f24072630 con 0x7f5f2409e410 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409e410 0x7f5f241383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f2937a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409d6f0 0x7f5f24137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409e410 0x7f5f241383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52434/0 (socket says 172.21.15.89:52434) 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 -- 172.21.15.89:0/1199118092 learned_addr learned my addr 172.21.15.89:0/1199118092 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f28b79700 1 --2- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2409e000 0x7f5f24137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 -- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409d6f0 msgr2=0x7f5f24137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 --2- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409d6f0 0x7f5f24137920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 -- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2409e000 msgr2=0x7f5f24137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 --2- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2409e000 0x7f5f24137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 -- 172.21.15.89:0/1199118092 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20009550 con 0x7f5f2409e410 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f29b7b700 1 --2- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409e410 0x7f5f241383a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00efc0 tx=0x7f5f1c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f19ffb700 1 -- 172.21.15.89:0/1199118092 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c009ea0 con 0x7f5f2409e410 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f19ffb700 1 -- 172.21.15.89:0/1199118092 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00f040 con 0x7f5f2409e410 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2409c700 con 0x7f5f2409e410 2023-09-30T22:18:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.432+0000 7f5f19ffb700 1 -- 172.21.15.89:0/1199118092 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00ceb0 con 0x7f5f2409e410 2023-09-30T22:18:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.433+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2411ae60 con 0x7f5f2409e410 2023-09-30T22:18:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.434+0000 7f5f19ffb700 1 -- 172.21.15.89:0/1199118092 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f1c014480 con 0x7f5f2409e410 2023-09-30T22:18:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.434+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f24074ea0 con 0x7f5f2409e410 2023-09-30T22:18:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.434+0000 7f5f19ffb700 1 -- 172.21.15.89:0/1199118092 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5f1c040510 con 0x7f5f2409e410 2023-09-30T22:18:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.443+0000 7f5f19ffb700 1 -- 172.21.15.89:0/1199118092 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f1c012930 con 0x7f5f2409e410 2023-09-30T22:18:22.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.471+0000 7fa62b59e700 1 Processor -- start 2023-09-30T22:18:22.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.471+0000 7fa62b59e700 1 -- start start 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa62b59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09f8a0 0x7fa62c138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa62b59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f150 0x7fa62c13b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa62b59e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c0a0240 con 0x7fa62c09e670 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa62b59e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c06c9a0 con 0x7fa62c09f8a0 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa62b59e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c06b880 con 0x7fa62c09f150 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa62a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa62c09e670 0x7fa62c09ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42148/0 (socket says 172.21.15.89:42148) 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa62a59c700 1 -- 172.21.15.89:0/1182963794 learned_addr learned my addr 172.21.15.89:0/1182963794 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa629d9b700 1 --2- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09f8a0 0x7fa62c138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa62ad9d700 1 --2- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f150 0x7fa62c13b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa629d9b700 1 -- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f150 msgr2=0x7fa62c13b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:22.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa629d9b700 1 --2- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f150 0x7fa62c13b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa629d9b700 1 -- 172.21.15.89:0/1182963794 >> v1:172.21.15.89:6789/0 conn(0x7fa62c09e670 legacy=0x7fa62c09ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.472+0000 7fa629d9b700 1 -- 172.21.15.89:0/1182963794 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c06bde0 con 0x7fa62c09f8a0 2023-09-30T22:18:22.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.473+0000 7fa629d9b700 1 --2- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09f8a0 0x7fa62c138ba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa620009870 tx=0x7fa620009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d3a66ac9589afc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:22.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.473+0000 7fa62959a700 1 -- 172.21.15.89:0/1182963794 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011830 con 0x7fa62c09f8a0 2023-09-30T22:18:22.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.473+0000 7fa62959a700 1 -- 172.21.15.89:0/1182963794 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa620011990 con 0x7fa62c09f8a0 2023-09-30T22:18:22.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.473+0000 7fa62959a700 1 -- 172.21.15.89:0/1182963794 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011d20 con 0x7fa62c09f8a0 2023-09-30T22:18:22.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.474+0000 7fa62b59e700 1 -- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09f8a0 msgr2=0x7fa62c138ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.474+0000 7fa62b59e700 1 --2- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09f8a0 0x7fa62c138ba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa620009870 tx=0x7fa620009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.474+0000 7fa62b59e700 1 -- 172.21.15.89:0/1182963794 shutdown_connections 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.474+0000 7fa62b59e700 1 --2- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f150 0x7fa62c13b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.474+0000 7fa62b59e700 1 --2- 172.21.15.89:0/1182963794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09f8a0 0x7fa62c138ba0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.474+0000 7fa62b59e700 1 -- 172.21.15.89:0/1182963794 >> 172.21.15.89:0/1182963794 conn(0x7fa62c0a0c90 msgr2=0x7fa62c0a1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.474+0000 7fa62b59e700 1 -- 172.21.15.89:0/1182963794 shutdown_connections 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.474+0000 7fa62b59e700 1 -- 172.21.15.89:0/1182963794 wait complete. 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.475+0000 7fa62b59e700 1 Processor -- start 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.475+0000 7fa62b59e700 1 -- start start 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.475+0000 7fa62b59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09e670 0x7fa62c139130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.475+0000 7fa62b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c09f150 0x7fa62c139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.475+0000 7fa62b59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f8a0 0x7fa62c139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.475+0000 7fa62b59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c11ee70 con 0x7fa62c09f150 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.475+0000 7fa62b59e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c09b470 con 0x7fa62c09e670 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.475+0000 7fa62b59e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c11f610 con 0x7fa62c09f8a0 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09e670 0x7fa62c139130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09e670 0x7fa62c139130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48778/0 (socket says 172.21.15.89:48778) 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 -- 172.21.15.89:0/2479002928 learned_addr learned my addr 172.21.15.89:0/2479002928 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62ad9d700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f8a0 0x7fa62c139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa629d9b700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c09f150 0x7fa62c139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 -- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f8a0 msgr2=0x7fa62c139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f8a0 0x7fa62c139bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 -- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c09f150 msgr2=0x7fa62c139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c09f150 0x7fa62c139670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 -- 172.21.15.89:0/2479002928 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6200094a0 con 0x7fa62c09e670 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa62a59c700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09e670 0x7fa62c139130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c480 tx=0x7fa61c00c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.476+0000 7fa61affd700 1 -- 172.21.15.89:0/2479002928 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c005d70 con 0x7fa62c09e670 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.477+0000 7fa61affd700 1 -- 172.21.15.89:0/2479002928 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c00ce90 con 0x7fa62c09e670 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.477+0000 7fa61affd700 1 -- 172.21.15.89:0/2479002928 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0072d0 con 0x7fa62c09e670 2023-09-30T22:18:22.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.477+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c11ff20 con 0x7fa62c09e670 2023-09-30T22:18:22.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.477+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c11fba0 con 0x7fa62c09e670 2023-09-30T22:18:22.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.478+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c04f2a0 con 0x7fa62c09e670 2023-09-30T22:18:22.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.478+0000 7fa61affd700 1 -- 172.21.15.89:0/2479002928 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa61c007430 con 0x7fa62c09e670 2023-09-30T22:18:22.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.479+0000 7fa61affd700 1 -- 172.21.15.89:0/2479002928 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa61c03ef00 con 0x7fa62c09e670 2023-09-30T22:18:22.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.482+0000 7fa61affd700 1 -- 172.21.15.89:0/2479002928 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa61c01b3b0 con 0x7fa62c09e670 2023-09-30T22:18:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.510+0000 7f15edd52700 1 Processor -- start 2023-09-30T22:18:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.510+0000 7f15edd52700 1 -- start start 2023-09-30T22:18:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.510+0000 7f15edd52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 0x7f15e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.510+0000 7f15edd52700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15edd52700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e80a2a70 con 0x7f15e809d630 2023-09-30T22:18:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15edd52700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80a01d0 con 0x7f15e8138d80 2023-09-30T22:18:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15edd52700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15e809e2b0 con 0x7f15e80a0a30 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e6ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f15e809d630 0x7f15e80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42154/0 (socket says 172.21.15.89:42154) 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e6ffd700 1 -- 172.21.15.89:0/981135204 learned_addr learned my addr 172.21.15.89:0/981135204 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e77fe700 1 --2- 172.21.15.89:0/981135204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 0x7f15e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e7fff700 1 -- 172.21.15.89:0/981135204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 msgr2=0x7f15e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e7fff700 1 --2- 172.21.15.89:0/981135204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 0x7f15e80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e7fff700 1 -- 172.21.15.89:0/981135204 >> v1:172.21.15.89:6789/0 conn(0x7f15e809d630 legacy=0x7f15e80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e7fff700 1 -- 172.21.15.89:0/981135204 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e809dc50 con 0x7f15e8138d80 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e7fff700 1 --2- 172.21.15.89:0/981135204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e813b160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009dc0 tx=0x7f15e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8e2b5e2b4af96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e67fc700 1 -- 172.21.15.89:0/981135204 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e000e070 con 0x7f15e8138d80 2023-09-30T22:18:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e67fc700 1 -- 172.21.15.89:0/981135204 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e000bb20 con 0x7f15e8138d80 2023-09-30T22:18:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.511+0000 7f15e67fc700 1 -- 172.21.15.89:0/981135204 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0012430 con 0x7f15e8138d80 2023-09-30T22:18:22.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.512+0000 7f15edd52700 1 -- 172.21.15.89:0/981135204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 msgr2=0x7f15e813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.512+0000 7f15edd52700 1 --2- 172.21.15.89:0/981135204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e813b160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009dc0 tx=0x7f15e000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:22.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.512+0000 7f15edd52700 1 -- 172.21.15.89:0/981135204 shutdown_connections 2023-09-30T22:18:22.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.512+0000 7f15edd52700 1 --2- 172.21.15.89:0/981135204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e813b160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.512+0000 7f15edd52700 1 --2- 172.21.15.89:0/981135204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 0x7f15e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.512+0000 7f15edd52700 1 -- 172.21.15.89:0/981135204 >> 172.21.15.89:0/981135204 conn(0x7f15e80a3050 msgr2=0x7f15e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:22.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.512+0000 7f15edd52700 1 -- 172.21.15.89:0/981135204 shutdown_connections 2023-09-30T22:18:22.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.512+0000 7f15edd52700 1 -- 172.21.15.89:0/981135204 wait complete. 2023-09-30T22:18:22.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.513+0000 7f15edd52700 1 Processor -- start 2023-09-30T22:18:22.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.513+0000 7f15edd52700 1 -- start start 2023-09-30T22:18:22.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15edd52700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e809d630 0x7f15e8133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15edd52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 0x7f15e8133f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15edd52700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e8134480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:22.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15edd52700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8138ba0 con 0x7f15e809d630 2023-09-30T22:18:22.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15edd52700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8138070 con 0x7f15e8138d80 2023-09-30T22:18:22.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15edd52700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15e813afc0 con 0x7f15e80a0a30 2023-09-30T22:18:22.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e809d630 0x7f15e8133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e809d630 0x7f15e8133a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45208/0 (socket says 172.21.15.89:45208) 2023-09-30T22:18:22.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 -- 172.21.15.89:0/910745701 learned_addr learned my addr 172.21.15.89:0/910745701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:22.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e6ffd700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 0x7f15e8133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e7fff700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e8134480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 -- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 msgr2=0x7f15e8133f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 0x7f15e8133f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 -- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 msgr2=0x7f15e8134480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e8134480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 -- 172.21.15.89:0/910745701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e0009a70 con 0x7f15e809d630 2023-09-30T22:18:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15e77fe700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e809d630 0x7f15e8133a00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f15d800e9b0 tx=0x7f15d800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.514+0000 7f15cffff700 1 -- 172.21.15.89:0/910745701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d800ee50 con 0x7f15e809d630 2023-09-30T22:18:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.515+0000 7f15cffff700 1 -- 172.21.15.89:0/910745701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d800c850 con 0x7f15e809d630 2023-09-30T22:18:22.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.515+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e8136ef0 con 0x7f15e809d630 2023-09-30T22:18:22.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.515+0000 7f15cffff700 1 -- 172.21.15.89:0/910745701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d800cbc0 con 0x7f15e809d630 2023-09-30T22:18:22.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.515+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e8137b90 con 0x7f15e809d630 2023-09-30T22:18:22.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.515+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c80052e0 con 0x7f15e809d630 2023-09-30T22:18:22.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.516+0000 7f15cffff700 1 -- 172.21.15.89:0/910745701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f15d800cd20 con 0x7f15e809d630 2023-09-30T22:18:22.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.516+0000 7f15cffff700 1 -- 172.21.15.89:0/910745701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f15d803fea0 con 0x7f15e809d630 2023-09-30T22:18:22.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.520+0000 7f15cffff700 1 -- 172.21.15.89:0/910745701 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f15d8011350 con 0x7f15e809d630 2023-09-30T22:18:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.641+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5f2411b140 con 0x7f5f2409e410 2023-09-30T22:18:22.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.642+0000 7f5f19ffb700 1 -- 172.21.15.89:0/1199118092 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f1c012340 con 0x7f5f2409e410 2023-09-30T22:18:22.643 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.642+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f18a411dc40 con 0x7f18a409e760 2023-09-30T22:18:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.643+0000 7f188b7fe700 1 -- 172.21.15.89:0/4259632654 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18a411dc40 con 0x7f18a409e760 2023-09-30T22:18:22.644 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.644+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409e410 msgr2=0x7f5f241383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.644+0000 7f5f2b5de700 1 --2- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409e410 0x7f5f241383a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00efc0 tx=0x7f5f1c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.644+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 shutdown_connections 2023-09-30T22:18:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.644+0000 7f5f2b5de700 1 --2- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f2409e410 0x7f5f241383a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.644+0000 7f5f2b5de700 1 --2- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f2409d6f0 0x7f5f24137920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.644+0000 7f5f2b5de700 1 --2- 172.21.15.89:0/1199118092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f2409e000 0x7f5f24137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.644+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 >> 172.21.15.89:0/1199118092 conn(0x7f5f24066f50 msgr2=0x7f5f2406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.645+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 shutdown_connections 2023-09-30T22:18:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a409e760 msgr2=0x7f18a411ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a409e760 0x7f18a411ac10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1898009fd0 tx=0x7f1898004750 comp rx=0 tx=0).stop 2023-09-30T22:18:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f5f2b5de700 1 -- 172.21.15.89:0/1199118092 wait complete. 2023-09-30T22:18:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 shutdown_connections 2023-09-30T22:18:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18a41338a0 0x7f18a411b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18a409de50 0x7f18a411a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 --2- 172.21.15.89:0/4259632654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18a409e760 0x7f18a411ac10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 >> 172.21.15.89:0/4259632654 conn(0x7f18a40a2aa0 msgr2=0x7f18a4130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 shutdown_connections 2023-09-30T22:18:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.646+0000 7f18a97e4700 1 -- 172.21.15.89:0/4259632654 wait complete. 2023-09-30T22:18:22.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:22.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.716+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa62c11f7d0 con 0x7fa62c09e670 2023-09-30T22:18:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.716+0000 7fa61affd700 1 -- 172.21.15.89:0/2479002928 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa61c0192c0 con 0x7fa62c09e670 2023-09-30T22:18:22.717 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09e670 msgr2=0x7fa62c139130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09e670 0x7fa62c139130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c480 tx=0x7fa61c00c790 comp rx=0 tx=0).stop 2023-09-30T22:18:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 shutdown_connections 2023-09-30T22:18:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa62c09f8a0 0x7fa62c139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa62c09e670 0x7fa62c139130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 --2- 172.21.15.89:0/2479002928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c09f150 0x7fa62c139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 >> 172.21.15.89:0/2479002928 conn(0x7fa62c0a0c90 msgr2=0x7fa62c09bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 shutdown_connections 2023-09-30T22:18:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.718+0000 7fa62b59e700 1 -- 172.21.15.89:0/2479002928 wait complete. 2023-09-30T22:18:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.721+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f15c8005860 con 0x7f15e809d630 2023-09-30T22:18:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.722+0000 7f15cffff700 1 -- 172.21.15.89:0/910745701 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f15d802a090 con 0x7f15e809d630 2023-09-30T22:18:22.722 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:22.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e809d630 msgr2=0x7f15e8133a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:22.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e809d630 0x7f15e8133a00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f15d800e9b0 tx=0x7f15d800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 shutdown_connections 2023-09-30T22:18:22.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15e8138d80 0x7f15e8134480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e809d630 0x7f15e8133a00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 --2- 172.21.15.89:0/910745701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15e80a0a30 0x7f15e8133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:22.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 >> 172.21.15.89:0/910745701 conn(0x7f15e80a3050 msgr2=0x7f15e809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:22.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 shutdown_connections 2023-09-30T22:18:22.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:22.724+0000 7f15edd52700 1 -- 172.21.15.89:0/910745701 wait complete. 2023-09-30T22:18:22.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:22.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:23.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:23.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:23.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:23.738 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.803+0000 7fe7929ba700 1 Processor -- start 2023-09-30T22:18:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.803+0000 7fe7929ba700 1 -- start start 2023-09-30T22:18:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.803+0000 7fe7929ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe78c130540 0x7fe78c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.803+0000 7fe7929ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c06fcf0 0x7fe78c138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.803+0000 7fe7929ba700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78c0716a0 con 0x7fe78c130e50 2023-09-30T22:18:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.803+0000 7fe7929ba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c09a390 con 0x7fe78c130540 2023-09-30T22:18:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.803+0000 7fe7929ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c071290 con 0x7fe78c06fcf0 2023-09-30T22:18:23.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe78c130540 0x7fe78c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe790f57700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c06fcf0 0x7fe78c138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe78c130540 0x7fe78c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45220/0 (socket says 172.21.15.89:45220) 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78bfff700 1 -- 172.21.15.89:0/2751426189 learned_addr learned my addr 172.21.15.89:0/2751426189 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78bfff700 1 -- 172.21.15.89:0/2751426189 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c06fcf0 msgr2=0x7fe78c138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78bfff700 1 --2- 172.21.15.89:0/2751426189 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c06fcf0 0x7fe78c138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78bfff700 1 -- 172.21.15.89:0/2751426189 >> v1:172.21.15.89:6789/0 conn(0x7fe78c130e50 legacy=0x7fe78c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78bfff700 1 -- 172.21.15.89:0/2751426189 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c130340 con 0x7fe78c130540 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78bfff700 1 --2- 172.21.15.89:0/2751426189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c130910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0098f0 tx=0x7fe77c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e814b1bf2cf3de67 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.804+0000 7fe78affd700 1 -- 172.21.15.89:0/2751426189 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011830 con 0x7fe78c130540 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe78affd700 1 -- 172.21.15.89:0/2751426189 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c011990 con 0x7fe78c130540 2023-09-30T22:18:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe78affd700 1 -- 172.21.15.89:0/2751426189 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011d20 con 0x7fe78c130540 2023-09-30T22:18:23.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe7929ba700 1 -- 172.21.15.89:0/2751426189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 msgr2=0x7fe78c130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe7929ba700 1 --2- 172.21.15.89:0/2751426189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c130910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0098f0 tx=0x7fe77c009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe7929ba700 1 -- 172.21.15.89:0/2751426189 shutdown_connections 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe7929ba700 1 --2- 172.21.15.89:0/2751426189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c130910 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe7929ba700 1 --2- 172.21.15.89:0/2751426189 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c06fcf0 0x7fe78c138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe7929ba700 1 -- 172.21.15.89:0/2751426189 >> 172.21.15.89:0/2751426189 conn(0x7fe78c0a2aa0 msgr2=0x7fe78c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe7929ba700 1 -- 172.21.15.89:0/2751426189 shutdown_connections 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.805+0000 7fe7929ba700 1 -- 172.21.15.89:0/2751426189 wait complete. 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.806+0000 7fe7929ba700 1 Processor -- start 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe7929ba700 1 -- start start 2023-09-30T22:18:23.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe7929ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe78c06fcf0 0x7fe78c1cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe7929ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c1cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe7929ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c130e50 0x7fe78c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe7929ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c119f30 con 0x7fe78c130540 2023-09-30T22:18:23.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe7929ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c09cdb0 con 0x7fe78c06fcf0 2023-09-30T22:18:23.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe7929ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c11c4c0 con 0x7fe78c130e50 2023-09-30T22:18:23.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe78b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c1cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe78b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c1cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45232/0 (socket says 172.21.15.89:45232) 2023-09-30T22:18:23.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe78b7fe700 1 -- 172.21.15.89:0/3224587267 learned_addr learned my addr 172.21.15.89:0/3224587267 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:23.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.807+0000 7fe790f57700 1 --2- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c130e50 0x7fe78c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe78b7fe700 1 -- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c130e50 msgr2=0x7fe78c118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c130e50 0x7fe78c118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe78b7fe700 1 -- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe78c06fcf0 msgr2=0x7fe78c1cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe78c06fcf0 0x7fe78c1cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe78b7fe700 1 -- 172.21.15.89:0/3224587267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c009550 con 0x7fe78c130540 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c1cce40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe78000c4e0 tx=0x7fe78000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe788ff9700 1 -- 172.21.15.89:0/3224587267 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800052d0 con 0x7fe78c130540 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c11a0b0 con 0x7fe78c130540 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c1cd380 con 0x7fe78c130540 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe788ff9700 1 -- 172.21.15.89:0/3224587267 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe780005430 con 0x7fe78c130540 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.808+0000 7fe788ff9700 1 -- 172.21.15.89:0/3224587267 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800038c0 con 0x7fe78c130540 2023-09-30T22:18:23.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.809+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76c0052e0 con 0x7fe78c130540 2023-09-30T22:18:23.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.810+0000 7fe788ff9700 1 -- 172.21.15.89:0/3224587267 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe780003a20 con 0x7fe78c130540 2023-09-30T22:18:23.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.810+0000 7fe788ff9700 1 -- 172.21.15.89:0/3224587267 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe78003eea0 con 0x7fe78c130540 2023-09-30T22:18:23.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.814+0000 7fe788ff9700 1 -- 172.21.15.89:0/3224587267 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe780018260 con 0x7fe78c130540 2023-09-30T22:18:23.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.845+0000 7f37e1da1700 1 Processor -- start 2023-09-30T22:18:23.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.845+0000 7f37e1da1700 1 -- start start 2023-09-30T22:18:23.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.845+0000 7f37e1da1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37dc09d8d0 0x7f37dc09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.845+0000 7f37e1da1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.845+0000 7f37e1da1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37dc09b6e0 con 0x7f37dc09e1e0 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.845+0000 7f37e1da1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc09b470 con 0x7f37dc09d8d0 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.845+0000 7f37e1da1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc09eb80 con 0x7f37dc09e610 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e0d9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37dc09d8d0 0x7f37dc09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e0d9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37dc09d8d0 0x7f37dc09dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45248/0 (socket says 172.21.15.89:45248) 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e0d9f700 1 -- 172.21.15.89:0/1665039463 learned_addr learned my addr 172.21.15.89:0/1665039463 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e15a0700 1 --2- 172.21.15.89:0/1665039463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e15a0700 1 -- 172.21.15.89:0/1665039463 >> v2:172.21.15.89:3300/0 conn(0x7f37dc09d8d0 msgr2=0x7f37dc09dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e15a0700 1 --2- 172.21.15.89:0/1665039463 >> v2:172.21.15.89:3300/0 conn(0x7f37dc09d8d0 0x7f37dc09dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e15a0700 1 -- 172.21.15.89:0/1665039463 >> v1:172.21.15.89:6789/0 conn(0x7f37dc09e1e0 legacy=0x7f37dc130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e15a0700 1 -- 172.21.15.89:0/1665039463 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc09ae00 con 0x7f37dc09e610 2023-09-30T22:18:23.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37e15a0700 1 --2- 172.21.15.89:0/1665039463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc130b70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f37d4009d90 tx=0x7f37d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a6a4af2a717c5e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:23.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.846+0000 7f37db7fe700 1 -- 172.21.15.89:0/1665039463 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400e070 con 0x7f37dc09e610 2023-09-30T22:18:23.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37db7fe700 1 -- 172.21.15.89:0/1665039463 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d400bb20 con 0x7f37dc09e610 2023-09-30T22:18:23.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37db7fe700 1 -- 172.21.15.89:0/1665039463 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4012430 con 0x7f37dc09e610 2023-09-30T22:18:23.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37e1da1700 1 -- 172.21.15.89:0/1665039463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 msgr2=0x7f37dc130b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37e1da1700 1 --2- 172.21.15.89:0/1665039463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc130b70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f37d4009d90 tx=0x7f37d400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:23.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37e1da1700 1 -- 172.21.15.89:0/1665039463 shutdown_connections 2023-09-30T22:18:23.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37e1da1700 1 --2- 172.21.15.89:0/1665039463 >> v2:172.21.15.89:3300/0 conn(0x7f37dc09d8d0 0x7f37dc09dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37e1da1700 1 --2- 172.21.15.89:0/1665039463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc130b70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37e1da1700 1 -- 172.21.15.89:0/1665039463 >> 172.21.15.89:0/1665039463 conn(0x7f37dc0a2e20 msgr2=0x7f37dc0a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37e1da1700 1 -- 172.21.15.89:0/1665039463 shutdown_connections 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.847+0000 7f37e1da1700 1 -- 172.21.15.89:0/1665039463 wait complete. 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.848+0000 7f37e1da1700 1 Processor -- start 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.848+0000 7f37e1da1700 1 -- start start 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e1da1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37dc09d8d0 0x7f37dc122250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e1da1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37dc09e1e0 0x7f37dc122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e1da1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e1da1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc1253b0 con 0x7f37dc09d8d0 2023-09-30T22:18:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e1da1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc124660 con 0x7f37dc09e1e0 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e1da1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc123dc0 con 0x7f37dc09e610 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e0d9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37dc09d8d0 0x7f37dc122250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e0d9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37dc09d8d0 0x7f37dc122250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45252/0 (socket says 172.21.15.89:45252) 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e0d9f700 1 -- 172.21.15.89:0/1375987567 learned_addr learned my addr 172.21.15.89:0/1375987567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e15a0700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37dbfff700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37dc09e1e0 0x7f37dc122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e0d9f700 1 -- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 msgr2=0x7f37dc122cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e0d9f700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc122cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e0d9f700 1 -- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37dc09e1e0 msgr2=0x7f37dc122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.849+0000 7f37e0d9f700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37dc09e1e0 0x7f37dc122790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.850+0000 7f37e0d9f700 1 -- 172.21.15.89:0/1375987567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d4009a70 con 0x7f37dc09d8d0 2023-09-30T22:18:23.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.850+0000 7f37dbfff700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37dc09e1e0 0x7f37dc122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:23.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.850+0000 7f37e0d9f700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37dc09d8d0 0x7f37dc122250 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b240 tx=0x7f37d000d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:23.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.850+0000 7f37d97fa700 1 -- 172.21.15.89:0/1375987567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d000de40 con 0x7f37dc09d8d0 2023-09-30T22:18:23.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.850+0000 7f37d97fa700 1 -- 172.21.15.89:0/1375987567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d0010440 con 0x7f37dc09d8d0 2023-09-30T22:18:23.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.850+0000 7f37d97fa700 1 -- 172.21.15.89:0/1375987567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d00107d0 con 0x7f37dc09d8d0 2023-09-30T22:18:23.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.850+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37dc125cd0 con 0x7f37dc09d8d0 2023-09-30T22:18:23.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.850+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37dc09c650 con 0x7f37dc09d8d0 2023-09-30T22:18:23.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.851+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37dc125b10 con 0x7f37dc09d8d0 2023-09-30T22:18:23.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.851+0000 7f37d97fa700 1 -- 172.21.15.89:0/1375987567 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f37d0010930 con 0x7f37dc09d8d0 2023-09-30T22:18:23.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.852+0000 7f37d97fa700 1 -- 172.21.15.89:0/1375987567 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f37d003fe80 con 0x7f37dc09d8d0 2023-09-30T22:18:23.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.860+0000 7f37d97fa700 1 -- 172.21.15.89:0/1375987567 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f37d0016d40 con 0x7f37dc09d8d0 2023-09-30T22:18:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.886+0000 7f8874ad5700 1 Processor -- start 2023-09-30T22:18:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.886+0000 7f8874ad5700 1 -- start start 2023-09-30T22:18:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f8874ad5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f887009d6f0 0x7f887009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f8874ad5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009e000 0x7f88701339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f8874ad5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887009b120 con 0x7f887009e410 2023-09-30T22:18:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f8874ad5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887006b460 con 0x7f887009d6f0 2023-09-30T22:18:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f8874ad5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f887009a7e0 con 0x7f887009e000 2023-09-30T22:18:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f886e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f887009d6f0 0x7f887009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f886dd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009e000 0x7f88701339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f886e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f887009d6f0 0x7f887009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45268/0 (socket says 172.21.15.89:45268) 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f886e59c700 1 -- 172.21.15.89:0/379690677 learned_addr learned my addr 172.21.15.89:0/379690677 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.887+0000 7f886ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f887009e410 0x7f8870133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42178/0 (socket says 172.21.15.89:42178) 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886e59c700 1 -- 172.21.15.89:0/379690677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009e000 msgr2=0x7f88701339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886e59c700 1 --2- 172.21.15.89:0/379690677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009e000 0x7f88701339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886e59c700 1 -- 172.21.15.89:0/379690677 >> v1:172.21.15.89:6789/0 conn(0x7f887009e410 legacy=0x7f8870133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886e59c700 1 -- 172.21.15.89:0/379690677 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887009a500 con 0x7f887009d6f0 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886dd9b700 1 --2- 172.21.15.89:0/379690677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009e000 0x7f88701339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886e59c700 1 --2- 172.21.15.89:0/379690677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009d6f0 0x7f887009dac0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f88600098f0 tx=0x7f8860009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cf83bfde68620f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886d59a700 1 -- 172.21.15.89:0/379690677 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8860011830 con 0x7f887009d6f0 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886d59a700 1 -- 172.21.15.89:0/379690677 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8860011990 con 0x7f887009d6f0 2023-09-30T22:18:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.888+0000 7f886d59a700 1 -- 172.21.15.89:0/379690677 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8860011d20 con 0x7f887009d6f0 2023-09-30T22:18:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.889+0000 7f8874ad5700 1 -- 172.21.15.89:0/379690677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009d6f0 msgr2=0x7f887009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.889+0000 7f8874ad5700 1 --2- 172.21.15.89:0/379690677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009d6f0 0x7f887009dac0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f88600098f0 tx=0x7f8860009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.889+0000 7f8874ad5700 1 -- 172.21.15.89:0/379690677 shutdown_connections 2023-09-30T22:18:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.889+0000 7f8874ad5700 1 --2- 172.21.15.89:0/379690677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009d6f0 0x7f887009dac0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.889+0000 7f8874ad5700 1 --2- 172.21.15.89:0/379690677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009e000 0x7f88701339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.889+0000 7f8874ad5700 1 -- 172.21.15.89:0/379690677 >> 172.21.15.89:0/379690677 conn(0x7f8870066f50 msgr2=0x7f887009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.889+0000 7f8874ad5700 1 -- 172.21.15.89:0/379690677 shutdown_connections 2023-09-30T22:18:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.889+0000 7f8874ad5700 1 -- 172.21.15.89:0/379690677 wait complete. 2023-09-30T22:18:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.890+0000 7fe57fdec700 1 Processor -- start 2023-09-30T22:18:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.890+0000 7f8874ad5700 1 Processor -- start 2023-09-30T22:18:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.890+0000 7fe57fdec700 1 -- start start 2023-09-30T22:18:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7fe57fdec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe57809e1d0 0x7fe578135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7fe57fdec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe578135e50 0x7fe578138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7fe57fdec700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57809f130 con 0x7fe57809d6f0 2023-09-30T22:18:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7fe57fdec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57809f520 con 0x7fe57809e1d0 2023-09-30T22:18:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7fe57fdec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57809e930 con 0x7fe578135e50 2023-09-30T22:18:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57d387700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe57809e1d0 0x7fe578135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7fe57db88700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe57809d6f0 0x7fe57809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42192/0 (socket says 172.21.15.89:42192) 2023-09-30T22:18:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57db88700 1 -- 172.21.15.89:0/3237116074 learned_addr learned my addr 172.21.15.89:0/3237116074 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57e389700 1 --2- 172.21.15.89:0/3237116074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe578135e50 0x7fe578138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f8874ad5700 1 -- start start 2023-09-30T22:18:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f8874ad5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009d6f0 0x7f8870119720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f8874ad5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009e000 0x7f8870119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f8874ad5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887009e410 0x7f887011a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f8874ad5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887011ca30 con 0x7f887009e000 2023-09-30T22:18:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f8874ad5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887011cfd0 con 0x7f887009e410 2023-09-30T22:18:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f8874ad5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88700a21d0 con 0x7f887009d6f0 2023-09-30T22:18:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f886e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009d6f0 0x7f8870119720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f886e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009d6f0 0x7f8870119720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52534/0 (socket says 172.21.15.89:52534) 2023-09-30T22:18:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f886dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009e000 0x7f8870119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.891+0000 7f886e59c700 1 -- 172.21.15.89:0/1659966365 learned_addr learned my addr 172.21.15.89:0/1659966365 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f886ed9d700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887009e410 0x7f887011a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57e389700 1 -- 172.21.15.89:0/3237116074 >> v2:172.21.15.89:3300/0 conn(0x7fe57809e1d0 msgr2=0x7fe578135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57e389700 1 --2- 172.21.15.89:0/3237116074 >> v2:172.21.15.89:3300/0 conn(0x7fe57809e1d0 0x7fe578135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57e389700 1 -- 172.21.15.89:0/3237116074 >> v1:172.21.15.89:6789/0 conn(0x7fe57809d6f0 legacy=0x7fe57809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57e389700 1 -- 172.21.15.89:0/3237116074 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57809b2d0 con 0x7fe578135e50 2023-09-30T22:18:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57cb86700 1 -- 172.21.15.89:0/3237116074 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415801136 0 0) 0x7fe57809f130 con 0x7fe57809d6f0 2023-09-30T22:18:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57e389700 1 --2- 172.21.15.89:0/3237116074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe578135e50 0x7fe578138230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe570009e10 tx=0x7fe57000d540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca045f61221594bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f886ed9d700 1 -- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009d6f0 msgr2=0x7f8870119720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f886ed9d700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009d6f0 0x7f8870119720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f886ed9d700 1 -- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009e000 msgr2=0x7f8870119c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f886ed9d700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009e000 0x7f8870119c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f886ed9d700 1 -- 172.21.15.89:0/1659966365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88640094a0 con 0x7f887009e410 2023-09-30T22:18:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f886e59c700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009d6f0 0x7f8870119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57d387700 1 --2- 172.21.15.89:0/3237116074 >> v2:172.21.15.89:3300/0 conn(0x7fe57809e1d0 0x7fe578135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57cb86700 1 -- 172.21.15.89:0/3237116074 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570011460 con 0x7fe578135e50 2023-09-30T22:18:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57cb86700 1 -- 172.21.15.89:0/3237116074 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5700115c0 con 0x7fe578135e50 2023-09-30T22:18:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7fe57cb86700 1 -- 172.21.15.89:0/3237116074 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570011950 con 0x7fe578135e50 2023-09-30T22:18:23.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f886ed9d700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887009e410 0x7f887011a1a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f886800cb10 tx=0x7f886800ce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:23.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.892+0000 7f885effd700 1 -- 172.21.15.89:0/1659966365 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886800aaa0 con 0x7f887009e410 2023-09-30T22:18:23.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.893+0000 7f885effd700 1 -- 172.21.15.89:0/1659966365 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886800ac00 con 0x7f887009e410 2023-09-30T22:18:23.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.893+0000 7f885effd700 1 -- 172.21.15.89:0/1659966365 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868007070 con 0x7f887009e410 2023-09-30T22:18:23.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.893+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8860009550 con 0x7f887009e410 2023-09-30T22:18:23.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.893+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887011ba70 con 0x7f887009e410 2023-09-30T22:18:23.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.893+0000 7fe57fdec700 1 -- 172.21.15.89:0/3237116074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe578135e50 msgr2=0x7fe578138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.893+0000 7fe57fdec700 1 --2- 172.21.15.89:0/3237116074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe578135e50 0x7fe578138230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe570009e10 tx=0x7fe57000d540 comp rx=0 tx=0).stop 2023-09-30T22:18:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7fe57fdec700 1 -- 172.21.15.89:0/3237116074 shutdown_connections 2023-09-30T22:18:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7fe57fdec700 1 --2- 172.21.15.89:0/3237116074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe578135e50 0x7fe578138230 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7fe57fdec700 1 --2- 172.21.15.89:0/3237116074 >> v2:172.21.15.89:3300/0 conn(0x7fe57809e1d0 0x7fe578135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7fe57fdec700 1 -- 172.21.15.89:0/3237116074 >> 172.21.15.89:0/3237116074 conn(0x7fe578066f50 msgr2=0x7fe57809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7fe57fdec700 1 -- 172.21.15.89:0/3237116074 shutdown_connections 2023-09-30T22:18:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7fe57fdec700 1 -- 172.21.15.89:0/3237116074 wait complete. 2023-09-30T22:18:23.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8870074ea0 con 0x7f887009e410 2023-09-30T22:18:23.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7f885effd700 1 -- 172.21.15.89:0/1659966365 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88680071d0 con 0x7f887009e410 2023-09-30T22:18:23.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.894+0000 7f885effd700 1 -- 172.21.15.89:0/1659966365 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f886803f410 con 0x7f887009e410 2023-09-30T22:18:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.895+0000 7fe57fdec700 1 Processor -- start 2023-09-30T22:18:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.895+0000 7fe57fdec700 1 -- start start 2023-09-30T22:18:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57fdec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe57809d6f0 0x7fe578119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57fdec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe57809e1d0 0x7fe5781199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57fdec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe578135e50 0x7fe578119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57fdec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57811b440 con 0x7fe578135e50 2023-09-30T22:18:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57fdec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57811cd70 con 0x7fe57809d6f0 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57fdec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780a1d00 con 0x7fe57809e1d0 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57db88700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe57809d6f0 0x7fe578119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57e389700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe578135e50 0x7fe578119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57db88700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe57809d6f0 0x7fe578119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48850/0 (socket says 172.21.15.89:48850) 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57db88700 1 -- 172.21.15.89:0/3576353230 learned_addr learned my addr 172.21.15.89:0/3576353230 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57e389700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe578135e50 0x7fe578119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45294/0 (socket says 172.21.15.89:45294) 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57d387700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe57809e1d0 0x7fe5781199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57db88700 1 -- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe57809e1d0 msgr2=0x7fe5781199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57db88700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe57809e1d0 0x7fe5781199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.896+0000 7fe57db88700 1 -- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe578135e50 msgr2=0x7fe578119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.897+0000 7fe57db88700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe578135e50 0x7fe578119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:23.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.897+0000 7fe57db88700 1 -- 172.21.15.89:0/3576353230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680094a0 con 0x7fe57809d6f0 2023-09-30T22:18:23.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.897+0000 7fe57e389700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe578135e50 0x7fe578119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:23.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.897+0000 7fe57db88700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe57809d6f0 0x7fe578119480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c520 tx=0x7fe57400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:23.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.900+0000 7fe56e7fc700 1 -- 172.21.15.89:0/3576353230 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574004c80 con 0x7fe57809d6f0 2023-09-30T22:18:23.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.900+0000 7fe56e7fc700 1 -- 172.21.15.89:0/3576353230 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574004de0 con 0x7fe57809d6f0 2023-09-30T22:18:23.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.900+0000 7fe56e7fc700 1 -- 172.21.15.89:0/3576353230 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574007310 con 0x7fe57809d6f0 2023-09-30T22:18:23.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.900+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe570009a70 con 0x7fe57809d6f0 2023-09-30T22:18:23.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.900+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57811c500 con 0x7fe57809d6f0 2023-09-30T22:18:23.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.900+0000 7f885effd700 1 -- 172.21.15.89:0/1659966365 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8868012400 con 0x7f887009e410 2023-09-30T22:18:23.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.901+0000 7fe56e7fc700 1 -- 172.21.15.89:0/3576353230 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe574007470 con 0x7fe57809d6f0 2023-09-30T22:18:23.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.901+0000 7fe56e7fc700 1 -- 172.21.15.89:0/3576353230 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe57403f340 con 0x7fe57809d6f0 2023-09-30T22:18:23.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.905+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe578074ea0 con 0x7fe57809d6f0 2023-09-30T22:18:23.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:23.909+0000 7fe56e7fc700 1 -- 172.21.15.89:0/3576353230 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5740123b0 con 0x7fe57809d6f0 2023-09-30T22:18:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.035+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe76c005150 con 0x7fe78c130540 2023-09-30T22:18:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.035+0000 7fe788ff9700 1 -- 172.21.15.89:0/3224587267 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe780017030 con 0x7fe78c130540 2023-09-30T22:18:24.036 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:24.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.037+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 msgr2=0x7fe78c1cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:24.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.037+0000 7fe7929ba700 1 --2- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c1cce40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe78000c4e0 tx=0x7fe78000c7f0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.037+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 shutdown_connections 2023-09-30T22:18:24.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.037+0000 7fe7929ba700 1 --2- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe78c130e50 0x7fe78c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.037+0000 7fe7929ba700 1 --2- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe78c06fcf0 0x7fe78c1cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.037+0000 7fe7929ba700 1 --2- 172.21.15.89:0/3224587267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c130540 0x7fe78c1cce40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.037+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 >> 172.21.15.89:0/3224587267 conn(0x7fe78c0a2aa0 msgr2=0x7fe78c0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:24.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.038+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 shutdown_connections 2023-09-30T22:18:24.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.038+0000 7fe7929ba700 1 -- 172.21.15.89:0/3224587267 wait complete. 2023-09-30T22:18:24.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.062+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f37dc0a1f50 con 0x7f37dc09d8d0 2023-09-30T22:18:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.062+0000 7f37d97fa700 1 -- 172.21.15.89:0/1375987567 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f37d0016300 con 0x7f37dc09d8d0 2023-09-30T22:18:24.063 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37dc09d8d0 msgr2=0x7f37dc122250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37dc09d8d0 0x7f37dc122250 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b240 tx=0x7f37d000d630 comp rx=0 tx=0).stop 2023-09-30T22:18:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 shutdown_connections 2023-09-30T22:18:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37dc09e610 0x7f37dc122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37dc09d8d0 0x7f37dc122250 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 --2- 172.21.15.89:0/1375987567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37dc09e1e0 0x7f37dc122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 >> 172.21.15.89:0/1375987567 conn(0x7f37dc0a2e20 msgr2=0x7f37dc130700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 shutdown_connections 2023-09-30T22:18:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.064+0000 7f37e1da1700 1 -- 172.21.15.89:0/1375987567 wait complete. 2023-09-30T22:18:24.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:24.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.105+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe57811b650 con 0x7fe57809d6f0 2023-09-30T22:18:24.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.106+0000 7fe56e7fc700 1 -- 172.21.15.89:0/3576353230 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe574011360 con 0x7fe57809d6f0 2023-09-30T22:18:24.106 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:24.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe57809d6f0 msgr2=0x7fe578119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:24.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe57809d6f0 0x7fe578119480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c520 tx=0x7fe57400c830 comp rx=0 tx=0).stop 2023-09-30T22:18:24.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 shutdown_connections 2023-09-30T22:18:24.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe578135e50 0x7fe578119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe57809d6f0 0x7fe578119480 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 --2- 172.21.15.89:0/3576353230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe57809e1d0 0x7fe5781199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 >> 172.21.15.89:0/3576353230 conn(0x7fe578066f50 msgr2=0x7fe5780a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:24.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 shutdown_connections 2023-09-30T22:18:24.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.108+0000 7fe57fdec700 1 -- 172.21.15.89:0/3576353230 wait complete. 2023-09-30T22:18:24.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.113+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f887006c360 con 0x7f887009e410 2023-09-30T22:18:24.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.113+0000 7f885effd700 1 -- 172.21.15.89:0/1659966365 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f88680113a0 con 0x7f887009e410 2023-09-30T22:18:24.114 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:24.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.115+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887009e410 msgr2=0x7f887011a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:24.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.115+0000 7f8874ad5700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887009e410 0x7f887011a1a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f886800cb10 tx=0x7f886800ce20 comp rx=0 tx=0).stop 2023-09-30T22:18:24.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.116+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 shutdown_connections 2023-09-30T22:18:24.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.116+0000 7f8874ad5700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f887009e410 0x7f887011a1a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.116+0000 7f8874ad5700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f887009d6f0 0x7f8870119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.116+0000 7f8874ad5700 1 --2- 172.21.15.89:0/1659966365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887009e000 0x7f8870119c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:24.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.116+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 >> 172.21.15.89:0/1659966365 conn(0x7f8870066f50 msgr2=0x7f88701bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:24.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.116+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 shutdown_connections 2023-09-30T22:18:24.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:24.116+0000 7f8874ad5700 1 -- 172.21.15.89:0/1659966365 wait complete. 2023-09-30T22:18:24.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:24.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:25.051 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:25.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:25.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:25.129 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.197+0000 7f48b2cf3700 1 Processor -- start 2023-09-30T22:18:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.197+0000 7f48b2cf3700 1 -- start start 2023-09-30T22:18:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48b2cf3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac09eca0 0x7f48ac13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48b2cf3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48ac06f050 0x7f48ac13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48b2cf3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac070ee0 con 0x7f48ac06e570 2023-09-30T22:18:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48b2cf3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ac071980 con 0x7f48ac06f050 2023-09-30T22:18:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48b2cf3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac071480 con 0x7f48ac09eca0 2023-09-30T22:18:25.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48abfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac09eca0 0x7f48ac13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48abfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac09eca0 0x7f48ac13cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48860/0 (socket says 172.21.15.89:48860) 2023-09-30T22:18:25.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48abfff700 1 -- 172.21.15.89:0/3352328936 learned_addr learned my addr 172.21.15.89:0/3352328936 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:25.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48b1290700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48ac06f050 0x7f48ac13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48abfff700 1 -- 172.21.15.89:0/3352328936 >> v2:172.21.15.89:3300/0 conn(0x7f48ac06f050 msgr2=0x7f48ac13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:25.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48abfff700 1 --2- 172.21.15.89:0/3352328936 >> v2:172.21.15.89:3300/0 conn(0x7f48ac06f050 0x7f48ac13f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48abfff700 1 -- 172.21.15.89:0/3352328936 >> v1:172.21.15.89:6789/0 conn(0x7f48ac06e570 legacy=0x7f48ac06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48abfff700 1 -- 172.21.15.89:0/3352328936 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac071b90 con 0x7f48ac09eca0 2023-09-30T22:18:25.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.198+0000 7f48b1290700 1 --2- 172.21.15.89:0/3352328936 >> v2:172.21.15.89:3300/0 conn(0x7f48ac06f050 0x7f48ac13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:25.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.199+0000 7f48abfff700 1 --2- 172.21.15.89:0/3352328936 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac09eca0 0x7f48ac13cea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48980098f0 tx=0x7f4898009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ec9ddae72cc2a3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:25.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.199+0000 7f48ab7fe700 1 -- 172.21.15.89:0/3352328936 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898011830 con 0x7f48ac09eca0 2023-09-30T22:18:25.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.199+0000 7f48ab7fe700 1 -- 172.21.15.89:0/3352328936 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898011990 con 0x7f48ac09eca0 2023-09-30T22:18:25.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.199+0000 7f48ab7fe700 1 -- 172.21.15.89:0/3352328936 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898011d20 con 0x7f48ac09eca0 2023-09-30T22:18:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.199+0000 7f48b2cf3700 1 -- 172.21.15.89:0/3352328936 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac09eca0 msgr2=0x7f48ac13cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.199+0000 7f48b2cf3700 1 --2- 172.21.15.89:0/3352328936 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac09eca0 0x7f48ac13cea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48980098f0 tx=0x7f4898009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.200+0000 7f48b2cf3700 1 -- 172.21.15.89:0/3352328936 shutdown_connections 2023-09-30T22:18:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.200+0000 7f48b2cf3700 1 --2- 172.21.15.89:0/3352328936 >> v2:172.21.15.89:3300/0 conn(0x7f48ac06f050 0x7f48ac13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.200+0000 7f48b2cf3700 1 --2- 172.21.15.89:0/3352328936 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac09eca0 0x7f48ac13cea0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.200+0000 7f48b2cf3700 1 -- 172.21.15.89:0/3352328936 >> 172.21.15.89:0/3352328936 conn(0x7f48ac0a2aa0 msgr2=0x7f48ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.200+0000 7f48b2cf3700 1 -- 172.21.15.89:0/3352328936 shutdown_connections 2023-09-30T22:18:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.200+0000 7f48b2cf3700 1 -- 172.21.15.89:0/3352328936 wait complete. 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.201+0000 7f48b2cf3700 1 Processor -- start 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.201+0000 7f48b2cf3700 1 -- start start 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b2cf3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ac06e570 0x7f48ac1334e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b2cf3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ac06f050 0x7f48ac133a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b2cf3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac1374b0 0x7f48ac133f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b2cf3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac13ed10 con 0x7f48ac06f050 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b2cf3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac1362f0 con 0x7f48ac1374b0 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b2cf3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac135220 con 0x7f48ac06e570 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b1290700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac1374b0 0x7f48ac133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b1290700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac1374b0 0x7f48ac133f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48864/0 (socket says 172.21.15.89:48864) 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b1290700 1 -- 172.21.15.89:0/2746206988 learned_addr learned my addr 172.21.15.89:0/2746206988 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b0a8f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ac06e570 0x7f48ac1334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48abfff700 1 --2- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ac06f050 0x7f48ac133a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b1290700 1 -- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ac06e570 msgr2=0x7f48ac1334e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b1290700 1 --2- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ac06e570 0x7f48ac1334e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b1290700 1 -- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ac06f050 msgr2=0x7f48ac133a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b1290700 1 --2- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ac06f050 0x7f48ac133a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.202+0000 7f48b1290700 1 -- 172.21.15.89:0/2746206988 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4009a70 con 0x7f48ac1374b0 2023-09-30T22:18:25.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.203+0000 7f48b1290700 1 --2- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac1374b0 0x7f48ac133f60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48a400efc0 tx=0x7f48a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.203+0000 7f48a97fa700 1 -- 172.21.15.89:0/2746206988 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400ca90 con 0x7f48ac1374b0 2023-09-30T22:18:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.203+0000 7f48a97fa700 1 -- 172.21.15.89:0/2746206988 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a400f040 con 0x7f48ac1374b0 2023-09-30T22:18:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.203+0000 7f48a97fa700 1 -- 172.21.15.89:0/2746206988 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400ce10 con 0x7f48ac1374b0 2023-09-30T22:18:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.203+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4898009550 con 0x7f48ac1374b0 2023-09-30T22:18:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.203+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac135900 con 0x7f48ac1374b0 2023-09-30T22:18:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.204+0000 7f48a97fa700 1 -- 172.21.15.89:0/2746206988 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48a4014480 con 0x7f48ac1374b0 2023-09-30T22:18:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.204+0000 7f48a97fa700 1 -- 172.21.15.89:0/2746206988 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f48a403fcc0 con 0x7f48ac1374b0 2023-09-30T22:18:25.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.204+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac074ea0 con 0x7f48ac1374b0 2023-09-30T22:18:25.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.209+0000 7f48a97fa700 1 -- 172.21.15.89:0/2746206988 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48a4012700 con 0x7f48ac1374b0 2023-09-30T22:18:25.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.232+0000 7f4d69806700 1 Processor -- start 2023-09-30T22:18:25.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.232+0000 7f4d69806700 1 -- start start 2023-09-30T22:18:25.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d69806700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 0x7f4d6412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d69806700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d64135c20 0x7f4d64138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d69806700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6406fa40 con 0x7f4d6412cf60 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d69806700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6412cd90 con 0x7f4d64135c20 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d69806700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64071780 con 0x7f4d6406f5d0 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d637fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d64135c20 0x7f4d64138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d637fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d64135c20 0x7f4d64138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45322/0 (socket says 172.21.15.89:45322) 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d637fe700 1 -- 172.21.15.89:0/1141456245 learned_addr learned my addr 172.21.15.89:0/1141456245 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d62ffd700 1 --1- 172.21.15.89:0/1141456245 >> v1:172.21.15.89:6789/0 conn(0x7f4d6412cf60 0x7f4d6412f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42206/0 (socket says 172.21.15.89:42206) 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d627fc700 1 --2- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 0x7f4d6412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d637fe700 1 -- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 msgr2=0x7f4d6412ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d637fe700 1 --2- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 0x7f4d6412ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d637fe700 1 -- 172.21.15.89:0/1141456245 >> v1:172.21.15.89:6789/0 conn(0x7f4d6412cf60 legacy=0x7f4d6412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d637fe700 1 -- 172.21.15.89:0/1141456245 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d64071a40 con 0x7f4d64135c20 2023-09-30T22:18:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.233+0000 7f4d627fc700 1 --2- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 0x7f4d6412ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d637fe700 1 --2- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d64138000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009e50 tx=0x7f4d5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0c613367523341 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d61ffb700 1 -- 172.21.15.89:0/1141456245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c00bbc0 con 0x7f4d64135c20 2023-09-30T22:18:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d61ffb700 1 -- 172.21.15.89:0/1141456245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c00bd20 con 0x7f4d64135c20 2023-09-30T22:18:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d61ffb700 1 -- 172.21.15.89:0/1141456245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0106c0 con 0x7f4d64135c20 2023-09-30T22:18:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d69806700 1 -- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 msgr2=0x7f4d64138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d69806700 1 --2- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d64138000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009e50 tx=0x7f4d5c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d69806700 1 -- 172.21.15.89:0/1141456245 shutdown_connections 2023-09-30T22:18:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d69806700 1 --2- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d64138000 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d69806700 1 --2- 172.21.15.89:0/1141456245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 0x7f4d6412ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.234+0000 7f4d69806700 1 -- 172.21.15.89:0/1141456245 >> 172.21.15.89:0/1141456245 conn(0x7f4d640a2aa0 msgr2=0x7f4d640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.235+0000 7f4d69806700 1 -- 172.21.15.89:0/1141456245 shutdown_connections 2023-09-30T22:18:25.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.235+0000 7f4d69806700 1 -- 172.21.15.89:0/1141456245 wait complete. 2023-09-30T22:18:25.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.236+0000 7f4d69806700 1 Processor -- start 2023-09-30T22:18:25.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.236+0000 7f4d69806700 1 -- start start 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d69806700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 0x7f4d641c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d69806700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d6412cf60 0x7f4d641c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d69806700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d641c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d69806700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6411a1a0 con 0x7f4d64135c20 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d69806700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6412bff0 con 0x7f4d6412cf60 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d69806700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64129fa0 con 0x7f4d6406f5d0 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d641c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d641c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45326/0 (socket says 172.21.15.89:45326) 2023-09-30T22:18:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d637fe700 1 -- 172.21.15.89:0/4133065997 learned_addr learned my addr 172.21.15.89:0/4133065997 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d637fe700 1 -- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 msgr2=0x7f4d641c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d637fe700 1 --2- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 0x7f4d641c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d637fe700 1 -- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d6412cf60 msgr2=0x7f4d641c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d637fe700 1 --2- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d6412cf60 0x7f4d641c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.237+0000 7f4d637fe700 1 -- 172.21.15.89:0/4133065997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c009b30 con 0x7f4d64135c20 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.238+0000 7f4d637fe700 1 --2- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d641c8d90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c014000 tx=0x7f4d5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.238+0000 7f4d4bfff700 1 -- 172.21.15.89:0/4133065997 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c010c80 con 0x7f4d64135c20 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.238+0000 7f4d4bfff700 1 -- 172.21.15.89:0/4133065997 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c00be90 con 0x7f4d64135c20 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.238+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6411abc0 con 0x7f4d64135c20 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.238+0000 7f4d4bfff700 1 -- 172.21.15.89:0/4133065997 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0099b0 con 0x7f4d64135c20 2023-09-30T22:18:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.238+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6409e160 con 0x7f4d64135c20 2023-09-30T22:18:25.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.239+0000 7f4d4bfff700 1 -- 172.21.15.89:0/4133065997 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d5c004040 con 0x7f4d64135c20 2023-09-30T22:18:25.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.239+0000 7f4d4bfff700 1 -- 172.21.15.89:0/4133065997 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4d5c00e040 con 0x7f4d64135c20 2023-09-30T22:18:25.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.239+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d64074ea0 con 0x7f4d64135c20 2023-09-30T22:18:25.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.248+0000 7f4d4bfff700 1 -- 172.21.15.89:0/4133065997 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d5c015080 con 0x7f4d64135c20 2023-09-30T22:18:25.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.273+0000 7fc67c99c700 1 Processor -- start 2023-09-30T22:18:25.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.273+0000 7fc67c99c700 1 -- start start 2023-09-30T22:18:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67c99c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc67809f410 0x7fc678136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67c99c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc678136fe0 0x7fc6781393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67c99c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc678070940 con 0x7fc67809e930 2023-09-30T22:18:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67c99c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67806fd60 con 0x7fc67809f410 2023-09-30T22:18:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67c99c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc67806fa80 con 0x7fc678136fe0 2023-09-30T22:18:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc675d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc67809f410 0x7fc678136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc67809e930 0x7fc67809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42218/0 (socket says 172.21.15.89:42218) 2023-09-30T22:18:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67659c700 1 -- 172.21.15.89:0/2454778301 learned_addr learned my addr 172.21.15.89:0/2454778301 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc675d9b700 1 -- 172.21.15.89:0/2454778301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc678136fe0 msgr2=0x7fc6781393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc675d9b700 1 --2- 172.21.15.89:0/2454778301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc678136fe0 0x7fc6781393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc675d9b700 1 -- 172.21.15.89:0/2454778301 >> v1:172.21.15.89:6789/0 conn(0x7fc67809e930 legacy=0x7fc67809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc675d9b700 1 -- 172.21.15.89:0/2454778301 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67809fb60 con 0x7fc67809f410 2023-09-30T22:18:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc675d9b700 1 --2- 172.21.15.89:0/2454778301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc67809f410 0x7fc678136aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009870 tx=0x7fc66c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db48a6c310596ec9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:25.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67559a700 1 -- 172.21.15.89:0/2454778301 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c011830 con 0x7fc67809f410 2023-09-30T22:18:25.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67559a700 1 -- 172.21.15.89:0/2454778301 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c011990 con 0x7fc67809f410 2023-09-30T22:18:25.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.274+0000 7fc67559a700 1 -- 172.21.15.89:0/2454778301 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c011d20 con 0x7fc67809f410 2023-09-30T22:18:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.275+0000 7fc67c99c700 1 -- 172.21.15.89:0/2454778301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc67809f410 msgr2=0x7fc678136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.275+0000 7fc67c99c700 1 --2- 172.21.15.89:0/2454778301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc67809f410 0x7fc678136aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009870 tx=0x7fc66c009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.275+0000 7fc67c99c700 1 -- 172.21.15.89:0/2454778301 shutdown_connections 2023-09-30T22:18:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.275+0000 7fc67c99c700 1 --2- 172.21.15.89:0/2454778301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc678136fe0 0x7fc6781393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.275+0000 7fc67c99c700 1 --2- 172.21.15.89:0/2454778301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc67809f410 0x7fc678136aa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.275+0000 7fc67c99c700 1 -- 172.21.15.89:0/2454778301 >> 172.21.15.89:0/2454778301 conn(0x7fc6780a4210 msgr2=0x7fc6780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.275+0000 7fc67c99c700 1 -- 172.21.15.89:0/2454778301 shutdown_connections 2023-09-30T22:18:25.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.275+0000 7fc67c99c700 1 -- 172.21.15.89:0/2454778301 wait complete. 2023-09-30T22:18:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f688d657700 1 Processor -- start 2023-09-30T22:18:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f688d657700 1 -- start start 2023-09-30T22:18:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f688d657700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a0a30 0x7f68880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f688d657700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68880a1340 0x7f6888133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f688d657700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688809edd0 con 0x7f6888133fa0 2023-09-30T22:18:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f688d657700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688809e1f0 con 0x7f68880a1340 2023-09-30T22:18:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f688d657700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f688809dc50 con 0x7f68880a0a30 2023-09-30T22:18:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f6886ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a0a30 0x7f68880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f6886ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a0a30 0x7f68880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52606/0 (socket says 172.21.15.89:52606) 2023-09-30T22:18:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.276+0000 7f6886ffd700 1 -- 172.21.15.89:0/2641615420 learned_addr learned my addr 172.21.15.89:0/2641615420 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f68867fc700 1 --2- 172.21.15.89:0/2641615420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68880a1340 0x7f6888133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f6886ffd700 1 -- 172.21.15.89:0/2641615420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68880a1340 msgr2=0x7f6888133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f6886ffd700 1 --2- 172.21.15.89:0/2641615420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68880a1340 0x7f6888133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f6886ffd700 1 -- 172.21.15.89:0/2641615420 >> v1:172.21.15.89:6789/0 conn(0x7f6888133fa0 legacy=0x7f688813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7fc67c99c700 1 Processor -- start 2023-09-30T22:18:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7fc67c99c700 1 -- start start 2023-09-30T22:18:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f6886ffd700 1 -- 172.21.15.89:0/2641615420 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880a1fe0 con 0x7f68880a0a30 2023-09-30T22:18:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f6886ffd700 1 --2- 172.21.15.89:0/2641615420 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a0a30 0x7f68880a0e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6878009920 tx=0x7f6878009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe26892faf5b0eb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f6885ffb700 1 -- 172.21.15.89:0/2641615420 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878011830 con 0x7f68880a0a30 2023-09-30T22:18:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f6885ffb700 1 -- 172.21.15.89:0/2641615420 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878011990 con 0x7f68880a0a30 2023-09-30T22:18:25.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.277+0000 7f6885ffb700 1 -- 172.21.15.89:0/2641615420 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878011d20 con 0x7f68880a0a30 2023-09-30T22:18:25.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67c99c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc67809e930 0x7fc67811b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67c99c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc67809f410 0x7fc67811be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67c99c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc678136fe0 0x7fc67811c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67c99c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67811d3b0 con 0x7fc67809e930 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67c99c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780a3140 con 0x7fc678136fe0 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67c99c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc67811ed20 con 0x7fc67809f410 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc67809e930 0x7fc67811b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc676d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc678136fe0 0x7fc67811c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc67809e930 0x7fc67811b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45332/0 (socket says 172.21.15.89:45332) 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc676d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc678136fe0 0x7fc67811c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48890/0 (socket says 172.21.15.89:48890) 2023-09-30T22:18:25.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc67659c700 1 -- 172.21.15.89:0/3673059094 learned_addr learned my addr 172.21.15.89:0/3673059094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7f688d657700 1 -- 172.21.15.89:0/2641615420 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a0a30 msgr2=0x7f68880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7f688d657700 1 --2- 172.21.15.89:0/2641615420 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a0a30 0x7f68880a0e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6878009920 tx=0x7f6878009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc676d9d700 1 -- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc67809f410 msgr2=0x7fc67811be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc676d9d700 1 --2- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc67809f410 0x7fc67811be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc676d9d700 1 -- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc67809e930 msgr2=0x7fc67811b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc676d9d700 1 --2- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc67809e930 0x7fc67811b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc676d9d700 1 -- 172.21.15.89:0/3673059094 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c0094a0 con 0x7fc678136fe0 2023-09-30T22:18:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7f688d657700 1 -- 172.21.15.89:0/2641615420 shutdown_connections 2023-09-30T22:18:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7f688d657700 1 --2- 172.21.15.89:0/2641615420 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a0a30 0x7f68880a0e00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7f688d657700 1 --2- 172.21.15.89:0/2641615420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68880a1340 0x7f6888133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7f688d657700 1 -- 172.21.15.89:0/2641615420 >> 172.21.15.89:0/2641615420 conn(0x7f68880a6550 msgr2=0x7f68880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7f688d657700 1 -- 172.21.15.89:0/2641615420 shutdown_connections 2023-09-30T22:18:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc676d9d700 1 --2- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc678136fe0 0x7fc67811c3b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc67000bdf0 tx=0x7fc670009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7fc666ffd700 1 -- 172.21.15.89:0/3673059094 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67000cbb0 con 0x7fc678136fe0 2023-09-30T22:18:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.279+0000 7fc666ffd700 1 -- 172.21.15.89:0/3673059094 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67000cd30 con 0x7fc678136fe0 2023-09-30T22:18:25.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.279+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc678071d20 con 0x7fc678136fe0 2023-09-30T22:18:25.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.279+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc678131670 con 0x7fc678136fe0 2023-09-30T22:18:25.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.279+0000 7fc666ffd700 1 -- 172.21.15.89:0/3673059094 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67000cea0 con 0x7fc678136fe0 2023-09-30T22:18:25.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.279+0000 7fc666ffd700 1 -- 172.21.15.89:0/3673059094 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc67001d3b0 con 0x7fc678136fe0 2023-09-30T22:18:25.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.278+0000 7f688d657700 1 -- 172.21.15.89:0/2641615420 wait complete. 2023-09-30T22:18:25.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.279+0000 7f688d657700 1 Processor -- start 2023-09-30T22:18:25.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f688d657700 1 -- start start 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f688d657700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68880a0a30 0x7f68881d5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f688d657700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a1340 0x7f68881d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f688d657700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6888133fa0 0x7f68881d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f688d657700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888133270 con 0x7f68880a0a30 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f688d657700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6888130c00 con 0x7f6888133fa0 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f688d657700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68880a0220 con 0x7f68880a1340 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f6886ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68880a0a30 0x7f68881d5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f6886ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68880a0a30 0x7f68881d5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45344/0 (socket says 172.21.15.89:45344) 2023-09-30T22:18:25.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f6886ffd700 1 -- 172.21.15.89:0/2218903328 learned_addr learned my addr 172.21.15.89:0/2218903328 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:25.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7fc666ffd700 1 -- 172.21.15.89:0/3673059094 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc670006e40 con 0x7fc678136fe0 2023-09-30T22:18:25.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c0052e0 con 0x7fc678136fe0 2023-09-30T22:18:25.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.280+0000 7f68867fc700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a1340 0x7f68881d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f68877fe700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6888133fa0 0x7f68881d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:25.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f6886ffd700 1 -- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a1340 msgr2=0x7f68881d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f6886ffd700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a1340 0x7f68881d5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f6886ffd700 1 -- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6888133fa0 msgr2=0x7f68881d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f6886ffd700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6888133fa0 0x7f68881d19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f6886ffd700 1 -- 172.21.15.89:0/2218903328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6878009550 con 0x7f68880a0a30 2023-09-30T22:18:25.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f6886ffd700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68880a0a30 0x7f68881d5670 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6878000c00 tx=0x7f6878004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:25.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f686f7fe700 1 -- 172.21.15.89:0/2218903328 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878004450 con 0x7f68880a0a30 2023-09-30T22:18:25.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f686f7fe700 1 -- 172.21.15.89:0/2218903328 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68780045b0 con 0x7f68880a0a30 2023-09-30T22:18:25.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f686f7fe700 1 -- 172.21.15.89:0/2218903328 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68780038c0 con 0x7f68880a0a30 2023-09-30T22:18:25.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68880a25e0 con 0x7f68880a0a30 2023-09-30T22:18:25.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.281+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6888132150 con 0x7f68880a0a30 2023-09-30T22:18:25.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.282+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68881330b0 con 0x7f68880a0a30 2023-09-30T22:18:25.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.285+0000 7fc666ffd700 1 -- 172.21.15.89:0/3673059094 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc670019080 con 0x7fc678136fe0 2023-09-30T22:18:25.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.286+0000 7f686f7fe700 1 -- 172.21.15.89:0/2218903328 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6878003a20 con 0x7f68880a0a30 2023-09-30T22:18:25.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.286+0000 7f686f7fe700 1 -- 172.21.15.89:0/2218903328 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6878021030 con 0x7f68880a0a30 2023-09-30T22:18:25.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.288+0000 7f686f7fe700 1 -- 172.21.15.89:0/2218903328 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6878003cd0 con 0x7f68880a0a30 2023-09-30T22:18:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.467+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f48ac06b9e0 con 0x7f48ac1374b0 2023-09-30T22:18:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.467+0000 7f48a97fa700 1 -- 172.21.15.89:0/2746206988 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f48a4009d70 con 0x7f48ac1374b0 2023-09-30T22:18:25.468 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:25.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.470+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac1374b0 msgr2=0x7f48ac133f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.470+0000 7f48b2cf3700 1 --2- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac1374b0 0x7f48ac133f60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48a400efc0 tx=0x7f48a400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.471+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 shutdown_connections 2023-09-30T22:18:25.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.471+0000 7f48b2cf3700 1 --2- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ac1374b0 0x7f48ac133f60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.471+0000 7f48b2cf3700 1 --2- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ac06e570 0x7f48ac1334e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.471+0000 7f48b2cf3700 1 --2- 172.21.15.89:0/2746206988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ac06f050 0x7f48ac133a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.471+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 >> 172.21.15.89:0/2746206988 conn(0x7f48ac0a2aa0 msgr2=0x7f48ac1311f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:25.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.471+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 shutdown_connections 2023-09-30T22:18:25.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.471+0000 7f48b2cf3700 1 -- 172.21.15.89:0/2746206988 wait complete. 2023-09-30T22:18:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.479+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc65c005590 con 0x7fc678136fe0 2023-09-30T22:18:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.480+0000 7fc666ffd700 1 -- 172.21.15.89:0/3673059094 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc67001c360 con 0x7fc678136fe0 2023-09-30T22:18:25.480 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc678136fe0 msgr2=0x7fc67811c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 --2- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc678136fe0 0x7fc67811c3b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc67000bdf0 tx=0x7fc670009ed0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 shutdown_connections 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 --2- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc678136fe0 0x7fc67811c3b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 --2- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc67809e930 0x7fc67811b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 --2- 172.21.15.89:0/3673059094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc67809f410 0x7fc67811be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 >> 172.21.15.89:0/3673059094 conn(0x7fc6780a4210 msgr2=0x7fc678136190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 shutdown_connections 2023-09-30T22:18:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.482+0000 7fc67c99c700 1 -- 172.21.15.89:0/3673059094 wait complete. 2023-09-30T22:18:25.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:25.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.493+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f68881330b0 con 0x7f68880a0a30 2023-09-30T22:18:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.493+0000 7f686f7fe700 1 -- 172.21.15.89:0/2218903328 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f687802b030 con 0x7f68880a0a30 2023-09-30T22:18:25.494 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:25.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:25.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68880a0a30 msgr2=0x7f68881d5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68880a0a30 0x7f68881d5670 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6878000c00 tx=0x7f6878004750 comp rx=0 tx=0).stop 2023-09-30T22:18:25.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 shutdown_connections 2023-09-30T22:18:25.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6888133fa0 0x7f68881d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68880a0a30 0x7f68881d5670 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 --2- 172.21.15.89:0/2218903328 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68880a1340 0x7f68881d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 >> 172.21.15.89:0/2218903328 conn(0x7f68880a6550 msgr2=0x7f688809abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:25.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 shutdown_connections 2023-09-30T22:18:25.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.495+0000 7f688d657700 1 -- 172.21.15.89:0/2218903328 wait complete. 2023-09-30T22:18:25.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.505+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4d640713a0 con 0x7f4d64135c20 2023-09-30T22:18:25.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.505+0000 7f4d4bfff700 1 -- 172.21.15.89:0/4133065997 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d5c019770 con 0x7f4d64135c20 2023-09-30T22:18:25.506 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:25.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 msgr2=0x7f4d641c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:25.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 --2- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d641c8d90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c014000 tx=0x7f4d5c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 shutdown_connections 2023-09-30T22:18:25.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 --2- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d64135c20 0x7f4d641c8d90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 --2- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d6406f5d0 0x7f4d641c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 --2- 172.21.15.89:0/4133065997 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d6412cf60 0x7f4d641c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:25.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 >> 172.21.15.89:0/4133065997 conn(0x7f4d640a2aa0 msgr2=0x7f4d6406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:25.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 shutdown_connections 2023-09-30T22:18:25.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:25.507+0000 7f4d69806700 1 -- 172.21.15.89:0/4133065997 wait complete. 2023-09-30T22:18:25.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:25.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:26.489 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:26.496 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:26.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:26.521 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:26.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.634+0000 7fc6d8f8c700 1 Processor -- start 2023-09-30T22:18:26.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.634+0000 7fc6d8f8c700 1 -- start start 2023-09-30T22:18:26.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d8f8c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d4130540 0x7fc6d4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d8f8c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6d406fcf0 0x7fc6d4138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d8f8c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d40716a0 con 0x7fc6d4130e50 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d8f8c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d409a390 con 0x7fc6d406fcf0 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d8f8c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4071290 con 0x7fc6d4130540 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d4130540 0x7fc6d4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d4130540 0x7fc6d4130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48908/0 (socket says 172.21.15.89:48908) 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d259c700 1 -- 172.21.15.89:0/1239038911 learned_addr learned my addr 172.21.15.89:0/1239038911 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d2d9d700 1 --2- 172.21.15.89:0/1239038911 >> v2:172.21.15.89:3300/0 conn(0x7fc6d406fcf0 0x7fc6d4138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d2d9d700 1 -- 172.21.15.89:0/1239038911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d4130540 msgr2=0x7fc6d4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:26.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d2d9d700 1 --2- 172.21.15.89:0/1239038911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d4130540 0x7fc6d4130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.635+0000 7fc6d2d9d700 1 -- 172.21.15.89:0/1239038911 >> v1:172.21.15.89:6789/0 conn(0x7fc6d4130e50 legacy=0x7fc6d406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.636+0000 7fc6d2d9d700 1 -- 172.21.15.89:0/1239038911 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d4130340 con 0x7fc6d406fcf0 2023-09-30T22:18:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.636+0000 7fc6d259c700 1 --2- 172.21.15.89:0/1239038911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d4130540 0x7fc6d4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.636+0000 7fc6d2d9d700 1 --2- 172.21.15.89:0/1239038911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d406fcf0 0x7fc6d4138570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc014a00 tx=0x7fc6cc014d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4165cf688091b763 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.636+0000 7fc6d159a700 1 -- 172.21.15.89:0/1239038911 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc014ea0 con 0x7fc6d406fcf0 2023-09-30T22:18:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.636+0000 7fc6d159a700 1 -- 172.21.15.89:0/1239038911 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc00fd50 con 0x7fc6d406fcf0 2023-09-30T22:18:26.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.636+0000 7fc6d159a700 1 -- 172.21.15.89:0/1239038911 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc003c60 con 0x7fc6d406fcf0 2023-09-30T22:18:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.637+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/1239038911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d406fcf0 msgr2=0x7fc6d4138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.637+0000 7fc6d8f8c700 1 --2- 172.21.15.89:0/1239038911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d406fcf0 0x7fc6d4138570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc014a00 tx=0x7fc6cc014d10 comp rx=0 tx=0).stop 2023-09-30T22:18:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.637+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/1239038911 shutdown_connections 2023-09-30T22:18:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.637+0000 7fc6d8f8c700 1 --2- 172.21.15.89:0/1239038911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d406fcf0 0x7fc6d4138570 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.637+0000 7fc6d8f8c700 1 --2- 172.21.15.89:0/1239038911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d4130540 0x7fc6d4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.637+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/1239038911 >> 172.21.15.89:0/1239038911 conn(0x7fc6d40a2aa0 msgr2=0x7fc6d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.637+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/1239038911 shutdown_connections 2023-09-30T22:18:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.637+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/1239038911 wait complete. 2023-09-30T22:18:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d8f8c700 1 Processor -- start 2023-09-30T22:18:26.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d8f8c700 1 -- start start 2023-09-30T22:18:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d8f8c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d406fcf0 0x7fc6d41c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d8f8c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d4130540 0x7fc6d41c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d8f8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6d4130e50 0x7fc6d4118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d8f8c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41c4af0 con 0x7fc6d4130540 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d8f8c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41c3c00 con 0x7fc6d406fcf0 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d8f8c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d411b260 con 0x7fc6d4130e50 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d406fcf0 0x7fc6d41c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d406fcf0 0x7fc6d41c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48914/0 (socket says 172.21.15.89:48914) 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.639+0000 7fc6d259c700 1 -- 172.21.15.89:0/4906751 learned_addr learned my addr 172.21.15.89:0/4906751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d1d9b700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d4130540 0x7fc6d41c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d259c700 1 -- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6d4130e50 msgr2=0x7fc6d4118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d2d9d700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6d4130e50 0x7fc6d4118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d259c700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6d4130e50 0x7fc6d4118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d259c700 1 -- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d4130540 msgr2=0x7fc6d41c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d259c700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d4130540 0x7fc6d41c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d259c700 1 -- 172.21.15.89:0/4906751 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c400d9f0 con 0x7fc6d406fcf0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d1d9b700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d4130540 0x7fc6d41c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d2d9d700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6d4130e50 0x7fc6d4118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d259c700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d406fcf0 0x7fc6d41c4070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400bef0 tx=0x7fc6c400d240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6c2ffd700 1 -- 172.21.15.89:0/4906751 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4008a40 con 0x7fc6d406fcf0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6c2ffd700 1 -- 172.21.15.89:0/4906751 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c4008ba0 con 0x7fc6d406fcf0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc014660 con 0x7fc6d406fcf0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6c2ffd700 1 -- 172.21.15.89:0/4906751 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4015400 con 0x7fc6d406fcf0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.640+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d411bbd0 con 0x7fc6d406fcf0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.641+0000 7fc6c2ffd700 1 -- 172.21.15.89:0/4906751 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6c4015560 con 0x7fc6d406fcf0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.641+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d4074ea0 con 0x7fc6d406fcf0 2023-09-30T22:18:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.641+0000 7fc6c2ffd700 1 -- 172.21.15.89:0/4906751 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc6c403f6b0 con 0x7fc6d406fcf0 2023-09-30T22:18:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.646+0000 7fc6c2ffd700 1 -- 172.21.15.89:0/4906751 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6c40136c0 con 0x7fc6d406fcf0 2023-09-30T22:18:26.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d1db81700 1 Processor -- start 2023-09-30T22:18:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d1db81700 1 -- start start 2023-09-30T22:18:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d1db81700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d18135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d1db81700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d18138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d1db81700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1809f130 con 0x7f7d1809d6f0 2023-09-30T22:18:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d1db81700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1809f520 con 0x7f7d18135e50 2023-09-30T22:18:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d1db81700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1809e930 con 0x7f7d1809e1d0 2023-09-30T22:18:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d177fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7d1809d6f0 0x7f7d1809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42246/0 (socket says 172.21.15.89:42246) 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d177fe700 1 -- 172.21.15.89:0/913613438 learned_addr learned my addr 172.21.15.89:0/913613438 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d17fff700 1 --2- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d18138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.671+0000 7f7d16ffd700 1 --2- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d18135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d17fff700 1 -- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 msgr2=0x7f7d18135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d17fff700 1 --2- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d18135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d17fff700 1 -- 172.21.15.89:0/913613438 >> v1:172.21.15.89:6789/0 conn(0x7f7d1809d6f0 legacy=0x7f7d1809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d17fff700 1 -- 172.21.15.89:0/913613438 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1809b2d0 con 0x7f7d18135e50 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d16ffd700 1 --2- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d18135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d17fff700 1 --2- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d18138230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009dc0 tx=0x7f7d1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc78214d13c7697b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d167fc700 1 -- 172.21.15.89:0/913613438 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000e070 con 0x7f7d18135e50 2023-09-30T22:18:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d167fc700 1 -- 172.21.15.89:0/913613438 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1000bb20 con 0x7f7d18135e50 2023-09-30T22:18:26.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.672+0000 7f7d167fc700 1 -- 172.21.15.89:0/913613438 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10012430 con 0x7f7d18135e50 2023-09-30T22:18:26.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.673+0000 7f7d1db81700 1 -- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 msgr2=0x7f7d18138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.673+0000 7f7d1db81700 1 --2- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d18138230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009dc0 tx=0x7f7d1000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.673+0000 7f7d1db81700 1 -- 172.21.15.89:0/913613438 shutdown_connections 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.673+0000 7f7d1db81700 1 --2- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d18138230 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.673+0000 7f7d1db81700 1 --2- 172.21.15.89:0/913613438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d18135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.673+0000 7f7d1db81700 1 -- 172.21.15.89:0/913613438 >> 172.21.15.89:0/913613438 conn(0x7f7d18066f50 msgr2=0x7f7d1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.673+0000 7f7d1db81700 1 -- 172.21.15.89:0/913613438 shutdown_connections 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.673+0000 7f7d1db81700 1 -- 172.21.15.89:0/913613438 wait complete. 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.674+0000 7f7d1db81700 1 Processor -- start 2023-09-30T22:18:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.674+0000 7f7d1db81700 1 -- start start 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d1db81700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d1809d6f0 0x7f7d181c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d1db81700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d181c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d1db81700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d181c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d1db81700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181303f0 con 0x7f7d1809d6f0 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d1db81700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180a1d00 con 0x7f7d18135e50 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d1db81700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1811f9d0 con 0x7f7d1809e1d0 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d16ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d181c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d16ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d181c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52646/0 (socket says 172.21.15.89:52646) 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d1809d6f0 0x7f7d181c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d16ffd700 1 -- 172.21.15.89:0/4112186483 learned_addr learned my addr 172.21.15.89:0/4112186483 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.675+0000 7f7d17fff700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d181c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d16ffd700 1 -- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 msgr2=0x7f7d181c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d16ffd700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d181c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d16ffd700 1 -- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d1809d6f0 msgr2=0x7f7d181c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d16ffd700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d1809d6f0 0x7f7d181c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d16ffd700 1 -- 172.21.15.89:0/4112186483 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c0094a0 con 0x7f7d1809e1d0 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d17fff700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d181c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d16ffd700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d181c87c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00ebe0 tx=0x7f7d0c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7cfffff700 1 -- 172.21.15.89:0/4112186483 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00c850 con 0x7f7d1809e1d0 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7cfffff700 1 -- 172.21.15.89:0/4112186483 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c00c9b0 con 0x7f7d1809e1d0 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7cfffff700 1 -- 172.21.15.89:0/4112186483 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00cd40 con 0x7f7d1809e1d0 2023-09-30T22:18:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d10009a70 con 0x7f7d1809e1d0 2023-09-30T22:18:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.676+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1811fc40 con 0x7f7d1809e1d0 2023-09-30T22:18:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.677+0000 7f7cfffff700 1 -- 172.21.15.89:0/4112186483 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d0c013470 con 0x7f7d1809e1d0 2023-09-30T22:18:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.677+0000 7f7cfffff700 1 -- 172.21.15.89:0/4112186483 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7d0c03f780 con 0x7f7d1809e1d0 2023-09-30T22:18:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.677+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18074ea0 con 0x7f7d1809e1d0 2023-09-30T22:18:26.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.682+0000 7f7cfffff700 1 -- 172.21.15.89:0/4112186483 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d0c013720 con 0x7f7d1809e1d0 2023-09-30T22:18:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.686+0000 7f9d3a05c700 1 Processor -- start 2023-09-30T22:18:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.686+0000 7f9d3a05c700 1 -- start start 2023-09-30T22:18:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.687+0000 7f9d3a05c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a1510 0x7f9d34138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.687+0000 7f9d3a05c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d3413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.687+0000 7f9d3a05c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3409edd0 con 0x7f9d340a0a30 2023-09-30T22:18:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.687+0000 7f9d3a05c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340a18e0 con 0x7f9d34139250 2023-09-30T22:18:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.687+0000 7f9d3a05c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3409e1f0 con 0x7f9d340a1510 2023-09-30T22:18:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.687+0000 7f9d337fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9d340a0a30 0x7f9d340a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42252/0 (socket says 172.21.15.89:42252) 2023-09-30T22:18:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.687+0000 7f9d337fe700 1 -- 172.21.15.89:0/2800722458 learned_addr learned my addr 172.21.15.89:0/2800722458 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.687+0000 7f9d33fff700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d3413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945561283 0 0) 0x7f9d3409edd0 con 0x7f9d340a0a30 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d1c0035b0 con 0x7f9d340a0a30 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933752021 0 0) 0x7f9d1c0035b0 con 0x7f9d340a0a30 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a1510 msgr2=0x7f9d34138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a1510 0x7f9d34138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 msgr2=0x7f9d3413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d3413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3409dc50 con 0x7f9d340a0a30 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7f9d24001670 con 0x7f9d340a0a30 2023-09-30T22:18:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d33fff700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d3413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 >> v1:172.21.15.89:6789/0 conn(0x7f9d340a0a30 legacy=0x7f9d340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d1c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a1510 0x7f9d1c004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340a18e0 con 0x7f9d1c003f40 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3409e1f0 con 0x7f9d34139250 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0053f0 con 0x7f9d340a1510 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d32ffd700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.688+0000 7f9d337fe700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d1c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d337fe700 1 -- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a1510 msgr2=0x7f9d1c004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d337fe700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a1510 0x7f9d1c004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d337fe700 1 -- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 msgr2=0x7f9d1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d337fe700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d337fe700 1 -- 172.21.15.89:0/2800722458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009a70 con 0x7f9d34139250 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d32ffd700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d1c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d337fe700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d1c003a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c190 tx=0x7f9d1c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c3fdffe1a16fde server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24007b50 con 0x7f9d34139250 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2400cd30 con 0x7f9d34139250 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.689+0000 7f9d327fc700 1 -- 172.21.15.89:0/2800722458 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400cea0 con 0x7f9d34139250 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 -- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 msgr2=0x7f9d1c003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d1c003a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c190 tx=0x7f9d1c005880 comp rx=0 tx=0).stop 2023-09-30T22:18:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 -- 172.21.15.89:0/2800722458 shutdown_connections 2023-09-30T22:18:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d1c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d34139250 0x7f9d1c003a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 --2- 172.21.15.89:0/2800722458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a1510 0x7f9d1c004d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 -- 172.21.15.89:0/2800722458 >> 172.21.15.89:0/2800722458 conn(0x7f9d340a6550 msgr2=0x7f9d340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 -- 172.21.15.89:0/2800722458 shutdown_connections 2023-09-30T22:18:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.690+0000 7f9d3a05c700 1 -- 172.21.15.89:0/2800722458 wait complete. 2023-09-30T22:18:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.691+0000 7f9d3a05c700 1 Processor -- start 2023-09-30T22:18:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d3a05c700 1 -- start start 2023-09-30T22:18:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d3a05c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d34122660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d3a05c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a0a30 0x7f9d34122ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d3a05c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d340a1510 0x7f9d341230e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d3a05c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d341241e0 con 0x7f9d1c003f40 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d3a05c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d34125b50 con 0x7f9d340a1510 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d3a05c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340a0220 con 0x7f9d340a0a30 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d33fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d340a1510 0x7f9d341230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d34122660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d33fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d340a1510 0x7f9d341230e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48964/0 (socket says 172.21.15.89:48964) 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.692+0000 7f9d33fff700 1 -- 172.21.15.89:0/1551177868 learned_addr learned my addr 172.21.15.89:0/1551177868 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f06984a5700 1 Processor -- start 2023-09-30T22:18:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f06984a5700 1 -- start start 2023-09-30T22:18:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d33fff700 1 -- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a0a30 msgr2=0x7f9d34122ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d33fff700 1 --2- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a0a30 0x7f9d34122ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d33fff700 1 -- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 msgr2=0x7f9d34122660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d33fff700 1 --2- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d34122660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d33fff700 1 -- 172.21.15.89:0/1551177868 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d341260f0 con 0x7f9d340a1510 2023-09-30T22:18:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d33fff700 1 --2- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d340a1510 0x7f9d341230e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00ef30 tx=0x7f9d2c00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d1bfff700 1 -- 172.21.15.89:0/1551177868 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c010040 con 0x7f9d340a1510 2023-09-30T22:18:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d1bfff700 1 -- 172.21.15.89:0/1551177868 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c00bd50 con 0x7f9d340a1510 2023-09-30T22:18:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d34124a10 con 0x7f9d340a1510 2023-09-30T22:18:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f06984a5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069006a770 0x7f069009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f06984a5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f069009dc30 0x7f069009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f06984a5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069009a810 con 0x7f069013c1e0 2023-09-30T22:18:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f06984a5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069006ab40 con 0x7f069009dc30 2023-09-30T22:18:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f06984a5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f069009ebc0 con 0x7f069006a770 2023-09-30T22:18:26.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.693+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d34125e10 con 0x7f9d340a1510 2023-09-30T22:18:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0696241700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069006a770 0x7f069009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0696241700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069006a770 0x7f069009d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52678/0 (socket says 172.21.15.89:52678) 2023-09-30T22:18:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0696241700 1 -- 172.21.15.89:0/717762122 learned_addr learned my addr 172.21.15.89:0/717762122 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0695a40700 1 --2- 172.21.15.89:0/717762122 >> v2:172.21.15.89:3300/0 conn(0x7f069009dc30 0x7f069009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0695a40700 1 -- 172.21.15.89:0/717762122 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069006a770 msgr2=0x7f069009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0695a40700 1 --2- 172.21.15.89:0/717762122 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069006a770 0x7f069009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0695a40700 1 -- 172.21.15.89:0/717762122 >> v1:172.21.15.89:6789/0 conn(0x7f069013c1e0 legacy=0x7f069013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0695a40700 1 -- 172.21.15.89:0/717762122 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069009e920 con 0x7f069009dc30 2023-09-30T22:18:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.694+0000 7f0695a40700 1 --2- 172.21.15.89:0/717762122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069009dc30 0x7f069009e080 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f067c009870 tx=0x7f067c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a221d2582ef3ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.695+0000 7f069523f700 1 -- 172.21.15.89:0/717762122 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c011830 con 0x7f069009dc30 2023-09-30T22:18:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.695+0000 7f069523f700 1 -- 172.21.15.89:0/717762122 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c011990 con 0x7f069009dc30 2023-09-30T22:18:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.695+0000 7f069523f700 1 -- 172.21.15.89:0/717762122 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c011d20 con 0x7f069009dc30 2023-09-30T22:18:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.695+0000 7f9d1bfff700 1 -- 172.21.15.89:0/1551177868 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c014640 con 0x7f9d340a1510 2023-09-30T22:18:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.695+0000 7f9d1bfff700 1 -- 172.21.15.89:0/1551177868 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d2c014860 con 0x7f9d340a1510 2023-09-30T22:18:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.695+0000 7f9d1bfff700 1 -- 172.21.15.89:0/1551177868 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d2c03ffa0 con 0x7f9d340a1510 2023-09-30T22:18:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.695+0000 7f06984a5700 1 -- 172.21.15.89:0/717762122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069009dc30 msgr2=0x7f069009e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.695+0000 7f06984a5700 1 --2- 172.21.15.89:0/717762122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069009dc30 0x7f069009e080 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f067c009870 tx=0x7f067c009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.696+0000 7f06984a5700 1 -- 172.21.15.89:0/717762122 shutdown_connections 2023-09-30T22:18:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.696+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d34126940 con 0x7f9d340a1510 2023-09-30T22:18:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.696+0000 7f06984a5700 1 --2- 172.21.15.89:0/717762122 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069006a770 0x7f069009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.696+0000 7f06984a5700 1 --2- 172.21.15.89:0/717762122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069009dc30 0x7f069009e080 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.696+0000 7f06984a5700 1 -- 172.21.15.89:0/717762122 >> 172.21.15.89:0/717762122 conn(0x7f0690066f50 msgr2=0x7f069009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.699+0000 7f06984a5700 1 -- 172.21.15.89:0/717762122 shutdown_connections 2023-09-30T22:18:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.699+0000 7f06984a5700 1 -- 172.21.15.89:0/717762122 wait complete. 2023-09-30T22:18:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.700+0000 7f06984a5700 1 Processor -- start 2023-09-30T22:18:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.700+0000 7f06984a5700 1 -- start start 2023-09-30T22:18:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.700+0000 7f9d1bfff700 1 -- 172.21.15.89:0/1551177868 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d2c018030 con 0x7f9d340a1510 2023-09-30T22:18:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f06984a5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069009dc30 0x7f06901cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f06984a5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069013c1e0 0x7f06901cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f06984a5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f069012a740 0x7f06901c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f06984a5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0690129540 con 0x7f069009dc30 2023-09-30T22:18:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f06984a5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069009f1a0 con 0x7f069012a740 2023-09-30T22:18:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f06984a5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0690129ca0 con 0x7f069013c1e0 2023-09-30T22:18:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f069012a740 0x7f06901c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f069012a740 0x7f06901c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:48968/0 (socket says 172.21.15.89:48968) 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 -- 172.21.15.89:0/4074093293 learned_addr learned my addr 172.21.15.89:0/4074093293 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 -- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069013c1e0 msgr2=0x7f06901cd410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 --2- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069013c1e0 0x7f06901cd410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 -- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069009dc30 msgr2=0x7f06901cced0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 --2- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069009dc30 0x7f06901cced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 -- 172.21.15.89:0/4074093293 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0094a0 con 0x7f069012a740 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.701+0000 7f0696a42700 1 --2- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f069012a740 0x7f06901c93b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f068800c4d0 tx=0x7f068800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.702+0000 7f068effd700 1 -- 172.21.15.89:0/4074093293 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880052d0 con 0x7f069012a740 2023-09-30T22:18:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.702+0000 7f068effd700 1 -- 172.21.15.89:0/4074093293 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688005430 con 0x7f069012a740 2023-09-30T22:18:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.702+0000 7f068effd700 1 -- 172.21.15.89:0/4074093293 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880038a0 con 0x7f069012a740 2023-09-30T22:18:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.705+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0690129080 con 0x7f069012a740 2023-09-30T22:18:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.705+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0690128510 con 0x7f069012a740 2023-09-30T22:18:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.706+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069006ac80 con 0x7f069012a740 2023-09-30T22:18:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.706+0000 7f068effd700 1 -- 172.21.15.89:0/4074093293 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0688003a00 con 0x7f069012a740 2023-09-30T22:18:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.706+0000 7f068effd700 1 -- 172.21.15.89:0/4074093293 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0688040a70 con 0x7f069012a740 2023-09-30T22:18:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.711+0000 7f068effd700 1 -- 172.21.15.89:0/4074093293 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0690128510 con 0x7f069012a740 2023-09-30T22:18:26.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.899+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc6d406b9e0 con 0x7fc6d406fcf0 2023-09-30T22:18:26.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.899+0000 7fc6c2ffd700 1 -- 172.21.15.89:0/4906751 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6c4013300 con 0x7fc6d406fcf0 2023-09-30T22:18:26.900 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:26.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.902+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d406fcf0 msgr2=0x7fc6d41c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.902+0000 7fc6d8f8c700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d406fcf0 0x7fc6d41c4070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400bef0 tx=0x7fc6c400d240 comp rx=0 tx=0).stop 2023-09-30T22:18:26.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.903+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 shutdown_connections 2023-09-30T22:18:26.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.903+0000 7fc6d8f8c700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6d4130e50 0x7fc6d4118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.903+0000 7fc6d8f8c700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6d406fcf0 0x7fc6d41c4070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.903+0000 7fc6d8f8c700 1 --2- 172.21.15.89:0/4906751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6d4130540 0x7fc6d41c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.903+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 >> 172.21.15.89:0/4906751 conn(0x7fc6d40a2aa0 msgr2=0x7fc6d412d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:26.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.903+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 shutdown_connections 2023-09-30T22:18:26.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.903+0000 7fc6d8f8c700 1 -- 172.21.15.89:0/4906751 wait complete. 2023-09-30T22:18:26.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:26.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.932+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7d1806c360 con 0x7f7d1809e1d0 2023-09-30T22:18:26.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.933+0000 7f7cfffff700 1 -- 172.21.15.89:0/4112186483 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d0c021030 con 0x7f7d1809e1d0 2023-09-30T22:18:26.934 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.935+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 msgr2=0x7f7d181c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.935+0000 7f7d1db81700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d181c87c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00ebe0 tx=0x7f7d0c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.935+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 shutdown_connections 2023-09-30T22:18:26.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.935+0000 7f7d1db81700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d18135e50 0x7f7d181c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.935+0000 7f7d1db81700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d1809d6f0 0x7f7d181c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.935+0000 7f7d1db81700 1 --2- 172.21.15.89:0/4112186483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1809e1d0 0x7f7d181c87c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.935+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 >> 172.21.15.89:0/4112186483 conn(0x7f7d18066f50 msgr2=0x7f7d180a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:26.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.935+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9d34074ea0 con 0x7f9d340a1510 2023-09-30T22:18:26.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.936+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 shutdown_connections 2023-09-30T22:18:26.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.936+0000 7f7d1db81700 1 -- 172.21.15.89:0/4112186483 wait complete. 2023-09-30T22:18:26.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.936+0000 7f9d1bfff700 1 -- 172.21.15.89:0/1551177868 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d2c01d400 con 0x7f9d340a1510 2023-09-30T22:18:26.939 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:26.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d340a1510 msgr2=0x7f9d341230e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 --2- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d340a1510 0x7f9d341230e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00ef30 tx=0x7f9d2c00b390 comp rx=0 tx=0).stop 2023-09-30T22:18:26.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 shutdown_connections 2023-09-30T22:18:26.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 --2- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d340a1510 0x7f9d341230e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 --2- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d1c003f40 0x7f9d34122660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 --2- 172.21.15.89:0/1551177868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d340a0a30 0x7f9d34122ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 >> 172.21.15.89:0/1551177868 conn(0x7f9d340a6550 msgr2=0x7f9d3411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:26.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 shutdown_connections 2023-09-30T22:18:26.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.939+0000 7f9d3a05c700 1 -- 172.21.15.89:0/1551177868 wait complete. 2023-09-30T22:18:26.950 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:26.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.951+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0690128510 con 0x7f069012a740 2023-09-30T22:18:26.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:26.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.954+0000 7f068effd700 1 -- 172.21.15.89:0/4074093293 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0690128510 con 0x7f069012a740 2023-09-30T22:18:26.955 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f069012a740 msgr2=0x7f06901c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 --2- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f069012a740 0x7f06901c93b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f068800c4d0 tx=0x7f068800c890 comp rx=0 tx=0).stop 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 shutdown_connections 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 --2- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f069012a740 0x7f06901c93b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 --2- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069009dc30 0x7f06901cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 --2- 172.21.15.89:0/4074093293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f069013c1e0 0x7f06901cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 >> 172.21.15.89:0/4074093293 conn(0x7f0690066f50 msgr2=0x7f069013b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 shutdown_connections 2023-09-30T22:18:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:26.957+0000 7f06984a5700 1 -- 172.21.15.89:0/4074093293 wait complete. 2023-09-30T22:18:26.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:27.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:27.951 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:27.954 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:27.970 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.071+0000 7ff734569700 1 Processor -- start 2023-09-30T22:18:28.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.071+0000 7ff734569700 1 -- start start 2023-09-30T22:18:28.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff734569700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09e1d0 0x7ff72c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff734569700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c135e50 0x7ff72c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff734569700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c09f130 con 0x7ff72c09d6f0 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff734569700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c09f520 con 0x7ff72c135e50 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff734569700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c09e930 con 0x7ff72c09e1d0 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff732305700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff72c09d6f0 0x7ff72c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42272/0 (socket says 172.21.15.89:42272) 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff732305700 1 -- 172.21.15.89:0/4246767679 learned_addr learned my addr 172.21.15.89:0/4246767679 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff732b06700 1 --2- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c135e50 0x7ff72c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff731b04700 1 --2- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09e1d0 0x7ff72c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff732b06700 1 -- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09e1d0 msgr2=0x7ff72c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff732b06700 1 --2- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09e1d0 0x7ff72c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff732b06700 1 -- 172.21.15.89:0/4246767679 >> v1:172.21.15.89:6789/0 conn(0x7ff72c09d6f0 legacy=0x7ff72c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff732b06700 1 -- 172.21.15.89:0/4246767679 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c09b2d0 con 0x7ff72c135e50 2023-09-30T22:18:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.072+0000 7ff732b06700 1 --2- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c135e50 0x7ff72c138230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff724009df0 tx=0x7ff72400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79854342d1f27f43 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff731303700 1 -- 172.21.15.89:0/4246767679 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72400e070 con 0x7ff72c135e50 2023-09-30T22:18:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff731303700 1 -- 172.21.15.89:0/4246767679 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72400bb90 con 0x7ff72c135e50 2023-09-30T22:18:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff731303700 1 -- 172.21.15.89:0/4246767679 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724012430 con 0x7ff72c135e50 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff734569700 1 -- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c135e50 msgr2=0x7ff72c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff734569700 1 --2- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c135e50 0x7ff72c138230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff724009df0 tx=0x7ff72400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff734569700 1 -- 172.21.15.89:0/4246767679 shutdown_connections 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff734569700 1 --2- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c135e50 0x7ff72c138230 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff734569700 1 --2- 172.21.15.89:0/4246767679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09e1d0 0x7ff72c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff734569700 1 -- 172.21.15.89:0/4246767679 >> 172.21.15.89:0/4246767679 conn(0x7ff72c066f50 msgr2=0x7ff72c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.073+0000 7ff734569700 1 -- 172.21.15.89:0/4246767679 shutdown_connections 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.074+0000 7ff734569700 1 -- 172.21.15.89:0/4246767679 wait complete. 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.074+0000 7ff734569700 1 Processor -- start 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.074+0000 7ff734569700 1 -- start start 2023-09-30T22:18:28.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff734569700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09d6f0 0x7ff72c11b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff734569700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09e1d0 0x7ff72c11bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff734569700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c135e50 0x7ff72c11c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff734569700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c138b70 con 0x7ff72c09e1d0 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff734569700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c11d190 con 0x7ff72c09d6f0 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff734569700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c11da20 con 0x7ff72c135e50 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732b06700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c135e50 0x7ff72c11c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732305700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09d6f0 0x7ff72c11b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732b06700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c135e50 0x7ff72c11c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52700/0 (socket says 172.21.15.89:52700) 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732b06700 1 -- 172.21.15.89:0/1165545011 learned_addr learned my addr 172.21.15.89:0/1165545011 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff731b04700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09e1d0 0x7ff72c11bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732b06700 1 -- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09d6f0 msgr2=0x7ff72c11b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732b06700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09d6f0 0x7ff72c11b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732b06700 1 -- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09e1d0 msgr2=0x7ff72c11bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732b06700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09e1d0 0x7ff72c11bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732b06700 1 -- 172.21.15.89:0/1165545011 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724009a70 con 0x7ff72c135e50 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.075+0000 7ff732305700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09d6f0 0x7ff72c11b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.076+0000 7ff732b06700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c135e50 0x7ff72c11c000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff724013000 tx=0x7ff724004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.076+0000 7ff722ffd700 1 -- 172.21.15.89:0/1165545011 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724012430 con 0x7ff72c135e50 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.076+0000 7ff722ffd700 1 -- 172.21.15.89:0/1165545011 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff724024070 con 0x7ff72c135e50 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.076+0000 7ff722ffd700 1 -- 172.21.15.89:0/1165545011 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724020810 con 0x7ff72c135e50 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.076+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c09c700 con 0x7ff72c135e50 2023-09-30T22:18:28.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.076+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c11edc0 con 0x7ff72c135e50 2023-09-30T22:18:28.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.077+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c071700 con 0x7ff72c135e50 2023-09-30T22:18:28.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.078+0000 7ff722ffd700 1 -- 172.21.15.89:0/1165545011 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff724020970 con 0x7ff72c135e50 2023-09-30T22:18:28.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.078+0000 7ff722ffd700 1 -- 172.21.15.89:0/1165545011 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff724026070 con 0x7ff72c135e50 2023-09-30T22:18:28.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.082+0000 7ff722ffd700 1 -- 172.21.15.89:0/1165545011 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff724004040 con 0x7ff72c135e50 2023-09-30T22:18:28.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48e11b6700 1 Processor -- start 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48e11b6700 1 -- start start 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48e11b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc06c0e0 0x7f48dc09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48e11b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc134ec0 0x7f48dc1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48e11b6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48dc137f20 0x7f48dc138390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48e11b6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc06c4b0 con 0x7f48dc137f20 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48e11b6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0a09c0 con 0x7f48dc06c0e0 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48e11b6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc134980 con 0x7f48dc134ec0 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48da59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc134ec0 0x7f48dc1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48da59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc134ec0 0x7f48dc1379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52716/0 (socket says 172.21.15.89:52716) 2023-09-30T22:18:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48da59c700 1 -- 172.21.15.89:0/4141324888 learned_addr learned my addr 172.21.15.89:0/4141324888 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48db59e700 1 --2- 172.21.15.89:0/4141324888 >> v2:172.21.15.89:3300/0 conn(0x7f48dc137f20 0x7f48dc138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.099+0000 7f48dad9d700 1 --2- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc06c0e0 0x7f48dc09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48dad9d700 1 -- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc134ec0 msgr2=0x7f48dc1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48dad9d700 1 --2- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc134ec0 0x7f48dc1379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48dad9d700 1 -- 172.21.15.89:0/4141324888 >> v2:172.21.15.89:3300/0 conn(0x7f48dc137f20 msgr2=0x7f48dc138390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48dad9d700 1 --2- 172.21.15.89:0/4141324888 >> v2:172.21.15.89:3300/0 conn(0x7f48dc137f20 0x7f48dc138390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48dad9d700 1 -- 172.21.15.89:0/4141324888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0a0600 con 0x7f48dc06c0e0 2023-09-30T22:18:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48da59c700 1 --2- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc134ec0 0x7f48dc1379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48db59e700 1 --2- 172.21.15.89:0/4141324888 >> v2:172.21.15.89:3300/0 conn(0x7f48dc137f20 0x7f48dc138390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48dad9d700 1 --2- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc06c0e0 0x7f48dc09ac90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009870 tx=0x7f48cc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3a7d62351370b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48d9d9b700 1 -- 172.21.15.89:0/4141324888 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc011830 con 0x7f48dc06c0e0 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48d9d9b700 1 -- 172.21.15.89:0/4141324888 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc011990 con 0x7f48dc06c0e0 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.100+0000 7f48d9d9b700 1 -- 172.21.15.89:0/4141324888 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc011d20 con 0x7f48dc06c0e0 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 -- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc06c0e0 msgr2=0x7f48dc09ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 --2- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc06c0e0 0x7f48dc09ac90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009870 tx=0x7f48cc009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 -- 172.21.15.89:0/4141324888 shutdown_connections 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 --2- 172.21.15.89:0/4141324888 >> v2:172.21.15.89:3300/0 conn(0x7f48dc137f20 0x7f48dc138390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 --2- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc06c0e0 0x7f48dc09ac90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 --2- 172.21.15.89:0/4141324888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc134ec0 0x7f48dc1379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 -- 172.21.15.89:0/4141324888 >> 172.21.15.89:0/4141324888 conn(0x7f48dc0a4210 msgr2=0x7f48dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 -- 172.21.15.89:0/4141324888 shutdown_connections 2023-09-30T22:18:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.101+0000 7f48e11b6700 1 -- 172.21.15.89:0/4141324888 wait complete. 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.102+0000 7f48e11b6700 1 Processor -- start 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.102+0000 7f48e11b6700 1 -- start start 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.102+0000 7f48e11b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48dc06c0e0 0x7f48dc133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48e11b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc134ec0 0x7f48dc11aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48e11b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc137f20 0x7f48dc11b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48e11b6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc11bf90 con 0x7f48dc06c0e0 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48e11b6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc11caf0 con 0x7f48dc134ec0 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48e11b6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0a1380 con 0x7f48dc137f20 2023-09-30T22:18:28.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48dc06c0e0 0x7f48dc133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48dc06c0e0 0x7f48dc133f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45412/0 (socket says 172.21.15.89:45412) 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 -- 172.21.15.89:0/4178104449 learned_addr learned my addr 172.21.15.89:0/4178104449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48da59c700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc134ec0 0x7f48dc11aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48db59e700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc137f20 0x7f48dc11b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 -- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc137f20 msgr2=0x7f48dc11b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc137f20 0x7f48dc11b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 -- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc134ec0 msgr2=0x7f48dc11aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc134ec0 0x7f48dc11aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 -- 172.21.15.89:0/4178104449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0009550 con 0x7f48dc06c0e0 2023-09-30T22:18:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48dad9d700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48dc06c0e0 0x7f48dc133f20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f48cc014040 tx=0x7f48cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:28.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.103+0000 7f48c37fe700 1 -- 172.21.15.89:0/4178104449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc004450 con 0x7f48dc06c0e0 2023-09-30T22:18:28.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.104+0000 7f48c37fe700 1 -- 172.21.15.89:0/4178104449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc01e070 con 0x7f48dc06c0e0 2023-09-30T22:18:28.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.104+0000 7f48c37fe700 1 -- 172.21.15.89:0/4178104449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc0038c0 con 0x7f48dc06c0e0 2023-09-30T22:18:28.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.104+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d4009a70 con 0x7f48dc06c0e0 2023-09-30T22:18:28.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.104+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc0094a0 con 0x7f48dc06c0e0 2023-09-30T22:18:28.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.105+0000 7f48c37fe700 1 -- 172.21.15.89:0/4178104449 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48cc003a20 con 0x7f48dc06c0e0 2023-09-30T22:18:28.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.105+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc138f70 con 0x7f48dc06c0e0 2023-09-30T22:18:28.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.105+0000 7f48c37fe700 1 -- 172.21.15.89:0/4178104449 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f48cc040320 con 0x7f48dc06c0e0 2023-09-30T22:18:28.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.110+0000 7f48c37fe700 1 -- 172.21.15.89:0/4178104449 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48cc01a420 con 0x7f48dc06c0e0 2023-09-30T22:18:28.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.113+0000 7f014846b700 1 Processor -- start 2023-09-30T22:18:28.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.113+0000 7f014846b700 1 -- start start 2023-09-30T22:18:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.113+0000 7f014846b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130540 0x7f0140130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.113+0000 7f014846b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0140130e50 0x7f014006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.113+0000 7f014846b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f014006fb20 0x7f014006ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f014846b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014009a390 con 0x7f014006fb20 2023-09-30T22:18:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f014846b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01400716a0 con 0x7f0140130e50 2023-09-30T22:18:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f014846b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0140071290 con 0x7f0140130540 2023-09-30T22:18:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0146207700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130540 0x7f0140130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0146207700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130540 0x7f0140130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52720/0 (socket says 172.21.15.89:52720) 2023-09-30T22:18:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0146207700 1 -- 172.21.15.89:0/2964393101 learned_addr learned my addr 172.21.15.89:0/2964393101 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0145a06700 1 --2- 172.21.15.89:0/2964393101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0140130e50 0x7f014006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0146a08700 1 --2- 172.21.15.89:0/2964393101 >> v2:172.21.15.89:3300/0 conn(0x7f014006fb20 0x7f014006ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0145a06700 1 -- 172.21.15.89:0/2964393101 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130540 msgr2=0x7f0140130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0145a06700 1 --2- 172.21.15.89:0/2964393101 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130540 0x7f0140130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0145a06700 1 -- 172.21.15.89:0/2964393101 >> v2:172.21.15.89:3300/0 conn(0x7f014006fb20 msgr2=0x7f014006ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0145a06700 1 --2- 172.21.15.89:0/2964393101 >> v2:172.21.15.89:3300/0 conn(0x7f014006fb20 0x7f014006ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0145a06700 1 -- 172.21.15.89:0/2964393101 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140130340 con 0x7f0140130e50 2023-09-30T22:18:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0145a06700 1 --2- 172.21.15.89:0/2964393101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0140130e50 0x7f014006f5e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0130009870 tx=0x7f0130009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5cf4dd082c1eda2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.114+0000 7f0145205700 1 -- 172.21.15.89:0/2964393101 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130011830 con 0x7f0140130e50 2023-09-30T22:18:28.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f0145205700 1 -- 172.21.15.89:0/2964393101 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0130011990 con 0x7f0140130e50 2023-09-30T22:18:28.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f0145205700 1 -- 172.21.15.89:0/2964393101 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130011d20 con 0x7f0140130e50 2023-09-30T22:18:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 -- 172.21.15.89:0/2964393101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0140130e50 msgr2=0x7f014006f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 --2- 172.21.15.89:0/2964393101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0140130e50 0x7f014006f5e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0130009870 tx=0x7f0130009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 -- 172.21.15.89:0/2964393101 shutdown_connections 2023-09-30T22:18:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 --2- 172.21.15.89:0/2964393101 >> v2:172.21.15.89:3300/0 conn(0x7f014006fb20 0x7f014006ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 --2- 172.21.15.89:0/2964393101 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130540 0x7f0140130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 --2- 172.21.15.89:0/2964393101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0140130e50 0x7f014006f5e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 -- 172.21.15.89:0/2964393101 >> 172.21.15.89:0/2964393101 conn(0x7f01400a2aa0 msgr2=0x7f01400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 -- 172.21.15.89:0/2964393101 shutdown_connections 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.115+0000 7f014846b700 1 -- 172.21.15.89:0/2964393101 wait complete. 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.117+0000 7f014846b700 1 Processor -- start 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.117+0000 7f014846b700 1 -- start start 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.117+0000 7f014846b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f014006fb20 0x7f014011a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.117+0000 7f014846b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0140130540 0x7f014011acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.117+0000 7f014846b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130e50 0x7f014011b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.117+0000 7f014846b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014011dc80 con 0x7f0140130540 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f014846b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014011c420 con 0x7f014006fb20 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f014846b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f014011bbb0 con 0x7f0140130e50 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0140130540 0x7f014011acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0140130540 0x7f014011acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45432/0 (socket says 172.21.15.89:45432) 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 -- 172.21.15.89:0/3993801713 learned_addr learned my addr 172.21.15.89:0/3993801713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0146a08700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130e50 0x7f014011b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0146207700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f014006fb20 0x7f014011a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 -- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130e50 msgr2=0x7f014011b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130e50 0x7f014011b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 -- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f014006fb20 msgr2=0x7f014011a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f014006fb20 0x7f014011a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 -- 172.21.15.89:0/3993801713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300094a0 con 0x7f0140130540 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0145a06700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0140130540 0x7f014011acd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0130000c00 tx=0x7f0130004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.118+0000 7f0136ffd700 1 -- 172.21.15.89:0/3993801713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130004510 con 0x7f0140130540 2023-09-30T22:18:28.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.119+0000 7f0136ffd700 1 -- 172.21.15.89:0/3993801713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01300036a0 con 0x7f0140130540 2023-09-30T22:18:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.119+0000 7f0136ffd700 1 -- 172.21.15.89:0/3993801713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130003a30 con 0x7f0140130540 2023-09-30T22:18:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.119+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014011cf80 con 0x7f0140130540 2023-09-30T22:18:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.119+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014011cb00 con 0x7f0140130540 2023-09-30T22:18:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.120+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01400a0a70 con 0x7f0140130540 2023-09-30T22:18:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.121+0000 7f0136ffd700 1 -- 172.21.15.89:0/3993801713 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0130003b90 con 0x7f0140130540 2023-09-30T22:18:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.121+0000 7f0136ffd700 1 -- 172.21.15.89:0/3993801713 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0130040770 con 0x7f0140130540 2023-09-30T22:18:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.129+0000 7f0136ffd700 1 -- 172.21.15.89:0/3993801713 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f013001a420 con 0x7f0140130540 2023-09-30T22:18:28.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.167+0000 7faa69d94700 1 Processor -- start 2023-09-30T22:18:28.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.168+0000 7faa69d94700 1 -- start start 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.168+0000 7faa69d94700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa640a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.168+0000 7faa69d94700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa640a0f70 0x7faa640a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.168+0000 7faa69d94700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa6413e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.168+0000 7faa69d94700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6406d060 con 0x7faa640a0f70 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.168+0000 7faa69d94700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa640a2550 con 0x7faa6406cc90 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.168+0000 7faa69d94700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa640a1e00 con 0x7faa6413c120 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa640a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa63fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa6413e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa640a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49036/0 (socket says 172.21.15.89:49036) 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa63fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa6413e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52742/0 (socket says 172.21.15.89:52742) 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 -- 172.21.15.89:0/4095514672 learned_addr learned my addr 172.21.15.89:0/4095514672 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa62ffd700 1 --2- 172.21.15.89:0/4095514672 >> v2:172.21.15.89:3300/0 conn(0x7faa640a0f70 0x7faa640a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 -- 172.21.15.89:0/4095514672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 msgr2=0x7faa6413e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 --2- 172.21.15.89:0/4095514672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa6413e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 -- 172.21.15.89:0/4095514672 >> v2:172.21.15.89:3300/0 conn(0x7faa640a0f70 msgr2=0x7faa640a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 --2- 172.21.15.89:0/4095514672 >> v2:172.21.15.89:3300/0 conn(0x7faa640a0f70 0x7faa640a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 -- 172.21.15.89:0/4095514672 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6409cfa0 con 0x7faa6406cc90 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa637fe700 1 --2- 172.21.15.89:0/4095514672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa640a0a30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa540098f0 tx=0x7faa54009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba179276dac60779 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa627fc700 1 -- 172.21.15.89:0/4095514672 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54011830 con 0x7faa6406cc90 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa627fc700 1 -- 172.21.15.89:0/4095514672 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa54011990 con 0x7faa6406cc90 2023-09-30T22:18:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.169+0000 7faa627fc700 1 -- 172.21.15.89:0/4095514672 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54011d20 con 0x7faa6406cc90 2023-09-30T22:18:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 -- 172.21.15.89:0/4095514672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 msgr2=0x7faa640a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 --2- 172.21.15.89:0/4095514672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa640a0a30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa540098f0 tx=0x7faa54009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 -- 172.21.15.89:0/4095514672 shutdown_connections 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 --2- 172.21.15.89:0/4095514672 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa6413e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 --2- 172.21.15.89:0/4095514672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa640a0a30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 --2- 172.21.15.89:0/4095514672 >> v2:172.21.15.89:3300/0 conn(0x7faa640a0f70 0x7faa640a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 -- 172.21.15.89:0/4095514672 >> 172.21.15.89:0/4095514672 conn(0x7faa640a3050 msgr2=0x7faa640a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 -- 172.21.15.89:0/4095514672 shutdown_connections 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.170+0000 7faa69d94700 1 -- 172.21.15.89:0/4095514672 wait complete. 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.172+0000 7faa69d94700 1 Processor -- start 2023-09-30T22:18:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.172+0000 7faa69d94700 1 -- start start 2023-09-30T22:18:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa69d94700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa64126eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa69d94700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640a0f70 0x7faa641273f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa69d94700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa641cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa69d94700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa641289c0 con 0x7faa640a0f70 2023-09-30T22:18:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa69d94700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa64129b00 con 0x7faa6406cc90 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa69d94700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa6413ae50 con 0x7faa6413c120 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa64126eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa64126eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49040/0 (socket says 172.21.15.89:49040) 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa637fe700 1 -- 172.21.15.89:0/2103218186 learned_addr learned my addr 172.21.15.89:0/2103218186 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa62ffd700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640a0f70 0x7faa641273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa63fff700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa641cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa637fe700 1 -- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 msgr2=0x7faa641cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa637fe700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa641cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa637fe700 1 -- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640a0f70 msgr2=0x7faa641273f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa637fe700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640a0f70 0x7faa641273f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa637fe700 1 -- 172.21.15.89:0/2103218186 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa54009550 con 0x7faa6406cc90 2023-09-30T22:18:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.173+0000 7faa62ffd700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640a0f70 0x7faa641273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.174+0000 7faa637fe700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa64126eb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa54009c40 tx=0x7faa54004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.174+0000 7faa4bfff700 1 -- 172.21.15.89:0/2103218186 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54004510 con 0x7faa6406cc90 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.174+0000 7faa4bfff700 1 -- 172.21.15.89:0/2103218186 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5401e070 con 0x7faa6406cc90 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.174+0000 7faa4bfff700 1 -- 172.21.15.89:0/2103218186 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa540038e0 con 0x7faa6406cc90 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.174+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa64129d00 con 0x7faa6406cc90 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.174+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa64128130 con 0x7faa6406cc90 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.175+0000 7faa4bfff700 1 -- 172.21.15.89:0/2103218186 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa54003a40 con 0x7faa6406cc90 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.175+0000 7faa4bfff700 1 -- 172.21.15.89:0/2103218186 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa54044170 con 0x7faa6406cc90 2023-09-30T22:18:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.175+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa64071e20 con 0x7faa6406cc90 2023-09-30T22:18:28.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.184+0000 7faa4bfff700 1 -- 172.21.15.89:0/2103218186 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa54015080 con 0x7faa6406cc90 2023-09-30T22:18:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.314+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f48dc074ea0 con 0x7f48dc06c0e0 2023-09-30T22:18:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.315+0000 7f48c37fe700 1 -- 172.21.15.89:0/4178104449 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f48cc015020 con 0x7f48dc06c0e0 2023-09-30T22:18:28.316 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.317+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48dc06c0e0 msgr2=0x7f48dc133f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.317+0000 7f48e11b6700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48dc06c0e0 0x7f48dc133f20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f48cc014040 tx=0x7f48cc004750 comp rx=0 tx=0).stop 2023-09-30T22:18:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.317+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 shutdown_connections 2023-09-30T22:18:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.317+0000 7f48e11b6700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48dc137f20 0x7f48dc11b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.317+0000 7f48e11b6700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48dc06c0e0 0x7f48dc133f20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.317+0000 7f48e11b6700 1 --2- 172.21.15.89:0/4178104449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48dc134ec0 0x7f48dc11aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.317+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 >> 172.21.15.89:0/4178104449 conn(0x7f48dc0a4210 msgr2=0x7f48dc099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.318+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 shutdown_connections 2023-09-30T22:18:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.321+0000 7f48e11b6700 1 -- 172.21.15.89:0/4178104449 wait complete. 2023-09-30T22:18:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.325+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff72c074ea0 con 0x7ff72c135e50 2023-09-30T22:18:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.326+0000 7ff722ffd700 1 -- 172.21.15.89:0/1165545011 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff724039090 con 0x7ff72c135e50 2023-09-30T22:18:28.327 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c135e50 msgr2=0x7ff72c11c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c135e50 0x7ff72c11c000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff724013000 tx=0x7ff724004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 shutdown_connections 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c135e50 0x7ff72c11c000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09d6f0 0x7ff72c11b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 --2- 172.21.15.89:0/1165545011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09e1d0 0x7ff72c11bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 >> 172.21.15.89:0/1165545011 conn(0x7ff72c066f50 msgr2=0x7ff72c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 shutdown_connections 2023-09-30T22:18:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.328+0000 7ff734569700 1 -- 172.21.15.89:0/1165545011 wait complete. 2023-09-30T22:18:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.330+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0140074ea0 con 0x7f0140130540 2023-09-30T22:18:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.331+0000 7f0136ffd700 1 -- 172.21.15.89:0/3993801713 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0130015030 con 0x7f0140130540 2023-09-30T22:18:28.332 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:28.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.334+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0140130540 msgr2=0x7f014011acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.334+0000 7f014846b700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0140130540 0x7f014011acd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0130000c00 tx=0x7f0130004750 comp rx=0 tx=0).stop 2023-09-30T22:18:28.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:28.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.336+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 shutdown_connections 2023-09-30T22:18:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.336+0000 7f014846b700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0140130e50 0x7f014011b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.336+0000 7f014846b700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f014006fb20 0x7f014011a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.336+0000 7f014846b700 1 --2- 172.21.15.89:0/3993801713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0140130540 0x7f014011acd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.336+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 >> 172.21.15.89:0/3993801713 conn(0x7f01400a2aa0 msgr2=0x7f014012d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.336+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 shutdown_connections 2023-09-30T22:18:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.336+0000 7f014846b700 1 -- 172.21.15.89:0/3993801713 wait complete. 2023-09-30T22:18:28.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:28.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.407+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faa640a0070 con 0x7faa6406cc90 2023-09-30T22:18:28.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.407+0000 7faa4bfff700 1 -- 172.21.15.89:0/2103218186 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa64128130 con 0x7faa6406cc90 2023-09-30T22:18:28.408 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 msgr2=0x7faa64126eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa64126eb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa54009c40 tx=0x7faa54004750 comp rx=0 tx=0).stop 2023-09-30T22:18:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 shutdown_connections 2023-09-30T22:18:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6413c120 0x7faa641cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa6406cc90 0x7faa64126eb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 --2- 172.21.15.89:0/2103218186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640a0f70 0x7faa641273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 >> 172.21.15.89:0/2103218186 conn(0x7faa640a3050 msgr2=0x7faa6413b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 shutdown_connections 2023-09-30T22:18:28.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:28.409+0000 7faa69d94700 1 -- 172.21.15.89:0/2103218186 wait complete. 2023-09-30T22:18:28.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:29.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:29.343 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:29.350 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:29.423 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.479+0000 7f53d849c700 1 Processor -- start 2023-09-30T22:18:31.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.479+0000 7f53d849c700 1 -- start start 2023-09-30T22:18:31.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d849c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d009e850 0x7f53d009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d849c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d0138e30 0x7f53d013b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d849c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d00a06b0 con 0x7f53d009ffb0 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d849c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d006b920 con 0x7f53d0138e30 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d849c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53d009aa30 con 0x7f53d009e850 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d6238700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d009e850 0x7f53d009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d6238700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d009e850 0x7f53d009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52762/0 (socket says 172.21.15.89:52762) 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d6238700 1 -- 172.21.15.89:0/2825930386 learned_addr learned my addr 172.21.15.89:0/2825930386 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.480+0000 7f53d6a39700 1 --2- 172.21.15.89:0/2825930386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d0138e30 0x7f53d013b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d6238700 1 -- 172.21.15.89:0/2825930386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d0138e30 msgr2=0x7f53d013b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d6238700 1 --2- 172.21.15.89:0/2825930386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d0138e30 0x7f53d013b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d6238700 1 -- 172.21.15.89:0/2825930386 >> v1:172.21.15.89:6789/0 conn(0x7f53d009ffb0 legacy=0x7f53d009f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d6238700 1 -- 172.21.15.89:0/2825930386 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d006caa0 con 0x7f53d009e850 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d5236700 1 -- 172.21.15.89:0/2825930386 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906456841 0 0) 0x7f53d00a06b0 con 0x7f53d009ffb0 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d6238700 1 --2- 172.21.15.89:0/2825930386 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d009e850 0x7f53d009ec20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009920 tx=0x7f53cc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8f6b195b435fcb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d5236700 1 -- 172.21.15.89:0/2825930386 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc011830 con 0x7f53d009e850 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d5236700 1 -- 172.21.15.89:0/2825930386 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc011990 con 0x7f53d009e850 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.481+0000 7f53d5236700 1 -- 172.21.15.89:0/2825930386 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc011d20 con 0x7f53d009e850 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.482+0000 7f53d849c700 1 -- 172.21.15.89:0/2825930386 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d009e850 msgr2=0x7f53d009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.482+0000 7f53d849c700 1 --2- 172.21.15.89:0/2825930386 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d009e850 0x7f53d009ec20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009920 tx=0x7f53cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.482+0000 7f53d849c700 1 -- 172.21.15.89:0/2825930386 shutdown_connections 2023-09-30T22:18:31.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.482+0000 7f53d849c700 1 --2- 172.21.15.89:0/2825930386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d0138e30 0x7f53d013b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.482+0000 7f53d849c700 1 --2- 172.21.15.89:0/2825930386 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d009e850 0x7f53d009ec20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.482+0000 7f53d849c700 1 -- 172.21.15.89:0/2825930386 >> 172.21.15.89:0/2825930386 conn(0x7f53d00a3f70 msgr2=0x7f53d00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.482+0000 7f53d849c700 1 -- 172.21.15.89:0/2825930386 shutdown_connections 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.482+0000 7f53d849c700 1 -- 172.21.15.89:0/2825930386 wait complete. 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d849c700 1 Processor -- start 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d849c700 1 -- start start 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d849c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d009e850 0x7f53d01c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d849c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d009ffb0 0x7f53d01c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d849c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d0138e30 0x7f53d0133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d849c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d01366f0 con 0x7f53d009e850 2023-09-30T22:18:31.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d849c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d0135d50 con 0x7f53d009ffb0 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d849c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53d00a3850 con 0x7f53d0138e30 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d6a39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d0138e30 0x7f53d0133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.484+0000 7f53d5a37700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d009ffb0 0x7f53d01c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6a39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d0138e30 0x7f53d0133fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52766/0 (socket says 172.21.15.89:52766) 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6a39700 1 -- 172.21.15.89:0/554504758 learned_addr learned my addr 172.21.15.89:0/554504758 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d5a37700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d009ffb0 0x7f53d01c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49066/0 (socket says 172.21.15.89:49066) 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6238700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d009e850 0x7f53d01c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6238700 1 -- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d0138e30 msgr2=0x7f53d0133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6238700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d0138e30 0x7f53d0133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6238700 1 -- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d009ffb0 msgr2=0x7f53d01c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6238700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d009ffb0 0x7f53d01c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6238700 1 -- 172.21.15.89:0/554504758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc009550 con 0x7f53d009e850 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6a39700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d0138e30 0x7f53d0133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d5a37700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d009ffb0 0x7f53d01c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d6238700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d009e850 0x7f53d01c8c50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f53cc016000 tx=0x7f53cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53c6ffd700 1 -- 172.21.15.89:0/554504758 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004450 con 0x7f53d009e850 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53c6ffd700 1 -- 172.21.15.89:0/554504758 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0045d0 con 0x7f53d009e850 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53c6ffd700 1 -- 172.21.15.89:0/554504758 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0038e0 con 0x7f53d009e850 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.485+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d006ccc0 con 0x7f53d009e850 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.486+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d0135fd0 con 0x7f53d009e850 2023-09-30T22:18:31.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.487+0000 7f53c6ffd700 1 -- 172.21.15.89:0/554504758 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53cc003a40 con 0x7f53d009e850 2023-09-30T22:18:31.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.487+0000 7f53c6ffd700 1 -- 172.21.15.89:0/554504758 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f53cc040930 con 0x7f53d009e850 2023-09-30T22:18:31.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.487+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d00747f0 con 0x7f53d009e850 2023-09-30T22:18:31.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.492+0000 7f53c6ffd700 1 -- 172.21.15.89:0/554504758 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53cc028070 con 0x7f53d009e850 2023-09-30T22:18:31.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.516+0000 7f145a7b8700 1 Processor -- start 2023-09-30T22:18:31.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.516+0000 7f145a7b8700 1 -- start start 2023-09-30T22:18:31.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.516+0000 7f145a7b8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409d6f0 0x7f145409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.516+0000 7f145a7b8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f145409e410 0x7f1454134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.516+0000 7f145a7b8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145409b120 con 0x7f145409e000 2023-09-30T22:18:31.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.516+0000 7f145a7b8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145406b460 con 0x7f145409e410 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.516+0000 7f145a7b8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145409a7e0 con 0x7f145409d6f0 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1453fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409d6f0 0x7f145409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1458d55700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f145409e410 0x7f1454134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1453fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409d6f0 0x7f145409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49072/0 (socket says 172.21.15.89:49072) 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1458d55700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f145409e410 0x7f1454134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45456/0 (socket says 172.21.15.89:45456) 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1453fff700 1 -- 172.21.15.89:0/1979466946 learned_addr learned my addr 172.21.15.89:0/1979466946 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1453fff700 1 -- 172.21.15.89:0/1979466946 >> v2:172.21.15.89:3300/0 conn(0x7f145409e410 msgr2=0x7f1454134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1453fff700 1 --2- 172.21.15.89:0/1979466946 >> v2:172.21.15.89:3300/0 conn(0x7f145409e410 0x7f1454134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1453fff700 1 -- 172.21.15.89:0/1979466946 >> v1:172.21.15.89:6789/0 conn(0x7f145409e000 legacy=0x7f14541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1453fff700 1 -- 172.21.15.89:0/1979466946 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145409a500 con 0x7f145409d6f0 2023-09-30T22:18:31.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1453fff700 1 --2- 172.21.15.89:0/1979466946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409d6f0 0x7f145409dac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1444009920 tx=0x7f1444009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d27852ce6c58c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:31.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1452ffd700 1 -- 172.21.15.89:0/1979466946 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444011830 con 0x7f145409d6f0 2023-09-30T22:18:31.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1452ffd700 1 -- 172.21.15.89:0/1979466946 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1444011990 con 0x7f145409d6f0 2023-09-30T22:18:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.517+0000 7f1452ffd700 1 -- 172.21.15.89:0/1979466946 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444011d20 con 0x7f145409d6f0 2023-09-30T22:18:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.518+0000 7f145a7b8700 1 -- 172.21.15.89:0/1979466946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409d6f0 msgr2=0x7f145409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.518+0000 7f145a7b8700 1 --2- 172.21.15.89:0/1979466946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409d6f0 0x7f145409dac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1444009920 tx=0x7f1444009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.518+0000 7f145a7b8700 1 -- 172.21.15.89:0/1979466946 shutdown_connections 2023-09-30T22:18:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.518+0000 7f145a7b8700 1 --2- 172.21.15.89:0/1979466946 >> v2:172.21.15.89:3300/0 conn(0x7f145409e410 0x7f1454134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.518+0000 7f145a7b8700 1 --2- 172.21.15.89:0/1979466946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409d6f0 0x7f145409dac0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.518+0000 7f145a7b8700 1 -- 172.21.15.89:0/1979466946 >> 172.21.15.89:0/1979466946 conn(0x7f1454066f50 msgr2=0x7f145409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.518+0000 7f145a7b8700 1 -- 172.21.15.89:0/1979466946 shutdown_connections 2023-09-30T22:18:31.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.518+0000 7f145a7b8700 1 -- 172.21.15.89:0/1979466946 wait complete. 2023-09-30T22:18:31.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7ff4feac1700 1 Processor -- start 2023-09-30T22:18:31.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7f145a7b8700 1 Processor -- start 2023-09-30T22:18:31.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7f145a7b8700 1 -- start start 2023-09-30T22:18:31.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7ff4feac1700 1 -- start start 2023-09-30T22:18:31.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7ff4feac1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7ff4feac1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e000 0x7ff4f8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7ff4feac1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f809f130 con 0x7ff4f809e410 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7ff4feac1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f809f520 con 0x7ff4f809e000 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.519+0000 7ff4feac1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f809e930 con 0x7ff4f809d6f0 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4fd05e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff4f809e410 0x7ff4f8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42316/0 (socket says 172.21.15.89:42316) 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4fd05e700 1 -- 172.21.15.89:0/2180369864 learned_addr learned my addr 172.21.15.89:0/2180369864 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4fc85d700 1 --2- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4f7fff700 1 --2- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e000 0x7ff4f8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4fc85d700 1 -- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e000 msgr2=0x7ff4f8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4fc85d700 1 --2- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e000 0x7ff4f8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4fc85d700 1 -- 172.21.15.89:0/2180369864 >> v1:172.21.15.89:6789/0 conn(0x7ff4f809e410 legacy=0x7ff4f8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4fc85d700 1 -- 172.21.15.89:0/2180369864 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f809b2d0 con 0x7ff4f809d6f0 2023-09-30T22:18:31.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f145a7b8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409d6f0 0x7f14541a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f145a7b8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409e000 0x7f14541a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f145a7b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145409e410 0x7f14541a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f145a7b8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14541ab880 con 0x7f145409e410 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f145a7b8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145409c710 con 0x7f145409e000 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f145a7b8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14541a9ba0 con 0x7f145409d6f0 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f1458d55700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145409e410 0x7f14541a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f14537fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409e000 0x7f14541a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f1458d55700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145409e410 0x7f14541a8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45460/0 (socket says 172.21.15.89:45460) 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f1458d55700 1 -- 172.21.15.89:0/2089751792 learned_addr learned my addr 172.21.15.89:0/2089751792 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f14537fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409e000 0x7f14541a85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49080/0 (socket says 172.21.15.89:49080) 2023-09-30T22:18:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f1453fff700 1 --2- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409d6f0 0x7f14541a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7ff4fc85d700 1 --2- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f809dac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0098a0 tx=0x7ff4ec009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599ab61691e78883 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7ff4f7fff700 1 --2- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e000 0x7ff4f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7ff4f77fe700 1 -- 172.21.15.89:0/2180369864 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec00f8a0 con 0x7ff4f809d6f0 2023-09-30T22:18:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f1458d55700 1 -- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409d6f0 msgr2=0x7f14541a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f1458d55700 1 --2- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409d6f0 0x7f14541a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f1458d55700 1 -- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409e000 msgr2=0x7f14541a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.520+0000 7f1458d55700 1 --2- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409e000 0x7f14541a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7f1458d55700 1 -- 172.21.15.89:0/2089751792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444009550 con 0x7f145409e410 2023-09-30T22:18:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7f1458d55700 1 --2- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145409e410 0x7f14541a8af0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f144c00efc0 tx=0x7f144c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7ff4f77fe700 1 -- 172.21.15.89:0/2180369864 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec00fa00 con 0x7ff4f809d6f0 2023-09-30T22:18:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7ff4f77fe700 1 -- 172.21.15.89:0/2180369864 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec00fd90 con 0x7ff4f809d6f0 2023-09-30T22:18:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7f1450ff9700 1 -- 172.21.15.89:0/2089751792 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c009ea0 con 0x7f145409e410 2023-09-30T22:18:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14541abae0 con 0x7f145409e410 2023-09-30T22:18:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14541aa290 con 0x7f145409e410 2023-09-30T22:18:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7f1450ff9700 1 -- 172.21.15.89:0/2089751792 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c00f040 con 0x7f145409e410 2023-09-30T22:18:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.521+0000 7f1450ff9700 1 -- 172.21.15.89:0/2089751792 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00ce90 con 0x7f145409e410 2023-09-30T22:18:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7ff4feac1700 1 -- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 msgr2=0x7ff4f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7ff4feac1700 1 --2- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f809dac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0098a0 tx=0x7ff4ec009bb0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7ff4feac1700 1 -- 172.21.15.89:0/2180369864 shutdown_connections 2023-09-30T22:18:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7ff4feac1700 1 --2- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f809dac0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7ff4feac1700 1 --2- 172.21.15.89:0/2180369864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e000 0x7ff4f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7ff4feac1700 1 -- 172.21.15.89:0/2180369864 >> 172.21.15.89:0/2180369864 conn(0x7ff4f8066f50 msgr2=0x7ff4f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7ff4feac1700 1 -- 172.21.15.89:0/2180369864 shutdown_connections 2023-09-30T22:18:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7ff4feac1700 1 -- 172.21.15.89:0/2180369864 wait complete. 2023-09-30T22:18:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14340052e0 con 0x7f145409e410 2023-09-30T22:18:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.522+0000 7f1450ff9700 1 -- 172.21.15.89:0/2089751792 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f144c014480 con 0x7f145409e410 2023-09-30T22:18:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.523+0000 7f1450ff9700 1 -- 172.21.15.89:0/2089751792 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f144c0401d0 con 0x7f145409e410 2023-09-30T22:18:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.523+0000 7ff4feac1700 1 Processor -- start 2023-09-30T22:18:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.523+0000 7ff4feac1700 1 -- start start 2023-09-30T22:18:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4feac1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f811dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4feac1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f809e000 0x7ff4f811e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4feac1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e410 0x7ff4f811ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4feac1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81212a0 con 0x7ff4f809e000 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4feac1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81214e0 con 0x7ff4f809e410 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4feac1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f811fcf0 con 0x7ff4f809d6f0 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e410 0x7ff4f811ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e410 0x7ff4f811ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49084/0 (socket says 172.21.15.89:49084) 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 -- 172.21.15.89:0/125474993 learned_addr learned my addr 172.21.15.89:0/125474993 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fc85d700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f811dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4f7fff700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f809e000 0x7ff4f811e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 -- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 msgr2=0x7ff4f811dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f811dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 -- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f809e000 msgr2=0x7ff4f811e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f809e000 0x7ff4f811e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 -- 172.21.15.89:0/125474993 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0094a0 con 0x7ff4f809e410 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4f7fff700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f809e000 0x7ff4f811e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.524+0000 7ff4fd05e700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e410 0x7ff4f811ea30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000cbc0 tx=0x7ff4f000ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.525+0000 7ff4f57fa700 1 -- 172.21.15.89:0/125474993 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000a9e0 con 0x7ff4f809e410 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.525+0000 7ff4f57fa700 1 -- 172.21.15.89:0/125474993 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f000ab40 con 0x7ff4f809e410 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.525+0000 7ff4f57fa700 1 -- 172.21.15.89:0/125474993 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0012400 con 0x7ff4f809e410 2023-09-30T22:18:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.525+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f8121a80 con 0x7ff4f809e410 2023-09-30T22:18:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.525+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f81217a0 con 0x7ff4f809e410 2023-09-30T22:18:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.526+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f8074ea0 con 0x7ff4f809e410 2023-09-30T22:18:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.527+0000 7ff4f57fa700 1 -- 172.21.15.89:0/125474993 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff4f0012560 con 0x7ff4f809e410 2023-09-30T22:18:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.527+0000 7ff4f57fa700 1 -- 172.21.15.89:0/125474993 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff4f003fb20 con 0x7ff4f809e410 2023-09-30T22:18:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.528+0000 7f1450ff9700 1 -- 172.21.15.89:0/2089751792 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f144c007920 con 0x7f145409e410 2023-09-30T22:18:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.531+0000 7ff4f57fa700 1 -- 172.21.15.89:0/125474993 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff4f001b3b0 con 0x7ff4f809e410 2023-09-30T22:18:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f9a56b700 1 Processor -- start 2023-09-30T22:18:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f9a56b700 1 -- start start 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f9a56b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f9409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f9a56b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f9a56b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9409a810 con 0x7f6f9409dc30 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f9a56b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9406ab40 con 0x7f6f9413d1f0 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f9a56b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9409ebc0 con 0x7f6f9406a770 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f98b08700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f98b08700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49086/0 (socket says 172.21.15.89:49086) 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f98b08700 1 -- 172.21.15.89:0/2447993740 learned_addr learned my addr 172.21.15.89:0/2447993740 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.591+0000 7f6f93fff700 1 --2- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f9409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f98b08700 1 -- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 msgr2=0x7f6f9409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f98b08700 1 --2- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f9409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f98b08700 1 -- 172.21.15.89:0/2447993740 >> v1:172.21.15.89:6789/0 conn(0x7f6f9409dc30 legacy=0x7f6f9409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f98b08700 1 -- 172.21.15.89:0/2447993740 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9409e920 con 0x7f6f9413d1f0 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f92ffd700 1 -- 172.21.15.89:0/2447993740 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297528898 0 0) 0x7f6f9409a810 con 0x7f6f9409dc30 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f93fff700 1 --2- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f9409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f98b08700 1 --2- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413f5d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009e80 tx=0x7f6f8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f715e0539e216238 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f92ffd700 1 -- 172.21.15.89:0/2447993740 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00e070 con 0x7f6f9413d1f0 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f92ffd700 1 -- 172.21.15.89:0/2447993740 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c00b910 con 0x7f6f9413d1f0 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.592+0000 7f6f92ffd700 1 -- 172.21.15.89:0/2447993740 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00bcc0 con 0x7f6f9413d1f0 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.593+0000 7f6f9a56b700 1 -- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 msgr2=0x7f6f9413f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.593+0000 7f6f9a56b700 1 --2- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413f5d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009e80 tx=0x7f6f8c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.593+0000 7f6f9a56b700 1 -- 172.21.15.89:0/2447993740 shutdown_connections 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.593+0000 7f6f9a56b700 1 --2- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413f5d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.593+0000 7f6f9a56b700 1 --2- 172.21.15.89:0/2447993740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f9409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.593+0000 7f6f9a56b700 1 -- 172.21.15.89:0/2447993740 >> 172.21.15.89:0/2447993740 conn(0x7f6f94066f50 msgr2=0x7f6f9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.593+0000 7f6f9a56b700 1 -- 172.21.15.89:0/2447993740 shutdown_connections 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.593+0000 7f6f9a56b700 1 -- 172.21.15.89:0/2447993740 wait complete. 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.594+0000 7f6f9a56b700 1 Processor -- start 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.594+0000 7f6f9a56b700 1 -- start start 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f9a56b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f94071490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f9a56b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9409dc30 0x7f6f9413b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f9a56b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f9a56b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94072280 con 0x7f6f9409dc30 2023-09-30T22:18:31.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f9a56b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94072090 con 0x7f6f9413d1f0 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f9a56b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94135cd0 con 0x7f6f9406a770 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f94071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f94071490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52816/0 (socket says 172.21.15.89:52816) 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 -- 172.21.15.89:0/3434431563 learned_addr learned my addr 172.21.15.89:0/3434431563 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f98b08700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f937fe700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9409dc30 0x7f6f9413b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 -- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 msgr2=0x7f6f9413bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 -- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9409dc30 msgr2=0x7f6f9413b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9409dc30 0x7f6f9413b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 -- 172.21.15.89:0/3434431563 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840094a0 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f98b08700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.595+0000 7f6f93fff700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f94071490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400ec10 tx=0x7f6f8400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.596+0000 7f6f90ff9700 1 -- 172.21.15.89:0/3434431563 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400c910 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.596+0000 7f6f90ff9700 1 -- 172.21.15.89:0/3434431563 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8400ca70 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.596+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c009b30 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.596+0000 7f6f90ff9700 1 -- 172.21.15.89:0/3434431563 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400cde0 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.596+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9409f1a0 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.597+0000 7f6f90ff9700 1 -- 172.21.15.89:0/3434431563 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f84013470 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.597+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f94074ea0 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.597+0000 7f6f90ff9700 1 -- 172.21.15.89:0/3434431563 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6f84013bf0 con 0x7f6f9406a770 2023-09-30T22:18:31.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.606+0000 7f6f90ff9700 1 -- 172.21.15.89:0/3434431563 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f84044060 con 0x7f6f9406a770 2023-09-30T22:18:31.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.723+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1434005590 con 0x7f145409e410 2023-09-30T22:18:31.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.723+0000 7f1450ff9700 1 -- 172.21.15.89:0/2089751792 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f144c018020 con 0x7f145409e410 2023-09-30T22:18:31.205 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:31.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145409e410 msgr2=0x7f14541a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 --2- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145409e410 0x7f14541a8af0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f144c00efc0 tx=0x7f144c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 shutdown_connections 2023-09-30T22:18:31.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 --2- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145409e410 0x7f14541a8af0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 --2- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409d6f0 0x7f14541a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 --2- 172.21.15.89:0/2089751792 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145409e000 0x7f14541a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 >> 172.21.15.89:0/2089751792 conn(0x7f1454066f50 msgr2=0x7f145406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:31.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 shutdown_connections 2023-09-30T22:18:31.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.725+0000 7f145a7b8700 1 -- 172.21.15.89:0/2089751792 wait complete. 2023-09-30T22:18:31.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.732+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff4f806c360 con 0x7ff4f809e410 2023-09-30T22:18:31.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.732+0000 7ff4f57fa700 1 -- 172.21.15.89:0/125474993 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff4f0007070 con 0x7ff4f809e410 2023-09-30T22:18:31.208 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:31.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.734+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f53d0072820 con 0x7f53d009e850 2023-09-30T22:18:31.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.734+0000 7f53c6ffd700 1 -- 172.21.15.89:0/554504758 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53cc015020 con 0x7f53d009e850 2023-09-30T22:18:31.208 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:31.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.735+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e410 msgr2=0x7ff4f811ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.735+0000 7ff4feac1700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e410 0x7ff4f811ea30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000cbc0 tx=0x7ff4f000ced0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 shutdown_connections 2023-09-30T22:18:31.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7ff4feac1700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4f809e410 0x7ff4f811ea30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7ff4feac1700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4f809d6f0 0x7ff4f811dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7ff4feac1700 1 --2- 172.21.15.89:0/125474993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f809e000 0x7ff4f811e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 >> 172.21.15.89:0/125474993 conn(0x7ff4f8066f50 msgr2=0x7ff4f8071150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:31.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 shutdown_connections 2023-09-30T22:18:31.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d009e850 msgr2=0x7f53d01c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7f53d849c700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d009e850 0x7f53d01c8c50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f53cc016000 tx=0x7f53cc004750 comp rx=0 tx=0).stop 2023-09-30T22:18:31.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7ff4feac1700 1 -- 172.21.15.89:0/125474993 wait complete. 2023-09-30T22:18:31.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 shutdown_connections 2023-09-30T22:18:31.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7f53d849c700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d0138e30 0x7f53d0133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7f53d849c700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d009e850 0x7f53d01c8c50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7f53d849c700 1 --2- 172.21.15.89:0/554504758 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d009ffb0 0x7f53d01c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.736+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 >> 172.21.15.89:0/554504758 conn(0x7f53d00a3f70 msgr2=0x7f53d0071800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:31.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.737+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 shutdown_connections 2023-09-30T22:18:31.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.737+0000 7f53d849c700 1 -- 172.21.15.89:0/554504758 wait complete. 2023-09-30T22:18:31.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:31.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:31.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:31.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.801+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f9411e240 con 0x7f6f9406a770 2023-09-30T22:18:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.801+0000 7f6f90ff9700 1 -- 172.21.15.89:0/3434431563 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f84017020 con 0x7f6f9406a770 2023-09-30T22:18:31.230 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.803+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 msgr2=0x7f6f94071490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.803+0000 7f6f9a56b700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f94071490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400ec10 tx=0x7f6f8400ef20 comp rx=0 tx=0).stop 2023-09-30T22:18:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.803+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 shutdown_connections 2023-09-30T22:18:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.803+0000 7f6f9a56b700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9413d1f0 0x7f6f9413bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.803+0000 7f6f9a56b700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9406a770 0x7f6f94071490 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.803+0000 7f6f9a56b700 1 --2- 172.21.15.89:0/3434431563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9409dc30 0x7f6f9413b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.803+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 >> 172.21.15.89:0/3434431563 conn(0x7f6f94066f50 msgr2=0x7f6f940a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.804+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 shutdown_connections 2023-09-30T22:18:31.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:29.804+0000 7f6f9a56b700 1 -- 172.21.15.89:0/3434431563 wait complete. 2023-09-30T22:18:31.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:32.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:32.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:32.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:32.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.358+0000 7f69f3016700 1 Processor -- start 2023-09-30T22:18:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.358+0000 7f69f3016700 1 -- start start 2023-09-30T22:18:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69f3016700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f69ec12cf60 0x7f69ec06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69f3016700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec12f8b0 0x7f69ec12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7fc06dfa8700 1 Processor -- start 2023-09-30T22:18:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7fc06dfa8700 1 -- start start 2023-09-30T22:18:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69f3016700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec130260 0x7f69ec136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69f3016700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec12d330 con 0x7f69ec12cf60 2023-09-30T22:18:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69f3016700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec12cd90 con 0x7f69ec130260 2023-09-30T22:18:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69f3016700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec1306d0 con 0x7f69ec12f8b0 2023-09-30T22:18:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69ebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec12f8b0 0x7f69ec12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69ebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec12f8b0 0x7f69ec12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51350/0 (socket says 172.21.15.89:51350) 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69ebfff700 1 -- 172.21.15.89:0/1242071049 learned_addr learned my addr 172.21.15.89:0/1242071049 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69f0db2700 1 --2- 172.21.15.89:0/1242071049 >> v2:172.21.15.89:3300/0 conn(0x7f69ec12cf60 0x7f69ec06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69ebfff700 1 -- 172.21.15.89:0/1242071049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec130260 msgr2=0x7f69ec136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69ebfff700 1 --2- 172.21.15.89:0/1242071049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec130260 0x7f69ec136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69ebfff700 1 -- 172.21.15.89:0/1242071049 >> v2:172.21.15.89:3300/0 conn(0x7f69ec12cf60 msgr2=0x7f69ec06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69ebfff700 1 --2- 172.21.15.89:0/1242071049 >> v2:172.21.15.89:3300/0 conn(0x7f69ec12cf60 0x7f69ec06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.359+0000 7f69ebfff700 1 -- 172.21.15.89:0/1242071049 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec071780 con 0x7f69ec12f8b0 2023-09-30T22:18:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7f69ebfff700 1 --2- 172.21.15.89:0/1242071049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec12f8b0 0x7f69ec12fd20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009920 tx=0x7f69d8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc9e77f6dd38990 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc06dfa8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc06806e490 0x7fc06806e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc06dfa8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc06806eda0 0x7fc068134610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc06dfa8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc068134b50 0x7fc068134f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc06dfa8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc068070ee0 con 0x7fc06806e490 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc06dfa8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681354d0 con 0x7fc06806eda0 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc06dfa8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc06806f8c0 con 0x7fc068134b50 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc068134b50 0x7fc068134f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc068134b50 0x7fc068134f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51352/0 (socket says 172.21.15.89:51352) 2023-09-30T22:18:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 -- 172.21.15.89:0/3064586375 learned_addr learned my addr 172.21.15.89:0/3064586375 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7f69eb7fe700 1 -- 172.21.15.89:0/1242071049 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800b810 con 0x7f69ec12f8b0 2023-09-30T22:18:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7f69eb7fe700 1 -- 172.21.15.89:0/1242071049 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d800b970 con 0x7f69ec12f8b0 2023-09-30T22:18:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7f69eb7fe700 1 -- 172.21.15.89:0/1242071049 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800bd00 con 0x7f69ec12f8b0 2023-09-30T22:18:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 -- 172.21.15.89:0/3064586375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc06806eda0 msgr2=0x7fc068134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 --2- 172.21.15.89:0/3064586375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc06806eda0 0x7fc068134610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 -- 172.21.15.89:0/3064586375 >> v2:172.21.15.89:3300/0 conn(0x7fc06806e490 msgr2=0x7fc06806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 --2- 172.21.15.89:0/3064586375 >> v2:172.21.15.89:3300/0 conn(0x7fc06806e490 0x7fc06806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 -- 172.21.15.89:0/3064586375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068071980 con 0x7fc068134b50 2023-09-30T22:18:32.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc067fff700 1 --2- 172.21.15.89:0/3064586375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc068134b50 0x7fc068134f90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc060009e80 tx=0x7fc06000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1aa46f1fd8a21fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc0667fc700 1 -- 172.21.15.89:0/3064586375 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000e070 con 0x7fc068134b50 2023-09-30T22:18:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc0667fc700 1 -- 172.21.15.89:0/3064586375 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06000b980 con 0x7fc068134b50 2023-09-30T22:18:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.360+0000 7fc0667fc700 1 -- 172.21.15.89:0/3064586375 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000bd30 con 0x7fc068134b50 2023-09-30T22:18:32.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 -- 172.21.15.89:0/1242071049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec12f8b0 msgr2=0x7f69ec12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 --2- 172.21.15.89:0/1242071049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec12f8b0 0x7f69ec12fd20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009920 tx=0x7f69d8009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:32.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 -- 172.21.15.89:0/3064586375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc068134b50 msgr2=0x7fc068134f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 --2- 172.21.15.89:0/3064586375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc068134b50 0x7fc068134f90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc060009e80 tx=0x7fc06000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:32.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 -- 172.21.15.89:0/1242071049 shutdown_connections 2023-09-30T22:18:32.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 --2- 172.21.15.89:0/1242071049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec130260 0x7f69ec136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 --2- 172.21.15.89:0/1242071049 >> v2:172.21.15.89:3300/0 conn(0x7f69ec12cf60 0x7f69ec06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 --2- 172.21.15.89:0/1242071049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec12f8b0 0x7f69ec12fd20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 -- 172.21.15.89:0/1242071049 >> 172.21.15.89:0/1242071049 conn(0x7f69ec0a2aa0 msgr2=0x7f69ec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:32.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 -- 172.21.15.89:0/1242071049 shutdown_connections 2023-09-30T22:18:32.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7f69f3016700 1 -- 172.21.15.89:0/1242071049 wait complete. 2023-09-30T22:18:32.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 -- 172.21.15.89:0/3064586375 shutdown_connections 2023-09-30T22:18:32.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 --2- 172.21.15.89:0/3064586375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc068134b50 0x7fc068134f90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 --2- 172.21.15.89:0/3064586375 >> v2:172.21.15.89:3300/0 conn(0x7fc06806e490 0x7fc06806e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 --2- 172.21.15.89:0/3064586375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc06806eda0 0x7fc068134610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 -- 172.21.15.89:0/3064586375 >> 172.21.15.89:0/3064586375 conn(0x7fc0680a2aa0 msgr2=0x7fc0680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:32.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 -- 172.21.15.89:0/3064586375 shutdown_connections 2023-09-30T22:18:32.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.361+0000 7fc06dfa8700 1 -- 172.21.15.89:0/3064586375 wait complete. 2023-09-30T22:18:32.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.362+0000 7fc06dfa8700 1 Processor -- start 2023-09-30T22:18:32.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.362+0000 7f69f3016700 1 Processor -- start 2023-09-30T22:18:32.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f3016700 1 -- start start 2023-09-30T22:18:32.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc06dfa8700 1 -- start start 2023-09-30T22:18:32.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc06dfa8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc06806e490 0x7fc0681232c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc06dfa8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc06806eda0 0x7fc068123800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc06dfa8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc068134b50 0x7fc068123d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc06dfa8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068126d00 con 0x7fc06806e490 2023-09-30T22:18:32.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc06dfa8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc068124df0 con 0x7fc068134b50 2023-09-30T22:18:32.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc06dfa8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681346c0 con 0x7fc06806eda0 2023-09-30T22:18:32.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc066ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc06806eda0 0x7fc068123800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc066ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc06806eda0 0x7fc068123800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51362/0 (socket says 172.21.15.89:51362) 2023-09-30T22:18:32.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7fc066ffd700 1 -- 172.21.15.89:0/1595047757 learned_addr learned my addr 172.21.15.89:0/1595047757 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:32.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc067fff700 1 --2- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc068134b50 0x7fc068123d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f3016700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec12cf60 0x7f69ec1c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f3016700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ec12f8b0 0x7f69ec1c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f3016700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec130260 0x7f69ec1c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f3016700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec09e710 con 0x7f69ec12f8b0 2023-09-30T22:18:32.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f3016700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec09d980 con 0x7f69ec12cf60 2023-09-30T22:18:32.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f3016700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec09ca00 con 0x7f69ec130260 2023-09-30T22:18:32.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f0db2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec12cf60 0x7f69ec1c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f0db2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec12cf60 0x7f69ec1c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35570/0 (socket says 172.21.15.89:35570) 2023-09-30T22:18:32.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.363+0000 7f69f0db2700 1 -- 172.21.15.89:0/1267645417 learned_addr learned my addr 172.21.15.89:0/1267645417 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:32.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69f15b3700 1 --2- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec130260 0x7f69ec1c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc066ffd700 1 -- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc068134b50 msgr2=0x7fc068123d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc066ffd700 1 --2- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc068134b50 0x7fc068123d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc066ffd700 1 -- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc06806e490 msgr2=0x7fc0681232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc066ffd700 1 --2- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc06806e490 0x7fc0681232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc066ffd700 1 -- 172.21.15.89:0/1595047757 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009b30 con 0x7fc06806eda0 2023-09-30T22:18:32.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69f0db2700 1 -- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec130260 msgr2=0x7f69ec1c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69f0db2700 1 --2- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec130260 0x7f69ec1c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69f0db2700 1 -- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ec12f8b0 msgr2=0x7f69ec1c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69f0db2700 1 --2- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ec12f8b0 0x7f69ec1c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69f0db2700 1 -- 172.21.15.89:0/1267645417 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d8009550 con 0x7f69ec12cf60 2023-09-30T22:18:32.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc066ffd700 1 --2- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc06806eda0 0x7fc068123800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b530 tx=0x7fc05c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:32.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f4b1818b700 1 Processor -- start 2023-09-30T22:18:32.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.365+0000 7f4b1818b700 1 -- start start 2023-09-30T22:18:32.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.365+0000 7f4b1818b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b1006e570 0x7f4b1006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.365+0000 7f4b1818b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006ee80 0x7f4b10134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.365+0000 7f4b1818b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b10070ee0 con 0x7f4b1009eca0 2023-09-30T22:18:32.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.365+0000 7f4b1818b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b10071480 con 0x7f4b1006e570 2023-09-30T22:18:32.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.365+0000 7f4b1818b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b10071b90 con 0x7f4b1006ee80 2023-09-30T22:18:32.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc04ffff700 1 -- 172.21.15.89:0/1595047757 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c011830 con 0x7fc06806eda0 2023-09-30T22:18:32.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc068070710 con 0x7fc06806eda0 2023-09-30T22:18:32.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc04ffff700 1 -- 172.21.15.89:0/1595047757 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c011990 con 0x7fc06806eda0 2023-09-30T22:18:32.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc04ffff700 1 -- 172.21.15.89:0/1595047757 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c011d40 con 0x7fc06806eda0 2023-09-30T22:18:32.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc068124690 con 0x7fc06806eda0 2023-09-30T22:18:32.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69f0db2700 1 --2- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec12cf60 0x7f69ec1c8380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69e000dd10 tx=0x7f69e0009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:32.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69e97fa700 1 -- 172.21.15.89:0/1267645417 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000b1c0 con 0x7f69ec12cf60 2023-09-30T22:18:32.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec129fa0 con 0x7f69ec12cf60 2023-09-30T22:18:32.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69e97fa700 1 -- 172.21.15.89:0/1267645417 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0009a10 con 0x7f69ec12cf60 2023-09-30T22:18:32.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.364+0000 7f69e97fa700 1 -- 172.21.15.89:0/1267645417 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0009dc0 con 0x7f69ec12cf60 2023-09-30T22:18:32.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.365+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec11d3e0 con 0x7f69ec12cf60 2023-09-30T22:18:32.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec074ea0 con 0x7f69ec12cf60 2023-09-30T22:18:32.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc068074ea0 con 0x7fc06806eda0 2023-09-30T22:18:32.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b15726700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4b1009eca0 0x7f4b10134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52322/0 (socket says 172.21.15.89:52322) 2023-09-30T22:18:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b15726700 1 -- 172.21.15.89:0/1851961060 learned_addr learned my addr 172.21.15.89:0/1851961060 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b15f27700 1 --2- 172.21.15.89:0/1851961060 >> v2:172.21.15.89:3300/0 conn(0x7f4b1006e570 0x7f4b1006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b16728700 1 --2- 172.21.15.89:0/1851961060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006ee80 0x7f4b10134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b15f27700 1 -- 172.21.15.89:0/1851961060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006ee80 msgr2=0x7f4b10134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b15f27700 1 --2- 172.21.15.89:0/1851961060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006ee80 0x7f4b10134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b15f27700 1 -- 172.21.15.89:0/1851961060 >> v1:172.21.15.89:6789/0 conn(0x7f4b1009eca0 legacy=0x7f4b10134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b14f25700 1 -- 172.21.15.89:0/1851961060 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565834703 0 0) 0x7f4b10070ee0 con 0x7f4b1009eca0 2023-09-30T22:18:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f69e97fa700 1 -- 172.21.15.89:0/1267645417 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f69e0020400 con 0x7f69ec12cf60 2023-09-30T22:18:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7f4b15f27700 1 -- 172.21.15.89:0/1851961060 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b100711a0 con 0x7f4b1006e570 2023-09-30T22:18:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7fc04ffff700 1 -- 172.21.15.89:0/1595047757 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc05c01a400 con 0x7fc06806eda0 2023-09-30T22:18:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.366+0000 7fc04ffff700 1 -- 172.21.15.89:0/1595047757 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc05c03f7a0 con 0x7fc06806eda0 2023-09-30T22:18:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.367+0000 7f69e97fa700 1 -- 172.21.15.89:0/1267645417 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f69e0040560 con 0x7f69ec12cf60 2023-09-30T22:18:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.367+0000 7f4b15f27700 1 --2- 172.21.15.89:0/1851961060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1006e570 0x7f4b1006e940 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009840 tx=0x7f4b0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bbae65b0820a4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.367+0000 7f4b14f25700 1 -- 172.21.15.89:0/1851961060 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011830 con 0x7f4b1006e570 2023-09-30T22:18:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.367+0000 7f4b14f25700 1 -- 172.21.15.89:0/1851961060 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c011990 con 0x7f4b1006e570 2023-09-30T22:18:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.367+0000 7f4b14f25700 1 -- 172.21.15.89:0/1851961060 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011d20 con 0x7f4b1006e570 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.368+0000 7f4b1818b700 1 -- 172.21.15.89:0/1851961060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1006e570 msgr2=0x7f4b1006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.368+0000 7f4b1818b700 1 --2- 172.21.15.89:0/1851961060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1006e570 0x7f4b1006e940 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009840 tx=0x7f4b0c009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.368+0000 7f4b1818b700 1 -- 172.21.15.89:0/1851961060 shutdown_connections 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.368+0000 7f4b1818b700 1 --2- 172.21.15.89:0/1851961060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006ee80 0x7f4b10134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.368+0000 7f4b1818b700 1 --2- 172.21.15.89:0/1851961060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1006e570 0x7f4b1006e940 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.368+0000 7f4b1818b700 1 -- 172.21.15.89:0/1851961060 >> 172.21.15.89:0/1851961060 conn(0x7f4b100a2aa0 msgr2=0x7f4b100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.368+0000 7f4b1818b700 1 -- 172.21.15.89:0/1851961060 shutdown_connections 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.368+0000 7f4b1818b700 1 -- 172.21.15.89:0/1851961060 wait complete. 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.369+0000 7f4b1818b700 1 Processor -- start 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.369+0000 7f4b1818b700 1 -- start start 2023-09-30T22:18:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.369+0000 7f4b1818b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006e570 0x7f4b1013ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.369+0000 7f4b1818b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b1006ee80 0x7f4b10124e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b1818b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1013db10 0x7f4b10125340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b1818b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b101356c0 con 0x7f4b1013db10 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b1818b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1013d780 con 0x7f4b1006e570 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b1818b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1013d330 con 0x7f4b1006ee80 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15f27700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006e570 0x7f4b1013ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15f27700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006e570 0x7f4b1013ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35602/0 (socket says 172.21.15.89:35602) 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15f27700 1 -- 172.21.15.89:0/2410113863 learned_addr learned my addr 172.21.15.89:0/2410113863 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15726700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b1006ee80 0x7f4b10124e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b16728700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1013db10 0x7f4b10125340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15726700 1 -- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006e570 msgr2=0x7f4b1013ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15726700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006e570 0x7f4b1013ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15726700 1 -- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1013db10 msgr2=0x7f4b10125340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15726700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1013db10 0x7f4b10125340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15726700 1 -- 172.21.15.89:0/2410113863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0094a0 con 0x7f4b1006ee80 2023-09-30T22:18:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15f27700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006e570 0x7f4b1013ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.370+0000 7f4b15726700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b1006ee80 0x7f4b10124e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b100a35d0 tx=0x7f4b0800cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.371+0000 7f69e97fa700 1 -- 172.21.15.89:0/1267645417 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f69e00206b0 con 0x7f69ec12cf60 2023-09-30T22:18:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.372+0000 7f4b067fc700 1 -- 172.21.15.89:0/2410113863 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800aac0 con 0x7f4b1006ee80 2023-09-30T22:18:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.372+0000 7f4b067fc700 1 -- 172.21.15.89:0/2410113863 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0800ac40 con 0x7f4b1006ee80 2023-09-30T22:18:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.372+0000 7f4b067fc700 1 -- 172.21.15.89:0/2410113863 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b080070d0 con 0x7f4b1006ee80 2023-09-30T22:18:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.372+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b101266d0 con 0x7f4b1006ee80 2023-09-30T22:18:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.372+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1013d070 con 0x7f4b1006ee80 2023-09-30T22:18:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.373+0000 7fc04ffff700 1 -- 172.21.15.89:0/1595047757 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc05c00f8d0 con 0x7fc06806eda0 2023-09-30T22:18:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.373+0000 7f4b067fc700 1 -- 172.21.15.89:0/2410113863 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b08007250 con 0x7f4b1006ee80 2023-09-30T22:18:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.373+0000 7f4b067fc700 1 -- 172.21.15.89:0/2410113863 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4b0803f1e0 con 0x7f4b1006ee80 2023-09-30T22:18:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.373+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b10074ea0 con 0x7f4b1006ee80 2023-09-30T22:18:32.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.378+0000 7f4b067fc700 1 -- 172.21.15.89:0/2410113863 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b08018080 con 0x7f4b1006ee80 2023-09-30T22:18:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.405+0000 7fd27189d700 1 Processor -- start 2023-09-30T22:18:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.405+0000 7fd27189d700 1 -- start start 2023-09-30T22:18:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd27189d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd27189d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd26c09f240 0x7fd26c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd27189d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c070940 con 0x7fd26c09f650 2023-09-30T22:18:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd27189d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26c06fd60 con 0x7fd26c09f240 2023-09-30T22:18:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd27189d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c06fa80 con 0x7fd26c09e930 2023-09-30T22:18:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd26c09f240 0x7fd26c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd26c09f240 0x7fd26c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48814/0 (socket says 172.21.15.89:48814) 2023-09-30T22:18:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51396/0 (socket says 172.21.15.89:51396) 2023-09-30T22:18:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26a7fc700 1 -- 172.21.15.89:0/781548155 learned_addr learned my addr 172.21.15.89:0/781548155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26a7fc700 1 -- 172.21.15.89:0/781548155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 msgr2=0x7fd26c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26a7fc700 1 --2- 172.21.15.89:0/781548155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26a7fc700 1 -- 172.21.15.89:0/781548155 >> v1:172.21.15.89:6789/0 conn(0x7fd26c09f650 legacy=0x7fd26c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.406+0000 7fd26a7fc700 1 -- 172.21.15.89:0/781548155 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c09fb60 con 0x7fd26c09f240 2023-09-30T22:18:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.407+0000 7fd26a7fc700 1 --2- 172.21.15.89:0/781548155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f240 0x7fd26c1317f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd2600098f0 tx=0x7fd260009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f649576ababc10 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.407+0000 7fd269ffb700 1 -- 172.21.15.89:0/781548155 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260011830 con 0x7fd26c09f240 2023-09-30T22:18:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.407+0000 7fd269ffb700 1 -- 172.21.15.89:0/781548155 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd260011990 con 0x7fd26c09f240 2023-09-30T22:18:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.407+0000 7fd269ffb700 1 -- 172.21.15.89:0/781548155 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260011d20 con 0x7fd26c09f240 2023-09-30T22:18:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.407+0000 7fd27189d700 1 -- 172.21.15.89:0/781548155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f240 msgr2=0x7fd26c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.407+0000 7fd27189d700 1 --2- 172.21.15.89:0/781548155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f240 0x7fd26c1317f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd2600098f0 tx=0x7fd260009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.408+0000 7fd27189d700 1 -- 172.21.15.89:0/781548155 shutdown_connections 2023-09-30T22:18:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.408+0000 7fd27189d700 1 --2- 172.21.15.89:0/781548155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.408+0000 7fd27189d700 1 --2- 172.21.15.89:0/781548155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f240 0x7fd26c1317f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.408+0000 7fd27189d700 1 -- 172.21.15.89:0/781548155 >> 172.21.15.89:0/781548155 conn(0x7fd26c0a4210 msgr2=0x7fd26c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.408+0000 7fd27189d700 1 -- 172.21.15.89:0/781548155 shutdown_connections 2023-09-30T22:18:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.408+0000 7fd27189d700 1 -- 172.21.15.89:0/781548155 wait complete. 2023-09-30T22:18:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.408+0000 7fd27189d700 1 Processor -- start 2023-09-30T22:18:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd27189d700 1 -- start start 2023-09-30T22:18:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd27189d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c1a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd27189d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26c09f240 0x7fd26c1a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd27189d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f650 0x7fd26c1a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd27189d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c1a7170 con 0x7fd26c09f650 2023-09-30T22:18:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd27189d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c1a78d0 con 0x7fd26c09f240 2023-09-30T22:18:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd27189d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c1a8b20 con 0x7fd26c09e930 2023-09-30T22:18:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd26affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c1a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd26affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c1a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51410/0 (socket says 172.21.15.89:51410) 2023-09-30T22:18:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd26affd700 1 -- 172.21.15.89:0/1962060349 learned_addr learned my addr 172.21.15.89:0/1962060349 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd26b7fe700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f650 0x7fd26c1a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.409+0000 7fd26a7fc700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26c09f240 0x7fd26c1a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd26affd700 1 -- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26c09f240 msgr2=0x7fd26c1a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd26affd700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26c09f240 0x7fd26c1a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd26affd700 1 -- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f650 msgr2=0x7fd26c1a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd26affd700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f650 0x7fd26c1a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd26affd700 1 -- 172.21.15.89:0/1962060349 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd260009550 con 0x7fd26c09e930 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd26a7fc700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26c09f240 0x7fd26c1a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd26affd700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c1a5cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00e9e0 tx=0x7fd25c00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd253fff700 1 -- 172.21.15.89:0/1962060349 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c009710 con 0x7fd26c09e930 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd253fff700 1 -- 172.21.15.89:0/1962060349 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c00c850 con 0x7fd26c09e930 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c0a0190 con 0x7fd26c09e930 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd253fff700 1 -- 172.21.15.89:0/1962060349 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00cbc0 con 0x7fd26c09e930 2023-09-30T22:18:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.410+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c0a3530 con 0x7fd26c09e930 2023-09-30T22:18:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.411+0000 7fd253fff700 1 -- 172.21.15.89:0/1962060349 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd25c00cd20 con 0x7fd26c09e930 2023-09-30T22:18:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.411+0000 7fd253fff700 1 -- 172.21.15.89:0/1962060349 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd25c03f7a0 con 0x7fd26c09e930 2023-09-30T22:18:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.411+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c1a2b40 con 0x7fd26c09e930 2023-09-30T22:18:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.416+0000 7fd253fff700 1 -- 172.21.15.89:0/1962060349 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd25c0232f0 con 0x7fd26c09e930 2023-09-30T22:18:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.574+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4b1013d550 con 0x7f4b1006ee80 2023-09-30T22:18:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.575+0000 7f4b067fc700 1 -- 172.21.15.89:0/2410113863 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b08007500 con 0x7f4b1006ee80 2023-09-30T22:18:32.575 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.577+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b1006ee80 msgr2=0x7f4b10124e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.577+0000 7f4b1818b700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b1006ee80 0x7f4b10124e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b100a35d0 tx=0x7f4b0800cee0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.577+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 shutdown_connections 2023-09-30T22:18:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.577+0000 7f4b1818b700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1013db10 0x7f4b10125340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.577+0000 7f4b1818b700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b1006e570 0x7f4b1013ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.577+0000 7f4b1818b700 1 --2- 172.21.15.89:0/2410113863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b1006ee80 0x7f4b10124e00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.577+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 >> 172.21.15.89:0/2410113863 conn(0x7f4b100a2aa0 msgr2=0x7f4b100a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.578+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 shutdown_connections 2023-09-30T22:18:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.578+0000 7f4b1818b700 1 -- 172.21.15.89:0/2410113863 wait complete. 2023-09-30T22:18:32.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.608+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f69ec0713a0 con 0x7f69ec12cf60 2023-09-30T22:18:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.609+0000 7f69e97fa700 1 -- 172.21.15.89:0/1267645417 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f69e001f360 con 0x7f69ec12cf60 2023-09-30T22:18:32.610 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.610+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc06806b9e0 con 0x7fc06806eda0 2023-09-30T22:18:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.610+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd26c071d20 con 0x7fd26c09e930 2023-09-30T22:18:32.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.610+0000 7fc04ffff700 1 -- 172.21.15.89:0/1595047757 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc05c015030 con 0x7fc06806eda0 2023-09-30T22:18:32.611 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:32.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.610+0000 7fd253fff700 1 -- 172.21.15.89:0/1962060349 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd25c021020 con 0x7fd26c09e930 2023-09-30T22:18:32.612 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:32.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.611+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec12cf60 msgr2=0x7f69ec1c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.611+0000 7f69f3016700 1 --2- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec12cf60 0x7f69ec1c8380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69e000dd10 tx=0x7f69e0009400 comp rx=0 tx=0).stop 2023-09-30T22:18:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 shutdown_connections 2023-09-30T22:18:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7f69f3016700 1 --2- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69ec130260 0x7f69ec1c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7f69f3016700 1 --2- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69ec12cf60 0x7f69ec1c8380 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7f69f3016700 1 --2- 172.21.15.89:0/1267645417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69ec12f8b0 0x7f69ec1c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 >> 172.21.15.89:0/1267645417 conn(0x7f69ec0a2aa0 msgr2=0x7f69ec06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 shutdown_connections 2023-09-30T22:18:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7f69f3016700 1 -- 172.21.15.89:0/1267645417 wait complete. 2023-09-30T22:18:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc06806eda0 msgr2=0x7fc068123800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7fc06dfa8700 1 --2- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc06806eda0 0x7fc068123800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b530 tx=0x7fc05c00b840 comp rx=0 tx=0).stop 2023-09-30T22:18:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 msgr2=0x7fd26c1a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7fd27189d700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c1a5cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00e9e0 tx=0x7fd25c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 shutdown_connections 2023-09-30T22:18:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7fc06dfa8700 1 --2- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc068134b50 0x7fc068123d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7fc06dfa8700 1 --2- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc06806e490 0x7fc0681232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.612+0000 7fc06dfa8700 1 --2- 172.21.15.89:0/1595047757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc06806eda0 0x7fc068123800 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 >> 172.21.15.89:0/1595047757 conn(0x7fc0680a2aa0 msgr2=0x7fc068136d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:32.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 shutdown_connections 2023-09-30T22:18:32.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fd27189d700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26c09f650 0x7fd26c1a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fd27189d700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26c09e930 0x7fd26c1a5cc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fd27189d700 1 --2- 172.21.15.89:0/1962060349 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26c09f240 0x7fd26c1a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:32.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 >> 172.21.15.89:0/1962060349 conn(0x7fd26c0a4210 msgr2=0x7fd26c09dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:32.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 shutdown_connections 2023-09-30T22:18:32.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fc06dfa8700 1 -- 172.21.15.89:0/1595047757 wait complete. 2023-09-30T22:18:32.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 shutdown_connections 2023-09-30T22:18:32.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:32.613+0000 7fd27189d700 1 -- 172.21.15.89:0/1962060349 wait complete. 2023-09-30T22:18:32.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:32.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:32.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:33.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:33.630 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:33.630 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:33.631 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:33.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.737+0000 7f951bf46700 1 Processor -- start 2023-09-30T22:18:33.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.737+0000 7f951bf46700 1 -- start start 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f951bf46700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951409d9f0 0x7f9514136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f951bf46700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 0x7f9514138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f951bf46700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951409e130 con 0x7f95141316c0 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f951bf46700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95141321a0 con 0x7f9514136a70 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f951bf46700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95141312b0 con 0x7f951409d9f0 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f95194e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951409d9f0 0x7f9514136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f95194e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951409d9f0 0x7f9514136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51420/0 (socket says 172.21.15.89:51420) 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f95194e1700 1 -- 172.21.15.89:0/3582085902 learned_addr learned my addr 172.21.15.89:0/3582085902 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f951a4e3700 1 --2- 172.21.15.89:0/3582085902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 0x7f9514138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f95194e1700 1 -- 172.21.15.89:0/3582085902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 msgr2=0x7f9514138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f95194e1700 1 --2- 172.21.15.89:0/3582085902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 0x7f9514138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f95194e1700 1 -- 172.21.15.89:0/3582085902 >> v1:172.21.15.89:6789/0 conn(0x7f95141316c0 legacy=0x7f9514131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.738+0000 7f95194e1700 1 -- 172.21.15.89:0/3582085902 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951409f6e0 con 0x7f951409d9f0 2023-09-30T22:18:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.739+0000 7f95194e1700 1 --2- 172.21.15.89:0/3582085902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951409d9f0 0x7f9514136530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9504009950 tx=0x7f9504009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd5a782007db59f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.739+0000 7f9518ce0700 1 -- 172.21.15.89:0/3582085902 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504009ea0 con 0x7f951409d9f0 2023-09-30T22:18:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.739+0000 7f9518ce0700 1 -- 172.21.15.89:0/3582085902 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950400ba20 con 0x7f951409d9f0 2023-09-30T22:18:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.739+0000 7f9518ce0700 1 -- 172.21.15.89:0/3582085902 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950400bdb0 con 0x7f951409d9f0 2023-09-30T22:18:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.740+0000 7f951bf46700 1 -- 172.21.15.89:0/3582085902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951409d9f0 msgr2=0x7f9514136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.740+0000 7f951bf46700 1 --2- 172.21.15.89:0/3582085902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951409d9f0 0x7f9514136530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9504009950 tx=0x7f9504009c60 comp rx=0 tx=0).stop 2023-09-30T22:18:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.740+0000 7f951bf46700 1 -- 172.21.15.89:0/3582085902 shutdown_connections 2023-09-30T22:18:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.740+0000 7f951bf46700 1 --2- 172.21.15.89:0/3582085902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 0x7f9514138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.740+0000 7f951bf46700 1 --2- 172.21.15.89:0/3582085902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951409d9f0 0x7f9514136530 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.740+0000 7f951bf46700 1 -- 172.21.15.89:0/3582085902 >> 172.21.15.89:0/3582085902 conn(0x7f9514066f50 msgr2=0x7f951409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.740+0000 7f951bf46700 1 -- 172.21.15.89:0/3582085902 shutdown_connections 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.740+0000 7f951bf46700 1 -- 172.21.15.89:0/3582085902 wait complete. 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.741+0000 7f951bf46700 1 Processor -- start 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.741+0000 7f951bf46700 1 -- start start 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.741+0000 7f951bf46700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951409d9f0 0x7f95141c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.741+0000 7f951bf46700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95141316c0 0x7f95141c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.741+0000 7f951bf46700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 0x7f95141c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.741+0000 7f951bf46700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514135120 con 0x7f951409d9f0 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.741+0000 7f951bf46700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9514134880 con 0x7f9514136a70 2023-09-30T22:18:33.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.741+0000 7f951bf46700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9514135740 con 0x7f95141316c0 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951409d9f0 0x7f95141c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951409d9f0 0x7f95141c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48824/0 (socket says 172.21.15.89:48824) 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 -- 172.21.15.89:0/1411632335 learned_addr learned my addr 172.21.15.89:0/1411632335 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f951a4e3700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 0x7f95141c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f95194e1700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95141316c0 0x7f95141c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 -- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95141316c0 msgr2=0x7f95141c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95141316c0 0x7f95141c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 -- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 msgr2=0x7f95141c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 0x7f95141c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 -- 172.21.15.89:0/1411632335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9504009550 con 0x7f951409d9f0 2023-09-30T22:18:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f9519ce2700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951409d9f0 0x7f95141c6240 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f951000c4e0 tx=0x7f951000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f950a7fc700 1 -- 172.21.15.89:0/1411632335 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100052d0 con 0x7f951409d9f0 2023-09-30T22:18:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f950a7fc700 1 -- 172.21.15.89:0/1411632335 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9510005430 con 0x7f951409d9f0 2023-09-30T22:18:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f950a7fc700 1 -- 172.21.15.89:0/1411632335 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100038c0 con 0x7f951409d9f0 2023-09-30T22:18:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.742+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951412a7b0 con 0x7f951409d9f0 2023-09-30T22:18:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.743+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95141324e0 con 0x7f951409d9f0 2023-09-30T22:18:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.744+0000 7f950a7fc700 1 -- 172.21.15.89:0/1411632335 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9510003a20 con 0x7f951409d9f0 2023-09-30T22:18:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.744+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951412ab00 con 0x7f951409d9f0 2023-09-30T22:18:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.744+0000 7f950a7fc700 1 -- 172.21.15.89:0/1411632335 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f951003e9a0 con 0x7f951409d9f0 2023-09-30T22:18:33.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.753+0000 7f950a7fc700 1 -- 172.21.15.89:0/1411632335 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9510011590 con 0x7f951409d9f0 2023-09-30T22:18:33.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.772+0000 7f6dce3e7700 1 Processor -- start 2023-09-30T22:18:33.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.772+0000 7f6dce3e7700 1 -- start start 2023-09-30T22:18:33.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.772+0000 7f6dce3e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc806f5d0 0x7f6dc812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.772+0000 7f6dce3e7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6dc8135c20 0x7f6dc8138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.772+0000 7f6dce3e7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc806fa40 con 0x7f6dc812cf60 2023-09-30T22:18:33.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.772+0000 7f6dce3e7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc812cd90 con 0x7f6dc8135c20 2023-09-30T22:18:33.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.772+0000 7f6dce3e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8071780 con 0x7f6dc806f5d0 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc806f5d0 0x7f6dc812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc806f5d0 0x7f6dc812ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51432/0 (socket says 172.21.15.89:51432) 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc77fe700 1 -- 172.21.15.89:0/2954203848 learned_addr learned my addr 172.21.15.89:0/2954203848 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc77fe700 1 -- 172.21.15.89:0/2954203848 >> v2:172.21.15.89:3300/0 conn(0x7f6dc8135c20 msgr2=0x7f6dc8138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc77fe700 1 --2- 172.21.15.89:0/2954203848 >> v2:172.21.15.89:3300/0 conn(0x7f6dc8135c20 0x7f6dc8138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc77fe700 1 -- 172.21.15.89:0/2954203848 >> v1:172.21.15.89:6789/0 conn(0x7f6dc812cf60 legacy=0x7f6dc812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc77fe700 1 -- 172.21.15.89:0/2954203848 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8071a40 con 0x7f6dc806f5d0 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc77fe700 1 --2- 172.21.15.89:0/2954203848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc806f5d0 0x7f6dc812ffc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0098f0 tx=0x7f6dbc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d11906363f706bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc6ffd700 1 -- 172.21.15.89:0/2954203848 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00b8a0 con 0x7f6dc806f5d0 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc6ffd700 1 -- 172.21.15.89:0/2954203848 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc00ba00 con 0x7f6dc806f5d0 2023-09-30T22:18:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.773+0000 7f6dc6ffd700 1 -- 172.21.15.89:0/2954203848 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00bd90 con 0x7f6dc806f5d0 2023-09-30T22:18:33.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.774+0000 7f6dce3e7700 1 -- 172.21.15.89:0/2954203848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc806f5d0 msgr2=0x7f6dc812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.774+0000 7f6dce3e7700 1 --2- 172.21.15.89:0/2954203848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc806f5d0 0x7f6dc812ffc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0098f0 tx=0x7f6dbc009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:33.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.774+0000 7f6dce3e7700 1 -- 172.21.15.89:0/2954203848 shutdown_connections 2023-09-30T22:18:33.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.774+0000 7f6dce3e7700 1 --2- 172.21.15.89:0/2954203848 >> v2:172.21.15.89:3300/0 conn(0x7f6dc8135c20 0x7f6dc8138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.774+0000 7f6dce3e7700 1 --2- 172.21.15.89:0/2954203848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc806f5d0 0x7f6dc812ffc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.774+0000 7f6dce3e7700 1 -- 172.21.15.89:0/2954203848 >> 172.21.15.89:0/2954203848 conn(0x7f6dc80a2aa0 msgr2=0x7f6dc80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:33.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.774+0000 7f6dce3e7700 1 -- 172.21.15.89:0/2954203848 shutdown_connections 2023-09-30T22:18:33.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.774+0000 7f6dce3e7700 1 -- 172.21.15.89:0/2954203848 wait complete. 2023-09-30T22:18:33.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.775+0000 7f6dce3e7700 1 Processor -- start 2023-09-30T22:18:33.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.775+0000 7f6dce3e7700 1 -- start start 2023-09-30T22:18:33.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dce3e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6dc806f5d0 0x7f6dc8099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dce3e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc812cf60 0x7f6dc809a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dce3e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dc8135c20 0x7f6dc809a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dce3e7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc809d8c0 con 0x7f6dc8135c20 2023-09-30T22:18:33.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dce3e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc809e330 con 0x7f6dc806f5d0 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dce3e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc809cf30 con 0x7f6dc812cf60 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6dc806f5d0 0x7f6dc8099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6dc806f5d0 0x7f6dc8099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35642/0 (socket says 172.21.15.89:35642) 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc7fff700 1 -- 172.21.15.89:0/3955221958 learned_addr learned my addr 172.21.15.89:0/3955221958 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc77fe700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc812cf60 0x7f6dc809a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dcc984700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dc8135c20 0x7f6dc809a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc77fe700 1 -- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6dc806f5d0 msgr2=0x7f6dc8099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc77fe700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6dc806f5d0 0x7f6dc8099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc77fe700 1 -- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dc8135c20 msgr2=0x7f6dc809a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc77fe700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dc8135c20 0x7f6dc809a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc77fe700 1 -- 172.21.15.89:0/3955221958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc009550 con 0x7f6dc812cf60 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc7fff700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6dc806f5d0 0x7f6dc8099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.776+0000 7f6dc77fe700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc812cf60 0x7f6dc809a290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0098f0 tx=0x7f6dbc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.777+0000 7f6dc4ff9700 1 -- 172.21.15.89:0/3955221958 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc004510 con 0x7f6dc812cf60 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.777+0000 7f6dc4ff9700 1 -- 172.21.15.89:0/3955221958 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc0036a0 con 0x7f6dc812cf60 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.777+0000 7f6dc4ff9700 1 -- 172.21.15.89:0/3955221958 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc003a10 con 0x7f6dc812cf60 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.777+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc809d470 con 0x7f6dc812cf60 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.777+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc809c030 con 0x7f6dc812cf60 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.778+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc809b900 con 0x7f6dc812cf60 2023-09-30T22:18:33.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.778+0000 7f6dc4ff9700 1 -- 172.21.15.89:0/3955221958 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6dbc021070 con 0x7f6dc812cf60 2023-09-30T22:18:33.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.778+0000 7f6dc4ff9700 1 -- 172.21.15.89:0/3955221958 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6dbc040ad0 con 0x7f6dc812cf60 2023-09-30T22:18:33.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.779+0000 7fb492e17700 1 Processor -- start 2023-09-30T22:18:33.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.779+0000 7fb492e17700 1 -- start start 2023-09-30T22:18:33.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb492e17700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09d6f0 0x7fb48c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb492e17700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb48c09e000 0x7fb48c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb492e17700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09e410 0x7fb48c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb492e17700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c06b460 con 0x7fb48c09e000 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb492e17700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c09b120 con 0x7fb48c09d6f0 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb492e17700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c09a7e0 con 0x7fb48c09e410 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb48c09e000 0x7fb48c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb48c09e000 0x7fb48c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48850/0 (socket says 172.21.15.89:48850) 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 -- 172.21.15.89:0/1681738083 learned_addr learned my addr 172.21.15.89:0/1681738083 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb4913b4700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09e410 0x7fb48c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb490bb3700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09d6f0 0x7fb48c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 -- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09e410 msgr2=0x7fb48c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:33.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09e410 0x7fb48c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 -- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09d6f0 msgr2=0x7fb48c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09d6f0 0x7fb48c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 -- 172.21.15.89:0/1681738083 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c09a500 con 0x7fb48c09e000 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.780+0000 7fb48bfff700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e000 0x7fb48c1339f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb4780098f0 tx=0x7fb478009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0aaa8b1e5aa9bdc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb48b7fe700 1 -- 172.21.15.89:0/1681738083 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47800b8a0 con 0x7fb48c09e000 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb48b7fe700 1 -- 172.21.15.89:0/1681738083 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47800ba00 con 0x7fb48c09e000 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb48b7fe700 1 -- 172.21.15.89:0/1681738083 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47800bd90 con 0x7fb48c09e000 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 -- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e000 msgr2=0x7fb48c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e000 0x7fb48c1339f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb4780098f0 tx=0x7fb478009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 -- 172.21.15.89:0/1681738083 shutdown_connections 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09e410 0x7fb48c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09d6f0 0x7fb48c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 --2- 172.21.15.89:0/1681738083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e000 0x7fb48c1339f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 -- 172.21.15.89:0/1681738083 >> 172.21.15.89:0/1681738083 conn(0x7fb48c066f50 msgr2=0x7fb48c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 -- 172.21.15.89:0/1681738083 shutdown_connections 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.781+0000 7fb492e17700 1 -- 172.21.15.89:0/1681738083 wait complete. 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.782+0000 7fb492e17700 1 Processor -- start 2023-09-30T22:18:33.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.782+0000 7fb492e17700 1 -- start start 2023-09-30T22:18:33.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb492e17700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09d6f0 0x7fb48c1cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7f6dc4ff9700 1 -- 172.21.15.89:0/3955221958 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6dbc02d760 con 0x7f6dc812cf60 2023-09-30T22:18:33.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7f9d69aad700 1 Processor -- start 2023-09-30T22:18:33.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb492e17700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09e000 0x7fb48c1ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb492e17700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e410 0x7fb48c127350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb492e17700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c09c710 con 0x7fb48c09e410 2023-09-30T22:18:33.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb492e17700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c1292d0 con 0x7fb48c09e000 2023-09-30T22:18:33.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb492e17700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c072840 con 0x7fb48c09d6f0 2023-09-30T22:18:33.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb490bb3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09d6f0 0x7fb48c1cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb490bb3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09d6f0 0x7fb48c1cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51470/0 (socket says 172.21.15.89:51470) 2023-09-30T22:18:33.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb490bb3700 1 -- 172.21.15.89:0/2583583831 learned_addr learned my addr 172.21.15.89:0/2583583831 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:33.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7fb4913b4700 1 --2- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e410 0x7fb48c127350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.783+0000 7f9d69aad700 1 -- start start 2023-09-30T22:18:33.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb490bb3700 1 -- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09e000 msgr2=0x7fb48c1ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb490bb3700 1 --2- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09e000 0x7fb48c1ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb490bb3700 1 -- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e410 msgr2=0x7fb48c127350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb490bb3700 1 --2- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e410 0x7fb48c127350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb490bb3700 1 -- 172.21.15.89:0/2583583831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478009550 con 0x7fb48c09d6f0 2023-09-30T22:18:33.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d69aad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409e1d0 0x7f9d64135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb490bb3700 1 --2- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09d6f0 0x7fb48c1cc8a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb48000e840 tx=0x7fb48000eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:33.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb4897fa700 1 -- 172.21.15.89:0/2583583831 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480013610 con 0x7fb48c09d6f0 2023-09-30T22:18:33.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb4897fa700 1 -- 172.21.15.89:0/2583583831 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb480013770 con 0x7fb48c09d6f0 2023-09-30T22:18:33.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7fb4897fa700 1 -- 172.21.15.89:0/2583583831 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480013b20 con 0x7fb48c09d6f0 2023-09-30T22:18:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d69aad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d64135e50 0x7f9d64138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d69aad700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6409f130 con 0x7f9d6409d6f0 2023-09-30T22:18:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d62ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409e1d0 0x7f9d64135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d63fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d64135e50 0x7f9d64138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d63fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d64135e50 0x7f9d64138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48874/0 (socket says 172.21.15.89:48874) 2023-09-30T22:18:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d63fff700 1 -- 172.21.15.89:0/3441173832 learned_addr learned my addr 172.21.15.89:0/3441173832 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d69aad700 1 -- 172.21.15.89:0/3441173832 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6409f520 con 0x7f9d64135e50 2023-09-30T22:18:33.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.784+0000 7f9d69aad700 1 -- 172.21.15.89:0/3441173832 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6409e930 con 0x7f9d6409e1d0 2023-09-30T22:18:33.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7f9d627fc700 1 -- 172.21.15.89:0/3441173832 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765082702 0 0) 0x7f9d6409f130 con 0x7f9d6409d6f0 2023-09-30T22:18:33.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7f9d627fc700 1 -- 172.21.15.89:0/3441173832 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d4c0035b0 con 0x7f9d6409d6f0 2023-09-30T22:18:33.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7f9d63fff700 1 -- 172.21.15.89:0/3441173832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409e1d0 msgr2=0x7f9d64135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:33.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7f9d63fff700 1 --2- 172.21.15.89:0/3441173832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409e1d0 0x7f9d64135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7f9d63fff700 1 -- 172.21.15.89:0/3441173832 >> v1:172.21.15.89:6789/0 conn(0x7f9d6409d6f0 legacy=0x7f9d6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7f9d63fff700 1 -- 172.21.15.89:0/3441173832 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6409b2d0 con 0x7f9d64135e50 2023-09-30T22:18:33.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7f9d63fff700 1 --2- 172.21.15.89:0/3441173832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d64135e50 0x7f9d64138230 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00f3a0 tx=0x7f9d5c00f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eaa46aa7f6c0e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:33.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c128de0 con 0x7fb48c09d6f0 2023-09-30T22:18:33.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.785+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c12a8e0 con 0x7fb48c09d6f0 2023-09-30T22:18:33.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.786+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c074ea0 con 0x7fb48c09d6f0 2023-09-30T22:18:33.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.786+0000 7fb4897fa700 1 -- 172.21.15.89:0/2583583831 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb480013c80 con 0x7fb48c09d6f0 2023-09-30T22:18:33.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.786+0000 7f9d627fc700 1 -- 172.21.15.89:0/3441173832 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c012070 con 0x7f9d64135e50 2023-09-30T22:18:33.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.786+0000 7f9d627fc700 1 -- 172.21.15.89:0/3441173832 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c00d0d0 con 0x7f9d64135e50 2023-09-30T22:18:33.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.786+0000 7f9d627fc700 1 -- 172.21.15.89:0/3441173832 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c00d440 con 0x7f9d64135e50 2023-09-30T22:18:33.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.786+0000 7fb4897fa700 1 -- 172.21.15.89:0/2583583831 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb48003f520 con 0x7fb48c09d6f0 2023-09-30T22:18:33.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.787+0000 7f9d69aad700 1 -- 172.21.15.89:0/3441173832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d64135e50 msgr2=0x7f9d64138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.787+0000 7f9d69aad700 1 --2- 172.21.15.89:0/3441173832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d64135e50 0x7f9d64138230 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00f3a0 tx=0x7f9d5c00f6b0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.790+0000 7f9d69aad700 1 -- 172.21.15.89:0/3441173832 shutdown_connections 2023-09-30T22:18:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.790+0000 7f9d69aad700 1 --2- 172.21.15.89:0/3441173832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d64135e50 0x7f9d64138230 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.790+0000 7f9d69aad700 1 --2- 172.21.15.89:0/3441173832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409e1d0 0x7f9d64135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.790+0000 7f9d69aad700 1 -- 172.21.15.89:0/3441173832 >> 172.21.15.89:0/3441173832 conn(0x7f9d64066f50 msgr2=0x7f9d6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.791+0000 7f9d69aad700 1 -- 172.21.15.89:0/3441173832 shutdown_connections 2023-09-30T22:18:33.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.791+0000 7f9d69aad700 1 -- 172.21.15.89:0/3441173832 wait complete. 2023-09-30T22:18:33.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.791+0000 7fb4897fa700 1 -- 172.21.15.89:0/2583583831 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb480012370 con 0x7fb48c09d6f0 2023-09-30T22:18:33.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.792+0000 7f9d69aad700 1 Processor -- start 2023-09-30T22:18:33.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.793+0000 7f9d69aad700 1 -- start start 2023-09-30T22:18:33.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.793+0000 7f9d69aad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409d6f0 0x7f9d641cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.793+0000 7f9d69aad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6409e1d0 0x7f9d641cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.793+0000 7f9d69aad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d64135e50 0x7f9d641cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.793+0000 7f9d69aad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d640a20d0 con 0x7f9d6409e1d0 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.793+0000 7f9d69aad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d641256b0 con 0x7f9d6409d6f0 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6409e1d0 0x7f9d641cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d69aad700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d641264b0 con 0x7f9d64135e50 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6409e1d0 0x7f9d641cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48880/0 (socket says 172.21.15.89:48880) 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 -- 172.21.15.89:0/27132534 learned_addr learned my addr 172.21.15.89:0/27132534 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d637fe700 1 --2- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409d6f0 0x7f9d641cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 -- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d64135e50 msgr2=0x7f9d641cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 --2- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d64135e50 0x7f9d641cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 -- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409d6f0 msgr2=0x7f9d641cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 --2- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409d6f0 0x7f9d641cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 -- 172.21.15.89:0/27132534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c00f050 con 0x7f9d6409e1d0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d62ffd700 1 --2- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6409e1d0 0x7f9d641cc900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e840 tx=0x7f9d5400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d4bfff700 1 -- 172.21.15.89:0/27132534 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54013610 con 0x7f9d6409e1d0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.794+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6409c760 con 0x7f9d6409e1d0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.795+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d64127bd0 con 0x7f9d6409e1d0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.795+0000 7f9d4bfff700 1 -- 172.21.15.89:0/27132534 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d54013770 con 0x7f9d6409e1d0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.795+0000 7f9d4bfff700 1 -- 172.21.15.89:0/27132534 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54013b20 con 0x7f9d6409e1d0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.796+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d440052e0 con 0x7f9d6409e1d0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.796+0000 7f9d4bfff700 1 -- 172.21.15.89:0/27132534 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d54013c80 con 0x7f9d6409e1d0 2023-09-30T22:18:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.796+0000 7f9d4bfff700 1 -- 172.21.15.89:0/27132534 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d5403f4a0 con 0x7f9d6409e1d0 2023-09-30T22:18:33.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.805+0000 7f9d4bfff700 1 -- 172.21.15.89:0/27132534 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d54012370 con 0x7f9d6409e1d0 2023-09-30T22:18:33.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.953+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f951409e130 con 0x7f951409d9f0 2023-09-30T22:18:33.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.953+0000 7f950a7fc700 1 -- 172.21.15.89:0/1411632335 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f951409e130 con 0x7f951409d9f0 2023-09-30T22:18:33.954 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:33.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.955+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951409d9f0 msgr2=0x7f95141c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.955+0000 7f951bf46700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951409d9f0 0x7f95141c6240 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f951000c4e0 tx=0x7f951000c7f0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.956+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 shutdown_connections 2023-09-30T22:18:33.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.956+0000 7f951bf46700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9514136a70 0x7f95141c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.956+0000 7f951bf46700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951409d9f0 0x7f95141c6240 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.956+0000 7f951bf46700 1 --2- 172.21.15.89:0/1411632335 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95141316c0 0x7f95141c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.956+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 >> 172.21.15.89:0/1411632335 conn(0x7f9514066f50 msgr2=0x7f95140a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:33.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.956+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 shutdown_connections 2023-09-30T22:18:33.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.956+0000 7f951bf46700 1 -- 172.21.15.89:0/1411632335 wait complete. 2023-09-30T22:18:33.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:33.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.985+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6dc8074ea0 con 0x7f6dc812cf60 2023-09-30T22:18:33.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.985+0000 7f6dc4ff9700 1 -- 172.21.15.89:0/3955221958 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6dbc029030 con 0x7f6dc812cf60 2023-09-30T22:18:33.986 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:33.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.986+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb48c12a560 con 0x7fb48c09d6f0 2023-09-30T22:18:33.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.986+0000 7fb4897fa700 1 -- 172.21.15.89:0/2583583831 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb480021020 con 0x7fb48c09d6f0 2023-09-30T22:18:33.987 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:33.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc812cf60 msgr2=0x7f6dc809a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc812cf60 0x7f6dc809a290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0098f0 tx=0x7f6dbc004750 comp rx=0 tx=0).stop 2023-09-30T22:18:33.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 shutdown_connections 2023-09-30T22:18:33.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dc8135c20 0x7f6dc809a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6dc806f5d0 0x7f6dc8099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 --2- 172.21.15.89:0/3955221958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dc812cf60 0x7f6dc809a290 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 >> 172.21.15.89:0/3955221958 conn(0x7f6dc80a2aa0 msgr2=0x7f6dc80a0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:33.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 shutdown_connections 2023-09-30T22:18:33.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.987+0000 7f6dce3e7700 1 -- 172.21.15.89:0/3955221958 wait complete. 2023-09-30T22:18:33.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.988+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09d6f0 msgr2=0x7fb48c1cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:33.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.988+0000 7fb492e17700 1 --2- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09d6f0 0x7fb48c1cc8a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb48000e840 tx=0x7fb48000eb50 comp rx=0 tx=0).stop 2023-09-30T22:18:33.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.989+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 shutdown_connections 2023-09-30T22:18:33.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.989+0000 7fb492e17700 1 --2- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb48c09e410 0x7fb48c127350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.989+0000 7fb492e17700 1 --2- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb48c09d6f0 0x7fb48c1cc8a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.989+0000 7fb492e17700 1 --2- 172.21.15.89:0/2583583831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb48c09e000 0x7fb48c1ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:33.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.989+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 >> 172.21.15.89:0/2583583831 conn(0x7fb48c066f50 msgr2=0x7fb48c06f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:33.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.989+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 shutdown_connections 2023-09-30T22:18:33.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.989+0000 7fb492e17700 1 -- 172.21.15.89:0/2583583831 wait complete. 2023-09-30T22:18:33.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.998+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9d44005590 con 0x7f9d6409e1d0 2023-09-30T22:18:33.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:33.999+0000 7f9d4bfff700 1 -- 172.21.15.89:0/27132534 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d54010310 con 0x7f9d6409e1d0 2023-09-30T22:18:34.000 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6409e1d0 msgr2=0x7f9d641cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 --2- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6409e1d0 0x7f9d641cc900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e840 tx=0x7f9d5400eb50 comp rx=0 tx=0).stop 2023-09-30T22:18:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 shutdown_connections 2023-09-30T22:18:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 --2- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d64135e50 0x7f9d641cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 --2- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d6409d6f0 0x7f9d641cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:34.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 --2- 172.21.15.89:0/27132534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d6409e1d0 0x7f9d641cc900 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:34.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 >> 172.21.15.89:0/27132534 conn(0x7f9d64066f50 msgr2=0x7f9d64132680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:34.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 shutdown_connections 2023-09-30T22:18:34.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:34.001+0000 7f9d69aad700 1 -- 172.21.15.89:0/27132534 wait complete. 2023-09-30T22:18:34.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:34.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:34.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:34.970 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:35.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:35.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:35.016 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:35.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.124+0000 7f9b49cf7700 1 Processor -- start 2023-09-30T22:18:35.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.124+0000 7f9b49cf7700 1 -- start start 2023-09-30T22:18:35.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b49cf7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b49cf7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b49cf7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b4409b120 con 0x7f9b4409e000 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b49cf7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4406b460 con 0x7f9b4409e410 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b49cf7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4409a7e0 con 0x7f9b4409d6f0 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b4409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51484/0 (socket says 172.21.15.89:51484) 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b437fe700 1 -- 172.21.15.89:0/877482657 learned_addr learned my addr 172.21.15.89:0/877482657 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b43fff700 1 --2- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b437fe700 1 -- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 msgr2=0x7f9b44134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b437fe700 1 --2- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b437fe700 1 -- 172.21.15.89:0/877482657 >> v1:172.21.15.89:6789/0 conn(0x7f9b4409e000 legacy=0x7f9b441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b437fe700 1 -- 172.21.15.89:0/877482657 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4409a500 con 0x7f9b4409d6f0 2023-09-30T22:18:35.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.125+0000 7f9b43fff700 1 --2- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:35.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.126+0000 7f9b437fe700 1 --2- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b4409dac0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009950 tx=0x7f9b34009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed35a6ad05fb4dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:35.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.126+0000 7f9b427fc700 1 -- 172.21.15.89:0/877482657 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34011830 con 0x7f9b4409d6f0 2023-09-30T22:18:35.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.126+0000 7f9b427fc700 1 -- 172.21.15.89:0/877482657 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b34011990 con 0x7f9b4409d6f0 2023-09-30T22:18:35.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.126+0000 7f9b427fc700 1 -- 172.21.15.89:0/877482657 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34011d40 con 0x7f9b4409d6f0 2023-09-30T22:18:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.127+0000 7f9b49cf7700 1 -- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 msgr2=0x7f9b4409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.127+0000 7f9b49cf7700 1 --2- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b4409dac0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009950 tx=0x7f9b34009c60 comp rx=0 tx=0).stop 2023-09-30T22:18:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.127+0000 7f9b49cf7700 1 -- 172.21.15.89:0/877482657 shutdown_connections 2023-09-30T22:18:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.127+0000 7f9b49cf7700 1 --2- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.127+0000 7f9b49cf7700 1 --2- 172.21.15.89:0/877482657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b4409dac0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.127+0000 7f9b49cf7700 1 -- 172.21.15.89:0/877482657 >> 172.21.15.89:0/877482657 conn(0x7f9b44066f50 msgr2=0x7f9b4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.127+0000 7f9b49cf7700 1 -- 172.21.15.89:0/877482657 shutdown_connections 2023-09-30T22:18:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.127+0000 7f9b49cf7700 1 -- 172.21.15.89:0/877482657 wait complete. 2023-09-30T22:18:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.128+0000 7f9b49cf7700 1 Processor -- start 2023-09-30T22:18:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.128+0000 7f9b49cf7700 1 -- start start 2023-09-30T22:18:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b49cf7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b441224e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b49cf7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b4409e000 0x7f9b44122a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b49cf7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44122f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b49cf7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441260a0 con 0x7f9b4409e000 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b49cf7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440a1e90 con 0x7f9b4409e410 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b49cf7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4409c710 con 0x7f9b4409d6f0 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b441224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b441224e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51498/0 (socket says 172.21.15.89:51498) 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b437fe700 1 -- 172.21.15.89:0/492647565 learned_addr learned my addr 172.21.15.89:0/492647565 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.129+0000 7f9b42ffd700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b4409e000 0x7f9b44122a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b43fff700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44122f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b437fe700 1 -- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 msgr2=0x7f9b44122f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b437fe700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44122f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b437fe700 1 -- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b4409e000 msgr2=0x7f9b44122a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b437fe700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b4409e000 0x7f9b44122a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b437fe700 1 -- 172.21.15.89:0/492647565 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c009a70 con 0x7f9b4409d6f0 2023-09-30T22:18:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b437fe700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b441224e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9b4409a970 tx=0x7f9b34004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b2bfff700 1 -- 172.21.15.89:0/492647565 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34004240 con 0x7f9b4409d6f0 2023-09-30T22:18:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b34009550 con 0x7f9b4409d6f0 2023-09-30T22:18:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b44125790 con 0x7f9b4409d6f0 2023-09-30T22:18:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b2bfff700 1 -- 172.21.15.89:0/492647565 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3401f070 con 0x7f9b4409d6f0 2023-09-30T22:18:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.130+0000 7f9b2bfff700 1 -- 172.21.15.89:0/492647565 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340045c0 con 0x7f9b4409d6f0 2023-09-30T22:18:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.132+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b240052e0 con 0x7f9b4409d6f0 2023-09-30T22:18:35.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.132+0000 7f9b2bfff700 1 -- 172.21.15.89:0/492647565 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b34021030 con 0x7f9b4409d6f0 2023-09-30T22:18:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.137+0000 7f9b2bfff700 1 -- 172.21.15.89:0/492647565 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b340036a0 con 0x7f9b4409d6f0 2023-09-30T22:18:35.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.141+0000 7f9b2bfff700 1 -- 172.21.15.89:0/492647565 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b3401a450 con 0x7f9b4409d6f0 2023-09-30T22:18:35.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.150+0000 7f5b8f813700 1 Processor -- start 2023-09-30T22:18:35.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.150+0000 7f5b8f813700 1 -- start start 2023-09-30T22:18:35.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.150+0000 7f5b8f813700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809d6f0 0x7f5b8809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.150+0000 7f5b8f813700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b8809e000 0x7f5b881339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8f813700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8809b120 con 0x7f5b8809e410 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8f813700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b8806b460 con 0x7f5b8809e000 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8f813700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8809a7e0 con 0x7f5b8809d6f0 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8d5af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809d6f0 0x7f5b8809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8d5af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809d6f0 0x7f5b8809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51514/0 (socket says 172.21.15.89:51514) 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8d5af700 1 -- 172.21.15.89:0/833460149 learned_addr learned my addr 172.21.15.89:0/833460149 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8cdae700 1 --2- 172.21.15.89:0/833460149 >> v2:172.21.15.89:3300/0 conn(0x7f5b8809e000 0x7f5b881339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8d5af700 1 -- 172.21.15.89:0/833460149 >> v2:172.21.15.89:3300/0 conn(0x7f5b8809e000 msgr2=0x7f5b881339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8d5af700 1 --2- 172.21.15.89:0/833460149 >> v2:172.21.15.89:3300/0 conn(0x7f5b8809e000 0x7f5b881339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8d5af700 1 -- 172.21.15.89:0/833460149 >> v1:172.21.15.89:6789/0 conn(0x7f5b8809e410 legacy=0x7f5b88133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8d5af700 1 -- 172.21.15.89:0/833460149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8809a500 con 0x7f5b8809d6f0 2023-09-30T22:18:35.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8cdae700 1 --2- 172.21.15.89:0/833460149 >> v2:172.21.15.89:3300/0 conn(0x7f5b8809e000 0x7f5b881339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b8d5af700 1 --2- 172.21.15.89:0/833460149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809d6f0 0x7f5b8809dac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009920 tx=0x7f5b84009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ca1b43f85faf7e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.151+0000 7f5b7ffff700 1 -- 172.21.15.89:0/833460149 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84004230 con 0x7f5b8809d6f0 2023-09-30T22:18:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b7ffff700 1 -- 172.21.15.89:0/833460149 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84004390 con 0x7f5b8809d6f0 2023-09-30T22:18:35.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b7ffff700 1 -- 172.21.15.89:0/833460149 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84009e70 con 0x7f5b8809d6f0 2023-09-30T22:18:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b8f813700 1 -- 172.21.15.89:0/833460149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809d6f0 msgr2=0x7f5b8809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b8f813700 1 --2- 172.21.15.89:0/833460149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809d6f0 0x7f5b8809dac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009920 tx=0x7f5b84009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b8f813700 1 -- 172.21.15.89:0/833460149 shutdown_connections 2023-09-30T22:18:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b8f813700 1 --2- 172.21.15.89:0/833460149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809d6f0 0x7f5b8809dac0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b8f813700 1 --2- 172.21.15.89:0/833460149 >> v2:172.21.15.89:3300/0 conn(0x7f5b8809e000 0x7f5b881339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b8f813700 1 -- 172.21.15.89:0/833460149 >> 172.21.15.89:0/833460149 conn(0x7f5b88066f50 msgr2=0x7f5b8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b8f813700 1 -- 172.21.15.89:0/833460149 shutdown_connections 2023-09-30T22:18:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.152+0000 7f5b8f813700 1 -- 172.21.15.89:0/833460149 wait complete. 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.153+0000 7f5b8f813700 1 Processor -- start 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.153+0000 7f5b8f813700 1 -- start start 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8f813700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8809d6f0 0x7f5b881aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8f813700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809e000 0x7f5b881abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8f813700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b8809e410 0x7f5b881ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8f813700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b881adc70 con 0x7f5b8809d6f0 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8f813700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8809c780 con 0x7f5b8809e410 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8f813700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880a32c0 con 0x7f5b8809e000 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8ddb0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b8809e410 0x7f5b881ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8ddb0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b8809e410 0x7f5b881ac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35714/0 (socket says 172.21.15.89:35714) 2023-09-30T22:18:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8ddb0700 1 -- 172.21.15.89:0/4054950504 learned_addr learned my addr 172.21.15.89:0/4054950504 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8d5af700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8809d6f0 0x7f5b881aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.154+0000 7f5b8cdae700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809e000 0x7f5b881abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b8ddb0700 1 -- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809e000 msgr2=0x7f5b881abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b8ddb0700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809e000 0x7f5b881abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b8ddb0700 1 -- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8809d6f0 msgr2=0x7f5b881aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b8ddb0700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8809d6f0 0x7f5b881aba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b8ddb0700 1 -- 172.21.15.89:0/4054950504 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b84009550 con 0x7f5b8809e410 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b8ddb0700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b8809e410 0x7f5b881ac4c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c7a0 tx=0x7f5b8000cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b7dffb700 1 -- 172.21.15.89:0/4054950504 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80003e10 con 0x7f5b8809e410 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b7dffb700 1 -- 172.21.15.89:0/4054950504 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80005070 con 0x7f5b8809e410 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b7dffb700 1 -- 172.21.15.89:0/4054950504 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80005420 con 0x7f5b8809e410 2023-09-30T22:18:35.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b881ae8a0 con 0x7f5b8809e410 2023-09-30T22:18:35.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.155+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b881ade40 con 0x7f5b8809e410 2023-09-30T22:18:35.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.156+0000 7f5b7dffb700 1 -- 172.21.15.89:0/4054950504 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b80005580 con 0x7f5b8809e410 2023-09-30T22:18:35.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.157+0000 7f5b7dffb700 1 -- 172.21.15.89:0/4054950504 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5b8003e940 con 0x7f5b8809e410 2023-09-30T22:18:35.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.157+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b88074ea0 con 0x7f5b8809e410 2023-09-30T22:18:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80f31a1700 1 Processor -- start 2023-09-30T22:18:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80f31a1700 1 -- start start 2023-09-30T22:18:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80f31a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f80ec09d6f0 0x7f80ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80f31a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec09e000 0x7f80ec13bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80f31a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f80ec13c470 0x7f80ec13e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80f31a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec0992c0 con 0x7f80ec09d6f0 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80f31a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec099520 con 0x7f80ec13c470 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80f31a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec0997e0 con 0x7f80ec09e000 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80ebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec09e000 0x7f80ec13bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80ebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec09e000 0x7f80ec13bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51530/0 (socket says 172.21.15.89:51530) 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.165+0000 7f80ebfff700 1 -- 172.21.15.89:0/389298798 learned_addr learned my addr 172.21.15.89:0/389298798 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80f0f3d700 1 --2- 172.21.15.89:0/389298798 >> v2:172.21.15.89:3300/0 conn(0x7f80ec09d6f0 0x7f80ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80ebfff700 1 -- 172.21.15.89:0/389298798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f80ec13c470 msgr2=0x7f80ec13e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80ebfff700 1 --2- 172.21.15.89:0/389298798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f80ec13c470 0x7f80ec13e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80ebfff700 1 -- 172.21.15.89:0/389298798 >> v2:172.21.15.89:3300/0 conn(0x7f80ec09d6f0 msgr2=0x7f80ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80ebfff700 1 --2- 172.21.15.89:0/389298798 >> v2:172.21.15.89:3300/0 conn(0x7f80ec09d6f0 0x7f80ec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80ebfff700 1 -- 172.21.15.89:0/389298798 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec099b30 con 0x7f80ec09e000 2023-09-30T22:18:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80ebfff700 1 --2- 172.21.15.89:0/389298798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec09e000 0x7f80ec13bf30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009920 tx=0x7f80d8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fbf1e8e79e43018 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80eb7fe700 1 -- 172.21.15.89:0/389298798 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8011830 con 0x7f80ec09e000 2023-09-30T22:18:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f5b7dffb700 1 -- 172.21.15.89:0/4054950504 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b8000eb00 con 0x7f5b8809e410 2023-09-30T22:18:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80eb7fe700 1 -- 172.21.15.89:0/389298798 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8011990 con 0x7f80ec09e000 2023-09-30T22:18:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.166+0000 7f80eb7fe700 1 -- 172.21.15.89:0/389298798 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8011d20 con 0x7f80ec09e000 2023-09-30T22:18:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.167+0000 7f80f31a1700 1 -- 172.21.15.89:0/389298798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec09e000 msgr2=0x7f80ec13bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.167+0000 7f80f31a1700 1 --2- 172.21.15.89:0/389298798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec09e000 0x7f80ec13bf30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009920 tx=0x7f80d8009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.167+0000 7f80f31a1700 1 -- 172.21.15.89:0/389298798 shutdown_connections 2023-09-30T22:18:35.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.167+0000 7f80f31a1700 1 --2- 172.21.15.89:0/389298798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f80ec13c470 0x7f80ec13e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.167+0000 7f80f31a1700 1 --2- 172.21.15.89:0/389298798 >> v2:172.21.15.89:3300/0 conn(0x7f80ec09d6f0 0x7f80ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.167+0000 7f80f31a1700 1 --2- 172.21.15.89:0/389298798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec09e000 0x7f80ec13bf30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.167+0000 7f80f31a1700 1 -- 172.21.15.89:0/389298798 >> 172.21.15.89:0/389298798 conn(0x7f80ec09f470 msgr2=0x7f80ec0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:35.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.170+0000 7f80f31a1700 1 -- 172.21.15.89:0/389298798 shutdown_connections 2023-09-30T22:18:35.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.170+0000 7f80f31a1700 1 -- 172.21.15.89:0/389298798 wait complete. 2023-09-30T22:18:35.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.171+0000 7f80f31a1700 1 Processor -- start 2023-09-30T22:18:35.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.171+0000 7f80f31a1700 1 -- start start 2023-09-30T22:18:35.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f31a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80ec09d6f0 0x7f80ec070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f31a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f80ec09e000 0x7f80ec070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f31a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec13c470 0x7f80ec071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f31a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec127960 con 0x7f80ec09d6f0 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f31a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec13e190 con 0x7f80ec09e000 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f31a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec098c70 con 0x7f80ec13c470 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f0f3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80ec09d6f0 0x7f80ec070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f0f3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80ec09d6f0 0x7f80ec070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48940/0 (socket says 172.21.15.89:48940) 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f0f3d700 1 -- 172.21.15.89:0/1916588919 learned_addr learned my addr 172.21.15.89:0/1916588919 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f173e700 1 --2- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec13c470 0x7f80ec071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f0f3d700 1 -- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec13c470 msgr2=0x7f80ec071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f0f3d700 1 --2- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec13c470 0x7f80ec071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f0f3d700 1 -- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f80ec09e000 msgr2=0x7f80ec070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f0f3d700 1 --2- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f80ec09e000 0x7f80ec070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.172+0000 7f80f0f3d700 1 -- 172.21.15.89:0/1916588919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8009550 con 0x7f80ec09d6f0 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.173+0000 7f80f0f3d700 1 --2- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80ec09d6f0 0x7f80ec070800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f80e000e9e0 tx=0x7f80e000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.173+0000 7f80e97fa700 1 -- 172.21.15.89:0/1916588919 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0009710 con 0x7f80ec09d6f0 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.173+0000 7f80e97fa700 1 -- 172.21.15.89:0/1916588919 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e000c850 con 0x7f80ec09d6f0 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.173+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec128370 con 0x7f80ec09d6f0 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.173+0000 7f80e97fa700 1 -- 172.21.15.89:0/1916588919 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e000cbc0 con 0x7f80ec09d6f0 2023-09-30T22:18:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.173+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec1271f0 con 0x7f80ec09d6f0 2023-09-30T22:18:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.174+0000 7f80e97fa700 1 -- 172.21.15.89:0/1916588919 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f80e000cd20 con 0x7f80ec09d6f0 2023-09-30T22:18:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.174+0000 7f80e97fa700 1 -- 172.21.15.89:0/1916588919 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f80e003f950 con 0x7f80ec09d6f0 2023-09-30T22:18:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.174+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec0747f0 con 0x7f80ec09d6f0 2023-09-30T22:18:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.179+0000 7f80e97fa700 1 -- 172.21.15.89:0/1916588919 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80e0021080 con 0x7f80ec09d6f0 2023-09-30T22:18:35.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.201+0000 7fdca2da4700 1 Processor -- start 2023-09-30T22:18:35.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.201+0000 7fdca2da4700 1 -- start start 2023-09-30T22:18:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.201+0000 7fdca2da4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c0a0a30 0x7fdc9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca2da4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc9c138d80 0x7fdc9c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca2da4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9c0a2a70 con 0x7fdc9c09d630 2023-09-30T22:18:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca2da4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c0a01d0 con 0x7fdc9c138d80 2023-09-30T22:18:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca2da4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c09e2b0 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca0b40700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c0a0a30 0x7fdc9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdc9bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdc9c09d630 0x7fdc9c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52392/0 (socket says 172.21.15.89:52392) 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca0b40700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c0a0a30 0x7fdc9c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35742/0 (socket says 172.21.15.89:35742) 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdc9bfff700 1 -- 172.21.15.89:0/3644959187 learned_addr learned my addr 172.21.15.89:0/3644959187 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca1341700 1 --2- 172.21.15.89:0/3644959187 >> v2:172.21.15.89:3300/0 conn(0x7fdc9c138d80 0x7fdc9c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca1341700 1 -- 172.21.15.89:0/3644959187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c0a0a30 msgr2=0x7fdc9c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca1341700 1 --2- 172.21.15.89:0/3644959187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c0a0a30 0x7fdc9c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca1341700 1 -- 172.21.15.89:0/3644959187 >> v1:172.21.15.89:6789/0 conn(0x7fdc9c09d630 legacy=0x7fdc9c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca1341700 1 -- 172.21.15.89:0/3644959187 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c09dc50 con 0x7fdc9c138d80 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.202+0000 7fdca1341700 1 --2- 172.21.15.89:0/3644959187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c138d80 0x7fdc9c13b160 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009d90 tx=0x7fdc9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42f3c34a17a00df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdc9b7fe700 1 -- 172.21.15.89:0/3644959187 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400e070 con 0x7fdc9c138d80 2023-09-30T22:18:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdc9b7fe700 1 -- 172.21.15.89:0/3644959187 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9400bb20 con 0x7fdc9c138d80 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdc9b7fe700 1 -- 172.21.15.89:0/3644959187 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94012430 con 0x7fdc9c138d80 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdca2da4700 1 -- 172.21.15.89:0/3644959187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c138d80 msgr2=0x7fdc9c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdca2da4700 1 --2- 172.21.15.89:0/3644959187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c138d80 0x7fdc9c13b160 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009d90 tx=0x7fdc9400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdca2da4700 1 -- 172.21.15.89:0/3644959187 shutdown_connections 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdca2da4700 1 --2- 172.21.15.89:0/3644959187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c138d80 0x7fdc9c13b160 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdca2da4700 1 --2- 172.21.15.89:0/3644959187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c0a0a30 0x7fdc9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdca2da4700 1 -- 172.21.15.89:0/3644959187 >> 172.21.15.89:0/3644959187 conn(0x7fdc9c0a3050 msgr2=0x7fdc9c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdca2da4700 1 -- 172.21.15.89:0/3644959187 shutdown_connections 2023-09-30T22:18:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.203+0000 7fdca2da4700 1 -- 172.21.15.89:0/3644959187 wait complete. 2023-09-30T22:18:35.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.205+0000 7fdca2da4700 1 Processor -- start 2023-09-30T22:18:35.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.205+0000 7fdca2da4700 1 -- start start 2023-09-30T22:18:35.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca2da4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c09d630 0x7fdc9c133930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca2da4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a0a30 0x7fdc9c133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca2da4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c138d80 0x7fdc9c1343b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:35.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca2da4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1380a0 con 0x7fdc9c09d630 2023-09-30T22:18:35.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca2da4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1370d0 con 0x7fdc9c138d80 2023-09-30T22:18:35.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca2da4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1362a0 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a0a30 0x7fdc9c133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a0a30 0x7fdc9c133e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51546/0 (socket says 172.21.15.89:51546) 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 -- 172.21.15.89:0/1843528986 learned_addr learned my addr 172.21.15.89:0/1843528986 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca0b40700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c09d630 0x7fdc9c133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca1341700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c138d80 0x7fdc9c1343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 -- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c138d80 msgr2=0x7fdc9c1343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c138d80 0x7fdc9c1343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 -- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c09d630 msgr2=0x7fdc9c133930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c09d630 0x7fdc9c133930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 -- 172.21.15.89:0/1843528986 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc94009a70 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca1341700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c138d80 0x7fdc9c1343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:35.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdca0b40700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c09d630 0x7fdc9c133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.206+0000 7fdc9bfff700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a0a30 0x7fdc9c133e70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800c500 tx=0x7fdc8800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.207+0000 7fdc997fa700 1 -- 172.21.15.89:0/1843528986 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88005d70 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.207+0000 7fdc997fa700 1 -- 172.21.15.89:0/1843528986 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc880070d0 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.207+0000 7fdc997fa700 1 -- 172.21.15.89:0/1843528986 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88007440 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.207+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c0a4d40 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.207+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c137b40 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.208+0000 7fdc997fa700 1 -- 172.21.15.89:0/1843528986 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc880075a0 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.208+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c0747f0 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.208+0000 7fdc997fa700 1 -- 172.21.15.89:0/1843528986 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdc88040d80 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.217+0000 7fdc997fa700 1 -- 172.21.15.89:0/1843528986 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc88044060 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.354+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9b24005860 con 0x7f9b4409d6f0 2023-09-30T22:18:35.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.354+0000 7f9b2bfff700 1 -- 172.21.15.89:0/492647565 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b34015070 con 0x7f9b4409d6f0 2023-09-30T22:18:35.354 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:35.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 msgr2=0x7f9b441224e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b441224e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9b4409a970 tx=0x7f9b34004750 comp rx=0 tx=0).stop 2023-09-30T22:18:35.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 shutdown_connections 2023-09-30T22:18:35.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b4409e410 0x7f9b44122f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b4409d6f0 0x7f9b441224e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 --2- 172.21.15.89:0/492647565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b4409e000 0x7f9b44122a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 >> 172.21.15.89:0/492647565 conn(0x7f9b44066f50 msgr2=0x7f9b44133e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:35.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 shutdown_connections 2023-09-30T22:18:35.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.356+0000 7f9b49cf7700 1 -- 172.21.15.89:0/492647565 wait complete. 2023-09-30T22:18:35.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.365+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5b8806c360 con 0x7f5b8809e410 2023-09-30T22:18:35.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.365+0000 7f5b7dffb700 1 -- 172.21.15.89:0/4054950504 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b80017070 con 0x7f5b8809e410 2023-09-30T22:18:35.365 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:35.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b8809e410 msgr2=0x7f5b881ac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b8809e410 0x7f5b881ac4c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c7a0 tx=0x7f5b8000cab0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 shutdown_connections 2023-09-30T22:18:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b8809e410 0x7f5b881ac4c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8809d6f0 0x7f5b881aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 --2- 172.21.15.89:0/4054950504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b8809e000 0x7f5b881abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 >> 172.21.15.89:0/4054950504 conn(0x7f5b88066f50 msgr2=0x7f5b88071430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 shutdown_connections 2023-09-30T22:18:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.369+0000 7f5b8f813700 1 -- 172.21.15.89:0/4054950504 wait complete. 2023-09-30T22:18:35.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.377+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f80ec097de0 con 0x7f80ec09d6f0 2023-09-30T22:18:35.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.377+0000 7f80e97fa700 1 -- 172.21.15.89:0/1916588919 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f80e0017020 con 0x7f80ec09d6f0 2023-09-30T22:18:35.377 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80ec09d6f0 msgr2=0x7f80ec070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 --2- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80ec09d6f0 0x7f80ec070800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f80e000e9e0 tx=0x7f80e000eda0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 shutdown_connections 2023-09-30T22:18:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 --2- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80ec13c470 0x7f80ec071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 --2- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80ec09d6f0 0x7f80ec070800 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 --2- 172.21.15.89:0/1916588919 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f80ec09e000 0x7f80ec070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 >> 172.21.15.89:0/1916588919 conn(0x7f80ec09f470 msgr2=0x7f80ec0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 shutdown_connections 2023-09-30T22:18:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.380+0000 7f80f31a1700 1 -- 172.21.15.89:0/1916588919 wait complete. 2023-09-30T22:18:35.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:35.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:35.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.421+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdc9c137210 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.421+0000 7fdc997fa700 1 -- 172.21.15.89:0/1843528986 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc9c137210 con 0x7fdc9c0a0a30 2023-09-30T22:18:35.422 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:35.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a0a30 msgr2=0x7fdc9c133e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:35.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a0a30 0x7fdc9c133e70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800c500 tx=0x7fdc8800c8c0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 shutdown_connections 2023-09-30T22:18:35.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c138d80 0x7fdc9c1343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc9c09d630 0x7fdc9c133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 --2- 172.21.15.89:0/1843528986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a0a30 0x7fdc9c133e70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:35.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 >> 172.21.15.89:0/1843528986 conn(0x7fdc9c0a3050 msgr2=0x7fdc9c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:35.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 shutdown_connections 2023-09-30T22:18:35.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:35.424+0000 7fdca2da4700 1 -- 172.21.15.89:0/1843528986 wait complete. 2023-09-30T22:18:35.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:36.369 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:36.387 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:36.393 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:36.436 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.522+0000 7f3fa1029700 1 Processor -- start 2023-09-30T22:18:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.522+0000 7f3fa1029700 1 -- start start 2023-09-30T22:18:36.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3fa1029700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1316c0 0x7f3f9c131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3fa1029700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c09d9f0 0x7f3f9c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3fa1029700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f9c06acb0 0x7f3f9c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3fa1029700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f9c09e130 con 0x7f3f9c06acb0 2023-09-30T22:18:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3fa1029700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c1312b0 con 0x7f3f9c09d9f0 2023-09-30T22:18:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3fa1029700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c09f6e0 con 0x7f3f9c1316c0 2023-09-30T22:18:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3f9ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1316c0 0x7f3f9c131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3f9b59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f9c06acb0 0x7f3f9c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3f9ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1316c0 0x7f3f9c131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51556/0 (socket says 172.21.15.89:51556) 2023-09-30T22:18:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3f9ad9d700 1 -- 172.21.15.89:0/928432804 learned_addr learned my addr 172.21.15.89:0/928432804 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.523+0000 7f3f9a59c700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c09d9f0 0x7f3f9c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f9b59e700 1 -- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1316c0 msgr2=0x7f3f9c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f9b59e700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1316c0 0x7f3f9c131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f9b59e700 1 -- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c09d9f0 msgr2=0x7f3f9c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f9b59e700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c09d9f0 0x7f3f9c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f9b59e700 1 -- 172.21.15.89:0/928432804 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c09e760 con 0x7f3f9c06acb0 2023-09-30T22:18:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f9a59c700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c09d9f0 0x7f3f9c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f9ad9d700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1316c0 0x7f3f9c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f9b59e700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c06b110 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009d90 tx=0x7f3f9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c668bb14f3a49e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f99d9b700 1 -- 172.21.15.89:0/928432804 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400e070 con 0x7f3f9c06acb0 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f99d9b700 1 -- 172.21.15.89:0/928432804 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9400bb20 con 0x7f3f9c06acb0 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.524+0000 7f3f99d9b700 1 -- 172.21.15.89:0/928432804 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94012430 con 0x7f3f9c06acb0 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 -- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 msgr2=0x7f3f9c06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c06b110 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009d90 tx=0x7f3f9400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 -- 172.21.15.89:0/928432804 shutdown_connections 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c06b110 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1316c0 0x7f3f9c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 --2- 172.21.15.89:0/928432804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c09d9f0 0x7f3f9c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 -- 172.21.15.89:0/928432804 >> 172.21.15.89:0/928432804 conn(0x7f3f9c066f50 msgr2=0x7f3f9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:36.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 -- 172.21.15.89:0/928432804 shutdown_connections 2023-09-30T22:18:36.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.525+0000 7f3fa1029700 1 -- 172.21.15.89:0/928432804 wait complete. 2023-09-30T22:18:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.527+0000 7f3fa1029700 1 Processor -- start 2023-09-30T22:18:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.527+0000 7f3fa1029700 1 -- start start 2023-09-30T22:18:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3fa1029700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3fa1029700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c09d9f0 0x7f3f9c070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3fa1029700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c1316c0 0x7f3f9c070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3fa1029700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c132990 con 0x7f3f9c06acb0 2023-09-30T22:18:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3fa1029700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c072310 con 0x7f3f9c1316c0 2023-09-30T22:18:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3fa1029700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c072120 con 0x7f3f9c09d9f0 2023-09-30T22:18:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c070510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48968/0 (socket says 172.21.15.89:48968) 2023-09-30T22:18:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 -- 172.21.15.89:0/959506244 learned_addr learned my addr 172.21.15.89:0/959506244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9b59e700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c1316c0 0x7f3f9c070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9a59c700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c09d9f0 0x7f3f9c070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 -- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c09d9f0 msgr2=0x7f3f9c070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c09d9f0 0x7f3f9c070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 -- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c1316c0 msgr2=0x7f3f9c070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c1316c0 0x7f3f9c070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 -- 172.21.15.89:0/959506244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900094a0 con 0x7f3f9c06acb0 2023-09-30T22:18:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.528+0000 7f3f9ad9d700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c070510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00ec10 tx=0x7f3f8c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.529+0000 7f3f837fe700 1 -- 172.21.15.89:0/959506244 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00c850 con 0x7f3f9c06acb0 2023-09-30T22:18:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.529+0000 7f3f837fe700 1 -- 172.21.15.89:0/959506244 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c00c9b0 con 0x7f3f9c06acb0 2023-09-30T22:18:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.529+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8c009550 con 0x7f3f9c06acb0 2023-09-30T22:18:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.529+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f94009a70 con 0x7f3f9c06acb0 2023-09-30T22:18:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.530+0000 7f3f837fe700 1 -- 172.21.15.89:0/959506244 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00cd40 con 0x7f3f9c06acb0 2023-09-30T22:18:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.530+0000 7f3f837fe700 1 -- 172.21.15.89:0/959506244 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f8c013470 con 0x7f3f9c06acb0 2023-09-30T22:18:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.530+0000 7f3f837fe700 1 -- 172.21.15.89:0/959506244 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3f8c03f970 con 0x7f3f9c06acb0 2023-09-30T22:18:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.533+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c0052e0 con 0x7f3f9c06acb0 2023-09-30T22:18:36.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.542+0000 7f3f837fe700 1 -- 172.21.15.89:0/959506244 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f8c013720 con 0x7f3f9c06acb0 2023-09-30T22:18:36.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.551+0000 7f5f767b9700 1 Processor -- start 2023-09-30T22:18:36.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.551+0000 7f5f767b9700 1 -- start start 2023-09-30T22:18:36.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f767b9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f7009d6f0 0x7f5f7009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f767b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009e000 0x7f5f70130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f767b9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7009f130 con 0x7f5f7009e410 2023-09-30T22:18:36.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f767b9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7009f520 con 0x7f5f7009d6f0 2023-09-30T22:18:36.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f767b9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7009e930 con 0x7f5f7009e000 2023-09-30T22:18:36.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f6ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f7009d6f0 0x7f5f7009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f6ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f7009d6f0 0x7f5f7009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48972/0 (socket says 172.21.15.89:48972) 2023-09-30T22:18:36.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f6ffff700 1 -- 172.21.15.89:0/3722415257 learned_addr learned my addr 172.21.15.89:0/3722415257 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:36.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.552+0000 7f5f6f7fe700 1 --2- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009e000 0x7f5f70130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6ffff700 1 -- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009e000 msgr2=0x7f5f70130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:36.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6ffff700 1 --2- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009e000 0x7f5f70130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6ffff700 1 -- 172.21.15.89:0/3722415257 >> v1:172.21.15.89:6789/0 conn(0x7f5f7009e410 legacy=0x7f5f70130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6ffff700 1 -- 172.21.15.89:0/3722415257 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7009b2d0 con 0x7f5f7009d6f0 2023-09-30T22:18:36.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6f7fe700 1 --2- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009e000 0x7f5f70130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:36.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6ffff700 1 --2- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009d6f0 0x7f5f7009dac0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5f600098f0 tx=0x7f5f60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1af64211cf297fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:36.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6effd700 1 -- 172.21.15.89:0/3722415257 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60011830 con 0x7f5f7009d6f0 2023-09-30T22:18:36.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6effd700 1 -- 172.21.15.89:0/3722415257 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60011990 con 0x7f5f7009d6f0 2023-09-30T22:18:36.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.553+0000 7f5f6effd700 1 -- 172.21.15.89:0/3722415257 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60011d20 con 0x7f5f7009d6f0 2023-09-30T22:18:36.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7f5f767b9700 1 -- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009d6f0 msgr2=0x7f5f7009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7f5f767b9700 1 --2- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009d6f0 0x7f5f7009dac0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5f600098f0 tx=0x7f5f60009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:36.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7f5f767b9700 1 -- 172.21.15.89:0/3722415257 shutdown_connections 2023-09-30T22:18:36.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7f5f767b9700 1 --2- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009d6f0 0x7f5f7009dac0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7f5f767b9700 1 --2- 172.21.15.89:0/3722415257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009e000 0x7f5f70130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7f5f767b9700 1 -- 172.21.15.89:0/3722415257 >> 172.21.15.89:0/3722415257 conn(0x7f5f70066f50 msgr2=0x7f5f7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:36.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7f5f767b9700 1 -- 172.21.15.89:0/3722415257 shutdown_connections 2023-09-30T22:18:36.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7f5f767b9700 1 -- 172.21.15.89:0/3722415257 wait complete. 2023-09-30T22:18:36.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.554+0000 7fd78c461700 1 Processor -- start 2023-09-30T22:18:36.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.555+0000 7fd78c461700 1 -- start start 2023-09-30T22:18:36.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.555+0000 7f5f767b9700 1 Processor -- start 2023-09-30T22:18:36.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.555+0000 7fd78c461700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd78409e930 0x7fd78409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.555+0000 7fd78c461700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409f650 0x7fd784131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.555+0000 7fd78c461700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd784070940 con 0x7fd78409f240 2023-09-30T22:18:36.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7fd78c461700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78406fd60 con 0x7fd78409e930 2023-09-30T22:18:36.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7fd78c461700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78406fa80 con 0x7fd78409f650 2023-09-30T22:18:36.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f767b9700 1 -- start start 2023-09-30T22:18:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7fd78a1fd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd78409e930 0x7fd78409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7fd78a1fd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd78409e930 0x7fd78409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48986/0 (socket says 172.21.15.89:48986) 2023-09-30T22:18:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7fd78a1fd700 1 -- 172.21.15.89:0/1218299873 learned_addr learned my addr 172.21.15.89:0/1218299873 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7fd78a1fd700 1 -- 172.21.15.89:0/1218299873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409f650 msgr2=0x7fd784131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7fd7891fb700 1 -- 172.21.15.89:0/1218299873 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027542416 0 0) 0x7fd784070940 con 0x7fd78409f240 2023-09-30T22:18:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7fd78a1fd700 1 --2- 172.21.15.89:0/1218299873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409f650 0x7fd784131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7fd78a1fd700 1 -- 172.21.15.89:0/1218299873 >> v1:172.21.15.89:6789/0 conn(0x7fd78409f240 legacy=0x7fd7841317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7fd78a1fd700 1 -- 172.21.15.89:0/1218299873 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78409fb60 con 0x7fd78409e930 2023-09-30T22:18:36.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f767b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009d6f0 0x7f5f70122380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f767b9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f7009e000 0x7f5f701228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f767b9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009e410 0x7f5f70122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f767b9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70125900 con 0x7f5f7009e410 2023-09-30T22:18:36.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f767b9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7009c700 con 0x7f5f7009e000 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f767b9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70125610 con 0x7f5f7009d6f0 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f74d56700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009e410 0x7f5f70122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f74d56700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009e410 0x7f5f70122e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48992/0 (socket says 172.21.15.89:48992) 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.556+0000 7f5f74d56700 1 -- 172.21.15.89:0/1115633359 learned_addr learned my addr 172.21.15.89:0/1115633359 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f74d56700 1 -- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009d6f0 msgr2=0x7f5f70122380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f74d56700 1 --2- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009d6f0 0x7f5f70122380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f74d56700 1 -- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f7009e000 msgr2=0x7f5f701228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f74d56700 1 --2- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f7009e000 0x7f5f701228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f74d56700 1 -- 172.21.15.89:0/1115633359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f640094a0 con 0x7f5f7009e410 2023-09-30T22:18:36.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f74d56700 1 --2- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009e410 0x7f5f70122e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800cc50 tx=0x7f5f6800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:36.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7fd78a1fd700 1 --2- 172.21.15.89:0/1218299873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409e930 0x7fd78409ed00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd7800098f0 tx=0x7fd780009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748eb72fb90043ae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:36.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f6cff9700 1 -- 172.21.15.89:0/1115633359 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800ad80 con 0x7f5f7009e410 2023-09-30T22:18:36.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f6cff9700 1 -- 172.21.15.89:0/1115633359 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6800ce50 con 0x7f5f7009e410 2023-09-30T22:18:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f6cff9700 1 -- 172.21.15.89:0/1115633359 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68003c40 con 0x7f5f7009e410 2023-09-30T22:18:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f60009550 con 0x7f5f7009e410 2023-09-30T22:18:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7fd7891fb700 1 -- 172.21.15.89:0/1218299873 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000b8a0 con 0x7fd78409e930 2023-09-30T22:18:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.557+0000 7fd7891fb700 1 -- 172.21.15.89:0/1218299873 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78000ba00 con 0x7fd78409e930 2023-09-30T22:18:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.558+0000 7fd7891fb700 1 -- 172.21.15.89:0/1218299873 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000bd90 con 0x7fd78409e930 2023-09-30T22:18:36.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.558+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f70124030 con 0x7f5f7009e410 2023-09-30T22:18:36.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.558+0000 7fd78c461700 1 -- 172.21.15.89:0/1218299873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409e930 msgr2=0x7fd78409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.558+0000 7fd78c461700 1 --2- 172.21.15.89:0/1218299873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409e930 0x7fd78409ed00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd7800098f0 tx=0x7fd780009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:36.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.559+0000 7fd78c461700 1 -- 172.21.15.89:0/1218299873 shutdown_connections 2023-09-30T22:18:36.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.559+0000 7fd78c461700 1 --2- 172.21.15.89:0/1218299873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409f650 0x7fd784131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.559+0000 7fd78c461700 1 --2- 172.21.15.89:0/1218299873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409e930 0x7fd78409ed00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.559+0000 7fd78c461700 1 -- 172.21.15.89:0/1218299873 >> 172.21.15.89:0/1218299873 conn(0x7fd7840a4210 msgr2=0x7fd7840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:36.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.559+0000 7fd78c461700 1 -- 172.21.15.89:0/1218299873 shutdown_connections 2023-09-30T22:18:36.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.559+0000 7f5f6cff9700 1 -- 172.21.15.89:0/1115633359 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f680050d0 con 0x7f5f7009e410 2023-09-30T22:18:36.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.559+0000 7fd78c461700 1 -- 172.21.15.89:0/1218299873 wait complete. 2023-09-30T22:18:36.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.559+0000 7f5f6cff9700 1 -- 172.21.15.89:0/1115633359 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5f6803f860 con 0x7f5f7009e410 2023-09-30T22:18:36.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78c461700 1 Processor -- start 2023-09-30T22:18:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78c461700 1 -- start start 2023-09-30T22:18:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78c461700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409e930 0x7fd7841b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78c461700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd78409f240 0x7fd7841b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78c461700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409f650 0x7fd7841a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78c461700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840a3160 con 0x7fd78409f650 2023-09-30T22:18:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78c461700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd784071d20 con 0x7fd78409e930 2023-09-30T22:18:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78c461700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7841a3db0 con 0x7fd78409f240 2023-09-30T22:18:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd7899fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd78409f240 0x7fd7841b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78a9fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409f650 0x7fd7841a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd7899fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd78409f240 0x7fd7841b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51584/0 (socket says 172.21.15.89:51584) 2023-09-30T22:18:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd7899fc700 1 -- 172.21.15.89:0/1614548060 learned_addr learned my addr 172.21.15.89:0/1614548060 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78a9fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409f650 0x7fd7841a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49006/0 (socket says 172.21.15.89:49006) 2023-09-30T22:18:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.560+0000 7fd78a1fd700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409e930 0x7fd7841b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd7899fc700 1 -- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409e930 msgr2=0x7fd7841b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd7899fc700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409e930 0x7fd7841b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd7899fc700 1 -- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409f650 msgr2=0x7fd7841a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd7899fc700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409f650 0x7fd7841a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd7899fc700 1 -- 172.21.15.89:0/1614548060 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780009550 con 0x7fd78409f240 2023-09-30T22:18:36.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd78a9fe700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409f650 0x7fd7841a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:36.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd7899fc700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd78409f240 0x7fd7841b48b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd774001eb0 tx=0x7fd77400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:36.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd77affd700 1 -- 172.21.15.89:0/1614548060 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7740054e0 con 0x7fd78409f240 2023-09-30T22:18:36.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd77affd700 1 -- 172.21.15.89:0/1614548060 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd774003680 con 0x7fd78409f240 2023-09-30T22:18:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7840a0190 con 0x7fd78409f240 2023-09-30T22:18:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd77affd700 1 -- 172.21.15.89:0/1614548060 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7740039f0 con 0x7fd78409f240 2023-09-30T22:18:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.561+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7841a4820 con 0x7fd78409f240 2023-09-30T22:18:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.562+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7841a3a50 con 0x7fd78409f240 2023-09-30T22:18:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.563+0000 7fd77affd700 1 -- 172.21.15.89:0/1614548060 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd774003b50 con 0x7fd78409f240 2023-09-30T22:18:36.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.563+0000 7fd77affd700 1 -- 172.21.15.89:0/1614548060 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd77403edf0 con 0x7fd78409f240 2023-09-30T22:18:36.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.565+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f70074ea0 con 0x7f5f7009e410 2023-09-30T22:18:36.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.567+0000 7fd77affd700 1 -- 172.21.15.89:0/1614548060 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd774003e00 con 0x7fd78409f240 2023-09-30T22:18:36.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.576+0000 7f5f6cff9700 1 -- 172.21.15.89:0/1115633359 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f68005380 con 0x7f5f7009e410 2023-09-30T22:18:36.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.599+0000 7fe2827d6700 1 Processor -- start 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe2827d6700 1 -- start start 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe2827d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c095690 0x7fe27c134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe2827d6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe27c138430 0x7fe27c13fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe2827d6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c09e800 con 0x7fe27c1378f0 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe2827d6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c095a60 con 0x7fe27c138430 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe2827d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c094bd0 con 0x7fe27c095690 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe27bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c095690 0x7fe27c134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe27b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe27c1378f0 0x7fe27c137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52420/0 (socket says 172.21.15.89:52420) 2023-09-30T22:18:36.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe27bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c095690 0x7fe27c134cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35814/0 (socket says 172.21.15.89:35814) 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe27b7fe700 1 -- 172.21.15.89:0/869135191 learned_addr learned my addr 172.21.15.89:0/869135191 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.600+0000 7fe280d73700 1 --2- 172.21.15.89:0/869135191 >> v2:172.21.15.89:3300/0 conn(0x7fe27c138430 0x7fe27c13fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.601+0000 7fe27bfff700 1 -- 172.21.15.89:0/869135191 >> v2:172.21.15.89:3300/0 conn(0x7fe27c138430 msgr2=0x7fe27c13fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.601+0000 7fe27bfff700 1 --2- 172.21.15.89:0/869135191 >> v2:172.21.15.89:3300/0 conn(0x7fe27c138430 0x7fe27c13fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.601+0000 7fe27bfff700 1 -- 172.21.15.89:0/869135191 >> v1:172.21.15.89:6789/0 conn(0x7fe27c1378f0 legacy=0x7fe27c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.601+0000 7fe27bfff700 1 -- 172.21.15.89:0/869135191 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c0940e0 con 0x7fe27c095690 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.601+0000 7fe27bfff700 1 --2- 172.21.15.89:0/869135191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c095690 0x7fe27c134cb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009840 tx=0x7fe26c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c00423639f517ab6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.601+0000 7fe27affd700 1 -- 172.21.15.89:0/869135191 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c011830 con 0x7fe27c095690 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.601+0000 7fe27affd700 1 -- 172.21.15.89:0/869135191 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c011990 con 0x7fe27c095690 2023-09-30T22:18:36.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.601+0000 7fe27affd700 1 -- 172.21.15.89:0/869135191 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c011d20 con 0x7fe27c095690 2023-09-30T22:18:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.602+0000 7fe2827d6700 1 -- 172.21.15.89:0/869135191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c095690 msgr2=0x7fe27c134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.602+0000 7fe2827d6700 1 --2- 172.21.15.89:0/869135191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c095690 0x7fe27c134cb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009840 tx=0x7fe26c009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.602+0000 7fe2827d6700 1 -- 172.21.15.89:0/869135191 shutdown_connections 2023-09-30T22:18:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.602+0000 7fe2827d6700 1 --2- 172.21.15.89:0/869135191 >> v2:172.21.15.89:3300/0 conn(0x7fe27c138430 0x7fe27c13fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.602+0000 7fe2827d6700 1 --2- 172.21.15.89:0/869135191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c095690 0x7fe27c134cb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.602+0000 7fe2827d6700 1 -- 172.21.15.89:0/869135191 >> 172.21.15.89:0/869135191 conn(0x7fe27c0a2360 msgr2=0x7fe27c0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.602+0000 7fe2827d6700 1 -- 172.21.15.89:0/869135191 shutdown_connections 2023-09-30T22:18:36.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.602+0000 7fe2827d6700 1 -- 172.21.15.89:0/869135191 wait complete. 2023-09-30T22:18:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe2827d6700 1 Processor -- start 2023-09-30T22:18:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe2827d6700 1 -- start start 2023-09-30T22:18:36.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe2827d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c095690 0x7fe27c1cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe2827d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c1378f0 0x7fe27c1cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe2827d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe27c138430 0x7fe27c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe2827d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0705a0 con 0x7fe27c095690 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe2827d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c09f780 con 0x7fe27c1378f0 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe2827d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0721e0 con 0x7fe27c138430 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c1378f0 0x7fe27c1cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c1378f0 0x7fe27c1cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35826/0 (socket says 172.21.15.89:35826) 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe27b7fe700 1 -- 172.21.15.89:0/3634652394 learned_addr learned my addr 172.21.15.89:0/3634652394 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.603+0000 7fe27bfff700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c095690 0x7fe27c1cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe280d73700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe27c138430 0x7fe27c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:36.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe27b7fe700 1 -- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe27c138430 msgr2=0x7fe27c06eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe27b7fe700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe27c138430 0x7fe27c06eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe27b7fe700 1 -- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c095690 msgr2=0x7fe27c1cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe27b7fe700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c095690 0x7fe27c1cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe27b7fe700 1 -- 172.21.15.89:0/3634652394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c0094a0 con 0x7fe27c1378f0 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe280d73700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe27c138430 0x7fe27c06eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe27bfff700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c095690 0x7fe27c1cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe27b7fe700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c1378f0 0x7fe27c1cd200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c500 tx=0x7fe27000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe278ff9700 1 -- 172.21.15.89:0/3634652394 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270005d70 con 0x7fe27c1378f0 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe278ff9700 1 -- 172.21.15.89:0/3634652394 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2700070d0 con 0x7fe27c1378f0 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c071eb0 con 0x7fe27c1378f0 2023-09-30T22:18:36.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe278ff9700 1 -- 172.21.15.89:0/3634652394 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270007440 con 0x7fe27c1378f0 2023-09-30T22:18:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.604+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c072450 con 0x7fe27c1378f0 2023-09-30T22:18:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.605+0000 7fe278ff9700 1 -- 172.21.15.89:0/3634652394 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe2700075a0 con 0x7fe27c1378f0 2023-09-30T22:18:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.605+0000 7fe278ff9700 1 -- 172.21.15.89:0/3634652394 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe270040d10 con 0x7fe27c1378f0 2023-09-30T22:18:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.605+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c133a60 con 0x7fe27c1378f0 2023-09-30T22:18:36.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.610+0000 7fe278ff9700 1 -- 172.21.15.89:0/3634652394 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe270017030 con 0x7fe27c1378f0 2023-09-30T22:18:36.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.766+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd7841a3a50 con 0x7fd78409f240 2023-09-30T22:18:36.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.767+0000 7fd77affd700 1 -- 172.21.15.89:0/1614548060 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7841a3a50 con 0x7fd78409f240 2023-09-30T22:18:36.766 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:36.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd78409f240 msgr2=0x7fd7841b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd78409f240 0x7fd7841b48b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd774001eb0 tx=0x7fd77400c800 comp rx=0 tx=0).stop 2023-09-30T22:18:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 shutdown_connections 2023-09-30T22:18:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd78409f650 0x7fd7841a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd78409e930 0x7fd7841b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 --2- 172.21.15.89:0/1614548060 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd78409f240 0x7fd7841b48b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 >> 172.21.15.89:0/1614548060 conn(0x7fd7840a4210 msgr2=0x7fd78406c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 shutdown_connections 2023-09-30T22:18:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.770+0000 7fd78c461700 1 -- 172.21.15.89:0/1614548060 wait complete. 2023-09-30T22:18:36.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:36.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.788+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3f7c005590 con 0x7f3f9c06acb0 2023-09-30T22:18:36.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.788+0000 7f3f837fe700 1 -- 172.21.15.89:0/959506244 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f8c01bb60 con 0x7f3f9c06acb0 2023-09-30T22:18:36.788 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 msgr2=0x7f3f9c070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c070510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00ec10 tx=0x7f3f8c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:18:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 shutdown_connections 2023-09-30T22:18:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f9c1316c0 0x7f3f9c070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f9c06acb0 0x7f3f9c070510 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 --2- 172.21.15.89:0/959506244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c09d9f0 0x7f3f9c070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 >> 172.21.15.89:0/959506244 conn(0x7f3f9c066f50 msgr2=0x7f3f9c09a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 shutdown_connections 2023-09-30T22:18:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.791+0000 7f3fa1029700 1 -- 172.21.15.89:0/959506244 wait complete. 2023-09-30T22:18:36.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.806+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe27c072040 con 0x7fe27c1378f0 2023-09-30T22:18:36.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.806+0000 7fe278ff9700 1 -- 172.21.15.89:0/3634652394 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe27c072040 con 0x7fe27c1378f0 2023-09-30T22:18:36.806 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:36.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c1378f0 msgr2=0x7fe27c1cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c1378f0 0x7fe27c1cd200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c500 tx=0x7fe27000c8c0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 shutdown_connections 2023-09-30T22:18:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe27c138430 0x7fe27c06eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c095690 0x7fe27c1cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 --2- 172.21.15.89:0/3634652394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe27c1378f0 0x7fe27c1cd200 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 >> 172.21.15.89:0/3634652394 conn(0x7fe27c0a2360 msgr2=0x7fe27c135970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 shutdown_connections 2023-09-30T22:18:36.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.808+0000 7fe2827d6700 1 -- 172.21.15.89:0/3634652394 wait complete. 2023-09-30T22:18:36.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.809+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5f70125f60 con 0x7f5f7009e410 2023-09-30T22:18:36.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.809+0000 7f5f6cff9700 1 -- 172.21.15.89:0/1115633359 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f68019300 con 0x7f5f7009e410 2023-09-30T22:18:36.811 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:36.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009e410 msgr2=0x7f5f70122e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:36.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 --2- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009e410 0x7f5f70122e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800cc50 tx=0x7f5f6800a340 comp rx=0 tx=0).stop 2023-09-30T22:18:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 shutdown_connections 2023-09-30T22:18:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 --2- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f7009e410 0x7f5f70122e00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 --2- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f7009d6f0 0x7f5f70122380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 --2- 172.21.15.89:0/1115633359 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f7009e000 0x7f5f701228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 >> 172.21.15.89:0/1115633359 conn(0x7f5f70066f50 msgr2=0x7f5f7006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 shutdown_connections 2023-09-30T22:18:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:36.811+0000 7f5f767b9700 1 -- 172.21.15.89:0/1115633359 wait complete. 2023-09-30T22:18:36.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:36.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:37.787 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:37.804 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:37.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:37.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:37.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.941+0000 7fc1e70de700 1 Processor -- start 2023-09-30T22:18:37.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.941+0000 7fc1e70de700 1 -- start start 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.942+0000 7fc1e70de700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1e009e1d0 0x7fc1e0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.942+0000 7fc1e70de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e0139170 0x7fc1e013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.942+0000 7fc1e70de700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e009b120 con 0x7fc1e009d6f0 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.942+0000 7fc1e70de700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e006b460 con 0x7fc1e009e1d0 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.942+0000 7fc1e70de700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e009a7e0 con 0x7fc1e0139170 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.942+0000 7fc1e567b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e0139170 0x7fc1e013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.942+0000 7fc1e567b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e0139170 0x7fc1e013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35840/0 (socket says 172.21.15.89:35840) 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.942+0000 7fc1e567b700 1 -- 172.21.15.89:0/3112502590 learned_addr learned my addr 172.21.15.89:0/3112502590 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:37.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1dffff700 1 --2- 172.21.15.89:0/3112502590 >> v2:172.21.15.89:3300/0 conn(0x7fc1e009e1d0 0x7fc1e0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1e567b700 1 -- 172.21.15.89:0/3112502590 >> v2:172.21.15.89:3300/0 conn(0x7fc1e009e1d0 msgr2=0x7fc1e0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1e567b700 1 --2- 172.21.15.89:0/3112502590 >> v2:172.21.15.89:3300/0 conn(0x7fc1e009e1d0 0x7fc1e0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1e567b700 1 -- 172.21.15.89:0/3112502590 >> v1:172.21.15.89:6789/0 conn(0x7fc1e009d6f0 legacy=0x7fc1e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1e567b700 1 -- 172.21.15.89:0/3112502590 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e009a500 con 0x7fc1e0139170 2023-09-30T22:18:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1e567b700 1 --2- 172.21.15.89:0/3112502590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e0139170 0x7fc1e013b550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009e80 tx=0x7fc1d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b35421c1639a8b92 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1df7fe700 1 -- 172.21.15.89:0/3112502590 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800e070 con 0x7fc1e0139170 2023-09-30T22:18:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1df7fe700 1 -- 172.21.15.89:0/3112502590 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d800b8d0 con 0x7fc1e0139170 2023-09-30T22:18:37.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.943+0000 7fc1df7fe700 1 -- 172.21.15.89:0/3112502590 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800bc80 con 0x7fc1e0139170 2023-09-30T22:18:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.944+0000 7fc1e70de700 1 -- 172.21.15.89:0/3112502590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e0139170 msgr2=0x7fc1e013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.944+0000 7fc1e70de700 1 --2- 172.21.15.89:0/3112502590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e0139170 0x7fc1e013b550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009e80 tx=0x7fc1d800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.944+0000 7fc1e70de700 1 -- 172.21.15.89:0/3112502590 shutdown_connections 2023-09-30T22:18:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.944+0000 7fc1e70de700 1 --2- 172.21.15.89:0/3112502590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e0139170 0x7fc1e013b550 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.944+0000 7fc1e70de700 1 --2- 172.21.15.89:0/3112502590 >> v2:172.21.15.89:3300/0 conn(0x7fc1e009e1d0 0x7fc1e0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.944+0000 7fc1e70de700 1 -- 172.21.15.89:0/3112502590 >> 172.21.15.89:0/3112502590 conn(0x7fc1e0066f50 msgr2=0x7fc1e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:37.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.944+0000 7fc1e70de700 1 -- 172.21.15.89:0/3112502590 shutdown_connections 2023-09-30T22:18:37.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.944+0000 7fc1e70de700 1 -- 172.21.15.89:0/3112502590 wait complete. 2023-09-30T22:18:37.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.946+0000 7fc1e70de700 1 Processor -- start 2023-09-30T22:18:37.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.946+0000 7fc1e70de700 1 -- start start 2023-09-30T22:18:37.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.946+0000 7fc1e70de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e009d6f0 0x7fc1e01c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e70de700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc1e009e1d0 0x7fc1e01c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e70de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e0139170 0x7fc1e0120780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e70de700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0136650 con 0x7fc1e0139170 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e70de700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e01be360 con 0x7fc1e009d6f0 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e70de700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0123d00 con 0x7fc1e009e1d0 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e4e7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e009d6f0 0x7fc1e01c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e4e7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e009d6f0 0x7fc1e01c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35846/0 (socket says 172.21.15.89:35846) 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e4e7a700 1 -- 172.21.15.89:0/2850578638 learned_addr learned my addr 172.21.15.89:0/2850578638 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1dffff700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc1e009e1d0 0x7fc1e01c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e567b700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e0139170 0x7fc1e0120780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1dffff700 1 -- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e009d6f0 msgr2=0x7fc1e01c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1dffff700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e009d6f0 0x7fc1e01c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1dffff700 1 -- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e0139170 msgr2=0x7fc1e0120780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1dffff700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e0139170 0x7fc1e0120780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1dffff700 1 -- 172.21.15.89:0/2850578638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8009b30 con 0x7fc1e009e1d0 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1e4e7a700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e009d6f0 0x7fc1e01c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.947+0000 7fc1dffff700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc1e009e1d0 0x7fc1e01c8cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00e900 tx=0x7fc1cc00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.948+0000 7fc1dd7fa700 1 -- 172.21.15.89:0/2850578638 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc013680 con 0x7fc1e009e1d0 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.948+0000 7fc1dd7fa700 1 -- 172.21.15.89:0/2850578638 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc0137e0 con 0x7fc1e009e1d0 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.948+0000 7fc1dd7fa700 1 -- 172.21.15.89:0/2850578638 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc013b50 con 0x7fc1e009e1d0 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.948+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e0123a20 con 0x7fc1e009e1d0 2023-09-30T22:18:37.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.948+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e0124a30 con 0x7fc1e009e1d0 2023-09-30T22:18:37.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.949+0000 7fc1dd7fa700 1 -- 172.21.15.89:0/2850578638 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc1cc013cb0 con 0x7fc1e009e1d0 2023-09-30T22:18:37.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.949+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e0074ea0 con 0x7fc1e009e1d0 2023-09-30T22:18:37.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.949+0000 7fc1dd7fa700 1 -- 172.21.15.89:0/2850578638 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc1cc03f9e0 con 0x7fc1e009e1d0 2023-09-30T22:18:37.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.954+0000 7fc1dd7fa700 1 -- 172.21.15.89:0/2850578638 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1cc01b7c0 con 0x7fc1e009e1d0 2023-09-30T22:18:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc41dba4700 1 Processor -- start 2023-09-30T22:18:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc41dba4700 1 -- start start 2023-09-30T22:18:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc41dba4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc41812cf60 0x7fc41806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc41dba4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41812f8b0 0x7fc41812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc41dba4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41812cd90 con 0x7fc418130260 2023-09-30T22:18:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc41dba4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41812d330 con 0x7fc41812cf60 2023-09-30T22:18:37.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc41dba4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4181306d0 con 0x7fc41812f8b0 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc4177fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc41812cf60 0x7fc41806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc4177fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc41812cf60 0x7fc41806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49046/0 (socket says 172.21.15.89:49046) 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc4177fe700 1 -- 172.21.15.89:0/3841277979 learned_addr learned my addr 172.21.15.89:0/3841277979 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.971+0000 7fc416ffd700 1 --2- 172.21.15.89:0/3841277979 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41812f8b0 0x7fc41812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.972+0000 7fc4177fe700 1 -- 172.21.15.89:0/3841277979 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41812f8b0 msgr2=0x7fc41812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.972+0000 7fc4177fe700 1 --2- 172.21.15.89:0/3841277979 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41812f8b0 0x7fc41812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.972+0000 7fc4177fe700 1 -- 172.21.15.89:0/3841277979 >> v1:172.21.15.89:6789/0 conn(0x7fc418130260 legacy=0x7fc418136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.972+0000 7fc4177fe700 1 -- 172.21.15.89:0/3841277979 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc418071780 con 0x7fc41812cf60 2023-09-30T22:18:37.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.972+0000 7fc4177fe700 1 --2- 172.21.15.89:0/3841277979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 0x7fc41806f5d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc4080098f0 tx=0x7fc408009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865071a4fd6a9e03 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.972+0000 7fc4167fc700 1 -- 172.21.15.89:0/3841277979 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800da00 con 0x7fc41812cf60 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.972+0000 7fc4167fc700 1 -- 172.21.15.89:0/3841277979 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800db60 con 0x7fc41812cf60 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.972+0000 7fc4167fc700 1 -- 172.21.15.89:0/3841277979 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408010470 con 0x7fc41812cf60 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.973+0000 7fc41dba4700 1 -- 172.21.15.89:0/3841277979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 msgr2=0x7fc41806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.973+0000 7fc41dba4700 1 --2- 172.21.15.89:0/3841277979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 0x7fc41806f5d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc4080098f0 tx=0x7fc408009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.973+0000 7fc41dba4700 1 -- 172.21.15.89:0/3841277979 shutdown_connections 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.973+0000 7fc41dba4700 1 --2- 172.21.15.89:0/3841277979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 0x7fc41806f5d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.973+0000 7fc41dba4700 1 --2- 172.21.15.89:0/3841277979 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41812f8b0 0x7fc41812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.973+0000 7fc41dba4700 1 -- 172.21.15.89:0/3841277979 >> 172.21.15.89:0/3841277979 conn(0x7fc4180a2aa0 msgr2=0x7fc4180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.973+0000 7fc41dba4700 1 -- 172.21.15.89:0/3841277979 shutdown_connections 2023-09-30T22:18:37.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.973+0000 7fc41dba4700 1 -- 172.21.15.89:0/3841277979 wait complete. 2023-09-30T22:18:37.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.974+0000 7fc41dba4700 1 Processor -- start 2023-09-30T22:18:37.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.974+0000 7fc41dba4700 1 -- start start 2023-09-30T22:18:37.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.974+0000 7fc41dba4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 0x7fc41809bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.974+0000 7fc41dba4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41812f8b0 0x7fc4181c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc41dba4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc418130260 0x7fc4181c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc41dba4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41809e5a0 con 0x7fc41812cf60 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc41dba4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc418129fa0 con 0x7fc41812f8b0 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc41dba4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc41809b850 con 0x7fc418130260 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc418130260 0x7fc4181c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc418130260 0x7fc4181c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51638/0 (socket says 172.21.15.89:51638) 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 -- 172.21.15.89:0/1350555155 learned_addr learned my addr 172.21.15.89:0/1350555155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc4177fe700 1 --2- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 0x7fc41809bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 -- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41812f8b0 msgr2=0x7fc4181c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 --2- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41812f8b0 0x7fc4181c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 -- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 msgr2=0x7fc41809bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 --2- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 0x7fc41809bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 -- 172.21.15.89:0/1350555155 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408009550 con 0x7fc418130260 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc417fff700 1 --2- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc418130260 0x7fc4181c45e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc41000cc50 tx=0x7fc41000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc3fffff700 1 -- 172.21.15.89:0/1350555155 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410003c00 con 0x7fc418130260 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc3fffff700 1 -- 172.21.15.89:0/1350555155 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc410003d60 con 0x7fc418130260 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41809b550 con 0x7fc418130260 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc3fffff700 1 -- 172.21.15.89:0/1350555155 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41000ab60 con 0x7fc418130260 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.975+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41809cda0 con 0x7fc418130260 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.976+0000 7fc3fffff700 1 -- 172.21.15.89:0/1350555155 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc41000acc0 con 0x7fc418130260 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.976+0000 7fc3fffff700 1 -- 172.21.15.89:0/1350555155 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc41003f520 con 0x7fc418130260 2023-09-30T22:18:37.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.977+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41809cda0 con 0x7fc418130260 2023-09-30T22:18:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.979+0000 7f38bd3a1700 1 Processor -- start 2023-09-30T22:18:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.979+0000 7f38bd3a1700 1 -- start start 2023-09-30T22:18:37.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38bd3a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f38b81348a0 0x7f38b8134c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38bd3a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b809dc40 0x7f38b813fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38bd3a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b809ebc0 con 0x7f38b809d100 2023-09-30T22:18:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38bd3a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b8095cb0 con 0x7f38b81348a0 2023-09-30T22:18:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38bd3a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b809e980 con 0x7f38b809dc40 2023-09-30T22:18:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38b7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f38b81348a0 0x7f38b8134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38b7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f38b81348a0 0x7f38b8134c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49062/0 (socket says 172.21.15.89:49062) 2023-09-30T22:18:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38bcba0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b809dc40 0x7f38b813fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38b7fff700 1 -- 172.21.15.89:0/320732525 learned_addr learned my addr 172.21.15.89:0/320732525 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38b7fff700 1 -- 172.21.15.89:0/320732525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b809dc40 msgr2=0x7f38b813fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38b7fff700 1 --2- 172.21.15.89:0/320732525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b809dc40 0x7f38b813fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38b7fff700 1 -- 172.21.15.89:0/320732525 >> v1:172.21.15.89:6789/0 conn(0x7f38b809d100 legacy=0x7f38b809d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38b7fff700 1 -- 172.21.15.89:0/320732525 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b8094ec0 con 0x7f38b81348a0 2023-09-30T22:18:37.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.980+0000 7f38bcba0700 1 --2- 172.21.15.89:0/320732525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b809dc40 0x7f38b813fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.981+0000 7f38b7fff700 1 --2- 172.21.15.89:0/320732525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b81348a0 0x7f38b8134c70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f38a8006fd0 tx=0x7f38a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798177315f6c8db0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.981+0000 7f38b6ffd700 1 -- 172.21.15.89:0/320732525 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8005e30 con 0x7f38b81348a0 2023-09-30T22:18:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.981+0000 7f38b6ffd700 1 -- 172.21.15.89:0/320732525 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a8004030 con 0x7f38b81348a0 2023-09-30T22:18:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.981+0000 7f38b6ffd700 1 -- 172.21.15.89:0/320732525 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80043e0 con 0x7f38b81348a0 2023-09-30T22:18:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.981+0000 7f28420eb700 1 Processor -- start 2023-09-30T22:18:37.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.981+0000 7f28420eb700 1 -- start start 2023-09-30T22:18:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.981+0000 7f38bd3a1700 1 -- 172.21.15.89:0/320732525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b81348a0 msgr2=0x7f38b8134c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.981+0000 7f38bd3a1700 1 --2- 172.21.15.89:0/320732525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b81348a0 0x7f38b8134c70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f38a8006fd0 tx=0x7f38a8009bd0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f38bd3a1700 1 -- 172.21.15.89:0/320732525 shutdown_connections 2023-09-30T22:18:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f38bd3a1700 1 --2- 172.21.15.89:0/320732525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b81348a0 0x7f38b8134c70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f38bd3a1700 1 --2- 172.21.15.89:0/320732525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b809dc40 0x7f38b813fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f38bd3a1700 1 -- 172.21.15.89:0/320732525 >> 172.21.15.89:0/320732525 conn(0x7f38b80a2540 msgr2=0x7f38b80a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:37.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f38bd3a1700 1 -- 172.21.15.89:0/320732525 shutdown_connections 2023-09-30T22:18:37.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f38bd3a1700 1 -- 172.21.15.89:0/320732525 wait complete. 2023-09-30T22:18:37.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f28420eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09e930 0x7f283c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f28420eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09f240 0x7f283c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f28420eb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283c070940 con 0x7f283c09f650 2023-09-30T22:18:37.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f28420eb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c06fd60 con 0x7f283c09e930 2023-09-30T22:18:37.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f28420eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f283c06fa80 con 0x7f283c09f240 2023-09-30T22:18:37.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09f240 0x7f283c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09f240 0x7f283c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51646/0 (socket says 172.21.15.89:51646) 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283affd700 1 -- 172.21.15.89:0/2057107366 learned_addr learned my addr 172.21.15.89:0/2057107366 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283b7fe700 1 --2- 172.21.15.89:0/2057107366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09e930 0x7f283c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283affd700 1 -- 172.21.15.89:0/2057107366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09e930 msgr2=0x7f283c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283affd700 1 --2- 172.21.15.89:0/2057107366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09e930 0x7f283c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283affd700 1 -- 172.21.15.89:0/2057107366 >> v1:172.21.15.89:6789/0 conn(0x7f283c09f650 legacy=0x7f283c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283affd700 1 -- 172.21.15.89:0/2057107366 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c09fb60 con 0x7f283c09f240 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.982+0000 7f283a7fc700 1 -- 172.21.15.89:0/2057107366 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788097088 0 0) 0x7f283c070940 con 0x7f283c09f650 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f283affd700 1 --2- 172.21.15.89:0/2057107366 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09f240 0x7f283c1317f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2830009920 tx=0x7f2830009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e572d9f418bdd52 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f283a7fc700 1 -- 172.21.15.89:0/2057107366 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000ba60 con 0x7f283c09f240 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f283a7fc700 1 -- 172.21.15.89:0/2057107366 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283000bbc0 con 0x7f283c09f240 2023-09-30T22:18:37.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f283a7fc700 1 -- 172.21.15.89:0/2057107366 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830009e70 con 0x7f283c09f240 2023-09-30T22:18:37.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f38bd3a1700 1 Processor -- start 2023-09-30T22:18:37.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f38bd3a1700 1 -- start start 2023-09-30T22:18:37.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f28420eb700 1 -- 172.21.15.89:0/2057107366 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09f240 msgr2=0x7f283c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f28420eb700 1 --2- 172.21.15.89:0/2057107366 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09f240 0x7f283c1317f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2830009920 tx=0x7f2830009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:37.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f28420eb700 1 -- 172.21.15.89:0/2057107366 shutdown_connections 2023-09-30T22:18:37.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f28420eb700 1 --2- 172.21.15.89:0/2057107366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09e930 0x7f283c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f28420eb700 1 --2- 172.21.15.89:0/2057107366 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09f240 0x7f283c1317f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f28420eb700 1 -- 172.21.15.89:0/2057107366 >> 172.21.15.89:0/2057107366 conn(0x7f283c0a4210 msgr2=0x7f283c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:37.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7fc3fffff700 1 -- 172.21.15.89:0/1350555155 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc41809cda0 con 0x7fc418130260 2023-09-30T22:18:37.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f38bd3a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b809d100 0x7f38b8071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f38bd3a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38b809dc40 0x7f38b8071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f38bd3a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b81348a0 0x7f38b806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f38bd3a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b80a1ad0 con 0x7f38b809d100 2023-09-30T22:18:37.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f38bd3a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8094080 con 0x7f38b809dc40 2023-09-30T22:18:37.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.983+0000 7f38bd3a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b80706e0 con 0x7f38b81348a0 2023-09-30T22:18:37.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f28420eb700 1 -- 172.21.15.89:0/2057107366 shutdown_connections 2023-09-30T22:18:37.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f28420eb700 1 -- 172.21.15.89:0/2057107366 wait complete. 2023-09-30T22:18:37.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b809d100 0x7f38b8071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b809d100 0x7f38b8071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49076/0 (socket says 172.21.15.89:49076) 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 -- 172.21.15.89:0/2963185782 learned_addr learned my addr 172.21.15.89:0/2963185782 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 -- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b81348a0 msgr2=0x7f38b806ec40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 --2- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b81348a0 0x7f38b806ec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 -- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38b809dc40 msgr2=0x7f38b8071890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 --2- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38b809dc40 0x7f38b8071890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 -- 172.21.15.89:0/2963185782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0009a70 con 0x7f38b809d100 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b7fff700 1 --2- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b809d100 0x7f38b8071350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f38a8000c00 tx=0x7f38a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.984+0000 7f38b4ff9700 1 -- 172.21.15.89:0/2963185782 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a801c620 con 0x7f38b809d100 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.985+0000 7f38b4ff9700 1 -- 172.21.15.89:0/2963185782 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a801c780 con 0x7f38b809d100 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.985+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a8009500 con 0x7f38b809d100 2023-09-30T22:18:37.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.985+0000 7f38b4ff9700 1 -- 172.21.15.89:0/2963185782 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a801cb30 con 0x7f38b809d100 2023-09-30T22:18:37.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.985+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b8071dd0 con 0x7f38b809d100 2023-09-30T22:18:37.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.985+0000 7f28420eb700 1 Processor -- start 2023-09-30T22:18:37.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f38b4ff9700 1 -- 172.21.15.89:0/2963185782 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38a801cc90 con 0x7f38b809d100 2023-09-30T22:18:37.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b804f2a0 con 0x7f38b809d100 2023-09-30T22:18:37.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f38b4ff9700 1 -- 172.21.15.89:0/2963185782 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f38a80407a0 con 0x7f38b809d100 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f28420eb700 1 -- start start 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f28420eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09e930 0x7f283c1a7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f28420eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283c09f240 0x7f283c1a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f28420eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09f650 0x7f283c06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f28420eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1a5010 con 0x7f283c09f240 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f28420eb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c0a3510 con 0x7f283c09f650 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.986+0000 7f28420eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f283c071d20 con 0x7f283c09e930 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09e930 0x7f283c1a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09e930 0x7f283c1a7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51652/0 (socket says 172.21.15.89:51652) 2023-09-30T22:18:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 -- 172.21.15.89:0/971812639 learned_addr learned my addr 172.21.15.89:0/971812639 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283bfff700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09f650 0x7f283c06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283affd700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283c09f240 0x7f283c1a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 -- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09f650 msgr2=0x7f283c06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09f650 0x7f283c06b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 -- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283c09f240 msgr2=0x7f283c1a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283c09f240 0x7f283c1a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 -- 172.21.15.89:0/971812639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2830009550 con 0x7f283c09e930 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283affd700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283c09f240 0x7f283c1a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.987+0000 7f283b7fe700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09e930 0x7f283c1a7460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e810 tx=0x7f282c00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.988+0000 7f2823fff700 1 -- 172.21.15.89:0/971812639 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c00c070 con 0x7f283c09e930 2023-09-30T22:18:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.988+0000 7f2823fff700 1 -- 172.21.15.89:0/971812639 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c013610 con 0x7f283c09e930 2023-09-30T22:18:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.988+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283c0a0190 con 0x7f283c09e930 2023-09-30T22:18:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.988+0000 7f2823fff700 1 -- 172.21.15.89:0/971812639 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c0139c0 con 0x7f283c09e930 2023-09-30T22:18:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.988+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c1a52d0 con 0x7f283c09e930 2023-09-30T22:18:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.989+0000 7f2823fff700 1 -- 172.21.15.89:0/971812639 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f282c013b20 con 0x7f283c09e930 2023-09-30T22:18:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.989+0000 7f2823fff700 1 -- 172.21.15.89:0/971812639 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f282c03f3e0 con 0x7f283c09e930 2023-09-30T22:18:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.989+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c06d870 con 0x7f283c09e930 2023-09-30T22:18:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.994+0000 7f38b4ff9700 1 -- 172.21.15.89:0/2963185782 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38a8011420 con 0x7f38b809d100 2023-09-30T22:18:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:37.997+0000 7f2823fff700 1 -- 172.21.15.89:0/971812639 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f282c0108d0 con 0x7f283c09e930 2023-09-30T22:18:38.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.172+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc1e009f420 con 0x7fc1e009e1d0 2023-09-30T22:18:38.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.172+0000 7fc1dd7fa700 1 -- 172.21.15.89:0/2850578638 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc1cc01b3b0 con 0x7fc1e009e1d0 2023-09-30T22:18:38.172 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc1e009e1d0 msgr2=0x7fc1e01c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc1e009e1d0 0x7fc1e01c8cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00e900 tx=0x7fc1cc00ec10 comp rx=0 tx=0).stop 2023-09-30T22:18:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 shutdown_connections 2023-09-30T22:18:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e0139170 0x7fc1e0120780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc1e009d6f0 0x7fc1e01c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 --2- 172.21.15.89:0/2850578638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc1e009e1d0 0x7fc1e01c8cf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 >> 172.21.15.89:0/2850578638 conn(0x7fc1e0066f50 msgr2=0x7fc1e00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:38.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 shutdown_connections 2023-09-30T22:18:38.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.175+0000 7fc1e70de700 1 -- 172.21.15.89:0/2850578638 wait complete. 2023-09-30T22:18:38.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.181+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc41809cda0 con 0x7fc418130260 2023-09-30T22:18:38.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.181+0000 7fc3fffff700 1 -- 172.21.15.89:0/1350555155 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc41809cda0 con 0x7fc418130260 2023-09-30T22:18:38.181 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.185+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc418130260 msgr2=0x7fc4181c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.185+0000 7fc41dba4700 1 --2- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc418130260 0x7fc4181c45e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc41000cc50 tx=0x7fc41000cf60 comp rx=0 tx=0).stop 2023-09-30T22:18:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.185+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 shutdown_connections 2023-09-30T22:18:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.185+0000 7fc41dba4700 1 --2- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc418130260 0x7fc4181c45e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.185+0000 7fc41dba4700 1 --2- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc41812cf60 0x7fc41809bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.185+0000 7fc41dba4700 1 --2- 172.21.15.89:0/1350555155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41812f8b0 0x7fc4181c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.185+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 >> 172.21.15.89:0/1350555155 conn(0x7fc4180a2aa0 msgr2=0x7fc41806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.186+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 shutdown_connections 2023-09-30T22:18:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.187+0000 7fc41dba4700 1 -- 172.21.15.89:0/1350555155 wait complete. 2023-09-30T22:18:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.189+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38b8074740 con 0x7f38b809d100 2023-09-30T22:18:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.190+0000 7f38b4ff9700 1 -- 172.21.15.89:0/2963185782 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38a802a020 con 0x7f38b809d100 2023-09-30T22:18:38.190 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:38.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:38.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b809d100 msgr2=0x7f38b8071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:38.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 --2- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b809d100 0x7f38b8071350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f38a8000c00 tx=0x7f38a8004750 comp rx=0 tx=0).stop 2023-09-30T22:18:38.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 shutdown_connections 2023-09-30T22:18:38.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 --2- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38b81348a0 0x7f38b806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 --2- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38b809d100 0x7f38b8071350 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 --2- 172.21.15.89:0/2963185782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38b809dc40 0x7f38b8071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 >> 172.21.15.89:0/2963185782 conn(0x7f38b80a2540 msgr2=0x7f38b8098190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:38.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 shutdown_connections 2023-09-30T22:18:38.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.192+0000 7f38bd3a1700 1 -- 172.21.15.89:0/2963185782 wait complete. 2023-09-30T22:18:38.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:38.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.206+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f283c074ea0 con 0x7f283c09e930 2023-09-30T22:18:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.207+0000 7f2823fff700 1 -- 172.21.15.89:0/971812639 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f282c02b090 con 0x7f283c09e930 2023-09-30T22:18:38.206 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09e930 msgr2=0x7f283c1a7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09e930 0x7f283c1a7460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e810 tx=0x7f282c00eb20 comp rx=0 tx=0).stop 2023-09-30T22:18:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 shutdown_connections 2023-09-30T22:18:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283c09f650 0x7f283c06b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283c09e930 0x7f283c1a7460 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 --2- 172.21.15.89:0/971812639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283c09f240 0x7f283c1a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 >> 172.21.15.89:0/971812639 conn(0x7f283c0a4210 msgr2=0x7f283c1b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 shutdown_connections 2023-09-30T22:18:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:38.209+0000 7f28420eb700 1 -- 172.21.15.89:0/971812639 wait complete. 2023-09-30T22:18:38.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:39.193 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:39.201 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:39.206 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:39.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:39.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.347+0000 7ffb72d9d700 1 Processor -- start 2023-09-30T22:18:39.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.347+0000 7ffb72d9d700 1 -- start start 2023-09-30T22:18:39.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb72d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb6c09d6f0 0x7ffb6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb72d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb72d9d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c09b120 con 0x7ffb6c09e000 2023-09-30T22:18:39.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb72d9d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c06b460 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb72d9d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c09a7e0 con 0x7ffb6c09e440 2023-09-30T22:18:39.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb70b39700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb6c09d6f0 0x7ffb6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb70b39700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb6c09d6f0 0x7ffb6c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49092/0 (socket says 172.21.15.89:49092) 2023-09-30T22:18:39.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb7133a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb70b39700 1 -- 172.21.15.89:0/3488849166 learned_addr learned my addr 172.21.15.89:0/3488849166 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:39.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb7133a700 1 -- 172.21.15.89:0/3488849166 >> v2:172.21.15.89:3300/0 conn(0x7ffb6c09d6f0 msgr2=0x7ffb6c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:39.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb7133a700 1 --2- 172.21.15.89:0/3488849166 >> v2:172.21.15.89:3300/0 conn(0x7ffb6c09d6f0 0x7ffb6c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb7133a700 1 -- 172.21.15.89:0/3488849166 >> v1:172.21.15.89:6789/0 conn(0x7ffb6c09e000 legacy=0x7ffb6c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb7133a700 1 -- 172.21.15.89:0/3488849166 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c09a500 con 0x7ffb6c09e440 2023-09-30T22:18:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.348+0000 7ffb70b39700 1 --2- 172.21.15.89:0/3488849166 >> v2:172.21.15.89:3300/0 conn(0x7ffb6c09d6f0 0x7ffb6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb7133a700 1 --2- 172.21.15.89:0/3488849166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c134100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009dc0 tx=0x7ffb6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb7985a540bce9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb6b7fe700 1 -- 172.21.15.89:0/3488849166 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400e070 con 0x7ffb6c09e440 2023-09-30T22:18:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb6b7fe700 1 -- 172.21.15.89:0/3488849166 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6400bae0 con 0x7ffb6c09e440 2023-09-30T22:18:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb6b7fe700 1 -- 172.21.15.89:0/3488849166 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400be90 con 0x7ffb6c09e440 2023-09-30T22:18:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb72d9d700 1 -- 172.21.15.89:0/3488849166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 msgr2=0x7ffb6c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb72d9d700 1 --2- 172.21.15.89:0/3488849166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c134100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009dc0 tx=0x7ffb6400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb72d9d700 1 -- 172.21.15.89:0/3488849166 shutdown_connections 2023-09-30T22:18:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb72d9d700 1 --2- 172.21.15.89:0/3488849166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c134100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.349+0000 7ffb72d9d700 1 --2- 172.21.15.89:0/3488849166 >> v2:172.21.15.89:3300/0 conn(0x7ffb6c09d6f0 0x7ffb6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.350+0000 7ffb72d9d700 1 -- 172.21.15.89:0/3488849166 >> 172.21.15.89:0/3488849166 conn(0x7ffb6c066f50 msgr2=0x7ffb6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.350+0000 7ffb72d9d700 1 -- 172.21.15.89:0/3488849166 shutdown_connections 2023-09-30T22:18:39.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.350+0000 7ffb72d9d700 1 -- 172.21.15.89:0/3488849166 wait complete. 2023-09-30T22:18:39.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.351+0000 7ffb72d9d700 1 Processor -- start 2023-09-30T22:18:39.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.351+0000 7ffb72d9d700 1 -- start start 2023-09-30T22:18:39.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb72d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb6c09d6f0 0x7ffb6c1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb72d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb6c09e000 0x7ffb6c1a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb72d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c1a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb72d9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c1a81f0 con 0x7ffb6c09e000 2023-09-30T22:18:39.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb72d9d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c1a6b10 con 0x7ffb6c09e440 2023-09-30T22:18:39.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb72d9d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c1a86a0 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb7133a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c1a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb70b39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb6c09d6f0 0x7ffb6c1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb7133a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c1a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35886/0 (socket says 172.21.15.89:35886) 2023-09-30T22:18:39.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb70b39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb6c09d6f0 0x7ffb6c1a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51666/0 (socket says 172.21.15.89:51666) 2023-09-30T22:18:39.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb7133a700 1 -- 172.21.15.89:0/1356503219 learned_addr learned my addr 172.21.15.89:0/1356503219 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:39.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb6bfff700 1 --2- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb6c09e000 0x7ffb6c1a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb70b39700 1 -- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 msgr2=0x7ffb6c1a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb70b39700 1 --2- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c1a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb70b39700 1 -- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb6c09e000 msgr2=0x7ffb6c1a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb70b39700 1 --2- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb6c09e000 0x7ffb6c1a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.352+0000 7ffb70b39700 1 -- 172.21.15.89:0/1356503219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60009550 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.353+0000 7ffb70b39700 1 --2- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb6c09d6f0 0x7ffb6c1a4e60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000e9a0 tx=0x7ffb6000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.353+0000 7ffb697fa700 1 -- 172.21.15.89:0/1356503219 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60004030 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.353+0000 7ffb697fa700 1 -- 172.21.15.89:0/1356503219 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60004190 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.353+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb64009a70 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.353+0000 7ffb697fa700 1 -- 172.21.15.89:0/1356503219 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60004520 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.353+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c1a7cc0 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.354+0000 7ffb697fa700 1 -- 172.21.15.89:0/1356503219 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb600036a0 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.354+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c074ea0 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.354+0000 7ffb697fa700 1 -- 172.21.15.89:0/1356503219 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffb6003f9d0 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.359+0000 7ffb697fa700 1 -- 172.21.15.89:0/1356503219 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb60011db0 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.377+0000 7f2146bde700 1 Processor -- start 2023-09-30T22:18:39.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.378+0000 7f2146bde700 1 -- start start 2023-09-30T22:18:39.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.378+0000 7f2146bde700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f21401316c0 0x7f2140131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.378+0000 7f2146bde700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f214009d9f0 0x7f214006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.378+0000 7f2146bde700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f214006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.378+0000 7f2146bde700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214009e130 con 0x7f21401316c0 2023-09-30T22:18:39.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.378+0000 7f2146bde700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21401312b0 con 0x7f214009d9f0 2023-09-30T22:18:39.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.378+0000 7f2146bde700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f214009f6e0 con 0x7f214006acb0 2023-09-30T22:18:39.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.378+0000 7f214517b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f214006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f213ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f214009d9f0 0x7f214006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214517b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f214006b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51668/0 (socket says 172.21.15.89:51668) 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214517b700 1 -- 172.21.15.89:0/823984293 learned_addr learned my addr 172.21.15.89:0/823984293 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214497a700 1 --2- 172.21.15.89:0/823984293 >> v2:172.21.15.89:3300/0 conn(0x7f21401316c0 0x7f2140131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214517b700 1 -- 172.21.15.89:0/823984293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f214009d9f0 msgr2=0x7f214006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214517b700 1 --2- 172.21.15.89:0/823984293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f214009d9f0 0x7f214006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214517b700 1 -- 172.21.15.89:0/823984293 >> v2:172.21.15.89:3300/0 conn(0x7f21401316c0 msgr2=0x7f2140131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214517b700 1 --2- 172.21.15.89:0/823984293 >> v2:172.21.15.89:3300/0 conn(0x7f21401316c0 0x7f2140131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214517b700 1 -- 172.21.15.89:0/823984293 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214009e760 con 0x7f214006acb0 2023-09-30T22:18:39.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f214517b700 1 --2- 172.21.15.89:0/823984293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f214006b150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21380086c0 tx=0x7f21380089d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbe7ae711f3ff35e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:39.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f213f7fe700 1 -- 172.21.15.89:0/823984293 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213800b540 con 0x7f214006acb0 2023-09-30T22:18:39.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f213f7fe700 1 -- 172.21.15.89:0/823984293 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213800d0d0 con 0x7f214006acb0 2023-09-30T22:18:39.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.379+0000 7f213f7fe700 1 -- 172.21.15.89:0/823984293 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213800d480 con 0x7f214006acb0 2023-09-30T22:18:39.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.380+0000 7f2146bde700 1 -- 172.21.15.89:0/823984293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 msgr2=0x7f214006b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.380+0000 7f2146bde700 1 --2- 172.21.15.89:0/823984293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f214006b150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21380086c0 tx=0x7f21380089d0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.380+0000 7f2146bde700 1 -- 172.21.15.89:0/823984293 shutdown_connections 2023-09-30T22:18:39.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.380+0000 7f2146bde700 1 --2- 172.21.15.89:0/823984293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f214006b150 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.380+0000 7f2146bde700 1 --2- 172.21.15.89:0/823984293 >> v2:172.21.15.89:3300/0 conn(0x7f21401316c0 0x7f2140131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.380+0000 7f2146bde700 1 --2- 172.21.15.89:0/823984293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f214009d9f0 0x7f214006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.380+0000 7f2146bde700 1 -- 172.21.15.89:0/823984293 >> 172.21.15.89:0/823984293 conn(0x7f2140066f50 msgr2=0x7f214009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:39.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.385+0000 7f2146bde700 1 -- 172.21.15.89:0/823984293 shutdown_connections 2023-09-30T22:18:39.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.385+0000 7f2146bde700 1 -- 172.21.15.89:0/823984293 wait complete. 2023-09-30T22:18:39.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.385+0000 7f2146bde700 1 Processor -- start 2023-09-30T22:18:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f2146bde700 1 -- start start 2023-09-30T22:18:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f2146bde700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f2140119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f2146bde700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f214009d9f0 0x7f21401199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f2146bde700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21401316c0 0x7f2140119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f2146bde700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214012b900 con 0x7f214009d9f0 2023-09-30T22:18:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f2146bde700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214011c560 con 0x7f21401316c0 2023-09-30T22:18:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f2146bde700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f214011ab70 con 0x7f214006acb0 2023-09-30T22:18:39.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f214517b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21401316c0 0x7f2140119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f214517b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21401316c0 0x7f2140119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35910/0 (socket says 172.21.15.89:35910) 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f214517b700 1 -- 172.21.15.89:0/268343193 learned_addr learned my addr 172.21.15.89:0/268343193 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f213ffff700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f214009d9f0 0x7f21401199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.386+0000 7f214497a700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f2140119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f214517b700 1 -- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 msgr2=0x7f2140119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f214517b700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f2140119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f214517b700 1 -- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f214009d9f0 msgr2=0x7f21401199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f214517b700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f214009d9f0 0x7f21401199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f214517b700 1 -- 172.21.15.89:0/268343193 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2138008320 con 0x7f21401316c0 2023-09-30T22:18:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f214517b700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21401316c0 0x7f2140119f00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f214009f820 tx=0x7f2138004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f213d7fa700 1 -- 172.21.15.89:0/268343193 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213801c070 con 0x7f21401316c0 2023-09-30T22:18:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f213d7fa700 1 -- 172.21.15.89:0/268343193 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213800ba10 con 0x7f21401316c0 2023-09-30T22:18:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f213d7fa700 1 -- 172.21.15.89:0/268343193 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213800bdc0 con 0x7f21401316c0 2023-09-30T22:18:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21400a3540 con 0x7f21401316c0 2023-09-30T22:18:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.387+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214011b910 con 0x7f21401316c0 2023-09-30T22:18:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7fd1eaf64700 1 Processor -- start 2023-09-30T22:18:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7fefdddc9700 1 Processor -- start 2023-09-30T22:18:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2140074ea0 con 0x7f21401316c0 2023-09-30T22:18:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7f213d7fa700 1 -- 172.21.15.89:0/268343193 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2138015400 con 0x7f21401316c0 2023-09-30T22:18:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7fd1eaf64700 1 -- start start 2023-09-30T22:18:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7fd1eaf64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e406a770 0x7fd1e409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7fd1eaf64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7fd1eaf64700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd1e413c1e0 0x7fd1e413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fd1eaf64700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e406ab40 con 0x7fd1e413c1e0 2023-09-30T22:18:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fd1eaf64700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e409ebc0 con 0x7fd1e406a770 2023-09-30T22:18:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fd1eaf64700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e409a810 con 0x7fd1e409dc30 2023-09-30T22:18:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.388+0000 7fefdddc9700 1 -- start start 2023-09-30T22:18:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefdddc9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefd809e1d0 0x7fefd813cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefdddc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd813d480 0x7fefd813f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefdddc9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd80992c0 con 0x7fefd809d6f0 2023-09-30T22:18:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefdddc9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd8099520 con 0x7fefd809e1d0 2023-09-30T22:18:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefdddc9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80997e0 con 0x7fefd813d480 2023-09-30T22:18:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e409e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51684/0 (socket says 172.21.15.89:51684) 2023-09-30T22:18:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fd1e3fff700 1 -- 172.21.15.89:0/622956177 learned_addr learned my addr 172.21.15.89:0/622956177 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefd6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefd809e1d0 0x7fefd813cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7f213d7fa700 1 -- 172.21.15.89:0/268343193 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f21380405a0 con 0x7f21401316c0 2023-09-30T22:18:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefd6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefd809e1d0 0x7fefd813cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49120/0 (socket says 172.21.15.89:49120) 2023-09-30T22:18:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefd6ffd700 1 -- 172.21.15.89:0/530913893 learned_addr learned my addr 172.21.15.89:0/530913893 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefd7fff700 1 --2- 172.21.15.89:0/530913893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd813d480 0x7fefd813f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fd1e9501700 1 --2- 172.21.15.89:0/622956177 >> v2:172.21.15.89:3300/0 conn(0x7fd1e413c1e0 0x7fd1e413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefd6ffd700 1 -- 172.21.15.89:0/530913893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd813d480 msgr2=0x7fefd813f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefd6ffd700 1 --2- 172.21.15.89:0/530913893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd813d480 0x7fefd813f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefd6ffd700 1 -- 172.21.15.89:0/530913893 >> v1:172.21.15.89:6789/0 conn(0x7fefd809d6f0 legacy=0x7fefd809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.389+0000 7fefd6ffd700 1 -- 172.21.15.89:0/530913893 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8099b30 con 0x7fefd809e1d0 2023-09-30T22:18:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.390+0000 7fefd6ffd700 1 --2- 172.21.15.89:0/530913893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefd809e1d0 0x7fefd813cf40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0098f0 tx=0x7fefcc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8174f38ac47095de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.390+0000 7fd1e9501700 1 -- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 msgr2=0x7fd1e409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.390+0000 7fd1e9501700 1 --2- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.390+0000 7fd1e9501700 1 -- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e406a770 msgr2=0x7fd1e409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.390+0000 7fd1e9501700 1 --2- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e406a770 0x7fd1e409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.390+0000 7fd1e9501700 1 -- 172.21.15.89:0/622956177 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e409e920 con 0x7fd1e413c1e0 2023-09-30T22:18:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.390+0000 7fd1e9501700 1 --2- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e413c1e0 0x7fd1e413e5c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009840 tx=0x7fd1cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa338a9b92ec2b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.390+0000 7fefd67fc700 1 -- 172.21.15.89:0/530913893 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc00b8a0 con 0x7fefd809e1d0 2023-09-30T22:18:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fefd67fc700 1 -- 172.21.15.89:0/530913893 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc00ba00 con 0x7fefd809e1d0 2023-09-30T22:18:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fefd67fc700 1 -- 172.21.15.89:0/530913893 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc00bd90 con 0x7fefd809e1d0 2023-09-30T22:18:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fd1e37fe700 1 -- 172.21.15.89:0/622956177 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc011830 con 0x7fd1e413c1e0 2023-09-30T22:18:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fd1e37fe700 1 -- 172.21.15.89:0/622956177 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc011990 con 0x7fd1e413c1e0 2023-09-30T22:18:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fd1e37fe700 1 -- 172.21.15.89:0/622956177 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc011d20 con 0x7fd1e413c1e0 2023-09-30T22:18:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fd1eaf64700 1 -- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e413c1e0 msgr2=0x7fd1e413e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fd1eaf64700 1 --2- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e413c1e0 0x7fd1e413e5c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009840 tx=0x7fd1cc009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fefdddc9700 1 -- 172.21.15.89:0/530913893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefd809e1d0 msgr2=0x7fefd813cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.391+0000 7fefdddc9700 1 --2- 172.21.15.89:0/530913893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefd809e1d0 0x7fefd813cf40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0098f0 tx=0x7fefcc009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:39.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.392+0000 7fd1eaf64700 1 -- 172.21.15.89:0/622956177 shutdown_connections 2023-09-30T22:18:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.392+0000 7fd1eaf64700 1 --2- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e413c1e0 0x7fd1e413e5c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.392+0000 7fd1eaf64700 1 --2- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e406a770 0x7fd1e409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.392+0000 7fd1eaf64700 1 --2- 172.21.15.89:0/622956177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.392+0000 7fd1eaf64700 1 -- 172.21.15.89:0/622956177 >> 172.21.15.89:0/622956177 conn(0x7fd1e4066f50 msgr2=0x7fd1e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.392+0000 7fd1eaf64700 1 -- 172.21.15.89:0/622956177 shutdown_connections 2023-09-30T22:18:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.392+0000 7fd1eaf64700 1 -- 172.21.15.89:0/622956177 wait complete. 2023-09-30T22:18:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.393+0000 7fd1eaf64700 1 Processor -- start 2023-09-30T22:18:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.393+0000 7fd1eaf64700 1 -- start start 2023-09-30T22:18:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1eaf64700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e406a770 0x7fd1e413bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1eaf64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e406f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1eaf64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e413c1e0 0x7fd1e406fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1eaf64700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e411d0b0 con 0x7fd1e406a770 2023-09-30T22:18:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1eaf64700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4071dc0 con 0x7fd1e413c1e0 2023-09-30T22:18:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1eaf64700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e412da30 con 0x7fd1e409dc30 2023-09-30T22:18:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e406f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e406f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51694/0 (socket says 172.21.15.89:51694) 2023-09-30T22:18:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e8d00700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e406a770 0x7fd1e413bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e3fff700 1 -- 172.21.15.89:0/3411700194 learned_addr learned my addr 172.21.15.89:0/3411700194 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e9501700 1 --2- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e413c1e0 0x7fd1e406fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e3fff700 1 -- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e413c1e0 msgr2=0x7fd1e406fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e3fff700 1 --2- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e413c1e0 0x7fd1e406fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e3fff700 1 -- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e406a770 msgr2=0x7fd1e413bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e3fff700 1 --2- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e406a770 0x7fd1e413bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fefdddc9700 1 -- 172.21.15.89:0/530913893 shutdown_connections 2023-09-30T22:18:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fefdddc9700 1 --2- 172.21.15.89:0/530913893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd813d480 0x7fefd813f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fefdddc9700 1 --2- 172.21.15.89:0/530913893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefd809e1d0 0x7fefd813cf40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fefdddc9700 1 -- 172.21.15.89:0/530913893 >> 172.21.15.89:0/530913893 conn(0x7fefd809f470 msgr2=0x7fefd80a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fefdddc9700 1 -- 172.21.15.89:0/530913893 shutdown_connections 2023-09-30T22:18:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fefdddc9700 1 -- 172.21.15.89:0/530913893 wait complete. 2023-09-30T22:18:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.395+0000 7f213d7fa700 1 -- 172.21.15.89:0/268343193 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f213800a400 con 0x7f21401316c0 2023-09-30T22:18:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.394+0000 7fd1e3fff700 1 -- 172.21.15.89:0/3411700194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0094a0 con 0x7fd1e409dc30 2023-09-30T22:18:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.395+0000 7fd1e3fff700 1 --2- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e406f890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009d70 tx=0x7fd1d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.395+0000 7fd1e17fa700 1 -- 172.21.15.89:0/3411700194 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400cbb0 con 0x7fd1e409dc30 2023-09-30T22:18:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.395+0000 7fd1e17fa700 1 -- 172.21.15.89:0/3411700194 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400f040 con 0x7fd1e409dc30 2023-09-30T22:18:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.395+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e409f1a0 con 0x7fd1e409dc30 2023-09-30T22:18:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.395+0000 7fd1e17fa700 1 -- 172.21.15.89:0/3411700194 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40092e0 con 0x7fd1e409dc30 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.395+0000 7fefdddc9700 1 Processor -- start 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefdddc9700 1 -- start start 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefdddc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd809d6f0 0x7fefd806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefdddc9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd809e1d0 0x7fefd806f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefdddc9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefd813d480 0x7fefd806fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefdddc9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8070780 con 0x7fefd813d480 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefdddc9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8071dd0 con 0x7fefd809e1d0 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefdddc9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8098c70 con 0x7fefd809d6f0 2023-09-30T22:18:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd809d6f0 0x7fefd806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd809d6f0 0x7fefd806f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51706/0 (socket says 172.21.15.89:51706) 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefd77fe700 1 -- 172.21.15.89:0/762818856 learned_addr learned my addr 172.21.15.89:0/762818856 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.396+0000 7fefd6ffd700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd809e1d0 0x7fefd806f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.397+0000 7fefd77fe700 1 -- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd809e1d0 msgr2=0x7fefd806f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.397+0000 7fefd77fe700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd809e1d0 0x7fefd806f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.397+0000 7fefd77fe700 1 -- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefd813d480 msgr2=0x7fefd806fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.397+0000 7fefd77fe700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefd813d480 0x7fefd806fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.397+0000 7fefd77fe700 1 -- 172.21.15.89:0/762818856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc009550 con 0x7fefd809d6f0 2023-09-30T22:18:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.397+0000 7fefd6ffd700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd809e1d0 0x7fefd806f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.395+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e4072280 con 0x7fd1e409dc30 2023-09-30T22:18:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.397+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e4071f00 con 0x7fd1e409dc30 2023-09-30T22:18:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.397+0000 7fefd77fe700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd809d6f0 0x7fefd806f2b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c510 tx=0x7fefc800c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.398+0000 7fd1e17fa700 1 -- 172.21.15.89:0/3411700194 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1d4014480 con 0x7fd1e409dc30 2023-09-30T22:18:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.398+0000 7fd1e17fa700 1 -- 172.21.15.89:0/3411700194 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd1d40405d0 con 0x7fd1e409dc30 2023-09-30T22:18:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.401+0000 7fefbffff700 1 -- 172.21.15.89:0/762818856 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80052d0 con 0x7fefd809d6f0 2023-09-30T22:18:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.401+0000 7fd1e17fa700 1 -- 172.21.15.89:0/3411700194 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1d4014730 con 0x7fd1e409dc30 2023-09-30T22:18:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.402+0000 7fefbffff700 1 -- 172.21.15.89:0/762818856 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8005430 con 0x7fefd809d6f0 2023-09-30T22:18:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.402+0000 7fefbffff700 1 -- 172.21.15.89:0/762818856 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80038c0 con 0x7fefd809d6f0 2023-09-30T22:18:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.402+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd808f820 con 0x7fefd809d6f0 2023-09-30T22:18:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.402+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd80716b0 con 0x7fefd809d6f0 2023-09-30T22:18:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.403+0000 7fefbffff700 1 -- 172.21.15.89:0/762818856 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefc8003a20 con 0x7fefd809d6f0 2023-09-30T22:18:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.403+0000 7fefbffff700 1 -- 172.21.15.89:0/762818856 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fefc803ec60 con 0x7fefd809d6f0 2023-09-30T22:18:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.403+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd80747f0 con 0x7fefd809d6f0 2023-09-30T22:18:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.408+0000 7fefbffff700 1 -- 172.21.15.89:0/762818856 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefc8017020 con 0x7fefd809d6f0 2023-09-30T22:18:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.586+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffb6c09f420 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.586+0000 7ffb697fa700 1 -- 172.21.15.89:0/1356503219 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb60011350 con 0x7ffb6c09d6f0 2023-09-30T22:18:39.586 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb6c09d6f0 msgr2=0x7ffb6c1a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 --2- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb6c09d6f0 0x7ffb6c1a4e60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000e9a0 tx=0x7ffb6000ecb0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 shutdown_connections 2023-09-30T22:18:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 --2- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb6c09e440 0x7ffb6c1a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 --2- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb6c09d6f0 0x7ffb6c1a4e60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 --2- 172.21.15.89:0/1356503219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb6c09e000 0x7ffb6c1a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 >> 172.21.15.89:0/1356503219 conn(0x7ffb6c066f50 msgr2=0x7ffb6c1b9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 shutdown_connections 2023-09-30T22:18:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.588+0000 7ffb72d9d700 1 -- 172.21.15.89:0/1356503219 wait complete. 2023-09-30T22:18:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.591+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f214011b560 con 0x7f21401316c0 2023-09-30T22:18:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.592+0000 7f213d7fa700 1 -- 172.21.15.89:0/268343193 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2138025030 con 0x7f21401316c0 2023-09-30T22:18:39.592 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21401316c0 msgr2=0x7f2140119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21401316c0 0x7f2140119f00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f214009f820 tx=0x7f2138004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 shutdown_connections 2023-09-30T22:18:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21401316c0 0x7f2140119f00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f214006acb0 0x7f2140119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 --2- 172.21.15.89:0/268343193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f214009d9f0 0x7f21401199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 >> 172.21.15.89:0/268343193 conn(0x7f2140066f50 msgr2=0x7f214009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 shutdown_connections 2023-09-30T22:18:39.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.595+0000 7f2146bde700 1 -- 172.21.15.89:0/268343193 wait complete. 2023-09-30T22:18:39.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.599+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd1e4074ea0 con 0x7fd1e409dc30 2023-09-30T22:18:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.599+0000 7fd1e17fa700 1 -- 172.21.15.89:0/3411700194 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1d40133e0 con 0x7fd1e409dc30 2023-09-30T22:18:39.599 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:39.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 msgr2=0x7fd1e406f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 --2- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e406f890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009d70 tx=0x7fd1d400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 shutdown_connections 2023-09-30T22:18:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 --2- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1e413c1e0 0x7fd1e406fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 --2- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1e406a770 0x7fd1e413bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 --2- 172.21.15.89:0/3411700194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1e409dc30 0x7fd1e406f890 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 >> 172.21.15.89:0/3411700194 conn(0x7fd1e4066f50 msgr2=0x7fd1e409b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 shutdown_connections 2023-09-30T22:18:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.603+0000 7fd1eaf64700 1 -- 172.21.15.89:0/3411700194 wait complete. 2023-09-30T22:18:39.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:39.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.625+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fefd808fbc0 con 0x7fefd809d6f0 2023-09-30T22:18:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.625+0000 7fefbffff700 1 -- 172.21.15.89:0/762818856 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefc8019e60 con 0x7fefd809d6f0 2023-09-30T22:18:39.625 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd809d6f0 msgr2=0x7fefd806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd809d6f0 0x7fefd806f2b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c510 tx=0x7fefc800c820 comp rx=0 tx=0).stop 2023-09-30T22:18:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 shutdown_connections 2023-09-30T22:18:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefd813d480 0x7fefd806fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd809d6f0 0x7fefd806f2b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 --2- 172.21.15.89:0/762818856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd809e1d0 0x7fefd806f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 >> 172.21.15.89:0/762818856 conn(0x7fefd809f470 msgr2=0x7fefd809a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 shutdown_connections 2023-09-30T22:18:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:39.627+0000 7fefdddc9700 1 -- 172.21.15.89:0/762818856 wait complete. 2023-09-30T22:18:39.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:40.603 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:40.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:40.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:40.640 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:40.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.758+0000 7f730fb96700 1 Processor -- start 2023-09-30T22:18:40.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.759+0000 7f730fb96700 1 -- start start 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.759+0000 7f730fb96700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 0x7f7308138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.759+0000 7f730fb96700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7308139170 0x7f730813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.759+0000 7f730fb96700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730809b120 con 0x7f730809d6f0 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.759+0000 7f730fb96700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730806b460 con 0x7f7308139170 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.759+0000 7f730fb96700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f730809a7e0 con 0x7f730809e1d0 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730d932700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f730809d6f0 0x7f730809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52948/0 (socket says 172.21.15.89:52948) 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730d932700 1 -- 172.21.15.89:0/2557918833 learned_addr learned my addr 172.21.15.89:0/2557918833 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730d131700 1 --2- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 0x7f7308138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730e133700 1 --2- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7308139170 0x7f730813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730c930700 1 -- 172.21.15.89:0/2557918833 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559325321 0 0) 0x7f730809b120 con 0x7f730809d6f0 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730e133700 1 -- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 msgr2=0x7f7308138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730e133700 1 --2- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 0x7f7308138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730e133700 1 -- 172.21.15.89:0/2557918833 >> v1:172.21.15.89:6789/0 conn(0x7f730809d6f0 legacy=0x7f730809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730e133700 1 -- 172.21.15.89:0/2557918833 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730809a500 con 0x7f7308139170 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730e133700 1 --2- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7308139170 0x7f730813b550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7300009dc0 tx=0x7f730000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=830241bc064c8645 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730c930700 1 -- 172.21.15.89:0/2557918833 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730000e070 con 0x7f7308139170 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730c930700 1 -- 172.21.15.89:0/2557918833 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730000bb20 con 0x7f7308139170 2023-09-30T22:18:40.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.760+0000 7f730c930700 1 -- 172.21.15.89:0/2557918833 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300012430 con 0x7f7308139170 2023-09-30T22:18:40.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.761+0000 7f730fb96700 1 -- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7308139170 msgr2=0x7f730813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.761+0000 7f730fb96700 1 --2- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7308139170 0x7f730813b550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7300009dc0 tx=0x7f730000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:40.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.761+0000 7f730fb96700 1 -- 172.21.15.89:0/2557918833 shutdown_connections 2023-09-30T22:18:40.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.761+0000 7f730fb96700 1 --2- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7308139170 0x7f730813b550 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.761+0000 7f730fb96700 1 --2- 172.21.15.89:0/2557918833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 0x7f7308138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.761+0000 7f730fb96700 1 -- 172.21.15.89:0/2557918833 >> 172.21.15.89:0/2557918833 conn(0x7f7308066f50 msgr2=0x7f730809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:40.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.761+0000 7f730fb96700 1 -- 172.21.15.89:0/2557918833 shutdown_connections 2023-09-30T22:18:40.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.761+0000 7f730fb96700 1 -- 172.21.15.89:0/2557918833 wait complete. 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.762+0000 7f730fb96700 1 Processor -- start 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.762+0000 7f730fb96700 1 -- start start 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730fb96700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730809d6f0 0x7f7308127c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730fb96700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 0x7f7308128140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730fb96700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308139170 0x7f73081cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730fb96700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7308127820 con 0x7f7308139170 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730fb96700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7308138d80 con 0x7f730809d6f0 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730fb96700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7308128680 con 0x7f730809e1d0 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730d932700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730809d6f0 0x7f7308127c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730d932700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730809d6f0 0x7f7308127c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34984/0 (socket says 172.21.15.89:34984) 2023-09-30T22:18:40.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730d932700 1 -- 172.21.15.89:0/2397526684 learned_addr learned my addr 172.21.15.89:0/2397526684 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.763+0000 7f730d131700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 0x7f7308128140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730e133700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308139170 0x7f73081cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730d932700 1 -- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 msgr2=0x7f7308128140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730d932700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 0x7f7308128140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730d932700 1 -- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308139170 msgr2=0x7f73081cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730d932700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308139170 0x7f73081cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730d932700 1 -- 172.21.15.89:0/2397526684 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7300009a70 con 0x7f730809d6f0 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730d932700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730809d6f0 0x7f7308127c00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f730400c520 tx=0x7f730400c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f72fe7fc700 1 -- 172.21.15.89:0/2397526684 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304004c80 con 0x7f730809d6f0 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f72fe7fc700 1 -- 172.21.15.89:0/2397526684 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7304004e00 con 0x7f730809d6f0 2023-09-30T22:18:40.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730809c710 con 0x7f730809d6f0 2023-09-30T22:18:40.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f72fe7fc700 1 -- 172.21.15.89:0/2397526684 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304007310 con 0x7f730809d6f0 2023-09-30T22:18:40.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.764+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7308128f20 con 0x7f730809d6f0 2023-09-30T22:18:40.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.765+0000 7f72fe7fc700 1 -- 172.21.15.89:0/2397526684 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7304007470 con 0x7f730809d6f0 2023-09-30T22:18:40.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.765+0000 7f72fe7fc700 1 -- 172.21.15.89:0/2397526684 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f730403f430 con 0x7f730809d6f0 2023-09-30T22:18:40.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.765+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73080a0720 con 0x7f730809d6f0 2023-09-30T22:18:40.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.774+0000 7f72fe7fc700 1 -- 172.21.15.89:0/2397526684 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73040123b0 con 0x7f730809d6f0 2023-09-30T22:18:40.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a159c5700 1 Processor -- start 2023-09-30T22:18:40.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a159c5700 1 -- start start 2023-09-30T22:18:40.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a159c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009d6f0 0x7f0a1009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a159c5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0a1009e000 0x7f0a10130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a159c5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009e410 0x7f0a10130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a159c5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1009f520 con 0x7f0a1009e000 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a159c5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1009f130 con 0x7f0a1009e410 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a159c5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1009e930 con 0x7f0a1009d6f0 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a0effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009d6f0 0x7f0a1009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a0effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009d6f0 0x7f0a1009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44438/0 (socket says 172.21.15.89:44438) 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a0effd700 1 -- 172.21.15.89:0/2493311426 learned_addr learned my addr 172.21.15.89:0/2493311426 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a0f7fe700 1 --2- 172.21.15.89:0/2493311426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009e410 0x7f0a10130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.784+0000 7f0a0e7fc700 1 --2- 172.21.15.89:0/2493311426 >> v2:172.21.15.89:3300/0 conn(0x7f0a1009e000 0x7f0a10130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0f7fe700 1 -- 172.21.15.89:0/2493311426 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009d6f0 msgr2=0x7f0a1009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0f7fe700 1 --2- 172.21.15.89:0/2493311426 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009d6f0 0x7f0a1009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0f7fe700 1 -- 172.21.15.89:0/2493311426 >> v2:172.21.15.89:3300/0 conn(0x7f0a1009e000 msgr2=0x7f0a10130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0f7fe700 1 --2- 172.21.15.89:0/2493311426 >> v2:172.21.15.89:3300/0 conn(0x7f0a1009e000 0x7f0a10130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0f7fe700 1 -- 172.21.15.89:0/2493311426 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1009b2d0 con 0x7f0a1009e410 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0e7fc700 1 --2- 172.21.15.89:0/2493311426 >> v2:172.21.15.89:3300/0 conn(0x7f0a1009e000 0x7f0a10130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0f7fe700 1 --2- 172.21.15.89:0/2493311426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009e410 0x7f0a10130ba0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009d90 tx=0x7f0a0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a87e6a80e50d8650 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0dffb700 1 -- 172.21.15.89:0/2493311426 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800e070 con 0x7f0a1009e410 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0dffb700 1 -- 172.21.15.89:0/2493311426 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0800bb20 con 0x7f0a1009e410 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.785+0000 7f0a0dffb700 1 -- 172.21.15.89:0/2493311426 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08012430 con 0x7f0a1009e410 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 -- 172.21.15.89:0/2493311426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009e410 msgr2=0x7f0a10130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 --2- 172.21.15.89:0/2493311426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009e410 0x7f0a10130ba0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009d90 tx=0x7f0a0800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:40.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 -- 172.21.15.89:0/2493311426 shutdown_connections 2023-09-30T22:18:40.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 --2- 172.21.15.89:0/2493311426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009e410 0x7f0a10130ba0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 --2- 172.21.15.89:0/2493311426 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009d6f0 0x7f0a1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 --2- 172.21.15.89:0/2493311426 >> v2:172.21.15.89:3300/0 conn(0x7f0a1009e000 0x7f0a10130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 -- 172.21.15.89:0/2493311426 >> 172.21.15.89:0/2493311426 conn(0x7f0a10066f50 msgr2=0x7f0a1009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:40.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 -- 172.21.15.89:0/2493311426 shutdown_connections 2023-09-30T22:18:40.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.786+0000 7f0a159c5700 1 -- 172.21.15.89:0/2493311426 wait complete. 2023-09-30T22:18:40.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.787+0000 7f0a159c5700 1 Processor -- start 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.787+0000 7f2484c6f700 1 Processor -- start 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.787+0000 7f2484c6f700 1 -- start start 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f2484c6f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f248009d6f0 0x7f248009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f2484c6f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009e000 0x7f2480130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f2484c6f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248009f130 con 0x7f248009e410 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f2484c6f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248009f520 con 0x7f248009d6f0 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f2484c6f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f248009e930 con 0x7f248009e000 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f247e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f248009d6f0 0x7f248009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f247e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f248009d6f0 0x7f248009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45244/0 (socket says 172.21.15.89:45244) 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f247e59c700 1 -- 172.21.15.89:0/1649274259 learned_addr learned my addr 172.21.15.89:0/1649274259 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:40.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.789+0000 7f247e59c700 1 -- 172.21.15.89:0/1649274259 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009e000 msgr2=0x7f2480130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:40.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.789+0000 7f247e59c700 1 --2- 172.21.15.89:0/1649274259 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009e000 0x7f2480130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.789+0000 7f247e59c700 1 -- 172.21.15.89:0/1649274259 >> v1:172.21.15.89:6789/0 conn(0x7f248009e410 legacy=0x7f2480130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.789+0000 7f247e59c700 1 -- 172.21.15.89:0/1649274259 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248009b2d0 con 0x7f248009d6f0 2023-09-30T22:18:40.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.789+0000 7f247e59c700 1 --2- 172.21.15.89:0/1649274259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009d6f0 0x7f248009dac0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f247000bb90 tx=0x7f247000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b33d79fa454fadf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:40.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.787+0000 7f0a159c5700 1 -- start start 2023-09-30T22:18:40.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a159c5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009d6f0 0x7f0a1011f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a159c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009e000 0x7f0a1011f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a159c5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a1009e410 0x7f0a1011fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a159c5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10122ac0 con 0x7f0a1009e410 2023-09-30T22:18:40.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a159c5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10122ca0 con 0x7f0a1009d6f0 2023-09-30T22:18:40.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a159c5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100a2a90 con 0x7f0a1009e000 2023-09-30T22:18:40.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a1009e410 0x7f0a1011fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a1009e410 0x7f0a1011fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45238/0 (socket says 172.21.15.89:45238) 2023-09-30T22:18:40.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0f7fe700 1 -- 172.21.15.89:0/184792030 learned_addr learned my addr 172.21.15.89:0/184792030 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:40.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0e7fc700 1 --2- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009e000 0x7f0a1011f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0e7fc700 1 -- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009d6f0 msgr2=0x7f0a1011f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0e7fc700 1 --2- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009d6f0 0x7f0a1011f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0e7fc700 1 -- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a1009e410 msgr2=0x7f0a1011fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0e7fc700 1 --2- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a1009e410 0x7f0a1011fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.788+0000 7f0a0e7fc700 1 -- 172.21.15.89:0/184792030 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040094a0 con 0x7f0a1009e000 2023-09-30T22:18:40.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.789+0000 7f0a0e7fc700 1 --2- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009e000 0x7f0a1011f790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e9b0 tx=0x7f0a0400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:40.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.789+0000 7f09f7fff700 1 -- 172.21.15.89:0/184792030 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400c850 con 0x7f0a1009e000 2023-09-30T22:18:40.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f247d59a700 1 -- 172.21.15.89:0/1649274259 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000dd50 con 0x7f248009d6f0 2023-09-30T22:18:40.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f247d59a700 1 -- 172.21.15.89:0/1649274259 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470010470 con 0x7f248009d6f0 2023-09-30T22:18:40.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f247d59a700 1 -- 172.21.15.89:0/1649274259 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470010800 con 0x7f248009d6f0 2023-09-30T22:18:40.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f09f7fff700 1 -- 172.21.15.89:0/184792030 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0400c9b0 con 0x7f0a1009e000 2023-09-30T22:18:40.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f2484c6f700 1 -- 172.21.15.89:0/1649274259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009d6f0 msgr2=0x7f248009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f2484c6f700 1 --2- 172.21.15.89:0/1649274259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009d6f0 0x7f248009dac0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f247000bb90 tx=0x7f247000bea0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f2484c6f700 1 -- 172.21.15.89:0/1649274259 shutdown_connections 2023-09-30T22:18:40.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f2484c6f700 1 --2- 172.21.15.89:0/1649274259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009d6f0 0x7f248009dac0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f2484c6f700 1 --2- 172.21.15.89:0/1649274259 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009e000 0x7f2480130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f2484c6f700 1 -- 172.21.15.89:0/1649274259 >> 172.21.15.89:0/1649274259 conn(0x7f2480066f50 msgr2=0x7f248009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:40.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a08009a70 con 0x7f0a1009e000 2023-09-30T22:18:40.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f09f7fff700 1 -- 172.21.15.89:0/184792030 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400cd40 con 0x7f0a1009e000 2023-09-30T22:18:40.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.790+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a10121560 con 0x7f0a1009e000 2023-09-30T22:18:40.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f09f7fff700 1 -- 172.21.15.89:0/184792030 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a04013470 con 0x7f0a1009e000 2023-09-30T22:18:40.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f09f7fff700 1 -- 172.21.15.89:0/184792030 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a0403f9a0 con 0x7f0a1009e000 2023-09-30T22:18:40.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a10074ea0 con 0x7f0a1009e000 2023-09-30T22:18:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f2484c6f700 1 -- 172.21.15.89:0/1649274259 shutdown_connections 2023-09-30T22:18:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.791+0000 7f2484c6f700 1 -- 172.21.15.89:0/1649274259 wait complete. 2023-09-30T22:18:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.792+0000 7f2484c6f700 1 Processor -- start 2023-09-30T22:18:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f2484c6f700 1 -- start start 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f2484c6f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009d6f0 0x7f248011e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f2484c6f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f248009e000 0x7f248011e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f2484c6f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009e410 0x7f248011eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f2484c6f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2480121670 con 0x7f248009e410 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f2484c6f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2480121bd0 con 0x7f248009e000 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f2484c6f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24800a1e20 con 0x7f248009d6f0 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f247dd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f248009e000 0x7f248011e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f247dd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f248009e000 0x7f248011e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35010/0 (socket says 172.21.15.89:35010) 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.793+0000 7f247ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009e410 0x7f248011eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247dd9b700 1 -- 172.21.15.89:0/2895793740 learned_addr learned my addr 172.21.15.89:0/2895793740 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247e59c700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009d6f0 0x7f248011e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247dd9b700 1 -- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009d6f0 msgr2=0x7f248011e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247dd9b700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009d6f0 0x7f248011e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247dd9b700 1 -- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009e410 msgr2=0x7f248011eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247dd9b700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009e410 0x7f248011eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247dd9b700 1 -- 172.21.15.89:0/2895793740 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247000b7f0 con 0x7f248009e000 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247e59c700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009d6f0 0x7f248011e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f247dd9b700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f248009e000 0x7f248011e5f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f247400b530 tx=0x7f247400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f246effd700 1 -- 172.21.15.89:0/2895793740 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474011830 con 0x7f248009e000 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f246effd700 1 -- 172.21.15.89:0/2895793740 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2474011990 con 0x7f248009e000 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2480120480 con 0x7f248009e000 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2480121850 con 0x7f248009e000 2023-09-30T22:18:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.794+0000 7f246effd700 1 -- 172.21.15.89:0/2895793740 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474011d40 con 0x7f248009e000 2023-09-30T22:18:40.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.795+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24640052e0 con 0x7f248009e000 2023-09-30T22:18:40.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.796+0000 7f246effd700 1 -- 172.21.15.89:0/2895793740 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f247401a400 con 0x7f248009e000 2023-09-30T22:18:40.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.796+0000 7f246effd700 1 -- 172.21.15.89:0/2895793740 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f247403f660 con 0x7f248009e000 2023-09-30T22:18:40.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.796+0000 7f9de5f05700 1 Processor -- start 2023-09-30T22:18:40.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.797+0000 7f9de5f05700 1 -- start start 2023-09-30T22:18:40.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.797+0000 7f09f7fff700 1 -- 172.21.15.89:0/184792030 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a04013720 con 0x7f0a1009e000 2023-09-30T22:18:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.797+0000 7f9de5f05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de0133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.797+0000 7f9de5f05700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9de01341d0 0x7f9de009de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.797+0000 7f9de5f05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de009e390 0x7f9de009e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.797+0000 7f9de5f05700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de0070c80 con 0x7f9de01341d0 2023-09-30T22:18:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.797+0000 7f9de5f05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de009ed20 con 0x7f9de009e390 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.797+0000 7f9de5f05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9de006f720 con 0x7f9de01338c0 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de0133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9de01341d0 0x7f9de009de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9de01341d0 0x7f9de009de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45272/0 (socket says 172.21.15.89:45272) 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de0133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44472/0 (socket says 172.21.15.89:44472) 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 -- 172.21.15.89:0/2115861511 learned_addr learned my addr 172.21.15.89:0/2115861511 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddffff700 1 --2- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de009e390 0x7f9de009e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 -- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 msgr2=0x7f9de0133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 --2- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de0133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 -- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de009e390 msgr2=0x7f9de009e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 --2- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de009e390 0x7f9de009e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 -- 172.21.15.89:0/2115861511 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de006fce0 con 0x7f9de01341d0 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9ddeffd700 1 --2- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de01341d0 0x7f9de009de50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40098f0 tx=0x7f9dd4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704c2ef3f20e29f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9dde7fc700 1 -- 172.21.15.89:0/2115861511 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4011830 con 0x7f9de01341d0 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9dde7fc700 1 -- 172.21.15.89:0/2115861511 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd4011990 con 0x7f9de01341d0 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.798+0000 7f9dde7fc700 1 -- 172.21.15.89:0/2115861511 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4011d20 con 0x7f9de01341d0 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 -- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de01341d0 msgr2=0x7f9de009de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 --2- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de01341d0 0x7f9de009de50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40098f0 tx=0x7f9dd4009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 -- 172.21.15.89:0/2115861511 shutdown_connections 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 --2- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de009e390 0x7f9de009e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 --2- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de0133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 --2- 172.21.15.89:0/2115861511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de01341d0 0x7f9de009de50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 -- 172.21.15.89:0/2115861511 >> 172.21.15.89:0/2115861511 conn(0x7f9de00a2aa0 msgr2=0x7f9de00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 -- 172.21.15.89:0/2115861511 shutdown_connections 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.799+0000 7f9de5f05700 1 -- 172.21.15.89:0/2115861511 wait complete. 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.800+0000 7f9de5f05700 1 Processor -- start 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9de5f05700 1 -- start start 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9de5f05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de009e390 0x7f9de011db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9de5f05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de011e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9de5f05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de01341d0 0x7f9de011e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9de5f05700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de0120df0 con 0x7f9de009e390 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9de5f05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de011f790 con 0x7f9de01341d0 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9de5f05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9de011efe0 con 0x7f9de01338c0 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9ddf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de009e390 0x7f9de011db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9ddf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de009e390 0x7f9de011db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45274/0 (socket says 172.21.15.89:45274) 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9ddf7fe700 1 -- 172.21.15.89:0/870042286 learned_addr learned my addr 172.21.15.89:0/870042286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9ddffff700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de01341d0 0x7f9de011e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f9ddeffd700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de011e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9ddffff700 1 -- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 msgr2=0x7f9de011e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9ddffff700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de011e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9ddffff700 1 -- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de009e390 msgr2=0x7f9de011db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9ddffff700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de009e390 0x7f9de011db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9ddffff700 1 -- 172.21.15.89:0/870042286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4009550 con 0x7f9de01341d0 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9ddf7fe700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de009e390 0x7f9de011db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9ddeffd700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de011e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9ddffff700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de01341d0 0x7f9de011e600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8007cf0 tx=0x7f9dd800e0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9dc7fff700 1 -- 172.21.15.89:0/870042286 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8018070 con 0x7f9de01341d0 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9dc7fff700 1 -- 172.21.15.89:0/870042286 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd8003a20 con 0x7f9de01341d0 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de01216a0 con 0x7f9de01341d0 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9dc7fff700 1 -- 172.21.15.89:0/870042286 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8003db0 con 0x7f9de01341d0 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.802+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de006f960 con 0x7f9de01341d0 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.803+0000 7f9dc7fff700 1 -- 172.21.15.89:0/870042286 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9dd800e970 con 0x7f9de01341d0 2023-09-30T22:18:40.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.804+0000 7f9dc7fff700 1 -- 172.21.15.89:0/870042286 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9dd803f550 con 0x7f9de01341d0 2023-09-30T22:18:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.805+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de0074ea0 con 0x7f9de01341d0 2023-09-30T22:18:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.801+0000 7f246effd700 1 -- 172.21.15.89:0/2895793740 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f247401ae60 con 0x7f248009e000 2023-09-30T22:18:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.809+0000 7f9dc7fff700 1 -- 172.21.15.89:0/870042286 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9dd801b310 con 0x7f9de01341d0 2023-09-30T22:18:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.990+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0a100a1b30 con 0x7f0a1009e000 2023-09-30T22:18:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.991+0000 7f09f7fff700 1 -- 172.21.15.89:0/184792030 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a0402a090 con 0x7f0a1009e000 2023-09-30T22:18:40.991 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.991+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7308074ea0 con 0x7f730809d6f0 2023-09-30T22:18:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.991+0000 7f72fe7fc700 1 -- 172.21.15.89:0/2397526684 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7304011360 con 0x7f730809d6f0 2023-09-30T22:18:40.991 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009e000 msgr2=0x7f0a1011f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 --2- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009e000 0x7f0a1011f790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e9b0 tx=0x7f0a0400ed70 comp rx=0 tx=0).stop 2023-09-30T22:18:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 shutdown_connections 2023-09-30T22:18:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 --2- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a1009e410 0x7f0a1011fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 --2- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a1009d6f0 0x7f0a1011f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 --2- 172.21.15.89:0/184792030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a1009e000 0x7f0a1011f790 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 >> 172.21.15.89:0/184792030 conn(0x7f0a10066f50 msgr2=0x7f0a10119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 shutdown_connections 2023-09-30T22:18:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f0a159c5700 1 -- 172.21.15.89:0/184792030 wait complete. 2023-09-30T22:18:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730809d6f0 msgr2=0x7f7308127c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730809d6f0 0x7f7308127c00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f730400c520 tx=0x7f730400c8e0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 shutdown_connections 2023-09-30T22:18:40.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308139170 0x7f73081cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f730809d6f0 0x7f7308127c00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 --2- 172.21.15.89:0/2397526684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f730809e1d0 0x7f7308128140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:40.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 >> 172.21.15.89:0/2397526684 conn(0x7f7308066f50 msgr2=0x7f7308070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:40.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 shutdown_connections 2023-09-30T22:18:40.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:40.994+0000 7f730fb96700 1 -- 172.21.15.89:0/2397526684 wait complete. 2023-09-30T22:18:41.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:41.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.013+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9de01304e0 con 0x7f9de01341d0 2023-09-30T22:18:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.013+0000 7f9dc7fff700 1 -- 172.21.15.89:0/870042286 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9dd801cb20 con 0x7f9de01341d0 2023-09-30T22:18:41.014 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de01341d0 msgr2=0x7f9de011e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de01341d0 0x7f9de011e600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8007cf0 tx=0x7f9dd800e0e0 comp rx=0 tx=0).stop 2023-09-30T22:18:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 shutdown_connections 2023-09-30T22:18:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9de01341d0 0x7f9de011e600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9de009e390 0x7f9de011db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 --2- 172.21.15.89:0/870042286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9de01338c0 0x7f9de011e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 >> 172.21.15.89:0/870042286 conn(0x7f9de00a2aa0 msgr2=0x7f9de009f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 shutdown_connections 2023-09-30T22:18:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.015+0000 7f9de5f05700 1 -- 172.21.15.89:0/870042286 wait complete. 2023-09-30T22:18:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.021+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2464005590 con 0x7f248009e000 2023-09-30T22:18:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.021+0000 7f246effd700 1 -- 172.21.15.89:0/2895793740 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2474022350 con 0x7f248009e000 2023-09-30T22:18:41.021 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f248009e000 msgr2=0x7f248011e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f248009e000 0x7f248011e5f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f247400b530 tx=0x7f247400b840 comp rx=0 tx=0).stop 2023-09-30T22:18:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 shutdown_connections 2023-09-30T22:18:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248009e410 0x7f248011eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248009d6f0 0x7f248011e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 --2- 172.21.15.89:0/2895793740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f248009e000 0x7f248011e5f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 >> 172.21.15.89:0/2895793740 conn(0x7f2480066f50 msgr2=0x7f2480071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 shutdown_connections 2023-09-30T22:18:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:41.024+0000 7f2484c6f700 1 -- 172.21.15.89:0/2895793740 wait complete. 2023-09-30T22:18:41.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:41.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:42.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:42.014 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:42.028 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:42.037 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:42.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.158+0000 7f818bd61700 1 Processor -- start 2023-09-30T22:18:42.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.158+0000 7fa466b95700 1 Processor -- start 2023-09-30T22:18:42.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.158+0000 7fa466b95700 1 -- start start 2023-09-30T22:18:42.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.158+0000 7f818bd61700 1 -- start start 2023-09-30T22:18:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7fa466b95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009c650 0x7fa46009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7fa466b95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009cf60 0x7fa460132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7fa466b95700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46009ba70 con 0x7fa4600a0090 2023-09-30T22:18:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7fa466b95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46009e8f0 con 0x7fa46009cf60 2023-09-30T22:18:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7fa466b95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa46009d400 con 0x7fa46009c650 2023-09-30T22:18:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f818bd61700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81841305f0 0x7f81841309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f818bd61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 0x7f818406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f818bd61700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81841301c0 con 0x7f818406ee80 2023-09-30T22:18:42.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f818bd61700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8184071ff0 con 0x7f81841305f0 2023-09-30T22:18:42.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f818bd61700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8184070e70 con 0x7f818406e550 2023-09-30T22:18:42.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7fa45ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa4600a0090 0x7fa460131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52974/0 (socket says 172.21.15.89:52974) 2023-09-30T22:18:42.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7fa45ffff700 1 -- 172.21.15.89:0/1851522305 learned_addr learned my addr 172.21.15.89:0/1851522305 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:42.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f8189afd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81841305f0 0x7f81841309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f8189afd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81841305f0 0x7f81841309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45288/0 (socket says 172.21.15.89:45288) 2023-09-30T22:18:42.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f8189afd700 1 -- 172.21.15.89:0/1972999572 learned_addr learned my addr 172.21.15.89:0/1972999572 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:42.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.159+0000 7f818a2fe700 1 --1- 172.21.15.89:0/1972999572 >> v1:172.21.15.89:6789/0 conn(0x7f818406ee80 0x7f818406f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52976/0 (socket says 172.21.15.89:52976) 2023-09-30T22:18:42.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa464931700 1 --2- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009c650 0x7fa46009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa465132700 1 --2- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009cf60 0x7fa460132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f81892fc700 1 --2- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 0x7f818406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f8189afd700 1 -- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 msgr2=0x7f818406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:42.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f8189afd700 1 --2- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 0x7f818406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f8189afd700 1 -- 172.21.15.89:0/1972999572 >> v1:172.21.15.89:6789/0 conn(0x7f818406ee80 legacy=0x7f818406f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f8189afd700 1 -- 172.21.15.89:0/1972999572 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840722b0 con 0x7f81841305f0 2023-09-30T22:18:42.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f81892fc700 1 --2- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 0x7f818406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:42.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f8189afd700 1 --2- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81841305f0 0x7f81841309c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f81800098f0 tx=0x7f8180009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e675f76c7bb9c965 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:42.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa465132700 1 -- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009c650 msgr2=0x7fa46009ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:42.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa465132700 1 --2- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009c650 0x7fa46009ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa465132700 1 -- 172.21.15.89:0/1851522305 >> v1:172.21.15.89:6789/0 conn(0x7fa4600a0090 legacy=0x7fa460131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa465132700 1 -- 172.21.15.89:0/1851522305 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600a0760 con 0x7fa46009cf60 2023-09-30T22:18:42.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa465132700 1 --2- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009cf60 0x7fa460132080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa458009dc0 tx=0x7fa45800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edae475ce2ebd00 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:42.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f8188afb700 1 -- 172.21.15.89:0/1972999572 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000b8a0 con 0x7f81841305f0 2023-09-30T22:18:42.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f8188afb700 1 -- 172.21.15.89:0/1972999572 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000ba00 con 0x7f81841305f0 2023-09-30T22:18:42.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa45f7fe700 1 -- 172.21.15.89:0/1851522305 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800e070 con 0x7fa46009cf60 2023-09-30T22:18:42.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa45f7fe700 1 -- 172.21.15.89:0/1851522305 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45800bb20 con 0x7fa46009cf60 2023-09-30T22:18:42.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7fa45f7fe700 1 -- 172.21.15.89:0/1851522305 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458012430 con 0x7fa46009cf60 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.160+0000 7f8188afb700 1 -- 172.21.15.89:0/1972999572 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000bd90 con 0x7f81841305f0 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7f818bd61700 1 -- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81841305f0 msgr2=0x7f81841309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7f818bd61700 1 --2- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81841305f0 0x7f81841309c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f81800098f0 tx=0x7f8180009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7f818bd61700 1 -- 172.21.15.89:0/1972999572 shutdown_connections 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7f818bd61700 1 --2- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81841305f0 0x7f81841309c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7f818bd61700 1 --2- 172.21.15.89:0/1972999572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 0x7f818406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7f818bd61700 1 -- 172.21.15.89:0/1972999572 >> 172.21.15.89:0/1972999572 conn(0x7f81840a2aa0 msgr2=0x7f81840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7f818bd61700 1 -- 172.21.15.89:0/1972999572 shutdown_connections 2023-09-30T22:18:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7f818bd61700 1 -- 172.21.15.89:0/1972999572 wait complete. 2023-09-30T22:18:42.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7fa466b95700 1 -- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009cf60 msgr2=0x7fa460132080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7fa466b95700 1 --2- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009cf60 0x7fa460132080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa458009dc0 tx=0x7fa45800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:42.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7fa466b95700 1 -- 172.21.15.89:0/1851522305 shutdown_connections 2023-09-30T22:18:42.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7fa466b95700 1 --2- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009cf60 0x7fa460132080 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7fa466b95700 1 --2- 172.21.15.89:0/1851522305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009c650 0x7fa46009ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.161+0000 7fa466b95700 1 -- 172.21.15.89:0/1851522305 >> 172.21.15.89:0/1851522305 conn(0x7fa4600a3f70 msgr2=0x7fa4600a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:42.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.162+0000 7fa466b95700 1 -- 172.21.15.89:0/1851522305 shutdown_connections 2023-09-30T22:18:42.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.162+0000 7fa466b95700 1 -- 172.21.15.89:0/1851522305 wait complete. 2023-09-30T22:18:42.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f818bd61700 1 Processor -- start 2023-09-30T22:18:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f818bd61700 1 -- start start 2023-09-30T22:18:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7fa466b95700 1 Processor -- start 2023-09-30T22:18:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7fa466b95700 1 -- start start 2023-09-30T22:18:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f818bd61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 0x7f8184119d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f818bd61700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f818406ee80 0x7f818411a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f818bd61700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81841305f0 0x7f818411a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f818bd61700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818411e260 con 0x7f818406ee80 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f818bd61700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8184071820 con 0x7f818406e550 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f818bd61700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f818411d040 con 0x7f81841305f0 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f81892fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f818406ee80 0x7f818411a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f81892fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f818406ee80 0x7f818411a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45292/0 (socket says 172.21.15.89:45292) 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.163+0000 7f81892fc700 1 -- 172.21.15.89:0/3775471085 learned_addr learned my addr 172.21.15.89:0/3775471085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f81892fc700 1 -- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81841305f0 msgr2=0x7f818411a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f81892fc700 1 --2- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81841305f0 0x7f818411a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f81892fc700 1 -- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 msgr2=0x7f8184119d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f81892fc700 1 --2- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 0x7f8184119d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f81892fc700 1 -- 172.21.15.89:0/3775471085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009550 con 0x7f818406ee80 2023-09-30T22:18:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa466b95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009c650 0x7fa46011a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa466b95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009cf60 0x7fa46011ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa466b95700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4600a0090 0x7fa46011b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa466b95700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46011c0f0 con 0x7fa4600a0090 2023-09-30T22:18:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa466b95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46011baa0 con 0x7fa46009c650 2023-09-30T22:18:42.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa466b95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa46009d980 con 0x7fa46009cf60 2023-09-30T22:18:42.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f81892fc700 1 --2- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f818406ee80 0x7f818411a260 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f817400c750 tx=0x7f817400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:42.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f817a7fc700 1 -- 172.21.15.89:0/3775471085 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174011800 con 0x7f818406ee80 2023-09-30T22:18:42.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f817a7fc700 1 -- 172.21.15.89:0/3775471085 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174011960 con 0x7f818406ee80 2023-09-30T22:18:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f817a7fc700 1 -- 172.21.15.89:0/3775471085 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174011cd0 con 0x7f818406ee80 2023-09-30T22:18:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa464931700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009c650 0x7fa46011a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa465132700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4600a0090 0x7fa46011b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa464931700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009c650 0x7fa46011a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35066/0 (socket says 172.21.15.89:35066) 2023-09-30T22:18:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7fa464931700 1 -- 172.21.15.89:0/43739275 learned_addr learned my addr 172.21.15.89:0/43739275 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818411dd70 con 0x7f818406ee80 2023-09-30T22:18:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.164+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818411b660 con 0x7f818406ee80 2023-09-30T22:18:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa464931700 1 -- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009cf60 msgr2=0x7fa46011ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa464931700 1 --2- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009cf60 0x7fa46011ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa464931700 1 -- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4600a0090 msgr2=0x7fa46011b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa464931700 1 --2- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4600a0090 0x7fa46011b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa464931700 1 -- 172.21.15.89:0/43739275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458009a70 con 0x7fa46009c650 2023-09-30T22:18:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa464931700 1 --2- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009c650 0x7fa46011a620 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e9b0 tx=0x7fa45000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7f817a7fc700 1 -- 172.21.15.89:0/3775471085 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8174021350 con 0x7f818406ee80 2023-09-30T22:18:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818411d390 con 0x7f818406ee80 2023-09-30T22:18:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.166+0000 7f817a7fc700 1 -- 172.21.15.89:0/3775471085 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f817403f680 con 0x7f818406ee80 2023-09-30T22:18:42.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa45d7fa700 1 -- 172.21.15.89:0/43739275 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450013860 con 0x7fa46009c650 2023-09-30T22:18:42.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa45d7fa700 1 -- 172.21.15.89:0/43739275 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4500139c0 con 0x7fa46009c650 2023-09-30T22:18:42.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46011d6d0 con 0x7fa46009c650 2023-09-30T22:18:42.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa45d7fa700 1 -- 172.21.15.89:0/43739275 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450013d30 con 0x7fa46009c650 2023-09-30T22:18:42.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.165+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46011ccf0 con 0x7fa46009c650 2023-09-30T22:18:42.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.166+0000 7fa45d7fa700 1 -- 172.21.15.89:0/43739275 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa45000c5e0 con 0x7fa46009c650 2023-09-30T22:18:42.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.166+0000 7fa45d7fa700 1 -- 172.21.15.89:0/43739275 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa450043660 con 0x7fa46009c650 2023-09-30T22:18:42.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.167+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4600747f0 con 0x7fa46009c650 2023-09-30T22:18:42.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.172+0000 7fa45d7fa700 1 -- 172.21.15.89:0/43739275 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa450020080 con 0x7fa46009c650 2023-09-30T22:18:42.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.175+0000 7f817a7fc700 1 -- 172.21.15.89:0/3775471085 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8174011e30 con 0x7f818406ee80 2023-09-30T22:18:42.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.201+0000 7fd42dad4700 1 Processor -- start 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.201+0000 7fd42dad4700 1 -- start start 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd42dad4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd42809d6f0 0x7fd42809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd42dad4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e000 0x7fd4281339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd42dad4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809e410 0x7fd428133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd42dad4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42806b460 con 0x7fd42809d6f0 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd42dad4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42809b120 con 0x7fd42809e000 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd42dad4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd42809a7e0 con 0x7fd42809e410 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e000 0x7fd4281339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e000 0x7fd4281339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35074/0 (socket says 172.21.15.89:35074) 2023-09-30T22:18:42.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 -- 172.21.15.89:0/163684644 learned_addr learned my addr 172.21.15.89:0/163684644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd4277fe700 1 --2- 172.21.15.89:0/163684644 >> v2:172.21.15.89:3300/0 conn(0x7fd42809d6f0 0x7fd42809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 -- 172.21.15.89:0/163684644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809e410 msgr2=0x7fd428133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 --2- 172.21.15.89:0/163684644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809e410 0x7fd428133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 -- 172.21.15.89:0/163684644 >> v2:172.21.15.89:3300/0 conn(0x7fd42809d6f0 msgr2=0x7fd42809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 --2- 172.21.15.89:0/163684644 >> v2:172.21.15.89:3300/0 conn(0x7fd42809d6f0 0x7fd42809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 -- 172.21.15.89:0/163684644 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42809a500 con 0x7fd42809e000 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd4277fe700 1 --2- 172.21.15.89:0/163684644 >> v2:172.21.15.89:3300/0 conn(0x7fd42809d6f0 0x7fd42809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.202+0000 7fd426ffd700 1 --2- 172.21.15.89:0/163684644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e000 0x7fd4281339f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009920 tx=0x7fd41c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d09f5fb5dc3296a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:42.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd4267fc700 1 -- 172.21.15.89:0/163684644 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00ba20 con 0x7fd42809e000 2023-09-30T22:18:42.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd4267fc700 1 -- 172.21.15.89:0/163684644 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c00bb80 con 0x7fd42809e000 2023-09-30T22:18:42.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd4267fc700 1 -- 172.21.15.89:0/163684644 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c009e70 con 0x7fd42809e000 2023-09-30T22:18:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 -- 172.21.15.89:0/163684644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e000 msgr2=0x7fd4281339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 --2- 172.21.15.89:0/163684644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e000 0x7fd4281339f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009920 tx=0x7fd41c009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 -- 172.21.15.89:0/163684644 shutdown_connections 2023-09-30T22:18:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 --2- 172.21.15.89:0/163684644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809e410 0x7fd428133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 --2- 172.21.15.89:0/163684644 >> v2:172.21.15.89:3300/0 conn(0x7fd42809d6f0 0x7fd42809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 --2- 172.21.15.89:0/163684644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e000 0x7fd4281339f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 -- 172.21.15.89:0/163684644 >> 172.21.15.89:0/163684644 conn(0x7fd428066f50 msgr2=0x7fd42809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 -- 172.21.15.89:0/163684644 shutdown_connections 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.203+0000 7fd42dad4700 1 -- 172.21.15.89:0/163684644 wait complete. 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.204+0000 7fd42dad4700 1 Processor -- start 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.204+0000 7fd42dad4700 1 -- start start 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd42dad4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809d6f0 0x7fd4281b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd42dad4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd42809e000 0x7fd4281a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd42dad4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e410 0x7fd4281a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd42dad4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281aa2c0 con 0x7fd42809e000 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd42dad4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281a9900 con 0x7fd42809e410 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd42dad4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281a9090 con 0x7fd42809d6f0 2023-09-30T22:18:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd427fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e410 0x7fd4281a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd4277fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809d6f0 0x7fd4281b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd426ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd42809e000 0x7fd4281a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd427fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e410 0x7fd4281a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35084/0 (socket says 172.21.15.89:35084) 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd427fff700 1 -- 172.21.15.89:0/2936794310 learned_addr learned my addr 172.21.15.89:0/2936794310 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd427fff700 1 -- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809d6f0 msgr2=0x7fd4281b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd427fff700 1 --2- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809d6f0 0x7fd4281b91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd427fff700 1 -- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd42809e000 msgr2=0x7fd4281a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd427fff700 1 --2- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd42809e000 0x7fd4281a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.205+0000 7fd427fff700 1 -- 172.21.15.89:0/2936794310 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c009550 con 0x7fd42809e410 2023-09-30T22:18:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.206+0000 7fd427fff700 1 --2- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e410 0x7fd4281a6320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b9a0 tx=0x7fd42000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.206+0000 7fd40ffff700 1 -- 172.21.15.89:0/2936794310 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42000d850 con 0x7fd42809e410 2023-09-30T22:18:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.206+0000 7fd40ffff700 1 -- 172.21.15.89:0/2936794310 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42000d9d0 con 0x7fd42809e410 2023-09-30T22:18:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.206+0000 7fd40ffff700 1 -- 172.21.15.89:0/2936794310 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42000dd80 con 0x7fd42809e410 2023-09-30T22:18:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.206+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4281a8c40 con 0x7fd42809e410 2023-09-30T22:18:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.206+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4281a79f0 con 0x7fd42809e410 2023-09-30T22:18:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.207+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4281a92c0 con 0x7fd42809e410 2023-09-30T22:18:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.208+0000 7fd40ffff700 1 -- 172.21.15.89:0/2936794310 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4200203a0 con 0x7fd42809e410 2023-09-30T22:18:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.208+0000 7fd40ffff700 1 -- 172.21.15.89:0/2936794310 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd420006e40 con 0x7fd42809e410 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.208+0000 7f5afa19a700 1 Processor -- start 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5afa19a700 1 -- start start 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5afa19a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5af41338a0 0x7f5af4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5afa19a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409de50 0x7f5af409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5afa19a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409e760 0x7f5af413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5afa19a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af406fef0 con 0x7f5af41338a0 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5afa19a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af406ef30 con 0x7f5af409e760 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5afa19a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5af41334b0 con 0x7f5af409de50 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5af37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5af41338a0 0x7f5af4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5af3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409e760 0x7f5af413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5af37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5af41338a0 0x7f5af4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45324/0 (socket says 172.21.15.89:45324) 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5af3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409e760 0x7f5af413a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35098/0 (socket says 172.21.15.89:35098) 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5af37fe700 1 -- 172.21.15.89:0/3030011732 learned_addr learned my addr 172.21.15.89:0/3030011732 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.209+0000 7f5af2ffd700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409de50 0x7f5af409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af37fe700 1 -- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409de50 msgr2=0x7f5af409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af37fe700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409de50 0x7f5af409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af37fe700 1 -- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409e760 msgr2=0x7f5af413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af37fe700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409e760 0x7f5af413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af37fe700 1 -- 172.21.15.89:0/3030011732 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af406fc30 con 0x7f5af41338a0 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af3fff700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409e760 0x7f5af413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af37fe700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 0x7f5af4133c70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098f0 tx=0x7f5ae4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7de7aa35f4f98 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af27fc700 1 -- 172.21.15.89:0/3030011732 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4011830 con 0x7f5af41338a0 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af27fc700 1 -- 172.21.15.89:0/3030011732 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae4011990 con 0x7f5af41338a0 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.210+0000 7f5af27fc700 1 -- 172.21.15.89:0/3030011732 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4011d20 con 0x7f5af41338a0 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 -- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 msgr2=0x7f5af4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 0x7f5af4133c70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098f0 tx=0x7f5ae4009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 -- 172.21.15.89:0/3030011732 shutdown_connections 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409e760 0x7f5af413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 0x7f5af4133c70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 --2- 172.21.15.89:0/3030011732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409de50 0x7f5af409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 -- 172.21.15.89:0/3030011732 >> 172.21.15.89:0/3030011732 conn(0x7f5af40a2aa0 msgr2=0x7f5af40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 -- 172.21.15.89:0/3030011732 shutdown_connections 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.211+0000 7f5afa19a700 1 -- 172.21.15.89:0/3030011732 wait complete. 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.212+0000 7f5afa19a700 1 Processor -- start 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.212+0000 7f5afa19a700 1 -- start start 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5afa19a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409de50 0x7f5af411b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5afa19a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409e760 0x7f5af411bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5afa19a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 0x7f5af411c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5afa19a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af411cb50 con 0x7f5af41338a0 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5afa19a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af411d260 con 0x7f5af409de50 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5afa19a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5af411e4b0 con 0x7f5af409e760 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409de50 0x7f5af411b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409de50 0x7f5af411b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35104/0 (socket says 172.21.15.89:35104) 2023-09-30T22:18:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 -- 172.21.15.89:0/3811918120 learned_addr learned my addr 172.21.15.89:0/3811918120 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af3fff700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 0x7f5af411c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af2ffd700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409e760 0x7f5af411bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 -- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409e760 msgr2=0x7f5af411bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409e760 0x7f5af411bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 -- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 msgr2=0x7f5af411c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 0x7f5af411c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 -- 172.21.15.89:0/3811918120 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec009a70 con 0x7f5af409de50 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af2ffd700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409e760 0x7f5af411bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.213+0000 7f5af37fe700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409de50 0x7f5af411b6a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4000c00 tx=0x7f5ae4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.216+0000 7f5adbfff700 1 -- 172.21.15.89:0/3811918120 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4004240 con 0x7f5af409de50 2023-09-30T22:18:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.216+0000 7f5adbfff700 1 -- 172.21.15.89:0/3811918120 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae401f070 con 0x7f5af409de50 2023-09-30T22:18:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.216+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae4009550 con 0x7f5af409de50 2023-09-30T22:18:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.216+0000 7f5adbfff700 1 -- 172.21.15.89:0/3811918120 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40045c0 con 0x7f5af409de50 2023-09-30T22:18:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.216+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af40a0f90 con 0x7f5af409de50 2023-09-30T22:18:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.216+0000 7fd40ffff700 1 -- 172.21.15.89:0/2936794310 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd420015ab0 con 0x7fd42809e410 2023-09-30T22:18:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.217+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af406f640 con 0x7f5af409de50 2023-09-30T22:18:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.218+0000 7f5adbfff700 1 -- 172.21.15.89:0/3811918120 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ae4021030 con 0x7f5af409de50 2023-09-30T22:18:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.218+0000 7f5adbfff700 1 -- 172.21.15.89:0/3811918120 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ae4040bc0 con 0x7f5af409de50 2023-09-30T22:18:42.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.226+0000 7f5adbfff700 1 -- 172.21.15.89:0/3811918120 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ae4003e00 con 0x7f5af409de50 2023-09-30T22:18:42.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.379+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f818411d1b0 con 0x7f818406ee80 2023-09-30T22:18:42.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.379+0000 7f817a7fc700 1 -- 172.21.15.89:0/3775471085 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8174010d10 con 0x7f818406ee80 2023-09-30T22:18:42.380 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:42.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.380+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa460072840 con 0x7fa46009c650 2023-09-30T22:18:42.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.380+0000 7fa45d7fa700 1 -- 172.21.15.89:0/43739275 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa45001a990 con 0x7fa46009c650 2023-09-30T22:18:42.381 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:42.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.382+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f818406ee80 msgr2=0x7f818411a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.382+0000 7f818bd61700 1 --2- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f818406ee80 0x7f818411a260 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f817400c750 tx=0x7f817400cb10 comp rx=0 tx=0).stop 2023-09-30T22:18:42.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 shutdown_connections 2023-09-30T22:18:42.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7f818bd61700 1 --2- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81841305f0 0x7f818411a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7f818bd61700 1 --2- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f818406e550 0x7f8184119d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7f818bd61700 1 --2- 172.21.15.89:0/3775471085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f818406ee80 0x7f818411a260 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 >> 172.21.15.89:0/3775471085 conn(0x7f81840a2aa0 msgr2=0x7f818412e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:42.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009c650 msgr2=0x7fa46011a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 --2- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009c650 0x7fa46011a620 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e9b0 tx=0x7fa45000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 shutdown_connections 2023-09-30T22:18:42.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 shutdown_connections 2023-09-30T22:18:42.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 --2- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4600a0090 0x7fa46011b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7f818bd61700 1 -- 172.21.15.89:0/3775471085 wait complete. 2023-09-30T22:18:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 --2- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa46009c650 0x7fa46011a620 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 --2- 172.21.15.89:0/43739275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa46009cf60 0x7fa46011ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 >> 172.21.15.89:0/43739275 conn(0x7fa4600a3f70 msgr2=0x7fa46012eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:42.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 shutdown_connections 2023-09-30T22:18:42.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.383+0000 7fa466b95700 1 -- 172.21.15.89:0/43739275 wait complete. 2023-09-30T22:18:42.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:42.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:42.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.412+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd4281a92c0 con 0x7fd42809e410 2023-09-30T22:18:42.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.412+0000 7fd40ffff700 1 -- 172.21.15.89:0/2936794310 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4281a92c0 con 0x7fd42809e410 2023-09-30T22:18:42.412 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:42.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.414+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e410 msgr2=0x7fd4281a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.414+0000 7fd42dad4700 1 --2- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e410 0x7fd4281a6320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b9a0 tx=0x7fd42000bcb0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.414+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 shutdown_connections 2023-09-30T22:18:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.414+0000 7fd42dad4700 1 --2- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd42809e410 0x7fd4281a6320 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.414+0000 7fd42dad4700 1 --2- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd42809d6f0 0x7fd4281b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.414+0000 7fd42dad4700 1 --2- 172.21.15.89:0/2936794310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd42809e000 0x7fd4281a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.414+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 >> 172.21.15.89:0/2936794310 conn(0x7fd428066f50 msgr2=0x7fd4281bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.415+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 shutdown_connections 2023-09-30T22:18:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.415+0000 7fd42dad4700 1 -- 172.21.15.89:0/2936794310 wait complete. 2023-09-30T22:18:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.422+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5af4074ea0 con 0x7f5af409de50 2023-09-30T22:18:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.422+0000 7f5adbfff700 1 -- 172.21.15.89:0/3811918120 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ae4015030 con 0x7f5af409de50 2023-09-30T22:18:42.422 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.424+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409de50 msgr2=0x7f5af411b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.424+0000 7f5afa19a700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409de50 0x7f5af411b6a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4000c00 tx=0x7f5ae4004750 comp rx=0 tx=0).stop 2023-09-30T22:18:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.424+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 shutdown_connections 2023-09-30T22:18:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.424+0000 7f5afa19a700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5af41338a0 0x7f5af411c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.424+0000 7f5afa19a700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5af409de50 0x7f5af411b6a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.424+0000 7f5afa19a700 1 --2- 172.21.15.89:0/3811918120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5af409e760 0x7f5af411bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.424+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 >> 172.21.15.89:0/3811918120 conn(0x7f5af40a2aa0 msgr2=0x7f5af4130630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.424+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 shutdown_connections 2023-09-30T22:18:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:42.425+0000 7f5afa19a700 1 -- 172.21.15.89:0/3811918120 wait complete. 2023-09-30T22:18:42.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:42.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:43.402 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:43.402 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:43.428 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:43.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.559+0000 7f9365fc0700 1 Processor -- start 2023-09-30T22:18:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.559+0000 7f723b504700 1 Processor -- start 2023-09-30T22:18:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.559+0000 7f9365fc0700 1 -- start start 2023-09-30T22:18:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.559+0000 7f723b504700 1 -- start start 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f9365fc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f936009e1d0 0x7f9360135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f9365fc0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9360135e50 0x7f9360138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f9365fc0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936009f130 con 0x7f936009d6f0 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f9365fc0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936009f520 con 0x7f9360135e50 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f9365fc0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f936009e930 con 0x7f936009e1d0 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f935f7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f936009d6f0 0x7f936009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52986/0 (socket says 172.21.15.89:52986) 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f935f7fe700 1 -- 172.21.15.89:0/1215439502 learned_addr learned my addr 172.21.15.89:0/1215439502 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f935effd700 1 --2- 172.21.15.89:0/1215439502 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f936009e1d0 0x7f9360135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f935e7fc700 1 -- 172.21.15.89:0/1215439502 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913044477 0 0) 0x7f936009f130 con 0x7f936009d6f0 2023-09-30T22:18:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f723b504700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409d6f0 0x7f723409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f723b504700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f723409e000 0x7f72341339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f723b504700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723409b120 con 0x7f723409e410 2023-09-30T22:18:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f723b504700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723406b460 con 0x7f723409e000 2023-09-30T22:18:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f935e7fc700 1 -- 172.21.15.89:0/1215439502 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93480035b0 con 0x7f936009d6f0 2023-09-30T22:18:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f723b504700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f723409a7e0 con 0x7f723409d6f0 2023-09-30T22:18:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f7238a9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f723409e000 0x7f72341339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f7238a9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f723409e000 0x7f72341339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45346/0 (socket says 172.21.15.89:45346) 2023-09-30T22:18:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.560+0000 7f7238a9f700 1 -- 172.21.15.89:0/2089217208 learned_addr learned my addr 172.21.15.89:0/2089217208 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f72392a0700 1 --2- 172.21.15.89:0/2089217208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409d6f0 0x7f723409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f7238a9f700 1 -- 172.21.15.89:0/2089217208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409d6f0 msgr2=0x7f723409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f7238a9f700 1 --2- 172.21.15.89:0/2089217208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409d6f0 0x7f723409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f7238a9f700 1 -- 172.21.15.89:0/2089217208 >> v1:172.21.15.89:6789/0 conn(0x7f723409e410 legacy=0x7f7234133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f7238a9f700 1 -- 172.21.15.89:0/2089217208 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723409a500 con 0x7f723409e000 2023-09-30T22:18:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f935effd700 1 -- 172.21.15.89:0/1215439502 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9360135e50 msgr2=0x7f9360138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f935effd700 1 --2- 172.21.15.89:0/1215439502 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9360135e50 0x7f9360138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f935effd700 1 -- 172.21.15.89:0/1215439502 >> v1:172.21.15.89:6789/0 conn(0x7f936009d6f0 legacy=0x7f936009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f935effd700 1 -- 172.21.15.89:0/1215439502 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936009b2d0 con 0x7f936009e1d0 2023-09-30T22:18:43.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f935effd700 1 --2- 172.21.15.89:0/1215439502 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f936009e1d0 0x7f9360135910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9354009870 tx=0x7f9354009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9078fa53ab4b5dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:43.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f7238a9f700 1 --2- 172.21.15.89:0/2089217208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e000 0x7f72341339f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72240098f0 tx=0x7f7224009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4540751e8091c3f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:43.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f935e7fc700 1 -- 172.21.15.89:0/1215439502 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354011830 con 0x7f936009e1d0 2023-09-30T22:18:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.561+0000 7f722bfff700 1 -- 172.21.15.89:0/2089217208 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224004230 con 0x7f723409e000 2023-09-30T22:18:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f935e7fc700 1 -- 172.21.15.89:0/1215439502 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9354011990 con 0x7f936009e1d0 2023-09-30T22:18:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f935e7fc700 1 -- 172.21.15.89:0/1215439502 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354011d00 con 0x7f936009e1d0 2023-09-30T22:18:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f722bfff700 1 -- 172.21.15.89:0/2089217208 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7224004390 con 0x7f723409e000 2023-09-30T22:18:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f722bfff700 1 -- 172.21.15.89:0/2089217208 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72240036a0 con 0x7f723409e000 2023-09-30T22:18:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f9365fc0700 1 -- 172.21.15.89:0/1215439502 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f936009e1d0 msgr2=0x7f9360135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f9365fc0700 1 --2- 172.21.15.89:0/1215439502 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f936009e1d0 0x7f9360135910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9354009870 tx=0x7f9354009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f9365fc0700 1 -- 172.21.15.89:0/1215439502 shutdown_connections 2023-09-30T22:18:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f9365fc0700 1 --2- 172.21.15.89:0/1215439502 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9360135e50 0x7f9360138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f9365fc0700 1 --2- 172.21.15.89:0/1215439502 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f936009e1d0 0x7f9360135910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f9365fc0700 1 -- 172.21.15.89:0/1215439502 >> 172.21.15.89:0/1215439502 conn(0x7f9360066f50 msgr2=0x7f936009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f723b504700 1 -- 172.21.15.89:0/2089217208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e000 msgr2=0x7f72341339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f723b504700 1 --2- 172.21.15.89:0/2089217208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e000 0x7f72341339f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72240098f0 tx=0x7f7224009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.562+0000 7f9365fc0700 1 -- 172.21.15.89:0/1215439502 shutdown_connections 2023-09-30T22:18:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.563+0000 7f9365fc0700 1 -- 172.21.15.89:0/1215439502 wait complete. 2023-09-30T22:18:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.563+0000 7f723b504700 1 -- 172.21.15.89:0/2089217208 shutdown_connections 2023-09-30T22:18:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.563+0000 7f723b504700 1 --2- 172.21.15.89:0/2089217208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409d6f0 0x7f723409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.563+0000 7f723b504700 1 --2- 172.21.15.89:0/2089217208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e000 0x7f72341339f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.563+0000 7f723b504700 1 -- 172.21.15.89:0/2089217208 >> 172.21.15.89:0/2089217208 conn(0x7f7234066f50 msgr2=0x7f723409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.563+0000 7f723b504700 1 -- 172.21.15.89:0/2089217208 shutdown_connections 2023-09-30T22:18:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.563+0000 7f723b504700 1 -- 172.21.15.89:0/2089217208 wait complete. 2023-09-30T22:18:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.563+0000 7f9365fc0700 1 Processor -- start 2023-09-30T22:18:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f9365fc0700 1 -- start start 2023-09-30T22:18:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f723b504700 1 Processor -- start 2023-09-30T22:18:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f9365fc0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f936009d6f0 0x7f9360127780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f9365fc0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f936009e1d0 0x7f9360127cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f9365fc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9360135e50 0x7f93601cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f9365fc0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936012aad0 con 0x7f936009d6f0 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f9365fc0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9360128f70 con 0x7f936009e1d0 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f9365fc0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9360129a20 con 0x7f9360135e50 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f935ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9360135e50 0x7f93601cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f935ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9360135e50 0x7f93601cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44574/0 (socket says 172.21.15.89:44574) 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f935ffff700 1 -- 172.21.15.89:0/13035425 learned_addr learned my addr 172.21.15.89:0/13035425 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f935ffff700 1 -- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f936009e1d0 msgr2=0x7f9360127cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f935ffff700 1 --2- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f936009e1d0 0x7f9360127cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f935ffff700 1 -- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f936009d6f0 msgr2=0x7f9360127780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f935ffff700 1 --2- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f936009d6f0 0x7f9360127780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f935ffff700 1 -- 172.21.15.89:0/13035425 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540094a0 con 0x7f9360135e50 2023-09-30T22:18:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.564+0000 7f723b504700 1 -- start start 2023-09-30T22:18:43.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f723b504700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f723409d6f0 0x7f7234139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f723b504700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409e000 0x7f7234139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f7238a9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409e000 0x7f7234139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f7238a9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409e000 0x7f7234139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44584/0 (socket says 172.21.15.89:44584) 2023-09-30T22:18:43.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f935ffff700 1 --2- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9360135e50 0x7f93601cc430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f935800b9e0 tx=0x7f935800bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:43.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f72392a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f723409d6f0 0x7f7234139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f72392a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f723409d6f0 0x7f7234139680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35146/0 (socket says 172.21.15.89:35146) 2023-09-30T22:18:43.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f9347fff700 1 -- 172.21.15.89:0/13035425 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935800c9b0 con 0x7f9360135e50 2023-09-30T22:18:43.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f723b504700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e410 0x7f723413a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f7238a9f700 1 -- 172.21.15.89:0/685974546 learned_addr learned my addr 172.21.15.89:0/685974546 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:43.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723413b400 con 0x7f723409e410 2023-09-30T22:18:43.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723413b210 con 0x7f723409d6f0 2023-09-30T22:18:43.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f723413d1a0 con 0x7f723409e000 2023-09-30T22:18:43.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9360127390 con 0x7f9360135e50 2023-09-30T22:18:43.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936012b0b0 con 0x7f9360135e50 2023-09-30T22:18:43.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f9347fff700 1 -- 172.21.15.89:0/13035425 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935800cb10 con 0x7f9360135e50 2023-09-30T22:18:43.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.565+0000 7f9347fff700 1 -- 172.21.15.89:0/13035425 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935800ce80 con 0x7f9360135e50 2023-09-30T22:18:43.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f7239aa1700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e410 0x7f723413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f72392a0700 1 -- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409e000 msgr2=0x7f7234139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f72392a0700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409e000 0x7f7234139bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f72392a0700 1 -- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e410 msgr2=0x7f723413a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f72392a0700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e410 0x7f723413a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f72392a0700 1 -- 172.21.15.89:0/685974546 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7224009550 con 0x7f723409d6f0 2023-09-30T22:18:43.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f7238a9f700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409e000 0x7f7234139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:43.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f9347fff700 1 -- 172.21.15.89:0/13035425 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9358014440 con 0x7f9360135e50 2023-09-30T22:18:43.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f9347fff700 1 -- 172.21.15.89:0/13035425 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9358006e40 con 0x7f9360135e50 2023-09-30T22:18:43.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f72392a0700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f723409d6f0 0x7f7234139680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f723000e900 tx=0x7f723000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:43.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.566+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93400052e0 con 0x7f9360135e50 2023-09-30T22:18:43.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.567+0000 7f7229ffb700 1 -- 172.21.15.89:0/685974546 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000ee50 con 0x7f723409d6f0 2023-09-30T22:18:43.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.567+0000 7f7229ffb700 1 -- 172.21.15.89:0/685974546 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723000c850 con 0x7f723409d6f0 2023-09-30T22:18:43.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.567+0000 7f7229ffb700 1 -- 172.21.15.89:0/685974546 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000cbc0 con 0x7f723409d6f0 2023-09-30T22:18:43.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.567+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7234144430 con 0x7f723409d6f0 2023-09-30T22:18:43.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.567+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723409c710 con 0x7f723409d6f0 2023-09-30T22:18:43.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.568+0000 7f7229ffb700 1 -- 172.21.15.89:0/685974546 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f723000cd20 con 0x7f723409d6f0 2023-09-30T22:18:43.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.568+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723413cb60 con 0x7f723409d6f0 2023-09-30T22:18:43.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.568+0000 7f7229ffb700 1 -- 172.21.15.89:0/685974546 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f723003f880 con 0x7f723409d6f0 2023-09-30T22:18:43.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.573+0000 7f7229ffb700 1 -- 172.21.15.89:0/685974546 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f723413cb60 con 0x7f723409d6f0 2023-09-30T22:18:43.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.575+0000 7f9347fff700 1 -- 172.21.15.89:0/13035425 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93580133a0 con 0x7f9360135e50 2023-09-30T22:18:43.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.594+0000 7faf83b3b700 1 Processor -- start 2023-09-30T22:18:43.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.594+0000 7faf83b3b700 1 -- start start 2023-09-30T22:18:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf83b3b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c09f410 0x7faf7c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf83b3b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c136fe0 0x7faf7c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf83b3b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7c070940 con 0x7faf7c09e930 2023-09-30T22:18:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf83b3b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c06fd60 con 0x7faf7c09f410 2023-09-30T22:18:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf83b3b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c06fa80 con 0x7faf7c136fe0 2023-09-30T22:18:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf818d7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faf7c09e930 0x7faf7c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52998/0 (socket says 172.21.15.89:52998) 2023-09-30T22:18:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf818d7700 1 -- 172.21.15.89:0/278930110 learned_addr learned my addr 172.21.15.89:0/278930110 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf810d6700 1 --2- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c09f410 0x7faf7c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf820d8700 1 --2- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c136fe0 0x7faf7c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf808d5700 1 -- 172.21.15.89:0/278930110 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238214767 0 0) 0x7faf7c070940 con 0x7faf7c09e930 2023-09-30T22:18:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf808d5700 1 -- 172.21.15.89:0/278930110 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf680035b0 con 0x7faf7c09e930 2023-09-30T22:18:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7f1e69acb700 1 Processor -- start 2023-09-30T22:18:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf820d8700 1 -- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c09f410 msgr2=0x7faf7c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf820d8700 1 --2- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c09f410 0x7faf7c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf820d8700 1 -- 172.21.15.89:0/278930110 >> v1:172.21.15.89:6789/0 conn(0x7faf7c09e930 legacy=0x7faf7c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf808d5700 1 -- 172.21.15.89:0/278930110 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2125116938 0 0) 0x7faf680035b0 con 0x7faf7c09e930 2023-09-30T22:18:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7faf820d8700 1 -- 172.21.15.89:0/278930110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c09fb60 con 0x7faf7c136fe0 2023-09-30T22:18:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.595+0000 7f1e69acb700 1 -- start start 2023-09-30T22:18:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7faf820d8700 1 --2- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c136fe0 0x7faf7c1393c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf74009dc0 tx=0x7faf7400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e87958934b6a9416 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7faf808d5700 1 -- 172.21.15.89:0/278930110 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7400e070 con 0x7faf7c136fe0 2023-09-30T22:18:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7faf808d5700 1 -- 172.21.15.89:0/278930110 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74003a20 con 0x7faf7c136fe0 2023-09-30T22:18:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7faf808d5700 1 -- 172.21.15.89:0/278930110 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78001670 con 0x7faf7c136fe0 2023-09-30T22:18:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e69acb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e6412cf60 0x7f1e6406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e69acb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e6412f8b0 0x7f1e6412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e69acb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6412cd90 con 0x7f1e64130260 2023-09-30T22:18:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e69acb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6412d330 con 0x7f1e6412f8b0 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e69acb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e641306d0 con 0x7f1e6412cf60 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e6412cf60 0x7f1e6406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e6412cf60 0x7f1e6406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44598/0 (socket says 172.21.15.89:44598) 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e637fe700 1 -- 172.21.15.89:0/3533469791 learned_addr learned my addr 172.21.15.89:0/3533469791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e637fe700 1 -- 172.21.15.89:0/3533469791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e6412f8b0 msgr2=0x7f1e6412fd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e637fe700 1 --2- 172.21.15.89:0/3533469791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e6412f8b0 0x7f1e6412fd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e637fe700 1 -- 172.21.15.89:0/3533469791 >> v1:172.21.15.89:6789/0 conn(0x7f1e64130260 legacy=0x7f1e64136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.596+0000 7f1e637fe700 1 -- 172.21.15.89:0/3533469791 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e64071780 con 0x7f1e6412cf60 2023-09-30T22:18:43.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7f1e637fe700 1 --2- 172.21.15.89:0/3533469791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e6412cf60 0x7f1e6406f5d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400bbc0 tx=0x7f1e5400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a80b761e714eb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7faf83b3b700 1 -- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c136fe0 msgr2=0x7faf7c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7faf83b3b700 1 --2- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c136fe0 0x7faf7c1393c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf74009dc0 tx=0x7faf7400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7f1e627fc700 1 -- 172.21.15.89:0/3533469791 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400dc40 con 0x7f1e6412cf60 2023-09-30T22:18:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7faf83b3b700 1 -- 172.21.15.89:0/278930110 shutdown_connections 2023-09-30T22:18:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7faf83b3b700 1 --2- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c136fe0 0x7faf7c1393c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7faf83b3b700 1 --2- 172.21.15.89:0/278930110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c09f410 0x7faf7c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7faf83b3b700 1 -- 172.21.15.89:0/278930110 >> 172.21.15.89:0/278930110 conn(0x7faf7c0a4210 msgr2=0x7faf7c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7faf83b3b700 1 -- 172.21.15.89:0/278930110 shutdown_connections 2023-09-30T22:18:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7faf83b3b700 1 -- 172.21.15.89:0/278930110 wait complete. 2023-09-30T22:18:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7f1e627fc700 1 -- 172.21.15.89:0/3533469791 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5400dda0 con 0x7f1e6412cf60 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.597+0000 7f1e627fc700 1 -- 172.21.15.89:0/3533469791 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54004270 con 0x7f1e6412cf60 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7f1e69acb700 1 -- 172.21.15.89:0/3533469791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e6412cf60 msgr2=0x7f1e6406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7f1e69acb700 1 --2- 172.21.15.89:0/3533469791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e6412cf60 0x7f1e6406f5d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400bbc0 tx=0x7f1e5400bed0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7f1e69acb700 1 -- 172.21.15.89:0/3533469791 shutdown_connections 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7f1e69acb700 1 --2- 172.21.15.89:0/3533469791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e6412cf60 0x7f1e6406f5d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7f1e69acb700 1 --2- 172.21.15.89:0/3533469791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e6412f8b0 0x7f1e6412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7f1e69acb700 1 -- 172.21.15.89:0/3533469791 >> 172.21.15.89:0/3533469791 conn(0x7f1e640a2aa0 msgr2=0x7f1e640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7f1e69acb700 1 -- 172.21.15.89:0/3533469791 shutdown_connections 2023-09-30T22:18:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7f1e69acb700 1 -- 172.21.15.89:0/3533469791 wait complete. 2023-09-30T22:18:43.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf83b3b700 1 Processor -- start 2023-09-30T22:18:43.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf83b3b700 1 -- start start 2023-09-30T22:18:43.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf83b3b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c09e930 0x7faf7c11c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf83b3b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c09f410 0x7faf7c11cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf83b3b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c136fe0 0x7faf7c11d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf83b3b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c11fea0 con 0x7faf7c09e930 2023-09-30T22:18:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf83b3b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c071d20 con 0x7faf7c136fe0 2023-09-30T22:18:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf83b3b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c11e380 con 0x7faf7c09f410 2023-09-30T22:18:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf820d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c136fe0 0x7faf7c11d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf810d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c09f410 0x7faf7c11cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf820d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c136fe0 0x7faf7c11d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35164/0 (socket says 172.21.15.89:35164) 2023-09-30T22:18:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.598+0000 7faf820d8700 1 -- 172.21.15.89:0/2368275309 learned_addr learned my addr 172.21.15.89:0/2368275309 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7f1e69acb700 1 Processor -- start 2023-09-30T22:18:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7faf818d7700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c09e930 0x7faf7c11c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7faf820d8700 1 -- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c09f410 msgr2=0x7faf7c11cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7faf820d8700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c09f410 0x7faf7c11cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7faf820d8700 1 -- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c09e930 msgr2=0x7faf7c11c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7faf820d8700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c09e930 0x7faf7c11c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7faf820d8700 1 -- 172.21.15.89:0/2368275309 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf74009a70 con 0x7faf7c136fe0 2023-09-30T22:18:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7faf818d7700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c09e930 0x7faf7c11c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7faf820d8700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c136fe0 0x7faf7c11d0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faf74004000 tx=0x7faf74004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.599+0000 7f1e69acb700 1 -- start start 2023-09-30T22:18:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e69acb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6412cf60 0x7f1e6409bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e69acb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e6412f8b0 0x7f1e6409c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7faf727fc700 1 -- 172.21.15.89:0/2368275309 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7401eb50 con 0x7faf7c136fe0 2023-09-30T22:18:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7faf727fc700 1 -- 172.21.15.89:0/2368275309 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74022070 con 0x7faf7c136fe0 2023-09-30T22:18:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7faf727fc700 1 -- 172.21.15.89:0/2368275309 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf740040b0 con 0x7faf7c136fe0 2023-09-30T22:18:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e69acb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e64130260 0x7f1e641c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e69acb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6409e950 con 0x7f1e6412cf60 2023-09-30T22:18:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e69acb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6409d990 con 0x7f1e6412f8b0 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e69acb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e64119180 con 0x7f1e64130260 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6412cf60 0x7f1e6409bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e63fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e64130260 0x7f1e641c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6412cf60 0x7f1e6409bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45380/0 (socket says 172.21.15.89:45380) 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 -- 172.21.15.89:0/2875305591 learned_addr learned my addr 172.21.15.89:0/2875305591 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 -- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e64130260 msgr2=0x7f1e641c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 --2- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e64130260 0x7f1e641c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 -- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e6412f8b0 msgr2=0x7f1e6409c1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 --2- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e6412f8b0 0x7f1e6409c1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 -- 172.21.15.89:0/2875305591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5400b7f0 con 0x7f1e6412cf60 2023-09-30T22:18:43.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c09b220 con 0x7faf7c136fe0 2023-09-30T22:18:43.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c120910 con 0x7faf7c136fe0 2023-09-30T22:18:43.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.600+0000 7f1e637fe700 1 --2- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6412cf60 0x7f1e6409bc90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1e54019040 tx=0x7f1e54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:43.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.601+0000 7f1e4bfff700 1 -- 172.21.15.89:0/2875305591 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54009040 con 0x7f1e6412cf60 2023-09-30T22:18:43.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.601+0000 7f1e4bfff700 1 -- 172.21.15.89:0/2875305591 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e540045a0 con 0x7f1e6412cf60 2023-09-30T22:18:43.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.601+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6409ec30 con 0x7f1e6412cf60 2023-09-30T22:18:43.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.601+0000 7f1e4bfff700 1 -- 172.21.15.89:0/2875305591 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540206b0 con 0x7f1e6412cf60 2023-09-30T22:18:43.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.601+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6409cf00 con 0x7f1e6412cf60 2023-09-30T22:18:43.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.601+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c074ea0 con 0x7faf7c136fe0 2023-09-30T22:18:43.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.602+0000 7f1e4bfff700 1 -- 172.21.15.89:0/2875305591 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e54015070 con 0x7f1e6412cf60 2023-09-30T22:18:43.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.602+0000 7f1e4bfff700 1 -- 172.21.15.89:0/2875305591 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e54040f00 con 0x7f1e6412cf60 2023-09-30T22:18:43.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.602+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6409cf00 con 0x7f1e6412cf60 2023-09-30T22:18:43.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.603+0000 7faf727fc700 1 -- 172.21.15.89:0/2368275309 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf74004210 con 0x7faf7c136fe0 2023-09-30T22:18:43.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.603+0000 7faf727fc700 1 -- 172.21.15.89:0/2368275309 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faf74024070 con 0x7faf7c136fe0 2023-09-30T22:18:43.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.607+0000 7f1e4bfff700 1 -- 172.21.15.89:0/2875305591 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e6409cf00 con 0x7f1e6412cf60 2023-09-30T22:18:43.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.612+0000 7faf727fc700 1 -- 172.21.15.89:0/2368275309 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf74027970 con 0x7faf7c136fe0 2023-09-30T22:18:43.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.771+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f723413cb60 con 0x7f723409d6f0 2023-09-30T22:18:43.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.771+0000 7f7229ffb700 1 -- 172.21.15.89:0/685974546 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f723413cb60 con 0x7f723409d6f0 2023-09-30T22:18:43.771 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:43.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.774+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f723409d6f0 msgr2=0x7f7234139680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.774+0000 7f723b504700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f723409d6f0 0x7f7234139680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f723000e900 tx=0x7f723000ec10 comp rx=0 tx=0).stop 2023-09-30T22:18:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.774+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 shutdown_connections 2023-09-30T22:18:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.775+0000 7f723b504700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723409e410 0x7f723413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.775+0000 7f723b504700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f723409d6f0 0x7f7234139680 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.775+0000 7f723b504700 1 --2- 172.21.15.89:0/685974546 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f723409e000 0x7f7234139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.775+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 >> 172.21.15.89:0/685974546 conn(0x7f7234066f50 msgr2=0x7f72340a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.775+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 shutdown_connections 2023-09-30T22:18:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.775+0000 7f723b504700 1 -- 172.21.15.89:0/685974546 wait complete. 2023-09-30T22:18:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.781+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9340005590 con 0x7f9360135e50 2023-09-30T22:18:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.782+0000 7f9347fff700 1 -- 172.21.15.89:0/13035425 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9358012e60 con 0x7f9360135e50 2023-09-30T22:18:43.782 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:43.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9360135e50 msgr2=0x7f93601cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 --2- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9360135e50 0x7f93601cc430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f935800b9e0 tx=0x7f935800bda0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 shutdown_connections 2023-09-30T22:18:43.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 --2- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9360135e50 0x7f93601cc430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 --2- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f936009d6f0 0x7f9360127780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 --2- 172.21.15.89:0/13035425 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f936009e1d0 0x7f9360127cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 >> 172.21.15.89:0/13035425 conn(0x7f9360066f50 msgr2=0x7f9360070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:43.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 shutdown_connections 2023-09-30T22:18:43.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.785+0000 7f9365fc0700 1 -- 172.21.15.89:0/13035425 wait complete. 2023-09-30T22:18:43.790 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:43.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.803+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1e6409cf00 con 0x7f1e6412cf60 2023-09-30T22:18:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.804+0000 7f1e4bfff700 1 -- 172.21.15.89:0/2875305591 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e6409cf00 con 0x7f1e6412cf60 2023-09-30T22:18:43.804 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.805+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faf7c0a33f0 con 0x7faf7c136fe0 2023-09-30T22:18:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.805+0000 7faf727fc700 1 -- 172.21.15.89:0/2368275309 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf740323a0 con 0x7faf7c136fe0 2023-09-30T22:18:43.805 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6412cf60 msgr2=0x7f1e6409bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 --2- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6412cf60 0x7f1e6409bc90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1e54019040 tx=0x7f1e54004750 comp rx=0 tx=0).stop 2023-09-30T22:18:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 shutdown_connections 2023-09-30T22:18:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 --2- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e64130260 0x7f1e641c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 --2- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e6412cf60 0x7f1e6409bc90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 --2- 172.21.15.89:0/2875305591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e6412f8b0 0x7f1e6409c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 >> 172.21.15.89:0/2875305591 conn(0x7f1e640a2aa0 msgr2=0x7f1e641b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 shutdown_connections 2023-09-30T22:18:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.806+0000 7f1e69acb700 1 -- 172.21.15.89:0/2875305591 wait complete. 2023-09-30T22:18:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.807+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c136fe0 msgr2=0x7faf7c11d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.808+0000 7faf83b3b700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c136fe0 0x7faf7c11d0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faf74004000 tx=0x7faf74004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.808+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 shutdown_connections 2023-09-30T22:18:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.808+0000 7faf83b3b700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf7c136fe0 0x7faf7c11d0e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.808+0000 7faf83b3b700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf7c09e930 0x7faf7c11c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.808+0000 7faf83b3b700 1 --2- 172.21.15.89:0/2368275309 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf7c09f410 0x7faf7c11cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.808+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 >> 172.21.15.89:0/2368275309 conn(0x7faf7c0a4210 msgr2=0x7faf7c134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.808+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 shutdown_connections 2023-09-30T22:18:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:43.808+0000 7faf83b3b700 1 -- 172.21.15.89:0/2368275309 wait complete. 2023-09-30T22:18:43.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:43.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:44.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:44.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:44.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:44.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:44.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.947+0000 7fa85a306700 1 Processor -- start 2023-09-30T22:18:44.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.947+0000 7fa85a306700 1 -- start start 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa85a306700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409d6f0 0x7fa85409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa85a306700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 0x7fa85413bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa85a306700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85409d2f0 con 0x7fa85409e000 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa85a306700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85409a810 con 0x7fa85409d6f0 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa85a306700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa85409a470 con 0x7fa854134100 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa853fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409d6f0 0x7fa85409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa853fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409d6f0 0x7fa85409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35180/0 (socket says 172.21.15.89:35180) 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa853fff700 1 -- 172.21.15.89:0/2174109352 learned_addr learned my addr 172.21.15.89:0/2174109352 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:44.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa853fff700 1 -- 172.21.15.89:0/2174109352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 msgr2=0x7fa85413bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa853fff700 1 --2- 172.21.15.89:0/2174109352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 0x7fa85413bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.948+0000 7fa853fff700 1 -- 172.21.15.89:0/2174109352 >> v1:172.21.15.89:6789/0 conn(0x7fa85409e000 legacy=0x7fa8541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.949+0000 7fa853fff700 1 -- 172.21.15.89:0/2174109352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85409ec40 con 0x7fa85409d6f0 2023-09-30T22:18:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.949+0000 7fa853fff700 1 --2- 172.21.15.89:0/2174109352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409d6f0 0x7fa85409dac0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa844009920 tx=0x7fa844009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574f21a03b163a0d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.949+0000 7fa852ffd700 1 -- 172.21.15.89:0/2174109352 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400ba60 con 0x7fa85409d6f0 2023-09-30T22:18:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.949+0000 7fa852ffd700 1 -- 172.21.15.89:0/2174109352 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84400bbc0 con 0x7fa85409d6f0 2023-09-30T22:18:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.949+0000 7fa852ffd700 1 -- 172.21.15.89:0/2174109352 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844009e70 con 0x7fa85409d6f0 2023-09-30T22:18:44.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.950+0000 7fa85a306700 1 -- 172.21.15.89:0/2174109352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409d6f0 msgr2=0x7fa85409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.950+0000 7fa85a306700 1 --2- 172.21.15.89:0/2174109352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409d6f0 0x7fa85409dac0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa844009920 tx=0x7fa844009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:44.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.950+0000 7fa85a306700 1 -- 172.21.15.89:0/2174109352 shutdown_connections 2023-09-30T22:18:44.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.950+0000 7fa85a306700 1 --2- 172.21.15.89:0/2174109352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409d6f0 0x7fa85409dac0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.950+0000 7fa85a306700 1 --2- 172.21.15.89:0/2174109352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 0x7fa85413bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.950+0000 7fa85a306700 1 -- 172.21.15.89:0/2174109352 >> 172.21.15.89:0/2174109352 conn(0x7fa854066f50 msgr2=0x7fa85409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:44.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.950+0000 7fa85a306700 1 -- 172.21.15.89:0/2174109352 shutdown_connections 2023-09-30T22:18:44.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.950+0000 7fa85a306700 1 -- 172.21.15.89:0/2174109352 wait complete. 2023-09-30T22:18:44.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.951+0000 7fa85a306700 1 Processor -- start 2023-09-30T22:18:44.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.951+0000 7fa85a306700 1 -- start start 2023-09-30T22:18:44.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa85a306700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85409d6f0 0x7fa854122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa85a306700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409e000 0x7fa854122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa85a306700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 0x7fa854123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa85a306700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa854125fc0 con 0x7fa85409d6f0 2023-09-30T22:18:44.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa85a306700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa854133980 con 0x7fa85409e000 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa85a306700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa854133740 con 0x7fa854134100 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa8537fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409e000 0x7fa854122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa8537fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409e000 0x7fa854122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35192/0 (socket says 172.21.15.89:35192) 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa8537fe700 1 -- 172.21.15.89:0/2348351468 learned_addr learned my addr 172.21.15.89:0/2348351468 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa853fff700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85409d6f0 0x7fa854122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.952+0000 7fa8588a3700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 0x7fa854123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa8537fe700 1 -- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 msgr2=0x7fa854123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa8537fe700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 0x7fa854123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa8537fe700 1 -- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85409d6f0 msgr2=0x7fa854122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa8537fe700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85409d6f0 0x7fa854122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa8537fe700 1 -- 172.21.15.89:0/2348351468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa844009550 con 0x7fa85409e000 2023-09-30T22:18:44.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa853fff700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85409d6f0 0x7fa854122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:44.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa8537fe700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409e000 0x7fa854122cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c4b0 tx=0x7fa84800c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:44.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa850ff9700 1 -- 172.21.15.89:0/2348351468 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480052d0 con 0x7fa85409e000 2023-09-30T22:18:44.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa850ff9700 1 -- 172.21.15.89:0/2348351468 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa848005430 con 0x7fa85409e000 2023-09-30T22:18:44.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8541244a0 con 0x7fa85409e000 2023-09-30T22:18:44.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa850ff9700 1 -- 172.21.15.89:0/2348351468 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480038c0 con 0x7fa85409e000 2023-09-30T22:18:44.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.953+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa854133280 con 0x7fa85409e000 2023-09-30T22:18:44.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.954+0000 7fa850ff9700 1 -- 172.21.15.89:0/2348351468 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa848003a20 con 0x7fa85409e000 2023-09-30T22:18:44.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.954+0000 7fa850ff9700 1 -- 172.21.15.89:0/2348351468 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa84803eb40 con 0x7fa85409e000 2023-09-30T22:18:44.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.954+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa854126af0 con 0x7fa85409e000 2023-09-30T22:18:44.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.963+0000 7fa850ff9700 1 -- 172.21.15.89:0/2348351468 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa848017020 con 0x7fa85409e000 2023-09-30T22:18:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.977+0000 7fb9c5e32700 1 Processor -- start 2023-09-30T22:18:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.977+0000 7fb9c5e32700 1 -- start start 2023-09-30T22:18:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9c5e32700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9c009d6f0 0x7fb9c009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9c5e32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e410 0x7fb9c0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9c5e32700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c009f130 con 0x7fb9c009e000 2023-09-30T22:18:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9c5e32700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c009f520 con 0x7fb9c009d6f0 2023-09-30T22:18:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9c5e32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c009e930 con 0x7fb9c009e410 2023-09-30T22:18:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9bf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9c009d6f0 0x7fb9c009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9bf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9c009d6f0 0x7fb9c009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45398/0 (socket says 172.21.15.89:45398) 2023-09-30T22:18:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9bf7fe700 1 -- 172.21.15.89:0/1008022224 learned_addr learned my addr 172.21.15.89:0/1008022224 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9bffff700 1 --2- 172.21.15.89:0/1008022224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e410 0x7fb9c0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.978+0000 7fb9bf7fe700 1 -- 172.21.15.89:0/1008022224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e410 msgr2=0x7fb9c0130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:44.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.979+0000 7fb9bf7fe700 1 --2- 172.21.15.89:0/1008022224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e410 0x7fb9c0130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.979+0000 7fb9bf7fe700 1 -- 172.21.15.89:0/1008022224 >> v1:172.21.15.89:6789/0 conn(0x7fb9c009e000 legacy=0x7fb9c0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.979+0000 7fb9bf7fe700 1 -- 172.21.15.89:0/1008022224 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c009b2d0 con 0x7fb9c009d6f0 2023-09-30T22:18:44.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.979+0000 7fb9bf7fe700 1 --2- 172.21.15.89:0/1008022224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c009d6f0 0x7fb9c009dac0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00098f0 tx=0x7fb9b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4eb9d25bd895a4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:44.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.981+0000 7fb9be7fc700 1 -- 172.21.15.89:0/1008022224 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0011830 con 0x7fb9c009d6f0 2023-09-30T22:18:44.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.981+0000 7fb9be7fc700 1 -- 172.21.15.89:0/1008022224 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b0011990 con 0x7fb9c009d6f0 2023-09-30T22:18:44.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.981+0000 7fb9be7fc700 1 -- 172.21.15.89:0/1008022224 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0011d20 con 0x7fb9c009d6f0 2023-09-30T22:18:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.982+0000 7fbcf120e700 1 Processor -- start 2023-09-30T22:18:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.982+0000 7fbcf120e700 1 -- start start 2023-09-30T22:18:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcf120e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09d6f0 0x7fbcec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcf120e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbcec09e000 0x7fbcec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcf120e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec09b120 con 0x7fbcec09e410 2023-09-30T22:18:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcf120e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec06b460 con 0x7fbcec09e000 2023-09-30T22:18:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcf120e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec09a7e0 con 0x7fbcec09d6f0 2023-09-30T22:18:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcead9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09d6f0 0x7fbcec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcead9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09d6f0 0x7fbcec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44658/0 (socket says 172.21.15.89:44658) 2023-09-30T22:18:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbceb59e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbcec09e410 0x7fbcec133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53030/0 (socket says 172.21.15.89:53030) 2023-09-30T22:18:44.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcead9d700 1 -- 172.21.15.89:0/1268479970 learned_addr learned my addr 172.21.15.89:0/1268479970 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:44.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fb9c5e32700 1 -- 172.21.15.89:0/1008022224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c009d6f0 msgr2=0x7fb9c009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fb9c5e32700 1 --2- 172.21.15.89:0/1008022224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c009d6f0 0x7fb9c009dac0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00098f0 tx=0x7fb9b0009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:44.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcea59c700 1 --2- 172.21.15.89:0/1268479970 >> v2:172.21.15.89:3300/0 conn(0x7fbcec09e000 0x7fbcec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fb9c5e32700 1 -- 172.21.15.89:0/1008022224 shutdown_connections 2023-09-30T22:18:44.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fb9c5e32700 1 --2- 172.21.15.89:0/1008022224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c009d6f0 0x7fb9c009dac0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fb9c5e32700 1 --2- 172.21.15.89:0/1008022224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e410 0x7fb9c0130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fb9c5e32700 1 -- 172.21.15.89:0/1008022224 >> 172.21.15.89:0/1008022224 conn(0x7fb9c0066f50 msgr2=0x7fb9c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:44.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fb9c5e32700 1 -- 172.21.15.89:0/1008022224 shutdown_connections 2023-09-30T22:18:44.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fb9c5e32700 1 -- 172.21.15.89:0/1008022224 wait complete. 2023-09-30T22:18:44.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcea59c700 1 -- 172.21.15.89:0/1268479970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09d6f0 msgr2=0x7fbcec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:44.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcea59c700 1 --2- 172.21.15.89:0/1268479970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09d6f0 0x7fbcec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcea59c700 1 -- 172.21.15.89:0/1268479970 >> v1:172.21.15.89:6789/0 conn(0x7fbcec09e410 legacy=0x7fbcec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:44.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcea59c700 1 -- 172.21.15.89:0/1268479970 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec09a500 con 0x7fbcec09e000 2023-09-30T22:18:44.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbcea59c700 1 --2- 172.21.15.89:0/1268479970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09e000 0x7fbcec1339f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009840 tx=0x7fbce0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6184a31a8fcf616f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:44.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.983+0000 7fbce9d9b700 1 -- 172.21.15.89:0/1268479970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004230 con 0x7fbcec09e000 2023-09-30T22:18:44.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbce9d9b700 1 -- 172.21.15.89:0/1268479970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0004390 con 0x7fbcec09e000 2023-09-30T22:18:44.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbce9d9b700 1 -- 172.21.15.89:0/1268479970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00036a0 con 0x7fbcec09e000 2023-09-30T22:18:44.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbcf120e700 1 -- 172.21.15.89:0/1268479970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09e000 msgr2=0x7fbcec1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbcf120e700 1 --2- 172.21.15.89:0/1268479970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09e000 0x7fbcec1339f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009840 tx=0x7fbce0009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:44.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbcf120e700 1 -- 172.21.15.89:0/1268479970 shutdown_connections 2023-09-30T22:18:44.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbcf120e700 1 --2- 172.21.15.89:0/1268479970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09d6f0 0x7fbcec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbcf120e700 1 --2- 172.21.15.89:0/1268479970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09e000 0x7fbcec1339f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbcf120e700 1 -- 172.21.15.89:0/1268479970 >> 172.21.15.89:0/1268479970 conn(0x7fbcec066f50 msgr2=0x7fbcec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:44.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbcf120e700 1 -- 172.21.15.89:0/1268479970 shutdown_connections 2023-09-30T22:18:44.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fbcf120e700 1 -- 172.21.15.89:0/1268479970 wait complete. 2023-09-30T22:18:44.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.984+0000 7fb9c5e32700 1 Processor -- start 2023-09-30T22:18:44.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9c5e32700 1 -- start start 2023-09-30T22:18:44.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9c5e32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9c009d6f0 0x7fb9c011e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9c5e32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e000 0x7fb9c011e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9c5e32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c009e410 0x7fb9c011eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:44.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9c5e32700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0120170 con 0x7fb9c009e410 2023-09-30T22:18:44.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9c5e32700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c011f490 con 0x7fb9c009d6f0 2023-09-30T22:18:44.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9c5e32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01302f0 con 0x7fb9c009e000 2023-09-30T22:18:44.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9bf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9c009d6f0 0x7fb9c011e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:44.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9bf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9c009d6f0 0x7fb9c011e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35194/0 (socket says 172.21.15.89:35194) 2023-09-30T22:18:44.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.985+0000 7fb9bf7fe700 1 -- 172.21.15.89:0/3413787405 learned_addr learned my addr 172.21.15.89:0/3413787405 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:44.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9bf7fe700 1 -- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e000 msgr2=0x7fb9c011e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9bf7fe700 1 --2- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e000 0x7fb9c011e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9bf7fe700 1 -- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c009e410 msgr2=0x7fb9c011eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:44.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9bf7fe700 1 --2- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c009e410 0x7fb9c011eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:44.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9bf7fe700 1 -- 172.21.15.89:0/3413787405 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0009550 con 0x7fb9c009d6f0 2023-09-30T22:18:44.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fbcf120e700 1 Processor -- start 2023-09-30T22:18:44.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fbcf120e700 1 -- start start 2023-09-30T22:18:44.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9bf7fe700 1 --2- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9c009d6f0 0x7fb9c011e030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009c40 tx=0x7fb9b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:44.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9a7fff700 1 -- 172.21.15.89:0/3413787405 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00044b0 con 0x7fb9c009d6f0 2023-09-30T22:18:44.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9a7fff700 1 -- 172.21.15.89:0/3413787405 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b001f070 con 0x7fb9c009d6f0 2023-09-30T22:18:44.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c009c700 con 0x7fb9c009d6f0 2023-09-30T22:18:44.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9a7fff700 1 -- 172.21.15.89:0/3413787405 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00038a0 con 0x7fb9c009d6f0 2023-09-30T22:18:44.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.986+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c0120d70 con 0x7fb9c009d6f0 2023-09-30T22:18:44.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbcf120e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09d6f0 0x7fbcec122510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:45.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbcf120e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09e000 0x7fbcec122a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:45.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbcf120e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcec09e410 0x7fbcec122f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:45.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbcf120e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec1249a0 con 0x7fbcec09d6f0 2023-09-30T22:18:45.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbcf120e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec09c710 con 0x7fbcec09e410 2023-09-30T22:18:45.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbcf120e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0a2cb0 con 0x7fbcec09e000 2023-09-30T22:18:45.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbceb59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcec09e410 0x7fbcec122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbceb59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcec09e410 0x7fbcec122f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35200/0 (socket says 172.21.15.89:35200) 2023-09-30T22:18:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.987+0000 7fbceb59e700 1 -- 172.21.15.89:0/1303365580 learned_addr learned my addr 172.21.15.89:0/1303365580 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbcead9d700 1 --2- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09d6f0 0x7fbcec122510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbceb59e700 1 -- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09e000 msgr2=0x7fbcec122a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:18:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbceb59e700 1 --2- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09e000 0x7fbcec122a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbceb59e700 1 -- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09d6f0 msgr2=0x7fbcec122510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbceb59e700 1 --2- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09d6f0 0x7fbcec122510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbceb59e700 1 -- 172.21.15.89:0/1303365580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00094a0 con 0x7fbcec09e410 2023-09-30T22:18:45.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c011ac20 con 0x7fb9c009d6f0 2023-09-30T22:18:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbceb59e700 1 --2- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcec09e410 0x7fbcec122f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbce400cba0 tx=0x7fbce400ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbcd37fe700 1 -- 172.21.15.89:0/1303365580 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce400aad0 con 0x7fbcec09e410 2023-09-30T22:18:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbcd37fe700 1 -- 172.21.15.89:0/1303365580 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce400ac50 con 0x7fbcec09e410 2023-09-30T22:18:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec124ef0 con 0x7fbcec09e410 2023-09-30T22:18:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbcd37fe700 1 -- 172.21.15.89:0/1303365580 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4007070 con 0x7fbcec09e410 2023-09-30T22:18:45.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.988+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec125d00 con 0x7fbcec09e410 2023-09-30T22:18:45.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.989+0000 7fbcd37fe700 1 -- 172.21.15.89:0/1303365580 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbce40071d0 con 0x7fbcec09e410 2023-09-30T22:18:45.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.989+0000 7fbcd37fe700 1 -- 172.21.15.89:0/1303365580 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbce403f520 con 0x7fbcec09e410 2023-09-30T22:18:45.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.989+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec125880 con 0x7fbcec09e410 2023-09-30T22:18:45.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.991+0000 7fb9a7fff700 1 -- 172.21.15.89:0/3413787405 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb9b0021030 con 0x7fb9c009d6f0 2023-09-30T22:18:45.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.991+0000 7fb9a7fff700 1 -- 172.21.15.89:0/3413787405 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb9b0040b10 con 0x7fb9c009d6f0 2023-09-30T22:18:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.992+0000 7fe174296700 1 Processor -- start 2023-09-30T22:18:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.992+0000 7fe174296700 1 -- start start 2023-09-30T22:18:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.993+0000 7fe174296700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c0a09c0 0x7fe16c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.993+0000 7fe174296700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe16c0a12d0 0x7fe16c134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.993+0000 7fe174296700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c09ea10 con 0x7fe16c09d5c0 2023-09-30T22:18:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.993+0000 7fe174296700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c09e210 con 0x7fe16c0a12d0 2023-09-30T22:18:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.993+0000 7fe174296700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0a26f0 con 0x7fe16c0a09c0 2023-09-30T22:18:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172032700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c0a09c0 0x7fe16c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172032700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c0a09c0 0x7fe16c0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44680/0 (socket says 172.21.15.89:44680) 2023-09-30T22:18:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172032700 1 -- 172.21.15.89:0/1313861973 learned_addr learned my addr 172.21.15.89:0/1313861973 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172833700 1 --2- 172.21.15.89:0/1313861973 >> v2:172.21.15.89:3300/0 conn(0x7fe16c0a12d0 0x7fe16c134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172032700 1 -- 172.21.15.89:0/1313861973 >> v2:172.21.15.89:3300/0 conn(0x7fe16c0a12d0 msgr2=0x7fe16c134030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172032700 1 --2- 172.21.15.89:0/1313861973 >> v2:172.21.15.89:3300/0 conn(0x7fe16c0a12d0 0x7fe16c134030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172032700 1 -- 172.21.15.89:0/1313861973 >> v1:172.21.15.89:6789/0 conn(0x7fe16c09d5c0 legacy=0x7fe16c133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172032700 1 -- 172.21.15.89:0/1313861973 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c09db70 con 0x7fe16c0a09c0 2023-09-30T22:18:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172833700 1 --2- 172.21.15.89:0/1313861973 >> v2:172.21.15.89:3300/0 conn(0x7fe16c0a12d0 0x7fe16c134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe172032700 1 --2- 172.21.15.89:0/1313861973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c0a09c0 0x7fe16c0a0d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098f0 tx=0x7fe168009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=672effd8e083702a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe171030700 1 -- 172.21.15.89:0/1313861973 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168011830 con 0x7fe16c0a09c0 2023-09-30T22:18:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe171030700 1 -- 172.21.15.89:0/1313861973 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168011990 con 0x7fe16c0a09c0 2023-09-30T22:18:45.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.994+0000 7fe171030700 1 -- 172.21.15.89:0/1313861973 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168011d20 con 0x7fe16c0a09c0 2023-09-30T22:18:45.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fe174296700 1 -- 172.21.15.89:0/1313861973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c0a09c0 msgr2=0x7fe16c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fe174296700 1 --2- 172.21.15.89:0/1313861973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c0a09c0 0x7fe16c0a0d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098f0 tx=0x7fe168009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:45.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fbcd37fe700 1 -- 172.21.15.89:0/1303365580 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbce4017080 con 0x7fbcec09e410 2023-09-30T22:18:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fe174296700 1 -- 172.21.15.89:0/1313861973 shutdown_connections 2023-09-30T22:18:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fe174296700 1 --2- 172.21.15.89:0/1313861973 >> v2:172.21.15.89:3300/0 conn(0x7fe16c0a12d0 0x7fe16c134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fe174296700 1 --2- 172.21.15.89:0/1313861973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c0a09c0 0x7fe16c0a0d90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fe174296700 1 -- 172.21.15.89:0/1313861973 >> 172.21.15.89:0/1313861973 conn(0x7fe16c0a2fe0 msgr2=0x7fe16c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fe174296700 1 -- 172.21.15.89:0/1313861973 shutdown_connections 2023-09-30T22:18:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.995+0000 7fe174296700 1 -- 172.21.15.89:0/1313861973 wait complete. 2023-09-30T22:18:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.996+0000 7fe174296700 1 Processor -- start 2023-09-30T22:18:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.996+0000 7fe174296700 1 -- start start 2023-09-30T22:18:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe174296700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c09d5c0 0x7fe16c122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe174296700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16c0a09c0 0x7fe16c122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe174296700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16c0a12d0 0x7fe16c122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe174296700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c125b60 con 0x7fe16c0a12d0 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe174296700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c125930 con 0x7fe16c0a09c0 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe174296700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c124dc0 con 0x7fe16c09d5c0 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe172032700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c09d5c0 0x7fe16c122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe172032700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c09d5c0 0x7fe16c122320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44690/0 (socket says 172.21.15.89:44690) 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe172032700 1 -- 172.21.15.89:0/1825345085 learned_addr learned my addr 172.21.15.89:0/1825345085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.997+0000 7fe172833700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16c0a12d0 0x7fe16c122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe172032700 1 -- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16c0a09c0 msgr2=0x7fe16c122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe172032700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16c0a09c0 0x7fe16c122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe172032700 1 -- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16c0a12d0 msgr2=0x7fe16c122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe172032700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16c0a12d0 0x7fe16c122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe172032700 1 -- 172.21.15.89:0/1825345085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe164009a70 con 0x7fe16c09d5c0 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe172833700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16c0a12d0 0x7fe16c122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe172032700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c09d5c0 0x7fe16c122320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe168009c40 tx=0x7fe168004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe162ffd700 1 -- 172.21.15.89:0/1825345085 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168004530 con 0x7fe16c09d5c0 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe162ffd700 1 -- 172.21.15.89:0/1825345085 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16801f070 con 0x7fe16c09d5c0 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe168009550 con 0x7fe16c09d5c0 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe162ffd700 1 -- 172.21.15.89:0/1825345085 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1680038c0 con 0x7fe16c09d5c0 2023-09-30T22:18:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.998+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c09cd10 con 0x7fe16c09d5c0 2023-09-30T22:18:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.999+0000 7fe162ffd700 1 -- 172.21.15.89:0/1825345085 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe168021030 con 0x7fe16c09d5c0 2023-09-30T22:18:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.999+0000 7fe162ffd700 1 -- 172.21.15.89:0/1825345085 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe168040c30 con 0x7fe16c09d5c0 2023-09-30T22:18:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:44.999+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c11f670 con 0x7fe16c09d5c0 2023-09-30T22:18:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.001+0000 7fb9a7fff700 1 -- 172.21.15.89:0/3413787405 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb9b00199a0 con 0x7fb9c009d6f0 2023-09-30T22:18:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.008+0000 7fe162ffd700 1 -- 172.21.15.89:0/1825345085 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe168015070 con 0x7fe16c09d5c0 2023-09-30T22:18:45.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.187+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa854074ea0 con 0x7fa85409e000 2023-09-30T22:18:45.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.187+0000 7fa850ff9700 1 -- 172.21.15.89:0/2348351468 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa848019e60 con 0x7fa85409e000 2023-09-30T22:18:45.187 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:45.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.189+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409e000 msgr2=0x7fa854122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fa85a306700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409e000 0x7fa854122cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c4b0 tx=0x7fa84800c7c0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 shutdown_connections 2023-09-30T22:18:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fa85a306700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa854134100 0x7fa854123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fa85a306700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85409d6f0 0x7fa854122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fa85a306700 1 --2- 172.21.15.89:0/2348351468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85409e000 0x7fa854122cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 >> 172.21.15.89:0/2348351468 conn(0x7fa854066f50 msgr2=0x7fa854070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 shutdown_connections 2023-09-30T22:18:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fa85a306700 1 -- 172.21.15.89:0/2348351468 wait complete. 2023-09-30T22:18:45.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.190+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbcec074ea0 con 0x7fbcec09e410 2023-09-30T22:18:45.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.191+0000 7fbcd37fe700 1 -- 172.21.15.89:0/1303365580 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbce40198b0 con 0x7fbcec09e410 2023-09-30T22:18:45.191 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.194+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcec09e410 msgr2=0x7fbcec122f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.194+0000 7fbcf120e700 1 --2- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcec09e410 0x7fbcec122f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbce400cba0 tx=0x7fbce400ceb0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.194+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 shutdown_connections 2023-09-30T22:18:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.194+0000 7fbcf120e700 1 --2- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcec09e410 0x7fbcec122f90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.194+0000 7fbcf120e700 1 --2- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcec09d6f0 0x7fbcec122510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.194+0000 7fbcf120e700 1 --2- 172.21.15.89:0/1303365580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcec09e000 0x7fbcec122a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.194+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 >> 172.21.15.89:0/1303365580 conn(0x7fbcec066f50 msgr2=0x7fbcec070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.195+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 shutdown_connections 2023-09-30T22:18:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.195+0000 7fbcf120e700 1 -- 172.21.15.89:0/1303365580 wait complete. 2023-09-30T22:18:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.195+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb9c0074ea0 con 0x7fb9c009d6f0 2023-09-30T22:18:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.195+0000 7fb9a7fff700 1 -- 172.21.15.89:0/3413787405 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb9b0015020 con 0x7fb9c009d6f0 2023-09-30T22:18:45.195 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9c009d6f0 msgr2=0x7fb9c011e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 --2- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9c009d6f0 0x7fb9c011e030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009c40 tx=0x7fb9b0004750 comp rx=0 tx=0).stop 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 shutdown_connections 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 --2- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c009e410 0x7fb9c011eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 --2- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9c009d6f0 0x7fb9c011e030 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 --2- 172.21.15.89:0/3413787405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9c009e000 0x7fb9c011e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 >> 172.21.15.89:0/3413787405 conn(0x7fb9c0066f50 msgr2=0x7fb9c0130ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 shutdown_connections 2023-09-30T22:18:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.198+0000 7fb9c5e32700 1 -- 172.21.15.89:0/3413787405 wait complete. 2023-09-30T22:18:45.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:45.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:45.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:45.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.220+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe16c0747f0 con 0x7fe16c09d5c0 2023-09-30T22:18:45.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.220+0000 7fe162ffd700 1 -- 172.21.15.89:0/1825345085 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe16801a7d0 con 0x7fe16c09d5c0 2023-09-30T22:18:45.220 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:45.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c09d5c0 msgr2=0x7fe16c122320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:45.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c09d5c0 0x7fe16c122320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe168009c40 tx=0x7fe168004750 comp rx=0 tx=0).stop 2023-09-30T22:18:45.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 shutdown_connections 2023-09-30T22:18:45.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe16c0a12d0 0x7fe16c122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe16c09d5c0 0x7fe16c122320 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 --2- 172.21.15.89:0/1825345085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe16c0a09c0 0x7fe16c122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:45.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 >> 172.21.15.89:0/1825345085 conn(0x7fe16c0a2fe0 msgr2=0x7fe16c11c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:45.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 shutdown_connections 2023-09-30T22:18:45.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:45.222+0000 7fe174296700 1 -- 172.21.15.89:0/1825345085 wait complete. 2023-09-30T22:18:45.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:46.209 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:46.212 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:46.213 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:46.236 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:46.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.354+0000 7f2464c64700 1 Processor -- start 2023-09-30T22:18:46.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.354+0000 7f8e214b9700 1 Processor -- start 2023-09-30T22:18:46.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.354+0000 7f8e214b9700 1 -- start start 2023-09-30T22:18:46.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f8e214b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.354+0000 7f2464c64700 1 -- start start 2023-09-30T22:18:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f8e214b9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e1c09e930 0x7f8e1c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f8e214b9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c06ef30 con 0x7f8e1c09de50 2023-09-30T22:18:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f8e214b9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c06fef0 con 0x7f8e1c09e930 2023-09-30T22:18:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f8e214b9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c1334b0 con 0x7f8e1c1338a0 2023-09-30T22:18:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f2464c64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f246009d6f0 0x7f246009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f2464c64700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f246009e410 0x7f2460130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f2464c64700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246009f130 con 0x7f246009e000 2023-09-30T22:18:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f2464c64700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246009f520 con 0x7f246009e410 2023-09-30T22:18:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f2464c64700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246009e930 con 0x7f246009d6f0 2023-09-30T22:18:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f8e1a7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8e1c09de50 0x7f8e1c09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53046/0 (socket says 172.21.15.89:53046) 2023-09-30T22:18:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f8e1a7fc700 1 -- 172.21.15.89:0/2473415660 learned_addr learned my addr 172.21.15.89:0/2473415660 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.355+0000 7f8e1affd700 1 --2- 172.21.15.89:0/2473415660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f246009e410 0x7f2460130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f246009e410 0x7f2460130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45470/0 (socket says 172.21.15.89:45470) 2023-09-30T22:18:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245ed9d700 1 -- 172.21.15.89:0/3353792452 learned_addr learned my addr 172.21.15.89:0/3353792452 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e1b7fe700 1 --2- 172.21.15.89:0/2473415660 >> v2:172.21.15.89:3300/0 conn(0x7f8e1c09e930 0x7f8e1c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e1affd700 1 -- 172.21.15.89:0/2473415660 >> v2:172.21.15.89:3300/0 conn(0x7f8e1c09e930 msgr2=0x7f8e1c13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e1affd700 1 --2- 172.21.15.89:0/2473415660 >> v2:172.21.15.89:3300/0 conn(0x7f8e1c09e930 0x7f8e1c13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e1affd700 1 -- 172.21.15.89:0/2473415660 >> v1:172.21.15.89:6789/0 conn(0x7f8e1c09de50 legacy=0x7f8e1c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e1affd700 1 -- 172.21.15.89:0/2473415660 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c06fc30 con 0x7f8e1c1338a0 2023-09-30T22:18:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245ed9d700 1 -- 172.21.15.89:0/3353792452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f246009d6f0 msgr2=0x7f246009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245ed9d700 1 --2- 172.21.15.89:0/3353792452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f246009d6f0 0x7f246009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245ed9d700 1 -- 172.21.15.89:0/3353792452 >> v1:172.21.15.89:6789/0 conn(0x7f246009e000 legacy=0x7f2460130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245ed9d700 1 -- 172.21.15.89:0/3353792452 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246009b2d0 con 0x7f246009e410 2023-09-30T22:18:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e1affd700 1 --2- 172.21.15.89:0/2473415660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c133c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009840 tx=0x7f8e14009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c18c9e82e7e704f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e19ffb700 1 -- 172.21.15.89:0/2473415660 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14011830 con 0x7f8e1c1338a0 2023-09-30T22:18:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e19ffb700 1 -- 172.21.15.89:0/2473415660 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e14011990 con 0x7f8e1c1338a0 2023-09-30T22:18:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f8e19ffb700 1 -- 172.21.15.89:0/2473415660 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14011d20 con 0x7f8e1c1338a0 2023-09-30T22:18:46.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245ed9d700 1 --2- 172.21.15.89:0/3353792452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 0x7f2460130d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f245800d150 tx=0x7f245800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6695fcf4689363 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:46.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.356+0000 7f245d59a700 1 -- 172.21.15.89:0/3353792452 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458010070 con 0x7f246009e410 2023-09-30T22:18:46.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f245d59a700 1 -- 172.21.15.89:0/3353792452 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245800dd80 con 0x7f246009e410 2023-09-30T22:18:46.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f245d59a700 1 -- 172.21.15.89:0/3353792452 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245800b310 con 0x7f246009e410 2023-09-30T22:18:46.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f8e214b9700 1 -- 172.21.15.89:0/2473415660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 msgr2=0x7f8e1c133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f8e214b9700 1 --2- 172.21.15.89:0/2473415660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c133c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009840 tx=0x7f8e14009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:46.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f8e214b9700 1 -- 172.21.15.89:0/2473415660 shutdown_connections 2023-09-30T22:18:46.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f8e214b9700 1 --2- 172.21.15.89:0/2473415660 >> v2:172.21.15.89:3300/0 conn(0x7f8e1c09e930 0x7f8e1c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f8e214b9700 1 --2- 172.21.15.89:0/2473415660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c133c70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f8e214b9700 1 -- 172.21.15.89:0/2473415660 >> 172.21.15.89:0/2473415660 conn(0x7f8e1c0a2aa0 msgr2=0x7f8e1c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:46.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f8e214b9700 1 -- 172.21.15.89:0/2473415660 shutdown_connections 2023-09-30T22:18:46.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f8e214b9700 1 -- 172.21.15.89:0/2473415660 wait complete. 2023-09-30T22:18:46.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f2464c64700 1 -- 172.21.15.89:0/3353792452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 msgr2=0x7f2460130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.357+0000 7f2464c64700 1 --2- 172.21.15.89:0/3353792452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 0x7f2460130d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f245800d150 tx=0x7f245800d460 comp rx=0 tx=0).stop 2023-09-30T22:18:46.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.358+0000 7f8e214b9700 1 Processor -- start 2023-09-30T22:18:46.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.358+0000 7f8e214b9700 1 -- start start 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e214b9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e1c09de50 0x7f8e1c11eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e214b9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e1c09e930 0x7f8e1c11f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e214b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c11f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e214b9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c121cf0 con 0x7f8e1c09de50 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e214b9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c12f6a0 con 0x7f8e1c09e930 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e214b9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c120700 con 0x7f8e1c1338a0 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e1c09de50 0x7f8e1c11eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c11f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e1c09de50 0x7f8e1c11eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45484/0 (socket says 172.21.15.89:45484) 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1affd700 1 -- 172.21.15.89:0/2545477040 learned_addr learned my addr 172.21.15.89:0/2545477040 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:46.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1b7fe700 1 -- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e1c09e930 msgr2=0x7f8e1c11f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1b7fe700 1 --2- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e1c09e930 0x7f8e1c11f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1b7fe700 1 -- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e1c09de50 msgr2=0x7f8e1c11eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1b7fe700 1 --2- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e1c09de50 0x7f8e1c11eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1b7fe700 1 -- 172.21.15.89:0/2545477040 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e140094a0 con 0x7f8e1c1338a0 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1affd700 1 --2- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e1c09de50 0x7f8e1c11eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.359+0000 7f8e1b7fe700 1 --2- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c11f570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c007dc0 tx=0x7f8e0c00e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f8e037fe700 1 -- 172.21.15.89:0/2545477040 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c018070 con 0x7f8e1c1338a0 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c0a10d0 con 0x7f8e1c1338a0 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c122660 con 0x7f8e1c1338a0 2023-09-30T22:18:46.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f8e037fe700 1 -- 172.21.15.89:0/2545477040 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c009950 con 0x7f8e1c1338a0 2023-09-30T22:18:46.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f8e037fe700 1 -- 172.21.15.89:0/2545477040 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00ede0 con 0x7f8e1c1338a0 2023-09-30T22:18:46.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f2464c64700 1 -- 172.21.15.89:0/3353792452 shutdown_connections 2023-09-30T22:18:46.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f2464c64700 1 --2- 172.21.15.89:0/3353792452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 0x7f2460130d70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f2464c64700 1 --2- 172.21.15.89:0/3353792452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f246009d6f0 0x7f246009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f2464c64700 1 -- 172.21.15.89:0/3353792452 >> 172.21.15.89:0/3353792452 conn(0x7f2460066f50 msgr2=0x7f246009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:46.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f8e037fe700 1 -- 172.21.15.89:0/2545477040 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e0c003a20 con 0x7f8e1c1338a0 2023-09-30T22:18:46.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.360+0000 7f8e037fe700 1 -- 172.21.15.89:0/2545477040 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e0c03f4a0 con 0x7f8e1c1338a0 2023-09-30T22:18:46.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.361+0000 7f2464c64700 1 -- 172.21.15.89:0/3353792452 shutdown_connections 2023-09-30T22:18:46.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.361+0000 7f2464c64700 1 -- 172.21.15.89:0/3353792452 wait complete. 2023-09-30T22:18:46.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.361+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc0052e0 con 0x7f8e1c1338a0 2023-09-30T22:18:46.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f2464c64700 1 Processor -- start 2023-09-30T22:18:46.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f2464c64700 1 -- start start 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f2464c64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f246009d6f0 0x7f246011d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f2464c64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f246009e000 0x7f246011d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f2464c64700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 0x7f246011dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f2464c64700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246011e620 con 0x7f246009e410 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f2464c64700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246011f550 con 0x7f246009e000 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f2464c64700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2460120c00 con 0x7f246009d6f0 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f245e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f246009d6f0 0x7f246011d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f245e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f246009d6f0 0x7f246011d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44716/0 (socket says 172.21.15.89:44716) 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f245ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 0x7f246011dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.362+0000 7f245e59c700 1 -- 172.21.15.89:0/528945028 learned_addr learned my addr 172.21.15.89:0/528945028 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.363+0000 7f245e59c700 1 -- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f246009e000 msgr2=0x7f246011d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.363+0000 7f245e59c700 1 --2- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f246009e000 0x7f246011d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.363+0000 7f245e59c700 1 -- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 msgr2=0x7f246011dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.363+0000 7f245e59c700 1 --2- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 0x7f246011dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.363+0000 7f245e59c700 1 -- 172.21.15.89:0/528945028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245800bd40 con 0x7f246009d6f0 2023-09-30T22:18:46.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.363+0000 7f245e59c700 1 --2- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f246009d6f0 0x7f246011d1c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f245000b560 tx=0x7f245000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f21076700 1 Processor -- start 2023-09-30T22:18:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f21076700 1 -- start start 2023-09-30T22:18:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f21076700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133420 0x7f9f1c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f21076700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133e30 0x7f9f1c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f21076700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f1c134260 0x7f9f1c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f21076700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c06b990 con 0x7f9f1c134260 2023-09-30T22:18:46.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f21076700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0a0590 con 0x7f9f1c133e30 2023-09-30T22:18:46.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f21076700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c09fb80 con 0x7f9f1c133420 2023-09-30T22:18:46.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f1ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133420 0x7f9f1c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f1ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133420 0x7f9f1c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44718/0 (socket says 172.21.15.89:44718) 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.364+0000 7f9f1ad9d700 1 -- 172.21.15.89:0/1003982036 learned_addr learned my addr 172.21.15.89:0/1003982036 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.365+0000 7f9f1b59e700 1 --2- 172.21.15.89:0/1003982036 >> v2:172.21.15.89:3300/0 conn(0x7f9f1c134260 0x7f9f1c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.365+0000 7f9f1a59c700 1 --2- 172.21.15.89:0/1003982036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133e30 0x7f9f1c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.365+0000 7f9f1ad9d700 1 -- 172.21.15.89:0/1003982036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133e30 msgr2=0x7f9f1c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.365+0000 7f9f1ad9d700 1 --2- 172.21.15.89:0/1003982036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133e30 0x7f9f1c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.365+0000 7f9f1ad9d700 1 -- 172.21.15.89:0/1003982036 >> v2:172.21.15.89:3300/0 conn(0x7f9f1c134260 msgr2=0x7f9f1c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.365+0000 7f9f1ad9d700 1 --2- 172.21.15.89:0/1003982036 >> v2:172.21.15.89:3300/0 conn(0x7f9f1c134260 0x7f9f1c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.365+0000 7f9f1ad9d700 1 -- 172.21.15.89:0/1003982036 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0a0030 con 0x7f9f1c133420 2023-09-30T22:18:46.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.365+0000 7f9f1ad9d700 1 --2- 172.21.15.89:0/1003982036 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133420 0x7f9f1c1337f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0131e0 tx=0x7f9f0c0134f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fadacd5b7a590266 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:46.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.366+0000 7f9f19d9b700 1 -- 172.21.15.89:0/1003982036 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c016070 con 0x7f9f1c133420 2023-09-30T22:18:46.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.366+0000 7f9f19d9b700 1 -- 172.21.15.89:0/1003982036 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c013e30 con 0x7f9f1c133420 2023-09-30T22:18:46.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.366+0000 7f9f19d9b700 1 -- 172.21.15.89:0/1003982036 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c0112f0 con 0x7f9f1c133420 2023-09-30T22:18:46.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.366+0000 7f8e037fe700 1 -- 172.21.15.89:0/2545477040 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e0c01c370 con 0x7f8e1c1338a0 2023-09-30T22:18:46.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.366+0000 7f244effd700 1 -- 172.21.15.89:0/528945028 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450011830 con 0x7f246009d6f0 2023-09-30T22:18:46.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.366+0000 7f244effd700 1 -- 172.21.15.89:0/528945028 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2450011990 con 0x7f246009d6f0 2023-09-30T22:18:46.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.366+0000 7f244effd700 1 -- 172.21.15.89:0/528945028 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450011d40 con 0x7f246009d6f0 2023-09-30T22:18:46.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246011f2a0 con 0x7f246009d6f0 2023-09-30T22:18:46.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24600a1e80 con 0x7f246009d6f0 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 -- 172.21.15.89:0/1003982036 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133420 msgr2=0x7f9f1c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 --2- 172.21.15.89:0/1003982036 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133420 0x7f9f1c1337f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0131e0 tx=0x7f9f0c0134f0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 -- 172.21.15.89:0/1003982036 shutdown_connections 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 --2- 172.21.15.89:0/1003982036 >> v2:172.21.15.89:3300/0 conn(0x7f9f1c134260 0x7f9f1c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 --2- 172.21.15.89:0/1003982036 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133420 0x7f9f1c1337f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 --2- 172.21.15.89:0/1003982036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133e30 0x7f9f1c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 -- 172.21.15.89:0/1003982036 >> 172.21.15.89:0/1003982036 conn(0x7f9f1c0a4210 msgr2=0x7f9f1c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 -- 172.21.15.89:0/1003982036 shutdown_connections 2023-09-30T22:18:46.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.367+0000 7f9f21076700 1 -- 172.21.15.89:0/1003982036 wait complete. 2023-09-30T22:18:46.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.368+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24601204b0 con 0x7f246009d6f0 2023-09-30T22:18:46.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.369+0000 7f9f21076700 1 Processor -- start 2023-09-30T22:18:46.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.369+0000 7f9f21076700 1 -- start start 2023-09-30T22:18:46.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.369+0000 7f9f21076700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133420 0x7f9f1c09be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.369+0000 7f9f21076700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133e30 0x7f9f1c09c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.369+0000 7f9f21076700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c134260 0x7f9f1c09c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.369+0000 7f9f21076700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c11b4c0 con 0x7f9f1c134260 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.369+0000 7f9f21076700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c12d680 con 0x7f9f1c133420 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f21076700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c131490 con 0x7f9f1c133e30 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c134260 0x7f9f1c09c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c134260 0x7f9f1c09c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45502/0 (socket says 172.21.15.89:45502) 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 -- 172.21.15.89:0/550128445 learned_addr learned my addr 172.21.15.89:0/550128445 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1ad9d700 1 --2- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133420 0x7f9f1c09be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 -- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133e30 msgr2=0x7f9f1c09c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 --2- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133e30 0x7f9f1c09c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 -- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133420 msgr2=0x7f9f1c09be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 --2- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133420 0x7f9f1c09be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 -- 172.21.15.89:0/550128445 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c011df0 con 0x7f9f1c134260 2023-09-30T22:18:46.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f9f1b59e700 1 --2- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c134260 0x7f9f1c09c8e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9f14007e40 tx=0x7f9f1400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:46.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.370+0000 7f244effd700 1 -- 172.21.15.89:0/528945028 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f245001a400 con 0x7f246009d6f0 2023-09-30T22:18:46.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.371+0000 7f244effd700 1 -- 172.21.15.89:0/528945028 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f245003fd80 con 0x7f246009d6f0 2023-09-30T22:18:46.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.371+0000 7f9f037fe700 1 -- 172.21.15.89:0/550128445 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400ee40 con 0x7f9f1c134260 2023-09-30T22:18:46.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.371+0000 7f9f037fe700 1 -- 172.21.15.89:0/550128445 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14014420 con 0x7f9f1c134260 2023-09-30T22:18:46.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.371+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c09da80 con 0x7f9f1c134260 2023-09-30T22:18:46.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.371+0000 7f9f037fe700 1 -- 172.21.15.89:0/550128445 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f140147d0 con 0x7f9f1c134260 2023-09-30T22:18:46.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.371+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c09e1d0 con 0x7f9f1c134260 2023-09-30T22:18:46.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.372+0000 7f9f037fe700 1 -- 172.21.15.89:0/550128445 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f14014930 con 0x7f9f1c134260 2023-09-30T22:18:46.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.372+0000 7f9f037fe700 1 -- 172.21.15.89:0/550128445 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9f1403ff30 con 0x7f9f1c134260 2023-09-30T22:18:46.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.376+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9efc0052e0 con 0x7f9f1c134260 2023-09-30T22:18:46.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.378+0000 7f244effd700 1 -- 172.21.15.89:0/528945028 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f245001ae60 con 0x7f246009d6f0 2023-09-30T22:18:46.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.381+0000 7f9f037fe700 1 -- 172.21.15.89:0/550128445 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f140122e0 con 0x7f9f1c134260 2023-09-30T22:18:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.405+0000 7fd269a66700 1 Processor -- start 2023-09-30T22:18:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.405+0000 7fd269a66700 1 -- start start 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd269a66700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd264135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd269a66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd264138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd269a66700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26409f130 con 0x7fd26409d6f0 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd269a66700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26409f520 con 0x7fd26409e1d0 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd269a66700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd26409e930 con 0x7fd264135e50 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd263fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd264138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd263fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd264138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44738/0 (socket says 172.21.15.89:44738) 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd263fff700 1 -- 172.21.15.89:0/3570982700 learned_addr learned my addr 172.21.15.89:0/3570982700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd262ffd700 1 --2- 172.21.15.89:0/3570982700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd264135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.406+0000 7fd2627fc700 1 -- 172.21.15.89:0/3570982700 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298954339 0 0) 0x7fd26409f130 con 0x7fd26409d6f0 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd2627fc700 1 -- 172.21.15.89:0/3570982700 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd24c0035b0 con 0x7fd26409d6f0 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd263fff700 1 -- 172.21.15.89:0/3570982700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 msgr2=0x7fd264135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd263fff700 1 --2- 172.21.15.89:0/3570982700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd264135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd263fff700 1 -- 172.21.15.89:0/3570982700 >> v1:172.21.15.89:6789/0 conn(0x7fd26409d6f0 legacy=0x7fd26409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd263fff700 1 -- 172.21.15.89:0/3570982700 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26409b2d0 con 0x7fd264135e50 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd263fff700 1 --2- 172.21.15.89:0/3570982700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd264138230 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00d180 tx=0x7fd25c00d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e4939a619abf2e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd2627fc700 1 -- 172.21.15.89:0/3570982700 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c010070 con 0x7fd264135e50 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd2627fc700 1 -- 172.21.15.89:0/3570982700 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c00dd40 con 0x7fd264135e50 2023-09-30T22:18:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.407+0000 7fd2627fc700 1 -- 172.21.15.89:0/3570982700 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00b310 con 0x7fd264135e50 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.408+0000 7fd269a66700 1 -- 172.21.15.89:0/3570982700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 msgr2=0x7fd264138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.408+0000 7fd269a66700 1 --2- 172.21.15.89:0/3570982700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd264138230 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00d180 tx=0x7fd25c00d490 comp rx=0 tx=0).stop 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.408+0000 7fd269a66700 1 -- 172.21.15.89:0/3570982700 shutdown_connections 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.408+0000 7fd269a66700 1 --2- 172.21.15.89:0/3570982700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd264138230 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.408+0000 7fd269a66700 1 --2- 172.21.15.89:0/3570982700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd264135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.408+0000 7fd269a66700 1 -- 172.21.15.89:0/3570982700 >> 172.21.15.89:0/3570982700 conn(0x7fd264066f50 msgr2=0x7fd26409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.408+0000 7fd269a66700 1 -- 172.21.15.89:0/3570982700 shutdown_connections 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.408+0000 7fd269a66700 1 -- 172.21.15.89:0/3570982700 wait complete. 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.409+0000 7fd269a66700 1 Processor -- start 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.409+0000 7fd269a66700 1 -- start start 2023-09-30T22:18:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd269a66700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd2641c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd269a66700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd2641c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd269a66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd2641c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd269a66700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264120db0 con 0x7fd26409d6f0 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd269a66700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26411f730 con 0x7fd26409e1d0 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd269a66700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640a2090 con 0x7fd264135e50 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd2641c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd2641c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45508/0 (socket says 172.21.15.89:45508) 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 -- 172.21.15.89:0/1211112908 learned_addr learned my addr 172.21.15.89:0/1211112908 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd263fff700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd2641c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd262ffd700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd2641c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 -- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 msgr2=0x7fd2641c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd2641c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 -- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 msgr2=0x7fd2641c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd2641c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 -- 172.21.15.89:0/1211112908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c00bd40 con 0x7fd26409d6f0 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd262ffd700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd2641c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.410+0000 7fd2637fe700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd2641c8070 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c500 tx=0x7fd25400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.411+0000 7fd24bfff700 1 -- 172.21.15.89:0/1211112908 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2540052d0 con 0x7fd26409d6f0 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.411+0000 7fd24bfff700 1 -- 172.21.15.89:0/1211112908 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd254005450 con 0x7fd26409d6f0 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.411+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26411f970 con 0x7fd26409d6f0 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.411+0000 7fd24bfff700 1 -- 172.21.15.89:0/1211112908 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2540038c0 con 0x7fd26409d6f0 2023-09-30T22:18:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.411+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd264121090 con 0x7fd26409d6f0 2023-09-30T22:18:46.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.412+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd264074ea0 con 0x7fd26409d6f0 2023-09-30T22:18:46.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.412+0000 7fd24bfff700 1 -- 172.21.15.89:0/1211112908 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd254003a20 con 0x7fd26409d6f0 2023-09-30T22:18:46.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.412+0000 7fd24bfff700 1 -- 172.21.15.89:0/1211112908 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd25403eff0 con 0x7fd26409d6f0 2023-09-30T22:18:46.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.421+0000 7fd24bfff700 1 -- 172.21.15.89:0/1211112908 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd25400e280 con 0x7fd26409d6f0 2023-09-30T22:18:46.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.569+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8dfc005590 con 0x7f8e1c1338a0 2023-09-30T22:18:46.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.569+0000 7f8e037fe700 1 -- 172.21.15.89:0/2545477040 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e0c01bac0 con 0x7f8e1c1338a0 2023-09-30T22:18:46.569 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 msgr2=0x7f8e1c11f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 --2- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c11f570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c007dc0 tx=0x7f8e0c00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 shutdown_connections 2023-09-30T22:18:46.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 --2- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e1c1338a0 0x7f8e1c11f570 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 --2- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e1c09de50 0x7f8e1c11eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 --2- 172.21.15.89:0/2545477040 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e1c09e930 0x7f8e1c11f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 >> 172.21.15.89:0/2545477040 conn(0x7f8e1c0a2aa0 msgr2=0x7f8e1c09ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 shutdown_connections 2023-09-30T22:18:46.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.572+0000 7f8e214b9700 1 -- 172.21.15.89:0/2545477040 wait complete. 2023-09-30T22:18:46.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:46.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.591+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2460074ea0 con 0x7f246009d6f0 2023-09-30T22:18:46.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.591+0000 7f244effd700 1 -- 172.21.15.89:0/528945028 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2450028030 con 0x7f246009d6f0 2023-09-30T22:18:46.591 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:46.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f246009d6f0 msgr2=0x7f246011d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 --2- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f246009d6f0 0x7f246011d1c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f245000b560 tx=0x7f245000b870 comp rx=0 tx=0).stop 2023-09-30T22:18:46.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 shutdown_connections 2023-09-30T22:18:46.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 --2- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f246009e410 0x7f246011dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 --2- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f246009d6f0 0x7f246011d1c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 --2- 172.21.15.89:0/528945028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f246009e000 0x7f246011d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 >> 172.21.15.89:0/528945028 conn(0x7f2460066f50 msgr2=0x7f246012f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:46.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 shutdown_connections 2023-09-30T22:18:46.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.593+0000 7f2464c64700 1 -- 172.21.15.89:0/528945028 wait complete. 2023-09-30T22:18:46.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:46.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.609+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9efc005590 con 0x7f9f1c134260 2023-09-30T22:18:46.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.610+0000 7f9f037fe700 1 -- 172.21.15.89:0/550128445 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f1401bb00 con 0x7f9f1c134260 2023-09-30T22:18:46.609 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:46.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c134260 msgr2=0x7f9f1c09c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 --2- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c134260 0x7f9f1c09c8e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9f14007e40 tx=0x7f9f1400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 shutdown_connections 2023-09-30T22:18:46.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 --2- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f1c134260 0x7f9f1c09c8e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 --2- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f1c133420 0x7f9f1c09be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 --2- 172.21.15.89:0/550128445 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f1c133e30 0x7f9f1c09c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 >> 172.21.15.89:0/550128445 conn(0x7f9f1c0a4210 msgr2=0x7f9f1c1301c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:46.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 shutdown_connections 2023-09-30T22:18:46.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.612+0000 7f9f21076700 1 -- 172.21.15.89:0/550128445 wait complete. 2023-09-30T22:18:46.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:46.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.634+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd26406c360 con 0x7fd26409d6f0 2023-09-30T22:18:46.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.634+0000 7fd24bfff700 1 -- 172.21.15.89:0/1211112908 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd254017020 con 0x7fd26409d6f0 2023-09-30T22:18:46.634 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:46.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 msgr2=0x7fd2641c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:46.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd2641c8070 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c500 tx=0x7fd25400c8c0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 shutdown_connections 2023-09-30T22:18:46.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd264135e50 0x7fd2641c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd2641c8070 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 --2- 172.21.15.89:0/1211112908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e1d0 0x7fd2641c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:46.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 >> 172.21.15.89:0/1211112908 conn(0x7fd264066f50 msgr2=0x7fd264130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:46.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 shutdown_connections 2023-09-30T22:18:46.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:46.636+0000 7fd269a66700 1 -- 172.21.15.89:0/1211112908 wait complete. 2023-09-30T22:18:46.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:47.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:47.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:47.625 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:47.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:47.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.733+0000 7f92a071b700 1 Processor -- start 2023-09-30T22:18:47.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.733+0000 7f92a071b700 1 -- start start 2023-09-30T22:18:47.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f92a071b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298131810 0x7f9298131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f92a071b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f929809a580 0x7f9298139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f92a071b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92981313c0 con 0x7f9298099a40 2023-09-30T22:18:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f92a071b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929806e8c0 con 0x7f9298131810 2023-09-30T22:18:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f92a071b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9298071d20 con 0x7f929809a580 2023-09-30T22:18:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929dcb6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9298099a40 0x7f9298099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53080/0 (socket says 172.21.15.89:53080) 2023-09-30T22:18:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929dcb6700 1 -- 172.21.15.89:0/4107605828 learned_addr learned my addr 172.21.15.89:0/4107605828 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929ecb8700 1 --2- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f929809a580 0x7f9298139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929e4b7700 1 --2- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298131810 0x7f9298131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929d4b5700 1 -- 172.21.15.89:0/4107605828 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558876870 0 0) 0x7f92981313c0 con 0x7f9298099a40 2023-09-30T22:18:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929ecb8700 1 -- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298131810 msgr2=0x7f9298131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929ecb8700 1 --2- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298131810 0x7f9298131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929ecb8700 1 -- 172.21.15.89:0/4107605828 >> v1:172.21.15.89:6789/0 conn(0x7f9298099a40 legacy=0x7f9298099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.734+0000 7f929ecb8700 1 -- 172.21.15.89:0/4107605828 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92980721a0 con 0x7f929809a580 2023-09-30T22:18:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.735+0000 7f929ecb8700 1 --2- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f929809a580 0x7f9298139660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9290009dc0 tx=0x7f929000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eda7102e2f95fd34 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:47.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.735+0000 7f929d4b5700 1 -- 172.21.15.89:0/4107605828 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000e070 con 0x7f929809a580 2023-09-30T22:18:47.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.735+0000 7f929d4b5700 1 -- 172.21.15.89:0/4107605828 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929000bb20 con 0x7f929809a580 2023-09-30T22:18:47.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.735+0000 7f929d4b5700 1 -- 172.21.15.89:0/4107605828 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290012430 con 0x7f929809a580 2023-09-30T22:18:47.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.736+0000 7f92a071b700 1 -- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f929809a580 msgr2=0x7f9298139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.736+0000 7f92a071b700 1 --2- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f929809a580 0x7f9298139660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9290009dc0 tx=0x7f929000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.736+0000 7f92a071b700 1 -- 172.21.15.89:0/4107605828 shutdown_connections 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.736+0000 7f92a071b700 1 --2- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298131810 0x7f9298131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.736+0000 7f92a071b700 1 --2- 172.21.15.89:0/4107605828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f929809a580 0x7f9298139660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.736+0000 7f92a071b700 1 -- 172.21.15.89:0/4107605828 >> 172.21.15.89:0/4107605828 conn(0x7f92980a2aa0 msgr2=0x7f92980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.736+0000 7f92a071b700 1 -- 172.21.15.89:0/4107605828 shutdown_connections 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.736+0000 7f92a071b700 1 -- 172.21.15.89:0/4107605828 wait complete. 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f92a071b700 1 Processor -- start 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f92a071b700 1 -- start start 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f92a071b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298099a40 0x7f929806f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f92a071b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f929809a580 0x7f929806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f92a071b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9298131810 0x7f9298070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f92a071b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9298121370 con 0x7f929809a580 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f92a071b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9298120b00 con 0x7f9298099a40 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f92a071b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f929812ec20 con 0x7f9298131810 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f929e4b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298099a40 0x7f929806f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f929e4b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298099a40 0x7f929806f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35282/0 (socket says 172.21.15.89:35282) 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.737+0000 7f929e4b7700 1 -- 172.21.15.89:0/1639355127 learned_addr learned my addr 172.21.15.89:0/1639355127 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f929dcb6700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f929809a580 0x7f929806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f929ecb8700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9298131810 0x7f9298070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f929e4b7700 1 -- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9298131810 msgr2=0x7f9298070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f929e4b7700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9298131810 0x7f9298070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f929e4b7700 1 -- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f929809a580 msgr2=0x7f929806fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f929e4b7700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f929809a580 0x7f929806fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f929e4b7700 1 -- 172.21.15.89:0/1639355127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9290009a70 con 0x7f9298099a40 2023-09-30T22:18:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f929e4b7700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298099a40 0x7f929806f610 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f929400e9b0 tx=0x7f929400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f928effd700 1 -- 172.21.15.89:0/1639355127 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294009710 con 0x7f9298099a40 2023-09-30T22:18:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f928effd700 1 -- 172.21.15.89:0/1639355127 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929400c850 con 0x7f9298099a40 2023-09-30T22:18:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9298120dd0 con 0x7f9298099a40 2023-09-30T22:18:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f928effd700 1 -- 172.21.15.89:0/1639355127 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929400cbc0 con 0x7f9298099a40 2023-09-30T22:18:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.738+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929806eb00 con 0x7f9298099a40 2023-09-30T22:18:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.739+0000 7f928effd700 1 -- 172.21.15.89:0/1639355127 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f929400cd20 con 0x7f9298099a40 2023-09-30T22:18:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.739+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9298074ea0 con 0x7f9298099a40 2023-09-30T22:18:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.740+0000 7f928effd700 1 -- 172.21.15.89:0/1639355127 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f929403fb40 con 0x7f9298099a40 2023-09-30T22:18:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.744+0000 7f928effd700 1 -- 172.21.15.89:0/1639355127 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f929401b3b0 con 0x7f9298099a40 2023-09-30T22:18:47.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.772+0000 7f6f309ad700 1 Processor -- start 2023-09-30T22:18:47.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.772+0000 7f6f309ad700 1 -- start start 2023-09-30T22:18:47.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f309ad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c06a770 0x7f6f2c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f309ad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c13d1f0 0x7f6f2c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f309ad700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2c09a810 con 0x7f6f2c09dc30 2023-09-30T22:18:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f309ad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c06ab40 con 0x7f6f2c06a770 2023-09-30T22:18:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f309ad700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c09ebc0 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f2ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c13d1f0 0x7f6f2c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f2ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c13d1f0 0x7f6f2c13f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44788/0 (socket says 172.21.15.89:44788) 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f2ad9d700 1 -- 172.21.15.89:0/3351436476 learned_addr learned my addr 172.21.15.89:0/3351436476 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.773+0000 7f6f2a59c700 1 --2- 172.21.15.89:0/3351436476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c06a770 0x7f6f2c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.774+0000 7f6f2ad9d700 1 -- 172.21.15.89:0/3351436476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c06a770 msgr2=0x7f6f2c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.774+0000 7f6f2ad9d700 1 --2- 172.21.15.89:0/3351436476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c06a770 0x7f6f2c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.774+0000 7f6f2ad9d700 1 -- 172.21.15.89:0/3351436476 >> v1:172.21.15.89:6789/0 conn(0x7f6f2c09dc30 legacy=0x7f6f2c09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.774+0000 7f6f2ad9d700 1 -- 172.21.15.89:0/3351436476 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c09e920 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.774+0000 7f6f2ad9d700 1 --2- 172.21.15.89:0/3351436476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c13d1f0 0x7f6f2c13f5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009e80 tx=0x7f6f2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cc37025b2e2f0ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.774+0000 7f6f2959a700 1 -- 172.21.15.89:0/3351436476 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400e070 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.774+0000 7f6f2959a700 1 -- 172.21.15.89:0/3351436476 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2400b9c0 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.774+0000 7f6f2959a700 1 -- 172.21.15.89:0/3351436476 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400bd70 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.775+0000 7f6f309ad700 1 -- 172.21.15.89:0/3351436476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c13d1f0 msgr2=0x7f6f2c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.775+0000 7f6f309ad700 1 --2- 172.21.15.89:0/3351436476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c13d1f0 0x7f6f2c13f5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009e80 tx=0x7f6f2400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.775+0000 7f6f309ad700 1 -- 172.21.15.89:0/3351436476 shutdown_connections 2023-09-30T22:18:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.775+0000 7f6f309ad700 1 --2- 172.21.15.89:0/3351436476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c06a770 0x7f6f2c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.775+0000 7f6f309ad700 1 --2- 172.21.15.89:0/3351436476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c13d1f0 0x7f6f2c13f5d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.775+0000 7f6f309ad700 1 -- 172.21.15.89:0/3351436476 >> 172.21.15.89:0/3351436476 conn(0x7f6f2c066f50 msgr2=0x7f6f2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.775+0000 7f6f309ad700 1 -- 172.21.15.89:0/3351436476 shutdown_connections 2023-09-30T22:18:47.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.775+0000 7f6f309ad700 1 -- 172.21.15.89:0/3351436476 wait complete. 2023-09-30T22:18:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.776+0000 7f6f309ad700 1 Processor -- start 2023-09-30T22:18:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.776+0000 7f6f309ad700 1 -- start start 2023-09-30T22:18:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f309ad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c06a770 0x7f6f2c1cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f309ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f2c09dc30 0x7f6f2c1ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f309ad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c13d1f0 0x7f6f2c1cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f309ad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c070080 con 0x7f6f2c09dc30 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f309ad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c06f450 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f309ad700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0a19d0 con 0x7f6f2c06a770 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c13d1f0 0x7f6f2c1cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c13d1f0 0x7f6f2c1cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35304/0 (socket says 172.21.15.89:35304) 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2ad9d700 1 -- 172.21.15.89:0/1616960921 learned_addr learned my addr 172.21.15.89:0/1616960921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f29d9b700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f2c09dc30 0x7f6f2c1ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2a59c700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c06a770 0x7f6f2c1cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2ad9d700 1 -- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c06a770 msgr2=0x7f6f2c1cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2ad9d700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c06a770 0x7f6f2c1cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2ad9d700 1 -- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f2c09dc30 msgr2=0x7f6f2c1ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2ad9d700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f2c09dc30 0x7f6f2c1ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2ad9d700 1 -- 172.21.15.89:0/1616960921 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f24009b30 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.777+0000 7f6f2a59c700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c06a770 0x7f6f2c1cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.778+0000 7f6f2ad9d700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c13d1f0 0x7f6f2c1cd440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c11d760 tx=0x7f6f24004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.778+0000 7f6f1affd700 1 -- 172.21.15.89:0/1616960921 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24004450 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.778+0000 7f6f1affd700 1 -- 172.21.15.89:0/1616960921 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f240045b0 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.778+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c09f1a0 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.778+0000 7f6f1affd700 1 -- 172.21.15.89:0/1616960921 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24004960 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.778+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c070810 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.779+0000 7f6f1affd700 1 -- 172.21.15.89:0/1616960921 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f2401c410 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.779+0000 7f6f1affd700 1 -- 172.21.15.89:0/1616960921 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6f24041140 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.779+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c06ee90 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.785+0000 7fd91b1d6700 1 Processor -- start 2023-09-30T22:18:47.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.785+0000 7fd91b1d6700 1 -- start start 2023-09-30T22:18:47.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd91b1d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd914066410 0x7fd91406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd91b1d6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd91406ef70 0x7fd91412fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd91b1d6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd914099b50 con 0x7fd91406eb80 2023-09-30T22:18:47.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd91b1d6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140667e0 con 0x7fd91406ef70 2023-09-30T22:18:47.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd91b1d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91406f3d0 con 0x7fd914066410 2023-09-30T22:18:47.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd918f72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd914066410 0x7fd91406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd918f72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd914066410 0x7fd91406e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35318/0 (socket says 172.21.15.89:35318) 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd918f72700 1 -- 172.21.15.89:0/2327631239 learned_addr learned my addr 172.21.15.89:0/2327631239 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd919773700 1 --2- 172.21.15.89:0/2327631239 >> v2:172.21.15.89:3300/0 conn(0x7fd91406ef70 0x7fd91412fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd918f72700 1 -- 172.21.15.89:0/2327631239 >> v2:172.21.15.89:3300/0 conn(0x7fd91406ef70 msgr2=0x7fd91412fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd918f72700 1 --2- 172.21.15.89:0/2327631239 >> v2:172.21.15.89:3300/0 conn(0x7fd91406ef70 0x7fd91412fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd918f72700 1 -- 172.21.15.89:0/2327631239 >> v1:172.21.15.89:6789/0 conn(0x7fd91406eb80 legacy=0x7fd91412f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd918f72700 1 -- 172.21.15.89:0/2327631239 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914099f60 con 0x7fd914066410 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd919773700 1 --2- 172.21.15.89:0/2327631239 >> v2:172.21.15.89:3300/0 conn(0x7fd91406ef70 0x7fd91412fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd918f72700 1 --2- 172.21.15.89:0/2327631239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd914066410 0x7fd91406e640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9080098f0 tx=0x7fd908009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a068ee02103f3cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.786+0000 7fd9137fe700 1 -- 172.21.15.89:0/2327631239 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908004230 con 0x7fd914066410 2023-09-30T22:18:47.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd9137fe700 1 -- 172.21.15.89:0/2327631239 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908004390 con 0x7fd914066410 2023-09-30T22:18:47.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd9137fe700 1 -- 172.21.15.89:0/2327631239 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080036a0 con 0x7fd914066410 2023-09-30T22:18:47.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7f6f1affd700 1 -- 172.21.15.89:0/1616960921 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f24023770 con 0x7f6f2c13d1f0 2023-09-30T22:18:47.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2327631239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd914066410 msgr2=0x7fd91406e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd91b1d6700 1 --2- 172.21.15.89:0/2327631239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd914066410 0x7fd91406e640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9080098f0 tx=0x7fd908009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:47.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2327631239 shutdown_connections 2023-09-30T22:18:47.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd91b1d6700 1 --2- 172.21.15.89:0/2327631239 >> v2:172.21.15.89:3300/0 conn(0x7fd91406ef70 0x7fd91412fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd91b1d6700 1 --2- 172.21.15.89:0/2327631239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd914066410 0x7fd91406e640 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2327631239 >> 172.21.15.89:0/2327631239 conn(0x7fd9140a2aa0 msgr2=0x7fd9140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2327631239 shutdown_connections 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.787+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2327631239 wait complete. 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.788+0000 7fd91b1d6700 1 Processor -- start 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.788+0000 7fd91b1d6700 1 -- start start 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd91b1d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd91406eb80 0x7fd91406f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd91b1d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd91406ef70 0x7fd91406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd91b1d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9140700f0 0x7fd9141ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd91b1d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91411f480 con 0x7fd9140700f0 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd91b1d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91411ddd0 con 0x7fd91406eb80 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd91b1d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd91411e7c0 con 0x7fd91406ef70 2023-09-30T22:18:47.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd91406eb80 0x7fd91406f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd91406eb80 0x7fd91406f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35326/0 (socket says 172.21.15.89:35326) 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 -- 172.21.15.89:0/2209768040 learned_addr learned my addr 172.21.15.89:0/2209768040 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 -- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd91406ef70 msgr2=0x7fd91406fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 --2- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd91406ef70 0x7fd91406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 -- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9140700f0 msgr2=0x7fd9141ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 --2- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9140700f0 0x7fd9141ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 -- 172.21.15.89:0/2209768040 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908009550 con 0x7fd91406eb80 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.789+0000 7fd918f72700 1 --2- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd91406eb80 0x7fd91406f670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd908004000 tx=0x7fd908004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.790+0000 7fd9117fa700 1 -- 172.21.15.89:0/2209768040 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908003c00 con 0x7fd91406eb80 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.790+0000 7fd9117fa700 1 -- 172.21.15.89:0/2209768040 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90801f070 con 0x7fd91406eb80 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.790+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9140a2030 con 0x7fd91406eb80 2023-09-30T22:18:47.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.790+0000 7fd9117fa700 1 -- 172.21.15.89:0/2209768040 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800b660 con 0x7fd91406eb80 2023-09-30T22:18:47.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.790+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91411ec10 con 0x7fd91406eb80 2023-09-30T22:18:47.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.791+0000 7fd9117fa700 1 -- 172.21.15.89:0/2209768040 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd908021030 con 0x7fd91406eb80 2023-09-30T22:18:47.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.791+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd914074ea0 con 0x7fd91406eb80 2023-09-30T22:18:47.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.791+0000 7fd9117fa700 1 -- 172.21.15.89:0/2209768040 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd908040300 con 0x7fd91406eb80 2023-09-30T22:18:47.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.795+0000 7fd9117fa700 1 -- 172.21.15.89:0/2209768040 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd908010970 con 0x7fd91406eb80 2023-09-30T22:18:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.816+0000 7fad23810700 1 Processor -- start 2023-09-30T22:18:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad23810700 1 -- start start 2023-09-30T22:18:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad23810700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c0a0a30 0x7fad1c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad23810700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c134170 0x7fad1c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad23810700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c09edd0 con 0x7fad1c0a1340 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad23810700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c09e1f0 con 0x7fad1c0a0a30 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad23810700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c09dc50 con 0x7fad1c134170 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad20dab700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fad1c0a1340 0x7fad1c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53106/0 (socket says 172.21.15.89:53106) 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad20dab700 1 -- 172.21.15.89:0/169725413 learned_addr learned my addr 172.21.15.89:0/169725413 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad215ac700 1 --2- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c0a0a30 0x7fad1c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.817+0000 7fad21dad700 1 --2- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c134170 0x7fad1c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad13fff700 1 -- 172.21.15.89:0/169725413 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931299311 0 0) 0x7fad1c09edd0 con 0x7fad1c0a1340 2023-09-30T22:18:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad13fff700 1 -- 172.21.15.89:0/169725413 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad080035b0 con 0x7fad1c0a1340 2023-09-30T22:18:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad215ac700 1 -- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c134170 msgr2=0x7fad1c13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad215ac700 1 --2- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c134170 0x7fad1c13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad215ac700 1 -- 172.21.15.89:0/169725413 >> v1:172.21.15.89:6789/0 conn(0x7fad1c0a1340 legacy=0x7fad1c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad215ac700 1 -- 172.21.15.89:0/169725413 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0a1fe0 con 0x7fad1c0a0a30 2023-09-30T22:18:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad215ac700 1 --2- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c0a0a30 0x7fad1c0a0e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fad18009870 tx=0x7fad18009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96211ff158ee35d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad13fff700 1 -- 172.21.15.89:0/169725413 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011830 con 0x7fad1c0a0a30 2023-09-30T22:18:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad13fff700 1 -- 172.21.15.89:0/169725413 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18011990 con 0x7fad1c0a0a30 2023-09-30T22:18:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.818+0000 7fad13fff700 1 -- 172.21.15.89:0/169725413 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011d20 con 0x7fad1c0a0a30 2023-09-30T22:18:47.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.819+0000 7fad23810700 1 -- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c0a0a30 msgr2=0x7fad1c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.819+0000 7fad23810700 1 --2- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c0a0a30 0x7fad1c0a0e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fad18009870 tx=0x7fad18009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.819+0000 7fad23810700 1 -- 172.21.15.89:0/169725413 shutdown_connections 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.819+0000 7fad23810700 1 --2- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c0a0a30 0x7fad1c0a0e00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.819+0000 7fad23810700 1 --2- 172.21.15.89:0/169725413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c134170 0x7fad1c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.819+0000 7fad23810700 1 -- 172.21.15.89:0/169725413 >> 172.21.15.89:0/169725413 conn(0x7fad1c0a6550 msgr2=0x7fad1c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.819+0000 7fad23810700 1 -- 172.21.15.89:0/169725413 shutdown_connections 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.819+0000 7fad23810700 1 -- 172.21.15.89:0/169725413 wait complete. 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad23810700 1 Processor -- start 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad23810700 1 -- start start 2023-09-30T22:18:47.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad23810700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1c0a0a30 0x7fad1c1226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad23810700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c0a1340 0x7fad1c122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad23810700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c134170 0x7fad1c123160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad23810700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c126200 con 0x7fad1c0a0a30 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad23810700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c124b80 con 0x7fad1c134170 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad23810700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c123b40 con 0x7fad1c0a1340 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad20dab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c0a1340 0x7fad1c122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad20dab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c0a1340 0x7fad1c122c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44812/0 (socket says 172.21.15.89:44812) 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.820+0000 7fad20dab700 1 -- 172.21.15.89:0/3171136250 learned_addr learned my addr 172.21.15.89:0/3171136250 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad215ac700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1c0a0a30 0x7fad1c1226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad21dad700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c134170 0x7fad1c123160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad20dab700 1 -- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c134170 msgr2=0x7fad1c123160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad20dab700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c134170 0x7fad1c123160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad20dab700 1 -- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1c0a0a30 msgr2=0x7fad1c1226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad20dab700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1c0a0a30 0x7fad1c1226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad20dab700 1 -- 172.21.15.89:0/3171136250 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad180094a0 con 0x7fad1c0a1340 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad20dab700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c0a1340 0x7fad1c122c20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00c700 tx=0x7fad0c00ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.821+0000 7fad11ffb700 1 -- 172.21.15.89:0/3171136250 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c005d70 con 0x7fad1c0a1340 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.822+0000 7fad11ffb700 1 -- 172.21.15.89:0/3171136250 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c0070d0 con 0x7fad1c0a1340 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.822+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c0a25e0 con 0x7fad1c0a1340 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.822+0000 7fad11ffb700 1 -- 172.21.15.89:0/3171136250 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c007460 con 0x7fad1c0a1340 2023-09-30T22:18:47.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.822+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c133630 con 0x7fad1c0a1340 2023-09-30T22:18:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.823+0000 7fad11ffb700 1 -- 172.21.15.89:0/3171136250 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad0c0075c0 con 0x7fad1c0a1340 2023-09-30T22:18:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.823+0000 7fad11ffb700 1 -- 172.21.15.89:0/3171136250 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fad0c03eae0 con 0x7fad1c0a1340 2023-09-30T22:18:47.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.823+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c125d80 con 0x7fad1c0a1340 2023-09-30T22:18:47.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.828+0000 7fad11ffb700 1 -- 172.21.15.89:0/3171136250 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad1c125d80 con 0x7fad1c0a1340 2023-09-30T22:18:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.989+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd91406b9e0 con 0x7fd91406eb80 2023-09-30T22:18:47.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.990+0000 7fd9117fa700 1 -- 172.21.15.89:0/2209768040 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd90802a070 con 0x7fd91406eb80 2023-09-30T22:18:47.990 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd91406eb80 msgr2=0x7fd91406f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 --2- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd91406eb80 0x7fd91406f670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd908004000 tx=0x7fd908004750 comp rx=0 tx=0).stop 2023-09-30T22:18:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 shutdown_connections 2023-09-30T22:18:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 --2- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9140700f0 0x7fd9141ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 --2- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd91406eb80 0x7fd91406f670 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 --2- 172.21.15.89:0/2209768040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd91406ef70 0x7fd91406fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 >> 172.21.15.89:0/2209768040 conn(0x7fd9140a2aa0 msgr2=0x7fd91412e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 shutdown_connections 2023-09-30T22:18:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:47.993+0000 7fd91b1d6700 1 -- 172.21.15.89:0/2209768040 wait complete. 2023-09-30T22:18:48.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.003+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f929806b9e0 con 0x7f9298099a40 2023-09-30T22:18:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.003+0000 7f928effd700 1 -- 172.21.15.89:0/1639355127 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9294017030 con 0x7f9298099a40 2023-09-30T22:18:48.003 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:48.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.006+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298099a40 msgr2=0x7f929806f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.006+0000 7f92a071b700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298099a40 0x7f929806f610 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f929400e9b0 tx=0x7f929400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.007+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 shutdown_connections 2023-09-30T22:18:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.007+0000 7f92a071b700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9298131810 0x7f9298070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.007+0000 7f92a071b700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9298099a40 0x7f929806f610 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.007+0000 7f92a071b700 1 --2- 172.21.15.89:0/1639355127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f929809a580 0x7f929806fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.007+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 >> 172.21.15.89:0/1639355127 conn(0x7f92980a2aa0 msgr2=0x7f92981b8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.007+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 shutdown_connections 2023-09-30T22:18:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.007+0000 7f92a071b700 1 -- 172.21.15.89:0/1639355127 wait complete. 2023-09-30T22:18:48.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:48.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.023+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fad1c125d80 con 0x7fad1c0a1340 2023-09-30T22:18:48.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.023+0000 7fad11ffb700 1 -- 172.21.15.89:0/3171136250 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad1c125d80 con 0x7fad1c0a1340 2023-09-30T22:18:48.023 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:48.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c0a1340 msgr2=0x7fad1c122c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c0a1340 0x7fad1c122c20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00c700 tx=0x7fad0c00ca10 comp rx=0 tx=0).stop 2023-09-30T22:18:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 shutdown_connections 2023-09-30T22:18:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad1c134170 0x7fad1c123160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad1c0a0a30 0x7fad1c1226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 --2- 172.21.15.89:0/3171136250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad1c0a1340 0x7fad1c122c20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 >> 172.21.15.89:0/3171136250 conn(0x7fad1c0a6550 msgr2=0x7fad1c09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 shutdown_connections 2023-09-30T22:18:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.026+0000 7fad23810700 1 -- 172.21.15.89:0/3171136250 wait complete. 2023-09-30T22:18:48.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.033+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6f2c074ea0 con 0x7f6f2c13d1f0 2023-09-30T22:18:48.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.033+0000 7f6f1affd700 1 -- 172.21.15.89:0/1616960921 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f2401b3a0 con 0x7f6f2c13d1f0 2023-09-30T22:18:48.033 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.035+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c13d1f0 msgr2=0x7f6f2c1cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.035+0000 7f6f309ad700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c13d1f0 0x7f6f2c1cd440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c11d760 tx=0x7f6f24004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.035+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 shutdown_connections 2023-09-30T22:18:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.035+0000 7f6f309ad700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f2c13d1f0 0x7f6f2c1cd440 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.035+0000 7f6f309ad700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f2c06a770 0x7f6f2c1cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.035+0000 7f6f309ad700 1 --2- 172.21.15.89:0/1616960921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f2c09dc30 0x7f6f2c1ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.035+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 >> 172.21.15.89:0/1616960921 conn(0x7f6f2c066f50 msgr2=0x7f6f2c13d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.036+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 shutdown_connections 2023-09-30T22:18:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:48.036+0000 7f6f309ad700 1 -- 172.21.15.89:0/1616960921 wait complete. 2023-09-30T22:18:48.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:48.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:49.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:49.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:49.039 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:49.049 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.155+0000 7f9a5a3ea700 1 Processor -- start 2023-09-30T22:18:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.155+0000 7f9a5a3ea700 1 -- start start 2023-09-30T22:18:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a5a3ea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412cf60 0x7f9a5406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a5a3ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a5412f8b0 0x7f9a5412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a5a3ea700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5412cd90 con 0x7f9a54130260 2023-09-30T22:18:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a5a3ea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5412d330 con 0x7f9a5412f8b0 2023-09-30T22:18:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a5a3ea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a541306d0 con 0x7f9a5412cf60 2023-09-30T22:18:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a53fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412cf60 0x7f9a5406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a53fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412cf60 0x7f9a5406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44828/0 (socket says 172.21.15.89:44828) 2023-09-30T22:18:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a53fff700 1 -- 172.21.15.89:0/3243310325 learned_addr learned my addr 172.21.15.89:0/3243310325 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a537fe700 1 --2- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a5412f8b0 0x7f9a5412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a53fff700 1 -- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a5412f8b0 msgr2=0x7f9a5412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a53fff700 1 --2- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a5412f8b0 0x7f9a5412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a53fff700 1 -- 172.21.15.89:0/3243310325 >> v1:172.21.15.89:6789/0 conn(0x7f9a54130260 legacy=0x7f9a54136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.156+0000 7f9a53fff700 1 -- 172.21.15.89:0/3243310325 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54071780 con 0x7f9a5412cf60 2023-09-30T22:18:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.157+0000 7f9a537fe700 1 --2- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a5412f8b0 0x7f9a5412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.157+0000 7f9a53fff700 1 --2- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412cf60 0x7f9a5406f5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009920 tx=0x7f9a44009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1780c8c8b1f93edd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.157+0000 7f9a52ffd700 1 -- 172.21.15.89:0/3243310325 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400ba60 con 0x7f9a5412cf60 2023-09-30T22:18:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.157+0000 7f9a52ffd700 1 -- 172.21.15.89:0/3243310325 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4400bbc0 con 0x7f9a5412cf60 2023-09-30T22:18:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.157+0000 7f9a52ffd700 1 -- 172.21.15.89:0/3243310325 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44009e70 con 0x7f9a5412cf60 2023-09-30T22:18:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.158+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412cf60 msgr2=0x7f9a5406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.158+0000 7f9a5a3ea700 1 --2- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412cf60 0x7f9a5406f5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009920 tx=0x7f9a44009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.158+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/3243310325 shutdown_connections 2023-09-30T22:18:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.158+0000 7f9a5a3ea700 1 --2- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412cf60 0x7f9a5406f5d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.158+0000 7f9a5a3ea700 1 --2- 172.21.15.89:0/3243310325 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a5412f8b0 0x7f9a5412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.158+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/3243310325 >> 172.21.15.89:0/3243310325 conn(0x7f9a540a2aa0 msgr2=0x7f9a540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.158+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/3243310325 shutdown_connections 2023-09-30T22:18:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.158+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/3243310325 wait complete. 2023-09-30T22:18:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.159+0000 7f9a5a3ea700 1 Processor -- start 2023-09-30T22:18:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.159+0000 7f9a5a3ea700 1 -- start start 2023-09-30T22:18:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a5a3ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5412cf60 0x7f9a541c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a5a3ea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412f8b0 0x7f9a541c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a5a3ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a54130260 0x7f9a541c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a5a3ea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5409e810 con 0x7f9a5412cf60 2023-09-30T22:18:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a5a3ea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5409b340 con 0x7f9a54130260 2023-09-30T22:18:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a5a3ea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54129fa0 con 0x7f9a5412f8b0 2023-09-30T22:18:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a58987700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a54130260 0x7f9a541c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a58987700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a54130260 0x7f9a541c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35362/0 (socket says 172.21.15.89:35362) 2023-09-30T22:18:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a58987700 1 -- 172.21.15.89:0/2558177124 learned_addr learned my addr 172.21.15.89:0/2558177124 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a537fe700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412f8b0 0x7f9a541c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a53fff700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5412cf60 0x7f9a541c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a58987700 1 -- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412f8b0 msgr2=0x7f9a541c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a58987700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412f8b0 0x7f9a541c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a58987700 1 -- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5412cf60 msgr2=0x7f9a541c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a58987700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5412cf60 0x7f9a541c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.160+0000 7f9a58987700 1 -- 172.21.15.89:0/2558177124 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44009550 con 0x7f9a54130260 2023-09-30T22:18:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.161+0000 7f9a58987700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a54130260 0x7f9a541c8e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00cbc0 tx=0x7f9a4c00ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.161+0000 7f9a50ff9700 1 -- 172.21.15.89:0/2558177124 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c003c00 con 0x7f9a54130260 2023-09-30T22:18:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.161+0000 7f9a50ff9700 1 -- 172.21.15.89:0/2558177124 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c003d60 con 0x7f9a54130260 2023-09-30T22:18:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.161+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5409cb00 con 0x7f9a54130260 2023-09-30T22:18:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.161+0000 7f9a50ff9700 1 -- 172.21.15.89:0/2558177124 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00aae0 con 0x7f9a54130260 2023-09-30T22:18:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.161+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5411d470 con 0x7f9a54130260 2023-09-30T22:18:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.162+0000 7f9a50ff9700 1 -- 172.21.15.89:0/2558177124 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9a4c00ac40 con 0x7f9a54130260 2023-09-30T22:18:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.162+0000 7f9a50ff9700 1 -- 172.21.15.89:0/2558177124 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9a4c03f7d0 con 0x7f9a54130260 2023-09-30T22:18:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.162+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a54074ea0 con 0x7f9a54130260 2023-09-30T22:18:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.171+0000 7f9a50ff9700 1 -- 172.21.15.89:0/2558177124 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a4c01b400 con 0x7f9a54130260 2023-09-30T22:18:49.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.189+0000 7f1e8d3d2700 1 Processor -- start 2023-09-30T22:18:49.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.189+0000 7f1e8d3d2700 1 -- start start 2023-09-30T22:18:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e8d3d2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e8809d6f0 0x7f1e8809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e8d3d2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 0x7f1e881339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e8d3d2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8809b120 con 0x7f1e8809e410 2023-09-30T22:18:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e8d3d2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8806b460 con 0x7f1e8809d6f0 2023-09-30T22:18:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e8d3d2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8809a7e0 con 0x7f1e8809e000 2023-09-30T22:18:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e86ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e8809d6f0 0x7f1e8809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e86ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e8809d6f0 0x7f1e8809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45572/0 (socket says 172.21.15.89:45572) 2023-09-30T22:18:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e86ffd700 1 -- 172.21.15.89:0/1566395665 learned_addr learned my addr 172.21.15.89:0/1566395665 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.190+0000 7f1e867fc700 1 --2- 172.21.15.89:0/1566395665 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 0x7f1e881339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.191+0000 7f1e867fc700 1 -- 172.21.15.89:0/1566395665 >> v2:172.21.15.89:3300/0 conn(0x7f1e8809d6f0 msgr2=0x7f1e8809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.191+0000 7f1e867fc700 1 --2- 172.21.15.89:0/1566395665 >> v2:172.21.15.89:3300/0 conn(0x7f1e8809d6f0 0x7f1e8809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.191+0000 7f1e867fc700 1 -- 172.21.15.89:0/1566395665 >> v1:172.21.15.89:6789/0 conn(0x7f1e8809e410 legacy=0x7f1e88133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.191+0000 7f1e867fc700 1 -- 172.21.15.89:0/1566395665 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8809a500 con 0x7f1e8809e000 2023-09-30T22:18:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.191+0000 7f1e867fc700 1 --2- 172.21.15.89:0/1566395665 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 0x7f1e881339f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009870 tx=0x7f1e7c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcea86169e94e1b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.191+0000 7f1e85ffb700 1 -- 172.21.15.89:0/1566395665 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c011830 con 0x7f1e8809e000 2023-09-30T22:18:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.191+0000 7f1e85ffb700 1 -- 172.21.15.89:0/1566395665 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c011990 con 0x7f1e8809e000 2023-09-30T22:18:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.191+0000 7f1e85ffb700 1 -- 172.21.15.89:0/1566395665 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c011d20 con 0x7f1e8809e000 2023-09-30T22:18:49.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f3905ae8700 1 Processor -- start 2023-09-30T22:18:49.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f3905ae8700 1 -- start start 2023-09-30T22:18:49.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f3905ae8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001338c0 0x7f3900133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f3905ae8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f390009e560 0x7f390013b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f3905ae8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390006f720 con 0x7f39001341d0 2023-09-30T22:18:49.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f3905ae8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3900070c80 con 0x7f39001338c0 2023-09-30T22:18:49.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f3905ae8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f390006fce0 con 0x7f390009e560 2023-09-30T22:18:49.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/1566395665 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 msgr2=0x7f1e881339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f1e8d3d2700 1 --2- 172.21.15.89:0/1566395665 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 0x7f1e881339f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009870 tx=0x7f1e7c009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:49.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001338c0 0x7f3900133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f38feffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f39001341d0 0x7f390009de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53146/0 (socket says 172.21.15.89:53146) 2023-09-30T22:18:49.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f38feffd700 1 -- 172.21.15.89:0/2708488044 learned_addr learned my addr 172.21.15.89:0/2708488044 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:49.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/1566395665 shutdown_connections 2023-09-30T22:18:49.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f1e8d3d2700 1 --2- 172.21.15.89:0/1566395665 >> v2:172.21.15.89:3300/0 conn(0x7f1e8809d6f0 0x7f1e8809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f1e8d3d2700 1 --2- 172.21.15.89:0/1566395665 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 0x7f1e881339f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/1566395665 >> 172.21.15.89:0/1566395665 conn(0x7f1e88066f50 msgr2=0x7f1e8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:49.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.192+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/1566395665 shutdown_connections 2023-09-30T22:18:49.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f38ff7fe700 1 -- 172.21.15.89:0/2708488044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f390009e560 msgr2=0x7f390013b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:49.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f38ff7fe700 1 --2- 172.21.15.89:0/2708488044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f390009e560 0x7f390013b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f38ff7fe700 1 -- 172.21.15.89:0/2708488044 >> v1:172.21.15.89:6789/0 conn(0x7f39001341d0 legacy=0x7f390009de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f38ff7fe700 1 -- 172.21.15.89:0/2708488044 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39001336a0 con 0x7f39001338c0 2023-09-30T22:18:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/1566395665 wait complete. 2023-09-30T22:18:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f38ff7fe700 1 --2- 172.21.15.89:0/2708488044 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001338c0 0x7f3900133c90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009870 tx=0x7f38f0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c536a597f893b23 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f38fe7fc700 1 -- 172.21.15.89:0/2708488044 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0011830 con 0x7f39001338c0 2023-09-30T22:18:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f38fe7fc700 1 -- 172.21.15.89:0/2708488044 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f0011990 con 0x7f39001338c0 2023-09-30T22:18:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.193+0000 7f38fe7fc700 1 -- 172.21.15.89:0/2708488044 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0011d20 con 0x7f39001338c0 2023-09-30T22:18:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f3905ae8700 1 -- 172.21.15.89:0/2708488044 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001338c0 msgr2=0x7f3900133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f3905ae8700 1 --2- 172.21.15.89:0/2708488044 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001338c0 0x7f3900133c90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009870 tx=0x7f38f0009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f1e8d3d2700 1 Processor -- start 2023-09-30T22:18:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f3905ae8700 1 -- 172.21.15.89:0/2708488044 shutdown_connections 2023-09-30T22:18:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f3905ae8700 1 --2- 172.21.15.89:0/2708488044 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001338c0 0x7f3900133c90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f3905ae8700 1 --2- 172.21.15.89:0/2708488044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f390009e560 0x7f390013b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f3905ae8700 1 -- 172.21.15.89:0/2708488044 >> 172.21.15.89:0/2708488044 conn(0x7f39000a2aa0 msgr2=0x7f39000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f1e8d3d2700 1 -- start start 2023-09-30T22:18:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.194+0000 7f3905ae8700 1 -- 172.21.15.89:0/2708488044 shutdown_connections 2023-09-30T22:18:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f3905ae8700 1 -- 172.21.15.89:0/2708488044 wait complete. 2023-09-30T22:18:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e8d3d2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e8809d6f0 0x7f1e881a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e8d3d2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 0x7f1e881a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e8d3d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e8809e410 0x7f1e881a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e8d3d2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e881a9830 con 0x7f1e8809e410 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e8d3d2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e881aa3f0 con 0x7f1e8809e000 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e8d3d2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8809c710 con 0x7f1e8809d6f0 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e8809d6f0 0x7f1e881a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e877fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e8809e410 0x7f1e881a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e8809d6f0 0x7f1e881a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44852/0 (socket says 172.21.15.89:44852) 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 -- 172.21.15.89:0/3920381280 learned_addr learned my addr 172.21.15.89:0/3920381280 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 -- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 msgr2=0x7f1e881a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 --2- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 0x7f1e881a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 -- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e8809e410 msgr2=0x7f1e881a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 --2- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e8809e410 0x7f1e881a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 -- 172.21.15.89:0/3920381280 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0094a0 con 0x7f1e8809d6f0 2023-09-30T22:18:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f1e86ffd700 1 --2- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e8809d6f0 0x7f1e881a6020 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009d90 tx=0x7f1e7800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.195+0000 7f3905ae8700 1 Processor -- start 2023-09-30T22:18:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f1e6f7fe700 1 -- 172.21.15.89:0/3920381280 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7800cbb0 con 0x7f1e8809d6f0 2023-09-30T22:18:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f1e6f7fe700 1 -- 172.21.15.89:0/3920381280 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7800f040 con 0x7f1e8809d6f0 2023-09-30T22:18:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f1e6f7fe700 1 -- 172.21.15.89:0/3920381280 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78014480 con 0x7f1e8809d6f0 2023-09-30T22:18:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e881a9a30 con 0x7f1e8809d6f0 2023-09-30T22:18:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e881a8860 con 0x7f1e8809d6f0 2023-09-30T22:18:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f3905ae8700 1 -- start start 2023-09-30T22:18:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f3905ae8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f390009e560 0x7f390011fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f3905ae8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39001338c0 0x7f390011ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f3905ae8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001341d0 0x7f39001204b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f3905ae8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900121640 con 0x7f390009e560 2023-09-30T22:18:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f3905ae8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39001309f0 con 0x7f39001341d0 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f3905ae8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3900123630 con 0x7f39001338c0 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f390009e560 0x7f390011fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f390009e560 0x7f390011fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45588/0 (socket says 172.21.15.89:45588) 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.196+0000 7f38ff7fe700 1 -- 172.21.15.89:0/1796062527 learned_addr learned my addr 172.21.15.89:0/1796062527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38ff7fe700 1 -- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39001338c0 msgr2=0x7f390011ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38ff7fe700 1 --2- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39001338c0 0x7f390011ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38ff7fe700 1 -- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001341d0 msgr2=0x7f39001204b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38ff7fe700 1 --2- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001341d0 0x7f39001204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38ff7fe700 1 -- 172.21.15.89:0/1796062527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f00094a0 con 0x7f390009e560 2023-09-30T22:18:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38ff7fe700 1 --2- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f390009e560 0x7f390011fa30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f38f00090b0 tx=0x7f38f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f1e6f7fe700 1 -- 172.21.15.89:0/3920381280 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e780145e0 con 0x7f1e8809d6f0 2023-09-30T22:18:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38e7fff700 1 -- 172.21.15.89:0/1796062527 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00036a0 con 0x7f390009e560 2023-09-30T22:18:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38e7fff700 1 -- 172.21.15.89:0/1796062527 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f001f070 con 0x7f390009e560 2023-09-30T22:18:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390006f960 con 0x7f390009e560 2023-09-30T22:18:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f38e7fff700 1 -- 172.21.15.89:0/1796062527 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0003a20 con 0x7f390009e560 2023-09-30T22:18:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3900130530 con 0x7f390009e560 2023-09-30T22:18:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.198+0000 7f38e7fff700 1 -- 172.21.15.89:0/1796062527 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38f0003b80 con 0x7f390009e560 2023-09-30T22:18:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.198+0000 7f38e7fff700 1 -- 172.21.15.89:0/1796062527 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f38f0021070 con 0x7f390009e560 2023-09-30T22:18:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.197+0000 7f1e6f7fe700 1 -- 172.21.15.89:0/3920381280 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e7803faf0 con 0x7f1e8809d6f0 2023-09-30T22:18:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.198+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3900123190 con 0x7f390009e560 2023-09-30T22:18:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.200+0000 7f375d125700 1 Processor -- start 2023-09-30T22:18:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.200+0000 7f375d125700 1 -- start start 2023-09-30T22:18:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e881a7b50 con 0x7f1e8809d6f0 2023-09-30T22:18:49.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f375d125700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f375809d6f0 0x7f375809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f375d125700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f3758130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f375d125700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375809f130 con 0x7f375809e410 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f375d125700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375809f520 con 0x7f375809d6f0 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f375d125700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375809e930 con 0x7f375809e000 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f3756d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f375809d6f0 0x7f375809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f3756d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f375809d6f0 0x7f375809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45594/0 (socket says 172.21.15.89:45594) 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f3756d9d700 1 -- 172.21.15.89:0/2495784159 learned_addr learned my addr 172.21.15.89:0/2495784159 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f375659c700 1 --2- 172.21.15.89:0/2495784159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f3758130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f3756d9d700 1 -- 172.21.15.89:0/2495784159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 msgr2=0x7f3758130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f3756d9d700 1 --2- 172.21.15.89:0/2495784159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f3758130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f3756d9d700 1 -- 172.21.15.89:0/2495784159 >> v1:172.21.15.89:6789/0 conn(0x7f375809e410 legacy=0x7f3758130ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:18:49.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.201+0000 7f3756d9d700 1 -- 172.21.15.89:0/2495784159 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375809b2d0 con 0x7f375809d6f0 2023-09-30T22:18:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.202+0000 7f3756d9d700 1 --2- 172.21.15.89:0/2495784159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809d6f0 0x7f375809dac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f37480098f0 tx=0x7f3748009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a9f824931e178d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.203+0000 7f3755d9b700 1 -- 172.21.15.89:0/2495784159 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748004230 con 0x7f375809d6f0 2023-09-30T22:18:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.203+0000 7f3755d9b700 1 -- 172.21.15.89:0/2495784159 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3748004390 con 0x7f375809d6f0 2023-09-30T22:18:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.203+0000 7f3755d9b700 1 -- 172.21.15.89:0/2495784159 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37480036a0 con 0x7f375809d6f0 2023-09-30T22:18:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.203+0000 7f38e7fff700 1 -- 172.21.15.89:0/1796062527 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3900123190 con 0x7f390009e560 2023-09-30T22:18:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.204+0000 7f375d125700 1 -- 172.21.15.89:0/2495784159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809d6f0 msgr2=0x7f375809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.204+0000 7f375d125700 1 --2- 172.21.15.89:0/2495784159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809d6f0 0x7f375809dac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f37480098f0 tx=0x7f3748009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.204+0000 7f375d125700 1 -- 172.21.15.89:0/2495784159 shutdown_connections 2023-09-30T22:18:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.204+0000 7f375d125700 1 --2- 172.21.15.89:0/2495784159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809d6f0 0x7f375809dac0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.204+0000 7f375d125700 1 --2- 172.21.15.89:0/2495784159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f3758130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.204+0000 7f375d125700 1 -- 172.21.15.89:0/2495784159 >> 172.21.15.89:0/2495784159 conn(0x7f3758066f50 msgr2=0x7f375809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.204+0000 7f375d125700 1 -- 172.21.15.89:0/2495784159 shutdown_connections 2023-09-30T22:18:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.204+0000 7f375d125700 1 -- 172.21.15.89:0/2495784159 wait complete. 2023-09-30T22:18:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.205+0000 7f375d125700 1 Processor -- start 2023-09-30T22:18:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375d125700 1 -- start start 2023-09-30T22:18:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375d125700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f375809d6f0 0x7f37581cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375d125700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f37581ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375d125700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809e410 0x7f37581cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375d125700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3758128560 con 0x7f375809e410 2023-09-30T22:18:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375d125700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3758128370 con 0x7f375809e000 2023-09-30T22:18:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375d125700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3758130350 con 0x7f375809d6f0 2023-09-30T22:18:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f37581ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f37581ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35418/0 (socket says 172.21.15.89:35418) 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375659c700 1 -- 172.21.15.89:0/233523332 learned_addr learned my addr 172.21.15.89:0/233523332 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.206+0000 7f375759e700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809e410 0x7f37581cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375659c700 1 -- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f375809d6f0 msgr2=0x7f37581cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375659c700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f375809d6f0 0x7f37581cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375659c700 1 -- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809e410 msgr2=0x7f37581cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375659c700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809e410 0x7f37581cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375659c700 1 -- 172.21.15.89:0/233523332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3748009550 con 0x7f375809e000 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375759e700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809e410 0x7f37581cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375659c700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f37581ccb50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e9e0 tx=0x7f374c00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f373f7fe700 1 -- 172.21.15.89:0/233523332 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c009710 con 0x7f375809e000 2023-09-30T22:18:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f373f7fe700 1 -- 172.21.15.89:0/233523332 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c00c850 con 0x7f375809e000 2023-09-30T22:18:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f373f7fe700 1 -- 172.21.15.89:0/233523332 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00cbc0 con 0x7f375809e000 2023-09-30T22:18:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37580a1bd0 con 0x7f375809e000 2023-09-30T22:18:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.207+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3758129fa0 con 0x7f375809e000 2023-09-30T22:18:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.208+0000 7f373f7fe700 1 -- 172.21.15.89:0/233523332 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f374c00cd20 con 0x7f375809e000 2023-09-30T22:18:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.208+0000 7f373f7fe700 1 -- 172.21.15.89:0/233523332 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f374c03fcf0 con 0x7f375809e000 2023-09-30T22:18:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.211+0000 7f1e6f7fe700 1 -- 172.21.15.89:0/3920381280 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e78012930 con 0x7f1e8809d6f0 2023-09-30T22:18:49.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.211+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3758129e00 con 0x7f375809e000 2023-09-30T22:18:49.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.220+0000 7f373f7fe700 1 -- 172.21.15.89:0/233523332 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3758129e00 con 0x7f375809e000 2023-09-30T22:18:49.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.397+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3900123190 con 0x7f390009e560 2023-09-30T22:18:49.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.397+0000 7f38e7fff700 1 -- 172.21.15.89:0/1796062527 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3900123190 con 0x7f390009e560 2023-09-30T22:18:49.397 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.399+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f390009e560 msgr2=0x7f390011fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.399+0000 7f3905ae8700 1 --2- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f390009e560 0x7f390011fa30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f38f00090b0 tx=0x7f38f0004750 comp rx=0 tx=0).stop 2023-09-30T22:18:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.400+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 shutdown_connections 2023-09-30T22:18:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.400+0000 7f3905ae8700 1 --2- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39001341d0 0x7f39001204b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.400+0000 7f3905ae8700 1 --2- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f390009e560 0x7f390011fa30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.400+0000 7f3905ae8700 1 --2- 172.21.15.89:0/1796062527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39001338c0 0x7f390011ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.400+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 >> 172.21.15.89:0/1796062527 conn(0x7f39000a2aa0 msgr2=0x7f39001316c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.400+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 shutdown_connections 2023-09-30T22:18:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.400+0000 7f3905ae8700 1 -- 172.21.15.89:0/1796062527 wait complete. 2023-09-30T22:18:49.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.402+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9a540713a0 con 0x7f9a54130260 2023-09-30T22:18:49.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.403+0000 7f9a50ff9700 1 -- 172.21.15.89:0/2558177124 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a4c0102e0 con 0x7f9a54130260 2023-09-30T22:18:49.403 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a54130260 msgr2=0x7f9a541c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a54130260 0x7f9a541c8e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00cbc0 tx=0x7f9a4c00ced0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 shutdown_connections 2023-09-30T22:18:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a54130260 0x7f9a541c8e00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5412cf60 0x7f9a541c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 --2- 172.21.15.89:0/2558177124 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a5412f8b0 0x7f9a541c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 >> 172.21.15.89:0/2558177124 conn(0x7f9a540a2aa0 msgr2=0x7f9a5406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:49.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 shutdown_connections 2023-09-30T22:18:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.407+0000 7f9a5a3ea700 1 -- 172.21.15.89:0/2558177124 wait complete. 2023-09-30T22:18:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.409+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1e88074ea0 con 0x7f1e8809d6f0 2023-09-30T22:18:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.410+0000 7f1e6f7fe700 1 -- 172.21.15.89:0/3920381280 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e78012340 con 0x7f1e8809d6f0 2023-09-30T22:18:49.413 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:49.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e8809d6f0 msgr2=0x7f1e881a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 --2- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e8809d6f0 0x7f1e881a6020 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009d90 tx=0x7f1e7800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 shutdown_connections 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 --2- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e8809e410 0x7f1e881a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 --2- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e8809d6f0 0x7f1e881a6020 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 --2- 172.21.15.89:0/3920381280 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e8809e000 0x7f1e881a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 >> 172.21.15.89:0/3920381280 conn(0x7f1e88066f50 msgr2=0x7f1e881baf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 shutdown_connections 2023-09-30T22:18:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.416+0000 7f1e8d3d2700 1 -- 172.21.15.89:0/3920381280 wait complete. 2023-09-30T22:18:49.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:49.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:49.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.434+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3758129e00 con 0x7f375809e000 2023-09-30T22:18:49.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.434+0000 7f373f7fe700 1 -- 172.21.15.89:0/233523332 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3758129e00 con 0x7f375809e000 2023-09-30T22:18:49.434 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 msgr2=0x7f37581ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f37581ccb50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e9e0 tx=0x7f374c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 shutdown_connections 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f375809e410 0x7f37581cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f375809d6f0 0x7f37581cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 --2- 172.21.15.89:0/233523332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f375809e000 0x7f37581ccb50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 >> 172.21.15.89:0/233523332 conn(0x7f3758066f50 msgr2=0x7f375806f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 shutdown_connections 2023-09-30T22:18:49.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:49.436+0000 7f375d125700 1 -- 172.21.15.89:0/233523332 wait complete. 2023-09-30T22:18:49.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:50.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:50.421 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:50.429 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:50.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.551+0000 7f36704e3700 1 Processor -- start 2023-09-30T22:18:50.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.552+0000 7f36704e3700 1 -- start start 2023-09-30T22:18:50.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.552+0000 7f36704e3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f366809f410 0x7f3668136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.552+0000 7f36704e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f36681393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.552+0000 7f36704e3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3668070940 con 0x7f366809e930 2023-09-30T22:18:50.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.552+0000 7f36704e3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366806fd60 con 0x7f366809f410 2023-09-30T22:18:50.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.552+0000 7f36704e3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f366806fa80 con 0x7f3668136fe0 2023-09-30T22:18:50.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366da7e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f366809f410 0x7f3668136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366da7e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f366809f410 0x7f3668136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47090/0 (socket says 172.21.15.89:47090) 2023-09-30T22:18:50.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366da7e700 1 -- 172.21.15.89:0/2674173891 learned_addr learned my addr 172.21.15.89:0/2674173891 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:50.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366ea80700 1 --2- 172.21.15.89:0/2674173891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f36681393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366ea80700 1 -- 172.21.15.89:0/2674173891 >> v2:172.21.15.89:3300/0 conn(0x7f366809f410 msgr2=0x7f3668136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:50.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366ea80700 1 --2- 172.21.15.89:0/2674173891 >> v2:172.21.15.89:3300/0 conn(0x7f366809f410 0x7f3668136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366ea80700 1 -- 172.21.15.89:0/2674173891 >> v1:172.21.15.89:6789/0 conn(0x7f366809e930 legacy=0x7f366809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366ea80700 1 -- 172.21.15.89:0/2674173891 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366809fb60 con 0x7f3668136fe0 2023-09-30T22:18:50.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366da7e700 1 --2- 172.21.15.89:0/2674173891 >> v2:172.21.15.89:3300/0 conn(0x7f366809f410 0x7f3668136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:50.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366ea80700 1 --2- 172.21.15.89:0/2674173891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f36681393c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3660009d90 tx=0x7f366000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646c495a70d832ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:50.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366d27d700 1 -- 172.21.15.89:0/2674173891 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366000e070 con 0x7f3668136fe0 2023-09-30T22:18:50.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366d27d700 1 -- 172.21.15.89:0/2674173891 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366000bb20 con 0x7f3668136fe0 2023-09-30T22:18:50.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.553+0000 7f366d27d700 1 -- 172.21.15.89:0/2674173891 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660012430 con 0x7f3668136fe0 2023-09-30T22:18:50.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.554+0000 7f36704e3700 1 -- 172.21.15.89:0/2674173891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 msgr2=0x7f36681393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.554+0000 7f36704e3700 1 --2- 172.21.15.89:0/2674173891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f36681393c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3660009d90 tx=0x7f366000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:50.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.555+0000 7f36704e3700 1 -- 172.21.15.89:0/2674173891 shutdown_connections 2023-09-30T22:18:50.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.555+0000 7f36704e3700 1 --2- 172.21.15.89:0/2674173891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f36681393c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.555+0000 7f36704e3700 1 --2- 172.21.15.89:0/2674173891 >> v2:172.21.15.89:3300/0 conn(0x7f366809f410 0x7f3668136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.555+0000 7f36704e3700 1 -- 172.21.15.89:0/2674173891 >> 172.21.15.89:0/2674173891 conn(0x7f36680a4210 msgr2=0x7f36680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:50.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.555+0000 7f36704e3700 1 -- 172.21.15.89:0/2674173891 shutdown_connections 2023-09-30T22:18:50.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.555+0000 7f36704e3700 1 -- 172.21.15.89:0/2674173891 wait complete. 2023-09-30T22:18:50.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.556+0000 7f36704e3700 1 Processor -- start 2023-09-30T22:18:50.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.556+0000 7f36704e3700 1 -- start start 2023-09-30T22:18:50.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f36704e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366809e930 0x7f366811c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f36704e3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f366809f410 0x7f366811cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f36704e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f366811d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f36704e3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366811e9c0 con 0x7f366809e930 2023-09-30T22:18:50.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f36704e3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366811e370 con 0x7f366809f410 2023-09-30T22:18:50.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f36704e3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36681204a0 con 0x7f3668136fe0 2023-09-30T22:18:50.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366ea80700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f366811d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366e27f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366809e930 0x7f366811c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366ea80700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f366811d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37976/0 (socket says 172.21.15.89:37976) 2023-09-30T22:18:50.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366ea80700 1 -- 172.21.15.89:0/532113599 learned_addr learned my addr 172.21.15.89:0/532113599 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:50.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366e27f700 1 -- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 msgr2=0x7f366811d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366e27f700 1 --2- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f366811d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366e27f700 1 -- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f366809f410 msgr2=0x7f366811cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366e27f700 1 --2- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f366809f410 0x7f366811cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366e27f700 1 -- 172.21.15.89:0/532113599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009550 con 0x7f366809e930 2023-09-30T22:18:50.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366ea80700 1 --2- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f366811d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:50.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.557+0000 7f366e27f700 1 --2- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366809e930 0x7f366811c760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3664000f60 tx=0x7f366400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:50.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.558+0000 7f365effd700 1 -- 172.21.15.89:0/532113599 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36640052d0 con 0x7f366809e930 2023-09-30T22:18:50.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.558+0000 7f365effd700 1 -- 172.21.15.89:0/532113599 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3664005430 con 0x7f366809e930 2023-09-30T22:18:50.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.558+0000 7f365effd700 1 -- 172.21.15.89:0/532113599 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36640038c0 con 0x7f366809e930 2023-09-30T22:18:50.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.558+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3660009a70 con 0x7f366809e930 2023-09-30T22:18:50.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.558+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366809df90 con 0x7f366809e930 2023-09-30T22:18:50.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.559+0000 7f365effd700 1 -- 172.21.15.89:0/532113599 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3664003a20 con 0x7f366809e930 2023-09-30T22:18:50.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.559+0000 7f365effd700 1 -- 172.21.15.89:0/532113599 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f366403ea00 con 0x7f366809e930 2023-09-30T22:18:50.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.559+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3668074ea0 con 0x7f366809e930 2023-09-30T22:18:50.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.564+0000 7f365effd700 1 -- 172.21.15.89:0/532113599 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3664018260 con 0x7f366809e930 2023-09-30T22:18:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.592+0000 7f88082bc700 1 Processor -- start 2023-09-30T22:18:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.592+0000 7f88082bc700 1 -- start start 2023-09-30T22:18:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f094c981700 1 Processor -- start 2023-09-30T22:18:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f094c981700 1 -- start start 2023-09-30T22:18:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f88082bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009d6f0 0x7f880009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f88082bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e000 0x7f8800130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f88082bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f880009e410 0x7f8800130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f88082bc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880009f520 con 0x7f880009e410 2023-09-30T22:18:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f88082bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880009f130 con 0x7f880009e000 2023-09-30T22:18:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f88082bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f880009e930 con 0x7f880009d6f0 2023-09-30T22:18:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f752b211700 1 Processor -- start 2023-09-30T22:18:50.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f8805857700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e000 0x7f8800130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f8805857700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e000 0x7f8800130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45322/0 (socket says 172.21.15.89:45322) 2023-09-30T22:18:50.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f8805857700 1 -- 172.21.15.89:0/3450848659 learned_addr learned my addr 172.21.15.89:0/3450848659 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:50.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f752b211700 1 -- start start 2023-09-30T22:18:50.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f094c981700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f094809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f094c981700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f094809e000 0x7f09481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f094c981700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f0948133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f094c981700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094806b460 con 0x7f094809e000 2023-09-30T22:18:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f094c981700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094809b120 con 0x7f094809e410 2023-09-30T22:18:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.593+0000 7f094c981700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f094809a7e0 con 0x7f094809d6f0 2023-09-30T22:18:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f094809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f094809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37988/0 (socket says 172.21.15.89:37988) 2023-09-30T22:18:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 -- 172.21.15.89:0/4206615159 learned_addr learned my addr 172.21.15.89:0/4206615159 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f0946d9d700 1 --2- 172.21.15.89:0/4206615159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f0948133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f0945d9b700 1 --2- 172.21.15.89:0/4206615159 >> v2:172.21.15.89:3300/0 conn(0x7f094809e000 0x7f09481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8806058700 1 --2- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009d6f0 0x7f880009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8806859700 1 --2- 172.21.15.89:0/3450848659 >> v2:172.21.15.89:3300/0 conn(0x7f880009e410 0x7f8800130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805857700 1 -- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009d6f0 msgr2=0x7f880009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805857700 1 --2- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009d6f0 0x7f880009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805857700 1 -- 172.21.15.89:0/3450848659 >> v2:172.21.15.89:3300/0 conn(0x7f880009e410 msgr2=0x7f8800130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805857700 1 --2- 172.21.15.89:0/3450848659 >> v2:172.21.15.89:3300/0 conn(0x7f880009e410 0x7f8800130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805857700 1 -- 172.21.15.89:0/3450848659 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880009b2d0 con 0x7f880009e000 2023-09-30T22:18:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8806058700 1 --2- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009d6f0 0x7f880009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8806859700 1 --2- 172.21.15.89:0/3450848659 >> v2:172.21.15.89:3300/0 conn(0x7f880009e410 0x7f8800130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805857700 1 --2- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e000 0x7f8800130660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f87f000bbc0 tx=0x7f87f000bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0e022235244cb62 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 -- 172.21.15.89:0/4206615159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 msgr2=0x7f0948133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 --2- 172.21.15.89:0/4206615159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f0948133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 -- 172.21.15.89:0/4206615159 >> v2:172.21.15.89:3300/0 conn(0x7f094809e000 msgr2=0x7f09481339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 --2- 172.21.15.89:0/4206615159 >> v2:172.21.15.89:3300/0 conn(0x7f094809e000 0x7f09481339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 -- 172.21.15.89:0/4206615159 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094809a500 con 0x7f094809d6f0 2023-09-30T22:18:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094659c700 1 --2- 172.21.15.89:0/4206615159 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f094809dac0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0938009920 tx=0x7f0938009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21596d886722e89 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805056700 1 -- 172.21.15.89:0/3450848659 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000dd50 con 0x7f880009e000 2023-09-30T22:18:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805056700 1 -- 172.21.15.89:0/3450848659 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f0010470 con 0x7f880009e000 2023-09-30T22:18:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f8805056700 1 -- 172.21.15.89:0/3450848659 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0010800 con 0x7f880009e000 2023-09-30T22:18:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.594+0000 7f094559a700 1 -- 172.21.15.89:0/4206615159 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938011830 con 0x7f094809d6f0 2023-09-30T22:18:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f094559a700 1 -- 172.21.15.89:0/4206615159 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0938011990 con 0x7f094809d6f0 2023-09-30T22:18:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f094559a700 1 -- 172.21.15.89:0/4206615159 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938011d20 con 0x7f094809d6f0 2023-09-30T22:18:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f752b211700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409e1d0 0x7f7524135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f752b211700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7524135e50 0x7f7524138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f752b211700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752409f130 con 0x7f752409d6f0 2023-09-30T22:18:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f752b211700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752409f520 con 0x7f7524135e50 2023-09-30T22:18:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f752b211700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752409e930 con 0x7f752409e1d0 2023-09-30T22:18:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f7523fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409e1d0 0x7f7524135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f7523fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409e1d0 0x7f7524135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45334/0 (socket says 172.21.15.89:45334) 2023-09-30T22:18:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f7523fff700 1 -- 172.21.15.89:0/646462626 learned_addr learned my addr 172.21.15.89:0/646462626 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f75297ae700 1 --2- 172.21.15.89:0/646462626 >> v2:172.21.15.89:3300/0 conn(0x7f7524135e50 0x7f7524138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f7523fff700 1 -- 172.21.15.89:0/646462626 >> v2:172.21.15.89:3300/0 conn(0x7f7524135e50 msgr2=0x7f7524138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f7523fff700 1 --2- 172.21.15.89:0/646462626 >> v2:172.21.15.89:3300/0 conn(0x7f7524135e50 0x7f7524138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f7523fff700 1 -- 172.21.15.89:0/646462626 >> v1:172.21.15.89:6789/0 conn(0x7f752409d6f0 legacy=0x7f752409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f7523fff700 1 -- 172.21.15.89:0/646462626 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752409b2d0 con 0x7f752409e1d0 2023-09-30T22:18:50.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f88082bc700 1 -- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e000 msgr2=0x7f8800130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f88082bc700 1 --2- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e000 0x7f8800130660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f87f000bbc0 tx=0x7f87f000bed0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f094c981700 1 -- 172.21.15.89:0/4206615159 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 msgr2=0x7f094809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f094c981700 1 --2- 172.21.15.89:0/4206615159 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f094809dac0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0938009920 tx=0x7f0938009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:50.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f88082bc700 1 -- 172.21.15.89:0/3450848659 shutdown_connections 2023-09-30T22:18:50.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f88082bc700 1 --2- 172.21.15.89:0/3450848659 >> v2:172.21.15.89:3300/0 conn(0x7f880009e410 0x7f8800130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f88082bc700 1 --2- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009d6f0 0x7f880009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f88082bc700 1 --2- 172.21.15.89:0/3450848659 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e000 0x7f8800130660 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f88082bc700 1 -- 172.21.15.89:0/3450848659 >> 172.21.15.89:0/3450848659 conn(0x7f8800066f50 msgr2=0x7f880009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:50.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f7523fff700 1 --2- 172.21.15.89:0/646462626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409e1d0 0x7f7524135910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7510009920 tx=0x7f7510009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f0d5dd330ee83c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:50.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.595+0000 7f88082bc700 1 -- 172.21.15.89:0/3450848659 shutdown_connections 2023-09-30T22:18:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f88082bc700 1 -- 172.21.15.89:0/3450848659 wait complete. 2023-09-30T22:18:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f094c981700 1 -- 172.21.15.89:0/4206615159 shutdown_connections 2023-09-30T22:18:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f094c981700 1 --2- 172.21.15.89:0/4206615159 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f0948133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f094c981700 1 --2- 172.21.15.89:0/4206615159 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f094809dac0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f094c981700 1 --2- 172.21.15.89:0/4206615159 >> v2:172.21.15.89:3300/0 conn(0x7f094809e000 0x7f09481339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f094c981700 1 -- 172.21.15.89:0/4206615159 >> 172.21.15.89:0/4206615159 conn(0x7f0948066f50 msgr2=0x7f094809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:50.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f75237fe700 1 -- 172.21.15.89:0/646462626 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510011830 con 0x7f752409e1d0 2023-09-30T22:18:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f094c981700 1 -- 172.21.15.89:0/4206615159 shutdown_connections 2023-09-30T22:18:50.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f094c981700 1 -- 172.21.15.89:0/4206615159 wait complete. 2023-09-30T22:18:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f75237fe700 1 -- 172.21.15.89:0/646462626 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7510011990 con 0x7f752409e1d0 2023-09-30T22:18:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f75237fe700 1 -- 172.21.15.89:0/646462626 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510011d20 con 0x7f752409e1d0 2023-09-30T22:18:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f752b211700 1 -- 172.21.15.89:0/646462626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409e1d0 msgr2=0x7f7524135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f752b211700 1 --2- 172.21.15.89:0/646462626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409e1d0 0x7f7524135910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7510009920 tx=0x7f7510009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f752b211700 1 -- 172.21.15.89:0/646462626 shutdown_connections 2023-09-30T22:18:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f752b211700 1 --2- 172.21.15.89:0/646462626 >> v2:172.21.15.89:3300/0 conn(0x7f7524135e50 0x7f7524138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f752b211700 1 --2- 172.21.15.89:0/646462626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409e1d0 0x7f7524135910 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f752b211700 1 -- 172.21.15.89:0/646462626 >> 172.21.15.89:0/646462626 conn(0x7f7524066f50 msgr2=0x7f752409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.596+0000 7f88082bc700 1 Processor -- start 2023-09-30T22:18:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f752b211700 1 -- 172.21.15.89:0/646462626 shutdown_connections 2023-09-30T22:18:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f752b211700 1 -- 172.21.15.89:0/646462626 wait complete. 2023-09-30T22:18:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f094c981700 1 Processor -- start 2023-09-30T22:18:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f094c981700 1 -- start start 2023-09-30T22:18:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f094c981700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f09481cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f094c981700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094809e000 0x7f09481ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f094c981700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f094812d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f094c981700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0948130c40 con 0x7f094809e000 2023-09-30T22:18:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f094c981700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09481cc490 con 0x7f094809e410 2023-09-30T22:18:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f094c981700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f094813a350 con 0x7f094809d6f0 2023-09-30T22:18:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f0946d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f094812d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f0946d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f094812d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45336/0 (socket says 172.21.15.89:45336) 2023-09-30T22:18:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f0946d9d700 1 -- 172.21.15.89:0/3260484190 learned_addr learned my addr 172.21.15.89:0/3260484190 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f752b211700 1 Processor -- start 2023-09-30T22:18:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f752b211700 1 -- start start 2023-09-30T22:18:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.597+0000 7f0945d9b700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094809e000 0x7f09481ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f094659c700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f09481cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f88082bc700 1 -- start start 2023-09-30T22:18:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f0946d9d700 1 -- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 msgr2=0x7f09481cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f0946d9d700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f09481cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f0946d9d700 1 -- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094809e000 msgr2=0x7f09481ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f0946d9d700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094809e000 0x7f09481ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f0946d9d700 1 -- 172.21.15.89:0/3260484190 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0938009550 con 0x7f094809e410 2023-09-30T22:18:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f0945d9b700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094809e000 0x7f09481ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f094659c700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f09481cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f0946d9d700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f094812d7d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f094000efc0 tx=0x7f094000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f752b211700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409d6f0 0x7f7524134db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f752b211700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f752409e1d0 0x7f75241352f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f752b211700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7524135e50 0x7f7524135830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f752b211700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7524132890 con 0x7f752409e1d0 2023-09-30T22:18:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f752b211700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7524133f60 con 0x7f752409d6f0 2023-09-30T22:18:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f752b211700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75241341d0 con 0x7f7524135e50 2023-09-30T22:18:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f88082bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f880009d6f0 0x7f880011dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f88082bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009e000 0x7f880011e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f88082bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e410 0x7f880011ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f88082bc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8800121ab0 con 0x7f880009d6f0 2023-09-30T22:18:50.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f88082bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8800120180 con 0x7f880009e410 2023-09-30T22:18:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f88082bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8800121240 con 0x7f880009e000 2023-09-30T22:18:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f75297ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7524135e50 0x7f7524135830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f75297ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7524135e50 0x7f7524135830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38014/0 (socket says 172.21.15.89:38014) 2023-09-30T22:18:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f7528fad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409d6f0 0x7f7524134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f75297ae700 1 -- 172.21.15.89:0/1742616670 learned_addr learned my addr 172.21.15.89:0/1742616670 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f8805857700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009e000 0x7f880011e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f8805857700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009e000 0x7f880011e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38012/0 (socket says 172.21.15.89:38012) 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f8805857700 1 -- 172.21.15.89:0/1734247794 learned_addr learned my addr 172.21.15.89:0/1734247794 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.598+0000 7f8806859700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e410 0x7f880011ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f8806058700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f880009d6f0 0x7f880011dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f8805857700 1 -- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e410 msgr2=0x7f880011ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f8805857700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e410 0x7f880011ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f8805857700 1 -- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f880009d6f0 msgr2=0x7f880011dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f8805857700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f880009d6f0 0x7f880011dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f8805857700 1 -- 172.21.15.89:0/1734247794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f000b7f0 con 0x7f880009e000 2023-09-30T22:18:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f8805857700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009e000 0x7f880011e520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f87f000bf10 tx=0x7f87f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f0936ffd700 1 -- 172.21.15.89:0/3260484190 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000cd40 con 0x7f094809e410 2023-09-30T22:18:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f0936ffd700 1 -- 172.21.15.89:0/3260484190 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000f040 con 0x7f094809e410 2023-09-30T22:18:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f0936ffd700 1 -- 172.21.15.89:0/3260484190 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400146c0 con 0x7f094809e410 2023-09-30T22:18:50.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f7523fff700 1 --2- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f752409e1d0 0x7f75241352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f7528fad700 1 -- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7524135e50 msgr2=0x7f7524135830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f7528fad700 1 --2- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7524135e50 0x7f7524135830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f7528fad700 1 -- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f752409e1d0 msgr2=0x7f75241352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f7528fad700 1 --2- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f752409e1d0 0x7f75241352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f7528fad700 1 -- 172.21.15.89:0/1742616670 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7510009550 con 0x7f752409d6f0 2023-09-30T22:18:50.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f7528fad700 1 --2- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409d6f0 0x7f7524134db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7518000f60 tx=0x7f751800c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094812fcc0 con 0x7f094809e410 2023-09-30T22:18:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09481341b0 con 0x7f094809e410 2023-09-30T22:18:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.599+0000 7f87f6ffd700 1 -- 172.21.15.89:0/1734247794 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f001d070 con 0x7f880009e000 2023-09-30T22:18:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.600+0000 7f87f6ffd700 1 -- 172.21.15.89:0/1734247794 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f00038b0 con 0x7f880009e000 2023-09-30T22:18:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.600+0000 7f87f6ffd700 1 -- 172.21.15.89:0/1734247794 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0003c40 con 0x7f880009e000 2023-09-30T22:18:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.600+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880009c700 con 0x7f880009e000 2023-09-30T22:18:50.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.600+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8800130370 con 0x7f880009e000 2023-09-30T22:18:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.600+0000 7f0936ffd700 1 -- 172.21.15.89:0/3260484190 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0940014820 con 0x7f094809e410 2023-09-30T22:18:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.600+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0948074ea0 con 0x7f094809e410 2023-09-30T22:18:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.600+0000 7f0936ffd700 1 -- 172.21.15.89:0/3260484190 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0940040220 con 0x7f094809e410 2023-09-30T22:18:50.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.602+0000 7f75217fa700 1 -- 172.21.15.89:0/1742616670 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180052d0 con 0x7f752409d6f0 2023-09-30T22:18:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.602+0000 7f75217fa700 1 -- 172.21.15.89:0/1742616670 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7518005430 con 0x7f752409d6f0 2023-09-30T22:18:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.602+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7524133510 con 0x7f752409d6f0 2023-09-30T22:18:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.602+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752409c700 con 0x7f752409d6f0 2023-09-30T22:18:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.602+0000 7f75217fa700 1 -- 172.21.15.89:0/1742616670 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180038a0 con 0x7f752409d6f0 2023-09-30T22:18:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.603+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7524130400 con 0x7f752409d6f0 2023-09-30T22:18:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.601+0000 7f87f6ffd700 1 -- 172.21.15.89:0/1734247794 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f87f0019400 con 0x7f880009e000 2023-09-30T22:18:50.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.601+0000 7f87f6ffd700 1 -- 172.21.15.89:0/1734247794 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f87f0040920 con 0x7f880009e000 2023-09-30T22:18:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.601+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8800121470 con 0x7f880009e000 2023-09-30T22:18:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.606+0000 7f0936ffd700 1 -- 172.21.15.89:0/3260484190 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0940007920 con 0x7f094809e410 2023-09-30T22:18:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.606+0000 7f75217fa700 1 -- 172.21.15.89:0/1742616670 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7518003a00 con 0x7f752409d6f0 2023-09-30T22:18:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.606+0000 7f75217fa700 1 -- 172.21.15.89:0/1742616670 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f751803f4f0 con 0x7f752409d6f0 2023-09-30T22:18:50.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.606+0000 7f87f6ffd700 1 -- 172.21.15.89:0/1734247794 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8800121470 con 0x7f880009e000 2023-09-30T22:18:50.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.610+0000 7f75217fa700 1 -- 172.21.15.89:0/1742616670 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7518003cb0 con 0x7f752409d6f0 2023-09-30T22:18:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.792+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3668120000 con 0x7f366809e930 2023-09-30T22:18:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.792+0000 7f365effd700 1 -- 172.21.15.89:0/532113599 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3664017020 con 0x7f366809e930 2023-09-30T22:18:50.792 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.795+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366809e930 msgr2=0x7f366811c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.795+0000 7f36704e3700 1 --2- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366809e930 0x7f366811c760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3664000f60 tx=0x7f366400c730 comp rx=0 tx=0).stop 2023-09-30T22:18:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.795+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 shutdown_connections 2023-09-30T22:18:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.795+0000 7f36704e3700 1 --2- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3668136fe0 0x7f366811d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.795+0000 7f36704e3700 1 --2- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366809e930 0x7f366811c760 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.795+0000 7f36704e3700 1 --2- 172.21.15.89:0/532113599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f366809f410 0x7f366811cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.795+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 >> 172.21.15.89:0/532113599 conn(0x7f36680a4210 msgr2=0x7f3668134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.796+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 shutdown_connections 2023-09-30T22:18:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.796+0000 7f36704e3700 1 -- 172.21.15.89:0/532113599 wait complete. 2023-09-30T22:18:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.802+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7524074ea0 con 0x7f752409d6f0 2023-09-30T22:18:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.802+0000 7f75217fa700 1 -- 172.21.15.89:0/1742616670 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7518017070 con 0x7f752409d6f0 2023-09-30T22:18:50.802 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.803+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8800121470 con 0x7f880009e000 2023-09-30T22:18:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.804+0000 7f87f6ffd700 1 -- 172.21.15.89:0/1734247794 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8800121470 con 0x7f880009e000 2023-09-30T22:18:50.803 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.805+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409d6f0 msgr2=0x7f7524134db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.805+0000 7f752b211700 1 --2- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409d6f0 0x7f7524134db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7518000f60 tx=0x7f751800c790 comp rx=0 tx=0).stop 2023-09-30T22:18:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 shutdown_connections 2023-09-30T22:18:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f752b211700 1 --2- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7524135e50 0x7f7524135830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f752b211700 1 --2- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f752409d6f0 0x7f7524134db0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f752b211700 1 --2- 172.21.15.89:0/1742616670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f752409e1d0 0x7f75241352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 >> 172.21.15.89:0/1742616670 conn(0x7f7524066f50 msgr2=0x7f75240a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 shutdown_connections 2023-09-30T22:18:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f752b211700 1 -- 172.21.15.89:0/1742616670 wait complete. 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009e000 msgr2=0x7f880011e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009e000 0x7f880011e520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f87f000bf10 tx=0x7f87f0004750 comp rx=0 tx=0).stop 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 shutdown_connections 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f880009e410 0x7f880011ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f880009d6f0 0x7f880011dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 --2- 172.21.15.89:0/1734247794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f880009e000 0x7f880011e520 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 >> 172.21.15.89:0/1734247794 conn(0x7f8800066f50 msgr2=0x7f8800130900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 shutdown_connections 2023-09-30T22:18:50.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.806+0000 7f88082bc700 1 -- 172.21.15.89:0/1734247794 wait complete. 2023-09-30T22:18:50.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:50.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:50.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.832+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f094812fe50 con 0x7f094809e410 2023-09-30T22:18:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.832+0000 7f0936ffd700 1 -- 172.21.15.89:0/3260484190 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0940018030 con 0x7f094809e410 2023-09-30T22:18:50.832 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 msgr2=0x7f094812d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f094812d7d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f094000efc0 tx=0x7f094000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 shutdown_connections 2023-09-30T22:18:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f094809e410 0x7f094812d7d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f094809d6f0 0x7f09481cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 --2- 172.21.15.89:0/3260484190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f094809e000 0x7f09481ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 >> 172.21.15.89:0/3260484190 conn(0x7f0948066f50 msgr2=0x7f09480a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 shutdown_connections 2023-09-30T22:18:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:50.834+0000 7f094c981700 1 -- 172.21.15.89:0/3260484190 wait complete. 2023-09-30T22:18:50.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:51.812 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:51.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:51.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:51.847 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:51.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.966+0000 7f2c1b685700 1 Processor -- start 2023-09-30T22:18:51.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.966+0000 7f2c1b685700 1 -- start start 2023-09-30T22:18:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c1b685700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c14130750 0x7f2c14130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c1b685700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c1409d6f0 0x7f2c1409db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c1b685700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1409c190 con 0x7f2c1409e080 2023-09-30T22:18:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c1b685700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1409b090 con 0x7f2c1409d6f0 2023-09-30T22:18:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c1b685700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1409a6c0 con 0x7f2c14130750 2023-09-30T22:18:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c18c20700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c1409d6f0 0x7f2c1409db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c19421700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c14130750 0x7f2c14130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c18c20700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c1409d6f0 0x7f2c1409db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45358/0 (socket says 172.21.15.89:45358) 2023-09-30T22:18:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c18c20700 1 -- 172.21.15.89:0/1375643998 learned_addr learned my addr 172.21.15.89:0/1375643998 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c19c22700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2c1409e080 0x7f2c141374b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40976/0 (socket says 172.21.15.89:40976) 2023-09-30T22:18:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c19421700 1 -- 172.21.15.89:0/1375643998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c1409d6f0 msgr2=0x7f2c1409db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c0bfff700 1 -- 172.21.15.89:0/1375643998 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165651727 0 0) 0x7f2c1409c190 con 0x7f2c1409e080 2023-09-30T22:18:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c19421700 1 --2- 172.21.15.89:0/1375643998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c1409d6f0 0x7f2c1409db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c19421700 1 -- 172.21.15.89:0/1375643998 >> v1:172.21.15.89:6789/0 conn(0x7f2c1409e080 legacy=0x7f2c141374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.967+0000 7f2c19421700 1 -- 172.21.15.89:0/1375643998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14131060 con 0x7f2c14130750 2023-09-30T22:18:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.968+0000 7f2c19421700 1 --2- 172.21.15.89:0/1375643998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c14130750 0x7f2c14130b20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009870 tx=0x7f2c10009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3345975b6ec552f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.968+0000 7f2c0bfff700 1 -- 172.21.15.89:0/1375643998 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10011830 con 0x7f2c14130750 2023-09-30T22:18:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.968+0000 7f2c0bfff700 1 -- 172.21.15.89:0/1375643998 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c10011990 con 0x7f2c14130750 2023-09-30T22:18:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.968+0000 7f2c0bfff700 1 -- 172.21.15.89:0/1375643998 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10011d20 con 0x7f2c14130750 2023-09-30T22:18:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.969+0000 7f2c1b685700 1 -- 172.21.15.89:0/1375643998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c14130750 msgr2=0x7f2c14130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.969+0000 7f2c1b685700 1 --2- 172.21.15.89:0/1375643998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c14130750 0x7f2c14130b20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009870 tx=0x7f2c10009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.969+0000 7f2c1b685700 1 -- 172.21.15.89:0/1375643998 shutdown_connections 2023-09-30T22:18:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.969+0000 7f2c1b685700 1 --2- 172.21.15.89:0/1375643998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c14130750 0x7f2c14130b20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.969+0000 7f2c1b685700 1 --2- 172.21.15.89:0/1375643998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c1409d6f0 0x7f2c1409db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.969+0000 7f2c1b685700 1 -- 172.21.15.89:0/1375643998 >> 172.21.15.89:0/1375643998 conn(0x7f2c1409f470 msgr2=0x7f2c140a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.969+0000 7f2c1b685700 1 -- 172.21.15.89:0/1375643998 shutdown_connections 2023-09-30T22:18:51.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.969+0000 7f2c1b685700 1 -- 172.21.15.89:0/1375643998 wait complete. 2023-09-30T22:18:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.970+0000 7f2c1b685700 1 Processor -- start 2023-09-30T22:18:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.970+0000 7f2c1b685700 1 -- start start 2023-09-30T22:18:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c1b685700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1409d6f0 0x7f2c1412abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c1b685700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409e080 0x7f2c1412b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c1b685700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14130750 0x7f2c1412b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c1b685700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c140a13d0 con 0x7f2c1409d6f0 2023-09-30T22:18:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c1b685700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1412df40 con 0x7f2c14130750 2023-09-30T22:18:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c1b685700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1412fd50 con 0x7f2c1409e080 2023-09-30T22:18:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c18c20700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409e080 0x7f2c1412b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c18c20700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409e080 0x7f2c1412b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38048/0 (socket says 172.21.15.89:38048) 2023-09-30T22:18:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c18c20700 1 -- 172.21.15.89:0/1623452834 learned_addr learned my addr 172.21.15.89:0/1623452834 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c19c22700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14130750 0x7f2c1412b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c19421700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1409d6f0 0x7f2c1412abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c19c22700 1 -- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409e080 msgr2=0x7f2c1412b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c19c22700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409e080 0x7f2c1412b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c19c22700 1 -- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1409d6f0 msgr2=0x7f2c1412abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c19c22700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1409d6f0 0x7f2c1412abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.971+0000 7f2c19c22700 1 -- 172.21.15.89:0/1623452834 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c100094a0 con 0x7f2c14130750 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.972+0000 7f2c19c22700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14130750 0x7f2c1412b650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00cc10 tx=0x7f2c0c00cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.972+0000 7f2c09ffb700 1 -- 172.21.15.89:0/1623452834 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00aa60 con 0x7f2c14130750 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.972+0000 7f2c09ffb700 1 -- 172.21.15.89:0/1623452834 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0c00abc0 con 0x7f2c14130750 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.972+0000 7f2c09ffb700 1 -- 172.21.15.89:0/1623452834 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c0070d0 con 0x7f2c14130750 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.972+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1409b2b0 con 0x7f2c14130750 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.973+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1412d2f0 con 0x7f2c14130750 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.973+0000 7f2c09ffb700 1 -- 172.21.15.89:0/1623452834 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c0c007230 con 0x7f2c14130750 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.973+0000 7f2c09ffb700 1 -- 172.21.15.89:0/1623452834 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2c0c0074e0 con 0x7f2c14130750 2023-09-30T22:18:51.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.974+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c141b7fd0 con 0x7f2c14130750 2023-09-30T22:18:51.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.982+0000 7f2c09ffb700 1 -- 172.21.15.89:0/1623452834 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c0c019680 con 0x7f2c14130750 2023-09-30T22:18:51.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.993+0000 7f87f4fd2700 1 Processor -- start 2023-09-30T22:18:51.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.993+0000 7f87f4fd2700 1 -- start start 2023-09-30T22:18:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87f4fd2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f87f009de00 0x7f87f009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87f4fd2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87f4fd2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f009a810 con 0x7f87f006a770 2023-09-30T22:18:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87f4fd2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f006ab40 con 0x7f87f009de00 2023-09-30T22:18:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87f4fd2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f009ebc0 con 0x7f87f013d1f0 2023-09-30T22:18:51.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87ee59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f87f006a770 0x7f87f009d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40988/0 (socket says 172.21.15.89:40988) 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87ee59c700 1 -- 172.21.15.89:0/4067590380 learned_addr learned my addr 172.21.15.89:0/4067590380 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87eed9d700 1 --2- 172.21.15.89:0/4067590380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.994+0000 7f87edd9b700 1 --2- 172.21.15.89:0/4067590380 >> v2:172.21.15.89:3300/0 conn(0x7f87f009de00 0x7f87f009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87eed9d700 1 -- 172.21.15.89:0/4067590380 >> v2:172.21.15.89:3300/0 conn(0x7f87f009de00 msgr2=0x7f87f009e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87eed9d700 1 --2- 172.21.15.89:0/4067590380 >> v2:172.21.15.89:3300/0 conn(0x7f87f009de00 0x7f87f009e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87eed9d700 1 -- 172.21.15.89:0/4067590380 >> v1:172.21.15.89:6789/0 conn(0x7f87f006a770 legacy=0x7f87f009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87eed9d700 1 -- 172.21.15.89:0/4067590380 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f009e920 con 0x7f87f013d1f0 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87ed59a700 1 -- 172.21.15.89:0/4067590380 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879476969 0 0) 0x7f87f009a810 con 0x7f87f006a770 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87eed9d700 1 --2- 172.21.15.89:0/4067590380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f013f5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009d90 tx=0x7f87e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb7cc06908ab8a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87ed59a700 1 -- 172.21.15.89:0/4067590380 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e800e070 con 0x7f87f013d1f0 2023-09-30T22:18:51.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87ed59a700 1 -- 172.21.15.89:0/4067590380 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e800bb20 con 0x7f87f013d1f0 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87ed59a700 1 -- 172.21.15.89:0/4067590380 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8012430 con 0x7f87f013d1f0 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87f4fd2700 1 -- 172.21.15.89:0/4067590380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 msgr2=0x7f87f013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.995+0000 7f87f4fd2700 1 --2- 172.21.15.89:0/4067590380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f013f5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009d90 tx=0x7f87e800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.996+0000 7f87f4fd2700 1 -- 172.21.15.89:0/4067590380 shutdown_connections 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.996+0000 7f87f4fd2700 1 --2- 172.21.15.89:0/4067590380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f013f5d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.996+0000 7f87f4fd2700 1 --2- 172.21.15.89:0/4067590380 >> v2:172.21.15.89:3300/0 conn(0x7f87f009de00 0x7f87f009e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.996+0000 7f87f4fd2700 1 -- 172.21.15.89:0/4067590380 >> 172.21.15.89:0/4067590380 conn(0x7f87f0066f50 msgr2=0x7f87f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.996+0000 7f87f4fd2700 1 -- 172.21.15.89:0/4067590380 shutdown_connections 2023-09-30T22:18:51.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.996+0000 7f87f4fd2700 1 -- 172.21.15.89:0/4067590380 wait complete. 2023-09-30T22:18:51.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7fe48eba8700 1 Processor -- start 2023-09-30T22:18:51.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7fe48eba8700 1 -- start start 2023-09-30T22:18:51.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7fe48eba8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe48809d6f0 0x7fe48809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7fe48eba8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 0x7fe4881339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:51.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7fe48eba8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe488133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7fe48eba8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48806b460 con 0x7fe48809d6f0 2023-09-30T22:18:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7fe48eba8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48809b120 con 0x7fe48809e410 2023-09-30T22:18:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7fe48eba8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe48809a7e0 con 0x7fe48809e000 2023-09-30T22:18:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87f4fd2700 1 Processor -- start 2023-09-30T22:18:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87f4fd2700 1 -- start start 2023-09-30T22:18:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87f4fd2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87f006a770 0x7f87f011b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87f4fd2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f009de00 0x7f87f011bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87f4fd2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f011c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87f4fd2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f011d280 con 0x7f87f009de00 2023-09-30T22:18:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87f4fd2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0119df0 con 0x7f87f013d1f0 2023-09-30T22:18:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87f4fd2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0071900 con 0x7f87f006a770 2023-09-30T22:18:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87eed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f011c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87eed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f011c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45386/0 (socket says 172.21.15.89:45386) 2023-09-30T22:18:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.997+0000 7f87eed9d700 1 -- 172.21.15.89:0/2316305728 learned_addr learned my addr 172.21.15.89:0/2316305728 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe488133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe488133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45400/0 (socket says 172.21.15.89:45400) 2023-09-30T22:18:52.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 -- 172.21.15.89:0/2253770892 learned_addr learned my addr 172.21.15.89:0/2253770892 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:52.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87edd9b700 1 --2- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f009de00 0x7f87f011bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48c944700 1 --2- 172.21.15.89:0/2253770892 >> v2:172.21.15.89:3300/0 conn(0x7fe48809d6f0 0x7fe48809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87eed9d700 1 -- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87f006a770 msgr2=0x7f87f011b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87eed9d700 1 --2- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87f006a770 0x7f87f011b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87eed9d700 1 -- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f009de00 msgr2=0x7f87f011bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87eed9d700 1 --2- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f009de00 0x7f87f011bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87eed9d700 1 -- 172.21.15.89:0/2316305728 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e8009a70 con 0x7f87f013d1f0 2023-09-30T22:18:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87eed9d700 1 --2- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f011c1c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f87f009a9a0 tx=0x7f87e8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 -- 172.21.15.89:0/2253770892 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 msgr2=0x7fe4881339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 --2- 172.21.15.89:0/2253770892 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 0x7fe4881339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 -- 172.21.15.89:0/2253770892 >> v2:172.21.15.89:3300/0 conn(0x7fe48809d6f0 msgr2=0x7fe48809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 --2- 172.21.15.89:0/2253770892 >> v2:172.21.15.89:3300/0 conn(0x7fe48809d6f0 0x7fe48809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 -- 172.21.15.89:0/2253770892 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48809a500 con 0x7fe48809e410 2023-09-30T22:18:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87deffd700 1 -- 172.21.15.89:0/2316305728 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8012430 con 0x7f87f013d1f0 2023-09-30T22:18:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87deffd700 1 -- 172.21.15.89:0/2316305728 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e8020070 con 0x7f87f013d1f0 2023-09-30T22:18:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f0072670 con 0x7f87f013d1f0 2023-09-30T22:18:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87deffd700 1 -- 172.21.15.89:0/2316305728 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8004280 con 0x7f87f013d1f0 2023-09-30T22:18:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f00a35a0 con 0x7f87f013d1f0 2023-09-30T22:18:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.998+0000 7fe48d145700 1 --2- 172.21.15.89:0/2253770892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe488133f30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe480009e80 tx=0x7fe48000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459c1a1c007d2e18 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.999+0000 7fe4877fe700 1 -- 172.21.15.89:0/2253770892 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48000e070 con 0x7fe48809e410 2023-09-30T22:18:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.999+0000 7fe4877fe700 1 -- 172.21.15.89:0/2253770892 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48000b980 con 0x7fe48809e410 2023-09-30T22:18:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.999+0000 7fe4877fe700 1 -- 172.21.15.89:0/2253770892 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48000bd30 con 0x7fe48809e410 2023-09-30T22:18:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.999+0000 7fe48eba8700 1 -- 172.21.15.89:0/2253770892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 msgr2=0x7fe488133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:51.999+0000 7fe48eba8700 1 --2- 172.21.15.89:0/2253770892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe488133f30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe480009e80 tx=0x7fe48000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:52.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.000+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f0074ea0 con 0x7f87f013d1f0 2023-09-30T22:18:52.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.000+0000 7f87deffd700 1 -- 172.21.15.89:0/2316305728 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f87e80043e0 con 0x7f87f013d1f0 2023-09-30T22:18:52.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.000+0000 7f87deffd700 1 -- 172.21.15.89:0/2316305728 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f87e8022070 con 0x7f87f013d1f0 2023-09-30T22:18:52.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.001+0000 7fe48eba8700 1 -- 172.21.15.89:0/2253770892 shutdown_connections 2023-09-30T22:18:52.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.001+0000 7fe48eba8700 1 --2- 172.21.15.89:0/2253770892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe488133f30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.001+0000 7fe48eba8700 1 --2- 172.21.15.89:0/2253770892 >> v2:172.21.15.89:3300/0 conn(0x7fe48809d6f0 0x7fe48809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.001+0000 7fe48eba8700 1 --2- 172.21.15.89:0/2253770892 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 0x7fe4881339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.001+0000 7fe48eba8700 1 -- 172.21.15.89:0/2253770892 >> 172.21.15.89:0/2253770892 conn(0x7fe488066f50 msgr2=0x7fe48809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:52.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.003+0000 7fe48eba8700 1 -- 172.21.15.89:0/2253770892 shutdown_connections 2023-09-30T22:18:52.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.003+0000 7fe48eba8700 1 -- 172.21.15.89:0/2253770892 wait complete. 2023-09-30T22:18:52.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.004+0000 7fe48eba8700 1 Processor -- start 2023-09-30T22:18:52.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.004+0000 7fe48eba8700 1 -- start start 2023-09-30T22:18:52.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48eba8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe48809d6f0 0x7fe4881ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48eba8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 0x7fe48811c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48eba8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe4881cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48eba8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48811bd90 con 0x7fe48809d6f0 2023-09-30T22:18:52.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48eba8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880a37f0 con 0x7fe48809e410 2023-09-30T22:18:52.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48eba8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881ac820 con 0x7fe48809e000 2023-09-30T22:18:52.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48c944700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe48809d6f0 0x7fe4881ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48c944700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe48809d6f0 0x7fe4881ab0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47182/0 (socket says 172.21.15.89:47182) 2023-09-30T22:18:52.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48c944700 1 -- 172.21.15.89:0/3633088978 learned_addr learned my addr 172.21.15.89:0/3633088978 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:52.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe487fff700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 0x7fe48811c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48d145700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe4881cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48c944700 1 -- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 msgr2=0x7fe48811c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48c944700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 0x7fe48811c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48c944700 1 -- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 msgr2=0x7fe4881cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.005+0000 7fe48c944700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe4881cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.006+0000 7fe48c944700 1 -- 172.21.15.89:0/3633088978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480009b30 con 0x7fe48809d6f0 2023-09-30T22:18:52.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.006+0000 7fe48c944700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe48809d6f0 0x7fe4881ab0f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e810 tx=0x7fe47c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:52.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.006+0000 7fe4857fa700 1 -- 172.21.15.89:0/3633088978 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00c070 con 0x7fe48809d6f0 2023-09-30T22:18:52.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.006+0000 7fe4857fa700 1 -- 172.21.15.89:0/3633088978 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c013610 con 0x7fe48809d6f0 2023-09-30T22:18:52.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.006+0000 7fe4857fa700 1 -- 172.21.15.89:0/3633088978 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c013980 con 0x7fe48809d6f0 2023-09-30T22:18:52.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.007+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4881ad290 con 0x7fe48809d6f0 2023-09-30T22:18:52.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.007+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48811ca10 con 0x7fe48809d6f0 2023-09-30T22:18:52.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.007+0000 7f87deffd700 1 -- 172.21.15.89:0/2316305728 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87e8025d50 con 0x7f87f013d1f0 2023-09-30T22:18:52.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.007+0000 7fe4857fa700 1 -- 172.21.15.89:0/3633088978 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe47c013ae0 con 0x7fe48809d6f0 2023-09-30T22:18:52.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.008+0000 7fe4857fa700 1 -- 172.21.15.89:0/3633088978 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe47c03f3d0 con 0x7fe48809d6f0 2023-09-30T22:18:52.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.008+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4881ac680 con 0x7fe48809d6f0 2023-09-30T22:18:52.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.008+0000 7fe6535ac700 1 Processor -- start 2023-09-30T22:18:52.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.008+0000 7fe6535ac700 1 -- start start 2023-09-30T22:18:52.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe6535ac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe64c12f640 0x7fe64c12fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe6535ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12ff50 0x7fe64c095070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe6535ac700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64c09b180 con 0x7fe64c073f60 2023-09-30T22:18:52.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe6535ac700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64c09eb40 con 0x7fe64c12f640 2023-09-30T22:18:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe6535ac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c09c580 con 0x7fe64c12ff50 2023-09-30T22:18:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe650b47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12ff50 0x7fe64c095070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe650b47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12ff50 0x7fe64c095070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38082/0 (socket says 172.21.15.89:38082) 2023-09-30T22:18:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe650b47700 1 -- 172.21.15.89:0/1881306447 learned_addr learned my addr 172.21.15.89:0/1881306447 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe650b47700 1 -- 172.21.15.89:0/1881306447 >> v2:172.21.15.89:3300/0 conn(0x7fe64c12f640 msgr2=0x7fe64c12fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe650b47700 1 --2- 172.21.15.89:0/1881306447 >> v2:172.21.15.89:3300/0 conn(0x7fe64c12f640 0x7fe64c12fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe650b47700 1 -- 172.21.15.89:0/1881306447 >> v1:172.21.15.89:6789/0 conn(0x7fe64c073f60 legacy=0x7fe64c136b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe650b47700 1 -- 172.21.15.89:0/1881306447 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c09aca0 con 0x7fe64c12ff50 2023-09-30T22:18:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.009+0000 7fe650b47700 1 --2- 172.21.15.89:0/1881306447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12ff50 0x7fe64c095070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009920 tx=0x7fe63c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91cb6120377fd457 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.010+0000 7fe643fff700 1 -- 172.21.15.89:0/1881306447 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c004230 con 0x7fe64c12ff50 2023-09-30T22:18:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.010+0000 7fe643fff700 1 -- 172.21.15.89:0/1881306447 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c004390 con 0x7fe64c12ff50 2023-09-30T22:18:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.010+0000 7fe643fff700 1 -- 172.21.15.89:0/1881306447 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c009e70 con 0x7fe64c12ff50 2023-09-30T22:18:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.010+0000 7fe6535ac700 1 -- 172.21.15.89:0/1881306447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12ff50 msgr2=0x7fe64c095070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.010+0000 7fe6535ac700 1 --2- 172.21.15.89:0/1881306447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12ff50 0x7fe64c095070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009920 tx=0x7fe63c009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:52.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.011+0000 7fe6535ac700 1 -- 172.21.15.89:0/1881306447 shutdown_connections 2023-09-30T22:18:52.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.011+0000 7fe6535ac700 1 --2- 172.21.15.89:0/1881306447 >> v2:172.21.15.89:3300/0 conn(0x7fe64c12f640 0x7fe64c12fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.011+0000 7fe6535ac700 1 --2- 172.21.15.89:0/1881306447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12ff50 0x7fe64c095070 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.011+0000 7fe6535ac700 1 -- 172.21.15.89:0/1881306447 >> 172.21.15.89:0/1881306447 conn(0x7fe64c0a2360 msgr2=0x7fe64c0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:52.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.011+0000 7fe6535ac700 1 -- 172.21.15.89:0/1881306447 shutdown_connections 2023-09-30T22:18:52.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.011+0000 7fe6535ac700 1 -- 172.21.15.89:0/1881306447 wait complete. 2023-09-30T22:18:52.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe6535ac700 1 Processor -- start 2023-09-30T22:18:52.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe6535ac700 1 -- start start 2023-09-30T22:18:52.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe6535ac700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe64c073f60 0x7fe64c070910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe6535ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12f640 0x7fe64c070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe6535ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe64c12ff50 0x7fe64c071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe6535ac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c09a800 con 0x7fe64c12ff50 2023-09-30T22:18:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe6535ac700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c071da0 con 0x7fe64c073f60 2023-09-30T22:18:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe6535ac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c128c10 con 0x7fe64c12f640 2023-09-30T22:18:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe651348700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe64c073f60 0x7fe64c070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe651b49700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe64c12ff50 0x7fe64c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe651348700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe64c073f60 0x7fe64c070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45420/0 (socket says 172.21.15.89:45420) 2023-09-30T22:18:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe651348700 1 -- 172.21.15.89:0/1072445321 learned_addr learned my addr 172.21.15.89:0/1072445321 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.012+0000 7fe650b47700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12f640 0x7fe64c070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.013+0000 7fe651b49700 1 -- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12f640 msgr2=0x7fe64c070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.013+0000 7fe651b49700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12f640 0x7fe64c070e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.013+0000 7fe651b49700 1 -- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe64c073f60 msgr2=0x7fe64c070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.013+0000 7fe651b49700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe64c073f60 0x7fe64c070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.013+0000 7fe651b49700 1 -- 172.21.15.89:0/1072445321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c009550 con 0x7fe64c12ff50 2023-09-30T22:18:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.013+0000 7fe651348700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe64c073f60 0x7fe64c070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.013+0000 7fe651b49700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe64c12ff50 0x7fe64c071390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c780 tx=0x7fe64400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.014+0000 7fe641ffb700 1 -- 172.21.15.89:0/1072445321 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644003e10 con 0x7fe64c12ff50 2023-09-30T22:18:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.014+0000 7fe641ffb700 1 -- 172.21.15.89:0/1072445321 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6440050d0 con 0x7fe64c12ff50 2023-09-30T22:18:52.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.014+0000 7fe641ffb700 1 -- 172.21.15.89:0/1072445321 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644005480 con 0x7fe64c12ff50 2023-09-30T22:18:52.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.014+0000 7fe4857fa700 1 -- 172.21.15.89:0/3633088978 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe4881ac680 con 0x7fe48809d6f0 2023-09-30T22:18:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.014+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64c072520 con 0x7fe64c12ff50 2023-09-30T22:18:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.014+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c127850 con 0x7fe64c12ff50 2023-09-30T22:18:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.015+0000 7fe641ffb700 1 -- 172.21.15.89:0/1072445321 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6440055e0 con 0x7fe64c12ff50 2023-09-30T22:18:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.015+0000 7fe641ffb700 1 -- 172.21.15.89:0/1072445321 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe64403ee30 con 0x7fe64c12ff50 2023-09-30T22:18:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.016+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64c128940 con 0x7fe64c12ff50 2023-09-30T22:18:52.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.021+0000 7fe641ffb700 1 -- 172.21.15.89:0/1072445321 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe64401bb10 con 0x7fe64c12ff50 2023-09-30T22:18:52.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.191+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2c140747f0 con 0x7f2c14130750 2023-09-30T22:18:52.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.191+0000 7f2c09ffb700 1 -- 172.21.15.89:0/1623452834 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c0c031020 con 0x7f2c14130750 2023-09-30T22:18:52.191 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:52.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.193+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14130750 msgr2=0x7f2c1412b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.193+0000 7f2c1b685700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14130750 0x7f2c1412b650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00cc10 tx=0x7f2c0c00cf20 comp rx=0 tx=0).stop 2023-09-30T22:18:52.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.193+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 shutdown_connections 2023-09-30T22:18:52.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.193+0000 7f2c1b685700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c14130750 0x7f2c1412b650 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.193+0000 7f2c1b685700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1409d6f0 0x7f2c1412abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.193+0000 7f2c1b685700 1 --2- 172.21.15.89:0/1623452834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c1409e080 0x7f2c1412b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.193+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 >> 172.21.15.89:0/1623452834 conn(0x7f2c1409f470 msgr2=0x7f2c141b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.193+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 shutdown_connections 2023-09-30T22:18:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.194+0000 7f2c1b685700 1 -- 172.21.15.89:0/1623452834 wait complete. 2023-09-30T22:18:52.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.220+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe64c074ea0 con 0x7fe64c12ff50 2023-09-30T22:18:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.220+0000 7fe641ffb700 1 -- 172.21.15.89:0/1072445321 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6440192c0 con 0x7fe64c12ff50 2023-09-30T22:18:52.220 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.222+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe64c12ff50 msgr2=0x7fe64c071390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.222+0000 7fe6535ac700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe64c12ff50 0x7fe64c071390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c780 tx=0x7fe64400ca90 comp rx=0 tx=0).stop 2023-09-30T22:18:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.222+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 shutdown_connections 2023-09-30T22:18:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.222+0000 7fe6535ac700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe64c12ff50 0x7fe64c071390 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.222+0000 7fe6535ac700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe64c073f60 0x7fe64c070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.222+0000 7fe6535ac700 1 --2- 172.21.15.89:0/1072445321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe64c12f640 0x7fe64c070e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.222+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 >> 172.21.15.89:0/1072445321 conn(0x7fe64c0a2360 msgr2=0x7fe64c0a0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.223+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 shutdown_connections 2023-09-30T22:18:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.223+0000 7fe6535ac700 1 -- 172.21.15.89:0/1072445321 wait complete. 2023-09-30T22:18:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.223+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f87f009a3d0 con 0x7f87f013d1f0 2023-09-30T22:18:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.223+0000 7f87deffd700 1 -- 172.21.15.89:0/2316305728 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87e8034020 con 0x7f87f013d1f0 2023-09-30T22:18:52.225 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.226+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 msgr2=0x7f87f011c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.227+0000 7f87f4fd2700 1 --2- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f011c1c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f87f009a9a0 tx=0x7f87e8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.227+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 shutdown_connections 2023-09-30T22:18:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.227+0000 7f87f4fd2700 1 --2- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87f013d1f0 0x7f87f011c1c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.227+0000 7f87f4fd2700 1 --2- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87f006a770 0x7f87f011b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.227+0000 7f87f4fd2700 1 --2- 172.21.15.89:0/2316305728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f009de00 0x7f87f011bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.227+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 >> 172.21.15.89:0/2316305728 conn(0x7f87f0066f50 msgr2=0x7f87f013c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.229+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 shutdown_connections 2023-09-30T22:18:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.229+0000 7f87f4fd2700 1 -- 172.21.15.89:0/2316305728 wait complete. 2023-09-30T22:18:52.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.234+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe4881ac680 con 0x7fe48809d6f0 2023-09-30T22:18:52.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.234+0000 7fe4857fa700 1 -- 172.21.15.89:0/3633088978 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe4881ac680 con 0x7fe48809d6f0 2023-09-30T22:18:52.234 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:52.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe48809d6f0 msgr2=0x7fe4881ab0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe48809d6f0 0x7fe4881ab0f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e810 tx=0x7fe47c00eb20 comp rx=0 tx=0).stop 2023-09-30T22:18:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 shutdown_connections 2023-09-30T22:18:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe48809e410 0x7fe4881cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe48809d6f0 0x7fe4881ab0f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 --2- 172.21.15.89:0/3633088978 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe48809e000 0x7fe48811c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 >> 172.21.15.89:0/3633088978 conn(0x7fe488066f50 msgr2=0x7fe4880a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 shutdown_connections 2023-09-30T22:18:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:52.236+0000 7fe48eba8700 1 -- 172.21.15.89:0/3633088978 wait complete. 2023-09-30T22:18:52.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:52.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:53.208 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:53.237 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:53.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:53.253 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:53.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.357+0000 7f97edcaa700 1 Processor -- start 2023-09-30T22:18:53.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.357+0000 7f97edcaa700 1 -- start start 2023-09-30T22:18:53.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97edcaa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809d6f0 0x7f97e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97edcaa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97e809e000 0x7f97e8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97edcaa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e809f130 con 0x7f97e809e410 2023-09-30T22:18:53.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97edcaa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e809f520 con 0x7f97e809e000 2023-09-30T22:18:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97edcaa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e809e930 con 0x7f97e809d6f0 2023-09-30T22:18:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97e6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97e809e000 0x7f97e8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97e6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97e809e000 0x7f97e8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47224/0 (socket says 172.21.15.89:47224) 2023-09-30T22:18:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97e6ffd700 1 -- 172.21.15.89:0/744012609 learned_addr learned my addr 172.21.15.89:0/744012609 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.358+0000 7f97e77fe700 1 --2- 172.21.15.89:0/744012609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809d6f0 0x7f97e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e77fe700 1 -- 172.21.15.89:0/744012609 >> v2:172.21.15.89:3300/0 conn(0x7f97e809e000 msgr2=0x7f97e8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e77fe700 1 --2- 172.21.15.89:0/744012609 >> v2:172.21.15.89:3300/0 conn(0x7f97e809e000 0x7f97e8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e77fe700 1 -- 172.21.15.89:0/744012609 >> v1:172.21.15.89:6789/0 conn(0x7f97e809e410 legacy=0x7f97e8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e77fe700 1 -- 172.21.15.89:0/744012609 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e809b2d0 con 0x7f97e809d6f0 2023-09-30T22:18:53.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e6ffd700 1 --2- 172.21.15.89:0/744012609 >> v2:172.21.15.89:3300/0 conn(0x7f97e809e000 0x7f97e8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:53.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e77fe700 1 --2- 172.21.15.89:0/744012609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809d6f0 0x7f97e809dac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009840 tx=0x7f97d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=383e8287bede104e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:53.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e67fc700 1 -- 172.21.15.89:0/744012609 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8011830 con 0x7f97e809d6f0 2023-09-30T22:18:53.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e67fc700 1 -- 172.21.15.89:0/744012609 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d8011990 con 0x7f97e809d6f0 2023-09-30T22:18:53.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.359+0000 7f97e67fc700 1 -- 172.21.15.89:0/744012609 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8011d20 con 0x7f97e809d6f0 2023-09-30T22:18:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.360+0000 7f97edcaa700 1 -- 172.21.15.89:0/744012609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809d6f0 msgr2=0x7f97e809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.360+0000 7f97edcaa700 1 --2- 172.21.15.89:0/744012609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809d6f0 0x7f97e809dac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009840 tx=0x7f97d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.360+0000 7f97edcaa700 1 -- 172.21.15.89:0/744012609 shutdown_connections 2023-09-30T22:18:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.360+0000 7f97edcaa700 1 --2- 172.21.15.89:0/744012609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809d6f0 0x7f97e809dac0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.360+0000 7f97edcaa700 1 --2- 172.21.15.89:0/744012609 >> v2:172.21.15.89:3300/0 conn(0x7f97e809e000 0x7f97e8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.360+0000 7f97edcaa700 1 -- 172.21.15.89:0/744012609 >> 172.21.15.89:0/744012609 conn(0x7f97e8066f50 msgr2=0x7f97e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.360+0000 7f97edcaa700 1 -- 172.21.15.89:0/744012609 shutdown_connections 2023-09-30T22:18:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.360+0000 7f97edcaa700 1 -- 172.21.15.89:0/744012609 wait complete. 2023-09-30T22:18:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97edcaa700 1 Processor -- start 2023-09-30T22:18:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97edcaa700 1 -- start start 2023-09-30T22:18:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97edcaa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97e809d6f0 0x7f97e811e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97edcaa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809e000 0x7f97e811e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97edcaa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97e809e410 0x7f97e811eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97edcaa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e811f4d0 con 0x7f97e809d6f0 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97edcaa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8121af0 con 0x7f97e809e000 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97edcaa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8120200 con 0x7f97e809e410 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809e000 0x7f97e811e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97e6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809e000 0x7f97e811e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45432/0 (socket says 172.21.15.89:45432) 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.361+0000 7f97e6ffd700 1 -- 172.21.15.89:0/3879498887 learned_addr learned my addr 172.21.15.89:0/3879498887 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e7fff700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97e809e410 0x7f97e811eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e77fe700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97e809d6f0 0x7f97e811e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e6ffd700 1 -- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97e809e410 msgr2=0x7f97e811eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e6ffd700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97e809e410 0x7f97e811eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e6ffd700 1 -- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97e809d6f0 msgr2=0x7f97e811e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e6ffd700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97e809d6f0 0x7f97e811e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e6ffd700 1 -- 172.21.15.89:0/3879498887 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc00b7f0 con 0x7f97e809e000 2023-09-30T22:18:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e7fff700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97e809e410 0x7f97e811eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97e6ffd700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809e000 0x7f97e811e560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00d680 tx=0x7f97dc00d990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97cffff700 1 -- 172.21.15.89:0/3879498887 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00c6f0 con 0x7f97e809e000 2023-09-30T22:18:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97cffff700 1 -- 172.21.15.89:0/3879498887 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc00c850 con 0x7f97e809e000 2023-09-30T22:18:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d80094a0 con 0x7f97e809e000 2023-09-30T22:18:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e80a1d90 con 0x7f97e809e000 2023-09-30T22:18:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.362+0000 7f97cffff700 1 -- 172.21.15.89:0/3879498887 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00cbc0 con 0x7f97e809e000 2023-09-30T22:18:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.363+0000 7f97cffff700 1 -- 172.21.15.89:0/3879498887 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f97dc00cd20 con 0x7f97e809e000 2023-09-30T22:18:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.363+0000 7f97cffff700 1 -- 172.21.15.89:0/3879498887 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f97dc043e20 con 0x7f97e809e000 2023-09-30T22:18:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.363+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c80052e0 con 0x7f97e809e000 2023-09-30T22:18:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.368+0000 7f97cffff700 1 -- 172.21.15.89:0/3879498887 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97dc0212a0 con 0x7f97e809e000 2023-09-30T22:18:53.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.387+0000 7f94072d1700 1 Processor -- start 2023-09-30T22:18:53.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f94072d1700 1 -- start start 2023-09-30T22:18:53.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f94072d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9400134630 0x7f9400136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f94072d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400136f50 0x7f9400139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f94072d1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940009b4f0 con 0x7f9400131810 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f94072d1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94001313c0 con 0x7f9400134630 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f94072d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f940006e8c0 con 0x7f9400136f50 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f940586e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400136f50 0x7f9400139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f940486c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9400134630 0x7f9400136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f940586e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400136f50 0x7f9400139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38106/0 (socket says 172.21.15.89:38106) 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.388+0000 7f940586e700 1 -- 172.21.15.89:0/3483977702 learned_addr learned my addr 172.21.15.89:0/3483977702 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f940586e700 1 -- 172.21.15.89:0/3483977702 >> v2:172.21.15.89:3300/0 conn(0x7f9400134630 msgr2=0x7f9400136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f940586e700 1 --2- 172.21.15.89:0/3483977702 >> v2:172.21.15.89:3300/0 conn(0x7f9400134630 0x7f9400136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f940586e700 1 -- 172.21.15.89:0/3483977702 >> v1:172.21.15.89:6789/0 conn(0x7f9400131810 legacy=0x7f9400131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f940586e700 1 -- 172.21.15.89:0/3483977702 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9400071d20 con 0x7f9400136f50 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f940486c700 1 --2- 172.21.15.89:0/3483977702 >> v2:172.21.15.89:3300/0 conn(0x7f9400134630 0x7f9400136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f940586e700 1 --2- 172.21.15.89:0/3483977702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400136f50 0x7f9400139330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009e50 tx=0x7f93f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21e46059a9865b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f93f7fff700 1 -- 172.21.15.89:0/3483977702 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800e070 con 0x7f9400136f50 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f93f7fff700 1 -- 172.21.15.89:0/3483977702 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800b910 con 0x7f9400136f50 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.389+0000 7f93f7fff700 1 -- 172.21.15.89:0/3483977702 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800bcc0 con 0x7f9400136f50 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.390+0000 7f94072d1700 1 -- 172.21.15.89:0/3483977702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400136f50 msgr2=0x7f9400139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.390+0000 7f94072d1700 1 --2- 172.21.15.89:0/3483977702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400136f50 0x7f9400139330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009e50 tx=0x7f93f800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.390+0000 7f94072d1700 1 -- 172.21.15.89:0/3483977702 shutdown_connections 2023-09-30T22:18:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.390+0000 7f94072d1700 1 --2- 172.21.15.89:0/3483977702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400136f50 0x7f9400139330 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.390+0000 7f94072d1700 1 --2- 172.21.15.89:0/3483977702 >> v2:172.21.15.89:3300/0 conn(0x7f9400134630 0x7f9400136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.390+0000 7f94072d1700 1 -- 172.21.15.89:0/3483977702 >> 172.21.15.89:0/3483977702 conn(0x7f94000a2aa0 msgr2=0x7f94000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.390+0000 7f94072d1700 1 -- 172.21.15.89:0/3483977702 shutdown_connections 2023-09-30T22:18:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.390+0000 7f94072d1700 1 -- 172.21.15.89:0/3483977702 wait complete. 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f94072d1700 1 Processor -- start 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f94072d1700 1 -- start start 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f94072d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9400131810 0x7f9400119be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f94072d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400134630 0x7f940011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f94072d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9400136f50 0x7f94001c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f94072d1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940006eb90 con 0x7f9400136f50 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f94072d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94001197b0 con 0x7f9400131810 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f94072d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f940011cbb0 con 0x7f9400134630 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f940486c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400134630 0x7f940011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.391+0000 7f940486c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400134630 0x7f940011a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38122/0 (socket says 172.21.15.89:38122) 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940486c700 1 -- 172.21.15.89:0/2137130375 learned_addr learned my addr 172.21.15.89:0/2137130375 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940506d700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9400131810 0x7f9400119be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940586e700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9400136f50 0x7f94001c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940586e700 1 -- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400134630 msgr2=0x7f940011a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940586e700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400134630 0x7f940011a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940586e700 1 -- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9400131810 msgr2=0x7f9400119be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940586e700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9400131810 0x7f9400119be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940586e700 1 -- 172.21.15.89:0/2137130375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f00094a0 con 0x7f9400136f50 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f940586e700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9400136f50 0x7f94001c61b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f93f8015000 tx=0x7f93f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f93f5ffb700 1 -- 172.21.15.89:0/2137130375 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8004570 con 0x7f9400136f50 2023-09-30T22:18:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f93f5ffb700 1 -- 172.21.15.89:0/2137130375 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f80092e0 con 0x7f9400136f50 2023-09-30T22:18:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f93f5ffb700 1 -- 172.21.15.89:0/2137130375 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8004910 con 0x7f9400136f50 2023-09-30T22:18:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.392+0000 7f94072d1700 1 -- 172.21.15.89:0/2137130375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f8009b30 con 0x7f9400136f50 2023-09-30T22:18:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.393+0000 7f94072d1700 1 -- 172.21.15.89:0/2137130375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940011a660 con 0x7f9400136f50 2023-09-30T22:18:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.393+0000 7f93e37fe700 1 -- 172.21.15.89:0/2137130375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93dc0052e0 con 0x7f9400136f50 2023-09-30T22:18:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.394+0000 7f93f5ffb700 1 -- 172.21.15.89:0/2137130375 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93f802d420 con 0x7f9400136f50 2023-09-30T22:18:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.394+0000 7f93f5ffb700 1 -- 172.21.15.89:0/2137130375 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f93f8051b40 con 0x7f9400136f50 2023-09-30T22:18:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.399+0000 7f93f5ffb700 1 -- 172.21.15.89:0/2137130375 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93f8034770 con 0x7f9400136f50 2023-09-30T22:18:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.404+0000 7f36a53f4700 1 Processor -- start 2023-09-30T22:18:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.404+0000 7f36a53f4700 1 -- start start 2023-09-30T22:18:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.404+0000 7f36a53f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a009e1d0 0x7f36a0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.404+0000 7f36a53f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36a0135e50 0x7f36a0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f36a53f4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a009f130 con 0x7f36a009d6f0 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f36a53f4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a009f520 con 0x7f36a0135e50 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f36a53f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a009e930 con 0x7f36a009e1d0 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a009e1d0 0x7f36a0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a009e1d0 0x7f36a0135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45454/0 (socket says 172.21.15.89:45454) 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f36a009d6f0 0x7f36a009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41026/0 (socket says 172.21.15.89:41026) 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369e7fc700 1 -- 172.21.15.89:0/1011787621 learned_addr learned my addr 172.21.15.89:0/1011787621 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369f7fe700 1 --2- 172.21.15.89:0/1011787621 >> v2:172.21.15.89:3300/0 conn(0x7f36a0135e50 0x7f36a0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369e7fc700 1 -- 172.21.15.89:0/1011787621 >> v2:172.21.15.89:3300/0 conn(0x7f36a0135e50 msgr2=0x7f36a0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369e7fc700 1 --2- 172.21.15.89:0/1011787621 >> v2:172.21.15.89:3300/0 conn(0x7f36a0135e50 0x7f36a0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369e7fc700 1 -- 172.21.15.89:0/1011787621 >> v1:172.21.15.89:6789/0 conn(0x7f36a009d6f0 legacy=0x7f36a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.405+0000 7f369e7fc700 1 -- 172.21.15.89:0/1011787621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a009b2d0 con 0x7f36a009e1d0 2023-09-30T22:18:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.406+0000 7f369e7fc700 1 --2- 172.21.15.89:0/1011787621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a009e1d0 0x7f36a0135910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3694009920 tx=0x7f3694009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a3d70f466c8fe5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.406+0000 7f369dffb700 1 -- 172.21.15.89:0/1011787621 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369400ba20 con 0x7f36a009e1d0 2023-09-30T22:18:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.406+0000 7f369dffb700 1 -- 172.21.15.89:0/1011787621 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369400bb80 con 0x7f36a009e1d0 2023-09-30T22:18:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.406+0000 7f369dffb700 1 -- 172.21.15.89:0/1011787621 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694009e70 con 0x7f36a009e1d0 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.407+0000 7f36a53f4700 1 -- 172.21.15.89:0/1011787621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a009e1d0 msgr2=0x7f36a0135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.407+0000 7f36a53f4700 1 --2- 172.21.15.89:0/1011787621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a009e1d0 0x7f36a0135910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3694009920 tx=0x7f3694009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.407+0000 7f36a53f4700 1 -- 172.21.15.89:0/1011787621 shutdown_connections 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.407+0000 7f36a53f4700 1 --2- 172.21.15.89:0/1011787621 >> v2:172.21.15.89:3300/0 conn(0x7f36a0135e50 0x7f36a0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.407+0000 7f36a53f4700 1 --2- 172.21.15.89:0/1011787621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a009e1d0 0x7f36a0135910 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.407+0000 7f36a53f4700 1 -- 172.21.15.89:0/1011787621 >> 172.21.15.89:0/1011787621 conn(0x7f36a0066f50 msgr2=0x7f36a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.407+0000 7f36a53f4700 1 -- 172.21.15.89:0/1011787621 shutdown_connections 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.407+0000 7f36a53f4700 1 -- 172.21.15.89:0/1011787621 wait complete. 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.408+0000 7f36a53f4700 1 Processor -- start 2023-09-30T22:18:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.408+0000 7f36a53f4700 1 -- start start 2023-09-30T22:18:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f36a53f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a009d6f0 0x7f36a01c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f36a53f4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36a009e1d0 0x7f36a01c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f36a53f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a0135e50 0x7f36a01c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f36a53f4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0070480 con 0x7f36a009d6f0 2023-09-30T22:18:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f36a53f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01343f0 con 0x7f36a0135e50 2023-09-30T22:18:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f36a53f4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0125320 con 0x7f36a009e1d0 2023-09-30T22:18:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f369effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a009d6f0 0x7f36a01c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f369e7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36a009e1d0 0x7f36a01c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f369effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a009d6f0 0x7f36a01c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47272/0 (socket says 172.21.15.89:47272) 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f369effd700 1 -- 172.21.15.89:0/1246881753 learned_addr learned my addr 172.21.15.89:0/1246881753 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.409+0000 7f369f7fe700 1 --2- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a0135e50 0x7f36a01c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f369effd700 1 -- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36a009e1d0 msgr2=0x7f36a01c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f369effd700 1 --2- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36a009e1d0 0x7f36a01c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f369effd700 1 -- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a0135e50 msgr2=0x7f36a01c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f369effd700 1 --2- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a0135e50 0x7f36a01c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f369effd700 1 -- 172.21.15.89:0/1246881753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694009550 con 0x7f36a009d6f0 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f369effd700 1 --2- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a009d6f0 0x7f36a01c8280 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f369000c520 tx=0x7f369000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f36877fe700 1 -- 172.21.15.89:0/1246881753 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690005ab0 con 0x7f36a009d6f0 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f36877fe700 1 -- 172.21.15.89:0/1246881753 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690005c10 con 0x7f36a009d6f0 2023-09-30T22:18:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a00a1f50 con 0x7f36a009d6f0 2023-09-30T22:18:53.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f36877fe700 1 -- 172.21.15.89:0/1246881753 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36900070d0 con 0x7f36a009d6f0 2023-09-30T22:18:53.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.410+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a0123c00 con 0x7f36a009d6f0 2023-09-30T22:18:53.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.411+0000 7f36877fe700 1 -- 172.21.15.89:0/1246881753 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3690007230 con 0x7f36a009d6f0 2023-09-30T22:18:53.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.411+0000 7f36877fe700 1 -- 172.21.15.89:0/1246881753 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f369004f8b0 con 0x7f36a009d6f0 2023-09-30T22:18:53.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.411+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a0074ea0 con 0x7f36a009d6f0 2023-09-30T22:18:53.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.416+0000 7f36877fe700 1 -- 172.21.15.89:0/1246881753 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3690028030 con 0x7f36a009d6f0 2023-09-30T22:18:53.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.459+0000 7fb3b4470700 1 Processor -- start 2023-09-30T22:18:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.459+0000 7fb3b4470700 1 -- start start 2023-09-30T22:18:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.459+0000 7fb3b4470700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac09eca0 0x7fb3ac13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.459+0000 7fb3b4470700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac06f050 0x7fb3ac13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.459+0000 7fb3b4470700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac070ee0 con 0x7fb3ac06e570 2023-09-30T22:18:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.459+0000 7fb3b4470700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac071980 con 0x7fb3ac09eca0 2023-09-30T22:18:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.459+0000 7fb3b4470700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac071480 con 0x7fb3ac06f050 2023-09-30T22:18:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b220c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb3ac06e570 0x7fb3ac06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41030/0 (socket says 172.21.15.89:41030) 2023-09-30T22:18:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b220c700 1 -- 172.21.15.89:0/306087165 learned_addr learned my addr 172.21.15.89:0/306087165 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b1a0b700 1 --2- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac09eca0 0x7fb3ac13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b2a0d700 1 --2- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac06f050 0x7fb3ac13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b1a0b700 1 -- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac06f050 msgr2=0x7fb3ac13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b1a0b700 1 --2- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac06f050 0x7fb3ac13f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b1a0b700 1 -- 172.21.15.89:0/306087165 >> v1:172.21.15.89:6789/0 conn(0x7fb3ac06e570 legacy=0x7fb3ac06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b120a700 1 -- 172.21.15.89:0/306087165 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720853443 0 0) 0x7fb3ac070ee0 con 0x7fb3ac06e570 2023-09-30T22:18:53.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b1a0b700 1 -- 172.21.15.89:0/306087165 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac071b90 con 0x7fb3ac09eca0 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b1a0b700 1 --2- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac09eca0 0x7fb3ac13cea0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009870 tx=0x7fb39c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e6f572685c8a7f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b120a700 1 -- 172.21.15.89:0/306087165 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c011830 con 0x7fb3ac09eca0 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b120a700 1 -- 172.21.15.89:0/306087165 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c011990 con 0x7fb3ac09eca0 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.460+0000 7fb3b120a700 1 -- 172.21.15.89:0/306087165 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c011d20 con 0x7fb3ac09eca0 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.461+0000 7fb3b4470700 1 -- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac09eca0 msgr2=0x7fb3ac13cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.461+0000 7fb3b4470700 1 --2- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac09eca0 0x7fb3ac13cea0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009870 tx=0x7fb39c009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.461+0000 7fb3b4470700 1 -- 172.21.15.89:0/306087165 shutdown_connections 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.461+0000 7fb3b4470700 1 --2- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac06f050 0x7fb3ac13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.461+0000 7fb3b4470700 1 --2- 172.21.15.89:0/306087165 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac09eca0 0x7fb3ac13cea0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.461+0000 7fb3b4470700 1 -- 172.21.15.89:0/306087165 >> 172.21.15.89:0/306087165 conn(0x7fb3ac0a2aa0 msgr2=0x7fb3ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.461+0000 7fb3b4470700 1 -- 172.21.15.89:0/306087165 shutdown_connections 2023-09-30T22:18:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.461+0000 7fb3b4470700 1 -- 172.21.15.89:0/306087165 wait complete. 2023-09-30T22:18:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.462+0000 7fb3b4470700 1 Processor -- start 2023-09-30T22:18:53.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.462+0000 7fb3b4470700 1 -- start start 2023-09-30T22:18:53.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b4470700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac06e570 0x7fb3ac136250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b4470700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ac06f050 0x7fb3ac12f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b4470700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac12f930 0x7fb3ac12fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b4470700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac136e50 con 0x7fb3ac06f050 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b4470700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac137bc0 con 0x7fb3ac06e570 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b4470700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac1387c0 con 0x7fb3ac12f930 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac12f930 0x7fb3ac12fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac12f930 0x7fb3ac12fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38164/0 (socket says 172.21.15.89:38164) 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 -- 172.21.15.89:0/2949646276 learned_addr learned my addr 172.21.15.89:0/2949646276 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b1a0b700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ac06f050 0x7fb3ac12f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 -- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac06e570 msgr2=0x7fb3ac136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac06e570 0x7fb3ac136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 -- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ac06f050 msgr2=0x7fb3ac12f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ac06f050 0x7fb3ac12f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 -- 172.21.15.89:0/2949646276 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0094a0 con 0x7fb3ac12f930 2023-09-30T22:18:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b1a0b700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ac06f050 0x7fb3ac12f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.463+0000 7fb3b2a0d700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac12f930 0x7fb3ac12fdd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400efc0 tx=0x7fb3a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.464+0000 7fb3a2ffd700 1 -- 172.21.15.89:0/2949646276 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4009ea0 con 0x7fb3ac12f930 2023-09-30T22:18:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.464+0000 7fb3a2ffd700 1 -- 172.21.15.89:0/2949646276 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a400f040 con 0x7fb3ac12f930 2023-09-30T22:18:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.464+0000 7fb3a2ffd700 1 -- 172.21.15.89:0/2949646276 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4014480 con 0x7fb3ac12f930 2023-09-30T22:18:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.464+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac138a80 con 0x7fb3ac12f930 2023-09-30T22:18:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.464+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac130bf0 con 0x7fb3ac12f930 2023-09-30T22:18:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.465+0000 7fb3a2ffd700 1 -- 172.21.15.89:0/2949646276 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb3a40145e0 con 0x7fb3ac12f930 2023-09-30T22:18:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.465+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac074ea0 con 0x7fb3ac12f930 2023-09-30T22:18:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.465+0000 7fb3a2ffd700 1 -- 172.21.15.89:0/2949646276 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb3a4040040 con 0x7fb3ac12f930 2023-09-30T22:18:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.474+0000 7fb3a2ffd700 1 -- 172.21.15.89:0/2949646276 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb3a401a8b0 con 0x7fb3ac12f930 2023-09-30T22:18:53.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.574+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f97c8005590 con 0x7f97e809e000 2023-09-30T22:18:53.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.574+0000 7f97cffff700 1 -- 172.21.15.89:0/3879498887 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97dc018aa0 con 0x7f97e809e000 2023-09-30T22:18:53.575 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809e000 msgr2=0x7f97e811e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809e000 0x7f97e811e560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00d680 tx=0x7f97dc00d990 comp rx=0 tx=0).stop 2023-09-30T22:18:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 shutdown_connections 2023-09-30T22:18:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97e809e410 0x7f97e811eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97e809d6f0 0x7f97e811e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 --2- 172.21.15.89:0/3879498887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97e809e000 0x7f97e811e560 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 >> 172.21.15.89:0/3879498887 conn(0x7f97e8066f50 msgr2=0x7f97e80710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 shutdown_connections 2023-09-30T22:18:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.577+0000 7f97edcaa700 1 -- 172.21.15.89:0/3879498887 wait complete. 2023-09-30T22:18:53.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.621+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f36a006c360 con 0x7f36a009d6f0 2023-09-30T22:18:53.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.621+0000 7f36877fe700 1 -- 172.21.15.89:0/1246881753 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f369002a6d0 con 0x7f36a009d6f0 2023-09-30T22:18:53.622 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:53.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.623+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a009d6f0 msgr2=0x7f36a01c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.623+0000 7f36a53f4700 1 --2- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a009d6f0 0x7f36a01c8280 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f369000c520 tx=0x7f369000c830 comp rx=0 tx=0).stop 2023-09-30T22:18:53.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.624+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 shutdown_connections 2023-09-30T22:18:53.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.624+0000 7f36a53f4700 1 --2- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36a0135e50 0x7f36a01c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.624+0000 7f36a53f4700 1 --2- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36a009d6f0 0x7f36a01c8280 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.624+0000 7f36a53f4700 1 --2- 172.21.15.89:0/1246881753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36a009e1d0 0x7f36a01c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.624+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 >> 172.21.15.89:0/1246881753 conn(0x7f36a0066f50 msgr2=0x7f36a0134840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:53.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.624+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 shutdown_connections 2023-09-30T22:18:53.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.624+0000 7f36a53f4700 1 -- 172.21.15.89:0/1246881753 wait complete. 2023-09-30T22:18:53.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.632+0000 7f93e37fe700 1 -- 172.21.15.89:0/2137130375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f93dc005590 con 0x7f9400136f50 2023-09-30T22:18:53.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.633+0000 7f93f5ffb700 1 -- 172.21.15.89:0/2137130375 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93f803b030 con 0x7f9400136f50 2023-09-30T22:18:53.633 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 -- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9400136f50 msgr2=0x7f94001c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9400136f50 0x7f94001c61b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f93f8015000 tx=0x7f93f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 -- 172.21.15.89:0/2137130375 shutdown_connections 2023-09-30T22:18:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9400136f50 0x7f94001c61b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9400131810 0x7f9400119be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 --2- 172.21.15.89:0/2137130375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9400134630 0x7f940011a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 -- 172.21.15.89:0/2137130375 >> 172.21.15.89:0/2137130375 conn(0x7f94000a2aa0 msgr2=0x7f940012f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 -- 172.21.15.89:0/2137130375 shutdown_connections 2023-09-30T22:18:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.635+0000 7f93e37fe700 1 -- 172.21.15.89:0/2137130375 wait complete. 2023-09-30T22:18:53.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:53.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.708+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb3ac06b9e0 con 0x7fb3ac12f930 2023-09-30T22:18:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.708+0000 7fb3a2ffd700 1 -- 172.21.15.89:0/2949646276 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb3a4018030 con 0x7fb3ac12f930 2023-09-30T22:18:53.709 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac12f930 msgr2=0x7fb3ac12fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac12f930 0x7fb3ac12fdd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400efc0 tx=0x7fb3a400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 shutdown_connections 2023-09-30T22:18:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb3ac12f930 0x7fb3ac12fdd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb3ac06e570 0x7fb3ac136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 --2- 172.21.15.89:0/2949646276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ac06f050 0x7fb3ac12f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 >> 172.21.15.89:0/2949646276 conn(0x7fb3ac0a2aa0 msgr2=0x7fb3ac130f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 shutdown_connections 2023-09-30T22:18:53.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:53.710+0000 7fb3b4470700 1 -- 172.21.15.89:0/2949646276 wait complete. 2023-09-30T22:18:53.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:54.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:54.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:54.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:54.724 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.740+0000 7f4625453700 1 Processor -- start 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.740+0000 7f4625453700 1 -- start start 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f4625453700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009e930 0x7f462009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f4625453700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f462009f240 0x7f46201317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f4625453700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4620070940 con 0x7f462009f650 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f4625453700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462006fd60 con 0x7f462009f240 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f4625453700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462006fa80 con 0x7f462009e930 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f461effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009e930 0x7f462009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f461effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009e930 0x7f462009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45472/0 (socket says 172.21.15.89:45472) 2023-09-30T22:18:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f461effd700 1 -- 172.21.15.89:0/912922190 learned_addr learned my addr 172.21.15.89:0/912922190 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.741+0000 7f461dffb700 1 -- 172.21.15.89:0/912922190 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128177655 0 0) 0x7f4620070940 con 0x7f462009f650 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461effd700 1 -- 172.21.15.89:0/912922190 >> v2:172.21.15.89:3300/0 conn(0x7f462009f240 msgr2=0x7f46201317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461e7fc700 1 --2- 172.21.15.89:0/912922190 >> v2:172.21.15.89:3300/0 conn(0x7f462009f240 0x7f46201317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461effd700 1 --2- 172.21.15.89:0/912922190 >> v2:172.21.15.89:3300/0 conn(0x7f462009f240 0x7f46201317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461effd700 1 -- 172.21.15.89:0/912922190 >> v1:172.21.15.89:6789/0 conn(0x7f462009f650 legacy=0x7f4620131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461effd700 1 -- 172.21.15.89:0/912922190 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462009fb60 con 0x7f462009e930 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461e7fc700 1 --2- 172.21.15.89:0/912922190 >> v2:172.21.15.89:3300/0 conn(0x7f462009f240 0x7f46201317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461effd700 1 --2- 172.21.15.89:0/912922190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009e930 0x7f462009ed00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f46100098f0 tx=0x7f4610009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57d936458b90a14c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461dffb700 1 -- 172.21.15.89:0/912922190 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461000b8a0 con 0x7f462009e930 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461dffb700 1 -- 172.21.15.89:0/912922190 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461000ba00 con 0x7f462009e930 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.742+0000 7f461dffb700 1 -- 172.21.15.89:0/912922190 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461000bd90 con 0x7f462009e930 2023-09-30T22:18:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.743+0000 7f4625453700 1 -- 172.21.15.89:0/912922190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009e930 msgr2=0x7f462009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.743+0000 7f4625453700 1 --2- 172.21.15.89:0/912922190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009e930 0x7f462009ed00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f46100098f0 tx=0x7f4610009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.743+0000 7f4625453700 1 -- 172.21.15.89:0/912922190 shutdown_connections 2023-09-30T22:18:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.743+0000 7f4625453700 1 --2- 172.21.15.89:0/912922190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009e930 0x7f462009ed00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.743+0000 7f4625453700 1 --2- 172.21.15.89:0/912922190 >> v2:172.21.15.89:3300/0 conn(0x7f462009f240 0x7f46201317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.743+0000 7f4625453700 1 -- 172.21.15.89:0/912922190 >> 172.21.15.89:0/912922190 conn(0x7f46200a4210 msgr2=0x7f46200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.743+0000 7f4625453700 1 -- 172.21.15.89:0/912922190 shutdown_connections 2023-09-30T22:18:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.743+0000 7f4625453700 1 -- 172.21.15.89:0/912922190 wait complete. 2023-09-30T22:18:54.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.744+0000 7f4625453700 1 Processor -- start 2023-09-30T22:18:54.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.744+0000 7f4625453700 1 -- start start 2023-09-30T22:18:54.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f4625453700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f462009e930 0x7f46201a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f4625453700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f462009f240 0x7f46201c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f4625453700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009f650 0x7f46201c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f4625453700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a6370 con 0x7f462009f240 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f4625453700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46201b7370 con 0x7f462009f650 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f4625453700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a7750 con 0x7f462009e930 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009f650 0x7f46201c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f461e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f462009f240 0x7f46201c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009f650 0x7f46201c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45482/0 (socket says 172.21.15.89:45482) 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.745+0000 7f461f7fe700 1 -- 172.21.15.89:0/4156391969 learned_addr learned my addr 172.21.15.89:0/4156391969 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f461e7fc700 1 -- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f462009e930 msgr2=0x7f46201a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f461e7fc700 1 --2- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f462009e930 0x7f46201a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f461e7fc700 1 -- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009f650 msgr2=0x7f46201c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f461e7fc700 1 --2- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009f650 0x7f46201c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f461e7fc700 1 -- 172.21.15.89:0/4156391969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610009550 con 0x7f462009f240 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f461f7fe700 1 --2- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009f650 0x7f46201c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f461e7fc700 1 --2- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f462009f240 0x7f46201c5cd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f46140109b0 tx=0x7f4614010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f46077fe700 1 -- 172.21.15.89:0/4156391969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614013070 con 0x7f462009f240 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f46077fe700 1 -- 172.21.15.89:0/4156391969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461400e2e0 con 0x7f462009f240 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f46077fe700 1 -- 172.21.15.89:0/4156391969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614010ea0 con 0x7f462009f240 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46200a0190 con 0x7f462009f240 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.746+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46201a6f70 con 0x7f462009f240 2023-09-30T22:18:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.747+0000 7f46077fe700 1 -- 172.21.15.89:0/4156391969 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4614004270 con 0x7f462009f240 2023-09-30T22:18:54.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.747+0000 7f46077fe700 1 -- 172.21.15.89:0/4156391969 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4614042a70 con 0x7f462009f240 2023-09-30T22:18:54.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.747+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46201a0b30 con 0x7f462009f240 2023-09-30T22:18:54.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.752+0000 7f46077fe700 1 -- 172.21.15.89:0/4156391969 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f461401c020 con 0x7f462009f240 2023-09-30T22:18:54.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.777+0000 7fbc6ad00700 1 Processor -- start 2023-09-30T22:18:54.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.777+0000 7fbc6ad00700 1 -- start start 2023-09-30T22:18:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.777+0000 7fbc6ad00700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc6409e1d0 0x7fbc64135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.777+0000 7fbc6ad00700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc64135e50 0x7fbc64138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.777+0000 7fbc6ad00700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6409f130 con 0x7fbc6409d6f0 2023-09-30T22:18:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.777+0000 7fbc6ad00700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6409f520 con 0x7fbc64135e50 2023-09-30T22:18:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.777+0000 7fbc6ad00700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6409e930 con 0x7fbc6409e1d0 2023-09-30T22:18:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc63fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc6409e1d0 0x7fbc64135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc63fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc6409e1d0 0x7fbc64135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38192/0 (socket says 172.21.15.89:38192) 2023-09-30T22:18:54.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc63fff700 1 -- 172.21.15.89:0/1653900464 learned_addr learned my addr 172.21.15.89:0/1653900464 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc6929d700 1 --2- 172.21.15.89:0/1653900464 >> v2:172.21.15.89:3300/0 conn(0x7fbc64135e50 0x7fbc64138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc6929d700 1 -- 172.21.15.89:0/1653900464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc6409e1d0 msgr2=0x7fbc64135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc6929d700 1 --2- 172.21.15.89:0/1653900464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc6409e1d0 0x7fbc64135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc6929d700 1 -- 172.21.15.89:0/1653900464 >> v1:172.21.15.89:6789/0 conn(0x7fbc6409d6f0 legacy=0x7fbc6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc6929d700 1 -- 172.21.15.89:0/1653900464 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6409b2d0 con 0x7fbc64135e50 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc63fff700 1 --2- 172.21.15.89:0/1653900464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc6409e1d0 0x7fbc64135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc6929d700 1 --2- 172.21.15.89:0/1653900464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc64135e50 0x7fbc64138230 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009dc0 tx=0x7fbc5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e3aebcee3dfd4b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc637fe700 1 -- 172.21.15.89:0/1653900464 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c00e070 con 0x7fbc64135e50 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc637fe700 1 -- 172.21.15.89:0/1653900464 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c00bae0 con 0x7fbc64135e50 2023-09-30T22:18:54.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.778+0000 7fbc637fe700 1 -- 172.21.15.89:0/1653900464 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c00be90 con 0x7fbc64135e50 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.779+0000 7fbc6ad00700 1 -- 172.21.15.89:0/1653900464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc64135e50 msgr2=0x7fbc64138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.779+0000 7fbc6ad00700 1 --2- 172.21.15.89:0/1653900464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc64135e50 0x7fbc64138230 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009dc0 tx=0x7fbc5c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.779+0000 7fbc6ad00700 1 -- 172.21.15.89:0/1653900464 shutdown_connections 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.779+0000 7fbc6ad00700 1 --2- 172.21.15.89:0/1653900464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc64135e50 0x7fbc64138230 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.779+0000 7fbc6ad00700 1 --2- 172.21.15.89:0/1653900464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc6409e1d0 0x7fbc64135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.779+0000 7fbc6ad00700 1 -- 172.21.15.89:0/1653900464 >> 172.21.15.89:0/1653900464 conn(0x7fbc64066f50 msgr2=0x7fbc6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.779+0000 7fbc6ad00700 1 -- 172.21.15.89:0/1653900464 shutdown_connections 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.779+0000 7fbc6ad00700 1 -- 172.21.15.89:0/1653900464 wait complete. 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.780+0000 7fbc6ad00700 1 Processor -- start 2023-09-30T22:18:54.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.780+0000 7fbc6ad00700 1 -- start start 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc6ad00700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc6409d6f0 0x7fbc641c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc6ad00700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6409e1d0 0x7fbc641c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc6ad00700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc64135e50 0x7fbc641c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc6ad00700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6411b140 con 0x7fbc6409e1d0 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc6ad00700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6411c8b0 con 0x7fbc6409d6f0 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc6ad00700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6412ff70 con 0x7fbc64135e50 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc68a9c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc6409d6f0 0x7fbc641c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc68a9c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc6409d6f0 0x7fbc641c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45484/0 (socket says 172.21.15.89:45484) 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc68a9c700 1 -- 172.21.15.89:0/3104408860 learned_addr learned my addr 172.21.15.89:0/3104408860 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc6929d700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc64135e50 0x7fbc641c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc63fff700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6409e1d0 0x7fbc641c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc68a9c700 1 -- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc64135e50 msgr2=0x7fbc641c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc68a9c700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc64135e50 0x7fbc641c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc68a9c700 1 -- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6409e1d0 msgr2=0x7fbc641c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc68a9c700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6409e1d0 0x7fbc641c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc68a9c700 1 -- 172.21.15.89:0/3104408860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5000b7f0 con 0x7fbc6409d6f0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.781+0000 7fbc6929d700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc64135e50 0x7fbc641c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.782+0000 7fbc68a9c700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc6409d6f0 0x7fbc641c4030 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c4e0 tx=0x7fbc5800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.782+0000 7fbc617fa700 1 -- 172.21.15.89:0/3104408860 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58015d70 con 0x7fbc6409d6f0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.782+0000 7fbc617fa700 1 -- 172.21.15.89:0/3104408860 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc580050d0 con 0x7fbc6409d6f0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.782+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c009a70 con 0x7fbc6409d6f0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.782+0000 7fbc617fa700 1 -- 172.21.15.89:0/3104408860 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58005480 con 0x7fbc6409d6f0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.782+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6409c700 con 0x7fbc6409d6f0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.783+0000 7fbc617fa700 1 -- 172.21.15.89:0/3104408860 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc58003680 con 0x7fbc6409d6f0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.783+0000 7fbc617fa700 1 -- 172.21.15.89:0/3104408860 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbc58047190 con 0x7fbc6409d6f0 2023-09-30T22:18:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.783+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc64074ea0 con 0x7fbc6409d6f0 2023-09-30T22:18:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.792+0000 7fbc617fa700 1 -- 172.21.15.89:0/3104408860 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc580263f0 con 0x7fbc6409d6f0 2023-09-30T22:18:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.827+0000 7fc000bb2700 1 Processor -- start 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.827+0000 7fc000bb2700 1 -- start start 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fc000bb2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fc000bb2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbffc13d1f0 0x7fbffc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fc000bb2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc09a810 con 0x7fbffc06a770 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fc000bb2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc06ab40 con 0x7fbffc13d1f0 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fc000bb2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc09ebc0 con 0x7fbffc09de00 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fbff9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fbff9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc09e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45494/0 (socket says 172.21.15.89:45494) 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fbffad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbffc13d1f0 0x7fbffc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fbff9d9b700 1 -- 172.21.15.89:0/1615114145 learned_addr learned my addr 172.21.15.89:0/1615114145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fbff9d9b700 1 -- 172.21.15.89:0/1615114145 >> v2:172.21.15.89:3300/0 conn(0x7fbffc13d1f0 msgr2=0x7fbffc13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fbff9d9b700 1 --2- 172.21.15.89:0/1615114145 >> v2:172.21.15.89:3300/0 conn(0x7fbffc13d1f0 0x7fbffc13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fbff9d9b700 1 -- 172.21.15.89:0/1615114145 >> v1:172.21.15.89:6789/0 conn(0x7fbffc06a770 legacy=0x7fbffc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.828+0000 7fbff9d9b700 1 -- 172.21.15.89:0/1615114145 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc09e920 con 0x7fbffc09de00 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.829+0000 7fbff9d9b700 1 --2- 172.21.15.89:0/1615114145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc09e250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098f0 tx=0x7fbff0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0d425634c77cde6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.829+0000 7fbff959a700 1 -- 172.21.15.89:0/1615114145 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0011830 con 0x7fbffc09de00 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.829+0000 7fbff959a700 1 -- 172.21.15.89:0/1615114145 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0011990 con 0x7fbffc09de00 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.829+0000 7fbff959a700 1 -- 172.21.15.89:0/1615114145 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0011d20 con 0x7fbffc09de00 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.830+0000 7fc000bb2700 1 -- 172.21.15.89:0/1615114145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 msgr2=0x7fbffc09e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.830+0000 7fc000bb2700 1 --2- 172.21.15.89:0/1615114145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc09e250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098f0 tx=0x7fbff0009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.830+0000 7fc000bb2700 1 -- 172.21.15.89:0/1615114145 shutdown_connections 2023-09-30T22:18:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.830+0000 7fc000bb2700 1 --2- 172.21.15.89:0/1615114145 >> v2:172.21.15.89:3300/0 conn(0x7fbffc13d1f0 0x7fbffc13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.830+0000 7fc000bb2700 1 --2- 172.21.15.89:0/1615114145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc09e250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.830+0000 7fc000bb2700 1 -- 172.21.15.89:0/1615114145 >> 172.21.15.89:0/1615114145 conn(0x7fbffc066f50 msgr2=0x7fbffc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.830+0000 7fc000bb2700 1 -- 172.21.15.89:0/1615114145 shutdown_connections 2023-09-30T22:18:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.830+0000 7fc000bb2700 1 -- 172.21.15.89:0/1615114145 wait complete. 2023-09-30T22:18:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fc000bb2700 1 Processor -- start 2023-09-30T22:18:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fc000bb2700 1 -- start start 2023-09-30T22:18:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fc000bb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbffc06a770 0x7fbffc13b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fc000bb2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc13bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fc000bb2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbffc13d1f0 0x7fbffc13c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fc000bb2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc135dd0 con 0x7fbffc06a770 2023-09-30T22:18:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fc000bb2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc11ea60 con 0x7fbffc09de00 2023-09-30T22:18:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fc000bb2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc11f0e0 con 0x7fbffc13d1f0 2023-09-30T22:18:54.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fbff9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc13bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fbff9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc13bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45496/0 (socket says 172.21.15.89:45496) 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.832+0000 7fbff9d9b700 1 -- 172.21.15.89:0/866015679 learned_addr learned my addr 172.21.15.89:0/866015679 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbffa59c700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbffc06a770 0x7fbffc13b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbffad9d700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbffc13d1f0 0x7fbffc13c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbff9d9b700 1 -- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbffc13d1f0 msgr2=0x7fbffc13c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbff9d9b700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbffc13d1f0 0x7fbffc13c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbff9d9b700 1 -- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbffc06a770 msgr2=0x7fbffc13b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbff9d9b700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbffc06a770 0x7fbffc13b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbff9d9b700 1 -- 172.21.15.89:0/866015679 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0009550 con 0x7fbffc09de00 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbff9d9b700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc13bbd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009c40 tx=0x7fbff0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbfeaffd700 1 -- 172.21.15.89:0/866015679 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004550 con 0x7fbffc09de00 2023-09-30T22:18:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbfeaffd700 1 -- 172.21.15.89:0/866015679 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff00036a0 con 0x7fbffc09de00 2023-09-30T22:18:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fbfeaffd700 1 -- 172.21.15.89:0/866015679 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0003a30 con 0x7fbffc09de00 2023-09-30T22:18:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.833+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc072520 con 0x7fbffc09de00 2023-09-30T22:18:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.834+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc11def0 con 0x7fbffc09de00 2023-09-30T22:18:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.835+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc11d5c0 con 0x7fbffc09de00 2023-09-30T22:18:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.835+0000 7fbfeaffd700 1 -- 172.21.15.89:0/866015679 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbff0021070 con 0x7fbffc09de00 2023-09-30T22:18:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.836+0000 7fbfeaffd700 1 -- 172.21.15.89:0/866015679 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbff0040ba0 con 0x7fbffc09de00 2023-09-30T22:18:54.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.840+0000 7fbfeaffd700 1 -- 172.21.15.89:0/866015679 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbff0019770 con 0x7fbffc09de00 2023-09-30T22:18:54.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.861+0000 7fdc68449700 1 Processor -- start 2023-09-30T22:18:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.862+0000 7fdc68449700 1 -- start start 2023-09-30T22:18:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.862+0000 7fdc68449700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012cf60 0x7fdc6006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.862+0000 7fdc68449700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 0x7fdc60137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.862+0000 7fdc68449700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6012cd90 con 0x7fdc6012f8b0 2023-09-30T22:18:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.862+0000 7fdc68449700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6012d330 con 0x7fdc6012cf60 2023-09-30T22:18:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.862+0000 7fdc68449700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60071780 con 0x7fdc60130430 2023-09-30T22:18:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.862+0000 7fdc661e5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012cf60 0x7fdc6006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc661e5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012cf60 0x7fdc6006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45500/0 (socket says 172.21.15.89:45500) 2023-09-30T22:18:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc661e5700 1 -- 172.21.15.89:0/3122610369 learned_addr learned my addr 172.21.15.89:0/3122610369 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc669e6700 1 --2- 172.21.15.89:0/3122610369 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 0x7fdc60137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc669e6700 1 -- 172.21.15.89:0/3122610369 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012cf60 msgr2=0x7fdc6006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:54.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc669e6700 1 --2- 172.21.15.89:0/3122610369 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012cf60 0x7fdc6006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc669e6700 1 -- 172.21.15.89:0/3122610369 >> v1:172.21.15.89:6789/0 conn(0x7fdc6012f8b0 legacy=0x7fdc6012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc651e3700 1 -- 172.21.15.89:0/3122610369 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819720188 0 0) 0x7fdc6012cd90 con 0x7fdc6012f8b0 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc669e6700 1 -- 172.21.15.89:0/3122610369 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60071a40 con 0x7fdc60130430 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc669e6700 1 --2- 172.21.15.89:0/3122610369 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 0x7fdc60137c30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009dc0 tx=0x7fdc5800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775a9385d9adf968 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc651e3700 1 -- 172.21.15.89:0/3122610369 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800e070 con 0x7fdc60130430 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc651e3700 1 -- 172.21.15.89:0/3122610369 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5800bb20 con 0x7fdc60130430 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.863+0000 7fdc651e3700 1 -- 172.21.15.89:0/3122610369 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58012430 con 0x7fdc60130430 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.864+0000 7fdc68449700 1 -- 172.21.15.89:0/3122610369 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 msgr2=0x7fdc60137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.864+0000 7fdc68449700 1 --2- 172.21.15.89:0/3122610369 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 0x7fdc60137c30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009dc0 tx=0x7fdc5800b330 comp rx=0 tx=0).stop 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.864+0000 7fdc68449700 1 -- 172.21.15.89:0/3122610369 shutdown_connections 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.864+0000 7fdc68449700 1 --2- 172.21.15.89:0/3122610369 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012cf60 0x7fdc6006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.864+0000 7fdc68449700 1 --2- 172.21.15.89:0/3122610369 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 0x7fdc60137c30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.864+0000 7fdc68449700 1 -- 172.21.15.89:0/3122610369 >> 172.21.15.89:0/3122610369 conn(0x7fdc600a2aa0 msgr2=0x7fdc600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.864+0000 7fdc68449700 1 -- 172.21.15.89:0/3122610369 shutdown_connections 2023-09-30T22:18:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.864+0000 7fdc68449700 1 -- 172.21.15.89:0/3122610369 wait complete. 2023-09-30T22:18:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.865+0000 7fdc68449700 1 Processor -- start 2023-09-30T22:18:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc68449700 1 -- start start 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc68449700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc6012cf60 0x7fdc6009bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc68449700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012f8b0 0x7fdc6009c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc68449700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 0x7fdc601c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc68449700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6012da40 con 0x7fdc6012cf60 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc68449700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6009b800 con 0x7fdc6012f8b0 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc68449700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6009c770 con 0x7fdc60130430 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc661e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc6012cf60 0x7fdc6009bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc659e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012f8b0 0x7fdc6009c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc661e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc6012cf60 0x7fdc6009bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47328/0 (socket says 172.21.15.89:47328) 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc661e5700 1 -- 172.21.15.89:0/2191771151 learned_addr learned my addr 172.21.15.89:0/2191771151 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.866+0000 7fdc659e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012f8b0 0x7fdc6009c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45504/0 (socket says 172.21.15.89:45504) 2023-09-30T22:18:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc659e4700 1 -- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 msgr2=0x7fdc601c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc659e4700 1 --2- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 0x7fdc601c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc659e4700 1 -- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc6012cf60 msgr2=0x7fdc6009bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc659e4700 1 --2- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc6012cf60 0x7fdc6009bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc659e4700 1 -- 172.21.15.89:0/2191771151 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc58009a70 con 0x7fdc6012f8b0 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc661e5700 1 --2- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc6012cf60 0x7fdc6009bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc659e4700 1 --2- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012f8b0 0x7fdc6009c230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdc50000f60 tx=0x7fdc50001c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc56ffd700 1 -- 172.21.15.89:0/2191771151 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50013070 con 0x7fdc6012f8b0 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc56ffd700 1 -- 172.21.15.89:0/2191771151 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5000ee40 con 0x7fdc6012f8b0 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc56ffd700 1 -- 172.21.15.89:0/2191771151 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500050d0 con 0x7fdc6012f8b0 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.867+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6009deb0 con 0x7fdc6012f8b0 2023-09-30T22:18:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.868+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6009cf30 con 0x7fdc6012f8b0 2023-09-30T22:18:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.869+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc60074ea0 con 0x7fdc6012f8b0 2023-09-30T22:18:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.869+0000 7fdc56ffd700 1 -- 172.21.15.89:0/2191771151 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc50005230 con 0x7fdc6012f8b0 2023-09-30T22:18:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.869+0000 7fdc56ffd700 1 -- 172.21.15.89:0/2191771151 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdc50043200 con 0x7fdc6012f8b0 2023-09-30T22:18:54.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.873+0000 7fdc56ffd700 1 -- 172.21.15.89:0/2191771151 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc500054e0 con 0x7fdc6012f8b0 2023-09-30T22:18:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.964+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4620074ea0 con 0x7f462009f240 2023-09-30T22:18:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.964+0000 7f46077fe700 1 -- 172.21.15.89:0/4156391969 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46140036a0 con 0x7f462009f240 2023-09-30T22:18:54.965 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f462009f240 msgr2=0x7f46201c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 --2- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f462009f240 0x7f46201c5cd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f46140109b0 tx=0x7f4614010cc0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 shutdown_connections 2023-09-30T22:18:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 --2- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f462009f650 0x7f46201c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 --2- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f462009e930 0x7f46201a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 --2- 172.21.15.89:0/4156391969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f462009f240 0x7f46201c5cd0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 >> 172.21.15.89:0/4156391969 conn(0x7f46200a4210 msgr2=0x7f46201b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 shutdown_connections 2023-09-30T22:18:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.966+0000 7f4625453700 1 -- 172.21.15.89:0/4156391969 wait complete. 2023-09-30T22:18:54.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.989+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbc640a3b80 con 0x7fbc6409d6f0 2023-09-30T22:18:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.989+0000 7fbc617fa700 1 -- 172.21.15.89:0/3104408860 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc5801c350 con 0x7fbc6409d6f0 2023-09-30T22:18:54.990 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc6409d6f0 msgr2=0x7fbc641c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc6409d6f0 0x7fbc641c4030 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c4e0 tx=0x7fbc5800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 shutdown_connections 2023-09-30T22:18:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc64135e50 0x7fbc641c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc6409d6f0 0x7fbc641c4030 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 --2- 172.21.15.89:0/3104408860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc6409e1d0 0x7fbc641c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:54.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 >> 172.21.15.89:0/3104408860 conn(0x7fbc64066f50 msgr2=0x7fbc6409a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:54.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 shutdown_connections 2023-09-30T22:18:54.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:54.991+0000 7fbc6ad00700 1 -- 172.21.15.89:0/3104408860 wait complete. 2023-09-30T22:18:55.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:55.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.043+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbffc074ea0 con 0x7fbffc09de00 2023-09-30T22:18:55.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.044+0000 7fbfeaffd700 1 -- 172.21.15.89:0/866015679 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbff0029030 con 0x7fbffc09de00 2023-09-30T22:18:55.045 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 msgr2=0x7fbffc13bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc13bbd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009c40 tx=0x7fbff0004750 comp rx=0 tx=0).stop 2023-09-30T22:18:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 shutdown_connections 2023-09-30T22:18:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbffc13d1f0 0x7fbffc13c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbffc06a770 0x7fbffc13b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 --2- 172.21.15.89:0/866015679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbffc09de00 0x7fbffc13bbd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 >> 172.21.15.89:0/866015679 conn(0x7fbffc066f50 msgr2=0x7fbffc09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 shutdown_connections 2023-09-30T22:18:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.046+0000 7fc000bb2700 1 -- 172.21.15.89:0/866015679 wait complete. 2023-09-30T22:18:55.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.077+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdc6012e2d0 con 0x7fdc6012f8b0 2023-09-30T22:18:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.077+0000 7fdc56ffd700 1 -- 172.21.15.89:0/2191771151 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc50018420 con 0x7fdc6012f8b0 2023-09-30T22:18:55.078 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012f8b0 msgr2=0x7fdc6009c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 --2- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012f8b0 0x7fdc6009c230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdc50000f60 tx=0x7fdc50001c30 comp rx=0 tx=0).stop 2023-09-30T22:18:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 shutdown_connections 2023-09-30T22:18:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 --2- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc60130430 0x7fdc601c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 --2- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc6012cf60 0x7fdc6009bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 --2- 172.21.15.89:0/2191771151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc6012f8b0 0x7fdc6009c230 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 >> 172.21.15.89:0/2191771151 conn(0x7fdc600a2aa0 msgr2=0x7fdc600a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 shutdown_connections 2023-09-30T22:18:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:55.079+0000 7fdc68449700 1 -- 172.21.15.89:0/2191771151 wait complete. 2023-09-30T22:18:55.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:55.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:56.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:56.059 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:56.092 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:56.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a3c44700 1 Processor -- start 2023-09-30T22:18:56.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a3c44700 1 -- start start 2023-09-30T22:18:56.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a3c44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c1338c0 0x7f189c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a3c44700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1341d0 0x7f189c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a3c44700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f189c09e390 0x7f189c09e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a3c44700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c070c80 con 0x7f189c09e390 2023-09-30T22:18:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a3c44700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c06f720 con 0x7f189c1341d0 2023-09-30T22:18:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a3c44700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f189c06fce0 con 0x7f189c1338c0 2023-09-30T22:18:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.117+0000 7f18a19e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c1338c0 0x7f189c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a19e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c1338c0 0x7f189c133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38228/0 (socket says 172.21.15.89:38228) 2023-09-30T22:18:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a19e0700 1 -- 172.21.15.89:0/3259779868 learned_addr learned my addr 172.21.15.89:0/3259779868 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:56.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a11df700 1 --2- 172.21.15.89:0/3259779868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1341d0 0x7f189c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a21e1700 1 --2- 172.21.15.89:0/3259779868 >> v2:172.21.15.89:3300/0 conn(0x7f189c09e390 0x7f189c09e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a11df700 1 -- 172.21.15.89:0/3259779868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c1338c0 msgr2=0x7f189c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a11df700 1 --2- 172.21.15.89:0/3259779868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c1338c0 0x7f189c133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a11df700 1 -- 172.21.15.89:0/3259779868 >> v2:172.21.15.89:3300/0 conn(0x7f189c09e390 msgr2=0x7f189c09e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a11df700 1 --2- 172.21.15.89:0/3259779868 >> v2:172.21.15.89:3300/0 conn(0x7f189c09e390 0x7f189c09e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a11df700 1 -- 172.21.15.89:0/3259779868 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c1336a0 con 0x7f189c1341d0 2023-09-30T22:18:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a11df700 1 --2- 172.21.15.89:0/3259779868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1341d0 0x7f189c09de50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f188c009840 tx=0x7f188c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5d9dd226d254ac9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a09de700 1 -- 172.21.15.89:0/3259779868 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c011830 con 0x7f189c1341d0 2023-09-30T22:18:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a09de700 1 -- 172.21.15.89:0/3259779868 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c011990 con 0x7f189c1341d0 2023-09-30T22:18:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.118+0000 7f18a09de700 1 -- 172.21.15.89:0/3259779868 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c011d20 con 0x7f189c1341d0 2023-09-30T22:18:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.119+0000 7f18a3c44700 1 -- 172.21.15.89:0/3259779868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1341d0 msgr2=0x7f189c09de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.119+0000 7f18a3c44700 1 --2- 172.21.15.89:0/3259779868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1341d0 0x7f189c09de50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f188c009840 tx=0x7f188c009b50 comp rx=0 tx=0).stop 2023-09-30T22:18:56.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.119+0000 7f18a3c44700 1 -- 172.21.15.89:0/3259779868 shutdown_connections 2023-09-30T22:18:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.119+0000 7f18a3c44700 1 --2- 172.21.15.89:0/3259779868 >> v2:172.21.15.89:3300/0 conn(0x7f189c09e390 0x7f189c09e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.119+0000 7f18a3c44700 1 --2- 172.21.15.89:0/3259779868 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c1338c0 0x7f189c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.119+0000 7f18a3c44700 1 --2- 172.21.15.89:0/3259779868 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1341d0 0x7f189c09de50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.119+0000 7f18a3c44700 1 -- 172.21.15.89:0/3259779868 >> 172.21.15.89:0/3259779868 conn(0x7f189c0a2aa0 msgr2=0x7f189c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.120+0000 7f18a3c44700 1 -- 172.21.15.89:0/3259779868 shutdown_connections 2023-09-30T22:18:56.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.120+0000 7f18a3c44700 1 -- 172.21.15.89:0/3259779868 wait complete. 2023-09-30T22:18:56.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.121+0000 7f18a3c44700 1 Processor -- start 2023-09-30T22:18:56.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.121+0000 7f18a3c44700 1 -- start start 2023-09-30T22:18:56.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a3c44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c09e390 0x7f189c11e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a3c44700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1338c0 0x7f189c11ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a3c44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f189c1341d0 0x7f189c11f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a3c44700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c1221c0 con 0x7f189c1341d0 2023-09-30T22:18:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a3c44700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c1212b0 con 0x7f189c1338c0 2023-09-30T22:18:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a3c44700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f189c12fd30 con 0x7f189c09e390 2023-09-30T22:18:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a11df700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1338c0 0x7f189c11ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a11df700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1338c0 0x7f189c11ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45520/0 (socket says 172.21.15.89:45520) 2023-09-30T22:18:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a11df700 1 -- 172.21.15.89:0/2385732045 learned_addr learned my addr 172.21.15.89:0/2385732045 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a19e0700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c09e390 0x7f189c11e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a21e1700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f189c1341d0 0x7f189c11f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a19e0700 1 -- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1338c0 msgr2=0x7f189c11ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a19e0700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1338c0 0x7f189c11ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a19e0700 1 -- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f189c1341d0 msgr2=0x7f189c11f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a19e0700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f189c1341d0 0x7f189c11f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.122+0000 7f18a19e0700 1 -- 172.21.15.89:0/2385732045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0094a0 con 0x7f189c09e390 2023-09-30T22:18:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.123+0000 7f18a19e0700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c09e390 0x7f189c11e940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f189800e8e0 tx=0x7f189800ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.123+0000 7f18927fc700 1 -- 172.21.15.89:0/2385732045 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898013680 con 0x7f189c09e390 2023-09-30T22:18:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.123+0000 7f18927fc700 1 -- 172.21.15.89:0/2385732045 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18980137e0 con 0x7f189c09e390 2023-09-30T22:18:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.123+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c130e50 con 0x7f189c09e390 2023-09-30T22:18:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.123+0000 7f18927fc700 1 -- 172.21.15.89:0/2385732045 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898013b70 con 0x7f189c09e390 2023-09-30T22:18:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.123+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c1228c0 con 0x7f189c09e390 2023-09-30T22:18:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.124+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c122000 con 0x7f189c09e390 2023-09-30T22:18:56.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.124+0000 7f18927fc700 1 -- 172.21.15.89:0/2385732045 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1898013cd0 con 0x7f189c09e390 2023-09-30T22:18:56.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.129+0000 7f18927fc700 1 -- 172.21.15.89:0/2385732045 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f189801ae10 con 0x7f189c09e390 2023-09-30T22:18:56.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.133+0000 7f18927fc700 1 -- 172.21.15.89:0/2385732045 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f189801bc30 con 0x7f189c09e390 2023-09-30T22:18:56.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.161+0000 7f7d19d1e700 1 Processor -- start 2023-09-30T22:18:56.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.161+0000 7f7d19d1e700 1 -- start start 2023-09-30T22:18:56.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.162+0000 7f7d19d1e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d14130540 0x7f7d14130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.162+0000 7f7d19d1e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d14130e50 0x7f7d1406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.162+0000 7f7d19d1e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d140716a0 con 0x7f7d1406fb20 2023-09-30T22:18:56.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.162+0000 7f7d19d1e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1409a390 con 0x7f7d14130540 2023-09-30T22:18:56.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.162+0000 7f7d19d1e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14071290 con 0x7f7d14130e50 2023-09-30T22:18:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.162+0000 7f7d12ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d14130e50 0x7f7d1406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.162+0000 7f7d12ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d14130e50 0x7f7d1406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38246/0 (socket says 172.21.15.89:38246) 2023-09-30T22:18:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.162+0000 7f7d12ffd700 1 -- 172.21.15.89:0/169325908 learned_addr learned my addr 172.21.15.89:0/169325908 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.163+0000 7f7d12ffd700 1 -- 172.21.15.89:0/169325908 >> v2:172.21.15.89:3300/0 conn(0x7f7d14130540 msgr2=0x7f7d14130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.163+0000 7f7d12ffd700 1 --2- 172.21.15.89:0/169325908 >> v2:172.21.15.89:3300/0 conn(0x7f7d14130540 0x7f7d14130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.163+0000 7f7d12ffd700 1 -- 172.21.15.89:0/169325908 >> v1:172.21.15.89:6789/0 conn(0x7f7d1406fb20 legacy=0x7f7d14137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.163+0000 7f7d12ffd700 1 -- 172.21.15.89:0/169325908 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14130340 con 0x7f7d14130e50 2023-09-30T22:18:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.163+0000 7f7d12ffd700 1 --2- 172.21.15.89:0/169325908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d14130e50 0x7f7d1406f5e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009920 tx=0x7f7d08009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2514480c18bba0cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.163+0000 7f7d127fc700 1 -- 172.21.15.89:0/169325908 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08004230 con 0x7f7d14130e50 2023-09-30T22:18:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.163+0000 7f7d127fc700 1 -- 172.21.15.89:0/169325908 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08004390 con 0x7f7d14130e50 2023-09-30T22:18:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.163+0000 7f7d127fc700 1 -- 172.21.15.89:0/169325908 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08009e70 con 0x7f7d14130e50 2023-09-30T22:18:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.164+0000 7f7d19d1e700 1 -- 172.21.15.89:0/169325908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d14130e50 msgr2=0x7f7d1406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.164+0000 7f7d19d1e700 1 --2- 172.21.15.89:0/169325908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d14130e50 0x7f7d1406f5e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009920 tx=0x7f7d08009c30 comp rx=0 tx=0).stop 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.164+0000 7f7d19d1e700 1 -- 172.21.15.89:0/169325908 shutdown_connections 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.164+0000 7f7d19d1e700 1 --2- 172.21.15.89:0/169325908 >> v2:172.21.15.89:3300/0 conn(0x7f7d14130540 0x7f7d14130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.164+0000 7f7d19d1e700 1 --2- 172.21.15.89:0/169325908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d14130e50 0x7f7d1406f5e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.164+0000 7f7d19d1e700 1 -- 172.21.15.89:0/169325908 >> 172.21.15.89:0/169325908 conn(0x7f7d140a2aa0 msgr2=0x7f7d140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.164+0000 7f7d19d1e700 1 -- 172.21.15.89:0/169325908 shutdown_connections 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.164+0000 7f7d19d1e700 1 -- 172.21.15.89:0/169325908 wait complete. 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.165+0000 7f7d19d1e700 1 Processor -- start 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.165+0000 7f7d19d1e700 1 -- start start 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.165+0000 7f7d19d1e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1406fb20 0x7f7d1409a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.165+0000 7f7d19d1e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14130540 0x7f7d1409ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.165+0000 7f7d19d1e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d14130e50 0x7f7d1409b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.165+0000 7f7d19d1e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1409c400 con 0x7f7d14130540 2023-09-30T22:18:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.165+0000 7f7d19d1e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1409c1d0 con 0x7f7d14130e50 2023-09-30T22:18:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.165+0000 7f7d19d1e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14120600 con 0x7f7d1406fb20 2023-09-30T22:18:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14130540 0x7f7d1409ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14130540 0x7f7d1409ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47360/0 (socket says 172.21.15.89:47360) 2023-09-30T22:18:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 -- 172.21.15.89:0/1485698013 learned_addr learned my addr 172.21.15.89:0/1485698013 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d13fff700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d14130e50 0x7f7d1409b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d137fe700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1406fb20 0x7f7d1409a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 -- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1406fb20 msgr2=0x7f7d1409a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1406fb20 0x7f7d1409a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 -- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d14130e50 msgr2=0x7f7d1409b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d14130e50 0x7f7d1409b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 -- 172.21.15.89:0/1485698013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08009550 con 0x7f7d14130540 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d13fff700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d14130e50 0x7f7d1409b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7d12ffd700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14130540 0x7f7d1409ab00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7d140713d0 tx=0x7f7d08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7cfbfff700 1 -- 172.21.15.89:0/1485698013 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0800be20 con 0x7f7d14130540 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7cfbfff700 1 -- 172.21.15.89:0/1485698013 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d080036a0 con 0x7f7d14130540 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.166+0000 7f7cfbfff700 1 -- 172.21.15.89:0/1485698013 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08003a30 con 0x7f7d14130540 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.167+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d141202b0 con 0x7f7d14130540 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.167+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d140718e0 con 0x7f7d14130540 2023-09-30T22:18:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.168+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1412d8f0 con 0x7f7d14130540 2023-09-30T22:18:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.168+0000 7f7cfbfff700 1 -- 172.21.15.89:0/1485698013 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d08022020 con 0x7f7d14130540 2023-09-30T22:18:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.169+0000 7f7cfbfff700 1 -- 172.21.15.89:0/1485698013 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7d08041160 con 0x7f7d14130540 2023-09-30T22:18:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.173+0000 7f7cfbfff700 1 -- 172.21.15.89:0/1485698013 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d080199e0 con 0x7f7d14130540 2023-09-30T22:18:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.200+0000 7f71f5b65700 1 Processor -- start 2023-09-30T22:18:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.200+0000 7f71f5b65700 1 -- start start 2023-09-30T22:18:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71f5b65700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f71f009d6f0 0x7f71f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71f5b65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71f5b65700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f009f130 con 0x7f71f009e000 2023-09-30T22:18:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71f5b65700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f009f520 con 0x7f71f009d6f0 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71f5b65700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71f009e930 con 0x7f71f009e410 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71ef7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f71f009d6f0 0x7f71f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71ef7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f71f009d6f0 0x7f71f009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47364/0 (socket says 172.21.15.89:47364) 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71ef7fe700 1 -- 172.21.15.89:0/3746855270 learned_addr learned my addr 172.21.15.89:0/3746855270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71effff700 1 --2- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71ef7fe700 1 -- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 msgr2=0x7f71f0130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71ef7fe700 1 --2- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71ef7fe700 1 -- 172.21.15.89:0/3746855270 >> v1:172.21.15.89:6789/0 conn(0x7f71f009e000 legacy=0x7f71f0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71ef7fe700 1 -- 172.21.15.89:0/3746855270 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f009b2d0 con 0x7f71f009d6f0 2023-09-30T22:18:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71effff700 1 --2- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.201+0000 7f71ef7fe700 1 --2- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009d6f0 0x7f71f009dac0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f71e000bb90 tx=0x7f71e000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566554e6d3be1efe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71ee7fc700 1 -- 172.21.15.89:0/3746855270 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000dd30 con 0x7f71f009d6f0 2023-09-30T22:18:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71ee7fc700 1 -- 172.21.15.89:0/3746855270 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e000de90 con 0x7f71f009d6f0 2023-09-30T22:18:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71ee7fc700 1 -- 172.21.15.89:0/3746855270 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0010690 con 0x7f71f009d6f0 2023-09-30T22:18:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71f5b65700 1 -- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009d6f0 msgr2=0x7f71f009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71f5b65700 1 --2- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009d6f0 0x7f71f009dac0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f71e000bb90 tx=0x7f71e000bea0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71f5b65700 1 -- 172.21.15.89:0/3746855270 shutdown_connections 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71f5b65700 1 --2- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009d6f0 0x7f71f009dac0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71f5b65700 1 --2- 172.21.15.89:0/3746855270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71f5b65700 1 -- 172.21.15.89:0/3746855270 >> 172.21.15.89:0/3746855270 conn(0x7f71f0066f50 msgr2=0x7f71f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71f5b65700 1 -- 172.21.15.89:0/3746855270 shutdown_connections 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.202+0000 7f71f5b65700 1 -- 172.21.15.89:0/3746855270 wait complete. 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.203+0000 7f71f5b65700 1 Processor -- start 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71f5b65700 1 -- start start 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71f5b65700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71f009d6f0 0x7f71f0122460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71f5b65700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009e000 0x7f71f01229a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71f5b65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0122ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71f5b65700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0125ca0 con 0x7f71f009e000 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71f5b65700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0124180 con 0x7f71f009d6f0 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71f5b65700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00a1d00 con 0x7f71f009e410 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71f009d6f0 0x7f71f0122460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71ef7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71f009d6f0 0x7f71f0122460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45552/0 (socket says 172.21.15.89:45552) 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71ef7fe700 1 -- 172.21.15.89:0/970140232 learned_addr learned my addr 172.21.15.89:0/970140232 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71eeffd700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009e000 0x7f71f01229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.204+0000 7f71effff700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0122ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71eeffd700 1 -- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 msgr2=0x7f71f0122ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71eeffd700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0122ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71eeffd700 1 -- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71f009d6f0 msgr2=0x7f71f0122460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71eeffd700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71f009d6f0 0x7f71f0122460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71eeffd700 1 -- 172.21.15.89:0/970140232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e8009a70 con 0x7f71f009e000 2023-09-30T22:18:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71eeffd700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009e000 0x7f71f01229a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f71e4000f60 tx=0x7f71e400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71d7fff700 1 -- 172.21.15.89:0/970140232 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40052d0 con 0x7f71f009e000 2023-09-30T22:18:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71d7fff700 1 -- 172.21.15.89:0/970140232 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e4005430 con 0x7f71f009e000 2023-09-30T22:18:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e000b7f0 con 0x7f71f009e000 2023-09-30T22:18:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71d7fff700 1 -- 172.21.15.89:0/970140232 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40038c0 con 0x7f71f009e000 2023-09-30T22:18:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.205+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f009c750 con 0x7f71f009e000 2023-09-30T22:18:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.206+0000 7f71d7fff700 1 -- 172.21.15.89:0/970140232 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71e4003a20 con 0x7f71f009e000 2023-09-30T22:18:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.206+0000 7f71d7fff700 1 -- 172.21.15.89:0/970140232 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f71e403e940 con 0x7f71f009e000 2023-09-30T22:18:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.206+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f01267d0 con 0x7f71f009e000 2023-09-30T22:18:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.215+0000 7f71d7fff700 1 -- 172.21.15.89:0/970140232 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71e4011590 con 0x7f71f009e000 2023-09-30T22:18:56.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.256+0000 7f6b0cc9d700 1 Processor -- start 2023-09-30T22:18:56.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.256+0000 7f6b0cc9d700 1 -- start start 2023-09-30T22:18:56.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0cc9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133420 0x7f6b081337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0cc9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133e30 0x7f6b0812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0cc9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b08134260 0x7f6b081304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0cc9d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0806b990 con 0x7f6b08134260 2023-09-30T22:18:56.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0cc9d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080a0590 con 0x7f6b08133e30 2023-09-30T22:18:56.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0cc9d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0809fb80 con 0x7f6b08133420 2023-09-30T22:18:56.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133420 0x7f6b081337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133420 0x7f6b081337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38290/0 (socket says 172.21.15.89:38290) 2023-09-30T22:18:56.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b0659c700 1 -- 172.21.15.89:0/480468377 learned_addr learned my addr 172.21.15.89:0/480468377 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:56.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b06d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b08134260 0x7f6b081304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b05d9b700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133e30 0x7f6b0812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b06d9d700 1 -- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133420 msgr2=0x7f6b081337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:56.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b06d9d700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133420 0x7f6b081337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b06d9d700 1 -- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133e30 msgr2=0x7f6b0812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:56.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b06d9d700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133e30 0x7f6b0812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b06d9d700 1 -- 172.21.15.89:0/480468377 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080a0030 con 0x7f6b08134260 2023-09-30T22:18:56.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.257+0000 7f6b05d9b700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133e30 0x7f6b0812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:18:56.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.258+0000 7f6b06d9d700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 0x7f6b081304d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009d90 tx=0x7f6b0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95ea6866495b14d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:56.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.258+0000 7f6b0559a700 1 -- 172.21.15.89:0/480468377 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000e070 con 0x7f6b08134260 2023-09-30T22:18:56.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.258+0000 7f6b0559a700 1 -- 172.21.15.89:0/480468377 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0000bbd0 con 0x7f6b08134260 2023-09-30T22:18:56.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.258+0000 7f6b0559a700 1 -- 172.21.15.89:0/480468377 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00012430 con 0x7f6b08134260 2023-09-30T22:18:56.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 msgr2=0x7f6b081304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 0x7f6b081304d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009d90 tx=0x7f6b0000b330 comp rx=0 tx=0).stop 2023-09-30T22:18:56.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/480468377 shutdown_connections 2023-09-30T22:18:56.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 0x7f6b081304d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133420 0x7f6b081337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 --2- 172.21.15.89:0/480468377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133e30 0x7f6b0812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/480468377 >> 172.21.15.89:0/480468377 conn(0x7f6b080a4210 msgr2=0x7f6b080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:56.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/480468377 shutdown_connections 2023-09-30T22:18:56.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.259+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/480468377 wait complete. 2023-09-30T22:18:56.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0cc9d700 1 Processor -- start 2023-09-30T22:18:56.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0cc9d700 1 -- start start 2023-09-30T22:18:56.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0cc9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133420 0x7f6b0812d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0cc9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133e30 0x7f6b0809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0cc9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 0x7f6b0809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:56.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0cc9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0811fe20 con 0x7f6b08134260 2023-09-30T22:18:56.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0cc9d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08120680 con 0x7f6b08133420 2023-09-30T22:18:56.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0cc9d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0811f390 con 0x7f6b08133e30 2023-09-30T22:18:56.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.260+0000 7f6b0659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133420 0x7f6b0812d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133420 0x7f6b0812d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45576/0 (socket says 172.21.15.89:45576) 2023-09-30T22:18:56.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0659c700 1 -- 172.21.15.89:0/178871407 learned_addr learned my addr 172.21.15.89:0/178871407 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:56.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b06d9d700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 0x7f6b0809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b05d9b700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133e30 0x7f6b0809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:56.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0659c700 1 -- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133e30 msgr2=0x7f6b0809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0659c700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133e30 0x7f6b0809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0659c700 1 -- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 msgr2=0x7f6b0809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0659c700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 0x7f6b0809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0659c700 1 -- 172.21.15.89:0/178871407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b00009a70 con 0x7f6b08133420 2023-09-30T22:18:56.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b06d9d700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 0x7f6b0809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:56.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0659c700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133420 0x7f6b0812d890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6af800e7e0 tx=0x7f6af800eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:56.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6af6ffd700 1 -- 172.21.15.89:0/178871407 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8013610 con 0x7f6b08133420 2023-09-30T22:18:56.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6af6ffd700 1 -- 172.21.15.89:0/178871407 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af8013770 con 0x7f6b08133420 2023-09-30T22:18:56.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b08120070 con 0x7f6b08133420 2023-09-30T22:18:56.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6af6ffd700 1 -- 172.21.15.89:0/178871407 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8013b20 con 0x7f6b08133420 2023-09-30T22:18:56.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.261+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0809e190 con 0x7f6b08133420 2023-09-30T22:18:56.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.262+0000 7f6af6ffd700 1 -- 172.21.15.89:0/178871407 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6af8013c80 con 0x7f6b08133420 2023-09-30T22:18:56.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.262+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b08074ea0 con 0x7f6b08133420 2023-09-30T22:18:56.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.263+0000 7f6af6ffd700 1 -- 172.21.15.89:0/178871407 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6af803f340 con 0x7f6b08133420 2023-09-30T22:18:56.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.271+0000 7f6af6ffd700 1 -- 172.21.15.89:0/178871407 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6af8012370 con 0x7f6b08133420 2023-09-30T22:18:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.341+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f189c074ea0 con 0x7f189c09e390 2023-09-30T22:18:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.341+0000 7f18927fc700 1 -- 172.21.15.89:0/2385732045 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1898017030 con 0x7f189c09e390 2023-09-30T22:18:56.342 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.343+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c09e390 msgr2=0x7f189c11e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.343+0000 7f18a3c44700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c09e390 0x7f189c11e940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f189800e8e0 tx=0x7f189800ebf0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.343+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 shutdown_connections 2023-09-30T22:18:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.343+0000 7f18a3c44700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f189c1341d0 0x7f189c11f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.343+0000 7f18a3c44700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f189c09e390 0x7f189c11e940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.344+0000 7f18a3c44700 1 --2- 172.21.15.89:0/2385732045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f189c1338c0 0x7f189c11ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.344+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 >> 172.21.15.89:0/2385732045 conn(0x7f189c0a2aa0 msgr2=0x7f189c1306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.344+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 shutdown_connections 2023-09-30T22:18:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.344+0000 7f18a3c44700 1 -- 172.21.15.89:0/2385732045 wait complete. 2023-09-30T22:18:56.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.371+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7d1412d8f0 con 0x7f7d14130540 2023-09-30T22:18:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.371+0000 7f7cfbfff700 1 -- 172.21.15.89:0/1485698013 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d1412d8f0 con 0x7f7d14130540 2023-09-30T22:18:56.372 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.374+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14130540 msgr2=0x7f7d1409ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.374+0000 7f7d19d1e700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14130540 0x7f7d1409ab00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7d140713d0 tx=0x7f7d08004750 comp rx=0 tx=0).stop 2023-09-30T22:18:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.374+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 shutdown_connections 2023-09-30T22:18:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.374+0000 7f7d19d1e700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d14130e50 0x7f7d1409b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.374+0000 7f7d19d1e700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d1406fb20 0x7f7d1409a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.374+0000 7f7d19d1e700 1 --2- 172.21.15.89:0/1485698013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d14130540 0x7f7d1409ab00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.374+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 >> 172.21.15.89:0/1485698013 conn(0x7f7d140a2aa0 msgr2=0x7f7d140a1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.375+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 shutdown_connections 2023-09-30T22:18:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.375+0000 7f7d19d1e700 1 -- 172.21.15.89:0/1485698013 wait complete. 2023-09-30T22:18:56.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:56.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.430+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f71f0074ea0 con 0x7f71f009e000 2023-09-30T22:18:56.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.430+0000 7f71d7fff700 1 -- 172.21.15.89:0/970140232 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71e4017020 con 0x7f71f009e000 2023-09-30T22:18:56.432 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:56.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009e000 msgr2=0x7f71f01229a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009e000 0x7f71f01229a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f71e4000f60 tx=0x7f71e400c730 comp rx=0 tx=0).stop 2023-09-30T22:18:56.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 shutdown_connections 2023-09-30T22:18:56.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f71f009e410 0x7f71f0122ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71f009d6f0 0x7f71f0122460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 --2- 172.21.15.89:0/970140232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f009e000 0x7f71f01229a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 >> 172.21.15.89:0/970140232 conn(0x7f71f0066f50 msgr2=0x7f71f00710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:56.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 shutdown_connections 2023-09-30T22:18:56.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.433+0000 7f71f5b65700 1 -- 172.21.15.89:0/970140232 wait complete. 2023-09-30T22:18:56.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.469+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6b0806b250 con 0x7f6b08133420 2023-09-30T22:18:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.469+0000 7f6af6ffd700 1 -- 172.21.15.89:0/178871407 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6af8021070 con 0x7f6b08133420 2023-09-30T22:18:56.471 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133420 msgr2=0x7f6b0812d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133420 0x7f6b0812d890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6af800e7e0 tx=0x7f6af800eaf0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 shutdown_connections 2023-09-30T22:18:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b08134260 0x7f6b0809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b08133420 0x7f6b0812d890 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 --2- 172.21.15.89:0/178871407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b08133e30 0x7f6b0809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 >> 172.21.15.89:0/178871407 conn(0x7f6b080a4210 msgr2=0x7f6b080a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:56.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 shutdown_connections 2023-09-30T22:18:56.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:56.471+0000 7f6b0cc9d700 1 -- 172.21.15.89:0/178871407 wait complete. 2023-09-30T22:18:56.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:57.358 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:57.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:57.446 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:57.485 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:57.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.508+0000 7ff17a5aa700 1 Processor -- start 2023-09-30T22:18:57.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.508+0000 7ff17a5aa700 1 -- start start 2023-09-30T22:18:57.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.508+0000 7ff17a5aa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1741338c0 0x7ff174133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.508+0000 7ff17a5aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff17409e560 0x7ff17413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff17a5aa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17406f720 con 0x7ff1741341d0 2023-09-30T22:18:57.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff17a5aa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff174070c80 con 0x7ff1741338c0 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff17a5aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff17406fce0 con 0x7ff17409e560 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff173fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1741338c0 0x7ff174133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff173fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1741338c0 0x7ff174133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47404/0 (socket says 172.21.15.89:47404) 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff173fff700 1 -- 172.21.15.89:0/2211552406 learned_addr learned my addr 172.21.15.89:0/2211552406 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff178b47700 1 --2- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff17409e560 0x7ff17413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff173fff700 1 -- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff17409e560 msgr2=0x7ff17413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff173fff700 1 --2- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff17409e560 0x7ff17413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff173fff700 1 -- 172.21.15.89:0/2211552406 >> v1:172.21.15.89:6789/0 conn(0x7ff1741341d0 legacy=0x7ff17409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff173fff700 1 -- 172.21.15.89:0/2211552406 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1741336a0 con 0x7ff1741338c0 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff178b47700 1 --2- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff17409e560 0x7ff17413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff173fff700 1 --2- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1741338c0 0x7ff174133c90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff1640098f0 tx=0x7ff164009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3ff3dac247fd33 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:57.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.509+0000 7ff172ffd700 1 -- 172.21.15.89:0/2211552406 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164011830 con 0x7ff1741338c0 2023-09-30T22:18:57.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff172ffd700 1 -- 172.21.15.89:0/2211552406 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff164011990 con 0x7ff1741338c0 2023-09-30T22:18:57.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff172ffd700 1 -- 172.21.15.89:0/2211552406 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164011d20 con 0x7ff1741338c0 2023-09-30T22:18:57.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff17a5aa700 1 -- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1741338c0 msgr2=0x7ff174133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff17a5aa700 1 --2- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1741338c0 0x7ff174133c90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff1640098f0 tx=0x7ff164009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:57.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff17a5aa700 1 -- 172.21.15.89:0/2211552406 shutdown_connections 2023-09-30T22:18:57.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff17a5aa700 1 --2- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1741338c0 0x7ff174133c90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff17a5aa700 1 --2- 172.21.15.89:0/2211552406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff17409e560 0x7ff17413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff17a5aa700 1 -- 172.21.15.89:0/2211552406 >> 172.21.15.89:0/2211552406 conn(0x7ff1740a2aa0 msgr2=0x7ff1740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff17a5aa700 1 -- 172.21.15.89:0/2211552406 shutdown_connections 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.510+0000 7ff17a5aa700 1 -- 172.21.15.89:0/2211552406 wait complete. 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.511+0000 7ff17a5aa700 1 Processor -- start 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff17a5aa700 1 -- start start 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff17a5aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17409e560 0x7ff17411b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff17a5aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1741338c0 0x7ff17411bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff17a5aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1741341d0 0x7ff17411c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff17a5aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17411d520 con 0x7ff17409e560 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff17a5aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17406f960 con 0x7ff1741338c0 2023-09-30T22:18:57.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff17a5aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff17411e660 con 0x7ff1741341d0 2023-09-30T22:18:57.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff178b47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1741341d0 0x7ff17411c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff178b47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1741341d0 0x7ff17411c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38306/0 (socket says 172.21.15.89:38306) 2023-09-30T22:18:57.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff178b47700 1 -- 172.21.15.89:0/765619354 learned_addr learned my addr 172.21.15.89:0/765619354 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:57.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.512+0000 7ff173fff700 1 --2- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17409e560 0x7ff17411b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff178b47700 1 -- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1741338c0 msgr2=0x7ff17411bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff178b47700 1 --2- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1741338c0 0x7ff17411bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff178b47700 1 -- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17409e560 msgr2=0x7ff17411b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff178b47700 1 --2- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17409e560 0x7ff17411b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff178b47700 1 -- 172.21.15.89:0/765619354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c009a70 con 0x7ff1741341d0 2023-09-30T22:18:57.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff178b47700 1 --2- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1741341d0 0x7ff17411c310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00efc0 tx=0x7ff16c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:57.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff170ff9700 1 -- 172.21.15.89:0/765619354 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00cb40 con 0x7ff1741341d0 2023-09-30T22:18:57.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff170ff9700 1 -- 172.21.15.89:0/765619354 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c00f040 con 0x7ff1741341d0 2023-09-30T22:18:57.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff170ff9700 1 -- 172.21.15.89:0/765619354 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c003a20 con 0x7ff1741341d0 2023-09-30T22:18:57.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff164009550 con 0x7ff1741341d0 2023-09-30T22:18:57.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.513+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17411f320 con 0x7ff1741341d0 2023-09-30T22:18:57.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.514+0000 7ff170ff9700 1 -- 172.21.15.89:0/765619354 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff16c014480 con 0x7ff1741341d0 2023-09-30T22:18:57.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.514+0000 7ff170ff9700 1 -- 172.21.15.89:0/765619354 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff16c040100 con 0x7ff1741341d0 2023-09-30T22:18:57.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.515+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17409d1f0 con 0x7ff1741341d0 2023-09-30T22:18:57.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.523+0000 7ff170ff9700 1 -- 172.21.15.89:0/765619354 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff16c012930 con 0x7ff1741341d0 2023-09-30T22:18:57.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f75764d5700 1 Processor -- start 2023-09-30T22:18:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f75764d5700 1 -- start start 2023-09-30T22:18:57.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f75764d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757006e570 0x7f757006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f75764d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757009eca0 0x7f757006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f75764d5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7570134610 0x7f7570134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f75764d5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7570070ee0 con 0x7f7570134610 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f75764d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7570071480 con 0x7f757006e570 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f75764d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f757006f3c0 con 0x7f757009eca0 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f756ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757006e570 0x7f757006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f756ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757006e570 0x7f757006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45594/0 (socket says 172.21.15.89:45594) 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f756ffff700 1 -- 172.21.15.89:0/1830914137 learned_addr learned my addr 172.21.15.89:0/1830914137 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.540+0000 7f7574a72700 1 --2- 172.21.15.89:0/1830914137 >> v2:172.21.15.89:3300/0 conn(0x7f7570134610 0x7f7570134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756ffff700 1 -- 172.21.15.89:0/1830914137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757009eca0 msgr2=0x7f757006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:57.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756ffff700 1 --2- 172.21.15.89:0/1830914137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757009eca0 0x7f757006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756ffff700 1 -- 172.21.15.89:0/1830914137 >> v2:172.21.15.89:3300/0 conn(0x7f7570134610 msgr2=0x7f7570134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:57.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756ffff700 1 --2- 172.21.15.89:0/1830914137 >> v2:172.21.15.89:3300/0 conn(0x7f7570134610 0x7f7570134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756ffff700 1 -- 172.21.15.89:0/1830914137 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570071b90 con 0x7f757006e570 2023-09-30T22:18:57.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756ffff700 1 --2- 172.21.15.89:0/1830914137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757006e570 0x7f757006e940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f75600098f0 tx=0x7f7560009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca32f5828670ca5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:57.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756effd700 1 -- 172.21.15.89:0/1830914137 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560011830 con 0x7f757006e570 2023-09-30T22:18:57.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756effd700 1 -- 172.21.15.89:0/1830914137 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7560011990 con 0x7f757006e570 2023-09-30T22:18:57.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.541+0000 7f756effd700 1 -- 172.21.15.89:0/1830914137 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560011d20 con 0x7f757006e570 2023-09-30T22:18:57.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 -- 172.21.15.89:0/1830914137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757006e570 msgr2=0x7f757006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 --2- 172.21.15.89:0/1830914137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757006e570 0x7f757006e940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f75600098f0 tx=0x7f7560009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:57.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 -- 172.21.15.89:0/1830914137 shutdown_connections 2023-09-30T22:18:57.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 --2- 172.21.15.89:0/1830914137 >> v2:172.21.15.89:3300/0 conn(0x7f7570134610 0x7f7570134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 --2- 172.21.15.89:0/1830914137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757006e570 0x7f757006e940 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 --2- 172.21.15.89:0/1830914137 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757009eca0 0x7f757006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 -- 172.21.15.89:0/1830914137 >> 172.21.15.89:0/1830914137 conn(0x7f75700a2aa0 msgr2=0x7f75700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:57.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 -- 172.21.15.89:0/1830914137 shutdown_connections 2023-09-30T22:18:57.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.542+0000 7f75764d5700 1 -- 172.21.15.89:0/1830914137 wait complete. 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f75764d5700 1 Processor -- start 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f75764d5700 1 -- start start 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f75764d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757006e570 0x7f7570127400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f75764d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7570134610 0x7f7570125fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f75764d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7570127940 0x7f75701264f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f75764d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570070610 con 0x7f757006e570 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f75764d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757013d0d0 con 0x7f7570127940 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f75764d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f757013de00 con 0x7f7570134610 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f7574a72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7570127940 0x7f75701264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f7574a72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7570127940 0x7f75701264f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45598/0 (socket says 172.21.15.89:45598) 2023-09-30T22:18:57.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f7574a72700 1 -- 172.21.15.89:0/4006547958 learned_addr learned my addr 172.21.15.89:0/4006547958 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.543+0000 7f756ffff700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757006e570 0x7f7570127400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f756f7fe700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7570134610 0x7f7570125fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f7574a72700 1 -- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7570134610 msgr2=0x7f7570125fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f7574a72700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7570134610 0x7f7570125fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f7574a72700 1 -- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757006e570 msgr2=0x7f7570127400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f7574a72700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757006e570 0x7f7570127400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f7574a72700 1 -- 172.21.15.89:0/4006547958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560009550 con 0x7f7570127940 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f7574a72700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7570127940 0x7f75701264f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f756800efc0 tx=0x7f756800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f756cff9700 1 -- 172.21.15.89:0/4006547958 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568009ea0 con 0x7f7570127940 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f756cff9700 1 -- 172.21.15.89:0/4006547958 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800f040 con 0x7f7570127940 2023-09-30T22:18:57.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757013d670 con 0x7f7570127940 2023-09-30T22:18:57.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f756cff9700 1 -- 172.21.15.89:0/4006547958 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800ceb0 con 0x7f7570127940 2023-09-30T22:18:57.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.544+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757013b940 con 0x7f7570127940 2023-09-30T22:18:57.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.545+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757013d4d0 con 0x7f7570127940 2023-09-30T22:18:57.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.546+0000 7f756cff9700 1 -- 172.21.15.89:0/4006547958 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7568014480 con 0x7f7570127940 2023-09-30T22:18:57.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.546+0000 7f756cff9700 1 -- 172.21.15.89:0/4006547958 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f756803fd30 con 0x7f7570127940 2023-09-30T22:18:57.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.554+0000 7f756cff9700 1 -- 172.21.15.89:0/4006547958 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f757013d4d0 con 0x7f7570127940 2023-09-30T22:18:57.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.588+0000 7f744b7d6700 1 Processor -- start 2023-09-30T22:18:57.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.588+0000 7f744b7d6700 1 -- start start 2023-09-30T22:18:57.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f744b7d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444134630 0x7f7444136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f744b7d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444136f50 0x7f7444139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f744b7d6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7444071a00 con 0x7f7444131630 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f744b7d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7444131430 con 0x7f7444136f50 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f744b7d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f744409a510 con 0x7f7444134630 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f7449572700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7444131630 0x7f7444131a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41088/0 (socket says 172.21.15.89:41088) 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f7449572700 1 -- 172.21.15.89:0/18345851 learned_addr learned my addr 172.21.15.89:0/18345851 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f7448d71700 1 --2- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444134630 0x7f7444136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f7449d73700 1 --2- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444136f50 0x7f7444139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f743bfff700 1 -- 172.21.15.89:0/18345851 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863749430 0 0) 0x7f7444071a00 con 0x7f7444131630 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f743bfff700 1 -- 172.21.15.89:0/18345851 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74300035b0 con 0x7f7444131630 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f7449d73700 1 -- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444134630 msgr2=0x7f7444136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f7449d73700 1 --2- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444134630 0x7f7444136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f7449d73700 1 -- 172.21.15.89:0/18345851 >> v1:172.21.15.89:6789/0 conn(0x7f7444131630 legacy=0x7f7444131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.589+0000 7f7449d73700 1 -- 172.21.15.89:0/18345851 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74440721c0 con 0x7f7444136f50 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.590+0000 7f7449d73700 1 --2- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444136f50 0x7f7444139330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f743c009dc0 tx=0x7f743c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3eff692ce8d4541 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.590+0000 7f743bfff700 1 -- 172.21.15.89:0/18345851 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c00e070 con 0x7f7444136f50 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.590+0000 7f743bfff700 1 -- 172.21.15.89:0/18345851 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c00bb20 con 0x7f7444136f50 2023-09-30T22:18:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.590+0000 7f743bfff700 1 -- 172.21.15.89:0/18345851 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c012430 con 0x7f7444136f50 2023-09-30T22:18:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.590+0000 7f744b7d6700 1 -- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444136f50 msgr2=0x7f7444139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.590+0000 7f744b7d6700 1 --2- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444136f50 0x7f7444139330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f743c009dc0 tx=0x7f743c00b330 comp rx=0 tx=0).stop 2023-09-30T22:18:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.591+0000 7f744b7d6700 1 -- 172.21.15.89:0/18345851 shutdown_connections 2023-09-30T22:18:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.591+0000 7f744b7d6700 1 --2- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444136f50 0x7f7444139330 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.591+0000 7f744b7d6700 1 --2- 172.21.15.89:0/18345851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444134630 0x7f7444136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.591+0000 7f744b7d6700 1 -- 172.21.15.89:0/18345851 >> 172.21.15.89:0/18345851 conn(0x7f74440a2aa0 msgr2=0x7f74440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.591+0000 7f744b7d6700 1 -- 172.21.15.89:0/18345851 shutdown_connections 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.591+0000 7f744b7d6700 1 -- 172.21.15.89:0/18345851 wait complete. 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.591+0000 7f744b7d6700 1 Processor -- start 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f744b7d6700 1 -- start start 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f744b7d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444131630 0x7f744406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f744b7d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444134630 0x7f744406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f744b7d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7444136f50 0x7f7444070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f744b7d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744411e220 con 0x7f7444136f50 2023-09-30T22:18:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f744b7d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7444136d20 con 0x7f7444134630 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f744b7d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7444071010 con 0x7f7444131630 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449572700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444131630 0x7f744406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449572700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444131630 0x7f744406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38336/0 (socket says 172.21.15.89:38336) 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449572700 1 -- 172.21.15.89:0/2769632880 learned_addr learned my addr 172.21.15.89:0/2769632880 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449d73700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7444136f50 0x7f7444070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7448d71700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444134630 0x7f744406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449572700 1 -- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444134630 msgr2=0x7f744406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449572700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444134630 0x7f744406fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449572700 1 -- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7444136f50 msgr2=0x7f7444070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449572700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7444136f50 0x7f7444070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.592+0000 7f7449572700 1 -- 172.21.15.89:0/2769632880 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c009a70 con 0x7f7444131630 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.593+0000 7f7449572700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444131630 0x7f744406f5e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f744000c700 tx=0x7f744000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.593+0000 7f7439ffb700 1 -- 172.21.15.89:0/2769632880 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440005d70 con 0x7f7444131630 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.593+0000 7f7439ffb700 1 -- 172.21.15.89:0/2769632880 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74400070d0 con 0x7f7444131630 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.593+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744411e470 con 0x7f7444131630 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.593+0000 7f7439ffb700 1 -- 172.21.15.89:0/2769632880 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440007460 con 0x7f7444131630 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.593+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744411eee0 con 0x7f7444131630 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.594+0000 7f7439ffb700 1 -- 172.21.15.89:0/2769632880 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74400075c0 con 0x7f7444131630 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.594+0000 7f7439ffb700 1 -- 172.21.15.89:0/2769632880 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f744003ebb0 con 0x7f7444131630 2023-09-30T22:18:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.594+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7444074ea0 con 0x7f7444131630 2023-09-30T22:18:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.599+0000 7f7439ffb700 1 -- 172.21.15.89:0/2769632880 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7440017020 con 0x7f7444131630 2023-09-30T22:18:57.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.672+0000 7f6acb270700 1 Processor -- start 2023-09-30T22:18:57.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6acb270700 1 -- start start 2023-09-30T22:18:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6acb270700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6acb270700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ac409f160 0x7f6ac4133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6acb270700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac406bf30 con 0x7f6ac409f5b0 2023-09-30T22:18:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6acb270700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac409ffb0 con 0x7f6ac409f160 2023-09-30T22:18:57.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6acb270700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac406cb10 con 0x7f6ac409e850 2023-09-30T22:18:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6ac900c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6ac900c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45620/0 (socket says 172.21.15.89:45620) 2023-09-30T22:18:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.673+0000 7f6ac900c700 1 -- 172.21.15.89:0/934186463 learned_addr learned my addr 172.21.15.89:0/934186463 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6ac880b700 1 --2- 172.21.15.89:0/934186463 >> v2:172.21.15.89:3300/0 conn(0x7f6ac409f160 0x7f6ac4133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6ac880b700 1 -- 172.21.15.89:0/934186463 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 msgr2=0x7f6ac409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6ac880b700 1 --2- 172.21.15.89:0/934186463 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6ac880b700 1 -- 172.21.15.89:0/934186463 >> v1:172.21.15.89:6789/0 conn(0x7f6ac409f5b0 legacy=0x7f6ac4134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6ac880b700 1 -- 172.21.15.89:0/934186463 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40a0270 con 0x7f6ac409f160 2023-09-30T22:18:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6ac880b700 1 --2- 172.21.15.89:0/934186463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f160 0x7f6ac4133bf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009870 tx=0x7f6ab4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928f390aa7993e69 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6abbfff700 1 -- 172.21.15.89:0/934186463 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4011830 con 0x7f6ac409f160 2023-09-30T22:18:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6abbfff700 1 -- 172.21.15.89:0/934186463 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab4011990 con 0x7f6ac409f160 2023-09-30T22:18:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.674+0000 7f6abbfff700 1 -- 172.21.15.89:0/934186463 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4011d20 con 0x7f6ac409f160 2023-09-30T22:18:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.675+0000 7f6acb270700 1 -- 172.21.15.89:0/934186463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f160 msgr2=0x7f6ac4133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.675+0000 7f6acb270700 1 --2- 172.21.15.89:0/934186463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f160 0x7f6ac4133bf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009870 tx=0x7f6ab4009b80 comp rx=0 tx=0).stop 2023-09-30T22:18:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.675+0000 7f6acb270700 1 -- 172.21.15.89:0/934186463 shutdown_connections 2023-09-30T22:18:57.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.675+0000 7f6acb270700 1 --2- 172.21.15.89:0/934186463 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.675+0000 7f6acb270700 1 --2- 172.21.15.89:0/934186463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f160 0x7f6ac4133bf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.675+0000 7f6acb270700 1 -- 172.21.15.89:0/934186463 >> 172.21.15.89:0/934186463 conn(0x7f6ac40a3f70 msgr2=0x7f6ac40a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:57.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.675+0000 7f6acb270700 1 -- 172.21.15.89:0/934186463 shutdown_connections 2023-09-30T22:18:57.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.675+0000 7f6acb270700 1 -- 172.21.15.89:0/934186463 wait complete. 2023-09-30T22:18:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.676+0000 7f6acb270700 1 Processor -- start 2023-09-30T22:18:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.676+0000 7f6acb270700 1 -- start start 2023-09-30T22:18:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6acb270700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac4122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6acb270700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ac409f160 0x7f6ac4122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6acb270700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f5b0 0x7f6ac4123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:57.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6acb270700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac406f9e0 con 0x7f6ac409f5b0 2023-09-30T22:18:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6acb270700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4133740 con 0x7f6ac409e850 2023-09-30T22:18:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6acb270700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41244a0 con 0x7f6ac409f160 2023-09-30T22:18:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac4122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac4122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45622/0 (socket says 172.21.15.89:45622) 2023-09-30T22:18:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 -- 172.21.15.89:0/3408758523 learned_addr learned my addr 172.21.15.89:0/3408758523 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac880b700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ac409f160 0x7f6ac4122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac980d700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f5b0 0x7f6ac4123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:57.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 -- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ac409f160 msgr2=0x7f6ac4122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ac409f160 0x7f6ac4122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 -- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f5b0 msgr2=0x7f6ac4123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f5b0 0x7f6ac4123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 -- 172.21.15.89:0/3408758523 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab40094a0 con 0x7f6ac409e850 2023-09-30T22:18:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac980d700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f5b0 0x7f6ac4123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:18:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.677+0000 7f6ac900c700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac4122780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e8b0 tx=0x7f6ac000ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.678+0000 7f6ab9ffb700 1 -- 172.21.15.89:0/3408758523 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0009600 con 0x7f6ac409e850 2023-09-30T22:18:57.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.678+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac406cd90 con 0x7f6ac409e850 2023-09-30T22:18:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.678+0000 7f6ab9ffb700 1 -- 172.21.15.89:0/3408758523 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac000eeb0 con 0x7f6ac409e850 2023-09-30T22:18:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.678+0000 7f6ab9ffb700 1 -- 172.21.15.89:0/3408758523 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000ca70 con 0x7f6ac409e850 2023-09-30T22:18:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.678+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac4126a30 con 0x7f6ac409e850 2023-09-30T22:18:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.679+0000 7f6ab9ffb700 1 -- 172.21.15.89:0/3408758523 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ac000cbd0 con 0x7f6ac409e850 2023-09-30T22:18:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.679+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac4125c60 con 0x7f6ac409e850 2023-09-30T22:18:57.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.679+0000 7f6ab9ffb700 1 -- 172.21.15.89:0/3408758523 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6ac003f6f0 con 0x7f6ac409e850 2023-09-30T22:18:57.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.688+0000 7f6ab9ffb700 1 -- 172.21.15.89:0/3408758523 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ac00232f0 con 0x7f6ac409e850 2023-09-30T22:18:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.750+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f757013b940 con 0x7f7570127940 2023-09-30T22:18:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.750+0000 7f756cff9700 1 -- 172.21.15.89:0/4006547958 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f757013b940 con 0x7f7570127940 2023-09-30T22:18:57.752 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.752+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7570127940 msgr2=0x7f75701264f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.752+0000 7f75764d5700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7570127940 0x7f75701264f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f756800efc0 tx=0x7f756800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.752+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 shutdown_connections 2023-09-30T22:18:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.752+0000 7f75764d5700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7570127940 0x7f75701264f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.752+0000 7f75764d5700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757006e570 0x7f7570127400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.752+0000 7f75764d5700 1 --2- 172.21.15.89:0/4006547958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7570134610 0x7f7570125fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.752+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 >> 172.21.15.89:0/4006547958 conn(0x7f75700a2aa0 msgr2=0x7f7570099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:57.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.753+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 shutdown_connections 2023-09-30T22:18:57.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.753+0000 7f75764d5700 1 -- 172.21.15.89:0/4006547958 wait complete. 2023-09-30T22:18:57.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:57.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.800+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f744406b9e0 con 0x7f7444131630 2023-09-30T22:18:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.801+0000 7f7439ffb700 1 -- 172.21.15.89:0/2769632880 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7440018260 con 0x7f7444131630 2023-09-30T22:18:57.802 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:57.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444131630 msgr2=0x7f744406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444131630 0x7f744406f5e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f744000c700 tx=0x7f744000ca10 comp rx=0 tx=0).stop 2023-09-30T22:18:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 shutdown_connections 2023-09-30T22:18:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7444136f50 0x7f7444070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7444131630 0x7f744406f5e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 --2- 172.21.15.89:0/2769632880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7444134630 0x7f744406fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 >> 172.21.15.89:0/2769632880 conn(0x7f74440a2aa0 msgr2=0x7f74440a0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 shutdown_connections 2023-09-30T22:18:57.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.803+0000 7f744b7d6700 1 -- 172.21.15.89:0/2769632880 wait complete. 2023-09-30T22:18:57.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:57.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.845+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff174074ea0 con 0x7ff1741341d0 2023-09-30T22:18:57.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.845+0000 7ff170ff9700 1 -- 172.21.15.89:0/765619354 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff16c02b090 con 0x7ff1741341d0 2023-09-30T22:18:57.846 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:57.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.847+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1741341d0 msgr2=0x7ff17411c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.847+0000 7ff17a5aa700 1 --2- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1741341d0 0x7ff17411c310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00efc0 tx=0x7ff16c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.847+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 shutdown_connections 2023-09-30T22:18:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.847+0000 7ff17a5aa700 1 --2- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1741341d0 0x7ff17411c310 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.847+0000 7ff17a5aa700 1 --2- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff17409e560 0x7ff17411b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.847+0000 7ff17a5aa700 1 --2- 172.21.15.89:0/765619354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1741338c0 0x7ff17411bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.847+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 >> 172.21.15.89:0/765619354 conn(0x7ff1740a2aa0 msgr2=0x7ff174131eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.848+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 shutdown_connections 2023-09-30T22:18:57.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.848+0000 7ff17a5aa700 1 -- 172.21.15.89:0/765619354 wait complete. 2023-09-30T22:18:57.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:57.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.934+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6ac4125e70 con 0x7f6ac409e850 2023-09-30T22:18:57.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.935+0000 7f6ab9ffb700 1 -- 172.21.15.89:0/3408758523 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ac001bb60 con 0x7f6ac409e850 2023-09-30T22:18:57.936 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:57.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 msgr2=0x7f6ac4122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac4122780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e8b0 tx=0x7f6ac000ec70 comp rx=0 tx=0).stop 2023-09-30T22:18:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 shutdown_connections 2023-09-30T22:18:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac409f5b0 0x7f6ac4123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ac409e850 0x7f6ac4122780 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 --2- 172.21.15.89:0/3408758523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ac409f160 0x7f6ac4122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 >> 172.21.15.89:0/3408758523 conn(0x7f6ac40a3f70 msgr2=0x7f6ac40a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 shutdown_connections 2023-09-30T22:18:57.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:57.937+0000 7f6acb270700 1 -- 172.21.15.89:0/3408758523 wait complete. 2023-09-30T22:18:57.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:18:58.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:18:58.816 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:18:58.861 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:18:58.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fdea19a3700 1 Processor -- start 2023-09-30T22:18:58.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fdea19a3700 1 -- start start 2023-09-30T22:18:58.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fdea19a3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09e1d0 0x7fde9c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fdea19a3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c135e50 0x7fde9c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fdea19a3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c09f130 con 0x7fde9c09d6f0 2023-09-30T22:18:58.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fdea19a3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c09f520 con 0x7fde9c09e1d0 2023-09-30T22:18:58.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fdea19a3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c09e930 con 0x7fde9c135e50 2023-09-30T22:18:58.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fde9a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09e1d0 0x7fde9c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fde9b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c135e50 0x7fde9c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fde9a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09e1d0 0x7fde9c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45632/0 (socket says 172.21.15.89:45632) 2023-09-30T22:18:58.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fde9a7fc700 1 -- 172.21.15.89:0/302924032 learned_addr learned my addr 172.21.15.89:0/302924032 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:58.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fde9b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c135e50 0x7fde9c138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38356/0 (socket says 172.21.15.89:38356) 2023-09-30T22:18:58.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.902+0000 7fde9affd700 1 --1- 172.21.15.89:0/302924032 >> v1:172.21.15.89:6789/0 conn(0x7fde9c09d6f0 0x7fde9c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41114/0 (socket says 172.21.15.89:41114) 2023-09-30T22:18:58.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde99ffb700 1 -- 172.21.15.89:0/302924032 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515893736 0 0) 0x7fde9c09f130 con 0x7fde9c09d6f0 2023-09-30T22:18:58.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde9b7fe700 1 -- 172.21.15.89:0/302924032 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09e1d0 msgr2=0x7fde9c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:58.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde9b7fe700 1 --2- 172.21.15.89:0/302924032 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09e1d0 0x7fde9c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde9b7fe700 1 -- 172.21.15.89:0/302924032 >> v1:172.21.15.89:6789/0 conn(0x7fde9c09d6f0 legacy=0x7fde9c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde9b7fe700 1 -- 172.21.15.89:0/302924032 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c09b2d0 con 0x7fde9c135e50 2023-09-30T22:18:58.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde9a7fc700 1 --2- 172.21.15.89:0/302924032 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09e1d0 0x7fde9c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:58.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde9b7fe700 1 --2- 172.21.15.89:0/302924032 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c135e50 0x7fde9c138230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fde94009dc0 tx=0x7fde9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9d8110de946783 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:58.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde99ffb700 1 -- 172.21.15.89:0/302924032 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400e070 con 0x7fde9c135e50 2023-09-30T22:18:58.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde99ffb700 1 -- 172.21.15.89:0/302924032 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400bb20 con 0x7fde9c135e50 2023-09-30T22:18:58.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.903+0000 7fde99ffb700 1 -- 172.21.15.89:0/302924032 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94012430 con 0x7fde9c135e50 2023-09-30T22:18:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.904+0000 7fdea19a3700 1 -- 172.21.15.89:0/302924032 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c135e50 msgr2=0x7fde9c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.904+0000 7fdea19a3700 1 --2- 172.21.15.89:0/302924032 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c135e50 0x7fde9c138230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fde94009dc0 tx=0x7fde9400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.904+0000 7fdea19a3700 1 -- 172.21.15.89:0/302924032 shutdown_connections 2023-09-30T22:18:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.904+0000 7fdea19a3700 1 --2- 172.21.15.89:0/302924032 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c135e50 0x7fde9c138230 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.904+0000 7fdea19a3700 1 --2- 172.21.15.89:0/302924032 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09e1d0 0x7fde9c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.904+0000 7fdea19a3700 1 -- 172.21.15.89:0/302924032 >> 172.21.15.89:0/302924032 conn(0x7fde9c066f50 msgr2=0x7fde9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.904+0000 7fdea19a3700 1 -- 172.21.15.89:0/302924032 shutdown_connections 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.904+0000 7fdea19a3700 1 -- 172.21.15.89:0/302924032 wait complete. 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.905+0000 7fdea19a3700 1 Processor -- start 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.905+0000 7fdea19a3700 1 -- start start 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fdea19a3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09d6f0 0x7fde9c1397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fdea19a3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c09e1d0 0x7fde9c139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fdea19a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde9c135e50 0x7fde9c13a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fdea19a3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c121130 con 0x7fde9c135e50 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fdea19a3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c13b7a0 con 0x7fde9c09d6f0 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fdea19a3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c09c700 con 0x7fde9c09e1d0 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fde9affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09d6f0 0x7fde9c1397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fde9affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09d6f0 0x7fde9c1397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45646/0 (socket says 172.21.15.89:45646) 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fde9affd700 1 -- 172.21.15.89:0/2610597577 learned_addr learned my addr 172.21.15.89:0/2610597577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fde9a7fc700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c09e1d0 0x7fde9c139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.906+0000 7fde9b7fe700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde9c135e50 0x7fde9c13a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde9affd700 1 -- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c09e1d0 msgr2=0x7fde9c139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde9affd700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c09e1d0 0x7fde9c139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde9affd700 1 -- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde9c135e50 msgr2=0x7fde9c13a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde9affd700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde9c135e50 0x7fde9c13a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde9affd700 1 -- 172.21.15.89:0/2610597577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90009550 con 0x7fde9c09d6f0 2023-09-30T22:18:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde9b7fe700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde9c135e50 0x7fde9c13a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde9affd700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09d6f0 0x7fde9c1397b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c520 tx=0x7fde8c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde83fff700 1 -- 172.21.15.89:0/2610597577 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c004c80 con 0x7fde9c09d6f0 2023-09-30T22:18:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde83fff700 1 -- 172.21.15.89:0/2610597577 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c004e00 con 0x7fde9c09d6f0 2023-09-30T22:18:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde94009a70 con 0x7fde9c09d6f0 2023-09-30T22:18:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fde83fff700 1 -- 172.21.15.89:0/2610597577 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c007310 con 0x7fde9c09d6f0 2023-09-30T22:18:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.907+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c0a3c40 con 0x7fde9c09d6f0 2023-09-30T22:18:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.908+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c071700 con 0x7fde9c09d6f0 2023-09-30T22:18:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.908+0000 7fde83fff700 1 -- 172.21.15.89:0/2610597577 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde8c007470 con 0x7fde9c09d6f0 2023-09-30T22:18:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.908+0000 7fde83fff700 1 -- 172.21.15.89:0/2610597577 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fde8c03f290 con 0x7fde9c09d6f0 2023-09-30T22:18:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.913+0000 7fde83fff700 1 -- 172.21.15.89:0/2610597577 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde8c0133c0 con 0x7fde9c09d6f0 2023-09-30T22:18:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.946+0000 7fba22016700 1 Processor -- start 2023-09-30T22:18:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.946+0000 7fba22016700 1 -- start start 2023-09-30T22:18:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba22016700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09f410 0x7fba1c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba22016700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fba1c136fe0 0x7fba1c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba22016700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c070940 con 0x7fba1c09e930 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba22016700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c06fd60 con 0x7fba1c136fe0 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba22016700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c06fa80 con 0x7fba1c09f410 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fba1c136fe0 0x7fba1c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fba1c136fe0 0x7fba1c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47456/0 (socket says 172.21.15.89:47456) 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1bfff700 1 -- 172.21.15.89:0/1686442688 learned_addr learned my addr 172.21.15.89:0/1686442688 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09f410 0x7fba1c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1bfff700 1 -- 172.21.15.89:0/1686442688 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09f410 msgr2=0x7fba1c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1bfff700 1 --2- 172.21.15.89:0/1686442688 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09f410 0x7fba1c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1bfff700 1 -- 172.21.15.89:0/1686442688 >> v1:172.21.15.89:6789/0 conn(0x7fba1c09e930 legacy=0x7fba1c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1bfff700 1 -- 172.21.15.89:0/1686442688 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c09fb60 con 0x7fba1c136fe0 2023-09-30T22:18:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1affd700 1 --2- 172.21.15.89:0/1686442688 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09f410 0x7fba1c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1bfff700 1 --2- 172.21.15.89:0/1686442688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c136fe0 0x7fba1c1393c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fba14009e50 tx=0x7fba1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d30de7411f65b2f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1a7fc700 1 -- 172.21.15.89:0/1686442688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400e070 con 0x7fba1c136fe0 2023-09-30T22:18:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1a7fc700 1 -- 172.21.15.89:0/1686442688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1400b910 con 0x7fba1c136fe0 2023-09-30T22:18:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.947+0000 7fba1a7fc700 1 -- 172.21.15.89:0/1686442688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400bcc0 con 0x7fba1c136fe0 2023-09-30T22:18:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.948+0000 7fba22016700 1 -- 172.21.15.89:0/1686442688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c136fe0 msgr2=0x7fba1c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.948+0000 7fba22016700 1 --2- 172.21.15.89:0/1686442688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c136fe0 0x7fba1c1393c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fba14009e50 tx=0x7fba1400b330 comp rx=0 tx=0).stop 2023-09-30T22:18:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.948+0000 7fba22016700 1 -- 172.21.15.89:0/1686442688 shutdown_connections 2023-09-30T22:18:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.948+0000 7fba22016700 1 --2- 172.21.15.89:0/1686442688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c136fe0 0x7fba1c1393c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.948+0000 7fba22016700 1 --2- 172.21.15.89:0/1686442688 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09f410 0x7fba1c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.948+0000 7fba22016700 1 -- 172.21.15.89:0/1686442688 >> 172.21.15.89:0/1686442688 conn(0x7fba1c0a4210 msgr2=0x7fba1c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.948+0000 7fba22016700 1 -- 172.21.15.89:0/1686442688 shutdown_connections 2023-09-30T22:18:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.948+0000 7fba22016700 1 -- 172.21.15.89:0/1686442688 wait complete. 2023-09-30T22:18:58.952 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:18:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.949+0000 7fba22016700 1 Processor -- start 2023-09-30T22:18:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.950+0000 7fba22016700 1 -- start start 2023-09-30T22:18:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.950+0000 7fba22016700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09e930 0x7fba1c11e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.950+0000 7fba22016700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c09f410 0x7fba1c11eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.950+0000 7fba22016700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba1c136fe0 0x7fba1c11f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.950+0000 7fba22016700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c133ef0 con 0x7fba1c09f410 2023-09-30T22:18:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.950+0000 7fba22016700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c1201d0 con 0x7fba1c09e930 2023-09-30T22:18:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.950+0000 7fba22016700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c1344b0 con 0x7fba1c136fe0 2023-09-30T22:18:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c09f410 0x7fba1c11eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c09f410 0x7fba1c11eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47470/0 (socket says 172.21.15.89:47470) 2023-09-30T22:18:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 -- 172.21.15.89:0/3180362086 learned_addr learned my addr 172.21.15.89:0/3180362086 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1b7fe700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09e930 0x7fba1c11e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 -- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba1c136fe0 msgr2=0x7fba1c11f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba1c136fe0 0x7fba1c11f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 -- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09e930 msgr2=0x7fba1c11e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09e930 0x7fba1c11e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 -- 172.21.15.89:0/3180362086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba100094a0 con 0x7fba1c09f410 2023-09-30T22:18:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1b7fe700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09e930 0x7fba1c11e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba1affd700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c09f410 0x7fba1c11eb00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fba1000ec10 tx=0x7fba1000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba03fff700 1 -- 172.21.15.89:0/3180362086 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1000c910 con 0x7fba1c09f410 2023-09-30T22:18:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba03fff700 1 -- 172.21.15.89:0/3180362086 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1000ca70 con 0x7fba1c09f410 2023-09-30T22:18:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba14009b30 con 0x7fba1c09f410 2023-09-30T22:18:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba03fff700 1 -- 172.21.15.89:0/3180362086 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1000cde0 con 0x7fba1c09f410 2023-09-30T22:18:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.951+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c120a30 con 0x7fba1c09f410 2023-09-30T22:18:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.952+0000 7fba03fff700 1 -- 172.21.15.89:0/3180362086 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba10013470 con 0x7fba1c09f410 2023-09-30T22:18:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.952+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c0a2f40 con 0x7fba1c09f410 2023-09-30T22:18:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.952+0000 7fba03fff700 1 -- 172.21.15.89:0/3180362086 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fba1003f860 con 0x7fba1c09f410 2023-09-30T22:18:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.957+0000 7fba03fff700 1 -- 172.21.15.89:0/3180362086 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba1001bcf0 con 0x7fba1c09f410 2023-09-30T22:18:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.992+0000 7fc6b36c8700 1 Processor -- start 2023-09-30T22:18:58.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.992+0000 7fc6b36c8700 1 -- start start 2023-09-30T22:18:58.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.992+0000 7fc6b36c8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6ac12cf60 0x7fc6ac06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.992+0000 7fc6b36c8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:58.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.992+0000 7fc6b36c8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ac12cd90 con 0x7fc6ac12f8b0 2023-09-30T22:18:58.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.992+0000 7fc6b36c8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ac12d330 con 0x7fc6ac12cf60 2023-09-30T22:18:58.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.992+0000 7fc6b36c8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac071780 con 0x7fc6ac130430 2023-09-30T22:18:58.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1464700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6ac12cf60 0x7fc6ac06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1c65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1464700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6ac12cf60 0x7fc6ac06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47476/0 (socket says 172.21.15.89:47476) 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1464700 1 -- 172.21.15.89:0/29731145 learned_addr learned my addr 172.21.15.89:0/29731145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1464700 1 -- 172.21.15.89:0/29731145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 msgr2=0x7fc6ac137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1464700 1 --2- 172.21.15.89:0/29731145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1464700 1 -- 172.21.15.89:0/29731145 >> v1:172.21.15.89:6789/0 conn(0x7fc6ac12f8b0 legacy=0x7fc6ac12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1464700 1 -- 172.21.15.89:0/29731145 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac071a40 con 0x7fc6ac12cf60 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6b1464700 1 --2- 172.21.15.89:0/29731145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12cf60 0x7fc6ac06f5d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80098f0 tx=0x7fc6a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c379e0eddf724ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6a3fff700 1 -- 172.21.15.89:0/29731145 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800f940 con 0x7fc6ac12cf60 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6a3fff700 1 -- 172.21.15.89:0/29731145 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a800faa0 con 0x7fc6ac12cf60 2023-09-30T22:18:58.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.993+0000 7fc6a3fff700 1 -- 172.21.15.89:0/29731145 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800fe30 con 0x7fc6ac12cf60 2023-09-30T22:18:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.994+0000 7fc6b36c8700 1 -- 172.21.15.89:0/29731145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12cf60 msgr2=0x7fc6ac06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.994+0000 7fc6b36c8700 1 --2- 172.21.15.89:0/29731145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12cf60 0x7fc6ac06f5d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80098f0 tx=0x7fc6a8009c00 comp rx=0 tx=0).stop 2023-09-30T22:18:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.994+0000 7fc6b36c8700 1 -- 172.21.15.89:0/29731145 shutdown_connections 2023-09-30T22:18:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.994+0000 7fc6b36c8700 1 --2- 172.21.15.89:0/29731145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12cf60 0x7fc6ac06f5d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.994+0000 7fc6b36c8700 1 --2- 172.21.15.89:0/29731145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.994+0000 7fc6b36c8700 1 -- 172.21.15.89:0/29731145 >> 172.21.15.89:0/29731145 conn(0x7fc6ac0a2aa0 msgr2=0x7fc6ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.994+0000 7fc6b36c8700 1 -- 172.21.15.89:0/29731145 shutdown_connections 2023-09-30T22:18:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.994+0000 7fc6b36c8700 1 -- 172.21.15.89:0/29731145 wait complete. 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.996+0000 7fc6b36c8700 1 Processor -- start 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.996+0000 7fc6b36c8700 1 -- start start 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.996+0000 7fc6b36c8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6ac12cf60 0x7fc6ac09b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b36c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12f8b0 0x7fc6ac09b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b36c8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac09bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b36c8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac09de40 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b36c8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac09e8b0 con 0x7fc6ac12cf60 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b36c8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac09cf40 con 0x7fc6ac130430 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b1c65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac09bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b0c63700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12f8b0 0x7fc6ac09b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b1c65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac09bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38398/0 (socket says 172.21.15.89:38398) 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b1c65700 1 -- 172.21.15.89:0/1377282053 learned_addr learned my addr 172.21.15.89:0/1377282053 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b0c63700 1 -- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 msgr2=0x7fc6ac09bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b0c63700 1 --2- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac09bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b0c63700 1 -- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6ac12cf60 msgr2=0x7fc6ac09b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b0c63700 1 --2- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6ac12cf60 0x7fc6ac09b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b0c63700 1 -- 172.21.15.89:0/1377282053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8009550 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b1c65700 1 --2- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac09bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b0c63700 1 --2- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12f8b0 0x7fc6ac09b870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000f60 tx=0x7fc69c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1377282053 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c0052d0 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1377282053 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c005430 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.997+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac09c790 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.998+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1377282053 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c0038c0 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.998+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac11d910 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.998+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1377282053 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc69c003a20 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.999+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1377282053 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc69c03ea70 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:58.999+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac067d80 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.008+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1377282053 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc69c018260 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.087+0000 7f58b9aa9700 1 Processor -- start 2023-09-30T22:18:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.087+0000 7f58b9aa9700 1 -- start start 2023-09-30T22:18:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.087+0000 7f58b9aa9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f58b409e1d0 0x7f58b4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.087+0000 7f58b9aa9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4135e50 0x7f58b4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.087+0000 7f58b9aa9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b409f130 con 0x7f58b409d6f0 2023-09-30T22:18:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b9aa9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b409f520 con 0x7f58b409e1d0 2023-09-30T22:18:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b9aa9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b409e930 con 0x7f58b4135e50 2023-09-30T22:18:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4135e50 0x7f58b4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4135e50 0x7f58b4138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45682/0 (socket says 172.21.15.89:45682) 2023-09-30T22:18:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b3fff700 1 -- 172.21.15.89:0/3940004692 learned_addr learned my addr 172.21.15.89:0/3940004692 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b2ffd700 1 --2- 172.21.15.89:0/3940004692 >> v2:172.21.15.89:3300/0 conn(0x7f58b409e1d0 0x7f58b4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b3fff700 1 -- 172.21.15.89:0/3940004692 >> v2:172.21.15.89:3300/0 conn(0x7f58b409e1d0 msgr2=0x7f58b4135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:18:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b3fff700 1 --2- 172.21.15.89:0/3940004692 >> v2:172.21.15.89:3300/0 conn(0x7f58b409e1d0 0x7f58b4135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b3fff700 1 -- 172.21.15.89:0/3940004692 >> v1:172.21.15.89:6789/0 conn(0x7f58b409d6f0 legacy=0x7f58b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b3fff700 1 -- 172.21.15.89:0/3940004692 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b409b2d0 con 0x7f58b4135e50 2023-09-30T22:18:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b3fff700 1 --2- 172.21.15.89:0/3940004692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4135e50 0x7f58b4138230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d150 tx=0x7f58ac00d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b229c5e6e11e96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b27fc700 1 -- 172.21.15.89:0/3940004692 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac010070 con 0x7f58b4135e50 2023-09-30T22:18:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.088+0000 7f58b27fc700 1 -- 172.21.15.89:0/3940004692 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00dd10 con 0x7f58b4135e50 2023-09-30T22:18:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b27fc700 1 -- 172.21.15.89:0/3940004692 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00b310 con 0x7f58b4135e50 2023-09-30T22:18:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3940004692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4135e50 msgr2=0x7f58b4138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b9aa9700 1 --2- 172.21.15.89:0/3940004692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4135e50 0x7f58b4138230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d150 tx=0x7f58ac00d460 comp rx=0 tx=0).stop 2023-09-30T22:18:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3940004692 shutdown_connections 2023-09-30T22:18:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b9aa9700 1 --2- 172.21.15.89:0/3940004692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b4135e50 0x7f58b4138230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b9aa9700 1 --2- 172.21.15.89:0/3940004692 >> v2:172.21.15.89:3300/0 conn(0x7f58b409e1d0 0x7f58b4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3940004692 >> 172.21.15.89:0/3940004692 conn(0x7f58b4066f50 msgr2=0x7f58b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3940004692 shutdown_connections 2023-09-30T22:18:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.089+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3940004692 wait complete. 2023-09-30T22:18:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.090+0000 7f58b9aa9700 1 Processor -- start 2023-09-30T22:18:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.090+0000 7f58b9aa9700 1 -- start start 2023-09-30T22:18:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b9aa9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b409d6f0 0x7f58b41c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b9aa9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b409e1d0 0x7f58b41c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b9aa9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4135e50 0x7f58b4119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:18:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b9aa9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41c4ef0 con 0x7f58b4135e50 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b9aa9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b411ca70 con 0x7f58b409d6f0 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b9aa9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41302a0 con 0x7f58b409e1d0 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b409d6f0 0x7f58b41c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b409d6f0 0x7f58b41c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:45696/0 (socket says 172.21.15.89:45696) 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 -- 172.21.15.89:0/3658796906 learned_addr learned my addr 172.21.15.89:0/3658796906 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b2ffd700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b409e1d0 0x7f58b41c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b3fff700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4135e50 0x7f58b4119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 -- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b409e1d0 msgr2=0x7f58b41c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b409e1d0 0x7f58b41c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 -- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4135e50 msgr2=0x7f58b4119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4135e50 0x7f58b4119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 -- 172.21.15.89:0/3658796906 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac00bd40 con 0x7f58b409d6f0 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.091+0000 7f58b37fe700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b409d6f0 0x7f58b41c4440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58a400c4e0 tx=0x7f58a400c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.092+0000 7f589bfff700 1 -- 172.21.15.89:0/3658796906 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40052d0 con 0x7f58b409d6f0 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.092+0000 7f589bfff700 1 -- 172.21.15.89:0/3658796906 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a4005430 con 0x7f58b409d6f0 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.092+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b411b940 con 0x7f58b409d6f0 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.092+0000 7f589bfff700 1 -- 172.21.15.89:0/3658796906 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40038c0 con 0x7f58b409d6f0 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.092+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b411b170 con 0x7f58b409d6f0 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.093+0000 7f589bfff700 1 -- 172.21.15.89:0/3658796906 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f58a4003a20 con 0x7f58b409d6f0 2023-09-30T22:18:59.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.093+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b4074ea0 con 0x7f58b409d6f0 2023-09-30T22:18:59.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.093+0000 7f589bfff700 1 -- 172.21.15.89:0/3658796906 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f58a403ea70 con 0x7f58b409d6f0 2023-09-30T22:18:59.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.102+0000 7f589bfff700 1 -- 172.21.15.89:0/3658796906 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58a4018260 con 0x7f58b409d6f0 2023-09-30T22:18:59.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.118+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fde9c120a00 con 0x7fde9c09d6f0 2023-09-30T22:18:59.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.118+0000 7fde83fff700 1 -- 172.21.15.89:0/2610597577 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde8c011360 con 0x7fde9c09d6f0 2023-09-30T22:18:59.119 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:18:59.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.121+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09d6f0 msgr2=0x7fde9c1397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.121+0000 7fdea19a3700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09d6f0 0x7fde9c1397b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c520 tx=0x7fde8c00c830 comp rx=0 tx=0).stop 2023-09-30T22:18:59.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.122+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 shutdown_connections 2023-09-30T22:18:59.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.122+0000 7fdea19a3700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde9c135e50 0x7fde9c13a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.122+0000 7fdea19a3700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde9c09d6f0 0x7fde9c1397b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.122+0000 7fdea19a3700 1 --2- 172.21.15.89:0/2610597577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde9c09e1d0 0x7fde9c139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.122+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 >> 172.21.15.89:0/2610597577 conn(0x7fde9c066f50 msgr2=0x7fde9c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:59.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.122+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 shutdown_connections 2023-09-30T22:18:59.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.122+0000 7fdea19a3700 1 -- 172.21.15.89:0/2610597577 wait complete. 2023-09-30T22:18:59.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:18:59.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.166+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fba1c074ea0 con 0x7fba1c09f410 2023-09-30T22:18:59.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.167+0000 7fba03fff700 1 -- 172.21.15.89:0/3180362086 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba100080a0 con 0x7fba1c09f410 2023-09-30T22:18:59.168 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c09f410 msgr2=0x7fba1c11eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c09f410 0x7fba1c11eb00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fba1000ec10 tx=0x7fba1000ef20 comp rx=0 tx=0).stop 2023-09-30T22:18:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 shutdown_connections 2023-09-30T22:18:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba1c136fe0 0x7fba1c11f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba1c09e930 0x7fba1c11e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 --2- 172.21.15.89:0/3180362086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c09f410 0x7fba1c11eb00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 >> 172.21.15.89:0/3180362086 conn(0x7fba1c0a4210 msgr2=0x7fba1c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 shutdown_connections 2023-09-30T22:18:59.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.169+0000 7fba22016700 1 -- 172.21.15.89:0/3180362086 wait complete. 2023-09-30T22:18:59.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:18:59.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.219+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc6ac074ea0 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.220+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1377282053 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc69c017030 con 0x7fc6ac12f8b0 2023-09-30T22:18:59.221 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:18:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12f8b0 msgr2=0x7fc6ac09b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 --2- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12f8b0 0x7fc6ac09b870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000f60 tx=0x7fc69c00c730 comp rx=0 tx=0).stop 2023-09-30T22:18:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 shutdown_connections 2023-09-30T22:18:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 --2- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6ac130430 0x7fc6ac09bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 --2- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6ac12cf60 0x7fc6ac09b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 --2- 172.21.15.89:0/1377282053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6ac12f8b0 0x7fc6ac09b870 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 >> 172.21.15.89:0/1377282053 conn(0x7fc6ac0a2aa0 msgr2=0x7fc6ac06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:59.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 shutdown_connections 2023-09-30T22:18:59.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.222+0000 7fc6b36c8700 1 -- 172.21.15.89:0/1377282053 wait complete. 2023-09-30T22:18:59.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:18:59.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.295+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f58b409ef10 con 0x7f58b409d6f0 2023-09-30T22:18:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.295+0000 7f589bfff700 1 -- 172.21.15.89:0/3658796906 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58a40102e0 con 0x7f58b409d6f0 2023-09-30T22:18:59.296 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:18:59.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b409d6f0 msgr2=0x7f58b41c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:18:59.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b409d6f0 0x7f58b41c4440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58a400c4e0 tx=0x7f58a400c7f0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 shutdown_connections 2023-09-30T22:18:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b4135e50 0x7f58b4119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f58b409d6f0 0x7f58b41c4440 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 --2- 172.21.15.89:0/3658796906 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f58b409e1d0 0x7f58b41c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:18:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 >> 172.21.15.89:0/3658796906 conn(0x7f58b4066f50 msgr2=0x7f58b40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:18:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 shutdown_connections 2023-09-30T22:18:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:18:59.297+0000 7f58b9aa9700 1 -- 172.21.15.89:0/3658796906 wait complete. 2023-09-30T22:18:59.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:00.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:00.183 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:00.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.280+0000 7fb18e5ac700 1 Processor -- start 2023-09-30T22:19:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.280+0000 7fb18e5ac700 1 -- start start 2023-09-30T22:19:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb18e5ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb188138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb18e5ac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb188139170 0x7fb18813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb18e5ac700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18809b120 con 0x7fb18809d6f0 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb18e5ac700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18806b460 con 0x7fb188139170 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb18e5ac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb18809a7e0 con 0x7fb18809e1d0 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb1877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb188138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb1877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb188138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54202/0 (socket says 172.21.15.89:54202) 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb1877fe700 1 -- 172.21.15.89:0/4289264728 learned_addr learned my addr 172.21.15.89:0/4289264728 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb18cb49700 1 --2- 172.21.15.89:0/4289264728 >> v2:172.21.15.89:3300/0 conn(0x7fb188139170 0x7fb18813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb1877fe700 1 -- 172.21.15.89:0/4289264728 >> v2:172.21.15.89:3300/0 conn(0x7fb188139170 msgr2=0x7fb18813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb1877fe700 1 --2- 172.21.15.89:0/4289264728 >> v2:172.21.15.89:3300/0 conn(0x7fb188139170 0x7fb18813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb1877fe700 1 -- 172.21.15.89:0/4289264728 >> v1:172.21.15.89:6789/0 conn(0x7fb18809d6f0 legacy=0x7fb18809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb1877fe700 1 -- 172.21.15.89:0/4289264728 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18809a500 con 0x7fb18809e1d0 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.281+0000 7fb1877fe700 1 --2- 172.21.15.89:0/4289264728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb188138c30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009920 tx=0x7fb17c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73644f142f77d663 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb186ffd700 1 -- 172.21.15.89:0/4289264728 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c011830 con 0x7fb18809e1d0 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb186ffd700 1 -- 172.21.15.89:0/4289264728 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c011990 con 0x7fb18809e1d0 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb186ffd700 1 -- 172.21.15.89:0/4289264728 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c011d20 con 0x7fb18809e1d0 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb18e5ac700 1 -- 172.21.15.89:0/4289264728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 msgr2=0x7fb188138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb18e5ac700 1 --2- 172.21.15.89:0/4289264728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb188138c30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009920 tx=0x7fb17c009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb18e5ac700 1 -- 172.21.15.89:0/4289264728 shutdown_connections 2023-09-30T22:19:00.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb18e5ac700 1 --2- 172.21.15.89:0/4289264728 >> v2:172.21.15.89:3300/0 conn(0x7fb188139170 0x7fb18813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb18e5ac700 1 --2- 172.21.15.89:0/4289264728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb188138c30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb18e5ac700 1 -- 172.21.15.89:0/4289264728 >> 172.21.15.89:0/4289264728 conn(0x7fb188066f50 msgr2=0x7fb18809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb18e5ac700 1 -- 172.21.15.89:0/4289264728 shutdown_connections 2023-09-30T22:19:00.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.282+0000 7fb18e5ac700 1 -- 172.21.15.89:0/4289264728 wait complete. 2023-09-30T22:19:00.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.283+0000 7fb18e5ac700 1 Processor -- start 2023-09-30T22:19:00.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb18e5ac700 1 -- start start 2023-09-30T22:19:00.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb18e5ac700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb18809d6f0 0x7fb1881c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb18e5ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb1881c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb18e5ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb188139170 0x7fb188121860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb18e5ac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb188122d20 con 0x7fb188139170 2023-09-30T22:19:00.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb18e5ac700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb188123660 con 0x7fb18809d6f0 2023-09-30T22:19:00.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb18e5ac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb18809c710 con 0x7fb18809e1d0 2023-09-30T22:19:00.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb1877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb1881c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb187fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb18809d6f0 0x7fb1881c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb1877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb1881c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54204/0 (socket says 172.21.15.89:54204) 2023-09-30T22:19:00.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.284+0000 7fb1877fe700 1 -- 172.21.15.89:0/3630420765 learned_addr learned my addr 172.21.15.89:0/3630420765 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:00.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb18cb49700 1 --2- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb188139170 0x7fb188121860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb187fff700 1 -- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 msgr2=0x7fb1881c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb187fff700 1 --2- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb1881c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb187fff700 1 -- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb188139170 msgr2=0x7fb188121860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb187fff700 1 --2- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb188139170 0x7fb188121860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb187fff700 1 -- 172.21.15.89:0/3630420765 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c009550 con 0x7fb18809d6f0 2023-09-30T22:19:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb187fff700 1 --2- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb18809d6f0 0x7fb1881c8590 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb178000f60 tx=0x7fb17800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb184ff9700 1 -- 172.21.15.89:0/3630420765 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780052d0 con 0x7fb18809d6f0 2023-09-30T22:19:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb184ff9700 1 -- 172.21.15.89:0/3630420765 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb178005430 con 0x7fb18809d6f0 2023-09-30T22:19:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb188125380 con 0x7fb18809d6f0 2023-09-30T22:19:00.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb184ff9700 1 -- 172.21.15.89:0/3630420765 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780038c0 con 0x7fb18809d6f0 2023-09-30T22:19:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.285+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1881250a0 con 0x7fb18809d6f0 2023-09-30T22:19:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.286+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb188124f00 con 0x7fb18809d6f0 2023-09-30T22:19:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.287+0000 7fb184ff9700 1 -- 172.21.15.89:0/3630420765 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb178003a20 con 0x7fb18809d6f0 2023-09-30T22:19:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.287+0000 7fb184ff9700 1 -- 172.21.15.89:0/3630420765 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb17803ec20 con 0x7fb18809d6f0 2023-09-30T22:19:00.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.291+0000 7fb184ff9700 1 -- 172.21.15.89:0/3630420765 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb178003cd0 con 0x7fb18809d6f0 2023-09-30T22:19:00.311 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:00.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.326+0000 7fb135731700 1 Processor -- start 2023-09-30T22:19:00.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.326+0000 7fb135731700 1 -- start start 2023-09-30T22:19:00.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb135731700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb13009d6f0 0x7fb13009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb135731700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb130130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb135731700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13009f130 con 0x7fb13009e410 2023-09-30T22:19:00.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb135731700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13009f520 con 0x7fb13009d6f0 2023-09-30T22:19:00.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb135731700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13009e930 con 0x7fb13009e000 2023-09-30T22:19:00.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb12effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb13009d6f0 0x7fb13009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb12effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb13009d6f0 0x7fb13009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42368/0 (socket says 172.21.15.89:42368) 2023-09-30T22:19:00.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb12effd700 1 -- 172.21.15.89:0/2361333666 learned_addr learned my addr 172.21.15.89:0/2361333666 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:00.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb12e7fc700 1 --2- 172.21.15.89:0/2361333666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb130130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb12e7fc700 1 -- 172.21.15.89:0/2361333666 >> v2:172.21.15.89:3300/0 conn(0x7fb13009d6f0 msgr2=0x7fb13009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:00.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb12e7fc700 1 --2- 172.21.15.89:0/2361333666 >> v2:172.21.15.89:3300/0 conn(0x7fb13009d6f0 0x7fb13009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb12e7fc700 1 -- 172.21.15.89:0/2361333666 >> v1:172.21.15.89:6789/0 conn(0x7fb13009e410 legacy=0x7fb130130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.327+0000 7fb12e7fc700 1 -- 172.21.15.89:0/2361333666 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13009b2d0 con 0x7fb13009e000 2023-09-30T22:19:00.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.328+0000 7fb12e7fc700 1 --2- 172.21.15.89:0/2361333666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb130130660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb12400dc60 tx=0x7fb12400df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=218a36ed6c0fcd35 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:00.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.328+0000 7fb12dffb700 1 -- 172.21.15.89:0/2361333666 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124012070 con 0x7fb13009e000 2023-09-30T22:19:00.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.328+0000 7fb12dffb700 1 -- 172.21.15.89:0/2361333666 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12400d2e0 con 0x7fb13009e000 2023-09-30T22:19:00.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.328+0000 7fb12dffb700 1 -- 172.21.15.89:0/2361333666 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12400f9b0 con 0x7fb13009e000 2023-09-30T22:19:00.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.328+0000 7fb135731700 1 -- 172.21.15.89:0/2361333666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 msgr2=0x7fb130130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.328+0000 7fb135731700 1 --2- 172.21.15.89:0/2361333666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb130130660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb12400dc60 tx=0x7fb12400df70 comp rx=0 tx=0).stop 2023-09-30T22:19:00.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.329+0000 7fb135731700 1 -- 172.21.15.89:0/2361333666 shutdown_connections 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.329+0000 7fb135731700 1 --2- 172.21.15.89:0/2361333666 >> v2:172.21.15.89:3300/0 conn(0x7fb13009d6f0 0x7fb13009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.329+0000 7fb135731700 1 --2- 172.21.15.89:0/2361333666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb130130660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.329+0000 7fb135731700 1 -- 172.21.15.89:0/2361333666 >> 172.21.15.89:0/2361333666 conn(0x7fb130066f50 msgr2=0x7fb13009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.329+0000 7fb135731700 1 -- 172.21.15.89:0/2361333666 shutdown_connections 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.329+0000 7fb135731700 1 -- 172.21.15.89:0/2361333666 wait complete. 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb135731700 1 Processor -- start 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb135731700 1 -- start start 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb135731700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb13009d6f0 0x7fb13011df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb135731700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb13011e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb135731700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13009e410 0x7fb13011e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb135731700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1301302a0 con 0x7fb13009e410 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb135731700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13011f3b0 con 0x7fb13009e000 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb135731700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb130120d20 con 0x7fb13009d6f0 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb12effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb13009d6f0 0x7fb13011df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb12effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb13009d6f0 0x7fb13011df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54208/0 (socket says 172.21.15.89:54208) 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb12effd700 1 -- 172.21.15.89:0/2617077600 learned_addr learned my addr 172.21.15.89:0/2617077600 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb12f7fe700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13009e410 0x7fb13011e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.330+0000 7fb12e7fc700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb13011e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb12effd700 1 -- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 msgr2=0x7fb13011e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb12effd700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb13011e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb12effd700 1 -- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13009e410 msgr2=0x7fb13011e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb12effd700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13009e410 0x7fb13011e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb12effd700 1 -- 172.21.15.89:0/2617077600 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12400d940 con 0x7fb13009d6f0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb12effd700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb13009d6f0 0x7fb13011df50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb12000dad0 tx=0x7fb12000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb1177fe700 1 -- 172.21.15.89:0/2617077600 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12000a970 con 0x7fb13009d6f0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb1177fe700 1 -- 172.21.15.89:0/2617077600 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12000aad0 con 0x7fb13009d6f0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb1177fe700 1 -- 172.21.15.89:0/2617077600 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12000ae80 con 0x7fb13009d6f0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1301200b0 con 0x7fb13009d6f0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.331+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1300a1d00 con 0x7fb13009d6f0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.332+0000 7fb1177fe700 1 -- 172.21.15.89:0/2617077600 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb120019400 con 0x7fb13009d6f0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.333+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13011ac20 con 0x7fb13009d6f0 2023-09-30T22:19:00.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.333+0000 7fb1177fe700 1 -- 172.21.15.89:0/2617077600 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb12003f820 con 0x7fb13009d6f0 2023-09-30T22:19:00.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.339+0000 7fb1177fe700 1 -- 172.21.15.89:0/2617077600 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb120018950 con 0x7fb13009d6f0 2023-09-30T22:19:00.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.375+0000 7f4bfb34e700 1 Processor -- start 2023-09-30T22:19:00.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.375+0000 7f4bfb34e700 1 -- start start 2023-09-30T22:19:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.375+0000 7f4bfb34e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4bf409d6f0 0x7f4bf409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.375+0000 7f4bfb34e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e440 0x7f4bf4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.375+0000 7f4bfb34e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf409b120 con 0x7f4bf409e000 2023-09-30T22:19:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.375+0000 7f4bfb34e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf406b460 con 0x7f4bf409d6f0 2023-09-30T22:19:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.375+0000 7f4bfb34e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf409a7e0 con 0x7f4bf409e440 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf98eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e440 0x7f4bf4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf90ea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4bf409d6f0 0x7f4bf409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf98eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e440 0x7f4bf4134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55468/0 (socket says 172.21.15.89:55468) 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf90ea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4bf409d6f0 0x7f4bf409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42398/0 (socket says 172.21.15.89:42398) 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf88e9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4bf409e000 0x7f4bf41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50042/0 (socket says 172.21.15.89:50042) 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf98eb700 1 -- 172.21.15.89:0/3358100113 learned_addr learned my addr 172.21.15.89:0/3358100113 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf90ea700 1 -- 172.21.15.89:0/3358100113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e440 msgr2=0x7f4bf4134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf90ea700 1 --2- 172.21.15.89:0/3358100113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e440 0x7f4bf4134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf90ea700 1 -- 172.21.15.89:0/3358100113 >> v1:172.21.15.89:6789/0 conn(0x7f4bf409e000 legacy=0x7f4bf41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf90ea700 1 -- 172.21.15.89:0/3358100113 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf409a500 con 0x7f4bf409d6f0 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bebfff700 1 -- 172.21.15.89:0/3358100113 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095443669 0 0) 0x7f4bf409b120 con 0x7f4bf409e000 2023-09-30T22:19:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf98eb700 1 --2- 172.21.15.89:0/3358100113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e440 0x7f4bf4134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:00.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bf90ea700 1 --2- 172.21.15.89:0/3358100113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf409dac0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009870 tx=0x7f4bf0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7d28f4b226b9ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:00.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bebfff700 1 -- 172.21.15.89:0/3358100113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0011830 con 0x7f4bf409d6f0 2023-09-30T22:19:00.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bebfff700 1 -- 172.21.15.89:0/3358100113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0011990 con 0x7f4bf409d6f0 2023-09-30T22:19:00.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.376+0000 7f4bebfff700 1 -- 172.21.15.89:0/3358100113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0011d20 con 0x7f4bf409d6f0 2023-09-30T22:19:00.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.377+0000 7f4bfb34e700 1 -- 172.21.15.89:0/3358100113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 msgr2=0x7f4bf409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.377+0000 7f4bfb34e700 1 --2- 172.21.15.89:0/3358100113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf409dac0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009870 tx=0x7f4bf0009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:00.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.377+0000 7f4bfb34e700 1 -- 172.21.15.89:0/3358100113 shutdown_connections 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.377+0000 7f4bfb34e700 1 --2- 172.21.15.89:0/3358100113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e440 0x7f4bf4134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.377+0000 7f4bfb34e700 1 --2- 172.21.15.89:0/3358100113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf409dac0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.377+0000 7f4bfb34e700 1 -- 172.21.15.89:0/3358100113 >> 172.21.15.89:0/3358100113 conn(0x7f4bf4066f50 msgr2=0x7f4bf409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.377+0000 7f4bfb34e700 1 -- 172.21.15.89:0/3358100113 shutdown_connections 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.377+0000 7f4bfb34e700 1 -- 172.21.15.89:0/3358100113 wait complete. 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.379+0000 7f4bfb34e700 1 Processor -- start 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.379+0000 7f4bfb34e700 1 -- start start 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.379+0000 7f4bfb34e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf41a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.379+0000 7f4bfb34e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e000 0x7f4bf41a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.379+0000 7f4bfb34e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf409e440 0x7f4bf41c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.379+0000 7f4bfb34e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40a2eb0 con 0x7f4bf409d6f0 2023-09-30T22:19:00.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.379+0000 7f4bfb34e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf41a4e30 con 0x7f4bf409e000 2023-09-30T22:19:00.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.379+0000 7f4bfb34e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf409c710 con 0x7f4bf409e440 2023-09-30T22:19:00.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf41a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf88e9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e000 0x7f4bf41a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf41a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42408/0 (socket says 172.21.15.89:42408) 2023-09-30T22:19:00.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 -- 172.21.15.89:0/1148654529 learned_addr learned my addr 172.21.15.89:0/1148654529 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:00.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf98eb700 1 --2- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf409e440 0x7f4bf41c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 -- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf409e440 msgr2=0x7f4bf41c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 --2- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf409e440 0x7f4bf41c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 -- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e000 msgr2=0x7f4bf41a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 --2- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e000 0x7f4bf41a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 -- 172.21.15.89:0/1148654529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec009b30 con 0x7f4bf409d6f0 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bf90ea700 1 --2- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf41a51f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000c00 tx=0x7f4bf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4be9ffb700 1 -- 172.21.15.89:0/1148654529 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf001a960 con 0x7f4bf409d6f0 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4be9ffb700 1 -- 172.21.15.89:0/1148654529 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf001aac0 con 0x7f4bf409d6f0 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4be9ffb700 1 -- 172.21.15.89:0/1148654529 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf001ae50 con 0x7f4bf409d6f0 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.380+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf00094a0 con 0x7f4bf409d6f0 2023-09-30T22:19:00.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.381+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf41a7d60 con 0x7f4bf409d6f0 2023-09-30T22:19:00.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.381+0000 7f4be9ffb700 1 -- 172.21.15.89:0/1148654529 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bf00243f0 con 0x7f4bf409d6f0 2023-09-30T22:19:00.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.382+0000 7f4be9ffb700 1 -- 172.21.15.89:0/1148654529 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4bf0040bd0 con 0x7f4bf409d6f0 2023-09-30T22:19:00.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.382+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf4074ea0 con 0x7f4bf409d6f0 2023-09-30T22:19:00.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.391+0000 7f4be9ffb700 1 -- 172.21.15.89:0/1148654529 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bf0015080 con 0x7f4bf409d6f0 2023-09-30T22:19:00.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.446+0000 7fc8a97ca700 1 Processor -- start 2023-09-30T22:19:00.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.446+0000 7fc8a97ca700 1 -- start start 2023-09-30T22:19:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a97ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc8a40a0a30 0x7fc8a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a97ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a4134170 0x7fc8a413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a97ca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a409edd0 con 0x7fc8a40a1340 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a97ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a409e1f0 con 0x7fc8a40a0a30 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a97ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a409dc50 con 0x7fc8a4134170 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc8a40a0a30 0x7fc8a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc8a40a0a30 0x7fc8a40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42422/0 (socket says 172.21.15.89:42422) 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a2ffd700 1 -- 172.21.15.89:0/3451004217 learned_addr learned my addr 172.21.15.89:0/3451004217 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.447+0000 7fc8a37fe700 1 --2- 172.21.15.89:0/3451004217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a4134170 0x7fc8a413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.448+0000 7fc8a2ffd700 1 -- 172.21.15.89:0/3451004217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a4134170 msgr2=0x7fc8a413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.448+0000 7fc8a2ffd700 1 --2- 172.21.15.89:0/3451004217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a4134170 0x7fc8a413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.448+0000 7fc8a2ffd700 1 -- 172.21.15.89:0/3451004217 >> v1:172.21.15.89:6789/0 conn(0x7fc8a40a1340 legacy=0x7fc8a4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.448+0000 7fc8a2ffd700 1 -- 172.21.15.89:0/3451004217 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40a1fe0 con 0x7fc8a40a0a30 2023-09-30T22:19:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.448+0000 7fc8a2ffd700 1 --2- 172.21.15.89:0/3451004217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a40a0a30 0x7fc8a40a0e00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc8940098f0 tx=0x7fc894009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6a31cb2e310a51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.448+0000 7fc8a1ffb700 1 -- 172.21.15.89:0/3451004217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894004230 con 0x7fc8a40a0a30 2023-09-30T22:19:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.448+0000 7fc8a1ffb700 1 -- 172.21.15.89:0/3451004217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894004390 con 0x7fc8a40a0a30 2023-09-30T22:19:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.448+0000 7fc8a1ffb700 1 -- 172.21.15.89:0/3451004217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8940036a0 con 0x7fc8a40a0a30 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.449+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3451004217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a40a0a30 msgr2=0x7fc8a40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.449+0000 7fc8a97ca700 1 --2- 172.21.15.89:0/3451004217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a40a0a30 0x7fc8a40a0e00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc8940098f0 tx=0x7fc894009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.449+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3451004217 shutdown_connections 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.449+0000 7fc8a97ca700 1 --2- 172.21.15.89:0/3451004217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a4134170 0x7fc8a413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.449+0000 7fc8a97ca700 1 --2- 172.21.15.89:0/3451004217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a40a0a30 0x7fc8a40a0e00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.449+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3451004217 >> 172.21.15.89:0/3451004217 conn(0x7fc8a40a6550 msgr2=0x7fc8a40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.449+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3451004217 shutdown_connections 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.449+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3451004217 wait complete. 2023-09-30T22:19:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.450+0000 7fc8a97ca700 1 Processor -- start 2023-09-30T22:19:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.450+0000 7fc8a97ca700 1 -- start start 2023-09-30T22:19:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a97ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8a40a0a30 0x7fc8a41c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a97ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a40a1340 0x7fc8a41c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a97ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a4134170 0x7fc8a411e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a97ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a41bd570 con 0x7fc8a4134170 2023-09-30T22:19:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a97ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40a5080 con 0x7fc8a40a1340 2023-09-30T22:19:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a97ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40a0220 con 0x7fc8a40a0a30 2023-09-30T22:19:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a27fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a40a1340 0x7fc8a41c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a27fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a40a1340 0x7fc8a41c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55484/0 (socket says 172.21.15.89:55484) 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8a40a0a30 0x7fc8a41c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a27fc700 1 -- 172.21.15.89:0/3756102306 learned_addr learned my addr 172.21.15.89:0/3756102306 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.451+0000 7fc8a37fe700 1 --2- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a4134170 0x7fc8a411e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc8a27fc700 1 -- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8a40a0a30 msgr2=0x7fc8a41c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc8a27fc700 1 --2- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8a40a0a30 0x7fc8a41c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc8a27fc700 1 -- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a4134170 msgr2=0x7fc8a411e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc8a27fc700 1 --2- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a4134170 0x7fc8a411e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc8a27fc700 1 -- 172.21.15.89:0/3756102306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc894009550 con 0x7fc8a40a1340 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc8a27fc700 1 --2- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a40a1340 0x7fc8a41c90b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc89800c4a0 tx=0x7fc89800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc88b7fe700 1 -- 172.21.15.89:0/3756102306 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980054e0 con 0x7fc8a40a1340 2023-09-30T22:19:00.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc88b7fe700 1 -- 172.21.15.89:0/3756102306 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc898003680 con 0x7fc8a40a1340 2023-09-30T22:19:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc88b7fe700 1 -- 172.21.15.89:0/3756102306 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898003a30 con 0x7fc8a40a1340 2023-09-30T22:19:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a40a25e0 con 0x7fc8a40a1340 2023-09-30T22:19:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.452+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a4121d50 con 0x7fc8a40a1340 2023-09-30T22:19:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.453+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a4074ea0 con 0x7fc8a40a1340 2023-09-30T22:19:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.454+0000 7fc88b7fe700 1 -- 172.21.15.89:0/3756102306 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc898003b90 con 0x7fc8a40a1340 2023-09-30T22:19:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.454+0000 7fc88b7fe700 1 -- 172.21.15.89:0/3756102306 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc89803e5b0 con 0x7fc8a40a1340 2023-09-30T22:19:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.458+0000 7fc88b7fe700 1 -- 172.21.15.89:0/3756102306 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc898003e40 con 0x7fc8a40a1340 2023-09-30T22:19:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.527+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb188124f00 con 0x7fb18809d6f0 2023-09-30T22:19:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.527+0000 7fb184ff9700 1 -- 172.21.15.89:0/3630420765 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb188124f00 con 0x7fb18809d6f0 2023-09-30T22:19:00.528 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.530+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb18809d6f0 msgr2=0x7fb1881c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.530+0000 7fb18e5ac700 1 --2- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb18809d6f0 0x7fb1881c8590 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb178000f60 tx=0x7fb17800c730 comp rx=0 tx=0).stop 2023-09-30T22:19:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.530+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 shutdown_connections 2023-09-30T22:19:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.530+0000 7fb18e5ac700 1 --2- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb188139170 0x7fb188121860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.530+0000 7fb18e5ac700 1 --2- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb18809d6f0 0x7fb1881c8590 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.530+0000 7fb18e5ac700 1 --2- 172.21.15.89:0/3630420765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb18809e1d0 0x7fb1881c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.530+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 >> 172.21.15.89:0/3630420765 conn(0x7fb188066f50 msgr2=0x7fb18806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.530+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 shutdown_connections 2023-09-30T22:19:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.531+0000 7fb18e5ac700 1 -- 172.21.15.89:0/3630420765 wait complete. 2023-09-30T22:19:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.537+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb130074ea0 con 0x7fb13009d6f0 2023-09-30T22:19:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.537+0000 7fb1177fe700 1 -- 172.21.15.89:0/2617077600 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb120018360 con 0x7fb13009d6f0 2023-09-30T22:19:00.538 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.539+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb13009d6f0 msgr2=0x7fb13011df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.539+0000 7fb135731700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb13009d6f0 0x7fb13011df50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb12000dad0 tx=0x7fb12000dde0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.540+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 shutdown_connections 2023-09-30T22:19:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.540+0000 7fb135731700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb13009e410 0x7fb13011e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.540+0000 7fb135731700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb13009d6f0 0x7fb13011df50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.540+0000 7fb135731700 1 --2- 172.21.15.89:0/2617077600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb13009e000 0x7fb13011e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.540+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 >> 172.21.15.89:0/2617077600 conn(0x7fb130066f50 msgr2=0x7fb130130ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.540+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 shutdown_connections 2023-09-30T22:19:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.540+0000 7fb135731700 1 -- 172.21.15.89:0/2617077600 wait complete. 2023-09-30T22:19:00.549 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:00.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.630+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4bf409f420 con 0x7f4bf409d6f0 2023-09-30T22:19:00.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.630+0000 7f4be9ffb700 1 -- 172.21.15.89:0/1148654529 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bf0035020 con 0x7f4bf409d6f0 2023-09-30T22:19:00.632 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.632+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 msgr2=0x7f4bf41a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.632+0000 7f4bfb34e700 1 --2- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf41a51f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000c00 tx=0x7f4bf0004750 comp rx=0 tx=0).stop 2023-09-30T22:19:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.633+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 shutdown_connections 2023-09-30T22:19:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.633+0000 7f4bfb34e700 1 --2- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf409e440 0x7f4bf41c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.633+0000 7f4bfb34e700 1 --2- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf409d6f0 0x7f4bf41a51f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.633+0000 7f4bfb34e700 1 --2- 172.21.15.89:0/1148654529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf409e000 0x7f4bf41a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.633+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 >> 172.21.15.89:0/1148654529 conn(0x7f4bf4066f50 msgr2=0x7f4bf40a35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.633+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 shutdown_connections 2023-09-30T22:19:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.633+0000 7f4bfb34e700 1 -- 172.21.15.89:0/1148654529 wait complete. 2023-09-30T22:19:00.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.669+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc8a40a28e0 con 0x7fc8a40a1340 2023-09-30T22:19:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.670+0000 7fc88b7fe700 1 -- 172.21.15.89:0/3756102306 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc898017020 con 0x7fc8a40a1340 2023-09-30T22:19:00.671 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a40a1340 msgr2=0x7fc8a41c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 --2- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a40a1340 0x7fc8a41c90b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc89800c4a0 tx=0x7fc89800c7b0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 shutdown_connections 2023-09-30T22:19:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 --2- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8a4134170 0x7fc8a411e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 --2- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8a40a0a30 0x7fc8a41c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 --2- 172.21.15.89:0/3756102306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8a40a1340 0x7fc8a41c90b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 >> 172.21.15.89:0/3756102306 conn(0x7fc8a40a6550 msgr2=0x7fc8a41b9320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 shutdown_connections 2023-09-30T22:19:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:00.672+0000 7fc8a97ca700 1 -- 172.21.15.89:0/3756102306 wait complete. 2023-09-30T22:19:00.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:01.551 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:01.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:01.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:01.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:01.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.694+0000 7fd8b2ef8700 1 Processor -- start 2023-09-30T22:19:01.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.694+0000 7fd8b2ef8700 1 -- start start 2023-09-30T22:19:01.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b2ef8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b2ef8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8ac130e50 0x7fd8ac06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b2ef8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 0x7fd8ac06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b2ef8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ac0704b0 con 0x7fd8ac130e50 2023-09-30T22:19:01.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b2ef8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac09a390 con 0x7fd8ac130540 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b2ef8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac0716a0 con 0x7fd8ac06fb20 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55492/0 (socket says 172.21.15.89:55492) 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 -- 172.21.15.89:0/1430007508 learned_addr learned my addr 172.21.15.89:0/1430007508 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 -- 172.21.15.89:0/1430007508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 msgr2=0x7fd8ac06ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 --2- 172.21.15.89:0/1430007508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 0x7fd8ac06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 -- 172.21.15.89:0/1430007508 >> v2:172.21.15.89:3300/0 conn(0x7fd8ac130e50 msgr2=0x7fd8ac06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 --2- 172.21.15.89:0/1430007508 >> v2:172.21.15.89:3300/0 conn(0x7fd8ac130e50 0x7fd8ac06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 -- 172.21.15.89:0/1430007508 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac071290 con 0x7fd8ac130540 2023-09-30T22:19:01.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.695+0000 7fd8b0c94700 1 --2- 172.21.15.89:0/1430007508 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac130910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00098f0 tx=0x7fd8a0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59dac7cb6d7c4617 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:01.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8ab7fe700 1 -- 172.21.15.89:0/1430007508 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000b8a0 con 0x7fd8ac130540 2023-09-30T22:19:01.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8ab7fe700 1 -- 172.21.15.89:0/1430007508 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a000ba00 con 0x7fd8ac130540 2023-09-30T22:19:01.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8ab7fe700 1 -- 172.21.15.89:0/1430007508 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000bd90 con 0x7fd8ac130540 2023-09-30T22:19:01.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/1430007508 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 msgr2=0x7fd8ac130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8b2ef8700 1 --2- 172.21.15.89:0/1430007508 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac130910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00098f0 tx=0x7fd8a0009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:01.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/1430007508 shutdown_connections 2023-09-30T22:19:01.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8b2ef8700 1 --2- 172.21.15.89:0/1430007508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 0x7fd8ac06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8b2ef8700 1 --2- 172.21.15.89:0/1430007508 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac130910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8b2ef8700 1 --2- 172.21.15.89:0/1430007508 >> v2:172.21.15.89:3300/0 conn(0x7fd8ac130e50 0x7fd8ac06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/1430007508 >> 172.21.15.89:0/1430007508 conn(0x7fd8ac0a2aa0 msgr2=0x7fd8ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.696+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/1430007508 shutdown_connections 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.697+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/1430007508 wait complete. 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.698+0000 7fd8b2ef8700 1 Processor -- start 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.698+0000 7fd8b2ef8700 1 -- start start 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b2ef8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 0x7fd8ac1c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b2ef8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac1c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b2ef8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8ac130e50 0x7fd8ac1c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b2ef8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac1212b0 con 0x7fd8ac130e50 2023-09-30T22:19:01.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b2ef8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac0a1f30 con 0x7fd8ac130540 2023-09-30T22:19:01.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b2ef8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac06f060 con 0x7fd8ac06fb20 2023-09-30T22:19:01.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b1495700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8ac130e50 0x7fd8ac1c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8abfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac1c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b1495700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8ac130e50 0x7fd8ac1c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42442/0 (socket says 172.21.15.89:42442) 2023-09-30T22:19:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b1495700 1 -- 172.21.15.89:0/622823146 learned_addr learned my addr 172.21.15.89:0/622823146 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.699+0000 7fd8b0c94700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 0x7fd8ac1c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8b1495700 1 -- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 msgr2=0x7fd8ac1c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8b1495700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 0x7fd8ac1c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8b1495700 1 -- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 msgr2=0x7fd8ac1c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8b1495700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac1c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8b1495700 1 -- 172.21.15.89:0/622823146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0009550 con 0x7fd8ac130e50 2023-09-30T22:19:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8abfff700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac1c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8b1495700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8ac130e50 0x7fd8ac1c8ff0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b9a0 tx=0x7fd8a400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8a97fa700 1 -- 172.21.15.89:0/622823146 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400c930 con 0x7fd8ac130e50 2023-09-30T22:19:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8a97fa700 1 -- 172.21.15.89:0/622823146 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a400cab0 con 0x7fd8ac130e50 2023-09-30T22:19:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac071940 con 0x7fd8ac130e50 2023-09-30T22:19:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8a97fa700 1 -- 172.21.15.89:0/622823146 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400ce60 con 0x7fd8ac130e50 2023-09-30T22:19:01.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.700+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac1216e0 con 0x7fd8ac130e50 2023-09-30T22:19:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.701+0000 7fd8a97fa700 1 -- 172.21.15.89:0/622823146 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd8a401d3b0 con 0x7fd8ac130e50 2023-09-30T22:19:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.701+0000 7fd8a97fa700 1 -- 172.21.15.89:0/622823146 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd8a4006e40 con 0x7fd8ac130e50 2023-09-30T22:19:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.701+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac074ea0 con 0x7fd8ac130e50 2023-09-30T22:19:01.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.707+0000 7fd8a97fa700 1 -- 172.21.15.89:0/622823146 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8a4019020 con 0x7fd8ac130e50 2023-09-30T22:19:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.736+0000 7f16a08e6700 1 Processor -- start 2023-09-30T22:19:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.736+0000 7f16a08e6700 1 -- start start 2023-09-30T22:19:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f16a08e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f16a08e6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f169c135e60 0x7f169c138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f16a08e6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c0716a0 con 0x7f169c130540 2023-09-30T22:19:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f16a08e6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169c071290 con 0x7f169c135e60 2023-09-30T22:19:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f16a08e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f169c130340 con 0x7f169c133540 2023-09-30T22:19:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f1699d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f1699d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54250/0 (socket says 172.21.15.89:54250) 2023-09-30T22:19:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f1699d9b700 1 -- 172.21.15.89:0/3098738998 learned_addr learned my addr 172.21.15.89:0/3098738998 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f169ad9d700 1 --2- 172.21.15.89:0/3098738998 >> v2:172.21.15.89:3300/0 conn(0x7f169c135e60 0x7f169c138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f1699d9b700 1 -- 172.21.15.89:0/3098738998 >> v2:172.21.15.89:3300/0 conn(0x7f169c135e60 msgr2=0x7f169c138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f1699d9b700 1 --2- 172.21.15.89:0/3098738998 >> v2:172.21.15.89:3300/0 conn(0x7f169c135e60 0x7f169c138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f1699d9b700 1 -- 172.21.15.89:0/3098738998 >> v1:172.21.15.89:6789/0 conn(0x7f169c130540 legacy=0x7f169c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.737+0000 7f1699d9b700 1 -- 172.21.15.89:0/3098738998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c070b80 con 0x7f169c133540 2023-09-30T22:19:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.738+0000 7f1699d9b700 1 --2- 172.21.15.89:0/3098738998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c135920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1690009920 tx=0x7f1690009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af3dd8294d86e2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.741+0000 7f169959a700 1 -- 172.21.15.89:0/3098738998 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690011830 con 0x7f169c133540 2023-09-30T22:19:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.741+0000 7f169959a700 1 -- 172.21.15.89:0/3098738998 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1690011990 con 0x7f169c133540 2023-09-30T22:19:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.741+0000 7f169959a700 1 -- 172.21.15.89:0/3098738998 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690011d20 con 0x7f169c133540 2023-09-30T22:19:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.742+0000 7f16a08e6700 1 -- 172.21.15.89:0/3098738998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 msgr2=0x7f169c135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.742+0000 7f16a08e6700 1 --2- 172.21.15.89:0/3098738998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c135920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1690009920 tx=0x7f1690009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:01.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.742+0000 7f16a08e6700 1 -- 172.21.15.89:0/3098738998 shutdown_connections 2023-09-30T22:19:01.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.742+0000 7f16a08e6700 1 --2- 172.21.15.89:0/3098738998 >> v2:172.21.15.89:3300/0 conn(0x7f169c135e60 0x7f169c138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.742+0000 7f16a08e6700 1 --2- 172.21.15.89:0/3098738998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c135920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.742+0000 7f16a08e6700 1 -- 172.21.15.89:0/3098738998 >> 172.21.15.89:0/3098738998 conn(0x7f169c0a2aa0 msgr2=0x7f169c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:01.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.742+0000 7f16a08e6700 1 -- 172.21.15.89:0/3098738998 shutdown_connections 2023-09-30T22:19:01.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.742+0000 7f16a08e6700 1 -- 172.21.15.89:0/3098738998 wait complete. 2023-09-30T22:19:01.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.743+0000 7f16a08e6700 1 Processor -- start 2023-09-30T22:19:01.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f16a08e6700 1 -- start start 2023-09-30T22:19:01.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f16a08e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130540 0x7f169c12d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f16a08e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c12d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f16a08e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c135e60 0x7f169c139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f16a08e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c09ace0 con 0x7f169c135e60 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f16a08e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c13ae40 con 0x7f169c130540 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f16a08e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f169c1393f0 con 0x7f169c133540 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f169a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130540 0x7f169c12d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f169a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130540 0x7f169c12d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55512/0 (socket says 172.21.15.89:55512) 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f169a59c700 1 -- 172.21.15.89:0/3635404567 learned_addr learned my addr 172.21.15.89:0/3635404567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.744+0000 7f169ad9d700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c135e60 0x7f169c139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f1699d9b700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c12d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f169a59c700 1 -- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 msgr2=0x7f169c12d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f169a59c700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c12d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f169a59c700 1 -- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c135e60 msgr2=0x7f169c139790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f169a59c700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c135e60 0x7f169c139790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f169a59c700 1 -- 172.21.15.89:0/3635404567 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690009550 con 0x7f169c130540 2023-09-30T22:19:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f169a59c700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130540 0x7f169c12d1e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f168c00c450 tx=0x7f168c00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f168affd700 1 -- 172.21.15.89:0/3635404567 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c0052d0 con 0x7f169c130540 2023-09-30T22:19:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c070ec0 con 0x7f169c130540 2023-09-30T22:19:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f168affd700 1 -- 172.21.15.89:0/3635404567 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c005430 con 0x7f169c130540 2023-09-30T22:19:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f168affd700 1 -- 172.21.15.89:0/3635404567 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c0038c0 con 0x7f169c130540 2023-09-30T22:19:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.745+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c09a6f0 con 0x7f169c130540 2023-09-30T22:19:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.746+0000 7f168affd700 1 -- 172.21.15.89:0/3635404567 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f168c003a20 con 0x7f169c130540 2023-09-30T22:19:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.747+0000 7f168affd700 1 -- 172.21.15.89:0/3635404567 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f168c0118c0 con 0x7f169c130540 2023-09-30T22:19:01.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.749+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c074ea0 con 0x7f169c130540 2023-09-30T22:19:01.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.753+0000 7f168affd700 1 -- 172.21.15.89:0/3635404567 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f168c017080 con 0x7f169c130540 2023-09-30T22:19:01.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.784+0000 7fb08ae6c700 1 Processor -- start 2023-09-30T22:19:01.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.784+0000 7fb08ae6c700 1 -- start start 2023-09-30T22:19:01.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.784+0000 7fb08ae6c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412cf60 0x7fb08406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.784+0000 7fb08ae6c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412f8b0 0x7fb08412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.784+0000 7fb08ae6c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb084130260 0x7fb084136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.784+0000 7fb08ae6c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08412d330 con 0x7fb084130260 2023-09-30T22:19:01.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.784+0000 7fb08ae6c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08412cd90 con 0x7fb08412cf60 2023-09-30T22:19:01.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.784+0000 7fb08ae6c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841306d0 con 0x7fb08412f8b0 2023-09-30T22:19:01.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb084130260 0x7fb084136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb084130260 0x7fb084136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42468/0 (socket says 172.21.15.89:42468) 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 -- 172.21.15.89:0/3176778993 learned_addr learned my addr 172.21.15.89:0/3176778993 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb083fff700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412f8b0 0x7fb08412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb088c08700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412cf60 0x7fb08406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 -- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412f8b0 msgr2=0x7fb08412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412f8b0 0x7fb08412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 -- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412cf60 msgr2=0x7fb08406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412cf60 0x7fb08406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 -- 172.21.15.89:0/3176778993 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb084071780 con 0x7fb084130260 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb089409700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 0x7fb084136c20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009e50 tx=0x7fb07c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93717bd931303a82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:01.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb0837fe700 1 -- 172.21.15.89:0/3176778993 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00b910 con 0x7fb084130260 2023-09-30T22:19:01.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb0837fe700 1 -- 172.21.15.89:0/3176778993 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c00ba70 con 0x7fb084130260 2023-09-30T22:19:01.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.785+0000 7fb0837fe700 1 -- 172.21.15.89:0/3176778993 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00be20 con 0x7fb084130260 2023-09-30T22:19:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 msgr2=0x7fb084136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 0x7fb084136c20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009e50 tx=0x7fb07c00b330 comp rx=0 tx=0).stop 2023-09-30T22:19:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3176778993 shutdown_connections 2023-09-30T22:19:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 0x7fb084136c20 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412cf60 0x7fb08406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 --2- 172.21.15.89:0/3176778993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412f8b0 0x7fb08412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3176778993 >> 172.21.15.89:0/3176778993 conn(0x7fb0840a2aa0 msgr2=0x7fb0840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3176778993 shutdown_connections 2023-09-30T22:19:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.786+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3176778993 wait complete. 2023-09-30T22:19:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.787+0000 7fb08ae6c700 1 Processor -- start 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.787+0000 7fb08ae6c700 1 -- start start 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb08ae6c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412cf60 0x7fb0841c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb08ae6c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412f8b0 0x7fb0841c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb08ae6c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 0x7fb0841c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb08ae6c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08411dd90 con 0x7fb084130260 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb08ae6c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841c3a80 con 0x7fb08412f8b0 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb08ae6c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb084129fa0 con 0x7fb08412cf60 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412cf60 0x7fb0841c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412cf60 0x7fb0841c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54292/0 (socket says 172.21.15.89:54292) 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 -- 172.21.15.89:0/3228388691 learned_addr learned my addr 172.21.15.89:0/3228388691 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb089409700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 0x7fb0841c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb083fff700 1 --2- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412f8b0 0x7fb0841c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 -- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412f8b0 msgr2=0x7fb0841c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 --2- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412f8b0 0x7fb0841c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 -- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 msgr2=0x7fb0841c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 --2- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 0x7fb0841c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 -- 172.21.15.89:0/3228388691 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c009b30 con 0x7fb08412cf60 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.788+0000 7fb088c08700 1 --2- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412cf60 0x7fb0841c3000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e810 tx=0x7fb07800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.789+0000 7fb0817fa700 1 -- 172.21.15.89:0/3228388691 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07800c070 con 0x7fb08412cf60 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.789+0000 7fb0817fa700 1 -- 172.21.15.89:0/3228388691 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb078013610 con 0x7fb08412cf60 2023-09-30T22:19:01.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.789+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08411c080 con 0x7fb08412cf60 2023-09-30T22:19:01.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.789+0000 7fb0817fa700 1 -- 172.21.15.89:0/3228388691 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780139c0 con 0x7fb08412cf60 2023-09-30T22:19:01.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.789+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08411df90 con 0x7fb08412cf60 2023-09-30T22:19:01.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.790+0000 7fb0817fa700 1 -- 172.21.15.89:0/3228388691 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb078013b20 con 0x7fb08412cf60 2023-09-30T22:19:01.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.790+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb084074ea0 con 0x7fb08412cf60 2023-09-30T22:19:01.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.790+0000 7fb0817fa700 1 -- 172.21.15.89:0/3228388691 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb07803f3e0 con 0x7fb08412cf60 2023-09-30T22:19:01.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.799+0000 7fb0817fa700 1 -- 172.21.15.89:0/3228388691 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0780108d0 con 0x7fb08412cf60 2023-09-30T22:19:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.837+0000 7f553da1b700 1 Processor -- start 2023-09-30T22:19:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.837+0000 7f553da1b700 1 -- start start 2023-09-30T22:19:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.837+0000 7f553da1b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f55381338a0 0x7f5538133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.837+0000 7f553da1b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f553809de50 0x7f553809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.837+0000 7f553da1b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809e760 0x7f553813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.837+0000 7f553da1b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553806fef0 con 0x7f553809de50 2023-09-30T22:19:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.837+0000 7f553da1b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553806ef30 con 0x7f553809e760 2023-09-30T22:19:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.837+0000 7f553da1b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55381334b0 con 0x7f55381338a0 2023-09-30T22:19:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f5536ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f55381338a0 0x7f5538133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55377fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809e760 0x7f553813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f5536ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f55381338a0 0x7f5538133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54294/0 (socket says 172.21.15.89:54294) 2023-09-30T22:19:01.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f5536ffd700 1 -- 172.21.15.89:0/2371565609 learned_addr learned my addr 172.21.15.89:0/2371565609 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:01.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55377fe700 1 --2- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809e760 0x7f553813a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55530/0 (socket says 172.21.15.89:55530) 2023-09-30T22:19:01.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55367fc700 1 --2- 172.21.15.89:0/2371565609 >> v2:172.21.15.89:3300/0 conn(0x7f553809de50 0x7f553809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55377fe700 1 -- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f55381338a0 msgr2=0x7f5538133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:01.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55377fe700 1 --2- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f55381338a0 0x7f5538133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55377fe700 1 -- 172.21.15.89:0/2371565609 >> v2:172.21.15.89:3300/0 conn(0x7f553809de50 msgr2=0x7f553809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:01.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55377fe700 1 --2- 172.21.15.89:0/2371565609 >> v2:172.21.15.89:3300/0 conn(0x7f553809de50 0x7f553809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55377fe700 1 -- 172.21.15.89:0/2371565609 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553806fc30 con 0x7f553809e760 2023-09-30T22:19:01.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f5536ffd700 1 --2- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f55381338a0 0x7f5538133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:01.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f55377fe700 1 --2- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809e760 0x7f553813a730 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5530009d90 tx=0x7f553000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb882c4a840a4e4c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:01.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f5535ffb700 1 -- 172.21.15.89:0/2371565609 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553000e070 con 0x7f553809e760 2023-09-30T22:19:01.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f5535ffb700 1 -- 172.21.15.89:0/2371565609 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553000bb20 con 0x7f553809e760 2023-09-30T22:19:01.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.838+0000 7f5535ffb700 1 -- 172.21.15.89:0/2371565609 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530012430 con 0x7f553809e760 2023-09-30T22:19:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 -- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809e760 msgr2=0x7f553813a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 --2- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809e760 0x7f553813a730 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5530009d90 tx=0x7f553000b330 comp rx=0 tx=0).stop 2023-09-30T22:19:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 -- 172.21.15.89:0/2371565609 shutdown_connections 2023-09-30T22:19:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 --2- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809e760 0x7f553813a730 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 --2- 172.21.15.89:0/2371565609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f55381338a0 0x7f5538133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 --2- 172.21.15.89:0/2371565609 >> v2:172.21.15.89:3300/0 conn(0x7f553809de50 0x7f553809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 -- 172.21.15.89:0/2371565609 >> 172.21.15.89:0/2371565609 conn(0x7f55380a2aa0 msgr2=0x7f55380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:01.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 -- 172.21.15.89:0/2371565609 shutdown_connections 2023-09-30T22:19:01.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.839+0000 7f553da1b700 1 -- 172.21.15.89:0/2371565609 wait complete. 2023-09-30T22:19:01.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.840+0000 7f553da1b700 1 Processor -- start 2023-09-30T22:19:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.840+0000 7f553da1b700 1 -- start start 2023-09-30T22:19:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f553da1b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809de50 0x7f553811a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f553da1b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f553809e760 0x7f553811aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f553da1b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55381338a0 0x7f553811afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f553da1b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553811d8b0 con 0x7f55381338a0 2023-09-30T22:19:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f553da1b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553811c1d0 con 0x7f553809de50 2023-09-30T22:19:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f553da1b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f553811dd60 con 0x7f553809e760 2023-09-30T22:19:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f5536ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809de50 0x7f553811a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f5536ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809de50 0x7f553811a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55534/0 (socket says 172.21.15.89:55534) 2023-09-30T22:19:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f5536ffd700 1 -- 172.21.15.89:0/1778737569 learned_addr learned my addr 172.21.15.89:0/1778737569 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f55367fc700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f553809e760 0x7f553811aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.841+0000 7f55377fe700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55381338a0 0x7f553811afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f55367fc700 1 -- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809de50 msgr2=0x7f553811a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f55367fc700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809de50 0x7f553811a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f55367fc700 1 -- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55381338a0 msgr2=0x7f553811afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f55367fc700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55381338a0 0x7f553811afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f55367fc700 1 -- 172.21.15.89:0/1778737569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5528009550 con 0x7f553809e760 2023-09-30T22:19:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f5536ffd700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809de50 0x7f553811a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f55377fe700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55381338a0 0x7f553811afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f55367fc700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f553809e760 0x7f553811aa60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f552c00e870 tx=0x7f552c00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f551ffff700 1 -- 172.21.15.89:0/1778737569 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c013680 con 0x7f553809e760 2023-09-30T22:19:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f551ffff700 1 -- 172.21.15.89:0/1778737569 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552c0137e0 con 0x7f553809e760 2023-09-30T22:19:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5530009a70 con 0x7f553809e760 2023-09-30T22:19:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f551ffff700 1 -- 172.21.15.89:0/1778737569 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c013b70 con 0x7f553809e760 2023-09-30T22:19:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.842+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553811d380 con 0x7f553809e760 2023-09-30T22:19:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.843+0000 7f551ffff700 1 -- 172.21.15.89:0/1778737569 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f552c013cd0 con 0x7f553809e760 2023-09-30T22:19:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.843+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5538074ea0 con 0x7f553809e760 2023-09-30T22:19:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.843+0000 7f551ffff700 1 -- 172.21.15.89:0/1778737569 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f552c03f8b0 con 0x7f553809e760 2023-09-30T22:19:01.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.852+0000 7f551ffff700 1 -- 172.21.15.89:0/1778737569 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f552c011b80 con 0x7f553809e760 2023-09-30T22:19:01.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.909+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd8ac06b9e0 con 0x7fd8ac130e50 2023-09-30T22:19:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.909+0000 7fd8a97fa700 1 -- 172.21.15.89:0/622823146 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd8a4007370 con 0x7fd8ac130e50 2023-09-30T22:19:01.910 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.911+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8ac130e50 msgr2=0x7fd8ac1c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.911+0000 7fd8b2ef8700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8ac130e50 0x7fd8ac1c8ff0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b9a0 tx=0x7fd8a400bcb0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.911+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 shutdown_connections 2023-09-30T22:19:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.911+0000 7fd8b2ef8700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8ac130e50 0x7fd8ac1c8ff0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.911+0000 7fd8b2ef8700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8ac06fb20 0x7fd8ac1c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.911+0000 7fd8b2ef8700 1 --2- 172.21.15.89:0/622823146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8ac130540 0x7fd8ac1c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.911+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 >> 172.21.15.89:0/622823146 conn(0x7fd8ac0a2aa0 msgr2=0x7fd8ac1b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.911+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 shutdown_connections 2023-09-30T22:19:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.912+0000 7fd8b2ef8700 1 -- 172.21.15.89:0/622823146 wait complete. 2023-09-30T22:19:01.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:01.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.974+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f169c09ecc0 con 0x7f169c130540 2023-09-30T22:19:01.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.975+0000 7f168affd700 1 -- 172.21.15.89:0/3635404567 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f168c044060 con 0x7f169c130540 2023-09-30T22:19:01.976 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130540 msgr2=0x7f169c12d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130540 0x7f169c12d1e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f168c00c450 tx=0x7f168c00c760 comp rx=0 tx=0).stop 2023-09-30T22:19:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 shutdown_connections 2023-09-30T22:19:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169c135e60 0x7f169c139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f169c130540 0x7f169c12d1e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 --2- 172.21.15.89:0/3635404567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169c133540 0x7f169c12d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 >> 172.21.15.89:0/3635404567 conn(0x7f169c0a2aa0 msgr2=0x7f169c131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 shutdown_connections 2023-09-30T22:19:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.977+0000 7f16a08e6700 1 -- 172.21.15.89:0/3635404567 wait complete. 2023-09-30T22:19:01.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:01.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.996+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb08411d9f0 con 0x7fb08412cf60 2023-09-30T22:19:01.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.996+0000 7fb0817fa700 1 -- 172.21.15.89:0/3228388691 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb078010310 con 0x7fb08412cf60 2023-09-30T22:19:01.997 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412cf60 msgr2=0x7fb0841c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 --2- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412cf60 0x7fb0841c3000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e810 tx=0x7fb07800eb20 comp rx=0 tx=0).stop 2023-09-30T22:19:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 shutdown_connections 2023-09-30T22:19:01.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 --2- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb084130260 0x7fb0841c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:02.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 --2- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb08412cf60 0x7fb0841c3000 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:02.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 --2- 172.21.15.89:0/3228388691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb08412f8b0 0x7fb0841c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:02.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 >> 172.21.15.89:0/3228388691 conn(0x7fb0840a2aa0 msgr2=0x7fb08409e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:02.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 shutdown_connections 2023-09-30T22:19:02.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:01.998+0000 7fb08ae6c700 1 -- 172.21.15.89:0/3228388691 wait complete. 2023-09-30T22:19:02.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:02.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.045+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f553806b9e0 con 0x7f553809e760 2023-09-30T22:19:02.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.045+0000 7f551ffff700 1 -- 172.21.15.89:0/1778737569 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f552c017070 con 0x7f553809e760 2023-09-30T22:19:02.046 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.047+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f553809e760 msgr2=0x7f553811aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:02.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.047+0000 7f553da1b700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f553809e760 0x7f553811aa60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f552c00e870 tx=0x7f552c00eb80 comp rx=0 tx=0).stop 2023-09-30T22:19:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.048+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 shutdown_connections 2023-09-30T22:19:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.048+0000 7f553da1b700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55381338a0 0x7f553811afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.048+0000 7f553da1b700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f553809de50 0x7f553811a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.048+0000 7f553da1b700 1 --2- 172.21.15.89:0/1778737569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f553809e760 0x7f553811aa60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.048+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 >> 172.21.15.89:0/1778737569 conn(0x7f55380a2aa0 msgr2=0x7f5538130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.048+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 shutdown_connections 2023-09-30T22:19:02.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:02.048+0000 7f553da1b700 1 -- 172.21.15.89:0/1778737569 wait complete. 2023-09-30T22:19:02.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:02.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:02.991 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:03.011 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:03.061 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.071+0000 7f44f9ffe700 1 Processor -- start 2023-09-30T22:19:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.071+0000 7f44f9ffe700 1 -- start start 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.071+0000 7f44f9ffe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.071+0000 7f44f9ffe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409de50 0x7f44f409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.071+0000 7f44f9ffe700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f406ef30 con 0x7f44f409e760 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.071+0000 7f44f9ffe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f406fef0 con 0x7f44f409de50 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.071+0000 7f44f9ffe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44f41334b0 con 0x7f44f41338a0 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f44f409e760 0x7f44f413a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50082/0 (socket says 172.21.15.89:50082) 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f3fff700 1 -- 172.21.15.89:0/3801574261 learned_addr learned my addr 172.21.15.89:0/3801574261 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54310/0 (socket says 172.21.15.89:54310) 2023-09-30T22:19:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f2ffd700 1 --2- 172.21.15.89:0/3801574261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409de50 0x7f44f409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f37fe700 1 -- 172.21.15.89:0/3801574261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409de50 msgr2=0x7f44f409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f37fe700 1 --2- 172.21.15.89:0/3801574261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409de50 0x7f44f409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f37fe700 1 -- 172.21.15.89:0/3801574261 >> v1:172.21.15.89:6789/0 conn(0x7f44f409e760 legacy=0x7f44f413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f37fe700 1 -- 172.21.15.89:0/3801574261 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f406fc30 con 0x7f44f41338a0 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f37fe700 1 --2- 172.21.15.89:0/3801574261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f4133c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f44e40098a0 tx=0x7f44e4009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df90813791e87800 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f27fc700 1 -- 172.21.15.89:0/3801574261 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4011830 con 0x7f44f41338a0 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f27fc700 1 -- 172.21.15.89:0/3801574261 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4011990 con 0x7f44f41338a0 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.072+0000 7f44f27fc700 1 -- 172.21.15.89:0/3801574261 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4011d20 con 0x7f44f41338a0 2023-09-30T22:19:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.073+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3801574261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 msgr2=0x7f44f4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.073+0000 7f44f9ffe700 1 --2- 172.21.15.89:0/3801574261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f4133c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f44e40098a0 tx=0x7f44e4009bb0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.073+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3801574261 shutdown_connections 2023-09-30T22:19:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.073+0000 7f44f9ffe700 1 --2- 172.21.15.89:0/3801574261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f4133c70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.073+0000 7f44f9ffe700 1 --2- 172.21.15.89:0/3801574261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409de50 0x7f44f409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.073+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3801574261 >> 172.21.15.89:0/3801574261 conn(0x7f44f40a2aa0 msgr2=0x7f44f40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.074+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3801574261 shutdown_connections 2023-09-30T22:19:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.074+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3801574261 wait complete. 2023-09-30T22:19:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.075+0000 7f44f9ffe700 1 Processor -- start 2023-09-30T22:19:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.075+0000 7f44f9ffe700 1 -- start start 2023-09-30T22:19:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f9ffe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44f409de50 0x7f44f411ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f9ffe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409e760 0x7f44f411f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f9ffe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f411f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f9ffe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f411ff30 con 0x7f44f409de50 2023-09-30T22:19:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f9ffe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f412f660 con 0x7f44f409e760 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f9ffe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4122050 con 0x7f44f41338a0 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f411f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f411f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54324/0 (socket says 172.21.15.89:54324) 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f3fff700 1 -- 172.21.15.89:0/3908505491 learned_addr learned my addr 172.21.15.89:0/3908505491 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f37fe700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44f409de50 0x7f44f411ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f2ffd700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409e760 0x7f44f411f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f2ffd700 1 -- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 msgr2=0x7f44f411f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f2ffd700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f411f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f2ffd700 1 -- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44f409de50 msgr2=0x7f44f411ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f2ffd700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44f409de50 0x7f44f411ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f2ffd700 1 -- 172.21.15.89:0/3908505491 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40094a0 con 0x7f44f409e760 2023-09-30T22:19:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f3fff700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f411f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.076+0000 7f44f37fe700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44f409de50 0x7f44f411ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.077+0000 7f44f2ffd700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409e760 0x7f44f411f010 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44e800e840 tx=0x7f44e800eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.077+0000 7f44dbfff700 1 -- 172.21.15.89:0/3908505491 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8013680 con 0x7f44f409e760 2023-09-30T22:19:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.077+0000 7f44dbfff700 1 -- 172.21.15.89:0/3908505491 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e80137e0 con 0x7f44f409e760 2023-09-30T22:19:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.077+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f41225f0 con 0x7f44f409e760 2023-09-30T22:19:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.077+0000 7f44dbfff700 1 -- 172.21.15.89:0/3908505491 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8013b50 con 0x7f44f409e760 2023-09-30T22:19:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.077+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f406f640 con 0x7f44f409e760 2023-09-30T22:19:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.078+0000 7f44dbfff700 1 -- 172.21.15.89:0/3908505491 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44e8013cb0 con 0x7f44f409e760 2023-09-30T22:19:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.078+0000 7f44dbfff700 1 -- 172.21.15.89:0/3908505491 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f44e803f5b0 con 0x7f44f409e760 2023-09-30T22:19:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.078+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d40052e0 con 0x7f44f409e760 2023-09-30T22:19:03.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.087+0000 7f44dbfff700 1 -- 172.21.15.89:0/3908505491 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44e8017020 con 0x7f44f409e760 2023-09-30T22:19:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.128+0000 7f33c4a7b700 1 Processor -- start 2023-09-30T22:19:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.128+0000 7f33c4a7b700 1 -- start start 2023-09-30T22:19:03.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.129+0000 7f33c4a7b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f33c0131810 0x7f33c0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.129+0000 7f33c4a7b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c009a580 0x7f33c0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.129+0000 7f33c4a7b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c01313c0 con 0x7f33c0099a40 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.129+0000 7f33c4a7b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c006e8c0 con 0x7f33c0131810 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.129+0000 7f33c4a7b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0071d20 con 0x7f33c009a580 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.129+0000 7f33be59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f33c0131810 0x7f33c0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.129+0000 7f33be59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f33c0131810 0x7f33c0131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42494/0 (socket says 172.21.15.89:42494) 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.129+0000 7f33be59c700 1 -- 172.21.15.89:0/2288754965 learned_addr learned my addr 172.21.15.89:0/2288754965 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.130+0000 7f33be59c700 1 -- 172.21.15.89:0/2288754965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c009a580 msgr2=0x7f33c0139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.130+0000 7f33be59c700 1 --2- 172.21.15.89:0/2288754965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c009a580 0x7f33c0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.130+0000 7f33be59c700 1 -- 172.21.15.89:0/2288754965 >> v1:172.21.15.89:6789/0 conn(0x7f33c0099a40 legacy=0x7f33c0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.130+0000 7f33be59c700 1 -- 172.21.15.89:0/2288754965 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c00721a0 con 0x7f33c0131810 2023-09-30T22:19:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.130+0000 7f33be59c700 1 --2- 172.21.15.89:0/2288754965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c0131810 0x7f33c0131be0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f33b00098f0 tx=0x7f33b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5468160ba95f9f00 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.130+0000 7f33bd59a700 1 -- 172.21.15.89:0/2288754965 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000b8a0 con 0x7f33c0131810 2023-09-30T22:19:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.130+0000 7f33bd59a700 1 -- 172.21.15.89:0/2288754965 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b000ba00 con 0x7f33c0131810 2023-09-30T22:19:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.130+0000 7f33bd59a700 1 -- 172.21.15.89:0/2288754965 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000bd90 con 0x7f33c0131810 2023-09-30T22:19:03.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.131+0000 7f33c4a7b700 1 -- 172.21.15.89:0/2288754965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c0131810 msgr2=0x7f33c0131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.131+0000 7f33c4a7b700 1 --2- 172.21.15.89:0/2288754965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c0131810 0x7f33c0131be0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f33b00098f0 tx=0x7f33b0009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:03.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.131+0000 7f33c4a7b700 1 -- 172.21.15.89:0/2288754965 shutdown_connections 2023-09-30T22:19:03.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.131+0000 7f33c4a7b700 1 --2- 172.21.15.89:0/2288754965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c0131810 0x7f33c0131be0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.131+0000 7f33c4a7b700 1 --2- 172.21.15.89:0/2288754965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c009a580 0x7f33c0139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.131+0000 7f33c4a7b700 1 -- 172.21.15.89:0/2288754965 >> 172.21.15.89:0/2288754965 conn(0x7f33c00a2aa0 msgr2=0x7f33c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:03.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.131+0000 7f33c4a7b700 1 -- 172.21.15.89:0/2288754965 shutdown_connections 2023-09-30T22:19:03.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.131+0000 7f33c4a7b700 1 -- 172.21.15.89:0/2288754965 wait complete. 2023-09-30T22:19:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.132+0000 7f33c4a7b700 1 Processor -- start 2023-09-30T22:19:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.132+0000 7f33c4a7b700 1 -- start start 2023-09-30T22:19:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33c4a7b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c0099a40 0x7f33c011b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33c4a7b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c009a580 0x7f33c011be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33c4a7b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33c0131810 0x7f33c011c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33c4a7b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c011f140 con 0x7f33c009a580 2023-09-30T22:19:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33c4a7b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c011cd50 con 0x7f33c0131810 2023-09-30T22:19:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33c4a7b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33c011d4b0 con 0x7f33c0099a40 2023-09-30T22:19:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33bed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33c0131810 0x7f33c011c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33be59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c0099a40 0x7f33c011b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33be59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c0099a40 0x7f33c011b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54338/0 (socket says 172.21.15.89:54338) 2023-09-30T22:19:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33bdd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c009a580 0x7f33c011be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33be59c700 1 -- 172.21.15.89:0/522045444 learned_addr learned my addr 172.21.15.89:0/522045444 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.133+0000 7f33bed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33c0131810 0x7f33c011c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55548/0 (socket says 172.21.15.89:55548) 2023-09-30T22:19:03.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33bdd9b700 1 -- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c0099a40 msgr2=0x7f33c011b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33bdd9b700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c0099a40 0x7f33c011b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33bdd9b700 1 -- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33c0131810 msgr2=0x7f33c011c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33bdd9b700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33c0131810 0x7f33c011c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33bdd9b700 1 -- 172.21.15.89:0/522045444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0009550 con 0x7f33c009a580 2023-09-30T22:19:03.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33be59c700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c0099a40 0x7f33c011b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:03.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33bed9d700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33c0131810 0x7f33c011c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:03.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33bdd9b700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c009a580 0x7f33c011be30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f33b4000f60 tx=0x7f33b400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:03.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33aeffd700 1 -- 172.21.15.89:0/522045444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b40050d0 con 0x7f33c009a580 2023-09-30T22:19:03.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33aeffd700 1 -- 172.21.15.89:0/522045444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b4005230 con 0x7f33c009a580 2023-09-30T22:19:03.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c00a1bb0 con 0x7f33c009a580 2023-09-30T22:19:03.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33aeffd700 1 -- 172.21.15.89:0/522045444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4003680 con 0x7f33c009a580 2023-09-30T22:19:03.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.134+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c006eb60 con 0x7f33c009a580 2023-09-30T22:19:03.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.135+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a40052e0 con 0x7f33c009a580 2023-09-30T22:19:03.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.135+0000 7f33aeffd700 1 -- 172.21.15.89:0/522045444 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f33b40037e0 con 0x7f33c009a580 2023-09-30T22:19:03.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.136+0000 7f33aeffd700 1 -- 172.21.15.89:0/522045444 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f33b403e910 con 0x7f33c009a580 2023-09-30T22:19:03.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.140+0000 7f33aeffd700 1 -- 172.21.15.89:0/522045444 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f33b4011590 con 0x7f33c009a580 2023-09-30T22:19:03.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.189+0000 7f99f814b700 1 Processor -- start 2023-09-30T22:19:03.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.189+0000 7f99f814b700 1 -- start start 2023-09-30T22:19:03.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f814b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99f006e570 0x7f99f006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f814b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f009eca0 0x7f99f006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f814b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f0070ee0 con 0x7f99f0134610 2023-09-30T22:19:03.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f814b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f0071480 con 0x7f99f006e570 2023-09-30T22:19:03.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f814b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f006f3c0 con 0x7f99f009eca0 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f56e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f009eca0 0x7f99f006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f5ee7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99f006e570 0x7f99f006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f56e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f009eca0 0x7f99f006ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55552/0 (socket says 172.21.15.89:55552) 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f56e6700 1 -- 172.21.15.89:0/2955938690 learned_addr learned my addr 172.21.15.89:0/2955938690 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f5ee7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99f006e570 0x7f99f006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42508/0 (socket says 172.21.15.89:42508) 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f56e6700 1 -- 172.21.15.89:0/2955938690 >> v2:172.21.15.89:3300/0 conn(0x7f99f006e570 msgr2=0x7f99f006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f56e6700 1 --2- 172.21.15.89:0/2955938690 >> v2:172.21.15.89:3300/0 conn(0x7f99f006e570 0x7f99f006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f56e6700 1 -- 172.21.15.89:0/2955938690 >> v1:172.21.15.89:6789/0 conn(0x7f99f0134610 legacy=0x7f99f0134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.190+0000 7f99f56e6700 1 -- 172.21.15.89:0/2955938690 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0071b90 con 0x7f99f009eca0 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f56e6700 1 --2- 172.21.15.89:0/2955938690 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f009eca0 0x7f99f006ee80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098f0 tx=0x7f99e0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f95083c2469577 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f4ee5700 1 -- 172.21.15.89:0/2955938690 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0011830 con 0x7f99f009eca0 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f4ee5700 1 -- 172.21.15.89:0/2955938690 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0011990 con 0x7f99f009eca0 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f4ee5700 1 -- 172.21.15.89:0/2955938690 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0011d20 con 0x7f99f009eca0 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f814b700 1 -- 172.21.15.89:0/2955938690 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f009eca0 msgr2=0x7f99f006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f814b700 1 --2- 172.21.15.89:0/2955938690 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f009eca0 0x7f99f006ee80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098f0 tx=0x7f99e0009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f814b700 1 -- 172.21.15.89:0/2955938690 shutdown_connections 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f814b700 1 --2- 172.21.15.89:0/2955938690 >> v2:172.21.15.89:3300/0 conn(0x7f99f006e570 0x7f99f006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f814b700 1 --2- 172.21.15.89:0/2955938690 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f009eca0 0x7f99f006ee80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.191+0000 7f99f814b700 1 -- 172.21.15.89:0/2955938690 >> 172.21.15.89:0/2955938690 conn(0x7f99f00a2aa0 msgr2=0x7f99f00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:03.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.192+0000 7f99f814b700 1 -- 172.21.15.89:0/2955938690 shutdown_connections 2023-09-30T22:19:03.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.192+0000 7f99f814b700 1 -- 172.21.15.89:0/2955938690 wait complete. 2023-09-30T22:19:03.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.192+0000 7f99f814b700 1 Processor -- start 2023-09-30T22:19:03.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.192+0000 7f99f814b700 1 -- start start 2023-09-30T22:19:03.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f814b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f006e570 0x7f99f0123250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f814b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99f0134610 0x7f99f0123790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f814b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99f0123cd0 0x7f99f01d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f814b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0126740 con 0x7f99f0123cd0 2023-09-30T22:19:03.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f814b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0126ce0 con 0x7f99f006e570 2023-09-30T22:19:03.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f814b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0124e70 con 0x7f99f0134610 2023-09-30T22:19:03.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99f0123cd0 0x7f99f01d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f5ee7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f006e570 0x7f99f0123250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99f0123cd0 0x7f99f01d2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42516/0 (socket says 172.21.15.89:42516) 2023-09-30T22:19:03.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f5ee7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f006e570 0x7f99f0123250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55566/0 (socket says 172.21.15.89:55566) 2023-09-30T22:19:03.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 -- 172.21.15.89:0/4106277815 learned_addr learned my addr 172.21.15.89:0/4106277815 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:03.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 -- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99f0134610 msgr2=0x7f99f0123790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 --2- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99f0134610 0x7f99f0123790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 -- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f006e570 msgr2=0x7f99f0123250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 --2- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f006e570 0x7f99f0123250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 -- 172.21.15.89:0/4106277815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0009550 con 0x7f99f0123cd0 2023-09-30T22:19:03.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f5ee7700 1 --2- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f006e570 0x7f99f0123250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:03.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.193+0000 7f99f66e8700 1 --2- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99f0123cd0 0x7f99f01d2a30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f99e800cc50 tx=0x7f99e800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.194+0000 7f99e67fc700 1 -- 172.21.15.89:0/4106277815 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8003c00 con 0x7f99f0123cd0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.194+0000 7f99e67fc700 1 -- 172.21.15.89:0/4106277815 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8003d60 con 0x7f99f0123cd0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.194+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f00a2320 con 0x7f99f0123cd0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.194+0000 7f99e67fc700 1 -- 172.21.15.89:0/4106277815 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e800ab60 con 0x7f99f0123cd0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.194+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f0070670 con 0x7f99f0123cd0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.195+0000 7f99e67fc700 1 -- 172.21.15.89:0/4106277815 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99e800acc0 con 0x7f99f0123cd0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.195+0000 7f99e67fc700 1 -- 172.21.15.89:0/4106277815 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f99e803f6b0 con 0x7f99f0123cd0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.195+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0074ea0 con 0x7f99f0123cd0 2023-09-30T22:19:03.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.200+0000 7f99e67fc700 1 -- 172.21.15.89:0/4106277815 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99e80198f0 con 0x7f99f0123cd0 2023-09-30T22:19:03.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.204+0000 7f2895642700 1 Processor -- start 2023-09-30T22:19:03.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.204+0000 7f2895642700 1 -- start start 2023-09-30T22:19:03.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f2895642700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009e1d0 0x7f2890138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f2895642700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2890139170 0x7f289013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f2895642700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289009b120 con 0x7f289009d6f0 2023-09-30T22:19:03.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f2895642700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289006b460 con 0x7f2890139170 2023-09-30T22:19:03.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f2895642700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289009a7e0 con 0x7f289009e1d0 2023-09-30T22:19:03.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f288e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009e1d0 0x7f2890138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f288effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f289009d6f0 0x7f289009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50098/0 (socket says 172.21.15.89:50098) 2023-09-30T22:19:03.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f288effd700 1 -- 172.21.15.89:0/4055871668 learned_addr learned my addr 172.21.15.89:0/4055871668 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:03.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f288e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009e1d0 0x7f2890138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55574/0 (socket says 172.21.15.89:55574) 2023-09-30T22:19:03.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.205+0000 7f288f7fe700 1 --2- 172.21.15.89:0/4055871668 >> v2:172.21.15.89:3300/0 conn(0x7f2890139170 0x7f289013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288dffb700 1 -- 172.21.15.89:0/4055871668 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1662177828 0 0) 0x7f289009b120 con 0x7f289009d6f0 2023-09-30T22:19:03.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288dffb700 1 -- 172.21.15.89:0/4055871668 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28780035b0 con 0x7f289009d6f0 2023-09-30T22:19:03.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288e7fc700 1 -- 172.21.15.89:0/4055871668 >> v2:172.21.15.89:3300/0 conn(0x7f2890139170 msgr2=0x7f289013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:03.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288e7fc700 1 --2- 172.21.15.89:0/4055871668 >> v2:172.21.15.89:3300/0 conn(0x7f2890139170 0x7f289013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288e7fc700 1 -- 172.21.15.89:0/4055871668 >> v1:172.21.15.89:6789/0 conn(0x7f289009d6f0 legacy=0x7f289009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288e7fc700 1 -- 172.21.15.89:0/4055871668 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289009a500 con 0x7f289009e1d0 2023-09-30T22:19:03.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288f7fe700 1 --2- 172.21.15.89:0/4055871668 >> v2:172.21.15.89:3300/0 conn(0x7f2890139170 0x7f289013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:03.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288e7fc700 1 --2- 172.21.15.89:0/4055871668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009e1d0 0x7f2890138c30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2884009870 tx=0x7f2884009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a4d70853b08a3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:03.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288dffb700 1 -- 172.21.15.89:0/4055871668 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884011830 con 0x7f289009e1d0 2023-09-30T22:19:03.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288dffb700 1 -- 172.21.15.89:0/4055871668 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2884011990 con 0x7f289009e1d0 2023-09-30T22:19:03.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.206+0000 7f288dffb700 1 -- 172.21.15.89:0/4055871668 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884011d00 con 0x7f289009e1d0 2023-09-30T22:19:03.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.207+0000 7f2895642700 1 -- 172.21.15.89:0/4055871668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009e1d0 msgr2=0x7f2890138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.207+0000 7f2895642700 1 --2- 172.21.15.89:0/4055871668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009e1d0 0x7f2890138c30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2884009870 tx=0x7f2884009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:03.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.207+0000 7f2895642700 1 -- 172.21.15.89:0/4055871668 shutdown_connections 2023-09-30T22:19:03.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.207+0000 7f2895642700 1 --2- 172.21.15.89:0/4055871668 >> v2:172.21.15.89:3300/0 conn(0x7f2890139170 0x7f289013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.207+0000 7f2895642700 1 --2- 172.21.15.89:0/4055871668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009e1d0 0x7f2890138c30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.207+0000 7f2895642700 1 -- 172.21.15.89:0/4055871668 >> 172.21.15.89:0/4055871668 conn(0x7f2890066f50 msgr2=0x7f289009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.207+0000 7f2895642700 1 -- 172.21.15.89:0/4055871668 shutdown_connections 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.207+0000 7f2895642700 1 -- 172.21.15.89:0/4055871668 wait complete. 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.208+0000 7f2895642700 1 Processor -- start 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.208+0000 7f2895642700 1 -- start start 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f2895642700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009d6f0 0x7f2890120690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f2895642700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f289009e1d0 0x7f2890120bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f2895642700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2890139170 0x7f2890121110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f2895642700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890135fc0 con 0x7f2890139170 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f2895642700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2890136580 con 0x7f289009d6f0 2023-09-30T22:19:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f2895642700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28901241b0 con 0x7f289009e1d0 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009d6f0 0x7f2890120690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009d6f0 0x7f2890120690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55590/0 (socket says 172.21.15.89:55590) 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 -- 172.21.15.89:0/2510812741 learned_addr learned my addr 172.21.15.89:0/2510812741 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288f7fe700 1 --2- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2890139170 0x7f2890121110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 -- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f289009e1d0 msgr2=0x7f2890120bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 --2- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f289009e1d0 0x7f2890120bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 -- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2890139170 msgr2=0x7f2890121110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 --2- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2890139170 0x7f2890121110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 -- 172.21.15.89:0/2510812741 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2888009a70 con 0x7f289009d6f0 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.209+0000 7f288effd700 1 --2- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009d6f0 0x7f2890120690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f288000c700 tx=0x7f288000cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.210+0000 7f28777fe700 1 -- 172.21.15.89:0/2510812741 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880005d70 con 0x7f289009d6f0 2023-09-30T22:19:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.210+0000 7f28777fe700 1 -- 172.21.15.89:0/2510812741 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800070d0 con 0x7f289009d6f0 2023-09-30T22:19:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.210+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28840094a0 con 0x7f289009d6f0 2023-09-30T22:19:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.210+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289009c710 con 0x7f289009d6f0 2023-09-30T22:19:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.210+0000 7f28777fe700 1 -- 172.21.15.89:0/2510812741 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880007440 con 0x7f289009d6f0 2023-09-30T22:19:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.211+0000 7f28777fe700 1 -- 172.21.15.89:0/2510812741 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28800075a0 con 0x7f289009d6f0 2023-09-30T22:19:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.211+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2890074ea0 con 0x7f289009d6f0 2023-09-30T22:19:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.211+0000 7f28777fe700 1 -- 172.21.15.89:0/2510812741 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2880040bd0 con 0x7f289009d6f0 2023-09-30T22:19:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.220+0000 7f28777fe700 1 -- 172.21.15.89:0/2510812741 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2880044060 con 0x7f289009d6f0 2023-09-30T22:19:03.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.307+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f44d4005150 con 0x7f44f409e760 2023-09-30T22:19:03.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.308+0000 7f44dbfff700 1 -- 172.21.15.89:0/3908505491 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44e801bde0 con 0x7f44f409e760 2023-09-30T22:19:03.309 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:03.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409e760 msgr2=0x7f44f411f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409e760 0x7f44f411f010 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44e800e840 tx=0x7f44e800eb50 comp rx=0 tx=0).stop 2023-09-30T22:19:03.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 shutdown_connections 2023-09-30T22:19:03.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44f41338a0 0x7f44f411f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44f409de50 0x7f44f411ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 --2- 172.21.15.89:0/3908505491 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44f409e760 0x7f44f411f010 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 >> 172.21.15.89:0/3908505491 conn(0x7f44f40a2aa0 msgr2=0x7f44f409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 shutdown_connections 2023-09-30T22:19:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.310+0000 7f44f9ffe700 1 -- 172.21.15.89:0/3908505491 wait complete. 2023-09-30T22:19:03.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:03.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.349+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f33a4005590 con 0x7f33c009a580 2023-09-30T22:19:03.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.349+0000 7f33aeffd700 1 -- 172.21.15.89:0/522045444 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33b4017030 con 0x7f33c009a580 2023-09-30T22:19:03.350 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:03.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c009a580 msgr2=0x7f33c011be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c009a580 0x7f33c011be30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f33b4000f60 tx=0x7f33b400c730 comp rx=0 tx=0).stop 2023-09-30T22:19:03.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 shutdown_connections 2023-09-30T22:19:03.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33c0131810 0x7f33c011c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33c0099a40 0x7f33c011b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 --2- 172.21.15.89:0/522045444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33c009a580 0x7f33c011be30 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 >> 172.21.15.89:0/522045444 conn(0x7f33c00a2aa0 msgr2=0x7f33c009d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:03.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 shutdown_connections 2023-09-30T22:19:03.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.352+0000 7f33c4a7b700 1 -- 172.21.15.89:0/522045444 wait complete. 2023-09-30T22:19:03.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.418+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f99f0071100 con 0x7f99f0123cd0 2023-09-30T22:19:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.418+0000 7f99e67fc700 1 -- 172.21.15.89:0/4106277815 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99e8019300 con 0x7f99f0123cd0 2023-09-30T22:19:03.419 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.420+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f289009f420 con 0x7f289009d6f0 2023-09-30T22:19:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.420+0000 7f28777fe700 1 -- 172.21.15.89:0/2510812741 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2880044070 con 0x7f289009d6f0 2023-09-30T22:19:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.420+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99f0123cd0 msgr2=0x7f99f01d2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.420+0000 7f99f814b700 1 --2- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99f0123cd0 0x7f99f01d2a30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f99e800cc50 tx=0x7f99e800cf60 comp rx=0 tx=0).stop 2023-09-30T22:19:03.422 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.421+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 shutdown_connections 2023-09-30T22:19:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.421+0000 7f99f814b700 1 --2- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99f0123cd0 0x7f99f01d2a30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.421+0000 7f99f814b700 1 --2- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99f006e570 0x7f99f0123250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.421+0000 7f99f814b700 1 --2- 172.21.15.89:0/4106277815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99f0134610 0x7f99f0123790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.421+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 >> 172.21.15.89:0/4106277815 conn(0x7f99f00a2aa0 msgr2=0x7f99f00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.421+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 shutdown_connections 2023-09-30T22:19:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.421+0000 7f99f814b700 1 -- 172.21.15.89:0/4106277815 wait complete. 2023-09-30T22:19:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.423+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009d6f0 msgr2=0x7f2890120690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.423+0000 7f2895642700 1 --2- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009d6f0 0x7f2890120690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f288000c700 tx=0x7f288000cac0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.423+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 shutdown_connections 2023-09-30T22:19:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.423+0000 7f2895642700 1 --2- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2890139170 0x7f2890121110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.423+0000 7f2895642700 1 --2- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009d6f0 0x7f2890120690 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.423+0000 7f2895642700 1 --2- 172.21.15.89:0/2510812741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f289009e1d0 0x7f2890120bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.423+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 >> 172.21.15.89:0/2510812741 conn(0x7f2890066f50 msgr2=0x7f28900a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:03.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.424+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 shutdown_connections 2023-09-30T22:19:03.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:03.424+0000 7f2895642700 1 -- 172.21.15.89:0/2510812741 wait complete. 2023-09-30T22:19:03.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:03.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:04.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:04.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:04.436 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:04.438 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:04.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.470+0000 7f5ce101a700 1 Processor -- start 2023-09-30T22:19:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.470+0000 7f5ce101a700 1 -- start start 2023-09-30T22:19:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.471+0000 7f5ce101a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5cdc09d9f0 0x7f5cdc136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.471+0000 7f5ce101a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc136a70 0x7f5cdc138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.471+0000 7f5ce101a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cdc09e130 con 0x7f5cdc1316c0 2023-09-30T22:19:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.471+0000 7f5ce101a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cdc1321a0 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.471+0000 7f5ce101a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc1312b0 con 0x7f5cdc136a70 2023-09-30T22:19:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cda59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5cdc09d9f0 0x7f5cdc136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cda59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5cdc09d9f0 0x7f5cdc136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42542/0 (socket says 172.21.15.89:42542) 2023-09-30T22:19:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cda59c700 1 -- 172.21.15.89:0/2561248819 learned_addr learned my addr 172.21.15.89:0/2561248819 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cdb59e700 1 --2- 172.21.15.89:0/2561248819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc136a70 0x7f5cdc138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cdb59e700 1 -- 172.21.15.89:0/2561248819 >> v2:172.21.15.89:3300/0 conn(0x7f5cdc09d9f0 msgr2=0x7f5cdc136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cdb59e700 1 --2- 172.21.15.89:0/2561248819 >> v2:172.21.15.89:3300/0 conn(0x7f5cdc09d9f0 0x7f5cdc136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cdb59e700 1 -- 172.21.15.89:0/2561248819 >> v1:172.21.15.89:6789/0 conn(0x7f5cdc1316c0 legacy=0x7f5cdc131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cdb59e700 1 -- 172.21.15.89:0/2561248819 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc09f6e0 con 0x7f5cdc136a70 2023-09-30T22:19:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cda59c700 1 --2- 172.21.15.89:0/2561248819 >> v2:172.21.15.89:3300/0 conn(0x7f5cdc09d9f0 0x7f5cdc136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cdb59e700 1 --2- 172.21.15.89:0/2561248819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc136a70 0x7f5cdc138e50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009dc0 tx=0x7f5cd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=845a7af38f147831 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cd9d9b700 1 -- 172.21.15.89:0/2561248819 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400e070 con 0x7f5cdc136a70 2023-09-30T22:19:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cd9d9b700 1 -- 172.21.15.89:0/2561248819 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd400bae0 con 0x7f5cdc136a70 2023-09-30T22:19:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.472+0000 7f5cd9d9b700 1 -- 172.21.15.89:0/2561248819 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400be90 con 0x7f5cdc136a70 2023-09-30T22:19:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.473+0000 7f5ce101a700 1 -- 172.21.15.89:0/2561248819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc136a70 msgr2=0x7f5cdc138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.473+0000 7f5ce101a700 1 --2- 172.21.15.89:0/2561248819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc136a70 0x7f5cdc138e50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009dc0 tx=0x7f5cd400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.473+0000 7f5ce101a700 1 -- 172.21.15.89:0/2561248819 shutdown_connections 2023-09-30T22:19:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.473+0000 7f5ce101a700 1 --2- 172.21.15.89:0/2561248819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc136a70 0x7f5cdc138e50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.473+0000 7f5ce101a700 1 --2- 172.21.15.89:0/2561248819 >> v2:172.21.15.89:3300/0 conn(0x7f5cdc09d9f0 0x7f5cdc136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.473+0000 7f5ce101a700 1 -- 172.21.15.89:0/2561248819 >> 172.21.15.89:0/2561248819 conn(0x7f5cdc066f50 msgr2=0x7f5cdc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.474+0000 7f5ce101a700 1 -- 172.21.15.89:0/2561248819 shutdown_connections 2023-09-30T22:19:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.474+0000 7f5ce101a700 1 -- 172.21.15.89:0/2561248819 wait complete. 2023-09-30T22:19:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5ce101a700 1 Processor -- start 2023-09-30T22:19:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5ce101a700 1 -- start start 2023-09-30T22:19:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5ce101a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cdc09d9f0 0x7f5cdc12ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5ce101a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc1316c0 0x7f5cdc12b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5ce101a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cdc136a70 0x7f5cdc09a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5ce101a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc06e8e0 con 0x7f5cdc136a70 2023-09-30T22:19:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5ce101a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc1343e0 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5ce101a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc134910 con 0x7f5cdc1316c0 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5cdad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cdc09d9f0 0x7f5cdc12ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5cdad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cdc09d9f0 0x7f5cdc12ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55606/0 (socket says 172.21.15.89:55606) 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5cdad9d700 1 -- 172.21.15.89:0/3368352602 learned_addr learned my addr 172.21.15.89:0/3368352602 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.475+0000 7f5cda59c700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc1316c0 0x7f5cdc12b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cdb59e700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cdc136a70 0x7f5cdc09a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cdad9d700 1 -- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc1316c0 msgr2=0x7f5cdc12b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cdad9d700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc1316c0 0x7f5cdc12b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cdad9d700 1 -- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cdc136a70 msgr2=0x7f5cdc09a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cdad9d700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cdc136a70 0x7f5cdc09a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cdad9d700 1 -- 172.21.15.89:0/3368352602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0009550 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cdad9d700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cdc09d9f0 0x7f5cdc12ab90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c510 tx=0x7f5ccc00c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cc37fe700 1 -- 172.21.15.89:0/3368352602 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0052d0 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cc37fe700 1 -- 172.21.15.89:0/3368352602 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc005430 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5cc37fe700 1 -- 172.21.15.89:0/3368352602 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0038c0 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd4009a70 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.476+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc12b610 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.478+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc074ea0 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.479+0000 7f5cc37fe700 1 -- 172.21.15.89:0/3368352602 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ccc003a20 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.479+0000 7f5cc37fe700 1 -- 172.21.15.89:0/3368352602 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ccc03ea00 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.488+0000 7f5cc37fe700 1 -- 172.21.15.89:0/3368352602 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ccc011590 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.511+0000 7f9884f9f700 1 Processor -- start 2023-09-30T22:19:04.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.511+0000 7f9884f9f700 1 -- start start 2023-09-30T22:19:04.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f9884f9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988009df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f9884f9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988012f8c0 0x7f988012fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f9884f9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9880130210 0x7f9880136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f9884f9f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988009d670 con 0x7f9880130210 2023-09-30T22:19:04.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f9884f9f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880099d80 con 0x7f988009dba0 2023-09-30T22:19:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f9884f9f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f988009b020 con 0x7f988012f8c0 2023-09-30T22:19:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988009df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988009df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55620/0 (socket says 172.21.15.89:55620) 2023-09-30T22:19:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987f7fe700 1 -- 172.21.15.89:0/2598858980 learned_addr learned my addr 172.21.15.89:0/2598858980 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987ffff700 1 --2- 172.21.15.89:0/2598858980 >> v2:172.21.15.89:3300/0 conn(0x7f9880130210 0x7f9880136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987effd700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988012f8c0 0x7f988012fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987ffff700 1 -- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988012f8c0 msgr2=0x7f988012fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987ffff700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988012f8c0 0x7f988012fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987ffff700 1 -- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 msgr2=0x7f988009df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987ffff700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988009df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987ffff700 1 -- 172.21.15.89:0/2598858980 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988009a440 con 0x7f9880130210 2023-09-30T22:19:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987f7fe700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988009df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.512+0000 7f987ffff700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9880130210 0x7f9880136c20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9878009d90 tx=0x7f987800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88763e81f46abc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.513+0000 7f987e7fc700 1 -- 172.21.15.89:0/2598858980 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987800e070 con 0x7f9880130210 2023-09-30T22:19:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.513+0000 7f987e7fc700 1 -- 172.21.15.89:0/2598858980 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987800bb20 con 0x7f9880130210 2023-09-30T22:19:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.513+0000 7f987e7fc700 1 -- 172.21.15.89:0/2598858980 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878012430 con 0x7f9880130210 2023-09-30T22:19:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 -- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9880130210 msgr2=0x7f9880136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9880130210 0x7f9880136c20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9878009d90 tx=0x7f987800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 -- 172.21.15.89:0/2598858980 shutdown_connections 2023-09-30T22:19:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9880130210 0x7f9880136c20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988009df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 --2- 172.21.15.89:0/2598858980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988012f8c0 0x7f988012fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 -- 172.21.15.89:0/2598858980 >> 172.21.15.89:0/2598858980 conn(0x7f98800a2540 msgr2=0x7f98800a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 -- 172.21.15.89:0/2598858980 shutdown_connections 2023-09-30T22:19:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.514+0000 7f9884f9f700 1 -- 172.21.15.89:0/2598858980 wait complete. 2023-09-30T22:19:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.515+0000 7f9884f9f700 1 Processor -- start 2023-09-30T22:19:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.515+0000 7f9884f9f700 1 -- start start 2023-09-30T22:19:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f9884f9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988012a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f9884f9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988012f8c0 0x7f988012a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f9884f9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9880130210 0x7f98801c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f9884f9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988012e9c0 con 0x7f988012f8c0 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f9884f9f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988009a920 con 0x7f988009dba0 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f9884f9f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9880129ea0 con 0x7f9880130210 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988012f8c0 0x7f988012a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988012f8c0 0x7f988012a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42582/0 (socket says 172.21.15.89:42582) 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987effd700 1 -- 172.21.15.89:0/786627982 learned_addr learned my addr 172.21.15.89:0/786627982 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987ffff700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9880130210 0x7f98801c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987f7fe700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988012a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987effd700 1 -- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9880130210 msgr2=0x7f98801c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987effd700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9880130210 0x7f98801c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987effd700 1 -- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 msgr2=0x7f988012a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987effd700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988012a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987effd700 1 -- 172.21.15.89:0/786627982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987000b7a0 con 0x7f988012f8c0 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.516+0000 7f987ffff700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9880130210 0x7f98801c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.517+0000 7f987effd700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988012f8c0 0x7f988012a670 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f987400e980 tx=0x7f987400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.517+0000 7f9867fff700 1 -- 172.21.15.89:0/786627982 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874009710 con 0x7f988012f8c0 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.517+0000 7f9867fff700 1 -- 172.21.15.89:0/786627982 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987400c690 con 0x7f988012f8c0 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.517+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9878009a70 con 0x7f988012f8c0 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.517+0000 7f9867fff700 1 -- 172.21.15.89:0/786627982 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987400ca20 con 0x7f988012f8c0 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.517+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988008b4e0 con 0x7f988012f8c0 2023-09-30T22:19:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.518+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988004f2a0 con 0x7f988012f8c0 2023-09-30T22:19:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.518+0000 7f9867fff700 1 -- 172.21.15.89:0/786627982 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f987400cb80 con 0x7f988012f8c0 2023-09-30T22:19:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.519+0000 7f9867fff700 1 -- 172.21.15.89:0/786627982 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f987403fe00 con 0x7f988012f8c0 2023-09-30T22:19:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.523+0000 7f9867fff700 1 -- 172.21.15.89:0/786627982 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9874013470 con 0x7f988012f8c0 2023-09-30T22:19:04.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.567+0000 7fc10a35f700 1 Processor -- start 2023-09-30T22:19:04.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.567+0000 7fc10a35f700 1 -- start start 2023-09-30T22:19:04.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc10a35f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc10406a770 0x7fc10406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc10a35f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc104131d10 0x7fc104139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc10a35f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10409ece0 con 0x7fc10406b080 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc10a35f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10409e760 con 0x7fc10406a770 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc10a35f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10409e040 con 0x7fc104131d10 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc1088fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc104131d10 0x7fc104139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc103fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc10406a770 0x7fc10406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc103fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc10406a770 0x7fc10406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42598/0 (socket says 172.21.15.89:42598) 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc103fff700 1 -- 172.21.15.89:0/2165665373 learned_addr learned my addr 172.21.15.89:0/2165665373 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.568+0000 7fc1088fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc104131d10 0x7fc104139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55642/0 (socket says 172.21.15.89:55642) 2023-09-30T22:19:04.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.569+0000 7fc103fff700 1 -- 172.21.15.89:0/2165665373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc104131d10 msgr2=0x7fc104139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.569+0000 7fc103fff700 1 --2- 172.21.15.89:0/2165665373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc104131d10 0x7fc104139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.569+0000 7fc103fff700 1 -- 172.21.15.89:0/2165665373 >> v1:172.21.15.89:6789/0 conn(0x7fc10406b080 legacy=0x7fc104131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.569+0000 7fc103fff700 1 -- 172.21.15.89:0/2165665373 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10409f630 con 0x7fc10406a770 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.569+0000 7fc103fff700 1 --2- 172.21.15.89:0/2165665373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 0x7fc10406ab40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40098f0 tx=0x7fc0f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80334f4ed06c673c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.569+0000 7fc102ffd700 1 -- 172.21.15.89:0/2165665373 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4011830 con 0x7fc10406a770 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.569+0000 7fc102ffd700 1 -- 172.21.15.89:0/2165665373 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f4011990 con 0x7fc10406a770 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.569+0000 7fc102ffd700 1 -- 172.21.15.89:0/2165665373 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4011d20 con 0x7fc10406a770 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.570+0000 7fc10a35f700 1 -- 172.21.15.89:0/2165665373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 msgr2=0x7fc10406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.570+0000 7fc10a35f700 1 --2- 172.21.15.89:0/2165665373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 0x7fc10406ab40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40098f0 tx=0x7fc0f4009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.570+0000 7fc10a35f700 1 -- 172.21.15.89:0/2165665373 shutdown_connections 2023-09-30T22:19:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.570+0000 7fc10a35f700 1 --2- 172.21.15.89:0/2165665373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc104131d10 0x7fc104139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.570+0000 7fc10a35f700 1 --2- 172.21.15.89:0/2165665373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 0x7fc10406ab40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.570+0000 7fc10a35f700 1 -- 172.21.15.89:0/2165665373 >> 172.21.15.89:0/2165665373 conn(0x7fc104066f50 msgr2=0x7fc10409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.570+0000 7fc10a35f700 1 -- 172.21.15.89:0/2165665373 shutdown_connections 2023-09-30T22:19:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.570+0000 7fc10a35f700 1 -- 172.21.15.89:0/2165665373 wait complete. 2023-09-30T22:19:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.571+0000 7fc10a35f700 1 Processor -- start 2023-09-30T22:19:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.571+0000 7fc10a35f700 1 -- start start 2023-09-30T22:19:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.571+0000 7fc10a35f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 0x7fc104120440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.571+0000 7fc10a35f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc10406b080 0x7fc104120980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.571+0000 7fc10a35f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc104131d10 0x7fc104120ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc10a35f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc104071e60 con 0x7fc10406a770 2023-09-30T22:19:04.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc10a35f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc104123c80 con 0x7fc10406b080 2023-09-30T22:19:04.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc10a35f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041228b0 con 0x7fc104131d10 2023-09-30T22:19:04.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc104131d10 0x7fc104120ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc104131d10 0x7fc104120ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54394/0 (socket says 172.21.15.89:54394) 2023-09-30T22:19:04.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 -- 172.21.15.89:0/1332799805 learned_addr learned my addr 172.21.15.89:0/1332799805 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc103fff700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 0x7fc104120440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1037fe700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc10406b080 0x7fc104120980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 -- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc10406b080 msgr2=0x7fc104120980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc10406b080 0x7fc104120980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 -- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 msgr2=0x7fc104120440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 0x7fc104120440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 -- 172.21.15.89:0/1332799805 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4009550 con 0x7fc104131d10 2023-09-30T22:19:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc1088fc700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc104131d10 0x7fc104120ec0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00efc0 tx=0x7fc0fc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.572+0000 7fc100ff9700 1 -- 172.21.15.89:0/1332799805 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc009ea0 con 0x7fc104131d10 2023-09-30T22:19:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.573+0000 7fc100ff9700 1 -- 172.21.15.89:0/1332799805 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc00f040 con 0x7fc104131d10 2023-09-30T22:19:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.573+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1041239c0 con 0x7fc104131d10 2023-09-30T22:19:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.573+0000 7fc100ff9700 1 -- 172.21.15.89:0/1332799805 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00ce90 con 0x7fc104131d10 2023-09-30T22:19:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.573+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1041315d0 con 0x7fc104131d10 2023-09-30T22:19:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.574+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc104074ea0 con 0x7fc104131d10 2023-09-30T22:19:04.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.574+0000 7fc100ff9700 1 -- 172.21.15.89:0/1332799805 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc0fc014480 con 0x7fc104131d10 2023-09-30T22:19:04.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.574+0000 7fc100ff9700 1 -- 172.21.15.89:0/1332799805 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc0fc040460 con 0x7fc104131d10 2023-09-30T22:19:04.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.578+0000 7fc100ff9700 1 -- 172.21.15.89:0/1332799805 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0fc012930 con 0x7fc104131d10 2023-09-30T22:19:04.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.587+0000 7ff30ee8f700 1 Processor -- start 2023-09-30T22:19:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.587+0000 7ff30ee8f700 1 -- start start 2023-09-30T22:19:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.588+0000 7ff30ee8f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff30806f5d0 0x7ff30812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.588+0000 7ff30ee8f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff308135c20 0x7ff308138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.588+0000 7ff30ee8f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30806fa40 con 0x7ff30812cf60 2023-09-30T22:19:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.588+0000 7ff30ee8f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30812cd90 con 0x7ff308135c20 2023-09-30T22:19:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.588+0000 7ff30ee8f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff308071780 con 0x7ff30806f5d0 2023-09-30T22:19:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.588+0000 7ff30cc2b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff30812cf60 0x7ff30812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50130/0 (socket says 172.21.15.89:50130) 2023-09-30T22:19:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.588+0000 7ff30cc2b700 1 -- 172.21.15.89:0/1072995795 learned_addr learned my addr 172.21.15.89:0/1072995795 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.588+0000 7ff307fff700 1 --2- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff30806f5d0 0x7ff30812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff30d42c700 1 --2- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff308135c20 0x7ff308138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff3077fe700 1 -- 172.21.15.89:0/1072995795 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884151774 0 0) 0x7ff30806fa40 con 0x7ff30812cf60 2023-09-30T22:19:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff3077fe700 1 -- 172.21.15.89:0/1072995795 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f80035b0 con 0x7ff30812cf60 2023-09-30T22:19:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff307fff700 1 -- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff308135c20 msgr2=0x7ff308138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff307fff700 1 --2- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff308135c20 0x7ff308138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff307fff700 1 -- 172.21.15.89:0/1072995795 >> v1:172.21.15.89:6789/0 conn(0x7ff30812cf60 legacy=0x7ff30812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff307fff700 1 -- 172.21.15.89:0/1072995795 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff308071a40 con 0x7ff30806f5d0 2023-09-30T22:19:04.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff307fff700 1 --2- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff30806f5d0 0x7ff30812ffc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400aa40 tx=0x7ff2f400ad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=666d1ccef149f225 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:04.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff3077fe700 1 -- 172.21.15.89:0/1072995795 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400cb30 con 0x7ff30806f5d0 2023-09-30T22:19:04.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff3077fe700 1 -- 172.21.15.89:0/1072995795 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f400cc90 con 0x7ff30806f5d0 2023-09-30T22:19:04.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.589+0000 7ff3077fe700 1 -- 172.21.15.89:0/1072995795 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4018400 con 0x7ff30806f5d0 2023-09-30T22:19:04.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.590+0000 7ff30ee8f700 1 -- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff30806f5d0 msgr2=0x7ff30812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.590+0000 7ff30ee8f700 1 --2- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff30806f5d0 0x7ff30812ffc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400aa40 tx=0x7ff2f400ad50 comp rx=0 tx=0).stop 2023-09-30T22:19:04.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.590+0000 7ff30ee8f700 1 -- 172.21.15.89:0/1072995795 shutdown_connections 2023-09-30T22:19:04.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.590+0000 7ff30ee8f700 1 --2- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff308135c20 0x7ff308138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.590+0000 7ff30ee8f700 1 --2- 172.21.15.89:0/1072995795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff30806f5d0 0x7ff30812ffc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.590+0000 7ff30ee8f700 1 -- 172.21.15.89:0/1072995795 >> 172.21.15.89:0/1072995795 conn(0x7ff3080a2aa0 msgr2=0x7ff3080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:04.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.590+0000 7ff30ee8f700 1 -- 172.21.15.89:0/1072995795 shutdown_connections 2023-09-30T22:19:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.590+0000 7ff30ee8f700 1 -- 172.21.15.89:0/1072995795 wait complete. 2023-09-30T22:19:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.591+0000 7ff30ee8f700 1 Processor -- start 2023-09-30T22:19:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.591+0000 7ff30ee8f700 1 -- start start 2023-09-30T22:19:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.591+0000 7ff30ee8f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff30806f5d0 0x7ff30808a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.591+0000 7ff30ee8f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30812cf60 0x7ff308126f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.591+0000 7ff30ee8f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff308135c20 0x7ff3081274c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.591+0000 7ff30ee8f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff308135580 con 0x7ff30812cf60 2023-09-30T22:19:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.591+0000 7ff30ee8f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff308129fa0 con 0x7ff30806f5d0 2023-09-30T22:19:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.591+0000 7ff30ee8f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081357b0 con 0x7ff308135c20 2023-09-30T22:19:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff308135c20 0x7ff3081274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff308135c20 0x7ff3081274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54406/0 (socket says 172.21.15.89:54406) 2023-09-30T22:19:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 -- 172.21.15.89:0/3948873693 learned_addr learned my addr 172.21.15.89:0/3948873693 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff307fff700 1 --2- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30812cf60 0x7ff308126f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 -- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff30806f5d0 msgr2=0x7ff30808a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 --2- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff30806f5d0 0x7ff30808a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 -- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30812cf60 msgr2=0x7ff308126f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 --2- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30812cf60 0x7ff308126f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 -- 172.21.15.89:0/3948873693 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f400a640 con 0x7ff308135c20 2023-09-30T22:19:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.592+0000 7ff30d42c700 1 --2- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff308135c20 0x7ff3081274c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff300007e30 tx=0x7ff30000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.593+0000 7ff3057fa700 1 -- 172.21.15.89:0/3948873693 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30000ede0 con 0x7ff308135c20 2023-09-30T22:19:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.593+0000 7ff3057fa700 1 -- 172.21.15.89:0/3948873693 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff300014420 con 0x7ff308135c20 2023-09-30T22:19:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.593+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30809a570 con 0x7ff308135c20 2023-09-30T22:19:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.593+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30809a8e0 con 0x7ff308135c20 2023-09-30T22:19:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.593+0000 7ff3057fa700 1 -- 172.21.15.89:0/3948873693 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3000147d0 con 0x7ff308135c20 2023-09-30T22:19:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.594+0000 7ff3057fa700 1 -- 172.21.15.89:0/3948873693 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff300014930 con 0x7ff308135c20 2023-09-30T22:19:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.594+0000 7ff3057fa700 1 -- 172.21.15.89:0/3948873693 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff30003fd70 con 0x7ff308135c20 2023-09-30T22:19:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.594+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e80052e0 con 0x7ff308135c20 2023-09-30T22:19:04.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.599+0000 7ff3057fa700 1 -- 172.21.15.89:0/3948873693 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff3000122e0 con 0x7ff308135c20 2023-09-30T22:19:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.709+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5cdc09e130 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.709+0000 7f5cc37fe700 1 -- 172.21.15.89:0/3368352602 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5cdc09e130 con 0x7f5cdc09d9f0 2023-09-30T22:19:04.710 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.712+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cdc09d9f0 msgr2=0x7f5cdc12ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.712+0000 7f5ce101a700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cdc09d9f0 0x7f5cdc12ab90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c510 tx=0x7f5ccc00c820 comp rx=0 tx=0).stop 2023-09-30T22:19:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.712+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 shutdown_connections 2023-09-30T22:19:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.712+0000 7f5ce101a700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cdc136a70 0x7f5cdc09a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.712+0000 7f5ce101a700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cdc09d9f0 0x7f5cdc12ab90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.712+0000 7f5ce101a700 1 --2- 172.21.15.89:0/3368352602 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cdc1316c0 0x7f5cdc12b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.712+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 >> 172.21.15.89:0/3368352602 conn(0x7f5cdc066f50 msgr2=0x7f5cdc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.713+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 shutdown_connections 2023-09-30T22:19:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.713+0000 7f5ce101a700 1 -- 172.21.15.89:0/3368352602 wait complete. 2023-09-30T22:19:04.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.723+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f988006b7c0 con 0x7f988012f8c0 2023-09-30T22:19:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.723+0000 7f9867fff700 1 -- 172.21.15.89:0/786627982 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9874021070 con 0x7f988012f8c0 2023-09-30T22:19:04.724 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:04.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.725+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988012f8c0 msgr2=0x7f988012a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.725+0000 7f9884f9f700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988012f8c0 0x7f988012a670 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f987400e980 tx=0x7f987400ec90 comp rx=0 tx=0).stop 2023-09-30T22:19:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.725+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 shutdown_connections 2023-09-30T22:19:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.725+0000 7f9884f9f700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9880130210 0x7f98801c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.725+0000 7f9884f9f700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988009dba0 0x7f988012a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.725+0000 7f9884f9f700 1 --2- 172.21.15.89:0/786627982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f988012f8c0 0x7f988012a670 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.725+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 >> 172.21.15.89:0/786627982 conn(0x7f98800a2540 msgr2=0x7f98800a0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.726+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 shutdown_connections 2023-09-30T22:19:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.726+0000 7f9884f9f700 1 -- 172.21.15.89:0/786627982 wait complete. 2023-09-30T22:19:04.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:04.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.774+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc10406c360 con 0x7fc104131d10 2023-09-30T22:19:04.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.774+0000 7fc100ff9700 1 -- 172.21.15.89:0/1332799805 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc0fc018020 con 0x7fc104131d10 2023-09-30T22:19:04.778 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc104131d10 msgr2=0x7fc104120ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc104131d10 0x7fc104120ec0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00efc0 tx=0x7fc0fc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 shutdown_connections 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc104131d10 0x7fc104120ec0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc10406a770 0x7fc104120440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 --2- 172.21.15.89:0/1332799805 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc10406b080 0x7fc104120980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 >> 172.21.15.89:0/1332799805 conn(0x7fc104066f50 msgr2=0x7fc1040700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 shutdown_connections 2023-09-30T22:19:04.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.776+0000 7fc10a35f700 1 -- 172.21.15.89:0/1332799805 wait complete. 2023-09-30T22:19:04.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:04.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.793+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff2e8005590 con 0x7ff308135c20 2023-09-30T22:19:04.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.793+0000 7ff3057fa700 1 -- 172.21.15.89:0/3948873693 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff30001bb00 con 0x7ff308135c20 2023-09-30T22:19:04.794 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:04.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff308135c20 msgr2=0x7ff3081274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:04.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 --2- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff308135c20 0x7ff3081274c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff300007e30 tx=0x7ff30000e3c0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 shutdown_connections 2023-09-30T22:19:04.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 --2- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff308135c20 0x7ff3081274c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 --2- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff30806f5d0 0x7ff30808a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 --2- 172.21.15.89:0/3948873693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30812cf60 0x7ff308126f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:04.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 >> 172.21.15.89:0/3948873693 conn(0x7ff3080a2aa0 msgr2=0x7ff30812a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:04.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 shutdown_connections 2023-09-30T22:19:04.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:04.795+0000 7ff30ee8f700 1 -- 172.21.15.89:0/3948873693 wait complete. 2023-09-30T22:19:04.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:05.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:05.740 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:05.790 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:05.809 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:05.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.870+0000 7fbd1be2c700 1 Processor -- start 2023-09-30T22:19:05.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.870+0000 7fbd1be2c700 1 -- start start 2023-09-30T22:19:05.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1be2c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409e1d0 0x7fbd14135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1be2c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd14135e50 0x7fbd14138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1be2c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1409f130 con 0x7fbd1409d6f0 2023-09-30T22:19:05.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1be2c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1409f520 con 0x7fbd14135e50 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1be2c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1409e930 con 0x7fbd1409e1d0 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1a3c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd14135e50 0x7fbd14138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd19bc8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbd1409d6f0 0x7fbd1409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50146/0 (socket says 172.21.15.89:50146) 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd19bc8700 1 -- 172.21.15.89:0/162480202 learned_addr learned my addr 172.21.15.89:0/162480202 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd193c7700 1 --2- 172.21.15.89:0/162480202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409e1d0 0x7fbd14135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1a3c9700 1 -- 172.21.15.89:0/162480202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409e1d0 msgr2=0x7fbd14135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1a3c9700 1 --2- 172.21.15.89:0/162480202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409e1d0 0x7fbd14135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1a3c9700 1 -- 172.21.15.89:0/162480202 >> v1:172.21.15.89:6789/0 conn(0x7fbd1409d6f0 legacy=0x7fbd1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd1a3c9700 1 -- 172.21.15.89:0/162480202 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1409b2d0 con 0x7fbd14135e50 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.871+0000 7fbd18bc6700 1 -- 172.21.15.89:0/162480202 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157379374 0 0) 0x7fbd1409f130 con 0x7fbd1409d6f0 2023-09-30T22:19:05.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.872+0000 7fbd1a3c9700 1 --2- 172.21.15.89:0/162480202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd14135e50 0x7fbd14138230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009dc0 tx=0x7fbd0c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7c2de2a96076597 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:05.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.872+0000 7fbd18bc6700 1 -- 172.21.15.89:0/162480202 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c00bd20 con 0x7fbd14135e50 2023-09-30T22:19:05.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.872+0000 7fbd18bc6700 1 -- 172.21.15.89:0/162480202 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c00be80 con 0x7fbd14135e50 2023-09-30T22:19:05.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.872+0000 7fbd18bc6700 1 -- 172.21.15.89:0/162480202 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0116c0 con 0x7fbd14135e50 2023-09-30T22:19:05.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.872+0000 7fbd1be2c700 1 -- 172.21.15.89:0/162480202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd14135e50 msgr2=0x7fbd14138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.872+0000 7fbd1be2c700 1 --2- 172.21.15.89:0/162480202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd14135e50 0x7fbd14138230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009dc0 tx=0x7fbd0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:19:05.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.873+0000 7fbd1be2c700 1 -- 172.21.15.89:0/162480202 shutdown_connections 2023-09-30T22:19:05.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.873+0000 7fbd1be2c700 1 --2- 172.21.15.89:0/162480202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd14135e50 0x7fbd14138230 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.873+0000 7fbd1be2c700 1 --2- 172.21.15.89:0/162480202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409e1d0 0x7fbd14135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.873+0000 7fbd1be2c700 1 -- 172.21.15.89:0/162480202 >> 172.21.15.89:0/162480202 conn(0x7fbd14066f50 msgr2=0x7fbd1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.873+0000 7fbd1be2c700 1 -- 172.21.15.89:0/162480202 shutdown_connections 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.873+0000 7fbd1be2c700 1 -- 172.21.15.89:0/162480202 wait complete. 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1be2c700 1 Processor -- start 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1be2c700 1 -- start start 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1be2c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409d6f0 0x7fbd141c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1be2c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd1409e1d0 0x7fbd141c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1be2c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd14135e50 0x7fbd14119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1be2c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1411b280 con 0x7fbd14135e50 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1be2c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1411b090 con 0x7fbd1409e1d0 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1be2c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1411b960 con 0x7fbd1409d6f0 2023-09-30T22:19:05.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd193c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd1409e1d0 0x7fbd141c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd193c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd1409e1d0 0x7fbd141c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55700/0 (socket says 172.21.15.89:55700) 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd193c7700 1 -- 172.21.15.89:0/3910447423 learned_addr learned my addr 172.21.15.89:0/3910447423 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd1a3c9700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd14135e50 0x7fbd14119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.874+0000 7fbd19bc8700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409d6f0 0x7fbd141c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd193c7700 1 -- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409d6f0 msgr2=0x7fbd141c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd193c7700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409d6f0 0x7fbd141c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd193c7700 1 -- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd14135e50 msgr2=0x7fbd14119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd193c7700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd14135e50 0x7fbd14119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd193c7700 1 -- 172.21.15.89:0/3910447423 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c009a70 con 0x7fbd1409e1d0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd193c7700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd1409e1d0 0x7fbd141c4880 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400e9b0 tx=0x7fbd0400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd0a7fc700 1 -- 172.21.15.89:0/3910447423 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04009710 con 0x7fbd1409e1d0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd0a7fc700 1 -- 172.21.15.89:0/3910447423 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0400ee50 con 0x7fbd1409e1d0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1409c700 con 0x7fbd1409e1d0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd0a7fc700 1 -- 172.21.15.89:0/3910447423 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400ca50 con 0x7fbd1409e1d0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.875+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd141c4dc0 con 0x7fbd1409e1d0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.876+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd14074ea0 con 0x7fbd1409e1d0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.876+0000 7fbd0a7fc700 1 -- 172.21.15.89:0/3910447423 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd0400cbb0 con 0x7fbd1409e1d0 2023-09-30T22:19:05.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.876+0000 7fbd0a7fc700 1 -- 172.21.15.89:0/3910447423 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbd0403f690 con 0x7fbd1409e1d0 2023-09-30T22:19:05.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.885+0000 7fbd0a7fc700 1 -- 172.21.15.89:0/3910447423 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd040232f0 con 0x7fbd1409e1d0 2023-09-30T22:19:05.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.911+0000 7f3120e43700 1 Processor -- start 2023-09-30T22:19:05.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.911+0000 7f3120e43700 1 -- start start 2023-09-30T22:19:05.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f3120e43700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c06c0e0 0x7f311c09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f3120e43700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c134ec0 0x7f311c1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f3120e43700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f311c137f20 0x7f311c138390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f3120e43700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c06c4b0 con 0x7f311c137f20 2023-09-30T22:19:05.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f3120e43700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0a09c0 con 0x7f311c134ec0 2023-09-30T22:19:05.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f3120e43700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f311c134980 con 0x7f311c06c0e0 2023-09-30T22:19:05.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f311a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c06c0e0 0x7f311c09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f311a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c06c0e0 0x7f311c09ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54426/0 (socket says 172.21.15.89:54426) 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f311a59c700 1 -- 172.21.15.89:0/3996844554 learned_addr learned my addr 172.21.15.89:0/3996844554 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f3119d9b700 1 --2- 172.21.15.89:0/3996844554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c134ec0 0x7f311c1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.912+0000 7f311ad9d700 1 --2- 172.21.15.89:0/3996844554 >> v2:172.21.15.89:3300/0 conn(0x7f311c137f20 0x7f311c138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311a59c700 1 -- 172.21.15.89:0/3996844554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c134ec0 msgr2=0x7f311c1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311a59c700 1 --2- 172.21.15.89:0/3996844554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c134ec0 0x7f311c1379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311a59c700 1 -- 172.21.15.89:0/3996844554 >> v2:172.21.15.89:3300/0 conn(0x7f311c137f20 msgr2=0x7f311c138390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311a59c700 1 --2- 172.21.15.89:0/3996844554 >> v2:172.21.15.89:3300/0 conn(0x7f311c137f20 0x7f311c138390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311a59c700 1 -- 172.21.15.89:0/3996844554 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0a0600 con 0x7f311c06c0e0 2023-09-30T22:19:05.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311a59c700 1 --2- 172.21.15.89:0/3996844554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c06c0e0 0x7f311c09ac90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f310c0098f0 tx=0x7f310c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=560767a5282f052f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:05.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311959a700 1 -- 172.21.15.89:0/3996844554 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c011830 con 0x7f311c06c0e0 2023-09-30T22:19:05.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311959a700 1 -- 172.21.15.89:0/3996844554 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c011990 con 0x7f311c06c0e0 2023-09-30T22:19:05.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.913+0000 7f311959a700 1 -- 172.21.15.89:0/3996844554 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c011d20 con 0x7f311c06c0e0 2023-09-30T22:19:05.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 -- 172.21.15.89:0/3996844554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c06c0e0 msgr2=0x7f311c09ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 --2- 172.21.15.89:0/3996844554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c06c0e0 0x7f311c09ac90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f310c0098f0 tx=0x7f310c009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:05.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 -- 172.21.15.89:0/3996844554 shutdown_connections 2023-09-30T22:19:05.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 --2- 172.21.15.89:0/3996844554 >> v2:172.21.15.89:3300/0 conn(0x7f311c137f20 0x7f311c138390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 --2- 172.21.15.89:0/3996844554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c06c0e0 0x7f311c09ac90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 --2- 172.21.15.89:0/3996844554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c134ec0 0x7f311c1379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 -- 172.21.15.89:0/3996844554 >> 172.21.15.89:0/3996844554 conn(0x7f311c0a4210 msgr2=0x7f311c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 -- 172.21.15.89:0/3996844554 shutdown_connections 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.914+0000 7f3120e43700 1 -- 172.21.15.89:0/3996844554 wait complete. 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.916+0000 7f3120e43700 1 Processor -- start 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.916+0000 7f3120e43700 1 -- start start 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.916+0000 7f3120e43700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c06c0e0 0x7f311c134020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.916+0000 7f3120e43700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c134ec0 0x7f311c11a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.916+0000 7f3120e43700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311c137f20 0x7f311c11aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.916+0000 7f3120e43700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c11dfe0 con 0x7f311c137f20 2023-09-30T22:19:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.916+0000 7f3120e43700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c11cc10 con 0x7f311c06c0e0 2023-09-30T22:19:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.916+0000 7f3120e43700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f311c11c4c0 con 0x7f311c134ec0 2023-09-30T22:19:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c06c0e0 0x7f311c134020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c06c0e0 0x7f311c134020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55726/0 (socket says 172.21.15.89:55726) 2023-09-30T22:19:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311a59c700 1 -- 172.21.15.89:0/1930477688 learned_addr learned my addr 172.21.15.89:0/1930477688 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f3119d9b700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c134ec0 0x7f311c11a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311ad9d700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311c137f20 0x7f311c11aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311ad9d700 1 -- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c134ec0 msgr2=0x7f311c11a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311ad9d700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c134ec0 0x7f311c11a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311ad9d700 1 -- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c06c0e0 msgr2=0x7f311c134020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311ad9d700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c06c0e0 0x7f311c134020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311ad9d700 1 -- 172.21.15.89:0/1930477688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c009550 con 0x7f311c137f20 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f3119d9b700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c134ec0 0x7f311c11a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f311ad9d700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311c137f20 0x7f311c11aea0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f311400efc0 tx=0x7f311400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.917+0000 7f310affd700 1 -- 172.21.15.89:0/1930477688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400cc90 con 0x7f311c137f20 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.918+0000 7f310affd700 1 -- 172.21.15.89:0/1930477688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311400f040 con 0x7f311c137f20 2023-09-30T22:19:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.918+0000 7f310affd700 1 -- 172.21.15.89:0/1930477688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31140146c0 con 0x7f311c137f20 2023-09-30T22:19:05.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.918+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c0a46e0 con 0x7f311c137f20 2023-09-30T22:19:05.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.918+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c134560 con 0x7f311c137f20 2023-09-30T22:19:05.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.919+0000 7f310affd700 1 -- 172.21.15.89:0/1930477688 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3114014820 con 0x7f311c137f20 2023-09-30T22:19:05.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.919+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c138f20 con 0x7f311c137f20 2023-09-30T22:19:05.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.919+0000 7f310affd700 1 -- 172.21.15.89:0/1930477688 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3114040130 con 0x7f311c137f20 2023-09-30T22:19:05.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.928+0000 7f310affd700 1 -- 172.21.15.89:0/1930477688 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3114014ad0 con 0x7f311c137f20 2023-09-30T22:19:05.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.951+0000 7f22a7754700 1 Processor -- start 2023-09-30T22:19:05.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.951+0000 7f22a7754700 1 -- start start 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a7754700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22a009e850 0x7f22a009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a7754700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009f160 0x7f22a0133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a7754700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a01345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a7754700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a009ae60 con 0x7f22a009e850 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a7754700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a009f960 con 0x7f22a0134130 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a7754700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22a006cb10 con 0x7f22a009f160 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a54f0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22a009e850 0x7f22a009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a54f0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22a009e850 0x7f22a009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42656/0 (socket says 172.21.15.89:42656) 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a54f0700 1 -- 172.21.15.89:0/3945024097 learned_addr learned my addr 172.21.15.89:0/3945024097 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a5cf1700 1 --2- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a01345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.952+0000 7f22a4cef700 1 --2- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009f160 0x7f22a0133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f22a5cf1700 1 -- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009f160 msgr2=0x7f22a0133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f22a5cf1700 1 --2- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009f160 0x7f22a0133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f22a5cf1700 1 -- 172.21.15.89:0/3945024097 >> v2:172.21.15.89:3300/0 conn(0x7f22a009e850 msgr2=0x7f22a009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f22a5cf1700 1 --2- 172.21.15.89:0/3945024097 >> v2:172.21.15.89:3300/0 conn(0x7f22a009e850 0x7f22a009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f22a5cf1700 1 -- 172.21.15.89:0/3945024097 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a006b990 con 0x7f22a0134130 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f22a54f0700 1 --2- 172.21.15.89:0/3945024097 >> v2:172.21.15.89:3300/0 conn(0x7f22a009e850 0x7f22a009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f22a5cf1700 1 --2- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a01345b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2298009d90 tx=0x7f229800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5e73196c337105e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f2297fff700 1 -- 172.21.15.89:0/3945024097 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800e070 con 0x7f22a0134130 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f2297fff700 1 -- 172.21.15.89:0/3945024097 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229800bb20 con 0x7f22a0134130 2023-09-30T22:19:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.953+0000 7f2297fff700 1 -- 172.21.15.89:0/3945024097 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298012430 con 0x7f22a0134130 2023-09-30T22:19:05.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 -- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 msgr2=0x7f22a01345b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 --2- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a01345b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2298009d90 tx=0x7f229800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 -- 172.21.15.89:0/3945024097 shutdown_connections 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 --2- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a01345b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 --2- 172.21.15.89:0/3945024097 >> v2:172.21.15.89:3300/0 conn(0x7f22a009e850 0x7f22a009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 --2- 172.21.15.89:0/3945024097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009f160 0x7f22a0133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 -- 172.21.15.89:0/3945024097 >> 172.21.15.89:0/3945024097 conn(0x7f22a00a3f70 msgr2=0x7f22a00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 -- 172.21.15.89:0/3945024097 shutdown_connections 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.954+0000 7f22a7754700 1 -- 172.21.15.89:0/3945024097 wait complete. 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.955+0000 7f22a7754700 1 Processor -- start 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.955+0000 7f22a7754700 1 -- start start 2023-09-30T22:19:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.955+0000 7f22a7754700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009e850 0x7f22a0122580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.955+0000 7f22a7754700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a009f160 0x7f22a0122ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.955+0000 7f22a7754700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a0123000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.955+0000 7f22a7754700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a0125cb0 con 0x7f22a009f160 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.955+0000 7f22a7754700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00a1780 con 0x7f22a0134130 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.955+0000 7f22a7754700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22a0124190 con 0x7f22a009e850 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a009f160 0x7f22a0122ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a009f160 0x7f22a0122ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42666/0 (socket says 172.21.15.89:42666) 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 -- 172.21.15.89:0/1843472394 learned_addr learned my addr 172.21.15.89:0/1843472394 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a54f0700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009e850 0x7f22a0122580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a5cf1700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a0123000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 -- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009e850 msgr2=0x7f22a0122580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009e850 0x7f22a0122580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 -- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 msgr2=0x7f22a0123000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a0123000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 -- 172.21.15.89:0/1843472394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c009550 con 0x7f22a009f160 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a54f0700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009e850 0x7f22a0122580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a4cef700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a009f160 0x7f22a0122ac0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f229000e9b0 tx=0x7f229000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f2295ffb700 1 -- 172.21.15.89:0/1843472394 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2290009710 con 0x7f22a009f160 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f2295ffb700 1 -- 172.21.15.89:0/1843472394 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229000ee50 con 0x7f22a009f160 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f2295ffb700 1 -- 172.21.15.89:0/1843472394 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000ca50 con 0x7f22a009f160 2023-09-30T22:19:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.956+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2298009a70 con 0x7f22a009f160 2023-09-30T22:19:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.957+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a01259f0 con 0x7f22a009f160 2023-09-30T22:19:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.958+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a00747f0 con 0x7f22a009f160 2023-09-30T22:19:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.958+0000 7f2295ffb700 1 -- 172.21.15.89:0/1843472394 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f229000cbb0 con 0x7f22a009f160 2023-09-30T22:19:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.959+0000 7f2295ffb700 1 -- 172.21.15.89:0/1843472394 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f229003fea0 con 0x7f22a009f160 2023-09-30T22:19:05.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.961+0000 7f54a45f1700 1 Processor -- start 2023-09-30T22:19:05.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.961+0000 7f54a45f1700 1 -- start start 2023-09-30T22:19:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a45f1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f549c1338c0 0x7f549c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a45f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a45f1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c06f720 con 0x7f549c09e390 2023-09-30T22:19:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a45f1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c070c80 con 0x7f549c1338c0 2023-09-30T22:19:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a45f1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f549c06fce0 con 0x7f549c1341d0 2023-09-30T22:19:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a1b8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a1b8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c09de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54478/0 (socket says 172.21.15.89:54478) 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a1b8c700 1 -- 172.21.15.89:0/173232600 learned_addr learned my addr 172.21.15.89:0/173232600 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a1b8c700 1 -- 172.21.15.89:0/173232600 >> v2:172.21.15.89:3300/0 conn(0x7f549c1338c0 msgr2=0x7f549c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a1b8c700 1 --2- 172.21.15.89:0/173232600 >> v2:172.21.15.89:3300/0 conn(0x7f549c1338c0 0x7f549c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a1b8c700 1 -- 172.21.15.89:0/173232600 >> v1:172.21.15.89:6789/0 conn(0x7f549c09e390 legacy=0x7f549c13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a1b8c700 1 -- 172.21.15.89:0/173232600 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c1336a0 con 0x7f549c1341d0 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a1b8c700 1 --2- 172.21.15.89:0/173232600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c09de50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f548c0098f0 tx=0x7f548c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee95820cf8c8059f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a138b700 1 -- 172.21.15.89:0/173232600 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c004230 con 0x7f549c1341d0 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a138b700 1 -- 172.21.15.89:0/173232600 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c004390 con 0x7f549c1341d0 2023-09-30T22:19:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.962+0000 7f54a138b700 1 -- 172.21.15.89:0/173232600 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0036a0 con 0x7f549c1341d0 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.963+0000 7f54a45f1700 1 -- 172.21.15.89:0/173232600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 msgr2=0x7f549c09de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.963+0000 7f54a45f1700 1 --2- 172.21.15.89:0/173232600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c09de50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f548c0098f0 tx=0x7f548c009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.963+0000 7f54a45f1700 1 -- 172.21.15.89:0/173232600 shutdown_connections 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.963+0000 7f54a45f1700 1 --2- 172.21.15.89:0/173232600 >> v2:172.21.15.89:3300/0 conn(0x7f549c1338c0 0x7f549c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.963+0000 7f54a45f1700 1 --2- 172.21.15.89:0/173232600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c09de50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.963+0000 7f54a45f1700 1 -- 172.21.15.89:0/173232600 >> 172.21.15.89:0/173232600 conn(0x7f549c0a2aa0 msgr2=0x7f549c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.963+0000 7f54a45f1700 1 -- 172.21.15.89:0/173232600 shutdown_connections 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.963+0000 7f54a45f1700 1 -- 172.21.15.89:0/173232600 wait complete. 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.964+0000 7f54a45f1700 1 Processor -- start 2023-09-30T22:19:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.964+0000 7f54a45f1700 1 -- start start 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.964+0000 7f54a45f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f549c09e390 0x7f549c11f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.964+0000 7f54a45f1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c1338c0 0x7f549c11ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.964+0000 7f54a45f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c120440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.964+0000 7f54a45f1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c06fa10 con 0x7f549c1338c0 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.964+0000 7f54a45f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c123200 con 0x7f549c09e390 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.964+0000 7f54a45f1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f549c1216e0 con 0x7f549c1341d0 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c120440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c120440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54488/0 (socket says 172.21.15.89:54488) 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 -- 172.21.15.89:0/3826983358 learned_addr learned my addr 172.21.15.89:0/3826983358 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a238d700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f549c09e390 0x7f549c11f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 -- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f549c09e390 msgr2=0x7f549c11f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f549c09e390 0x7f549c11f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 -- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c1338c0 msgr2=0x7f549c11ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c1338c0 0x7f549c11ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 -- 172.21.15.89:0/3826983358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c009550 con 0x7f549c1341d0 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a238d700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f549c09e390 0x7f549c11f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a2b8e700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c120440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f549400c7d0 tx=0x7f549400cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f5492ffd700 1 -- 172.21.15.89:0/3826983358 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494003e10 con 0x7f549c1341d0 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f5492ffd700 1 -- 172.21.15.89:0/3826983358 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494005070 con 0x7f549c1341d0 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f5492ffd700 1 -- 172.21.15.89:0/3826983358 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494005420 con 0x7f549c1341d0 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c122fa0 con 0x7f549c1341d0 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.965+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c123c70 con 0x7f549c1341d0 2023-09-30T22:19:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.966+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c123440 con 0x7f549c1341d0 2023-09-30T22:19:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.967+0000 7f5492ffd700 1 -- 172.21.15.89:0/3826983358 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5494006ba0 con 0x7f549c1341d0 2023-09-30T22:19:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.967+0000 7f2295ffb700 1 -- 172.21.15.89:0/1843472394 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f229001a360 con 0x7f22a009f160 2023-09-30T22:19:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.967+0000 7f5492ffd700 1 -- 172.21.15.89:0/3826983358 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f549403ea10 con 0x7f549c1341d0 2023-09-30T22:19:05.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:05.971+0000 7f5492ffd700 1 -- 172.21.15.89:0/3826983358 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5494019900 con 0x7f549c1341d0 2023-09-30T22:19:06.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.086+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbd14130260 con 0x7fbd1409e1d0 2023-09-30T22:19:06.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.086+0000 7fbd0a7fc700 1 -- 172.21.15.89:0/3910447423 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd04021070 con 0x7fbd1409e1d0 2023-09-30T22:19:06.087 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:06.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.089+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd1409e1d0 msgr2=0x7fbd141c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:06.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.089+0000 7fbd1be2c700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd1409e1d0 0x7fbd141c4880 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400e9b0 tx=0x7fbd0400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.089+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 shutdown_connections 2023-09-30T22:19:06.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.089+0000 7fbd1be2c700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd14135e50 0x7fbd14119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.089+0000 7fbd1be2c700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd1409d6f0 0x7fbd141c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.089+0000 7fbd1be2c700 1 --2- 172.21.15.89:0/3910447423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd1409e1d0 0x7fbd141c4880 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.089+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 >> 172.21.15.89:0/3910447423 conn(0x7fbd14066f50 msgr2=0x7fbd14132810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:06.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.090+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 shutdown_connections 2023-09-30T22:19:06.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.090+0000 7fbd1be2c700 1 -- 172.21.15.89:0/3910447423 wait complete. 2023-09-30T22:19:06.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.126+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f311c074ea0 con 0x7f311c137f20 2023-09-30T22:19:06.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.127+0000 7f310affd700 1 -- 172.21.15.89:0/1930477688 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3114007330 con 0x7f311c137f20 2023-09-30T22:19:06.127 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:06.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311c137f20 msgr2=0x7f311c11aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311c137f20 0x7f311c11aea0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f311400efc0 tx=0x7f311400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 shutdown_connections 2023-09-30T22:19:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311c137f20 0x7f311c11aea0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311c06c0e0 0x7f311c134020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 --2- 172.21.15.89:0/1930477688 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311c134ec0 0x7f311c11a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 >> 172.21.15.89:0/1930477688 conn(0x7f311c0a4210 msgr2=0x7f311c099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 shutdown_connections 2023-09-30T22:19:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.129+0000 7f3120e43700 1 -- 172.21.15.89:0/1930477688 wait complete. 2023-09-30T22:19:06.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.164+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f549c122fa0 con 0x7f549c1341d0 2023-09-30T22:19:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.165+0000 7f5492ffd700 1 -- 172.21.15.89:0/3826983358 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f549c122fa0 con 0x7f549c1341d0 2023-09-30T22:19:06.165 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 msgr2=0x7f549c120440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c120440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f549400c7d0 tx=0x7f549400cb90 comp rx=0 tx=0).stop 2023-09-30T22:19:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 shutdown_connections 2023-09-30T22:19:06.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f549c1341d0 0x7f549c120440 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f549c09e390 0x7f549c11f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 --2- 172.21.15.89:0/3826983358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549c1338c0 0x7f549c11ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 >> 172.21.15.89:0/3826983358 conn(0x7f549c0a2aa0 msgr2=0x7f549c09f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 shutdown_connections 2023-09-30T22:19:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.167+0000 7f54a45f1700 1 -- 172.21.15.89:0/3826983358 wait complete. 2023-09-30T22:19:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.171+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f22a006bbb0 con 0x7f22a009f160 2023-09-30T22:19:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.171+0000 7f2295ffb700 1 -- 172.21.15.89:0/1843472394 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2290017070 con 0x7f22a009f160 2023-09-30T22:19:06.172 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a009f160 msgr2=0x7f22a0122ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a009f160 0x7f22a0122ac0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f229000e9b0 tx=0x7f229000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 shutdown_connections 2023-09-30T22:19:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22a0134130 0x7f22a0123000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22a009e850 0x7f22a0122580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 --2- 172.21.15.89:0/1843472394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a009f160 0x7f22a0122ac0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 >> 172.21.15.89:0/1843472394 conn(0x7f22a00a3f70 msgr2=0x7f22a0070980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 shutdown_connections 2023-09-30T22:19:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:06.174+0000 7f22a7754700 1 -- 172.21.15.89:0/1843472394 wait complete. 2023-09-30T22:19:06.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:06.188 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:07.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:07.142 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:07.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:07.188 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:07.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.256+0000 7fbc764eb700 1 Processor -- start 2023-09-30T22:19:07.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.257+0000 7fbc764eb700 1 -- start start 2023-09-30T22:19:07.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.257+0000 7fbc764eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc701338c0 0x7fbc70133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.257+0000 7fbc764eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701341d0 0x7fbc7009de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.257+0000 7fbc764eb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7006f720 con 0x7fbc7009e390 2023-09-30T22:19:07.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.257+0000 7fbc764eb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70070c80 con 0x7fbc701338c0 2023-09-30T22:19:07.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.257+0000 7fbc764eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7006fce0 con 0x7fbc701341d0 2023-09-30T22:19:07.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.257+0000 7fbc74a88700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbc7009e390 0x7fbc7013a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50154/0 (socket says 172.21.15.89:50154) 2023-09-30T22:19:07.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701341d0 0x7fbc7009de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc74a88700 1 -- 172.21.15.89:0/3609187295 learned_addr learned my addr 172.21.15.89:0/3609187295 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:07.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6ffff700 1 --2- 172.21.15.89:0/3609187295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc701338c0 0x7fbc70133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6effd700 1 -- 172.21.15.89:0/3609187295 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293695300 0 0) 0x7fbc7006f720 con 0x7fbc7009e390 2023-09-30T22:19:07.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6effd700 1 -- 172.21.15.89:0/3609187295 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc580035b0 con 0x7fbc7009e390 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6f7fe700 1 -- 172.21.15.89:0/3609187295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc701338c0 msgr2=0x7fbc70133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6f7fe700 1 --2- 172.21.15.89:0/3609187295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc701338c0 0x7fbc70133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6f7fe700 1 -- 172.21.15.89:0/3609187295 >> v1:172.21.15.89:6789/0 conn(0x7fbc7009e390 legacy=0x7fbc7013a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6f7fe700 1 -- 172.21.15.89:0/3609187295 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc701336a0 con 0x7fbc701341d0 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6f7fe700 1 --2- 172.21.15.89:0/3609187295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701341d0 0x7fbc7009de50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009870 tx=0x7fbc64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfedbb8141aeddb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6effd700 1 -- 172.21.15.89:0/3609187295 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011830 con 0x7fbc701341d0 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6effd700 1 -- 172.21.15.89:0/3609187295 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc64011990 con 0x7fbc701341d0 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.258+0000 7fbc6effd700 1 -- 172.21.15.89:0/3609187295 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011d20 con 0x7fbc701341d0 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.259+0000 7fbc764eb700 1 -- 172.21.15.89:0/3609187295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701341d0 msgr2=0x7fbc7009de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.259+0000 7fbc764eb700 1 --2- 172.21.15.89:0/3609187295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701341d0 0x7fbc7009de50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009870 tx=0x7fbc64009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:07.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.259+0000 7fbc764eb700 1 -- 172.21.15.89:0/3609187295 shutdown_connections 2023-09-30T22:19:07.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.259+0000 7fbc764eb700 1 --2- 172.21.15.89:0/3609187295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc701338c0 0x7fbc70133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.259+0000 7fbc764eb700 1 --2- 172.21.15.89:0/3609187295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701341d0 0x7fbc7009de50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.259+0000 7fbc764eb700 1 -- 172.21.15.89:0/3609187295 >> 172.21.15.89:0/3609187295 conn(0x7fbc700a2aa0 msgr2=0x7fbc700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:07.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.259+0000 7fbc764eb700 1 -- 172.21.15.89:0/3609187295 shutdown_connections 2023-09-30T22:19:07.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.259+0000 7fbc764eb700 1 -- 172.21.15.89:0/3609187295 wait complete. 2023-09-30T22:19:07.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.260+0000 7fbc764eb700 1 Processor -- start 2023-09-30T22:19:07.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.261+0000 7fbc764eb700 1 -- start start 2023-09-30T22:19:07.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.261+0000 7fbc764eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc7009e390 0x7fbc7011f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.261+0000 7fbc764eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701338c0 0x7fbc7011fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.261+0000 7fbc764eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc701341d0 0x7fbc701202e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.261+0000 7fbc764eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701221e0 con 0x7fbc701341d0 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.261+0000 7fbc764eb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70130bc0 con 0x7fbc7009e390 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.261+0000 7fbc764eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70122d50 con 0x7fbc701338c0 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc7009e390 0x7fbc7011f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701338c0 0x7fbc7011fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701338c0 0x7fbc7011fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54508/0 (socket says 172.21.15.89:54508) 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc7009e390 0x7fbc7011f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55784/0 (socket says 172.21.15.89:55784) 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 -- 172.21.15.89:0/788262878 learned_addr learned my addr 172.21.15.89:0/788262878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc74a88700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc701341d0 0x7fbc701202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 -- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc7009e390 msgr2=0x7fbc7011f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc7009e390 0x7fbc7011f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 -- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc701341d0 msgr2=0x7fbc701202e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc701341d0 0x7fbc701202e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 -- 172.21.15.89:0/788262878 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640094a0 con 0x7fbc701338c0 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc74a88700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc701341d0 0x7fbc701202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6f7fe700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701338c0 0x7fbc7011fda0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc640090b0 tx=0x7fbc64004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6cff9700 1 -- 172.21.15.89:0/788262878 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64004510 con 0x7fbc701338c0 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.262+0000 7fbc6cff9700 1 -- 172.21.15.89:0/788262878 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc64028070 con 0x7fbc701338c0 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.263+0000 7fbc6cff9700 1 -- 172.21.15.89:0/788262878 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64024a70 con 0x7fbc701338c0 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.263+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7006f960 con 0x7fbc701338c0 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.263+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc701237f0 con 0x7fbc701338c0 2023-09-30T22:19:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.264+0000 7fbc6cff9700 1 -- 172.21.15.89:0/788262878 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc64024bd0 con 0x7fbc701338c0 2023-09-30T22:19:07.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.264+0000 7fbc6cff9700 1 -- 172.21.15.89:0/788262878 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbc6402a070 con 0x7fbc701338c0 2023-09-30T22:19:07.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.264+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc70074ea0 con 0x7fbc701338c0 2023-09-30T22:19:07.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.269+0000 7fbc6cff9700 1 -- 172.21.15.89:0/788262878 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc64034080 con 0x7fbc701338c0 2023-09-30T22:19:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.291+0000 7f8961793700 1 Processor -- start 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f8961793700 1 -- start start 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f8961793700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c06a770 0x7f895c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f8961793700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f895c09dc30 0x7f895c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f8961793700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f8961793700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c06ab40 con 0x7f895c09dc30 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f8961793700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c09ebc0 con 0x7f895c13c1e0 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f8961793700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f895c09a810 con 0x7f895c06a770 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f895a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f895c09dc30 0x7f895c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f895a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f895c09dc30 0x7f895c09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42690/0 (socket says 172.21.15.89:42690) 2023-09-30T22:19:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f895a7fc700 1 -- 172.21.15.89:0/74021253 learned_addr learned my addr 172.21.15.89:0/74021253 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:07.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f895b7fe700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.292+0000 7f895affd700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c06a770 0x7f895c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f895a7fc700 1 -- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c06a770 msgr2=0x7f895c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:07.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f895a7fc700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c06a770 0x7f895c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f895a7fc700 1 -- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 msgr2=0x7f895c13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:07.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f895a7fc700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f895a7fc700 1 -- 172.21.15.89:0/74021253 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c09e920 con 0x7f895c09dc30 2023-09-30T22:19:07.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f895a7fc700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c09dc30 0x7f895c09e080 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8950009920 tx=0x7f8950009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6371080495c84d0c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:07.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f8959ffb700 1 -- 172.21.15.89:0/74021253 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950011830 con 0x7f895c09dc30 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f8959ffb700 1 -- 172.21.15.89:0/74021253 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8950011990 con 0x7f895c09dc30 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.293+0000 7f8959ffb700 1 -- 172.21.15.89:0/74021253 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950011d20 con 0x7f895c09dc30 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 -- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c09dc30 msgr2=0x7f895c09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c09dc30 0x7f895c09e080 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8950009920 tx=0x7f8950009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 -- 172.21.15.89:0/74021253 shutdown_connections 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c06a770 0x7f895c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 --2- 172.21.15.89:0/74021253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c09dc30 0x7f895c09e080 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 -- 172.21.15.89:0/74021253 >> 172.21.15.89:0/74021253 conn(0x7f895c066f50 msgr2=0x7f895c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 -- 172.21.15.89:0/74021253 shutdown_connections 2023-09-30T22:19:07.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.294+0000 7f8961793700 1 -- 172.21.15.89:0/74021253 wait complete. 2023-09-30T22:19:07.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.295+0000 7f8961793700 1 Processor -- start 2023-09-30T22:19:07.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.295+0000 7f8961793700 1 -- start start 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f8961793700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c06a770 0x7f895c13bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f8961793700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c09dc30 0x7f895c13a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f8961793700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f8961793700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0723b0 con 0x7f895c06a770 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f8961793700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0721c0 con 0x7f895c13c1e0 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f8961793700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0a1b50 con 0x7f895c09dc30 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f895affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c06a770 0x7f895c13bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f895affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c06a770 0x7f895c13bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42698/0 (socket says 172.21.15.89:42698) 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f895affd700 1 -- 172.21.15.89:0/405258381 learned_addr learned my addr 172.21.15.89:0/405258381 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.296+0000 7f895b7fe700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f895affd700 1 -- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c09dc30 msgr2=0x7f895c13a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f895affd700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c09dc30 0x7f895c13a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f895affd700 1 -- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 msgr2=0x7f895c13aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f895affd700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f895affd700 1 -- 172.21.15.89:0/405258381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009550 con 0x7f895c06a770 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f895b7fe700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f895affd700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c06a770 0x7f895c13bc20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f894c00e9b0 tx=0x7f894c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f89437fe700 1 -- 172.21.15.89:0/405258381 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00c850 con 0x7f895c06a770 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f89437fe700 1 -- 172.21.15.89:0/405258381 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c00c9b0 con 0x7f895c06a770 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.297+0000 7f89437fe700 1 -- 172.21.15.89:0/405258381 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00cd20 con 0x7f895c06a770 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.298+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c09f1a0 con 0x7f895c06a770 2023-09-30T22:19:07.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.298+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c11f320 con 0x7f895c06a770 2023-09-30T22:19:07.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.299+0000 7f89437fe700 1 -- 172.21.15.89:0/405258381 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f894c013470 con 0x7f895c06a770 2023-09-30T22:19:07.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.299+0000 7f89437fe700 1 -- 172.21.15.89:0/405258381 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f894c03f790 con 0x7f895c06a770 2023-09-30T22:19:07.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.299+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c11cc30 con 0x7f895c06a770 2023-09-30T22:19:07.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.308+0000 7f89437fe700 1 -- 172.21.15.89:0/405258381 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f894c021080 con 0x7f895c06a770 2023-09-30T22:19:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.328+0000 7f7e8001b700 1 Processor -- start 2023-09-30T22:19:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.328+0000 7f7e8001b700 1 -- start start 2023-09-30T22:19:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e8001b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7e7809d6f0 0x7f7e7809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e8001b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e000 0x7f7e78130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e8001b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7809f130 con 0x7f7e7809e410 2023-09-30T22:19:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e8001b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7809f520 con 0x7f7e7809d6f0 2023-09-30T22:19:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e8001b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7809e930 con 0x7f7e7809e000 2023-09-30T22:19:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e7ddb7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7e7809d6f0 0x7f7e7809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e7ddb7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7e7809d6f0 0x7f7e7809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42702/0 (socket says 172.21.15.89:42702) 2023-09-30T22:19:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e7ddb7700 1 -- 172.21.15.89:0/2020095639 learned_addr learned my addr 172.21.15.89:0/2020095639 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.329+0000 7f7e7d5b6700 1 --2- 172.21.15.89:0/2020095639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e000 0x7f7e78130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.330+0000 7f7e7d5b6700 1 -- 172.21.15.89:0/2020095639 >> v2:172.21.15.89:3300/0 conn(0x7f7e7809d6f0 msgr2=0x7f7e7809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.330+0000 7f7e7d5b6700 1 --2- 172.21.15.89:0/2020095639 >> v2:172.21.15.89:3300/0 conn(0x7f7e7809d6f0 0x7f7e7809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.330+0000 7f7e7d5b6700 1 -- 172.21.15.89:0/2020095639 >> v1:172.21.15.89:6789/0 conn(0x7f7e7809e410 legacy=0x7f7e78130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.330+0000 7f7e7d5b6700 1 -- 172.21.15.89:0/2020095639 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7809b2d0 con 0x7f7e7809e000 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.330+0000 7f7e7d5b6700 1 --2- 172.21.15.89:0/2020095639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e000 0x7f7e78130660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009840 tx=0x7f7e68009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b131d22e36f70e3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.330+0000 7f7e7cdb5700 1 -- 172.21.15.89:0/2020095639 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68011830 con 0x7f7e7809e000 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.330+0000 7f7e7cdb5700 1 -- 172.21.15.89:0/2020095639 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68011990 con 0x7f7e7809e000 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.330+0000 7f7e7cdb5700 1 -- 172.21.15.89:0/2020095639 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68011d20 con 0x7f7e7809e000 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.331+0000 7f7e8001b700 1 -- 172.21.15.89:0/2020095639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e000 msgr2=0x7f7e78130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.331+0000 7f7e8001b700 1 --2- 172.21.15.89:0/2020095639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e000 0x7f7e78130660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009840 tx=0x7f7e68009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.331+0000 7f7e8001b700 1 -- 172.21.15.89:0/2020095639 shutdown_connections 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.331+0000 7f7e8001b700 1 --2- 172.21.15.89:0/2020095639 >> v2:172.21.15.89:3300/0 conn(0x7f7e7809d6f0 0x7f7e7809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.331+0000 7f7e8001b700 1 --2- 172.21.15.89:0/2020095639 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e000 0x7f7e78130660 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.331+0000 7f7e8001b700 1 -- 172.21.15.89:0/2020095639 >> 172.21.15.89:0/2020095639 conn(0x7f7e78066f50 msgr2=0x7f7e7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.331+0000 7f7e8001b700 1 -- 172.21.15.89:0/2020095639 shutdown_connections 2023-09-30T22:19:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.331+0000 7f7e8001b700 1 -- 172.21.15.89:0/2020095639 wait complete. 2023-09-30T22:19:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.333+0000 7f7e8001b700 1 Processor -- start 2023-09-30T22:19:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.333+0000 7f7e8001b700 1 -- start start 2023-09-30T22:19:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.333+0000 7f7e8001b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7809d6f0 0x7f7e781c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.333+0000 7f7e8001b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e7809e000 0x7f7e781c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.333+0000 7f7e8001b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e410 0x7f7e781c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.333+0000 7f7e8001b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78130d90 con 0x7f7e7809e000 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.333+0000 7f7e8001b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e780722e0 con 0x7f7e7809e410 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.333+0000 7f7e8001b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7809c700 con 0x7f7e7809d6f0 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e7809e000 0x7f7e781c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7ddb7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7809d6f0 0x7f7e781c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e7809e000 0x7f7e781c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42706/0 (socket says 172.21.15.89:42706) 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 -- 172.21.15.89:0/1507098152 learned_addr learned my addr 172.21.15.89:0/1507098152 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7e5b8700 1 --2- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e410 0x7f7e781c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 -- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7809d6f0 msgr2=0x7f7e781c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 --2- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7809d6f0 0x7f7e781c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 -- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e410 msgr2=0x7f7e781c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 --2- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e410 0x7f7e781c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 -- 172.21.15.89:0/1507098152 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680094a0 con 0x7f7e7809e000 2023-09-30T22:19:07.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7f7e7d5b6700 1 --2- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e7809e000 0x7f7e781c8bf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009fd0 tx=0x7f7e68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:07.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.334+0000 7fdbb5519700 1 Processor -- start 2023-09-30T22:19:07.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbb5519700 1 -- start start 2023-09-30T22:19:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7f7e6e7fc700 1 -- 172.21.15.89:0/1507098152 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680036a0 con 0x7f7e7809e000 2023-09-30T22:19:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7f7e6e7fc700 1 -- 172.21.15.89:0/1507098152 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6801f070 con 0x7f7e7809e000 2023-09-30T22:19:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7f7e6e7fc700 1 -- 172.21.15.89:0/1507098152 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68003a20 con 0x7f7e7809e000 2023-09-30T22:19:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7f7e8001b700 1 -- 172.21.15.89:0/1507098152 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e78131b60 con 0x7f7e7809e000 2023-09-30T22:19:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7f7e8001b700 1 -- 172.21.15.89:0/1507098152 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e78130740 con 0x7f7e7809e000 2023-09-30T22:19:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbb5519700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdbb01316d0 0x7fdbb0131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbb5519700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb006e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbb5519700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb006ea90 0x7fdbb006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbb5519700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb009a5f0 con 0x7fdbb01316d0 2023-09-30T22:19:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbb5519700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb0131510 con 0x7fdbb006ea90 2023-09-30T22:19:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbb5519700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb0071670 con 0x7fdbb0131fe0 2023-09-30T22:19:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbae7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb006e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbae7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb006e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54536/0 (socket says 172.21.15.89:54536) 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.335+0000 7fdbae7fc700 1 -- 172.21.15.89:0/3110665713 learned_addr learned my addr 172.21.15.89:0/3110665713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaeffd700 1 --2- 172.21.15.89:0/3110665713 >> v2:172.21.15.89:3300/0 conn(0x7fdbb01316d0 0x7fdbb0131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaf7fe700 1 --2- 172.21.15.89:0/3110665713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb006ea90 0x7fdbb006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaf7fe700 1 -- 172.21.15.89:0/3110665713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 msgr2=0x7fdbb006e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaf7fe700 1 --2- 172.21.15.89:0/3110665713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb006e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaf7fe700 1 -- 172.21.15.89:0/3110665713 >> v2:172.21.15.89:3300/0 conn(0x7fdbb01316d0 msgr2=0x7fdbb0131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaf7fe700 1 --2- 172.21.15.89:0/3110665713 >> v2:172.21.15.89:3300/0 conn(0x7fdbb01316d0 0x7fdbb0131aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaf7fe700 1 -- 172.21.15.89:0/3110665713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb0099e70 con 0x7fdbb006ea90 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaeffd700 1 --2- 172.21.15.89:0/3110665713 >> v2:172.21.15.89:3300/0 conn(0x7fdbb01316d0 0x7fdbb0131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7fdbaf7fe700 1 --2- 172.21.15.89:0/3110665713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb006ea90 0x7fdbb006eee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009d90 tx=0x7fdba800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd1951e5de51d26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.336+0000 7f7e5bfff700 1 -- 172.21.15.89:0/1507098152 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e540052e0 con 0x7f7e7809e000 2023-09-30T22:19:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.337+0000 7f7e6e7fc700 1 -- 172.21.15.89:0/1507098152 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7e68003b80 con 0x7f7e7809e000 2023-09-30T22:19:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.337+0000 7f7e6e7fc700 1 -- 172.21.15.89:0/1507098152 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7e68021070 con 0x7f7e7809e000 2023-09-30T22:19:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.340+0000 7fdbadffb700 1 -- 172.21.15.89:0/3110665713 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800e070 con 0x7fdbb006ea90 2023-09-30T22:19:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.341+0000 7fdbadffb700 1 -- 172.21.15.89:0/3110665713 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba800bb20 con 0x7fdbb006ea90 2023-09-30T22:19:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.341+0000 7fdbadffb700 1 -- 172.21.15.89:0/3110665713 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8012430 con 0x7fdbb006ea90 2023-09-30T22:19:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.341+0000 7fdbb5519700 1 -- 172.21.15.89:0/3110665713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb006ea90 msgr2=0x7fdbb006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.341+0000 7fdbb5519700 1 --2- 172.21.15.89:0/3110665713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb006ea90 0x7fdbb006eee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009d90 tx=0x7fdba800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.341+0000 7f7e6e7fc700 1 -- 172.21.15.89:0/1507098152 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e68003e30 con 0x7f7e7809e000 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.342+0000 7fdbb5519700 1 -- 172.21.15.89:0/3110665713 shutdown_connections 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.342+0000 7fdbb5519700 1 --2- 172.21.15.89:0/3110665713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb006ea90 0x7fdbb006eee0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.342+0000 7fdbb5519700 1 --2- 172.21.15.89:0/3110665713 >> v2:172.21.15.89:3300/0 conn(0x7fdbb01316d0 0x7fdbb0131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.342+0000 7fdbb5519700 1 --2- 172.21.15.89:0/3110665713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb006e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.342+0000 7fdbb5519700 1 -- 172.21.15.89:0/3110665713 >> 172.21.15.89:0/3110665713 conn(0x7fdbb00a2aa0 msgr2=0x7fdbb00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.342+0000 7fdbb5519700 1 -- 172.21.15.89:0/3110665713 shutdown_connections 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.342+0000 7fdbb5519700 1 -- 172.21.15.89:0/3110665713 wait complete. 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbb5519700 1 Processor -- start 2023-09-30T22:19:07.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbb5519700 1 -- start start 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbb5519700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb006ea90 0x7fdbb009d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbb5519700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb01316d0 0x7fdbb009da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbb5519700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb01cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbb5519700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb0127730 con 0x7fdbb006ea90 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbb5519700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb009e6c0 con 0x7fdbb01316d0 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbb5519700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb0128b90 con 0x7fdbb0131fe0 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.343+0000 7fdbae7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb01316d0 0x7fdbb009da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbae7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb01316d0 0x7fdbb009da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55856/0 (socket says 172.21.15.89:55856) 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbae7fc700 1 -- 172.21.15.89:0/2925369162 learned_addr learned my addr 172.21.15.89:0/2925369162 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbaf7fe700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb01cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbaeffd700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb006ea90 0x7fdbb009d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbae7fc700 1 -- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 msgr2=0x7fdbb01cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbae7fc700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb01cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbae7fc700 1 -- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb006ea90 msgr2=0x7fdbb009d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbae7fc700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb006ea90 0x7fdbb009d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbae7fc700 1 -- 172.21.15.89:0/2925369162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8009a70 con 0x7fdbb01316d0 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbae7fc700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb01316d0 0x7fdbb009da50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdba400e910 tx=0x7fdba400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdb977fe700 1 -- 172.21.15.89:0/2925369162 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40096a0 con 0x7fdbb01316d0 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdb977fe700 1 -- 172.21.15.89:0/2925369162 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba400ee60 con 0x7fdbb01316d0 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb009d120 con 0x7fdbb01316d0 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdb977fe700 1 -- 172.21.15.89:0/2925369162 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400ca90 con 0x7fdbb01316d0 2023-09-30T22:19:07.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.344+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb009df90 con 0x7fdbb01316d0 2023-09-30T22:19:07.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.345+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb0074ea0 con 0x7fdbb01316d0 2023-09-30T22:19:07.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.346+0000 7fdb977fe700 1 -- 172.21.15.89:0/2925369162 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdba400cbf0 con 0x7fdbb01316d0 2023-09-30T22:19:07.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.346+0000 7fdb977fe700 1 -- 172.21.15.89:0/2925369162 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdba403f660 con 0x7fdbb01316d0 2023-09-30T22:19:07.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.350+0000 7fdb977fe700 1 -- 172.21.15.89:0/2925369162 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdba4013470 con 0x7fdbb01316d0 2023-09-30T22:19:07.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.470+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbc70122f70 con 0x7fbc701338c0 2023-09-30T22:19:07.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.471+0000 7fbc6cff9700 1 -- 172.21.15.89:0/788262878 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc64015020 con 0x7fbc701338c0 2023-09-30T22:19:07.472 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701338c0 msgr2=0x7fbc7011fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701338c0 0x7fbc7011fda0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc640090b0 tx=0x7fbc64004750 comp rx=0 tx=0).stop 2023-09-30T22:19:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 shutdown_connections 2023-09-30T22:19:07.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc701341d0 0x7fbc701202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc7009e390 0x7fbc7011f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 --2- 172.21.15.89:0/788262878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc701338c0 0x7fbc7011fda0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 >> 172.21.15.89:0/788262878 conn(0x7fbc700a2aa0 msgr2=0x7fbc7009f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:07.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 shutdown_connections 2023-09-30T22:19:07.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.473+0000 7fbc764eb700 1 -- 172.21.15.89:0/788262878 wait complete. 2023-09-30T22:19:07.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:07.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.546+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdbb006b9e0 con 0x7fdbb01316d0 2023-09-30T22:19:07.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.547+0000 7fdb977fe700 1 -- 172.21.15.89:0/2925369162 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdba4021020 con 0x7fdbb01316d0 2023-09-30T22:19:07.548 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:07.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb01316d0 msgr2=0x7fdbb009da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb01316d0 0x7fdbb009da50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdba400e910 tx=0x7fdba400ec20 comp rx=0 tx=0).stop 2023-09-30T22:19:07.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 shutdown_connections 2023-09-30T22:19:07.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdbb0131fe0 0x7fdbb01cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb006ea90 0x7fdbb009d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 --2- 172.21.15.89:0/2925369162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdbb01316d0 0x7fdbb009da50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 >> 172.21.15.89:0/2925369162 conn(0x7fdbb00a2aa0 msgr2=0x7fdbb0125f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:07.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 shutdown_connections 2023-09-30T22:19:07.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.550+0000 7fdbb5519700 1 -- 172.21.15.89:0/2925369162 wait complete. 2023-09-30T22:19:07.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.576+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f895c074ea0 con 0x7f895c06a770 2023-09-30T22:19:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.576+0000 7f89437fe700 1 -- 172.21.15.89:0/405258381 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f894c017020 con 0x7f895c06a770 2023-09-30T22:19:07.577 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c06a770 msgr2=0x7f895c13bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c06a770 0x7f895c13bc20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f894c00e9b0 tx=0x7f894c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 shutdown_connections 2023-09-30T22:19:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f895c13c1e0 0x7f895c13aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f895c06a770 0x7f895c13bc20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 --2- 172.21.15.89:0/405258381 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f895c09dc30 0x7f895c13a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 >> 172.21.15.89:0/405258381 conn(0x7f895c066f50 msgr2=0x7f895c0a2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 shutdown_connections 2023-09-30T22:19:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.578+0000 7f8961793700 1 -- 172.21.15.89:0/405258381 wait complete. 2023-09-30T22:19:07.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:07.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.593+0000 7f7e5bfff700 1 -- 172.21.15.89:0/1507098152 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7e54005860 con 0x7f7e7809e000 2023-09-30T22:19:07.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.593+0000 7f7e6e7fc700 1 -- 172.21.15.89:0/1507098152 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e68034020 con 0x7f7e7809e000 2023-09-30T22:19:07.594 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:07.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.595+0000 7f7e5bfff700 1 -- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e7809e000 msgr2=0x7f7e781c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:07.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.595+0000 7f7e5bfff700 1 --2- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e7809e000 0x7f7e781c8bf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009fd0 tx=0x7f7e68004750 comp rx=0 tx=0).stop 2023-09-30T22:19:07.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.595+0000 7f7e5bfff700 1 -- 172.21.15.89:0/1507098152 shutdown_connections 2023-09-30T22:19:07.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.595+0000 7f7e5bfff700 1 --2- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e7809e410 0x7f7e781c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.595+0000 7f7e5bfff700 1 --2- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7809d6f0 0x7f7e781c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.595+0000 7f7e5bfff700 1 --2- 172.21.15.89:0/1507098152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e7809e000 0x7f7e781c8bf0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:07.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.595+0000 7f7e5bfff700 1 -- 172.21.15.89:0/1507098152 >> 172.21.15.89:0/1507098152 conn(0x7f7e78066f50 msgr2=0x7f7e780a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:07.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.596+0000 7f7e5bfff700 1 -- 172.21.15.89:0/1507098152 shutdown_connections 2023-09-30T22:19:07.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:07.596+0000 7f7e5bfff700 1 -- 172.21.15.89:0/1507098152 wait complete. 2023-09-30T22:19:07.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:08.487 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:08.568 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:08.592 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:08.608 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:08.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45fe2ca700 1 Processor -- start 2023-09-30T22:19:08.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45fe2ca700 1 -- start start 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45fe2ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45fe2ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f45f809e000 0x7f45f8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45fe2ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45fe2ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f809f520 con 0x7f45f809e000 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45fe2ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f809f130 con 0x7f45f809d6f0 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45fe2ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45f809e930 con 0x7f45f809e410 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45f7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45f7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55870/0 (socket says 172.21.15.89:55870) 2023-09-30T22:19:08.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.623+0000 7f45f7fff700 1 -- 172.21.15.89:0/3630453116 learned_addr learned my addr 172.21.15.89:0/3630453116 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45fc867700 1 --2- 172.21.15.89:0/3630453116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f7fff700 1 -- 172.21.15.89:0/3630453116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 msgr2=0x7f45f8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f7fff700 1 --2- 172.21.15.89:0/3630453116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f7fff700 1 -- 172.21.15.89:0/3630453116 >> v2:172.21.15.89:3300/0 conn(0x7f45f809e000 msgr2=0x7f45f8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f7fff700 1 --2- 172.21.15.89:0/3630453116 >> v2:172.21.15.89:3300/0 conn(0x7f45f809e000 0x7f45f8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f7fff700 1 -- 172.21.15.89:0/3630453116 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f809b2d0 con 0x7f45f809d6f0 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f7fff700 1 --2- 172.21.15.89:0/3630453116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f809dac0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f45e800bb90 tx=0x7f45e800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bbb21cb6663977b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f6ffd700 1 -- 172.21.15.89:0/3630453116 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e800dd30 con 0x7f45f809d6f0 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f6ffd700 1 -- 172.21.15.89:0/3630453116 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e800de90 con 0x7f45f809d6f0 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.624+0000 7f45f6ffd700 1 -- 172.21.15.89:0/3630453116 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8010690 con 0x7f45f809d6f0 2023-09-30T22:19:08.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 -- 172.21.15.89:0/3630453116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 msgr2=0x7f45f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 --2- 172.21.15.89:0/3630453116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f809dac0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f45e800bb90 tx=0x7f45e800bea0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 -- 172.21.15.89:0/3630453116 shutdown_connections 2023-09-30T22:19:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 --2- 172.21.15.89:0/3630453116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 --2- 172.21.15.89:0/3630453116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f809dac0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 --2- 172.21.15.89:0/3630453116 >> v2:172.21.15.89:3300/0 conn(0x7f45f809e000 0x7f45f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 -- 172.21.15.89:0/3630453116 >> 172.21.15.89:0/3630453116 conn(0x7f45f8066f50 msgr2=0x7f45f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 -- 172.21.15.89:0/3630453116 shutdown_connections 2023-09-30T22:19:08.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.625+0000 7f45fe2ca700 1 -- 172.21.15.89:0/3630453116 wait complete. 2023-09-30T22:19:08.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.626+0000 7f45fe2ca700 1 Processor -- start 2023-09-30T22:19:08.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.626+0000 7f45fe2ca700 1 -- start start 2023-09-30T22:19:08.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fe2ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f811e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fe2ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f809e000 0x7f45f811e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fe2ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f811ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fe2ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f8121a30 con 0x7f45f809e000 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fe2ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f80a1a70 con 0x7f45f809d6f0 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fe2ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45f811ff10 con 0x7f45f809e410 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f811ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f811ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54544/0 (socket says 172.21.15.89:54544) 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 -- 172.21.15.89:0/825277464 learned_addr learned my addr 172.21.15.89:0/825277464 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 -- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 msgr2=0x7f45f811e1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 --2- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f811e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 -- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f809e000 msgr2=0x7f45f811e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 --2- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f809e000 0x7f45f811e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 -- 172.21.15.89:0/825277464 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e800b7f0 con 0x7f45f809e410 2023-09-30T22:19:08.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.627+0000 7f45fc867700 1 --2- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f811ec70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f45f000efc0 tx=0x7f45f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:08.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.628+0000 7f45f4ff9700 1 -- 172.21.15.89:0/825277464 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000cd40 con 0x7f45f809e410 2023-09-30T22:19:08.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.628+0000 7f45f4ff9700 1 -- 172.21.15.89:0/825277464 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f000f040 con 0x7f45f809e410 2023-09-30T22:19:08.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.628+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f809c760 con 0x7f45f809e410 2023-09-30T22:19:08.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.628+0000 7f45f4ff9700 1 -- 172.21.15.89:0/825277464 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f00146c0 con 0x7f45f809e410 2023-09-30T22:19:08.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.628+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f8121d10 con 0x7f45f809e410 2023-09-30T22:19:08.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.629+0000 7f45f4ff9700 1 -- 172.21.15.89:0/825277464 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f45f0014820 con 0x7f45f809e410 2023-09-30T22:19:08.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.629+0000 7f45f4ff9700 1 -- 172.21.15.89:0/825277464 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f45f003ffc0 con 0x7f45f809e410 2023-09-30T22:19:08.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.633+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f8074ea0 con 0x7f45f809e410 2023-09-30T22:19:08.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.638+0000 7f45f4ff9700 1 -- 172.21.15.89:0/825277464 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f45f0018020 con 0x7f45f809e410 2023-09-30T22:19:08.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a6771d700 1 Processor -- start 2023-09-30T22:19:08.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a6771d700 1 -- start start 2023-09-30T22:19:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a6771d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a0a30 0x7f7a600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a6771d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a60134170 0x7f7a6013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a6771d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6009edd0 con 0x7f7a600a1340 2023-09-30T22:19:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a6771d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6009e1f0 con 0x7f7a60134170 2023-09-30T22:19:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a6771d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6009dc50 con 0x7f7a600a0a30 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a64cb8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7a600a1340 0x7f7a60133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50170/0 (socket says 172.21.15.89:50170) 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a654b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a0a30 0x7f7a600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.701+0000 7f7a64cb8700 1 -- 172.21.15.89:0/3015188702 learned_addr learned my addr 172.21.15.89:0/3015188702 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a65cba700 1 --2- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a60134170 0x7f7a6013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a654b9700 1 -- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a60134170 msgr2=0x7f7a6013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a654b9700 1 --2- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a60134170 0x7f7a6013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a654b9700 1 -- 172.21.15.89:0/3015188702 >> v1:172.21.15.89:6789/0 conn(0x7f7a600a1340 legacy=0x7f7a60133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a654b9700 1 -- 172.21.15.89:0/3015188702 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a600a1fe0 con 0x7f7a600a0a30 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a65cba700 1 --2- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a60134170 0x7f7a6013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a654b9700 1 --2- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a0a30 0x7f7a600a0e00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009870 tx=0x7f7a5c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=759fd5cdca9df872 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a57fff700 1 -- 172.21.15.89:0/3015188702 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c011830 con 0x7f7a600a0a30 2023-09-30T22:19:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a57fff700 1 -- 172.21.15.89:0/3015188702 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c011990 con 0x7f7a600a0a30 2023-09-30T22:19:08.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.702+0000 7f7a57fff700 1 -- 172.21.15.89:0/3015188702 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c011d20 con 0x7f7a600a0a30 2023-09-30T22:19:08.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.703+0000 7f7a6771d700 1 -- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a0a30 msgr2=0x7f7a600a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.703+0000 7f7a6771d700 1 --2- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a0a30 0x7f7a600a0e00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009870 tx=0x7f7a5c009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.703+0000 7f7a6771d700 1 -- 172.21.15.89:0/3015188702 shutdown_connections 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.703+0000 7f7a6771d700 1 --2- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a60134170 0x7f7a6013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.703+0000 7f7a6771d700 1 --2- 172.21.15.89:0/3015188702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a0a30 0x7f7a600a0e00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.703+0000 7f7a6771d700 1 -- 172.21.15.89:0/3015188702 >> 172.21.15.89:0/3015188702 conn(0x7f7a600a6550 msgr2=0x7f7a600a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.703+0000 7f7a6771d700 1 -- 172.21.15.89:0/3015188702 shutdown_connections 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.703+0000 7f7a6771d700 1 -- 172.21.15.89:0/3015188702 wait complete. 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a6771d700 1 Processor -- start 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a6771d700 1 -- start start 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a6771d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a600a0a30 0x7f7a60123290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a6771d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a1340 0x7f7a601237d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a6771d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a60134170 0x7f7a601c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a6771d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a60122eb0 con 0x7f7a60134170 2023-09-30T22:19:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a6771d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a60133400 con 0x7f7a600a0a30 2023-09-30T22:19:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a6771d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a60126500 con 0x7f7a600a1340 2023-09-30T22:19:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a64cb8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a1340 0x7f7a601237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a64cb8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a1340 0x7f7a601237d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54548/0 (socket says 172.21.15.89:54548) 2023-09-30T22:19:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.705+0000 7f7a64cb8700 1 -- 172.21.15.89:0/2404971435 learned_addr learned my addr 172.21.15.89:0/2404971435 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:08.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a654b9700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a600a0a30 0x7f7a60123290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a65cba700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a60134170 0x7f7a601c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a64cb8700 1 -- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a600a0a30 msgr2=0x7f7a60123290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a64cb8700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a600a0a30 0x7f7a60123290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a64cb8700 1 -- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a60134170 msgr2=0x7f7a601c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a64cb8700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a60134170 0x7f7a601c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a64cb8700 1 -- 172.21.15.89:0/2404971435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a58009a70 con 0x7f7a600a1340 2023-09-30T22:19:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a64cb8700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a1340 0x7f7a601237d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000c520 tx=0x7f7a5000c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a55ffb700 1 -- 172.21.15.89:0/2404971435 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50005d70 con 0x7f7a600a1340 2023-09-30T22:19:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a55ffb700 1 -- 172.21.15.89:0/2404971435 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a500070d0 con 0x7f7a600a1340 2023-09-30T22:19:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c0094a0 con 0x7f7a600a1340 2023-09-30T22:19:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.706+0000 7f7a55ffb700 1 -- 172.21.15.89:0/2404971435 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50007440 con 0x7f7a600a1340 2023-09-30T22:19:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.707+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a600a25e0 con 0x7f7a600a1340 2023-09-30T22:19:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.708+0000 7f7a55ffb700 1 -- 172.21.15.89:0/2404971435 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a500075a0 con 0x7f7a600a1340 2023-09-30T22:19:08.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.708+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a60074ea0 con 0x7f7a600a1340 2023-09-30T22:19:08.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.708+0000 7f7a55ffb700 1 -- 172.21.15.89:0/2404971435 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7a5003ea70 con 0x7f7a600a1340 2023-09-30T22:19:08.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.716+0000 7f7a55ffb700 1 -- 172.21.15.89:0/2404971435 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a50040e60 con 0x7f7a600a1340 2023-09-30T22:19:08.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2f7c9700 1 Processor -- start 2023-09-30T22:19:08.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2f7c9700 1 -- start start 2023-09-30T22:19:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2f7c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28133420 0x7fda281337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2f7c9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda28133e30 0x7fda2812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2f7c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28134260 0x7fda281304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2f7c9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2806b990 con 0x7fda28133e30 2023-09-30T22:19:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2f7c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda280a0590 con 0x7fda28133420 2023-09-30T22:19:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2f7c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda2809fb80 con 0x7fda28134260 2023-09-30T22:19:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.741+0000 7fda2dd66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28134260 0x7fda281304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2dd66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28134260 0x7fda281304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54562/0 (socket says 172.21.15.89:54562) 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2d565700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28133420 0x7fda281337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2dd66700 1 -- 172.21.15.89:0/4000343298 learned_addr learned my addr 172.21.15.89:0/4000343298 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2cd64700 1 --2- 172.21.15.89:0/4000343298 >> v2:172.21.15.89:3300/0 conn(0x7fda28133e30 0x7fda2812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2cd64700 1 -- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28134260 msgr2=0x7fda281304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2cd64700 1 --2- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28134260 0x7fda281304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2cd64700 1 -- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28133420 msgr2=0x7fda281337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2cd64700 1 --2- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28133420 0x7fda281337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2cd64700 1 -- 172.21.15.89:0/4000343298 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280a0030 con 0x7fda28133e30 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2d565700 1 --2- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28133420 0x7fda281337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda2cd64700 1 --2- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 0x7fda2812ff90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fda18009840 tx=0x7fda18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdd52fef975dd57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda1ffff700 1 -- 172.21.15.89:0/4000343298 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18011830 con 0x7fda28133e30 2023-09-30T22:19:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda1ffff700 1 -- 172.21.15.89:0/4000343298 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18011990 con 0x7fda28133e30 2023-09-30T22:19:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.742+0000 7fda1ffff700 1 -- 172.21.15.89:0/4000343298 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18011d20 con 0x7fda28133e30 2023-09-30T22:19:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 -- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 msgr2=0x7fda2812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 --2- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 0x7fda2812ff90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fda18009840 tx=0x7fda18009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 -- 172.21.15.89:0/4000343298 shutdown_connections 2023-09-30T22:19:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 --2- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28134260 0x7fda281304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 --2- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28133420 0x7fda281337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 --2- 172.21.15.89:0/4000343298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 0x7fda2812ff90 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 -- 172.21.15.89:0/4000343298 >> 172.21.15.89:0/4000343298 conn(0x7fda280a4210 msgr2=0x7fda280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:08.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7f3a7003a700 1 Processor -- start 2023-09-30T22:19:08.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 -- 172.21.15.89:0/4000343298 shutdown_connections 2023-09-30T22:19:08.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7fda2f7c9700 1 -- 172.21.15.89:0/4000343298 wait complete. 2023-09-30T22:19:08.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.743+0000 7f3a7003a700 1 -- start start 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a7003a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a6806e570 0x7f3a6806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a7003a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a6809eca0 0x7f3a6806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a7003a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a68070ee0 con 0x7f3a68134610 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a7003a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a68071480 con 0x7f3a6806e570 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a7003a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6806f3c0 con 0x7f3a6809eca0 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6ddd6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a6806e570 0x7f3a6806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6ddd6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a6806e570 0x7f3a6806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42772/0 (socket says 172.21.15.89:42772) 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6ddd6700 1 -- 172.21.15.89:0/3315602687 learned_addr learned my addr 172.21.15.89:0/3315602687 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6d5d5700 1 --2- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a6809eca0 0x7f3a6806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6ddd6700 1 -- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a6809eca0 msgr2=0x7f3a6806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6ddd6700 1 --2- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a6809eca0 0x7f3a6806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6ddd6700 1 -- 172.21.15.89:0/3315602687 >> v1:172.21.15.89:6789/0 conn(0x7f3a68134610 legacy=0x7f3a68134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6ddd6700 1 -- 172.21.15.89:0/3315602687 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68071b90 con 0x7f3a6806e570 2023-09-30T22:19:08.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.744+0000 7f3a6d5d5700 1 --2- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a6809eca0 0x7f3a6806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:19:08.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7f3a6ddd6700 1 --2- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 0x7f3a6806e940 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3a640098f0 tx=0x7f3a64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f20e38f53f49790 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:08.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7fda2f7c9700 1 Processor -- start 2023-09-30T22:19:08.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7fda2f7c9700 1 -- start start 2023-09-30T22:19:08.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7f3a6cdd4700 1 -- 172.21.15.89:0/3315602687 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6400b8a0 con 0x7f3a6806e570 2023-09-30T22:19:08.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7f3a6cdd4700 1 -- 172.21.15.89:0/3315602687 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6400ba00 con 0x7f3a6806e570 2023-09-30T22:19:08.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7f3a6cdd4700 1 -- 172.21.15.89:0/3315602687 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6400bd90 con 0x7f3a6806e570 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7fda2f7c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28133420 0x7fda2809c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7fda2f7c9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 0x7fda2809c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7fda2f7c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28134260 0x7fda2809cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7fda2f7c9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2812da20 con 0x7fda28133e30 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7fda2f7c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2811ff20 con 0x7fda28134260 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.745+0000 7fda2f7c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda28120170 con 0x7fda28133420 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28133420 0x7fda2809c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28133420 0x7fda2809c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54570/0 (socket says 172.21.15.89:54570) 2023-09-30T22:19:08.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 -- 172.21.15.89:0/1140720212 learned_addr learned my addr 172.21.15.89:0/1140720212 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:08.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2dd66700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28134260 0x7fda2809cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7f3a7003a700 1 -- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 msgr2=0x7f3a6806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7f3a7003a700 1 --2- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 0x7f3a6806e940 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3a640098f0 tx=0x7f3a64009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:08.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7f3a7003a700 1 -- 172.21.15.89:0/3315602687 shutdown_connections 2023-09-30T22:19:08.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7f3a7003a700 1 --2- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 0x7f3a6806e940 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7f3a7003a700 1 --2- 172.21.15.89:0/3315602687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a6809eca0 0x7f3a6806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7f3a7003a700 1 -- 172.21.15.89:0/3315602687 >> 172.21.15.89:0/3315602687 conn(0x7f3a680a2aa0 msgr2=0x7f3a680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:08.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2cd64700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 0x7fda2809c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 -- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28134260 msgr2=0x7fda2809cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28134260 0x7fda2809cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 -- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 msgr2=0x7fda2809c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 0x7fda2809c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 -- 172.21.15.89:0/1140720212 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda240094a0 con 0x7fda28133420 2023-09-30T22:19:08.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda2d565700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28133420 0x7fda2809c160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fda2400ebe0 tx=0x7fda2400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:08.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7f3a7003a700 1 -- 172.21.15.89:0/3315602687 shutdown_connections 2023-09-30T22:19:08.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7f3a7003a700 1 -- 172.21.15.89:0/3315602687 wait complete. 2023-09-30T22:19:08.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.746+0000 7fda1dffb700 1 -- 172.21.15.89:0/1140720212 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400c850 con 0x7fda28133420 2023-09-30T22:19:08.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.747+0000 7fda1dffb700 1 -- 172.21.15.89:0/1140720212 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2400c9b0 con 0x7fda28133420 2023-09-30T22:19:08.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.747+0000 7fda1dffb700 1 -- 172.21.15.89:0/1140720212 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400cd40 con 0x7fda28133420 2023-09-30T22:19:08.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.747+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda180094a0 con 0x7fda28133420 2023-09-30T22:19:08.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.747+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda28131490 con 0x7fda28133420 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.747+0000 7f3a7003a700 1 Processor -- start 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.747+0000 7f3a7003a700 1 -- start start 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a7003a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 0x7f3a68122130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a7003a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a68134610 0x7f3a68122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a7003a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a68122bb0 0x7f3a681cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a7003a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a68125030 con 0x7f3a6806e570 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a7003a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a68124210 con 0x7f3a68122bb0 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a7003a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680a2320 con 0x7f3a68134610 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6d5d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a68134610 0x7f3a68122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6d5d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a68134610 0x7f3a68122670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54584/0 (socket says 172.21.15.89:54584) 2023-09-30T22:19:08.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6d5d5700 1 -- 172.21.15.89:0/2726647890 learned_addr learned my addr 172.21.15.89:0/2726647890 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:08.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7fda1dffb700 1 -- 172.21.15.89:0/1140720212 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda24013470 con 0x7fda28133420 2023-09-30T22:19:08.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6ddd6700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 0x7f3a68122130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6e5d7700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a68122bb0 0x7f3a681cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:08.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6d5d5700 1 -- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a68122bb0 msgr2=0x7f3a681cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6d5d5700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a68122bb0 0x7f3a681cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6d5d5700 1 -- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 msgr2=0x7f3a68122130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7f3a6d5d5700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 0x7f3a68122130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.749+0000 7f3a6d5d5700 1 -- 172.21.15.89:0/2726647890 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a64009550 con 0x7f3a68134610 2023-09-30T22:19:08.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.749+0000 7f3a6d5d5700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a68134610 0x7f3a68122670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800c770 tx=0x7f3a5800ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:08.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.748+0000 7fda1dffb700 1 -- 172.21.15.89:0/1140720212 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fda2403fc20 con 0x7fda28133420 2023-09-30T22:19:08.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.749+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda28074ea0 con 0x7fda28133420 2023-09-30T22:19:08.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.749+0000 7f3a5e7fc700 1 -- 172.21.15.89:0/2726647890 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58004220 con 0x7f3a68134610 2023-09-30T22:19:08.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.749+0000 7f3a5e7fc700 1 -- 172.21.15.89:0/2726647890 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58004380 con 0x7f3a68134610 2023-09-30T22:19:08.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.749+0000 7f3a5e7fc700 1 -- 172.21.15.89:0/2726647890 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a580036a0 con 0x7f3a68134610 2023-09-30T22:19:08.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.750+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a68070610 con 0x7f3a68134610 2023-09-30T22:19:08.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.750+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a681253a0 con 0x7f3a68134610 2023-09-30T22:19:08.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.751+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a68074ea0 con 0x7f3a68134610 2023-09-30T22:19:08.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.758+0000 7fda1dffb700 1 -- 172.21.15.89:0/1140720212 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda24017080 con 0x7fda28133420 2023-09-30T22:19:08.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.758+0000 7f3a5e7fc700 1 -- 172.21.15.89:0/2726647890 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3a58003800 con 0x7f3a68134610 2023-09-30T22:19:08.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.758+0000 7f3a5e7fc700 1 -- 172.21.15.89:0/2726647890 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3a5803fff0 con 0x7f3a68134610 2023-09-30T22:19:08.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.766+0000 7f3a5e7fc700 1 -- 172.21.15.89:0/2726647890 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a58021b00 con 0x7f3a68134610 2023-09-30T22:19:08.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.850+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f45f806c360 con 0x7f45f809e410 2023-09-30T22:19:08.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.850+0000 7f45f4ff9700 1 -- 172.21.15.89:0/825277464 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f45f001a680 con 0x7f45f809e410 2023-09-30T22:19:08.854 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:08.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.852+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 msgr2=0x7f45f811ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.852+0000 7f45fe2ca700 1 --2- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f811ec70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f45f000efc0 tx=0x7f45f000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.852+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 shutdown_connections 2023-09-30T22:19:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.852+0000 7f45fe2ca700 1 --2- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f45f809e410 0x7f45f811ec70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.852+0000 7f45fe2ca700 1 --2- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f45f809d6f0 0x7f45f811e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.852+0000 7f45fe2ca700 1 --2- 172.21.15.89:0/825277464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f809e000 0x7f45f811e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.852+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 >> 172.21.15.89:0/825277464 conn(0x7f45f8066f50 msgr2=0x7f45f806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.853+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 shutdown_connections 2023-09-30T22:19:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.853+0000 7f45fe2ca700 1 -- 172.21.15.89:0/825277464 wait complete. 2023-09-30T22:19:08.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.913+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7a600a28e0 con 0x7f7a600a1340 2023-09-30T22:19:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.914+0000 7f7a55ffb700 1 -- 172.21.15.89:0/2404971435 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a50017030 con 0x7f7a600a1340 2023-09-30T22:19:08.915 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a1340 msgr2=0x7f7a601237d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a1340 0x7f7a601237d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000c520 tx=0x7f7a5000c8e0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 shutdown_connections 2023-09-30T22:19:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a60134170 0x7f7a601c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a600a0a30 0x7f7a60123290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 --2- 172.21.15.89:0/2404971435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a600a1340 0x7f7a601237d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 >> 172.21.15.89:0/2404971435 conn(0x7f7a600a6550 msgr2=0x7f7a6011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 shutdown_connections 2023-09-30T22:19:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.916+0000 7f7a6771d700 1 -- 172.21.15.89:0/2404971435 wait complete. 2023-09-30T22:19:08.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.952+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fda2811f4b0 con 0x7fda28133420 2023-09-30T22:19:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.953+0000 7fda1dffb700 1 -- 172.21.15.89:0/1140720212 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda24021020 con 0x7fda28133420 2023-09-30T22:19:08.954 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28133420 msgr2=0x7fda2809c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28133420 0x7fda2809c160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fda2400ebe0 tx=0x7fda2400eef0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 shutdown_connections 2023-09-30T22:19:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda28134260 0x7fda2809cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda28133420 0x7fda2809c160 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 --2- 172.21.15.89:0/1140720212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda28133e30 0x7fda2809c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 >> 172.21.15.89:0/1140720212 conn(0x7fda280a4210 msgr2=0x7fda280a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 shutdown_connections 2023-09-30T22:19:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.955+0000 7fda2f7c9700 1 -- 172.21.15.89:0/1140720212 wait complete. 2023-09-30T22:19:08.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.963+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3a68071100 con 0x7f3a68134610 2023-09-30T22:19:08.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.963+0000 7f3a5e7fc700 1 -- 172.21.15.89:0/2726647890 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a58027020 con 0x7f3a68134610 2023-09-30T22:19:08.964 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a68134610 msgr2=0x7f3a68122670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a68134610 0x7f3a68122670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800c770 tx=0x7f3a5800ca80 comp rx=0 tx=0).stop 2023-09-30T22:19:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 shutdown_connections 2023-09-30T22:19:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a68122bb0 0x7f3a681cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a6806e570 0x7f3a68122130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 --2- 172.21.15.89:0/2726647890 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a68134610 0x7f3a68122670 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 >> 172.21.15.89:0/2726647890 conn(0x7f3a680a2aa0 msgr2=0x7f3a680a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 shutdown_connections 2023-09-30T22:19:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:08.965+0000 7f3a7003a700 1 -- 172.21.15.89:0/2726647890 wait complete. 2023-09-30T22:19:08.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:08.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:09.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:09.930 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:09.969 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:09.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:10.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.005+0000 7f326d79d700 1 Processor -- start 2023-09-30T22:19:10.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.005+0000 7f326d79d700 1 -- start start 2023-09-30T22:19:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f326d79d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32681316c0 0x7f3268131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f326d79d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326806ae80 0x7f3268139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f326d79d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326809e130 con 0x7f326809d9f0 2023-09-30T22:19:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f326d79d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32681312b0 con 0x7f32681316c0 2023-09-30T22:19:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f326d79d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f326809f6e0 con 0x7f326806ae80 2023-09-30T22:19:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f32677fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326806ae80 0x7f3268139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f3266ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32681316c0 0x7f3268131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f32677fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326806ae80 0x7f3268139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54586/0 (socket says 172.21.15.89:54586) 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f32677fe700 1 -- 172.21.15.89:0/211850205 learned_addr learned my addr 172.21.15.89:0/211850205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f32677fe700 1 -- 172.21.15.89:0/211850205 >> v2:172.21.15.89:3300/0 conn(0x7f32681316c0 msgr2=0x7f3268131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f32677fe700 1 --2- 172.21.15.89:0/211850205 >> v2:172.21.15.89:3300/0 conn(0x7f32681316c0 0x7f3268131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f32677fe700 1 -- 172.21.15.89:0/211850205 >> v1:172.21.15.89:6789/0 conn(0x7f326809d9f0 legacy=0x7f326806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f32677fe700 1 -- 172.21.15.89:0/211850205 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326809e760 con 0x7f326806ae80 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f3266ffd700 1 --2- 172.21.15.89:0/211850205 >> v2:172.21.15.89:3300/0 conn(0x7f32681316c0 0x7f3268131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.006+0000 7f32677fe700 1 --2- 172.21.15.89:0/211850205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326806ae80 0x7f3268139550 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3260009e50 tx=0x7f326000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddcb571b2a5db472 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.007+0000 7f3265ffb700 1 -- 172.21.15.89:0/211850205 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000e070 con 0x7f326806ae80 2023-09-30T22:19:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.007+0000 7f3265ffb700 1 -- 172.21.15.89:0/211850205 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326000b910 con 0x7f326806ae80 2023-09-30T22:19:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.007+0000 7f3265ffb700 1 -- 172.21.15.89:0/211850205 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000bcc0 con 0x7f326806ae80 2023-09-30T22:19:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.008+0000 7f326d79d700 1 -- 172.21.15.89:0/211850205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326806ae80 msgr2=0x7f3268139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.008+0000 7f326d79d700 1 --2- 172.21.15.89:0/211850205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326806ae80 0x7f3268139550 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3260009e50 tx=0x7f326000b330 comp rx=0 tx=0).stop 2023-09-30T22:19:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.008+0000 7f326d79d700 1 -- 172.21.15.89:0/211850205 shutdown_connections 2023-09-30T22:19:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.008+0000 7f326d79d700 1 --2- 172.21.15.89:0/211850205 >> v2:172.21.15.89:3300/0 conn(0x7f32681316c0 0x7f3268131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.008+0000 7f326d79d700 1 --2- 172.21.15.89:0/211850205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326806ae80 0x7f3268139550 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.008+0000 7f326d79d700 1 -- 172.21.15.89:0/211850205 >> 172.21.15.89:0/211850205 conn(0x7f3268066f50 msgr2=0x7f326809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.008+0000 7f326d79d700 1 -- 172.21.15.89:0/211850205 shutdown_connections 2023-09-30T22:19:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.008+0000 7f326d79d700 1 -- 172.21.15.89:0/211850205 wait complete. 2023-09-30T22:19:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.009+0000 7f326d79d700 1 Processor -- start 2023-09-30T22:19:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.009+0000 7f326d79d700 1 -- start start 2023-09-30T22:19:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f326d79d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f326806ae80 0x7f3268070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f326d79d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326809d9f0 0x7f3268071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f326d79d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32681316c0 0x7f3268071780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f326d79d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326806fed0 con 0x7f326806ae80 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f326d79d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32680a3540 con 0x7f32681316c0 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f326d79d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f326812b900 con 0x7f326809d9f0 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f326806ae80 0x7f3268070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f326806ae80 0x7f3268070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42808/0 (socket says 172.21.15.89:42808) 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 -- 172.21.15.89:0/2350314113 learned_addr learned my addr 172.21.15.89:0/2350314113 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f32677fe700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32681316c0 0x7f3268071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f32667fc700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326809d9f0 0x7f3268071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 -- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326809d9f0 msgr2=0x7f3268071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326809d9f0 0x7f3268071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 -- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32681316c0 msgr2=0x7f3268071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32681316c0 0x7f3268071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 -- 172.21.15.89:0/2350314113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325800d940 con 0x7f326806ae80 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f32677fe700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32681316c0 0x7f3268071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f32667fc700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326809d9f0 0x7f3268071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.010+0000 7f3266ffd700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f326806ae80 0x7f3268070d00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3258012fc0 tx=0x7f32580103a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.011+0000 7f324f7fe700 1 -- 172.21.15.89:0/2350314113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258010b00 con 0x7f326806ae80 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.011+0000 7f324f7fe700 1 -- 172.21.15.89:0/2350314113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258013040 con 0x7f326806ae80 2023-09-30T22:19:10.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.011+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3260009b30 con 0x7f326806ae80 2023-09-30T22:19:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.011+0000 7f324f7fe700 1 -- 172.21.15.89:0/2350314113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258010e80 con 0x7f326806ae80 2023-09-30T22:19:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.011+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326806f010 con 0x7f326806ae80 2023-09-30T22:19:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.011+0000 7f324f7fe700 1 -- 172.21.15.89:0/2350314113 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3258018480 con 0x7f326806ae80 2023-09-30T22:19:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.012+0000 7f324f7fe700 1 -- 172.21.15.89:0/2350314113 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3258043720 con 0x7f326806ae80 2023-09-30T22:19:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.012+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3268131cb0 con 0x7f326806ae80 2023-09-30T22:19:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.021+0000 7f324f7fe700 1 -- 172.21.15.89:0/2350314113 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3258016910 con 0x7f326806ae80 2023-09-30T22:19:10.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e3840700 1 Processor -- start 2023-09-30T22:19:10.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e3840700 1 -- start start 2023-09-30T22:19:10.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e3840700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09dc09d6f0 0x7f09dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e3840700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e000 0x7f09dc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e3840700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09dc09f130 con 0x7f09dc09e410 2023-09-30T22:19:10.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e3840700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09dc09f520 con 0x7f09dc09d6f0 2023-09-30T22:19:10.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e3840700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc09e930 con 0x7f09dc09e000 2023-09-30T22:19:10.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e15dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09dc09d6f0 0x7f09dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e15dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09dc09d6f0 0x7f09dc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42818/0 (socket says 172.21.15.89:42818) 2023-09-30T22:19:10.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09e15dc700 1 -- 172.21.15.89:0/1325601371 learned_addr learned my addr 172.21.15.89:0/1325601371 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:10.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.063+0000 7f09e0ddb700 1 --2- 172.21.15.89:0/1325601371 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e000 0x7f09dc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09e15dc700 1 -- 172.21.15.89:0/1325601371 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e000 msgr2=0x7f09dc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:10.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09e15dc700 1 --2- 172.21.15.89:0/1325601371 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e000 0x7f09dc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09e15dc700 1 -- 172.21.15.89:0/1325601371 >> v1:172.21.15.89:6789/0 conn(0x7f09dc09e410 legacy=0x7f09dc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09e15dc700 1 -- 172.21.15.89:0/1325601371 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc09b2d0 con 0x7f09dc09d6f0 2023-09-30T22:19:10.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09e15dc700 1 --2- 172.21.15.89:0/1325601371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09d6f0 0x7f09dc09dac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f09d800bb90 tx=0x7f09d800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5028ced40dd5e14 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:10.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09d3fff700 1 -- 172.21.15.89:0/1325601371 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d800dd30 con 0x7f09dc09d6f0 2023-09-30T22:19:10.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09d3fff700 1 -- 172.21.15.89:0/1325601371 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d800de90 con 0x7f09dc09d6f0 2023-09-30T22:19:10.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.064+0000 7f09d3fff700 1 -- 172.21.15.89:0/1325601371 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8010690 con 0x7f09dc09d6f0 2023-09-30T22:19:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.065+0000 7f09e3840700 1 -- 172.21.15.89:0/1325601371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09d6f0 msgr2=0x7f09dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.065+0000 7f09e3840700 1 --2- 172.21.15.89:0/1325601371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09d6f0 0x7f09dc09dac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f09d800bb90 tx=0x7f09d800bea0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.065+0000 7f09e3840700 1 -- 172.21.15.89:0/1325601371 shutdown_connections 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.065+0000 7f09e3840700 1 --2- 172.21.15.89:0/1325601371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09d6f0 0x7f09dc09dac0 secure :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f09d800bb90 tx=0x7f09d800bea0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.065+0000 7f09e3840700 1 --2- 172.21.15.89:0/1325601371 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e000 0x7f09dc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.065+0000 7f09e3840700 1 -- 172.21.15.89:0/1325601371 >> 172.21.15.89:0/1325601371 conn(0x7f09dc066f50 msgr2=0x7f09dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.065+0000 7f09e3840700 1 -- 172.21.15.89:0/1325601371 shutdown_connections 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.065+0000 7f09e3840700 1 -- 172.21.15.89:0/1325601371 wait complete. 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.066+0000 7f09e3840700 1 Processor -- start 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.066+0000 7f09e3840700 1 -- start start 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e3840700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09e000 0x7f09dc11e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e3840700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e410 0x7f09dc11e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e3840700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09dc11ec40 0x7f09dc1c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e3840700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc120650 con 0x7f09dc09e000 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e3840700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc121c90 con 0x7f09dc09e410 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e3840700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc0a1f90 con 0x7f09dc11ec40 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09e000 0x7f09dc11e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09e000 0x7f09dc11e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42826/0 (socket says 172.21.15.89:42826) 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 -- 172.21.15.89:0/1080186085 learned_addr learned my addr 172.21.15.89:0/1080186085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e1ddd700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09dc11ec40 0x7f09dc1c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e0ddb700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e410 0x7f09dc11e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 -- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09dc11ec40 msgr2=0x7f09dc1c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09dc11ec40 0x7f09dc1c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 -- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e410 msgr2=0x7f09dc11e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e410 0x7f09dc11e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 -- 172.21.15.89:0/1080186085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d800b7f0 con 0x7f09dc09e000 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e0ddb700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e410 0x7f09dc11e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.067+0000 7f09e15dc700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09e000 0x7f09dc11e1c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c00 tx=0x7f09d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.068+0000 7f09d1ffb700 1 -- 172.21.15.89:0/1080186085 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8003ab0 con 0x7f09dc09e000 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.068+0000 7f09d1ffb700 1 -- 172.21.15.89:0/1080186085 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d8003c10 con 0x7f09dc09e000 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.068+0000 7f09d1ffb700 1 -- 172.21.15.89:0/1080186085 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8010470 con 0x7f09dc09e000 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.068+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc11fd80 con 0x7f09dc09e000 2023-09-30T22:19:10.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.068+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc1216f0 con 0x7f09dc09e000 2023-09-30T22:19:10.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.069+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc074ea0 con 0x7f09dc09e000 2023-09-30T22:19:10.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.069+0000 7f09d1ffb700 1 -- 172.21.15.89:0/1080186085 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09d80333d0 con 0x7f09dc09e000 2023-09-30T22:19:10.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.070+0000 7f09d1ffb700 1 -- 172.21.15.89:0/1080186085 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f09d8049e60 con 0x7f09dc09e000 2023-09-30T22:19:10.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.074+0000 7f09d1ffb700 1 -- 172.21.15.89:0/1080186085 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09d800e720 con 0x7f09dc09e000 2023-09-30T22:19:10.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.128+0000 7ff7daa6b700 1 Processor -- start 2023-09-30T22:19:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.128+0000 7ff7daa6b700 1 -- start start 2023-09-30T22:19:10.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.128+0000 7ff7daa6b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409d6f0 0x7ff7d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.128+0000 7ff7daa6b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7d409e410 0x7ff7d4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.128+0000 7ff7daa6b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d409f130 con 0x7ff7d409e000 2023-09-30T22:19:10.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.128+0000 7ff7daa6b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d409f520 con 0x7ff7d409e410 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.128+0000 7ff7daa6b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d409e930 con 0x7ff7d409d6f0 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d8807700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409d6f0 0x7ff7d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d8807700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409d6f0 0x7ff7d409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55932/0 (socket says 172.21.15.89:55932) 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d8807700 1 -- 172.21.15.89:0/444446383 learned_addr learned my addr 172.21.15.89:0/444446383 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d9008700 1 --2- 172.21.15.89:0/444446383 >> v2:172.21.15.89:3300/0 conn(0x7ff7d409e410 0x7ff7d4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d8807700 1 -- 172.21.15.89:0/444446383 >> v2:172.21.15.89:3300/0 conn(0x7ff7d409e410 msgr2=0x7ff7d4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d8807700 1 --2- 172.21.15.89:0/444446383 >> v2:172.21.15.89:3300/0 conn(0x7ff7d409e410 0x7ff7d4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d8807700 1 -- 172.21.15.89:0/444446383 >> v1:172.21.15.89:6789/0 conn(0x7ff7d409e000 legacy=0x7ff7d4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d8807700 1 -- 172.21.15.89:0/444446383 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d409b2d0 con 0x7ff7d409d6f0 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d8807700 1 --2- 172.21.15.89:0/444446383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409d6f0 0x7ff7d409dac0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80098f0 tx=0x7ff7c8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df7a2ada1d5d9c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d37fe700 1 -- 172.21.15.89:0/444446383 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8011830 con 0x7ff7d409d6f0 2023-09-30T22:19:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d37fe700 1 -- 172.21.15.89:0/444446383 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c8011990 con 0x7ff7d409d6f0 2023-09-30T22:19:10.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.129+0000 7ff7d37fe700 1 -- 172.21.15.89:0/444446383 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8011d20 con 0x7ff7d409d6f0 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.130+0000 7ff7daa6b700 1 -- 172.21.15.89:0/444446383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409d6f0 msgr2=0x7ff7d409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.130+0000 7ff7daa6b700 1 --2- 172.21.15.89:0/444446383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409d6f0 0x7ff7d409dac0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80098f0 tx=0x7ff7c8009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.130+0000 7ff7daa6b700 1 -- 172.21.15.89:0/444446383 shutdown_connections 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.130+0000 7ff7daa6b700 1 --2- 172.21.15.89:0/444446383 >> v2:172.21.15.89:3300/0 conn(0x7ff7d409e410 0x7ff7d4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.130+0000 7ff7daa6b700 1 --2- 172.21.15.89:0/444446383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409d6f0 0x7ff7d409dac0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.130+0000 7ff7daa6b700 1 -- 172.21.15.89:0/444446383 >> 172.21.15.89:0/444446383 conn(0x7ff7d4066f50 msgr2=0x7ff7d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.130+0000 7ff7daa6b700 1 -- 172.21.15.89:0/444446383 shutdown_connections 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.130+0000 7ff7daa6b700 1 -- 172.21.15.89:0/444446383 wait complete. 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.131+0000 7ff7daa6b700 1 Processor -- start 2023-09-30T22:19:10.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.131+0000 7ff7daa6b700 1 -- start start 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7daa6b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7d409d6f0 0x7ff7d41c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7daa6b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409e000 0x7ff7d41c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7daa6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d409e410 0x7ff7d41cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7daa6b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d412fe80 con 0x7ff7d409e410 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7daa6b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4129e10 con 0x7ff7d409e000 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7daa6b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d41270f0 con 0x7ff7d409d6f0 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409e000 0x7ff7d41c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409e000 0x7ff7d41c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55934/0 (socket says 172.21.15.89:55934) 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7d3fff700 1 -- 172.21.15.89:0/1341499201 learned_addr learned my addr 172.21.15.89:0/1341499201 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7d9008700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d409e410 0x7ff7d41cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.132+0000 7ff7d8807700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7d409d6f0 0x7ff7d41c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d3fff700 1 -- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7d409d6f0 msgr2=0x7ff7d41c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d3fff700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7d409d6f0 0x7ff7d41c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d3fff700 1 -- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d409e410 msgr2=0x7ff7d41cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d3fff700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d409e410 0x7ff7d41cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d3fff700 1 -- 172.21.15.89:0/1341499201 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c8009550 con 0x7ff7d409e000 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d3fff700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409e000 0x7ff7d41c9d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c4b0 tx=0x7ff7c000c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d17fa700 1 -- 172.21.15.89:0/1341499201 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00052d0 con 0x7ff7d409e000 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d41284b0 con 0x7ff7d409e000 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d17fa700 1 -- 172.21.15.89:0/1341499201 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c0005430 con 0x7ff7d409e000 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7d17fa700 1 -- 172.21.15.89:0/1341499201 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0003880 con 0x7ff7d409e000 2023-09-30T22:19:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.133+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d41286f0 con 0x7ff7d409e000 2023-09-30T22:19:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.134+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b40052e0 con 0x7ff7d409e000 2023-09-30T22:19:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.134+0000 7ff7d17fa700 1 -- 172.21.15.89:0/1341499201 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff7c00039e0 con 0x7ff7d409e000 2023-09-30T22:19:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.135+0000 7ff7d17fa700 1 -- 172.21.15.89:0/1341499201 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff7c003eaf0 con 0x7ff7d409e000 2023-09-30T22:19:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.143+0000 7ff7d17fa700 1 -- 172.21.15.89:0/1341499201 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7c00198b0 con 0x7ff7d409e000 2023-09-30T22:19:10.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.145+0000 7f81746d8700 1 Processor -- start 2023-09-30T22:19:10.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f81746d8700 1 -- start start 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f81746d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c06f5d0 0x7f816c12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f81746d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f816c135c20 0x7f816c138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f81746d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816c06fa40 con 0x7f816c12cf60 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f81746d8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816c12cd90 con 0x7f816c135c20 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f81746d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c071780 con 0x7f816c06f5d0 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f8172474700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f816c12cf60 0x7f816c12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50206/0 (socket says 172.21.15.89:50206) 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f8172474700 1 -- 172.21.15.89:0/3290233889 learned_addr learned my addr 172.21.15.89:0/3290233889 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.146+0000 7f8172c75700 1 --2- 172.21.15.89:0/3290233889 >> v2:172.21.15.89:3300/0 conn(0x7f816c135c20 0x7f816c138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8171c73700 1 --2- 172.21.15.89:0/3290233889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c06f5d0 0x7f816c12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8172c75700 1 -- 172.21.15.89:0/3290233889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c06f5d0 msgr2=0x7f816c12ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8172c75700 1 --2- 172.21.15.89:0/3290233889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c06f5d0 0x7f816c12ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8172c75700 1 -- 172.21.15.89:0/3290233889 >> v1:172.21.15.89:6789/0 conn(0x7f816c12cf60 legacy=0x7f816c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8171472700 1 -- 172.21.15.89:0/3290233889 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317779483 0 0) 0x7f816c06fa40 con 0x7f816c12cf60 2023-09-30T22:19:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8172c75700 1 -- 172.21.15.89:0/3290233889 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c071a40 con 0x7f816c135c20 2023-09-30T22:19:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8172c75700 1 --2- 172.21.15.89:0/3290233889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c135c20 0x7f816c138000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f816400bfa0 tx=0x7f816400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed48295c844487a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8171472700 1 -- 172.21.15.89:0/3290233889 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164010070 con 0x7f816c135c20 2023-09-30T22:19:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8171472700 1 -- 172.21.15.89:0/3290233889 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816400dab0 con 0x7f816c135c20 2023-09-30T22:19:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.147+0000 7f8171472700 1 -- 172.21.15.89:0/3290233889 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816400de60 con 0x7f816c135c20 2023-09-30T22:19:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.148+0000 7f81746d8700 1 -- 172.21.15.89:0/3290233889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c135c20 msgr2=0x7f816c138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.148+0000 7f81746d8700 1 --2- 172.21.15.89:0/3290233889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c135c20 0x7f816c138000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f816400bfa0 tx=0x7f816400d330 comp rx=0 tx=0).stop 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.148+0000 7f81746d8700 1 -- 172.21.15.89:0/3290233889 shutdown_connections 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.148+0000 7f81746d8700 1 --2- 172.21.15.89:0/3290233889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c135c20 0x7f816c138000 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.148+0000 7f81746d8700 1 --2- 172.21.15.89:0/3290233889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c06f5d0 0x7f816c12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.148+0000 7f81746d8700 1 -- 172.21.15.89:0/3290233889 >> 172.21.15.89:0/3290233889 conn(0x7f816c0a2aa0 msgr2=0x7f816c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.148+0000 7f81746d8700 1 -- 172.21.15.89:0/3290233889 shutdown_connections 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.148+0000 7f81746d8700 1 -- 172.21.15.89:0/3290233889 wait complete. 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.149+0000 7f81746d8700 1 Processor -- start 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.149+0000 7f81746d8700 1 -- start start 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f81746d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c06f5d0 0x7f816c12f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f81746d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c12cf60 0x7f816c12dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f81746d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f816c135c20 0x7f816c12e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f81746d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c1313c0 con 0x7f816c06f5d0 2023-09-30T22:19:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f81746d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c131fa0 con 0x7f816c12cf60 2023-09-30T22:19:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f81746d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f816c130e20 con 0x7f816c135c20 2023-09-30T22:19:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8172c75700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f816c135c20 0x7f816c12e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8172c75700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f816c135c20 0x7f816c12e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54612/0 (socket says 172.21.15.89:54612) 2023-09-30T22:19:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8172c75700 1 -- 172.21.15.89:0/3281454292 learned_addr learned my addr 172.21.15.89:0/3281454292 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8172474700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c06f5d0 0x7f816c12f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8171c73700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c12cf60 0x7f816c12dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8171c73700 1 -- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f816c135c20 msgr2=0x7f816c12e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8171c73700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f816c135c20 0x7f816c12e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8171c73700 1 -- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c06f5d0 msgr2=0x7f816c12f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8171c73700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c06f5d0 0x7f816c12f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8171c73700 1 -- 172.21.15.89:0/3281454292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816400bc80 con 0x7f816c12cf60 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.150+0000 7f8171c73700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c12cf60 0x7f816c12dbd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f815c00e7e0 tx=0x7f815c00eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.151+0000 7f8162ffd700 1 -- 172.21.15.89:0/3281454292 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c00c070 con 0x7f816c12cf60 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.151+0000 7f8162ffd700 1 -- 172.21.15.89:0/3281454292 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c013610 con 0x7f816c12cf60 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.151+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816c1310e0 con 0x7f816c12cf60 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.151+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c12e650 con 0x7f816c12cf60 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.151+0000 7f8162ffd700 1 -- 172.21.15.89:0/3281454292 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c013980 con 0x7f816c12cf60 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.152+0000 7f8162ffd700 1 -- 172.21.15.89:0/3281454292 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f815c013ae0 con 0x7f816c12cf60 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.152+0000 7f8162ffd700 1 -- 172.21.15.89:0/3281454292 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f815c03f250 con 0x7f816c12cf60 2023-09-30T22:19:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.152+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81540052e0 con 0x7f816c12cf60 2023-09-30T22:19:10.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.161+0000 7f8162ffd700 1 -- 172.21.15.89:0/3281454292 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f815c010900 con 0x7f816c12cf60 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.240+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3268131f50 con 0x7f326806ae80 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.241+0000 7f324f7fe700 1 -- 172.21.15.89:0/2350314113 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f325800dca0 con 0x7f326806ae80 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.243+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f326806ae80 msgr2=0x7f3268070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.243+0000 7f326d79d700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f326806ae80 0x7f3268070d00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3258012fc0 tx=0x7f32580103a0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.243+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 shutdown_connections 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.243+0000 7f326d79d700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32681316c0 0x7f3268071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.243+0000 7f326d79d700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f326806ae80 0x7f3268070d00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.243+0000 7f326d79d700 1 --2- 172.21.15.89:0/2350314113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f326809d9f0 0x7f3268071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.243+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 >> 172.21.15.89:0/2350314113 conn(0x7f3268066f50 msgr2=0x7f3268126880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.244+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 shutdown_connections 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.244+0000 7f326d79d700 1 -- 172.21.15.89:0/2350314113 wait complete. 2023-09-30T22:19:10.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.284+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f09dc1202d0 con 0x7f09dc09e000 2023-09-30T22:19:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.284+0000 7f09d1ffb700 1 -- 172.21.15.89:0/1080186085 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09d8029390 con 0x7f09dc09e000 2023-09-30T22:19:10.286 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:10.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09e000 msgr2=0x7f09dc11e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09e000 0x7f09dc11e1c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c00 tx=0x7f09d8004750 comp rx=0 tx=0).stop 2023-09-30T22:19:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 shutdown_connections 2023-09-30T22:19:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09dc11ec40 0x7f09dc1c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09dc09e000 0x7f09dc11e1c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 --2- 172.21.15.89:0/1080186085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09dc09e410 0x7f09dc11e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 >> 172.21.15.89:0/1080186085 conn(0x7f09dc066f50 msgr2=0x7f09dc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 shutdown_connections 2023-09-30T22:19:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.286+0000 7f09e3840700 1 -- 172.21.15.89:0/1080186085 wait complete. 2023-09-30T22:19:10.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.380+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff7b4005590 con 0x7ff7d409e000 2023-09-30T22:19:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.380+0000 7ff7d17fa700 1 -- 172.21.15.89:0/1341499201 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7c0017020 con 0x7ff7d409e000 2023-09-30T22:19:10.381 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409e000 msgr2=0x7ff7d41c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409e000 0x7ff7d41c9d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c4b0 tx=0x7ff7c000c7c0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 shutdown_connections 2023-09-30T22:19:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d409e410 0x7ff7d41cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff7d409d6f0 0x7ff7d41c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 --2- 172.21.15.89:0/1341499201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff7d409e000 0x7ff7d41c9d40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 >> 172.21.15.89:0/1341499201 conn(0x7ff7d4066f50 msgr2=0x7ff7d406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 shutdown_connections 2023-09-30T22:19:10.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.382+0000 7ff7daa6b700 1 -- 172.21.15.89:0/1341499201 wait complete. 2023-09-30T22:19:10.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.396+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8154005590 con 0x7f816c12cf60 2023-09-30T22:19:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.397+0000 7f8162ffd700 1 -- 172.21.15.89:0/3281454292 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f815c02b090 con 0x7f816c12cf60 2023-09-30T22:19:10.397 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c12cf60 msgr2=0x7f816c12dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c12cf60 0x7f816c12dbd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f815c00e7e0 tx=0x7f815c00eaf0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 shutdown_connections 2023-09-30T22:19:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f816c135c20 0x7f816c12e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f816c06f5d0 0x7f816c12f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 --2- 172.21.15.89:0/3281454292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f816c12cf60 0x7f816c12dbd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 >> 172.21.15.89:0/3281454292 conn(0x7f816c0a2aa0 msgr2=0x7f816c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 shutdown_connections 2023-09-30T22:19:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:10.399+0000 7f81746d8700 1 -- 172.21.15.89:0/3281454292 wait complete. 2023-09-30T22:19:10.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:11.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:11.300 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:11.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:11.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.399+0000 7fb0dd873700 1 Processor -- start 2023-09-30T22:19:11.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.399+0000 7fb0dd873700 1 -- start start 2023-09-30T22:19:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0dd873700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809e1d0 0x7fb0d8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0dd873700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d8135e50 0x7fb0d8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0dd873700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d809f130 con 0x7fb0d809d6f0 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0dd873700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d809f520 con 0x7fb0d809e1d0 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0dd873700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d809e930 con 0x7fb0d8135e50 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d6ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb0d809d6f0 0x7fb0d809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39496/0 (socket says 172.21.15.89:39496) 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d6ffd700 1 -- 172.21.15.89:0/3293490804 learned_addr learned my addr 172.21.15.89:0/3293490804 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d77fe700 1 --2- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d8135e50 0x7fb0d8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d67fc700 1 --2- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809e1d0 0x7fb0d8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d77fe700 1 -- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809e1d0 msgr2=0x7fb0d8135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d77fe700 1 --2- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809e1d0 0x7fb0d8135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d77fe700 1 -- 172.21.15.89:0/3293490804 >> v1:172.21.15.89:6789/0 conn(0x7fb0d809d6f0 legacy=0x7fb0d809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d5ffb700 1 -- 172.21.15.89:0/3293490804 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748845159 0 0) 0x7fb0d809f130 con 0x7fb0d809d6f0 2023-09-30T22:19:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.400+0000 7fb0d77fe700 1 -- 172.21.15.89:0/3293490804 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d809b2d0 con 0x7fb0d8135e50 2023-09-30T22:19:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.401+0000 7fb0d77fe700 1 --2- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d8135e50 0x7fb0d8138230 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009dc0 tx=0x7fb0d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2643891b9ee88ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.401+0000 7fb0d5ffb700 1 -- 172.21.15.89:0/3293490804 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000e070 con 0x7fb0d8135e50 2023-09-30T22:19:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.401+0000 7fb0d5ffb700 1 -- 172.21.15.89:0/3293490804 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d000bb20 con 0x7fb0d8135e50 2023-09-30T22:19:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.401+0000 7fb0d5ffb700 1 -- 172.21.15.89:0/3293490804 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0012430 con 0x7fb0d8135e50 2023-09-30T22:19:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.401+0000 7fb0dd873700 1 -- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d8135e50 msgr2=0x7fb0d8138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.401+0000 7fb0dd873700 1 --2- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d8135e50 0x7fb0d8138230 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009dc0 tx=0x7fb0d000b330 comp rx=0 tx=0).stop 2023-09-30T22:19:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.402+0000 7fb0dd873700 1 -- 172.21.15.89:0/3293490804 shutdown_connections 2023-09-30T22:19:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.402+0000 7fb0dd873700 1 --2- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d8135e50 0x7fb0d8138230 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.402+0000 7fb0dd873700 1 --2- 172.21.15.89:0/3293490804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809e1d0 0x7fb0d8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.402+0000 7fb0dd873700 1 -- 172.21.15.89:0/3293490804 >> 172.21.15.89:0/3293490804 conn(0x7fb0d8066f50 msgr2=0x7fb0d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.402+0000 7fb0dd873700 1 -- 172.21.15.89:0/3293490804 shutdown_connections 2023-09-30T22:19:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.402+0000 7fb0dd873700 1 -- 172.21.15.89:0/3293490804 wait complete. 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.402+0000 7fb0dd873700 1 Processor -- start 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0dd873700 1 -- start start 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0dd873700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809d6f0 0x7fb0d81397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0dd873700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d809e1d0 0x7fb0d8139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0dd873700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d8135e50 0x7fb0d813a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0dd873700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d813b3c0 con 0x7fb0d8135e50 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0dd873700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d8120480 con 0x7fb0d809d6f0 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0dd873700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d813ac10 con 0x7fb0d809e1d0 2023-09-30T22:19:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0d6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809d6f0 0x7fb0d81397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0d67fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d809e1d0 0x7fb0d8139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0d6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809d6f0 0x7fb0d81397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37650/0 (socket says 172.21.15.89:37650) 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.403+0000 7fb0d6ffd700 1 -- 172.21.15.89:0/1747110609 learned_addr learned my addr 172.21.15.89:0/1747110609 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0d6ffd700 1 -- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d809e1d0 msgr2=0x7fb0d8139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0d6ffd700 1 --2- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d809e1d0 0x7fb0d8139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0d6ffd700 1 -- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d8135e50 msgr2=0x7fb0d813a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0d6ffd700 1 --2- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d8135e50 0x7fb0d813a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0d6ffd700 1 -- 172.21.15.89:0/1747110609 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0009a70 con 0x7fb0d809d6f0 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0d67fc700 1 --2- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d809e1d0 0x7fb0d8139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0d6ffd700 1 --2- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809d6f0 0x7fb0d81397b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800c550 tx=0x7fb0c800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0bffff700 1 -- 172.21.15.89:0/1747110609 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8004c80 con 0x7fb0d809d6f0 2023-09-30T22:19:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0bffff700 1 -- 172.21.15.89:0/1747110609 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c8004e00 con 0x7fb0d809d6f0 2023-09-30T22:19:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0bffff700 1 -- 172.21.15.89:0/1747110609 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8007310 con 0x7fb0d809d6f0 2023-09-30T22:19:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d811fa90 con 0x7fb0d809d6f0 2023-09-30T22:19:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.404+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d81208d0 con 0x7fb0d809d6f0 2023-09-30T22:19:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.405+0000 7fb0bffff700 1 -- 172.21.15.89:0/1747110609 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0c8007470 con 0x7fb0d809d6f0 2023-09-30T22:19:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.405+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d8071700 con 0x7fb0d809d6f0 2023-09-30T22:19:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.405+0000 7fb0bffff700 1 -- 172.21.15.89:0/1747110609 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb0c803f490 con 0x7fb0d809d6f0 2023-09-30T22:19:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.410+0000 7fb0bffff700 1 -- 172.21.15.89:0/1747110609 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0c80106d0 con 0x7fb0d809d6f0 2023-09-30T22:19:11.413 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.440+0000 7fbb420a7700 1 Processor -- start 2023-09-30T22:19:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.440+0000 7fbb420a7700 1 -- start start 2023-09-30T22:19:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb420a7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb3c09e1d0 0x7fbb3c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb420a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb420a7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c09b120 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb420a7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c06b460 con 0x7fbb3c09e1d0 2023-09-30T22:19:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb420a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c09a7e0 con 0x7fbb3c139170 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb3bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb3bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37664/0 (socket says 172.21.15.89:37664) 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb3bfff700 1 -- 172.21.15.89:0/1109426861 learned_addr learned my addr 172.21.15.89:0/1109426861 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb3bfff700 1 -- 172.21.15.89:0/1109426861 >> v2:172.21.15.89:3300/0 conn(0x7fbb3c09e1d0 msgr2=0x7fbb3c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb3bfff700 1 --2- 172.21.15.89:0/1109426861 >> v2:172.21.15.89:3300/0 conn(0x7fbb3c09e1d0 0x7fbb3c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb3bfff700 1 -- 172.21.15.89:0/1109426861 >> v1:172.21.15.89:6789/0 conn(0x7fbb3c09d6f0 legacy=0x7fbb3c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb3bfff700 1 -- 172.21.15.89:0/1109426861 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c09a500 con 0x7fbb3c139170 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.441+0000 7fbb3bfff700 1 --2- 172.21.15.89:0/1109426861 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c13b550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009e80 tx=0x7fbb3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de4e35ec483b6c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb3a7fc700 1 -- 172.21.15.89:0/1109426861 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400e070 con 0x7fbb3c139170 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb3a7fc700 1 -- 172.21.15.89:0/1109426861 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34003a20 con 0x7fbb3c139170 2023-09-30T22:19:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb3a7fc700 1 -- 172.21.15.89:0/1109426861 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34003dd0 con 0x7fbb3c139170 2023-09-30T22:19:11.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb420a7700 1 -- 172.21.15.89:0/1109426861 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 msgr2=0x7fbb3c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb420a7700 1 --2- 172.21.15.89:0/1109426861 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c13b550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009e80 tx=0x7fbb3400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:11.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb420a7700 1 -- 172.21.15.89:0/1109426861 shutdown_connections 2023-09-30T22:19:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb420a7700 1 --2- 172.21.15.89:0/1109426861 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c13b550 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb420a7700 1 --2- 172.21.15.89:0/1109426861 >> v2:172.21.15.89:3300/0 conn(0x7fbb3c09e1d0 0x7fbb3c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb420a7700 1 -- 172.21.15.89:0/1109426861 >> 172.21.15.89:0/1109426861 conn(0x7fbb3c066f50 msgr2=0x7fbb3c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb420a7700 1 -- 172.21.15.89:0/1109426861 shutdown_connections 2023-09-30T22:19:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.442+0000 7fbb420a7700 1 -- 172.21.15.89:0/1109426861 wait complete. 2023-09-30T22:19:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.443+0000 7fbb420a7700 1 Processor -- start 2023-09-30T22:19:11.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb420a7700 1 -- start start 2023-09-30T22:19:11.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb420a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb3c09d6f0 0x7fbb3c11f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb420a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c09e1d0 0x7fbb3c11f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb420a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c1c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb420a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c121fa0 con 0x7fbb3c09e1d0 2023-09-30T22:19:11.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb420a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0a2c90 con 0x7fbb3c139170 2023-09-30T22:19:11.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb420a7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c09c710 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb3b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb3c09d6f0 0x7fbb3c11f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb3b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb3c09d6f0 0x7fbb3c11f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60628/0 (socket says 172.21.15.89:60628) 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb3b7fe700 1 -- 172.21.15.89:0/909780405 learned_addr learned my addr 172.21.15.89:0/909780405 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb3affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c09e1d0 0x7fbb3c11f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.445+0000 7fbb3bfff700 1 --2- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c1c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb3b7fe700 1 -- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 msgr2=0x7fbb3c1c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb3b7fe700 1 --2- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c1c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb3b7fe700 1 -- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c09e1d0 msgr2=0x7fbb3c11f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb3b7fe700 1 --2- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c09e1d0 0x7fbb3c11f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb3b7fe700 1 -- 172.21.15.89:0/909780405 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34009b30 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb3b7fe700 1 --2- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb3c09d6f0 0x7fbb3c11f330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c4a0 tx=0x7fbb2c00c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb23fff700 1 -- 172.21.15.89:0/909780405 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0054e0 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb23fff700 1 -- 172.21.15.89:0/909780405 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c003680 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c122cd0 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb23fff700 1 -- 172.21.15.89:0/909780405 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c003a30 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.446+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c11fdb0 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.447+0000 7fbb23fff700 1 -- 172.21.15.89:0/909780405 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbb2c003b90 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.447+0000 7fbb23fff700 1 -- 172.21.15.89:0/909780405 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbb2c03e6a0 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.447+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c11ffd0 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.452+0000 7fbb23fff700 1 -- 172.21.15.89:0/909780405 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb3c11ffd0 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.571+0000 7fa553024700 1 Processor -- start 2023-09-30T22:19:11.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.571+0000 7fa553024700 1 -- start start 2023-09-30T22:19:11.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa553024700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa54c09d6f0 0x7fa54c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa553024700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09e440 0x7fa54c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa553024700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54c09b120 con 0x7fa54c09e000 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa553024700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c06b460 con 0x7fa54c09d6f0 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa553024700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c09a7e0 con 0x7fa54c09e440 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa550dc0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa54c09d6f0 0x7fa54c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa5515c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09e440 0x7fa54c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa550dc0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa54c09d6f0 0x7fa54c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46584/0 (socket says 172.21.15.89:46584) 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa550dc0700 1 -- 172.21.15.89:0/1550713075 learned_addr learned my addr 172.21.15.89:0/1550713075 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.572+0000 7fa54bfff700 1 --1- 172.21.15.89:0/1550713075 >> v1:172.21.15.89:6789/0 conn(0x7fa54c09e000 0x7fa54c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39522/0 (socket says 172.21.15.89:39522) 2023-09-30T22:19:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.573+0000 7fa5515c1700 1 -- 172.21.15.89:0/1550713075 >> v2:172.21.15.89:3300/0 conn(0x7fa54c09d6f0 msgr2=0x7fa54c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.573+0000 7fa5515c1700 1 --2- 172.21.15.89:0/1550713075 >> v2:172.21.15.89:3300/0 conn(0x7fa54c09d6f0 0x7fa54c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.573+0000 7fa5515c1700 1 -- 172.21.15.89:0/1550713075 >> v1:172.21.15.89:6789/0 conn(0x7fa54c09e000 legacy=0x7fa54c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.573+0000 7fa5515c1700 1 -- 172.21.15.89:0/1550713075 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c09a500 con 0x7fa54c09e440 2023-09-30T22:19:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.573+0000 7fa5515c1700 1 --2- 172.21.15.89:0/1550713075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09e440 0x7fa54c134100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa544009dc0 tx=0x7fa54400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eaee7d99f355d35 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.573+0000 7fa54b7fe700 1 -- 172.21.15.89:0/1550713075 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54400e070 con 0x7fa54c09e440 2023-09-30T22:19:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.573+0000 7fa54b7fe700 1 -- 172.21.15.89:0/1550713075 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54400bae0 con 0x7fa54c09e440 2023-09-30T22:19:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.573+0000 7fa54b7fe700 1 -- 172.21.15.89:0/1550713075 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54400be90 con 0x7fa54c09e440 2023-09-30T22:19:11.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.574+0000 7fa553024700 1 -- 172.21.15.89:0/1550713075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09e440 msgr2=0x7fa54c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.574+0000 7fa553024700 1 --2- 172.21.15.89:0/1550713075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09e440 0x7fa54c134100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa544009dc0 tx=0x7fa54400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:11.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.574+0000 7fa553024700 1 -- 172.21.15.89:0/1550713075 shutdown_connections 2023-09-30T22:19:11.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.574+0000 7fa553024700 1 --2- 172.21.15.89:0/1550713075 >> v2:172.21.15.89:3300/0 conn(0x7fa54c09d6f0 0x7fa54c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.574+0000 7fa553024700 1 --2- 172.21.15.89:0/1550713075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09e440 0x7fa54c134100 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.574+0000 7fa553024700 1 -- 172.21.15.89:0/1550713075 >> 172.21.15.89:0/1550713075 conn(0x7fa54c066f50 msgr2=0x7fa54c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.574+0000 7fa553024700 1 -- 172.21.15.89:0/1550713075 shutdown_connections 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.574+0000 7fa553024700 1 -- 172.21.15.89:0/1550713075 wait complete. 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa553024700 1 Processor -- start 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa553024700 1 -- start start 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa553024700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09d6f0 0x7fa54c1c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa553024700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa54c09e000 0x7fa54c1c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa553024700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa54c09e440 0x7fa54c1c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa553024700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1a80b0 con 0x7fa54c09e440 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa553024700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c0a1f90 con 0x7fa54c09e000 2023-09-30T22:19:11.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa553024700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1b9550 con 0x7fa54c09d6f0 2023-09-30T22:19:11.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa550dc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09d6f0 0x7fa54c1c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa5515c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa54c09e440 0x7fa54c1c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa550dc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09d6f0 0x7fa54c1c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60652/0 (socket says 172.21.15.89:60652) 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa5515c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa54c09e440 0x7fa54c1c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46598/0 (socket says 172.21.15.89:46598) 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa54bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa54c09e000 0x7fa54c1c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.575+0000 7fa550dc0700 1 -- 172.21.15.89:0/2434803643 learned_addr learned my addr 172.21.15.89:0/2434803643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5515c1700 1 -- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09d6f0 msgr2=0x7fa54c1c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5515c1700 1 --2- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09d6f0 0x7fa54c1c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5515c1700 1 -- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa54c09e000 msgr2=0x7fa54c1c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5515c1700 1 --2- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa54c09e000 0x7fa54c1c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5515c1700 1 -- 172.21.15.89:0/2434803643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa544009a70 con 0x7fa54c09e440 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5515c1700 1 --2- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa54c09e440 0x7fa54c1c49d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa544016000 tx=0x7fa544004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:11.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5497fa700 1 -- 172.21.15.89:0/2434803643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440045d0 con 0x7fa54c09e440 2023-09-30T22:19:11.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5497fa700 1 -- 172.21.15.89:0/2434803643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa544020070 con 0x7fa54c09e440 2023-09-30T22:19:11.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa5497fa700 1 -- 172.21.15.89:0/2434803643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544027480 con 0x7fa54c09e440 2023-09-30T22:19:11.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c1a8980 con 0x7fa54c09e440 2023-09-30T22:19:11.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.576+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c1a79c0 con 0x7fa54c09e440 2023-09-30T22:19:11.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.577+0000 7fa5497fa700 1 -- 172.21.15.89:0/2434803643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa544012430 con 0x7fa54c09e440 2023-09-30T22:19:11.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.577+0000 7fa5497fa700 1 -- 172.21.15.89:0/2434803643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa544022070 con 0x7fa54c09e440 2023-09-30T22:19:11.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.577+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c1a79c0 con 0x7fa54c09e440 2023-09-30T22:19:11.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.582+0000 7fa5497fa700 1 -- 172.21.15.89:0/2434803643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa54c1a79c0 con 0x7fa54c09e440 2023-09-30T22:19:11.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.591+0000 7f1306b2e700 1 Processor -- start 2023-09-30T22:19:11.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.591+0000 7f1306b2e700 1 -- start start 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.591+0000 7f1306b2e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f1300136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.591+0000 7f1306b2e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1300136fe0 0x7f13001393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.591+0000 7f1306b2e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1300070940 con 0x7f130009e930 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.591+0000 7f1306b2e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130006fd60 con 0x7f130009f410 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.591+0000 7f1306b2e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f130006fa80 con 0x7f1300136fe0 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12fffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f1300136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12fffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f1300136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37680/0 (socket says 172.21.15.89:37680) 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12fffff700 1 -- 172.21.15.89:0/1451063145 learned_addr learned my addr 172.21.15.89:0/1451063145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f13050cb700 1 --2- 172.21.15.89:0/1451063145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1300136fe0 0x7f13001393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12fffff700 1 -- 172.21.15.89:0/1451063145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1300136fe0 msgr2=0x7f13001393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12fffff700 1 --2- 172.21.15.89:0/1451063145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1300136fe0 0x7f13001393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12fffff700 1 -- 172.21.15.89:0/1451063145 >> v1:172.21.15.89:6789/0 conn(0x7f130009e930 legacy=0x7f130009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12fffff700 1 -- 172.21.15.89:0/1451063145 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130009fb60 con 0x7f130009f410 2023-09-30T22:19:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12fffff700 1 --2- 172.21.15.89:0/1451063145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f1300136aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009920 tx=0x7f12ec009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4daa15bcfc4b55e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12ff7fe700 1 -- 172.21.15.89:0/1451063145 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec004230 con 0x7f130009f410 2023-09-30T22:19:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12ff7fe700 1 -- 172.21.15.89:0/1451063145 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec004390 con 0x7f130009f410 2023-09-30T22:19:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.592+0000 7f12ff7fe700 1 -- 172.21.15.89:0/1451063145 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec009e70 con 0x7f130009f410 2023-09-30T22:19:11.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.593+0000 7f1306b2e700 1 -- 172.21.15.89:0/1451063145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 msgr2=0x7f1300136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.593+0000 7f1306b2e700 1 --2- 172.21.15.89:0/1451063145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f1300136aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009920 tx=0x7f12ec009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:11.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.593+0000 7f1306b2e700 1 -- 172.21.15.89:0/1451063145 shutdown_connections 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.593+0000 7f1306b2e700 1 --2- 172.21.15.89:0/1451063145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1300136fe0 0x7f13001393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.593+0000 7f1306b2e700 1 --2- 172.21.15.89:0/1451063145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f1300136aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.593+0000 7f1306b2e700 1 -- 172.21.15.89:0/1451063145 >> 172.21.15.89:0/1451063145 conn(0x7f13000a4210 msgr2=0x7f13000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.593+0000 7f1306b2e700 1 -- 172.21.15.89:0/1451063145 shutdown_connections 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.593+0000 7f1306b2e700 1 -- 172.21.15.89:0/1451063145 wait complete. 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.594+0000 7f1306b2e700 1 Processor -- start 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.594+0000 7f1306b2e700 1 -- start start 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.594+0000 7f1306b2e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f130009e930 0x7f1300123950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.594+0000 7f1306b2e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f13001c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.594+0000 7f1306b2e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1300136fe0 0x7f13001c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.594+0000 7f1306b2e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1300125600 con 0x7f1300136fe0 2023-09-30T22:19:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.594+0000 7f1306b2e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1300136600 con 0x7f130009f410 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.594+0000 7f1306b2e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1300126200 con 0x7f130009e930 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1300136fe0 0x7f13001c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1300136fe0 0x7f13001c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46600/0 (socket says 172.21.15.89:46600) 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 -- 172.21.15.89:0/3465013423 learned_addr learned my addr 172.21.15.89:0/3465013423 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13048ca700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f130009e930 0x7f1300123950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f12fffff700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f13001c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 -- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f130009e930 msgr2=0x7f1300123950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f130009e930 0x7f1300123950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 -- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 msgr2=0x7f13001c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f13001c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 -- 172.21.15.89:0/3465013423 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec009550 con 0x7f1300136fe0 2023-09-30T22:19:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f13050cb700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1300136fe0 0x7f13001c87a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f12f8007e10 tx=0x7f12f800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f12fd7fa700 1 -- 172.21.15.89:0/3465013423 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800ed30 con 0x7f1300136fe0 2023-09-30T22:19:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f12fd7fa700 1 -- 172.21.15.89:0/3465013423 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f800ee90 con 0x7f1300136fe0 2023-09-30T22:19:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13001232f0 con 0x7f1300136fe0 2023-09-30T22:19:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f12fd7fa700 1 -- 172.21.15.89:0/3465013423 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8014660 con 0x7f1300136fe0 2023-09-30T22:19:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.595+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13001264c0 con 0x7f1300136fe0 2023-09-30T22:19:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.596+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1300074ea0 con 0x7f1300136fe0 2023-09-30T22:19:11.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.597+0000 7f12fd7fa700 1 -- 172.21.15.89:0/3465013423 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12f80147c0 con 0x7f1300136fe0 2023-09-30T22:19:11.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.597+0000 7f12fd7fa700 1 -- 172.21.15.89:0/3465013423 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f12f8040c10 con 0x7f1300136fe0 2023-09-30T22:19:11.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.601+0000 7f12fd7fa700 1 -- 172.21.15.89:0/3465013423 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12f8013d50 con 0x7f1300136fe0 2023-09-30T22:19:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.639+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb0d809c700 con 0x7fb0d809d6f0 2023-09-30T22:19:11.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.640+0000 7fb0bffff700 1 -- 172.21.15.89:0/1747110609 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0c8011360 con 0x7fb0d809d6f0 2023-09-30T22:19:11.641 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:11.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.642+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809d6f0 msgr2=0x7fb0d81397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.642+0000 7fb0dd873700 1 --2- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809d6f0 0x7fb0d81397b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800c550 tx=0x7fb0c800c860 comp rx=0 tx=0).stop 2023-09-30T22:19:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.642+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 shutdown_connections 2023-09-30T22:19:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.642+0000 7fb0dd873700 1 --2- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d8135e50 0x7fb0d813a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.642+0000 7fb0dd873700 1 --2- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d809d6f0 0x7fb0d81397b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.642+0000 7fb0dd873700 1 --2- 172.21.15.89:0/1747110609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d809e1d0 0x7fb0d8139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.642+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 >> 172.21.15.89:0/1747110609 conn(0x7fb0d8066f50 msgr2=0x7fb0d806e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.643+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 shutdown_connections 2023-09-30T22:19:11.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.643+0000 7fb0dd873700 1 -- 172.21.15.89:0/1747110609 wait complete. 2023-09-30T22:19:11.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.654+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbb3c11ffd0 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.655+0000 7fbb23fff700 1 -- 172.21.15.89:0/909780405 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb3c11ffd0 con 0x7fbb3c09d6f0 2023-09-30T22:19:11.655 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:11.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb3c09d6f0 msgr2=0x7fbb3c11f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 --2- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb3c09d6f0 0x7fbb3c11f330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c4a0 tx=0x7fbb2c00c7b0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 shutdown_connections 2023-09-30T22:19:11.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 --2- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb3c139170 0x7fbb3c1c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 --2- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb3c09d6f0 0x7fbb3c11f330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 --2- 172.21.15.89:0/909780405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb3c09e1d0 0x7fbb3c11f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 >> 172.21.15.89:0/909780405 conn(0x7fbb3c066f50 msgr2=0x7fbb3c136cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:11.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 shutdown_connections 2023-09-30T22:19:11.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.657+0000 7fbb420a7700 1 -- 172.21.15.89:0/909780405 wait complete. 2023-09-30T22:19:11.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:11.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.801+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f13001369d0 con 0x7f1300136fe0 2023-09-30T22:19:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.801+0000 7f12fd7fa700 1 -- 172.21.15.89:0/3465013423 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12f8013b70 con 0x7f1300136fe0 2023-09-30T22:19:11.802 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:11.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.803+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa54c1a79c0 con 0x7fa54c09e440 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1300136fe0 msgr2=0x7f13001c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1300136fe0 0x7f13001c87a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f12f8007e10 tx=0x7f12f800e3c0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 shutdown_connections 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1300136fe0 0x7f13001c87a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f130009e930 0x7f1300123950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 --2- 172.21.15.89:0/3465013423 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f130009f410 0x7f13001c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 >> 172.21.15.89:0/3465013423 conn(0x7f13000a4210 msgr2=0x7f13000a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 shutdown_connections 2023-09-30T22:19:11.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7f1306b2e700 1 -- 172.21.15.89:0/3465013423 wait complete. 2023-09-30T22:19:11.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.804+0000 7fa5497fa700 1 -- 172.21.15.89:0/2434803643 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa54c1a79c0 con 0x7fa54c09e440 2023-09-30T22:19:11.806 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:11.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa54c09e440 msgr2=0x7fa54c1c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:11.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 --2- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa54c09e440 0x7fa54c1c49d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa544016000 tx=0x7fa544004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 shutdown_connections 2023-09-30T22:19:11.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 --2- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa54c09e440 0x7fa54c1c49d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 --2- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa54c09d6f0 0x7fa54c1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 --2- 172.21.15.89:0/2434803643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa54c09e000 0x7fa54c1c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:11.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 >> 172.21.15.89:0/2434803643 conn(0x7fa54c066f50 msgr2=0x7fa54c1bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:11.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 shutdown_connections 2023-09-30T22:19:11.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:11.807+0000 7fa553024700 1 -- 172.21.15.89:0/2434803643 wait complete. 2023-09-30T22:19:11.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:11.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:12.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:12.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.800+0000 7f8e4125b700 1 Processor -- start 2023-09-30T22:19:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.800+0000 7f8e4125b700 1 -- start start 2023-09-30T22:19:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e4125b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e4125b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c133e30 0x7f8e3c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e4125b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c06b990 con 0x7f8e3c134260 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e4125b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0a0590 con 0x7f8e3c133e30 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e4125b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c09fb80 con 0x7f8e3c133420 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c133e30 0x7f8e3c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60680/0 (socket says 172.21.15.89:60680) 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3ad9d700 1 -- 172.21.15.89:0/143382496 learned_addr learned my addr 172.21.15.89:0/143382496 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3b59e700 1 --1- 172.21.15.89:0/143382496 >> v1:172.21.15.89:6789/0 conn(0x7f8e3c134260 0x7f8e3c1304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39540/0 (socket says 172.21.15.89:39540) 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e39d9b700 1 -- 172.21.15.89:0/143382496 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591707700 0 0) 0x7f8e3c06b990 con 0x7f8e3c134260 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3ad9d700 1 -- 172.21.15.89:0/143382496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c133e30 msgr2=0x7f8e3c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3ad9d700 1 --2- 172.21.15.89:0/143382496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c133e30 0x7f8e3c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3ad9d700 1 -- 172.21.15.89:0/143382496 >> v1:172.21.15.89:6789/0 conn(0x7f8e3c134260 legacy=0x7f8e3c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3ad9d700 1 -- 172.21.15.89:0/143382496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c0a0030 con 0x7f8e3c133420 2023-09-30T22:19:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3a59c700 1 --2- 172.21.15.89:0/143382496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c133e30 0x7f8e3c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e3ad9d700 1 --2- 172.21.15.89:0/143382496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c1337f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009920 tx=0x7f8e2c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6c5fcf765b4050 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e39d9b700 1 -- 172.21.15.89:0/143382496 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c011830 con 0x7f8e3c133420 2023-09-30T22:19:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e39d9b700 1 -- 172.21.15.89:0/143382496 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c011990 con 0x7f8e3c133420 2023-09-30T22:19:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.801+0000 7f8e39d9b700 1 -- 172.21.15.89:0/143382496 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c011d20 con 0x7f8e3c133420 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.802+0000 7f8e4125b700 1 -- 172.21.15.89:0/143382496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 msgr2=0x7f8e3c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.802+0000 7f8e4125b700 1 --2- 172.21.15.89:0/143382496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c1337f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009920 tx=0x7f8e2c009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.802+0000 7f8e4125b700 1 -- 172.21.15.89:0/143382496 shutdown_connections 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.802+0000 7f8e4125b700 1 --2- 172.21.15.89:0/143382496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c1337f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.802+0000 7f8e4125b700 1 --2- 172.21.15.89:0/143382496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c133e30 0x7f8e3c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.802+0000 7f8e4125b700 1 -- 172.21.15.89:0/143382496 >> 172.21.15.89:0/143382496 conn(0x7f8e3c0a4210 msgr2=0x7f8e3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.802+0000 7f8e4125b700 1 -- 172.21.15.89:0/143382496 shutdown_connections 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.802+0000 7f8e4125b700 1 -- 172.21.15.89:0/143382496 wait complete. 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.804+0000 7f8e4125b700 1 Processor -- start 2023-09-30T22:19:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.804+0000 7f8e4125b700 1 -- start start 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.804+0000 7f8e4125b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.804+0000 7f8e4125b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c133e30 0x7f8e3c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.804+0000 7f8e4125b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c134260 0x7f8e3c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.804+0000 7f8e4125b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c11fcb0 con 0x7f8e3c133e30 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.804+0000 7f8e4125b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c09ded0 con 0x7f8e3c134260 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.804+0000 7f8e4125b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c131570 con 0x7f8e3c133420 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3b59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c134260 0x7f8e3c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60686/0 (socket says 172.21.15.89:60686) 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3ad9d700 1 -- 172.21.15.89:0/1188795689 learned_addr learned my addr 172.21.15.89:0/1188795689 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3a59c700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c133e30 0x7f8e3c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3a59c700 1 -- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 msgr2=0x7f8e3c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3a59c700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3a59c700 1 -- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c134260 msgr2=0x7f8e3c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3a59c700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c134260 0x7f8e3c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3a59c700 1 -- 172.21.15.89:0/1188795689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300094a0 con 0x7f8e3c133e30 2023-09-30T22:19:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3ad9d700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3b59e700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c134260 0x7f8e3c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e3a59c700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c133e30 0x7f8e3c09c9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000e980 tx=0x7f8e3000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e237fe700 1 -- 172.21.15.89:0/1188795689 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000c690 con 0x7f8e3c133e30 2023-09-30T22:19:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e237fe700 1 -- 172.21.15.89:0/1188795689 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3000c7f0 con 0x7f8e3c133e30 2023-09-30T22:19:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c009550 con 0x7f8e3c133e30 2023-09-30T22:19:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e237fe700 1 -- 172.21.15.89:0/1188795689 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000cb80 con 0x7f8e3c133e30 2023-09-30T22:19:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.805+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c11fe90 con 0x7f8e3c133e30 2023-09-30T22:19:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.806+0000 7f8e237fe700 1 -- 172.21.15.89:0/1188795689 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e3000cce0 con 0x7f8e3c133e30 2023-09-30T22:19:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.806+0000 7f8e237fe700 1 -- 172.21.15.89:0/1188795689 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e3003fa70 con 0x7f8e3c133e30 2023-09-30T22:19:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.807+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c074ea0 con 0x7f8e3c133e30 2023-09-30T22:19:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.811+0000 7f8e237fe700 1 -- 172.21.15.89:0/1188795689 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e30017080 con 0x7f8e3c133e30 2023-09-30T22:19:12.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:12.824 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.836+0000 7f9afc36c700 1 Processor -- start 2023-09-30T22:19:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.836+0000 7f9afc36c700 1 -- start start 2023-09-30T22:19:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.837+0000 7f9afc36c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9af412cf60 0x7f9af406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.837+0000 7f9afc36c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af4137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.837+0000 7f9afc36c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af412cd90 con 0x7f9af412f8b0 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.837+0000 7f9afc36c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af412d330 con 0x7f9af412cf60 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.837+0000 7f9afc36c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af4071780 con 0x7f9af4130430 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.837+0000 7f9afa108700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9af412cf60 0x7f9af406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.837+0000 7f9afa108700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9af412cf60 0x7f9af406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46620/0 (socket says 172.21.15.89:46620) 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.837+0000 7f9afa108700 1 -- 172.21.15.89:0/2461817957 learned_addr learned my addr 172.21.15.89:0/2461817957 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9afa909700 1 --2- 172.21.15.89:0/2461817957 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af4137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9afa108700 1 -- 172.21.15.89:0/2461817957 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 msgr2=0x7f9af4137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9afa108700 1 --2- 172.21.15.89:0/2461817957 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af4137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9afa108700 1 -- 172.21.15.89:0/2461817957 >> v1:172.21.15.89:6789/0 conn(0x7f9af412f8b0 legacy=0x7f9af412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9afa108700 1 -- 172.21.15.89:0/2461817957 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4071a40 con 0x7f9af412cf60 2023-09-30T22:19:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9afa108700 1 --2- 172.21.15.89:0/2461817957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af406f5d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9af00098f0 tx=0x7f9af0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9cf70f97cf06d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9af9106700 1 -- 172.21.15.89:0/2461817957 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af000c900 con 0x7f9af412cf60 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9af9106700 1 -- 172.21.15.89:0/2461817957 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af000ca60 con 0x7f9af412cf60 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.838+0000 7f9af9106700 1 -- 172.21.15.89:0/2461817957 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af000cdf0 con 0x7f9af412cf60 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.839+0000 7f9afc36c700 1 -- 172.21.15.89:0/2461817957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 msgr2=0x7f9af406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.839+0000 7f9afc36c700 1 --2- 172.21.15.89:0/2461817957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af406f5d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9af00098f0 tx=0x7f9af0009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.839+0000 7f9afc36c700 1 -- 172.21.15.89:0/2461817957 shutdown_connections 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.839+0000 7f9afc36c700 1 --2- 172.21.15.89:0/2461817957 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af4137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.839+0000 7f9afc36c700 1 --2- 172.21.15.89:0/2461817957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af406f5d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.839+0000 7f9afc36c700 1 -- 172.21.15.89:0/2461817957 >> 172.21.15.89:0/2461817957 conn(0x7f9af40a2aa0 msgr2=0x7f9af40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.839+0000 7f9afc36c700 1 -- 172.21.15.89:0/2461817957 shutdown_connections 2023-09-30T22:19:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.839+0000 7f9afc36c700 1 -- 172.21.15.89:0/2461817957 wait complete. 2023-09-30T22:19:12.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.840+0000 7f9afc36c700 1 Processor -- start 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afc36c700 1 -- start start 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afc36c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af41c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afc36c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9af412f8b0 0x7f9af41c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afc36c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af41c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afc36c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af409c700 con 0x7f9af412cf60 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afc36c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af41c6000 con 0x7f9af4130430 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afc36c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9af409ead0 con 0x7f9af412f8b0 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afa108700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af41c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afa108700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af41c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46626/0 (socket says 172.21.15.89:46626) 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.841+0000 7f9afa108700 1 -- 172.21.15.89:0/1491916194 learned_addr learned my addr 172.21.15.89:0/1491916194 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:12.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afa909700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af41c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9af9907700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9af412f8b0 0x7f9af41c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afa108700 1 -- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9af412f8b0 msgr2=0x7f9af41c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afa108700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9af412f8b0 0x7f9af41c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afa108700 1 -- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 msgr2=0x7f9af41c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afa108700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af41c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afa108700 1 -- 172.21.15.89:0/1491916194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af0009550 con 0x7f9af412cf60 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afa909700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af41c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afa108700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af41c5580 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009c40 tx=0x7f9af0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9aeaffd700 1 -- 172.21.15.89:0/1491916194 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00038a0 con 0x7f9af412cf60 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9aeaffd700 1 -- 172.21.15.89:0/1491916194 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0003a00 con 0x7f9af412cf60 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9aeaffd700 1 -- 172.21.15.89:0/1491916194 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0003db0 con 0x7f9af412cf60 2023-09-30T22:19:12.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.842+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af412f590 con 0x7f9af412cf60 2023-09-30T22:19:12.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.843+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af409d420 con 0x7f9af412cf60 2023-09-30T22:19:12.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.843+0000 7f9aeaffd700 1 -- 172.21.15.89:0/1491916194 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9af0018400 con 0x7f9af412cf60 2023-09-30T22:19:12.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.843+0000 7f9aeaffd700 1 -- 172.21.15.89:0/1491916194 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9af0040920 con 0x7f9af412cf60 2023-09-30T22:19:12.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.844+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af409e910 con 0x7f9af412cf60 2023-09-30T22:19:12.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.849+0000 7f9aeaffd700 1 -- 172.21.15.89:0/1491916194 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9af0026080 con 0x7f9af412cf60 2023-09-30T22:19:12.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.948+0000 7f43c36dc700 1 Processor -- start 2023-09-30T22:19:12.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.948+0000 7f43c36dc700 1 -- start start 2023-09-30T22:19:12.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c36dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09d6f0 0x7f43bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c36dc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e000 0x7f43bc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c36dc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f43bc09e410 0x7f43bc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c36dc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc06b460 con 0x7f43bc09e410 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c36dc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc09b120 con 0x7f43bc09d6f0 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c36dc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc09a7e0 con 0x7f43bc09e000 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c1478700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09d6f0 0x7f43bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c0c77700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e000 0x7f43bc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c1478700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09d6f0 0x7f43bc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37734/0 (socket says 172.21.15.89:37734) 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c1478700 1 -- 172.21.15.89:0/3013017632 learned_addr learned my addr 172.21.15.89:0/3013017632 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c0c77700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e000 0x7f43bc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60698/0 (socket says 172.21.15.89:60698) 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.949+0000 7f43c1c79700 1 --2- 172.21.15.89:0/3013017632 >> v2:172.21.15.89:3300/0 conn(0x7f43bc09e410 0x7f43bc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43c1478700 1 -- 172.21.15.89:0/3013017632 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e000 msgr2=0x7f43bc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43c1478700 1 --2- 172.21.15.89:0/3013017632 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e000 0x7f43bc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43c1478700 1 -- 172.21.15.89:0/3013017632 >> v2:172.21.15.89:3300/0 conn(0x7f43bc09e410 msgr2=0x7f43bc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43c1478700 1 --2- 172.21.15.89:0/3013017632 >> v2:172.21.15.89:3300/0 conn(0x7f43bc09e410 0x7f43bc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43c1478700 1 -- 172.21.15.89:0/3013017632 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc09a500 con 0x7f43bc09d6f0 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43c1478700 1 --2- 172.21.15.89:0/3013017632 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09d6f0 0x7f43bc09dac0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009920 tx=0x7f43b8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8892ae3d89922de9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43b3fff700 1 -- 172.21.15.89:0/3013017632 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8011830 con 0x7f43bc09d6f0 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43b3fff700 1 -- 172.21.15.89:0/3013017632 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b8011990 con 0x7f43bc09d6f0 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.950+0000 7f43b3fff700 1 -- 172.21.15.89:0/3013017632 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8011d20 con 0x7f43bc09d6f0 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 -- 172.21.15.89:0/3013017632 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09d6f0 msgr2=0x7f43bc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 --2- 172.21.15.89:0/3013017632 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09d6f0 0x7f43bc09dac0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009920 tx=0x7f43b8009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 -- 172.21.15.89:0/3013017632 shutdown_connections 2023-09-30T22:19:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 --2- 172.21.15.89:0/3013017632 >> v2:172.21.15.89:3300/0 conn(0x7f43bc09e410 0x7f43bc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 --2- 172.21.15.89:0/3013017632 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09d6f0 0x7f43bc09dac0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 --2- 172.21.15.89:0/3013017632 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e000 0x7f43bc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 -- 172.21.15.89:0/3013017632 >> 172.21.15.89:0/3013017632 conn(0x7f43bc066f50 msgr2=0x7f43bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:12.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 -- 172.21.15.89:0/3013017632 shutdown_connections 2023-09-30T22:19:12.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.951+0000 7f43c36dc700 1 -- 172.21.15.89:0/3013017632 wait complete. 2023-09-30T22:19:12.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.952+0000 7f43c36dc700 1 Processor -- start 2023-09-30T22:19:12.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.952+0000 7f43c36dc700 1 -- start start 2023-09-30T22:19:12.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c36dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43bc09d6f0 0x7f43bc1a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c36dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09e000 0x7f43bc1a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c36dc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e410 0x7f43bc1a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c36dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc1a9a60 con 0x7f43bc09d6f0 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c36dc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc1a8130 con 0x7f43bc09e000 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c36dc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc1a91f0 con 0x7f43bc09e410 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1478700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43bc09d6f0 0x7f43bc1a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1478700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43bc09d6f0 0x7f43bc1a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46644/0 (socket says 172.21.15.89:46644) 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1478700 1 -- 172.21.15.89:0/2352494073 learned_addr learned my addr 172.21.15.89:0/2352494073 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1c79700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e410 0x7f43bc1a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c0c77700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09e000 0x7f43bc1a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7ff3022a8700 1 Processor -- start 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1478700 1 -- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e410 msgr2=0x7f43bc1a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1478700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e410 0x7f43bc1a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1478700 1 -- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09e000 msgr2=0x7f43bc1a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1478700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09e000 0x7f43bc1a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7f43c1478700 1 -- 172.21.15.89:0/2352494073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009550 con 0x7f43bc09d6f0 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7f43c1478700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43bc09d6f0 0x7f43bc1a6020 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f43b8000c00 tx=0x7f43b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7f43b1ffb700 1 -- 172.21.15.89:0/2352494073 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8004300 con 0x7f43bc09d6f0 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7f43b1ffb700 1 -- 172.21.15.89:0/2352494073 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b801f070 con 0x7f43bc09d6f0 2023-09-30T22:19:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc09c710 con 0x7f43bc09d6f0 2023-09-30T22:19:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.953+0000 7ff3022a8700 1 -- start start 2023-09-30T22:19:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff3022a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff3022a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff3022a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff3022a8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2fc09f520 con 0x7ff2fc09d6f0 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff3022a8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc09f130 con 0x7ff2fc09e000 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff3022a8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc09e930 con 0x7ff2fc09e410 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37754/0 (socket says 172.21.15.89:37754) 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff2fb7fe700 1 -- 172.21.15.89:0/2732620538 learned_addr learned my addr 172.21.15.89:0/2732620538 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff300845700 1 --2- 172.21.15.89:0/2732620538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7ff2fbfff700 1 --2- 172.21.15.89:0/2732620538 >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc1a7b50 con 0x7f43bc09d6f0 2023-09-30T22:19:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.954+0000 7f43b1ffb700 1 -- 172.21.15.89:0/2352494073 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80036a0 con 0x7f43bc09d6f0 2023-09-30T22:19:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff300845700 1 -- 172.21.15.89:0/2732620538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 msgr2=0x7ff2fc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff300845700 1 --2- 172.21.15.89:0/2732620538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff300845700 1 -- 172.21.15.89:0/2732620538 >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09d6f0 msgr2=0x7ff2fc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff300845700 1 --2- 172.21.15.89:0/2732620538 >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff300845700 1 -- 172.21.15.89:0/2732620538 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc09b2d0 con 0x7ff2fc09e410 2023-09-30T22:19:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc1a9420 con 0x7f43bc09d6f0 2023-09-30T22:19:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff300845700 1 --2- 172.21.15.89:0/2732620538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc130ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009870 tx=0x7ff2f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ebf52a18a989316 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff2faffd700 1 -- 172.21.15.89:0/2732620538 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0011830 con 0x7ff2fc09e410 2023-09-30T22:19:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff2faffd700 1 -- 172.21.15.89:0/2732620538 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f0011990 con 0x7ff2fc09e410 2023-09-30T22:19:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.955+0000 7ff2faffd700 1 -- 172.21.15.89:0/2732620538 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0011d20 con 0x7ff2fc09e410 2023-09-30T22:19:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7f43b1ffb700 1 -- 172.21.15.89:0/2352494073 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f43b8021030 con 0x7f43bc09d6f0 2023-09-30T22:19:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 -- 172.21.15.89:0/2732620538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 msgr2=0x7ff2fc130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 --2- 172.21.15.89:0/2732620538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc130ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009870 tx=0x7ff2f0009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7f43b1ffb700 1 -- 172.21.15.89:0/2352494073 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f43b8040d00 con 0x7f43bc09d6f0 2023-09-30T22:19:12.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 -- 172.21.15.89:0/2732620538 shutdown_connections 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 --2- 172.21.15.89:0/2732620538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc130ba0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 --2- 172.21.15.89:0/2732620538 >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 --2- 172.21.15.89:0/2732620538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 -- 172.21.15.89:0/2732620538 >> 172.21.15.89:0/2732620538 conn(0x7ff2fc066f50 msgr2=0x7ff2fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 -- 172.21.15.89:0/2732620538 shutdown_connections 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.956+0000 7ff3022a8700 1 -- 172.21.15.89:0/2732620538 wait complete. 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.957+0000 7ff3022a8700 1 Processor -- start 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff3022a8700 1 -- start start 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff3022a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff3022a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc11e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff3022a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc11e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff3022a8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc120040 con 0x7ff2fc09d6f0 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff3022a8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc11fa10 con 0x7ff2fc09e000 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff3022a8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc11f2a0 con 0x7ff2fc09e410 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff2fbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff2fbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46656/0 (socket says 172.21.15.89:46656) 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff2fbfff700 1 -- 172.21.15.89:0/1542269514 learned_addr learned my addr 172.21.15.89:0/1542269514 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff300845700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc11e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.958+0000 7ff2fb7fe700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc11e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2fb7fe700 1 -- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 msgr2=0x7ff2fc11e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2fb7fe700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc11e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2fb7fe700 1 -- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 msgr2=0x7ff2fc11ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:12.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2fb7fe700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2fb7fe700 1 -- 172.21.15.89:0/1542269514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f00094a0 con 0x7ff2fc09e000 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2fb7fe700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc11e410 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec010820 tx=0x7ff2ec010b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2f8ff9700 1 -- 172.21.15.89:0/1542269514 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec011920 con 0x7ff2fc09e000 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2f8ff9700 1 -- 172.21.15.89:0/1542269514 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec011a80 con 0x7ff2fc09e000 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc09c700 con 0x7ff2fc09e000 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff2f8ff9700 1 -- 172.21.15.89:0/1542269514 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec011e10 con 0x7ff2fc09e000 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.959+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc120c60 con 0x7ff2fc09e000 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.960+0000 7ff2f8ff9700 1 -- 172.21.15.89:0/1542269514 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff2ec019440 con 0x7ff2fc09e000 2023-09-30T22:19:12.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.960+0000 7ff2f8ff9700 1 -- 172.21.15.89:0/1542269514 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff2ec041970 con 0x7ff2fc09e000 2023-09-30T22:19:12.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.961+0000 7f43b1ffb700 1 -- 172.21.15.89:0/2352494073 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43b80237e0 con 0x7f43bc09d6f0 2023-09-30T22:19:12.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.961+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc074ea0 con 0x7ff2fc09e000 2023-09-30T22:19:12.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:12.970+0000 7ff2f8ff9700 1 -- 172.21.15.89:0/1542269514 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2ec01e020 con 0x7ff2fc09e000 2023-09-30T22:19:13.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.017+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8e3c120230 con 0x7f8e3c133e30 2023-09-30T22:19:13.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.018+0000 7f8e237fe700 1 -- 172.21.15.89:0/1188795689 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e30021020 con 0x7f8e3c133e30 2023-09-30T22:19:13.019 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:13.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c133e30 msgr2=0x7f8e3c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:13.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c133e30 0x7f8e3c09c9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000e980 tx=0x7f8e3000ec90 comp rx=0 tx=0).stop 2023-09-30T22:19:13.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 shutdown_connections 2023-09-30T22:19:13.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c134260 0x7f8e3c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c133420 0x7f8e3c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 --2- 172.21.15.89:0/1188795689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c133e30 0x7f8e3c09c9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 >> 172.21.15.89:0/1188795689 conn(0x7f8e3c0a4210 msgr2=0x7f8e3c0a3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:13.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 shutdown_connections 2023-09-30T22:19:13.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.020+0000 7f8e4125b700 1 -- 172.21.15.89:0/1188795689 wait complete. 2023-09-30T22:19:13.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:13.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.050+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9af4074ea0 con 0x7f9af412cf60 2023-09-30T22:19:13.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.050+0000 7f9aeaffd700 1 -- 172.21.15.89:0/1491916194 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9af0018e60 con 0x7f9af412cf60 2023-09-30T22:19:13.051 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 msgr2=0x7f9af41c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af41c5580 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009c40 tx=0x7f9af0004750 comp rx=0 tx=0).stop 2023-09-30T22:19:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 shutdown_connections 2023-09-30T22:19:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9af4130430 0x7f9af41c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af412cf60 0x7f9af41c5580 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 --2- 172.21.15.89:0/1491916194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9af412f8b0 0x7f9af41c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 >> 172.21.15.89:0/1491916194 conn(0x7f9af40a2aa0 msgr2=0x7f9af406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 shutdown_connections 2023-09-30T22:19:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.052+0000 7f9afc36c700 1 -- 172.21.15.89:0/1491916194 wait complete. 2023-09-30T22:19:13.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:13.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.159+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f43bc1a9420 con 0x7f43bc09d6f0 2023-09-30T22:19:13.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.160+0000 7f43b1ffb700 1 -- 172.21.15.89:0/2352494073 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43bc1a9420 con 0x7f43bc09d6f0 2023-09-30T22:19:13.161 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:13.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43bc09d6f0 msgr2=0x7f43bc1a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:13.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43bc09d6f0 0x7f43bc1a6020 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f43b8000c00 tx=0x7f43b8004750 comp rx=0 tx=0).stop 2023-09-30T22:19:13.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 shutdown_connections 2023-09-30T22:19:13.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43bc09e410 0x7f43bc1a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43bc09d6f0 0x7f43bc1a6020 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 --2- 172.21.15.89:0/2352494073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43bc09e000 0x7f43bc1a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 >> 172.21.15.89:0/2352494073 conn(0x7f43bc066f50 msgr2=0x7f43bc1bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:13.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 shutdown_connections 2023-09-30T22:19:13.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.162+0000 7f43c36dc700 1 -- 172.21.15.89:0/2352494073 wait complete. 2023-09-30T22:19:13.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.164+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff2fc06c360 con 0x7ff2fc09e000 2023-09-30T22:19:13.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.164+0000 7ff2f8ff9700 1 -- 172.21.15.89:0/1542269514 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2ec0183a0 con 0x7ff2fc09e000 2023-09-30T22:19:13.165 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:13.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 msgr2=0x7ff2fc11e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:13.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc11e410 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec010820 tx=0x7ff2ec010b30 comp rx=0 tx=0).stop 2023-09-30T22:19:13.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 shutdown_connections 2023-09-30T22:19:13.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc11e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 --2- 172.21.15.89:0/1542269514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc11e410 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:13.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 >> 172.21.15.89:0/1542269514 conn(0x7ff2fc066f50 msgr2=0x7ff2fc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:13.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 shutdown_connections 2023-09-30T22:19:13.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:13.166+0000 7ff3022a8700 1 -- 172.21.15.89:0/1542269514 wait complete. 2023-09-30T22:19:13.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:13.179 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:14.034 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:14.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:14.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f099da31700 1 Processor -- start 2023-09-30T22:19:14.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f099da31700 1 -- start start 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f099da31700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09980a1510 0x7f0998138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f099da31700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0998139250 0x7f099813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f099da31700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099809edd0 con 0x7f09980a0a30 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f099da31700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09980a18e0 con 0x7f09980a1510 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f099da31700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f099809e1f0 con 0x7f0998139250 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f09967fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09980a1510 0x7f0998138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f09967fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09980a1510 0x7f0998138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46660/0 (socket says 172.21.15.89:46660) 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f09967fc700 1 -- 172.21.15.89:0/1770156263 learned_addr learned my addr 172.21.15.89:0/1770156263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.171+0000 7f09977fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0998139250 0x7f099813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.172+0000 7f09967fc700 1 -- 172.21.15.89:0/1770156263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0998139250 msgr2=0x7f099813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.172+0000 7f09967fc700 1 --2- 172.21.15.89:0/1770156263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0998139250 0x7f099813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.172+0000 7f09967fc700 1 -- 172.21.15.89:0/1770156263 >> v1:172.21.15.89:6789/0 conn(0x7f09980a0a30 legacy=0x7f09980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.172+0000 7f09967fc700 1 -- 172.21.15.89:0/1770156263 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099809dc50 con 0x7f09980a1510 2023-09-30T22:19:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.172+0000 7f09967fc700 1 --2- 172.21.15.89:0/1770156263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09980a1510 0x7f0998138d10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f098c0098f0 tx=0x7f098c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7079267201b32e91 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.172+0000 7f0995ffb700 1 -- 172.21.15.89:0/1770156263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c011830 con 0x7f09980a1510 2023-09-30T22:19:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.172+0000 7f0995ffb700 1 -- 172.21.15.89:0/1770156263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c011990 con 0x7f09980a1510 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.172+0000 7f0995ffb700 1 -- 172.21.15.89:0/1770156263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c011d20 con 0x7f09980a1510 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.173+0000 7f099da31700 1 -- 172.21.15.89:0/1770156263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09980a1510 msgr2=0x7f0998138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.173+0000 7f099da31700 1 --2- 172.21.15.89:0/1770156263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09980a1510 0x7f0998138d10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f098c0098f0 tx=0x7f098c009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.173+0000 7f099da31700 1 -- 172.21.15.89:0/1770156263 shutdown_connections 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.173+0000 7f099da31700 1 --2- 172.21.15.89:0/1770156263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0998139250 0x7f099813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.173+0000 7f099da31700 1 --2- 172.21.15.89:0/1770156263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09980a1510 0x7f0998138d10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.173+0000 7f099da31700 1 -- 172.21.15.89:0/1770156263 >> 172.21.15.89:0/1770156263 conn(0x7f09980a6550 msgr2=0x7f09980a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.173+0000 7f099da31700 1 -- 172.21.15.89:0/1770156263 shutdown_connections 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.173+0000 7f099da31700 1 -- 172.21.15.89:0/1770156263 wait complete. 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.174+0000 7f099da31700 1 Processor -- start 2023-09-30T22:19:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.174+0000 7f099da31700 1 -- start start 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f099da31700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09980a0a30 0x7f09981c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f099da31700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09980a1510 0x7f09981c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f099da31700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998139250 0x7f09981229c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f099da31700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0998125f40 con 0x7f0998139250 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f099da31700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09981be330 con 0x7f09980a1510 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f099da31700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09980a0220 con 0x7f09980a0a30 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998139250 0x7f09981229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998139250 0x7f09981229c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46664/0 (socket says 172.21.15.89:46664) 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 -- 172.21.15.89:0/1420870462 learned_addr learned my addr 172.21.15.89:0/1420870462 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09967fc700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09980a1510 0x7f09981c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f0996ffd700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09980a0a30 0x7f09981c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 -- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09980a0a30 msgr2=0x7f09981c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09980a0a30 0x7f09981c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 -- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09980a1510 msgr2=0x7f09981c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09980a1510 0x7f09981c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 -- 172.21.15.89:0/1420870462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c009550 con 0x7f0998139250 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09967fc700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09980a1510 0x7f09981c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f09977fe700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998139250 0x7f09981229c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f099000efc0 tx=0x7f099000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.175+0000 7f097ffff700 1 -- 172.21.15.89:0/1420870462 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099000cc90 con 0x7f0998139250 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.176+0000 7f097ffff700 1 -- 172.21.15.89:0/1420870462 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099000f040 con 0x7f0998139250 2023-09-30T22:19:14.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.176+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0998138890 con 0x7f0998139250 2023-09-30T22:19:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.176+0000 7f097ffff700 1 -- 172.21.15.89:0/1420870462 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09900146c0 con 0x7f0998139250 2023-09-30T22:19:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.176+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09981246d0 con 0x7f0998139250 2023-09-30T22:19:14.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:14.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:14.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.176+0000 7f097ffff700 1 -- 172.21.15.89:0/1420870462 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0990014820 con 0x7f0998139250 2023-09-30T22:19:14.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.177+0000 7f097ffff700 1 -- 172.21.15.89:0/1420870462 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0990040260 con 0x7f0998139250 2023-09-30T22:19:14.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.177+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09981260d0 con 0x7f0998139250 2023-09-30T22:19:14.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.182+0000 7f097ffff700 1 -- 172.21.15.89:0/1420870462 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09981260d0 con 0x7f0998139250 2023-09-30T22:19:14.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.211+0000 7fc4e3dce700 1 Processor -- start 2023-09-30T22:19:14.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.211+0000 7fc4e3dce700 1 -- start start 2023-09-30T22:19:14.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e3dce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc4dc071b40 0x7fc4dc130420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e3dce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc130d80 0x7fc4dc133c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e3dce700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4dc099ef0 con 0x7fc4dc130960 2023-09-30T22:19:14.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e3dce700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4dc099500 con 0x7fc4dc071b40 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e3dce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc099c10 con 0x7fc4dc130d80 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e1b6a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc4dc071b40 0x7fc4dc130420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e1b6a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc4dc071b40 0x7fc4dc130420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46680/0 (socket says 172.21.15.89:46680) 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e1b6a700 1 -- 172.21.15.89:0/3819256138 learned_addr learned my addr 172.21.15.89:0/3819256138 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e236b700 1 --2- 172.21.15.89:0/3819256138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc130d80 0x7fc4dc133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e236b700 1 -- 172.21.15.89:0/3819256138 >> v2:172.21.15.89:3300/0 conn(0x7fc4dc071b40 msgr2=0x7fc4dc130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e236b700 1 --2- 172.21.15.89:0/3819256138 >> v2:172.21.15.89:3300/0 conn(0x7fc4dc071b40 0x7fc4dc130420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e236b700 1 -- 172.21.15.89:0/3819256138 >> v1:172.21.15.89:6789/0 conn(0x7fc4dc130960 legacy=0x7fc4dc133570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e236b700 1 -- 172.21.15.89:0/3819256138 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc09e370 con 0x7fc4dc130d80 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e1b6a700 1 --2- 172.21.15.89:0/3819256138 >> v2:172.21.15.89:3300/0 conn(0x7fc4dc071b40 0x7fc4dc130420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e236b700 1 --2- 172.21.15.89:0/3819256138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc130d80 0x7fc4dc133c80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009d90 tx=0x7fc4d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca59820a4b74bfd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:14.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.212+0000 7fc4e0b68700 1 -- 172.21.15.89:0/3819256138 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400e070 con 0x7fc4dc130d80 2023-09-30T22:19:14.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.213+0000 7fc4e0b68700 1 -- 172.21.15.89:0/3819256138 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d400bb20 con 0x7fc4dc130d80 2023-09-30T22:19:14.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.213+0000 7fc4e0b68700 1 -- 172.21.15.89:0/3819256138 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4012430 con 0x7fc4dc130d80 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.213+0000 7fc4e3dce700 1 -- 172.21.15.89:0/3819256138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc130d80 msgr2=0x7fc4dc133c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.213+0000 7fc4e3dce700 1 --2- 172.21.15.89:0/3819256138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc130d80 0x7fc4dc133c80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009d90 tx=0x7fc4d400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.214+0000 7fc4e3dce700 1 -- 172.21.15.89:0/3819256138 shutdown_connections 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.214+0000 7fc4e3dce700 1 --2- 172.21.15.89:0/3819256138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc130d80 0x7fc4dc133c80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.214+0000 7fc4e3dce700 1 --2- 172.21.15.89:0/3819256138 >> v2:172.21.15.89:3300/0 conn(0x7fc4dc071b40 0x7fc4dc130420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.214+0000 7fc4e3dce700 1 -- 172.21.15.89:0/3819256138 >> 172.21.15.89:0/3819256138 conn(0x7fc4dc09e7e0 msgr2=0x7fc4dc09ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.214+0000 7fc4e3dce700 1 -- 172.21.15.89:0/3819256138 shutdown_connections 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.214+0000 7fc4e3dce700 1 -- 172.21.15.89:0/3819256138 wait complete. 2023-09-30T22:19:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.215+0000 7fc4e3dce700 1 Processor -- start 2023-09-30T22:19:14.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.215+0000 7fc4e3dce700 1 -- start start 2023-09-30T22:19:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e3dce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc071b40 0x7fc4dc126e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e3dce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4dc130960 0x7fc4dc127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e3dce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4dc127880 0x7fc4dc1c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e3dce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc129410 con 0x7fc4dc127880 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e3dce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc129f20 con 0x7fc4dc071b40 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e3dce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc12fab0 con 0x7fc4dc130960 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e236b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4dc127880 0x7fc4dc1c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e236b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4dc127880 0x7fc4dc1c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46686/0 (socket says 172.21.15.89:46686) 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e236b700 1 -- 172.21.15.89:0/783334298 learned_addr learned my addr 172.21.15.89:0/783334298 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e1369700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4dc130960 0x7fc4dc127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.216+0000 7fc4e1b6a700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc071b40 0x7fc4dc126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e1369700 1 -- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc071b40 msgr2=0x7fc4dc126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e1369700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc071b40 0x7fc4dc126e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e1369700 1 -- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4dc127880 msgr2=0x7fc4dc1c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e1369700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4dc127880 0x7fc4dc1c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e1369700 1 -- 172.21.15.89:0/783334298 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d8009550 con 0x7fc4dc130960 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e236b700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4dc127880 0x7fc4dc1c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e1369700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4dc130960 0x7fc4dc127340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc000f60 tx=0x7fc4cc00c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4d27fc700 1 -- 172.21.15.89:0/783334298 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0052d0 con 0x7fc4dc130960 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4d27fc700 1 -- 172.21.15.89:0/783334298 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc005430 con 0x7fc4dc130960 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4d27fc700 1 -- 172.21.15.89:0/783334298 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0038c0 con 0x7fc4dc130960 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d4009a70 con 0x7fc4dc130960 2023-09-30T22:19:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.217+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc129bf0 con 0x7fc4dc130960 2023-09-30T22:19:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.218+0000 7fc4d27fc700 1 -- 172.21.15.89:0/783334298 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc4cc003a20 con 0x7fc4dc130960 2023-09-30T22:19:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.218+0000 7fc4d27fc700 1 -- 172.21.15.89:0/783334298 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc4cc03e950 con 0x7fc4dc130960 2023-09-30T22:19:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.219+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc129560 con 0x7fc4dc130960 2023-09-30T22:19:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.228+0000 7fc4d27fc700 1 -- 172.21.15.89:0/783334298 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc4cc017080 con 0x7fc4dc130960 2023-09-30T22:19:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.313+0000 7fdc25ca6700 1 Processor -- start 2023-09-30T22:19:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.313+0000 7fdc25ca6700 1 -- start start 2023-09-30T22:19:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc25ca6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc20131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc25ca6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc20131f40 0x7fdc2006e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc25ca6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc2006e9d0 0x7fdc2006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc25ca6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc20071a00 con 0x7fdc20131f40 2023-09-30T22:19:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc25ca6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20131430 con 0x7fdc2006e9d0 2023-09-30T22:19:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc25ca6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2009a510 con 0x7fdc20131630 2023-09-30T22:19:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7f7ff18ad700 1 Processor -- start 2023-09-30T22:19:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc1f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc20131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc1effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc20131f40 0x7fdc2006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc1effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc20131f40 0x7fdc2006e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46688/0 (socket says 172.21.15.89:46688) 2023-09-30T22:19:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc1f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc20131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60760/0 (socket says 172.21.15.89:60760) 2023-09-30T22:19:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc1effd700 1 -- 172.21.15.89:0/2087428802 learned_addr learned my addr 172.21.15.89:0/2087428802 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7fdc1ffff700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc2006e9d0 0x7fdc2006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.314+0000 7f7ff18ad700 1 -- start start 2023-09-30T22:19:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1effd700 1 -- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 msgr2=0x7fdc20131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1effd700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc20131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1effd700 1 -- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc2006e9d0 msgr2=0x7fdc2006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1effd700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc2006e9d0 0x7fdc2006ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1effd700 1 -- 172.21.15.89:0/2087428802 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200721c0 con 0x7fdc20131f40 2023-09-30T22:19:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1ffff700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc2006e9d0 0x7fdc2006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1effd700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc20131f40 0x7fdc2006e490 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdc140098f0 tx=0x7fdc14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf641ccf8db195a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7ff18ad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7fec09d6f0 0x7f7fec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7ff18ad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09e000 0x7f7fec130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7ff18ad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e410 0x7f7fec130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7ff18ad700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fec09f520 con 0x7f7fec09d6f0 2023-09-30T22:19:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1e7fc700 1 -- 172.21.15.89:0/2087428802 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14011830 con 0x7fdc20131f40 2023-09-30T22:19:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7ff18ad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec09f130 con 0x7f7fec09e410 2023-09-30T22:19:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7ff18ad700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec09e930 con 0x7f7fec09e000 2023-09-30T22:19:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1e7fc700 1 -- 172.21.15.89:0/2087428802 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc14011990 con 0x7fdc20131f40 2023-09-30T22:19:14.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7fdc1e7fc700 1 -- 172.21.15.89:0/2087428802 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14011d20 con 0x7fdc20131f40 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7fea7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09e000 0x7f7fec130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7fea7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09e000 0x7f7fec130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60764/0 (socket says 172.21.15.89:60764) 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7fea7fc700 1 -- 172.21.15.89:0/1119160327 learned_addr learned my addr 172.21.15.89:0/1119160327 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7feb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e410 0x7f7fec130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7feaffd700 1 --2- 172.21.15.89:0/1119160327 >> v2:172.21.15.89:3300/0 conn(0x7f7fec09d6f0 0x7f7fec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7feb7fe700 1 -- 172.21.15.89:0/1119160327 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09e000 msgr2=0x7f7fec130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7feb7fe700 1 --2- 172.21.15.89:0/1119160327 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09e000 0x7f7fec130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7feb7fe700 1 -- 172.21.15.89:0/1119160327 >> v2:172.21.15.89:3300/0 conn(0x7f7fec09d6f0 msgr2=0x7f7fec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7feb7fe700 1 --2- 172.21.15.89:0/1119160327 >> v2:172.21.15.89:3300/0 conn(0x7f7fec09d6f0 0x7f7fec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.315+0000 7f7feb7fe700 1 -- 172.21.15.89:0/1119160327 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec09b2d0 con 0x7f7fec09e410 2023-09-30T22:19:14.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7feb7fe700 1 --2- 172.21.15.89:0/1119160327 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e410 0x7f7fec130ba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009d90 tx=0x7f7fe400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d72ce0742c3354 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:14.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7fe9ffb700 1 -- 172.21.15.89:0/1119160327 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400e070 con 0x7f7fec09e410 2023-09-30T22:19:14.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7fe9ffb700 1 -- 172.21.15.89:0/1119160327 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe400bb20 con 0x7f7fec09e410 2023-09-30T22:19:14.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7fe9ffb700 1 -- 172.21.15.89:0/1119160327 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4012430 con 0x7f7fec09e410 2023-09-30T22:19:14.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7fdc25ca6700 1 -- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc20131f40 msgr2=0x7fdc2006e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7fdc25ca6700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc20131f40 0x7fdc2006e490 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdc140098f0 tx=0x7fdc14009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:14.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7ff18ad700 1 -- 172.21.15.89:0/1119160327 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e410 msgr2=0x7f7fec130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7ff18ad700 1 --2- 172.21.15.89:0/1119160327 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e410 0x7f7fec130ba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009d90 tx=0x7f7fe400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:14.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7ff18ad700 1 -- 172.21.15.89:0/1119160327 shutdown_connections 2023-09-30T22:19:14.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7ff18ad700 1 --2- 172.21.15.89:0/1119160327 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e410 0x7f7fec130ba0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7ff18ad700 1 --2- 172.21.15.89:0/1119160327 >> v2:172.21.15.89:3300/0 conn(0x7f7fec09d6f0 0x7f7fec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7ff18ad700 1 --2- 172.21.15.89:0/1119160327 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09e000 0x7f7fec130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7f7ff18ad700 1 -- 172.21.15.89:0/1119160327 >> 172.21.15.89:0/1119160327 conn(0x7f7fec066f50 msgr2=0x7f7fec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:14.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7fdc25ca6700 1 -- 172.21.15.89:0/2087428802 shutdown_connections 2023-09-30T22:19:14.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7fdc25ca6700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc2006e9d0 0x7fdc2006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7fdc25ca6700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc20131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7fdc25ca6700 1 --2- 172.21.15.89:0/2087428802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc20131f40 0x7fdc2006e490 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.316+0000 7fdc25ca6700 1 -- 172.21.15.89:0/2087428802 >> 172.21.15.89:0/2087428802 conn(0x7fdc200a2aa0 msgr2=0x7fdc200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:14.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.317+0000 7fdc25ca6700 1 -- 172.21.15.89:0/2087428802 shutdown_connections 2023-09-30T22:19:14.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.317+0000 7fdc25ca6700 1 -- 172.21.15.89:0/2087428802 wait complete. 2023-09-30T22:19:14.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.317+0000 7f7ff18ad700 1 -- 172.21.15.89:0/1119160327 shutdown_connections 2023-09-30T22:19:14.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.317+0000 7f7ff18ad700 1 -- 172.21.15.89:0/1119160327 wait complete. 2023-09-30T22:19:14.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7f7ff18ad700 1 Processor -- start 2023-09-30T22:19:14.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7fdc25ca6700 1 Processor -- start 2023-09-30T22:19:14.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7fdc25ca6700 1 -- start start 2023-09-30T22:19:14.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7fdc25ca6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc2006e9d0 0x7fdc2006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7fdc25ca6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc2006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7fdc25ca6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc20131f40 0x7fdc20070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7fdc25ca6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20071190 con 0x7fdc2006e9d0 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7fdc25ca6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20120bf0 con 0x7fdc20131f40 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.318+0000 7fdc25ca6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc201201a0 con 0x7fdc20131630 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc2006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc2006fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60768/0 (socket says 172.21.15.89:60768) 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 -- 172.21.15.89:0/4114423436 learned_addr learned my addr 172.21.15.89:0/4114423436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 -- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc20131f40 msgr2=0x7fdc20070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 --2- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc20131f40 0x7fdc20070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 -- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc2006e9d0 msgr2=0x7fdc2006f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 --2- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc2006e9d0 0x7fdc2006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 -- 172.21.15.89:0/4114423436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009550 con 0x7fdc20131630 2023-09-30T22:19:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7fdc1effd700 1 --2- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc2006fb50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc14013000 tx=0x7fdc14004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.319+0000 7f7ff18ad700 1 -- start start 2023-09-30T22:19:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7ff18ad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09d6f0 0x7f7fec1c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7ff18ad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e000 0x7f7fec1c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7ff18ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fec09e410 0x7f7fec1c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7ff18ad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec135be0 con 0x7f7fec09e410 2023-09-30T22:19:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7ff18ad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec136950 con 0x7f7fec09e000 2023-09-30T22:19:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7ff18ad700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec134ae0 con 0x7f7fec09d6f0 2023-09-30T22:19:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7fdc07fff700 1 -- 172.21.15.89:0/4114423436 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140036a0 con 0x7fdc20131630 2023-09-30T22:19:14.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7fdc25ca6700 1 -- 172.21.15.89:0/4114423436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2009f8f0 con 0x7fdc20131630 2023-09-30T22:19:14.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7fdc25ca6700 1 -- 172.21.15.89:0/4114423436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc20120eb0 con 0x7fdc20131630 2023-09-30T22:19:14.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7feaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09d6f0 0x7f7fec1c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7feaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09d6f0 0x7f7fec1c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60782/0 (socket says 172.21.15.89:60782) 2023-09-30T22:19:14.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7feaffd700 1 -- 172.21.15.89:0/383000735 learned_addr learned my addr 172.21.15.89:0/383000735 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:14.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7fdc07fff700 1 -- 172.21.15.89:0/4114423436 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc14003820 con 0x7fdc20131630 2023-09-30T22:19:14.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7fdc07fff700 1 -- 172.21.15.89:0/4114423436 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14003bd0 con 0x7fdc20131630 2023-09-30T22:19:14.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7fea7fc700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e000 0x7f7fec1c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7feb7fe700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fec09e410 0x7f7fec1c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7fea7fc700 1 -- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09d6f0 msgr2=0x7f7fec1c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7fea7fc700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09d6f0 0x7f7fec1c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7fea7fc700 1 -- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fec09e410 msgr2=0x7f7fec1c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7fea7fc700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fec09e410 0x7f7fec1c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.320+0000 7f7fea7fc700 1 -- 172.21.15.89:0/383000735 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4009a70 con 0x7f7fec09e000 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.321+0000 7f7feb7fe700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fec09e410 0x7f7fec1c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.321+0000 7f7fea7fc700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e000 0x7f7fec1c6330 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000e8e0 tx=0x7f7fe000ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.321+0000 7f7fd3fff700 1 -- 172.21.15.89:0/383000735 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000c690 con 0x7f7fec09e000 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.321+0000 7f7fd3fff700 1 -- 172.21.15.89:0/383000735 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe000c7f0 con 0x7f7fec09e000 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.321+0000 7f7fd3fff700 1 -- 172.21.15.89:0/383000735 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000cb60 con 0x7f7fec09e000 2023-09-30T22:19:14.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.321+0000 7f7feaffd700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09d6f0 0x7f7fec1c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:14.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.321+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec0a3880 con 0x7f7fec09e000 2023-09-30T22:19:14.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.321+0000 7fdc05ffb700 1 -- 172.21.15.89:0/4114423436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf80052e0 con 0x7fdc20131630 2023-09-30T22:19:14.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.322+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec137530 con 0x7f7fec09e000 2023-09-30T22:19:14.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.322+0000 7fdc07fff700 1 -- 172.21.15.89:0/4114423436 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc14003d30 con 0x7fdc20131630 2023-09-30T22:19:14.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.322+0000 7fdc07fff700 1 -- 172.21.15.89:0/4114423436 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdc14040b10 con 0x7fdc20131630 2023-09-30T22:19:14.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.322+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fec074ea0 con 0x7f7fec09e000 2023-09-30T22:19:14.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.325+0000 7f7fd3fff700 1 -- 172.21.15.89:0/383000735 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7fe000ccc0 con 0x7f7fec09e000 2023-09-30T22:19:14.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.325+0000 7f7fd3fff700 1 -- 172.21.15.89:0/383000735 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7fe003fb80 con 0x7f7fec09e000 2023-09-30T22:19:14.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.327+0000 7fdc07fff700 1 -- 172.21.15.89:0/4114423436 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc1402b360 con 0x7fdc20131630 2023-09-30T22:19:14.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.332+0000 7f7fd3fff700 1 -- 172.21.15.89:0/383000735 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7fe00119a0 con 0x7f7fec09e000 2023-09-30T22:19:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.408+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f09980a1d10 con 0x7f0998139250 2023-09-30T22:19:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.408+0000 7f097ffff700 1 -- 172.21.15.89:0/1420870462 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0990018080 con 0x7f0998139250 2023-09-30T22:19:14.410 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.411+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998139250 msgr2=0x7f09981229c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.411+0000 7f099da31700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998139250 0x7f09981229c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f099000efc0 tx=0x7f099000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.411+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 shutdown_connections 2023-09-30T22:19:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.411+0000 7f099da31700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998139250 0x7f09981229c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.411+0000 7f099da31700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09980a0a30 0x7f09981c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.411+0000 7f099da31700 1 --2- 172.21.15.89:0/1420870462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09980a1510 0x7f09981c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.411+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 >> 172.21.15.89:0/1420870462 conn(0x7f09980a6550 msgr2=0x7f09980a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.412+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 shutdown_connections 2023-09-30T22:19:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.412+0000 7f099da31700 1 -- 172.21.15.89:0/1420870462 wait complete. 2023-09-30T22:19:14.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.445+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc4dc133530 con 0x7fc4dc130960 2023-09-30T22:19:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.445+0000 7fc4d27fc700 1 -- 172.21.15.89:0/783334298 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc4cc011590 con 0x7fc4dc130960 2023-09-30T22:19:14.446 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4dc130960 msgr2=0x7fc4dc127340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4dc130960 0x7fc4dc127340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc000f60 tx=0x7fc4cc00c730 comp rx=0 tx=0).stop 2023-09-30T22:19:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 shutdown_connections 2023-09-30T22:19:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4dc127880 0x7fc4dc1c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4dc071b40 0x7fc4dc126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 --2- 172.21.15.89:0/783334298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4dc130960 0x7fc4dc127340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 >> 172.21.15.89:0/783334298 conn(0x7fc4dc09e7e0 msgr2=0x7fc4dc131470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 shutdown_connections 2023-09-30T22:19:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.447+0000 7fc4e3dce700 1 -- 172.21.15.89:0/783334298 wait complete. 2023-09-30T22:19:14.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:14.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.521+0000 7fdc05ffb700 1 -- 172.21.15.89:0/4114423436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdbf8005590 con 0x7fdc20131630 2023-09-30T22:19:14.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.522+0000 7fdc07fff700 1 -- 172.21.15.89:0/4114423436 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc14015020 con 0x7fdc20131630 2023-09-30T22:19:14.523 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.523+0000 7fdc05ffb700 1 -- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 msgr2=0x7fdc2006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.523+0000 7fdc05ffb700 1 --2- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc2006fb50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc14013000 tx=0x7fdc14004750 comp rx=0 tx=0).stop 2023-09-30T22:19:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.523+0000 7fdc05ffb700 1 -- 172.21.15.89:0/4114423436 shutdown_connections 2023-09-30T22:19:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.523+0000 7fdc05ffb700 1 --2- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc20131f40 0x7fdc20070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.524+0000 7fdc05ffb700 1 --2- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc2006e9d0 0x7fdc2006f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.524+0000 7fdc05ffb700 1 --2- 172.21.15.89:0/4114423436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc20131630 0x7fdc2006fb50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.524+0000 7fdc05ffb700 1 -- 172.21.15.89:0/4114423436 >> 172.21.15.89:0/4114423436 conn(0x7fdc200a2aa0 msgr2=0x7fdc201b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.524+0000 7fdc05ffb700 1 -- 172.21.15.89:0/4114423436 shutdown_connections 2023-09-30T22:19:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.524+0000 7fdc05ffb700 1 -- 172.21.15.89:0/4114423436 wait complete. 2023-09-30T22:19:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.526+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7fec06c360 con 0x7f7fec09e000 2023-09-30T22:19:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.526+0000 7f7fd3fff700 1 -- 172.21.15.89:0/383000735 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7fe0017070 con 0x7f7fec09e000 2023-09-30T22:19:14.527 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e000 msgr2=0x7f7fec1c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e000 0x7f7fec1c6330 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000e8e0 tx=0x7f7fe000ebf0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 shutdown_connections 2023-09-30T22:19:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fec09e410 0x7f7fec1c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fec09d6f0 0x7f7fec1c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 --2- 172.21.15.89:0/383000735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fec09e000 0x7f7fec1c6330 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 >> 172.21.15.89:0/383000735 conn(0x7f7fec066f50 msgr2=0x7f7fec0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 shutdown_connections 2023-09-30T22:19:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:14.528+0000 7f7ff18ad700 1 -- 172.21.15.89:0/383000735 wait complete. 2023-09-30T22:19:14.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:14.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:15.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:15.461 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:15.540 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:15.543 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd417150700 1 Processor -- start 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd417150700 1 -- start start 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd417150700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd41009d6f0 0x7fd41009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd417150700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e440 0x7fd410134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd417150700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41009b120 con 0x7fd41009e000 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd417150700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41006b460 con 0x7fd41009d6f0 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd417150700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd41009a7e0 con 0x7fd41009e440 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd4156ed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e440 0x7fd410134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd414eec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd41009d6f0 0x7fd41009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd4156ed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e440 0x7fd410134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60794/0 (socket says 172.21.15.89:60794) 2023-09-30T22:19:15.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.561+0000 7fd4156ed700 1 -- 172.21.15.89:0/3636372192 learned_addr learned my addr 172.21.15.89:0/3636372192 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:15.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd414eec700 1 -- 172.21.15.89:0/3636372192 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e440 msgr2=0x7fd410134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:15.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd414eec700 1 --2- 172.21.15.89:0/3636372192 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e440 0x7fd410134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd414eec700 1 -- 172.21.15.89:0/3636372192 >> v1:172.21.15.89:6789/0 conn(0x7fd41009e000 legacy=0x7fd4101339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd414eec700 1 -- 172.21.15.89:0/3636372192 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41009a500 con 0x7fd41009d6f0 2023-09-30T22:19:15.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd4156ed700 1 --2- 172.21.15.89:0/3636372192 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e440 0x7fd410134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd414eec700 1 --2- 172.21.15.89:0/3636372192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009d6f0 0x7fd41009dac0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd404009870 tx=0x7fd404009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f496f00ef07cde5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd40f7fe700 1 -- 172.21.15.89:0/3636372192 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404011830 con 0x7fd41009d6f0 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd40f7fe700 1 -- 172.21.15.89:0/3636372192 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd404011990 con 0x7fd41009d6f0 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.562+0000 7fd40f7fe700 1 -- 172.21.15.89:0/3636372192 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404011d20 con 0x7fd41009d6f0 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.563+0000 7fd417150700 1 -- 172.21.15.89:0/3636372192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009d6f0 msgr2=0x7fd41009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.563+0000 7fd417150700 1 --2- 172.21.15.89:0/3636372192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009d6f0 0x7fd41009dac0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd404009870 tx=0x7fd404009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.563+0000 7fd417150700 1 -- 172.21.15.89:0/3636372192 shutdown_connections 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.563+0000 7fd417150700 1 --2- 172.21.15.89:0/3636372192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009d6f0 0x7fd41009dac0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.563+0000 7fd417150700 1 --2- 172.21.15.89:0/3636372192 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e440 0x7fd410134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.563+0000 7fd417150700 1 -- 172.21.15.89:0/3636372192 >> 172.21.15.89:0/3636372192 conn(0x7fd410066f50 msgr2=0x7fd41009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.563+0000 7fd417150700 1 -- 172.21.15.89:0/3636372192 shutdown_connections 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.563+0000 7fd417150700 1 -- 172.21.15.89:0/3636372192 wait complete. 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.564+0000 7fd417150700 1 Processor -- start 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.564+0000 7fd417150700 1 -- start start 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd417150700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd41009d6f0 0x7fd4101a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd417150700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e000 0x7fd4101a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd417150700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009e440 0x7fd4101a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd417150700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4101a80c0 con 0x7fd41009e440 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd417150700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4101a9180 con 0x7fd41009d6f0 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd417150700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd41009c710 con 0x7fd41009e000 2023-09-30T22:19:15.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009e440 0x7fd4101a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd40ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e000 0x7fd4101a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009e440 0x7fd4101a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46742/0 (socket says 172.21.15.89:46742) 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 -- 172.21.15.89:0/908684937 learned_addr learned my addr 172.21.15.89:0/908684937 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd414eec700 1 --2- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd41009d6f0 0x7fd4101a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 -- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e000 msgr2=0x7fd4101a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 --2- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e000 0x7fd4101a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 -- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd41009d6f0 msgr2=0x7fd4101a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 --2- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd41009d6f0 0x7fd4101a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 -- 172.21.15.89:0/908684937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd408009b30 con 0x7fd41009e440 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.565+0000 7fd4156ed700 1 --2- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009e440 0x7fd4101a6950 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd40800efc0 tx=0x7fd40800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.566+0000 7fd40d7fa700 1 -- 172.21.15.89:0/908684937 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408018070 con 0x7fd41009e440 2023-09-30T22:19:15.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.566+0000 7fd40d7fa700 1 -- 172.21.15.89:0/908684937 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40800f040 con 0x7fd41009e440 2023-09-30T22:19:15.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.566+0000 7fd40d7fa700 1 -- 172.21.15.89:0/908684937 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408003c20 con 0x7fd41009e440 2023-09-30T22:19:15.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.566+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4040094a0 con 0x7fd41009e440 2023-09-30T22:19:15.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.566+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4101a7ae0 con 0x7fd41009e440 2023-09-30T22:19:15.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.567+0000 7fd40d7fa700 1 -- 172.21.15.89:0/908684937 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd40800c880 con 0x7fd41009e440 2023-09-30T22:19:15.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.567+0000 7fd40d7fa700 1 -- 172.21.15.89:0/908684937 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd40803f7f0 con 0x7fd41009e440 2023-09-30T22:19:15.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.567+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4101a93b0 con 0x7fd41009e440 2023-09-30T22:19:15.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.573+0000 7fd40d7fa700 1 -- 172.21.15.89:0/908684937 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd40800ce10 con 0x7fd41009e440 2023-09-30T22:19:15.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f536f09b700 1 Processor -- start 2023-09-30T22:19:15.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f536f09b700 1 -- start start 2023-09-30T22:19:15.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f536f09b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5368133540 0x7f5368135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f536f09b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368135e60 0x7f5368138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f536f09b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53680716a0 con 0x7f5368130540 2023-09-30T22:19:15.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f536f09b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5368071290 con 0x7f5368133540 2023-09-30T22:19:15.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f536f09b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5368130340 con 0x7f5368135e60 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f5367fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5368133540 0x7f5368135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f5367fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5368133540 0x7f5368135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46758/0 (socket says 172.21.15.89:46758) 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f5367fff700 1 -- 172.21.15.89:0/4127959431 learned_addr learned my addr 172.21.15.89:0/4127959431 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.614+0000 7f536d638700 1 --2- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368135e60 0x7f5368138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f5367fff700 1 -- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368135e60 msgr2=0x7f5368138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f5367fff700 1 --2- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368135e60 0x7f5368138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f5367fff700 1 -- 172.21.15.89:0/4127959431 >> v1:172.21.15.89:6789/0 conn(0x7f5368130540 legacy=0x7f5368130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f5367fff700 1 -- 172.21.15.89:0/4127959431 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368070b80 con 0x7f5368133540 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f536d638700 1 --2- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368135e60 0x7f5368138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f5367fff700 1 --2- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368133540 0x7f5368135920 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f53540098f0 tx=0x7f5354009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe56997bd5cd480 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f53677fe700 1 -- 172.21.15.89:0/4127959431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354004230 con 0x7f5368133540 2023-09-30T22:19:15.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f53677fe700 1 -- 172.21.15.89:0/4127959431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5354004390 con 0x7f5368133540 2023-09-30T22:19:15.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.615+0000 7f53677fe700 1 -- 172.21.15.89:0/4127959431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53540036a0 con 0x7f5368133540 2023-09-30T22:19:15.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.616+0000 7f536f09b700 1 -- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368133540 msgr2=0x7f5368135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.616+0000 7f536f09b700 1 --2- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368133540 0x7f5368135920 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f53540098f0 tx=0x7f5354009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.616+0000 7f536f09b700 1 -- 172.21.15.89:0/4127959431 shutdown_connections 2023-09-30T22:19:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.616+0000 7f536f09b700 1 --2- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368135e60 0x7f5368138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.616+0000 7f536f09b700 1 --2- 172.21.15.89:0/4127959431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368133540 0x7f5368135920 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.616+0000 7f536f09b700 1 -- 172.21.15.89:0/4127959431 >> 172.21.15.89:0/4127959431 conn(0x7f53680a2aa0 msgr2=0x7f53680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.616+0000 7f536f09b700 1 -- 172.21.15.89:0/4127959431 shutdown_connections 2023-09-30T22:19:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.616+0000 7f536f09b700 1 -- 172.21.15.89:0/4127959431 wait complete. 2023-09-30T22:19:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.617+0000 7f536f09b700 1 Processor -- start 2023-09-30T22:19:15.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.617+0000 7f536f09b700 1 -- start start 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.617+0000 7f536f09b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368130540 0x7f53681c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.617+0000 7f536f09b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368133540 0x7f53681c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.617+0000 7f536f09b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5368135e60 0x7f536806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.617+0000 7f536f09b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53681c8290 con 0x7f5368130540 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.617+0000 7f536f09b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536811a740 con 0x7f5368135e60 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.617+0000 7f536f09b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53680a23b0 con 0x7f5368133540 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536ce37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368130540 0x7f53681c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536ce37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368130540 0x7f53681c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46770/0 (socket says 172.21.15.89:46770) 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536ce37700 1 -- 172.21.15.89:0/1196411441 learned_addr learned my addr 172.21.15.89:0/1196411441 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536d638700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5368135e60 0x7f536806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f5367fff700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368133540 0x7f53681c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536d638700 1 -- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368133540 msgr2=0x7f53681c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536d638700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368133540 0x7f53681c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536d638700 1 -- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368130540 msgr2=0x7f53681c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536d638700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368130540 0x7f53681c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536d638700 1 -- 172.21.15.89:0/1196411441 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5360009a70 con 0x7f5368135e60 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f5367fff700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368133540 0x7f53681c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536ce37700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368130540 0x7f53681c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536d638700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5368135e60 0x7f536806e4d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f536000efc0 tx=0x7f536000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f53657fa700 1 -- 172.21.15.89:0/1196411441 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536000cb40 con 0x7f5368135e60 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f53657fa700 1 -- 172.21.15.89:0/1196411441 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536000f040 con 0x7f5368135e60 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5354009550 con 0x7f5368135e60 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.618+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53680718e0 con 0x7f5368135e60 2023-09-30T22:19:15.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.619+0000 7f53657fa700 1 -- 172.21.15.89:0/1196411441 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536000cee0 con 0x7f5368135e60 2023-09-30T22:19:15.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.619+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5368074ea0 con 0x7f5368135e60 2023-09-30T22:19:15.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.620+0000 7f53657fa700 1 -- 172.21.15.89:0/1196411441 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5360014480 con 0x7f5368135e60 2023-09-30T22:19:15.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.621+0000 7f53657fa700 1 -- 172.21.15.89:0/1196411441 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f536001c7b0 con 0x7f5368135e60 2023-09-30T22:19:15.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.629+0000 7f53657fa700 1 -- 172.21.15.89:0/1196411441 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5360012930 con 0x7f5368135e60 2023-09-30T22:19:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.678+0000 7faeade1f700 1 Processor -- start 2023-09-30T22:19:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.678+0000 7faeade1f700 1 -- start start 2023-09-30T22:19:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.678+0000 7faeade1f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.678+0000 7faeade1f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e410 0x7faea8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.678+0000 7faeade1f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea809b120 con 0x7faea809e000 2023-09-30T22:19:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.678+0000 7faeade1f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea806b460 con 0x7faea809e410 2023-09-30T22:19:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faeade1f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faea809a7e0 con 0x7faea809d6f0 2023-09-30T22:19:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e410 0x7faea8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60820/0 (socket says 172.21.15.89:60820) 2023-09-30T22:19:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea77fe700 1 -- 172.21.15.89:0/1424037296 learned_addr learned my addr 172.21.15.89:0/1424037296 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea77fe700 1 -- 172.21.15.89:0/1424037296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e410 msgr2=0x7faea8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea77fe700 1 --2- 172.21.15.89:0/1424037296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e410 0x7faea8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea77fe700 1 -- 172.21.15.89:0/1424037296 >> v1:172.21.15.89:6789/0 conn(0x7faea809e000 legacy=0x7faea81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea77fe700 1 -- 172.21.15.89:0/1424037296 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea809a500 con 0x7faea809d6f0 2023-09-30T22:19:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea77fe700 1 --2- 172.21.15.89:0/1424037296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea809dac0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fae98009950 tx=0x7fae98009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcc536e17c30e34 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.679+0000 7faea67fc700 1 -- 172.21.15.89:0/1424037296 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98004230 con 0x7faea809d6f0 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.680+0000 7faea67fc700 1 -- 172.21.15.89:0/1424037296 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae98004390 con 0x7faea809d6f0 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.680+0000 7faea67fc700 1 -- 172.21.15.89:0/1424037296 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae980036a0 con 0x7faea809d6f0 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.680+0000 7faeade1f700 1 -- 172.21.15.89:0/1424037296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 msgr2=0x7faea809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.680+0000 7faeade1f700 1 --2- 172.21.15.89:0/1424037296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea809dac0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fae98009950 tx=0x7fae98009c60 comp rx=0 tx=0).stop 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.681+0000 7faeade1f700 1 -- 172.21.15.89:0/1424037296 shutdown_connections 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.681+0000 7faeade1f700 1 --2- 172.21.15.89:0/1424037296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e410 0x7faea8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.681+0000 7faeade1f700 1 --2- 172.21.15.89:0/1424037296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea809dac0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.681+0000 7faeade1f700 1 -- 172.21.15.89:0/1424037296 >> 172.21.15.89:0/1424037296 conn(0x7faea8066f50 msgr2=0x7faea809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:15.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.681+0000 7faeade1f700 1 -- 172.21.15.89:0/1424037296 shutdown_connections 2023-09-30T22:19:15.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.681+0000 7faeade1f700 1 -- 172.21.15.89:0/1424037296 wait complete. 2023-09-30T22:19:15.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.682+0000 7faeade1f700 1 Processor -- start 2023-09-30T22:19:15.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.682+0000 7faeade1f700 1 -- start start 2023-09-30T22:19:15.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faeade1f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea8127b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faeade1f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e000 0x7faea8128050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faeade1f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea809e410 0x7faea8128590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faeade1f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea812a830 con 0x7faea809e410 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faeade1f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea809c710 con 0x7faea809e000 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faeade1f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faea8129ac0 con 0x7faea809d6f0 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e000 0x7faea8128050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e000 0x7faea8128050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37850/0 (socket says 172.21.15.89:37850) 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 -- 172.21.15.89:0/656637536 learned_addr learned my addr 172.21.15.89:0/656637536 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea77fe700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea8127b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea7fff700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea809e410 0x7faea8128590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 -- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 msgr2=0x7faea8127b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea8127b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 -- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea809e410 msgr2=0x7faea8128590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea809e410 0x7faea8128590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 -- 172.21.15.89:0/656637536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae98009550 con 0x7faea809e000 2023-09-30T22:19:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.683+0000 7faea6ffd700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e000 0x7faea8128050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00c780 tx=0x7fae9c00ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.684+0000 7fae8ffff700 1 -- 172.21.15.89:0/656637536 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0054e0 con 0x7faea809e000 2023-09-30T22:19:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.684+0000 7fae8ffff700 1 -- 172.21.15.89:0/656637536 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c003680 con 0x7faea809e000 2023-09-30T22:19:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.684+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea812b4d0 con 0x7faea809e000 2023-09-30T22:19:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.684+0000 7fae8ffff700 1 -- 172.21.15.89:0/656637536 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0039f0 con 0x7faea809e000 2023-09-30T22:19:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.685+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea8129030 con 0x7faea809e000 2023-09-30T22:19:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.685+0000 7fae8ffff700 1 -- 172.21.15.89:0/656637536 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae9c003b50 con 0x7faea809e000 2023-09-30T22:19:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.686+0000 7fae8ffff700 1 -- 172.21.15.89:0/656637536 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fae9c03ea50 con 0x7faea809e000 2023-09-30T22:19:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.686+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea8129030 con 0x7faea809e000 2023-09-30T22:19:15.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.695+0000 7fae8ffff700 1 -- 172.21.15.89:0/656637536 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faea8129030 con 0x7faea809e000 2023-09-30T22:19:15.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.696+0000 7fc7547df700 1 Processor -- start 2023-09-30T22:19:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.696+0000 7fc7547df700 1 -- start start 2023-09-30T22:19:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.696+0000 7fc7547df700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c06a770 0x7fc74c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.696+0000 7fc7547df700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c09dc30 0x7fc74c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.696+0000 7fc7547df700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc74c13c1e0 0x7fc74c13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.696+0000 7fc7547df700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c06ab40 con 0x7fc74c13c1e0 2023-09-30T22:19:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.696+0000 7fc7547df700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c09ebc0 con 0x7fc74c09dc30 2023-09-30T22:19:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.696+0000 7fc7547df700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c09a810 con 0x7fc74c06a770 2023-09-30T22:19:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c09dc30 0x7fc74c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c09dc30 0x7fc74c09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37858/0 (socket says 172.21.15.89:37858) 2023-09-30T22:19:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 -- 172.21.15.89:0/3673382674 learned_addr learned my addr 172.21.15.89:0/3673382674 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc75257b700 1 --2- 172.21.15.89:0/3673382674 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c06a770 0x7fc74c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 -- 172.21.15.89:0/3673382674 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c06a770 msgr2=0x7fc74c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 --2- 172.21.15.89:0/3673382674 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c06a770 0x7fc74c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 -- 172.21.15.89:0/3673382674 >> v2:172.21.15.89:3300/0 conn(0x7fc74c13c1e0 msgr2=0x7fc74c13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 --2- 172.21.15.89:0/3673382674 >> v2:172.21.15.89:3300/0 conn(0x7fc74c13c1e0 0x7fc74c13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 -- 172.21.15.89:0/3673382674 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c09e920 con 0x7fc74c09dc30 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751d7a700 1 --2- 172.21.15.89:0/3673382674 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c09dc30 0x7fc74c09e080 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009920 tx=0x7fc73c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=978f9d1dccec2bae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751579700 1 -- 172.21.15.89:0/3673382674 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c011830 con 0x7fc74c09dc30 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751579700 1 -- 172.21.15.89:0/3673382674 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c011990 con 0x7fc74c09dc30 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.697+0000 7fc751579700 1 -- 172.21.15.89:0/3673382674 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c011d20 con 0x7fc74c09dc30 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 -- 172.21.15.89:0/3673382674 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c09dc30 msgr2=0x7fc74c09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 --2- 172.21.15.89:0/3673382674 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c09dc30 0x7fc74c09e080 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009920 tx=0x7fc73c009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 -- 172.21.15.89:0/3673382674 shutdown_connections 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 --2- 172.21.15.89:0/3673382674 >> v2:172.21.15.89:3300/0 conn(0x7fc74c13c1e0 0x7fc74c13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 --2- 172.21.15.89:0/3673382674 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c06a770 0x7fc74c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 --2- 172.21.15.89:0/3673382674 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c09dc30 0x7fc74c09e080 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 -- 172.21.15.89:0/3673382674 >> 172.21.15.89:0/3673382674 conn(0x7fc74c066f50 msgr2=0x7fc74c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 -- 172.21.15.89:0/3673382674 shutdown_connections 2023-09-30T22:19:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.698+0000 7fc7547df700 1 -- 172.21.15.89:0/3673382674 wait complete. 2023-09-30T22:19:15.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.699+0000 7fc7547df700 1 Processor -- start 2023-09-30T22:19:15.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.699+0000 7fc7547df700 1 -- start start 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc7547df700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c06a770 0x7fc74c13a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc7547df700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c09dc30 0x7fc74c13ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc7547df700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc74c13c1e0 0x7fc74c13b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc7547df700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c11fb40 con 0x7fc74c13c1e0 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc7547df700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c11ceb0 con 0x7fc74c06a770 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc7547df700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c11f490 con 0x7fc74c09dc30 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc74c13c1e0 0x7fc74c13b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc751d7a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c09dc30 0x7fc74c13ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc74c13c1e0 0x7fc74c13b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46810/0 (socket says 172.21.15.89:46810) 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 -- 172.21.15.89:0/3959396254 learned_addr learned my addr 172.21.15.89:0/3959396254 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:15.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc75257b700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c06a770 0x7fc74c13a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 -- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c09dc30 msgr2=0x7fc74c13ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c09dc30 0x7fc74c13ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 -- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c06a770 msgr2=0x7fc74c13a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c06a770 0x7fc74c13a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 -- 172.21.15.89:0/3959396254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c009550 con 0x7fc74c13c1e0 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc751d7a700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c09dc30 0x7fc74c13ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.700+0000 7fc752d7c700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc74c13c1e0 0x7fc74c13b0d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc74400b970 tx=0x7fc74400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.702+0000 7fc742ffd700 1 -- 172.21.15.89:0/3959396254 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400d860 con 0x7fc74c13c1e0 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.702+0000 7fc742ffd700 1 -- 172.21.15.89:0/3959396254 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74400d9e0 con 0x7fc74c13c1e0 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.702+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74c09f1a0 con 0x7fc74c13c1e0 2023-09-30T22:19:15.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.702+0000 7fc742ffd700 1 -- 172.21.15.89:0/3959396254 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400dd90 con 0x7fc74c13c1e0 2023-09-30T22:19:15.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.702+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c1200e0 con 0x7fc74c13c1e0 2023-09-30T22:19:15.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.703+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c071ff0 con 0x7fc74c13c1e0 2023-09-30T22:19:15.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.703+0000 7fc742ffd700 1 -- 172.21.15.89:0/3959396254 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc74400e430 con 0x7fc74c13c1e0 2023-09-30T22:19:15.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.703+0000 7fc742ffd700 1 -- 172.21.15.89:0/3959396254 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc744006e40 con 0x7fc74c13c1e0 2023-09-30T22:19:15.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.712+0000 7fc742ffd700 1 -- 172.21.15.89:0/3959396254 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc744007960 con 0x7fc74c13c1e0 2023-09-30T22:19:15.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.783+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd41009f420 con 0x7fd41009e440 2023-09-30T22:19:15.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.783+0000 7fd40d7fa700 1 -- 172.21.15.89:0/908684937 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd40801cbb0 con 0x7fd41009e440 2023-09-30T22:19:15.784 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009e440 msgr2=0x7fd4101a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 --2- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009e440 0x7fd4101a6950 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd40800efc0 tx=0x7fd40800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 shutdown_connections 2023-09-30T22:19:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 --2- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd41009e440 0x7fd4101a6950 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 --2- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd41009d6f0 0x7fd4101a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 --2- 172.21.15.89:0/908684937 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd41009e000 0x7fd4101a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 >> 172.21.15.89:0/908684937 conn(0x7fd410066f50 msgr2=0x7fd4101baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 shutdown_connections 2023-09-30T22:19:15.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.785+0000 7fd417150700 1 -- 172.21.15.89:0/908684937 wait complete. 2023-09-30T22:19:15.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.843+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5368119220 con 0x7f5368135e60 2023-09-30T22:19:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.844+0000 7f53657fa700 1 -- 172.21.15.89:0/1196411441 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5360012340 con 0x7f5368135e60 2023-09-30T22:19:15.845 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5368135e60 msgr2=0x7f536806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5368135e60 0x7f536806e4d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f536000efc0 tx=0x7f536000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 shutdown_connections 2023-09-30T22:19:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5368135e60 0x7f536806e4d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5368130540 0x7f53681c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 --2- 172.21.15.89:0/1196411441 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5368133540 0x7f53681c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 >> 172.21.15.89:0/1196411441 conn(0x7f53680a2aa0 msgr2=0x7f536806f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 shutdown_connections 2023-09-30T22:19:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.846+0000 7f536f09b700 1 -- 172.21.15.89:0/1196411441 wait complete. 2023-09-30T22:19:15.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:15.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.896+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faea8129030 con 0x7faea809e000 2023-09-30T22:19:15.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.897+0000 7fae8ffff700 1 -- 172.21.15.89:0/656637536 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faea8129030 con 0x7faea809e000 2023-09-30T22:19:15.898 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:15.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e000 msgr2=0x7faea8128050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e000 0x7faea8128050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00c780 tx=0x7fae9c00ca90 comp rx=0 tx=0).stop 2023-09-30T22:19:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 shutdown_connections 2023-09-30T22:19:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea809e410 0x7faea8128590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea809d6f0 0x7faea8127b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 --2- 172.21.15.89:0/656637536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea809e000 0x7faea8128050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 >> 172.21.15.89:0/656637536 conn(0x7faea8066f50 msgr2=0x7faea80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 shutdown_connections 2023-09-30T22:19:15.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.899+0000 7faeade1f700 1 -- 172.21.15.89:0/656637536 wait complete. 2023-09-30T22:19:15.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.910+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc74c11fcb0 con 0x7fc74c13c1e0 2023-09-30T22:19:15.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.910+0000 7fc742ffd700 1 -- 172.21.15.89:0/3959396254 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc744007370 con 0x7fc74c13c1e0 2023-09-30T22:19:15.911 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:15.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:15.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.912+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc74c13c1e0 msgr2=0x7fc74c13b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:15.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.912+0000 7fc7547df700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc74c13c1e0 0x7fc74c13b0d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc74400b970 tx=0x7fc74400bc80 comp rx=0 tx=0).stop 2023-09-30T22:19:15.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.913+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 shutdown_connections 2023-09-30T22:19:15.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.913+0000 7fc7547df700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc74c13c1e0 0x7fc74c13b0d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.913+0000 7fc7547df700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc74c06a770 0x7fc74c13a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.913+0000 7fc7547df700 1 --2- 172.21.15.89:0/3959396254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc74c09dc30 0x7fc74c13ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:15.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.913+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 >> 172.21.15.89:0/3959396254 conn(0x7fc74c066f50 msgr2=0x7fc74c0a1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:15.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.913+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 shutdown_connections 2023-09-30T22:19:15.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:15.913+0000 7fc7547df700 1 -- 172.21.15.89:0/3959396254 wait complete. 2023-09-30T22:19:15.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:16.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:16.859 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:16.912 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:16.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.939+0000 7f3e5198b700 1 Processor -- start 2023-09-30T22:19:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e5198b700 1 -- start start 2023-09-30T22:19:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e5198b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09d6f0 0x7f3e4c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e5198b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e4c09e000 0x7f3e4c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e5198b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c09b120 con 0x7f3e4c09e410 2023-09-30T22:19:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e5198b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c06b460 con 0x7f3e4c09e000 2023-09-30T22:19:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e5198b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c09a7e0 con 0x7f3e4c09d6f0 2023-09-30T22:19:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e4a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e4c09e000 0x7f3e4c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e4a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e4c09e000 0x7f3e4c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46826/0 (socket says 172.21.15.89:46826) 2023-09-30T22:19:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e4a7fc700 1 -- 172.21.15.89:0/1695494687 learned_addr learned my addr 172.21.15.89:0/1695494687 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e4affd700 1 --2- 172.21.15.89:0/1695494687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09d6f0 0x7f3e4c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e4a7fc700 1 -- 172.21.15.89:0/1695494687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09d6f0 msgr2=0x7f3e4c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.940+0000 7f3e4a7fc700 1 --2- 172.21.15.89:0/1695494687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09d6f0 0x7f3e4c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.941+0000 7f3e4a7fc700 1 -- 172.21.15.89:0/1695494687 >> v1:172.21.15.89:6789/0 conn(0x7f3e4c09e410 legacy=0x7f3e4c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.941+0000 7f3e4a7fc700 1 -- 172.21.15.89:0/1695494687 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c09a500 con 0x7f3e4c09e000 2023-09-30T22:19:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.941+0000 7f3e4a7fc700 1 --2- 172.21.15.89:0/1695494687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09e000 0x7f3e4c1339f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3e340098f0 tx=0x7f3e34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80e104866699f09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.941+0000 7f3e49ffb700 1 -- 172.21.15.89:0/1695494687 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34011830 con 0x7f3e4c09e000 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.941+0000 7f3e49ffb700 1 -- 172.21.15.89:0/1695494687 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e34011990 con 0x7f3e4c09e000 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.941+0000 7f3e49ffb700 1 -- 172.21.15.89:0/1695494687 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34011d20 con 0x7f3e4c09e000 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.942+0000 7f3e5198b700 1 -- 172.21.15.89:0/1695494687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09e000 msgr2=0x7f3e4c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.942+0000 7f3e5198b700 1 --2- 172.21.15.89:0/1695494687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09e000 0x7f3e4c1339f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3e340098f0 tx=0x7f3e34009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.942+0000 7f3e5198b700 1 -- 172.21.15.89:0/1695494687 shutdown_connections 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.942+0000 7f3e5198b700 1 --2- 172.21.15.89:0/1695494687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09d6f0 0x7f3e4c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.942+0000 7f3e5198b700 1 --2- 172.21.15.89:0/1695494687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09e000 0x7f3e4c1339f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.942+0000 7f3e5198b700 1 -- 172.21.15.89:0/1695494687 >> 172.21.15.89:0/1695494687 conn(0x7f3e4c066f50 msgr2=0x7f3e4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.942+0000 7f3e5198b700 1 -- 172.21.15.89:0/1695494687 shutdown_connections 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.942+0000 7f3e5198b700 1 -- 172.21.15.89:0/1695494687 wait complete. 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.943+0000 7f3e5198b700 1 Processor -- start 2023-09-30T22:19:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.944+0000 7f3e5198b700 1 -- start start 2023-09-30T22:19:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.944+0000 7f3e5198b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09d6f0 0x7f3e4c1a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.944+0000 7f3e5198b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09e000 0x7f3e4c1a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.944+0000 7f3e5198b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e4c09e410 0x7f3e4c1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.944+0000 7f3e5198b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1a91f0 con 0x7f3e4c09d6f0 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.944+0000 7f3e5198b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c09c710 con 0x7f3e4c09e000 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.944+0000 7f3e5198b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0a3160 con 0x7f3e4c09e410 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09e000 0x7f3e4c1a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09d6f0 0x7f3e4c1a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09e000 0x7f3e4c1a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37896/0 (socket says 172.21.15.89:37896) 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 -- 172.21.15.89:0/2951643408 learned_addr learned my addr 172.21.15.89:0/2951643408 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 -- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e4c09e410 msgr2=0x7f3e4c1a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 --2- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e4c09e410 0x7f3e4c1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 -- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09d6f0 msgr2=0x7f3e4c1a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 --2- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09d6f0 0x7f3e4c1a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 -- 172.21.15.89:0/2951643408 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e34009550 con 0x7f3e4c09e000 2023-09-30T22:19:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e4a7fc700 1 --2- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09e000 0x7f3e4c1a6510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009c40 tx=0x7f3e34004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e43fff700 1 -- 172.21.15.89:0/2951643408 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34004400 con 0x7f3e4c09e000 2023-09-30T22:19:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e43fff700 1 -- 172.21.15.89:0/2951643408 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3401f070 con 0x7f3e4c09e000 2023-09-30T22:19:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e43fff700 1 -- 172.21.15.89:0/2951643408 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e340038a0 con 0x7f3e4c09e000 2023-09-30T22:19:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c1a7b50 con 0x7f3e4c09e000 2023-09-30T22:19:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.945+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c1a9f60 con 0x7f3e4c09e000 2023-09-30T22:19:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.946+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c074ea0 con 0x7f3e4c09e000 2023-09-30T22:19:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.947+0000 7f3e43fff700 1 -- 172.21.15.89:0/2951643408 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3e34021030 con 0x7f3e4c09e000 2023-09-30T22:19:16.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.947+0000 7f3e43fff700 1 -- 172.21.15.89:0/2951643408 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3e34040ea0 con 0x7f3e4c09e000 2023-09-30T22:19:16.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.951+0000 7f3e43fff700 1 -- 172.21.15.89:0/2951643408 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3e340199a0 con 0x7f3e4c09e000 2023-09-30T22:19:16.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.996+0000 7fcf65a85700 1 Processor -- start 2023-09-30T22:19:16.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.996+0000 7fcf65a85700 1 -- start start 2023-09-30T22:19:16.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.996+0000 7fcf65a85700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf6009e1d0 0x7fcf60138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.996+0000 7fcf65a85700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf60139170 0x7fcf6013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.996+0000 7fcf65a85700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6009b120 con 0x7fcf6009d6f0 2023-09-30T22:19:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.996+0000 7fcf65a85700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6006b460 con 0x7fcf6009e1d0 2023-09-30T22:19:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.996+0000 7fcf65a85700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6009a7e0 con 0x7fcf60139170 2023-09-30T22:19:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf6009e1d0 0x7fcf60138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf6009e1d0 0x7fcf60138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46858/0 (socket says 172.21.15.89:46858) 2023-09-30T22:19:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5effd700 1 -- 172.21.15.89:0/2355233093 learned_addr learned my addr 172.21.15.89:0/2355233093 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:16.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5ffff700 1 --2- 172.21.15.89:0/2355233093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf60139170 0x7fcf6013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:16.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5effd700 1 -- 172.21.15.89:0/2355233093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf60139170 msgr2=0x7fcf6013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:16.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5effd700 1 --2- 172.21.15.89:0/2355233093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf60139170 0x7fcf6013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5effd700 1 -- 172.21.15.89:0/2355233093 >> v1:172.21.15.89:6789/0 conn(0x7fcf6009d6f0 legacy=0x7fcf6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5effd700 1 -- 172.21.15.89:0/2355233093 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6009a500 con 0x7fcf6009e1d0 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5effd700 1 --2- 172.21.15.89:0/2355233093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6009e1d0 0x7fcf60138c30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009920 tx=0x7fcf54009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b21822a5d5f2a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5e7fc700 1 -- 172.21.15.89:0/2355233093 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54011830 con 0x7fcf6009e1d0 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5e7fc700 1 -- 172.21.15.89:0/2355233093 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf54011990 con 0x7fcf6009e1d0 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.997+0000 7fcf5e7fc700 1 -- 172.21.15.89:0/2355233093 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54011d20 con 0x7fcf6009e1d0 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.998+0000 7fcf65a85700 1 -- 172.21.15.89:0/2355233093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6009e1d0 msgr2=0x7fcf60138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.998+0000 7fcf65a85700 1 --2- 172.21.15.89:0/2355233093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6009e1d0 0x7fcf60138c30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009920 tx=0x7fcf54009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.998+0000 7fcf65a85700 1 -- 172.21.15.89:0/2355233093 shutdown_connections 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.998+0000 7fcf65a85700 1 --2- 172.21.15.89:0/2355233093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf60139170 0x7fcf6013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.998+0000 7fcf65a85700 1 --2- 172.21.15.89:0/2355233093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6009e1d0 0x7fcf60138c30 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.998+0000 7fcf65a85700 1 -- 172.21.15.89:0/2355233093 >> 172.21.15.89:0/2355233093 conn(0x7fcf60066f50 msgr2=0x7fcf6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.998+0000 7fcf65a85700 1 -- 172.21.15.89:0/2355233093 shutdown_connections 2023-09-30T22:19:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.998+0000 7fcf65a85700 1 -- 172.21.15.89:0/2355233093 wait complete. 2023-09-30T22:19:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.999+0000 7fcf65a85700 1 Processor -- start 2023-09-30T22:19:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:16.999+0000 7fcf65a85700 1 -- start start 2023-09-30T22:19:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf65a85700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf6009d6f0 0x7fcf6011f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf65a85700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf6009e1d0 0x7fcf6011fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf65a85700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf60139170 0x7fcf601c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf65a85700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60121f20 con 0x7fcf60139170 2023-09-30T22:19:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf65a85700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf600a1b40 con 0x7fcf6009e1d0 2023-09-30T22:19:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf65a85700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6009c710 con 0x7fcf6009d6f0 2023-09-30T22:19:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf60139170 0x7fcf601c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf60139170 0x7fcf601c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46866/0 (socket says 172.21.15.89:46866) 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 -- 172.21.15.89:0/3225939525 learned_addr learned my addr 172.21.15.89:0/3225939525 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5effd700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf6009e1d0 0x7fcf6011fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5f7fe700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf6009d6f0 0x7fcf6011f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 -- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf6009d6f0 msgr2=0x7fcf6011f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf6009d6f0 0x7fcf6011f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 -- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf6009e1d0 msgr2=0x7fcf6011fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf6009e1d0 0x7fcf6011fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 -- 172.21.15.89:0/3225939525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54009550 con 0x7fcf60139170 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.000+0000 7fcf5ffff700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf60139170 0x7fcf601c8670 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800efc0 tx=0x7fcf5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.001+0000 7fcf47fff700 1 -- 172.21.15.89:0/3225939525 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800cc90 con 0x7fcf60139170 2023-09-30T22:19:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.001+0000 7fcf47fff700 1 -- 172.21.15.89:0/3225939525 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5800f040 con 0x7fcf60139170 2023-09-30T22:19:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.001+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf60120890 con 0x7fcf60139170 2023-09-30T22:19:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.001+0000 7fcf47fff700 1 -- 172.21.15.89:0/3225939525 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58014480 con 0x7fcf60139170 2023-09-30T22:19:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.001+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6011f090 con 0x7fcf60139170 2023-09-30T22:19:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.002+0000 7fcf47fff700 1 -- 172.21.15.89:0/3225939525 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf580145e0 con 0x7fcf60139170 2023-09-30T22:19:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.002+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf60074ea0 con 0x7fcf60139170 2023-09-30T22:19:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.002+0000 7fcf47fff700 1 -- 172.21.15.89:0/3225939525 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcf580402e0 con 0x7fcf60139170 2023-09-30T22:19:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.007+0000 7fcf47fff700 1 -- 172.21.15.89:0/3225939525 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf58007900 con 0x7fcf60139170 2023-09-30T22:19:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.048+0000 7fd824f5b700 1 Processor -- start 2023-09-30T22:19:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.048+0000 7fd824f5b700 1 -- start start 2023-09-30T22:19:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.048+0000 7fd824f5b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8200a0a30 0x7fd8200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.048+0000 7fd824f5b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd82009d630 0x7fd8200a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.048+0000 7fd824f5b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8200a2a70 con 0x7fd820133930 2023-09-30T22:19:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.048+0000 7fd824f5b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8200a01d0 con 0x7fd82009d630 2023-09-30T22:19:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.048+0000 7fd824f5b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd82009e2b0 con 0x7fd8200a0a30 2023-09-30T22:19:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8200a0a30 0x7fd8200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd82009d630 0x7fd8200a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8200a0a30 0x7fd8200a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60864/0 (socket says 172.21.15.89:60864) 2023-09-30T22:19:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81e59c700 1 -- 172.21.15.89:0/1315891194 learned_addr learned my addr 172.21.15.89:0/1315891194 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81e59c700 1 -- 172.21.15.89:0/1315891194 >> v2:172.21.15.89:3300/0 conn(0x7fd82009d630 msgr2=0x7fd8200a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81e59c700 1 --2- 172.21.15.89:0/1315891194 >> v2:172.21.15.89:3300/0 conn(0x7fd82009d630 0x7fd8200a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81e59c700 1 -- 172.21.15.89:0/1315891194 >> v1:172.21.15.89:6789/0 conn(0x7fd820133930 legacy=0x7fd820133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81e59c700 1 -- 172.21.15.89:0/1315891194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82009dc50 con 0x7fd8200a0a30 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81dd9b700 1 --2- 172.21.15.89:0/1315891194 >> v2:172.21.15.89:3300/0 conn(0x7fd82009d630 0x7fd8200a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81e59c700 1 --2- 172.21.15.89:0/1315891194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8200a0a30 0x7fd8200a0e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098f0 tx=0x7fd810009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c61a56b8be00d686 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81d59a700 1 -- 172.21.15.89:0/1315891194 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011830 con 0x7fd8200a0a30 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81d59a700 1 -- 172.21.15.89:0/1315891194 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810011990 con 0x7fd8200a0a30 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.049+0000 7fd81d59a700 1 -- 172.21.15.89:0/1315891194 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011d20 con 0x7fd8200a0a30 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.050+0000 7fd824f5b700 1 -- 172.21.15.89:0/1315891194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8200a0a30 msgr2=0x7fd8200a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.050+0000 7fd824f5b700 1 --2- 172.21.15.89:0/1315891194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8200a0a30 0x7fd8200a0e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098f0 tx=0x7fd810009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.050+0000 7fd824f5b700 1 -- 172.21.15.89:0/1315891194 shutdown_connections 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.050+0000 7fd824f5b700 1 --2- 172.21.15.89:0/1315891194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8200a0a30 0x7fd8200a0e00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.050+0000 7fd824f5b700 1 --2- 172.21.15.89:0/1315891194 >> v2:172.21.15.89:3300/0 conn(0x7fd82009d630 0x7fd8200a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.050+0000 7fd824f5b700 1 -- 172.21.15.89:0/1315891194 >> 172.21.15.89:0/1315891194 conn(0x7fd8200a3050 msgr2=0x7fd8200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.050+0000 7fd824f5b700 1 -- 172.21.15.89:0/1315891194 shutdown_connections 2023-09-30T22:19:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.050+0000 7fd824f5b700 1 -- 172.21.15.89:0/1315891194 wait complete. 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.051+0000 7fd824f5b700 1 Processor -- start 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.051+0000 7fd824f5b700 1 -- start start 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd824f5b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82009d630 0x7fd8201c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd824f5b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8200a0a30 0x7fd8201c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd824f5b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd820133930 0x7fd8201c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd824f5b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820122430 con 0x7fd82009d630 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd824f5b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd820120720 con 0x7fd8200a0a30 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd824f5b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200a5040 con 0x7fd820133930 2023-09-30T22:19:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd820133930 0x7fd8201c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82009d630 0x7fd8201c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd820133930 0x7fd8201c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60874/0 (socket says 172.21.15.89:60874) 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 -- 172.21.15.89:0/2228019581 learned_addr learned my addr 172.21.15.89:0/2228019581 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81dd9b700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8200a0a30 0x7fd8201c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 -- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8200a0a30 msgr2=0x7fd8201c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8200a0a30 0x7fd8201c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 -- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82009d630 msgr2=0x7fd8201c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82009d630 0x7fd8201c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 -- 172.21.15.89:0/2228019581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8140094a0 con 0x7fd820133930 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81dd9b700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8200a0a30 0x7fd8201c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.052+0000 7fd81ed9d700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd820133930 0x7fd8201c8c20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd81800cc80 tx=0x7fd81800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.053+0000 7fd80effd700 1 -- 172.21.15.89:0/2228019581 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818003c00 con 0x7fd820133930 2023-09-30T22:19:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.053+0000 7fd80effd700 1 -- 172.21.15.89:0/2228019581 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818003d60 con 0x7fd820133930 2023-09-30T22:19:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.053+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd810009550 con 0x7fd820133930 2023-09-30T22:19:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.053+0000 7fd80effd700 1 -- 172.21.15.89:0/2228019581 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81800ab50 con 0x7fd820133930 2023-09-30T22:19:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.053+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8201335c0 con 0x7fd820133930 2023-09-30T22:19:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.054+0000 7fd80effd700 1 -- 172.21.15.89:0/2228019581 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd81800acb0 con 0x7fd820133930 2023-09-30T22:19:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.054+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8200747f0 con 0x7fd820133930 2023-09-30T22:19:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.054+0000 7fd80effd700 1 -- 172.21.15.89:0/2228019581 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd81803f700 con 0x7fd820133930 2023-09-30T22:19:17.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.058+0000 7fd80effd700 1 -- 172.21.15.89:0/2228019581 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8180198f0 con 0x7fd820133930 2023-09-30T22:19:17.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.105+0000 7f60e10c2700 1 Processor -- start 2023-09-30T22:19:17.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.105+0000 7f60e10c2700 1 -- start start 2023-09-30T22:19:17.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.105+0000 7f60e10c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc0a0a30 0x7f60dc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.105+0000 7f60e10c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.105+0000 7f60e10c2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60dc0a2a70 con 0x7f60dc09d630 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.105+0000 7f60e10c2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc0a01d0 con 0x7f60dc138d80 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.105+0000 7f60e10c2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc09e2b0 con 0x7f60dc0a0a30 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60db59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60dad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc0a0a30 0x7f60dc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60db59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37924/0 (socket says 172.21.15.89:37924) 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60dad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc0a0a30 0x7f60dc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60882/0 (socket says 172.21.15.89:60882) 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60db59e700 1 -- 172.21.15.89:0/679006034 learned_addr learned my addr 172.21.15.89:0/679006034 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60da59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f60dc09d630 0x7f60dc0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39636/0 (socket says 172.21.15.89:39636) 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60d9d9b700 1 -- 172.21.15.89:0/679006034 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104695182 0 0) 0x7f60dc0a2a70 con 0x7f60dc09d630 2023-09-30T22:19:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60d9d9b700 1 -- 172.21.15.89:0/679006034 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60c40035b0 con 0x7f60dc09d630 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60dad9d700 1 -- 172.21.15.89:0/679006034 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 msgr2=0x7f60dc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60dad9d700 1 --2- 172.21.15.89:0/679006034 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60dad9d700 1 -- 172.21.15.89:0/679006034 >> v1:172.21.15.89:6789/0 conn(0x7f60dc09d630 legacy=0x7f60dc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60dad9d700 1 -- 172.21.15.89:0/679006034 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc09dc50 con 0x7f60dc0a0a30 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60db59e700 1 --2- 172.21.15.89:0/679006034 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60dad9d700 1 --2- 172.21.15.89:0/679006034 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc0a0a30 0x7f60dc0a0e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009870 tx=0x7f60cc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f6b3b5dd462a1d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60d9d9b700 1 -- 172.21.15.89:0/679006034 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc004230 con 0x7f60dc0a0a30 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60d9d9b700 1 -- 172.21.15.89:0/679006034 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc004390 con 0x7f60dc0a0a30 2023-09-30T22:19:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.106+0000 7f60d9d9b700 1 -- 172.21.15.89:0/679006034 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0036a0 con 0x7f60dc0a0a30 2023-09-30T22:19:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.107+0000 7f60e10c2700 1 -- 172.21.15.89:0/679006034 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc0a0a30 msgr2=0x7f60dc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.107+0000 7f60e10c2700 1 --2- 172.21.15.89:0/679006034 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc0a0a30 0x7f60dc0a0e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009870 tx=0x7f60cc009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.107+0000 7f60e10c2700 1 -- 172.21.15.89:0/679006034 shutdown_connections 2023-09-30T22:19:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.107+0000 7f60e10c2700 1 --2- 172.21.15.89:0/679006034 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.107+0000 7f60e10c2700 1 --2- 172.21.15.89:0/679006034 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc0a0a30 0x7f60dc0a0e00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.107+0000 7f60e10c2700 1 -- 172.21.15.89:0/679006034 >> 172.21.15.89:0/679006034 conn(0x7f60dc0a3050 msgr2=0x7f60dc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:17.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.107+0000 7f60e10c2700 1 -- 172.21.15.89:0/679006034 shutdown_connections 2023-09-30T22:19:17.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.107+0000 7f60e10c2700 1 -- 172.21.15.89:0/679006034 wait complete. 2023-09-30T22:19:17.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.109+0000 7f60e10c2700 1 Processor -- start 2023-09-30T22:19:17.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.109+0000 7f60e10c2700 1 -- start start 2023-09-30T22:19:17.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60e10c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc09d630 0x7f60dc137eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60e10c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60dc0a0a30 0x7f60dc1383f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60e10c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:17.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60e10c2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc12f5e0 con 0x7f60dc0a0a30 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60e10c2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc06eee0 con 0x7f60dc138d80 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60e10c2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc071550 con 0x7f60dc09d630 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60da59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60dc0a0a30 0x7f60dc1383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc06dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37932/0 (socket says 172.21.15.89:37932) 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 -- 172.21.15.89:0/1677792778 learned_addr learned my addr 172.21.15.89:0/1677792778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60dad9d700 1 --2- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc09d630 0x7f60dc137eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 -- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc09d630 msgr2=0x7f60dc137eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 --2- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc09d630 0x7f60dc137eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 -- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60dc0a0a30 msgr2=0x7f60dc1383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 --2- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60dc0a0a30 0x7f60dc1383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 -- 172.21.15.89:0/1677792778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d4009b30 con 0x7f60dc138d80 2023-09-30T22:19:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.110+0000 7f60db59e700 1 --2- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc06dac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f60d40070b0 tx=0x7f60d400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.111+0000 7f60c37fe700 1 -- 172.21.15.89:0/1677792778 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4003c20 con 0x7f60dc138d80 2023-09-30T22:19:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.111+0000 7f60c37fe700 1 -- 172.21.15.89:0/1677792778 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d400f040 con 0x7f60dc138d80 2023-09-30T22:19:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.111+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60cc0094a0 con 0x7f60dc138d80 2023-09-30T22:19:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.111+0000 7f60c37fe700 1 -- 172.21.15.89:0/1677792778 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d400c8c0 con 0x7f60dc138d80 2023-09-30T22:19:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.111+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc0a0410 con 0x7f60dc138d80 2023-09-30T22:19:17.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.112+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60dc0747f0 con 0x7f60dc138d80 2023-09-30T22:19:17.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.112+0000 7f60c37fe700 1 -- 172.21.15.89:0/1677792778 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f60d400ca20 con 0x7f60dc138d80 2023-09-30T22:19:17.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.113+0000 7f60c37fe700 1 -- 172.21.15.89:0/1677792778 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f60d403fab0 con 0x7f60dc138d80 2023-09-30T22:19:17.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.117+0000 7f60c37fe700 1 -- 172.21.15.89:0/1677792778 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60d401c770 con 0x7f60dc138d80 2023-09-30T22:19:17.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.154+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3e4c06c360 con 0x7f3e4c09e000 2023-09-30T22:19:17.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.154+0000 7f3e43fff700 1 -- 172.21.15.89:0/2951643408 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e34015030 con 0x7f3e4c09e000 2023-09-30T22:19:17.156 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:17.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.157+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09e000 msgr2=0x7f3e4c1a6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.157+0000 7f3e5198b700 1 --2- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09e000 0x7f3e4c1a6510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009c40 tx=0x7f3e34004750 comp rx=0 tx=0).stop 2023-09-30T22:19:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.157+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 shutdown_connections 2023-09-30T22:19:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.157+0000 7f3e5198b700 1 --2- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e4c09e410 0x7f3e4c1a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.157+0000 7f3e5198b700 1 --2- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4c09d6f0 0x7f3e4c1a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.157+0000 7f3e5198b700 1 --2- 172.21.15.89:0/2951643408 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4c09e000 0x7f3e4c1a6510 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.157+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 >> 172.21.15.89:0/2951643408 conn(0x7f3e4c066f50 msgr2=0x7f3e4c1baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.158+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 shutdown_connections 2023-09-30T22:19:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.158+0000 7f3e5198b700 1 -- 172.21.15.89:0/2951643408 wait complete. 2023-09-30T22:19:17.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:17.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.245+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcf6009f420 con 0x7fcf60139170 2023-09-30T22:19:17.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.245+0000 7fcf47fff700 1 -- 172.21.15.89:0/3225939525 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf58007330 con 0x7fcf60139170 2023-09-30T22:19:17.246 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.247+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf60139170 msgr2=0x7fcf601c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.247+0000 7fcf65a85700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf60139170 0x7fcf601c8670 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800efc0 tx=0x7fcf5800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.247+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 shutdown_connections 2023-09-30T22:19:17.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.247+0000 7fcf65a85700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf60139170 0x7fcf601c8670 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.247+0000 7fcf65a85700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf6009d6f0 0x7fcf6011f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.247+0000 7fcf65a85700 1 --2- 172.21.15.89:0/3225939525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf6009e1d0 0x7fcf6011fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.248+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 >> 172.21.15.89:0/3225939525 conn(0x7fcf60066f50 msgr2=0x7fcf60136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:17.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.248+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 shutdown_connections 2023-09-30T22:19:17.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.248+0000 7fcf65a85700 1 -- 172.21.15.89:0/3225939525 wait complete. 2023-09-30T22:19:17.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:17.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.300+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd820072980 con 0x7fd820133930 2023-09-30T22:19:17.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.301+0000 7fd80effd700 1 -- 172.21.15.89:0/2228019581 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd818019300 con 0x7fd820133930 2023-09-30T22:19:17.302 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd820133930 msgr2=0x7fd8201c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd820133930 0x7fd8201c8c20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd81800cc80 tx=0x7fd81800cf90 comp rx=0 tx=0).stop 2023-09-30T22:19:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 shutdown_connections 2023-09-30T22:19:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd820133930 0x7fd8201c8c20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82009d630 0x7fd8201c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 --2- 172.21.15.89:0/2228019581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8200a0a30 0x7fd8201c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 >> 172.21.15.89:0/2228019581 conn(0x7fd8200a3050 msgr2=0x7fd8200a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 shutdown_connections 2023-09-30T22:19:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.304+0000 7fd824f5b700 1 -- 172.21.15.89:0/2228019581 wait complete. 2023-09-30T22:19:17.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.365+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f60dc138930 con 0x7f60dc138d80 2023-09-30T22:19:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.365+0000 7f60c37fe700 1 -- 172.21.15.89:0/1677792778 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60d40133e0 con 0x7f60dc138d80 2023-09-30T22:19:17.366 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 msgr2=0x7f60dc06dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 --2- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc06dac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f60d40070b0 tx=0x7f60d400ef90 comp rx=0 tx=0).stop 2023-09-30T22:19:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 shutdown_connections 2023-09-30T22:19:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 --2- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60dc138d80 0x7f60dc06dac0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 --2- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60dc09d630 0x7f60dc137eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 --2- 172.21.15.89:0/1677792778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60dc0a0a30 0x7f60dc1383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 >> 172.21.15.89:0/1677792778 conn(0x7f60dc0a3050 msgr2=0x7f60dc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 shutdown_connections 2023-09-30T22:19:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:17.367+0000 7f60e10c2700 1 -- 172.21.15.89:0/1677792778 wait complete. 2023-09-30T22:19:17.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:18.172 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:18.261 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:18.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.308+0000 7faf37360700 1 Processor -- start 2023-09-30T22:19:18.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf37360700 1 -- start start 2023-09-30T22:19:18.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf37360700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf300a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf37360700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf300a1340 0x7faf30133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf37360700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3009edd0 con 0x7faf30133fa0 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf37360700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3009e1f0 con 0x7faf300a1340 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf37360700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf3009dc50 con 0x7faf300a0a30 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf348fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf300a1340 0x7faf30133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf350fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf300a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf348fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf300a1340 0x7faf30133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37946/0 (socket says 172.21.15.89:37946) 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf358fd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faf30133fa0 0x7faf3013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39648/0 (socket says 172.21.15.89:39648) 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf350fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf300a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60900/0 (socket says 172.21.15.89:60900) 2023-09-30T22:19:18.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.309+0000 7faf348fb700 1 -- 172.21.15.89:0/616996946 learned_addr learned my addr 172.21.15.89:0/616996946 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf27fff700 1 -- 172.21.15.89:0/616996946 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803531348 0 0) 0x7faf3009edd0 con 0x7faf30133fa0 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf348fb700 1 -- 172.21.15.89:0/616996946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 msgr2=0x7faf300a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf348fb700 1 --2- 172.21.15.89:0/616996946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf300a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf348fb700 1 -- 172.21.15.89:0/616996946 >> v1:172.21.15.89:6789/0 conn(0x7faf30133fa0 legacy=0x7faf3013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf348fb700 1 -- 172.21.15.89:0/616996946 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf300a1fe0 con 0x7faf300a1340 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf350fc700 1 --2- 172.21.15.89:0/616996946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf300a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf348fb700 1 --2- 172.21.15.89:0/616996946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf300a1340 0x7faf30133a60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faf20009920 tx=0x7faf20009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eda0aa9b67c2a955 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf27fff700 1 -- 172.21.15.89:0/616996946 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20011830 con 0x7faf300a1340 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf27fff700 1 -- 172.21.15.89:0/616996946 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf20011990 con 0x7faf300a1340 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.310+0000 7faf27fff700 1 -- 172.21.15.89:0/616996946 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20011d20 con 0x7faf300a1340 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.311+0000 7faf37360700 1 -- 172.21.15.89:0/616996946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf300a1340 msgr2=0x7faf30133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.311+0000 7faf37360700 1 --2- 172.21.15.89:0/616996946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf300a1340 0x7faf30133a60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faf20009920 tx=0x7faf20009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:18.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.311+0000 7faf37360700 1 -- 172.21.15.89:0/616996946 shutdown_connections 2023-09-30T22:19:18.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.311+0000 7faf37360700 1 --2- 172.21.15.89:0/616996946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf300a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.311+0000 7faf37360700 1 --2- 172.21.15.89:0/616996946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf300a1340 0x7faf30133a60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.311+0000 7faf37360700 1 -- 172.21.15.89:0/616996946 >> 172.21.15.89:0/616996946 conn(0x7faf300a6550 msgr2=0x7faf300a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:18.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.311+0000 7faf37360700 1 -- 172.21.15.89:0/616996946 shutdown_connections 2023-09-30T22:19:18.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.311+0000 7faf37360700 1 -- 172.21.15.89:0/616996946 wait complete. 2023-09-30T22:19:18.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf37360700 1 Processor -- start 2023-09-30T22:19:18.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf37360700 1 -- start start 2023-09-30T22:19:18.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf37360700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf301231b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf37360700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf300a1340 0x7faf301236f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf37360700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf30133fa0 0x7faf301c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf37360700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf30126160 con 0x7faf300a1340 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf37360700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf30122e30 con 0x7faf30133fa0 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf37360700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf30124e00 con 0x7faf300a0a30 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf348fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf300a1340 0x7faf301236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf348fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf300a1340 0x7faf301236f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46886/0 (socket says 172.21.15.89:46886) 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf348fb700 1 -- 172.21.15.89:0/4002958856 learned_addr learned my addr 172.21.15.89:0/4002958856 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf358fd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf30133fa0 0x7faf301c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.312+0000 7faf350fc700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf301231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf350fc700 1 -- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf30133fa0 msgr2=0x7faf301c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf350fc700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf30133fa0 0x7faf301c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf350fc700 1 -- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf300a1340 msgr2=0x7faf301236f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf350fc700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf300a1340 0x7faf301236f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf350fc700 1 -- 172.21.15.89:0/4002958856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0094a0 con 0x7faf300a0a30 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf348fb700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf300a1340 0x7faf301236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf358fd700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf30133fa0 0x7faf301c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf350fc700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf301231b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00e9b0 tx=0x7faf2c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf25ffb700 1 -- 172.21.15.89:0/4002958856 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00ee50 con 0x7faf300a0a30 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf25ffb700 1 -- 172.21.15.89:0/4002958856 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c00c850 con 0x7faf300a0a30 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf20009550 con 0x7faf300a0a30 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf25ffb700 1 -- 172.21.15.89:0/4002958856 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00cc00 con 0x7faf300a0a30 2023-09-30T22:19:18.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.313+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf300a25e0 con 0x7faf300a0a30 2023-09-30T22:19:18.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.314+0000 7faf25ffb700 1 -- 172.21.15.89:0/4002958856 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf2c013470 con 0x7faf300a0a30 2023-09-30T22:19:18.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.314+0000 7faf25ffb700 1 -- 172.21.15.89:0/4002958856 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faf2c03f6c0 con 0x7faf300a0a30 2023-09-30T22:19:18.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.314+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf30074ea0 con 0x7faf300a0a30 2023-09-30T22:19:18.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.319+0000 7faf25ffb700 1 -- 172.21.15.89:0/4002958856 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf2c013720 con 0x7faf300a0a30 2023-09-30T22:19:18.322 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:18.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:18.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.393+0000 7f24906f3700 1 Processor -- start 2023-09-30T22:19:18.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.393+0000 7f24906f3700 1 -- start start 2023-09-30T22:19:18.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f24906f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f2488135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f24906f3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2488135e50 0x7f2488138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f24906f3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248809f130 con 0x7f248809d6f0 2023-09-30T22:19:18.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f24906f3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248809f520 con 0x7f2488135e50 2023-09-30T22:19:18.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f24906f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f248809e930 con 0x7f248809e1d0 2023-09-30T22:19:18.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248dc8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f2488135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248dc8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f2488135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60924/0 (socket says 172.21.15.89:60924) 2023-09-30T22:19:18.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248dc8e700 1 -- 172.21.15.89:0/1248575908 learned_addr learned my addr 172.21.15.89:0/1248575908 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:18.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248ec90700 1 --2- 172.21.15.89:0/1248575908 >> v2:172.21.15.89:3300/0 conn(0x7f2488135e50 0x7f2488138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248dc8e700 1 -- 172.21.15.89:0/1248575908 >> v2:172.21.15.89:3300/0 conn(0x7f2488135e50 msgr2=0x7f2488138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248dc8e700 1 --2- 172.21.15.89:0/1248575908 >> v2:172.21.15.89:3300/0 conn(0x7f2488135e50 0x7f2488138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248dc8e700 1 -- 172.21.15.89:0/1248575908 >> v1:172.21.15.89:6789/0 conn(0x7f248809d6f0 legacy=0x7f248809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248dc8e700 1 -- 172.21.15.89:0/1248575908 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248809b2d0 con 0x7f248809e1d0 2023-09-30T22:19:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.394+0000 7f248dc8e700 1 --2- 172.21.15.89:0/1248575908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f2488135910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2478009920 tx=0x7f2478009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca4ba6022fd46292 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f248d48d700 1 -- 172.21.15.89:0/1248575908 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478011830 con 0x7f248809e1d0 2023-09-30T22:19:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f248d48d700 1 -- 172.21.15.89:0/1248575908 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2478011990 con 0x7f248809e1d0 2023-09-30T22:19:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f248d48d700 1 -- 172.21.15.89:0/1248575908 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478011d20 con 0x7f248809e1d0 2023-09-30T22:19:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f24906f3700 1 -- 172.21.15.89:0/1248575908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 msgr2=0x7f2488135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f24906f3700 1 --2- 172.21.15.89:0/1248575908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f2488135910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2478009920 tx=0x7f2478009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f24906f3700 1 -- 172.21.15.89:0/1248575908 shutdown_connections 2023-09-30T22:19:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f24906f3700 1 --2- 172.21.15.89:0/1248575908 >> v2:172.21.15.89:3300/0 conn(0x7f2488135e50 0x7f2488138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f24906f3700 1 --2- 172.21.15.89:0/1248575908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f2488135910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f24906f3700 1 -- 172.21.15.89:0/1248575908 >> 172.21.15.89:0/1248575908 conn(0x7f2488066f50 msgr2=0x7f248809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.395+0000 7f24906f3700 1 -- 172.21.15.89:0/1248575908 shutdown_connections 2023-09-30T22:19:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.396+0000 7f24906f3700 1 -- 172.21.15.89:0/1248575908 wait complete. 2023-09-30T22:19:18.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.396+0000 7f24906f3700 1 Processor -- start 2023-09-30T22:19:18.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.396+0000 7f24906f3700 1 -- start start 2023-09-30T22:19:18.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f24906f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248809d6f0 0x7f2488134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f24906f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f24881353e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f24906f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2488135e50 0x7f2488139780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f24906f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881342c0 con 0x7f248809d6f0 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f24906f3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24880a22f0 con 0x7f2488135e50 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f24906f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2488133a50 con 0x7f248809e1d0 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248dc8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f24881353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248dc8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f24881353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60938/0 (socket says 172.21.15.89:60938) 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248dc8e700 1 -- 172.21.15.89:0/3111377966 learned_addr learned my addr 172.21.15.89:0/3111377966 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248e48f700 1 --2- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248809d6f0 0x7f2488134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248dc8e700 1 -- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2488135e50 msgr2=0x7f2488139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248dc8e700 1 --2- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2488135e50 0x7f2488139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248dc8e700 1 -- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248809d6f0 msgr2=0x7f2488134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248dc8e700 1 --2- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248809d6f0 0x7f2488134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.397+0000 7f248dc8e700 1 -- 172.21.15.89:0/3111377966 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478009550 con 0x7f248809e1d0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.398+0000 7f248dc8e700 1 --2- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f24881353e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24780098f0 tx=0x7f2478004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.398+0000 7f247effd700 1 -- 172.21.15.89:0/3111377966 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478004300 con 0x7f248809e1d0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.398+0000 7f247effd700 1 -- 172.21.15.89:0/3111377966 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247801f070 con 0x7f248809e1d0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.398+0000 7f247effd700 1 -- 172.21.15.89:0/3111377966 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24780036a0 con 0x7f248809e1d0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.398+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2488133600 con 0x7f248809e1d0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.398+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881303a0 con 0x7f248809e1d0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.399+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2488074ea0 con 0x7f248809e1d0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.399+0000 7f247effd700 1 -- 172.21.15.89:0/3111377966 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2478021030 con 0x7f248809e1d0 2023-09-30T22:19:18.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.399+0000 7f247effd700 1 -- 172.21.15.89:0/3111377966 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f24780408c0 con 0x7f248809e1d0 2023-09-30T22:19:18.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.407+0000 7f247effd700 1 -- 172.21.15.89:0/3111377966 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24780239c0 con 0x7f248809e1d0 2023-09-30T22:19:18.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.450+0000 7f0891b83700 1 Processor -- start 2023-09-30T22:19:18.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.450+0000 7f0891b83700 1 -- start start 2023-09-30T22:19:18.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f0891b83700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f088c0a0a30 0x7f088c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f0891b83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c09d630 0x7f088c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f0891b83700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0a2a70 con 0x7f088c133930 2023-09-30T22:19:18.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f0891b83700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0a01d0 con 0x7f088c0a0a30 2023-09-30T22:19:18.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f0891b83700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f088c09e2b0 con 0x7f088c09d630 2023-09-30T22:19:18.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f088affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c09d630 0x7f088c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f088affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c09d630 0x7f088c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60948/0 (socket says 172.21.15.89:60948) 2023-09-30T22:19:18.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f088affd700 1 -- 172.21.15.89:0/978208825 learned_addr learned my addr 172.21.15.89:0/978208825 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:18.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f088b7fe700 1 --2- 172.21.15.89:0/978208825 >> v2:172.21.15.89:3300/0 conn(0x7f088c0a0a30 0x7f088c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f088affd700 1 -- 172.21.15.89:0/978208825 >> v2:172.21.15.89:3300/0 conn(0x7f088c0a0a30 msgr2=0x7f088c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f088affd700 1 --2- 172.21.15.89:0/978208825 >> v2:172.21.15.89:3300/0 conn(0x7f088c0a0a30 0x7f088c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.451+0000 7f088affd700 1 -- 172.21.15.89:0/978208825 >> v1:172.21.15.89:6789/0 conn(0x7f088c133930 legacy=0x7f088c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.452+0000 7f088affd700 1 -- 172.21.15.89:0/978208825 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c09dc50 con 0x7f088c09d630 2023-09-30T22:19:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.452+0000 7f088affd700 1 --2- 172.21.15.89:0/978208825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c09d630 0x7f088c0a1340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f08800098f0 tx=0x7f0880009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fee0c334b0facdb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.452+0000 7f088a7fc700 1 -- 172.21.15.89:0/978208825 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880011830 con 0x7f088c09d630 2023-09-30T22:19:18.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.452+0000 7f088a7fc700 1 -- 172.21.15.89:0/978208825 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0880011990 con 0x7f088c09d630 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.452+0000 7f088a7fc700 1 -- 172.21.15.89:0/978208825 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880011d20 con 0x7f088c09d630 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.452+0000 7f0891b83700 1 -- 172.21.15.89:0/978208825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c09d630 msgr2=0x7f088c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.452+0000 7f0891b83700 1 --2- 172.21.15.89:0/978208825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c09d630 0x7f088c0a1340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f08800098f0 tx=0x7f0880009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.453+0000 7f0891b83700 1 -- 172.21.15.89:0/978208825 shutdown_connections 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.453+0000 7f0891b83700 1 --2- 172.21.15.89:0/978208825 >> v2:172.21.15.89:3300/0 conn(0x7f088c0a0a30 0x7f088c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.453+0000 7f0891b83700 1 --2- 172.21.15.89:0/978208825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c09d630 0x7f088c0a1340 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.453+0000 7f0891b83700 1 -- 172.21.15.89:0/978208825 >> 172.21.15.89:0/978208825 conn(0x7f088c0a3050 msgr2=0x7f088c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.453+0000 7f0891b83700 1 -- 172.21.15.89:0/978208825 shutdown_connections 2023-09-30T22:19:18.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.453+0000 7f0891b83700 1 -- 172.21.15.89:0/978208825 wait complete. 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.454+0000 7f0891b83700 1 Processor -- start 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.454+0000 7f0891b83700 1 -- start start 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f0891b83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f088c09d630 0x7f088c11f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f0891b83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c0a0a30 0x7f088c11f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f0891b83700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f088c133930 0x7f088c1c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f0891b83700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c121fb0 con 0x7f088c09d630 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f0891b83700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c11eef0 con 0x7f088c133930 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f0891b83700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f088c120640 con 0x7f088c0a0a30 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f088c09d630 0x7f088c11f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f088c09d630 0x7f088c11f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46912/0 (socket says 172.21.15.89:46912) 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088b7fe700 1 -- 172.21.15.89:0/2311747543 learned_addr learned my addr 172.21.15.89:0/2311747543 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088affd700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c0a0a30 0x7f088c11f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088bfff700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f088c133930 0x7f088c1c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088affd700 1 -- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f088c133930 msgr2=0x7f088c1c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088affd700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f088c133930 0x7f088c1c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088affd700 1 -- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f088c09d630 msgr2=0x7f088c11f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088affd700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f088c09d630 0x7f088c11f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088affd700 1 -- 172.21.15.89:0/2311747543 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0880009550 con 0x7f088c0a0a30 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.455+0000 7f088bfff700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f088c133930 0x7f088c1c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.456+0000 7f088affd700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c0a0a30 0x7f088c11f9d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0880016000 tx=0x7f0880004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:18.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.456+0000 7f0873fff700 1 -- 172.21.15.89:0/2311747543 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800036a0 con 0x7f088c0a0a30 2023-09-30T22:19:18.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.456+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c0a5040 con 0x7f088c0a0a30 2023-09-30T22:19:18.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.456+0000 7f0873fff700 1 -- 172.21.15.89:0/2311747543 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0880003800 con 0x7f088c0a0a30 2023-09-30T22:19:18.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.456+0000 7f0873fff700 1 -- 172.21.15.89:0/2311747543 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880003bb0 con 0x7f088c0a0a30 2023-09-30T22:19:18.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.456+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c1334e0 con 0x7f088c0a0a30 2023-09-30T22:19:18.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.457+0000 7f0873fff700 1 -- 172.21.15.89:0/2311747543 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0880021070 con 0x7f088c0a0a30 2023-09-30T22:19:18.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.457+0000 7f0873fff700 1 -- 172.21.15.89:0/2311747543 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0880040b10 con 0x7f088c0a0a30 2023-09-30T22:19:18.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.457+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c11ff10 con 0x7f088c0a0a30 2023-09-30T22:19:18.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.466+0000 7f0873fff700 1 -- 172.21.15.89:0/2311747543 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08800233d0 con 0x7f088c0a0a30 2023-09-30T22:19:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.512+0000 7f3e54a93700 1 Processor -- start 2023-09-30T22:19:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.512+0000 7f3e54a93700 1 -- start start 2023-09-30T22:19:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.513+0000 7f3e54a93700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009d6f0 0x7f3e5009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.513+0000 7f3e54a93700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009e410 0x7f3e50130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.513+0000 7f3e54a93700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5009f130 con 0x7f3e5009e000 2023-09-30T22:19:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.513+0000 7f3e54a93700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5009f520 con 0x7f3e5009e410 2023-09-30T22:19:18.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.513+0000 7f3e54a93700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5009e930 con 0x7f3e5009d6f0 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4dd9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3e5009e000 0x7f3e50130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39670/0 (socket says 172.21.15.89:39670) 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4dd9b700 1 -- 172.21.15.89:0/325292808 learned_addr learned my addr 172.21.15.89:0/325292808 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4e59c700 1 --2- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009d6f0 0x7f3e5009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4ed9d700 1 --2- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009e410 0x7f3e50130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4d59a700 1 -- 172.21.15.89:0/325292808 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252294156 0 0) 0x7f3e5009f130 con 0x7f3e5009e000 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4ed9d700 1 -- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009d6f0 msgr2=0x7f3e5009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4ed9d700 1 --2- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009d6f0 0x7f3e5009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4ed9d700 1 -- 172.21.15.89:0/325292808 >> v1:172.21.15.89:6789/0 conn(0x7f3e5009e000 legacy=0x7f3e50130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4ed9d700 1 -- 172.21.15.89:0/325292808 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5009b2d0 con 0x7f3e5009e410 2023-09-30T22:19:18.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4ed9d700 1 --2- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009e410 0x7f3e50130d70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009dc0 tx=0x7f3e4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2614e9bdb5e57da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4d59a700 1 -- 172.21.15.89:0/325292808 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800e070 con 0x7f3e5009e410 2023-09-30T22:19:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4d59a700 1 -- 172.21.15.89:0/325292808 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4800bb20 con 0x7f3e5009e410 2023-09-30T22:19:18.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.514+0000 7f3e4d59a700 1 -- 172.21.15.89:0/325292808 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48012430 con 0x7f3e5009e410 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.515+0000 7f3e54a93700 1 -- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009e410 msgr2=0x7f3e50130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.515+0000 7f3e54a93700 1 --2- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009e410 0x7f3e50130d70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009dc0 tx=0x7f3e4800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.515+0000 7f3e54a93700 1 -- 172.21.15.89:0/325292808 shutdown_connections 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.515+0000 7f3e54a93700 1 --2- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009e410 0x7f3e50130d70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.515+0000 7f3e54a93700 1 --2- 172.21.15.89:0/325292808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009d6f0 0x7f3e5009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.515+0000 7f3e54a93700 1 -- 172.21.15.89:0/325292808 >> 172.21.15.89:0/325292808 conn(0x7f3e50066f50 msgr2=0x7f3e5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.515+0000 7f3e54a93700 1 -- 172.21.15.89:0/325292808 shutdown_connections 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.515+0000 7f3e54a93700 1 -- 172.21.15.89:0/325292808 wait complete. 2023-09-30T22:19:18.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.517+0000 7f3e54a93700 1 Processor -- start 2023-09-30T22:19:18.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.517+0000 7f3e54a93700 1 -- start start 2023-09-30T22:19:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e54a93700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009d6f0 0x7f3e50136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e54a93700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009e000 0x7f3e50137060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e54a93700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e5009e410 0x7f3e501375a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e54a93700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500a3860 con 0x7f3e5009e410 2023-09-30T22:19:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e54a93700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5013b330 con 0x7f3e5009d6f0 2023-09-30T22:19:18.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e54a93700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5013a360 con 0x7f3e5009e000 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e5009e410 0x7f3e501375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4e59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009d6f0 0x7f3e50136b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e5009e410 0x7f3e501375a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46914/0 (socket says 172.21.15.89:46914) 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 -- 172.21.15.89:0/373013396 learned_addr learned my addr 172.21.15.89:0/373013396 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4dd9b700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009e000 0x7f3e50137060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 -- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009e000 msgr2=0x7f3e50137060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009e000 0x7f3e50137060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 -- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009d6f0 msgr2=0x7f3e50136b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009d6f0 0x7f3e50136b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 -- 172.21.15.89:0/373013396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48009a70 con 0x7f3e5009e410 2023-09-30T22:19:18.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4e59c700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009d6f0 0x7f3e50136b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e4ed9d700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e5009e410 0x7f3e501375a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3e48014040 tx=0x7f3e48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.518+0000 7f3e3effd700 1 -- 172.21.15.89:0/373013396 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48004930 con 0x7f3e5009e410 2023-09-30T22:19:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.519+0000 7f3e3effd700 1 -- 172.21.15.89:0/373013396 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e48020070 con 0x7f3e5009e410 2023-09-30T22:19:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.519+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e50138f20 con 0x7f3e5009e410 2023-09-30T22:19:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.519+0000 7f3e3effd700 1 -- 172.21.15.89:0/373013396 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e480276c0 con 0x7f3e5009e410 2023-09-30T22:19:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.519+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e50138c90 con 0x7f3e5009e410 2023-09-30T22:19:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.519+0000 7f3e3effd700 1 -- 172.21.15.89:0/373013396 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3e48012430 con 0x7f3e5009e410 2023-09-30T22:19:18.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.519+0000 7f3e3effd700 1 -- 172.21.15.89:0/373013396 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3e48022070 con 0x7f3e5009e410 2023-09-30T22:19:18.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.520+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50131240 con 0x7f3e5009e410 2023-09-30T22:19:18.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.529+0000 7f3e3effd700 1 -- 172.21.15.89:0/373013396 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3e48025970 con 0x7f3e5009e410 2023-09-30T22:19:18.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.533+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faf300a28e0 con 0x7faf300a0a30 2023-09-30T22:19:18.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.534+0000 7faf25ffb700 1 -- 172.21.15.89:0/4002958856 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf2c021030 con 0x7faf300a0a30 2023-09-30T22:19:18.534 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:18.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.536+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 msgr2=0x7faf301231b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.536+0000 7faf37360700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf301231b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00e9b0 tx=0x7faf2c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.538+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 shutdown_connections 2023-09-30T22:19:18.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.538+0000 7faf37360700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf30133fa0 0x7faf301c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.538+0000 7faf37360700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf300a0a30 0x7faf301231b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.538+0000 7faf37360700 1 --2- 172.21.15.89:0/4002958856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf300a1340 0x7faf301236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.538+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 >> 172.21.15.89:0/4002958856 conn(0x7faf300a6550 msgr2=0x7faf3011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.538+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 shutdown_connections 2023-09-30T22:19:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.538+0000 7faf37360700 1 -- 172.21.15.89:0/4002958856 wait complete. 2023-09-30T22:19:18.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:18.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.608+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f248806c360 con 0x7f248809e1d0 2023-09-30T22:19:18.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.609+0000 7f247effd700 1 -- 172.21.15.89:0/3111377966 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2478015030 con 0x7f248809e1d0 2023-09-30T22:19:18.610 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 msgr2=0x7f24881353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 --2- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f24881353e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24780098f0 tx=0x7f2478004750 comp rx=0 tx=0).stop 2023-09-30T22:19:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 shutdown_connections 2023-09-30T22:19:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 --2- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2488135e50 0x7f2488139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 --2- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f248809d6f0 0x7f2488134ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 --2- 172.21.15.89:0/3111377966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f248809e1d0 0x7f24881353e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 >> 172.21.15.89:0/3111377966 conn(0x7f2488066f50 msgr2=0x7f248806e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 shutdown_connections 2023-09-30T22:19:18.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.611+0000 7f24906f3700 1 -- 172.21.15.89:0/3111377966 wait complete. 2023-09-30T22:19:18.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.699+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f088c0747f0 con 0x7f088c0a0a30 2023-09-30T22:19:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.700+0000 7f0873fff700 1 -- 172.21.15.89:0/2311747543 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0880015020 con 0x7f088c0a0a30 2023-09-30T22:19:18.701 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c0a0a30 msgr2=0x7f088c11f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c0a0a30 0x7f088c11f9d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0880016000 tx=0x7f0880004750 comp rx=0 tx=0).stop 2023-09-30T22:19:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 shutdown_connections 2023-09-30T22:19:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f088c133930 0x7f088c1c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f088c09d630 0x7f088c11f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 --2- 172.21.15.89:0/2311747543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f088c0a0a30 0x7f088c11f9d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 >> 172.21.15.89:0/2311747543 conn(0x7f088c0a3050 msgr2=0x7f088c1bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 shutdown_connections 2023-09-30T22:19:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.702+0000 7f0891b83700 1 -- 172.21.15.89:0/2311747543 wait complete. 2023-09-30T22:19:18.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.758+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3e50074ea0 con 0x7f3e5009e410 2023-09-30T22:19:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.758+0000 7f3e3effd700 1 -- 172.21.15.89:0/373013396 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e48025380 con 0x7f3e5009e410 2023-09-30T22:19:18.760 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e5009e410 msgr2=0x7f3e501375a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e5009e410 0x7f3e501375a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3e48014040 tx=0x7f3e48004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 shutdown_connections 2023-09-30T22:19:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e5009e410 0x7f3e501375a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e5009d6f0 0x7f3e50136b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 --2- 172.21.15.89:0/373013396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e5009e000 0x7f3e50137060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 >> 172.21.15.89:0/373013396 conn(0x7f3e50066f50 msgr2=0x7f3e50070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 shutdown_connections 2023-09-30T22:19:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:18.760+0000 7f3e54a93700 1 -- 172.21.15.89:0/373013396 wait complete. 2023-09-30T22:19:18.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:19.553 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:19.626 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:19.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.691+0000 7fb11cf2c700 1 Processor -- start 2023-09-30T22:19:19.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.691+0000 7fb11cf2c700 1 -- start start 2023-09-30T22:19:19.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.691+0000 7fb11cf2c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.691+0000 7fb11cf2c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb11813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.691+0000 7fb11cf2c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11809edd0 con 0x7fb1180a1340 2023-09-30T22:19:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.691+0000 7fb11cf2c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11809e1f0 con 0x7fb118134170 2023-09-30T22:19:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.691+0000 7fb11cf2c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb11809dc50 con 0x7fb1180a0a30 2023-09-30T22:19:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb116d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb11813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb115d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb1180a1340 0x7fb118133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39672/0 (socket says 172.21.15.89:39672) 2023-09-30T22:19:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb115d9b700 1 -- 172.21.15.89:0/159547714 learned_addr learned my addr 172.21.15.89:0/159547714 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11559a700 1 -- 172.21.15.89:0/159547714 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465574039 0 0) 0x7fb11809edd0 con 0x7fb1180a1340 2023-09-30T22:19:19.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11659c700 1 -- 172.21.15.89:0/159547714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 msgr2=0x7fb11813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11659c700 1 --2- 172.21.15.89:0/159547714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb11813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11659c700 1 -- 172.21.15.89:0/159547714 >> v1:172.21.15.89:6789/0 conn(0x7fb1180a1340 legacy=0x7fb118133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11659c700 1 -- 172.21.15.89:0/159547714 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1180a1fe0 con 0x7fb1180a0a30 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb116d9d700 1 --2- 172.21.15.89:0/159547714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb11813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11659c700 1 --2- 172.21.15.89:0/159547714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1180a0e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb108009870 tx=0x7fb108009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16fe407d407f3604 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11559a700 1 -- 172.21.15.89:0/159547714 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108011830 con 0x7fb1180a0a30 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11559a700 1 -- 172.21.15.89:0/159547714 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb108011990 con 0x7fb1180a0a30 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.692+0000 7fb11559a700 1 -- 172.21.15.89:0/159547714 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108011d20 con 0x7fb1180a0a30 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.693+0000 7fb11cf2c700 1 -- 172.21.15.89:0/159547714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 msgr2=0x7fb1180a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.693+0000 7fb11cf2c700 1 --2- 172.21.15.89:0/159547714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1180a0e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb108009870 tx=0x7fb108009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.693+0000 7fb11cf2c700 1 -- 172.21.15.89:0/159547714 shutdown_connections 2023-09-30T22:19:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.693+0000 7fb11cf2c700 1 --2- 172.21.15.89:0/159547714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb11813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.693+0000 7fb11cf2c700 1 --2- 172.21.15.89:0/159547714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1180a0e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.693+0000 7fb11cf2c700 1 -- 172.21.15.89:0/159547714 >> 172.21.15.89:0/159547714 conn(0x7fb1180a6550 msgr2=0x7fb1180a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.693+0000 7fb11cf2c700 1 -- 172.21.15.89:0/159547714 shutdown_connections 2023-09-30T22:19:19.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.693+0000 7fb11cf2c700 1 -- 172.21.15.89:0/159547714 wait complete. 2023-09-30T22:19:19.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.694+0000 7fb11cf2c700 1 Processor -- start 2023-09-30T22:19:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.694+0000 7fb11cf2c700 1 -- start start 2023-09-30T22:19:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb11cf2c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1181227b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb11cf2c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180a1340 0x7fb118122cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb11cf2c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb118123230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb11cf2c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb118125f60 con 0x7fb1180a1340 2023-09-30T22:19:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb11cf2c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb118124440 con 0x7fb118134170 2023-09-30T22:19:19.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb11cf2c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180a4d60 con 0x7fb1180a0a30 2023-09-30T22:19:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb115d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180a1340 0x7fb118122cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb116d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb118123230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb116d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb118123230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38024/0 (socket says 172.21.15.89:38024) 2023-09-30T22:19:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb115d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180a1340 0x7fb118122cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46930/0 (socket says 172.21.15.89:46930) 2023-09-30T22:19:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb116d9d700 1 -- 172.21.15.89:0/1627089096 learned_addr learned my addr 172.21.15.89:0/1627089096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.695+0000 7fb11659c700 1 --2- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1181227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb116d9d700 1 -- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 msgr2=0x7fb1181227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb116d9d700 1 --2- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1181227b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb116d9d700 1 -- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180a1340 msgr2=0x7fb118122cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb116d9d700 1 --2- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180a1340 0x7fb118122cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb116d9d700 1 -- 172.21.15.89:0/1627089096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb110009a70 con 0x7fb118134170 2023-09-30T22:19:19.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb116d9d700 1 --2- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb118123230 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb11000efc0 tx=0x7fb11000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:19.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb106ffd700 1 -- 172.21.15.89:0/1627089096 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000ca90 con 0x7fb118134170 2023-09-30T22:19:19.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb106ffd700 1 -- 172.21.15.89:0/1627089096 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11000f040 con 0x7fb118134170 2023-09-30T22:19:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1080094a0 con 0x7fb118134170 2023-09-30T22:19:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb106ffd700 1 -- 172.21.15.89:0/1627089096 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000ce10 con 0x7fb118134170 2023-09-30T22:19:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.696+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1180a25e0 con 0x7fb118134170 2023-09-30T22:19:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.697+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb118074ea0 con 0x7fb118134170 2023-09-30T22:19:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.697+0000 7fb106ffd700 1 -- 172.21.15.89:0/1627089096 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb110014480 con 0x7fb118134170 2023-09-30T22:19:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.697+0000 7fb106ffd700 1 -- 172.21.15.89:0/1627089096 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb11003fcf0 con 0x7fb118134170 2023-09-30T22:19:19.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.707+0000 7fb106ffd700 1 -- 172.21.15.89:0/1627089096 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb110012930 con 0x7fb118134170 2023-09-30T22:19:19.717 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:19.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.761+0000 7ff5b57b4700 1 Processor -- start 2023-09-30T22:19:19.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.761+0000 7ff5b57b4700 1 -- start start 2023-09-30T22:19:19.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5b57b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5b009d9f0 0x7ff5b0136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5b57b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b0138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5b57b4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b009e130 con 0x7ff5b01316c0 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5b57b4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b01321a0 con 0x7ff5b009d9f0 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5b57b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01312b0 con 0x7ff5b0136a70 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5ae7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5b009d9f0 0x7ff5b0136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5ae7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5b009d9f0 0x7ff5b0136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46940/0 (socket says 172.21.15.89:46940) 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5ae7fc700 1 -- 172.21.15.89:0/2384126109 learned_addr learned my addr 172.21.15.89:0/2384126109 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5af7fe700 1 --2- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b0138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5ae7fc700 1 -- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 msgr2=0x7ff5b0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5ae7fc700 1 --2- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b0138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5ae7fc700 1 -- 172.21.15.89:0/2384126109 >> v1:172.21.15.89:6789/0 conn(0x7ff5b01316c0 legacy=0x7ff5b0131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5ae7fc700 1 -- 172.21.15.89:0/2384126109 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b009f6e0 con 0x7ff5b009d9f0 2023-09-30T22:19:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5af7fe700 1 --2- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b0138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:19.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.762+0000 7ff5ae7fc700 1 --2- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b009d9f0 0x7ff5b0136530 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40098f0 tx=0x7ff5a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c96fdd607b13b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:19.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5adffb700 1 -- 172.21.15.89:0/2384126109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4004230 con 0x7ff5b009d9f0 2023-09-30T22:19:19.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5adffb700 1 -- 172.21.15.89:0/2384126109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a4004390 con 0x7ff5b009d9f0 2023-09-30T22:19:19.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5adffb700 1 -- 172.21.15.89:0/2384126109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40036a0 con 0x7ff5b009d9f0 2023-09-30T22:19:19.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5b57b4700 1 -- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b009d9f0 msgr2=0x7ff5b0136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5b57b4700 1 --2- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b009d9f0 0x7ff5b0136530 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40098f0 tx=0x7ff5a4009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:19.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5b57b4700 1 -- 172.21.15.89:0/2384126109 shutdown_connections 2023-09-30T22:19:19.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5b57b4700 1 --2- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b0138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5b57b4700 1 --2- 172.21.15.89:0/2384126109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b009d9f0 0x7ff5b0136530 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.763+0000 7ff5b57b4700 1 -- 172.21.15.89:0/2384126109 >> 172.21.15.89:0/2384126109 conn(0x7ff5b0066f50 msgr2=0x7ff5b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:19.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.764+0000 7ff5b57b4700 1 -- 172.21.15.89:0/2384126109 shutdown_connections 2023-09-30T22:19:19.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.764+0000 7ff5b57b4700 1 -- 172.21.15.89:0/2384126109 wait complete. 2023-09-30T22:19:19.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.765+0000 7ff5b57b4700 1 Processor -- start 2023-09-30T22:19:19.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.765+0000 7ff5b57b4700 1 -- start start 2023-09-30T22:19:19.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5b57b4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5b009d9f0 0x7ff5b0070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5b57b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b01316c0 0x7ff5b0071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5b57b4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b00719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5b57b4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b006fdc0 con 0x7ff5b01316c0 2023-09-30T22:19:19.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5b57b4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b012b900 con 0x7ff5b009d9f0 2023-09-30T22:19:19.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5b57b4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0135f50 con 0x7ff5b0136a70 2023-09-30T22:19:19.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b01316c0 0x7ff5b0071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b01316c0 0x7ff5b0071480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46944/0 (socket says 172.21.15.89:46944) 2023-09-30T22:19:19.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 -- 172.21.15.89:0/3301865224 learned_addr learned my addr 172.21.15.89:0/3301865224 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:19.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5af7fe700 1 --2- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b00719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 -- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 msgr2=0x7ff5b00719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 --2- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b00719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 -- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5b009d9f0 msgr2=0x7ff5b0070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 --2- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5b009d9f0 0x7ff5b0070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 -- 172.21.15.89:0/3301865224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a8009a70 con 0x7ff5b01316c0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5ae7fc700 1 --2- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b01316c0 0x7ff5b0071480 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.766+0000 7ff5977fe700 1 -- 172.21.15.89:0/3301865224 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4003c50 con 0x7ff5b01316c0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.767+0000 7ff5977fe700 1 -- 172.21.15.89:0/3301865224 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a401f070 con 0x7ff5b01316c0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.767+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a4009550 con 0x7ff5b01316c0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.767+0000 7ff5977fe700 1 -- 172.21.15.89:0/3301865224 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4011630 con 0x7ff5b01316c0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.767+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b0070af0 con 0x7ff5b01316c0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.768+0000 7ff5977fe700 1 -- 172.21.15.89:0/3301865224 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff5a4021030 con 0x7ff5b01316c0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.768+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b012d2b0 con 0x7ff5b01316c0 2023-09-30T22:19:19.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.768+0000 7ff5977fe700 1 -- 172.21.15.89:0/3301865224 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff5a4040ac0 con 0x7ff5b01316c0 2023-09-30T22:19:19.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.773+0000 7ff5977fe700 1 -- 172.21.15.89:0/3301865224 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff5a40239c0 con 0x7ff5b01316c0 2023-09-30T22:19:19.774 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:19.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.846+0000 7f4eb8d4e700 1 Processor -- start 2023-09-30T22:19:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.846+0000 7f4eb8d4e700 1 -- start start 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.846+0000 7f4eb8d4e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409d6f0 0x7f4eb409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.846+0000 7f4eb8d4e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4eb409e410 0x7f4eb4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.846+0000 7f4eb8d4e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb409b120 con 0x7f4eb409e000 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.846+0000 7f4eb8d4e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb406b460 con 0x7f4eb409e410 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.846+0000 7f4eb8d4e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb409a7e0 con 0x7f4eb409d6f0 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.846+0000 7f4eb2d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4eb409e410 0x7f4eb4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb2d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4eb409e410 0x7f4eb4134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46958/0 (socket says 172.21.15.89:46958) 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb2d9d700 1 -- 172.21.15.89:0/1334068898 learned_addr learned my addr 172.21.15.89:0/1334068898 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb259c700 1 --2- 172.21.15.89:0/1334068898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409d6f0 0x7f4eb409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb259c700 1 -- 172.21.15.89:0/1334068898 >> v2:172.21.15.89:3300/0 conn(0x7f4eb409e410 msgr2=0x7f4eb4134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb259c700 1 --2- 172.21.15.89:0/1334068898 >> v2:172.21.15.89:3300/0 conn(0x7f4eb409e410 0x7f4eb4134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb259c700 1 -- 172.21.15.89:0/1334068898 >> v1:172.21.15.89:6789/0 conn(0x7f4eb409e000 legacy=0x7f4eb41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb259c700 1 -- 172.21.15.89:0/1334068898 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb409a500 con 0x7f4eb409d6f0 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb259c700 1 --2- 172.21.15.89:0/1334068898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409d6f0 0x7f4eb409dac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009870 tx=0x7f4ea4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4713a4d19940e21 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb159a700 1 -- 172.21.15.89:0/1334068898 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4011830 con 0x7f4eb409d6f0 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb159a700 1 -- 172.21.15.89:0/1334068898 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea4011990 con 0x7f4eb409d6f0 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.847+0000 7f4eb159a700 1 -- 172.21.15.89:0/1334068898 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4011d20 con 0x7f4eb409d6f0 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.848+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1334068898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409d6f0 msgr2=0x7f4eb409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.848+0000 7f4eb8d4e700 1 --2- 172.21.15.89:0/1334068898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409d6f0 0x7f4eb409dac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009870 tx=0x7f4ea4009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:19.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.848+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1334068898 shutdown_connections 2023-09-30T22:19:19.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.848+0000 7f4eb8d4e700 1 --2- 172.21.15.89:0/1334068898 >> v2:172.21.15.89:3300/0 conn(0x7f4eb409e410 0x7f4eb4134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.848+0000 7f4eb8d4e700 1 --2- 172.21.15.89:0/1334068898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409d6f0 0x7f4eb409dac0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.848+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1334068898 >> 172.21.15.89:0/1334068898 conn(0x7f4eb4066f50 msgr2=0x7f4eb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:19.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.848+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1334068898 shutdown_connections 2023-09-30T22:19:19.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.848+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1334068898 wait complete. 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.849+0000 7f4eb8d4e700 1 Processor -- start 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.849+0000 7f4eb8d4e700 1 -- start start 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb8d4e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4eb409d6f0 0x7f4eb41a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb8d4e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409e000 0x7f4eb41a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb8d4e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb409e410 0x7f4eb41a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb8d4e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41a7ac0 con 0x7f4eb409e410 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb8d4e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41a8d10 con 0x7f4eb409d6f0 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb8d4e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41a80a0 con 0x7f4eb409e000 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409e000 0x7f4eb41a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409e000 0x7f4eb41a6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32790/0 (socket says 172.21.15.89:32790) 2023-09-30T22:19:19.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 -- 172.21.15.89:0/1606441469 learned_addr learned my addr 172.21.15.89:0/1606441469 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb2d9d700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb409e410 0x7f4eb41a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb259c700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4eb409d6f0 0x7f4eb41a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 -- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4eb409d6f0 msgr2=0x7f4eb41a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4eb409d6f0 0x7f4eb41a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 -- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb409e410 msgr2=0x7f4eb41a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb409e410 0x7f4eb41a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 -- 172.21.15.89:0/1606441469 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40094a0 con 0x7f4eb409e000 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.850+0000 7f4eb1d9b700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409e000 0x7f4eb41a6480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c4b0 tx=0x7f4ea800c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.851+0000 7f4ea2ffd700 1 -- 172.21.15.89:0/1606441469 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8011790 con 0x7f4eb409e000 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.851+0000 7f4ea2ffd700 1 -- 172.21.15.89:0/1606441469 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea80118f0 con 0x7f4eb409e000 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.851+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb41a96f0 con 0x7f4eb409e000 2023-09-30T22:19:19.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.851+0000 7f4ea2ffd700 1 -- 172.21.15.89:0/1606441469 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8011c60 con 0x7f4eb409e000 2023-09-30T22:19:19.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.851+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb41a91c0 con 0x7f4eb409e000 2023-09-30T22:19:19.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.852+0000 7f4ea2ffd700 1 -- 172.21.15.89:0/1606441469 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ea801c460 con 0x7f4eb409e000 2023-09-30T22:19:19.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.852+0000 7f4ea2ffd700 1 -- 172.21.15.89:0/1606441469 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4ea803f200 con 0x7f4eb409e000 2023-09-30T22:19:19.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.852+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb41a9160 con 0x7f4eb409e000 2023-09-30T22:19:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.861+0000 7f4ea2ffd700 1 -- 172.21.15.89:0/1606441469 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ea80102e0 con 0x7f4eb409e000 2023-09-30T22:19:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd8a1da3700 1 Processor -- start 2023-09-30T22:19:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd8a1da3700 1 -- start start 2023-09-30T22:19:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd8a1da3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd8a1da3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd89c139170 0x7fd89c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd8a1da3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c09b120 con 0x7fd89c09d6f0 2023-09-30T22:19:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd8a1da3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c06b460 con 0x7fd89c139170 2023-09-30T22:19:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd8a1da3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c09a7e0 con 0x7fd89c09e1d0 2023-09-30T22:19:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd89affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd89affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32798/0 (socket says 172.21.15.89:32798) 2023-09-30T22:19:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd89affd700 1 -- 172.21.15.89:0/2295784932 learned_addr learned my addr 172.21.15.89:0/2295784932 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.904+0000 7fd89bfff700 1 --2- 172.21.15.89:0/2295784932 >> v2:172.21.15.89:3300/0 conn(0x7fd89c139170 0x7fd89c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89affd700 1 -- 172.21.15.89:0/2295784932 >> v2:172.21.15.89:3300/0 conn(0x7fd89c139170 msgr2=0x7fd89c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89affd700 1 --2- 172.21.15.89:0/2295784932 >> v2:172.21.15.89:3300/0 conn(0x7fd89c139170 0x7fd89c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89affd700 1 -- 172.21.15.89:0/2295784932 >> v1:172.21.15.89:6789/0 conn(0x7fd89c09d6f0 legacy=0x7fd89c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89affd700 1 -- 172.21.15.89:0/2295784932 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c09a500 con 0x7fd89c09e1d0 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89bfff700 1 --2- 172.21.15.89:0/2295784932 >> v2:172.21.15.89:3300/0 conn(0x7fd89c139170 0x7fd89c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89affd700 1 --2- 172.21.15.89:0/2295784932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c138c30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd890009920 tx=0x7fd890009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf9b10bdbc9c399 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89a7fc700 1 -- 172.21.15.89:0/2295784932 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000ba20 con 0x7fd89c09e1d0 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89a7fc700 1 -- 172.21.15.89:0/2295784932 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89000bb80 con 0x7fd89c09e1d0 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.905+0000 7fd89a7fc700 1 -- 172.21.15.89:0/2295784932 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890009e70 con 0x7fd89c09e1d0 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.906+0000 7fd8a1da3700 1 -- 172.21.15.89:0/2295784932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 msgr2=0x7fd89c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.906+0000 7fd8a1da3700 1 --2- 172.21.15.89:0/2295784932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c138c30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd890009920 tx=0x7fd890009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.906+0000 7fd8a1da3700 1 -- 172.21.15.89:0/2295784932 shutdown_connections 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.906+0000 7fd8a1da3700 1 --2- 172.21.15.89:0/2295784932 >> v2:172.21.15.89:3300/0 conn(0x7fd89c139170 0x7fd89c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.906+0000 7fd8a1da3700 1 --2- 172.21.15.89:0/2295784932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c138c30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.906+0000 7fd8a1da3700 1 -- 172.21.15.89:0/2295784932 >> 172.21.15.89:0/2295784932 conn(0x7fd89c066f50 msgr2=0x7fd89c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.906+0000 7fd8a1da3700 1 -- 172.21.15.89:0/2295784932 shutdown_connections 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.906+0000 7fd8a1da3700 1 -- 172.21.15.89:0/2295784932 wait complete. 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.907+0000 7fd8a1da3700 1 Processor -- start 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.907+0000 7fd8a1da3700 1 -- start start 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.907+0000 7fd8a1da3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd89c09d6f0 0x7fd89c1c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.907+0000 7fd8a1da3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c1c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.907+0000 7fd8a1da3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd89c139170 0x7fd89c11e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.907+0000 7fd8a1da3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c1220a0 con 0x7fd89c09d6f0 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.907+0000 7fd8a1da3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c122280 con 0x7fd89c139170 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd8a1da3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c1be250 con 0x7fd89c09e1d0 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd89c09d6f0 0x7fd89c1c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c1c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd89c09d6f0 0x7fd89c1c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46992/0 (socket says 172.21.15.89:46992) 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c1c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32814/0 (socket says 172.21.15.89:32814) 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 -- 172.21.15.89:0/3125412157 learned_addr learned my addr 172.21.15.89:0/3125412157 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89bfff700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd89c139170 0x7fd89c11e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 -- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd89c139170 msgr2=0x7fd89c11e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd89c139170 0x7fd89c11e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 -- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd89c09d6f0 msgr2=0x7fd89c1c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd89c09d6f0 0x7fd89c1c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 -- 172.21.15.89:0/3125412157 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd890009550 con 0x7fd89c09e1d0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89bfff700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd89c139170 0x7fd89c11e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd89affd700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c1c8bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd890009870 tx=0x7fd890004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd883fff700 1 -- 172.21.15.89:0/3125412157 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890004300 con 0x7fd89c09e1d0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd883fff700 1 -- 172.21.15.89:0/3125412157 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89001f070 con 0x7fd89c09e1d0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd883fff700 1 -- 172.21.15.89:0/3125412157 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900036a0 con 0x7fd89c09e1d0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c0a29f0 con 0x7fd89c09e1d0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.908+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c09c710 con 0x7fd89c09e1d0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.909+0000 7fd883fff700 1 -- 172.21.15.89:0/3125412157 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd890021030 con 0x7fd89c09e1d0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.909+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c121ee0 con 0x7fd89c09e1d0 2023-09-30T22:19:19.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.910+0000 7fd883fff700 1 -- 172.21.15.89:0/3125412157 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd890040990 con 0x7fd89c09e1d0 2023-09-30T22:19:19.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.912+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb118133670 con 0x7fb118134170 2023-09-30T22:19:19.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.912+0000 7fb106ffd700 1 -- 172.21.15.89:0/1627089096 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb11001c3b0 con 0x7fb118134170 2023-09-30T22:19:19.916 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 msgr2=0x7fb118123230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 --2- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb118123230 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb11000efc0 tx=0x7fb11000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 shutdown_connections 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 --2- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb118134170 0x7fb118123230 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 --2- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1180a0a30 0x7fb1181227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 --2- 172.21.15.89:0/1627089096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180a1340 0x7fb118122cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 >> 172.21.15.89:0/1627089096 conn(0x7fb1180a6550 msgr2=0x7fb11809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 shutdown_connections 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fb11cf2c700 1 -- 172.21.15.89:0/1627089096 wait complete. 2023-09-30T22:19:19.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.914+0000 7fd883fff700 1 -- 172.21.15.89:0/3125412157 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd89c121ee0 con 0x7fd89c09e1d0 2023-09-30T22:19:19.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:19.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.970+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff5b009e130 con 0x7ff5b01316c0 2023-09-30T22:19:19.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.971+0000 7ff5977fe700 1 -- 172.21.15.89:0/3301865224 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff5b009e130 con 0x7ff5b01316c0 2023-09-30T22:19:19.971 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:19.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b01316c0 msgr2=0x7ff5b0071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:19.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 --2- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b01316c0 0x7ff5b0071480 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4004750 comp rx=0 tx=0).stop 2023-09-30T22:19:19.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 shutdown_connections 2023-09-30T22:19:19.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 --2- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5b0136a70 0x7ff5b00719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 --2- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff5b009d9f0 0x7ff5b0070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 --2- 172.21.15.89:0/3301865224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5b01316c0 0x7ff5b0071480 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:19.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 >> 172.21.15.89:0/3301865224 conn(0x7ff5b0066f50 msgr2=0x7ff5b01303d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:19.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 shutdown_connections 2023-09-30T22:19:19.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:19.973+0000 7ff5b57b4700 1 -- 172.21.15.89:0/3301865224 wait complete. 2023-09-30T22:19:19.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.058+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4eb4074ea0 con 0x7f4eb409e000 2023-09-30T22:19:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.058+0000 7f4ea2ffd700 1 -- 172.21.15.89:0/1606441469 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ea8019b00 con 0x7f4eb409e000 2023-09-30T22:19:20.059 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409e000 msgr2=0x7f4eb41a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409e000 0x7f4eb41a6480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c4b0 tx=0x7f4ea800c7c0 comp rx=0 tx=0).stop 2023-09-30T22:19:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 shutdown_connections 2023-09-30T22:19:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb409e410 0x7f4eb41a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4eb409d6f0 0x7f4eb41a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 --2- 172.21.15.89:0/1606441469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4eb409e000 0x7f4eb41a6480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 >> 172.21.15.89:0/1606441469 conn(0x7f4eb4066f50 msgr2=0x7f4eb41baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 shutdown_connections 2023-09-30T22:19:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.060+0000 7f4eb8d4e700 1 -- 172.21.15.89:0/1606441469 wait complete. 2023-09-30T22:19:20.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:20.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.107+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd89c121ee0 con 0x7fd89c09e1d0 2023-09-30T22:19:20.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.107+0000 7fd883fff700 1 -- 172.21.15.89:0/3125412157 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd89c121ee0 con 0x7fd89c09e1d0 2023-09-30T22:19:20.108 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:20.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 msgr2=0x7fd89c1c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:20.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c1c8bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd890009870 tx=0x7fd890004750 comp rx=0 tx=0).stop 2023-09-30T22:19:20.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 shutdown_connections 2023-09-30T22:19:20.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd89c139170 0x7fd89c11e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:20.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd89c09d6f0 0x7fd89c1c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:20.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 --2- 172.21.15.89:0/3125412157 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd89c09e1d0 0x7fd89c1c8bf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:20.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 >> 172.21.15.89:0/3125412157 conn(0x7fd89c066f50 msgr2=0x7fd89c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:20.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 shutdown_connections 2023-09-30T22:19:20.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:20.109+0000 7fd8a1da3700 1 -- 172.21.15.89:0/3125412157 wait complete. 2023-09-30T22:19:20.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:20.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:20.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:21.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.068+0000 7f9163e57700 1 Processor -- start 2023-09-30T22:19:21.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.068+0000 7f9163e57700 1 -- start start 2023-09-30T22:19:21.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9163e57700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f915c0a0a30 0x7f915c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9163e57700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a1340 0x7f915c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9163e57700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c133fa0 0x7f915c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9163e57700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915c09edd0 con 0x7f915c0a0a30 2023-09-30T22:19:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9163e57700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c09e1f0 con 0x7f915c133fa0 2023-09-30T22:19:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9163e57700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f915c09dc50 con 0x7f915c0a1340 2023-09-30T22:19:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9161bf3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f915c0a0a30 0x7f915c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f91613f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a1340 0x7f915c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9161bf3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f915c0a0a30 0x7f915c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40804/0 (socket says 172.21.15.89:40804) 2023-09-30T22:19:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f91623f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c133fa0 0x7f915c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9161bf3700 1 -- 172.21.15.89:0/907601990 learned_addr learned my addr 172.21.15.89:0/907601990 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f91623f4700 1 -- 172.21.15.89:0/907601990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a1340 msgr2=0x7f915c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f91623f4700 1 --2- 172.21.15.89:0/907601990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a1340 0x7f915c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f91623f4700 1 -- 172.21.15.89:0/907601990 >> v2:172.21.15.89:3300/0 conn(0x7f915c0a0a30 msgr2=0x7f915c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:21.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f91623f4700 1 --2- 172.21.15.89:0/907601990 >> v2:172.21.15.89:3300/0 conn(0x7f915c0a0a30 0x7f915c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f91623f4700 1 -- 172.21.15.89:0/907601990 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c0a1fe0 con 0x7f915c133fa0 2023-09-30T22:19:21.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9161bf3700 1 --2- 172.21.15.89:0/907601990 >> v2:172.21.15.89:3300/0 conn(0x7f915c0a0a30 0x7f915c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:21.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f91623f4700 1 --2- 172.21.15.89:0/907601990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c133fa0 0x7f915c134410 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9154009dc0 tx=0x7f915400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8419f81079747951 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:21.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9160bf1700 1 -- 172.21.15.89:0/907601990 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915400e070 con 0x7f915c133fa0 2023-09-30T22:19:21.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9160bf1700 1 -- 172.21.15.89:0/907601990 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915400bae0 con 0x7f915c133fa0 2023-09-30T22:19:21.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.069+0000 7f9160bf1700 1 -- 172.21.15.89:0/907601990 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915400be90 con 0x7f915c133fa0 2023-09-30T22:19:21.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.070+0000 7f9163e57700 1 -- 172.21.15.89:0/907601990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c133fa0 msgr2=0x7f915c134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.070+0000 7f9163e57700 1 --2- 172.21.15.89:0/907601990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c133fa0 0x7f915c134410 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9154009dc0 tx=0x7f915400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.070+0000 7f9163e57700 1 -- 172.21.15.89:0/907601990 shutdown_connections 2023-09-30T22:19:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.070+0000 7f9163e57700 1 --2- 172.21.15.89:0/907601990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c133fa0 0x7f915c134410 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.070+0000 7f9163e57700 1 --2- 172.21.15.89:0/907601990 >> v2:172.21.15.89:3300/0 conn(0x7f915c0a0a30 0x7f915c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.070+0000 7f9163e57700 1 --2- 172.21.15.89:0/907601990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a1340 0x7f915c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.070+0000 7f9163e57700 1 -- 172.21.15.89:0/907601990 >> 172.21.15.89:0/907601990 conn(0x7f915c0a6550 msgr2=0x7f915c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.071+0000 7f9163e57700 1 -- 172.21.15.89:0/907601990 shutdown_connections 2023-09-30T22:19:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.071+0000 7f9163e57700 1 -- 172.21.15.89:0/907601990 wait complete. 2023-09-30T22:19:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.072+0000 7f9163e57700 1 Processor -- start 2023-09-30T22:19:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.072+0000 7f9163e57700 1 -- start start 2023-09-30T22:19:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9163e57700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a0a30 0x7f915c120170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9163e57700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c0a1340 0x7f915c1206b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9163e57700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915c133fa0 0x7f915c120bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9163e57700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c121d30 con 0x7f915c133fa0 2023-09-30T22:19:21.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9163e57700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c122f80 con 0x7f915c0a1340 2023-09-30T22:19:21.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9163e57700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f915c122320 con 0x7f915c0a0a30 2023-09-30T22:19:21.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9161bf3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a0a30 0x7f915c120170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91613f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c0a1340 0x7f915c1206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9161bf3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a0a30 0x7f915c120170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46596/0 (socket says 172.21.15.89:46596) 2023-09-30T22:19:21.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9161bf3700 1 -- 172.21.15.89:0/1218896002 learned_addr learned my addr 172.21.15.89:0/1218896002 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:21.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91623f4700 1 --2- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915c133fa0 0x7f915c120bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91613f2700 1 -- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a0a30 msgr2=0x7f915c120170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91613f2700 1 --2- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a0a30 0x7f915c120170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91613f2700 1 -- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915c133fa0 msgr2=0x7f915c120bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91613f2700 1 --2- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915c133fa0 0x7f915c120bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91613f2700 1 -- 172.21.15.89:0/1218896002 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158009550 con 0x7f915c0a1340 2023-09-30T22:19:21.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91613f2700 1 --2- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c0a1340 0x7f915c1206b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f914c00e8a0 tx=0x7f914c00ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:21.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91527fc700 1 -- 172.21.15.89:0/1218896002 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c013680 con 0x7f915c0a1340 2023-09-30T22:19:21.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91527fc700 1 -- 172.21.15.89:0/1218896002 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c0137e0 con 0x7f915c0a1340 2023-09-30T22:19:21.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9154009a70 con 0x7f915c0a1340 2023-09-30T22:19:21.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f91527fc700 1 -- 172.21.15.89:0/1218896002 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c013b70 con 0x7f915c0a1340 2023-09-30T22:19:21.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.073+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c0a0220 con 0x7f915c0a1340 2023-09-30T22:19:21.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.074+0000 7f91527fc700 1 -- 172.21.15.89:0/1218896002 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f914c013cd0 con 0x7f915c0a1340 2023-09-30T22:19:21.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.074+0000 7f91527fc700 1 -- 172.21.15.89:0/1218896002 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f914c03f7b0 con 0x7f915c0a1340 2023-09-30T22:19:21.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.074+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c074ea0 con 0x7f915c0a1340 2023-09-30T22:19:21.083 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:21.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.079+0000 7f91527fc700 1 -- 172.21.15.89:0/1218896002 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f914c011ba0 con 0x7f915c0a1340 2023-09-30T22:19:21.123 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.128+0000 7f6927e9c700 1 Processor -- start 2023-09-30T22:19:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.128+0000 7f6927e9c700 1 -- start start 2023-09-30T22:19:21.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6927e9c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6920131810 0x7f6920131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6927e9c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f692009a580 0x7f6920139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6927e9c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69201313c0 con 0x7f6920099a40 2023-09-30T22:19:21.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6927e9c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692009b580 con 0x7f692009a580 2023-09-30T22:19:21.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6927e9c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692006e8c0 con 0x7f6920131810 2023-09-30T22:19:21.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6925c38700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6920131810 0x7f6920131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6926439700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f692009a580 0x7f6920139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6925c38700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6920131810 0x7f6920131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46146/0 (socket says 172.21.15.89:46146) 2023-09-30T22:19:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6926439700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f692009a580 0x7f6920139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40826/0 (socket says 172.21.15.89:40826) 2023-09-30T22:19:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6925c38700 1 -- 172.21.15.89:0/2008267511 learned_addr learned my addr 172.21.15.89:0/2008267511 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6925c38700 1 -- 172.21.15.89:0/2008267511 >> v2:172.21.15.89:3300/0 conn(0x7f692009a580 msgr2=0x7f6920139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6925c38700 1 --2- 172.21.15.89:0/2008267511 >> v2:172.21.15.89:3300/0 conn(0x7f692009a580 0x7f6920139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6924c36700 1 -- 172.21.15.89:0/2008267511 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854185530 0 0) 0x7f69201313c0 con 0x7f6920099a40 2023-09-30T22:19:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6925c38700 1 -- 172.21.15.89:0/2008267511 >> v1:172.21.15.89:6789/0 conn(0x7f6920099a40 legacy=0x7f6920099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.129+0000 7f6925c38700 1 -- 172.21.15.89:0/2008267511 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920071d20 con 0x7f6920131810 2023-09-30T22:19:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.130+0000 7f6925c38700 1 --2- 172.21.15.89:0/2008267511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6920131810 0x7f6920131be0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f691c009920 tx=0x7f691c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebbafaa0d284a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.130+0000 7f6924c36700 1 -- 172.21.15.89:0/2008267511 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c010a90 con 0x7f6920131810 2023-09-30T22:19:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.130+0000 7f6924c36700 1 -- 172.21.15.89:0/2008267511 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c010bf0 con 0x7f6920131810 2023-09-30T22:19:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.130+0000 7f6924c36700 1 -- 172.21.15.89:0/2008267511 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c018440 con 0x7f6920131810 2023-09-30T22:19:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.130+0000 7f6927e9c700 1 -- 172.21.15.89:0/2008267511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6920131810 msgr2=0x7f6920131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.130+0000 7f6927e9c700 1 --2- 172.21.15.89:0/2008267511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6920131810 0x7f6920131be0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f691c009920 tx=0x7f691c009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.130+0000 7f6927e9c700 1 -- 172.21.15.89:0/2008267511 shutdown_connections 2023-09-30T22:19:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.130+0000 7f6927e9c700 1 --2- 172.21.15.89:0/2008267511 >> v2:172.21.15.89:3300/0 conn(0x7f692009a580 0x7f6920139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.131+0000 7f6927e9c700 1 --2- 172.21.15.89:0/2008267511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6920131810 0x7f6920131be0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.131+0000 7f6927e9c700 1 -- 172.21.15.89:0/2008267511 >> 172.21.15.89:0/2008267511 conn(0x7f69200a2aa0 msgr2=0x7f69200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.131+0000 7f6927e9c700 1 -- 172.21.15.89:0/2008267511 shutdown_connections 2023-09-30T22:19:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.131+0000 7f6927e9c700 1 -- 172.21.15.89:0/2008267511 wait complete. 2023-09-30T22:19:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.131+0000 7f6927e9c700 1 Processor -- start 2023-09-30T22:19:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6927e9c700 1 -- start start 2023-09-30T22:19:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6927e9c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6920099a40 0x7f692011ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6927e9c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f692009a580 0x7f692011bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6927e9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6920131810 0x7f692011c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6927e9c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200a1d60 con 0x7f6920131810 2023-09-30T22:19:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6927e9c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692006eb00 con 0x7f692009a580 2023-09-30T22:19:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6927e9c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f692011ce20 con 0x7f6920099a40 2023-09-30T22:19:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6926439700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6920131810 0x7f692011c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6925c38700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6920099a40 0x7f692011ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6926439700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6920131810 0x7f692011c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40832/0 (socket says 172.21.15.89:40832) 2023-09-30T22:19:21.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6926439700 1 -- 172.21.15.89:0/888489541 learned_addr learned my addr 172.21.15.89:0/888489541 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:21.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6926439700 1 -- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6920099a40 msgr2=0x7f692011ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6926439700 1 --2- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6920099a40 0x7f692011ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6926439700 1 -- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f692009a580 msgr2=0x7f692011bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6926439700 1 --2- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f692009a580 0x7f692011bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6926439700 1 -- 172.21.15.89:0/888489541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c009550 con 0x7f6920131810 2023-09-30T22:19:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.132+0000 7f6925c38700 1 --2- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6920099a40 0x7f692011ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.133+0000 7f6926439700 1 --2- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6920131810 0x7f692011c4b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f691800efc0 tx=0x7f691800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.133+0000 7f69167fc700 1 -- 172.21.15.89:0/888489541 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800cd40 con 0x7f6920131810 2023-09-30T22:19:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.133+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6920072250 con 0x7f6920131810 2023-09-30T22:19:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.133+0000 7f69167fc700 1 -- 172.21.15.89:0/888489541 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691800f040 con 0x7f6920131810 2023-09-30T22:19:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.133+0000 7f69167fc700 1 -- 172.21.15.89:0/888489541 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180146c0 con 0x7f6920131810 2023-09-30T22:19:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.133+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692011f5a0 con 0x7f6920131810 2023-09-30T22:19:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.134+0000 7f69167fc700 1 -- 172.21.15.89:0/888489541 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6918014820 con 0x7f6920131810 2023-09-30T22:19:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.134+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692011ee70 con 0x7f6920131810 2023-09-30T22:19:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.134+0000 7f69167fc700 1 -- 172.21.15.89:0/888489541 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6918040400 con 0x7f6920131810 2023-09-30T22:19:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.143+0000 7f69167fc700 1 -- 172.21.15.89:0/888489541 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6918014e00 con 0x7f6920131810 2023-09-30T22:19:21.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.224+0000 7f8457d07700 1 Processor -- start 2023-09-30T22:19:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.224+0000 7f8457d07700 1 -- start start 2023-09-30T22:19:21.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f8457d07700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009d6f0 0x7f845009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f8457d07700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009e410 0x7f8450130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f8457d07700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845009f130 con 0x7f845009e000 2023-09-30T22:19:21.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f8457d07700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845009f520 con 0x7f845009e410 2023-09-30T22:19:21.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f8457d07700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f845009e930 con 0x7f845009d6f0 2023-09-30T22:19:21.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f84562a4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009e410 0x7f8450130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f8455aa3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009d6f0 0x7f845009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f84552a2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f845009e000 0x7f8450130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54998/0 (socket says 172.21.15.89:54998) 2023-09-30T22:19:21.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f84562a4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009e410 0x7f8450130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46160/0 (socket says 172.21.15.89:46160) 2023-09-30T22:19:21.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.225+0000 7f84552a2700 1 -- 172.21.15.89:0/150680151 learned_addr learned my addr 172.21.15.89:0/150680151 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:21.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8454aa1700 1 -- 172.21.15.89:0/150680151 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949032144 0 0) 0x7f845009f130 con 0x7f845009e000 2023-09-30T22:19:21.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8455aa3700 1 -- 172.21.15.89:0/150680151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009e410 msgr2=0x7f8450130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:21.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8455aa3700 1 --2- 172.21.15.89:0/150680151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009e410 0x7f8450130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8455aa3700 1 -- 172.21.15.89:0/150680151 >> v1:172.21.15.89:6789/0 conn(0x7f845009e000 legacy=0x7f8450130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8455aa3700 1 -- 172.21.15.89:0/150680151 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845009b2d0 con 0x7f845009d6f0 2023-09-30T22:19:21.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8455aa3700 1 --2- 172.21.15.89:0/150680151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009d6f0 0x7f845009dac0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f844c009870 tx=0x7f844c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1925af8b8df3f42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:21.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8454aa1700 1 -- 172.21.15.89:0/150680151 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c011830 con 0x7f845009d6f0 2023-09-30T22:19:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8454aa1700 1 -- 172.21.15.89:0/150680151 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c011990 con 0x7f845009d6f0 2023-09-30T22:19:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.226+0000 7f8454aa1700 1 -- 172.21.15.89:0/150680151 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c011d20 con 0x7f845009d6f0 2023-09-30T22:19:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.227+0000 7f8457d07700 1 -- 172.21.15.89:0/150680151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009d6f0 msgr2=0x7f845009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.227+0000 7f8457d07700 1 --2- 172.21.15.89:0/150680151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009d6f0 0x7f845009dac0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f844c009870 tx=0x7f844c009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.227+0000 7f8457d07700 1 -- 172.21.15.89:0/150680151 shutdown_connections 2023-09-30T22:19:21.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.227+0000 7f8457d07700 1 --2- 172.21.15.89:0/150680151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009e410 0x7f8450130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.227+0000 7f8457d07700 1 --2- 172.21.15.89:0/150680151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009d6f0 0x7f845009dac0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.227+0000 7f8457d07700 1 -- 172.21.15.89:0/150680151 >> 172.21.15.89:0/150680151 conn(0x7f8450066f50 msgr2=0x7f845009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:21.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.227+0000 7f8457d07700 1 -- 172.21.15.89:0/150680151 shutdown_connections 2023-09-30T22:19:21.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.227+0000 7f8457d07700 1 -- 172.21.15.89:0/150680151 wait complete. 2023-09-30T22:19:21.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.228+0000 7f8457d07700 1 Processor -- start 2023-09-30T22:19:21.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8457d07700 1 -- start start 2023-09-30T22:19:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8457d07700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009d6f0 0x7f84501222b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8457d07700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009e000 0x7f84501227f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8457d07700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f845009e410 0x7f8450122d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8457d07700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8450125ae0 con 0x7f845009e410 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8457d07700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8450125cc0 con 0x7f845009d6f0 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8457d07700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f845009c700 con 0x7f845009e000 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8455aa3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009d6f0 0x7f84501222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8455aa3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009d6f0 0x7f84501222b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46170/0 (socket says 172.21.15.89:46170) 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f8455aa3700 1 -- 172.21.15.89:0/3293253394 learned_addr learned my addr 172.21.15.89:0/3293253394 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.229+0000 7f84552a2700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009e000 0x7f84501227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f84562a4700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f845009e410 0x7f8450122d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f8455aa3700 1 -- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009e000 msgr2=0x7f84501227f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f8455aa3700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009e000 0x7f84501227f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f8455aa3700 1 -- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f845009e410 msgr2=0x7f8450122d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f8455aa3700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f845009e410 0x7f8450122d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f8455aa3700 1 -- 172.21.15.89:0/3293253394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c0094a0 con 0x7f845009d6f0 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f8455aa3700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009d6f0 0x7f84501222b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f844c0090b0 tx=0x7f844c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f84467fc700 1 -- 172.21.15.89:0/3293253394 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0036a0 con 0x7f845009d6f0 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f84467fc700 1 -- 172.21.15.89:0/3293253394 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c01f070 con 0x7f845009d6f0 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8450123ec0 con 0x7f845009d6f0 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f84467fc700 1 -- 172.21.15.89:0/3293253394 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c003a60 con 0x7f845009d6f0 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.230+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84501245a0 con 0x7f845009d6f0 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.231+0000 7f84467fc700 1 -- 172.21.15.89:0/3293253394 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f844c003bc0 con 0x7f845009d6f0 2023-09-30T22:19:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.231+0000 7f84467fc700 1 -- 172.21.15.89:0/3293253394 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f844c021070 con 0x7f845009d6f0 2023-09-30T22:19:21.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.232+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8450125900 con 0x7f845009d6f0 2023-09-30T22:19:21.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.240+0000 7f84467fc700 1 -- 172.21.15.89:0/3293253394 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f844c015020 con 0x7f845009d6f0 2023-09-30T22:19:21.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.273+0000 7fb6a7a6b700 1 Processor -- start 2023-09-30T22:19:21.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.273+0000 7fb6a7a6b700 1 -- start start 2023-09-30T22:19:21.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a7a6b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb6a009d6f0 0x7fb6a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a7a6b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009e410 0x7fb6a0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a7a6b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a009f130 con 0x7fb6a009e000 2023-09-30T22:19:21.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a7a6b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a009f520 con 0x7fb6a009d6f0 2023-09-30T22:19:21.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a7a6b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a009e930 con 0x7fb6a009e410 2023-09-30T22:19:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a6008700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009e410 0x7fb6a0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a5006700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb6a009e000 0x7fb6a0130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55002/0 (socket says 172.21.15.89:55002) 2023-09-30T22:19:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a5006700 1 -- 172.21.15.89:0/381162208 learned_addr learned my addr 172.21.15.89:0/381162208 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a5807700 1 --2- 172.21.15.89:0/381162208 >> v2:172.21.15.89:3300/0 conn(0x7fb6a009d6f0 0x7fb6a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a5807700 1 -- 172.21.15.89:0/381162208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009e410 msgr2=0x7fb6a0130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a5807700 1 --2- 172.21.15.89:0/381162208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009e410 0x7fb6a0130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a5807700 1 -- 172.21.15.89:0/381162208 >> v1:172.21.15.89:6789/0 conn(0x7fb6a009e000 legacy=0x7fb6a0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a4805700 1 -- 172.21.15.89:0/381162208 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098407106 0 0) 0x7fb6a009f130 con 0x7fb6a009e000 2023-09-30T22:19:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a5807700 1 -- 172.21.15.89:0/381162208 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a009b2d0 con 0x7fb6a009d6f0 2023-09-30T22:19:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.274+0000 7fb6a5807700 1 --2- 172.21.15.89:0/381162208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009d6f0 0x7fb6a009dac0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00dc60 tx=0x7fb69c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c90b72cef91ab05 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a4805700 1 -- 172.21.15.89:0/381162208 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c012070 con 0x7fb6a009d6f0 2023-09-30T22:19:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a4805700 1 -- 172.21.15.89:0/381162208 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c00d2e0 con 0x7fb6a009d6f0 2023-09-30T22:19:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a4805700 1 -- 172.21.15.89:0/381162208 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00f900 con 0x7fb6a009d6f0 2023-09-30T22:19:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/381162208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009d6f0 msgr2=0x7fb6a009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a7a6b700 1 --2- 172.21.15.89:0/381162208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009d6f0 0x7fb6a009dac0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00dc60 tx=0x7fb69c00df70 comp rx=0 tx=0).stop 2023-09-30T22:19:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/381162208 shutdown_connections 2023-09-30T22:19:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a7a6b700 1 --2- 172.21.15.89:0/381162208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009e410 0x7fb6a0130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a7a6b700 1 --2- 172.21.15.89:0/381162208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009d6f0 0x7fb6a009dac0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/381162208 >> 172.21.15.89:0/381162208 conn(0x7fb6a0066f50 msgr2=0x7fb6a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/381162208 shutdown_connections 2023-09-30T22:19:21.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.275+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/381162208 wait complete. 2023-09-30T22:19:21.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a7a6b700 1 Processor -- start 2023-09-30T22:19:21.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a7a6b700 1 -- start start 2023-09-30T22:19:21.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a7a6b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009d6f0 0x7fb6a01d5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a7a6b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6a009e000 0x7fb6a01d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a7a6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009e410 0x7fb6a01d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a7a6b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a0130600 con 0x7fb6a009e410 2023-09-30T22:19:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a7a6b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00a3c20 con 0x7fb6a009d6f0 2023-09-30T22:19:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a7a6b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a009c700 con 0x7fb6a009e000 2023-09-30T22:19:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a5807700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009d6f0 0x7fb6a01d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a5807700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009d6f0 0x7fb6a01d5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46200/0 (socket says 172.21.15.89:46200) 2023-09-30T22:19:21.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a5807700 1 -- 172.21.15.89:0/1647501127 learned_addr learned my addr 172.21.15.89:0/1647501127 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a5006700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6a009e000 0x7fb6a01d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.277+0000 7fb6a6008700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009e410 0x7fb6a01d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6a5807700 1 -- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6a009e000 msgr2=0x7fb6a01d56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6a5807700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6a009e000 0x7fb6a01d56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6a5807700 1 -- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009e410 msgr2=0x7fb6a01d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6a5807700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009e410 0x7fb6a01d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6a5807700 1 -- 172.21.15.89:0/1647501127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c00d940 con 0x7fb6a009d6f0 2023-09-30T22:19:21.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6a5807700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009d6f0 0x7fb6a01d5190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb69c000c00 tx=0x7fb69c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:21.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6967fc700 1 -- 172.21.15.89:0/1647501127 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00b070 con 0x7fb6a009d6f0 2023-09-30T22:19:21.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a01b7500 con 0x7fb6a009d6f0 2023-09-30T22:19:21.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a01b8d60 con 0x7fb6a009d6f0 2023-09-30T22:19:21.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6967fc700 1 -- 172.21.15.89:0/1647501127 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c017980 con 0x7fb6a009d6f0 2023-09-30T22:19:21.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.278+0000 7fb6967fc700 1 -- 172.21.15.89:0/1647501127 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c017cf0 con 0x7fb6a009d6f0 2023-09-30T22:19:21.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.279+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6880052e0 con 0x7fb6a009d6f0 2023-09-30T22:19:21.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.280+0000 7fb6967fc700 1 -- 172.21.15.89:0/1647501127 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb69c01f450 con 0x7fb6a009d6f0 2023-09-30T22:19:21.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.280+0000 7fb6967fc700 1 -- 172.21.15.89:0/1647501127 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb69c040b40 con 0x7fb6a009d6f0 2023-09-30T22:19:21.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.288+0000 7fb6967fc700 1 -- 172.21.15.89:0/1647501127 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb69c017e50 con 0x7fb6a009d6f0 2023-09-30T22:19:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.293+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f915c0a28e0 con 0x7f915c0a1340 2023-09-30T22:19:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.293+0000 7f91527fc700 1 -- 172.21.15.89:0/1218896002 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f914c021030 con 0x7f915c0a1340 2023-09-30T22:19:21.293 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.296+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c0a1340 msgr2=0x7f915c1206b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.296+0000 7f9163e57700 1 --2- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c0a1340 0x7f915c1206b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f914c00e8a0 tx=0x7f914c00ebb0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.296+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 shutdown_connections 2023-09-30T22:19:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.296+0000 7f9163e57700 1 --2- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f915c133fa0 0x7f915c120bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.296+0000 7f9163e57700 1 --2- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f915c0a0a30 0x7f915c120170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.296+0000 7f9163e57700 1 --2- 172.21.15.89:0/1218896002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f915c0a1340 0x7f915c1206b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.296+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 >> 172.21.15.89:0/1218896002 conn(0x7f915c0a6550 msgr2=0x7f915c1bb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.297+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 shutdown_connections 2023-09-30T22:19:21.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.297+0000 7f9163e57700 1 -- 172.21.15.89:0/1218896002 wait complete. 2023-09-30T22:19:21.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.351+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6920074ea0 con 0x7f6920131810 2023-09-30T22:19:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.351+0000 7f69167fc700 1 -- 172.21.15.89:0/888489541 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6918018030 con 0x7f6920131810 2023-09-30T22:19:21.351 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:21.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6920131810 msgr2=0x7f692011c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 --2- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6920131810 0x7f692011c4b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f691800efc0 tx=0x7f691800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 shutdown_connections 2023-09-30T22:19:21.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 --2- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6920131810 0x7f692011c4b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 --2- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6920099a40 0x7f692011ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 --2- 172.21.15.89:0/888489541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f692009a580 0x7f692011bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 >> 172.21.15.89:0/888489541 conn(0x7f69200a2aa0 msgr2=0x7f692009d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:21.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 shutdown_connections 2023-09-30T22:19:21.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.353+0000 7f6927e9c700 1 -- 172.21.15.89:0/888489541 wait complete. 2023-09-30T22:19:21.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:21.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.449+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8450072080 con 0x7f845009d6f0 2023-09-30T22:19:21.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.449+0000 7f84467fc700 1 -- 172.21.15.89:0/3293253394 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f844c010380 con 0x7f845009d6f0 2023-09-30T22:19:21.449 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:21.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.451+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009d6f0 msgr2=0x7f84501222b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.451+0000 7f8457d07700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009d6f0 0x7f84501222b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f844c0090b0 tx=0x7f844c004750 comp rx=0 tx=0).stop 2023-09-30T22:19:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.451+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 shutdown_connections 2023-09-30T22:19:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.451+0000 7f8457d07700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f845009e410 0x7f8450122d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.451+0000 7f8457d07700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f845009d6f0 0x7f84501222b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.451+0000 7f8457d07700 1 --2- 172.21.15.89:0/3293253394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f845009e000 0x7f84501227f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.452+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 >> 172.21.15.89:0/3293253394 conn(0x7f8450066f50 msgr2=0x7f8450130a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.452+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 shutdown_connections 2023-09-30T22:19:21.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.452+0000 7f8457d07700 1 -- 172.21.15.89:0/3293253394 wait complete. 2023-09-30T22:19:21.463 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:21.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.487+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb688005150 con 0x7fb6a009d6f0 2023-09-30T22:19:21.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.488+0000 7fb6967fc700 1 -- 172.21.15.89:0/1647501127 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb69c015390 con 0x7fb6a009d6f0 2023-09-30T22:19:21.487 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.489+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009d6f0 msgr2=0x7fb6a01d5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.489+0000 7fb6a7a6b700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009d6f0 0x7fb6a01d5190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb69c000c00 tx=0x7fb69c004750 comp rx=0 tx=0).stop 2023-09-30T22:19:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.490+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 shutdown_connections 2023-09-30T22:19:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.490+0000 7fb6a7a6b700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6a009e410 0x7fb6a01d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.490+0000 7fb6a7a6b700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6a009d6f0 0x7fb6a01d5190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.490+0000 7fb6a7a6b700 1 --2- 172.21.15.89:0/1647501127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6a009e000 0x7fb6a01d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.490+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 >> 172.21.15.89:0/1647501127 conn(0x7fb6a0066f50 msgr2=0x7fb6a00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.490+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 shutdown_connections 2023-09-30T22:19:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:21.490+0000 7fb6a7a6b700 1 -- 172.21.15.89:0/1647501127 wait complete. 2023-09-30T22:19:21.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:22.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:22.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.451+0000 7fa943e83700 1 Processor -- start 2023-09-30T22:19:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.451+0000 7fa943e83700 1 -- start start 2023-09-30T22:19:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa943e83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c09d630 0x7fa93c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa943e83700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c138d80 0x7fa93c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa943e83700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93c0a2a70 con 0x7fa93c0a0a30 2023-09-30T22:19:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa943e83700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0a01d0 con 0x7fa93c138d80 2023-09-30T22:19:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa943e83700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c09e2b0 con 0x7fa93c09d630 2023-09-30T22:19:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa94141e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c09d630 0x7fa93c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa941c1f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa93c0a0a30 0x7fa93c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55012/0 (socket says 172.21.15.89:55012) 2023-09-30T22:19:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa94141e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c09d630 0x7fa93c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46648/0 (socket says 172.21.15.89:46648) 2023-09-30T22:19:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa941c1f700 1 -- 172.21.15.89:0/1764156498 learned_addr learned my addr 172.21.15.89:0/1764156498 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa942420700 1 --2- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c138d80 0x7fa93c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa94141e700 1 -- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c138d80 msgr2=0x7fa93c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa94141e700 1 --2- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c138d80 0x7fa93c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa94141e700 1 -- 172.21.15.89:0/1764156498 >> v1:172.21.15.89:6789/0 conn(0x7fa93c0a0a30 legacy=0x7fa93c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa940c1d700 1 -- 172.21.15.89:0/1764156498 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909717289 0 0) 0x7fa93c0a2a70 con 0x7fa93c0a0a30 2023-09-30T22:19:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa94141e700 1 -- 172.21.15.89:0/1764156498 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c09dc50 con 0x7fa93c09d630 2023-09-30T22:19:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.452+0000 7fa942420700 1 --2- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c138d80 0x7fa93c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa94141e700 1 --2- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c09d630 0x7fa93c0a1510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009870 tx=0x7fa92c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e7da7983d52305 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa940c1d700 1 -- 172.21.15.89:0/1764156498 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c011830 con 0x7fa93c09d630 2023-09-30T22:19:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa940c1d700 1 -- 172.21.15.89:0/1764156498 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c011990 con 0x7fa93c09d630 2023-09-30T22:19:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa940c1d700 1 -- 172.21.15.89:0/1764156498 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c011d20 con 0x7fa93c09d630 2023-09-30T22:19:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa943e83700 1 -- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c09d630 msgr2=0x7fa93c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa943e83700 1 --2- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c09d630 0x7fa93c0a1510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009870 tx=0x7fa92c009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa943e83700 1 -- 172.21.15.89:0/1764156498 shutdown_connections 2023-09-30T22:19:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa943e83700 1 --2- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c138d80 0x7fa93c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa943e83700 1 --2- 172.21.15.89:0/1764156498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c09d630 0x7fa93c0a1510 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa943e83700 1 -- 172.21.15.89:0/1764156498 >> 172.21.15.89:0/1764156498 conn(0x7fa93c0a3050 msgr2=0x7fa93c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa943e83700 1 -- 172.21.15.89:0/1764156498 shutdown_connections 2023-09-30T22:19:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.453+0000 7fa943e83700 1 -- 172.21.15.89:0/1764156498 wait complete. 2023-09-30T22:19:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.454+0000 7fa943e83700 1 Processor -- start 2023-09-30T22:19:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.454+0000 7fa943e83700 1 -- start start 2023-09-30T22:19:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa943e83700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa93c09d630 0x7fa93c1337c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa943e83700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c0a0a30 0x7fa93c133d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa943e83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c138d80 0x7fa93c134240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa943e83700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c134c90 con 0x7fa93c09d630 2023-09-30T22:19:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa943e83700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c137310 con 0x7fa93c0a0a30 2023-09-30T22:19:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa943e83700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c136c60 con 0x7fa93c138d80 2023-09-30T22:19:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa93c09d630 0x7fa93c1337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa93c09d630 0x7fa93c1337c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40864/0 (socket says 172.21.15.89:40864) 2023-09-30T22:19:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 -- 172.21.15.89:0/2726058258 learned_addr learned my addr 172.21.15.89:0/2726058258 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa94141e700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c0a0a30 0x7fa93c133d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa942420700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c138d80 0x7fa93c134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 -- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c138d80 msgr2=0x7fa93c134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c138d80 0x7fa93c134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 -- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c0a0a30 msgr2=0x7fa93c133d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c0a0a30 0x7fa93c133d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 -- 172.21.15.89:0/2726058258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa934009a70 con 0x7fa93c09d630 2023-09-30T22:19:22.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa94141e700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c0a0a30 0x7fa93c133d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:22.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa941c1f700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa93c09d630 0x7fa93c1337c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c550 tx=0x7fa93800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:22.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa9327fc700 1 -- 172.21.15.89:0/2726058258 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938004c80 con 0x7fa93c09d630 2023-09-30T22:19:22.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa9327fc700 1 -- 172.21.15.89:0/2726058258 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938004e00 con 0x7fa93c09d630 2023-09-30T22:19:22.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.455+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c0094a0 con 0x7fa93c09d630 2023-09-30T22:19:22.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.456+0000 7fa9327fc700 1 -- 172.21.15.89:0/2726058258 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938007310 con 0x7fa93c09d630 2023-09-30T22:19:22.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.456+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c136f20 con 0x7fa93c09d630 2023-09-30T22:19:22.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.456+0000 7fa9327fc700 1 -- 172.21.15.89:0/2726058258 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa938007470 con 0x7fa93c09d630 2023-09-30T22:19:22.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.456+0000 7fa9327fc700 1 -- 172.21.15.89:0/2726058258 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa93803f2d0 con 0x7fa93c09d630 2023-09-30T22:19:22.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.456+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93c0747f0 con 0x7fa93c09d630 2023-09-30T22:19:22.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:22.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.461+0000 7fa9327fc700 1 -- 172.21.15.89:0/2726058258 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9380123b0 con 0x7fa93c09d630 2023-09-30T22:19:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.501+0000 7f518ed5f700 1 Processor -- start 2023-09-30T22:19:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.501+0000 7f518ed5f700 1 -- start start 2023-09-30T22:19:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.501+0000 7f518ed5f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5188136710 0x7f5188138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.501+0000 7f518ed5f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5188139030 0x7f518813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.501+0000 7f518ed5f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518806ef30 con 0x7f51881338a0 2023-09-30T22:19:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f518ed5f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518806fef0 con 0x7f5188136710 2023-09-30T22:19:22.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f518ed5f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51881334b0 con 0x7f5188139030 2023-09-30T22:19:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f5187fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5188136710 0x7f5188138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f518d2fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5188139030 0x7f518813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f5187fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5188136710 0x7f5188138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40878/0 (socket says 172.21.15.89:40878) 2023-09-30T22:19:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f518d2fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5188139030 0x7f518813b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46224/0 (socket says 172.21.15.89:46224) 2023-09-30T22:19:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f5187fff700 1 -- 172.21.15.89:0/2832682620 learned_addr learned my addr 172.21.15.89:0/2832682620 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f5187fff700 1 -- 172.21.15.89:0/2832682620 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5188139030 msgr2=0x7f518813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f5187fff700 1 --2- 172.21.15.89:0/2832682620 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5188139030 0x7f518813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f5187fff700 1 -- 172.21.15.89:0/2832682620 >> v1:172.21.15.89:6789/0 conn(0x7f51881338a0 legacy=0x7f5188133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f5187fff700 1 -- 172.21.15.89:0/2832682620 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518806fc30 con 0x7f5188136710 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f518d2fc700 1 --2- 172.21.15.89:0/2832682620 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5188139030 0x7f518813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f5187fff700 1 --2- 172.21.15.89:0/2832682620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188136710 0x7f5188138af0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f51740098f0 tx=0x7f5174009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519792fb260847d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f51877fe700 1 -- 172.21.15.89:0/2832682620 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174011830 con 0x7f5188136710 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f51877fe700 1 -- 172.21.15.89:0/2832682620 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174011990 con 0x7f5188136710 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.502+0000 7f51877fe700 1 -- 172.21.15.89:0/2832682620 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174011d20 con 0x7f5188136710 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.503+0000 7f518ed5f700 1 -- 172.21.15.89:0/2832682620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188136710 msgr2=0x7f5188138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.503+0000 7f518ed5f700 1 --2- 172.21.15.89:0/2832682620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188136710 0x7f5188138af0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f51740098f0 tx=0x7f5174009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.503+0000 7f518ed5f700 1 -- 172.21.15.89:0/2832682620 shutdown_connections 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.503+0000 7f518ed5f700 1 --2- 172.21.15.89:0/2832682620 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5188139030 0x7f518813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.503+0000 7f518ed5f700 1 --2- 172.21.15.89:0/2832682620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188136710 0x7f5188138af0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.503+0000 7f518ed5f700 1 -- 172.21.15.89:0/2832682620 >> 172.21.15.89:0/2832682620 conn(0x7f51880a2aa0 msgr2=0x7f51880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.503+0000 7f518ed5f700 1 -- 172.21.15.89:0/2832682620 shutdown_connections 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.503+0000 7f518ed5f700 1 -- 172.21.15.89:0/2832682620 wait complete. 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.504+0000 7f518ed5f700 1 Processor -- start 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.504+0000 7f518ed5f700 1 -- start start 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518ed5f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51881338a0 0x7f518811ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518ed5f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5188136710 0x7f518811b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518ed5f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188139030 0x7f518811b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518ed5f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518811cad0 con 0x7f5188139030 2023-09-30T22:19:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518ed5f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518811e330 con 0x7f51881338a0 2023-09-30T22:19:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518ed5f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5188138b00 con 0x7f5188136710 2023-09-30T22:19:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518cafb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51881338a0 0x7f518811ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518d2fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188139030 0x7f518811b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518cafb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51881338a0 0x7f518811ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46226/0 (socket says 172.21.15.89:46226) 2023-09-30T22:19:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518cafb700 1 -- 172.21.15.89:0/2588399952 learned_addr learned my addr 172.21.15.89:0/2588399952 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518d2fc700 1 -- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5188136710 msgr2=0x7f518811b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518d2fc700 1 --2- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5188136710 0x7f518811b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518d2fc700 1 -- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51881338a0 msgr2=0x7f518811ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518d2fc700 1 --2- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51881338a0 0x7f518811ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518d2fc700 1 -- 172.21.15.89:0/2588399952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5180009a70 con 0x7f5188139030 2023-09-30T22:19:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518cafb700 1 --2- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51881338a0 0x7f518811ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518d2fc700 1 --2- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188139030 0x7f518811b8d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f518000efc0 tx=0x7f518000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f51857fa700 1 -- 172.21.15.89:0/2588399952 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518000cb40 con 0x7f5188139030 2023-09-30T22:19:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5174009550 con 0x7f5188139030 2023-09-30T22:19:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f51857fa700 1 -- 172.21.15.89:0/2588399952 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518000f040 con 0x7f5188139030 2023-09-30T22:19:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f51857fa700 1 -- 172.21.15.89:0/2588399952 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180003a20 con 0x7f5188139030 2023-09-30T22:19:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.505+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518811f060 con 0x7f5188139030 2023-09-30T22:19:22.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.506+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5188074ea0 con 0x7f5188139030 2023-09-30T22:19:22.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.507+0000 7f51857fa700 1 -- 172.21.15.89:0/2588399952 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5180014480 con 0x7f5188139030 2023-09-30T22:19:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.507+0000 7f51857fa700 1 -- 172.21.15.89:0/2588399952 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f51800406a0 con 0x7f5188139030 2023-09-30T22:19:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.515+0000 7f51857fa700 1 -- 172.21.15.89:0/2588399952 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f518001b930 con 0x7f5188139030 2023-09-30T22:19:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.600+0000 7f95799ae700 1 Processor -- start 2023-09-30T22:19:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.600+0000 7f95799ae700 1 -- start start 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.600+0000 7f95799ae700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f957412cf60 0x7f957406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.600+0000 7f95799ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066410 0x7f957412f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.600+0000 7f95799ae700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957412cd90 con 0x7f9574066880 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.600+0000 7f95799ae700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957412d330 con 0x7f957412cf60 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.600+0000 7f95799ae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9574071780 con 0x7f9574066410 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f95727fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066410 0x7f957412f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f9572ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f957412cf60 0x7f957406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f95727fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066410 0x7f957412f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46684/0 (socket says 172.21.15.89:46684) 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f95727fc700 1 -- 172.21.15.89:0/2230628194 learned_addr learned my addr 172.21.15.89:0/2230628194 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f95727fc700 1 -- 172.21.15.89:0/2230628194 >> v2:172.21.15.89:3300/0 conn(0x7f957412cf60 msgr2=0x7f957406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f95727fc700 1 --2- 172.21.15.89:0/2230628194 >> v2:172.21.15.89:3300/0 conn(0x7f957412cf60 0x7f957406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f95727fc700 1 -- 172.21.15.89:0/2230628194 >> v1:172.21.15.89:6789/0 conn(0x7f9574066880 legacy=0x7f957412fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f95727fc700 1 -- 172.21.15.89:0/2230628194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9574071a40 con 0x7f9574066410 2023-09-30T22:19:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f95727fc700 1 --2- 172.21.15.89:0/2230628194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066410 0x7f957412f8b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f95680098f0 tx=0x7f9568009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03922180cf9db12 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f9571ffb700 1 -- 172.21.15.89:0/2230628194 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568011830 con 0x7f9574066410 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f9571ffb700 1 -- 172.21.15.89:0/2230628194 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9568011990 con 0x7f9574066410 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.601+0000 7f9571ffb700 1 -- 172.21.15.89:0/2230628194 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568011d00 con 0x7f9574066410 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.602+0000 7f95799ae700 1 -- 172.21.15.89:0/2230628194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066410 msgr2=0x7f957412f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.602+0000 7f95799ae700 1 --2- 172.21.15.89:0/2230628194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066410 0x7f957412f8b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f95680098f0 tx=0x7f9568009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.602+0000 7f95799ae700 1 -- 172.21.15.89:0/2230628194 shutdown_connections 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.602+0000 7f95799ae700 1 --2- 172.21.15.89:0/2230628194 >> v2:172.21.15.89:3300/0 conn(0x7f957412cf60 0x7f957406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.602+0000 7f95799ae700 1 --2- 172.21.15.89:0/2230628194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066410 0x7f957412f8b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.602+0000 7f95799ae700 1 -- 172.21.15.89:0/2230628194 >> 172.21.15.89:0/2230628194 conn(0x7f95740a2aa0 msgr2=0x7f95740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.602+0000 7f95799ae700 1 -- 172.21.15.89:0/2230628194 shutdown_connections 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.602+0000 7f95799ae700 1 -- 172.21.15.89:0/2230628194 wait complete. 2023-09-30T22:19:22.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.603+0000 7f95799ae700 1 Processor -- start 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.603+0000 7f95799ae700 1 -- start start 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95799ae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9574066410 0x7f9574130500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95799ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066880 0x7f9574130a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95799ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957412cf60 0x7f95741c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95799ae700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95741327f0 con 0x7f957412cf60 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95799ae700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9574130f80 con 0x7f9574066410 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95799ae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95741300f0 con 0x7f9574066880 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957412cf60 0x7f95741c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957412cf60 0x7f95741c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40906/0 (socket says 172.21.15.89:40906) 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 -- 172.21.15.89:0/3631530472 learned_addr learned my addr 172.21.15.89:0/3631530472 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95727fc700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066880 0x7f9574130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f9572ffd700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9574066410 0x7f9574130500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 -- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066880 msgr2=0x7f9574130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066880 0x7f9574130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 -- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9574066410 msgr2=0x7f9574130500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9574066410 0x7f9574130500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 -- 172.21.15.89:0/3631530472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9568009550 con 0x7f957412cf60 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95727fc700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066880 0x7f9574130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95737fe700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957412cf60 0x7f95741c8430 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f956c00cbc0 tx=0x7f956c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f955bfff700 1 -- 172.21.15.89:0/3631530472 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c003c00 con 0x7f957412cf60 2023-09-30T22:19:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f955bfff700 1 -- 172.21.15.89:0/3631530472 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c003d60 con 0x7f957412cf60 2023-09-30T22:19:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f955bfff700 1 -- 172.21.15.89:0/3631530472 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c00ab00 con 0x7f957412cf60 2023-09-30T22:19:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.604+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957412c080 con 0x7f957412cf60 2023-09-30T22:19:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.605+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9574133380 con 0x7f957412cf60 2023-09-30T22:19:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.606+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9574133020 con 0x7f957412cf60 2023-09-30T22:19:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.606+0000 7f955bfff700 1 -- 172.21.15.89:0/3631530472 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f956c00ac60 con 0x7f957412cf60 2023-09-30T22:19:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.606+0000 7f955bfff700 1 -- 172.21.15.89:0/3631530472 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f956c03f700 con 0x7f957412cf60 2023-09-30T22:19:22.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.615+0000 7f955bfff700 1 -- 172.21.15.89:0/3631530472 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9574133020 con 0x7f957412cf60 2023-09-30T22:19:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.644+0000 7fc64ec27700 1 Processor -- start 2023-09-30T22:19:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.644+0000 7fc64ec27700 1 -- start start 2023-09-30T22:19:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.644+0000 7fc64ec27700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6481338a0 0x7fc648133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.644+0000 7fc64ec27700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.644+0000 7fc64ec27700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64806ef30 con 0x7fc64809de50 2023-09-30T22:19:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.644+0000 7fc64ec27700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64806fef0 con 0x7fc64809e930 2023-09-30T22:19:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.644+0000 7fc64ec27700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6481334b0 con 0x7fc6481338a0 2023-09-30T22:19:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc647fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc64809de50 0x7fc64809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55048/0 (socket says 172.21.15.89:55048) 2023-09-30T22:19:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc647fff700 1 -- 172.21.15.89:0/1286890095 learned_addr learned my addr 172.21.15.89:0/1286890095 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc64d1c4700 1 --2- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc64c9c3700 1 --2- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6481338a0 0x7fc648133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc6477fe700 1 -- 172.21.15.89:0/1286890095 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182192451 0 0) 0x7fc64806ef30 con 0x7fc64809de50 2023-09-30T22:19:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc6477fe700 1 -- 172.21.15.89:0/1286890095 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6380035b0 con 0x7fc64809de50 2023-09-30T22:19:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc64d1c4700 1 -- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6481338a0 msgr2=0x7fc648133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc64d1c4700 1 --2- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6481338a0 0x7fc648133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc64d1c4700 1 -- 172.21.15.89:0/1286890095 >> v1:172.21.15.89:6789/0 conn(0x7fc64809de50 legacy=0x7fc64809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc64d1c4700 1 -- 172.21.15.89:0/1286890095 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64806fc30 con 0x7fc64809e930 2023-09-30T22:19:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.645+0000 7fc64d1c4700 1 --2- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64813b740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc640009dc0 tx=0x7fc64000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d9e0f6771c9ab4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc6477fe700 1 -- 172.21.15.89:0/1286890095 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000e070 con 0x7fc64809e930 2023-09-30T22:19:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc6477fe700 1 -- 172.21.15.89:0/1286890095 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64000bb20 con 0x7fc64809e930 2023-09-30T22:19:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc6477fe700 1 -- 172.21.15.89:0/1286890095 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640012430 con 0x7fc64809e930 2023-09-30T22:19:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc64ec27700 1 -- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 msgr2=0x7fc64813b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc64ec27700 1 --2- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64813b740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc640009dc0 tx=0x7fc64000b330 comp rx=0 tx=0).stop 2023-09-30T22:19:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc64ec27700 1 -- 172.21.15.89:0/1286890095 shutdown_connections 2023-09-30T22:19:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc64ec27700 1 --2- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64813b740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc64ec27700 1 --2- 172.21.15.89:0/1286890095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6481338a0 0x7fc648133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc64ec27700 1 -- 172.21.15.89:0/1286890095 >> 172.21.15.89:0/1286890095 conn(0x7fc6480a2aa0 msgr2=0x7fc6480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc64ec27700 1 -- 172.21.15.89:0/1286890095 shutdown_connections 2023-09-30T22:19:22.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.646+0000 7fc64ec27700 1 -- 172.21.15.89:0/1286890095 wait complete. 2023-09-30T22:19:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.647+0000 7fc64ec27700 1 Processor -- start 2023-09-30T22:19:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.647+0000 7fc64ec27700 1 -- start start 2023-09-30T22:19:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64ec27700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc64809de50 0x7fc64811a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64ec27700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64811ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64ec27700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6481338a0 0x7fc64811b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64ec27700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64811e0e0 con 0x7fc6481338a0 2023-09-30T22:19:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64ec27700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64811d4b0 con 0x7fc64809e930 2023-09-30T22:19:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64ec27700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480a0b50 con 0x7fc64809de50 2023-09-30T22:19:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc64809de50 0x7fc64811a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc64809de50 0x7fc64811a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46710/0 (socket says 172.21.15.89:46710) 2023-09-30T22:19:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 -- 172.21.15.89:0/2852803151 learned_addr learned my addr 172.21.15.89:0/2852803151 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64d1c4700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6481338a0 0x7fc64811b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc647fff700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64811ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 -- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 msgr2=0x7fc64811ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64811ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 -- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6481338a0 msgr2=0x7fc64811b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6481338a0 0x7fc64811b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 -- 172.21.15.89:0/2852803151 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640009a70 con 0x7fc64809de50 2023-09-30T22:19:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64d1c4700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6481338a0 0x7fc64811b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.648+0000 7fc64c9c3700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc64809de50 0x7fc64811a5c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00e9b0 tx=0x7fc63c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.649+0000 7fc6457fa700 1 -- 172.21.15.89:0/2852803151 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c009710 con 0x7fc64809de50 2023-09-30T22:19:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.649+0000 7fc6457fa700 1 -- 172.21.15.89:0/2852803151 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c00c850 con 0x7fc64809de50 2023-09-30T22:19:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.649+0000 7fc6457fa700 1 -- 172.21.15.89:0/2852803151 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00cbc0 con 0x7fc64809de50 2023-09-30T22:19:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.649+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64806f640 con 0x7fc64809de50 2023-09-30T22:19:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.649+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64811d870 con 0x7fc64809de50 2023-09-30T22:19:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.650+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc648074ea0 con 0x7fc64809de50 2023-09-30T22:19:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.650+0000 7fc6457fa700 1 -- 172.21.15.89:0/2852803151 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc63c00cd20 con 0x7fc64809de50 2023-09-30T22:19:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.650+0000 7fc6457fa700 1 -- 172.21.15.89:0/2852803151 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc63c03fb90 con 0x7fc64809de50 2023-09-30T22:19:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.655+0000 7fc6457fa700 1 -- 172.21.15.89:0/2852803151 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc63c01b3b0 con 0x7fc64809de50 2023-09-30T22:19:22.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.673+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa93c072980 con 0x7fa93c09d630 2023-09-30T22:19:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.673+0000 7fa9327fc700 1 -- 172.21.15.89:0/2726058258 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa938011360 con 0x7fa93c09d630 2023-09-30T22:19:22.673 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa93c09d630 msgr2=0x7fa93c1337c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa93c09d630 0x7fa93c1337c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c550 tx=0x7fa93800c860 comp rx=0 tx=0).stop 2023-09-30T22:19:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 shutdown_connections 2023-09-30T22:19:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa93c138d80 0x7fa93c134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa93c09d630 0x7fa93c1337c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 --2- 172.21.15.89:0/2726058258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa93c0a0a30 0x7fa93c133d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 >> 172.21.15.89:0/2726058258 conn(0x7fa93c0a3050 msgr2=0x7fa93c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 shutdown_connections 2023-09-30T22:19:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.676+0000 7fa943e83700 1 -- 172.21.15.89:0/2726058258 wait complete. 2023-09-30T22:19:22.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:22.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.714+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f518806b9e0 con 0x7f5188139030 2023-09-30T22:19:22.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.715+0000 7f51857fa700 1 -- 172.21.15.89:0/2588399952 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f518001b360 con 0x7f5188139030 2023-09-30T22:19:22.715 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188139030 msgr2=0x7f518811b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 --2- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188139030 0x7f518811b8d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f518000efc0 tx=0x7f518000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 shutdown_connections 2023-09-30T22:19:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 --2- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5188139030 0x7f518811b8d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 --2- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f51881338a0 0x7f518811ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 --2- 172.21.15.89:0/2588399952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5188136710 0x7f518811b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 >> 172.21.15.89:0/2588399952 conn(0x7f51880a2aa0 msgr2=0x7f5188137360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 shutdown_connections 2023-09-30T22:19:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.717+0000 7f518ed5f700 1 -- 172.21.15.89:0/2588399952 wait complete. 2023-09-30T22:19:22.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.820+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9574133020 con 0x7f957412cf60 2023-09-30T22:19:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.820+0000 7f955bfff700 1 -- 172.21.15.89:0/3631530472 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9574133020 con 0x7f957412cf60 2023-09-30T22:19:22.820 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.822+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957412cf60 msgr2=0x7f95741c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.822+0000 7f95799ae700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957412cf60 0x7f95741c8430 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f956c00cbc0 tx=0x7f956c00ced0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.823+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 shutdown_connections 2023-09-30T22:19:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.823+0000 7f95799ae700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957412cf60 0x7f95741c8430 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.823+0000 7f95799ae700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9574066410 0x7f9574130500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.823+0000 7f95799ae700 1 --2- 172.21.15.89:0/3631530472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9574066880 0x7f9574130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.823+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 >> 172.21.15.89:0/3631530472 conn(0x7f95740a2aa0 msgr2=0x7f957412a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.823+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 shutdown_connections 2023-09-30T22:19:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.823+0000 7f95799ae700 1 -- 172.21.15.89:0/3631530472 wait complete. 2023-09-30T22:19:22.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:22.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.850+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc64811dbb0 con 0x7fc64809de50 2023-09-30T22:19:22.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.851+0000 7fc6457fa700 1 -- 172.21.15.89:0/2852803151 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc63c01acb0 con 0x7fc64809de50 2023-09-30T22:19:22.851 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc64809de50 msgr2=0x7fc64811a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc64809de50 0x7fc64811a5c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00e9b0 tx=0x7fc63c00ed70 comp rx=0 tx=0).stop 2023-09-30T22:19:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 shutdown_connections 2023-09-30T22:19:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6481338a0 0x7fc64811b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc64809de50 0x7fc64811a5c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 --2- 172.21.15.89:0/2852803151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc64809e930 0x7fc64811ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:22.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 >> 172.21.15.89:0/2852803151 conn(0x7fc6480a2aa0 msgr2=0x7fc648130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:22.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 shutdown_connections 2023-09-30T22:19:22.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:22.853+0000 7fc64ec27700 1 -- 172.21.15.89:0/2852803151 wait complete. 2023-09-30T22:19:22.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:23.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:23.730 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:23.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.825+0000 7f3ec1164700 1 Processor -- start 2023-09-30T22:19:23.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.825+0000 7f3ec1164700 1 -- start start 2023-09-30T22:19:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.825+0000 7f3ec1164700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc0a0a30 0x7f3ebc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.825+0000 7f3ec1164700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3ebc138d80 0x7f3ebc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.825+0000 7f3ec1164700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc0a2a70 con 0x7f3ebc09d630 2023-09-30T22:19:23.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.825+0000 7f3ec1164700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ebc0a01d0 con 0x7f3ebc138d80 2023-09-30T22:19:23.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.825+0000 7f3ec1164700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc09e2b0 con 0x7f3ebc0a0a30 2023-09-30T22:19:23.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc0a0a30 0x7f3ebc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebb59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3ebc138d80 0x7f3ebc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc0a0a30 0x7f3ebc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46712/0 (socket says 172.21.15.89:46712) 2023-09-30T22:19:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebad9d700 1 -- 172.21.15.89:0/2406922571 learned_addr learned my addr 172.21.15.89:0/2406922571 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebad9d700 1 -- 172.21.15.89:0/2406922571 >> v2:172.21.15.89:3300/0 conn(0x7f3ebc138d80 msgr2=0x7f3ebc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebad9d700 1 --2- 172.21.15.89:0/2406922571 >> v2:172.21.15.89:3300/0 conn(0x7f3ebc138d80 0x7f3ebc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebad9d700 1 -- 172.21.15.89:0/2406922571 >> v1:172.21.15.89:6789/0 conn(0x7f3ebc09d630 legacy=0x7f3ebc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:23.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebad9d700 1 -- 172.21.15.89:0/2406922571 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc09dc50 con 0x7f3ebc0a0a30 2023-09-30T22:19:23.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebb59e700 1 --2- 172.21.15.89:0/2406922571 >> v2:172.21.15.89:3300/0 conn(0x7f3ebc138d80 0x7f3ebc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:23.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3ebad9d700 1 --2- 172.21.15.89:0/2406922571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc0a0a30 0x7f3ebc0a0e00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0098f0 tx=0x7f3eac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45365a6ad2351f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:23.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3eb9d9b700 1 -- 172.21.15.89:0/2406922571 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac011830 con 0x7f3ebc0a0a30 2023-09-30T22:19:23.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3eb9d9b700 1 -- 172.21.15.89:0/2406922571 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac011990 con 0x7f3ebc0a0a30 2023-09-30T22:19:23.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.826+0000 7f3eb9d9b700 1 -- 172.21.15.89:0/2406922571 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac011d20 con 0x7f3ebc0a0a30 2023-09-30T22:19:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.827+0000 7f3ec1164700 1 -- 172.21.15.89:0/2406922571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc0a0a30 msgr2=0x7f3ebc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.827+0000 7f3ec1164700 1 --2- 172.21.15.89:0/2406922571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc0a0a30 0x7f3ebc0a0e00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0098f0 tx=0x7f3eac009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.827+0000 7f3ec1164700 1 -- 172.21.15.89:0/2406922571 shutdown_connections 2023-09-30T22:19:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.827+0000 7f3ec1164700 1 --2- 172.21.15.89:0/2406922571 >> v2:172.21.15.89:3300/0 conn(0x7f3ebc138d80 0x7f3ebc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.827+0000 7f3ec1164700 1 --2- 172.21.15.89:0/2406922571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc0a0a30 0x7f3ebc0a0e00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.827+0000 7f3ec1164700 1 -- 172.21.15.89:0/2406922571 >> 172.21.15.89:0/2406922571 conn(0x7f3ebc0a3050 msgr2=0x7f3ebc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:23.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.827+0000 7f3ec1164700 1 -- 172.21.15.89:0/2406922571 shutdown_connections 2023-09-30T22:19:23.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.827+0000 7f3ec1164700 1 -- 172.21.15.89:0/2406922571 wait complete. 2023-09-30T22:19:23.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.828+0000 7f3ec1164700 1 Processor -- start 2023-09-30T22:19:23.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.828+0000 7f3ec1164700 1 -- start start 2023-09-30T22:19:23.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.828+0000 7f3ec1164700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc09d630 0x7f3ebc120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.828+0000 7f3ec1164700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ebc0a0a30 0x7f3ebc120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.828+0000 7f3ec1164700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ebc138d80 0x7f3ebc1211e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ec1164700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc123670 con 0x7f3ebc138d80 2023-09-30T22:19:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ec1164700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc121bc0 con 0x7f3ebc0a0a30 2023-09-30T22:19:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ec1164700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc124230 con 0x7f3ebc09d630 2023-09-30T22:19:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc09d630 0x7f3ebc120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc09d630 0x7f3ebc120760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46716/0 (socket says 172.21.15.89:46716) 2023-09-30T22:19:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebad9d700 1 -- 172.21.15.89:0/2535057698 learned_addr learned my addr 172.21.15.89:0/2535057698 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3eba59c700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ebc0a0a30 0x7f3ebc120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebb59e700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ebc138d80 0x7f3ebc1211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebb59e700 1 -- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc09d630 msgr2=0x7f3ebc120760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebb59e700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc09d630 0x7f3ebc120760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebb59e700 1 -- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ebc0a0a30 msgr2=0x7f3ebc120ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebb59e700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ebc0a0a30 0x7f3ebc120ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebb59e700 1 -- 172.21.15.89:0/2535057698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb4009a70 con 0x7f3ebc138d80 2023-09-30T22:19:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebad9d700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc09d630 0x7f3ebc120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:23.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3eba59c700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ebc0a0a30 0x7f3ebc120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ebb59e700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ebc138d80 0x7f3ebc1211e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0a2430 tx=0x7f3eb400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ea37fe700 1 -- 172.21.15.89:0/2535057698 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400cc20 con 0x7f3ebc138d80 2023-09-30T22:19:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ea37fe700 1 -- 172.21.15.89:0/2535057698 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb400f040 con 0x7f3ebc138d80 2023-09-30T22:19:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ea37fe700 1 -- 172.21.15.89:0/2535057698 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4014480 con 0x7f3ebc138d80 2023-09-30T22:19:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac009550 con 0x7f3ebc138d80 2023-09-30T22:19:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.829+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc0a0410 con 0x7f3ebc138d80 2023-09-30T22:19:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.830+0000 7f3ea37fe700 1 -- 172.21.15.89:0/2535057698 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3eb4014600 con 0x7f3ebc138d80 2023-09-30T22:19:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.830+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc0747f0 con 0x7f3ebc138d80 2023-09-30T22:19:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.831+0000 7f3ea37fe700 1 -- 172.21.15.89:0/2535057698 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3eb4040450 con 0x7f3ebc138d80 2023-09-30T22:19:23.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:23.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.835+0000 7f3ea37fe700 1 -- 172.21.15.89:0/2535057698 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3eb4012930 con 0x7f3ebc138d80 2023-09-30T22:19:23.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.865+0000 7f2f34b85700 1 Processor -- start 2023-09-30T22:19:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.865+0000 7f2f34b85700 1 -- start start 2023-09-30T22:19:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f34b85700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f301338a0 0x7f2f30133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f34b85700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f34b85700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3006ef30 con 0x7f2f3009de50 2023-09-30T22:19:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f34b85700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3006fef0 con 0x7f2f301338a0 2023-09-30T22:19:23.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f34b85700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301334b0 con 0x7f2f3009e930 2023-09-30T22:19:23.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f301338a0 0x7f2f30133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f301338a0 0x7f2f30133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40934/0 (socket says 172.21.15.89:40934) 2023-09-30T22:19:23.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3013b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46720/0 (socket says 172.21.15.89:46720) 2023-09-30T22:19:23.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2e59c700 1 -- 172.21.15.89:0/1041039883 learned_addr learned my addr 172.21.15.89:0/1041039883 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:23.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2ed9d700 1 -- 172.21.15.89:0/1041039883 >> v2:172.21.15.89:3300/0 conn(0x7f2f301338a0 msgr2=0x7f2f30133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:23.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2ed9d700 1 --2- 172.21.15.89:0/1041039883 >> v2:172.21.15.89:3300/0 conn(0x7f2f301338a0 0x7f2f30133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2ed9d700 1 -- 172.21.15.89:0/1041039883 >> v1:172.21.15.89:6789/0 conn(0x7f2f3009de50 legacy=0x7f2f3009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:23.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2ed9d700 1 -- 172.21.15.89:0/1041039883 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3006fc30 con 0x7f2f3009e930 2023-09-30T22:19:23.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2e59c700 1 --2- 172.21.15.89:0/1041039883 >> v2:172.21.15.89:3300/0 conn(0x7f2f301338a0 0x7f2f30133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2ed9d700 1 --2- 172.21.15.89:0/1041039883 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3013b740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009d90 tx=0x7f2f2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b19d4940d4ca05 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.866+0000 7f2f2d59a700 1 -- 172.21.15.89:0/1041039883 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800e070 con 0x7f2f3009e930 2023-09-30T22:19:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f2d59a700 1 -- 172.21.15.89:0/1041039883 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2800bb20 con 0x7f2f3009e930 2023-09-30T22:19:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f2d59a700 1 -- 172.21.15.89:0/1041039883 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28012430 con 0x7f2f3009e930 2023-09-30T22:19:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f34b85700 1 -- 172.21.15.89:0/1041039883 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 msgr2=0x7f2f3013b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f34b85700 1 --2- 172.21.15.89:0/1041039883 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3013b740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009d90 tx=0x7f2f2800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f34b85700 1 -- 172.21.15.89:0/1041039883 shutdown_connections 2023-09-30T22:19:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f34b85700 1 --2- 172.21.15.89:0/1041039883 >> v2:172.21.15.89:3300/0 conn(0x7f2f301338a0 0x7f2f30133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f34b85700 1 --2- 172.21.15.89:0/1041039883 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3013b740 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f34b85700 1 -- 172.21.15.89:0/1041039883 >> 172.21.15.89:0/1041039883 conn(0x7f2f300a2aa0 msgr2=0x7f2f300a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f34b85700 1 -- 172.21.15.89:0/1041039883 shutdown_connections 2023-09-30T22:19:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.867+0000 7f2f34b85700 1 -- 172.21.15.89:0/1041039883 wait complete. 2023-09-30T22:19:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.868+0000 7f2f34b85700 1 Processor -- start 2023-09-30T22:19:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.868+0000 7f2f34b85700 1 -- start start 2023-09-30T22:19:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f34b85700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f3009de50 0x7f2f3011d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f34b85700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3011dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f34b85700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f301338a0 0x7f2f3011e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f34b85700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30121260 con 0x7f2f301338a0 2023-09-30T22:19:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f34b85700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3011f930 con 0x7f2f3009de50 2023-09-30T22:19:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f34b85700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301209f0 con 0x7f2f3009e930 2023-09-30T22:19:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f301338a0 0x7f2f3011e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f301338a0 0x7f2f3011e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40938/0 (socket says 172.21.15.89:40938) 2023-09-30T22:19:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2ed9d700 1 -- 172.21.15.89:0/1505137050 learned_addr learned my addr 172.21.15.89:0/1505137050 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2dd9b700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3011dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2e59c700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f3009de50 0x7f2f3011d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2ed9d700 1 -- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 msgr2=0x7f2f3011dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2ed9d700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3011dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2ed9d700 1 -- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f3009de50 msgr2=0x7f2f3011d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2ed9d700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f3009de50 0x7f2f3011d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2ed9d700 1 -- 172.21.15.89:0/1505137050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20009550 con 0x7f2f301338a0 2023-09-30T22:19:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.869+0000 7f2f2e59c700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f3009de50 0x7f2f3011d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.870+0000 7f2f2ed9d700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f301338a0 0x7f2f3011e210 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2f3006b6d0 tx=0x7f2f28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.870+0000 7f2f1effd700 1 -- 172.21.15.89:0/1505137050 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28012430 con 0x7f2f301338a0 2023-09-30T22:19:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.870+0000 7f2f1effd700 1 -- 172.21.15.89:0/1505137050 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28020070 con 0x7f2f301338a0 2023-09-30T22:19:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.870+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f28009a70 con 0x7f2f301338a0 2023-09-30T22:19:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.870+0000 7f2f1effd700 1 -- 172.21.15.89:0/1505137050 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28003ea0 con 0x7f2f301338a0 2023-09-30T22:19:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.870+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3006f690 con 0x7f2f301338a0 2023-09-30T22:19:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.871+0000 7f2f1effd700 1 -- 172.21.15.89:0/1505137050 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f28004280 con 0x7f2f301338a0 2023-09-30T22:19:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.871+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3011a470 con 0x7f2f301338a0 2023-09-30T22:19:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.871+0000 7f2f1effd700 1 -- 172.21.15.89:0/1505137050 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2f28022070 con 0x7f2f301338a0 2023-09-30T22:19:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.876+0000 7f2f1effd700 1 -- 172.21.15.89:0/1505137050 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f2801a970 con 0x7f2f301338a0 2023-09-30T22:19:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.973+0000 7fc36ffa7700 1 Processor -- start 2023-09-30T22:19:24.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.973+0000 7fc36ffa7700 1 -- start start 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.973+0000 7fc36ffa7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc368130540 0x7fc368130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.973+0000 7fc36ffa7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.973+0000 7fc36ffa7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36806ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.973+0000 7fc36ffa7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36809a390 con 0x7fc368130540 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.973+0000 7fc36ffa7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680716a0 con 0x7fc368130e50 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.973+0000 7fc36ffa7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc368071290 con 0x7fc36806fb20 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36dd43700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc368130540 0x7fc368130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36dd43700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc368130540 0x7fc368130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40940/0 (socket says 172.21.15.89:40940) 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36dd43700 1 -- 172.21.15.89:0/4019111252 learned_addr learned my addr 172.21.15.89:0/4019111252 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36d542700 1 --2- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36e544700 1 --2- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36806ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36d542700 1 -- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 msgr2=0x7fc36806ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36d542700 1 --2- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36806ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36d542700 1 -- 172.21.15.89:0/4019111252 >> v2:172.21.15.89:3300/0 conn(0x7fc368130540 msgr2=0x7fc368130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36d542700 1 --2- 172.21.15.89:0/4019111252 >> v2:172.21.15.89:3300/0 conn(0x7fc368130540 0x7fc368130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36d542700 1 -- 172.21.15.89:0/4019111252 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368130340 con 0x7fc368130e50 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36d542700 1 --2- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36806f5e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc35800dc60 tx=0x7fc35800df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991d31feb3085950 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36cd41700 1 -- 172.21.15.89:0/4019111252 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35800b070 con 0x7fc368130e50 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36cd41700 1 -- 172.21.15.89:0/4019111252 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc358008a30 con 0x7fc368130e50 2023-09-30T22:19:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.974+0000 7fc36cd41700 1 -- 172.21.15.89:0/4019111252 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358008de0 con 0x7fc368130e50 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 -- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 msgr2=0x7fc36806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 --2- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36806f5e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc35800dc60 tx=0x7fc35800df70 comp rx=0 tx=0).stop 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 -- 172.21.15.89:0/4019111252 shutdown_connections 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 --2- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36806ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 --2- 172.21.15.89:0/4019111252 >> v2:172.21.15.89:3300/0 conn(0x7fc368130540 0x7fc368130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 --2- 172.21.15.89:0/4019111252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36806f5e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 -- 172.21.15.89:0/4019111252 >> 172.21.15.89:0/4019111252 conn(0x7fc3680a2aa0 msgr2=0x7fc3680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 -- 172.21.15.89:0/4019111252 shutdown_connections 2023-09-30T22:19:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.975+0000 7fc36ffa7700 1 -- 172.21.15.89:0/4019111252 wait complete. 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36ffa7700 1 Processor -- start 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36ffa7700 1 -- start start 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36ffa7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36811a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36ffa7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc368130540 0x7fc36811abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36ffa7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36811b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36ffa7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36811d4b0 con 0x7fc368130540 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36ffa7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36811bb20 con 0x7fc368130e50 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36ffa7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc36811df00 con 0x7fc36806fb20 2023-09-30T22:19:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.976+0000 7fc36dd43700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36811a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36dd43700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36811a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46738/0 (socket says 172.21.15.89:46738) 2023-09-30T22:19:24.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36dd43700 1 -- 172.21.15.89:0/415505535 learned_addr learned my addr 172.21.15.89:0/415505535 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:24.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36d542700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc368130540 0x7fc36811abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36e544700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36811b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36dd43700 1 -- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 msgr2=0x7fc36811b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36dd43700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36811b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36dd43700 1 -- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc368130540 msgr2=0x7fc36811abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36dd43700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc368130540 0x7fc36811abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36dd43700 1 -- 172.21.15.89:0/415505535 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35800d940 con 0x7fc36806fb20 2023-09-30T22:19:24.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36d542700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc368130540 0x7fc36811abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:24.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36dd43700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36811a670 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc36400fb80 tx=0x7fc36400ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:24.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc35e7fc700 1 -- 172.21.15.89:0/415505535 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36400b750 con 0x7fc36806fb20 2023-09-30T22:19:24.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc35e7fc700 1 -- 172.21.15.89:0/415505535 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36400de20 con 0x7fc36806fb20 2023-09-30T22:19:24.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36811cff0 con 0x7fc36806fb20 2023-09-30T22:19:24.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc35e7fc700 1 -- 172.21.15.89:0/415505535 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364018680 con 0x7fc36806fb20 2023-09-30T22:19:24.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.977+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3680718e0 con 0x7fc36806fb20 2023-09-30T22:19:24.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.978+0000 7fc35e7fc700 1 -- 172.21.15.89:0/415505535 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc3640187e0 con 0x7fc36806fb20 2023-09-30T22:19:24.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.978+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36809cd40 con 0x7fc36806fb20 2023-09-30T22:19:24.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.979+0000 7fc35e7fc700 1 -- 172.21.15.89:0/415505535 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc36403fdd0 con 0x7fc36806fb20 2023-09-30T22:19:24.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:23.983+0000 7fc35e7fc700 1 -- 172.21.15.89:0/415505535 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc364010440 con 0x7fc36806fb20 2023-09-30T22:19:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.014+0000 7f8198344700 1 Processor -- start 2023-09-30T22:19:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.014+0000 7f8198344700 1 -- start start 2023-09-30T22:19:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f8198344700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009e850 0x7f819009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f8198344700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f819009f160 0x7f819013c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f8198344700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819009aa30 con 0x7f819009f590 2023-09-30T22:19:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f8198344700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819006c930 con 0x7f819009f160 2023-09-30T22:19:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f8198344700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819006bf30 con 0x7f819009e850 2023-09-30T22:19:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81958df700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f819009f160 0x7f819013c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81958df700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f819009f160 0x7f819013c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40962/0 (socket says 172.21.15.89:40962) 2023-09-30T22:19:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81958df700 1 -- 172.21.15.89:0/1549986644 learned_addr learned my addr 172.21.15.89:0/1549986644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81960e0700 1 --2- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009e850 0x7f819009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81958df700 1 -- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009e850 msgr2=0x7f819009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81958df700 1 --2- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009e850 0x7f819009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81958df700 1 -- 172.21.15.89:0/1549986644 >> v1:172.21.15.89:6789/0 conn(0x7f819009f590 legacy=0x7f819013e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81958df700 1 -- 172.21.15.89:0/1549986644 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81900a0250 con 0x7f819009f160 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.015+0000 7f81960e0700 1 --2- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009e850 0x7f819009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.016+0000 7f81958df700 1 --2- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f160 0x7f819013c370 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f81800098f0 tx=0x7f8180009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9f268cb6bcb15d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.016+0000 7f81950de700 1 -- 172.21.15.89:0/1549986644 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000b8a0 con 0x7f819009f160 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.016+0000 7f81950de700 1 -- 172.21.15.89:0/1549986644 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000ba00 con 0x7f819009f160 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.016+0000 7f81950de700 1 -- 172.21.15.89:0/1549986644 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000bd90 con 0x7f819009f160 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.017+0000 7f8198344700 1 -- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f160 msgr2=0x7f819013c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.017+0000 7f8198344700 1 --2- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f160 0x7f819013c370 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f81800098f0 tx=0x7f8180009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.017+0000 7f8198344700 1 -- 172.21.15.89:0/1549986644 shutdown_connections 2023-09-30T22:19:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.017+0000 7f8198344700 1 --2- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009e850 0x7f819009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.017+0000 7f8198344700 1 --2- 172.21.15.89:0/1549986644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f160 0x7f819013c370 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.017+0000 7f8198344700 1 -- 172.21.15.89:0/1549986644 >> 172.21.15.89:0/1549986644 conn(0x7f81900a3f70 msgr2=0x7f81900a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.017+0000 7f8198344700 1 -- 172.21.15.89:0/1549986644 shutdown_connections 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.017+0000 7f8198344700 1 -- 172.21.15.89:0/1549986644 wait complete. 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f8198344700 1 Processor -- start 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f8198344700 1 -- start start 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f8198344700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f819009e850 0x7f81901cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f8198344700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009f160 0x7f81901ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f8198344700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f590 0x7f81901cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f8198344700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81901ad6c0 con 0x7f819009f590 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f8198344700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81901ae840 con 0x7f819009f160 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f8198344700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81901ac060 con 0x7f819009e850 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f81968e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f590 0x7f81901cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f81968e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f590 0x7f81901cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40970/0 (socket says 172.21.15.89:40970) 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f81968e1700 1 -- 172.21.15.89:0/2107894889 learned_addr learned my addr 172.21.15.89:0/2107894889 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.018+0000 7f81958df700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009f160 0x7f81901ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f81960e0700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f819009e850 0x7f81901cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f81968e1700 1 -- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f819009e850 msgr2=0x7f81901cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f81968e1700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f819009e850 0x7f81901cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f81968e1700 1 -- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009f160 msgr2=0x7f81901ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f81968e1700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009f160 0x7f81901ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f81968e1700 1 -- 172.21.15.89:0/2107894889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009550 con 0x7f819009f590 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f81958df700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009f160 0x7f81901ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f81968e1700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f590 0x7f81901cd290 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f818800cbc0 tx=0x7f818800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f8186ffd700 1 -- 172.21.15.89:0/2107894889 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188003c00 con 0x7f819009f590 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f8186ffd700 1 -- 172.21.15.89:0/2107894889 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188003d60 con 0x7f819009f590 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819006fa90 con 0x7f819009f590 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f8186ffd700 1 -- 172.21.15.89:0/2107894889 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818800abf0 con 0x7f819009f590 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.019+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819011dd90 con 0x7f819009f590 2023-09-30T22:19:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.020+0000 7f8186ffd700 1 -- 172.21.15.89:0/2107894889 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f818800ad50 con 0x7f819009f590 2023-09-30T22:19:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.020+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81900747f0 con 0x7f819009f590 2023-09-30T22:19:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.020+0000 7f8186ffd700 1 -- 172.21.15.89:0/2107894889 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f818803f6e0 con 0x7f819009f590 2023-09-30T22:19:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.025+0000 7f8186ffd700 1 -- 172.21.15.89:0/2107894889 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81880196c0 con 0x7f819009f590 2023-09-30T22:19:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.036+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ebc072980 con 0x7f3ebc138d80 2023-09-30T22:19:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.036+0000 7f3ea37fe700 1 -- 172.21.15.89:0/2535057698 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3eb4012340 con 0x7f3ebc138d80 2023-09-30T22:19:24.394 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.038+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ebc138d80 msgr2=0x7f3ebc1211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.038+0000 7f3ec1164700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ebc138d80 0x7f3ebc1211e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0a2430 tx=0x7f3eb400ef90 comp rx=0 tx=0).stop 2023-09-30T22:19:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.038+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 shutdown_connections 2023-09-30T22:19:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.038+0000 7f3ec1164700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ebc138d80 0x7f3ebc1211e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.038+0000 7f3ec1164700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ebc09d630 0x7f3ebc120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.038+0000 7f3ec1164700 1 --2- 172.21.15.89:0/2535057698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ebc0a0a30 0x7f3ebc120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.038+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 >> 172.21.15.89:0/2535057698 conn(0x7f3ebc0a3050 msgr2=0x7f3ebc139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.039+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 shutdown_connections 2023-09-30T22:19:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.039+0000 7f3ec1164700 1 -- 172.21.15.89:0/2535057698 wait complete. 2023-09-30T22:19:24.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.077+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2f30074ea0 con 0x7f2f301338a0 2023-09-30T22:19:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.078+0000 7f2f1effd700 1 -- 172.21.15.89:0/1505137050 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f2801a380 con 0x7f2f301338a0 2023-09-30T22:19:24.397 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f301338a0 msgr2=0x7f2f3011e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f301338a0 0x7f2f3011e210 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2f3006b6d0 tx=0x7f2f28004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 shutdown_connections 2023-09-30T22:19:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f301338a0 0x7f2f3011e210 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f3009de50 0x7f2f3011d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 --2- 172.21.15.89:0/1505137050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f3009e930 0x7f2f3011dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 >> 172.21.15.89:0/1505137050 conn(0x7f2f300a2aa0 msgr2=0x7f2f301b9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 shutdown_connections 2023-09-30T22:19:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.080+0000 7f2f34b85700 1 -- 172.21.15.89:0/1505137050 wait complete. 2023-09-30T22:19:24.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.180+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc368074ea0 con 0x7fc36806fb20 2023-09-30T22:19:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.180+0000 7fc35e7fc700 1 -- 172.21.15.89:0/415505535 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc36401d030 con 0x7fc36806fb20 2023-09-30T22:19:24.401 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 msgr2=0x7fc36811a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36811a670 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc36400fb80 tx=0x7fc36400ff40 comp rx=0 tx=0).stop 2023-09-30T22:19:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 shutdown_connections 2023-09-30T22:19:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc368130e50 0x7fc36811b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc36806fb20 0x7fc36811a670 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 --2- 172.21.15.89:0/415505535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc368130540 0x7fc36811abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 >> 172.21.15.89:0/415505535 conn(0x7fc3680a2aa0 msgr2=0x7fc36812d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 shutdown_connections 2023-09-30T22:19:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.182+0000 7fc36ffa7700 1 -- 172.21.15.89:0/415505535 wait complete. 2023-09-30T22:19:24.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.219+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f81901ae980 con 0x7f819009f590 2023-09-30T22:19:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.219+0000 7f8186ffd700 1 -- 172.21.15.89:0/2107894889 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8188019300 con 0x7f819009f590 2023-09-30T22:19:24.404 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.221+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f590 msgr2=0x7f81901cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.221+0000 7f8198344700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f590 0x7f81901cd290 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f818800cbc0 tx=0x7f818800cf80 comp rx=0 tx=0).stop 2023-09-30T22:19:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.222+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 shutdown_connections 2023-09-30T22:19:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.222+0000 7f8198344700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f819009f590 0x7f81901cd290 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.222+0000 7f8198344700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f819009e850 0x7f81901cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.222+0000 7f8198344700 1 --2- 172.21.15.89:0/2107894889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f819009f160 0x7f81901ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.222+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 >> 172.21.15.89:0/2107894889 conn(0x7f81900a3f70 msgr2=0x7f81900a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.222+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 shutdown_connections 2023-09-30T22:19:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:24.222+0000 7f8198344700 1 -- 172.21.15.89:0/2107894889 wait complete. 2023-09-30T22:19:24.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:25.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:25.400 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:25.404 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:25.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:25.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.543+0000 7f991f77c700 1 Processor -- start 2023-09-30T22:19:25.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.543+0000 7f991f77c700 1 -- start start 2023-09-30T22:19:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991f77c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99181338a0 0x7f9918133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991f77c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809de50 0x7f991809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991f77c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991806ef30 con 0x7f991809e760 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991f77c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991806fef0 con 0x7f99181338a0 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991f77c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99181334b0 con 0x7f991809de50 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991dd19700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f991809e760 0x7f991813a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55100/0 (socket says 172.21.15.89:55100) 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991dd19700 1 -- 172.21.15.89:0/1608593506 learned_addr learned my addr 172.21.15.89:0/1608593506 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991cd17700 1 --2- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809de50 0x7f991809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f991d518700 1 --2- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99181338a0 0x7f9918133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.544+0000 7f990ffff700 1 -- 172.21.15.89:0/1608593506 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446010932 0 0) 0x7f991806ef30 con 0x7f991809e760 2023-09-30T22:19:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.545+0000 7f991cd17700 1 -- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99181338a0 msgr2=0x7f9918133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.545+0000 7f991cd17700 1 --2- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99181338a0 0x7f9918133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.545+0000 7f991cd17700 1 -- 172.21.15.89:0/1608593506 >> v1:172.21.15.89:6789/0 conn(0x7f991809e760 legacy=0x7f991813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.545+0000 7f991cd17700 1 -- 172.21.15.89:0/1608593506 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991806fc30 con 0x7f991809de50 2023-09-30T22:19:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.545+0000 7f991cd17700 1 --2- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809de50 0x7f991809e220 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f99080098a0 tx=0x7f9908009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfcc5f59a5e1c202 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.545+0000 7f990ffff700 1 -- 172.21.15.89:0/1608593506 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908011830 con 0x7f991809de50 2023-09-30T22:19:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.545+0000 7f990ffff700 1 -- 172.21.15.89:0/1608593506 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9908011990 con 0x7f991809de50 2023-09-30T22:19:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.545+0000 7f990ffff700 1 -- 172.21.15.89:0/1608593506 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908011d20 con 0x7f991809de50 2023-09-30T22:19:25.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.546+0000 7f991f77c700 1 -- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809de50 msgr2=0x7f991809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.546+0000 7f991f77c700 1 --2- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809de50 0x7f991809e220 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f99080098a0 tx=0x7f9908009bb0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.546+0000 7f991f77c700 1 -- 172.21.15.89:0/1608593506 shutdown_connections 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.546+0000 7f991f77c700 1 --2- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99181338a0 0x7f9918133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.546+0000 7f991f77c700 1 --2- 172.21.15.89:0/1608593506 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809de50 0x7f991809e220 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.546+0000 7f991f77c700 1 -- 172.21.15.89:0/1608593506 >> 172.21.15.89:0/1608593506 conn(0x7f99180a2aa0 msgr2=0x7f99180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.546+0000 7f991f77c700 1 -- 172.21.15.89:0/1608593506 shutdown_connections 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.546+0000 7f991f77c700 1 -- 172.21.15.89:0/1608593506 wait complete. 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991f77c700 1 Processor -- start 2023-09-30T22:19:25.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991f77c700 1 -- start start 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991f77c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991809de50 0x7f991811d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991f77c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809e760 0x7f991811dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991f77c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99181338a0 0x7f991811e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991f77c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991811ebe0 con 0x7f99181338a0 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991f77c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9918120fb0 con 0x7f991809de50 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991f77c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9918121190 con 0x7f991809e760 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.547+0000 7f991cd17700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809e760 0x7f991811dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991cd17700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809e760 0x7f991811dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46762/0 (socket says 172.21.15.89:46762) 2023-09-30T22:19:25.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991cd17700 1 -- 172.21.15.89:0/199138013 learned_addr learned my addr 172.21.15.89:0/199138013 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991dd19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99181338a0 0x7f991811e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991d518700 1 --2- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991809de50 0x7f991811d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991cd17700 1 -- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991809de50 msgr2=0x7f991811d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991cd17700 1 --2- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991809de50 0x7f991811d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991cd17700 1 -- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99181338a0 msgr2=0x7f991811e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991cd17700 1 --2- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99181338a0 0x7f991811e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991cd17700 1 -- 172.21.15.89:0/199138013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99080094a0 con 0x7f991809e760 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991cd17700 1 --2- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809e760 0x7f991811dc50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9908009870 tx=0x7f9908004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f990dffb700 1 -- 172.21.15.89:0/199138013 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908004450 con 0x7f991809e760 2023-09-30T22:19:25.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f990dffb700 1 -- 172.21.15.89:0/199138013 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990801e070 con 0x7f991809e760 2023-09-30T22:19:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f990dffb700 1 -- 172.21.15.89:0/199138013 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99080038e0 con 0x7f991809e760 2023-09-30T22:19:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.548+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991811f320 con 0x7f991809e760 2023-09-30T22:19:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.549+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991806f640 con 0x7f991809e760 2023-09-30T22:19:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.550+0000 7f990dffb700 1 -- 172.21.15.89:0/199138013 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9908003a40 con 0x7f991809e760 2023-09-30T22:19:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.550+0000 7f990dffb700 1 -- 172.21.15.89:0/199138013 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9908040420 con 0x7f991809e760 2023-09-30T22:19:25.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.550+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9918120e10 con 0x7f991809e760 2023-09-30T22:19:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.555+0000 7f990dffb700 1 -- 172.21.15.89:0/199138013 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9918120e10 con 0x7f991809e760 2023-09-30T22:19:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b59aea700 1 Processor -- start 2023-09-30T22:19:25.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b59aea700 1 -- start start 2023-09-30T22:19:25.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b59aea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412cf60 0x7f4b5406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b59aea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b5412f8b0 0x7f4b5412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b59aea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b54130260 0x7f4b54136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b59aea700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5412d330 con 0x7f4b54130260 2023-09-30T22:19:25.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b59aea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5412cd90 con 0x7f4b5412f8b0 2023-09-30T22:19:25.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b59aea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541306d0 con 0x7f4b5412cf60 2023-09-30T22:19:25.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b53fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b54130260 0x7f4b54136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b53fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b54130260 0x7f4b54136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40986/0 (socket says 172.21.15.89:40986) 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.579+0000 7f4b53fff700 1 -- 172.21.15.89:0/261560893 learned_addr learned my addr 172.21.15.89:0/261560893 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b537fe700 1 --2- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412cf60 0x7f4b5406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b53fff700 1 -- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412cf60 msgr2=0x7f4b5406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b53fff700 1 --2- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412cf60 0x7f4b5406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b53fff700 1 -- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b5412f8b0 msgr2=0x7f4b5412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b53fff700 1 --2- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b5412f8b0 0x7f4b5412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b53fff700 1 -- 172.21.15.89:0/261560893 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54071780 con 0x7f4b54130260 2023-09-30T22:19:25.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b53fff700 1 --2- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b54130260 0x7f4b54136c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00af60 tx=0x7f4b4c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3ba5426cd231c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b527fc700 1 -- 172.21.15.89:0/261560893 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00a2d0 con 0x7f4b54130260 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b527fc700 1 -- 172.21.15.89:0/261560893 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c00c960 con 0x7f4b54130260 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.580+0000 7f4b527fc700 1 -- 172.21.15.89:0/261560893 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00cd10 con 0x7f4b54130260 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 -- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b54130260 msgr2=0x7f4b54136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 --2- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b54130260 0x7f4b54136c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00af60 tx=0x7f4b4c00af90 comp rx=0 tx=0).stop 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 -- 172.21.15.89:0/261560893 shutdown_connections 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 --2- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b54130260 0x7f4b54136c20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 --2- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412cf60 0x7f4b5406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 --2- 172.21.15.89:0/261560893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b5412f8b0 0x7f4b5412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 -- 172.21.15.89:0/261560893 >> 172.21.15.89:0/261560893 conn(0x7f4b540a2aa0 msgr2=0x7f4b540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 -- 172.21.15.89:0/261560893 shutdown_connections 2023-09-30T22:19:25.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.581+0000 7f4b59aea700 1 -- 172.21.15.89:0/261560893 wait complete. 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.582+0000 7f4b59aea700 1 Processor -- start 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.582+0000 7f4b59aea700 1 -- start start 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.582+0000 7f4b59aea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5412cf60 0x7f4b5409b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.582+0000 7f4b59aea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412f8b0 0x7f4b5409be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.582+0000 7f4b59aea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b54130260 0x7f4b541c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.582+0000 7f4b59aea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54129fa0 con 0x7f4b5412cf60 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b59aea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5409e480 con 0x7f4b54130260 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b59aea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5409c3b0 con 0x7f4b5412f8b0 2023-09-30T22:19:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5412cf60 0x7f4b5409b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5412cf60 0x7f4b5409b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41002/0 (socket says 172.21.15.89:41002) 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b537fe700 1 -- 172.21.15.89:0/2685237404 learned_addr learned my addr 172.21.15.89:0/2685237404 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b53fff700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b54130260 0x7f4b541c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b53fff700 1 -- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412f8b0 msgr2=0x7f4b5409be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b53fff700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412f8b0 0x7f4b5409be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b53fff700 1 -- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5412cf60 msgr2=0x7f4b5409b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b53fff700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5412cf60 0x7f4b5409b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b53fff700 1 -- 172.21.15.89:0/2685237404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c00ac40 con 0x7f4b54130260 2023-09-30T22:19:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b537fe700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5412cf60 0x7f4b5409b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b53fff700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b54130260 0x7f4b541c3f90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c015000 tx=0x7f4b4c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b3bfff700 1 -- 172.21.15.89:0/2685237404 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c003e30 con 0x7f4b54130260 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b3bfff700 1 -- 172.21.15.89:0/2685237404 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c004030 con 0x7f4b54130260 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5409b3d0 con 0x7f4b54130260 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b3bfff700 1 -- 172.21.15.89:0/2685237404 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004420 con 0x7f4b54130260 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.583+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5409d780 con 0x7f4b54130260 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.584+0000 7f4b3bfff700 1 -- 172.21.15.89:0/2685237404 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b4c011070 con 0x7f4b54130260 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.584+0000 7f4b3bfff700 1 -- 172.21.15.89:0/2685237404 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4b4c040ab0 con 0x7f4b54130260 2023-09-30T22:19:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.585+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b54074ea0 con 0x7f4b54130260 2023-09-30T22:19:25.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.587+0000 7f9522104700 1 Processor -- start 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.587+0000 7f9522104700 1 -- start start 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f9522104700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f951c134630 0x7f951c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f9522104700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c136f50 0x7f951c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f9522104700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c09b4f0 con 0x7f951c131810 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f9522104700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c1313c0 con 0x7f951c134630 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f9522104700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c06e8c0 con 0x7f951c136f50 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f951bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c136f50 0x7f951c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f951bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c136f50 0x7f951c139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46346/0 (socket says 172.21.15.89:46346) 2023-09-30T22:19:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f951bfff700 1 -- 172.21.15.89:0/3510674449 learned_addr learned my addr 172.21.15.89:0/3510674449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f951bfff700 1 -- 172.21.15.89:0/3510674449 >> v2:172.21.15.89:3300/0 conn(0x7f951c134630 msgr2=0x7f951c136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f951bfff700 1 --2- 172.21.15.89:0/3510674449 >> v2:172.21.15.89:3300/0 conn(0x7f951c134630 0x7f951c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f951bfff700 1 -- 172.21.15.89:0/3510674449 >> v1:172.21.15.89:6789/0 conn(0x7f951c131810 legacy=0x7f951c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f951bfff700 1 -- 172.21.15.89:0/3510674449 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c071d20 con 0x7f951c136f50 2023-09-30T22:19:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.588+0000 7f951bfff700 1 --2- 172.21.15.89:0/3510674449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c136f50 0x7f951c139330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f951400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcd741ca3ce77986 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.590+0000 7f4b3bfff700 1 -- 172.21.15.89:0/2685237404 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b4c029460 con 0x7f4b54130260 2023-09-30T22:19:25.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.590+0000 7f951a7fc700 1 -- 172.21.15.89:0/3510674449 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951400e070 con 0x7f951c136f50 2023-09-30T22:19:25.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.590+0000 7f951a7fc700 1 -- 172.21.15.89:0/3510674449 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514003a20 con 0x7f951c136f50 2023-09-30T22:19:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.590+0000 7f951a7fc700 1 -- 172.21.15.89:0/3510674449 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514003dd0 con 0x7f951c136f50 2023-09-30T22:19:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.590+0000 7f9522104700 1 -- 172.21.15.89:0/3510674449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c136f50 msgr2=0x7f951c139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.590+0000 7f9522104700 1 --2- 172.21.15.89:0/3510674449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c136f50 0x7f951c139330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f951400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.591+0000 7f9522104700 1 -- 172.21.15.89:0/3510674449 shutdown_connections 2023-09-30T22:19:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.591+0000 7f9522104700 1 --2- 172.21.15.89:0/3510674449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c136f50 0x7f951c139330 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.591+0000 7f9522104700 1 --2- 172.21.15.89:0/3510674449 >> v2:172.21.15.89:3300/0 conn(0x7f951c134630 0x7f951c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.591+0000 7f9522104700 1 -- 172.21.15.89:0/3510674449 >> 172.21.15.89:0/3510674449 conn(0x7f951c0a2aa0 msgr2=0x7f951c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.591+0000 7f9522104700 1 -- 172.21.15.89:0/3510674449 shutdown_connections 2023-09-30T22:19:25.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.591+0000 7f9522104700 1 -- 172.21.15.89:0/3510674449 wait complete. 2023-09-30T22:19:25.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.591+0000 7f616805d700 1 Processor -- start 2023-09-30T22:19:25.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f616805d700 1 -- start start 2023-09-30T22:19:25.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f9522104700 1 Processor -- start 2023-09-30T22:19:25.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f9522104700 1 -- start start 2023-09-30T22:19:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f616805d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f616805d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6160134170 0x7f616013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f616805d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616009edd0 con 0x7f61600a1340 2023-09-30T22:19:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f616805d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616009e1f0 con 0x7f6160134170 2023-09-30T22:19:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f616805d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f616009dc50 con 0x7f61600a0a30 2023-09-30T22:19:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.592+0000 7f6165df9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f6165df9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46802/0 (socket says 172.21.15.89:46802) 2023-09-30T22:19:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f6165df9700 1 -- 172.21.15.89:0/3058397939 learned_addr learned my addr 172.21.15.89:0/3058397939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f9522104700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c131810 0x7f951c1c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f9522104700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951c134630 0x7f951c1c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f9522104700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951c136f50 0x7f951c1c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f9522104700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c09e5d0 con 0x7f951c134630 2023-09-30T22:19:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f9522104700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c136ab0 con 0x7f951c131810 2023-09-30T22:19:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f9522104700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f951c11a3a0 con 0x7f951c136f50 2023-09-30T22:19:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951c134630 0x7f951c1c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951c134630 0x7f951c1c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41014/0 (socket says 172.21.15.89:41014) 2023-09-30T22:19:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951affd700 1 -- 172.21.15.89:0/3966095642 learned_addr learned my addr 172.21.15.89:0/3966095642 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951b7fe700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c131810 0x7f951c1c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951bfff700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951c136f50 0x7f951c1c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f61665fa700 1 --2- 172.21.15.89:0/3058397939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6160134170 0x7f616013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f6165df9700 1 -- 172.21.15.89:0/3058397939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6160134170 msgr2=0x7f616013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f6165df9700 1 --2- 172.21.15.89:0/3058397939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6160134170 0x7f616013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f6165df9700 1 -- 172.21.15.89:0/3058397939 >> v1:172.21.15.89:6789/0 conn(0x7f61600a1340 legacy=0x7f6160133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f6165df9700 1 -- 172.21.15.89:0/3058397939 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61600a1fe0 con 0x7f61600a0a30 2023-09-30T22:19:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f6165df9700 1 --2- 172.21.15.89:0/3058397939 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61600a0a30 0x7f61600a0e00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f615c009920 tx=0x7f615c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1bb15273bea697b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f6164df7700 1 -- 172.21.15.89:0/3058397939 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c004230 con 0x7f61600a0a30 2023-09-30T22:19:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f6164df7700 1 -- 172.21.15.89:0/3058397939 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c004390 con 0x7f61600a0a30 2023-09-30T22:19:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f6164df7700 1 -- 172.21.15.89:0/3058397939 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c009e70 con 0x7f61600a0a30 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951affd700 1 -- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951c136f50 msgr2=0x7f951c1c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951affd700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951c136f50 0x7f951c1c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951affd700 1 -- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c131810 msgr2=0x7f951c1c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951affd700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c131810 0x7f951c1c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.593+0000 7f951affd700 1 -- 172.21.15.89:0/3966095642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514009b30 con 0x7f951c134630 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f951affd700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951c134630 0x7f951c1c4660 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f951000b500 tx=0x7f951000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f9503fff700 1 -- 172.21.15.89:0/3966095642 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510014d70 con 0x7f951c134630 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f9503fff700 1 -- 172.21.15.89:0/3966095642 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951001c440 con 0x7f951c134630 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f9503fff700 1 -- 172.21.15.89:0/3966095642 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951001c7d0 con 0x7f951c134630 2023-09-30T22:19:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c06eb00 con 0x7f951c134630 2023-09-30T22:19:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c119220 con 0x7f951c134630 2023-09-30T22:19:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f616805d700 1 -- 172.21.15.89:0/3058397939 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61600a0a30 msgr2=0x7f61600a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f616805d700 1 --2- 172.21.15.89:0/3058397939 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61600a0a30 0x7f61600a0e00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f615c009920 tx=0x7f615c009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f616805d700 1 -- 172.21.15.89:0/3058397939 shutdown_connections 2023-09-30T22:19:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f616805d700 1 --2- 172.21.15.89:0/3058397939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6160134170 0x7f616013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.594+0000 7f616805d700 1 --2- 172.21.15.89:0/3058397939 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61600a0a30 0x7f61600a0e00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.595+0000 7f616805d700 1 -- 172.21.15.89:0/3058397939 >> 172.21.15.89:0/3058397939 conn(0x7f61600a6550 msgr2=0x7f61600a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.595+0000 7f616805d700 1 -- 172.21.15.89:0/3058397939 shutdown_connections 2023-09-30T22:19:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.595+0000 7f616805d700 1 -- 172.21.15.89:0/3058397939 wait complete. 2023-09-30T22:19:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.595+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c06ee10 con 0x7f951c134630 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f616805d700 1 Processor -- start 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f616805d700 1 -- start start 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f616805d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61600a0a30 0x7f616011e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f616805d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61600a1340 0x7f616011ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f616805d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6160134170 0x7f616011f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f616805d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61600a0220 con 0x7f61600a0a30 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f616805d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6160120420 con 0x7f61600a1340 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f616805d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6160121f40 con 0x7f6160134170 2023-09-30T22:19:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f61665fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6160134170 0x7f616011f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:25.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f61665fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6160134170 0x7f616011f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46826/0 (socket says 172.21.15.89:46826) 2023-09-30T22:19:25.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f61665fa700 1 -- 172.21.15.89:0/1690202858 learned_addr learned my addr 172.21.15.89:0/1690202858 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:25.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.597+0000 7f9503fff700 1 -- 172.21.15.89:0/3966095642 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f95100253b0 con 0x7f951c134630 2023-09-30T22:19:25.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f61665fa700 1 -- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61600a1340 msgr2=0x7f616011ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f61665fa700 1 --2- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61600a1340 0x7f616011ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f61665fa700 1 -- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61600a0a30 msgr2=0x7f616011e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f61665fa700 1 --2- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61600a0a30 0x7f616011e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.596+0000 7f61665fa700 1 -- 172.21.15.89:0/1690202858 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c009550 con 0x7f6160134170 2023-09-30T22:19:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.597+0000 7f61665fa700 1 --2- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6160134170 0x7f616011f180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6158007e40 tx=0x7f615800e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.597+0000 7f9503fff700 1 -- 172.21.15.89:0/3966095642 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f95100474b0 con 0x7f951c134630 2023-09-30T22:19:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.597+0000 7f61567fc700 1 -- 172.21.15.89:0/1690202858 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800eea0 con 0x7f6160134170 2023-09-30T22:19:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.597+0000 7f61567fc700 1 -- 172.21.15.89:0/1690202858 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6158014420 con 0x7f6160134170 2023-09-30T22:19:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.597+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61600a25e0 con 0x7f6160134170 2023-09-30T22:19:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.597+0000 7f61567fc700 1 -- 172.21.15.89:0/1690202858 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61580147b0 con 0x7f6160134170 2023-09-30T22:19:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.597+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6160120ad0 con 0x7f6160134170 2023-09-30T22:19:25.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.598+0000 7f61567fc700 1 -- 172.21.15.89:0/1690202858 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6158014910 con 0x7f6160134170 2023-09-30T22:19:25.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.598+0000 7f61567fc700 1 -- 172.21.15.89:0/1690202858 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f615803ffd0 con 0x7f6160134170 2023-09-30T22:19:25.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.599+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6160074ea0 con 0x7f6160134170 2023-09-30T22:19:25.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.606+0000 7f9503fff700 1 -- 172.21.15.89:0/3966095642 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9510024360 con 0x7f951c134630 2023-09-30T22:19:25.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.607+0000 7f61567fc700 1 -- 172.21.15.89:0/1690202858 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f61580122e0 con 0x7f6160134170 2023-09-30T22:19:25.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.788+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4b540713a0 con 0x7f4b54130260 2023-09-30T22:19:25.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.788+0000 7f4b3bfff700 1 -- 172.21.15.89:0/2685237404 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b4c018020 con 0x7f4b54130260 2023-09-30T22:19:25.788 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:25.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b54130260 msgr2=0x7f4b541c3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b54130260 0x7f4b541c3f90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c015000 tx=0x7f4b4c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 shutdown_connections 2023-09-30T22:19:25.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b54130260 0x7f4b541c3f90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b5412cf60 0x7f4b5409b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 --2- 172.21.15.89:0/2685237404 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b5412f8b0 0x7f4b5409be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 >> 172.21.15.89:0/2685237404 conn(0x7f4b540a2aa0 msgr2=0x7f4b5406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:25.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 shutdown_connections 2023-09-30T22:19:25.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.791+0000 7f4b59aea700 1 -- 172.21.15.89:0/2685237404 wait complete. 2023-09-30T22:19:25.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.798+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f951c074ea0 con 0x7f951c134630 2023-09-30T22:19:25.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.799+0000 7f9503fff700 1 -- 172.21.15.89:0/3966095642 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9510021030 con 0x7f951c134630 2023-09-30T22:19:25.798 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951c134630 msgr2=0x7f951c1c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951c134630 0x7f951c1c4660 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f951000b500 tx=0x7f951000b810 comp rx=0 tx=0).stop 2023-09-30T22:19:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 shutdown_connections 2023-09-30T22:19:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f951c136f50 0x7f951c1c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f951c131810 0x7f951c1c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 --2- 172.21.15.89:0/3966095642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f951c134630 0x7f951c1c4660 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 >> 172.21.15.89:0/3966095642 conn(0x7f951c0a2aa0 msgr2=0x7f951c06f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 shutdown_connections 2023-09-30T22:19:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.801+0000 7f9522104700 1 -- 172.21.15.89:0/3966095642 wait complete. 2023-09-30T22:19:25.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:25.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:25.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.834+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9918120e10 con 0x7f991809e760 2023-09-30T22:19:25.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.835+0000 7f990dffb700 1 -- 172.21.15.89:0/199138013 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9918120e10 con 0x7f991809e760 2023-09-30T22:19:25.835 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:25.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809e760 msgr2=0x7f991811dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 --2- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809e760 0x7f991811dc50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9908009870 tx=0x7f9908004750 comp rx=0 tx=0).stop 2023-09-30T22:19:25.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 shutdown_connections 2023-09-30T22:19:25.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 --2- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99181338a0 0x7f991811e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 --2- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991809de50 0x7f991811d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 --2- 172.21.15.89:0/199138013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991809e760 0x7f991811dc50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 >> 172.21.15.89:0/199138013 conn(0x7f99180a2aa0 msgr2=0x7f99181bac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:25.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 shutdown_connections 2023-09-30T22:19:25.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.837+0000 7f991f77c700 1 -- 172.21.15.89:0/199138013 wait complete. 2023-09-30T22:19:25.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:25.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.859+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f61600a28e0 con 0x7f6160134170 2023-09-30T22:19:25.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.860+0000 7f61567fc700 1 -- 172.21.15.89:0/1690202858 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f615801bb00 con 0x7f6160134170 2023-09-30T22:19:25.860 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:25.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6160134170 msgr2=0x7f616011f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:25.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 --2- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6160134170 0x7f616011f180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6158007e40 tx=0x7f615800e3c0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 shutdown_connections 2023-09-30T22:19:25.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 --2- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6160134170 0x7f616011f180 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 --2- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61600a0a30 0x7f616011e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 --2- 172.21.15.89:0/1690202858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61600a1340 0x7f616011ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:25.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 >> 172.21.15.89:0/1690202858 conn(0x7f61600a6550 msgr2=0x7f61601b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:25.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 shutdown_connections 2023-09-30T22:19:25.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:25.862+0000 7f616805d700 1 -- 172.21.15.89:0/1690202858 wait complete. 2023-09-30T22:19:25.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:26.811 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:26.814 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:26.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:26.875 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:26.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.956+0000 7f02eb574700 1 Processor -- start 2023-09-30T22:19:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02eb574700 1 -- start start 2023-09-30T22:19:26.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02eb574700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4136820 0x7f02e4138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02eb574700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4139140 0x7f02e413b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02eb574700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e406bc00 con 0x7f02e4133a80 2023-09-30T22:19:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02eb574700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e406c570 con 0x7f02e4139140 2023-09-30T22:19:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02eb574700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40a0040 con 0x7f02e4136820 2023-09-30T22:19:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02e9310700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f02e4133a80 0x7f02e4133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55134/0 (socket says 172.21.15.89:55134) 2023-09-30T22:19:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02e9310700 1 -- 172.21.15.89:0/5074494 learned_addr learned my addr 172.21.15.89:0/5074494 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.957+0000 7f02e8b0f700 1 --2- 172.21.15.89:0/5074494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4136820 0x7f02e4138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:26.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02dbfff700 1 -- 172.21.15.89:0/5074494 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715464272 0 0) 0x7f02e406bc00 con 0x7f02e4133a80 2023-09-30T22:19:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02dbfff700 1 -- 172.21.15.89:0/5074494 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d00035b0 con 0x7f02e4133a80 2023-09-30T22:19:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02e8b0f700 1 -- 172.21.15.89:0/5074494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4139140 msgr2=0x7f02e413b520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:19:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02e8b0f700 1 --2- 172.21.15.89:0/5074494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4139140 0x7f02e413b520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02e8b0f700 1 -- 172.21.15.89:0/5074494 >> v1:172.21.15.89:6789/0 conn(0x7f02e4133a80 legacy=0x7f02e4133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02e8b0f700 1 -- 172.21.15.89:0/5074494 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4133860 con 0x7f02e4136820 2023-09-30T22:19:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02e8b0f700 1 --2- 172.21.15.89:0/5074494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4136820 0x7f02e4138c00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009870 tx=0x7f02d4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d268c56d6f538278 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02dbfff700 1 -- 172.21.15.89:0/5074494 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4011830 con 0x7f02e4136820 2023-09-30T22:19:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02dbfff700 1 -- 172.21.15.89:0/5074494 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d4011990 con 0x7f02e4136820 2023-09-30T22:19:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.958+0000 7f02dbfff700 1 -- 172.21.15.89:0/5074494 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4011d20 con 0x7f02e4136820 2023-09-30T22:19:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.959+0000 7f02eb574700 1 -- 172.21.15.89:0/5074494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4136820 msgr2=0x7f02e4138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.959+0000 7f02eb574700 1 --2- 172.21.15.89:0/5074494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4136820 0x7f02e4138c00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009870 tx=0x7f02d4009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.960+0000 7f02eb574700 1 -- 172.21.15.89:0/5074494 shutdown_connections 2023-09-30T22:19:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.960+0000 7f02eb574700 1 --2- 172.21.15.89:0/5074494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4139140 0x7f02e413b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.960+0000 7f02eb574700 1 --2- 172.21.15.89:0/5074494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4136820 0x7f02e4138c00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.960+0000 7f02eb574700 1 -- 172.21.15.89:0/5074494 >> 172.21.15.89:0/5074494 conn(0x7f02e40a4210 msgr2=0x7f02e40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.960+0000 7f02eb574700 1 -- 172.21.15.89:0/5074494 shutdown_connections 2023-09-30T22:19:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.960+0000 7f02eb574700 1 -- 172.21.15.89:0/5074494 wait complete. 2023-09-30T22:19:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.961+0000 7f02eb574700 1 Processor -- start 2023-09-30T22:19:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.961+0000 7f02eb574700 1 -- start start 2023-09-30T22:19:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02eb574700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4133a80 0x7f02e41c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02eb574700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02e4136820 0x7f02e41c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02eb574700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4139140 0x7f02e41c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02eb574700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e409d040 con 0x7f02e4136820 2023-09-30T22:19:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02eb574700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e409c980 con 0x7f02e4133a80 2023-09-30T22:19:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02eb574700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02e41364c0 con 0x7f02e4139140 2023-09-30T22:19:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9310700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4133a80 0x7f02e41c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9310700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4133a80 0x7f02e41c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46378/0 (socket says 172.21.15.89:46378) 2023-09-30T22:19:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9310700 1 -- 172.21.15.89:0/1827460432 learned_addr learned my addr 172.21.15.89:0/1827460432 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e8b0f700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02e4136820 0x7f02e41c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9b11700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4139140 0x7f02e41c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9310700 1 -- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4139140 msgr2=0x7f02e41c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9310700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4139140 0x7f02e41c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9310700 1 -- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02e4136820 msgr2=0x7f02e41c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9310700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02e4136820 0x7f02e41c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e9310700 1 -- 172.21.15.89:0/1827460432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40094a0 con 0x7f02e4133a80 2023-09-30T22:19:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.962+0000 7f02e8b0f700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02e4136820 0x7f02e41c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.963+0000 7f02e9310700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4133a80 0x7f02e41c8330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c700 tx=0x7f02e000ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.963+0000 7fa42c335700 1 Processor -- start 2023-09-30T22:19:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.963+0000 7fa42c335700 1 -- start start 2023-09-30T22:19:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42c335700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa424071b40 0x7fa424130420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42c335700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130960 0x7fa424133570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42c335700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa424099ef0 con 0x7fa424130d50 2023-09-30T22:19:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42c335700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa424099500 con 0x7fa424071b40 2023-09-30T22:19:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42c335700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa424099c10 con 0x7fa424130960 2023-09-30T22:19:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7f02d9ffb700 1 -- 172.21.15.89:0/1827460432 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0005d70 con 0x7f02e4133a80 2023-09-30T22:19:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7f02d9ffb700 1 -- 172.21.15.89:0/1827460432 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e00070d0 con 0x7f02e4133a80 2023-09-30T22:19:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e409b740 con 0x7f02e4133a80 2023-09-30T22:19:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7f02d9ffb700 1 -- 172.21.15.89:0/1827460432 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0007460 con 0x7f02e4133a80 2023-09-30T22:19:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42a0d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa424071b40 0x7fa424130420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42a0d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa424071b40 0x7fa424130420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41052/0 (socket says 172.21.15.89:41052) 2023-09-30T22:19:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42a0d1700 1 -- 172.21.15.89:0/2104127038 learned_addr learned my addr 172.21.15.89:0/2104127038 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:26.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4071e20 con 0x7f02e4133a80 2023-09-30T22:19:26.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42a0d1700 1 -- 172.21.15.89:0/2104127038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130960 msgr2=0x7fa424133570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:26.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42a0d1700 1 --2- 172.21.15.89:0/2104127038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130960 0x7fa424133570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42a0d1700 1 -- 172.21.15.89:0/2104127038 >> v1:172.21.15.89:6789/0 conn(0x7fa424130d50 legacy=0x7fa424133ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.964+0000 7fa42a0d1700 1 -- 172.21.15.89:0/2104127038 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42409e370 con 0x7fa424071b40 2023-09-30T22:19:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.965+0000 7fa42a0d1700 1 --2- 172.21.15.89:0/2104127038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa424071b40 0x7fa424130420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa4200098f0 tx=0x7fa420009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a3917ae2bb05cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.965+0000 7f02d9ffb700 1 -- 172.21.15.89:0/1827460432 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02e00075c0 con 0x7f02e4133a80 2023-09-30T22:19:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.965+0000 7f02d9ffb700 1 -- 172.21.15.89:0/1827460432 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f02e003eb10 con 0x7f02e4133a80 2023-09-30T22:19:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.965+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e4074ea0 con 0x7f02e4133a80 2023-09-30T22:19:26.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa4290cf700 1 -- 172.21.15.89:0/2104127038 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420004230 con 0x7fa424071b40 2023-09-30T22:19:26.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa4290cf700 1 -- 172.21.15.89:0/2104127038 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa420004390 con 0x7fa424071b40 2023-09-30T22:19:26.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa4290cf700 1 -- 172.21.15.89:0/2104127038 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4200036a0 con 0x7fa424071b40 2023-09-30T22:19:26.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa42c335700 1 -- 172.21.15.89:0/2104127038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa424071b40 msgr2=0x7fa424130420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:26.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa42c335700 1 --2- 172.21.15.89:0/2104127038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa424071b40 0x7fa424130420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa4200098f0 tx=0x7fa420009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa42c335700 1 -- 172.21.15.89:0/2104127038 shutdown_connections 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa42c335700 1 --2- 172.21.15.89:0/2104127038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa424071b40 0x7fa424130420 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa42c335700 1 --2- 172.21.15.89:0/2104127038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130960 0x7fa424133570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa42c335700 1 -- 172.21.15.89:0/2104127038 >> 172.21.15.89:0/2104127038 conn(0x7fa42409e7e0 msgr2=0x7fa42409ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.966+0000 7fa42c335700 1 -- 172.21.15.89:0/2104127038 shutdown_connections 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.967+0000 7fa42c335700 1 -- 172.21.15.89:0/2104127038 wait complete. 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.968+0000 7fa42c335700 1 Processor -- start 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.968+0000 7fa42c335700 1 -- start start 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.968+0000 7fa42c335700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa424071b40 0x7fa42412ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.968+0000 7fa42c335700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa424130960 0x7fa42412d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.968+0000 7fa42c335700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130d50 0x7fa42412d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.968+0000 7fa42c335700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42412e8a0 con 0x7fa424071b40 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.968+0000 7fa42c335700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42408b110 con 0x7fa424130d50 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.968+0000 7fa42c335700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa42408bb30 con 0x7fa424130960 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa4298d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa424130960 0x7fa42412d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa42a8d2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130d50 0x7fa42412d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa4298d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa424130960 0x7fa42412d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46870/0 (socket says 172.21.15.89:46870) 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa4298d0700 1 -- 172.21.15.89:0/2096073488 learned_addr learned my addr 172.21.15.89:0/2096073488 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa42a8d2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130d50 0x7fa42412d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46386/0 (socket says 172.21.15.89:46386) 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa42a8d2700 1 -- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa424130960 msgr2=0x7fa42412d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa42a8d2700 1 --2- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa424130960 0x7fa42412d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa42a8d2700 1 -- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa424071b40 msgr2=0x7fa42412ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa42a8d2700 1 --2- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa424071b40 0x7fa42412ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa42a8d2700 1 -- 172.21.15.89:0/2096073488 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009550 con 0x7fa424130d50 2023-09-30T22:19:26.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa4298d0700 1 --2- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa424130960 0x7fa42412d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:26.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.969+0000 7fa42a8d2700 1 --2- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130d50 0x7fa42412d8f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa42409dff0 tx=0x7fa41c00cdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:26.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.974+0000 7fa41affd700 1 -- 172.21.15.89:0/2096073488 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00a980 con 0x7fa424130d50 2023-09-30T22:19:26.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.974+0000 7fa41affd700 1 -- 172.21.15.89:0/2096073488 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c00aae0 con 0x7fa424130d50 2023-09-30T22:19:26.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.974+0000 7f02d9ffb700 1 -- 172.21.15.89:0/1827460432 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02e0019d20 con 0x7f02e4133a80 2023-09-30T22:19:26.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.974+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42409c150 con 0x7fa424130d50 2023-09-30T22:19:26.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.974+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42412f090 con 0x7fa424130d50 2023-09-30T22:19:26.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.974+0000 7fa41affd700 1 -- 172.21.15.89:0/2096073488 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00ae90 con 0x7fa424130d50 2023-09-30T22:19:26.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.975+0000 7fa41affd700 1 -- 172.21.15.89:0/2096073488 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa41c012420 con 0x7fa424130d50 2023-09-30T22:19:26.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.975+0000 7fa41affd700 1 -- 172.21.15.89:0/2096073488 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa41c03fd90 con 0x7fa424130d50 2023-09-30T22:19:26.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.975+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c0052e0 con 0x7fa424130d50 2023-09-30T22:19:26.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:26.983+0000 7fa41affd700 1 -- 172.21.15.89:0/2096073488 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa41c0228f0 con 0x7fa424130d50 2023-09-30T22:19:27.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.002+0000 7f16e173b700 1 Processor -- start 2023-09-30T22:19:27.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.002+0000 7f16e173b700 1 -- start start 2023-09-30T22:19:27.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16e173b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc06a770 0x7f16dc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16e173b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16e173b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16dc09a810 con 0x7f16dc09dc30 2023-09-30T22:19:27.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16e173b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc06ab40 con 0x7f16dc13d1f0 2023-09-30T22:19:27.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16e173b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc09ebc0 con 0x7f16dc06a770 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16daffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc06a770 0x7f16dc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16db7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16daffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc06a770 0x7f16dc09d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46886/0 (socket says 172.21.15.89:46886) 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16daffd700 1 -- 172.21.15.89:0/680328147 learned_addr learned my addr 172.21.15.89:0/680328147 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16daffd700 1 -- 172.21.15.89:0/680328147 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 msgr2=0x7f16dc13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16d9ffb700 1 -- 172.21.15.89:0/680328147 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238375213 0 0) 0x7f16dc09a810 con 0x7f16dc09dc30 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16daffd700 1 --2- 172.21.15.89:0/680328147 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16daffd700 1 -- 172.21.15.89:0/680328147 >> v1:172.21.15.89:6789/0 conn(0x7f16dc09dc30 legacy=0x7f16dc09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.003+0000 7f16daffd700 1 -- 172.21.15.89:0/680328147 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc09e920 con 0x7f16dc06a770 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16daffd700 1 --2- 172.21.15.89:0/680328147 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc06a770 0x7f16dc09d6f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009920 tx=0x7f16cc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c155b591beb3475c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16d9ffb700 1 -- 172.21.15.89:0/680328147 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc011830 con 0x7f16dc06a770 2023-09-30T22:19:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16d9ffb700 1 -- 172.21.15.89:0/680328147 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc011990 con 0x7f16dc06a770 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16d9ffb700 1 -- 172.21.15.89:0/680328147 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc011d20 con 0x7f16dc06a770 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16e173b700 1 -- 172.21.15.89:0/680328147 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc06a770 msgr2=0x7f16dc09d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16e173b700 1 --2- 172.21.15.89:0/680328147 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc06a770 0x7f16dc09d6f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009920 tx=0x7f16cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16e173b700 1 -- 172.21.15.89:0/680328147 shutdown_connections 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16e173b700 1 --2- 172.21.15.89:0/680328147 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16e173b700 1 --2- 172.21.15.89:0/680328147 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc06a770 0x7f16dc09d6f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16e173b700 1 -- 172.21.15.89:0/680328147 >> 172.21.15.89:0/680328147 conn(0x7f16dc066f50 msgr2=0x7f16dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.004+0000 7f16e173b700 1 -- 172.21.15.89:0/680328147 shutdown_connections 2023-09-30T22:19:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.005+0000 7f16e173b700 1 -- 172.21.15.89:0/680328147 wait complete. 2023-09-30T22:19:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16e173b700 1 Processor -- start 2023-09-30T22:19:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16e173b700 1 -- start start 2023-09-30T22:19:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16e173b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16dc09dc30 0x7f16dc071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16e173b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16e173b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc13bb30 0x7f16dc13bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16e173b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc11eac0 con 0x7f16dc09dc30 2023-09-30T22:19:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16e173b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc11e2f0 con 0x7f16dc13d1f0 2023-09-30T22:19:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16e173b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc11f160 con 0x7f16dc13bb30 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16daffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16dc09dc30 0x7f16dc071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16daffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16dc09dc30 0x7f16dc071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41074/0 (socket says 172.21.15.89:41074) 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16daffd700 1 -- 172.21.15.89:0/431803738 learned_addr learned my addr 172.21.15.89:0/431803738 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16db7fe700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc13bb30 0x7f16dc13bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.006+0000 7f16da7fc700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16daffd700 1 -- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc13bb30 msgr2=0x7f16dc13bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16daffd700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc13bb30 0x7f16dc13bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16daffd700 1 -- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 msgr2=0x7f16dc13b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16daffd700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16daffd700 1 -- 172.21.15.89:0/431803738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc009550 con 0x7f16dc09dc30 2023-09-30T22:19:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16da7fc700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16db7fe700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc13bb30 0x7f16dc13bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16daffd700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16dc09dc30 0x7f16dc071590 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0098f0 tx=0x7f16cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16c37fe700 1 -- 172.21.15.89:0/431803738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc004240 con 0x7f16dc09dc30 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc072190 con 0x7f16dc09dc30 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16c37fe700 1 -- 172.21.15.89:0/431803738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc01f070 con 0x7f16dc09dc30 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16c37fe700 1 -- 172.21.15.89:0/431803738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0045e0 con 0x7f16dc09dc30 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.007+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc13cd80 con 0x7f16dc09dc30 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.008+0000 7f16c37fe700 1 -- 172.21.15.89:0/431803738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f16cc021030 con 0x7f16dc09dc30 2023-09-30T22:19:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.008+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16dc074ea0 con 0x7f16dc09dc30 2023-09-30T22:19:27.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.008+0000 7f16c37fe700 1 -- 172.21.15.89:0/431803738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f16cc0407b0 con 0x7f16dc09dc30 2023-09-30T22:19:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.017+0000 7f16c37fe700 1 -- 172.21.15.89:0/431803738 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16cc019980 con 0x7f16dc09dc30 2023-09-30T22:19:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.038+0000 7fd653de1700 1 Processor -- start 2023-09-30T22:19:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.038+0000 7fd653de1700 1 -- start start 2023-09-30T22:19:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd653de1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd64c133420 0x7fd64c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd653de1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd653de1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd653de1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c06b990 con 0x7fd64c133420 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd653de1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0a0590 con 0x7fd64c134260 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd653de1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c09fb80 con 0x7fd64c133e30 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46906/0 (socket says 172.21.15.89:46906) 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 -- 172.21.15.89:0/172037996 learned_addr learned my addr 172.21.15.89:0/172037996 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd651b7d700 1 --2- 172.21.15.89:0/172037996 >> v2:172.21.15.89:3300/0 conn(0x7fd64c133420 0x7fd64c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65237e700 1 --2- 172.21.15.89:0/172037996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 -- 172.21.15.89:0/172037996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 msgr2=0x7fd64c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:27.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 --2- 172.21.15.89:0/172037996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 -- 172.21.15.89:0/172037996 >> v2:172.21.15.89:3300/0 conn(0x7fd64c133420 msgr2=0x7fd64c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 --2- 172.21.15.89:0/172037996 >> v2:172.21.15.89:3300/0 conn(0x7fd64c133420 0x7fd64c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 -- 172.21.15.89:0/172037996 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0a0030 con 0x7fd64c133e30 2023-09-30T22:19:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd651b7d700 1 --2- 172.21.15.89:0/172037996 >> v2:172.21.15.89:3300/0 conn(0x7fd64c133420 0x7fd64c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.039+0000 7fd65137c700 1 --2- 172.21.15.89:0/172037996 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c12ff90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0098f0 tx=0x7fd63c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb185fee2e60e0c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd650b7b700 1 -- 172.21.15.89:0/172037996 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c011830 con 0x7fd64c133e30 2023-09-30T22:19:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd650b7b700 1 -- 172.21.15.89:0/172037996 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c011990 con 0x7fd64c133e30 2023-09-30T22:19:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd650b7b700 1 -- 172.21.15.89:0/172037996 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c011d20 con 0x7fd64c133e30 2023-09-30T22:19:27.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd653de1700 1 -- 172.21.15.89:0/172037996 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 msgr2=0x7fd64c12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd653de1700 1 --2- 172.21.15.89:0/172037996 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c12ff90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0098f0 tx=0x7fd63c009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd653de1700 1 -- 172.21.15.89:0/172037996 shutdown_connections 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd653de1700 1 --2- 172.21.15.89:0/172037996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd653de1700 1 --2- 172.21.15.89:0/172037996 >> v2:172.21.15.89:3300/0 conn(0x7fd64c133420 0x7fd64c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd653de1700 1 --2- 172.21.15.89:0/172037996 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c12ff90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.040+0000 7fd653de1700 1 -- 172.21.15.89:0/172037996 >> 172.21.15.89:0/172037996 conn(0x7fd64c0a4210 msgr2=0x7fd64c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.041+0000 7fd653de1700 1 -- 172.21.15.89:0/172037996 shutdown_connections 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.041+0000 7fd653de1700 1 -- 172.21.15.89:0/172037996 wait complete. 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.042+0000 7fd653de1700 1 Processor -- start 2023-09-30T22:19:27.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.042+0000 7fd653de1700 1 -- start start 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.042+0000 7fd653de1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64c133420 0x7fd64c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd653de1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd653de1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd653de1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c12d9e0 con 0x7fd64c133420 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd653de1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c11f4e0 con 0x7fd64c134260 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd653de1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c120230 con 0x7fd64c133e30 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd651b7d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64c133420 0x7fd64c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c09cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46414/0 (socket says 172.21.15.89:46414) 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 -- 172.21.15.89:0/1140686300 learned_addr learned my addr 172.21.15.89:0/1140686300 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65137c700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 -- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 msgr2=0x7fd64c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 -- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64c133420 msgr2=0x7fd64c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64c133420 0x7fd64c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 -- 172.21.15.89:0/1140686300 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480094a0 con 0x7fd64c134260 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65137c700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.043+0000 7fd65237e700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c09cf00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd644009e30 tx=0x7fd64400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.044+0000 7fd6427fc700 1 -- 172.21.15.89:0/1140686300 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644009ca0 con 0x7fd64c134260 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.044+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c009550 con 0x7fd64c134260 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.044+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c11f8c0 con 0x7fd64c134260 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.044+0000 7fd6427fc700 1 -- 172.21.15.89:0/1140686300 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64400f040 con 0x7fd64c134260 2023-09-30T22:19:27.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.044+0000 7fd6427fc700 1 -- 172.21.15.89:0/1140686300 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644014480 con 0x7fd64c134260 2023-09-30T22:19:27.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.045+0000 7fd6427fc700 1 -- 172.21.15.89:0/1140686300 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd644014660 con 0x7fd64c134260 2023-09-30T22:19:27.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.045+0000 7fd6427fc700 1 -- 172.21.15.89:0/1140686300 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd6440401f0 con 0x7fd64c134260 2023-09-30T22:19:27.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.045+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6340052e0 con 0x7fd64c134260 2023-09-30T22:19:27.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.054+0000 7fd6427fc700 1 -- 172.21.15.89:0/1140686300 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd644007920 con 0x7fd64c134260 2023-09-30T22:19:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.175+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f02e40720d0 con 0x7f02e4133a80 2023-09-30T22:19:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.175+0000 7f02d9ffb700 1 -- 172.21.15.89:0/1827460432 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02e00192c0 con 0x7f02e4133a80 2023-09-30T22:19:27.175 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.177+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4133a80 msgr2=0x7f02e41c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.177+0000 7f02eb574700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4133a80 0x7f02e41c8330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c700 tx=0x7f02e000ca10 comp rx=0 tx=0).stop 2023-09-30T22:19:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.177+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 shutdown_connections 2023-09-30T22:19:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.177+0000 7f02eb574700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02e4139140 0x7f02e41c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.177+0000 7f02eb574700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02e4133a80 0x7f02e41c8330 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.177+0000 7f02eb574700 1 --2- 172.21.15.89:0/1827460432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02e4136820 0x7f02e41c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.177+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 >> 172.21.15.89:0/1827460432 conn(0x7f02e40a4210 msgr2=0x7f02e4134600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.178+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 shutdown_connections 2023-09-30T22:19:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.178+0000 7f02eb574700 1 -- 172.21.15.89:0/1827460432 wait complete. 2023-09-30T22:19:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.186+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa40c005590 con 0x7fa424130d50 2023-09-30T22:19:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.187+0000 7fa41affd700 1 -- 172.21.15.89:0/2096073488 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa41c029070 con 0x7fa424130d50 2023-09-30T22:19:27.187 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:27.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130d50 msgr2=0x7fa42412d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 --2- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130d50 0x7fa42412d8f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa42409dff0 tx=0x7fa41c00cdc0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 shutdown_connections 2023-09-30T22:19:27.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 --2- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa424130d50 0x7fa42412d8f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 --2- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa424071b40 0x7fa42412ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 --2- 172.21.15.89:0/2096073488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa424130960 0x7fa42412d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 >> 172.21.15.89:0/2096073488 conn(0x7fa42409e7e0 msgr2=0x7fa42409c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 shutdown_connections 2023-09-30T22:19:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.189+0000 7fa42c335700 1 -- 172.21.15.89:0/2096073488 wait complete. 2023-09-30T22:19:27.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:27.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:27.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.213+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f16dc09a3d0 con 0x7f16dc09dc30 2023-09-30T22:19:27.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.214+0000 7f16c37fe700 1 -- 172.21.15.89:0/431803738 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16cc015020 con 0x7f16dc09dc30 2023-09-30T22:19:27.214 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:27.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16dc09dc30 msgr2=0x7f16dc071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16dc09dc30 0x7f16dc071590 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0098f0 tx=0x7f16cc004750 comp rx=0 tx=0).stop 2023-09-30T22:19:27.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 shutdown_connections 2023-09-30T22:19:27.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16dc13bb30 0x7f16dc13bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16dc09dc30 0x7f16dc071590 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 --2- 172.21.15.89:0/431803738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16dc13d1f0 0x7f16dc13b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 >> 172.21.15.89:0/431803738 conn(0x7f16dc066f50 msgr2=0x7f16dc09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:27.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 shutdown_connections 2023-09-30T22:19:27.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.216+0000 7f16e173b700 1 -- 172.21.15.89:0/431803738 wait complete. 2023-09-30T22:19:27.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:27.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.248+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd634005590 con 0x7fd64c134260 2023-09-30T22:19:27.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.249+0000 7fd6427fc700 1 -- 172.21.15.89:0/1140686300 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd64402b090 con 0x7fd64c134260 2023-09-30T22:19:27.248 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:27.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 msgr2=0x7fd64c09cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:27.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c09cf00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd644009e30 tx=0x7fd64400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 shutdown_connections 2023-09-30T22:19:27.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64c134260 0x7fd64c09cf00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64c133420 0x7fd64c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 --2- 172.21.15.89:0/1140686300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64c133e30 0x7fd64c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:27.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 >> 172.21.15.89:0/1140686300 conn(0x7fd64c0a4210 msgr2=0x7fd64c1302a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:27.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 shutdown_connections 2023-09-30T22:19:27.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:27.251+0000 7fd653de1700 1 -- 172.21.15.89:0/1140686300 wait complete. 2023-09-30T22:19:27.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:28.191 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:28.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:28.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:28.264 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:28.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.345+0000 7f99aeb43700 1 Processor -- start 2023-09-30T22:19:28.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99aeb43700 1 -- start start 2023-09-30T22:19:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99aeb43700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99aeb43700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a809eca0 0x7f99a806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99aeb43700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a8070ee0 con 0x7f99a8134610 2023-09-30T22:19:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99aeb43700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8071480 con 0x7f99a809eca0 2023-09-30T22:19:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99aeb43700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99a806f3c0 con 0x7f99a806e570 2023-09-30T22:19:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99ad0e0700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f99a8134610 0x7f99a8134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55168/0 (socket says 172.21.15.89:55168) 2023-09-30T22:19:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99ad0e0700 1 -- 172.21.15.89:0/750264236 learned_addr learned my addr 172.21.15.89:0/750264236 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.346+0000 7f99a7fff700 1 --2- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a809eca0 0x7f99a806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99ac8df700 1 --2- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a7fff700 1 -- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 msgr2=0x7f99a806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a7fff700 1 --2- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a77fe700 1 -- 172.21.15.89:0/750264236 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51895696 0 0) 0x7f99a8070ee0 con 0x7f99a8134610 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a7fff700 1 -- 172.21.15.89:0/750264236 >> v1:172.21.15.89:6789/0 conn(0x7f99a8134610 legacy=0x7f99a8134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a7fff700 1 -- 172.21.15.89:0/750264236 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8071b90 con 0x7f99a809eca0 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a7fff700 1 --2- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a809eca0 0x7f99a806ee80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9994009870 tx=0x7f9994009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ac8a5a56acba0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a77fe700 1 -- 172.21.15.89:0/750264236 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994011830 con 0x7f99a809eca0 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a77fe700 1 -- 172.21.15.89:0/750264236 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9994011990 con 0x7f99a809eca0 2023-09-30T22:19:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.347+0000 7f99a77fe700 1 -- 172.21.15.89:0/750264236 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994011d20 con 0x7f99a809eca0 2023-09-30T22:19:28.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.348+0000 7f99aeb43700 1 -- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a809eca0 msgr2=0x7f99a806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.348+0000 7f99aeb43700 1 --2- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a809eca0 0x7f99a806ee80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9994009870 tx=0x7f9994009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.348+0000 7f99aeb43700 1 -- 172.21.15.89:0/750264236 shutdown_connections 2023-09-30T22:19:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.348+0000 7f99aeb43700 1 --2- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.348+0000 7f99aeb43700 1 --2- 172.21.15.89:0/750264236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a809eca0 0x7f99a806ee80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.348+0000 7f99aeb43700 1 -- 172.21.15.89:0/750264236 >> 172.21.15.89:0/750264236 conn(0x7f99a80a2aa0 msgr2=0x7f99a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.348+0000 7f99aeb43700 1 -- 172.21.15.89:0/750264236 shutdown_connections 2023-09-30T22:19:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.348+0000 7f99aeb43700 1 -- 172.21.15.89:0/750264236 wait complete. 2023-09-30T22:19:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.349+0000 7f99aeb43700 1 Processor -- start 2023-09-30T22:19:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.350+0000 7f99aeb43700 1 -- start start 2023-09-30T22:19:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.350+0000 7f99aeb43700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a81214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.350+0000 7f99aeb43700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8134610 0x7f99a8121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.350+0000 7f99aeb43700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a809e450 0x7f99a8121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.350+0000 7f99aeb43700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a81229b0 con 0x7f99a809e450 2023-09-30T22:19:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.350+0000 7f99aeb43700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8122fb0 con 0x7f99a8134610 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.350+0000 7f99aeb43700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8123be0 con 0x7f99a806e570 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99ac8df700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a81214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99ac8df700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a81214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46936/0 (socket says 172.21.15.89:46936) 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99ac8df700 1 -- 172.21.15.89:0/2615929775 learned_addr learned my addr 172.21.15.89:0/2615929775 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a7fff700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8134610 0x7f99a8121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99ad0e0700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a809e450 0x7f99a8121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a7fff700 1 -- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 msgr2=0x7f99a81214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a7fff700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a81214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a7fff700 1 -- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a809e450 msgr2=0x7f99a8121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a7fff700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a809e450 0x7f99a8121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a7fff700 1 -- 172.21.15.89:0/2615929775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940094a0 con 0x7f99a8134610 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a7fff700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8134610 0x7f99a8121a20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f99a8071f90 tx=0x7f9994004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a57fa700 1 -- 172.21.15.89:0/2615929775 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994004510 con 0x7f99a8134610 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a57fa700 1 -- 172.21.15.89:0/2615929775 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999401f070 con 0x7f99a8134610 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a80a2240 con 0x7f99a8134610 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.351+0000 7f99a57fa700 1 -- 172.21.15.89:0/2615929775 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940038e0 con 0x7f99a8134610 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.352+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a8124030 con 0x7f99a8134610 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.352+0000 7f99a57fa700 1 -- 172.21.15.89:0/2615929775 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9994003a40 con 0x7f99a8134610 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.353+0000 7f99a57fa700 1 -- 172.21.15.89:0/2615929775 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9994021070 con 0x7f99a8134610 2023-09-30T22:19:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.353+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a809b1e0 con 0x7f99a8134610 2023-09-30T22:19:28.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.358+0000 7f99a57fa700 1 -- 172.21.15.89:0/2615929775 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9994015080 con 0x7f99a8134610 2023-09-30T22:19:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.373+0000 7fcf29942700 1 Processor -- start 2023-09-30T22:19:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.373+0000 7fcf29942700 1 -- start start 2023-09-30T22:19:28.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf29942700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 0x7fcf24131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf29942700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf24099a40 0x7fcf24099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf29942700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf241313c0 con 0x7fcf2409a3b0 2023-09-30T22:19:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf29942700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2409b580 con 0x7fcf24131810 2023-09-30T22:19:28.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf29942700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2406e8c0 con 0x7fcf24099a40 2023-09-30T22:19:28.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf24099a40 0x7fcf24099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf24099a40 0x7fcf24099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46944/0 (socket says 172.21.15.89:46944) 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf237fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcf2409a3b0 0x7fcf24138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55174/0 (socket says 172.21.15.89:55174) 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf227fc700 1 -- 172.21.15.89:0/2593437024 learned_addr learned my addr 172.21.15.89:0/2593437024 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf22ffd700 1 --2- 172.21.15.89:0/2593437024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 0x7fcf24131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf227fc700 1 -- 172.21.15.89:0/2593437024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 msgr2=0x7fcf24131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf21ffb700 1 -- 172.21.15.89:0/2593437024 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887706208 0 0) 0x7fcf241313c0 con 0x7fcf2409a3b0 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf227fc700 1 --2- 172.21.15.89:0/2593437024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 0x7fcf24131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf227fc700 1 -- 172.21.15.89:0/2593437024 >> v1:172.21.15.89:6789/0 conn(0x7fcf2409a3b0 legacy=0x7fcf24138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf227fc700 1 -- 172.21.15.89:0/2593437024 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2409a820 con 0x7fcf24099a40 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.374+0000 7fcf227fc700 1 --2- 172.21.15.89:0/2593437024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf24099a40 0x7fcf24099e70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009920 tx=0x7fcf18009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de2b300d521640c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf21ffb700 1 -- 172.21.15.89:0/2593437024 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800ba60 con 0x7fcf24099a40 2023-09-30T22:19:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf21ffb700 1 -- 172.21.15.89:0/2593437024 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1800bbc0 con 0x7fcf24099a40 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf21ffb700 1 -- 172.21.15.89:0/2593437024 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18009e70 con 0x7fcf24099a40 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf29942700 1 -- 172.21.15.89:0/2593437024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf24099a40 msgr2=0x7fcf24099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf29942700 1 --2- 172.21.15.89:0/2593437024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf24099a40 0x7fcf24099e70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009920 tx=0x7fcf18009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf29942700 1 -- 172.21.15.89:0/2593437024 shutdown_connections 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf29942700 1 --2- 172.21.15.89:0/2593437024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 0x7fcf24131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf29942700 1 --2- 172.21.15.89:0/2593437024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf24099a40 0x7fcf24099e70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf29942700 1 -- 172.21.15.89:0/2593437024 >> 172.21.15.89:0/2593437024 conn(0x7fcf240a2aa0 msgr2=0x7fcf240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf29942700 1 -- 172.21.15.89:0/2593437024 shutdown_connections 2023-09-30T22:19:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.375+0000 7fcf29942700 1 -- 172.21.15.89:0/2593437024 wait complete. 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.376+0000 7fcf29942700 1 Processor -- start 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.376+0000 7fcf29942700 1 -- start start 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf29942700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf24099a40 0x7fcf2406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf29942700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf2409a3b0 0x7fcf2406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf29942700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 0x7fcf24070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf29942700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2411eff0 con 0x7fcf24099a40 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf29942700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2406eb00 con 0x7fcf24131810 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf29942700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2411f2d0 con 0x7fcf2409a3b0 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf2409a3b0 0x7fcf2406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf2409a3b0 0x7fcf2406fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46946/0 (socket says 172.21.15.89:46946) 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf227fc700 1 -- 172.21.15.89:0/96173459 learned_addr learned my addr 172.21.15.89:0/96173459 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf24099a40 0x7fcf2406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf237fe700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 0x7fcf24070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf22ffd700 1 -- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf2409a3b0 msgr2=0x7fcf2406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf22ffd700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf2409a3b0 0x7fcf2406fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf22ffd700 1 -- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 msgr2=0x7fcf24070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.377+0000 7fcf22ffd700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 0x7fcf24070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7fcf22ffd700 1 -- 172.21.15.89:0/96173459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf18009550 con 0x7fcf24099a40 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7fcf227fc700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf2409a3b0 0x7fcf2406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7f0659f6a700 1 Processor -- start 2023-09-30T22:19:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7fcf22ffd700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf24099a40 0x7fcf2406f610 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400e7e0 tx=0x7fcf1400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7f0659f6a700 1 -- start start 2023-09-30T22:19:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7fcf0bfff700 1 -- 172.21.15.89:0/96173459 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400c070 con 0x7fcf24099a40 2023-09-30T22:19:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7fcf0bfff700 1 -- 172.21.15.89:0/96173459 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf14013610 con 0x7fcf24099a40 2023-09-30T22:19:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7fcf0bfff700 1 -- 172.21.15.89:0/96173459 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140139a0 con 0x7fcf24099a40 2023-09-30T22:19:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf241307c0 con 0x7fcf24099a40 2023-09-30T22:19:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2411dc20 con 0x7fcf24099a40 2023-09-30T22:19:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7fcf0bfff700 1 -- 172.21.15.89:0/96173459 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf14013b00 con 0x7fcf24099a40 2023-09-30T22:19:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7f0659f6a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06541316c0 0x7f0654131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.378+0000 7f0659f6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f065409d9f0 0x7f065406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f0659f6a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065409e130 con 0x7f065406acb0 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f0659f6a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06541312b0 con 0x7f06541316c0 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f0659f6a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f065409f6e0 con 0x7f065409d9f0 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f0652ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f065409d9f0 0x7f065406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f0652ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f065409d9f0 0x7f065406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46960/0 (socket says 172.21.15.89:46960) 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f0652ffd700 1 -- 172.21.15.89:0/3776104848 learned_addr learned my addr 172.21.15.89:0/3776104848 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f06537fe700 1 --2- 172.21.15.89:0/3776104848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06541316c0 0x7f0654131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f06537fe700 1 -- 172.21.15.89:0/3776104848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f065409d9f0 msgr2=0x7f065406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f06537fe700 1 --2- 172.21.15.89:0/3776104848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f065409d9f0 0x7f065406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f06537fe700 1 -- 172.21.15.89:0/3776104848 >> v1:172.21.15.89:6789/0 conn(0x7f065406acb0 legacy=0x7f0654138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:28.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f06537fe700 1 -- 172.21.15.89:0/3776104848 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065409e760 con 0x7f06541316c0 2023-09-30T22:19:28.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7fcf0bfff700 1 -- 172.21.15.89:0/96173459 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcf1403f4e0 con 0x7fcf24099a40 2023-09-30T22:19:28.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.379+0000 7f06537fe700 1 --2- 172.21.15.89:0/3776104848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06541316c0 0x7f0654131a90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0644009870 tx=0x7f0644009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b43758a26a7a038 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:28.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2411ee50 con 0x7fcf24099a40 2023-09-30T22:19:28.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f06527fc700 1 -- 172.21.15.89:0/3776104848 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644004230 con 0x7f06541316c0 2023-09-30T22:19:28.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f06527fc700 1 -- 172.21.15.89:0/3776104848 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0644004390 con 0x7f06541316c0 2023-09-30T22:19:28.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f06527fc700 1 -- 172.21.15.89:0/3776104848 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440036a0 con 0x7f06541316c0 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f0659f6a700 1 -- 172.21.15.89:0/3776104848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06541316c0 msgr2=0x7f0654131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f0659f6a700 1 --2- 172.21.15.89:0/3776104848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06541316c0 0x7f0654131a90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0644009870 tx=0x7f0644009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f0659f6a700 1 -- 172.21.15.89:0/3776104848 shutdown_connections 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f0659f6a700 1 --2- 172.21.15.89:0/3776104848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06541316c0 0x7f0654131a90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f0659f6a700 1 --2- 172.21.15.89:0/3776104848 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f065409d9f0 0x7f065406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f0659f6a700 1 -- 172.21.15.89:0/3776104848 >> 172.21.15.89:0/3776104848 conn(0x7f0654066f50 msgr2=0x7f065409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f0659f6a700 1 -- 172.21.15.89:0/3776104848 shutdown_connections 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.380+0000 7f0659f6a700 1 -- 172.21.15.89:0/3776104848 wait complete. 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.382+0000 7f0659f6a700 1 Processor -- start 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.382+0000 7f0659f6a700 1 -- start start 2023-09-30T22:19:28.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.382+0000 7f0659f6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f065406acb0 0x7f0654119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.382+0000 7f0659f6a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f065409d9f0 0x7f06541199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.382+0000 7f0659f6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06541316c0 0x7f0654119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.382+0000 7f0659f6a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065411c550 con 0x7f065406acb0 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.382+0000 7f0659f6a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06540a3540 con 0x7f065409d9f0 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.382+0000 7f0659f6a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06540722f0 con 0x7f06541316c0 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f065406acb0 0x7f0654119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f065406acb0 0x7f0654119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41122/0 (socket says 172.21.15.89:41122) 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 -- 172.21.15.89:0/683519372 learned_addr learned my addr 172.21.15.89:0/683519372 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f0652ffd700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f065409d9f0 0x7f06541199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f0653fff700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06541316c0 0x7f0654119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 -- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06541316c0 msgr2=0x7f0654119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06541316c0 0x7f0654119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 -- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f065409d9f0 msgr2=0x7f06541199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f065409d9f0 0x7f06541199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 -- 172.21.15.89:0/683519372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440094a0 con 0x7f065406acb0 2023-09-30T22:19:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.383+0000 7f06537fe700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f065406acb0 0x7f0654119480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0644009840 tx=0x7f0644004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.385+0000 7fcf0bfff700 1 -- 172.21.15.89:0/96173459 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf2411ee50 con 0x7fcf24099a40 2023-09-30T22:19:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.385+0000 7f063bfff700 1 -- 172.21.15.89:0/683519372 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644011830 con 0x7f065406acb0 2023-09-30T22:19:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.385+0000 7f063bfff700 1 -- 172.21.15.89:0/683519372 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064401f070 con 0x7f065406acb0 2023-09-30T22:19:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.385+0000 7f063bfff700 1 -- 172.21.15.89:0/683519372 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644011bf0 con 0x7f065406acb0 2023-09-30T22:19:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.385+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065412b960 con 0x7f065406acb0 2023-09-30T22:19:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.385+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06540718c0 con 0x7f065406acb0 2023-09-30T22:19:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.386+0000 7f063bfff700 1 -- 172.21.15.89:0/683519372 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0644011d50 con 0x7f065406acb0 2023-09-30T22:19:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.387+0000 7f063bfff700 1 -- 172.21.15.89:0/683519372 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0644021070 con 0x7f065406acb0 2023-09-30T22:19:28.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.387+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065409e130 con 0x7f065406acb0 2023-09-30T22:19:28.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.396+0000 7f063bfff700 1 -- 172.21.15.89:0/683519372 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f065409e130 con 0x7f065406acb0 2023-09-30T22:19:28.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.438+0000 7fc7584d1700 1 Processor -- start 2023-09-30T22:19:28.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.438+0000 7fc7584d1700 1 -- start start 2023-09-30T22:19:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc7584d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc75009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc7584d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f240 0x7fc7501317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc7584d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc75009f650 0x7fc750131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc7584d1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc750070940 con 0x7fc75009f650 2023-09-30T22:19:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc7584d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75006fd60 con 0x7fc75009f240 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc7584d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc75006fa80 con 0x7fc75009e930 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc75626d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc75009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc755a6c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f240 0x7fc7501317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc75626d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc75009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46972/0 (socket says 172.21.15.89:46972) 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc75626d700 1 -- 172.21.15.89:0/292312445 learned_addr learned my addr 172.21.15.89:0/292312445 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc756a6e700 1 --2- 172.21.15.89:0/292312445 >> v2:172.21.15.89:3300/0 conn(0x7fc75009f650 0x7fc750131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc756a6e700 1 -- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 msgr2=0x7fc75009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc756a6e700 1 --2- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc75009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc756a6e700 1 -- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f240 msgr2=0x7fc7501317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc756a6e700 1 --2- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f240 0x7fc7501317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc756a6e700 1 -- 172.21.15.89:0/292312445 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75009fb60 con 0x7fc75009f650 2023-09-30T22:19:28.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc755a6c700 1 --2- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f240 0x7fc7501317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:28.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.439+0000 7fc756a6e700 1 --2- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f650 0x7fc750131d30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc748009d90 tx=0x7fc74800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcb5badf8b5ebd4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:28.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.440+0000 7fc75526b700 1 -- 172.21.15.89:0/292312445 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800e070 con 0x7fc75009f650 2023-09-30T22:19:28.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.440+0000 7fc75526b700 1 -- 172.21.15.89:0/292312445 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74800bb20 con 0x7fc75009f650 2023-09-30T22:19:28.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.440+0000 7fc75526b700 1 -- 172.21.15.89:0/292312445 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748012430 con 0x7fc75009f650 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.440+0000 7fc7584d1700 1 -- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f650 msgr2=0x7fc750131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.441+0000 7fc7584d1700 1 --2- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f650 0x7fc750131d30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc748009d90 tx=0x7fc74800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.441+0000 7fc7584d1700 1 -- 172.21.15.89:0/292312445 shutdown_connections 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.441+0000 7fc7584d1700 1 --2- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f650 0x7fc750131d30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.441+0000 7fc7584d1700 1 --2- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc75009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.441+0000 7fc7584d1700 1 --2- 172.21.15.89:0/292312445 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f240 0x7fc7501317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.441+0000 7fc7584d1700 1 -- 172.21.15.89:0/292312445 >> 172.21.15.89:0/292312445 conn(0x7fc7500a4210 msgr2=0x7fc7500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.441+0000 7fc7584d1700 1 -- 172.21.15.89:0/292312445 shutdown_connections 2023-09-30T22:19:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.441+0000 7fc7584d1700 1 -- 172.21.15.89:0/292312445 wait complete. 2023-09-30T22:19:28.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.442+0000 7fc7584d1700 1 Processor -- start 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.442+0000 7fc7584d1700 1 -- start start 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc7584d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc7501d5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc7584d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f240 0x7fc7501d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc7584d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f650 0x7fc7501d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc7584d1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7501b7740 con 0x7fc75009f240 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc7584d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc750071d20 con 0x7fc75009f650 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc7584d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7501b7220 con 0x7fc75009e930 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f240 0x7fc7501d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f240 0x7fc7501d58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41128/0 (socket says 172.21.15.89:41128) 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 -- 172.21.15.89:0/2975113174 learned_addr learned my addr 172.21.15.89:0/2975113174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc756a6e700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f650 0x7fc7501d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc75626d700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc7501d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 -- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 msgr2=0x7fc7501d5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc7501d5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 -- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f650 msgr2=0x7fc7501d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f650 0x7fc7501d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 -- 172.21.15.89:0/2975113174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7400094a0 con 0x7fc75009f240 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc75626d700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc7501d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.443+0000 7fc755a6c700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f240 0x7fc7501d58a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc74000ebb0 tx=0x7fc74000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.444+0000 7fc746ffd700 1 -- 172.21.15.89:0/2975113174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74000c850 con 0x7fc75009f240 2023-09-30T22:19:28.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.444+0000 7fc746ffd700 1 -- 172.21.15.89:0/2975113174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74000c9b0 con 0x7fc75009f240 2023-09-30T22:19:28.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.444+0000 7fc746ffd700 1 -- 172.21.15.89:0/2975113174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74000cd40 con 0x7fc75009f240 2023-09-30T22:19:28.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.444+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc748009a70 con 0x7fc75009f240 2023-09-30T22:19:28.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.444+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7500a0190 con 0x7fc75009f240 2023-09-30T22:19:28.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.445+0000 7fc746ffd700 1 -- 172.21.15.89:0/2975113174 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc740013470 con 0x7fc75009f240 2023-09-30T22:19:28.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.445+0000 7fc746ffd700 1 -- 172.21.15.89:0/2975113174 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc74003f950 con 0x7fc75009f240 2023-09-30T22:19:28.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.445+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc750074ea0 con 0x7fc75009f240 2023-09-30T22:19:28.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.454+0000 7fc746ffd700 1 -- 172.21.15.89:0/2975113174 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc740021080 con 0x7fc75009f240 2023-09-30T22:19:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.589+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcf2411e0c0 con 0x7fcf24099a40 2023-09-30T22:19:28.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.589+0000 7fcf0bfff700 1 -- 172.21.15.89:0/96173459 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf1401c400 con 0x7fcf24099a40 2023-09-30T22:19:28.589 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:28.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf24099a40 msgr2=0x7fcf2406f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf24099a40 0x7fcf2406f610 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400e7e0 tx=0x7fcf1400eaf0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 shutdown_connections 2023-09-30T22:19:28.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf24131810 0x7fcf24070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf24099a40 0x7fcf2406f610 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 --2- 172.21.15.89:0/96173459 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf2409a3b0 0x7fcf2406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 >> 172.21.15.89:0/96173459 conn(0x7fcf240a2aa0 msgr2=0x7fcf2412e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:28.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 shutdown_connections 2023-09-30T22:19:28.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.591+0000 7fcf29942700 1 -- 172.21.15.89:0/96173459 wait complete. 2023-09-30T22:19:28.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.592+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f065409e130 con 0x7f065406acb0 2023-09-30T22:19:28.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.593+0000 7f063bfff700 1 -- 172.21.15.89:0/683519372 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f065409e130 con 0x7f065406acb0 2023-09-30T22:19:28.593 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.594+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f065406acb0 msgr2=0x7f0654119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.594+0000 7f0659f6a700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f065406acb0 0x7f0654119480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0644009840 tx=0x7f0644004750 comp rx=0 tx=0).stop 2023-09-30T22:19:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.595+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 shutdown_connections 2023-09-30T22:19:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.595+0000 7f0659f6a700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06541316c0 0x7f0654119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.595+0000 7f0659f6a700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f065406acb0 0x7f0654119480 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.595+0000 7f0659f6a700 1 --2- 172.21.15.89:0/683519372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f065409d9f0 0x7f06541199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.595+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 >> 172.21.15.89:0/683519372 conn(0x7f0654066f50 msgr2=0x7f06540a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:28.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.595+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 shutdown_connections 2023-09-30T22:19:28.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.595+0000 7f0659f6a700 1 -- 172.21.15.89:0/683519372 wait complete. 2023-09-30T22:19:28.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.604+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f99a8070610 con 0x7f99a8134610 2023-09-30T22:19:28.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.605+0000 7f99a57fa700 1 -- 172.21.15.89:0/2615929775 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9994019ba0 con 0x7f99a8134610 2023-09-30T22:19:28.604 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8134610 msgr2=0x7f99a8121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8134610 0x7f99a8121a20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f99a8071f90 tx=0x7f9994004750 comp rx=0 tx=0).stop 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 shutdown_connections 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a809e450 0x7f99a8121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99a806e570 0x7f99a81214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 --2- 172.21.15.89:0/2615929775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99a8134610 0x7f99a8121a20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 >> 172.21.15.89:0/2615929775 conn(0x7f99a80a2aa0 msgr2=0x7f99a81bf160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 shutdown_connections 2023-09-30T22:19:28.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.607+0000 7f99aeb43700 1 -- 172.21.15.89:0/2615929775 wait complete. 2023-09-30T22:19:28.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:28.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:28.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.670+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc750131050 con 0x7fc75009f240 2023-09-30T22:19:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.670+0000 7fc746ffd700 1 -- 172.21.15.89:0/2975113174 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7400232f0 con 0x7fc75009f240 2023-09-30T22:19:28.670 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f240 msgr2=0x7fc7501d58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f240 0x7fc7501d58a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc74000ebb0 tx=0x7fc74000eec0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 shutdown_connections 2023-09-30T22:19:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc75009f650 0x7fc7501d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc75009e930 0x7fc7501d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 --2- 172.21.15.89:0/2975113174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc75009f240 0x7fc7501d58a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 >> 172.21.15.89:0/2975113174 conn(0x7fc7500a4210 msgr2=0x7fc75006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 shutdown_connections 2023-09-30T22:19:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:28.672+0000 7fc7584d1700 1 -- 172.21.15.89:0/2975113174 wait complete. 2023-09-30T22:19:28.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:29.609 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:29.610 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:29.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:29.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:29.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f1eb5837700 1 Processor -- start 2023-09-30T22:19:29.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f1eb5837700 1 -- start start 2023-09-30T22:19:29.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f62f61c7700 1 Processor -- start 2023-09-30T22:19:29.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f62f61c7700 1 -- start start 2023-09-30T22:19:29.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f1eb5837700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f1eb5837700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1eb009eca0 0x7f1eb006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f1eb5837700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb0070ee0 con 0x7f1eb0134610 2023-09-30T22:19:29.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f1eb5837700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb0071480 con 0x7f1eb009eca0 2023-09-30T22:19:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.755+0000 7f1eb5837700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb006f3c0 con 0x7f1eb006e570 2023-09-30T22:19:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46496/0 (socket says 172.21.15.89:46496) 2023-09-30T22:19:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f1eaeffd700 1 -- 172.21.15.89:0/3469281077 learned_addr learned my addr 172.21.15.89:0/3469281077 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f1eae7fc700 1 --2- 172.21.15.89:0/3469281077 >> v2:172.21.15.89:3300/0 conn(0x7f1eb009eca0 0x7f1eb006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62f61c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f0136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62f61c7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f62f0136f50 0x7f62f0139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62f61c7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f0071a00 con 0x7f62f0131630 2023-09-30T22:19:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62f61c7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f0131430 con 0x7f62f0136f50 2023-09-30T22:19:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62f61c7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f009a510 con 0x7f62f0134630 2023-09-30T22:19:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f1eae7fc700 1 -- 172.21.15.89:0/3469281077 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 msgr2=0x7f1eb006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f1eae7fc700 1 --2- 172.21.15.89:0/3469281077 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f1eae7fc700 1 -- 172.21.15.89:0/3469281077 >> v1:172.21.15.89:6789/0 conn(0x7f1eb0134610 legacy=0x7f1eb0134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f1eae7fc700 1 -- 172.21.15.89:0/3469281077 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0071b90 con 0x7f1eb009eca0 2023-09-30T22:19:29.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62effff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f62f0136f50 0x7f62f0139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62effff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f62f0136f50 0x7f62f0139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41156/0 (socket says 172.21.15.89:41156) 2023-09-30T22:19:29.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62effff700 1 -- 172.21.15.89:0/3346584036 learned_addr learned my addr 172.21.15.89:0/3346584036 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:29.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62eeffd700 1 --2- 172.21.15.89:0/3346584036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f0136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62eeffd700 1 -- 172.21.15.89:0/3346584036 >> v2:172.21.15.89:3300/0 conn(0x7f62f0136f50 msgr2=0x7f62f0139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:29.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62eeffd700 1 --2- 172.21.15.89:0/3346584036 >> v2:172.21.15.89:3300/0 conn(0x7f62f0136f50 0x7f62f0139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62eeffd700 1 -- 172.21.15.89:0/3346584036 >> v1:172.21.15.89:6789/0 conn(0x7f62f0131630 legacy=0x7f62f0131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.756+0000 7f62eeffd700 1 -- 172.21.15.89:0/3346584036 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f00721c0 con 0x7f62f0134630 2023-09-30T22:19:29.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.757+0000 7f62eeffd700 1 --2- 172.21.15.89:0/3346584036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f0136a10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009840 tx=0x7f62e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=105e0bb15b2b8840 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:29.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.757+0000 7f1eae7fc700 1 --2- 172.21.15.89:0/3469281077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb009eca0 0x7f1eb006ee80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009840 tx=0x7f1ea4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fb2b4b1fc33623 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:29.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.757+0000 7f62ee7fc700 1 -- 172.21.15.89:0/3346584036 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4011830 con 0x7f62f0134630 2023-09-30T22:19:29.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.757+0000 7f62ee7fc700 1 -- 172.21.15.89:0/3346584036 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e4011990 con 0x7f62f0134630 2023-09-30T22:19:29.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.757+0000 7f62ee7fc700 1 -- 172.21.15.89:0/3346584036 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4011d20 con 0x7f62f0134630 2023-09-30T22:19:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.757+0000 7f1eadffb700 1 -- 172.21.15.89:0/3469281077 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4011830 con 0x7f1eb009eca0 2023-09-30T22:19:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.757+0000 7f1eadffb700 1 -- 172.21.15.89:0/3469281077 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea4011990 con 0x7f1eb009eca0 2023-09-30T22:19:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.757+0000 7f1eadffb700 1 -- 172.21.15.89:0/3469281077 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4011d20 con 0x7f1eb009eca0 2023-09-30T22:19:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f62f61c7700 1 -- 172.21.15.89:0/3346584036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 msgr2=0x7f62f0136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f62f61c7700 1 --2- 172.21.15.89:0/3346584036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f0136a10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009840 tx=0x7f62e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:29.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f1eb5837700 1 -- 172.21.15.89:0/3469281077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb009eca0 msgr2=0x7f1eb006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f1eb5837700 1 --2- 172.21.15.89:0/3469281077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb009eca0 0x7f1eb006ee80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009840 tx=0x7f1ea4009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:29.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f62f61c7700 1 -- 172.21.15.89:0/3346584036 shutdown_connections 2023-09-30T22:19:29.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f62f61c7700 1 --2- 172.21.15.89:0/3346584036 >> v2:172.21.15.89:3300/0 conn(0x7f62f0136f50 0x7f62f0139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f62f61c7700 1 --2- 172.21.15.89:0/3346584036 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f0136a10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f62f61c7700 1 -- 172.21.15.89:0/3346584036 >> 172.21.15.89:0/3346584036 conn(0x7f62f00a2aa0 msgr2=0x7f62f00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:29.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f1eb5837700 1 -- 172.21.15.89:0/3469281077 shutdown_connections 2023-09-30T22:19:29.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f1eb5837700 1 --2- 172.21.15.89:0/3469281077 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f1eb5837700 1 --2- 172.21.15.89:0/3469281077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb009eca0 0x7f1eb006ee80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f1eb5837700 1 -- 172.21.15.89:0/3469281077 >> 172.21.15.89:0/3469281077 conn(0x7f1eb00a2aa0 msgr2=0x7f1eb00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:29.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f62f61c7700 1 -- 172.21.15.89:0/3346584036 shutdown_connections 2023-09-30T22:19:29.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f1eb5837700 1 -- 172.21.15.89:0/3469281077 shutdown_connections 2023-09-30T22:19:29.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f1eb5837700 1 -- 172.21.15.89:0/3469281077 wait complete. 2023-09-30T22:19:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.758+0000 7f62f61c7700 1 -- 172.21.15.89:0/3346584036 wait complete. 2023-09-30T22:19:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.759+0000 7f62f61c7700 1 Processor -- start 2023-09-30T22:19:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.759+0000 7f62f61c7700 1 -- start start 2023-09-30T22:19:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.759+0000 7f1eb5837700 1 Processor -- start 2023-09-30T22:19:29.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.759+0000 7f1eb5837700 1 -- start start 2023-09-30T22:19:29.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62f61c7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62f0131630 0x7f62f01c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62f61c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f01c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62f61c7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62f0136f50 0x7f62f0118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62f61c7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f011c530 con 0x7f62f0131630 2023-09-30T22:19:29.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62f61c7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f012eae0 con 0x7f62f0134630 2023-09-30T22:19:29.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62f61c7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62f011a820 con 0x7f62f0136f50 2023-09-30T22:19:29.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62ef7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62f0131630 0x7f62f01c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62effff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62f0136f50 0x7f62f0118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62effff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62f0136f50 0x7f62f0118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:46988/0 (socket says 172.21.15.89:46988) 2023-09-30T22:19:29.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62ef7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62f0131630 0x7f62f01c62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41172/0 (socket says 172.21.15.89:41172) 2023-09-30T22:19:29.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62effff700 1 -- 172.21.15.89:0/3237801288 learned_addr learned my addr 172.21.15.89:0/3237801288 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:29.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62eeffd700 1 --2- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f01c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f1eb5837700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb01221d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f1eb5837700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1eb0134610 0x7f1eb0122710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f1eb5837700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0122c50 0x7f1eb01cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f1eb5837700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0126850 con 0x7f1eb0122c50 2023-09-30T22:19:29.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f1eb5837700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0070610 con 0x7f1eb006e570 2023-09-30T22:19:29.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f1eb5837700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0125990 con 0x7f1eb0134610 2023-09-30T22:19:29.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62ef7fe700 1 -- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62f0136f50 msgr2=0x7f62f0118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62ef7fe700 1 --2- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62f0136f50 0x7f62f0118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62ef7fe700 1 -- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 msgr2=0x7f62f01c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62ef7fe700 1 --2- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f01c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f62ef7fe700 1 -- 172.21.15.89:0/3237801288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40094a0 con 0x7f62f0131630 2023-09-30T22:19:29.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.760+0000 7f1eae7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1eb0134610 0x7f1eb0122710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eae7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1eb0134610 0x7f1eb0122710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47000/0 (socket says 172.21.15.89:47000) 2023-09-30T22:19:29.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eae7fc700 1 -- 172.21.15.89:0/2991965212 learned_addr learned my addr 172.21.15.89:0/2991965212 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:29.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eaeffd700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb01221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eaf7fe700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0122c50 0x7f1eb01cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eaeffd700 1 -- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1eb0134610 msgr2=0x7f1eb0122710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f62ef7fe700 1 --2- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62f0131630 0x7f62f01c62b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f62e0000f60 tx=0x7f62e000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:29.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eaeffd700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1eb0134610 0x7f1eb0122710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eaeffd700 1 -- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0122c50 msgr2=0x7f1eb01cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eaeffd700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0122c50 0x7f1eb01cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eaeffd700 1 -- 172.21.15.89:0/2991965212 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40094a0 con 0x7f1eb006e570 2023-09-30T22:19:29.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f62d7fff700 1 -- 172.21.15.89:0/3237801288 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0011970 con 0x7f62f0131630 2023-09-30T22:19:29.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f62d7fff700 1 -- 172.21.15.89:0/3237801288 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e0011ad0 con 0x7f62f0131630 2023-09-30T22:19:29.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f1eaeffd700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb01221d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000e8e0 tx=0x7f1ea000ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:29.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f0136d40 con 0x7f62f0131630 2023-09-30T22:19:29.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f62d7fff700 1 -- 172.21.15.89:0/3237801288 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0011e40 con 0x7f62f0131630 2023-09-30T22:19:29.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.761+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f011cec0 con 0x7f62f0131630 2023-09-30T22:19:29.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.762+0000 7f62d7fff700 1 -- 172.21.15.89:0/3237801288 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f62e0005080 con 0x7f62f0131630 2023-09-30T22:19:29.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.762+0000 7f62d7fff700 1 -- 172.21.15.89:0/3237801288 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f62e0042e10 con 0x7f62f0131630 2023-09-30T22:19:29.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.762+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f0074ea0 con 0x7f62f0131630 2023-09-30T22:19:29.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.762+0000 7f1e97fff700 1 -- 172.21.15.89:0/2991965212 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0013680 con 0x7f1eb006e570 2023-09-30T22:19:29.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.762+0000 7f1e97fff700 1 -- 172.21.15.89:0/2991965212 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea00137e0 con 0x7f1eb006e570 2023-09-30T22:19:29.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.762+0000 7f1e97fff700 1 -- 172.21.15.89:0/2991965212 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0013b50 con 0x7f1eb006e570 2023-09-30T22:19:29.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.762+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb00a2440 con 0x7f1eb006e570 2023-09-30T22:19:29.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.762+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb0123cf0 con 0x7f1eb006e570 2023-09-30T22:19:29.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.763+0000 7f1e97fff700 1 -- 172.21.15.89:0/2991965212 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ea0013cb0 con 0x7f1eb006e570 2023-09-30T22:19:29.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.763+0000 7f1e97fff700 1 -- 172.21.15.89:0/2991965212 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1ea003f520 con 0x7f1eb006e570 2023-09-30T22:19:29.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.764+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb00a2440 con 0x7f1eb006e570 2023-09-30T22:19:29.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.768+0000 7f62d7fff700 1 -- 172.21.15.89:0/3237801288 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f62e000ce30 con 0x7f62f0131630 2023-09-30T22:19:29.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.769+0000 7f1e97fff700 1 -- 172.21.15.89:0/2991965212 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1eb00a2440 con 0x7f1eb006e570 2023-09-30T22:19:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.801+0000 7f04e461e700 1 Processor -- start 2023-09-30T22:19:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.801+0000 7f04e461e700 1 -- start start 2023-09-30T22:19:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e461e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04dc09d6f0 0x7f04dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e461e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e461e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04dc09b120 con 0x7f04dc09e000 2023-09-30T22:19:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e461e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04dc06b460 con 0x7f04dc09d6f0 2023-09-30T22:19:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e461e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc09a7e0 con 0x7f04dc09e440 2023-09-30T22:19:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e2bbb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e23ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04dc09d6f0 0x7f04dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e1bb9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f04dc09e000 0x7f04dc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55220/0 (socket says 172.21.15.89:55220) 2023-09-30T22:19:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e1bb9700 1 -- 172.21.15.89:0/885643319 learned_addr learned my addr 172.21.15.89:0/885643319 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e2bbb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47004/0 (socket says 172.21.15.89:47004) 2023-09-30T22:19:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e23ba700 1 -- 172.21.15.89:0/885643319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 msgr2=0x7f04dc134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e23ba700 1 --2- 172.21.15.89:0/885643319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e23ba700 1 -- 172.21.15.89:0/885643319 >> v1:172.21.15.89:6789/0 conn(0x7f04dc09e000 legacy=0x7f04dc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e13b8700 1 -- 172.21.15.89:0/885643319 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515694455 0 0) 0x7f04dc09b120 con 0x7f04dc09e000 2023-09-30T22:19:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.802+0000 7f04e23ba700 1 -- 172.21.15.89:0/885643319 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc09a500 con 0x7f04dc09d6f0 2023-09-30T22:19:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.803+0000 7f04e23ba700 1 --2- 172.21.15.89:0/885643319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09d6f0 0x7f04dc09dac0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009870 tx=0x7f04d8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8a17f8003236f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.803+0000 7f04e13b8700 1 -- 172.21.15.89:0/885643319 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8011830 con 0x7f04dc09d6f0 2023-09-30T22:19:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.803+0000 7f04e13b8700 1 -- 172.21.15.89:0/885643319 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d8011990 con 0x7f04dc09d6f0 2023-09-30T22:19:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.803+0000 7f04e13b8700 1 -- 172.21.15.89:0/885643319 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8011d20 con 0x7f04dc09d6f0 2023-09-30T22:19:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.803+0000 7f04e461e700 1 -- 172.21.15.89:0/885643319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09d6f0 msgr2=0x7f04dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.803+0000 7f04e461e700 1 --2- 172.21.15.89:0/885643319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09d6f0 0x7f04dc09dac0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009870 tx=0x7f04d8009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.804+0000 7f04e461e700 1 -- 172.21.15.89:0/885643319 shutdown_connections 2023-09-30T22:19:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.804+0000 7f04e461e700 1 --2- 172.21.15.89:0/885643319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09d6f0 0x7f04dc09dac0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.804+0000 7f04e461e700 1 --2- 172.21.15.89:0/885643319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.804+0000 7f04e461e700 1 -- 172.21.15.89:0/885643319 >> 172.21.15.89:0/885643319 conn(0x7f04dc066f50 msgr2=0x7f04dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.804+0000 7f04e461e700 1 -- 172.21.15.89:0/885643319 shutdown_connections 2023-09-30T22:19:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.804+0000 7f04e461e700 1 -- 172.21.15.89:0/885643319 wait complete. 2023-09-30T22:19:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e461e700 1 Processor -- start 2023-09-30T22:19:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e461e700 1 -- start start 2023-09-30T22:19:29.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e461e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04dc09d6f0 0x7f04dc1a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e461e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09e000 0x7f04dc1a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e461e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc1c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e461e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a5ce0 con 0x7f04dc09e000 2023-09-30T22:19:29.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e461e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a4d70 con 0x7f04dc09d6f0 2023-09-30T22:19:29.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e461e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a8430 con 0x7f04dc09e440 2023-09-30T22:19:29.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e2bbb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc1c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e1bb9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09e000 0x7f04dc1a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e2bbb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc1c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47018/0 (socket says 172.21.15.89:47018) 2023-09-30T22:19:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.805+0000 7f04e2bbb700 1 -- 172.21.15.89:0/4014165912 learned_addr learned my addr 172.21.15.89:0/4014165912 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04e1bb9700 1 -- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 msgr2=0x7f04dc1c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04e1bb9700 1 --2- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc1c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04e1bb9700 1 -- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04dc09d6f0 msgr2=0x7f04dc1a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04e1bb9700 1 --2- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04dc09d6f0 0x7f04dc1a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04e1bb9700 1 -- 172.21.15.89:0/4014165912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80094a0 con 0x7f04dc09e000 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04e1bb9700 1 --2- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09e000 0x7f04dc1a57a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00c4b0 tx=0x7f04cc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04d2ffd700 1 -- 172.21.15.89:0/4014165912 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc004c80 con 0x7f04dc09e000 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04d2ffd700 1 -- 172.21.15.89:0/4014165912 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc004e00 con 0x7f04dc09e000 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc1a6420 con 0x7f04dc09e000 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04d2ffd700 1 -- 172.21.15.89:0/4014165912 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc007310 con 0x7f04dc09e000 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.806+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc09c710 con 0x7f04dc09e000 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.807+0000 7f04d2ffd700 1 -- 172.21.15.89:0/4014165912 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f04cc007470 con 0x7f04dc09e000 2023-09-30T22:19:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.807+0000 7f04d2ffd700 1 -- 172.21.15.89:0/4014165912 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f04cc03f350 con 0x7f04dc09e000 2023-09-30T22:19:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.807+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc1a7e50 con 0x7f04dc09e000 2023-09-30T22:19:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.812+0000 7f04d2ffd700 1 -- 172.21.15.89:0/4014165912 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f04cc0123b0 con 0x7f04dc09e000 2023-09-30T22:19:29.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.840+0000 7fd9c61f2700 1 Processor -- start 2023-09-30T22:19:29.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.840+0000 7fd9c61f2700 1 -- start start 2023-09-30T22:19:29.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.840+0000 7fd9c61f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0133420 0x7fd9c01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.840+0000 7fd9c61f2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd9c0133e30 0x7fd9c012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.840+0000 7fd9c61f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0134260 0x7fd9c01304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.840+0000 7fd9c61f2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c006b990 con 0x7fd9c0133e30 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.840+0000 7fd9c61f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00a0590 con 0x7fd9c0133420 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.840+0000 7fd9c61f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009fb80 con 0x7fd9c0134260 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0133420 0x7fd9c01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0133420 0x7fd9c01337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46534/0 (socket says 172.21.15.89:46534) 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 -- 172.21.15.89:0/3849966650 learned_addr learned my addr 172.21.15.89:0/3849966650 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bffff700 1 --2- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0134260 0x7fd9c01304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9beffd700 1 --2- 172.21.15.89:0/3849966650 >> v2:172.21.15.89:3300/0 conn(0x7fd9c0133e30 0x7fd9c012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 -- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0134260 msgr2=0x7fd9c01304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 --2- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0134260 0x7fd9c01304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 -- 172.21.15.89:0/3849966650 >> v2:172.21.15.89:3300/0 conn(0x7fd9c0133e30 msgr2=0x7fd9c012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:29.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 --2- 172.21.15.89:0/3849966650 >> v2:172.21.15.89:3300/0 conn(0x7fd9c0133e30 0x7fd9c012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 -- 172.21.15.89:0/3849966650 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00a0030 con 0x7fd9c0133420 2023-09-30T22:19:29.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9beffd700 1 --2- 172.21.15.89:0/3849966650 >> v2:172.21.15.89:3300/0 conn(0x7fd9c0133e30 0x7fd9c012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:29.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bffff700 1 --2- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0134260 0x7fd9c01304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:29.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9bf7fe700 1 --2- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0133420 0x7fd9c01337f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098f0 tx=0x7fd9b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe51e0ba69ae3c6a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:29.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9be7fc700 1 -- 172.21.15.89:0/3849966650 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0011830 con 0x7fd9c0133420 2023-09-30T22:19:29.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9be7fc700 1 -- 172.21.15.89:0/3849966650 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0011990 con 0x7fd9c0133420 2023-09-30T22:19:29.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.841+0000 7fd9be7fc700 1 -- 172.21.15.89:0/3849966650 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0011d20 con 0x7fd9c0133420 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 -- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0133420 msgr2=0x7fd9c01337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 --2- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0133420 0x7fd9c01337f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098f0 tx=0x7fd9b0009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 -- 172.21.15.89:0/3849966650 shutdown_connections 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 --2- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0134260 0x7fd9c01304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 --2- 172.21.15.89:0/3849966650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0133420 0x7fd9c01337f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 --2- 172.21.15.89:0/3849966650 >> v2:172.21.15.89:3300/0 conn(0x7fd9c0133e30 0x7fd9c012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 -- 172.21.15.89:0/3849966650 >> 172.21.15.89:0/3849966650 conn(0x7fd9c00a4210 msgr2=0x7fd9c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 -- 172.21.15.89:0/3849966650 shutdown_connections 2023-09-30T22:19:29.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.842+0000 7fd9c61f2700 1 -- 172.21.15.89:0/3849966650 wait complete. 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.843+0000 7fd9c61f2700 1 Processor -- start 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9c61f2700 1 -- start start 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9c61f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0133420 0x7fd9c009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9c61f2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9c0133e30 0x7fd9c009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9c61f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0134260 0x7fd9c009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9c61f2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009d850 con 0x7fd9c0133e30 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9c61f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0131590 con 0x7fd9c0134260 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9c61f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c011fc40 con 0x7fd9c0133420 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9beffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9c0133e30 0x7fd9c009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0133420 0x7fd9c009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:29.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9beffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9c0133e30 0x7fd9c009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41206/0 (socket says 172.21.15.89:41206) 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9beffd700 1 -- 172.21.15.89:0/239366908 learned_addr learned my addr 172.21.15.89:0/239366908 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9bf7fe700 1 -- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0134260 msgr2=0x7fd9c009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9bf7fe700 1 --2- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0134260 0x7fd9c009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9bf7fe700 1 -- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9c0133e30 msgr2=0x7fd9c009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9bf7fe700 1 --2- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9c0133e30 0x7fd9c009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.844+0000 7fd9bf7fe700 1 -- 172.21.15.89:0/239366908 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8009a70 con 0x7fd9c0133420 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.845+0000 7fd9bf7fe700 1 --2- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0133420 0x7fd9c009c480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0016000 tx=0x7fd9b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.845+0000 7fd9a7fff700 1 -- 172.21.15.89:0/239366908 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0004510 con 0x7fd9c0133420 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.845+0000 7fd9a7fff700 1 -- 172.21.15.89:0/239366908 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b00036a0 con 0x7fd9c0133420 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.845+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b0009550 con 0x7fd9c0133420 2023-09-30T22:19:29.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.845+0000 7fd9a7fff700 1 -- 172.21.15.89:0/239366908 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0003a10 con 0x7fd9c0133420 2023-09-30T22:19:29.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.845+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c012db80 con 0x7fd9c0133420 2023-09-30T22:19:29.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.846+0000 7fd9a7fff700 1 -- 172.21.15.89:0/239366908 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9b0021070 con 0x7fd9c0133420 2023-09-30T22:19:29.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.846+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c011fe60 con 0x7fd9c0133420 2023-09-30T22:19:29.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.846+0000 7fd9a7fff700 1 -- 172.21.15.89:0/239366908 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd9b0048350 con 0x7fd9c0133420 2023-09-30T22:19:29.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.851+0000 7fd9a7fff700 1 -- 172.21.15.89:0/239366908 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9b0045060 con 0x7fd9c0133420 2023-09-30T22:19:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.989+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f62f011c320 con 0x7f62f0131630 2023-09-30T22:19:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.989+0000 7f62d7fff700 1 -- 172.21.15.89:0/3237801288 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f62e00168b0 con 0x7f62f0131630 2023-09-30T22:19:29.989 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.991+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1eb00a2440 con 0x7f1eb006e570 2023-09-30T22:19:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.991+0000 7f1e97fff700 1 -- 172.21.15.89:0/2991965212 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1eb00a2440 con 0x7f1eb006e570 2023-09-30T22:19:29.991 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.992+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62f0131630 msgr2=0x7f62f01c62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.992+0000 7f62f61c7700 1 --2- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62f0131630 0x7f62f01c62b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f62e0000f60 tx=0x7f62e000c730 comp rx=0 tx=0).stop 2023-09-30T22:19:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.992+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 shutdown_connections 2023-09-30T22:19:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.992+0000 7f62f61c7700 1 --2- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62f0136f50 0x7f62f0118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.992+0000 7f62f61c7700 1 --2- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62f0131630 0x7f62f01c62b0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.992+0000 7f62f61c7700 1 --2- 172.21.15.89:0/3237801288 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62f0134630 0x7f62f01c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.992+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 >> 172.21.15.89:0/3237801288 conn(0x7f62f00a2aa0 msgr2=0x7f62f0132410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:29.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 shutdown_connections 2023-09-30T22:19:29.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f62f61c7700 1 -- 172.21.15.89:0/3237801288 wait complete. 2023-09-30T22:19:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 msgr2=0x7f1eb01221d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb01221d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000e8e0 tx=0x7f1ea000ebf0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 shutdown_connections 2023-09-30T22:19:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1eb0122c50 0x7f1eb01cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1eb006e570 0x7f1eb01221d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 --2- 172.21.15.89:0/2991965212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1eb0134610 0x7f1eb0122710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 >> 172.21.15.89:0/2991965212 conn(0x7f1eb00a2aa0 msgr2=0x7f1eb00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:29.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 shutdown_connections 2023-09-30T22:19:29.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:29.993+0000 7f1eb5837700 1 -- 172.21.15.89:0/2991965212 wait complete. 2023-09-30T22:19:30.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:30.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:30.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.010+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f04dc09f420 con 0x7f04dc09e000 2023-09-30T22:19:30.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.010+0000 7f04d2ffd700 1 -- 172.21.15.89:0/4014165912 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04cc01b340 con 0x7f04dc09e000 2023-09-30T22:19:30.010 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:30.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09e000 msgr2=0x7f04dc1a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:30.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 --2- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09e000 0x7f04dc1a57a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00c4b0 tx=0x7f04cc00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:19:30.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 shutdown_connections 2023-09-30T22:19:30.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 --2- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04dc09e440 0x7f04dc1c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:30.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 --2- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04dc09d6f0 0x7f04dc1a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:30.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 --2- 172.21.15.89:0/4014165912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04dc09e000 0x7f04dc1a57a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:30.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 >> 172.21.15.89:0/4014165912 conn(0x7f04dc066f50 msgr2=0x7f04dc06f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:30.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 shutdown_connections 2023-09-30T22:19:30.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.012+0000 7f04e461e700 1 -- 172.21.15.89:0/4014165912 wait complete. 2023-09-30T22:19:30.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.046+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd9c0074ea0 con 0x7fd9c0133420 2023-09-30T22:19:30.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.046+0000 7fd9a7fff700 1 -- 172.21.15.89:0/239366908 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9b0015070 con 0x7fd9c0133420 2023-09-30T22:19:30.046 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:30.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0133420 msgr2=0x7fd9c009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:30.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 --2- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0133420 0x7fd9c009c480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0016000 tx=0x7fd9b0004750 comp rx=0 tx=0).stop 2023-09-30T22:19:30.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 shutdown_connections 2023-09-30T22:19:30.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 --2- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9c0134260 0x7fd9c009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:30.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 --2- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9c0133420 0x7fd9c009c480 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:30.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 --2- 172.21.15.89:0/239366908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9c0133e30 0x7fd9c009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:30.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 >> 172.21.15.89:0/239366908 conn(0x7fd9c00a4210 msgr2=0x7fd9c00a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:30.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 shutdown_connections 2023-09-30T22:19:30.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:30.048+0000 7fd9c61f2700 1 -- 172.21.15.89:0/239366908 wait complete. 2023-09-30T22:19:30.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:31.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:31.009 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:31.024 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:31.061 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:31.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.150+0000 7f8fb56af700 1 Processor -- start 2023-09-30T22:19:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8fb56af700 1 -- start start 2023-09-30T22:19:31.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8fb56af700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8fb009d6f0 0x7f8fb009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8fb56af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 0x7f8fb0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8fb56af700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb009f130 con 0x7f8fb009e000 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8fb56af700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb009f520 con 0x7f8fb009d6f0 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8fb56af700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb009e930 con 0x7f8fb009e410 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8faeffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8fb009d6f0 0x7f8fb009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8faeffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8fb009d6f0 0x7f8fb009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38890/0 (socket says 172.21.15.89:38890) 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8faeffd700 1 -- 172.21.15.89:0/4067877486 learned_addr learned my addr 172.21.15.89:0/4067877486 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.151+0000 7f8faf7fe700 1 --2- 172.21.15.89:0/4067877486 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 0x7f8fb0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.152+0000 7f8faeffd700 1 -- 172.21.15.89:0/4067877486 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 msgr2=0x7f8fb0130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.152+0000 7f8faeffd700 1 --2- 172.21.15.89:0/4067877486 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 0x7f8fb0130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.152+0000 7f8faeffd700 1 -- 172.21.15.89:0/4067877486 >> v1:172.21.15.89:6789/0 conn(0x7f8fb009e000 legacy=0x7f8fb0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.152+0000 7f8faeffd700 1 -- 172.21.15.89:0/4067877486 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb009b2d0 con 0x7f8fb009d6f0 2023-09-30T22:19:31.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.152+0000 7f8faeffd700 1 --2- 172.21.15.89:0/4067877486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb009dac0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00098f0 tx=0x7f8fa0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c53b66666985c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.152+0000 7f8fadffb700 1 -- 172.21.15.89:0/4067877486 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0011830 con 0x7f8fb009d6f0 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.152+0000 7f8fadffb700 1 -- 172.21.15.89:0/4067877486 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa0011990 con 0x7f8fb009d6f0 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.152+0000 7f8fadffb700 1 -- 172.21.15.89:0/4067877486 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0011d20 con 0x7f8fb009d6f0 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.153+0000 7f8fb56af700 1 -- 172.21.15.89:0/4067877486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 msgr2=0x7f8fb009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.153+0000 7f8fb56af700 1 --2- 172.21.15.89:0/4067877486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb009dac0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00098f0 tx=0x7f8fa0009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.153+0000 7f8fb56af700 1 -- 172.21.15.89:0/4067877486 shutdown_connections 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.153+0000 7f8fb56af700 1 --2- 172.21.15.89:0/4067877486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb009dac0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.153+0000 7f8fb56af700 1 --2- 172.21.15.89:0/4067877486 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 0x7f8fb0130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.153+0000 7f8fb56af700 1 -- 172.21.15.89:0/4067877486 >> 172.21.15.89:0/4067877486 conn(0x7f8fb0066f50 msgr2=0x7f8fb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.153+0000 7f8fb56af700 1 -- 172.21.15.89:0/4067877486 shutdown_connections 2023-09-30T22:19:31.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.153+0000 7f8fb56af700 1 -- 172.21.15.89:0/4067877486 wait complete. 2023-09-30T22:19:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.154+0000 7f8fb56af700 1 Processor -- start 2023-09-30T22:19:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.154+0000 7f8fb56af700 1 -- start start 2023-09-30T22:19:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.154+0000 7fab9740f700 1 Processor -- start 2023-09-30T22:19:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.154+0000 7fab9740f700 1 -- start start 2023-09-30T22:19:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8fb56af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb0127f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8fb56af700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fb009e000 0x7f8fb0128470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8fb56af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 0x7f8fb01cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8fb56af700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb012ab70 con 0x7f8fb009d6f0 2023-09-30T22:19:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8fb56af700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0129c40 con 0x7f8fb009e000 2023-09-30T22:19:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8fb56af700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb009c700 con 0x7f8fb009e410 2023-09-30T22:19:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb0127f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb0127f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38906/0 (socket says 172.21.15.89:38906) 2023-09-30T22:19:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faeffd700 1 -- 172.21.15.89:0/889292720 learned_addr learned my addr 172.21.15.89:0/889292720 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:31.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faf7fe700 1 --2- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 0x7f8fb01cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7fab9740f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab90130670 0x7fab90130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faeffd700 1 -- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 msgr2=0x7f8fb01cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faeffd700 1 --2- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 0x7f8fb01cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faeffd700 1 -- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fb009e000 msgr2=0x7f8fb0128470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faeffd700 1 --2- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fb009e000 0x7f8fb0128470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7f8faeffd700 1 -- 172.21.15.89:0/889292720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0009550 con 0x7f8fb009d6f0 2023-09-30T22:19:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7fab951ab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab90130670 0x7fab90130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7fab951ab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab90130670 0x7fab90130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38912/0 (socket says 172.21.15.89:38912) 2023-09-30T22:19:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7fab9740f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab900701a0 0x7fab90138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7fab9740f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab90130260 con 0x7fab9006f680 2023-09-30T22:19:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7fab9740f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9009a350 con 0x7fab90130670 2023-09-30T22:19:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.155+0000 7fab9740f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab90071ff0 con 0x7fab900701a0 2023-09-30T22:19:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab951ab700 1 -- 172.21.15.89:0/2834460070 learned_addr learned my addr 172.21.15.89:0/2834460070 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab959ac700 1 --2- 172.21.15.89:0/2834460070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab900701a0 0x7fab90138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7f8faeffd700 1 --2- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb0127f30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009c40 tx=0x7f8fa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab959ac700 1 -- 172.21.15.89:0/2834460070 >> v2:172.21.15.89:3300/0 conn(0x7fab90130670 msgr2=0x7fab90130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab959ac700 1 --2- 172.21.15.89:0/2834460070 >> v2:172.21.15.89:3300/0 conn(0x7fab90130670 0x7fab90130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab959ac700 1 -- 172.21.15.89:0/2834460070 >> v1:172.21.15.89:6789/0 conn(0x7fab9006f680 legacy=0x7fab9006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab959ac700 1 -- 172.21.15.89:0/2834460070 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9009ecf0 con 0x7fab900701a0 2023-09-30T22:19:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab959ac700 1 --2- 172.21.15.89:0/2834460070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab900701a0 0x7fab90138540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fab88009d90 tx=0x7fab8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20127bf1bac50c81 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab87fff700 1 -- 172.21.15.89:0/2834460070 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8800e070 con 0x7fab900701a0 2023-09-30T22:19:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab87fff700 1 -- 172.21.15.89:0/2834460070 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8800bb20 con 0x7fab900701a0 2023-09-30T22:19:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7fab87fff700 1 -- 172.21.15.89:0/2834460070 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88012430 con 0x7fab900701a0 2023-09-30T22:19:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7f8f977fe700 1 -- 172.21.15.89:0/889292720 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0004480 con 0x7f8fb009d6f0 2023-09-30T22:19:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7f8f977fe700 1 -- 172.21.15.89:0/889292720 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa001f070 con 0x7f8fb009d6f0 2023-09-30T22:19:31.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7f8f977fe700 1 -- 172.21.15.89:0/889292720 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00038c0 con 0x7f8fb009d6f0 2023-09-30T22:19:31.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.156+0000 7f8fb56af700 1 -- 172.21.15.89:0/889292720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb01279c0 con 0x7f8fb009d6f0 2023-09-30T22:19:31.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7f8fb56af700 1 -- 172.21.15.89:0/889292720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb01291f0 con 0x7f8fb009d6f0 2023-09-30T22:19:31.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7fab9740f700 1 -- 172.21.15.89:0/2834460070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab900701a0 msgr2=0x7fab90138540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7fab9740f700 1 --2- 172.21.15.89:0/2834460070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab900701a0 0x7fab90138540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fab88009d90 tx=0x7fab8800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:31.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7fab9740f700 1 -- 172.21.15.89:0/2834460070 shutdown_connections 2023-09-30T22:19:31.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7fab9740f700 1 --2- 172.21.15.89:0/2834460070 >> v2:172.21.15.89:3300/0 conn(0x7fab90130670 0x7fab90130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7fab9740f700 1 --2- 172.21.15.89:0/2834460070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab900701a0 0x7fab90138540 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7fab9740f700 1 -- 172.21.15.89:0/2834460070 >> 172.21.15.89:0/2834460070 conn(0x7fab900a2aa0 msgr2=0x7fab900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:31.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7fab9740f700 1 -- 172.21.15.89:0/2834460070 shutdown_connections 2023-09-30T22:19:31.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.157+0000 7f8f957fa700 1 -- 172.21.15.89:0/889292720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f880052e0 con 0x7f8fb009d6f0 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.158+0000 7fab9740f700 1 -- 172.21.15.89:0/2834460070 wait complete. 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.158+0000 7fab9740f700 1 Processor -- start 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.158+0000 7fab9740f700 1 -- start start 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab9740f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab9006f680 0x7fab90118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab9740f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab900701a0 0x7fab90119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab9740f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab90130670 0x7fab90119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab9740f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9011b840 con 0x7fab90130670 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab9740f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9011ad80 con 0x7fab900701a0 2023-09-30T22:19:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab9740f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab9011c400 con 0x7fab9006f680 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab949aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab900701a0 0x7fab90119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab951ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab9006f680 0x7fab90118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab949aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab900701a0 0x7fab90119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56760/0 (socket says 172.21.15.89:56760) 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab949aa700 1 -- 172.21.15.89:0/2522199088 learned_addr learned my addr 172.21.15.89:0/2522199088 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab951ab700 1 -- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab900701a0 msgr2=0x7fab90119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab951ab700 1 --2- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab900701a0 0x7fab90119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab951ab700 1 -- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab90130670 msgr2=0x7fab90119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab951ab700 1 --2- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab90130670 0x7fab90119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.159+0000 7fab951ab700 1 -- 172.21.15.89:0/2522199088 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88009a70 con 0x7fab9006f680 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.160+0000 7fab949aa700 1 --2- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab900701a0 0x7fab90119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:31.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.160+0000 7fab951ab700 1 --2- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab9006f680 0x7fab90118cd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00b170 tx=0x7fab8c00b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:31.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.160+0000 7fab85ffb700 1 -- 172.21.15.89:0/2522199088 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c019070 con 0x7fab9006f680 2023-09-30T22:19:31.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.160+0000 7fab85ffb700 1 -- 172.21.15.89:0/2522199088 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c00bd60 con 0x7fab9006f680 2023-09-30T22:19:31.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.160+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9009fbc0 con 0x7fab9006f680 2023-09-30T22:19:31.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.160+0000 7fab85ffb700 1 -- 172.21.15.89:0/2522199088 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c01d630 con 0x7fab9006f680 2023-09-30T22:19:31.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.160+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9011c120 con 0x7fab9006f680 2023-09-30T22:19:31.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.161+0000 7fab85ffb700 1 -- 172.21.15.89:0/2522199088 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab8c01d790 con 0x7fab9006f680 2023-09-30T22:19:31.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.162+0000 7fab85ffb700 1 -- 172.21.15.89:0/2522199088 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab8c047320 con 0x7fab9006f680 2023-09-30T22:19:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.162+0000 7f8f977fe700 1 -- 172.21.15.89:0/889292720 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8fa0021030 con 0x7f8fb009d6f0 2023-09-30T22:19:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.162+0000 7f8f977fe700 1 -- 172.21.15.89:0/889292720 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8fa0040e60 con 0x7f8fb009d6f0 2023-09-30T22:19:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.163+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9006f400 con 0x7fab9006f680 2023-09-30T22:19:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.163+0000 7f8f977fe700 1 -- 172.21.15.89:0/889292720 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8fa00199a0 con 0x7f8fb009d6f0 2023-09-30T22:19:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.172+0000 7fab85ffb700 1 -- 172.21.15.89:0/2522199088 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab8c022080 con 0x7fab9006f680 2023-09-30T22:19:31.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.192+0000 7f5136bbc700 1 Processor -- start 2023-09-30T22:19:31.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5136bbc700 1 -- start start 2023-09-30T22:19:31.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5136bbc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5136bbc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5130130430 0x7f5130137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5136bbc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513012cd90 con 0x7f513012f8b0 2023-09-30T22:19:31.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5136bbc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513012d330 con 0x7f5130130430 2023-09-30T22:19:31.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5136bbc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5130071780 con 0x7f513012cf60 2023-09-30T22:19:31.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5134958700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5134958700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41970/0 (socket says 172.21.15.89:41970) 2023-09-30T22:19:31.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.193+0000 7f5134958700 1 -- 172.21.15.89:0/3815742600 learned_addr learned my addr 172.21.15.89:0/3815742600 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:31.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f5135159700 1 --2- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5130130430 0x7f5130137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f5134958700 1 -- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5130130430 msgr2=0x7f5130137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:31.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f5134958700 1 --2- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5130130430 0x7f5130137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f5134958700 1 -- 172.21.15.89:0/3815742600 >> v1:172.21.15.89:6789/0 conn(0x7f513012f8b0 legacy=0x7f513012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f5134958700 1 -- 172.21.15.89:0/3815742600 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130071a40 con 0x7f513012cf60 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f5135159700 1 --2- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5130130430 0x7f5130137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f5134958700 1 --2- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513006f5d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f512400aac0 tx=0x7f512400add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c358e31773ea0429 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f512f7fe700 1 -- 172.21.15.89:0/3815742600 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400cb70 con 0x7f513012cf60 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f512f7fe700 1 -- 172.21.15.89:0/3815742600 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400ccd0 con 0x7f513012cf60 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.194+0000 7f512f7fe700 1 -- 172.21.15.89:0/3815742600 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400f690 con 0x7f513012cf60 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.195+0000 7f5136bbc700 1 -- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 msgr2=0x7f513006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.195+0000 7f5136bbc700 1 --2- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513006f5d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f512400aac0 tx=0x7f512400add0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.195+0000 7f5136bbc700 1 -- 172.21.15.89:0/3815742600 shutdown_connections 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.195+0000 7f5136bbc700 1 --2- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5130130430 0x7f5130137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.195+0000 7f5136bbc700 1 --2- 172.21.15.89:0/3815742600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513006f5d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.195+0000 7f5136bbc700 1 -- 172.21.15.89:0/3815742600 >> 172.21.15.89:0/3815742600 conn(0x7f51300a2aa0 msgr2=0x7f51300a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:31.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.195+0000 7f5136bbc700 1 -- 172.21.15.89:0/3815742600 shutdown_connections 2023-09-30T22:19:31.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.195+0000 7f5136bbc700 1 -- 172.21.15.89:0/3815742600 wait complete. 2023-09-30T22:19:31.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.196+0000 7f5136bbc700 1 Processor -- start 2023-09-30T22:19:31.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.196+0000 7f5136bbc700 1 -- start start 2023-09-30T22:19:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.196+0000 7f5136bbc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513009be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.196+0000 7f5136bbc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f513012f8b0 0x7f513009c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.196+0000 7f5136bbc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5130130430 0x7f51301c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.196+0000 7f5136bbc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513009b980 con 0x7f5130130430 2023-09-30T22:19:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.196+0000 7f5136bbc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513009ebc0 con 0x7f513012f8b0 2023-09-30T22:19:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.196+0000 7f5136bbc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5130129fa0 con 0x7f513012cf60 2023-09-30T22:19:31.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5130130430 0x7f51301c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5130130430 0x7f51301c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38940/0 (socket says 172.21.15.89:38940) 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 -- 172.21.15.89:0/804566031 learned_addr learned my addr 172.21.15.89:0/804566031 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5134958700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513009be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f512ffff700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f513012f8b0 0x7f513009c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 -- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 msgr2=0x7f513009be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513009be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 -- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f513012f8b0 msgr2=0x7f513009c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f513012f8b0 0x7f513009c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 -- 172.21.15.89:0/804566031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512400a6f0 con 0x7f5130130430 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f5135159700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5130130430 0x7f51301c4380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5128007fc0 tx=0x7f512800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:31.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.197+0000 7f512d7fa700 1 -- 172.21.15.89:0/804566031 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800ad40 con 0x7f5130130430 2023-09-30T22:19:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.198+0000 7f512d7fa700 1 -- 172.21.15.89:0/804566031 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128011840 con 0x7f5130130430 2023-09-30T22:19:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.198+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513009e900 con 0x7f5130130430 2023-09-30T22:19:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.198+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513009d030 con 0x7f5130130430 2023-09-30T22:19:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.198+0000 7f512d7fa700 1 -- 172.21.15.89:0/804566031 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800aea0 con 0x7f5130130430 2023-09-30T22:19:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.199+0000 7f512d7fa700 1 -- 172.21.15.89:0/804566031 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5128011bf0 con 0x7f5130130430 2023-09-30T22:19:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.199+0000 7f512d7fa700 1 -- 172.21.15.89:0/804566031 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5128047220 con 0x7f5130130430 2023-09-30T22:19:31.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.199+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51100052e0 con 0x7f5130130430 2023-09-30T22:19:31.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.207+0000 7f512d7fa700 1 -- 172.21.15.89:0/804566031 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f512801e070 con 0x7f5130130430 2023-09-30T22:19:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.238+0000 7fcb79e7c700 1 Processor -- start 2023-09-30T22:19:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.238+0000 7fcb79e7c700 1 -- start start 2023-09-30T22:19:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.238+0000 7fcb79e7c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb7409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.238+0000 7fcb79e7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f240 0x7fcb741317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.238+0000 7fcb79e7c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb74070940 con 0x7fcb7409f650 2023-09-30T22:19:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.238+0000 7fcb79e7c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7406fd60 con 0x7fcb7409e930 2023-09-30T22:19:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.238+0000 7fcb79e7c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7406fa80 con 0x7fcb7409f240 2023-09-30T22:19:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb73fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcb7409f650 0x7fcb74131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47074/0 (socket says 172.21.15.89:47074) 2023-09-30T22:19:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb72ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f240 0x7fcb741317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb73fff700 1 -- 172.21.15.89:0/2739613866 learned_addr learned my addr 172.21.15.89:0/2739613866 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb737fe700 1 --2- 172.21.15.89:0/2739613866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb7409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb727fc700 1 -- 172.21.15.89:0/2739613866 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928805313 0 0) 0x7fcb74070940 con 0x7fcb7409f650 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb727fc700 1 -- 172.21.15.89:0/2739613866 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb5c0035b0 con 0x7fcb7409f650 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb737fe700 1 -- 172.21.15.89:0/2739613866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f240 msgr2=0x7fcb741317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb737fe700 1 --2- 172.21.15.89:0/2739613866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f240 0x7fcb741317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb737fe700 1 -- 172.21.15.89:0/2739613866 >> v1:172.21.15.89:6789/0 conn(0x7fcb7409f650 legacy=0x7fcb74131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb737fe700 1 -- 172.21.15.89:0/2739613866 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7409fb60 con 0x7fcb7409e930 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb737fe700 1 --2- 172.21.15.89:0/2739613866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb7409ed00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb64009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6ac71c26c23506 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb727fc700 1 -- 172.21.15.89:0/2739613866 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64011830 con 0x7fcb7409e930 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb727fc700 1 -- 172.21.15.89:0/2739613866 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb64011990 con 0x7fcb7409e930 2023-09-30T22:19:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.239+0000 7fcb727fc700 1 -- 172.21.15.89:0/2739613866 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64011d00 con 0x7fcb7409e930 2023-09-30T22:19:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.240+0000 7fcb79e7c700 1 -- 172.21.15.89:0/2739613866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 msgr2=0x7fcb7409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.240+0000 7fcb79e7c700 1 --2- 172.21.15.89:0/2739613866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb7409ed00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb64009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.240+0000 7fcb79e7c700 1 -- 172.21.15.89:0/2739613866 shutdown_connections 2023-09-30T22:19:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.240+0000 7fcb79e7c700 1 --2- 172.21.15.89:0/2739613866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb7409ed00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.240+0000 7fcb79e7c700 1 --2- 172.21.15.89:0/2739613866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f240 0x7fcb741317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.240+0000 7fcb79e7c700 1 -- 172.21.15.89:0/2739613866 >> 172.21.15.89:0/2739613866 conn(0x7fcb740a4210 msgr2=0x7fcb740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.240+0000 7fcb79e7c700 1 -- 172.21.15.89:0/2739613866 shutdown_connections 2023-09-30T22:19:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.240+0000 7fcb79e7c700 1 -- 172.21.15.89:0/2739613866 wait complete. 2023-09-30T22:19:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.241+0000 7fcb79e7c700 1 Processor -- start 2023-09-30T22:19:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb79e7c700 1 -- start start 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb79e7c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb741a3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb79e7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb7409f240 0x7fcb741a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb79e7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f650 0x7fcb741c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb79e7c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb74071d20 con 0x7fcb7409f240 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb79e7c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb741a6100 con 0x7fcb7409e930 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb79e7c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb741b7250 con 0x7fcb7409f650 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb73fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f650 0x7fcb741c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb73fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f650 0x7fcb741c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42010/0 (socket says 172.21.15.89:42010) 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb73fff700 1 -- 172.21.15.89:0/4003055580 learned_addr learned my addr 172.21.15.89:0/4003055580 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb737fe700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb741a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.242+0000 7fcb72ffd700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb7409f240 0x7fcb741a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb737fe700 1 -- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f650 msgr2=0x7fcb741c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb737fe700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f650 0x7fcb741c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb737fe700 1 -- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb7409f240 msgr2=0x7fcb741a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb737fe700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb7409f240 0x7fcb741a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb737fe700 1 -- 172.21.15.89:0/4003055580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640094a0 con 0x7fcb7409e930 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb73fff700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f650 0x7fcb741c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb72ffd700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb7409f240 0x7fcb741a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb737fe700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb741a3220 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcb64016000 tx=0x7fcb64004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb5bfff700 1 -- 172.21.15.89:0/4003055580 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640044d0 con 0x7fcb7409e930 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb5bfff700 1 -- 172.21.15.89:0/4003055580 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6401f070 con 0x7fcb7409e930 2023-09-30T22:19:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb740a0190 con 0x7fcb7409e930 2023-09-30T22:19:31.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb5bfff700 1 -- 172.21.15.89:0/4003055580 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640038e0 con 0x7fcb7409e930 2023-09-30T22:19:31.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.243+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb741a3ca0 con 0x7fcb7409e930 2023-09-30T22:19:31.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.244+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb741a6240 con 0x7fcb7409e930 2023-09-30T22:19:31.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.245+0000 7fcb5bfff700 1 -- 172.21.15.89:0/4003055580 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb64003a40 con 0x7fcb7409e930 2023-09-30T22:19:31.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.245+0000 7fcb5bfff700 1 -- 172.21.15.89:0/4003055580 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb64021070 con 0x7fcb7409e930 2023-09-30T22:19:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.253+0000 7fcb5bfff700 1 -- 172.21.15.89:0/4003055580 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb640193b0 con 0x7fcb7409e930 2023-09-30T22:19:31.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.375+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fab90074ea0 con 0x7fab9006f680 2023-09-30T22:19:31.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.375+0000 7fab85ffb700 1 -- 172.21.15.89:0/2522199088 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab8c0248b0 con 0x7fab9006f680 2023-09-30T22:19:31.376 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:31.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.378+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab9006f680 msgr2=0x7fab90118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.378+0000 7fab9740f700 1 --2- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab9006f680 0x7fab90118cd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00b170 tx=0x7fab8c00b480 comp rx=0 tx=0).stop 2023-09-30T22:19:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.379+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 shutdown_connections 2023-09-30T22:19:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.379+0000 7fab9740f700 1 --2- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab90130670 0x7fab90119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.379+0000 7fab9740f700 1 --2- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab9006f680 0x7fab90118cd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.379+0000 7fab9740f700 1 --2- 172.21.15.89:0/2522199088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab900701a0 0x7fab90119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.379+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 >> 172.21.15.89:0/2522199088 conn(0x7fab900a2aa0 msgr2=0x7fab9012d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.379+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 shutdown_connections 2023-09-30T22:19:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.379+0000 7fab9740f700 1 -- 172.21.15.89:0/2522199088 wait complete. 2023-09-30T22:19:31.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.385+0000 7f8f957fa700 1 -- 172.21.15.89:0/889292720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8f88005860 con 0x7f8fb009d6f0 2023-09-30T22:19:31.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.385+0000 7f8f977fe700 1 -- 172.21.15.89:0/889292720 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8fa001b430 con 0x7f8fb009d6f0 2023-09-30T22:19:31.386 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:31.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.387+0000 7f8f957fa700 1 -- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 msgr2=0x7f8fb0127f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.387+0000 7f8f957fa700 1 --2- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb0127f30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009c40 tx=0x7f8fa0004750 comp rx=0 tx=0).stop 2023-09-30T22:19:31.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.387+0000 7f8f957fa700 1 -- 172.21.15.89:0/889292720 shutdown_connections 2023-09-30T22:19:31.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.387+0000 7f8f957fa700 1 --2- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fb009e410 0x7f8fb01cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.387+0000 7f8f957fa700 1 --2- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb009d6f0 0x7f8fb0127f30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.387+0000 7f8f957fa700 1 --2- 172.21.15.89:0/889292720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fb009e000 0x7f8fb0128470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.387+0000 7f8f957fa700 1 -- 172.21.15.89:0/889292720 >> 172.21.15.89:0/889292720 conn(0x7f8fb0066f50 msgr2=0x7f8fb00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:31.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.388+0000 7f8f957fa700 1 -- 172.21.15.89:0/889292720 shutdown_connections 2023-09-30T22:19:31.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.388+0000 7f8f957fa700 1 -- 172.21.15.89:0/889292720 wait complete. 2023-09-30T22:19:31.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:31.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:31.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.419+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5110005590 con 0x7f5130130430 2023-09-30T22:19:31.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.419+0000 7f512d7fa700 1 -- 172.21.15.89:0/804566031 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f512801e020 con 0x7f5130130430 2023-09-30T22:19:31.419 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:31.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5130130430 msgr2=0x7f51301c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5130130430 0x7f51301c4380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5128007fc0 tx=0x7f512800cef0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 shutdown_connections 2023-09-30T22:19:31.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5130130430 0x7f51301c4380 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f513012cf60 0x7f513009be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 --2- 172.21.15.89:0/804566031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f513012f8b0 0x7f513009c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 >> 172.21.15.89:0/804566031 conn(0x7f51300a2aa0 msgr2=0x7f513006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:31.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 shutdown_connections 2023-09-30T22:19:31.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.421+0000 7f5136bbc700 1 -- 172.21.15.89:0/804566031 wait complete. 2023-09-30T22:19:31.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.451+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcb741a6240 con 0x7fcb7409e930 2023-09-30T22:19:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.451+0000 7fcb5bfff700 1 -- 172.21.15.89:0/4003055580 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb741a6240 con 0x7fcb7409e930 2023-09-30T22:19:31.451 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 msgr2=0x7fcb741a3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb741a3220 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcb64016000 tx=0x7fcb64004750 comp rx=0 tx=0).stop 2023-09-30T22:19:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 shutdown_connections 2023-09-30T22:19:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb7409f650 0x7fcb741c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7409e930 0x7fcb741a3220 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 --2- 172.21.15.89:0/4003055580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb7409f240 0x7fcb741a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 >> 172.21.15.89:0/4003055580 conn(0x7fcb740a4210 msgr2=0x7fcb741b9820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 shutdown_connections 2023-09-30T22:19:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:31.453+0000 7fcb79e7c700 1 -- 172.21.15.89:0/4003055580 wait complete. 2023-09-30T22:19:31.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:32.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:32.402 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:32.436 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:32.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.539+0000 7ff1c2e86700 1 Processor -- start 2023-09-30T22:19:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.539+0000 7ff1c2e86700 1 -- start start 2023-09-30T22:19:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c2e86700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c2e86700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c2e86700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1bc09f130 con 0x7ff1bc09e000 2023-09-30T22:19:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c2e86700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc09f520 con 0x7ff1bc09e410 2023-09-30T22:19:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c2e86700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc09e930 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1bbfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff1bc09e000 0x7ff1bc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47088/0 (socket says 172.21.15.89:47088) 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c0c22700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1bbfff700 1 -- 172.21.15.89:0/3004732342 learned_addr learned my addr 172.21.15.89:0/3004732342 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c1423700 1 --2- 172.21.15.89:0/3004732342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c0c22700 1 -- 172.21.15.89:0/3004732342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 msgr2=0x7ff1bc130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c0c22700 1 --2- 172.21.15.89:0/3004732342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c0c22700 1 -- 172.21.15.89:0/3004732342 >> v1:172.21.15.89:6789/0 conn(0x7ff1bc09e000 legacy=0x7ff1bc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.540+0000 7ff1c0c22700 1 -- 172.21.15.89:0/3004732342 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc09b2d0 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.541+0000 7ff1c0c22700 1 --2- 172.21.15.89:0/3004732342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc09dac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c2555265d5e0136 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:33.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.541+0000 7ff1bb7fe700 1 -- 172.21.15.89:0/3004732342 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0011830 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.541+0000 7ff1bb7fe700 1 -- 172.21.15.89:0/3004732342 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b0011990 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.541+0000 7ff1bb7fe700 1 -- 172.21.15.89:0/3004732342 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0011d20 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.541+0000 7ff1c2e86700 1 -- 172.21.15.89:0/3004732342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 msgr2=0x7ff1bc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.541+0000 7ff1c2e86700 1 --2- 172.21.15.89:0/3004732342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc09dac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.542+0000 7ff1c2e86700 1 -- 172.21.15.89:0/3004732342 shutdown_connections 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.542+0000 7ff1c2e86700 1 --2- 172.21.15.89:0/3004732342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.542+0000 7ff1c2e86700 1 --2- 172.21.15.89:0/3004732342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc09dac0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.542+0000 7ff1c2e86700 1 -- 172.21.15.89:0/3004732342 >> 172.21.15.89:0/3004732342 conn(0x7ff1bc066f50 msgr2=0x7ff1bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:33.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.542+0000 7ff1c2e86700 1 -- 172.21.15.89:0/3004732342 shutdown_connections 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.542+0000 7ff1c2e86700 1 -- 172.21.15.89:0/3004732342 wait complete. 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c2e86700 1 Processor -- start 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c2e86700 1 -- start start 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c2e86700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc12dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c2e86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1bc09e000 0x7ff1bc12e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c2e86700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc12e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c2e86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc09c700 con 0x7ff1bc09e000 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c2e86700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc139380 con 0x7ff1bc09e410 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c2e86700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc072570 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c0c22700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc12dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c0c22700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc12dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42022/0 (socket says 172.21.15.89:42022) 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1c0c22700 1 -- 172.21.15.89:0/40434292 learned_addr learned my addr 172.21.15.89:0/40434292 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.543+0000 7ff1bbfff700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1bc09e000 0x7ff1bc12e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c1423700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc12e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c0c22700 1 -- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 msgr2=0x7ff1bc12e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c0c22700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc12e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c0c22700 1 -- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1bc09e000 msgr2=0x7ff1bc12e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c0c22700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1bc09e000 0x7ff1bc12e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c0c22700 1 -- 172.21.15.89:0/40434292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b00094a0 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c1423700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc12e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1bbfff700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1bc09e000 0x7ff1bc12e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c0c22700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc12dd50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00090b0 tx=0x7ff1b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1b97fa700 1 -- 172.21.15.89:0/40434292 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0004450 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1b97fa700 1 -- 172.21.15.89:0/40434292 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b001f070 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc131ca0 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1b97fa700 1 -- 172.21.15.89:0/40434292 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00038e0 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.544+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc12f8d0 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.546+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc074ea0 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.546+0000 7ff1b97fa700 1 -- 172.21.15.89:0/40434292 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1b0003a40 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.546+0000 7ff1b97fa700 1 -- 172.21.15.89:0/40434292 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff1b0021070 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.550+0000 7ff1b97fa700 1 -- 172.21.15.89:0/40434292 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1b00253e0 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc729020700 1 Processor -- start 2023-09-30T22:19:33.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc729020700 1 -- start start 2023-09-30T22:19:33.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc729020700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc72409d630 0x7fc7240a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc729020700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc72413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc729020700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240a2a70 con 0x7fc7240a0a30 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc729020700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240a01d0 con 0x7fc72409d630 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc729020700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc72409e2b0 con 0x7fc724138d80 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc72359e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc72413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc72259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc72409d630 0x7fc7240a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.577+0000 7fc72359e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc72413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42030/0 (socket says 172.21.15.89:42030) 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc72359e700 1 -- 172.21.15.89:0/1121426116 learned_addr learned my addr 172.21.15.89:0/1121426116 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc72359e700 1 -- 172.21.15.89:0/1121426116 >> v2:172.21.15.89:3300/0 conn(0x7fc72409d630 msgr2=0x7fc7240a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc72359e700 1 --2- 172.21.15.89:0/1121426116 >> v2:172.21.15.89:3300/0 conn(0x7fc72409d630 0x7fc7240a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc72359e700 1 -- 172.21.15.89:0/1121426116 >> v1:172.21.15.89:6789/0 conn(0x7fc7240a0a30 legacy=0x7fc7240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc72359e700 1 -- 172.21.15.89:0/1121426116 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72409dc50 con 0x7fc724138d80 2023-09-30T22:19:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc72359e700 1 --2- 172.21.15.89:0/1121426116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc72413b160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009e50 tx=0x7fc71c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8e28ebfe0b597c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc721d9b700 1 -- 172.21.15.89:0/1121426116 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00e070 con 0x7fc724138d80 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc721d9b700 1 -- 172.21.15.89:0/1121426116 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c00b910 con 0x7fc724138d80 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.578+0000 7fc721d9b700 1 -- 172.21.15.89:0/1121426116 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00bc80 con 0x7fc724138d80 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.579+0000 7fc729020700 1 -- 172.21.15.89:0/1121426116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 msgr2=0x7fc72413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.579+0000 7fc729020700 1 --2- 172.21.15.89:0/1121426116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc72413b160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009e50 tx=0x7fc71c00b330 comp rx=0 tx=0).stop 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.579+0000 7fc729020700 1 -- 172.21.15.89:0/1121426116 shutdown_connections 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.579+0000 7fc729020700 1 --2- 172.21.15.89:0/1121426116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc72413b160 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.579+0000 7fc729020700 1 --2- 172.21.15.89:0/1121426116 >> v2:172.21.15.89:3300/0 conn(0x7fc72409d630 0x7fc7240a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.579+0000 7fc729020700 1 -- 172.21.15.89:0/1121426116 >> 172.21.15.89:0/1121426116 conn(0x7fc7240a3050 msgr2=0x7fc7240a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.579+0000 7fc729020700 1 -- 172.21.15.89:0/1121426116 shutdown_connections 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.579+0000 7fc729020700 1 -- 172.21.15.89:0/1121426116 wait complete. 2023-09-30T22:19:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.581+0000 7fc729020700 1 Processor -- start 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.581+0000 7fc729020700 1 -- start start 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.581+0000 7fc729020700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc72409d630 0x7fc724133820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc729020700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7240a0a30 0x7fc724133d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc729020700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc7241342a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc729020700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc724137140 con 0x7fc7240a0a30 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc729020700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc724137f20 con 0x7fc72409d630 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc729020700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc724137340 con 0x7fc724138d80 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc72409d630 0x7fc724133820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc72359e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc7241342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc72409d630 0x7fc724133820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56816/0 (socket says 172.21.15.89:56816) 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 -- 172.21.15.89:0/1879553964 learned_addr learned my addr 172.21.15.89:0/1879553964 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc72259c700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7240a0a30 0x7fc724133d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 -- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 msgr2=0x7fc7241342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc7241342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 -- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7240a0a30 msgr2=0x7fc724133d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7240a0a30 0x7fc724133d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 -- 172.21.15.89:0/1879553964 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c009b30 con 0x7fc72409d630 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc72359e700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc7241342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc722d9d700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc72409d630 0x7fc724133820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c510 tx=0x7fc71400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc70b7fe700 1 -- 172.21.15.89:0/1879553964 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140052d0 con 0x7fc72409d630 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc724136a20 con 0x7fc72409d630 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7240a0410 con 0x7fc72409d630 2023-09-30T22:19:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc70b7fe700 1 -- 172.21.15.89:0/1879553964 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc714005430 con 0x7fc72409d630 2023-09-30T22:19:33.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.582+0000 7fc70b7fe700 1 -- 172.21.15.89:0/1879553964 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714003880 con 0x7fc72409d630 2023-09-30T22:19:33.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.583+0000 7fc70b7fe700 1 -- 172.21.15.89:0/1879553964 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7140039e0 con 0x7fc72409d630 2023-09-30T22:19:33.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.583+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7040052e0 con 0x7fc72409d630 2023-09-30T22:19:33.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.584+0000 7fc70b7fe700 1 -- 172.21.15.89:0/1879553964 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc71403ea30 con 0x7fc72409d630 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.590+0000 7f40375be700 1 Processor -- start 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.590+0000 7f40375be700 1 -- start start 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.590+0000 7f40375be700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403009eca0 0x7f403013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.590+0000 7f40375be700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f403006f050 0x7f403013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.590+0000 7f40375be700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4030070ee0 con 0x7f403006e570 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.590+0000 7f40375be700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4030071980 con 0x7f403006f050 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.590+0000 7f40375be700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4030071480 con 0x7f403009eca0 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.590+0000 7f4034b59700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403009eca0 0x7f403013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4034b59700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403009eca0 0x7f403013cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42038/0 (socket says 172.21.15.89:42038) 2023-09-30T22:19:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4034b59700 1 -- 172.21.15.89:0/3894451284 learned_addr learned my addr 172.21.15.89:0/3894451284 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4035b5b700 1 --2- 172.21.15.89:0/3894451284 >> v2:172.21.15.89:3300/0 conn(0x7f403006f050 0x7f403013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4034b59700 1 -- 172.21.15.89:0/3894451284 >> v2:172.21.15.89:3300/0 conn(0x7f403006f050 msgr2=0x7f403013f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4034b59700 1 --2- 172.21.15.89:0/3894451284 >> v2:172.21.15.89:3300/0 conn(0x7f403006f050 0x7f403013f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4034b59700 1 -- 172.21.15.89:0/3894451284 >> v1:172.21.15.89:6789/0 conn(0x7f403006e570 legacy=0x7f403006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4034b59700 1 -- 172.21.15.89:0/3894451284 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030071b90 con 0x7f403009eca0 2023-09-30T22:19:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7fc70b7fe700 1 -- 172.21.15.89:0/1879553964 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc714010b60 con 0x7fc72409d630 2023-09-30T22:19:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4034b59700 1 --2- 172.21.15.89:0/3894451284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403009eca0 0x7f403013cea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f40200098f0 tx=0x7f4020009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16549721be05ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4027fff700 1 -- 172.21.15.89:0/3894451284 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020011830 con 0x7f403009eca0 2023-09-30T22:19:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4027fff700 1 -- 172.21.15.89:0/3894451284 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4020011990 con 0x7f403009eca0 2023-09-30T22:19:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.591+0000 7f4027fff700 1 -- 172.21.15.89:0/3894451284 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020011d20 con 0x7f403009eca0 2023-09-30T22:19:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.592+0000 7f40375be700 1 -- 172.21.15.89:0/3894451284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403009eca0 msgr2=0x7f403013cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.592+0000 7f40375be700 1 --2- 172.21.15.89:0/3894451284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403009eca0 0x7f403013cea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f40200098f0 tx=0x7f4020009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.592+0000 7f40375be700 1 -- 172.21.15.89:0/3894451284 shutdown_connections 2023-09-30T22:19:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.592+0000 7f40375be700 1 --2- 172.21.15.89:0/3894451284 >> v2:172.21.15.89:3300/0 conn(0x7f403006f050 0x7f403013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.592+0000 7f40375be700 1 --2- 172.21.15.89:0/3894451284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403009eca0 0x7f403013cea0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.592+0000 7f40375be700 1 -- 172.21.15.89:0/3894451284 >> 172.21.15.89:0/3894451284 conn(0x7f40300a2aa0 msgr2=0x7f40300a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.592+0000 7f40375be700 1 -- 172.21.15.89:0/3894451284 shutdown_connections 2023-09-30T22:19:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.592+0000 7f40375be700 1 -- 172.21.15.89:0/3894451284 wait complete. 2023-09-30T22:19:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f40375be700 1 Processor -- start 2023-09-30T22:19:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f40375be700 1 -- start start 2023-09-30T22:19:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f40375be700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f403006e570 0x7f403013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f40375be700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f403006f050 0x7f403013d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f40375be700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403012e1a0 0x7f403013d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f40375be700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40301387c0 con 0x7f403006f050 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f40375be700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403013e880 con 0x7f403006e570 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f40375be700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4030070680 con 0x7f403012e1a0 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f4034b59700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f403006f050 0x7f403013d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f4034b59700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f403006f050 0x7f403013d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38994/0 (socket says 172.21.15.89:38994) 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f403535a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f403006e570 0x7f403013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f4034b59700 1 -- 172.21.15.89:0/901402851 learned_addr learned my addr 172.21.15.89:0/901402851 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.593+0000 7f4035b5b700 1 --2- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403012e1a0 0x7f403013d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4034b59700 1 -- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403012e1a0 msgr2=0x7f403013d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4034b59700 1 --2- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403012e1a0 0x7f403013d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4034b59700 1 -- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f403006e570 msgr2=0x7f403013cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4034b59700 1 --2- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f403006e570 0x7f403013cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4034b59700 1 -- 172.21.15.89:0/901402851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4020009550 con 0x7f403006f050 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4034b59700 1 --2- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f403006f050 0x7f403013d2c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4020013000 tx=0x7f4020004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4025ffb700 1 -- 172.21.15.89:0/901402851 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020004510 con 0x7f403006f050 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4025ffb700 1 -- 172.21.15.89:0/901402851 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40200036a0 con 0x7f403006f050 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403012c380 con 0x7f403006f050 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f4025ffb700 1 -- 172.21.15.89:0/901402851 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020003a50 con 0x7f403006f050 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.594+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403012d430 con 0x7f403006f050 2023-09-30T22:19:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.595+0000 7f4025ffb700 1 -- 172.21.15.89:0/901402851 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4020003bb0 con 0x7f403006f050 2023-09-30T22:19:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.595+0000 7f4025ffb700 1 -- 172.21.15.89:0/901402851 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4020040920 con 0x7f403006f050 2023-09-30T22:19:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.596+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4030074ea0 con 0x7f403006f050 2023-09-30T22:19:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.600+0000 7f4025ffb700 1 -- 172.21.15.89:0/901402851 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4020028080 con 0x7f403006f050 2023-09-30T22:19:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.761+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff1bc0a3b60 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.762+0000 7ff1b97fa700 1 -- 172.21.15.89:0/40434292 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1b0015020 con 0x7ff1bc09d6f0 2023-09-30T22:19:33.032 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 msgr2=0x7ff1bc12dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc12dd50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00090b0 tx=0x7ff1b0004750 comp rx=0 tx=0).stop 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 shutdown_connections 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1bc09e410 0x7ff1bc12e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1bc09d6f0 0x7ff1bc12dd50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 --2- 172.21.15.89:0/40434292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1bc09e000 0x7ff1bc12e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 >> 172.21.15.89:0/40434292 conn(0x7ff1bc066f50 msgr2=0x7ff1bc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 shutdown_connections 2023-09-30T22:19:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.765+0000 7ff1c2e86700 1 -- 172.21.15.89:0/40434292 wait complete. 2023-09-30T22:19:33.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.792+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc704005150 con 0x7fc72409d630 2023-09-30T22:19:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.792+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f403012d900 con 0x7f403006f050 2023-09-30T22:19:33.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.792+0000 7fc70b7fe700 1 -- 172.21.15.89:0/1879553964 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc714017070 con 0x7fc72409d630 2023-09-30T22:19:33.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.792+0000 7f4025ffb700 1 -- 172.21.15.89:0/901402851 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f403012d900 con 0x7f403006f050 2023-09-30T22:19:33.035 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:33.035 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:33.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc72409d630 msgr2=0x7fc724133820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc72409d630 0x7fc724133820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c510 tx=0x7fc71400c820 comp rx=0 tx=0).stop 2023-09-30T22:19:33.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 shutdown_connections 2023-09-30T22:19:33.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc724138d80 0x7fc7241342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc72409d630 0x7fc724133820 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f403006f050 msgr2=0x7f403013d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 --2- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f403006f050 0x7f403013d2c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4020013000 tx=0x7f4020004750 comp rx=0 tx=0).stop 2023-09-30T22:19:33.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 --2- 172.21.15.89:0/1879553964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7240a0a30 0x7fc724133d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 >> 172.21.15.89:0/1879553964 conn(0x7fc7240a3050 msgr2=0x7fc7240a4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:33.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 shutdown_connections 2023-09-30T22:19:33.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 --2- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f403012e1a0 0x7f403013d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 shutdown_connections 2023-09-30T22:19:33.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 --2- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f403006e570 0x7f403013cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 --2- 172.21.15.89:0/901402851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f403006f050 0x7f403013d2c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7fc729020700 1 -- 172.21.15.89:0/1879553964 wait complete. 2023-09-30T22:19:33.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 >> 172.21.15.89:0/901402851 conn(0x7f40300a2aa0 msgr2=0x7f4030099af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:33.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 shutdown_connections 2023-09-30T22:19:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:32.794+0000 7f40375be700 1 -- 172.21.15.89:0/901402851 wait complete. 2023-09-30T22:19:33.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:33.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.195+0000 7f4170fa6700 1 Processor -- start 2023-09-30T22:19:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.195+0000 7f4170fa6700 1 -- start start 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f4170fa6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f416c06a770 0x7f416c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f4170fa6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c131d10 0x7f416c139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f4170fa6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416c09ece0 con 0x7f416c06b080 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f4170fa6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c09e760 con 0x7f416c06a770 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f4170fa6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f416c09e040 con 0x7f416c131d10 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c131d10 0x7f416c139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f416c06a770 0x7f416c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c131d10 0x7f416c139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42066/0 (socket says 172.21.15.89:42066) 2023-09-30T22:19:33.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416ad9d700 1 -- 172.21.15.89:0/3352768109 learned_addr learned my addr 172.21.15.89:0/3352768109 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416a59c700 1 -- 172.21.15.89:0/3352768109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c131d10 msgr2=0x7f416c139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416a59c700 1 --2- 172.21.15.89:0/3352768109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c131d10 0x7f416c139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416a59c700 1 -- 172.21.15.89:0/3352768109 >> v1:172.21.15.89:6789/0 conn(0x7f416c06b080 legacy=0x7f416c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416a59c700 1 -- 172.21.15.89:0/3352768109 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c09f630 con 0x7f416c06a770 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416959a700 1 -- 172.21.15.89:0/3352768109 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919615390 0 0) 0x7f416c09ece0 con 0x7f416c06b080 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416ad9d700 1 --2- 172.21.15.89:0/3352768109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c131d10 0x7f416c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.196+0000 7f416a59c700 1 --2- 172.21.15.89:0/3352768109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06a770 0x7f416c06ab40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4160009840 tx=0x7f4160009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a912d83585435c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f416959a700 1 -- 172.21.15.89:0/3352768109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160011830 con 0x7f416c06a770 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f416959a700 1 -- 172.21.15.89:0/3352768109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160011990 con 0x7f416c06a770 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f416959a700 1 -- 172.21.15.89:0/3352768109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160011d20 con 0x7f416c06a770 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f4170fa6700 1 -- 172.21.15.89:0/3352768109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06a770 msgr2=0x7f416c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f4170fa6700 1 --2- 172.21.15.89:0/3352768109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06a770 0x7f416c06ab40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4160009840 tx=0x7f4160009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f4170fa6700 1 -- 172.21.15.89:0/3352768109 shutdown_connections 2023-09-30T22:19:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f4170fa6700 1 --2- 172.21.15.89:0/3352768109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c06a770 0x7f416c06ab40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f4170fa6700 1 --2- 172.21.15.89:0/3352768109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c131d10 0x7f416c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f4170fa6700 1 -- 172.21.15.89:0/3352768109 >> 172.21.15.89:0/3352768109 conn(0x7f416c066f50 msgr2=0x7f416c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f4170fa6700 1 -- 172.21.15.89:0/3352768109 shutdown_connections 2023-09-30T22:19:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.197+0000 7f4170fa6700 1 -- 172.21.15.89:0/3352768109 wait complete. 2023-09-30T22:19:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.198+0000 7f4170fa6700 1 Processor -- start 2023-09-30T22:19:33.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.198+0000 7f4170fa6700 1 -- start start 2023-09-30T22:19:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f4170fa6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f416c06a770 0x7f416c120370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f4170fa6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c06b080 0x7f416c1208b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f4170fa6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c131d10 0x7f416c120df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f4170fa6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c123e40 con 0x7f416c131d10 2023-09-30T22:19:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f4170fa6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c130e20 con 0x7f416c06a770 2023-09-30T22:19:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f4170fa6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f416c121fd0 con 0x7f416c06b080 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f416c06a770 0x7f416c120370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c131d10 0x7f416c120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f4169d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c06b080 0x7f416c1208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f416c06a770 0x7f416c120370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56828/0 (socket says 172.21.15.89:56828) 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c131d10 0x7f416c120df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39012/0 (socket says 172.21.15.89:39012) 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416a59c700 1 -- 172.21.15.89:0/3677663148 learned_addr learned my addr 172.21.15.89:0/3677663148 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416ad9d700 1 -- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c06b080 msgr2=0x7f416c1208b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416ad9d700 1 --2- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c06b080 0x7f416c1208b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416ad9d700 1 -- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f416c06a770 msgr2=0x7f416c120370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416ad9d700 1 --2- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f416c06a770 0x7f416c120370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416ad9d700 1 -- 172.21.15.89:0/3677663148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164009b30 con 0x7f416c131d10 2023-09-30T22:19:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f416ad9d700 1 --2- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c131d10 0x7f416c120df0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f416400efc0 tx=0x7f416400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f415affd700 1 -- 172.21.15.89:0/3677663148 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164018070 con 0x7f416c131d10 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.199+0000 7f415affd700 1 -- 172.21.15.89:0/3677663148 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416400f040 con 0x7f416c131d10 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.200+0000 7f415affd700 1 -- 172.21.15.89:0/3677663148 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164003c60 con 0x7f416c131d10 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.200+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41600094a0 con 0x7f416c131d10 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.200+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c071e60 con 0x7f416c131d10 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.200+0000 7f415affd700 1 -- 172.21.15.89:0/3677663148 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f416400c880 con 0x7f416c131d10 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.200+0000 7f415affd700 1 -- 172.21.15.89:0/3677663148 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f416403f780 con 0x7f416c131d10 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.200+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c074ea0 con 0x7f416c131d10 2023-09-30T22:19:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.205+0000 7f415affd700 1 -- 172.21.15.89:0/3677663148 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4164003dc0 con 0x7f416c131d10 2023-09-30T22:19:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.391+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f416c06c360 con 0x7f416c131d10 2023-09-30T22:19:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.391+0000 7f415affd700 1 -- 172.21.15.89:0/3677663148 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f416401de00 con 0x7f416c131d10 2023-09-30T22:19:33.391 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c131d10 msgr2=0x7f416c120df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 --2- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c131d10 0x7f416c120df0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f416400efc0 tx=0x7f416400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 shutdown_connections 2023-09-30T22:19:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 --2- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f416c131d10 0x7f416c120df0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 --2- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f416c06a770 0x7f416c120370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 --2- 172.21.15.89:0/3677663148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f416c06b080 0x7f416c1208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 >> 172.21.15.89:0/3677663148 conn(0x7f416c066f50 msgr2=0x7f416c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 shutdown_connections 2023-09-30T22:19:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:33.393+0000 7f4170fa6700 1 -- 172.21.15.89:0/3677663148 wait complete. 2023-09-30T22:19:33.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:34.034 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:34.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:34.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:34.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.172+0000 7f01ab3fa700 1 Processor -- start 2023-09-30T22:19:34.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.172+0000 7f01ab3fa700 1 -- start start 2023-09-30T22:19:34.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01ab3fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01ab3fa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01a412f8b0 0x7f01a412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01ab3fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a4130260 0x7f01a4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01ab3fa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a412d330 con 0x7f01a412f8b0 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01ab3fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a412cd90 con 0x7f01a412cf60 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01ab3fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01a41306d0 con 0x7f01a4130260 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01a9196700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01a9997700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a4130260 0x7f01a4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01a8995700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01a412f8b0 0x7f01a412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01a9196700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56832/0 (socket says 172.21.15.89:56832) 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01a9196700 1 -- 172.21.15.89:0/1020991435 learned_addr learned my addr 172.21.15.89:0/1020991435 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.173+0000 7f01a9196700 1 -- 172.21.15.89:0/1020991435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a4130260 msgr2=0x7f01a4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f01a9196700 1 --2- 172.21.15.89:0/1020991435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a4130260 0x7f01a4136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f01a9196700 1 -- 172.21.15.89:0/1020991435 >> v2:172.21.15.89:3300/0 conn(0x7f01a412f8b0 msgr2=0x7f01a412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:34.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f01a9196700 1 --2- 172.21.15.89:0/1020991435 >> v2:172.21.15.89:3300/0 conn(0x7f01a412f8b0 0x7f01a412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f01a9196700 1 -- 172.21.15.89:0/1020991435 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a4071780 con 0x7f01a412cf60 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f01a8995700 1 --2- 172.21.15.89:0/1020991435 >> v2:172.21.15.89:3300/0 conn(0x7f01a412f8b0 0x7f01a412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f01a9997700 1 --2- 172.21.15.89:0/1020991435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a4130260 0x7f01a4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f01a9196700 1 --2- 172.21.15.89:0/1020991435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a406f5d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009920 tx=0x7f01a0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b37d7af51853fee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f019bfff700 1 -- 172.21.15.89:0/1020991435 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011830 con 0x7f01a412cf60 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f019bfff700 1 -- 172.21.15.89:0/1020991435 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0011990 con 0x7f01a412cf60 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.174+0000 7f019bfff700 1 -- 172.21.15.89:0/1020991435 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011d20 con 0x7f01a412cf60 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 -- 172.21.15.89:0/1020991435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 msgr2=0x7f01a406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 --2- 172.21.15.89:0/1020991435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a406f5d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009920 tx=0x7f01a0009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 -- 172.21.15.89:0/1020991435 shutdown_connections 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 --2- 172.21.15.89:0/1020991435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a4130260 0x7f01a4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 --2- 172.21.15.89:0/1020991435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a406f5d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 --2- 172.21.15.89:0/1020991435 >> v2:172.21.15.89:3300/0 conn(0x7f01a412f8b0 0x7f01a412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 -- 172.21.15.89:0/1020991435 >> 172.21.15.89:0/1020991435 conn(0x7f01a40a2aa0 msgr2=0x7f01a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:34.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 -- 172.21.15.89:0/1020991435 shutdown_connections 2023-09-30T22:19:34.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.175+0000 7f01ab3fa700 1 -- 172.21.15.89:0/1020991435 wait complete. 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.176+0000 7f01ab3fa700 1 Processor -- start 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.176+0000 7f01ab3fa700 1 -- start start 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01ab3fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a41309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01ab3fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a412f8b0 0x7f01a4130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01ab3fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a4130260 0x7f01a4131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01ab3fa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4132d90 con 0x7f01a4130260 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01ab3fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4132aa0 con 0x7f01a412cf60 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01ab3fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4133c10 con 0x7f01a412f8b0 2023-09-30T22:19:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a412f8b0 0x7f01a4130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a412f8b0 0x7f01a4130f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42098/0 (socket says 172.21.15.89:42098) 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 -- 172.21.15.89:0/3485515898 learned_addr learned my addr 172.21.15.89:0/3485515898 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a9997700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a4130260 0x7f01a4131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a9196700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a41309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 -- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 msgr2=0x7f01a41309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a41309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 -- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a4130260 msgr2=0x7f01a4131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a4130260 0x7f01a4131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 -- 172.21.15.89:0/3485515898 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940094a0 con 0x7f01a412f8b0 2023-09-30T22:19:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.177+0000 7f01a8995700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a412f8b0 0x7f01a4130f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0194009ea0 tx=0x7f019400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.178+0000 7f0199ffb700 1 -- 172.21.15.89:0/3485515898 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194013860 con 0x7f01a412f8b0 2023-09-30T22:19:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.178+0000 7f0199ffb700 1 -- 172.21.15.89:0/3485515898 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01940139c0 con 0x7f01a412f8b0 2023-09-30T22:19:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.178+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c009a70 con 0x7f01a412f8b0 2023-09-30T22:19:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.178+0000 7f0199ffb700 1 -- 172.21.15.89:0/3485515898 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194013d50 con 0x7f01a412f8b0 2023-09-30T22:19:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.178+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a0009550 con 0x7f01a412f8b0 2023-09-30T22:19:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.179+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a4074ea0 con 0x7f01a412f8b0 2023-09-30T22:19:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.180+0000 7f0199ffb700 1 -- 172.21.15.89:0/3485515898 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f019400c5e0 con 0x7f01a412f8b0 2023-09-30T22:19:34.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.180+0000 7f0199ffb700 1 -- 172.21.15.89:0/3485515898 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0194043b10 con 0x7f01a412f8b0 2023-09-30T22:19:34.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.184+0000 7f0199ffb700 1 -- 172.21.15.89:0/3485515898 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f019400c890 con 0x7f01a412f8b0 2023-09-30T22:19:34.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.211+0000 7fdb47317700 1 Processor -- start 2023-09-30T22:19:34.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.211+0000 7f0ffd8cf700 1 Processor -- start 2023-09-30T22:19:34.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.211+0000 7fdb47317700 1 -- start start 2023-09-30T22:19:34.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ffd8cf700 1 -- start start 2023-09-30T22:19:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb47317700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb4006e570 0x7fdb4006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb47317700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb40134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb47317700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb40070ee0 con 0x7fdb4009eca0 2023-09-30T22:19:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb47317700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb40071480 con 0x7fdb4006e570 2023-09-30T22:19:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb47317700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb40071b90 con 0x7fdb4006ee80 2023-09-30T22:19:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb450b3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb4006e570 0x7fdb4006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb450b3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb4006e570 0x7fdb4006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39028/0 (socket says 172.21.15.89:39028) 2023-09-30T22:19:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb450b3700 1 -- 172.21.15.89:0/313761604 learned_addr learned my addr 172.21.15.89:0/313761604 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ffd8cf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ff812cf60 0x7f0ff806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7fdb458b4700 1 --2- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb40134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ffd8cf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812f8b0 0x7f0ff812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ffd8cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 0x7f0ff8136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ffd8cf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff812d330 con 0x7f0ff812cf60 2023-09-30T22:19:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ffd8cf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff812cd90 con 0x7f0ff812f8b0 2023-09-30T22:19:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ffd8cf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff81306d0 con 0x7f0ff8130260 2023-09-30T22:19:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ff6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ff812cf60 0x7f0ff806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ff6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ff812cf60 0x7f0ff806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39030/0 (socket says 172.21.15.89:39030) 2023-09-30T22:19:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.212+0000 7f0ff6ffd700 1 -- 172.21.15.89:0/4239895530 learned_addr learned my addr 172.21.15.89:0/4239895530 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb450b3700 1 -- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 msgr2=0x7fdb40134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb450b3700 1 --2- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb40134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb450b3700 1 -- 172.21.15.89:0/313761604 >> v1:172.21.15.89:6789/0 conn(0x7fdb4009eca0 legacy=0x7fdb40134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb450b3700 1 -- 172.21.15.89:0/313761604 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400711a0 con 0x7fdb4006e570 2023-09-30T22:19:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff67fc700 1 --2- 172.21.15.89:0/4239895530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812f8b0 0x7f0ff812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb458b4700 1 --2- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb40134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff67fc700 1 -- 172.21.15.89:0/4239895530 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 msgr2=0x7f0ff8136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff67fc700 1 --2- 172.21.15.89:0/4239895530 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 0x7f0ff8136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff67fc700 1 -- 172.21.15.89:0/4239895530 >> v2:172.21.15.89:3300/0 conn(0x7f0ff812cf60 msgr2=0x7f0ff806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff67fc700 1 --2- 172.21.15.89:0/4239895530 >> v2:172.21.15.89:3300/0 conn(0x7f0ff812cf60 0x7f0ff806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff67fc700 1 -- 172.21.15.89:0/4239895530 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8071780 con 0x7f0ff812f8b0 2023-09-30T22:19:34.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff6ffd700 1 --2- 172.21.15.89:0/4239895530 >> v2:172.21.15.89:3300/0 conn(0x7f0ff812cf60 0x7f0ff806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:34.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb450b3700 1 --2- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb4006e570 0x7fdb4006e940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0098f0 tx=0x7fdb3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756701d4a5b80598 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:34.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff67fc700 1 --2- 172.21.15.89:0/4239895530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812f8b0 0x7f0ff812fd20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00bae0 tx=0x7f0fec00bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7f5484650fa147 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:34.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb37fff700 1 -- 172.21.15.89:0/313761604 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c011830 con 0x7fdb4006e570 2023-09-30T22:19:34.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb37fff700 1 -- 172.21.15.89:0/313761604 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c011990 con 0x7fdb4006e570 2023-09-30T22:19:34.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7fdb37fff700 1 -- 172.21.15.89:0/313761604 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c011d20 con 0x7fdb4006e570 2023-09-30T22:19:34.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff5ffb700 1 -- 172.21.15.89:0/4239895530 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00db60 con 0x7f0ff812f8b0 2023-09-30T22:19:34.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff5ffb700 1 -- 172.21.15.89:0/4239895530 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec00dcc0 con 0x7f0ff812f8b0 2023-09-30T22:19:34.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.213+0000 7f0ff5ffb700 1 -- 172.21.15.89:0/4239895530 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec017630 con 0x7f0ff812f8b0 2023-09-30T22:19:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7fdb47317700 1 -- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb4006e570 msgr2=0x7fdb4006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7fdb47317700 1 --2- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb4006e570 0x7fdb4006e940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0098f0 tx=0x7fdb3c009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7fdb47317700 1 -- 172.21.15.89:0/313761604 shutdown_connections 2023-09-30T22:19:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7fdb47317700 1 --2- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb40134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7fdb47317700 1 --2- 172.21.15.89:0/313761604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb4006e570 0x7fdb4006e940 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7fdb47317700 1 -- 172.21.15.89:0/313761604 >> 172.21.15.89:0/313761604 conn(0x7fdb400a2aa0 msgr2=0x7fdb400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239895530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812f8b0 msgr2=0x7f0ff812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 --2- 172.21.15.89:0/4239895530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812f8b0 0x7f0ff812fd20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00bae0 tx=0x7f0fec00bdf0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239895530 shutdown_connections 2023-09-30T22:19:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 --2- 172.21.15.89:0/4239895530 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 0x7f0ff8136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 --2- 172.21.15.89:0/4239895530 >> v2:172.21.15.89:3300/0 conn(0x7f0ff812cf60 0x7f0ff806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 --2- 172.21.15.89:0/4239895530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812f8b0 0x7f0ff812fd20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239895530 >> 172.21.15.89:0/4239895530 conn(0x7f0ff80a2aa0 msgr2=0x7f0ff80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239895530 shutdown_connections 2023-09-30T22:19:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7fdb47317700 1 -- 172.21.15.89:0/313761604 shutdown_connections 2023-09-30T22:19:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239895530 wait complete. 2023-09-30T22:19:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.214+0000 7fdb47317700 1 -- 172.21.15.89:0/313761604 wait complete. 2023-09-30T22:19:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.215+0000 7fdb47317700 1 Processor -- start 2023-09-30T22:19:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.215+0000 7fdb47317700 1 -- start start 2023-09-30T22:19:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7f0ffd8cf700 1 Processor -- start 2023-09-30T22:19:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb47317700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb4006e570 0x7fdb40123370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb47317700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb401238b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb47317700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb40123df0 0x7fdb401cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb47317700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb40124ec0 con 0x7fdb40123df0 2023-09-30T22:19:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb47317700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb401264b0 con 0x7fdb4006ee80 2023-09-30T22:19:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb47317700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb40135cf0 con 0x7fdb4006e570 2023-09-30T22:19:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb448b2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb401238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb4006e570 0x7fdb40123370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb4006e570 0x7fdb40123370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42120/0 (socket says 172.21.15.89:42120) 2023-09-30T22:19:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 -- 172.21.15.89:0/655529771 learned_addr learned my addr 172.21.15.89:0/655529771 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb448b2700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb401238b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56886/0 (socket says 172.21.15.89:56886) 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb458b4700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb40123df0 0x7fdb401cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 -- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 msgr2=0x7fdb401238b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb401238b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 -- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb40123df0 msgr2=0x7fdb401cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb40123df0 0x7fdb401cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 -- 172.21.15.89:0/655529771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb38009a70 con 0x7fdb4006e570 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb448b2700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb401238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7fdb450b3700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb4006e570 0x7fdb40123370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009c40 tx=0x7fdb3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:34.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7f0ffd8cf700 1 -- start start 2023-09-30T22:19:34.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7f0ffd8cf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812cf60 0x7f0ff809b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7f0ffd8cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff812f8b0 0x7f0ff809b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7f0ffd8cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 0x7f0ff809bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7f0ffd8cf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff809c540 con 0x7f0ff812f8b0 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7f0ffd8cf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8118e60 con 0x7f0ff812cf60 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.216+0000 7f0ffd8cf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff81190e0 con 0x7f0ff8130260 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812cf60 0x7f0ff809b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812cf60 0x7f0ff809b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56890/0 (socket says 172.21.15.89:56890) 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 0x7f0ff809bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff812f8b0 0x7f0ff809b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff6ffd700 1 -- 172.21.15.89:0/4239322969 learned_addr learned my addr 172.21.15.89:0/4239322969 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7fdb35ffb700 1 -- 172.21.15.89:0/655529771 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c0043c0 con 0x7fdb4006e570 2023-09-30T22:19:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7fdb35ffb700 1 -- 172.21.15.89:0/655529771 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c01f070 con 0x7fdb4006e570 2023-09-30T22:19:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7fdb35ffb700 1 -- 172.21.15.89:0/655529771 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c0036a0 con 0x7fdb4006e570 2023-09-30T22:19:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c009550 con 0x7fdb4006e570 2023-09-30T22:19:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb40070610 con 0x7fdb4006e570 2023-09-30T22:19:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff77fe700 1 -- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812cf60 msgr2=0x7f0ff809b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff77fe700 1 --2- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812cf60 0x7f0ff809b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff77fe700 1 -- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff812f8b0 msgr2=0x7f0ff809b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff77fe700 1 --2- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff812f8b0 0x7f0ff809b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff77fe700 1 -- 172.21.15.89:0/4239322969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe8009550 con 0x7f0ff8130260 2023-09-30T22:19:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0ff77fe700 1 --2- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 0x7f0ff809bb10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000bdf0 tx=0x7f0ff0009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0fdffff700 1 -- 172.21.15.89:0/4239322969 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000cb30 con 0x7f0ff8130260 2023-09-30T22:19:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0fdffff700 1 -- 172.21.15.89:0/4239322969 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff000cc90 con 0x7f0ff8130260 2023-09-30T22:19:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.217+0000 7f0fdffff700 1 -- 172.21.15.89:0/4239322969 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0014440 con 0x7f0ff8130260 2023-09-30T22:19:34.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.218+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec00b740 con 0x7f0ff8130260 2023-09-30T22:19:34.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.218+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff8129fa0 con 0x7f0ff8130260 2023-09-30T22:19:34.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.218+0000 7fdb35ffb700 1 -- 172.21.15.89:0/655529771 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdb3c021030 con 0x7fdb4006e570 2023-09-30T22:19:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.218+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb40074ea0 con 0x7fdb4006e570 2023-09-30T22:19:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.218+0000 7fdb35ffb700 1 -- 172.21.15.89:0/655529771 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdb3c040a20 con 0x7fdb4006e570 2023-09-30T22:19:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.219+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff809e670 con 0x7f0ff8130260 2023-09-30T22:19:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.223+0000 7fdb35ffb700 1 -- 172.21.15.89:0/655529771 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdb3c0199a0 con 0x7fdb4006e570 2023-09-30T22:19:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.223+0000 7f0fdffff700 1 -- 172.21.15.89:0/4239322969 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ff001d3b0 con 0x7f0ff8130260 2023-09-30T22:19:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.223+0000 7f0fdffff700 1 -- 172.21.15.89:0/4239322969 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0ff0006e40 con 0x7f0ff8130260 2023-09-30T22:19:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.227+0000 7f0fdffff700 1 -- 172.21.15.89:0/4239322969 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ff00072b0 con 0x7f0ff8130260 2023-09-30T22:19:34.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.383+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f01a40713a0 con 0x7f01a412f8b0 2023-09-30T22:19:34.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.383+0000 7f0199ffb700 1 -- 172.21.15.89:0/3485515898 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0194020030 con 0x7f01a412f8b0 2023-09-30T22:19:34.384 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.385+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a412f8b0 msgr2=0x7f01a4130f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.385+0000 7f01ab3fa700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a412f8b0 0x7f01a4130f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0194009ea0 tx=0x7f019400ee90 comp rx=0 tx=0).stop 2023-09-30T22:19:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.386+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 shutdown_connections 2023-09-30T22:19:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.386+0000 7f01ab3fa700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a4130260 0x7f01a4131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.386+0000 7f01ab3fa700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f01a412cf60 0x7f01a41309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.386+0000 7f01ab3fa700 1 --2- 172.21.15.89:0/3485515898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01a412f8b0 0x7f01a4130f20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.386+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 >> 172.21.15.89:0/3485515898 conn(0x7f01a40a2aa0 msgr2=0x7f01a412f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:34.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.386+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 shutdown_connections 2023-09-30T22:19:34.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.386+0000 7f01ab3fa700 1 -- 172.21.15.89:0/3485515898 wait complete. 2023-09-30T22:19:34.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:34.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.421+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdb4006b9e0 con 0x7fdb4006e570 2023-09-30T22:19:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.421+0000 7fdb35ffb700 1 -- 172.21.15.89:0/655529771 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdb3c03c070 con 0x7fdb4006e570 2023-09-30T22:19:34.423 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.423+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb4006e570 msgr2=0x7fdb40123370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.423+0000 7fdb47317700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb4006e570 0x7fdb40123370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009c40 tx=0x7fdb3c004750 comp rx=0 tx=0).stop 2023-09-30T22:19:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.424+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 shutdown_connections 2023-09-30T22:19:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.424+0000 7fdb47317700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb40123df0 0x7fdb401cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.424+0000 7fdb47317700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb4006e570 0x7fdb40123370 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.424+0000 7fdb47317700 1 --2- 172.21.15.89:0/655529771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdb4006ee80 0x7fdb401238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.424+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 >> 172.21.15.89:0/655529771 conn(0x7fdb400a2aa0 msgr2=0x7fdb400a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:34.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.424+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 shutdown_connections 2023-09-30T22:19:34.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.424+0000 7fdb47317700 1 -- 172.21.15.89:0/655529771 wait complete. 2023-09-30T22:19:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.427+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0ff8074ea0 con 0x7f0ff8130260 2023-09-30T22:19:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.427+0000 7f0fdffff700 1 -- 172.21.15.89:0/4239322969 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ff0019030 con 0x7f0ff8130260 2023-09-30T22:19:34.428 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.429+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 msgr2=0x7f0ff809bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.429+0000 7f0ffd8cf700 1 --2- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 0x7f0ff809bb10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000bdf0 tx=0x7f0ff0009ed0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.429+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 shutdown_connections 2023-09-30T22:19:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.429+0000 7f0ffd8cf700 1 --2- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ff8130260 0x7f0ff809bb10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.429+0000 7f0ffd8cf700 1 --2- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ff812cf60 0x7f0ff809b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.429+0000 7f0ffd8cf700 1 --2- 172.21.15.89:0/4239322969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ff812f8b0 0x7f0ff809b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.429+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 >> 172.21.15.89:0/4239322969 conn(0x7f0ff80a2aa0 msgr2=0x7f0ff806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.430+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 shutdown_connections 2023-09-30T22:19:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.430+0000 7f0ffd8cf700 1 -- 172.21.15.89:0/4239322969 wait complete. 2023-09-30T22:19:34.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:34.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:34.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.533+0000 7f6ed6781700 1 Processor -- start 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.533+0000 7f6ed6781700 1 -- start start 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.533+0000 7f6ed6781700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed0136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ed6781700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed0138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ed6781700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed009e130 con 0x7f6ed01316c0 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ed6781700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01321a0 con 0x7f6ed009d9f0 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ed6781700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01312b0 con 0x7f6ed0136a70 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed0136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ecffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6ed01316c0 0x7f6ed0131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47148/0 (socket says 172.21.15.89:47148) 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ecffff700 1 -- 172.21.15.89:0/517906249 learned_addr learned my addr 172.21.15.89:0/517906249 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:34.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ed4d1e700 1 --2- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed0138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6eceffd700 1 -- 172.21.15.89:0/517906249 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917370655 0 0) 0x7f6ec4001110 con 0x7f6ed01316c0 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ecf7fe700 1 -- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 msgr2=0x7f6ed0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ecf7fe700 1 --2- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed0138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ecf7fe700 1 -- 172.21.15.89:0/517906249 >> v1:172.21.15.89:6789/0 conn(0x7f6ed01316c0 legacy=0x7f6ed0131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ecf7fe700 1 -- 172.21.15.89:0/517906249 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed009f6e0 con 0x7f6ed009d9f0 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ed4d1e700 1 --2- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed0138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6ecf7fe700 1 --2- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed0136530 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009870 tx=0x7f6ec0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4f076bdda5c12b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6eceffd700 1 -- 172.21.15.89:0/517906249 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0011830 con 0x7f6ed009d9f0 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6eceffd700 1 -- 172.21.15.89:0/517906249 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec0011990 con 0x7f6ed009d9f0 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.534+0000 7f6eceffd700 1 -- 172.21.15.89:0/517906249 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0011d20 con 0x7f6ed009d9f0 2023-09-30T22:19:34.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.535+0000 7f6ed6781700 1 -- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 msgr2=0x7f6ed0136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.535+0000 7f6ed6781700 1 --2- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed0136530 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009870 tx=0x7f6ec0009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:34.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.535+0000 7f6ed6781700 1 -- 172.21.15.89:0/517906249 shutdown_connections 2023-09-30T22:19:34.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.535+0000 7f6ed6781700 1 --2- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed0138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.535+0000 7f6ed6781700 1 --2- 172.21.15.89:0/517906249 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed0136530 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.535+0000 7f6ed6781700 1 -- 172.21.15.89:0/517906249 >> 172.21.15.89:0/517906249 conn(0x7f6ed0066f50 msgr2=0x7f6ed009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:34.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.535+0000 7f6ed6781700 1 -- 172.21.15.89:0/517906249 shutdown_connections 2023-09-30T22:19:34.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.535+0000 7f6ed6781700 1 -- 172.21.15.89:0/517906249 wait complete. 2023-09-30T22:19:34.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed6781700 1 Processor -- start 2023-09-30T22:19:34.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed6781700 1 -- start start 2023-09-30T22:19:34.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed6781700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed012ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed6781700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed01316c0 0x7f6ed012b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed6781700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed01341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed6781700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed009a7a0 con 0x7f6ed01316c0 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed6781700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed009c2e0 con 0x7f6ed009d9f0 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed6781700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed0138e60 con 0x7f6ed0136a70 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ecffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed012ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed4d1e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed01341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed4d1e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed01341e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42146/0 (socket says 172.21.15.89:42146) 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ecffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed012ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56910/0 (socket says 172.21.15.89:56910) 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ed4d1e700 1 -- 172.21.15.89:0/1259312079 learned_addr learned my addr 172.21.15.89:0/1259312079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.536+0000 7f6ecf7fe700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed01316c0 0x7f6ed012b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ed4d1e700 1 -- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 msgr2=0x7f6ed012ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ed4d1e700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed012ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ed4d1e700 1 -- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed01316c0 msgr2=0x7f6ed012b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ed4d1e700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed01316c0 0x7f6ed012b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ed4d1e700 1 -- 172.21.15.89:0/1259312079 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec8009a70 con 0x7f6ed0136a70 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ecffff700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed012ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ed4d1e700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed01341e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009ea0 tx=0x7f6ec800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6eccff9700 1 -- 172.21.15.89:0/1259312079 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800cb50 con 0x7f6ed0136a70 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6eccff9700 1 -- 172.21.15.89:0/1259312079 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec800f040 con 0x7f6ed0136a70 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6eccff9700 1 -- 172.21.15.89:0/1259312079 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8003a20 con 0x7f6ed0136a70 2023-09-30T22:19:34.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec00094a0 con 0x7f6ed0136a70 2023-09-30T22:19:34.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.537+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed009bb00 con 0x7f6ed0136a70 2023-09-30T22:19:34.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.538+0000 7f6eccff9700 1 -- 172.21.15.89:0/1259312079 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ec8014480 con 0x7f6ed0136a70 2023-09-30T22:19:34.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.538+0000 7f6eccff9700 1 -- 172.21.15.89:0/1259312079 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6ec80400c0 con 0x7f6ed0136a70 2023-09-30T22:19:34.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.538+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed009b780 con 0x7f6ed0136a70 2023-09-30T22:19:34.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.543+0000 7f6eccff9700 1 -- 172.21.15.89:0/1259312079 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ed009b780 con 0x7f6ed0136a70 2023-09-30T22:19:34.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.725+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6ed009b780 con 0x7f6ed0136a70 2023-09-30T22:19:34.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.725+0000 7f6eccff9700 1 -- 172.21.15.89:0/1259312079 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ed009b780 con 0x7f6ed0136a70 2023-09-30T22:19:34.726 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:34.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 msgr2=0x7f6ed01341e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:34.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed01341e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009ea0 tx=0x7f6ec800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 shutdown_connections 2023-09-30T22:19:34.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ed0136a70 0x7f6ed01341e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ed009d9f0 0x7f6ed012ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 --2- 172.21.15.89:0/1259312079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ed01316c0 0x7f6ed012b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:34.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 >> 172.21.15.89:0/1259312079 conn(0x7f6ed0066f50 msgr2=0x7f6ed0136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:34.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 shutdown_connections 2023-09-30T22:19:34.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:34.728+0000 7f6ed6781700 1 -- 172.21.15.89:0/1259312079 wait complete. 2023-09-30T22:19:34.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:35.398 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:35.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:35.446 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:35.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.528+0000 7fb43f9a6700 1 Processor -- start 2023-09-30T22:19:35.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43f9a6700 1 -- start start 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43f9a6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130540 0x7fb438130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43f9a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb438130e50 0x7fb43806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43f9a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4380716a0 con 0x7fb43806fb20 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43f9a6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43809a390 con 0x7fb438130e50 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43f9a6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb438071290 con 0x7fb438130540 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43d742700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130540 0x7fb438130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43d742700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130540 0x7fb438130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42148/0 (socket says 172.21.15.89:42148) 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43d742700 1 -- 172.21.15.89:0/3722019199 learned_addr learned my addr 172.21.15.89:0/3722019199 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:35.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.529+0000 7fb43cf41700 1 --2- 172.21.15.89:0/3722019199 >> v2:172.21.15.89:3300/0 conn(0x7fb438130e50 0x7fb43806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb43cf41700 1 -- 172.21.15.89:0/3722019199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130540 msgr2=0x7fb438130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb43cf41700 1 --2- 172.21.15.89:0/3722019199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130540 0x7fb438130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb43cf41700 1 -- 172.21.15.89:0/3722019199 >> v1:172.21.15.89:6789/0 conn(0x7fb43806fb20 legacy=0x7fb438137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb43cf41700 1 -- 172.21.15.89:0/3722019199 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb438130340 con 0x7fb438130e50 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb43d742700 1 --2- 172.21.15.89:0/3722019199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130540 0x7fb438130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb43cf41700 1 --2- 172.21.15.89:0/3722019199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb438130e50 0x7fb43806f5e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb428009840 tx=0x7fb428009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe2309ff6476463 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb42ffff700 1 -- 172.21.15.89:0/3722019199 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428011830 con 0x7fb438130e50 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb42ffff700 1 -- 172.21.15.89:0/3722019199 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb428011990 con 0x7fb438130e50 2023-09-30T22:19:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.530+0000 7fb42ffff700 1 -- 172.21.15.89:0/3722019199 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428011d20 con 0x7fb438130e50 2023-09-30T22:19:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.531+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3722019199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb438130e50 msgr2=0x7fb43806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.531+0000 7fb43f9a6700 1 --2- 172.21.15.89:0/3722019199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb438130e50 0x7fb43806f5e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb428009840 tx=0x7fb428009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.531+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3722019199 shutdown_connections 2023-09-30T22:19:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.531+0000 7fb43f9a6700 1 --2- 172.21.15.89:0/3722019199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130540 0x7fb438130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.531+0000 7fb43f9a6700 1 --2- 172.21.15.89:0/3722019199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb438130e50 0x7fb43806f5e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.531+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3722019199 >> 172.21.15.89:0/3722019199 conn(0x7fb4380a2aa0 msgr2=0x7fb4380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.531+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3722019199 shutdown_connections 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.531+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3722019199 wait complete. 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.532+0000 7fb43f9a6700 1 Processor -- start 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.532+0000 7fb43f9a6700 1 -- start start 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43f9a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb43806fb20 0x7fb438125da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43f9a6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb438130540 0x7fb4381262e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43f9a6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130e50 0x7fb438126820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43f9a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43812ce40 con 0x7fb43806fb20 2023-09-30T22:19:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43f9a6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43812d670 con 0x7fb438130540 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43f9a6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb43812c8a0 con 0x7fb438130e50 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb43806fb20 0x7fb438125da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb43806fb20 0x7fb438125da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39072/0 (socket says 172.21.15.89:39072) 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 -- 172.21.15.89:0/3789956358 learned_addr learned my addr 172.21.15.89:0/3789956358 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43cf41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb438130540 0x7fb4381262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43df43700 1 --2- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130e50 0x7fb438126820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 -- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130e50 msgr2=0x7fb438126820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 --2- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130e50 0x7fb438126820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 -- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb438130540 msgr2=0x7fb4381262e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 --2- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb438130540 0x7fb4381262e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 -- 172.21.15.89:0/3789956358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434009550 con 0x7fb43806fb20 2023-09-30T22:19:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43d742700 1 --2- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb43806fb20 0x7fb438125da0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb43400ebe0 tx=0x7fb43400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb42dffb700 1 -- 172.21.15.89:0/3789956358 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43400c850 con 0x7fb43806fb20 2023-09-30T22:19:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb42dffb700 1 -- 172.21.15.89:0/3789956358 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43400c9b0 con 0x7fb43806fb20 2023-09-30T22:19:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4280094a0 con 0x7fb43806fb20 2023-09-30T22:19:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.533+0000 7fb42dffb700 1 -- 172.21.15.89:0/3789956358 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43400cd40 con 0x7fb43806fb20 2023-09-30T22:19:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.534+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43812b1e0 con 0x7fb43806fb20 2023-09-30T22:19:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.534+0000 7fb42dffb700 1 -- 172.21.15.89:0/3789956358 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb434013470 con 0x7fb43806fb20 2023-09-30T22:19:35.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.535+0000 7fb42dffb700 1 -- 172.21.15.89:0/3789956358 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb43403f810 con 0x7fb43806fb20 2023-09-30T22:19:35.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.535+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43812ca60 con 0x7fb43806fb20 2023-09-30T22:19:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.540+0000 7fb42dffb700 1 -- 172.21.15.89:0/3789956358 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb434021080 con 0x7fb43806fb20 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.569+0000 7f8752841700 1 Processor -- start 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.569+0000 7f8752841700 1 -- start start 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f8752841700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09e1d0 0x7f874c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f8752841700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f874c139170 0x7f874c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f8752841700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c09b120 con 0x7f874c09d6f0 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f8752841700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c06b460 con 0x7f874c139170 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f8752841700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f874c09a7e0 con 0x7f874c09e1d0 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f874b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09e1d0 0x7f874c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f8750dde700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f874c139170 0x7f874c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f874b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09e1d0 0x7f874c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42170/0 (socket says 172.21.15.89:42170) 2023-09-30T22:19:35.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f874b7fe700 1 -- 172.21.15.89:0/3821225470 learned_addr learned my addr 172.21.15.89:0/3821225470 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:35.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f874b7fe700 1 -- 172.21.15.89:0/3821225470 >> v2:172.21.15.89:3300/0 conn(0x7f874c139170 msgr2=0x7f874c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:35.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f874b7fe700 1 --2- 172.21.15.89:0/3821225470 >> v2:172.21.15.89:3300/0 conn(0x7f874c139170 0x7f874c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f874b7fe700 1 -- 172.21.15.89:0/3821225470 >> v1:172.21.15.89:6789/0 conn(0x7f874c09d6f0 legacy=0x7f874c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f874b7fe700 1 -- 172.21.15.89:0/3821225470 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c09a500 con 0x7f874c09e1d0 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.570+0000 7f8750dde700 1 --2- 172.21.15.89:0/3821225470 >> v2:172.21.15.89:3300/0 conn(0x7f874c139170 0x7f874c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f874b7fe700 1 --2- 172.21.15.89:0/3821225470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09e1d0 0x7f874c138c30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8740009920 tx=0x7f8740009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe51e35beb07a399 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f874affd700 1 -- 172.21.15.89:0/3821225470 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740011830 con 0x7f874c09e1d0 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f874affd700 1 -- 172.21.15.89:0/3821225470 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8740011990 con 0x7f874c09e1d0 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f874affd700 1 -- 172.21.15.89:0/3821225470 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740011d20 con 0x7f874c09e1d0 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f8752841700 1 -- 172.21.15.89:0/3821225470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09e1d0 msgr2=0x7f874c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f8752841700 1 --2- 172.21.15.89:0/3821225470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09e1d0 0x7f874c138c30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8740009920 tx=0x7f8740009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f8752841700 1 -- 172.21.15.89:0/3821225470 shutdown_connections 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f8752841700 1 --2- 172.21.15.89:0/3821225470 >> v2:172.21.15.89:3300/0 conn(0x7f874c139170 0x7f874c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f8752841700 1 --2- 172.21.15.89:0/3821225470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09e1d0 0x7f874c138c30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f8752841700 1 -- 172.21.15.89:0/3821225470 >> 172.21.15.89:0/3821225470 conn(0x7f874c066f50 msgr2=0x7f874c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f8752841700 1 -- 172.21.15.89:0/3821225470 shutdown_connections 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.571+0000 7f8752841700 1 -- 172.21.15.89:0/3821225470 wait complete. 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.572+0000 7f8752841700 1 Processor -- start 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8752841700 1 -- start start 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8752841700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09d6f0 0x7f874c122470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8752841700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f874c09e1d0 0x7f874c1229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8752841700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f874c139170 0x7f874c1c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8752841700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c122000 con 0x7f874c09e1d0 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8752841700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c124fd0 con 0x7f874c139170 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8752841700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0a2a90 con 0x7f874c09d6f0 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f874c09e1d0 0x7f874c1229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f874c09e1d0 0x7f874c1229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39098/0 (socket says 172.21.15.89:39098) 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874b7fe700 1 -- 172.21.15.89:0/2373264592 learned_addr learned my addr 172.21.15.89:0/2373264592 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8750dde700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f874c139170 0x7f874c1c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874bfff700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09d6f0 0x7f874c122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874b7fe700 1 -- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09d6f0 msgr2=0x7f874c122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874b7fe700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09d6f0 0x7f874c122470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874b7fe700 1 -- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f874c139170 msgr2=0x7f874c1c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874b7fe700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f874c139170 0x7f874c1c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f874b7fe700 1 -- 172.21.15.89:0/2373264592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744009a70 con 0x7f874c09e1d0 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.573+0000 7f8750dde700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f874c139170 0x7f874c1c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.574+0000 7f874b7fe700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f874c09e1d0 0x7f874c1229b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8740009870 tx=0x7f8740004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.574+0000 7f8748ff9700 1 -- 172.21.15.89:0/2373264592 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400042c0 con 0x7f874c09e1d0 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.574+0000 7f8748ff9700 1 -- 172.21.15.89:0/2373264592 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874001f070 con 0x7f874c09e1d0 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.574+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8740009550 con 0x7f874c09e1d0 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.574+0000 7f8748ff9700 1 -- 172.21.15.89:0/2373264592 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400036a0 con 0x7f874c09e1d0 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.574+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c125d00 con 0x7f874c09e1d0 2023-09-30T22:19:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.575+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c074ea0 con 0x7f874c09e1d0 2023-09-30T22:19:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.575+0000 7f8748ff9700 1 -- 172.21.15.89:0/2373264592 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8740021030 con 0x7f874c09e1d0 2023-09-30T22:19:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.575+0000 7f8748ff9700 1 -- 172.21.15.89:0/2373264592 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8740040870 con 0x7f874c09e1d0 2023-09-30T22:19:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.584+0000 7f8748ff9700 1 -- 172.21.15.89:0/2373264592 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87400239c0 con 0x7f874c09e1d0 2023-09-30T22:19:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.618+0000 7f1abd2cc700 1 Processor -- start 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.618+0000 7f1abd2cc700 1 -- start start 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1abd2cc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a0a30 0x7f1ab80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1abd2cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a1340 0x7f1ab8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1abd2cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1ab8133fa0 0x7f1ab8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1abd2cc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab809edd0 con 0x7f1ab8133fa0 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1abd2cc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab809e1f0 con 0x7f1ab80a0a30 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1abd2cc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab809dc50 con 0x7f1ab80a1340 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a1340 0x7f1ab8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a0a30 0x7f1ab80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a1340 0x7f1ab8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42194/0 (socket says 172.21.15.89:42194) 2023-09-30T22:19:35.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab67fc700 1 -- 172.21.15.89:0/2960923908 learned_addr learned my addr 172.21.15.89:0/2960923908 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:35.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab77fe700 1 --2- 172.21.15.89:0/2960923908 >> v2:172.21.15.89:3300/0 conn(0x7f1ab8133fa0 0x7f1ab8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab77fe700 1 -- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a1340 msgr2=0x7f1ab8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab77fe700 1 --2- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a1340 0x7f1ab8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab77fe700 1 -- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a0a30 msgr2=0x7f1ab80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab77fe700 1 --2- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a0a30 0x7f1ab80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.619+0000 7f1ab77fe700 1 -- 172.21.15.89:0/2960923908 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80a1fe0 con 0x7f1ab8133fa0 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1ab77fe700 1 --2- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 0x7f1ab8134410 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009d90 tx=0x7f1ab000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cc58ebd692dc8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1ab5ffb700 1 -- 172.21.15.89:0/2960923908 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000e070 con 0x7f1ab8133fa0 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1ab5ffb700 1 -- 172.21.15.89:0/2960923908 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab000bb20 con 0x7f1ab8133fa0 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1ab5ffb700 1 -- 172.21.15.89:0/2960923908 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0012430 con 0x7f1ab8133fa0 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 -- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 msgr2=0x7f1ab8134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 --2- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 0x7f1ab8134410 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009d90 tx=0x7f1ab000b330 comp rx=0 tx=0).stop 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 -- 172.21.15.89:0/2960923908 shutdown_connections 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 --2- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 0x7f1ab8134410 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 --2- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a0a30 0x7f1ab80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 --2- 172.21.15.89:0/2960923908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a1340 0x7f1ab8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 -- 172.21.15.89:0/2960923908 >> 172.21.15.89:0/2960923908 conn(0x7f1ab80a6550 msgr2=0x7f1ab80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 -- 172.21.15.89:0/2960923908 shutdown_connections 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.620+0000 7f1abd2cc700 1 -- 172.21.15.89:0/2960923908 wait complete. 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.621+0000 7f1abd2cc700 1 Processor -- start 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.621+0000 7f1abd2cc700 1 -- start start 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1abd2cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a0a30 0x7f1ab8120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1abd2cc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a1340 0x7f1ab8120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1abd2cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 0x7f1ab8120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1abd2cc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8121d40 con 0x7f1ab8133fa0 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1abd2cc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8122f90 con 0x7f1ab80a1340 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1abd2cc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8122330 con 0x7f1ab80a0a30 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a0a30 0x7f1ab8120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 0x7f1ab8120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a0a30 0x7f1ab8120130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42210/0 (socket says 172.21.15.89:42210) 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab6ffd700 1 -- 172.21.15.89:0/927826879 learned_addr learned my addr 172.21.15.89:0/927826879 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab67fc700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a1340 0x7f1ab8120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab6ffd700 1 -- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a1340 msgr2=0x7f1ab8120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab6ffd700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a1340 0x7f1ab8120670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab6ffd700 1 -- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 msgr2=0x7f1ab8120bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab6ffd700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 0x7f1ab8120bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab6ffd700 1 -- 172.21.15.89:0/927826879 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab0009a70 con 0x7f1ab80a0a30 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.622+0000 7f1ab67fc700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a1340 0x7f1ab8120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.623+0000 7f1ab6ffd700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a0a30 0x7f1ab8120130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800e9e0 tx=0x7f1aa800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.623+0000 7f1a9f7fe700 1 -- 172.21.15.89:0/927826879 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8009710 con 0x7f1ab80a0a30 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.623+0000 7f1a9f7fe700 1 -- 172.21.15.89:0/927826879 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa800ee80 con 0x7f1ab80a0a30 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.623+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab81233e0 con 0x7f1ab80a0a30 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.623+0000 7f1a9f7fe700 1 -- 172.21.15.89:0/927826879 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa800ca50 con 0x7f1ab80a0a30 2023-09-30T22:19:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.623+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab80a3480 con 0x7f1ab80a0a30 2023-09-30T22:19:35.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.623+0000 7f1a9f7fe700 1 -- 172.21.15.89:0/927826879 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1aa800cbb0 con 0x7f1ab80a0a30 2023-09-30T22:19:35.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.624+0000 7f1a9f7fe700 1 -- 172.21.15.89:0/927826879 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1aa803fa90 con 0x7f1ab80a0a30 2023-09-30T22:19:35.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.624+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab80a1cc0 con 0x7f1ab80a0a30 2023-09-30T22:19:35.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.629+0000 7f1a9f7fe700 1 -- 172.21.15.89:0/927826879 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1aa8021080 con 0x7f1ab80a0a30 2023-09-30T22:19:35.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.742+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb43812cc70 con 0x7fb43806fb20 2023-09-30T22:19:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.742+0000 7fb42dffb700 1 -- 172.21.15.89:0/3789956358 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4340232f0 con 0x7fb43806fb20 2023-09-30T22:19:35.742 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:35.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb43806fb20 msgr2=0x7fb438125da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 --2- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb43806fb20 0x7fb438125da0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb43400ebe0 tx=0x7fb43400eef0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 shutdown_connections 2023-09-30T22:19:35.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 --2- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb438130e50 0x7fb438126820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 --2- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb43806fb20 0x7fb438125da0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 --2- 172.21.15.89:0/3789956358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb438130540 0x7fb4381262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 >> 172.21.15.89:0/3789956358 conn(0x7fb4380a2aa0 msgr2=0x7fb438126eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:35.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 shutdown_connections 2023-09-30T22:19:35.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.745+0000 7fb43f9a6700 1 -- 172.21.15.89:0/3789956358 wait complete. 2023-09-30T22:19:35.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.791+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f874c1251f0 con 0x7f874c09e1d0 2023-09-30T22:19:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.791+0000 7f8748ff9700 1 -- 172.21.15.89:0/2373264592 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f874c1251f0 con 0x7f874c09e1d0 2023-09-30T22:19:35.791 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f874c09e1d0 msgr2=0x7f874c1229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f874c09e1d0 0x7f874c1229b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8740009870 tx=0x7f8740004750 comp rx=0 tx=0).stop 2023-09-30T22:19:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 shutdown_connections 2023-09-30T22:19:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f874c139170 0x7f874c1c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f874c09d6f0 0x7f874c122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 --2- 172.21.15.89:0/2373264592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f874c09e1d0 0x7f874c1229b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 >> 172.21.15.89:0/2373264592 conn(0x7f874c066f50 msgr2=0x7f874c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 shutdown_connections 2023-09-30T22:19:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.794+0000 7f8752841700 1 -- 172.21.15.89:0/2373264592 wait complete. 2023-09-30T22:19:35.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:35.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.830+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1ab8074ea0 con 0x7f1ab80a0a30 2023-09-30T22:19:35.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.831+0000 7f1a9f7fe700 1 -- 172.21.15.89:0/927826879 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1aa8017020 con 0x7f1ab80a0a30 2023-09-30T22:19:35.831 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:35.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a0a30 msgr2=0x7f1ab8120130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a0a30 0x7f1ab8120130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800e9e0 tx=0x7f1aa800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 shutdown_connections 2023-09-30T22:19:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab8133fa0 0x7f1ab8120bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ab80a0a30 0x7f1ab8120130 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 --2- 172.21.15.89:0/927826879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ab80a1340 0x7f1ab8120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 >> 172.21.15.89:0/927826879 conn(0x7f1ab80a6550 msgr2=0x7f1ab81bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 shutdown_connections 2023-09-30T22:19:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.833+0000 7f1abd2cc700 1 -- 172.21.15.89:0/927826879 wait complete. 2023-09-30T22:19:35.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:35.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.869+0000 7f13faafb700 1 Processor -- start 2023-09-30T22:19:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.869+0000 7f13faafb700 1 -- start start 2023-09-30T22:19:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13faafb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13faafb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f240 0x7f13f41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13faafb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f4070940 con 0x7f13f409f650 2023-09-30T22:19:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13faafb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f406fd60 con 0x7f13f409f240 2023-09-30T22:19:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13faafb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13f406fa80 con 0x7f13f409e930 2023-09-30T22:19:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f8897700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f240 0x7f13f41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f8897700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42226/0 (socket says 172.21.15.89:42226) 2023-09-30T22:19:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f9098700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f13f409f650 0x7f13f4131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47182/0 (socket says 172.21.15.89:47182) 2023-09-30T22:19:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f8897700 1 -- 172.21.15.89:0/529687472 learned_addr learned my addr 172.21.15.89:0/529687472 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f3fff700 1 -- 172.21.15.89:0/529687472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 msgr2=0x7f13f409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f3fff700 1 --2- 172.21.15.89:0/529687472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f3fff700 1 -- 172.21.15.89:0/529687472 >> v1:172.21.15.89:6789/0 conn(0x7f13f409f650 legacy=0x7f13f4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f37fe700 1 -- 172.21.15.89:0/529687472 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569609475 0 0) 0x7f13f4070940 con 0x7f13f409f650 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f3fff700 1 -- 172.21.15.89:0/529687472 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f409fb60 con 0x7f13f409f240 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f8897700 1 --2- 172.21.15.89:0/529687472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f3fff700 1 --2- 172.21.15.89:0/529687472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f240 0x7f13f41317f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009870 tx=0x7f13e0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=539e0f100d66e6fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.870+0000 7f13f37fe700 1 -- 172.21.15.89:0/529687472 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0011830 con 0x7f13f409f240 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13f37fe700 1 -- 172.21.15.89:0/529687472 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e0011990 con 0x7f13f409f240 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13f37fe700 1 -- 172.21.15.89:0/529687472 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0011d20 con 0x7f13f409f240 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13faafb700 1 -- 172.21.15.89:0/529687472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f240 msgr2=0x7f13f41317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13faafb700 1 --2- 172.21.15.89:0/529687472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f240 0x7f13f41317f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009870 tx=0x7f13e0009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13faafb700 1 -- 172.21.15.89:0/529687472 shutdown_connections 2023-09-30T22:19:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13faafb700 1 --2- 172.21.15.89:0/529687472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13faafb700 1 --2- 172.21.15.89:0/529687472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f240 0x7f13f41317f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13faafb700 1 -- 172.21.15.89:0/529687472 >> 172.21.15.89:0/529687472 conn(0x7f13f40a4210 msgr2=0x7f13f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13faafb700 1 -- 172.21.15.89:0/529687472 shutdown_connections 2023-09-30T22:19:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.871+0000 7f13faafb700 1 -- 172.21.15.89:0/529687472 wait complete. 2023-09-30T22:19:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.872+0000 7f13faafb700 1 Processor -- start 2023-09-30T22:19:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.872+0000 7f13faafb700 1 -- start start 2023-09-30T22:19:35.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.872+0000 7f13faafb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f41a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13faafb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13f409f240 0x7f13f41a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13faafb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f650 0x7f13f41a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13faafb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f41a5de0 con 0x7f13f409f240 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13faafb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f41a76a0 con 0x7f13f409f650 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13faafb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13f41a6a70 con 0x7f13f409e930 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13f409f240 0x7f13f41a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f9098700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f650 0x7f13f41a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f8897700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f41a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13f409f240 0x7f13f41a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39130/0 (socket says 172.21.15.89:39130) 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f3fff700 1 -- 172.21.15.89:0/783312125 learned_addr learned my addr 172.21.15.89:0/783312125 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:35.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f8897700 1 -- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f650 msgr2=0x7f13f41a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f8897700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f650 0x7f13f41a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f8897700 1 -- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13f409f240 msgr2=0x7f13f41a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f8897700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13f409f240 0x7f13f41a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f8897700 1 -- 172.21.15.89:0/783312125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e8009550 con 0x7f13f409e930 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f9098700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f650 0x7f13f41a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f3fff700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13f409f240 0x7f13f41a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f8897700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f41a3c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13e800e9a0 tx=0x7f13e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f17fa700 1 -- 172.21.15.89:0/783312125 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8004030 con 0x7f13f409e930 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f17fa700 1 -- 172.21.15.89:0/783312125 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e8004190 con 0x7f13f409e930 2023-09-30T22:19:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13f17fa700 1 -- 172.21.15.89:0/783312125 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8004520 con 0x7f13f409e930 2023-09-30T22:19:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e00094a0 con 0x7f13f409e930 2023-09-30T22:19:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.873+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f40a0190 con 0x7f13f409e930 2023-09-30T22:19:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.874+0000 7f13f17fa700 1 -- 172.21.15.89:0/783312125 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f13e80036a0 con 0x7f13f409e930 2023-09-30T22:19:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.874+0000 7f13f17fa700 1 -- 172.21.15.89:0/783312125 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f13e803f890 con 0x7f13f409e930 2023-09-30T22:19:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.874+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f41a0b30 con 0x7f13f409e930 2023-09-30T22:19:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:35.879+0000 7f13f17fa700 1 -- 172.21.15.89:0/783312125 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13e8011db0 con 0x7f13f409e930 2023-09-30T22:19:36.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.061+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f13f41a7220 con 0x7f13f409e930 2023-09-30T22:19:36.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.062+0000 7f13f17fa700 1 -- 172.21.15.89:0/783312125 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13f41a7220 con 0x7f13f409e930 2023-09-30T22:19:36.062 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:36.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 msgr2=0x7f13f41a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f41a3c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13e800e9a0 tx=0x7f13e800ecb0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 shutdown_connections 2023-09-30T22:19:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13f409f650 0x7f13f41a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13f409e930 0x7f13f41a3c20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 --2- 172.21.15.89:0/783312125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13f409f240 0x7f13f41a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 >> 172.21.15.89:0/783312125 conn(0x7f13f40a4210 msgr2=0x7f13f41b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 shutdown_connections 2023-09-30T22:19:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.064+0000 7f13faafb700 1 -- 172.21.15.89:0/783312125 wait complete. 2023-09-30T22:19:36.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:36.759 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:36.812 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:36.846 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:36.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.892+0000 7f609336b700 1 Processor -- start 2023-09-30T22:19:36.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.892+0000 7f609336b700 1 -- start start 2023-09-30T22:19:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.892+0000 7f609336b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09d6f0 0x7f608c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.892+0000 7f609336b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e000 0x7f608c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.892+0000 7f609336b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f608c09e410 0x7f608c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.892+0000 7f609336b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c09f520 con 0x7f608c09e410 2023-09-30T22:19:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.892+0000 7f609336b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c09f130 con 0x7f608c09d6f0 2023-09-30T22:19:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.892+0000 7f609336b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f608c09e930 con 0x7f608c09e000 2023-09-30T22:19:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e000 0x7f608c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e000 0x7f608c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42246/0 (socket says 172.21.15.89:42246) 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 -- 172.21.15.89:0/2434261523 learned_addr learned my addr 172.21.15.89:0/2434261523 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6091107700 1 --2- 172.21.15.89:0/2434261523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09d6f0 0x7f608c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6091908700 1 --2- 172.21.15.89:0/2434261523 >> v2:172.21.15.89:3300/0 conn(0x7f608c09e410 0x7f608c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 -- 172.21.15.89:0/2434261523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09d6f0 msgr2=0x7f608c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 --2- 172.21.15.89:0/2434261523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09d6f0 0x7f608c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 -- 172.21.15.89:0/2434261523 >> v2:172.21.15.89:3300/0 conn(0x7f608c09e410 msgr2=0x7f608c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 --2- 172.21.15.89:0/2434261523 >> v2:172.21.15.89:3300/0 conn(0x7f608c09e410 0x7f608c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 -- 172.21.15.89:0/2434261523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c09b2d0 con 0x7f608c09e000 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6090906700 1 --2- 172.21.15.89:0/2434261523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e000 0x7f608c130660 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f607c009920 tx=0x7f607c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e378f4f6a654ae38 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6083fff700 1 -- 172.21.15.89:0/2434261523 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c011830 con 0x7f608c09e000 2023-09-30T22:19:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6083fff700 1 -- 172.21.15.89:0/2434261523 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c011990 con 0x7f608c09e000 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.893+0000 7f6083fff700 1 -- 172.21.15.89:0/2434261523 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c011d20 con 0x7f608c09e000 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 -- 172.21.15.89:0/2434261523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e000 msgr2=0x7f608c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 --2- 172.21.15.89:0/2434261523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e000 0x7f608c130660 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f607c009920 tx=0x7f607c009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 -- 172.21.15.89:0/2434261523 shutdown_connections 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 --2- 172.21.15.89:0/2434261523 >> v2:172.21.15.89:3300/0 conn(0x7f608c09e410 0x7f608c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 --2- 172.21.15.89:0/2434261523 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09d6f0 0x7f608c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 --2- 172.21.15.89:0/2434261523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e000 0x7f608c130660 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 -- 172.21.15.89:0/2434261523 >> 172.21.15.89:0/2434261523 conn(0x7f608c066f50 msgr2=0x7f608c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 -- 172.21.15.89:0/2434261523 shutdown_connections 2023-09-30T22:19:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.894+0000 7f609336b700 1 -- 172.21.15.89:0/2434261523 wait complete. 2023-09-30T22:19:36.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.895+0000 7f609336b700 1 Processor -- start 2023-09-30T22:19:36.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.895+0000 7f609336b700 1 -- start start 2023-09-30T22:19:36.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.895+0000 7f609336b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c09d6f0 0x7f608c122290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.895+0000 7f609336b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09e000 0x7f608c1227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.895+0000 7f609336b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e410 0x7f608c122d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.895+0000 7f609336b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c125ad0 con 0x7f608c09d6f0 2023-09-30T22:19:36.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.895+0000 7f609336b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c123fb0 con 0x7f608c09e000 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.895+0000 7f609336b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0a3af0 con 0x7f608c09e410 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09e000 0x7f608c1227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6091107700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c09d6f0 0x7f608c122290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6091908700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e410 0x7f608c122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09e000 0x7f608c1227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56978/0 (socket says 172.21.15.89:56978) 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 -- 172.21.15.89:0/127361796 learned_addr learned my addr 172.21.15.89:0/127361796 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 -- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e410 msgr2=0x7f608c122d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 --2- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e410 0x7f608c122d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 -- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c09d6f0 msgr2=0x7f608c122290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 --2- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c09d6f0 0x7f608c122290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 -- 172.21.15.89:0/127361796 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c009550 con 0x7f608c09e000 2023-09-30T22:19:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6090906700 1 --2- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09e000 0x7f608c1227d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f607c000c00 tx=0x7f607c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6081ffb700 1 -- 172.21.15.89:0/127361796 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c004450 con 0x7f608c09e000 2023-09-30T22:19:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6081ffb700 1 -- 172.21.15.89:0/127361796 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c0045b0 con 0x7f608c09e000 2023-09-30T22:19:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c09c700 con 0x7f608c09e000 2023-09-30T22:19:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f6081ffb700 1 -- 172.21.15.89:0/127361796 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c0038c0 con 0x7f608c09e000 2023-09-30T22:19:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.896+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c124710 con 0x7f608c09e000 2023-09-30T22:19:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.897+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c074ea0 con 0x7f608c09e000 2023-09-30T22:19:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.898+0000 7f6081ffb700 1 -- 172.21.15.89:0/127361796 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f607c029070 con 0x7f608c09e000 2023-09-30T22:19:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.898+0000 7f6081ffb700 1 -- 172.21.15.89:0/127361796 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f607c049510 con 0x7f608c09e000 2023-09-30T22:19:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.902+0000 7f6081ffb700 1 -- 172.21.15.89:0/127361796 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f607c035990 con 0x7f608c09e000 2023-09-30T22:19:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.944+0000 7fc9d9926700 1 Processor -- start 2023-09-30T22:19:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.944+0000 7fc9d9926700 1 -- start start 2023-09-30T22:19:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d9926700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d41316c0 0x7fc9d4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d9926700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9d406ae80 0x7fc9d4139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d9926700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d409e130 con 0x7fc9d409d9f0 2023-09-30T22:19:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d9926700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d41312b0 con 0x7fc9d406ae80 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d9926700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d409f6e0 con 0x7fc9d41316c0 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d41316c0 0x7fc9d4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d27fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc9d409d9f0 0x7fc9d406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47190/0 (socket says 172.21.15.89:47190) 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d27fc700 1 -- 172.21.15.89:0/3313242756 learned_addr learned my addr 172.21.15.89:0/3313242756 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d37fe700 1 --2- 172.21.15.89:0/3313242756 >> v2:172.21.15.89:3300/0 conn(0x7fc9d406ae80 0x7fc9d4139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d1ffb700 1 -- 172.21.15.89:0/3313242756 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273885289 0 0) 0x7fc9c8001110 con 0x7fc9d409d9f0 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d1ffb700 1 -- 172.21.15.89:0/3313242756 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9bc0035b0 con 0x7fc9d409d9f0 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d2ffd700 1 -- 172.21.15.89:0/3313242756 >> v2:172.21.15.89:3300/0 conn(0x7fc9d406ae80 msgr2=0x7fc9d4139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d2ffd700 1 --2- 172.21.15.89:0/3313242756 >> v2:172.21.15.89:3300/0 conn(0x7fc9d406ae80 0x7fc9d4139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d2ffd700 1 -- 172.21.15.89:0/3313242756 >> v1:172.21.15.89:6789/0 conn(0x7fc9d409d9f0 legacy=0x7fc9d406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.945+0000 7fc9d2ffd700 1 -- 172.21.15.89:0/3313242756 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d409e760 con 0x7fc9d41316c0 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d2ffd700 1 --2- 172.21.15.89:0/3313242756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d41316c0 0x7fc9d4131a90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009840 tx=0x7fc9c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5762f640e6a2629 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d1ffb700 1 -- 172.21.15.89:0/3313242756 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4011830 con 0x7fc9d41316c0 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d1ffb700 1 -- 172.21.15.89:0/3313242756 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4011990 con 0x7fc9d41316c0 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d1ffb700 1 -- 172.21.15.89:0/3313242756 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4011d20 con 0x7fc9d41316c0 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d9926700 1 -- 172.21.15.89:0/3313242756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d41316c0 msgr2=0x7fc9d4131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d9926700 1 --2- 172.21.15.89:0/3313242756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d41316c0 0x7fc9d4131a90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009840 tx=0x7fc9c4009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d9926700 1 -- 172.21.15.89:0/3313242756 shutdown_connections 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d9926700 1 --2- 172.21.15.89:0/3313242756 >> v2:172.21.15.89:3300/0 conn(0x7fc9d406ae80 0x7fc9d4139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d9926700 1 --2- 172.21.15.89:0/3313242756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d41316c0 0x7fc9d4131a90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.946+0000 7fc9d9926700 1 -- 172.21.15.89:0/3313242756 >> 172.21.15.89:0/3313242756 conn(0x7fc9d4066f50 msgr2=0x7fc9d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.947+0000 7fc9d9926700 1 -- 172.21.15.89:0/3313242756 shutdown_connections 2023-09-30T22:19:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.947+0000 7fc9d9926700 1 -- 172.21.15.89:0/3313242756 wait complete. 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d9926700 1 Processor -- start 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d9926700 1 -- start start 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d9926700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d406ae80 0x7fc9d4070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d9926700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d409d9f0 0x7fc9d4071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d9926700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9d41316c0 0x7fc9d40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d9926700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4119a30 con 0x7fc9d409d9f0 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d9926700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40722f0 con 0x7fc9d41316c0 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d9926700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d411b180 con 0x7fc9d406ae80 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.948+0000 7fc9d2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d406ae80 0x7fc9d4070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d406ae80 0x7fc9d4070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42278/0 (socket says 172.21.15.89:42278) 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d2ffd700 1 -- 172.21.15.89:0/4012578801 learned_addr learned my addr 172.21.15.89:0/4012578801 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d27fc700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d409d9f0 0x7fc9d4071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d37fe700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9d41316c0 0x7fc9d40719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d2ffd700 1 -- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9d41316c0 msgr2=0x7fc9d40719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d2ffd700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9d41316c0 0x7fc9d40719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d2ffd700 1 -- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d409d9f0 msgr2=0x7fc9d4071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d2ffd700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d409d9f0 0x7fc9d4071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d2ffd700 1 -- 172.21.15.89:0/4012578801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c40094a0 con 0x7fc9d406ae80 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d27fc700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d409d9f0 0x7fc9d4071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d2ffd700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d406ae80 0x7fc9d4070f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40090b0 tx=0x7fc9c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9bbfff700 1 -- 172.21.15.89:0/4012578801 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4004590 con 0x7fc9d406ae80 2023-09-30T22:19:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9bbfff700 1 -- 172.21.15.89:0/4012578801 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c401f070 con 0x7fc9d406ae80 2023-09-30T22:19:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d9926700 1 -- 172.21.15.89:0/4012578801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d411abe0 con 0x7fc9d406ae80 2023-09-30T22:19:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9bbfff700 1 -- 172.21.15.89:0/4012578801 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40038e0 con 0x7fc9d406ae80 2023-09-30T22:19:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.949+0000 7fc9d9926700 1 -- 172.21.15.89:0/4012578801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d4119480 con 0x7fc9d406ae80 2023-09-30T22:19:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.950+0000 7fc9bbfff700 1 -- 172.21.15.89:0/4012578801 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc9c4003a40 con 0x7fc9d406ae80 2023-09-30T22:19:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.950+0000 7fc9bbfff700 1 -- 172.21.15.89:0/4012578801 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc9c4021070 con 0x7fc9d406ae80 2023-09-30T22:19:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.950+0000 7fc9b9ffb700 1 -- 172.21.15.89:0/4012578801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac0052e0 con 0x7fc9d406ae80 2023-09-30T22:19:36.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.955+0000 7fc9bbfff700 1 -- 172.21.15.89:0/4012578801 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9c4010380 con 0x7fc9d406ae80 2023-09-30T22:19:36.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.991+0000 7fe465bd7700 1 Processor -- start 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.991+0000 7fe465bd7700 1 -- start start 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe465bd7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46009de00 0x7fe46009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe465bd7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe46013d1f0 0x7fe46013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe465bd7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46009a810 con 0x7fe46006a770 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe465bd7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46006ab40 con 0x7fe46013d1f0 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe465bd7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe46009ebc0 con 0x7fe46009de00 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45f7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe46006a770 0x7fe46009d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47192/0 (socket says 172.21.15.89:47192) 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46009de00 0x7fe46009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45f7fe700 1 -- 172.21.15.89:0/1760970669 learned_addr learned my addr 172.21.15.89:0/1760970669 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:36.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45ffff700 1 --2- 172.21.15.89:0/1760970669 >> v2:172.21.15.89:3300/0 conn(0x7fe46013d1f0 0x7fe46013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45effd700 1 -- 172.21.15.89:0/1760970669 >> v2:172.21.15.89:3300/0 conn(0x7fe46013d1f0 msgr2=0x7fe46013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45effd700 1 --2- 172.21.15.89:0/1760970669 >> v2:172.21.15.89:3300/0 conn(0x7fe46013d1f0 0x7fe46013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45effd700 1 -- 172.21.15.89:0/1760970669 >> v1:172.21.15.89:6789/0 conn(0x7fe46006a770 legacy=0x7fe46009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45effd700 1 -- 172.21.15.89:0/1760970669 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46009e920 con 0x7fe46009de00 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45effd700 1 --2- 172.21.15.89:0/1760970669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46009de00 0x7fe46009e250 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe454009840 tx=0x7fe454009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d516ad0433201e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45e7fc700 1 -- 172.21.15.89:0/1760970669 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454011830 con 0x7fe46009de00 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45e7fc700 1 -- 172.21.15.89:0/1760970669 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe454011990 con 0x7fe46009de00 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.992+0000 7fe45e7fc700 1 -- 172.21.15.89:0/1760970669 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454011d20 con 0x7fe46009de00 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.993+0000 7fe465bd7700 1 -- 172.21.15.89:0/1760970669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46009de00 msgr2=0x7fe46009e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.993+0000 7fe465bd7700 1 --2- 172.21.15.89:0/1760970669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46009de00 0x7fe46009e250 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe454009840 tx=0x7fe454009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:36.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.993+0000 7fe465bd7700 1 -- 172.21.15.89:0/1760970669 shutdown_connections 2023-09-30T22:19:36.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.993+0000 7fe465bd7700 1 --2- 172.21.15.89:0/1760970669 >> v2:172.21.15.89:3300/0 conn(0x7fe46013d1f0 0x7fe46013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.993+0000 7fe465bd7700 1 --2- 172.21.15.89:0/1760970669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46009de00 0x7fe46009e250 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.993+0000 7fe465bd7700 1 -- 172.21.15.89:0/1760970669 >> 172.21.15.89:0/1760970669 conn(0x7fe460066f50 msgr2=0x7fe46009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:36.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.993+0000 7fe465bd7700 1 -- 172.21.15.89:0/1760970669 shutdown_connections 2023-09-30T22:19:36.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.993+0000 7fe465bd7700 1 -- 172.21.15.89:0/1760970669 wait complete. 2023-09-30T22:19:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.994+0000 7fe465bd7700 1 Processor -- start 2023-09-30T22:19:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe465bd7700 1 -- start start 2023-09-30T22:19:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe465bd7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46006a770 0x7fe4601260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe465bd7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe46009de00 0x7fe4601265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe465bd7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe46013d1f0 0x7fe460126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe465bd7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601290c0 con 0x7fe46013d1f0 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe465bd7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600a3840 con 0x7fe46009de00 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe465bd7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe46009f1a0 con 0x7fe46006a770 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe45f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46006a770 0x7fe4601260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe45f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46006a770 0x7fe4601260a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42294/0 (socket says 172.21.15.89:42294) 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe45f7fe700 1 -- 172.21.15.89:0/2988585087 learned_addr learned my addr 172.21.15.89:0/2988585087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe45effd700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe46009de00 0x7fe4601265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.995+0000 7fe45ffff700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe46013d1f0 0x7fe460126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe45f7fe700 1 -- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe46009de00 msgr2=0x7fe4601265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe45f7fe700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe46009de00 0x7fe4601265e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe45f7fe700 1 -- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe46013d1f0 msgr2=0x7fe460126b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe45f7fe700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe46013d1f0 0x7fe460126b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe45f7fe700 1 -- 172.21.15.89:0/2988585087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540094a0 con 0x7fe46006a770 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe45effd700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe46009de00 0x7fe4601265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe45f7fe700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46006a770 0x7fe4601260a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe45000c520 tx=0x7fe45000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe447fff700 1 -- 172.21.15.89:0/2988585087 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450005d70 con 0x7fe46006a770 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe447fff700 1 -- 172.21.15.89:0/2988585087 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4500070d0 con 0x7fe46006a770 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe460127a90 con 0x7fe46006a770 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe447fff700 1 -- 172.21.15.89:0/2988585087 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450007440 con 0x7fe46006a770 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.996+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe460071840 con 0x7fe46006a770 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.997+0000 7fe447fff700 1 -- 172.21.15.89:0/2988585087 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe4500075a0 con 0x7fe46006a770 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.997+0000 7fe447fff700 1 -- 172.21.15.89:0/2988585087 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe45003ea00 con 0x7fe46006a770 2023-09-30T22:19:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:36.997+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe460074ea0 con 0x7fe46006a770 2023-09-30T22:19:37.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.002+0000 7fe447fff700 1 -- 172.21.15.89:0/2988585087 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe450019b40 con 0x7fe46006a770 2023-09-30T22:19:37.076 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:37.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.100+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f608c125e10 con 0x7f608c09e000 2023-09-30T22:19:37.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.100+0000 7f6081ffb700 1 -- 172.21.15.89:0/127361796 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f607c031070 con 0x7f608c09e000 2023-09-30T22:19:37.101 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.102+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09e000 msgr2=0x7f608c1227d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.102+0000 7f609336b700 1 --2- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09e000 0x7f608c1227d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f607c000c00 tx=0x7f607c004750 comp rx=0 tx=0).stop 2023-09-30T22:19:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.102+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 shutdown_connections 2023-09-30T22:19:37.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.102+0000 7f609336b700 1 --2- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f608c09e410 0x7f608c122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.102+0000 7f609336b700 1 --2- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c09d6f0 0x7f608c122290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.103+0000 7f609336b700 1 --2- 172.21.15.89:0/127361796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f608c09e000 0x7f608c1227d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.103+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 >> 172.21.15.89:0/127361796 conn(0x7f608c066f50 msgr2=0x7f608c130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:37.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.103+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 shutdown_connections 2023-09-30T22:19:37.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.103+0000 7f609336b700 1 -- 172.21.15.89:0/127361796 wait complete. 2023-09-30T22:19:37.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:37.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.161+0000 7fc9b9ffb700 1 -- 172.21.15.89:0/4012578801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc9ac005590 con 0x7fc9d406ae80 2023-09-30T22:19:37.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.161+0000 7fc9bbfff700 1 -- 172.21.15.89:0/4012578801 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9c402eb90 con 0x7fc9d406ae80 2023-09-30T22:19:37.161 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 -- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d406ae80 msgr2=0x7fc9d4070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d406ae80 0x7fc9d4070f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40090b0 tx=0x7fc9c4004750 comp rx=0 tx=0).stop 2023-09-30T22:19:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 -- 172.21.15.89:0/4012578801 shutdown_connections 2023-09-30T22:19:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9d41316c0 0x7fc9d40719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9d406ae80 0x7fc9d4070f20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 --2- 172.21.15.89:0/4012578801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d409d9f0 0x7fc9d4071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 -- 172.21.15.89:0/4012578801 >> 172.21.15.89:0/4012578801 conn(0x7fc9d4066f50 msgr2=0x7fc9d4130190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:37.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 -- 172.21.15.89:0/4012578801 shutdown_connections 2023-09-30T22:19:37.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.163+0000 7fc9b9ffb700 1 -- 172.21.15.89:0/4012578801 wait complete. 2023-09-30T22:19:37.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:37.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.200+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe46009a3d0 con 0x7fe46006a770 2023-09-30T22:19:37.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.200+0000 7fe447fff700 1 -- 172.21.15.89:0/2988585087 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe450017030 con 0x7fe46006a770 2023-09-30T22:19:37.200 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:37.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46006a770 msgr2=0x7fe4601260a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:37.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46006a770 0x7fe4601260a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe45000c520 tx=0x7fe45000c830 comp rx=0 tx=0).stop 2023-09-30T22:19:37.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 shutdown_connections 2023-09-30T22:19:37.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe46013d1f0 0x7fe460126b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe46006a770 0x7fe4601260a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 --2- 172.21.15.89:0/2988585087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe46009de00 0x7fe4601265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 >> 172.21.15.89:0/2988585087 conn(0x7fe460066f50 msgr2=0x7fe4600a1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:37.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 shutdown_connections 2023-09-30T22:19:37.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.202+0000 7fe465bd7700 1 -- 172.21.15.89:0/2988585087 wait complete. 2023-09-30T22:19:37.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.205+0000 7f84150ab700 1 Processor -- start 2023-09-30T22:19:37.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.205+0000 7f84150ab700 1 -- start start 2023-09-30T22:19:37.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.205+0000 7f84150ab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f841009e1d0 0x7f8410135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:37.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.205+0000 7f84150ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f8410138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:37.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.205+0000 7f84150ab700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841009f130 con 0x7f841009d6f0 2023-09-30T22:19:37.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.205+0000 7f84150ab700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841009f520 con 0x7f841009e1d0 2023-09-30T22:19:37.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.205+0000 7f84150ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f841009e930 con 0x7f8410135e50 2023-09-30T22:19:37.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840f59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f8410138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:37.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f841009e1d0 0x7f8410135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:37.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840f59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f8410138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42306/0 (socket says 172.21.15.89:42306) 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840f59e700 1 -- 172.21.15.89:0/3296280791 learned_addr learned my addr 172.21.15.89:0/3296280791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f841009e1d0 0x7f8410135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39204/0 (socket says 172.21.15.89:39204) 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840f59e700 1 -- 172.21.15.89:0/3296280791 >> v2:172.21.15.89:3300/0 conn(0x7f841009e1d0 msgr2=0x7f8410135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840f59e700 1 --2- 172.21.15.89:0/3296280791 >> v2:172.21.15.89:3300/0 conn(0x7f841009e1d0 0x7f8410135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840f59e700 1 -- 172.21.15.89:0/3296280791 >> v1:172.21.15.89:6789/0 conn(0x7f841009d6f0 legacy=0x7f841009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840f59e700 1 -- 172.21.15.89:0/3296280791 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841009b2d0 con 0x7f8410135e50 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840f59e700 1 --2- 172.21.15.89:0/3296280791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f8410138230 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f840800d150 tx=0x7f840800d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d84fe33768d94c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840dd9b700 1 -- 172.21.15.89:0/3296280791 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408010070 con 0x7f8410135e50 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840dd9b700 1 -- 172.21.15.89:0/3296280791 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840800dd10 con 0x7f8410135e50 2023-09-30T22:19:37.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.206+0000 7f840dd9b700 1 -- 172.21.15.89:0/3296280791 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800b310 con 0x7f8410135e50 2023-09-30T22:19:37.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.207+0000 7f84150ab700 1 -- 172.21.15.89:0/3296280791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 msgr2=0x7f8410138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:37.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.207+0000 7f84150ab700 1 --2- 172.21.15.89:0/3296280791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f8410138230 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f840800d150 tx=0x7f840800d460 comp rx=0 tx=0).stop 2023-09-30T22:19:37.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.207+0000 7f84150ab700 1 -- 172.21.15.89:0/3296280791 shutdown_connections 2023-09-30T22:19:37.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.207+0000 7f84150ab700 1 --2- 172.21.15.89:0/3296280791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f8410138230 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.207+0000 7f84150ab700 1 --2- 172.21.15.89:0/3296280791 >> v2:172.21.15.89:3300/0 conn(0x7f841009e1d0 0x7f8410135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.207+0000 7f84150ab700 1 -- 172.21.15.89:0/3296280791 >> 172.21.15.89:0/3296280791 conn(0x7f8410066f50 msgr2=0x7f841009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:37.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.207+0000 7f84150ab700 1 -- 172.21.15.89:0/3296280791 shutdown_connections 2023-09-30T22:19:37.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.207+0000 7f84150ab700 1 -- 172.21.15.89:0/3296280791 wait complete. 2023-09-30T22:19:37.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f84150ab700 1 Processor -- start 2023-09-30T22:19:37.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f84150ab700 1 -- start start 2023-09-30T22:19:37.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f84150ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841009d6f0 0x7f841011b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:37.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f84150ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f841009e1d0 0x7f841011b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:37.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f84150ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f841011bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:37.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f84150ab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841011d100 con 0x7f841009d6f0 2023-09-30T22:19:37.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f84150ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841011ec20 con 0x7f841009e1d0 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f84150ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84100a1e20 con 0x7f8410135e50 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841009d6f0 0x7f841011b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f840f59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f841011bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841009d6f0 0x7f841011b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39206/0 (socket says 172.21.15.89:39206) 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f840ed9d700 1 -- 172.21.15.89:0/3902397490 learned_addr learned my addr 172.21.15.89:0/3902397490 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.209+0000 7f840e59c700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f841009e1d0 0x7f841011b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f840f59e700 1 -- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f841009e1d0 msgr2=0x7f841011b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f840f59e700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f841009e1d0 0x7f841011b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f840f59e700 1 -- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841009d6f0 msgr2=0x7f841011b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f840f59e700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841009d6f0 0x7f841011b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f840f59e700 1 -- 172.21.15.89:0/3902397490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840800bd40 con 0x7f8410135e50 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f840ed9d700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841009d6f0 0x7f841011b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f840f59e700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f841011bf00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8408017000 tx=0x7f8408004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f83f77fe700 1 -- 172.21.15.89:0/3902397490 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408009e40 con 0x7f8410135e50 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f83f77fe700 1 -- 172.21.15.89:0/3902397490 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840800b4c0 con 0x7f8410135e50 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841009c700 con 0x7f8410135e50 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f83f77fe700 1 -- 172.21.15.89:0/3902397490 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408004260 con 0x7f8410135e50 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.210+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841011f690 con 0x7f8410135e50 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.211+0000 7f83f77fe700 1 -- 172.21.15.89:0/3902397490 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f84080043c0 con 0x7f8410135e50 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.211+0000 7f83f77fe700 1 -- 172.21.15.89:0/3902397490 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f84080435c0 con 0x7f8410135e50 2023-09-30T22:19:37.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.211+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8410074ea0 con 0x7f8410135e50 2023-09-30T22:19:37.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.216+0000 7f83f77fe700 1 -- 172.21.15.89:0/3902397490 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f840801e410 con 0x7f8410135e50 2023-09-30T22:19:37.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:37.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.400+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f841006c360 con 0x7f8410135e50 2023-09-30T22:19:37.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.401+0000 7f83f77fe700 1 -- 172.21.15.89:0/3902397490 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f840801db90 con 0x7f8410135e50 2023-09-30T22:19:37.401 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 msgr2=0x7f841011bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f841011bf00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8408017000 tx=0x7f8408004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 shutdown_connections 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8410135e50 0x7f841011bf00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841009d6f0 0x7f841011b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 --2- 172.21.15.89:0/3902397490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f841009e1d0 0x7f841011b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 >> 172.21.15.89:0/3902397490 conn(0x7f8410066f50 msgr2=0x7f84100a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 shutdown_connections 2023-09-30T22:19:37.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:37.403+0000 7f84150ab700 1 -- 172.21.15.89:0/3902397490 wait complete. 2023-09-30T22:19:37.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:38.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:38.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:38.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.248+0000 7ff3b348b700 1 Processor -- start 2023-09-30T22:19:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b348b700 1 -- start start 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b348b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 0x7ff3ac133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b348b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3ac09de50 0x7ff3ac09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b348b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac06ef30 con 0x7ff3ac09e760 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b348b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac06fef0 con 0x7ff3ac09de50 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b348b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1334b0 con 0x7ff3ac1338a0 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b0a26700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3ac09de50 0x7ff3ac09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b0a26700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3ac09de50 0x7ff3ac09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39218/0 (socket says 172.21.15.89:39218) 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b0a26700 1 -- 172.21.15.89:0/1093843957 learned_addr learned my addr 172.21.15.89:0/1093843957 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.249+0000 7ff3b1227700 1 --2- 172.21.15.89:0/1093843957 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 0x7ff3ac133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.250+0000 7ff3b0a26700 1 -- 172.21.15.89:0/1093843957 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 msgr2=0x7ff3ac133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.250+0000 7ff3b0a26700 1 --2- 172.21.15.89:0/1093843957 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 0x7ff3ac133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.250+0000 7ff3b0a26700 1 -- 172.21.15.89:0/1093843957 >> v1:172.21.15.89:6789/0 conn(0x7ff3ac09e760 legacy=0x7ff3ac13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.250+0000 7ff3b0a26700 1 -- 172.21.15.89:0/1093843957 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac06fc30 con 0x7ff3ac09de50 2023-09-30T22:19:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.250+0000 7ff3b0a26700 1 --2- 172.21.15.89:0/1093843957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac09e220 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0098f0 tx=0x7ff39c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f4c711f8a3303c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.250+0000 7ff3a3fff700 1 -- 172.21.15.89:0/1093843957 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c011830 con 0x7ff3ac09de50 2023-09-30T22:19:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.250+0000 7ff3a3fff700 1 -- 172.21.15.89:0/1093843957 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c011990 con 0x7ff3ac09de50 2023-09-30T22:19:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.250+0000 7ff3a3fff700 1 -- 172.21.15.89:0/1093843957 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c011d20 con 0x7ff3ac09de50 2023-09-30T22:19:38.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.251+0000 7ff3b348b700 1 -- 172.21.15.89:0/1093843957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 msgr2=0x7ff3ac09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.251+0000 7ff3b348b700 1 --2- 172.21.15.89:0/1093843957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac09e220 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0098f0 tx=0x7ff39c009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:38.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.251+0000 7ff3b348b700 1 -- 172.21.15.89:0/1093843957 shutdown_connections 2023-09-30T22:19:38.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.251+0000 7ff3b348b700 1 --2- 172.21.15.89:0/1093843957 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 0x7ff3ac133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.251+0000 7ff3b348b700 1 --2- 172.21.15.89:0/1093843957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac09e220 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.251+0000 7ff3b348b700 1 -- 172.21.15.89:0/1093843957 >> 172.21.15.89:0/1093843957 conn(0x7ff3ac0a2aa0 msgr2=0x7ff3ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:38.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.251+0000 7ff3b348b700 1 -- 172.21.15.89:0/1093843957 shutdown_connections 2023-09-30T22:19:38.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.251+0000 7ff3b348b700 1 -- 172.21.15.89:0/1093843957 wait complete. 2023-09-30T22:19:38.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b348b700 1 Processor -- start 2023-09-30T22:19:38.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b348b700 1 -- start start 2023-09-30T22:19:38.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b348b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac11f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b348b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3ac09e760 0x7ff3ac11fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b348b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 0x7ff3ac1c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b348b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac122870 con 0x7ff3ac09de50 2023-09-30T22:19:38.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b348b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac121410 con 0x7ff3ac1338a0 2023-09-30T22:19:38.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b348b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac122a50 con 0x7ff3ac09e760 2023-09-30T22:19:38.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b1227700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac11f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b1227700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac11f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39224/0 (socket says 172.21.15.89:39224) 2023-09-30T22:19:38.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b1227700 1 -- 172.21.15.89:0/299309964 learned_addr learned my addr 172.21.15.89:0/299309964 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:38.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.252+0000 7ff3b0a26700 1 --2- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3ac09e760 0x7ff3ac11fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3b1227700 1 -- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3ac09e760 msgr2=0x7ff3ac11fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3b1227700 1 --2- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3ac09e760 0x7ff3ac11fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3b1227700 1 -- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 msgr2=0x7ff3ac1c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3b1227700 1 --2- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 0x7ff3ac1c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3b1227700 1 -- 172.21.15.89:0/299309964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c009550 con 0x7ff3ac09de50 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3b1227700 1 --2- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac11f800 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800e9b0 tx=0x7ff3a800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3a1ffb700 1 -- 172.21.15.89:0/299309964 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8009710 con 0x7ff3ac09de50 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3a1ffb700 1 -- 172.21.15.89:0/299309964 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a800c850 con 0x7ff3ac09de50 2023-09-30T22:19:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac0a1110 con 0x7ff3ac09de50 2023-09-30T22:19:38.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3a1ffb700 1 -- 172.21.15.89:0/299309964 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800cbc0 con 0x7ff3ac09de50 2023-09-30T22:19:38.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.253+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac121070 con 0x7ff3ac09de50 2023-09-30T22:19:38.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.254+0000 7ff3a1ffb700 1 -- 172.21.15.89:0/299309964 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff3a800cd20 con 0x7ff3ac09de50 2023-09-30T22:19:38.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.254+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac074ea0 con 0x7ff3ac09de50 2023-09-30T22:19:38.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.254+0000 7ff3a1ffb700 1 -- 172.21.15.89:0/299309964 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff3a803f840 con 0x7ff3ac09de50 2023-09-30T22:19:38.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.263+0000 7ff3a1ffb700 1 -- 172.21.15.89:0/299309964 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff3a8021080 con 0x7ff3ac09de50 2023-09-30T22:19:38.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.306+0000 7fd79512c700 1 Processor -- start 2023-09-30T22:19:38.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.306+0000 7fd79512c700 1 -- start start 2023-09-30T22:19:38.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd79512c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd79009e930 0x7fd79009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd79512c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009f240 0x7fd7901317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd79512c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f650 0x7fd790131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd79512c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd790070940 con 0x7fd79009e930 2023-09-30T22:19:38.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd79512c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79006fd60 con 0x7fd79009f240 2023-09-30T22:19:38.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd79512c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd79006fa80 con 0x7fd79009f650 2023-09-30T22:19:38.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd79009e930 0x7fd79009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78f59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f650 0x7fd790131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd79009e930 0x7fd79009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39226/0 (socket says 172.21.15.89:39226) 2023-09-30T22:19:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78ed9d700 1 -- 172.21.15.89:0/2957012666 learned_addr learned my addr 172.21.15.89:0/2957012666 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78f59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f650 0x7fd790131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42336/0 (socket says 172.21.15.89:42336) 2023-09-30T22:19:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78e59c700 1 --2- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009f240 0x7fd7901317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78f59e700 1 -- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009f240 msgr2=0x7fd7901317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78f59e700 1 --2- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009f240 0x7fd7901317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78f59e700 1 -- 172.21.15.89:0/2957012666 >> v2:172.21.15.89:3300/0 conn(0x7fd79009e930 msgr2=0x7fd79009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:38.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78f59e700 1 --2- 172.21.15.89:0/2957012666 >> v2:172.21.15.89:3300/0 conn(0x7fd79009e930 0x7fd79009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78f59e700 1 -- 172.21.15.89:0/2957012666 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79009fb60 con 0x7fd79009f650 2023-09-30T22:19:38.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78ed9d700 1 --2- 172.21.15.89:0/2957012666 >> v2:172.21.15.89:3300/0 conn(0x7fd79009e930 0x7fd79009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:38.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.307+0000 7fd78e59c700 1 --2- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009f240 0x7fd7901317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:38.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd78f59e700 1 --2- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f650 0x7fd790131d30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd788009d90 tx=0x7fd78800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f51f6ee7f45f74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:38.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd78dd9b700 1 -- 172.21.15.89:0/2957012666 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800e070 con 0x7fd79009f650 2023-09-30T22:19:38.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd78dd9b700 1 -- 172.21.15.89:0/2957012666 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78800bb20 con 0x7fd79009f650 2023-09-30T22:19:38.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd78dd9b700 1 -- 172.21.15.89:0/2957012666 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788012430 con 0x7fd79009f650 2023-09-30T22:19:38.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 -- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f650 msgr2=0x7fd790131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 --2- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f650 0x7fd790131d30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd788009d90 tx=0x7fd78800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:38.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 -- 172.21.15.89:0/2957012666 shutdown_connections 2023-09-30T22:19:38.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 --2- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f650 0x7fd790131d30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 --2- 172.21.15.89:0/2957012666 >> v2:172.21.15.89:3300/0 conn(0x7fd79009e930 0x7fd79009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 --2- 172.21.15.89:0/2957012666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009f240 0x7fd7901317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 -- 172.21.15.89:0/2957012666 >> 172.21.15.89:0/2957012666 conn(0x7fd7900a4210 msgr2=0x7fd7900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:38.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 -- 172.21.15.89:0/2957012666 shutdown_connections 2023-09-30T22:19:38.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.308+0000 7fd79512c700 1 -- 172.21.15.89:0/2957012666 wait complete. 2023-09-30T22:19:38.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd79512c700 1 Processor -- start 2023-09-30T22:19:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd79512c700 1 -- start start 2023-09-30T22:19:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd79512c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009e930 0x7fd79006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd79512c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f240 0x7fd79006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd79512c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd79009f650 0x7fd79006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd79512c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901a8810 con 0x7fd79009f650 2023-09-30T22:19:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd79512c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901a9850 con 0x7fd79009e930 2023-09-30T22:19:38.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd79512c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd79006cd10 con 0x7fd79009f240 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd78e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f240 0x7fd79006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009e930 0x7fd79006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd78f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd79009f650 0x7fd79006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd78e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f240 0x7fd79006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42348/0 (socket says 172.21.15.89:42348) 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009e930 0x7fd79006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57046/0 (socket says 172.21.15.89:57046) 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.310+0000 7fd78e59c700 1 -- 172.21.15.89:0/2492069538 learned_addr learned my addr 172.21.15.89:0/2492069538 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd78ed9d700 1 -- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f240 msgr2=0x7fd79006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd78ed9d700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f240 0x7fd79006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd78ed9d700 1 -- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd79009f650 msgr2=0x7fd79006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd78ed9d700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd79009f650 0x7fd79006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd78ed9d700 1 -- 172.21.15.89:0/2492069538 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780009550 con 0x7fd79009e930 2023-09-30T22:19:38.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd78e59c700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f240 0x7fd79006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd78f59e700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd79009f650 0x7fd79006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd78ed9d700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009e930 0x7fd79006b8b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd78000e9e0 tx=0x7fd78000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd7777fe700 1 -- 172.21.15.89:0/2492069538 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000c640 con 0x7fd79009e930 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd7777fe700 1 -- 172.21.15.89:0/2492069538 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78000c7a0 con 0x7fd79009e930 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd7777fe700 1 -- 172.21.15.89:0/2492069538 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000cb30 con 0x7fd79009e930 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7840094a0 con 0x7fd79009e930 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.311+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd788009a70 con 0x7fd79009e930 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.312+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7901a0b30 con 0x7fd79009e930 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.312+0000 7fd7777fe700 1 -- 172.21.15.89:0/2492069538 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd78000cc90 con 0x7fd79009e930 2023-09-30T22:19:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.312+0000 7fd7777fe700 1 -- 172.21.15.89:0/2492069538 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd78003f660 con 0x7fd79009e930 2023-09-30T22:19:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.317+0000 7fd7777fe700 1 -- 172.21.15.89:0/2492069538 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd78001bdc0 con 0x7fd79009e930 2023-09-30T22:19:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.363+0000 7f4682e63700 1 Processor -- start 2023-09-30T22:19:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.363+0000 7f4682e63700 1 -- start start 2023-09-30T22:19:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.363+0000 7f4682e63700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c0a0a30 0x7f467c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4682e63700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4682e63700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c0a2a70 con 0x7f467c133930 2023-09-30T22:19:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4682e63700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0a01d0 con 0x7f467c0a0a30 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4682e63700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f467c09e2b0 con 0x7f467c09d630 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4680bff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c0a0a30 0x7f467c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4681400700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f467c133930 0x7f467c133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47226/0 (socket says 172.21.15.89:47226) 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4681400700 1 -- 172.21.15.89:0/1482552292 learned_addr learned my addr 172.21.15.89:0/1482552292 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f467bfff700 1 --2- 172.21.15.89:0/1482552292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f467b7fe700 1 -- 172.21.15.89:0/1482552292 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917106398 0 0) 0x7f467c0a2a70 con 0x7f467c133930 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4680bff700 1 -- 172.21.15.89:0/1482552292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 msgr2=0x7f467c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4680bff700 1 --2- 172.21.15.89:0/1482552292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4680bff700 1 -- 172.21.15.89:0/1482552292 >> v1:172.21.15.89:6789/0 conn(0x7f467c133930 legacy=0x7f467c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4680bff700 1 -- 172.21.15.89:0/1482552292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c09dc50 con 0x7f467c0a0a30 2023-09-30T22:19:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f4680bff700 1 --2- 172.21.15.89:0/1482552292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c0a0a30 0x7f467c0a0e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4670009870 tx=0x7f4670009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6fe1ca06e1e3762 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f467b7fe700 1 -- 172.21.15.89:0/1482552292 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670011830 con 0x7f467c0a0a30 2023-09-30T22:19:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f467b7fe700 1 -- 172.21.15.89:0/1482552292 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4670011990 con 0x7f467c0a0a30 2023-09-30T22:19:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.364+0000 7f467b7fe700 1 -- 172.21.15.89:0/1482552292 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670011d20 con 0x7f467c0a0a30 2023-09-30T22:19:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.365+0000 7f4682e63700 1 -- 172.21.15.89:0/1482552292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c0a0a30 msgr2=0x7f467c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.365+0000 7f4682e63700 1 --2- 172.21.15.89:0/1482552292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c0a0a30 0x7f467c0a0e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4670009870 tx=0x7f4670009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.365+0000 7f4682e63700 1 -- 172.21.15.89:0/1482552292 shutdown_connections 2023-09-30T22:19:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.365+0000 7f4682e63700 1 --2- 172.21.15.89:0/1482552292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c0a0a30 0x7f467c0a0e00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.365+0000 7f4682e63700 1 --2- 172.21.15.89:0/1482552292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.365+0000 7f4682e63700 1 -- 172.21.15.89:0/1482552292 >> 172.21.15.89:0/1482552292 conn(0x7f467c0a3050 msgr2=0x7f467c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.365+0000 7f4682e63700 1 -- 172.21.15.89:0/1482552292 shutdown_connections 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.365+0000 7f4682e63700 1 -- 172.21.15.89:0/1482552292 wait complete. 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.366+0000 7f4682e63700 1 Processor -- start 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.366+0000 7f4682e63700 1 -- start start 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4682e63700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c11f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4682e63700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467c0a0a30 0x7f467c11f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4682e63700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c133930 0x7f467c1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4682e63700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c11ed60 con 0x7f467c0a0a30 2023-09-30T22:19:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4682e63700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1224a0 con 0x7f467c133930 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4682e63700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f467c120490 con 0x7f467c09d630 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f467bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467c0a0a30 0x7f467c11f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f467bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467c0a0a30 0x7f467c11f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39240/0 (socket says 172.21.15.89:39240) 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f467bfff700 1 -- 172.21.15.89:0/2433130091 learned_addr learned my addr 172.21.15.89:0/2433130091 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4681400700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c133930 0x7f467c1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4680bff700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c11f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4681400700 1 -- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 msgr2=0x7f467c11f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4681400700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c11f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4681400700 1 -- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467c0a0a30 msgr2=0x7f467c11f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4681400700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467c0a0a30 0x7f467c11f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4681400700 1 -- 172.21.15.89:0/2433130091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46700094a0 con 0x7f467c133930 2023-09-30T22:19:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4680bff700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c11f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f4681400700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c133930 0x7f467c1c7e80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f467400cbe0 tx=0x7f467400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.367+0000 7f46797fa700 1 -- 172.21.15.89:0/2433130091 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467400aa40 con 0x7f467c133930 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.368+0000 7f46797fa700 1 -- 172.21.15.89:0/2433130091 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467400abc0 con 0x7f467c133930 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.368+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c0a5360 con 0x7f467c133930 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.368+0000 7f46797fa700 1 -- 172.21.15.89:0/2433130091 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740070d0 con 0x7f467c133930 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.368+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c121e00 con 0x7f467c133930 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.369+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c1222e0 con 0x7f467c133930 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.369+0000 7f46797fa700 1 -- 172.21.15.89:0/2433130091 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4674007230 con 0x7f467c133930 2023-09-30T22:19:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.369+0000 7f46797fa700 1 -- 172.21.15.89:0/2433130091 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4674045470 con 0x7f467c133930 2023-09-30T22:19:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.373+0000 7f46797fa700 1 -- 172.21.15.89:0/2433130091 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f467c1222e0 con 0x7f467c133930 2023-09-30T22:19:38.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:38.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.479+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff3ac1226a0 con 0x7ff3ac09de50 2023-09-30T22:19:38.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.480+0000 7ff3a1ffb700 1 -- 172.21.15.89:0/299309964 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3a8017020 con 0x7ff3ac09de50 2023-09-30T22:19:38.481 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:38.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 msgr2=0x7ff3ac11f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 --2- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac11f800 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800e9b0 tx=0x7ff3a800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 shutdown_connections 2023-09-30T22:19:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 --2- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3ac1338a0 0x7ff3ac1c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 --2- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac09de50 0x7ff3ac11f800 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 --2- 172.21.15.89:0/299309964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3ac09e760 0x7ff3ac11fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 >> 172.21.15.89:0/299309964 conn(0x7ff3ac0a2aa0 msgr2=0x7ff3ac09edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 shutdown_connections 2023-09-30T22:19:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.482+0000 7ff3b348b700 1 -- 172.21.15.89:0/299309964 wait complete. 2023-09-30T22:19:38.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.513+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd79006d520 con 0x7fd79009e930 2023-09-30T22:19:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.514+0000 7fd7777fe700 1 -- 172.21.15.89:0/2492069538 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd780021020 con 0x7fd79009e930 2023-09-30T22:19:38.514 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009e930 msgr2=0x7fd79006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009e930 0x7fd79006b8b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd78000e9e0 tx=0x7fd78000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 shutdown_connections 2023-09-30T22:19:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd79009f650 0x7fd79006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd79009e930 0x7fd79006b8b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 --2- 172.21.15.89:0/2492069538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009f240 0x7fd79006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 >> 172.21.15.89:0/2492069538 conn(0x7fd7900a4210 msgr2=0x7fd7901b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 shutdown_connections 2023-09-30T22:19:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.516+0000 7fd79512c700 1 -- 172.21.15.89:0/2492069538 wait complete. 2023-09-30T22:19:38.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:38.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.543+0000 7f8b88daf700 1 Processor -- start 2023-09-30T22:19:38.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b88daf700 1 -- start start 2023-09-30T22:19:38.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b88daf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b88daf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b841317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b88daf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b8409f650 0x7f8b84131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b88daf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b84070940 con 0x7f8b8409f650 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b88daf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8406fd60 con 0x7f8b8409f240 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b88daf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8406fa80 con 0x7f8b8409e930 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b8259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b82d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b8409f650 0x7f8b84131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b81d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b841317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b8259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42360/0 (socket says 172.21.15.89:42360) 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b82d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b8409f650 0x7f8b84131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39254/0 (socket says 172.21.15.89:39254) 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b8259c700 1 -- 172.21.15.89:0/524672507 learned_addr learned my addr 172.21.15.89:0/524672507 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:38.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.544+0000 7f8b81d9b700 1 --2- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b841317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57074/0 (socket says 172.21.15.89:57074) 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8259c700 1 -- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 msgr2=0x7f8b841317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8259c700 1 --2- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b841317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8259c700 1 -- 172.21.15.89:0/524672507 >> v2:172.21.15.89:3300/0 conn(0x7f8b8409f650 msgr2=0x7f8b84131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8259c700 1 --2- 172.21.15.89:0/524672507 >> v2:172.21.15.89:3300/0 conn(0x7f8b8409f650 0x7f8b84131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8259c700 1 -- 172.21.15.89:0/524672507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8409fb60 con 0x7f8b8409e930 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b81d9b700 1 --2- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b841317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b82d9d700 1 --2- 172.21.15.89:0/524672507 >> v2:172.21.15.89:3300/0 conn(0x7f8b8409f650 0x7f8b84131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8259c700 1 --2- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8409ed00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b740098f0 tx=0x7f8b74009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c1d7cb03758d57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8159a700 1 -- 172.21.15.89:0/524672507 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74011830 con 0x7f8b8409e930 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8159a700 1 -- 172.21.15.89:0/524672507 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74011990 con 0x7f8b8409e930 2023-09-30T22:19:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b8159a700 1 -- 172.21.15.89:0/524672507 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74011d20 con 0x7f8b8409e930 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b88daf700 1 -- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 msgr2=0x7f8b8409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.545+0000 7f8b88daf700 1 --2- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8409ed00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b740098f0 tx=0x7f8b74009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.546+0000 7f8b88daf700 1 -- 172.21.15.89:0/524672507 shutdown_connections 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.546+0000 7f8b88daf700 1 --2- 172.21.15.89:0/524672507 >> v2:172.21.15.89:3300/0 conn(0x7f8b8409f650 0x7f8b84131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.546+0000 7f8b88daf700 1 --2- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8409ed00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.546+0000 7f8b88daf700 1 --2- 172.21.15.89:0/524672507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b841317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.546+0000 7f8b88daf700 1 -- 172.21.15.89:0/524672507 >> 172.21.15.89:0/524672507 conn(0x7f8b840a4210 msgr2=0x7f8b840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.546+0000 7f8b88daf700 1 -- 172.21.15.89:0/524672507 shutdown_connections 2023-09-30T22:19:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.546+0000 7f8b88daf700 1 -- 172.21.15.89:0/524672507 wait complete. 2023-09-30T22:19:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b88daf700 1 Processor -- start 2023-09-30T22:19:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b88daf700 1 -- start start 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b88daf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b88daf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b8406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b88daf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b8409f650 0x7f8b8406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b88daf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8406cd80 con 0x7f8b8409f650 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b88daf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b841b71d0 con 0x7f8b8409f240 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b88daf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b841a9380 con 0x7f8b8409e930 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b8259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b8259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42374/0 (socket says 172.21.15.89:42374) 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b8259c700 1 -- 172.21.15.89:0/3366695552 learned_addr learned my addr 172.21.15.89:0/3366695552 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b81d9b700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b8406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b82d9d700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b8409f650 0x7f8b8406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b8259c700 1 -- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 msgr2=0x7f8b8406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b8259c700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b8406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b8259c700 1 -- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b8409f650 msgr2=0x7f8b8406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b8259c700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b8409f650 0x7f8b8406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b8259c700 1 -- 172.21.15.89:0/3366695552 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780094a0 con 0x7f8b8409e930 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.547+0000 7f8b82d9d700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b8409f650 0x7f8b8406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.548+0000 7f8b8259c700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8406b8b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009c40 tx=0x7f8b74004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.548+0000 7f8b72ffd700 1 -- 172.21.15.89:0/3366695552 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74004240 con 0x7f8b8409e930 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.548+0000 7f8b72ffd700 1 -- 172.21.15.89:0/3366695552 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7401f070 con 0x7f8b8409e930 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.548+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c009a70 con 0x7f8b8409e930 2023-09-30T22:19:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.548+0000 7f8b72ffd700 1 -- 172.21.15.89:0/3366695552 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740045c0 con 0x7f8b8409e930 2023-09-30T22:19:38.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.548+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b74009550 con 0x7f8b8409e930 2023-09-30T22:19:38.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.548+0000 7f8b72ffd700 1 -- 172.21.15.89:0/3366695552 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b74021030 con 0x7f8b8409e930 2023-09-30T22:19:38.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.549+0000 7f8b72ffd700 1 -- 172.21.15.89:0/3366695552 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8b74040740 con 0x7f8b8409e930 2023-09-30T22:19:38.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.549+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b841a0b30 con 0x7f8b8409e930 2023-09-30T22:19:38.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.553+0000 7f8b72ffd700 1 -- 172.21.15.89:0/3366695552 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b74015080 con 0x7f8b8409e930 2023-09-30T22:19:38.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.573+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f467c1222e0 con 0x7f467c133930 2023-09-30T22:19:38.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.573+0000 7f46797fa700 1 -- 172.21.15.89:0/2433130091 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f467c1222e0 con 0x7f467c133930 2023-09-30T22:19:38.574 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:38.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c133930 msgr2=0x7f467c1c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c133930 0x7f467c1c7e80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f467400cbe0 tx=0x7f467400cef0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 shutdown_connections 2023-09-30T22:19:38.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f467c133930 0x7f467c1c7e80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f467c09d630 0x7f467c11f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 --2- 172.21.15.89:0/2433130091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467c0a0a30 0x7f467c11f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 >> 172.21.15.89:0/2433130091 conn(0x7f467c0a3050 msgr2=0x7f467c1bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 shutdown_connections 2023-09-30T22:19:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.575+0000 7f4682e63700 1 -- 172.21.15.89:0/2433130091 wait complete. 2023-09-30T22:19:38.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.740+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8b840a3610 con 0x7f8b8409e930 2023-09-30T22:19:38.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.741+0000 7f8b72ffd700 1 -- 172.21.15.89:0/3366695552 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b7401a450 con 0x7f8b8409e930 2023-09-30T22:19:38.742 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 msgr2=0x7f8b8406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8406b8b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009c40 tx=0x7f8b74004750 comp rx=0 tx=0).stop 2023-09-30T22:19:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 shutdown_connections 2023-09-30T22:19:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b8409f650 0x7f8b8406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b8409e930 0x7f8b8406b8b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 --2- 172.21.15.89:0/3366695552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b8409f240 0x7f8b8406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:38.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 >> 172.21.15.89:0/3366695552 conn(0x7f8b840a4210 msgr2=0x7f8b841b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:38.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 shutdown_connections 2023-09-30T22:19:38.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:38.743+0000 7f8b88daf700 1 -- 172.21.15.89:0/3366695552 wait complete. 2023-09-30T22:19:38.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:39.496 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:39.531 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:39.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:39.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.636+0000 7f8f772bb700 1 Processor -- start 2023-09-30T22:19:39.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.636+0000 7f8f772bb700 1 -- start start 2023-09-30T22:19:39.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f772bb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f70138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f772bb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f7013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f772bb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7009b120 con 0x7f8f7009d6f0 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f772bb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7006b460 con 0x7f8f7009e1d0 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f772bb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7009a7e0 con 0x7f8f70139170 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f74856700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f70138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f75858700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f7013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f75858700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f7013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42388/0 (socket says 172.21.15.89:42388) 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f74856700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f70138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57094/0 (socket says 172.21.15.89:57094) 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f75858700 1 -- 172.21.15.89:0/3212106925 learned_addr learned my addr 172.21.15.89:0/3212106925 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f75858700 1 -- 172.21.15.89:0/3212106925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 msgr2=0x7f8f70138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f75858700 1 --2- 172.21.15.89:0/3212106925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f70138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f75858700 1 -- 172.21.15.89:0/3212106925 >> v1:172.21.15.89:6789/0 conn(0x7f8f7009d6f0 legacy=0x7f8f7009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f75858700 1 -- 172.21.15.89:0/3212106925 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7009a500 con 0x7f8f70139170 2023-09-30T22:19:39.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f74856700 1 --2- 172.21.15.89:0/3212106925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f70138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:39.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f75858700 1 --2- 172.21.15.89:0/3212106925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f7013b550 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009eb0 tx=0x7f8f6800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9428a17f9e8edbf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:39.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.637+0000 7f8f67fff700 1 -- 172.21.15.89:0/3212106925 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6800e070 con 0x7f8f70139170 2023-09-30T22:19:39.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f67fff700 1 -- 172.21.15.89:0/3212106925 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6800b8d0 con 0x7f8f70139170 2023-09-30T22:19:39.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f67fff700 1 -- 172.21.15.89:0/3212106925 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6800bc80 con 0x7f8f70139170 2023-09-30T22:19:39.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f772bb700 1 -- 172.21.15.89:0/3212106925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 msgr2=0x7f8f7013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f772bb700 1 --2- 172.21.15.89:0/3212106925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f7013b550 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009eb0 tx=0x7f8f6800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f772bb700 1 -- 172.21.15.89:0/3212106925 shutdown_connections 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f772bb700 1 --2- 172.21.15.89:0/3212106925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f7013b550 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f772bb700 1 --2- 172.21.15.89:0/3212106925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f70138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f772bb700 1 -- 172.21.15.89:0/3212106925 >> 172.21.15.89:0/3212106925 conn(0x7f8f70066f50 msgr2=0x7f8f7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f772bb700 1 -- 172.21.15.89:0/3212106925 shutdown_connections 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.638+0000 7f8f772bb700 1 -- 172.21.15.89:0/3212106925 wait complete. 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.639+0000 7f8f772bb700 1 Processor -- start 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.639+0000 7f8f772bb700 1 -- start start 2023-09-30T22:19:39.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f772bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f7009d6f0 0x7f8f701c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f772bb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f701c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f772bb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f70120890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f772bb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f70122400 con 0x7f8f7009d6f0 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f772bb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f70124460 con 0x7f8f7009e1d0 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f772bb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7009c780 con 0x7f8f70139170 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f74856700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f701c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f74856700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f701c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57100/0 (socket says 172.21.15.89:57100) 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f74856700 1 -- 172.21.15.89:0/3174650398 learned_addr learned my addr 172.21.15.89:0/3174650398 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f75858700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f70120890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f75057700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f7009d6f0 0x7f8f701c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f74856700 1 -- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 msgr2=0x7f8f70120890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f74856700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f70120890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f74856700 1 -- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f7009d6f0 msgr2=0x7f8f701c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f74856700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f7009d6f0 0x7f8f701c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.640+0000 7f8f74856700 1 -- 172.21.15.89:0/3174650398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600094a0 con 0x7f8f7009e1d0 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.641+0000 7f8f74856700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f701c8980 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000ec10 tx=0x7f8f6000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.641+0000 7f8f65ffb700 1 -- 172.21.15.89:0/3174650398 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000c7f0 con 0x7f8f7009e1d0 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.641+0000 7f8f65ffb700 1 -- 172.21.15.89:0/3174650398 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6000c950 con 0x7f8f7009e1d0 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.641+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f68009b30 con 0x7f8f7009e1d0 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.641+0000 7f8f65ffb700 1 -- 172.21.15.89:0/3174650398 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000ccc0 con 0x7f8f7009e1d0 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.641+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f70124aa0 con 0x7f8f7009e1d0 2023-09-30T22:19:39.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.642+0000 7f8f65ffb700 1 -- 172.21.15.89:0/3174650398 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f60013470 con 0x7f8f7009e1d0 2023-09-30T22:19:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.642+0000 7f8f65ffb700 1 -- 172.21.15.89:0/3174650398 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8f6003f740 con 0x7f8f7009e1d0 2023-09-30T22:19:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.642+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f70074ea0 con 0x7f8f7009e1d0 2023-09-30T22:19:39.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.651+0000 7f8f65ffb700 1 -- 172.21.15.89:0/3174650398 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f60013720 con 0x7f8f7009e1d0 2023-09-30T22:19:39.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.676+0000 7f32b1c3f700 1 Processor -- start 2023-09-30T22:19:39.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.676+0000 7f32b1c3f700 1 -- start start 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32b1c3f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09e930 0x7f32ac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32b1c3f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32ac09f240 0x7f32ac1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32b1c3f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ac070940 con 0x7f32ac09f650 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32b1c3f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac06fd60 con 0x7f32ac09f240 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32b1c3f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac06fa80 con 0x7f32ac09e930 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32ab7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09e930 0x7f32ac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32ab7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09e930 0x7f32ac09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42408/0 (socket says 172.21.15.89:42408) 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32ab7fe700 1 -- 172.21.15.89:0/1455184867 learned_addr learned my addr 172.21.15.89:0/1455184867 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:39.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32a2ffd700 1 --2- 172.21.15.89:0/1455184867 >> v2:172.21.15.89:3300/0 conn(0x7f32ac09f240 0x7f32ac1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32ab7fe700 1 -- 172.21.15.89:0/1455184867 >> v2:172.21.15.89:3300/0 conn(0x7f32ac09f240 msgr2=0x7f32ac1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:39.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32ab7fe700 1 --2- 172.21.15.89:0/1455184867 >> v2:172.21.15.89:3300/0 conn(0x7f32ac09f240 0x7f32ac1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32ab7fe700 1 -- 172.21.15.89:0/1455184867 >> v1:172.21.15.89:6789/0 conn(0x7f32ac09f650 legacy=0x7f32ac131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32ab7fe700 1 -- 172.21.15.89:0/1455184867 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac09fb60 con 0x7f32ac09e930 2023-09-30T22:19:39.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32ab7fe700 1 --2- 172.21.15.89:0/1455184867 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09e930 0x7f32ac09ed00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098f0 tx=0x7f329c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75398e25d6624979 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:39.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.677+0000 7f32aaffd700 1 -- 172.21.15.89:0/1455184867 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c011830 con 0x7f32ac09e930 2023-09-30T22:19:39.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32aaffd700 1 -- 172.21.15.89:0/1455184867 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c011990 con 0x7f32ac09e930 2023-09-30T22:19:39.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32aaffd700 1 -- 172.21.15.89:0/1455184867 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c011d20 con 0x7f32ac09e930 2023-09-30T22:19:39.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1455184867 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09e930 msgr2=0x7f32ac09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32b1c3f700 1 --2- 172.21.15.89:0/1455184867 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09e930 0x7f32ac09ed00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098f0 tx=0x7f329c009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:39.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1455184867 shutdown_connections 2023-09-30T22:19:39.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32b1c3f700 1 --2- 172.21.15.89:0/1455184867 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09e930 0x7f32ac09ed00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32b1c3f700 1 --2- 172.21.15.89:0/1455184867 >> v2:172.21.15.89:3300/0 conn(0x7f32ac09f240 0x7f32ac1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1455184867 >> 172.21.15.89:0/1455184867 conn(0x7f32ac0a4210 msgr2=0x7f32ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1455184867 shutdown_connections 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.678+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1455184867 wait complete. 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.680+0000 7f32b1c3f700 1 Processor -- start 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.680+0000 7f32b1c3f700 1 -- start start 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.680+0000 7f32b1c3f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32ac09e930 0x7f32ac1a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.680+0000 7f32b1c3f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32ac09f240 0x7f32ac1a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.680+0000 7f32b1c3f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09f650 0x7f32ac1a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.680+0000 7f32b1c3f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1a5f20 con 0x7f32ac09f240 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.680+0000 7f32b1c3f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1a6fe0 con 0x7f32ac09e930 2023-09-30T22:19:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.680+0000 7f32b1c3f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac071d20 con 0x7f32ac09f650 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32ac09f240 0x7f32ac1a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32abfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09f650 0x7f32ac1a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32ac09f240 0x7f32ac1a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39294/0 (socket says 172.21.15.89:39294) 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 -- 172.21.15.89:0/1560830542 learned_addr learned my addr 172.21.15.89:0/1560830542 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32ab7fe700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32ac09e930 0x7f32ac1a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 -- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09f650 msgr2=0x7f32ac1a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09f650 0x7f32ac1a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 -- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32ac09e930 msgr2=0x7f32ac1a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32ac09e930 0x7f32ac1a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 -- 172.21.15.89:0/1560830542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c009550 con 0x7f32ac09f240 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32abfff700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09f650 0x7f32ac1a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a2ffd700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32ac09f240 0x7f32ac1a4270 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f329400e980 tx=0x7f329400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a8ff9700 1 -- 172.21.15.89:0/1560830542 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294009710 con 0x7f32ac09f240 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a8ff9700 1 -- 172.21.15.89:0/1560830542 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400c690 con 0x7f32ac09f240 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac0a0190 con 0x7f32ac09f240 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32a8ff9700 1 -- 172.21.15.89:0/1560830542 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400ca20 con 0x7f32ac09f240 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.681+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac1a5940 con 0x7f32ac09f240 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.682+0000 7f32a8ff9700 1 -- 172.21.15.89:0/1560830542 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f329400cb80 con 0x7f32ac09f240 2023-09-30T22:19:39.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.682+0000 7f32a8ff9700 1 -- 172.21.15.89:0/1560830542 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f329403f6d0 con 0x7f32ac09f240 2023-09-30T22:19:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.683+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac074ea0 con 0x7f32ac09f240 2023-09-30T22:19:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.688+0000 7f32a8ff9700 1 -- 172.21.15.89:0/1560830542 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32940232f0 con 0x7f32ac09f240 2023-09-30T22:19:39.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43bacc8700 1 Processor -- start 2023-09-30T22:19:39.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43bacc8700 1 -- start start 2023-09-30T22:19:39.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43bacc8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409e930 0x7f43b409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43bacc8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f43b409f240 0x7f43b41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43bacc8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 0x7f43b4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43bacc8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b4070940 con 0x7f43b409f240 2023-09-30T22:19:39.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43bacc8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b406fd60 con 0x7f43b409e930 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43bacc8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43b406fa80 con 0x7f43b409f650 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43b3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f43b409f240 0x7f43b41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43b3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f43b409f240 0x7f43b41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39300/0 (socket says 172.21.15.89:39300) 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43b3fff700 1 -- 172.21.15.89:0/3049353960 learned_addr learned my addr 172.21.15.89:0/3049353960 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.720+0000 7f43b8a64700 1 --2- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409e930 0x7f43b409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b3fff700 1 -- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 msgr2=0x7f43b4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b3fff700 1 --2- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 0x7f43b4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b3fff700 1 -- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409e930 msgr2=0x7f43b409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b3fff700 1 --2- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409e930 0x7f43b409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b3fff700 1 -- 172.21.15.89:0/3049353960 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b409fb60 con 0x7f43b409f240 2023-09-30T22:19:39.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b3fff700 1 --2- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409f240 0x7f43b41317f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f43a00098f0 tx=0x7f43a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de318ee796777b78 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:39.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b37fe700 1 -- 172.21.15.89:0/3049353960 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0011830 con 0x7f43b409f240 2023-09-30T22:19:39.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b37fe700 1 -- 172.21.15.89:0/3049353960 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a0011990 con 0x7f43b409f240 2023-09-30T22:19:39.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43b37fe700 1 -- 172.21.15.89:0/3049353960 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0011d20 con 0x7f43b409f240 2023-09-30T22:19:39.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43bacc8700 1 -- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409f240 msgr2=0x7f43b41317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.721+0000 7f43bacc8700 1 --2- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409f240 0x7f43b41317f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f43a00098f0 tx=0x7f43a0009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:39.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.722+0000 7f43bacc8700 1 -- 172.21.15.89:0/3049353960 shutdown_connections 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.722+0000 7f43bacc8700 1 --2- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 0x7f43b4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.722+0000 7f43bacc8700 1 --2- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409e930 0x7f43b409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.722+0000 7f43bacc8700 1 --2- 172.21.15.89:0/3049353960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409f240 0x7f43b41317f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.722+0000 7f43bacc8700 1 -- 172.21.15.89:0/3049353960 >> 172.21.15.89:0/3049353960 conn(0x7f43b40a4210 msgr2=0x7f43b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.722+0000 7f43bacc8700 1 -- 172.21.15.89:0/3049353960 shutdown_connections 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.722+0000 7f43bacc8700 1 -- 172.21.15.89:0/3049353960 wait complete. 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.722+0000 7f43bacc8700 1 Processor -- start 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43bacc8700 1 -- start start 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43bacc8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409e930 0x7f43b41c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43bacc8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409f240 0x7f43b41c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43bacc8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 0x7f43b41c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43bacc8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b41a7370 con 0x7f43b409e930 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43bacc8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b41a99e0 con 0x7f43b409f240 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43bacc8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43b41c5c90 con 0x7f43b409f650 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409f240 0x7f43b41c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43b8a64700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409e930 0x7f43b41c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409f240 0x7f43b41c5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57138/0 (socket says 172.21.15.89:57138) 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43b8a64700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409e930 0x7f43b41c5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39316/0 (socket says 172.21.15.89:39316) 2023-09-30T22:19:39.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43b3fff700 1 -- 172.21.15.89:0/1978454832 learned_addr learned my addr 172.21.15.89:0/1978454832 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:39.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.723+0000 7f43b9265700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 0x7f43b41c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b3fff700 1 -- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 msgr2=0x7f43b41c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b3fff700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 0x7f43b41c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b3fff700 1 -- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409e930 msgr2=0x7f43b41c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b3fff700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409e930 0x7f43b41c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b3fff700 1 -- 172.21.15.89:0/1978454832 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a0009550 con 0x7f43b409f240 2023-09-30T22:19:39.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b9265700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 0x7f43b41c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:39.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b8a64700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409e930 0x7f43b41c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:39.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b3fff700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409f240 0x7f43b41c5750 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f43a0000c00 tx=0x7f43a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:39.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b17fa700 1 -- 172.21.15.89:0/1978454832 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0004300 con 0x7f43b409f240 2023-09-30T22:19:39.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b17fa700 1 -- 172.21.15.89:0/1978454832 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a001f070 con 0x7f43b409f240 2023-09-30T22:19:39.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43b17fa700 1 -- 172.21.15.89:0/1978454832 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a00036a0 con 0x7f43b409f240 2023-09-30T22:19:39.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b40a0190 con 0x7f43b409f240 2023-09-30T22:19:39.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.724+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b41a8d20 con 0x7f43b409f240 2023-09-30T22:19:39.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.725+0000 7f43b17fa700 1 -- 172.21.15.89:0/1978454832 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f43a0021030 con 0x7f43b409f240 2023-09-30T22:19:39.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.725+0000 7f43b17fa700 1 -- 172.21.15.89:0/1978454832 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f43a0040740 con 0x7f43b409f240 2023-09-30T22:19:39.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.725+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b4074ea0 con 0x7f43b409f240 2023-09-30T22:19:39.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.730+0000 7f43b17fa700 1 -- 172.21.15.89:0/1978454832 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43a0015080 con 0x7f43b409f240 2023-09-30T22:19:39.756 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:39.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.849+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8f7006c360 con 0x7f8f7009e1d0 2023-09-30T22:19:39.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.850+0000 7f8f65ffb700 1 -- 172.21.15.89:0/3174650398 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f60021020 con 0x7f8f7009e1d0 2023-09-30T22:19:39.852 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 msgr2=0x7f8f701c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f701c8980 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000ec10 tx=0x7f8f6000ef20 comp rx=0 tx=0).stop 2023-09-30T22:19:39.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 shutdown_connections 2023-09-30T22:19:39.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f70139170 0x7f8f70120890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f7009d6f0 0x7f8f701c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 --2- 172.21.15.89:0/3174650398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f7009e1d0 0x7f8f701c8980 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 >> 172.21.15.89:0/3174650398 conn(0x7f8f70066f50 msgr2=0x7f8f70136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:39.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 shutdown_connections 2023-09-30T22:19:39.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.852+0000 7f8f772bb700 1 -- 172.21.15.89:0/3174650398 wait complete. 2023-09-30T22:19:39.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:39.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.882+0000 7f8e99431700 1 Processor -- start 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.882+0000 7f8e99431700 1 -- start start 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e99431700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e9406b250 0x7f8e94136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e99431700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e94136d80 0x7f8e94139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e99431700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9409ece0 con 0x7f8e9406a770 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e99431700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9409e760 con 0x7f8e9406b250 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e99431700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9409e040 con 0x7f8e94136d80 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e927fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e9406b250 0x7f8e94136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e927fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e9406b250 0x7f8e94136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39322/0 (socket says 172.21.15.89:39322) 2023-09-30T22:19:39.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e927fc700 1 -- 172.21.15.89:0/3183146126 learned_addr learned my addr 172.21.15.89:0/3183146126 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:39.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e92ffd700 1 --1- 172.21.15.89:0/3183146126 >> v1:172.21.15.89:6789/0 conn(0x7f8e9406a770 0x7f8e9406ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47248/0 (socket says 172.21.15.89:47248) 2023-09-30T22:19:39.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e937fe700 1 --2- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e94136d80 0x7f8e94139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e927fc700 1 -- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e94136d80 msgr2=0x7f8e94139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:39.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e927fc700 1 --2- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e94136d80 0x7f8e94139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e927fc700 1 -- 172.21.15.89:0/3183146126 >> v1:172.21.15.89:6789/0 conn(0x7f8e9406a770 legacy=0x7f8e9406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e927fc700 1 -- 172.21.15.89:0/3183146126 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9409f630 con 0x7f8e9406b250 2023-09-30T22:19:39.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.883+0000 7f8e937fe700 1 --2- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e94136d80 0x7f8e94139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e927fc700 1 --2- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e9406b250 0x7f8e94136840 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8e880098f0 tx=0x7f8e88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d800e122e37f92c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e91ffb700 1 -- 172.21.15.89:0/3183146126 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88011830 con 0x7f8e9406b250 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e91ffb700 1 -- 172.21.15.89:0/3183146126 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e88011990 con 0x7f8e9406b250 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e91ffb700 1 -- 172.21.15.89:0/3183146126 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88011d20 con 0x7f8e9406b250 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e99431700 1 -- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e9406b250 msgr2=0x7f8e94136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e99431700 1 --2- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e9406b250 0x7f8e94136840 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8e880098f0 tx=0x7f8e88009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e99431700 1 -- 172.21.15.89:0/3183146126 shutdown_connections 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e99431700 1 --2- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e94136d80 0x7f8e94139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e99431700 1 --2- 172.21.15.89:0/3183146126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e9406b250 0x7f8e94136840 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e99431700 1 -- 172.21.15.89:0/3183146126 >> 172.21.15.89:0/3183146126 conn(0x7f8e94066f50 msgr2=0x7f8e9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:39.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e99431700 1 -- 172.21.15.89:0/3183146126 shutdown_connections 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.884+0000 7f8e99431700 1 -- 172.21.15.89:0/3183146126 wait complete. 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.885+0000 7f8e99431700 1 Processor -- start 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.885+0000 7f8e99431700 1 -- start start 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e99431700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e9406a770 0x7f8e9411d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e99431700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e9406b250 0x7f8e9411d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e99431700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e94136d80 0x7f8e941c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e99431700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9411cda0 con 0x7f8e94136d80 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e99431700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9406ea70 con 0x7f8e9406b250 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e99431700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94071e60 con 0x7f8e9406a770 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e927fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e9406b250 0x7f8e9411d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e94136d80 0x7f8e941c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e94136d80 0x7f8e941c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39338/0 (socket says 172.21.15.89:39338) 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e927fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e9406b250 0x7f8e9411d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57140/0 (socket says 172.21.15.89:57140) 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e937fe700 1 -- 172.21.15.89:0/3953226585 learned_addr learned my addr 172.21.15.89:0/3953226585 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e92ffd700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e9406a770 0x7f8e9411d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e927fc700 1 -- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e9406a770 msgr2=0x7f8e9411d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e927fc700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e9406a770 0x7f8e9411d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e927fc700 1 -- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e94136d80 msgr2=0x7f8e941c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e927fc700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e94136d80 0x7f8e941c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e927fc700 1 -- 172.21.15.89:0/3953226585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c009a70 con 0x7f8e9406b250 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e937fe700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e94136d80 0x7f8e941c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:39.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.886+0000 7f8e927fc700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e9406b250 0x7f8e9411d650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8e88016000 tx=0x7f8e88004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:39.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.887+0000 7f8e7b7fe700 1 -- 172.21.15.89:0/3953226585 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88004400 con 0x7f8e9406b250 2023-09-30T22:19:39.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.887+0000 7f8e7b7fe700 1 -- 172.21.15.89:0/3953226585 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8801f070 con 0x7f8e9406b250 2023-09-30T22:19:39.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.887+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e88009550 con 0x7f8e9406b250 2023-09-30T22:19:39.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.887+0000 7f8e7b7fe700 1 -- 172.21.15.89:0/3953226585 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e880038e0 con 0x7f8e9406b250 2023-09-30T22:19:39.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.887+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9411fff0 con 0x7f8e9406b250 2023-09-30T22:19:39.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.887+0000 7f8e7b7fe700 1 -- 172.21.15.89:0/3953226585 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e88021030 con 0x7f8e9406b250 2023-09-30T22:19:39.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.888+0000 7f8e7b7fe700 1 -- 172.21.15.89:0/3953226585 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e88040890 con 0x7f8e9406b250 2023-09-30T22:19:39.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.888+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9411fc70 con 0x7f8e9406b250 2023-09-30T22:19:39.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.888+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f32ac1a73d0 con 0x7f32ac09f240 2023-09-30T22:19:39.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.889+0000 7f32a8ff9700 1 -- 172.21.15.89:0/1560830542 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32ac1a73d0 con 0x7f32ac09f240 2023-09-30T22:19:39.902 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.891+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32ac09f240 msgr2=0x7f32ac1a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.891+0000 7f32b1c3f700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32ac09f240 0x7f32ac1a4270 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f329400e980 tx=0x7f329400ec90 comp rx=0 tx=0).stop 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.891+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 shutdown_connections 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.891+0000 7f32b1c3f700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32ac09f650 0x7f32ac1a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.891+0000 7f32b1c3f700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32ac09e930 0x7f32ac1a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.891+0000 7f32b1c3f700 1 --2- 172.21.15.89:0/1560830542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32ac09f240 0x7f32ac1a4270 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.891+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 >> 172.21.15.89:0/1560830542 conn(0x7f32ac0a4210 msgr2=0x7f32ac1b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.892+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 shutdown_connections 2023-09-30T22:19:39.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.892+0000 7f32b1c3f700 1 -- 172.21.15.89:0/1560830542 wait complete. 2023-09-30T22:19:39.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.892+0000 7f8e7b7fe700 1 -- 172.21.15.89:0/3953226585 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e88015020 con 0x7f8e9406b250 2023-09-30T22:19:39.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:39.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.929+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f43b41a7ad0 con 0x7f43b409f240 2023-09-30T22:19:39.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.929+0000 7f43b17fa700 1 -- 172.21.15.89:0/1978454832 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43a00239c0 con 0x7f43b409f240 2023-09-30T22:19:39.930 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409f240 msgr2=0x7f43b41c5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409f240 0x7f43b41c5750 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f43a0000c00 tx=0x7f43a0004750 comp rx=0 tx=0).stop 2023-09-30T22:19:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 shutdown_connections 2023-09-30T22:19:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43b409f650 0x7f43b41c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43b409e930 0x7f43b41c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 --2- 172.21.15.89:0/1978454832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43b409f240 0x7f43b41c5750 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 >> 172.21.15.89:0/1978454832 conn(0x7f43b40a4210 msgr2=0x7f43b41bb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 shutdown_connections 2023-09-30T22:19:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:39.931+0000 7f43bacc8700 1 -- 172.21.15.89:0/1978454832 wait complete. 2023-09-30T22:19:39.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:40.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.082+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8e94074ea0 con 0x7f8e9406b250 2023-09-30T22:19:40.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.082+0000 7f8e7b7fe700 1 -- 172.21.15.89:0/3953226585 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e88004560 con 0x7f8e9406b250 2023-09-30T22:19:40.086 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:40.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e9406b250 msgr2=0x7f8e9411d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:40.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e9406b250 0x7f8e9411d650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8e88016000 tx=0x7f8e88004750 comp rx=0 tx=0).stop 2023-09-30T22:19:40.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 shutdown_connections 2023-09-30T22:19:40.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e94136d80 0x7f8e941c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:40.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e9406a770 0x7f8e9411d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:40.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 --2- 172.21.15.89:0/3953226585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e9406b250 0x7f8e9411d650 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:40.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 >> 172.21.15.89:0/3953226585 conn(0x7f8e94066f50 msgr2=0x7f8e94135da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:40.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 shutdown_connections 2023-09-30T22:19:40.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.084+0000 7f8e99431700 1 -- 172.21.15.89:0/3953226585 wait complete. 2023-09-30T22:19:40.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:40.866 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:40.906 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:40.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:40.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.998+0000 7fad54cf5700 1 Processor -- start 2023-09-30T22:19:40.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.998+0000 7fad54cf5700 1 -- start start 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad54cf5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad54cf5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad5009e000 0x7fad501339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad54cf5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e410 0x7fad50133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad54cf5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5006b460 con 0x7fad5009e000 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad54cf5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5009b120 con 0x7fad5009d6f0 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad54cf5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad5009a7e0 con 0x7fad5009e410 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4e59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e410 0x7fad50133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad5009e000 0x7fad501339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4e59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54892/0 (socket says 172.21.15.89:54892) 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad5009e000 0x7fad501339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35626/0 (socket says 172.21.15.89:35626) 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e410 0x7fad50133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44072/0 (socket says 172.21.15.89:44072) 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4e59c700 1 -- 172.21.15.89:0/3123634457 learned_addr learned my addr 172.21.15.89:0/3123634457 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4ed9d700 1 -- 172.21.15.89:0/3123634457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 msgr2=0x7fad5009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4ed9d700 1 --2- 172.21.15.89:0/3123634457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4ed9d700 1 -- 172.21.15.89:0/3123634457 >> v2:172.21.15.89:3300/0 conn(0x7fad5009e000 msgr2=0x7fad501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4ed9d700 1 --2- 172.21.15.89:0/3123634457 >> v2:172.21.15.89:3300/0 conn(0x7fad5009e000 0x7fad501339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4ed9d700 1 -- 172.21.15.89:0/3123634457 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5009a500 con 0x7fad5009e410 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4dd9b700 1 --2- 172.21.15.89:0/3123634457 >> v2:172.21.15.89:3300/0 conn(0x7fad5009e000 0x7fad501339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:41.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:40.999+0000 7fad4e59c700 1 --2- 172.21.15.89:0/3123634457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:41.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.000+0000 7fad4ed9d700 1 --2- 172.21.15.89:0/3123634457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e410 0x7fad50133f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad48009dc0 tx=0x7fad4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1785cbc143f14437 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:41.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.000+0000 7fad4d59a700 1 -- 172.21.15.89:0/3123634457 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800e070 con 0x7fad5009e410 2023-09-30T22:19:41.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.000+0000 7fad4d59a700 1 -- 172.21.15.89:0/3123634457 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4800bae0 con 0x7fad5009e410 2023-09-30T22:19:41.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.000+0000 7fad4d59a700 1 -- 172.21.15.89:0/3123634457 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800be90 con 0x7fad5009e410 2023-09-30T22:19:41.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.000+0000 7fad54cf5700 1 -- 172.21.15.89:0/3123634457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e410 msgr2=0x7fad50133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.000+0000 7fad54cf5700 1 --2- 172.21.15.89:0/3123634457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e410 0x7fad50133f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad48009dc0 tx=0x7fad4800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.001+0000 7fad54cf5700 1 -- 172.21.15.89:0/3123634457 shutdown_connections 2023-09-30T22:19:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.001+0000 7fad54cf5700 1 --2- 172.21.15.89:0/3123634457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e410 0x7fad50133f30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.001+0000 7fad54cf5700 1 --2- 172.21.15.89:0/3123634457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.001+0000 7fad54cf5700 1 --2- 172.21.15.89:0/3123634457 >> v2:172.21.15.89:3300/0 conn(0x7fad5009e000 0x7fad501339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.001+0000 7fad54cf5700 1 -- 172.21.15.89:0/3123634457 >> 172.21.15.89:0/3123634457 conn(0x7fad50066f50 msgr2=0x7fad5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.001+0000 7fad54cf5700 1 -- 172.21.15.89:0/3123634457 shutdown_connections 2023-09-30T22:19:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.001+0000 7fad54cf5700 1 -- 172.21.15.89:0/3123634457 wait complete. 2023-09-30T22:19:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad54cf5700 1 Processor -- start 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad54cf5700 1 -- start start 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad54cf5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009d6f0 0x7fad501c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad54cf5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e000 0x7fad501c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad54cf5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e410 0x7fad501c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad54cf5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad501a78d0 con 0x7fad5009e000 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad54cf5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad501a81a0 con 0x7fad5009e410 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad54cf5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad500a2060 con 0x7fad5009d6f0 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad4dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e000 0x7fad501c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad4e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009d6f0 0x7fad501c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad4dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e000 0x7fad501c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35636/0 (socket says 172.21.15.89:35636) 2023-09-30T22:19:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad4dd9b700 1 -- 172.21.15.89:0/2124578847 learned_addr learned my addr 172.21.15.89:0/2124578847 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad4e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009d6f0 0x7fad501c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44078/0 (socket says 172.21.15.89:44078) 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.002+0000 7fad4ed9d700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e410 0x7fad501c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad4e59c700 1 -- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e410 msgr2=0x7fad501c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad4e59c700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e410 0x7fad501c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad4e59c700 1 -- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e000 msgr2=0x7fad501c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad4e59c700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e000 0x7fad501c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad4e59c700 1 -- 172.21.15.89:0/2124578847 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40009550 con 0x7fad5009d6f0 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad4ed9d700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e410 0x7fad501c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad4e59c700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009d6f0 0x7fad501c5050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fad4000e9d0 tx=0x7fad4000ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad3effd700 1 -- 172.21.15.89:0/2124578847 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000ee70 con 0x7fad5009d6f0 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad3effd700 1 -- 172.21.15.89:0/2124578847 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad40004030 con 0x7fad5009d6f0 2023-09-30T22:19:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad3effd700 1 -- 172.21.15.89:0/2124578847 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad400043c0 con 0x7fad5009d6f0 2023-09-30T22:19:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad440094a0 con 0x7fad5009d6f0 2023-09-30T22:19:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.003+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad48009a70 con 0x7fad5009d6f0 2023-09-30T22:19:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.004+0000 7fad3effd700 1 -- 172.21.15.89:0/2124578847 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad4000c850 con 0x7fad5009d6f0 2023-09-30T22:19:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.004+0000 7fad3effd700 1 -- 172.21.15.89:0/2124578847 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fad4003f520 con 0x7fad5009d6f0 2023-09-30T22:19:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.004+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad50074ea0 con 0x7fad5009d6f0 2023-09-30T22:19:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.013+0000 7fad3effd700 1 -- 172.21.15.89:0/2124578847 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad40019950 con 0x7fad5009d6f0 2023-09-30T22:19:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.038+0000 7fb408da0700 1 Processor -- start 2023-09-30T22:19:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb408da0700 1 -- start start 2023-09-30T22:19:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb408da0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404136820 0x7fb404138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb408da0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb40413b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb408da0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40406bc00 con 0x7fb404133a80 2023-09-30T22:19:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb408da0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40406c570 con 0x7fb404139140 2023-09-30T22:19:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb408da0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4040a0040 con 0x7fb404136820 2023-09-30T22:19:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb401d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404136820 0x7fb404138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb401d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404136820 0x7fb404138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44088/0 (socket says 172.21.15.89:44088) 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb401d9b700 1 -- 172.21.15.89:0/1872433236 learned_addr learned my addr 172.21.15.89:0/1872433236 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.039+0000 7fb402d9d700 1 --2- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb40413b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb40159a700 1 -- 172.21.15.89:0/1872433236 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250134822 0 0) 0x7fb40406bc00 con 0x7fb404133a80 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb401d9b700 1 -- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 msgr2=0x7fb40413b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb401d9b700 1 --2- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb40413b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb401d9b700 1 -- 172.21.15.89:0/1872433236 >> v1:172.21.15.89:6789/0 conn(0x7fb404133a80 legacy=0x7fb404133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb401d9b700 1 -- 172.21.15.89:0/1872433236 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404133860 con 0x7fb404136820 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb402d9d700 1 --2- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb40413b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb401d9b700 1 --2- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404136820 0x7fb404138c00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009920 tx=0x7fb3f8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5ca64ee5e3e633 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb40159a700 1 -- 172.21.15.89:0/1872433236 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8011830 con 0x7fb404136820 2023-09-30T22:19:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb40159a700 1 -- 172.21.15.89:0/1872433236 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f8011990 con 0x7fb404136820 2023-09-30T22:19:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.040+0000 7fb40159a700 1 -- 172.21.15.89:0/1872433236 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8011d20 con 0x7fb404136820 2023-09-30T22:19:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.041+0000 7fb408da0700 1 -- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404136820 msgr2=0x7fb404138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.041+0000 7fb408da0700 1 --2- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404136820 0x7fb404138c00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009920 tx=0x7fb3f8009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.041+0000 7fb408da0700 1 -- 172.21.15.89:0/1872433236 shutdown_connections 2023-09-30T22:19:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.041+0000 7fb408da0700 1 --2- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb40413b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.041+0000 7fb408da0700 1 --2- 172.21.15.89:0/1872433236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404136820 0x7fb404138c00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.041+0000 7fb408da0700 1 -- 172.21.15.89:0/1872433236 >> 172.21.15.89:0/1872433236 conn(0x7fb4040a4210 msgr2=0x7fb4040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.041+0000 7fb408da0700 1 -- 172.21.15.89:0/1872433236 shutdown_connections 2023-09-30T22:19:41.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.041+0000 7fb408da0700 1 -- 172.21.15.89:0/1872433236 wait complete. 2023-09-30T22:19:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb408da0700 1 Processor -- start 2023-09-30T22:19:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb408da0700 1 -- start start 2023-09-30T22:19:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb408da0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404133a80 0x7fb4041c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb408da0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb404136820 0x7fb4041c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb408da0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb4041c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb408da0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb404070e10 con 0x7fb404136820 2023-09-30T22:19:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb408da0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40406fad0 con 0x7fb404139140 2023-09-30T22:19:41.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb408da0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb404130cc0 con 0x7fb404133a80 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb401d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb404136820 0x7fb4041c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb40259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404133a80 0x7fb4041c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb401d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb404136820 0x7fb4041c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35644/0 (socket says 172.21.15.89:35644) 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.042+0000 7fb401d9b700 1 -- 172.21.15.89:0/3910274306 learned_addr learned my addr 172.21.15.89:0/3910274306 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb402d9d700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb4041c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb40259c700 1 -- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 msgr2=0x7fb4041c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb40259c700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb4041c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb40259c700 1 -- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb404136820 msgr2=0x7fb4041c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb40259c700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb404136820 0x7fb4041c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb40259c700 1 -- 172.21.15.89:0/3910274306 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc009a70 con 0x7fb404133a80 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb401d9b700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb404136820 0x7fb4041c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb40259c700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404133a80 0x7fb4041c8430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c430 tx=0x7fb3f400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb3f2ffd700 1 -- 172.21.15.89:0/3910274306 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40054e0 con 0x7fb404133a80 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb3f2ffd700 1 -- 172.21.15.89:0/3910274306 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f4003680 con 0x7fb404133a80 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb3f2ffd700 1 -- 172.21.15.89:0/3910274306 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4003a30 con 0x7fb404133a80 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f8009550 con 0x7fb404133a80 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.043+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb404072490 con 0x7fb404133a80 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.044+0000 7fb3f2ffd700 1 -- 172.21.15.89:0/3910274306 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb3f4003b90 con 0x7fb404133a80 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.044+0000 7fb3f2ffd700 1 -- 172.21.15.89:0/3910274306 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb3f403eac0 con 0x7fb404133a80 2023-09-30T22:19:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.044+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4040a1240 con 0x7fb404133a80 2023-09-30T22:19:41.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.053+0000 7fb3f2ffd700 1 -- 172.21.15.89:0/3910274306 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb3f4017020 con 0x7fb404133a80 2023-09-30T22:19:41.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.082+0000 7f9c4439f700 1 Processor -- start 2023-09-30T22:19:41.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.082+0000 7f9c4439f700 1 -- start start 2023-09-30T22:19:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4439f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c0a0a30 0x7f9c3c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4439f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c3c138d80 0x7f9c3c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4439f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c0a2a70 con 0x7f9c3c09d630 2023-09-30T22:19:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4439f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c0a01d0 con 0x7f9c3c138d80 2023-09-30T22:19:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4439f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c09e2b0 con 0x7f9c3c0a0a30 2023-09-30T22:19:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4213b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c0a0a30 0x7f9c3c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4213b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c0a0a30 0x7f9c3c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44092/0 (socket says 172.21.15.89:44092) 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4293c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c3c138d80 0x7f9c3c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.083+0000 7f9c4213b700 1 -- 172.21.15.89:0/2177087119 learned_addr learned my addr 172.21.15.89:0/2177087119 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c4293c700 1 -- 172.21.15.89:0/2177087119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c0a0a30 msgr2=0x7f9c3c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c4293c700 1 --2- 172.21.15.89:0/2177087119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c0a0a30 0x7f9c3c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c4293c700 1 -- 172.21.15.89:0/2177087119 >> v1:172.21.15.89:6789/0 conn(0x7f9c3c09d630 legacy=0x7f9c3c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c4293c700 1 -- 172.21.15.89:0/2177087119 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c09dc50 con 0x7f9c3c138d80 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c4293c700 1 --2- 172.21.15.89:0/2177087119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 0x7f9c3c13b160 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009d90 tx=0x7f9c3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268c1d83579b3bb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c41139700 1 -- 172.21.15.89:0/2177087119 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400e070 con 0x7f9c3c138d80 2023-09-30T22:19:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c41139700 1 -- 172.21.15.89:0/2177087119 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3400bb20 con 0x7f9c3c138d80 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c41139700 1 -- 172.21.15.89:0/2177087119 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34012430 con 0x7f9c3c138d80 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c4439f700 1 -- 172.21.15.89:0/2177087119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 msgr2=0x7f9c3c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.084+0000 7f9c4439f700 1 --2- 172.21.15.89:0/2177087119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 0x7f9c3c13b160 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009d90 tx=0x7f9c3400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.085+0000 7f9c4439f700 1 -- 172.21.15.89:0/2177087119 shutdown_connections 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.085+0000 7f9c4439f700 1 --2- 172.21.15.89:0/2177087119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 0x7f9c3c13b160 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.085+0000 7f9c4439f700 1 --2- 172.21.15.89:0/2177087119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c0a0a30 0x7f9c3c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.085+0000 7f9c4439f700 1 -- 172.21.15.89:0/2177087119 >> 172.21.15.89:0/2177087119 conn(0x7f9c3c0a3050 msgr2=0x7f9c3c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.085+0000 7f9c4439f700 1 -- 172.21.15.89:0/2177087119 shutdown_connections 2023-09-30T22:19:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.085+0000 7f9c4439f700 1 -- 172.21.15.89:0/2177087119 wait complete. 2023-09-30T22:19:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.085+0000 7f9c4439f700 1 Processor -- start 2023-09-30T22:19:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4439f700 1 -- start start 2023-09-30T22:19:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4439f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c09d630 0x7f9c3c1339c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4439f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c3c0a0a30 0x7f9c3c133f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4439f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 0x7f9c3c134440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4439f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c136c30 con 0x7f9c3c138d80 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4439f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c1380c0 con 0x7f9c3c0a0a30 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4439f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0a0410 con 0x7f9c3c09d630 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4213b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c09d630 0x7f9c3c1339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4213b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c09d630 0x7f9c3c1339c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44102/0 (socket says 172.21.15.89:44102) 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4213b700 1 -- 172.21.15.89:0/1427157100 learned_addr learned my addr 172.21.15.89:0/1427157100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4193a700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c3c0a0a30 0x7f9c3c133f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4293c700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 0x7f9c3c134440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4213b700 1 -- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c3c0a0a30 msgr2=0x7f9c3c133f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4213b700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c3c0a0a30 0x7f9c3c133f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4213b700 1 -- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 msgr2=0x7f9c3c134440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4213b700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 0x7f9c3c134440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.086+0000 7f9c4213b700 1 -- 172.21.15.89:0/1427157100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c34009a70 con 0x7f9c3c09d630 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.087+0000 7f9c4213b700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c09d630 0x7f9c3c1339c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800e940 tx=0x7f9c3800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.087+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1427157100 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c380096a0 con 0x7f9c3c09d630 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.087+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1427157100 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c38013680 con 0x7f9c3c09d630 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.087+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c137540 con 0x7f9c3c09d630 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.087+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1427157100 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38013a30 con 0x7f9c3c09d630 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.087+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c137200 con 0x7f9c3c09d630 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.088+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c136e80 con 0x7f9c3c09d630 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.088+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1427157100 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c38013b90 con 0x7f9c3c09d630 2023-09-30T22:19:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.088+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1427157100 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9c3803f5e0 con 0x7f9c3c09d630 2023-09-30T22:19:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.093+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1427157100 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c3c136e80 con 0x7f9c3c09d630 2023-09-30T22:19:41.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:41.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.218+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fad501a8b20 con 0x7fad5009d6f0 2023-09-30T22:19:41.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.218+0000 7fad3effd700 1 -- 172.21.15.89:0/2124578847 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad40019300 con 0x7fad5009d6f0 2023-09-30T22:19:41.219 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009d6f0 msgr2=0x7fad501c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009d6f0 0x7fad501c5050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fad4000e9d0 tx=0x7fad4000ece0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 shutdown_connections 2023-09-30T22:19:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e410 0x7fad501c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009d6f0 0x7fad501c5050 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 --2- 172.21.15.89:0/2124578847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e000 0x7fad501c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 >> 172.21.15.89:0/2124578847 conn(0x7fad50066f50 msgr2=0x7fad501bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:41.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 shutdown_connections 2023-09-30T22:19:41.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.220+0000 7fad54cf5700 1 -- 172.21.15.89:0/2124578847 wait complete. 2023-09-30T22:19:41.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.225+0000 7f9bda701700 1 Processor -- start 2023-09-30T22:19:41.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.225+0000 7f9bda701700 1 -- start start 2023-09-30T22:19:41.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.225+0000 7f9bda701700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bd406f5d0 0x7f9bd412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.225+0000 7f9bda701700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 0x7f9bd4138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.225+0000 7f9bda701700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd406fa40 con 0x7f9bd412cf60 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.225+0000 7f9bda701700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd412cd90 con 0x7f9bd406f5d0 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.225+0000 7f9bda701700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd4071780 con 0x7f9bd4135c20 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.225+0000 7f9bd37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bd406f5d0 0x7f9bd412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bd406f5d0 0x7f9bd412ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35680/0 (socket says 172.21.15.89:35680) 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1950967574 learned_addr learned my addr 172.21.15.89:0/1950967574 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd8c9e700 1 --2- 172.21.15.89:0/1950967574 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 0x7f9bd4138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1950967574 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 msgr2=0x7f9bd4138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1950967574 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 0x7f9bd4138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1950967574 >> v1:172.21.15.89:6789/0 conn(0x7f9bd412cf60 legacy=0x7f9bd412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:41.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1950967574 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4071a40 con 0x7f9bd406f5d0 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1950967574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd406f5d0 0x7f9bd412ffc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80098f0 tx=0x7f9bc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a290debd1ff8fc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd2ffd700 1 -- 172.21.15.89:0/1950967574 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8004230 con 0x7f9bd406f5d0 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd2ffd700 1 -- 172.21.15.89:0/1950967574 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc8004390 con 0x7f9bd406f5d0 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.226+0000 7f9bd2ffd700 1 -- 172.21.15.89:0/1950967574 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80036a0 con 0x7f9bd406f5d0 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.227+0000 7f9bda701700 1 -- 172.21.15.89:0/1950967574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd406f5d0 msgr2=0x7f9bd412ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.227+0000 7f9bda701700 1 --2- 172.21.15.89:0/1950967574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd406f5d0 0x7f9bd412ffc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80098f0 tx=0x7f9bc8009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.227+0000 7f9bda701700 1 -- 172.21.15.89:0/1950967574 shutdown_connections 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.227+0000 7f9bda701700 1 --2- 172.21.15.89:0/1950967574 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 0x7f9bd4138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.227+0000 7f9bda701700 1 --2- 172.21.15.89:0/1950967574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd406f5d0 0x7f9bd412ffc0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.227+0000 7f9bda701700 1 -- 172.21.15.89:0/1950967574 >> 172.21.15.89:0/1950967574 conn(0x7f9bd40a2aa0 msgr2=0x7f9bd40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:41.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.227+0000 7f9bda701700 1 -- 172.21.15.89:0/1950967574 shutdown_connections 2023-09-30T22:19:41.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.227+0000 7f9bda701700 1 -- 172.21.15.89:0/1950967574 wait complete. 2023-09-30T22:19:41.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.228+0000 7f9bda701700 1 Processor -- start 2023-09-30T22:19:41.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.228+0000 7f9bda701700 1 -- start start 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bda701700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bd406f5d0 0x7f9bd412e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bda701700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd412cf60 0x7f9bd412eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bda701700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 0x7f9bd41c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bda701700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd412e1a0 con 0x7f9bd412cf60 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bda701700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd41312b0 con 0x7f9bd4135c20 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bda701700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd412bff0 con 0x7f9bd406f5d0 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd412cf60 0x7f9bd412eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd412cf60 0x7f9bd412eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35684/0 (socket says 172.21.15.89:35684) 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1740965859 learned_addr learned my addr 172.21.15.89:0/1740965859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd8c9e700 1 --2- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 0x7f9bd41c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:41.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bd406f5d0 msgr2=0x7f9bd412e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bd406f5d0 0x7f9bd412e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 msgr2=0x7f9bd41c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 0x7f9bd41c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 -- 172.21.15.89:0/1740965859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8009550 con 0x7f9bd412cf60 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd37fe700 1 --2- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd412cf60 0x7f9bd412eac0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009130 tx=0x7f9bc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.229+0000 7f9bd0ff9700 1 -- 172.21.15.89:0/1740965859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80036a0 con 0x7f9bd412cf60 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.230+0000 7f9bd0ff9700 1 -- 172.21.15.89:0/1740965859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc8003800 con 0x7f9bd412cf60 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.230+0000 7f9bd0ff9700 1 -- 172.21.15.89:0/1740965859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8003b70 con 0x7f9bd412cf60 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.230+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd412f000 con 0x7f9bd412cf60 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.230+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd4131d20 con 0x7f9bd412cf60 2023-09-30T22:19:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.231+0000 7f9bd0ff9700 1 -- 172.21.15.89:0/1740965859 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9bc8003cd0 con 0x7f9bd412cf60 2023-09-30T22:19:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.231+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd4129fa0 con 0x7f9bd412cf60 2023-09-30T22:19:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.231+0000 7f9bd0ff9700 1 -- 172.21.15.89:0/1740965859 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9bc80445d0 con 0x7f9bd412cf60 2023-09-30T22:19:41.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:41.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.240+0000 7f9bd0ff9700 1 -- 172.21.15.89:0/1740965859 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9bc802d310 con 0x7f9bd412cf60 2023-09-30T22:19:41.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.255+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb404074ea0 con 0x7fb404133a80 2023-09-30T22:19:41.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.256+0000 7fb3f2ffd700 1 -- 172.21.15.89:0/3910274306 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb3f4003e40 con 0x7fb404133a80 2023-09-30T22:19:41.257 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404133a80 msgr2=0x7fb4041c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404133a80 0x7fb4041c8430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c430 tx=0x7fb3f400c740 comp rx=0 tx=0).stop 2023-09-30T22:19:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 shutdown_connections 2023-09-30T22:19:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb404139140 0x7fb4041c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb404133a80 0x7fb4041c8430 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 --2- 172.21.15.89:0/3910274306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb404136820 0x7fb4041c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 >> 172.21.15.89:0/3910274306 conn(0x7fb4040a4210 msgr2=0x7fb404137450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 shutdown_connections 2023-09-30T22:19:41.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.258+0000 7fb408da0700 1 -- 172.21.15.89:0/3910274306 wait complete. 2023-09-30T22:19:41.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:41.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.298+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9c3c136e80 con 0x7f9c3c09d630 2023-09-30T22:19:41.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.298+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1427157100 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c38021020 con 0x7f9c3c09d630 2023-09-30T22:19:41.299 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:41.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c09d630 msgr2=0x7f9c3c1339c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c09d630 0x7f9c3c1339c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800e940 tx=0x7f9c3800ec50 comp rx=0 tx=0).stop 2023-09-30T22:19:41.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 shutdown_connections 2023-09-30T22:19:41.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c3c138d80 0x7f9c3c134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c3c09d630 0x7f9c3c1339c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 --2- 172.21.15.89:0/1427157100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c3c0a0a30 0x7f9c3c133f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 >> 172.21.15.89:0/1427157100 conn(0x7f9c3c0a3050 msgr2=0x7f9c3c09f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:41.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 shutdown_connections 2023-09-30T22:19:41.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.300+0000 7f9c4439f700 1 -- 172.21.15.89:0/1427157100 wait complete. 2023-09-30T22:19:41.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:41.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.429+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9bd4074ea0 con 0x7f9bd412cf60 2023-09-30T22:19:41.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.430+0000 7f9bd0ff9700 1 -- 172.21.15.89:0/1740965859 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9bc802b070 con 0x7f9bd412cf60 2023-09-30T22:19:41.431 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:41.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.431+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd412cf60 msgr2=0x7f9bd412eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.432+0000 7f9bda701700 1 --2- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd412cf60 0x7f9bd412eac0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009130 tx=0x7f9bc8004750 comp rx=0 tx=0).stop 2023-09-30T22:19:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.432+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 shutdown_connections 2023-09-30T22:19:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.432+0000 7f9bda701700 1 --2- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bd4135c20 0x7f9bd41c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.432+0000 7f9bda701700 1 --2- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bd406f5d0 0x7f9bd412e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.432+0000 7f9bda701700 1 --2- 172.21.15.89:0/1740965859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bd412cf60 0x7f9bd412eac0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.432+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 >> 172.21.15.89:0/1740965859 conn(0x7f9bd40a2aa0 msgr2=0x7f9bd406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.432+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 shutdown_connections 2023-09-30T22:19:41.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:41.432+0000 7f9bda701700 1 -- 172.21.15.89:0/1740965859 wait complete. 2023-09-30T22:19:41.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:42.236 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:42.272 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:42.314 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:42.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.378+0000 7f438b876700 1 Processor -- start 2023-09-30T22:19:42.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.378+0000 7f438b876700 1 -- start start 2023-09-30T22:19:42.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f438b876700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409d6f0 0x7f438409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f438b876700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409e000 0x7f4384130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f438b876700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f438409e410 0x7f4384130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f438b876700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438409f520 con 0x7f438409e410 2023-09-30T22:19:42.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f438b876700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438409f130 con 0x7f438409e000 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f438b876700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f438409e930 con 0x7f438409d6f0 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4389e13700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f438409e410 0x7f4384130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4389612700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409d6f0 0x7f438409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4389e13700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f438409e410 0x7f4384130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35690/0 (socket says 172.21.15.89:35690) 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4389e13700 1 -- 172.21.15.89:0/1803908368 learned_addr learned my addr 172.21.15.89:0/1803908368 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4388e11700 1 --2- 172.21.15.89:0/1803908368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409e000 0x7f4384130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4388e11700 1 -- 172.21.15.89:0/1803908368 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409d6f0 msgr2=0x7f438409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4388e11700 1 --2- 172.21.15.89:0/1803908368 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409d6f0 0x7f438409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4388e11700 1 -- 172.21.15.89:0/1803908368 >> v2:172.21.15.89:3300/0 conn(0x7f438409e410 msgr2=0x7f4384130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:42.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4388e11700 1 --2- 172.21.15.89:0/1803908368 >> v2:172.21.15.89:3300/0 conn(0x7f438409e410 0x7f4384130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4388e11700 1 -- 172.21.15.89:0/1803908368 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438409b2d0 con 0x7f438409e000 2023-09-30T22:19:42.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4389e13700 1 --2- 172.21.15.89:0/1803908368 >> v2:172.21.15.89:3300/0 conn(0x7f438409e410 0x7f4384130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f4388e11700 1 --2- 172.21.15.89:0/1803908368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409e000 0x7f4384130660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4374009870 tx=0x7f4374009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b446bb484e52f5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f437bfff700 1 -- 172.21.15.89:0/1803908368 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374011830 con 0x7f438409e000 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.379+0000 7f437bfff700 1 -- 172.21.15.89:0/1803908368 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4374011990 con 0x7f438409e000 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f437bfff700 1 -- 172.21.15.89:0/1803908368 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374011d20 con 0x7f438409e000 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 -- 172.21.15.89:0/1803908368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409e000 msgr2=0x7f4384130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 --2- 172.21.15.89:0/1803908368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409e000 0x7f4384130660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4374009870 tx=0x7f4374009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 -- 172.21.15.89:0/1803908368 shutdown_connections 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 --2- 172.21.15.89:0/1803908368 >> v2:172.21.15.89:3300/0 conn(0x7f438409e410 0x7f4384130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 --2- 172.21.15.89:0/1803908368 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409d6f0 0x7f438409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 --2- 172.21.15.89:0/1803908368 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409e000 0x7f4384130660 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 -- 172.21.15.89:0/1803908368 >> 172.21.15.89:0/1803908368 conn(0x7f4384066f50 msgr2=0x7f438409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:42.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 -- 172.21.15.89:0/1803908368 shutdown_connections 2023-09-30T22:19:42.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.380+0000 7f438b876700 1 -- 172.21.15.89:0/1803908368 wait complete. 2023-09-30T22:19:42.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.381+0000 7f438b876700 1 Processor -- start 2023-09-30T22:19:42.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.381+0000 7f438b876700 1 -- start start 2023-09-30T22:19:42.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f438b876700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409d6f0 0x7f43841c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f438b876700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f438409e000 0x7f43841c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f438b876700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409e410 0x7f4384131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f438b876700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4384135100 con 0x7f438409e000 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f438b876700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43841c9070 con 0x7f438409d6f0 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f438b876700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f438409c700 con 0x7f438409e410 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409d6f0 0x7f43841c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409d6f0 0x7f43841c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54952/0 (socket says 172.21.15.89:54952) 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 -- 172.21.15.89:0/2842665585 learned_addr learned my addr 172.21.15.89:0/2842665585 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4388e11700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f438409e000 0x7f43841c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389e13700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409e410 0x7f4384131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 -- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409e410 msgr2=0x7f4384131650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409e410 0x7f4384131650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 -- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f438409e000 msgr2=0x7f43841c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f438409e000 0x7f43841c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 -- 172.21.15.89:0/2842665585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c009b30 con 0x7f438409d6f0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4388e11700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f438409e000 0x7f43841c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.382+0000 7f4389612700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409d6f0 0x7f43841c85f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f438000e930 tx=0x7f438000ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.383+0000 7f4379ffb700 1 -- 172.21.15.89:0/2842665585 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380013680 con 0x7f438409d6f0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.383+0000 7f4379ffb700 1 -- 172.21.15.89:0/2842665585 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43800137e0 con 0x7f438409d6f0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.383+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43740094a0 con 0x7f438409d6f0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.383+0000 7f4379ffb700 1 -- 172.21.15.89:0/2842665585 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380013b70 con 0x7f438409d6f0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.383+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4384133990 con 0x7f438409d6f0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.384+0000 7f4379ffb700 1 -- 172.21.15.89:0/2842665585 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4380013cd0 con 0x7f438409d6f0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.384+0000 7f4379ffb700 1 -- 172.21.15.89:0/2842665585 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f438003f6c0 con 0x7f438409d6f0 2023-09-30T22:19:42.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.384+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4384134360 con 0x7f438409d6f0 2023-09-30T22:19:42.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.393+0000 7f4379ffb700 1 -- 172.21.15.89:0/2842665585 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4380017020 con 0x7f438409d6f0 2023-09-30T22:19:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.414+0000 7f7d6bb54700 1 Processor -- start 2023-09-30T22:19:42.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.414+0000 7f7d6bb54700 1 -- start start 2023-09-30T22:19:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6bb54700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409d6f0 0x7f7d6409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6bb54700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6bb54700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d6409e410 0x7f7d64133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6bb54700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6406b460 con 0x7f7d6409e410 2023-09-30T22:19:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6bb54700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6409b120 con 0x7f7d6409d6f0 2023-09-30T22:19:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6bb54700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6409a7e0 con 0x7f7d6409e000 2023-09-30T22:19:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6a0f1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d6409e410 0x7f7d64133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d698f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409d6f0 0x7f7d6409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6a0f1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d6409e410 0x7f7d64133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35700/0 (socket says 172.21.15.89:35700) 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d6a0f1700 1 -- 172.21.15.89:0/596940180 learned_addr learned my addr 172.21.15.89:0/596940180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d690ef700 1 --2- 172.21.15.89:0/596940180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d698f0700 1 -- 172.21.15.89:0/596940180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 msgr2=0x7f7d641339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d698f0700 1 --2- 172.21.15.89:0/596940180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d698f0700 1 -- 172.21.15.89:0/596940180 >> v2:172.21.15.89:3300/0 conn(0x7f7d6409e410 msgr2=0x7f7d64133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d698f0700 1 --2- 172.21.15.89:0/596940180 >> v2:172.21.15.89:3300/0 conn(0x7f7d6409e410 0x7f7d64133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d698f0700 1 -- 172.21.15.89:0/596940180 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6409a500 con 0x7f7d6409d6f0 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.415+0000 7f7d698f0700 1 --2- 172.21.15.89:0/596940180 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409d6f0 0x7f7d6409dac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009870 tx=0x7f7d60009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c573701f25c9d5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d688ee700 1 -- 172.21.15.89:0/596940180 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60011830 con 0x7f7d6409d6f0 2023-09-30T22:19:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d688ee700 1 -- 172.21.15.89:0/596940180 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d60011990 con 0x7f7d6409d6f0 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d688ee700 1 -- 172.21.15.89:0/596940180 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60011d20 con 0x7f7d6409d6f0 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 -- 172.21.15.89:0/596940180 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409d6f0 msgr2=0x7f7d6409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 --2- 172.21.15.89:0/596940180 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409d6f0 0x7f7d6409dac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009870 tx=0x7f7d60009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 -- 172.21.15.89:0/596940180 shutdown_connections 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 --2- 172.21.15.89:0/596940180 >> v2:172.21.15.89:3300/0 conn(0x7f7d6409e410 0x7f7d64133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 --2- 172.21.15.89:0/596940180 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409d6f0 0x7f7d6409dac0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 --2- 172.21.15.89:0/596940180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 -- 172.21.15.89:0/596940180 >> 172.21.15.89:0/596940180 conn(0x7f7d64066f50 msgr2=0x7f7d6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 -- 172.21.15.89:0/596940180 shutdown_connections 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.416+0000 7f7d6bb54700 1 -- 172.21.15.89:0/596940180 wait complete. 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.417+0000 7f7d6bb54700 1 Processor -- start 2023-09-30T22:19:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.417+0000 7f7d6bb54700 1 -- start start 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d6bb54700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6409d6f0 0x7f7d641a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d6bb54700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d6bb54700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409e410 0x7f7d641a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d6bb54700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641a8600 con 0x7f7d6409d6f0 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d6bb54700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640a31a0 con 0x7f7d6409e410 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d6bb54700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641a6ae0 con 0x7f7d6409e000 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44148/0 (socket says 172.21.15.89:44148) 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 -- 172.21.15.89:0/3592630507 learned_addr learned my addr 172.21.15.89:0/3592630507 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d698f0700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6409d6f0 0x7f7d641a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d6a0f1700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409e410 0x7f7d641a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 -- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409e410 msgr2=0x7f7d641a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409e410 0x7f7d641a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 -- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6409d6f0 msgr2=0x7f7d641a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6409d6f0 0x7f7d641a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 -- 172.21.15.89:0/3592630507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d600094a0 con 0x7f7d6409e000 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d698f0700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6409d6f0 0x7f7d641a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.418+0000 7f7d690ef700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641a5410 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400e9b0 tx=0x7f7d5400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.419+0000 7f7d5a7fc700 1 -- 172.21.15.89:0/3592630507 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54009710 con 0x7f7d6409e000 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.419+0000 7f7d5a7fc700 1 -- 172.21.15.89:0/3592630507 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5400c850 con 0x7f7d6409e000 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.419+0000 7f7d5a7fc700 1 -- 172.21.15.89:0/3592630507 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5400cbe0 con 0x7f7d6409e000 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.419+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6409c710 con 0x7f7d6409e000 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.419+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d641a88e0 con 0x7f7d6409e000 2023-09-30T22:19:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.420+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d641a8170 con 0x7f7d6409e000 2023-09-30T22:19:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.420+0000 7f7d5a7fc700 1 -- 172.21.15.89:0/3592630507 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d5400cd40 con 0x7f7d6409e000 2023-09-30T22:19:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.420+0000 7f7d5a7fc700 1 -- 172.21.15.89:0/3592630507 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7d5403fbe0 con 0x7f7d6409e000 2023-09-30T22:19:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.424+0000 7f7d5a7fc700 1 -- 172.21.15.89:0/3592630507 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d5401b3b0 con 0x7f7d6409e000 2023-09-30T22:19:42.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.461+0000 7f8cc7150700 1 Processor -- start 2023-09-30T22:19:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.461+0000 7f8cc7150700 1 -- start start 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc7150700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc012cf60 0x7f8cc006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc7150700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8cc012f8b0 0x7f8cc012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc7150700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc0130260 0x7f8cc0066410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc7150700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc012d330 con 0x7f8cc012f8b0 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc7150700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc012cd90 con 0x7f8cc0130260 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc7150700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc01306d0 con 0x7f8cc012cf60 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cbffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8cc012f8b0 0x7f8cc012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cbffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8cc012f8b0 0x7f8cc012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35730/0 (socket says 172.21.15.89:35730) 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc56ed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc0130260 0x7f8cc0066410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cbffff700 1 -- 172.21.15.89:0/4124297043 learned_addr learned my addr 172.21.15.89:0/4124297043 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc4eec700 1 --2- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc012cf60 0x7f8cc006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc56ed700 1 -- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc012cf60 msgr2=0x7f8cc006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc56ed700 1 --2- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc012cf60 0x7f8cc006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc56ed700 1 -- 172.21.15.89:0/4124297043 >> v2:172.21.15.89:3300/0 conn(0x7f8cc012f8b0 msgr2=0x7f8cc012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc56ed700 1 --2- 172.21.15.89:0/4124297043 >> v2:172.21.15.89:3300/0 conn(0x7f8cc012f8b0 0x7f8cc012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc56ed700 1 -- 172.21.15.89:0/4124297043 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0071780 con 0x7f8cc0130260 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.462+0000 7f8cc56ed700 1 --2- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc0130260 0x7f8cc0066410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009d90 tx=0x7f8cb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=261e0c3964f15ddf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.463+0000 7f8cc4eec700 1 --2- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc012cf60 0x7f8cc006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.463+0000 7f8cbf7fe700 1 -- 172.21.15.89:0/4124297043 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800e070 con 0x7f8cc0130260 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.463+0000 7f8cbf7fe700 1 -- 172.21.15.89:0/4124297043 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb800bb20 con 0x7f8cc0130260 2023-09-30T22:19:42.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.463+0000 7f8cbf7fe700 1 -- 172.21.15.89:0/4124297043 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8012430 con 0x7f8cc0130260 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.463+0000 7f8cc7150700 1 -- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc0130260 msgr2=0x7f8cc0066410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.463+0000 7f8cc7150700 1 --2- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc0130260 0x7f8cc0066410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009d90 tx=0x7f8cb800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.464+0000 7f8cc7150700 1 -- 172.21.15.89:0/4124297043 shutdown_connections 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.464+0000 7f8cc7150700 1 --2- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc0130260 0x7f8cc0066410 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.464+0000 7f8cc7150700 1 --2- 172.21.15.89:0/4124297043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc012cf60 0x7f8cc006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.464+0000 7f8cc7150700 1 --2- 172.21.15.89:0/4124297043 >> v2:172.21.15.89:3300/0 conn(0x7f8cc012f8b0 0x7f8cc012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.464+0000 7f8cc7150700 1 -- 172.21.15.89:0/4124297043 >> 172.21.15.89:0/4124297043 conn(0x7f8cc00a2aa0 msgr2=0x7f8cc00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.464+0000 7f8cc7150700 1 -- 172.21.15.89:0/4124297043 shutdown_connections 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.464+0000 7f8cc7150700 1 -- 172.21.15.89:0/4124297043 wait complete. 2023-09-30T22:19:42.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.465+0000 7f8cc7150700 1 Processor -- start 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.465+0000 7f8cc7150700 1 -- start start 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.465+0000 7f8cc7150700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc012cf60 0x7f8cc0133c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc7150700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cc012f8b0 0x7f8cc01341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc7150700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc0130260 0x7f8cc01c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc7150700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc012d9d0 con 0x7f8cc012f8b0 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc7150700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc009b910 con 0x7f8cc012cf60 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc7150700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0134700 con 0x7f8cc0130260 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc012cf60 0x7f8cc0133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc012cf60 0x7f8cc0133c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54988/0 (socket says 172.21.15.89:54988) 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 -- 172.21.15.89:0/1963557376 learned_addr learned my addr 172.21.15.89:0/1963557376 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc56ed700 1 --2- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc0130260 0x7f8cc01c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 -- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc0130260 msgr2=0x7f8cc01c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 --2- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc0130260 0x7f8cc01c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 -- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cc012f8b0 msgr2=0x7f8cc01341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 --2- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cc012f8b0 0x7f8cc01341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 -- 172.21.15.89:0/1963557376 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb8009a70 con 0x7f8cc012cf60 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc4eec700 1 --2- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc012cf60 0x7f8cc0133c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400eda0 tx=0x7f8cb4007380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cbd7fa700 1 -- 172.21.15.89:0/1963557376 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4007b90 con 0x7f8cc012cf60 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cbd7fa700 1 -- 172.21.15.89:0/1963557376 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb4007cf0 con 0x7f8cc012cf60 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.466+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc0133640 con 0x7f8cc012cf60 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.467+0000 7f8cbd7fa700 1 -- 172.21.15.89:0/1963557376 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4010640 con 0x7f8cc012cf60 2023-09-30T22:19:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.467+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc009afb0 con 0x7f8cc012cf60 2023-09-30T22:19:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.467+0000 7f8cbd7fa700 1 -- 172.21.15.89:0/1963557376 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8cb40107a0 con 0x7f8cc012cf60 2023-09-30T22:19:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.468+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc009bb30 con 0x7f8cc012cf60 2023-09-30T22:19:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.468+0000 7f8cbd7fa700 1 -- 172.21.15.89:0/1963557376 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8cb4014070 con 0x7f8cc012cf60 2023-09-30T22:19:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.473+0000 7f8cbd7fa700 1 -- 172.21.15.89:0/1963557376 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8cb4007e60 con 0x7f8cc012cf60 2023-09-30T22:19:42.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.572+0000 7f95023e1700 1 Processor -- start 2023-09-30T22:19:42.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.572+0000 7f95023e1700 1 -- start start 2023-09-30T22:19:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f95023e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f94fc133b10 0x7f94fc133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f95023e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc09de50 0x7f94fc09e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f95023e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc13a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f95023e1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc134420 con 0x7f94fc133b10 2023-09-30T22:19:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f95023e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1334b0 con 0x7f94fc09e7e0 2023-09-30T22:19:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f95023e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc070bc0 con 0x7f94fc09de50 2023-09-30T22:19:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f950097e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc13a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f94fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc09de50 0x7f94fc09e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f950097e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc13a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54998/0 (socket says 172.21.15.89:54998) 2023-09-30T22:19:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f950097e700 1 -- 172.21.15.89:0/3991667387 learned_addr learned my addr 172.21.15.89:0/3991667387 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.573+0000 7f94fbfff700 1 --2- 172.21.15.89:0/3991667387 >> v2:172.21.15.89:3300/0 conn(0x7f94fc133b10 0x7f94fc133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f950097e700 1 -- 172.21.15.89:0/3991667387 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc09de50 msgr2=0x7f94fc09e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f950097e700 1 --2- 172.21.15.89:0/3991667387 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc09de50 0x7f94fc09e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f950097e700 1 -- 172.21.15.89:0/3991667387 >> v2:172.21.15.89:3300/0 conn(0x7f94fc133b10 msgr2=0x7f94fc133ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:42.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f950097e700 1 --2- 172.21.15.89:0/3991667387 >> v2:172.21.15.89:3300/0 conn(0x7f94fc133b10 0x7f94fc133ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f950097e700 1 -- 172.21.15.89:0/3991667387 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc070350 con 0x7f94fc09e7e0 2023-09-30T22:19:42.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f94fb7fe700 1 --2- 172.21.15.89:0/3991667387 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc09de50 0x7f94fc09e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:42.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f950097e700 1 --2- 172.21.15.89:0/3991667387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc13a7e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009e80 tx=0x7f94f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=469a7607f30d8555 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:42.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f94faffd700 1 -- 172.21.15.89:0/3991667387 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f400e070 con 0x7f94fc09e7e0 2023-09-30T22:19:42.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f94faffd700 1 -- 172.21.15.89:0/3991667387 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f400b8d0 con 0x7f94fc09e7e0 2023-09-30T22:19:42.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.574+0000 7f94faffd700 1 -- 172.21.15.89:0/3991667387 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f400bc80 con 0x7f94fc09e7e0 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 -- 172.21.15.89:0/3991667387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 msgr2=0x7f94fc13a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 --2- 172.21.15.89:0/3991667387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc13a7e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009e80 tx=0x7f94f400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 -- 172.21.15.89:0/3991667387 shutdown_connections 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 --2- 172.21.15.89:0/3991667387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc13a7e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 --2- 172.21.15.89:0/3991667387 >> v2:172.21.15.89:3300/0 conn(0x7f94fc133b10 0x7f94fc133ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 --2- 172.21.15.89:0/3991667387 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc09de50 0x7f94fc09e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 -- 172.21.15.89:0/3991667387 >> 172.21.15.89:0/3991667387 conn(0x7f94fc0a2aa0 msgr2=0x7f94fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 -- 172.21.15.89:0/3991667387 shutdown_connections 2023-09-30T22:19:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.575+0000 7f95023e1700 1 -- 172.21.15.89:0/3991667387 wait complete. 2023-09-30T22:19:42.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.576+0000 7f95023e1700 1 Processor -- start 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.576+0000 7f95023e1700 1 -- start start 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f95023e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc09de50 0x7f94fc11e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f95023e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc11ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f95023e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc133b10 0x7f94fc11f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f95023e1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1213e0 con 0x7f94fc09de50 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f95023e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc06f910 con 0x7f94fc09e7e0 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f95023e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc12fe90 con 0x7f94fc133b10 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f94fb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc11ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f94fb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc11ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55004/0 (socket says 172.21.15.89:55004) 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f94fb7fe700 1 -- 172.21.15.89:0/2157574269 learned_addr learned my addr 172.21.15.89:0/2157574269 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f950097e700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc133b10 0x7f94fc11f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f94fbfff700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc09de50 0x7f94fc11e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f94fb7fe700 1 -- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc133b10 msgr2=0x7f94fc11f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f94fb7fe700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc133b10 0x7f94fc11f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f94fb7fe700 1 -- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc09de50 msgr2=0x7f94fc11e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.577+0000 7f94fb7fe700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc09de50 0x7f94fc11e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.578+0000 7f94fb7fe700 1 -- 172.21.15.89:0/2157574269 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00094a0 con 0x7f94fc09e7e0 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.578+0000 7f950097e700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc133b10 0x7f94fc11f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.578+0000 7f94fb7fe700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc11ee80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f94f000ebe0 tx=0x7f94f000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.578+0000 7f94f8ff9700 1 -- 172.21.15.89:0/2157574269 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0009760 con 0x7f94fc09e7e0 2023-09-30T22:19:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.578+0000 7f94f8ff9700 1 -- 172.21.15.89:0/2157574269 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f000c850 con 0x7f94fc09e7e0 2023-09-30T22:19:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.578+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f4009b30 con 0x7f94fc09e7e0 2023-09-30T22:19:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.578+0000 7f94f8ff9700 1 -- 172.21.15.89:0/2157574269 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f000cbe0 con 0x7f94fc09e7e0 2023-09-30T22:19:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.579+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc121ec0 con 0x7f94fc09e7e0 2023-09-30T22:19:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.579+0000 7f94f8ff9700 1 -- 172.21.15.89:0/2157574269 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94f000cd40 con 0x7f94fc09e7e0 2023-09-30T22:19:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.579+0000 7f94f8ff9700 1 -- 172.21.15.89:0/2157574269 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f94f00421c0 con 0x7f94fc09e7e0 2023-09-30T22:19:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.580+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc074ea0 con 0x7f94fc09e7e0 2023-09-30T22:19:42.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.584+0000 7f94f8ff9700 1 -- 172.21.15.89:0/2157574269 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94f0017020 con 0x7f94fc09e7e0 2023-09-30T22:19:42.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.601+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4384074ea0 con 0x7f438409d6f0 2023-09-30T22:19:42.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.601+0000 7f4379ffb700 1 -- 172.21.15.89:0/2842665585 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f438001ba00 con 0x7f438409d6f0 2023-09-30T22:19:42.602 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:42.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.604+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409d6f0 msgr2=0x7f43841c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.604+0000 7f438b876700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409d6f0 0x7f43841c85f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f438000e930 tx=0x7f438000ec40 comp rx=0 tx=0).stop 2023-09-30T22:19:42.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.604+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 shutdown_connections 2023-09-30T22:19:42.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.604+0000 7f438b876700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f438409e410 0x7f4384131650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.604+0000 7f438b876700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f438409d6f0 0x7f43841c85f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.604+0000 7f438b876700 1 --2- 172.21.15.89:0/2842665585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f438409e000 0x7f43841c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.604+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 >> 172.21.15.89:0/2842665585 conn(0x7f4384066f50 msgr2=0x7f4384070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:42.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.604+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 shutdown_connections 2023-09-30T22:19:42.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.605+0000 7f438b876700 1 -- 172.21.15.89:0/2842665585 wait complete. 2023-09-30T22:19:42.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.629+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7d641a8740 con 0x7f7d6409e000 2023-09-30T22:19:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.629+0000 7f7d5a7fc700 1 -- 172.21.15.89:0/3592630507 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d641a8740 con 0x7f7d6409e000 2023-09-30T22:19:42.630 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 msgr2=0x7f7d641a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641a5410 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400e9b0 tx=0x7f7d5400ed70 comp rx=0 tx=0).stop 2023-09-30T22:19:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 shutdown_connections 2023-09-30T22:19:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d6409e410 0x7f7d641a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6409d6f0 0x7f7d641a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 --2- 172.21.15.89:0/3592630507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d6409e000 0x7f7d641a5410 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 >> 172.21.15.89:0/3592630507 conn(0x7f7d64066f50 msgr2=0x7f7d641b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 shutdown_connections 2023-09-30T22:19:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.631+0000 7f7d6bb54700 1 -- 172.21.15.89:0/3592630507 wait complete. 2023-09-30T22:19:42.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.679+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8cc0074ea0 con 0x7f8cc012cf60 2023-09-30T22:19:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.680+0000 7f8cbd7fa700 1 -- 172.21.15.89:0/1963557376 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8cb40203a0 con 0x7f8cc012cf60 2023-09-30T22:19:42.680 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc012cf60 msgr2=0x7f8cc0133c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 --2- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc012cf60 0x7f8cc0133c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400eda0 tx=0x7f8cb4007380 comp rx=0 tx=0).stop 2023-09-30T22:19:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 shutdown_connections 2023-09-30T22:19:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 --2- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8cc0130260 0x7f8cc01c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 --2- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cc012cf60 0x7f8cc0133c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 --2- 172.21.15.89:0/1963557376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cc012f8b0 0x7f8cc01341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 >> 172.21.15.89:0/1963557376 conn(0x7f8cc00a2aa0 msgr2=0x7f8cc01291e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 shutdown_connections 2023-09-30T22:19:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.682+0000 7f8cc7150700 1 -- 172.21.15.89:0/1963557376 wait complete. 2023-09-30T22:19:42.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:42.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.778+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f94fc122240 con 0x7f94fc09e7e0 2023-09-30T22:19:42.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.779+0000 7f94f8ff9700 1 -- 172.21.15.89:0/2157574269 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94f0044060 con 0x7f94fc09e7e0 2023-09-30T22:19:42.780 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 msgr2=0x7f94fc11ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc11ee80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f94f000ebe0 tx=0x7f94f000eef0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 shutdown_connections 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94fc133b10 0x7f94fc11f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc09de50 0x7f94fc11e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 --2- 172.21.15.89:0/2157574269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94fc09e7e0 0x7f94fc11ee80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 >> 172.21.15.89:0/2157574269 conn(0x7f94fc0a2aa0 msgr2=0x7f94fc09f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 shutdown_connections 2023-09-30T22:19:42.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:42.781+0000 7f95023e1700 1 -- 172.21.15.89:0/2157574269 wait complete. 2023-09-30T22:19:42.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:43.623 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:43.645 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:43.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.774+0000 7efe06ff0700 1 Processor -- start 2023-09-30T22:19:43.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.774+0000 7efe06ff0700 1 -- start start 2023-09-30T22:19:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe06ff0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe00130560 0x7efe00130930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe06ff0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0009eca0 0x7efe0006f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe06ff0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe00070a50 con 0x7efe00130e70 2023-09-30T22:19:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe06ff0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000717d0 con 0x7efe00130560 2023-09-30T22:19:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe06ff0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe00130340 con 0x7efe0009eca0 2023-09-30T22:19:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe04d8c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe00130560 0x7efe00130930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efdfffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0009eca0 0x7efe0006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe04d8c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe00130560 0x7efe00130930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35778/0 (socket says 172.21.15.89:35778) 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe04d8c700 1 -- 172.21.15.89:0/2493671517 learned_addr learned my addr 172.21.15.89:0/2493671517 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe04d8c700 1 -- 172.21.15.89:0/2493671517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0009eca0 msgr2=0x7efe0006f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe04d8c700 1 --2- 172.21.15.89:0/2493671517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0009eca0 0x7efe0006f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe04d8c700 1 -- 172.21.15.89:0/2493671517 >> v1:172.21.15.89:6789/0 conn(0x7efe00130e70 legacy=0x7efe0006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efe04d8c700 1 -- 172.21.15.89:0/2493671517 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe00071060 con 0x7efe00130560 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.775+0000 7efdfffff700 1 --2- 172.21.15.89:0/2493671517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0009eca0 0x7efe0006f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efe04d8c700 1 --2- 172.21.15.89:0/2493671517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe00130560 0x7efe00130930 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efdf40098f0 tx=0x7efdf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c84ae9ac3b53fb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efdff7fe700 1 -- 172.21.15.89:0/2493671517 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4011830 con 0x7efe00130560 2023-09-30T22:19:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efdff7fe700 1 -- 172.21.15.89:0/2493671517 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf4011990 con 0x7efe00130560 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efdff7fe700 1 -- 172.21.15.89:0/2493671517 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4011d20 con 0x7efe00130560 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efe06ff0700 1 -- 172.21.15.89:0/2493671517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe00130560 msgr2=0x7efe00130930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efe06ff0700 1 --2- 172.21.15.89:0/2493671517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe00130560 0x7efe00130930 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efdf40098f0 tx=0x7efdf4009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efe06ff0700 1 -- 172.21.15.89:0/2493671517 shutdown_connections 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efe06ff0700 1 --2- 172.21.15.89:0/2493671517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe00130560 0x7efe00130930 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efe06ff0700 1 --2- 172.21.15.89:0/2493671517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0009eca0 0x7efe0006f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efe06ff0700 1 -- 172.21.15.89:0/2493671517 >> 172.21.15.89:0/2493671517 conn(0x7efe000a2aa0 msgr2=0x7efe000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.776+0000 7efe06ff0700 1 -- 172.21.15.89:0/2493671517 shutdown_connections 2023-09-30T22:19:43.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.777+0000 7efe06ff0700 1 -- 172.21.15.89:0/2493671517 wait complete. 2023-09-30T22:19:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.778+0000 7efe06ff0700 1 Processor -- start 2023-09-30T22:19:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.778+0000 7efe06ff0700 1 -- start start 2023-09-30T22:19:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe06ff0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0009eca0 0x7efe0009d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe06ff0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe00130560 0x7efe0009d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe06ff0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe00130e70 0x7efe0009db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe06ff0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0011abf0 con 0x7efe0009eca0 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe06ff0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0011a120 con 0x7efe00130560 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe06ff0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe000719f0 con 0x7efe00130e70 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efdfffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe00130560 0x7efe0009d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efdfffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe00130560 0x7efe0009d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55018/0 (socket says 172.21.15.89:55018) 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efdfffff700 1 -- 172.21.15.89:0/3579343833 learned_addr learned my addr 172.21.15.89:0/3579343833 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe0558d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe00130e70 0x7efe0009db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe04d8c700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0009eca0 0x7efe0009d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe0558d700 1 -- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe00130560 msgr2=0x7efe0009d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe0558d700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe00130560 0x7efe0009d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe0558d700 1 -- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0009eca0 msgr2=0x7efe0009d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe0558d700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0009eca0 0x7efe0009d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe0558d700 1 -- 172.21.15.89:0/3579343833 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec00d940 con 0x7efe00130e70 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efdfffff700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe00130560 0x7efe0009d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.779+0000 7efe0558d700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe00130e70 0x7efe0009db70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efdf800cb90 tx=0x7efdf800cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.780+0000 7efdfd7fa700 1 -- 172.21.15.89:0/3579343833 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8003c00 con 0x7efe00130e70 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.780+0000 7efdfd7fa700 1 -- 172.21.15.89:0/3579343833 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8003d60 con 0x7efe00130e70 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.780+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4009550 con 0x7efe00130e70 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.780+0000 7efdfd7fa700 1 -- 172.21.15.89:0/3579343833 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf800aa40 con 0x7efe00130e70 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.780+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0011b190 con 0x7efe00130e70 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.781+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0011a540 con 0x7efe00130e70 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.781+0000 7efdfd7fa700 1 -- 172.21.15.89:0/3579343833 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efdf800aba0 con 0x7efe00130e70 2023-09-30T22:19:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.781+0000 7efdfd7fa700 1 -- 172.21.15.89:0/3579343833 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efdf803f5c0 con 0x7efe00130e70 2023-09-30T22:19:43.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.790+0000 7efdfd7fa700 1 -- 172.21.15.89:0/3579343833 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efdf8019710 con 0x7efe00130e70 2023-09-30T22:19:43.794 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.803+0000 7f12c67e1700 1 Processor -- start 2023-09-30T22:19:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.803+0000 7f12c67e1700 1 -- start start 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12c67e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012cf60 0x7f12c006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12c67e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f12c012f8b0 0x7f12c012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12c67e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c0130260 0x7f12c0136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12c67e1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c012d330 con 0x7f12c012f8b0 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12c67e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c012cd90 con 0x7f12c0130260 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12c67e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01306d0 con 0x7f12c012cf60 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012cf60 0x7f12c006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012cf60 0x7f12c006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44220/0 (socket says 172.21.15.89:44220) 2023-09-30T22:19:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bffff700 1 -- 172.21.15.89:0/1435859922 learned_addr learned my addr 172.21.15.89:0/1435859922 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12c4d7e700 1 --2- 172.21.15.89:0/1435859922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c0130260 0x7f12c0136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bf7fe700 1 --2- 172.21.15.89:0/1435859922 >> v2:172.21.15.89:3300/0 conn(0x7f12c012f8b0 0x7f12c012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bffff700 1 -- 172.21.15.89:0/1435859922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c0130260 msgr2=0x7f12c0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bffff700 1 --2- 172.21.15.89:0/1435859922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c0130260 0x7f12c0136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bffff700 1 -- 172.21.15.89:0/1435859922 >> v2:172.21.15.89:3300/0 conn(0x7f12c012f8b0 msgr2=0x7f12c012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bffff700 1 --2- 172.21.15.89:0/1435859922 >> v2:172.21.15.89:3300/0 conn(0x7f12c012f8b0 0x7f12c012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.804+0000 7f12bffff700 1 -- 172.21.15.89:0/1435859922 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c0071780 con 0x7f12c012cf60 2023-09-30T22:19:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.805+0000 7f12bffff700 1 --2- 172.21.15.89:0/1435859922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012cf60 0x7f12c006f5d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009920 tx=0x7f12b0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9c55c0311680e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.805+0000 7f12beffd700 1 -- 172.21.15.89:0/1435859922 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0011830 con 0x7f12c012cf60 2023-09-30T22:19:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.805+0000 7f12beffd700 1 -- 172.21.15.89:0/1435859922 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b0011990 con 0x7f12c012cf60 2023-09-30T22:19:43.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.805+0000 7f12beffd700 1 -- 172.21.15.89:0/1435859922 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0011d20 con 0x7f12c012cf60 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 -- 172.21.15.89:0/1435859922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012cf60 msgr2=0x7f12c006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 --2- 172.21.15.89:0/1435859922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012cf60 0x7f12c006f5d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009920 tx=0x7f12b0009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 -- 172.21.15.89:0/1435859922 shutdown_connections 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 --2- 172.21.15.89:0/1435859922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c0130260 0x7f12c0136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 --2- 172.21.15.89:0/1435859922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012cf60 0x7f12c006f5d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 --2- 172.21.15.89:0/1435859922 >> v2:172.21.15.89:3300/0 conn(0x7f12c012f8b0 0x7f12c012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 -- 172.21.15.89:0/1435859922 >> 172.21.15.89:0/1435859922 conn(0x7f12c00a2aa0 msgr2=0x7f12c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 -- 172.21.15.89:0/1435859922 shutdown_connections 2023-09-30T22:19:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.806+0000 7f12c67e1700 1 -- 172.21.15.89:0/1435859922 wait complete. 2023-09-30T22:19:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.807+0000 7f12c67e1700 1 Processor -- start 2023-09-30T22:19:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12c67e1700 1 -- start start 2023-09-30T22:19:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12c67e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c012cf60 0x7f12c009bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12c67e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012f8b0 0x7f12c009c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12c67e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12c0130260 0x7f12c01c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12c67e1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c009e840 con 0x7f12c0130260 2023-09-30T22:19:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12c67e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c012da40 con 0x7f12c012cf60 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12c67e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12c009b720 con 0x7f12c012f8b0 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12bffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c012cf60 0x7f12c009bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12bffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c012cf60 0x7f12c009bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55044/0 (socket says 172.21.15.89:55044) 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12bffff700 1 -- 172.21.15.89:0/906292955 learned_addr learned my addr 172.21.15.89:0/906292955 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12c4d7e700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12c0130260 0x7f12c01c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.808+0000 7f12bf7fe700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012f8b0 0x7f12c009c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bffff700 1 -- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012f8b0 msgr2=0x7f12c009c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bffff700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012f8b0 0x7f12c009c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bffff700 1 -- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12c0130260 msgr2=0x7f12c01c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bffff700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12c0130260 0x7f12c01c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bffff700 1 -- 172.21.15.89:0/906292955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0009550 con 0x7f12c012cf60 2023-09-30T22:19:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bf7fe700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012f8b0 0x7f12c009c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bffff700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c012cf60 0x7f12c009bbc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009c70 tx=0x7f12b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bcff9700 1 -- 172.21.15.89:0/906292955 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0004240 con 0x7f12c012cf60 2023-09-30T22:19:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bcff9700 1 -- 172.21.15.89:0/906292955 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b001f070 con 0x7f12c012cf60 2023-09-30T22:19:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c0129fa0 con 0x7f12c012cf60 2023-09-30T22:19:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12bcff9700 1 -- 172.21.15.89:0/906292955 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00036a0 con 0x7f12c012cf60 2023-09-30T22:19:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.809+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c009cc10 con 0x7f12c012cf60 2023-09-30T22:19:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.810+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c009e4e0 con 0x7f12c012cf60 2023-09-30T22:19:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.810+0000 7f12bcff9700 1 -- 172.21.15.89:0/906292955 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12b0021030 con 0x7f12c012cf60 2023-09-30T22:19:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.811+0000 7f12bcff9700 1 -- 172.21.15.89:0/906292955 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f12b0040800 con 0x7f12c012cf60 2023-09-30T22:19:43.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.818+0000 7f12bcff9700 1 -- 172.21.15.89:0/906292955 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12b00199a0 con 0x7f12c012cf60 2023-09-30T22:19:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.855+0000 7fc6bdbd3700 1 Processor -- start 2023-09-30T22:19:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.855+0000 7fc6bdbd3700 1 -- start start 2023-09-30T22:19:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6bdbd3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6b809de00 0x7fc6b809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6bdbd3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b813d1f0 0x7fc6b813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6bdbd3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b809a810 con 0x7fc6b806a770 2023-09-30T22:19:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6bdbd3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b806ab40 con 0x7fc6b809de00 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6bdbd3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b809ebc0 con 0x7fc6b813d1f0 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b813d1f0 0x7fc6b813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6b809de00 0x7fc6b809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b813d1f0 0x7fc6b813f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55046/0 (socket says 172.21.15.89:55046) 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b7fff700 1 -- 172.21.15.89:0/559708365 learned_addr learned my addr 172.21.15.89:0/559708365 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b7fff700 1 -- 172.21.15.89:0/559708365 >> v2:172.21.15.89:3300/0 conn(0x7fc6b809de00 msgr2=0x7fc6b809e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b7fff700 1 --2- 172.21.15.89:0/559708365 >> v2:172.21.15.89:3300/0 conn(0x7fc6b809de00 0x7fc6b809e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b7fff700 1 -- 172.21.15.89:0/559708365 >> v1:172.21.15.89:6789/0 conn(0x7fc6b806a770 legacy=0x7fc6b809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b7fff700 1 -- 172.21.15.89:0/559708365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b809e920 con 0x7fc6b813d1f0 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b6ffd700 1 --2- 172.21.15.89:0/559708365 >> v2:172.21.15.89:3300/0 conn(0x7fc6b809de00 0x7fc6b809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b7fff700 1 --2- 172.21.15.89:0/559708365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b813d1f0 0x7fc6b813f5d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009e50 tx=0x7fc6b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74a155e045ca415 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.856+0000 7fc6b67fc700 1 -- 172.21.15.89:0/559708365 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000e070 con 0x7fc6b813d1f0 2023-09-30T22:19:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6b67fc700 1 -- 172.21.15.89:0/559708365 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b000b910 con 0x7fc6b813d1f0 2023-09-30T22:19:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6b67fc700 1 -- 172.21.15.89:0/559708365 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000bcc0 con 0x7fc6b813d1f0 2023-09-30T22:19:43.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/559708365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b813d1f0 msgr2=0x7fc6b813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6bdbd3700 1 --2- 172.21.15.89:0/559708365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b813d1f0 0x7fc6b813f5d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009e50 tx=0x7fc6b000b330 comp rx=0 tx=0).stop 2023-09-30T22:19:43.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/559708365 shutdown_connections 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6bdbd3700 1 --2- 172.21.15.89:0/559708365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b813d1f0 0x7fc6b813f5d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6bdbd3700 1 --2- 172.21.15.89:0/559708365 >> v2:172.21.15.89:3300/0 conn(0x7fc6b809de00 0x7fc6b809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/559708365 >> 172.21.15.89:0/559708365 conn(0x7fc6b8066f50 msgr2=0x7fc6b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/559708365 shutdown_connections 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.857+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/559708365 wait complete. 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.859+0000 7fc6bdbd3700 1 Processor -- start 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.859+0000 7fc6bdbd3700 1 -- start start 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.859+0000 7fc6bdbd3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6b806a770 0x7fc6b8071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.859+0000 7fc6bdbd3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b809de00 0x7fc6b813b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.859+0000 7fc6bdbd3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6b813d1f0 0x7fc6b813bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.859+0000 7fc6bdbd3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b809f1a0 con 0x7fc6b806a770 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.859+0000 7fc6bdbd3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b811dfb0 con 0x7fc6b809de00 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.859+0000 7fc6bdbd3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b811eb00 con 0x7fc6b813d1f0 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6b806a770 0x7fc6b8071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6b806a770 0x7fc6b8071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35838/0 (socket says 172.21.15.89:35838) 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 -- 172.21.15.89:0/1392022678 learned_addr learned my addr 172.21.15.89:0/1392022678 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b6ffd700 1 --2- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b809de00 0x7fc6b813b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 -- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6b813d1f0 msgr2=0x7fc6b813bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 --2- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6b813d1f0 0x7fc6b813bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 -- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b809de00 msgr2=0x7fc6b813b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 --2- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b809de00 0x7fc6b813b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 -- 172.21.15.89:0/1392022678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac0094a0 con 0x7fc6b806a770 2023-09-30T22:19:43.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6b77fe700 1 --2- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6b806a770 0x7fc6b8071710 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c4e0 tx=0x7fc6a800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:43.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc69ffff700 1 -- 172.21.15.89:0/1392022678 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80052d0 con 0x7fc6b806a770 2023-09-30T22:19:43.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc69ffff700 1 -- 172.21.15.89:0/1392022678 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a8005430 con 0x7fc6b806a770 2023-09-30T22:19:43.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc69ffff700 1 -- 172.21.15.89:0/1392022678 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80038c0 con 0x7fc6b806a770 2023-09-30T22:19:43.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b0009b30 con 0x7fc6b806a770 2023-09-30T22:19:43.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.860+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b811f830 con 0x7fc6b806a770 2023-09-30T22:19:43.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.861+0000 7fc69ffff700 1 -- 172.21.15.89:0/1392022678 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6a8003a20 con 0x7fc6b806a770 2023-09-30T22:19:43.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.861+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b811edf0 con 0x7fc6b806a770 2023-09-30T22:19:43.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.862+0000 7fc69ffff700 1 -- 172.21.15.89:0/1392022678 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc6a803eb60 con 0x7fc6b806a770 2023-09-30T22:19:43.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.870+0000 7fc69ffff700 1 -- 172.21.15.89:0/1392022678 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6a8018260 con 0x7fc6b806a770 2023-09-30T22:19:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.928+0000 7fb273095700 1 Processor -- start 2023-09-30T22:19:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.928+0000 7fb273095700 1 -- start start 2023-09-30T22:19:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb273095700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb273095700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb26c06eba0 0x7fb26c139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb273095700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c071a00 con 0x7fb26c131f40 2023-09-30T22:19:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb273095700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c131430 con 0x7fb26c06eba0 2023-09-30T22:19:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb273095700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c09a510 con 0x7fb26c131630 2023-09-30T22:19:43.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb270e31700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb271632700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb26c06eba0 0x7fb26c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb270e31700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55058/0 (socket says 172.21.15.89:55058) 2023-09-30T22:19:43.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb271632700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb26c06eba0 0x7fb26c139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35846/0 (socket says 172.21.15.89:35846) 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.929+0000 7fb270e31700 1 -- 172.21.15.89:0/1329114424 learned_addr learned my addr 172.21.15.89:0/1329114424 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb271632700 1 -- 172.21.15.89:0/1329114424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 msgr2=0x7fb26c131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb271632700 1 --2- 172.21.15.89:0/1329114424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb271632700 1 -- 172.21.15.89:0/1329114424 >> v1:172.21.15.89:6789/0 conn(0x7fb26c131f40 legacy=0x7fb26c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb271632700 1 -- 172.21.15.89:0/1329114424 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c0721c0 con 0x7fb26c06eba0 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb270e31700 1 --2- 172.21.15.89:0/1329114424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb271632700 1 --2- 172.21.15.89:0/1329114424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c139660 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb264009d90 tx=0x7fb26400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19889acc48ecae7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb26b7fe700 1 -- 172.21.15.89:0/1329114424 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26400e070 con 0x7fb26c06eba0 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb26b7fe700 1 -- 172.21.15.89:0/1329114424 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26400bb20 con 0x7fb26c06eba0 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.930+0000 7fb26b7fe700 1 -- 172.21.15.89:0/1329114424 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264012430 con 0x7fb26c06eba0 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.931+0000 7fb273095700 1 -- 172.21.15.89:0/1329114424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 msgr2=0x7fb26c139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.931+0000 7fb273095700 1 --2- 172.21.15.89:0/1329114424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c139660 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb264009d90 tx=0x7fb26400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.931+0000 7fb273095700 1 -- 172.21.15.89:0/1329114424 shutdown_connections 2023-09-30T22:19:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.931+0000 7fb273095700 1 --2- 172.21.15.89:0/1329114424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c139660 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.931+0000 7fb273095700 1 --2- 172.21.15.89:0/1329114424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.931+0000 7fb273095700 1 -- 172.21.15.89:0/1329114424 >> 172.21.15.89:0/1329114424 conn(0x7fb26c0a2aa0 msgr2=0x7fb26c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.931+0000 7fb273095700 1 -- 172.21.15.89:0/1329114424 shutdown_connections 2023-09-30T22:19:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.931+0000 7fb273095700 1 -- 172.21.15.89:0/1329114424 wait complete. 2023-09-30T22:19:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.932+0000 7fb273095700 1 Processor -- start 2023-09-30T22:19:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.932+0000 7fb273095700 1 -- start start 2023-09-30T22:19:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb273095700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c13d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb273095700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c13d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb273095700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb26c131f40 0x7fb26c13dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb273095700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c1323f0 con 0x7fb26c06eba0 2023-09-30T22:19:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb273095700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c13a730 con 0x7fb26c131630 2023-09-30T22:19:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb273095700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c099f20 con 0x7fb26c131f40 2023-09-30T22:19:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb270e31700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c13d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c13d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb270e31700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c13d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35856/0 (socket says 172.21.15.89:35856) 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb270e31700 1 -- 172.21.15.89:0/529367975 learned_addr learned my addr 172.21.15.89:0/529367975 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb270e31700 1 -- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb26c131f40 msgr2=0x7fb26c13dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb270e31700 1 --2- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb26c131f40 0x7fb26c13dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb270e31700 1 -- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 msgr2=0x7fb26c13d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb270e31700 1 --2- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c13d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb270e31700 1 -- 172.21.15.89:0/529367975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb260009550 con 0x7fb26c06eba0 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.933+0000 7fb26bfff700 1 --2- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c13d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.934+0000 7fb270e31700 1 --2- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c13d130 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb26000ebe0 tx=0x7fb26000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.934+0000 7fb2697fa700 1 -- 172.21.15.89:0/529367975 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26000c850 con 0x7fb26c06eba0 2023-09-30T22:19:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.934+0000 7fb2697fa700 1 -- 172.21.15.89:0/529367975 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26000c9b0 con 0x7fb26c06eba0 2023-09-30T22:19:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.934+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb264009a70 con 0x7fb26c06eba0 2023-09-30T22:19:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.934+0000 7fb2697fa700 1 -- 172.21.15.89:0/529367975 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26000cd20 con 0x7fb26c06eba0 2023-09-30T22:19:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.934+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c1328d0 con 0x7fb26c06eba0 2023-09-30T22:19:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.935+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c133cc0 con 0x7fb26c06eba0 2023-09-30T22:19:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.935+0000 7fb2697fa700 1 -- 172.21.15.89:0/529367975 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb260004020 con 0x7fb26c06eba0 2023-09-30T22:19:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.935+0000 7fb2697fa700 1 -- 172.21.15.89:0/529367975 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb260040010 con 0x7fb26c06eba0 2023-09-30T22:19:43.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.944+0000 7fb2697fa700 1 -- 172.21.15.89:0/529367975 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb260024350 con 0x7fb26c06eba0 2023-09-30T22:19:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.992+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efe0011ae10 con 0x7efe00130e70 2023-09-30T22:19:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.993+0000 7efdfd7fa700 1 -- 172.21.15.89:0/3579343833 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efdf801a360 con 0x7efe00130e70 2023-09-30T22:19:43.994 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe00130e70 msgr2=0x7efe0009db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe00130e70 0x7efe0009db70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efdf800cb90 tx=0x7efdf800cea0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 shutdown_connections 2023-09-30T22:19:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe00130e70 0x7efe0009db70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0009eca0 0x7efe0009d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 --2- 172.21.15.89:0/3579343833 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe00130560 0x7efe0009d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 >> 172.21.15.89:0/3579343833 conn(0x7efe000a2aa0 msgr2=0x7efe001b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 shutdown_connections 2023-09-30T22:19:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:43.995+0000 7efe06ff0700 1 -- 172.21.15.89:0/3579343833 wait complete. 2023-09-30T22:19:44.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.018+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f12c009e980 con 0x7f12c012cf60 2023-09-30T22:19:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.019+0000 7f12bcff9700 1 -- 172.21.15.89:0/906292955 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12c009e980 con 0x7f12c012cf60 2023-09-30T22:19:44.020 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:44.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c012cf60 msgr2=0x7f12c009bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:44.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c012cf60 0x7f12c009bbc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009c70 tx=0x7f12b0004750 comp rx=0 tx=0).stop 2023-09-30T22:19:44.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 shutdown_connections 2023-09-30T22:19:44.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12c0130260 0x7f12c01c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12c012cf60 0x7f12c009bbc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 --2- 172.21.15.89:0/906292955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12c012f8b0 0x7f12c009c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 >> 172.21.15.89:0/906292955 conn(0x7f12c00a2aa0 msgr2=0x7f12c00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:44.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 shutdown_connections 2023-09-30T22:19:44.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.021+0000 7f12c67e1700 1 -- 172.21.15.89:0/906292955 wait complete. 2023-09-30T22:19:44.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:44.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.072+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc6b8074ea0 con 0x7fc6b806a770 2023-09-30T22:19:44.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.072+0000 7fc69ffff700 1 -- 172.21.15.89:0/1392022678 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6a8017070 con 0x7fc6b806a770 2023-09-30T22:19:44.073 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:44.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.074+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6b806a770 msgr2=0x7fc6b8071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:44.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.074+0000 7fc6bdbd3700 1 --2- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6b806a770 0x7fc6b8071710 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c4e0 tx=0x7fc6a800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.075+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 shutdown_connections 2023-09-30T22:19:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.075+0000 7fc6bdbd3700 1 --2- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6b813d1f0 0x7fc6b813bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.075+0000 7fc6bdbd3700 1 --2- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6b806a770 0x7fc6b8071710 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.075+0000 7fc6bdbd3700 1 --2- 172.21.15.89:0/1392022678 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6b809de00 0x7fc6b813b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.075+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 >> 172.21.15.89:0/1392022678 conn(0x7fc6b8066f50 msgr2=0x7fc6b809b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.075+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 shutdown_connections 2023-09-30T22:19:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.075+0000 7fc6bdbd3700 1 -- 172.21.15.89:0/1392022678 wait complete. 2023-09-30T22:19:44.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:44.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.139+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb26c074ea0 con 0x7fb26c06eba0 2023-09-30T22:19:44.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.139+0000 7fb2697fa700 1 -- 172.21.15.89:0/529367975 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb260021070 con 0x7fb26c06eba0 2023-09-30T22:19:44.140 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:44.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.141+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 msgr2=0x7fb26c13d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:44.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.141+0000 7fb273095700 1 --2- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c13d130 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb26000ebe0 tx=0x7fb26000eef0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.141+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 shutdown_connections 2023-09-30T22:19:44.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.141+0000 7fb273095700 1 --2- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb26c131f40 0x7fb26c13dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.141+0000 7fb273095700 1 --2- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26c06eba0 0x7fb26c13d130 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.141+0000 7fb273095700 1 --2- 172.21.15.89:0/529367975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb26c131630 0x7fb26c13d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.141+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 >> 172.21.15.89:0/529367975 conn(0x7fb26c0a2aa0 msgr2=0x7fb26c0708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.142+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 shutdown_connections 2023-09-30T22:19:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:44.142+0000 7fb273095700 1 -- 172.21.15.89:0/529367975 wait complete. 2023-09-30T22:19:44.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:45.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:45.035 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:45.088 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.149+0000 7f24ee4d3700 1 Processor -- start 2023-09-30T22:19:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.149+0000 7f24ee4d3700 1 -- start start 2023-09-30T22:19:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.149+0000 7f24ee4d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e809d630 0x7f24e80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.149+0000 7f24ee4d3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f24e8138d80 0x7f24e813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.149+0000 7f24ee4d3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e80a2a70 con 0x7f24e80a0a30 2023-09-30T22:19:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.149+0000 7f24ee4d3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e80a01d0 con 0x7f24e8138d80 2023-09-30T22:19:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.149+0000 7f24ee4d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24e809e2b0 con 0x7f24e809d630 2023-09-30T22:19:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24e77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e809d630 0x7f24e80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24e77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e809d630 0x7f24e80a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44258/0 (socket says 172.21.15.89:44258) 2023-09-30T22:19:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24e77fe700 1 -- 172.21.15.89:0/4028813497 learned_addr learned my addr 172.21.15.89:0/4028813497 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24eca70700 1 --2- 172.21.15.89:0/4028813497 >> v2:172.21.15.89:3300/0 conn(0x7f24e8138d80 0x7f24e813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24eca70700 1 -- 172.21.15.89:0/4028813497 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e809d630 msgr2=0x7f24e80a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24eca70700 1 --2- 172.21.15.89:0/4028813497 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e809d630 0x7f24e80a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24eca70700 1 -- 172.21.15.89:0/4028813497 >> v1:172.21.15.89:6789/0 conn(0x7f24e80a0a30 legacy=0x7f24e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24eca70700 1 -- 172.21.15.89:0/4028813497 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e809dc50 con 0x7f24e8138d80 2023-09-30T22:19:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24e77fe700 1 --2- 172.21.15.89:0/4028813497 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e809d630 0x7f24e80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24eca70700 1 --2- 172.21.15.89:0/4028813497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 0x7f24e813b160 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009d90 tx=0x7f24e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98226deb9872b88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24e6ffd700 1 -- 172.21.15.89:0/4028813497 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000e070 con 0x7f24e8138d80 2023-09-30T22:19:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24e6ffd700 1 -- 172.21.15.89:0/4028813497 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000bb20 con 0x7f24e8138d80 2023-09-30T22:19:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.150+0000 7f24e6ffd700 1 -- 172.21.15.89:0/4028813497 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0012430 con 0x7f24e8138d80 2023-09-30T22:19:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.151+0000 7f24ee4d3700 1 -- 172.21.15.89:0/4028813497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 msgr2=0x7f24e813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.151+0000 7f24ee4d3700 1 --2- 172.21.15.89:0/4028813497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 0x7f24e813b160 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009d90 tx=0x7f24e000b330 comp rx=0 tx=0).stop 2023-09-30T22:19:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.151+0000 7f24ee4d3700 1 -- 172.21.15.89:0/4028813497 shutdown_connections 2023-09-30T22:19:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.151+0000 7f24ee4d3700 1 --2- 172.21.15.89:0/4028813497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 0x7f24e813b160 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.151+0000 7f24ee4d3700 1 --2- 172.21.15.89:0/4028813497 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e809d630 0x7f24e80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.151+0000 7f24ee4d3700 1 -- 172.21.15.89:0/4028813497 >> 172.21.15.89:0/4028813497 conn(0x7f24e80a3050 msgr2=0x7f24e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.151+0000 7f24ee4d3700 1 -- 172.21.15.89:0/4028813497 shutdown_connections 2023-09-30T22:19:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.151+0000 7f24ee4d3700 1 -- 172.21.15.89:0/4028813497 wait complete. 2023-09-30T22:19:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.152+0000 7f24ee4d3700 1 Processor -- start 2023-09-30T22:19:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24ee4d3700 1 -- start start 2023-09-30T22:19:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24ee4d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24e809d630 0x7f24e8133750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24ee4d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e80a0a30 0x7f24e8133c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24ee4d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 0x7f24e81341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24ee4d3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e8134bb0 con 0x7f24e8138d80 2023-09-30T22:19:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24ee4d3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e8136c40 con 0x7f24e809d630 2023-09-30T22:19:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24ee4d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24e81389f0 con 0x7f24e80a0a30 2023-09-30T22:19:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24e77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e80a0a30 0x7f24e8133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24e77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e80a0a30 0x7f24e8133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44272/0 (socket says 172.21.15.89:44272) 2023-09-30T22:19:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24e77fe700 1 -- 172.21.15.89:0/2427308712 learned_addr learned my addr 172.21.15.89:0/2427308712 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24e7fff700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24e809d630 0x7f24e8133750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.153+0000 7f24eca70700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 0x7f24e81341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e77fe700 1 -- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24e809d630 msgr2=0x7f24e8133750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e77fe700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24e809d630 0x7f24e8133750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e77fe700 1 -- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 msgr2=0x7f24e81341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e77fe700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 0x7f24e81341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e77fe700 1 -- 172.21.15.89:0/2427308712 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc009550 con 0x7f24e80a0a30 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e7fff700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24e809d630 0x7f24e8133750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24eca70700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 0x7f24e81341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e77fe700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e80a0a30 0x7f24e8133c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00ebe0 tx=0x7f24dc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e4ff9700 1 -- 172.21.15.89:0/2427308712 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00c850 con 0x7f24e80a0a30 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e4ff9700 1 -- 172.21.15.89:0/2427308712 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc00c9b0 con 0x7f24e80a0a30 2023-09-30T22:19:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e0009a70 con 0x7f24e80a0a30 2023-09-30T22:19:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24e4ff9700 1 -- 172.21.15.89:0/2427308712 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00cd40 con 0x7f24e80a0a30 2023-09-30T22:19:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.154+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e80a0410 con 0x7f24e80a0a30 2023-09-30T22:19:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.155+0000 7f24e4ff9700 1 -- 172.21.15.89:0/2427308712 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24dc013470 con 0x7f24e80a0a30 2023-09-30T22:19:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.155+0000 7f24e4ff9700 1 -- 172.21.15.89:0/2427308712 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f24dc03fc30 con 0x7f24e80a0a30 2023-09-30T22:19:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.155+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e80747f0 con 0x7f24e80a0a30 2023-09-30T22:19:45.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.160+0000 7f24e4ff9700 1 -- 172.21.15.89:0/2427308712 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24dc01b3b0 con 0x7f24e80a0a30 2023-09-30T22:19:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.189+0000 7fda0fc41700 1 Processor -- start 2023-09-30T22:19:45.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.189+0000 7fda0fc41700 1 -- start start 2023-09-30T22:19:45.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0fc41700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0809d6f0 0x7fda0809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0fc41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda0813d650 0x7fda0813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0fc41700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080992c0 con 0x7fda0809e000 2023-09-30T22:19:45.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0fc41700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda08099520 con 0x7fda0813d650 2023-09-30T22:19:45.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0fc41700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda080997e0 con 0x7fda0809d6f0 2023-09-30T22:19:45.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0d9dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0809d6f0 0x7fda0809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0d9dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0809d6f0 0x7fda0809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44286/0 (socket says 172.21.15.89:44286) 2023-09-30T22:19:45.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0d9dd700 1 -- 172.21.15.89:0/3623986987 learned_addr learned my addr 172.21.15.89:0/3623986987 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:45.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.190+0000 7fda0d1dc700 1 --1- 172.21.15.89:0/3623986987 >> v1:172.21.15.89:6789/0 conn(0x7fda0809e000 0x7fda0813bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37932/0 (socket says 172.21.15.89:37932) 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0d9dd700 1 -- 172.21.15.89:0/3623986987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda0813d650 msgr2=0x7fda0813fa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0d9dd700 1 --2- 172.21.15.89:0/3623986987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda0813d650 0x7fda0813fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0d9dd700 1 -- 172.21.15.89:0/3623986987 >> v1:172.21.15.89:6789/0 conn(0x7fda0809e000 legacy=0x7fda0813bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0c9db700 1 -- 172.21.15.89:0/3623986987 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886268839 0 0) 0x7fda080992c0 con 0x7fda0809e000 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0d9dd700 1 -- 172.21.15.89:0/3623986987 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08099b30 con 0x7fda0809d6f0 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0d9dd700 1 --2- 172.21.15.89:0/3623986987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0809d6f0 0x7fda0809dac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda04009920 tx=0x7fda04009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55365d1c735dddbe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0c9db700 1 -- 172.21.15.89:0/3623986987 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400ba60 con 0x7fda0809d6f0 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0c9db700 1 -- 172.21.15.89:0/3623986987 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0400bbc0 con 0x7fda0809d6f0 2023-09-30T22:19:45.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.191+0000 7fda0c9db700 1 -- 172.21.15.89:0/3623986987 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04009e70 con 0x7fda0809d6f0 2023-09-30T22:19:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.192+0000 7fda0fc41700 1 -- 172.21.15.89:0/3623986987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0809d6f0 msgr2=0x7fda0809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.192+0000 7fda0fc41700 1 --2- 172.21.15.89:0/3623986987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0809d6f0 0x7fda0809dac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda04009920 tx=0x7fda04009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.192+0000 7fda0fc41700 1 -- 172.21.15.89:0/3623986987 shutdown_connections 2023-09-30T22:19:45.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.192+0000 7fda0fc41700 1 --2- 172.21.15.89:0/3623986987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda0813d650 0x7fda0813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.192+0000 7fda0fc41700 1 --2- 172.21.15.89:0/3623986987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0809d6f0 0x7fda0809dac0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.192+0000 7fda0fc41700 1 -- 172.21.15.89:0/3623986987 >> 172.21.15.89:0/3623986987 conn(0x7fda0809f470 msgr2=0x7fda080a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.192+0000 7fda0fc41700 1 -- 172.21.15.89:0/3623986987 shutdown_connections 2023-09-30T22:19:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.192+0000 7fda0fc41700 1 -- 172.21.15.89:0/3623986987 wait complete. 2023-09-30T22:19:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0fc41700 1 Processor -- start 2023-09-30T22:19:45.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0fc41700 1 -- start start 2023-09-30T22:19:45.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0fc41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0809d6f0 0x7fda08070830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0fc41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda0809e000 0x7fda08070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0fc41700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0813d650 0x7fda080712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0fc41700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08128910 con 0x7fda0809d6f0 2023-09-30T22:19:45.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0fc41700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda08127cc0 con 0x7fda0809e000 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0fc41700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda080724e0 con 0x7fda0813d650 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0d9dd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0809d6f0 0x7fda08070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0d9dd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0809d6f0 0x7fda08070830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35876/0 (socket says 172.21.15.89:35876) 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.194+0000 7fda0d9dd700 1 -- 172.21.15.89:0/782259964 learned_addr learned my addr 172.21.15.89:0/782259964 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0e1de700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0813d650 0x7fda080712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0d9dd700 1 -- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0813d650 msgr2=0x7fda080712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0d9dd700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0813d650 0x7fda080712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0d9dd700 1 -- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda0809e000 msgr2=0x7fda08070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0d9dd700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda0809e000 0x7fda08070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0d9dd700 1 -- 172.21.15.89:0/782259964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04009550 con 0x7fda0809d6f0 2023-09-30T22:19:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0e1de700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0813d650 0x7fda080712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0d9dd700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0809d6f0 0x7fda08070830 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fda04000c00 tx=0x7fda04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fd9fa7fc700 1 -- 172.21.15.89:0/782259964 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040043c0 con 0x7fda0809d6f0 2023-09-30T22:19:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fd9fa7fc700 1 -- 172.21.15.89:0/782259964 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0401f070 con 0x7fda0809d6f0 2023-09-30T22:19:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fd9fa7fc700 1 -- 172.21.15.89:0/782259964 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040036a0 con 0x7fda0809d6f0 2023-09-30T22:19:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda080a1c90 con 0x7fda0809d6f0 2023-09-30T22:19:45.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.195+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda08128e10 con 0x7fda0809d6f0 2023-09-30T22:19:45.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.196+0000 7fd9fa7fc700 1 -- 172.21.15.89:0/782259964 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda04021030 con 0x7fda0809d6f0 2023-09-30T22:19:45.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.196+0000 7fd9fa7fc700 1 -- 172.21.15.89:0/782259964 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fda04040b40 con 0x7fda0809d6f0 2023-09-30T22:19:45.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.196+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda080747f0 con 0x7fda0809d6f0 2023-09-30T22:19:45.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.205+0000 7fd9fa7fc700 1 -- 172.21.15.89:0/782259964 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda04015080 con 0x7fda0809d6f0 2023-09-30T22:19:45.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.233+0000 7f94607ee700 1 Processor -- start 2023-09-30T22:19:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.233+0000 7f94607ee700 1 -- start start 2023-09-30T22:19:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f94607ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f945809d6f0 0x7f945809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f94607ee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809e000 0x7f94581339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f94607ee700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945809b120 con 0x7f945809e410 2023-09-30T22:19:45.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f94607ee700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945806b460 con 0x7f945809d6f0 2023-09-30T22:19:45.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f94607ee700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945809a7e0 con 0x7f945809e000 2023-09-30T22:19:45.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945dd89700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809e000 0x7f94581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945e58a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f945809d6f0 0x7f945809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945e58a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f945809d6f0 0x7f945809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35878/0 (socket says 172.21.15.89:35878) 2023-09-30T22:19:45.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945dd89700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809e000 0x7f94581339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55086/0 (socket says 172.21.15.89:55086) 2023-09-30T22:19:45.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945e58a700 1 -- 172.21.15.89:0/2718752609 learned_addr learned my addr 172.21.15.89:0/2718752609 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:45.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945dd89700 1 -- 172.21.15.89:0/2718752609 >> v2:172.21.15.89:3300/0 conn(0x7f945809d6f0 msgr2=0x7f945809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945dd89700 1 --2- 172.21.15.89:0/2718752609 >> v2:172.21.15.89:3300/0 conn(0x7f945809d6f0 0x7f945809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945dd89700 1 -- 172.21.15.89:0/2718752609 >> v1:172.21.15.89:6789/0 conn(0x7f945809e410 legacy=0x7f9458133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945dd89700 1 -- 172.21.15.89:0/2718752609 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945809a500 con 0x7f945809e000 2023-09-30T22:19:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945e58a700 1 --2- 172.21.15.89:0/2718752609 >> v2:172.21.15.89:3300/0 conn(0x7f945809d6f0 0x7f945809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945dd89700 1 --2- 172.21.15.89:0/2718752609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809e000 0x7f94581339f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9448009870 tx=0x7f9448009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c130e4b9dfb867 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.234+0000 7f945d588700 1 -- 172.21.15.89:0/2718752609 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448011830 con 0x7f945809e000 2023-09-30T22:19:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f945d588700 1 -- 172.21.15.89:0/2718752609 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448011990 con 0x7f945809e000 2023-09-30T22:19:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f945d588700 1 -- 172.21.15.89:0/2718752609 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448011d20 con 0x7f945809e000 2023-09-30T22:19:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f94607ee700 1 -- 172.21.15.89:0/2718752609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809e000 msgr2=0x7f94581339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f94607ee700 1 --2- 172.21.15.89:0/2718752609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809e000 0x7f94581339f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9448009870 tx=0x7f9448009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f94607ee700 1 -- 172.21.15.89:0/2718752609 shutdown_connections 2023-09-30T22:19:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f94607ee700 1 --2- 172.21.15.89:0/2718752609 >> v2:172.21.15.89:3300/0 conn(0x7f945809d6f0 0x7f945809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f94607ee700 1 --2- 172.21.15.89:0/2718752609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809e000 0x7f94581339f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f94607ee700 1 -- 172.21.15.89:0/2718752609 >> 172.21.15.89:0/2718752609 conn(0x7f9458066f50 msgr2=0x7f945809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f94607ee700 1 -- 172.21.15.89:0/2718752609 shutdown_connections 2023-09-30T22:19:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.235+0000 7f94607ee700 1 -- 172.21.15.89:0/2718752609 wait complete. 2023-09-30T22:19:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.236+0000 7f94607ee700 1 Processor -- start 2023-09-30T22:19:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.236+0000 7f94607ee700 1 -- start start 2023-09-30T22:19:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f94607ee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809d6f0 0x7f94581a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f94607ee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f945809e000 0x7f94581a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f94607ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945809e410 0x7f94581a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f94607ee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94581a6b00 con 0x7f945809e410 2023-09-30T22:19:45.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f94607ee700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94581a8690 con 0x7f945809d6f0 2023-09-30T22:19:45.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f94607ee700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94581a7cb0 con 0x7f945809e000 2023-09-30T22:19:45.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945e58a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809d6f0 0x7f94581a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945ed8b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945809e410 0x7f94581a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945e58a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809d6f0 0x7f94581a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55094/0 (socket says 172.21.15.89:55094) 2023-09-30T22:19:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945ed8b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945809e410 0x7f94581a58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35894/0 (socket says 172.21.15.89:35894) 2023-09-30T22:19:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945e58a700 1 -- 172.21.15.89:0/1772138354 learned_addr learned my addr 172.21.15.89:0/1772138354 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945ed8b700 1 -- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f945809e000 msgr2=0x7f94581a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945ed8b700 1 --2- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f945809e000 0x7f94581a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945ed8b700 1 -- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809d6f0 msgr2=0x7f94581a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945ed8b700 1 --2- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809d6f0 0x7f94581a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945ed8b700 1 -- 172.21.15.89:0/1772138354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9454009550 con 0x7f945809e410 2023-09-30T22:19:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945e58a700 1 --2- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809d6f0 0x7f94581a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.237+0000 7f945ed8b700 1 --2- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945809e410 0x7f94581a58d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f945000cb90 tx=0x7f945000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.238+0000 7f944effd700 1 -- 172.21.15.89:0/1772138354 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450003c00 con 0x7f945809e410 2023-09-30T22:19:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.238+0000 7f944effd700 1 -- 172.21.15.89:0/1772138354 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9450003d60 con 0x7f945809e410 2023-09-30T22:19:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.238+0000 7f944effd700 1 -- 172.21.15.89:0/1772138354 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000aaa0 con 0x7f945809e410 2023-09-30T22:19:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.238+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94480094a0 con 0x7f945809e410 2023-09-30T22:19:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.238+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945809c710 con 0x7f945809e410 2023-09-30T22:19:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.239+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94581a8330 con 0x7f945809e410 2023-09-30T22:19:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.239+0000 7f944effd700 1 -- 172.21.15.89:0/1772138354 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f945000ac00 con 0x7f945809e410 2023-09-30T22:19:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.240+0000 7f944effd700 1 -- 172.21.15.89:0/1772138354 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f94500400f0 con 0x7f945809e410 2023-09-30T22:19:45.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.244+0000 7f944effd700 1 -- 172.21.15.89:0/1772138354 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9450011560 con 0x7f945809e410 2023-09-30T22:19:45.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.340+0000 7f7934a63700 1 Processor -- start 2023-09-30T22:19:45.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.340+0000 7f7934a63700 1 -- start start 2023-09-30T22:19:45.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f7934a63700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 0x7f79300a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f7934a63700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7930138d80 0x7f793013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f7934a63700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79300a2a70 con 0x7f79300a0a30 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f7934a63700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79300a01d0 con 0x7f7930138d80 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792ed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7930138d80 0x7f793013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f7934a63700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f793009e2b0 con 0x7f793009d630 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792ed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7930138d80 0x7f793013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55110/0 (socket says 172.21.15.89:55110) 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792ed9d700 1 -- 172.21.15.89:0/3858647193 learned_addr learned my addr 172.21.15.89:0/3858647193 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792e59c700 1 --1- 172.21.15.89:0/3858647193 >> v1:172.21.15.89:6789/0 conn(0x7f79300a0a30 0x7f79300a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37942/0 (socket says 172.21.15.89:37942) 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792dd9b700 1 --2- 172.21.15.89:0/3858647193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 0x7f79300a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792ed9d700 1 -- 172.21.15.89:0/3858647193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 msgr2=0x7f79300a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792ed9d700 1 --2- 172.21.15.89:0/3858647193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 0x7f79300a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792ed9d700 1 -- 172.21.15.89:0/3858647193 >> v1:172.21.15.89:6789/0 conn(0x7f79300a0a30 legacy=0x7f79300a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792ed9d700 1 -- 172.21.15.89:0/3858647193 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793009dc50 con 0x7f7930138d80 2023-09-30T22:19:45.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792ed9d700 1 --2- 172.21.15.89:0/3858647193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7930138d80 0x7f793013b160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7928009e80 tx=0x7f792800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a498e283d01d7a89 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:45.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.341+0000 7f792d59a700 1 -- 172.21.15.89:0/3858647193 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800e070 con 0x7f7930138d80 2023-09-30T22:19:45.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.342+0000 7f792d59a700 1 -- 172.21.15.89:0/3858647193 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800b910 con 0x7f7930138d80 2023-09-30T22:19:45.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.342+0000 7f792d59a700 1 -- 172.21.15.89:0/3858647193 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800bcc0 con 0x7f7930138d80 2023-09-30T22:19:45.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.342+0000 7f7934a63700 1 -- 172.21.15.89:0/3858647193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7930138d80 msgr2=0x7f793013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.342+0000 7f7934a63700 1 --2- 172.21.15.89:0/3858647193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7930138d80 0x7f793013b160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7928009e80 tx=0x7f792800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:45.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.342+0000 7f7934a63700 1 -- 172.21.15.89:0/3858647193 shutdown_connections 2023-09-30T22:19:45.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.342+0000 7f7934a63700 1 --2- 172.21.15.89:0/3858647193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7930138d80 0x7f793013b160 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.342+0000 7f7934a63700 1 --2- 172.21.15.89:0/3858647193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 0x7f79300a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.342+0000 7f7934a63700 1 -- 172.21.15.89:0/3858647193 >> 172.21.15.89:0/3858647193 conn(0x7f79300a3050 msgr2=0x7f79300a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:45.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.343+0000 7f7934a63700 1 -- 172.21.15.89:0/3858647193 shutdown_connections 2023-09-30T22:19:45.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.343+0000 7f7934a63700 1 -- 172.21.15.89:0/3858647193 wait complete. 2023-09-30T22:19:45.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f7934a63700 1 Processor -- start 2023-09-30T22:19:45.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f7934a63700 1 -- start start 2023-09-30T22:19:45.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f7934a63700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 0x7f79301338f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f7934a63700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79300a0a30 0x7f7930133e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f7934a63700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7930138d80 0x7f7930134370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f7934a63700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7930136da0 con 0x7f7930138d80 2023-09-30T22:19:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f7934a63700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79300a0410 con 0x7f79300a0a30 2023-09-30T22:19:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f7934a63700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7930136a30 con 0x7f793009d630 2023-09-30T22:19:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f792dd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79300a0a30 0x7f7930133e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f792dd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79300a0a30 0x7f7930133e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55124/0 (socket says 172.21.15.89:55124) 2023-09-30T22:19:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f792dd9b700 1 -- 172.21.15.89:0/412460791 learned_addr learned my addr 172.21.15.89:0/412460791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.344+0000 7f792ed9d700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7930138d80 0x7f7930134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f792e59c700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 0x7f79301338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f792dd9b700 1 -- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 msgr2=0x7f79301338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f792dd9b700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 0x7f79301338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f792dd9b700 1 -- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7930138d80 msgr2=0x7f7930134370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f792dd9b700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7930138d80 0x7f7930134370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f792dd9b700 1 -- 172.21.15.89:0/412460791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928009b30 con 0x7f79300a0a30 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f792dd9b700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79300a0a30 0x7f7930133e30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f792400e9b0 tx=0x7f792400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f791effd700 1 -- 172.21.15.89:0/412460791 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924009710 con 0x7f79300a0a30 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f7934a63700 1 -- 172.21.15.89:0/412460791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7930137ad0 con 0x7f79300a0a30 2023-09-30T22:19:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f791effd700 1 -- 172.21.15.89:0/412460791 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792400c850 con 0x7f79300a0a30 2023-09-30T22:19:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f791effd700 1 -- 172.21.15.89:0/412460791 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792400cbc0 con 0x7f79300a0a30 2023-09-30T22:19:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.345+0000 7f7934a63700 1 -- 172.21.15.89:0/412460791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7930137340 con 0x7f79300a0a30 2023-09-30T22:19:45.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.347+0000 7f791cff9700 1 -- 172.21.15.89:0/412460791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79301386b0 con 0x7f79300a0a30 2023-09-30T22:19:45.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.347+0000 7f791effd700 1 -- 172.21.15.89:0/412460791 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f792400cd20 con 0x7f79300a0a30 2023-09-30T22:19:45.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.348+0000 7f791effd700 1 -- 172.21.15.89:0/412460791 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f792403f9d0 con 0x7f79300a0a30 2023-09-30T22:19:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.356+0000 7f791effd700 1 -- 172.21.15.89:0/412460791 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79240232f0 con 0x7f79300a0a30 2023-09-30T22:19:45.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.370+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f24e8136590 con 0x7f24e80a0a30 2023-09-30T22:19:45.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.371+0000 7f24e4ff9700 1 -- 172.21.15.89:0/2427308712 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24dc011b00 con 0x7f24e80a0a30 2023-09-30T22:19:45.372 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:45.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e80a0a30 msgr2=0x7f24e8133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e80a0a30 0x7f24e8133c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00ebe0 tx=0x7f24dc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 shutdown_connections 2023-09-30T22:19:45.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e8138d80 0x7f24e81341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24e809d630 0x7f24e8133750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 --2- 172.21.15.89:0/2427308712 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f24e80a0a30 0x7f24e8133c90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 >> 172.21.15.89:0/2427308712 conn(0x7f24e80a3050 msgr2=0x7f24e809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:45.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 shutdown_connections 2023-09-30T22:19:45.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.373+0000 7f24ee4d3700 1 -- 172.21.15.89:0/2427308712 wait complete. 2023-09-30T22:19:45.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.440+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9458074ea0 con 0x7f945809e410 2023-09-30T22:19:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.440+0000 7f944effd700 1 -- 172.21.15.89:0/1772138354 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9450011380 con 0x7f945809e410 2023-09-30T22:19:45.442 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945809e410 msgr2=0x7f94581a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 --2- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945809e410 0x7f94581a58d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f945000cb90 tx=0x7f945000cea0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 shutdown_connections 2023-09-30T22:19:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 --2- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f945809e410 0x7f94581a58d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 --2- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f945809d6f0 0x7f94581a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 --2- 172.21.15.89:0/1772138354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f945809e000 0x7f94581a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 >> 172.21.15.89:0/1772138354 conn(0x7f9458066f50 msgr2=0x7f94581baf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 shutdown_connections 2023-09-30T22:19:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.443+0000 7f94607ee700 1 -- 172.21.15.89:0/1772138354 wait complete. 2023-09-30T22:19:45.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:45.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.466+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fda08097de0 con 0x7fda0809d6f0 2023-09-30T22:19:45.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.466+0000 7fd9fa7fc700 1 -- 172.21.15.89:0/782259964 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda040199a0 con 0x7fda0809d6f0 2023-09-30T22:19:45.467 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:45.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.469+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0809d6f0 msgr2=0x7fda08070830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.469+0000 7fda0fc41700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0809d6f0 0x7fda08070830 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fda04000c00 tx=0x7fda04004750 comp rx=0 tx=0).stop 2023-09-30T22:19:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.469+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 shutdown_connections 2023-09-30T22:19:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.469+0000 7fda0fc41700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda0813d650 0x7fda080712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.469+0000 7fda0fc41700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0809d6f0 0x7fda08070830 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.469+0000 7fda0fc41700 1 --2- 172.21.15.89:0/782259964 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda0809e000 0x7fda08070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.469+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 >> 172.21.15.89:0/782259964 conn(0x7fda0809f470 msgr2=0x7fda080a0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.469+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 shutdown_connections 2023-09-30T22:19:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.470+0000 7fda0fc41700 1 -- 172.21.15.89:0/782259964 wait complete. 2023-09-30T22:19:45.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:45.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.558+0000 7f791cff9700 1 -- 172.21.15.89:0/412460791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f79300747f0 con 0x7f79300a0a30 2023-09-30T22:19:45.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.558+0000 7f791effd700 1 -- 172.21.15.89:0/412460791 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f792401ae80 con 0x7f79300a0a30 2023-09-30T22:19:45.559 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 -- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79300a0a30 msgr2=0x7f7930133e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79300a0a30 0x7f7930133e30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f792400e9b0 tx=0x7f792400ed70 comp rx=0 tx=0).stop 2023-09-30T22:19:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 -- 172.21.15.89:0/412460791 shutdown_connections 2023-09-30T22:19:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7930138d80 0x7f7930134370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f793009d630 0x7f79301338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 --2- 172.21.15.89:0/412460791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79300a0a30 0x7f7930133e30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 -- 172.21.15.89:0/412460791 >> 172.21.15.89:0/412460791 conn(0x7f79300a3050 msgr2=0x7f793009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 -- 172.21.15.89:0/412460791 shutdown_connections 2023-09-30T22:19:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:45.560+0000 7f791cff9700 1 -- 172.21.15.89:0/412460791 wait complete. 2023-09-30T22:19:45.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:46.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:46.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:46.483 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:46.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.529+0000 7fdf9637c700 1 Processor -- start 2023-09-30T22:19:46.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.529+0000 7fdf9637c700 1 -- start start 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf9637c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf9637c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf9009a580 0x7fdf90139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf9637c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf901313c0 con 0x7fdf90099a40 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf9637c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9009b580 con 0x7fdf9009a580 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf9637c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9006e8c0 con 0x7fdf90131810 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf94919700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf9009a580 0x7fdf90139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf94919700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf9009a580 0x7fdf90139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35912/0 (socket says 172.21.15.89:35912) 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf94919700 1 -- 172.21.15.89:0/3210609728 learned_addr learned my addr 172.21.15.89:0/3210609728 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:46.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8f7fe700 1 --1- 172.21.15.89:0/3210609728 >> v1:172.21.15.89:6789/0 conn(0x7fdf90099a40 0x7fdf90099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37944/0 (socket says 172.21.15.89:37944) 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8effd700 1 -- 172.21.15.89:0/3210609728 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646017279 0 0) 0x7fdf901313c0 con 0x7fdf90099a40 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8effd700 1 -- 172.21.15.89:0/3210609728 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf780035b0 con 0x7fdf90099a40 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8ffff700 1 -- 172.21.15.89:0/3210609728 >> v2:172.21.15.89:3300/0 conn(0x7fdf9009a580 msgr2=0x7fdf90139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8ffff700 1 --2- 172.21.15.89:0/3210609728 >> v2:172.21.15.89:3300/0 conn(0x7fdf9009a580 0x7fdf90139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8ffff700 1 -- 172.21.15.89:0/3210609728 >> v1:172.21.15.89:6789/0 conn(0x7fdf90099a40 legacy=0x7fdf90099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8ffff700 1 -- 172.21.15.89:0/3210609728 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf90071d20 con 0x7fdf90131810 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf94919700 1 --2- 172.21.15.89:0/3210609728 >> v2:172.21.15.89:3300/0 conn(0x7fdf9009a580 0x7fdf90139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.530+0000 7fdf8ffff700 1 --2- 172.21.15.89:0/3210609728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90131be0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009870 tx=0x7fdf80009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8ffe3964ba4d7e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf8effd700 1 -- 172.21.15.89:0/3210609728 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80011830 con 0x7fdf90131810 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf8effd700 1 -- 172.21.15.89:0/3210609728 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf80011990 con 0x7fdf90131810 2023-09-30T22:19:46.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf8effd700 1 -- 172.21.15.89:0/3210609728 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80011d20 con 0x7fdf90131810 2023-09-30T22:19:46.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf9637c700 1 -- 172.21.15.89:0/3210609728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 msgr2=0x7fdf90131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf9637c700 1 --2- 172.21.15.89:0/3210609728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90131be0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009870 tx=0x7fdf80009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:46.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf9637c700 1 -- 172.21.15.89:0/3210609728 shutdown_connections 2023-09-30T22:19:46.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf9637c700 1 --2- 172.21.15.89:0/3210609728 >> v2:172.21.15.89:3300/0 conn(0x7fdf9009a580 0x7fdf90139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf9637c700 1 --2- 172.21.15.89:0/3210609728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90131be0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf9637c700 1 -- 172.21.15.89:0/3210609728 >> 172.21.15.89:0/3210609728 conn(0x7fdf900a2aa0 msgr2=0x7fdf900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:46.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf9637c700 1 -- 172.21.15.89:0/3210609728 shutdown_connections 2023-09-30T22:19:46.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.531+0000 7fdf9637c700 1 -- 172.21.15.89:0/3210609728 wait complete. 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.532+0000 7fdf9637c700 1 Processor -- start 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf9637c700 1 -- start start 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf9637c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90099a40 0x7fdf901194d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf9637c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf9009a580 0x7fdf90119a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf9637c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90119f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf9637c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9011ca50 con 0x7fdf90099a40 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf9637c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9011c3a0 con 0x7fdf9009a580 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf9637c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9006eb00 con 0x7fdf90131810 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90099a40 0x7fdf901194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90099a40 0x7fdf901194d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35926/0 (socket says 172.21.15.89:35926) 2023-09-30T22:19:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf9009a580 0x7fdf90119a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8ffff700 1 -- 172.21.15.89:0/3605003864 learned_addr learned my addr 172.21.15.89:0/3605003864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf94919700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90119f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8ffff700 1 -- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 msgr2=0x7fdf90119f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8ffff700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90119f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8ffff700 1 -- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf9009a580 msgr2=0x7fdf90119a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8ffff700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf9009a580 0x7fdf90119a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8ffff700 1 -- 172.21.15.89:0/3605003864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88009b30 con 0x7fdf90099a40 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.533+0000 7fdf8f7fe700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf9009a580 0x7fdf90119a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.534+0000 7fdf8ffff700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90099a40 0x7fdf901194d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdf80000c00 tx=0x7fdf80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.534+0000 7fdf8cff9700 1 -- 172.21.15.89:0/3605003864 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80004400 con 0x7fdf90099a40 2023-09-30T22:19:46.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.534+0000 7fdf8cff9700 1 -- 172.21.15.89:0/3605003864 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8001f070 con 0x7fdf90099a40 2023-09-30T22:19:46.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.534+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf800094a0 con 0x7fdf90099a40 2023-09-30T22:19:46.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.534+0000 7fdf8cff9700 1 -- 172.21.15.89:0/3605003864 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf800038e0 con 0x7fdf90099a40 2023-09-30T22:19:46.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.534+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9011bee0 con 0x7fdf90099a40 2023-09-30T22:19:46.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.535+0000 7fdf8cff9700 1 -- 172.21.15.89:0/3605003864 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf80021030 con 0x7fdf90099a40 2023-09-30T22:19:46.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.535+0000 7fdf8cff9700 1 -- 172.21.15.89:0/3605003864 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdf80040900 con 0x7fdf90099a40 2023-09-30T22:19:46.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.536+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf90074ea0 con 0x7fdf90099a40 2023-09-30T22:19:46.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.545+0000 7fdf8cff9700 1 -- 172.21.15.89:0/3605003864 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf80015080 con 0x7fdf90099a40 2023-09-30T22:19:46.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:46.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.598+0000 7f898db65700 1 Processor -- start 2023-09-30T22:19:46.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.598+0000 7f898db65700 1 -- start start 2023-09-30T22:19:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f898db65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809e1d0 0x7f8988135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f898db65700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f8988138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f898db65700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898809f130 con 0x7f898809d6f0 2023-09-30T22:19:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f898db65700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898809f520 con 0x7f8988135e50 2023-09-30T22:19:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f898db65700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f898809e930 con 0x7f898809e1d0 2023-09-30T22:19:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f8987fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f8988138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f8987fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f8988138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55136/0 (socket says 172.21.15.89:55136) 2023-09-30T22:19:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f8987fff700 1 -- 172.21.15.89:0/1802623830 learned_addr learned my addr 172.21.15.89:0/1802623830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.599+0000 7f8986ffd700 1 --2- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809e1d0 0x7f8988135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f8987fff700 1 -- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809e1d0 msgr2=0x7f8988135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f8987fff700 1 --2- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809e1d0 0x7f8988135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f8987fff700 1 -- 172.21.15.89:0/1802623830 >> v1:172.21.15.89:6789/0 conn(0x7f898809d6f0 legacy=0x7f898809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f8987fff700 1 -- 172.21.15.89:0/1802623830 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898809b2d0 con 0x7f8988135e50 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f89867fc700 1 -- 172.21.15.89:0/1802623830 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751580157 0 0) 0x7f898809f130 con 0x7f898809d6f0 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f8986ffd700 1 --2- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809e1d0 0x7f8988135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f8987fff700 1 --2- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f8988138230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8980009e80 tx=0x7f898000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35de4b0452a5b75b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f89867fc700 1 -- 172.21.15.89:0/1802623830 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898000e070 con 0x7f8988135e50 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f89867fc700 1 -- 172.21.15.89:0/1802623830 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898000b910 con 0x7f8988135e50 2023-09-30T22:19:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.600+0000 7f89867fc700 1 -- 172.21.15.89:0/1802623830 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898000bcc0 con 0x7f8988135e50 2023-09-30T22:19:46.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.601+0000 7f898db65700 1 -- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 msgr2=0x7f8988138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.601+0000 7f898db65700 1 --2- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f8988138230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8980009e80 tx=0x7f898000b330 comp rx=0 tx=0).stop 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.601+0000 7f898db65700 1 -- 172.21.15.89:0/1802623830 shutdown_connections 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.601+0000 7f898db65700 1 --2- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f8988138230 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.601+0000 7f898db65700 1 --2- 172.21.15.89:0/1802623830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809e1d0 0x7f8988135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.601+0000 7f898db65700 1 -- 172.21.15.89:0/1802623830 >> 172.21.15.89:0/1802623830 conn(0x7f8988066f50 msgr2=0x7f898809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.601+0000 7f898db65700 1 -- 172.21.15.89:0/1802623830 shutdown_connections 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.601+0000 7f898db65700 1 -- 172.21.15.89:0/1802623830 wait complete. 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.603+0000 7f898db65700 1 Processor -- start 2023-09-30T22:19:46.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.603+0000 7f898db65700 1 -- start start 2023-09-30T22:19:46.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.603+0000 7f898db65700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809d6f0 0x7f898811d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.603+0000 7f898db65700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898809e1d0 0x7f898811de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.603+0000 7f898db65700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f89881c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.603+0000 7f898db65700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898811e8c0 con 0x7f898809e1d0 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.603+0000 7f898db65700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8988124120 con 0x7f8988135e50 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.603+0000 7f898db65700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f898811d5b0 con 0x7f898809d6f0 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f89881c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f89881c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55148/0 (socket says 172.21.15.89:55148) 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 -- 172.21.15.89:0/1185274091 learned_addr learned my addr 172.21.15.89:0/1185274091 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f89877fe700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809d6f0 0x7f898811d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8986ffd700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898809e1d0 0x7f898811de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 -- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809d6f0 msgr2=0x7f898811d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809d6f0 0x7f898811d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 -- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898809e1d0 msgr2=0x7f898811de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898809e1d0 0x7f898811de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 -- 172.21.15.89:0/1185274091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0094a0 con 0x7f8988135e50 2023-09-30T22:19:46.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f8987fff700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f89881c8280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f898000b680 tx=0x7f8980004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:46.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f896ffff700 1 -- 172.21.15.89:0/1185274091 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980004450 con 0x7f8988135e50 2023-09-30T22:19:46.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.604+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8980009b30 con 0x7f8988135e50 2023-09-30T22:19:46.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.605+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898806a9e0 con 0x7f8988135e50 2023-09-30T22:19:46.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.605+0000 7f896ffff700 1 -- 172.21.15.89:0/1185274091 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89800092e0 con 0x7f8988135e50 2023-09-30T22:19:46.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.605+0000 7f896ffff700 1 -- 172.21.15.89:0/1185274091 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89800047b0 con 0x7f8988135e50 2023-09-30T22:19:46.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.606+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89680052e0 con 0x7f8988135e50 2023-09-30T22:19:46.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.606+0000 7f896ffff700 1 -- 172.21.15.89:0/1185274091 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8980025460 con 0x7f8988135e50 2023-09-30T22:19:46.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.606+0000 7f896ffff700 1 -- 172.21.15.89:0/1185274091 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f89800412e0 con 0x7f8988135e50 2023-09-30T22:19:46.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.614+0000 7f896ffff700 1 -- 172.21.15.89:0/1185274091 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8980025710 con 0x7f8988135e50 2023-09-30T22:19:46.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.644+0000 7f1dc51c1700 1 Processor -- start 2023-09-30T22:19:46.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.644+0000 7f1dc51c1700 1 -- start start 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dc51c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc009d6f0 0x7f1dc009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dc51c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dc51c1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc009f130 con 0x7f1dc009e410 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dc51c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc009f520 con 0x7f1dc009e000 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dc51c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc009e930 con 0x7f1dc009d6f0 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dbe59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dbe59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55156/0 (socket says 172.21.15.89:55156) 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dbe59c700 1 -- 172.21.15.89:0/3402153427 learned_addr learned my addr 172.21.15.89:0/3402153427 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dbe59c700 1 -- 172.21.15.89:0/3402153427 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc009d6f0 msgr2=0x7f1dc009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dbe59c700 1 --2- 172.21.15.89:0/3402153427 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc009d6f0 0x7f1dc009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dbe59c700 1 -- 172.21.15.89:0/3402153427 >> v1:172.21.15.89:6789/0 conn(0x7f1dc009e410 legacy=0x7f1dc0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:46.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dbe59c700 1 -- 172.21.15.89:0/3402153427 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc009b2d0 con 0x7f1dc009e000 2023-09-30T22:19:46.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.645+0000 7f1dbe59c700 1 --2- 172.21.15.89:0/3402153427 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0130660 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009920 tx=0x7f1db4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=870cbaf30d5ab950 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:46.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dbdd9b700 1 -- 172.21.15.89:0/3402153427 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db4004230 con 0x7f1dc009e000 2023-09-30T22:19:46.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dbdd9b700 1 -- 172.21.15.89:0/3402153427 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db4004390 con 0x7f1dc009e000 2023-09-30T22:19:46.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dbdd9b700 1 -- 172.21.15.89:0/3402153427 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db4009e70 con 0x7f1dc009e000 2023-09-30T22:19:46.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dc51c1700 1 -- 172.21.15.89:0/3402153427 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 msgr2=0x7f1dc0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dc51c1700 1 --2- 172.21.15.89:0/3402153427 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0130660 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009920 tx=0x7f1db4009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:46.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dc51c1700 1 -- 172.21.15.89:0/3402153427 shutdown_connections 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dc51c1700 1 --2- 172.21.15.89:0/3402153427 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc009d6f0 0x7f1dc009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dc51c1700 1 --2- 172.21.15.89:0/3402153427 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0130660 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dc51c1700 1 -- 172.21.15.89:0/3402153427 >> 172.21.15.89:0/3402153427 conn(0x7f1dc0066f50 msgr2=0x7f1dc009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dc51c1700 1 -- 172.21.15.89:0/3402153427 shutdown_connections 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.646+0000 7f1dc51c1700 1 -- 172.21.15.89:0/3402153427 wait complete. 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dc51c1700 1 Processor -- start 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dc51c1700 1 -- start start 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dc51c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc009d6f0 0x7f1dc0122410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dc51c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0122950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dc51c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc009e410 0x7f1dc0122e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dc51c1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc0125f30 con 0x7f1dc009e410 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dc51c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc0124020 con 0x7f1dc009e000 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dc51c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc00a1be0 con 0x7f1dc009d6f0 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dbf59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc009e410 0x7f1dc0122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dbf59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc009e410 0x7f1dc0122e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35938/0 (socket says 172.21.15.89:35938) 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dbf59e700 1 -- 172.21.15.89:0/736265142 learned_addr learned my addr 172.21.15.89:0/736265142 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.648+0000 7f1dbe59c700 1 --2- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0122950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1dbf59e700 1 -- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc009d6f0 msgr2=0x7f1dc0122410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1dbf59e700 1 --2- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc009d6f0 0x7f1dc0122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1dbf59e700 1 -- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 msgr2=0x7f1dc0122950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1dbf59e700 1 --2- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0122950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1dbf59e700 1 -- 172.21.15.89:0/736265142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4009550 con 0x7f1dc009e410 2023-09-30T22:19:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1dbf59e700 1 --2- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc009e410 0x7f1dc0122e90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c7a0 tx=0x7f1db800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1da77fe700 1 -- 172.21.15.89:0/736265142 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8003e10 con 0x7f1dc009e410 2023-09-30T22:19:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1da77fe700 1 -- 172.21.15.89:0/736265142 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db8005070 con 0x7f1dc009e410 2023-09-30T22:19:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1da77fe700 1 -- 172.21.15.89:0/736265142 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8005420 con 0x7f1dc009e410 2023-09-30T22:19:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc009c700 con 0x7f1dc009e410 2023-09-30T22:19:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.649+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc0124810 con 0x7f1dc009e410 2023-09-30T22:19:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.650+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc0074ea0 con 0x7f1dc009e410 2023-09-30T22:19:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.651+0000 7f1da77fe700 1 -- 172.21.15.89:0/736265142 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1db8006ba0 con 0x7f1dc009e410 2023-09-30T22:19:46.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.651+0000 7f1da77fe700 1 -- 172.21.15.89:0/736265142 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1db803eb70 con 0x7f1dc009e410 2023-09-30T22:19:46.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.655+0000 7f1da77fe700 1 -- 172.21.15.89:0/736265142 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1db80108f0 con 0x7f1dc009e410 2023-09-30T22:19:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.728+0000 7fb5e5b07700 1 Processor -- start 2023-09-30T22:19:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.728+0000 7fb5e5b07700 1 -- start start 2023-09-30T22:19:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.728+0000 7fb5e5b07700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb5e01338a0 0x7fb5e0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.728+0000 7fb5e5b07700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e009de50 0x7fb5e009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.728+0000 7fb5e5b07700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009e760 0x7fb5e013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.728+0000 7fb5e5b07700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e006fef0 con 0x7fb5e01338a0 2023-09-30T22:19:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.728+0000 7fb5e5b07700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e006ef30 con 0x7fb5e009de50 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.728+0000 7fb5e5b07700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e01334b0 con 0x7fb5e009e760 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009e760 0x7fb5e013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009e760 0x7fb5e013a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44408/0 (socket says 172.21.15.89:44408) 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 -- 172.21.15.89:0/3817433546 learned_addr learned my addr 172.21.15.89:0/3817433546 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5df7fe700 1 --2- 172.21.15.89:0/3817433546 >> v2:172.21.15.89:3300/0 conn(0x7fb5e01338a0 0x7fb5e0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5deffd700 1 --2- 172.21.15.89:0/3817433546 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e009de50 0x7fb5e009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 -- 172.21.15.89:0/3817433546 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e009de50 msgr2=0x7fb5e009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 --2- 172.21.15.89:0/3817433546 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e009de50 0x7fb5e009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 -- 172.21.15.89:0/3817433546 >> v2:172.21.15.89:3300/0 conn(0x7fb5e01338a0 msgr2=0x7fb5e0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:46.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 --2- 172.21.15.89:0/3817433546 >> v2:172.21.15.89:3300/0 conn(0x7fb5e01338a0 0x7fb5e0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 -- 172.21.15.89:0/3817433546 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e006fc30 con 0x7fb5e009e760 2023-09-30T22:19:46.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5df7fe700 1 --2- 172.21.15.89:0/3817433546 >> v2:172.21.15.89:3300/0 conn(0x7fb5e01338a0 0x7fb5e0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:46.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5dffff700 1 --2- 172.21.15.89:0/3817433546 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009e760 0x7fb5e013a730 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009e50 tx=0x7fb5d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff7d5115fabb8a90 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:46.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5de7fc700 1 -- 172.21.15.89:0/3817433546 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800e070 con 0x7fb5e009e760 2023-09-30T22:19:46.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5de7fc700 1 -- 172.21.15.89:0/3817433546 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d800b910 con 0x7fb5e009e760 2023-09-30T22:19:46.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.729+0000 7fb5de7fc700 1 -- 172.21.15.89:0/3817433546 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800bcc0 con 0x7fb5e009e760 2023-09-30T22:19:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 -- 172.21.15.89:0/3817433546 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009e760 msgr2=0x7fb5e013a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 --2- 172.21.15.89:0/3817433546 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009e760 0x7fb5e013a730 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009e50 tx=0x7fb5d800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 -- 172.21.15.89:0/3817433546 shutdown_connections 2023-09-30T22:19:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 --2- 172.21.15.89:0/3817433546 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009e760 0x7fb5e013a730 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 --2- 172.21.15.89:0/3817433546 >> v2:172.21.15.89:3300/0 conn(0x7fb5e01338a0 0x7fb5e0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 --2- 172.21.15.89:0/3817433546 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e009de50 0x7fb5e009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 -- 172.21.15.89:0/3817433546 >> 172.21.15.89:0/3817433546 conn(0x7fb5e00a2aa0 msgr2=0x7fb5e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 -- 172.21.15.89:0/3817433546 shutdown_connections 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.730+0000 7fb5e5b07700 1 -- 172.21.15.89:0/3817433546 wait complete. 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.731+0000 7fb5e5b07700 1 Processor -- start 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5e5b07700 1 -- start start 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5e5b07700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009de50 0x7fb5e011d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5e5b07700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e009e760 0x7fb5e011de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5e5b07700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e01338a0 0x7fb5e011e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5e5b07700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e011f990 con 0x7fb5e009e760 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5e5b07700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e0130b40 con 0x7fb5e01338a0 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5e5b07700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e006f640 con 0x7fb5e009de50 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5df7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009de50 0x7fb5e011d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5df7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009de50 0x7fb5e011d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44410/0 (socket says 172.21.15.89:44410) 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5df7fe700 1 -- 172.21.15.89:0/1290053333 learned_addr learned my addr 172.21.15.89:0/1290053333 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.732+0000 7fb5deffd700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e009e760 0x7fb5e011de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5dffff700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e01338a0 0x7fb5e011e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5df7fe700 1 -- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e01338a0 msgr2=0x7fb5e011e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5df7fe700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e01338a0 0x7fb5e011e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5df7fe700 1 -- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e009e760 msgr2=0x7fb5e011de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5df7fe700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e009e760 0x7fb5e011de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5df7fe700 1 -- 172.21.15.89:0/1290053333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00094a0 con 0x7fb5e009de50 2023-09-30T22:19:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5dffff700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e01338a0 0x7fb5e011e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:19:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5df7fe700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009de50 0x7fb5e011d900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000ec10 tx=0x7fb5d000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5c7fff700 1 -- 172.21.15.89:0/1290053333 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d000c910 con 0x7fb5e009de50 2023-09-30T22:19:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5c7fff700 1 -- 172.21.15.89:0/1290053333 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d000ca70 con 0x7fb5e009de50 2023-09-30T22:19:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5c7fff700 1 -- 172.21.15.89:0/1290053333 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d000cde0 con 0x7fb5e009de50 2023-09-30T22:19:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d8009b30 con 0x7fb5e009de50 2023-09-30T22:19:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.733+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e01205b0 con 0x7fb5e009de50 2023-09-30T22:19:46.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.734+0000 7fb5c7fff700 1 -- 172.21.15.89:0/1290053333 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb5d0013470 con 0x7fb5e009de50 2023-09-30T22:19:46.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.734+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e0074ea0 con 0x7fb5e009de50 2023-09-30T22:19:46.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.734+0000 7fb5c7fff700 1 -- 172.21.15.89:0/1290053333 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb5d003f7a0 con 0x7fb5e009de50 2023-09-30T22:19:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.743+0000 7fb5c7fff700 1 -- 172.21.15.89:0/1290053333 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5d00232f0 con 0x7fb5e009de50 2023-09-30T22:19:46.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.794+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdf9006b9e0 con 0x7fdf90099a40 2023-09-30T22:19:46.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.794+0000 7fdf8cff9700 1 -- 172.21.15.89:0/3605003864 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf800239c0 con 0x7fdf90099a40 2023-09-30T22:19:46.795 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:46.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90099a40 msgr2=0x7fdf901194d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90099a40 0x7fdf901194d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdf80000c00 tx=0x7fdf80004750 comp rx=0 tx=0).stop 2023-09-30T22:19:46.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 shutdown_connections 2023-09-30T22:19:46.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf90131810 0x7fdf90119f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90099a40 0x7fdf901194d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 --2- 172.21.15.89:0/3605003864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf9009a580 0x7fdf90119a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 >> 172.21.15.89:0/3605003864 conn(0x7fdf900a2aa0 msgr2=0x7fdf9012e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:46.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 shutdown_connections 2023-09-30T22:19:46.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.796+0000 7fdf9637c700 1 -- 172.21.15.89:0/3605003864 wait complete. 2023-09-30T22:19:46.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.813+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8968005590 con 0x7f8988135e50 2023-09-30T22:19:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.813+0000 7f896ffff700 1 -- 172.21.15.89:0/1185274091 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f898002a020 con 0x7f8988135e50 2023-09-30T22:19:46.814 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 msgr2=0x7f89881c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f89881c8280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f898000b680 tx=0x7f8980004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 shutdown_connections 2023-09-30T22:19:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8988135e50 0x7f89881c8280 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f898809d6f0 0x7f898811d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 --2- 172.21.15.89:0/1185274091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898809e1d0 0x7f898811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 >> 172.21.15.89:0/1185274091 conn(0x7f8988066f50 msgr2=0x7f89880a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 shutdown_connections 2023-09-30T22:19:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.815+0000 7f898db65700 1 -- 172.21.15.89:0/1185274091 wait complete. 2023-09-30T22:19:46.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:46.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.851+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1dc006c360 con 0x7f1dc009e410 2023-09-30T22:19:46.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.851+0000 7f1da77fe700 1 -- 172.21.15.89:0/736265142 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1db8017070 con 0x7f1dc009e410 2023-09-30T22:19:46.852 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:46.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc009e410 msgr2=0x7f1dc0122e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 --2- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc009e410 0x7f1dc0122e90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c7a0 tx=0x7f1db800cab0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 shutdown_connections 2023-09-30T22:19:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 --2- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc009e410 0x7f1dc0122e90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 --2- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc009d6f0 0x7f1dc0122410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 --2- 172.21.15.89:0/736265142 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc009e000 0x7f1dc0122950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 >> 172.21.15.89:0/736265142 conn(0x7f1dc0066f50 msgr2=0x7f1dc006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 shutdown_connections 2023-09-30T22:19:46.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.853+0000 7f1dc51c1700 1 -- 172.21.15.89:0/736265142 wait complete. 2023-09-30T22:19:46.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:46.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.936+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb5e006b9e0 con 0x7fb5e009de50 2023-09-30T22:19:46.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.936+0000 7fb5c7fff700 1 -- 172.21.15.89:0/1290053333 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5d0011350 con 0x7fb5e009de50 2023-09-30T22:19:46.937 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:46.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009de50 msgr2=0x7fb5e011d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:46.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009de50 0x7fb5e011d900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000ec10 tx=0x7fb5d000ef20 comp rx=0 tx=0).stop 2023-09-30T22:19:46.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 shutdown_connections 2023-09-30T22:19:46.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5e01338a0 0x7fb5e011e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5e009de50 0x7fb5e011d900 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 --2- 172.21.15.89:0/1290053333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5e009e760 0x7fb5e011de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:46.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 >> 172.21.15.89:0/1290053333 conn(0x7fb5e00a2aa0 msgr2=0x7fb5e01b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:46.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 shutdown_connections 2023-09-30T22:19:46.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:46.938+0000 7fb5e5b07700 1 -- 172.21.15.89:0/1290053333 wait complete. 2023-09-30T22:19:46.950 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:47.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:47.829 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:47.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:47.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.939+0000 7fd597a36700 1 Processor -- start 2023-09-30T22:19:47.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.939+0000 7fd597a36700 1 -- start start 2023-09-30T22:19:47.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd597a36700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009d6f0 0x7fd59009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd597a36700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e000 0x7fd5901339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd597a36700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59009b120 con 0x7fd59009e410 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd597a36700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59006b460 con 0x7fd59009d6f0 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd597a36700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd59009a7e0 con 0x7fd59009e000 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd594fd1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e000 0x7fd5901339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd5957d2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009d6f0 0x7fd59009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd5957d2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009d6f0 0x7fd59009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55188/0 (socket says 172.21.15.89:55188) 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd594fd1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e000 0x7fd5901339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44412/0 (socket says 172.21.15.89:44412) 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd5957d2700 1 -- 172.21.15.89:0/2469713709 learned_addr learned my addr 172.21.15.89:0/2469713709 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd5957d2700 1 -- 172.21.15.89:0/2469713709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e000 msgr2=0x7fd5901339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd5957d2700 1 --2- 172.21.15.89:0/2469713709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e000 0x7fd5901339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd5957d2700 1 -- 172.21.15.89:0/2469713709 >> v1:172.21.15.89:6789/0 conn(0x7fd59009e410 legacy=0x7fd590133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd5957d2700 1 -- 172.21.15.89:0/2469713709 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59009a500 con 0x7fd59009d6f0 2023-09-30T22:19:47.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.940+0000 7fd587fff700 1 -- 172.21.15.89:0/2469713709 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343061423 0 0) 0x7fd59009b120 con 0x7fd59009e410 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd5957d2700 1 --2- 172.21.15.89:0/2469713709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009d6f0 0x7fd59009dac0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009920 tx=0x7fd58c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60965fe1de89c700 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd587fff700 1 -- 172.21.15.89:0/2469713709 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c011830 con 0x7fd59009d6f0 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd587fff700 1 -- 172.21.15.89:0/2469713709 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c011990 con 0x7fd59009d6f0 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd587fff700 1 -- 172.21.15.89:0/2469713709 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c011d20 con 0x7fd59009d6f0 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd597a36700 1 -- 172.21.15.89:0/2469713709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009d6f0 msgr2=0x7fd59009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd597a36700 1 --2- 172.21.15.89:0/2469713709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009d6f0 0x7fd59009dac0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009920 tx=0x7fd58c009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd597a36700 1 -- 172.21.15.89:0/2469713709 shutdown_connections 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd597a36700 1 --2- 172.21.15.89:0/2469713709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009d6f0 0x7fd59009dac0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd597a36700 1 --2- 172.21.15.89:0/2469713709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e000 0x7fd5901339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.941+0000 7fd597a36700 1 -- 172.21.15.89:0/2469713709 >> 172.21.15.89:0/2469713709 conn(0x7fd590066f50 msgr2=0x7fd59009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.942+0000 7fd597a36700 1 -- 172.21.15.89:0/2469713709 shutdown_connections 2023-09-30T22:19:47.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.942+0000 7fd597a36700 1 -- 172.21.15.89:0/2469713709 wait complete. 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.943+0000 7fd597a36700 1 Processor -- start 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.943+0000 7fd597a36700 1 -- start start 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd597a36700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd59009d6f0 0x7fd5901c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd597a36700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009e000 0x7fd5901c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd597a36700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e410 0x7fd5901c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd597a36700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901a80a0 con 0x7fd59009d6f0 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd597a36700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901a6390 con 0x7fd59009e000 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd597a36700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900a1e00 con 0x7fd59009e410 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009e000 0x7fd5901c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009e000 0x7fd5901c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55194/0 (socket says 172.21.15.89:55194) 2023-09-30T22:19:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 -- 172.21.15.89:0/2662796024 learned_addr learned my addr 172.21.15.89:0/2662796024 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd595fd3700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e410 0x7fd5901c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd5957d2700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd59009d6f0 0x7fd5901c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 -- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e410 msgr2=0x7fd5901c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e410 0x7fd5901c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 -- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd59009d6f0 msgr2=0x7fd5901c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd59009d6f0 0x7fd5901c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 -- 172.21.15.89:0/2662796024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c009550 con 0x7fd59009e000 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd594fd1700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009e000 0x7fd5901c8560 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e980 tx=0x7fd58000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd585ffb700 1 -- 172.21.15.89:0/2662796024 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580009710 con 0x7fd59009e000 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd585ffb700 1 -- 172.21.15.89:0/2662796024 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58000c690 con 0x7fd59009e000 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59009c710 con 0x7fd59009e000 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.944+0000 7fd585ffb700 1 -- 172.21.15.89:0/2662796024 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58000ca20 con 0x7fd59009e000 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.945+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5901a7000 con 0x7fd59009e000 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.946+0000 7fd585ffb700 1 -- 172.21.15.89:0/2662796024 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd58000cb80 con 0x7fd59009e000 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.946+0000 7fd585ffb700 1 -- 172.21.15.89:0/2662796024 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd58003f750 con 0x7fd59009e000 2023-09-30T22:19:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.946+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd590074ea0 con 0x7fd59009e000 2023-09-30T22:19:47.951 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.951+0000 7fd585ffb700 1 -- 172.21.15.89:0/2662796024 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd580017080 con 0x7fd59009e000 2023-09-30T22:19:47.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f9003b700 1 Processor -- start 2023-09-30T22:19:47.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f9003b700 1 -- start start 2023-09-30T22:19:47.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f9003b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f8809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f9003b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f8809dfe0 0x7f2f88130600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f9003b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8809b940 con 0x7f2f88130b40 2023-09-30T22:19:47.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f9003b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8809a920 con 0x7f2f8809dfe0 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f9003b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8809ab10 con 0x7f2f8809d6d0 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f8ddd7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f8809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f8ddd7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f8809daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44434/0 (socket says 172.21.15.89:44434) 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.985+0000 7f2f8ddd7700 1 -- 172.21.15.89:0/2846550361 learned_addr learned my addr 172.21.15.89:0/2846550361 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8d5d6700 1 --2- 172.21.15.89:0/2846550361 >> v2:172.21.15.89:3300/0 conn(0x7f2f8809dfe0 0x7f2f88130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8ddd7700 1 -- 172.21.15.89:0/2846550361 >> v2:172.21.15.89:3300/0 conn(0x7f2f8809dfe0 msgr2=0x7f2f88130600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8ddd7700 1 --2- 172.21.15.89:0/2846550361 >> v2:172.21.15.89:3300/0 conn(0x7f2f8809dfe0 0x7f2f88130600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8ddd7700 1 -- 172.21.15.89:0/2846550361 >> v1:172.21.15.89:6789/0 conn(0x7f2f88130b40 legacy=0x7f2f881378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8ddd7700 1 -- 172.21.15.89:0/2846550361 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8809c520 con 0x7f2f8809d6d0 2023-09-30T22:19:47.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8ddd7700 1 --2- 172.21.15.89:0/2846550361 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f8809daa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400bbc0 tx=0x7f2f8400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc9a3518adc29700 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:47.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8cdd5700 1 -- 172.21.15.89:0/2846550361 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400dcf0 con 0x7f2f8809d6d0 2023-09-30T22:19:47.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8cdd5700 1 -- 172.21.15.89:0/2846550361 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8400de50 con 0x7f2f8809d6d0 2023-09-30T22:19:47.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.986+0000 7f2f8cdd5700 1 -- 172.21.15.89:0/2846550361 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84010690 con 0x7f2f8809d6d0 2023-09-30T22:19:47.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.987+0000 7f2f9003b700 1 -- 172.21.15.89:0/2846550361 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 msgr2=0x7f2f8809daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.987+0000 7f2f9003b700 1 --2- 172.21.15.89:0/2846550361 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f8809daa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400bbc0 tx=0x7f2f8400bed0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.987+0000 7f2f9003b700 1 -- 172.21.15.89:0/2846550361 shutdown_connections 2023-09-30T22:19:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.987+0000 7f2f9003b700 1 --2- 172.21.15.89:0/2846550361 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f8809daa0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.987+0000 7f2f9003b700 1 --2- 172.21.15.89:0/2846550361 >> v2:172.21.15.89:3300/0 conn(0x7f2f8809dfe0 0x7f2f88130600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.987+0000 7f2f9003b700 1 -- 172.21.15.89:0/2846550361 >> 172.21.15.89:0/2846550361 conn(0x7f2f88099c20 msgr2=0x7f2f880a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.987+0000 7f2f9003b700 1 -- 172.21.15.89:0/2846550361 shutdown_connections 2023-09-30T22:19:47.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.987+0000 7f2f9003b700 1 -- 172.21.15.89:0/2846550361 wait complete. 2023-09-30T22:19:47.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.988+0000 7f2f9003b700 1 Processor -- start 2023-09-30T22:19:47.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.988+0000 7f2f9003b700 1 -- start start 2023-09-30T22:19:47.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f9003b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f88126820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f9003b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8809dfe0 0x7f2f88126d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f9003b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f88130b40 0x7f2f881272a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:47.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f9003b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881296f0 con 0x7f2f8809dfe0 2023-09-30T22:19:47.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f9003b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88128540 con 0x7f2f88130b40 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f9003b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88129da0 con 0x7f2f8809d6d0 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8d5d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8809dfe0 0x7f2f88126d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8e5d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f88130b40 0x7f2f881272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8d5d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8809dfe0 0x7f2f88126d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35986/0 (socket says 172.21.15.89:35986) 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8d5d6700 1 -- 172.21.15.89:0/861091940 learned_addr learned my addr 172.21.15.89:0/861091940 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8ddd7700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f88126820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8e5d8700 1 -- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 msgr2=0x7f2f88126820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8e5d8700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f88126820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8e5d8700 1 -- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8809dfe0 msgr2=0x7f2f88126d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8e5d8700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8809dfe0 0x7f2f88126d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:47.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8e5d8700 1 -- 172.21.15.89:0/861091940 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8400b7f0 con 0x7f2f88130b40 2023-09-30T22:19:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8d5d6700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8809dfe0 0x7f2f88126d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f8e5d8700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f88130b40 0x7f2f881272a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000cb90 tx=0x7f2f8000cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.989+0000 7f2f7e7fc700 1 -- 172.21.15.89:0/861091940 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80003c00 con 0x7f2f88130b40 2023-09-30T22:19:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.990+0000 7f2f7e7fc700 1 -- 172.21.15.89:0/861091940 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f80003d60 con 0x7f2f88130b40 2023-09-30T22:19:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.990+0000 7f2f7e7fc700 1 -- 172.21.15.89:0/861091940 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000aaa0 con 0x7f2f88130b40 2023-09-30T22:19:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.990+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8809c740 con 0x7f2f88130b40 2023-09-30T22:19:47.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.990+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8812a340 con 0x7f2f88130b40 2023-09-30T22:19:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.991+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8811abc0 con 0x7f2f88130b40 2023-09-30T22:19:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.991+0000 7f2f7e7fc700 1 -- 172.21.15.89:0/861091940 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f8000ac00 con 0x7f2f88130b40 2023-09-30T22:19:47.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.991+0000 7f2f7e7fc700 1 -- 172.21.15.89:0/861091940 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2f8003f690 con 0x7f2f88130b40 2023-09-30T22:19:47.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:47.996+0000 7f2f7e7fc700 1 -- 172.21.15.89:0/861091940 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f800196c0 con 0x7f2f88130b40 2023-09-30T22:19:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.032+0000 7f0b92199700 1 Processor -- start 2023-09-30T22:19:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.032+0000 7f0b92199700 1 -- start start 2023-09-30T22:19:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.032+0000 7f0b92199700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b8c09d630 0x7f0b8c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.032+0000 7f0b92199700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c138d80 0x7f0b8c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.032+0000 7f0b92199700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8c0a2a70 con 0x7f0b8c0a0a30 2023-09-30T22:19:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.032+0000 7f0b92199700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8c0a01d0 con 0x7f0b8c09d630 2023-09-30T22:19:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.032+0000 7f0b92199700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c09e2b0 con 0x7f0b8c138d80 2023-09-30T22:19:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b8c09d630 0x7f0b8c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c138d80 0x7f0b8c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b8c09d630 0x7f0b8c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35992/0 (socket says 172.21.15.89:35992) 2023-09-30T22:19:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8affd700 1 -- 172.21.15.89:0/3235048199 learned_addr learned my addr 172.21.15.89:0/3235048199 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8affd700 1 -- 172.21.15.89:0/3235048199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c138d80 msgr2=0x7f0b8c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8affd700 1 --2- 172.21.15.89:0/3235048199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c138d80 0x7f0b8c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8affd700 1 -- 172.21.15.89:0/3235048199 >> v1:172.21.15.89:6789/0 conn(0x7f0b8c0a0a30 legacy=0x7f0b8c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8affd700 1 -- 172.21.15.89:0/3235048199 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c09dc50 con 0x7f0b8c09d630 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8bfff700 1 --2- 172.21.15.89:0/3235048199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c138d80 0x7f0b8c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8affd700 1 --2- 172.21.15.89:0/3235048199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c09d630 0x7f0b8c0a1510 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0b800098f0 tx=0x7f0b80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282fcfbd15207027 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8a7fc700 1 -- 172.21.15.89:0/3235048199 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80011830 con 0x7f0b8c09d630 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8a7fc700 1 -- 172.21.15.89:0/3235048199 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b80011990 con 0x7f0b8c09d630 2023-09-30T22:19:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.033+0000 7f0b8a7fc700 1 -- 172.21.15.89:0/3235048199 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80011d20 con 0x7f0b8c09d630 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.034+0000 7f0b92199700 1 -- 172.21.15.89:0/3235048199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c09d630 msgr2=0x7f0b8c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.034+0000 7f0b92199700 1 --2- 172.21.15.89:0/3235048199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c09d630 0x7f0b8c0a1510 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0b800098f0 tx=0x7f0b80009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.034+0000 7f0b92199700 1 -- 172.21.15.89:0/3235048199 shutdown_connections 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.034+0000 7f0b92199700 1 --2- 172.21.15.89:0/3235048199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c138d80 0x7f0b8c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.034+0000 7f0b92199700 1 --2- 172.21.15.89:0/3235048199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c09d630 0x7f0b8c0a1510 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.034+0000 7f0b92199700 1 -- 172.21.15.89:0/3235048199 >> 172.21.15.89:0/3235048199 conn(0x7f0b8c0a3050 msgr2=0x7f0b8c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.034+0000 7f0b92199700 1 -- 172.21.15.89:0/3235048199 shutdown_connections 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.034+0000 7f0b92199700 1 -- 172.21.15.89:0/3235048199 wait complete. 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.035+0000 7f0b92199700 1 Processor -- start 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.035+0000 7f0b92199700 1 -- start start 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b92199700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c09d630 0x7f0b8c120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b92199700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b8c0a0a30 0x7f0b8c120d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b92199700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c138d80 0x7f0b8c121280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b92199700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c123fb0 con 0x7f0b8c138d80 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b92199700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0a4e00 con 0x7f0b8c09d630 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b92199700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0a0410 con 0x7f0b8c0a0a30 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c09d630 0x7f0b8c120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c138d80 0x7f0b8c121280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c09d630 0x7f0b8c120800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55226/0 (socket says 172.21.15.89:55226) 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c138d80 0x7f0b8c121280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35998/0 (socket says 172.21.15.89:35998) 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8b7fe700 1 -- 172.21.15.89:0/307911493 learned_addr learned my addr 172.21.15.89:0/307911493 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8affd700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b8c0a0a30 0x7f0b8c120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8bfff700 1 -- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b8c0a0a30 msgr2=0x7f0b8c120d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8bfff700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b8c0a0a30 0x7f0b8c120d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8bfff700 1 -- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c09d630 msgr2=0x7f0b8c120800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8bfff700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c09d630 0x7f0b8c120800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8bfff700 1 -- 172.21.15.89:0/307911493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b84009a70 con 0x7f0b8c138d80 2023-09-30T22:19:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8b7fe700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c09d630 0x7f0b8c120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8affd700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b8c0a0a30 0x7f0b8c120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b8bfff700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c138d80 0x7f0b8c121280 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400ef90 tx=0x7f0b8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.036+0000 7f0b73fff700 1 -- 172.21.15.89:0/307911493 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400cd40 con 0x7f0b8c138d80 2023-09-30T22:19:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.037+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b80009550 con 0x7f0b8c138d80 2023-09-30T22:19:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.037+0000 7f0b73fff700 1 -- 172.21.15.89:0/307911493 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8400f040 con 0x7f0b8c138d80 2023-09-30T22:19:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.037+0000 7f0b73fff700 1 -- 172.21.15.89:0/307911493 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400cea0 con 0x7f0b8c138d80 2023-09-30T22:19:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.037+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c124a20 con 0x7f0b8c138d80 2023-09-30T22:19:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.038+0000 7f0b73fff700 1 -- 172.21.15.89:0/307911493 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b840146c0 con 0x7f0b8c138d80 2023-09-30T22:19:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.038+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c0747f0 con 0x7f0b8c138d80 2023-09-30T22:19:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.038+0000 7f0b73fff700 1 -- 172.21.15.89:0/307911493 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0b84040410 con 0x7f0b8c138d80 2023-09-30T22:19:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.047+0000 7f0b73fff700 1 -- 172.21.15.89:0/307911493 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b840076f0 con 0x7f0b8c138d80 2023-09-30T22:19:48.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.127+0000 7f75e7e78700 1 Processor -- start 2023-09-30T22:19:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.127+0000 7f75e7e78700 1 -- start start 2023-09-30T22:19:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.127+0000 7f75e7e78700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f75e0131810 0x7f75e0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e7e78700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0099a40 0x7f75e0099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e7e78700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e009a3b0 0x7f75e0138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e7e78700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e01313c0 con 0x7f75e0131810 2023-09-30T22:19:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e7e78700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e006e8c0 con 0x7f75e0099a40 2023-09-30T22:19:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e7e78700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0071d20 con 0x7f75e009a3b0 2023-09-30T22:19:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e5c14700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f75e0131810 0x7f75e0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e5c14700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f75e0131810 0x7f75e0131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36006/0 (socket says 172.21.15.89:36006) 2023-09-30T22:19:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e5c14700 1 -- 172.21.15.89:0/2081941747 learned_addr learned my addr 172.21.15.89:0/2081941747 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e6415700 1 --2- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e009a3b0 0x7f75e0138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e5413700 1 --2- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0099a40 0x7f75e0099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e6415700 1 -- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0099a40 msgr2=0x7f75e0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e6415700 1 --2- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0099a40 0x7f75e0099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e6415700 1 -- 172.21.15.89:0/2081941747 >> v2:172.21.15.89:3300/0 conn(0x7f75e0131810 msgr2=0x7f75e0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e6415700 1 --2- 172.21.15.89:0/2081941747 >> v2:172.21.15.89:3300/0 conn(0x7f75e0131810 0x7f75e0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e6415700 1 -- 172.21.15.89:0/2081941747 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00721a0 con 0x7f75e009a3b0 2023-09-30T22:19:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e5c14700 1 --2- 172.21.15.89:0/2081941747 >> v2:172.21.15.89:3300/0 conn(0x7f75e0131810 0x7f75e0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:48.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.128+0000 7f75e6415700 1 --2- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e009a3b0 0x7f75e0138650 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009d90 tx=0x7f75d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e23da2b8f04027a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:48.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e4c12700 1 -- 172.21.15.89:0/2081941747 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d800e070 con 0x7f75e009a3b0 2023-09-30T22:19:48.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e4c12700 1 -- 172.21.15.89:0/2081941747 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d800bb20 con 0x7f75e009a3b0 2023-09-30T22:19:48.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e4c12700 1 -- 172.21.15.89:0/2081941747 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8012430 con 0x7f75e009a3b0 2023-09-30T22:19:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e7e78700 1 -- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e009a3b0 msgr2=0x7f75e0138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e7e78700 1 --2- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e009a3b0 0x7f75e0138650 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009d90 tx=0x7f75d800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e7e78700 1 -- 172.21.15.89:0/2081941747 shutdown_connections 2023-09-30T22:19:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e7e78700 1 --2- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e009a3b0 0x7f75e0138650 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e7e78700 1 --2- 172.21.15.89:0/2081941747 >> v2:172.21.15.89:3300/0 conn(0x7f75e0131810 0x7f75e0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e7e78700 1 --2- 172.21.15.89:0/2081941747 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0099a40 0x7f75e0099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.129+0000 7f75e7e78700 1 -- 172.21.15.89:0/2081941747 >> 172.21.15.89:0/2081941747 conn(0x7f75e00a2aa0 msgr2=0x7f75e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.130+0000 7f75e7e78700 1 -- 172.21.15.89:0/2081941747 shutdown_connections 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.130+0000 7f75e7e78700 1 -- 172.21.15.89:0/2081941747 wait complete. 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.131+0000 7f75e7e78700 1 Processor -- start 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.131+0000 7f75e7e78700 1 -- start start 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.131+0000 7f75e7e78700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e0099a40 0x7f75e006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.131+0000 7f75e7e78700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75e009a3b0 0x7f75e006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.131+0000 7f75e7e78700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0131810 0x7f75e0070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.131+0000 7f75e7e78700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e011f2d0 con 0x7f75e009a3b0 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.131+0000 7f75e7e78700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0070a70 con 0x7f75e0131810 2023-09-30T22:19:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e7e78700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75e011ea60 con 0x7f75e0099a40 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0131810 0x7f75e0070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0131810 0x7f75e0070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55246/0 (socket says 172.21.15.89:55246) 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 -- 172.21.15.89:0/4245944824 learned_addr learned my addr 172.21.15.89:0/4245944824 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e5c14700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e0099a40 0x7f75e006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e5413700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75e009a3b0 0x7f75e006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 -- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e0099a40 msgr2=0x7f75e006f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e0099a40 0x7f75e006f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 -- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75e009a3b0 msgr2=0x7f75e006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75e009a3b0 0x7f75e006fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 -- 172.21.15.89:0/4245944824 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc009550 con 0x7f75e0131810 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e5413700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75e009a3b0 0x7f75e006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75e6415700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0131810 0x7f75e0070090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f75d8014040 tx=0x7f75d8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.132+0000 7f75d67fc700 1 -- 172.21.15.89:0/4245944824 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8012a40 con 0x7f75e0131810 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.133+0000 7f75d67fc700 1 -- 172.21.15.89:0/4245944824 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d8020070 con 0x7f75e0131810 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.133+0000 7f75d67fc700 1 -- 172.21.15.89:0/4245944824 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8012e20 con 0x7f75e0131810 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.133+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d8009a70 con 0x7f75e0131810 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.133+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e006eb00 con 0x7f75e0131810 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.134+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e0074ea0 con 0x7f75e0131810 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.134+0000 7f75d67fc700 1 -- 172.21.15.89:0/4245944824 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f75d8004040 con 0x7f75e0131810 2023-09-30T22:19:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.134+0000 7f75d67fc700 1 -- 172.21.15.89:0/4245944824 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f75d8022070 con 0x7f75e0131810 2023-09-30T22:19:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.143+0000 7f75d67fc700 1 -- 172.21.15.89:0/4245944824 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f75d801a970 con 0x7f75e0131810 2023-09-30T22:19:48.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.157+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd59006c360 con 0x7fd59009e000 2023-09-30T22:19:48.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.157+0000 7fd585ffb700 1 -- 172.21.15.89:0/2662796024 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5800232f0 con 0x7fd59009e000 2023-09-30T22:19:48.160 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009e000 msgr2=0x7fd5901c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009e000 0x7fd5901c8560 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e980 tx=0x7fd58000ec90 comp rx=0 tx=0).stop 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 shutdown_connections 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd59009e410 0x7fd5901c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd59009d6f0 0x7fd5901c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 --2- 172.21.15.89:0/2662796024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd59009e000 0x7fd5901c8560 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 >> 172.21.15.89:0/2662796024 conn(0x7fd590066f50 msgr2=0x7fd5901bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 shutdown_connections 2023-09-30T22:19:48.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.160+0000 7fd597a36700 1 -- 172.21.15.89:0/2662796024 wait complete. 2023-09-30T22:19:48.173 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:48.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.252+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0b8c072980 con 0x7f0b8c138d80 2023-09-30T22:19:48.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.252+0000 7f0b73fff700 1 -- 172.21.15.89:0/307911493 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b84018030 con 0x7f0b8c138d80 2023-09-30T22:19:48.253 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c138d80 msgr2=0x7f0b8c121280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c138d80 0x7f0b8c121280 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400ef90 tx=0x7f0b8400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 shutdown_connections 2023-09-30T22:19:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b8c138d80 0x7f0b8c121280 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b8c09d630 0x7f0b8c120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 --2- 172.21.15.89:0/307911493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b8c0a0a30 0x7f0b8c120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 >> 172.21.15.89:0/307911493 conn(0x7f0b8c0a3050 msgr2=0x7f0b8c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 shutdown_connections 2023-09-30T22:19:48.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.254+0000 7f0b92199700 1 -- 172.21.15.89:0/307911493 wait complete. 2023-09-30T22:19:48.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:48.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.268+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2f8809b3f0 con 0x7f2f88130b40 2023-09-30T22:19:48.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.268+0000 7f2f7e7fc700 1 -- 172.21.15.89:0/861091940 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f80011380 con 0x7f2f88130b40 2023-09-30T22:19:48.269 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:48.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f88130b40 msgr2=0x7f2f881272a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f88130b40 0x7f2f881272a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000cb90 tx=0x7f2f8000cea0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 shutdown_connections 2023-09-30T22:19:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f88130b40 0x7f2f881272a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f8809d6d0 0x7f2f88126820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 --2- 172.21.15.89:0/861091940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f8809dfe0 0x7f2f88126d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 >> 172.21.15.89:0/861091940 conn(0x7f2f88099c20 msgr2=0x7f2f880a1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 shutdown_connections 2023-09-30T22:19:48.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.271+0000 7f2f9003b700 1 -- 172.21.15.89:0/861091940 wait complete. 2023-09-30T22:19:48.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:48.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.354+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f75e012cb80 con 0x7f75e0131810 2023-09-30T22:19:48.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.354+0000 7f75d67fc700 1 -- 172.21.15.89:0/4245944824 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75d801a380 con 0x7f75e0131810 2023-09-30T22:19:48.355 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:48.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.356+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0131810 msgr2=0x7f75e0070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:48.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.356+0000 7f75e7e78700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0131810 0x7f75e0070090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f75d8014040 tx=0x7f75d8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.356+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 shutdown_connections 2023-09-30T22:19:48.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.356+0000 7f75e7e78700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75e0131810 0x7f75e0070090 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.356+0000 7f75e7e78700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f75e0099a40 0x7f75e006f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.356+0000 7f75e7e78700 1 --2- 172.21.15.89:0/4245944824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75e009a3b0 0x7f75e006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:48.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.356+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 >> 172.21.15.89:0/4245944824 conn(0x7f75e00a2aa0 msgr2=0x7f75e012e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:48.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.356+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 shutdown_connections 2023-09-30T22:19:48.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:48.357+0000 7f75e7e78700 1 -- 172.21.15.89:0/4245944824 wait complete. 2023-09-30T22:19:48.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:49.174 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:49.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:49.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.305+0000 7f18bbada700 1 Processor -- start 2023-09-30T22:19:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.305+0000 7f18bbada700 1 -- start start 2023-09-30T22:19:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.305+0000 7f18bbada700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.305+0000 7f18bbada700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e000 0x7f18b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.305+0000 7f18bbada700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18b409e410 0x7f18b4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.305+0000 7f18bbada700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b406b460 con 0x7f18b409e410 2023-09-30T22:19:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.305+0000 7f18bbada700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b409b120 con 0x7f18b409e000 2023-09-30T22:19:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.305+0000 7f18bbada700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18b409a7e0 con 0x7f18b409d6f0 2023-09-30T22:19:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e000 0x7f18b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18ba077700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18b409e410 0x7f18b4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9876700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e000 0x7f18b41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55252/0 (socket says 172.21.15.89:55252) 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 -- 172.21.15.89:0/1453550413 learned_addr learned my addr 172.21.15.89:0/1453550413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 -- 172.21.15.89:0/1453550413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 msgr2=0x7f18b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 --2- 172.21.15.89:0/1453550413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 -- 172.21.15.89:0/1453550413 >> v2:172.21.15.89:3300/0 conn(0x7f18b409e410 msgr2=0x7f18b4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 --2- 172.21.15.89:0/1453550413 >> v2:172.21.15.89:3300/0 conn(0x7f18b409e410 0x7f18b4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 -- 172.21.15.89:0/1453550413 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b409a500 con 0x7f18b409e000 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9876700 1 --2- 172.21.15.89:0/1453550413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18ba077700 1 --2- 172.21.15.89:0/1453550413 >> v2:172.21.15.89:3300/0 conn(0x7f18b409e410 0x7f18b4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b9075700 1 --2- 172.21.15.89:0/1453550413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e000 0x7f18b41339f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009920 tx=0x7f18a4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58af85e0743ff5f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b8874700 1 -- 172.21.15.89:0/1453550413 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4011830 con 0x7f18b409e000 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b8874700 1 -- 172.21.15.89:0/1453550413 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a4011990 con 0x7f18b409e000 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.306+0000 7f18b8874700 1 -- 172.21.15.89:0/1453550413 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4011d20 con 0x7f18b409e000 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 -- 172.21.15.89:0/1453550413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e000 msgr2=0x7f18b41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 --2- 172.21.15.89:0/1453550413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e000 0x7f18b41339f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009920 tx=0x7f18a4009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 -- 172.21.15.89:0/1453550413 shutdown_connections 2023-09-30T22:19:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 --2- 172.21.15.89:0/1453550413 >> v2:172.21.15.89:3300/0 conn(0x7f18b409e410 0x7f18b4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 --2- 172.21.15.89:0/1453550413 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 --2- 172.21.15.89:0/1453550413 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e000 0x7f18b41339f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 -- 172.21.15.89:0/1453550413 >> 172.21.15.89:0/1453550413 conn(0x7f18b4066f50 msgr2=0x7f18b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 -- 172.21.15.89:0/1453550413 shutdown_connections 2023-09-30T22:19:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.307+0000 7f18bbada700 1 -- 172.21.15.89:0/1453550413 wait complete. 2023-09-30T22:19:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.308+0000 7f18bbada700 1 Processor -- start 2023-09-30T22:19:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.308+0000 7f18bbada700 1 -- start start 2023-09-30T22:19:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.308+0000 7f18bbada700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b41a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18bbada700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18b409e000 0x7f18b41a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18bbada700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e410 0x7f18b41a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18bbada700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41a7360 con 0x7f18b409e000 2023-09-30T22:19:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18bbada700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41a7a30 con 0x7f18b409e410 2023-09-30T22:19:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18bbada700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41a8c80 con 0x7f18b409d6f0 2023-09-30T22:19:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9876700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b41a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9876700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b41a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44478/0 (socket says 172.21.15.89:44478) 2023-09-30T22:19:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9876700 1 -- 172.21.15.89:0/3929744907 learned_addr learned my addr 172.21.15.89:0/3929744907 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9075700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18b409e000 0x7f18b41a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18ba077700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e410 0x7f18b41a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9075700 1 -- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 msgr2=0x7f18b41a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9075700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b41a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9075700 1 -- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e410 msgr2=0x7f18b41a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9075700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e410 0x7f18b41a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9075700 1 -- 172.21.15.89:0/3929744907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00094a0 con 0x7f18b409e000 2023-09-30T22:19:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18ba077700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e410 0x7f18b41a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18b9075700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18b409e000 0x7f18b41a63f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f18a4016000 tx=0x7f18a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18aa7fc700 1 -- 172.21.15.89:0/3929744907 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4004240 con 0x7f18b409e000 2023-09-30T22:19:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18aa7fc700 1 -- 172.21.15.89:0/3929744907 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a401f070 con 0x7f18b409e000 2023-09-30T22:19:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.309+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac009a70 con 0x7f18b409e000 2023-09-30T22:19:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.310+0000 7f18aa7fc700 1 -- 172.21.15.89:0/3929744907 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40036a0 con 0x7f18b409e000 2023-09-30T22:19:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.310+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a4009550 con 0x7f18b409e000 2023-09-30T22:19:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.310+0000 7f18aa7fc700 1 -- 172.21.15.89:0/3929744907 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18a4021030 con 0x7f18b409e000 2023-09-30T22:19:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.310+0000 7f18aa7fc700 1 -- 172.21.15.89:0/3929744907 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f18a4040720 con 0x7f18b409e000 2023-09-30T22:19:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.311+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b40a3140 con 0x7f18b409e000 2023-09-30T22:19:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.317+0000 7f18aa7fc700 1 -- 172.21.15.89:0/3929744907 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18a4003800 con 0x7f18b409e000 2023-09-30T22:19:49.370 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.396+0000 7f9a04d00700 1 Processor -- start 2023-09-30T22:19:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f9a04d00700 1 -- start start 2023-09-30T22:19:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f9a04d00700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006b250 0x7f9a00136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f9a04d00700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9a00136d80 0x7f9a00139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f9a04d00700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0009ece0 con 0x7f9a0006a770 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f9a04d00700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0006b640 con 0x7f9a00136d80 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f9a04d00700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0009e760 con 0x7f9a0006b250 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f99fe59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9a0006a770 0x7f9a0006ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37988/0 (socket says 172.21.15.89:37988) 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f99fe59c700 1 -- 172.21.15.89:0/4131434609 learned_addr learned my addr 172.21.15.89:0/4131434609 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f99fdd9b700 1 --2- 172.21.15.89:0/4131434609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006b250 0x7f9a00136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.397+0000 7f99fed9d700 1 --2- 172.21.15.89:0/4131434609 >> v2:172.21.15.89:3300/0 conn(0x7f9a00136d80 0x7f9a00139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.398+0000 7f99fed9d700 1 -- 172.21.15.89:0/4131434609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006b250 msgr2=0x7f9a00136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.398+0000 7f99fed9d700 1 --2- 172.21.15.89:0/4131434609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006b250 0x7f9a00136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.398+0000 7f99fed9d700 1 -- 172.21.15.89:0/4131434609 >> v1:172.21.15.89:6789/0 conn(0x7f9a0006a770 legacy=0x7f9a0006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.398+0000 7f99fed9d700 1 -- 172.21.15.89:0/4131434609 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0009e040 con 0x7f9a00136d80 2023-09-30T22:19:49.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.398+0000 7f99fed9d700 1 --2- 172.21.15.89:0/4131434609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 0x7f9a00139160 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009d90 tx=0x7f99f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bffde4f46bab8eb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:49.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.398+0000 7f99fd59a700 1 -- 172.21.15.89:0/4131434609 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800e070 con 0x7f9a00136d80 2023-09-30T22:19:49.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.398+0000 7f99fd59a700 1 -- 172.21.15.89:0/4131434609 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f800bb20 con 0x7f9a00136d80 2023-09-30T22:19:49.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.398+0000 7f99fd59a700 1 -- 172.21.15.89:0/4131434609 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8012430 con 0x7f9a00136d80 2023-09-30T22:19:49.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.399+0000 7f9a04d00700 1 -- 172.21.15.89:0/4131434609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 msgr2=0x7f9a00139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.399+0000 7f9a04d00700 1 --2- 172.21.15.89:0/4131434609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 0x7f9a00139160 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009d90 tx=0x7f99f800b330 comp rx=0 tx=0).stop 2023-09-30T22:19:49.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.399+0000 7f9a04d00700 1 -- 172.21.15.89:0/4131434609 shutdown_connections 2023-09-30T22:19:49.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.399+0000 7f9a04d00700 1 --2- 172.21.15.89:0/4131434609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 0x7f9a00139160 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.399+0000 7f9a04d00700 1 --2- 172.21.15.89:0/4131434609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006b250 0x7f9a00136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.399+0000 7f9a04d00700 1 -- 172.21.15.89:0/4131434609 >> 172.21.15.89:0/4131434609 conn(0x7f9a00066f50 msgr2=0x7f9a0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.399+0000 7f9a04d00700 1 -- 172.21.15.89:0/4131434609 shutdown_connections 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.399+0000 7f9a04d00700 1 -- 172.21.15.89:0/4131434609 wait complete. 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.400+0000 7f9a04d00700 1 Processor -- start 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.400+0000 7f9a04d00700 1 -- start start 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f9a04d00700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006a770 0x7f9a00120330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f9a04d00700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a0006b250 0x7f9a0011ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f9a04d00700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 0x7f9a0011ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f9a04d00700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0006e8e0 con 0x7f9a00136d80 2023-09-30T22:19:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f9a04d00700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a00071e60 con 0x7f9a0006a770 2023-09-30T22:19:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f9a04d00700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0011c6c0 con 0x7f9a0006b250 2023-09-30T22:19:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f99fe59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006a770 0x7f9a00120330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f99fe59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006a770 0x7f9a00120330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55280/0 (socket says 172.21.15.89:55280) 2023-09-30T22:19:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f99fe59c700 1 -- 172.21.15.89:0/1276335775 learned_addr learned my addr 172.21.15.89:0/1276335775 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f99fed9d700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 0x7f9a0011ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.401+0000 7f99fdd9b700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a0006b250 0x7f9a0011ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99fed9d700 1 -- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a0006b250 msgr2=0x7f9a0011ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99fed9d700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a0006b250 0x7f9a0011ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99fed9d700 1 -- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006a770 msgr2=0x7f9a00120330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99fed9d700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006a770 0x7f9a00120330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99fed9d700 1 -- 172.21.15.89:0/1276335775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8009a70 con 0x7f9a00136d80 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99fe59c700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006a770 0x7f9a00120330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99fed9d700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 0x7f9a0011ef50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f99f8016000 tx=0x7f99f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99eeffd700 1 -- 172.21.15.89:0/1276335775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8012430 con 0x7f9a00136d80 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99eeffd700 1 -- 172.21.15.89:0/1276335775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f8020070 con 0x7f9a00136d80 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0011db50 con 0x7f9a00136d80 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f99eeffd700 1 -- 172.21.15.89:0/1276335775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f80098f0 con 0x7f9a00136d80 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.402+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0011de30 con 0x7f9a00136d80 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.403+0000 7f99eeffd700 1 -- 172.21.15.89:0/1276335775 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99f8004040 con 0x7f9a00136d80 2023-09-30T22:19:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.403+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a00074ea0 con 0x7f9a00136d80 2023-09-30T22:19:49.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.403+0000 7f99eeffd700 1 -- 172.21.15.89:0/1276335775 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f99f8022070 con 0x7f9a00136d80 2023-09-30T22:19:49.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.412+0000 7f99eeffd700 1 -- 172.21.15.89:0/1276335775 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99f802c020 con 0x7f9a00136d80 2023-09-30T22:19:49.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.434+0000 7f3ef20f2700 1 Processor -- start 2023-09-30T22:19:49.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3ef20f2700 1 -- start start 2023-09-30T22:19:49.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3ef20f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3ef20f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e000 0x7f3eec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3ef20f2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eec09e410 0x7f3eec133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3ef20f2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec06b460 con 0x7f3eec09e410 2023-09-30T22:19:49.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3ef20f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec09b120 con 0x7f3eec09e000 2023-09-30T22:19:49.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3ef20f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec09a7e0 con 0x7f3eec09d6f0 2023-09-30T22:19:49.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3eeb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3eebfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eec09e410 0x7f3eec133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.435+0000 7f3eeb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44500/0 (socket says 172.21.15.89:44500) 2023-09-30T22:19:49.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eeb7fe700 1 -- 172.21.15.89:0/2033572230 learned_addr learned my addr 172.21.15.89:0/2033572230 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:49.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eeaffd700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e000 0x7f3eec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eebfff700 1 -- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 msgr2=0x7f3eec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eebfff700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eebfff700 1 -- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e000 msgr2=0x7f3eec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eebfff700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e000 0x7f3eec1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eebfff700 1 -- 172.21.15.89:0/2033572230 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec09a500 con 0x7f3eec09e410 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eeb7fe700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eeaffd700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e000 0x7f3eec1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eebfff700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e410 0x7f3eec133f30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d90 tx=0x7f3ee400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b0d3d559863fac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eea7fc700 1 -- 172.21.15.89:0/2033572230 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400e070 con 0x7f3eec09e410 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eea7fc700 1 -- 172.21.15.89:0/2033572230 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee400bb20 con 0x7f3eec09e410 2023-09-30T22:19:49.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.436+0000 7f3eea7fc700 1 -- 172.21.15.89:0/2033572230 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4012430 con 0x7f3eec09e410 2023-09-30T22:19:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 -- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e410 msgr2=0x7f3eec133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e410 0x7f3eec133f30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d90 tx=0x7f3ee400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 -- 172.21.15.89:0/2033572230 shutdown_connections 2023-09-30T22:19:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e410 0x7f3eec133f30 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 --2- 172.21.15.89:0/2033572230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e000 0x7f3eec1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 -- 172.21.15.89:0/2033572230 >> 172.21.15.89:0/2033572230 conn(0x7f3eec066f50 msgr2=0x7f3eec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:49.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 -- 172.21.15.89:0/2033572230 shutdown_connections 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.437+0000 7f3ef20f2700 1 -- 172.21.15.89:0/2033572230 wait complete. 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.438+0000 7f3ef20f2700 1 Processor -- start 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.438+0000 7f3ef20f2700 1 -- start start 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3ef20f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec1a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3ef20f2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e000 0x7f3eec1a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3ef20f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e410 0x7f3eec1a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3ef20f2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1a8560 con 0x7f3eec09e000 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3ef20f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1a8920 con 0x7f3eec09e410 2023-09-30T22:19:49.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3ef20f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1a9180 con 0x7f3eec09d6f0 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e000 0x7f3eec1a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e000 0x7f3eec1a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36088/0 (socket says 172.21.15.89:36088) 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eeaffd700 1 -- 172.21.15.89:0/1550359474 learned_addr learned my addr 172.21.15.89:0/1550359474 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eebfff700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e410 0x7f3eec1a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eeb7fe700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec1a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eebfff700 1 -- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 msgr2=0x7f3eec1a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eebfff700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec1a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eebfff700 1 -- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e000 msgr2=0x7f3eec1a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eebfff700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e000 0x7f3eec1a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.439+0000 7f3eebfff700 1 -- 172.21.15.89:0/1550359474 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc009550 con 0x7f3eec09e410 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.440+0000 7f3eeaffd700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e000 0x7f3eec1a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.440+0000 7f3eeb7fe700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec1a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.440+0000 7f3eebfff700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e410 0x7f3eec1a6ca0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3eec09e8b0 tx=0x7f3ee4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.440+0000 7f3ed3fff700 1 -- 172.21.15.89:0/1550359474 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004900 con 0x7f3eec09e410 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.440+0000 7f3ed3fff700 1 -- 172.21.15.89:0/1550359474 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee4020070 con 0x7f3eec09e410 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.440+0000 7f3ed3fff700 1 -- 172.21.15.89:0/1550359474 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40276c0 con 0x7f3eec09e410 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.440+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee4009a70 con 0x7f3eec09e410 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.440+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec1a9eb0 con 0x7f3eec09e410 2023-09-30T22:19:49.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.441+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec074ea0 con 0x7f3eec09e410 2023-09-30T22:19:49.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.442+0000 7f3ed3fff700 1 -- 172.21.15.89:0/1550359474 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ee4012430 con 0x7f3eec09e410 2023-09-30T22:19:49.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.442+0000 7f3ed3fff700 1 -- 172.21.15.89:0/1550359474 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3ee4022070 con 0x7f3eec09e410 2023-09-30T22:19:49.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.450+0000 7f3ed3fff700 1 -- 172.21.15.89:0/1550359474 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ee401b400 con 0x7f3eec09e410 2023-09-30T22:19:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.525+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f18b409c710 con 0x7f18b409e000 2023-09-30T22:19:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.525+0000 7f18aa7fc700 1 -- 172.21.15.89:0/3929744907 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18a4015020 con 0x7f18b409e000 2023-09-30T22:19:49.526 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18b409e000 msgr2=0x7f18b41a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18b409e000 0x7f18b41a63f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f18a4016000 tx=0x7f18a4004750 comp rx=0 tx=0).stop 2023-09-30T22:19:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 shutdown_connections 2023-09-30T22:19:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b409e410 0x7f18b41a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18b409d6f0 0x7f18b41a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 --2- 172.21.15.89:0/3929744907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18b409e000 0x7f18b41a63f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 >> 172.21.15.89:0/3929744907 conn(0x7f18b4066f50 msgr2=0x7f18b41bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 shutdown_connections 2023-09-30T22:19:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.528+0000 7f18bbada700 1 -- 172.21.15.89:0/3929744907 wait complete. 2023-09-30T22:19:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.531+0000 7fde57297700 1 Processor -- start 2023-09-30T22:19:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.532+0000 7fde57297700 1 -- start start 2023-09-30T22:19:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.532+0000 7fde57297700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde50136830 0x7fde50138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.532+0000 7fde57297700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde5013b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.532+0000 7fde57297700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5006f720 con 0x7fde501338c0 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.532+0000 7fde57297700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde50070c80 con 0x7fde50136830 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.532+0000 7fde57297700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde5006fce0 con 0x7fde50139150 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde54832700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde50136830 0x7fde50138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde54832700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde50136830 0x7fde50138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36104/0 (socket says 172.21.15.89:36104) 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde54832700 1 -- 172.21.15.89:0/3306275450 learned_addr learned my addr 172.21.15.89:0/3306275450 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde55834700 1 --2- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde5013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde54832700 1 -- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 msgr2=0x7fde5013b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde54832700 1 --2- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde5013b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde54832700 1 -- 172.21.15.89:0/3306275450 >> v1:172.21.15.89:6789/0 conn(0x7fde501338c0 legacy=0x7fde50133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde54832700 1 -- 172.21.15.89:0/3306275450 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde501336a0 con 0x7fde50136830 2023-09-30T22:19:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde55834700 1 --2- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde5013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde54832700 1 --2- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde50138c10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fde400098f0 tx=0x7fde40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b3dd847c5ca1ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde47fff700 1 -- 172.21.15.89:0/3306275450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40011830 con 0x7fde50136830 2023-09-30T22:19:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde47fff700 1 -- 172.21.15.89:0/3306275450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40011990 con 0x7fde50136830 2023-09-30T22:19:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.533+0000 7fde47fff700 1 -- 172.21.15.89:0/3306275450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40011d20 con 0x7fde50136830 2023-09-30T22:19:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.534+0000 7fde57297700 1 -- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 msgr2=0x7fde50138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.534+0000 7fde57297700 1 --2- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde50138c10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fde400098f0 tx=0x7fde40009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.535+0000 7fde57297700 1 -- 172.21.15.89:0/3306275450 shutdown_connections 2023-09-30T22:19:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.535+0000 7fde57297700 1 --2- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde5013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.535+0000 7fde57297700 1 --2- 172.21.15.89:0/3306275450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde50138c10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.535+0000 7fde57297700 1 -- 172.21.15.89:0/3306275450 >> 172.21.15.89:0/3306275450 conn(0x7fde500a2aa0 msgr2=0x7fde500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.535+0000 7fde57297700 1 -- 172.21.15.89:0/3306275450 shutdown_connections 2023-09-30T22:19:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.535+0000 7fde57297700 1 -- 172.21.15.89:0/3306275450 wait complete. 2023-09-30T22:19:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.536+0000 7fde57297700 1 Processor -- start 2023-09-30T22:19:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.536+0000 7fde57297700 1 -- start start 2023-09-30T22:19:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde57297700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501338c0 0x7fde50119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde57297700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde5011a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde57297700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde501c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde57297700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5011c9a0 con 0x7fde50136830 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde57297700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde50119640 con 0x7fde501338c0 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde57297700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde5006f960 con 0x7fde50139150 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde54832700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde5011a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde54832700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde5011a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36114/0 (socket says 172.21.15.89:36114) 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde54832700 1 -- 172.21.15.89:0/3426341537 learned_addr learned my addr 172.21.15.89:0/3426341537 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde55033700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501338c0 0x7fde50119c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde55834700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde501c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde54832700 1 -- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 msgr2=0x7fde501c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde54832700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde501c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde54832700 1 -- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501338c0 msgr2=0x7fde50119c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde54832700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501338c0 0x7fde50119c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.537+0000 7fde54832700 1 -- 172.21.15.89:0/3426341537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48009a70 con 0x7fde50136830 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.538+0000 7fde54832700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde5011a160 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fde40009c40 tx=0x7fde40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.538+0000 7fde45ffb700 1 -- 172.21.15.89:0/3426341537 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40004400 con 0x7fde50136830 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.538+0000 7fde45ffb700 1 -- 172.21.15.89:0/3426341537 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4001f070 con 0x7fde50136830 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.538+0000 7fde45ffb700 1 -- 172.21.15.89:0/3426341537 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde400038e0 con 0x7fde50136830 2023-09-30T22:19:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.538+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde40009550 con 0x7fde50136830 2023-09-30T22:19:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.538+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde50136110 con 0x7fde50136830 2023-09-30T22:19:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.539+0000 7fde45ffb700 1 -- 172.21.15.89:0/3426341537 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde40021030 con 0x7fde50136830 2023-09-30T22:19:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.539+0000 7fde45ffb700 1 -- 172.21.15.89:0/3426341537 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fde40040990 con 0x7fde50136830 2023-09-30T22:19:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.539+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde50074ea0 con 0x7fde50136830 2023-09-30T22:19:49.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.545+0000 7fde45ffb700 1 -- 172.21.15.89:0/3426341537 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde40015080 con 0x7fde50136830 2023-09-30T22:19:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.615+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9a0006c360 con 0x7f9a00136d80 2023-09-30T22:19:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.616+0000 7f99eeffd700 1 -- 172.21.15.89:0/1276335775 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99f8025970 con 0x7f9a00136d80 2023-09-30T22:19:49.617 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 msgr2=0x7f9a0011ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 0x7f9a0011ef50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f99f8016000 tx=0x7f99f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 shutdown_connections 2023-09-30T22:19:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a00136d80 0x7f9a0011ef50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a0006a770 0x7f9a00120330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 --2- 172.21.15.89:0/1276335775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a0006b250 0x7f9a0011ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 >> 172.21.15.89:0/1276335775 conn(0x7f9a00066f50 msgr2=0x7f9a000a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 shutdown_connections 2023-09-30T22:19:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.618+0000 7f9a04d00700 1 -- 172.21.15.89:0/1276335775 wait complete. 2023-09-30T22:19:49.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.720+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3eec09f420 con 0x7f3eec09e410 2023-09-30T22:19:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.720+0000 7f3ed3fff700 1 -- 172.21.15.89:0/1550359474 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ee402c030 con 0x7f3eec09e410 2023-09-30T22:19:49.721 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e410 msgr2=0x7f3eec1a6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e410 0x7f3eec1a6ca0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3eec09e8b0 tx=0x7f3ee4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 shutdown_connections 2023-09-30T22:19:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3eec09e410 0x7f3eec1a6ca0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3eec09d6f0 0x7f3eec1a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 --2- 172.21.15.89:0/1550359474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eec09e000 0x7f3eec1a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 >> 172.21.15.89:0/1550359474 conn(0x7f3eec066f50 msgr2=0x7f3eec1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 shutdown_connections 2023-09-30T22:19:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.722+0000 7f3ef20f2700 1 -- 172.21.15.89:0/1550359474 wait complete. 2023-09-30T22:19:49.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:49.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.772+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde50072120 con 0x7fde50136830 2023-09-30T22:19:49.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.773+0000 7fde45ffb700 1 -- 172.21.15.89:0/3426341537 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde40004560 con 0x7fde50136830 2023-09-30T22:19:49.773 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:49.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 msgr2=0x7fde5011a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde5011a160 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fde40009c40 tx=0x7fde40004750 comp rx=0 tx=0).stop 2023-09-30T22:19:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 shutdown_connections 2023-09-30T22:19:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde50139150 0x7fde501c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501338c0 0x7fde50119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 --2- 172.21.15.89:0/3426341537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde50136830 0x7fde5011a160 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 >> 172.21.15.89:0/3426341537 conn(0x7fde500a2aa0 msgr2=0x7fde5012f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 shutdown_connections 2023-09-30T22:19:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:49.775+0000 7fde57297700 1 -- 172.21.15.89:0/3426341537 wait complete. 2023-09-30T22:19:49.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:50.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:50.631 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:50.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.669+0000 7f6184f0a700 1 Processor -- start 2023-09-30T22:19:50.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.669+0000 7f6184f0a700 1 -- start start 2023-09-30T22:19:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f6184f0a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61800a1510 0x7f6180138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f6184f0a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6180139250 0x7f618013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f6184f0a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618009edd0 con 0x7f61800a0a30 2023-09-30T22:19:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f6184f0a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61800a18e0 con 0x7f6180139250 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f6184f0a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f618009e1f0 con 0x7f61800a1510 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617ed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6180139250 0x7f618013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617e59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61800a0a30 0x7f61800a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53474/0 (socket says 172.21.15.89:53474) 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617ed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6180139250 0x7f618013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41858/0 (socket says 172.21.15.89:41858) 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617dd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61800a1510 0x7f6180138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617e59c700 1 -- 172.21.15.89:0/4001280607 learned_addr learned my addr 172.21.15.89:0/4001280607 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617dd9b700 1 -- 172.21.15.89:0/4001280607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6180139250 msgr2=0x7f618013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617dd9b700 1 --2- 172.21.15.89:0/4001280607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6180139250 0x7f618013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617dd9b700 1 -- 172.21.15.89:0/4001280607 >> v1:172.21.15.89:6789/0 conn(0x7f61800a0a30 legacy=0x7f61800a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617d59a700 1 -- 172.21.15.89:0/4001280607 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088614619 0 0) 0x7f618009edd0 con 0x7f61800a0a30 2023-09-30T22:19:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617dd9b700 1 -- 172.21.15.89:0/4001280607 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618009dc50 con 0x7f61800a1510 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617ed9d700 1 --2- 172.21.15.89:0/4001280607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6180139250 0x7f618013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617dd9b700 1 --2- 172.21.15.89:0/4001280607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61800a1510 0x7f6180138d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6174009870 tx=0x7f6174009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d254c4a72d25b582 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617d59a700 1 -- 172.21.15.89:0/4001280607 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174011830 con 0x7f61800a1510 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617d59a700 1 -- 172.21.15.89:0/4001280607 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174011990 con 0x7f61800a1510 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.670+0000 7f617d59a700 1 -- 172.21.15.89:0/4001280607 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174011d20 con 0x7f61800a1510 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.671+0000 7f6184f0a700 1 -- 172.21.15.89:0/4001280607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61800a1510 msgr2=0x7f6180138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.671+0000 7f6184f0a700 1 --2- 172.21.15.89:0/4001280607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61800a1510 0x7f6180138d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6174009870 tx=0x7f6174009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.671+0000 7f6184f0a700 1 -- 172.21.15.89:0/4001280607 shutdown_connections 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.671+0000 7f6184f0a700 1 --2- 172.21.15.89:0/4001280607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6180139250 0x7f618013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.671+0000 7f6184f0a700 1 --2- 172.21.15.89:0/4001280607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61800a1510 0x7f6180138d10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.671+0000 7f6184f0a700 1 -- 172.21.15.89:0/4001280607 >> 172.21.15.89:0/4001280607 conn(0x7f61800a6550 msgr2=0x7f61800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.671+0000 7f6184f0a700 1 -- 172.21.15.89:0/4001280607 shutdown_connections 2023-09-30T22:19:50.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.671+0000 7f6184f0a700 1 -- 172.21.15.89:0/4001280607 wait complete. 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f6184f0a700 1 Processor -- start 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f6184f0a700 1 -- start start 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f6184f0a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61800a0a30 0x7f6180122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f6184f0a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61800a1510 0x7f6180122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f6184f0a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6180139250 0x7f6180123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f6184f0a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6180125c00 con 0x7f61800a0a30 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f6184f0a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6180138180 con 0x7f61800a1510 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f6184f0a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61801389a0 con 0x7f6180139250 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f617ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6180139250 0x7f6180123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f617ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6180139250 0x7f6180123380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44840/0 (socket says 172.21.15.89:44840) 2023-09-30T22:19:50.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f617ed9d700 1 -- 172.21.15.89:0/4103612100 learned_addr learned my addr 172.21.15.89:0/4103612100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:50.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f617dd9b700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61800a1510 0x7f6180122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.673+0000 7f617e59c700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61800a0a30 0x7f6180122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f617ed9d700 1 -- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61800a1510 msgr2=0x7f6180122e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f617ed9d700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61800a1510 0x7f6180122e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f617ed9d700 1 -- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61800a0a30 msgr2=0x7f6180122900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f617ed9d700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61800a0a30 0x7f6180122900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f617ed9d700 1 -- 172.21.15.89:0/4103612100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178009a70 con 0x7f6180139250 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f617dd9b700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61800a1510 0x7f6180122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f617ed9d700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6180139250 0x7f6180123380 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6178009f00 tx=0x7f617800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f616effd700 1 -- 172.21.15.89:0/4103612100 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178009d70 con 0x7f6180139250 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f616effd700 1 -- 172.21.15.89:0/4103612100 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617800f040 con 0x7f6180139250 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f616effd700 1 -- 172.21.15.89:0/4103612100 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617800cd70 con 0x7f6180139250 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61740094a0 con 0x7f6180139250 2023-09-30T22:19:50.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.674+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6180138760 con 0x7f6180139250 2023-09-30T22:19:50.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.675+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61800a1d00 con 0x7f6180139250 2023-09-30T22:19:50.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.676+0000 7f616effd700 1 -- 172.21.15.89:0/4103612100 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6178003a20 con 0x7f6180139250 2023-09-30T22:19:50.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.676+0000 7f616effd700 1 -- 172.21.15.89:0/4103612100 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6178040730 con 0x7f6180139250 2023-09-30T22:19:50.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.680+0000 7f616effd700 1 -- 172.21.15.89:0/4103612100 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f617801c7c0 con 0x7f6180139250 2023-09-30T22:19:50.736 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.757+0000 7efe090b2700 1 Processor -- start 2023-09-30T22:19:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.757+0000 7efe090b2700 1 -- start start 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe090b2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe0409d6f0 0x7efe0409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe090b2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409e440 0x7efe04134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe090b2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0409b120 con 0x7efe0409e000 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe090b2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0406b460 con 0x7efe0409d6f0 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe090b2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe0409a7e0 con 0x7efe0409e440 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe02d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe0409d6f0 0x7efe0409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe02d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe0409d6f0 0x7efe0409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53084/0 (socket says 172.21.15.89:53084) 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe02d9d700 1 -- 172.21.15.89:0/167418266 learned_addr learned my addr 172.21.15.89:0/167418266 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe0259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efe0409e000 0x7efe041339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53480/0 (socket says 172.21.15.89:53480) 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe0359e700 1 --2- 172.21.15.89:0/167418266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409e440 0x7efe04134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe0359e700 1 -- 172.21.15.89:0/167418266 >> v2:172.21.15.89:3300/0 conn(0x7efe0409d6f0 msgr2=0x7efe0409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe0359e700 1 --2- 172.21.15.89:0/167418266 >> v2:172.21.15.89:3300/0 conn(0x7efe0409d6f0 0x7efe0409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe0359e700 1 -- 172.21.15.89:0/167418266 >> v1:172.21.15.89:6789/0 conn(0x7efe0409e000 legacy=0x7efe041339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe0359e700 1 -- 172.21.15.89:0/167418266 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0409a500 con 0x7efe0409e440 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe02d9d700 1 --2- 172.21.15.89:0/167418266 >> v2:172.21.15.89:3300/0 conn(0x7efe0409d6f0 0x7efe0409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.758+0000 7efe0359e700 1 --2- 172.21.15.89:0/167418266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409e440 0x7efe04134100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009dc0 tx=0x7efdfc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a63113a32760242 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe01d9b700 1 -- 172.21.15.89:0/167418266 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00e070 con 0x7efe0409e440 2023-09-30T22:19:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe01d9b700 1 -- 172.21.15.89:0/167418266 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc00bae0 con 0x7efe0409e440 2023-09-30T22:19:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe01d9b700 1 -- 172.21.15.89:0/167418266 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00be90 con 0x7efe0409e440 2023-09-30T22:19:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe090b2700 1 -- 172.21.15.89:0/167418266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409e440 msgr2=0x7efe04134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe090b2700 1 --2- 172.21.15.89:0/167418266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409e440 0x7efe04134100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009dc0 tx=0x7efdfc00b330 comp rx=0 tx=0).stop 2023-09-30T22:19:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe090b2700 1 -- 172.21.15.89:0/167418266 shutdown_connections 2023-09-30T22:19:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe090b2700 1 --2- 172.21.15.89:0/167418266 >> v2:172.21.15.89:3300/0 conn(0x7efe0409d6f0 0x7efe0409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe090b2700 1 --2- 172.21.15.89:0/167418266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409e440 0x7efe04134100 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe090b2700 1 -- 172.21.15.89:0/167418266 >> 172.21.15.89:0/167418266 conn(0x7efe04066f50 msgr2=0x7efe0409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe090b2700 1 -- 172.21.15.89:0/167418266 shutdown_connections 2023-09-30T22:19:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.759+0000 7efe090b2700 1 -- 172.21.15.89:0/167418266 wait complete. 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.760+0000 7efe090b2700 1 Processor -- start 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe090b2700 1 -- start start 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe090b2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409d6f0 0x7efe041a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe090b2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe0409e000 0x7efe041a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe090b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0409e440 0x7efe041a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe090b2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041aa360 con 0x7efe0409e440 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe090b2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe041a99a0 con 0x7efe0409e000 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe090b2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe041a8ce0 con 0x7efe0409d6f0 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe02d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409d6f0 0x7efe041a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe0259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe0409e000 0x7efe041a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe02d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409d6f0 0x7efe041a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44860/0 (socket says 172.21.15.89:44860) 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe02d9d700 1 -- 172.21.15.89:0/94998592 learned_addr learned my addr 172.21.15.89:0/94998592 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.761+0000 7efe0359e700 1 --2- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0409e440 0x7efe041a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efe02d9d700 1 -- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe0409e000 msgr2=0x7efe041a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efe02d9d700 1 --2- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe0409e000 0x7efe041a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efe02d9d700 1 -- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0409e440 msgr2=0x7efe041a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efe02d9d700 1 --2- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0409e440 0x7efe041a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efe02d9d700 1 -- 172.21.15.89:0/94998592 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009550 con 0x7efe0409d6f0 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efe02d9d700 1 --2- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409d6f0 0x7efe041a5ed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efdf400ec10 tx=0x7efdf400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efdeb7fe700 1 -- 172.21.15.89:0/94998592 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400c850 con 0x7efe0409d6f0 2023-09-30T22:19:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efdeb7fe700 1 -- 172.21.15.89:0/94998592 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400c9b0 con 0x7efe0409d6f0 2023-09-30T22:19:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efdeb7fe700 1 -- 172.21.15.89:0/94998592 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400cd20 con 0x7efe0409d6f0 2023-09-30T22:19:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc009a70 con 0x7efe0409d6f0 2023-09-30T22:19:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.762+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe040a2050 con 0x7efe0409d6f0 2023-09-30T22:19:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.763+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe041a9540 con 0x7efe0409d6f0 2023-09-30T22:19:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.763+0000 7efdeb7fe700 1 -- 172.21.15.89:0/94998592 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efdf4013470 con 0x7efe0409d6f0 2023-09-30T22:19:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.764+0000 7efdeb7fe700 1 -- 172.21.15.89:0/94998592 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efdf403fc00 con 0x7efe0409d6f0 2023-09-30T22:19:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.768+0000 7efdeb7fe700 1 -- 172.21.15.89:0/94998592 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efdf4013720 con 0x7efe0409d6f0 2023-09-30T22:19:50.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.863+0000 7f47bd412700 1 Processor -- start 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.863+0000 7f47bd412700 1 -- start start 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47bd412700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47b8133420 0x7f47b81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47bd412700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b8133e30 0x7f47b812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47bd412700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b806b990 con 0x7f47b8134260 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47bd412700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b80a0590 con 0x7f47b8133420 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47bd412700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b809fb80 con 0x7f47b8133e30 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47b77fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f47b8134260 0x7f47b81304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53492/0 (socket says 172.21.15.89:53492) 2023-09-30T22:19:50.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47b6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47b8133420 0x7f47b81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47b77fe700 1 -- 172.21.15.89:0/2934164750 learned_addr learned my addr 172.21.15.89:0/2934164750 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:50.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.864+0000 7f47b67fc700 1 --2- 172.21.15.89:0/2934164750 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b8133e30 0x7f47b812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b5ffb700 1 -- 172.21.15.89:0/2934164750 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059866370 0 0) 0x7f47b806b990 con 0x7f47b8134260 2023-09-30T22:19:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b5ffb700 1 -- 172.21.15.89:0/2934164750 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a00035b0 con 0x7f47b8134260 2023-09-30T22:19:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b67fc700 1 -- 172.21.15.89:0/2934164750 >> v2:172.21.15.89:3300/0 conn(0x7f47b8133420 msgr2=0x7f47b81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b67fc700 1 --2- 172.21.15.89:0/2934164750 >> v2:172.21.15.89:3300/0 conn(0x7f47b8133420 0x7f47b81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b67fc700 1 -- 172.21.15.89:0/2934164750 >> v1:172.21.15.89:6789/0 conn(0x7f47b8134260 legacy=0x7f47b81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b67fc700 1 -- 172.21.15.89:0/2934164750 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b80a0030 con 0x7f47b8133e30 2023-09-30T22:19:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b67fc700 1 --2- 172.21.15.89:0/2934164750 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b8133e30 0x7f47b812ff90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009840 tx=0x7f47ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3141424b21ca4080 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b5ffb700 1 -- 172.21.15.89:0/2934164750 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac011830 con 0x7f47b8133e30 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b5ffb700 1 -- 172.21.15.89:0/2934164750 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac011990 con 0x7f47b8133e30 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.865+0000 7f47b5ffb700 1 -- 172.21.15.89:0/2934164750 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac011d00 con 0x7f47b8133e30 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.866+0000 7f47bd412700 1 -- 172.21.15.89:0/2934164750 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b8133e30 msgr2=0x7f47b812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.866+0000 7f47bd412700 1 --2- 172.21.15.89:0/2934164750 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b8133e30 0x7f47b812ff90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009840 tx=0x7f47ac009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.866+0000 7f47bd412700 1 -- 172.21.15.89:0/2934164750 shutdown_connections 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.866+0000 7f47bd412700 1 --2- 172.21.15.89:0/2934164750 >> v2:172.21.15.89:3300/0 conn(0x7f47b8133420 0x7f47b81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.866+0000 7f47bd412700 1 --2- 172.21.15.89:0/2934164750 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b8133e30 0x7f47b812ff90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.866+0000 7f47bd412700 1 -- 172.21.15.89:0/2934164750 >> 172.21.15.89:0/2934164750 conn(0x7f47b80a4210 msgr2=0x7f47b80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.866+0000 7f47bd412700 1 -- 172.21.15.89:0/2934164750 shutdown_connections 2023-09-30T22:19:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.866+0000 7f47bd412700 1 -- 172.21.15.89:0/2934164750 wait complete. 2023-09-30T22:19:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.867+0000 7f47bd412700 1 Processor -- start 2023-09-30T22:19:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.867+0000 7f47bd412700 1 -- start start 2023-09-30T22:19:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.867+0000 7f47bd412700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47b8133420 0x7f47b809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.867+0000 7f47bd412700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8134260 0x7f47b809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.867+0000 7f47bd412700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b809cf00 0x7f47b81c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.867+0000 7f47bd412700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b812dc60 con 0x7f47b8134260 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.867+0000 7f47bd412700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8131650 con 0x7f47b809cf00 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.867+0000 7f47bd412700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47b809ded0 con 0x7f47b8133420 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47b8133420 0x7f47b809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47b8133420 0x7f47b809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44876/0 (socket says 172.21.15.89:44876) 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 -- 172.21.15.89:0/2164280977 learned_addr learned my addr 172.21.15.89:0/2164280977 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b77fe700 1 --2- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b809cf00 0x7f47b81c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 -- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b809cf00 msgr2=0x7f47b81c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 --2- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b809cf00 0x7f47b81c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 -- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8134260 msgr2=0x7f47b809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 --2- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8134260 0x7f47b809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 -- 172.21.15.89:0/2164280977 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0094a0 con 0x7f47b8133420 2023-09-30T22:19:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47b6ffd700 1 --2- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47b8133420 0x7f47b809c480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e9b0 tx=0x7f47a800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f479f7fe700 1 -- 172.21.15.89:0/2164280977 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8009710 con 0x7f47b8133420 2023-09-30T22:19:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b811fc60 con 0x7f47b8133420 2023-09-30T22:19:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f479f7fe700 1 -- 172.21.15.89:0/2164280977 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a800c7f0 con 0x7f47b8133420 2023-09-30T22:19:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f479f7fe700 1 -- 172.21.15.89:0/2164280977 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800cb60 con 0x7f47b8133420 2023-09-30T22:19:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.868+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b8120990 con 0x7f47b8133420 2023-09-30T22:19:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.870+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8074ea0 con 0x7f47b8133420 2023-09-30T22:19:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.870+0000 7f479f7fe700 1 -- 172.21.15.89:0/2164280977 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f47a800ccc0 con 0x7f47b8133420 2023-09-30T22:19:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.870+0000 7f479f7fe700 1 -- 172.21.15.89:0/2164280977 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f47a803ff60 con 0x7f47b8133420 2023-09-30T22:19:50.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.878+0000 7f479f7fe700 1 -- 172.21.15.89:0/2164280977 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f47a806b750 con 0x7f47b8133420 2023-09-30T22:19:50.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.885+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6180074ea0 con 0x7f6180139250 2023-09-30T22:19:50.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.885+0000 7f616effd700 1 -- 172.21.15.89:0/4103612100 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f617802b090 con 0x7f6180139250 2023-09-30T22:19:50.886 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.888+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6180139250 msgr2=0x7f6180123380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.888+0000 7f6184f0a700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6180139250 0x7f6180123380 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6178009f00 tx=0x7f617800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.889+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 shutdown_connections 2023-09-30T22:19:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.889+0000 7f6184f0a700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6180139250 0x7f6180123380 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.889+0000 7f6184f0a700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61800a0a30 0x7f6180122900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.889+0000 7f6184f0a700 1 --2- 172.21.15.89:0/4103612100 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61800a1510 0x7f6180122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.889+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 >> 172.21.15.89:0/4103612100 conn(0x7f61800a6550 msgr2=0x7f618009ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.889+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 shutdown_connections 2023-09-30T22:19:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.889+0000 7f6184f0a700 1 -- 172.21.15.89:0/4103612100 wait complete. 2023-09-30T22:19:50.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.917+0000 7f2327b45700 1 Processor -- start 2023-09-30T22:19:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f2327b45700 1 -- start start 2023-09-30T22:19:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f2327b45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009d6f0 0x7f232009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f2327b45700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009e410 0x7f2320134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f2327b45700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232009b120 con 0x7f232009e000 2023-09-30T22:19:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f2327b45700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232006b460 con 0x7f232009e410 2023-09-30T22:19:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f2327b45700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f232009a7e0 con 0x7f232009d6f0 2023-09-30T22:19:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f23258e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009d6f0 0x7f232009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f23250e0700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f232009e000 0x7f23201339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53496/0 (socket says 172.21.15.89:53496) 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f23250e0700 1 -- 172.21.15.89:0/2733431798 learned_addr learned my addr 172.21.15.89:0/2733431798 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.918+0000 7f23260e2700 1 --2- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009e410 0x7f2320134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23248df700 1 -- 172.21.15.89:0/2733431798 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960264826 0 0) 0x7f232009b120 con 0x7f232009e000 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23248df700 1 -- 172.21.15.89:0/2733431798 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f230c0035b0 con 0x7f232009e000 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23258e1700 1 -- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009e410 msgr2=0x7f2320134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23258e1700 1 --2- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009e410 0x7f2320134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23258e1700 1 -- 172.21.15.89:0/2733431798 >> v1:172.21.15.89:6789/0 conn(0x7f232009e000 legacy=0x7f23201339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23258e1700 1 -- 172.21.15.89:0/2733431798 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232009a500 con 0x7f232009d6f0 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23260e2700 1 --2- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009e410 0x7f2320134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23258e1700 1 --2- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009d6f0 0x7f232009dac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f231c0098a0 tx=0x7f231c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b7d284d9e00005 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23248df700 1 -- 172.21.15.89:0/2733431798 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c004230 con 0x7f232009d6f0 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23248df700 1 -- 172.21.15.89:0/2733431798 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c004390 con 0x7f232009d6f0 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.919+0000 7f23248df700 1 -- 172.21.15.89:0/2733431798 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c0036a0 con 0x7f232009d6f0 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.920+0000 7f2327b45700 1 -- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009d6f0 msgr2=0x7f232009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.920+0000 7f2327b45700 1 --2- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009d6f0 0x7f232009dac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f231c0098a0 tx=0x7f231c009bb0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.920+0000 7f2327b45700 1 -- 172.21.15.89:0/2733431798 shutdown_connections 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.920+0000 7f2327b45700 1 --2- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009e410 0x7f2320134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.920+0000 7f2327b45700 1 --2- 172.21.15.89:0/2733431798 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009d6f0 0x7f232009dac0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.920+0000 7f2327b45700 1 -- 172.21.15.89:0/2733431798 >> 172.21.15.89:0/2733431798 conn(0x7f2320066f50 msgr2=0x7f232009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.920+0000 7f2327b45700 1 -- 172.21.15.89:0/2733431798 shutdown_connections 2023-09-30T22:19:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.920+0000 7f2327b45700 1 -- 172.21.15.89:0/2733431798 wait complete. 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f2327b45700 1 Processor -- start 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f2327b45700 1 -- start start 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f2327b45700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009d6f0 0x7f2320122700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f2327b45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009e000 0x7f2320122c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f2327b45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232009e410 0x7f2320123180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f2327b45700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2320125e10 con 0x7f232009e410 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f2327b45700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23200a19d0 con 0x7f232009d6f0 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f2327b45700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23201242f0 con 0x7f232009e000 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f23250e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009e000 0x7f2320122c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f23250e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009e000 0x7f2320122c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44894/0 (socket says 172.21.15.89:44894) 2023-09-30T22:19:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f23250e0700 1 -- 172.21.15.89:0/1819916102 learned_addr learned my addr 172.21.15.89:0/1819916102 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f23260e2700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232009e410 0x7f2320123180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.921+0000 7f23258e1700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009d6f0 0x7f2320122700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23260e2700 1 -- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009e000 msgr2=0x7f2320122c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23260e2700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009e000 0x7f2320122c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23260e2700 1 -- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009d6f0 msgr2=0x7f2320122700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23260e2700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009d6f0 0x7f2320122700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23260e2700 1 -- 172.21.15.89:0/1819916102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2318009a70 con 0x7f232009e410 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23258e1700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009d6f0 0x7f2320122700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23250e0700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009e000 0x7f2320122c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23260e2700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232009e410 0x7f2320123180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f231800efc0 tx=0x7f231800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23167fc700 1 -- 172.21.15.89:0/1819916102 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800ca90 con 0x7f232009e410 2023-09-30T22:19:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23167fc700 1 -- 172.21.15.89:0/1819916102 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231800f040 con 0x7f232009e410 2023-09-30T22:19:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c0094a0 con 0x7f232009e410 2023-09-30T22:19:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f23167fc700 1 -- 172.21.15.89:0/1819916102 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800ce30 con 0x7f232009e410 2023-09-30T22:19:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.922+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23201330f0 con 0x7f232009e410 2023-09-30T22:19:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.923+0000 7f23167fc700 1 -- 172.21.15.89:0/1819916102 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2318014480 con 0x7f232009e410 2023-09-30T22:19:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.923+0000 7f23167fc700 1 -- 172.21.15.89:0/1819916102 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f231803fc60 con 0x7f232009e410 2023-09-30T22:19:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.924+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2320125f50 con 0x7f232009e410 2023-09-30T22:19:50.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.928+0000 7f23167fc700 1 -- 172.21.15.89:0/1819916102 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2320125f50 con 0x7f232009e410 2023-09-30T22:19:50.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.970+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efe0409f420 con 0x7efe0409d6f0 2023-09-30T22:19:50.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.970+0000 7efdeb7fe700 1 -- 172.21.15.89:0/94998592 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efdf4021030 con 0x7efe0409d6f0 2023-09-30T22:19:50.971 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409d6f0 msgr2=0x7efe041a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 --2- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409d6f0 0x7efe041a5ed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efdf400ec10 tx=0x7efdf400ef20 comp rx=0 tx=0).stop 2023-09-30T22:19:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 shutdown_connections 2023-09-30T22:19:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 --2- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe0409e440 0x7efe041a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 --2- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe0409d6f0 0x7efe041a5ed0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 --2- 172.21.15.89:0/94998592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe0409e000 0x7efe041a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 >> 172.21.15.89:0/94998592 conn(0x7efe04066f50 msgr2=0x7efe041baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 shutdown_connections 2023-09-30T22:19:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:50.972+0000 7efe090b2700 1 -- 172.21.15.89:0/94998592 wait complete. 2023-09-30T22:19:50.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.078+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f47b8120160 con 0x7f47b8133420 2023-09-30T22:19:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.078+0000 7f479f7fe700 1 -- 172.21.15.89:0/2164280977 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47a8021020 con 0x7f47b8133420 2023-09-30T22:19:51.079 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:51.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47b8133420 msgr2=0x7f47b809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 --2- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47b8133420 0x7f47b809c480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e9b0 tx=0x7f47a800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:19:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 shutdown_connections 2023-09-30T22:19:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 --2- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47b809cf00 0x7f47b81c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 --2- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47b8133420 0x7f47b809c480 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 --2- 172.21.15.89:0/2164280977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8134260 0x7f47b809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 >> 172.21.15.89:0/2164280977 conn(0x7f47b80a4210 msgr2=0x7f47b80a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 shutdown_connections 2023-09-30T22:19:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.081+0000 7f47bd412700 1 -- 172.21.15.89:0/2164280977 wait complete. 2023-09-30T22:19:51.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:51.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.120+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2320125f50 con 0x7f232009e410 2023-09-30T22:19:51.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.121+0000 7f23167fc700 1 -- 172.21.15.89:0/1819916102 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2320125f50 con 0x7f232009e410 2023-09-30T22:19:51.121 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:51.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.122+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232009e410 msgr2=0x7f2320123180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.122+0000 7f2327b45700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232009e410 0x7f2320123180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f231800efc0 tx=0x7f231800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.123+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 shutdown_connections 2023-09-30T22:19:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.123+0000 7f2327b45700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232009e410 0x7f2320123180 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.123+0000 7f2327b45700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f232009d6f0 0x7f2320122700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.123+0000 7f2327b45700 1 --2- 172.21.15.89:0/1819916102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f232009e000 0x7f2320122c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.123+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 >> 172.21.15.89:0/1819916102 conn(0x7f2320066f50 msgr2=0x7f2320133f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.123+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 shutdown_connections 2023-09-30T22:19:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:51.123+0000 7f2327b45700 1 -- 172.21.15.89:0/1819916102 wait complete. 2023-09-30T22:19:51.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:51.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:51.986 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.029+0000 7f4d23033700 1 Processor -- start 2023-09-30T22:19:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.029+0000 7f4d23033700 1 -- start start 2023-09-30T22:19:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d23033700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a0a30 0x7f4d1c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d23033700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d1c134170 0x7f4d1c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d23033700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c09edd0 con 0x7f4d1c0a1340 2023-09-30T22:19:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d23033700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c09e1f0 con 0x7f4d1c134170 2023-09-30T22:19:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d23033700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c09dc50 con 0x7f4d1c0a0a30 2023-09-30T22:19:52.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d20dcf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a0a30 0x7f4d1c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d215d0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d1c134170 0x7f4d1c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d20dcf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a0a30 0x7f4d1c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44904/0 (socket says 172.21.15.89:44904) 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d215d0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d1c134170 0x7f4d1c13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53116/0 (socket says 172.21.15.89:53116) 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d20dcf700 1 -- 172.21.15.89:0/3829978028 learned_addr learned my addr 172.21.15.89:0/3829978028 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d215d0700 1 -- 172.21.15.89:0/3829978028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a0a30 msgr2=0x7f4d1c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d215d0700 1 --2- 172.21.15.89:0/3829978028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a0a30 0x7f4d1c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d215d0700 1 -- 172.21.15.89:0/3829978028 >> v1:172.21.15.89:6789/0 conn(0x7f4d1c0a1340 legacy=0x7f4d1c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d215d0700 1 -- 172.21.15.89:0/3829978028 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0a1fe0 con 0x7f4d1c134170 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d20dcf700 1 --2- 172.21.15.89:0/3829978028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a0a30 0x7f4d1c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d215d0700 1 --2- 172.21.15.89:0/3829978028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 0x7f4d1c13bd30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d90 tx=0x7f4d1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa5236006d9bf3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d1b7fe700 1 -- 172.21.15.89:0/3829978028 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400e070 con 0x7f4d1c134170 2023-09-30T22:19:52.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d1b7fe700 1 -- 172.21.15.89:0/3829978028 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400bb20 con 0x7f4d1c134170 2023-09-30T22:19:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.030+0000 7f4d1b7fe700 1 -- 172.21.15.89:0/3829978028 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14012430 con 0x7f4d1c134170 2023-09-30T22:19:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.031+0000 7f4d23033700 1 -- 172.21.15.89:0/3829978028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 msgr2=0x7f4d1c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.031+0000 7f4d23033700 1 --2- 172.21.15.89:0/3829978028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 0x7f4d1c13bd30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d90 tx=0x7f4d1400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.031+0000 7f4d23033700 1 -- 172.21.15.89:0/3829978028 shutdown_connections 2023-09-30T22:19:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.031+0000 7f4d23033700 1 --2- 172.21.15.89:0/3829978028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 0x7f4d1c13bd30 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.031+0000 7f4d23033700 1 --2- 172.21.15.89:0/3829978028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a0a30 0x7f4d1c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.031+0000 7f4d23033700 1 -- 172.21.15.89:0/3829978028 >> 172.21.15.89:0/3829978028 conn(0x7f4d1c0a6550 msgr2=0x7f4d1c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:52.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.031+0000 7f4d23033700 1 -- 172.21.15.89:0/3829978028 shutdown_connections 2023-09-30T22:19:52.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.031+0000 7f4d23033700 1 -- 172.21.15.89:0/3829978028 wait complete. 2023-09-30T22:19:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.032+0000 7f4d23033700 1 Processor -- start 2023-09-30T22:19:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.032+0000 7f4d23033700 1 -- start start 2023-09-30T22:19:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d23033700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c0a0a30 0x7f4d1c11e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d23033700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a1340 0x7f4d1c11ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d23033700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 0x7f4d1c11f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d23033700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c122190 con 0x7f4d1c134170 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d23033700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c121bf0 con 0x7f4d1c0a0a30 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d23033700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c120280 con 0x7f4d1c0a1340 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a1340 0x7f4d1c11ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a1340 0x7f4d1c11ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44918/0 (socket says 172.21.15.89:44918) 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 -- 172.21.15.89:0/3773456988 learned_addr learned my addr 172.21.15.89:0/3773456988 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d20dcf700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c0a0a30 0x7f4d1c11e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d215d0700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 0x7f4d1c11f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 -- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c0a0a30 msgr2=0x7f4d1c11e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c0a0a30 0x7f4d1c11e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 -- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 msgr2=0x7f4d1c11f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 0x7f4d1c11f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 -- 172.21.15.89:0/3773456988 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10009550 con 0x7f4d1c0a1340 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d20dcf700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c0a0a30 0x7f4d1c11e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d215d0700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 0x7f4d1c11f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d1bfff700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a1340 0x7f4d1c11ec40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800c4e0 tx=0x7f4d0800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d197fa700 1 -- 172.21.15.89:0/3773456988 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d080052d0 con 0x7f4d1c0a1340 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.033+0000 7f4d197fa700 1 -- 172.21.15.89:0/3773456988 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08005430 con 0x7f4d1c0a1340 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.034+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d14009a70 con 0x7f4d1c0a1340 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.034+0000 7f4d197fa700 1 -- 172.21.15.89:0/3773456988 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d080038c0 con 0x7f4d1c0a1340 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.034+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c120a30 con 0x7f4d1c0a1340 2023-09-30T22:19:52.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.034+0000 7f4d197fa700 1 -- 172.21.15.89:0/3773456988 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d08003a20 con 0x7f4d1c0a1340 2023-09-30T22:19:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.034+0000 7f4d197fa700 1 -- 172.21.15.89:0/3773456988 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4d0803ea60 con 0x7f4d1c0a1340 2023-09-30T22:19:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.034+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c074ea0 con 0x7f4d1c0a1340 2023-09-30T22:19:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.039+0000 7f4d197fa700 1 -- 172.21.15.89:0/3773456988 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d08017080 con 0x7f4d1c0a1340 2023-09-30T22:19:52.094 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.112+0000 7f06d161f700 1 Processor -- start 2023-09-30T22:19:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.112+0000 7f06d161f700 1 -- start start 2023-09-30T22:19:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06d161f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06d161f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e000 0x7f06cc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06d161f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc09b120 con 0x7f06cc09e410 2023-09-30T22:19:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06d161f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc06b460 con 0x7f06cc09d6f0 2023-09-30T22:19:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06d161f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc09a7e0 con 0x7f06cc09e000 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06ca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e000 0x7f06cc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06ca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e000 0x7f06cc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44926/0 (socket says 172.21.15.89:44926) 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06ca7fc700 1 -- 172.21.15.89:0/2112747567 learned_addr learned my addr 172.21.15.89:0/2112747567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06caffd700 1 --2- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06caffd700 1 -- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e000 msgr2=0x7f06cc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06caffd700 1 --2- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e000 0x7f06cc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06caffd700 1 -- 172.21.15.89:0/2112747567 >> v1:172.21.15.89:6789/0 conn(0x7f06cc09e410 legacy=0x7f06cc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06caffd700 1 -- 172.21.15.89:0/2112747567 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc09a500 con 0x7f06cc09d6f0 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06ca7fc700 1 --2- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e000 0x7f06cc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.113+0000 7f06caffd700 1 --2- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009870 tx=0x7f06bc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2adb62a4a71d061 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06c9ffb700 1 -- 172.21.15.89:0/2112747567 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc004230 con 0x7f06cc09d6f0 2023-09-30T22:19:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06c9ffb700 1 -- 172.21.15.89:0/2112747567 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc004390 con 0x7f06cc09d6f0 2023-09-30T22:19:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06c9ffb700 1 -- 172.21.15.89:0/2112747567 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0036a0 con 0x7f06cc09d6f0 2023-09-30T22:19:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06d161f700 1 -- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 msgr2=0x7f06cc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06d161f700 1 --2- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009870 tx=0x7f06bc009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06d161f700 1 -- 172.21.15.89:0/2112747567 shutdown_connections 2023-09-30T22:19:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06d161f700 1 --2- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc09dac0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06d161f700 1 --2- 172.21.15.89:0/2112747567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e000 0x7f06cc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06d161f700 1 -- 172.21.15.89:0/2112747567 >> 172.21.15.89:0/2112747567 conn(0x7f06cc066f50 msgr2=0x7f06cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06d161f700 1 -- 172.21.15.89:0/2112747567 shutdown_connections 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.114+0000 7f06d161f700 1 -- 172.21.15.89:0/2112747567 wait complete. 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.115+0000 7f06d161f700 1 Processor -- start 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.115+0000 7f06d161f700 1 -- start start 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06d161f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc1225c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06d161f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc09e000 0x7f06cc122b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06d161f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e410 0x7f06cc123040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06d161f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc133580 con 0x7f06cc09e000 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06d161f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc1242e0 con 0x7f06cc09d6f0 2023-09-30T22:19:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06d161f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc09c710 con 0x7f06cc09e410 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06caffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc1225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06caffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc1225c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:41944/0 (socket says 172.21.15.89:41944) 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06caffd700 1 -- 172.21.15.89:0/1697736781 learned_addr learned my addr 172.21.15.89:0/1697736781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06ca7fc700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc09e000 0x7f06cc122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.116+0000 7f06cb7fe700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e410 0x7f06cc123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06caffd700 1 -- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e410 msgr2=0x7f06cc123040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06caffd700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e410 0x7f06cc123040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06caffd700 1 -- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc09e000 msgr2=0x7f06cc122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06caffd700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc09e000 0x7f06cc122b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06caffd700 1 -- 172.21.15.89:0/1697736781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0009550 con 0x7f06cc09d6f0 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06ca7fc700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc09e000 0x7f06cc122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06caffd700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc1225c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009ed0 tx=0x7f06bc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06b37fe700 1 -- 172.21.15.89:0/1697736781 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc003da0 con 0x7f06cc09d6f0 2023-09-30T22:19:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06b37fe700 1 -- 172.21.15.89:0/1697736781 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc01e070 con 0x7f06cc09d6f0 2023-09-30T22:19:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc0094a0 con 0x7f06cc09d6f0 2023-09-30T22:19:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06b37fe700 1 -- 172.21.15.89:0/1697736781 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc011870 con 0x7f06cc09d6f0 2023-09-30T22:19:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.117+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc125b90 con 0x7f06cc09d6f0 2023-09-30T22:19:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.118+0000 7f06b37fe700 1 -- 172.21.15.89:0/1697736781 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f06bc0119d0 con 0x7f06cc09d6f0 2023-09-30T22:19:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.118+0000 7f06b37fe700 1 -- 172.21.15.89:0/1697736781 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f06bc040780 con 0x7f06cc09d6f0 2023-09-30T22:19:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.118+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc0729a0 con 0x7f06cc09d6f0 2023-09-30T22:19:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.124+0000 7f06b37fe700 1 -- 172.21.15.89:0/1697736781 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f06bc010970 con 0x7f06cc09d6f0 2023-09-30T22:19:52.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.223+0000 7f6125efe700 1 Processor -- start 2023-09-30T22:19:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.223+0000 7f6125efe700 1 -- start start 2023-09-30T22:19:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.223+0000 7f6125efe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6120131630 0x7f6120131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.223+0000 7f6125efe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612006eba0 0x7f6120139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.223+0000 7f6125efe700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6120071a00 con 0x7f6120131f40 2023-09-30T22:19:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.223+0000 7f6125efe700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6120131430 con 0x7f6120131630 2023-09-30T22:19:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.223+0000 7f6125efe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612009a510 con 0x7f612006eba0 2023-09-30T22:19:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6120131630 0x7f6120131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6120131630 0x7f6120131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53142/0 (socket says 172.21.15.89:53142) 2023-09-30T22:19:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611f7fe700 1 -- 172.21.15.89:0/417742612 learned_addr learned my addr 172.21.15.89:0/417742612 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611ffff700 1 --2- 172.21.15.89:0/417742612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612006eba0 0x7f6120139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611f7fe700 1 -- 172.21.15.89:0/417742612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612006eba0 msgr2=0x7f6120139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611f7fe700 1 --2- 172.21.15.89:0/417742612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612006eba0 0x7f6120139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611f7fe700 1 -- 172.21.15.89:0/417742612 >> v1:172.21.15.89:6789/0 conn(0x7f6120131f40 legacy=0x7f612006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611f7fe700 1 -- 172.21.15.89:0/417742612 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61200721c0 con 0x7f6120131630 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611f7fe700 1 --2- 172.21.15.89:0/417742612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120131630 0x7f6120131a00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f61100098f0 tx=0x7f6110009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef69a7de45752988 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611e7fc700 1 -- 172.21.15.89:0/417742612 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110011830 con 0x7f6120131630 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611e7fc700 1 -- 172.21.15.89:0/417742612 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6110011990 con 0x7f6120131630 2023-09-30T22:19:52.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.224+0000 7f611e7fc700 1 -- 172.21.15.89:0/417742612 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110011d20 con 0x7f6120131630 2023-09-30T22:19:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.225+0000 7f6125efe700 1 -- 172.21.15.89:0/417742612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120131630 msgr2=0x7f6120131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.225+0000 7f6125efe700 1 --2- 172.21.15.89:0/417742612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120131630 0x7f6120131a00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f61100098f0 tx=0x7f6110009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.225+0000 7f6125efe700 1 -- 172.21.15.89:0/417742612 shutdown_connections 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.225+0000 7f6125efe700 1 --2- 172.21.15.89:0/417742612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612006eba0 0x7f6120139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.225+0000 7f6125efe700 1 --2- 172.21.15.89:0/417742612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120131630 0x7f6120131a00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.225+0000 7f6125efe700 1 -- 172.21.15.89:0/417742612 >> 172.21.15.89:0/417742612 conn(0x7f61200a2aa0 msgr2=0x7f61200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.225+0000 7f6125efe700 1 -- 172.21.15.89:0/417742612 shutdown_connections 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.225+0000 7f6125efe700 1 -- 172.21.15.89:0/417742612 wait complete. 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.226+0000 7f6125efe700 1 Processor -- start 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.226+0000 7f6125efe700 1 -- start start 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f6125efe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612006eba0 0x7f612006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f6125efe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6120131630 0x7f612006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f6125efe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120131f40 0x7f6120070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f6125efe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6120070ae0 con 0x7f612006eba0 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f6125efe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612012e700 con 0x7f6120131630 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f6125efe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61201210b0 con 0x7f6120131f40 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6120131630 0x7f612006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612006eba0 0x7f612006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120131f40 0x7f6120070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612006eba0 0x7f612006f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53148/0 (socket says 172.21.15.89:53148) 2023-09-30T22:19:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611f7fe700 1 -- 172.21.15.89:0/2960953757 learned_addr learned my addr 172.21.15.89:0/2960953757 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611f7fe700 1 -- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120131f40 msgr2=0x7f6120070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611f7fe700 1 --2- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120131f40 0x7f6120070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611f7fe700 1 -- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6120131630 msgr2=0x7f612006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611f7fe700 1 --2- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6120131630 0x7f612006fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611f7fe700 1 -- 172.21.15.89:0/2960953757 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6110009550 con 0x7f612006eba0 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.227+0000 7f611ffff700 1 --2- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120131f40 0x7f6120070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.228+0000 7f611f7fe700 1 --2- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612006eba0 0x7f612006f610 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6110009c40 tx=0x7f6110004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.228+0000 7f6107fff700 1 -- 172.21.15.89:0/2960953757 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110004400 con 0x7f612006eba0 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.228+0000 7f6107fff700 1 -- 172.21.15.89:0/2960953757 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611001f070 con 0x7f612006eba0 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.228+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6120121650 con 0x7f612006eba0 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.228+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6120121370 con 0x7f612006eba0 2023-09-30T22:19:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.228+0000 7f6107fff700 1 -- 172.21.15.89:0/2960953757 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61100038e0 con 0x7f612006eba0 2023-09-30T22:19:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.229+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61000052e0 con 0x7f612006eba0 2023-09-30T22:19:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.229+0000 7f6107fff700 1 -- 172.21.15.89:0/2960953757 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6110021030 con 0x7f612006eba0 2023-09-30T22:19:52.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.230+0000 7f6107fff700 1 -- 172.21.15.89:0/2960953757 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6110040ad0 con 0x7f612006eba0 2023-09-30T22:19:52.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.238+0000 7f6107fff700 1 -- 172.21.15.89:0/2960953757 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f611000b9d0 con 0x7f612006eba0 2023-09-30T22:19:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.268+0000 7fd795684700 1 Processor -- start 2023-09-30T22:19:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.268+0000 7fd795684700 1 -- start start 2023-09-30T22:19:52.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd795684700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd7900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd795684700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd7900a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd795684700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7900a2a70 con 0x7fd790133930 2023-09-30T22:19:52.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd795684700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900a01d0 con 0x7fd7900a0a30 2023-09-30T22:19:52.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd795684700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd79009e2b0 con 0x7fd79009d630 2023-09-30T22:19:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd7900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78f7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd790133930 0x7fd790133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53528/0 (socket says 172.21.15.89:53528) 2023-09-30T22:19:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78f7fe700 1 -- 172.21.15.89:0/1890629981 learned_addr learned my addr 172.21.15.89:0/1890629981 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78e7fc700 1 --2- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd7900a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78dffb700 1 -- 172.21.15.89:0/1890629981 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024653210 0 0) 0x7fd7900a2a70 con 0x7fd790133930 2023-09-30T22:19:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78dffb700 1 -- 172.21.15.89:0/1890629981 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7780035b0 con 0x7fd790133930 2023-09-30T22:19:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78e7fc700 1 -- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 msgr2=0x7fd7900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78e7fc700 1 --2- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd7900a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78e7fc700 1 -- 172.21.15.89:0/1890629981 >> v1:172.21.15.89:6789/0 conn(0x7fd790133930 legacy=0x7fd790133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78e7fc700 1 -- 172.21.15.89:0/1890629981 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79009dc50 con 0x7fd79009d630 2023-09-30T22:19:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.269+0000 7fd78effd700 1 --2- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd7900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.270+0000 7fd78e7fc700 1 --2- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd7900a1340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd784009870 tx=0x7fd784009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71efd476a9429e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.270+0000 7fd78dffb700 1 -- 172.21.15.89:0/1890629981 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784011830 con 0x7fd79009d630 2023-09-30T22:19:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.270+0000 7fd78dffb700 1 -- 172.21.15.89:0/1890629981 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd784011990 con 0x7fd79009d630 2023-09-30T22:19:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.270+0000 7fd78dffb700 1 -- 172.21.15.89:0/1890629981 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784011d20 con 0x7fd79009d630 2023-09-30T22:19:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.271+0000 7fd795684700 1 -- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 msgr2=0x7fd7900a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.271+0000 7fd795684700 1 --2- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd7900a1340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd784009870 tx=0x7fd784009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.271+0000 7fd795684700 1 -- 172.21.15.89:0/1890629981 shutdown_connections 2023-09-30T22:19:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.271+0000 7fd795684700 1 --2- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd7900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.271+0000 7fd795684700 1 --2- 172.21.15.89:0/1890629981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd7900a1340 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.271+0000 7fd795684700 1 -- 172.21.15.89:0/1890629981 >> 172.21.15.89:0/1890629981 conn(0x7fd7900a3050 msgr2=0x7fd7900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.271+0000 7fd795684700 1 -- 172.21.15.89:0/1890629981 shutdown_connections 2023-09-30T22:19:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.271+0000 7fd795684700 1 -- 172.21.15.89:0/1890629981 wait complete. 2023-09-30T22:19:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd795684700 1 Processor -- start 2023-09-30T22:19:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd795684700 1 -- start start 2023-09-30T22:19:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd795684700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd79011f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd795684700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd79011f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd795684700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd790133930 0x7fd7901c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd795684700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790121e70 con 0x7fd790133930 2023-09-30T22:19:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd795684700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79011eea0 con 0x7fd7900a0a30 2023-09-30T22:19:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd795684700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901211e0 con 0x7fd79009d630 2023-09-30T22:19:52.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.272+0000 7fd78f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd790133930 0x7fd7901c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd79011f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd790133930 0x7fd7901c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53160/0 (socket says 172.21.15.89:53160) 2023-09-30T22:19:52.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78f7fe700 1 -- 172.21.15.89:0/787310550 learned_addr learned my addr 172.21.15.89:0/787310550 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:52.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78effd700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd79011f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:52.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78f7fe700 1 -- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 msgr2=0x7fd79011f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78f7fe700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd79011f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78f7fe700 1 -- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 msgr2=0x7fd79011f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78f7fe700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd79011f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78f7fe700 1 -- 172.21.15.89:0/787310550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7800094a0 con 0x7fd790133930 2023-09-30T22:19:52.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78effd700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd79011f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:52.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd78f7fe700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd790133930 0x7fd7901c7f20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd78800cb40 tx=0x7fd78800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:52.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd7777fe700 1 -- 172.21.15.89:0/787310550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800aaa0 con 0x7fd790133930 2023-09-30T22:19:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd7777fe700 1 -- 172.21.15.89:0/787310550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78800ac00 con 0x7fd790133930 2023-09-30T22:19:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd7777fe700 1 -- 172.21.15.89:0/787310550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788012400 con 0x7fd790133930 2023-09-30T22:19:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7840094a0 con 0x7fd790133930 2023-09-30T22:19:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.273+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd790122ba0 con 0x7fd790133930 2023-09-30T22:19:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.274+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7900747f0 con 0x7fd790133930 2023-09-30T22:19:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.275+0000 7fd7777fe700 1 -- 172.21.15.89:0/787310550 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd78801b3b0 con 0x7fd790133930 2023-09-30T22:19:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.275+0000 7fd7777fe700 1 -- 172.21.15.89:0/787310550 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd78803f990 con 0x7fd790133930 2023-09-30T22:19:52.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.279+0000 7fd7777fe700 1 -- 172.21.15.89:0/787310550 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd7880192c0 con 0x7fd790133930 2023-09-30T22:19:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.293+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d1c0a28e0 con 0x7f4d1c0a1340 2023-09-30T22:19:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.293+0000 7f4d197fa700 1 -- 172.21.15.89:0/3773456988 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d08019e60 con 0x7f4d1c0a1340 2023-09-30T22:19:52.294 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a1340 msgr2=0x7f4d1c11ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a1340 0x7f4d1c11ec40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800c4e0 tx=0x7f4d0800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 shutdown_connections 2023-09-30T22:19:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c134170 0x7f4d1c11f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c0a0a30 0x7f4d1c11e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 --2- 172.21.15.89:0/3773456988 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c0a1340 0x7f4d1c11ec40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 >> 172.21.15.89:0/3773456988 conn(0x7f4d1c0a6550 msgr2=0x7f4d1c09c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 shutdown_connections 2023-09-30T22:19:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.295+0000 7f4d23033700 1 -- 172.21.15.89:0/3773456988 wait complete. 2023-09-30T22:19:52.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:52.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.386+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f06cc126930 con 0x7f06cc09d6f0 2023-09-30T22:19:52.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.386+0000 7f06b37fe700 1 -- 172.21.15.89:0/1697736781 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f06bc0193c0 con 0x7f06cc09d6f0 2023-09-30T22:19:52.387 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 msgr2=0x7f06cc1225c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc1225c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009ed0 tx=0x7f06bc004750 comp rx=0 tx=0).stop 2023-09-30T22:19:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 shutdown_connections 2023-09-30T22:19:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06cc09e410 0x7f06cc123040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06cc09d6f0 0x7f06cc1225c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 --2- 172.21.15.89:0/1697736781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc09e000 0x7f06cc122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 >> 172.21.15.89:0/1697736781 conn(0x7f06cc066f50 msgr2=0x7f06cc133f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:52.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 shutdown_connections 2023-09-30T22:19:52.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.389+0000 7f06d161f700 1 -- 172.21.15.89:0/1697736781 wait complete. 2023-09-30T22:19:52.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:52.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.434+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6100005590 con 0x7f612006eba0 2023-09-30T22:19:52.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.434+0000 7f6107fff700 1 -- 172.21.15.89:0/2960953757 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6110015070 con 0x7f612006eba0 2023-09-30T22:19:52.435 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:52.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.436+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612006eba0 msgr2=0x7f612006f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.436+0000 7f6125efe700 1 --2- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612006eba0 0x7f612006f610 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6110009c40 tx=0x7f6110004750 comp rx=0 tx=0).stop 2023-09-30T22:19:52.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.436+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 shutdown_connections 2023-09-30T22:19:52.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.436+0000 7f6125efe700 1 --2- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120131f40 0x7f6120070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.436+0000 7f6125efe700 1 --2- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f612006eba0 0x7f612006f610 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.436+0000 7f6125efe700 1 --2- 172.21.15.89:0/2960953757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6120131630 0x7f612006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.436+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 >> 172.21.15.89:0/2960953757 conn(0x7f61200a2aa0 msgr2=0x7f61201b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:52.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.437+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 shutdown_connections 2023-09-30T22:19:52.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.437+0000 7f6125efe700 1 -- 172.21.15.89:0/2960953757 wait complete. 2023-09-30T22:19:52.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.488+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd790072980 con 0x7fd790133930 2023-09-30T22:19:52.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.488+0000 7fd7777fe700 1 -- 172.21.15.89:0/787310550 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd788017030 con 0x7fd790133930 2023-09-30T22:19:52.489 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.490+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd790133930 msgr2=0x7fd7901c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:52.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.490+0000 7fd795684700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd790133930 0x7fd7901c7f20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd78800cb40 tx=0x7fd78800ce50 comp rx=0 tx=0).stop 2023-09-30T22:19:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.491+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 shutdown_connections 2023-09-30T22:19:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.491+0000 7fd795684700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd790133930 0x7fd7901c7f20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.491+0000 7fd795684700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd79009d630 0x7fd79011f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.491+0000 7fd795684700 1 --2- 172.21.15.89:0/787310550 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7900a0a30 0x7fd79011f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.491+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 >> 172.21.15.89:0/787310550 conn(0x7fd7900a3050 msgr2=0x7fd79009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.491+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 shutdown_connections 2023-09-30T22:19:52.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:52.491+0000 7fd795684700 1 -- 172.21.15.89:0/787310550 wait complete. 2023-09-30T22:19:52.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:53.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:53.407 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.438+0000 7f84b2f44700 1 Processor -- start 2023-09-30T22:19:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.438+0000 7f84b2f44700 1 -- start start 2023-09-30T22:19:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b2f44700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84ac09ffb0 0x7f84ac09f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b2f44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac138e30 0x7f84ac13b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b2f44700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac0a06b0 con 0x7f84ac09e850 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b2f44700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ac06b920 con 0x7f84ac09ffb0 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b2f44700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac09aa30 con 0x7f84ac138e30 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84abfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84ac09ffb0 0x7f84ac09f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b14e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac138e30 0x7f84ac13b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b14e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac138e30 0x7f84ac13b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44988/0 (socket says 172.21.15.89:44988) 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b14e1700 1 -- 172.21.15.89:0/1629006785 learned_addr learned my addr 172.21.15.89:0/1629006785 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84abfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84ac09ffb0 0x7f84ac09f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53168/0 (socket says 172.21.15.89:53168) 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b14e1700 1 -- 172.21.15.89:0/1629006785 >> v2:172.21.15.89:3300/0 conn(0x7f84ac09ffb0 msgr2=0x7f84ac09f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b14e1700 1 --2- 172.21.15.89:0/1629006785 >> v2:172.21.15.89:3300/0 conn(0x7f84ac09ffb0 0x7f84ac09f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b14e1700 1 -- 172.21.15.89:0/1629006785 >> v1:172.21.15.89:6789/0 conn(0x7f84ac09e850 legacy=0x7f84ac09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b14e1700 1 -- 172.21.15.89:0/1629006785 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac06caa0 con 0x7f84ac138e30 2023-09-30T22:19:53.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84b14e1700 1 --2- 172.21.15.89:0/1629006785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac138e30 0x7f84ac13b210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009e50 tx=0x7f84a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4596f46fdcb904b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:53.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84ab7fe700 1 -- 172.21.15.89:0/1629006785 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400e070 con 0x7f84ac138e30 2023-09-30T22:19:53.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84ab7fe700 1 -- 172.21.15.89:0/1629006785 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a400b910 con 0x7f84ac138e30 2023-09-30T22:19:53.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.439+0000 7f84ab7fe700 1 -- 172.21.15.89:0/1629006785 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400bcc0 con 0x7f84ac138e30 2023-09-30T22:19:53.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.440+0000 7f84b2f44700 1 -- 172.21.15.89:0/1629006785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac138e30 msgr2=0x7f84ac13b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.440+0000 7f84b2f44700 1 --2- 172.21.15.89:0/1629006785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac138e30 0x7f84ac13b210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009e50 tx=0x7f84a400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.440+0000 7f84b2f44700 1 -- 172.21.15.89:0/1629006785 shutdown_connections 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.440+0000 7f84b2f44700 1 --2- 172.21.15.89:0/1629006785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac138e30 0x7f84ac13b210 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.440+0000 7f84b2f44700 1 --2- 172.21.15.89:0/1629006785 >> v2:172.21.15.89:3300/0 conn(0x7f84ac09ffb0 0x7f84ac09f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.440+0000 7f84b2f44700 1 -- 172.21.15.89:0/1629006785 >> 172.21.15.89:0/1629006785 conn(0x7f84ac0a3f70 msgr2=0x7f84ac0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.440+0000 7f84b2f44700 1 -- 172.21.15.89:0/1629006785 shutdown_connections 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.440+0000 7f84b2f44700 1 -- 172.21.15.89:0/1629006785 wait complete. 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.441+0000 7f84b2f44700 1 Processor -- start 2023-09-30T22:19:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.441+0000 7f84b2f44700 1 -- start start 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.441+0000 7f84b2f44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac09e850 0x7f84ac12fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b2f44700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84ac09ffb0 0x7f84ac130080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b2f44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84ac138e30 0x7f84ac1305c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b2f44700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac137e70 con 0x7f84ac138e30 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b2f44700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac135550 con 0x7f84ac09ffb0 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b2f44700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac06ccc0 con 0x7f84ac09e850 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac09e850 0x7f84ac12fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac09e850 0x7f84ac12fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45004/0 (socket says 172.21.15.89:45004) 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 -- 172.21.15.89:0/127443943 learned_addr learned my addr 172.21.15.89:0/127443943 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:53.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84abfff700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84ac09ffb0 0x7f84ac130080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b14e1700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84ac138e30 0x7f84ac1305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 -- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84ac09ffb0 msgr2=0x7f84ac130080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84ac09ffb0 0x7f84ac130080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 -- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84ac138e30 msgr2=0x7f84ac1305c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84ac138e30 0x7f84ac1305c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 -- 172.21.15.89:0/127443943 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4009b30 con 0x7f84ac09e850 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b0ce0700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac09e850 0x7f84ac12fb40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f84a000c4e0 tx=0x7f84a000c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84a97fa700 1 -- 172.21.15.89:0/127443943 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0011790 con 0x7f84ac09e850 2023-09-30T22:19:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84a97fa700 1 -- 172.21.15.89:0/127443943 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a00118f0 con 0x7f84ac09e850 2023-09-30T22:19:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac1340e0 con 0x7f84ac09e850 2023-09-30T22:19:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84a97fa700 1 -- 172.21.15.89:0/127443943 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0011c60 con 0x7f84ac09e850 2023-09-30T22:19:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.442+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac135af0 con 0x7f84ac09e850 2023-09-30T22:19:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.443+0000 7f84a97fa700 1 -- 172.21.15.89:0/127443943 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f84a0005080 con 0x7f84ac09e850 2023-09-30T22:19:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.443+0000 7f84a97fa700 1 -- 172.21.15.89:0/127443943 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f84a003eec0 con 0x7f84ac09e850 2023-09-30T22:19:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.443+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ac09a790 con 0x7f84ac09e850 2023-09-30T22:19:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.448+0000 7f84a97fa700 1 -- 172.21.15.89:0/127443943 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f84a001c460 con 0x7f84ac09e850 2023-09-30T22:19:53.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:53.505 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.547+0000 7f6127f1b700 1 Processor -- start 2023-09-30T22:19:53.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.547+0000 7f6127f1b700 1 -- start start 2023-09-30T22:19:53.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.547+0000 7f6127f1b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f612012cf60 0x7f612006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.547+0000 7f6127f1b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.547+0000 7f6127f1b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120130260 0x7f6120136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.547+0000 7f6127f1b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612012d330 con 0x7f612012cf60 2023-09-30T22:19:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.547+0000 7f6127f1b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612012cd90 con 0x7f612012f8b0 2023-09-30T22:19:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.547+0000 7f6127f1b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61201306d0 con 0x7f6120130260 2023-09-30T22:19:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42004/0 (socket says 172.21.15.89:42004) 2023-09-30T22:19:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 -- 172.21.15.89:0/3448833949 learned_addr learned my addr 172.21.15.89:0/3448833949 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f6125cb7700 1 --2- 172.21.15.89:0/3448833949 >> v2:172.21.15.89:3300/0 conn(0x7f612012cf60 0x7f612006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61264b8700 1 --2- 172.21.15.89:0/3448833949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120130260 0x7f6120136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 -- 172.21.15.89:0/3448833949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120130260 msgr2=0x7f6120136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 --2- 172.21.15.89:0/3448833949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120130260 0x7f6120136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 -- 172.21.15.89:0/3448833949 >> v2:172.21.15.89:3300/0 conn(0x7f612012cf60 msgr2=0x7f612006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 --2- 172.21.15.89:0/3448833949 >> v2:172.21.15.89:3300/0 conn(0x7f612012cf60 0x7f612006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 -- 172.21.15.89:0/3448833949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120071780 con 0x7f612012f8b0 2023-09-30T22:19:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f61254b6700 1 --2- 172.21.15.89:0/3448833949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612012fd20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f611000bb90 tx=0x7f611000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbab8c1d284cb8c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f6124cb5700 1 -- 172.21.15.89:0/3448833949 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611000dc20 con 0x7f612012f8b0 2023-09-30T22:19:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f6124cb5700 1 -- 172.21.15.89:0/3448833949 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611000dd80 con 0x7f612012f8b0 2023-09-30T22:19:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.548+0000 7f6124cb5700 1 -- 172.21.15.89:0/3448833949 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110017630 con 0x7f612012f8b0 2023-09-30T22:19:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.549+0000 7f6127f1b700 1 -- 172.21.15.89:0/3448833949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 msgr2=0x7f612012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.549+0000 7f6127f1b700 1 --2- 172.21.15.89:0/3448833949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612012fd20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f611000bb90 tx=0x7f611000bea0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.549+0000 7f6127f1b700 1 -- 172.21.15.89:0/3448833949 shutdown_connections 2023-09-30T22:19:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.549+0000 7f6127f1b700 1 --2- 172.21.15.89:0/3448833949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6120130260 0x7f6120136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.549+0000 7f6127f1b700 1 --2- 172.21.15.89:0/3448833949 >> v2:172.21.15.89:3300/0 conn(0x7f612012cf60 0x7f612006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.549+0000 7f6127f1b700 1 --2- 172.21.15.89:0/3448833949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612012fd20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.549+0000 7f6127f1b700 1 -- 172.21.15.89:0/3448833949 >> 172.21.15.89:0/3448833949 conn(0x7f61200a2aa0 msgr2=0x7f61200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:53.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.550+0000 7f6127f1b700 1 -- 172.21.15.89:0/3448833949 shutdown_connections 2023-09-30T22:19:53.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.550+0000 7f6127f1b700 1 -- 172.21.15.89:0/3448833949 wait complete. 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.551+0000 7f6127f1b700 1 Processor -- start 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.551+0000 7f6127f1b700 1 -- start start 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6127f1b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612012cf60 0x7f612009bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6127f1b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612009c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6127f1b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120130260 0x7f61201c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6127f1b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612012a010 con 0x7f6120130260 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6127f1b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612009b6e0 con 0x7f612012f8b0 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6127f1b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f612009d9b0 con 0x7f612012cf60 2023-09-30T22:19:53.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6125cb7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612012cf60 0x7f612009bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f61264b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120130260 0x7f61201c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6125cb7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612012cf60 0x7f612009bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45024/0 (socket says 172.21.15.89:45024) 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f6125cb7700 1 -- 172.21.15.89:0/4170577617 learned_addr learned my addr 172.21.15.89:0/4170577617 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f61254b6700 1 --2- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612009c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f61264b8700 1 -- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612012cf60 msgr2=0x7f612009bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f61264b8700 1 --2- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612012cf60 0x7f612009bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f61264b8700 1 -- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 msgr2=0x7f612009c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f61264b8700 1 --2- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612009c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f61264b8700 1 -- 172.21.15.89:0/4170577617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611000b7f0 con 0x7f6120130260 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.552+0000 7f61264b8700 1 --2- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120130260 0x7f61201c8250 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f611800ef90 tx=0x7f611800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.553+0000 7f61167fc700 1 -- 172.21.15.89:0/4170577617 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611800cc90 con 0x7f6120130260 2023-09-30T22:19:53.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.553+0000 7f61167fc700 1 -- 172.21.15.89:0/4170577617 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611800f040 con 0x7f6120130260 2023-09-30T22:19:53.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.553+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612009c540 con 0x7f6120130260 2023-09-30T22:19:53.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.553+0000 7f61167fc700 1 -- 172.21.15.89:0/4170577617 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180146c0 con 0x7f6120130260 2023-09-30T22:19:53.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.553+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612009cde0 con 0x7f6120130260 2023-09-30T22:19:53.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.554+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6120074ea0 con 0x7f6120130260 2023-09-30T22:19:53.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.554+0000 7f61167fc700 1 -- 172.21.15.89:0/4170577617 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6118014820 con 0x7f6120130260 2023-09-30T22:19:53.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.554+0000 7f61167fc700 1 -- 172.21.15.89:0/4170577617 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f61180400b0 con 0x7f6120130260 2023-09-30T22:19:53.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.559+0000 7f61167fc700 1 -- 172.21.15.89:0/4170577617 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6118014ad0 con 0x7f6120130260 2023-09-30T22:19:53.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.592+0000 7f759820e700 1 Processor -- start 2023-09-30T22:19:53.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.593+0000 7f759820e700 1 -- start start 2023-09-30T22:19:53.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.593+0000 7f759820e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f759009d6f0 0x7f759009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f759820e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e000 0x7f75901339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f759820e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e410 0x7f7590133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f759820e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759006b460 con 0x7f759009d6f0 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f759820e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759009b120 con 0x7f759009e000 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f759820e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f759009a7e0 con 0x7f759009e410 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f7595faa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f759009d6f0 0x7f759009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f7595faa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f759009d6f0 0x7f759009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53210/0 (socket says 172.21.15.89:53210) 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f7595faa700 1 -- 172.21.15.89:0/475056850 learned_addr learned my addr 172.21.15.89:0/475056850 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f75967ab700 1 --2- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e410 0x7f7590133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f75957a9700 1 --2- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e000 0x7f75901339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f75967ab700 1 -- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e000 msgr2=0x7f75901339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f75967ab700 1 --2- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e000 0x7f75901339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f75967ab700 1 -- 172.21.15.89:0/475056850 >> v2:172.21.15.89:3300/0 conn(0x7f759009d6f0 msgr2=0x7f759009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f75967ab700 1 --2- 172.21.15.89:0/475056850 >> v2:172.21.15.89:3300/0 conn(0x7f759009d6f0 0x7f759009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f75967ab700 1 -- 172.21.15.89:0/475056850 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759009a500 con 0x7f759009e410 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f7595faa700 1 --2- 172.21.15.89:0/475056850 >> v2:172.21.15.89:3300/0 conn(0x7f759009d6f0 0x7f759009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.594+0000 7f75967ab700 1 --2- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e410 0x7f7590133f30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7588009870 tx=0x7f7588009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2caae5255ca91e75 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.595+0000 7f7594fa8700 1 -- 172.21.15.89:0/475056850 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588011830 con 0x7f759009e410 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.595+0000 7f7594fa8700 1 -- 172.21.15.89:0/475056850 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588011990 con 0x7f759009e410 2023-09-30T22:19:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.595+0000 7f7594fa8700 1 -- 172.21.15.89:0/475056850 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588011d20 con 0x7f759009e410 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 -- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e410 msgr2=0x7f7590133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 --2- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e410 0x7f7590133f30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7588009870 tx=0x7f7588009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 -- 172.21.15.89:0/475056850 shutdown_connections 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 --2- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e410 0x7f7590133f30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 --2- 172.21.15.89:0/475056850 >> v2:172.21.15.89:3300/0 conn(0x7f759009d6f0 0x7f759009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 --2- 172.21.15.89:0/475056850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e000 0x7f75901339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 -- 172.21.15.89:0/475056850 >> 172.21.15.89:0/475056850 conn(0x7f7590066f50 msgr2=0x7f759009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 -- 172.21.15.89:0/475056850 shutdown_connections 2023-09-30T22:19:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.596+0000 7f759820e700 1 -- 172.21.15.89:0/475056850 wait complete. 2023-09-30T22:19:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.597+0000 7f759820e700 1 Processor -- start 2023-09-30T22:19:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.597+0000 7f759820e700 1 -- start start 2023-09-30T22:19:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f759820e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f759009d6f0 0x7f75901c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f759820e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e000 0x7f75901c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f759820e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e410 0x7f75901c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f759820e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75901a71d0 con 0x7f759009d6f0 2023-09-30T22:19:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f759820e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75901a97f0 con 0x7f759009e410 2023-09-30T22:19:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f759820e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75901c5aa0 con 0x7f759009e000 2023-09-30T22:19:53.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75957a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e000 0x7f75901c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75957a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e000 0x7f75901c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45052/0 (socket says 172.21.15.89:45052) 2023-09-30T22:19:53.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75957a9700 1 -- 172.21.15.89:0/3016698424 learned_addr learned my addr 172.21.15.89:0/3016698424 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:53.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75967ab700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e410 0x7f75901c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f7595faa700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f759009d6f0 0x7f75901c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75957a9700 1 -- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e410 msgr2=0x7f75901c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75957a9700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e410 0x7f75901c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75957a9700 1 -- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f759009d6f0 msgr2=0x7f75901c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75957a9700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f759009d6f0 0x7f75901c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f75957a9700 1 -- 172.21.15.89:0/3016698424 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580009b30 con 0x7f759009e000 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.598+0000 7f7595faa700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f759009d6f0 0x7f75901c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.599+0000 7f75957a9700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e000 0x7f75901c5560 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9b0 tx=0x7f758400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.599+0000 7f758e7fc700 1 -- 172.21.15.89:0/3016698424 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584009710 con 0x7f759009e000 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.599+0000 7f758e7fc700 1 -- 172.21.15.89:0/3016698424 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758400c850 con 0x7f759009e000 2023-09-30T22:19:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.599+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75880094a0 con 0x7f759009e000 2023-09-30T22:19:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.599+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75901a8b30 con 0x7f759009e000 2023-09-30T22:19:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.599+0000 7f758e7fc700 1 -- 172.21.15.89:0/3016698424 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400cbc0 con 0x7f759009e000 2023-09-30T22:19:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.600+0000 7f758e7fc700 1 -- 172.21.15.89:0/3016698424 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f758400cd20 con 0x7f759009e000 2023-09-30T22:19:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.600+0000 7f758e7fc700 1 -- 172.21.15.89:0/3016698424 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f758403f720 con 0x7f759009e000 2023-09-30T22:19:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.600+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75700052e0 con 0x7f759009e000 2023-09-30T22:19:53.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.609+0000 7f758e7fc700 1 -- 172.21.15.89:0/3016698424 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7584021020 con 0x7f759009e000 2023-09-30T22:19:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.636+0000 7f390bbef700 1 Processor -- start 2023-09-30T22:19:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.636+0000 7f390bbef700 1 -- start start 2023-09-30T22:19:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.636+0000 7f390bbef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3904136710 0x7f3904138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.636+0000 7f390bbef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3904139030 0x7f390413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.636+0000 7f390bbef700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390406ef30 con 0x7f39041338a0 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.636+0000 7f390bbef700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390406fef0 con 0x7f3904139030 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.636+0000 7f390bbef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39041334b0 con 0x7f3904136710 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390998b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f39041338a0 0x7f3904133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53548/0 (socket says 172.21.15.89:53548) 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390918a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3904136710 0x7f3904138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390998b700 1 -- 172.21.15.89:0/2437653732 learned_addr learned my addr 172.21.15.89:0/2437653732 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390a18c700 1 --2- 172.21.15.89:0/2437653732 >> v2:172.21.15.89:3300/0 conn(0x7f3904139030 0x7f390413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390918a700 1 -- 172.21.15.89:0/2437653732 >> v2:172.21.15.89:3300/0 conn(0x7f3904139030 msgr2=0x7f390413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390918a700 1 --2- 172.21.15.89:0/2437653732 >> v2:172.21.15.89:3300/0 conn(0x7f3904139030 0x7f390413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390918a700 1 -- 172.21.15.89:0/2437653732 >> v1:172.21.15.89:6789/0 conn(0x7f39041338a0 legacy=0x7f3904133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390918a700 1 -- 172.21.15.89:0/2437653732 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390406fc30 con 0x7f3904136710 2023-09-30T22:19:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f390918a700 1 --2- 172.21.15.89:0/2437653732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3904136710 0x7f3904138af0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009840 tx=0x7f38f4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa64e31a0164083 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f3908989700 1 -- 172.21.15.89:0/2437653732 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4011830 con 0x7f3904136710 2023-09-30T22:19:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f3908989700 1 -- 172.21.15.89:0/2437653732 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4011990 con 0x7f3904136710 2023-09-30T22:19:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.637+0000 7f3908989700 1 -- 172.21.15.89:0/2437653732 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4011d20 con 0x7f3904136710 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.638+0000 7f390bbef700 1 -- 172.21.15.89:0/2437653732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3904136710 msgr2=0x7f3904138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.638+0000 7f390bbef700 1 --2- 172.21.15.89:0/2437653732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3904136710 0x7f3904138af0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009840 tx=0x7f38f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.638+0000 7f390bbef700 1 -- 172.21.15.89:0/2437653732 shutdown_connections 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.638+0000 7f390bbef700 1 --2- 172.21.15.89:0/2437653732 >> v2:172.21.15.89:3300/0 conn(0x7f3904139030 0x7f390413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.638+0000 7f390bbef700 1 --2- 172.21.15.89:0/2437653732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3904136710 0x7f3904138af0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.638+0000 7f390bbef700 1 -- 172.21.15.89:0/2437653732 >> 172.21.15.89:0/2437653732 conn(0x7f39040a2aa0 msgr2=0x7f39040a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.638+0000 7f390bbef700 1 -- 172.21.15.89:0/2437653732 shutdown_connections 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.638+0000 7f390bbef700 1 -- 172.21.15.89:0/2437653732 wait complete. 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.639+0000 7f390bbef700 1 Processor -- start 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.639+0000 7f390bbef700 1 -- start start 2023-09-30T22:19:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.639+0000 7f390bbef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39041338a0 0x7f390411adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.639+0000 7f390bbef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3904136710 0x7f390411b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.639+0000 7f390bbef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3904139030 0x7f390411b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.639+0000 7f390bbef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390406f640 con 0x7f3904136710 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390bbef700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390411eb20 con 0x7f3904139030 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390bbef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3904130cb0 con 0x7f39041338a0 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390998b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39041338a0 0x7f390411adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390998b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39041338a0 0x7f390411adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45070/0 (socket says 172.21.15.89:45070) 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390998b700 1 -- 172.21.15.89:0/121492899 learned_addr learned my addr 172.21.15.89:0/121492899 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390918a700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3904136710 0x7f390411b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390a18c700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3904139030 0x7f390411b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390a18c700 1 -- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39041338a0 msgr2=0x7f390411adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390a18c700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39041338a0 0x7f390411adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390a18c700 1 -- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3904136710 msgr2=0x7f390411b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390a18c700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3904136710 0x7f390411b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390a18c700 1 -- 172.21.15.89:0/121492899 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f40094a0 con 0x7f3904139030 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390998b700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39041338a0 0x7f390411adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390918a700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3904136710 0x7f390411b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390a18c700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3904139030 0x7f390411b840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00ef90 tx=0x7f38fc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f38fa7fc700 1 -- 172.21.15.89:0/121492899 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00cd40 con 0x7f3904139030 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f38fa7fc700 1 -- 172.21.15.89:0/121492899 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc00f040 con 0x7f3904139030 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390411d230 con 0x7f3904139030 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f38fa7fc700 1 -- 172.21.15.89:0/121492899 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0146c0 con 0x7f3904139030 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.640+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390411e070 con 0x7f3904139030 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.641+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390411e2a0 con 0x7f3904139030 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.642+0000 7f38fa7fc700 1 -- 172.21.15.89:0/121492899 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38fc014820 con 0x7f3904139030 2023-09-30T22:19:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.642+0000 7f38fa7fc700 1 -- 172.21.15.89:0/121492899 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f38fc0407d0 con 0x7f3904139030 2023-09-30T22:19:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.650+0000 7f38fa7fc700 1 -- 172.21.15.89:0/121492899 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38fc014ad0 con 0x7f3904139030 2023-09-30T22:19:53.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.675+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f84ac0747f0 con 0x7f84ac09e850 2023-09-30T22:19:53.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.675+0000 7f84a97fa700 1 -- 172.21.15.89:0/127443943 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f84a0019b00 con 0x7f84ac09e850 2023-09-30T22:19:53.676 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac09e850 msgr2=0x7f84ac12fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac09e850 0x7f84ac12fb40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f84a000c4e0 tx=0x7f84a000c7f0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 shutdown_connections 2023-09-30T22:19:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84ac138e30 0x7f84ac1305c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84ac09e850 0x7f84ac12fb40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 --2- 172.21.15.89:0/127443943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84ac09ffb0 0x7f84ac130080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 >> 172.21.15.89:0/127443943 conn(0x7f84ac0a3f70 msgr2=0x7f84ac13a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 shutdown_connections 2023-09-30T22:19:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.677+0000 7f84b2f44700 1 -- 172.21.15.89:0/127443943 wait complete. 2023-09-30T22:19:53.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:53.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.774+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f61200713a0 con 0x7f6120130260 2023-09-30T22:19:53.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.774+0000 7f61167fc700 1 -- 172.21.15.89:0/4170577617 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6118012340 con 0x7f6120130260 2023-09-30T22:19:53.775 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120130260 msgr2=0x7f61201c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 --2- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120130260 0x7f61201c8250 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f611800ef90 tx=0x7f611800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 shutdown_connections 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 --2- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6120130260 0x7f61201c8250 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 --2- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f612012cf60 0x7f612009bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 --2- 172.21.15.89:0/4170577617 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f612012f8b0 0x7f612009c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 >> 172.21.15.89:0/4170577617 conn(0x7f61200a2aa0 msgr2=0x7f61201b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 shutdown_connections 2023-09-30T22:19:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.777+0000 7f6127f1b700 1 -- 172.21.15.89:0/4170577617 wait complete. 2023-09-30T22:19:53.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:53.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.806+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7570005590 con 0x7f759009e000 2023-09-30T22:19:53.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.807+0000 7f758e7fc700 1 -- 172.21.15.89:0/3016698424 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75840232f0 con 0x7f759009e000 2023-09-30T22:19:53.807 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:53.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e000 msgr2=0x7f75901c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e000 0x7f75901c5560 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9b0 tx=0x7f758400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 shutdown_connections 2023-09-30T22:19:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f759009e410 0x7f75901c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f759009d6f0 0x7f75901c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 --2- 172.21.15.89:0/3016698424 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f759009e000 0x7f75901c5560 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 >> 172.21.15.89:0/3016698424 conn(0x7f7590066f50 msgr2=0x7f75901bac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 shutdown_connections 2023-09-30T22:19:53.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.809+0000 7f759820e700 1 -- 172.21.15.89:0/3016698424 wait complete. 2023-09-30T22:19:53.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.846+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f390411e460 con 0x7f3904139030 2023-09-30T22:19:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.847+0000 7f38fa7fc700 1 -- 172.21.15.89:0/121492899 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f390411e460 con 0x7f3904139030 2023-09-30T22:19:53.848 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3904139030 msgr2=0x7f390411b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3904139030 0x7f390411b840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00ef90 tx=0x7f38fc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 shutdown_connections 2023-09-30T22:19:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3904139030 0x7f390411b840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39041338a0 0x7f390411adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 --2- 172.21.15.89:0/121492899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3904136710 0x7f390411b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 >> 172.21.15.89:0/121492899 conn(0x7f39040a2aa0 msgr2=0x7f39041344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 shutdown_connections 2023-09-30T22:19:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:53.849+0000 7f390bbef700 1 -- 172.21.15.89:0/121492899 wait complete. 2023-09-30T22:19:53.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:54.692 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:54.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:54.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.827+0000 7fbb67b5e700 1 Processor -- start 2023-09-30T22:19:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.827+0000 7fbb67b5e700 1 -- start start 2023-09-30T22:19:54.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb67b5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009e930 0x7fbb6009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb67b5e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb67b5e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb60070940 con 0x7fbb6009f650 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb67b5e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6006fd60 con 0x7fbb6009e930 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb67b5e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6006fa80 con 0x7fbb6009f240 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb650f9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb658fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009e930 0x7fbb6009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb660fb700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbb6009f650 0x7fbb60131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53550/0 (socket says 172.21.15.89:53550) 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb660fb700 1 -- 172.21.15.89:0/2328140498 learned_addr learned my addr 172.21.15.89:0/2328140498 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb650f9700 1 -- 172.21.15.89:0/2328140498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009e930 msgr2=0x7fbb6009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb650f9700 1 --2- 172.21.15.89:0/2328140498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009e930 0x7fbb6009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb650f9700 1 -- 172.21.15.89:0/2328140498 >> v1:172.21.15.89:6789/0 conn(0x7fbb6009f650 legacy=0x7fbb60131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb650f9700 1 -- 172.21.15.89:0/2328140498 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6009fb60 con 0x7fbb6009f240 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb648f8700 1 -- 172.21.15.89:0/2328140498 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219369649 0 0) 0x7fbb60070940 con 0x7fbb6009f650 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb650f9700 1 --2- 172.21.15.89:0/2328140498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601317f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009870 tx=0x7fbb50009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fdbe657bcabbfe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb648f8700 1 -- 172.21.15.89:0/2328140498 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011830 con 0x7fbb6009f240 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.828+0000 7fbb648f8700 1 -- 172.21.15.89:0/2328140498 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb50011990 con 0x7fbb6009f240 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb648f8700 1 -- 172.21.15.89:0/2328140498 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011d20 con 0x7fbb6009f240 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb67b5e700 1 -- 172.21.15.89:0/2328140498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 msgr2=0x7fbb601317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb67b5e700 1 --2- 172.21.15.89:0/2328140498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601317f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009870 tx=0x7fbb50009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb67b5e700 1 -- 172.21.15.89:0/2328140498 shutdown_connections 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb67b5e700 1 --2- 172.21.15.89:0/2328140498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009e930 0x7fbb6009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb67b5e700 1 --2- 172.21.15.89:0/2328140498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601317f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb67b5e700 1 -- 172.21.15.89:0/2328140498 >> 172.21.15.89:0/2328140498 conn(0x7fbb600a4210 msgr2=0x7fbb600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb67b5e700 1 -- 172.21.15.89:0/2328140498 shutdown_connections 2023-09-30T22:19:54.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.829+0000 7fbb67b5e700 1 -- 172.21.15.89:0/2328140498 wait complete. 2023-09-30T22:19:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.831+0000 7fbb67b5e700 1 Processor -- start 2023-09-30T22:19:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.831+0000 7fbb67b5e700 1 -- start start 2023-09-30T22:19:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.831+0000 7fbb67b5e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6009e930 0x7fbb601a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.831+0000 7fbb67b5e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.831+0000 7fbb67b5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009f650 0x7fbb601a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.831+0000 7fbb67b5e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600a3460 con 0x7fbb6009e930 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.831+0000 7fbb67b5e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb601a5830 con 0x7fbb6009f650 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.831+0000 7fbb67b5e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60071d20 con 0x7fbb6009f240 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb650f9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb660fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009f650 0x7fbb601a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb650f9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601a41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45084/0 (socket says 172.21.15.89:45084) 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb650f9700 1 -- 172.21.15.89:0/3426062068 learned_addr learned my addr 172.21.15.89:0/3426062068 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb658fa700 1 --2- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6009e930 0x7fbb601a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb658fa700 1 -- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 msgr2=0x7fbb601a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb658fa700 1 --2- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb658fa700 1 -- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009f650 msgr2=0x7fbb601a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb658fa700 1 --2- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009f650 0x7fbb601a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb658fa700 1 -- 172.21.15.89:0/3426062068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500094a0 con 0x7fbb6009e930 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb658fa700 1 --2- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6009e930 0x7fbb601a3cb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00e7e0 tx=0x7fbb5c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb567fc700 1 -- 172.21.15.89:0/3426062068 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c00c070 con 0x7fbb6009e930 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb567fc700 1 -- 172.21.15.89:0/3426062068 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c013610 con 0x7fbb6009e930 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb567fc700 1 -- 172.21.15.89:0/3426062068 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c013980 con 0x7fbb6009e930 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.832+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb600a0190 con 0x7fbb6009e930 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.833+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb601a5f80 con 0x7fbb6009e930 2023-09-30T22:19:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.833+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb60074ea0 con 0x7fbb6009e930 2023-09-30T22:19:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.834+0000 7fbb567fc700 1 -- 172.21.15.89:0/3426062068 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbb5c013ae0 con 0x7fbb6009e930 2023-09-30T22:19:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.834+0000 7fbb567fc700 1 -- 172.21.15.89:0/3426062068 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbb5c03f7e0 con 0x7fbb6009e930 2023-09-30T22:19:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.838+0000 7fbb567fc700 1 -- 172.21.15.89:0/3426062068 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb5c012db0 con 0x7fbb6009e930 2023-09-30T22:19:54.862 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.932+0000 7f1ca26c5700 1 Processor -- start 2023-09-30T22:19:54.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.932+0000 7f1ca26c5700 1 -- start start 2023-09-30T22:19:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.932+0000 7f1ca26c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c09f410 0x7f1c9c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.932+0000 7f1ca26c5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c136fe0 0x7f1c9c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.932+0000 7f1ca26c5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9c070940 con 0x7f1c9c09e930 2023-09-30T22:19:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.932+0000 7f1ca26c5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c06fd60 con 0x7f1c9c136fe0 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.932+0000 7f1ca26c5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c06fa80 con 0x7f1c9c09f410 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1c9c09e930 0x7f1c9c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53554/0 (socket says 172.21.15.89:53554) 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9bfff700 1 -- 172.21.15.89:0/3416956292 learned_addr learned my addr 172.21.15.89:0/3416956292 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9b7fe700 1 --2- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c09f410 0x7f1c9c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1ca0c62700 1 --2- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c136fe0 0x7f1c9c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9affd700 1 -- 172.21.15.89:0/3416956292 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201642842 0 0) 0x7f1c9c070940 con 0x7f1c9c09e930 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9affd700 1 -- 172.21.15.89:0/3416956292 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c840035b0 con 0x7f1c9c09e930 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9b7fe700 1 -- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c136fe0 msgr2=0x7f1c9c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9b7fe700 1 --2- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c136fe0 0x7f1c9c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9b7fe700 1 -- 172.21.15.89:0/3416956292 >> v1:172.21.15.89:6789/0 conn(0x7f1c9c09e930 legacy=0x7f1c9c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9b7fe700 1 -- 172.21.15.89:0/3416956292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c09fb60 con 0x7f1c9c09f410 2023-09-30T22:19:54.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1ca0c62700 1 --2- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c136fe0 0x7f1c9c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:54.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9b7fe700 1 --2- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c09f410 0x7f1c9c136aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009870 tx=0x7f1c90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94dd2e01367c752d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:54.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.933+0000 7f1c9affd700 1 -- 172.21.15.89:0/3416956292 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90011830 con 0x7f1c9c09f410 2023-09-30T22:19:54.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1c9affd700 1 -- 172.21.15.89:0/3416956292 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c90011990 con 0x7f1c9c09f410 2023-09-30T22:19:54.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1c9affd700 1 -- 172.21.15.89:0/3416956292 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90011d20 con 0x7f1c9c09f410 2023-09-30T22:19:54.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1ca26c5700 1 -- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c09f410 msgr2=0x7f1c9c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1ca26c5700 1 --2- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c09f410 0x7f1c9c136aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009870 tx=0x7f1c90009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1ca26c5700 1 -- 172.21.15.89:0/3416956292 shutdown_connections 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1ca26c5700 1 --2- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c136fe0 0x7f1c9c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1ca26c5700 1 --2- 172.21.15.89:0/3416956292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c09f410 0x7f1c9c136aa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1ca26c5700 1 -- 172.21.15.89:0/3416956292 >> 172.21.15.89:0/3416956292 conn(0x7f1c9c0a4210 msgr2=0x7f1c9c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1ca26c5700 1 -- 172.21.15.89:0/3416956292 shutdown_connections 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.934+0000 7f1ca26c5700 1 -- 172.21.15.89:0/3416956292 wait complete. 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.936+0000 7f1ca26c5700 1 Processor -- start 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.936+0000 7f1ca26c5700 1 -- start start 2023-09-30T22:19:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1ca26c5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c09e930 0x7f1c9c1c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1ca26c5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c09f410 0x7f1c9c1c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1ca26c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c136fe0 0x7f1c9c1c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1ca26c5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c071d20 con 0x7f1c9c09e930 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1ca26c5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c122050 con 0x7f1c9c09f410 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1ca26c5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c06d470 con 0x7f1c9c136fe0 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c09e930 0x7f1c9c1c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c09e930 0x7f1c9c1c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53252/0 (socket says 172.21.15.89:53252) 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 -- 172.21.15.89:0/1403957768 learned_addr learned my addr 172.21.15.89:0/1403957768 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1ca0c62700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c136fe0 0x7f1c9c1c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9b7fe700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c09f410 0x7f1c9c1c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 -- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c136fe0 msgr2=0x7f1c9c1c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c136fe0 0x7f1c9c1c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 -- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c09f410 msgr2=0x7f1c9c1c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c09f410 0x7f1c9c1c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 -- 172.21.15.89:0/1403957768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94009a70 con 0x7f1c9c09e930 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1c9bfff700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c09e930 0x7f1c9c1c81f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00c730 tx=0x7f1c8c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.937+0000 7f1ca0c62700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c136fe0 0x7f1c9c1c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.938+0000 7f1c98ff9700 1 -- 172.21.15.89:0/1403957768 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c005d70 con 0x7f1c9c09e930 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.938+0000 7f1c98ff9700 1 -- 172.21.15.89:0/1403957768 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c0070d0 con 0x7f1c9c09e930 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.938+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c900094a0 con 0x7f1c9c09e930 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.938+0000 7f1c98ff9700 1 -- 172.21.15.89:0/1403957768 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c007440 con 0x7f1c9c09e930 2023-09-30T22:19:54.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.938+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c120fb0 con 0x7f1c9c09e930 2023-09-30T22:19:54.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.939+0000 7f1c98ff9700 1 -- 172.21.15.89:0/1403957768 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c8c0075a0 con 0x7f1c9c09e930 2023-09-30T22:19:54.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.939+0000 7f1c98ff9700 1 -- 172.21.15.89:0/1403957768 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1c8c03eb80 con 0x7f1c9c09e930 2023-09-30T22:19:54.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.939+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c0052e0 con 0x7f1c9c09e930 2023-09-30T22:19:54.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.944+0000 7f1c98ff9700 1 -- 172.21.15.89:0/1403957768 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c8c03ee60 con 0x7f1c9c09e930 2023-09-30T22:19:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.957+0000 7fe058226700 1 Processor -- start 2023-09-30T22:19:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.957+0000 7fe058226700 1 -- start start 2023-09-30T22:19:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.957+0000 7fe058226700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.957+0000 7fe058226700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05013d1f0 0x7fe05013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.957+0000 7fe058226700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05009a810 con 0x7fe05006a770 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.957+0000 7fe058226700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05006ab40 con 0x7fe05013d1f0 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.957+0000 7fe058226700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe05009ebc0 con 0x7fe05009de00 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0567c3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05013d1f0 0x7fe05013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0557c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe055fc2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe05006a770 0x7fe05009d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53568/0 (socket says 172.21.15.89:53568) 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe055fc2700 1 -- 172.21.15.89:0/3418438277 learned_addr learned my addr 172.21.15.89:0/3418438277 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0567c3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05013d1f0 0x7fe05013f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42078/0 (socket says 172.21.15.89:42078) 2023-09-30T22:19:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0557c1700 1 -- 172.21.15.89:0/3418438277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05013d1f0 msgr2=0x7fe05013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0557c1700 1 --2- 172.21.15.89:0/3418438277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05013d1f0 0x7fe05013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0557c1700 1 -- 172.21.15.89:0/3418438277 >> v1:172.21.15.89:6789/0 conn(0x7fe05006a770 legacy=0x7fe05009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0557c1700 1 -- 172.21.15.89:0/3418438277 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05009e920 con 0x7fe05009de00 2023-09-30T22:19:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe054fc0700 1 -- 172.21.15.89:0/3418438277 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119867124 0 0) 0x7fe05009a810 con 0x7fe05006a770 2023-09-30T22:19:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0567c3700 1 --2- 172.21.15.89:0/3418438277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05013d1f0 0x7fe05013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe0557c1700 1 --2- 172.21.15.89:0/3418438277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05009e250 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe040009870 tx=0x7fe040009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237b3db052318c91 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe054fc0700 1 -- 172.21.15.89:0/3418438277 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040011830 con 0x7fe05009de00 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe054fc0700 1 -- 172.21.15.89:0/3418438277 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe040011990 con 0x7fe05009de00 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.958+0000 7fe054fc0700 1 -- 172.21.15.89:0/3418438277 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040011d20 con 0x7fe05009de00 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.959+0000 7fe058226700 1 -- 172.21.15.89:0/3418438277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 msgr2=0x7fe05009e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.959+0000 7fe058226700 1 --2- 172.21.15.89:0/3418438277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05009e250 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe040009870 tx=0x7fe040009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.959+0000 7fe058226700 1 -- 172.21.15.89:0/3418438277 shutdown_connections 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.959+0000 7fe058226700 1 --2- 172.21.15.89:0/3418438277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05013d1f0 0x7fe05013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.959+0000 7fe058226700 1 --2- 172.21.15.89:0/3418438277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05009e250 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.959+0000 7fe058226700 1 -- 172.21.15.89:0/3418438277 >> 172.21.15.89:0/3418438277 conn(0x7fe050066f50 msgr2=0x7fe05009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.959+0000 7fe058226700 1 -- 172.21.15.89:0/3418438277 shutdown_connections 2023-09-30T22:19:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.959+0000 7fe058226700 1 -- 172.21.15.89:0/3418438277 wait complete. 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.960+0000 7fe058226700 1 Processor -- start 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.960+0000 7fe058226700 1 -- start start 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.960+0000 7fe058226700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05006a770 0x7fe05011c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.960+0000 7fe058226700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05013b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe058226700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05013d1f0 0x7fe05013baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe058226700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05011ec60 con 0x7fe05013d1f0 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe058226700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe050135c30 con 0x7fe05006a770 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe058226700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe05009f1a0 con 0x7fe05009de00 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe055fc2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05006a770 0x7fe05011c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe055fc2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05006a770 0x7fe05011c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42094/0 (socket says 172.21.15.89:42094) 2023-09-30T22:19:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe055fc2700 1 -- 172.21.15.89:0/2973750971 learned_addr learned my addr 172.21.15.89:0/2973750971 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0567c3700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05013d1f0 0x7fe05013baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0557c1700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05013b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0567c3700 1 -- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 msgr2=0x7fe05013b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0567c3700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05013b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0567c3700 1 -- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05006a770 msgr2=0x7fe05011c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0567c3700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05006a770 0x7fe05011c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0567c3700 1 -- 172.21.15.89:0/2973750971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048009a70 con 0x7fe05013d1f0 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0567c3700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05013d1f0 0x7fe05013baa0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe04800efc0 tx=0x7fe04800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0467fc700 1 -- 172.21.15.89:0/2973750971 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800ca90 con 0x7fe05013d1f0 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe0467fc700 1 -- 172.21.15.89:0/2973750971 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04800f040 con 0x7fe05013d1f0 2023-09-30T22:19:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0400094a0 con 0x7fe05013d1f0 2023-09-30T22:19:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.961+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05011ee60 con 0x7fe05013d1f0 2023-09-30T22:19:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.962+0000 7fe0467fc700 1 -- 172.21.15.89:0/2973750971 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800ce30 con 0x7fe05013d1f0 2023-09-30T22:19:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.962+0000 7fe0467fc700 1 -- 172.21.15.89:0/2973750971 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe048014480 con 0x7fe05013d1f0 2023-09-30T22:19:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.963+0000 7fe0467fc700 1 -- 172.21.15.89:0/2973750971 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe04803fbb0 con 0x7fe05013d1f0 2023-09-30T22:19:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.963+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05011cc70 con 0x7fe05013d1f0 2023-09-30T22:19:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.968+0000 7fe0467fc700 1 -- 172.21.15.89:0/2973750971 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe05011cc70 con 0x7fe05013d1f0 2023-09-30T22:19:54.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.993+0000 7fad552d1700 1 Processor -- start 2023-09-30T22:19:54.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.993+0000 7fad552d1700 1 -- start start 2023-09-30T22:19:54.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad552d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad552d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e440 0x7fad50134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad552d1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5009b120 con 0x7fad5009e000 2023-09-30T22:19:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad552d1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5006b460 con 0x7fad5009d6f0 2023-09-30T22:19:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad552d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5009a7e0 con 0x7fad5009e440 2023-09-30T22:19:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4e7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fad5009e000 0x7fad501339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53582/0 (socket says 172.21.15.89:53582) 2023-09-30T22:19:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4e7fc700 1 -- 172.21.15.89:0/1667296892 learned_addr learned my addr 172.21.15.89:0/1667296892 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4f7fe700 1 --2- 172.21.15.89:0/1667296892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e440 0x7fad50134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4effd700 1 -- 172.21.15.89:0/1667296892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e440 msgr2=0x7fad50134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4effd700 1 --2- 172.21.15.89:0/1667296892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e440 0x7fad50134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4effd700 1 -- 172.21.15.89:0/1667296892 >> v1:172.21.15.89:6789/0 conn(0x7fad5009e000 legacy=0x7fad501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4effd700 1 -- 172.21.15.89:0/1667296892 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5009a500 con 0x7fad5009d6f0 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.994+0000 7fad4effd700 1 --2- 172.21.15.89:0/1667296892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009d6f0 0x7fad5009dac0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fad40009870 tx=0x7fad40009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54495b88670cac03 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad4dffb700 1 -- 172.21.15.89:0/1667296892 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40011830 con 0x7fad5009d6f0 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad4dffb700 1 -- 172.21.15.89:0/1667296892 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad40011990 con 0x7fad5009d6f0 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad4dffb700 1 -- 172.21.15.89:0/1667296892 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40011d20 con 0x7fad5009d6f0 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad552d1700 1 -- 172.21.15.89:0/1667296892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009d6f0 msgr2=0x7fad5009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad552d1700 1 --2- 172.21.15.89:0/1667296892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009d6f0 0x7fad5009dac0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fad40009870 tx=0x7fad40009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad552d1700 1 -- 172.21.15.89:0/1667296892 shutdown_connections 2023-09-30T22:19:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad552d1700 1 --2- 172.21.15.89:0/1667296892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009e440 0x7fad50134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad552d1700 1 --2- 172.21.15.89:0/1667296892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009d6f0 0x7fad5009dac0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad552d1700 1 -- 172.21.15.89:0/1667296892 >> 172.21.15.89:0/1667296892 conn(0x7fad50066f50 msgr2=0x7fad5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad552d1700 1 -- 172.21.15.89:0/1667296892 shutdown_connections 2023-09-30T22:19:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.995+0000 7fad552d1700 1 -- 172.21.15.89:0/1667296892 wait complete. 2023-09-30T22:19:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.997+0000 7fad552d1700 1 Processor -- start 2023-09-30T22:19:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.997+0000 7fad552d1700 1 -- start start 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad552d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad501a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad552d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e000 0x7fad501a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad552d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e440 0x7fad501a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad552d1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad501a8d00 con 0x7fad5009e440 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad552d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad501a72e0 con 0x7fad5009d6f0 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad552d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad501a9750 con 0x7fad5009e000 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e440 0x7fad501a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e440 0x7fad501a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53278/0 (socket says 172.21.15.89:53278) 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 -- 172.21.15.89:0/3799034930 learned_addr learned my addr 172.21.15.89:0/3799034930 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4effd700 1 --2- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad501a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 -- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e000 msgr2=0x7fad501a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 --2- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e000 0x7fad501a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 -- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 msgr2=0x7fad501a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 --2- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad501a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 -- 172.21.15.89:0/3799034930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad400094a0 con 0x7fad5009e440 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad4f7fe700 1 --2- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e440 0x7fad501a68b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fad48007e40 tx=0x7fad4800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.998+0000 7fad377fe700 1 -- 172.21.15.89:0/3799034930 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800ede0 con 0x7fad5009e440 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.999+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad501a8840 con 0x7fad5009e440 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.999+0000 7fad377fe700 1 -- 172.21.15.89:0/3799034930 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad48014420 con 0x7fad5009e440 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.999+0000 7fad377fe700 1 -- 172.21.15.89:0/3799034930 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480147d0 con 0x7fad5009e440 2023-09-30T22:19:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:54.999+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad501a7b30 con 0x7fad5009e440 2023-09-30T22:19:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.000+0000 7fad377fe700 1 -- 172.21.15.89:0/3799034930 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad48014930 con 0x7fad5009e440 2023-09-30T22:19:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.000+0000 7fad377fe700 1 -- 172.21.15.89:0/3799034930 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fad4803fe40 con 0x7fad5009e440 2023-09-30T22:19:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.000+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad50074ea0 con 0x7fad5009e440 2023-09-30T22:19:55.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.009+0000 7fad377fe700 1 -- 172.21.15.89:0/3799034930 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad48014be0 con 0x7fad5009e440 2023-09-30T22:19:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.076+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbb6006b250 con 0x7fbb6009e930 2023-09-30T22:19:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.076+0000 7fbb567fc700 1 -- 172.21.15.89:0/3426062068 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb5c012370 con 0x7fbb6009e930 2023-09-30T22:19:55.077 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6009e930 msgr2=0x7fbb601a3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 --2- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6009e930 0x7fbb601a3cb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00e7e0 tx=0x7fbb5c00eaf0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 shutdown_connections 2023-09-30T22:19:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 --2- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6009f650 0x7fbb601a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 --2- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6009e930 0x7fbb601a3cb0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 --2- 172.21.15.89:0/3426062068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6009f240 0x7fbb601a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 >> 172.21.15.89:0/3426062068 conn(0x7fbb600a4210 msgr2=0x7fbb601b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 shutdown_connections 2023-09-30T22:19:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.078+0000 7fbb67b5e700 1 -- 172.21.15.89:0/3426062068 wait complete. 2023-09-30T22:19:55.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.161+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe05011cc70 con 0x7fe05013d1f0 2023-09-30T22:19:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.162+0000 7fe0467fc700 1 -- 172.21.15.89:0/2973750971 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe05011cc70 con 0x7fe05013d1f0 2023-09-30T22:19:55.163 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:55.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05013d1f0 msgr2=0x7fe05013baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:55.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05013d1f0 0x7fe05013baa0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe04800efc0 tx=0x7fe04800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 shutdown_connections 2023-09-30T22:19:55.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05013d1f0 0x7fe05013baa0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe05006a770 0x7fe05011c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 --2- 172.21.15.89:0/2973750971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009de00 0x7fe05013b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 >> 172.21.15.89:0/2973750971 conn(0x7fe050066f50 msgr2=0x7fe0500a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:55.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 shutdown_connections 2023-09-30T22:19:55.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7fe058226700 1 -- 172.21.15.89:0/2973750971 wait complete. 2023-09-30T22:19:55.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1c7c005590 con 0x7f1c9c09e930 2023-09-30T22:19:55.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.164+0000 7f1c98ff9700 1 -- 172.21.15.89:0/1403957768 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c8c011d70 con 0x7f1c9c09e930 2023-09-30T22:19:55.167 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:55.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.166+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c09e930 msgr2=0x7f1c9c1c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:55.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.166+0000 7f1ca26c5700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c09e930 0x7f1c9c1c81f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00c730 tx=0x7f1c8c00ca40 comp rx=0 tx=0).stop 2023-09-30T22:19:55.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.167+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 shutdown_connections 2023-09-30T22:19:55.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.167+0000 7f1ca26c5700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1c9c136fe0 0x7f1c9c1c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.167+0000 7f1ca26c5700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9c09e930 0x7f1c9c1c81f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.167+0000 7f1ca26c5700 1 --2- 172.21.15.89:0/1403957768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1c9c09f410 0x7f1c9c1c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.167+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 >> 172.21.15.89:0/1403957768 conn(0x7f1c9c0a4210 msgr2=0x7f1c9c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:55.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.167+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 shutdown_connections 2023-09-30T22:19:55.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.167+0000 7f1ca26c5700 1 -- 172.21.15.89:0/1403957768 wait complete. 2023-09-30T22:19:55.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:55.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:55.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.201+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fad5006c360 con 0x7fad5009e440 2023-09-30T22:19:55.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.202+0000 7fad377fe700 1 -- 172.21.15.89:0/3799034930 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad4801b310 con 0x7fad5009e440 2023-09-30T22:19:55.202 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:55.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.203+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e440 msgr2=0x7fad501a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:55.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.203+0000 7fad552d1700 1 --2- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e440 0x7fad501a68b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fad48007e40 tx=0x7fad4800e3c0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.204+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 shutdown_connections 2023-09-30T22:19:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.204+0000 7fad552d1700 1 --2- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5009e440 0x7fad501a68b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.204+0000 7fad552d1700 1 --2- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5009d6f0 0x7fad501a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.204+0000 7fad552d1700 1 --2- 172.21.15.89:0/3799034930 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5009e000 0x7fad501a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.204+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 >> 172.21.15.89:0/3799034930 conn(0x7fad50066f50 msgr2=0x7fad501baea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.204+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 shutdown_connections 2023-09-30T22:19:55.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:55.204+0000 7fad552d1700 1 -- 172.21.15.89:0/3799034930 wait complete. 2023-09-30T22:19:55.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:56.093 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:56.178 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:56.181 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:56.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:56.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.232+0000 7f98335cf700 1 Processor -- start 2023-09-30T22:19:56.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.232+0000 7f98335cf700 1 -- start start 2023-09-30T22:19:56.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.233+0000 7f98335cf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f982c1338a0 0x7f982c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.233+0000 7f98335cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c09de50 0x7f982c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.233+0000 7f98335cf700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982c06ef30 con 0x7f982c09e760 2023-09-30T22:19:56.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.233+0000 7f98335cf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f982c06fef0 con 0x7f982c1338a0 2023-09-30T22:19:56.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.233+0000 7f98335cf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f982c1334b0 con 0x7f982c09de50 2023-09-30T22:19:56.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.233+0000 7f9830b6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c09de50 0x7f982c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.233+0000 7f9830b6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c09de50 0x7f982c09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45138/0 (socket says 172.21.15.89:45138) 2023-09-30T22:19:56.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.233+0000 7f9830b6a700 1 -- 172.21.15.89:0/943429880 learned_addr learned my addr 172.21.15.89:0/943429880 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:56.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f983136b700 1 --2- 172.21.15.89:0/943429880 >> v2:172.21.15.89:3300/0 conn(0x7f982c1338a0 0x7f982c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f9830b6a700 1 -- 172.21.15.89:0/943429880 >> v2:172.21.15.89:3300/0 conn(0x7f982c1338a0 msgr2=0x7f982c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:56.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f9830b6a700 1 --2- 172.21.15.89:0/943429880 >> v2:172.21.15.89:3300/0 conn(0x7f982c1338a0 0x7f982c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f9830b6a700 1 -- 172.21.15.89:0/943429880 >> v1:172.21.15.89:6789/0 conn(0x7f982c09e760 legacy=0x7f982c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f9830b6a700 1 -- 172.21.15.89:0/943429880 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c06fc30 con 0x7f982c09de50 2023-09-30T22:19:56.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f9830b6a700 1 --2- 172.21.15.89:0/943429880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c09de50 0x7f982c09e220 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f981c0098f0 tx=0x7f981c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b486e01d33b1fce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:56.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f9823fff700 1 -- 172.21.15.89:0/943429880 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c011830 con 0x7f982c09de50 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f9823fff700 1 -- 172.21.15.89:0/943429880 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981c011990 con 0x7f982c09de50 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.234+0000 7f9823fff700 1 -- 172.21.15.89:0/943429880 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c011d20 con 0x7f982c09de50 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.235+0000 7f98335cf700 1 -- 172.21.15.89:0/943429880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c09de50 msgr2=0x7f982c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.235+0000 7f98335cf700 1 --2- 172.21.15.89:0/943429880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c09de50 0x7f982c09e220 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f981c0098f0 tx=0x7f981c009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.235+0000 7f98335cf700 1 -- 172.21.15.89:0/943429880 shutdown_connections 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.235+0000 7f98335cf700 1 --2- 172.21.15.89:0/943429880 >> v2:172.21.15.89:3300/0 conn(0x7f982c1338a0 0x7f982c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.235+0000 7f98335cf700 1 --2- 172.21.15.89:0/943429880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c09de50 0x7f982c09e220 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.235+0000 7f98335cf700 1 -- 172.21.15.89:0/943429880 >> 172.21.15.89:0/943429880 conn(0x7f982c0a2aa0 msgr2=0x7f982c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.235+0000 7f98335cf700 1 -- 172.21.15.89:0/943429880 shutdown_connections 2023-09-30T22:19:56.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.236+0000 7f98335cf700 1 -- 172.21.15.89:0/943429880 wait complete. 2023-09-30T22:19:56.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.236+0000 7f98335cf700 1 Processor -- start 2023-09-30T22:19:56.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.236+0000 7f98335cf700 1 -- start start 2023-09-30T22:19:56.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f98335cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c09de50 0x7f982c11ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f98335cf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c09e760 0x7f982c11ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f98335cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c1338a0 0x7f982c11f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f98335cf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c120ea0 con 0x7f982c09de50 2023-09-30T22:19:56.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f98335cf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982c120720 con 0x7f982c09e760 2023-09-30T22:19:56.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f98335cf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f982c122240 con 0x7f982c1338a0 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f9831b6c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c1338a0 0x7f982c11f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f9831b6c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c1338a0 0x7f982c11f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45152/0 (socket says 172.21.15.89:45152) 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f9831b6c700 1 -- 172.21.15.89:0/3533412989 learned_addr learned my addr 172.21.15.89:0/3533412989 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f983136b700 1 --2- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c09de50 0x7f982c11ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f9831b6c700 1 -- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c09e760 msgr2=0x7f982c11ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f9831b6c700 1 --2- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c09e760 0x7f982c11ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f9831b6c700 1 -- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c09de50 msgr2=0x7f982c11ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f9831b6c700 1 --2- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c09de50 0x7f982c11ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.237+0000 7f9831b6c700 1 -- 172.21.15.89:0/3533412989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c009550 con 0x7f982c1338a0 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.238+0000 7f9831b6c700 1 --2- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c1338a0 0x7f982c11f480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f982400cc50 tx=0x7f982400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:56.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.238+0000 7f9821ffb700 1 -- 172.21.15.89:0/3533412989 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824003c00 con 0x7f982c1338a0 2023-09-30T22:19:56.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.238+0000 7f9821ffb700 1 -- 172.21.15.89:0/3533412989 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9824003d60 con 0x7f982c1338a0 2023-09-30T22:19:56.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.238+0000 7f9821ffb700 1 -- 172.21.15.89:0/3533412989 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982400ab60 con 0x7f982c1338a0 2023-09-30T22:19:56.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.238+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982c0a1020 con 0x7f982c1338a0 2023-09-30T22:19:56.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.238+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982c122cb0 con 0x7f982c1338a0 2023-09-30T22:19:56.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.239+0000 7f9821ffb700 1 -- 172.21.15.89:0/3533412989 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f982400acc0 con 0x7f982c1338a0 2023-09-30T22:19:56.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.239+0000 7f9821ffb700 1 -- 172.21.15.89:0/3533412989 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9824011540 con 0x7f982c1338a0 2023-09-30T22:19:56.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.239+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982c074ea0 con 0x7f982c1338a0 2023-09-30T22:19:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.244+0000 7f9821ffb700 1 -- 172.21.15.89:0/3533412989 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9824044060 con 0x7f982c1338a0 2023-09-30T22:19:56.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.311+0000 7f2e08cff700 1 Processor -- start 2023-09-30T22:19:56.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.311+0000 7f2e08cff700 1 -- start start 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.312+0000 7f2e08cff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409d6f0 0x7f2e0409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.312+0000 7f2e08cff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e0409e000 0x7f2e04130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.312+0000 7f2e08cff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0409f130 con 0x7f2e0409e410 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.312+0000 7f2e08cff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0409f520 con 0x7f2e0409e000 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.312+0000 7f2e08cff700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0409e930 con 0x7f2e0409d6f0 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.312+0000 7f2e01d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e0409e000 0x7f2e04130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.312+0000 7f2e01d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e0409e000 0x7f2e04130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53298/0 (socket says 172.21.15.89:53298) 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.312+0000 7f2e01d9b700 1 -- 172.21.15.89:0/3931981161 learned_addr learned my addr 172.21.15.89:0/3931981161 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:56.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.313+0000 7f2e01d9b700 1 -- 172.21.15.89:0/3931981161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409d6f0 msgr2=0x7f2e0409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:56.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.313+0000 7f2e01d9b700 1 --2- 172.21.15.89:0/3931981161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409d6f0 0x7f2e0409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.313+0000 7f2e01d9b700 1 -- 172.21.15.89:0/3931981161 >> v1:172.21.15.89:6789/0 conn(0x7f2e0409e410 legacy=0x7f2e04130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.313+0000 7f2e01d9b700 1 -- 172.21.15.89:0/3931981161 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0409b2d0 con 0x7f2e0409e000 2023-09-30T22:19:56.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.313+0000 7f2e01d9b700 1 --2- 172.21.15.89:0/3931981161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 0x7f2e04130660 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2df80098f0 tx=0x7f2df8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e977225ef30a4e13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:56.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.313+0000 7f2e0159a700 1 -- 172.21.15.89:0/3931981161 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800b8a0 con 0x7f2e0409e000 2023-09-30T22:19:56.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.313+0000 7f2e0159a700 1 -- 172.21.15.89:0/3931981161 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df800ba00 con 0x7f2e0409e000 2023-09-30T22:19:56.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.313+0000 7f2e0159a700 1 -- 172.21.15.89:0/3931981161 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800bd90 con 0x7f2e0409e000 2023-09-30T22:19:56.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.314+0000 7f2e08cff700 1 -- 172.21.15.89:0/3931981161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 msgr2=0x7f2e04130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.314+0000 7f2e08cff700 1 --2- 172.21.15.89:0/3931981161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 0x7f2e04130660 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2df80098f0 tx=0x7f2df8009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.314+0000 7f2e08cff700 1 -- 172.21.15.89:0/3931981161 shutdown_connections 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.314+0000 7f2e08cff700 1 --2- 172.21.15.89:0/3931981161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409d6f0 0x7f2e0409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.314+0000 7f2e08cff700 1 --2- 172.21.15.89:0/3931981161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 0x7f2e04130660 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.314+0000 7f2e08cff700 1 -- 172.21.15.89:0/3931981161 >> 172.21.15.89:0/3931981161 conn(0x7f2e04066f50 msgr2=0x7f2e0409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.314+0000 7f2e08cff700 1 -- 172.21.15.89:0/3931981161 shutdown_connections 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.314+0000 7f2e08cff700 1 -- 172.21.15.89:0/3931981161 wait complete. 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.315+0000 7f2e08cff700 1 Processor -- start 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.315+0000 7f2e08cff700 1 -- start start 2023-09-30T22:19:56.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.315+0000 7f2e08cff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e0409d6f0 0x7f2e0412dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.315+0000 7f2e08cff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 0x7f2e0412e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.315+0000 7f2e08cff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409e410 0x7f2e0412e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.315+0000 7f2e08cff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04130050 con 0x7f2e0409e000 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.315+0000 7f2e08cff700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04131a80 con 0x7f2e0409e410 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.315+0000 7f2e08cff700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04130e20 con 0x7f2e0409d6f0 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e0259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e0409d6f0 0x7f2e0412dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e0259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e0409d6f0 0x7f2e0412dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45166/0 (socket says 172.21.15.89:45166) 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e0259c700 1 -- 172.21.15.89:0/4198205874 learned_addr learned my addr 172.21.15.89:0/4198205874 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:56.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e01d9b700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 0x7f2e0412e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e02d9d700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409e410 0x7f2e0412e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e01d9b700 1 -- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e0409d6f0 msgr2=0x7f2e0412dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e01d9b700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e0409d6f0 0x7f2e0412dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e01d9b700 1 -- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409e410 msgr2=0x7f2e0412e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e01d9b700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409e410 0x7f2e0412e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e01d9b700 1 -- 172.21.15.89:0/4198205874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8009550 con 0x7f2e0409e000 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e01d9b700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 0x7f2e0412e260 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2df8016000 tx=0x7f2df8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2df2ffd700 1 -- 172.21.15.89:0/4198205874 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80036a0 con 0x7f2e0409e000 2023-09-30T22:19:56.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2df2ffd700 1 -- 172.21.15.89:0/4198205874 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df801f070 con 0x7f2e0409e000 2023-09-30T22:19:56.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2df2ffd700 1 -- 172.21.15.89:0/4198205874 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8003a00 con 0x7f2e0409e000 2023-09-30T22:19:56.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.316+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e040a1d00 con 0x7f2e0409e000 2023-09-30T22:19:56.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.317+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0412f780 con 0x7f2e0409e000 2023-09-30T22:19:56.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.317+0000 7f2df2ffd700 1 -- 172.21.15.89:0/4198205874 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2df8003b60 con 0x7f2e0409e000 2023-09-30T22:19:56.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.318+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e04074ea0 con 0x7f2e0409e000 2023-09-30T22:19:56.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.318+0000 7f2df2ffd700 1 -- 172.21.15.89:0/4198205874 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2df8021070 con 0x7f2e0409e000 2023-09-30T22:19:56.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.326+0000 7f2df2ffd700 1 -- 172.21.15.89:0/4198205874 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2df801a760 con 0x7f2e0409e000 2023-09-30T22:19:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.362+0000 7f04866cc700 1 Processor -- start 2023-09-30T22:19:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.362+0000 7f04866cc700 1 -- start start 2023-09-30T22:19:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f04866cc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f048006f5d0 0x7f048012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f04866cc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0480135c20 0x7f0480138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f04866cc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048006fa40 con 0x7f048012cf60 2023-09-30T22:19:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f04866cc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048012cd90 con 0x7f048006f5d0 2023-09-30T22:19:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f04866cc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0480071780 con 0x7f0480135c20 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f047f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f048006f5d0 0x7f048012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f047f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f048006f5d0 0x7f048012ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53316/0 (socket says 172.21.15.89:53316) 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f047f7fe700 1 -- 172.21.15.89:0/2235061143 learned_addr learned my addr 172.21.15.89:0/2235061143 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f047f7fe700 1 -- 172.21.15.89:0/2235061143 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0480135c20 msgr2=0x7f0480138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f047f7fe700 1 --2- 172.21.15.89:0/2235061143 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0480135c20 0x7f0480138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f047f7fe700 1 -- 172.21.15.89:0/2235061143 >> v1:172.21.15.89:6789/0 conn(0x7f048012cf60 legacy=0x7f048012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f047f7fe700 1 -- 172.21.15.89:0/2235061143 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480071a40 con 0x7f048006f5d0 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.363+0000 7f047f7fe700 1 --2- 172.21.15.89:0/2235061143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048006f5d0 0x7f048012ffc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f04740098f0 tx=0x7f0474009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deafa3dabfa1e337 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f047effd700 1 -- 172.21.15.89:0/2235061143 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047400b8a0 con 0x7f048006f5d0 2023-09-30T22:19:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f047effd700 1 -- 172.21.15.89:0/2235061143 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047400ba00 con 0x7f048006f5d0 2023-09-30T22:19:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f047effd700 1 -- 172.21.15.89:0/2235061143 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047400bd90 con 0x7f048006f5d0 2023-09-30T22:19:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f04866cc700 1 -- 172.21.15.89:0/2235061143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048006f5d0 msgr2=0x7f048012ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f04866cc700 1 --2- 172.21.15.89:0/2235061143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048006f5d0 0x7f048012ffc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f04740098f0 tx=0x7f0474009c00 comp rx=0 tx=0).stop 2023-09-30T22:19:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f04866cc700 1 -- 172.21.15.89:0/2235061143 shutdown_connections 2023-09-30T22:19:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f04866cc700 1 --2- 172.21.15.89:0/2235061143 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0480135c20 0x7f0480138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f04866cc700 1 --2- 172.21.15.89:0/2235061143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048006f5d0 0x7f048012ffc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f04866cc700 1 -- 172.21.15.89:0/2235061143 >> 172.21.15.89:0/2235061143 conn(0x7f04800a2aa0 msgr2=0x7f04800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f04866cc700 1 -- 172.21.15.89:0/2235061143 shutdown_connections 2023-09-30T22:19:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.364+0000 7f04866cc700 1 -- 172.21.15.89:0/2235061143 wait complete. 2023-09-30T22:19:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.365+0000 7f74d3301700 1 Processor -- start 2023-09-30T22:19:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.365+0000 7f74d3301700 1 -- start start 2023-09-30T22:19:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f04866cc700 1 Processor -- start 2023-09-30T22:19:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d3301700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12cf60 0x7f74cc06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d3301700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc12f8b0 0x7f74cc12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d3301700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f74cc130260 0x7f74cc136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d3301700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc12d330 con 0x7f74cc130260 2023-09-30T22:19:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d3301700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc12cd90 con 0x7f74cc12f8b0 2023-09-30T22:19:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d3301700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1306d0 con 0x7f74cc12cf60 2023-09-30T22:19:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d089c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc12f8b0 0x7f74cc12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d089c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc12f8b0 0x7f74cc12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42148/0 (socket says 172.21.15.89:42148) 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d089c700 1 -- 172.21.15.89:0/1226879845 learned_addr learned my addr 172.21.15.89:0/1226879845 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d189e700 1 --2- 172.21.15.89:0/1226879845 >> v2:172.21.15.89:3300/0 conn(0x7f74cc130260 0x7f74cc136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d189e700 1 -- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12cf60 msgr2=0x7f74cc06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d189e700 1 --2- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12cf60 0x7f74cc06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d189e700 1 -- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc12f8b0 msgr2=0x7f74cc12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d189e700 1 --2- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc12f8b0 0x7f74cc12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d189e700 1 -- 172.21.15.89:0/1226879845 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc071780 con 0x7f74cc130260 2023-09-30T22:19:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f74d089c700 1 --2- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc12f8b0 0x7f74cc12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.366+0000 7f04866cc700 1 -- start start 2023-09-30T22:19:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f04866cc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048006f5d0 0x7f0480099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f04866cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048012cf60 0x7f048009a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f04866cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0480135c20 0x7f048009a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f04866cc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048009aec0 con 0x7f048012cf60 2023-09-30T22:19:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f04866cc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048009caf0 con 0x7f048006f5d0 2023-09-30T22:19:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f04866cc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f048009c830 con 0x7f0480135c20 2023-09-30T22:19:56.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048006f5d0 0x7f0480099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048012cf60 0x7f048009a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048006f5d0 0x7f0480099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42160/0 (socket says 172.21.15.89:42160) 2023-09-30T22:19:56.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 -- 172.21.15.89:0/1606391650 learned_addr learned my addr 172.21.15.89:0/1606391650 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:56.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f74d189e700 1 --2- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc130260 0x7f74cc136c20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009d90 tx=0x7f74c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a9f7996ad78f37 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:56.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 -- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0480135c20 msgr2=0x7f048009a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 --2- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0480135c20 0x7f048009a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 -- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048012cf60 msgr2=0x7f048009a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 --2- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048012cf60 0x7f048009a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 -- 172.21.15.89:0/1606391650 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0474009550 con 0x7f048006f5d0 2023-09-30T22:19:56.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047ffff700 1 --2- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048006f5d0 0x7f0480099d50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f04700014a0 tx=0x7f04700017b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:56.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047cff9700 1 -- 172.21.15.89:0/1606391650 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04700050d0 con 0x7f048006f5d0 2023-09-30T22:19:56.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047cff9700 1 -- 172.21.15.89:0/1606391650 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0470005230 con 0x7f048006f5d0 2023-09-30T22:19:56.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f74c3fff700 1 -- 172.21.15.89:0/1226879845 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c400e070 con 0x7f74cc130260 2023-09-30T22:19:56.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f74c3fff700 1 -- 172.21.15.89:0/1226879845 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c400bb20 con 0x7f74cc130260 2023-09-30T22:19:56.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f74c3fff700 1 -- 172.21.15.89:0/1226879845 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4012430 con 0x7f74cc130260 2023-09-30T22:19:56.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048009ed00 con 0x7f048006f5d0 2023-09-30T22:19:56.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.367+0000 7f047cff9700 1 -- 172.21.15.89:0/1606391650 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000f690 con 0x7f048006f5d0 2023-09-30T22:19:56.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.368+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0480135270 con 0x7f048006f5d0 2023-09-30T22:19:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.368+0000 7f74d3301700 1 -- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc130260 msgr2=0x7f74cc136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.368+0000 7f74d3301700 1 --2- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc130260 0x7f74cc136c20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009d90 tx=0x7f74c400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.368+0000 7f74d3301700 1 -- 172.21.15.89:0/1226879845 shutdown_connections 2023-09-30T22:19:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.368+0000 7f74d3301700 1 --2- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc130260 0x7f74cc136c20 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.368+0000 7f74d3301700 1 --2- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12cf60 0x7f74cc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.368+0000 7f74d3301700 1 --2- 172.21.15.89:0/1226879845 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc12f8b0 0x7f74cc12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.368+0000 7f74d3301700 1 -- 172.21.15.89:0/1226879845 >> 172.21.15.89:0/1226879845 conn(0x7f74cc0a2aa0 msgr2=0x7f74cc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.369+0000 7f74d3301700 1 -- 172.21.15.89:0/1226879845 shutdown_connections 2023-09-30T22:19:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.369+0000 7f74d3301700 1 -- 172.21.15.89:0/1226879845 wait complete. 2023-09-30T22:19:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.369+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0480074ea0 con 0x7f048006f5d0 2023-09-30T22:19:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.369+0000 7f047cff9700 1 -- 172.21.15.89:0/1606391650 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f047000f7f0 con 0x7f048006f5d0 2023-09-30T22:19:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.369+0000 7f047cff9700 1 -- 172.21.15.89:0/1606391650 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f047003f810 con 0x7f048006f5d0 2023-09-30T22:19:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d3301700 1 Processor -- start 2023-09-30T22:19:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d3301700 1 -- start start 2023-09-30T22:19:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d3301700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc12cf60 0x7f74cc1309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d3301700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12f8b0 0x7f74cc130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d3301700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc130260 0x7f74cc131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d3301700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc133d10 con 0x7f74cc12cf60 2023-09-30T22:19:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d3301700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc129fa0 con 0x7f74cc130260 2023-09-30T22:19:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d3301700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc133050 con 0x7f74cc12f8b0 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d189e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc130260 0x7f74cc131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d189e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc130260 0x7f74cc131460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42166/0 (socket says 172.21.15.89:42166) 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d189e700 1 -- 172.21.15.89:0/2189926958 learned_addr learned my addr 172.21.15.89:0/2189926958 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.370+0000 7f74d089c700 1 --2- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12f8b0 0x7f74cc130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.371+0000 7f74d189e700 1 -- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12f8b0 msgr2=0x7f74cc130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.371+0000 7f74d189e700 1 --2- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12f8b0 0x7f74cc130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.371+0000 7f74d189e700 1 -- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc12cf60 msgr2=0x7f74cc1309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.371+0000 7f74d189e700 1 --2- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc12cf60 0x7f74cc1309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.371+0000 7f74d189e700 1 -- 172.21.15.89:0/2189926958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009550 con 0x7f74cc130260 2023-09-30T22:19:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.371+0000 7f74d189e700 1 --2- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc130260 0x7f74cc131460 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f74c4014040 tx=0x7f74c4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.374+0000 7f74c1ffb700 1 -- 172.21.15.89:0/2189926958 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4012430 con 0x7f74cc130260 2023-09-30T22:19:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.374+0000 7f74c1ffb700 1 -- 172.21.15.89:0/2189926958 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c4020070 con 0x7f74cc130260 2023-09-30T22:19:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.374+0000 7f74c1ffb700 1 -- 172.21.15.89:0/2189926958 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4003ea0 con 0x7f74cc130260 2023-09-30T22:19:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.374+0000 7f047cff9700 1 -- 172.21.15.89:0/1606391650 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f047000faa0 con 0x7f048006f5d0 2023-09-30T22:19:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.375+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c4009a70 con 0x7f74cc130260 2023-09-30T22:19:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.375+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc09ea50 con 0x7f74cc130260 2023-09-30T22:19:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.376+0000 7f74c1ffb700 1 -- 172.21.15.89:0/2189926958 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74c4004280 con 0x7f74cc130260 2023-09-30T22:19:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.376+0000 7f74c1ffb700 1 -- 172.21.15.89:0/2189926958 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f74c4022070 con 0x7f74cc130260 2023-09-30T22:19:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.376+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc0679c0 con 0x7f74cc130260 2023-09-30T22:19:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.385+0000 7f74c1ffb700 1 -- 172.21.15.89:0/2189926958 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74c4004530 con 0x7f74cc130260 2023-09-30T22:19:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.456+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f982c06b9e0 con 0x7f982c1338a0 2023-09-30T22:19:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.457+0000 7f9821ffb700 1 -- 172.21.15.89:0/3533412989 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98240198f0 con 0x7f982c1338a0 2023-09-30T22:19:56.458 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c1338a0 msgr2=0x7f982c11f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 --2- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c1338a0 0x7f982c11f480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f982400cc50 tx=0x7f982400cf60 comp rx=0 tx=0).stop 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 shutdown_connections 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 --2- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f982c1338a0 0x7f982c11f480 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 --2- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c09de50 0x7f982c11ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 --2- 172.21.15.89:0/3533412989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c09e760 0x7f982c11ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 >> 172.21.15.89:0/3533412989 conn(0x7f982c0a2aa0 msgr2=0x7f982c09ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 shutdown_connections 2023-09-30T22:19:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.460+0000 7f98335cf700 1 -- 172.21.15.89:0/3533412989 wait complete. 2023-09-30T22:19:56.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:56.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.538+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2e0406c360 con 0x7f2e0409e000 2023-09-30T22:19:56.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.538+0000 7f2df2ffd700 1 -- 172.21.15.89:0/4198205874 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2df8015020 con 0x7f2e0409e000 2023-09-30T22:19:56.539 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 msgr2=0x7f2e0412e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 0x7f2e0412e260 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2df8016000 tx=0x7f2df8004750 comp rx=0 tx=0).stop 2023-09-30T22:19:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 shutdown_connections 2023-09-30T22:19:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e0409e410 0x7f2e0412e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e0409d6f0 0x7f2e0412dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 --2- 172.21.15.89:0/4198205874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e0409e000 0x7f2e0412e260 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 >> 172.21.15.89:0/4198205874 conn(0x7f2e04066f50 msgr2=0x7f2e040a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 shutdown_connections 2023-09-30T22:19:56.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.540+0000 7f2e08cff700 1 -- 172.21.15.89:0/4198205874 wait complete. 2023-09-30T22:19:56.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.592+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f04800a21d0 con 0x7f048006f5d0 2023-09-30T22:19:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.592+0000 7f047cff9700 1 -- 172.21.15.89:0/1606391650 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04700183a0 con 0x7f048006f5d0 2023-09-30T22:19:56.593 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048006f5d0 msgr2=0x7f0480099d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 --2- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048006f5d0 0x7f0480099d50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f04700014a0 tx=0x7f04700017b0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 shutdown_connections 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 --2- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0480135c20 0x7f048009a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 --2- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048006f5d0 0x7f0480099d50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 --2- 172.21.15.89:0/1606391650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048012cf60 0x7f048009a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 >> 172.21.15.89:0/1606391650 conn(0x7f04800a2aa0 msgr2=0x7f0480128fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 shutdown_connections 2023-09-30T22:19:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.595+0000 7f04866cc700 1 -- 172.21.15.89:0/1606391650 wait complete. 2023-09-30T22:19:56.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:56.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.613+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f74cc133290 con 0x7f74cc130260 2023-09-30T22:19:56.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.613+0000 7f74c1ffb700 1 -- 172.21.15.89:0/2189926958 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74c4036090 con 0x7f74cc130260 2023-09-30T22:19:56.614 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:56.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.615+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc130260 msgr2=0x7f74cc131460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.615+0000 7f74d3301700 1 --2- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc130260 0x7f74cc131460 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f74c4014040 tx=0x7f74c4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.616+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 shutdown_connections 2023-09-30T22:19:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.616+0000 7f74d3301700 1 --2- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc130260 0x7f74cc131460 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.616+0000 7f74d3301700 1 --2- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc12cf60 0x7f74cc1309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.616+0000 7f74d3301700 1 --2- 172.21.15.89:0/2189926958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc12f8b0 0x7f74cc130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.616+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 >> 172.21.15.89:0/2189926958 conn(0x7f74cc0a2aa0 msgr2=0x7f74cc0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.616+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 shutdown_connections 2023-09-30T22:19:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:56.616+0000 7f74d3301700 1 -- 172.21.15.89:0/2189926958 wait complete. 2023-09-30T22:19:56.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:57.479 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:57.554 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:57.608 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:57.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22de886700 1 Processor -- start 2023-09-30T22:19:57.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22de886700 1 -- start start 2023-09-30T22:19:57.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22de886700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d8134630 0x7f22d8136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22de886700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8136f50 0x7f22d8139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22de886700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d809b4f0 con 0x7f22d8131810 2023-09-30T22:19:57.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22de886700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81313c0 con 0x7f22d8136f50 2023-09-30T22:19:57.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22de886700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22d806e8c0 con 0x7f22d8134630 2023-09-30T22:19:57.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22d77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d8134630 0x7f22d8136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22dce23700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8136f50 0x7f22d8139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22d77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d8134630 0x7f22d8136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45192/0 (socket says 172.21.15.89:45192) 2023-09-30T22:19:57.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.617+0000 7f22d77fe700 1 -- 172.21.15.89:0/3158226234 learned_addr learned my addr 172.21.15.89:0/3158226234 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:57.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d77fe700 1 -- 172.21.15.89:0/3158226234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8136f50 msgr2=0x7f22d8139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:57.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d77fe700 1 --2- 172.21.15.89:0/3158226234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8136f50 0x7f22d8139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d77fe700 1 -- 172.21.15.89:0/3158226234 >> v1:172.21.15.89:6789/0 conn(0x7f22d8131810 legacy=0x7f22d8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d77fe700 1 -- 172.21.15.89:0/3158226234 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8071d20 con 0x7f22d8134630 2023-09-30T22:19:57.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d6ffd700 1 -- 172.21.15.89:0/3158226234 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427547810 0 0) 0x7f22d809b4f0 con 0x7f22d8131810 2023-09-30T22:19:57.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d77fe700 1 --2- 172.21.15.89:0/3158226234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d8134630 0x7f22d8136a10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009920 tx=0x7f22cc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b02cd082dbaa10c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:57.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d6ffd700 1 -- 172.21.15.89:0/3158226234 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc011830 con 0x7f22d8134630 2023-09-30T22:19:57.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d6ffd700 1 -- 172.21.15.89:0/3158226234 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc011990 con 0x7f22d8134630 2023-09-30T22:19:57.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.618+0000 7f22d6ffd700 1 -- 172.21.15.89:0/3158226234 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc011d20 con 0x7f22d8134630 2023-09-30T22:19:57.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.619+0000 7f22de886700 1 -- 172.21.15.89:0/3158226234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d8134630 msgr2=0x7f22d8136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.619+0000 7f22de886700 1 --2- 172.21.15.89:0/3158226234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d8134630 0x7f22d8136a10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009920 tx=0x7f22cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:57.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.619+0000 7f22de886700 1 -- 172.21.15.89:0/3158226234 shutdown_connections 2023-09-30T22:19:57.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.619+0000 7f22de886700 1 --2- 172.21.15.89:0/3158226234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8136f50 0x7f22d8139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.619+0000 7f22de886700 1 --2- 172.21.15.89:0/3158226234 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d8134630 0x7f22d8136a10 secure :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009920 tx=0x7f22cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:57.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.619+0000 7f22de886700 1 -- 172.21.15.89:0/3158226234 >> 172.21.15.89:0/3158226234 conn(0x7f22d80a2aa0 msgr2=0x7f22d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:57.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.619+0000 7f22de886700 1 -- 172.21.15.89:0/3158226234 shutdown_connections 2023-09-30T22:19:57.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.619+0000 7f22de886700 1 -- 172.21.15.89:0/3158226234 wait complete. 2023-09-30T22:19:57.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.620+0000 7f22de886700 1 Processor -- start 2023-09-30T22:19:57.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.620+0000 7f22de886700 1 -- start start 2023-09-30T22:19:57.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.620+0000 7f22de886700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8131810 0x7f22d8070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22de886700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d8136f50 0x7f22d8070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22de886700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d81c8470 0x7f22d81c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22de886700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d811ea00 con 0x7f22d8136f50 2023-09-30T22:19:57.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22de886700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d806fb30 con 0x7f22d8131810 2023-09-30T22:19:57.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22de886700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22d812ecb0 con 0x7f22d81c8470 2023-09-30T22:19:57.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d81c8470 0x7f22d81c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d81c8470 0x7f22d81c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45196/0 (socket says 172.21.15.89:45196) 2023-09-30T22:19:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8131810 0x7f22d8070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 -- 172.21.15.89:0/1278108740 learned_addr learned my addr 172.21.15.89:0/1278108740 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22d77fe700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d8136f50 0x7f22d8070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 -- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8131810 msgr2=0x7f22d8070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8131810 0x7f22d8070030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 -- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d8136f50 msgr2=0x7f22d8070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d8136f50 0x7f22d8070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 -- 172.21.15.89:0/1278108740 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc009550 con 0x7f22d81c8470 2023-09-30T22:19:57.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22d7fff700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8131810 0x7f22d8070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:57.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22d77fe700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d8136f50 0x7f22d8070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:57.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22dce23700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d81c8470 0x7f22d81c88e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009e30 tx=0x7f22d000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:57.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.621+0000 7f22d4ff9700 1 -- 172.21.15.89:0/1278108740 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0009ca0 con 0x7f22d81c8470 2023-09-30T22:19:57.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.622+0000 7f22d4ff9700 1 -- 172.21.15.89:0/1278108740 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d000f040 con 0x7f22d81c8470 2023-09-30T22:19:57.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.622+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d811ec00 con 0x7f22d81c8470 2023-09-30T22:19:57.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.622+0000 7f22d4ff9700 1 -- 172.21.15.89:0/1278108740 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000ced0 con 0x7f22d81c8470 2023-09-30T22:19:57.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.622+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d8133950 con 0x7f22d81c8470 2023-09-30T22:19:57.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.623+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d806eb00 con 0x7f22d81c8470 2023-09-30T22:19:57.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.623+0000 7f22d4ff9700 1 -- 172.21.15.89:0/1278108740 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f22d0014480 con 0x7f22d81c8470 2023-09-30T22:19:57.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.623+0000 7f22d4ff9700 1 -- 172.21.15.89:0/1278108740 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f22d0041810 con 0x7f22d81c8470 2023-09-30T22:19:57.631 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:57.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.628+0000 7f22d4ff9700 1 -- 172.21.15.89:0/1278108740 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f22d0045060 con 0x7f22d81c8470 2023-09-30T22:19:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.687+0000 7ffa51f0e700 1 Processor -- start 2023-09-30T22:19:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.687+0000 7ffa51f0e700 1 -- start start 2023-09-30T22:19:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa51f0e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa51f0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c136d80 0x7ffa4c139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa51f0e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c09ece0 con 0x7ffa4c06a770 2023-09-30T22:19:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa51f0e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c06b640 con 0x7ffa4c06b250 2023-09-30T22:19:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa51f0e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c09e760 con 0x7ffa4c136d80 2023-09-30T22:19:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ffa4c06a770 0x7ffa4c06ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53626/0 (socket says 172.21.15.89:53626) 2023-09-30T22:19:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4b7fe700 1 -- 172.21.15.89:0/345164311 learned_addr learned my addr 172.21.15.89:0/345164311 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4bfff700 1 --2- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c136d80 0x7ffa4c139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4affd700 1 -- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c136d80 msgr2=0x7ffa4c139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4affd700 1 --2- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c136d80 0x7ffa4c139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4affd700 1 -- 172.21.15.89:0/345164311 >> v1:172.21.15.89:6789/0 conn(0x7ffa4c06a770 legacy=0x7ffa4c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4affd700 1 -- 172.21.15.89:0/345164311 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c09e040 con 0x7ffa4c06b250 2023-09-30T22:19:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4a7fc700 1 -- 172.21.15.89:0/345164311 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545192716 0 0) 0x7ffa4c09ece0 con 0x7ffa4c06a770 2023-09-30T22:19:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4bfff700 1 --2- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c136d80 0x7ffa4c139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.688+0000 7ffa4affd700 1 --2- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c136840 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009870 tx=0x7ffa40009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce06d56c8d339aaf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.689+0000 7ffa4a7fc700 1 -- 172.21.15.89:0/345164311 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40011830 con 0x7ffa4c06b250 2023-09-30T22:19:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.689+0000 7ffa4a7fc700 1 -- 172.21.15.89:0/345164311 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa40011990 con 0x7ffa4c06b250 2023-09-30T22:19:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.689+0000 7ffa4a7fc700 1 -- 172.21.15.89:0/345164311 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40011d20 con 0x7ffa4c06b250 2023-09-30T22:19:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.689+0000 7ffa51f0e700 1 -- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 msgr2=0x7ffa4c136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.689+0000 7ffa51f0e700 1 --2- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c136840 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009870 tx=0x7ffa40009b80 comp rx=0 tx=0).stop 2023-09-30T22:19:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.690+0000 7ffa51f0e700 1 -- 172.21.15.89:0/345164311 shutdown_connections 2023-09-30T22:19:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.690+0000 7ffa51f0e700 1 --2- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c136d80 0x7ffa4c139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.690+0000 7ffa51f0e700 1 --2- 172.21.15.89:0/345164311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c136840 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.690+0000 7ffa51f0e700 1 -- 172.21.15.89:0/345164311 >> 172.21.15.89:0/345164311 conn(0x7ffa4c066f50 msgr2=0x7ffa4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.690+0000 7ffa51f0e700 1 -- 172.21.15.89:0/345164311 shutdown_connections 2023-09-30T22:19:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.690+0000 7ffa51f0e700 1 -- 172.21.15.89:0/345164311 wait complete. 2023-09-30T22:19:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa51f0e700 1 Processor -- start 2023-09-30T22:19:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa51f0e700 1 -- start start 2023-09-30T22:19:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa51f0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c06a770 0x7ffa4c11b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa51f0e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c11b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa51f0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa4c136d80 0x7ffa4c11bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa51f0e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c11d030 con 0x7ffa4c136d80 2023-09-30T22:19:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa51f0e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c11dad0 con 0x7ffa4c06b250 2023-09-30T22:19:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa51f0e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c11e280 con 0x7ffa4c06a770 2023-09-30T22:19:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa4b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c06a770 0x7ffa4c11b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa4b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c06a770 0x7ffa4c11b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45222/0 (socket says 172.21.15.89:45222) 2023-09-30T22:19:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa4b7fe700 1 -- 172.21.15.89:0/2387407208 learned_addr learned my addr 172.21.15.89:0/2387407208 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.691+0000 7ffa4affd700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c11b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa4bfff700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa4c136d80 0x7ffa4c11bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa4affd700 1 -- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c06a770 msgr2=0x7ffa4c11b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa4affd700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c06a770 0x7ffa4c11b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa4affd700 1 -- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa4c136d80 msgr2=0x7ffa4c11bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa4affd700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa4c136d80 0x7ffa4c11bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa4affd700 1 -- 172.21.15.89:0/2387407208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4400f050 con 0x7ffa4c06b250 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa4b7fe700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c06a770 0x7ffa4c11b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa4affd700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c11b9e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c09d2a0 tx=0x7ffa40004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa33fff700 1 -- 172.21.15.89:0/2387407208 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40004590 con 0x7ffa4c06b250 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa33fff700 1 -- 172.21.15.89:0/2387407208 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4001f070 con 0x7ffa4c06b250 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa33fff700 1 -- 172.21.15.89:0/2387407208 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa400038c0 con 0x7ffa4c06b250 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa400094a0 con 0x7ffa4c06b250 2023-09-30T22:19:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.692+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c071e60 con 0x7ffa4c06b250 2023-09-30T22:19:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.693+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c074ea0 con 0x7ffa4c06b250 2023-09-30T22:19:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.694+0000 7ffa33fff700 1 -- 172.21.15.89:0/2387407208 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa40003a20 con 0x7ffa4c06b250 2023-09-30T22:19:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.695+0000 7ffa33fff700 1 -- 172.21.15.89:0/2387407208 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffa40021070 con 0x7ffa4c06b250 2023-09-30T22:19:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.698+0000 7ffa33fff700 1 -- 172.21.15.89:0/2387407208 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa400253e0 con 0x7ffa4c06b250 2023-09-30T22:19:57.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.738+0000 7f7f80452700 1 Processor -- start 2023-09-30T22:19:57.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.738+0000 7f7f80452700 1 -- start start 2023-09-30T22:19:57.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f80452700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f7809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f80452700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f80452700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7809f130 con 0x7f7f7809e410 2023-09-30T22:19:57.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f80452700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7809f520 con 0x7f7f7809e000 2023-09-30T22:19:57.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f80452700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7809e930 con 0x7f7f7809d6f0 2023-09-30T22:19:57.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7e9ef700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7f7809e410 0x7f7f78130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53630/0 (socket says 172.21.15.89:53630) 2023-09-30T22:19:57.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7e9ef700 1 -- 172.21.15.89:0/3508427513 learned_addr learned my addr 172.21.15.89:0/3508427513 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:57.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7e1ee700 1 --2- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f7809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7d9ed700 1 --2- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7e1ee700 1 -- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 msgr2=0x7f7f78130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:57.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7e1ee700 1 --2- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7e1ee700 1 -- 172.21.15.89:0/3508427513 >> v1:172.21.15.89:6789/0 conn(0x7f7f7809e410 legacy=0x7f7f78130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7d1ec700 1 -- 172.21.15.89:0/3508427513 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276635351 0 0) 0x7f7f7809f130 con 0x7f7f7809e410 2023-09-30T22:19:57.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7e1ee700 1 -- 172.21.15.89:0/3508427513 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7809b2d0 con 0x7f7f7809d6f0 2023-09-30T22:19:57.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.739+0000 7f7f7d9ed700 1 --2- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:57.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.740+0000 7f7f7e1ee700 1 --2- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f7809dac0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7f740098a0 tx=0x7f7f74009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a48c87f27a331993 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:57.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.740+0000 7f7f7d1ec700 1 -- 172.21.15.89:0/3508427513 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74011830 con 0x7f7f7809d6f0 2023-09-30T22:19:57.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.740+0000 7f7f7d1ec700 1 -- 172.21.15.89:0/3508427513 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f74011990 con 0x7f7f7809d6f0 2023-09-30T22:19:57.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.740+0000 7f7f7d1ec700 1 -- 172.21.15.89:0/3508427513 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74011d20 con 0x7f7f7809d6f0 2023-09-30T22:19:57.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.741+0000 7f7f80452700 1 -- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 msgr2=0x7f7f7809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.741+0000 7f7f80452700 1 --2- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f7809dac0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7f740098a0 tx=0x7f7f74009bb0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.741+0000 7f7f80452700 1 -- 172.21.15.89:0/3508427513 shutdown_connections 2023-09-30T22:19:57.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.741+0000 7f7f80452700 1 --2- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f7809dac0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.741+0000 7f7f80452700 1 --2- 172.21.15.89:0/3508427513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.741+0000 7f7f80452700 1 -- 172.21.15.89:0/3508427513 >> 172.21.15.89:0/3508427513 conn(0x7f7f78066f50 msgr2=0x7f7f7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:57.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.741+0000 7f7f80452700 1 -- 172.21.15.89:0/3508427513 shutdown_connections 2023-09-30T22:19:57.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.741+0000 7f7f80452700 1 -- 172.21.15.89:0/3508427513 wait complete. 2023-09-30T22:19:57.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f80452700 1 Processor -- start 2023-09-30T22:19:57.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f80452700 1 -- start start 2023-09-30T22:19:57.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f80452700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f78122220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f80452700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78122760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f80452700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7809e410 0x7f7f78122ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f80452700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78124450 con 0x7f7f7809e410 2023-09-30T22:19:57.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f80452700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7809c770 con 0x7f7f7809e000 2023-09-30T22:19:57.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f80452700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78123660 con 0x7f7f7809d6f0 2023-09-30T22:19:57.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f7d9ed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78122760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f7d9ed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78122760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42218/0 (socket says 172.21.15.89:42218) 2023-09-30T22:19:57.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.742+0000 7f7f7d9ed700 1 -- 172.21.15.89:0/2868743608 learned_addr learned my addr 172.21.15.89:0/2868743608 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:57.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7e1ee700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f78122220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7e9ef700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7809e410 0x7f7f78122ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7e1ee700 1 -- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 msgr2=0x7f7f78122760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7e1ee700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78122760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7e1ee700 1 -- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7809e410 msgr2=0x7f7f78122ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7e1ee700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7809e410 0x7f7f78122ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7e1ee700 1 -- 172.21.15.89:0/2868743608 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680094a0 con 0x7f7f7809d6f0 2023-09-30T22:19:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7d9ed700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78122760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f7e1ee700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f78122220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7f74016000 tx=0x7f7f74004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f6effd700 1 -- 172.21.15.89:0/2868743608 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74004450 con 0x7f7f7809d6f0 2023-09-30T22:19:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f6effd700 1 -- 172.21.15.89:0/2868743608 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7401f070 con 0x7f7f7809d6f0 2023-09-30T22:19:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f740094a0 con 0x7f7f7809d6f0 2023-09-30T22:19:57.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f6effd700 1 -- 172.21.15.89:0/2868743608 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f740038e0 con 0x7f7f7809d6f0 2023-09-30T22:19:57.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.743+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f78125a60 con 0x7f7f7809d6f0 2023-09-30T22:19:57.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.744+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f78074ea0 con 0x7f7f7809d6f0 2023-09-30T22:19:57.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.745+0000 7f7f6effd700 1 -- 172.21.15.89:0/2868743608 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f74003a40 con 0x7f7f7809d6f0 2023-09-30T22:19:57.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.745+0000 7f7f6effd700 1 -- 172.21.15.89:0/2868743608 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7f74021070 con 0x7f7f7809d6f0 2023-09-30T22:19:57.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.749+0000 7f7f6effd700 1 -- 172.21.15.89:0/2868743608 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f740045b0 con 0x7f7f7809d6f0 2023-09-30T22:19:57.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.791+0000 7f672b39a700 1 Processor -- start 2023-09-30T22:19:57.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.791+0000 7f672b39a700 1 -- start start 2023-09-30T22:19:57.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f672b39a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67241337f0 0x7f6724138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f672b39a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6724138dc0 0x7f672413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f672b39a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672406b990 con 0x7f6724133420 2023-09-30T22:19:57.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f672b39a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672409b0e0 con 0x7f6724138dc0 2023-09-30T22:19:57.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f672b39a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6724133c20 con 0x7f67241337f0 2023-09-30T22:19:57.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6729937700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6724138dc0 0x7f672413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6729937700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6724138dc0 0x7f672413b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53396/0 (socket says 172.21.15.89:53396) 2023-09-30T22:19:57.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6729937700 1 -- 172.21.15.89:0/1685698808 learned_addr learned my addr 172.21.15.89:0/1685698808 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:57.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6728935700 1 --2- 172.21.15.89:0/1685698808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67241337f0 0x7f6724138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6729937700 1 -- 172.21.15.89:0/1685698808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67241337f0 msgr2=0x7f6724138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:57.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6729937700 1 --2- 172.21.15.89:0/1685698808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67241337f0 0x7f6724138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6729937700 1 -- 172.21.15.89:0/1685698808 >> v1:172.21.15.89:6789/0 conn(0x7f6724133420 legacy=0x7f6724133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6729937700 1 -- 172.21.15.89:0/1685698808 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240a0590 con 0x7f6724138dc0 2023-09-30T22:19:57.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.792+0000 7f6729937700 1 --2- 172.21.15.89:0/1685698808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724138dc0 0x7f672413b1a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f671c009e50 tx=0x7f671c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1656728cb506d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:57.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.793+0000 7f671bfff700 1 -- 172.21.15.89:0/1685698808 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c00e070 con 0x7f6724138dc0 2023-09-30T22:19:57.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.793+0000 7f671bfff700 1 -- 172.21.15.89:0/1685698808 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c00b910 con 0x7f6724138dc0 2023-09-30T22:19:57.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.793+0000 7f671bfff700 1 -- 172.21.15.89:0/1685698808 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c00bcc0 con 0x7f6724138dc0 2023-09-30T22:19:57.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.793+0000 7f672b39a700 1 -- 172.21.15.89:0/1685698808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724138dc0 msgr2=0x7f672413b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.793+0000 7f672b39a700 1 --2- 172.21.15.89:0/1685698808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724138dc0 0x7f672413b1a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f671c009e50 tx=0x7f671c00b330 comp rx=0 tx=0).stop 2023-09-30T22:19:57.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.794+0000 7f672b39a700 1 -- 172.21.15.89:0/1685698808 shutdown_connections 2023-09-30T22:19:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.794+0000 7f672b39a700 1 --2- 172.21.15.89:0/1685698808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724138dc0 0x7f672413b1a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.794+0000 7f672b39a700 1 --2- 172.21.15.89:0/1685698808 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67241337f0 0x7f6724138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.794+0000 7f672b39a700 1 -- 172.21.15.89:0/1685698808 >> 172.21.15.89:0/1685698808 conn(0x7f67240a4210 msgr2=0x7f67240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.794+0000 7f672b39a700 1 -- 172.21.15.89:0/1685698808 shutdown_connections 2023-09-30T22:19:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.794+0000 7f672b39a700 1 -- 172.21.15.89:0/1685698808 wait complete. 2023-09-30T22:19:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.795+0000 7f672b39a700 1 Processor -- start 2023-09-30T22:19:57.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.795+0000 7f672b39a700 1 -- start start 2023-09-30T22:19:57.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f672b39a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724133420 0x7f67241c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f672b39a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67241337f0 0x7f67241c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f672b39a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6724138dc0 0x7f67241c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:57.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f672b39a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6724130390 con 0x7f6724133420 2023-09-30T22:19:57.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f672b39a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672409e1e0 con 0x7f67241337f0 2023-09-30T22:19:57.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f672b39a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f672409c8f0 con 0x7f6724138dc0 2023-09-30T22:19:57.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6724138dc0 0x7f67241c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6724138dc0 0x7f67241c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45246/0 (socket says 172.21.15.89:45246) 2023-09-30T22:19:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 -- 172.21.15.89:0/830153263 learned_addr learned my addr 172.21.15.89:0/830153263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729136700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724133420 0x7f67241c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6728935700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67241337f0 0x7f67241c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 -- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67241337f0 msgr2=0x7f67241c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67241337f0 0x7f67241c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 -- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724133420 msgr2=0x7f67241c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724133420 0x7f67241c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 -- 172.21.15.89:0/830153263 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009b30 con 0x7f6724138dc0 2023-09-30T22:19:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729136700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724133420 0x7f67241c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.796+0000 7f6729937700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6724138dc0 0x7f67241c90e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f671c012000 tx=0x7f671c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.797+0000 7f6719ffb700 1 -- 172.21.15.89:0/830153263 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0048b0 con 0x7f6724138dc0 2023-09-30T22:19:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.797+0000 7f6719ffb700 1 -- 172.21.15.89:0/830153263 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c0092e0 con 0x7f6724138dc0 2023-09-30T22:19:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.797+0000 7f6719ffb700 1 -- 172.21.15.89:0/830153263 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c01c660 con 0x7f6724138dc0 2023-09-30T22:19:57.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.797+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672409cbb0 con 0x7f6724138dc0 2023-09-30T22:19:57.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.797+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6724130b50 con 0x7f6724138dc0 2023-09-30T22:19:57.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.798+0000 7f6719ffb700 1 -- 172.21.15.89:0/830153263 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f671c01c7c0 con 0x7f6724138dc0 2023-09-30T22:19:57.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.799+0000 7f6719ffb700 1 -- 172.21.15.89:0/830153263 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f671c01cd50 con 0x7f6724138dc0 2023-09-30T22:19:57.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.799+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6724131e90 con 0x7f6724138dc0 2023-09-30T22:19:57.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.808+0000 7f6719ffb700 1 -- 172.21.15.89:0/830153263 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f671c046060 con 0x7f6724138dc0 2023-09-30T22:19:57.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.834+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f22d8074ea0 con 0x7f22d81c8470 2023-09-30T22:19:57.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.835+0000 7f22d4ff9700 1 -- 172.21.15.89:0/1278108740 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22d0018020 con 0x7f22d81c8470 2023-09-30T22:19:57.836 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d81c8470 msgr2=0x7f22d81c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d81c8470 0x7f22d81c88e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009e30 tx=0x7f22d000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 shutdown_connections 2023-09-30T22:19:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f22d81c8470 0x7f22d81c88e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f22d8131810 0x7f22d8070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 --2- 172.21.15.89:0/1278108740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d8136f50 0x7f22d8070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 >> 172.21.15.89:0/1278108740 conn(0x7f22d80a2aa0 msgr2=0x7f22d812f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 shutdown_connections 2023-09-30T22:19:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.837+0000 7f22de886700 1 -- 172.21.15.89:0/1278108740 wait complete. 2023-09-30T22:19:57.855 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:57.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.940+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa4c09f6b0 con 0x7ffa4c06b250 2023-09-30T22:19:57.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.940+0000 7ffa33fff700 1 -- 172.21.15.89:0/2387407208 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa4002b030 con 0x7ffa4c06b250 2023-09-30T22:19:57.941 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.943+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 msgr2=0x7ffa4c11b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.943+0000 7ffa51f0e700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c11b9e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c09d2a0 tx=0x7ffa40004750 comp rx=0 tx=0).stop 2023-09-30T22:19:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.943+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 shutdown_connections 2023-09-30T22:19:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.943+0000 7ffa51f0e700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa4c136d80 0x7ffa4c11bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.943+0000 7ffa51f0e700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa4c06a770 0x7ffa4c11b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.943+0000 7ffa51f0e700 1 --2- 172.21.15.89:0/2387407208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa4c06b250 0x7ffa4c11b9e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.943+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 >> 172.21.15.89:0/2387407208 conn(0x7ffa4c066f50 msgr2=0x7ffa4c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.944+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 shutdown_connections 2023-09-30T22:19:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.944+0000 7ffa51f0e700 1 -- 172.21.15.89:0/2387407208 wait complete. 2023-09-30T22:19:57.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.951+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7f78124620 con 0x7f7f7809d6f0 2023-09-30T22:19:57.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.952+0000 7f7f6effd700 1 -- 172.21.15.89:0/2868743608 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f7402b030 con 0x7f7f7809d6f0 2023-09-30T22:19:57.952 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:57.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 msgr2=0x7f7f78122220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:57.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f78122220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7f74016000 tx=0x7f7f74004750 comp rx=0 tx=0).stop 2023-09-30T22:19:57.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 shutdown_connections 2023-09-30T22:19:57.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7809e410 0x7f7f78122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7809d6f0 0x7f7f78122220 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 --2- 172.21.15.89:0/2868743608 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7809e000 0x7f7f78122760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:57.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 >> 172.21.15.89:0/2868743608 conn(0x7f7f78066f50 msgr2=0x7f7f78130a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:57.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 shutdown_connections 2023-09-30T22:19:57.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:57.954+0000 7f7f80452700 1 -- 172.21.15.89:0/2868743608 wait complete. 2023-09-30T22:19:57.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:57.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:58.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.005+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6724074ea0 con 0x7f6724138dc0 2023-09-30T22:19:58.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.005+0000 7f6719ffb700 1 -- 172.21.15.89:0/830153263 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f671c0123f0 con 0x7f6724138dc0 2023-09-30T22:19:58.006 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:58.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6724138dc0 msgr2=0x7f67241c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:58.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6724138dc0 0x7f67241c90e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f671c012000 tx=0x7f671c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 shutdown_connections 2023-09-30T22:19:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6724138dc0 0x7f67241c90e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6724133420 0x7f67241c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 --2- 172.21.15.89:0/830153263 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67241337f0 0x7f67241c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 >> 172.21.15.89:0/830153263 conn(0x7f67240a4210 msgr2=0x7f67240a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 shutdown_connections 2023-09-30T22:19:58.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.007+0000 7f672b39a700 1 -- 172.21.15.89:0/830153263 wait complete. 2023-09-30T22:19:58.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:19:58.855 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:19:58.958 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:19:58.968 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:19:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.978+0000 7ffbacb05700 1 Processor -- start 2023-09-30T22:19:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.978+0000 7ffbacb05700 1 -- start start 2023-09-30T22:19:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffbacb05700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffba809d6f0 0x7ffba809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffbacb05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e440 0x7ffba8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffbacb05700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba809b120 con 0x7ffba809e000 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffbacb05700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba806b460 con 0x7ffba809d6f0 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffbacb05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffba809a7e0 con 0x7ffba809e440 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffba809d6f0 0x7ffba809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e440 0x7ffba8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffba809d6f0 0x7ffba809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53404/0 (socket says 172.21.15.89:53404) 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba659c700 1 -- 172.21.15.89:0/2319176489 learned_addr learned my addr 172.21.15.89:0/2319176489 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba659c700 1 -- 172.21.15.89:0/2319176489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e440 msgr2=0x7ffba8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba659c700 1 --2- 172.21.15.89:0/2319176489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e440 0x7ffba8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba659c700 1 -- 172.21.15.89:0/2319176489 >> v1:172.21.15.89:6789/0 conn(0x7ffba809e000 legacy=0x7ffba81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba659c700 1 -- 172.21.15.89:0/2319176489 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba809a500 con 0x7ffba809d6f0 2023-09-30T22:19:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba559a700 1 -- 172.21.15.89:0/2319176489 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219939325 0 0) 0x7ffba809b120 con 0x7ffba809e000 2023-09-30T22:19:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba6d9d700 1 --2- 172.21.15.89:0/2319176489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e440 0x7ffba8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.979+0000 7ffba659c700 1 --2- 172.21.15.89:0/2319176489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809d6f0 0x7ffba809dac0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009920 tx=0x7ffb98009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac7bb2752846ec3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffba559a700 1 -- 172.21.15.89:0/2319176489 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98011830 con 0x7ffba809d6f0 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffba559a700 1 -- 172.21.15.89:0/2319176489 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98011990 con 0x7ffba809d6f0 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffba559a700 1 -- 172.21.15.89:0/2319176489 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98011d20 con 0x7ffba809d6f0 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffbacb05700 1 -- 172.21.15.89:0/2319176489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809d6f0 msgr2=0x7ffba809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffbacb05700 1 --2- 172.21.15.89:0/2319176489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809d6f0 0x7ffba809dac0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009920 tx=0x7ffb98009c30 comp rx=0 tx=0).stop 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffbacb05700 1 -- 172.21.15.89:0/2319176489 shutdown_connections 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffbacb05700 1 --2- 172.21.15.89:0/2319176489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809d6f0 0x7ffba809dac0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffbacb05700 1 --2- 172.21.15.89:0/2319176489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e440 0x7ffba8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.980+0000 7ffbacb05700 1 -- 172.21.15.89:0/2319176489 >> 172.21.15.89:0/2319176489 conn(0x7ffba8066f50 msgr2=0x7ffba809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.981+0000 7ffbacb05700 1 -- 172.21.15.89:0/2319176489 shutdown_connections 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.981+0000 7ffbacb05700 1 -- 172.21.15.89:0/2319176489 wait complete. 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.982+0000 7ffbacb05700 1 Processor -- start 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.982+0000 7ffbacb05700 1 -- start start 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.982+0000 7ffbacb05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffba809d6f0 0x7ffba81a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.982+0000 7ffbacb05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e000 0x7ffba81a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.982+0000 7ffbacb05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809e440 0x7ffba81c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.982+0000 7ffbacb05700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba81a8140 con 0x7ffba809e440 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.982+0000 7ffbacb05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba81a8320 con 0x7ffba809d6f0 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.982+0000 7ffbacb05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffba81b93c0 con 0x7ffba809e000 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffba809d6f0 0x7ffba81a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffba809d6f0 0x7ffba81a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:42244/0 (socket says 172.21.15.89:42244) 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809e440 0x7ffba81c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 -- 172.21.15.89:0/2524516602 learned_addr learned my addr 172.21.15.89:0/2524516602 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba5d9b700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e000 0x7ffba81a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 -- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e000 msgr2=0x7ffba81a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e000 0x7ffba81a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 -- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809e440 msgr2=0x7ffba81c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809e440 0x7ffba81c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 -- 172.21.15.89:0/2524516602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba0009a70 con 0x7ffba809d6f0 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba6d9d700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809e440 0x7ffba81c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffba659c700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffba809d6f0 0x7ffba81a5380 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ffb98000c00 tx=0x7ffb98004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffb96ffd700 1 -- 172.21.15.89:0/2524516602 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980042c0 con 0x7ffba809d6f0 2023-09-30T22:19:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb98009550 con 0x7ffba809d6f0 2023-09-30T22:19:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.983+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba809c710 con 0x7ffba809d6f0 2023-09-30T22:19:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.984+0000 7ffb96ffd700 1 -- 172.21.15.89:0/2524516602 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9801f070 con 0x7ffba809d6f0 2023-09-30T22:19:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.984+0000 7ffb96ffd700 1 -- 172.21.15.89:0/2524516602 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980036a0 con 0x7ffba809d6f0 2023-09-30T22:19:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.984+0000 7ffb96ffd700 1 -- 172.21.15.89:0/2524516602 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb98021030 con 0x7ffba809d6f0 2023-09-30T22:19:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.985+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c0052e0 con 0x7ffba809d6f0 2023-09-30T22:19:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.985+0000 7ffb96ffd700 1 -- 172.21.15.89:0/2524516602 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffb98040a60 con 0x7ffba809d6f0 2023-09-30T22:19:58.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:58.989+0000 7ffb96ffd700 1 -- 172.21.15.89:0/2524516602 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb980199a0 con 0x7ffba809d6f0 2023-09-30T22:19:59.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:19:59.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.087+0000 7f4942612700 1 Processor -- start 2023-09-30T22:19:59.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.087+0000 7f4942612700 1 -- start start 2023-09-30T22:19:59.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.087+0000 7f4942612700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c12cf60 0x7f493c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.087+0000 7f4942612700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f493c12f8b0 0x7f493c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.087+0000 7f4942612700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c12cd90 con 0x7f493c130260 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.087+0000 7f4942612700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c12d330 con 0x7f493c12f8b0 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.087+0000 7f4942612700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f493c1306e0 con 0x7f493c12cf60 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.087+0000 7f493b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f493c12f8b0 0x7f493c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f493bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c12cf60 0x7f493c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f4940baf700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f493c130260 0x7f493c136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53652/0 (socket says 172.21.15.89:53652) 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f493b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f493c12f8b0 0x7f493c12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53418/0 (socket says 172.21.15.89:53418) 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f4940baf700 1 -- 172.21.15.89:0/1209919405 learned_addr learned my addr 172.21.15.89:0/1209919405 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f493bfff700 1 -- 172.21.15.89:0/1209919405 >> v2:172.21.15.89:3300/0 conn(0x7f493c12f8b0 msgr2=0x7f493c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:59.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f493bfff700 1 --2- 172.21.15.89:0/1209919405 >> v2:172.21.15.89:3300/0 conn(0x7f493c12f8b0 0x7f493c12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f493bfff700 1 -- 172.21.15.89:0/1209919405 >> v1:172.21.15.89:6789/0 conn(0x7f493c130260 legacy=0x7f493c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f493bfff700 1 -- 172.21.15.89:0/1209919405 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c071780 con 0x7f493c12cf60 2023-09-30T22:19:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f493bfff700 1 --2- 172.21.15.89:0/1209919405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c12cf60 0x7f493c06f5d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f492c009840 tx=0x7f492c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b55b17c80e21c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.088+0000 7f493affd700 1 -- 172.21.15.89:0/1209919405 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c011830 con 0x7f493c12cf60 2023-09-30T22:19:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f493affd700 1 -- 172.21.15.89:0/1209919405 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c011990 con 0x7f493c12cf60 2023-09-30T22:19:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f493affd700 1 -- 172.21.15.89:0/1209919405 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c011d20 con 0x7f493c12cf60 2023-09-30T22:19:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f4942612700 1 -- 172.21.15.89:0/1209919405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c12cf60 msgr2=0x7f493c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f4942612700 1 --2- 172.21.15.89:0/1209919405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c12cf60 0x7f493c06f5d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f492c009840 tx=0x7f492c009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:59.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f4942612700 1 -- 172.21.15.89:0/1209919405 shutdown_connections 2023-09-30T22:19:59.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f4942612700 1 --2- 172.21.15.89:0/1209919405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c12cf60 0x7f493c06f5d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f4942612700 1 --2- 172.21.15.89:0/1209919405 >> v2:172.21.15.89:3300/0 conn(0x7f493c12f8b0 0x7f493c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f4942612700 1 -- 172.21.15.89:0/1209919405 >> 172.21.15.89:0/1209919405 conn(0x7f493c0a2aa0 msgr2=0x7f493c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:59.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f4942612700 1 -- 172.21.15.89:0/1209919405 shutdown_connections 2023-09-30T22:19:59.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.089+0000 7f4942612700 1 -- 172.21.15.89:0/1209919405 wait complete. 2023-09-30T22:19:59.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.091+0000 7f4942612700 1 Processor -- start 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.091+0000 7f4942612700 1 -- start start 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.091+0000 7f4942612700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c12cf60 0x7f493c1c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.091+0000 7f4942612700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f493c12f8b0 0x7f493c1c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.091+0000 7f4942612700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c130260 0x7f493c1c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.091+0000 7f4942612700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c129fa0 con 0x7f493c12cf60 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.091+0000 7f4942612700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c09cf10 con 0x7f493c12f8b0 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.091+0000 7f4942612700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f493c12d7b0 con 0x7f493c130260 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c130260 0x7f493c1c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c130260 0x7f493c1c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45268/0 (socket says 172.21.15.89:45268) 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 -- 172.21.15.89:0/3341077915 learned_addr learned my addr 172.21.15.89:0/3341077915 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f493b7fe700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f493c12f8b0 0x7f493c1c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f493bfff700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c12cf60 0x7f493c1c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 -- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f493c12f8b0 msgr2=0x7f493c1c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f493c12f8b0 0x7f493c1c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 -- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c12cf60 msgr2=0x7f493c1c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c12cf60 0x7f493c1c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 -- 172.21.15.89:0/3341077915 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0094a0 con 0x7f493c130260 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f493b7fe700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f493c12f8b0 0x7f493c1c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4940baf700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c130260 0x7f493c1c4a40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f493400cbc0 tx=0x7f493400ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:59.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4938ff9700 1 -- 172.21.15.89:0/3341077915 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400a9e0 con 0x7f493c130260 2023-09-30T22:19:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4938ff9700 1 -- 172.21.15.89:0/3341077915 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400ab40 con 0x7f493c130260 2023-09-30T22:19:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c09e9c0 con 0x7f493c130260 2023-09-30T22:19:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4938ff9700 1 -- 172.21.15.89:0/3341077915 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934012400 con 0x7f493c130260 2023-09-30T22:19:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.092+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c129ab0 con 0x7f493c130260 2023-09-30T22:19:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.093+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c09e660 con 0x7f493c130260 2023-09-30T22:19:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.094+0000 7f4938ff9700 1 -- 172.21.15.89:0/3341077915 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4934012560 con 0x7f493c130260 2023-09-30T22:19:59.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.094+0000 7f4938ff9700 1 -- 172.21.15.89:0/3341077915 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4934047350 con 0x7f493c130260 2023-09-30T22:19:59.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.099+0000 7f4938ff9700 1 -- 172.21.15.89:0/3341077915 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f493c09e660 con 0x7f493c130260 2023-09-30T22:19:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.107+0000 7f5522c99700 1 Processor -- start 2023-09-30T22:19:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.107+0000 7f5522c99700 1 -- start start 2023-09-30T22:19:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5522c99700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f551c09d9f0 0x7f551c136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5522c99700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c136a70 0x7f551c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5522c99700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c09e130 con 0x7f551c1316c0 2023-09-30T22:19:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5522c99700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551c1321a0 con 0x7f551c09d9f0 2023-09-30T22:19:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5522c99700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f551c1312b0 con 0x7f551c136a70 2023-09-30T22:19:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5521236700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c136a70 0x7f551c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5521236700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c136a70 0x7f551c138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45270/0 (socket says 172.21.15.89:45270) 2023-09-30T22:19:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f551bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f551c09d9f0 0x7f551c136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5520a35700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f551c1316c0 0x7f551c131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53666/0 (socket says 172.21.15.89:53666) 2023-09-30T22:19:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5521236700 1 -- 172.21.15.89:0/4203141481 learned_addr learned my addr 172.21.15.89:0/4203141481 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5521236700 1 -- 172.21.15.89:0/4203141481 >> v2:172.21.15.89:3300/0 conn(0x7f551c09d9f0 msgr2=0x7f551c136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5521236700 1 --2- 172.21.15.89:0/4203141481 >> v2:172.21.15.89:3300/0 conn(0x7f551c09d9f0 0x7f551c136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5521236700 1 -- 172.21.15.89:0/4203141481 >> v1:172.21.15.89:6789/0 conn(0x7f551c1316c0 legacy=0x7f551c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5521236700 1 -- 172.21.15.89:0/4203141481 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c09f6e0 con 0x7f551c136a70 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.108+0000 7f5521236700 1 --2- 172.21.15.89:0/4203141481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c136a70 0x7f551c138e50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5514009e80 tx=0x7f551400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f68d4497904420 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.109+0000 7f551b7fe700 1 -- 172.21.15.89:0/4203141481 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400e070 con 0x7f551c136a70 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.109+0000 7f551b7fe700 1 -- 172.21.15.89:0/4203141481 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551400b8d0 con 0x7f551c136a70 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.109+0000 7f551b7fe700 1 -- 172.21.15.89:0/4203141481 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400bc80 con 0x7f551c136a70 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.109+0000 7f5522c99700 1 -- 172.21.15.89:0/4203141481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c136a70 msgr2=0x7f551c138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.109+0000 7f5522c99700 1 --2- 172.21.15.89:0/4203141481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c136a70 0x7f551c138e50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5514009e80 tx=0x7f551400b330 comp rx=0 tx=0).stop 2023-09-30T22:19:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.110+0000 7f5522c99700 1 -- 172.21.15.89:0/4203141481 shutdown_connections 2023-09-30T22:19:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.110+0000 7f5522c99700 1 --2- 172.21.15.89:0/4203141481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c136a70 0x7f551c138e50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.110+0000 7f5522c99700 1 --2- 172.21.15.89:0/4203141481 >> v2:172.21.15.89:3300/0 conn(0x7f551c09d9f0 0x7f551c136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.110+0000 7f5522c99700 1 -- 172.21.15.89:0/4203141481 >> 172.21.15.89:0/4203141481 conn(0x7f551c066f50 msgr2=0x7f551c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.110+0000 7f5522c99700 1 -- 172.21.15.89:0/4203141481 shutdown_connections 2023-09-30T22:19:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.110+0000 7f5522c99700 1 -- 172.21.15.89:0/4203141481 wait complete. 2023-09-30T22:19:59.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.111+0000 7f5522c99700 1 Processor -- start 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.111+0000 7f5522c99700 1 -- start start 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5522c99700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09d9f0 0x7f551c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5522c99700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1316c0 0x7f551c071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5522c99700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c136a70 0x7f551c071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5522c99700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c135570 con 0x7f551c136a70 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5522c99700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c06fc60 con 0x7f551c1316c0 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5522c99700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f551c070200 con 0x7f551c09d9f0 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5520a35700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09d9f0 0x7f551c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5520a35700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09d9f0 0x7f551c070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45286/0 (socket says 172.21.15.89:45286) 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5520a35700 1 -- 172.21.15.89:0/1536520614 learned_addr learned my addr 172.21.15.89:0/1536520614 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5521236700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c136a70 0x7f551c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f551bfff700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1316c0 0x7f551c071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5521236700 1 -- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09d9f0 msgr2=0x7f551c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5521236700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09d9f0 0x7f551c070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5521236700 1 -- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1316c0 msgr2=0x7f551c071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5521236700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1316c0 0x7f551c071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5521236700 1 -- 172.21.15.89:0/1536520614 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514009b30 con 0x7f551c136a70 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5520a35700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09d9f0 0x7f551c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.112+0000 7f5521236700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c136a70 0x7f551c071860 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5514015000 tx=0x7f5514004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.113+0000 7f55197fa700 1 -- 172.21.15.89:0/1536520614 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514004800 con 0x7f551c136a70 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.113+0000 7f55197fa700 1 -- 172.21.15.89:0/1536520614 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55140092e0 con 0x7f551c136a70 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.113+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c071da0 con 0x7f551c136a70 2023-09-30T22:19:59.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.113+0000 7f55197fa700 1 -- 172.21.15.89:0/1536520614 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551401c680 con 0x7f551c136a70 2023-09-30T22:19:59.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.113+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c070990 con 0x7f551c136a70 2023-09-30T22:19:59.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.114+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c074ea0 con 0x7f551c136a70 2023-09-30T22:19:59.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.115+0000 7f55197fa700 1 -- 172.21.15.89:0/1536520614 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f551401c7e0 con 0x7f551c136a70 2023-09-30T22:19:59.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.115+0000 7f55197fa700 1 -- 172.21.15.89:0/1536520614 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5514041800 con 0x7f551c136a70 2023-09-30T22:19:59.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.123+0000 7f55197fa700 1 -- 172.21.15.89:0/1536520614 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f551402e430 con 0x7f551c136a70 2023-09-30T22:19:59.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.151+0000 7f0cbca49700 1 Processor -- start 2023-09-30T22:19:59.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.151+0000 7f0cbca49700 1 -- start start 2023-09-30T22:19:59.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.151+0000 7f0cbca49700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb806c0e0 0x7f0cb809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.151+0000 7f0cbca49700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0cb81380f0 0x7f0cb813fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.151+0000 7f0cbca49700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb8134980 con 0x7f0cb8134ec0 2023-09-30T22:19:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.151+0000 7f0cbca49700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb806c4b0 con 0x7f0cb81380f0 2023-09-30T22:19:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cbca49700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb80a09c0 con 0x7f0cb806c0e0 2023-09-30T22:19:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0cb81380f0 0x7f0cb813fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0cb81380f0 0x7f0cb813fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53468/0 (socket says 172.21.15.89:53468) 2023-09-30T22:19:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb6d9d700 1 -- 172.21.15.89:0/4273941850 learned_addr learned my addr 172.21.15.89:0/4273941850 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb659c700 1 --2- 172.21.15.89:0/4273941850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb806c0e0 0x7f0cb809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb659c700 1 -- 172.21.15.89:0/4273941850 >> v2:172.21.15.89:3300/0 conn(0x7f0cb81380f0 msgr2=0x7f0cb813fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:19:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb659c700 1 --2- 172.21.15.89:0/4273941850 >> v2:172.21.15.89:3300/0 conn(0x7f0cb81380f0 0x7f0cb813fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb659c700 1 -- 172.21.15.89:0/4273941850 >> v1:172.21.15.89:6789/0 conn(0x7f0cb8134ec0 legacy=0x7f0cb81379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb659c700 1 -- 172.21.15.89:0/4273941850 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb80a0600 con 0x7f0cb806c0e0 2023-09-30T22:19:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb559a700 1 -- 172.21.15.89:0/4273941850 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640988624 0 0) 0x7f0cb8134980 con 0x7f0cb8134ec0 2023-09-30T22:19:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb6d9d700 1 --2- 172.21.15.89:0/4273941850 >> v2:172.21.15.89:3300/0 conn(0x7f0cb81380f0 0x7f0cb813fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb659c700 1 --2- 172.21.15.89:0/4273941850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb806c0e0 0x7f0cb809ac90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009840 tx=0x7f0ca8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c48be1a464e8fc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.152+0000 7f0cb559a700 1 -- 172.21.15.89:0/4273941850 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8011830 con 0x7f0cb806c0e0 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cb559a700 1 -- 172.21.15.89:0/4273941850 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca8011990 con 0x7f0cb806c0e0 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cb559a700 1 -- 172.21.15.89:0/4273941850 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8011d20 con 0x7f0cb806c0e0 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cbca49700 1 -- 172.21.15.89:0/4273941850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb806c0e0 msgr2=0x7f0cb809ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cbca49700 1 --2- 172.21.15.89:0/4273941850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb806c0e0 0x7f0cb809ac90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009840 tx=0x7f0ca8009b50 comp rx=0 tx=0).stop 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cbca49700 1 -- 172.21.15.89:0/4273941850 shutdown_connections 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cbca49700 1 --2- 172.21.15.89:0/4273941850 >> v2:172.21.15.89:3300/0 conn(0x7f0cb81380f0 0x7f0cb813fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cbca49700 1 --2- 172.21.15.89:0/4273941850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb806c0e0 0x7f0cb809ac90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cbca49700 1 -- 172.21.15.89:0/4273941850 >> 172.21.15.89:0/4273941850 conn(0x7f0cb80a4210 msgr2=0x7f0cb80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cbca49700 1 -- 172.21.15.89:0/4273941850 shutdown_connections 2023-09-30T22:19:59.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.153+0000 7f0cbca49700 1 -- 172.21.15.89:0/4273941850 wait complete. 2023-09-30T22:19:59.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.155+0000 7f0cbca49700 1 Processor -- start 2023-09-30T22:19:59.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.155+0000 7f0cbca49700 1 -- start start 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.155+0000 7f0cbca49700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb806c0e0 0x7f0cb811a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.155+0000 7f0cbca49700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cb8134ec0 0x7f0cb811ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.155+0000 7f0cbca49700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb81380f0 0x7f0cb811b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cbca49700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb8140d60 con 0x7f0cb806c0e0 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cbca49700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb811dee0 con 0x7f0cb81380f0 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cbca49700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb80a03f0 con 0x7f0cb8134ec0 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb806c0e0 0x7f0cb811a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb806c0e0 0x7f0cb811a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53484/0 (socket says 172.21.15.89:53484) 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb659c700 1 -- 172.21.15.89:0/741462551 learned_addr learned my addr 172.21.15.89:0/741462551 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb6d9d700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb81380f0 0x7f0cb811b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb5d9b700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cb8134ec0 0x7f0cb811ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb6d9d700 1 -- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cb8134ec0 msgr2=0x7f0cb811ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb6d9d700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cb8134ec0 0x7f0cb811ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb6d9d700 1 -- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb806c0e0 msgr2=0x7f0cb811a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb6d9d700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb806c0e0 0x7f0cb811a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb6d9d700 1 -- 172.21.15.89:0/741462551 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb0009b30 con 0x7f0cb81380f0 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb659c700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb806c0e0 0x7f0cb811a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0cb6d9d700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb81380f0 0x7f0cb811b370 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000efc0 tx=0x7f0cb000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0ca6ffd700 1 -- 172.21.15.89:0/741462551 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0018070 con 0x7f0cb81380f0 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.156+0000 7f0ca6ffd700 1 -- 172.21.15.89:0/741462551 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb000f040 con 0x7f0cb81380f0 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.157+0000 7f0ca6ffd700 1 -- 172.21.15.89:0/741462551 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0003c60 con 0x7f0cb81380f0 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.157+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca80094a0 con 0x7f0cb81380f0 2023-09-30T22:19:59.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.157+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb811dc00 con 0x7f0cb81380f0 2023-09-30T22:19:59.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.158+0000 7f0ca6ffd700 1 -- 172.21.15.89:0/741462551 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0cb000c880 con 0x7f0cb81380f0 2023-09-30T22:19:59.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.158+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb811da60 con 0x7f0cb81380f0 2023-09-30T22:19:59.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.158+0000 7f0ca6ffd700 1 -- 172.21.15.89:0/741462551 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0cb003f790 con 0x7f0cb81380f0 2023-09-30T22:19:59.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.167+0000 7f0ca6ffd700 1 -- 172.21.15.89:0/741462551 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0cb811da60 con 0x7f0cb81380f0 2023-09-30T22:19:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.199+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffb8c005590 con 0x7ffba809d6f0 2023-09-30T22:19:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.199+0000 7ffb96ffd700 1 -- 172.21.15.89:0/2524516602 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb98015030 con 0x7ffba809d6f0 2023-09-30T22:19:59.200 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:19:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffba809d6f0 msgr2=0x7ffba81a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffba809d6f0 0x7ffba81a5380 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ffb98000c00 tx=0x7ffb98004750 comp rx=0 tx=0).stop 2023-09-30T22:19:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 shutdown_connections 2023-09-30T22:19:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffba809e440 0x7ffba81c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffba809d6f0 0x7ffba81a5380 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 --2- 172.21.15.89:0/2524516602 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffba809e000 0x7ffba81a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 >> 172.21.15.89:0/2524516602 conn(0x7ffba8066f50 msgr2=0x7ffba81bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:59.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 shutdown_connections 2023-09-30T22:19:59.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.201+0000 7ffbacb05700 1 -- 172.21.15.89:0/2524516602 wait complete. 2023-09-30T22:19:59.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:19:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.301+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f493c09e660 con 0x7f493c130260 2023-09-30T22:19:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.302+0000 7f4938ff9700 1 -- 172.21.15.89:0/3341077915 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f493c09e660 con 0x7f493c130260 2023-09-30T22:19:59.303 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:19:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c130260 msgr2=0x7f493c1c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c130260 0x7f493c1c4a40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f493400cbc0 tx=0x7f493400ced0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 shutdown_connections 2023-09-30T22:19:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f493c130260 0x7f493c1c4a40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c12cf60 0x7f493c1c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 --2- 172.21.15.89:0/3341077915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f493c12f8b0 0x7f493c1c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 >> 172.21.15.89:0/3341077915 conn(0x7f493c0a2aa0 msgr2=0x7f493c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 shutdown_connections 2023-09-30T22:19:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.304+0000 7f4942612700 1 -- 172.21.15.89:0/3341077915 wait complete. 2023-09-30T22:19:59.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:19:59.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.321+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f551c09e980 con 0x7f551c136a70 2023-09-30T22:19:59.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.321+0000 7f55197fa700 1 -- 172.21.15.89:0/1536520614 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5514023b50 con 0x7f551c136a70 2023-09-30T22:19:59.322 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c136a70 msgr2=0x7f551c071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c136a70 0x7f551c071860 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5514015000 tx=0x7f5514004ad0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 shutdown_connections 2023-09-30T22:19:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c136a70 0x7f551c071860 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09d9f0 0x7f551c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 --2- 172.21.15.89:0/1536520614 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1316c0 0x7f551c071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 >> 172.21.15.89:0/1536520614 conn(0x7f551c066f50 msgr2=0x7f551c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 shutdown_connections 2023-09-30T22:19:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.323+0000 7f5522c99700 1 -- 172.21.15.89:0/1536520614 wait complete. 2023-09-30T22:19:59.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:19:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.364+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0cb811da60 con 0x7f0cb81380f0 2023-09-30T22:19:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.364+0000 7f0ca6ffd700 1 -- 172.21.15.89:0/741462551 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0cb811da60 con 0x7f0cb81380f0 2023-09-30T22:19:59.365 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:19:59.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb81380f0 msgr2=0x7f0cb811b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:19:59.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb81380f0 0x7f0cb811b370 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000efc0 tx=0x7f0cb000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 shutdown_connections 2023-09-30T22:19:59.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cb81380f0 0x7f0cb811b370 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cb806c0e0 0x7f0cb811a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 --2- 172.21.15.89:0/741462551 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cb8134ec0 0x7f0cb811ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:19:59.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 >> 172.21.15.89:0/741462551 conn(0x7f0cb80a4210 msgr2=0x7f0cb8137050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:19:59.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 shutdown_connections 2023-09-30T22:19:59.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:19:59.366+0000 7f0cbca49700 1 -- 172.21.15.89:0/741462551 wait complete. 2023-09-30T22:19:59.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:00.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:00.318 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:00.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.352+0000 7ff66546d700 1 Processor -- start 2023-09-30T22:20:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.352+0000 7ff66546d700 1 -- start start 2023-09-30T22:20:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff66546d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6600a0a30 0x7ff6600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff66546d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff660138d80 0x7ff66013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff66546d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6600a2a70 con 0x7ff66009d630 2023-09-30T22:20:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff66546d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6600a01d0 con 0x7ff6600a0a30 2023-09-30T22:20:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff66546d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66009e2b0 con 0x7ff660138d80 2023-09-30T22:20:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff660138d80 0x7ff66013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6600a0a30 0x7ff6600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff660138d80 0x7ff66013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47174/0 (socket says 172.21.15.89:47174) 2023-09-30T22:20:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6600a0a30 0x7ff6600a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59500/0 (socket says 172.21.15.89:59500) 2023-09-30T22:20:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65f7fe700 1 -- 172.21.15.89:0/4056098713 learned_addr learned my addr 172.21.15.89:0/4056098713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65effd700 1 -- 172.21.15.89:0/4056098713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff660138d80 msgr2=0x7ff66013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65effd700 1 --2- 172.21.15.89:0/4056098713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff660138d80 0x7ff66013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65effd700 1 -- 172.21.15.89:0/4056098713 >> v1:172.21.15.89:6789/0 conn(0x7ff66009d630 legacy=0x7ff6600a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65effd700 1 -- 172.21.15.89:0/4056098713 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66009dc50 con 0x7ff6600a0a30 2023-09-30T22:20:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.353+0000 7ff65f7fe700 1 --2- 172.21.15.89:0/4056098713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff660138d80 0x7ff66013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff65effd700 1 --2- 172.21.15.89:0/4056098713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6600a0a30 0x7ff6600a0e00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff650009840 tx=0x7ff650009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d971309ec69cda7e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff65dffb700 1 -- 172.21.15.89:0/4056098713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650011830 con 0x7ff6600a0a30 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff65dffb700 1 -- 172.21.15.89:0/4056098713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff650011990 con 0x7ff6600a0a30 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff65dffb700 1 -- 172.21.15.89:0/4056098713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650011d20 con 0x7ff6600a0a30 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff66546d700 1 -- 172.21.15.89:0/4056098713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6600a0a30 msgr2=0x7ff6600a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff66546d700 1 --2- 172.21.15.89:0/4056098713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6600a0a30 0x7ff6600a0e00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff650009840 tx=0x7ff650009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff66546d700 1 -- 172.21.15.89:0/4056098713 shutdown_connections 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff66546d700 1 --2- 172.21.15.89:0/4056098713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff660138d80 0x7ff66013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff66546d700 1 --2- 172.21.15.89:0/4056098713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6600a0a30 0x7ff6600a0e00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.354+0000 7ff66546d700 1 -- 172.21.15.89:0/4056098713 >> 172.21.15.89:0/4056098713 conn(0x7ff6600a3050 msgr2=0x7ff6600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.355+0000 7ff66546d700 1 -- 172.21.15.89:0/4056098713 shutdown_connections 2023-09-30T22:20:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.355+0000 7ff66546d700 1 -- 172.21.15.89:0/4056098713 wait complete. 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.356+0000 7ff66546d700 1 Processor -- start 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.356+0000 7ff66546d700 1 -- start start 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.356+0000 7ff66546d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66009d630 0x7ff6601207c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.356+0000 7ff66546d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6600a0a30 0x7ff660120d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.356+0000 7ff66546d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff660138d80 0x7ff660121240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.356+0000 7ff66546d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6601223d0 con 0x7ff660138d80 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.356+0000 7ff66546d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff660122c30 con 0x7ff66009d630 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.356+0000 7ff66546d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff660124000 con 0x7ff6600a0a30 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66009d630 0x7ff6601207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65e7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6600a0a30 0x7ff660120d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff660138d80 0x7ff660121240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66009d630 0x7ff6601207c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47178/0 (socket says 172.21.15.89:47178) 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 -- 172.21.15.89:0/1244608647 learned_addr learned my addr 172.21.15.89:0/1244608647 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 -- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6600a0a30 msgr2=0x7ff660120d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 --2- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6600a0a30 0x7ff660120d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 -- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff660138d80 msgr2=0x7ff660121240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 --2- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff660138d80 0x7ff660121240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 -- 172.21.15.89:0/1244608647 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658009b30 con 0x7ff66009d630 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65e7fc700 1 --2- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6600a0a30 0x7ff660120d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff65effd700 1 --2- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66009d630 0x7ff6601207c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff650009fd0 tx=0x7ff650004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.357+0000 7ff6477fe700 1 -- 172.21.15.89:0/1244608647 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6500036a0 con 0x7ff66009d630 2023-09-30T22:20:00.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.358+0000 7ff6477fe700 1 -- 172.21.15.89:0/1244608647 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65001f070 con 0x7ff66009d630 2023-09-30T22:20:00.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.358+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6500094a0 con 0x7ff66009d630 2023-09-30T22:20:00.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.358+0000 7ff6477fe700 1 -- 172.21.15.89:0/1244608647 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650003a60 con 0x7ff66009d630 2023-09-30T22:20:00.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.358+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff660123ac0 con 0x7ff66009d630 2023-09-30T22:20:00.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.358+0000 7ff6477fe700 1 -- 172.21.15.89:0/1244608647 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff650003bc0 con 0x7ff66009d630 2023-09-30T22:20:00.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.358+0000 7ff6477fe700 1 -- 172.21.15.89:0/1244608647 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff650021070 con 0x7ff66009d630 2023-09-30T22:20:00.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.359+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff660123620 con 0x7ff66009d630 2023-09-30T22:20:00.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.364+0000 7ff6477fe700 1 -- 172.21.15.89:0/1244608647 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff650015080 con 0x7ff66009d630 2023-09-30T22:20:00.380 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.460+0000 7f5a390f6700 1 Processor -- start 2023-09-30T22:20:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.460+0000 7f5a390f6700 1 -- start start 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.461+0000 7f5a390f6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 0x7f5a3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.461+0000 7f5a390f6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a34130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.461+0000 7f5a390f6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3409f130 con 0x7f5a3409e000 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.461+0000 7f5a390f6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3409f520 con 0x7f5a3409e410 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.461+0000 7f5a390f6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3409e930 con 0x7f5a3409d6f0 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.461+0000 7f5a3259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5a3409e000 0x7f5a34130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35550/0 (socket says 172.21.15.89:35550) 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.461+0000 7f5a3259c700 1 -- 172.21.15.89:0/4011533600 learned_addr learned my addr 172.21.15.89:0/4011533600 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.461+0000 7f5a3359e700 1 --2- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a34130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a32d9d700 1 --2- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 0x7f5a3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a3359e700 1 -- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 msgr2=0x7f5a3409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a3359e700 1 --2- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 0x7f5a3409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a3359e700 1 -- 172.21.15.89:0/4011533600 >> v1:172.21.15.89:6789/0 conn(0x7f5a3409e000 legacy=0x7f5a34130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a3359e700 1 -- 172.21.15.89:0/4011533600 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3409b2d0 con 0x7f5a3409e410 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a32d9d700 1 --2- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 0x7f5a3409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a3359e700 1 --2- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a34130d70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009dc0 tx=0x7f5a2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3125eefc9e44488 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a31d9b700 1 -- 172.21.15.89:0/4011533600 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00e070 con 0x7f5a3409e410 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a31d9b700 1 -- 172.21.15.89:0/4011533600 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2c00bbd0 con 0x7f5a3409e410 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.462+0000 7f5a31d9b700 1 -- 172.21.15.89:0/4011533600 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c012430 con 0x7f5a3409e410 2023-09-30T22:20:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.463+0000 7f5a390f6700 1 -- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 msgr2=0x7f5a34130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.463+0000 7f5a390f6700 1 --2- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a34130d70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009dc0 tx=0x7f5a2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.463+0000 7f5a390f6700 1 -- 172.21.15.89:0/4011533600 shutdown_connections 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.463+0000 7f5a390f6700 1 --2- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a34130d70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.463+0000 7f5a390f6700 1 --2- 172.21.15.89:0/4011533600 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 0x7f5a3409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.463+0000 7f5a390f6700 1 -- 172.21.15.89:0/4011533600 >> 172.21.15.89:0/4011533600 conn(0x7f5a34066f50 msgr2=0x7f5a3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.463+0000 7f5a390f6700 1 -- 172.21.15.89:0/4011533600 shutdown_connections 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.463+0000 7f5a390f6700 1 -- 172.21.15.89:0/4011533600 wait complete. 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.464+0000 7f5a390f6700 1 Processor -- start 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.464+0000 7f5a390f6700 1 -- start start 2023-09-30T22:20:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.464+0000 7f5a390f6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 0x7f5a3406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.464+0000 7f5a390f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3409e000 0x7f5a3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.464+0000 7f5a390f6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a3406f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a390f6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a34125c70 con 0x7f5a3409e000 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a390f6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a34126b30 con 0x7f5a3409e410 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a390f6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a341303e0 con 0x7f5a3409d6f0 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3409e000 0x7f5a3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3409e000 0x7f5a3406edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59506/0 (socket says 172.21.15.89:59506) 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 -- 172.21.15.89:0/503620112 learned_addr learned my addr 172.21.15.89:0/503620112 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3359e700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a3406f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 -- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 msgr2=0x7f5a3406e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 0x7f5a3406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 -- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 msgr2=0x7f5a3406f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a3406f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 -- 172.21.15.89:0/503620112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c009a70 con 0x7f5a3409e000 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3359e700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a3406f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a3259c700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3409e000 0x7f5a3406edb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800c500 tx=0x7f5a2800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a1b7fe700 1 -- 172.21.15.89:0/503620112 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28005d70 con 0x7f5a3409e000 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a1b7fe700 1 -- 172.21.15.89:0/503620112 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a280070d0 con 0x7f5a3409e000 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a340a1c30 con 0x7f5a3409e000 2023-09-30T22:20:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.465+0000 7f5a1b7fe700 1 -- 172.21.15.89:0/503620112 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28007440 con 0x7f5a3409e000 2023-09-30T22:20:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.466+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a34070430 con 0x7f5a3409e000 2023-09-30T22:20:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.467+0000 7f5a1b7fe700 1 -- 172.21.15.89:0/503620112 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a280075a0 con 0x7f5a3409e000 2023-09-30T22:20:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.467+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3409c700 con 0x7f5a3409e000 2023-09-30T22:20:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.467+0000 7f5a1b7fe700 1 -- 172.21.15.89:0/503620112 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5a2803ead0 con 0x7f5a3409e000 2023-09-30T22:20:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.476+0000 7f5a1b7fe700 1 -- 172.21.15.89:0/503620112 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a28019b40 con 0x7f5a3409e000 2023-09-30T22:20:00.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.485+0000 7fb2e2a96700 1 Processor -- start 2023-09-30T22:20:00.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.485+0000 7fb2e2a96700 1 -- start start 2023-09-30T22:20:00.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e2a96700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2dc06a770 0x7fb2dc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e2a96700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc09dc30 0x7fb2dc09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e2a96700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc13c1e0 0x7fb2dc13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e2a96700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc06ab40 con 0x7fb2dc06a770 2023-09-30T22:20:00.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e2a96700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc09ebc0 con 0x7fb2dc09dc30 2023-09-30T22:20:00.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e2a96700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc09a810 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e0832700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2dc06a770 0x7fb2dc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2dbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc09dc30 0x7fb2dc09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e0832700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2dc06a770 0x7fb2dc09d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59518/0 (socket says 172.21.15.89:59518) 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e0832700 1 -- 172.21.15.89:0/1250119711 learned_addr learned my addr 172.21.15.89:0/1250119711 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2dbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc09dc30 0x7fb2dc09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47208/0 (socket says 172.21.15.89:47208) 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e1033700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc13c1e0 0x7fb2dc13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e0832700 1 -- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc13c1e0 msgr2=0x7fb2dc13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e0832700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc13c1e0 0x7fb2dc13e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e0832700 1 -- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc09dc30 msgr2=0x7fb2dc09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e0832700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc09dc30 0x7fb2dc09e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e0832700 1 -- 172.21.15.89:0/1250119711 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc09e920 con 0x7fb2dc06a770 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2e1033700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc13c1e0 0x7fb2dc13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.486+0000 7fb2dbfff700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc09dc30 0x7fb2dc09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e0832700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc06a770 0x7fb2dc09d6f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009920 tx=0x7fb2d0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c9e198c8d03a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2db7fe700 1 -- 172.21.15.89:0/1250119711 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0011830 con 0x7fb2dc06a770 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2db7fe700 1 -- 172.21.15.89:0/1250119711 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0011990 con 0x7fb2dc06a770 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2db7fe700 1 -- 172.21.15.89:0/1250119711 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0011d20 con 0x7fb2dc06a770 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc06a770 msgr2=0x7fb2dc09d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc06a770 0x7fb2dc09d6f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009920 tx=0x7fb2d0009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:00.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1250119711 shutdown_connections 2023-09-30T22:20:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc13c1e0 0x7fb2dc13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc06a770 0x7fb2dc09d6f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 --2- 172.21.15.89:0/1250119711 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc09dc30 0x7fb2dc09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1250119711 >> 172.21.15.89:0/1250119711 conn(0x7fb2dc066f50 msgr2=0x7fb2dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1250119711 shutdown_connections 2023-09-30T22:20:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.487+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1250119711 wait complete. 2023-09-30T22:20:00.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.489+0000 7fb2e2a96700 1 Processor -- start 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.489+0000 7fb2e2a96700 1 -- start start 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.489+0000 7fb2e2a96700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc06a770 0x7fb2dc1cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.489+0000 7fb2e2a96700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc09dc30 0x7fb2dc1cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.489+0000 7fb2e2a96700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc13c1e0 0x7fb2dc1ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.489+0000 7fb2e2a96700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc128080 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.489+0000 7fb2e2a96700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc127e90 con 0x7fb2dc06a770 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.489+0000 7fb2e2a96700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc129da0 con 0x7fb2dc09dc30 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e0832700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc06a770 0x7fb2dc1cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc13c1e0 0x7fb2dc1ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc13c1e0 0x7fb2dc1ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59520/0 (socket says 172.21.15.89:59520) 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e0832700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc06a770 0x7fb2dc1cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47224/0 (socket says 172.21.15.89:47224) 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 -- 172.21.15.89:0/1597495083 learned_addr learned my addr 172.21.15.89:0/1597495083 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2dbfff700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc09dc30 0x7fb2dc1cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 -- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc09dc30 msgr2=0x7fb2dc1cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc09dc30 0x7fb2dc1cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 -- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc06a770 msgr2=0x7fb2dc1cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc06a770 0x7fb2dc1cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 -- 172.21.15.89:0/1597495083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80094a0 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e0832700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc06a770 0x7fb2dc1cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e1033700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc13c1e0 0x7fb2dc1ccf60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400efc0 tx=0x7fb2d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2d97fa700 1 -- 172.21.15.89:0/1597495083 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400cb40 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d4009a70 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2d97fa700 1 -- 172.21.15.89:0/1597495083 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d400f040 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.490+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d0009550 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.491+0000 7fb2d97fa700 1 -- 172.21.15.89:0/1597495083 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4003a20 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.491+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc0052e0 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.491+0000 7fb2d97fa700 1 -- 172.21.15.89:0/1597495083 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb2d4014480 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.492+0000 7fb2d97fa700 1 -- 172.21.15.89:0/1597495083 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb2d403ffe0 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.496+0000 7fb2d97fa700 1 -- 172.21.15.89:0/1597495083 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2d4012930 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6539347700 1 Processor -- start 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6539347700 1 -- start start 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6539347700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f653406a770 0x7f653409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6539347700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653409dc30 0x7f653409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6539347700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653413c1e0 0x7f653413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6539347700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653406ab40 con 0x7f653406a770 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6539347700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653409ebc0 con 0x7f653413c1e0 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6539347700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f653409a810 con 0x7f653409dc30 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6532ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f653406a770 0x7f653409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6532ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f653406a770 0x7f653409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59530/0 (socket says 172.21.15.89:59530) 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f6532ffd700 1 -- 172.21.15.89:0/3092887706 learned_addr learned my addr 172.21.15.89:0/3092887706 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.530+0000 7f65337fe700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653413c1e0 0x7f653413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f65327fc700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653409dc30 0x7f653409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6532ffd700 1 -- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653409dc30 msgr2=0x7f653409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6532ffd700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653409dc30 0x7f653409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6532ffd700 1 -- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653413c1e0 msgr2=0x7f653413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6532ffd700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653413c1e0 0x7f653413e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6532ffd700 1 -- 172.21.15.89:0/3092887706 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653409e920 con 0x7f653406a770 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f65327fc700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653409dc30 0x7f653409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6532ffd700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653406a770 0x7f653409d6f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6524009920 tx=0x7f6524009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bdd67f37a39856 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6531ffb700 1 -- 172.21.15.89:0/3092887706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524011830 con 0x7f653406a770 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6531ffb700 1 -- 172.21.15.89:0/3092887706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6524011990 con 0x7f653406a770 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.531+0000 7f6531ffb700 1 -- 172.21.15.89:0/3092887706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524011d20 con 0x7f653406a770 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 -- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653406a770 msgr2=0x7f653409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653406a770 0x7f653409d6f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6524009920 tx=0x7f6524009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 -- 172.21.15.89:0/3092887706 shutdown_connections 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653413c1e0 0x7f653413e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653406a770 0x7f653409d6f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 --2- 172.21.15.89:0/3092887706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653409dc30 0x7f653409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 -- 172.21.15.89:0/3092887706 >> 172.21.15.89:0/3092887706 conn(0x7f6534066f50 msgr2=0x7f653409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 -- 172.21.15.89:0/3092887706 shutdown_connections 2023-09-30T22:20:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.532+0000 7f6539347700 1 -- 172.21.15.89:0/3092887706 wait complete. 2023-09-30T22:20:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.533+0000 7f6539347700 1 Processor -- start 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.533+0000 7f6539347700 1 -- start start 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f6539347700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653406a770 0x7f653411d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f6539347700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653409dc30 0x7f653411dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f6539347700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653413c1e0 0x7f653411e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f6539347700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653409f1a0 con 0x7f653409dc30 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f6539347700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653411eee0 con 0x7f653406a770 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f6539347700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f653413bf80 con 0x7f653413c1e0 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f65327fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653409dc30 0x7f653411dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f65327fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653409dc30 0x7f653411dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59538/0 (socket says 172.21.15.89:59538) 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f65327fc700 1 -- 172.21.15.89:0/2800652030 learned_addr learned my addr 172.21.15.89:0/2800652030 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f6532ffd700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653406a770 0x7f653411d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.534+0000 7f65337fe700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653413c1e0 0x7f653411e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f65327fc700 1 -- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653413c1e0 msgr2=0x7f653411e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f65327fc700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653413c1e0 0x7f653411e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f65327fc700 1 -- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653406a770 msgr2=0x7f653411d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f65327fc700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653406a770 0x7f653411d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f65327fc700 1 -- 172.21.15.89:0/2800652030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6524009550 con 0x7f653409dc30 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f65327fc700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653409dc30 0x7f653411dd30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f652800e810 tx=0x7f652800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f651b7fe700 1 -- 172.21.15.89:0/2800652030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528013610 con 0x7f653409dc30 2023-09-30T22:20:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f651b7fe700 1 -- 172.21.15.89:0/2800652030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6528013770 con 0x7f653409dc30 2023-09-30T22:20:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653411ffd0 con 0x7f653409dc30 2023-09-30T22:20:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653411d0e0 con 0x7f653409dc30 2023-09-30T22:20:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.535+0000 7f651b7fe700 1 -- 172.21.15.89:0/2800652030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528013b00 con 0x7f653409dc30 2023-09-30T22:20:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.536+0000 7f651b7fe700 1 -- 172.21.15.89:0/2800652030 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6528013c60 con 0x7f653409dc30 2023-09-30T22:20:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.536+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65140052e0 con 0x7f653409dc30 2023-09-30T22:20:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.537+0000 7f651b7fe700 1 -- 172.21.15.89:0/2800652030 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f652800efa0 con 0x7f653409dc30 2023-09-30T22:20:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.541+0000 7f651b7fe700 1 -- 172.21.15.89:0/2800652030 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f653411d0e0 con 0x7f653409dc30 2023-09-30T22:20:00.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.566+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff6600747f0 con 0x7ff66009d630 2023-09-30T22:20:00.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.566+0000 7ff6477fe700 1 -- 172.21.15.89:0/1244608647 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff650035090 con 0x7ff66009d630 2023-09-30T22:20:00.567 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:00.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66009d630 msgr2=0x7ff6601207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 --2- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66009d630 0x7ff6601207c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff650009fd0 tx=0x7ff650004750 comp rx=0 tx=0).stop 2023-09-30T22:20:00.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 shutdown_connections 2023-09-30T22:20:00.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 --2- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff660138d80 0x7ff660121240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 --2- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66009d630 0x7ff6601207c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 --2- 172.21.15.89:0/1244608647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6600a0a30 0x7ff660120d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 >> 172.21.15.89:0/1244608647 conn(0x7ff6600a3050 msgr2=0x7ff66009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:00.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 shutdown_connections 2023-09-30T22:20:00.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.568+0000 7ff66546d700 1 -- 172.21.15.89:0/1244608647 wait complete. 2023-09-30T22:20:00.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:00.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.691+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5a3409ca30 con 0x7f5a3409e000 2023-09-30T22:20:00.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.691+0000 7f5a1b7fe700 1 -- 172.21.15.89:0/503620112 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a28017070 con 0x7f5a3409e000 2023-09-30T22:20:00.692 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3409e000 msgr2=0x7f5a3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3409e000 0x7f5a3406edb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800c500 tx=0x7f5a2800c810 comp rx=0 tx=0).stop 2023-09-30T22:20:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 shutdown_connections 2023-09-30T22:20:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a3409e410 0x7f5a3406f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a3409d6f0 0x7f5a3406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 --2- 172.21.15.89:0/503620112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3409e000 0x7f5a3406edb0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 >> 172.21.15.89:0/503620112 conn(0x7f5a34066f50 msgr2=0x7f5a34071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 shutdown_connections 2023-09-30T22:20:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.693+0000 7f5a390f6700 1 -- 172.21.15.89:0/503620112 wait complete. 2023-09-30T22:20:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.695+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb2bc005590 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.695+0000 7fb2d97fa700 1 -- 172.21.15.89:0/1597495083 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2d4012340 con 0x7fb2dc13c1e0 2023-09-30T22:20:00.696 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc13c1e0 msgr2=0x7fb2dc1ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc13c1e0 0x7fb2dc1ccf60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400efc0 tx=0x7fb2d400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 shutdown_connections 2023-09-30T22:20:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2dc13c1e0 0x7fb2dc1ccf60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc06a770 0x7fb2dc1cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 --2- 172.21.15.89:0/1597495083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc09dc30 0x7fb2dc1cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 >> 172.21.15.89:0/1597495083 conn(0x7fb2dc066f50 msgr2=0x7fb2dc0a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:00.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 shutdown_connections 2023-09-30T22:20:00.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.697+0000 7fb2e2a96700 1 -- 172.21.15.89:0/1597495083 wait complete. 2023-09-30T22:20:00.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:00.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:00.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.737+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6514005590 con 0x7f653409dc30 2023-09-30T22:20:00.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.737+0000 7f651b7fe700 1 -- 172.21.15.89:0/2800652030 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6528044060 con 0x7f653409dc30 2023-09-30T22:20:00.739 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:00.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653409dc30 msgr2=0x7f653411dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:00.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653409dc30 0x7f653411dd30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f652800e810 tx=0x7f652800eb20 comp rx=0 tx=0).stop 2023-09-30T22:20:00.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 shutdown_connections 2023-09-30T22:20:00.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f653413c1e0 0x7f653411e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f653406a770 0x7f653411d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 --2- 172.21.15.89:0/2800652030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653409dc30 0x7f653411dd30 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:00.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 >> 172.21.15.89:0/2800652030 conn(0x7f6534066f50 msgr2=0x7f65340a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:00.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 shutdown_connections 2023-09-30T22:20:00.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:00.739+0000 7f6539347700 1 -- 172.21.15.89:0/2800652030 wait complete. 2023-09-30T22:20:00.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:01.583 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:01.708 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:01.711 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:01.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.718+0000 7f335b207700 1 Processor -- start 2023-09-30T22:20:01.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.718+0000 7f335b207700 1 -- start start 2023-09-30T22:20:01.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.718+0000 7f335b207700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f33541338a0 0x7f3354133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.718+0000 7f335b207700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f335b207700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335406ef30 con 0x7f335409e760 2023-09-30T22:20:01.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f335b207700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335406fef0 con 0x7f33541338a0 2023-09-30T22:20:01.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f335b207700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33541334b0 con 0x7f335409de50 2023-09-30T22:20:01.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3353fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3358fa3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f33541338a0 0x7f3354133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3358fa3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f33541338a0 0x7f3354133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59554/0 (socket says 172.21.15.89:59554) 2023-09-30T22:20:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3353fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36364/0 (socket says 172.21.15.89:36364) 2023-09-30T22:20:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3358fa3700 1 -- 172.21.15.89:0/2702877163 learned_addr learned my addr 172.21.15.89:0/2702877163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3358fa3700 1 -- 172.21.15.89:0/2702877163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 msgr2=0x7f335409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3358fa3700 1 --2- 172.21.15.89:0/2702877163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3358fa3700 1 -- 172.21.15.89:0/2702877163 >> v1:172.21.15.89:6789/0 conn(0x7f335409e760 legacy=0x7f335413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3358fa3700 1 -- 172.21.15.89:0/2702877163 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335406fc30 con 0x7f33541338a0 2023-09-30T22:20:01.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3353fff700 1 --2- 172.21.15.89:0/2702877163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f3358fa3700 1 --2- 172.21.15.89:0/2702877163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 0x7f3354133c70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f33480098f0 tx=0x7f3348009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1798173425d6ec88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.719+0000 7f33537fe700 1 -- 172.21.15.89:0/2702877163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348011830 con 0x7f33541338a0 2023-09-30T22:20:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f33537fe700 1 -- 172.21.15.89:0/2702877163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3348011990 con 0x7f33541338a0 2023-09-30T22:20:01.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f33537fe700 1 -- 172.21.15.89:0/2702877163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348011d20 con 0x7f33541338a0 2023-09-30T22:20:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f335b207700 1 -- 172.21.15.89:0/2702877163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 msgr2=0x7f3354133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f335b207700 1 --2- 172.21.15.89:0/2702877163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 0x7f3354133c70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f33480098f0 tx=0x7f3348009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f335b207700 1 -- 172.21.15.89:0/2702877163 shutdown_connections 2023-09-30T22:20:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f335b207700 1 --2- 172.21.15.89:0/2702877163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 0x7f3354133c70 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f335b207700 1 --2- 172.21.15.89:0/2702877163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f335b207700 1 -- 172.21.15.89:0/2702877163 >> 172.21.15.89:0/2702877163 conn(0x7f33540a2aa0 msgr2=0x7f33540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:01.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f335b207700 1 -- 172.21.15.89:0/2702877163 shutdown_connections 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.720+0000 7f335b207700 1 -- 172.21.15.89:0/2702877163 wait complete. 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.721+0000 7f335b207700 1 Processor -- start 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.721+0000 7f335b207700 1 -- start start 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f335b207700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335411eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f335b207700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335409e760 0x7f335411f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f335b207700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 0x7f335411f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f335b207700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335406f640 con 0x7f33541338a0 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f335b207700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33541222b0 con 0x7f335409e760 2023-09-30T22:20:01.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f335b207700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3354120790 con 0x7f335409de50 2023-09-30T22:20:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3358fa3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335411eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3358fa3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335411eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36370/0 (socket says 172.21.15.89:36370) 2023-09-30T22:20:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3358fa3700 1 -- 172.21.15.89:0/1723880812 learned_addr learned my addr 172.21.15.89:0/1723880812 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3353fff700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335409e760 0x7f335411f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f33597a4700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 0x7f335411f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3358fa3700 1 -- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335409e760 msgr2=0x7f335411f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3358fa3700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335409e760 0x7f335411f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3358fa3700 1 -- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 msgr2=0x7f335411f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3358fa3700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 0x7f335411f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.722+0000 7f3358fa3700 1 -- 172.21.15.89:0/1723880812 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400094a0 con 0x7f335409de50 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.723+0000 7f3358fa3700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335411eb00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3348009c40 tx=0x7f3348004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.723+0000 7f33517fa700 1 -- 172.21.15.89:0/1723880812 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348004400 con 0x7f335409de50 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.723+0000 7f33517fa700 1 -- 172.21.15.89:0/1723880812 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334801f070 con 0x7f335409de50 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.723+0000 7f335b207700 1 -- 172.21.15.89:0/1723880812 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3348009550 con 0x7f335409de50 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.723+0000 7f33517fa700 1 -- 172.21.15.89:0/1723880812 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33480038e0 con 0x7f335409de50 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.723+0000 7f335b207700 1 -- 172.21.15.89:0/1723880812 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335412f590 con 0x7f335409de50 2023-09-30T22:20:01.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.724+0000 7f33517fa700 1 -- 172.21.15.89:0/1723880812 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3348021030 con 0x7f335409de50 2023-09-30T22:20:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.724+0000 7f33517fa700 1 -- 172.21.15.89:0/1723880812 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3348040d70 con 0x7f335409de50 2023-09-30T22:20:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.724+0000 7f333effd700 1 -- 172.21.15.89:0/1723880812 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c0052e0 con 0x7f335409de50 2023-09-30T22:20:01.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.729+0000 7f33517fa700 1 -- 172.21.15.89:0/1723880812 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f334800b9d0 con 0x7f335409de50 2023-09-30T22:20:01.753 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.848+0000 7feabd32c700 1 Processor -- start 2023-09-30T22:20:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feabd32c700 1 -- start start 2023-09-30T22:20:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feabd32c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab81316c0 0x7feab8131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feabd32c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feab809d9f0 0x7feab806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feabd32c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab806acb0 0x7feab806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feabd32c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab806b630 con 0x7feab809d9f0 2023-09-30T22:20:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feabd32c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab809e130 con 0x7feab806acb0 2023-09-30T22:20:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feabd32c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feab81312b0 con 0x7feab81316c0 2023-09-30T22:20:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feab6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab81316c0 0x7feab8131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feab6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab81316c0 0x7feab8131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36378/0 (socket says 172.21.15.89:36378) 2023-09-30T22:20:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.849+0000 7feab6ffd700 1 -- 172.21.15.89:0/3364688496 learned_addr learned my addr 172.21.15.89:0/3364688496 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab77fe700 1 --2- 172.21.15.89:0/3364688496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab806acb0 0x7feab806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab67fc700 1 --2- 172.21.15.89:0/3364688496 >> v2:172.21.15.89:3300/0 conn(0x7feab809d9f0 0x7feab806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab6ffd700 1 -- 172.21.15.89:0/3364688496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab806acb0 msgr2=0x7feab806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab6ffd700 1 --2- 172.21.15.89:0/3364688496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab806acb0 0x7feab806b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab6ffd700 1 -- 172.21.15.89:0/3364688496 >> v2:172.21.15.89:3300/0 conn(0x7feab809d9f0 msgr2=0x7feab806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab6ffd700 1 --2- 172.21.15.89:0/3364688496 >> v2:172.21.15.89:3300/0 conn(0x7feab809d9f0 0x7feab806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab6ffd700 1 -- 172.21.15.89:0/3364688496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab809f6e0 con 0x7feab81316c0 2023-09-30T22:20:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab67fc700 1 --2- 172.21.15.89:0/3364688496 >> v2:172.21.15.89:3300/0 conn(0x7feab809d9f0 0x7feab806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab6ffd700 1 --2- 172.21.15.89:0/3364688496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab81316c0 0x7feab8131a90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009920 tx=0x7feaa8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4ccc2910ae5948 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:01.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab5ffb700 1 -- 172.21.15.89:0/3364688496 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8011830 con 0x7feab81316c0 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab5ffb700 1 -- 172.21.15.89:0/3364688496 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa8011990 con 0x7feab81316c0 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.850+0000 7feab5ffb700 1 -- 172.21.15.89:0/3364688496 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8011d20 con 0x7feab81316c0 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 -- 172.21.15.89:0/3364688496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab81316c0 msgr2=0x7feab8131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 --2- 172.21.15.89:0/3364688496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab81316c0 0x7feab8131a90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009920 tx=0x7feaa8009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 -- 172.21.15.89:0/3364688496 shutdown_connections 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 --2- 172.21.15.89:0/3364688496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab806acb0 0x7feab806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 --2- 172.21.15.89:0/3364688496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab81316c0 0x7feab8131a90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 --2- 172.21.15.89:0/3364688496 >> v2:172.21.15.89:3300/0 conn(0x7feab809d9f0 0x7feab806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 -- 172.21.15.89:0/3364688496 >> 172.21.15.89:0/3364688496 conn(0x7feab8066f50 msgr2=0x7feab809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:01.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 -- 172.21.15.89:0/3364688496 shutdown_connections 2023-09-30T22:20:01.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.851+0000 7feabd32c700 1 -- 172.21.15.89:0/3364688496 wait complete. 2023-09-30T22:20:01.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.852+0000 7f0ea9fba700 1 Processor -- start 2023-09-30T22:20:01.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.852+0000 7feabd32c700 1 Processor -- start 2023-09-30T22:20:01.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.852+0000 7f0ea9fba700 1 -- start start 2023-09-30T22:20:01.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.852+0000 7feabd32c700 1 -- start start 2023-09-30T22:20:01.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feabd32c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab806acb0 0x7feab8070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feabd32c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab809d9f0 0x7feab8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feabd32c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab81316c0 0x7feab80711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feabd32c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab811a320 con 0x7feab81316c0 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feabd32c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab811a840 con 0x7feab809d9f0 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feabd32c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feab80a3740 con 0x7feab806acb0 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feab6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab806acb0 0x7feab8070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feab6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab806acb0 0x7feab8070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36388/0 (socket says 172.21.15.89:36388) 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feab6ffd700 1 -- 172.21.15.89:0/3853322745 learned_addr learned my addr 172.21.15.89:0/3853322745 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feab77fe700 1 --2- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab81316c0 0x7feab80711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feab6ffd700 1 -- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab809d9f0 msgr2=0x7feab8070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feab6ffd700 1 --2- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab809d9f0 0x7feab8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feab6ffd700 1 -- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab81316c0 msgr2=0x7feab80711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.853+0000 7feab6ffd700 1 --2- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab81316c0 0x7feab80711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7feab6ffd700 1 -- 172.21.15.89:0/3853322745 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa8009550 con 0x7feab806acb0 2023-09-30T22:20:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7feab6ffd700 1 --2- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab806acb0 0x7feab8070740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feaa80098f0 tx=0x7feaa8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea9fba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea4136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea9fba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ea4136f50 0x7f0ea4139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7fea9f7fe700 1 -- 172.21.15.89:0/3853322745 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8004590 con 0x7feab806acb0 2023-09-30T22:20:01.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7fea9f7fe700 1 -- 172.21.15.89:0/3853322745 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa80036a0 con 0x7feab806acb0 2023-09-30T22:20:01.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7fea9f7fe700 1 -- 172.21.15.89:0/3853322745 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8003a50 con 0x7feab806acb0 2023-09-30T22:20:01.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea9fba700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea409b4f0 con 0x7f0ea4131810 2023-09-30T22:20:01.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea9fba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea41313c0 con 0x7f0ea4136f50 2023-09-30T22:20:01.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea9fba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea406e8c0 con 0x7f0ea4134630 2023-09-30T22:20:01.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab811a580 con 0x7feab806acb0 2023-09-30T22:20:01.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea4136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab811b2b0 con 0x7feab806acb0 2023-09-30T22:20:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea4136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36396/0 (socket says 172.21.15.89:36396) 2023-09-30T22:20:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea2ffd700 1 -- 172.21.15.89:0/542775405 learned_addr learned my addr 172.21.15.89:0/542775405 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea2ffd700 1 -- 172.21.15.89:0/542775405 >> v2:172.21.15.89:3300/0 conn(0x7f0ea4136f50 msgr2=0x7f0ea4139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea2ffd700 1 --2- 172.21.15.89:0/542775405 >> v2:172.21.15.89:3300/0 conn(0x7f0ea4136f50 0x7f0ea4139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.854+0000 7f0ea2ffd700 1 -- 172.21.15.89:0/542775405 >> v1:172.21.15.89:6789/0 conn(0x7f0ea4131810 legacy=0x7f0ea4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.855+0000 7f0ea2ffd700 1 -- 172.21.15.89:0/542775405 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4071d20 con 0x7f0ea4134630 2023-09-30T22:20:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.855+0000 7f0ea2ffd700 1 --2- 172.21.15.89:0/542775405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea4136a10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0e980098f0 tx=0x7f0e98009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b164d5b3e088c63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.855+0000 7f0ea27fc700 1 -- 172.21.15.89:0/542775405 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800b8a0 con 0x7f0ea4134630 2023-09-30T22:20:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.855+0000 7f0ea27fc700 1 -- 172.21.15.89:0/542775405 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9800ba00 con 0x7f0ea4134630 2023-09-30T22:20:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.855+0000 7f0ea27fc700 1 -- 172.21.15.89:0/542775405 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800bd90 con 0x7f0ea4134630 2023-09-30T22:20:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.855+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab812b900 con 0x7feab806acb0 2023-09-30T22:20:01.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.856+0000 7fea9f7fe700 1 -- 172.21.15.89:0/3853322745 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feaa8003bb0 con 0x7feab806acb0 2023-09-30T22:20:01.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.856+0000 7fea9f7fe700 1 -- 172.21.15.89:0/3853322745 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feaa80407f0 con 0x7feab806acb0 2023-09-30T22:20:01.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.856+0000 7f0ea9fba700 1 -- 172.21.15.89:0/542775405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 msgr2=0x7f0ea4136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.856+0000 7f0ea9fba700 1 --2- 172.21.15.89:0/542775405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea4136a10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0e980098f0 tx=0x7f0e98009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:01.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.857+0000 7f0ea9fba700 1 -- 172.21.15.89:0/542775405 shutdown_connections 2023-09-30T22:20:01.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.857+0000 7f0ea9fba700 1 --2- 172.21.15.89:0/542775405 >> v2:172.21.15.89:3300/0 conn(0x7f0ea4136f50 0x7f0ea4139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.857+0000 7f0ea9fba700 1 --2- 172.21.15.89:0/542775405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea4136a10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.857+0000 7f0ea9fba700 1 -- 172.21.15.89:0/542775405 >> 172.21.15.89:0/542775405 conn(0x7f0ea40a2aa0 msgr2=0x7f0ea40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:01.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.857+0000 7f0ea9fba700 1 -- 172.21.15.89:0/542775405 shutdown_connections 2023-09-30T22:20:01.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.857+0000 7f0ea9fba700 1 -- 172.21.15.89:0/542775405 wait complete. 2023-09-30T22:20:01.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.858+0000 7f0ea9fba700 1 Processor -- start 2023-09-30T22:20:01.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.858+0000 7f0ea9fba700 1 -- start start 2023-09-30T22:20:01.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea9fba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea4131810 0x7f0ea41c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea9fba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea41c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea9fba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea4136f50 0x7f0ea4118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea9fba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea411b950 con 0x7f0ea4131810 2023-09-30T22:20:01.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea9fba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40a2060 con 0x7f0ea4136f50 2023-09-30T22:20:01.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea9fba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea406eb00 con 0x7f0ea4134630 2023-09-30T22:20:01.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea4136f50 0x7f0ea4118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea4131810 0x7f0ea41c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea4136f50 0x7f0ea4118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47290/0 (socket says 172.21.15.89:47290) 2023-09-30T22:20:01.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 -- 172.21.15.89:0/3273525434 learned_addr learned my addr 172.21.15.89:0/3273525434 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:01.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea4131810 0x7f0ea41c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59604/0 (socket says 172.21.15.89:59604) 2023-09-30T22:20:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea2ffd700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea41c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 -- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 msgr2=0x7f0ea41c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea41c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 -- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea4131810 msgr2=0x7f0ea41c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea4131810 0x7f0ea41c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 -- 172.21.15.89:0/3273525434 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e98009550 con 0x7f0ea4136f50 2023-09-30T22:20:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea37fe700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea4131810 0x7f0ea41c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.859+0000 7f0ea3fff700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea4136f50 0x7f0ea4118cd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b9e0 tx=0x7f0e9c00bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.860+0000 7f0e8bfff700 1 -- 172.21.15.89:0/3273525434 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00d8e0 con 0x7f0ea4136f50 2023-09-30T22:20:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.860+0000 7f0e8bfff700 1 -- 172.21.15.89:0/3273525434 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c00da40 con 0x7f0ea4136f50 2023-09-30T22:20:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.860+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea41c62e0 con 0x7f0ea4136f50 2023-09-30T22:20:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.860+0000 7f0e8bfff700 1 -- 172.21.15.89:0/3273525434 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00ddf0 con 0x7f0ea4136f50 2023-09-30T22:20:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.860+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea409e510 con 0x7f0ea4136f50 2023-09-30T22:20:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.861+0000 7f0e8bfff700 1 -- 172.21.15.89:0/3273525434 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e9c015420 con 0x7f0ea4136f50 2023-09-30T22:20:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.861+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea411b590 con 0x7f0ea4136f50 2023-09-30T22:20:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.861+0000 7f0e8bfff700 1 -- 172.21.15.89:0/3273525434 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0e9c011070 con 0x7f0ea4136f50 2023-09-30T22:20:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.861+0000 7fea9f7fe700 1 -- 172.21.15.89:0/3853322745 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feaa8003e60 con 0x7feab806acb0 2023-09-30T22:20:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.866+0000 7f0e8bfff700 1 -- 172.21.15.89:0/3273525434 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e9c01c900 con 0x7f0ea4136f50 2023-09-30T22:20:01.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.901+0000 7f3031949700 1 Processor -- start 2023-09-30T22:20:01.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f3031949700 1 -- start start 2023-09-30T22:20:01.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f3031949700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133420 0x7f302c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f3031949700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133e30 0x7f302c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f3031949700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c06b990 con 0x7f302c134260 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f3031949700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0a0590 con 0x7f302c133420 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f3031949700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f302c09fb80 con 0x7f302c133e30 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f302a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133e30 0x7f302c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f302a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133e30 0x7f302c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36426/0 (socket says 172.21.15.89:36426) 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.902+0000 7f302a7fc700 1 -- 172.21.15.89:0/3393907230 learned_addr learned my addr 172.21.15.89:0/3393907230 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f3029ffb700 1 -- 172.21.15.89:0/3393907230 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250498538 0 0) 0x7f302c06b990 con 0x7f302c134260 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f302affd700 1 --2- 172.21.15.89:0/3393907230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133420 0x7f302c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f3029ffb700 1 -- 172.21.15.89:0/3393907230 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30140035b0 con 0x7f302c134260 2023-09-30T22:20:01.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f302a7fc700 1 -- 172.21.15.89:0/3393907230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133420 msgr2=0x7f302c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:01.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f302a7fc700 1 --2- 172.21.15.89:0/3393907230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133420 0x7f302c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f302a7fc700 1 -- 172.21.15.89:0/3393907230 >> v1:172.21.15.89:6789/0 conn(0x7f302c134260 legacy=0x7f302c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f302a7fc700 1 -- 172.21.15.89:0/3393907230 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c0a0030 con 0x7f302c133e30 2023-09-30T22:20:01.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f302a7fc700 1 --2- 172.21.15.89:0/3393907230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133e30 0x7f302c12ff90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3020009920 tx=0x7f3020009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564e2096ed440fc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:01.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f3029ffb700 1 -- 172.21.15.89:0/3393907230 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020011830 con 0x7f302c133e30 2023-09-30T22:20:01.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f3029ffb700 1 -- 172.21.15.89:0/3393907230 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3020011990 con 0x7f302c133e30 2023-09-30T22:20:01.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.903+0000 7f3029ffb700 1 -- 172.21.15.89:0/3393907230 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020011d20 con 0x7f302c133e30 2023-09-30T22:20:01.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.904+0000 7f3031949700 1 -- 172.21.15.89:0/3393907230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133e30 msgr2=0x7f302c12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.904+0000 7f3031949700 1 --2- 172.21.15.89:0/3393907230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133e30 0x7f302c12ff90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3020009920 tx=0x7f3020009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:01.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.904+0000 7f3031949700 1 -- 172.21.15.89:0/3393907230 shutdown_connections 2023-09-30T22:20:01.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.904+0000 7f3031949700 1 --2- 172.21.15.89:0/3393907230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133420 0x7f302c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.904+0000 7f3031949700 1 --2- 172.21.15.89:0/3393907230 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133e30 0x7f302c12ff90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.904+0000 7f3031949700 1 -- 172.21.15.89:0/3393907230 >> 172.21.15.89:0/3393907230 conn(0x7f302c0a4210 msgr2=0x7f302c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.904+0000 7f3031949700 1 -- 172.21.15.89:0/3393907230 shutdown_connections 2023-09-30T22:20:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.904+0000 7f3031949700 1 -- 172.21.15.89:0/3393907230 wait complete. 2023-09-30T22:20:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.905+0000 7f3031949700 1 Processor -- start 2023-09-30T22:20:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f3031949700 1 -- start start 2023-09-30T22:20:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f3031949700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133420 0x7f302c09c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f3031949700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133e30 0x7f302c09cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f3031949700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302c134260 0x7f302c09d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f3031949700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c09e2e0 con 0x7f302c134260 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f3031949700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c12db90 con 0x7f302c133e30 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f3031949700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1209e0 con 0x7f302c133420 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f302affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133420 0x7f302c09c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f302affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133420 0x7f302c09c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36434/0 (socket says 172.21.15.89:36434) 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f302affd700 1 -- 172.21.15.89:0/2103159943 learned_addr learned my addr 172.21.15.89:0/2103159943 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.906+0000 7f302b7fe700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302c134260 0x7f302c09d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f302a7fc700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133e30 0x7f302c09cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:01.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f302b7fe700 1 -- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133420 msgr2=0x7f302c09c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f302b7fe700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133420 0x7f302c09c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f302b7fe700 1 -- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133e30 msgr2=0x7f302c09cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f302b7fe700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133e30 0x7f302c09cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f302b7fe700 1 -- 172.21.15.89:0/2103159943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020009550 con 0x7f302c134260 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f302affd700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133420 0x7f302c09c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f302b7fe700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302c134260 0x7f302c09d160 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f302400a480 tx=0x7f302400a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f3013fff700 1 -- 172.21.15.89:0/2103159943 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024003a00 con 0x7f302c134260 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f3013fff700 1 -- 172.21.15.89:0/2103159943 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024003b60 con 0x7f302c134260 2023-09-30T22:20:01.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f3013fff700 1 -- 172.21.15.89:0/2103159943 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240050d0 con 0x7f302c134260 2023-09-30T22:20:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c120250 con 0x7f302c134260 2023-09-30T22:20:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.907+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c11a680 con 0x7f302c134260 2023-09-30T22:20:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.908+0000 7f3013fff700 1 -- 172.21.15.89:0/2103159943 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3024005230 con 0x7f302c134260 2023-09-30T22:20:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.909+0000 7f3013fff700 1 -- 172.21.15.89:0/2103159943 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f30240056c0 con 0x7f302c134260 2023-09-30T22:20:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.909+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c09f6b0 con 0x7f302c134260 2023-09-30T22:20:01.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.918+0000 7f3013fff700 1 -- 172.21.15.89:0/2103159943 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f30240054e0 con 0x7f302c134260 2023-09-30T22:20:01.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.931+0000 7f333effd700 1 -- 172.21.15.89:0/1723880812 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f332c005590 con 0x7f335409de50 2023-09-30T22:20:01.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.931+0000 7f33517fa700 1 -- 172.21.15.89:0/1723880812 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3348006ce0 con 0x7f335409de50 2023-09-30T22:20:01.932 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:01.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.933+0000 7f333effd700 1 -- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 msgr2=0x7f335411eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:01.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.933+0000 7f333effd700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335411eb00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3348009c40 tx=0x7f3348004750 comp rx=0 tx=0).stop 2023-09-30T22:20:01.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.933+0000 7f333effd700 1 -- 172.21.15.89:0/1723880812 shutdown_connections 2023-09-30T22:20:01.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.933+0000 7f333effd700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33541338a0 0x7f335411f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.933+0000 7f333effd700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335409de50 0x7f335411eb00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.933+0000 7f333effd700 1 --2- 172.21.15.89:0/1723880812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335409e760 0x7f335411f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:01.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.933+0000 7f333effd700 1 -- 172.21.15.89:0/1723880812 >> 172.21.15.89:0/1723880812 conn(0x7f33540a2aa0 msgr2=0x7f335409ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:01.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.934+0000 7f333effd700 1 -- 172.21.15.89:0/1723880812 shutdown_connections 2023-09-30T22:20:01.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:01.934+0000 7f333effd700 1 -- 172.21.15.89:0/1723880812 wait complete. 2023-09-30T22:20:01.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.067+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0ea411b0e0 con 0x7f0ea4136f50 2023-09-30T22:20:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.068+0000 7f0e8bfff700 1 -- 172.21.15.89:0/3273525434 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e9c01c310 con 0x7f0ea4136f50 2023-09-30T22:20:02.069 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea4136f50 msgr2=0x7f0ea4118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea4136f50 0x7f0ea4118cd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b9e0 tx=0x7f0e9c00bcf0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 shutdown_connections 2023-09-30T22:20:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea4136f50 0x7f0ea4118cd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea4131810 0x7f0ea41c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 --2- 172.21.15.89:0/3273525434 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea4134630 0x7f0ea41c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 >> 172.21.15.89:0/3273525434 conn(0x7f0ea40a2aa0 msgr2=0x7f0ea412e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 shutdown_connections 2023-09-30T22:20:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.070+0000 7f0ea9fba700 1 -- 172.21.15.89:0/3273525434 wait complete. 2023-09-30T22:20:02.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:02.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.113+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feab8074ea0 con 0x7feab806acb0 2023-09-30T22:20:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.114+0000 7fea9f7fe700 1 -- 172.21.15.89:0/3853322745 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feaa8015030 con 0x7feab806acb0 2023-09-30T22:20:02.115 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:02.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab806acb0 msgr2=0x7feab8070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:02.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 --2- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab806acb0 0x7feab8070740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feaa80098f0 tx=0x7feaa8004750 comp rx=0 tx=0).stop 2023-09-30T22:20:02.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 shutdown_connections 2023-09-30T22:20:02.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 --2- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab81316c0 0x7feab80711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 --2- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feab806acb0 0x7feab8070740 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 --2- 172.21.15.89:0/3853322745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feab809d9f0 0x7feab8070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 >> 172.21.15.89:0/3853322745 conn(0x7feab8066f50 msgr2=0x7feab80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:02.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 shutdown_connections 2023-09-30T22:20:02.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.116+0000 7feabd32c700 1 -- 172.21.15.89:0/3853322745 wait complete. 2023-09-30T22:20:02.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:02.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.153+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f302c074ea0 con 0x7f302c134260 2023-09-30T22:20:02.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.154+0000 7f3013fff700 1 -- 172.21.15.89:0/2103159943 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3024023b00 con 0x7f302c134260 2023-09-30T22:20:02.155 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:02.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302c134260 msgr2=0x7f302c09d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:02.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302c134260 0x7f302c09d160 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f302400a480 tx=0x7f302400a790 comp rx=0 tx=0).stop 2023-09-30T22:20:02.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 shutdown_connections 2023-09-30T22:20:02.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302c134260 0x7f302c09d160 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302c133420 0x7f302c09c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 --2- 172.21.15.89:0/2103159943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302c133e30 0x7f302c09cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:02.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 >> 172.21.15.89:0/2103159943 conn(0x7f302c0a4210 msgr2=0x7f302c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:02.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 shutdown_connections 2023-09-30T22:20:02.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:02.156+0000 7f3031949700 1 -- 172.21.15.89:0/2103159943 wait complete. 2023-09-30T22:20:02.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:02.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:03.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.076+0000 7efcdcccb700 1 Processor -- start 2023-09-30T22:20:03.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.076+0000 7efcdcccb700 1 -- start start 2023-09-30T22:20:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcdcccb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809d6f0 0x7efcd809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcdcccb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcd809e000 0x7efcd81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcdcccb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809e410 0x7efcd8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcdcccb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd806b460 con 0x7efcd809e000 2023-09-30T22:20:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcdcccb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd809b120 con 0x7efcd809e410 2023-09-30T22:20:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcdcccb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcd809a7e0 con 0x7efcd809d6f0 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd5d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcd809e000 0x7efcd81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809d6f0 0x7efcd809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809e410 0x7efcd8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd5d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcd809e000 0x7efcd81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59632/0 (socket says 172.21.15.89:59632) 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd5d9b700 1 -- 172.21.15.89:0/1684199496 learned_addr learned my addr 172.21.15.89:0/1684199496 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd6d9d700 1 -- 172.21.15.89:0/1684199496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809d6f0 msgr2=0x7efcd809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd6d9d700 1 --2- 172.21.15.89:0/1684199496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809d6f0 0x7efcd809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd6d9d700 1 -- 172.21.15.89:0/1684199496 >> v2:172.21.15.89:3300/0 conn(0x7efcd809e000 msgr2=0x7efcd81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd6d9d700 1 --2- 172.21.15.89:0/1684199496 >> v2:172.21.15.89:3300/0 conn(0x7efcd809e000 0x7efcd81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd6d9d700 1 -- 172.21.15.89:0/1684199496 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd809a500 con 0x7efcd809e410 2023-09-30T22:20:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd5d9b700 1 --2- 172.21.15.89:0/1684199496 >> v2:172.21.15.89:3300/0 conn(0x7efcd809e000 0x7efcd81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd659c700 1 --2- 172.21.15.89:0/1684199496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809d6f0 0x7efcd809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd6d9d700 1 --2- 172.21.15.89:0/1684199496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809e410 0x7efcd8133f30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009dc0 tx=0x7efcd000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=681d865234467cbe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd559a700 1 -- 172.21.15.89:0/1684199496 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000e070 con 0x7efcd809e410 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd559a700 1 -- 172.21.15.89:0/1684199496 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd000bae0 con 0x7efcd809e410 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.077+0000 7efcd559a700 1 -- 172.21.15.89:0/1684199496 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000be90 con 0x7efcd809e410 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 -- 172.21.15.89:0/1684199496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809e410 msgr2=0x7efcd8133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 --2- 172.21.15.89:0/1684199496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809e410 0x7efcd8133f30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009dc0 tx=0x7efcd000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 -- 172.21.15.89:0/1684199496 shutdown_connections 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 --2- 172.21.15.89:0/1684199496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809e410 0x7efcd8133f30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 --2- 172.21.15.89:0/1684199496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809d6f0 0x7efcd809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 --2- 172.21.15.89:0/1684199496 >> v2:172.21.15.89:3300/0 conn(0x7efcd809e000 0x7efcd81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 -- 172.21.15.89:0/1684199496 >> 172.21.15.89:0/1684199496 conn(0x7efcd8066f50 msgr2=0x7efcd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 -- 172.21.15.89:0/1684199496 shutdown_connections 2023-09-30T22:20:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.078+0000 7efcdcccb700 1 -- 172.21.15.89:0/1684199496 wait complete. 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.079+0000 7efcdcccb700 1 Processor -- start 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.079+0000 7efcdcccb700 1 -- start start 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.079+0000 7efcdcccb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809d6f0 0x7efcd81c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcdcccb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809e000 0x7efcd81c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcdcccb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd809e410 0x7efcd81c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcdcccb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd81a9040 con 0x7efcd809e410 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcdcccb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd81a7160 con 0x7efcd809d6f0 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcdcccb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcd81c5aa0 con 0x7efcd809e000 2023-09-30T22:20:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809d6f0 0x7efcd81c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd809e410 0x7efcd81c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809d6f0 0x7efcd81c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47340/0 (socket says 172.21.15.89:47340) 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd659c700 1 -- 172.21.15.89:0/3181953889 learned_addr learned my addr 172.21.15.89:0/3181953889 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd5d9b700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809e000 0x7efcd81c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd6d9d700 1 -- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809e000 msgr2=0x7efcd81c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd6d9d700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809e000 0x7efcd81c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd6d9d700 1 -- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809d6f0 msgr2=0x7efcd81c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd6d9d700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809d6f0 0x7efcd81c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd6d9d700 1 -- 172.21.15.89:0/3181953889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc0094a0 con 0x7efcd809e410 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd659c700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809d6f0 0x7efcd81c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcd6d9d700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd809e410 0x7efcd81c8040 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efcd0016000 tx=0x7efcd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcc6ffd700 1 -- 172.21.15.89:0/3181953889 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004550 con 0x7efcd809e410 2023-09-30T22:20:03.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcc6ffd700 1 -- 172.21.15.89:0/3181953889 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0020070 con 0x7efcd809e410 2023-09-30T22:20:03.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc8009550 con 0x7efcd809e410 2023-09-30T22:20:03.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcc6ffd700 1 -- 172.21.15.89:0/3181953889 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004950 con 0x7efcd809e410 2023-09-30T22:20:03.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.080+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd0009a70 con 0x7efcd809e410 2023-09-30T22:20:03.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.081+0000 7efcc6ffd700 1 -- 172.21.15.89:0/3181953889 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efcd0027480 con 0x7efcd809e410 2023-09-30T22:20:03.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.081+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd80a1ef0 con 0x7efcd809e410 2023-09-30T22:20:03.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.081+0000 7efcc6ffd700 1 -- 172.21.15.89:0/3181953889 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efcd0022070 con 0x7efcd809e410 2023-09-30T22:20:03.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:03.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.086+0000 7efcc6ffd700 1 -- 172.21.15.89:0/3181953889 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efcd0011980 con 0x7efcd809e410 2023-09-30T22:20:03.130 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:03.170 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:03.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.213+0000 7efc140e2700 1 Processor -- start 2023-09-30T22:20:03.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.213+0000 7efc140e2700 1 -- start start 2023-09-30T22:20:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.213+0000 7efc140e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09d6f0 0x7efc0c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.213+0000 7efc140e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e410 0x7efc0c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.213+0000 7efc140e2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0c09b120 con 0x7efc0c09e000 2023-09-30T22:20:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.213+0000 7efc140e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c06b460 con 0x7efc0c09e410 2023-09-30T22:20:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.213+0000 7efc140e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c09a7e0 con 0x7efc0c09d6f0 2023-09-30T22:20:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc11e7e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09d6f0 0x7efc0c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc11e7e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09d6f0 0x7efc0c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36448/0 (socket says 172.21.15.89:36448) 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc11e7e700 1 -- 172.21.15.89:0/3816225272 learned_addr learned my addr 172.21.15.89:0/3816225272 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc1267f700 1 --2- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e410 0x7efc0c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc11e7e700 1 -- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e410 msgr2=0x7efc0c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc11e7e700 1 --2- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e410 0x7efc0c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc11e7e700 1 -- 172.21.15.89:0/3816225272 >> v1:172.21.15.89:6789/0 conn(0x7efc0c09e000 legacy=0x7efc0c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc11e7e700 1 -- 172.21.15.89:0/3816225272 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c09a500 con 0x7efc0c09d6f0 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc1267f700 1 --2- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e410 0x7efc0c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc11e7e700 1 --2- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09d6f0 0x7efc0c09dac0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efc08009950 tx=0x7efc08009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f518370440f353 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc10e7c700 1 -- 172.21.15.89:0/3816225272 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08009ea0 con 0x7efc0c09d6f0 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc10e7c700 1 -- 172.21.15.89:0/3816225272 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0800ba20 con 0x7efc0c09d6f0 2023-09-30T22:20:03.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.214+0000 7efc10e7c700 1 -- 172.21.15.89:0/3816225272 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800bdb0 con 0x7efc0c09d6f0 2023-09-30T22:20:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.215+0000 7efc140e2700 1 -- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09d6f0 msgr2=0x7efc0c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.215+0000 7efc140e2700 1 --2- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09d6f0 0x7efc0c09dac0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efc08009950 tx=0x7efc08009c60 comp rx=0 tx=0).stop 2023-09-30T22:20:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.215+0000 7efc140e2700 1 -- 172.21.15.89:0/3816225272 shutdown_connections 2023-09-30T22:20:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.215+0000 7efc140e2700 1 --2- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e410 0x7efc0c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.215+0000 7efc140e2700 1 --2- 172.21.15.89:0/3816225272 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09d6f0 0x7efc0c09dac0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.215+0000 7efc140e2700 1 -- 172.21.15.89:0/3816225272 >> 172.21.15.89:0/3816225272 conn(0x7efc0c066f50 msgr2=0x7efc0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.216+0000 7efc140e2700 1 -- 172.21.15.89:0/3816225272 shutdown_connections 2023-09-30T22:20:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.216+0000 7efc140e2700 1 -- 172.21.15.89:0/3816225272 wait complete. 2023-09-30T22:20:03.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.216+0000 7efc140e2700 1 Processor -- start 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc140e2700 1 -- start start 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc140e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc0c09d6f0 0x7efc0c1224c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc140e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e000 0x7efc0c122a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc140e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09e410 0x7efc0c122f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc140e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c1338d0 con 0x7efc0c09d6f0 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc140e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c124f10 con 0x7efc0c09e000 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc140e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c125f90 con 0x7efc0c09e410 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc1267f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09e410 0x7efc0c122f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc1267f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09e410 0x7efc0c122f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36456/0 (socket says 172.21.15.89:36456) 2023-09-30T22:20:03.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc1267f700 1 -- 172.21.15.89:0/4190969718 learned_addr learned my addr 172.21.15.89:0/4190969718 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:03.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc11e7e700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc0c09d6f0 0x7efc0c1224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.217+0000 7efc1167d700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e000 0x7efc0c122a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc1267f700 1 -- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e000 msgr2=0x7efc0c122a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc1267f700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e000 0x7efc0c122a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc1267f700 1 -- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc0c09d6f0 msgr2=0x7efc0c1224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc1267f700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc0c09d6f0 0x7efc0c1224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc1267f700 1 -- 172.21.15.89:0/4190969718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08009550 con 0x7efc0c09e410 2023-09-30T22:20:03.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc11e7e700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc0c09d6f0 0x7efc0c1224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:03.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc1267f700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09e410 0x7efc0c122f40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc0400cef0 tx=0x7efc0400cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:03.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc027fc700 1 -- 172.21.15.89:0/4190969718 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0400ad00 con 0x7efc0c09e410 2023-09-30T22:20:03.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc027fc700 1 -- 172.21.15.89:0/4190969718 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0400ae60 con 0x7efc0c09e410 2023-09-30T22:20:03.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc027fc700 1 -- 172.21.15.89:0/4190969718 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04015680 con 0x7efc0c09e410 2023-09-30T22:20:03.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0c09c710 con 0x7efc0c09e410 2023-09-30T22:20:03.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.218+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0c0a1df0 con 0x7efc0c09e410 2023-09-30T22:20:03.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.219+0000 7efc027fc700 1 -- 172.21.15.89:0/4190969718 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc0400d430 con 0x7efc0c09e410 2023-09-30T22:20:03.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.219+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0c074ea0 con 0x7efc0c09e410 2023-09-30T22:20:03.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.219+0000 7efc027fc700 1 -- 172.21.15.89:0/4190969718 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efc0403fb70 con 0x7efc0c09e410 2023-09-30T22:20:03.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.228+0000 7efc027fc700 1 -- 172.21.15.89:0/4190969718 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc0401e3b0 con 0x7efc0c09e410 2023-09-30T22:20:03.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.262+0000 7f23be927700 1 Processor -- start 2023-09-30T22:20:03.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.262+0000 7f23be927700 1 -- start start 2023-09-30T22:20:03.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23be927700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23b809f410 0x7f23b8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23be927700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23be927700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b8070940 con 0x7f23b809e930 2023-09-30T22:20:03.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23be927700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b806fd60 con 0x7f23b809f410 2023-09-30T22:20:03.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23be927700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23b806fa80 con 0x7f23b8136fe0 2023-09-30T22:20:03.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23bcec4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23b77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23b809f410 0x7f23b8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23bcec4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36464/0 (socket says 172.21.15.89:36464) 2023-09-30T22:20:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23bcec4700 1 -- 172.21.15.89:0/379210542 learned_addr learned my addr 172.21.15.89:0/379210542 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23bcec4700 1 -- 172.21.15.89:0/379210542 >> v2:172.21.15.89:3300/0 conn(0x7f23b809f410 msgr2=0x7f23b8136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23bcec4700 1 --2- 172.21.15.89:0/379210542 >> v2:172.21.15.89:3300/0 conn(0x7f23b809f410 0x7f23b8136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23bcec4700 1 -- 172.21.15.89:0/379210542 >> v1:172.21.15.89:6789/0 conn(0x7f23b809e930 legacy=0x7f23b809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23bcec4700 1 -- 172.21.15.89:0/379210542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b809fb60 con 0x7f23b8136fe0 2023-09-30T22:20:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.263+0000 7f23bcec4700 1 --2- 172.21.15.89:0/379210542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81393c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009e50 tx=0x7f23b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d0aa312b699259 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.264+0000 7f23b6ffd700 1 -- 172.21.15.89:0/379210542 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b000e070 con 0x7f23b8136fe0 2023-09-30T22:20:03.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.264+0000 7f23b6ffd700 1 -- 172.21.15.89:0/379210542 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b000b910 con 0x7f23b8136fe0 2023-09-30T22:20:03.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.264+0000 7f23b6ffd700 1 -- 172.21.15.89:0/379210542 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b000bcc0 con 0x7f23b8136fe0 2023-09-30T22:20:03.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.264+0000 7f23be927700 1 -- 172.21.15.89:0/379210542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 msgr2=0x7f23b81393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.264+0000 7f23be927700 1 --2- 172.21.15.89:0/379210542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81393c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009e50 tx=0x7f23b000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:03.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.265+0000 7f23be927700 1 -- 172.21.15.89:0/379210542 shutdown_connections 2023-09-30T22:20:03.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.265+0000 7f23be927700 1 --2- 172.21.15.89:0/379210542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81393c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.265+0000 7f23be927700 1 --2- 172.21.15.89:0/379210542 >> v2:172.21.15.89:3300/0 conn(0x7f23b809f410 0x7f23b8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.265+0000 7f23be927700 1 -- 172.21.15.89:0/379210542 >> 172.21.15.89:0/379210542 conn(0x7f23b80a4210 msgr2=0x7f23b80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.265+0000 7f23be927700 1 -- 172.21.15.89:0/379210542 shutdown_connections 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.265+0000 7f23be927700 1 -- 172.21.15.89:0/379210542 wait complete. 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.265+0000 7f23be927700 1 Processor -- start 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23be927700 1 -- start start 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23be927700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b809e930 0x7f23b806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23be927700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23b809f410 0x7f23b806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23be927700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23be927700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b809df30 con 0x7f23b809e930 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23be927700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b81322e0 con 0x7f23b809f410 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23be927700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23b806cd00 con 0x7f23b8136fe0 2023-09-30T22:20:03.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23b7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b809e930 0x7f23b806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23bcec4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23b7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b809e930 0x7f23b806c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59666/0 (socket says 172.21.15.89:59666) 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23b7fff700 1 -- 172.21.15.89:0/2125228350 learned_addr learned my addr 172.21.15.89:0/2125228350 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.266+0000 7f23b77fe700 1 --2- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23b809f410 0x7f23b806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23bcec4700 1 -- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23b809f410 msgr2=0x7f23b806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23bcec4700 1 --2- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23b809f410 0x7f23b806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23bcec4700 1 -- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b809e930 msgr2=0x7f23b806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23bcec4700 1 --2- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b809e930 0x7f23b806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23bcec4700 1 -- 172.21.15.89:0/2125228350 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b0009b30 con 0x7f23b8136fe0 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23bcec4700 1 --2- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81c8360 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f23b0015000 tx=0x7f23b0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23b4ff9700 1 -- 172.21.15.89:0/2125228350 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0004800 con 0x7f23b8136fe0 2023-09-30T22:20:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23b4ff9700 1 -- 172.21.15.89:0/2125228350 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0004960 con 0x7f23b8136fe0 2023-09-30T22:20:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b806bd70 con 0x7f23b8136fe0 2023-09-30T22:20:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23b4ff9700 1 -- 172.21.15.89:0/2125228350 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b001c650 con 0x7f23b8136fe0 2023-09-30T22:20:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.267+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b81324b0 con 0x7f23b8136fe0 2023-09-30T22:20:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.268+0000 7f23b4ff9700 1 -- 172.21.15.89:0/2125228350 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f23b001c7b0 con 0x7f23b8136fe0 2023-09-30T22:20:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.268+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b8120dc0 con 0x7f23b8136fe0 2023-09-30T22:20:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.269+0000 7f23b4ff9700 1 -- 172.21.15.89:0/2125228350 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f23b0040dd0 con 0x7f23b8136fe0 2023-09-30T22:20:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.277+0000 7f23b4ff9700 1 -- 172.21.15.89:0/2125228350 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23b001ca60 con 0x7f23b8136fe0 2023-09-30T22:20:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.312+0000 7fbdefdb2700 1 Processor -- start 2023-09-30T22:20:03.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.312+0000 7fbdefdb2700 1 -- start start 2023-09-30T22:20:03.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbdefdb2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbde8133420 0x7fbde81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbdefdb2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8133e30 0x7fbde812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbdefdb2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde806b990 con 0x7fbde8134260 2023-09-30T22:20:03.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbdefdb2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde80a0590 con 0x7fbde8133420 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbdefdb2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde809fb80 con 0x7fbde8133e30 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbded34d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8133e30 0x7fbde812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbdedb4e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbde8133420 0x7fbde81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbded34d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8133e30 0x7fbde812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47370/0 (socket says 172.21.15.89:47370) 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbded34d700 1 -- 172.21.15.89:0/2550345435 learned_addr learned my addr 172.21.15.89:0/2550345435 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbded34d700 1 -- 172.21.15.89:0/2550345435 >> v2:172.21.15.89:3300/0 conn(0x7fbde8133420 msgr2=0x7fbde81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbded34d700 1 --2- 172.21.15.89:0/2550345435 >> v2:172.21.15.89:3300/0 conn(0x7fbde8133420 0x7fbde81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbded34d700 1 -- 172.21.15.89:0/2550345435 >> v1:172.21.15.89:6789/0 conn(0x7fbde8134260 legacy=0x7fbde81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.313+0000 7fbded34d700 1 -- 172.21.15.89:0/2550345435 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80a0030 con 0x7fbde8133e30 2023-09-30T22:20:03.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbded34d700 1 --2- 172.21.15.89:0/2550345435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8133e30 0x7fbde812ff90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80098f0 tx=0x7fbdd8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf65e370f6d9632e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdecb4c700 1 -- 172.21.15.89:0/2550345435 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8011830 con 0x7fbde8133e30 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdecb4c700 1 -- 172.21.15.89:0/2550345435 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd8011990 con 0x7fbde8133e30 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdecb4c700 1 -- 172.21.15.89:0/2550345435 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8011d20 con 0x7fbde8133e30 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdefdb2700 1 -- 172.21.15.89:0/2550345435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8133e30 msgr2=0x7fbde812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdefdb2700 1 --2- 172.21.15.89:0/2550345435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8133e30 0x7fbde812ff90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80098f0 tx=0x7fbdd8009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdefdb2700 1 -- 172.21.15.89:0/2550345435 shutdown_connections 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdefdb2700 1 --2- 172.21.15.89:0/2550345435 >> v2:172.21.15.89:3300/0 conn(0x7fbde8133420 0x7fbde81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdefdb2700 1 --2- 172.21.15.89:0/2550345435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8133e30 0x7fbde812ff90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdefdb2700 1 -- 172.21.15.89:0/2550345435 >> 172.21.15.89:0/2550345435 conn(0x7fbde80a4210 msgr2=0x7fbde80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.314+0000 7fbdefdb2700 1 -- 172.21.15.89:0/2550345435 shutdown_connections 2023-09-30T22:20:03.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.315+0000 7fbdefdb2700 1 -- 172.21.15.89:0/2550345435 wait complete. 2023-09-30T22:20:03.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.316+0000 7fbdefdb2700 1 Processor -- start 2023-09-30T22:20:03.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.316+0000 7fbdefdb2700 1 -- start start 2023-09-30T22:20:03.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.316+0000 7fbdefdb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbde8133420 0x7fbde809c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.316+0000 7fbdefdb2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbde8133e30 0x7fbde809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.316+0000 7fbdefdb2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8134260 0x7fbde809d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:03.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.316+0000 7fbdefdb2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde811ffb0 con 0x7fbde8133420 2023-09-30T22:20:03.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.316+0000 7fbdefdb2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81203e0 con 0x7fbde8134260 2023-09-30T22:20:03.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.316+0000 7fbdefdb2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbde809dbb0 con 0x7fbde8133e30 2023-09-30T22:20:03.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbde8133420 0x7fbde809c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbde8133420 0x7fbde809c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59682/0 (socket says 172.21.15.89:59682) 2023-09-30T22:20:03.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 -- 172.21.15.89:0/1263096707 learned_addr learned my addr 172.21.15.89:0/1263096707 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:03.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbded34d700 1 --2- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbde8133e30 0x7fbde809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:03.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 -- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbde8133e30 msgr2=0x7fbde809cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 --2- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbde8133e30 0x7fbde809cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 -- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8134260 msgr2=0x7fbde809d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 --2- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8134260 0x7fbde809d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 -- 172.21.15.89:0/1263096707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd8009550 con 0x7fbde8133420 2023-09-30T22:20:03.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdedb4e700 1 --2- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbde8133420 0x7fbde809c6e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbde400e9e0 tx=0x7fbde400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:03.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdde7fc700 1 -- 172.21.15.89:0/1263096707 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4009710 con 0x7fbde8133420 2023-09-30T22:20:03.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdde7fc700 1 -- 172.21.15.89:0/1263096707 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde400c850 con 0x7fbde8133420 2023-09-30T22:20:03.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde811fcf0 con 0x7fbde8133420 2023-09-30T22:20:03.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdde7fc700 1 -- 172.21.15.89:0/1263096707 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde400cbc0 con 0x7fbde8133420 2023-09-30T22:20:03.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.317+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde8120a20 con 0x7fbde8133420 2023-09-30T22:20:03.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.318+0000 7fbdde7fc700 1 -- 172.21.15.89:0/1263096707 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbde400cd20 con 0x7fbde8133420 2023-09-30T22:20:03.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.318+0000 7fbdde7fc700 1 -- 172.21.15.89:0/1263096707 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbde403fc40 con 0x7fbde8133420 2023-09-30T22:20:03.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.319+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde8074ea0 con 0x7fbde8133420 2023-09-30T22:20:03.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.320+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efcd8074ea0 con 0x7efcd809e410 2023-09-30T22:20:03.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.328+0000 7efcc6ffd700 1 -- 172.21.15.89:0/3181953889 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efcd00303f0 con 0x7efcd809e410 2023-09-30T22:20:03.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.328+0000 7fbdde7fc700 1 -- 172.21.15.89:0/1263096707 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbde4021080 con 0x7fbde8133420 2023-09-30T22:20:03.330 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:03.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.331+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd809e410 msgr2=0x7efcd81c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.331+0000 7efcdcccb700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd809e410 0x7efcd81c8040 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efcd0016000 tx=0x7efcd0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.331+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 shutdown_connections 2023-09-30T22:20:03.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.331+0000 7efcdcccb700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd809e410 0x7efcd81c8040 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.331+0000 7efcdcccb700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcd809d6f0 0x7efcd81c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.331+0000 7efcdcccb700 1 --2- 172.21.15.89:0/3181953889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efcd809e000 0x7efcd81c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.331+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 >> 172.21.15.89:0/3181953889 conn(0x7efcd8066f50 msgr2=0x7efcd81bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:03.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.332+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 shutdown_connections 2023-09-30T22:20:03.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.332+0000 7efcdcccb700 1 -- 172.21.15.89:0/3181953889 wait complete. 2023-09-30T22:20:03.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:03.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.451+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efc0c06c360 con 0x7efc0c09e410 2023-09-30T22:20:03.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.451+0000 7efc027fc700 1 -- 172.21.15.89:0/4190969718 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc0401a030 con 0x7efc0c09e410 2023-09-30T22:20:03.452 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09e410 msgr2=0x7efc0c122f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09e410 0x7efc0c122f40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc0400cef0 tx=0x7efc0400cf20 comp rx=0 tx=0).stop 2023-09-30T22:20:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 shutdown_connections 2023-09-30T22:20:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efc0c09e410 0x7efc0c122f40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc0c09d6f0 0x7efc0c1224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 --2- 172.21.15.89:0/4190969718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc0c09e000 0x7efc0c122a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 >> 172.21.15.89:0/4190969718 conn(0x7efc0c066f50 msgr2=0x7efc0c0711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 shutdown_connections 2023-09-30T22:20:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.453+0000 7efc140e2700 1 -- 172.21.15.89:0/4190969718 wait complete. 2023-09-30T22:20:03.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:03.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.502+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f23b8074ea0 con 0x7f23b8136fe0 2023-09-30T22:20:03.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.502+0000 7f23b4ff9700 1 -- 172.21.15.89:0/2125228350 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23b0029070 con 0x7f23b8136fe0 2023-09-30T22:20:03.503 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:03.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 msgr2=0x7f23b81c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 --2- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81c8360 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f23b0015000 tx=0x7f23b0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 shutdown_connections 2023-09-30T22:20:03.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 --2- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23b8136fe0 0x7f23b81c8360 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 --2- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23b809e930 0x7f23b806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 --2- 172.21.15.89:0/2125228350 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23b809f410 0x7f23b806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 >> 172.21.15.89:0/2125228350 conn(0x7f23b80a4210 msgr2=0x7f23b8134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:03.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 shutdown_connections 2023-09-30T22:20:03.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.504+0000 7f23be927700 1 -- 172.21.15.89:0/2125228350 wait complete. 2023-09-30T22:20:03.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.534+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbde81201d0 con 0x7fbde8133420 2023-09-30T22:20:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.534+0000 7fbdde7fc700 1 -- 172.21.15.89:0/1263096707 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbde4017020 con 0x7fbde8133420 2023-09-30T22:20:03.535 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.536+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbde8133420 msgr2=0x7fbde809c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.536+0000 7fbdefdb2700 1 --2- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbde8133420 0x7fbde809c6e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbde400e9e0 tx=0x7fbde400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.536+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 shutdown_connections 2023-09-30T22:20:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.536+0000 7fbdefdb2700 1 --2- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbde8134260 0x7fbde809d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.536+0000 7fbdefdb2700 1 --2- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbde8133420 0x7fbde809c6e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.536+0000 7fbdefdb2700 1 --2- 172.21.15.89:0/1263096707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbde8133e30 0x7fbde809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.536+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 >> 172.21.15.89:0/1263096707 conn(0x7fbde80a4210 msgr2=0x7fbde80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.537+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 shutdown_connections 2023-09-30T22:20:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:03.537+0000 7fbdefdb2700 1 -- 172.21.15.89:0/1263096707 wait complete. 2023-09-30T22:20:03.549 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:04.351 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:04.467 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f4256cb7700 1 Processor -- start 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f4256cb7700 1 -- start start 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f4256cb7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f425012cd70 0x7f425012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f4256cb7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4250135c10 0x7f4250137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f4256cb7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425009a870 con 0x7f425012c9a0 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f4256cb7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425009b890 con 0x7f4250135c10 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f4256cb7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425009b2f0 con 0x7f425012cd70 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f424ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f425012cd70 0x7f425012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f4255254700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4250135c10 0x7f4250137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f424ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f425012cd70 0x7f425012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47380/0 (socket says 172.21.15.89:47380) 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.486+0000 7f424ffff700 1 -- 172.21.15.89:0/1126287786 learned_addr learned my addr 172.21.15.89:0/1126287786 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f424ffff700 1 -- 172.21.15.89:0/1126287786 >> v2:172.21.15.89:3300/0 conn(0x7f4250135c10 msgr2=0x7f4250137ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f424ffff700 1 --2- 172.21.15.89:0/1126287786 >> v2:172.21.15.89:3300/0 conn(0x7f4250135c10 0x7f4250137ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f424ffff700 1 -- 172.21.15.89:0/1126287786 >> v1:172.21.15.89:6789/0 conn(0x7f425012c9a0 legacy=0x7f425012f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f424ffff700 1 -- 172.21.15.89:0/1126287786 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425009a170 con 0x7f425012cd70 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f4255254700 1 --2- 172.21.15.89:0/1126287786 >> v2:172.21.15.89:3300/0 conn(0x7f4250135c10 0x7f4250137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f424ffff700 1 --2- 172.21.15.89:0/1126287786 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f425012cd70 0x7f425012ff90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f423c00aac0 tx=0x7f423c00add0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=839d70391bf1ee06 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f424f7fe700 1 -- 172.21.15.89:0/1126287786 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c00cb30 con 0x7f425012cd70 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f424f7fe700 1 -- 172.21.15.89:0/1126287786 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c00cc90 con 0x7f425012cd70 2023-09-30T22:20:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f424f7fe700 1 -- 172.21.15.89:0/1126287786 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c00f470 con 0x7f425012cd70 2023-09-30T22:20:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f4256cb7700 1 -- 172.21.15.89:0/1126287786 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f425012cd70 msgr2=0x7f425012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f4256cb7700 1 --2- 172.21.15.89:0/1126287786 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f425012cd70 0x7f425012ff90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f423c00aac0 tx=0x7f423c00add0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f4256cb7700 1 -- 172.21.15.89:0/1126287786 shutdown_connections 2023-09-30T22:20:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f4256cb7700 1 --2- 172.21.15.89:0/1126287786 >> v2:172.21.15.89:3300/0 conn(0x7f4250135c10 0x7f4250137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f4256cb7700 1 --2- 172.21.15.89:0/1126287786 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f425012cd70 0x7f425012ff90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.487+0000 7f4256cb7700 1 -- 172.21.15.89:0/1126287786 >> 172.21.15.89:0/1126287786 conn(0x7f42500a2360 msgr2=0x7f42500a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.488+0000 7f4256cb7700 1 -- 172.21.15.89:0/1126287786 shutdown_connections 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.488+0000 7f4256cb7700 1 -- 172.21.15.89:0/1126287786 wait complete. 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.488+0000 7f4256cb7700 1 Processor -- start 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4256cb7700 1 -- start start 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4256cb7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f425012c9a0 0x7f425012abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4256cb7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f425012cd70 0x7f425012b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4256cb7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4250135c10 0x7f425012b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4256cb7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425006f0d0 con 0x7f425012cd70 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4256cb7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42501306c0 con 0x7f4250135c10 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4256cb7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f425006db30 con 0x7f425012c9a0 2023-09-30T22:20:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f424ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f425012cd70 0x7f425012b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4254a53700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f425012c9a0 0x7f425012abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f424ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f425012cd70 0x7f425012b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59704/0 (socket says 172.21.15.89:59704) 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f424ffff700 1 -- 172.21.15.89:0/194950286 learned_addr learned my addr 172.21.15.89:0/194950286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.489+0000 7f4255254700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4250135c10 0x7f425012b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424ffff700 1 -- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f425012c9a0 msgr2=0x7f425012abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424ffff700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f425012c9a0 0x7f425012abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424ffff700 1 -- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4250135c10 msgr2=0x7f425012b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424ffff700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4250135c10 0x7f425012b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424ffff700 1 -- 172.21.15.89:0/194950286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4248009a70 con 0x7f425012cd70 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f4255254700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4250135c10 0x7f425012b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f4254a53700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f425012c9a0 0x7f425012abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424ffff700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f425012cd70 0x7f425012b100 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f423c008010 tx=0x7f423c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424d7fa700 1 -- 172.21.15.89:0/194950286 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0036a0 con 0x7f425012cd70 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424d7fa700 1 -- 172.21.15.89:0/194950286 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c003800 con 0x7f425012cd70 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c00a6f0 con 0x7f425012cd70 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f424d7fa700 1 -- 172.21.15.89:0/194950286 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c003bb0 con 0x7f425012cd70 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.490+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425009c5f0 con 0x7f425012cd70 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.491+0000 7f424d7fa700 1 -- 172.21.15.89:0/194950286 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f423c003d10 con 0x7f425012cd70 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.491+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4250074ea0 con 0x7f425012cd70 2023-09-30T22:20:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.491+0000 7f424d7fa700 1 -- 172.21.15.89:0/194950286 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f423c013070 con 0x7f425012cd70 2023-09-30T22:20:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.496+0000 7f424d7fa700 1 -- 172.21.15.89:0/194950286 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f423c020e10 con 0x7f425012cd70 2023-09-30T22:20:04.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:04.550 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.595+0000 7fd8b57d6700 1 Processor -- start 2023-09-30T22:20:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.595+0000 7fd8b57d6700 1 -- start start 2023-09-30T22:20:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8b57d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b012cf60 0x7fd8b006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8b57d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8b57d6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b012cd90 con 0x7fd8b0130260 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8b57d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b012d330 con 0x7fd8b012f8b0 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8b57d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01306d0 con 0x7fd8b012cf60 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8af7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd8b0130260 0x7fd8b0136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35618/0 (socket says 172.21.15.89:35618) 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8af7fe700 1 -- 172.21.15.89:0/3009763802 learned_addr learned my addr 172.21.15.89:0/3009763802 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8aeffd700 1 --2- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b012cf60 0x7fd8b006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8ae7fc700 1 --2- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8aeffd700 1 -- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 msgr2=0x7fd8b012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8aeffd700 1 --2- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8aeffd700 1 -- 172.21.15.89:0/3009763802 >> v1:172.21.15.89:6789/0 conn(0x7fd8b0130260 legacy=0x7fd8b0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8aeffd700 1 -- 172.21.15.89:0/3009763802 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0071780 con 0x7fd8b012cf60 2023-09-30T22:20:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.596+0000 7fd8aeffd700 1 --2- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b012cf60 0x7fd8b006f5d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009870 tx=0x7fd8a0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d82ed1927625d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8adffb700 1 -- 172.21.15.89:0/3009763802 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0011830 con 0x7fd8b012cf60 2023-09-30T22:20:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8adffb700 1 -- 172.21.15.89:0/3009763802 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0011990 con 0x7fd8b012cf60 2023-09-30T22:20:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8adffb700 1 -- 172.21.15.89:0/3009763802 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0011d20 con 0x7fd8b012cf60 2023-09-30T22:20:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8b57d6700 1 -- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b012cf60 msgr2=0x7fd8b006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8b57d6700 1 --2- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b012cf60 0x7fd8b006f5d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009870 tx=0x7fd8a0009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8b57d6700 1 -- 172.21.15.89:0/3009763802 shutdown_connections 2023-09-30T22:20:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8b57d6700 1 --2- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b012cf60 0x7fd8b006f5d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8b57d6700 1 --2- 172.21.15.89:0/3009763802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8b57d6700 1 -- 172.21.15.89:0/3009763802 >> 172.21.15.89:0/3009763802 conn(0x7fd8b00a2aa0 msgr2=0x7fd8b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:04.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8b57d6700 1 -- 172.21.15.89:0/3009763802 shutdown_connections 2023-09-30T22:20:04.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.597+0000 7fd8b57d6700 1 -- 172.21.15.89:0/3009763802 wait complete. 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.599+0000 7fd8b57d6700 1 Processor -- start 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.599+0000 7fd8b57d6700 1 -- start start 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8b57d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b012cf60 0x7fd8b01309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8b57d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b0130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8b57d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b0130260 0x7fd8b0131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8b57d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0133f20 con 0x7fd8b012cf60 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8b57d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0139570 con 0x7fd8b012f8b0 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8b57d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b009ea70 con 0x7fd8b0130260 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b0130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b0130f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47402/0 (socket says 172.21.15.89:47402) 2023-09-30T22:20:04.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 -- 172.21.15.89:0/1686000089 learned_addr learned my addr 172.21.15.89:0/1686000089 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8aeffd700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b012cf60 0x7fd8b01309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8af7fe700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b0130260 0x7fd8b0131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 -- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b0130260 msgr2=0x7fd8b0131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b0130260 0x7fd8b0131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 -- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b012cf60 msgr2=0x7fd8b01309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b012cf60 0x7fd8b01309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 -- 172.21.15.89:0/1686000089 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a00094a0 con 0x7fd8b012f8b0 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8af7fe700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b0130260 0x7fd8b0131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.600+0000 7fd8ae7fc700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b0130f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400e980 tx=0x7fd8a400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.601+0000 7fd8977fe700 1 -- 172.21.15.89:0/1686000089 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4009710 con 0x7fd8b012f8b0 2023-09-30T22:20:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.601+0000 7fd8977fe700 1 -- 172.21.15.89:0/1686000089 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a400eed0 con 0x7fd8b012f8b0 2023-09-30T22:20:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.601+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b0133260 con 0x7fd8b012f8b0 2023-09-30T22:20:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.601+0000 7fd8977fe700 1 -- 172.21.15.89:0/1686000089 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400ca70 con 0x7fd8b012f8b0 2023-09-30T22:20:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.601+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b0131e20 con 0x7fd8b012f8b0 2023-09-30T22:20:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.602+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0074ea0 con 0x7fd8b012f8b0 2023-09-30T22:20:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.602+0000 7fd8977fe700 1 -- 172.21.15.89:0/1686000089 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd8a400cbd0 con 0x7fd8b012f8b0 2023-09-30T22:20:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.603+0000 7fd8977fe700 1 -- 172.21.15.89:0/1686000089 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd8a403ffe0 con 0x7fd8b012f8b0 2023-09-30T22:20:04.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.610+0000 7fd8977fe700 1 -- 172.21.15.89:0/1686000089 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8a4011350 con 0x7fd8b012f8b0 2023-09-30T22:20:04.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.657+0000 7f9d78fa8700 1 Processor -- start 2023-09-30T22:20:04.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.657+0000 7f9d78fa8700 1 -- start start 2023-09-30T22:20:04.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.658+0000 7f9d78fa8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d74133420 0x7f9d741337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.658+0000 7f9d78fa8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74133e30 0x7f9d7412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.658+0000 7f9d78fa8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7406b990 con 0x7f9d74134260 2023-09-30T22:20:04.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.658+0000 7f9d78fa8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d740a0590 con 0x7f9d74133420 2023-09-30T22:20:04.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.658+0000 7f9d78fa8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7409fb80 con 0x7f9d74133e30 2023-09-30T22:20:04.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.658+0000 7f9d7259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d74133420 0x7f9d741337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d74133420 0x7f9d741337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59720/0 (socket says 172.21.15.89:59720) 2023-09-30T22:20:04.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d71d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74133e30 0x7f9d7412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7259c700 1 -- 172.21.15.89:0/4274353752 learned_addr learned my addr 172.21.15.89:0/4274353752 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:04.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7259c700 1 -- 172.21.15.89:0/4274353752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74133e30 msgr2=0x7f9d7412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:04.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7259c700 1 --2- 172.21.15.89:0/4274353752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74133e30 0x7f9d7412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7259c700 1 -- 172.21.15.89:0/4274353752 >> v1:172.21.15.89:6789/0 conn(0x7f9d74134260 legacy=0x7f9d741304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7259c700 1 -- 172.21.15.89:0/4274353752 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d740a0030 con 0x7f9d74133420 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d71d9b700 1 --2- 172.21.15.89:0/4274353752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74133e30 0x7f9d7412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7259c700 1 --2- 172.21.15.89:0/4274353752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 0x7f9d741337f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9d640098f0 tx=0x7f9d64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c168c52e877b0c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7159a700 1 -- 172.21.15.89:0/4274353752 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64011830 con 0x7f9d74133420 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7159a700 1 -- 172.21.15.89:0/4274353752 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d64011990 con 0x7f9d74133420 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.659+0000 7f9d7159a700 1 -- 172.21.15.89:0/4274353752 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64011d20 con 0x7f9d74133420 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.660+0000 7f9d78fa8700 1 -- 172.21.15.89:0/4274353752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 msgr2=0x7f9d741337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.660+0000 7f9d78fa8700 1 --2- 172.21.15.89:0/4274353752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 0x7f9d741337f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9d640098f0 tx=0x7f9d64009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.660+0000 7f9d78fa8700 1 -- 172.21.15.89:0/4274353752 shutdown_connections 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.660+0000 7f9d78fa8700 1 --2- 172.21.15.89:0/4274353752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 0x7f9d741337f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.660+0000 7f9d78fa8700 1 --2- 172.21.15.89:0/4274353752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74133e30 0x7f9d7412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.660+0000 7f9d78fa8700 1 -- 172.21.15.89:0/4274353752 >> 172.21.15.89:0/4274353752 conn(0x7f9d740a4210 msgr2=0x7f9d740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:04.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.660+0000 7f9d78fa8700 1 -- 172.21.15.89:0/4274353752 shutdown_connections 2023-09-30T22:20:04.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.661+0000 7f9d78fa8700 1 -- 172.21.15.89:0/4274353752 wait complete. 2023-09-30T22:20:04.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.662+0000 7f9d78fa8700 1 Processor -- start 2023-09-30T22:20:04.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.662+0000 7f9d78fa8700 1 -- start start 2023-09-30T22:20:04.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d78fa8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 0x7f9d7409c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d78fa8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d74133e30 0x7f9d7409c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d78fa8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74134260 0x7f9d7409cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d78fa8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74120010 con 0x7f9d74133420 2023-09-30T22:20:04.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d78fa8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d741201f0 con 0x7f9d74133e30 2023-09-30T22:20:04.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d78fa8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7412da70 con 0x7f9d74134260 2023-09-30T22:20:04.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d7259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 0x7f9d7409c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d72d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74134260 0x7f9d7409cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d7259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 0x7f9d7409c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59730/0 (socket says 172.21.15.89:59730) 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d7259c700 1 -- 172.21.15.89:0/1478957351 learned_addr learned my addr 172.21.15.89:0/1478957351 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d71d9b700 1 --2- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d74133e30 0x7f9d7409c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d72d9d700 1 -- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d74133e30 msgr2=0x7f9d7409c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d72d9d700 1 --2- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d74133e30 0x7f9d7409c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d72d9d700 1 -- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 msgr2=0x7f9d7409c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d72d9d700 1 --2- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 0x7f9d7409c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d72d9d700 1 -- 172.21.15.89:0/1478957351 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d680094a0 con 0x7f9d74134260 2023-09-30T22:20:04.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d72d9d700 1 --2- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74134260 0x7f9d7409cc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00cb90 tx=0x7f9d6c00cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:04.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.663+0000 7f9d62ffd700 1 -- 172.21.15.89:0/1478957351 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c003c00 con 0x7f9d74134260 2023-09-30T22:20:04.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.664+0000 7f9d62ffd700 1 -- 172.21.15.89:0/1478957351 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c003d60 con 0x7f9d74134260 2023-09-30T22:20:04.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.664+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d64009550 con 0x7f9d74134260 2023-09-30T22:20:04.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.664+0000 7f9d62ffd700 1 -- 172.21.15.89:0/1478957351 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00aa40 con 0x7f9d74134260 2023-09-30T22:20:04.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.664+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7412d620 con 0x7f9d74134260 2023-09-30T22:20:04.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.664+0000 7f9d62ffd700 1 -- 172.21.15.89:0/1478957351 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d6c00aba0 con 0x7f9d74134260 2023-09-30T22:20:04.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.665+0000 7f9d62ffd700 1 -- 172.21.15.89:0/1478957351 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d6c03f630 con 0x7f9d74134260 2023-09-30T22:20:04.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.665+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d74074ea0 con 0x7f9d74134260 2023-09-30T22:20:04.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.674+0000 7f9d62ffd700 1 -- 172.21.15.89:0/1478957351 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d6c012420 con 0x7f9d74134260 2023-09-30T22:20:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.700+0000 7fcc1683c700 1 Processor -- start 2023-09-30T22:20:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.700+0000 7fcc1683c700 1 -- start start 2023-09-30T22:20:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.700+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f425009adc0 con 0x7f425012cd70 2023-09-30T22:20:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.700+0000 7fcc1683c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009e930 0x7fcc1009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.700+0000 7fcc1683c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc1009f650 0x7fcc10131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.700+0000 7fcc1683c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc10070940 con 0x7fcc1009f240 2023-09-30T22:20:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.700+0000 7fcc1683c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1006fd60 con 0x7fcc1009f650 2023-09-30T22:20:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.700+0000 7fcc1683c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1006fa80 con 0x7fcc1009e930 2023-09-30T22:20:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009e930 0x7fcc1009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc14dd9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc1009f650 0x7fcc10131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7f424d7fa700 1 -- 172.21.15.89:0/194950286 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f425009adc0 con 0x7f425012cd70 2023-09-30T22:20:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009e930 0x7fcc1009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36562/0 (socket says 172.21.15.89:36562) 2023-09-30T22:20:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc14dd9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc1009f650 0x7fcc10131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59732/0 (socket says 172.21.15.89:59732) 2023-09-30T22:20:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0ffff700 1 -- 172.21.15.89:0/562050022 learned_addr learned my addr 172.21.15.89:0/562050022 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:04.703 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0ffff700 1 -- 172.21.15.89:0/562050022 >> v2:172.21.15.89:3300/0 conn(0x7fcc1009f650 msgr2=0x7fcc10131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0ffff700 1 --2- 172.21.15.89:0/562050022 >> v2:172.21.15.89:3300/0 conn(0x7fcc1009f650 0x7fcc10131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0ffff700 1 -- 172.21.15.89:0/562050022 >> v1:172.21.15.89:6789/0 conn(0x7fcc1009f240 legacy=0x7fcc101317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0ffff700 1 -- 172.21.15.89:0/562050022 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1009fb60 con 0x7fcc1009e930 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0ffff700 1 --2- 172.21.15.89:0/562050022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009e930 0x7fcc1009ed00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098f0 tx=0x7fcc00009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88c5cd76c1535f97 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0effd700 1 -- 172.21.15.89:0/562050022 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00011830 con 0x7fcc1009e930 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0effd700 1 -- 172.21.15.89:0/562050022 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00011990 con 0x7fcc1009e930 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.701+0000 7fcc0effd700 1 -- 172.21.15.89:0/562050022 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00011d20 con 0x7fcc1009e930 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.702+0000 7fcc1683c700 1 -- 172.21.15.89:0/562050022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009e930 msgr2=0x7fcc1009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.702+0000 7fcc1683c700 1 --2- 172.21.15.89:0/562050022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009e930 0x7fcc1009ed00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098f0 tx=0x7fcc00009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.702+0000 7fcc1683c700 1 -- 172.21.15.89:0/562050022 shutdown_connections 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.702+0000 7fcc1683c700 1 --2- 172.21.15.89:0/562050022 >> v2:172.21.15.89:3300/0 conn(0x7fcc1009f650 0x7fcc10131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.702+0000 7fcc1683c700 1 --2- 172.21.15.89:0/562050022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009e930 0x7fcc1009ed00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.702+0000 7fcc1683c700 1 -- 172.21.15.89:0/562050022 >> 172.21.15.89:0/562050022 conn(0x7fcc100a4210 msgr2=0x7fcc100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.702+0000 7fcc1683c700 1 -- 172.21.15.89:0/562050022 shutdown_connections 2023-09-30T22:20:04.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.702+0000 7fcc1683c700 1 -- 172.21.15.89:0/562050022 wait complete. 2023-09-30T22:20:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7fcc1683c700 1 Processor -- start 2023-09-30T22:20:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7fcc1683c700 1 -- start start 2023-09-30T22:20:04.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f425012cd70 msgr2=0x7f425012b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f425012cd70 0x7f425012b100 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f423c008010 tx=0x7f423c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 shutdown_connections 2023-09-30T22:20:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4250135c10 0x7f425012b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f425012c9a0 0x7f425012abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 --2- 172.21.15.89:0/194950286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f425012cd70 0x7f425012b100 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 >> 172.21.15.89:0/194950286 conn(0x7f42500a2360 msgr2=0x7f42500a0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 shutdown_connections 2023-09-30T22:20:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.704+0000 7f4256cb7700 1 -- 172.21.15.89:0/194950286 wait complete. 2023-09-30T22:20:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc1683c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1009e930 0x7fcc1012d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc1683c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009f240 0x7fcc1012d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc1683c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc1009f650 0x7fcc1012db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc1683c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100a33c0 con 0x7fcc1009e930 2023-09-30T22:20:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc1683c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10071d20 con 0x7fcc1009f650 2023-09-30T22:20:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc1683c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc101307e0 con 0x7fcc1009f240 2023-09-30T22:20:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1009e930 0x7fcc1012d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1009e930 0x7fcc1012d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59738/0 (socket says 172.21.15.89:59738) 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 -- 172.21.15.89:0/922137173 learned_addr learned my addr 172.21.15.89:0/922137173 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0f7fe700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009f240 0x7fcc1012d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc14dd9700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc1009f650 0x7fcc1012db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 -- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009f240 msgr2=0x7fcc1012d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009f240 0x7fcc1012d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 -- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc1009f650 msgr2=0x7fcc1012db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc1009f650 0x7fcc1012db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 -- 172.21.15.89:0/922137173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc00009550 con 0x7fcc1009e930 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc14dd9700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc1009f650 0x7fcc1012db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.705+0000 7fcc0ffff700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1009e930 0x7fcc1012d0a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcc00000c00 tx=0x7fcc00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.706+0000 7fcc0cff9700 1 -- 172.21.15.89:0/922137173 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00004480 con 0x7fcc1009e930 2023-09-30T22:20:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.706+0000 7fcc0cff9700 1 -- 172.21.15.89:0/922137173 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0001f070 con 0x7fcc1009e930 2023-09-30T22:20:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.706+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc100a0190 con 0x7fcc1009e930 2023-09-30T22:20:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.706+0000 7fcc0cff9700 1 -- 172.21.15.89:0/922137173 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000038c0 con 0x7fcc1009e930 2023-09-30T22:20:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.706+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1012f050 con 0x7fcc1009e930 2023-09-30T22:20:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.706+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc10074ea0 con 0x7fcc1009e930 2023-09-30T22:20:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.707+0000 7fcc0cff9700 1 -- 172.21.15.89:0/922137173 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc00021030 con 0x7fcc1009e930 2023-09-30T22:20:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.707+0000 7fcc0cff9700 1 -- 172.21.15.89:0/922137173 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcc00023a20 con 0x7fcc1009e930 2023-09-30T22:20:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.716+0000 7fcc0cff9700 1 -- 172.21.15.89:0/922137173 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc0001a7d0 con 0x7fcc1009e930 2023-09-30T22:20:04.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.819+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd8b00713a0 con 0x7fd8b012f8b0 2023-09-30T22:20:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.820+0000 7fd8977fe700 1 -- 172.21.15.89:0/1686000089 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd8a4017030 con 0x7fd8b012f8b0 2023-09-30T22:20:04.821 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 msgr2=0x7fd8b0130f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b0130f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400e980 tx=0x7fd8a400ec90 comp rx=0 tx=0).stop 2023-09-30T22:20:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 shutdown_connections 2023-09-30T22:20:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8b0130260 0x7fd8b0131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b012cf60 0x7fd8b01309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 --2- 172.21.15.89:0/1686000089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8b012f8b0 0x7fd8b0130f20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 >> 172.21.15.89:0/1686000089 conn(0x7fd8b00a2aa0 msgr2=0x7fd8b00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 shutdown_connections 2023-09-30T22:20:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.822+0000 7fd8b57d6700 1 -- 172.21.15.89:0/1686000089 wait complete. 2023-09-30T22:20:04.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.899+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9d7411fe20 con 0x7f9d74134260 2023-09-30T22:20:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.899+0000 7f9d62ffd700 1 -- 172.21.15.89:0/1478957351 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d6c011380 con 0x7f9d74134260 2023-09-30T22:20:04.900 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74134260 msgr2=0x7f9d7409cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 --2- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74134260 0x7f9d7409cc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00cb90 tx=0x7f9d6c00cea0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 shutdown_connections 2023-09-30T22:20:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 --2- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d74134260 0x7f9d7409cc80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 --2- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d74133420 0x7f9d7409c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 --2- 172.21.15.89:0/1478957351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d74133e30 0x7f9d7409c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 >> 172.21.15.89:0/1478957351 conn(0x7f9d740a4210 msgr2=0x7f9d7412f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 shutdown_connections 2023-09-30T22:20:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.901+0000 7f9d78fa8700 1 -- 172.21.15.89:0/1478957351 wait complete. 2023-09-30T22:20:04.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:04.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.932+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcc1006b250 con 0x7fcc1009e930 2023-09-30T22:20:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.932+0000 7fcc0cff9700 1 -- 172.21.15.89:0/922137173 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc0000bbd0 con 0x7fcc1009e930 2023-09-30T22:20:04.933 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1009e930 msgr2=0x7fcc1012d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1009e930 0x7fcc1012d0a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcc00000c00 tx=0x7fcc00004750 comp rx=0 tx=0).stop 2023-09-30T22:20:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 shutdown_connections 2023-09-30T22:20:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc1009f650 0x7fcc1012db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1009e930 0x7fcc1012d0a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 --2- 172.21.15.89:0/922137173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc1009f240 0x7fcc1012d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 >> 172.21.15.89:0/922137173 conn(0x7fcc100a4210 msgr2=0x7fcc1006c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 shutdown_connections 2023-09-30T22:20:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:04.934+0000 7fcc1683c700 1 -- 172.21.15.89:0/922137173 wait complete. 2023-09-30T22:20:04.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:05.724 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:05.835 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:05.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f3025b1a700 1 Processor -- start 2023-09-30T22:20:05.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f3025b1a700 1 -- start start 2023-09-30T22:20:05.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f3025b1a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009d6f0 0x7f302009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f3025b1a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e000 0x7f3020130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f3025b1a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f302009e410 0x7f3020130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f3025b1a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302009f520 con 0x7f302009e410 2023-09-30T22:20:05.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f3025b1a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302009f130 con 0x7f302009d6f0 2023-09-30T22:20:05.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f3025b1a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f302009e930 con 0x7f302009e000 2023-09-30T22:20:05.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f301f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009d6f0 0x7f302009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f301ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f302009e410 0x7f3020130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f301f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009d6f0 0x7f302009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47418/0 (socket says 172.21.15.89:47418) 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f301ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f302009e410 0x7f3020130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59740/0 (socket says 172.21.15.89:59740) 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f301f7fe700 1 -- 172.21.15.89:0/1805459449 learned_addr learned my addr 172.21.15.89:0/1805459449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.861+0000 7f301effd700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e000 0x7f3020130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301ffff700 1 -- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e000 msgr2=0x7f3020130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301ffff700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e000 0x7f3020130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301ffff700 1 -- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009d6f0 msgr2=0x7f302009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301ffff700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009d6f0 0x7f302009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301ffff700 1 -- 172.21.15.89:0/1805459449 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302009b2d0 con 0x7f302009e410 2023-09-30T22:20:05.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301effd700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e000 0x7f3020130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301f7fe700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009d6f0 0x7f302009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301ffff700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009e410 0x7f3020130ba0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3018009d90 tx=0x7f301800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcb58ed9749dcf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301e7fc700 1 -- 172.21.15.89:0/1805459449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301800e070 con 0x7f302009e410 2023-09-30T22:20:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301e7fc700 1 -- 172.21.15.89:0/1805459449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301800bd20 con 0x7f302009e410 2023-09-30T22:20:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.862+0000 7f301e7fc700 1 -- 172.21.15.89:0/1805459449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018012920 con 0x7f302009e410 2023-09-30T22:20:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 -- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009e410 msgr2=0x7f3020130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009e410 0x7f3020130ba0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3018009d90 tx=0x7f301800b330 comp rx=0 tx=0).stop 2023-09-30T22:20:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 -- 172.21.15.89:0/1805459449 shutdown_connections 2023-09-30T22:20:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009e410 0x7f3020130ba0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009d6f0 0x7f302009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 --2- 172.21.15.89:0/1805459449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e000 0x7f3020130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 -- 172.21.15.89:0/1805459449 >> 172.21.15.89:0/1805459449 conn(0x7f3020066f50 msgr2=0x7f302009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 -- 172.21.15.89:0/1805459449 shutdown_connections 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.863+0000 7f3025b1a700 1 -- 172.21.15.89:0/1805459449 wait complete. 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.864+0000 7f3025b1a700 1 Processor -- start 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.864+0000 7f3025b1a700 1 -- start start 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3025b1a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009d6f0 0x7f302011dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3025b1a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009e000 0x7f302011e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3025b1a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e410 0x7f302011ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3025b1a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3020120220 con 0x7f302009d6f0 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3025b1a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3020121b10 con 0x7f302009e000 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3025b1a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f302009c700 con 0x7f302009e410 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009e000 0x7f302011e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e410 0x7f302011ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009e000 0x7f302011e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47430/0 (socket says 172.21.15.89:47430) 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301effd700 1 -- 172.21.15.89:0/2347709982 learned_addr learned my addr 172.21.15.89:0/2347709982 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e410 0x7f302011ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36580/0 (socket says 172.21.15.89:36580) 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301f7fe700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009d6f0 0x7f302011dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301ffff700 1 -- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009e000 msgr2=0x7f302011e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301ffff700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009e000 0x7f302011e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301ffff700 1 -- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009d6f0 msgr2=0x7f302011dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301ffff700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009d6f0 0x7f302011dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301ffff700 1 -- 172.21.15.89:0/2347709982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010009550 con 0x7f302009e410 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301effd700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009e000 0x7f302011e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301f7fe700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009d6f0 0x7f302011dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:05.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f301ffff700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e410 0x7f302011ea70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f301801a000 tx=0x7f3018004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:05.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3007fff700 1 -- 172.21.15.89:0/2347709982 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018021070 con 0x7f302009e410 2023-09-30T22:20:05.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3007fff700 1 -- 172.21.15.89:0/2347709982 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301800be90 con 0x7f302009e410 2023-09-30T22:20:05.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30140094a0 con 0x7f302009e410 2023-09-30T22:20:05.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3007fff700 1 -- 172.21.15.89:0/2347709982 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301800e070 con 0x7f302009e410 2023-09-30T22:20:05.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.865+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3018009a70 con 0x7f302009e410 2023-09-30T22:20:05.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.866+0000 7f3007fff700 1 -- 172.21.15.89:0/2347709982 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3018004250 con 0x7f302009e410 2023-09-30T22:20:05.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.866+0000 7f3007fff700 1 -- 172.21.15.89:0/2347709982 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3018044690 con 0x7f302009e410 2023-09-30T22:20:05.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.867+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3020074ea0 con 0x7f302009e410 2023-09-30T22:20:05.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.871+0000 7f3007fff700 1 -- 172.21.15.89:0/2347709982 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f301801b400 con 0x7f302009e410 2023-09-30T22:20:05.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:05.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.964+0000 7f5bd2e15700 1 Processor -- start 2023-09-30T22:20:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.964+0000 7f5bd2e15700 1 -- start start 2023-09-30T22:20:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bd2e15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc133540 0x7f5bcc135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bd2e15700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc135e60 0x7f5bcc138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bd2e15700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bcc0716a0 con 0x7f5bcc130540 2023-09-30T22:20:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bd2e15700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc071290 con 0x7f5bcc135e60 2023-09-30T22:20:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bd2e15700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc130340 con 0x7f5bcc133540 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bcbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc133540 0x7f5bcc135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bcbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc133540 0x7f5bcc135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36594/0 (socket says 172.21.15.89:36594) 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bcbfff700 1 -- 172.21.15.89:0/4139754680 learned_addr learned my addr 172.21.15.89:0/4139754680 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bd13b2700 1 --2- 172.21.15.89:0/4139754680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc135e60 0x7f5bcc138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bcbfff700 1 -- 172.21.15.89:0/4139754680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc135e60 msgr2=0x7f5bcc138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bcbfff700 1 --2- 172.21.15.89:0/4139754680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc135e60 0x7f5bcc138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bcbfff700 1 -- 172.21.15.89:0/4139754680 >> v1:172.21.15.89:6789/0 conn(0x7f5bcc130540 legacy=0x7f5bcc130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bcbfff700 1 -- 172.21.15.89:0/4139754680 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc070b80 con 0x7f5bcc133540 2023-09-30T22:20:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.965+0000 7f5bcbfff700 1 --2- 172.21.15.89:0/4139754680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc133540 0x7f5bcc135920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009920 tx=0x7f5bb8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5feefe61079e63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bcb7fe700 1 -- 172.21.15.89:0/4139754680 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8004230 con 0x7f5bcc133540 2023-09-30T22:20:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bcb7fe700 1 -- 172.21.15.89:0/4139754680 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb8004390 con 0x7f5bcc133540 2023-09-30T22:20:05.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bcb7fe700 1 -- 172.21.15.89:0/4139754680 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80036a0 con 0x7f5bcc133540 2023-09-30T22:20:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bd2e15700 1 -- 172.21.15.89:0/4139754680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc133540 msgr2=0x7f5bcc135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bd2e15700 1 --2- 172.21.15.89:0/4139754680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc133540 0x7f5bcc135920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009920 tx=0x7f5bb8009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bd2e15700 1 -- 172.21.15.89:0/4139754680 shutdown_connections 2023-09-30T22:20:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bd2e15700 1 --2- 172.21.15.89:0/4139754680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc135e60 0x7f5bcc138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bd2e15700 1 --2- 172.21.15.89:0/4139754680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc133540 0x7f5bcc135920 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.966+0000 7f5bd2e15700 1 -- 172.21.15.89:0/4139754680 >> 172.21.15.89:0/4139754680 conn(0x7f5bcc0a2aa0 msgr2=0x7f5bcc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.967+0000 7f5bd2e15700 1 -- 172.21.15.89:0/4139754680 shutdown_connections 2023-09-30T22:20:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.967+0000 7f5bd2e15700 1 -- 172.21.15.89:0/4139754680 wait complete. 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.967+0000 7f5bd2e15700 1 Processor -- start 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd2e15700 1 -- start start 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd2e15700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bcc130540 0x7f5bcc09c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd2e15700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc133540 0x7f5bcc09cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd2e15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc135e60 0x7f5bcc09d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd2e15700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc1191e0 con 0x7f5bcc130540 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd2e15700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc1196f0 con 0x7f5bcc133540 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd2e15700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc135930 con 0x7f5bcc135e60 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd13b2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc135e60 0x7f5bcc09d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd13b2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc135e60 0x7f5bcc09d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36598/0 (socket says 172.21.15.89:36598) 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bd13b2700 1 -- 172.21.15.89:0/1499647881 learned_addr learned my addr 172.21.15.89:0/1499647881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.968+0000 7f5bcbfff700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc133540 0x7f5bcc09cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd0bb1700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bcc130540 0x7f5bcc09c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd13b2700 1 -- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc133540 msgr2=0x7f5bcc09cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd13b2700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc133540 0x7f5bcc09cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd13b2700 1 -- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bcc130540 msgr2=0x7f5bcc09c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd13b2700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bcc130540 0x7f5bcc09c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd13b2700 1 -- 172.21.15.89:0/1499647881 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8009550 con 0x7f5bcc135e60 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bcbfff700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc133540 0x7f5bcc09cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd13b2700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc135e60 0x7f5bcc09d300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400efc0 tx=0x7f5bc400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bc97fa700 1 -- 172.21.15.89:0/1499647881 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4009ea0 con 0x7f5bcc135e60 2023-09-30T22:20:05.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bc97fa700 1 -- 172.21.15.89:0/1499647881 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc400f040 con 0x7f5bcc135e60 2023-09-30T22:20:05.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bc97fa700 1 -- 172.21.15.89:0/1499647881 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4014480 con 0x7f5bcc135e60 2023-09-30T22:20:05.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc11a590 con 0x7f5bcc135e60 2023-09-30T22:20:05.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.969+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc09e350 con 0x7f5bcc135e60 2023-09-30T22:20:05.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.970+0000 7f5bc97fa700 1 -- 172.21.15.89:0/1499647881 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5bc40145e0 con 0x7f5bcc135e60 2023-09-30T22:20:05.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.970+0000 7f5bc97fa700 1 -- 172.21.15.89:0/1499647881 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5bc403ff30 con 0x7f5bcc135e60 2023-09-30T22:20:05.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.970+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc0718e0 con 0x7f5bcc135e60 2023-09-30T22:20:05.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:05.979+0000 7f5bc97fa700 1 -- 172.21.15.89:0/1499647881 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5bc4018020 con 0x7f5bcc135e60 2023-09-30T22:20:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.045+0000 7fa4ae3c6700 1 Processor -- start 2023-09-30T22:20:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.045+0000 7fa4ae3c6700 1 -- start start 2023-09-30T22:20:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4ae3c6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a806a770 0x7fa4a809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4ae3c6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4ae3c6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a809a810 con 0x7fa4a813c1e0 2023-09-30T22:20:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4ae3c6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a806ab40 con 0x7fa4a806a770 2023-09-30T22:20:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4ae3c6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a809ebc0 con 0x7fa4a809dc30 2023-09-30T22:20:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a806a770 0x7fa4a809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a806a770 0x7fa4a809d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47454/0 (socket says 172.21.15.89:47454) 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4a7fff700 1 -- 172.21.15.89:0/611117364 learned_addr learned my addr 172.21.15.89:0/611117364 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.046+0000 7fa4a77fe700 1 --2- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a7fff700 1 -- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 msgr2=0x7fa4a809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a7fff700 1 --2- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a809e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a7fff700 1 -- 172.21.15.89:0/611117364 >> v1:172.21.15.89:6789/0 conn(0x7fa4a813c1e0 legacy=0x7fa4a813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a7fff700 1 -- 172.21.15.89:0/611117364 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a809e920 con 0x7fa4a806a770 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a77fe700 1 --2- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a7fff700 1 --2- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a806a770 0x7fa4a809d6f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa498009920 tx=0x7fa498009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a31c82d2bd4992e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a6ffd700 1 -- 172.21.15.89:0/611117364 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011830 con 0x7fa4a806a770 2023-09-30T22:20:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a6ffd700 1 -- 172.21.15.89:0/611117364 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498011990 con 0x7fa4a806a770 2023-09-30T22:20:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.047+0000 7fa4a6ffd700 1 -- 172.21.15.89:0/611117364 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011d20 con 0x7fa4a806a770 2023-09-30T22:20:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.048+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a806a770 msgr2=0x7fa4a809d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.048+0000 7fa4ae3c6700 1 --2- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a806a770 0x7fa4a809d6f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa498009920 tx=0x7fa498009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.048+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/611117364 shutdown_connections 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.048+0000 7fa4ae3c6700 1 --2- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a806a770 0x7fa4a809d6f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.048+0000 7fa4ae3c6700 1 --2- 172.21.15.89:0/611117364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.048+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/611117364 >> 172.21.15.89:0/611117364 conn(0x7fa4a8066f50 msgr2=0x7fa4a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.048+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/611117364 shutdown_connections 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.048+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/611117364 wait complete. 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.049+0000 7fa4ae3c6700 1 Processor -- start 2023-09-30T22:20:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.049+0000 7fa4ae3c6700 1 -- start start 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4ae3c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a806a770 0x7fa4a813a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4ae3c6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a813ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4ae3c6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a813c1e0 0x7fa4a813b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4ae3c6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a8071e40 con 0x7fa4a806a770 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4ae3c6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a8134ff0 con 0x7fa4a813c1e0 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4ae3c6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a813f1d0 con 0x7fa4a809dc30 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a806a770 0x7fa4a813a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a806a770 0x7fa4a813a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59764/0 (socket says 172.21.15.89:59764) 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4a7fff700 1 -- 172.21.15.89:0/913141533 learned_addr learned my addr 172.21.15.89:0/913141533 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4ac963700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a813c1e0 0x7fa4a813b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.050+0000 7fa4a77fe700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a813ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a77fe700 1 -- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a813c1e0 msgr2=0x7fa4a813b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a77fe700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a813c1e0 0x7fa4a813b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a77fe700 1 -- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a806a770 msgr2=0x7fa4a813a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a77fe700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a806a770 0x7fa4a813a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a77fe700 1 -- 172.21.15.89:0/913141533 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c0094a0 con 0x7fa4a809dc30 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a77fe700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a813ad40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00e980 tx=0x7fa49c00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a4ff9700 1 -- 172.21.15.89:0/913141533 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00c690 con 0x7fa4a809dc30 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a4ff9700 1 -- 172.21.15.89:0/913141533 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c00c7f0 con 0x7fa4a809dc30 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa498009550 con 0x7fa4a809dc30 2023-09-30T22:20:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4a4ff9700 1 -- 172.21.15.89:0/913141533 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00cb80 con 0x7fa4a809dc30 2023-09-30T22:20:06.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.051+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a8140360 con 0x7fa4a809dc30 2023-09-30T22:20:06.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.052+0000 7fa4a4ff9700 1 -- 172.21.15.89:0/913141533 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa49c00cce0 con 0x7fa4a809dc30 2023-09-30T22:20:06.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.052+0000 7fa4a4ff9700 1 -- 172.21.15.89:0/913141533 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa49c03f700 con 0x7fa4a809dc30 2023-09-30T22:20:06.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.052+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a813f7a0 con 0x7fa4a809dc30 2023-09-30T22:20:06.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.057+0000 7fa4a4ff9700 1 -- 172.21.15.89:0/913141533 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa49c017020 con 0x7fa4a809dc30 2023-09-30T22:20:06.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.078+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f30201303d0 con 0x7f302009e410 2023-09-30T22:20:06.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.079+0000 7f3007fff700 1 -- 172.21.15.89:0/2347709982 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f301802a030 con 0x7f302009e410 2023-09-30T22:20:06.079 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:06.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e410 msgr2=0x7f302011ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e410 0x7f302011ea70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f301801a000 tx=0x7f3018004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 shutdown_connections 2023-09-30T22:20:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f302009e410 0x7f302011ea70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f302009d6f0 0x7f302011dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 --2- 172.21.15.89:0/2347709982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f302009e000 0x7f302011e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 >> 172.21.15.89:0/2347709982 conn(0x7f3020066f50 msgr2=0x7f302006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 shutdown_connections 2023-09-30T22:20:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.082+0000 7f3025b1a700 1 -- 172.21.15.89:0/2347709982 wait complete. 2023-09-30T22:20:06.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.086+0000 7f82089ab700 1 Processor -- start 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f82089ab700 1 -- start start 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f82089ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f820412cf60 0x7f820406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f82089ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f82089ab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8204130260 0x7f8204136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f82089ab700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820412d330 con 0x7f8204130260 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f82089ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820412cd90 con 0x7f820412cf60 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f82089ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82041306d0 con 0x7f820412f8b0 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f8201d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f820259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f820412cf60 0x7f820406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f8201d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36608/0 (socket says 172.21.15.89:36608) 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.087+0000 7f8201d9b700 1 -- 172.21.15.89:0/64808405 learned_addr learned my addr 172.21.15.89:0/64808405 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f8202d9d700 1 --2- 172.21.15.89:0/64808405 >> v2:172.21.15.89:3300/0 conn(0x7f8204130260 0x7f8204136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f8201d9b700 1 -- 172.21.15.89:0/64808405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f820412cf60 msgr2=0x7f820406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f8201d9b700 1 --2- 172.21.15.89:0/64808405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f820412cf60 0x7f820406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f8201d9b700 1 -- 172.21.15.89:0/64808405 >> v2:172.21.15.89:3300/0 conn(0x7f8204130260 msgr2=0x7f8204136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f8201d9b700 1 --2- 172.21.15.89:0/64808405 >> v2:172.21.15.89:3300/0 conn(0x7f8204130260 0x7f8204136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f8201d9b700 1 -- 172.21.15.89:0/64808405 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204071780 con 0x7f820412f8b0 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f820259c700 1 --2- 172.21.15.89:0/64808405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f820412cf60 0x7f820406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f8201d9b700 1 --2- 172.21.15.89:0/64808405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820412fd20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f81f80098f0 tx=0x7f81f8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b812355686b51db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f820159a700 1 -- 172.21.15.89:0/64808405 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8011830 con 0x7f820412f8b0 2023-09-30T22:20:06.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f820159a700 1 -- 172.21.15.89:0/64808405 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8011990 con 0x7f820412f8b0 2023-09-30T22:20:06.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.088+0000 7f820159a700 1 -- 172.21.15.89:0/64808405 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8011d20 con 0x7f820412f8b0 2023-09-30T22:20:06.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 -- 172.21.15.89:0/64808405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 msgr2=0x7f820412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 --2- 172.21.15.89:0/64808405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820412fd20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f81f80098f0 tx=0x7f81f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 -- 172.21.15.89:0/64808405 shutdown_connections 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 --2- 172.21.15.89:0/64808405 >> v2:172.21.15.89:3300/0 conn(0x7f8204130260 0x7f8204136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 --2- 172.21.15.89:0/64808405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f820412cf60 0x7f820406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 --2- 172.21.15.89:0/64808405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820412fd20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 -- 172.21.15.89:0/64808405 >> 172.21.15.89:0/64808405 conn(0x7f82040a2aa0 msgr2=0x7f82040a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 -- 172.21.15.89:0/64808405 shutdown_connections 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.089+0000 7f82089ab700 1 -- 172.21.15.89:0/64808405 wait complete. 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.090+0000 7f82089ab700 1 Processor -- start 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.090+0000 7f82089ab700 1 -- start start 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f82089ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820412cf60 0x7f820409b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f82089ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820409bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f82089ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8204130260 0x7f82041c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f82089ab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820409cd00 con 0x7f820412cf60 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f82089ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820409b610 con 0x7f8204130260 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f82089ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f820409da50 con 0x7f820412f8b0 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f820259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820412cf60 0x7f820409b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f820259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820412cf60 0x7f820409b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59780/0 (socket says 172.21.15.89:59780) 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f820259c700 1 -- 172.21.15.89:0/1776837733 learned_addr learned my addr 172.21.15.89:0/1776837733 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f8202d9d700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8204130260 0x7f82041c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f8201d9b700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820409bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f820259c700 1 -- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 msgr2=0x7f820409bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f820259c700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820409bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f820259c700 1 -- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8204130260 msgr2=0x7f82041c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f820259c700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8204130260 0x7f82041c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f820259c700 1 -- 172.21.15.89:0/1776837733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f400a5a0 con 0x7f820412cf60 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.091+0000 7f8202d9d700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8204130260 0x7f82041c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.092+0000 7f820259c700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820412cf60 0x7f820409b9e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f81f400ec30 tx=0x7f81f400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.092+0000 7f81f2ffd700 1 -- 172.21.15.89:0/1776837733 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f400cbb0 con 0x7f820412cf60 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.092+0000 7f81f2ffd700 1 -- 172.21.15.89:0/1776837733 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f400cd10 con 0x7f820412cf60 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.092+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f8009550 con 0x7f820412cf60 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.092+0000 7f81f2ffd700 1 -- 172.21.15.89:0/1776837733 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4011630 con 0x7f820412cf60 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.093+0000 7f81f2ffd700 1 -- 172.21.15.89:0/1776837733 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81f4011790 con 0x7f820412cf60 2023-09-30T22:20:06.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.093+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820412f460 con 0x7f820412cf60 2023-09-30T22:20:06.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.093+0000 7f81f2ffd700 1 -- 172.21.15.89:0/1776837733 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f81f401d450 con 0x7f820412cf60 2023-09-30T22:20:06.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.094+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8204129fa0 con 0x7f820412cf60 2023-09-30T22:20:06.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:06.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.103+0000 7f81f2ffd700 1 -- 172.21.15.89:0/1776837733 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81f400f8d0 con 0x7f820412cf60 2023-09-30T22:20:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.183+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5bcc074ea0 con 0x7f5bcc135e60 2023-09-30T22:20:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.184+0000 7f5bc97fa700 1 -- 172.21.15.89:0/1499647881 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5bc401a680 con 0x7f5bcc135e60 2023-09-30T22:20:06.184 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc135e60 msgr2=0x7f5bcc09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc135e60 0x7f5bcc09d300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400efc0 tx=0x7f5bc400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 shutdown_connections 2023-09-30T22:20:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5bcc135e60 0x7f5bcc09d300 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bcc130540 0x7f5bcc09c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 --2- 172.21.15.89:0/1499647881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5bcc133540 0x7f5bcc09cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 >> 172.21.15.89:0/1499647881 conn(0x7f5bcc0a2aa0 msgr2=0x7f5bcc131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 shutdown_connections 2023-09-30T22:20:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.187+0000 7f5bd2e15700 1 -- 172.21.15.89:0/1499647881 wait complete. 2023-09-30T22:20:06.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:06.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.255+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa4a8074ea0 con 0x7fa4a809dc30 2023-09-30T22:20:06.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.255+0000 7fa4a4ff9700 1 -- 172.21.15.89:0/913141533 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa49c0232f0 con 0x7fa4a809dc30 2023-09-30T22:20:06.255 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 msgr2=0x7fa4a813ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a813ad40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00e980 tx=0x7fa49c00ec90 comp rx=0 tx=0).stop 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 shutdown_connections 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4a813c1e0 0x7fa4a813b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a806a770 0x7fa4a813a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 --2- 172.21.15.89:0/913141533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4a809dc30 0x7fa4a813ad40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 >> 172.21.15.89:0/913141533 conn(0x7fa4a8066f50 msgr2=0x7fa4a80a1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 shutdown_connections 2023-09-30T22:20:06.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.258+0000 7fa4ae3c6700 1 -- 172.21.15.89:0/913141533 wait complete. 2023-09-30T22:20:06.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:06.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.297+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8204074ea0 con 0x7f820412cf60 2023-09-30T22:20:06.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.297+0000 7f81f2ffd700 1 -- 172.21.15.89:0/1776837733 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81f400f2e0 con 0x7f820412cf60 2023-09-30T22:20:06.297 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:06.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820412cf60 msgr2=0x7f820409b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:06.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820412cf60 0x7f820409b9e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f81f400ec30 tx=0x7f81f400ef40 comp rx=0 tx=0).stop 2023-09-30T22:20:06.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 shutdown_connections 2023-09-30T22:20:06.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8204130260 0x7f82041c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f820412cf60 0x7f820409b9e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 --2- 172.21.15.89:0/1776837733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f820412f8b0 0x7f820409bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:06.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 >> 172.21.15.89:0/1776837733 conn(0x7f82040a2aa0 msgr2=0x7f820406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:06.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 shutdown_connections 2023-09-30T22:20:06.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:06.299+0000 7f82089ab700 1 -- 172.21.15.89:0/1776837733 wait complete. 2023-09-30T22:20:06.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:07.098 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:07.200 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc7d9ac700 1 Processor -- start 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc7d9ac700 1 -- start start 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc7d9ac700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809d6f0 0x7fdc7809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc7d9ac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc7809e410 0x7fdc78134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc7d9ac700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7809b120 con 0x7fdc7809e000 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc7d9ac700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7806b460 con 0x7fdc7809e410 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc7d9ac700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7809a7e0 con 0x7fdc7809d6f0 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc777fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc7809e410 0x7fdc78134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc76ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809d6f0 0x7fdc7809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc777fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc7809e410 0x7fdc78134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59784/0 (socket says 172.21.15.89:59784) 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.232+0000 7fdc777fe700 1 -- 172.21.15.89:0/1134963591 learned_addr learned my addr 172.21.15.89:0/1134963591 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc76ffd700 1 -- 172.21.15.89:0/1134963591 >> v2:172.21.15.89:3300/0 conn(0x7fdc7809e410 msgr2=0x7fdc78134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc76ffd700 1 --2- 172.21.15.89:0/1134963591 >> v2:172.21.15.89:3300/0 conn(0x7fdc7809e410 0x7fdc78134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc76ffd700 1 -- 172.21.15.89:0/1134963591 >> v1:172.21.15.89:6789/0 conn(0x7fdc7809e000 legacy=0x7fdc781339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc76ffd700 1 -- 172.21.15.89:0/1134963591 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7809a500 con 0x7fdc7809d6f0 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc777fe700 1 --2- 172.21.15.89:0/1134963591 >> v2:172.21.15.89:3300/0 conn(0x7fdc7809e410 0x7fdc78134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc76ffd700 1 --2- 172.21.15.89:0/1134963591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809d6f0 0x7fdc7809dac0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009870 tx=0x7fdc68009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b535ca4a6813601a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc75ffb700 1 -- 172.21.15.89:0/1134963591 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68011830 con 0x7fdc7809d6f0 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc75ffb700 1 -- 172.21.15.89:0/1134963591 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc68011990 con 0x7fdc7809d6f0 2023-09-30T22:20:07.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.233+0000 7fdc75ffb700 1 -- 172.21.15.89:0/1134963591 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68011d20 con 0x7fdc7809d6f0 2023-09-30T22:20:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.234+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/1134963591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809d6f0 msgr2=0x7fdc7809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.234+0000 7fdc7d9ac700 1 --2- 172.21.15.89:0/1134963591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809d6f0 0x7fdc7809dac0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009870 tx=0x7fdc68009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.234+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/1134963591 shutdown_connections 2023-09-30T22:20:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.234+0000 7fdc7d9ac700 1 --2- 172.21.15.89:0/1134963591 >> v2:172.21.15.89:3300/0 conn(0x7fdc7809e410 0x7fdc78134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.234+0000 7fdc7d9ac700 1 --2- 172.21.15.89:0/1134963591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809d6f0 0x7fdc7809dac0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.234+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/1134963591 >> 172.21.15.89:0/1134963591 conn(0x7fdc78066f50 msgr2=0x7fdc7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.234+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/1134963591 shutdown_connections 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.234+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/1134963591 wait complete. 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc7d9ac700 1 Processor -- start 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc7d9ac700 1 -- start start 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc7d9ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc7809d6f0 0x7fdc781a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc7d9ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7809e000 0x7fdc781a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc7d9ac700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809e410 0x7fdc781a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc7d9ac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781a9b00 con 0x7fdc7809e000 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc7d9ac700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781a81d0 con 0x7fdc7809e410 2023-09-30T22:20:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc7d9ac700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781a9290 con 0x7fdc7809d6f0 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc777fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809e410 0x7fdc781a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc767fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7809e000 0x7fdc781a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc777fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809e410 0x7fdc781a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47502/0 (socket says 172.21.15.89:47502) 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc777fe700 1 -- 172.21.15.89:0/2221180412 learned_addr learned my addr 172.21.15.89:0/2221180412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.235+0000 7fdc76ffd700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc7809d6f0 0x7fdc781a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc767fc700 1 -- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc7809d6f0 msgr2=0x7fdc781a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc767fc700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc7809d6f0 0x7fdc781a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc767fc700 1 -- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809e410 msgr2=0x7fdc781a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc767fc700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809e410 0x7fdc781a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc767fc700 1 -- 172.21.15.89:0/2221180412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc70009b30 con 0x7fdc7809e000 2023-09-30T22:20:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc76ffd700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc7809d6f0 0x7fdc781a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc777fe700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809e410 0x7fdc781a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc767fc700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7809e000 0x7fdc781a65b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdc7806e510 tx=0x7fdc6c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc5ffff700 1 -- 172.21.15.89:0/2221180412 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0052d0 con 0x7fdc7809e000 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc5ffff700 1 -- 172.21.15.89:0/2221180412 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c005430 con 0x7fdc7809e000 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc5ffff700 1 -- 172.21.15.89:0/2221180412 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0038c0 con 0x7fdc7809e000 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc680094a0 con 0x7fdc7809e000 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.236+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7809c760 con 0x7fdc7809e000 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.237+0000 7fdc5ffff700 1 -- 172.21.15.89:0/2221180412 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc6c003c30 con 0x7fdc7809e000 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.237+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc78074ea0 con 0x7fdc7809e000 2023-09-30T22:20:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.237+0000 7fdc5ffff700 1 -- 172.21.15.89:0/2221180412 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdc6c03ec60 con 0x7fdc7809e000 2023-09-30T22:20:07.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.246+0000 7fdc5ffff700 1 -- 172.21.15.89:0/2221180412 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc6c018260 con 0x7fdc7809e000 2023-09-30T22:20:07.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:07.312 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.333+0000 7f4fd7930700 1 Processor -- start 2023-09-30T22:20:07.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.334+0000 7f4fd7930700 1 -- start start 2023-09-30T22:20:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.334+0000 7f4fd7930700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4fd012fc70 0x7f4fd0130040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.334+0000 7f4fd7930700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd012c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.334+0000 7f4fd7930700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd009be70 con 0x7f4fd012c970 2023-09-30T22:20:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.334+0000 7f4fd7930700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd0130580 con 0x7f4fd012fc70 2023-09-30T22:20:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.334+0000 7f4fd7930700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd009a6e0 con 0x7f4fd012c060 2023-09-30T22:20:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd4ecb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd012c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd4ecb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd012c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36640/0 (socket says 172.21.15.89:36640) 2023-09-30T22:20:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd4ecb700 1 -- 172.21.15.89:0/247143040 learned_addr learned my addr 172.21.15.89:0/247143040 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd56cc700 1 --2- 172.21.15.89:0/247143040 >> v2:172.21.15.89:3300/0 conn(0x7f4fd012fc70 0x7f4fd0130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd4ecb700 1 -- 172.21.15.89:0/247143040 >> v2:172.21.15.89:3300/0 conn(0x7f4fd012fc70 msgr2=0x7f4fd0130040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:07.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd4ecb700 1 --2- 172.21.15.89:0/247143040 >> v2:172.21.15.89:3300/0 conn(0x7f4fd012fc70 0x7f4fd0130040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd4ecb700 1 -- 172.21.15.89:0/247143040 >> v1:172.21.15.89:6789/0 conn(0x7f4fd012c970 legacy=0x7f4fd0136940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd4ecb700 1 -- 172.21.15.89:0/247143040 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd009ac80 con 0x7f4fd012c060 2023-09-30T22:20:07.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.335+0000 7f4fd4ecb700 1 --2- 172.21.15.89:0/247143040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd012c430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00dd10 tx=0x7f4fcc00f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe1749635d6305d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.336+0000 7f4fc7fff700 1 -- 172.21.15.89:0/247143040 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc012070 con 0x7f4fd012c060 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.336+0000 7f4fc7fff700 1 -- 172.21.15.89:0/247143040 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc00d2e0 con 0x7f4fd012c060 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.336+0000 7f4fc7fff700 1 -- 172.21.15.89:0/247143040 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00fbc0 con 0x7f4fd012c060 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.337+0000 7f4fd7930700 1 -- 172.21.15.89:0/247143040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 msgr2=0x7f4fd012c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.337+0000 7f4fd7930700 1 --2- 172.21.15.89:0/247143040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd012c430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00dd10 tx=0x7f4fcc00f330 comp rx=0 tx=0).stop 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.337+0000 7f4fd7930700 1 -- 172.21.15.89:0/247143040 shutdown_connections 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.337+0000 7f4fd7930700 1 --2- 172.21.15.89:0/247143040 >> v2:172.21.15.89:3300/0 conn(0x7f4fd012fc70 0x7f4fd0130040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.337+0000 7f4fd7930700 1 --2- 172.21.15.89:0/247143040 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd012c430 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.337+0000 7f4fd7930700 1 -- 172.21.15.89:0/247143040 >> 172.21.15.89:0/247143040 conn(0x7f4fd009e7e0 msgr2=0x7f4fd009ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:07.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.337+0000 7f4fd7930700 1 -- 172.21.15.89:0/247143040 shutdown_connections 2023-09-30T22:20:07.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.337+0000 7f4fd7930700 1 -- 172.21.15.89:0/247143040 wait complete. 2023-09-30T22:20:07.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.338+0000 7f4fd7930700 1 Processor -- start 2023-09-30T22:20:07.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.338+0000 7f4fd7930700 1 -- start start 2023-09-30T22:20:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd7930700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd0128fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd7930700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd012c970 0x7f4fd01294f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd7930700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fd012fc70 0x7f4fd01c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd7930700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0128b60 con 0x7f4fd012c970 2023-09-30T22:20:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd7930700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd006f2b0 con 0x7f4fd012fc70 2023-09-30T22:20:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd7930700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd006ea50 con 0x7f4fd012c060 2023-09-30T22:20:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd56cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd0128fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd56cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd0128fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36656/0 (socket says 172.21.15.89:36656) 2023-09-30T22:20:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd56cc700 1 -- 172.21.15.89:0/3413282185 learned_addr learned my addr 172.21.15.89:0/3413282185 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd4ecb700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd012c970 0x7f4fd01294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.339+0000 7f4fd5ecd700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fd012fc70 0x7f4fd01c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.340+0000 7f4fd56cc700 1 -- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fd012fc70 msgr2=0x7f4fd01c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.340+0000 7f4fd56cc700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fd012fc70 0x7f4fd01c4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.340+0000 7f4fd56cc700 1 -- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd012c970 msgr2=0x7f4fd01294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.340+0000 7f4fd56cc700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd012c970 0x7f4fd01294f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.340+0000 7f4fd56cc700 1 -- 172.21.15.89:0/3413282185 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc00d9f0 con 0x7f4fd012c060 2023-09-30T22:20:07.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.340+0000 7f4fd56cc700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd0128fb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8007ef0 tx=0x7f4fc800e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.341+0000 7f4fc5ffb700 1 -- 172.21.15.89:0/3413282185 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800edf0 con 0x7f4fd012c060 2023-09-30T22:20:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.341+0000 7f4fc5ffb700 1 -- 172.21.15.89:0/3413282185 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc8014420 con 0x7f4fd012c060 2023-09-30T22:20:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.341+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd006f570 con 0x7f4fd012c060 2023-09-30T22:20:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.341+0000 7f4fc5ffb700 1 -- 172.21.15.89:0/3413282185 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc80147b0 con 0x7f4fd012c060 2023-09-30T22:20:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.341+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd006e5b0 con 0x7f4fd012c060 2023-09-30T22:20:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.342+0000 7f4fc5ffb700 1 -- 172.21.15.89:0/3413282185 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4fc8014910 con 0x7f4fd012c060 2023-09-30T22:20:07.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.342+0000 7f4fc5ffb700 1 -- 172.21.15.89:0/3413282185 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4fc803fea0 con 0x7f4fd012c060 2023-09-30T22:20:07.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.343+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd00747f0 con 0x7f4fd012c060 2023-09-30T22:20:07.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.349+0000 7f4fc5ffb700 1 -- 172.21.15.89:0/3413282185 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4fc8014da0 con 0x7f4fd012c060 2023-09-30T22:20:07.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.414+0000 7f4d37819700 1 Processor -- start 2023-09-30T22:20:07.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.415+0000 7f4d37819700 1 -- start start 2023-09-30T22:20:07.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.415+0000 7f4d37819700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009d6f0 0x7f4d3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.415+0000 7f4d37819700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d3009e410 0x7f4d30134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.415+0000 7f4d37819700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3009b120 con 0x7f4d3009e000 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.415+0000 7f4d37819700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3006b460 con 0x7f4d3009e410 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.415+0000 7f4d37819700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3009a7e0 con 0x7f4d3009d6f0 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.415+0000 7f4d355b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009d6f0 0x7f4d3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.415+0000 7f4d355b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009d6f0 0x7f4d3009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47524/0 (socket says 172.21.15.89:47524) 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d355b5700 1 -- 172.21.15.89:0/1797756427 learned_addr learned my addr 172.21.15.89:0/1797756427 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d35db6700 1 --2- 172.21.15.89:0/1797756427 >> v2:172.21.15.89:3300/0 conn(0x7f4d3009e410 0x7f4d30134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d355b5700 1 -- 172.21.15.89:0/1797756427 >> v2:172.21.15.89:3300/0 conn(0x7f4d3009e410 msgr2=0x7f4d30134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d355b5700 1 --2- 172.21.15.89:0/1797756427 >> v2:172.21.15.89:3300/0 conn(0x7f4d3009e410 0x7f4d30134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d355b5700 1 -- 172.21.15.89:0/1797756427 >> v1:172.21.15.89:6789/0 conn(0x7f4d3009e000 legacy=0x7f4d301339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d355b5700 1 -- 172.21.15.89:0/1797756427 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3009a500 con 0x7f4d3009d6f0 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d355b5700 1 --2- 172.21.15.89:0/1797756427 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009d6f0 0x7f4d3009dac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009920 tx=0x7f4d2c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27fff4d00a43e074 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d27fff700 1 -- 172.21.15.89:0/1797756427 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c011830 con 0x7f4d3009d6f0 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d27fff700 1 -- 172.21.15.89:0/1797756427 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c011990 con 0x7f4d3009d6f0 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.416+0000 7f4d27fff700 1 -- 172.21.15.89:0/1797756427 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c011d00 con 0x7f4d3009d6f0 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.417+0000 7f4d37819700 1 -- 172.21.15.89:0/1797756427 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009d6f0 msgr2=0x7f4d3009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.417+0000 7f4d37819700 1 --2- 172.21.15.89:0/1797756427 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009d6f0 0x7f4d3009dac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009920 tx=0x7f4d2c009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.417+0000 7f4d37819700 1 -- 172.21.15.89:0/1797756427 shutdown_connections 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.417+0000 7f4d37819700 1 --2- 172.21.15.89:0/1797756427 >> v2:172.21.15.89:3300/0 conn(0x7f4d3009e410 0x7f4d30134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.417+0000 7f4d37819700 1 --2- 172.21.15.89:0/1797756427 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009d6f0 0x7f4d3009dac0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.417+0000 7f4d37819700 1 -- 172.21.15.89:0/1797756427 >> 172.21.15.89:0/1797756427 conn(0x7f4d30066f50 msgr2=0x7f4d3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.417+0000 7f4d37819700 1 -- 172.21.15.89:0/1797756427 shutdown_connections 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.417+0000 7f4d37819700 1 -- 172.21.15.89:0/1797756427 wait complete. 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.418+0000 7f4d37819700 1 Processor -- start 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.418+0000 7f4d37819700 1 -- start start 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d37819700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d3009d6f0 0x7f4d301a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d37819700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009e000 0x7f4d301a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d37819700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d3009e410 0x7f4d301a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d37819700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301a6ad0 con 0x7f4d3009d6f0 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d37819700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301a7d20 con 0x7f4d3009e000 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d37819700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301a70c0 con 0x7f4d3009e410 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d355b5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d3009d6f0 0x7f4d301a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d355b5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d3009d6f0 0x7f4d301a4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59852/0 (socket says 172.21.15.89:59852) 2023-09-30T22:20:07.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d355b5700 1 -- 172.21.15.89:0/1490398200 learned_addr learned my addr 172.21.15.89:0/1490398200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.419+0000 7f4d34db4700 1 --2- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009e000 0x7f4d301a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d355b5700 1 -- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d3009e410 msgr2=0x7f4d301a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d355b5700 1 --2- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d3009e410 0x7f4d301a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d355b5700 1 -- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009e000 msgr2=0x7f4d301a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d355b5700 1 --2- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009e000 0x7f4d301a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d355b5700 1 -- 172.21.15.89:0/1490398200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c009550 con 0x7f4d3009d6f0 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d355b5700 1 --2- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d3009d6f0 0x7f4d301a4ec0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009870 tx=0x7f4d2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d25ffb700 1 -- 172.21.15.89:0/1490398200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c004300 con 0x7f4d3009d6f0 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d25ffb700 1 -- 172.21.15.89:0/1490398200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c01f070 con 0x7f4d3009d6f0 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.420+0000 7f4d25ffb700 1 -- 172.21.15.89:0/1490398200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0036a0 con 0x7f4d3009d6f0 2023-09-30T22:20:07.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.421+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d301a8170 con 0x7f4d3009d6f0 2023-09-30T22:20:07.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.421+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3009c710 con 0x7f4d3009d6f0 2023-09-30T22:20:07.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.422+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d30074ea0 con 0x7f4d3009d6f0 2023-09-30T22:20:07.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.422+0000 7f4d25ffb700 1 -- 172.21.15.89:0/1490398200 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d2c021030 con 0x7f4d3009d6f0 2023-09-30T22:20:07.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.422+0000 7f4d25ffb700 1 -- 172.21.15.89:0/1490398200 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4d2c0408c0 con 0x7f4d3009d6f0 2023-09-30T22:20:07.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.431+0000 7f4d25ffb700 1 -- 172.21.15.89:0/1490398200 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d2c0199a0 con 0x7f4d3009d6f0 2023-09-30T22:20:07.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.457+0000 7fef20987700 1 Processor -- start 2023-09-30T22:20:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.457+0000 7fef20987700 1 -- start start 2023-09-30T22:20:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef20987700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c0a0a30 0x7fef1c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef20987700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c138d80 0x7fef1c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef20987700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c0a2a70 con 0x7fef1c09d630 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef20987700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0a01d0 con 0x7fef1c138d80 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef20987700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c09e2b0 con 0x7fef1c0a0a30 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef1a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c0a0a30 0x7fef1c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef1a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c0a0a30 0x7fef1c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36674/0 (socket says 172.21.15.89:36674) 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef1a59c700 1 -- 172.21.15.89:0/1226846377 learned_addr learned my addr 172.21.15.89:0/1226846377 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef1ad9d700 1 --2- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c138d80 0x7fef1c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef1ad9d700 1 -- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c0a0a30 msgr2=0x7fef1c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef1ad9d700 1 --2- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c0a0a30 0x7fef1c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.458+0000 7fef1ad9d700 1 -- 172.21.15.89:0/1226846377 >> v1:172.21.15.89:6789/0 conn(0x7fef1c09d630 legacy=0x7fef1c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.459+0000 7fef1ad9d700 1 -- 172.21.15.89:0/1226846377 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c09dc50 con 0x7fef1c138d80 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.459+0000 7fef1a59c700 1 --2- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c0a0a30 0x7fef1c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.459+0000 7fef1ad9d700 1 --2- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c138d80 0x7fef1c13b160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef14009dc0 tx=0x7fef1400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2025f97a39a42f8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.459+0000 7fef1959a700 1 -- 172.21.15.89:0/1226846377 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400e070 con 0x7fef1c138d80 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.459+0000 7fef1959a700 1 -- 172.21.15.89:0/1226846377 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1400bb20 con 0x7fef1c138d80 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.459+0000 7fef1959a700 1 -- 172.21.15.89:0/1226846377 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14012430 con 0x7fef1c138d80 2023-09-30T22:20:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.459+0000 7fef20987700 1 -- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c138d80 msgr2=0x7fef1c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.459+0000 7fef20987700 1 --2- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c138d80 0x7fef1c13b160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef14009dc0 tx=0x7fef1400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.460+0000 7fef20987700 1 -- 172.21.15.89:0/1226846377 shutdown_connections 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.460+0000 7fef20987700 1 --2- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c138d80 0x7fef1c13b160 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.460+0000 7fef20987700 1 --2- 172.21.15.89:0/1226846377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c0a0a30 0x7fef1c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.460+0000 7fef20987700 1 -- 172.21.15.89:0/1226846377 >> 172.21.15.89:0/1226846377 conn(0x7fef1c0a3050 msgr2=0x7fef1c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.460+0000 7fef20987700 1 -- 172.21.15.89:0/1226846377 shutdown_connections 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.460+0000 7fef20987700 1 -- 172.21.15.89:0/1226846377 wait complete. 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef20987700 1 Processor -- start 2023-09-30T22:20:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef20987700 1 -- start start 2023-09-30T22:20:07.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdc781a94c0 con 0x7fdc7809e000 2023-09-30T22:20:07.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef20987700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c09d630 0x7fef1c133850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef20987700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c0a0a30 0x7fef1c133d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef20987700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef1c138d80 0x7fef1c1342d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:07.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef20987700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c136fb0 con 0x7fef1c138d80 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef20987700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0a0410 con 0x7fef1c0a0a30 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef20987700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c134c40 con 0x7fef1c09d630 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef1a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c09d630 0x7fef1c133850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef1a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c09d630 0x7fef1c133850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36684/0 (socket says 172.21.15.89:36684) 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef1a59c700 1 -- 172.21.15.89:0/3672365057 learned_addr learned my addr 172.21.15.89:0/3672365057 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.461+0000 7fef19d9b700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c0a0a30 0x7fef1c133d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef1a59c700 1 -- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c0a0a30 msgr2=0x7fef1c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef1a59c700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c0a0a30 0x7fef1c133d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef1a59c700 1 -- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef1c138d80 msgr2=0x7fef1c1342d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef1a59c700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef1c138d80 0x7fef1c1342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef1a59c700 1 -- 172.21.15.89:0/3672365057 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c009550 con 0x7fef1c09d630 2023-09-30T22:20:07.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef19d9b700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c0a0a30 0x7fef1c133d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:07.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef1a59c700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c09d630 0x7fef1c133850 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00ebe0 tx=0x7fef0c009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:07.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef0affd700 1 -- 172.21.15.89:0/3672365057 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00c8b0 con 0x7fef1c09d630 2023-09-30T22:20:07.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef0affd700 1 -- 172.21.15.89:0/3672365057 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c00ca10 con 0x7fef1c09d630 2023-09-30T22:20:07.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef14009a70 con 0x7fef1c09d630 2023-09-30T22:20:07.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef0affd700 1 -- 172.21.15.89:0/3672365057 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00cd80 con 0x7fef1c09d630 2023-09-30T22:20:07.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fdc5ffff700 1 -- 172.21.15.89:0/2221180412 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc781a94c0 con 0x7fdc7809e000 2023-09-30T22:20:07.731 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:07.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.462+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c136cf0 con 0x7fef1c09d630 2023-09-30T22:20:07.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.463+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c0747f0 con 0x7fef1c09d630 2023-09-30T22:20:07.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.464+0000 7fef0affd700 1 -- 172.21.15.89:0/3672365057 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef0c026480 con 0x7fef1c09d630 2023-09-30T22:20:07.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.464+0000 7fef0affd700 1 -- 172.21.15.89:0/3672365057 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef0c040260 con 0x7fef1c09d630 2023-09-30T22:20:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7809e000 msgr2=0x7fdc781a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7809e000 0x7fdc781a65b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdc7806e510 tx=0x7fdc6c00c730 comp rx=0 tx=0).stop 2023-09-30T22:20:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 shutdown_connections 2023-09-30T22:20:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc7809e410 0x7fdc781a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc7809d6f0 0x7fdc781a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 --2- 172.21.15.89:0/2221180412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7809e000 0x7fdc781a65b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 >> 172.21.15.89:0/2221180412 conn(0x7fdc78066f50 msgr2=0x7fdc781baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 shutdown_connections 2023-09-30T22:20:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.465+0000 7fdc7d9ac700 1 -- 172.21.15.89:0/2221180412 wait complete. 2023-09-30T22:20:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.468+0000 7fef0affd700 1 -- 172.21.15.89:0/3672365057 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef0c011760 con 0x7fef1c09d630 2023-09-30T22:20:07.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:07.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.570+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4fd00a2ad0 con 0x7f4fd012c060 2023-09-30T22:20:07.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.570+0000 7f4fc5ffb700 1 -- 172.21.15.89:0/3413282185 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4fc801bb00 con 0x7f4fd012c060 2023-09-30T22:20:07.735 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:07.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 msgr2=0x7f4fd0128fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd0128fb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8007ef0 tx=0x7f4fc800e3c0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 shutdown_connections 2023-09-30T22:20:07.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4fd012fc70 0x7f4fd01c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4fd012c060 0x7f4fd0128fb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 --2- 172.21.15.89:0/3413282185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4fd012c970 0x7f4fd01294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 >> 172.21.15.89:0/3413282185 conn(0x7f4fd009e7e0 msgr2=0x7f4fd01b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 shutdown_connections 2023-09-30T22:20:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.572+0000 7f4fd7930700 1 -- 172.21.15.89:0/3413282185 wait complete. 2023-09-30T22:20:07.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.627+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4d3006c360 con 0x7f4d3009d6f0 2023-09-30T22:20:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.627+0000 7f4d25ffb700 1 -- 172.21.15.89:0/1490398200 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d2c015030 con 0x7f4d3009d6f0 2023-09-30T22:20:07.738 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:07.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d3009d6f0 msgr2=0x7f4d301a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 --2- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d3009d6f0 0x7f4d301a4ec0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009870 tx=0x7f4d2c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:07.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 shutdown_connections 2023-09-30T22:20:07.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 --2- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d3009e410 0x7f4d301a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 --2- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d3009d6f0 0x7f4d301a4ec0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 --2- 172.21.15.89:0/1490398200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d3009e000 0x7f4d301a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 >> 172.21.15.89:0/1490398200 conn(0x7f4d30066f50 msgr2=0x7f4d301bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:07.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 shutdown_connections 2023-09-30T22:20:07.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.629+0000 7f4d37819700 1 -- 172.21.15.89:0/1490398200 wait complete. 2023-09-30T22:20:07.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:07.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.671+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fef1c1371f0 con 0x7fef1c09d630 2023-09-30T22:20:07.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.672+0000 7fef0affd700 1 -- 172.21.15.89:0/3672365057 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef0c017020 con 0x7fef1c09d630 2023-09-30T22:20:07.741 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:07.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c09d630 msgr2=0x7fef1c133850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:07.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c09d630 0x7fef1c133850 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00ebe0 tx=0x7fef0c009e70 comp rx=0 tx=0).stop 2023-09-30T22:20:07.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 shutdown_connections 2023-09-30T22:20:07.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef1c138d80 0x7fef1c1342d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef1c09d630 0x7fef1c133850 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 --2- 172.21.15.89:0/3672365057 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef1c0a0a30 0x7fef1c133d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:07.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 >> 172.21.15.89:0/3672365057 conn(0x7fef1c0a3050 msgr2=0x7fef1c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:07.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 shutdown_connections 2023-09-30T22:20:07.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:07.674+0000 7fef20987700 1 -- 172.21.15.89:0/3672365057 wait complete. 2023-09-30T22:20:07.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:08.735 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:08.737 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:08.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:08.744 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.878+0000 7f9fad604700 1 Processor -- start 2023-09-30T22:20:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fad604700 1 -- start start 2023-09-30T22:20:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fad604700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa8133830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fad604700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9fa8133d70 0x7f9fa81303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fad604700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa81341f0 con 0x7f9fa8133280 2023-09-30T22:20:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fad604700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa806ab40 con 0x7f9fa8133d70 2023-09-30T22:20:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fad604700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa809a2c0 con 0x7f9fa806a770 2023-09-30T22:20:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fa6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa8133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fa6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa8133830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36696/0 (socket says 172.21.15.89:36696) 2023-09-30T22:20:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.879+0000 7f9fa6ffd700 1 -- 172.21.15.89:0/4158895631 learned_addr learned my addr 172.21.15.89:0/4158895631 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa77fe700 1 --2- 172.21.15.89:0/4158895631 >> v2:172.21.15.89:3300/0 conn(0x7f9fa8133d70 0x7f9fa81303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa6ffd700 1 -- 172.21.15.89:0/4158895631 >> v2:172.21.15.89:3300/0 conn(0x7f9fa8133d70 msgr2=0x7f9fa81303d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa6ffd700 1 --2- 172.21.15.89:0/4158895631 >> v2:172.21.15.89:3300/0 conn(0x7f9fa8133d70 0x7f9fa81303d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa6ffd700 1 -- 172.21.15.89:0/4158895631 >> v1:172.21.15.89:6789/0 conn(0x7f9fa8133280 legacy=0x7f9fa812fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:08.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa6ffd700 1 -- 172.21.15.89:0/4158895631 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa809f440 con 0x7f9fa806a770 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa6ffd700 1 --2- 172.21.15.89:0/4158895631 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa8133830 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9f980098f0 tx=0x7f9f98009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f5cd286f0ac352 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa5ffb700 1 -- 172.21.15.89:0/4158895631 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98004230 con 0x7f9fa806a770 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa5ffb700 1 -- 172.21.15.89:0/4158895631 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f98004390 con 0x7f9fa806a770 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.880+0000 7f9fa5ffb700 1 -- 172.21.15.89:0/4158895631 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980036a0 con 0x7f9fa806a770 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.881+0000 7f9fad604700 1 -- 172.21.15.89:0/4158895631 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 msgr2=0x7f9fa8133830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.881+0000 7f9fad604700 1 --2- 172.21.15.89:0/4158895631 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa8133830 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9f980098f0 tx=0x7f9f98009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.881+0000 7f9fad604700 1 -- 172.21.15.89:0/4158895631 shutdown_connections 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.881+0000 7f9fad604700 1 --2- 172.21.15.89:0/4158895631 >> v2:172.21.15.89:3300/0 conn(0x7f9fa8133d70 0x7f9fa81303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.881+0000 7f9fad604700 1 --2- 172.21.15.89:0/4158895631 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa8133830 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.881+0000 7f9fad604700 1 -- 172.21.15.89:0/4158895631 >> 172.21.15.89:0/4158895631 conn(0x7f9fa80a2cc0 msgr2=0x7f9fa80a30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.881+0000 7f9fad604700 1 -- 172.21.15.89:0/4158895631 shutdown_connections 2023-09-30T22:20:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.881+0000 7f9fad604700 1 -- 172.21.15.89:0/4158895631 wait complete. 2023-09-30T22:20:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.882+0000 7f9fad604700 1 Processor -- start 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fad604700 1 -- start start 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fad604700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa81343b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fad604700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fa8133280 0x7f9fa81348f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fad604700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fa8133d70 0x7f9fa8134e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fad604700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8137720 con 0x7f9fa8133280 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fad604700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80722b0 con 0x7f9fa8133d70 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fad604700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa813ad00 con 0x7f9fa806a770 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fa6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa81343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fa6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa81343b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36702/0 (socket says 172.21.15.89:36702) 2023-09-30T22:20:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fa6ffd700 1 -- 172.21.15.89:0/2464421882 learned_addr learned my addr 172.21.15.89:0/2464421882 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.883+0000 7f9fa67fc700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fa8133280 0x7f9fa81348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fa77fe700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fa8133d70 0x7f9fa8134e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fa6ffd700 1 -- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fa8133d70 msgr2=0x7f9fa8134e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fa6ffd700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fa8133d70 0x7f9fa8134e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fa6ffd700 1 -- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fa8133280 msgr2=0x7f9fa81348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fa6ffd700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fa8133280 0x7f9fa81348f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fa6ffd700 1 -- 172.21.15.89:0/2464421882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98009550 con 0x7f9fa806a770 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fa77fe700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fa8133d70 0x7f9fa8134e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fa6ffd700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa81343b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800b0e0 tx=0x7f9f98004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9f8f7fe700 1 -- 172.21.15.89:0/2464421882 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98003e50 con 0x7f9fa806a770 2023-09-30T22:20:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9f8f7fe700 1 -- 172.21.15.89:0/2464421882 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9800b660 con 0x7f9fa806a770 2023-09-30T22:20:08.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9f8f7fe700 1 -- 172.21.15.89:0/2464421882 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800ba10 con 0x7f9fa806a770 2023-09-30T22:20:08.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa809c560 con 0x7f9fa806a770 2023-09-30T22:20:08.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.884+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa8071810 con 0x7f9fa806a770 2023-09-30T22:20:08.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.885+0000 7febe4ceb700 1 Processor -- start 2023-09-30T22:20:08.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.885+0000 7febe4ceb700 1 -- start start 2023-09-30T22:20:08.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7f9f8f7fe700 1 -- 172.21.15.89:0/2464421882 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f98021070 con 0x7f9fa806a770 2023-09-30T22:20:08.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.885+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa804f2a0 con 0x7f9fa806a770 2023-09-30T22:20:08.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7f9f8f7fe700 1 -- 172.21.15.89:0/2464421882 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9f98040c20 con 0x7f9fa806a770 2023-09-30T22:20:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febe4ceb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009d6f0 0x7febe009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febe4ceb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7febe009e410 0x7febe0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febe4ceb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe009b120 con 0x7febe009e000 2023-09-30T22:20:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febe4ceb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe006b460 con 0x7febe009e410 2023-09-30T22:20:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febe4ceb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febe009a7e0 con 0x7febe009d6f0 2023-09-30T22:20:08.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febddd9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7febe009e000 0x7febe01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35708/0 (socket says 172.21.15.89:35708) 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febde59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009d6f0 0x7febe009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febddd9b700 1 -- 172.21.15.89:0/3003857780 learned_addr learned my addr 172.21.15.89:0/3003857780 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febded9d700 1 --2- 172.21.15.89:0/3003857780 >> v2:172.21.15.89:3300/0 conn(0x7febe009e410 0x7febe0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.887+0000 7febde59c700 1 -- 172.21.15.89:0/3003857780 >> v2:172.21.15.89:3300/0 conn(0x7febe009e410 msgr2=0x7febe0134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.886+0000 7febdd59a700 1 -- 172.21.15.89:0/3003857780 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453948655 0 0) 0x7febe009b120 con 0x7febe009e000 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.887+0000 7febde59c700 1 --2- 172.21.15.89:0/3003857780 >> v2:172.21.15.89:3300/0 conn(0x7febe009e410 0x7febe0134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.887+0000 7febde59c700 1 -- 172.21.15.89:0/3003857780 >> v1:172.21.15.89:6789/0 conn(0x7febe009e000 legacy=0x7febe01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.887+0000 7febde59c700 1 -- 172.21.15.89:0/3003857780 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe009a500 con 0x7febe009d6f0 2023-09-30T22:20:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.887+0000 7febde59c700 1 --2- 172.21.15.89:0/3003857780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009d6f0 0x7febe009dac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7febc8009870 tx=0x7febc8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff7d130adaefe05 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.891+0000 7febdd59a700 1 -- 172.21.15.89:0/3003857780 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8011830 con 0x7febe009d6f0 2023-09-30T22:20:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.891+0000 7febdd59a700 1 -- 172.21.15.89:0/3003857780 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc8011990 con 0x7febe009d6f0 2023-09-30T22:20:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.891+0000 7febdd59a700 1 -- 172.21.15.89:0/3003857780 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8011d20 con 0x7febe009d6f0 2023-09-30T22:20:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.891+0000 7f9f8f7fe700 1 -- 172.21.15.89:0/2464421882 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f9802e7b0 con 0x7f9fa806a770 2023-09-30T22:20:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.892+0000 7febe4ceb700 1 -- 172.21.15.89:0/3003857780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009d6f0 msgr2=0x7febe009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.892+0000 7febe4ceb700 1 --2- 172.21.15.89:0/3003857780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009d6f0 0x7febe009dac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7febc8009870 tx=0x7febc8009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.892+0000 7febe4ceb700 1 -- 172.21.15.89:0/3003857780 shutdown_connections 2023-09-30T22:20:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.892+0000 7febe4ceb700 1 --2- 172.21.15.89:0/3003857780 >> v2:172.21.15.89:3300/0 conn(0x7febe009e410 0x7febe0134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.892+0000 7febe4ceb700 1 --2- 172.21.15.89:0/3003857780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009d6f0 0x7febe009dac0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.892+0000 7febe4ceb700 1 -- 172.21.15.89:0/3003857780 >> 172.21.15.89:0/3003857780 conn(0x7febe0066f50 msgr2=0x7febe009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.892+0000 7febe4ceb700 1 -- 172.21.15.89:0/3003857780 shutdown_connections 2023-09-30T22:20:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.892+0000 7febe4ceb700 1 -- 172.21.15.89:0/3003857780 wait complete. 2023-09-30T22:20:08.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.893+0000 7febe4ceb700 1 Processor -- start 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.893+0000 7febe4ceb700 1 -- start start 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febe4ceb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febe009d6f0 0x7febe01c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febe4ceb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7febe009e000 0x7febe01c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febe4ceb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009e410 0x7febe01a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febe4ceb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe01a5250 con 0x7febe009d6f0 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febe4ceb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe01a5b20 con 0x7febe009e000 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febe4ceb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febe01b7ff0 con 0x7febe009e410 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febddd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7febe009e000 0x7febe01c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febddd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7febe009e000 0x7febe01c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47564/0 (socket says 172.21.15.89:47564) 2023-09-30T22:20:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febddd9b700 1 -- 172.21.15.89:0/2229395171 learned_addr learned my addr 172.21.15.89:0/2229395171 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febded9d700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009e410 0x7febe01a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febde59c700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febe009d6f0 0x7febe01c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febded9d700 1 -- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7febe009e000 msgr2=0x7febe01c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febded9d700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7febe009e000 0x7febe01c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febded9d700 1 -- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febe009d6f0 msgr2=0x7febe01c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febded9d700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febe009d6f0 0x7febe01c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febded9d700 1 -- 172.21.15.89:0/2229395171 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80094a0 con 0x7febe009e410 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febddd9b700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7febe009e000 0x7febe01c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.894+0000 7febde59c700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febe009d6f0 0x7febe01c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.895+0000 7febded9d700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009e410 0x7febe01a3690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7febd8009d90 tx=0x7febd800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:08.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.895+0000 7febd6ffd700 1 -- 172.21.15.89:0/2229395171 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8009c00 con 0x7febe009e410 2023-09-30T22:20:08.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.895+0000 7febd6ffd700 1 -- 172.21.15.89:0/2229395171 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd800f040 con 0x7febe009e410 2023-09-30T22:20:08.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.895+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe01a71c0 con 0x7febe009e410 2023-09-30T22:20:08.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.895+0000 7febd6ffd700 1 -- 172.21.15.89:0/2229395171 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8014480 con 0x7febe009e410 2023-09-30T22:20:08.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.895+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe00a2fb0 con 0x7febe009e410 2023-09-30T22:20:08.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.896+0000 7febd6ffd700 1 -- 172.21.15.89:0/2229395171 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7febd80145e0 con 0x7febe009e410 2023-09-30T22:20:08.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.896+0000 7febd6ffd700 1 -- 172.21.15.89:0/2229395171 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7febd8040240 con 0x7febe009e410 2023-09-30T22:20:08.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.896+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe009c710 con 0x7febe009e410 2023-09-30T22:20:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.901+0000 7febd6ffd700 1 -- 172.21.15.89:0/2229395171 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7febd8007900 con 0x7febe009e410 2023-09-30T22:20:08.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41efe0d700 1 Processor -- start 2023-09-30T22:20:08.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41efe0d700 1 -- start start 2023-09-30T22:20:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41efe0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f41e809d6f0 0x7f41e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41efe0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41efe0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809e410 0x7f41e8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41efe0d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e806b460 con 0x7f41e809d6f0 2023-09-30T22:20:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41efe0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e809b120 con 0x7f41e809e000 2023-09-30T22:20:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41efe0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41e809a7e0 con 0x7f41e809e410 2023-09-30T22:20:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41edba9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f41e809d6f0 0x7f41e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41edba9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f41e809d6f0 0x7f41e809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59884/0 (socket says 172.21.15.89:59884) 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41edba9700 1 -- 172.21.15.89:0/2966929494 learned_addr learned my addr 172.21.15.89:0/2966929494 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.919+0000 7f41ed3a8700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41ee3aa700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809e410 0x7f41e8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41edba9700 1 -- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809e410 msgr2=0x7f41e8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41edba9700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809e410 0x7f41e8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41edba9700 1 -- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 msgr2=0x7f41e81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41edba9700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41edba9700 1 -- 172.21.15.89:0/2966929494 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e809a500 con 0x7f41e809d6f0 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41ed3a8700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41edba9700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809d6f0 0x7f41e809dac0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f41e40098f0 tx=0x7f41e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb40e917b80e2abd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:08.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41ecba7700 1 -- 172.21.15.89:0/2966929494 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4011830 con 0x7f41e809d6f0 2023-09-30T22:20:08.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41ecba7700 1 -- 172.21.15.89:0/2966929494 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4011990 con 0x7f41e809d6f0 2023-09-30T22:20:08.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.920+0000 7f41ecba7700 1 -- 172.21.15.89:0/2966929494 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4011d20 con 0x7f41e809d6f0 2023-09-30T22:20:08.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 -- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809d6f0 msgr2=0x7f41e809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809d6f0 0x7f41e809dac0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f41e40098f0 tx=0x7f41e4009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 -- 172.21.15.89:0/2966929494 shutdown_connections 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809e410 0x7f41e8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809d6f0 0x7f41e809dac0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 --2- 172.21.15.89:0/2966929494 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 -- 172.21.15.89:0/2966929494 >> 172.21.15.89:0/2966929494 conn(0x7f41e8066f50 msgr2=0x7f41e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 -- 172.21.15.89:0/2966929494 shutdown_connections 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.921+0000 7f41efe0d700 1 -- 172.21.15.89:0/2966929494 wait complete. 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41efe0d700 1 Processor -- start 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41efe0d700 1 -- start start 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41efe0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809d6f0 0x7f41e81a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41efe0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41efe0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809e410 0x7f41e81c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41efe0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e80a1f20 con 0x7f41e809e410 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41efe0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81a80a0 con 0x7f41e809e000 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41efe0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81b91f0 con 0x7f41e809d6f0 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41ee3aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809e410 0x7f41e81c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41edba9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809d6f0 0x7f41e81a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41ee3aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809e410 0x7f41e81c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59898/0 (socket says 172.21.15.89:59898) 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41ed3a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.922+0000 7f41ee3aa700 1 -- 172.21.15.89:0/475443474 learned_addr learned my addr 172.21.15.89:0/475443474 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41ee3aa700 1 -- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809d6f0 msgr2=0x7f41e81a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41ee3aa700 1 --2- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809d6f0 0x7f41e81a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41ee3aa700 1 -- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 msgr2=0x7f41e81a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41ee3aa700 1 --2- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41ee3aa700 1 -- 172.21.15.89:0/475443474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80094a0 con 0x7f41e809e410 2023-09-30T22:20:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41ee3aa700 1 --2- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809e410 0x7f41e81c8040 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009dd0 tx=0x7f41e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7faf69a1a700 1 Processor -- start 2023-09-30T22:20:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7faf69a1a700 1 -- start start 2023-09-30T22:20:08.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41de7fc700 1 -- 172.21.15.89:0/475443474 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0009c40 con 0x7f41e809e410 2023-09-30T22:20:08.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41de7fc700 1 -- 172.21.15.89:0/475443474 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e000f040 con 0x7f41e809e410 2023-09-30T22:20:08.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e4009550 con 0x7f41e809e410 2023-09-30T22:20:08.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e81a6c60 con 0x7f41e809e410 2023-09-30T22:20:08.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.923+0000 7f41de7fc700 1 -- 172.21.15.89:0/475443474 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e000ceb0 con 0x7f41e809e410 2023-09-30T22:20:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf69a1a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faf6409e1d0 0x7faf64135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf69a1a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf64135e50 0x7faf64138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf69a1a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6409f130 con 0x7faf6409d6f0 2023-09-30T22:20:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf69a1a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6409f520 con 0x7faf6409e1d0 2023-09-30T22:20:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf69a1a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6409e930 con 0x7faf64135e50 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf64135e50 0x7faf64138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf62ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faf6409d6f0 0x7faf6409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35720/0 (socket says 172.21.15.89:35720) 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf64135e50 0x7faf64138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47600/0 (socket says 172.21.15.89:47600) 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf62ffd700 1 -- 172.21.15.89:0/2291745465 learned_addr learned my addr 172.21.15.89:0/2291745465 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf627fc700 1 --2- 172.21.15.89:0/2291745465 >> v2:172.21.15.89:3300/0 conn(0x7faf6409e1d0 0x7faf64135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf637fe700 1 -- 172.21.15.89:0/2291745465 >> v2:172.21.15.89:3300/0 conn(0x7faf6409e1d0 msgr2=0x7faf64135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf637fe700 1 --2- 172.21.15.89:0/2291745465 >> v2:172.21.15.89:3300/0 conn(0x7faf6409e1d0 0x7faf64135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf637fe700 1 -- 172.21.15.89:0/2291745465 >> v1:172.21.15.89:6789/0 conn(0x7faf6409d6f0 legacy=0x7faf6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf637fe700 1 -- 172.21.15.89:0/2291745465 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6409b2d0 con 0x7faf64135e50 2023-09-30T22:20:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.924+0000 7faf637fe700 1 --2- 172.21.15.89:0/2291745465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf64135e50 0x7faf64138230 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faf58009840 tx=0x7faf58009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d53c99a1a6bd07 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:08.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.925+0000 7f41de7fc700 1 -- 172.21.15.89:0/475443474 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f41e00144a0 con 0x7f41e809e410 2023-09-30T22:20:08.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.925+0000 7faf61ffb700 1 -- 172.21.15.89:0/2291745465 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58011830 con 0x7faf64135e50 2023-09-30T22:20:08.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.925+0000 7faf61ffb700 1 -- 172.21.15.89:0/2291745465 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf58011990 con 0x7faf64135e50 2023-09-30T22:20:08.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.925+0000 7faf61ffb700 1 -- 172.21.15.89:0/2291745465 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58011d00 con 0x7faf64135e50 2023-09-30T22:20:08.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.925+0000 7f41de7fc700 1 -- 172.21.15.89:0/475443474 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f41e003ff60 con 0x7f41e809e410 2023-09-30T22:20:08.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.925+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d00052e0 con 0x7f41e809e410 2023-09-30T22:20:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.926+0000 7faf69a1a700 1 -- 172.21.15.89:0/2291745465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf64135e50 msgr2=0x7faf64138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.926+0000 7faf69a1a700 1 --2- 172.21.15.89:0/2291745465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf64135e50 0x7faf64138230 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faf58009840 tx=0x7faf58009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.926+0000 7faf69a1a700 1 -- 172.21.15.89:0/2291745465 shutdown_connections 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.926+0000 7faf69a1a700 1 --2- 172.21.15.89:0/2291745465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf64135e50 0x7faf64138230 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.926+0000 7faf69a1a700 1 --2- 172.21.15.89:0/2291745465 >> v2:172.21.15.89:3300/0 conn(0x7faf6409e1d0 0x7faf64135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.926+0000 7faf69a1a700 1 -- 172.21.15.89:0/2291745465 >> 172.21.15.89:0/2291745465 conn(0x7faf64066f50 msgr2=0x7faf6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.926+0000 7faf69a1a700 1 -- 172.21.15.89:0/2291745465 shutdown_connections 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.926+0000 7faf69a1a700 1 -- 172.21.15.89:0/2291745465 wait complete. 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf69a1a700 1 Processor -- start 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf69a1a700 1 -- start start 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf69a1a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf6409d6f0 0x7faf6411d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf69a1a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf6409e1d0 0x7faf6411ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf69a1a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf64135e50 0x7faf6411e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf69a1a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6411fd20 con 0x7faf64135e50 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf69a1a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6409c700 con 0x7faf6409d6f0 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf69a1a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf6413a140 con 0x7faf6409e1d0 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf62ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf6409d6f0 0x7faf6411d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf62ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf6409d6f0 0x7faf6411d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47610/0 (socket says 172.21.15.89:47610) 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf62ffd700 1 -- 172.21.15.89:0/2440081340 learned_addr learned my addr 172.21.15.89:0/2440081340 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.927+0000 7faf627fc700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf6409e1d0 0x7faf6411ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf62ffd700 1 -- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf6409e1d0 msgr2=0x7faf6411ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf62ffd700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf6409e1d0 0x7faf6411ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf62ffd700 1 -- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf64135e50 msgr2=0x7faf6411e310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf62ffd700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf64135e50 0x7faf6411e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf62ffd700 1 -- 172.21.15.89:0/2440081340 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580094a0 con 0x7faf6409d6f0 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf627fc700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf6409e1d0 0x7faf6411ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf62ffd700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf6409d6f0 0x7faf6411d890 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00cbc0 tx=0x7faf5c00cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf4bfff700 1 -- 172.21.15.89:0/2440081340 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00aaf0 con 0x7faf6409d6f0 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf4bfff700 1 -- 172.21.15.89:0/2440081340 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c00ac50 con 0x7faf6409d6f0 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6411fa60 con 0x7faf6409d6f0 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf4bfff700 1 -- 172.21.15.89:0/2440081340 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c012400 con 0x7faf6409d6f0 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.928+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf64139160 con 0x7faf6409d6f0 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.929+0000 7faf4bfff700 1 -- 172.21.15.89:0/2440081340 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf5c01b3b0 con 0x7faf6409d6f0 2023-09-30T22:20:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.929+0000 7faf4bfff700 1 -- 172.21.15.89:0/2440081340 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faf5c03f430 con 0x7faf6409d6f0 2023-09-30T22:20:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.931+0000 7f41de7fc700 1 -- 172.21.15.89:0/475443474 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41e006b750 con 0x7f41e809e410 2023-09-30T22:20:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.931+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf64074ea0 con 0x7faf6409d6f0 2023-09-30T22:20:08.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:08.939+0000 7faf4bfff700 1 -- 172.21.15.89:0/2440081340 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf5c0192c0 con 0x7faf6409d6f0 2023-09-30T22:20:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.122+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9fa81341f0 con 0x7f9fa806a770 2023-09-30T22:20:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.122+0000 7f9f8f7fe700 1 -- 172.21.15.89:0/2464421882 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9fa81341f0 con 0x7f9fa806a770 2023-09-30T22:20:09.122 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:09.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.123+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f41d0005590 con 0x7f41e809e410 2023-09-30T22:20:09.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.123+0000 7f41de7fc700 1 -- 172.21.15.89:0/475443474 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f41e0018020 con 0x7f41e809e410 2023-09-30T22:20:09.123 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:09.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.124+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 msgr2=0x7f9fa81343b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:09.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.124+0000 7f9fad604700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa81343b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800b0e0 tx=0x7f9f98004750 comp rx=0 tx=0).stop 2023-09-30T22:20:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 shutdown_connections 2023-09-30T22:20:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f9fad604700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fa8133d70 0x7f9fa8134e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f9fad604700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fa806a770 0x7f9fa81343b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f9fad604700 1 --2- 172.21.15.89:0/2464421882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fa8133280 0x7f9fa81348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 >> 172.21.15.89:0/2464421882 conn(0x7f9fa80a2cc0 msgr2=0x7f9fa809b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 shutdown_connections 2023-09-30T22:20:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f9fad604700 1 -- 172.21.15.89:0/2464421882 wait complete. 2023-09-30T22:20:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809e410 msgr2=0x7f41e81c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 --2- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809e410 0x7f41e81c8040 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009dd0 tx=0x7f41e000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 shutdown_connections 2023-09-30T22:20:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 --2- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41e809e410 0x7f41e81c8040 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 --2- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41e809d6f0 0x7f41e81a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 --2- 172.21.15.89:0/475443474 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41e809e000 0x7f41e81a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 >> 172.21.15.89:0/475443474 conn(0x7f41e8066f50 msgr2=0x7f41e81bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 shutdown_connections 2023-09-30T22:20:09.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.125+0000 7f41efe0d700 1 -- 172.21.15.89:0/475443474 wait complete. 2023-09-30T22:20:09.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.130+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7febe0074ea0 con 0x7febe009e410 2023-09-30T22:20:09.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.131+0000 7febd6ffd700 1 -- 172.21.15.89:0/2229395171 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7febd802c090 con 0x7febe009e410 2023-09-30T22:20:09.130 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.133+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009e410 msgr2=0x7febe01a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.133+0000 7febe4ceb700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009e410 0x7febe01a3690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7febd8009d90 tx=0x7febd800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.133+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faf6406c360 con 0x7faf6409d6f0 2023-09-30T22:20:09.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.133+0000 7faf4bfff700 1 -- 172.21.15.89:0/2440081340 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf5c017020 con 0x7faf6409d6f0 2023-09-30T22:20:09.133 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:09.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.135+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf6409d6f0 msgr2=0x7faf6411d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:09.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.135+0000 7faf69a1a700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf6409d6f0 0x7faf6411d890 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00cbc0 tx=0x7faf5c00cf80 comp rx=0 tx=0).stop 2023-09-30T22:20:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.135+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 shutdown_connections 2023-09-30T22:20:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.135+0000 7faf69a1a700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf64135e50 0x7faf6411e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.135+0000 7faf69a1a700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf6409d6f0 0x7faf6411d890 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.135+0000 7faf69a1a700 1 --2- 172.21.15.89:0/2440081340 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf6409e1d0 0x7faf6411ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.135+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 >> 172.21.15.89:0/2440081340 conn(0x7faf64066f50 msgr2=0x7faf640a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.136+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 shutdown_connections 2023-09-30T22:20:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.136+0000 7faf69a1a700 1 -- 172.21.15.89:0/2440081340 wait complete. 2023-09-30T22:20:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.137+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 shutdown_connections 2023-09-30T22:20:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.137+0000 7febe4ceb700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febe009e410 0x7febe01a3690 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.137+0000 7febe4ceb700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febe009d6f0 0x7febe01c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.137+0000 7febe4ceb700 1 --2- 172.21.15.89:0/2229395171 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7febe009e000 0x7febe01c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.137+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 >> 172.21.15.89:0/2229395171 conn(0x7febe0066f50 msgr2=0x7febe00a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.137+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 shutdown_connections 2023-09-30T22:20:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:09.137+0000 7febe4ceb700 1 -- 172.21.15.89:0/2229395171 wait complete. 2023-09-30T22:20:09.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:09.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:09.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:09.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:10.140 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:10.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:10.152 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:10.152 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:10.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.283+0000 7f3cd0c4c700 1 Processor -- start 2023-09-30T22:20:10.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.284+0000 7f3cd0c4c700 1 -- start start 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.284+0000 7f3cd0c4c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09d6f0 0x7f3ccc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.284+0000 7f3cd0c4c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e000 0x7f3ccc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.284+0000 7f3cd0c4c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3ccc09e410 0x7f3ccc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.284+0000 7f3cd0c4c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc06b460 con 0x7f3ccc09e410 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.284+0000 7f3cd0c4c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc09b120 con 0x7f3ccc09d6f0 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.284+0000 7f3cd0c4c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc09a7e0 con 0x7f3ccc09e000 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09d6f0 0x7f3ccc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cc9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e000 0x7f3ccc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09d6f0 0x7f3ccc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37862/0 (socket says 172.21.15.89:37862) 2023-09-30T22:20:10.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 -- 172.21.15.89:0/3862296975 learned_addr learned my addr 172.21.15.89:0/3862296975 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3ccad9d700 1 --2- 172.21.15.89:0/3862296975 >> v2:172.21.15.89:3300/0 conn(0x7f3ccc09e410 0x7f3ccc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 -- 172.21.15.89:0/3862296975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e000 msgr2=0x7f3ccc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 --2- 172.21.15.89:0/3862296975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e000 0x7f3ccc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 -- 172.21.15.89:0/3862296975 >> v2:172.21.15.89:3300/0 conn(0x7f3ccc09e410 msgr2=0x7f3ccc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 --2- 172.21.15.89:0/3862296975 >> v2:172.21.15.89:3300/0 conn(0x7f3ccc09e410 0x7f3ccc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 -- 172.21.15.89:0/3862296975 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc09a500 con 0x7f3ccc09d6f0 2023-09-30T22:20:10.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cca59c700 1 --2- 172.21.15.89:0/3862296975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09d6f0 0x7f3ccc09dac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009920 tx=0x7f3cbc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f9f3552bc78e09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:10.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.285+0000 7f3cc959a700 1 -- 172.21.15.89:0/3862296975 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011830 con 0x7f3ccc09d6f0 2023-09-30T22:20:10.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cc959a700 1 -- 172.21.15.89:0/3862296975 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc011990 con 0x7f3ccc09d6f0 2023-09-30T22:20:10.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cc959a700 1 -- 172.21.15.89:0/3862296975 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011d20 con 0x7f3ccc09d6f0 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/3862296975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09d6f0 msgr2=0x7f3ccc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cd0c4c700 1 --2- 172.21.15.89:0/3862296975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09d6f0 0x7f3ccc09dac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009920 tx=0x7f3cbc009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/3862296975 shutdown_connections 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cd0c4c700 1 --2- 172.21.15.89:0/3862296975 >> v2:172.21.15.89:3300/0 conn(0x7f3ccc09e410 0x7f3ccc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cd0c4c700 1 --2- 172.21.15.89:0/3862296975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09d6f0 0x7f3ccc09dac0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cd0c4c700 1 --2- 172.21.15.89:0/3862296975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e000 0x7f3ccc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/3862296975 >> 172.21.15.89:0/3862296975 conn(0x7f3ccc066f50 msgr2=0x7f3ccc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.286+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/3862296975 shutdown_connections 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.287+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/3862296975 wait complete. 2023-09-30T22:20:10.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cd0c4c700 1 Processor -- start 2023-09-30T22:20:10.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f8c8be82700 1 Processor -- start 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cd0c4c700 1 -- start start 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cd0c4c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ccc09d6f0 0x7f3ccc1a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cd0c4c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09e000 0x7f3ccc1a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cd0c4c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e410 0x7f3ccc1a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cd0c4c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a9a60 con 0x7f3ccc09d6f0 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cd0c4c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a8130 con 0x7f3ccc09e000 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cd0c4c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a91f0 con 0x7f3ccc09e410 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cc9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09e000 0x7f3ccc1a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cc9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09e000 0x7f3ccc1a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37876/0 (socket says 172.21.15.89:37876) 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3cc9d9b700 1 -- 172.21.15.89:0/4071883804 learned_addr learned my addr 172.21.15.89:0/4071883804 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:10.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f3ccad9d700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e410 0x7f3ccc1a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f8c8be82700 1 -- start start 2023-09-30T22:20:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cca59c700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ccc09d6f0 0x7f3ccc1a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cc9d9b700 1 -- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e410 msgr2=0x7f3ccc1a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cc9d9b700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e410 0x7f3ccc1a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cc9d9b700 1 -- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ccc09d6f0 msgr2=0x7f3ccc1a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cc9d9b700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ccc09d6f0 0x7f3ccc1a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cc9d9b700 1 -- 172.21.15.89:0/4071883804 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc009550 con 0x7f3ccc09e000 2023-09-30T22:20:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.288+0000 7f8c8be82700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8be82700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c8409e000 0x7f8c84130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8be82700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e410 0x7f8c84130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8be82700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8409f520 con 0x7f8c8409e000 2023-09-30T22:20:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c89c1e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cc9d9b700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09e000 0x7f3ccc1a6560 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000e980 tx=0x7f3cc000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8be82700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8409f130 con 0x7f8c8409e410 2023-09-30T22:20:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8be82700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8409e930 con 0x7f8c8409d6f0 2023-09-30T22:20:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c89c1e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41012/0 (socket says 172.21.15.89:41012) 2023-09-30T22:20:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c89c1e700 1 -- 172.21.15.89:0/4168297656 learned_addr learned my addr 172.21.15.89:0/4168297656 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:10.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8a41f700 1 --2- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e410 0x7f8c84130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cbaffd700 1 -- 172.21.15.89:0/4071883804 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0009710 con 0x7f3ccc09e000 2023-09-30T22:20:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cbaffd700 1 -- 172.21.15.89:0/4071883804 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc000eed0 con 0x7f3ccc09e000 2023-09-30T22:20:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc09c710 con 0x7f3ccc09e000 2023-09-30T22:20:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cbaffd700 1 -- 172.21.15.89:0/4071883804 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000ca50 con 0x7f3ccc09e000 2023-09-30T22:20:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc0a3190 con 0x7f3ccc09e000 2023-09-30T22:20:10.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8a41f700 1 -- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 msgr2=0x7f8c8409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8a41f700 1 --2- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8a41f700 1 -- 172.21.15.89:0/4168297656 >> v2:172.21.15.89:3300/0 conn(0x7f8c8409e000 msgr2=0x7f8c84130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8a41f700 1 --2- 172.21.15.89:0/4168297656 >> v2:172.21.15.89:3300/0 conn(0x7f8c8409e000 0x7f8c84130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c8a41f700 1 -- 172.21.15.89:0/4168297656 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8409b2d0 con 0x7f8c8409e410 2023-09-30T22:20:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.289+0000 7f8c89c1e700 1 --2- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:10.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.290+0000 7f3cca59c700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ccc09d6f0 0x7f3ccc1a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.290+0000 7f8c8a41f700 1 --2- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e410 0x7f8c84130ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009d90 tx=0x7f8c7c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b07168050904df6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.290+0000 7f8c88c1c700 1 -- 172.21.15.89:0/4168297656 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00e070 con 0x7f8c8409e410 2023-09-30T22:20:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.290+0000 7f8c88c1c700 1 -- 172.21.15.89:0/4168297656 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c00bd20 con 0x7f8c8409e410 2023-09-30T22:20:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.290+0000 7f8c88c1c700 1 -- 172.21.15.89:0/4168297656 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c012920 con 0x7f8c8409e410 2023-09-30T22:20:10.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.290+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc1a9600 con 0x7f3ccc09e000 2023-09-30T22:20:10.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f3cbaffd700 1 -- 172.21.15.89:0/4071883804 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3cc000cbf0 con 0x7f3ccc09e000 2023-09-30T22:20:10.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f3cbaffd700 1 -- 172.21.15.89:0/4071883804 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3cc0047350 con 0x7f3ccc09e000 2023-09-30T22:20:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f8c8be82700 1 -- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e410 msgr2=0x7f8c84130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f8c8be82700 1 --2- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e410 0x7f8c84130ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009d90 tx=0x7f8c7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f8c8be82700 1 -- 172.21.15.89:0/4168297656 shutdown_connections 2023-09-30T22:20:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f8c8be82700 1 --2- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e410 0x7f8c84130ba0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f8c8be82700 1 --2- 172.21.15.89:0/4168297656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f8c8be82700 1 --2- 172.21.15.89:0/4168297656 >> v2:172.21.15.89:3300/0 conn(0x7f8c8409e000 0x7f8c84130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.291+0000 7f8c8be82700 1 -- 172.21.15.89:0/4168297656 >> 172.21.15.89:0/4168297656 conn(0x7f8c84066f50 msgr2=0x7f8c8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.294+0000 7f8c8be82700 1 -- 172.21.15.89:0/4168297656 shutdown_connections 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.294+0000 7f8c8be82700 1 -- 172.21.15.89:0/4168297656 wait complete. 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8be82700 1 Processor -- start 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8be82700 1 -- start start 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8be82700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8411e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8be82700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e000 0x7f8c8411e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8be82700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c8409e410 0x7f8c8411ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8be82700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c84121c50 con 0x7f8c8409e410 2023-09-30T22:20:10.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8be82700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841202f0 con 0x7f8c8409e000 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8be82700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c84072590 con 0x7f8c8409d6f0 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c89c1e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8411e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c89c1e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8411e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41018/0 (socket says 172.21.15.89:41018) 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c89c1e700 1 -- 172.21.15.89:0/253199461 learned_addr learned my addr 172.21.15.89:0/253199461 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c8941d700 1 --2- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e000 0x7f8c8411e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c89c1e700 1 -- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e000 msgr2=0x7f8c8411e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c89c1e700 1 --2- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e000 0x7f8c8411e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c89c1e700 1 -- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c8409e410 msgr2=0x7f8c8411ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.295+0000 7f8c89c1e700 1 --2- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c8409e410 0x7f8c8411ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.296+0000 7f8c89c1e700 1 -- 172.21.15.89:0/253199461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80009550 con 0x7f8c8409d6f0 2023-09-30T22:20:10.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.296+0000 7f8c89c1e700 1 --2- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8411e130 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000ebe0 tx=0x7f8c8000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.296+0000 7f8c7a7fc700 1 -- 172.21.15.89:0/253199461 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000c850 con 0x7f8c8409d6f0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.296+0000 7f8c7a7fc700 1 -- 172.21.15.89:0/253199461 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8000c9b0 con 0x7f8c8409d6f0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.296+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c009a70 con 0x7f8c8409d6f0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.296+0000 7f8c7a7fc700 1 -- 172.21.15.89:0/253199461 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000cd20 con 0x7f8c8409d6f0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.296+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c841216b0 con 0x7f8c8409d6f0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.297+0000 7f8c7a7fc700 1 -- 172.21.15.89:0/253199461 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c80013470 con 0x7f8c8409d6f0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.297+0000 7f8c7a7fc700 1 -- 172.21.15.89:0/253199461 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8c8003fbe0 con 0x7f8c8409d6f0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.297+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c841214a0 con 0x7f8c8409d6f0 2023-09-30T22:20:10.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.299+0000 7f3cbaffd700 1 -- 172.21.15.89:0/4071883804 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ccc1a9600 con 0x7f3ccc09e000 2023-09-30T22:20:10.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.302+0000 7f8c7a7fc700 1 -- 172.21.15.89:0/253199461 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c8001b3b0 con 0x7f8c8409d6f0 2023-09-30T22:20:10.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.324+0000 7fbd7c1d3700 1 Processor -- start 2023-09-30T22:20:10.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.324+0000 7fbd7c1d3700 1 -- start start 2023-09-30T22:20:10.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.324+0000 7f66b8471700 1 Processor -- start 2023-09-30T22:20:10.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b8471700 1 -- start start 2023-09-30T22:20:10.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd7c1d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd740a0a30 0x7fbd740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b8471700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 0x7f66b006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b8471700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b8471700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b012cd90 con 0x7f66b0130260 2023-09-30T22:20:10.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b8471700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b012d330 con 0x7f66b012f8b0 2023-09-30T22:20:10.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b8471700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66b01306d0 con 0x7f66b012cf60 2023-09-30T22:20:10.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd7c1d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd74138d80 0x7fbd7413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd7c1d3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd740a2a70 con 0x7fbd7409d630 2023-09-30T22:20:10.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd7c1d3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740a01d0 con 0x7fbd74138d80 2023-09-30T22:20:10.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd7c1d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7409e2b0 con 0x7fbd740a0a30 2023-09-30T22:20:10.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd79f6f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd740a0a30 0x7fbd740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd79f6f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd740a0a30 0x7fbd740a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41024/0 (socket says 172.21.15.89:41024) 2023-09-30T22:20:10.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd79f6f700 1 -- 172.21.15.89:0/1222978080 learned_addr learned my addr 172.21.15.89:0/1222978080 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:10.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7fbd7a770700 1 --2- 172.21.15.89:0/1222978080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd74138d80 0x7fbd7413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b5a0c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b620d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 0x7f66b006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.325+0000 7f66b5a0c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37912/0 (socket says 172.21.15.89:37912) 2023-09-30T22:20:10.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b5a0c700 1 -- 172.21.15.89:0/3853083406 learned_addr learned my addr 172.21.15.89:0/3853083406 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7fbd7a770700 1 -- 172.21.15.89:0/1222978080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd740a0a30 msgr2=0x7fbd740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7fbd7a770700 1 --2- 172.21.15.89:0/1222978080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd740a0a30 0x7fbd740a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7fbd7a770700 1 -- 172.21.15.89:0/1222978080 >> v1:172.21.15.89:6789/0 conn(0x7fbd7409d630 legacy=0x7fbd740a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7fbd7a770700 1 -- 172.21.15.89:0/1222978080 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7409dc50 con 0x7fbd74138d80 2023-09-30T22:20:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b620d700 1 -- 172.21.15.89:0/3853083406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 msgr2=0x7f66b012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b620d700 1 --2- 172.21.15.89:0/3853083406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b620d700 1 -- 172.21.15.89:0/3853083406 >> v1:172.21.15.89:6789/0 conn(0x7f66b0130260 legacy=0x7f66b0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b620d700 1 -- 172.21.15.89:0/3853083406 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b0071780 con 0x7f66b012cf60 2023-09-30T22:20:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7fbd7a770700 1 --2- 172.21.15.89:0/1222978080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd74138d80 0x7fbd7413b160 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009dc0 tx=0x7fbd6c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54387fedc1cd21c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b620d700 1 --2- 172.21.15.89:0/3853083406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 0x7f66b006f5d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009870 tx=0x7f66ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656446b61f2d42db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b520b700 1 -- 172.21.15.89:0/3853083406 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00f940 con 0x7f66b012cf60 2023-09-30T22:20:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b520b700 1 -- 172.21.15.89:0/3853083406 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac00faa0 con 0x7f66b012cf60 2023-09-30T22:20:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7f66b520b700 1 -- 172.21.15.89:0/3853083406 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00fe30 con 0x7f66b012cf60 2023-09-30T22:20:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7fbd78f6d700 1 -- 172.21.15.89:0/1222978080 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00e070 con 0x7fbd74138d80 2023-09-30T22:20:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7fbd78f6d700 1 -- 172.21.15.89:0/1222978080 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c00bbd0 con 0x7fbd74138d80 2023-09-30T22:20:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.326+0000 7fbd78f6d700 1 -- 172.21.15.89:0/1222978080 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c012430 con 0x7fbd74138d80 2023-09-30T22:20:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7f66b8471700 1 -- 172.21.15.89:0/3853083406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 msgr2=0x7f66b006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7f66b8471700 1 --2- 172.21.15.89:0/3853083406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 0x7f66b006f5d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009870 tx=0x7f66ac009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1222978080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd74138d80 msgr2=0x7fbd7413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7fbd7c1d3700 1 --2- 172.21.15.89:0/1222978080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd74138d80 0x7fbd7413b160 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009dc0 tx=0x7fbd6c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7f66b8471700 1 -- 172.21.15.89:0/3853083406 shutdown_connections 2023-09-30T22:20:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1222978080 shutdown_connections 2023-09-30T22:20:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7fbd7c1d3700 1 --2- 172.21.15.89:0/1222978080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd74138d80 0x7fbd7413b160 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7fbd7c1d3700 1 --2- 172.21.15.89:0/1222978080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd740a0a30 0x7fbd740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1222978080 >> 172.21.15.89:0/1222978080 conn(0x7fbd740a3050 msgr2=0x7fbd740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7f66b8471700 1 --2- 172.21.15.89:0/3853083406 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 0x7f66b006f5d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7f66b8471700 1 --2- 172.21.15.89:0/3853083406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7f66b8471700 1 -- 172.21.15.89:0/3853083406 >> 172.21.15.89:0/3853083406 conn(0x7f66b00a2aa0 msgr2=0x7f66b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7f66b8471700 1 -- 172.21.15.89:0/3853083406 shutdown_connections 2023-09-30T22:20:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7f66b8471700 1 -- 172.21.15.89:0/3853083406 wait complete. 2023-09-30T22:20:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1222978080 shutdown_connections 2023-09-30T22:20:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.327+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1222978080 wait complete. 2023-09-30T22:20:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.328+0000 7fbd7c1d3700 1 Processor -- start 2023-09-30T22:20:10.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.328+0000 7f66b8471700 1 Processor -- start 2023-09-30T22:20:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.328+0000 7fbd7c1d3700 1 -- start start 2023-09-30T22:20:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.328+0000 7f66b8471700 1 -- start start 2023-09-30T22:20:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b8471700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 0x7f66b009afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b8471700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b009b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b8471700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0130260 0x7f66b009ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b8471700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b009cd30 con 0x7f66b0130260 2023-09-30T22:20:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b8471700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b009cb40 con 0x7f66b012f8b0 2023-09-30T22:20:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b8471700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66b012d7b0 con 0x7f66b012cf60 2023-09-30T22:20:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b5a0c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b009b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b5a0c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b009b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37936/0 (socket says 172.21.15.89:37936) 2023-09-30T22:20:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b5a0c700 1 -- 172.21.15.89:0/4131233228 learned_addr learned my addr 172.21.15.89:0/4131233228 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b6a0e700 1 --2- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0130260 0x7f66b009ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.328+0000 7fbd7c1d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd7409d630 0x7fbd741c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7c1d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd740a0a30 0x7fbd741c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7c1d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd74138d80 0x7fbd7411ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7c1d3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741bde60 con 0x7fbd74138d80 2023-09-30T22:20:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7c1d3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741227d0 con 0x7fbd740a0a30 2023-09-30T22:20:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7c1d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741207c0 con 0x7fbd7409d630 2023-09-30T22:20:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b5a0c700 1 -- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 msgr2=0x7f66b009afb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b5a0c700 1 --2- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 0x7f66b009afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b5a0c700 1 -- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0130260 msgr2=0x7f66b009ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b5a0c700 1 --2- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0130260 0x7f66b009ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7f66b5a0c700 1 -- 172.21.15.89:0/4131233228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0094a0 con 0x7f66b012f8b0 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7a770700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd74138d80 0x7fbd7411ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7a770700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd74138d80 0x7fbd7411ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49788/0 (socket says 172.21.15.89:49788) 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7a770700 1 -- 172.21.15.89:0/1282882276 learned_addr learned my addr 172.21.15.89:0/1282882276 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd79f6f700 1 --2- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd7409d630 0x7fbd741c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7a770700 1 -- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd7409d630 msgr2=0x7fbd741c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7a770700 1 --2- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd7409d630 0x7fbd741c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7a770700 1 -- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd740a0a30 msgr2=0x7fbd741c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7a770700 1 --2- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd740a0a30 0x7fbd741c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.329+0000 7fbd7a770700 1 -- 172.21.15.89:0/1282882276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c009a70 con 0x7fbd74138d80 2023-09-30T22:20:10.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7f66b5a0c700 1 --2- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b009b4f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e7e0 tx=0x7f66a000eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:10.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7f66a6ffd700 1 -- 172.21.15.89:0/4131233228 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0004230 con 0x7f66b012f8b0 2023-09-30T22:20:10.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7fbd7a770700 1 --2- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd74138d80 0x7fbd7411ec80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b370 tx=0x7fbd6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:10.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7f66a6ffd700 1 -- 172.21.15.89:0/4131233228 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0004390 con 0x7f66b012f8b0 2023-09-30T22:20:10.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7f66a6ffd700 1 -- 172.21.15.89:0/4131233228 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a00036a0 con 0x7f66b012f8b0 2023-09-30T22:20:10.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7fbd6a7fc700 1 -- 172.21.15.89:0/1282882276 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c012430 con 0x7fbd74138d80 2023-09-30T22:20:10.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7fbd6a7fc700 1 -- 172.21.15.89:0/1282882276 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c020070 con 0x7fbd74138d80 2023-09-30T22:20:10.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7fbd6a7fc700 1 -- 172.21.15.89:0/1282882276 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c004040 con 0x7fbd74138d80 2023-09-30T22:20:10.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.331+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd740a0410 con 0x7fbd74138d80 2023-09-30T22:20:10.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.331+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd74122130 con 0x7fbd74138d80 2023-09-30T22:20:10.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b0129fa0 con 0x7f66b012f8b0 2023-09-30T22:20:10.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.330+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b009eac0 con 0x7f66b012f8b0 2023-09-30T22:20:10.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.331+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b009e690 con 0x7f66b012f8b0 2023-09-30T22:20:10.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.332+0000 7f66a6ffd700 1 -- 172.21.15.89:0/4131233228 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66a0003800 con 0x7f66b012f8b0 2023-09-30T22:20:10.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.332+0000 7fbd6a7fc700 1 -- 172.21.15.89:0/1282882276 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd6c0041a0 con 0x7fbd74138d80 2023-09-30T22:20:10.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.332+0000 7fbd6a7fc700 1 -- 172.21.15.89:0/1282882276 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbd6c022070 con 0x7fbd74138d80 2023-09-30T22:20:10.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.332+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd740747f0 con 0x7fbd74138d80 2023-09-30T22:20:10.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.332+0000 7f66a6ffd700 1 -- 172.21.15.89:0/4131233228 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f66a003f5b0 con 0x7f66b012f8b0 2023-09-30T22:20:10.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.337+0000 7f66a6ffd700 1 -- 172.21.15.89:0/4131233228 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f66a0024940 con 0x7f66b012f8b0 2023-09-30T22:20:10.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.337+0000 7fbd6a7fc700 1 -- 172.21.15.89:0/1282882276 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd6c004450 con 0x7fbd74138d80 2023-09-30T22:20:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.527+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbd7406d5c0 con 0x7fbd74138d80 2023-09-30T22:20:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.527+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f66b0074ea0 con 0x7f66b012f8b0 2023-09-30T22:20:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.528+0000 7fbd6a7fc700 1 -- 172.21.15.89:0/1282882276 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd6c02c030 con 0x7fbd74138d80 2023-09-30T22:20:10.528 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.528+0000 7f66a6ffd700 1 -- 172.21.15.89:0/4131233228 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66a0024350 con 0x7f66b012f8b0 2023-09-30T22:20:10.529 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd74138d80 msgr2=0x7fbd7411ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 --2- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd74138d80 0x7fbd7411ec80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b370 tx=0x7fbd6c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 shutdown_connections 2023-09-30T22:20:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 --2- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd74138d80 0x7fbd7411ec80 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 --2- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd7409d630 0x7fbd741c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 --2- 172.21.15.89:0/1282882276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd740a0a30 0x7fbd741c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 >> 172.21.15.89:0/1282882276 conn(0x7fbd740a3050 msgr2=0x7fbd7409c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 msgr2=0x7f66b009b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 shutdown_connections 2023-09-30T22:20:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7fbd7c1d3700 1 -- 172.21.15.89:0/1282882276 wait complete. 2023-09-30T22:20:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7f66b8471700 1 --2- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b009b4f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e7e0 tx=0x7f66a000eaf0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 shutdown_connections 2023-09-30T22:20:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7f66b8471700 1 --2- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0130260 0x7f66b009ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7f66b8471700 1 --2- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66b012cf60 0x7f66b009afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7f66b8471700 1 --2- 172.21.15.89:0/4131233228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66b012f8b0 0x7f66b009b4f0 secure :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e7e0 tx=0x7f66a000eaf0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.530+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 >> 172.21.15.89:0/4131233228 conn(0x7f66b00a2aa0 msgr2=0x7f66b01b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.531+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 shutdown_connections 2023-09-30T22:20:10.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.531+0000 7f66b8471700 1 -- 172.21.15.89:0/4131233228 wait complete. 2023-09-30T22:20:10.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:10.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:10.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.560+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3ccc1a9600 con 0x7f3ccc09e000 2023-09-30T22:20:10.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.561+0000 7f3cbaffd700 1 -- 172.21.15.89:0/4071883804 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ccc1a9600 con 0x7f3ccc09e000 2023-09-30T22:20:10.560 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09e000 msgr2=0x7f3ccc1a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09e000 0x7f3ccc1a6560 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000e980 tx=0x7f3cc000ec90 comp rx=0 tx=0).stop 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 shutdown_connections 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ccc09e410 0x7f3ccc1a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ccc09d6f0 0x7f3ccc1a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 --2- 172.21.15.89:0/4071883804 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ccc09e000 0x7f3ccc1a6560 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 >> 172.21.15.89:0/4071883804 conn(0x7f3ccc066f50 msgr2=0x7f3ccc1bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 shutdown_connections 2023-09-30T22:20:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.563+0000 7f3cd0c4c700 1 -- 172.21.15.89:0/4071883804 wait complete. 2023-09-30T22:20:10.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.565+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8c84074ea0 con 0x7f8c8409d6f0 2023-09-30T22:20:10.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.565+0000 7f8c7a7fc700 1 -- 172.21.15.89:0/253199461 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c80017030 con 0x7f8c8409d6f0 2023-09-30T22:20:10.565 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:10.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 msgr2=0x7f8c8411e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:10.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 --2- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8411e130 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000ebe0 tx=0x7f8c8000eef0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 shutdown_connections 2023-09-30T22:20:10.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 --2- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c8409e410 0x7f8c8411ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 --2- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c8409d6f0 0x7f8c8411e130 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 --2- 172.21.15.89:0/253199461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c8409e000 0x7f8c8411e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:10.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 >> 172.21.15.89:0/253199461 conn(0x7f8c84066f50 msgr2=0x7f8c84070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:10.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 shutdown_connections 2023-09-30T22:20:10.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:10.567+0000 7f8c8be82700 1 -- 172.21.15.89:0/253199461 wait complete. 2023-09-30T22:20:10.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:10.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:11.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:11.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:11.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:11.581 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:11.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.684+0000 7f47f48e7700 1 Processor -- start 2023-09-30T22:20:11.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.684+0000 7f47f48e7700 1 -- start start 2023-09-30T22:20:11.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47f48e7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47f009e1d0 0x7f47f0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47f48e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47f48e7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f009b120 con 0x7f47f009d6f0 2023-09-30T22:20:11.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47f48e7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f006b460 con 0x7f47f009e1d0 2023-09-30T22:20:11.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47f48e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f009a7e0 con 0x7f47f0139170 2023-09-30T22:20:11.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47edd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47f009e1d0 0x7f47f0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47edd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47f009e1d0 0x7f47f0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49802/0 (socket says 172.21.15.89:49802) 2023-09-30T22:20:11.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47edd9b700 1 -- 172.21.15.89:0/4080902418 learned_addr learned my addr 172.21.15.89:0/4080902418 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:11.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.685+0000 7f47eed9d700 1 --2- 172.21.15.89:0/4080902418 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.686+0000 7f47edd9b700 1 -- 172.21.15.89:0/4080902418 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 msgr2=0x7f47f013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:11.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.686+0000 7f47edd9b700 1 --2- 172.21.15.89:0/4080902418 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.686+0000 7f47edd9b700 1 -- 172.21.15.89:0/4080902418 >> v1:172.21.15.89:6789/0 conn(0x7f47f009d6f0 legacy=0x7f47f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.686+0000 7f47edd9b700 1 -- 172.21.15.89:0/4080902418 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f009a500 con 0x7f47f009e1d0 2023-09-30T22:20:11.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.686+0000 7f47edd9b700 1 --2- 172.21.15.89:0/4080902418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f0138c30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009920 tx=0x7f47e4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7a14a429dae55c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.686+0000 7f47ed59a700 1 -- 172.21.15.89:0/4080902418 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4011830 con 0x7f47f009e1d0 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.686+0000 7f47ed59a700 1 -- 172.21.15.89:0/4080902418 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e4011990 con 0x7f47f009e1d0 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.686+0000 7f47ed59a700 1 -- 172.21.15.89:0/4080902418 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4011d20 con 0x7f47f009e1d0 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.687+0000 7f47f48e7700 1 -- 172.21.15.89:0/4080902418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 msgr2=0x7f47f0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.687+0000 7f47f48e7700 1 --2- 172.21.15.89:0/4080902418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f0138c30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009920 tx=0x7f47e4009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.687+0000 7f47f48e7700 1 -- 172.21.15.89:0/4080902418 shutdown_connections 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.687+0000 7f47f48e7700 1 --2- 172.21.15.89:0/4080902418 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.687+0000 7f47f48e7700 1 --2- 172.21.15.89:0/4080902418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f0138c30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.687+0000 7f47f48e7700 1 -- 172.21.15.89:0/4080902418 >> 172.21.15.89:0/4080902418 conn(0x7f47f0066f50 msgr2=0x7f47f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.687+0000 7f47f48e7700 1 -- 172.21.15.89:0/4080902418 shutdown_connections 2023-09-30T22:20:11.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.687+0000 7f47f48e7700 1 -- 172.21.15.89:0/4080902418 wait complete. 2023-09-30T22:20:11.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.688+0000 7f47f48e7700 1 Processor -- start 2023-09-30T22:20:11.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.688+0000 7f47f48e7700 1 -- start start 2023-09-30T22:20:11.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47f48e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f009d6f0 0x7f47f011f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47f48e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f011fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47f48e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f01c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47f48e7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0121f20 con 0x7f47f009e1d0 2023-09-30T22:20:11.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47f48e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00a1b40 con 0x7f47f0139170 2023-09-30T22:20:11.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47f48e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47f009c710 con 0x7f47f009d6f0 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47eed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f01c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47edd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f011fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47eed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f01c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37964/0 (socket says 172.21.15.89:37964) 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47eed9d700 1 -- 172.21.15.89:0/2956996671 learned_addr learned my addr 172.21.15.89:0/2956996671 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47edd9b700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f011fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49810/0 (socket says 172.21.15.89:49810) 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47eed9d700 1 -- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f009d6f0 msgr2=0x7f47f011f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47ee59c700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f009d6f0 0x7f47f011f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47eed9d700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f009d6f0 0x7f47f011f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47eed9d700 1 -- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 msgr2=0x7f47f011fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47eed9d700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f011fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47eed9d700 1 -- 172.21.15.89:0/2956996671 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e4009550 con 0x7f47f0139170 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47edd9b700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f011fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.689+0000 7f47ee59c700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f009d6f0 0x7f47f011f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.690+0000 7f47eed9d700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f01c8670 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f47e800efc0 tx=0x7f47e800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.690+0000 7f47deffd700 1 -- 172.21.15.89:0/2956996671 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800cd40 con 0x7f47f0139170 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.690+0000 7f47deffd700 1 -- 172.21.15.89:0/2956996671 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800f040 con 0x7f47f0139170 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.690+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f0120890 con 0x7f47f0139170 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.690+0000 7f47deffd700 1 -- 172.21.15.89:0/2956996671 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e80146c0 con 0x7f47f0139170 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.690+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f011f090 con 0x7f47f0139170 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.691+0000 7f47deffd700 1 -- 172.21.15.89:0/2956996671 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f47e8014820 con 0x7f47f0139170 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.691+0000 7f47deffd700 1 -- 172.21.15.89:0/2956996671 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f47e8040420 con 0x7f47f0139170 2023-09-30T22:20:11.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.691+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f009c920 con 0x7f47f0139170 2023-09-30T22:20:11.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.695+0000 7fea279a9700 1 Processor -- start 2023-09-30T22:20:11.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.695+0000 7fea279a9700 1 -- start start 2023-09-30T22:20:11.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.695+0000 7fea279a9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea2012cf60 0x7fea2006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.695+0000 7fea279a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea2012f8b0 0x7fea2012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.695+0000 7fea279a9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2012cd90 con 0x7fea20130260 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.695+0000 7fea279a9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2012d330 con 0x7fea2012cf60 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.695+0000 7fea279a9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea201306d0 con 0x7fea2012f8b0 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea25745700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea2012cf60 0x7fea2006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea25745700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea2012cf60 0x7fea2006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49812/0 (socket says 172.21.15.89:49812) 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea25745700 1 -- 172.21.15.89:0/1563165678 learned_addr learned my addr 172.21.15.89:0/1563165678 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea25745700 1 -- 172.21.15.89:0/1563165678 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea2012f8b0 msgr2=0x7fea2012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea25745700 1 --2- 172.21.15.89:0/1563165678 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea2012f8b0 0x7fea2012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea25745700 1 -- 172.21.15.89:0/1563165678 >> v1:172.21.15.89:6789/0 conn(0x7fea20130260 legacy=0x7fea20136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea25745700 1 -- 172.21.15.89:0/1563165678 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea20071780 con 0x7fea2012cf60 2023-09-30T22:20:11.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea25745700 1 --2- 172.21.15.89:0/1563165678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2006f5d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0098f0 tx=0x7fea1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a356904b82df0d0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:11.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea17fff700 1 -- 172.21.15.89:0/1563165678 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00b8a0 con 0x7fea2012cf60 2023-09-30T22:20:11.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea17fff700 1 -- 172.21.15.89:0/1563165678 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c00ba00 con 0x7fea2012cf60 2023-09-30T22:20:11.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.696+0000 7fea17fff700 1 -- 172.21.15.89:0/1563165678 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00bd90 con 0x7fea2012cf60 2023-09-30T22:20:11.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.697+0000 7fea279a9700 1 -- 172.21.15.89:0/1563165678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 msgr2=0x7fea2006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.697+0000 7fea279a9700 1 --2- 172.21.15.89:0/1563165678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2006f5d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0098f0 tx=0x7fea1c009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:11.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.697+0000 7fea279a9700 1 -- 172.21.15.89:0/1563165678 shutdown_connections 2023-09-30T22:20:11.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.697+0000 7fea279a9700 1 --2- 172.21.15.89:0/1563165678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2006f5d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.697+0000 7fea279a9700 1 --2- 172.21.15.89:0/1563165678 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea2012f8b0 0x7fea2012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.697+0000 7fea279a9700 1 -- 172.21.15.89:0/1563165678 >> 172.21.15.89:0/1563165678 conn(0x7fea200a2aa0 msgr2=0x7fea200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.697+0000 7fea279a9700 1 -- 172.21.15.89:0/1563165678 shutdown_connections 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.697+0000 7fea279a9700 1 -- 172.21.15.89:0/1563165678 wait complete. 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea279a9700 1 Processor -- start 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea279a9700 1 -- start start 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea279a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2009b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea279a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea2012f8b0 0x7fea2009bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea279a9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea20130260 0x7fea201c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea279a9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea20129fa0 con 0x7fea2012cf60 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea279a9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2009e430 con 0x7fea20130260 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea279a9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea2009b400 con 0x7fea2012f8b0 2023-09-30T22:20:11.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea25745700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2009b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea25745700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2009b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49816/0 (socket says 172.21.15.89:49816) 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.698+0000 7fea25745700 1 -- 172.21.15.89:0/2701957310 learned_addr learned my addr 172.21.15.89:0/2701957310 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea25f46700 1 --2- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea20130260 0x7fea201c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea25745700 1 -- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea2012f8b0 msgr2=0x7fea2009bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea25745700 1 --2- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea2012f8b0 0x7fea2009bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea25745700 1 -- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea20130260 msgr2=0x7fea201c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea25745700 1 --2- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea20130260 0x7fea201c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea25745700 1 -- 172.21.15.89:0/2701957310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c009550 con 0x7fea2012cf60 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea25745700 1 --2- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2009b640 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fea2006fba0 tx=0x7fea1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea15ffb700 1 -- 172.21.15.89:0/2701957310 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c003ab0 con 0x7fea2012cf60 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea15ffb700 1 -- 172.21.15.89:0/2701957310 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c003c30 con 0x7fea2012cf60 2023-09-30T22:20:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2009d9b0 con 0x7fea2012cf60 2023-09-30T22:20:11.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7fea15ffb700 1 -- 172.21.15.89:0/2701957310 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0045a0 con 0x7fea2012cf60 2023-09-30T22:20:11.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.699+0000 7f47deffd700 1 -- 172.21.15.89:0/2956996671 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f47e8018030 con 0x7f47f0139170 2023-09-30T22:20:11.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.700+0000 7fea15ffb700 1 -- 172.21.15.89:0/2701957310 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea1c018400 con 0x7fea2012cf60 2023-09-30T22:20:11.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.700+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2009cac0 con 0x7fea2012cf60 2023-09-30T22:20:11.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.701+0000 7fea15ffb700 1 -- 172.21.15.89:0/2701957310 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea1c029520 con 0x7fea2012cf60 2023-09-30T22:20:11.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.702+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea20074ea0 con 0x7fea2012cf60 2023-09-30T22:20:11.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.706+0000 7fea15ffb700 1 -- 172.21.15.89:0/2701957310 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea1c026020 con 0x7fea2012cf60 2023-09-30T22:20:11.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.738+0000 7f8b7726d700 1 Processor -- start 2023-09-30T22:20:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b7726d700 1 -- start start 2023-09-30T22:20:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b7726d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 0x7f8b7009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b7726d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b7726d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b7009f650 0x7f8b70131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b7726d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b70070940 con 0x7f8b7009f650 2023-09-30T22:20:11.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b7726d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7006fd60 con 0x7f8b7009f240 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b7726d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7006fa80 con 0x7f8b7009e930 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b74808700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b74808700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37984/0 (socket says 172.21.15.89:37984) 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b74808700 1 -- 172.21.15.89:0/4080763341 learned_addr learned my addr 172.21.15.89:0/4080763341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.739+0000 7f8b7580a700 1 --2- 172.21.15.89:0/4080763341 >> v2:172.21.15.89:3300/0 conn(0x7f8b7009f650 0x7f8b70131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b75009700 1 --2- 172.21.15.89:0/4080763341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 0x7f8b7009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b74808700 1 -- 172.21.15.89:0/4080763341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 msgr2=0x7f8b7009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b74808700 1 --2- 172.21.15.89:0/4080763341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 0x7f8b7009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b74808700 1 -- 172.21.15.89:0/4080763341 >> v2:172.21.15.89:3300/0 conn(0x7f8b7009f650 msgr2=0x7f8b70131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b74808700 1 --2- 172.21.15.89:0/4080763341 >> v2:172.21.15.89:3300/0 conn(0x7f8b7009f650 0x7f8b70131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b74808700 1 -- 172.21.15.89:0/4080763341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7009fb60 con 0x7f8b7009f240 2023-09-30T22:20:11.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b74808700 1 --2- 172.21.15.89:0/4080763341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701317f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8b600098f0 tx=0x7f8b60009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ad57a385f5591d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:11.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b67fff700 1 -- 172.21.15.89:0/4080763341 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60011830 con 0x7f8b7009f240 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b67fff700 1 -- 172.21.15.89:0/4080763341 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b60011990 con 0x7f8b7009f240 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.740+0000 7f8b67fff700 1 -- 172.21.15.89:0/4080763341 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60011d20 con 0x7f8b7009f240 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 -- 172.21.15.89:0/4080763341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 msgr2=0x7f8b701317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 --2- 172.21.15.89:0/4080763341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701317f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8b600098f0 tx=0x7f8b60009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 -- 172.21.15.89:0/4080763341 shutdown_connections 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 --2- 172.21.15.89:0/4080763341 >> v2:172.21.15.89:3300/0 conn(0x7f8b7009f650 0x7f8b70131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 --2- 172.21.15.89:0/4080763341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 0x7f8b7009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 --2- 172.21.15.89:0/4080763341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701317f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 -- 172.21.15.89:0/4080763341 >> 172.21.15.89:0/4080763341 conn(0x7f8b700a4210 msgr2=0x7f8b700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 -- 172.21.15.89:0/4080763341 shutdown_connections 2023-09-30T22:20:11.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.741+0000 7f8b7726d700 1 -- 172.21.15.89:0/4080763341 wait complete. 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.742+0000 7f8b7726d700 1 Processor -- start 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.742+0000 7f8b7726d700 1 -- start start 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.742+0000 7f8b7726d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 0x7f8b701a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.742+0000 7f8b7726d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.742+0000 7f8b7726d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b7009f650 0x7f8b701a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.742+0000 7f8b7726d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b701a5740 con 0x7f8b7009f650 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.742+0000 7f8b7726d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b701a6280 con 0x7f8b7009f240 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.742+0000 7f8b7726d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700a3580 con 0x7f8b7009e930 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b7009f650 0x7f8b701a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b7009f650 0x7f8b701a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49838/0 (socket says 172.21.15.89:49838) 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 -- 172.21.15.89:0/4269841937 learned_addr learned my addr 172.21.15.89:0/4269841937 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:11.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b74808700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 -- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 msgr2=0x7f8b701a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 0x7f8b701a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 -- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 msgr2=0x7f8b701a3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701a3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 -- 172.21.15.89:0/4269841937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60009550 con 0x7f8b7009f650 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b74808700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7580a700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b7009f650 0x7f8b701a3890 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009e30 tx=0x7f8b6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b65ffb700 1 -- 172.21.15.89:0/4269841937 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68009ca0 con 0x7f8b7009f650 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b65ffb700 1 -- 172.21.15.89:0/4269841937 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800f040 con 0x7f8b7009f650 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700a0190 con 0x7f8b7009f650 2023-09-30T22:20:11.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b65ffb700 1 -- 172.21.15.89:0/4269841937 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68014480 con 0x7f8b7009f650 2023-09-30T22:20:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.743+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b70071d20 con 0x7f8b7009f650 2023-09-30T22:20:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.745+0000 7f8b65ffb700 1 -- 172.21.15.89:0/4269841937 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b680145e0 con 0x7f8b7009f650 2023-09-30T22:20:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.745+0000 7f8b65ffb700 1 -- 172.21.15.89:0/4269841937 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8b680404f0 con 0x7f8b7009f650 2023-09-30T22:20:11.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.745+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b70074ea0 con 0x7f8b7009f650 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.752+0000 7f2ba08b2700 1 Processor -- start 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.752+0000 7f2ba08b2700 1 -- start start 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2ba08b2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b9c09e930 0x7f2b9c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2ba08b2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f240 0x7f2b9c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2ba08b2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b9c09f650 0x7f2b9c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2ba08b2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c070940 con 0x7f2b9c09f650 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2ba08b2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c06fd60 con 0x7f2b9c09f240 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2ba08b2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c06fa80 con 0x7f2b9c09e930 2023-09-30T22:20:11.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f240 0x7f2b9c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f240 0x7f2b9c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38010/0 (socket says 172.21.15.89:38010) 2023-09-30T22:20:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 -- 172.21.15.89:0/4121692044 learned_addr learned my addr 172.21.15.89:0/4121692044 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b9ad9d700 1 --2- 172.21.15.89:0/4121692044 >> v2:172.21.15.89:3300/0 conn(0x7f2b9c09f650 0x7f2b9c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 -- 172.21.15.89:0/4121692044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b9c09e930 msgr2=0x7f2b9c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 --2- 172.21.15.89:0/4121692044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b9c09e930 0x7f2b9c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 -- 172.21.15.89:0/4121692044 >> v2:172.21.15.89:3300/0 conn(0x7f2b9c09f650 msgr2=0x7f2b9c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 --2- 172.21.15.89:0/4121692044 >> v2:172.21.15.89:3300/0 conn(0x7f2b9c09f650 0x7f2b9c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 -- 172.21.15.89:0/4121692044 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c09fb60 con 0x7f2b9c09f240 2023-09-30T22:20:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.753+0000 7f2b99d9b700 1 --2- 172.21.15.89:0/4121692044 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f240 0x7f2b9c1317f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098f0 tx=0x7f2b90009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a321118d138cee47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2b9959a700 1 -- 172.21.15.89:0/4121692044 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90010a90 con 0x7f2b9c09f240 2023-09-30T22:20:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2b9959a700 1 -- 172.21.15.89:0/4121692044 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90010bf0 con 0x7f2b9c09f240 2023-09-30T22:20:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2b9959a700 1 -- 172.21.15.89:0/4121692044 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90018440 con 0x7f2b9c09f240 2023-09-30T22:20:11.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f8b65ffb700 1 -- 172.21.15.89:0/4269841937 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b68018070 con 0x7f8b7009f650 2023-09-30T22:20:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4121692044 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f240 msgr2=0x7f2b9c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2ba08b2700 1 --2- 172.21.15.89:0/4121692044 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f240 0x7f2b9c1317f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098f0 tx=0x7f2b90009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:11.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4121692044 shutdown_connections 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2ba08b2700 1 --2- 172.21.15.89:0/4121692044 >> v2:172.21.15.89:3300/0 conn(0x7f2b9c09f650 0x7f2b9c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2ba08b2700 1 --2- 172.21.15.89:0/4121692044 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b9c09e930 0x7f2b9c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2ba08b2700 1 --2- 172.21.15.89:0/4121692044 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f240 0x7f2b9c1317f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.754+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4121692044 >> 172.21.15.89:0/4121692044 conn(0x7f2b9c0a4210 msgr2=0x7f2b9c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.755+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4121692044 shutdown_connections 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.755+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4121692044 wait complete. 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.756+0000 7f2ba08b2700 1 Processor -- start 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.756+0000 7f2ba08b2700 1 -- start start 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.756+0000 7f2ba08b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b9c09e930 0x7f2b9c1bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.756+0000 7f2ba08b2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b9c09f240 0x7f2b9c1befb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.756+0000 7f2ba08b2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f650 0x7f2b9c1bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.756+0000 7f2ba08b2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c071d90 con 0x7f2b9c09e930 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.756+0000 7f2ba08b2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1ba730 con 0x7f2b9c09f650 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.756+0000 7f2ba08b2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1bd8f0 con 0x7f2b9c09f240 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b9c09e930 0x7f2b9c1bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f650 0x7f2b9c1bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b9c09e930 0x7f2b9c1bea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49850/0 (socket says 172.21.15.89:49850) 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 -- 172.21.15.89:0/4022030248 learned_addr learned my addr 172.21.15.89:0/4022030248 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 -- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b9c09f240 msgr2=0x7f2b9c1befb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 --2- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b9c09f240 0x7f2b9c1befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 -- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f650 msgr2=0x7f2b9c1bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 --2- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f650 0x7f2b9c1bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 -- 172.21.15.89:0/4022030248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90009550 con 0x7f2b9c09e930 2023-09-30T22:20:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9ad9d700 1 --2- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f650 0x7f2b9c1bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.757+0000 7f2b9a59c700 1 --2- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b9c09e930 0x7f2b9c1bea70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00b530 tx=0x7f2b8c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:11.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.761+0000 7f2b8affd700 1 -- 172.21.15.89:0/4022030248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c011830 con 0x7f2b9c09e930 2023-09-30T22:20:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.761+0000 7f2b8affd700 1 -- 172.21.15.89:0/4022030248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c0119b0 con 0x7f2b9c09e930 2023-09-30T22:20:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.761+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c0a0190 con 0x7f2b9c09e930 2023-09-30T22:20:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.761+0000 7f2b8affd700 1 -- 172.21.15.89:0/4022030248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c011d60 con 0x7f2b9c09e930 2023-09-30T22:20:11.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.762+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c1bc1c0 con 0x7f2b9c09e930 2023-09-30T22:20:11.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.762+0000 7f2b8affd700 1 -- 172.21.15.89:0/4022030248 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b8c01a400 con 0x7f2b9c09e930 2023-09-30T22:20:11.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.762+0000 7f2b8affd700 1 -- 172.21.15.89:0/4022030248 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2b8c040d80 con 0x7f2b9c09e930 2023-09-30T22:20:11.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.763+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c074ea0 con 0x7f2b9c09e930 2023-09-30T22:20:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.768+0000 7f2b8affd700 1 -- 172.21.15.89:0/4022030248 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b8c015070 con 0x7f2b9c09e930 2023-09-30T22:20:11.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.933+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fea200713a0 con 0x7fea2012cf60 2023-09-30T22:20:11.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.933+0000 7fea15ffb700 1 -- 172.21.15.89:0/2701957310 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea1c046060 con 0x7fea2012cf60 2023-09-30T22:20:11.933 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 msgr2=0x7fea2009b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 --2- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2009b640 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fea2006fba0 tx=0x7fea1c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 shutdown_connections 2023-09-30T22:20:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 --2- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea20130260 0x7fea201c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 --2- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2012cf60 0x7fea2009b640 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 --2- 172.21.15.89:0/2701957310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea2012f8b0 0x7fea2009bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 >> 172.21.15.89:0/2701957310 conn(0x7fea200a2aa0 msgr2=0x7fea201b7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:11.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 shutdown_connections 2023-09-30T22:20:11.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.935+0000 7fea279a9700 1 -- 172.21.15.89:0/2701957310 wait complete. 2023-09-30T22:20:11.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:11.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.953+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f47f009f420 con 0x7f47f0139170 2023-09-30T22:20:11.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.953+0000 7f47deffd700 1 -- 172.21.15.89:0/2956996671 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47e802c090 con 0x7f47f0139170 2023-09-30T22:20:11.953 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:11.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 msgr2=0x7f47f01c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f01c8670 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f47e800efc0 tx=0x7f47e800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 shutdown_connections 2023-09-30T22:20:11.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f0139170 0x7f47f01c8670 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f009d6f0 0x7f47f011f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 --2- 172.21.15.89:0/2956996671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f009e1d0 0x7f47f011fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 >> 172.21.15.89:0/2956996671 conn(0x7f47f0066f50 msgr2=0x7f47f0136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:11.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 shutdown_connections 2023-09-30T22:20:11.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.955+0000 7f47f48e7700 1 -- 172.21.15.89:0/2956996671 wait complete. 2023-09-30T22:20:11.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:11.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.974+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2b9c06b250 con 0x7f2b9c09e930 2023-09-30T22:20:11.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.974+0000 7f2b8affd700 1 -- 172.21.15.89:0/4022030248 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b8c015020 con 0x7f2b9c09e930 2023-09-30T22:20:11.974 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:11.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b9c09e930 msgr2=0x7f2b9c1bea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 --2- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b9c09e930 0x7f2b9c1bea70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00b530 tx=0x7f2b8c00b840 comp rx=0 tx=0).stop 2023-09-30T22:20:11.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 shutdown_connections 2023-09-30T22:20:11.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 --2- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b9c09f650 0x7f2b9c1bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 --2- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b9c09e930 0x7f2b9c1bea70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 --2- 172.21.15.89:0/4022030248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b9c09f240 0x7f2b9c1befb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 >> 172.21.15.89:0/4022030248 conn(0x7f2b9c0a4210 msgr2=0x7f2b9c1c0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:11.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 shutdown_connections 2023-09-30T22:20:11.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.976+0000 7f2ba08b2700 1 -- 172.21.15.89:0/4022030248 wait complete. 2023-09-30T22:20:11.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.986+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8b7006b250 con 0x7f8b7009f650 2023-09-30T22:20:11.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.987+0000 7f8b65ffb700 1 -- 172.21.15.89:0/4269841937 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b6802c090 con 0x7f8b7009f650 2023-09-30T22:20:11.987 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:11.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:11.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b7009f650 msgr2=0x7f8b701a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:11.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b7009f650 0x7f8b701a3890 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009e30 tx=0x7f8b6800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 shutdown_connections 2023-09-30T22:20:11.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b7009f650 0x7f8b701a3890 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b7009e930 0x7f8b701a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 --2- 172.21.15.89:0/4269841937 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b7009f240 0x7f8b701a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:11.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 >> 172.21.15.89:0/4269841937 conn(0x7f8b700a4210 msgr2=0x7f8b701b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:11.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 shutdown_connections 2023-09-30T22:20:11.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:11.989+0000 7f8b7726d700 1 -- 172.21.15.89:0/4269841937 wait complete. 2023-09-30T22:20:12.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:12.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:12.967 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:12.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:13.002 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:13.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.099+0000 7f69484f1700 1 Processor -- start 2023-09-30T22:20:13.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.099+0000 7f69484f1700 1 -- start start 2023-09-30T22:20:13.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f69484f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f6940133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f69484f1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6940130a40 0x7f694013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f69484f1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694006bc00 con 0x7f694012ff20 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f69484f1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694006c570 con 0x7f6940130a40 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f69484f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69400a0040 con 0x7f6940133a80 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f694628d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f6940133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f694628d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f6940133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38026/0 (socket says 172.21.15.89:38026) 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f6946a8e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6940130a40 0x7f694013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.100+0000 7f694628d700 1 -- 172.21.15.89:0/3354351644 learned_addr learned my addr 172.21.15.89:0/3354351644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f694628d700 1 -- 172.21.15.89:0/3354351644 >> v2:172.21.15.89:3300/0 conn(0x7f6940130a40 msgr2=0x7f694013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f694628d700 1 --2- 172.21.15.89:0/3354351644 >> v2:172.21.15.89:3300/0 conn(0x7f6940130a40 0x7f694013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f694628d700 1 -- 172.21.15.89:0/3354351644 >> v1:172.21.15.89:6789/0 conn(0x7f694012ff20 legacy=0x7f6940130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f694628d700 1 -- 172.21.15.89:0/3354351644 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940133860 con 0x7f6940133a80 2023-09-30T22:20:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f6946a8e700 1 --2- 172.21.15.89:0/3354351644 >> v2:172.21.15.89:3300/0 conn(0x7f6940130a40 0x7f694013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f694628d700 1 --2- 172.21.15.89:0/3354351644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f6940133e50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f693c0098f0 tx=0x7f693c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d577424dbbe61b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f694528b700 1 -- 172.21.15.89:0/3354351644 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c011830 con 0x7f6940133a80 2023-09-30T22:20:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f694528b700 1 -- 172.21.15.89:0/3354351644 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c011990 con 0x7f6940133a80 2023-09-30T22:20:13.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.101+0000 7f694528b700 1 -- 172.21.15.89:0/3354351644 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c011d20 con 0x7f6940133a80 2023-09-30T22:20:13.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.102+0000 7f69484f1700 1 -- 172.21.15.89:0/3354351644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 msgr2=0x7f6940133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.102+0000 7f69484f1700 1 --2- 172.21.15.89:0/3354351644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f6940133e50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f693c0098f0 tx=0x7f693c009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:13.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.102+0000 7f69484f1700 1 -- 172.21.15.89:0/3354351644 shutdown_connections 2023-09-30T22:20:13.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.102+0000 7f69484f1700 1 --2- 172.21.15.89:0/3354351644 >> v2:172.21.15.89:3300/0 conn(0x7f6940130a40 0x7f694013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.102+0000 7f69484f1700 1 --2- 172.21.15.89:0/3354351644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f6940133e50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.102+0000 7f69484f1700 1 -- 172.21.15.89:0/3354351644 >> 172.21.15.89:0/3354351644 conn(0x7f69400a4210 msgr2=0x7f69400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:13.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.102+0000 7f69484f1700 1 -- 172.21.15.89:0/3354351644 shutdown_connections 2023-09-30T22:20:13.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.102+0000 7f69484f1700 1 -- 172.21.15.89:0/3354351644 wait complete. 2023-09-30T22:20:13.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f69484f1700 1 Processor -- start 2023-09-30T22:20:13.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f69484f1700 1 -- start start 2023-09-30T22:20:13.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f69484f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694012ff20 0x7f694009c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f69484f1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6940130a40 0x7f694009c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f69484f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f694009ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f69484f1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694009d7c0 con 0x7f6940130a40 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f69484f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694011fad0 con 0x7f6940133a80 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f69484f1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f694011fd10 con 0x7f694012ff20 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f694628d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694012ff20 0x7f694009c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f694628d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694012ff20 0x7f694009c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41116/0 (socket says 172.21.15.89:41116) 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.104+0000 7f694628d700 1 -- 172.21.15.89:0/925183504 learned_addr learned my addr 172.21.15.89:0/925183504 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f6946a8e700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f694009ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f6945a8c700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6940130a40 0x7f694009c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f694628d700 1 -- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 msgr2=0x7f694009ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f694628d700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f694009ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f694628d700 1 -- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6940130a40 msgr2=0x7f694009c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f694628d700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6940130a40 0x7f694009c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f694628d700 1 -- 172.21.15.89:0/925183504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6938009a70 con 0x7f694012ff20 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f694628d700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694012ff20 0x7f694009c3d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f693c000c00 tx=0x7f693c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f6936ffd700 1 -- 172.21.15.89:0/925183504 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c0043c0 con 0x7f694012ff20 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693c009550 con 0x7f694012ff20 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f6936ffd700 1 -- 172.21.15.89:0/925183504 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c01f070 con 0x7f694012ff20 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f6936ffd700 1 -- 172.21.15.89:0/925183504 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c0038c0 con 0x7f694012ff20 2023-09-30T22:20:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.105+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694012d770 con 0x7f694012ff20 2023-09-30T22:20:13.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.106+0000 7f6936ffd700 1 -- 172.21.15.89:0/925183504 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f693c021030 con 0x7f694012ff20 2023-09-30T22:20:13.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.107+0000 7f6936ffd700 1 -- 172.21.15.89:0/925183504 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f693c004560 con 0x7f694012ff20 2023-09-30T22:20:13.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.108+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6940074ea0 con 0x7f694012ff20 2023-09-30T22:20:13.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.116+0000 7f6936ffd700 1 -- 172.21.15.89:0/925183504 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f693c00b9d0 con 0x7f694012ff20 2023-09-30T22:20:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c1d812700 1 Processor -- start 2023-09-30T22:20:13.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c1d812700 1 -- start start 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c1d812700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7c1809d6f0 0x7f7c1809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c1d812700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e000 0x7f7c18130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c1d812700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809e410 0x7f7c18130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c1d812700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1809f520 con 0x7f7c1809d6f0 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c1d812700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1809f130 con 0x7f7c1809e000 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c1d812700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1809e930 con 0x7f7c1809e410 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.129+0000 7f7c167fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e000 0x7f7c18130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c167fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e000 0x7f7c18130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38032/0 (socket says 172.21.15.89:38032) 2023-09-30T22:20:13.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c167fc700 1 -- 172.21.15.89:0/2697371383 learned_addr learned my addr 172.21.15.89:0/2697371383 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c177fe700 1 --2- 172.21.15.89:0/2697371383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809e410 0x7f7c18130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c167fc700 1 -- 172.21.15.89:0/2697371383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809e410 msgr2=0x7f7c18130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c167fc700 1 --2- 172.21.15.89:0/2697371383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809e410 0x7f7c18130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c167fc700 1 -- 172.21.15.89:0/2697371383 >> v2:172.21.15.89:3300/0 conn(0x7f7c1809d6f0 msgr2=0x7f7c1809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c167fc700 1 --2- 172.21.15.89:0/2697371383 >> v2:172.21.15.89:3300/0 conn(0x7f7c1809d6f0 0x7f7c1809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c167fc700 1 -- 172.21.15.89:0/2697371383 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1809b2d0 con 0x7f7c1809e000 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c167fc700 1 --2- 172.21.15.89:0/2697371383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e000 0x7f7c18130660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0098f0 tx=0x7f7c0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c18fa4de74fa99d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c15ffb700 1 -- 172.21.15.89:0/2697371383 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c011830 con 0x7f7c1809e000 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c15ffb700 1 -- 172.21.15.89:0/2697371383 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c011990 con 0x7f7c1809e000 2023-09-30T22:20:13.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7f7c15ffb700 1 -- 172.21.15.89:0/2697371383 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c011d20 con 0x7f7c1809e000 2023-09-30T22:20:13.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7fbb01732700 1 Processor -- start 2023-09-30T22:20:13.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.130+0000 7fbb01732700 1 -- start start 2023-09-30T22:20:13.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbb01732700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09d6f0 0x7fbafc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbb01732700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbafc09e000 0x7fbafc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbb01732700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbb01732700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc09f520 con 0x7fbafc09e000 2023-09-30T22:20:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbb01732700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09f130 con 0x7fbafc09d6f0 2023-09-30T22:20:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbb01732700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09e930 con 0x7fbafc09e410 2023-09-30T22:20:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7f7c1d812700 1 -- 172.21.15.89:0/2697371383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e000 msgr2=0x7f7c18130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7f7c1d812700 1 --2- 172.21.15.89:0/2697371383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e000 0x7f7c18130660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0098f0 tx=0x7f7c0c009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41126/0 (socket says 172.21.15.89:41126) 2023-09-30T22:20:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafb7fe700 1 -- 172.21.15.89:0/1779440785 learned_addr learned my addr 172.21.15.89:0/1779440785 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafa7fc700 1 --2- 172.21.15.89:0/1779440785 >> v2:172.21.15.89:3300/0 conn(0x7fbafc09e000 0x7fbafc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafa7fc700 1 -- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 msgr2=0x7fbafc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafa7fc700 1 --2- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafa7fc700 1 -- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09d6f0 msgr2=0x7fbafc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:13.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafa7fc700 1 --2- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09d6f0 0x7fbafc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.131+0000 7fbafa7fc700 1 -- 172.21.15.89:0/1779440785 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc09b2d0 con 0x7fbafc09e000 2023-09-30T22:20:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7f7c1d812700 1 -- 172.21.15.89:0/2697371383 shutdown_connections 2023-09-30T22:20:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7f7c1d812700 1 --2- 172.21.15.89:0/2697371383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809e410 0x7f7c18130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7f7c1d812700 1 --2- 172.21.15.89:0/2697371383 >> v2:172.21.15.89:3300/0 conn(0x7f7c1809d6f0 0x7f7c1809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7f7c1d812700 1 --2- 172.21.15.89:0/2697371383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e000 0x7f7c18130660 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7f7c1d812700 1 -- 172.21.15.89:0/2697371383 >> 172.21.15.89:0/2697371383 conn(0x7f7c18066f50 msgr2=0x7f7c1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7fbafa7fc700 1 --2- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09e000 0x7fbafc130660 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009870 tx=0x7fbaf0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31040d4f3e27106 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7f7c1d812700 1 -- 172.21.15.89:0/2697371383 shutdown_connections 2023-09-30T22:20:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7f7c1d812700 1 -- 172.21.15.89:0/2697371383 wait complete. 2023-09-30T22:20:13.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7fbaf9ffb700 1 -- 172.21.15.89:0/1779440785 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0011830 con 0x7fbafc09e000 2023-09-30T22:20:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7fbaf9ffb700 1 -- 172.21.15.89:0/1779440785 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf0011990 con 0x7fbafc09e000 2023-09-30T22:20:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.132+0000 7fbaf9ffb700 1 -- 172.21.15.89:0/1779440785 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0011d20 con 0x7fbafc09e000 2023-09-30T22:20:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c1d812700 1 Processor -- start 2023-09-30T22:20:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c1d812700 1 -- start start 2023-09-30T22:20:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 -- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09e000 msgr2=0x7fbafc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 --2- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09e000 0x7fbafc130660 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009870 tx=0x7fbaf0009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 -- 172.21.15.89:0/1779440785 shutdown_connections 2023-09-30T22:20:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 --2- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 --2- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09d6f0 0x7fbafc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 --2- 172.21.15.89:0/1779440785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09e000 0x7fbafc130660 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 -- 172.21.15.89:0/1779440785 >> 172.21.15.89:0/1779440785 conn(0x7fbafc066f50 msgr2=0x7fbafc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 -- 172.21.15.89:0/1779440785 shutdown_connections 2023-09-30T22:20:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7fbb01732700 1 -- 172.21.15.89:0/1779440785 wait complete. 2023-09-30T22:20:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c1d812700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809d6f0 0x7f7c1811e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c1d812700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c1809e000 0x7f7c1811e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c1d812700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e410 0x7f7c1811eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c1d812700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1811fb20 con 0x7f7c1809e000 2023-09-30T22:20:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c1d812700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c18120d70 con 0x7f7c1809e410 2023-09-30T22:20:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c1d812700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c181217f0 con 0x7f7c1809d6f0 2023-09-30T22:20:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.133+0000 7f7c177fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e410 0x7f7c1811eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c177fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e410 0x7f7c1811eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38050/0 (socket says 172.21.15.89:38050) 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c177fe700 1 -- 172.21.15.89:0/902301191 learned_addr learned my addr 172.21.15.89:0/902301191 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c167fc700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c1809e000 0x7f7c1811e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c16ffd700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809d6f0 0x7f7c1811e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c177fe700 1 -- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809d6f0 msgr2=0x7f7c1811e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c177fe700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809d6f0 0x7f7c1811e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c177fe700 1 -- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c1809e000 msgr2=0x7f7c1811e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c177fe700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c1809e000 0x7f7c1811e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c177fe700 1 -- 172.21.15.89:0/902301191 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c009550 con 0x7f7c1809e410 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c16ffd700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809d6f0 0x7f7c1811e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c177fe700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e410 0x7f7c1811eac0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000efc0 tx=0x7f7c1000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:13.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7bfffff700 1 -- 172.21.15.89:0/902301191 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10009ea0 con 0x7f7c1809e410 2023-09-30T22:20:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7bfffff700 1 -- 172.21.15.89:0/902301191 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1000f040 con 0x7f7c1809e410 2023-09-30T22:20:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c18120110 con 0x7f7c1809e410 2023-09-30T22:20:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7fbb01732700 1 Processor -- start 2023-09-30T22:20:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7bfffff700 1 -- 172.21.15.89:0/902301191 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10014480 con 0x7f7c1809e410 2023-09-30T22:20:13.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.134+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1809c700 con 0x7f7c1809e410 2023-09-30T22:20:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.135+0000 7fbb01732700 1 -- start start 2023-09-30T22:20:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.135+0000 7fbb01732700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09d6f0 0x7fbafc1324e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.135+0000 7fbb01732700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09e000 0x7fbafc132a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.135+0000 7fbb01732700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc132f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.135+0000 7fbb01732700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc130ec0 con 0x7fbafc09d6f0 2023-09-30T22:20:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.135+0000 7fbb01732700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc135a10 con 0x7fbafc09e000 2023-09-30T22:20:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.135+0000 7fbb01732700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc136140 con 0x7fbafc09e410 2023-09-30T22:20:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.135+0000 7f7bfffff700 1 -- 172.21.15.89:0/902301191 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c100145e0 con 0x7f7c1809e410 2023-09-30T22:20:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7f7bfffff700 1 -- 172.21.15.89:0/902301191 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c10040060 con 0x7f7c1809e410 2023-09-30T22:20:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41140/0 (socket says 172.21.15.89:41140) 2023-09-30T22:20:13.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafb7fe700 1 -- 172.21.15.89:0/211383990 learned_addr learned my addr 172.21.15.89:0/211383990 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:13.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c18074ea0 con 0x7f7c1809e410 2023-09-30T22:20:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafaffd700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09d6f0 0x7fbafc1324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafa7fc700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09e000 0x7fbafc132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafaffd700 1 -- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 msgr2=0x7fbafc132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafaffd700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc132f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafaffd700 1 -- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09e000 msgr2=0x7fbafc132a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafaffd700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09e000 0x7fbafc132a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafaffd700 1 -- 172.21.15.89:0/211383990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf00094a0 con 0x7fbafc09d6f0 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafa7fc700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09e000 0x7fbafc132a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafb7fe700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc132f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbafaffd700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09d6f0 0x7fbafc1324e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00b480 tx=0x7fbaec00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbae37fe700 1 -- 172.21.15.89:0/211383990 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec011830 con 0x7fbafc09d6f0 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.136+0000 7fbae37fe700 1 -- 172.21.15.89:0/211383990 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec011990 con 0x7fbafc09d6f0 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.137+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc135ca0 con 0x7fbafc09d6f0 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.137+0000 7fbae37fe700 1 -- 172.21.15.89:0/211383990 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec011d40 con 0x7fbafc09d6f0 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.137+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc072450 con 0x7fbafc09d6f0 2023-09-30T22:20:13.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.138+0000 7fbae37fe700 1 -- 172.21.15.89:0/211383990 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbaec01a400 con 0x7fbafc09d6f0 2023-09-30T22:20:13.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.138+0000 7fbae37fe700 1 -- 172.21.15.89:0/211383990 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbaec03f500 con 0x7fbafc09d6f0 2023-09-30T22:20:13.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.139+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbadc0052e0 con 0x7fbafc09d6f0 2023-09-30T22:20:13.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.141+0000 7f7bfffff700 1 -- 172.21.15.89:0/902301191 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c100248f0 con 0x7f7c1809e410 2023-09-30T22:20:13.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.143+0000 7fbae37fe700 1 -- 172.21.15.89:0/211383990 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbaec01a6b0 con 0x7fbafc09d6f0 2023-09-30T22:20:13.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.213+0000 7ff4ed4b5700 1 Processor -- start 2023-09-30T22:20:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.213+0000 7ff4ed4b5700 1 -- start start 2023-09-30T22:20:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4ed4b5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4ed4b5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff4e8133e30 0x7ff4e812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4ed4b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e81304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4ed4b5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e806b990 con 0x7ff4e8133e30 2023-09-30T22:20:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4ed4b5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80a0590 con 0x7ff4e8134260 2023-09-30T22:20:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4ed4b5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e809fb80 con 0x7ff4e8133420 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e81304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e81304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38072/0 (socket says 172.21.15.89:38072) 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e77fe700 1 -- 172.21.15.89:0/525740858 learned_addr learned my addr 172.21.15.89:0/525740858 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e67fc700 1 --2- 172.21.15.89:0/525740858 >> v2:172.21.15.89:3300/0 conn(0x7ff4e8133e30 0x7ff4e812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e6ffd700 1 -- 172.21.15.89:0/525740858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 msgr2=0x7ff4e81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e6ffd700 1 --2- 172.21.15.89:0/525740858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e81304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e6ffd700 1 -- 172.21.15.89:0/525740858 >> v2:172.21.15.89:3300/0 conn(0x7ff4e8133e30 msgr2=0x7ff4e812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e6ffd700 1 --2- 172.21.15.89:0/525740858 >> v2:172.21.15.89:3300/0 conn(0x7ff4e8133e30 0x7ff4e812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.214+0000 7ff4e6ffd700 1 -- 172.21.15.89:0/525740858 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80a0030 con 0x7ff4e8133420 2023-09-30T22:20:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.215+0000 7ff4e6ffd700 1 --2- 172.21.15.89:0/525740858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e81337f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009840 tx=0x7ff4d8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53d0f30426c2a7ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.215+0000 7ff4e5ffb700 1 -- 172.21.15.89:0/525740858 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8011830 con 0x7ff4e8133420 2023-09-30T22:20:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.215+0000 7ff4e5ffb700 1 -- 172.21.15.89:0/525740858 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d8011990 con 0x7ff4e8133420 2023-09-30T22:20:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.215+0000 7ff4e5ffb700 1 -- 172.21.15.89:0/525740858 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8011d20 con 0x7ff4e8133420 2023-09-30T22:20:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.215+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/525740858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 msgr2=0x7ff4e81337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.215+0000 7ff4ed4b5700 1 --2- 172.21.15.89:0/525740858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e81337f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009840 tx=0x7ff4d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.216+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/525740858 shutdown_connections 2023-09-30T22:20:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.216+0000 7ff4ed4b5700 1 --2- 172.21.15.89:0/525740858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e81304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.216+0000 7ff4ed4b5700 1 --2- 172.21.15.89:0/525740858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e81337f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.216+0000 7ff4ed4b5700 1 --2- 172.21.15.89:0/525740858 >> v2:172.21.15.89:3300/0 conn(0x7ff4e8133e30 0x7ff4e812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.216+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/525740858 >> 172.21.15.89:0/525740858 conn(0x7ff4e80a4210 msgr2=0x7ff4e80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.216+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/525740858 shutdown_connections 2023-09-30T22:20:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.216+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/525740858 wait complete. 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.217+0000 7ff4ed4b5700 1 Processor -- start 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.217+0000 7ff4ed4b5700 1 -- start start 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4ed4b5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4ed4b5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8133e30 0x7ff4e809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4ed4b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4ed4b5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e809d870 con 0x7ff4e8133e30 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4ed4b5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e812da40 con 0x7ff4e8134260 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4ed4b5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e811ff40 con 0x7ff4e8133420 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8133e30 0x7ff4e809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8133e30 0x7ff4e809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49924/0 (socket says 172.21.15.89:49924) 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e67fc700 1 -- 172.21.15.89:0/907179742 learned_addr learned my addr 172.21.15.89:0/907179742 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e6ffd700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e77fe700 1 -- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 msgr2=0x7ff4e809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e77fe700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e77fe700 1 -- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8133e30 msgr2=0x7ff4e809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e77fe700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8133e30 0x7ff4e809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e77fe700 1 -- 172.21.15.89:0/907179742 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80094a0 con 0x7ff4e8134260 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.218+0000 7ff4e67fc700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8133e30 0x7ff4e809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.219+0000 7ff4e77fe700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e809cf00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000efc0 tx=0x7ff4e000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.219+0000 7ff4cf7fe700 1 -- 172.21.15.89:0/907179742 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0003c20 con 0x7ff4e8134260 2023-09-30T22:20:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.219+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e809ddd0 con 0x7ff4e8134260 2023-09-30T22:20:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.219+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e811f4d0 con 0x7ff4e8134260 2023-09-30T22:20:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.220+0000 7ff4cf7fe700 1 -- 172.21.15.89:0/907179742 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e000f040 con 0x7ff4e8134260 2023-09-30T22:20:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.220+0000 7ff4cf7fe700 1 -- 172.21.15.89:0/907179742 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0009e60 con 0x7ff4e8134260 2023-09-30T22:20:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.221+0000 7ff4cf7fe700 1 -- 172.21.15.89:0/907179742 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff4e000c880 con 0x7ff4e8134260 2023-09-30T22:20:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.221+0000 7ff4cf7fe700 1 -- 172.21.15.89:0/907179742 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff4e003f8f0 con 0x7ff4e8134260 2023-09-30T22:20:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.221+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e811fa20 con 0x7ff4e8134260 2023-09-30T22:20:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.230+0000 7ff4cf7fe700 1 -- 172.21.15.89:0/907179742 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff4e001c360 con 0x7ff4e8134260 2023-09-30T22:20:13.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.340+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbadc005590 con 0x7fbafc09d6f0 2023-09-30T22:20:13.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.340+0000 7fbae37fe700 1 -- 172.21.15.89:0/211383990 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbaec029090 con 0x7fbafc09d6f0 2023-09-30T22:20:13.340 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:13.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.341+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7c1806c360 con 0x7f7c1809e410 2023-09-30T22:20:13.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.341+0000 7f7bfffff700 1 -- 172.21.15.89:0/902301191 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c10024300 con 0x7f7c1809e410 2023-09-30T22:20:13.342 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:13.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09d6f0 msgr2=0x7fbafc1324e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09d6f0 0x7fbafc1324e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00b480 tx=0x7fbaec00b790 comp rx=0 tx=0).stop 2023-09-30T22:20:13.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 shutdown_connections 2023-09-30T22:20:13.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbafc09e410 0x7fbafc132f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbafc09d6f0 0x7fbafc1324e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 --2- 172.21.15.89:0/211383990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbafc09e000 0x7fbafc132a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 >> 172.21.15.89:0/211383990 conn(0x7fbafc066f50 msgr2=0x7fbafc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:13.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 shutdown_connections 2023-09-30T22:20:13.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7fbb01732700 1 -- 172.21.15.89:0/211383990 wait complete. 2023-09-30T22:20:13.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e410 msgr2=0x7f7c1811eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.343+0000 7f7c1d812700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e410 0x7f7c1811eac0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000efc0 tx=0x7f7c1000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.344+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 shutdown_connections 2023-09-30T22:20:13.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.344+0000 7f7c1d812700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c1809e410 0x7f7c1811eac0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.344+0000 7f7c1d812700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c1809d6f0 0x7f7c1811e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.344+0000 7f7c1d812700 1 --2- 172.21.15.89:0/902301191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c1809e000 0x7f7c1811e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.344+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 >> 172.21.15.89:0/902301191 conn(0x7f7c18066f50 msgr2=0x7f7c1806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:13.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.344+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 shutdown_connections 2023-09-30T22:20:13.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.344+0000 7f7c1d812700 1 -- 172.21.15.89:0/902301191 wait complete. 2023-09-30T22:20:13.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:13.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.409+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f694006b250 con 0x7f694012ff20 2023-09-30T22:20:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.409+0000 7f6936ffd700 1 -- 172.21.15.89:0/925183504 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f693c0233d0 con 0x7f694012ff20 2023-09-30T22:20:13.409 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694012ff20 msgr2=0x7f694009c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694012ff20 0x7f694009c3d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f693c000c00 tx=0x7f693c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 shutdown_connections 2023-09-30T22:20:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6940133a80 0x7f694009ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f694012ff20 0x7f694009c3d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 --2- 172.21.15.89:0/925183504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6940130a40 0x7f694009c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 >> 172.21.15.89:0/925183504 conn(0x7f69400a4210 msgr2=0x7f694009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 shutdown_connections 2023-09-30T22:20:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.411+0000 7f69484f1700 1 -- 172.21.15.89:0/925183504 wait complete. 2023-09-30T22:20:13.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.475+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff4e8074ea0 con 0x7ff4e8134260 2023-09-30T22:20:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.475+0000 7ff4cf7fe700 1 -- 172.21.15.89:0/907179742 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff4e0014e00 con 0x7ff4e8134260 2023-09-30T22:20:13.476 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.477+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 msgr2=0x7ff4e809cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.477+0000 7ff4ed4b5700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e809cf00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000efc0 tx=0x7ff4e000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.478+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 shutdown_connections 2023-09-30T22:20:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.478+0000 7ff4ed4b5700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4e8134260 0x7ff4e809cf00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.478+0000 7ff4ed4b5700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff4e8133420 0x7ff4e809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.478+0000 7ff4ed4b5700 1 --2- 172.21.15.89:0/907179742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8133e30 0x7ff4e809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.478+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 >> 172.21.15.89:0/907179742 conn(0x7ff4e80a4210 msgr2=0x7ff4e80a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.478+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 shutdown_connections 2023-09-30T22:20:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:13.478+0000 7ff4ed4b5700 1 -- 172.21.15.89:0/907179742 wait complete. 2023-09-30T22:20:13.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:14.358 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:14.358 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:14.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:14.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.493+0000 7f7c62a7a700 1 Processor -- start 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.493+0000 7f7c62a7a700 1 -- start start 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.493+0000 7f7c62a7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.493+0000 7f7c62a7a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c12f8b0 0x7f7c5c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.493+0000 7f7c62a7a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5c12cd90 con 0x7f7c5c130260 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.493+0000 7f7c62a7a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c12d330 con 0x7f7c5c12cf60 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.493+0000 7f7c62a7a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1306d0 con 0x7f7c5c12f8b0 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c60816700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c60816700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38084/0 (socket says 172.21.15.89:38084) 2023-09-30T22:20:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c60816700 1 -- 172.21.15.89:0/3652404382 learned_addr learned my addr 172.21.15.89:0/3652404382 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c5bfff700 1 --2- 172.21.15.89:0/3652404382 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c12f8b0 0x7f7c5c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c60816700 1 -- 172.21.15.89:0/3652404382 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c12f8b0 msgr2=0x7f7c5c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c60816700 1 --2- 172.21.15.89:0/3652404382 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c12f8b0 0x7f7c5c12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c60816700 1 -- 172.21.15.89:0/3652404382 >> v1:172.21.15.89:6789/0 conn(0x7f7c5c130260 legacy=0x7f7c5c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c60816700 1 -- 172.21.15.89:0/3652404382 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c071780 con 0x7f7c5c12cf60 2023-09-30T22:20:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c60816700 1 --2- 172.21.15.89:0/3652404382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c06f5d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009920 tx=0x7f7c50009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ce77088c9a8588 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7fcb947d7700 1 Processor -- start 2023-09-30T22:20:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c5b7fe700 1 -- 172.21.15.89:0/3652404382 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004230 con 0x7f7c5c12cf60 2023-09-30T22:20:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c5b7fe700 1 -- 172.21.15.89:0/3652404382 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50004390 con 0x7f7c5c12cf60 2023-09-30T22:20:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7f7c5b7fe700 1 -- 172.21.15.89:0/3652404382 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50009e70 con 0x7f7c5c12cf60 2023-09-30T22:20:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.494+0000 7fcb947d7700 1 -- start start 2023-09-30T22:20:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7fcb947d7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133420 0x7fcb8c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7f7c62a7a700 1 -- 172.21.15.89:0/3652404382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 msgr2=0x7f7c5c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7f7c62a7a700 1 --2- 172.21.15.89:0/3652404382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c06f5d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009920 tx=0x7f7c50009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7fcb947d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb8c133e30 0x7fcb8c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7fcb947d7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c134260 0x7fcb8c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7fcb947d7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8c06b990 con 0x7fcb8c133e30 2023-09-30T22:20:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7fcb947d7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0a0590 con 0x7fcb8c133420 2023-09-30T22:20:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7fcb947d7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c09fb80 con 0x7fcb8c134260 2023-09-30T22:20:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.495+0000 7fcb92573700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133420 0x7fcb8c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92573700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133420 0x7fcb8c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38094/0 (socket says 172.21.15.89:38094) 2023-09-30T22:20:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7f7c62a7a700 1 -- 172.21.15.89:0/3652404382 shutdown_connections 2023-09-30T22:20:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7f7c62a7a700 1 --2- 172.21.15.89:0/3652404382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c06f5d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92573700 1 -- 172.21.15.89:0/1017648604 learned_addr learned my addr 172.21.15.89:0/1017648604 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7f7c62a7a700 1 --2- 172.21.15.89:0/3652404382 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c12f8b0 0x7f7c5c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7f7c62a7a700 1 -- 172.21.15.89:0/3652404382 >> 172.21.15.89:0/3652404382 conn(0x7f7c5c0a2aa0 msgr2=0x7f7c5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7f7c62a7a700 1 -- 172.21.15.89:0/3652404382 shutdown_connections 2023-09-30T22:20:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7f7c62a7a700 1 -- 172.21.15.89:0/3652404382 wait complete. 2023-09-30T22:20:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb91d72700 1 --2- 172.21.15.89:0/1017648604 >> v2:172.21.15.89:3300/0 conn(0x7fcb8c133e30 0x7fcb8c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92d74700 1 --2- 172.21.15.89:0/1017648604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c134260 0x7fcb8c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92573700 1 -- 172.21.15.89:0/1017648604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c134260 msgr2=0x7fcb8c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92573700 1 --2- 172.21.15.89:0/1017648604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c134260 0x7fcb8c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92573700 1 -- 172.21.15.89:0/1017648604 >> v2:172.21.15.89:3300/0 conn(0x7fcb8c133e30 msgr2=0x7fcb8c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92573700 1 --2- 172.21.15.89:0/1017648604 >> v2:172.21.15.89:3300/0 conn(0x7fcb8c133e30 0x7fcb8c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92573700 1 -- 172.21.15.89:0/1017648604 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0a0030 con 0x7fcb8c133420 2023-09-30T22:20:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb92573700 1 --2- 172.21.15.89:0/1017648604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133420 0x7fcb8c1337f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcb880098f0 tx=0x7fcb88009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa8aa426de173b67 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb91571700 1 -- 172.21.15.89:0/1017648604 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88011830 con 0x7fcb8c133420 2023-09-30T22:20:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb91571700 1 -- 172.21.15.89:0/1017648604 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88011990 con 0x7fcb8c133420 2023-09-30T22:20:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.496+0000 7fcb91571700 1 -- 172.21.15.89:0/1017648604 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88011d20 con 0x7fcb8c133420 2023-09-30T22:20:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.497+0000 7f7c62a7a700 1 Processor -- start 2023-09-30T22:20:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.497+0000 7f7c62a7a700 1 -- start start 2023-09-30T22:20:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.497+0000 7fcb947d7700 1 -- 172.21.15.89:0/1017648604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133420 msgr2=0x7fcb8c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.497+0000 7fcb947d7700 1 --2- 172.21.15.89:0/1017648604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133420 0x7fcb8c1337f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcb880098f0 tx=0x7fcb88009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7fcb947d7700 1 -- 172.21.15.89:0/1017648604 shutdown_connections 2023-09-30T22:20:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7fcb947d7700 1 --2- 172.21.15.89:0/1017648604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c134260 0x7fcb8c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7fcb947d7700 1 --2- 172.21.15.89:0/1017648604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133420 0x7fcb8c1337f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7fcb947d7700 1 --2- 172.21.15.89:0/1017648604 >> v2:172.21.15.89:3300/0 conn(0x7fcb8c133e30 0x7fcb8c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7fcb947d7700 1 -- 172.21.15.89:0/1017648604 >> 172.21.15.89:0/1017648604 conn(0x7fcb8c0a4210 msgr2=0x7fcb8c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c62a7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c125b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c62a7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c5c12f8b0 0x7f7c5c126090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c62a7a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c130260 0x7f7c5c1265d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c62a7a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c129170 con 0x7f7c5c12f8b0 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c62a7a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c129350 con 0x7f7c5c12cf60 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c62a7a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c12f680 con 0x7f7c5c130260 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c5bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c5c12f8b0 0x7f7c5c126090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c60816700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c125b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c5bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c5c12f8b0 0x7f7c5c126090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49928/0 (socket says 172.21.15.89:49928) 2023-09-30T22:20:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c5bfff700 1 -- 172.21.15.89:0/2821856410 learned_addr learned my addr 172.21.15.89:0/2821856410 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c61017700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c130260 0x7f7c5c1265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7fcb947d7700 1 -- 172.21.15.89:0/1017648604 shutdown_connections 2023-09-30T22:20:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c61017700 1 -- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 msgr2=0x7f7c5c125b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c61017700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c125b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c61017700 1 -- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c5c12f8b0 msgr2=0x7f7c5c126090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c61017700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c5c12f8b0 0x7f7c5c126090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c61017700 1 -- 172.21.15.89:0/2821856410 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009550 con 0x7f7c5c130260 2023-09-30T22:20:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7fcb947d7700 1 -- 172.21.15.89:0/1017648604 wait complete. 2023-09-30T22:20:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.498+0000 7f7c5bfff700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c5c12f8b0 0x7f7c5c126090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.499+0000 7f7c61017700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c130260 0x7f7c5c1265d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400c780 tx=0x7f7c5400ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.499+0000 7fcb947d7700 1 Processor -- start 2023-09-30T22:20:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.499+0000 7f7c597fa700 1 -- 172.21.15.89:0/2821856410 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54003e10 con 0x7f7c5c130260 2023-09-30T22:20:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.499+0000 7f7c597fa700 1 -- 172.21.15.89:0/2821856410 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c540050d0 con 0x7f7c5c130260 2023-09-30T22:20:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.499+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c129fa0 con 0x7f7c5c130260 2023-09-30T22:20:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7f7c597fa700 1 -- 172.21.15.89:0/2821856410 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54005480 con 0x7f7c5c130260 2023-09-30T22:20:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c128db0 con 0x7f7c5c130260 2023-09-30T22:20:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.499+0000 7fcb947d7700 1 -- start start 2023-09-30T22:20:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb947d7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c133420 0x7fcb8c09bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb947d7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133e30 0x7fcb8c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb947d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8c134260 0x7fcb8c09c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb947d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c09d390 con 0x7fcb8c134260 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb947d7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c11b9b0 con 0x7fcb8c133e30 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb947d7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c11b160 con 0x7fcb8c133420 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb91d72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133e30 0x7fcb8c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb91d72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133e30 0x7fcb8c09c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38114/0 (socket says 172.21.15.89:38114) 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.500+0000 7fcb91d72700 1 -- 172.21.15.89:0/3759583641 learned_addr learned my addr 172.21.15.89:0/3759583641 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb92573700 1 --2- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c133420 0x7fcb8c09bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb91d72700 1 -- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c133420 msgr2=0x7fcb8c09bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb91d72700 1 --2- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c133420 0x7fcb8c09bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb91d72700 1 -- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8c134260 msgr2=0x7fcb8c09c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb91d72700 1 --2- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8c134260 0x7fcb8c09c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb91d72700 1 -- 172.21.15.89:0/3759583641 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb88009550 con 0x7fcb8c133e30 2023-09-30T22:20:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb91d72700 1 --2- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133e30 0x7fcb8c09c400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00e9b0 tx=0x7fcb7c00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7f7c597fa700 1 -- 172.21.15.89:0/2821856410 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c540055e0 con 0x7f7c5c130260 2023-09-30T22:20:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb82ffd700 1 -- 172.21.15.89:0/3759583641 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c009710 con 0x7fcb8c133e30 2023-09-30T22:20:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb82ffd700 1 -- 172.21.15.89:0/3759583641 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c00c850 con 0x7fcb8c133e30 2023-09-30T22:20:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7fcb82ffd700 1 -- 172.21.15.89:0/3759583641 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00cbc0 con 0x7fcb8c133e30 2023-09-30T22:20:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.501+0000 7f7c597fa700 1 -- 172.21.15.89:0/2821856410 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c5403ea20 con 0x7f7c5c130260 2023-09-30T22:20:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.502+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c131520 con 0x7fcb8c133e30 2023-09-30T22:20:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.502+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c09d9f0 con 0x7fcb8c133e30 2023-09-30T22:20:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.502+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5c128a70 con 0x7f7c5c130260 2023-09-30T22:20:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.502+0000 7fcb82ffd700 1 -- 172.21.15.89:0/3759583641 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb7c00cd20 con 0x7fcb8c133e30 2023-09-30T22:20:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.503+0000 7fcb82ffd700 1 -- 172.21.15.89:0/3759583641 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb7c03fd10 con 0x7fcb8c133e30 2023-09-30T22:20:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.503+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c09e240 con 0x7fcb8c133e30 2023-09-30T22:20:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.508+0000 7f7c597fa700 1 -- 172.21.15.89:0/2821856410 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c54011770 con 0x7f7c5c130260 2023-09-30T22:20:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.508+0000 7fcb82ffd700 1 -- 172.21.15.89:0/3759583641 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb7c021080 con 0x7fcb8c133e30 2023-09-30T22:20:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.559+0000 7fa0882ba700 1 Processor -- start 2023-09-30T22:20:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.559+0000 7fa0882ba700 1 -- start start 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.560+0000 7fa0882ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa08006a770 0x7fa08006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.560+0000 7fa0882ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa08006b080 0x7fa080131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.560+0000 7fa0882ba700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08009ece0 con 0x7fa080131b40 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.560+0000 7fa0882ba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08009e760 con 0x7fa08006b080 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.560+0000 7fa0882ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08009e040 con 0x7fa08006a770 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.560+0000 7fa086056700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa08006a770 0x7fa08006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.560+0000 7fa086056700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa08006a770 0x7fa08006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38130/0 (socket says 172.21.15.89:38130) 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.560+0000 7fa086056700 1 -- 172.21.15.89:0/4220602260 learned_addr learned my addr 172.21.15.89:0/4220602260 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.561+0000 7fa086056700 1 -- 172.21.15.89:0/4220602260 >> v2:172.21.15.89:3300/0 conn(0x7fa08006b080 msgr2=0x7fa080131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.561+0000 7fa086056700 1 --2- 172.21.15.89:0/4220602260 >> v2:172.21.15.89:3300/0 conn(0x7fa08006b080 0x7fa080131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.561+0000 7fa086056700 1 -- 172.21.15.89:0/4220602260 >> v1:172.21.15.89:6789/0 conn(0x7fa080131b40 legacy=0x7fa080138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.561+0000 7fa086056700 1 -- 172.21.15.89:0/4220602260 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08009f630 con 0x7fa08006a770 2023-09-30T22:20:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.561+0000 7fa086056700 1 --2- 172.21.15.89:0/4220602260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa08006a770 0x7fa08006ab40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0098f0 tx=0x7fa07c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b8382583fc113a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.561+0000 7fa085054700 1 -- 172.21.15.89:0/4220602260 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c004230 con 0x7fa08006a770 2023-09-30T22:20:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.561+0000 7fa085054700 1 -- 172.21.15.89:0/4220602260 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c004390 con 0x7fa08006a770 2023-09-30T22:20:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.561+0000 7fa085054700 1 -- 172.21.15.89:0/4220602260 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0036a0 con 0x7fa08006a770 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.562+0000 7fa0882ba700 1 -- 172.21.15.89:0/4220602260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa08006a770 msgr2=0x7fa08006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.562+0000 7fa0882ba700 1 --2- 172.21.15.89:0/4220602260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa08006a770 0x7fa08006ab40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0098f0 tx=0x7fa07c009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.562+0000 7fa0882ba700 1 -- 172.21.15.89:0/4220602260 shutdown_connections 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.562+0000 7fa0882ba700 1 --2- 172.21.15.89:0/4220602260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa08006a770 0x7fa08006ab40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.562+0000 7fa0882ba700 1 --2- 172.21.15.89:0/4220602260 >> v2:172.21.15.89:3300/0 conn(0x7fa08006b080 0x7fa080131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.562+0000 7fa0882ba700 1 -- 172.21.15.89:0/4220602260 >> 172.21.15.89:0/4220602260 conn(0x7fa080066f50 msgr2=0x7fa08009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.562+0000 7fa0882ba700 1 -- 172.21.15.89:0/4220602260 shutdown_connections 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.562+0000 7fa0882ba700 1 -- 172.21.15.89:0/4220602260 wait complete. 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa0882ba700 1 Processor -- start 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa0882ba700 1 -- start start 2023-09-30T22:20:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa0882ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa08006a770 0x7fa0801c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa0882ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa08006b080 0x7fa0801c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa0882ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa080131b40 0x7fa0801c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa0882ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa080071e60 con 0x7fa08006b080 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa0882ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08011e330 con 0x7fa080131b40 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa0882ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa08011fb20 con 0x7fa08006a770 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa086857700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa080131b40 0x7fa0801c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa086857700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa080131b40 0x7fa0801c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38140/0 (socket says 172.21.15.89:38140) 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa086857700 1 -- 172.21.15.89:0/1077959729 learned_addr learned my addr 172.21.15.89:0/1077959729 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.563+0000 7fa086056700 1 --2- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa08006a770 0x7fa0801c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa086857700 1 -- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa08006a770 msgr2=0x7fa0801c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa086857700 1 --2- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa08006a770 0x7fa0801c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa086857700 1 -- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa08006b080 msgr2=0x7fa0801c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa086857700 1 --2- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa08006b080 0x7fa0801c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa086857700 1 -- 172.21.15.89:0/1077959729 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c009550 con 0x7fa080131b40 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa086857700 1 --2- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa080131b40 0x7fa0801c6ac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c7a0 tx=0x7fa07800cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa076ffd700 1 -- 172.21.15.89:0/1077959729 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078003e10 con 0x7fa080131b40 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa080131050 con 0x7fa080131b40 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa076ffd700 1 -- 172.21.15.89:0/1077959729 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa078005070 con 0x7fa080131b40 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa076ffd700 1 -- 172.21.15.89:0/1077959729 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078005420 con 0x7fa080131b40 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.564+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa080120850 con 0x7fa080131b40 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.565+0000 7fa076ffd700 1 -- 172.21.15.89:0/1077959729 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa078005580 con 0x7fa080131b40 2023-09-30T22:20:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.565+0000 7fa076ffd700 1 -- 172.21.15.89:0/1077959729 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa07803e970 con 0x7fa080131b40 2023-09-30T22:20:14.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.565+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa080074ea0 con 0x7fa080131b40 2023-09-30T22:20:14.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.574+0000 7fa076ffd700 1 -- 172.21.15.89:0/1077959729 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa07800ab60 con 0x7fa080131b40 2023-09-30T22:20:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.621+0000 7f83c359d700 1 Processor -- start 2023-09-30T22:20:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.621+0000 7f83c359d700 1 -- start start 2023-09-30T22:20:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c359d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09d8d0 0x7f83bc09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c359d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e1e0 0x7f83bc130460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c359d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83bc09b6e0 con 0x7f83bc09e610 2023-09-30T22:20:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c359d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc09b470 con 0x7f83bc09d8d0 2023-09-30T22:20:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c359d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc09eb80 con 0x7f83bc09e1e0 2023-09-30T22:20:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c259b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09d8d0 0x7f83bc09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c259b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09d8d0 0x7f83bc09dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38148/0 (socket says 172.21.15.89:38148) 2023-09-30T22:20:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c259b700 1 -- 172.21.15.89:0/2261876106 learned_addr learned my addr 172.21.15.89:0/2261876106 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.622+0000 7f83c1d9a700 1 --2- 172.21.15.89:0/2261876106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e1e0 0x7f83bc130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c259b700 1 -- 172.21.15.89:0/2261876106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e1e0 msgr2=0x7f83bc130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c259b700 1 --2- 172.21.15.89:0/2261876106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e1e0 0x7f83bc130460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c1599700 1 -- 172.21.15.89:0/2261876106 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546427904 0 0) 0x7f83bc09b6e0 con 0x7f83bc09e610 2023-09-30T22:20:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c259b700 1 -- 172.21.15.89:0/2261876106 >> v1:172.21.15.89:6789/0 conn(0x7f83bc09e610 legacy=0x7f83bc1309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c259b700 1 -- 172.21.15.89:0/2261876106 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc09ae00 con 0x7f83bc09d8d0 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c259b700 1 --2- 172.21.15.89:0/2261876106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09d8d0 0x7f83bc09dca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f83b800bbc0 tx=0x7f83b800bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3657d01324dc9988 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c1599700 1 -- 172.21.15.89:0/2261876106 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800dc70 con 0x7f83bc09d8d0 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c1599700 1 -- 172.21.15.89:0/2261876106 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b800ddd0 con 0x7f83bc09d8d0 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.623+0000 7f83c1599700 1 -- 172.21.15.89:0/2261876106 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8012990 con 0x7f83bc09d8d0 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.624+0000 7f83c359d700 1 -- 172.21.15.89:0/2261876106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09d8d0 msgr2=0x7f83bc09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.624+0000 7f83c359d700 1 --2- 172.21.15.89:0/2261876106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09d8d0 0x7f83bc09dca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f83b800bbc0 tx=0x7f83b800bed0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.624+0000 7f83c359d700 1 -- 172.21.15.89:0/2261876106 shutdown_connections 2023-09-30T22:20:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.624+0000 7f83c359d700 1 --2- 172.21.15.89:0/2261876106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09d8d0 0x7f83bc09dca0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.624+0000 7f83c359d700 1 --2- 172.21.15.89:0/2261876106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e1e0 0x7f83bc130460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.624+0000 7f83c359d700 1 -- 172.21.15.89:0/2261876106 >> 172.21.15.89:0/2261876106 conn(0x7f83bc0a2e20 msgr2=0x7f83bc0a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.624+0000 7f83c359d700 1 -- 172.21.15.89:0/2261876106 shutdown_connections 2023-09-30T22:20:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.624+0000 7f83c359d700 1 -- 172.21.15.89:0/2261876106 wait complete. 2023-09-30T22:20:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c359d700 1 Processor -- start 2023-09-30T22:20:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c359d700 1 -- start start 2023-09-30T22:20:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c359d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83bc09d8d0 0x7f83bc11dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c359d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09e1e0 0x7f83bc11e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c359d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e610 0x7f83bc11e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c359d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc071d00 con 0x7f83bc09d8d0 2023-09-30T22:20:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c359d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc120f20 con 0x7f83bc09e1e0 2023-09-30T22:20:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c359d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc09c600 con 0x7f83bc09e610 2023-09-30T22:20:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c259b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83bc09d8d0 0x7f83bc11dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c259b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83bc09d8d0 0x7f83bc11dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49958/0 (socket says 172.21.15.89:49958) 2023-09-30T22:20:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c259b700 1 -- 172.21.15.89:0/637116226 learned_addr learned my addr 172.21.15.89:0/637116226 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c1d9a700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09e1e0 0x7f83bc11e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.625+0000 7f83c2d9c700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e610 0x7f83bc11e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c259b700 1 -- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e610 msgr2=0x7f83bc11e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c259b700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e610 0x7f83bc11e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c259b700 1 -- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09e1e0 msgr2=0x7f83bc11e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c259b700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09e1e0 0x7f83bc11e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c259b700 1 -- 172.21.15.89:0/637116226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b800b7f0 con 0x7f83bc09d8d0 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c2d9c700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e610 0x7f83bc11e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c1d9a700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09e1e0 0x7f83bc11e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c259b700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83bc09d8d0 0x7f83bc11dc70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000c00 tx=0x7f83b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83b2ffd700 1 -- 172.21.15.89:0/637116226 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8020070 con 0x7f83bc09d8d0 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83b2ffd700 1 -- 172.21.15.89:0/637116226 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8012770 con 0x7f83bc09d8d0 2023-09-30T22:20:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c359d700 1 -- 172.21.15.89:0/637116226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc11f880 con 0x7f83bc09d8d0 2023-09-30T22:20:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83b2ffd700 1 -- 172.21.15.89:0/637116226 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8012b20 con 0x7f83bc09d8d0 2023-09-30T22:20:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.626+0000 7f83c359d700 1 -- 172.21.15.89:0/637116226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc121c90 con 0x7f83bc09d8d0 2023-09-30T22:20:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.627+0000 7f83b2ffd700 1 -- 172.21.15.89:0/637116226 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f83b8024070 con 0x7f83bc09d8d0 2023-09-30T22:20:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.627+0000 7f83b0ff9700 1 -- 172.21.15.89:0/637116226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc04f2a0 con 0x7f83bc09d8d0 2023-09-30T22:20:14.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.627+0000 7f83b2ffd700 1 -- 172.21.15.89:0/637116226 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f83b80445a0 con 0x7f83bc09d8d0 2023-09-30T22:20:14.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.637+0000 7f83b2ffd700 1 -- 172.21.15.89:0/637116226 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f83b801b400 con 0x7f83bc09d8d0 2023-09-30T22:20:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.707+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb8c074ea0 con 0x7fcb8c133e30 2023-09-30T22:20:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.707+0000 7fcb82ffd700 1 -- 172.21.15.89:0/3759583641 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb7c017020 con 0x7fcb8c133e30 2023-09-30T22:20:14.707 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.709+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7c5c074ea0 con 0x7f7c5c130260 2023-09-30T22:20:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.710+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133e30 msgr2=0x7fcb8c09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.710+0000 7fcb947d7700 1 --2- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133e30 0x7fcb8c09c400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00e9b0 tx=0x7fcb7c00ed70 comp rx=0 tx=0).stop 2023-09-30T22:20:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.710+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 shutdown_connections 2023-09-30T22:20:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.710+0000 7fcb947d7700 1 --2- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb8c134260 0x7fcb8c09c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.710+0000 7fcb947d7700 1 --2- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb8c133420 0x7fcb8c09bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.710+0000 7fcb947d7700 1 --2- 172.21.15.89:0/3759583641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb8c133e30 0x7fcb8c09c400 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.710+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 >> 172.21.15.89:0/3759583641 conn(0x7fcb8c0a4210 msgr2=0x7fcb8c130150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:14.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.710+0000 7f7c597fa700 1 -- 172.21.15.89:0/2821856410 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c54010300 con 0x7f7c5c130260 2023-09-30T22:20:14.711 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.711+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 shutdown_connections 2023-09-30T22:20:14.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.711+0000 7fcb947d7700 1 -- 172.21.15.89:0/3759583641 wait complete. 2023-09-30T22:20:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.712+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c130260 msgr2=0x7f7c5c1265d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.712+0000 7f7c62a7a700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c130260 0x7f7c5c1265d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400c780 tx=0x7f7c5400ca90 comp rx=0 tx=0).stop 2023-09-30T22:20:14.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.712+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 shutdown_connections 2023-09-30T22:20:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.712+0000 7f7c62a7a700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5c130260 0x7f7c5c1265d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.712+0000 7f7c62a7a700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c5c12cf60 0x7f7c5c125b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.712+0000 7f7c62a7a700 1 --2- 172.21.15.89:0/2821856410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c5c12f8b0 0x7f7c5c126090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.712+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 >> 172.21.15.89:0/2821856410 conn(0x7f7c5c0a2aa0 msgr2=0x7f7c5c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.713+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 shutdown_connections 2023-09-30T22:20:14.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.713+0000 7f7c62a7a700 1 -- 172.21.15.89:0/2821856410 wait complete. 2023-09-30T22:20:14.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:14.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:14.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.772+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa08011fce0 con 0x7fa080131b40 2023-09-30T22:20:14.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.772+0000 7fa076ffd700 1 -- 172.21.15.89:0/1077959729 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa078017030 con 0x7fa080131b40 2023-09-30T22:20:14.772 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:14.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa080131b40 msgr2=0x7fa0801c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 --2- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa080131b40 0x7fa0801c6ac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c7a0 tx=0x7fa07800cab0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 shutdown_connections 2023-09-30T22:20:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 --2- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa080131b40 0x7fa0801c6ac0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 --2- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa08006a770 0x7fa0801c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 --2- 172.21.15.89:0/1077959729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa08006b080 0x7fa0801c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 >> 172.21.15.89:0/1077959729 conn(0x7fa080066f50 msgr2=0x7fa0800a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 shutdown_connections 2023-09-30T22:20:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.774+0000 7fa0882ba700 1 -- 172.21.15.89:0/1077959729 wait complete. 2023-09-30T22:20:14.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:14.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.829+0000 7f83b0ff9700 1 -- 172.21.15.89:0/637116226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f83bc121310 con 0x7f83bc09d8d0 2023-09-30T22:20:14.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.829+0000 7f83b2ffd700 1 -- 172.21.15.89:0/637116226 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83b802b070 con 0x7f83bc09d8d0 2023-09-30T22:20:14.830 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:14.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 -- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83bc09d8d0 msgr2=0x7f83bc11dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:14.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83bc09d8d0 0x7f83bc11dc70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000c00 tx=0x7f83b8004750 comp rx=0 tx=0).stop 2023-09-30T22:20:14.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 -- 172.21.15.89:0/637116226 shutdown_connections 2023-09-30T22:20:14.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f83bc09e610 0x7f83bc11e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83bc09d8d0 0x7f83bc11dc70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 --2- 172.21.15.89:0/637116226 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f83bc09e1e0 0x7f83bc11e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 -- 172.21.15.89:0/637116226 >> 172.21.15.89:0/637116226 conn(0x7f83bc0a2e20 msgr2=0x7f83bc130820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 -- 172.21.15.89:0/637116226 shutdown_connections 2023-09-30T22:20:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:14.831+0000 7f83b0ff9700 1 -- 172.21.15.89:0/637116226 wait complete. 2023-09-30T22:20:14.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:15.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:15.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:15.787 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:15.845 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f601def4700 1 Processor -- start 2023-09-30T22:20:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f601def4700 1 -- start start 2023-09-30T22:20:15.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f601def4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809d6f0 0x7f601809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f601def4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f601809e000 0x7f6018130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f601def4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809e410 0x7f6018130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f601def4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601809f520 con 0x7f601809e000 2023-09-30T22:20:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f601def4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601809f130 con 0x7f601809d6f0 2023-09-30T22:20:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f601def4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f601809e930 con 0x7f601809e410 2023-09-30T22:20:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f6017fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809e410 0x7f6018130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f6017fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809e410 0x7f6018130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41242/0 (socket says 172.21.15.89:41242) 2023-09-30T22:20:15.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f6017fff700 1 -- 172.21.15.89:0/2221092681 learned_addr learned my addr 172.21.15.89:0/2221092681 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:15.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.866+0000 7f60177fe700 1 --2- 172.21.15.89:0/2221092681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809d6f0 0x7f601809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f6017fff700 1 -- 172.21.15.89:0/2221092681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809d6f0 msgr2=0x7f601809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:15.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f6017fff700 1 --2- 172.21.15.89:0/2221092681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809d6f0 0x7f601809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f6017fff700 1 -- 172.21.15.89:0/2221092681 >> v2:172.21.15.89:3300/0 conn(0x7f601809e000 msgr2=0x7f6018130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:15.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f6017fff700 1 --2- 172.21.15.89:0/2221092681 >> v2:172.21.15.89:3300/0 conn(0x7f601809e000 0x7f6018130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f6017fff700 1 -- 172.21.15.89:0/2221092681 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601809b2d0 con 0x7f601809e410 2023-09-30T22:20:15.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f6017fff700 1 --2- 172.21.15.89:0/2221092681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809e410 0x7f6018130ba0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6010009e80 tx=0x7f601000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c72d419bbbc656a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:15.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f6be2b39700 1 Processor -- start 2023-09-30T22:20:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f60167fc700 1 -- 172.21.15.89:0/2221092681 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000e070 con 0x7f601809e410 2023-09-30T22:20:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f60167fc700 1 -- 172.21.15.89:0/2221092681 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601000b8d0 con 0x7f601809e410 2023-09-30T22:20:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f60167fc700 1 -- 172.21.15.89:0/2221092681 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000bc80 con 0x7f601809e410 2023-09-30T22:20:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.867+0000 7f6be2b39700 1 -- start start 2023-09-30T22:20:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6be2b39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc1338c0 0x7f6bdc133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6be2b39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6be2b39700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bdc06f720 con 0x7f6bdc09e390 2023-09-30T22:20:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6be2b39700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc070c80 con 0x7f6bdc1338c0 2023-09-30T22:20:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6be2b39700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06fce0 con 0x7f6bdc1341d0 2023-09-30T22:20:15.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6bdbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6be10d6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6bdc09e390 0x7f6bdc13a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39240/0 (socket says 172.21.15.89:39240) 2023-09-30T22:20:15.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6be10d6700 1 -- 172.21.15.89:0/647547516 learned_addr learned my addr 172.21.15.89:0/647547516 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:15.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 -- 172.21.15.89:0/2221092681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809e410 msgr2=0x7f6018130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 --2- 172.21.15.89:0/2221092681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809e410 0x7f6018130ba0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6010009e80 tx=0x7f601000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:15.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 -- 172.21.15.89:0/2221092681 shutdown_connections 2023-09-30T22:20:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 --2- 172.21.15.89:0/2221092681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809e410 0x7f6018130ba0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 --2- 172.21.15.89:0/2221092681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809d6f0 0x7f601809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 --2- 172.21.15.89:0/2221092681 >> v2:172.21.15.89:3300/0 conn(0x7f601809e000 0x7f6018130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 -- 172.21.15.89:0/2221092681 >> 172.21.15.89:0/2221092681 conn(0x7f6018066f50 msgr2=0x7f601809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6bdb7fe700 1 -- 172.21.15.89:0/647547516 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994817407 0 0) 0x7f6bdc06f720 con 0x7f6bdc09e390 2023-09-30T22:20:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6bdb7fe700 1 -- 172.21.15.89:0/647547516 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc80035b0 con 0x7f6bdc09e390 2023-09-30T22:20:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 -- 172.21.15.89:0/2221092681 shutdown_connections 2023-09-30T22:20:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f601def4700 1 -- 172.21.15.89:0/2221092681 wait complete. 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6bdbfff700 1 -- 172.21.15.89:0/647547516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc1338c0 msgr2=0x7f6bdc133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6bdbfff700 1 --2- 172.21.15.89:0/647547516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc1338c0 0x7f6bdc133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.868+0000 7f6bdbfff700 1 -- 172.21.15.89:0/647547516 >> v1:172.21.15.89:6789/0 conn(0x7f6bdc09e390 legacy=0x7f6bdc13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.869+0000 7f6bdbfff700 1 -- 172.21.15.89:0/647547516 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc1336a0 con 0x7f6bdc1341d0 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.869+0000 7f6bdbfff700 1 --2- 172.21.15.89:0/647547516 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc09de50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009870 tx=0x7f6bd4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9660857f5c393b26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.869+0000 7f6bdb7fe700 1 -- 172.21.15.89:0/647547516 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4011830 con 0x7f6bdc1341d0 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.869+0000 7f6bdb7fe700 1 -- 172.21.15.89:0/647547516 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4011990 con 0x7f6bdc1341d0 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.869+0000 7f6bdb7fe700 1 -- 172.21.15.89:0/647547516 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4011d20 con 0x7f6bdc1341d0 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f6be2b39700 1 -- 172.21.15.89:0/647547516 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 msgr2=0x7f6bdc09de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f6be2b39700 1 --2- 172.21.15.89:0/647547516 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc09de50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009870 tx=0x7f6bd4009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:15.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f601def4700 1 Processor -- start 2023-09-30T22:20:15.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f6be2b39700 1 -- 172.21.15.89:0/647547516 shutdown_connections 2023-09-30T22:20:15.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f6be2b39700 1 --2- 172.21.15.89:0/647547516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc1338c0 0x7f6bdc133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f6be2b39700 1 --2- 172.21.15.89:0/647547516 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc09de50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f6be2b39700 1 -- 172.21.15.89:0/647547516 >> 172.21.15.89:0/647547516 conn(0x7f6bdc0a2aa0 msgr2=0x7f6bdc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f6be2b39700 1 -- 172.21.15.89:0/647547516 shutdown_connections 2023-09-30T22:20:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f6be2b39700 1 -- 172.21.15.89:0/647547516 wait complete. 2023-09-30T22:20:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.870+0000 7f601def4700 1 -- start start 2023-09-30T22:20:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6be2b39700 1 Processor -- start 2023-09-30T22:20:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6be2b39700 1 -- start start 2023-09-30T22:20:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f601def4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809d6f0 0x7f60181370b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f601def4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601809e000 0x7f60181375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f601def4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809e410 0x7f60181c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f601def4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6018072410 con 0x7f601809e000 2023-09-30T22:20:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f601def4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6018119d90 con 0x7f601809e410 2023-09-30T22:20:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f601def4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6018136b80 con 0x7f601809d6f0 2023-09-30T22:20:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601809e000 0x7f60181375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601809e000 0x7f60181375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49974/0 (socket says 172.21.15.89:49974) 2023-09-30T22:20:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 -- 172.21.15.89:0/502547933 learned_addr learned my addr 172.21.15.89:0/502547933 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f60177fe700 1 --2- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809d6f0 0x7f60181370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be2b39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc09e390 0x7f6bdc11f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be2b39700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc1338c0 0x7f6bdc11fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 -- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809d6f0 msgr2=0x7f60181370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 --2- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809d6f0 0x7f60181370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 -- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809e410 msgr2=0x7f60181c8100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 --2- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809e410 0x7f60181c8100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 -- 172.21.15.89:0/502547933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6010009b30 con 0x7f601809e000 2023-09-30T22:20:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.871+0000 7f6016ffd700 1 --2- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601809e000 0x7f60181375f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f600c00b500 tx=0x7f600c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f5ffffff700 1 -- 172.21.15.89:0/502547933 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c011830 con 0x7f601809e000 2023-09-30T22:20:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f5ffffff700 1 -- 172.21.15.89:0/502547933 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c011990 con 0x7f601809e000 2023-09-30T22:20:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f5ffffff700 1 -- 172.21.15.89:0/502547933 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c011d40 con 0x7f601809e000 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be2b39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc1201f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be2b39700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06f960 con 0x7f6bdc1338c0 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be2b39700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1212e0 con 0x7f6bdc09e390 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be2b39700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc122c50 con 0x7f6bdc1341d0 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be10d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc1201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be10d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc1201f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41260/0 (socket says 172.21.15.89:41260) 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be10d6700 1 -- 172.21.15.89:0/4096797962 learned_addr learned my addr 172.21.15.89:0/4096797962 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.872+0000 7f6be08d5700 1 --2- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc09e390 0x7f6bdc11f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.873+0000 7f6be08d5700 1 -- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 msgr2=0x7f6bdc1201f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.873+0000 7f6be08d5700 1 --2- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc1201f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.873+0000 7f6be08d5700 1 -- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc1338c0 msgr2=0x7f6bdc11fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.873+0000 7f6be08d5700 1 --2- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc1338c0 0x7f6bdc11fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.873+0000 7f6be08d5700 1 -- 172.21.15.89:0/4096797962 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40094a0 con 0x7f6bdc09e390 2023-09-30T22:20:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.873+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6018130e60 con 0x7f601809e000 2023-09-30T22:20:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.873+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6018138330 con 0x7f601809e000 2023-09-30T22:20:15.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f5ffffff700 1 -- 172.21.15.89:0/502547933 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f600c01a400 con 0x7f601809e000 2023-09-30T22:20:15.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f6be08d5700 1 --2- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc09e390 0x7f6bdc11f770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00bd70 tx=0x7f6bcc00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:15.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6018074ea0 con 0x7f601809e000 2023-09-30T22:20:15.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f5ffffff700 1 -- 172.21.15.89:0/502547933 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f600c03f6a0 con 0x7f601809e000 2023-09-30T22:20:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f6bd97fa700 1 -- 172.21.15.89:0/4096797962 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00cd50 con 0x7f6bdc09e390 2023-09-30T22:20:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f6bd97fa700 1 -- 172.21.15.89:0/4096797962 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc014440 con 0x7f6bdc09e390 2023-09-30T22:20:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc1232d0 con 0x7f6bdc09e390 2023-09-30T22:20:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f6bd97fa700 1 -- 172.21.15.89:0/4096797962 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0147f0 con 0x7f6bdc09e390 2023-09-30T22:20:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.874+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc1220e0 con 0x7f6bdc09e390 2023-09-30T22:20:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.875+0000 7f6bd97fa700 1 -- 172.21.15.89:0/4096797962 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6bcc01d3b0 con 0x7f6bdc09e390 2023-09-30T22:20:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.875+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc074ea0 con 0x7f6bdc09e390 2023-09-30T22:20:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.875+0000 7f6bd97fa700 1 -- 172.21.15.89:0/4096797962 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6bcc006e40 con 0x7f6bdc09e390 2023-09-30T22:20:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.880+0000 7f5ffffff700 1 -- 172.21.15.89:0/502547933 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f600c015020 con 0x7f601809e000 2023-09-30T22:20:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.880+0000 7f6bd97fa700 1 -- 172.21.15.89:0/4096797962 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bcc01c360 con 0x7f6bdc09e390 2023-09-30T22:20:15.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f24627c1700 1 Processor -- start 2023-09-30T22:20:15.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f24627c1700 1 -- start start 2023-09-30T22:20:15.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f24627c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09e1d0 0x7f245c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f24627c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f24627c1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245c09f130 con 0x7f245c09d6f0 2023-09-30T22:20:15.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f24627c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245c09f520 con 0x7f245c09e1d0 2023-09-30T22:20:15.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f24627c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f245c09e930 con 0x7f245c135e50 2023-09-30T22:20:15.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09e1d0 0x7f245c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09e1d0 0x7f245c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38206/0 (socket says 172.21.15.89:38206) 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.946+0000 7f245b7fe700 1 -- 172.21.15.89:0/1792105986 learned_addr learned my addr 172.21.15.89:0/1792105986 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f2460d5e700 1 --2- 172.21.15.89:0/1792105986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245affd700 1 -- 172.21.15.89:0/1792105986 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055820707 0 0) 0x7f245c09f130 con 0x7f245c09d6f0 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245affd700 1 -- 172.21.15.89:0/1792105986 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24440035b0 con 0x7f245c09d6f0 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245b7fe700 1 -- 172.21.15.89:0/1792105986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 msgr2=0x7f245c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245b7fe700 1 --2- 172.21.15.89:0/1792105986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245b7fe700 1 -- 172.21.15.89:0/1792105986 >> v1:172.21.15.89:6789/0 conn(0x7f245c09d6f0 legacy=0x7f245c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245b7fe700 1 -- 172.21.15.89:0/1792105986 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c09b2d0 con 0x7f245c09e1d0 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245b7fe700 1 --2- 172.21.15.89:0/1792105986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09e1d0 0x7f245c135910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f245000bbc0 tx=0x7f245000bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53ff21a155f6b9f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:15.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245affd700 1 -- 172.21.15.89:0/1792105986 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245000dc70 con 0x7f245c09e1d0 2023-09-30T22:20:15.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245affd700 1 -- 172.21.15.89:0/1792105986 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245000ddd0 con 0x7f245c09e1d0 2023-09-30T22:20:15.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.947+0000 7f245affd700 1 -- 172.21.15.89:0/1792105986 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450012990 con 0x7f245c09e1d0 2023-09-30T22:20:15.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.948+0000 7f24627c1700 1 -- 172.21.15.89:0/1792105986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09e1d0 msgr2=0x7f245c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.948+0000 7f24627c1700 1 --2- 172.21.15.89:0/1792105986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09e1d0 0x7f245c135910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f245000bbc0 tx=0x7f245000bed0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.948+0000 7f24627c1700 1 -- 172.21.15.89:0/1792105986 shutdown_connections 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.948+0000 7f24627c1700 1 --2- 172.21.15.89:0/1792105986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.948+0000 7f24627c1700 1 --2- 172.21.15.89:0/1792105986 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09e1d0 0x7f245c135910 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.948+0000 7f24627c1700 1 -- 172.21.15.89:0/1792105986 >> 172.21.15.89:0/1792105986 conn(0x7f245c066f50 msgr2=0x7f245c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.948+0000 7f24627c1700 1 -- 172.21.15.89:0/1792105986 shutdown_connections 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.948+0000 7f24627c1700 1 -- 172.21.15.89:0/1792105986 wait complete. 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.949+0000 7f24627c1700 1 Processor -- start 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.949+0000 7f24627c1700 1 -- start start 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f24627c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09d6f0 0x7f245c1196e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f24627c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245c09e1d0 0x7f245c119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f24627c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c11a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f24627c1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c11b9d0 con 0x7f245c09e1d0 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f24627c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245c11aba0 con 0x7f245c09d6f0 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f24627c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f245c11b250 con 0x7f245c135e50 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f245bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09d6f0 0x7f245c1196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f245bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09d6f0 0x7f245c1196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38222/0 (socket says 172.21.15.89:38222) 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f245bfff700 1 -- 172.21.15.89:0/817553899 learned_addr learned my addr 172.21.15.89:0/817553899 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f2460d5e700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f245b7fe700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245c09e1d0 0x7f245c119c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f2460d5e700 1 -- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09d6f0 msgr2=0x7f245c1196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f2460d5e700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09d6f0 0x7f245c1196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f2460d5e700 1 -- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245c09e1d0 msgr2=0x7f245c119c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f2460d5e700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245c09e1d0 0x7f245c119c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f2460d5e700 1 -- 172.21.15.89:0/817553899 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245000b7f0 con 0x7f245c135e50 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f245bfff700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09d6f0 0x7f245c1196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f245b7fe700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245c09e1d0 0x7f245c119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.950+0000 7f2460d5e700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c11a160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f245400efc0 tx=0x7f245400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.951+0000 7f2458ff9700 1 -- 172.21.15.89:0/817553899 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245400cd40 con 0x7f245c135e50 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.951+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c11c790 con 0x7f245c135e50 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.951+0000 7f2458ff9700 1 -- 172.21.15.89:0/817553899 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245400f040 con 0x7f245c135e50 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.951+0000 7f2458ff9700 1 -- 172.21.15.89:0/817553899 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24540146c0 con 0x7f245c135e50 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.951+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c0a21e0 con 0x7f245c135e50 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.952+0000 7f2458ff9700 1 -- 172.21.15.89:0/817553899 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2454014820 con 0x7f245c135e50 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.952+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c11c980 con 0x7f245c135e50 2023-09-30T22:20:15.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.952+0000 7f2458ff9700 1 -- 172.21.15.89:0/817553899 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2454040280 con 0x7f245c135e50 2023-09-30T22:20:15.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.957+0000 7f2458ff9700 1 -- 172.21.15.89:0/817553899 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f245c11c980 con 0x7f245c135e50 2023-09-30T22:20:15.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f5074dcc700 1 Processor -- start 2023-09-30T22:20:15.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f5074dcc700 1 -- start start 2023-09-30T22:20:15.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f5074dcc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009d6f0 0x7f507009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f5074dcc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f507009e000 0x7f5070130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f5074dcc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507009f130 con 0x7f507009e410 2023-09-30T22:20:15.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f5074dcc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507009f520 con 0x7f507009e000 2023-09-30T22:20:15.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f5074dcc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507009e930 con 0x7f507009d6f0 2023-09-30T22:20:15.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f506dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f507009e000 0x7f5070130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f506dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f507009e000 0x7f5070130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50004/0 (socket says 172.21.15.89:50004) 2023-09-30T22:20:15.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.977+0000 7f506dd9b700 1 -- 172.21.15.89:0/278356128 learned_addr learned my addr 172.21.15.89:0/278356128 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:15.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.978+0000 7f506dd9b700 1 -- 172.21.15.89:0/278356128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009d6f0 msgr2=0x7f507009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:15.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.978+0000 7f506dd9b700 1 --2- 172.21.15.89:0/278356128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009d6f0 0x7f507009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.978+0000 7f506dd9b700 1 -- 172.21.15.89:0/278356128 >> v1:172.21.15.89:6789/0 conn(0x7f507009e410 legacy=0x7f5070130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.978+0000 7f506dd9b700 1 -- 172.21.15.89:0/278356128 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507009b2d0 con 0x7f507009e000 2023-09-30T22:20:15.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.978+0000 7f506dd9b700 1 --2- 172.21.15.89:0/278356128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 0x7f5070130660 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f506400bb90 tx=0x7f506400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119ebbce51fd48eb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:15.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.978+0000 7f506d59a700 1 -- 172.21.15.89:0/278356128 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506400dd50 con 0x7f507009e000 2023-09-30T22:20:15.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.978+0000 7f506d59a700 1 -- 172.21.15.89:0/278356128 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5064010470 con 0x7f507009e000 2023-09-30T22:20:15.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.978+0000 7f506d59a700 1 -- 172.21.15.89:0/278356128 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064010800 con 0x7f507009e000 2023-09-30T22:20:15.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.979+0000 7f5074dcc700 1 -- 172.21.15.89:0/278356128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 msgr2=0x7f5070130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.979+0000 7f5074dcc700 1 --2- 172.21.15.89:0/278356128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 0x7f5070130660 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f506400bb90 tx=0x7f506400bea0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.979+0000 7f5074dcc700 1 -- 172.21.15.89:0/278356128 shutdown_connections 2023-09-30T22:20:15.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.979+0000 7f5074dcc700 1 --2- 172.21.15.89:0/278356128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009d6f0 0x7f507009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.979+0000 7f5074dcc700 1 --2- 172.21.15.89:0/278356128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 0x7f5070130660 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.979+0000 7f5074dcc700 1 -- 172.21.15.89:0/278356128 >> 172.21.15.89:0/278356128 conn(0x7f5070066f50 msgr2=0x7f507009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.979+0000 7f5074dcc700 1 -- 172.21.15.89:0/278356128 shutdown_connections 2023-09-30T22:20:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.979+0000 7f5074dcc700 1 -- 172.21.15.89:0/278356128 wait complete. 2023-09-30T22:20:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.980+0000 7f5074dcc700 1 Processor -- start 2023-09-30T22:20:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.980+0000 7f5074dcc700 1 -- start start 2023-09-30T22:20:15.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.980+0000 7f5074dcc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f507009d6f0 0x7f50701c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.980+0000 7f5074dcc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 0x7f50701c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f5074dcc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009e410 0x7f50701c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:15.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f5074dcc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50701246e0 con 0x7f507009e000 2023-09-30T22:20:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f5074dcc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5070124d30 con 0x7f507009e410 2023-09-30T22:20:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f5074dcc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50701265f0 con 0x7f507009d6f0 2023-09-30T22:20:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f507009d6f0 0x7f50701c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506ed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009e410 0x7f50701c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506ed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009e410 0x7f50701c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38238/0 (socket says 172.21.15.89:38238) 2023-09-30T22:20:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506ed9d700 1 -- 172.21.15.89:0/4208064482 learned_addr learned my addr 172.21.15.89:0/4208064482 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:15.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506e59c700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f507009d6f0 0x7f50701c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41290/0 (socket says 172.21.15.89:41290) 2023-09-30T22:20:15.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506dd9b700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 0x7f50701c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:15.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506e59c700 1 -- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009e410 msgr2=0x7f50701c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506e59c700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009e410 0x7f50701c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506e59c700 1 -- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 msgr2=0x7f50701c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:15.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506e59c700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 0x7f50701c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:15.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506e59c700 1 -- 172.21.15.89:0/4208064482 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506400b7f0 con 0x7f507009d6f0 2023-09-30T22:20:15.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f506e59c700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f507009d6f0 0x7f50701c85c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f506000b480 tx=0x7f506000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:15.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.981+0000 7f505effd700 1 -- 172.21.15.89:0/4208064482 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060011830 con 0x7f507009d6f0 2023-09-30T22:20:15.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.982+0000 7f505effd700 1 -- 172.21.15.89:0/4208064482 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5060011990 con 0x7f507009d6f0 2023-09-30T22:20:15.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.982+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50700a1f00 con 0x7f507009d6f0 2023-09-30T22:20:15.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.982+0000 7f505effd700 1 -- 172.21.15.89:0/4208064482 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060011d40 con 0x7f507009d6f0 2023-09-30T22:20:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.982+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5070124ed0 con 0x7f507009d6f0 2023-09-30T22:20:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.983+0000 7f505effd700 1 -- 172.21.15.89:0/4208064482 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f506001a400 con 0x7f507009d6f0 2023-09-30T22:20:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.983+0000 7f505effd700 1 -- 172.21.15.89:0/4208064482 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f506003f590 con 0x7f507009d6f0 2023-09-30T22:20:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.983+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5070126170 con 0x7f507009d6f0 2023-09-30T22:20:15.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:15.992+0000 7f505effd700 1 -- 172.21.15.89:0/4208064482 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5070126170 con 0x7f507009d6f0 2023-09-30T22:20:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.079+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6bdc06b9e0 con 0x7f6bdc09e390 2023-09-30T22:20:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.079+0000 7f6bd97fa700 1 -- 172.21.15.89:0/4096797962 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bcc019070 con 0x7f6bdc09e390 2023-09-30T22:20:16.079 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.082+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc09e390 msgr2=0x7f6bdc11f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.082+0000 7f6be2b39700 1 --2- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc09e390 0x7f6bdc11f770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00bd70 tx=0x7f6bcc00be50 comp rx=0 tx=0).stop 2023-09-30T22:20:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.083+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 shutdown_connections 2023-09-30T22:20:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.083+0000 7f6be2b39700 1 --2- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bdc1341d0 0x7f6bdc1201f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.083+0000 7f6be2b39700 1 --2- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bdc09e390 0x7f6bdc11f770 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.083+0000 7f6be2b39700 1 --2- 172.21.15.89:0/4096797962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc1338c0 0x7f6bdc11fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.083+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 >> 172.21.15.89:0/4096797962 conn(0x7f6bdc0a2aa0 msgr2=0x7f6bdc09f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.083+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 shutdown_connections 2023-09-30T22:20:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.083+0000 7f6be2b39700 1 -- 172.21.15.89:0/4096797962 wait complete. 2023-09-30T22:20:16.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.118+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6018137b30 con 0x7f601809e000 2023-09-30T22:20:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.119+0000 7f5ffffff700 1 -- 172.21.15.89:0/502547933 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f600c029090 con 0x7f601809e000 2023-09-30T22:20:16.119 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:16.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601809e000 msgr2=0x7f60181375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:16.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 --2- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601809e000 0x7f60181375f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f600c00b500 tx=0x7f600c00b810 comp rx=0 tx=0).stop 2023-09-30T22:20:16.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 shutdown_connections 2023-09-30T22:20:16.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 --2- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f601809e410 0x7f60181c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 --2- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f601809d6f0 0x7f60181370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 --2- 172.21.15.89:0/502547933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601809e000 0x7f60181375f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 >> 172.21.15.89:0/502547933 conn(0x7f6018066f50 msgr2=0x7f601806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:16.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 shutdown_connections 2023-09-30T22:20:16.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.121+0000 7f601def4700 1 -- 172.21.15.89:0/502547933 wait complete. 2023-09-30T22:20:16.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:16.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.153+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f245c11cb60 con 0x7f245c135e50 2023-09-30T22:20:16.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.153+0000 7f2458ff9700 1 -- 172.21.15.89:0/817553899 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f245c11cb60 con 0x7f245c135e50 2023-09-30T22:20:16.154 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:16.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 msgr2=0x7f245c11a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:16.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c11a160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f245400efc0 tx=0x7f245400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 shutdown_connections 2023-09-30T22:20:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f245c135e50 0x7f245c11a160 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f245c09d6f0 0x7f245c1196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 --2- 172.21.15.89:0/817553899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f245c09e1d0 0x7f245c119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 >> 172.21.15.89:0/817553899 conn(0x7f245c066f50 msgr2=0x7f245c132780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 shutdown_connections 2023-09-30T22:20:16.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.156+0000 7f24627c1700 1 -- 172.21.15.89:0/817553899 wait complete. 2023-09-30T22:20:16.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.184+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5070126170 con 0x7f507009d6f0 2023-09-30T22:20:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.185+0000 7f505effd700 1 -- 172.21.15.89:0/4208064482 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5070126170 con 0x7f507009d6f0 2023-09-30T22:20:16.184 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f507009d6f0 msgr2=0x7f50701c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f507009d6f0 0x7f50701c85c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f506000b480 tx=0x7f506000b790 comp rx=0 tx=0).stop 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 shutdown_connections 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f507009e410 0x7f50701c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f507009d6f0 0x7f50701c85c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 --2- 172.21.15.89:0/4208064482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f507009e000 0x7f50701c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 >> 172.21.15.89:0/4208064482 conn(0x7f5070066f50 msgr2=0x7f5070071150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 shutdown_connections 2023-09-30T22:20:16.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:16.187+0000 7f5074dcc700 1 -- 172.21.15.89:0/4208064482 wait complete. 2023-09-30T22:20:16.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:17.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:17.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:17.169 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:17.199 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:17.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.239+0000 7f3e4fdc8700 1 Processor -- start 2023-09-30T22:20:17.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.239+0000 7f3e4fdc8700 1 -- start start 2023-09-30T22:20:17.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4fdc8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e4809e1d0 0x7f3e48135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4fdc8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e48135e50 0x7f3e48138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4fdc8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4809f130 con 0x7f3e4809d6f0 2023-09-30T22:20:17.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4fdc8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4809f520 con 0x7f3e4809e1d0 2023-09-30T22:20:17.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4fdc8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4809e930 con 0x7f3e48135e50 2023-09-30T22:20:17.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4d363700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e4809e1d0 0x7f3e48135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4d363700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e4809e1d0 0x7f3e48135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50028/0 (socket says 172.21.15.89:50028) 2023-09-30T22:20:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4d363700 1 -- 172.21.15.89:0/2130980552 learned_addr learned my addr 172.21.15.89:0/2130980552 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4e365700 1 --2- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e48135e50 0x7f3e48138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4d363700 1 -- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e48135e50 msgr2=0x7f3e48138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4d363700 1 --2- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e48135e50 0x7f3e48138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4d363700 1 -- 172.21.15.89:0/2130980552 >> v1:172.21.15.89:6789/0 conn(0x7f3e4809d6f0 legacy=0x7f3e4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4d363700 1 -- 172.21.15.89:0/2130980552 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4809b2d0 con 0x7f3e4809e1d0 2023-09-30T22:20:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.240+0000 7f3e4e365700 1 --2- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e48135e50 0x7f3e48138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.241+0000 7f3e4d363700 1 --2- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4809e1d0 0x7f3e48135910 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800bb90 tx=0x7f3e3800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8825b22aa5149da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.241+0000 7f3e4cb62700 1 -- 172.21.15.89:0/2130980552 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800dd30 con 0x7f3e4809e1d0 2023-09-30T22:20:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.241+0000 7f3e4cb62700 1 -- 172.21.15.89:0/2130980552 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3800de90 con 0x7f3e4809e1d0 2023-09-30T22:20:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.241+0000 7f3e4cb62700 1 -- 172.21.15.89:0/2130980552 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e38010690 con 0x7f3e4809e1d0 2023-09-30T22:20:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.242+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4809e1d0 msgr2=0x7f3e48135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.242+0000 7f3e4fdc8700 1 --2- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4809e1d0 0x7f3e48135910 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800bb90 tx=0x7f3e3800bea0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.242+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2130980552 shutdown_connections 2023-09-30T22:20:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.242+0000 7f3e4fdc8700 1 --2- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e48135e50 0x7f3e48138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.242+0000 7f3e4fdc8700 1 --2- 172.21.15.89:0/2130980552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e4809e1d0 0x7f3e48135910 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.242+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2130980552 >> 172.21.15.89:0/2130980552 conn(0x7f3e48066f50 msgr2=0x7f3e4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.242+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2130980552 shutdown_connections 2023-09-30T22:20:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.242+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2130980552 wait complete. 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.243+0000 7f3e4fdc8700 1 Processor -- start 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.243+0000 7f3e4fdc8700 1 -- start start 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.243+0000 7f3e4fdc8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4809d6f0 0x7f3e48120690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.243+0000 7f3e4fdc8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e4809e1d0 0x7f3e48120bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.243+0000 7f3e4fdc8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e48135e50 0x7f3e481c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.243+0000 7f3e4fdc8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48122870 con 0x7f3e48135e50 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4fdc8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e481234b0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4fdc8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48134600 con 0x7f3e4809e1d0 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4809d6f0 0x7f3e48120690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4809d6f0 0x7f3e48120690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38254/0 (socket says 172.21.15.89:38254) 2023-09-30T22:20:17.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 -- 172.21.15.89:0/2783294516 learned_addr learned my addr 172.21.15.89:0/2783294516 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4e365700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e48135e50 0x7f3e481c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 -- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e4809e1d0 msgr2=0x7f3e48120bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e4809e1d0 0x7f3e48120bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 -- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e48135e50 msgr2=0x7f3e481c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e48135e50 0x7f3e481c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 -- 172.21.15.89:0/2783294516 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40009a70 con 0x7f3e4809d6f0 2023-09-30T22:20:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4e365700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e48135e50 0x7f3e481c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4db64700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4809d6f0 0x7f3e48120690 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c4b0 tx=0x7f3e4400c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e3e7fc700 1 -- 172.21.15.89:0/2783294516 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440052d0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e3e7fc700 1 -- 172.21.15.89:0/2783294516 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e44005430 con 0x7f3e4809d6f0 2023-09-30T22:20:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3800b7f0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.244+0000 7f3e3e7fc700 1 -- 172.21.15.89:0/2783294516 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440038a0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.245+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e48121110 con 0x7f3e4809d6f0 2023-09-30T22:20:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.245+0000 7f3e3e7fc700 1 -- 172.21.15.89:0/2783294516 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3e44003a00 con 0x7f3e4809d6f0 2023-09-30T22:20:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.246+0000 7f3e3e7fc700 1 -- 172.21.15.89:0/2783294516 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3e4403eac0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.246+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e481235f0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.255+0000 7f3e3e7fc700 1 -- 172.21.15.89:0/2783294516 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3e481235f0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.280+0000 7fc2ed1bd700 1 Processor -- start 2023-09-30T22:20:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.280+0000 7fc2ed1bd700 1 -- start start 2023-09-30T22:20:17.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2ed1bd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc2e806e570 0x7fc2e806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2ed1bd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e809eca0 0x7fc2e806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2ed1bd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8134a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2ed1bd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e8070ee0 con 0x7fc2e806e570 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2ed1bd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e8071480 con 0x7fc2e809eca0 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2ed1bd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e806f3c0 con 0x7fc2e8134610 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e759e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e759e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8134a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41314/0 (socket says 172.21.15.89:41314) 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e759e700 1 -- 172.21.15.89:0/1195528357 learned_addr learned my addr 172.21.15.89:0/1195528357 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e659c700 1 --2- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e809eca0 0x7fc2e806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e6d9d700 1 --2- 172.21.15.89:0/1195528357 >> v2:172.21.15.89:3300/0 conn(0x7fc2e806e570 0x7fc2e806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e659c700 1 -- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 msgr2=0x7fc2e8134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e659c700 1 --2- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8134a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e659c700 1 -- 172.21.15.89:0/1195528357 >> v2:172.21.15.89:3300/0 conn(0x7fc2e806e570 msgr2=0x7fc2e806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e659c700 1 --2- 172.21.15.89:0/1195528357 >> v2:172.21.15.89:3300/0 conn(0x7fc2e806e570 0x7fc2e806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e659c700 1 -- 172.21.15.89:0/1195528357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e8071b90 con 0x7fc2e809eca0 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.281+0000 7fc2e759e700 1 --2- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2e659c700 1 --2- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e809eca0 0x7fc2e806ee80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009840 tx=0x7fc2dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ef88ea3e5163be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2e5d9b700 1 -- 172.21.15.89:0/1195528357 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc011830 con 0x7fc2e809eca0 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2e5d9b700 1 -- 172.21.15.89:0/1195528357 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc011990 con 0x7fc2e809eca0 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2e5d9b700 1 -- 172.21.15.89:0/1195528357 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc011d20 con 0x7fc2e809eca0 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e809eca0 msgr2=0x7fc2e806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2ed1bd700 1 --2- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e809eca0 0x7fc2e806ee80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009840 tx=0x7fc2dc009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/1195528357 shutdown_connections 2023-09-30T22:20:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2ed1bd700 1 --2- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2ed1bd700 1 --2- 172.21.15.89:0/1195528357 >> v2:172.21.15.89:3300/0 conn(0x7fc2e806e570 0x7fc2e806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.282+0000 7fc2ed1bd700 1 --2- 172.21.15.89:0/1195528357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e809eca0 0x7fc2e806ee80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.283+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/1195528357 >> 172.21.15.89:0/1195528357 conn(0x7fc2e80a2aa0 msgr2=0x7fc2e80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:17.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.283+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/1195528357 shutdown_connections 2023-09-30T22:20:17.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.283+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/1195528357 wait complete. 2023-09-30T22:20:17.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.284+0000 7fc2ed1bd700 1 Processor -- start 2023-09-30T22:20:17.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.284+0000 7fc2ed1bd700 1 -- start start 2023-09-30T22:20:17.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2ed1bd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2e806e570 0x7fc2e81214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2ed1bd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2ed1bd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e8121f60 0x7fc2e81d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2ed1bd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e8134360 con 0x7fc2e806e570 2023-09-30T22:20:17.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2ed1bd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e81231a0 con 0x7fc2e8121f60 2023-09-30T22:20:17.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2ed1bd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e8123dd0 con 0x7fc2e8134610 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41316/0 (socket says 172.21.15.89:41316) 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e659c700 1 -- 172.21.15.89:0/2351594356 learned_addr learned my addr 172.21.15.89:0/2351594356 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e759e700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e8121f60 0x7fc2e81d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e6d9d700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2e806e570 0x7fc2e81214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e759e700 1 -- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 msgr2=0x7fc2e8121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e759e700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e759e700 1 -- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2e806e570 msgr2=0x7fc2e81214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.285+0000 7fc2e759e700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2e806e570 0x7fc2e81214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.286+0000 7fc2e759e700 1 -- 172.21.15.89:0/2351594356 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0009b30 con 0x7fc2e8121f60 2023-09-30T22:20:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.286+0000 7fc2e659c700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.286+0000 7fc2e759e700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e8121f60 0x7fc2e81d2e00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009e50 tx=0x7fc2e000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.286+0000 7fc2cf7fe700 1 -- 172.21.15.89:0/2351594356 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0018070 con 0x7fc2e8121f60 2023-09-30T22:20:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.286+0000 7fc2cf7fe700 1 -- 172.21.15.89:0/2351594356 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e000f040 con 0x7fc2e8121f60 2023-09-30T22:20:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.286+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc0094a0 con 0x7fc2e8121f60 2023-09-30T22:20:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.286+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e8124220 con 0x7fc2e8121f60 2023-09-30T22:20:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.286+0000 7fc2cf7fe700 1 -- 172.21.15.89:0/2351594356 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0003c40 con 0x7fc2e8121f60 2023-09-30T22:20:17.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.287+0000 7fc2cf7fe700 1 -- 172.21.15.89:0/2351594356 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc2e000c880 con 0x7fc2e8121f60 2023-09-30T22:20:17.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.287+0000 7fc2cf7fe700 1 -- 172.21.15.89:0/2351594356 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc2e003f690 con 0x7fc2e8121f60 2023-09-30T22:20:17.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.287+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c80052e0 con 0x7fc2e8121f60 2023-09-30T22:20:17.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.296+0000 7fc2cf7fe700 1 -- 172.21.15.89:0/2351594356 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2e0012340 con 0x7fc2e8121f60 2023-09-30T22:20:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.326+0000 7f66fbd79700 1 Processor -- start 2023-09-30T22:20:17.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.326+0000 7f66fbd79700 1 -- start start 2023-09-30T22:20:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66fbd79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f41316c0 0x7f66f4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66fbd79700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66f409d9f0 0x7f66f406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66fbd79700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f406acb0 0x7f66f406b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66fbd79700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f406b630 con 0x7f66f409d9f0 2023-09-30T22:20:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66fbd79700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f409e130 con 0x7f66f406acb0 2023-09-30T22:20:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66fbd79700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41312b0 con 0x7f66f41316c0 2023-09-30T22:20:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66f9314700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66f409d9f0 0x7f66f406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66f9314700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66f409d9f0 0x7f66f406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50060/0 (socket says 172.21.15.89:50060) 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66f9b15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f41316c0 0x7f66f4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66f9314700 1 -- 172.21.15.89:0/2422854038 learned_addr learned my addr 172.21.15.89:0/2422854038 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.327+0000 7f66fa316700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f406acb0 0x7f66f406b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f9314700 1 -- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f41316c0 msgr2=0x7f66f4131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f9314700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f41316c0 0x7f66f4131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f9314700 1 -- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f406acb0 msgr2=0x7f66f406b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f9314700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f406acb0 0x7f66f406b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f9314700 1 -- 172.21.15.89:0/2422854038 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f409f6e0 con 0x7f66f409d9f0 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f9b15700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f41316c0 0x7f66f4131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66fa316700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f406acb0 0x7f66f406b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f9314700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f409d9f0 0x7f66f406a770 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f66e40098f0 tx=0x7f66e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae17a3d36bad5567 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:17.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f8b13700 1 -- 172.21.15.89:0/2422854038 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011830 con 0x7f66f409d9f0 2023-09-30T22:20:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f8b13700 1 -- 172.21.15.89:0/2422854038 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4011990 con 0x7f66f409d9f0 2023-09-30T22:20:17.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.328+0000 7f66f8b13700 1 -- 172.21.15.89:0/2422854038 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011d20 con 0x7f66f409d9f0 2023-09-30T22:20:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 -- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f409d9f0 msgr2=0x7f66f406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f409d9f0 0x7f66f406a770 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f66e40098f0 tx=0x7f66e4009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 -- 172.21.15.89:0/2422854038 shutdown_connections 2023-09-30T22:20:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f406acb0 0x7f66f406b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f41316c0 0x7f66f4131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 --2- 172.21.15.89:0/2422854038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f409d9f0 0x7f66f406a770 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 -- 172.21.15.89:0/2422854038 >> 172.21.15.89:0/2422854038 conn(0x7f66f4066f50 msgr2=0x7f66f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 -- 172.21.15.89:0/2422854038 shutdown_connections 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.329+0000 7f66fbd79700 1 -- 172.21.15.89:0/2422854038 wait complete. 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.330+0000 7f66fbd79700 1 Processor -- start 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.330+0000 7f66fbd79700 1 -- start start 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66fbd79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f406acb0 0x7f66f4119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66fbd79700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f409d9f0 0x7f66f41199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66fbd79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f41316c0 0x7f66f4119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66fbd79700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f411a8a0 con 0x7f66f41316c0 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66fbd79700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f412b720 con 0x7f66f409d9f0 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66fbd79700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40a3540 con 0x7f66f406acb0 2023-09-30T22:20:17.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9b15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f406acb0 0x7f66f4119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9b15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f406acb0 0x7f66f4119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41334/0 (socket says 172.21.15.89:41334) 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9b15700 1 -- 172.21.15.89:0/2034232692 learned_addr learned my addr 172.21.15.89:0/2034232692 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9314700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f409d9f0 0x7f66f41199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66fa316700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f41316c0 0x7f66f4119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9b15700 1 -- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f409d9f0 msgr2=0x7f66f41199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9b15700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f409d9f0 0x7f66f41199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9b15700 1 -- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f41316c0 msgr2=0x7f66f4119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9b15700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f41316c0 0x7f66f4119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.331+0000 7f66f9b15700 1 -- 172.21.15.89:0/2034232692 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec00bc50 con 0x7f66f406acb0 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.332+0000 7f66f9b15700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f406acb0 0x7f66f4119480 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f66f000ec10 tx=0x7f66f000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:17.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.332+0000 7f66ea7fc700 1 -- 172.21.15.89:0/2034232692 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f000c850 con 0x7f66f406acb0 2023-09-30T22:20:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.332+0000 7f66ea7fc700 1 -- 172.21.15.89:0/2034232692 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f000c9b0 con 0x7f66f406acb0 2023-09-30T22:20:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.332+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f00094a0 con 0x7f66f406acb0 2023-09-30T22:20:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.332+0000 7f66ea7fc700 1 -- 172.21.15.89:0/2034232692 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f000cd20 con 0x7f66f406acb0 2023-09-30T22:20:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.332+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e4009550 con 0x7f66f406acb0 2023-09-30T22:20:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.333+0000 7f66ea7fc700 1 -- 172.21.15.89:0/2034232692 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66f0013470 con 0x7f66f406acb0 2023-09-30T22:20:17.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.333+0000 7f66ea7fc700 1 -- 172.21.15.89:0/2034232692 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f66f003f730 con 0x7f66f406acb0 2023-09-30T22:20:17.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.333+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f4074ea0 con 0x7f66f406acb0 2023-09-30T22:20:17.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.338+0000 7f66ea7fc700 1 -- 172.21.15.89:0/2034232692 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f66f0013720 con 0x7f66f406acb0 2023-09-30T22:20:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.363+0000 7f45701bc700 1 Processor -- start 2023-09-30T22:20:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.363+0000 7f45701bc700 1 -- start start 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f45701bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809e930 0x7f456809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f45701bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f456809f240 0x7f45681317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f45701bc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4568070940 con 0x7f456809f650 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f45701bc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456806fd60 con 0x7f456809f240 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f45701bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f456806fa80 con 0x7f456809e930 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f456d757700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f456809f240 0x7f45681317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f456d757700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f456809f240 0x7f45681317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50082/0 (socket says 172.21.15.89:50082) 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f456d757700 1 -- 172.21.15.89:0/1495565465 learned_addr learned my addr 172.21.15.89:0/1495565465 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.364+0000 7f456df58700 1 --2- 172.21.15.89:0/1495565465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809e930 0x7f456809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.365+0000 7f456d757700 1 -- 172.21.15.89:0/1495565465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809e930 msgr2=0x7f456809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.365+0000 7f456d757700 1 --2- 172.21.15.89:0/1495565465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809e930 0x7f456809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.365+0000 7f456d757700 1 -- 172.21.15.89:0/1495565465 >> v1:172.21.15.89:6789/0 conn(0x7f456809f650 legacy=0x7f4568131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.365+0000 7f456d757700 1 -- 172.21.15.89:0/1495565465 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456809fb60 con 0x7f456809f240 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.365+0000 7f456d757700 1 --2- 172.21.15.89:0/1495565465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f240 0x7f45681317f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f45580098f0 tx=0x7f4558009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d532b1d711abd853 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.365+0000 7f456cf56700 1 -- 172.21.15.89:0/1495565465 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800b8a0 con 0x7f456809f240 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.365+0000 7f456cf56700 1 -- 172.21.15.89:0/1495565465 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455800ba00 con 0x7f456809f240 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.365+0000 7f456cf56700 1 -- 172.21.15.89:0/1495565465 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800bd90 con 0x7f456809f240 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.366+0000 7f45701bc700 1 -- 172.21.15.89:0/1495565465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f240 msgr2=0x7f45681317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.366+0000 7f45701bc700 1 --2- 172.21.15.89:0/1495565465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f240 0x7f45681317f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f45580098f0 tx=0x7f4558009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.366+0000 7f45701bc700 1 -- 172.21.15.89:0/1495565465 shutdown_connections 2023-09-30T22:20:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.366+0000 7f45701bc700 1 --2- 172.21.15.89:0/1495565465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809e930 0x7f456809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.366+0000 7f45701bc700 1 --2- 172.21.15.89:0/1495565465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f240 0x7f45681317f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.366+0000 7f45701bc700 1 -- 172.21.15.89:0/1495565465 >> 172.21.15.89:0/1495565465 conn(0x7f45680a4210 msgr2=0x7f45680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.366+0000 7f45701bc700 1 -- 172.21.15.89:0/1495565465 shutdown_connections 2023-09-30T22:20:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.366+0000 7f45701bc700 1 -- 172.21.15.89:0/1495565465 wait complete. 2023-09-30T22:20:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.367+0000 7f45701bc700 1 Processor -- start 2023-09-30T22:20:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f45701bc700 1 -- start start 2023-09-30T22:20:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f45701bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f456809e930 0x7f45681a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f45701bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809f240 0x7f45681a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f45701bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f650 0x7f45681a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f45701bc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681a8f50 con 0x7f456809f650 2023-09-30T22:20:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f45701bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4568071d20 con 0x7f456809e930 2023-09-30T22:20:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f45701bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45680a35c0 con 0x7f456809f240 2023-09-30T22:20:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f456d757700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809f240 0x7f45681a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f456d757700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809f240 0x7f45681a61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41342/0 (socket says 172.21.15.89:41342) 2023-09-30T22:20:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f456d757700 1 -- 172.21.15.89:0/3293980180 learned_addr learned my addr 172.21.15.89:0/3293980180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.368+0000 7f456e759700 1 --2- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f650 0x7f45681a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f456e759700 1 -- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809f240 msgr2=0x7f45681a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f456e759700 1 --2- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809f240 0x7f45681a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f456e759700 1 -- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f456809e930 msgr2=0x7f45681a5ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f456e759700 1 --2- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f456809e930 0x7f45681a5ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f456e759700 1 -- 172.21.15.89:0/3293980180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558009550 con 0x7f456809f650 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f456e759700 1 --2- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f650 0x7f45681a6720 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f456000cb90 tx=0x7f456000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f455e7fc700 1 -- 172.21.15.89:0/3293980180 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560003c00 con 0x7f456809f650 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f455e7fc700 1 -- 172.21.15.89:0/3293980180 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4560003d60 con 0x7f456809f650 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45680a0190 con 0x7f456809f650 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f455e7fc700 1 -- 172.21.15.89:0/3293980180 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456000aa80 con 0x7f456809f650 2023-09-30T22:20:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.369+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45681a8b00 con 0x7f456809f650 2023-09-30T22:20:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.370+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45681a2b40 con 0x7f456809f650 2023-09-30T22:20:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.370+0000 7f455e7fc700 1 -- 172.21.15.89:0/3293980180 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f456000abe0 con 0x7f456809f650 2023-09-30T22:20:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.371+0000 7f455e7fc700 1 -- 172.21.15.89:0/3293980180 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f456003f5f0 con 0x7f456809f650 2023-09-30T22:20:17.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.379+0000 7f455e7fc700 1 -- 172.21.15.89:0/3293980180 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f45600198f0 con 0x7f456809f650 2023-09-30T22:20:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.457+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3e481235f0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.457+0000 7f3e3e7fc700 1 -- 172.21.15.89:0/2783294516 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e481235f0 con 0x7f3e4809d6f0 2023-09-30T22:20:17.458 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.459+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4809d6f0 msgr2=0x7f3e48120690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.459+0000 7f3e4fdc8700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4809d6f0 0x7f3e48120690 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c4b0 tx=0x7f3e4400c870 comp rx=0 tx=0).stop 2023-09-30T22:20:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.459+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 shutdown_connections 2023-09-30T22:20:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.459+0000 7f3e4fdc8700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e48135e50 0x7f3e481c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.460+0000 7f3e4fdc8700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e4809d6f0 0x7f3e48120690 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.460+0000 7f3e4fdc8700 1 --2- 172.21.15.89:0/2783294516 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e4809e1d0 0x7f3e48120bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.460+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 >> 172.21.15.89:0/2783294516 conn(0x7f3e48066f50 msgr2=0x7f3e48134d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.460+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 shutdown_connections 2023-09-30T22:20:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.460+0000 7f3e4fdc8700 1 -- 172.21.15.89:0/2783294516 wait complete. 2023-09-30T22:20:17.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:17.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.506+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc2c8005590 con 0x7fc2e8121f60 2023-09-30T22:20:17.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.506+0000 7fc2cf7fe700 1 -- 172.21.15.89:0/2351594356 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2e001de00 con 0x7fc2e8121f60 2023-09-30T22:20:17.506 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:17.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.509+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e8121f60 msgr2=0x7fc2e81d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.509+0000 7fc2ed1bd700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e8121f60 0x7fc2e81d2e00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009e50 tx=0x7fc2e000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.509+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 shutdown_connections 2023-09-30T22:20:17.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.509+0000 7fc2ed1bd700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2e8121f60 0x7fc2e81d2e00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.509+0000 7fc2ed1bd700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2e806e570 0x7fc2e81214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.509+0000 7fc2ed1bd700 1 --2- 172.21.15.89:0/2351594356 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc2e8134610 0x7fc2e8121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.509+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 >> 172.21.15.89:0/2351594356 conn(0x7fc2e80a2aa0 msgr2=0x7fc2e80a0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:17.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.509+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 shutdown_connections 2023-09-30T22:20:17.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.511+0000 7fc2ed1bd700 1 -- 172.21.15.89:0/2351594356 wait complete. 2023-09-30T22:20:17.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:17.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.533+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f66f409e980 con 0x7f66f406acb0 2023-09-30T22:20:17.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.533+0000 7f66ea7fc700 1 -- 172.21.15.89:0/2034232692 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66f001bb60 con 0x7f66f406acb0 2023-09-30T22:20:17.533 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:17.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f406acb0 msgr2=0x7f66f4119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f406acb0 0x7f66f4119480 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f66f000ec10 tx=0x7f66f000ef20 comp rx=0 tx=0).stop 2023-09-30T22:20:17.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 shutdown_connections 2023-09-30T22:20:17.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f41316c0 0x7f66f4119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f406acb0 0x7f66f4119480 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 --2- 172.21.15.89:0/2034232692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f409d9f0 0x7f66f41199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 >> 172.21.15.89:0/2034232692 conn(0x7f66f4066f50 msgr2=0x7f66f409bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:17.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 shutdown_connections 2023-09-30T22:20:17.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.535+0000 7f66fbd79700 1 -- 172.21.15.89:0/2034232692 wait complete. 2023-09-30T22:20:17.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.575+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f45681a9360 con 0x7f456809f650 2023-09-30T22:20:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.576+0000 7f455e7fc700 1 -- 172.21.15.89:0/3293980180 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4560011380 con 0x7f456809f650 2023-09-30T22:20:17.575 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f650 msgr2=0x7f45681a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 --2- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f650 0x7f45681a6720 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f456000cb90 tx=0x7f456000cea0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 shutdown_connections 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 --2- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f456809f650 0x7f45681a6720 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 --2- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f456809e930 0x7f45681a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 --2- 172.21.15.89:0/3293980180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f456809f240 0x7f45681a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 >> 172.21.15.89:0/3293980180 conn(0x7f45680a4210 msgr2=0x7f456809dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 shutdown_connections 2023-09-30T22:20:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:17.578+0000 7f45701bc700 1 -- 172.21.15.89:0/3293980180 wait complete. 2023-09-30T22:20:17.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:18.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:18.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:18.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:18.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.615+0000 7fd55d7a1700 1 Processor -- start 2023-09-30T22:20:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd55d7a1700 1 -- start start 2023-09-30T22:20:18.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd55d7a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5581338a0 0x7fd558133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd55d7a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd55d7a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55806ef30 con 0x7fd55809de50 2023-09-30T22:20:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd55d7a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55806fef0 con 0x7fd5581338a0 2023-09-30T22:20:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd55d7a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581334b0 con 0x7fd55809e930 2023-09-30T22:20:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd556ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5581338a0 0x7fd558133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd556ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5581338a0 0x7fd558133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50096/0 (socket says 172.21.15.89:50096) 2023-09-30T22:20:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd556ffd700 1 -- 172.21.15.89:0/3389535760 learned_addr learned my addr 172.21.15.89:0/3389535760 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd5577fe700 1 --2- 172.21.15.89:0/3389535760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.616+0000 7fd556ffd700 1 -- 172.21.15.89:0/3389535760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 msgr2=0x7fd55813b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.617+0000 7fd556ffd700 1 --2- 172.21.15.89:0/3389535760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55813b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.617+0000 7fd556ffd700 1 -- 172.21.15.89:0/3389535760 >> v1:172.21.15.89:6789/0 conn(0x7fd55809de50 legacy=0x7fd55809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.617+0000 7fd556ffd700 1 -- 172.21.15.89:0/3389535760 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55806fc30 con 0x7fd5581338a0 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.617+0000 7fd556ffd700 1 --2- 172.21.15.89:0/3389535760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5581338a0 0x7fd558133c70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd5480098f0 tx=0x7fd548009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2cfe9e1f4fd7a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.617+0000 7fd555ffb700 1 -- 172.21.15.89:0/3389535760 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548011830 con 0x7fd5581338a0 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.617+0000 7fd555ffb700 1 -- 172.21.15.89:0/3389535760 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548011990 con 0x7fd5581338a0 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.617+0000 7fd555ffb700 1 -- 172.21.15.89:0/3389535760 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548011d20 con 0x7fd5581338a0 2023-09-30T22:20:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.618+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3389535760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5581338a0 msgr2=0x7fd558133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.618+0000 7fd55d7a1700 1 --2- 172.21.15.89:0/3389535760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5581338a0 0x7fd558133c70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd5480098f0 tx=0x7fd548009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.618+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3389535760 shutdown_connections 2023-09-30T22:20:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.618+0000 7fd55d7a1700 1 --2- 172.21.15.89:0/3389535760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.618+0000 7fd55d7a1700 1 --2- 172.21.15.89:0/3389535760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5581338a0 0x7fd558133c70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.618+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3389535760 >> 172.21.15.89:0/3389535760 conn(0x7fd5580a2aa0 msgr2=0x7fd5580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.618+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3389535760 shutdown_connections 2023-09-30T22:20:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.618+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3389535760 wait complete. 2023-09-30T22:20:18.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.619+0000 7fd55d7a1700 1 Processor -- start 2023-09-30T22:20:18.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.619+0000 7fd55d7a1700 1 -- start start 2023-09-30T22:20:18.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd55d7a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd55809de50 0x7fd55811eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd55d7a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55811f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd55d7a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5581338a0 0x7fd55811f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd55d7a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd558121cd0 con 0x7fd55809de50 2023-09-30T22:20:18.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd55d7a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55812f6a0 con 0x7fd55809e930 2023-09-30T22:20:18.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd55d7a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581206e0 con 0x7fd5581338a0 2023-09-30T22:20:18.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd556ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd55809de50 0x7fd55811eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd556ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd55809de50 0x7fd55811eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50108/0 (socket says 172.21.15.89:50108) 2023-09-30T22:20:18.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd556ffd700 1 -- 172.21.15.89:0/3109368016 learned_addr learned my addr 172.21.15.89:0/3109368016 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:18.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd5567fc700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55811f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd556ffd700 1 -- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5581338a0 msgr2=0x7fd55811f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd556ffd700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5581338a0 0x7fd55811f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd556ffd700 1 -- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 msgr2=0x7fd55811f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd556ffd700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55811f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd556ffd700 1 -- 172.21.15.89:0/3109368016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd548009550 con 0x7fd55809de50 2023-09-30T22:20:18.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.620+0000 7fd5567fc700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55811f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:18.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.621+0000 7fd556ffd700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd55809de50 0x7fd55811eb10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd548004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:18.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.621+0000 7fd53f7fe700 1 -- 172.21.15.89:0/3109368016 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548004480 con 0x7fd55809de50 2023-09-30T22:20:18.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.621+0000 7fd53f7fe700 1 -- 172.21.15.89:0/3109368016 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54801f070 con 0x7fd55809de50 2023-09-30T22:20:18.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.621+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5581225f0 con 0x7fd55809de50 2023-09-30T22:20:18.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.621+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd558122310 con 0x7fd55809de50 2023-09-30T22:20:18.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.621+0000 7fd53f7fe700 1 -- 172.21.15.89:0/3109368016 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480038c0 con 0x7fd55809de50 2023-09-30T22:20:18.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.622+0000 7fd53f7fe700 1 -- 172.21.15.89:0/3109368016 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd548021030 con 0x7fd55809de50 2023-09-30T22:20:18.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.622+0000 7fd53f7fe700 1 -- 172.21.15.89:0/3109368016 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd5480409a0 con 0x7fd55809de50 2023-09-30T22:20:18.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.622+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5380052e0 con 0x7fd55809de50 2023-09-30T22:20:18.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.628+0000 7fd53f7fe700 1 -- 172.21.15.89:0/3109368016 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd548015080 con 0x7fd55809de50 2023-09-30T22:20:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.678+0000 7f2afaf74700 1 Processor -- start 2023-09-30T22:20:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.678+0000 7f2afaf74700 1 -- start start 2023-09-30T22:20:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2afaf74700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 0x7f2af409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2afaf74700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e000 0x7f2af41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2afaf74700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2af409e410 0x7f2af4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2afaf74700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af406b460 con 0x7f2af409e410 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2afaf74700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af409b120 con 0x7f2af409d6f0 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2afaf74700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2af409a7e0 con 0x7f2af409e000 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e000 0x7f2af41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e000 0x7f2af41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41352/0 (socket says 172.21.15.89:41352) 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af3fff700 1 -- 172.21.15.89:0/584079668 learned_addr learned my addr 172.21.15.89:0/584079668 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af9511700 1 --2- 172.21.15.89:0/584079668 >> v2:172.21.15.89:3300/0 conn(0x7f2af409e410 0x7f2af4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af8d10700 1 --2- 172.21.15.89:0/584079668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 0x7f2af409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af3fff700 1 -- 172.21.15.89:0/584079668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 msgr2=0x7f2af409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af3fff700 1 --2- 172.21.15.89:0/584079668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 0x7f2af409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af3fff700 1 -- 172.21.15.89:0/584079668 >> v2:172.21.15.89:3300/0 conn(0x7f2af409e410 msgr2=0x7f2af4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af3fff700 1 --2- 172.21.15.89:0/584079668 >> v2:172.21.15.89:3300/0 conn(0x7f2af409e410 0x7f2af4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af3fff700 1 -- 172.21.15.89:0/584079668 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af409a500 con 0x7f2af409e000 2023-09-30T22:20:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.679+0000 7f2af9511700 1 --2- 172.21.15.89:0/584079668 >> v2:172.21.15.89:3300/0 conn(0x7f2af409e410 0x7f2af4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.680+0000 7f2af3fff700 1 --2- 172.21.15.89:0/584079668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e000 0x7f2af41339f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009920 tx=0x7f2ae0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8d87d6036d7c6d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.680+0000 7f2af37fe700 1 -- 172.21.15.89:0/584079668 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0011830 con 0x7f2af409e000 2023-09-30T22:20:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.680+0000 7f2af37fe700 1 -- 172.21.15.89:0/584079668 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae0011990 con 0x7f2af409e000 2023-09-30T22:20:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.680+0000 7f2af37fe700 1 -- 172.21.15.89:0/584079668 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0011d20 con 0x7f2af409e000 2023-09-30T22:20:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 -- 172.21.15.89:0/584079668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e000 msgr2=0x7f2af41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 --2- 172.21.15.89:0/584079668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e000 0x7f2af41339f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009920 tx=0x7f2ae0009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 -- 172.21.15.89:0/584079668 shutdown_connections 2023-09-30T22:20:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 --2- 172.21.15.89:0/584079668 >> v2:172.21.15.89:3300/0 conn(0x7f2af409e410 0x7f2af4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 --2- 172.21.15.89:0/584079668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 0x7f2af409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 --2- 172.21.15.89:0/584079668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e000 0x7f2af41339f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 -- 172.21.15.89:0/584079668 >> 172.21.15.89:0/584079668 conn(0x7f2af4066f50 msgr2=0x7f2af409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 -- 172.21.15.89:0/584079668 shutdown_connections 2023-09-30T22:20:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.681+0000 7f2afaf74700 1 -- 172.21.15.89:0/584079668 wait complete. 2023-09-30T22:20:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.682+0000 7f2afaf74700 1 Processor -- start 2023-09-30T22:20:18.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.682+0000 7f2afaf74700 1 -- start start 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2afaf74700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 0x7f2af41a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2afaf74700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af409e000 0x7f2af41a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2afaf74700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e410 0x7f2af41a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2afaf74700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41a9650 con 0x7f2af409e000 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2afaf74700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41aa210 con 0x7f2af409d6f0 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2afaf74700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41a8c00 con 0x7f2af409e410 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2af9511700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e410 0x7f2af41a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2af9511700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e410 0x7f2af41a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41360/0 (socket says 172.21.15.89:41360) 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2af9511700 1 -- 172.21.15.89:0/366478430 learned_addr learned my addr 172.21.15.89:0/366478430 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2af8d10700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 0x7f2af41a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.683+0000 7f2af3fff700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af409e000 0x7f2af41a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af9511700 1 -- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 msgr2=0x7f2af41a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af9511700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 0x7f2af41a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af9511700 1 -- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af409e000 msgr2=0x7f2af41a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af9511700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af409e000 0x7f2af41a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af9511700 1 -- 172.21.15.89:0/366478430 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec009a70 con 0x7f2af409e410 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af9511700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e410 0x7f2af41a67b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009f00 tx=0x7f2aec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af17fa700 1 -- 172.21.15.89:0/366478430 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec009d70 con 0x7f2af409e410 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af17fa700 1 -- 172.21.15.89:0/366478430 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec00f040 con 0x7f2af409e410 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae0009550 con 0x7f2af409e410 2023-09-30T22:20:18.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2af17fa700 1 -- 172.21.15.89:0/366478430 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec014480 con 0x7f2af409e410 2023-09-30T22:20:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.684+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af41a79e0 con 0x7f2af409e410 2023-09-30T22:20:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.685+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af40a31a0 con 0x7f2af409e410 2023-09-30T22:20:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.686+0000 7f2af17fa700 1 -- 172.21.15.89:0/366478430 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2aec0145e0 con 0x7f2af409e410 2023-09-30T22:20:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.686+0000 7f2af17fa700 1 -- 172.21.15.89:0/366478430 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2aec040490 con 0x7f2af409e410 2023-09-30T22:20:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.695+0000 7f2af17fa700 1 -- 172.21.15.89:0/366478430 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2aec012930 con 0x7f2af409e410 2023-09-30T22:20:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.705+0000 7f5ce66af700 1 Processor -- start 2023-09-30T22:20:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.705+0000 7f5ce66af700 1 -- start start 2023-09-30T22:20:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.705+0000 7f5ce66af700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ce012cf60 0x7f5ce006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.705+0000 7f5ce66af700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012f8b0 0x7f5ce012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.705+0000 7f5ce66af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce0130260 0x7f5ce0136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.705+0000 7f5ce66af700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce012d330 con 0x7f5ce012cf60 2023-09-30T22:20:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.705+0000 7f5ce66af700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce012cd90 con 0x7f5ce012f8b0 2023-09-30T22:20:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.705+0000 7f5ce66af700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce01306d0 con 0x7f5ce0130260 2023-09-30T22:20:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012f8b0 0x7f5ce012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012f8b0 0x7f5ce012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38346/0 (socket says 172.21.15.89:38346) 2023-09-30T22:20:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 -- 172.21.15.89:0/693266446 learned_addr learned my addr 172.21.15.89:0/693266446 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdffff700 1 --2- 172.21.15.89:0/693266446 >> v2:172.21.15.89:3300/0 conn(0x7f5ce012cf60 0x7f5ce006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5ce4c4c700 1 --2- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce0130260 0x7f5ce0136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 -- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce0130260 msgr2=0x7f5ce0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 --2- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce0130260 0x7f5ce0136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 -- 172.21.15.89:0/693266446 >> v2:172.21.15.89:3300/0 conn(0x7f5ce012cf60 msgr2=0x7f5ce006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 --2- 172.21.15.89:0/693266446 >> v2:172.21.15.89:3300/0 conn(0x7f5ce012cf60 0x7f5ce006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 -- 172.21.15.89:0/693266446 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce0071780 con 0x7f5ce012f8b0 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5ce4c4c700 1 --2- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce0130260 0x7f5ce0136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdffff700 1 --2- 172.21.15.89:0/693266446 >> v2:172.21.15.89:3300/0 conn(0x7f5ce012cf60 0x7f5ce006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdf7fe700 1 --2- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012f8b0 0x7f5ce012fd20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40098f0 tx=0x7f5cd4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34de71879c65e132 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdeffd700 1 -- 172.21.15.89:0/693266446 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400f940 con 0x7f5ce012f8b0 2023-09-30T22:20:18.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdeffd700 1 -- 172.21.15.89:0/693266446 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd400faa0 con 0x7f5ce012f8b0 2023-09-30T22:20:18.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.706+0000 7f5cdeffd700 1 -- 172.21.15.89:0/693266446 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400fe30 con 0x7f5ce012f8b0 2023-09-30T22:20:18.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 -- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012f8b0 msgr2=0x7f5ce012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 --2- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012f8b0 0x7f5ce012fd20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40098f0 tx=0x7f5cd4009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:18.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 -- 172.21.15.89:0/693266446 shutdown_connections 2023-09-30T22:20:18.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 --2- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce0130260 0x7f5ce0136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 --2- 172.21.15.89:0/693266446 >> v2:172.21.15.89:3300/0 conn(0x7f5ce012cf60 0x7f5ce006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 --2- 172.21.15.89:0/693266446 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012f8b0 0x7f5ce012fd20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 -- 172.21.15.89:0/693266446 >> 172.21.15.89:0/693266446 conn(0x7f5ce00a2aa0 msgr2=0x7f5ce00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:18.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 -- 172.21.15.89:0/693266446 shutdown_connections 2023-09-30T22:20:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.707+0000 7f5ce66af700 1 -- 172.21.15.89:0/693266446 wait complete. 2023-09-30T22:20:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.708+0000 7f5ce66af700 1 Processor -- start 2023-09-30T22:20:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.708+0000 7f5ce66af700 1 -- start start 2023-09-30T22:20:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce66af700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012cf60 0x7f5ce009b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce66af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce012f8b0 0x7f5ce009b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce66af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce0130260 0x7f5ce009be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce66af700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce009dd20 con 0x7f5ce0130260 2023-09-30T22:20:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce66af700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce009c8c0 con 0x7f5ce012cf60 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce66af700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce011d0e0 con 0x7f5ce012f8b0 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5cdffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012cf60 0x7f5ce009b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce4c4c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce0130260 0x7f5ce009be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5cdffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012cf60 0x7f5ce009b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38356/0 (socket says 172.21.15.89:38356) 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce4c4c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce0130260 0x7f5ce009be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50138/0 (socket says 172.21.15.89:50138) 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5cdffff700 1 -- 172.21.15.89:0/1066444961 learned_addr learned my addr 172.21.15.89:0/1066444961 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5cdf7fe700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce012f8b0 0x7f5ce009b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce4c4c700 1 -- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce012f8b0 msgr2=0x7f5ce009b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce4c4c700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce012f8b0 0x7f5ce009b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce4c4c700 1 -- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012cf60 msgr2=0x7f5ce009b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce4c4c700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012cf60 0x7f5ce009b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5ce4c4c700 1 -- 172.21.15.89:0/1066444961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd00094a0 con 0x7f5ce0130260 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5cdffff700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012cf60 0x7f5ce009b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.709+0000 7f5cdf7fe700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce012f8b0 0x7f5ce009b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.710+0000 7f5ce4c4c700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce0130260 0x7f5ce009be90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8007ec0 tx=0x7f5cd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.710+0000 7f5cdcff9700 1 -- 172.21.15.89:0/1066444961 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800de90 con 0x7f5ce0130260 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.710+0000 7f5cdcff9700 1 -- 172.21.15.89:0/1066444961 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd800f040 con 0x7f5ce0130260 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.710+0000 7f5cdcff9700 1 -- 172.21.15.89:0/1066444961 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8015620 con 0x7f5ce0130260 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.710+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd4009550 con 0x7f5ce0130260 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.710+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce009d030 con 0x7f5ce0130260 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.711+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce009d8d0 con 0x7f5ce0130260 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.711+0000 7f5cdcff9700 1 -- 172.21.15.89:0/1066444961 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5cd801e3b0 con 0x7f5ce0130260 2023-09-30T22:20:18.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.711+0000 7f5cdcff9700 1 -- 172.21.15.89:0/1066444961 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5cd803f5f0 con 0x7f5ce0130260 2023-09-30T22:20:18.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.716+0000 7f5cdcff9700 1 -- 172.21.15.89:0/1066444961 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5cd801e660 con 0x7f5ce0130260 2023-09-30T22:20:18.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.757+0000 7fdf811da700 1 Processor -- start 2023-09-30T22:20:18.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.757+0000 7fdf811da700 1 -- start start 2023-09-30T22:20:18.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.757+0000 7fdf811da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c09e1d0 0x7fdf7c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.757+0000 7fdf811da700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf7c139170 0x7fdf7c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.757+0000 7fdf811da700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c09d2f0 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.757+0000 7fdf811da700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c09b0f0 con 0x7fdf7c139170 2023-09-30T22:20:18.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.757+0000 7fdf811da700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c09a810 con 0x7fdf7c09e1d0 2023-09-30T22:20:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7ad9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdf7c09d6f0 0x7fdf7c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39290/0 (socket says 172.21.15.89:39290) 2023-09-30T22:20:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c09e1d0 0x7fdf7c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7ad9d700 1 -- 172.21.15.89:0/385344120 learned_addr learned my addr 172.21.15.89:0/385344120 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7b59e700 1 --2- 172.21.15.89:0/385344120 >> v2:172.21.15.89:3300/0 conn(0x7fdf7c139170 0x7fdf7c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf79d9b700 1 -- 172.21.15.89:0/385344120 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2907739108 0 0) 0x7fdf7c09d2f0 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf79d9b700 1 -- 172.21.15.89:0/385344120 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf640035b0 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7a59c700 1 -- 172.21.15.89:0/385344120 >> v2:172.21.15.89:3300/0 conn(0x7fdf7c139170 msgr2=0x7fdf7c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7a59c700 1 --2- 172.21.15.89:0/385344120 >> v2:172.21.15.89:3300/0 conn(0x7fdf7c139170 0x7fdf7c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7a59c700 1 -- 172.21.15.89:0/385344120 >> v1:172.21.15.89:6789/0 conn(0x7fdf7c09d6f0 legacy=0x7fdf7c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7a59c700 1 -- 172.21.15.89:0/385344120 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c09a470 con 0x7fdf7c09e1d0 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7b59e700 1 --2- 172.21.15.89:0/385344120 >> v2:172.21.15.89:3300/0 conn(0x7fdf7c139170 0x7fdf7c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf7a59c700 1 --2- 172.21.15.89:0/385344120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c09e1d0 0x7fdf7c138c30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009840 tx=0x7fdf70009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=890efc48c9705e96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.758+0000 7fdf79d9b700 1 -- 172.21.15.89:0/385344120 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70011830 con 0x7fdf7c09e1d0 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf79d9b700 1 -- 172.21.15.89:0/385344120 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf70011990 con 0x7fdf7c09e1d0 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf79d9b700 1 -- 172.21.15.89:0/385344120 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70011d20 con 0x7fdf7c09e1d0 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf811da700 1 -- 172.21.15.89:0/385344120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c09e1d0 msgr2=0x7fdf7c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf811da700 1 --2- 172.21.15.89:0/385344120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c09e1d0 0x7fdf7c138c30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009840 tx=0x7fdf70009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf811da700 1 -- 172.21.15.89:0/385344120 shutdown_connections 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf811da700 1 --2- 172.21.15.89:0/385344120 >> v2:172.21.15.89:3300/0 conn(0x7fdf7c139170 0x7fdf7c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf811da700 1 --2- 172.21.15.89:0/385344120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c09e1d0 0x7fdf7c138c30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf811da700 1 -- 172.21.15.89:0/385344120 >> 172.21.15.89:0/385344120 conn(0x7fdf7c066f50 msgr2=0x7fdf7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf811da700 1 -- 172.21.15.89:0/385344120 shutdown_connections 2023-09-30T22:20:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.759+0000 7fdf811da700 1 -- 172.21.15.89:0/385344120 wait complete. 2023-09-30T22:20:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.761+0000 7fdf811da700 1 Processor -- start 2023-09-30T22:20:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.761+0000 7fdf811da700 1 -- start start 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.761+0000 7fdf811da700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf7c09d6f0 0x7fdf7c127cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.761+0000 7fdf811da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c09e1d0 0x7fdf7c1281f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.761+0000 7fdf811da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c139170 0x7fdf7c1cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.761+0000 7fdf811da700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c138ba0 con 0x7fdf7c09e1d0 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.761+0000 7fdf811da700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c1277f0 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.761+0000 7fdf811da700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c09c640 con 0x7fdf7c139170 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf7c09d6f0 0x7fdf7c127cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf7c09d6f0 0x7fdf7c127cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38362/0 (socket says 172.21.15.89:38362) 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7b59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c139170 0x7fdf7c1cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 -- 172.21.15.89:0/794755873 learned_addr learned my addr 172.21.15.89:0/794755873 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7a59c700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c09e1d0 0x7fdf7c1281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 -- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c139170 msgr2=0x7fdf7c1cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c139170 0x7fdf7c1cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 -- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c09e1d0 msgr2=0x7fdf7c1281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c09e1d0 0x7fdf7c1281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 -- 172.21.15.89:0/794755873 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74009a70 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7a59c700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c09e1d0 0x7fdf7c1281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7b59e700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c139170 0x7fdf7c1cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.762+0000 7fdf7ad9d700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf7c09d6f0 0x7fdf7c127cb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c500 tx=0x7fdf6c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.763+0000 7fdf637fe700 1 -- 172.21.15.89:0/794755873 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c005d70 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.763+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf700094a0 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.763+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c12aa80 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.763+0000 7fdf637fe700 1 -- 172.21.15.89:0/794755873 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0070d0 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.763+0000 7fdf637fe700 1 -- 172.21.15.89:0/794755873 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c007460 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.763+0000 7fdf637fe700 1 -- 172.21.15.89:0/794755873 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf6c0075c0 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.764+0000 7fdf637fe700 1 -- 172.21.15.89:0/794755873 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdf6c040b20 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.764+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c0052e0 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.773+0000 7fdf637fe700 1 -- 172.21.15.89:0/794755873 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf6c017070 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.873+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd538005590 con 0x7fd55809de50 2023-09-30T22:20:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.874+0000 7fd53f7fe700 1 -- 172.21.15.89:0/3109368016 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd548003cb0 con 0x7fd55809de50 2023-09-30T22:20:18.873 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:18.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.876+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd55809de50 msgr2=0x7fd55811eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.876+0000 7fd55d7a1700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd55809de50 0x7fd55811eb10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd548004750 comp rx=0 tx=0).stop 2023-09-30T22:20:18.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.877+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 shutdown_connections 2023-09-30T22:20:18.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.877+0000 7fd55d7a1700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5581338a0 0x7fd55811f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.877+0000 7fd55d7a1700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd55809de50 0x7fd55811eb10 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.877+0000 7fd55d7a1700 1 --2- 172.21.15.89:0/3109368016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd55809e930 0x7fd55811f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.877+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 >> 172.21.15.89:0/3109368016 conn(0x7fd5580a2aa0 msgr2=0x7fd55809ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.877+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 shutdown_connections 2023-09-30T22:20:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.877+0000 7fd55d7a1700 1 -- 172.21.15.89:0/3109368016 wait complete. 2023-09-30T22:20:18.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:18.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.899+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2af41a94b0 con 0x7f2af409e410 2023-09-30T22:20:18.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.900+0000 7f2af17fa700 1 -- 172.21.15.89:0/366478430 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2af41a94b0 con 0x7f2af409e410 2023-09-30T22:20:18.900 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:18.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e410 msgr2=0x7f2af41a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e410 0x7f2af41a67b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009f00 tx=0x7f2aec00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 shutdown_connections 2023-09-30T22:20:18.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2af409e410 0x7f2af41a67b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2af409d6f0 0x7f2af41a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 --2- 172.21.15.89:0/366478430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af409e000 0x7f2af41a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 >> 172.21.15.89:0/366478430 conn(0x7f2af4066f50 msgr2=0x7f2af41bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:18.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 shutdown_connections 2023-09-30T22:20:18.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.902+0000 7f2afaf74700 1 -- 172.21.15.89:0/366478430 wait complete. 2023-09-30T22:20:18.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:18.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.925+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5ce009d030 con 0x7f5ce0130260 2023-09-30T22:20:18.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.925+0000 7f5cdcff9700 1 -- 172.21.15.89:0/1066444961 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ce009d030 con 0x7f5ce0130260 2023-09-30T22:20:18.925 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce0130260 msgr2=0x7f5ce009be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce0130260 0x7f5ce009be90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8007ec0 tx=0x7f5cd800d3b0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 shutdown_connections 2023-09-30T22:20:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce0130260 0x7f5ce009be90 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ce012cf60 0x7f5ce009b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 --2- 172.21.15.89:0/1066444961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ce012f8b0 0x7f5ce009b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 >> 172.21.15.89:0/1066444961 conn(0x7f5ce00a2aa0 msgr2=0x7f5ce00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 shutdown_connections 2023-09-30T22:20:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.927+0000 7f5ce66af700 1 -- 172.21.15.89:0/1066444961 wait complete. 2023-09-30T22:20:18.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.981+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdf5c005590 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.981+0000 7fdf637fe700 1 -- 172.21.15.89:0/794755873 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf6c017020 con 0x7fdf7c09d6f0 2023-09-30T22:20:18.981 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:18.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf7c09d6f0 msgr2=0x7fdf7c127cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:18.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf7c09d6f0 0x7fdf7c127cb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c500 tx=0x7fdf6c00c8c0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 shutdown_connections 2023-09-30T22:20:18.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf7c139170 0x7fdf7c1cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf7c09d6f0 0x7fdf7c127cb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 --2- 172.21.15.89:0/794755873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c09e1d0 0x7fdf7c1281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:18.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 >> 172.21.15.89:0/794755873 conn(0x7fdf7c066f50 msgr2=0x7fdf7c070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:18.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 shutdown_connections 2023-09-30T22:20:18.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:18.983+0000 7fdf811da700 1 -- 172.21.15.89:0/794755873 wait complete. 2023-09-30T22:20:18.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:19.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:19.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:19.941 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:19.996 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.039+0000 7f8711d35700 1 Processor -- start 2023-09-30T22:20:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.039+0000 7f8711d35700 1 -- start start 2023-09-30T22:20:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f8711d35700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12cf60 0x7f870c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f8711d35700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f870c12f8b0 0x7f870c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f8711d35700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870c12cd90 con 0x7f870c130260 2023-09-30T22:20:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f8711d35700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870c12d330 con 0x7f870c12f8b0 2023-09-30T22:20:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f8711d35700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c1306e0 con 0x7f870c12cf60 2023-09-30T22:20:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f870b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12cf60 0x7f870c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f870b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12cf60 0x7f870c06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38368/0 (socket says 172.21.15.89:38368) 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f870b7fe700 1 -- 172.21.15.89:0/2196119709 learned_addr learned my addr 172.21.15.89:0/2196119709 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.040+0000 7f870affd700 1 --2- 172.21.15.89:0/2196119709 >> v2:172.21.15.89:3300/0 conn(0x7f870c12f8b0 0x7f870c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.041+0000 7f870b7fe700 1 -- 172.21.15.89:0/2196119709 >> v2:172.21.15.89:3300/0 conn(0x7f870c12f8b0 msgr2=0x7f870c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.041+0000 7f870b7fe700 1 --2- 172.21.15.89:0/2196119709 >> v2:172.21.15.89:3300/0 conn(0x7f870c12f8b0 0x7f870c12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.041+0000 7f870b7fe700 1 -- 172.21.15.89:0/2196119709 >> v1:172.21.15.89:6789/0 conn(0x7f870c130260 legacy=0x7f870c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.041+0000 7f870b7fe700 1 -- 172.21.15.89:0/2196119709 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c071780 con 0x7f870c12cf60 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.041+0000 7f870b7fe700 1 --2- 172.21.15.89:0/2196119709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12cf60 0x7f870c06f5d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0098f0 tx=0x7f86fc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c591f9ecd07289 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.041+0000 7f870a7fc700 1 -- 172.21.15.89:0/2196119709 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00b8a0 con 0x7f870c12cf60 2023-09-30T22:20:20.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.041+0000 7f870a7fc700 1 -- 172.21.15.89:0/2196119709 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc00ba00 con 0x7f870c12cf60 2023-09-30T22:20:20.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.041+0000 7f870a7fc700 1 -- 172.21.15.89:0/2196119709 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00bd90 con 0x7f870c12cf60 2023-09-30T22:20:20.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.042+0000 7f8711d35700 1 -- 172.21.15.89:0/2196119709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12cf60 msgr2=0x7f870c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.042+0000 7f8711d35700 1 --2- 172.21.15.89:0/2196119709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12cf60 0x7f870c06f5d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0098f0 tx=0x7f86fc009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.042+0000 7f8711d35700 1 -- 172.21.15.89:0/2196119709 shutdown_connections 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.042+0000 7f8711d35700 1 --2- 172.21.15.89:0/2196119709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12cf60 0x7f870c06f5d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.042+0000 7f8711d35700 1 --2- 172.21.15.89:0/2196119709 >> v2:172.21.15.89:3300/0 conn(0x7f870c12f8b0 0x7f870c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.042+0000 7f8711d35700 1 -- 172.21.15.89:0/2196119709 >> 172.21.15.89:0/2196119709 conn(0x7f870c0a2aa0 msgr2=0x7f870c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.042+0000 7f8711d35700 1 -- 172.21.15.89:0/2196119709 shutdown_connections 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.042+0000 7f8711d35700 1 -- 172.21.15.89:0/2196119709 wait complete. 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.043+0000 7f8711d35700 1 Processor -- start 2023-09-30T22:20:20.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.044+0000 7f8711d35700 1 -- start start 2023-09-30T22:20:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.044+0000 7f8711d35700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f870c12cf60 0x7f870c129120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.044+0000 7f8711d35700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12f8b0 0x7f870c129660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.044+0000 7f8711d35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f870c130260 0x7f870c13a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.044+0000 7f8711d35700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c135370 con 0x7f870c130260 2023-09-30T22:20:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.044+0000 7f8711d35700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c135550 con 0x7f870c12f8b0 2023-09-30T22:20:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.044+0000 7f8711d35700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f870c136130 con 0x7f870c12cf60 2023-09-30T22:20:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12f8b0 0x7f870c129660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12f8b0 0x7f870c129660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38384/0 (socket says 172.21.15.89:38384) 2023-09-30T22:20:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f870c12cf60 0x7f870c129120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870affd700 1 -- 172.21.15.89:0/18404707 learned_addr learned my addr 172.21.15.89:0/18404707 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870bfff700 1 --2- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f870c130260 0x7f870c13a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870b7fe700 1 -- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12f8b0 msgr2=0x7f870c129660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870b7fe700 1 --2- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12f8b0 0x7f870c129660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870b7fe700 1 -- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f870c130260 msgr2=0x7f870c13a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870b7fe700 1 --2- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f870c130260 0x7f870c13a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870b7fe700 1 -- 172.21.15.89:0/18404707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc009550 con 0x7f870c12cf60 2023-09-30T22:20:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f870b7fe700 1 --2- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f870c12cf60 0x7f870c129120 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f86fc016000 tx=0x7f86fc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f86f3fff700 1 -- 172.21.15.89:0/18404707 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc004510 con 0x7f870c12cf60 2023-09-30T22:20:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f86f3fff700 1 -- 172.21.15.89:0/18404707 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc01e070 con 0x7f870c12cf60 2023-09-30T22:20:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f86f3fff700 1 -- 172.21.15.89:0/18404707 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0038a0 con 0x7f870c12cf60 2023-09-30T22:20:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c12d810 con 0x7f870c12cf60 2023-09-30T22:20:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.045+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c09e930 con 0x7f870c12cf60 2023-09-30T22:20:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.046+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c074ea0 con 0x7f870c12cf60 2023-09-30T22:20:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.047+0000 7f86f3fff700 1 -- 172.21.15.89:0/18404707 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f86fc003a00 con 0x7f870c12cf60 2023-09-30T22:20:20.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.047+0000 7f86f3fff700 1 -- 172.21.15.89:0/18404707 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f86fc020070 con 0x7f870c12cf60 2023-09-30T22:20:20.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.055+0000 7f86f3fff700 1 -- 172.21.15.89:0/18404707 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f86fc0199a0 con 0x7f870c12cf60 2023-09-30T22:20:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.077+0000 7f6fa2fa1700 1 Processor -- start 2023-09-30T22:20:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.077+0000 7f6fa2fa1700 1 -- start start 2023-09-30T22:20:20.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa2fa1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a0870 0x7f6f9c0a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa2fa1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f9c0a15b0 0x7f6f9c133ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa2fa1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c09db90 con 0x7f6f9c0a1180 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa2fa1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a1ce0 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa2fa1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a24f0 con 0x7f6f9c0a0870 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa179e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6f9c0a1180 0x7f6f9c133790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39310/0 (socket says 172.21.15.89:39310) 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa179e700 1 -- 172.21.15.89:0/529375594 learned_addr learned my addr 172.21.15.89:0/529375594 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa27a0700 1 --2- 172.21.15.89:0/529375594 >> v2:172.21.15.89:3300/0 conn(0x7f6f9c0a15b0 0x7f6f9c133ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa27a0700 1 -- 172.21.15.89:0/529375594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a0870 msgr2=0x7f6f9c0a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:20.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa27a0700 1 --2- 172.21.15.89:0/529375594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a0870 0x7f6f9c0a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa27a0700 1 -- 172.21.15.89:0/529375594 >> v1:172.21.15.89:6789/0 conn(0x7f6f9c0a1180 legacy=0x7f6f9c133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa0f9d700 1 -- 172.21.15.89:0/529375594 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766427391 0 0) 0x7f6f9c09db90 con 0x7f6f9c0a1180 2023-09-30T22:20:20.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.078+0000 7f6fa27a0700 1 -- 172.21.15.89:0/529375594 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0a1aa0 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f6fa27a0700 1 --2- 172.21.15.89:0/529375594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a15b0 0x7f6f9c133ea0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009d90 tx=0x7f6f9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade1e096d93af033 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:20.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f6fa0f9d700 1 -- 172.21.15.89:0/529375594 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400e070 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f6fa0f9d700 1 -- 172.21.15.89:0/529375594 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9400bbd0 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f6fa0f9d700 1 -- 172.21.15.89:0/529375594 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94012430 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f57f8b98700 1 Processor -- start 2023-09-30T22:20:20.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/529375594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a15b0 msgr2=0x7f6f9c133ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f6fa2fa1700 1 --2- 172.21.15.89:0/529375594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a15b0 0x7f6f9c133ea0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009d90 tx=0x7f6f9400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:20.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/529375594 shutdown_connections 2023-09-30T22:20:20.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f6fa2fa1700 1 --2- 172.21.15.89:0/529375594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a15b0 0x7f6f9c133ea0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f6fa2fa1700 1 --2- 172.21.15.89:0/529375594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a0870 0x7f6f9c0a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.079+0000 7f57f8b98700 1 -- start start 2023-09-30T22:20:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/529375594 >> 172.21.15.89:0/529375594 conn(0x7f6f9c0a6390 msgr2=0x7f6f9c0a6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f8b98700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f409de00 0x7f57f409e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f8b98700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f8b98700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f409a810 con 0x7f57f406a770 2023-09-30T22:20:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f8b98700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f406ab40 con 0x7f57f409de00 2023-09-30T22:20:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f8b98700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57f409ebc0 con 0x7f57f413d1f0 2023-09-30T22:20:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/529375594 shutdown_connections 2023-09-30T22:20:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/529375594 wait complete. 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f57f406a770 0x7f57f409d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39320/0 (socket says 172.21.15.89:39320) 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f259c700 1 -- 172.21.15.89:0/3265786342 learned_addr learned my addr 172.21.15.89:0/3265786342 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f1d9b700 1 --2- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f409de00 0x7f57f409e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f2d9d700 1 --2- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f1d9b700 1 -- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 msgr2=0x7f57f413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f1d9b700 1 --2- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f413f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f1d9b700 1 -- 172.21.15.89:0/3265786342 >> v1:172.21.15.89:6789/0 conn(0x7f57f406a770 legacy=0x7f57f409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f1d9b700 1 -- 172.21.15.89:0/3265786342 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f409e920 con 0x7f57f409de00 2023-09-30T22:20:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f2d9d700 1 --2- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.080+0000 7f57f1d9b700 1 --2- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f409de00 0x7f57f409e250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009870 tx=0x7f57e8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c36d9f439637cc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f159a700 1 -- 172.21.15.89:0/3265786342 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8011830 con 0x7f57f409de00 2023-09-30T22:20:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f159a700 1 -- 172.21.15.89:0/3265786342 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e8011990 con 0x7f57f409de00 2023-09-30T22:20:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f159a700 1 -- 172.21.15.89:0/3265786342 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8011d20 con 0x7f57f409de00 2023-09-30T22:20:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f6fa2fa1700 1 Processor -- start 2023-09-30T22:20:20.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f6fa2fa1700 1 -- start start 2023-09-30T22:20:20.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f8b98700 1 -- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f409de00 msgr2=0x7f57f409e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f8b98700 1 --2- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f409de00 0x7f57f409e250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009870 tx=0x7f57e8009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:20.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f8b98700 1 -- 172.21.15.89:0/3265786342 shutdown_connections 2023-09-30T22:20:20.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f8b98700 1 --2- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f8b98700 1 --2- 172.21.15.89:0/3265786342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f409de00 0x7f57f409e250 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.081+0000 7f57f8b98700 1 -- 172.21.15.89:0/3265786342 >> 172.21.15.89:0/3265786342 conn(0x7f57f4066f50 msgr2=0x7f57f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:20.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f57f8b98700 1 -- 172.21.15.89:0/3265786342 shutdown_connections 2023-09-30T22:20:20.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f57f8b98700 1 -- 172.21.15.89:0/3265786342 wait complete. 2023-09-30T22:20:20.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa2fa1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a0870 0x7f6f9c126ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa2fa1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9c0a1180 0x7f6f9c126ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa2fa1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a15b0 0x7f6f9c127530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa2fa1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c09cec0 con 0x7f6f9c0a0870 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa2fa1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c129540 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa2fa1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c12a260 con 0x7f6f9c0a1180 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa27a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a15b0 0x7f6f9c127530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa27a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a15b0 0x7f6f9c127530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38408/0 (socket says 172.21.15.89:38408) 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa27a0700 1 -- 172.21.15.89:0/3721428466 learned_addr learned my addr 172.21.15.89:0/3721428466 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa1f9f700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a0870 0x7f6f9c126ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa179e700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9c0a1180 0x7f6f9c126ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa27a0700 1 -- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9c0a1180 msgr2=0x7f6f9c126ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa27a0700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9c0a1180 0x7f6f9c126ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa27a0700 1 -- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a0870 msgr2=0x7f6f9c126ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa27a0700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a0870 0x7f6f9c126ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.082+0000 7f6fa27a0700 1 -- 172.21.15.89:0/3721428466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94009a70 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f6fa27a0700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a15b0 0x7f6f9c127530 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6f94014040 tx=0x7f6f94004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:20.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f8b98700 1 Processor -- start 2023-09-30T22:20:20.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f6fa179e700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9c0a1180 0x7f6f9c126ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:20.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f6fa1f9f700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a0870 0x7f6f9c126ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:20.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f8b98700 1 -- start start 2023-09-30T22:20:20.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f8b98700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f406a770 0x7f57f41ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f8b98700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f409de00 0x7f57f41ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f8b98700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f41cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f8b98700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f409f210 con 0x7f57f409de00 2023-09-30T22:20:20.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f8b98700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f413bea0 con 0x7f57f406a770 2023-09-30T22:20:20.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f8b98700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57f406fec0 con 0x7f57f413d1f0 2023-09-30T22:20:20.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f406a770 0x7f57f41ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f406a770 0x7f57f41ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38416/0 (socket says 172.21.15.89:38416) 2023-09-30T22:20:20.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.083+0000 7f57f259c700 1 -- 172.21.15.89:0/1836936965 learned_addr learned my addr 172.21.15.89:0/1836936965 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f6f927fc700 1 -- 172.21.15.89:0/3721428466 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94012430 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f6f927fc700 1 -- 172.21.15.89:0/3721428466 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f94020070 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f6f927fc700 1 -- 172.21.15.89:0/3721428466 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94004280 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f1d9b700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f409de00 0x7f57f41ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f2d9d700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f41cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c09bf20 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c12acd0 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f1d9b700 1 -- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 msgr2=0x7f57f41cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f1d9b700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f41cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f1d9b700 1 -- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f406a770 msgr2=0x7f57f41ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f1d9b700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f406a770 0x7f57f41ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f1d9b700 1 -- 172.21.15.89:0/1836936965 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009a70 con 0x7f57f409de00 2023-09-30T22:20:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f1d9b700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f409de00 0x7f57f41ccfe0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f57e8016000 tx=0x7f57e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57e2ffd700 1 -- 172.21.15.89:0/1836936965 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8004550 con 0x7f57f409de00 2023-09-30T22:20:20.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57e2ffd700 1 -- 172.21.15.89:0/1836936965 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e801f070 con 0x7f57f409de00 2023-09-30T22:20:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57e2ffd700 1 -- 172.21.15.89:0/1836936965 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80038e0 con 0x7f57f409de00 2023-09-30T22:20:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e80094a0 con 0x7f57f409de00 2023-09-30T22:20:20.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.084+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f4072490 con 0x7f57f409de00 2023-09-30T22:20:20.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.085+0000 7f6f927fc700 1 -- 172.21.15.89:0/3721428466 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f940043e0 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.085+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c070240 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.085+0000 7f6f927fc700 1 -- 172.21.15.89:0/3721428466 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6f94022070 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.085+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f40709f0 con 0x7f57f409de00 2023-09-30T22:20:20.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.086+0000 7f57e2ffd700 1 -- 172.21.15.89:0/1836936965 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f57e8003a40 con 0x7f57f409de00 2023-09-30T22:20:20.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.086+0000 7f57e2ffd700 1 -- 172.21.15.89:0/1836936965 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f57e8021070 con 0x7f57f409de00 2023-09-30T22:20:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.090+0000 7f6f927fc700 1 -- 172.21.15.89:0/3721428466 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f94004690 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.091+0000 7f57e2ffd700 1 -- 172.21.15.89:0/1836936965 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57e802f3a0 con 0x7f57f409de00 2023-09-30T22:20:20.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.169+0000 7f98aec8e700 1 Processor -- start 2023-09-30T22:20:20.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.169+0000 7f98aec8e700 1 -- start start 2023-09-30T22:20:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98aec8e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98aec8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98aec8e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a8070940 con 0x7f98a809f650 2023-09-30T22:20:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98aec8e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a806fd60 con 0x7f98a809e930 2023-09-30T22:20:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98aec8e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98a806fa80 con 0x7f98a809f240 2023-09-30T22:20:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98aca2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98aca2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38428/0 (socket says 172.21.15.89:38428) 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98aca2a700 1 -- 172.21.15.89:0/1378933920 learned_addr learned my addr 172.21.15.89:0/1378933920 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.170+0000 7f98a7fff700 1 --2- 172.21.15.89:0/1378933920 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98a77fe700 1 -- 172.21.15.89:0/1378933920 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332820616 0 0) 0x7f98a8070940 con 0x7f98a809f650 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98aca2a700 1 -- 172.21.15.89:0/1378933920 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 msgr2=0x7f98a81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98aca2a700 1 --2- 172.21.15.89:0/1378933920 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a81317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98aca2a700 1 -- 172.21.15.89:0/1378933920 >> v1:172.21.15.89:6789/0 conn(0x7f98a809f650 legacy=0x7f98a8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98aca2a700 1 -- 172.21.15.89:0/1378933920 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a809fb60 con 0x7f98a809e930 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98aca2a700 1 --2- 172.21.15.89:0/1378933920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a809ed00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f989c009920 tx=0x7f989c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a270f51cc217238d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98a77fe700 1 -- 172.21.15.89:0/1378933920 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c011830 con 0x7f98a809e930 2023-09-30T22:20:20.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98a77fe700 1 -- 172.21.15.89:0/1378933920 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c011990 con 0x7f98a809e930 2023-09-30T22:20:20.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.171+0000 7f98a77fe700 1 -- 172.21.15.89:0/1378933920 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c011d20 con 0x7f98a809e930 2023-09-30T22:20:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.172+0000 7f98aec8e700 1 -- 172.21.15.89:0/1378933920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 msgr2=0x7f98a809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.172+0000 7f98aec8e700 1 --2- 172.21.15.89:0/1378933920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a809ed00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f989c009920 tx=0x7f989c009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.172+0000 7f98aec8e700 1 -- 172.21.15.89:0/1378933920 shutdown_connections 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.172+0000 7f98aec8e700 1 --2- 172.21.15.89:0/1378933920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a809ed00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.172+0000 7f98aec8e700 1 --2- 172.21.15.89:0/1378933920 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.172+0000 7f98aec8e700 1 -- 172.21.15.89:0/1378933920 >> 172.21.15.89:0/1378933920 conn(0x7f98a80a4210 msgr2=0x7f98a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.172+0000 7f98aec8e700 1 -- 172.21.15.89:0/1378933920 shutdown_connections 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.172+0000 7f98aec8e700 1 -- 172.21.15.89:0/1378933920 wait complete. 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.173+0000 7f98aec8e700 1 Processor -- start 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.173+0000 7f98aec8e700 1 -- start start 2023-09-30T22:20:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aec8e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a806b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aec8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aec8e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a809f650 0x7f98a806c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aec8e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81a9990 con 0x7f98a809f650 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aec8e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81b71d0 con 0x7f98a809e930 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aec8e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81a8020 con 0x7f98a809f240 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a806b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:38434/0 (socket says 172.21.15.89:38434) 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 -- 172.21.15.89:0/3426528407 learned_addr learned my addr 172.21.15.89:0/3426528407 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98a7fff700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98ad22b700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a809f650 0x7f98a806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 -- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 msgr2=0x7f98a806bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a806bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 -- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a809f650 msgr2=0x7f98a806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a809f650 0x7f98a806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 -- 172.21.15.89:0/3426528407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009550 con 0x7f98a809e930 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98a7fff700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aca2a700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a806b880 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f989c000c00 tx=0x7f989c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98a57fa700 1 -- 172.21.15.89:0/3426528407 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c004510 con 0x7f98a809e930 2023-09-30T22:20:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98a57fa700 1 -- 172.21.15.89:0/3426528407 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c0036a0 con 0x7f98a809e930 2023-09-30T22:20:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aec8e700 1 -- 172.21.15.89:0/3426528407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a80a0190 con 0x7f98a809e930 2023-09-30T22:20:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98a57fa700 1 -- 172.21.15.89:0/3426528407 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c003a10 con 0x7f98a809e930 2023-09-30T22:20:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.174+0000 7f98aec8e700 1 -- 172.21.15.89:0/3426528407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a81a92f0 con 0x7f98a809e930 2023-09-30T22:20:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.175+0000 7f98a57fa700 1 -- 172.21.15.89:0/3426528407 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f989c003b70 con 0x7f98a809e930 2023-09-30T22:20:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.175+0000 7f9892ffd700 1 -- 172.21.15.89:0/3426528407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98800052e0 con 0x7f98a809e930 2023-09-30T22:20:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.176+0000 7f98a57fa700 1 -- 172.21.15.89:0/3426528407 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f989c021030 con 0x7f98a809e930 2023-09-30T22:20:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.180+0000 7f98a57fa700 1 -- 172.21.15.89:0/3426528407 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f989c01a430 con 0x7f98a809e930 2023-09-30T22:20:20.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.302+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f870c0713a0 con 0x7f870c12cf60 2023-09-30T22:20:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.303+0000 7f86f3fff700 1 -- 172.21.15.89:0/18404707 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f86fc02a030 con 0x7f870c12cf60 2023-09-30T22:20:20.303 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f870c12cf60 msgr2=0x7f870c129120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 --2- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f870c12cf60 0x7f870c129120 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f86fc016000 tx=0x7f86fc004750 comp rx=0 tx=0).stop 2023-09-30T22:20:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 shutdown_connections 2023-09-30T22:20:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 --2- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f870c130260 0x7f870c13a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 --2- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f870c12cf60 0x7f870c129120 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 --2- 172.21.15.89:0/18404707 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f870c12f8b0 0x7f870c129660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 >> 172.21.15.89:0/18404707 conn(0x7f870c0a2aa0 msgr2=0x7f870c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 shutdown_connections 2023-09-30T22:20:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.305+0000 7f8711d35700 1 -- 172.21.15.89:0/18404707 wait complete. 2023-09-30T22:20:20.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:20.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.347+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f9c04f2a0 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.348+0000 7f6f927fc700 1 -- 172.21.15.89:0/3721428466 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f94025e00 con 0x7f6f9c0a15b0 2023-09-30T22:20:20.348 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.349+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a15b0 msgr2=0x7f6f9c127530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.349+0000 7f6fa2fa1700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a15b0 0x7f6f9c127530 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6f94014040 tx=0x7f6f94004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.350+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 shutdown_connections 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.350+0000 7f6fa2fa1700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f9c0a15b0 0x7f6f9c127530 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.350+0000 7f6fa2fa1700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f9c0a0870 0x7f6f9c126ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.350+0000 7f6fa2fa1700 1 --2- 172.21.15.89:0/3721428466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f9c0a1180 0x7f6f9c126ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.350+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 >> 172.21.15.89:0/3721428466 conn(0x7f6f9c0a6390 msgr2=0x7f6f9c09c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.350+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 shutdown_connections 2023-09-30T22:20:20.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.350+0000 7f6fa2fa1700 1 -- 172.21.15.89:0/3721428466 wait complete. 2023-09-30T22:20:20.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.352+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f57f4074ea0 con 0x7f57f409de00 2023-09-30T22:20:20.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.352+0000 7f57e2ffd700 1 -- 172.21.15.89:0/1836936965 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f57e802b070 con 0x7f57f409de00 2023-09-30T22:20:20.352 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:20.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f409de00 msgr2=0x7f57f41ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f409de00 0x7f57f41ccfe0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f57e8016000 tx=0x7f57e8004750 comp rx=0 tx=0).stop 2023-09-30T22:20:20.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 shutdown_connections 2023-09-30T22:20:20.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57f413d1f0 0x7f57f41cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57f406a770 0x7f57f41ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 --2- 172.21.15.89:0/1836936965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f409de00 0x7f57f41ccfe0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 >> 172.21.15.89:0/1836936965 conn(0x7f57f4066f50 msgr2=0x7f57f413cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:20.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 shutdown_connections 2023-09-30T22:20:20.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.354+0000 7f57f8b98700 1 -- 172.21.15.89:0/1836936965 wait complete. 2023-09-30T22:20:20.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:20.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:20.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.375+0000 7f9892ffd700 1 -- 172.21.15.89:0/3426528407 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9880005590 con 0x7f98a809e930 2023-09-30T22:20:20.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.375+0000 7f98a57fa700 1 -- 172.21.15.89:0/3426528407 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f989c015030 con 0x7f98a809e930 2023-09-30T22:20:20.375 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:20.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.377+0000 7f9892ffd700 1 -- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 msgr2=0x7f98a806b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.377+0000 7f9892ffd700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a806b880 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f989c000c00 tx=0x7f989c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.378+0000 7f9892ffd700 1 -- 172.21.15.89:0/3426528407 shutdown_connections 2023-09-30T22:20:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.378+0000 7f9892ffd700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a809f650 0x7f98a806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.378+0000 7f9892ffd700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98a809e930 0x7f98a806b880 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.378+0000 7f9892ffd700 1 --2- 172.21.15.89:0/3426528407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98a809f240 0x7f98a806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.378+0000 7f9892ffd700 1 -- 172.21.15.89:0/3426528407 >> 172.21.15.89:0/3426528407 conn(0x7f98a80a4210 msgr2=0x7f98a81b8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.378+0000 7f9892ffd700 1 -- 172.21.15.89:0/3426528407 shutdown_connections 2023-09-30T22:20:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:20.378+0000 7f9892ffd700 1 -- 172.21.15.89:0/3426528407 wait complete. 2023-09-30T22:20:20.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:21.318 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:21.364 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:21.368 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:21.390 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.466+0000 7f2e419c1700 1 Processor -- start 2023-09-30T22:20:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.466+0000 7f2e419c1700 1 -- start start 2023-09-30T22:20:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e419c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09d6f0 0x7f2e3c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e419c1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e3c09e000 0x7f2e3c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e419c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09e410 0x7f2e3c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e419c1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c06b460 con 0x7f2e3c09e000 2023-09-30T22:20:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e419c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c09b120 con 0x7f2e3c09d6f0 2023-09-30T22:20:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e419c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c09a7e0 con 0x7f2e3c09e410 2023-09-30T22:20:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09d6f0 0x7f2e3c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09d6f0 0x7f2e3c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58644/0 (socket says 172.21.15.89:58644) 2023-09-30T22:20:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3affd700 1 -- 172.21.15.89:0/496142567 learned_addr learned my addr 172.21.15.89:0/496142567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3b7fe700 1 --2- 172.21.15.89:0/496142567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09e410 0x7f2e3c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3a7fc700 1 --2- 172.21.15.89:0/496142567 >> v2:172.21.15.89:3300/0 conn(0x7f2e3c09e000 0x7f2e3c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3b7fe700 1 -- 172.21.15.89:0/496142567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09d6f0 msgr2=0x7f2e3c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3b7fe700 1 --2- 172.21.15.89:0/496142567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09d6f0 0x7f2e3c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3b7fe700 1 -- 172.21.15.89:0/496142567 >> v2:172.21.15.89:3300/0 conn(0x7f2e3c09e000 msgr2=0x7f2e3c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3b7fe700 1 --2- 172.21.15.89:0/496142567 >> v2:172.21.15.89:3300/0 conn(0x7f2e3c09e000 0x7f2e3c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.467+0000 7f2e3b7fe700 1 -- 172.21.15.89:0/496142567 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c09a500 con 0x7f2e3c09e410 2023-09-30T22:20:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.468+0000 7f2e3b7fe700 1 --2- 172.21.15.89:0/496142567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09e410 0x7f2e3c133f30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009dc0 tx=0x7f2e3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb06ab9759bfe382 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.468+0000 7f2e39ffb700 1 -- 172.21.15.89:0/496142567 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400e070 con 0x7f2e3c09e410 2023-09-30T22:20:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.468+0000 7f2e39ffb700 1 -- 172.21.15.89:0/496142567 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3400bae0 con 0x7f2e3c09e410 2023-09-30T22:20:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.468+0000 7f2e39ffb700 1 -- 172.21.15.89:0/496142567 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400be70 con 0x7f2e3c09e410 2023-09-30T22:20:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 -- 172.21.15.89:0/496142567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09e410 msgr2=0x7f2e3c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 --2- 172.21.15.89:0/496142567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09e410 0x7f2e3c133f30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009dc0 tx=0x7f2e3400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 -- 172.21.15.89:0/496142567 shutdown_connections 2023-09-30T22:20:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 --2- 172.21.15.89:0/496142567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09e410 0x7f2e3c133f30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 --2- 172.21.15.89:0/496142567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09d6f0 0x7f2e3c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 --2- 172.21.15.89:0/496142567 >> v2:172.21.15.89:3300/0 conn(0x7f2e3c09e000 0x7f2e3c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 -- 172.21.15.89:0/496142567 >> 172.21.15.89:0/496142567 conn(0x7f2e3c066f50 msgr2=0x7f2e3c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 -- 172.21.15.89:0/496142567 shutdown_connections 2023-09-30T22:20:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.469+0000 7f2e419c1700 1 -- 172.21.15.89:0/496142567 wait complete. 2023-09-30T22:20:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.470+0000 7f2e419c1700 1 Processor -- start 2023-09-30T22:20:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e419c1700 1 -- start start 2023-09-30T22:20:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e419c1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09d6f0 0x7f2e3c1c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e419c1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e3c09e000 0x7f2e3c1c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e419c1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09e410 0x7f2e3c1c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e419c1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c1aa9a0 con 0x7f2e3c09e000 2023-09-30T22:20:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e419c1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0a1d00 con 0x7f2e3c09e410 2023-09-30T22:20:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e419c1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c09c710 con 0x7f2e3c09d6f0 2023-09-30T22:20:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e3affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09d6f0 0x7f2e3c1c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e3affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09d6f0 0x7f2e3c1c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36486/0 (socket says 172.21.15.89:36486) 2023-09-30T22:20:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e3affd700 1 -- 172.21.15.89:0/2941701866 learned_addr learned my addr 172.21.15.89:0/2941701866 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e3a7fc700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e3c09e000 0x7f2e3c1c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.471+0000 7f2e3b7fe700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09e410 0x7f2e3c1c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e3a7fc700 1 -- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09d6f0 msgr2=0x7f2e3c1c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e3a7fc700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09d6f0 0x7f2e3c1c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e3a7fc700 1 -- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09e410 msgr2=0x7f2e3c1c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e3a7fc700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09e410 0x7f2e3c1c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e3a7fc700 1 -- 172.21.15.89:0/2941701866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34009a70 con 0x7f2e3c09e000 2023-09-30T22:20:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e3a7fc700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e3c09e000 0x7f2e3c1c8be0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e8d0 tx=0x7f2e3000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e23fff700 1 -- 172.21.15.89:0/2941701866 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30013680 con 0x7f2e3c09e000 2023-09-30T22:20:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e23fff700 1 -- 172.21.15.89:0/2941701866 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e300137e0 con 0x7f2e3c09e000 2023-09-30T22:20:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e23fff700 1 -- 172.21.15.89:0/2941701866 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30013b50 con 0x7f2e3c09e000 2023-09-30T22:20:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c1a9b80 con 0x7f2e3c09e000 2023-09-30T22:20:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.472+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c1ac150 con 0x7f2e3c09e000 2023-09-30T22:20:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.473+0000 7f2e23fff700 1 -- 172.21.15.89:0/2941701866 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e30013cb0 con 0x7f2e3c09e000 2023-09-30T22:20:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.473+0000 7f2e23fff700 1 -- 172.21.15.89:0/2941701866 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2e3003f6a0 con 0x7f2e3c09e000 2023-09-30T22:20:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.473+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c074ea0 con 0x7f2e3c09e000 2023-09-30T22:20:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.478+0000 7f2e23fff700 1 -- 172.21.15.89:0/2941701866 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e30017080 con 0x7f2e3c09e000 2023-09-30T22:20:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.520+0000 7fc97a851700 1 Processor -- start 2023-09-30T22:20:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.520+0000 7fc97a851700 1 -- start start 2023-09-30T22:20:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc97a851700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc97409e930 0x7fc97409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc97a851700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409f240 0x7fc9741317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc97a851700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc97a851700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc974070940 con 0x7fc97409e930 2023-09-30T22:20:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc97a851700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97406fd60 con 0x7fc97409f240 2023-09-30T22:20:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc97a851700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc97406fa80 con 0x7fc97409f650 2023-09-30T22:20:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc973fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc97409e930 0x7fc97409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc973fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc97409e930 0x7fc97409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58368/0 (socket says 172.21.15.89:58368) 2023-09-30T22:20:21.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc973fff700 1 -- 172.21.15.89:0/3015573211 learned_addr learned my addr 172.21.15.89:0/3015573211 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc978dee700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.521+0000 7fc9737fe700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409f240 0x7fc9741317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc973fff700 1 -- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 msgr2=0x7fc974131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc973fff700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc973fff700 1 -- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409f240 msgr2=0x7fc9741317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc973fff700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409f240 0x7fc9741317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc973fff700 1 -- 172.21.15.89:0/3015573211 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97409fb60 con 0x7fc97409e930 2023-09-30T22:20:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc973fff700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409e930 0x7fc97409ed00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc9640098f0 tx=0x7fc964009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4f2c0e3c6ff079 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc972ffd700 1 -- 172.21.15.89:0/3015573211 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964011830 con 0x7fc97409e930 2023-09-30T22:20:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc972ffd700 1 -- 172.21.15.89:0/3015573211 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964011990 con 0x7fc97409e930 2023-09-30T22:20:21.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.522+0000 7fc972ffd700 1 -- 172.21.15.89:0/3015573211 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964011d00 con 0x7fc97409e930 2023-09-30T22:20:21.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 -- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409e930 msgr2=0x7fc97409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409e930 0x7fc97409ed00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc9640098f0 tx=0x7fc964009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:21.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 -- 172.21.15.89:0/3015573211 shutdown_connections 2023-09-30T22:20:21.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409e930 0x7fc97409ed00 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 --2- 172.21.15.89:0/3015573211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409f240 0x7fc9741317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 -- 172.21.15.89:0/3015573211 >> 172.21.15.89:0/3015573211 conn(0x7fc9740a4210 msgr2=0x7fc9740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:21.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 -- 172.21.15.89:0/3015573211 shutdown_connections 2023-09-30T22:20:21.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.523+0000 7fc97a851700 1 -- 172.21.15.89:0/3015573211 wait complete. 2023-09-30T22:20:21.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.524+0000 7fc97a851700 1 Processor -- start 2023-09-30T22:20:21.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.524+0000 7fc97a851700 1 -- start start 2023-09-30T22:20:21.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc97a851700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409e930 0x7fc9741241a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc97a851700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409f240 0x7fc974122ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc97a851700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974123010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc97a851700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740a3160 con 0x7fc97409f240 2023-09-30T22:20:21.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc97a851700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc974121c10 con 0x7fc97409e930 2023-09-30T22:20:21.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc97a851700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc974122680 con 0x7fc97409f650 2023-09-30T22:20:21.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc9737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409f240 0x7fc974122ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc978dee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974123010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc9737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409f240 0x7fc974122ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58378/0 (socket says 172.21.15.89:58378) 2023-09-30T22:20:21.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc9737fe700 1 -- 172.21.15.89:0/3603358661 learned_addr learned my addr 172.21.15.89:0/3603358661 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:21.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.525+0000 7fc973fff700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409e930 0x7fc9741241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc978dee700 1 -- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409e930 msgr2=0x7fc9741241a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc978dee700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409e930 0x7fc9741241a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc978dee700 1 -- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409f240 msgr2=0x7fc974122ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc978dee700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409f240 0x7fc974122ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc978dee700 1 -- 172.21.15.89:0/3603358661 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964009550 con 0x7fc97409f650 2023-09-30T22:20:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc9737fe700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409f240 0x7fc974122ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc973fff700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409e930 0x7fc9741241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc978dee700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974123010 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00efc0 tx=0x7fc96c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc970ff9700 1 -- 172.21.15.89:0/3603358661 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00cc90 con 0x7fc97409f650 2023-09-30T22:20:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc970ff9700 1 -- 172.21.15.89:0/3603358661 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00f040 con 0x7fc97409f650 2023-09-30T22:20:21.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9740a0190 con 0x7fc97409f650 2023-09-30T22:20:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc970ff9700 1 -- 172.21.15.89:0/3603358661 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0146c0 con 0x7fc97409f650 2023-09-30T22:20:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.526+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9741315c0 con 0x7fc97409f650 2023-09-30T22:20:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.527+0000 7fc970ff9700 1 -- 172.21.15.89:0/3603358661 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc96c014820 con 0x7fc97409f650 2023-09-30T22:20:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.527+0000 7fc970ff9700 1 -- 172.21.15.89:0/3603358661 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc96c040180 con 0x7fc97409f650 2023-09-30T22:20:21.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.528+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97406cc70 con 0x7fc97409f650 2023-09-30T22:20:21.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.536+0000 7fc970ff9700 1 -- 172.21.15.89:0/3603358661 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc96c014ad0 con 0x7fc97409f650 2023-09-30T22:20:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f99867bc700 1 Processor -- start 2023-09-30T22:20:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f99867bc700 1 -- start start 2023-09-30T22:20:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f99867bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009d6f0 0x7f998009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f99867bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f998009e000 0x7f99801339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f99867bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009e410 0x7f9980133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f99867bc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998006b460 con 0x7f998009e000 2023-09-30T22:20:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f99867bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998009b120 con 0x7f998009d6f0 2023-09-30T22:20:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f99867bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f998009a7e0 con 0x7f998009e410 2023-09-30T22:20:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.544+0000 7f997ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009d6f0 0x7f998009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009d6f0 0x7f998009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58672/0 (socket says 172.21.15.89:58672) 2023-09-30T22:20:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997ffff700 1 -- 172.21.15.89:0/2521997858 learned_addr learned my addr 172.21.15.89:0/2521997858 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997f7fe700 1 --2- 172.21.15.89:0/2521997858 >> v2:172.21.15.89:3300/0 conn(0x7f998009e000 0x7f99801339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f9984d59700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009e410 0x7f9980133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997f7fe700 1 -- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009e410 msgr2=0x7f9980133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:21.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997f7fe700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009e410 0x7f9980133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997f7fe700 1 -- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009d6f0 msgr2=0x7f998009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:21.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997f7fe700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009d6f0 0x7f998009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997f7fe700 1 -- 172.21.15.89:0/2521997858 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998009a500 con 0x7f998009e000 2023-09-30T22:20:21.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997ffff700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009d6f0 0x7f998009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:21.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997f7fe700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e000 0x7f99801339f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9974009840 tx=0x7f9974009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcab72f7856f480 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:21.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997effd700 1 -- 172.21.15.89:0/2521997858 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974011830 con 0x7f998009e000 2023-09-30T22:20:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997effd700 1 -- 172.21.15.89:0/2521997858 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9974011990 con 0x7f998009e000 2023-09-30T22:20:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.545+0000 7f997effd700 1 -- 172.21.15.89:0/2521997858 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974011d20 con 0x7f998009e000 2023-09-30T22:20:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 -- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e000 msgr2=0x7f99801339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e000 0x7f99801339f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9974009840 tx=0x7f9974009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 -- 172.21.15.89:0/2521997858 shutdown_connections 2023-09-30T22:20:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009e410 0x7f9980133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009d6f0 0x7f998009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 --2- 172.21.15.89:0/2521997858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e000 0x7f99801339f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 -- 172.21.15.89:0/2521997858 >> 172.21.15.89:0/2521997858 conn(0x7f9980066f50 msgr2=0x7f998009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 -- 172.21.15.89:0/2521997858 shutdown_connections 2023-09-30T22:20:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f99867bc700 1 -- 172.21.15.89:0/2521997858 wait complete. 2023-09-30T22:20:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f105e081700 1 Processor -- start 2023-09-30T22:20:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.546+0000 7f105e081700 1 -- start start 2023-09-30T22:20:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f105e081700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f105e081700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f1058130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f99867bc700 1 Processor -- start 2023-09-30T22:20:21.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f105e081700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f105809e410 0x7f1058130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f105e081700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105809f520 con 0x7f105809e410 2023-09-30T22:20:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f105e081700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105809f130 con 0x7f105809e000 2023-09-30T22:20:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f105e081700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f105809e930 con 0x7f105809d6f0 2023-09-30T22:20:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1056ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f1058130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1056ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f1058130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58676/0 (socket says 172.21.15.89:58676) 2023-09-30T22:20:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1056ffd700 1 -- 172.21.15.89:0/3557229626 learned_addr learned my addr 172.21.15.89:0/3557229626 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1057fff700 1 --2- 172.21.15.89:0/3557229626 >> v2:172.21.15.89:3300/0 conn(0x7f105809e410 0x7f1058130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f10577fe700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f99867bc700 1 -- start start 2023-09-30T22:20:21.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1057fff700 1 -- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 msgr2=0x7f105809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:21.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1057fff700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1057fff700 1 -- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 msgr2=0x7f1058130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1057fff700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f1058130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.547+0000 7f1057fff700 1 -- 172.21.15.89:0/3557229626 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105809b2d0 con 0x7f105809e410 2023-09-30T22:20:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f1056ffd700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f1058130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f99867bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009d6f0 0x7f99801a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f99867bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009e000 0x7f99801a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f99867bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e410 0x7f99801a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f99867bc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801a9cf0 con 0x7f998009e410 2023-09-30T22:20:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f99867bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99800a1fe0 con 0x7f998009e000 2023-09-30T22:20:21.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f99867bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f998009c710 con 0x7f998009d6f0 2023-09-30T22:20:21.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f1057fff700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 0x7f1058130ba0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1050009e10 tx=0x7f105000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1abdf79ef0d1cc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e410 0x7f99801a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e410 0x7f99801a6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58410/0 (socket says 172.21.15.89:58410) 2023-09-30T22:20:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 -- 172.21.15.89:0/3345914857 learned_addr learned my addr 172.21.15.89:0/3345914857 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f997f7fe700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009e000 0x7f99801a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f997ffff700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009d6f0 0x7f99801a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f10567fc700 1 -- 172.21.15.89:0/3557229626 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050017410 con 0x7f105809e410 2023-09-30T22:20:21.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f10567fc700 1 -- 172.21.15.89:0/3557229626 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050017570 con 0x7f105809e410 2023-09-30T22:20:21.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f10567fc700 1 -- 172.21.15.89:0/3557229626 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050017920 con 0x7f105809e410 2023-09-30T22:20:21.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 -- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009d6f0 msgr2=0x7f99801a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009d6f0 0x7f99801a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 -- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009e000 msgr2=0x7f99801a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009e000 0x7f99801a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 -- 172.21.15.89:0/3345914857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970009550 con 0x7f998009e410 2023-09-30T22:20:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.548+0000 7f9984d59700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e410 0x7f99801a6c50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f997800efc0 tx=0x7f997800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f997cff9700 1 -- 172.21.15.89:0/3345914857 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978009ea0 con 0x7f998009e410 2023-09-30T22:20:21.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f997cff9700 1 -- 172.21.15.89:0/3345914857 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997800f040 con 0x7f998009e410 2023-09-30T22:20:21.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f997cff9700 1 -- 172.21.15.89:0/3345914857 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978014480 con 0x7f998009e410 2023-09-30T22:20:21.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99740094a0 con 0x7f998009e410 2023-09-30T22:20:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 -- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 msgr2=0x7f1058130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 0x7f1058130ba0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1050009e10 tx=0x7f105000f740 comp rx=0 tx=0).stop 2023-09-30T22:20:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 -- 172.21.15.89:0/3557229626 shutdown_connections 2023-09-30T22:20:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 0x7f1058130ba0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 --2- 172.21.15.89:0/3557229626 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f1058130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 -- 172.21.15.89:0/3557229626 >> 172.21.15.89:0/3557229626 conn(0x7f1058066f50 msgr2=0x7f105809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 -- 172.21.15.89:0/3557229626 shutdown_connections 2023-09-30T22:20:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.549+0000 7f105e081700 1 -- 172.21.15.89:0/3557229626 wait complete. 2023-09-30T22:20:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.550+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99801a9480 con 0x7f998009e410 2023-09-30T22:20:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.550+0000 7f105e081700 1 Processor -- start 2023-09-30T22:20:21.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.550+0000 7f997cff9700 1 -- 172.21.15.89:0/3345914857 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99780145e0 con 0x7f998009e410 2023-09-30T22:20:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.550+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9980074ea0 con 0x7f998009e410 2023-09-30T22:20:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f105e081700 1 -- start start 2023-09-30T22:20:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f997cff9700 1 -- 172.21.15.89:0/3345914857 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f997803fee0 con 0x7f998009e410 2023-09-30T22:20:21.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f105e081700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105811e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f105e081700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f105811e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f105e081700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 0x7f105811eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:21.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f105e081700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105809c770 con 0x7f105809e410 2023-09-30T22:20:21.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f105e081700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580a1920 con 0x7f105809e000 2023-09-30T22:20:21.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f105e081700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10581201f0 con 0x7f105809d6f0 2023-09-30T22:20:21.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f10577fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105811e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f10577fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105811e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36532/0 (socket says 172.21.15.89:36532) 2023-09-30T22:20:21.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f10577fe700 1 -- 172.21.15.89:0/733081590 learned_addr learned my addr 172.21.15.89:0/733081590 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:21.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.551+0000 7f1056ffd700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f105811e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f1057fff700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 0x7f105811eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:21.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f10577fe700 1 -- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 msgr2=0x7f105811e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f10577fe700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f105811e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f10577fe700 1 -- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 msgr2=0x7f105811eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f10577fe700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 0x7f105811eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f10577fe700 1 -- 172.21.15.89:0/733081590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c009550 con 0x7f105809d6f0 2023-09-30T22:20:21.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f10577fe700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105811e030 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f104800e930 tx=0x7f104800ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:21.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f103ffff700 1 -- 172.21.15.89:0/733081590 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048017d80 con 0x7f105809d6f0 2023-09-30T22:20:21.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1050009a70 con 0x7f105809d6f0 2023-09-30T22:20:21.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.552+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10581215b0 con 0x7f105809d6f0 2023-09-30T22:20:21.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.553+0000 7f103ffff700 1 -- 172.21.15.89:0/733081590 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104801f440 con 0x7f105809d6f0 2023-09-30T22:20:21.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.553+0000 7f103ffff700 1 -- 172.21.15.89:0/733081590 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104801f7b0 con 0x7f105809d6f0 2023-09-30T22:20:21.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.554+0000 7f103ffff700 1 -- 172.21.15.89:0/733081590 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f104801f910 con 0x7f105809d6f0 2023-09-30T22:20:21.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.554+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10581213a0 con 0x7f105809d6f0 2023-09-30T22:20:21.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.554+0000 7f103ffff700 1 -- 172.21.15.89:0/733081590 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1048047a20 con 0x7f105809d6f0 2023-09-30T22:20:21.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.556+0000 7f997cff9700 1 -- 172.21.15.89:0/3345914857 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f997801a8b0 con 0x7f998009e410 2023-09-30T22:20:21.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.559+0000 7f103ffff700 1 -- 172.21.15.89:0/733081590 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1048024080 con 0x7f105809d6f0 2023-09-30T22:20:21.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.721+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2e3c06c360 con 0x7f2e3c09e000 2023-09-30T22:20:21.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.722+0000 7f2e23fff700 1 -- 172.21.15.89:0/2941701866 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e3001bc30 con 0x7f2e3c09e000 2023-09-30T22:20:21.724 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:21.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.725+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e3c09e000 msgr2=0x7f2e3c1c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.725+0000 7f2e419c1700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e3c09e000 0x7f2e3c1c8be0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e8d0 tx=0x7f2e3000ebe0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.725+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 shutdown_connections 2023-09-30T22:20:21.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.725+0000 7f2e419c1700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e3c09e410 0x7f2e3c1c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.725+0000 7f2e419c1700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e3c09d6f0 0x7f2e3c1c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.725+0000 7f2e419c1700 1 --2- 172.21.15.89:0/2941701866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e3c09e000 0x7f2e3c1c8be0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.725+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 >> 172.21.15.89:0/2941701866 conn(0x7f2e3c066f50 msgr2=0x7f2e3c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:21.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.726+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 shutdown_connections 2023-09-30T22:20:21.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.726+0000 7f2e419c1700 1 -- 172.21.15.89:0/2941701866 wait complete. 2023-09-30T22:20:21.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.751+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f998009cb20 con 0x7f998009e410 2023-09-30T22:20:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.751+0000 7f997cff9700 1 -- 172.21.15.89:0/3345914857 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9978018020 con 0x7f998009e410 2023-09-30T22:20:21.752 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1058074ea0 con 0x7f105809d6f0 2023-09-30T22:20:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f103ffff700 1 -- 172.21.15.89:0/733081590 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f104800c920 con 0x7f105809d6f0 2023-09-30T22:20:21.754 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e410 msgr2=0x7f99801a6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f99867bc700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e410 0x7f99801a6c50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f997800efc0 tx=0x7f997800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 shutdown_connections 2023-09-30T22:20:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f99867bc700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f998009e410 0x7f99801a6c50 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f99867bc700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f998009d6f0 0x7f99801a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f99867bc700 1 --2- 172.21.15.89:0/3345914857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f998009e000 0x7f99801a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.753+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 >> 172.21.15.89:0/3345914857 conn(0x7f9980066f50 msgr2=0x7f99801bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.754+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 shutdown_connections 2023-09-30T22:20:21.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.754+0000 7f99867bc700 1 -- 172.21.15.89:0/3345914857 wait complete. 2023-09-30T22:20:21.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 msgr2=0x7f105811e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105811e030 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f104800e930 tx=0x7f104800ec40 comp rx=0 tx=0).stop 2023-09-30T22:20:21.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 shutdown_connections 2023-09-30T22:20:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f105809e410 0x7f105811eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f105809d6f0 0x7f105811e030 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 --2- 172.21.15.89:0/733081590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f105809e000 0x7f105811e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 >> 172.21.15.89:0/733081590 conn(0x7f1058066f50 msgr2=0x7f105806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 shutdown_connections 2023-09-30T22:20:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.755+0000 7f105e081700 1 -- 172.21.15.89:0/733081590 wait complete. 2023-09-30T22:20:21.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.765+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc9740727c0 con 0x7fc97409f650 2023-09-30T22:20:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.765+0000 7fc970ff9700 1 -- 172.21.15.89:0/3603358661 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc96c02b090 con 0x7fc97409f650 2023-09-30T22:20:21.765 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.767+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 msgr2=0x7fc974123010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.767+0000 7fc97a851700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974123010 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00efc0 tx=0x7fc96c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.768+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 shutdown_connections 2023-09-30T22:20:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.768+0000 7fc97a851700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97409f650 0x7fc974123010 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.768+0000 7fc97a851700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc97409e930 0x7fc9741241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.768+0000 7fc97a851700 1 --2- 172.21.15.89:0/3603358661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97409f240 0x7fc974122ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.768+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 >> 172.21.15.89:0/3603358661 conn(0x7fc9740a4210 msgr2=0x7fc97409bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.768+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 shutdown_connections 2023-09-30T22:20:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:21.768+0000 7fc97a851700 1 -- 172.21.15.89:0/3603358661 wait complete. 2023-09-30T22:20:21.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:21.772 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:21.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:22.738 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:22.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:22.774 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:22.781 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.879+0000 7f7884466700 1 Processor -- start 2023-09-30T22:20:22.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.880+0000 7f7884466700 1 -- start start 2023-09-30T22:20:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.880+0000 7f7884466700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09e930 0x7f787c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.880+0000 7f7884466700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09f240 0x7f787c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7884466700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c070940 con 0x7f787c09f650 2023-09-30T22:20:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7884466700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c06fd60 con 0x7f787c09f240 2023-09-30T22:20:22.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7884466700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787c06fa80 con 0x7f787c09e930 2023-09-30T22:20:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7882a03700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f787c09f650 0x7f787c131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58556/0 (socket says 172.21.15.89:58556) 2023-09-30T22:20:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7882a03700 1 -- 172.21.15.89:0/3173348744 learned_addr learned my addr 172.21.15.89:0/3173348744 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7881a01700 1 --2- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09f240 0x7f787c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7882202700 1 --2- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09e930 0x7f787c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7881200700 1 -- 172.21.15.89:0/3173348744 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258425009 0 0) 0x7f787c070940 con 0x7f787c09f650 2023-09-30T22:20:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7881a01700 1 -- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09e930 msgr2=0x7f787c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7881a01700 1 --2- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09e930 0x7f787c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7881a01700 1 -- 172.21.15.89:0/3173348744 >> v1:172.21.15.89:6789/0 conn(0x7f787c09f650 legacy=0x7f787c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.881+0000 7f7881a01700 1 -- 172.21.15.89:0/3173348744 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c09fb60 con 0x7f787c09f240 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.882+0000 7f7881a01700 1 --2- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09f240 0x7f787c1317f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f786c009870 tx=0x7f786c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38aedb7edcde2d27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.882+0000 7f7881200700 1 -- 172.21.15.89:0/3173348744 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c011830 con 0x7f787c09f240 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.882+0000 7f7881200700 1 -- 172.21.15.89:0/3173348744 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c011990 con 0x7f787c09f240 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.882+0000 7f7881200700 1 -- 172.21.15.89:0/3173348744 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c011d20 con 0x7f787c09f240 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.883+0000 7f7884466700 1 -- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09f240 msgr2=0x7f787c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.883+0000 7f7884466700 1 --2- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09f240 0x7f787c1317f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f786c009870 tx=0x7f786c009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.883+0000 7f7884466700 1 -- 172.21.15.89:0/3173348744 shutdown_connections 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.883+0000 7f7884466700 1 --2- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09e930 0x7f787c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.883+0000 7f7884466700 1 --2- 172.21.15.89:0/3173348744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09f240 0x7f787c1317f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.883+0000 7f7884466700 1 -- 172.21.15.89:0/3173348744 >> 172.21.15.89:0/3173348744 conn(0x7f787c0a4210 msgr2=0x7f787c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.883+0000 7f7884466700 1 -- 172.21.15.89:0/3173348744 shutdown_connections 2023-09-30T22:20:22.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.883+0000 7f7884466700 1 -- 172.21.15.89:0/3173348744 wait complete. 2023-09-30T22:20:22.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.884+0000 7f7884466700 1 Processor -- start 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7884466700 1 -- start start 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7884466700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09e930 0x7f787c1a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7884466700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09f240 0x7f787c1a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7884466700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f787c09f650 0x7f787c1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7884466700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1a8100 con 0x7f787c09f650 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7884466700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1a5830 con 0x7f787c09e930 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7884466700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1a5e50 con 0x7f787c09f240 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7882202700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09e930 0x7f787c1a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7881a01700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09f240 0x7f787c1a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7882a03700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f787c09f650 0x7f787c1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7881a01700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09f240 0x7f787c1a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36556/0 (socket says 172.21.15.89:36556) 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7882202700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09e930 0x7f787c1a3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58704/0 (socket says 172.21.15.89:58704) 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.885+0000 7f7881a01700 1 -- 172.21.15.89:0/1890876641 learned_addr learned my addr 172.21.15.89:0/1890876641 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7881a01700 1 -- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09e930 msgr2=0x7f787c1a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7881a01700 1 --2- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09e930 0x7f787c1a3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7881a01700 1 -- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f787c09f650 msgr2=0x7f787c1a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7881a01700 1 --2- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f787c09f650 0x7f787c1a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7881a01700 1 -- 172.21.15.89:0/1890876641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0094a0 con 0x7f787c09f240 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7881a01700 1 --2- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09f240 0x7f787c1a4160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f786c013000 tx=0x7f786c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7872ffd700 1 -- 172.21.15.89:0/1890876641 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c004510 con 0x7f787c09f240 2023-09-30T22:20:22.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7872ffd700 1 -- 172.21.15.89:0/1890876641 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c01d7c0 con 0x7f787c09f240 2023-09-30T22:20:22.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c0a0190 con 0x7f787c09f240 2023-09-30T22:20:22.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7872ffd700 1 -- 172.21.15.89:0/1890876641 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c01db70 con 0x7f787c09f240 2023-09-30T22:20:22.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.886+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c0a33f0 con 0x7f787c09f240 2023-09-30T22:20:22.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.887+0000 7f7872ffd700 1 -- 172.21.15.89:0/1890876641 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f786c01dcd0 con 0x7f787c09f240 2023-09-30T22:20:22.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.888+0000 7f7872ffd700 1 -- 172.21.15.89:0/1890876641 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f786c0449b0 con 0x7f787c09f240 2023-09-30T22:20:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.888+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c1a0b30 con 0x7f787c09f240 2023-09-30T22:20:22.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.893+0000 7f7872ffd700 1 -- 172.21.15.89:0/1890876641 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f786c015070 con 0x7f787c09f240 2023-09-30T22:20:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.912+0000 7f5834267700 1 Processor -- start 2023-09-30T22:20:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.912+0000 7f5834267700 1 -- start start 2023-09-30T22:20:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.913+0000 7fc03e94b700 1 Processor -- start 2023-09-30T22:20:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.913+0000 7fc03e94b700 1 -- start start 2023-09-30T22:20:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.913+0000 7f5834267700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f582c134630 0x7f582c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.913+0000 7f5834267700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c136f50 0x7f582c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.913+0000 7f5834267700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c09b4f0 con 0x7f582c131810 2023-09-30T22:20:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.913+0000 7f5834267700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c1313c0 con 0x7f582c134630 2023-09-30T22:20:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.913+0000 7f5834267700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f582c06e8c0 con 0x7f582c136f50 2023-09-30T22:20:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.913+0000 7f5832804700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c136f50 0x7f582c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5832804700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c136f50 0x7f582c139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36564/0 (socket says 172.21.15.89:36564) 2023-09-30T22:20:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5832804700 1 -- 172.21.15.89:0/2457234447 learned_addr learned my addr 172.21.15.89:0/2457234447 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc03e94b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc038131810 0x7fc038131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc03e94b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc038099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5831802700 1 --2- 172.21.15.89:0/2457234447 >> v2:172.21.15.89:3300/0 conn(0x7f582c134630 0x7f582c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc03e94b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0381313c0 con 0x7fc03809a3b0 2023-09-30T22:20:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc03e94b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03809b580 con 0x7fc038131810 2023-09-30T22:20:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc03e94b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc03806e8c0 con 0x7fc038099a40 2023-09-30T22:20:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc03cee8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc03809a3b0 0x7fc038138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58584/0 (socket says 172.21.15.89:58584) 2023-09-30T22:20:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc03cee8700 1 -- 172.21.15.89:0/3403091727 learned_addr learned my addr 172.21.15.89:0/3403091727 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3403091727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc038099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5832804700 1 -- 172.21.15.89:0/2457234447 >> v2:172.21.15.89:3300/0 conn(0x7f582c134630 msgr2=0x7f582c136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5832804700 1 --2- 172.21.15.89:0/2457234447 >> v2:172.21.15.89:3300/0 conn(0x7f582c134630 0x7f582c136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5832804700 1 -- 172.21.15.89:0/2457234447 >> v1:172.21.15.89:6789/0 conn(0x7f582c131810 legacy=0x7f582c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5832804700 1 -- 172.21.15.89:0/2457234447 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c071d20 con 0x7f582c136f50 2023-09-30T22:20:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5832804700 1 --2- 172.21.15.89:0/2457234447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c136f50 0x7f582c139330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5824009e50 tx=0x7f582400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a8a7cef09f8d426 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc0377fe700 1 -- 172.21.15.89:0/3403091727 >> v2:172.21.15.89:3300/0 conn(0x7fc038131810 msgr2=0x7fc038131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3403091727 >> v2:172.21.15.89:3300/0 conn(0x7fc038131810 0x7fc038131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc0377fe700 1 -- 172.21.15.89:0/3403091727 >> v1:172.21.15.89:6789/0 conn(0x7fc03809a3b0 legacy=0x7fc038138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7fc0377fe700 1 -- 172.21.15.89:0/3403091727 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03809a830 con 0x7fc038099a40 2023-09-30T22:20:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.915+0000 7fc0377fe700 1 --2- 172.21.15.89:0/3403091727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc038099e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009870 tx=0x7fc02c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=145454ef87d7723c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.914+0000 7f5831001700 1 -- 172.21.15.89:0/2457234447 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400e070 con 0x7f582c136f50 2023-09-30T22:20:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.915+0000 7f5831001700 1 -- 172.21.15.89:0/2457234447 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5824003a20 con 0x7f582c136f50 2023-09-30T22:20:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.915+0000 7f5831001700 1 -- 172.21.15.89:0/2457234447 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824003db0 con 0x7f582c136f50 2023-09-30T22:20:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.915+0000 7fc036ffd700 1 -- 172.21.15.89:0/3403091727 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c011830 con 0x7fc038099a40 2023-09-30T22:20:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.915+0000 7fc036ffd700 1 -- 172.21.15.89:0/3403091727 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c011990 con 0x7fc038099a40 2023-09-30T22:20:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.915+0000 7fc036ffd700 1 -- 172.21.15.89:0/3403091727 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c011d20 con 0x7fc038099a40 2023-09-30T22:20:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7f5834267700 1 -- 172.21.15.89:0/2457234447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c136f50 msgr2=0x7f582c139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7f5834267700 1 --2- 172.21.15.89:0/2457234447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c136f50 0x7f582c139330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5824009e50 tx=0x7f582400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7fc03e94b700 1 -- 172.21.15.89:0/3403091727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 msgr2=0x7fc038099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7fc03e94b700 1 --2- 172.21.15.89:0/3403091727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc038099e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009870 tx=0x7fc02c009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7fc03e94b700 1 -- 172.21.15.89:0/3403091727 shutdown_connections 2023-09-30T22:20:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7fc03e94b700 1 --2- 172.21.15.89:0/3403091727 >> v2:172.21.15.89:3300/0 conn(0x7fc038131810 0x7fc038131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7fc03e94b700 1 --2- 172.21.15.89:0/3403091727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc038099e70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7fc03e94b700 1 -- 172.21.15.89:0/3403091727 >> 172.21.15.89:0/3403091727 conn(0x7fc0380a2aa0 msgr2=0x7fc0380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7fc03e94b700 1 -- 172.21.15.89:0/3403091727 shutdown_connections 2023-09-30T22:20:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7f5834267700 1 -- 172.21.15.89:0/2457234447 shutdown_connections 2023-09-30T22:20:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7f5834267700 1 --2- 172.21.15.89:0/2457234447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c136f50 0x7f582c139330 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7f5834267700 1 --2- 172.21.15.89:0/2457234447 >> v2:172.21.15.89:3300/0 conn(0x7f582c134630 0x7f582c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7f5834267700 1 -- 172.21.15.89:0/2457234447 >> 172.21.15.89:0/2457234447 conn(0x7f582c0a2aa0 msgr2=0x7f582c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7fc03e94b700 1 -- 172.21.15.89:0/3403091727 wait complete. 2023-09-30T22:20:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7f5834267700 1 -- 172.21.15.89:0/2457234447 shutdown_connections 2023-09-30T22:20:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.916+0000 7f5834267700 1 -- 172.21.15.89:0/2457234447 wait complete. 2023-09-30T22:20:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.917+0000 7fc03e94b700 1 Processor -- start 2023-09-30T22:20:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.917+0000 7f5834267700 1 Processor -- start 2023-09-30T22:20:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.917+0000 7f5834267700 1 -- start start 2023-09-30T22:20:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5834267700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c131810 0x7f582c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5834267700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f582c134630 0x7f582c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5834267700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f582c136f50 0x7f582c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5834267700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c11c2c0 con 0x7f582c136f50 2023-09-30T22:20:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5834267700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c11c4a0 con 0x7f582c134630 2023-09-30T22:20:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5834267700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f582c136c90 con 0x7f582c131810 2023-09-30T22:20:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832003700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c131810 0x7f582c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832003700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c131810 0x7f582c118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36568/0 (socket says 172.21.15.89:36568) 2023-09-30T22:20:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832003700 1 -- 172.21.15.89:0/3222379468 learned_addr learned my addr 172.21.15.89:0/3222379468 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832804700 1 --2- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f582c136f50 0x7f582c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.917+0000 7fc03e94b700 1 -- start start 2023-09-30T22:20:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc03e94b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc03811b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc03e94b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc03809a3b0 0x7fc03811bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc03e94b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038131810 0x7fc03811c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc03e94b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03811cd70 con 0x7fc038131810 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc03e94b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03812ebf0 con 0x7fc03809a3b0 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc03e94b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc03811e660 con 0x7fc038099a40 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc037fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc03811b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc037fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc03811b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36584/0 (socket says 172.21.15.89:36584) 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc037fff700 1 -- 172.21.15.89:0/1132644111 learned_addr learned my addr 172.21.15.89:0/1132644111 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc03cee8700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038131810 0x7fc03811c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc037fff700 1 -- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc03809a3b0 msgr2=0x7fc03811bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc037fff700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc03809a3b0 0x7fc03811bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc037fff700 1 -- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038131810 msgr2=0x7fc03811c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc037fff700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038131810 0x7fc03811c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc037fff700 1 -- 172.21.15.89:0/1132644111 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0094a0 con 0x7fc038099a40 2023-09-30T22:20:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832804700 1 -- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c131810 msgr2=0x7f582c118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832804700 1 --2- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c131810 0x7f582c118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832804700 1 -- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f582c134630 msgr2=0x7f582c119210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832804700 1 --2- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f582c134630 0x7f582c119210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7f5832804700 1 -- 172.21.15.89:0/3222379468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5824009b30 con 0x7f582c136f50 2023-09-30T22:20:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.918+0000 7fc03cee8700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038131810 0x7fc03811c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7fc037fff700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc03811b8a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b530 tx=0x7fc02800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7f5832804700 1 --2- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f582c136f50 0x7f582c119750 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f582c06dd30 tx=0x7f5824004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7f5822ffd700 1 -- 172.21.15.89:0/3222379468 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824020070 con 0x7f582c136f50 2023-09-30T22:20:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7fc034ff9700 1 -- 172.21.15.89:0/1132644111 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028011830 con 0x7fc038099a40 2023-09-30T22:20:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7f5822ffd700 1 -- 172.21.15.89:0/3222379468 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58240092e0 con 0x7f582c136f50 2023-09-30T22:20:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7f5822ffd700 1 -- 172.21.15.89:0/3222379468 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582401ba50 con 0x7f582c136f50 2023-09-30T22:20:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7fc034ff9700 1 -- 172.21.15.89:0/1132644111 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0280119b0 con 0x7fc038099a40 2023-09-30T22:20:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7fc034ff9700 1 -- 172.21.15.89:0/1132644111 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028011d60 con 0x7fc038099a40 2023-09-30T22:20:22.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03811f0a0 con 0x7fc038099a40 2023-09-30T22:20:22.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.919+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0380a1cb0 con 0x7fc038099a40 2023-09-30T22:20:22.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.920+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c06eb00 con 0x7f582c136f50 2023-09-30T22:20:22.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.920+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c11cf10 con 0x7f582c136f50 2023-09-30T22:20:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.920+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03806eb00 con 0x7fc038099a40 2023-09-30T22:20:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.920+0000 7fc034ff9700 1 -- 172.21.15.89:0/1132644111 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc02801a400 con 0x7fc038099a40 2023-09-30T22:20:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.921+0000 7fc034ff9700 1 -- 172.21.15.89:0/1132644111 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc02803f6a0 con 0x7fc038099a40 2023-09-30T22:20:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.921+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c11c0c0 con 0x7f582c136f50 2023-09-30T22:20:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.921+0000 7f5822ffd700 1 -- 172.21.15.89:0/3222379468 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f582401bbb0 con 0x7f582c136f50 2023-09-30T22:20:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.921+0000 7f5822ffd700 1 -- 172.21.15.89:0/3222379468 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5824044230 con 0x7f582c136f50 2023-09-30T22:20:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.925+0000 7fc034ff9700 1 -- 172.21.15.89:0/1132644111 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc02801ae60 con 0x7fc038099a40 2023-09-30T22:20:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.927+0000 7f5822ffd700 1 -- 172.21.15.89:0/3222379468 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5824004030 con 0x7f582c136f50 2023-09-30T22:20:22.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.934+0000 7f2b4f788700 1 Processor -- start 2023-09-30T22:20:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.934+0000 7f2b4f788700 1 -- start start 2023-09-30T22:20:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4f788700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b4806e570 0x7f2b4806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4f788700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b4809eca0 0x7f2b4806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4f788700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b48134610 0x7f2b48134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4f788700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b48070ee0 con 0x7f2b4809eca0 2023-09-30T22:20:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4f788700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48071480 con 0x7f2b4806e570 2023-09-30T22:20:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4f788700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4806f3c0 con 0x7f2b48134610 2023-09-30T22:20:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4dd25700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b48134610 0x7f2b48134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4dd25700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b48134610 0x7f2b48134a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36596/0 (socket says 172.21.15.89:36596) 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4dd25700 1 -- 172.21.15.89:0/1241872354 learned_addr learned my addr 172.21.15.89:0/1241872354 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.935+0000 7f2b4d524700 1 --2- 172.21.15.89:0/1241872354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b4806e570 0x7f2b4806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b4cd23700 1 --2- 172.21.15.89:0/1241872354 >> v2:172.21.15.89:3300/0 conn(0x7f2b4809eca0 0x7f2b4806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b4dd25700 1 -- 172.21.15.89:0/1241872354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b4806e570 msgr2=0x7f2b4806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b4dd25700 1 --2- 172.21.15.89:0/1241872354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b4806e570 0x7f2b4806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b4dd25700 1 -- 172.21.15.89:0/1241872354 >> v2:172.21.15.89:3300/0 conn(0x7f2b4809eca0 msgr2=0x7f2b4806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b4dd25700 1 --2- 172.21.15.89:0/1241872354 >> v2:172.21.15.89:3300/0 conn(0x7f2b4809eca0 0x7f2b4806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b4dd25700 1 -- 172.21.15.89:0/1241872354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48071b90 con 0x7f2b48134610 2023-09-30T22:20:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b4dd25700 1 --2- 172.21.15.89:0/1241872354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b48134610 0x7f2b48134a40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009e50 tx=0x7f2b4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e45e1f3e2d58a03 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b3ffff700 1 -- 172.21.15.89:0/1241872354 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000e070 con 0x7f2b48134610 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b3ffff700 1 -- 172.21.15.89:0/1241872354 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4000b910 con 0x7f2b48134610 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.936+0000 7f2b3ffff700 1 -- 172.21.15.89:0/1241872354 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000bcc0 con 0x7f2b48134610 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 -- 172.21.15.89:0/1241872354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b48134610 msgr2=0x7f2b48134a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 --2- 172.21.15.89:0/1241872354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b48134610 0x7f2b48134a40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009e50 tx=0x7f2b4000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 -- 172.21.15.89:0/1241872354 shutdown_connections 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 --2- 172.21.15.89:0/1241872354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b48134610 0x7f2b48134a40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 --2- 172.21.15.89:0/1241872354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b4806e570 0x7f2b4806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 --2- 172.21.15.89:0/1241872354 >> v2:172.21.15.89:3300/0 conn(0x7f2b4809eca0 0x7f2b4806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 -- 172.21.15.89:0/1241872354 >> 172.21.15.89:0/1241872354 conn(0x7f2b480a2aa0 msgr2=0x7f2b480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 -- 172.21.15.89:0/1241872354 shutdown_connections 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.937+0000 7f2b4f788700 1 -- 172.21.15.89:0/1241872354 wait complete. 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.938+0000 7f2b4f788700 1 Processor -- start 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.938+0000 7f2b4f788700 1 -- start start 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.938+0000 7f2b4f788700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b4806e570 0x7f2b48127410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.938+0000 7f2b4f788700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b48134610 0x7f2b48139cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.938+0000 7f2b4f788700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48127950 0x7f2b4813a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.938+0000 7f2b4f788700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4813d140 con 0x7f2b48127950 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.938+0000 7f2b4f788700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48070610 con 0x7f2b48134610 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.938+0000 7f2b4f788700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4813e600 con 0x7f2b4806e570 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4d524700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b4806e570 0x7f2b48127410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4d524700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b4806e570 0x7f2b48127410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36602/0 (socket says 172.21.15.89:36602) 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4d524700 1 -- 172.21.15.89:0/1082921173 learned_addr learned my addr 172.21.15.89:0/1082921173 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4cd23700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b48134610 0x7f2b48139cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4dd25700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48127950 0x7f2b4813a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4dd25700 1 -- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b4806e570 msgr2=0x7f2b48127410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4dd25700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b4806e570 0x7f2b48127410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4dd25700 1 -- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b48134610 msgr2=0x7f2b48139cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4dd25700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b48134610 0x7f2b48139cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4dd25700 1 -- 172.21.15.89:0/1082921173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40009b30 con 0x7f2b48127950 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.939+0000 7f2b4dd25700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48127950 0x7f2b4813a210 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2b40015000 tx=0x7f2b40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.940+0000 7f2b3dffb700 1 -- 172.21.15.89:0/1082921173 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40004450 con 0x7f2b48127950 2023-09-30T22:20:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.940+0000 7f2b3dffb700 1 -- 172.21.15.89:0/1082921173 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b400045d0 con 0x7f2b48127950 2023-09-30T22:20:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.940+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4813d400 con 0x7f2b48127950 2023-09-30T22:20:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.940+0000 7f2b3dffb700 1 -- 172.21.15.89:0/1082921173 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b400092e0 con 0x7f2b48127950 2023-09-30T22:20:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.940+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4813de70 con 0x7f2b48127950 2023-09-30T22:20:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.941+0000 7f2b3dffb700 1 -- 172.21.15.89:0/1082921173 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b40025460 con 0x7f2b48127950 2023-09-30T22:20:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.941+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4809d1f0 con 0x7f2b48127950 2023-09-30T22:20:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.941+0000 7f2b3dffb700 1 -- 172.21.15.89:0/1082921173 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2b40041100 con 0x7f2b48127950 2023-09-30T22:20:22.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:22.950+0000 7f2b3dffb700 1 -- 172.21.15.89:0/1082921173 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b40025710 con 0x7f2b48127950 2023-09-30T22:20:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.104+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f787c071d20 con 0x7f787c09f240 2023-09-30T22:20:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.104+0000 7f7872ffd700 1 -- 172.21.15.89:0/1890876641 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f786c030770 con 0x7f787c09f240 2023-09-30T22:20:23.105 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09f240 msgr2=0x7f787c1a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 --2- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09f240 0x7f787c1a4160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f786c013000 tx=0x7f786c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 shutdown_connections 2023-09-30T22:20:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 --2- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f787c09f650 0x7f787c1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 --2- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f787c09e930 0x7f787c1a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 --2- 172.21.15.89:0/1890876641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f787c09f240 0x7f787c1a4160 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 >> 172.21.15.89:0/1890876641 conn(0x7f787c0a4210 msgr2=0x7f787c1b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 shutdown_connections 2023-09-30T22:20:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.106+0000 7f7884466700 1 -- 172.21.15.89:0/1890876641 wait complete. 2023-09-30T22:20:23.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:23.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.120+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc038074ea0 con 0x7fc038099a40 2023-09-30T22:20:23.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.120+0000 7fc034ff9700 1 -- 172.21.15.89:0/1132644111 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc028015020 con 0x7fc038099a40 2023-09-30T22:20:23.122 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:23.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.122+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 msgr2=0x7fc03811b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:23.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.122+0000 7fc03e94b700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc03811b8a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b530 tx=0x7fc02800b8f0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.122+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 shutdown_connections 2023-09-30T22:20:23.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.122+0000 7fc03e94b700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc038131810 0x7fc03811c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.122+0000 7fc03e94b700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc038099a40 0x7fc03811b8a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.122+0000 7fc03e94b700 1 --2- 172.21.15.89:0/1132644111 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc03809a3b0 0x7fc03811bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.122+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 >> 172.21.15.89:0/1132644111 conn(0x7fc0380a2aa0 msgr2=0x7fc038130220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:23.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.124+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 shutdown_connections 2023-09-30T22:20:23.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.124+0000 7fc03e94b700 1 -- 172.21.15.89:0/1132644111 wait complete. 2023-09-30T22:20:23.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.131+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f582c074ea0 con 0x7f582c136f50 2023-09-30T22:20:23.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.131+0000 7f5822ffd700 1 -- 172.21.15.89:0/3222379468 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f582402a020 con 0x7f582c136f50 2023-09-30T22:20:23.132 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f582c136f50 msgr2=0x7f582c119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 --2- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f582c136f50 0x7f582c119750 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f582c06dd30 tx=0x7f5824004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 shutdown_connections 2023-09-30T22:20:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 --2- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f582c136f50 0x7f582c119750 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 --2- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f582c131810 0x7f582c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 --2- 172.21.15.89:0/3222379468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f582c134630 0x7f582c119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 >> 172.21.15.89:0/3222379468 conn(0x7f582c0a2aa0 msgr2=0x7f582c1300d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 shutdown_connections 2023-09-30T22:20:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.133+0000 7f5834267700 1 -- 172.21.15.89:0/3222379468 wait complete. 2023-09-30T22:20:23.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.145+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2b48134f60 con 0x7f2b48127950 2023-09-30T22:20:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.145+0000 7f2b3dffb700 1 -- 172.21.15.89:0/1082921173 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b48134f60 con 0x7f2b48127950 2023-09-30T22:20:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:23.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:23.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.147+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48127950 msgr2=0x7f2b4813a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:23.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.147+0000 7f2b4f788700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48127950 0x7f2b4813a210 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2b40015000 tx=0x7f2b40004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.147+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 shutdown_connections 2023-09-30T22:20:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.147+0000 7f2b4f788700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48127950 0x7f2b4813a210 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.147+0000 7f2b4f788700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b4806e570 0x7f2b48127410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.147+0000 7f2b4f788700 1 --2- 172.21.15.89:0/1082921173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b48134610 0x7f2b48139cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.147+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 >> 172.21.15.89:0/1082921173 conn(0x7f2b480a2aa0 msgr2=0x7f2b48137b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.148+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 shutdown_connections 2023-09-30T22:20:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:23.148+0000 7f2b4f788700 1 -- 172.21.15.89:0/1082921173 wait complete. 2023-09-30T22:20:23.160 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:24.121 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:24.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:24.148 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:24.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89faddb700 1 Processor -- start 2023-09-30T22:20:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89faddb700 1 -- start start 2023-09-30T22:20:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89faddb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89faddb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f89f409e000 0x7f89f41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89faddb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f409b120 con 0x7f89f409e410 2023-09-30T22:20:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89faddb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f406b460 con 0x7f89f409e000 2023-09-30T22:20:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89faddb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89f409a7e0 con 0x7f89f409d6f0 2023-09-30T22:20:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89f8b77700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89f8b77700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36610/0 (socket says 172.21.15.89:36610) 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.266+0000 7f89f8b77700 1 -- 172.21.15.89:0/75693556 learned_addr learned my addr 172.21.15.89:0/75693556 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f3fff700 1 --2- 172.21.15.89:0/75693556 >> v2:172.21.15.89:3300/0 conn(0x7f89f409e000 0x7f89f41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f3fff700 1 -- 172.21.15.89:0/75693556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 msgr2=0x7f89f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f3fff700 1 --2- 172.21.15.89:0/75693556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f3fff700 1 -- 172.21.15.89:0/75693556 >> v1:172.21.15.89:6789/0 conn(0x7f89f409e410 legacy=0x7f89f4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f3fff700 1 -- 172.21.15.89:0/75693556 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f409a500 con 0x7f89f409e000 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f3fff700 1 --2- 172.21.15.89:0/75693556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e000 0x7f89f41339f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009840 tx=0x7f89e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a768784ed9c5ab19 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f37fe700 1 -- 172.21.15.89:0/75693556 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0011830 con 0x7f89f409e000 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f37fe700 1 -- 172.21.15.89:0/75693556 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e0011990 con 0x7f89f409e000 2023-09-30T22:20:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.267+0000 7f89f37fe700 1 -- 172.21.15.89:0/75693556 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0011d20 con 0x7f89f409e000 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.268+0000 7f89faddb700 1 -- 172.21.15.89:0/75693556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e000 msgr2=0x7f89f41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.268+0000 7f89faddb700 1 --2- 172.21.15.89:0/75693556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e000 0x7f89f41339f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009840 tx=0x7f89e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.268+0000 7f89faddb700 1 -- 172.21.15.89:0/75693556 shutdown_connections 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.268+0000 7f89faddb700 1 --2- 172.21.15.89:0/75693556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.268+0000 7f89faddb700 1 --2- 172.21.15.89:0/75693556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e000 0x7f89f41339f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.268+0000 7f89faddb700 1 -- 172.21.15.89:0/75693556 >> 172.21.15.89:0/75693556 conn(0x7f89f4066f50 msgr2=0x7f89f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.268+0000 7f89faddb700 1 -- 172.21.15.89:0/75693556 shutdown_connections 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.268+0000 7f89faddb700 1 -- 172.21.15.89:0/75693556 wait complete. 2023-09-30T22:20:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.269+0000 7f89faddb700 1 Processor -- start 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.269+0000 7f89faddb700 1 -- start start 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89faddb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f41a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89faddb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f409e000 0x7f89f41a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89faddb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e410 0x7f89f41a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89faddb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41a8800 con 0x7f89f409e410 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89faddb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41a6510 con 0x7f89f409e000 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89faddb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40a2fd0 con 0x7f89f409d6f0 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f8b77700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f41a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f9378700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e410 0x7f89f41a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f8b77700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f41a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36614/0 (socket says 172.21.15.89:36614) 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f8b77700 1 -- 172.21.15.89:0/29769437 learned_addr learned my addr 172.21.15.89:0/29769437 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f3fff700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f409e000 0x7f89f41a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f8b77700 1 -- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f409e000 msgr2=0x7f89f41a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f8b77700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f409e000 0x7f89f41a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f8b77700 1 -- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e410 msgr2=0x7f89f41a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f8b77700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e410 0x7f89f41a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f8b77700 1 -- 172.21.15.89:0/29769437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e00094a0 con 0x7f89f409d6f0 2023-09-30T22:20:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.270+0000 7f89f9378700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e410 0x7f89f41a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.271+0000 7f89f8b77700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f41a5120 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f89e800e810 tx=0x7f89e800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.271+0000 7f89f17fa700 1 -- 172.21.15.89:0/29769437 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8013610 con 0x7f89f409d6f0 2023-09-30T22:20:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.271+0000 7f89f17fa700 1 -- 172.21.15.89:0/29769437 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8013770 con 0x7f89f409d6f0 2023-09-30T22:20:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.271+0000 7f89f17fa700 1 -- 172.21.15.89:0/29769437 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8013b20 con 0x7f89f409d6f0 2023-09-30T22:20:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.271+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f41a6ce0 con 0x7f89f409d6f0 2023-09-30T22:20:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.271+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f41a7e20 con 0x7f89f409d6f0 2023-09-30T22:20:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.272+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f4074ea0 con 0x7f89f409d6f0 2023-09-30T22:20:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.272+0000 7f89f17fa700 1 -- 172.21.15.89:0/29769437 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89e8013c80 con 0x7f89f409d6f0 2023-09-30T22:20:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.272+0000 7f89f17fa700 1 -- 172.21.15.89:0/29769437 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f89e803f3f0 con 0x7f89f409d6f0 2023-09-30T22:20:24.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.277+0000 7f89f17fa700 1 -- 172.21.15.89:0/29769437 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89e8012370 con 0x7f89f409d6f0 2023-09-30T22:20:24.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.301+0000 7f33645a1700 1 Processor -- start 2023-09-30T22:20:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.301+0000 7f33645a1700 1 -- start start 2023-09-30T22:20:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f33645a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133420 0x7f335c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f33645a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f335c133e30 0x7f335c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f33645a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c06b990 con 0x7f335c134260 2023-09-30T22:20:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f33645a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335c0a0590 con 0x7f335c133e30 2023-09-30T22:20:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f33645a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335c09fb80 con 0x7f335c133420 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f336233d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133420 0x7f335c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f336233d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133420 0x7f335c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36626/0 (socket says 172.21.15.89:36626) 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f336233d700 1 -- 172.21.15.89:0/3002631949 learned_addr learned my addr 172.21.15.89:0/3002631949 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f3361b3c700 1 --2- 172.21.15.89:0/3002631949 >> v2:172.21.15.89:3300/0 conn(0x7f335c133e30 0x7f335c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f336233d700 1 -- 172.21.15.89:0/3002631949 >> v2:172.21.15.89:3300/0 conn(0x7f335c133e30 msgr2=0x7f335c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f336233d700 1 --2- 172.21.15.89:0/3002631949 >> v2:172.21.15.89:3300/0 conn(0x7f335c133e30 0x7f335c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f336233d700 1 -- 172.21.15.89:0/3002631949 >> v1:172.21.15.89:6789/0 conn(0x7f335c134260 legacy=0x7f335c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f336233d700 1 -- 172.21.15.89:0/3002631949 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0a0030 con 0x7f335c133420 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f3361b3c700 1 --2- 172.21.15.89:0/3002631949 >> v2:172.21.15.89:3300/0 conn(0x7f335c133e30 0x7f335c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.302+0000 7f336233d700 1 --2- 172.21.15.89:0/3002631949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133420 0x7f335c1337f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f33580098f0 tx=0x7f3358009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64562436156fc8c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f336133b700 1 -- 172.21.15.89:0/3002631949 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358011830 con 0x7f335c133420 2023-09-30T22:20:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f336133b700 1 -- 172.21.15.89:0/3002631949 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3358011990 con 0x7f335c133420 2023-09-30T22:20:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f336133b700 1 -- 172.21.15.89:0/3002631949 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358011d20 con 0x7f335c133420 2023-09-30T22:20:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f33645a1700 1 -- 172.21.15.89:0/3002631949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133420 msgr2=0x7f335c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f33645a1700 1 --2- 172.21.15.89:0/3002631949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133420 0x7f335c1337f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f33580098f0 tx=0x7f3358009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7fbe6240f700 1 Processor -- start 2023-09-30T22:20:24.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f33645a1700 1 -- 172.21.15.89:0/3002631949 shutdown_connections 2023-09-30T22:20:24.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f33645a1700 1 --2- 172.21.15.89:0/3002631949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133420 0x7f335c1337f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f33645a1700 1 --2- 172.21.15.89:0/3002631949 >> v2:172.21.15.89:3300/0 conn(0x7f335c133e30 0x7f335c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7f33645a1700 1 -- 172.21.15.89:0/3002631949 >> 172.21.15.89:0/3002631949 conn(0x7f335c0a4210 msgr2=0x7f335c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.303+0000 7fbe6240f700 1 -- start start 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe6240f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c0a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe6240f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c13d130 0x7fbe5c13f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe6240f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe5c0a27e0 con 0x7fbe5c06ccc0 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe6240f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c0a1dc0 con 0x7fbe5c13d130 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe6240f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c09dde0 con 0x7fbe5c0a2360 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe5bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbe5c06ccc0 0x7fbe5c0a0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58602/0 (socket says 172.21.15.89:58602) 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe609ac700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c13d130 0x7fbe5c13f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe5bfff700 1 -- 172.21.15.89:0/1299765114 learned_addr learned my addr 172.21.15.89:0/1299765114 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:24.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7fbe5b7fe700 1 --2- 172.21.15.89:0/1299765114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c0a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7f33645a1700 1 -- 172.21.15.89:0/3002631949 shutdown_connections 2023-09-30T22:20:24.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.304+0000 7f33645a1700 1 -- 172.21.15.89:0/3002631949 wait complete. 2023-09-30T22:20:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.305+0000 7fbe5b7fe700 1 -- 172.21.15.89:0/1299765114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c13d130 msgr2=0x7fbe5c13f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.305+0000 7fbe5b7fe700 1 --2- 172.21.15.89:0/1299765114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c13d130 0x7fbe5c13f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.305+0000 7fbe5b7fe700 1 -- 172.21.15.89:0/1299765114 >> v1:172.21.15.89:6789/0 conn(0x7fbe5c06ccc0 legacy=0x7fbe5c0a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.305+0000 7fbe5b7fe700 1 -- 172.21.15.89:0/1299765114 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c071b00 con 0x7fbe5c0a2360 2023-09-30T22:20:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.305+0000 7fbe5b7fe700 1 --2- 172.21.15.89:0/1299765114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c0a1140 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbe500098a0 tx=0x7fbe50009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3dd0bc3faeda9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.305+0000 7fbe5affd700 1 -- 172.21.15.89:0/1299765114 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50011830 con 0x7fbe5c0a2360 2023-09-30T22:20:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.305+0000 7fbe5affd700 1 -- 172.21.15.89:0/1299765114 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe50011990 con 0x7fbe5c0a2360 2023-09-30T22:20:24.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.305+0000 7fbe5affd700 1 -- 172.21.15.89:0/1299765114 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50011d20 con 0x7fbe5c0a2360 2023-09-30T22:20:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f33645a1700 1 Processor -- start 2023-09-30T22:20:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f33645a1700 1 -- start start 2023-09-30T22:20:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7fbe6240f700 1 -- 172.21.15.89:0/1299765114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 msgr2=0x7fbe5c0a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7fbe6240f700 1 --2- 172.21.15.89:0/1299765114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c0a1140 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbe500098a0 tx=0x7fbe50009bb0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7fbe6240f700 1 -- 172.21.15.89:0/1299765114 shutdown_connections 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7fbe6240f700 1 --2- 172.21.15.89:0/1299765114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c13d130 0x7fbe5c13f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7fbe6240f700 1 --2- 172.21.15.89:0/1299765114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c0a1140 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7fbe6240f700 1 -- 172.21.15.89:0/1299765114 >> 172.21.15.89:0/1299765114 conn(0x7fbe5c0a6550 msgr2=0x7fbe5c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f33645a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f335c133420 0x7f335c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f33645a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133e30 0x7f335c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f33645a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335c134260 0x7f335c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f33645a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c11fc20 con 0x7f335c133420 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f33645a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c11eb90 con 0x7f335c134260 2023-09-30T22:20:24.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f33645a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335c11f520 con 0x7f335c133e30 2023-09-30T22:20:24.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7fbe6240f700 1 -- 172.21.15.89:0/1299765114 shutdown_connections 2023-09-30T22:20:24.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f336233d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f335c133420 0x7f335c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f336233d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f335c133420 0x7f335c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58504/0 (socket says 172.21.15.89:58504) 2023-09-30T22:20:24.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f336233d700 1 -- 172.21.15.89:0/504281332 learned_addr learned my addr 172.21.15.89:0/504281332 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:24.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7f3362b3e700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335c134260 0x7f335c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.306+0000 7fbe6240f700 1 -- 172.21.15.89:0/1299765114 wait complete. 2023-09-30T22:20:24.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3361b3c700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133e30 0x7f335c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3362b3e700 1 -- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133e30 msgr2=0x7f335c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3362b3e700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133e30 0x7f335c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3362b3e700 1 -- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f335c133420 msgr2=0x7f335c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3362b3e700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f335c133420 0x7f335c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3362b3e700 1 -- 172.21.15.89:0/504281332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c0094a0 con 0x7f335c134260 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3362b3e700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335c134260 0x7f335c09cf00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f335c09f3e0 tx=0x7f335400ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3352ffd700 1 -- 172.21.15.89:0/504281332 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354003c00 con 0x7f335c134260 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3352ffd700 1 -- 172.21.15.89:0/504281332 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3354003d60 con 0x7f335c134260 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3358009550 con 0x7f335c134260 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c1208e0 con 0x7f335c134260 2023-09-30T22:20:24.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.307+0000 7f3352ffd700 1 -- 172.21.15.89:0/504281332 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335400aa80 con 0x7f335c134260 2023-09-30T22:20:24.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.308+0000 7fbe6240f700 1 Processor -- start 2023-09-30T22:20:24.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.308+0000 7fbe6240f700 1 -- start start 2023-09-30T22:20:24.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.308+0000 7f3352ffd700 1 -- 172.21.15.89:0/504281332 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f335400abe0 con 0x7f335c134260 2023-09-30T22:20:24.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.308+0000 7f3352ffd700 1 -- 172.21.15.89:0/504281332 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f335403f010 con 0x7f335c134260 2023-09-30T22:20:24.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7f4a25b12700 1 Processor -- start 2023-09-30T22:20:24.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe6240f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c06ccc0 0x7fbe5c124fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe6240f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c125510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe6240f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe5c13d130 0x7fbe5c125a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe6240f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c13c5f0 con 0x7fbe5c13d130 2023-09-30T22:20:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe6240f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c127850 con 0x7fbe5c06ccc0 2023-09-30T22:20:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe6240f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c09d9f0 con 0x7fbe5c0a2360 2023-09-30T22:20:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe609ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe5c13d130 0x7fbe5c125a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe609ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe5c13d130 0x7fbe5c125a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58516/0 (socket says 172.21.15.89:58516) 2023-09-30T22:20:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe609ac700 1 -- 172.21.15.89:0/873289279 learned_addr learned my addr 172.21.15.89:0/873289279 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7f4a25b12700 1 -- start start 2023-09-30T22:20:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe5b7fe700 1 --2- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c125510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe609ac700 1 -- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 msgr2=0x7fbe5c125510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe609ac700 1 --2- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c125510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe609ac700 1 -- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c06ccc0 msgr2=0x7fbe5c124fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe609ac700 1 --2- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c06ccc0 0x7fbe5c124fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.309+0000 7fbe609ac700 1 -- 172.21.15.89:0/873289279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500094a0 con 0x7fbe5c13d130 2023-09-30T22:20:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a25b12700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4a2009d6f0 0x7f4a2009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7fbe609ac700 1 --2- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe5c13d130 0x7fbe5c125a50 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009dd0 tx=0x7fbe5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a25b12700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e000 0x7f4a20130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a25b12700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e410 0x7f4a20130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a25b12700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2009f520 con 0x7f4a2009d6f0 2023-09-30T22:20:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a25b12700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2009f130 con 0x7f4a2009e410 2023-09-30T22:20:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a25b12700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2009e930 con 0x7f4a2009e000 2023-09-30T22:20:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e000 0x7f4a20130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e000 0x7f4a20130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36676/0 (socket says 172.21.15.89:36676) 2023-09-30T22:20:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 -- 172.21.15.89:0/973996683 learned_addr learned my addr 172.21.15.89:0/973996683 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33440052e0 con 0x7f335c134260 2023-09-30T22:20:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1ffff700 1 --2- 172.21.15.89:0/973996683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e410 0x7f4a20130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1f7fe700 1 --2- 172.21.15.89:0/973996683 >> v2:172.21.15.89:3300/0 conn(0x7f4a2009d6f0 0x7f4a2009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7fbe58ff9700 1 -- 172.21.15.89:0/873289279 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5400cce0 con 0x7fbe5c13d130 2023-09-30T22:20:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 -- 172.21.15.89:0/973996683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e410 msgr2=0x7f4a20130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 --2- 172.21.15.89:0/973996683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e410 0x7f4a20130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 -- 172.21.15.89:0/973996683 >> v2:172.21.15.89:3300/0 conn(0x7f4a2009d6f0 msgr2=0x7f4a2009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:24.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 --2- 172.21.15.89:0/973996683 >> v2:172.21.15.89:3300/0 conn(0x7f4a2009d6f0 0x7f4a2009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 -- 172.21.15.89:0/973996683 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2009b2d0 con 0x7f4a2009e000 2023-09-30T22:20:24.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7fbe58ff9700 1 -- 172.21.15.89:0/873289279 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5400f040 con 0x7fbe5c13d130 2023-09-30T22:20:24.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c128570 con 0x7fbe5c13d130 2023-09-30T22:20:24.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7fbe58ff9700 1 -- 172.21.15.89:0/873289279 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54014480 con 0x7fbe5c13d130 2023-09-30T22:20:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7f4a1effd700 1 --2- 172.21.15.89:0/973996683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e000 0x7f4a20130660 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009920 tx=0x7f4a10009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b63d49678a49f68a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.311+0000 7f4a1e7fc700 1 -- 172.21.15.89:0/973996683 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000ba20 con 0x7f4a2009e000 2023-09-30T22:20:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.311+0000 7f4a1e7fc700 1 -- 172.21.15.89:0/973996683 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1000bb80 con 0x7f4a2009e000 2023-09-30T22:20:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.311+0000 7f4a1e7fc700 1 -- 172.21.15.89:0/973996683 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10009e70 con 0x7f4a2009e000 2023-09-30T22:20:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.310+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c127130 con 0x7fbe5c13d130 2023-09-30T22:20:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.311+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c074ea0 con 0x7fbe5c13d130 2023-09-30T22:20:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.311+0000 7fbe58ff9700 1 -- 172.21.15.89:0/873289279 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe540145e0 con 0x7fbe5c13d130 2023-09-30T22:20:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.312+0000 7fbe58ff9700 1 -- 172.21.15.89:0/873289279 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbe54040130 con 0x7fbe5c13d130 2023-09-30T22:20:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.312+0000 7f4a25b12700 1 -- 172.21.15.89:0/973996683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e000 msgr2=0x7f4a20130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.312+0000 7f4a25b12700 1 --2- 172.21.15.89:0/973996683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e000 0x7f4a20130660 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009920 tx=0x7f4a10009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.312+0000 7f4a25b12700 1 -- 172.21.15.89:0/973996683 shutdown_connections 2023-09-30T22:20:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.312+0000 7f4a25b12700 1 --2- 172.21.15.89:0/973996683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e410 0x7f4a20130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.312+0000 7f4a25b12700 1 --2- 172.21.15.89:0/973996683 >> v2:172.21.15.89:3300/0 conn(0x7f4a2009d6f0 0x7f4a2009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.312+0000 7f4a25b12700 1 --2- 172.21.15.89:0/973996683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e000 0x7f4a20130660 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.312+0000 7f4a25b12700 1 -- 172.21.15.89:0/973996683 >> 172.21.15.89:0/973996683 conn(0x7f4a20066f50 msgr2=0x7f4a2009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.313+0000 7f4a25b12700 1 -- 172.21.15.89:0/973996683 shutdown_connections 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.313+0000 7f4a25b12700 1 -- 172.21.15.89:0/973996683 wait complete. 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.314+0000 7f4a25b12700 1 Processor -- start 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.314+0000 7f4a25b12700 1 -- start start 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a25b12700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a2009d6f0 0x7f4a20131b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a25b12700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e000 0x7f4a20132040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a25b12700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e410 0x7f4a20132580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a25b12700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a201346f0 con 0x7f4a2009d6f0 2023-09-30T22:20:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a25b12700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a20134b40 con 0x7f4a2009e000 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a25b12700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a20133c90 con 0x7f4a2009e410 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e410 0x7f4a20132580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e410 0x7f4a20132580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36688/0 (socket says 172.21.15.89:36688) 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1ffff700 1 -- 172.21.15.89:0/1552317043 learned_addr learned my addr 172.21.15.89:0/1552317043 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1effd700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e000 0x7f4a20132040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1effd700 1 -- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e410 msgr2=0x7f4a20132580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1effd700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e410 0x7f4a20132580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1effd700 1 -- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a2009d6f0 msgr2=0x7f4a20131b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1effd700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a2009d6f0 0x7f4a20131b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1effd700 1 -- 172.21.15.89:0/1552317043 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10009550 con 0x7f4a2009e000 2023-09-30T22:20:24.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f4a1ffff700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e410 0x7f4a20132580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:24.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.315+0000 7f3352ffd700 1 -- 172.21.15.89:0/504281332 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3354017020 con 0x7f335c134260 2023-09-30T22:20:24.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.316+0000 7f4a1effd700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e000 0x7f4a20132040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4a10016000 tx=0x7f4a10004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:24.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.316+0000 7f4a07fff700 1 -- 172.21.15.89:0/1552317043 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10004510 con 0x7f4a2009e000 2023-09-30T22:20:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.316+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a201339f0 con 0x7f4a2009e000 2023-09-30T22:20:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.316+0000 7f4a07fff700 1 -- 172.21.15.89:0/1552317043 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a100036a0 con 0x7f4a2009e000 2023-09-30T22:20:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.316+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a20132ca0 con 0x7f4a2009e000 2023-09-30T22:20:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.316+0000 7f4a07fff700 1 -- 172.21.15.89:0/1552317043 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10003a50 con 0x7f4a2009e000 2023-09-30T22:20:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.317+0000 7f4a07fff700 1 -- 172.21.15.89:0/1552317043 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a10003bb0 con 0x7f4a2009e000 2023-09-30T22:20:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.317+0000 7f4a07fff700 1 -- 172.21.15.89:0/1552317043 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4a10040b90 con 0x7f4a2009e000 2023-09-30T22:20:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.317+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a000052e0 con 0x7f4a2009e000 2023-09-30T22:20:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.317+0000 7fbe58ff9700 1 -- 172.21.15.89:0/873289279 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe5401a890 con 0x7fbe5c13d130 2023-09-30T22:20:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.323+0000 7f4a07fff700 1 -- 172.21.15.89:0/1552317043 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a10015020 con 0x7f4a2009e000 2023-09-30T22:20:24.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.476+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f89f406c360 con 0x7f89f409d6f0 2023-09-30T22:20:24.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.477+0000 7f89f17fa700 1 -- 172.21.15.89:0/29769437 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89e8021030 con 0x7f89f409d6f0 2023-09-30T22:20:24.478 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 msgr2=0x7f89f41a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f41a5120 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f89e800e810 tx=0x7f89e800eb20 comp rx=0 tx=0).stop 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 shutdown_connections 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f409e410 0x7f89f41a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89f409d6f0 0x7f89f41a5120 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 --2- 172.21.15.89:0/29769437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89f409e000 0x7f89f41a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 >> 172.21.15.89:0/29769437 conn(0x7f89f4066f50 msgr2=0x7f89f41bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 shutdown_connections 2023-09-30T22:20:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.479+0000 7f89faddb700 1 -- 172.21.15.89:0/29769437 wait complete. 2023-09-30T22:20:24.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:24.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.521+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4a00005150 con 0x7f4a2009e000 2023-09-30T22:20:24.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.522+0000 7f4a07fff700 1 -- 172.21.15.89:0/1552317043 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a10024440 con 0x7f4a2009e000 2023-09-30T22:20:24.523 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e000 msgr2=0x7f4a20132040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e000 0x7f4a20132040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4a10016000 tx=0x7f4a10004750 comp rx=0 tx=0).stop 2023-09-30T22:20:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 shutdown_connections 2023-09-30T22:20:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a2009e410 0x7f4a20132580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a2009d6f0 0x7f4a20131b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 --2- 172.21.15.89:0/1552317043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a2009e000 0x7f4a20132040 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 >> 172.21.15.89:0/1552317043 conn(0x7f4a20066f50 msgr2=0x7f4a2012fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 shutdown_connections 2023-09-30T22:20:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.524+0000 7f4a25b12700 1 -- 172.21.15.89:0/1552317043 wait complete. 2023-09-30T22:20:24.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.531+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3344005150 con 0x7f335c134260 2023-09-30T22:20:24.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.533+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbe5c072980 con 0x7fbe5c13d130 2023-09-30T22:20:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.534+0000 7fbe58ff9700 1 -- 172.21.15.89:0/873289279 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe54018030 con 0x7fbe5c13d130 2023-09-30T22:20:24.535 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.535+0000 7f3352ffd700 1 -- 172.21.15.89:0/504281332 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3354011380 con 0x7f335c134260 2023-09-30T22:20:24.536 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe5c13d130 msgr2=0x7fbe5c125a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7fbe6240f700 1 --2- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe5c13d130 0x7fbe5c125a50 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009dd0 tx=0x7fbe5400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 shutdown_connections 2023-09-30T22:20:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7fbe6240f700 1 --2- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe5c13d130 0x7fbe5c125a50 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7fbe6240f700 1 --2- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe5c06ccc0 0x7fbe5c124fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7fbe6240f700 1 --2- 172.21.15.89:0/873289279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe5c0a2360 0x7fbe5c125510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 >> 172.21.15.89:0/873289279 conn(0x7fbe5c0a6550 msgr2=0x7fbe5c0a53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335c134260 msgr2=0x7f335c09cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335c134260 0x7f335c09cf00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f335c09f3e0 tx=0x7f335400ce70 comp rx=0 tx=0).stop 2023-09-30T22:20:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 shutdown_connections 2023-09-30T22:20:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f335c134260 0x7f335c09cf00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f335c133420 0x7f335c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 --2- 172.21.15.89:0/504281332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f335c133e30 0x7f335c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 >> 172.21.15.89:0/504281332 conn(0x7f335c0a4210 msgr2=0x7f335c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 shutdown_connections 2023-09-30T22:20:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.537+0000 7f33645a1700 1 -- 172.21.15.89:0/504281332 wait complete. 2023-09-30T22:20:24.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:24.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.538+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 shutdown_connections 2023-09-30T22:20:24.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:24.538+0000 7fbe6240f700 1 -- 172.21.15.89:0/873289279 wait complete. 2023-09-30T22:20:24.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:24.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:25.492 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:25.542 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:25.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:25.553 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:25.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.639+0000 7f52230e5700 1 Processor -- start 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f52230e5700 1 -- start start 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f52230e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f521c09d6f0 0x7f521c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f52230e5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f52230e5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f52230e5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c09f520 con 0x7f521c09d6f0 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f52230e5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c09f130 con 0x7f521c09e000 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f52230e5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f521c09e930 con 0x7f521c09e410 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f521bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f521bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58780/0 (socket says 172.21.15.89:58780) 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f521bfff700 1 -- 172.21.15.89:0/3275032593 learned_addr learned my addr 172.21.15.89:0/3275032593 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:25.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f5221682700 1 --2- 172.21.15.89:0/3275032593 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f5220e81700 1 --2- 172.21.15.89:0/3275032593 >> v2:172.21.15.89:3300/0 conn(0x7f521c09d6f0 0x7f521c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.640+0000 7f521bfff700 1 -- 172.21.15.89:0/3275032593 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 msgr2=0x7f521c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.641+0000 7f521bfff700 1 --2- 172.21.15.89:0/3275032593 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.641+0000 7f521bfff700 1 -- 172.21.15.89:0/3275032593 >> v2:172.21.15.89:3300/0 conn(0x7f521c09d6f0 msgr2=0x7f521c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.641+0000 7f521bfff700 1 --2- 172.21.15.89:0/3275032593 >> v2:172.21.15.89:3300/0 conn(0x7f521c09d6f0 0x7f521c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.641+0000 7f521bfff700 1 -- 172.21.15.89:0/3275032593 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c09b2d0 con 0x7f521c09e000 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.641+0000 7f521bfff700 1 --2- 172.21.15.89:0/3275032593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c130660 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f52080098f0 tx=0x7f5208009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5725ee1a19cf8181 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.641+0000 7f521b7fe700 1 -- 172.21.15.89:0/3275032593 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800b8a0 con 0x7f521c09e000 2023-09-30T22:20:25.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.641+0000 7f521b7fe700 1 -- 172.21.15.89:0/3275032593 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520800ba00 con 0x7f521c09e000 2023-09-30T22:20:25.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.641+0000 7f521b7fe700 1 -- 172.21.15.89:0/3275032593 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800bd90 con 0x7f521c09e000 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 -- 172.21.15.89:0/3275032593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 msgr2=0x7f521c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 --2- 172.21.15.89:0/3275032593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c130660 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f52080098f0 tx=0x7f5208009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 -- 172.21.15.89:0/3275032593 shutdown_connections 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 --2- 172.21.15.89:0/3275032593 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 --2- 172.21.15.89:0/3275032593 >> v2:172.21.15.89:3300/0 conn(0x7f521c09d6f0 0x7f521c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 --2- 172.21.15.89:0/3275032593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c130660 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 -- 172.21.15.89:0/3275032593 >> 172.21.15.89:0/3275032593 conn(0x7f521c066f50 msgr2=0x7f521c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 -- 172.21.15.89:0/3275032593 shutdown_connections 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.642+0000 7f52230e5700 1 -- 172.21.15.89:0/3275032593 wait complete. 2023-09-30T22:20:25.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.643+0000 7f52230e5700 1 Processor -- start 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.643+0000 7f52230e5700 1 -- start start 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f52230e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f521c09d6f0 0x7f521c1c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f52230e5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c1c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f52230e5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c1c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f52230e5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c136690 con 0x7f521c09d6f0 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f52230e5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c136870 con 0x7f521c09e000 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f52230e5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f521c09c700 con 0x7f521c09e410 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f5220e81700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f521c09d6f0 0x7f521c1c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f5220e81700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f521c09d6f0 0x7f521c1c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58546/0 (socket says 172.21.15.89:58546) 2023-09-30T22:20:25.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f5220e81700 1 -- 172.21.15.89:0/1214909358 learned_addr learned my addr 172.21.15.89:0/1214909358 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f521bfff700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c1c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f5221682700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c1c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f5221682700 1 -- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 msgr2=0x7f521c1c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.644+0000 7f5221682700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c1c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f5221682700 1 -- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f521c09d6f0 msgr2=0x7f521c1c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f5221682700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f521c09d6f0 0x7f521c1c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f5221682700 1 -- 172.21.15.89:0/1214909358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208009550 con 0x7f521c09e410 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f521bfff700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c1c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f5220e81700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f521c09d6f0 0x7f521c1c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:25.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f5221682700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c1c6970 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5214007d90 tx=0x7f521400e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:25.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f52197fa700 1 -- 172.21.15.89:0/1214909358 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214018070 con 0x7f521c09e410 2023-09-30T22:20:25.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f52197fa700 1 -- 172.21.15.89:0/1214909358 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521400eb30 con 0x7f521c09e410 2023-09-30T22:20:25.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f52197fa700 1 -- 172.21.15.89:0/1214909358 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214003a20 con 0x7f521c09e410 2023-09-30T22:20:25.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c134a70 con 0x7f521c09e410 2023-09-30T22:20:25.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.645+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c137450 con 0x7f521c09e410 2023-09-30T22:20:25.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.646+0000 7f52197fa700 1 -- 172.21.15.89:0/1214909358 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5214014420 con 0x7f521c09e410 2023-09-30T22:20:25.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.646+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c136490 con 0x7f521c09e410 2023-09-30T22:20:25.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.646+0000 7f52197fa700 1 -- 172.21.15.89:0/1214909358 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f521403f630 con 0x7f521c09e410 2023-09-30T22:20:25.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.655+0000 7f52197fa700 1 -- 172.21.15.89:0/1214909358 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f52140146d0 con 0x7f521c09e410 2023-09-30T22:20:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.689+0000 7f85e4425700 1 Processor -- start 2023-09-30T22:20:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.689+0000 7f85e4425700 1 -- start start 2023-09-30T22:20:25.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.690+0000 7f85e4425700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc09e1d0 0x7f85dc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.690+0000 7f85e4425700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc139170 0x7f85dc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.690+0000 7f85e4425700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc09b120 con 0x7f85dc09d6f0 2023-09-30T22:20:25.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.690+0000 7f85e4425700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc06b460 con 0x7f85dc09e1d0 2023-09-30T22:20:25.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.690+0000 7f85e4425700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc09a7e0 con 0x7f85dc139170 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.690+0000 7f85e21c1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f85dc09d6f0 0x7f85dc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58612/0 (socket says 172.21.15.89:58612) 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e21c1700 1 -- 172.21.15.89:0/660558637 learned_addr learned my addr 172.21.15.89:0/660558637 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e29c2700 1 --2- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc139170 0x7f85dc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e19c0700 1 --2- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc09e1d0 0x7f85dc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e29c2700 1 -- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc09e1d0 msgr2=0x7f85dc138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e29c2700 1 --2- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc09e1d0 0x7f85dc138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e29c2700 1 -- 172.21.15.89:0/660558637 >> v1:172.21.15.89:6789/0 conn(0x7f85dc09d6f0 legacy=0x7f85dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e29c2700 1 -- 172.21.15.89:0/660558637 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc09a500 con 0x7f85dc139170 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e29c2700 1 --2- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc139170 0x7f85dc13b550 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009df0 tx=0x7f85d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f43f6ace8e0057dd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e11bf700 1 -- 172.21.15.89:0/660558637 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d400e070 con 0x7f85dc139170 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e11bf700 1 -- 172.21.15.89:0/660558637 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d4003a20 con 0x7f85dc139170 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.691+0000 7f85e11bf700 1 -- 172.21.15.89:0/660558637 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4003db0 con 0x7f85dc139170 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.692+0000 7f85e4425700 1 -- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc139170 msgr2=0x7f85dc13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.692+0000 7f85e4425700 1 --2- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc139170 0x7f85dc13b550 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009df0 tx=0x7f85d400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.692+0000 7f85e4425700 1 -- 172.21.15.89:0/660558637 shutdown_connections 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.692+0000 7f85e4425700 1 --2- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc139170 0x7f85dc13b550 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.692+0000 7f85e4425700 1 --2- 172.21.15.89:0/660558637 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc09e1d0 0x7f85dc138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.692+0000 7f85e4425700 1 -- 172.21.15.89:0/660558637 >> 172.21.15.89:0/660558637 conn(0x7f85dc066f50 msgr2=0x7f85dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.692+0000 7f85e4425700 1 -- 172.21.15.89:0/660558637 shutdown_connections 2023-09-30T22:20:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.692+0000 7f85e4425700 1 -- 172.21.15.89:0/660558637 wait complete. 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.693+0000 7f85e4425700 1 Processor -- start 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.693+0000 7f85e4425700 1 -- start start 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e4425700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc09d6f0 0x7f85dc11e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e4425700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc09e1d0 0x7f85dc11edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e4425700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc139170 0x7f85dc11f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e4425700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc1205a0 con 0x7f85dc09d6f0 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e4425700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc1220c0 con 0x7f85dc139170 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e4425700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0a1d00 con 0x7f85dc09e1d0 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e29c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc139170 0x7f85dc11f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e19c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc09e1d0 0x7f85dc11edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e19c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc09e1d0 0x7f85dc11edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36740/0 (socket says 172.21.15.89:36740) 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e19c0700 1 -- 172.21.15.89:0/4093542844 learned_addr learned my addr 172.21.15.89:0/4093542844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e21c1700 1 --2- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc09d6f0 0x7f85dc11e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.694+0000 7f85e29c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc139170 0x7f85dc11f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58794/0 (socket says 172.21.15.89:58794) 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85e19c0700 1 -- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc139170 msgr2=0x7f85dc11f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85e19c0700 1 --2- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc139170 0x7f85dc11f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85e19c0700 1 -- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc09d6f0 msgr2=0x7f85dc11e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85e19c0700 1 --2- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc09d6f0 0x7f85dc11e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85e19c0700 1 -- 172.21.15.89:0/4093542844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d4009a70 con 0x7f85dc09e1d0 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85e19c0700 1 --2- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc09e1d0 0x7f85dc11edc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00e9b0 tx=0x7f85cc00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85d2ffd700 1 -- 172.21.15.89:0/4093542844 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc00ee50 con 0x7f85dc09e1d0 2023-09-30T22:20:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85d2ffd700 1 -- 172.21.15.89:0/4093542844 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc00c850 con 0x7f85dc09e1d0 2023-09-30T22:20:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85d2ffd700 1 -- 172.21.15.89:0/4093542844 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc00cbc0 con 0x7f85dc09e1d0 2023-09-30T22:20:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc09c710 con 0x7f85dc09e1d0 2023-09-30T22:20:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.695+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc122b30 con 0x7f85dc09e1d0 2023-09-30T22:20:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.696+0000 7f85d2ffd700 1 -- 172.21.15.89:0/4093542844 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85cc00cd20 con 0x7f85dc09e1d0 2023-09-30T22:20:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.697+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc121d60 con 0x7f85dc09e1d0 2023-09-30T22:20:25.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.697+0000 7f85d2ffd700 1 -- 172.21.15.89:0/4093542844 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f85cc03fbb0 con 0x7f85dc09e1d0 2023-09-30T22:20:25.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.702+0000 7f85d2ffd700 1 -- 172.21.15.89:0/4093542844 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85dc121d60 con 0x7f85dc09e1d0 2023-09-30T22:20:25.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.724+0000 7ff8d4a93700 1 Processor -- start 2023-09-30T22:20:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.724+0000 7ff8d4a93700 1 -- start start 2023-09-30T22:20:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8d4a93700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8d4a93700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d630 0x7ff8d00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8d4a93700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d00a2a70 con 0x7ff8d0133930 2023-09-30T22:20:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8d4a93700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d00a01d0 con 0x7ff8d009d630 2023-09-30T22:20:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8d4a93700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d009e2b0 con 0x7ff8d00a0a30 2023-09-30T22:20:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8cdd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d630 0x7ff8d00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8cdd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d630 0x7ff8d00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58568/0 (socket says 172.21.15.89:58568) 2023-09-30T22:20:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8cdd9b700 1 -- 172.21.15.89:0/3302782667 learned_addr learned my addr 172.21.15.89:0/3302782667 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:25.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.725+0000 7ff8ce59c700 1 --2- 172.21.15.89:0/3302782667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8cd59a700 1 -- 172.21.15.89:0/3302782667 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132879830 0 0) 0x7ff8d00a2a70 con 0x7ff8d0133930 2023-09-30T22:20:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8cd59a700 1 -- 172.21.15.89:0/3302782667 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b80035b0 con 0x7ff8d0133930 2023-09-30T22:20:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8ce59c700 1 -- 172.21.15.89:0/3302782667 >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d630 msgr2=0x7ff8d00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8ce59c700 1 --2- 172.21.15.89:0/3302782667 >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d630 0x7ff8d00a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8ce59c700 1 -- 172.21.15.89:0/3302782667 >> v1:172.21.15.89:6789/0 conn(0x7ff8d0133930 legacy=0x7ff8d0133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8ce59c700 1 -- 172.21.15.89:0/3302782667 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d009dc50 con 0x7ff8d00a0a30 2023-09-30T22:20:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8ce59c700 1 --2- 172.21.15.89:0/3302782667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d00a0e00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009840 tx=0x7ff8c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f1979fd06d084f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:25.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8cd59a700 1 -- 172.21.15.89:0/3302782667 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0011830 con 0x7ff8d00a0a30 2023-09-30T22:20:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8cd59a700 1 -- 172.21.15.89:0/3302782667 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c0011990 con 0x7ff8d00a0a30 2023-09-30T22:20:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.726+0000 7ff8cd59a700 1 -- 172.21.15.89:0/3302782667 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0011d20 con 0x7ff8d00a0a30 2023-09-30T22:20:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.727+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3302782667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 msgr2=0x7ff8d00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.727+0000 7ff8d4a93700 1 --2- 172.21.15.89:0/3302782667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d00a0e00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009840 tx=0x7ff8c0009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.727+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3302782667 shutdown_connections 2023-09-30T22:20:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.727+0000 7ff8d4a93700 1 --2- 172.21.15.89:0/3302782667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d00a0e00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.727+0000 7ff8d4a93700 1 --2- 172.21.15.89:0/3302782667 >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d630 0x7ff8d00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.727+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3302782667 >> 172.21.15.89:0/3302782667 conn(0x7ff8d00a3050 msgr2=0x7ff8d00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:25.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.727+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3302782667 shutdown_connections 2023-09-30T22:20:25.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.727+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3302782667 wait complete. 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8d4a93700 1 Processor -- start 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8d4a93700 1 -- start start 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8d4a93700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009d630 0x7ff8d011f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8d4a93700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d011fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8d4a93700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d0133930 0x7ff8d01c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8d4a93700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0122350 con 0x7ff8d009d630 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8d4a93700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00a0410 con 0x7ff8d0133930 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8d4a93700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d011ef60 con 0x7ff8d00a0a30 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.729+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009d630 0x7ff8d011f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009d630 0x7ff8d011f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58582/0 (socket says 172.21.15.89:58582) 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8ce59c700 1 -- 172.21.15.89:0/3565320281 learned_addr learned my addr 172.21.15.89:0/3565320281 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8cdd9b700 1 --2- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d011fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8ce59c700 1 -- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 msgr2=0x7ff8d011fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8ce59c700 1 --2- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d011fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8ce59c700 1 -- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d0133930 msgr2=0x7ff8d01c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8ce59c700 1 --2- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d0133930 0x7ff8d01c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8ce59c700 1 -- 172.21.15.89:0/3565320281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c00094a0 con 0x7ff8d009d630 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8ce59c700 1 --2- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009d630 0x7ff8d011f500 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00090b0 tx=0x7ff8c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8beffd700 1 -- 172.21.15.89:0/3565320281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0004340 con 0x7ff8d009d630 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8beffd700 1 -- 172.21.15.89:0/3565320281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c001f070 con 0x7ff8d009d630 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8beffd700 1 -- 172.21.15.89:0/3565320281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00036a0 con 0x7ff8d009d630 2023-09-30T22:20:25.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d00a5410 con 0x7ff8d009d630 2023-09-30T22:20:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.730+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d0120620 con 0x7ff8d009d630 2023-09-30T22:20:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.731+0000 7ff8beffd700 1 -- 172.21.15.89:0/3565320281 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8c0021030 con 0x7ff8d009d630 2023-09-30T22:20:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.732+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d011ff80 con 0x7ff8d009d630 2023-09-30T22:20:25.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.732+0000 7ff8beffd700 1 -- 172.21.15.89:0/3565320281 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff8c00408d0 con 0x7ff8d009d630 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.733+0000 7fb299421700 1 Processor -- start 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.733+0000 7fb299421700 1 -- start start 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb299421700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb29406e570 0x7fb29406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb299421700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29406ee80 0x7fb294134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb299421700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb294070ee0 con 0x7fb29409eca0 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb299421700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb294071480 con 0x7fb29406e570 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb299421700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb294071b90 con 0x7fb29406ee80 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb2937fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29406ee80 0x7fb294134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb2937fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29406ee80 0x7fb294134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58806/0 (socket says 172.21.15.89:58806) 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb2937fe700 1 -- 172.21.15.89:0/3051259655 learned_addr learned my addr 172.21.15.89:0/3051259655 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb2937fe700 1 -- 172.21.15.89:0/3051259655 >> v2:172.21.15.89:3300/0 conn(0x7fb29406e570 msgr2=0x7fb29406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb2937fe700 1 --2- 172.21.15.89:0/3051259655 >> v2:172.21.15.89:3300/0 conn(0x7fb29406e570 0x7fb29406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb2937fe700 1 -- 172.21.15.89:0/3051259655 >> v1:172.21.15.89:6789/0 conn(0x7fb29409eca0 legacy=0x7fb294134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb2937fe700 1 -- 172.21.15.89:0/3051259655 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2940711a0 con 0x7fb29406ee80 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.734+0000 7fb2937fe700 1 --2- 172.21.15.89:0/3051259655 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29406ee80 0x7fb294134d20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009e50 tx=0x7fb28c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29305cc8a7761901 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb291ffb700 1 -- 172.21.15.89:0/3051259655 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00e070 con 0x7fb29406ee80 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb291ffb700 1 -- 172.21.15.89:0/3051259655 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c003a20 con 0x7fb29406ee80 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb291ffb700 1 -- 172.21.15.89:0/3051259655 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c003dd0 con 0x7fb29406ee80 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb299421700 1 -- 172.21.15.89:0/3051259655 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29406ee80 msgr2=0x7fb294134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb299421700 1 --2- 172.21.15.89:0/3051259655 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29406ee80 0x7fb294134d20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009e50 tx=0x7fb28c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:25.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb299421700 1 -- 172.21.15.89:0/3051259655 shutdown_connections 2023-09-30T22:20:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb299421700 1 --2- 172.21.15.89:0/3051259655 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29406ee80 0x7fb294134d20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb299421700 1 --2- 172.21.15.89:0/3051259655 >> v2:172.21.15.89:3300/0 conn(0x7fb29406e570 0x7fb29406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb299421700 1 -- 172.21.15.89:0/3051259655 >> 172.21.15.89:0/3051259655 conn(0x7fb2940a2aa0 msgr2=0x7fb2940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb299421700 1 -- 172.21.15.89:0/3051259655 shutdown_connections 2023-09-30T22:20:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.735+0000 7fb299421700 1 -- 172.21.15.89:0/3051259655 wait complete. 2023-09-30T22:20:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7fb299421700 1 Processor -- start 2023-09-30T22:20:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7fb299421700 1 -- start start 2023-09-30T22:20:25.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7ff8beffd700 1 -- 172.21.15.89:0/3565320281 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8c0019770 con 0x7ff8d009d630 2023-09-30T22:20:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7fb299421700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb29406e570 0x7fb294134a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7fb299421700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb29406ee80 0x7fb294139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7fb299421700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29413a3c0 0x7fb29413a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7fb299421700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294070610 con 0x7fb29406ee80 2023-09-30T22:20:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7fb299421700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb294135860 con 0x7fb29413a3c0 2023-09-30T22:20:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.737+0000 7fb299421700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb29413c260 con 0x7fb29406e570 2023-09-30T22:20:25.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb29406ee80 0x7fb294139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb29406ee80 0x7fb294139e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58606/0 (socket says 172.21.15.89:58606) 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 -- 172.21.15.89:0/3017906844 learned_addr learned my addr 172.21.15.89:0/3017906844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 -- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb29406e570 msgr2=0x7fb294134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 --2- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb29406e570 0x7fb294134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 -- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29413a3c0 msgr2=0x7fb29413a860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 --2- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29413a3c0 0x7fb29413a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 -- 172.21.15.89:0/3017906844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c009b30 con 0x7fb29406ee80 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb2927fc700 1 --2- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb29406ee80 0x7fb294139e80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb28800c4d0 tx=0x7fb28800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb27b7fe700 1 -- 172.21.15.89:0/3017906844 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880054e0 con 0x7fb29406ee80 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb27b7fe700 1 -- 172.21.15.89:0/3017906844 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb288003680 con 0x7fb29406ee80 2023-09-30T22:20:25.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29413bb80 con 0x7fb29406ee80 2023-09-30T22:20:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb27b7fe700 1 -- 172.21.15.89:0/3017906844 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288003a10 con 0x7fb29406ee80 2023-09-30T22:20:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.738+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29413d630 con 0x7fb29406ee80 2023-09-30T22:20:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.739+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29413c010 con 0x7fb29406ee80 2023-09-30T22:20:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.740+0000 7fb27b7fe700 1 -- 172.21.15.89:0/3017906844 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb288003b70 con 0x7fb29406ee80 2023-09-30T22:20:25.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.740+0000 7fb27b7fe700 1 -- 172.21.15.89:0/3017906844 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb288003e20 con 0x7fb29406ee80 2023-09-30T22:20:25.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.748+0000 7fb27b7fe700 1 -- 172.21.15.89:0/3017906844 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb29413c010 con 0x7fb29406ee80 2023-09-30T22:20:25.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.860+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f521c136230 con 0x7f521c09e410 2023-09-30T22:20:25.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.861+0000 7f52197fa700 1 -- 172.21.15.89:0/1214909358 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f52140122e0 con 0x7f521c09e410 2023-09-30T22:20:25.862 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:25.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 msgr2=0x7f521c1c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c1c6970 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5214007d90 tx=0x7f521400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 shutdown_connections 2023-09-30T22:20:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f521c09e410 0x7f521c1c6970 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f521c09d6f0 0x7f521c1c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 --2- 172.21.15.89:0/1214909358 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f521c09e000 0x7f521c1c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 >> 172.21.15.89:0/1214909358 conn(0x7f521c066f50 msgr2=0x7f521c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 shutdown_connections 2023-09-30T22:20:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.864+0000 7f52230e5700 1 -- 172.21.15.89:0/1214909358 wait complete. 2023-09-30T22:20:25.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:25.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.930+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f85dc121d60 con 0x7f85dc09e1d0 2023-09-30T22:20:25.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.931+0000 7f85d2ffd700 1 -- 172.21.15.89:0/4093542844 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85dc121d60 con 0x7f85dc09e1d0 2023-09-30T22:20:25.931 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:25.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc09e1d0 msgr2=0x7f85dc11edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 --2- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc09e1d0 0x7f85dc11edc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00e9b0 tx=0x7f85cc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 shutdown_connections 2023-09-30T22:20:25.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 --2- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f85dc139170 0x7f85dc11f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 --2- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc09d6f0 0x7f85dc11e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 --2- 172.21.15.89:0/4093542844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85dc09e1d0 0x7f85dc11edc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 >> 172.21.15.89:0/4093542844 conn(0x7f85dc066f50 msgr2=0x7f85dc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:25.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 shutdown_connections 2023-09-30T22:20:25.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.933+0000 7f85e4425700 1 -- 172.21.15.89:0/4093542844 wait complete. 2023-09-30T22:20:25.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.942+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb29413c010 con 0x7fb29406ee80 2023-09-30T22:20:25.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.942+0000 7fb27b7fe700 1 -- 172.21.15.89:0/3017906844 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb29413c010 con 0x7fb29406ee80 2023-09-30T22:20:25.943 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:25.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:25.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb29406ee80 msgr2=0x7fb294139e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 --2- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb29406ee80 0x7fb294139e80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb28800c4d0 tx=0x7fb28800c890 comp rx=0 tx=0).stop 2023-09-30T22:20:25.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 shutdown_connections 2023-09-30T22:20:25.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 --2- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb29413a3c0 0x7fb29413a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 --2- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb29406e570 0x7fb294134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 --2- 172.21.15.89:0/3017906844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb29406ee80 0x7fb294139e80 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 >> 172.21.15.89:0/3017906844 conn(0x7fb2940a2aa0 msgr2=0x7fb2940a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 shutdown_connections 2023-09-30T22:20:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.945+0000 7fb299421700 1 -- 172.21.15.89:0/3017906844 wait complete. 2023-09-30T22:20:25.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:25.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.963+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff8d00a1800 con 0x7ff8d009d630 2023-09-30T22:20:25.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.964+0000 7ff8beffd700 1 -- 172.21.15.89:0/3565320281 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8c00193b0 con 0x7ff8d009d630 2023-09-30T22:20:25.964 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:25.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009d630 msgr2=0x7ff8d011f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:25.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 --2- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009d630 0x7ff8d011f500 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00090b0 tx=0x7ff8c0004750 comp rx=0 tx=0).stop 2023-09-30T22:20:25.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 shutdown_connections 2023-09-30T22:20:25.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 --2- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d0133930 0x7ff8d01c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 --2- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009d630 0x7ff8d011f500 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 --2- 172.21.15.89:0/3565320281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d00a0a30 0x7ff8d011fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:25.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 >> 172.21.15.89:0/3565320281 conn(0x7ff8d00a3050 msgr2=0x7ff8d01bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:25.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 shutdown_connections 2023-09-30T22:20:25.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:25.966+0000 7ff8d4a93700 1 -- 172.21.15.89:0/3565320281 wait complete. 2023-09-30T22:20:25.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:26.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:26.947 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:27.003 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:27.003 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f346609b700 1 Processor -- start 2023-09-30T22:20:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f346609b700 1 -- start start 2023-09-30T22:20:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f346609b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009d6f0 0x7f346009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f346609b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e000 0x7f34601339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f346609b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f346009e410 0x7f3460133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f346609b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346006b460 con 0x7f346009e410 2023-09-30T22:20:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f346609b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346009b120 con 0x7f346009d6f0 2023-09-30T22:20:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f346609b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f346009a7e0 con 0x7f346009e000 2023-09-30T22:20:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f345ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f346009e410 0x7f3460133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f345effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e000 0x7f34601339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f345ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f346009e410 0x7f3460133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58610/0 (socket says 172.21.15.89:58610) 2023-09-30T22:20:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.012+0000 7f345ffff700 1 -- 172.21.15.89:0/1165647150 learned_addr learned my addr 172.21.15.89:0/1165647150 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345f7fe700 1 --2- 172.21.15.89:0/1165647150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009d6f0 0x7f346009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345effd700 1 -- 172.21.15.89:0/1165647150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009d6f0 msgr2=0x7f346009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345effd700 1 --2- 172.21.15.89:0/1165647150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009d6f0 0x7f346009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345effd700 1 -- 172.21.15.89:0/1165647150 >> v2:172.21.15.89:3300/0 conn(0x7f346009e410 msgr2=0x7f3460133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345effd700 1 --2- 172.21.15.89:0/1165647150 >> v2:172.21.15.89:3300/0 conn(0x7f346009e410 0x7f3460133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345effd700 1 -- 172.21.15.89:0/1165647150 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346009a500 con 0x7f346009e000 2023-09-30T22:20:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345ffff700 1 --2- 172.21.15.89:0/1165647150 >> v2:172.21.15.89:3300/0 conn(0x7f346009e410 0x7f3460133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345effd700 1 --2- 172.21.15.89:0/1165647150 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e000 0x7f34601339f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3454009870 tx=0x7f3454009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=874688abb2559a60 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345e7fc700 1 -- 172.21.15.89:0/1165647150 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454011830 con 0x7f346009e000 2023-09-30T22:20:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345e7fc700 1 -- 172.21.15.89:0/1165647150 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3454011990 con 0x7f346009e000 2023-09-30T22:20:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.013+0000 7f345e7fc700 1 -- 172.21.15.89:0/1165647150 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454011d20 con 0x7f346009e000 2023-09-30T22:20:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 -- 172.21.15.89:0/1165647150 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e000 msgr2=0x7f34601339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 --2- 172.21.15.89:0/1165647150 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e000 0x7f34601339f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3454009870 tx=0x7f3454009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 -- 172.21.15.89:0/1165647150 shutdown_connections 2023-09-30T22:20:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 --2- 172.21.15.89:0/1165647150 >> v2:172.21.15.89:3300/0 conn(0x7f346009e410 0x7f3460133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 --2- 172.21.15.89:0/1165647150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009d6f0 0x7f346009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 --2- 172.21.15.89:0/1165647150 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e000 0x7f34601339f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 -- 172.21.15.89:0/1165647150 >> 172.21.15.89:0/1165647150 conn(0x7f3460066f50 msgr2=0x7f346009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 -- 172.21.15.89:0/1165647150 shutdown_connections 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.014+0000 7f346609b700 1 -- 172.21.15.89:0/1165647150 wait complete. 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.015+0000 7f346609b700 1 Processor -- start 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f346609b700 1 -- start start 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f346609b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f346009d6f0 0x7f34601a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f346609b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009e000 0x7f34601a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f346609b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e410 0x7f34601a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f346609b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34601aa210 con 0x7f346009d6f0 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f346609b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34601a9850 con 0x7f346009e000 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f346609b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34601a8c20 con 0x7f346009e410 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f345ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e410 0x7f34601a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f345ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e410 0x7f34601a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36776/0 (socket says 172.21.15.89:36776) 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.016+0000 7f345ffff700 1 -- 172.21.15.89:0/3786008770 learned_addr learned my addr 172.21.15.89:0/3786008770 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345f7fe700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f346009d6f0 0x7f34601a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345effd700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009e000 0x7f34601a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345ffff700 1 -- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009e000 msgr2=0x7f34601a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345ffff700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009e000 0x7f34601a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345ffff700 1 -- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f346009d6f0 msgr2=0x7f34601a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345ffff700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f346009d6f0 0x7f34601a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345ffff700 1 -- 172.21.15.89:0/3786008770 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34540094a0 con 0x7f346009e410 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345effd700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009e000 0x7f34601a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f345ffff700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e410 0x7f34601a6840 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3458007c00 tx=0x7f3458007f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f3447fff700 1 -- 172.21.15.89:0/3786008770 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800db20 con 0x7f346009e410 2023-09-30T22:20:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f3447fff700 1 -- 172.21.15.89:0/3786008770 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345800f040 con 0x7f346009e410 2023-09-30T22:20:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f3447fff700 1 -- 172.21.15.89:0/3786008770 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800dec0 con 0x7f346009e410 2023-09-30T22:20:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346009c710 con 0x7f346009e410 2023-09-30T22:20:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.017+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34600a30e0 con 0x7f346009e410 2023-09-30T22:20:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.018+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34601a9210 con 0x7f346009e410 2023-09-30T22:20:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.019+0000 7f3447fff700 1 -- 172.21.15.89:0/3786008770 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f345801e3b0 con 0x7f346009e410 2023-09-30T22:20:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.019+0000 7f3447fff700 1 -- 172.21.15.89:0/3786008770 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f345803f670 con 0x7f346009e410 2023-09-30T22:20:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.025+0000 7f3447fff700 1 -- 172.21.15.89:0/3786008770 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f345801c2c0 con 0x7f346009e410 2023-09-30T22:20:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.137+0000 7f5c4e8d8700 1 Processor -- start 2023-09-30T22:20:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.137+0000 7f5c4e8d8700 1 -- start start 2023-09-30T22:20:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.137+0000 7f5c4e8d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5c4812cf60 0x7f5c4806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.137+0000 7f5c4e8d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812f8b0 0x7f5c4812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.137+0000 7f5c4e8d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c48130260 0x7f5c48136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.137+0000 7f5c4e8d8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4812d330 con 0x7f5c4812cf60 2023-09-30T22:20:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.137+0000 7f5c4e8d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4812cd90 con 0x7f5c4812f8b0 2023-09-30T22:20:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.137+0000 7f5c4e8d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c481306d0 con 0x7f5c48130260 2023-09-30T22:20:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812f8b0 0x7f5c4812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812f8b0 0x7f5c4812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58822/0 (socket says 172.21.15.89:58822) 2023-09-30T22:20:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 -- 172.21.15.89:0/1411038682 learned_addr learned my addr 172.21.15.89:0/1411038682 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c47fff700 1 --2- 172.21.15.89:0/1411038682 >> v2:172.21.15.89:3300/0 conn(0x7f5c4812cf60 0x7f5c4806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c4ce75700 1 --2- 172.21.15.89:0/1411038682 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c48130260 0x7f5c48136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 -- 172.21.15.89:0/1411038682 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c48130260 msgr2=0x7f5c48136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:27.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 --2- 172.21.15.89:0/1411038682 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c48130260 0x7f5c48136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 -- 172.21.15.89:0/1411038682 >> v2:172.21.15.89:3300/0 conn(0x7f5c4812cf60 msgr2=0x7f5c4806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 --2- 172.21.15.89:0/1411038682 >> v2:172.21.15.89:3300/0 conn(0x7f5c4812cf60 0x7f5c4806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 -- 172.21.15.89:0/1411038682 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48071780 con 0x7f5c4812f8b0 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.138+0000 7f5c477fe700 1 --2- 172.21.15.89:0/1411038682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812f8b0 0x7f5c4812fd20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00b9a0 tx=0x7f5c3c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6023f530b0de5c61 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c46ffd700 1 -- 172.21.15.89:0/1411038682 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c009030 con 0x7f5c4812f8b0 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c46ffd700 1 -- 172.21.15.89:0/1411038682 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00cba0 con 0x7f5c4812f8b0 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c46ffd700 1 -- 172.21.15.89:0/1411038682 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00d450 con 0x7f5c4812f8b0 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/1411038682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812f8b0 msgr2=0x7f5c4812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 --2- 172.21.15.89:0/1411038682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812f8b0 0x7f5c4812fd20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00b9a0 tx=0x7f5c3c00bcb0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/1411038682 shutdown_connections 2023-09-30T22:20:27.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 --2- 172.21.15.89:0/1411038682 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c48130260 0x7f5c48136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 --2- 172.21.15.89:0/1411038682 >> v2:172.21.15.89:3300/0 conn(0x7f5c4812cf60 0x7f5c4806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 --2- 172.21.15.89:0/1411038682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812f8b0 0x7f5c4812fd20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/1411038682 >> 172.21.15.89:0/1411038682 conn(0x7f5c480a2aa0 msgr2=0x7f5c480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/1411038682 shutdown_connections 2023-09-30T22:20:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.139+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/1411038682 wait complete. 2023-09-30T22:20:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.141+0000 7f5c4e8d8700 1 Processor -- start 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.141+0000 7f5c4e8d8700 1 -- start start 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.141+0000 7f5c4e8d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812cf60 0x7f5c481c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.141+0000 7f5c4e8d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c4812f8b0 0x7f5c481c1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.141+0000 7f5c4e8d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c48130260 0x7f5c481c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.141+0000 7f5c4e8d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48129ce0 con 0x7f5c48130260 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.141+0000 7f5c4e8d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48118f50 con 0x7f5c4812cf60 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.141+0000 7f5c4e8d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4812da60 con 0x7f5c4812f8b0 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812cf60 0x7f5c481c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812cf60 0x7f5c481c1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58830/0 (socket says 172.21.15.89:58830) 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 -- 172.21.15.89:0/3073165902 learned_addr learned my addr 172.21.15.89:0/3073165902 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c477fe700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c4812f8b0 0x7f5c481c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c4ce75700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c48130260 0x7f5c481c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 -- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c4812f8b0 msgr2=0x7f5c481c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c4812f8b0 0x7f5c481c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 -- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c48130260 msgr2=0x7f5c481c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c48130260 0x7f5c481c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 -- 172.21.15.89:0/3073165902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c00b600 con 0x7f5c4812cf60 2023-09-30T22:20:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c47fff700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812cf60 0x7f5c481c1330 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5c38012fc0 tx=0x7f5c380103a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c44ff9700 1 -- 172.21.15.89:0/3073165902 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800dbb0 con 0x7f5c4812cf60 2023-09-30T22:20:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c44ff9700 1 -- 172.21.15.89:0/3073165902 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c38013040 con 0x7f5c4812cf60 2023-09-30T22:20:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c44ff9700 1 -- 172.21.15.89:0/3073165902 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38010db0 con 0x7f5c4812cf60 2023-09-30T22:20:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c48129fa0 con 0x7f5c4812cf60 2023-09-30T22:20:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.142+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4809d250 con 0x7f5c4812cf60 2023-09-30T22:20:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.143+0000 7f5c44ff9700 1 -- 172.21.15.89:0/3073165902 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c38018480 con 0x7f5c4812cf60 2023-09-30T22:20:27.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.144+0000 7f5c44ff9700 1 -- 172.21.15.89:0/3073165902 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5c38047200 con 0x7f5c4812cf60 2023-09-30T22:20:27.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.145+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4806d5a0 con 0x7f5c4812cf60 2023-09-30T22:20:27.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.147+0000 7fe9e1aa9700 1 Processor -- start 2023-09-30T22:20:27.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.147+0000 7fe9e1aa9700 1 -- start start 2023-09-30T22:20:27.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7f46d13f3700 1 Processor -- start 2023-09-30T22:20:27.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7f46d13f3700 1 -- start start 2023-09-30T22:20:27.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7fe9e1aa9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09d6f0 0x7fe9dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7fe9e1aa9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe9dc09e000 0x7fe9dc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7fe9e1aa9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9dc09e410 0x7fe9dc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7fe9e1aa9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc09f520 con 0x7fe9dc09e000 2023-09-30T22:20:27.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7fe9e1aa9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc09f130 con 0x7fe9dc09e410 2023-09-30T22:20:27.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7fe9e1aa9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc09e930 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7f46d13f3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46cc130790 0x7f46cc130b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7f46d13f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc06f640 0x7f46cc06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7f46d13f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46cc06ffb0 0x7f46cc1374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7f46d13f3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46cc09a370 con 0x7f46cc130790 2023-09-30T22:20:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7f46d13f3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc09ef50 con 0x7f46cc06ffb0 2023-09-30T22:20:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.148+0000 7f46d13f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc070860 con 0x7f46cc06f640 2023-09-30T22:20:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09d6f0 0x7fe9dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc06f640 0x7f46cc06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09d6f0 0x7fe9dc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36804/0 (socket says 172.21.15.89:36804) 2023-09-30T22:20:27.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 -- 172.21.15.89:0/3414264704 learned_addr learned my addr 172.21.15.89:0/3414264704 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:27.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc06f640 0x7f46cc06fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36812/0 (socket says 172.21.15.89:36812) 2023-09-30T22:20:27.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 -- 172.21.15.89:0/3467240261 learned_addr learned my addr 172.21.15.89:0/3467240261 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:27.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46caffd700 1 --2- 172.21.15.89:0/3467240261 >> v2:172.21.15.89:3300/0 conn(0x7f46cc130790 0x7f46cc130b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9daffd700 1 --2- 172.21.15.89:0/3414264704 >> v2:172.21.15.89:3300/0 conn(0x7fe9dc09e000 0x7fe9dc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 -- 172.21.15.89:0/3467240261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46cc06ffb0 msgr2=0x7f46cc1374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 --2- 172.21.15.89:0/3467240261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46cc06ffb0 0x7f46cc1374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 -- 172.21.15.89:0/3467240261 >> v2:172.21.15.89:3300/0 conn(0x7f46cc130790 msgr2=0x7f46cc130b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 --2- 172.21.15.89:0/3467240261 >> v2:172.21.15.89:3300/0 conn(0x7f46cc130790 0x7f46cc130b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 -- 172.21.15.89:0/3414264704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9dc09e410 msgr2=0x7fe9dc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 --2- 172.21.15.89:0/3414264704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9dc09e410 0x7fe9dc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 -- 172.21.15.89:0/3414264704 >> v2:172.21.15.89:3300/0 conn(0x7fe9dc09e000 msgr2=0x7fe9dc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 --2- 172.21.15.89:0/3414264704 >> v2:172.21.15.89:3300/0 conn(0x7fe9dc09e000 0x7fe9dc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 -- 172.21.15.89:0/3467240261 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc1310a0 con 0x7f46cc06f640 2023-09-30T22:20:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 -- 172.21.15.89:0/3414264704 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc09b2d0 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46ca7fc700 1 --2- 172.21.15.89:0/3467240261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc06f640 0x7f46cc06fa70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f46c00098f0 tx=0x7f46c0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b86fcfca87c0d07 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9db7fe700 1 --2- 172.21.15.89:0/3414264704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09d6f0 0x7fe9dc09dac0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009920 tx=0x7fe9cc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf55556b996418bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9da7fc700 1 -- 172.21.15.89:0/3414264704 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc011830 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9da7fc700 1 -- 172.21.15.89:0/3414264704 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc011990 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7fe9da7fc700 1 -- 172.21.15.89:0/3414264704 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc011d20 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.150+0000 7f46c9ffb700 1 -- 172.21.15.89:0/3467240261 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0011830 con 0x7f46cc06f640 2023-09-30T22:20:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46c9ffb700 1 -- 172.21.15.89:0/3467240261 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0011990 con 0x7f46cc06f640 2023-09-30T22:20:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46c9ffb700 1 -- 172.21.15.89:0/3467240261 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0011d20 con 0x7f46cc06f640 2023-09-30T22:20:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3414264704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09d6f0 msgr2=0x7fe9dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 --2- 172.21.15.89:0/3414264704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09d6f0 0x7fe9dc09dac0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009920 tx=0x7fe9cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3414264704 shutdown_connections 2023-09-30T22:20:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 --2- 172.21.15.89:0/3414264704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9dc09e410 0x7fe9dc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 --2- 172.21.15.89:0/3414264704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09d6f0 0x7fe9dc09dac0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 --2- 172.21.15.89:0/3414264704 >> v2:172.21.15.89:3300/0 conn(0x7fe9dc09e000 0x7fe9dc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3414264704 >> 172.21.15.89:0/3414264704 conn(0x7fe9dc066f50 msgr2=0x7fe9dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:27.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3414264704 shutdown_connections 2023-09-30T22:20:27.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46d13f3700 1 -- 172.21.15.89:0/3467240261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc06f640 msgr2=0x7f46cc06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46d13f3700 1 --2- 172.21.15.89:0/3467240261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc06f640 0x7f46cc06fa70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f46c00098f0 tx=0x7f46c0009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:27.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3414264704 wait complete. 2023-09-30T22:20:27.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46d13f3700 1 -- 172.21.15.89:0/3467240261 shutdown_connections 2023-09-30T22:20:27.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46d13f3700 1 --2- 172.21.15.89:0/3467240261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46cc06ffb0 0x7f46cc1374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46d13f3700 1 --2- 172.21.15.89:0/3467240261 >> v2:172.21.15.89:3300/0 conn(0x7f46cc130790 0x7f46cc130b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46d13f3700 1 --2- 172.21.15.89:0/3467240261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc06f640 0x7f46cc06fa70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46d13f3700 1 -- 172.21.15.89:0/3467240261 >> 172.21.15.89:0/3467240261 conn(0x7f46cc0a2aa0 msgr2=0x7f46cc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:27.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.151+0000 7f46d13f3700 1 -- 172.21.15.89:0/3467240261 shutdown_connections 2023-09-30T22:20:27.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.152+0000 7f46d13f3700 1 -- 172.21.15.89:0/3467240261 wait complete. 2023-09-30T22:20:27.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.152+0000 7fe9e1aa9700 1 Processor -- start 2023-09-30T22:20:27.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.152+0000 7fe9e1aa9700 1 -- start start 2023-09-30T22:20:27.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46d13f3700 1 Processor -- start 2023-09-30T22:20:27.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46d13f3700 1 -- start start 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9e1aa9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc09d6f0 0x7fe9dc11df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9e1aa9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09e000 0x7fe9dc11e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9e1aa9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9dc09e410 0x7fe9dc11ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9e1aa9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc121820 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9e1aa9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc120180 con 0x7fe9dc09e410 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9e1aa9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc130320 con 0x7fe9dc09e000 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc09d6f0 0x7fe9dc11df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc09d6f0 0x7fe9dc11df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58654/0 (socket says 172.21.15.89:58654) 2023-09-30T22:20:27.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9db7fe700 1 -- 172.21.15.89:0/3366466874 learned_addr learned my addr 172.21.15.89:0/3366466874 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:27.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46d13f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46cc06f640 0x7f46cc11d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46d13f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46cc06ffb0 0x7f46cc11d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9db7fe700 1 -- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09e000 msgr2=0x7fe9dc11e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9db7fe700 1 --2- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09e000 0x7fe9dc11e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9db7fe700 1 -- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9dc09e410 msgr2=0x7fe9dc11ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9db7fe700 1 --2- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9dc09e410 0x7fe9dc11ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7fe9db7fe700 1 -- 172.21.15.89:0/3366466874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc009550 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46d13f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc130790 0x7f46cc11db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:27.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46d13f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc120f00 con 0x7f46cc06f640 2023-09-30T22:20:27.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46d13f3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc071800 con 0x7f46cc06ffb0 2023-09-30T22:20:27.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46d13f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc11ea50 con 0x7f46cc130790 2023-09-30T22:20:27.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46caffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46cc06f640 0x7f46cc11d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46caffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46cc06f640 0x7f46cc11d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58664/0 (socket says 172.21.15.89:58664) 2023-09-30T22:20:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.153+0000 7f46caffd700 1 -- 172.21.15.89:0/443275558 learned_addr learned my addr 172.21.15.89:0/443275558 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7fe9db7fe700 1 --2- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc09d6f0 0x7fe9dc11df80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009c70 tx=0x7fe9cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7fe9c3fff700 1 -- 172.21.15.89:0/3366466874 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc004400 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7f46caffd700 1 -- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc130790 msgr2=0x7f46cc11db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7f46caffd700 1 --2- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc130790 0x7f46cc11db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7f46caffd700 1 -- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46cc06ffb0 msgr2=0x7f46cc11d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7f46caffd700 1 --2- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46cc06ffb0 0x7f46cc11d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7f46caffd700 1 -- 172.21.15.89:0/443275558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009550 con 0x7f46cc06f640 2023-09-30T22:20:27.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7f5c44ff9700 1 -- 172.21.15.89:0/3073165902 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c4809d250 con 0x7f5c4812cf60 2023-09-30T22:20:27.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc09c700 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.154+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc1211f0 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.155+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc11ac20 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.155+0000 7f46caffd700 1 --2- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46cc06f640 0x7f46cc11d0e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00ed40 tx=0x7f46bc00d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.155+0000 7fe9c3fff700 1 -- 172.21.15.89:0/3366466874 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc01f070 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.155+0000 7fe9c3fff700 1 -- 172.21.15.89:0/3366466874 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc0038a0 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.155+0000 7f46b37fe700 1 -- 172.21.15.89:0/443275558 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00de40 con 0x7f46cc06f640 2023-09-30T22:20:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.155+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc11ffc0 con 0x7f46cc06f640 2023-09-30T22:20:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.155+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc120500 con 0x7f46cc06f640 2023-09-30T22:20:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.155+0000 7fe9c3fff700 1 -- 172.21.15.89:0/3366466874 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe9cc021030 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.156+0000 7fe9c3fff700 1 -- 172.21.15.89:0/3366466874 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe9cc040f50 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.156+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc12b970 con 0x7f46cc06f640 2023-09-30T22:20:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.160+0000 7f46b37fe700 1 -- 172.21.15.89:0/443275558 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc005a20 con 0x7f46cc06f640 2023-09-30T22:20:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.160+0000 7f46b37fe700 1 -- 172.21.15.89:0/443275558 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc005d90 con 0x7f46cc06f640 2023-09-30T22:20:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.160+0000 7fe9c3fff700 1 -- 172.21.15.89:0/3366466874 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe9cc0239a0 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.161+0000 7f46b37fe700 1 -- 172.21.15.89:0/443275558 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46bc010650 con 0x7f46cc06f640 2023-09-30T22:20:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.161+0000 7f46b37fe700 1 -- 172.21.15.89:0/443275558 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f46bc040540 con 0x7f46cc06f640 2023-09-30T22:20:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.165+0000 7f46b37fe700 1 -- 172.21.15.89:0/443275558 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46bc016300 con 0x7f46cc06f640 2023-09-30T22:20:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.268+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f34601a9210 con 0x7f346009e410 2023-09-30T22:20:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.268+0000 7f3447fff700 1 -- 172.21.15.89:0/3786008770 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34601a9210 con 0x7f346009e410 2023-09-30T22:20:27.269 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e410 msgr2=0x7f34601a6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e410 0x7f34601a6840 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3458007c00 tx=0x7f3458007f10 comp rx=0 tx=0).stop 2023-09-30T22:20:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 shutdown_connections 2023-09-30T22:20:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f346009e410 0x7f34601a6840 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f346009d6f0 0x7f34601a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 --2- 172.21.15.89:0/3786008770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f346009e000 0x7f34601a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 >> 172.21.15.89:0/3786008770 conn(0x7f3460066f50 msgr2=0x7f34601bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 shutdown_connections 2023-09-30T22:20:27.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.270+0000 7f346609b700 1 -- 172.21.15.89:0/3786008770 wait complete. 2023-09-30T22:20:27.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.351+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe9dc074ea0 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.352+0000 7fe9c3fff700 1 -- 172.21.15.89:0/3366466874 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe9cc015020 con 0x7fe9dc09d6f0 2023-09-30T22:20:27.352 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc09d6f0 msgr2=0x7fe9dc11df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 --2- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc09d6f0 0x7fe9dc11df80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009c70 tx=0x7fe9cc004750 comp rx=0 tx=0).stop 2023-09-30T22:20:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 shutdown_connections 2023-09-30T22:20:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 --2- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9dc09e410 0x7fe9dc11ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 --2- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc09d6f0 0x7fe9dc11df80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 --2- 172.21.15.89:0/3366466874 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe9dc09e000 0x7fe9dc11e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 >> 172.21.15.89:0/3366466874 conn(0x7fe9dc066f50 msgr2=0x7fe9dc130870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 shutdown_connections 2023-09-30T22:20:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.355+0000 7fe9e1aa9700 1 -- 172.21.15.89:0/3366466874 wait complete. 2023-09-30T22:20:27.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.357+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f46cc074ea0 con 0x7f46cc06f640 2023-09-30T22:20:27.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.358+0000 7f46b37fe700 1 -- 172.21.15.89:0/443275558 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46bc020b90 con 0x7f46cc06f640 2023-09-30T22:20:27.358 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46cc06f640 msgr2=0x7f46cc11d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 --2- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46cc06f640 0x7f46cc11d0e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00ed40 tx=0x7f46bc00d630 comp rx=0 tx=0).stop 2023-09-30T22:20:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 shutdown_connections 2023-09-30T22:20:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 --2- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46cc130790 0x7f46cc11db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 --2- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46cc06f640 0x7f46cc11d0e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 --2- 172.21.15.89:0/443275558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46cc06ffb0 0x7f46cc11d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 >> 172.21.15.89:0/443275558 conn(0x7f46cc0a2aa0 msgr2=0x7f46cc12d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 shutdown_connections 2023-09-30T22:20:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.360+0000 7f46d13f3700 1 -- 172.21.15.89:0/443275558 wait complete. 2023-09-30T22:20:27.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:27.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:27.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.388+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5c4809d250 con 0x7f5c4812cf60 2023-09-30T22:20:27.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.389+0000 7f5c44ff9700 1 -- 172.21.15.89:0/3073165902 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c4809d250 con 0x7f5c4812cf60 2023-09-30T22:20:27.389 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:27.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812cf60 msgr2=0x7f5c481c1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:27.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812cf60 0x7f5c481c1330 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5c38012fc0 tx=0x7f5c380103a0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 shutdown_connections 2023-09-30T22:20:27.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c48130260 0x7f5c481c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c4812cf60 0x7f5c481c1330 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 --2- 172.21.15.89:0/3073165902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c4812f8b0 0x7f5c481c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:27.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 >> 172.21.15.89:0/3073165902 conn(0x7f5c480a2aa0 msgr2=0x7f5c4812a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:27.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 shutdown_connections 2023-09-30T22:20:27.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:27.391+0000 7f5c4e8d8700 1 -- 172.21.15.89:0/3073165902 wait complete. 2023-09-30T22:20:27.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:28.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:28.370 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:28.375 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:28.404 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:28.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.417+0000 7f5d7d7fc700 1 Processor -- start 2023-09-30T22:20:28.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.417+0000 7f5d7d7fc700 1 -- start start 2023-09-30T22:20:28.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d7d7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809d6f0 0x7f5d7809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d7d7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d7809e410 0x7f5d78130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d7d7fc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7809f130 con 0x7f5d7809e000 2023-09-30T22:20:28.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d7d7fc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7809f520 con 0x7f5d7809e410 2023-09-30T22:20:28.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d7d7fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7809e930 con 0x7f5d7809d6f0 2023-09-30T22:20:28.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d777fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d7809e410 0x7f5d78130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d777fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d7809e410 0x7f5d78130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58678/0 (socket says 172.21.15.89:58678) 2023-09-30T22:20:28.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d777fe700 1 -- 172.21.15.89:0/1951841544 learned_addr learned my addr 172.21.15.89:0/1951841544 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:28.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d767fc700 1 --1- 172.21.15.89:0/1951841544 >> v1:172.21.15.89:6789/0 conn(0x7f5d7809e000 0x7f5d78130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58640/0 (socket says 172.21.15.89:58640) 2023-09-30T22:20:28.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d76ffd700 1 --2- 172.21.15.89:0/1951841544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809d6f0 0x7f5d7809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d75ffb700 1 -- 172.21.15.89:0/1951841544 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736879585 0 0) 0x7f5d7809f130 con 0x7f5d7809e000 2023-09-30T22:20:28.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.418+0000 7f5d75ffb700 1 -- 172.21.15.89:0/1951841544 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d600035b0 con 0x7f5d7809e000 2023-09-30T22:20:28.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d76ffd700 1 -- 172.21.15.89:0/1951841544 >> v2:172.21.15.89:3300/0 conn(0x7f5d7809e410 msgr2=0x7f5d78130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d76ffd700 1 --2- 172.21.15.89:0/1951841544 >> v2:172.21.15.89:3300/0 conn(0x7f5d7809e410 0x7f5d78130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d76ffd700 1 -- 172.21.15.89:0/1951841544 >> v1:172.21.15.89:6789/0 conn(0x7f5d7809e000 legacy=0x7f5d78130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d76ffd700 1 -- 172.21.15.89:0/1951841544 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7809b2d0 con 0x7f5d7809d6f0 2023-09-30T22:20:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d777fe700 1 --2- 172.21.15.89:0/1951841544 >> v2:172.21.15.89:3300/0 conn(0x7f5d7809e410 0x7f5d78130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d76ffd700 1 --2- 172.21.15.89:0/1951841544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809d6f0 0x7f5d7809dac0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009840 tx=0x7f5d68009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76957288c10483e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d75ffb700 1 -- 172.21.15.89:0/1951841544 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68011830 con 0x7f5d7809d6f0 2023-09-30T22:20:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d75ffb700 1 -- 172.21.15.89:0/1951841544 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d68011990 con 0x7f5d7809d6f0 2023-09-30T22:20:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.419+0000 7f5d75ffb700 1 -- 172.21.15.89:0/1951841544 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68011d20 con 0x7f5d7809d6f0 2023-09-30T22:20:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.420+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/1951841544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809d6f0 msgr2=0x7f5d7809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.420+0000 7f5d7d7fc700 1 --2- 172.21.15.89:0/1951841544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809d6f0 0x7f5d7809dac0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009840 tx=0x7f5d68009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:28.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.420+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/1951841544 shutdown_connections 2023-09-30T22:20:28.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.420+0000 7f5d7d7fc700 1 --2- 172.21.15.89:0/1951841544 >> v2:172.21.15.89:3300/0 conn(0x7f5d7809e410 0x7f5d78130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.420+0000 7f5d7d7fc700 1 --2- 172.21.15.89:0/1951841544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809d6f0 0x7f5d7809dac0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.420+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/1951841544 >> 172.21.15.89:0/1951841544 conn(0x7f5d78066f50 msgr2=0x7f5d7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:28.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.420+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/1951841544 shutdown_connections 2023-09-30T22:20:28.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.420+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/1951841544 wait complete. 2023-09-30T22:20:28.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.422+0000 7f5d7d7fc700 1 Processor -- start 2023-09-30T22:20:28.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.422+0000 7f5d7d7fc700 1 -- start start 2023-09-30T22:20:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.422+0000 7f5d7d7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d7809d6f0 0x7f5d78139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.422+0000 7f5d7d7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809e000 0x7f5d78139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.422+0000 7f5d7d7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7809e410 0x7f5d7813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.422+0000 7f5d7d7fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7813b1e0 con 0x7f5d7809e410 2023-09-30T22:20:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.422+0000 7f5d7d7fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7809c770 con 0x7f5d7809e000 2023-09-30T22:20:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.422+0000 7f5d7d7fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7813ad60 con 0x7f5d7809d6f0 2023-09-30T22:20:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809e000 0x7f5d78139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809e000 0x7f5d78139e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58892/0 (socket says 172.21.15.89:58892) 2023-09-30T22:20:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 -- 172.21.15.89:0/941744317 learned_addr learned my addr 172.21.15.89:0/941744317 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d777fe700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7809e410 0x7f5d7813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d76ffd700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d7809d6f0 0x7f5d78139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 -- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d7809d6f0 msgr2=0x7f5d78139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d7809d6f0 0x7f5d78139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 -- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7809e410 msgr2=0x7f5d7813a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7809e410 0x7f5d7813a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 -- 172.21.15.89:0/941744317 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70009b30 con 0x7f5d7809e000 2023-09-30T22:20:28.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.423+0000 7f5d767fc700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809e000 0x7f5d78139e80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c550 tx=0x7f5d6c00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:28.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.424+0000 7f5d5f7fe700 1 -- 172.21.15.89:0/941744317 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c005cb0 con 0x7f5d7809e000 2023-09-30T22:20:28.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.424+0000 7f5d5f7fe700 1 -- 172.21.15.89:0/941744317 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c005e10 con 0x7f5d7809e000 2023-09-30T22:20:28.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.424+0000 7f5d5f7fe700 1 -- 172.21.15.89:0/941744317 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0072f0 con 0x7f5d7809e000 2023-09-30T22:20:28.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.424+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d680094a0 con 0x7f5d7809e000 2023-09-30T22:20:28.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.424+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7813b3b0 con 0x7f5d7809e000 2023-09-30T22:20:28.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.425+0000 7f5d5f7fe700 1 -- 172.21.15.89:0/941744317 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d6c007450 con 0x7f5d7809e000 2023-09-30T22:20:28.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.425+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d78074ea0 con 0x7f5d7809e000 2023-09-30T22:20:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.425+0000 7f5d5f7fe700 1 -- 172.21.15.89:0/941744317 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5d6c03eb80 con 0x7f5d7809e000 2023-09-30T22:20:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.431+0000 7f5d5f7fe700 1 -- 172.21.15.89:0/941744317 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d6c007700 con 0x7f5d7809e000 2023-09-30T22:20:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.517+0000 7fa81a7f5700 1 Processor -- start 2023-09-30T22:20:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.517+0000 7fa81a7f5700 1 -- start start 2023-09-30T22:20:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa81a7f5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa81409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa81a7f5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa81409e000 0x7fa8141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa81a7f5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e410 0x7fa814133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa81a7f5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81406b460 con 0x7fa81409e000 2023-09-30T22:20:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa81a7f5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81409b120 con 0x7fa81409e410 2023-09-30T22:20:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa81a7f5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa81409a7e0 con 0x7fa81409d6f0 2023-09-30T22:20:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa81409e000 0x7fa8141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa81409e000 0x7fa8141339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58692/0 (socket says 172.21.15.89:58692) 2023-09-30T22:20:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 -- 172.21.15.89:0/3523176475 learned_addr learned my addr 172.21.15.89:0/3523176475 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa818d92700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e410 0x7fa814133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa813fff700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa81409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 -- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 msgr2=0x7fa81409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa81409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 -- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e410 msgr2=0x7fa814133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e410 0x7fa814133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 -- 172.21.15.89:0/3523176475 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81409a500 con 0x7fa81409e000 2023-09-30T22:20:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa813fff700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa81409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.518+0000 7fa8137fe700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e000 0x7fa8141339f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa8080098f0 tx=0x7fa808009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd106a60ad31843a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.519+0000 7fa812ffd700 1 -- 172.21.15.89:0/3523176475 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808011830 con 0x7fa81409e000 2023-09-30T22:20:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.519+0000 7fa812ffd700 1 -- 172.21.15.89:0/3523176475 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa808011990 con 0x7fa81409e000 2023-09-30T22:20:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.519+0000 7fa812ffd700 1 -- 172.21.15.89:0/3523176475 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808011d20 con 0x7fa81409e000 2023-09-30T22:20:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 -- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e000 msgr2=0x7fa8141339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e000 0x7fa8141339f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa8080098f0 tx=0x7fa808009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 -- 172.21.15.89:0/3523176475 shutdown_connections 2023-09-30T22:20:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e410 0x7fa814133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa81409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 --2- 172.21.15.89:0/3523176475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e000 0x7fa8141339f0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 -- 172.21.15.89:0/3523176475 >> 172.21.15.89:0/3523176475 conn(0x7fa814066f50 msgr2=0x7fa81409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 -- 172.21.15.89:0/3523176475 shutdown_connections 2023-09-30T22:20:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.520+0000 7fa81a7f5700 1 -- 172.21.15.89:0/3523176475 wait complete. 2023-09-30T22:20:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.521+0000 7fa81a7f5700 1 Processor -- start 2023-09-30T22:20:28.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.521+0000 7fa81a7f5700 1 -- start start 2023-09-30T22:20:28.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa81a7f5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa8141c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa81a7f5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e000 0x7fa8141c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa81a7f5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e410 0x7fa8141c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa81a7f5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141a9630 con 0x7fa81409e410 2023-09-30T22:20:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa81a7f5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141a9810 con 0x7fa81409e000 2023-09-30T22:20:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa81a7f5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141c5aa0 con 0x7fa81409d6f0 2023-09-30T22:20:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa8137fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e000 0x7fa8141c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa818d92700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e410 0x7fa8141c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa818d92700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e410 0x7fa8141c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58700/0 (socket says 172.21.15.89:58700) 2023-09-30T22:20:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa818d92700 1 -- 172.21.15.89:0/972061219 learned_addr learned my addr 172.21.15.89:0/972061219 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.522+0000 7fa813fff700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa8141c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa813fff700 1 -- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e000 msgr2=0x7fa8141c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa813fff700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e000 0x7fa8141c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa813fff700 1 -- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e410 msgr2=0x7fa8141c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa813fff700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e410 0x7fa8141c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa813fff700 1 -- 172.21.15.89:0/972061219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa808009550 con 0x7fa81409d6f0 2023-09-30T22:20:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa818d92700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e410 0x7fa8141c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa813fff700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa8141c5020 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c820 tx=0x7fa80400cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa810ff9700 1 -- 172.21.15.89:0/972061219 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804011800 con 0x7fa81409d6f0 2023-09-30T22:20:28.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa810ff9700 1 -- 172.21.15.89:0/972061219 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa804011960 con 0x7fa81409d6f0 2023-09-30T22:20:28.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81409c710 con 0x7fa81409d6f0 2023-09-30T22:20:28.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa810ff9700 1 -- 172.21.15.89:0/972061219 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804011cf0 con 0x7fa81409d6f0 2023-09-30T22:20:28.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.523+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8141a7900 con 0x7fa81409d6f0 2023-09-30T22:20:28.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.524+0000 7fa810ff9700 1 -- 172.21.15.89:0/972061219 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa804018360 con 0x7fa81409d6f0 2023-09-30T22:20:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.524+0000 7fa810ff9700 1 -- 172.21.15.89:0/972061219 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa80403f5b0 con 0x7fa81409d6f0 2023-09-30T22:20:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.524+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8141a91d0 con 0x7fa81409d6f0 2023-09-30T22:20:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.533+0000 7fa810ff9700 1 -- 172.21.15.89:0/972061219 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa804018610 con 0x7fa81409d6f0 2023-09-30T22:20:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.546+0000 7f52c1b3e700 1 Processor -- start 2023-09-30T22:20:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.546+0000 7f52c1b3e700 1 -- start start 2023-09-30T22:20:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.547+0000 7f52c1b3e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc09c650 0x7f52bc09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.547+0000 7f52c1b3e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.547+0000 7f52c1b3e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52bc09ba70 con 0x7f52bc0a0090 2023-09-30T22:20:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.547+0000 7f52c1b3e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc09e8f0 con 0x7f52bc09c650 2023-09-30T22:20:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.547+0000 7f52c1b3e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc09d400 con 0x7f52bc09cf60 2023-09-30T22:20:28.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.547+0000 7f52baffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f52bc0a0090 0x7f52bc131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58642/0 (socket says 172.21.15.89:58642) 2023-09-30T22:20:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.547+0000 7f52baffd700 1 -- 172.21.15.89:0/3313653748 learned_addr learned my addr 172.21.15.89:0/3313653748 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.547+0000 7f52bbfff700 1 --2- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52ba7fc700 1 -- 172.21.15.89:0/3313653748 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536657964 0 0) 0x7f52bc09ba70 con 0x7f52bc0a0090 2023-09-30T22:20:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52ba7fc700 1 -- 172.21.15.89:0/3313653748 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a40035b0 con 0x7f52bc0a0090 2023-09-30T22:20:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52bb7fe700 1 --2- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc09c650 0x7f52bc09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52bbfff700 1 -- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc09c650 msgr2=0x7f52bc09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52bbfff700 1 --2- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc09c650 0x7f52bc09ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52bbfff700 1 -- 172.21.15.89:0/3313653748 >> v1:172.21.15.89:6789/0 conn(0x7f52bc0a0090 legacy=0x7f52bc131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52bbfff700 1 -- 172.21.15.89:0/3313653748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc0a0760 con 0x7f52bc09cf60 2023-09-30T22:20:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52bbfff700 1 --2- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc132080 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009dc0 tx=0x7f52b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0c7b299ca2dc1d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52ba7fc700 1 -- 172.21.15.89:0/3313653748 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400e070 con 0x7f52bc09cf60 2023-09-30T22:20:28.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52ba7fc700 1 -- 172.21.15.89:0/3313653748 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b400bae0 con 0x7f52bc09cf60 2023-09-30T22:20:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.548+0000 7f52ba7fc700 1 -- 172.21.15.89:0/3313653748 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400be90 con 0x7f52bc09cf60 2023-09-30T22:20:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.549+0000 7f52c1b3e700 1 -- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 msgr2=0x7f52bc132080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.549+0000 7f52c1b3e700 1 --2- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc132080 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009dc0 tx=0x7f52b400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.549+0000 7f52c1b3e700 1 -- 172.21.15.89:0/3313653748 shutdown_connections 2023-09-30T22:20:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.549+0000 7f52c1b3e700 1 --2- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc09c650 0x7f52bc09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.549+0000 7f52c1b3e700 1 --2- 172.21.15.89:0/3313653748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc132080 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.549+0000 7f52c1b3e700 1 -- 172.21.15.89:0/3313653748 >> 172.21.15.89:0/3313653748 conn(0x7f52bc0a3f70 msgr2=0x7f52bc0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.550+0000 7f52c1b3e700 1 -- 172.21.15.89:0/3313653748 shutdown_connections 2023-09-30T22:20:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.550+0000 7f52c1b3e700 1 -- 172.21.15.89:0/3313653748 wait complete. 2023-09-30T22:20:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.551+0000 7f52c1b3e700 1 Processor -- start 2023-09-30T22:20:28.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.551+0000 7f52c1b3e700 1 -- start start 2023-09-30T22:20:28.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52c1b3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52bc09c650 0x7f52bc11a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52c1b3e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc11ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52c1b3e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc0a0090 0x7f52bc11b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52c1b3e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc12fb00 con 0x7f52bc09c650 2023-09-30T22:20:28.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52c1b3e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc11bc80 con 0x7f52bc0a0090 2023-09-30T22:20:28.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52c1b3e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc09d980 con 0x7f52bc09cf60 2023-09-30T22:20:28.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52bc09c650 0x7f52bc11a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52bc09c650 0x7f52bc11a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58712/0 (socket says 172.21.15.89:58712) 2023-09-30T22:20:28.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 -- 172.21.15.89:0/835604778 learned_addr learned my addr 172.21.15.89:0/835604778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:28.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bbfff700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc0a0090 0x7f52bc11b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52baffd700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc11ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 -- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 msgr2=0x7f52bc11ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc11ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 -- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc0a0090 msgr2=0x7f52bc11b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc0a0090 0x7f52bc11b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 -- 172.21.15.89:0/835604778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4009a70 con 0x7f52bc09c650 2023-09-30T22:20:28.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.552+0000 7f52bb7fe700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52bc09c650 0x7f52bc11a620 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e9b0 tx=0x7f52ac00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:28.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.553+0000 7f52a3fff700 1 -- 172.21.15.89:0/835604778 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac009710 con 0x7f52bc09c650 2023-09-30T22:20:28.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.553+0000 7f52a3fff700 1 -- 172.21.15.89:0/835604778 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00c850 con 0x7f52bc09c650 2023-09-30T22:20:28.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.553+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc11d7a0 con 0x7f52bc09c650 2023-09-30T22:20:28.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.553+0000 7f52a3fff700 1 -- 172.21.15.89:0/835604778 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00cbc0 con 0x7f52bc09c650 2023-09-30T22:20:28.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.553+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc072420 con 0x7f52bc09c650 2023-09-30T22:20:28.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.554+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52bc11d5e0 con 0x7f52bc09c650 2023-09-30T22:20:28.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.554+0000 7f52a3fff700 1 -- 172.21.15.89:0/835604778 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f52ac00cd20 con 0x7f52bc09c650 2023-09-30T22:20:28.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.554+0000 7f52a3fff700 1 -- 172.21.15.89:0/835604778 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f52ac040b20 con 0x7f52bc09c650 2023-09-30T22:20:28.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.559+0000 7f52a3fff700 1 -- 172.21.15.89:0/835604778 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f52ac01abe0 con 0x7f52bc09c650 2023-09-30T22:20:28.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.571+0000 7f5810254700 1 Processor -- start 2023-09-30T22:20:28.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.571+0000 7f5810254700 1 -- start start 2023-09-30T22:20:28.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.571+0000 7f5810254700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 0x7f580809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.571+0000 7f5810254700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f580809e000 0x7f5808130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.571+0000 7f5810254700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f5808130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.571+0000 7f5810254700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580809f520 con 0x7f580809e000 2023-09-30T22:20:28.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.571+0000 7f5810254700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580809f130 con 0x7f580809d6f0 2023-09-30T22:20:28.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.571+0000 7f5810254700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f580809e930 con 0x7f580809e410 2023-09-30T22:20:28.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f580809e000 0x7f5808130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f580809e000 0x7f5808130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58716/0 (socket says 172.21.15.89:58716) 2023-09-30T22:20:28.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 -- 172.21.15.89:0/1737221201 learned_addr learned my addr 172.21.15.89:0/1737221201 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:28.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 -- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 msgr2=0x7f5808130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:28.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580e7f1700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f5808130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f5808130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 -- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 msgr2=0x7f580809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 0x7f580809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 -- 172.21.15.89:0/1737221201 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580809b2d0 con 0x7f580809e000 2023-09-30T22:20:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580e7f1700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f5808130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580d7ef700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f5808130660 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f57f800bbc0 tx=0x7f57f800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6f5a7ee98462e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580cfee700 1 -- 172.21.15.89:0/1737221201 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f800dd50 con 0x7f580809e000 2023-09-30T22:20:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580cfee700 1 -- 172.21.15.89:0/1737221201 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f8010470 con 0x7f580809e000 2023-09-30T22:20:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.572+0000 7f580cfee700 1 -- 172.21.15.89:0/1737221201 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8010800 con 0x7f580809e000 2023-09-30T22:20:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 -- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 msgr2=0x7f5808130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f5808130660 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f57f800bbc0 tx=0x7f57f800bed0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 -- 172.21.15.89:0/1737221201 shutdown_connections 2023-09-30T22:20:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f5808130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 0x7f580809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 --2- 172.21.15.89:0/1737221201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f5808130660 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 -- 172.21.15.89:0/1737221201 >> 172.21.15.89:0/1737221201 conn(0x7f5808066f50 msgr2=0x7f580809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 -- 172.21.15.89:0/1737221201 shutdown_connections 2023-09-30T22:20:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.573+0000 7f5810254700 1 -- 172.21.15.89:0/1737221201 wait complete. 2023-09-30T22:20:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.574+0000 7f5810254700 1 Processor -- start 2023-09-30T22:20:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.574+0000 7f5810254700 1 -- start start 2023-09-30T22:20:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f5810254700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 0x7f580811e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f5810254700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f580811e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f5810254700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f580811ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f5810254700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081214d0 con 0x7f580809e000 2023-09-30T22:20:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f5810254700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580811fec0 con 0x7f580809d6f0 2023-09-30T22:20:28.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f5810254700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f580811f6f0 con 0x7f580809e410 2023-09-30T22:20:28.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f580811e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f580811e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58726/0 (socket says 172.21.15.89:58726) 2023-09-30T22:20:28.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 -- 172.21.15.89:0/646929641 learned_addr learned my addr 172.21.15.89:0/646929641 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:28.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580e7f1700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f580811ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580dff0700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 0x7f580811e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 -- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 msgr2=0x7f580811ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f580811ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 -- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 msgr2=0x7f580811e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 0x7f580811e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 -- 172.21.15.89:0/646929641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f800b7f0 con 0x7f580809e000 2023-09-30T22:20:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.575+0000 7f580d7ef700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f580811e7f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f57f8015000 tx=0x7f57f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.576+0000 7f57fe7fc700 1 -- 172.21.15.89:0/646929641 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f801f070 con 0x7f580809e000 2023-09-30T22:20:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.576+0000 7f57fe7fc700 1 -- 172.21.15.89:0/646929641 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f8003930 con 0x7f580809e000 2023-09-30T22:20:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.576+0000 7f57fe7fc700 1 -- 172.21.15.89:0/646929641 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003cc0 con 0x7f580809e000 2023-09-30T22:20:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.576+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5808121dd0 con 0x7f580809e000 2023-09-30T22:20:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.576+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080a1d60 con 0x7f580809e000 2023-09-30T22:20:28.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.577+0000 7f57fe7fc700 1 -- 172.21.15.89:0/646929641 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f57f8019400 con 0x7f580809e000 2023-09-30T22:20:28.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.577+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5808121790 con 0x7f580809e000 2023-09-30T22:20:28.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.578+0000 7f57fe7fc700 1 -- 172.21.15.89:0/646929641 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f57f8043190 con 0x7f580809e000 2023-09-30T22:20:28.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.586+0000 7f57fe7fc700 1 -- 172.21.15.89:0/646929641 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5808121790 con 0x7f580809e000 2023-09-30T22:20:28.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.640+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5d7806c360 con 0x7f5d7809e000 2023-09-30T22:20:28.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.640+0000 7f5d5f7fe700 1 -- 172.21.15.89:0/941744317 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d6c017070 con 0x7f5d7809e000 2023-09-30T22:20:28.642 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:28.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.643+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809e000 msgr2=0x7f5d78139e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.643+0000 7f5d7d7fc700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809e000 0x7f5d78139e80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c550 tx=0x7f5d6c00c860 comp rx=0 tx=0).stop 2023-09-30T22:20:28.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.643+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 shutdown_connections 2023-09-30T22:20:28.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.643+0000 7f5d7d7fc700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7809e410 0x7f5d7813a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.643+0000 7f5d7d7fc700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d7809d6f0 0x7f5d78139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.643+0000 7f5d7d7fc700 1 --2- 172.21.15.89:0/941744317 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d7809e000 0x7f5d78139e80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.643+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 >> 172.21.15.89:0/941744317 conn(0x7f5d78066f50 msgr2=0x7f5d78070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:28.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.644+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 shutdown_connections 2023-09-30T22:20:28.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.644+0000 7f5d7d7fc700 1 -- 172.21.15.89:0/941744317 wait complete. 2023-09-30T22:20:28.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:28.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.739+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa814074ea0 con 0x7fa81409d6f0 2023-09-30T22:20:28.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.739+0000 7fa810ff9700 1 -- 172.21.15.89:0/972061219 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa804019ed0 con 0x7fa81409d6f0 2023-09-30T22:20:28.740 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:28.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 msgr2=0x7fa8141c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa8141c5020 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c820 tx=0x7fa80400cb30 comp rx=0 tx=0).stop 2023-09-30T22:20:28.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 shutdown_connections 2023-09-30T22:20:28.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa81409e410 0x7fa8141c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa81409d6f0 0x7fa8141c5020 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 --2- 172.21.15.89:0/972061219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa81409e000 0x7fa8141c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 >> 172.21.15.89:0/972061219 conn(0x7fa814066f50 msgr2=0x7fa8141badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:28.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 shutdown_connections 2023-09-30T22:20:28.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.741+0000 7fa81a7f5700 1 -- 172.21.15.89:0/972061219 wait complete. 2023-09-30T22:20:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.752+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f52bc11cda0 con 0x7f52bc09c650 2023-09-30T22:20:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.752+0000 7f52a3fff700 1 -- 172.21.15.89:0/835604778 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f52ac017070 con 0x7f52bc09c650 2023-09-30T22:20:28.754 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:28.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52bc09c650 msgr2=0x7f52bc11a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52bc09c650 0x7f52bc11a620 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e9b0 tx=0x7f52ac00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 shutdown_connections 2023-09-30T22:20:28.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52bc0a0090 0x7f52bc11b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52bc09c650 0x7f52bc11a620 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 --2- 172.21.15.89:0/835604778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52bc09cf60 0x7f52bc11ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 >> 172.21.15.89:0/835604778 conn(0x7f52bc0a3f70 msgr2=0x7f52bc12ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 shutdown_connections 2023-09-30T22:20:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.754+0000 7f52c1b3e700 1 -- 172.21.15.89:0/835604778 wait complete. 2023-09-30T22:20:28.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:28.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:28.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.794+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5808121790 con 0x7f580809e000 2023-09-30T22:20:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.794+0000 7f57fe7fc700 1 -- 172.21.15.89:0/646929641 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5808121790 con 0x7f580809e000 2023-09-30T22:20:28.795 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:28.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 msgr2=0x7f580811e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:28.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f580811e7f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f57f8015000 tx=0x7f57f8004750 comp rx=0 tx=0).stop 2023-09-30T22:20:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 shutdown_connections 2023-09-30T22:20:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f580809e410 0x7f580811ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f580809d6f0 0x7f580811e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 --2- 172.21.15.89:0/646929641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f580809e000 0x7f580811e7f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 >> 172.21.15.89:0/646929641 conn(0x7f5808066f50 msgr2=0x7f580806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 shutdown_connections 2023-09-30T22:20:28.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:28.796+0000 7f5810254700 1 -- 172.21.15.89:0/646929641 wait complete. 2023-09-30T22:20:28.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:29.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:29.759 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:29.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:29.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.794+0000 7fcdd73d8700 1 Processor -- start 2023-09-30T22:20:29.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd73d8700 1 -- start start 2023-09-30T22:20:29.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd73d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd73d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd73d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd009b120 con 0x7fcdd009d6f0 2023-09-30T22:20:29.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd73d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd006b460 con 0x7fcdd009e1d0 2023-09-30T22:20:29.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd73d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd009a7e0 con 0x7fcdd0139170 2023-09-30T22:20:29.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd4973700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd5975700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd4973700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58950/0 (socket says 172.21.15.89:58950) 2023-09-30T22:20:29.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd5975700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36900/0 (socket says 172.21.15.89:36900) 2023-09-30T22:20:29.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.795+0000 7fcdd4973700 1 -- 172.21.15.89:0/3715836011 learned_addr learned my addr 172.21.15.89:0/3715836011 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:29.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdc7fff700 1 -- 172.21.15.89:0/3715836011 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027633585 0 0) 0x7fcdd009b120 con 0x7fcdd009d6f0 2023-09-30T22:20:29.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdd5975700 1 -- 172.21.15.89:0/3715836011 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 msgr2=0x7fcdd0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdd5975700 1 --2- 172.21.15.89:0/3715836011 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdd5975700 1 -- 172.21.15.89:0/3715836011 >> v1:172.21.15.89:6789/0 conn(0x7fcdd009d6f0 legacy=0x7fcdd009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdd5975700 1 -- 172.21.15.89:0/3715836011 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd009a500 con 0x7fcdd0139170 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdd5975700 1 --2- 172.21.15.89:0/3715836011 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd013b550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009df0 tx=0x7fcdc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d03536b26a628d28 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdc7fff700 1 -- 172.21.15.89:0/3715836011 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc800e070 con 0x7fcdd0139170 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdc7fff700 1 -- 172.21.15.89:0/3715836011 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc800bae0 con 0x7fcdd0139170 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.796+0000 7fcdc7fff700 1 -- 172.21.15.89:0/3715836011 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc800be90 con 0x7fcdd0139170 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.797+0000 7fcdd73d8700 1 -- 172.21.15.89:0/3715836011 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 msgr2=0x7fcdd013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.797+0000 7fcdd73d8700 1 --2- 172.21.15.89:0/3715836011 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd013b550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009df0 tx=0x7fcdc800b330 comp rx=0 tx=0).stop 2023-09-30T22:20:29.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.797+0000 7fcdd73d8700 1 -- 172.21.15.89:0/3715836011 shutdown_connections 2023-09-30T22:20:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.797+0000 7fcdd73d8700 1 --2- 172.21.15.89:0/3715836011 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd013b550 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.797+0000 7fcdd73d8700 1 --2- 172.21.15.89:0/3715836011 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.797+0000 7fcdd73d8700 1 -- 172.21.15.89:0/3715836011 >> 172.21.15.89:0/3715836011 conn(0x7fcdd0066f50 msgr2=0x7fcdd009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.797+0000 7fcdd73d8700 1 -- 172.21.15.89:0/3715836011 shutdown_connections 2023-09-30T22:20:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.797+0000 7fcdd73d8700 1 -- 172.21.15.89:0/3715836011 wait complete. 2023-09-30T22:20:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd73d8700 1 Processor -- start 2023-09-30T22:20:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd73d8700 1 -- start start 2023-09-30T22:20:29.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd73d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd009d6f0 0x7fcdd0121870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd73d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0121db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd73d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd01222f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd73d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0125110 con 0x7fcdd009d6f0 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd73d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd009c710 con 0x7fcdd009e1d0 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd73d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0123390 con 0x7fcdd0139170 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd5975700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd01222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd4973700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0121db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd5975700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd01222f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36904/0 (socket says 172.21.15.89:36904) 2023-09-30T22:20:29.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd5975700 1 -- 172.21.15.89:0/2632550558 learned_addr learned my addr 172.21.15.89:0/2632550558 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.798+0000 7fcdd5174700 1 --2- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd009d6f0 0x7fcdd0121870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdd5975700 1 -- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 msgr2=0x7fcdd0121db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdd5975700 1 --2- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0121db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdd5975700 1 -- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd009d6f0 msgr2=0x7fcdd0121870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdd5975700 1 --2- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd009d6f0 0x7fcdd0121870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdd5975700 1 -- 172.21.15.89:0/2632550558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc8009a70 con 0x7fcdd0139170 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdd5975700 1 --2- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd01222f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800d010 tx=0x7fcdc8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdc5ffb700 1 -- 172.21.15.89:0/2632550558 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80045d0 con 0x7fcdd0139170 2023-09-30T22:20:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdc5ffb700 1 -- 172.21.15.89:0/2632550558 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc8020070 con 0x7fcdd0139170 2023-09-30T22:20:29.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdc5ffb700 1 -- 172.21.15.89:0/2632550558 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8004970 con 0x7fcdd0139170 2023-09-30T22:20:29.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd00a3a90 con 0x7fcdd0139170 2023-09-30T22:20:29.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.799+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd0125aa0 con 0x7fcdd0139170 2023-09-30T22:20:29.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.800+0000 7fcdc5ffb700 1 -- 172.21.15.89:0/2632550558 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcdc8012430 con 0x7fcdd0139170 2023-09-30T22:20:29.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.800+0000 7fcdc5ffb700 1 -- 172.21.15.89:0/2632550558 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcdc8022070 con 0x7fcdd0139170 2023-09-30T22:20:29.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.801+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd0074ea0 con 0x7fcdd0139170 2023-09-30T22:20:29.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.805+0000 7fcdc5ffb700 1 -- 172.21.15.89:0/2632550558 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcdc801b400 con 0x7fcdd0139170 2023-09-30T22:20:29.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:29.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77fe742700 1 Processor -- start 2023-09-30T22:20:29.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77fe742700 1 -- start start 2023-09-30T22:20:29.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77fe742700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77fe742700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77f8139170 0x7f77f813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77fe742700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f809b120 con 0x7f77f809d6f0 2023-09-30T22:20:29.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77fe742700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f806b460 con 0x7f77f8139170 2023-09-30T22:20:29.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77fe742700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f809a7e0 con 0x7f77f809e1d0 2023-09-30T22:20:29.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77fccdf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77f8139170 0x7f77f813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.891+0000 7f77f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77fccdf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77f8139170 0x7f77f813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58748/0 (socket says 172.21.15.89:58748) 2023-09-30T22:20:29.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58968/0 (socket says 172.21.15.89:58968) 2023-09-30T22:20:29.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77fccdf700 1 -- 172.21.15.89:0/3865648666 learned_addr learned my addr 172.21.15.89:0/3865648666 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:29.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f77fe700 1 -- 172.21.15.89:0/3865648666 >> v2:172.21.15.89:3300/0 conn(0x7f77f8139170 msgr2=0x7f77f813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:29.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f77fe700 1 --2- 172.21.15.89:0/3865648666 >> v2:172.21.15.89:3300/0 conn(0x7f77f8139170 0x7f77f813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f77fe700 1 -- 172.21.15.89:0/3865648666 >> v1:172.21.15.89:6789/0 conn(0x7f77f809d6f0 legacy=0x7f77f809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f77fe700 1 -- 172.21.15.89:0/3865648666 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f809a500 con 0x7f77f809e1d0 2023-09-30T22:20:29.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f77fe700 1 --2- 172.21.15.89:0/3865648666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8138c30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009870 tx=0x7f77ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55279d6fe3053052 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:29.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f6ffd700 1 -- 172.21.15.89:0/3865648666 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011830 con 0x7f77f809e1d0 2023-09-30T22:20:29.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f6ffd700 1 -- 172.21.15.89:0/3865648666 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec011990 con 0x7f77f809e1d0 2023-09-30T22:20:29.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.892+0000 7f77f6ffd700 1 -- 172.21.15.89:0/3865648666 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011d20 con 0x7f77f809e1d0 2023-09-30T22:20:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.893+0000 7f77fe742700 1 -- 172.21.15.89:0/3865648666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 msgr2=0x7f77f8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.893+0000 7f77fe742700 1 --2- 172.21.15.89:0/3865648666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8138c30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009870 tx=0x7f77ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.893+0000 7f77fe742700 1 -- 172.21.15.89:0/3865648666 shutdown_connections 2023-09-30T22:20:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.893+0000 7f77fe742700 1 --2- 172.21.15.89:0/3865648666 >> v2:172.21.15.89:3300/0 conn(0x7f77f8139170 0x7f77f813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.893+0000 7f77fe742700 1 --2- 172.21.15.89:0/3865648666 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8138c30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.893+0000 7f77fe742700 1 -- 172.21.15.89:0/3865648666 >> 172.21.15.89:0/3865648666 conn(0x7f77f8066f50 msgr2=0x7f77f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.893+0000 7f77fe742700 1 -- 172.21.15.89:0/3865648666 shutdown_connections 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.893+0000 7f77fe742700 1 -- 172.21.15.89:0/3865648666 wait complete. 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.894+0000 7f77fe742700 1 Processor -- start 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.894+0000 7f77fe742700 1 -- start start 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77fe742700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f809d6f0 0x7f77f8133860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77fe742700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8133da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77fe742700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f8139170 0x7f77f811e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77fe742700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f809c710 con 0x7f77f809d6f0 2023-09-30T22:20:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77fe742700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8120af0 con 0x7f77f809e1d0 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77fe742700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80a2b90 con 0x7f77f8139170 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f809d6f0 0x7f77f8133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f809d6f0 0x7f77f8133860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58762/0 (socket says 172.21.15.89:58762) 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 -- 172.21.15.89:0/3122595517 learned_addr learned my addr 172.21.15.89:0/3122595517 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77fccdf700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f8139170 0x7f77f811e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f77fe700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 -- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f8139170 msgr2=0x7f77f811e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f8139170 0x7f77f811e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 -- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 msgr2=0x7f77f8133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 -- 172.21.15.89:0/3122595517 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec0094a0 con 0x7f77f809d6f0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.895+0000 7f77f7fff700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f809d6f0 0x7f77f8133860 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c4e0 tx=0x7f77e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.896+0000 7f77f4ff9700 1 -- 172.21.15.89:0/3122595517 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e80052d0 con 0x7f77f809d6f0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.896+0000 7f77f4ff9700 1 -- 172.21.15.89:0/3122595517 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8005430 con 0x7f77f809d6f0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.896+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f81205e0 con 0x7f77f809d6f0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.896+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f8120d70 con 0x7f77f809d6f0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.896+0000 7f77f4ff9700 1 -- 172.21.15.89:0/3122595517 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e80038c0 con 0x7f77f809d6f0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.897+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d80052e0 con 0x7f77f809d6f0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.897+0000 7f77f4ff9700 1 -- 172.21.15.89:0/3122595517 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f77e8003a20 con 0x7f77f809d6f0 2023-09-30T22:20:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.898+0000 7f77f4ff9700 1 -- 172.21.15.89:0/3122595517 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f77e803eb40 con 0x7f77f809d6f0 2023-09-30T22:20:29.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.902+0000 7f77f4ff9700 1 -- 172.21.15.89:0/3122595517 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f77e80102e0 con 0x7f77f809d6f0 2023-09-30T22:20:29.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.933+0000 7f9c16d2a700 1 Processor -- start 2023-09-30T22:20:29.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.933+0000 7f9c16d2a700 1 -- start start 2023-09-30T22:20:29.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.933+0000 7f9c16d2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009e930 0x7f9c1009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.933+0000 7f9c16d2a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c1009f240 0x7f9c101317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.933+0000 7f9c16d2a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c10070940 con 0x7f9c1009f650 2023-09-30T22:20:29.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.933+0000 7f9c16d2a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1006fd60 con 0x7f9c1009f240 2023-09-30T22:20:29.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.933+0000 7f9c16d2a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1006fa80 con 0x7f9c1009e930 2023-09-30T22:20:29.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c1009f240 0x7f9c101317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c14ac6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009e930 0x7f9c1009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c14ac6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009e930 0x7f9c1009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58990/0 (socket says 172.21.15.89:58990) 2023-09-30T22:20:29.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c1009f240 0x7f9c101317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58766/0 (socket says 172.21.15.89:58766) 2023-09-30T22:20:29.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c14ac6700 1 -- 172.21.15.89:0/3175043388 learned_addr learned my addr 172.21.15.89:0/3175043388 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:29.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0ffff700 1 -- 172.21.15.89:0/3175043388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009e930 msgr2=0x7f9c1009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:29.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0ffff700 1 --2- 172.21.15.89:0/3175043388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009e930 0x7f9c1009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0ffff700 1 -- 172.21.15.89:0/3175043388 >> v1:172.21.15.89:6789/0 conn(0x7f9c1009f650 legacy=0x7f9c10131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0ffff700 1 -- 172.21.15.89:0/3175043388 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1009fb60 con 0x7f9c1009f240 2023-09-30T22:20:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c14ac6700 1 --2- 172.21.15.89:0/3175043388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009e930 0x7f9c1009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0ffff700 1 --2- 172.21.15.89:0/3175043388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c101317f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009840 tx=0x7f9bfc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5853bdf6626b6df4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0f7fe700 1 -- 172.21.15.89:0/3175043388 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc011830 con 0x7f9c1009f240 2023-09-30T22:20:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0f7fe700 1 -- 172.21.15.89:0/3175043388 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc011990 con 0x7f9c1009f240 2023-09-30T22:20:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.934+0000 7f9c0f7fe700 1 -- 172.21.15.89:0/3175043388 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc011d20 con 0x7f9c1009f240 2023-09-30T22:20:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.935+0000 7f9c16d2a700 1 -- 172.21.15.89:0/3175043388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 msgr2=0x7f9c101317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.935+0000 7f9c16d2a700 1 --2- 172.21.15.89:0/3175043388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c101317f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009840 tx=0x7f9bfc009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:29.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.935+0000 7f9c16d2a700 1 -- 172.21.15.89:0/3175043388 shutdown_connections 2023-09-30T22:20:29.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.935+0000 7f9c16d2a700 1 --2- 172.21.15.89:0/3175043388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009e930 0x7f9c1009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.935+0000 7f9c16d2a700 1 --2- 172.21.15.89:0/3175043388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c101317f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.935+0000 7f9c16d2a700 1 -- 172.21.15.89:0/3175043388 >> 172.21.15.89:0/3175043388 conn(0x7f9c100a4210 msgr2=0x7f9c100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:29.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.935+0000 7f9c16d2a700 1 -- 172.21.15.89:0/3175043388 shutdown_connections 2023-09-30T22:20:29.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.935+0000 7f9c16d2a700 1 -- 172.21.15.89:0/3175043388 wait complete. 2023-09-30T22:20:29.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.936+0000 7f9c16d2a700 1 Processor -- start 2023-09-30T22:20:29.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.936+0000 7f9c16d2a700 1 -- start start 2023-09-30T22:20:29.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c16d2a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c1009e930 0x7f9c1006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c16d2a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c1006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c16d2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009f650 0x7f9c1006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c16d2a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1006cce0 con 0x7f9c1009f240 2023-09-30T22:20:29.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c16d2a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c101a9af0 con 0x7f9c1009f650 2023-09-30T22:20:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c16d2a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c101a8a90 con 0x7f9c1009e930 2023-09-30T22:20:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c0ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c1006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c0ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c1006bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58768/0 (socket says 172.21.15.89:58768) 2023-09-30T22:20:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c0ffff700 1 -- 172.21.15.89:0/1897758456 learned_addr learned my addr 172.21.15.89:0/1897758456 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.937+0000 7f9c152c7700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009f650 0x7f9c1006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c14ac6700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c1009e930 0x7f9c1006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0ffff700 1 -- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c1009e930 msgr2=0x7f9c1006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0ffff700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c1009e930 0x7f9c1006b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0ffff700 1 -- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009f650 msgr2=0x7f9c1006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0ffff700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009f650 0x7f9c1006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0ffff700 1 -- 172.21.15.89:0/1897758456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c04009550 con 0x7f9c1009f240 2023-09-30T22:20:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c152c7700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009f650 0x7f9c1006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0ffff700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c1006bdc0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc000c00 tx=0x7f9bfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0d7fa700 1 -- 172.21.15.89:0/1897758456 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0036a0 con 0x7f9c1009f240 2023-09-30T22:20:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0d7fa700 1 -- 172.21.15.89:0/1897758456 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc01f070 con 0x7f9c1009f240 2023-09-30T22:20:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc0094a0 con 0x7f9c1009f240 2023-09-30T22:20:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c0d7fa700 1 -- 172.21.15.89:0/1897758456 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc003a40 con 0x7f9c1009f240 2023-09-30T22:20:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.938+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c100a0190 con 0x7f9c1009f240 2023-09-30T22:20:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.939+0000 7f9c0d7fa700 1 -- 172.21.15.89:0/1897758456 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9bfc003ba0 con 0x7f9c1009f240 2023-09-30T22:20:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.940+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c10071d20 con 0x7f9c1009f240 2023-09-30T22:20:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.940+0000 7f9c0d7fa700 1 -- 172.21.15.89:0/1897758456 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9bfc021070 con 0x7f9c1009f240 2023-09-30T22:20:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.944+0000 7f2055c91700 1 Processor -- start 2023-09-30T22:20:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.944+0000 7f9c0d7fa700 1 -- 172.21.15.89:0/1897758456 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9bfc015070 con 0x7f9c1009f240 2023-09-30T22:20:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.944+0000 7f2055c91700 1 -- start start 2023-09-30T22:20:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f2055c91700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f205009d6f0 0x7f205009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f2055c91700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f205009e410 0x7f2050130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f2055c91700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205009f130 con 0x7f205009e000 2023-09-30T22:20:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f2055c91700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205009f520 con 0x7f205009e410 2023-09-30T22:20:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f2055c91700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f205009e930 con 0x7f205009d6f0 2023-09-30T22:20:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f205009e410 0x7f2050130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f205009e410 0x7f2050130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58770/0 (socket says 172.21.15.89:58770) 2023-09-30T22:20:29.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204ffff700 1 -- 172.21.15.89:0/1048187180 learned_addr learned my addr 172.21.15.89:0/1048187180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:29.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204effd700 1 --1- 172.21.15.89:0/1048187180 >> v1:172.21.15.89:6789/0 conn(0x7f205009e000 0x7f2050130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58682/0 (socket says 172.21.15.89:58682) 2023-09-30T22:20:29.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204f7fe700 1 --2- 172.21.15.89:0/1048187180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f205009d6f0 0x7f205009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204ffff700 1 -- 172.21.15.89:0/1048187180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f205009d6f0 msgr2=0x7f205009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:29.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204ffff700 1 --2- 172.21.15.89:0/1048187180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f205009d6f0 0x7f205009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204ffff700 1 -- 172.21.15.89:0/1048187180 >> v1:172.21.15.89:6789/0 conn(0x7f205009e000 legacy=0x7f2050130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204ffff700 1 -- 172.21.15.89:0/1048187180 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205009b2d0 con 0x7f205009e410 2023-09-30T22:20:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.945+0000 7f204e7fc700 1 -- 172.21.15.89:0/1048187180 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864291822 0 0) 0x7f205009f130 con 0x7f205009e000 2023-09-30T22:20:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f204ffff700 1 --2- 172.21.15.89:0/1048187180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 0x7f2050130d70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2048009e50 tx=0x7f204800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9c9f5b9ed2960f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f204e7fc700 1 -- 172.21.15.89:0/1048187180 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800e070 con 0x7f205009e410 2023-09-30T22:20:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f204e7fc700 1 -- 172.21.15.89:0/1048187180 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204800b9c0 con 0x7f205009e410 2023-09-30T22:20:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f204e7fc700 1 -- 172.21.15.89:0/1048187180 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800bd70 con 0x7f205009e410 2023-09-30T22:20:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f2055c91700 1 -- 172.21.15.89:0/1048187180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 msgr2=0x7f2050130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f2055c91700 1 --2- 172.21.15.89:0/1048187180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 0x7f2050130d70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2048009e50 tx=0x7f204800b330 comp rx=0 tx=0).stop 2023-09-30T22:20:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f2055c91700 1 -- 172.21.15.89:0/1048187180 shutdown_connections 2023-09-30T22:20:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f2055c91700 1 --2- 172.21.15.89:0/1048187180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 0x7f2050130d70 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f2055c91700 1 --2- 172.21.15.89:0/1048187180 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f205009d6f0 0x7f205009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.946+0000 7f2055c91700 1 -- 172.21.15.89:0/1048187180 >> 172.21.15.89:0/1048187180 conn(0x7f2050066f50 msgr2=0x7f205009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.949+0000 7f2055c91700 1 -- 172.21.15.89:0/1048187180 shutdown_connections 2023-09-30T22:20:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.949+0000 7f2055c91700 1 -- 172.21.15.89:0/1048187180 wait complete. 2023-09-30T22:20:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.949+0000 7f2055c91700 1 Processor -- start 2023-09-30T22:20:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f2055c91700 1 -- start start 2023-09-30T22:20:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f2055c91700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f205009e000 0x7f2050122280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f2055c91700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 0x7f20501227c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f2055c91700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2050122d00 0x7f20501c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f2055c91700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050123fa0 con 0x7f205009e410 2023-09-30T22:20:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f2055c91700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2050125ac0 con 0x7f205009e000 2023-09-30T22:20:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f2055c91700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20500a1d00 con 0x7f2050122d00 2023-09-30T22:20:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f205009e000 0x7f2050122280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f205009e000 0x7f2050122280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59012/0 (socket says 172.21.15.89:59012) 2023-09-30T22:20:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f204ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2050122d00 0x7f20501c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f204f7fe700 1 -- 172.21.15.89:0/2333405652 learned_addr learned my addr 172.21.15.89:0/2333405652 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.950+0000 7f204effd700 1 --2- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 0x7f20501227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f204ffff700 1 -- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f205009e000 msgr2=0x7f2050122280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f204ffff700 1 --2- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f205009e000 0x7f2050122280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f204ffff700 1 -- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 msgr2=0x7f20501227c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f204ffff700 1 --2- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 0x7f20501227c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f204ffff700 1 -- 172.21.15.89:0/2333405652 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2048009b30 con 0x7f2050122d00 2023-09-30T22:20:29.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f204ffff700 1 --2- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2050122d00 0x7f20501c84a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2048015000 tx=0x7f2048004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:29.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f2037fff700 1 -- 172.21.15.89:0/2333405652 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048004880 con 0x7f2050122d00 2023-09-30T22:20:29.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f2037fff700 1 -- 172.21.15.89:0/2333405652 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20480092e0 con 0x7f2050122d00 2023-09-30T22:20:29.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f2037fff700 1 -- 172.21.15.89:0/2333405652 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204801c660 con 0x7f2050122d00 2023-09-30T22:20:29.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205009c700 con 0x7f2050122d00 2023-09-30T22:20:29.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.951+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2050126530 con 0x7f2050122d00 2023-09-30T22:20:29.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.952+0000 7f2037fff700 1 -- 172.21.15.89:0/2333405652 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f204801c7c0 con 0x7f2050122d00 2023-09-30T22:20:29.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.952+0000 7f2037fff700 1 -- 172.21.15.89:0/2333405652 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2048040e20 con 0x7f2050122d00 2023-09-30T22:20:29.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.952+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2050074ea0 con 0x7f2050122d00 2023-09-30T22:20:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:29.957+0000 7f2037fff700 1 -- 172.21.15.89:0/2333405652 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20480123f0 con 0x7f2050122d00 2023-09-30T22:20:30.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.057+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcdd006c360 con 0x7fcdd0139170 2023-09-30T22:20:30.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.057+0000 7fcdc5ffb700 1 -- 172.21.15.89:0/2632550558 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdc80263e0 con 0x7fcdd0139170 2023-09-30T22:20:30.058 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 msgr2=0x7fcdd01222f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 --2- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd01222f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800d010 tx=0x7fcdc8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 shutdown_connections 2023-09-30T22:20:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 --2- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd0139170 0x7fcdd01222f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 --2- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd009d6f0 0x7fcdd0121870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 --2- 172.21.15.89:0/2632550558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd009e1d0 0x7fcdd0121db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 >> 172.21.15.89:0/2632550558 conn(0x7fcdd0066f50 msgr2=0x7fcdd0070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 shutdown_connections 2023-09-30T22:20:30.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.060+0000 7fcdd73d8700 1 -- 172.21.15.89:0/2632550558 wait complete. 2023-09-30T22:20:30.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:30.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.106+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f77d8005150 con 0x7f77f809d6f0 2023-09-30T22:20:30.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.106+0000 7f77f4ff9700 1 -- 172.21.15.89:0/3122595517 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f77e8017030 con 0x7f77f809d6f0 2023-09-30T22:20:30.107 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:30.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f809d6f0 msgr2=0x7f77f8133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:30.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f809d6f0 0x7f77f8133860 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c4e0 tx=0x7f77e800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 shutdown_connections 2023-09-30T22:20:30.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f8139170 0x7f77f811e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f809d6f0 0x7f77f8133860 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 --2- 172.21.15.89:0/3122595517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f809e1d0 0x7f77f8133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 >> 172.21.15.89:0/3122595517 conn(0x7f77f8066f50 msgr2=0x7f77f80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:30.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 shutdown_connections 2023-09-30T22:20:30.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.108+0000 7f77fe742700 1 -- 172.21.15.89:0/3122595517 wait complete. 2023-09-30T22:20:30.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:30.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.151+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f20501301a0 con 0x7f2050122d00 2023-09-30T22:20:30.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.152+0000 7f2037fff700 1 -- 172.21.15.89:0/2333405652 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2048034090 con 0x7f2050122d00 2023-09-30T22:20:30.153 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:30.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.153+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2050122d00 msgr2=0x7f20501c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:30.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.153+0000 7f2055c91700 1 --2- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2050122d00 0x7f20501c84a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2048015000 tx=0x7f2048004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.153+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 shutdown_connections 2023-09-30T22:20:30.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.153+0000 7f2055c91700 1 --2- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2050122d00 0x7f20501c84a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.153+0000 7f2055c91700 1 --2- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f205009e000 0x7f2050122280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.153+0000 7f2055c91700 1 --2- 172.21.15.89:0/2333405652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205009e410 0x7f20501227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.153+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 >> 172.21.15.89:0/2333405652 conn(0x7f2050066f50 msgr2=0x7f2050070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:30.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.154+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 shutdown_connections 2023-09-30T22:20:30.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.154+0000 7f2055c91700 1 -- 172.21.15.89:0/2333405652 wait complete. 2023-09-30T22:20:30.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:30.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.187+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9c10074ea0 con 0x7f9c1009f240 2023-09-30T22:20:30.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.188+0000 7f9c0d7fa700 1 -- 172.21.15.89:0/1897758456 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9bfc026480 con 0x7f9c1009f240 2023-09-30T22:20:30.188 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 msgr2=0x7f9c1006bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c1006bdc0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc000c00 tx=0x7f9bfc004750 comp rx=0 tx=0).stop 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 shutdown_connections 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c1009f650 0x7f9c1006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c1009e930 0x7f9c1006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 --2- 172.21.15.89:0/1897758456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c1009f240 0x7f9c1006bdc0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 >> 172.21.15.89:0/1897758456 conn(0x7f9c100a4210 msgr2=0x7f9c101b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 shutdown_connections 2023-09-30T22:20:30.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:30.190+0000 7f9c16d2a700 1 -- 172.21.15.89:0/1897758456 wait complete. 2023-09-30T22:20:30.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:31.073 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:31.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:31.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:31.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:31.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.216+0000 7fa8ca187700 1 Processor -- start 2023-09-30T22:20:31.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.216+0000 7fa8ca187700 1 -- start start 2023-09-30T22:20:31.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8ca187700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8c4130540 0x7fa8c4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8ca187700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c4130e50 0x7fa8c406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8ca187700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40716a0 con 0x7fa8c406fb20 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8ca187700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c409a390 con 0x7fa8c4130540 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8ca187700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4071290 con 0x7fa8c4130e50 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8c37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8c4130540 0x7fa8c4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8c37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8c4130540 0x7fa8c4130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58440/0 (socket says 172.21.15.89:58440) 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8c37fe700 1 -- 172.21.15.89:0/3188321431 learned_addr learned my addr 172.21.15.89:0/3188321431 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8c2ffd700 1 --2- 172.21.15.89:0/3188321431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c4130e50 0x7fa8c406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8c37fe700 1 -- 172.21.15.89:0/3188321431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c4130e50 msgr2=0x7fa8c406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8c37fe700 1 --2- 172.21.15.89:0/3188321431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c4130e50 0x7fa8c406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8c37fe700 1 -- 172.21.15.89:0/3188321431 >> v1:172.21.15.89:6789/0 conn(0x7fa8c406fb20 legacy=0x7fa8c4137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.217+0000 7fa8c37fe700 1 -- 172.21.15.89:0/3188321431 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c4130340 con 0x7fa8c4130540 2023-09-30T22:20:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.218+0000 7fa8c37fe700 1 --2- 172.21.15.89:0/3188321431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 0x7fa8c4130910 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40098f0 tx=0x7fa8b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec961dd8b4ae2ee5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.218+0000 7fa8c27fc700 1 -- 172.21.15.89:0/3188321431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4011830 con 0x7fa8c4130540 2023-09-30T22:20:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.218+0000 7fa8c27fc700 1 -- 172.21.15.89:0/3188321431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4011990 con 0x7fa8c4130540 2023-09-30T22:20:31.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.218+0000 7fa8c27fc700 1 -- 172.21.15.89:0/3188321431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4011d20 con 0x7fa8c4130540 2023-09-30T22:20:31.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.219+0000 7fa8ca187700 1 -- 172.21.15.89:0/3188321431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 msgr2=0x7fa8c4130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.219+0000 7fa8ca187700 1 --2- 172.21.15.89:0/3188321431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 0x7fa8c4130910 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40098f0 tx=0x7fa8b4009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.219+0000 7fa8ca187700 1 -- 172.21.15.89:0/3188321431 shutdown_connections 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.219+0000 7fa8ca187700 1 --2- 172.21.15.89:0/3188321431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 0x7fa8c4130910 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.219+0000 7fa8ca187700 1 --2- 172.21.15.89:0/3188321431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c4130e50 0x7fa8c406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.219+0000 7fa8ca187700 1 -- 172.21.15.89:0/3188321431 >> 172.21.15.89:0/3188321431 conn(0x7fa8c40a2aa0 msgr2=0x7fa8c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.219+0000 7fa8ca187700 1 -- 172.21.15.89:0/3188321431 shutdown_connections 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.219+0000 7fa8ca187700 1 -- 172.21.15.89:0/3188321431 wait complete. 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.220+0000 7fa8ca187700 1 Processor -- start 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.220+0000 7fa8ca187700 1 -- start start 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8ca187700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c406fb20 0x7fa8c413acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8ca187700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 0x7fa8c4137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8ca187700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c4130e50 0x7fa8c4138190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8ca187700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c412dba0 con 0x7fa8c4130540 2023-09-30T22:20:31.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8ca187700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4136ad0 con 0x7fa8c406fb20 2023-09-30T22:20:31.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8ca187700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40a05f0 con 0x7fa8c4130e50 2023-09-30T22:20:31.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c4130e50 0x7fa8c4138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c4130e50 0x7fa8c4138190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55668/0 (socket says 172.21.15.89:55668) 2023-09-30T22:20:31.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c3fff700 1 -- 172.21.15.89:0/99886875 learned_addr learned my addr 172.21.15.89:0/99886875 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:31.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c2ffd700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 0x7fa8c4137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c3fff700 1 -- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c406fb20 msgr2=0x7fa8c413acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c3fff700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c406fb20 0x7fa8c413acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c3fff700 1 -- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 msgr2=0x7fa8c4137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c3fff700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 0x7fa8c4137c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c3fff700 1 -- 172.21.15.89:0/99886875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4009550 con 0x7fa8c4130e50 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.221+0000 7fa8c2ffd700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 0x7fa8c4137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.222+0000 7fa8c3fff700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c4130e50 0x7fa8c4138190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00cc50 tx=0x7fa8bc00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.222+0000 7fa8abfff700 1 -- 172.21.15.89:0/99886875 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00ad80 con 0x7fa8c4130e50 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.222+0000 7fa8abfff700 1 -- 172.21.15.89:0/99886875 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc00ce50 con 0x7fa8c4130e50 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.222+0000 7fa8abfff700 1 -- 172.21.15.89:0/99886875 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc003c40 con 0x7fa8c4130e50 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.222+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c4135dd0 con 0x7fa8c4130e50 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.222+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c40718e0 con 0x7fa8c4130e50 2023-09-30T22:20:31.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.223+0000 7fa8abfff700 1 -- 172.21.15.89:0/99886875 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa8bc0050d0 con 0x7fa8c4130e50 2023-09-30T22:20:31.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.223+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c4074ea0 con 0x7fa8c4130e50 2023-09-30T22:20:31.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.223+0000 7fa8abfff700 1 -- 172.21.15.89:0/99886875 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa8bc03f7c0 con 0x7fa8c4130e50 2023-09-30T22:20:31.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.228+0000 7fa8abfff700 1 -- 172.21.15.89:0/99886875 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa8bc003da0 con 0x7fa8c4130e50 2023-09-30T22:20:31.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.264+0000 7f64e5fe2700 1 Processor -- start 2023-09-30T22:20:31.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.264+0000 7f64e5fe2700 1 -- start start 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64e5fe2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64e5fe2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64e5fe2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64e5fe2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e009f520 con 0x7f64e009d6f0 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64e5fe2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e009f130 con 0x7f64e009e410 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64e5fe2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64e009e930 con 0x7f64e009e000 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64deffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64dffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64deffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55682/0 (socket says 172.21.15.89:55682) 2023-09-30T22:20:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64deffd700 1 -- 172.21.15.89:0/1088443261 learned_addr learned my addr 172.21.15.89:0/1088443261 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.265+0000 7f64df7fe700 1 --2- 172.21.15.89:0/1088443261 >> v2:172.21.15.89:3300/0 conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64deffd700 1 -- 172.21.15.89:0/1088443261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 msgr2=0x7f64e0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64deffd700 1 --2- 172.21.15.89:0/1088443261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64deffd700 1 -- 172.21.15.89:0/1088443261 >> v2:172.21.15.89:3300/0 conn(0x7f64e009d6f0 msgr2=0x7f64e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64deffd700 1 --2- 172.21.15.89:0/1088443261 >> v2:172.21.15.89:3300/0 conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64deffd700 1 -- 172.21.15.89:0/1088443261 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e009b2d0 con 0x7f64e009e000 2023-09-30T22:20:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64df7fe700 1 --2- 172.21.15.89:0/1088443261 >> v2:172.21.15.89:3300/0 conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64deffd700 1 --2- 172.21.15.89:0/1088443261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009e000 0x7f64e0130660 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f64d400dd40 tx=0x7f64d400f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c4b820351a6e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64de7fc700 1 -- 172.21.15.89:0/1088443261 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4012070 con 0x7f64e009e000 2023-09-30T22:20:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64de7fc700 1 -- 172.21.15.89:0/1088443261 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d400de90 con 0x7f64e009e000 2023-09-30T22:20:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.266+0000 7f64de7fc700 1 -- 172.21.15.89:0/1088443261 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d400faf0 con 0x7f64e009e000 2023-09-30T22:20:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 -- 172.21.15.89:0/1088443261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009e000 msgr2=0x7f64e0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 --2- 172.21.15.89:0/1088443261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009e000 0x7f64e0130660 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f64d400dd40 tx=0x7f64d400f330 comp rx=0 tx=0).stop 2023-09-30T22:20:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 -- 172.21.15.89:0/1088443261 shutdown_connections 2023-09-30T22:20:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 --2- 172.21.15.89:0/1088443261 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 --2- 172.21.15.89:0/1088443261 >> v2:172.21.15.89:3300/0 conn(0x7f64e009d6f0 0x7f64e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 --2- 172.21.15.89:0/1088443261 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009e000 0x7f64e0130660 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 -- 172.21.15.89:0/1088443261 >> 172.21.15.89:0/1088443261 conn(0x7f64e0066f50 msgr2=0x7f64e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 -- 172.21.15.89:0/1088443261 shutdown_connections 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.267+0000 7f64e5fe2700 1 -- 172.21.15.89:0/1088443261 wait complete. 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.268+0000 7f64e5fe2700 1 Processor -- start 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.268+0000 7f64e5fe2700 1 -- start start 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64e5fe2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009d6f0 0x7f64e011df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64e5fe2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e009e000 0x7f64e011e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64e5fe2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e011e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64e5fe2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e011f3a0 con 0x7f64e009e000 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64e5fe2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0120130 con 0x7f64e009e410 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64e5fe2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01211f0 con 0x7f64e009d6f0 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64df7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009d6f0 0x7f64e011df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64df7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009d6f0 0x7f64e011df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55690/0 (socket says 172.21.15.89:55690) 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64df7fe700 1 -- 172.21.15.89:0/3451562645 learned_addr learned my addr 172.21.15.89:0/3451562645 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64deffd700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e009e000 0x7f64e011e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.269+0000 7f64dffff700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e011e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64deffd700 1 -- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009d6f0 msgr2=0x7f64e011df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64deffd700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009d6f0 0x7f64e011df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64deffd700 1 -- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 msgr2=0x7f64e011e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64deffd700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e011e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64deffd700 1 -- 172.21.15.89:0/3451562645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00094a0 con 0x7f64e009e000 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64dffff700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e011e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64deffd700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e009e000 0x7f64e011e480 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f64e009f6f0 tx=0x7f64d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64c7fff700 1 -- 172.21.15.89:0/3451562645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4010e70 con 0x7f64e009e000 2023-09-30T22:20:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64c7fff700 1 -- 172.21.15.89:0/3451562645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d40036a0 con 0x7f64e009e000 2023-09-30T22:20:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64c7fff700 1 -- 172.21.15.89:0/3451562645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4003a50 con 0x7f64e009e000 2023-09-30T22:20:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d400d9f0 con 0x7f64e009e000 2023-09-30T22:20:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.270+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e00a16b0 con 0x7f64e009e000 2023-09-30T22:20:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.271+0000 7f64c7fff700 1 -- 172.21.15.89:0/3451562645 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64d4003bb0 con 0x7f64e009e000 2023-09-30T22:20:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.271+0000 7f64c7fff700 1 -- 172.21.15.89:0/3451562645 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f64d4044b80 con 0x7f64e009e000 2023-09-30T22:20:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.272+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e011ac20 con 0x7f64e009e000 2023-09-30T22:20:31.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.280+0000 7f64c7fff700 1 -- 172.21.15.89:0/3451562645 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64d4044e60 con 0x7f64e009e000 2023-09-30T22:20:31.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.323+0000 7f03a42de700 1 Processor -- start 2023-09-30T22:20:31.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.323+0000 7f03a42de700 1 -- start start 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a42de700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c06a770 0x7f039c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a42de700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f039c13d1f0 0x7f039c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a42de700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c09a810 con 0x7f039c09dc30 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a42de700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c06ab40 con 0x7f039c13d1f0 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a42de700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f039c09ebc0 con 0x7f039c06a770 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a207a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c06a770 0x7f039c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a207a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c06a770 0x7f039c09d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55696/0 (socket says 172.21.15.89:55696) 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a207a700 1 -- 172.21.15.89:0/2397387181 learned_addr learned my addr 172.21.15.89:0/2397387181 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a287b700 1 --2- 172.21.15.89:0/2397387181 >> v2:172.21.15.89:3300/0 conn(0x7f039c13d1f0 0x7f039c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a287b700 1 -- 172.21.15.89:0/2397387181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c06a770 msgr2=0x7f039c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a287b700 1 --2- 172.21.15.89:0/2397387181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c06a770 0x7f039c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a287b700 1 -- 172.21.15.89:0/2397387181 >> v1:172.21.15.89:6789/0 conn(0x7f039c09dc30 legacy=0x7f039c09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.324+0000 7f03a287b700 1 -- 172.21.15.89:0/2397387181 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c09e920 con 0x7f039c13d1f0 2023-09-30T22:20:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.325+0000 7f03a287b700 1 --2- 172.21.15.89:0/2397387181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c13f5d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0394009dc0 tx=0x7f039400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32f1785c4a8d1aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.325+0000 7f03a1078700 1 -- 172.21.15.89:0/2397387181 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039400e070 con 0x7f039c13d1f0 2023-09-30T22:20:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.325+0000 7f03a1078700 1 -- 172.21.15.89:0/2397387181 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039400bae0 con 0x7f039c13d1f0 2023-09-30T22:20:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.325+0000 7f03a1078700 1 -- 172.21.15.89:0/2397387181 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039400be70 con 0x7f039c13d1f0 2023-09-30T22:20:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.325+0000 7f03a42de700 1 -- 172.21.15.89:0/2397387181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 msgr2=0x7f039c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.325+0000 7f03a42de700 1 --2- 172.21.15.89:0/2397387181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c13f5d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0394009dc0 tx=0x7f039400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.326+0000 7f03a42de700 1 -- 172.21.15.89:0/2397387181 shutdown_connections 2023-09-30T22:20:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.326+0000 7f03a42de700 1 --2- 172.21.15.89:0/2397387181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c13f5d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.326+0000 7f03a42de700 1 --2- 172.21.15.89:0/2397387181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c06a770 0x7f039c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.326+0000 7f03a42de700 1 -- 172.21.15.89:0/2397387181 >> 172.21.15.89:0/2397387181 conn(0x7f039c066f50 msgr2=0x7f039c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.326+0000 7f03a42de700 1 -- 172.21.15.89:0/2397387181 shutdown_connections 2023-09-30T22:20:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.326+0000 7f03a42de700 1 -- 172.21.15.89:0/2397387181 wait complete. 2023-09-30T22:20:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.327+0000 7f03a42de700 1 Processor -- start 2023-09-30T22:20:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.327+0000 7f03a42de700 1 -- start start 2023-09-30T22:20:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a42de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c06a770 0x7f039c1260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a42de700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09dc30 0x7f039c1265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a42de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a42de700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c128710 con 0x7f039c13d1f0 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a42de700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1288f0 con 0x7f039c06a770 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a42de700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f039c11d0c0 con 0x7f039c09dc30 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c126b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58476/0 (socket says 172.21.15.89:58476) 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 -- 172.21.15.89:0/1701055113 learned_addr learned my addr 172.21.15.89:0/1701055113 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a1879700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09dc30 0x7f039c1265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 -- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09dc30 msgr2=0x7f039c1265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 --2- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09dc30 0x7f039c1265e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 -- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c06a770 msgr2=0x7f039c1260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 --2- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c06a770 0x7f039c1260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 -- 172.21.15.89:0/1701055113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394009a70 con 0x7f039c13d1f0 2023-09-30T22:20:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.328+0000 7f03a287b700 1 --2- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c126b20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f039c09eff0 tx=0x7f0394004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:31.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.329+0000 7f0392ffd700 1 -- 172.21.15.89:0/1701055113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394003a20 con 0x7f039c13d1f0 2023-09-30T22:20:31.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.329+0000 7f0392ffd700 1 -- 172.21.15.89:0/1701055113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394020070 con 0x7f039c13d1f0 2023-09-30T22:20:31.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.329+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c09bf10 con 0x7f039c13d1f0 2023-09-30T22:20:31.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.329+0000 7f0392ffd700 1 -- 172.21.15.89:0/1701055113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394003de0 con 0x7f039c13d1f0 2023-09-30T22:20:31.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.329+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c1273d0 con 0x7f039c13d1f0 2023-09-30T22:20:31.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.330+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c074ea0 con 0x7f039c13d1f0 2023-09-30T22:20:31.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.330+0000 7f0392ffd700 1 -- 172.21.15.89:0/1701055113 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0394004030 con 0x7f039c13d1f0 2023-09-30T22:20:31.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.331+0000 7f0392ffd700 1 -- 172.21.15.89:0/1701055113 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0394022070 con 0x7f039c13d1f0 2023-09-30T22:20:31.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.336+0000 7f0392ffd700 1 -- 172.21.15.89:0/1701055113 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03940042e0 con 0x7f039c13d1f0 2023-09-30T22:20:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.353+0000 7fa779ed1700 1 Processor -- start 2023-09-30T22:20:31.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa779ed1700 1 -- start start 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa779ed1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa77409d6f0 0x7fa77409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa779ed1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e000 0x7fa7741339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa779ed1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409e410 0x7fa774133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa779ed1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77406b460 con 0x7fa77409d6f0 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa779ed1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77409b120 con 0x7fa77409e000 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa779ed1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa77409a7e0 con 0x7fa77409e410 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa7737fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa77409d6f0 0x7fa77409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa7737fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa77409d6f0 0x7fa77409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58482/0 (socket says 172.21.15.89:58482) 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa7737fe700 1 -- 172.21.15.89:0/4072537265 learned_addr learned my addr 172.21.15.89:0/4072537265 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:31.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.354+0000 7fa772ffd700 1 --2- 172.21.15.89:0/4072537265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e000 0x7fa7741339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa772ffd700 1 -- 172.21.15.89:0/4072537265 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409e410 msgr2=0x7fa774133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa772ffd700 1 --2- 172.21.15.89:0/4072537265 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409e410 0x7fa774133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa772ffd700 1 -- 172.21.15.89:0/4072537265 >> v2:172.21.15.89:3300/0 conn(0x7fa77409d6f0 msgr2=0x7fa77409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa772ffd700 1 --2- 172.21.15.89:0/4072537265 >> v2:172.21.15.89:3300/0 conn(0x7fa77409d6f0 0x7fa77409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa772ffd700 1 -- 172.21.15.89:0/4072537265 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77409a500 con 0x7fa77409e000 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa7737fe700 1 --2- 172.21.15.89:0/4072537265 >> v2:172.21.15.89:3300/0 conn(0x7fa77409d6f0 0x7fa77409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa772ffd700 1 --2- 172.21.15.89:0/4072537265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e000 0x7fa7741339f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa768009870 tx=0x7fa768009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea9acf5e08e0cdf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa7727fc700 1 -- 172.21.15.89:0/4072537265 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768011830 con 0x7fa77409e000 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa7727fc700 1 -- 172.21.15.89:0/4072537265 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768011990 con 0x7fa77409e000 2023-09-30T22:20:31.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.355+0000 7fa7727fc700 1 -- 172.21.15.89:0/4072537265 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768011d20 con 0x7fa77409e000 2023-09-30T22:20:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 -- 172.21.15.89:0/4072537265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e000 msgr2=0x7fa7741339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 --2- 172.21.15.89:0/4072537265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e000 0x7fa7741339f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa768009870 tx=0x7fa768009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 -- 172.21.15.89:0/4072537265 shutdown_connections 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 --2- 172.21.15.89:0/4072537265 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409e410 0x7fa774133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 --2- 172.21.15.89:0/4072537265 >> v2:172.21.15.89:3300/0 conn(0x7fa77409d6f0 0x7fa77409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 --2- 172.21.15.89:0/4072537265 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e000 0x7fa7741339f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 -- 172.21.15.89:0/4072537265 >> 172.21.15.89:0/4072537265 conn(0x7fa774066f50 msgr2=0x7fa77409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 -- 172.21.15.89:0/4072537265 shutdown_connections 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.356+0000 7fa779ed1700 1 -- 172.21.15.89:0/4072537265 wait complete. 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.357+0000 7fa779ed1700 1 Processor -- start 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.357+0000 7fa779ed1700 1 -- start start 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.357+0000 7fa779ed1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409d6f0 0x7fa7741a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.357+0000 7fa779ed1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa77409e000 0x7fa7741a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.357+0000 7fa779ed1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e410 0x7fa7741a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.357+0000 7fa779ed1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740a2090 con 0x7fa77409e000 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.357+0000 7fa779ed1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7741a8420 con 0x7fa77409e410 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.357+0000 7fa779ed1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7741a86e0 con 0x7fa77409d6f0 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e410 0x7fa7741a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e410 0x7fa7741a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59386/0 (socket says 172.21.15.89:59386) 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 -- 172.21.15.89:0/2333418345 learned_addr learned my addr 172.21.15.89:0/2333418345 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa7737fe700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409d6f0 0x7fa7741a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa772ffd700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa77409e000 0x7fa7741a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 -- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409d6f0 msgr2=0x7fa7741a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409d6f0 0x7fa7741a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 -- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa77409e000 msgr2=0x7fa7741a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa77409e000 0x7fa7741a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 -- 172.21.15.89:0/2333418345 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764009550 con 0x7fa77409e410 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa773fff700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e410 0x7fa7741a59f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00b9a0 tx=0x7fa76c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa772ffd700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa77409e000 0x7fa7741a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa75bfff700 1 -- 172.21.15.89:0/2333418345 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00c930 con 0x7fa77409e410 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7680094a0 con 0x7fa77409e410 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7741a89c0 con 0x7fa77409e410 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa75bfff700 1 -- 172.21.15.89:0/2333418345 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c00cab0 con 0x7fa77409e410 2023-09-30T22:20:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.358+0000 7fa75bfff700 1 -- 172.21.15.89:0/2333418345 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00ce60 con 0x7fa77409e410 2023-09-30T22:20:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.359+0000 7fa75bfff700 1 -- 172.21.15.89:0/2333418345 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa76c01d3b0 con 0x7fa77409e410 2023-09-30T22:20:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.360+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7540052e0 con 0x7fa77409e410 2023-09-30T22:20:31.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.360+0000 7fa75bfff700 1 -- 172.21.15.89:0/2333418345 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa76c006e40 con 0x7fa77409e410 2023-09-30T22:20:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.364+0000 7fa75bfff700 1 -- 172.21.15.89:0/2333418345 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa76c007370 con 0x7fa77409e410 2023-09-30T22:20:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.474+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa8c406b9e0 con 0x7fa8c4130e50 2023-09-30T22:20:31.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.474+0000 7fa8abfff700 1 -- 172.21.15.89:0/99886875 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa8bc019300 con 0x7fa8c4130e50 2023-09-30T22:20:31.475 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c4130e50 msgr2=0x7fa8c4138190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c4130e50 0x7fa8c4138190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00cc50 tx=0x7fa8bc00a340 comp rx=0 tx=0).stop 2023-09-30T22:20:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 shutdown_connections 2023-09-30T22:20:31.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8c4130e50 0x7fa8c4138190 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8c406fb20 0x7fa8c413acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 --2- 172.21.15.89:0/99886875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8c4130540 0x7fa8c4137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 >> 172.21.15.89:0/99886875 conn(0x7fa8c40a2aa0 msgr2=0x7fa8c406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 shutdown_connections 2023-09-30T22:20:31.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.476+0000 7fa8ca187700 1 -- 172.21.15.89:0/99886875 wait complete. 2023-09-30T22:20:31.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:31.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.498+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f64e0074ea0 con 0x7f64e009e000 2023-09-30T22:20:31.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.499+0000 7f64c7fff700 1 -- 172.21.15.89:0/3451562645 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64d400b450 con 0x7f64e009e000 2023-09-30T22:20:31.499 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:31.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e009e000 msgr2=0x7f64e011e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e009e000 0x7f64e011e480 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f64e009f6f0 tx=0x7f64d4004750 comp rx=0 tx=0).stop 2023-09-30T22:20:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 shutdown_connections 2023-09-30T22:20:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64e009e410 0x7f64e011e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64e009d6f0 0x7f64e011df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 --2- 172.21.15.89:0/3451562645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e009e000 0x7f64e011e480 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 >> 172.21.15.89:0/3451562645 conn(0x7f64e0066f50 msgr2=0x7f64e0130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 shutdown_connections 2023-09-30T22:20:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.501+0000 7f64e5fe2700 1 -- 172.21.15.89:0/3451562645 wait complete. 2023-09-30T22:20:31.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:31.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.536+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f039c09a3d0 con 0x7f039c13d1f0 2023-09-30T22:20:31.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.536+0000 7f0392ffd700 1 -- 172.21.15.89:0/1701055113 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f039401b400 con 0x7f039c13d1f0 2023-09-30T22:20:31.537 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 msgr2=0x7f039c126b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 --2- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c126b20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f039c09eff0 tx=0x7f0394004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 shutdown_connections 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 --2- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f039c13d1f0 0x7f039c126b20 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 --2- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f039c06a770 0x7f039c1260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 --2- 172.21.15.89:0/1701055113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f039c09dc30 0x7f039c1265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 >> 172.21.15.89:0/1701055113 conn(0x7f039c066f50 msgr2=0x7f039c13e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 shutdown_connections 2023-09-30T22:20:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.538+0000 7f03a42de700 1 -- 172.21.15.89:0/1701055113 wait complete. 2023-09-30T22:20:31.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:31.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.601+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa754005590 con 0x7fa77409e410 2023-09-30T22:20:31.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.601+0000 7fa75bfff700 1 -- 172.21.15.89:0/2333418345 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa76c019070 con 0x7fa77409e410 2023-09-30T22:20:31.602 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e410 msgr2=0x7fa7741a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e410 0x7fa7741a59f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00b9a0 tx=0x7fa76c00bcb0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 shutdown_connections 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa77409e410 0x7fa7741a59f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa77409d6f0 0x7fa7741a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 --2- 172.21.15.89:0/2333418345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa77409e000 0x7fa7741a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 >> 172.21.15.89:0/2333418345 conn(0x7fa774066f50 msgr2=0x7fa77406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 shutdown_connections 2023-09-30T22:20:31.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:31.603+0000 7fa779ed1700 1 -- 172.21.15.89:0/2333418345 wait complete. 2023-09-30T22:20:31.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:32.494 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:32.514 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:32.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:32.617 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.636+0000 7f31474f0700 1 Processor -- start 2023-09-30T22:20:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.636+0000 7f31474f0700 1 -- start start 2023-09-30T22:20:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f31474f0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3140131630 0x7f3140131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f31474f0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3140131f40 0x7f314006e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f31474f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f314006e9d0 0x7f314006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f31474f0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3140071a00 con 0x7f3140131f40 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f31474f0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3140131430 con 0x7f314006e9d0 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f31474f0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f314009a510 con 0x7f3140131630 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f314528c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3140131630 0x7f3140131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f314528c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3140131630 0x7f3140131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55724/0 (socket says 172.21.15.89:55724) 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f314528c700 1 -- 172.21.15.89:0/1179890319 learned_addr learned my addr 172.21.15.89:0/1179890319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f3144a8b700 1 --2- 172.21.15.89:0/1179890319 >> v2:172.21.15.89:3300/0 conn(0x7f3140131f40 0x7f314006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f3145a8d700 1 --2- 172.21.15.89:0/1179890319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f314006e9d0 0x7f314006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f3145a8d700 1 -- 172.21.15.89:0/1179890319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3140131630 msgr2=0x7f3140131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f3145a8d700 1 --2- 172.21.15.89:0/1179890319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3140131630 0x7f3140131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f3145a8d700 1 -- 172.21.15.89:0/1179890319 >> v2:172.21.15.89:3300/0 conn(0x7f3140131f40 msgr2=0x7f314006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f3145a8d700 1 --2- 172.21.15.89:0/1179890319 >> v2:172.21.15.89:3300/0 conn(0x7f3140131f40 0x7f314006e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f3145a8d700 1 -- 172.21.15.89:0/1179890319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31400721c0 con 0x7f314006e9d0 2023-09-30T22:20:32.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.637+0000 7f3144a8b700 1 --2- 172.21.15.89:0/1179890319 >> v2:172.21.15.89:3300/0 conn(0x7f3140131f40 0x7f314006e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:32.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.638+0000 7f3145a8d700 1 --2- 172.21.15.89:0/1179890319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f314006e9d0 0x7f314006ee70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3138009d90 tx=0x7f313800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8d51c567b57758 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.638+0000 7f3137fff700 1 -- 172.21.15.89:0/1179890319 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800e070 con 0x7f314006e9d0 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.638+0000 7f3137fff700 1 -- 172.21.15.89:0/1179890319 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313800bb20 con 0x7f314006e9d0 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.638+0000 7f3137fff700 1 -- 172.21.15.89:0/1179890319 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138012430 con 0x7f314006e9d0 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 -- 172.21.15.89:0/1179890319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f314006e9d0 msgr2=0x7f314006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 --2- 172.21.15.89:0/1179890319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f314006e9d0 0x7f314006ee70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3138009d90 tx=0x7f313800b330 comp rx=0 tx=0).stop 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 -- 172.21.15.89:0/1179890319 shutdown_connections 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 --2- 172.21.15.89:0/1179890319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f314006e9d0 0x7f314006ee70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 --2- 172.21.15.89:0/1179890319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3140131630 0x7f3140131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 --2- 172.21.15.89:0/1179890319 >> v2:172.21.15.89:3300/0 conn(0x7f3140131f40 0x7f314006e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 -- 172.21.15.89:0/1179890319 >> 172.21.15.89:0/1179890319 conn(0x7f31400a2aa0 msgr2=0x7f31400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 -- 172.21.15.89:0/1179890319 shutdown_connections 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.639+0000 7f31474f0700 1 -- 172.21.15.89:0/1179890319 wait complete. 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.640+0000 7f31474f0700 1 Processor -- start 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.640+0000 7f31474f0700 1 -- start start 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f31474f0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f314006e9d0 0x7f3140119600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f31474f0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3140131630 0x7f3140119b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f31474f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3140131f40 0x7f314011a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f31474f0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314011cd60 con 0x7f3140131630 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f31474f0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314009f860 con 0x7f3140131f40 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f31474f0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f314011b240 con 0x7f314006e9d0 2023-09-30T22:20:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3140131f40 0x7f314011a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3140131f40 0x7f314011a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59416/0 (socket says 172.21.15.89:59416) 2023-09-30T22:20:32.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 -- 172.21.15.89:0/2954893636 learned_addr learned my addr 172.21.15.89:0/2954893636 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:32.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 -- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f314006e9d0 msgr2=0x7f3140119600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 --2- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f314006e9d0 0x7f3140119600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 -- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3140131630 msgr2=0x7f3140119b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:32.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 --2- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3140131630 0x7f3140119b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 -- 172.21.15.89:0/2954893636 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3138009a70 con 0x7f3140131f40 2023-09-30T22:20:32.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3145a8d700 1 --2- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3140131f40 0x7f314011a080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3138014040 tx=0x7f3138004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:32.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.641+0000 7f3135ffb700 1 -- 172.21.15.89:0/2954893636 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31380048e0 con 0x7f3140131f40 2023-09-30T22:20:32.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.642+0000 7f3135ffb700 1 -- 172.21.15.89:0/2954893636 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3138020070 con 0x7f3140131f40 2023-09-30T22:20:32.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.642+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3140099f20 con 0x7f3140131f40 2023-09-30T22:20:32.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.642+0000 7f3135ffb700 1 -- 172.21.15.89:0/2954893636 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31380276c0 con 0x7f3140131f40 2023-09-30T22:20:32.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.642+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314011d040 con 0x7f3140131f40 2023-09-30T22:20:32.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.643+0000 7f3135ffb700 1 -- 172.21.15.89:0/2954893636 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3138012430 con 0x7f3140131f40 2023-09-30T22:20:32.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.643+0000 7f3135ffb700 1 -- 172.21.15.89:0/2954893636 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3138022070 con 0x7f3140131f40 2023-09-30T22:20:32.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.643+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3140074ea0 con 0x7f3140131f40 2023-09-30T22:20:32.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.648+0000 7f3135ffb700 1 -- 172.21.15.89:0/2954893636 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31380269d0 con 0x7f3140131f40 2023-09-30T22:20:32.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.674+0000 7f918f5a0700 1 Processor -- start 2023-09-30T22:20:32.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.674+0000 7f918f5a0700 1 -- start start 2023-09-30T22:20:32.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918f5a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91881338a0 0x7f9188133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918f5a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918f5a0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918806ef30 con 0x7f918809e760 2023-09-30T22:20:32.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918f5a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918806fef0 con 0x7f918809de50 2023-09-30T22:20:32.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918f5a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91881334b0 con 0x7f91881338a0 2023-09-30T22:20:32.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918cb3b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918d33c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91881338a0 0x7f9188133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918cb3b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59428/0 (socket says 172.21.15.89:59428) 2023-09-30T22:20:32.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918cb3b700 1 -- 172.21.15.89:0/1231125244 learned_addr learned my addr 172.21.15.89:0/1231125244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:32.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.675+0000 7f918db3d700 1 --1- 172.21.15.89:0/1231125244 >> v1:172.21.15.89:6789/0 conn(0x7f918809e760 0x7f918813a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55484/0 (socket says 172.21.15.89:55484) 2023-09-30T22:20:32.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.676+0000 7f918d33c700 1 -- 172.21.15.89:0/1231125244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 msgr2=0x7f918809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.676+0000 7f918d33c700 1 --2- 172.21.15.89:0/1231125244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.676+0000 7f918d33c700 1 -- 172.21.15.89:0/1231125244 >> v1:172.21.15.89:6789/0 conn(0x7f918809e760 legacy=0x7f918813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.676+0000 7f918d33c700 1 -- 172.21.15.89:0/1231125244 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918806fc30 con 0x7f91881338a0 2023-09-30T22:20:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.676+0000 7f918d33c700 1 --2- 172.21.15.89:0/1231125244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91881338a0 0x7f9188133c70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f91840098a0 tx=0x7f9184009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9b4f0dbe5b703b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:32.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.676+0000 7f917ffff700 1 -- 172.21.15.89:0/1231125244 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184011830 con 0x7f91881338a0 2023-09-30T22:20:32.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.676+0000 7f917ffff700 1 -- 172.21.15.89:0/1231125244 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9184011990 con 0x7f91881338a0 2023-09-30T22:20:32.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.676+0000 7f917ffff700 1 -- 172.21.15.89:0/1231125244 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184011d20 con 0x7f91881338a0 2023-09-30T22:20:32.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.677+0000 7f918f5a0700 1 -- 172.21.15.89:0/1231125244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91881338a0 msgr2=0x7f9188133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.677+0000 7f918f5a0700 1 --2- 172.21.15.89:0/1231125244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91881338a0 0x7f9188133c70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f91840098a0 tx=0x7f9184009bb0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.677+0000 7f918f5a0700 1 -- 172.21.15.89:0/1231125244 shutdown_connections 2023-09-30T22:20:32.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.677+0000 7f918f5a0700 1 --2- 172.21.15.89:0/1231125244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91881338a0 0x7f9188133c70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.677+0000 7f918f5a0700 1 --2- 172.21.15.89:0/1231125244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.677+0000 7f918f5a0700 1 -- 172.21.15.89:0/1231125244 >> 172.21.15.89:0/1231125244 conn(0x7f91880a2aa0 msgr2=0x7f91880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.677+0000 7f918f5a0700 1 -- 172.21.15.89:0/1231125244 shutdown_connections 2023-09-30T22:20:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.677+0000 7f918f5a0700 1 -- 172.21.15.89:0/1231125244 wait complete. 2023-09-30T22:20:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.678+0000 7f918f5a0700 1 Processor -- start 2023-09-30T22:20:32.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.678+0000 7f918f5a0700 1 -- start start 2023-09-30T22:20:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918f5a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918811ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918f5a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918809e760 0x7f918811f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918f5a0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91881338a0 0x7f918811f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918f5a0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9188120730 con 0x7f91881338a0 2023-09-30T22:20:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918f5a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91881222c0 con 0x7f918809de50 2023-09-30T22:20:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918f5a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91880a10d0 con 0x7f918809e760 2023-09-30T22:20:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918d33c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918811ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918d33c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918811ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59430/0 (socket says 172.21.15.89:59430) 2023-09-30T22:20:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918d33c700 1 -- 172.21.15.89:0/2884930297 learned_addr learned my addr 172.21.15.89:0/2884930297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918cb3b700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918809e760 0x7f918811f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.679+0000 7f918db3d700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91881338a0 0x7f918811f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918d33c700 1 -- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918809e760 msgr2=0x7f918811f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918d33c700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918809e760 0x7f918811f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918d33c700 1 -- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91881338a0 msgr2=0x7f918811f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918d33c700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91881338a0 0x7f918811f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918d33c700 1 -- 172.21.15.89:0/2884930297 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91840094a0 con 0x7f918809de50 2023-09-30T22:20:32.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918cb3b700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918809e760 0x7f918811f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:32.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918d33c700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918811ead0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f91840090b0 tx=0x7f9184004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:32.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f917dffb700 1 -- 172.21.15.89:0/2884930297 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184004550 con 0x7f918809de50 2023-09-30T22:20:32.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f917dffb700 1 -- 172.21.15.89:0/2884930297 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91840036a0 con 0x7f918809de50 2023-09-30T22:20:32.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f917dffb700 1 -- 172.21.15.89:0/2884930297 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184003a10 con 0x7f918809de50 2023-09-30T22:20:32.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918806f640 con 0x7f918809de50 2023-09-30T22:20:32.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.680+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9188122d30 con 0x7f918809de50 2023-09-30T22:20:32.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.681+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9188122100 con 0x7f918809de50 2023-09-30T22:20:32.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.681+0000 7f917dffb700 1 -- 172.21.15.89:0/2884930297 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9184003b70 con 0x7f918809de50 2023-09-30T22:20:32.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.682+0000 7f917dffb700 1 -- 172.21.15.89:0/2884930297 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9184040b20 con 0x7f918809de50 2023-09-30T22:20:32.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.690+0000 7f917dffb700 1 -- 172.21.15.89:0/2884930297 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9184010bb0 con 0x7f918809de50 2023-09-30T22:20:32.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.721+0000 7fb44f557700 1 Processor -- start 2023-09-30T22:20:32.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.721+0000 7fb44f557700 1 -- start start 2023-09-30T22:20:32.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.721+0000 7fb44f557700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.721+0000 7fb44f557700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb44806b080 0x7fb448131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.721+0000 7fb44f557700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb448131b40 0x7fb448131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.721+0000 7fb44f557700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44809ece0 con 0x7fb44806b080 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.721+0000 7fb44f557700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44809e760 con 0x7fb44806a770 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.721+0000 7fb44f557700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb44809e040 con 0x7fb448131b40 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44d2f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb448131b40 0x7fb448131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb448131b40 0x7fb448131fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55756/0 (socket says 172.21.15.89:55756) 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 -- 172.21.15.89:0/1341310109 learned_addr learned my addr 172.21.15.89:0/1341310109 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44d2f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59442/0 (socket says 172.21.15.89:59442) 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44caf2700 1 --2- 172.21.15.89:0/1341310109 >> v2:172.21.15.89:3300/0 conn(0x7fb44806b080 0x7fb448131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 -- 172.21.15.89:0/1341310109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 msgr2=0x7fb44806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:32.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 --2- 172.21.15.89:0/1341310109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 -- 172.21.15.89:0/1341310109 >> v2:172.21.15.89:3300/0 conn(0x7fb44806b080 msgr2=0x7fb448131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 --2- 172.21.15.89:0/1341310109 >> v2:172.21.15.89:3300/0 conn(0x7fb44806b080 0x7fb448131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 -- 172.21.15.89:0/1341310109 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44809f630 con 0x7fb448131b40 2023-09-30T22:20:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44caf2700 1 --2- 172.21.15.89:0/1341310109 >> v2:172.21.15.89:3300/0 conn(0x7fb44806b080 0x7fb448131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb44daf4700 1 --2- 172.21.15.89:0/1341310109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb448131b40 0x7fb448131fc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb44000f370 tx=0x7fb44000f680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f4fc0801c91317b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.722+0000 7fb43ffff700 1 -- 172.21.15.89:0/1341310109 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440012070 con 0x7fb448131b40 2023-09-30T22:20:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb43ffff700 1 -- 172.21.15.89:0/1341310109 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44000d0d0 con 0x7fb448131b40 2023-09-30T22:20:32.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb43ffff700 1 -- 172.21.15.89:0/1341310109 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44000d480 con 0x7fb448131b40 2023-09-30T22:20:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 -- 172.21.15.89:0/1341310109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb448131b40 msgr2=0x7fb448131fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 --2- 172.21.15.89:0/1341310109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb448131b40 0x7fb448131fc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb44000f370 tx=0x7fb44000f680 comp rx=0 tx=0).stop 2023-09-30T22:20:32.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 -- 172.21.15.89:0/1341310109 shutdown_connections 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 --2- 172.21.15.89:0/1341310109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb448131b40 0x7fb448131fc0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 --2- 172.21.15.89:0/1341310109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 --2- 172.21.15.89:0/1341310109 >> v2:172.21.15.89:3300/0 conn(0x7fb44806b080 0x7fb448131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 -- 172.21.15.89:0/1341310109 >> 172.21.15.89:0/1341310109 conn(0x7fb448066f50 msgr2=0x7fb44809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 -- 172.21.15.89:0/1341310109 shutdown_connections 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.723+0000 7fb44f557700 1 -- 172.21.15.89:0/1341310109 wait complete. 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.725+0000 7fb44f557700 1 Processor -- start 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.725+0000 7fb44f557700 1 -- start start 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.725+0000 7fb44f557700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44811afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.725+0000 7fb44f557700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb44806b080 0x7fb44811b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.725+0000 7fb44f557700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb448131b40 0x7fb44811ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.725+0000 7fb44f557700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44811c480 con 0x7fb448131b40 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.725+0000 7fb44f557700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44811d1f0 con 0x7fb44806a770 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.725+0000 7fb44f557700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb44811ead0 con 0x7fb44806b080 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44811afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44811afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59446/0 (socket says 172.21.15.89:59446) 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 -- 172.21.15.89:0/3921743781 learned_addr learned my addr 172.21.15.89:0/3921743781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44caf2700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb44806b080 0x7fb44811b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 -- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb44806b080 msgr2=0x7fb44811b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb44806b080 0x7fb44811b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44daf4700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb448131b40 0x7fb44811ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 -- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb448131b40 msgr2=0x7fb44811ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb448131b40 0x7fb44811ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 -- 172.21.15.89:0/3921743781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44000f050 con 0x7fb44806a770 2023-09-30T22:20:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44daf4700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb448131b40 0x7fb44811ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44d2f3700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44811afb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e9e0 tx=0x7fb44400ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb43dffb700 1 -- 172.21.15.89:0/3921743781 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444009710 con 0x7fb44806a770 2023-09-30T22:20:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb43dffb700 1 -- 172.21.15.89:0/3921743781 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44400c850 con 0x7fb44806a770 2023-09-30T22:20:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.726+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44811e260 con 0x7fb44806a770 2023-09-30T22:20:32.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.727+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44811de10 con 0x7fb44806a770 2023-09-30T22:20:32.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.728+0000 7fb43dffb700 1 -- 172.21.15.89:0/3921743781 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44400cbc0 con 0x7fb44806a770 2023-09-30T22:20:32.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.728+0000 7fb43dffb700 1 -- 172.21.15.89:0/3921743781 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb44400cd60 con 0x7fb44806a770 2023-09-30T22:20:32.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.728+0000 7fb43dffb700 1 -- 172.21.15.89:0/3921743781 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb44403f8b0 con 0x7fb44806a770 2023-09-30T22:20:32.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.728+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb448074ea0 con 0x7fb44806a770 2023-09-30T22:20:32.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.733+0000 7fb43dffb700 1 -- 172.21.15.89:0/3921743781 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb444021020 con 0x7fb44806a770 2023-09-30T22:20:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.772+0000 7fa767d93700 1 Processor -- start 2023-09-30T22:20:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.773+0000 7fa767d93700 1 -- start start 2023-09-30T22:20:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.773+0000 7fa767d93700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa76012cc20 0x7fa76012f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.773+0000 7fa767d93700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012c2f0 0x7fa76012fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.773+0000 7fa767d93700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 0x7fa7601303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.773+0000 7fa767d93700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76009a510 con 0x7fa76012cc20 2023-09-30T22:20:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.773+0000 7fa767d93700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76012c7d0 con 0x7fa76012cff0 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.773+0000 7fa767d93700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa76009c190 con 0x7fa76012c2f0 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012c2f0 0x7fa76012fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012c2f0 0x7fa76012fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55776/0 (socket says 172.21.15.89:55776) 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 -- 172.21.15.89:0/1529534925 learned_addr learned my addr 172.21.15.89:0/1529534925 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa766330700 1 --2- 172.21.15.89:0/1529534925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 0x7fa7601303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa765b2f700 1 --2- 172.21.15.89:0/1529534925 >> v2:172.21.15.89:3300/0 conn(0x7fa76012cc20 0x7fa76012f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 -- 172.21.15.89:0/1529534925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 msgr2=0x7fa7601303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 --2- 172.21.15.89:0/1529534925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 0x7fa7601303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 -- 172.21.15.89:0/1529534925 >> v2:172.21.15.89:3300/0 conn(0x7fa76012cc20 msgr2=0x7fa76012f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 --2- 172.21.15.89:0/1529534925 >> v2:172.21.15.89:3300/0 conn(0x7fa76012cc20 0x7fa76012f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 -- 172.21.15.89:0/1529534925 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76009b540 con 0x7fa76012c2f0 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa765b2f700 1 --2- 172.21.15.89:0/1529534925 >> v2:172.21.15.89:3300/0 conn(0x7fa76012cc20 0x7fa76012f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa76532e700 1 --2- 172.21.15.89:0/1529534925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012c2f0 0x7fa76012fe70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa7500098f0 tx=0x7fa750009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a154162029a377ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa764b2d700 1 -- 172.21.15.89:0/1529534925 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750011830 con 0x7fa76012c2f0 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa764b2d700 1 -- 172.21.15.89:0/1529534925 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa750011990 con 0x7fa76012c2f0 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.774+0000 7fa764b2d700 1 -- 172.21.15.89:0/1529534925 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750011d20 con 0x7fa76012c2f0 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 -- 172.21.15.89:0/1529534925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012c2f0 msgr2=0x7fa76012fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 --2- 172.21.15.89:0/1529534925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012c2f0 0x7fa76012fe70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa7500098f0 tx=0x7fa750009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 -- 172.21.15.89:0/1529534925 shutdown_connections 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 --2- 172.21.15.89:0/1529534925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 0x7fa7601303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 --2- 172.21.15.89:0/1529534925 >> v2:172.21.15.89:3300/0 conn(0x7fa76012cc20 0x7fa76012f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 --2- 172.21.15.89:0/1529534925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012c2f0 0x7fa76012fe70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 -- 172.21.15.89:0/1529534925 >> 172.21.15.89:0/1529534925 conn(0x7fa76009f470 msgr2=0x7fa7600a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 -- 172.21.15.89:0/1529534925 shutdown_connections 2023-09-30T22:20:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.775+0000 7fa767d93700 1 -- 172.21.15.89:0/1529534925 wait complete. 2023-09-30T22:20:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.776+0000 7fa767d93700 1 Processor -- start 2023-09-30T22:20:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.776+0000 7fa767d93700 1 -- start start 2023-09-30T22:20:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa767d93700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa76012c2f0 0x7fa7601c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa767d93700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012cc20 0x7fa7601c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa767d93700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 0x7fa7601c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa767d93700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601292e0 con 0x7fa76012c2f0 2023-09-30T22:20:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa767d93700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601306d0 con 0x7fa76012cff0 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa767d93700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601b9650 con 0x7fa76012cc20 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012cc20 0x7fa7601c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012cc20 0x7fa7601c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55782/0 (socket says 172.21.15.89:55782) 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 -- 172.21.15.89:0/600327013 learned_addr learned my addr 172.21.15.89:0/600327013 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa765b2f700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa76012c2f0 0x7fa7601c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa766330700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 0x7fa7601c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 -- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 msgr2=0x7fa7601c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 0x7fa7601c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 -- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa76012c2f0 msgr2=0x7fa7601c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa76012c2f0 0x7fa7601c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 -- 172.21.15.89:0/600327013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c00b740 con 0x7fa76012cc20 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa765b2f700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa76012c2f0 0x7fa7601c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.777+0000 7fa76532e700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012cc20 0x7fa7601c88c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa750000c00 tx=0x7fa750004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.778+0000 7fa7567fc700 1 -- 172.21.15.89:0/600327013 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750004240 con 0x7fa76012cc20 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.778+0000 7fa7567fc700 1 -- 172.21.15.89:0/600327013 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75001f070 con 0x7fa76012cc20 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.778+0000 7fa7567fc700 1 -- 172.21.15.89:0/600327013 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7500045e0 con 0x7fa76012cc20 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.778+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa750009550 con 0x7fa76012cc20 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.778+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7601b6ff0 con 0x7fa76012cc20 2023-09-30T22:20:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.779+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa760129f10 con 0x7fa76012cc20 2023-09-30T22:20:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.779+0000 7fa7567fc700 1 -- 172.21.15.89:0/600327013 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa750021030 con 0x7fa76012cc20 2023-09-30T22:20:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.779+0000 7fa7567fc700 1 -- 172.21.15.89:0/600327013 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa750040de0 con 0x7fa76012cc20 2023-09-30T22:20:32.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.788+0000 7fa7567fc700 1 -- 172.21.15.89:0/600327013 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa75000b7d0 con 0x7fa76012cc20 2023-09-30T22:20:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.860+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f314006b9e0 con 0x7f3140131f40 2023-09-30T22:20:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.860+0000 7f3135ffb700 1 -- 172.21.15.89:0/2954893636 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3138036090 con 0x7f3140131f40 2023-09-30T22:20:32.861 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3140131f40 msgr2=0x7f314011a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 --2- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3140131f40 0x7f314011a080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3138014040 tx=0x7f3138004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 shutdown_connections 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 --2- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3140131f40 0x7f314011a080 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 --2- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f314006e9d0 0x7f3140119600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 --2- 172.21.15.89:0/2954893636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3140131630 0x7f3140119b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 >> 172.21.15.89:0/2954893636 conn(0x7f31400a2aa0 msgr2=0x7f314012e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 shutdown_connections 2023-09-30T22:20:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.864+0000 7f31474f0700 1 -- 172.21.15.89:0/2954893636 wait complete. 2023-09-30T22:20:32.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:32.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.891+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9188121f20 con 0x7f918809de50 2023-09-30T22:20:32.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.891+0000 7f917dffb700 1 -- 172.21.15.89:0/2884930297 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9184015070 con 0x7f918809de50 2023-09-30T22:20:32.893 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:32.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 msgr2=0x7f918811ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918811ead0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f91840090b0 tx=0x7f9184004750 comp rx=0 tx=0).stop 2023-09-30T22:20:32.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 shutdown_connections 2023-09-30T22:20:32.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91881338a0 0x7f918811f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918809de50 0x7f918811ead0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 --2- 172.21.15.89:0/2884930297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918809e760 0x7f918811f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 >> 172.21.15.89:0/2884930297 conn(0x7f91880a2aa0 msgr2=0x7f918809ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:32.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 shutdown_connections 2023-09-30T22:20:32.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.894+0000 7f918f5a0700 1 -- 172.21.15.89:0/2884930297 wait complete. 2023-09-30T22:20:32.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.948+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb448071910 con 0x7fb44806a770 2023-09-30T22:20:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.948+0000 7fb43dffb700 1 -- 172.21.15.89:0/3921743781 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb444017070 con 0x7fb44806a770 2023-09-30T22:20:32.949 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 msgr2=0x7fb44811afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44811afb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e9e0 tx=0x7fb44400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 shutdown_connections 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb448131b40 0x7fb44811ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb44806a770 0x7fb44811afb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 --2- 172.21.15.89:0/3921743781 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb44806b080 0x7fb44811b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 >> 172.21.15.89:0/3921743781 conn(0x7fb448066f50 msgr2=0x7fb4480708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 shutdown_connections 2023-09-30T22:20:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.951+0000 7fb44f557700 1 -- 172.21.15.89:0/3921743781 wait complete. 2023-09-30T22:20:32.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:32.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.989+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa76006acf0 con 0x7fa76012cc20 2023-09-30T22:20:32.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.990+0000 7fa7567fc700 1 -- 172.21.15.89:0/600327013 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa76006acf0 con 0x7fa76012cc20 2023-09-30T22:20:32.991 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:32.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012cc20 msgr2=0x7fa7601c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:32.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012cc20 0x7fa7601c88c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa750000c00 tx=0x7fa750004750 comp rx=0 tx=0).stop 2023-09-30T22:20:32.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 shutdown_connections 2023-09-30T22:20:32.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa76012cff0 0x7fa7601c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa76012c2f0 0x7fa7601c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 --2- 172.21.15.89:0/600327013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa76012cc20 0x7fa7601c88c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:32.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 >> 172.21.15.89:0/600327013 conn(0x7fa76009f470 msgr2=0x7fa76009fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:32.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 shutdown_connections 2023-09-30T22:20:32.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:32.992+0000 7fa767d93700 1 -- 172.21.15.89:0/600327013 wait complete. 2023-09-30T22:20:33.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:33.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:33.908 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:33.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:34.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:34.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.025+0000 7f20ad3ea700 1 Processor -- start 2023-09-30T22:20:34.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.025+0000 7f20ad3ea700 1 -- start start 2023-09-30T22:20:34.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.025+0000 7f20ad3ea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a8131630 0x7f20a8131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.025+0000 7f20ad3ea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20a8131f40 0x7f20a806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.025+0000 7f20ad3ea700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a8071a00 con 0x7f20a806e9d0 2023-09-30T22:20:34.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.025+0000 7f20ad3ea700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a8131430 con 0x7f20a8131f40 2023-09-30T22:20:34.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.025+0000 7f20ad3ea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20a809a510 con 0x7f20a8131630 2023-09-30T22:20:34.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a8131630 0x7f20a8131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a8131630 0x7f20a8131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55794/0 (socket says 172.21.15.89:55794) 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a6ffd700 1 -- 172.21.15.89:0/2544319858 learned_addr learned my addr 172.21.15.89:0/2544319858 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a67fc700 1 --2- 172.21.15.89:0/2544319858 >> v2:172.21.15.89:3300/0 conn(0x7f20a8131f40 0x7f20a806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a67fc700 1 -- 172.21.15.89:0/2544319858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a8131630 msgr2=0x7f20a8131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a67fc700 1 --2- 172.21.15.89:0/2544319858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a8131630 0x7f20a8131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a67fc700 1 -- 172.21.15.89:0/2544319858 >> v1:172.21.15.89:6789/0 conn(0x7f20a806e9d0 legacy=0x7f20a8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a67fc700 1 -- 172.21.15.89:0/2544319858 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80721c0 con 0x7f20a8131f40 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a6ffd700 1 --2- 172.21.15.89:0/2544319858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a8131630 0x7f20a8131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.026+0000 7f20a67fc700 1 --2- 172.21.15.89:0/2544319858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20a8131f40 0x7f20a806e490 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f209c00dc60 tx=0x7f209c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69f9c24170ceaa2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.027+0000 7f20a5ffb700 1 -- 172.21.15.89:0/2544319858 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c00b070 con 0x7f20a8131f40 2023-09-30T22:20:34.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.027+0000 7f20a5ffb700 1 -- 172.21.15.89:0/2544319858 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c016c60 con 0x7f20a8131f40 2023-09-30T22:20:34.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.027+0000 7f20a5ffb700 1 -- 172.21.15.89:0/2544319858 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c008620 con 0x7f20a8131f40 2023-09-30T22:20:34.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.027+0000 7f20ad3ea700 1 -- 172.21.15.89:0/2544319858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20a8131f40 msgr2=0x7f20a806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.027+0000 7f20ad3ea700 1 --2- 172.21.15.89:0/2544319858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20a8131f40 0x7f20a806e490 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f209c00dc60 tx=0x7f209c00df70 comp rx=0 tx=0).stop 2023-09-30T22:20:34.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.028+0000 7f20ad3ea700 1 -- 172.21.15.89:0/2544319858 shutdown_connections 2023-09-30T22:20:34.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.028+0000 7f20ad3ea700 1 --2- 172.21.15.89:0/2544319858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a8131630 0x7f20a8131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.028+0000 7f20ad3ea700 1 --2- 172.21.15.89:0/2544319858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20a8131f40 0x7f20a806e490 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.028+0000 7f20ad3ea700 1 -- 172.21.15.89:0/2544319858 >> 172.21.15.89:0/2544319858 conn(0x7f20a80a2aa0 msgr2=0x7f20a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.028+0000 7f20ad3ea700 1 -- 172.21.15.89:0/2544319858 shutdown_connections 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.028+0000 7f20ad3ea700 1 -- 172.21.15.89:0/2544319858 wait complete. 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20ad3ea700 1 Processor -- start 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20ad3ea700 1 -- start start 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20ad3ea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a806e9d0 0x7f20a8119870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20ad3ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20a8131630 0x7f20a8119db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20ad3ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20a8131f40 0x7f20a811a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20ad3ea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a811ad40 con 0x7f20a8131630 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20ad3ea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a811d300 con 0x7f20a8131f40 2023-09-30T22:20:34.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20ad3ea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20a811c640 con 0x7f20a806e9d0 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20a6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a806e9d0 0x7f20a8119870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20a6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a806e9d0 0x7f20a8119870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55798/0 (socket says 172.21.15.89:55798) 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.029+0000 7f20a6ffd700 1 -- 172.21.15.89:0/1099319014 learned_addr learned my addr 172.21.15.89:0/1099319014 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20a77fe700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20a8131f40 0x7f20a811a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20a6ffd700 1 -- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20a8131f40 msgr2=0x7f20a811a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20a6ffd700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20a8131f40 0x7f20a811a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20a6ffd700 1 -- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20a8131630 msgr2=0x7f20a8119db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20a6ffd700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20a8131630 0x7f20a8119db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20a6ffd700 1 -- 172.21.15.89:0/1099319014 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098009550 con 0x7f20a806e9d0 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20a77fe700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20a8131f40 0x7f20a811a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20a6ffd700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a806e9d0 0x7f20a8119870 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f209800ec10 tx=0x7f2098009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f208f7fe700 1 -- 172.21.15.89:0/1099319014 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800c8e0 con 0x7f20a806e9d0 2023-09-30T22:20:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f208f7fe700 1 -- 172.21.15.89:0/1099319014 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209800ca40 con 0x7f20a806e9d0 2023-09-30T22:20:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c00d940 con 0x7f20a806e9d0 2023-09-30T22:20:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f208f7fe700 1 -- 172.21.15.89:0/1099319014 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800cdb0 con 0x7f20a806e9d0 2023-09-30T22:20:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.030+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a811ca90 con 0x7f20a806e9d0 2023-09-30T22:20:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.031+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a804ea50 con 0x7f20a806e9d0 2023-09-30T22:20:34.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.031+0000 7f208f7fe700 1 -- 172.21.15.89:0/1099319014 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2098013470 con 0x7f20a806e9d0 2023-09-30T22:20:34.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.032+0000 7f208f7fe700 1 -- 172.21.15.89:0/1099319014 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f209803f8e0 con 0x7f20a806e9d0 2023-09-30T22:20:34.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.037+0000 7f208f7fe700 1 -- 172.21.15.89:0/1099319014 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20980232f0 con 0x7f20a806e9d0 2023-09-30T22:20:34.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.076+0000 7fd0864a2700 1 Processor -- start 2023-09-30T22:20:34.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.076+0000 7fd0864a2700 1 -- start start 2023-09-30T22:20:34.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.076+0000 7fd0864a2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd0800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.076+0000 7fd0864a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd080133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.076+0000 7fd0864a2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08009edd0 con 0x7fd080133fa0 2023-09-30T22:20:34.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.076+0000 7fd0864a2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08009e1f0 con 0x7fd0800a0a30 2023-09-30T22:20:34.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.076+0000 7fd0864a2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd08009dc50 con 0x7fd0800a1340 2023-09-30T22:20:34.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd0800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd0800a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59480/0 (socket says 172.21.15.89:59480) 2023-09-30T22:20:34.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07ffff700 1 -- 172.21.15.89:0/1211306461 learned_addr learned my addr 172.21.15.89:0/1211306461 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:34.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07f7fe700 1 --2- 172.21.15.89:0/1211306461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd080133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07ffff700 1 -- 172.21.15.89:0/1211306461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 msgr2=0x7fd080133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:34.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07ffff700 1 --2- 172.21.15.89:0/1211306461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd080133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07ffff700 1 -- 172.21.15.89:0/1211306461 >> v1:172.21.15.89:6789/0 conn(0x7fd080133fa0 legacy=0x7fd08013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07ffff700 1 -- 172.21.15.89:0/1211306461 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0800a1fe0 con 0x7fd0800a0a30 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07ffff700 1 --2- 172.21.15.89:0/1211306461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd0800a0e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd070009920 tx=0x7fd070009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29a077aca5a08b5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07effd700 1 -- 172.21.15.89:0/1211306461 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070011830 con 0x7fd0800a0a30 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07effd700 1 -- 172.21.15.89:0/1211306461 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd070011990 con 0x7fd0800a0a30 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.077+0000 7fd07effd700 1 -- 172.21.15.89:0/1211306461 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070011d20 con 0x7fd0800a0a30 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.078+0000 7fd0864a2700 1 -- 172.21.15.89:0/1211306461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 msgr2=0x7fd0800a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.078+0000 7fd0864a2700 1 --2- 172.21.15.89:0/1211306461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd0800a0e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd070009920 tx=0x7fd070009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.078+0000 7fd0864a2700 1 -- 172.21.15.89:0/1211306461 shutdown_connections 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.078+0000 7fd0864a2700 1 --2- 172.21.15.89:0/1211306461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd0800a0e00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.078+0000 7fd0864a2700 1 --2- 172.21.15.89:0/1211306461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd080133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.078+0000 7fd0864a2700 1 -- 172.21.15.89:0/1211306461 >> 172.21.15.89:0/1211306461 conn(0x7fd0800a6550 msgr2=0x7fd0800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:34.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.078+0000 7fd0864a2700 1 -- 172.21.15.89:0/1211306461 shutdown_connections 2023-09-30T22:20:34.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.078+0000 7fd0864a2700 1 -- 172.21.15.89:0/1211306461 wait complete. 2023-09-30T22:20:34.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.080+0000 7fd0864a2700 1 Processor -- start 2023-09-30T22:20:34.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.080+0000 7fd0864a2700 1 -- start start 2023-09-30T22:20:34.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.080+0000 7fd0864a2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd08011e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.080+0000 7fd0864a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd08011ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.080+0000 7fd0864a2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd080133fa0 0x7fd08011f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.080+0000 7fd0864a2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080120520 con 0x7fd080133fa0 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.080+0000 7fd0864a2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800a38a0 con 0x7fd0800a0a30 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.080+0000 7fd0864a2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd080120980 con 0x7fd0800a1340 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd084a3f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd080133fa0 0x7fd08011f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd08011e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd08011ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd084a3f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd080133fa0 0x7fd08011f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58562/0 (socket says 172.21.15.89:58562) 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd08011e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59482/0 (socket says 172.21.15.89:59482) 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd084a3f700 1 -- 172.21.15.89:0/396806085 learned_addr learned my addr 172.21.15.89:0/396806085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07f7fe700 1 -- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 msgr2=0x7fd08011e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07f7fe700 1 --2- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd08011e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07f7fe700 1 -- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd080133fa0 msgr2=0x7fd08011f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07f7fe700 1 --2- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd080133fa0 0x7fd08011f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07f7fe700 1 -- 172.21.15.89:0/396806085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070009550 con 0x7fd0800a1340 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.081+0000 7fd07f7fe700 1 --2- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd08011ed60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e900 tx=0x7fd07400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.082+0000 7fd07cff9700 1 -- 172.21.15.89:0/396806085 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074013680 con 0x7fd0800a1340 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.082+0000 7fd07cff9700 1 -- 172.21.15.89:0/396806085 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0740137e0 con 0x7fd0800a1340 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.082+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0800a25e0 con 0x7fd0800a1340 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.082+0000 7fd07cff9700 1 -- 172.21.15.89:0/396806085 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074013b50 con 0x7fd0800a1340 2023-09-30T22:20:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.082+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd080120b90 con 0x7fd0800a1340 2023-09-30T22:20:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.083+0000 7fd07cff9700 1 -- 172.21.15.89:0/396806085 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd074013cb0 con 0x7fd0800a1340 2023-09-30T22:20:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.083+0000 7fd07cff9700 1 -- 172.21.15.89:0/396806085 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd07403f760 con 0x7fd0800a1340 2023-09-30T22:20:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.083+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd080074ea0 con 0x7fd0800a1340 2023-09-30T22:20:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.092+0000 7fd07cff9700 1 -- 172.21.15.89:0/396806085 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd074021080 con 0x7fd0800a1340 2023-09-30T22:20:34.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f145a4c0700 1 Processor -- start 2023-09-30T22:20:34.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f145a4c0700 1 -- start start 2023-09-30T22:20:34.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f145a4c0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f145406a770 0x7f145409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f145a4c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f145409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f145a4c0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145409a810 con 0x7f145413c1e0 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f145a4c0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145406ab40 con 0x7f145406a770 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f145a4c0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f145409ebc0 con 0x7f145409dc30 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f14537fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f145409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f14537fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f145409e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55822/0 (socket says 172.21.15.89:55822) 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f14537fe700 1 -- 172.21.15.89:0/890991952 learned_addr learned my addr 172.21.15.89:0/890991952 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.104+0000 7f1453fff700 1 --2- 172.21.15.89:0/890991952 >> v2:172.21.15.89:3300/0 conn(0x7f145406a770 0x7f145409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f14537fe700 1 -- 172.21.15.89:0/890991952 >> v2:172.21.15.89:3300/0 conn(0x7f145406a770 msgr2=0x7f145409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f14537fe700 1 --2- 172.21.15.89:0/890991952 >> v2:172.21.15.89:3300/0 conn(0x7f145406a770 0x7f145409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f14537fe700 1 -- 172.21.15.89:0/890991952 >> v1:172.21.15.89:6789/0 conn(0x7f145413c1e0 legacy=0x7f145413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f14537fe700 1 -- 172.21.15.89:0/890991952 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145409e920 con 0x7f145409dc30 2023-09-30T22:20:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f1453fff700 1 --2- 172.21.15.89:0/890991952 >> v2:172.21.15.89:3300/0 conn(0x7f145406a770 0x7f145409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f14537fe700 1 --2- 172.21.15.89:0/890991952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f145409e080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f14480098f0 tx=0x7f1448009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c4cec0a8b0e2e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f1452ffd700 1 -- 172.21.15.89:0/890991952 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448011830 con 0x7f145409dc30 2023-09-30T22:20:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f1452ffd700 1 -- 172.21.15.89:0/890991952 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448011990 con 0x7f145409dc30 2023-09-30T22:20:34.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.105+0000 7f1452ffd700 1 -- 172.21.15.89:0/890991952 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448011d20 con 0x7f145409dc30 2023-09-30T22:20:34.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.106+0000 7f145a4c0700 1 -- 172.21.15.89:0/890991952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 msgr2=0x7f145409e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.106+0000 7f145a4c0700 1 --2- 172.21.15.89:0/890991952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f145409e080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f14480098f0 tx=0x7f1448009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.106+0000 7f145a4c0700 1 -- 172.21.15.89:0/890991952 shutdown_connections 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.106+0000 7f145a4c0700 1 --2- 172.21.15.89:0/890991952 >> v2:172.21.15.89:3300/0 conn(0x7f145406a770 0x7f145409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.106+0000 7f145a4c0700 1 --2- 172.21.15.89:0/890991952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f145409e080 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.106+0000 7f145a4c0700 1 -- 172.21.15.89:0/890991952 >> 172.21.15.89:0/890991952 conn(0x7f1454066f50 msgr2=0x7f145409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.106+0000 7f145a4c0700 1 -- 172.21.15.89:0/890991952 shutdown_connections 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.106+0000 7f145a4c0700 1 -- 172.21.15.89:0/890991952 wait complete. 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.107+0000 7f145a4c0700 1 Processor -- start 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.107+0000 7f145a4c0700 1 -- start start 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.107+0000 7f145a4c0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145406a770 0x7f1454126fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.107+0000 7f145a4c0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f14541274e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.107+0000 7f145a4c0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145413c1e0 0x7f14541cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.107+0000 7f145a4c0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454127a20 con 0x7f145413c1e0 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.107+0000 7f145a4c0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1454126a70 con 0x7f145406a770 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.107+0000 7f145a4c0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1454072490 con 0x7f145409dc30 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f14541274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f1453fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145406a770 0x7f1454126fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f14541274e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55830/0 (socket says 172.21.15.89:55830) 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 -- 172.21.15.89:0/351529503 learned_addr learned my addr 172.21.15.89:0/351529503 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f1458a5d700 1 --2- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145413c1e0 0x7f14541cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 -- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145406a770 msgr2=0x7f1454126fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 --2- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145406a770 0x7f1454126fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 -- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145413c1e0 msgr2=0x7f14541cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 --2- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145413c1e0 0x7f14541cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 -- 172.21.15.89:0/351529503 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440094a0 con 0x7f145409dc30 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f14537fe700 1 --2- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f14541274e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1448000c00 tx=0x7f1448004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f1450ff9700 1 -- 172.21.15.89:0/351529503 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14480045d0 con 0x7f145409dc30 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f1450ff9700 1 -- 172.21.15.89:0/351529503 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144801f070 con 0x7f145409dc30 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1448009550 con 0x7f145409dc30 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f1450ff9700 1 -- 172.21.15.89:0/351529503 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14480038c0 con 0x7f145409dc30 2023-09-30T22:20:34.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.108+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1454129930 con 0x7f145409dc30 2023-09-30T22:20:34.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.109+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1454074ea0 con 0x7f145409dc30 2023-09-30T22:20:34.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.109+0000 7f1450ff9700 1 -- 172.21.15.89:0/351529503 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1448003a20 con 0x7f145409dc30 2023-09-30T22:20:34.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.110+0000 7f1450ff9700 1 -- 172.21.15.89:0/351529503 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1448021070 con 0x7f145409dc30 2023-09-30T22:20:34.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.118+0000 7f1450ff9700 1 -- 172.21.15.89:0/351529503 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f144802e3a0 con 0x7f145409dc30 2023-09-30T22:20:34.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.150+0000 7fe8056b9700 1 Processor -- start 2023-09-30T22:20:34.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.150+0000 7fe8056b9700 1 -- start start 2023-09-30T22:20:34.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.150+0000 7fe8056b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe80009c650 0x7fe80009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.150+0000 7fe8056b9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009cf60 0x7fe800132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.150+0000 7fe8056b9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80009ba70 con 0x7fe8000a0090 2023-09-30T22:20:34.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.150+0000 7fe8056b9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80009e8f0 con 0x7fe80009cf60 2023-09-30T22:20:34.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.150+0000 7fe8056b9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe80009d400 con 0x7fe80009c650 2023-09-30T22:20:34.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7feffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe80009c650 0x7fe80009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7fe7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe8000a0090 0x7fe800131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55526/0 (socket says 172.21.15.89:55526) 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7feffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe80009c650 0x7fe80009ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55834/0 (socket says 172.21.15.89:55834) 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7fe7fc700 1 -- 172.21.15.89:0/1440472885 learned_addr learned my addr 172.21.15.89:0/1440472885 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7ff7fe700 1 --2- 172.21.15.89:0/1440472885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009cf60 0x7fe800132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7feffd700 1 -- 172.21.15.89:0/1440472885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009cf60 msgr2=0x7fe800132080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7feffd700 1 --2- 172.21.15.89:0/1440472885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009cf60 0x7fe800132080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7feffd700 1 -- 172.21.15.89:0/1440472885 >> v1:172.21.15.89:6789/0 conn(0x7fe8000a0090 legacy=0x7fe800131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7feffd700 1 -- 172.21.15.89:0/1440472885 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8000a0760 con 0x7fe80009c650 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7feffd700 1 --2- 172.21.15.89:0/1440472885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe80009c650 0x7fe80009ca20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009870 tx=0x7fe7f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7669397c1533c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7fdffb700 1 -- 172.21.15.89:0/1440472885 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0011830 con 0x7fe80009c650 2023-09-30T22:20:34.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7fdffb700 1 -- 172.21.15.89:0/1440472885 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f0011990 con 0x7fe80009c650 2023-09-30T22:20:34.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.151+0000 7fe7fdffb700 1 -- 172.21.15.89:0/1440472885 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0011d20 con 0x7fe80009c650 2023-09-30T22:20:34.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.152+0000 7fe8056b9700 1 -- 172.21.15.89:0/1440472885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe80009c650 msgr2=0x7fe80009ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.152+0000 7fe8056b9700 1 --2- 172.21.15.89:0/1440472885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe80009c650 0x7fe80009ca20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009870 tx=0x7fe7f0009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:34.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.152+0000 7fe8056b9700 1 -- 172.21.15.89:0/1440472885 shutdown_connections 2023-09-30T22:20:34.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.152+0000 7fe8056b9700 1 --2- 172.21.15.89:0/1440472885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009cf60 0x7fe800132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.152+0000 7fe8056b9700 1 --2- 172.21.15.89:0/1440472885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe80009c650 0x7fe80009ca20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.152+0000 7fe8056b9700 1 -- 172.21.15.89:0/1440472885 >> 172.21.15.89:0/1440472885 conn(0x7fe8000a3f70 msgr2=0x7fe8000a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:34.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.152+0000 7fe8056b9700 1 -- 172.21.15.89:0/1440472885 shutdown_connections 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.152+0000 7fe8056b9700 1 -- 172.21.15.89:0/1440472885 wait complete. 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.153+0000 7fe8056b9700 1 Processor -- start 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.153+0000 7fe8056b9700 1 -- start start 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe8056b9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009c650 0x7fe80006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe8056b9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80009cf60 0x7fe80006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe8056b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8000a0090 0x7fe80006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe8056b9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe800120720 con 0x7fe80009cf60 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe8056b9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80012d5f0 con 0x7fe80009c650 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe8056b9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe80011f7a0 con 0x7fe8000a0090 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7fe7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80009cf60 0x7fe80006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7fe7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80009cf60 0x7fe80006bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58592/0 (socket says 172.21.15.89:58592) 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7fe7fc700 1 -- 172.21.15.89:0/1825365646 learned_addr learned my addr 172.21.15.89:0/1825365646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7ff7fe700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8000a0090 0x7fe80006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7feffd700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009c650 0x7fe80006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7feffd700 1 -- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8000a0090 msgr2=0x7fe80006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7feffd700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8000a0090 0x7fe80006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7feffd700 1 -- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80009cf60 msgr2=0x7fe80006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7feffd700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80009cf60 0x7fe80006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7feffd700 1 -- 172.21.15.89:0/1825365646 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f00094a0 con 0x7fe80009c650 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7feffd700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009c650 0x7fe80006b880 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe8000a1ae0 tx=0x7fe7f0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.154+0000 7fe7e77fe700 1 -- 172.21.15.89:0/1825365646 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0004450 con 0x7fe80009c650 2023-09-30T22:20:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.155+0000 7fe7e77fe700 1 -- 172.21.15.89:0/1825365646 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f001f070 con 0x7fe80009c650 2023-09-30T22:20:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.155+0000 7fe7e77fe700 1 -- 172.21.15.89:0/1825365646 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00038e0 con 0x7fe80009c650 2023-09-30T22:20:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.155+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80011ecc0 con 0x7fe80009c650 2023-09-30T22:20:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.155+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe800130040 con 0x7fe80009c650 2023-09-30T22:20:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.155+0000 7fe7e77fe700 1 -- 172.21.15.89:0/1825365646 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe7f0003a40 con 0x7fe80009c650 2023-09-30T22:20:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.156+0000 7fe7e77fe700 1 -- 172.21.15.89:0/1825365646 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe7f0021070 con 0x7fe80009c650 2023-09-30T22:20:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.156+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8000747f0 con 0x7fe80009c650 2023-09-30T22:20:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.161+0000 7fe7e77fe700 1 -- 172.21.15.89:0/1825365646 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe7f002b080 con 0x7fe80009c650 2023-09-30T22:20:34.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.288+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20a8074ea0 con 0x7f20a806e9d0 2023-09-30T22:20:34.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.288+0000 7f208f7fe700 1 -- 172.21.15.89:0/1099319014 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2098021070 con 0x7f20a806e9d0 2023-09-30T22:20:34.289 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:34.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.291+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a806e9d0 msgr2=0x7f20a8119870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.291+0000 7f20ad3ea700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a806e9d0 0x7f20a8119870 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f209800ec10 tx=0x7f2098009e70 comp rx=0 tx=0).stop 2023-09-30T22:20:34.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.292+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 shutdown_connections 2023-09-30T22:20:34.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.292+0000 7f20ad3ea700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20a8131f40 0x7f20a811a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.292+0000 7f20ad3ea700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20a806e9d0 0x7f20a8119870 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.292+0000 7f20ad3ea700 1 --2- 172.21.15.89:0/1099319014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20a8131630 0x7f20a8119db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.292+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 >> 172.21.15.89:0/1099319014 conn(0x7f20a80a2aa0 msgr2=0x7f20a812e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:34.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.293+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 shutdown_connections 2023-09-30T22:20:34.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.293+0000 7f20ad3ea700 1 -- 172.21.15.89:0/1099319014 wait complete. 2023-09-30T22:20:34.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:34.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.315+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f145406c360 con 0x7f145409dc30 2023-09-30T22:20:34.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.315+0000 7f1450ff9700 1 -- 172.21.15.89:0/351529503 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f144802a070 con 0x7f145409dc30 2023-09-30T22:20:34.316 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:34.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 msgr2=0x7f14541274e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 --2- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f14541274e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1448000c00 tx=0x7f1448004750 comp rx=0 tx=0).stop 2023-09-30T22:20:34.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 shutdown_connections 2023-09-30T22:20:34.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 --2- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f145413c1e0 0x7f14541cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 --2- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f145406a770 0x7f1454126fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 --2- 172.21.15.89:0/351529503 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f145409dc30 0x7f14541274e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 >> 172.21.15.89:0/351529503 conn(0x7f1454066f50 msgr2=0x7f145413b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:34.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 shutdown_connections 2023-09-30T22:20:34.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.317+0000 7f145a4c0700 1 -- 172.21.15.89:0/351529503 wait complete. 2023-09-30T22:20:34.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:34.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.333+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd080121e30 con 0x7fd0800a1340 2023-09-30T22:20:34.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.333+0000 7fd07cff9700 1 -- 172.21.15.89:0/396806085 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd080121e30 con 0x7fd0800a1340 2023-09-30T22:20:34.334 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 msgr2=0x7fd08011ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 --2- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd08011ed60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e900 tx=0x7fd07400ec10 comp rx=0 tx=0).stop 2023-09-30T22:20:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 shutdown_connections 2023-09-30T22:20:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 --2- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd080133fa0 0x7fd08011f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 --2- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0800a0a30 0x7fd08011e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 --2- 172.21.15.89:0/396806085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0800a1340 0x7fd08011ed60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 >> 172.21.15.89:0/396806085 conn(0x7fd0800a6550 msgr2=0x7fd0801b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:34.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 shutdown_connections 2023-09-30T22:20:34.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.335+0000 7fd0864a2700 1 -- 172.21.15.89:0/396806085 wait complete. 2023-09-30T22:20:34.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:34.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.362+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe800072840 con 0x7fe80009c650 2023-09-30T22:20:34.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.362+0000 7fe7e77fe700 1 -- 172.21.15.89:0/1825365646 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe7f0015020 con 0x7fe80009c650 2023-09-30T22:20:34.363 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:34.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.364+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009c650 msgr2=0x7fe80006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:34.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.364+0000 7fe8056b9700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009c650 0x7fe80006b880 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe8000a1ae0 tx=0x7fe7f0004750 comp rx=0 tx=0).stop 2023-09-30T22:20:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.364+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 shutdown_connections 2023-09-30T22:20:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.364+0000 7fe8056b9700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8000a0090 0x7fe80006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.364+0000 7fe8056b9700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe80009c650 0x7fe80006b880 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.364+0000 7fe8056b9700 1 --2- 172.21.15.89:0/1825365646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe80009cf60 0x7fe80006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.365+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 >> 172.21.15.89:0/1825365646 conn(0x7fe8000a3f70 msgr2=0x7fe80012eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.365+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 shutdown_connections 2023-09-30T22:20:34.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:34.365+0000 7fe8056b9700 1 -- 172.21.15.89:0/1825365646 wait complete. 2023-09-30T22:20:34.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:35.311 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:35.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:35.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:35.377 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:35.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.446+0000 7f9dfb8a3700 1 Processor -- start 2023-09-30T22:20:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.446+0000 7f9dfb8a3700 1 -- start start 2023-09-30T22:20:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9dfb8a3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9dfb8a3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9dfb8a3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df41313c0 con 0x7f9df409a3b0 2023-09-30T22:20:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9dfb8a3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df409b580 con 0x7f9df4099a40 2023-09-30T22:20:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9dfb8a3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9df406e8c0 con 0x7f9df4131810 2023-09-30T22:20:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9df963f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9df963f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55840/0 (socket says 172.21.15.89:55840) 2023-09-30T22:20:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9df963f700 1 -- 172.21.15.89:0/1871128429 learned_addr learned my addr 172.21.15.89:0/1871128429 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9df8e3e700 1 --2- 172.21.15.89:0/1871128429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9df963f700 1 -- 172.21.15.89:0/1871128429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 msgr2=0x7f9df4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9df963f700 1 --2- 172.21.15.89:0/1871128429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9df963f700 1 -- 172.21.15.89:0/1871128429 >> v1:172.21.15.89:6789/0 conn(0x7f9df409a3b0 legacy=0x7f9df4138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.447+0000 7f9df963f700 1 -- 172.21.15.89:0/1871128429 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df409a820 con 0x7f9df4131810 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9df963f700 1 --2- 172.21.15.89:0/1871128429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4131be0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009920 tx=0x7f9df0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe48117661f2951 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9debfff700 1 -- 172.21.15.89:0/1871128429 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0011830 con 0x7f9df4131810 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9debfff700 1 -- 172.21.15.89:0/1871128429 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0011990 con 0x7f9df4131810 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9debfff700 1 -- 172.21.15.89:0/1871128429 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0011d20 con 0x7f9df4131810 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/1871128429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 msgr2=0x7f9df4131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9dfb8a3700 1 --2- 172.21.15.89:0/1871128429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4131be0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009920 tx=0x7f9df0009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:35.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/1871128429 shutdown_connections 2023-09-30T22:20:35.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9dfb8a3700 1 --2- 172.21.15.89:0/1871128429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4131be0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9dfb8a3700 1 --2- 172.21.15.89:0/1871128429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.448+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/1871128429 >> 172.21.15.89:0/1871128429 conn(0x7f9df40a2aa0 msgr2=0x7f9df40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:35.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.449+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/1871128429 shutdown_connections 2023-09-30T22:20:35.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.449+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/1871128429 wait complete. 2023-09-30T22:20:35.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9dfb8a3700 1 Processor -- start 2023-09-30T22:20:35.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9dfb8a3700 1 -- start start 2023-09-30T22:20:35.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9dfb8a3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9dfb8a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df409a3b0 0x7f9df406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9dfb8a3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9dfb8a3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4121300 con 0x7f9df409a3b0 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9dfb8a3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4120d60 con 0x7f9df4099a40 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9dfb8a3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4130b20 con 0x7f9df4131810 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9df9e40700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9df8e3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df409a3b0 0x7f9df406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9df9e40700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55848/0 (socket says 172.21.15.89:55848) 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9df9e40700 1 -- 172.21.15.89:0/605428354 learned_addr learned my addr 172.21.15.89:0/605428354 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.450+0000 7f9df963f700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9df9e40700 1 -- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 msgr2=0x7f9df406f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9df9e40700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df406f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9df9e40700 1 -- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df409a3b0 msgr2=0x7f9df406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9df9e40700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df409a3b0 0x7f9df406fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9df9e40700 1 -- 172.21.15.89:0/605428354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0009550 con 0x7f9df4131810 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9df963f700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9df8e3e700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df409a3b0 0x7f9df406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9df9e40700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4070090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00cc50 tx=0x7f9dec00cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9de9ffb700 1 -- 172.21.15.89:0/605428354 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec003c00 con 0x7f9df4131810 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9de9ffb700 1 -- 172.21.15.89:0/605428354 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec003d60 con 0x7f9df4131810 2023-09-30T22:20:35.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df406eb00 con 0x7f9df4131810 2023-09-30T22:20:35.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9de9ffb700 1 -- 172.21.15.89:0/605428354 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec00ab60 con 0x7f9df4131810 2023-09-30T22:20:35.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.451+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df4121a90 con 0x7f9df4131810 2023-09-30T22:20:35.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.452+0000 7f9de9ffb700 1 -- 172.21.15.89:0/605428354 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9dec00acc0 con 0x7f9df4131810 2023-09-30T22:20:35.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.453+0000 7f9de9ffb700 1 -- 172.21.15.89:0/605428354 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9dec03f6b0 con 0x7f9df4131810 2023-09-30T22:20:35.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.453+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df4074ea0 con 0x7f9df4131810 2023-09-30T22:20:35.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.457+0000 7f9de9ffb700 1 -- 172.21.15.89:0/605428354 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9dec0198f0 con 0x7f9df4131810 2023-09-30T22:20:35.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.485+0000 7fe0a330c700 1 Processor -- start 2023-09-30T22:20:35.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.485+0000 7fe0a330c700 1 -- start start 2023-09-30T22:20:35.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a330c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe09c09d6f0 0x7fe09c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a330c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09e440 0x7fe09c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a330c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09c09b120 con 0x7fe09c09e000 2023-09-30T22:20:35.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a330c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c06b460 con 0x7fe09c09d6f0 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a330c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c09a7e0 con 0x7fe09c09e440 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a18a9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09e440 0x7fe09c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a08a7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe09c09e000 0x7fe09c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55540/0 (socket says 172.21.15.89:55540) 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a10a8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe09c09d6f0 0x7fe09c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a08a7700 1 -- 172.21.15.89:0/1046403197 learned_addr learned my addr 172.21.15.89:0/1046403197 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a18a9700 1 -- 172.21.15.89:0/1046403197 >> v2:172.21.15.89:3300/0 conn(0x7fe09c09d6f0 msgr2=0x7fe09c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a18a9700 1 --2- 172.21.15.89:0/1046403197 >> v2:172.21.15.89:3300/0 conn(0x7fe09c09d6f0 0x7fe09c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a18a9700 1 -- 172.21.15.89:0/1046403197 >> v1:172.21.15.89:6789/0 conn(0x7fe09c09e000 legacy=0x7fe09c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a18a9700 1 -- 172.21.15.89:0/1046403197 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c09a500 con 0x7fe09c09e440 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.486+0000 7fe0a10a8700 1 --2- 172.21.15.89:0/1046403197 >> v2:172.21.15.89:3300/0 conn(0x7fe09c09d6f0 0x7fe09c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:35.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe0a18a9700 1 --2- 172.21.15.89:0/1046403197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09e440 0x7fe09c134100 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe094009dc0 tx=0x7fe09400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db23b82a9fa5184 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:35.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe093fff700 1 -- 172.21.15.89:0/1046403197 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400e070 con 0x7fe09c09e440 2023-09-30T22:20:35.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe093fff700 1 -- 172.21.15.89:0/1046403197 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09400bae0 con 0x7fe09c09e440 2023-09-30T22:20:35.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe093fff700 1 -- 172.21.15.89:0/1046403197 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400be90 con 0x7fe09c09e440 2023-09-30T22:20:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe0a330c700 1 -- 172.21.15.89:0/1046403197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09e440 msgr2=0x7fe09c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe0a330c700 1 --2- 172.21.15.89:0/1046403197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09e440 0x7fe09c134100 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe094009dc0 tx=0x7fe09400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe0a330c700 1 -- 172.21.15.89:0/1046403197 shutdown_connections 2023-09-30T22:20:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe0a330c700 1 --2- 172.21.15.89:0/1046403197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09e440 0x7fe09c134100 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe0a330c700 1 --2- 172.21.15.89:0/1046403197 >> v2:172.21.15.89:3300/0 conn(0x7fe09c09d6f0 0x7fe09c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.487+0000 7fe0a330c700 1 -- 172.21.15.89:0/1046403197 >> 172.21.15.89:0/1046403197 conn(0x7fe09c066f50 msgr2=0x7fe09c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.488+0000 7fe0a330c700 1 -- 172.21.15.89:0/1046403197 shutdown_connections 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.488+0000 7fe0a330c700 1 -- 172.21.15.89:0/1046403197 wait complete. 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.488+0000 7fe0a330c700 1 Processor -- start 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.488+0000 7fe0a330c700 1 -- start start 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a330c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09d6f0 0x7fe09c1a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a330c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe09c09e000 0x7fe09c1a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a330c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe09c09e440 0x7fe09c1a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a330c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a99d0 con 0x7fe09c09e000 2023-09-30T22:20:35.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a330c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a7b60 con 0x7fe09c09d6f0 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a330c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0a1f90 con 0x7fe09c09e440 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a10a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09d6f0 0x7fe09c1a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a10a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09d6f0 0x7fe09c1a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59556/0 (socket says 172.21.15.89:59556) 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a10a8700 1 -- 172.21.15.89:0/3770081958 learned_addr learned my addr 172.21.15.89:0/3770081958 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a18a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe09c09e440 0x7fe09c1a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a08a7700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe09c09e000 0x7fe09c1a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a18a9700 1 -- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09d6f0 msgr2=0x7fe09c1a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a18a9700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09d6f0 0x7fe09c1a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a18a9700 1 -- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe09c09e000 msgr2=0x7fe09c1a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a18a9700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe09c09e000 0x7fe09c1a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.489+0000 7fe0a18a9700 1 -- 172.21.15.89:0/3770081958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980094a0 con 0x7fe09c09e440 2023-09-30T22:20:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.490+0000 7fe0a10a8700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09d6f0 0x7fe09c1a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.490+0000 7fe0a18a9700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe09c09e440 0x7fe09c1a6930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe094016000 tx=0x7fe094004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.490+0000 7fe091ffb700 1 -- 172.21.15.89:0/3770081958 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940045d0 con 0x7fe09c09e440 2023-09-30T22:20:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.490+0000 7fe091ffb700 1 -- 172.21.15.89:0/3770081958 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094020070 con 0x7fe09c09e440 2023-09-30T22:20:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.490+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe094009a70 con 0x7fe09c09e440 2023-09-30T22:20:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.490+0000 7fe091ffb700 1 -- 172.21.15.89:0/3770081958 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094027480 con 0x7fe09c09e440 2023-09-30T22:20:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.490+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c1a88c0 con 0x7fe09c09e440 2023-09-30T22:20:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.491+0000 7fe091ffb700 1 -- 172.21.15.89:0/3770081958 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe0940275e0 con 0x7fe09c09e440 2023-09-30T22:20:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.491+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c074ea0 con 0x7fe09c09e440 2023-09-30T22:20:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.491+0000 7fe091ffb700 1 -- 172.21.15.89:0/3770081958 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe094022070 con 0x7fe09c09e440 2023-09-30T22:20:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.494+0000 7f9c48bae700 1 Processor -- start 2023-09-30T22:20:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.494+0000 7f9c48bae700 1 -- start start 2023-09-30T22:20:35.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.495+0000 7f9c48bae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409d6f0 0x7f9c4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.495+0000 7f9c48bae700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c4409e410 0x7f9c44134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.495+0000 7f9c48bae700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4409b120 con 0x7f9c4409e000 2023-09-30T22:20:35.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.495+0000 7f9c48bae700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4406b460 con 0x7f9c4409e410 2023-09-30T22:20:35.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.495+0000 7f9c48bae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4409a7e0 con 0x7f9c4409d6f0 2023-09-30T22:20:35.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c4259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409d6f0 0x7f9c4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c42d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c4409e410 0x7f9c44134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c4259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409d6f0 0x7f9c4409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55866/0 (socket says 172.21.15.89:55866) 2023-09-30T22:20:35.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c42d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c4409e410 0x7f9c44134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58638/0 (socket says 172.21.15.89:58638) 2023-09-30T22:20:35.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c4259c700 1 -- 172.21.15.89:0/1898089436 learned_addr learned my addr 172.21.15.89:0/1898089436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c42d9d700 1 -- 172.21.15.89:0/1898089436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409d6f0 msgr2=0x7f9c4409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c42d9d700 1 --2- 172.21.15.89:0/1898089436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409d6f0 0x7f9c4409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c42d9d700 1 -- 172.21.15.89:0/1898089436 >> v1:172.21.15.89:6789/0 conn(0x7f9c4409e000 legacy=0x7f9c441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c42d9d700 1 -- 172.21.15.89:0/1898089436 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4409a500 con 0x7f9c4409e410 2023-09-30T22:20:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c42d9d700 1 --2- 172.21.15.89:0/1898089436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409e410 0x7f9c44134100 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009d90 tx=0x7f9c3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895ca4671cea9337 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c4159a700 1 -- 172.21.15.89:0/1898089436 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00e070 con 0x7f9c4409e410 2023-09-30T22:20:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c4159a700 1 -- 172.21.15.89:0/1898089436 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00bbd0 con 0x7f9c4409e410 2023-09-30T22:20:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7f9c4159a700 1 -- 172.21.15.89:0/1898089436 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c012430 con 0x7f9c4409e410 2023-09-30T22:20:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.496+0000 7fe091ffb700 1 -- 172.21.15.89:0/3770081958 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe094027890 con 0x7fe09c09e440 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.497+0000 7f9c48bae700 1 -- 172.21.15.89:0/1898089436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409e410 msgr2=0x7f9c44134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.497+0000 7f9c48bae700 1 --2- 172.21.15.89:0/1898089436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409e410 0x7f9c44134100 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009d90 tx=0x7f9c3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.497+0000 7f9c48bae700 1 -- 172.21.15.89:0/1898089436 shutdown_connections 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.497+0000 7f9c48bae700 1 --2- 172.21.15.89:0/1898089436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409e410 0x7f9c44134100 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.497+0000 7f9c48bae700 1 --2- 172.21.15.89:0/1898089436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409d6f0 0x7f9c4409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.497+0000 7f9c48bae700 1 -- 172.21.15.89:0/1898089436 >> 172.21.15.89:0/1898089436 conn(0x7f9c44066f50 msgr2=0x7f9c4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.497+0000 7f9c48bae700 1 -- 172.21.15.89:0/1898089436 shutdown_connections 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.497+0000 7f9c48bae700 1 -- 172.21.15.89:0/1898089436 wait complete. 2023-09-30T22:20:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.498+0000 7f9c48bae700 1 Processor -- start 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.498+0000 7f9c48bae700 1 -- start start 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c48bae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409d6f0 0x7f9c441a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c48bae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c4409e000 0x7f9c441a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c48bae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409e410 0x7f9c441a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c48bae700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c441a8940 con 0x7f9c4409d6f0 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c48bae700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c441a7470 con 0x7f9c4409e000 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c48bae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c441a7b40 con 0x7f9c4409e410 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c4259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409d6f0 0x7f9c441a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c41d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c4409e000 0x7f9c441a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c4259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409d6f0 0x7f9c441a5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58640/0 (socket says 172.21.15.89:58640) 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c4259c700 1 -- 172.21.15.89:0/1094548951 learned_addr learned my addr 172.21.15.89:0/1094548951 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c42d9d700 1 --2- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409e410 0x7f9c441a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c42d9d700 1 -- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c4409e000 msgr2=0x7f9c441a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c42d9d700 1 --2- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c4409e000 0x7f9c441a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c42d9d700 1 -- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409d6f0 msgr2=0x7f9c441a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c42d9d700 1 --2- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409d6f0 0x7f9c441a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.499+0000 7f9c42d9d700 1 -- 172.21.15.89:0/1094548951 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009a70 con 0x7f9c4409e410 2023-09-30T22:20:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.500+0000 7f9c42d9d700 1 --2- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409e410 0x7f9c441a6a40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c016000 tx=0x7f9c3c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.500+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1094548951 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c012430 con 0x7f9c4409e410 2023-09-30T22:20:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.500+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1094548951 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c020070 con 0x7f9c4409e410 2023-09-30T22:20:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.500+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1094548951 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c004040 con 0x7f9c4409e410 2023-09-30T22:20:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.501+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c441a8e00 con 0x7f9c4409e410 2023-09-30T22:20:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.501+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4409c710 con 0x7f9c4409e410 2023-09-30T22:20:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.502+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1094548951 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c3c0041a0 con 0x7f9c4409e410 2023-09-30T22:20:35.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.502+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1094548951 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9c3c022070 con 0x7f9c4409e410 2023-09-30T22:20:35.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.505+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c44074ea0 con 0x7f9c4409e410 2023-09-30T22:20:35.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.514+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1094548951 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c3c004450 con 0x7f9c4409e410 2023-09-30T22:20:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.531+0000 7ffb53f58700 1 Processor -- start 2023-09-30T22:20:35.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.531+0000 7ffb53f58700 1 -- start start 2023-09-30T22:20:35.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb53f58700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb4c06e570 0x7ffb4c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb53f58700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c09eca0 0x7ffb4c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb53f58700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c134a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb53f58700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c070ee0 con 0x7ffb4c06e570 2023-09-30T22:20:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb53f58700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c071480 con 0x7ffb4c09eca0 2023-09-30T22:20:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb53f58700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c06f3c0 con 0x7ffb4c134610 2023-09-30T22:20:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb51cf4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb4c06e570 0x7ffb4c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb524f5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb51cf4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb4c06e570 0x7ffb4c06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58650/0 (socket says 172.21.15.89:58650) 2023-09-30T22:20:35.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb51cf4700 1 -- 172.21.15.89:0/516748706 learned_addr learned my addr 172.21.15.89:0/516748706 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb51cf4700 1 -- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 msgr2=0x7ffb4c134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb51cf4700 1 --2- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c134a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb51cf4700 1 -- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c09eca0 msgr2=0x7ffb4c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb51cf4700 1 --2- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c09eca0 0x7ffb4c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb51cf4700 1 -- 172.21.15.89:0/516748706 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c071b90 con 0x7ffb4c06e570 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.532+0000 7ffb524f5700 1 --2- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb51cf4700 1 --2- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c06e570 0x7ffb4c06e940 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffb480098f0 tx=0x7ffb48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4705e6648ccd578 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb50cf2700 1 -- 172.21.15.89:0/516748706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011830 con 0x7ffb4c06e570 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb50cf2700 1 -- 172.21.15.89:0/516748706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48011990 con 0x7ffb4c06e570 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb50cf2700 1 -- 172.21.15.89:0/516748706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011d20 con 0x7ffb4c06e570 2023-09-30T22:20:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb53f58700 1 -- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c06e570 msgr2=0x7ffb4c06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb53f58700 1 --2- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c06e570 0x7ffb4c06e940 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffb480098f0 tx=0x7ffb48009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb53f58700 1 -- 172.21.15.89:0/516748706 shutdown_connections 2023-09-30T22:20:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb53f58700 1 --2- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb53f58700 1 --2- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c06e570 0x7ffb4c06e940 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb53f58700 1 --2- 172.21.15.89:0/516748706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c09eca0 0x7ffb4c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.533+0000 7ffb53f58700 1 -- 172.21.15.89:0/516748706 >> 172.21.15.89:0/516748706 conn(0x7ffb4c0a2aa0 msgr2=0x7ffb4c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.534+0000 7ffb53f58700 1 -- 172.21.15.89:0/516748706 shutdown_connections 2023-09-30T22:20:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.534+0000 7ffb53f58700 1 -- 172.21.15.89:0/516748706 wait complete. 2023-09-30T22:20:35.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.535+0000 7ffb53f58700 1 Processor -- start 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.535+0000 7ffb53f58700 1 -- start start 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb53f58700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c06e570 0x7ffb4c123440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb53f58700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c123980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb53f58700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c123ec0 0x7ffb4c1cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb53f58700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c070610 con 0x7ffb4c123ec0 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb53f58700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c1258b0 con 0x7ffb4c06e570 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb53f58700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c125160 con 0x7ffb4c134610 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb51cf4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c06e570 0x7ffb4c123440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb51cf4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c06e570 0x7ffb4c123440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59568/0 (socket says 172.21.15.89:59568) 2023-09-30T22:20:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb51cf4700 1 -- 172.21.15.89:0/4107109704 learned_addr learned my addr 172.21.15.89:0/4107109704 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb514f3700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c123980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb524f5700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c123ec0 0x7ffb4c1cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb51cf4700 1 -- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 msgr2=0x7ffb4c123980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb51cf4700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c123980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb51cf4700 1 -- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c123ec0 msgr2=0x7ffb4c1cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb51cf4700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c123ec0 0x7ffb4c1cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.536+0000 7ffb51cf4700 1 -- 172.21.15.89:0/4107109704 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb44009a70 con 0x7ffb4c06e570 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.537+0000 7ffb51cf4700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c06e570 0x7ffb4c123440 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009c40 tx=0x7ffb48004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.537+0000 7ffb427fc700 1 -- 172.21.15.89:0/4107109704 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480044d0 con 0x7ffb4c06e570 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.537+0000 7ffb427fc700 1 -- 172.21.15.89:0/4107109704 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb480036a0 con 0x7ffb4c06e570 2023-09-30T22:20:35.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.537+0000 7ffb427fc700 1 -- 172.21.15.89:0/4107109704 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48003a30 con 0x7ffb4c06e570 2023-09-30T22:20:35.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.538+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb48009550 con 0x7ffb4c06e570 2023-09-30T22:20:35.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.538+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c0a1fd0 con 0x7ffb4c06e570 2023-09-30T22:20:35.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.539+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c06d5a0 con 0x7ffb4c06e570 2023-09-30T22:20:35.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.539+0000 7ffb427fc700 1 -- 172.21.15.89:0/4107109704 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb48003b90 con 0x7ffb4c06e570 2023-09-30T22:20:35.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.539+0000 7ffb427fc700 1 -- 172.21.15.89:0/4107109704 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffb48040d50 con 0x7ffb4c06e570 2023-09-30T22:20:35.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.543+0000 7ffb427fc700 1 -- 172.21.15.89:0/4107109704 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb48026450 con 0x7ffb4c06e570 2023-09-30T22:20:35.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.665+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9df406b9e0 con 0x7f9df4131810 2023-09-30T22:20:35.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.665+0000 7f9de9ffb700 1 -- 172.21.15.89:0/605428354 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9dec011380 con 0x7f9df4131810 2023-09-30T22:20:35.666 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:35.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 msgr2=0x7f9df4070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4070090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00cc50 tx=0x7f9dec00cf60 comp rx=0 tx=0).stop 2023-09-30T22:20:35.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 shutdown_connections 2023-09-30T22:20:35.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9df4131810 0x7f9df4070090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9df4099a40 0x7f9df406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 --2- 172.21.15.89:0/605428354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df409a3b0 0x7f9df406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 >> 172.21.15.89:0/605428354 conn(0x7f9df40a2aa0 msgr2=0x7f9df41b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:35.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 shutdown_connections 2023-09-30T22:20:35.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.668+0000 7f9dfb8a3700 1 -- 172.21.15.89:0/605428354 wait complete. 2023-09-30T22:20:35.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:35.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.704+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe09c09f420 con 0x7fe09c09e440 2023-09-30T22:20:35.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.704+0000 7fe091ffb700 1 -- 172.21.15.89:0/3770081958 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe09402c030 con 0x7fe09c09e440 2023-09-30T22:20:35.705 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe09c09e440 msgr2=0x7fe09c1a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe09c09e440 0x7fe09c1a6930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe094016000 tx=0x7fe094004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 shutdown_connections 2023-09-30T22:20:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe09c09e440 0x7fe09c1a6930 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe09c09d6f0 0x7fe09c1a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 --2- 172.21.15.89:0/3770081958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe09c09e000 0x7fe09c1a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 >> 172.21.15.89:0/3770081958 conn(0x7fe09c066f50 msgr2=0x7fe09c1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:35.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 shutdown_connections 2023-09-30T22:20:35.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.706+0000 7fe0a330c700 1 -- 172.21.15.89:0/3770081958 wait complete. 2023-09-30T22:20:35.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:35.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.727+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9c4406c360 con 0x7f9c4409e410 2023-09-30T22:20:35.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.727+0000 7f9c32ffd700 1 -- 172.21.15.89:0/1094548951 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c3c011390 con 0x7f9c4409e410 2023-09-30T22:20:35.728 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:35.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.729+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409e410 msgr2=0x7f9c441a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.729+0000 7f9c48bae700 1 --2- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409e410 0x7f9c441a6a40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c016000 tx=0x7f9c3c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.729+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 shutdown_connections 2023-09-30T22:20:35.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.729+0000 7f9c48bae700 1 --2- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c4409e410 0x7f9c441a6a40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.729+0000 7f9c48bae700 1 --2- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c4409d6f0 0x7f9c441a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.729+0000 7f9c48bae700 1 --2- 172.21.15.89:0/1094548951 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c4409e000 0x7f9c441a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.729+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 >> 172.21.15.89:0/1094548951 conn(0x7f9c44066f50 msgr2=0x7f9c441baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:35.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.730+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 shutdown_connections 2023-09-30T22:20:35.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.730+0000 7f9c48bae700 1 -- 172.21.15.89:0/1094548951 wait complete. 2023-09-30T22:20:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.738+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffb4c074ea0 con 0x7ffb4c06e570 2023-09-30T22:20:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.738+0000 7ffb427fc700 1 -- 172.21.15.89:0/4107109704 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb4802b030 con 0x7ffb4c06e570 2023-09-30T22:20:35.739 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:35.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c06e570 msgr2=0x7ffb4c123440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c06e570 0x7ffb4c123440 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009c40 tx=0x7ffb48004750 comp rx=0 tx=0).stop 2023-09-30T22:20:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 shutdown_connections 2023-09-30T22:20:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb4c123ec0 0x7ffb4c1cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb4c06e570 0x7ffb4c123440 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 --2- 172.21.15.89:0/4107109704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffb4c134610 0x7ffb4c123980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 >> 172.21.15.89:0/4107109704 conn(0x7ffb4c0a2aa0 msgr2=0x7ffb4c0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 shutdown_connections 2023-09-30T22:20:35.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:35.741+0000 7ffb53f58700 1 -- 172.21.15.89:0/4107109704 wait complete. 2023-09-30T22:20:35.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:35.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:36.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:36.720 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:36.744 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:36.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:36.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.825+0000 7f9f8b568700 1 Processor -- start 2023-09-30T22:20:36.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.825+0000 7f9f8b568700 1 -- start start 2023-09-30T22:20:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.825+0000 7f9f8b568700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f841338a0 0x7f9f84133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.825+0000 7f9f8b568700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f8409e930 0x7f9f8413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.825+0000 7f9f8b568700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8406ef30 con 0x7f9f8409de50 2023-09-30T22:20:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.825+0000 7f9f8b568700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8406fef0 con 0x7f9f8409e930 2023-09-30T22:20:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.825+0000 7f9f8b568700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f841334b0 con 0x7f9f841338a0 2023-09-30T22:20:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f89b05700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f8409e930 0x7f9f8413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f88b03700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9f8409de50 0x7f9f8409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55556/0 (socket says 172.21.15.89:55556) 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f88b03700 1 -- 172.21.15.89:0/3192713610 learned_addr learned my addr 172.21.15.89:0/3192713610 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f89304700 1 --2- 172.21.15.89:0/3192713610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f841338a0 0x7f9f84133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f89b05700 1 -- 172.21.15.89:0/3192713610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f841338a0 msgr2=0x7f9f84133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f89b05700 1 --2- 172.21.15.89:0/3192713610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f841338a0 0x7f9f84133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f7bfff700 1 -- 172.21.15.89:0/3192713610 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437582360 0 0) 0x7f9f8406ef30 con 0x7f9f8409de50 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f89b05700 1 -- 172.21.15.89:0/3192713610 >> v1:172.21.15.89:6789/0 conn(0x7f9f8409de50 legacy=0x7f9f8409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f89b05700 1 -- 172.21.15.89:0/3192713610 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8406fc30 con 0x7f9f8409e930 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f89b05700 1 --2- 172.21.15.89:0/3192713610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f8409e930 0x7f9f8413b740 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009d90 tx=0x7f9f7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5960ae984f4be3dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f7bfff700 1 -- 172.21.15.89:0/3192713610 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00e070 con 0x7f9f8409e930 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f7bfff700 1 -- 172.21.15.89:0/3192713610 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c00bb20 con 0x7f9f8409e930 2023-09-30T22:20:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.826+0000 7f9f7bfff700 1 -- 172.21.15.89:0/3192713610 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c012430 con 0x7f9f8409e930 2023-09-30T22:20:36.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.827+0000 7f9f8b568700 1 -- 172.21.15.89:0/3192713610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f8409e930 msgr2=0x7f9f8413b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.827+0000 7f9f8b568700 1 --2- 172.21.15.89:0/3192713610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f8409e930 0x7f9f8413b740 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009d90 tx=0x7f9f7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.827+0000 7f9f8b568700 1 -- 172.21.15.89:0/3192713610 shutdown_connections 2023-09-30T22:20:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.827+0000 7f9f8b568700 1 --2- 172.21.15.89:0/3192713610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f8409e930 0x7f9f8413b740 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.827+0000 7f9f8b568700 1 --2- 172.21.15.89:0/3192713610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f841338a0 0x7f9f84133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.827+0000 7f9f8b568700 1 -- 172.21.15.89:0/3192713610 >> 172.21.15.89:0/3192713610 conn(0x7f9f840a2aa0 msgr2=0x7f9f840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.827+0000 7f9f8b568700 1 -- 172.21.15.89:0/3192713610 shutdown_connections 2023-09-30T22:20:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.827+0000 7f9f8b568700 1 -- 172.21.15.89:0/3192713610 wait complete. 2023-09-30T22:20:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.829+0000 7f9f8b568700 1 Processor -- start 2023-09-30T22:20:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.829+0000 7f9f8b568700 1 -- start start 2023-09-30T22:20:36.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.829+0000 7f9f8b568700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f8409de50 0x7f9f8411b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.829+0000 7f9f8b568700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8409e930 0x7f9f8411bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.829+0000 7f9f8b568700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f841338a0 0x7f9f8411c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.829+0000 7f9f8b568700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8411d280 con 0x7f9f841338a0 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.829+0000 7f9f8b568700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8411e4d0 con 0x7f9f8409e930 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.829+0000 7f9f8b568700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8411d860 con 0x7f9f8409de50 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f88b03700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8409e930 0x7f9f8411bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f88b03700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8409e930 0x7f9f8411bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59584/0 (socket says 172.21.15.89:59584) 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f88b03700 1 -- 172.21.15.89:0/1345264997 learned_addr learned my addr 172.21.15.89:0/1345264997 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f89b05700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f841338a0 0x7f9f8411c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f89304700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f8409de50 0x7f9f8411b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f89b05700 1 -- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f8409de50 msgr2=0x7f9f8411b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f89b05700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f8409de50 0x7f9f8411b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f89b05700 1 -- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8409e930 msgr2=0x7f9f8411bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f89b05700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8409e930 0x7f9f8411bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f89b05700 1 -- 172.21.15.89:0/1345264997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c009a70 con 0x7f9f841338a0 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f88b03700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8409e930 0x7f9f8411bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f89b05700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f841338a0 0x7f9f8411c0f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c016000 tx=0x7f9f7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f79ffb700 1 -- 172.21.15.89:0/1345264997 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c012430 con 0x7f9f841338a0 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f79ffb700 1 -- 172.21.15.89:0/1345264997 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c020070 con 0x7f9f841338a0 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8411eeb0 con 0x7f9f841338a0 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.830+0000 7f9f79ffb700 1 -- 172.21.15.89:0/1345264997 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c004280 con 0x7f9f841338a0 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.831+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8406f640 con 0x7f9f841338a0 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.831+0000 7f9f79ffb700 1 -- 172.21.15.89:0/1345264997 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f7c0043e0 con 0x7f9f841338a0 2023-09-30T22:20:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.832+0000 7f9f79ffb700 1 -- 172.21.15.89:0/1345264997 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9f7c022070 con 0x7f9f841338a0 2023-09-30T22:20:36.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.832+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f84132a40 con 0x7f9f841338a0 2023-09-30T22:20:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.837+0000 7f9f79ffb700 1 -- 172.21.15.89:0/1345264997 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f7c01a380 con 0x7f9f841338a0 2023-09-30T22:20:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.880+0000 7feb2aafb700 1 Processor -- start 2023-09-30T22:20:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.880+0000 7feb2aafb700 1 -- start start 2023-09-30T22:20:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.880+0000 7feb2aafb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb2409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.880+0000 7feb2aafb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e000 0x7feb241339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.880+0000 7feb2aafb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2409b120 con 0x7feb2409e410 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.880+0000 7feb2aafb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2406b460 con 0x7feb2409d6f0 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.880+0000 7feb2aafb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb2409a7e0 con 0x7feb2409e000 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb28897700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb2409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb28897700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb2409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59590/0 (socket says 172.21.15.89:59590) 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb28897700 1 -- 172.21.15.89:0/3767169818 learned_addr learned my addr 172.21.15.89:0/3767169818 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb23fff700 1 --2- 172.21.15.89:0/3767169818 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e000 0x7feb241339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb28897700 1 -- 172.21.15.89:0/3767169818 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e000 msgr2=0x7feb241339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb28897700 1 --2- 172.21.15.89:0/3767169818 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e000 0x7feb241339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb28897700 1 -- 172.21.15.89:0/3767169818 >> v1:172.21.15.89:6789/0 conn(0x7feb2409e410 legacy=0x7feb24133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb28897700 1 -- 172.21.15.89:0/3767169818 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2409a500 con 0x7feb2409d6f0 2023-09-30T22:20:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb28897700 1 --2- 172.21.15.89:0/3767169818 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb2409dac0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7feb18009920 tx=0x7feb18009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8afc3c79bfde01ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb237fe700 1 -- 172.21.15.89:0/3767169818 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18011830 con 0x7feb2409d6f0 2023-09-30T22:20:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb237fe700 1 -- 172.21.15.89:0/3767169818 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb18011990 con 0x7feb2409d6f0 2023-09-30T22:20:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.881+0000 7feb237fe700 1 -- 172.21.15.89:0/3767169818 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18011d20 con 0x7feb2409d6f0 2023-09-30T22:20:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.882+0000 7feb2aafb700 1 -- 172.21.15.89:0/3767169818 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 msgr2=0x7feb2409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.882+0000 7feb2aafb700 1 --2- 172.21.15.89:0/3767169818 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb2409dac0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7feb18009920 tx=0x7feb18009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.882+0000 7feb2aafb700 1 -- 172.21.15.89:0/3767169818 shutdown_connections 2023-09-30T22:20:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.882+0000 7feb2aafb700 1 --2- 172.21.15.89:0/3767169818 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb2409dac0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.882+0000 7feb2aafb700 1 --2- 172.21.15.89:0/3767169818 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e000 0x7feb241339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.882+0000 7feb2aafb700 1 -- 172.21.15.89:0/3767169818 >> 172.21.15.89:0/3767169818 conn(0x7feb24066f50 msgr2=0x7feb2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.882+0000 7feb2aafb700 1 -- 172.21.15.89:0/3767169818 shutdown_connections 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.883+0000 7feb2aafb700 1 -- 172.21.15.89:0/3767169818 wait complete. 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.884+0000 7feb2aafb700 1 Processor -- start 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.884+0000 7feb2aafb700 1 -- start start 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.884+0000 7feb2aafb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb24122600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.884+0000 7feb2aafb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb2409e000 0x7feb24122b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.884+0000 7feb2aafb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e410 0x7feb24123080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.884+0000 7feb2aafb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240a1e20 con 0x7feb2409e000 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.884+0000 7feb2aafb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2409c710 con 0x7feb2409d6f0 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.884+0000 7feb2aafb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb24124f90 con 0x7feb2409e410 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb24122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb24122600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59602/0 (socket says 172.21.15.89:59602) 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 -- 172.21.15.89:0/2251533753 learned_addr learned my addr 172.21.15.89:0/2251533753 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb23fff700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb2409e000 0x7feb24122b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb29098700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e410 0x7feb24123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 -- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e410 msgr2=0x7feb24123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e410 0x7feb24123080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 -- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb2409e000 msgr2=0x7feb24122b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb2409e000 0x7feb24122b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 -- 172.21.15.89:0/2251533753 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb18009550 con 0x7feb2409d6f0 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb28897700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb24122600 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feb18000c00 tx=0x7feb18004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb217fa700 1 -- 172.21.15.89:0/2251533753 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18004450 con 0x7feb2409d6f0 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb217fa700 1 -- 172.21.15.89:0/2251533753 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb180045b0 con 0x7feb2409d6f0 2023-09-30T22:20:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb24125810 con 0x7feb2409d6f0 2023-09-30T22:20:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb217fa700 1 -- 172.21.15.89:0/2251533753 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180038a0 con 0x7feb2409d6f0 2023-09-30T22:20:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.885+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb24126810 con 0x7feb2409d6f0 2023-09-30T22:20:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.886+0000 7feb217fa700 1 -- 172.21.15.89:0/2251533753 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb18003a00 con 0x7feb2409d6f0 2023-09-30T22:20:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.886+0000 7feb217fa700 1 -- 172.21.15.89:0/2251533753 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feb18029030 con 0x7feb2409d6f0 2023-09-30T22:20:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.887+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb24074ea0 con 0x7feb2409d6f0 2023-09-30T22:20:36.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.891+0000 7feb217fa700 1 -- 172.21.15.89:0/2251533753 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb18033080 con 0x7feb2409d6f0 2023-09-30T22:20:36.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.906+0000 7fcdbe73f700 1 Processor -- start 2023-09-30T22:20:36.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7f25d41c4700 1 Processor -- start 2023-09-30T22:20:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7f25d41c4700 1 -- start start 2023-09-30T22:20:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdbe73f700 1 -- start start 2023-09-30T22:20:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdbe73f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb809e1d0 0x7fcdb8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdbe73f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb8135e50 0x7fcdb8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdbe73f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb809f130 con 0x7fcdb809d6f0 2023-09-30T22:20:36.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdbe73f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb809f520 con 0x7fcdb8135e50 2023-09-30T22:20:36.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdbe73f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb809e930 con 0x7fcdb809e1d0 2023-09-30T22:20:36.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdb77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb809e1d0 0x7fcdb8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdb77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb809e1d0 0x7fcdb8135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55898/0 (socket says 172.21.15.89:55898) 2023-09-30T22:20:36.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdb77fe700 1 -- 172.21.15.89:0/1967015146 learned_addr learned my addr 172.21.15.89:0/1967015146 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:36.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdbccdc700 1 --2- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb8135e50 0x7fcdb8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7f25d41c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25cc06a770 0x7f25cc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7f25d41c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06b080 0x7f25cc133940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7f25d41c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 0x7f25cc1342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7f25d41c4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25cc09a500 con 0x7f25cc06a770 2023-09-30T22:20:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7f25d41c4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc09a6e0 con 0x7f25cc06b080 2023-09-30T22:20:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7f25d41c4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc09e940 con 0x7f25cc133e80 2023-09-30T22:20:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.907+0000 7fcdb7fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcdb809d6f0 0x7fcdb809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55572/0 (socket says 172.21.15.89:55572) 2023-09-30T22:20:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdbccdc700 1 -- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb809e1d0 msgr2=0x7fcdb8135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdbccdc700 1 --2- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb809e1d0 0x7fcdb8135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdbccdc700 1 -- 172.21.15.89:0/1967015146 >> v1:172.21.15.89:6789/0 conn(0x7fcdb809d6f0 legacy=0x7fcdb809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdbccdc700 1 -- 172.21.15.89:0/1967015146 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb809b2d0 con 0x7fcdb8135e50 2023-09-30T22:20:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06b080 0x7f25cc133940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06b080 0x7f25cc133940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59618/0 (socket says 172.21.15.89:59618) 2023-09-30T22:20:36.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 -- 172.21.15.89:0/889222715 learned_addr learned my addr 172.21.15.89:0/889222715 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:36.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d1f60700 1 --2- 172.21.15.89:0/889222715 >> v2:172.21.15.89:3300/0 conn(0x7f25cc06a770 0x7f25cc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdb77fe700 1 --2- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb809e1d0 0x7fcdb8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:36.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdbccdc700 1 --2- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb8135e50 0x7fcdb8138230 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009dc0 tx=0x7fcdb000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb26ded41d464627 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:36.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 -- 172.21.15.89:0/889222715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 msgr2=0x7f25cc1342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:36.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 --2- 172.21.15.89:0/889222715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 0x7f25cc1342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 -- 172.21.15.89:0/889222715 >> v2:172.21.15.89:3300/0 conn(0x7f25cc06a770 msgr2=0x7f25cc06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:36.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 --2- 172.21.15.89:0/889222715 >> v2:172.21.15.89:3300/0 conn(0x7f25cc06a770 0x7f25cc06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 -- 172.21.15.89:0/889222715 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc09f220 con 0x7f25cc06b080 2023-09-30T22:20:36.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdb6ffd700 1 -- 172.21.15.89:0/1967015146 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000e070 con 0x7fcdb8135e50 2023-09-30T22:20:36.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7f25d175f700 1 --2- 172.21.15.89:0/889222715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06b080 0x7f25cc133940 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009920 tx=0x7f25bc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1971bbe48e08234 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:36.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdb6ffd700 1 -- 172.21.15.89:0/1967015146 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb000bb20 con 0x7fcdb8135e50 2023-09-30T22:20:36.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.908+0000 7fcdb6ffd700 1 -- 172.21.15.89:0/1967015146 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0012430 con 0x7fcdb8135e50 2023-09-30T22:20:36.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d0f5e700 1 -- 172.21.15.89:0/889222715 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc011830 con 0x7f25cc06b080 2023-09-30T22:20:36.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d0f5e700 1 -- 172.21.15.89:0/889222715 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc011990 con 0x7f25cc06b080 2023-09-30T22:20:36.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d0f5e700 1 -- 172.21.15.89:0/889222715 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc011d20 con 0x7f25cc06b080 2023-09-30T22:20:36.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7fcdbe73f700 1 -- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb8135e50 msgr2=0x7fcdb8138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7fcdbe73f700 1 --2- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb8135e50 0x7fcdb8138230 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009dc0 tx=0x7fcdb000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:36.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7fcdbe73f700 1 -- 172.21.15.89:0/1967015146 shutdown_connections 2023-09-30T22:20:36.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7fcdbe73f700 1 --2- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb8135e50 0x7fcdb8138230 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7fcdbe73f700 1 --2- 172.21.15.89:0/1967015146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb809e1d0 0x7fcdb8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7fcdbe73f700 1 -- 172.21.15.89:0/1967015146 >> 172.21.15.89:0/1967015146 conn(0x7fcdb8066f50 msgr2=0x7fcdb809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:36.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 -- 172.21.15.89:0/889222715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06b080 msgr2=0x7f25cc133940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 --2- 172.21.15.89:0/889222715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06b080 0x7f25cc133940 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009920 tx=0x7f25bc009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:36.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 -- 172.21.15.89:0/889222715 shutdown_connections 2023-09-30T22:20:36.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 --2- 172.21.15.89:0/889222715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 0x7f25cc1342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 --2- 172.21.15.89:0/889222715 >> v2:172.21.15.89:3300/0 conn(0x7f25cc06a770 0x7f25cc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 --2- 172.21.15.89:0/889222715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06b080 0x7f25cc133940 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 -- 172.21.15.89:0/889222715 >> 172.21.15.89:0/889222715 conn(0x7f25cc066d10 msgr2=0x7f25cc0a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:36.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7fcdbe73f700 1 -- 172.21.15.89:0/1967015146 shutdown_connections 2023-09-30T22:20:36.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7fcdbe73f700 1 -- 172.21.15.89:0/1967015146 wait complete. 2023-09-30T22:20:36.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 -- 172.21.15.89:0/889222715 shutdown_connections 2023-09-30T22:20:36.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.909+0000 7f25d41c4700 1 -- 172.21.15.89:0/889222715 wait complete. 2023-09-30T22:20:36.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.910+0000 7fcdbe73f700 1 Processor -- start 2023-09-30T22:20:36.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.910+0000 7f25d41c4700 1 Processor -- start 2023-09-30T22:20:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.910+0000 7f25d41c4700 1 -- start start 2023-09-30T22:20:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d41c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06a770 0x7f25cc122610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d41c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25cc06b080 0x7f25cc122b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d41c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 0x7f25cc123090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d41c4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc09c4f0 con 0x7f25cc06b080 2023-09-30T22:20:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d41c4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc124270 con 0x7f25cc06a770 2023-09-30T22:20:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d41c4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc125b40 con 0x7f25cc133e80 2023-09-30T22:20:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdbe73f700 1 -- start start 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdbe73f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb809d6f0 0x7fcdb811b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdbe73f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdb809e1d0 0x7fcdb811b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdbe73f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb8135e50 0x7fcdb811bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdbe73f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb811e870 con 0x7fcdb809e1d0 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdbe73f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb806b510 con 0x7fcdb809d6f0 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdbe73f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb811dbb0 con 0x7fcdb8135e50 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdb77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdb809e1d0 0x7fcdb811b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdb77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdb809e1d0 0x7fcdb811b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58736/0 (socket says 172.21.15.89:58736) 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7fcdb77fe700 1 -- 172.21.15.89:0/3053453511 learned_addr learned my addr 172.21.15.89:0/3053453511 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb7fff700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb809d6f0 0x7fcdb811b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdbccdc700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb8135e50 0x7fcdb811bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d1f60700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06a770 0x7f25cc122610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d1f60700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06a770 0x7f25cc122610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59620/0 (socket says 172.21.15.89:59620) 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d1f60700 1 -- 172.21.15.89:0/1630066076 learned_addr learned my addr 172.21.15.89:0/1630066076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.911+0000 7f25d2761700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 0x7f25cc123090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d2761700 1 -- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06a770 msgr2=0x7f25cc122610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d2761700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06a770 0x7f25cc122610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d175f700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25cc06b080 0x7f25cc122b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d2761700 1 -- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25cc06b080 msgr2=0x7f25cc122b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d2761700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25cc06b080 0x7f25cc122b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d2761700 1 -- 172.21.15.89:0/1630066076 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc009550 con 0x7f25cc133e80 2023-09-30T22:20:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d1f60700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06a770 0x7f25cc122610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb7fff700 1 -- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb8135e50 msgr2=0x7fcdb811bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb7fff700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb8135e50 0x7fcdb811bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb7fff700 1 -- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdb809e1d0 msgr2=0x7fcdb811b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb7fff700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdb809e1d0 0x7fcdb811b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb7fff700 1 -- 172.21.15.89:0/3053453511 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac009550 con 0x7fcdb809d6f0 2023-09-30T22:20:36.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb7fff700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb809d6f0 0x7fcdb811b350 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcda800c450 tx=0x7fcda800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:36.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d175f700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25cc06b080 0x7f25cc122b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:36.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7f25d2761700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 0x7f25cc123090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f25c400bdf0 tx=0x7f25c4009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:36.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb4ff9700 1 -- 172.21.15.89:0/3053453511 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8005cb0 con 0x7fcdb809d6f0 2023-09-30T22:20:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb4ff9700 1 -- 172.21.15.89:0/3053453511 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda8005e10 con 0x7fcdb809d6f0 2023-09-30T22:20:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.912+0000 7fcdb4ff9700 1 -- 172.21.15.89:0/3053453511 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda80072d0 con 0x7fcdb809d6f0 2023-09-30T22:20:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.913+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb0009a70 con 0x7fcdb809d6f0 2023-09-30T22:20:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.913+0000 7f25c27fc700 1 -- 172.21.15.89:0/1630066076 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400cc10 con 0x7f25cc133e80 2023-09-30T22:20:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.913+0000 7f25c27fc700 1 -- 172.21.15.89:0/1630066076 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400cdb0 con 0x7f25cc133e80 2023-09-30T22:20:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.913+0000 7f25c27fc700 1 -- 172.21.15.89:0/1630066076 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4014680 con 0x7f25cc133e80 2023-09-30T22:20:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.913+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc133580 con 0x7f25cc133e80 2023-09-30T22:20:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.913+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc126870 con 0x7f25cc133e80 2023-09-30T22:20:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.913+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb811c770 con 0x7fcdb809d6f0 2023-09-30T22:20:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.913+0000 7fcdb4ff9700 1 -- 172.21.15.89:0/3053453511 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcda8007430 con 0x7fcdb809d6f0 2023-09-30T22:20:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.914+0000 7fcdb4ff9700 1 -- 172.21.15.89:0/3053453511 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcda803eac0 con 0x7fcdb809d6f0 2023-09-30T22:20:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.914+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb811d850 con 0x7fcdb809d6f0 2023-09-30T22:20:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.914+0000 7f25c27fc700 1 -- 172.21.15.89:0/1630066076 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f25c40147e0 con 0x7f25cc133e80 2023-09-30T22:20:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.914+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc04f2a0 con 0x7f25cc133e80 2023-09-30T22:20:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.914+0000 7f25c27fc700 1 -- 172.21.15.89:0/1630066076 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f25c4006e40 con 0x7f25cc133e80 2023-09-30T22:20:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.919+0000 7fcdb4ff9700 1 -- 172.21.15.89:0/3053453511 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcdb811d850 con 0x7fcdb809d6f0 2023-09-30T22:20:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:36.919+0000 7f25c27fc700 1 -- 172.21.15.89:0/1630066076 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f25c401b8b0 con 0x7f25cc133e80 2023-09-30T22:20:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.077+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9f84074ea0 con 0x7f9f841338a0 2023-09-30T22:20:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.077+0000 7f9f79ffb700 1 -- 172.21.15.89:0/1345264997 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f7c035070 con 0x7f9f841338a0 2023-09-30T22:20:37.078 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.079+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f841338a0 msgr2=0x7f9f8411c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.079+0000 7f9f8b568700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f841338a0 0x7f9f8411c0f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c016000 tx=0x7f9f7c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.079+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 shutdown_connections 2023-09-30T22:20:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.079+0000 7f9f8b568700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f841338a0 0x7f9f8411c0f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.079+0000 7f9f8b568700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f8409de50 0x7f9f8411b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.079+0000 7f9f8b568700 1 --2- 172.21.15.89:0/1345264997 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8409e930 0x7f9f8411bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.079+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 >> 172.21.15.89:0/1345264997 conn(0x7f9f840a2aa0 msgr2=0x7f9f841305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.080+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 shutdown_connections 2023-09-30T22:20:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.080+0000 7f9f8b568700 1 -- 172.21.15.89:0/1345264997 wait complete. 2023-09-30T22:20:37.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:37.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.113+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f25cc0a2190 con 0x7f25cc133e80 2023-09-30T22:20:37.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.114+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcdb811d850 con 0x7fcdb809d6f0 2023-09-30T22:20:37.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.114+0000 7f25c27fc700 1 -- 172.21.15.89:0/1630066076 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25c4019020 con 0x7f25cc133e80 2023-09-30T22:20:37.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.114+0000 7fcdb4ff9700 1 -- 172.21.15.89:0/3053453511 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdb811d850 con 0x7fcdb809d6f0 2023-09-30T22:20:37.115 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:37.116 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:37.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.115+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feb2406c360 con 0x7feb2409d6f0 2023-09-30T22:20:37.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.115+0000 7feb217fa700 1 -- 172.21.15.89:0/2251533753 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb18015020 con 0x7feb2409d6f0 2023-09-30T22:20:37.117 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:37.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 msgr2=0x7f25cc123090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:37.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7f25d41c4700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 0x7f25cc123090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f25c400bdf0 tx=0x7f25c4009ed0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 shutdown_connections 2023-09-30T22:20:37.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7f25d41c4700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f25cc133e80 0x7f25cc123090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7f25d41c4700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f25cc06a770 0x7f25cc122610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7f25d41c4700 1 --2- 172.21.15.89:0/1630066076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25cc06b080 0x7f25cc122b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 >> 172.21.15.89:0/1630066076 conn(0x7f25cc066d10 msgr2=0x7f25cc06e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb809d6f0 msgr2=0x7fcdb811b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.116+0000 7fcdbe73f700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb809d6f0 0x7fcdb811b350 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcda800c450 tx=0x7fcda800c760 comp rx=0 tx=0).stop 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.117+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 shutdown_connections 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.117+0000 7fcdbe73f700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdb8135e50 0x7fcdb811bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.117+0000 7fcdbe73f700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdb809d6f0 0x7fcdb811b350 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.117+0000 7fcdbe73f700 1 --2- 172.21.15.89:0/3053453511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdb809e1d0 0x7fcdb811b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.117+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 >> 172.21.15.89:0/3053453511 conn(0x7fcdb8066f50 msgr2=0x7fcdb81339c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.117+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 shutdown_connections 2023-09-30T22:20:37.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.117+0000 7fcdbe73f700 1 -- 172.21.15.89:0/3053453511 wait complete. 2023-09-30T22:20:37.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 shutdown_connections 2023-09-30T22:20:37.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7f25d41c4700 1 -- 172.21.15.89:0/1630066076 wait complete. 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 msgr2=0x7feb24122600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb24122600 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feb18000c00 tx=0x7feb18004750 comp rx=0 tx=0).stop 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 shutdown_connections 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb2409e410 0x7feb24123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb2409d6f0 0x7feb24122600 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 --2- 172.21.15.89:0/2251533753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb2409e000 0x7feb24122b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 >> 172.21.15.89:0/2251533753 conn(0x7feb24066f50 msgr2=0x7feb240a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 shutdown_connections 2023-09-30T22:20:37.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:37.118+0000 7feb2aafb700 1 -- 172.21.15.89:0/2251533753 wait complete. 2023-09-30T22:20:37.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:37.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:37.134 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:38.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:38.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:38.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:38.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:38.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.243+0000 7f8531cb9700 1 Processor -- start 2023-09-30T22:20:38.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.243+0000 7f8531cb9700 1 -- start start 2023-09-30T22:20:38.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.243+0000 7f8531cb9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131630 0x7f852c131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f8531cb9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c06eba0 0x7f852c139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f8531cb9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c071a00 con 0x7f852c131f40 2023-09-30T22:20:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f8531cb9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c131430 con 0x7f852c06eba0 2023-09-30T22:20:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f8531cb9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f852c09a510 con 0x7f852c131630 2023-09-30T22:20:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131630 0x7f852c131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131630 0x7f852c131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55932/0 (socket says 172.21.15.89:55932) 2023-09-30T22:20:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852b7fe700 1 -- 172.21.15.89:0/3073560647 learned_addr learned my addr 172.21.15.89:0/3073560647 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852bfff700 1 --2- 172.21.15.89:0/3073560647 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c06eba0 0x7f852c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852b7fe700 1 -- 172.21.15.89:0/3073560647 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c06eba0 msgr2=0x7f852c139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852b7fe700 1 --2- 172.21.15.89:0/3073560647 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c06eba0 0x7f852c139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852b7fe700 1 -- 172.21.15.89:0/3073560647 >> v1:172.21.15.89:6789/0 conn(0x7f852c131f40 legacy=0x7f852c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852b7fe700 1 -- 172.21.15.89:0/3073560647 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0721c0 con 0x7f852c131630 2023-09-30T22:20:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852b7fe700 1 --2- 172.21.15.89:0/3073560647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131630 0x7f852c131a00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f851c009920 tx=0x7f851c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b716c803969c57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.244+0000 7f852a7fc700 1 -- 172.21.15.89:0/3073560647 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00ba60 con 0x7f852c131630 2023-09-30T22:20:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f852a7fc700 1 -- 172.21.15.89:0/3073560647 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00bbc0 con 0x7f852c131630 2023-09-30T22:20:38.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f852a7fc700 1 -- 172.21.15.89:0/3073560647 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c009e70 con 0x7f852c131630 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f8531cb9700 1 -- 172.21.15.89:0/3073560647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131630 msgr2=0x7f852c131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f8531cb9700 1 --2- 172.21.15.89:0/3073560647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131630 0x7f852c131a00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f851c009920 tx=0x7f851c009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f8531cb9700 1 -- 172.21.15.89:0/3073560647 shutdown_connections 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f8531cb9700 1 --2- 172.21.15.89:0/3073560647 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c06eba0 0x7f852c139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f8531cb9700 1 --2- 172.21.15.89:0/3073560647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131630 0x7f852c131a00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f8531cb9700 1 -- 172.21.15.89:0/3073560647 >> 172.21.15.89:0/3073560647 conn(0x7f852c0a2aa0 msgr2=0x7f852c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f8531cb9700 1 -- 172.21.15.89:0/3073560647 shutdown_connections 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.245+0000 7f8531cb9700 1 -- 172.21.15.89:0/3073560647 wait complete. 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.246+0000 7f8531cb9700 1 Processor -- start 2023-09-30T22:20:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.246+0000 7f8531cb9700 1 -- start start 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f8531cb9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852c06eba0 0x7f852c12e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f8531cb9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c131630 0x7f852c12e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f8531cb9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131f40 0x7f852c1c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f8531cb9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c12ca10 con 0x7f852c06eba0 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f8531cb9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c12bd30 con 0x7f852c131630 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f8531cb9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f852c12c140 con 0x7f852c131f40 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131f40 0x7f852c1c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131f40 0x7f852c1c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55948/0 (socket says 172.21.15.89:55948) 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852bfff700 1 -- 172.21.15.89:0/1493776363 learned_addr learned my addr 172.21.15.89:0/1493776363 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852b7fe700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852c06eba0 0x7f852c12e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852affd700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c131630 0x7f852c12e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852bfff700 1 -- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c131630 msgr2=0x7f852c12e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852bfff700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c131630 0x7f852c12e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852bfff700 1 -- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852c06eba0 msgr2=0x7f852c12e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852bfff700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852c06eba0 0x7f852c12e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852bfff700 1 -- 172.21.15.89:0/1493776363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009550 con 0x7f852c131f40 2023-09-30T22:20:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.247+0000 7f852affd700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c131630 0x7f852c12e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.248+0000 7f852bfff700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131f40 0x7f852c1c8100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8524007e40 tx=0x7f852400e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:38.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.248+0000 7f8513fff700 1 -- 172.21.15.89:0/1493776363 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400ede0 con 0x7f852c131f40 2023-09-30T22:20:38.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.248+0000 7f8513fff700 1 -- 172.21.15.89:0/1493776363 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8524014420 con 0x7f852c131f40 2023-09-30T22:20:38.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.248+0000 7f8513fff700 1 -- 172.21.15.89:0/1493776363 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240147d0 con 0x7f852c131f40 2023-09-30T22:20:38.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.248+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c12b140 con 0x7f852c131f40 2023-09-30T22:20:38.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.248+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c12d4b0 con 0x7f852c131f40 2023-09-30T22:20:38.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.249+0000 7f8513fff700 1 -- 172.21.15.89:0/1493776363 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8524014930 con 0x7f852c131f40 2023-09-30T22:20:38.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.249+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c074ea0 con 0x7f852c131f40 2023-09-30T22:20:38.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.249+0000 7f8513fff700 1 -- 172.21.15.89:0/1493776363 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f852403ff50 con 0x7f852c131f40 2023-09-30T22:20:38.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.258+0000 7f8513fff700 1 -- 172.21.15.89:0/1493776363 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8524014be0 con 0x7f852c131f40 2023-09-30T22:20:38.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.277+0000 7fec8a83c700 1 Processor -- start 2023-09-30T22:20:38.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.277+0000 7fec8a83c700 1 -- start start 2023-09-30T22:20:38.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.277+0000 7fec8a83c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec8406c0e0 0x7fec8409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec8a83c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fec841380f0 0x7fec8413fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec8a83c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec84134980 con 0x7fec84134ec0 2023-09-30T22:20:38.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec8a83c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8406c4b0 con 0x7fec841380f0 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec8a83c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec840a09c0 con 0x7fec8406c0e0 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec837fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fec84134ec0 0x7fec841379e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55596/0 (socket says 172.21.15.89:55596) 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec837fe700 1 -- 172.21.15.89:0/1059156700 learned_addr learned my addr 172.21.15.89:0/1059156700 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec88dd9700 1 --2- 172.21.15.89:0/1059156700 >> v2:172.21.15.89:3300/0 conn(0x7fec841380f0 0x7fec8413fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec83fff700 1 --2- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec8406c0e0 0x7fec8409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec88dd9700 1 -- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec8406c0e0 msgr2=0x7fec8409ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec88dd9700 1 --2- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec8406c0e0 0x7fec8409ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec88dd9700 1 -- 172.21.15.89:0/1059156700 >> v1:172.21.15.89:6789/0 conn(0x7fec84134ec0 legacy=0x7fec841379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec82ffd700 1 -- 172.21.15.89:0/1059156700 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911268518 0 0) 0x7fec84134980 con 0x7fec84134ec0 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec88dd9700 1 -- 172.21.15.89:0/1059156700 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840a0600 con 0x7fec841380f0 2023-09-30T22:20:38.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec83fff700 1 --2- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec8406c0e0 0x7fec8409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.278+0000 7fec88dd9700 1 --2- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec841380f0 0x7fec8413fd20 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009d90 tx=0x7fec7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951452f5fe564192 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.279+0000 7fec82ffd700 1 -- 172.21.15.89:0/1059156700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00e070 con 0x7fec841380f0 2023-09-30T22:20:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.279+0000 7fec82ffd700 1 -- 172.21.15.89:0/1059156700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c00bbd0 con 0x7fec841380f0 2023-09-30T22:20:38.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.279+0000 7fec82ffd700 1 -- 172.21.15.89:0/1059156700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c012430 con 0x7fec841380f0 2023-09-30T22:20:38.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.279+0000 7fec8a83c700 1 -- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec841380f0 msgr2=0x7fec8413fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.279+0000 7fec8a83c700 1 --2- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec841380f0 0x7fec8413fd20 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009d90 tx=0x7fec7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:38.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.280+0000 7fec8a83c700 1 -- 172.21.15.89:0/1059156700 shutdown_connections 2023-09-30T22:20:38.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.280+0000 7fec8a83c700 1 --2- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec841380f0 0x7fec8413fd20 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.280+0000 7fec8a83c700 1 --2- 172.21.15.89:0/1059156700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec8406c0e0 0x7fec8409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.280+0000 7fec8a83c700 1 -- 172.21.15.89:0/1059156700 >> 172.21.15.89:0/1059156700 conn(0x7fec840a4210 msgr2=0x7fec840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.280+0000 7fec8a83c700 1 -- 172.21.15.89:0/1059156700 shutdown_connections 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.280+0000 7fec8a83c700 1 -- 172.21.15.89:0/1059156700 wait complete. 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.280+0000 7fec8a83c700 1 Processor -- start 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec8a83c700 1 -- start start 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec8a83c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec8406c0e0 0x7fec8411a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec8a83c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec84134ec0 0x7fec8411abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec8a83c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec841380f0 0x7fec8411b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec8a83c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8411c820 con 0x7fec8406c0e0 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec8a83c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec840a0460 con 0x7fec84134ec0 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec8a83c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec84140e80 con 0x7fec841380f0 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec837fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec84134ec0 0x7fec8411abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec837fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec84134ec0 0x7fec8411abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59654/0 (socket says 172.21.15.89:59654) 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec88dd9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec841380f0 0x7fec8411b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.281+0000 7fec837fe700 1 -- 172.21.15.89:0/53464219 learned_addr learned my addr 172.21.15.89:0/53464219 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec837fe700 1 -- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec841380f0 msgr2=0x7fec8411b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec837fe700 1 --2- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec841380f0 0x7fec8411b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec837fe700 1 -- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec8406c0e0 msgr2=0x7fec8411a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec837fe700 1 --2- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec8406c0e0 0x7fec8411a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec837fe700 1 -- 172.21.15.89:0/53464219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c009a70 con 0x7fec84134ec0 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec88dd9700 1 --2- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec841380f0 0x7fec8411b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec837fe700 1 --2- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec84134ec0 0x7fec8411abc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fec7800c520 tx=0x7fec7800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec80ff9700 1 -- 172.21.15.89:0/53464219 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78004c80 con 0x7fec84134ec0 2023-09-30T22:20:38.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec80ff9700 1 -- 172.21.15.89:0/53464219 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec78004de0 con 0x7fec84134ec0 2023-09-30T22:20:38.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8411c030 con 0x7fec84134ec0 2023-09-30T22:20:38.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec80ff9700 1 -- 172.21.15.89:0/53464219 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78007310 con 0x7fec84134ec0 2023-09-30T22:20:38.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.282+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8411d6f0 con 0x7fec84134ec0 2023-09-30T22:20:38.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.283+0000 7fec80ff9700 1 -- 172.21.15.89:0/53464219 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fec78007470 con 0x7fec84134ec0 2023-09-30T22:20:38.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.283+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8411d8e0 con 0x7fec84134ec0 2023-09-30T22:20:38.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.283+0000 7fec80ff9700 1 -- 172.21.15.89:0/53464219 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fec78040d80 con 0x7fec84134ec0 2023-09-30T22:20:38.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.287+0000 7ff738276700 1 Processor -- start 2023-09-30T22:20:38.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.287+0000 7ff738276700 1 -- start start 2023-09-30T22:20:38.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7fec80ff9700 1 -- 172.21.15.89:0/53464219 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec78044060 con 0x7fec84134ec0 2023-09-30T22:20:38.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7ff738276700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff73009de00 0x7ff73009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7ff738276700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73013d1f0 0x7ff73013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7ff738276700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73009a810 con 0x7ff73006a770 2023-09-30T22:20:38.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7ff738276700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73006ab40 con 0x7ff73009de00 2023-09-30T22:20:38.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7ff738276700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73009ebc0 con 0x7ff73013d1f0 2023-09-30T22:20:38.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7ff736813700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73013d1f0 0x7ff73013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7ff736813700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73013d1f0 0x7ff73013f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59664/0 (socket says 172.21.15.89:59664) 2023-09-30T22:20:38.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.288+0000 7ff736813700 1 -- 172.21.15.89:0/3880867249 learned_addr learned my addr 172.21.15.89:0/3880867249 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:38.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff735811700 1 --2- 172.21.15.89:0/3880867249 >> v2:172.21.15.89:3300/0 conn(0x7ff73009de00 0x7ff73009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7f3fdfb45700 1 Processor -- start 2023-09-30T22:20:38.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff736813700 1 -- 172.21.15.89:0/3880867249 >> v2:172.21.15.89:3300/0 conn(0x7ff73009de00 msgr2=0x7ff73009e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:38.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff736813700 1 --2- 172.21.15.89:0/3880867249 >> v2:172.21.15.89:3300/0 conn(0x7ff73009de00 0x7ff73009e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff736813700 1 -- 172.21.15.89:0/3880867249 >> v1:172.21.15.89:6789/0 conn(0x7ff73006a770 legacy=0x7ff73009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:38.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff736813700 1 -- 172.21.15.89:0/3880867249 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73009e920 con 0x7ff73013d1f0 2023-09-30T22:20:38.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff736813700 1 --2- 172.21.15.89:0/3880867249 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73013d1f0 0x7ff73013f5d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff728009e50 tx=0x7ff72800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84f7bef2b6559d88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:38.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff735010700 1 -- 172.21.15.89:0/3880867249 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800e070 con 0x7ff73013d1f0 2023-09-30T22:20:38.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff735010700 1 -- 172.21.15.89:0/3880867249 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff728003a20 con 0x7ff73013d1f0 2023-09-30T22:20:38.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7ff735010700 1 -- 172.21.15.89:0/3880867249 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728003dd0 con 0x7ff73013d1f0 2023-09-30T22:20:38.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.289+0000 7f3fdfb45700 1 -- start start 2023-09-30T22:20:38.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdfb45700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd809e1d0 0x7f3fd8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdfb45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd8135e50 0x7f3fd8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdfb45700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd809f130 con 0x7f3fd809d6f0 2023-09-30T22:20:38.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdfb45700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd809f520 con 0x7f3fd809e1d0 2023-09-30T22:20:38.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdfb45700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd809e930 con 0x7f3fd8135e50 2023-09-30T22:20:38.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdd0e0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd809e1d0 0x7f3fd8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdd8e1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3fd809d6f0 0x7f3fd809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55612/0 (socket says 172.21.15.89:55612) 2023-09-30T22:20:38.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdd0e0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd809e1d0 0x7f3fd8135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59668/0 (socket says 172.21.15.89:59668) 2023-09-30T22:20:38.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdd8e1700 1 -- 172.21.15.89:0/1585768736 learned_addr learned my addr 172.21.15.89:0/1585768736 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:38.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7ff738276700 1 -- 172.21.15.89:0/3880867249 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73013d1f0 msgr2=0x7ff73013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7ff738276700 1 --2- 172.21.15.89:0/3880867249 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73013d1f0 0x7ff73013f5d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff728009e50 tx=0x7ff72800b330 comp rx=0 tx=0).stop 2023-09-30T22:20:38.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdc8df700 1 -- 172.21.15.89:0/1585768736 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742642275 0 0) 0x7f3fd809f130 con 0x7f3fd809d6f0 2023-09-30T22:20:38.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdc8df700 1 -- 172.21.15.89:0/1585768736 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc40057c0 con 0x7f3fd809d6f0 2023-09-30T22:20:38.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdd0e0700 1 -- 172.21.15.89:0/1585768736 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd8135e50 msgr2=0x7f3fd8138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:38.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdd0e0700 1 --2- 172.21.15.89:0/1585768736 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd8135e50 0x7f3fd8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdd0e0700 1 -- 172.21.15.89:0/1585768736 >> v1:172.21.15.89:6789/0 conn(0x7f3fd809d6f0 legacy=0x7f3fd809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.290+0000 7f3fdd0e0700 1 -- 172.21.15.89:0/1585768736 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd809b2d0 con 0x7f3fd809e1d0 2023-09-30T22:20:38.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7f3fdd0e0700 1 --2- 172.21.15.89:0/1585768736 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd809e1d0 0x7f3fd8135910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009870 tx=0x7f3fc8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6d735665a8c6067 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:38.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7ff738276700 1 -- 172.21.15.89:0/3880867249 shutdown_connections 2023-09-30T22:20:38.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7ff738276700 1 --2- 172.21.15.89:0/3880867249 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73013d1f0 0x7ff73013f5d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7ff738276700 1 --2- 172.21.15.89:0/3880867249 >> v2:172.21.15.89:3300/0 conn(0x7ff73009de00 0x7ff73009e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7ff738276700 1 -- 172.21.15.89:0/3880867249 >> 172.21.15.89:0/3880867249 conn(0x7ff730066f50 msgr2=0x7ff73009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:38.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7ff738276700 1 -- 172.21.15.89:0/3880867249 shutdown_connections 2023-09-30T22:20:38.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7ff738276700 1 -- 172.21.15.89:0/3880867249 wait complete. 2023-09-30T22:20:38.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7f3fdc8df700 1 -- 172.21.15.89:0/1585768736 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8011830 con 0x7f3fd809e1d0 2023-09-30T22:20:38.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7f3fdc8df700 1 -- 172.21.15.89:0/1585768736 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc8011990 con 0x7f3fd809e1d0 2023-09-30T22:20:38.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.291+0000 7f3fdc8df700 1 -- 172.21.15.89:0/1585768736 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8011d20 con 0x7f3fd809e1d0 2023-09-30T22:20:38.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff738276700 1 Processor -- start 2023-09-30T22:20:38.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff738276700 1 -- start start 2023-09-30T22:20:38.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7f3fdfb45700 1 -- 172.21.15.89:0/1585768736 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd809e1d0 msgr2=0x7f3fd8135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7f3fdfb45700 1 --2- 172.21.15.89:0/1585768736 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd809e1d0 0x7f3fd8135910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009870 tx=0x7f3fc8009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:38.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7f3fdfb45700 1 -- 172.21.15.89:0/1585768736 shutdown_connections 2023-09-30T22:20:38.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7f3fdfb45700 1 --2- 172.21.15.89:0/1585768736 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd8135e50 0x7f3fd8138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7f3fdfb45700 1 --2- 172.21.15.89:0/1585768736 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd809e1d0 0x7f3fd8135910 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7f3fdfb45700 1 -- 172.21.15.89:0/1585768736 >> 172.21.15.89:0/1585768736 conn(0x7f3fd8066f50 msgr2=0x7f3fd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:38.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7f3fdfb45700 1 -- 172.21.15.89:0/1585768736 shutdown_connections 2023-09-30T22:20:38.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7f3fdfb45700 1 -- 172.21.15.89:0/1585768736 wait complete. 2023-09-30T22:20:38.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff738276700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73006a770 0x7ff7301260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff738276700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73009de00 0x7ff7301265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff738276700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73013d1f0 0x7ff730126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff738276700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730128510 con 0x7ff73006a770 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff738276700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73013c770 con 0x7ff73009de00 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff738276700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff73009f1a0 con 0x7ff73013d1f0 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff735811700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73009de00 0x7ff7301265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff735811700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73009de00 0x7ff7301265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59676/0 (socket says 172.21.15.89:59676) 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff735811700 1 -- 172.21.15.89:0/116771472 learned_addr learned my addr 172.21.15.89:0/116771472 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.292+0000 7ff736012700 1 --2- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73006a770 0x7ff7301260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff735811700 1 -- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73013d1f0 msgr2=0x7ff730126b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff735811700 1 --2- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73013d1f0 0x7ff730126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff735811700 1 -- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73006a770 msgr2=0x7ff7301260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff735811700 1 --2- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73006a770 0x7ff7301260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff735811700 1 -- 172.21.15.89:0/116771472 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff728009b30 con 0x7ff73009de00 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff735811700 1 --2- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73009de00 0x7ff7301265e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff72000e810 tx=0x7ff72000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff726ffd700 1 -- 172.21.15.89:0/116771472 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720013610 con 0x7ff73009de00 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff726ffd700 1 -- 172.21.15.89:0/116771472 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720013770 con 0x7ff73009de00 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff726ffd700 1 -- 172.21.15.89:0/116771472 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720013b20 con 0x7ff73009de00 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7300a3940 con 0x7ff73009de00 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7301281c0 con 0x7ff73009de00 2023-09-30T22:20:38.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.294+0000 7ff726ffd700 1 -- 172.21.15.89:0/116771472 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff720013c80 con 0x7ff73009de00 2023-09-30T22:20:38.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7f3fdfb45700 1 Processor -- start 2023-09-30T22:20:38.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.293+0000 7f3fdfb45700 1 -- start start 2023-09-30T22:20:38.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.294+0000 7f3fdfb45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fd809d6f0 0x7f3fd81196e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.294+0000 7f3fdd8e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fd809d6f0 0x7f3fd81196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fdfb45700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd8135e50 0x7f3fd8119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fdd8e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fd809d6f0 0x7f3fd81196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58786/0 (socket says 172.21.15.89:58786) 2023-09-30T22:20:38.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7ff726ffd700 1 -- 172.21.15.89:0/116771472 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff72003f2d0 con 0x7ff73009de00 2023-09-30T22:20:38.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7180052e0 con 0x7ff73009de00 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fdfb45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd811a160 0x7f3fd812ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fdfb45700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd811b110 con 0x7f3fd809d6f0 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fdfb45700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd811b9a0 con 0x7f3fd8135e50 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fdfb45700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd811b300 con 0x7f3fd811a160 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd811a160 0x7f3fd812ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd811a160 0x7f3fd812ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56012/0 (socket says 172.21.15.89:56012) 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 -- 172.21.15.89:0/118124026 learned_addr learned my addr 172.21.15.89:0/118124026 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fdd0e0700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd8135e50 0x7f3fd8119c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 -- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd8135e50 msgr2=0x7f3fd8119c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd8135e50 0x7f3fd8119c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 -- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fd809d6f0 msgr2=0x7f3fd81196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fd809d6f0 0x7f3fd81196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 -- 172.21.15.89:0/118124026 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80094a0 con 0x7f3fd811a160 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.295+0000 7f3fde0e2700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd811a160 0x7f3fd812ef60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000ba40 tx=0x7f3fd000bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.297+0000 7f3fdd8e1700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fd809d6f0 0x7f3fd81196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.297+0000 7f3fce7fc700 1 -- 172.21.15.89:0/118124026 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000d8e0 con 0x7f3fd811a160 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.297+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd811c800 con 0x7f3fd811a160 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.297+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd809c700 con 0x7f3fd811a160 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.297+0000 7f3fce7fc700 1 -- 172.21.15.89:0/118124026 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd000da40 con 0x7f3fd811a160 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.297+0000 7f3fce7fc700 1 -- 172.21.15.89:0/118124026 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000ddb0 con 0x7f3fd811a160 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.297+0000 7f3fce7fc700 1 -- 172.21.15.89:0/118124026 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3fd0015420 con 0x7f3fd811a160 2023-09-30T22:20:38.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.298+0000 7f3fce7fc700 1 -- 172.21.15.89:0/118124026 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3fd0011070 con 0x7f3fd811a160 2023-09-30T22:20:38.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.298+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd8074ea0 con 0x7f3fd811a160 2023-09-30T22:20:38.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.304+0000 7ff726ffd700 1 -- 172.21.15.89:0/116771472 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff720012370 con 0x7ff73009de00 2023-09-30T22:20:38.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.307+0000 7f3fce7fc700 1 -- 172.21.15.89:0/118124026 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3fd001c900 con 0x7f3fd811a160 2023-09-30T22:20:38.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.464+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f852c12c6b0 con 0x7f852c131f40 2023-09-30T22:20:38.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.465+0000 7f8513fff700 1 -- 172.21.15.89:0/1493776363 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f852401b310 con 0x7f852c131f40 2023-09-30T22:20:38.466 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:38.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131f40 msgr2=0x7f852c1c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131f40 0x7f852c1c8100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8524007e40 tx=0x7f852400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 shutdown_connections 2023-09-30T22:20:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852c131f40 0x7f852c1c8100 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852c06eba0 0x7f852c12e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 --2- 172.21.15.89:0/1493776363 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852c131630 0x7f852c12e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 >> 172.21.15.89:0/1493776363 conn(0x7f852c0a2aa0 msgr2=0x7f852c130220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 shutdown_connections 2023-09-30T22:20:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.467+0000 7f8531cb9700 1 -- 172.21.15.89:0/1493776363 wait complete. 2023-09-30T22:20:38.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.491+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fec84074ea0 con 0x7fec84134ec0 2023-09-30T22:20:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.492+0000 7fec80ff9700 1 -- 172.21.15.89:0/53464219 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec78044030 con 0x7fec84134ec0 2023-09-30T22:20:38.493 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec84134ec0 msgr2=0x7fec8411abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 --2- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec84134ec0 0x7fec8411abc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fec7800c520 tx=0x7fec7800c830 comp rx=0 tx=0).stop 2023-09-30T22:20:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 shutdown_connections 2023-09-30T22:20:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 --2- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec841380f0 0x7fec8411b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 --2- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec8406c0e0 0x7fec8411a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 --2- 172.21.15.89:0/53464219 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec84134ec0 0x7fec8411abc0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 >> 172.21.15.89:0/53464219 conn(0x7fec840a4210 msgr2=0x7fec841376d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 shutdown_connections 2023-09-30T22:20:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.494+0000 7fec8a83c700 1 -- 172.21.15.89:0/53464219 wait complete. 2023-09-30T22:20:38.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:38.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.531+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff718005590 con 0x7ff73009de00 2023-09-30T22:20:38.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.531+0000 7ff726ffd700 1 -- 172.21.15.89:0/116771472 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff720021020 con 0x7ff73009de00 2023-09-30T22:20:38.532 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:38.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.533+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73009de00 msgr2=0x7ff7301265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.533+0000 7ff738276700 1 --2- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73009de00 0x7ff7301265e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff72000e810 tx=0x7ff72000eb20 comp rx=0 tx=0).stop 2023-09-30T22:20:38.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.534+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 shutdown_connections 2023-09-30T22:20:38.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.534+0000 7ff738276700 1 --2- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73013d1f0 0x7ff730126b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.534+0000 7ff738276700 1 --2- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73006a770 0x7ff7301260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.534+0000 7ff738276700 1 --2- 172.21.15.89:0/116771472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73009de00 0x7ff7301265e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.534+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 >> 172.21.15.89:0/116771472 conn(0x7ff730066f50 msgr2=0x7ff7300a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:38.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.534+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 shutdown_connections 2023-09-30T22:20:38.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.534+0000 7ff738276700 1 -- 172.21.15.89:0/116771472 wait complete. 2023-09-30T22:20:38.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.535+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3fd806c360 con 0x7f3fd811a160 2023-09-30T22:20:38.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.535+0000 7f3fce7fc700 1 -- 172.21.15.89:0/118124026 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fd001d360 con 0x7f3fd811a160 2023-09-30T22:20:38.536 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:38.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.537+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd811a160 msgr2=0x7f3fd812ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:38.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.537+0000 7f3fdfb45700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd811a160 0x7f3fd812ef60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000ba40 tx=0x7f3fd000bd50 comp rx=0 tx=0).stop 2023-09-30T22:20:38.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.538+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 shutdown_connections 2023-09-30T22:20:38.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.538+0000 7f3fdfb45700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fd811a160 0x7f3fd812ef60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.538+0000 7f3fdfb45700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fd809d6f0 0x7f3fd81196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.538+0000 7f3fdfb45700 1 --2- 172.21.15.89:0/118124026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fd8135e50 0x7f3fd8119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:38.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.538+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 >> 172.21.15.89:0/118124026 conn(0x7f3fd8066f50 msgr2=0x7f3fd8132ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:38.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.538+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 shutdown_connections 2023-09-30T22:20:38.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:38.538+0000 7f3fdfb45700 1 -- 172.21.15.89:0/118124026 wait complete. 2023-09-30T22:20:38.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:38.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:39.481 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:39.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:39.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:39.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.620+0000 7fd0d839d700 1 Processor -- start 2023-09-30T22:20:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.620+0000 7fd0d839d700 1 -- start start 2023-09-30T22:20:39.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d839d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd0d01338a0 0x7fd0d0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d839d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009de50 0x7fd0d009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d839d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d006ef30 con 0x7fd0d009e760 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d839d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d006fef0 con 0x7fd0d01338a0 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d839d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01334b0 con 0x7fd0d009de50 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d5938700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009de50 0x7fd0d009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d5938700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009de50 0x7fd0d009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59696/0 (socket says 172.21.15.89:59696) 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d5938700 1 -- 172.21.15.89:0/2919167905 learned_addr learned my addr 172.21.15.89:0/2919167905 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d6139700 1 --2- 172.21.15.89:0/2919167905 >> v2:172.21.15.89:3300/0 conn(0x7fd0d01338a0 0x7fd0d0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d5938700 1 -- 172.21.15.89:0/2919167905 >> v2:172.21.15.89:3300/0 conn(0x7fd0d01338a0 msgr2=0x7fd0d0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d5938700 1 --2- 172.21.15.89:0/2919167905 >> v2:172.21.15.89:3300/0 conn(0x7fd0d01338a0 0x7fd0d0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d5938700 1 -- 172.21.15.89:0/2919167905 >> v1:172.21.15.89:6789/0 conn(0x7fd0d009e760 legacy=0x7fd0d013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.621+0000 7fd0d5938700 1 -- 172.21.15.89:0/2919167905 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d006fc30 con 0x7fd0d009de50 2023-09-30T22:20:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.622+0000 7fd0d6139700 1 --2- 172.21.15.89:0/2919167905 >> v2:172.21.15.89:3300/0 conn(0x7fd0d01338a0 0x7fd0d0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.622+0000 7fd0d5938700 1 --2- 172.21.15.89:0/2919167905 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009de50 0x7fd0d009e220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00098f0 tx=0x7fd0c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=448a69b22419f05b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.622+0000 7fd0d5137700 1 -- 172.21.15.89:0/2919167905 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000b8a0 con 0x7fd0d009de50 2023-09-30T22:20:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.622+0000 7fd0d5137700 1 -- 172.21.15.89:0/2919167905 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c000ba00 con 0x7fd0d009de50 2023-09-30T22:20:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.622+0000 7fd0d5137700 1 -- 172.21.15.89:0/2919167905 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000bd90 con 0x7fd0d009de50 2023-09-30T22:20:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.623+0000 7fd0d839d700 1 -- 172.21.15.89:0/2919167905 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009de50 msgr2=0x7fd0d009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.623+0000 7fd0d839d700 1 --2- 172.21.15.89:0/2919167905 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009de50 0x7fd0d009e220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00098f0 tx=0x7fd0c0009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.623+0000 7fd0d839d700 1 -- 172.21.15.89:0/2919167905 shutdown_connections 2023-09-30T22:20:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.623+0000 7fd0d839d700 1 --2- 172.21.15.89:0/2919167905 >> v2:172.21.15.89:3300/0 conn(0x7fd0d01338a0 0x7fd0d0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.623+0000 7fd0d839d700 1 --2- 172.21.15.89:0/2919167905 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009de50 0x7fd0d009e220 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.623+0000 7fd0d839d700 1 -- 172.21.15.89:0/2919167905 >> 172.21.15.89:0/2919167905 conn(0x7fd0d00a2aa0 msgr2=0x7fd0d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.623+0000 7fd0d839d700 1 -- 172.21.15.89:0/2919167905 shutdown_connections 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.623+0000 7fd0d839d700 1 -- 172.21.15.89:0/2919167905 wait complete. 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d839d700 1 Processor -- start 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d839d700 1 -- start start 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d839d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d009de50 0x7fd0d011b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d839d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009e760 0x7fd0d011bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d839d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0d01338a0 0x7fd0d011c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d839d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d011f330 con 0x7fd0d009de50 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d839d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d011e700 con 0x7fd0d009e760 2023-09-30T22:20:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d839d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d006f640 con 0x7fd0d01338a0 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d6139700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d009de50 0x7fd0d011b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d6139700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d009de50 0x7fd0d011b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58802/0 (socket says 172.21.15.89:58802) 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d6139700 1 -- 172.21.15.89:0/2431185959 learned_addr learned my addr 172.21.15.89:0/2431185959 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d693a700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0d01338a0 0x7fd0d011c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.624+0000 7fd0d5938700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009e760 0x7fd0d011bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0d6139700 1 -- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0d01338a0 msgr2=0x7fd0d011c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0d6139700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0d01338a0 0x7fd0d011c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0d6139700 1 -- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009e760 msgr2=0x7fd0d011bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0d6139700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009e760 0x7fd0d011bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0d6139700 1 -- 172.21.15.89:0/2431185959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c0009550 con 0x7fd0d009de50 2023-09-30T22:20:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0d6139700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d009de50 0x7fd0d011b810 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00e840 tx=0x7fd0cc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0c6ffd700 1 -- 172.21.15.89:0/2431185959 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc013610 con 0x7fd0d009de50 2023-09-30T22:20:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0c6ffd700 1 -- 172.21.15.89:0/2431185959 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc013770 con 0x7fd0d009de50 2023-09-30T22:20:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d011da00 con 0x7fd0d009de50 2023-09-30T22:20:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0c6ffd700 1 -- 172.21.15.89:0/2431185959 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc013b00 con 0x7fd0d009de50 2023-09-30T22:20:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.625+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d012ec00 con 0x7fd0d009de50 2023-09-30T22:20:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.626+0000 7fd0c6ffd700 1 -- 172.21.15.89:0/2431185959 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd0cc013c60 con 0x7fd0d009de50 2023-09-30T22:20:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.626+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d004ea50 con 0x7fd0d009de50 2023-09-30T22:20:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.626+0000 7fd0c6ffd700 1 -- 172.21.15.89:0/2431185959 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd0cc03f390 con 0x7fd0d009de50 2023-09-30T22:20:39.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.635+0000 7fd0c6ffd700 1 -- 172.21.15.89:0/2431185959 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd0cc012370 con 0x7fd0d009de50 2023-09-30T22:20:39.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3de024700 1 Processor -- start 2023-09-30T22:20:39.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3de024700 1 -- start start 2023-09-30T22:20:39.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3de024700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d81302b0 0x7fc3d80700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3de024700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3d812cd80 0x7fc3d8137b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3de024700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d8130090 con 0x7fc3d812c260 2023-09-30T22:20:39.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3de024700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d809a3a0 con 0x7fc3d812cd80 2023-09-30T22:20:39.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3de024700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d8070610 con 0x7fc3d81302b0 2023-09-30T22:20:39.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3d7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3d812cd80 0x7fc3d8137b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3d7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3d812cd80 0x7fc3d8137b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58812/0 (socket says 172.21.15.89:58812) 2023-09-30T22:20:39.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3d7fff700 1 -- 172.21.15.89:0/4021659023 learned_addr learned my addr 172.21.15.89:0/4021659023 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:39.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.664+0000 7fc3d77fe700 1 --2- 172.21.15.89:0/4021659023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d81302b0 0x7fc3d80700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d77fe700 1 -- 172.21.15.89:0/4021659023 >> v2:172.21.15.89:3300/0 conn(0x7fc3d812cd80 msgr2=0x7fc3d8137b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:39.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d67fc700 1 -- 172.21.15.89:0/4021659023 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242821033 0 0) 0x7fc3d8130090 con 0x7fc3d812c260 2023-09-30T22:20:39.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d77fe700 1 --2- 172.21.15.89:0/4021659023 >> v2:172.21.15.89:3300/0 conn(0x7fc3d812cd80 0x7fc3d8137b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d77fe700 1 -- 172.21.15.89:0/4021659023 >> v1:172.21.15.89:6789/0 conn(0x7fc3d812c260 legacy=0x7fc3d812c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d77fe700 1 -- 172.21.15.89:0/4021659023 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8099c70 con 0x7fc3d81302b0 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d7fff700 1 --2- 172.21.15.89:0/4021659023 >> v2:172.21.15.89:3300/0 conn(0x7fc3d812cd80 0x7fc3d8137b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d77fe700 1 --2- 172.21.15.89:0/4021659023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d81302b0 0x7fc3d80700d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ce969b38c985ef server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d67fc700 1 -- 172.21.15.89:0/4021659023 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8011830 con 0x7fc3d81302b0 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d67fc700 1 -- 172.21.15.89:0/4021659023 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8011990 con 0x7fc3d81302b0 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.665+0000 7fc3d67fc700 1 -- 172.21.15.89:0/4021659023 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8011d20 con 0x7fc3d81302b0 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.666+0000 7fc3de024700 1 -- 172.21.15.89:0/4021659023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d81302b0 msgr2=0x7fc3d80700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.666+0000 7fc3de024700 1 --2- 172.21.15.89:0/4021659023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d81302b0 0x7fc3d80700d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.666+0000 7fc3de024700 1 -- 172.21.15.89:0/4021659023 shutdown_connections 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.666+0000 7fc3de024700 1 --2- 172.21.15.89:0/4021659023 >> v2:172.21.15.89:3300/0 conn(0x7fc3d812cd80 0x7fc3d8137b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.666+0000 7fc3de024700 1 --2- 172.21.15.89:0/4021659023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d81302b0 0x7fc3d80700d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.666+0000 7fc3de024700 1 -- 172.21.15.89:0/4021659023 >> 172.21.15.89:0/4021659023 conn(0x7fc3d80a27f0 msgr2=0x7fc3d80a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:39.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.666+0000 7fc3de024700 1 -- 172.21.15.89:0/4021659023 shutdown_connections 2023-09-30T22:20:39.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.666+0000 7fc3de024700 1 -- 172.21.15.89:0/4021659023 wait complete. 2023-09-30T22:20:39.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.667+0000 7fc3de024700 1 Processor -- start 2023-09-30T22:20:39.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.668+0000 7fc3de024700 1 -- start start 2023-09-30T22:20:39.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.668+0000 7fc3de024700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d812c260 0x7fc3d809c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.668+0000 7fc3de024700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3d812cd80 0x7fc3d809cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.668+0000 7fc3de024700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3d81302b0 0x7fc3d81c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.668+0000 7fc3de024700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d809f4e0 con 0x7fc3d81302b0 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.668+0000 7fc3de024700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d81296b0 con 0x7fc3d812cd80 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.668+0000 7fc3de024700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d809c3b0 con 0x7fc3d812c260 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3d81302b0 0x7fc3d81c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3d81302b0 0x7fc3d81c3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58818/0 (socket says 172.21.15.89:58818) 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 -- 172.21.15.89:0/3838916864 learned_addr learned my addr 172.21.15.89:0/3838916864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d77fe700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d812c260 0x7fc3d809c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 -- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d812c260 msgr2=0x7fc3d809c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d812c260 0x7fc3d809c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 -- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3d812cd80 msgr2=0x7fc3d809cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3d812cd80 0x7fc3d809cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 -- 172.21.15.89:0/3838916864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d000bd40 con 0x7fc3d81302b0 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d77fe700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d812c260 0x7fc3d809c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3d7fff700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3d81302b0 0x7fc3d81c3c10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0007270 tx=0x7fc3d00106a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3bffff700 1 -- 172.21.15.89:0/3838916864 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0017410 con 0x7fc3d81302b0 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3bffff700 1 -- 172.21.15.89:0/3838916864 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0011040 con 0x7fc3d81302b0 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c80094a0 con 0x7fc3d81302b0 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.669+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d80a0510 con 0x7fc3d81302b0 2023-09-30T22:20:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.670+0000 7fc3bffff700 1 -- 172.21.15.89:0/3838916864 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0017790 con 0x7fc3d81302b0 2023-09-30T22:20:39.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.671+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b80052e0 con 0x7fc3d81302b0 2023-09-30T22:20:39.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.671+0000 7fc3bffff700 1 -- 172.21.15.89:0/3838916864 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc3d00178f0 con 0x7fc3d81302b0 2023-09-30T22:20:39.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.671+0000 7fc3bffff700 1 -- 172.21.15.89:0/3838916864 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc3d0042170 con 0x7fc3d81302b0 2023-09-30T22:20:39.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.680+0000 7fc3bffff700 1 -- 172.21.15.89:0/3838916864 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3d0017ba0 con 0x7fc3d81302b0 2023-09-30T22:20:39.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f616d6fb700 1 Processor -- start 2023-09-30T22:20:39.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f616d6fb700 1 -- start start 2023-09-30T22:20:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7fea6106d700 1 Processor -- start 2023-09-30T22:20:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7fea6106d700 1 -- start start 2023-09-30T22:20:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f616d6fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f616d6fb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f616812f8b0 0x7f616812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f616d6fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6168130260 0x7f6168136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f616d6fb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616812d330 con 0x7f616812f8b0 2023-09-30T22:20:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f616d6fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616812cd90 con 0x7f616812cf60 2023-09-30T22:20:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f616d6fb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61681306d0 con 0x7f6168130260 2023-09-30T22:20:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f61667fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f616812f8b0 0x7f616812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f61667fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f616812f8b0 0x7f616812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58832/0 (socket says 172.21.15.89:58832) 2023-09-30T22:20:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f61667fc700 1 -- 172.21.15.89:0/1177665785 learned_addr learned my addr 172.21.15.89:0/1177665785 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.700+0000 7f61677fe700 1 --2- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6168130260 0x7f6168136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f6166ffd700 1 --2- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea6106d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea5c131630 0x7fea5c131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea6106d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c131f40 0x7fea5c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea6106d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c06e9d0 0x7fea5c06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea6106d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5c071a00 con 0x7fea5c131630 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea6106d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c131430 con 0x7fea5c06e9d0 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea6106d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c09a510 con 0x7fea5c131f40 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f61677fe700 1 -- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 msgr2=0x7f616806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f61677fe700 1 --2- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f61677fe700 1 -- 172.21.15.89:0/1177665785 >> v2:172.21.15.89:3300/0 conn(0x7f616812f8b0 msgr2=0x7f616812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f61677fe700 1 --2- 172.21.15.89:0/1177665785 >> v2:172.21.15.89:3300/0 conn(0x7f616812f8b0 0x7f616812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f61677fe700 1 -- 172.21.15.89:0/1177665785 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168071780 con 0x7f6168130260 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f61667fc700 1 --2- 172.21.15.89:0/1177665785 >> v2:172.21.15.89:3300/0 conn(0x7f616812f8b0 0x7f616812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f61677fe700 1 --2- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6168130260 0x7f6168136c20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6160009dc0 tx=0x7f616000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3708af225a220339 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea5c131630 0x7fea5c131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea5c131630 0x7fea5c131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58848/0 (socket says 172.21.15.89:58848) 2023-09-30T22:20:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 -- 172.21.15.89:0/585262065 learned_addr learned my addr 172.21.15.89:0/585262065 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5b59e700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c06e9d0 0x7fea5c06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5a59c700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c131f40 0x7fea5c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 -- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c131f40 msgr2=0x7fea5c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c131f40 0x7fea5c06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 -- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c06e9d0 msgr2=0x7fea5c06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c06e9d0 0x7fea5c06ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 -- 172.21.15.89:0/585262065 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c0721c0 con 0x7fea5c131630 2023-09-30T22:20:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5a59c700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c131f40 0x7fea5c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f6165ffb700 1 -- 172.21.15.89:0/1177665785 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000e070 con 0x7f6168130260 2023-09-30T22:20:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f6165ffb700 1 -- 172.21.15.89:0/1177665785 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616000bae0 con 0x7f6168130260 2023-09-30T22:20:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7f6165ffb700 1 -- 172.21.15.89:0/1177665785 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000be70 con 0x7f6168130260 2023-09-30T22:20:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.701+0000 7fea5ad9d700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 0x7fea5c131a00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0098f0 tx=0x7fea4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f691cfd1e4a0ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7fea59d9b700 1 -- 172.21.15.89:0/585262065 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c010a90 con 0x7fea5c131630 2023-09-30T22:20:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7fea59d9b700 1 -- 172.21.15.89:0/585262065 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c010bf0 con 0x7fea5c131630 2023-09-30T22:20:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7fea59d9b700 1 -- 172.21.15.89:0/585262065 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c018440 con 0x7fea5c131630 2023-09-30T22:20:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 -- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6168130260 msgr2=0x7f6168136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 --2- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6168130260 0x7f6168136c20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6160009dc0 tx=0x7f616000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 -- 172.21.15.89:0/1177665785 shutdown_connections 2023-09-30T22:20:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 --2- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6168130260 0x7f6168136c20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 --2- 172.21.15.89:0/1177665785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 --2- 172.21.15.89:0/1177665785 >> v2:172.21.15.89:3300/0 conn(0x7f616812f8b0 0x7f616812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 -- 172.21.15.89:0/1177665785 >> 172.21.15.89:0/1177665785 conn(0x7f61680a2aa0 msgr2=0x7f61680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 -- 172.21.15.89:0/1177665785 shutdown_connections 2023-09-30T22:20:39.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.702+0000 7f616d6fb700 1 -- 172.21.15.89:0/1177665785 wait complete. 2023-09-30T22:20:39.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7fea6106d700 1 -- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 msgr2=0x7fea5c131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7fea6106d700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 0x7fea5c131a00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0098f0 tx=0x7fea4c009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:39.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7fea6106d700 1 -- 172.21.15.89:0/585262065 shutdown_connections 2023-09-30T22:20:39.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7fea6106d700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c06e9d0 0x7fea5c06ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7fea6106d700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 0x7fea5c131a00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7fea6106d700 1 --2- 172.21.15.89:0/585262065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c131f40 0x7fea5c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7fea6106d700 1 -- 172.21.15.89:0/585262065 >> 172.21.15.89:0/585262065 conn(0x7fea5c0a2aa0 msgr2=0x7fea5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:39.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7f616d6fb700 1 Processor -- start 2023-09-30T22:20:39.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7fea6106d700 1 -- 172.21.15.89:0/585262065 shutdown_connections 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.703+0000 7f616d6fb700 1 -- start start 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f616d6fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616809bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f616d6fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f616812f8b0 0x7f616809c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f616d6fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6168130260 0x7f61681c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f616d6fb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6168129900 con 0x7f6168130260 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f616d6fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616809b610 con 0x7f616812cf60 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f616d6fb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f616809e650 con 0x7f616812f8b0 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6168130260 0x7f61681c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f6166ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616809bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6168130260 0x7f61681c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58856/0 (socket says 172.21.15.89:58856) 2023-09-30T22:20:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f6166ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616809bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59734/0 (socket says 172.21.15.89:59734) 2023-09-30T22:20:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61677fe700 1 -- 172.21.15.89:0/2914003264 learned_addr learned my addr 172.21.15.89:0/2914003264 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61667fc700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f616812f8b0 0x7f616809c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7fea6106d700 1 -- 172.21.15.89:0/585262065 wait complete. 2023-09-30T22:20:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61677fe700 1 -- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f616812f8b0 msgr2=0x7f616809c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61677fe700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f616812f8b0 0x7f616809c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61677fe700 1 -- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 msgr2=0x7f616809bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61677fe700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616809bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61677fe700 1 -- 172.21.15.89:0/2914003264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c009550 con 0x7f6168130260 2023-09-30T22:20:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f61667fc700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f616812f8b0 0x7f616809c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:39.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.704+0000 7f6166ffd700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616809bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:39.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.705+0000 7f61677fe700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6168130260 0x7f61681c3fb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6160013000 tx=0x7f6160004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:39.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.705+0000 7f614f7fe700 1 -- 172.21.15.89:0/2914003264 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160004680 con 0x7f6168130260 2023-09-30T22:20:39.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.705+0000 7f614f7fe700 1 -- 172.21.15.89:0/2914003264 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6160004800 con 0x7f6168130260 2023-09-30T22:20:39.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.705+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6160009a70 con 0x7f6168130260 2023-09-30T22:20:39.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.705+0000 7f614f7fe700 1 -- 172.21.15.89:0/2914003264 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160004970 con 0x7f6168130260 2023-09-30T22:20:39.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.705+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6168129fa0 con 0x7f6168130260 2023-09-30T22:20:39.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.706+0000 7fea6106d700 1 Processor -- start 2023-09-30T22:20:39.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.706+0000 7fea6106d700 1 -- start start 2023-09-30T22:20:39.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.706+0000 7fea6106d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c06e9d0 0x7fea5c1d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.706+0000 7fea6106d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 0x7fea5c1d5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.706+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616809e870 con 0x7f6168130260 2023-09-30T22:20:39.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.706+0000 7f614f7fe700 1 -- 172.21.15.89:0/2914003264 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6160012670 con 0x7f6168130260 2023-09-30T22:20:39.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7f614f7fe700 1 -- 172.21.15.89:0/2914003264 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6160041760 con 0x7f6168130260 2023-09-30T22:20:39.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.706+0000 7fea6106d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c131f40 0x7fea5c1d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:39.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.706+0000 7fea6106d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c11ecb0 con 0x7fea5c131630 2023-09-30T22:20:39.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea6106d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c11ded0 con 0x7fea5c131f40 2023-09-30T22:20:39.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea6106d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c11e2b0 con 0x7fea5c06e9d0 2023-09-30T22:20:39.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c06e9d0 0x7fea5c1d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c06e9d0 0x7fea5c1d51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56078/0 (socket says 172.21.15.89:56078) 2023-09-30T22:20:39.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 -- 172.21.15.89:0/2305216529 learned_addr learned my addr 172.21.15.89:0/2305216529 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:39.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5a59c700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 0x7fea5c1d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 -- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c131f40 msgr2=0x7fea5c1d1650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c131f40 0x7fea5c1d1650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 -- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 msgr2=0x7fea5c1d5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 0x7fea5c1d5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 -- 172.21.15.89:0/2305216529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c009550 con 0x7fea5c06e9d0 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5a59c700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 0x7fea5c1d5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea5ad9d700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c06e9d0 0x7fea5c1d51e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea4c01b040 tx=0x7fea4c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.707+0000 7fea437fe700 1 -- 172.21.15.89:0/2305216529 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0038b0 con 0x7fea5c06e9d0 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.708+0000 7fea437fe700 1 -- 172.21.15.89:0/2305216529 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c003a30 con 0x7fea5c06e9d0 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.708+0000 7fea437fe700 1 -- 172.21.15.89:0/2305216529 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c003de0 con 0x7fea5c06e9d0 2023-09-30T22:20:39.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.708+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c11e9f0 con 0x7fea5c06e9d0 2023-09-30T22:20:39.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.708+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c11d260 con 0x7fea5c06e9d0 2023-09-30T22:20:39.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.709+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5c11eef0 con 0x7fea5c06e9d0 2023-09-30T22:20:39.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.709+0000 7fea437fe700 1 -- 172.21.15.89:0/2305216529 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea4c018440 con 0x7fea5c06e9d0 2023-09-30T22:20:39.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.709+0000 7fea437fe700 1 -- 172.21.15.89:0/2305216529 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea4c040c50 con 0x7fea5c06e9d0 2023-09-30T22:20:39.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.714+0000 7f614f7fe700 1 -- 172.21.15.89:0/2914003264 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6160030470 con 0x7f6168130260 2023-09-30T22:20:39.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.716+0000 7fea437fe700 1 -- 172.21.15.89:0/2305216529 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea4c016520 con 0x7fea5c06e9d0 2023-09-30T22:20:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.865+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd0d0074ea0 con 0x7fd0d009de50 2023-09-30T22:20:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.865+0000 7fd0c6ffd700 1 -- 172.21.15.89:0/2431185959 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd0cc02a090 con 0x7fd0d009de50 2023-09-30T22:20:39.866 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.867+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d009de50 msgr2=0x7fd0d011b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.867+0000 7fd0d839d700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d009de50 0x7fd0d011b810 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00e840 tx=0x7fd0cc00eb50 comp rx=0 tx=0).stop 2023-09-30T22:20:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.868+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 shutdown_connections 2023-09-30T22:20:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.868+0000 7fd0d839d700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0d01338a0 0x7fd0d011c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.868+0000 7fd0d839d700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d009de50 0x7fd0d011b810 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.868+0000 7fd0d839d700 1 --2- 172.21.15.89:0/2431185959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0d009e760 0x7fd0d011bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.868+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 >> 172.21.15.89:0/2431185959 conn(0x7fd0d00a2aa0 msgr2=0x7fd0d0130610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.868+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 shutdown_connections 2023-09-30T22:20:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.868+0000 7fd0d839d700 1 -- 172.21.15.89:0/2431185959 wait complete. 2023-09-30T22:20:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.877+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc3b8005590 con 0x7fc3d81302b0 2023-09-30T22:20:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.877+0000 7fc3bffff700 1 -- 172.21.15.89:0/3838916864 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc3d0014b10 con 0x7fc3d81302b0 2023-09-30T22:20:39.878 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3d81302b0 msgr2=0x7fc3d81c3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3d81302b0 0x7fc3d81c3c10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0007270 tx=0x7fc3d00106a0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 shutdown_connections 2023-09-30T22:20:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3d81302b0 0x7fc3d81c3c10 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc3d812c260 0x7fc3d809c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 --2- 172.21.15.89:0/3838916864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc3d812cd80 0x7fc3d809cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 >> 172.21.15.89:0/3838916864 conn(0x7fc3d80a27f0 msgr2=0x7fc3d806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:39.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 shutdown_connections 2023-09-30T22:20:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.880+0000 7fc3de024700 1 -- 172.21.15.89:0/3838916864 wait complete. 2023-09-30T22:20:39.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:39.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.909+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fea5c11d260 con 0x7fea5c06e9d0 2023-09-30T22:20:39.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.910+0000 7fea437fe700 1 -- 172.21.15.89:0/2305216529 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea5c11d260 con 0x7fea5c06e9d0 2023-09-30T22:20:39.910 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:39.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.911+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c06e9d0 msgr2=0x7fea5c1d51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.911+0000 7fea6106d700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c06e9d0 0x7fea5c1d51e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea4c01b040 tx=0x7fea4c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.911+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 shutdown_connections 2023-09-30T22:20:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.911+0000 7fea6106d700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea5c131f40 0x7fea5c1d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.912+0000 7fea6106d700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea5c06e9d0 0x7fea5c1d51e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.912+0000 7fea6106d700 1 --2- 172.21.15.89:0/2305216529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea5c131630 0x7fea5c1d5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.912+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 >> 172.21.15.89:0/2305216529 conn(0x7fea5c0a2aa0 msgr2=0x7fea5c070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.912+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 shutdown_connections 2023-09-30T22:20:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.912+0000 7fea6106d700 1 -- 172.21.15.89:0/2305216529 wait complete. 2023-09-30T22:20:39.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:39.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.936+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6168074ea0 con 0x7f6168130260 2023-09-30T22:20:39.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.936+0000 7f614f7fe700 1 -- 172.21.15.89:0/2914003264 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6160035090 con 0x7f6168130260 2023-09-30T22:20:39.937 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6168130260 msgr2=0x7f61681c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6168130260 0x7f61681c3fb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6160013000 tx=0x7f6160004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 shutdown_connections 2023-09-30T22:20:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6168130260 0x7f61681c3fb0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f616812cf60 0x7f616809bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 --2- 172.21.15.89:0/2914003264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f616812f8b0 0x7f616809c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:39.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 >> 172.21.15.89:0/2914003264 conn(0x7f61680a2aa0 msgr2=0x7f61681b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:39.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 shutdown_connections 2023-09-30T22:20:39.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:39.938+0000 7f616d6fb700 1 -- 172.21.15.89:0/2914003264 wait complete. 2023-09-30T22:20:39.950 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:40.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:40.893 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:40.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:40.952 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:41.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.033+0000 7f774e12e700 1 Processor -- start 2023-09-30T22:20:41.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.033+0000 7f774e12e700 1 -- start start 2023-09-30T22:20:41.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f774e12e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f774809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f774e12e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f774809e000 0x7f77481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f774e12e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774809b120 con 0x7f774809e410 2023-09-30T22:20:41.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f774e12e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774806b460 con 0x7f774809e000 2023-09-30T22:20:41.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f774e12e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774809a7e0 con 0x7f774809d6f0 2023-09-30T22:20:41.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f77477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f774809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f77477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f774809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60456/0 (socket says 172.21.15.89:60456) 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f7746ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f774809e000 0x7f77481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.034+0000 7f77477fe700 1 -- 172.21.15.89:0/3171400570 learned_addr learned my addr 172.21.15.89:0/3171400570 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f7746ffd700 1 -- 172.21.15.89:0/3171400570 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 msgr2=0x7f774809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f7746ffd700 1 --2- 172.21.15.89:0/3171400570 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f774809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f7746ffd700 1 -- 172.21.15.89:0/3171400570 >> v1:172.21.15.89:6789/0 conn(0x7f774809e410 legacy=0x7f7748133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f7746ffd700 1 -- 172.21.15.89:0/3171400570 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774809a500 con 0x7f774809e000 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f77477fe700 1 --2- 172.21.15.89:0/3171400570 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f774809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f7746ffd700 1 --2- 172.21.15.89:0/3171400570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 0x7f77481339f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f773c009840 tx=0x7f773c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de8d6ebab4affae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f77467fc700 1 -- 172.21.15.89:0/3171400570 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c011830 con 0x7f774809e000 2023-09-30T22:20:41.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f77467fc700 1 -- 172.21.15.89:0/3171400570 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c011990 con 0x7f774809e000 2023-09-30T22:20:41.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.035+0000 7f77467fc700 1 -- 172.21.15.89:0/3171400570 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c011d20 con 0x7f774809e000 2023-09-30T22:20:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.036+0000 7f774e12e700 1 -- 172.21.15.89:0/3171400570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 msgr2=0x7f77481339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.036+0000 7f774e12e700 1 --2- 172.21.15.89:0/3171400570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 0x7f77481339f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f773c009840 tx=0x7f773c009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.036+0000 7f774e12e700 1 -- 172.21.15.89:0/3171400570 shutdown_connections 2023-09-30T22:20:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.036+0000 7f774e12e700 1 --2- 172.21.15.89:0/3171400570 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f774809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.036+0000 7f774e12e700 1 --2- 172.21.15.89:0/3171400570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 0x7f77481339f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.036+0000 7f774e12e700 1 -- 172.21.15.89:0/3171400570 >> 172.21.15.89:0/3171400570 conn(0x7f7748066f50 msgr2=0x7f774809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.036+0000 7f774e12e700 1 -- 172.21.15.89:0/3171400570 shutdown_connections 2023-09-30T22:20:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.036+0000 7f774e12e700 1 -- 172.21.15.89:0/3171400570 wait complete. 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.037+0000 7f774e12e700 1 Processor -- start 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f774e12e700 1 -- start start 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f774e12e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f77481a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f774e12e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 0x7f77481a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f774e12e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f774809e410 0x7f77481a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f774e12e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77481a91f0 con 0x7f774809e000 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f774e12e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774809c710 con 0x7f774809d6f0 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f774e12e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77480a3160 con 0x7f774809e410 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f77477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f77481a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.038+0000 7f77477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f77481a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60462/0 (socket says 172.21.15.89:60462) 2023-09-30T22:20:41.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f77477fe700 1 -- 172.21.15.89:0/461976933 learned_addr learned my addr 172.21.15.89:0/461976933 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f7746ffd700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 0x7f77481a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f77477fe700 1 -- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f774809e410 msgr2=0x7f77481a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f77477fe700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f774809e410 0x7f77481a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f77477fe700 1 -- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 msgr2=0x7f77481a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f77477fe700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 0x7f77481a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f77477fe700 1 -- 172.21.15.89:0/461976933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738009550 con 0x7f774809d6f0 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f7746ffd700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 0x7f77481a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f77477fe700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f77481a5fd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f773800ebe0 tx=0x7f773800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f772ffff700 1 -- 172.21.15.89:0/461976933 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800c850 con 0x7f774809d6f0 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f772ffff700 1 -- 172.21.15.89:0/461976933 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773800c9b0 con 0x7f774809d6f0 2023-09-30T22:20:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f772ffff700 1 -- 172.21.15.89:0/461976933 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800cd20 con 0x7f774809d6f0 2023-09-30T22:20:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c0094a0 con 0x7f774809d6f0 2023-09-30T22:20:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.039+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77481a8da0 con 0x7f774809d6f0 2023-09-30T22:20:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.040+0000 7f772ffff700 1 -- 172.21.15.89:0/461976933 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7738013470 con 0x7f774809d6f0 2023-09-30T22:20:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.041+0000 7f772ffff700 1 -- 172.21.15.89:0/461976933 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f773803f850 con 0x7f774809d6f0 2023-09-30T22:20:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.041+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7748074ea0 con 0x7f774809d6f0 2023-09-30T22:20:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.049+0000 7f772ffff700 1 -- 172.21.15.89:0/461976933 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f77380232f0 con 0x7f774809d6f0 2023-09-30T22:20:41.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.062+0000 7f6f506c3700 1 Processor -- start 2023-09-30T22:20:41.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.062+0000 7f6f506c3700 1 -- start start 2023-09-30T22:20:41.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.062+0000 7f6f506c3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f4809e1d0 0x7f6f48138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.062+0000 7f6f506c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f48139170 0x7f6f4813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.062+0000 7f6f506c3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4809b120 con 0x7f6f4809d6f0 2023-09-30T22:20:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.062+0000 7f6f506c3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4806b460 con 0x7f6f4809e1d0 2023-09-30T22:20:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.062+0000 7f6f506c3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4809a7e0 con 0x7f6f48139170 2023-09-30T22:20:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4dc5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f4809e1d0 0x7f6f48138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4dc5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f4809e1d0 0x7f6f48138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60470/0 (socket says 172.21.15.89:60470) 2023-09-30T22:20:41.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4dc5e700 1 -- 172.21.15.89:0/1220492295 learned_addr learned my addr 172.21.15.89:0/1220492295 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4ec60700 1 --2- 172.21.15.89:0/1220492295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f48139170 0x7f6f4813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4dc5e700 1 -- 172.21.15.89:0/1220492295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f48139170 msgr2=0x7f6f4813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4dc5e700 1 --2- 172.21.15.89:0/1220492295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f48139170 0x7f6f4813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4d45d700 1 -- 172.21.15.89:0/1220492295 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063918401 0 0) 0x7f6f4809b120 con 0x7f6f4809d6f0 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4dc5e700 1 -- 172.21.15.89:0/1220492295 >> v1:172.21.15.89:6789/0 conn(0x7f6f4809d6f0 legacy=0x7f6f4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4dc5e700 1 -- 172.21.15.89:0/1220492295 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4809a500 con 0x7f6f4809e1d0 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4dc5e700 1 --2- 172.21.15.89:0/1220492295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f4809e1d0 0x7f6f48138c30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009920 tx=0x7f6f38009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dba95c1e291bae20 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4d45d700 1 -- 172.21.15.89:0/1220492295 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38011830 con 0x7f6f4809e1d0 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4d45d700 1 -- 172.21.15.89:0/1220492295 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38011990 con 0x7f6f4809e1d0 2023-09-30T22:20:41.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.063+0000 7f6f4d45d700 1 -- 172.21.15.89:0/1220492295 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38011d20 con 0x7f6f4809e1d0 2023-09-30T22:20:41.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.064+0000 7f6f506c3700 1 -- 172.21.15.89:0/1220492295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f4809e1d0 msgr2=0x7f6f48138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.064+0000 7f6f506c3700 1 --2- 172.21.15.89:0/1220492295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f4809e1d0 0x7f6f48138c30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009920 tx=0x7f6f38009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.064+0000 7f6f506c3700 1 -- 172.21.15.89:0/1220492295 shutdown_connections 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.064+0000 7f6f506c3700 1 --2- 172.21.15.89:0/1220492295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f48139170 0x7f6f4813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.064+0000 7f6f506c3700 1 --2- 172.21.15.89:0/1220492295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f4809e1d0 0x7f6f48138c30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.064+0000 7f6f506c3700 1 -- 172.21.15.89:0/1220492295 >> 172.21.15.89:0/1220492295 conn(0x7f6f48066f50 msgr2=0x7f6f4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.064+0000 7f6f506c3700 1 -- 172.21.15.89:0/1220492295 shutdown_connections 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.064+0000 7f6f506c3700 1 -- 172.21.15.89:0/1220492295 wait complete. 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.065+0000 7f6f506c3700 1 Processor -- start 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.065+0000 7f6f506c3700 1 -- start start 2023-09-30T22:20:41.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f506c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f4809d6f0 0x7f6f4811e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f506c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f4809e1d0 0x7f6f4811ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f506c3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f48139170 0x7f6f4811f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f506c3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f481221d0 con 0x7f6f4809d6f0 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f506c3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f481345a0 con 0x7f6f48139170 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f506c3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4809c710 con 0x7f6f4809e1d0 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4e45f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f4809d6f0 0x7f6f4811e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4dc5e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f4809e1d0 0x7f6f4811ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4e45f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f4809d6f0 0x7f6f4811e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36516/0 (socket says 172.21.15.89:36516) 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4dc5e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f4809e1d0 0x7f6f4811ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59640/0 (socket says 172.21.15.89:59640) 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4e45f700 1 -- 172.21.15.89:0/1000621540 learned_addr learned my addr 172.21.15.89:0/1000621540 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:41.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4ec60700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f48139170 0x7f6f4811f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4ec60700 1 -- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f4809e1d0 msgr2=0x7f6f4811ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4ec60700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f4809e1d0 0x7f6f4811ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4ec60700 1 -- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f4809d6f0 msgr2=0x7f6f4811e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4ec60700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f4809d6f0 0x7f6f4811e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4ec60700 1 -- 172.21.15.89:0/1000621540 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f38009550 con 0x7f6f48139170 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4e45f700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f4809d6f0 0x7f6f4811e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f4ec60700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f48139170 0x7f6f4811f130 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000efc0 tx=0x7f6f4000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.066+0000 7f6f3effd700 1 -- 172.21.15.89:0/1000621540 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000cc90 con 0x7f6f48139170 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.067+0000 7f6f3effd700 1 -- 172.21.15.89:0/1000621540 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4000f040 con 0x7f6f48139170 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.067+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f48120360 con 0x7f6f48139170 2023-09-30T22:20:41.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.067+0000 7f6f3effd700 1 -- 172.21.15.89:0/1000621540 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f400146c0 con 0x7f6f48139170 2023-09-30T22:20:41.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.067+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f48122960 con 0x7f6f48139170 2023-09-30T22:20:41.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.067+0000 7f6f3effd700 1 -- 172.21.15.89:0/1000621540 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f40014820 con 0x7f6f48139170 2023-09-30T22:20:41.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.068+0000 7f6f3effd700 1 -- 172.21.15.89:0/1000621540 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6f4001c570 con 0x7f6f48139170 2023-09-30T22:20:41.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.068+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f48121cf0 con 0x7f6f48139170 2023-09-30T22:20:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.072+0000 7f6f3effd700 1 -- 172.21.15.89:0/1000621540 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f40018020 con 0x7f6f48139170 2023-09-30T22:20:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.085+0000 7fae5b869700 1 Processor -- start 2023-09-30T22:20:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.085+0000 7fae5b869700 1 -- start start 2023-09-30T22:20:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae5b869700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae54133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae5b869700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae5b869700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5406ef30 con 0x7fae5409e760 2023-09-30T22:20:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae5b869700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5406fef0 con 0x7fae5409de50 2023-09-30T22:20:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae5b869700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae541334b0 con 0x7fae541338a0 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae59605700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae54133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae59605700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae54133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59644/0 (socket says 172.21.15.89:59644) 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae59605700 1 -- 172.21.15.89:0/1035581830 learned_addr learned my addr 172.21.15.89:0/1035581830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.086+0000 7fae58e04700 1 --2- 172.21.15.89:0/1035581830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.087+0000 7fae59605700 1 -- 172.21.15.89:0/1035581830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 msgr2=0x7fae5409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.087+0000 7fae59605700 1 --2- 172.21.15.89:0/1035581830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.087+0000 7fae59605700 1 -- 172.21.15.89:0/1035581830 >> v1:172.21.15.89:6789/0 conn(0x7fae5409e760 legacy=0x7fae5413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.087+0000 7fae59605700 1 -- 172.21.15.89:0/1035581830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5406fc30 con 0x7fae541338a0 2023-09-30T22:20:41.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.087+0000 7fae59605700 1 --2- 172.21.15.89:0/1035581830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae54133c70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fae50009950 tx=0x7fae50009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5952d2b833c6b50e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.087+0000 7fae4bfff700 1 -- 172.21.15.89:0/1035581830 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50009ea0 con 0x7fae541338a0 2023-09-30T22:20:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.087+0000 7fae4bfff700 1 -- 172.21.15.89:0/1035581830 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5000ba20 con 0x7fae541338a0 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.087+0000 7fae4bfff700 1 -- 172.21.15.89:0/1035581830 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5000bdb0 con 0x7fae541338a0 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.088+0000 7fae5b869700 1 -- 172.21.15.89:0/1035581830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 msgr2=0x7fae54133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.088+0000 7fae5b869700 1 --2- 172.21.15.89:0/1035581830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae54133c70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fae50009950 tx=0x7fae50009c60 comp rx=0 tx=0).stop 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.088+0000 7fae5b869700 1 -- 172.21.15.89:0/1035581830 shutdown_connections 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.088+0000 7fae5b869700 1 --2- 172.21.15.89:0/1035581830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae54133c70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.088+0000 7fae5b869700 1 --2- 172.21.15.89:0/1035581830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.088+0000 7fae5b869700 1 -- 172.21.15.89:0/1035581830 >> 172.21.15.89:0/1035581830 conn(0x7fae540a2aa0 msgr2=0x7fae540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.088+0000 7fae5b869700 1 -- 172.21.15.89:0/1035581830 shutdown_connections 2023-09-30T22:20:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.088+0000 7fae5b869700 1 -- 172.21.15.89:0/1035581830 wait complete. 2023-09-30T22:20:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.089+0000 7fae5b869700 1 Processor -- start 2023-09-30T22:20:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae5b869700 1 -- start start 2023-09-30T22:20:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae5b869700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5411eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae5b869700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5409e760 0x7fae5411f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae5b869700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae5411f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae5b869700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5412f650 con 0x7fae5409e760 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae5b869700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae54122040 con 0x7fae5409de50 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae5b869700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae541225e0 con 0x7fae541338a0 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae58e04700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5409e760 0x7fae5411f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae58e04700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5409e760 0x7fae5411f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36528/0 (socket says 172.21.15.89:36528) 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae58e04700 1 -- 172.21.15.89:0/3207487895 learned_addr learned my addr 172.21.15.89:0/3207487895 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.090+0000 7fae59e06700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae5411f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae59605700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5411eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae59605700 1 -- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 msgr2=0x7fae5411f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae59605700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae5411f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae59605700 1 -- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5409e760 msgr2=0x7fae5411f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae59605700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5409e760 0x7fae5411f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae59605700 1 -- 172.21.15.89:0/3207487895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50009550 con 0x7fae5409de50 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae58e04700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5409e760 0x7fae5411f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae59605700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5411eac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fae50016000 tx=0x7fae50004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.091+0000 7fae49ffb700 1 -- 172.21.15.89:0/3207487895 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50009ea0 con 0x7fae5409de50 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.092+0000 7fae49ffb700 1 -- 172.21.15.89:0/3207487895 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae500036a0 con 0x7fae5409de50 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.092+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae540a10a0 con 0x7fae5409de50 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.092+0000 7fae49ffb700 1 -- 172.21.15.89:0/3207487895 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50003a50 con 0x7fae5409de50 2023-09-30T22:20:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.092+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae54122300 con 0x7fae5409de50 2023-09-30T22:20:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.093+0000 7fae49ffb700 1 -- 172.21.15.89:0/3207487895 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae50003bb0 con 0x7fae5409de50 2023-09-30T22:20:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.093+0000 7fae49ffb700 1 -- 172.21.15.89:0/3207487895 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fae50040be0 con 0x7fae5409de50 2023-09-30T22:20:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.093+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae54074ea0 con 0x7fae5409de50 2023-09-30T22:20:41.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.098+0000 7fae49ffb700 1 -- 172.21.15.89:0/3207487895 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae50015020 con 0x7fae5409de50 2023-09-30T22:20:41.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.106+0000 7fd15c321700 1 Processor -- start 2023-09-30T22:20:41.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.106+0000 7fd15c321700 1 -- start start 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd15c321700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15406a770 0x7fd15409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd15c321700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd15409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd15c321700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15409a810 con 0x7fd15413c1e0 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd15c321700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15406ab40 con 0x7fd15409dc30 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd15c321700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd15409ebc0 con 0x7fd15406a770 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd1598bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd15409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd1598bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd15409e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60492/0 (socket says 172.21.15.89:60492) 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd1598bc700 1 -- 172.21.15.89:0/2249225545 learned_addr learned my addr 172.21.15.89:0/2249225545 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:41.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd15a0bd700 1 --2- 172.21.15.89:0/2249225545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15406a770 0x7fd15409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd1598bc700 1 -- 172.21.15.89:0/2249225545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15406a770 msgr2=0x7fd15409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd1598bc700 1 --2- 172.21.15.89:0/2249225545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15406a770 0x7fd15409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd1598bc700 1 -- 172.21.15.89:0/2249225545 >> v1:172.21.15.89:6789/0 conn(0x7fd15413c1e0 legacy=0x7fd15413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.107+0000 7fd1598bc700 1 -- 172.21.15.89:0/2249225545 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15409e920 con 0x7fd15409dc30 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd1598bc700 1 --2- 172.21.15.89:0/2249225545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd15409e080 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd144009920 tx=0x7fd144009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e364777224446b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd1590bb700 1 -- 172.21.15.89:0/2249225545 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144011830 con 0x7fd15409dc30 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd1590bb700 1 -- 172.21.15.89:0/2249225545 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144011990 con 0x7fd15409dc30 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd1590bb700 1 -- 172.21.15.89:0/2249225545 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144011d20 con 0x7fd15409dc30 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd15c321700 1 -- 172.21.15.89:0/2249225545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 msgr2=0x7fd15409e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd15c321700 1 --2- 172.21.15.89:0/2249225545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd15409e080 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd144009920 tx=0x7fd144009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd15c321700 1 -- 172.21.15.89:0/2249225545 shutdown_connections 2023-09-30T22:20:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd15c321700 1 --2- 172.21.15.89:0/2249225545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15406a770 0x7fd15409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd15c321700 1 --2- 172.21.15.89:0/2249225545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd15409e080 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd15c321700 1 -- 172.21.15.89:0/2249225545 >> 172.21.15.89:0/2249225545 conn(0x7fd154066f50 msgr2=0x7fd15409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.108+0000 7fd15c321700 1 -- 172.21.15.89:0/2249225545 shutdown_connections 2023-09-30T22:20:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.109+0000 7fd15c321700 1 -- 172.21.15.89:0/2249225545 wait complete. 2023-09-30T22:20:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.109+0000 7fd15c321700 1 Processor -- start 2023-09-30T22:20:41.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.109+0000 7fd15c321700 1 -- start start 2023-09-30T22:20:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd15c321700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd15406a770 0x7fd154126ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd15c321700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd154127410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd15c321700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15413c1e0 0x7fd1541cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd15c321700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd154129220 con 0x7fd15406a770 2023-09-30T22:20:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd15c321700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15413afb0 con 0x7fd15409dc30 2023-09-30T22:20:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd15c321700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd154126b10 con 0x7fd15413c1e0 2023-09-30T22:20:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd1598bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd154127410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd1598bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd154127410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60504/0 (socket says 172.21.15.89:60504) 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd1598bc700 1 -- 172.21.15.89:0/2234563471 learned_addr learned my addr 172.21.15.89:0/2234563471 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd15a0bd700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd15406a770 0x7fd154126ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd15a8be700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15413c1e0 0x7fd1541cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd1598bc700 1 -- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15413c1e0 msgr2=0x7fd1541cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd1598bc700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15413c1e0 0x7fd1541cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd1598bc700 1 -- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd15406a770 msgr2=0x7fd154126ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd1598bc700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd15406a770 0x7fd154126ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.110+0000 7fd1598bc700 1 -- 172.21.15.89:0/2234563471 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144009550 con 0x7fd15409dc30 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.111+0000 7fd1598bc700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd154127410 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd144000c00 tx=0x7fd144004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:41.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.111+0000 7fd14affd700 1 -- 172.21.15.89:0/2234563471 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144004450 con 0x7fd15409dc30 2023-09-30T22:20:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.111+0000 7fd14affd700 1 -- 172.21.15.89:0/2234563471 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1440045b0 con 0x7fd15409dc30 2023-09-30T22:20:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.111+0000 7fd14affd700 1 -- 172.21.15.89:0/2234563471 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440038c0 con 0x7fd15409dc30 2023-09-30T22:20:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.111+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1540724f0 con 0x7fd15409dc30 2023-09-30T22:20:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.111+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15412a8b0 con 0x7fd15409dc30 2023-09-30T22:20:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.112+0000 7fd14affd700 1 -- 172.21.15.89:0/2234563471 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd144003a20 con 0x7fd15409dc30 2023-09-30T22:20:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.112+0000 7fd14affd700 1 -- 172.21.15.89:0/2234563471 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd144021030 con 0x7fd15409dc30 2023-09-30T22:20:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.112+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd154071f70 con 0x7fd15409dc30 2023-09-30T22:20:41.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.117+0000 7fd14affd700 1 -- 172.21.15.89:0/2234563471 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd14401a7b0 con 0x7fd15409dc30 2023-09-30T22:20:41.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.277+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6f48074ea0 con 0x7f6f48139170 2023-09-30T22:20:41.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.278+0000 7f6f3effd700 1 -- 172.21.15.89:0/1000621540 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f40045060 con 0x7f6f48139170 2023-09-30T22:20:41.279 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f48139170 msgr2=0x7f6f4811f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f48139170 0x7f6f4811f130 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000efc0 tx=0x7f6f4000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 shutdown_connections 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f48139170 0x7f6f4811f130 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f4809d6f0 0x7f6f4811e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 --2- 172.21.15.89:0/1000621540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6f4809e1d0 0x7f6f4811ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 >> 172.21.15.89:0/1000621540 conn(0x7f6f48066f50 msgr2=0x7f6f48136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 shutdown_connections 2023-09-30T22:20:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.280+0000 7f6f506c3700 1 -- 172.21.15.89:0/1000621540 wait complete. 2023-09-30T22:20:41.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.281+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f774806c360 con 0x7f774809d6f0 2023-09-30T22:20:41.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.281+0000 7f772ffff700 1 -- 172.21.15.89:0/461976933 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7738021030 con 0x7f774809d6f0 2023-09-30T22:20:41.282 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:41.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.283+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 msgr2=0x7f77481a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.284+0000 7f774e12e700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f77481a5fd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f773800ebe0 tx=0x7f773800eef0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.284+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 shutdown_connections 2023-09-30T22:20:41.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.284+0000 7f774e12e700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f774809e410 0x7f77481a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.284+0000 7f774e12e700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f774809d6f0 0x7f77481a5fd0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.284+0000 7f774e12e700 1 --2- 172.21.15.89:0/461976933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f774809e000 0x7f77481a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.284+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 >> 172.21.15.89:0/461976933 conn(0x7f7748066f50 msgr2=0x7f77481baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:41.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.287+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 shutdown_connections 2023-09-30T22:20:41.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.287+0000 7f774e12e700 1 -- 172.21.15.89:0/461976933 wait complete. 2023-09-30T22:20:41.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:41.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.312+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd154074ea0 con 0x7fd15409dc30 2023-09-30T22:20:41.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.312+0000 7fd14affd700 1 -- 172.21.15.89:0/2234563471 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd144015020 con 0x7fd15409dc30 2023-09-30T22:20:41.313 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:41.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 msgr2=0x7fd154127410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd154127410 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd144000c00 tx=0x7fd144004750 comp rx=0 tx=0).stop 2023-09-30T22:20:41.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 shutdown_connections 2023-09-30T22:20:41.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd15413c1e0 0x7fd1541cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd15406a770 0x7fd154126ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 --2- 172.21.15.89:0/2234563471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd15409dc30 0x7fd154127410 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 >> 172.21.15.89:0/2234563471 conn(0x7fd154066f50 msgr2=0x7fd15413b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:41.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 shutdown_connections 2023-09-30T22:20:41.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.314+0000 7fd15c321700 1 -- 172.21.15.89:0/2234563471 wait complete. 2023-09-30T22:20:41.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.324+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae5406b9e0 con 0x7fae5409de50 2023-09-30T22:20:41.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.325+0000 7fae49ffb700 1 -- 172.21.15.89:0/3207487895 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae50003e60 con 0x7fae5409de50 2023-09-30T22:20:41.325 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:41.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 msgr2=0x7fae5411eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5411eac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fae50016000 tx=0x7fae50004750 comp rx=0 tx=0).stop 2023-09-30T22:20:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 shutdown_connections 2023-09-30T22:20:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae541338a0 0x7fae5411f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae5409de50 0x7fae5411eac0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 --2- 172.21.15.89:0/3207487895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae5409e760 0x7fae5411f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 >> 172.21.15.89:0/3207487895 conn(0x7fae540a2aa0 msgr2=0x7fae5409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 shutdown_connections 2023-09-30T22:20:41.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:41.327+0000 7fae5b869700 1 -- 172.21.15.89:0/3207487895 wait complete. 2023-09-30T22:20:41.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:42.294 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:42.306 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:42.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:42.342 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:42.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.443+0000 7f4a7f59e700 1 Processor -- start 2023-09-30T22:20:42.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.443+0000 7f4a7f59e700 1 -- start start 2023-09-30T22:20:42.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7f59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a78131810 0x7f4a78131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7f59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78099a40 0x7f4a78099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7f59e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a781313c0 con 0x7f4a7809a3b0 2023-09-30T22:20:42.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7f59e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7809b580 con 0x7f4a78099a40 2023-09-30T22:20:42.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7f59e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7806e8c0 con 0x7f4a78131810 2023-09-30T22:20:42.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7cb39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78099a40 0x7f4a78099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7cb39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78099a40 0x7f4a78099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60508/0 (socket says 172.21.15.89:60508) 2023-09-30T22:20:42.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7cb39700 1 -- 172.21.15.89:0/2993968984 learned_addr learned my addr 172.21.15.89:0/2993968984 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:42.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7db3b700 1 --1- 172.21.15.89:0/2993968984 >> v1:172.21.15.89:6789/0 conn(0x7f4a7809a3b0 0x7f4a78138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58086/0 (socket says 172.21.15.89:58086) 2023-09-30T22:20:42.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.444+0000 7f4a7d33a700 1 --2- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a78131810 0x7f4a78131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a6ffff700 1 -- 172.21.15.89:0/2993968984 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413841453 0 0) 0x7f4a781313c0 con 0x7f4a7809a3b0 2023-09-30T22:20:42.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a7cb39700 1 -- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a78131810 msgr2=0x7f4a78131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:42.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a7cb39700 1 --2- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a78131810 0x7f4a78131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a7cb39700 1 -- 172.21.15.89:0/2993968984 >> v1:172.21.15.89:6789/0 conn(0x7f4a7809a3b0 legacy=0x7f4a78138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a7cb39700 1 -- 172.21.15.89:0/2993968984 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7809a820 con 0x7f4a78099a40 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a7d33a700 1 --2- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a78131810 0x7f4a78131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a7cb39700 1 --2- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78099a40 0x7f4a78099e70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009920 tx=0x7f4a68009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e11e334586fb1ba4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a6ffff700 1 -- 172.21.15.89:0/2993968984 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6800ba60 con 0x7f4a78099a40 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a6ffff700 1 -- 172.21.15.89:0/2993968984 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6800bbc0 con 0x7f4a78099a40 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.445+0000 7f4a6ffff700 1 -- 172.21.15.89:0/2993968984 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68009e70 con 0x7f4a78099a40 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.446+0000 7f4a7f59e700 1 -- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78099a40 msgr2=0x7f4a78099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.446+0000 7f4a7f59e700 1 --2- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78099a40 0x7f4a78099e70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009920 tx=0x7f4a68009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.446+0000 7f4a7f59e700 1 -- 172.21.15.89:0/2993968984 shutdown_connections 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.446+0000 7f4a7f59e700 1 --2- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a78131810 0x7f4a78131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.446+0000 7f4a7f59e700 1 --2- 172.21.15.89:0/2993968984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78099a40 0x7f4a78099e70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.446+0000 7f4a7f59e700 1 -- 172.21.15.89:0/2993968984 >> 172.21.15.89:0/2993968984 conn(0x7f4a780a2aa0 msgr2=0x7f4a780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:42.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.447+0000 7f4a7f59e700 1 -- 172.21.15.89:0/2993968984 shutdown_connections 2023-09-30T22:20:42.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.447+0000 7f4a7f59e700 1 -- 172.21.15.89:0/2993968984 wait complete. 2023-09-30T22:20:42.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.448+0000 7f4a7f59e700 1 Processor -- start 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.448+0000 7f4a7f59e700 1 -- start start 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a78099a40 0x7f4a7806f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7f59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a7809a3b0 0x7f4a7806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7f59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78131810 0x7f4a78070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7f59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78120280 con 0x7f4a78099a40 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7f59e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78121290 con 0x7f4a78131810 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7f59e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78070a70 con 0x7f4a7809a3b0 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7cb39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a7809a3b0 0x7f4a7806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7d33a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a78099a40 0x7f4a7806f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7d33a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a78099a40 0x7f4a7806f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36550/0 (socket says 172.21.15.89:36550) 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7cb39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a7809a3b0 0x7f4a7806fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59696/0 (socket says 172.21.15.89:59696) 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7d33a700 1 -- 172.21.15.89:0/1090962556 learned_addr learned my addr 172.21.15.89:0/1090962556 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7db3b700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78131810 0x7f4a78070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7db3b700 1 -- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a7809a3b0 msgr2=0x7f4a7806fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7db3b700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a7809a3b0 0x7f4a7806fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7db3b700 1 -- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a78099a40 msgr2=0x7f4a7806f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7db3b700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a78099a40 0x7f4a7806f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7db3b700 1 -- 172.21.15.89:0/1090962556 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68009550 con 0x7f4a78131810 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7d33a700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a78099a40 0x7f4a7806f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.449+0000 7f4a7db3b700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78131810 0x7f4a78070090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000cbe0 tx=0x7f4a7000cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.450+0000 7f4a6dffb700 1 -- 172.21.15.89:0/1090962556 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000aa40 con 0x7f4a78131810 2023-09-30T22:20:42.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.450+0000 7f4a6dffb700 1 -- 172.21.15.89:0/1090962556 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7000aba0 con 0x7f4a78131810 2023-09-30T22:20:42.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.450+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a78130780 con 0x7f4a78131810 2023-09-30T22:20:42.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.450+0000 7f4a6dffb700 1 -- 172.21.15.89:0/1090962556 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a700070d0 con 0x7f4a78131810 2023-09-30T22:20:42.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.450+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a78071220 con 0x7f4a78131810 2023-09-30T22:20:42.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.451+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78120db0 con 0x7f4a78131810 2023-09-30T22:20:42.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.451+0000 7f4a6dffb700 1 -- 172.21.15.89:0/1090962556 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a70007230 con 0x7f4a78131810 2023-09-30T22:20:42.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.451+0000 7f4a6dffb700 1 -- 172.21.15.89:0/1090962556 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4a7003f160 con 0x7f4a78131810 2023-09-30T22:20:42.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.459+0000 7f4a6dffb700 1 -- 172.21.15.89:0/1090962556 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a70019860 con 0x7f4a78131810 2023-09-30T22:20:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.476+0000 7f1f6bc14700 1 Processor -- start 2023-09-30T22:20:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.476+0000 7f1f6bc14700 1 -- start start 2023-09-30T22:20:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f6bc14700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f6406e570 0x7f1f6406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f6bc14700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409eca0 0x7f1f6406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f6bc14700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f64070ee0 con 0x7f1f64134610 2023-09-30T22:20:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f6bc14700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f64071480 con 0x7f1f6406e570 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f6bc14700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6406f3c0 con 0x7f1f6409eca0 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f699b0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f6406e570 0x7f1f6406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f699b0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f6406e570 0x7f1f6406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36562/0 (socket says 172.21.15.89:36562) 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f699b0700 1 -- 172.21.15.89:0/283333416 learned_addr learned my addr 172.21.15.89:0/283333416 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f699b0700 1 -- 172.21.15.89:0/283333416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409eca0 msgr2=0x7f1f6406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f699b0700 1 --2- 172.21.15.89:0/283333416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409eca0 0x7f1f6406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f699b0700 1 -- 172.21.15.89:0/283333416 >> v1:172.21.15.89:6789/0 conn(0x7f1f64134610 legacy=0x7f1f64134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f699b0700 1 -- 172.21.15.89:0/283333416 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64071b90 con 0x7f1f6406e570 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.477+0000 7f1f699b0700 1 --2- 172.21.15.89:0/283333416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6406e570 0x7f1f6406e940 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1f600098f0 tx=0x7f1f60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36209193cfe98b9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f689ae700 1 -- 172.21.15.89:0/283333416 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000b8a0 con 0x7f1f6406e570 2023-09-30T22:20:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f689ae700 1 -- 172.21.15.89:0/283333416 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6000ba00 con 0x7f1f6406e570 2023-09-30T22:20:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f689ae700 1 -- 172.21.15.89:0/283333416 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000bd90 con 0x7f1f6406e570 2023-09-30T22:20:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f6bc14700 1 -- 172.21.15.89:0/283333416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6406e570 msgr2=0x7f1f6406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f6bc14700 1 --2- 172.21.15.89:0/283333416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6406e570 0x7f1f6406e940 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1f600098f0 tx=0x7f1f60009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f6bc14700 1 -- 172.21.15.89:0/283333416 shutdown_connections 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f6bc14700 1 --2- 172.21.15.89:0/283333416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6406e570 0x7f1f6406e940 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f6bc14700 1 --2- 172.21.15.89:0/283333416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409eca0 0x7f1f6406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.478+0000 7f1f6bc14700 1 -- 172.21.15.89:0/283333416 >> 172.21.15.89:0/283333416 conn(0x7f1f640a2aa0 msgr2=0x7f1f640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.479+0000 7f1f6bc14700 1 -- 172.21.15.89:0/283333416 shutdown_connections 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.479+0000 7f1f6bc14700 1 -- 172.21.15.89:0/283333416 wait complete. 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.479+0000 7f1f6bc14700 1 Processor -- start 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6bc14700 1 -- start start 2023-09-30T22:20:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6bc14700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6406e570 0x7f1f641233b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6bc14700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f64134610 0x7f1f641238f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6bc14700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f641273d0 0x7f1f64123e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6bc14700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64126ed0 con 0x7f1f641273d0 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6bc14700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64124fc0 con 0x7f1f6406e570 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6bc14700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640a1eb0 con 0x7f1f64134610 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6a1b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f641273d0 0x7f1f64123e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6a1b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f641273d0 0x7f1f64123e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36570/0 (socket says 172.21.15.89:36570) 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f6a1b1700 1 -- 172.21.15.89:0/659863266 learned_addr learned my addr 172.21.15.89:0/659863266 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.480+0000 7f1f691af700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f64134610 0x7f1f641238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f699b0700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6406e570 0x7f1f641233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f691af700 1 -- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6406e570 msgr2=0x7f1f641233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f691af700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6406e570 0x7f1f641233b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f691af700 1 -- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f641273d0 msgr2=0x7f1f64123e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f691af700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f641273d0 0x7f1f64123e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f691af700 1 -- 172.21.15.89:0/659863266 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60009550 con 0x7f1f64134610 2023-09-30T22:20:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f6a1b1700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f641273d0 0x7f1f64123e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f9e931d9700 1 Processor -- start 2023-09-30T22:20:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f691af700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f64134610 0x7f1f641238f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400b450 tx=0x7f1f5400b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f5a7fc700 1 -- 172.21.15.89:0/659863266 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54011830 con 0x7f1f64134610 2023-09-30T22:20:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f5a7fc700 1 -- 172.21.15.89:0/659863266 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f54011990 con 0x7f1f64134610 2023-09-30T22:20:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f5a7fc700 1 -- 172.21.15.89:0/659863266 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54011d40 con 0x7f1f64134610 2023-09-30T22:20:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f64070610 con 0x7f1f64134610 2023-09-30T22:20:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f64133ed0 con 0x7f1f64134610 2023-09-30T22:20:42.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.481+0000 7f9e931d9700 1 -- start start 2023-09-30T22:20:42.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e931d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e8c09d630 0x7f9e8c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e931d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c138d80 0x7f9e8c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e931d9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8c0a2a70 con 0x7f9e8c0a0a30 2023-09-30T22:20:42.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e931d9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8c0a01d0 con 0x7f9e8c09d630 2023-09-30T22:20:42.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e931d9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c09e2b0 con 0x7f9e8c138d80 2023-09-30T22:20:42.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f1f699b0700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6406e570 0x7f1f641233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e83fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e8c09d630 0x7f9e8c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e83fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e8c09d630 0x7f9e8c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36580/0 (socket says 172.21.15.89:36580) 2023-09-30T22:20:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e83fff700 1 -- 172.21.15.89:0/808009311 learned_addr learned my addr 172.21.15.89:0/808009311 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e83fff700 1 -- 172.21.15.89:0/808009311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c138d80 msgr2=0x7f9e8c13b160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:20:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e83fff700 1 --2- 172.21.15.89:0/808009311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c138d80 0x7f9e8c13b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e83fff700 1 -- 172.21.15.89:0/808009311 >> v1:172.21.15.89:6789/0 conn(0x7f9e8c0a0a30 legacy=0x7f9e8c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f9e83fff700 1 -- 172.21.15.89:0/808009311 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c09dc50 con 0x7f9e8c09d630 2023-09-30T22:20:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.482+0000 7f1f5a7fc700 1 -- 172.21.15.89:0/659863266 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f5401a400 con 0x7f1f64134610 2023-09-30T22:20:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.483+0000 7f9e83fff700 1 --2- 172.21.15.89:0/808009311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c0a1510 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9e880098f0 tx=0x7f9e88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4346c3d928adf355 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.483+0000 7f1f5a7fc700 1 -- 172.21.15.89:0/659863266 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1f5400f4a0 con 0x7f1f64134610 2023-09-30T22:20:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.483+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f64126a70 con 0x7f1f64134610 2023-09-30T22:20:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.483+0000 7f9e837fe700 1 -- 172.21.15.89:0/808009311 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88004230 con 0x7f9e8c09d630 2023-09-30T22:20:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.484+0000 7f9e837fe700 1 -- 172.21.15.89:0/808009311 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e88004390 con 0x7f9e8c09d630 2023-09-30T22:20:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.484+0000 7f9e837fe700 1 -- 172.21.15.89:0/808009311 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e880036a0 con 0x7f9e8c09d630 2023-09-30T22:20:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.484+0000 7f9e931d9700 1 -- 172.21.15.89:0/808009311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 msgr2=0x7f9e8c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.484+0000 7f9e931d9700 1 --2- 172.21.15.89:0/808009311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c0a1510 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9e880098f0 tx=0x7f9e88009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.485+0000 7f9e931d9700 1 -- 172.21.15.89:0/808009311 shutdown_connections 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.485+0000 7f9e931d9700 1 --2- 172.21.15.89:0/808009311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c138d80 0x7f9e8c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.485+0000 7f9e931d9700 1 --2- 172.21.15.89:0/808009311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c0a1510 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.485+0000 7f9e931d9700 1 -- 172.21.15.89:0/808009311 >> 172.21.15.89:0/808009311 conn(0x7f9e8c0a3050 msgr2=0x7f9e8c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.485+0000 7f9e931d9700 1 -- 172.21.15.89:0/808009311 shutdown_connections 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.485+0000 7f9e931d9700 1 -- 172.21.15.89:0/808009311 wait complete. 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.486+0000 7f9e931d9700 1 Processor -- start 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.486+0000 7f9e931d9700 1 -- start start 2023-09-30T22:20:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e931d9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e931d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c0a0a30 0x7f9e8c120d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e931d9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e8c138d80 0x7f9e8c121280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e931d9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c124260 con 0x7f9e8c09d630 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e931d9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0a4f30 con 0x7f9e8c0a0a30 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e931d9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c1223f0 con 0x7f9e8c138d80 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e90f75700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e90f75700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c120800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36596/0 (socket says 172.21.15.89:36596) 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e90f75700 1 -- 172.21.15.89:0/2866177072 learned_addr learned my addr 172.21.15.89:0/2866177072 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.487+0000 7f9e83fff700 1 --2- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c0a0a30 0x7f9e8c120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e90f75700 1 -- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e8c138d80 msgr2=0x7f9e8c121280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e90f75700 1 --2- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e8c138d80 0x7f9e8c121280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e90f75700 1 -- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c0a0a30 msgr2=0x7f9e8c120d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e90f75700 1 --2- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c0a0a30 0x7f9e8c120d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e90f75700 1 -- 172.21.15.89:0/2866177072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e88009550 con 0x7f9e8c09d630 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e90f75700 1 --2- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c120800 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c4a0 tx=0x7f9e7c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e817fa700 1 -- 172.21.15.89:0/2866177072 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c0054e0 con 0x7f9e8c09d630 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e817fa700 1 -- 172.21.15.89:0/2866177072 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c003680 con 0x7f9e8c09d630 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c0a0410 con 0x7f9e8c09d630 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e817fa700 1 -- 172.21.15.89:0/2866177072 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c003a30 con 0x7f9e8c09d630 2023-09-30T22:20:42.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.488+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c1249f0 con 0x7f9e8c09d630 2023-09-30T22:20:42.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.490+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c0747f0 con 0x7f9e8c09d630 2023-09-30T22:20:42.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.490+0000 7f9e817fa700 1 -- 172.21.15.89:0/2866177072 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e7c003b90 con 0x7f9e8c09d630 2023-09-30T22:20:42.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.490+0000 7f1f5a7fc700 1 -- 172.21.15.89:0/659863266 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f54015070 con 0x7f1f64134610 2023-09-30T22:20:42.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.493+0000 7f9e817fa700 1 -- 172.21.15.89:0/2866177072 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9e7c010b90 con 0x7f9e8c09d630 2023-09-30T22:20:42.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.495+0000 7f9e817fa700 1 -- 172.21.15.89:0/2866177072 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e7c044060 con 0x7f9e8c09d630 2023-09-30T22:20:42.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.521+0000 7f1d52cf8700 1 Processor -- start 2023-09-30T22:20:42.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.521+0000 7f1d52cf8700 1 -- start start 2023-09-30T22:20:42.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.521+0000 7f1d52cf8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c1316c0 0x7f1d4c131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d52cf8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1d4c06ae80 0x7f1d4c139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d52cf8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c09e130 con 0x7f1d4c09d9f0 2023-09-30T22:20:42.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d52cf8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c1312b0 con 0x7f1d4c06ae80 2023-09-30T22:20:42.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d52cf8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c09f6e0 con 0x7f1d4c1316c0 2023-09-30T22:20:42.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d50a94700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c1316c0 0x7f1d4c131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d50a94700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c1316c0 0x7f1d4c131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59736/0 (socket says 172.21.15.89:59736) 2023-09-30T22:20:42.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d50a94700 1 -- 172.21.15.89:0/2175241168 learned_addr learned my addr 172.21.15.89:0/2175241168 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d51295700 1 --2- 172.21.15.89:0/2175241168 >> v2:172.21.15.89:3300/0 conn(0x7f1d4c06ae80 0x7f1d4c139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d51295700 1 -- 172.21.15.89:0/2175241168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c1316c0 msgr2=0x7f1d4c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d51295700 1 --2- 172.21.15.89:0/2175241168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c1316c0 0x7f1d4c131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d51295700 1 -- 172.21.15.89:0/2175241168 >> v1:172.21.15.89:6789/0 conn(0x7f1d4c09d9f0 legacy=0x7f1d4c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d51295700 1 -- 172.21.15.89:0/2175241168 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c09e760 con 0x7f1d4c06ae80 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d51295700 1 --2- 172.21.15.89:0/2175241168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c06ae80 0x7f1d4c139550 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009d90 tx=0x7f1d4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f680f9b3439a9c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.522+0000 7f1d4b7fe700 1 -- 172.21.15.89:0/2175241168 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400e070 con 0x7f1d4c06ae80 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d4b7fe700 1 -- 172.21.15.89:0/2175241168 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4400bb20 con 0x7f1d4c06ae80 2023-09-30T22:20:42.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d4b7fe700 1 -- 172.21.15.89:0/2175241168 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44012430 con 0x7f1d4c06ae80 2023-09-30T22:20:42.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d52cf8700 1 -- 172.21.15.89:0/2175241168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c06ae80 msgr2=0x7f1d4c139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d52cf8700 1 --2- 172.21.15.89:0/2175241168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c06ae80 0x7f1d4c139550 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009d90 tx=0x7f1d4400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:42.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d52cf8700 1 -- 172.21.15.89:0/2175241168 shutdown_connections 2023-09-30T22:20:42.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d52cf8700 1 --2- 172.21.15.89:0/2175241168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c06ae80 0x7f1d4c139550 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d52cf8700 1 --2- 172.21.15.89:0/2175241168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c1316c0 0x7f1d4c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d52cf8700 1 -- 172.21.15.89:0/2175241168 >> 172.21.15.89:0/2175241168 conn(0x7f1d4c066f50 msgr2=0x7f1d4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:42.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d52cf8700 1 -- 172.21.15.89:0/2175241168 shutdown_connections 2023-09-30T22:20:42.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.523+0000 7f1d52cf8700 1 -- 172.21.15.89:0/2175241168 wait complete. 2023-09-30T22:20:42.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d52cf8700 1 Processor -- start 2023-09-30T22:20:42.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d52cf8700 1 -- start start 2023-09-30T22:20:42.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d52cf8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d4c06ae80 0x7f1d4c119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d52cf8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c09d9f0 0x7f1d4c1199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d52cf8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c1316c0 0x7f1d4c119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d52cf8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c11ae00 con 0x7f1d4c1316c0 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d52cf8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c12f3c0 con 0x7f1d4c06ae80 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d52cf8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c11ab10 con 0x7f1d4c09d9f0 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.525+0000 7f1d51295700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c1316c0 0x7f1d4c119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d51295700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c1316c0 0x7f1d4c119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36618/0 (socket says 172.21.15.89:36618) 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d51295700 1 -- 172.21.15.89:0/567924756 learned_addr learned my addr 172.21.15.89:0/567924756 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d50a94700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d4c06ae80 0x7f1d4c119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d51295700 1 -- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c09d9f0 msgr2=0x7f1d4c1199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d51295700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c09d9f0 0x7f1d4c1199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d51295700 1 -- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d4c06ae80 msgr2=0x7f1d4c119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d51295700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d4c06ae80 0x7f1d4c119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d51295700 1 -- 172.21.15.89:0/567924756 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009a70 con 0x7f1d4c1316c0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d50a94700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d4c06ae80 0x7f1d4c119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d51295700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c1316c0 0x7f1d4c119f00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c1313f0 tx=0x7f1d44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d497fa700 1 -- 172.21.15.89:0/567924756 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44012430 con 0x7f1d4c1316c0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d497fa700 1 -- 172.21.15.89:0/567924756 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d44020070 con 0x7f1d4c1316c0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c09ead0 con 0x7f1d4c1316c0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d497fa700 1 -- 172.21.15.89:0/567924756 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44003ea0 con 0x7f1d4c1316c0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.526+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c11b180 con 0x7f1d4c1316c0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.527+0000 7f1d497fa700 1 -- 172.21.15.89:0/567924756 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d44004280 con 0x7f1d4c1316c0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.527+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c074ea0 con 0x7f1d4c1316c0 2023-09-30T22:20:42.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.528+0000 7f1d497fa700 1 -- 172.21.15.89:0/567924756 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d44022070 con 0x7f1d4c1316c0 2023-09-30T22:20:42.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.532+0000 7f1d497fa700 1 -- 172.21.15.89:0/567924756 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d44004530 con 0x7f1d4c1316c0 2023-09-30T22:20:42.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.685+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1f64074ea0 con 0x7f1f64134610 2023-09-30T22:20:42.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.686+0000 7f1f5a7fc700 1 -- 172.21.15.89:0/659863266 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f54015020 con 0x7f1f64134610 2023-09-30T22:20:42.687 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:42.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f64134610 msgr2=0x7f1f641238f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f64134610 0x7f1f641238f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400b450 tx=0x7f1f5400b760 comp rx=0 tx=0).stop 2023-09-30T22:20:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 shutdown_connections 2023-09-30T22:20:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f641273d0 0x7f1f64123e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6406e570 0x7f1f641233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 --2- 172.21.15.89:0/659863266 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f64134610 0x7f1f641238f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 >> 172.21.15.89:0/659863266 conn(0x7f1f640a2aa0 msgr2=0x7f1f640a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 shutdown_connections 2023-09-30T22:20:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.688+0000 7f1f6bc14700 1 -- 172.21.15.89:0/659863266 wait complete. 2023-09-30T22:20:42.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:42.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.717+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4a78074ea0 con 0x7f4a78131810 2023-09-30T22:20:42.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.717+0000 7f4a6dffb700 1 -- 172.21.15.89:0/1090962556 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a70018030 con 0x7f4a78131810 2023-09-30T22:20:42.718 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:42.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78131810 msgr2=0x7f4a78070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78131810 0x7f4a78070090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000cbe0 tx=0x7f4a7000cef0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 shutdown_connections 2023-09-30T22:20:42.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a78131810 0x7f4a78070090 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a78099a40 0x7f4a7806f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 --2- 172.21.15.89:0/1090962556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a7809a3b0 0x7f4a7806fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 >> 172.21.15.89:0/1090962556 conn(0x7f4a780a2aa0 msgr2=0x7f4a781b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:42.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 shutdown_connections 2023-09-30T22:20:42.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.719+0000 7f4a7f59e700 1 -- 172.21.15.89:0/1090962556 wait complete. 2023-09-30T22:20:42.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:42.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.733+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1d4c11bb30 con 0x7f1d4c1316c0 2023-09-30T22:20:42.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.733+0000 7f1d497fa700 1 -- 172.21.15.89:0/567924756 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d4401a380 con 0x7f1d4c1316c0 2023-09-30T22:20:42.734 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:42.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.735+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c1316c0 msgr2=0x7f1d4c119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.735+0000 7f1d52cf8700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c1316c0 0x7f1d4c119f00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c1313f0 tx=0x7f1d44004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.735+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 shutdown_connections 2023-09-30T22:20:42.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.735+0000 7f1d52cf8700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d4c1316c0 0x7f1d4c119f00 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.735+0000 7f1d52cf8700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d4c06ae80 0x7f1d4c119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.735+0000 7f1d52cf8700 1 --2- 172.21.15.89:0/567924756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d4c09d9f0 0x7f1d4c1199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.735+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 >> 172.21.15.89:0/567924756 conn(0x7f1d4c066f50 msgr2=0x7f1d4c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:42.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.736+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 shutdown_connections 2023-09-30T22:20:42.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.736+0000 7f1d52cf8700 1 -- 172.21.15.89:0/567924756 wait complete. 2023-09-30T22:20:42.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.741+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e8c072980 con 0x7f9e8c09d630 2023-09-30T22:20:42.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.742+0000 7f9e817fa700 1 -- 172.21.15.89:0/2866177072 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e7c017020 con 0x7f9e8c09d630 2023-09-30T22:20:42.743 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 msgr2=0x7f9e8c120800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 --2- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c120800 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c4a0 tx=0x7f9e7c00c7b0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 shutdown_connections 2023-09-30T22:20:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 --2- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e8c138d80 0x7f9e8c121280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 --2- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e8c09d630 0x7f9e8c120800 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 --2- 172.21.15.89:0/2866177072 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e8c0a0a30 0x7f9e8c120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 >> 172.21.15.89:0/2866177072 conn(0x7f9e8c0a3050 msgr2=0x7f9e8c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 shutdown_connections 2023-09-30T22:20:42.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:42.744+0000 7f9e931d9700 1 -- 172.21.15.89:0/2866177072 wait complete. 2023-09-30T22:20:42.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:42.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:43.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:43.733 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:43.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:43.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:43.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.850+0000 7f373b45b700 1 Processor -- start 2023-09-30T22:20:43.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.850+0000 7f373b45b700 1 -- start start 2023-09-30T22:20:43.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f373b45b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f373b45b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f373b45b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373409edd0 con 0x7f37340a1340 2023-09-30T22:20:43.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f373b45b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373409e1f0 con 0x7f37340a0a30 2023-09-30T22:20:43.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f373b45b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373409dc50 con 0x7f3734134170 2023-09-30T22:20:43.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f37391f7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f37391f7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36628/0 (socket says 172.21.15.89:36628) 2023-09-30T22:20:43.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f37391f7700 1 -- 172.21.15.89:0/3077160772 learned_addr learned my addr 172.21.15.89:0/3077160772 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:43.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.851+0000 7f37399f8700 1 --2- 172.21.15.89:0/3077160772 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.852+0000 7f37399f8700 1 -- 172.21.15.89:0/3077160772 >> v2:172.21.15.89:3300/0 conn(0x7f37340a0a30 msgr2=0x7f37340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.852+0000 7f37399f8700 1 --2- 172.21.15.89:0/3077160772 >> v2:172.21.15.89:3300/0 conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.852+0000 7f37399f8700 1 -- 172.21.15.89:0/3077160772 >> v1:172.21.15.89:6789/0 conn(0x7f37340a1340 legacy=0x7f3734133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.852+0000 7f37399f8700 1 -- 172.21.15.89:0/3077160772 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37340a1fe0 con 0x7f3734134170 2023-09-30T22:20:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.852+0000 7f37399f8700 1 --2- 172.21.15.89:0/3077160772 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373413bd30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f372c009d90 tx=0x7f372c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b270e2a359e258 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.852+0000 7f372bfff700 1 -- 172.21.15.89:0/3077160772 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c00e070 con 0x7f3734134170 2023-09-30T22:20:43.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.852+0000 7f372bfff700 1 -- 172.21.15.89:0/3077160772 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c00bb20 con 0x7f3734134170 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.852+0000 7f372bfff700 1 -- 172.21.15.89:0/3077160772 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c012430 con 0x7f3734134170 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.853+0000 7f373b45b700 1 -- 172.21.15.89:0/3077160772 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 msgr2=0x7f373413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.853+0000 7f373b45b700 1 --2- 172.21.15.89:0/3077160772 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373413bd30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f372c009d90 tx=0x7f372c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.853+0000 7f373b45b700 1 -- 172.21.15.89:0/3077160772 shutdown_connections 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.853+0000 7f373b45b700 1 --2- 172.21.15.89:0/3077160772 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373413bd30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.853+0000 7f373b45b700 1 --2- 172.21.15.89:0/3077160772 >> v2:172.21.15.89:3300/0 conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.853+0000 7f373b45b700 1 -- 172.21.15.89:0/3077160772 >> 172.21.15.89:0/3077160772 conn(0x7f37340a6550 msgr2=0x7f37340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.853+0000 7f373b45b700 1 -- 172.21.15.89:0/3077160772 shutdown_connections 2023-09-30T22:20:43.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.853+0000 7f373b45b700 1 -- 172.21.15.89:0/3077160772 wait complete. 2023-09-30T22:20:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.855+0000 7f373b45b700 1 Processor -- start 2023-09-30T22:20:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.855+0000 7f373b45b700 1 -- start start 2023-09-30T22:20:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.855+0000 7f373b45b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37340a0a30 0x7f373411e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.855+0000 7f373b45b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37340a1340 0x7f373411ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.855+0000 7f373b45b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373411f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.855+0000 7f373b45b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3734120220 con 0x7f37340a1340 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.855+0000 7f373b45b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37341335a0 con 0x7f3734134170 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.855+0000 7f373b45b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3734122130 con 0x7f37340a0a30 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373411f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37391f7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37340a0a30 0x7f373411e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373411f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60566/0 (socket says 172.21.15.89:60566) 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 -- 172.21.15.89:0/805165714 learned_addr learned my addr 172.21.15.89:0/805165714 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37389f6700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37340a1340 0x7f373411ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 -- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37340a0a30 msgr2=0x7f373411e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37340a0a30 0x7f373411e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 -- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37340a1340 msgr2=0x7f373411ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37340a1340 0x7f373411ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 -- 172.21.15.89:0/805165714 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c009a70 con 0x7f3734134170 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37389f6700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37340a1340 0x7f373411ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f37399f8700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373411f120 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f372c014040 tx=0x7f372c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f3729ffb700 1 -- 172.21.15.89:0/805165714 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c012430 con 0x7f3734134170 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f3729ffb700 1 -- 172.21.15.89:0/805165714 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c020070 con 0x7f3734134170 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.856+0000 7f3729ffb700 1 -- 172.21.15.89:0/805165714 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c003ea0 con 0x7f3734134170 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.857+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37340a25e0 con 0x7f3734134170 2023-09-30T22:20:43.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.857+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37340a0220 con 0x7f3734134170 2023-09-30T22:20:43.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.858+0000 7f3729ffb700 1 -- 172.21.15.89:0/805165714 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f372c004040 con 0x7f3734134170 2023-09-30T22:20:43.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.858+0000 7f3729ffb700 1 -- 172.21.15.89:0/805165714 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f372c022070 con 0x7f3734134170 2023-09-30T22:20:43.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.858+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3734074ea0 con 0x7f3734134170 2023-09-30T22:20:43.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.863+0000 7f3729ffb700 1 -- 172.21.15.89:0/805165714 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f372c01a970 con 0x7f3734134170 2023-09-30T22:20:43.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b880c3700 1 Processor -- start 2023-09-30T22:20:43.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b880c3700 1 -- start start 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b880c3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 0x7f9b8009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b880c3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b8009e000 0x7f9b80130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b880c3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8009f130 con 0x7f9b8009e410 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b880c3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8009f520 con 0x7f9b8009e000 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b880c3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8009e930 con 0x7f9b8009d6f0 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b8565e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b8009e000 0x7f9b80130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b8565e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b8009e000 0x7f9b80130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36652/0 (socket says 172.21.15.89:36652) 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.880+0000 7f9b8565e700 1 -- 172.21.15.89:0/1450726089 learned_addr learned my addr 172.21.15.89:0/1450726089 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:43.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b85e5f700 1 --2- 172.21.15.89:0/1450726089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 0x7f9b8009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b8565e700 1 -- 172.21.15.89:0/1450726089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 msgr2=0x7f9b8009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:43.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b8565e700 1 --2- 172.21.15.89:0/1450726089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 0x7f9b8009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b8565e700 1 -- 172.21.15.89:0/1450726089 >> v1:172.21.15.89:6789/0 conn(0x7f9b8009e410 legacy=0x7f9b80130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b8565e700 1 -- 172.21.15.89:0/1450726089 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8009b2d0 con 0x7f9b8009e000 2023-09-30T22:20:43.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b8565e700 1 --2- 172.21.15.89:0/1450726089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e000 0x7f9b80130660 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000bb90 tx=0x7f9b7000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473cdf353dd1f3fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:43.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b84e5d700 1 -- 172.21.15.89:0/1450726089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7000dd50 con 0x7f9b8009e000 2023-09-30T22:20:43.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b84e5d700 1 -- 172.21.15.89:0/1450726089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b70010470 con 0x7f9b8009e000 2023-09-30T22:20:43.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.881+0000 7f9b84e5d700 1 -- 172.21.15.89:0/1450726089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70010800 con 0x7f9b8009e000 2023-09-30T22:20:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.882+0000 7f9b880c3700 1 -- 172.21.15.89:0/1450726089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e000 msgr2=0x7f9b80130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.882+0000 7f9b880c3700 1 --2- 172.21.15.89:0/1450726089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e000 0x7f9b80130660 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000bb90 tx=0x7f9b7000bea0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.882+0000 7f9b880c3700 1 -- 172.21.15.89:0/1450726089 shutdown_connections 2023-09-30T22:20:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.882+0000 7f9b880c3700 1 --2- 172.21.15.89:0/1450726089 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 0x7f9b8009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.882+0000 7f9b880c3700 1 --2- 172.21.15.89:0/1450726089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e000 0x7f9b80130660 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.882+0000 7f9b880c3700 1 -- 172.21.15.89:0/1450726089 >> 172.21.15.89:0/1450726089 conn(0x7f9b80066f50 msgr2=0x7f9b8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:43.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.882+0000 7f9b880c3700 1 -- 172.21.15.89:0/1450726089 shutdown_connections 2023-09-30T22:20:43.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.882+0000 7f9b880c3700 1 -- 172.21.15.89:0/1450726089 wait complete. 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.883+0000 7f9b880c3700 1 Processor -- start 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.883+0000 7f9b880c3700 1 -- start start 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.883+0000 7f9b880c3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 0x7f9b80122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.883+0000 7f9b880c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8009e000 0x7f9b80122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b880c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e410 0x7f9b80123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b880c3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801249e0 con 0x7f9b8009e410 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b880c3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801262a0 con 0x7f9b8009d6f0 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b880c3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80124390 con 0x7f9b8009e000 2023-09-30T22:20:43.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b86660700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e410 0x7f9b80123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b85e5f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 0x7f9b80122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b86660700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e410 0x7f9b80123200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36666/0 (socket says 172.21.15.89:36666) 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b86660700 1 -- 172.21.15.89:0/2828474203 learned_addr learned my addr 172.21.15.89:0/2828474203 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b8565e700 1 --2- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8009e000 0x7f9b80122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b86660700 1 -- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8009e000 msgr2=0x7f9b80122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b86660700 1 --2- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8009e000 0x7f9b80122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b86660700 1 -- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 msgr2=0x7f9b80122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b86660700 1 --2- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 0x7f9b80122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.884+0000 7f9b86660700 1 -- 172.21.15.89:0/2828474203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7000b7f0 con 0x7f9b8009e410 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.885+0000 7f9b86660700 1 --2- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e410 0x7f9b80123200 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800cbc0 tx=0x7f9b7800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:43.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.885+0000 7f9b767fc700 1 -- 172.21.15.89:0/2828474203 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78003c00 con 0x7f9b8009e410 2023-09-30T22:20:43.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.885+0000 7f9b767fc700 1 -- 172.21.15.89:0/2828474203 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78003d60 con 0x7f9b8009e410 2023-09-30T22:20:43.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.885+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b800a1d70 con 0x7f9b8009e410 2023-09-30T22:20:43.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.885+0000 7f9b767fc700 1 -- 172.21.15.89:0/2828474203 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800aae0 con 0x7f9b8009e410 2023-09-30T22:20:43.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.885+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b80124b80 con 0x7f9b8009e410 2023-09-30T22:20:43.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.886+0000 7f9b767fc700 1 -- 172.21.15.89:0/2828474203 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b7800ac40 con 0x7f9b8009e410 2023-09-30T22:20:43.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.886+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b80074ea0 con 0x7f9b8009e410 2023-09-30T22:20:43.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.886+0000 7f9b767fc700 1 -- 172.21.15.89:0/2828474203 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b7803f5d0 con 0x7f9b8009e410 2023-09-30T22:20:43.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f250a50a700 1 Processor -- start 2023-09-30T22:20:43.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f250a50a700 1 -- start start 2023-09-30T22:20:43.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f250a50a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f250409e930 0x7f250409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f250a50a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f650 0x7f2504131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f250a50a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2504070940 con 0x7f250409f240 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f250a50a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250406fd60 con 0x7f250409e930 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f250a50a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250406fa80 con 0x7f250409f650 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f2503fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f250409e930 0x7f250409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f2503fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f250409e930 0x7f250409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36670/0 (socket says 172.21.15.89:36670) 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f2503fff700 1 -- 172.21.15.89:0/2645480778 learned_addr learned my addr 172.21.15.89:0/2645480778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.893+0000 7f2508aa7700 1 --2- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f650 0x7f2504131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.894+0000 7f2503fff700 1 -- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f650 msgr2=0x7f2504131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.894+0000 7f2503fff700 1 --2- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f650 0x7f2504131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.894+0000 7f2503fff700 1 -- 172.21.15.89:0/2645480778 >> v1:172.21.15.89:6789/0 conn(0x7f250409f240 legacy=0x7f25041317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.894+0000 7f2503fff700 1 -- 172.21.15.89:0/2645480778 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250409fb60 con 0x7f250409e930 2023-09-30T22:20:43.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.894+0000 7f2508aa7700 1 --2- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f650 0x7f2504131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:43.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.894+0000 7f2503fff700 1 --2- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409e930 0x7f250409ed00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f24f40098f0 tx=0x7f24f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dda6f7d286999a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:43.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f2502ffd700 1 -- 172.21.15.89:0/2645480778 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4011830 con 0x7f250409e930 2023-09-30T22:20:43.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f2502ffd700 1 -- 172.21.15.89:0/2645480778 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f4011990 con 0x7f250409e930 2023-09-30T22:20:43.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f2502ffd700 1 -- 172.21.15.89:0/2645480778 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4011d20 con 0x7f250409e930 2023-09-30T22:20:43.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f9b767fc700 1 -- 172.21.15.89:0/2828474203 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b780198f0 con 0x7f9b8009e410 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f250a50a700 1 -- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409e930 msgr2=0x7f250409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f250a50a700 1 --2- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409e930 0x7f250409ed00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f24f40098f0 tx=0x7f24f4009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f250a50a700 1 -- 172.21.15.89:0/2645480778 shutdown_connections 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f250a50a700 1 --2- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f650 0x7f2504131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f250a50a700 1 --2- 172.21.15.89:0/2645480778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409e930 0x7f250409ed00 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f250a50a700 1 -- 172.21.15.89:0/2645480778 >> 172.21.15.89:0/2645480778 conn(0x7f25040a4210 msgr2=0x7f25040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.895+0000 7f250a50a700 1 -- 172.21.15.89:0/2645480778 shutdown_connections 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.896+0000 7f250a50a700 1 -- 172.21.15.89:0/2645480778 wait complete. 2023-09-30T22:20:43.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.896+0000 7f250a50a700 1 Processor -- start 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f250a50a700 1 -- start start 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f250a50a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f250409e930 0x7f250412d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f250a50a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f240 0x7f250412d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f250a50a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409f650 0x7f250412dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f250a50a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25041308c0 con 0x7f250409f650 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f250a50a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25040a35c0 con 0x7f250409f240 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f250a50a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2504071d20 con 0x7f250409e930 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f25037fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f240 0x7f250412d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f25037fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f240 0x7f250412d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60604/0 (socket says 172.21.15.89:60604) 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f25037fe700 1 -- 172.21.15.89:0/2764621785 learned_addr learned my addr 172.21.15.89:0/2764621785 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f2508aa7700 1 --2- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409f650 0x7f250412dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f25037fe700 1 -- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f250409e930 msgr2=0x7f250412d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f25037fe700 1 --2- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f250409e930 0x7f250412d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f25037fe700 1 -- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409f650 msgr2=0x7f250412dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.897+0000 7f25037fe700 1 --2- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409f650 0x7f250412dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.898+0000 7f25037fe700 1 -- 172.21.15.89:0/2764621785 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc009a70 con 0x7f250409f240 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.898+0000 7f25037fe700 1 --2- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f240 0x7f250412d680 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f24f800c4e0 tx=0x7f24f800c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.898+0000 7f2500ff9700 1 -- 172.21.15.89:0/2764621785 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80052d0 con 0x7f250409f240 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.898+0000 7f2500ff9700 1 -- 172.21.15.89:0/2764621785 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f8005430 con 0x7f250409f240 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.898+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f4009550 con 0x7f250409f240 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.898+0000 7f2500ff9700 1 -- 172.21.15.89:0/2764621785 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80038c0 con 0x7f250409f240 2023-09-30T22:20:43.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.898+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25040a0190 con 0x7f250409f240 2023-09-30T22:20:43.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.899+0000 7f2500ff9700 1 -- 172.21.15.89:0/2764621785 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24f8003a20 con 0x7f250409f240 2023-09-30T22:20:43.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.899+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e40052e0 con 0x7f250409f240 2023-09-30T22:20:43.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.899+0000 7f2500ff9700 1 -- 172.21.15.89:0/2764621785 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f24f803eae0 con 0x7f250409f240 2023-09-30T22:20:43.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.908+0000 7f2500ff9700 1 -- 172.21.15.89:0/2764621785 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24f8018260 con 0x7f250409f240 2023-09-30T22:20:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.937+0000 7fd75a08e700 1 Processor -- start 2023-09-30T22:20:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd75a08e700 1 -- start start 2023-09-30T22:20:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd75a08e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409d6f0 0x7fd75409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd75a08e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e410 0x7fd754134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd75a08e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75409b120 con 0x7fd75409e000 2023-09-30T22:20:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd75a08e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75406b460 con 0x7fd75409d6f0 2023-09-30T22:20:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd75a08e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd75409a7e0 con 0x7fd75409e410 2023-09-30T22:20:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd7537fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409d6f0 0x7fd75409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd7537fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409d6f0 0x7fd75409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60614/0 (socket says 172.21.15.89:60614) 2023-09-30T22:20:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.938+0000 7fd7537fe700 1 -- 172.21.15.89:0/4136738814 learned_addr learned my addr 172.21.15.89:0/4136738814 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd753fff700 1 --2- 172.21.15.89:0/4136738814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e410 0x7fd754134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7537fe700 1 -- 172.21.15.89:0/4136738814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e410 msgr2=0x7fd754134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7537fe700 1 --2- 172.21.15.89:0/4136738814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e410 0x7fd754134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7537fe700 1 -- 172.21.15.89:0/4136738814 >> v1:172.21.15.89:6789/0 conn(0x7fd75409e000 legacy=0x7fd7541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7527fc700 1 -- 172.21.15.89:0/4136738814 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452125824 0 0) 0x7fd75409b120 con 0x7fd75409e000 2023-09-30T22:20:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7537fe700 1 -- 172.21.15.89:0/4136738814 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75409a500 con 0x7fd75409d6f0 2023-09-30T22:20:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7537fe700 1 --2- 172.21.15.89:0/4136738814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409d6f0 0x7fd75409dac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd744009920 tx=0x7fd744009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50571dd5cd747ade server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7527fc700 1 -- 172.21.15.89:0/4136738814 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744011830 con 0x7fd75409d6f0 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7527fc700 1 -- 172.21.15.89:0/4136738814 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd744011990 con 0x7fd75409d6f0 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.939+0000 7fd7527fc700 1 -- 172.21.15.89:0/4136738814 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744011d20 con 0x7fd75409d6f0 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.940+0000 7fd75a08e700 1 -- 172.21.15.89:0/4136738814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409d6f0 msgr2=0x7fd75409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.940+0000 7fd75a08e700 1 --2- 172.21.15.89:0/4136738814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409d6f0 0x7fd75409dac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd744009920 tx=0x7fd744009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.940+0000 7fd75a08e700 1 -- 172.21.15.89:0/4136738814 shutdown_connections 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.940+0000 7fd75a08e700 1 --2- 172.21.15.89:0/4136738814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409d6f0 0x7fd75409dac0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.940+0000 7fd75a08e700 1 --2- 172.21.15.89:0/4136738814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e410 0x7fd754134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.940+0000 7fd75a08e700 1 -- 172.21.15.89:0/4136738814 >> 172.21.15.89:0/4136738814 conn(0x7fd754066f50 msgr2=0x7fd75409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.940+0000 7fd75a08e700 1 -- 172.21.15.89:0/4136738814 shutdown_connections 2023-09-30T22:20:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.940+0000 7fd75a08e700 1 -- 172.21.15.89:0/4136738814 wait complete. 2023-09-30T22:20:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.942+0000 7fd75a08e700 1 Processor -- start 2023-09-30T22:20:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.942+0000 7fd75a08e700 1 -- start start 2023-09-30T22:20:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.942+0000 7fd75a08e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd75409d6f0 0x7fd7541a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.942+0000 7fd75a08e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e000 0x7fd7541a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd75a08e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409e410 0x7fd7541a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:43.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd75a08e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7541a9140 con 0x7fd75409d6f0 2023-09-30T22:20:43.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd75a08e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75409c710 con 0x7fd75409e410 2023-09-30T22:20:43.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd75a08e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7540a1f90 con 0x7fd75409e000 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409e410 0x7fd7541a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409e410 0x7fd7541a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60622/0 (socket says 172.21.15.89:60622) 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 -- 172.21.15.89:0/2362086791 learned_addr learned my addr 172.21.15.89:0/2362086791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd7537fe700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd75409d6f0 0x7fd7541a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd752ffd700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e000 0x7fd7541a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 -- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e000 msgr2=0x7fd7541a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e000 0x7fd7541a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 -- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd75409d6f0 msgr2=0x7fd7541a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd75409d6f0 0x7fd7541a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 -- 172.21.15.89:0/2362086791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd744009550 con 0x7fd75409e410 2023-09-30T22:20:43.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd753fff700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409e410 0x7fd7541a69a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00efc0 tx=0x7fd74c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:43.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd73bfff700 1 -- 172.21.15.89:0/2362086791 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c009ea0 con 0x7fd75409e410 2023-09-30T22:20:43.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd73bfff700 1 -- 172.21.15.89:0/2362086791 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74c00f040 con 0x7fd75409e410 2023-09-30T22:20:43.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd73bfff700 1 -- 172.21.15.89:0/2362086791 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00ced0 con 0x7fd75409e410 2023-09-30T22:20:43.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7541a7aa0 con 0x7fd75409e410 2023-09-30T22:20:43.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.943+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7541a9eb0 con 0x7fd75409e410 2023-09-30T22:20:43.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.945+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd75406ada0 con 0x7fd75409e410 2023-09-30T22:20:43.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.945+0000 7fd73bfff700 1 -- 172.21.15.89:0/2362086791 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd74c014480 con 0x7fd75409e410 2023-09-30T22:20:43.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.945+0000 7fd73bfff700 1 -- 172.21.15.89:0/2362086791 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd74c0405f0 con 0x7fd75409e410 2023-09-30T22:20:43.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:43.954+0000 7fd73bfff700 1 -- 172.21.15.89:0/2362086791 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd74c012520 con 0x7fd75409e410 2023-09-30T22:20:44.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.072+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f37340a28e0 con 0x7f3734134170 2023-09-30T22:20:44.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.073+0000 7f3729ffb700 1 -- 172.21.15.89:0/805165714 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f372c02c020 con 0x7f3734134170 2023-09-30T22:20:44.074 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.075+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 msgr2=0x7f373411f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.075+0000 7f373b45b700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373411f120 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f372c014040 tx=0x7f372c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.075+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 shutdown_connections 2023-09-30T22:20:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.075+0000 7f373b45b700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3734134170 0x7f373411f120 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.075+0000 7f373b45b700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37340a0a30 0x7f373411e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.075+0000 7f373b45b700 1 --2- 172.21.15.89:0/805165714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37340a1340 0x7f373411ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.075+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 >> 172.21.15.89:0/805165714 conn(0x7f37340a6550 msgr2=0x7f373409ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:44.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.076+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 shutdown_connections 2023-09-30T22:20:44.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.077+0000 7f373b45b700 1 -- 172.21.15.89:0/805165714 wait complete. 2023-09-30T22:20:44.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:44.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.109+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f24e4005590 con 0x7f250409f240 2023-09-30T22:20:44.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.109+0000 7f2500ff9700 1 -- 172.21.15.89:0/2764621785 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24f8017070 con 0x7f250409f240 2023-09-30T22:20:44.110 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:44.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f240 msgr2=0x7f250412d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:44.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 --2- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f240 0x7f250412d680 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f24f800c4e0 tx=0x7f24f800c8a0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 shutdown_connections 2023-09-30T22:20:44.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 --2- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f250409f650 0x7f250412dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 --2- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f250409e930 0x7f250412d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 --2- 172.21.15.89:0/2764621785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f250409f240 0x7f250412d680 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 >> 172.21.15.89:0/2764621785 conn(0x7f25040a4210 msgr2=0x7f250406c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:44.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 shutdown_connections 2023-09-30T22:20:44.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.111+0000 7f250a50a700 1 -- 172.21.15.89:0/2764621785 wait complete. 2023-09-30T22:20:44.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:44.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.189+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9b8006c360 con 0x7f9b8009e410 2023-09-30T22:20:44.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.190+0000 7f9b767fc700 1 -- 172.21.15.89:0/2828474203 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b78011380 con 0x7f9b8009e410 2023-09-30T22:20:44.190 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e410 msgr2=0x7f9b80123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 --2- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e410 0x7f9b80123200 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800cbc0 tx=0x7f9b7800ced0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 shutdown_connections 2023-09-30T22:20:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 --2- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b8009e410 0x7f9b80123200 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 --2- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d6f0 0x7f9b80122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 --2- 172.21.15.89:0/2828474203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8009e000 0x7f9b80122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 >> 172.21.15.89:0/2828474203 conn(0x7f9b80066f50 msgr2=0x7f9b800710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 shutdown_connections 2023-09-30T22:20:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.192+0000 7f9b880c3700 1 -- 172.21.15.89:0/2828474203 wait complete. 2023-09-30T22:20:44.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:44.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.230+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd7541a9370 con 0x7fd75409e410 2023-09-30T22:20:44.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.231+0000 7fd73bfff700 1 -- 172.21.15.89:0/2362086791 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7541a9370 con 0x7fd75409e410 2023-09-30T22:20:44.232 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409e410 msgr2=0x7fd7541a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409e410 0x7fd7541a69a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00efc0 tx=0x7fd74c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 shutdown_connections 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd75409e410 0x7fd7541a69a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd75409d6f0 0x7fd7541a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 --2- 172.21.15.89:0/2362086791 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd75409e000 0x7fd7541a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 >> 172.21.15.89:0/2362086791 conn(0x7fd754066f50 msgr2=0x7fd7541badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 shutdown_connections 2023-09-30T22:20:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:44.233+0000 7fd75a08e700 1 -- 172.21.15.89:0/2362086791 wait complete. 2023-09-30T22:20:44.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:45.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:45.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:45.205 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.234+0000 7f4b30823700 1 Processor -- start 2023-09-30T22:20:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.234+0000 7f4b30823700 1 -- start start 2023-09-30T22:20:45.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b30823700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c09e1d0 0x7f4b2c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b30823700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b2c139170 0x7f4b2c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b30823700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c09b120 con 0x7f4b2c09d6f0 2023-09-30T22:20:45.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b30823700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c06b460 con 0x7f4b2c139170 2023-09-30T22:20:45.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b30823700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c09a7e0 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b21d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c09e1d0 0x7f4b2c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b21d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c09e1d0 0x7f4b2c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60636/0 (socket says 172.21.15.89:60636) 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b21d9b700 1 -- 172.21.15.89:0/2504642246 learned_addr learned my addr 172.21.15.89:0/2504642246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b2ad9d700 1 --2- 172.21.15.89:0/2504642246 >> v2:172.21.15.89:3300/0 conn(0x7f4b2c139170 0x7f4b2c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b21d9b700 1 -- 172.21.15.89:0/2504642246 >> v2:172.21.15.89:3300/0 conn(0x7f4b2c139170 msgr2=0x7f4b2c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b21d9b700 1 --2- 172.21.15.89:0/2504642246 >> v2:172.21.15.89:3300/0 conn(0x7f4b2c139170 0x7f4b2c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b21d9b700 1 -- 172.21.15.89:0/2504642246 >> v1:172.21.15.89:6789/0 conn(0x7f4b2c09d6f0 legacy=0x7f4b2c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b21d9b700 1 -- 172.21.15.89:0/2504642246 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c09a500 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.235+0000 7f4b21d9b700 1 --2- 172.21.15.89:0/2504642246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c09e1d0 0x7f4b2c138c30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009920 tx=0x7f4b14009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aeb91f0b3c89b0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b29d9b700 1 -- 172.21.15.89:0/2504642246 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011830 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b29d9b700 1 -- 172.21.15.89:0/2504642246 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14011990 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b29d9b700 1 -- 172.21.15.89:0/2504642246 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011d20 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b30823700 1 -- 172.21.15.89:0/2504642246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c09e1d0 msgr2=0x7f4b2c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b30823700 1 --2- 172.21.15.89:0/2504642246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c09e1d0 0x7f4b2c138c30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009920 tx=0x7f4b14009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b30823700 1 -- 172.21.15.89:0/2504642246 shutdown_connections 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b30823700 1 --2- 172.21.15.89:0/2504642246 >> v2:172.21.15.89:3300/0 conn(0x7f4b2c139170 0x7f4b2c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b30823700 1 --2- 172.21.15.89:0/2504642246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c09e1d0 0x7f4b2c138c30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b30823700 1 -- 172.21.15.89:0/2504642246 >> 172.21.15.89:0/2504642246 conn(0x7f4b2c066f50 msgr2=0x7f4b2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b30823700 1 -- 172.21.15.89:0/2504642246 shutdown_connections 2023-09-30T22:20:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.236+0000 7f4b30823700 1 -- 172.21.15.89:0/2504642246 wait complete. 2023-09-30T22:20:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b30823700 1 Processor -- start 2023-09-30T22:20:45.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b30823700 1 -- start start 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b30823700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c09d6f0 0x7f4b2c1c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b30823700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2c09e1d0 0x7f4b2c1c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b30823700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c139170 0x7f4b2c11e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b30823700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c134730 con 0x7f4b2c09d6f0 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b30823700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1205d0 con 0x7f4b2c139170 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b30823700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c072690 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b21d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2c09e1d0 0x7f4b2c1c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b21d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2c09e1d0 0x7f4b2c1c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59804/0 (socket says 172.21.15.89:59804) 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b21d9b700 1 -- 172.21.15.89:0/4161913168 learned_addr learned my addr 172.21.15.89:0/4161913168 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b2ad9d700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c139170 0x7f4b2c11e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.238+0000 7f4b2a59c700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c09d6f0 0x7f4b2c1c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b21d9b700 1 -- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c139170 msgr2=0x7f4b2c11e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b21d9b700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c139170 0x7f4b2c11e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b21d9b700 1 -- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c09d6f0 msgr2=0x7f4b2c1c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b21d9b700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c09d6f0 0x7f4b2c1c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b21d9b700 1 -- 172.21.15.89:0/4161913168 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b14009550 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b21d9b700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2c09e1d0 0x7f4b2c1c8b60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009130 tx=0x7f4b14004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b237fe700 1 -- 172.21.15.89:0/4161913168 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b140045d0 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b237fe700 1 -- 172.21.15.89:0/4161913168 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1401f070 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b237fe700 1 -- 172.21.15.89:0/4161913168 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b140038c0 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c09c710 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.239+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c120cb0 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.240+0000 7f4b237fe700 1 -- 172.21.15.89:0/4161913168 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b14003a20 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.240+0000 7f4b237fe700 1 -- 172.21.15.89:0/4161913168 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4b14021070 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.240+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c074ea0 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.249+0000 7f4b237fe700 1 -- 172.21.15.89:0/4161913168 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b1401a7b0 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.279+0000 7faeea5ab700 1 Processor -- start 2023-09-30T22:20:45.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.279+0000 7faeea5ab700 1 -- start start 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faeea5ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4090b60 0x7faee408b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faeea5ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee408d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faeea5ab700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee409be70 con 0x7faee4098a30 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faeea5ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee409ab80 con 0x7faee4090f90 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faeea5ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faee409ca10 con 0x7faee4090b60 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faee8b48700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee408d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faee8b48700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee408d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60656/0 (socket says 172.21.15.89:60656) 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faee8b48700 1 -- 172.21.15.89:0/1784470729 learned_addr learned my addr 172.21.15.89:0/1784470729 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.280+0000 7faee37fe700 1 --2- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4090b60 0x7faee408b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.281+0000 7faee2ffd700 1 -- 172.21.15.89:0/1784470729 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539989380 0 0) 0x7faee409be70 con 0x7faee4098a30 2023-09-30T22:20:45.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.282+0000 7faee2ffd700 1 -- 172.21.15.89:0/1784470729 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faecc003590 con 0x7faee4098a30 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee8b48700 1 -- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4090b60 msgr2=0x7faee408b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee8b48700 1 --2- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4090b60 0x7faee408b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee8b48700 1 -- 172.21.15.89:0/1784470729 >> v1:172.21.15.89:6789/0 conn(0x7faee4098a30 legacy=0x7faee4090450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee2ffd700 1 -- 172.21.15.89:0/1784470729 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3803363075 0 0) 0x7faecc003590 con 0x7faee4098a30 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee8b48700 1 -- 172.21.15.89:0/1784470729 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee409c1a0 con 0x7faee4090f90 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee37fe700 1 --2- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4090b60 0x7faee408b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee8b48700 1 --2- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee408d600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faedc00bf20 tx=0x7faedc00a4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e014ec88fb19f0f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee2ffd700 1 -- 172.21.15.89:0/1784470729 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4001670 con 0x7faee4090f90 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee2ffd700 1 -- 172.21.15.89:0/1784470729 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc003a20 con 0x7faee4090f90 2023-09-30T22:20:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.283+0000 7faee2ffd700 1 -- 172.21.15.89:0/1784470729 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc00a2e0 con 0x7faee4090f90 2023-09-30T22:20:45.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.284+0000 7faeea5ab700 1 -- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 msgr2=0x7faee408d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.284+0000 7faeea5ab700 1 --2- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee408d600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faedc00bf20 tx=0x7faedc00a4c0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.284+0000 7faeea5ab700 1 -- 172.21.15.89:0/1784470729 shutdown_connections 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.284+0000 7faeea5ab700 1 --2- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee408d600 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.284+0000 7faeea5ab700 1 --2- 172.21.15.89:0/1784470729 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4090b60 0x7faee408b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.284+0000 7faeea5ab700 1 -- 172.21.15.89:0/1784470729 >> 172.21.15.89:0/1784470729 conn(0x7faee409e7e0 msgr2=0x7faee409ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.284+0000 7faeea5ab700 1 -- 172.21.15.89:0/1784470729 shutdown_connections 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.284+0000 7faeea5ab700 1 -- 172.21.15.89:0/1784470729 wait complete. 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.286+0000 7faeea5ab700 1 Processor -- start 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.286+0000 7faeea5ab700 1 -- start start 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.286+0000 7faeea5ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faee4090b60 0x7faee40919c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.286+0000 7faeea5ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee4091f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.286+0000 7faeea5ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4098a30 0x7faee4092440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.286+0000 7faeea5ab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee4094f50 con 0x7faee4090b60 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.286+0000 7faeea5ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee4095380 con 0x7faee4090f90 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.286+0000 7faeea5ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faee40a1690 con 0x7faee4098a30 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee4091f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee4091f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60670/0 (socket says 172.21.15.89:60670) 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 -- 172.21.15.89:0/1765214987 learned_addr learned my addr 172.21.15.89:0/1765214987 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee3fff700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faee4090b60 0x7faee40919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee8b48700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4098a30 0x7faee4092440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 -- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4098a30 msgr2=0x7faee4092440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4098a30 0x7faee4092440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 -- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faee4090b60 msgr2=0x7faee40919c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faee4090b60 0x7faee40919c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 -- 172.21.15.89:0/1765214987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc00bb50 con 0x7faee4090f90 2023-09-30T22:20:45.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee37fe700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee4091f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7faed800dba0 tx=0x7faed800deb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:45.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.287+0000 7faee0ff9700 1 -- 172.21.15.89:0/1765214987 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8006ea0 con 0x7faee4090f90 2023-09-30T22:20:45.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.288+0000 7faee0ff9700 1 -- 172.21.15.89:0/1765214987 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed8004020 con 0x7faee4090f90 2023-09-30T22:20:45.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.288+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee4094cf0 con 0x7faee4090f90 2023-09-30T22:20:45.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.288+0000 7faee0ff9700 1 -- 172.21.15.89:0/1765214987 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed80041d0 con 0x7faee4090f90 2023-09-30T22:20:45.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.288+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee40959c0 con 0x7faee4090f90 2023-09-30T22:20:45.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.288+0000 7faee0ff9700 1 -- 172.21.15.89:0/1765214987 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faed8004330 con 0x7faee4090f90 2023-09-30T22:20:45.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.289+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee40747f0 con 0x7faee4090f90 2023-09-30T22:20:45.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.289+0000 7faee0ff9700 1 -- 172.21.15.89:0/1765214987 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faed803f2d0 con 0x7faee4090f90 2023-09-30T22:20:45.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.297+0000 7faee0ff9700 1 -- 172.21.15.89:0/1765214987 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faed801c9b0 con 0x7faee4090f90 2023-09-30T22:20:45.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.351+0000 7fc7b3f6c700 1 Processor -- start 2023-09-30T22:20:45.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.352+0000 7fc7b3f6c700 1 -- start start 2023-09-30T22:20:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.352+0000 7fc7b3f6c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7ac0a0a30 0x7fc7ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.352+0000 7fc7b3f6c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.352+0000 7fc7b3f6c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ac0a2a70 con 0x7fc7ac133930 2023-09-30T22:20:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.352+0000 7fc7b3f6c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ac0a01d0 con 0x7fc7ac0a0a30 2023-09-30T22:20:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.352+0000 7fc7b3f6c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac09e2b0 con 0x7fc7ac09d630 2023-09-30T22:20:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1507700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1d08700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7ac0a0a30 0x7fc7ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1d08700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7ac0a0a30 0x7fc7ac0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36718/0 (socket says 172.21.15.89:36718) 2023-09-30T22:20:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1d08700 1 -- 172.21.15.89:0/1251651794 learned_addr learned my addr 172.21.15.89:0/1251651794 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1507700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60684/0 (socket says 172.21.15.89:60684) 2023-09-30T22:20:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1d08700 1 -- 172.21.15.89:0/1251651794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 msgr2=0x7fc7ac0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1d08700 1 --2- 172.21.15.89:0/1251651794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1d08700 1 -- 172.21.15.89:0/1251651794 >> v1:172.21.15.89:6789/0 conn(0x7fc7ac133930 legacy=0x7fc7ac133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1d08700 1 -- 172.21.15.89:0/1251651794 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac09dc50 con 0x7fc7ac0a0a30 2023-09-30T22:20:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1507700 1 --2- 172.21.15.89:0/1251651794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b1d08700 1 --2- 172.21.15.89:0/1251651794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac0a0a30 0x7fc7ac0a0e00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80098f0 tx=0x7fc7a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a39f4ac5cfc58e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b0d06700 1 -- 172.21.15.89:0/1251651794 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8011830 con 0x7fc7ac0a0a30 2023-09-30T22:20:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b0d06700 1 -- 172.21.15.89:0/1251651794 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8011990 con 0x7fc7ac0a0a30 2023-09-30T22:20:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.353+0000 7fc7b0d06700 1 -- 172.21.15.89:0/1251651794 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8011d20 con 0x7fc7ac0a0a30 2023-09-30T22:20:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.354+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1251651794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac0a0a30 msgr2=0x7fc7ac0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.354+0000 7fc7b3f6c700 1 --2- 172.21.15.89:0/1251651794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac0a0a30 0x7fc7ac0a0e00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80098f0 tx=0x7fc7a8009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:45.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.354+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1251651794 shutdown_connections 2023-09-30T22:20:45.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.354+0000 7fc7b3f6c700 1 --2- 172.21.15.89:0/1251651794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac0a0a30 0x7fc7ac0a0e00 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.354+0000 7fc7b3f6c700 1 --2- 172.21.15.89:0/1251651794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.354+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1251651794 >> 172.21.15.89:0/1251651794 conn(0x7fc7ac0a3050 msgr2=0x7fc7ac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:45.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.354+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1251651794 shutdown_connections 2023-09-30T22:20:45.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.354+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1251651794 wait complete. 2023-09-30T22:20:45.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.355+0000 7fc7b3f6c700 1 Processor -- start 2023-09-30T22:20:45.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b3f6c700 1 -- start start 2023-09-30T22:20:45.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b3f6c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac1c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b3f6c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc7ac0a0a30 0x7fc7ac1c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b3f6c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac133930 0x7fc7ac1c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b3f6c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac11eea0 con 0x7fc7ac133930 2023-09-30T22:20:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b3f6c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1207f0 con 0x7fc7ac09d630 2023-09-30T22:20:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b3f6c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac120600 con 0x7fc7ac0a0a30 2023-09-30T22:20:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b2509700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac133930 0x7fc7ac1c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b1d08700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac1c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b2509700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac133930 0x7fc7ac1c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36724/0 (socket says 172.21.15.89:36724) 2023-09-30T22:20:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.356+0000 7fc7b2509700 1 -- 172.21.15.89:0/1749222912 learned_addr learned my addr 172.21.15.89:0/1749222912 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b1507700 1 --2- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc7ac0a0a30 0x7fc7ac1c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b2509700 1 -- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc7ac0a0a30 msgr2=0x7fc7ac1c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b2509700 1 --2- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc7ac0a0a30 0x7fc7ac1c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b2509700 1 -- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 msgr2=0x7fc7ac1c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b2509700 1 --2- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac1c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b2509700 1 -- 172.21.15.89:0/1749222912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c0094a0 con 0x7fc7ac133930 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b2509700 1 --2- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac133930 0x7fc7ac1c8ba0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400cc50 tx=0x7fc7a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7a27fc700 1 -- 172.21.15.89:0/1749222912 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4003c00 con 0x7fc7ac133930 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7a27fc700 1 -- 172.21.15.89:0/1749222912 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a4003d60 con 0x7fc7ac133930 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a8009550 con 0x7fc7ac133930 2023-09-30T22:20:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7a27fc700 1 -- 172.21.15.89:0/1749222912 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400ab60 con 0x7fc7ac133930 2023-09-30T22:20:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.357+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac122230 con 0x7fc7ac133930 2023-09-30T22:20:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.358+0000 7fc7a27fc700 1 -- 172.21.15.89:0/1749222912 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7a400acc0 con 0x7fc7ac133930 2023-09-30T22:20:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.358+0000 7fc7a27fc700 1 -- 172.21.15.89:0/1749222912 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc7a40421c0 con 0x7fc7ac133930 2023-09-30T22:20:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.359+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac121ed0 con 0x7fc7ac133930 2023-09-30T22:20:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.364+0000 7fc7a27fc700 1 -- 172.21.15.89:0/1749222912 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc7ac121ed0 con 0x7fc7ac133930 2023-09-30T22:20:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.387+0000 7ff656fa1700 1 Processor -- start 2023-09-30T22:20:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.387+0000 7ff656fa1700 1 -- start start 2023-09-30T22:20:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.387+0000 7ff656fa1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6500a0a30 0x7ff6500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.387+0000 7ff656fa1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6500a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff656fa1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6500a2a70 con 0x7ff650133930 2023-09-30T22:20:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff656fa1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6500a01d0 con 0x7ff6500a0a30 2023-09-30T22:20:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff656fa1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65009e2b0 con 0x7ff65009d630 2023-09-30T22:20:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff654d3d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6500a0a30 0x7ff6500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff654d3d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6500a0a30 0x7ff6500a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36738/0 (socket says 172.21.15.89:36738) 2023-09-30T22:20:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff654d3d700 1 -- 172.21.15.89:0/2226836357 learned_addr learned my addr 172.21.15.89:0/2226836357 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff65553e700 1 --1- 172.21.15.89:0/2226836357 >> v1:172.21.15.89:6789/0 conn(0x7ff650133930 0x7ff650133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58216/0 (socket says 172.21.15.89:58216) 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff64ffff700 1 --2- 172.21.15.89:0/2226836357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6500a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff654d3d700 1 -- 172.21.15.89:0/2226836357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 msgr2=0x7ff6500a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff654d3d700 1 --2- 172.21.15.89:0/2226836357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6500a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff654d3d700 1 -- 172.21.15.89:0/2226836357 >> v1:172.21.15.89:6789/0 conn(0x7ff650133930 legacy=0x7ff650133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff654d3d700 1 -- 172.21.15.89:0/2226836357 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65009dc50 con 0x7ff6500a0a30 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.388+0000 7ff654d3d700 1 --2- 172.21.15.89:0/2226836357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6500a0a30 0x7ff6500a0e00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff6440098f0 tx=0x7ff644009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6eb2e7e7904528 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.389+0000 7ff64f7fe700 1 -- 172.21.15.89:0/2226836357 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644011830 con 0x7ff6500a0a30 2023-09-30T22:20:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.389+0000 7ff64f7fe700 1 -- 172.21.15.89:0/2226836357 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644011990 con 0x7ff6500a0a30 2023-09-30T22:20:45.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.389+0000 7ff64f7fe700 1 -- 172.21.15.89:0/2226836357 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644011d20 con 0x7ff6500a0a30 2023-09-30T22:20:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.389+0000 7ff656fa1700 1 -- 172.21.15.89:0/2226836357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6500a0a30 msgr2=0x7ff6500a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.389+0000 7ff656fa1700 1 --2- 172.21.15.89:0/2226836357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6500a0a30 0x7ff6500a0e00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff6440098f0 tx=0x7ff644009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.389+0000 7ff656fa1700 1 -- 172.21.15.89:0/2226836357 shutdown_connections 2023-09-30T22:20:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.389+0000 7ff656fa1700 1 --2- 172.21.15.89:0/2226836357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6500a0a30 0x7ff6500a0e00 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.390+0000 7ff656fa1700 1 --2- 172.21.15.89:0/2226836357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6500a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.390+0000 7ff656fa1700 1 -- 172.21.15.89:0/2226836357 >> 172.21.15.89:0/2226836357 conn(0x7ff6500a3050 msgr2=0x7ff6500a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.390+0000 7ff656fa1700 1 -- 172.21.15.89:0/2226836357 shutdown_connections 2023-09-30T22:20:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.390+0000 7ff656fa1700 1 -- 172.21.15.89:0/2226836357 wait complete. 2023-09-30T22:20:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.391+0000 7ff656fa1700 1 Processor -- start 2023-09-30T22:20:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.391+0000 7ff656fa1700 1 -- start start 2023-09-30T22:20:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff656fa1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6501c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff656fa1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6500a0a30 0x7ff6501c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff656fa1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff650133930 0x7ff6501d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff656fa1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6501313a0 con 0x7ff650133930 2023-09-30T22:20:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff656fa1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6501320a0 con 0x7ff65009d630 2023-09-30T22:20:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff656fa1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6500a4030 con 0x7ff6500a0a30 2023-09-30T22:20:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff65553e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff650133930 0x7ff6501d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff654d3d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6501c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff654d3d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6501c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60716/0 (socket says 172.21.15.89:60716) 2023-09-30T22:20:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff654d3d700 1 -- 172.21.15.89:0/1540522700 learned_addr learned my addr 172.21.15.89:0/1540522700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff65553e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff650133930 0x7ff6501d4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36742/0 (socket says 172.21.15.89:36742) 2023-09-30T22:20:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff64ffff700 1 --2- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6500a0a30 0x7ff6501c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff65553e700 1 -- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6500a0a30 msgr2=0x7ff6501c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff65553e700 1 --2- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6500a0a30 0x7ff6501c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff65553e700 1 -- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 msgr2=0x7ff6501c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff65553e700 1 --2- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6501c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.392+0000 7ff65553e700 1 -- 172.21.15.89:0/1540522700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff644009550 con 0x7ff650133930 2023-09-30T22:20:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.393+0000 7ff65553e700 1 --2- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff650133930 0x7ff6501d4d70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff64800cb10 tx=0x7ff64800ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.393+0000 7ff64d7fa700 1 -- 172.21.15.89:0/1540522700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64800a9e0 con 0x7ff650133930 2023-09-30T22:20:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.393+0000 7ff64d7fa700 1 -- 172.21.15.89:0/1540522700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64800ab40 con 0x7ff650133930 2023-09-30T22:20:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.393+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff650133190 con 0x7ff650133930 2023-09-30T22:20:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.393+0000 7ff64d7fa700 1 -- 172.21.15.89:0/1540522700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648007070 con 0x7ff650133930 2023-09-30T22:20:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.393+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff650130c20 con 0x7ff650133930 2023-09-30T22:20:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.394+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6500747f0 con 0x7ff650133930 2023-09-30T22:20:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.395+0000 7ff64d7fa700 1 -- 172.21.15.89:0/1540522700 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff6480071d0 con 0x7ff650133930 2023-09-30T22:20:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.395+0000 7ff64d7fa700 1 -- 172.21.15.89:0/1540522700 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff64803fcf0 con 0x7ff650133930 2023-09-30T22:20:45.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.403+0000 7ff64d7fa700 1 -- 172.21.15.89:0/1540522700 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff648012400 con 0x7ff650133930 2023-09-30T22:20:45.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.459+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4b2c1c90a0 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.459+0000 7f4b237fe700 1 -- 172.21.15.89:0/4161913168 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b14015030 con 0x7f4b2c09e1d0 2023-09-30T22:20:45.462 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2c09e1d0 msgr2=0x7f4b2c1c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2c09e1d0 0x7f4b2c1c8b60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009130 tx=0x7f4b14004750 comp rx=0 tx=0).stop 2023-09-30T22:20:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 shutdown_connections 2023-09-30T22:20:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4b2c139170 0x7f4b2c11e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c09d6f0 0x7f4b2c1c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 --2- 172.21.15.89:0/4161913168 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4b2c09e1d0 0x7f4b2c1c8b60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 >> 172.21.15.89:0/4161913168 conn(0x7f4b2c066f50 msgr2=0x7f4b2c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:45.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 shutdown_connections 2023-09-30T22:20:45.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.461+0000 7f4b30823700 1 -- 172.21.15.89:0/4161913168 wait complete. 2023-09-30T22:20:45.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:45.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.497+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faee4094cf0 con 0x7faee4090f90 2023-09-30T22:20:45.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.497+0000 7faee0ff9700 1 -- 172.21.15.89:0/1765214987 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faed801c3c0 con 0x7faee4090f90 2023-09-30T22:20:45.498 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:45.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 msgr2=0x7faee4091f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee4091f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7faed800dba0 tx=0x7faed800deb0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 shutdown_connections 2023-09-30T22:20:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faee4098a30 0x7faee4092440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faee4090b60 0x7faee40919c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 --2- 172.21.15.89:0/1765214987 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faee4090f90 0x7faee4091f00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 >> 172.21.15.89:0/1765214987 conn(0x7faee409e7e0 msgr2=0x7faee406dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 shutdown_connections 2023-09-30T22:20:45.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.500+0000 7faeea5ab700 1 -- 172.21.15.89:0/1765214987 wait complete. 2023-09-30T22:20:45.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:45.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.572+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc7ac122420 con 0x7fc7ac133930 2023-09-30T22:20:45.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.572+0000 7fc7a27fc700 1 -- 172.21.15.89:0/1749222912 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7a4044060 con 0x7fc7ac133930 2023-09-30T22:20:45.573 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:45.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.574+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac133930 msgr2=0x7fc7ac1c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.574+0000 7fc7b3f6c700 1 --2- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac133930 0x7fc7ac1c8ba0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400cc50 tx=0x7fc7a400cf60 comp rx=0 tx=0).stop 2023-09-30T22:20:45.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.574+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 shutdown_connections 2023-09-30T22:20:45.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.574+0000 7fc7b3f6c700 1 --2- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7ac133930 0x7fc7ac1c8ba0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.574+0000 7fc7b3f6c700 1 --2- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc7ac09d630 0x7fc7ac1c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.574+0000 7fc7b3f6c700 1 --2- 172.21.15.89:0/1749222912 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc7ac0a0a30 0x7fc7ac1c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.574+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 >> 172.21.15.89:0/1749222912 conn(0x7fc7ac0a3050 msgr2=0x7fc7ac09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:45.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.575+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 shutdown_connections 2023-09-30T22:20:45.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.575+0000 7fc7b3f6c700 1 -- 172.21.15.89:0/1749222912 wait complete. 2023-09-30T22:20:45.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:45.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.602+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff650130e90 con 0x7ff650133930 2023-09-30T22:20:45.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.603+0000 7ff64d7fa700 1 -- 172.21.15.89:0/1540522700 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff648010300 con 0x7ff650133930 2023-09-30T22:20:45.604 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff650133930 msgr2=0x7ff6501d4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 --2- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff650133930 0x7ff6501d4d70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff64800cb10 tx=0x7ff64800ce20 comp rx=0 tx=0).stop 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 shutdown_connections 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 --2- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff650133930 0x7ff6501d4d70 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 --2- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff65009d630 0x7ff6501c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 --2- 172.21.15.89:0/1540522700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6500a0a30 0x7ff6501c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 >> 172.21.15.89:0/1540522700 conn(0x7ff6500a3050 msgr2=0x7ff65009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 shutdown_connections 2023-09-30T22:20:45.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:45.605+0000 7ff656fa1700 1 -- 172.21.15.89:0/1540522700 wait complete. 2023-09-30T22:20:45.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:46.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:46.513 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:46.588 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:46.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:46.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.618+0000 7f76ec007700 1 Processor -- start 2023-09-30T22:20:46.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.618+0000 7f76ec007700 1 -- start start 2023-09-30T22:20:46.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.618+0000 7f76ec007700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e41338a0 0x7f76e4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.618+0000 7f76ec007700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f76e409de50 0x7f76e409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.618+0000 7f76ec007700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e406ef30 con 0x7f76e409e760 2023-09-30T22:20:46.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.618+0000 7f76ec007700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e406fef0 con 0x7f76e409de50 2023-09-30T22:20:46.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.618+0000 7f76ec007700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e41334b0 con 0x7f76e41338a0 2023-09-30T22:20:46.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e95a2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f76e409de50 0x7f76e409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e95a2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f76e409de50 0x7f76e409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36750/0 (socket says 172.21.15.89:36750) 2023-09-30T22:20:46.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e95a2700 1 -- 172.21.15.89:0/851504002 learned_addr learned my addr 172.21.15.89:0/851504002 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:46.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e9da3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e41338a0 0x7f76e4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e95a2700 1 -- 172.21.15.89:0/851504002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e41338a0 msgr2=0x7f76e4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:46.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e95a2700 1 --2- 172.21.15.89:0/851504002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e41338a0 0x7f76e4133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e95a2700 1 -- 172.21.15.89:0/851504002 >> v1:172.21.15.89:6789/0 conn(0x7f76e409e760 legacy=0x7f76e413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e95a2700 1 -- 172.21.15.89:0/851504002 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e406fc30 con 0x7f76e409de50 2023-09-30T22:20:46.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e9da3700 1 --2- 172.21.15.89:0/851504002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e41338a0 0x7f76e4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:46.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e95a2700 1 --2- 172.21.15.89:0/851504002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 0x7f76e409e220 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f76d40098f0 tx=0x7f76d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2e1837e5b8fdf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:46.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e8da1700 1 -- 172.21.15.89:0/851504002 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4011830 con 0x7f76e409de50 2023-09-30T22:20:46.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e8da1700 1 -- 172.21.15.89:0/851504002 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d4011990 con 0x7f76e409de50 2023-09-30T22:20:46.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.619+0000 7f76e8da1700 1 -- 172.21.15.89:0/851504002 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4011d20 con 0x7f76e409de50 2023-09-30T22:20:46.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.620+0000 7f76ec007700 1 -- 172.21.15.89:0/851504002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 msgr2=0x7f76e409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.620+0000 7f76ec007700 1 --2- 172.21.15.89:0/851504002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 0x7f76e409e220 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f76d40098f0 tx=0x7f76d4009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:46.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.621+0000 7f76ec007700 1 -- 172.21.15.89:0/851504002 shutdown_connections 2023-09-30T22:20:46.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.621+0000 7f76ec007700 1 --2- 172.21.15.89:0/851504002 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e41338a0 0x7f76e4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.621+0000 7f76ec007700 1 --2- 172.21.15.89:0/851504002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 0x7f76e409e220 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.621+0000 7f76ec007700 1 -- 172.21.15.89:0/851504002 >> 172.21.15.89:0/851504002 conn(0x7f76e40a2aa0 msgr2=0x7f76e40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:46.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.621+0000 7f76ec007700 1 -- 172.21.15.89:0/851504002 shutdown_connections 2023-09-30T22:20:46.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.621+0000 7f76ec007700 1 -- 172.21.15.89:0/851504002 wait complete. 2023-09-30T22:20:46.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.622+0000 7f76ec007700 1 Processor -- start 2023-09-30T22:20:46.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.622+0000 7f76ec007700 1 -- start start 2023-09-30T22:20:46.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ec007700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 0x7f76e411eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ec007700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e409e760 0x7f76e411f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ec007700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76e41338a0 0x7f76e411f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ec007700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4121ea0 con 0x7f76e409de50 2023-09-30T22:20:46.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ec007700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4122080 con 0x7f76e409e760 2023-09-30T22:20:46.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ec007700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76e412f690 con 0x7f76e41338a0 2023-09-30T22:20:46.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76e41338a0 0x7f76e411f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76e41338a0 0x7f76e411f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59858/0 (socket says 172.21.15.89:59858) 2023-09-30T22:20:46.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 -- 172.21.15.89:0/484294066 learned_addr learned my addr 172.21.15.89:0/484294066 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:46.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76e9da3700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 0x7f76e411eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76e95a2700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e409e760 0x7f76e411f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 -- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e409e760 msgr2=0x7f76e411f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e409e760 0x7f76e411f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 -- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 msgr2=0x7f76e411eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 0x7f76e411eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 -- 172.21.15.89:0/484294066 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00094a0 con 0x7f76e41338a0 2023-09-30T22:20:46.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76e95a2700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e409e760 0x7f76e411f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:46.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76e9da3700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 0x7f76e411eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:46.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.623+0000 7f76ea5a4700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76e41338a0 0x7f76e411f580 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00cc80 tx=0x7f76dc00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:46.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.624+0000 7f76da7fc700 1 -- 172.21.15.89:0/484294066 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc003c00 con 0x7f76e41338a0 2023-09-30T22:20:46.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.624+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d4009550 con 0x7f76e41338a0 2023-09-30T22:20:46.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.624+0000 7f76da7fc700 1 -- 172.21.15.89:0/484294066 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76dc003d60 con 0x7f76e41338a0 2023-09-30T22:20:46.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.624+0000 7f76da7fc700 1 -- 172.21.15.89:0/484294066 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00ab50 con 0x7f76e41338a0 2023-09-30T22:20:46.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.624+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e406f640 con 0x7f76e41338a0 2023-09-30T22:20:46.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.624+0000 7f76da7fc700 1 -- 172.21.15.89:0/484294066 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76dc00acb0 con 0x7f76e41338a0 2023-09-30T22:20:46.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.625+0000 7f76da7fc700 1 -- 172.21.15.89:0/484294066 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f76dc03f7b0 con 0x7f76e41338a0 2023-09-30T22:20:46.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.625+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e4074ea0 con 0x7f76e41338a0 2023-09-30T22:20:46.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.631+0000 7f76da7fc700 1 -- 172.21.15.89:0/484294066 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76dc0198f0 con 0x7f76e41338a0 2023-09-30T22:20:46.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.661+0000 7f35ce409700 1 Processor -- start 2023-09-30T22:20:46.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.661+0000 7f35ce409700 1 -- start start 2023-09-30T22:20:46.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35ce409700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35ce409700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35c809e000 0x7f35c8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35ce409700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e410 0x7f35c8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35ce409700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c809f520 con 0x7f35c809e000 2023-09-30T22:20:46.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35ce409700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c809f130 con 0x7f35c809e410 2023-09-30T22:20:46.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35ce409700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35c809e930 con 0x7f35c809d6f0 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35c77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35c809e000 0x7f35c8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35c7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35c7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59866/0 (socket says 172.21.15.89:59866) 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35c77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35c809e000 0x7f35c8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36780/0 (socket says 172.21.15.89:36780) 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35c7fff700 1 -- 172.21.15.89:0/1021621307 learned_addr learned my addr 172.21.15.89:0/1021621307 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.662+0000 7f35cc9a6700 1 --2- 172.21.15.89:0/1021621307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e410 0x7f35c8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c7fff700 1 -- 172.21.15.89:0/1021621307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e410 msgr2=0x7f35c8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c7fff700 1 --2- 172.21.15.89:0/1021621307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e410 0x7f35c8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c7fff700 1 -- 172.21.15.89:0/1021621307 >> v2:172.21.15.89:3300/0 conn(0x7f35c809e000 msgr2=0x7f35c8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c7fff700 1 --2- 172.21.15.89:0/1021621307 >> v2:172.21.15.89:3300/0 conn(0x7f35c809e000 0x7f35c8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c7fff700 1 -- 172.21.15.89:0/1021621307 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c809b2d0 con 0x7f35c809d6f0 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c7fff700 1 --2- 172.21.15.89:0/1021621307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c809dac0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009920 tx=0x7f35b8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1758130ad1347bc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c6ffd700 1 -- 172.21.15.89:0/1021621307 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8011830 con 0x7f35c809d6f0 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c6ffd700 1 -- 172.21.15.89:0/1021621307 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b8011990 con 0x7f35c809d6f0 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.663+0000 7f35c6ffd700 1 -- 172.21.15.89:0/1021621307 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8011d20 con 0x7f35c809d6f0 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 -- 172.21.15.89:0/1021621307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 msgr2=0x7f35c809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 --2- 172.21.15.89:0/1021621307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c809dac0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009920 tx=0x7f35b8009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 -- 172.21.15.89:0/1021621307 shutdown_connections 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 --2- 172.21.15.89:0/1021621307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e410 0x7f35c8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 --2- 172.21.15.89:0/1021621307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c809dac0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 --2- 172.21.15.89:0/1021621307 >> v2:172.21.15.89:3300/0 conn(0x7f35c809e000 0x7f35c8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 -- 172.21.15.89:0/1021621307 >> 172.21.15.89:0/1021621307 conn(0x7f35c8066f50 msgr2=0x7f35c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:46.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 -- 172.21.15.89:0/1021621307 shutdown_connections 2023-09-30T22:20:46.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.664+0000 7f35ce409700 1 -- 172.21.15.89:0/1021621307 wait complete. 2023-09-30T22:20:46.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.665+0000 7f35ce409700 1 Processor -- start 2023-09-30T22:20:46.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.665+0000 7f35ce409700 1 -- start start 2023-09-30T22:20:46.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.665+0000 7f35ce409700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c811dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.665+0000 7f35ce409700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e000 0x7f35c811e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.665+0000 7f35ce409700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c809e410 0x7f35c811e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.665+0000 7f35ce409700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8121960 con 0x7f35c809e410 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.665+0000 7f35ce409700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8120070 con 0x7f35c809e000 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.665+0000 7f35ce409700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8120d30 con 0x7f35c809d6f0 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c811dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c811dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59874/0 (socket says 172.21.15.89:59874) 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 -- 172.21.15.89:0/699726013 learned_addr learned my addr 172.21.15.89:0/699726013 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35cc9a6700 1 --2- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c809e410 0x7f35c811e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e000 0x7f35c811e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 -- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e000 msgr2=0x7f35c811e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 --2- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e000 0x7f35c811e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 -- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c809e410 msgr2=0x7f35c811e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 --2- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c809e410 0x7f35c811e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 -- 172.21.15.89:0/699726013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b8009550 con 0x7f35c809d6f0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c7fff700 1 --2- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c811dee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f35b8000c00 tx=0x7f35b8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c4ff9700 1 -- 172.21.15.89:0/699726013 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8004240 con 0x7f35c809d6f0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c4ff9700 1 -- 172.21.15.89:0/699726013 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b801f070 con 0x7f35c809d6f0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.666+0000 7f35c4ff9700 1 -- 172.21.15.89:0/699726013 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b80036a0 con 0x7f35c809d6f0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.667+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c811f340 con 0x7f35c809d6f0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.667+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c809c700 con 0x7f35c809d6f0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.668+0000 7f35c4ff9700 1 -- 172.21.15.89:0/699726013 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f35b8021030 con 0x7f35c809d6f0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.668+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c8121320 con 0x7f35c809d6f0 2023-09-30T22:20:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.668+0000 7f35c4ff9700 1 -- 172.21.15.89:0/699726013 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f35b8040760 con 0x7f35c809d6f0 2023-09-30T22:20:46.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.677+0000 7f35c4ff9700 1 -- 172.21.15.89:0/699726013 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35c8121320 con 0x7f35c809d6f0 2023-09-30T22:20:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.720+0000 7fc5af526700 1 Processor -- start 2023-09-30T22:20:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.720+0000 7fc5af526700 1 -- start start 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5af526700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a8136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5af526700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc5a8136d80 0x7fc5a8139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5af526700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a809ece0 con 0x7fc5a806a770 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5af526700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a806b640 con 0x7fc5a8136d80 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5af526700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a809e760 con 0x7fc5a806b250 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5ad2c2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc5a806a770 0x7fc5a806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58234/0 (socket says 172.21.15.89:58234) 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5ad2c2700 1 -- 172.21.15.89:0/1193064082 learned_addr learned my addr 172.21.15.89:0/1193064082 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5adac3700 1 --2- 172.21.15.89:0/1193064082 >> v2:172.21.15.89:3300/0 conn(0x7fc5a8136d80 0x7fc5a8139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.721+0000 7fc5acac1700 1 --2- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a8136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc59ffff700 1 -- 172.21.15.89:0/1193064082 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853071556 0 0) 0x7fc5a809ece0 con 0x7fc5a806a770 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc5adac3700 1 -- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 msgr2=0x7fc5a8136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc5adac3700 1 --2- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a8136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc5adac3700 1 -- 172.21.15.89:0/1193064082 >> v1:172.21.15.89:6789/0 conn(0x7fc5a806a770 legacy=0x7fc5a806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc5adac3700 1 -- 172.21.15.89:0/1193064082 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a809e040 con 0x7fc5a8136d80 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc5acac1700 1 --2- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a8136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc5adac3700 1 --2- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a8136d80 0x7fc5a8139160 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009d90 tx=0x7fc5a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c833a9dbceddc9cb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc59ffff700 1 -- 172.21.15.89:0/1193064082 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000e070 con 0x7fc5a8136d80 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc59ffff700 1 -- 172.21.15.89:0/1193064082 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000bb20 con 0x7fc5a8136d80 2023-09-30T22:20:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.722+0000 7fc59ffff700 1 -- 172.21.15.89:0/1193064082 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0012430 con 0x7fc5a8136d80 2023-09-30T22:20:46.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.723+0000 7fc5af526700 1 -- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a8136d80 msgr2=0x7fc5a8139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.723+0000 7fc5af526700 1 --2- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a8136d80 0x7fc5a8139160 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009d90 tx=0x7fc5a000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:46.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.723+0000 7fc5af526700 1 -- 172.21.15.89:0/1193064082 shutdown_connections 2023-09-30T22:20:46.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.723+0000 7fc5af526700 1 --2- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a8136d80 0x7fc5a8139160 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.723+0000 7fc5af526700 1 --2- 172.21.15.89:0/1193064082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a8136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.723+0000 7fc5af526700 1 -- 172.21.15.89:0/1193064082 >> 172.21.15.89:0/1193064082 conn(0x7fc5a8066f50 msgr2=0x7fc5a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.723+0000 7fc5af526700 1 -- 172.21.15.89:0/1193064082 shutdown_connections 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.723+0000 7fc5af526700 1 -- 172.21.15.89:0/1193064082 wait complete. 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5af526700 1 Processor -- start 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5af526700 1 -- start start 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5af526700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a806a770 0x7fc5a809acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5af526700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a809b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5af526700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5a8136d80 0x7fc5a81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5af526700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8121c30 con 0x7fc5a806a770 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5af526700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a806ec70 con 0x7fc5a806b250 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5af526700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8071e60 con 0x7fc5a8136d80 2023-09-30T22:20:46.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5ad2c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a806a770 0x7fc5a809acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5ad2c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a806a770 0x7fc5a809acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36810/0 (socket says 172.21.15.89:36810) 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5ad2c2700 1 -- 172.21.15.89:0/3222206942 learned_addr learned my addr 172.21.15.89:0/3222206942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.724+0000 7fc5acac1700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a809b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5adac3700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5a8136d80 0x7fc5a81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5ad2c2700 1 -- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5a8136d80 msgr2=0x7fc5a81c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5ad2c2700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5a8136d80 0x7fc5a81c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5ad2c2700 1 -- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 msgr2=0x7fc5a809b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5ad2c2700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a809b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5ad2c2700 1 -- 172.21.15.89:0/3222206942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5980094a0 con 0x7fc5a806a770 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5ad2c2700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a806a770 0x7fc5a809acf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400c520 tx=0x7fc5a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc59dffb700 1 -- 172.21.15.89:0/3222206942 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4004c80 con 0x7fc5a806a770 2023-09-30T22:20:46.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc59dffb700 1 -- 172.21.15.89:0/3222206942 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a4004de0 con 0x7fc5a806a770 2023-09-30T22:20:46.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc59dffb700 1 -- 172.21.15.89:0/3222206942 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4007310 con 0x7fc5a806a770 2023-09-30T22:20:46.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a0009a70 con 0x7fc5a806a770 2023-09-30T22:20:46.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.725+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a809b770 con 0x7fc5a806a770 2023-09-30T22:20:46.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.726+0000 7fc59dffb700 1 -- 172.21.15.89:0/3222206942 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5a4007470 con 0x7fc5a806a770 2023-09-30T22:20:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.726+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8121e30 con 0x7fc5a806a770 2023-09-30T22:20:46.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.727+0000 7fc59dffb700 1 -- 172.21.15.89:0/3222206942 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc5a403f320 con 0x7fc5a806a770 2023-09-30T22:20:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.736+0000 7fc59dffb700 1 -- 172.21.15.89:0/3222206942 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5a8121e30 con 0x7fc5a806a770 2023-09-30T22:20:46.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f958b089700 1 Processor -- start 2023-09-30T22:20:46.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f958b089700 1 -- start start 2023-09-30T22:20:46.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f958b089700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958406a770 0x7f958409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f958b089700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958409dc30 0x7f958409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f958b089700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958409a810 con 0x7f958413c1e0 2023-09-30T22:20:46.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f958b089700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958406ab40 con 0x7f958406a770 2023-09-30T22:20:46.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f958b089700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f958409ebc0 con 0x7f958409dc30 2023-09-30T22:20:46.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f9583fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958409dc30 0x7f958409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f9583fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958409dc30 0x7f958409e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59884/0 (socket says 172.21.15.89:59884) 2023-09-30T22:20:46.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.753+0000 7f9583fff700 1 -- 172.21.15.89:0/3190375043 learned_addr learned my addr 172.21.15.89:0/3190375043 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:46.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f9588e25700 1 --2- 172.21.15.89:0/3190375043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958406a770 0x7f958409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f9583fff700 1 -- 172.21.15.89:0/3190375043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958406a770 msgr2=0x7f958409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f9583fff700 1 --2- 172.21.15.89:0/3190375043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958406a770 0x7f958409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f9583fff700 1 -- 172.21.15.89:0/3190375043 >> v1:172.21.15.89:6789/0 conn(0x7f958413c1e0 legacy=0x7f958413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f9583fff700 1 -- 172.21.15.89:0/3190375043 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958409e920 con 0x7f958409dc30 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f95837fe700 1 -- 172.21.15.89:0/3190375043 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147090484 0 0) 0x7f958409a810 con 0x7f958413c1e0 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f9583fff700 1 --2- 172.21.15.89:0/3190375043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958409dc30 0x7f958409e080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9570009950 tx=0x7f9570009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75dff589ad64590c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f95837fe700 1 -- 172.21.15.89:0/3190375043 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570011830 con 0x7f958409dc30 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f95837fe700 1 -- 172.21.15.89:0/3190375043 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9570011990 con 0x7f958409dc30 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.754+0000 7f95837fe700 1 -- 172.21.15.89:0/3190375043 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570011d40 con 0x7f958409dc30 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.755+0000 7f958b089700 1 -- 172.21.15.89:0/3190375043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958409dc30 msgr2=0x7f958409e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.755+0000 7f958b089700 1 --2- 172.21.15.89:0/3190375043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958409dc30 0x7f958409e080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9570009950 tx=0x7f9570009c60 comp rx=0 tx=0).stop 2023-09-30T22:20:46.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.755+0000 7f958b089700 1 -- 172.21.15.89:0/3190375043 shutdown_connections 2023-09-30T22:20:46.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.755+0000 7f958b089700 1 --2- 172.21.15.89:0/3190375043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958406a770 0x7f958409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.755+0000 7f958b089700 1 --2- 172.21.15.89:0/3190375043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958409dc30 0x7f958409e080 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.755+0000 7f958b089700 1 -- 172.21.15.89:0/3190375043 >> 172.21.15.89:0/3190375043 conn(0x7f9584066f50 msgr2=0x7f958409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:46.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.755+0000 7f958b089700 1 -- 172.21.15.89:0/3190375043 shutdown_connections 2023-09-30T22:20:46.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.755+0000 7f958b089700 1 -- 172.21.15.89:0/3190375043 wait complete. 2023-09-30T22:20:46.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.756+0000 7f958b089700 1 Processor -- start 2023-09-30T22:20:46.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.756+0000 7f958b089700 1 -- start start 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f958b089700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958406a770 0x7f958413e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f958b089700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958409dc30 0x7f958413a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f958b089700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f958413c1e0 0x7f958413a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f958b089700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958413ee00 con 0x7f958413c1e0 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f958b089700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958413bc40 con 0x7f958409dc30 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f958b089700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9584136cc0 con 0x7f958406a770 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9588e25700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958406a770 0x7f958413e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9583fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958409dc30 0x7f958413a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9588e25700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958406a770 0x7f958413e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59896/0 (socket says 172.21.15.89:59896) 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9588e25700 1 -- 172.21.15.89:0/2098841959 learned_addr learned my addr 172.21.15.89:0/2098841959 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9589626700 1 --2- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f958413c1e0 0x7f958413a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9583fff700 1 -- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958406a770 msgr2=0x7f958413e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9583fff700 1 --2- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958406a770 0x7f958413e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9583fff700 1 -- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f958413c1e0 msgr2=0x7f958413a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9583fff700 1 --2- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f958413c1e0 0x7f958413a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9583fff700 1 -- 172.21.15.89:0/2098841959 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570009550 con 0x7f958409dc30 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f9583fff700 1 --2- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958409dc30 0x7f958413a460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9570016000 tx=0x7f9570004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.757+0000 7f95817fa700 1 -- 172.21.15.89:0/2098841959 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000b9f0 con 0x7f958409dc30 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.758+0000 7f95817fa700 1 -- 172.21.15.89:0/2098841959 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957001f070 con 0x7f958409dc30 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.758+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958409a4e0 con 0x7f958409dc30 2023-09-30T22:20:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.758+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958411d930 con 0x7f958409dc30 2023-09-30T22:20:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.758+0000 7f95817fa700 1 -- 172.21.15.89:0/2098841959 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000bd90 con 0x7f958409dc30 2023-09-30T22:20:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.759+0000 7f95817fa700 1 -- 172.21.15.89:0/2098841959 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9570026480 con 0x7f958409dc30 2023-09-30T22:20:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.759+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95640052e0 con 0x7f958409dc30 2023-09-30T22:20:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.759+0000 7f95817fa700 1 -- 172.21.15.89:0/2098841959 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9570021070 con 0x7f958409dc30 2023-09-30T22:20:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.768+0000 7f95817fa700 1 -- 172.21.15.89:0/2098841959 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9570015030 con 0x7f958409dc30 2023-09-30T22:20:46.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.842+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f76e406b9e0 con 0x7f76e41338a0 2023-09-30T22:20:46.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.842+0000 7f76da7fc700 1 -- 172.21.15.89:0/484294066 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76dc02a090 con 0x7f76e41338a0 2023-09-30T22:20:46.843 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.845+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76e41338a0 msgr2=0x7f76e411f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.845+0000 7f76ec007700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76e41338a0 0x7f76e411f580 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00cc80 tx=0x7f76dc00cf90 comp rx=0 tx=0).stop 2023-09-30T22:20:46.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.845+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 shutdown_connections 2023-09-30T22:20:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.845+0000 7f76ec007700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76e41338a0 0x7f76e411f580 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.845+0000 7f76ec007700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76e409de50 0x7f76e411eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.845+0000 7f76ec007700 1 --2- 172.21.15.89:0/484294066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76e409e760 0x7f76e411f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.845+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 >> 172.21.15.89:0/484294066 conn(0x7f76e40a2aa0 msgr2=0x7f76e409edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:46.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.846+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 shutdown_connections 2023-09-30T22:20:46.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.848+0000 7f76ec007700 1 -- 172.21.15.89:0/484294066 wait complete. 2023-09-30T22:20:46.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:46.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.876+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f35c8121320 con 0x7f35c809d6f0 2023-09-30T22:20:46.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.876+0000 7f35c4ff9700 1 -- 172.21.15.89:0/699726013 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35c8121320 con 0x7f35c809d6f0 2023-09-30T22:20:46.877 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:46.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 msgr2=0x7f35c811dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 --2- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c811dee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f35b8000c00 tx=0x7f35b8004750 comp rx=0 tx=0).stop 2023-09-30T22:20:46.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 shutdown_connections 2023-09-30T22:20:46.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 --2- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c809e410 0x7f35c811e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 --2- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f35c809d6f0 0x7f35c811dee0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 --2- 172.21.15.89:0/699726013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f35c809e000 0x7f35c811e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 >> 172.21.15.89:0/699726013 conn(0x7f35c8066f50 msgr2=0x7f35c806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:46.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 shutdown_connections 2023-09-30T22:20:46.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.879+0000 7f35ce409700 1 -- 172.21.15.89:0/699726013 wait complete. 2023-09-30T22:20:46.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:46.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.946+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc5a8121e00 con 0x7fc5a806a770 2023-09-30T22:20:46.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.946+0000 7fc59dffb700 1 -- 172.21.15.89:0/3222206942 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5a40123b0 con 0x7fc5a806a770 2023-09-30T22:20:46.947 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a806a770 msgr2=0x7fc5a809acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a806a770 0x7fc5a809acf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400c520 tx=0x7fc5a400c830 comp rx=0 tx=0).stop 2023-09-30T22:20:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 shutdown_connections 2023-09-30T22:20:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5a8136d80 0x7fc5a81c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5a806a770 0x7fc5a809acf0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 --2- 172.21.15.89:0/3222206942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5a806b250 0x7fc5a809b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 >> 172.21.15.89:0/3222206942 conn(0x7fc5a8066f50 msgr2=0x7fc5a8134810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:46.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 shutdown_connections 2023-09-30T22:20:46.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.948+0000 7fc5af526700 1 -- 172.21.15.89:0/3222206942 wait complete. 2023-09-30T22:20:46.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:46.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.974+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9564005590 con 0x7f958409dc30 2023-09-30T22:20:46.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.974+0000 7f95817fa700 1 -- 172.21.15.89:0/2098841959 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9570010970 con 0x7f958409dc30 2023-09-30T22:20:46.975 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.976+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958409dc30 msgr2=0x7f958413a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.977+0000 7f958b089700 1 --2- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958409dc30 0x7f958413a460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9570016000 tx=0x7f9570004750 comp rx=0 tx=0).stop 2023-09-30T22:20:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.977+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 shutdown_connections 2023-09-30T22:20:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.977+0000 7f958b089700 1 --2- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f958413c1e0 0x7f958413a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.977+0000 7f958b089700 1 --2- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f958406a770 0x7f958413e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.977+0000 7f958b089700 1 --2- 172.21.15.89:0/2098841959 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f958409dc30 0x7f958413a460 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.977+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 >> 172.21.15.89:0/2098841959 conn(0x7f9584066f50 msgr2=0x7f9584072260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.977+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 shutdown_connections 2023-09-30T22:20:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:46.977+0000 7f958b089700 1 -- 172.21.15.89:0/2098841959 wait complete. 2023-09-30T22:20:46.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:47.866 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:47.897 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:47.961 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:47.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:48.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.000+0000 7fb7c50f1700 1 Processor -- start 2023-09-30T22:20:48.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.000+0000 7fb7c50f1700 1 -- start start 2023-09-30T22:20:48.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7c50f1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7c009d6f0 0x7fb7c009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7c50f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7c50f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009e410 0x7fb7c0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7c50f1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c006b460 con 0x7fb7c009d6f0 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7c50f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c009b120 con 0x7fb7c009e410 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7c50f1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c009a7e0 con 0x7fb7c009e000 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7be59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7bed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7c009d6f0 0x7fb7c009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7be59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59908/0 (socket says 172.21.15.89:59908) 2023-09-30T22:20:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7be59c700 1 -- 172.21.15.89:0/4078252124 learned_addr learned my addr 172.21.15.89:0/4078252124 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7bf59e700 1 --2- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009e410 0x7fb7c0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7be59c700 1 -- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009e410 msgr2=0x7fb7c0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7be59c700 1 --2- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009e410 0x7fb7c0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7be59c700 1 -- 172.21.15.89:0/4078252124 >> v2:172.21.15.89:3300/0 conn(0x7fb7c009d6f0 msgr2=0x7fb7c009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7be59c700 1 --2- 172.21.15.89:0/4078252124 >> v2:172.21.15.89:3300/0 conn(0x7fb7c009d6f0 0x7fb7c009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7be59c700 1 -- 172.21.15.89:0/4078252124 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c009a500 con 0x7fb7c009e000 2023-09-30T22:20:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7bed9d700 1 --2- 172.21.15.89:0/4078252124 >> v2:172.21.15.89:3300/0 conn(0x7fb7c009d6f0 0x7fb7c009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.001+0000 7fb7bf59e700 1 --2- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009e410 0x7fb7c0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:48.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.002+0000 7fb7be59c700 1 --2- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01339f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009920 tx=0x7fb7b4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd89a9413b6a410b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:48.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.002+0000 7fb7bdd9b700 1 -- 172.21.15.89:0/4078252124 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4011830 con 0x7fb7c009e000 2023-09-30T22:20:48.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.002+0000 7fb7bdd9b700 1 -- 172.21.15.89:0/4078252124 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b4011990 con 0x7fb7c009e000 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.002+0000 7fb7bdd9b700 1 -- 172.21.15.89:0/4078252124 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4011d20 con 0x7fb7c009e000 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.003+0000 7fb7c50f1700 1 -- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 msgr2=0x7fb7c01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.003+0000 7fb7c50f1700 1 --2- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01339f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009920 tx=0x7fb7b4009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.003+0000 7fb7c50f1700 1 -- 172.21.15.89:0/4078252124 shutdown_connections 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.003+0000 7fb7c50f1700 1 --2- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009e410 0x7fb7c0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.003+0000 7fb7c50f1700 1 --2- 172.21.15.89:0/4078252124 >> v2:172.21.15.89:3300/0 conn(0x7fb7c009d6f0 0x7fb7c009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.003+0000 7fb7c50f1700 1 --2- 172.21.15.89:0/4078252124 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01339f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.003+0000 7fb7c50f1700 1 -- 172.21.15.89:0/4078252124 >> 172.21.15.89:0/4078252124 conn(0x7fb7c0066f50 msgr2=0x7fb7c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.004+0000 7fb7c50f1700 1 -- 172.21.15.89:0/4078252124 shutdown_connections 2023-09-30T22:20:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.004+0000 7fb7c50f1700 1 -- 172.21.15.89:0/4078252124 wait complete. 2023-09-30T22:20:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.005+0000 7fb7c50f1700 1 Processor -- start 2023-09-30T22:20:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.005+0000 7fb7c50f1700 1 -- start start 2023-09-30T22:20:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7c50f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009d6f0 0x7fb7c01a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7c50f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7c50f1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c009e410 0x7fb7c01a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7c50f1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c01a95b0 con 0x7fb7c009e410 2023-09-30T22:20:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7c50f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00a2190 con 0x7fb7c009d6f0 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7c50f1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c01a7a90 con 0x7fb7c009e000 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7bed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009d6f0 0x7fb7c01a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7bed9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009d6f0 0x7fb7c01a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60786/0 (socket says 172.21.15.89:60786) 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7bed9d700 1 -- 172.21.15.89:0/172517535 learned_addr learned my addr 172.21.15.89:0/172517535 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7be59c700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.006+0000 7fb7bf59e700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c009e410 0x7fb7c01a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7bed9d700 1 -- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 msgr2=0x7fb7c01a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7bed9d700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7bed9d700 1 -- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c009e410 msgr2=0x7fb7c01a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7bed9d700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c009e410 0x7fb7c01a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7bed9d700 1 -- 172.21.15.89:0/172517535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00094a0 con 0x7fb7c009d6f0 2023-09-30T22:20:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7bed9d700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009d6f0 0x7fb7c01a5e30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000ec10 tx=0x7fb7b000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7a77fe700 1 -- 172.21.15.89:0/172517535 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000c910 con 0x7fb7c009d6f0 2023-09-30T22:20:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7a77fe700 1 -- 172.21.15.89:0/172517535 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b000ca70 con 0x7fb7c009d6f0 2023-09-30T22:20:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b8009a70 con 0x7fb7c009d6f0 2023-09-30T22:20:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7a77fe700 1 -- 172.21.15.89:0/172517535 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000cde0 con 0x7fb7c009d6f0 2023-09-30T22:20:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.007+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b4009550 con 0x7fb7c009d6f0 2023-09-30T22:20:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.008+0000 7fb7a77fe700 1 -- 172.21.15.89:0/172517535 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb7b0013470 con 0x7fb7c009d6f0 2023-09-30T22:20:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.008+0000 7fb7a77fe700 1 -- 172.21.15.89:0/172517535 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb7b003f9b0 con 0x7fb7c009d6f0 2023-09-30T22:20:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.008+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c0074ea0 con 0x7fb7c009d6f0 2023-09-30T22:20:48.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.017+0000 7fb7a77fe700 1 -- 172.21.15.89:0/172517535 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb7b00232f0 con 0x7fb7c009d6f0 2023-09-30T22:20:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.048+0000 7f045a519700 1 Processor -- start 2023-09-30T22:20:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.048+0000 7f045a519700 1 -- start start 2023-09-30T22:20:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.048+0000 7f045a519700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f045409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.048+0000 7f045a519700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f045409e000 0x7f04541339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.048+0000 7f045a519700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045409b120 con 0x7f045409e410 2023-09-30T22:20:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.048+0000 7f045a519700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f045406b460 con 0x7f045409e000 2023-09-30T22:20:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.048+0000 7f045a519700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f045409a7e0 con 0x7f045409d6f0 2023-09-30T22:20:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f04537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f045409e000 0x7f04541339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0453fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f045409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0458ab6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f045409e410 0x7f0454133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58258/0 (socket says 172.21.15.89:58258) 2023-09-30T22:20:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0458ab6700 1 -- 172.21.15.89:0/4081238454 learned_addr learned my addr 172.21.15.89:0/4081238454 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0453fff700 1 -- 172.21.15.89:0/4081238454 >> v2:172.21.15.89:3300/0 conn(0x7f045409e000 msgr2=0x7f04541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0453fff700 1 --2- 172.21.15.89:0/4081238454 >> v2:172.21.15.89:3300/0 conn(0x7f045409e000 0x7f04541339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0453fff700 1 -- 172.21.15.89:0/4081238454 >> v1:172.21.15.89:6789/0 conn(0x7f045409e410 legacy=0x7f0454133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0453fff700 1 -- 172.21.15.89:0/4081238454 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045409a500 con 0x7f045409d6f0 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0452ffd700 1 -- 172.21.15.89:0/4081238454 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521310540 0 0) 0x7f045409b120 con 0x7f045409e410 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f04537fe700 1 --2- 172.21.15.89:0/4081238454 >> v2:172.21.15.89:3300/0 conn(0x7f045409e000 0x7f04541339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0453fff700 1 --2- 172.21.15.89:0/4081238454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f045409dac0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0444009870 tx=0x7f0444009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60835e8c9f5ce55b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0452ffd700 1 -- 172.21.15.89:0/4081238454 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444011830 con 0x7f045409d6f0 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0452ffd700 1 -- 172.21.15.89:0/4081238454 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0444011990 con 0x7f045409d6f0 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.049+0000 7f0452ffd700 1 -- 172.21.15.89:0/4081238454 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444011d20 con 0x7f045409d6f0 2023-09-30T22:20:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.050+0000 7f045a519700 1 -- 172.21.15.89:0/4081238454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 msgr2=0x7f045409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.050+0000 7f045a519700 1 --2- 172.21.15.89:0/4081238454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f045409dac0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0444009870 tx=0x7f0444009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.050+0000 7f045a519700 1 -- 172.21.15.89:0/4081238454 shutdown_connections 2023-09-30T22:20:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.050+0000 7f045a519700 1 --2- 172.21.15.89:0/4081238454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f045409dac0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.050+0000 7f045a519700 1 --2- 172.21.15.89:0/4081238454 >> v2:172.21.15.89:3300/0 conn(0x7f045409e000 0x7f04541339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.050+0000 7f045a519700 1 -- 172.21.15.89:0/4081238454 >> 172.21.15.89:0/4081238454 conn(0x7f0454066f50 msgr2=0x7f045409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.051+0000 7f045a519700 1 -- 172.21.15.89:0/4081238454 shutdown_connections 2023-09-30T22:20:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.051+0000 7f045a519700 1 -- 172.21.15.89:0/4081238454 wait complete. 2023-09-30T22:20:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.052+0000 7f045a519700 1 Processor -- start 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.052+0000 7f045a519700 1 -- start start 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f045a519700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f04541a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f045a519700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f045409e000 0x7f04541a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f045a519700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f045409e410 0x7f04541c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f045a519700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04541a8270 con 0x7f045409e000 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f045a519700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04541a6e30 con 0x7f045409e410 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f045a519700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04541a8450 con 0x7f045409d6f0 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f045409e000 0x7f04541a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f045409e000 0x7f04541a5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36850/0 (socket says 172.21.15.89:36850) 2023-09-30T22:20:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 -- 172.21.15.89:0/1503650961 learned_addr learned my addr 172.21.15.89:0/1503650961 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f0453fff700 1 --2- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f04541a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 -- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 msgr2=0x7f04541a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 --2- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f04541a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 -- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f045409e410 msgr2=0x7f04541c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 --2- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f045409e410 0x7f04541c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 -- 172.21.15.89:0/1503650961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04480094a0 con 0x7f045409e000 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.053+0000 7f04537fe700 1 --2- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f045409e000 0x7f04541a5a70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f044800ebe0 tx=0x7f044800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.054+0000 7f0450ff9700 1 -- 172.21.15.89:0/1503650961 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044800c910 con 0x7f045409e000 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.054+0000 7f0450ff9700 1 -- 172.21.15.89:0/1503650961 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044800ca70 con 0x7f045409e000 2023-09-30T22:20:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.054+0000 7f0450ff9700 1 -- 172.21.15.89:0/1503650961 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044800ce00 con 0x7f045409e000 2023-09-30T22:20:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.054+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04440094a0 con 0x7f045409e000 2023-09-30T22:20:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.054+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045409c710 con 0x7f045409e000 2023-09-30T22:20:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.055+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0454074ea0 con 0x7f045409e000 2023-09-30T22:20:48.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.056+0000 7f0450ff9700 1 -- 172.21.15.89:0/1503650961 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0448013470 con 0x7f045409e000 2023-09-30T22:20:48.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.056+0000 7f0450ff9700 1 -- 172.21.15.89:0/1503650961 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0448040110 con 0x7f045409e000 2023-09-30T22:20:48.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.065+0000 7f0450ff9700 1 -- 172.21.15.89:0/1503650961 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0448011760 con 0x7f045409e000 2023-09-30T22:20:48.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.094+0000 7fed68da1700 1 Processor -- start 2023-09-30T22:20:48.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.094+0000 7fed68da1700 1 -- start start 2023-09-30T22:20:48.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed68da1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6409dba0 0x7fed6409df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed68da1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed6412f8c0 0x7fed6412fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed68da1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6409d670 con 0x7fed64130210 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed68da1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed64099d80 con 0x7fed6412f8c0 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed68da1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6409b020 con 0x7fed6409dba0 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6409dba0 0x7fed6409df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed63fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fed64130210 0x7fed64136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58266/0 (socket says 172.21.15.89:58266) 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed63fff700 1 -- 172.21.15.89:0/3879252108 learned_addr learned my addr 172.21.15.89:0/3879252108 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.095+0000 7fed62ffd700 1 --2- 172.21.15.89:0/3879252108 >> v2:172.21.15.89:3300/0 conn(0x7fed6412f8c0 0x7fed6412fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.096+0000 7fed637fe700 1 -- 172.21.15.89:0/3879252108 >> v2:172.21.15.89:3300/0 conn(0x7fed6412f8c0 msgr2=0x7fed6412fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.096+0000 7fed637fe700 1 --2- 172.21.15.89:0/3879252108 >> v2:172.21.15.89:3300/0 conn(0x7fed6412f8c0 0x7fed6412fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.096+0000 7fed637fe700 1 -- 172.21.15.89:0/3879252108 >> v1:172.21.15.89:6789/0 conn(0x7fed64130210 legacy=0x7fed64136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.096+0000 7fed637fe700 1 -- 172.21.15.89:0/3879252108 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6409a440 con 0x7fed6409dba0 2023-09-30T22:20:48.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.096+0000 7fed637fe700 1 --2- 172.21.15.89:0/3879252108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6409dba0 0x7fed6409df70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fed54009820 tx=0x7fed54009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791a5e8c8002fe94 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.096+0000 7fed627fc700 1 -- 172.21.15.89:0/3879252108 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54005e30 con 0x7fed6409dba0 2023-09-30T22:20:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.096+0000 7fed627fc700 1 -- 172.21.15.89:0/3879252108 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54006030 con 0x7fed6409dba0 2023-09-30T22:20:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.096+0000 7fed627fc700 1 -- 172.21.15.89:0/3879252108 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed540063c0 con 0x7fed6409dba0 2023-09-30T22:20:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.097+0000 7fed68da1700 1 -- 172.21.15.89:0/3879252108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6409dba0 msgr2=0x7fed6409df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.097+0000 7fed68da1700 1 --2- 172.21.15.89:0/3879252108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6409dba0 0x7fed6409df70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fed54009820 tx=0x7fed54009b30 comp rx=0 tx=0).stop 2023-09-30T22:20:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.097+0000 7fed68da1700 1 -- 172.21.15.89:0/3879252108 shutdown_connections 2023-09-30T22:20:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.097+0000 7fed68da1700 1 --2- 172.21.15.89:0/3879252108 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6409dba0 0x7fed6409df70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.097+0000 7fed68da1700 1 --2- 172.21.15.89:0/3879252108 >> v2:172.21.15.89:3300/0 conn(0x7fed6412f8c0 0x7fed6412fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.097+0000 7fed68da1700 1 -- 172.21.15.89:0/3879252108 >> 172.21.15.89:0/3879252108 conn(0x7fed640a2540 msgr2=0x7fed640a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.097+0000 7fed68da1700 1 -- 172.21.15.89:0/3879252108 shutdown_connections 2023-09-30T22:20:48.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.097+0000 7fed68da1700 1 -- 172.21.15.89:0/3879252108 wait complete. 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed68da1700 1 Processor -- start 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed68da1700 1 -- start start 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed68da1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed6409dba0 0x7fed641b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed68da1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6412f8c0 0x7fed641b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed68da1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64130210 0x7fed641b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed68da1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6412f5f0 con 0x7fed64130210 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed68da1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed641ba5f0 con 0x7fed6412f8c0 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed68da1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed64127d70 con 0x7fed6409dba0 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed6409dba0 0x7fed641b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed6409dba0 0x7fed641b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59956/0 (socket says 172.21.15.89:59956) 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed637fe700 1 -- 172.21.15.89:0/4120335110 learned_addr learned my addr 172.21.15.89:0/4120335110 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:48.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.098+0000 7fed62ffd700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6412f8c0 0x7fed641b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed63fff700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64130210 0x7fed641b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed637fe700 1 -- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6412f8c0 msgr2=0x7fed641b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed637fe700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6412f8c0 0x7fed641b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed637fe700 1 -- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64130210 msgr2=0x7fed641b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed637fe700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64130210 0x7fed641b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed637fe700 1 -- 172.21.15.89:0/4120335110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54009430 con 0x7fed6409dba0 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed637fe700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed6409dba0 0x7fed641b76a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fed54000c00 tx=0x7fed54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed4bfff700 1 -- 172.21.15.89:0/4120335110 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54004240 con 0x7fed6409dba0 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed4bfff700 1 -- 172.21.15.89:0/4120335110 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed540043a0 con 0x7fed6409dba0 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed64127620 con 0x7fed6409dba0 2023-09-30T22:20:48.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed4bfff700 1 -- 172.21.15.89:0/4120335110 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed540036a0 con 0x7fed6409dba0 2023-09-30T22:20:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.099+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed641b8e60 con 0x7fed6409dba0 2023-09-30T22:20:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.100+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6404f2a0 con 0x7fed6409dba0 2023-09-30T22:20:48.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.100+0000 7fed4bfff700 1 -- 172.21.15.89:0/4120335110 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fed54022070 con 0x7fed6409dba0 2023-09-30T22:20:48.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.109+0000 7fed4bfff700 1 -- 172.21.15.89:0/4120335110 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fed54003800 con 0x7fed6409dba0 2023-09-30T22:20:48.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.109+0000 7fed4bfff700 1 -- 172.21.15.89:0/4120335110 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fed641b8e60 con 0x7fed6409dba0 2023-09-30T22:20:48.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.133+0000 7f3634758700 1 Processor -- start 2023-09-30T22:20:48.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.133+0000 7f3634758700 1 -- start start 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3634758700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09d6f0 0x7f362c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3634758700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f362c09e410 0x7f362c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3634758700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c09f130 con 0x7f362c09e000 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3634758700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c09f520 con 0x7f362c09e410 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3634758700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f362c09e930 con 0x7f362c09d6f0 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3632cf5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f362c09e410 0x7f362c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3631cf3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f362c09e000 0x7f362c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58276/0 (socket says 172.21.15.89:58276) 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3632cf5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f362c09e410 0x7f362c130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36872/0 (socket says 172.21.15.89:36872) 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f3631cf3700 1 -- 172.21.15.89:0/3963576618 learned_addr learned my addr 172.21.15.89:0/3963576618 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.134+0000 7f36324f4700 1 --2- 172.21.15.89:0/3963576618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09d6f0 0x7f362c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f3632cf5700 1 -- 172.21.15.89:0/3963576618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09d6f0 msgr2=0x7f362c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f3632cf5700 1 --2- 172.21.15.89:0/3963576618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09d6f0 0x7f362c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f3632cf5700 1 -- 172.21.15.89:0/3963576618 >> v1:172.21.15.89:6789/0 conn(0x7f362c09e000 legacy=0x7f362c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f3632cf5700 1 -- 172.21.15.89:0/3963576618 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c09b2d0 con 0x7f362c09e410 2023-09-30T22:20:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f36314f2700 1 -- 172.21.15.89:0/3963576618 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343323831 0 0) 0x7f362c09f130 con 0x7f362c09e000 2023-09-30T22:20:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f3632cf5700 1 --2- 172.21.15.89:0/3963576618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e410 0x7f362c130d70 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3624009d90 tx=0x7f362400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a0f9323e0faaac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f36314f2700 1 -- 172.21.15.89:0/3963576618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362400e070 con 0x7f362c09e410 2023-09-30T22:20:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f36314f2700 1 -- 172.21.15.89:0/3963576618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362400bbd0 con 0x7f362c09e410 2023-09-30T22:20:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.135+0000 7f36314f2700 1 -- 172.21.15.89:0/3963576618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624012430 con 0x7f362c09e410 2023-09-30T22:20:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.136+0000 7f3634758700 1 -- 172.21.15.89:0/3963576618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e410 msgr2=0x7f362c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.136+0000 7f3634758700 1 --2- 172.21.15.89:0/3963576618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e410 0x7f362c130d70 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3624009d90 tx=0x7f362400b330 comp rx=0 tx=0).stop 2023-09-30T22:20:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.136+0000 7f3634758700 1 -- 172.21.15.89:0/3963576618 shutdown_connections 2023-09-30T22:20:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.136+0000 7f3634758700 1 --2- 172.21.15.89:0/3963576618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e410 0x7f362c130d70 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.136+0000 7f3634758700 1 --2- 172.21.15.89:0/3963576618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09d6f0 0x7f362c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.136+0000 7f3634758700 1 -- 172.21.15.89:0/3963576618 >> 172.21.15.89:0/3963576618 conn(0x7f362c066f50 msgr2=0x7f362c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.136+0000 7f3634758700 1 -- 172.21.15.89:0/3963576618 shutdown_connections 2023-09-30T22:20:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.136+0000 7f3634758700 1 -- 172.21.15.89:0/3963576618 wait complete. 2023-09-30T22:20:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.137+0000 7f3634758700 1 Processor -- start 2023-09-30T22:20:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f3634758700 1 -- start start 2023-09-30T22:20:48.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f3634758700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f362c09d6f0 0x7f362c120220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f3634758700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e000 0x7f362c120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f3634758700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09e410 0x7f362c120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:48.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f3634758700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c123ca0 con 0x7f362c09e000 2023-09-30T22:20:48.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f3634758700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1223b0 con 0x7f362c09d6f0 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f3634758700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f362c09c700 con 0x7f362c09e410 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f36324f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f362c09d6f0 0x7f362c120220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f36324f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f362c09d6f0 0x7f362c120220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60816/0 (socket says 172.21.15.89:60816) 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.138+0000 7f36324f4700 1 -- 172.21.15.89:0/2575590702 learned_addr learned my addr 172.21.15.89:0/2575590702 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f3631cf3700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e000 0x7f362c120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f3632cf5700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09e410 0x7f362c120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f36324f4700 1 -- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09e410 msgr2=0x7f362c120ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f36324f4700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09e410 0x7f362c120ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f36324f4700 1 -- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e000 msgr2=0x7f362c120760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f36324f4700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e000 0x7f362c120760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f36324f4700 1 -- 172.21.15.89:0/2575590702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624009a70 con 0x7f362c09d6f0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f3632cf5700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09e410 0x7f362c120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f3631cf3700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e000 0x7f362c120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f36324f4700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f362c09d6f0 0x7f362c120220 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f362800d800 tx=0x7f362800db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f3622ffd700 1 -- 172.21.15.89:0/2575590702 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36280125f0 con 0x7f362c09d6f0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f3622ffd700 1 -- 172.21.15.89:0/2575590702 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3628012750 con 0x7f362c09d6f0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.139+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c121d90 con 0x7f362c09d6f0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.140+0000 7f3622ffd700 1 -- 172.21.15.89:0/2575590702 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628012b00 con 0x7f362c09d6f0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.140+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c124430 con 0x7f362c09d6f0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.141+0000 7f3622ffd700 1 -- 172.21.15.89:0/2575590702 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3628012c60 con 0x7f362c09d6f0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.141+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c074ea0 con 0x7f362c09d6f0 2023-09-30T22:20:48.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.141+0000 7f3622ffd700 1 -- 172.21.15.89:0/2575590702 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f362801e640 con 0x7f362c09d6f0 2023-09-30T22:20:48.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.150+0000 7f3622ffd700 1 -- 172.21.15.89:0/2575590702 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3628014430 con 0x7f362c09d6f0 2023-09-30T22:20:48.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.261+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb7c01a8bd0 con 0x7fb7c009d6f0 2023-09-30T22:20:48.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.261+0000 7fb7a77fe700 1 -- 172.21.15.89:0/172517535 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb7b002a090 con 0x7fb7c009d6f0 2023-09-30T22:20:48.262 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.264+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009d6f0 msgr2=0x7fb7c01a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.264+0000 7fb7c50f1700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009d6f0 0x7fb7c01a5e30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000ec10 tx=0x7fb7b000ef20 comp rx=0 tx=0).stop 2023-09-30T22:20:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.264+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 shutdown_connections 2023-09-30T22:20:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.264+0000 7fb7c50f1700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c009e410 0x7fb7c01a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.264+0000 7fb7c50f1700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7c009d6f0 0x7fb7c01a5e30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.264+0000 7fb7c50f1700 1 --2- 172.21.15.89:0/172517535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7c009e000 0x7fb7c01a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.264+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 >> 172.21.15.89:0/172517535 conn(0x7fb7c0066f50 msgr2=0x7fb7c01bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.265+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 shutdown_connections 2023-09-30T22:20:48.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.265+0000 7fb7c50f1700 1 -- 172.21.15.89:0/172517535 wait complete. 2023-09-30T22:20:48.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.299+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f04541b92c0 con 0x7f045409e000 2023-09-30T22:20:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.299+0000 7f0450ff9700 1 -- 172.21.15.89:0/1503650961 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0448021020 con 0x7f045409e000 2023-09-30T22:20:48.300 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f045409e000 msgr2=0x7f04541a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 --2- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f045409e000 0x7f04541a5a70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f044800ebe0 tx=0x7f044800eef0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 shutdown_connections 2023-09-30T22:20:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 --2- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f045409e410 0x7f04541c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 --2- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f045409d6f0 0x7f04541a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 --2- 172.21.15.89:0/1503650961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f045409e000 0x7f04541a5a70 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 >> 172.21.15.89:0/1503650961 conn(0x7f0454066f50 msgr2=0x7f04540a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 shutdown_connections 2023-09-30T22:20:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.301+0000 7f045a519700 1 -- 172.21.15.89:0/1503650961 wait complete. 2023-09-30T22:20:48.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.337+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fed641b8e60 con 0x7fed6409dba0 2023-09-30T22:20:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.337+0000 7fed4bfff700 1 -- 172.21.15.89:0/4120335110 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fed641b8e60 con 0x7fed6409dba0 2023-09-30T22:20:48.338 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed6409dba0 msgr2=0x7fed641b76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed6409dba0 0x7fed641b76a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fed54000c00 tx=0x7fed54004750 comp rx=0 tx=0).stop 2023-09-30T22:20:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 shutdown_connections 2023-09-30T22:20:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed64130210 0x7fed641b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed6409dba0 0x7fed641b76a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 --2- 172.21.15.89:0/4120335110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed6412f8c0 0x7fed641b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 >> 172.21.15.89:0/4120335110 conn(0x7fed640a2540 msgr2=0x7fed6412d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 shutdown_connections 2023-09-30T22:20:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.339+0000 7fed68da1700 1 -- 172.21.15.89:0/4120335110 wait complete. 2023-09-30T22:20:48.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:48.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.365+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f362c06c360 con 0x7f362c09d6f0 2023-09-30T22:20:48.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.365+0000 7f3622ffd700 1 -- 172.21.15.89:0/2575590702 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3628011340 con 0x7f362c09d6f0 2023-09-30T22:20:48.366 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:48.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.367+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f362c09d6f0 msgr2=0x7f362c120220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:48.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.367+0000 7f3634758700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f362c09d6f0 0x7f362c120220 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f362800d800 tx=0x7f362800db10 comp rx=0 tx=0).stop 2023-09-30T22:20:48.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.368+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 shutdown_connections 2023-09-30T22:20:48.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.368+0000 7f3634758700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f362c09e410 0x7f362c120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.368+0000 7f3634758700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f362c09d6f0 0x7f362c120220 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.368+0000 7f3634758700 1 --2- 172.21.15.89:0/2575590702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362c09e000 0x7f362c120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:48.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.368+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 >> 172.21.15.89:0/2575590702 conn(0x7f362c066f50 msgr2=0x7f362c0710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:48.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.368+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 shutdown_connections 2023-09-30T22:20:48.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:48.368+0000 7f3634758700 1 -- 172.21.15.89:0/2575590702 wait complete. 2023-09-30T22:20:48.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:49.283 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:49.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:49.353 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:49.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:49.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.423+0000 7f9b85689700 1 Processor -- start 2023-09-30T22:20:49.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.423+0000 7f9b85689700 1 -- start start 2023-09-30T22:20:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b85689700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b801316c0 0x7f9b80131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b85689700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b8006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b85689700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8009e130 con 0x7f9b8006acb0 2023-09-30T22:20:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b85689700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b801312b0 con 0x7f9b801316c0 2023-09-30T22:20:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b85689700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8009f6e0 con 0x7f9b8009d9f0 2023-09-30T22:20:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b8006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b801316c0 0x7f9b80131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b8006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60832/0 (socket says 172.21.15.89:60832) 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7e7fc700 1 -- 172.21.15.89:0/2466590794 learned_addr learned my addr 172.21.15.89:0/2466590794 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7e7fc700 1 -- 172.21.15.89:0/2466590794 >> v2:172.21.15.89:3300/0 conn(0x7f9b801316c0 msgr2=0x7f9b80131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7e7fc700 1 --2- 172.21.15.89:0/2466590794 >> v2:172.21.15.89:3300/0 conn(0x7f9b801316c0 0x7f9b80131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7e7fc700 1 -- 172.21.15.89:0/2466590794 >> v1:172.21.15.89:6789/0 conn(0x7f9b8006acb0 legacy=0x7f9b80138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7e7fc700 1 -- 172.21.15.89:0/2466590794 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8009e760 con 0x7f9b8009d9f0 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.424+0000 7f9b7effd700 1 --2- 172.21.15.89:0/2466590794 >> v2:172.21.15.89:3300/0 conn(0x7f9b801316c0 0x7f9b80131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.425+0000 7f9b7e7fc700 1 --2- 172.21.15.89:0/2466590794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b8006a770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9b740098f0 tx=0x7f9b74009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a3f576680465db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.425+0000 7f9b7dffb700 1 -- 172.21.15.89:0/2466590794 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74011830 con 0x7f9b8009d9f0 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.425+0000 7f9b7dffb700 1 -- 172.21.15.89:0/2466590794 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b74011990 con 0x7f9b8009d9f0 2023-09-30T22:20:49.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.425+0000 7f9b7dffb700 1 -- 172.21.15.89:0/2466590794 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74011d20 con 0x7f9b8009d9f0 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.426+0000 7f9b85689700 1 -- 172.21.15.89:0/2466590794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 msgr2=0x7f9b8006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.426+0000 7f9b85689700 1 --2- 172.21.15.89:0/2466590794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b8006a770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9b740098f0 tx=0x7f9b74009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.426+0000 7f9b85689700 1 -- 172.21.15.89:0/2466590794 shutdown_connections 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.426+0000 7f9b85689700 1 --2- 172.21.15.89:0/2466590794 >> v2:172.21.15.89:3300/0 conn(0x7f9b801316c0 0x7f9b80131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.426+0000 7f9b85689700 1 --2- 172.21.15.89:0/2466590794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b8006a770 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.426+0000 7f9b85689700 1 -- 172.21.15.89:0/2466590794 >> 172.21.15.89:0/2466590794 conn(0x7f9b80066f50 msgr2=0x7f9b8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.426+0000 7f9b85689700 1 -- 172.21.15.89:0/2466590794 shutdown_connections 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.426+0000 7f9b85689700 1 -- 172.21.15.89:0/2466590794 wait complete. 2023-09-30T22:20:49.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b85689700 1 Processor -- start 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b85689700 1 -- start start 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b85689700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8006acb0 0x7f9b80119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b85689700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b801199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b85689700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b801316c0 0x7f9b80119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b85689700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80072370 con 0x7f9b801316c0 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b85689700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8011b850 con 0x7f9b8009d9f0 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b85689700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8011bed0 con 0x7f9b8006acb0 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b7effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8006acb0 0x7f9b80119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b7effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8006acb0 0x7f9b80119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59990/0 (socket says 172.21.15.89:59990) 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.427+0000 7f9b7effd700 1 -- 172.21.15.89:0/2406858363 learned_addr learned my addr 172.21.15.89:0/2406858363 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b7effd700 1 -- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 msgr2=0x7f9b801199c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:49.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b7effd700 1 --2- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b801199c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b7effd700 1 -- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b801316c0 msgr2=0x7f9b80119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b7effd700 1 --2- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b801316c0 0x7f9b80119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b7effd700 1 -- 172.21.15.89:0/2406858363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7000d940 con 0x7f9b8006acb0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b7effd700 1 --2- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8006acb0 0x7f9b80119480 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9b70012fc0 tx=0x7f9b700103a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b677fe700 1 -- 172.21.15.89:0/2406858363 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70010bb0 con 0x7f9b8006acb0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b677fe700 1 -- 172.21.15.89:0/2406858363 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b70013040 con 0x7f9b8006acb0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b677fe700 1 -- 172.21.15.89:0/2406858363 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7000d2e0 con 0x7f9b8006acb0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b74009550 con 0x7f9b8006acb0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.428+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8009ead0 con 0x7f9b8006acb0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.429+0000 7f9b677fe700 1 -- 172.21.15.89:0/2406858363 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b70018480 con 0x7f9b8006acb0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.429+0000 7f9b677fe700 1 -- 172.21.15.89:0/2406858363 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b700439c0 con 0x7f9b8006acb0 2023-09-30T22:20:49.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.429+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b80074ea0 con 0x7f9b8006acb0 2023-09-30T22:20:49.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.435+0000 7f9b677fe700 1 -- 172.21.15.89:0/2406858363 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b70016930 con 0x7f9b8006acb0 2023-09-30T22:20:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.459+0000 7fdec770b700 1 Processor -- start 2023-09-30T22:20:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.459+0000 7fdec770b700 1 -- start start 2023-09-30T22:20:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec770b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009e1d0 0x7fdec0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec770b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec0139170 0x7fdec013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec770b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec009b120 con 0x7fdec009d6f0 2023-09-30T22:20:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec770b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec006b460 con 0x7fdec0139170 2023-09-30T22:20:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec770b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdec009a7e0 con 0x7fdec009e1d0 2023-09-30T22:20:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec54a7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdec009d6f0 0x7fdec009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58282/0 (socket says 172.21.15.89:58282) 2023-09-30T22:20:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec54a7700 1 -- 172.21.15.89:0/3470299095 learned_addr learned my addr 172.21.15.89:0/3470299095 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec4ca6700 1 --2- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009e1d0 0x7fdec0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec5ca8700 1 --2- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec0139170 0x7fdec013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdeb7fff700 1 -- 172.21.15.89:0/3470299095 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186249694 0 0) 0x7fdec009b120 con 0x7fdec009d6f0 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdeb7fff700 1 -- 172.21.15.89:0/3470299095 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeac0035b0 con 0x7fdec009d6f0 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec4ca6700 1 -- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec0139170 msgr2=0x7fdec013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec4ca6700 1 --2- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec0139170 0x7fdec013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec4ca6700 1 -- 172.21.15.89:0/3470299095 >> v1:172.21.15.89:6789/0 conn(0x7fdec009d6f0 legacy=0x7fdec009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.460+0000 7fdec4ca6700 1 -- 172.21.15.89:0/3470299095 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec009a500 con 0x7fdec009e1d0 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdec4ca6700 1 --2- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009e1d0 0x7fdec0138c30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00098a0 tx=0x7fdeb0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd406e932d5eb7c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdeb7fff700 1 -- 172.21.15.89:0/3470299095 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0004230 con 0x7fdec009e1d0 2023-09-30T22:20:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdeb7fff700 1 -- 172.21.15.89:0/3470299095 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb0004390 con 0x7fdec009e1d0 2023-09-30T22:20:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdeb7fff700 1 -- 172.21.15.89:0/3470299095 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb00036a0 con 0x7fdec009e1d0 2023-09-30T22:20:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdec770b700 1 -- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009e1d0 msgr2=0x7fdec0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdec770b700 1 --2- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009e1d0 0x7fdec0138c30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00098a0 tx=0x7fdeb0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdec770b700 1 -- 172.21.15.89:0/3470299095 shutdown_connections 2023-09-30T22:20:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdec770b700 1 --2- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec0139170 0x7fdec013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdec770b700 1 --2- 172.21.15.89:0/3470299095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009e1d0 0x7fdec0138c30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.461+0000 7fdec770b700 1 -- 172.21.15.89:0/3470299095 >> 172.21.15.89:0/3470299095 conn(0x7fdec0066f50 msgr2=0x7fdec009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.462+0000 7fdec770b700 1 -- 172.21.15.89:0/3470299095 shutdown_connections 2023-09-30T22:20:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.462+0000 7fdec770b700 1 -- 172.21.15.89:0/3470299095 wait complete. 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.462+0000 7fdec770b700 1 Processor -- start 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec770b700 1 -- start start 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec770b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009d6f0 0x7fdec0135b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec770b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec009e1d0 0x7fdec0136080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec770b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec0139170 0x7fdec01365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec770b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec009c710 con 0x7fdec0139170 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec770b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00a3a90 con 0x7fdec009e1d0 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec770b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdec0138df0 con 0x7fdec009d6f0 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec5ca8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec0139170 0x7fdec01365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec5ca8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec0139170 0x7fdec01365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36906/0 (socket says 172.21.15.89:36906) 2023-09-30T22:20:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec5ca8700 1 -- 172.21.15.89:0/2378832775 learned_addr learned my addr 172.21.15.89:0/2378832775 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec54a7700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009d6f0 0x7fdec0135b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec4ca6700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec009e1d0 0x7fdec0136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec5ca8700 1 -- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009d6f0 msgr2=0x7fdec0135b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec5ca8700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009d6f0 0x7fdec0135b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec5ca8700 1 -- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec009e1d0 msgr2=0x7fdec0136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec5ca8700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec009e1d0 0x7fdec0136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.463+0000 7fdec5ca8700 1 -- 172.21.15.89:0/2378832775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb00094a0 con 0x7fdec0139170 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.464+0000 7fdec5ca8700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec0139170 0x7fdec01365c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004270 tx=0x7fdeb8004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.464+0000 7fdeb5ffb700 1 -- 172.21.15.89:0/2378832775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8019040 con 0x7fdec0139170 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.464+0000 7fdeb5ffb700 1 -- 172.21.15.89:0/2378832775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb80110d0 con 0x7fdec0139170 2023-09-30T22:20:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.464+0000 7fdeb5ffb700 1 -- 172.21.15.89:0/2378832775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8011460 con 0x7fdec0139170 2023-09-30T22:20:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.464+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec01302b0 con 0x7fdec0139170 2023-09-30T22:20:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.464+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec011a100 con 0x7fdec0139170 2023-09-30T22:20:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.465+0000 7fdeb5ffb700 1 -- 172.21.15.89:0/2378832775 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdeb80115c0 con 0x7fdec0139170 2023-09-30T22:20:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.465+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec0074ea0 con 0x7fdec0139170 2023-09-30T22:20:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.465+0000 7fdeb5ffb700 1 -- 172.21.15.89:0/2378832775 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdeb80485b0 con 0x7fdec0139170 2023-09-30T22:20:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.470+0000 7fdeb5ffb700 1 -- 172.21.15.89:0/2378832775 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdeb801c770 con 0x7fdec0139170 2023-09-30T22:20:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.534+0000 7f3a55567700 1 Processor -- start 2023-09-30T22:20:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.534+0000 7f3a55567700 1 -- start start 2023-09-30T22:20:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.534+0000 7f3a55567700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a5012cf60 0x7f3a5006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.534+0000 7f3a55567700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.534+0000 7f3a55567700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 0x7f3a50136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.534+0000 7f3a55567700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5012d330 con 0x7f3a5012cf60 2023-09-30T22:20:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.534+0000 7f3a55567700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5012cd90 con 0x7f3a5012f8b0 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.534+0000 7f3a55567700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501306d0 con 0x7f3a50130260 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 0x7f3a50136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a5012cf60 0x7f3a5006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 0x7f3a50136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60022/0 (socket says 172.21.15.89:60022) 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4f7fe700 1 -- 172.21.15.89:0/1038901158 learned_addr learned my addr 172.21.15.89:0/1038901158 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4e7fc700 1 --2- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4effd700 1 -- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 msgr2=0x7f3a50136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4effd700 1 --2- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 0x7f3a50136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4effd700 1 -- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 msgr2=0x7f3a5012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4effd700 1 --2- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4effd700 1 -- 172.21.15.89:0/1038901158 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50071780 con 0x7f3a5012cf60 2023-09-30T22:20:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4effd700 1 --2- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 0x7f3a5006f5d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009870 tx=0x7f3a40009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b612d8c1c660b608 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4dffb700 1 -- 172.21.15.89:0/1038901158 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000f940 con 0x7f3a5012cf60 2023-09-30T22:20:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4dffb700 1 -- 172.21.15.89:0/1038901158 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4000faa0 con 0x7f3a5012cf60 2023-09-30T22:20:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.535+0000 7f3a4dffb700 1 -- 172.21.15.89:0/1038901158 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000fe10 con 0x7f3a5012cf60 2023-09-30T22:20:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 -- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 msgr2=0x7f3a5006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 --2- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 0x7f3a5006f5d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009870 tx=0x7f3a40009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 -- 172.21.15.89:0/1038901158 shutdown_connections 2023-09-30T22:20:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 --2- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 0x7f3a50136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 --2- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 0x7f3a5006f5d0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 --2- 172.21.15.89:0/1038901158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 -- 172.21.15.89:0/1038901158 >> 172.21.15.89:0/1038901158 conn(0x7f3a500a2aa0 msgr2=0x7f3a500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 -- 172.21.15.89:0/1038901158 shutdown_connections 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.536+0000 7f3a55567700 1 -- 172.21.15.89:0/1038901158 wait complete. 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.538+0000 7f3a55567700 1 Processor -- start 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.538+0000 7f3a55567700 1 -- start start 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.538+0000 7f3a55567700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 0x7f3a5009b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.538+0000 7f3a55567700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5009b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a55567700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 0x7f3a5009be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a55567700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5011d180 con 0x7f3a5012cf60 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a55567700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5009c8a0 con 0x7f3a5012f8b0 2023-09-30T22:20:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a55567700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5012da40 con 0x7f3a50130260 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5009b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5009b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60874/0 (socket says 172.21.15.89:60874) 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 -- 172.21.15.89:0/67697105 learned_addr learned my addr 172.21.15.89:0/67697105 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4effd700 1 --2- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 0x7f3a5009b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 -- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 msgr2=0x7f3a5009be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 --2- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 0x7f3a5009be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 -- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 msgr2=0x7f3a5009b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 --2- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 0x7f3a5009b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 -- 172.21.15.89:0/67697105 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a400094a0 con 0x7f3a5012f8b0 2023-09-30T22:20:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a4e7fc700 1 --2- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5009b930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400e9e0 tx=0x7f3a4400ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a377fe700 1 -- 172.21.15.89:0/67697105 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44009710 con 0x7f3a5012f8b0 2023-09-30T22:20:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a377fe700 1 -- 172.21.15.89:0/67697105 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4400c7d0 con 0x7f3a5012f8b0 2023-09-30T22:20:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a50129fa0 con 0x7f3a5012f8b0 2023-09-30T22:20:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a377fe700 1 -- 172.21.15.89:0/67697105 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400cb40 con 0x7f3a5012f8b0 2023-09-30T22:20:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.539+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5009d760 con 0x7f3a5012f8b0 2023-09-30T22:20:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.540+0000 7f3a377fe700 1 -- 172.21.15.89:0/67697105 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3a4400cca0 con 0x7f3a5012f8b0 2023-09-30T22:20:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.540+0000 7f3a377fe700 1 -- 172.21.15.89:0/67697105 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3a4403f750 con 0x7f3a5012f8b0 2023-09-30T22:20:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.542+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a50074ea0 con 0x7f3a5012f8b0 2023-09-30T22:20:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.546+0000 7fcd8d57e700 1 Processor -- start 2023-09-30T22:20:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.546+0000 7fcd8d57e700 1 -- start start 2023-09-30T22:20:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd8d57e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd8806a770 0x7fcd8809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd8d57e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd8809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd8d57e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 0x7fcd8813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd8d57e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8806ab40 con 0x7fcd8806a770 2023-09-30T22:20:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd8d57e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8809ebc0 con 0x7fcd8809dc30 2023-09-30T22:20:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd8d57e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8809a810 con 0x7fcd8813c1e0 2023-09-30T22:20:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7f3a377fe700 1 -- 172.21.15.89:0/67697105 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a440232f0 con 0x7f3a5012f8b0 2023-09-30T22:20:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd867fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd8809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd867fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd8809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60880/0 (socket says 172.21.15.89:60880) 2023-09-30T22:20:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd867fc700 1 -- 172.21.15.89:0/2943274462 learned_addr learned my addr 172.21.15.89:0/2943274462 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd867fc700 1 -- 172.21.15.89:0/2943274462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 msgr2=0x7fcd8813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd867fc700 1 --2- 172.21.15.89:0/2943274462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 0x7fcd8813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd867fc700 1 -- 172.21.15.89:0/2943274462 >> v2:172.21.15.89:3300/0 conn(0x7fcd8806a770 msgr2=0x7fcd8809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd867fc700 1 --2- 172.21.15.89:0/2943274462 >> v2:172.21.15.89:3300/0 conn(0x7fcd8806a770 0x7fcd8809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.547+0000 7fcd867fc700 1 -- 172.21.15.89:0/2943274462 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8809e920 con 0x7fcd8809dc30 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd867fc700 1 --2- 172.21.15.89:0/2943274462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd8809e080 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0098f0 tx=0x7fcd7c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69c9648ea2d949b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd85ffb700 1 -- 172.21.15.89:0/2943274462 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00b8a0 con 0x7fcd8809dc30 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd85ffb700 1 -- 172.21.15.89:0/2943274462 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7c00ba00 con 0x7fcd8809dc30 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd85ffb700 1 -- 172.21.15.89:0/2943274462 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00bd90 con 0x7fcd8809dc30 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd8d57e700 1 -- 172.21.15.89:0/2943274462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 msgr2=0x7fcd8809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd8d57e700 1 --2- 172.21.15.89:0/2943274462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd8809e080 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0098f0 tx=0x7fcd7c009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd8d57e700 1 -- 172.21.15.89:0/2943274462 shutdown_connections 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd8d57e700 1 --2- 172.21.15.89:0/2943274462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 0x7fcd8813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd8d57e700 1 --2- 172.21.15.89:0/2943274462 >> v2:172.21.15.89:3300/0 conn(0x7fcd8806a770 0x7fcd8809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd8d57e700 1 --2- 172.21.15.89:0/2943274462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd8809e080 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.548+0000 7fcd8d57e700 1 -- 172.21.15.89:0/2943274462 >> 172.21.15.89:0/2943274462 conn(0x7fcd88066f50 msgr2=0x7fcd8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.549+0000 7fcd8d57e700 1 -- 172.21.15.89:0/2943274462 shutdown_connections 2023-09-30T22:20:49.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.549+0000 7fcd8d57e700 1 -- 172.21.15.89:0/2943274462 wait complete. 2023-09-30T22:20:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.550+0000 7fcd8d57e700 1 Processor -- start 2023-09-30T22:20:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.550+0000 7fcd8d57e700 1 -- start start 2023-09-30T22:20:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd8d57e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8806a770 0x7fcd881cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd8d57e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd881cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd8d57e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 0x7fcd881ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd8d57e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd88128ea0 con 0x7fcd8806a770 2023-09-30T22:20:49.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd8d57e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8812a810 con 0x7fcd8809dc30 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd8d57e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd880a3980 con 0x7fcd8813c1e0 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd867fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd881cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd867fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd881cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60888/0 (socket says 172.21.15.89:60888) 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd867fc700 1 -- 172.21.15.89:0/3986297749 learned_addr learned my addr 172.21.15.89:0/3986297749 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd877fe700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 0x7fcd881ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd86ffd700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8806a770 0x7fcd881cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd867fc700 1 -- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 msgr2=0x7fcd881ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd867fc700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 0x7fcd881ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd867fc700 1 -- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8806a770 msgr2=0x7fcd881cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd867fc700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8806a770 0x7fcd881cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.551+0000 7fcd867fc700 1 -- 172.21.15.89:0/3986297749 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c009550 con 0x7fcd8809dc30 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.552+0000 7fcd867fc700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd881cca90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00b0e0 tx=0x7fcd7c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.552+0000 7fcd6f7fe700 1 -- 172.21.15.89:0/3986297749 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c004480 con 0x7fcd8809dc30 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.552+0000 7fcd6f7fe700 1 -- 172.21.15.89:0/3986297749 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7c01f070 con 0x7fcd8809dc30 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.552+0000 7fcd6f7fe700 1 -- 172.21.15.89:0/3986297749 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0038e0 con 0x7fcd8809dc30 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.552+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8809f1a0 con 0x7fcd8809dc30 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.552+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8812b540 con 0x7fcd8809dc30 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.553+0000 7fcd6f7fe700 1 -- 172.21.15.89:0/3986297749 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd7c021030 con 0x7fcd8809dc30 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.553+0000 7fcd6f7fe700 1 -- 172.21.15.89:0/3986297749 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcd7c040c00 con 0x7fcd8809dc30 2023-09-30T22:20:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.553+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd88074ea0 con 0x7fcd8809dc30 2023-09-30T22:20:49.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.558+0000 7fcd6f7fe700 1 -- 172.21.15.89:0/3986297749 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd7c003a40 con 0x7fcd8809dc30 2023-09-30T22:20:49.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.671+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9b8011b1a0 con 0x7f9b8006acb0 2023-09-30T22:20:49.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.672+0000 7f9b677fe700 1 -- 172.21.15.89:0/2406858363 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b70016340 con 0x7f9b8006acb0 2023-09-30T22:20:49.673 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8006acb0 msgr2=0x7f9b80119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 --2- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8006acb0 0x7f9b80119480 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9b70012fc0 tx=0x7f9b700103a0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 shutdown_connections 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 --2- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b801316c0 0x7f9b80119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 --2- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8006acb0 0x7f9b80119480 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 --2- 172.21.15.89:0/2406858363 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8009d9f0 0x7f9b801199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 >> 172.21.15.89:0/2406858363 conn(0x7f9b80066f50 msgr2=0x7f9b800a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 shutdown_connections 2023-09-30T22:20:49.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.674+0000 7f9b85689700 1 -- 172.21.15.89:0/2406858363 wait complete. 2023-09-30T22:20:49.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.685+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdec006c360 con 0x7fdec0139170 2023-09-30T22:20:49.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.685+0000 7fdeb5ffb700 1 -- 172.21.15.89:0/2378832775 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdeb80173c0 con 0x7fdec0139170 2023-09-30T22:20:49.686 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:49.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:49.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.687+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec0139170 msgr2=0x7fdec01365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.688+0000 7fdec770b700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec0139170 0x7fdec01365c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004270 tx=0x7fdeb8004630 comp rx=0 tx=0).stop 2023-09-30T22:20:49.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.688+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 shutdown_connections 2023-09-30T22:20:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.688+0000 7fdec770b700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec0139170 0x7fdec01365c0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.688+0000 7fdec770b700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdec009d6f0 0x7fdec0135b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.688+0000 7fdec770b700 1 --2- 172.21.15.89:0/2378832775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdec009e1d0 0x7fdec0136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.688+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 >> 172.21.15.89:0/2378832775 conn(0x7fdec0066f50 msgr2=0x7fdec0132700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.688+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 shutdown_connections 2023-09-30T22:20:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.688+0000 7fdec770b700 1 -- 172.21.15.89:0/2378832775 wait complete. 2023-09-30T22:20:49.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.740+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3a5009e700 con 0x7f3a5012f8b0 2023-09-30T22:20:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.741+0000 7f3a377fe700 1 -- 172.21.15.89:0/67697105 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a4402a090 con 0x7f3a5012f8b0 2023-09-30T22:20:49.742 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 msgr2=0x7f3a5009b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 --2- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5009b930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400e9e0 tx=0x7f3a4400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 shutdown_connections 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 --2- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a50130260 0x7f3a5009be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 --2- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a5012cf60 0x7f3a5009b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 --2- 172.21.15.89:0/67697105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a5012f8b0 0x7f3a5009b930 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 >> 172.21.15.89:0/67697105 conn(0x7f3a500a2aa0 msgr2=0x7f3a501b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 shutdown_connections 2023-09-30T22:20:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.743+0000 7f3a55567700 1 -- 172.21.15.89:0/67697105 wait complete. 2023-09-30T22:20:49.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:49.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.766+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcd8806c360 con 0x7fcd8809dc30 2023-09-30T22:20:49.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.766+0000 7fcd6f7fe700 1 -- 172.21.15.89:0/3986297749 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd7c0233d0 con 0x7fcd8809dc30 2023-09-30T22:20:49.767 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:49.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 msgr2=0x7fcd881cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:49.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd881cca90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00b0e0 tx=0x7fcd7c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:49.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 shutdown_connections 2023-09-30T22:20:49.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8813c1e0 0x7fcd881ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8806a770 0x7fcd881cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 --2- 172.21.15.89:0/3986297749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8809dc30 0x7fcd881cca90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:49.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 >> 172.21.15.89:0/3986297749 conn(0x7fcd88066f50 msgr2=0x7fcd88070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:49.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 shutdown_connections 2023-09-30T22:20:49.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:49.769+0000 7fcd8d57e700 1 -- 172.21.15.89:0/3986297749 wait complete. 2023-09-30T22:20:49.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:50.688 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:50.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:50.756 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:50.781 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.832+0000 7f0aa7de7700 1 Processor -- start 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa7de7700 1 -- start start 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa7de7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa01338a0 0x7f0aa0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa7de7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009e930 0x7f0aa013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa7de7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa006ef30 con 0x7f0aa009de50 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa7de7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa006fef0 con 0x7f0aa009e930 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa7de7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa01334b0 con 0x7f0aa01338a0 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa5382700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0aa009de50 0x7f0aa009e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34410/0 (socket says 172.21.15.89:34410) 2023-09-30T22:20:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa5382700 1 -- 172.21.15.89:0/2123909222 learned_addr learned my addr 172.21.15.89:0/2123909222 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa6384700 1 --2- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009e930 0x7f0aa013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.833+0000 7f0aa5b83700 1 --2- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa01338a0 0x7f0aa0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.834+0000 7f0aa6384700 1 -- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa01338a0 msgr2=0x7f0aa0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.834+0000 7f0aa6384700 1 --2- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa01338a0 0x7f0aa0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.834+0000 7f0aa6384700 1 -- 172.21.15.89:0/2123909222 >> v1:172.21.15.89:6789/0 conn(0x7f0aa009de50 legacy=0x7f0aa009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.834+0000 7f0aa6384700 1 -- 172.21.15.89:0/2123909222 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa006fc30 con 0x7f0aa009e930 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.834+0000 7f0aa6384700 1 --2- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009e930 0x7f0aa013b740 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009dc0 tx=0x7f0a9800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6644877aa1248a0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.834+0000 7f0aa4b81700 1 -- 172.21.15.89:0/2123909222 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800e070 con 0x7f0aa009e930 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.834+0000 7f0aa4b81700 1 -- 172.21.15.89:0/2123909222 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9800bb20 con 0x7f0aa009e930 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.834+0000 7f0aa4b81700 1 -- 172.21.15.89:0/2123909222 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98012430 con 0x7f0aa009e930 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.835+0000 7f0aa7de7700 1 -- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009e930 msgr2=0x7f0aa013b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.835+0000 7f0aa7de7700 1 --2- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009e930 0x7f0aa013b740 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009dc0 tx=0x7f0a9800b330 comp rx=0 tx=0).stop 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.835+0000 7f0aa7de7700 1 -- 172.21.15.89:0/2123909222 shutdown_connections 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.835+0000 7f0aa7de7700 1 --2- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009e930 0x7f0aa013b740 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.835+0000 7f0aa7de7700 1 --2- 172.21.15.89:0/2123909222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa01338a0 0x7f0aa0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.835+0000 7f0aa7de7700 1 -- 172.21.15.89:0/2123909222 >> 172.21.15.89:0/2123909222 conn(0x7f0aa00a2aa0 msgr2=0x7f0aa00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.835+0000 7f0aa7de7700 1 -- 172.21.15.89:0/2123909222 shutdown_connections 2023-09-30T22:20:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.835+0000 7f0aa7de7700 1 -- 172.21.15.89:0/2123909222 wait complete. 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.836+0000 7f0aa7de7700 1 Processor -- start 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.836+0000 7f0aa7de7700 1 -- start start 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa7de7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009de50 0x7f0aa011eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa7de7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa009e930 0x7f0aa011f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa7de7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa01338a0 0x7f0aa011f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa7de7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa012f610 con 0x7f0aa01338a0 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa7de7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa01206c0 con 0x7f0aa009de50 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa7de7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa01222c0 con 0x7f0aa009e930 2023-09-30T22:20:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009de50 0x7f0aa011eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009de50 0x7f0aa011eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54072/0 (socket says 172.21.15.89:54072) 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 -- 172.21.15.89:0/3710123879 learned_addr learned my addr 172.21.15.89:0/3710123879 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa6384700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa01338a0 0x7f0aa011f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 -- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa009e930 msgr2=0x7f0aa011f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa009e930 0x7f0aa011f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 -- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa01338a0 msgr2=0x7f0aa011f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa01338a0 0x7f0aa011f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 -- 172.21.15.89:0/3710123879 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98009a70 con 0x7f0aa009de50 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa6384700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa01338a0 0x7f0aa011f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:50.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.837+0000 7f0aa5b83700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009de50 0x7f0aa011eb50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00e9b0 tx=0x7f0a9c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.838+0000 7f0a967fc700 1 -- 172.21.15.89:0/3710123879 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c009710 con 0x7f0aa009de50 2023-09-30T22:20:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.838+0000 7f0a967fc700 1 -- 172.21.15.89:0/3710123879 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c00c850 con 0x7f0aa009de50 2023-09-30T22:20:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.838+0000 7f0a967fc700 1 -- 172.21.15.89:0/3710123879 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00cbc0 con 0x7f0aa009de50 2023-09-30T22:20:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.838+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa0121d40 con 0x7f0aa009de50 2023-09-30T22:20:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.838+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa006f640 con 0x7f0aa009de50 2023-09-30T22:20:50.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.840+0000 7f0a967fc700 1 -- 172.21.15.89:0/3710123879 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a9c00cd20 con 0x7f0aa009de50 2023-09-30T22:20:50.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.840+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa0121f60 con 0x7f0aa009de50 2023-09-30T22:20:50.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.840+0000 7f0a967fc700 1 -- 172.21.15.89:0/3710123879 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a9c03fb60 con 0x7f0aa009de50 2023-09-30T22:20:50.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.846+0000 7f0a967fc700 1 -- 172.21.15.89:0/3710123879 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a9c01adc0 con 0x7f0aa009de50 2023-09-30T22:20:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.869+0000 7fe3eefc4700 1 Processor -- start 2023-09-30T22:20:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.869+0000 7fe3eefc4700 1 -- start start 2023-09-30T22:20:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3eefc4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3eefc4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e8135c20 0x7fe3e8138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3eefc4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e806fa40 con 0x7fe3e812cf60 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3eefc4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e812cd90 con 0x7fe3e806f5d0 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3eefc4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8071780 con 0x7fe3e8135c20 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3ecd60700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe3e812cf60 0x7fe3e812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34414/0 (socket says 172.21.15.89:34414) 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3ecd60700 1 -- 172.21.15.89:0/3084488744 learned_addr learned my addr 172.21.15.89:0/3084488744 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3ed561700 1 --2- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e8135c20 0x7fe3e8138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3e7fff700 1 --2- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3ed561700 1 -- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 msgr2=0x7fe3e812ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3ed561700 1 --2- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e812ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3ed561700 1 -- 172.21.15.89:0/3084488744 >> v1:172.21.15.89:6789/0 conn(0x7fe3e812cf60 legacy=0x7fe3e812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3e77fe700 1 -- 172.21.15.89:0/3084488744 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862102917 0 0) 0x7fe3e806fa40 con 0x7fe3e812cf60 2023-09-30T22:20:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3ed561700 1 -- 172.21.15.89:0/3084488744 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8071a40 con 0x7fe3e8135c20 2023-09-30T22:20:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.870+0000 7fe3ed561700 1 --2- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e8135c20 0x7fe3e8138000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000af00 tx=0x7fe3e000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd73f17798404da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.871+0000 7fe3e77fe700 1 -- 172.21.15.89:0/3084488744 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000f070 con 0x7fe3e8135c20 2023-09-30T22:20:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.871+0000 7fe3e77fe700 1 -- 172.21.15.89:0/3084488744 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e000a2e0 con 0x7fe3e8135c20 2023-09-30T22:20:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.871+0000 7fe3e77fe700 1 -- 172.21.15.89:0/3084488744 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000cbd0 con 0x7fe3e8135c20 2023-09-30T22:20:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.871+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e8135c20 msgr2=0x7fe3e8138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.871+0000 7fe3eefc4700 1 --2- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e8135c20 0x7fe3e8138000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000af00 tx=0x7fe3e000c330 comp rx=0 tx=0).stop 2023-09-30T22:20:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.872+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3084488744 shutdown_connections 2023-09-30T22:20:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.872+0000 7fe3eefc4700 1 --2- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e8135c20 0x7fe3e8138000 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.872+0000 7fe3eefc4700 1 --2- 172.21.15.89:0/3084488744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e812ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.872+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3084488744 >> 172.21.15.89:0/3084488744 conn(0x7fe3e80a2aa0 msgr2=0x7fe3e80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.872+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3084488744 shutdown_connections 2023-09-30T22:20:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.872+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3084488744 wait complete. 2023-09-30T22:20:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.873+0000 7fe3eefc4700 1 Processor -- start 2023-09-30T22:20:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.873+0000 7fe3eefc4700 1 -- start start 2023-09-30T22:20:50.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.873+0000 7fe3eefc4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e81c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.873+0000 7fe3eefc4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e812cf60 0x7fe3e81c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.873+0000 7fe3eefc4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e8135c20 0x7fe3e8131ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.873+0000 7fe3eefc4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8129fa0 con 0x7fe3e8135c20 2023-09-30T22:20:50.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.873+0000 7fe3eefc4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e81353f0 con 0x7fe3e806f5d0 2023-09-30T22:20:50.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.873+0000 7fe3eefc4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e81298e0 con 0x7fe3e812cf60 2023-09-30T22:20:50.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3ecd60700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e81c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3ecd60700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e81c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54092/0 (socket says 172.21.15.89:54092) 2023-09-30T22:20:50.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3ecd60700 1 -- 172.21.15.89:0/3518864244 learned_addr learned my addr 172.21.15.89:0/3518864244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:50.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e7fff700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e812cf60 0x7fe3e81c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e7fff700 1 -- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 msgr2=0x7fe3e81c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e7fff700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e81c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e7fff700 1 -- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e8135c20 msgr2=0x7fe3e8131ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e7fff700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e8135c20 0x7fe3e8131ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e7fff700 1 -- 172.21.15.89:0/3518864244 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e000ab80 con 0x7fe3e812cf60 2023-09-30T22:20:50.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3ecd60700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e81c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e7fff700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e812cf60 0x7fe3e81c6930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400e930 tx=0x7fe3d400ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e57fa700 1 -- 172.21.15.89:0/3518864244 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400ee80 con 0x7fe3e812cf60 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e57fa700 1 -- 172.21.15.89:0/3518864244 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d400c850 con 0x7fe3e812cf60 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e806f100 con 0x7fe3e812cf60 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e81c6e70 con 0x7fe3e812cf60 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.874+0000 7fe3e57fa700 1 -- 172.21.15.89:0/3518864244 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400cbc0 con 0x7fe3e812cf60 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.875+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c80052e0 con 0x7fe3e812cf60 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.876+0000 7fe3e57fa700 1 -- 172.21.15.89:0/3518864244 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe3d400cd20 con 0x7fe3e812cf60 2023-09-30T22:20:50.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.876+0000 7fe3e57fa700 1 -- 172.21.15.89:0/3518864244 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe3d403fac0 con 0x7fe3e812cf60 2023-09-30T22:20:50.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.885+0000 7fe3e57fa700 1 -- 172.21.15.89:0/3518864244 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe3d401b3b0 con 0x7fe3e812cf60 2023-09-30T22:20:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.917+0000 7fd1aea95700 1 Processor -- start 2023-09-30T22:20:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.917+0000 7fd1aea95700 1 -- start start 2023-09-30T22:20:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1aea95700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd1a809e1d0 0x7fd1a8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1aea95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a8139170 0x7fd1a813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1aea95700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a809b120 con 0x7fd1a809d6f0 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1aea95700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a806b460 con 0x7fd1a809e1d0 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1aea95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a809a7e0 con 0x7fd1a8139170 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1a7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd1a809e1d0 0x7fd1a8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1ad032700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a8139170 0x7fd1a813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1a7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd1a809e1d0 0x7fd1a8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33942/0 (socket says 172.21.15.89:33942) 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1a7fff700 1 -- 172.21.15.89:0/1120937732 learned_addr learned my addr 172.21.15.89:0/1120937732 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.918+0000 7fd1ad032700 1 -- 172.21.15.89:0/1120937732 >> v2:172.21.15.89:3300/0 conn(0x7fd1a809e1d0 msgr2=0x7fd1a8138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.919+0000 7fd1ad032700 1 --2- 172.21.15.89:0/1120937732 >> v2:172.21.15.89:3300/0 conn(0x7fd1a809e1d0 0x7fd1a8138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.919+0000 7fd1ad032700 1 -- 172.21.15.89:0/1120937732 >> v1:172.21.15.89:6789/0 conn(0x7fd1a809d6f0 legacy=0x7fd1a809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.919+0000 7fd1ad032700 1 -- 172.21.15.89:0/1120937732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a809a500 con 0x7fd1a8139170 2023-09-30T22:20:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.919+0000 7fd1ad032700 1 --2- 172.21.15.89:0/1120937732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a8139170 0x7fd1a813b550 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009dc0 tx=0x7fd1a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a2abc52ced2150 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.919+0000 7fd1a77fe700 1 -- 172.21.15.89:0/1120937732 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000e070 con 0x7fd1a8139170 2023-09-30T22:20:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.919+0000 7fd1a77fe700 1 -- 172.21.15.89:0/1120937732 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a000bae0 con 0x7fd1a8139170 2023-09-30T22:20:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.919+0000 7fd1a77fe700 1 -- 172.21.15.89:0/1120937732 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000be90 con 0x7fd1a8139170 2023-09-30T22:20:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.920+0000 7fd1aea95700 1 -- 172.21.15.89:0/1120937732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a8139170 msgr2=0x7fd1a813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.920+0000 7fd1aea95700 1 --2- 172.21.15.89:0/1120937732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a8139170 0x7fd1a813b550 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009dc0 tx=0x7fd1a000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.920+0000 7fd1aea95700 1 -- 172.21.15.89:0/1120937732 shutdown_connections 2023-09-30T22:20:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.920+0000 7fd1aea95700 1 --2- 172.21.15.89:0/1120937732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a8139170 0x7fd1a813b550 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.920+0000 7fd1aea95700 1 --2- 172.21.15.89:0/1120937732 >> v2:172.21.15.89:3300/0 conn(0x7fd1a809e1d0 0x7fd1a8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.920+0000 7fd1aea95700 1 -- 172.21.15.89:0/1120937732 >> 172.21.15.89:0/1120937732 conn(0x7fd1a8066f50 msgr2=0x7fd1a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.920+0000 7fd1aea95700 1 -- 172.21.15.89:0/1120937732 shutdown_connections 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.920+0000 7fd1aea95700 1 -- 172.21.15.89:0/1120937732 wait complete. 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.921+0000 7fd1aea95700 1 Processor -- start 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.921+0000 7fd1aea95700 1 -- start start 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.921+0000 7fd1aea95700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1a809d6f0 0x7fd1a81c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.921+0000 7fd1aea95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a809e1d0 0x7fd1a81c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.921+0000 7fd1aea95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1a8139170 0x7fd1a811e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.921+0000 7fd1aea95700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a81220c0 con 0x7fd1a809d6f0 2023-09-30T22:20:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.921+0000 7fd1aea95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a81203b0 con 0x7fd1a809e1d0 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.921+0000 7fd1aea95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8134670 con 0x7fd1a8139170 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1ad032700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1a8139170 0x7fd1a811e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1ad032700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1a8139170 0x7fd1a811e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41984/0 (socket says 172.21.15.89:41984) 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1ad032700 1 -- 172.21.15.89:0/3371453341 learned_addr learned my addr 172.21.15.89:0/3371453341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1a7fff700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a809e1d0 0x7fd1a81c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1ac831700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1a809d6f0 0x7fd1a81c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1a7fff700 1 -- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1a8139170 msgr2=0x7fd1a811e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1a7fff700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1a8139170 0x7fd1a811e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1a7fff700 1 -- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1a809d6f0 msgr2=0x7fd1a81c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1a7fff700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1a809d6f0 0x7fd1a81c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1a7fff700 1 -- 172.21.15.89:0/3371453341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a0009a70 con 0x7fd1a809e1d0 2023-09-30T22:20:50.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1ad032700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1a8139170 0x7fd1a811e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:50.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1a7fff700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a809e1d0 0x7fd1a81c8bb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd19400e8e0 tx=0x7fd19400ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:50.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.922+0000 7fd1a57fa700 1 -- 172.21.15.89:0/3371453341 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194013680 con 0x7fd1a809e1d0 2023-09-30T22:20:50.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.923+0000 7fd1a57fa700 1 -- 172.21.15.89:0/3371453341 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1940137e0 con 0x7fd1a809e1d0 2023-09-30T22:20:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.923+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a80726b0 con 0x7fd1a809e1d0 2023-09-30T22:20:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.923+0000 7fd1a57fa700 1 -- 172.21.15.89:0/3371453341 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194013b70 con 0x7fd1a809e1d0 2023-09-30T22:20:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.923+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a81c90f0 con 0x7fd1a809e1d0 2023-09-30T22:20:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.924+0000 7fd1a57fa700 1 -- 172.21.15.89:0/3371453341 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd194013cd0 con 0x7fd1a809e1d0 2023-09-30T22:20:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.924+0000 7fd1a57fa700 1 -- 172.21.15.89:0/3371453341 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd19403f980 con 0x7fd1a809e1d0 2023-09-30T22:20:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.924+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a8074ea0 con 0x7fd1a809e1d0 2023-09-30T22:20:50.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.933+0000 7fd1a57fa700 1 -- 172.21.15.89:0/3371453341 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd194021070 con 0x7fd1a809e1d0 2023-09-30T22:20:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.966+0000 7feb4c7d0700 1 Processor -- start 2023-09-30T22:20:50.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.967+0000 7feb4c7d0700 1 -- start start 2023-09-30T22:20:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.967+0000 7feb4c7d0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feb440a0a30 0x7feb440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4c7d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb44138d80 0x7feb4413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4c7d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb440a2a70 con 0x7feb4409d630 2023-09-30T22:20:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4c7d0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb440a01d0 con 0x7feb440a0a30 2023-09-30T22:20:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4c7d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4409e2b0 con 0x7feb44138d80 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4ad6d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb44138d80 0x7feb4413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4a56c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feb440a0a30 0x7feb440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb49d6b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7feb4409d630 0x7feb440a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34434/0 (socket says 172.21.15.89:34434) 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4ad6d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb44138d80 0x7feb4413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54112/0 (socket says 172.21.15.89:54112) 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb49d6b700 1 -- 172.21.15.89:0/852233137 learned_addr learned my addr 172.21.15.89:0/852233137 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4a56c700 1 --2- 172.21.15.89:0/852233137 >> v2:172.21.15.89:3300/0 conn(0x7feb440a0a30 0x7feb440a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33956/0 (socket says 172.21.15.89:33956) 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4ad6d700 1 -- 172.21.15.89:0/852233137 >> v2:172.21.15.89:3300/0 conn(0x7feb440a0a30 msgr2=0x7feb440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4ad6d700 1 --2- 172.21.15.89:0/852233137 >> v2:172.21.15.89:3300/0 conn(0x7feb440a0a30 0x7feb440a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4ad6d700 1 -- 172.21.15.89:0/852233137 >> v1:172.21.15.89:6789/0 conn(0x7feb4409d630 legacy=0x7feb440a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4ad6d700 1 -- 172.21.15.89:0/852233137 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4409dc50 con 0x7feb44138d80 2023-09-30T22:20:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4a56c700 1 --2- 172.21.15.89:0/852233137 >> v2:172.21.15.89:3300/0 conn(0x7feb440a0a30 0x7feb440a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:50.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4ad6d700 1 --2- 172.21.15.89:0/852233137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb44138d80 0x7feb4413b160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009d90 tx=0x7feb3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5538310863911a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:50.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.968+0000 7feb4956a700 1 -- 172.21.15.89:0/852233137 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00e070 con 0x7feb44138d80 2023-09-30T22:20:50.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.969+0000 7feb4956a700 1 -- 172.21.15.89:0/852233137 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c00bb20 con 0x7feb44138d80 2023-09-30T22:20:50.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.969+0000 7feb4956a700 1 -- 172.21.15.89:0/852233137 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c012430 con 0x7feb44138d80 2023-09-30T22:20:50.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.969+0000 7feb4c7d0700 1 -- 172.21.15.89:0/852233137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb44138d80 msgr2=0x7feb4413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.969+0000 7feb4c7d0700 1 --2- 172.21.15.89:0/852233137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb44138d80 0x7feb4413b160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009d90 tx=0x7feb3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.970+0000 7feb4c7d0700 1 -- 172.21.15.89:0/852233137 shutdown_connections 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.970+0000 7feb4c7d0700 1 --2- 172.21.15.89:0/852233137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb44138d80 0x7feb4413b160 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.970+0000 7feb4c7d0700 1 --2- 172.21.15.89:0/852233137 >> v2:172.21.15.89:3300/0 conn(0x7feb440a0a30 0x7feb440a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.970+0000 7feb4c7d0700 1 -- 172.21.15.89:0/852233137 >> 172.21.15.89:0/852233137 conn(0x7feb440a3050 msgr2=0x7feb440a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.970+0000 7feb4c7d0700 1 -- 172.21.15.89:0/852233137 shutdown_connections 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.970+0000 7feb4c7d0700 1 -- 172.21.15.89:0/852233137 wait complete. 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.971+0000 7feb4c7d0700 1 Processor -- start 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.971+0000 7feb4c7d0700 1 -- start start 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4c7d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb4409d630 0x7feb441380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4c7d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb440a0a30 0x7feb44138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4c7d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb44138d80 0x7feb4406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4c7d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb44071650 con 0x7feb440a0a30 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4c7d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb440a0410 con 0x7feb4409d630 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4c7d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb4406f6e0 con 0x7feb44138d80 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb44138d80 0x7feb4406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb44138d80 0x7feb4406dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41992/0 (socket says 172.21.15.89:41992) 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 -- 172.21.15.89:0/526777110 learned_addr learned my addr 172.21.15.89:0/526777110 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb49d6b700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb440a0a30 0x7feb44138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4a56c700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb4409d630 0x7feb441380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 -- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb4409d630 msgr2=0x7feb441380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb4409d630 0x7feb441380c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 -- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb440a0a30 msgr2=0x7feb44138600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:50.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb440a0a30 0x7feb44138600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:50.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 -- 172.21.15.89:0/526777110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400094a0 con 0x7feb44138d80 2023-09-30T22:20:50.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.972+0000 7feb4ad6d700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb44138d80 0x7feb4406dac0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7feb3c014040 tx=0x7feb3c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:50.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.973+0000 7feb3affd700 1 -- 172.21.15.89:0/526777110 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c012430 con 0x7feb44138d80 2023-09-30T22:20:50.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.973+0000 7feb3affd700 1 -- 172.21.15.89:0/526777110 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c020070 con 0x7feb44138d80 2023-09-30T22:20:50.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.973+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c009a70 con 0x7feb44138d80 2023-09-30T22:20:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.973+0000 7feb3affd700 1 -- 172.21.15.89:0/526777110 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c003ea0 con 0x7feb44138d80 2023-09-30T22:20:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.973+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb44070fb0 con 0x7feb44138d80 2023-09-30T22:20:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.974+0000 7feb3affd700 1 -- 172.21.15.89:0/526777110 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb3c004280 con 0x7feb44138d80 2023-09-30T22:20:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.974+0000 7feb3affd700 1 -- 172.21.15.89:0/526777110 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feb3c022070 con 0x7feb44138d80 2023-09-30T22:20:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.974+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb440a26f0 con 0x7feb44138d80 2023-09-30T22:20:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:50.979+0000 7feb3affd700 1 -- 172.21.15.89:0/526777110 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb3c012e60 con 0x7feb44138d80 2023-09-30T22:20:51.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.058+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0aa0074ea0 con 0x7f0aa009de50 2023-09-30T22:20:51.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.058+0000 7f0a967fc700 1 -- 172.21.15.89:0/3710123879 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a9c017020 con 0x7f0aa009de50 2023-09-30T22:20:51.058 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:51.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009de50 msgr2=0x7f0aa011eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:51.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009de50 0x7f0aa011eb50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00e9b0 tx=0x7f0a9c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 shutdown_connections 2023-09-30T22:20:51.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa01338a0 0x7f0aa011f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0aa009de50 0x7f0aa011eb50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 --2- 172.21.15.89:0/3710123879 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0aa009e930 0x7f0aa011f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 >> 172.21.15.89:0/3710123879 conn(0x7f0aa00a2aa0 msgr2=0x7f0aa0131650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:51.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 shutdown_connections 2023-09-30T22:20:51.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.060+0000 7f0aa7de7700 1 -- 172.21.15.89:0/3710123879 wait complete. 2023-09-30T22:20:51.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.099+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe3c8005590 con 0x7fe3e812cf60 2023-09-30T22:20:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.099+0000 7fe3e57fa700 1 -- 172.21.15.89:0/3518864244 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe3d4021020 con 0x7fe3e812cf60 2023-09-30T22:20:51.099 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e812cf60 msgr2=0x7fe3e81c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e812cf60 0x7fe3e81c6930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400e930 tx=0x7fe3d400ec40 comp rx=0 tx=0).stop 2023-09-30T22:20:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 shutdown_connections 2023-09-30T22:20:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3e8135c20 0x7fe3e8131ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3e806f5d0 0x7fe3e81c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 --2- 172.21.15.89:0/3518864244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3e812cf60 0x7fe3e81c6930 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 >> 172.21.15.89:0/3518864244 conn(0x7fe3e80a2aa0 msgr2=0x7fe3e812aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 shutdown_connections 2023-09-30T22:20:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.101+0000 7fe3eefc4700 1 -- 172.21.15.89:0/3518864244 wait complete. 2023-09-30T22:20:51.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:51.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.128+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd1a809f420 con 0x7fd1a809e1d0 2023-09-30T22:20:51.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.129+0000 7fd1a57fa700 1 -- 172.21.15.89:0/3371453341 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd19402a090 con 0x7fd1a809e1d0 2023-09-30T22:20:51.129 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:51.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a809e1d0 msgr2=0x7fd1a81c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:51.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a809e1d0 0x7fd1a81c8bb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd19400e8e0 tx=0x7fd19400ebf0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 shutdown_connections 2023-09-30T22:20:51.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1a8139170 0x7fd1a811e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1a809d6f0 0x7fd1a81c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 --2- 172.21.15.89:0/3371453341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1a809e1d0 0x7fd1a81c8bb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 >> 172.21.15.89:0/3371453341 conn(0x7fd1a8066f50 msgr2=0x7fd1a80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:51.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 shutdown_connections 2023-09-30T22:20:51.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.131+0000 7fd1aea95700 1 -- 172.21.15.89:0/3371453341 wait complete. 2023-09-30T22:20:51.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.178+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feb440747f0 con 0x7feb44138d80 2023-09-30T22:20:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.178+0000 7feb3affd700 1 -- 172.21.15.89:0/526777110 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb3c035090 con 0x7feb44138d80 2023-09-30T22:20:51.178 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:51.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.180+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb44138d80 msgr2=0x7feb4406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:51.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.180+0000 7feb4c7d0700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb44138d80 0x7feb4406dac0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7feb3c014040 tx=0x7feb3c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.180+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 shutdown_connections 2023-09-30T22:20:51.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.180+0000 7feb4c7d0700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb44138d80 0x7feb4406dac0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.180+0000 7feb4c7d0700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb4409d630 0x7feb441380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.180+0000 7feb4c7d0700 1 --2- 172.21.15.89:0/526777110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb440a0a30 0x7feb44138600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:51.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.180+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 >> 172.21.15.89:0/526777110 conn(0x7feb440a3050 msgr2=0x7feb4409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:51.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.181+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 shutdown_connections 2023-09-30T22:20:51.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:51.181+0000 7feb4c7d0700 1 -- 172.21.15.89:0/526777110 wait complete. 2023-09-30T22:20:51.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:52.073 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:52.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:52.144 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:52.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:52.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.215+0000 7f171c6d0700 1 Processor -- start 2023-09-30T22:20:52.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.215+0000 7f171c6d0700 1 -- start start 2023-09-30T22:20:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.216+0000 7f171c6d0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f171409e1d0 0x7f1714138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.216+0000 7f171c6d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f171413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.216+0000 7f171c6d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171409b120 con 0x7f171409d6f0 2023-09-30T22:20:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.216+0000 7f171c6d0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171406b460 con 0x7f171409e1d0 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.216+0000 7f171c6d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171409a7e0 con 0x7f1714139170 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.216+0000 7f1719c6b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f171409e1d0 0x7f1714138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.216+0000 7f171a46c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f171409d6f0 0x7f171409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34444/0 (socket says 172.21.15.89:34444) 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.216+0000 7f171a46c700 1 -- 172.21.15.89:0/1898108286 learned_addr learned my addr 172.21.15.89:0/1898108286 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.217+0000 7f171ac6d700 1 --2- 172.21.15.89:0/1898108286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f171413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.217+0000 7f1719c6b700 1 -- 172.21.15.89:0/1898108286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 msgr2=0x7f171413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.217+0000 7f1719c6b700 1 --2- 172.21.15.89:0/1898108286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f171413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.217+0000 7f1719c6b700 1 -- 172.21.15.89:0/1898108286 >> v1:172.21.15.89:6789/0 conn(0x7f171409d6f0 legacy=0x7f171409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.217+0000 7f1719c6b700 1 -- 172.21.15.89:0/1898108286 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171409a500 con 0x7f171409e1d0 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.217+0000 7f1719c6b700 1 --2- 172.21.15.89:0/1898108286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409e1d0 0x7f1714138c30 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1704009870 tx=0x7f1704009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7890fe388c0229bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.217+0000 7f171946a700 1 -- 172.21.15.89:0/1898108286 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704011830 con 0x7f171409e1d0 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171946a700 1 -- 172.21.15.89:0/1898108286 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1704011990 con 0x7f171409e1d0 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171946a700 1 -- 172.21.15.89:0/1898108286 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704011d20 con 0x7f171409e1d0 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171c6d0700 1 -- 172.21.15.89:0/1898108286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409e1d0 msgr2=0x7f1714138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171c6d0700 1 --2- 172.21.15.89:0/1898108286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409e1d0 0x7f1714138c30 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1704009870 tx=0x7f1704009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171c6d0700 1 -- 172.21.15.89:0/1898108286 shutdown_connections 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171c6d0700 1 --2- 172.21.15.89:0/1898108286 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f171413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171c6d0700 1 --2- 172.21.15.89:0/1898108286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409e1d0 0x7f1714138c30 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171c6d0700 1 -- 172.21.15.89:0/1898108286 >> 172.21.15.89:0/1898108286 conn(0x7f1714066f50 msgr2=0x7f171409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.218+0000 7f171c6d0700 1 -- 172.21.15.89:0/1898108286 shutdown_connections 2023-09-30T22:20:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.219+0000 7f171c6d0700 1 -- 172.21.15.89:0/1898108286 wait complete. 2023-09-30T22:20:52.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.220+0000 7f171c6d0700 1 Processor -- start 2023-09-30T22:20:52.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.220+0000 7f171c6d0700 1 -- start start 2023-09-30T22:20:52.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171c6d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409d6f0 0x7f171411f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171c6d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f171409e1d0 0x7f171411f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171c6d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f17141c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171c6d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171409c710 con 0x7f171409d6f0 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171c6d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171411eee0 con 0x7f1714139170 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171c6d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1714121eb0 con 0x7f171409e1d0 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171a46c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409d6f0 0x7f171411f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171a46c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409d6f0 0x7f171411f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33978/0 (socket says 172.21.15.89:33978) 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171a46c700 1 -- 172.21.15.89:0/1438840777 learned_addr learned my addr 172.21.15.89:0/1438840777 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:52.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171ac6d700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f17141c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f1719c6b700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f171409e1d0 0x7f171411f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171ac6d700 1 -- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f171409e1d0 msgr2=0x7f171411f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171ac6d700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f171409e1d0 0x7f171411f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171ac6d700 1 -- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409d6f0 msgr2=0x7f171411f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171ac6d700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409d6f0 0x7f171411f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171ac6d700 1 -- 172.21.15.89:0/1438840777 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17040094a0 con 0x7f1714139170 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.221+0000 7f171ac6d700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f17141c8360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f170c007d90 tx=0x7f170c00e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.222+0000 7f170affd700 1 -- 172.21.15.89:0/1438840777 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c018070 con 0x7f1714139170 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.222+0000 7f170affd700 1 -- 172.21.15.89:0/1438840777 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c00eb30 con 0x7f1714139170 2023-09-30T22:20:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.222+0000 7f170affd700 1 -- 172.21.15.89:0/1438840777 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00eec0 con 0x7f1714139170 2023-09-30T22:20:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.222+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171409a0c0 con 0x7f1714139170 2023-09-30T22:20:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.222+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1714122450 con 0x7f1714139170 2023-09-30T22:20:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.224+0000 7f170affd700 1 -- 172.21.15.89:0/1438840777 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f170c014420 con 0x7f1714139170 2023-09-30T22:20:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.224+0000 7f170affd700 1 -- 172.21.15.89:0/1438840777 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f170c03f3b0 con 0x7f1714139170 2023-09-30T22:20:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.224+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1714074ea0 con 0x7f1714139170 2023-09-30T22:20:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.229+0000 7f170affd700 1 -- 172.21.15.89:0/1438840777 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f170c0128d0 con 0x7f1714139170 2023-09-30T22:20:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.253+0000 7fee1ddf1700 1 Processor -- start 2023-09-30T22:20:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee1ddf1700 1 -- start start 2023-09-30T22:20:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee1ddf1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809d6f0 0x7fee1809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee1ddf1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809e000 0x7fee181339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee1ddf1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1809b120 con 0x7fee1809e410 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee1ddf1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1806b460 con 0x7fee1809d6f0 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee1ddf1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee1809a7e0 con 0x7fee1809e000 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee17fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fee1809e410 0x7fee18133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34458/0 (socket says 172.21.15.89:34458) 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee17fff700 1 -- 172.21.15.89:0/26100046 learned_addr learned my addr 172.21.15.89:0/26100046 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee16ffd700 1 --2- 172.21.15.89:0/26100046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809e000 0x7fee181339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.254+0000 7fee167fc700 1 -- 172.21.15.89:0/26100046 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083290781 0 0) 0x7fee1809b120 con 0x7fee1809e410 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee167fc700 1 -- 172.21.15.89:0/26100046 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee000035b0 con 0x7fee1809e410 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee167fc700 1 -- 172.21.15.89:0/26100046 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3523442487 0 0) 0x7fee000035b0 con 0x7fee1809e410 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee16ffd700 1 -- 172.21.15.89:0/26100046 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809d6f0 msgr2=0x7fee1809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee16ffd700 1 --2- 172.21.15.89:0/26100046 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809d6f0 0x7fee1809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee16ffd700 1 -- 172.21.15.89:0/26100046 >> v1:172.21.15.89:6789/0 conn(0x7fee1809e410 legacy=0x7fee18133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee16ffd700 1 -- 172.21.15.89:0/26100046 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1809a500 con 0x7fee1809e000 2023-09-30T22:20:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee16ffd700 1 --2- 172.21.15.89:0/26100046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809e000 0x7fee181339f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0098a0 tx=0x7fee0c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9526f91e0ec416b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee167fc700 1 -- 172.21.15.89:0/26100046 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee100019f0 con 0x7fee1809e000 2023-09-30T22:20:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee167fc700 1 -- 172.21.15.89:0/26100046 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c004030 con 0x7fee1809e000 2023-09-30T22:20:52.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.255+0000 7fee167fc700 1 -- 172.21.15.89:0/26100046 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0043c0 con 0x7fee1809e000 2023-09-30T22:20:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.256+0000 7fee1ddf1700 1 -- 172.21.15.89:0/26100046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809e000 msgr2=0x7fee181339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.256+0000 7fee1ddf1700 1 --2- 172.21.15.89:0/26100046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809e000 0x7fee181339f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0098a0 tx=0x7fee0c009bb0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.256+0000 7fee1ddf1700 1 -- 172.21.15.89:0/26100046 shutdown_connections 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.256+0000 7fee1ddf1700 1 --2- 172.21.15.89:0/26100046 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809d6f0 0x7fee1809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.256+0000 7fee1ddf1700 1 --2- 172.21.15.89:0/26100046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809e000 0x7fee181339f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.256+0000 7fee1ddf1700 1 -- 172.21.15.89:0/26100046 >> 172.21.15.89:0/26100046 conn(0x7fee18066f50 msgr2=0x7fee1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.256+0000 7fee1ddf1700 1 -- 172.21.15.89:0/26100046 shutdown_connections 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.256+0000 7fee1ddf1700 1 -- 172.21.15.89:0/26100046 wait complete. 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.258+0000 7fee1ddf1700 1 Processor -- start 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.258+0000 7fee1ddf1700 1 -- start start 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.258+0000 7fee1ddf1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809d6f0 0x7fee181224a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.258+0000 7fee1ddf1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809e000 0x7fee181229e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.258+0000 7fee1ddf1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809e410 0x7fee18122f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.258+0000 7fee1ddf1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee18125fc0 con 0x7fee1809e000 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee1ddf1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee18072150 con 0x7fee1809e410 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee1ddf1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee1809c710 con 0x7fee1809d6f0 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809d6f0 0x7fee181224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee17fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809e410 0x7fee18122f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809d6f0 0x7fee181224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42026/0 (socket says 172.21.15.89:42026) 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 -- 172.21.15.89:0/1861196794 learned_addr learned my addr 172.21.15.89:0/1861196794 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee16ffd700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809e000 0x7fee181229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 -- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809e410 msgr2=0x7fee18122f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809e410 0x7fee18122f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 -- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809e000 msgr2=0x7fee181229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809e000 0x7fee181229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 -- 172.21.15.89:0/1861196794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c0094a0 con 0x7fee1809d6f0 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee17fff700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809e410 0x7fee18122f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fee177fe700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809d6f0 0x7fee181224a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fee0800b530 tx=0x7fee0800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fedfffff700 1 -- 172.21.15.89:0/1861196794 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08011830 con 0x7fee1809d6f0 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.259+0000 7fedfffff700 1 -- 172.21.15.89:0/1861196794 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee080119b0 con 0x7fee1809d6f0 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.260+0000 7fedfffff700 1 -- 172.21.15.89:0/1861196794 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08011d60 con 0x7fee1809d6f0 2023-09-30T22:20:52.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.260+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee18125300 con 0x7fee1809d6f0 2023-09-30T22:20:52.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.260+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee18125750 con 0x7fee1809d6f0 2023-09-30T22:20:52.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.260+0000 7fedfffff700 1 -- 172.21.15.89:0/1861196794 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee0801a400 con 0x7fee1809d6f0 2023-09-30T22:20:52.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.261+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee18125b40 con 0x7fee1809d6f0 2023-09-30T22:20:52.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.261+0000 7fedfffff700 1 -- 172.21.15.89:0/1861196794 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fee0803f750 con 0x7fee1809d6f0 2023-09-30T22:20:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.270+0000 7fedfffff700 1 -- 172.21.15.89:0/1861196794 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee18125b40 con 0x7fee1809d6f0 2023-09-30T22:20:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.303+0000 7fcc0c6a1700 1 Processor -- start 2023-09-30T22:20:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.303+0000 7fcc0c6a1700 1 -- start start 2023-09-30T22:20:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0c6a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc0412cc20 0x7fcc0412f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0c6a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc0412fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0c6a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0409a510 con 0x7fcc0412cff0 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0c6a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0412c7d0 con 0x7fcc0412cc20 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0c6a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0409c190 con 0x7fcc0412c2f0 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0a43d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc0412cc20 0x7fcc0412f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc09c3c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc0412fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0a43d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc0412cc20 0x7fcc0412f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33992/0 (socket says 172.21.15.89:33992) 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0a43d700 1 -- 172.21.15.89:0/1179291379 learned_addr learned my addr 172.21.15.89:0/1179291379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0a43d700 1 -- 172.21.15.89:0/1179291379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 msgr2=0x7fcc0412fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0a43d700 1 --2- 172.21.15.89:0/1179291379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc0412fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0a43d700 1 -- 172.21.15.89:0/1179291379 >> v1:172.21.15.89:6789/0 conn(0x7fcc0412cff0 legacy=0x7fcc04136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc0a43d700 1 -- 172.21.15.89:0/1179291379 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0409b540 con 0x7fcc0412cc20 2023-09-30T22:20:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.304+0000 7fcc09c3c700 1 --2- 172.21.15.89:0/1179291379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc0412fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.305+0000 7fcc0a43d700 1 --2- 172.21.15.89:0/1179291379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cc20 0x7fcc0412f930 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098f0 tx=0x7fcc00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f23c7fe0fc5dcf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.305+0000 7fcc0943b700 1 -- 172.21.15.89:0/1179291379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00011830 con 0x7fcc0412cc20 2023-09-30T22:20:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.305+0000 7fcc0943b700 1 -- 172.21.15.89:0/1179291379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00011990 con 0x7fcc0412cc20 2023-09-30T22:20:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.305+0000 7fcc0943b700 1 -- 172.21.15.89:0/1179291379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00011d20 con 0x7fcc0412cc20 2023-09-30T22:20:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.305+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/1179291379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cc20 msgr2=0x7fcc0412f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.305+0000 7fcc0c6a1700 1 --2- 172.21.15.89:0/1179291379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cc20 0x7fcc0412f930 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098f0 tx=0x7fcc00009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.306+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/1179291379 shutdown_connections 2023-09-30T22:20:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.306+0000 7fcc0c6a1700 1 --2- 172.21.15.89:0/1179291379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cc20 0x7fcc0412f930 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.306+0000 7fcc0c6a1700 1 --2- 172.21.15.89:0/1179291379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc0412fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.306+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/1179291379 >> 172.21.15.89:0/1179291379 conn(0x7fcc0409f470 msgr2=0x7fcc040a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.306+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/1179291379 shutdown_connections 2023-09-30T22:20:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.306+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/1179291379 wait complete. 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.307+0000 7fcc0c6a1700 1 Processor -- start 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.307+0000 7fcc0c6a1700 1 -- start start 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0c6a1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc04071040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0c6a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc0412cc20 0x7fcc04071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0c6a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cff0 0x7fcc04071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0c6a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0412b4e0 con 0x7fcc0412cff0 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0c6a1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc04129f10 con 0x7fcc0412cc20 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0c6a1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0408b4c0 con 0x7fcc0412c2f0 2023-09-30T22:20:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0ac3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cff0 0x7fcc04071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0ac3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cff0 0x7fcc04071ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34004/0 (socket says 172.21.15.89:34004) 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0ac3e700 1 -- 172.21.15.89:0/2149190455 learned_addr learned my addr 172.21.15.89:0/2149190455 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc09c3c700 1 --2- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc0412cc20 0x7fcc04071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0a43d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc04071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0ac3e700 1 -- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 msgr2=0x7fcc04071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0ac3e700 1 --2- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc04071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0ac3e700 1 -- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc0412cc20 msgr2=0x7fcc04071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0ac3e700 1 --2- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc0412cc20 0x7fcc04071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.308+0000 7fcc0ac3e700 1 -- 172.21.15.89:0/2149190455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf40094a0 con 0x7fcc0412cff0 2023-09-30T22:20:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.309+0000 7fcc0ac3e700 1 --2- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cff0 0x7fcc04071ac0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00cc50 tx=0x7fcbfc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.309+0000 7fcbfaffd700 1 -- 172.21.15.89:0/2149190455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc003c00 con 0x7fcc0412cff0 2023-09-30T22:20:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.309+0000 7fcbfaffd700 1 -- 172.21.15.89:0/2149190455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc003d60 con 0x7fcc0412cff0 2023-09-30T22:20:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.309+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc00009550 con 0x7fcc0412cff0 2023-09-30T22:20:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.309+0000 7fcbfaffd700 1 -- 172.21.15.89:0/2149190455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc00aab0 con 0x7fcc0412cff0 2023-09-30T22:20:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.309+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0408bb70 con 0x7fcc0412cff0 2023-09-30T22:20:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.310+0000 7fcbfaffd700 1 -- 172.21.15.89:0/2149190455 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcbfc00ac10 con 0x7fcc0412cff0 2023-09-30T22:20:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.310+0000 7fcbfaffd700 1 -- 172.21.15.89:0/2149190455 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcbfc03f5f0 con 0x7fcc0412cff0 2023-09-30T22:20:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.310+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc041285a0 con 0x7fcc0412cff0 2023-09-30T22:20:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.319+0000 7fcbfaffd700 1 -- 172.21.15.89:0/2149190455 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcbfc0198f0 con 0x7fcc0412cff0 2023-09-30T22:20:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.345+0000 7f0577b6a700 1 Processor -- start 2023-09-30T22:20:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0577b6a700 1 -- start start 2023-09-30T22:20:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0577b6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05700a0a30 0x7f05700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0577b6a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f057009d630 0x7f05700a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0577b6a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05700a2a70 con 0x7f0570133930 2023-09-30T22:20:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0577b6a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05700a01d0 con 0x7f057009d630 2023-09-30T22:20:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0577b6a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f057009e2b0 con 0x7f05700a0a30 2023-09-30T22:20:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0575105700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f057009d630 0x7f05700a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0575105700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f057009d630 0x7f05700a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34018/0 (socket says 172.21.15.89:34018) 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0575105700 1 -- 172.21.15.89:0/1974299025 learned_addr learned my addr 172.21.15.89:0/1974299025 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.346+0000 7f0575906700 1 --2- 172.21.15.89:0/1974299025 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05700a0a30 0x7f05700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.347+0000 7f0575105700 1 -- 172.21.15.89:0/1974299025 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05700a0a30 msgr2=0x7f05700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.347+0000 7f0575105700 1 --2- 172.21.15.89:0/1974299025 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05700a0a30 0x7f05700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.347+0000 7f0575105700 1 -- 172.21.15.89:0/1974299025 >> v1:172.21.15.89:6789/0 conn(0x7f0570133930 legacy=0x7f0570133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.347+0000 7f0575105700 1 -- 172.21.15.89:0/1974299025 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057009dc50 con 0x7f057009d630 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.347+0000 7f0575105700 1 --2- 172.21.15.89:0/1974299025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 0x7f05700a1340 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f05600098f0 tx=0x7f0560009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8b4adb67955840 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.347+0000 7f0574904700 1 -- 172.21.15.89:0/1974299025 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560011830 con 0x7f057009d630 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.347+0000 7f0574904700 1 -- 172.21.15.89:0/1974299025 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0560011990 con 0x7f057009d630 2023-09-30T22:20:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.347+0000 7f0574904700 1 -- 172.21.15.89:0/1974299025 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560011d20 con 0x7f057009d630 2023-09-30T22:20:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.348+0000 7f0577b6a700 1 -- 172.21.15.89:0/1974299025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 msgr2=0x7f05700a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.348+0000 7f0577b6a700 1 --2- 172.21.15.89:0/1974299025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 0x7f05700a1340 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f05600098f0 tx=0x7f0560009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.348+0000 7f0577b6a700 1 -- 172.21.15.89:0/1974299025 shutdown_connections 2023-09-30T22:20:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.348+0000 7f0577b6a700 1 --2- 172.21.15.89:0/1974299025 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05700a0a30 0x7f05700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.348+0000 7f0577b6a700 1 --2- 172.21.15.89:0/1974299025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 0x7f05700a1340 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.348+0000 7f0577b6a700 1 -- 172.21.15.89:0/1974299025 >> 172.21.15.89:0/1974299025 conn(0x7f05700a3050 msgr2=0x7f05700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.348+0000 7f0577b6a700 1 -- 172.21.15.89:0/1974299025 shutdown_connections 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.348+0000 7f0577b6a700 1 -- 172.21.15.89:0/1974299025 wait complete. 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.349+0000 7f0577b6a700 1 Processor -- start 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.349+0000 7f0577b6a700 1 -- start start 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.349+0000 7f0577b6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 0x7f057011f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0577b6a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05700a0a30 0x7f057011faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0577b6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0570133930 0x7f05701c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0577b6a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057011f160 con 0x7f057009d630 2023-09-30T22:20:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0577b6a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05700a0410 con 0x7f05700a0a30 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0577b6a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0570120030 con 0x7f0570133930 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0575105700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05700a0a30 0x7f057011faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0576107700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0570133930 0x7f05701c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0575105700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05700a0a30 0x7f057011faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54176/0 (socket says 172.21.15.89:54176) 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0576107700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0570133930 0x7f05701c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42062/0 (socket says 172.21.15.89:42062) 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0575105700 1 -- 172.21.15.89:0/726733689 learned_addr learned my addr 172.21.15.89:0/726733689 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0575906700 1 --2- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 0x7f057011f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0576107700 1 -- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05700a0a30 msgr2=0x7f057011faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0576107700 1 --2- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05700a0a30 0x7f057011faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0576107700 1 -- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 msgr2=0x7f057011f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0576107700 1 --2- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 0x7f057011f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0576107700 1 -- 172.21.15.89:0/726733689 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009550 con 0x7f0570133930 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0576107700 1 --2- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0570133930 0x7f05701c80b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f056800cb90 tx=0x7f056800cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f05667fc700 1 -- 172.21.15.89:0/726733689 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568003c00 con 0x7f0570133930 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f05667fc700 1 -- 172.21.15.89:0/726733689 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568003d60 con 0x7f0570133930 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f05667fc700 1 -- 172.21.15.89:0/726733689 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056800aa80 con 0x7f0570133930 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.350+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0570122100 con 0x7f0570133930 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.351+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05701335d0 con 0x7f0570133930 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.351+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05701222c0 con 0x7f0570133930 2023-09-30T22:20:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.352+0000 7f05667fc700 1 -- 172.21.15.89:0/726733689 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f056800abe0 con 0x7f0570133930 2023-09-30T22:20:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.352+0000 7f05667fc700 1 -- 172.21.15.89:0/726733689 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f056803f560 con 0x7f0570133930 2023-09-30T22:20:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.356+0000 7f05667fc700 1 -- 172.21.15.89:0/726733689 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05680050d0 con 0x7f0570133930 2023-09-30T22:20:52.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.437+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f17141220b0 con 0x7f1714139170 2023-09-30T22:20:52.437 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:52.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.437+0000 7f170affd700 1 -- 172.21.15.89:0/1438840777 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17141220b0 con 0x7f1714139170 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 msgr2=0x7f17141c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f17141c8360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f170c007d90 tx=0x7f170c00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 shutdown_connections 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1714139170 0x7f17141c8360 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f171409d6f0 0x7f171411f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 --2- 172.21.15.89:0/1438840777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f171409e1d0 0x7f171411f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 >> 172.21.15.89:0/1438840777 conn(0x7f1714066f50 msgr2=0x7f1714136d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 shutdown_connections 2023-09-30T22:20:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.440+0000 7f171c6d0700 1 -- 172.21.15.89:0/1438840777 wait complete. 2023-09-30T22:20:52.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:52.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.515+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fee18125940 con 0x7fee1809d6f0 2023-09-30T22:20:52.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.516+0000 7fedfffff700 1 -- 172.21.15.89:0/1861196794 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee08029090 con 0x7fee1809d6f0 2023-09-30T22:20:52.516 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809d6f0 msgr2=0x7fee181224a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809d6f0 0x7fee181224a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fee0800b530 tx=0x7fee0800b840 comp rx=0 tx=0).stop 2023-09-30T22:20:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 shutdown_connections 2023-09-30T22:20:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee1809e410 0x7fee18122f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee1809d6f0 0x7fee181224a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 --2- 172.21.15.89:0/1861196794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1809e000 0x7fee181229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 >> 172.21.15.89:0/1861196794 conn(0x7fee18066f50 msgr2=0x7fee18070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 shutdown_connections 2023-09-30T22:20:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.518+0000 7fee1ddf1700 1 -- 172.21.15.89:0/1861196794 wait complete. 2023-09-30T22:20:52.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:52.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.551+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f05701224d0 con 0x7f0570133930 2023-09-30T22:20:52.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.551+0000 7f05667fc700 1 -- 172.21.15.89:0/726733689 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0568011380 con 0x7f0570133930 2023-09-30T22:20:52.551 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0570133930 msgr2=0x7f05701c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 --2- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0570133930 0x7f05701c80b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f056800cb90 tx=0x7f056800cea0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 shutdown_connections 2023-09-30T22:20:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 --2- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0570133930 0x7f05701c80b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 --2- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057009d630 0x7f057011f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 --2- 172.21.15.89:0/726733689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05700a0a30 0x7f057011faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 >> 172.21.15.89:0/726733689 conn(0x7f05700a3050 msgr2=0x7f05701bb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:52.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 shutdown_connections 2023-09-30T22:20:52.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.553+0000 7f0577b6a700 1 -- 172.21.15.89:0/726733689 wait complete. 2023-09-30T22:20:52.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:52.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.565+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcc040747f0 con 0x7fcc0412cff0 2023-09-30T22:20:52.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.565+0000 7fcbfaffd700 1 -- 172.21.15.89:0/2149190455 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcbfc019300 con 0x7fcc0412cff0 2023-09-30T22:20:52.565 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:52.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cff0 msgr2=0x7fcc04071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:52.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 --2- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cff0 0x7fcc04071ac0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00cc50 tx=0x7fcbfc00cf60 comp rx=0 tx=0).stop 2023-09-30T22:20:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 shutdown_connections 2023-09-30T22:20:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 --2- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc0412cff0 0x7fcc04071ac0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 --2- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc0412c2f0 0x7fcc04071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 --2- 172.21.15.89:0/2149190455 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc0412cc20 0x7fcc04071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 >> 172.21.15.89:0/2149190455 conn(0x7fcc0409f470 msgr2=0x7fcc040a1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 shutdown_connections 2023-09-30T22:20:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:52.567+0000 7fcc0c6a1700 1 -- 172.21.15.89:0/2149190455 wait complete. 2023-09-30T22:20:52.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:53.453 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:53.531 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:53.565 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:53.580 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:53.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.595+0000 7f3b508fb700 1 Processor -- start 2023-09-30T22:20:53.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.595+0000 7f3b508fb700 1 -- start start 2023-09-30T22:20:53.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b508fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b508fb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3b4c09e000 0x7f3b4c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b508fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e410 0x7f3b4c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b508fb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4c09f520 con 0x7f3b4c09e000 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b508fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c09f130 con 0x7f3b4c09d6f0 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b508fb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c09e930 con 0x7f3b4c09e410 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54178/0 (socket says 172.21.15.89:54178) 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4a59c700 1 -- 172.21.15.89:0/2114831244 learned_addr learned my addr 172.21.15.89:0/2114831244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b49d9b700 1 --2- 172.21.15.89:0/2114831244 >> v2:172.21.15.89:3300/0 conn(0x7f3b4c09e000 0x7f3b4c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4ad9d700 1 --2- 172.21.15.89:0/2114831244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e410 0x7f3b4c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4a59c700 1 -- 172.21.15.89:0/2114831244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e410 msgr2=0x7f3b4c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4a59c700 1 --2- 172.21.15.89:0/2114831244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e410 0x7f3b4c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4a59c700 1 -- 172.21.15.89:0/2114831244 >> v2:172.21.15.89:3300/0 conn(0x7f3b4c09e000 msgr2=0x7f3b4c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4a59c700 1 --2- 172.21.15.89:0/2114831244 >> v2:172.21.15.89:3300/0 conn(0x7f3b4c09e000 0x7f3b4c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.596+0000 7f3b4a59c700 1 -- 172.21.15.89:0/2114831244 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c09b2d0 con 0x7f3b4c09d6f0 2023-09-30T22:20:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.597+0000 7f3b4a59c700 1 --2- 172.21.15.89:0/2114831244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c09dac0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00bb90 tx=0x7f3b3c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e448ac1a2d16c9d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.597+0000 7f3b4959a700 1 -- 172.21.15.89:0/2114831244 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00dc70 con 0x7f3b4c09d6f0 2023-09-30T22:20:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.597+0000 7f3b4959a700 1 -- 172.21.15.89:0/2114831244 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3c00ddd0 con 0x7f3b4c09d6f0 2023-09-30T22:20:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.597+0000 7f3b4959a700 1 -- 172.21.15.89:0/2114831244 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c012970 con 0x7f3b4c09d6f0 2023-09-30T22:20:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 -- 172.21.15.89:0/2114831244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 msgr2=0x7f3b4c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 --2- 172.21.15.89:0/2114831244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c09dac0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00bb90 tx=0x7f3b3c00bea0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 -- 172.21.15.89:0/2114831244 shutdown_connections 2023-09-30T22:20:53.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 --2- 172.21.15.89:0/2114831244 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e410 0x7f3b4c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 --2- 172.21.15.89:0/2114831244 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c09dac0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 --2- 172.21.15.89:0/2114831244 >> v2:172.21.15.89:3300/0 conn(0x7f3b4c09e000 0x7f3b4c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 -- 172.21.15.89:0/2114831244 >> 172.21.15.89:0/2114831244 conn(0x7f3b4c066f50 msgr2=0x7f3b4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 -- 172.21.15.89:0/2114831244 shutdown_connections 2023-09-30T22:20:53.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.598+0000 7f3b508fb700 1 -- 172.21.15.89:0/2114831244 wait complete. 2023-09-30T22:20:53.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b508fb700 1 Processor -- start 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b508fb700 1 -- start start 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b508fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c11e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b508fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e000 0x7f3b4c11e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b4a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c11e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b49d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e000 0x7f3b4c11e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b508fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c09e410 0x7f3b4c11eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b508fb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c121b80 con 0x7f3b4c09e410 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b508fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0a18f0 con 0x7f3b4c09d6f0 2023-09-30T22:20:53.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b4a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c11e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54184/0 (socket says 172.21.15.89:54184) 2023-09-30T22:20:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c09c700 con 0x7f3b4c09e000 2023-09-30T22:20:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.600+0000 7f3b4a59c700 1 -- 172.21.15.89:0/2671312824 learned_addr learned my addr 172.21.15.89:0/2671312824 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b49d9b700 1 -- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 msgr2=0x7f3b4c11e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b49d9b700 1 --2- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c11e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b49d9b700 1 -- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c09e410 msgr2=0x7f3b4c11eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b49d9b700 1 --2- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c09e410 0x7f3b4c11eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b49d9b700 1 -- 172.21.15.89:0/2671312824 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c00b7f0 con 0x7f3b4c09e000 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b49d9b700 1 --2- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e000 0x7f3b4c11e5a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e840 tx=0x7f3b4000eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b4a59c700 1 --2- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c11e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b3affd700 1 -- 172.21.15.89:0/2671312824 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40013680 con 0x7f3b4c09e000 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c1215e0 con 0x7f3b4c09e000 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b3affd700 1 -- 172.21.15.89:0/2671312824 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b400137e0 con 0x7f3b4c09e000 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b3affd700 1 -- 172.21.15.89:0/2671312824 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40013b70 con 0x7f3b4c09e000 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.601+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c1201c0 con 0x7f3b4c09e000 2023-09-30T22:20:53.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.602+0000 7f3b3affd700 1 -- 172.21.15.89:0/2671312824 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3b40013cd0 con 0x7f3b4c09e000 2023-09-30T22:20:53.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.602+0000 7f3b3affd700 1 -- 172.21.15.89:0/2671312824 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3b4003f370 con 0x7f3b4c09e000 2023-09-30T22:20:53.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.602+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4c120f50 con 0x7f3b4c09e000 2023-09-30T22:20:53.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.608+0000 7f3b3affd700 1 -- 172.21.15.89:0/2671312824 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3b4001bdc0 con 0x7f3b4c09e000 2023-09-30T22:20:53.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.676+0000 7fe998e39700 1 Processor -- start 2023-09-30T22:20:53.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.676+0000 7fe998e39700 1 -- start start 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe998e39700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe99409e930 0x7fe99409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe998e39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409f240 0x7fe9941317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe998e39700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe994070940 con 0x7fe99409f650 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe998e39700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99406fd60 con 0x7fe99409e930 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe998e39700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe99406fa80 con 0x7fe99409f240 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe99259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe99409e930 0x7fe99409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe991d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409f240 0x7fe9941317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe99259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe99409e930 0x7fe99409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34058/0 (socket says 172.21.15.89:34058) 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe99259c700 1 -- 172.21.15.89:0/1241299751 learned_addr learned my addr 172.21.15.89:0/1241299751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe991d9b700 1 -- 172.21.15.89:0/1241299751 >> v2:172.21.15.89:3300/0 conn(0x7fe99409e930 msgr2=0x7fe99409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe991d9b700 1 --2- 172.21.15.89:0/1241299751 >> v2:172.21.15.89:3300/0 conn(0x7fe99409e930 0x7fe99409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe991d9b700 1 -- 172.21.15.89:0/1241299751 >> v1:172.21.15.89:6789/0 conn(0x7fe99409f650 legacy=0x7fe994131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.677+0000 7fe991d9b700 1 -- 172.21.15.89:0/1241299751 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99409fb60 con 0x7fe99409f240 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.678+0000 7fe991d9b700 1 --2- 172.21.15.89:0/1241299751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409f240 0x7fe9941317f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe988009840 tx=0x7fe988009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b96956987fe9822 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.678+0000 7fe99159a700 1 -- 172.21.15.89:0/1241299751 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988011830 con 0x7fe99409f240 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.678+0000 7fe99159a700 1 -- 172.21.15.89:0/1241299751 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe988011990 con 0x7fe99409f240 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.678+0000 7fe99159a700 1 -- 172.21.15.89:0/1241299751 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988011d20 con 0x7fe99409f240 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.678+0000 7fe998e39700 1 -- 172.21.15.89:0/1241299751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409f240 msgr2=0x7fe9941317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.678+0000 7fe998e39700 1 --2- 172.21.15.89:0/1241299751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409f240 0x7fe9941317f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe988009840 tx=0x7fe988009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.679+0000 7fe998e39700 1 -- 172.21.15.89:0/1241299751 shutdown_connections 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.679+0000 7fe998e39700 1 --2- 172.21.15.89:0/1241299751 >> v2:172.21.15.89:3300/0 conn(0x7fe99409e930 0x7fe99409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.679+0000 7fe998e39700 1 --2- 172.21.15.89:0/1241299751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409f240 0x7fe9941317f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.679+0000 7fe998e39700 1 -- 172.21.15.89:0/1241299751 >> 172.21.15.89:0/1241299751 conn(0x7fe9940a4210 msgr2=0x7fe9940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.679+0000 7fe998e39700 1 -- 172.21.15.89:0/1241299751 shutdown_connections 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.679+0000 7fe998e39700 1 -- 172.21.15.89:0/1241299751 wait complete. 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.680+0000 7fe998e39700 1 Processor -- start 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.680+0000 7fe998e39700 1 -- start start 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe998e39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409e930 0x7fe9941ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe998e39700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe99409f240 0x7fe9941cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe998e39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe99409f650 0x7fe9941a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe998e39700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9941acd90 con 0x7fe99409f240 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe998e39700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9941ac160 con 0x7fe99409f650 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe998e39700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9941a2850 con 0x7fe99409e930 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe99259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409e930 0x7fe9941ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe992d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe99409f650 0x7fe9941a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe991d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe99409f240 0x7fe9941cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe992d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe99409f650 0x7fe9941a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54200/0 (socket says 172.21.15.89:54200) 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe991d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe99409f240 0x7fe9941cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34070/0 (socket says 172.21.15.89:34070) 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe992d9d700 1 -- 172.21.15.89:0/4259127895 learned_addr learned my addr 172.21.15.89:0/4259127895 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe991d9b700 1 -- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409e930 msgr2=0x7fe9941ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe991d9b700 1 --2- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409e930 0x7fe9941ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe991d9b700 1 -- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe99409f650 msgr2=0x7fe9941a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe991d9b700 1 --2- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe99409f650 0x7fe9941a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe991d9b700 1 -- 172.21.15.89:0/4259127895 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880094a0 con 0x7fe99409f240 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe991d9b700 1 --2- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe99409f240 0x7fe9941cd090 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe988016000 tx=0x7fe988004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.681+0000 7fe982ffd700 1 -- 172.21.15.89:0/4259127895 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988004400 con 0x7fe99409f240 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.682+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9940a0190 con 0x7fe99409f240 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.682+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9941ac5d0 con 0x7fe99409f240 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.682+0000 7fe982ffd700 1 -- 172.21.15.89:0/4259127895 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98801f070 con 0x7fe99409f240 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.682+0000 7fe982ffd700 1 -- 172.21.15.89:0/4259127895 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880038e0 con 0x7fe99409f240 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.683+0000 7fe982ffd700 1 -- 172.21.15.89:0/4259127895 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe988021030 con 0x7fe99409f240 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.683+0000 7fe982ffd700 1 -- 172.21.15.89:0/4259127895 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe988040910 con 0x7fe99409f240 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.683+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9780052e0 con 0x7fe99409f240 2023-09-30T22:20:53.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.692+0000 7fe982ffd700 1 -- 172.21.15.89:0/4259127895 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe988015070 con 0x7fe99409f240 2023-09-30T22:20:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.714+0000 7fe61006d700 1 Processor -- start 2023-09-30T22:20:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe61006d700 1 -- start start 2023-09-30T22:20:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe61006d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe60809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe61006d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e000 0x7fe6081339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe61006d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe60809e410 0x7fe608133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe61006d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60806b460 con 0x7fe60809e410 2023-09-30T22:20:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe61006d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60809b120 con 0x7fe60809d6f0 2023-09-30T22:20:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe61006d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe60809a7e0 con 0x7fe60809e000 2023-09-30T22:20:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe60e60a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe60809e410 0x7fe608133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe60d608700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e000 0x7fe6081339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe60de09700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe60809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe60e60a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe60809e410 0x7fe608133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34076/0 (socket says 172.21.15.89:34076) 2023-09-30T22:20:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.715+0000 7fe60e60a700 1 -- 172.21.15.89:0/3448275458 learned_addr learned my addr 172.21.15.89:0/3448275458 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60de09700 1 -- 172.21.15.89:0/3448275458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e000 msgr2=0x7fe6081339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60de09700 1 --2- 172.21.15.89:0/3448275458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e000 0x7fe6081339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60de09700 1 -- 172.21.15.89:0/3448275458 >> v2:172.21.15.89:3300/0 conn(0x7fe60809e410 msgr2=0x7fe608133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60de09700 1 --2- 172.21.15.89:0/3448275458 >> v2:172.21.15.89:3300/0 conn(0x7fe60809e410 0x7fe608133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60de09700 1 -- 172.21.15.89:0/3448275458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60809a500 con 0x7fe60809d6f0 2023-09-30T22:20:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60de09700 1 --2- 172.21.15.89:0/3448275458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe60809dac0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe604009870 tx=0x7fe604009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ba47aa628500c3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60ce07700 1 -- 172.21.15.89:0/3448275458 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604011830 con 0x7fe60809d6f0 2023-09-30T22:20:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60ce07700 1 -- 172.21.15.89:0/3448275458 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604011990 con 0x7fe60809d6f0 2023-09-30T22:20:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.716+0000 7fe60ce07700 1 -- 172.21.15.89:0/3448275458 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604011d20 con 0x7fe60809d6f0 2023-09-30T22:20:53.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 -- 172.21.15.89:0/3448275458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 msgr2=0x7fe60809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 --2- 172.21.15.89:0/3448275458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe60809dac0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe604009870 tx=0x7fe604009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:53.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 -- 172.21.15.89:0/3448275458 shutdown_connections 2023-09-30T22:20:53.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 --2- 172.21.15.89:0/3448275458 >> v2:172.21.15.89:3300/0 conn(0x7fe60809e410 0x7fe608133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 --2- 172.21.15.89:0/3448275458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe60809dac0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 --2- 172.21.15.89:0/3448275458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e000 0x7fe6081339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 -- 172.21.15.89:0/3448275458 >> 172.21.15.89:0/3448275458 conn(0x7fe608066f50 msgr2=0x7fe60809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 -- 172.21.15.89:0/3448275458 shutdown_connections 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.717+0000 7fe61006d700 1 -- 172.21.15.89:0/3448275458 wait complete. 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe61006d700 1 Processor -- start 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe61006d700 1 -- start start 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe61006d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe6081a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe61006d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe60809e000 0x7fe6081a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe61006d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e410 0x7fe6081a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe61006d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081a8600 con 0x7fe60809e000 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe61006d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080a31a0 con 0x7fe60809d6f0 2023-09-30T22:20:53.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe61006d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081a6ae0 con 0x7fe60809e410 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe60e60a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e410 0x7fe6081a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe60de09700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe6081a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe60de09700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe6081a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54226/0 (socket says 172.21.15.89:54226) 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe60e60a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e410 0x7fe6081a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42096/0 (socket says 172.21.15.89:42096) 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe60de09700 1 -- 172.21.15.89:0/2941259352 learned_addr learned my addr 172.21.15.89:0/2941259352 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.718+0000 7fe60d608700 1 --2- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe60809e000 0x7fe6081a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe60de09700 1 -- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e410 msgr2=0x7fe6081a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe60de09700 1 --2- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e410 0x7fe6081a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe60de09700 1 -- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe60809e000 msgr2=0x7fe6081a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe60de09700 1 --2- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe60809e000 0x7fe6081a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe60de09700 1 -- 172.21.15.89:0/2941259352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040094a0 con 0x7fe60809d6f0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe60de09700 1 --2- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe6081a4ed0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe6040090b0 tx=0x7fe604004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe5fe7fc700 1 -- 172.21.15.89:0/2941259352 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604004450 con 0x7fe60809d6f0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe5fe7fc700 1 -- 172.21.15.89:0/2941259352 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60401f070 con 0x7fe60809d6f0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60809c710 con 0x7fe60809d6f0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe5fe7fc700 1 -- 172.21.15.89:0/2941259352 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040038a0 con 0x7fe60809d6f0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.719+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6081a88e0 con 0x7fe60809d6f0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.720+0000 7fe5fe7fc700 1 -- 172.21.15.89:0/2941259352 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe604003a00 con 0x7fe60809d6f0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.720+0000 7fe5fe7fc700 1 -- 172.21.15.89:0/2941259352 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe604021070 con 0x7fe60809d6f0 2023-09-30T22:20:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.720+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6081a8740 con 0x7fe60809d6f0 2023-09-30T22:20:53.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.729+0000 7fe5fe7fc700 1 -- 172.21.15.89:0/2941259352 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6081a8740 con 0x7fe60809d6f0 2023-09-30T22:20:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.741+0000 7efe89c5d700 1 Processor -- start 2023-09-30T22:20:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe89c5d700 1 -- start start 2023-09-30T22:20:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe89c5d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe840a0a30 0x7efe840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe89c5d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe840a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe89c5d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe840a2a70 con 0x7efe84133930 2023-09-30T22:20:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe89c5d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe840a01d0 con 0x7efe840a0a30 2023-09-30T22:20:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe89c5d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe8409e2b0 con 0x7efe8409d630 2023-09-30T22:20:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe82ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe840a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe837fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe840a0a30 0x7efe840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe82ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe840a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42112/0 (socket says 172.21.15.89:42112) 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.742+0000 7efe82ffd700 1 -- 172.21.15.89:0/52100599 learned_addr learned my addr 172.21.15.89:0/52100599 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe82ffd700 1 -- 172.21.15.89:0/52100599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe840a0a30 msgr2=0x7efe840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe82ffd700 1 --2- 172.21.15.89:0/52100599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe840a0a30 0x7efe840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe82ffd700 1 -- 172.21.15.89:0/52100599 >> v1:172.21.15.89:6789/0 conn(0x7efe84133930 legacy=0x7efe84133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe82ffd700 1 -- 172.21.15.89:0/52100599 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8409dc50 con 0x7efe8409d630 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe827fc700 1 -- 172.21.15.89:0/52100599 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050175156 0 0) 0x7efe840a2a70 con 0x7efe84133930 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe837fe700 1 --2- 172.21.15.89:0/52100599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe840a0a30 0x7efe840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe82ffd700 1 --2- 172.21.15.89:0/52100599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe840a1340 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe78009920 tx=0x7efe78009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b66f9dcddbdc9c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe827fc700 1 -- 172.21.15.89:0/52100599 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78011830 con 0x7efe8409d630 2023-09-30T22:20:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe827fc700 1 -- 172.21.15.89:0/52100599 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe78011990 con 0x7efe8409d630 2023-09-30T22:20:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.743+0000 7efe827fc700 1 -- 172.21.15.89:0/52100599 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78011d20 con 0x7efe8409d630 2023-09-30T22:20:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.744+0000 7efe89c5d700 1 -- 172.21.15.89:0/52100599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 msgr2=0x7efe840a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.744+0000 7efe89c5d700 1 --2- 172.21.15.89:0/52100599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe840a1340 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe78009920 tx=0x7efe78009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.744+0000 7efe89c5d700 1 -- 172.21.15.89:0/52100599 shutdown_connections 2023-09-30T22:20:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.744+0000 7efe89c5d700 1 --2- 172.21.15.89:0/52100599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe840a0a30 0x7efe840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.744+0000 7efe89c5d700 1 --2- 172.21.15.89:0/52100599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe840a1340 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.744+0000 7efe89c5d700 1 -- 172.21.15.89:0/52100599 >> 172.21.15.89:0/52100599 conn(0x7efe840a3050 msgr2=0x7efe840a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.744+0000 7efe89c5d700 1 -- 172.21.15.89:0/52100599 shutdown_connections 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.744+0000 7efe89c5d700 1 -- 172.21.15.89:0/52100599 wait complete. 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.745+0000 7efe89c5d700 1 Processor -- start 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.745+0000 7efe89c5d700 1 -- start start 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe89c5d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe8411f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe89c5d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe840a0a30 0x7efe8411f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe89c5d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe84133930 0x7efe841c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe89c5d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8411eef0 con 0x7efe840a0a30 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe89c5d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe840a0410 con 0x7efe84133930 2023-09-30T22:20:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe89c5d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe8411fd30 con 0x7efe8409d630 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe8411f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe83fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe84133930 0x7efe841c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe8411f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42118/0 (socket says 172.21.15.89:42118) 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe83fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe84133930 0x7efe841c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54248/0 (socket says 172.21.15.89:54248) 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 -- 172.21.15.89:0/2348276218 learned_addr learned my addr 172.21.15.89:0/2348276218 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe82ffd700 1 --2- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe840a0a30 0x7efe8411f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 -- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe84133930 msgr2=0x7efe841c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 --2- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe84133930 0x7efe841c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 -- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe840a0a30 msgr2=0x7efe8411f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 --2- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe840a0a30 0x7efe8411f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 -- 172.21.15.89:0/2348276218 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740094a0 con 0x7efe8409d630 2023-09-30T22:20:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.746+0000 7efe837fe700 1 --2- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe8411f2b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efe7400ec10 tx=0x7efe7400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.747+0000 7efe6bfff700 1 -- 172.21.15.89:0/2348276218 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7400c850 con 0x7efe8409d630 2023-09-30T22:20:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.747+0000 7efe6bfff700 1 -- 172.21.15.89:0/2348276218 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7400c9b0 con 0x7efe8409d630 2023-09-30T22:20:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.747+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe78009550 con 0x7efe8409d630 2023-09-30T22:20:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.747+0000 7efe6bfff700 1 -- 172.21.15.89:0/2348276218 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7400cd20 con 0x7efe8409d630 2023-09-30T22:20:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.747+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe841205b0 con 0x7efe8409d630 2023-09-30T22:20:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.748+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe84121f90 con 0x7efe8409d630 2023-09-30T22:20:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.749+0000 7efe6bfff700 1 -- 172.21.15.89:0/2348276218 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efe74013470 con 0x7efe8409d630 2023-09-30T22:20:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.752+0000 7efe6bfff700 1 -- 172.21.15.89:0/2348276218 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efe7401bd40 con 0x7efe8409d630 2023-09-30T22:20:53.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.758+0000 7efe6bfff700 1 -- 172.21.15.89:0/2348276218 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efe74013720 con 0x7efe8409d630 2023-09-30T22:20:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.812+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3b4c074ea0 con 0x7f3b4c09e000 2023-09-30T22:20:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.813+0000 7f3b3affd700 1 -- 172.21.15.89:0/2671312824 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b40017020 con 0x7f3b4c09e000 2023-09-30T22:20:53.813 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.815+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e000 msgr2=0x7f3b4c11e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.815+0000 7f3b508fb700 1 --2- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e000 0x7f3b4c11e5a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e840 tx=0x7f3b4000eb50 comp rx=0 tx=0).stop 2023-09-30T22:20:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.815+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 shutdown_connections 2023-09-30T22:20:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.815+0000 7f3b508fb700 1 --2- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c09e410 0x7f3b4c11eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.815+0000 7f3b508fb700 1 --2- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b4c09d6f0 0x7f3b4c11e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.815+0000 7f3b508fb700 1 --2- 172.21.15.89:0/2671312824 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b4c09e000 0x7f3b4c11e5a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.815+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 >> 172.21.15.89:0/2671312824 conn(0x7f3b4c066f50 msgr2=0x7f3b4c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.816+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 shutdown_connections 2023-09-30T22:20:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.816+0000 7f3b508fb700 1 -- 172.21.15.89:0/2671312824 wait complete. 2023-09-30T22:20:53.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.919+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe978005590 con 0x7fe99409f240 2023-09-30T22:20:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.920+0000 7fe982ffd700 1 -- 172.21.15.89:0/4259127895 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe98801a450 con 0x7fe99409f240 2023-09-30T22:20:53.924 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:53.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.921+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe99409f240 msgr2=0x7fe9941cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.921+0000 7fe998e39700 1 --2- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe99409f240 0x7fe9941cd090 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe988016000 tx=0x7fe988004750 comp rx=0 tx=0).stop 2023-09-30T22:20:53.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.922+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 shutdown_connections 2023-09-30T22:20:53.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.922+0000 7fe998e39700 1 --2- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe99409f650 0x7fe9941a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.922+0000 7fe998e39700 1 --2- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe99409e930 0x7fe9941ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.922+0000 7fe998e39700 1 --2- 172.21.15.89:0/4259127895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe99409f240 0x7fe9941cd090 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.922+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 >> 172.21.15.89:0/4259127895 conn(0x7fe9940a4210 msgr2=0x7fe99406c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:53.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.922+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 shutdown_connections 2023-09-30T22:20:53.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.922+0000 7fe998e39700 1 -- 172.21.15.89:0/4259127895 wait complete. 2023-09-30T22:20:53.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.927+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe6081a8740 con 0x7fe60809d6f0 2023-09-30T22:20:53.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.927+0000 7fe5fe7fc700 1 -- 172.21.15.89:0/2941259352 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6081a8740 con 0x7fe60809d6f0 2023-09-30T22:20:53.927 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:53.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 msgr2=0x7fe6081a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 --2- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe6081a4ed0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe6040090b0 tx=0x7fe604004750 comp rx=0 tx=0).stop 2023-09-30T22:20:53.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 shutdown_connections 2023-09-30T22:20:53.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 --2- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe60809e410 0x7fe6081a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 --2- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe60809d6f0 0x7fe6081a4ed0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 --2- 172.21.15.89:0/2941259352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe60809e000 0x7fe6081a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 >> 172.21.15.89:0/2941259352 conn(0x7fe608066f50 msgr2=0x7fe6081b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:53.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 shutdown_connections 2023-09-30T22:20:53.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.929+0000 7fe61006d700 1 -- 172.21.15.89:0/2941259352 wait complete. 2023-09-30T22:20:53.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:53.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:53.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.957+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efe840747f0 con 0x7efe8409d630 2023-09-30T22:20:53.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.957+0000 7efe6bfff700 1 -- 172.21.15.89:0/2348276218 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efe74017070 con 0x7efe8409d630 2023-09-30T22:20:53.957 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:53.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 msgr2=0x7efe8411f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:53.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 --2- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe8411f2b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efe7400ec10 tx=0x7efe7400ef20 comp rx=0 tx=0).stop 2023-09-30T22:20:53.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 shutdown_connections 2023-09-30T22:20:53.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 --2- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe84133930 0x7efe841c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 --2- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe8409d630 0x7efe8411f2b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 --2- 172.21.15.89:0/2348276218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe840a0a30 0x7efe8411f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:53.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 >> 172.21.15.89:0/2348276218 conn(0x7efe840a3050 msgr2=0x7efe8409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:53.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 shutdown_connections 2023-09-30T22:20:53.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:53.959+0000 7efe89c5d700 1 -- 172.21.15.89:0/2348276218 wait complete. 2023-09-30T22:20:53.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:54.830 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:54.935 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:54.943 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.957+0000 7f8ad5538700 1 Processor -- start 2023-09-30T22:20:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.957+0000 7f8ad5538700 1 -- start start 2023-09-30T22:20:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ad5538700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 0x7f8ad01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ad5538700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ad5538700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad006b990 con 0x7f8ad0134260 2023-09-30T22:20:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ad5538700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad00a0590 con 0x7f8ad0133420 2023-09-30T22:20:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ad5538700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad009fb80 con 0x7f8ad0133e30 2023-09-30T22:20:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ace7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8acf7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8ad0134260 0x7f8ad01304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34498/0 (socket says 172.21.15.89:34498) 2023-09-30T22:20:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8acf7fe700 1 -- 172.21.15.89:0/2204066676 learned_addr learned my addr 172.21.15.89:0/2204066676 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8aceffd700 1 --2- 172.21.15.89:0/2204066676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 0x7f8ad01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:54.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ace7fc700 1 -- 172.21.15.89:0/2204066676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 msgr2=0x7f8ad01337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:54.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ace7fc700 1 --2- 172.21.15.89:0/2204066676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 0x7f8ad01337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:54.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ace7fc700 1 -- 172.21.15.89:0/2204066676 >> v1:172.21.15.89:6789/0 conn(0x7f8ad0134260 legacy=0x7f8ad01304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.958+0000 7f8ace7fc700 1 -- 172.21.15.89:0/2204066676 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00a0030 con 0x7f8ad0133e30 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.959+0000 7f8ace7fc700 1 --2- 172.21.15.89:0/2204066676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad012ff90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009870 tx=0x7f8ac4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec986cb5e3b4536f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.959+0000 7f8acdffb700 1 -- 172.21.15.89:0/2204066676 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4011830 con 0x7f8ad0133e30 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.959+0000 7f8acdffb700 1 -- 172.21.15.89:0/2204066676 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4011990 con 0x7f8ad0133e30 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.959+0000 7f8acdffb700 1 -- 172.21.15.89:0/2204066676 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4011d20 con 0x7f8ad0133e30 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.960+0000 7f8ad5538700 1 -- 172.21.15.89:0/2204066676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 msgr2=0x7f8ad012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.960+0000 7f8ad5538700 1 --2- 172.21.15.89:0/2204066676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad012ff90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009870 tx=0x7f8ac4009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.960+0000 7f8ad5538700 1 -- 172.21.15.89:0/2204066676 shutdown_connections 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.960+0000 7f8ad5538700 1 --2- 172.21.15.89:0/2204066676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 0x7f8ad01337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.960+0000 7f8ad5538700 1 --2- 172.21.15.89:0/2204066676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad012ff90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:54.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.960+0000 7f8ad5538700 1 -- 172.21.15.89:0/2204066676 >> 172.21.15.89:0/2204066676 conn(0x7f8ad00a4210 msgr2=0x7f8ad00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:54.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.960+0000 7f8ad5538700 1 -- 172.21.15.89:0/2204066676 shutdown_connections 2023-09-30T22:20:54.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.960+0000 7f8ad5538700 1 -- 172.21.15.89:0/2204066676 wait complete. 2023-09-30T22:20:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.961+0000 7f8ad5538700 1 Processor -- start 2023-09-30T22:20:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.961+0000 7f8ad5538700 1 -- start start 2023-09-30T22:20:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ad5538700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 0x7f8ad009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ad5538700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ad5538700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad0134260 0x7f8ad009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ad5538700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad011fcc0 con 0x7f8ad0134260 2023-09-30T22:20:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ad5538700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad009df90 con 0x7f8ad0133420 2023-09-30T22:20:54.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ad5538700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad01209f0 con 0x7f8ad0133e30 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ace7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ace7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42126/0 (socket says 172.21.15.89:42126) 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ace7fc700 1 -- 172.21.15.89:0/2748438479 learned_addr learned my addr 172.21.15.89:0/2748438479 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8aceffd700 1 --2- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 0x7f8ad009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ace7fc700 1 -- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 msgr2=0x7f8ad009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ace7fc700 1 --2- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 0x7f8ad009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ace7fc700 1 -- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad0134260 msgr2=0x7f8ad009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ace7fc700 1 --2- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad0134260 0x7f8ad009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.962+0000 7f8ace7fc700 1 -- 172.21.15.89:0/2748438479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40094a0 con 0x7f8ad0133e30 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.963+0000 7f8ace7fc700 1 --2- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad009c9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009bc0 tx=0x7f8ac4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.963+0000 7f8ab77fe700 1 -- 172.21.15.89:0/2748438479 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40043c0 con 0x7f8ad0133e30 2023-09-30T22:20:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.963+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad0120260 con 0x7f8ad0133e30 2023-09-30T22:20:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.963+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad01ce690 con 0x7f8ad0133e30 2023-09-30T22:20:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.963+0000 7f8ab77fe700 1 -- 172.21.15.89:0/2748438479 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac401f070 con 0x7f8ad0133e30 2023-09-30T22:20:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.963+0000 7f8ab77fe700 1 -- 172.21.15.89:0/2748438479 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40036a0 con 0x7f8ad0133e30 2023-09-30T22:20:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.964+0000 7f8ab77fe700 1 -- 172.21.15.89:0/2748438479 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ac4021030 con 0x7f8ad0133e30 2023-09-30T22:20:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.964+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab00052e0 con 0x7f8ad0133e30 2023-09-30T22:20:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.964+0000 7f8ab77fe700 1 -- 172.21.15.89:0/2748438479 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8ac4040a90 con 0x7f8ad0133e30 2023-09-30T22:20:54.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:54.969+0000 7f8ab77fe700 1 -- 172.21.15.89:0/2748438479 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ac40239c0 con 0x7f8ad0133e30 2023-09-30T22:20:54.972 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.066+0000 7f77d091e700 1 Processor -- start 2023-09-30T22:20:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.066+0000 7f77d091e700 1 -- start start 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77d091e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77cc1338a0 0x7f77cc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77d091e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77d091e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc09e760 0x7f77cc13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77d091e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77cc06fef0 con 0x7f77cc1338a0 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77d091e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc06ef30 con 0x7f77cc09de50 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77d091e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc1334b0 con 0x7f77cc09e760 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77c9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77c9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54284/0 (socket says 172.21.15.89:54284) 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77ca59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77cc1338a0 0x7f77cc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77c9d9b700 1 -- 172.21.15.89:0/208181192 learned_addr learned my addr 172.21.15.89:0/208181192 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77cad9d700 1 --2- 172.21.15.89:0/208181192 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc09e760 0x7f77cc13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77c9d9b700 1 -- 172.21.15.89:0/208181192 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc09e760 msgr2=0x7f77cc13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77c9d9b700 1 --2- 172.21.15.89:0/208181192 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc09e760 0x7f77cc13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77c9d9b700 1 -- 172.21.15.89:0/208181192 >> v2:172.21.15.89:3300/0 conn(0x7f77cc1338a0 msgr2=0x7f77cc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77c9d9b700 1 --2- 172.21.15.89:0/208181192 >> v2:172.21.15.89:3300/0 conn(0x7f77cc1338a0 0x7f77cc133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77c9d9b700 1 -- 172.21.15.89:0/208181192 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc06fc30 con 0x7f77cc09de50 2023-09-30T22:20:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.067+0000 7f77ca59c700 1 --2- 172.21.15.89:0/208181192 >> v2:172.21.15.89:3300/0 conn(0x7f77cc1338a0 0x7f77cc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.068+0000 7f77c9d9b700 1 --2- 172.21.15.89:0/208181192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc09e220 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f77c00098f0 tx=0x7f77c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef0027222a46fb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.068+0000 7f77c959a700 1 -- 172.21.15.89:0/208181192 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0011830 con 0x7f77cc09de50 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.068+0000 7f77c959a700 1 -- 172.21.15.89:0/208181192 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c0011990 con 0x7f77cc09de50 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.068+0000 7f77c959a700 1 -- 172.21.15.89:0/208181192 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0011d20 con 0x7f77cc09de50 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.068+0000 7f77d091e700 1 -- 172.21.15.89:0/208181192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 msgr2=0x7f77cc09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.068+0000 7f77d091e700 1 --2- 172.21.15.89:0/208181192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc09e220 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f77c00098f0 tx=0x7f77c0009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.069+0000 7f77d091e700 1 -- 172.21.15.89:0/208181192 shutdown_connections 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.069+0000 7f77d091e700 1 --2- 172.21.15.89:0/208181192 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc09e760 0x7f77cc13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.069+0000 7f77d091e700 1 --2- 172.21.15.89:0/208181192 >> v2:172.21.15.89:3300/0 conn(0x7f77cc1338a0 0x7f77cc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.069+0000 7f77d091e700 1 --2- 172.21.15.89:0/208181192 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc09e220 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.069+0000 7f77d091e700 1 -- 172.21.15.89:0/208181192 >> 172.21.15.89:0/208181192 conn(0x7f77cc0a2aa0 msgr2=0x7f77cc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.069+0000 7f77d091e700 1 -- 172.21.15.89:0/208181192 shutdown_connections 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.069+0000 7f77d091e700 1 -- 172.21.15.89:0/208181192 wait complete. 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77d091e700 1 Processor -- start 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77d091e700 1 -- start start 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77d091e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc11e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77d091e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77cc09e760 0x7f77cc11eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77d091e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc1338a0 0x7f77cc11f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77d091e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc1224c0 con 0x7f77cc09e760 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77d091e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc06f640 con 0x7f77cc09de50 2023-09-30T22:20:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77d091e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0a1020 con 0x7f77cc1338a0 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77ca59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc11e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77ca59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc11e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54292/0 (socket says 172.21.15.89:54292) 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77ca59c700 1 -- 172.21.15.89:0/2410494875 learned_addr learned my addr 172.21.15.89:0/2410494875 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77cad9d700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc1338a0 0x7f77cc11f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.070+0000 7f77c9d9b700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77cc09e760 0x7f77cc11eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77cad9d700 1 -- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 msgr2=0x7f77cc11e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77cad9d700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc11e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77cad9d700 1 -- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77cc09e760 msgr2=0x7f77cc11eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77cad9d700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77cc09e760 0x7f77cc11eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77cad9d700 1 -- 172.21.15.89:0/2410494875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc0094a0 con 0x7f77cc1338a0 2023-09-30T22:20:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77cad9d700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc1338a0 0x7f77cc11f420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f77c400efc0 tx=0x7f77c400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:55.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77baffd700 1 -- 172.21.15.89:0/2410494875 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c400cc90 con 0x7f77cc1338a0 2023-09-30T22:20:55.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77baffd700 1 -- 172.21.15.89:0/2410494875 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c400f040 con 0x7f77cc1338a0 2023-09-30T22:20:55.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c0009550 con 0x7f77cc1338a0 2023-09-30T22:20:55.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77baffd700 1 -- 172.21.15.89:0/2410494875 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c40146c0 con 0x7f77cc1338a0 2023-09-30T22:20:55.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.071+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc12f4c0 con 0x7f77cc1338a0 2023-09-30T22:20:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.072+0000 7f77baffd700 1 -- 172.21.15.89:0/2410494875 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f77c4014820 con 0x7f77cc1338a0 2023-09-30T22:20:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.072+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc074ea0 con 0x7f77cc1338a0 2023-09-30T22:20:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.073+0000 7f77baffd700 1 -- 172.21.15.89:0/2410494875 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f77c40401b0 con 0x7f77cc1338a0 2023-09-30T22:20:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.081+0000 7f77baffd700 1 -- 172.21.15.89:0/2410494875 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f77c4007900 con 0x7f77cc1338a0 2023-09-30T22:20:55.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.092+0000 7f5608d32700 1 Processor -- start 2023-09-30T22:20:55.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.092+0000 7f5608d32700 1 -- start start 2023-09-30T22:20:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.093+0000 7f5608d32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409d6f0 0x7f560409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.093+0000 7f5608d32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409e410 0x7f5604130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.093+0000 7f5608d32700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560409f130 con 0x7f560409e000 2023-09-30T22:20:55.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.093+0000 7f5608d32700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560409f520 con 0x7f560409d6f0 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.093+0000 7f5608d32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f560409e930 con 0x7f560409e410 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.093+0000 7f560259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409d6f0 0x7f560409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f560259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409d6f0 0x7f560409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54296/0 (socket says 172.21.15.89:54296) 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f5601d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f560409e000 0x7f5604130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34508/0 (socket says 172.21.15.89:34508) 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f560259c700 1 -- 172.21.15.89:0/2723412092 learned_addr learned my addr 172.21.15.89:0/2723412092 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f5602d9d700 1 --2- 172.21.15.89:0/2723412092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409e410 0x7f5604130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f5602d9d700 1 -- 172.21.15.89:0/2723412092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409d6f0 msgr2=0x7f560409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f5602d9d700 1 --2- 172.21.15.89:0/2723412092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409d6f0 0x7f560409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f5602d9d700 1 -- 172.21.15.89:0/2723412092 >> v1:172.21.15.89:6789/0 conn(0x7f560409e000 legacy=0x7f5604130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f5602d9d700 1 -- 172.21.15.89:0/2723412092 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560409b2d0 con 0x7f560409e410 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f5602d9d700 1 --2- 172.21.15.89:0/2723412092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409e410 0x7f5604130d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009dc0 tx=0x7f55fc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9bf6b7d0559cb09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f560159a700 1 -- 172.21.15.89:0/2723412092 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00e070 con 0x7f560409e410 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f560159a700 1 -- 172.21.15.89:0/2723412092 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc00bb20 con 0x7f560409e410 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.094+0000 7f560159a700 1 -- 172.21.15.89:0/2723412092 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc012430 con 0x7f560409e410 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.095+0000 7f5608d32700 1 -- 172.21.15.89:0/2723412092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409e410 msgr2=0x7f5604130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.095+0000 7f5608d32700 1 --2- 172.21.15.89:0/2723412092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409e410 0x7f5604130d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009dc0 tx=0x7f55fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.095+0000 7f5608d32700 1 -- 172.21.15.89:0/2723412092 shutdown_connections 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.095+0000 7f5608d32700 1 --2- 172.21.15.89:0/2723412092 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409d6f0 0x7f560409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.095+0000 7f5608d32700 1 --2- 172.21.15.89:0/2723412092 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409e410 0x7f5604130d70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.095+0000 7f5608d32700 1 -- 172.21.15.89:0/2723412092 >> 172.21.15.89:0/2723412092 conn(0x7f5604066f50 msgr2=0x7f560409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.095+0000 7f5608d32700 1 -- 172.21.15.89:0/2723412092 shutdown_connections 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.095+0000 7f5608d32700 1 -- 172.21.15.89:0/2723412092 wait complete. 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.096+0000 7f5608d32700 1 Processor -- start 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.096+0000 7f5608d32700 1 -- start start 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5608d32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409d6f0 0x7f560411df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5608d32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409e000 0x7f560411e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5608d32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560409e410 0x7f560411e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5608d32700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560409c700 con 0x7f560409e410 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5608d32700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560411fc20 con 0x7f560409e000 2023-09-30T22:20:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5608d32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5604120080 con 0x7f560409d6f0 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5602d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560409e410 0x7f560411e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5602d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560409e410 0x7f560411e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34150/0 (socket says 172.21.15.89:34150) 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5602d9d700 1 -- 172.21.15.89:0/2101728307 learned_addr learned my addr 172.21.15.89:0/2101728307 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5601d9b700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409e000 0x7f560411e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5602d9d700 1 -- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409d6f0 msgr2=0x7f560411df30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5602d9d700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409d6f0 0x7f560411df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5602d9d700 1 -- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409e000 msgr2=0x7f560411e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5602d9d700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409e000 0x7f560411e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5602d9d700 1 -- 172.21.15.89:0/2101728307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc009a70 con 0x7f560409e410 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.097+0000 7f5601d9b700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409e000 0x7f560411e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.098+0000 7f5602d9d700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560409e410 0x7f560411e9b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f55fc013000 tx=0x7f55fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.100+0000 7f55f2ffd700 1 -- 172.21.15.89:0/2101728307 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc012630 con 0x7f560409e410 2023-09-30T22:20:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.100+0000 7f55f2ffd700 1 -- 172.21.15.89:0/2101728307 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc0127b0 con 0x7f560409e410 2023-09-30T22:20:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.100+0000 7f55f2ffd700 1 -- 172.21.15.89:0/2101728307 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc012b60 con 0x7f560409e410 2023-09-30T22:20:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.100+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56041216d0 con 0x7f560409e410 2023-09-30T22:20:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.100+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5604120710 con 0x7f560409e410 2023-09-30T22:20:55.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.101+0000 7f55f2ffd700 1 -- 172.21.15.89:0/2101728307 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f55fc012cc0 con 0x7f560409e410 2023-09-30T22:20:55.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.101+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5604074ea0 con 0x7f560409e410 2023-09-30T22:20:55.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.101+0000 7f55f2ffd700 1 -- 172.21.15.89:0/2101728307 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f55fc0418e0 con 0x7f560409e410 2023-09-30T22:20:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.106+0000 7f55f2ffd700 1 -- 172.21.15.89:0/2101728307 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55fc009430 con 0x7f560409e410 2023-09-30T22:20:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.108+0000 7f709de95700 1 Processor -- start 2023-09-30T22:20:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.108+0000 7f709de95700 1 -- start start 2023-09-30T22:20:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f709de95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809d6f0 0x7f709809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f709de95700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f709809e000 0x7f7098130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f709de95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809e410 0x7f7098130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f709de95700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709809f520 con 0x7f709809e000 2023-09-30T22:20:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f709de95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709809f130 con 0x7f709809e410 2023-09-30T22:20:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f709de95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f709809e930 con 0x7f709809d6f0 2023-09-30T22:20:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f70977fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809d6f0 0x7f709809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f70977fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809d6f0 0x7f709809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42162/0 (socket says 172.21.15.89:42162) 2023-09-30T22:20:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f70977fe700 1 -- 172.21.15.89:0/4112497116 learned_addr learned my addr 172.21.15.89:0/4112497116 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.109+0000 7f7097fff700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809e410 0x7f7098130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f7096ffd700 1 --2- 172.21.15.89:0/4112497116 >> v2:172.21.15.89:3300/0 conn(0x7f709809e000 0x7f7098130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f7096ffd700 1 -- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809d6f0 msgr2=0x7f709809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f7096ffd700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809d6f0 0x7f709809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f7096ffd700 1 -- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809e410 msgr2=0x7f7098130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f7096ffd700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809e410 0x7f7098130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f7096ffd700 1 -- 172.21.15.89:0/4112497116 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709809b2d0 con 0x7f709809e000 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f7097fff700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809e410 0x7f7098130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f7096ffd700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e000 0x7f7098130660 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f708c009870 tx=0x7f708c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bba91493c475c50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f70967fc700 1 -- 172.21.15.89:0/4112497116 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c011830 con 0x7f709809e000 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f70967fc700 1 -- 172.21.15.89:0/4112497116 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c011990 con 0x7f709809e000 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.110+0000 7f70967fc700 1 -- 172.21.15.89:0/4112497116 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c011d20 con 0x7f709809e000 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 -- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e000 msgr2=0x7f7098130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e000 0x7f7098130660 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f708c009870 tx=0x7f708c009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 -- 172.21.15.89:0/4112497116 shutdown_connections 2023-09-30T22:20:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809e410 0x7f7098130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809d6f0 0x7f709809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 --2- 172.21.15.89:0/4112497116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e000 0x7f7098130660 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 -- 172.21.15.89:0/4112497116 >> 172.21.15.89:0/4112497116 conn(0x7f7098066f50 msgr2=0x7f709809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 -- 172.21.15.89:0/4112497116 shutdown_connections 2023-09-30T22:20:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.111+0000 7f709de95700 1 -- 172.21.15.89:0/4112497116 wait complete. 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.113+0000 7f709de95700 1 Processor -- start 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.113+0000 7f709de95700 1 -- start start 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f709de95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809d6f0 0x7f709811f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f709de95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809e000 0x7f709811f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f709de95700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e410 0x7f709811faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f709de95700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70981204f0 con 0x7f709809e410 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f709de95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70981303f0 con 0x7f709809d6f0 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f709de95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70981225a0 con 0x7f709809e000 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809e000 0x7f709811f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809e000 0x7f709811f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42172/0 (socket says 172.21.15.89:42172) 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 -- 172.21.15.89:0/154807118 learned_addr learned my addr 172.21.15.89:0/154807118 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f70977fe700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809d6f0 0x7f709811f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7097fff700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e410 0x7f709811faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 -- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809d6f0 msgr2=0x7f709811f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809d6f0 0x7f709811f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 -- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e410 msgr2=0x7f709811faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e410 0x7f709811faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 -- 172.21.15.89:0/154807118 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7090009a70 con 0x7f709809e000 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.114+0000 7f7096ffd700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809e000 0x7f709811f560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f708c0090b0 tx=0x7f708c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.115+0000 7f707ffff700 1 -- 172.21.15.89:0/154807118 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c004510 con 0x7f709809e000 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.115+0000 7f707ffff700 1 -- 172.21.15.89:0/154807118 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c01f070 con 0x7f709809e000 2023-09-30T22:20:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.115+0000 7f707ffff700 1 -- 172.21.15.89:0/154807118 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c0038c0 con 0x7f709809e000 2023-09-30T22:20:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.115+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c0094a0 con 0x7f709809e000 2023-09-30T22:20:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.115+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709806eb40 con 0x7f709809e000 2023-09-30T22:20:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.116+0000 7f707ffff700 1 -- 172.21.15.89:0/154807118 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f708c003a20 con 0x7f709809e000 2023-09-30T22:20:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.116+0000 7f707ffff700 1 -- 172.21.15.89:0/154807118 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f708c021070 con 0x7f709809e000 2023-09-30T22:20:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.116+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7098074ea0 con 0x7f709809e000 2023-09-30T22:20:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.121+0000 7f707ffff700 1 -- 172.21.15.89:0/154807118 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f708c015020 con 0x7f709809e000 2023-09-30T22:20:55.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.174+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8ab0005590 con 0x7f8ad0133e30 2023-09-30T22:20:55.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.174+0000 7f8ab77fe700 1 -- 172.21.15.89:0/2748438479 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ac401b430 con 0x7f8ad0133e30 2023-09-30T22:20:55.174 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:55.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 msgr2=0x7f8ad009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 --2- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad009c9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009bc0 tx=0x7f8ac4004750 comp rx=0 tx=0).stop 2023-09-30T22:20:55.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 shutdown_connections 2023-09-30T22:20:55.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 --2- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad0134260 0x7f8ad009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 --2- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad0133420 0x7f8ad009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 --2- 172.21.15.89:0/2748438479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8ad0133e30 0x7f8ad009c9c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 >> 172.21.15.89:0/2748438479 conn(0x7f8ad00a4210 msgr2=0x7f8ad00a0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:55.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 shutdown_connections 2023-09-30T22:20:55.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.177+0000 7f8ad5538700 1 -- 172.21.15.89:0/2748438479 wait complete. 2023-09-30T22:20:55.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:55.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.283+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f77cc122060 con 0x7f77cc1338a0 2023-09-30T22:20:55.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.283+0000 7f77baffd700 1 -- 172.21.15.89:0/2410494875 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f77c4018020 con 0x7f77cc1338a0 2023-09-30T22:20:55.283 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc1338a0 msgr2=0x7f77cc11f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc1338a0 0x7f77cc11f420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f77c400efc0 tx=0x7f77c400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 shutdown_connections 2023-09-30T22:20:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77cc1338a0 0x7f77cc11f420 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77cc09de50 0x7f77cc11e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 --2- 172.21.15.89:0/2410494875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77cc09e760 0x7f77cc11eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 >> 172.21.15.89:0/2410494875 conn(0x7f77cc0a2aa0 msgr2=0x7f77cc09ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:55.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 shutdown_connections 2023-09-30T22:20:55.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.286+0000 7f77d091e700 1 -- 172.21.15.89:0/2410494875 wait complete. 2023-09-30T22:20:55.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:55.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.304+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f560406c360 con 0x7f560409e410 2023-09-30T22:20:55.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.304+0000 7f55f2ffd700 1 -- 172.21.15.89:0/2101728307 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55fc011b80 con 0x7f560409e410 2023-09-30T22:20:55.304 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:55.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560409e410 msgr2=0x7f560411e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560409e410 0x7f560411e9b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f55fc013000 tx=0x7f55fc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 shutdown_connections 2023-09-30T22:20:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560409e410 0x7f560411e9b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560409d6f0 0x7f560411df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 --2- 172.21.15.89:0/2101728307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560409e000 0x7f560411e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 >> 172.21.15.89:0/2101728307 conn(0x7f5604066f50 msgr2=0x7f5604070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 shutdown_connections 2023-09-30T22:20:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.307+0000 7f5608d32700 1 -- 172.21.15.89:0/2101728307 wait complete. 2023-09-30T22:20:55.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:55.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.333+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f709806c360 con 0x7f709809e000 2023-09-30T22:20:55.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.333+0000 7f707ffff700 1 -- 172.21.15.89:0/154807118 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f708c02b070 con 0x7f709809e000 2023-09-30T22:20:55.333 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:55.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809e000 msgr2=0x7f709811f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:55.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809e000 0x7f709811f560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f708c0090b0 tx=0x7f708c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:55.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 shutdown_connections 2023-09-30T22:20:55.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f709809e410 0x7f709811faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f709809d6f0 0x7f709811f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 --2- 172.21.15.89:0/154807118 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f709809e000 0x7f709811f560 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:55.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 >> 172.21.15.89:0/154807118 conn(0x7f7098066f50 msgr2=0x7f7098119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:55.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 shutdown_connections 2023-09-30T22:20:55.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:55.336+0000 7f709de95700 1 -- 172.21.15.89:0/154807118 wait complete. 2023-09-30T22:20:55.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:56.195 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:56.303 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:56.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:56.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.331+0000 7f0d5a441700 1 Processor -- start 2023-09-30T22:20:56.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.331+0000 7f0d5a441700 1 -- start start 2023-09-30T22:20:56.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.331+0000 7f0d5a441700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d5409d6f0 0x7f0d5409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.331+0000 7f0d5a441700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e410 0x7f0d54130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.331+0000 7f0d5a441700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d5409f130 con 0x7f0d5409e000 2023-09-30T22:20:56.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.331+0000 7f0d5a441700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5409f520 con 0x7f0d5409d6f0 2023-09-30T22:20:56.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.331+0000 7f0d5a441700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5409e930 con 0x7f0d5409e410 2023-09-30T22:20:56.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d53fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d5409d6f0 0x7f0d5409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d589de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e410 0x7f0d54130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d53fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d5409d6f0 0x7f0d5409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34180/0 (socket says 172.21.15.89:34180) 2023-09-30T22:20:56.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d589de700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e410 0x7f0d54130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54342/0 (socket says 172.21.15.89:54342) 2023-09-30T22:20:56.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d53fff700 1 -- 172.21.15.89:0/129770870 learned_addr learned my addr 172.21.15.89:0/129770870 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:56.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d53fff700 1 -- 172.21.15.89:0/129770870 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e410 msgr2=0x7f0d54130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:56.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d53fff700 1 --2- 172.21.15.89:0/129770870 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e410 0x7f0d54130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d53fff700 1 -- 172.21.15.89:0/129770870 >> v1:172.21.15.89:6789/0 conn(0x7f0d5409e000 legacy=0x7f0d54130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d53fff700 1 -- 172.21.15.89:0/129770870 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5409b2d0 con 0x7f0d5409d6f0 2023-09-30T22:20:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d589de700 1 --2- 172.21.15.89:0/129770870 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e410 0x7f0d54130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d53fff700 1 --2- 172.21.15.89:0/129770870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409d6f0 0x7f0d5409dac0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400dd20 tx=0x7f0d4400f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444680b7d040682c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d52ffd700 1 -- 172.21.15.89:0/129770870 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44012070 con 0x7f0d5409d6f0 2023-09-30T22:20:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d52ffd700 1 -- 172.21.15.89:0/129770870 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4400de70 con 0x7f0d5409d6f0 2023-09-30T22:20:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.332+0000 7f0d52ffd700 1 -- 172.21.15.89:0/129770870 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400fd20 con 0x7f0d5409d6f0 2023-09-30T22:20:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.333+0000 7f0d5a441700 1 -- 172.21.15.89:0/129770870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409d6f0 msgr2=0x7f0d5409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.333+0000 7f0d5a441700 1 --2- 172.21.15.89:0/129770870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409d6f0 0x7f0d5409dac0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400dd20 tx=0x7f0d4400f330 comp rx=0 tx=0).stop 2023-09-30T22:20:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.333+0000 7f0d5a441700 1 -- 172.21.15.89:0/129770870 shutdown_connections 2023-09-30T22:20:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.333+0000 7f0d5a441700 1 --2- 172.21.15.89:0/129770870 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e410 0x7f0d54130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.333+0000 7f0d5a441700 1 --2- 172.21.15.89:0/129770870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409d6f0 0x7f0d5409dac0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.333+0000 7f0d5a441700 1 -- 172.21.15.89:0/129770870 >> 172.21.15.89:0/129770870 conn(0x7f0d54066f50 msgr2=0x7f0d5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.333+0000 7f0d5a441700 1 -- 172.21.15.89:0/129770870 shutdown_connections 2023-09-30T22:20:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.333+0000 7f0d5a441700 1 -- 172.21.15.89:0/129770870 wait complete. 2023-09-30T22:20:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.334+0000 7f0d5a441700 1 Processor -- start 2023-09-30T22:20:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.334+0000 7f0d5a441700 1 -- start start 2023-09-30T22:20:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d5a441700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d5409d6f0 0x7f0d5411df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d5a441700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e000 0x7f0d5411e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d5a441700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409e410 0x7f0d5411e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d5a441700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54121a50 con 0x7f0d5409e410 2023-09-30T22:20:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d5a441700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54120160 con 0x7f0d5409e000 2023-09-30T22:20:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d5a441700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54120e20 con 0x7f0d5409d6f0 2023-09-30T22:20:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d5409d6f0 0x7f0d5411df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d589de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409e410 0x7f0d5411e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d5409d6f0 0x7f0d5411df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42184/0 (socket says 172.21.15.89:42184) 2023-09-30T22:20:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 -- 172.21.15.89:0/558030416 learned_addr learned my addr 172.21.15.89:0/558030416 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d589de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409e410 0x7f0d5411e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34196/0 (socket says 172.21.15.89:34196) 2023-09-30T22:20:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d537fe700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e000 0x7f0d5411e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 -- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e000 msgr2=0x7f0d5411e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e000 0x7f0d5411e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 -- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409e410 msgr2=0x7f0d5411e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409e410 0x7f0d5411e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 -- 172.21.15.89:0/558030416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c009a70 con 0x7f0d5409d6f0 2023-09-30T22:20:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d537fe700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e000 0x7f0d5411e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.335+0000 7f0d53fff700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d5409d6f0 0x7f0d5411df30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0d44018040 tx=0x7f0d44004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.336+0000 7f0d50ff9700 1 -- 172.21.15.89:0/558030416 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44003b30 con 0x7f0d5409d6f0 2023-09-30T22:20:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.336+0000 7f0d50ff9700 1 -- 172.21.15.89:0/558030416 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d44003c90 con 0x7f0d5409d6f0 2023-09-30T22:20:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.336+0000 7f0d50ff9700 1 -- 172.21.15.89:0/558030416 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44016430 con 0x7f0d5409d6f0 2023-09-30T22:20:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.336+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4400da00 con 0x7f0d5409d6f0 2023-09-30T22:20:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.336+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d54121240 con 0x7f0d5409d6f0 2023-09-30T22:20:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.336+0000 7f0d50ff9700 1 -- 172.21.15.89:0/558030416 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d44016590 con 0x7f0d5409d6f0 2023-09-30T22:20:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.337+0000 7f0d50ff9700 1 -- 172.21.15.89:0/558030416 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0d44046870 con 0x7f0d5409d6f0 2023-09-30T22:20:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.339+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d541215d0 con 0x7f0d5409d6f0 2023-09-30T22:20:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.345+0000 7f0d50ff9700 1 -- 172.21.15.89:0/558030416 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d541215d0 con 0x7f0d5409d6f0 2023-09-30T22:20:56.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:56.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.435+0000 7f1b692cd700 1 Processor -- start 2023-09-30T22:20:56.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.435+0000 7f1b692cd700 1 -- start start 2023-09-30T22:20:56.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b692cd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b641338a0 0x7f1b64133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b692cd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b6409de50 0x7f1b6409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b692cd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409e760 0x7f1b6413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b692cd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6406fef0 con 0x7f1b6409de50 2023-09-30T22:20:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b692cd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6406ef30 con 0x7f1b641338a0 2023-09-30T22:20:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b692cd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b641334b0 con 0x7f1b6409e760 2023-09-30T22:20:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b641338a0 0x7f1b64133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b641338a0 0x7f1b64133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54352/0 (socket says 172.21.15.89:54352) 2023-09-30T22:20:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 -- 172.21.15.89:0/3195660505 learned_addr learned my addr 172.21.15.89:0/3195660505 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b637fe700 1 --2- 172.21.15.89:0/3195660505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409e760 0x7f1b6413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 -- 172.21.15.89:0/3195660505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409e760 msgr2=0x7f1b6413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 --2- 172.21.15.89:0/3195660505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409e760 0x7f1b6413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 -- 172.21.15.89:0/3195660505 >> v2:172.21.15.89:3300/0 conn(0x7f1b6409de50 msgr2=0x7f1b6409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 --2- 172.21.15.89:0/3195660505 >> v2:172.21.15.89:3300/0 conn(0x7f1b6409de50 0x7f1b6409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 -- 172.21.15.89:0/3195660505 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6406fc30 con 0x7f1b641338a0 2023-09-30T22:20:56.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b62ffd700 1 --2- 172.21.15.89:0/3195660505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b641338a0 0x7f1b64133c70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1b540098f0 tx=0x7f1b54009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca1d4ebb6beccd7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:56.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.436+0000 7f1b61ffb700 1 -- 172.21.15.89:0/3195660505 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54011830 con 0x7f1b641338a0 2023-09-30T22:20:56.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b61ffb700 1 -- 172.21.15.89:0/3195660505 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b54011990 con 0x7f1b641338a0 2023-09-30T22:20:56.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b61ffb700 1 -- 172.21.15.89:0/3195660505 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54011d20 con 0x7f1b641338a0 2023-09-30T22:20:56.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b692cd700 1 -- 172.21.15.89:0/3195660505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b641338a0 msgr2=0x7f1b64133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b692cd700 1 --2- 172.21.15.89:0/3195660505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b641338a0 0x7f1b64133c70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1b540098f0 tx=0x7f1b54009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:56.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b692cd700 1 -- 172.21.15.89:0/3195660505 shutdown_connections 2023-09-30T22:20:56.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b692cd700 1 --2- 172.21.15.89:0/3195660505 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409e760 0x7f1b6413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b692cd700 1 --2- 172.21.15.89:0/3195660505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b641338a0 0x7f1b64133c70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b692cd700 1 --2- 172.21.15.89:0/3195660505 >> v2:172.21.15.89:3300/0 conn(0x7f1b6409de50 0x7f1b6409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.437+0000 7f1b692cd700 1 -- 172.21.15.89:0/3195660505 >> 172.21.15.89:0/3195660505 conn(0x7f1b640a2aa0 msgr2=0x7f1b640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:56.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.438+0000 7f1b692cd700 1 -- 172.21.15.89:0/3195660505 shutdown_connections 2023-09-30T22:20:56.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.438+0000 7f1b692cd700 1 -- 172.21.15.89:0/3195660505 wait complete. 2023-09-30T22:20:56.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.439+0000 7f1b692cd700 1 Processor -- start 2023-09-30T22:20:56.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.439+0000 7f1b692cd700 1 -- start start 2023-09-30T22:20:56.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b692cd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409de50 0x7f1b6411ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b692cd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b6409e760 0x7f1b6411ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b692cd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b641338a0 0x7f1b6411f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b692cd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6412f9c0 con 0x7f1b641338a0 2023-09-30T22:20:56.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b692cd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64120720 con 0x7f1b6409e760 2023-09-30T22:20:56.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b692cd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64121f80 con 0x7f1b6409de50 2023-09-30T22:20:56.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b627fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b6409e760 0x7f1b6411ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b641338a0 0x7f1b6411f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b627fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b6409e760 0x7f1b6411ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54358/0 (socket says 172.21.15.89:54358) 2023-09-30T22:20:56.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b627fc700 1 -- 172.21.15.89:0/1349382552 learned_addr learned my addr 172.21.15.89:0/1349382552 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:56.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b62ffd700 1 --2- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409de50 0x7f1b6411ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b637fe700 1 -- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409de50 msgr2=0x7f1b6411ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b637fe700 1 --2- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409de50 0x7f1b6411ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b637fe700 1 -- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b6409e760 msgr2=0x7f1b6411ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b637fe700 1 --2- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b6409e760 0x7f1b6411ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.440+0000 7f1b637fe700 1 -- 172.21.15.89:0/1349382552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b54009550 con 0x7f1b641338a0 2023-09-30T22:20:56.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.441+0000 7f1b637fe700 1 --2- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b641338a0 0x7f1b6411f480 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00efc0 tx=0x7f1b5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:56.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.441+0000 7f1b4b7fe700 1 -- 172.21.15.89:0/1349382552 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00cc90 con 0x7f1b641338a0 2023-09-30T22:20:56.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.441+0000 7f1b4b7fe700 1 -- 172.21.15.89:0/1349382552 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c00f040 con 0x7f1b641338a0 2023-09-30T22:20:56.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.441+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6406f640 con 0x7f1b641338a0 2023-09-30T22:20:56.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.441+0000 7f1b4b7fe700 1 -- 172.21.15.89:0/1349382552 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0146c0 con 0x7f1b641338a0 2023-09-30T22:20:56.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.441+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b64122cb0 con 0x7f1b641338a0 2023-09-30T22:20:56.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.442+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b64074ea0 con 0x7f1b641338a0 2023-09-30T22:20:56.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.442+0000 7f1b4b7fe700 1 -- 172.21.15.89:0/1349382552 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b5c014820 con 0x7f1b641338a0 2023-09-30T22:20:56.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.442+0000 7f1b4b7fe700 1 -- 172.21.15.89:0/1349382552 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1b5c0401b0 con 0x7f1b641338a0 2023-09-30T22:20:56.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.451+0000 7f1b4b7fe700 1 -- 172.21.15.89:0/1349382552 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b5c014ad0 con 0x7f1b641338a0 2023-09-30T22:20:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.462+0000 7fd5b2ad6700 1 Processor -- start 2023-09-30T22:20:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.462+0000 7fd5b2ad6700 1 -- start start 2023-09-30T22:20:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b2ad6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b2ad6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09e000 0x7fd5ac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b2ad6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac09b120 con 0x7fd5ac09e410 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b2ad6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac06b460 con 0x7fd5ac09d6f0 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b2ad6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac09a7e0 con 0x7fd5ac09e000 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b0872700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b0872700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54366/0 (socket says 172.21.15.89:54366) 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b0872700 1 -- 172.21.15.89:0/1089914075 learned_addr learned my addr 172.21.15.89:0/1089914075 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b1073700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd5ac09e410 0x7fd5ac133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34516/0 (socket says 172.21.15.89:34516) 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5abfff700 1 --2- 172.21.15.89:0/1089914075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09e000 0x7fd5ac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b0872700 1 -- 172.21.15.89:0/1089914075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09e000 msgr2=0x7fd5ac1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b0872700 1 --2- 172.21.15.89:0/1089914075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09e000 0x7fd5ac1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b0872700 1 -- 172.21.15.89:0/1089914075 >> v1:172.21.15.89:6789/0 conn(0x7fd5ac09e410 legacy=0x7fd5ac133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5ab7fe700 1 -- 172.21.15.89:0/1089914075 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690052277 0 0) 0x7fd5ac09b120 con 0x7fd5ac09e410 2023-09-30T22:20:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.463+0000 7fd5b0872700 1 -- 172.21.15.89:0/1089914075 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac09a500 con 0x7fd5ac09d6f0 2023-09-30T22:20:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5b0872700 1 --2- 172.21.15.89:0/1089914075 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009920 tx=0x7fd5a0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce28f5900d830f95 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5ab7fe700 1 -- 172.21.15.89:0/1089914075 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000ba60 con 0x7fd5ac09d6f0 2023-09-30T22:20:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5ab7fe700 1 -- 172.21.15.89:0/1089914075 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a000bbc0 con 0x7fd5ac09d6f0 2023-09-30T22:20:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5ab7fe700 1 -- 172.21.15.89:0/1089914075 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0009e70 con 0x7fd5ac09d6f0 2023-09-30T22:20:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/1089914075 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09d6f0 msgr2=0x7fd5ac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5b2ad6700 1 --2- 172.21.15.89:0/1089914075 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009920 tx=0x7fd5a0009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/1089914075 shutdown_connections 2023-09-30T22:20:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5b2ad6700 1 --2- 172.21.15.89:0/1089914075 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09d6f0 0x7fd5ac09dac0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5b2ad6700 1 --2- 172.21.15.89:0/1089914075 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09e000 0x7fd5ac1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.464+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/1089914075 >> 172.21.15.89:0/1089914075 conn(0x7fd5ac066f50 msgr2=0x7fd5ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.465+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/1089914075 shutdown_connections 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.465+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/1089914075 wait complete. 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.465+0000 7fd5b2ad6700 1 Processor -- start 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.465+0000 7fd5b2ad6700 1 -- start start 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5b2ad6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09d6f0 0x7fd5ac1a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5b2ad6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5ac09e000 0x7fd5ac1a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5b2ad6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09e410 0x7fd5ac1c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5b2ad6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1a8120 con 0x7fd5ac09e000 2023-09-30T22:20:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5b2ad6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1a6ce0 con 0x7fd5ac09e410 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5b2ad6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1a8300 con 0x7fd5ac09d6f0 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5abfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5ac09e000 0x7fd5ac1a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5b1073700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09e410 0x7fd5ac1c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5abfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5ac09e000 0x7fd5ac1a58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34226/0 (socket says 172.21.15.89:34226) 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5abfff700 1 -- 172.21.15.89:0/58518932 learned_addr learned my addr 172.21.15.89:0/58518932 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.466+0000 7fd5b0872700 1 --2- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09d6f0 0x7fd5ac1a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5b1073700 1 -- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09d6f0 msgr2=0x7fd5ac1a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5b1073700 1 --2- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09d6f0 0x7fd5ac1a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5b1073700 1 -- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5ac09e000 msgr2=0x7fd5ac1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5b1073700 1 --2- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5ac09e000 0x7fd5ac1a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5b1073700 1 -- 172.21.15.89:0/58518932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a0009550 con 0x7fd5ac09e410 2023-09-30T22:20:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5b1073700 1 --2- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09e410 0x7fd5ac1c7fd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400cc10 tx=0x7fd5a400cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5a97fa700 1 -- 172.21.15.89:0/58518932 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400a9c0 con 0x7fd5ac09e410 2023-09-30T22:20:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5a97fa700 1 -- 172.21.15.89:0/58518932 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a400ab20 con 0x7fd5ac09e410 2023-09-30T22:20:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac1a7ab0 con 0x7fd5ac09e410 2023-09-30T22:20:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5a97fa700 1 -- 172.21.15.89:0/58518932 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a40070d0 con 0x7fd5ac09e410 2023-09-30T22:20:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.467+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac09c710 con 0x7fd5ac09e410 2023-09-30T22:20:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.468+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac074ea0 con 0x7fd5ac09e410 2023-09-30T22:20:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.468+0000 7fd5a97fa700 1 -- 172.21.15.89:0/58518932 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5a4007230 con 0x7fd5ac09e410 2023-09-30T22:20:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.468+0000 7fd5a97fa700 1 -- 172.21.15.89:0/58518932 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd5a4042030 con 0x7fd5ac09e410 2023-09-30T22:20:56.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.473+0000 7fd5a97fa700 1 -- 172.21.15.89:0/58518932 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5a401a8b0 con 0x7fd5ac09e410 2023-09-30T22:20:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.483+0000 7f965780c700 1 Processor -- start 2023-09-30T22:20:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.483+0000 7f965780c700 1 -- start start 2023-09-30T22:20:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f965780c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009e930 0x7f965009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f965780c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f965009f240 0x7f96501317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f965780c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9650070940 con 0x7f965009f650 2023-09-30T22:20:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f965780c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965006fd60 con 0x7f965009f240 2023-09-30T22:20:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f965780c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f965006fa80 con 0x7f965009e930 2023-09-30T22:20:56.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f9655da9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f965009f650 0x7f9650131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34520/0 (socket says 172.21.15.89:34520) 2023-09-30T22:20:56.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f9655da9700 1 -- 172.21.15.89:0/1647555418 learned_addr learned my addr 172.21.15.89:0/1647555418 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:56.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f96555a8700 1 --2- 172.21.15.89:0/1647555418 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009e930 0x7f965009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f9654da7700 1 --2- 172.21.15.89:0/1647555418 >> v2:172.21.15.89:3300/0 conn(0x7f965009f240 0x7f96501317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f9647fff700 1 -- 172.21.15.89:0/1647555418 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577334152 0 0) 0x7f9650070940 con 0x7f965009f650 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f9647fff700 1 -- 172.21.15.89:0/1647555418 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f963c0035b0 con 0x7f965009f650 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f96555a8700 1 -- 172.21.15.89:0/1647555418 >> v2:172.21.15.89:3300/0 conn(0x7f965009f240 msgr2=0x7f96501317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f96555a8700 1 --2- 172.21.15.89:0/1647555418 >> v2:172.21.15.89:3300/0 conn(0x7f965009f240 0x7f96501317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f96555a8700 1 -- 172.21.15.89:0/1647555418 >> v1:172.21.15.89:6789/0 conn(0x7f965009f650 legacy=0x7f9650131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f96555a8700 1 -- 172.21.15.89:0/1647555418 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965009fb60 con 0x7f965009e930 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.484+0000 7f9654da7700 1 --2- 172.21.15.89:0/1647555418 >> v2:172.21.15.89:3300/0 conn(0x7f965009f240 0x7f96501317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.485+0000 7f96555a8700 1 --2- 172.21.15.89:0/1647555418 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009e930 0x7f965009ed00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f964c009840 tx=0x7f964c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb0006fc51cf4ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.485+0000 7f9647fff700 1 -- 172.21.15.89:0/1647555418 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c004230 con 0x7f965009e930 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.485+0000 7f9647fff700 1 -- 172.21.15.89:0/1647555418 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c004390 con 0x7f965009e930 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.485+0000 7f9647fff700 1 -- 172.21.15.89:0/1647555418 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0036a0 con 0x7f965009e930 2023-09-30T22:20:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.485+0000 7f965780c700 1 -- 172.21.15.89:0/1647555418 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009e930 msgr2=0x7f965009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.485+0000 7f965780c700 1 --2- 172.21.15.89:0/1647555418 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009e930 0x7f965009ed00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f964c009840 tx=0x7f964c009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.486+0000 7f965780c700 1 -- 172.21.15.89:0/1647555418 shutdown_connections 2023-09-30T22:20:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.486+0000 7f965780c700 1 --2- 172.21.15.89:0/1647555418 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009e930 0x7f965009ed00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.486+0000 7f965780c700 1 --2- 172.21.15.89:0/1647555418 >> v2:172.21.15.89:3300/0 conn(0x7f965009f240 0x7f96501317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.486+0000 7f965780c700 1 -- 172.21.15.89:0/1647555418 >> 172.21.15.89:0/1647555418 conn(0x7f96500a4210 msgr2=0x7f96500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.486+0000 7f965780c700 1 -- 172.21.15.89:0/1647555418 shutdown_connections 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.486+0000 7f965780c700 1 -- 172.21.15.89:0/1647555418 wait complete. 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.487+0000 7f965780c700 1 Processor -- start 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.487+0000 7f965780c700 1 -- start start 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f965780c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f965009e930 0x7f965013a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f965780c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009f240 0x7f96501326e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f96555a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f965009e930 0x7f965013a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f96555a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f965009e930 0x7f965013a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54382/0 (socket says 172.21.15.89:54382) 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f965780c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f965009f650 0x7f9650132c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f96555a8700 1 -- 172.21.15.89:0/397814221 learned_addr learned my addr 172.21.15.89:0/397814221 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650135060 con 0x7f965009f650 2023-09-30T22:20:56.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965006d5e0 con 0x7f965009e930 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9650135420 con 0x7f965009f240 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f9654da7700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009f240 0x7f96501326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f9655da9700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f965009f650 0x7f9650132c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f96555a8700 1 -- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009f240 msgr2=0x7f96501326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f96555a8700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009f240 0x7f96501326e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f96555a8700 1 -- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f965009f650 msgr2=0x7f9650132c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f96555a8700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f965009f650 0x7f9650132c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.488+0000 7f96555a8700 1 -- 172.21.15.89:0/397814221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400094a0 con 0x7f965009e930 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.489+0000 7f96555a8700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f965009e930 0x7f965013a080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f965006fbc0 tx=0x7f964c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.489+0000 7f9645ffb700 1 -- 172.21.15.89:0/397814221 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c003cf0 con 0x7f965009e930 2023-09-30T22:20:56.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.489+0000 7f9645ffb700 1 -- 172.21.15.89:0/397814221 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c003e70 con 0x7f965009e930 2023-09-30T22:20:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.489+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c0094a0 con 0x7f965009e930 2023-09-30T22:20:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.489+0000 7f9645ffb700 1 -- 172.21.15.89:0/397814221 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c011850 con 0x7f965009e930 2023-09-30T22:20:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.489+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96500a0190 con 0x7f965009e930 2023-09-30T22:20:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.490+0000 7f9645ffb700 1 -- 172.21.15.89:0/397814221 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f964c018050 con 0x7f965009e930 2023-09-30T22:20:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.490+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9650135830 con 0x7f965009e930 2023-09-30T22:20:56.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.490+0000 7f9645ffb700 1 -- 172.21.15.89:0/397814221 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f964c040090 con 0x7f965009e930 2023-09-30T22:20:56.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.495+0000 7f9645ffb700 1 -- 172.21.15.89:0/397814221 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f964c024dc0 con 0x7f965009e930 2023-09-30T22:20:56.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.606+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0d541215d0 con 0x7f0d5409d6f0 2023-09-30T22:20:56.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.607+0000 7f0d50ff9700 1 -- 172.21.15.89:0/558030416 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d541215d0 con 0x7f0d5409d6f0 2023-09-30T22:20:56.607 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d5409d6f0 msgr2=0x7f0d5411df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d5409d6f0 0x7f0d5411df30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0d44018040 tx=0x7f0d44004750 comp rx=0 tx=0).stop 2023-09-30T22:20:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 shutdown_connections 2023-09-30T22:20:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d5409e410 0x7f0d5411e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d5409d6f0 0x7f0d5411df30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 --2- 172.21.15.89:0/558030416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d5409e000 0x7f0d5411e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 >> 172.21.15.89:0/558030416 conn(0x7f0d54066f50 msgr2=0x7f0d540710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 shutdown_connections 2023-09-30T22:20:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.609+0000 7f0d5a441700 1 -- 172.21.15.89:0/558030416 wait complete. 2023-09-30T22:20:56.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.670+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd5ac06c360 con 0x7fd5ac09e410 2023-09-30T22:20:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.671+0000 7fd5a97fa700 1 -- 172.21.15.89:0/58518932 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5a4018030 con 0x7fd5ac09e410 2023-09-30T22:20:56.671 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09e410 msgr2=0x7fd5ac1c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 --2- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09e410 0x7fd5ac1c7fd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400cc10 tx=0x7fd5a400cf20 comp rx=0 tx=0).stop 2023-09-30T22:20:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 shutdown_connections 2023-09-30T22:20:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 --2- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5ac09e410 0x7fd5ac1c7fd0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 --2- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5ac09d6f0 0x7fd5ac1a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 --2- 172.21.15.89:0/58518932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5ac09e000 0x7fd5ac1a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 >> 172.21.15.89:0/58518932 conn(0x7fd5ac066f50 msgr2=0x7fd5ac0a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 shutdown_connections 2023-09-30T22:20:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.673+0000 7fd5b2ad6700 1 -- 172.21.15.89:0/58518932 wait complete. 2023-09-30T22:20:56.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:56.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.692+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9650074ea0 con 0x7f965009e930 2023-09-30T22:20:56.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.692+0000 7f9645ffb700 1 -- 172.21.15.89:0/397814221 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f964c016020 con 0x7f965009e930 2023-09-30T22:20:56.692 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:56.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f965009e930 msgr2=0x7f965013a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f965009e930 0x7f965013a080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f965006fbc0 tx=0x7f964c004750 comp rx=0 tx=0).stop 2023-09-30T22:20:56.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 shutdown_connections 2023-09-30T22:20:56.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f965009f650 0x7f9650132c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f965009e930 0x7f965013a080 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 --2- 172.21.15.89:0/397814221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f965009f240 0x7f96501326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 >> 172.21.15.89:0/397814221 conn(0x7f96500a4210 msgr2=0x7f965006cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:56.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 shutdown_connections 2023-09-30T22:20:56.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.694+0000 7f965780c700 1 -- 172.21.15.89:0/397814221 wait complete. 2023-09-30T22:20:56.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.700+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1b6406b9e0 con 0x7f1b641338a0 2023-09-30T22:20:56.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.700+0000 7f1b4b7fe700 1 -- 172.21.15.89:0/1349382552 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b5c012340 con 0x7f1b641338a0 2023-09-30T22:20:56.700 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b641338a0 msgr2=0x7f1b6411f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 --2- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b641338a0 0x7f1b6411f480 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00efc0 tx=0x7f1b5c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 shutdown_connections 2023-09-30T22:20:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 --2- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b641338a0 0x7f1b6411f480 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 --2- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b6409de50 0x7f1b6411ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 --2- 172.21.15.89:0/1349382552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b6409e760 0x7f1b6411ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 >> 172.21.15.89:0/1349382552 conn(0x7f1b640a2aa0 msgr2=0x7f1b6409ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 shutdown_connections 2023-09-30T22:20:56.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:56.702+0000 7f1b692cd700 1 -- 172.21.15.89:0/1349382552 wait complete. 2023-09-30T22:20:56.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:56.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:57.623 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:57.687 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:57.708 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:57.714 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:57.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2bf2996700 1 Processor -- start 2023-09-30T22:20:57.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2bf2996700 1 -- start start 2023-09-30T22:20:57.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2bf2996700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2bf2996700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2bf2996700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec09a810 con 0x7f2bec13c1e0 2023-09-30T22:20:57.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2bf2996700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec06ab40 con 0x7f2bec09dc30 2023-09-30T22:20:57.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2bf2996700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec09ebc0 con 0x7f2bec06a770 2023-09-30T22:20:57.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2beb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2beb7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34244/0 (socket says 172.21.15.89:34244) 2023-09-30T22:20:57.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.763+0000 7f2beb7fe700 1 -- 172.21.15.89:0/1076557375 learned_addr learned my addr 172.21.15.89:0/1076557375 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:57.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2bebfff700 1 --2- 172.21.15.89:0/1076557375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2bebfff700 1 -- 172.21.15.89:0/1076557375 >> v2:172.21.15.89:3300/0 conn(0x7f2bec09dc30 msgr2=0x7f2bec09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2bebfff700 1 --2- 172.21.15.89:0/1076557375 >> v2:172.21.15.89:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2bebfff700 1 -- 172.21.15.89:0/1076557375 >> v1:172.21.15.89:6789/0 conn(0x7f2bec13c1e0 legacy=0x7f2bec13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2bebfff700 1 -- 172.21.15.89:0/1076557375 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec09e920 con 0x7f2bec06a770 2023-09-30T22:20:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2beb7fe700 1 --2- 172.21.15.89:0/1076557375 >> v2:172.21.15.89:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2bebfff700 1 --2- 172.21.15.89:0/1076557375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4354ca32d9fb95c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2beaffd700 1 -- 172.21.15.89:0/1076557375 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011830 con 0x7f2bec06a770 2023-09-30T22:20:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2beaffd700 1 -- 172.21.15.89:0/1076557375 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc011990 con 0x7f2bec06a770 2023-09-30T22:20:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.764+0000 7f2beaffd700 1 -- 172.21.15.89:0/1076557375 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011d00 con 0x7f2bec06a770 2023-09-30T22:20:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.765+0000 7f2bf2996700 1 -- 172.21.15.89:0/1076557375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec06a770 msgr2=0x7f2bec09d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.765+0000 7f2bf2996700 1 --2- 172.21.15.89:0/1076557375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.765+0000 7f2bf2996700 1 -- 172.21.15.89:0/1076557375 shutdown_connections 2023-09-30T22:20:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.765+0000 7f2bf2996700 1 --2- 172.21.15.89:0/1076557375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.765+0000 7f2bf2996700 1 --2- 172.21.15.89:0/1076557375 >> v2:172.21.15.89:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.765+0000 7f2bf2996700 1 -- 172.21.15.89:0/1076557375 >> 172.21.15.89:0/1076557375 conn(0x7f2bec066f50 msgr2=0x7f2bec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:57.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.765+0000 7f2bf2996700 1 -- 172.21.15.89:0/1076557375 shutdown_connections 2023-09-30T22:20:57.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.765+0000 7f2bf2996700 1 -- 172.21.15.89:0/1076557375 wait complete. 2023-09-30T22:20:57.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.766+0000 7f2bf2996700 1 Processor -- start 2023-09-30T22:20:57.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.766+0000 7f2bf2996700 1 -- start start 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2bf2996700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2bec06a770 0x7f2bec13e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2bf2996700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec09dc30 0x7f2bec13a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2bf2996700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bec13c1e0 0x7f2bec13ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2bf2996700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1401c0 con 0x7f2bec13c1e0 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2bf2996700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec13efa0 con 0x7f2bec09dc30 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2bf2996700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec13bee0 con 0x7f2bec06a770 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec09dc30 0x7f2bec13a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec09dc30 0x7f2bec13a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54392/0 (socket says 172.21.15.89:54392) 2023-09-30T22:20:57.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 -- 172.21.15.89:0/3526728620 learned_addr learned my addr 172.21.15.89:0/3526728620 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2bf0f33700 1 --2- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bec13c1e0 0x7f2bec13ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 -- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2bec06a770 msgr2=0x7f2bec13e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 --2- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2bec06a770 0x7f2bec13e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 -- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bec13c1e0 msgr2=0x7f2bec13ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 --2- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bec13c1e0 0x7f2bec13ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 -- 172.21.15.89:0/3526728620 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0009550 con 0x7f2bec09dc30 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.767+0000 7f2beb7fe700 1 --2- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec09dc30 0x7f2bec13a6c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ebe0 tx=0x7f2be000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.768+0000 7f2be8ff9700 1 -- 172.21.15.89:0/3526728620 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be000c850 con 0x7f2bec09dc30 2023-09-30T22:20:57.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.768+0000 7f2be8ff9700 1 -- 172.21.15.89:0/3526728620 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be000c9b0 con 0x7f2bec09dc30 2023-09-30T22:20:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.768+0000 7f2be8ff9700 1 -- 172.21.15.89:0/3526728620 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be000cd60 con 0x7f2bec09dc30 2023-09-30T22:20:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.768+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc0094a0 con 0x7f2bec09dc30 2023-09-30T22:20:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.768+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec13f790 con 0x7f2bec09dc30 2023-09-30T22:20:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.769+0000 7f2be8ff9700 1 -- 172.21.15.89:0/3526728620 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2be0013470 con 0x7f2bec09dc30 2023-09-30T22:20:57.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.769+0000 7f2be8ff9700 1 -- 172.21.15.89:0/3526728620 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2be003f820 con 0x7f2bec09dc30 2023-09-30T22:20:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.771+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec09da50 con 0x7f2bec09dc30 2023-09-30T22:20:57.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.775+0000 7f2be8ff9700 1 -- 172.21.15.89:0/3526728620 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2be00232f0 con 0x7f2bec09dc30 2023-09-30T22:20:57.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.819+0000 7fee0ac0d700 1 Processor -- start 2023-09-30T22:20:57.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.819+0000 7fee0ac0d700 1 -- start start 2023-09-30T22:20:57.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.819+0000 7fee0ac0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee040a0a30 0x7fee040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.819+0000 7fee0ac0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee040a1340 0x7fee04133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee0ac0d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0409edd0 con 0x7fee04133fa0 2023-09-30T22:20:57.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee0ac0d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0409e1f0 con 0x7fee040a1340 2023-09-30T22:20:57.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee0ac0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee0409dc50 con 0x7fee040a0a30 2023-09-30T22:20:57.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee03fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee040a1340 0x7fee04133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee089a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee040a0a30 0x7fee040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee03fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee040a1340 0x7fee04133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34260/0 (socket says 172.21.15.89:34260) 2023-09-30T22:20:57.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee03fff700 1 -- 172.21.15.89:0/976724955 learned_addr learned my addr 172.21.15.89:0/976724955 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:57.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee089a9700 1 -- 172.21.15.89:0/976724955 >> v2:172.21.15.89:3300/0 conn(0x7fee040a1340 msgr2=0x7fee04133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:57.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee089a9700 1 --2- 172.21.15.89:0/976724955 >> v2:172.21.15.89:3300/0 conn(0x7fee040a1340 0x7fee04133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee089a9700 1 -- 172.21.15.89:0/976724955 >> v1:172.21.15.89:6789/0 conn(0x7fee04133fa0 legacy=0x7fee0413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee089a9700 1 -- 172.21.15.89:0/976724955 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040a1fe0 con 0x7fee040a0a30 2023-09-30T22:20:57.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee089a9700 1 --2- 172.21.15.89:0/976724955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee040a0a30 0x7fee040a0e00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009840 tx=0x7fedf8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2f08c10d13fa62 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:57.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.820+0000 7fee037fe700 1 -- 172.21.15.89:0/976724955 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8011830 con 0x7fee040a0a30 2023-09-30T22:20:57.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee037fe700 1 -- 172.21.15.89:0/976724955 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf8011990 con 0x7fee040a0a30 2023-09-30T22:20:57.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee037fe700 1 -- 172.21.15.89:0/976724955 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8011d20 con 0x7fee040a0a30 2023-09-30T22:20:57.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee0ac0d700 1 -- 172.21.15.89:0/976724955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee040a0a30 msgr2=0x7fee040a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee0ac0d700 1 --2- 172.21.15.89:0/976724955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee040a0a30 0x7fee040a0e00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009840 tx=0x7fedf8009b50 comp rx=0 tx=0).stop 2023-09-30T22:20:57.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee0ac0d700 1 -- 172.21.15.89:0/976724955 shutdown_connections 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee0ac0d700 1 --2- 172.21.15.89:0/976724955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee040a0a30 0x7fee040a0e00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee0ac0d700 1 --2- 172.21.15.89:0/976724955 >> v2:172.21.15.89:3300/0 conn(0x7fee040a1340 0x7fee04133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee0ac0d700 1 -- 172.21.15.89:0/976724955 >> 172.21.15.89:0/976724955 conn(0x7fee040a6550 msgr2=0x7fee040a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee0ac0d700 1 -- 172.21.15.89:0/976724955 shutdown_connections 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.821+0000 7fee0ac0d700 1 -- 172.21.15.89:0/976724955 wait complete. 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.822+0000 7fee0ac0d700 1 Processor -- start 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.822+0000 7fee0ac0d700 1 -- start start 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee0ac0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee040a0a30 0x7fee0411e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee0ac0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee040a1340 0x7fee0411ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee0ac0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee04133fa0 0x7fee0411f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee0ac0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee04122090 con 0x7fee040a0a30 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee0ac0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee04120570 con 0x7fee040a1340 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee0ac0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee040a37e0 con 0x7fee04133fa0 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee089a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee040a0a30 0x7fee0411e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee089a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee040a0a30 0x7fee0411e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34276/0 (socket says 172.21.15.89:34276) 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee089a9700 1 -- 172.21.15.89:0/2278268007 learned_addr learned my addr 172.21.15.89:0/2278268007 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee091aa700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee04133fa0 0x7fee0411f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.823+0000 7fee03fff700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee040a1340 0x7fee0411ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee091aa700 1 -- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee040a1340 msgr2=0x7fee0411ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee091aa700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee040a1340 0x7fee0411ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee091aa700 1 -- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee040a0a30 msgr2=0x7fee0411e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee091aa700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee040a0a30 0x7fee0411e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee091aa700 1 -- 172.21.15.89:0/2278268007 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80094a0 con 0x7fee04133fa0 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee091aa700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee04133fa0 0x7fee0411f360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00cb90 tx=0x7fedfc00cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee017fa700 1 -- 172.21.15.89:0/2278268007 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc003c00 con 0x7fee04133fa0 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee040a0220 con 0x7fee04133fa0 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee017fa700 1 -- 172.21.15.89:0/2278268007 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc003d60 con 0x7fee04133fa0 2023-09-30T22:20:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee017fa700 1 -- 172.21.15.89:0/2278268007 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00aa80 con 0x7fee04133fa0 2023-09-30T22:20:57.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.824+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee04122a70 con 0x7fee04133fa0 2023-09-30T22:20:57.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.825+0000 7fee017fa700 1 -- 172.21.15.89:0/2278268007 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fedfc00abe0 con 0x7fee04133fa0 2023-09-30T22:20:57.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.825+0000 7fee017fa700 1 -- 172.21.15.89:0/2278268007 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fedfc03f5a0 con 0x7fee04133fa0 2023-09-30T22:20:57.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.826+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee040a1d40 con 0x7fee04133fa0 2023-09-30T22:20:57.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.830+0000 7fee017fa700 1 -- 172.21.15.89:0/2278268007 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fedfc0198f0 con 0x7fee04133fa0 2023-09-30T22:20:57.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.861+0000 7fe5b2b1d700 1 Processor -- start 2023-09-30T22:20:57.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.861+0000 7fe5b2b1d700 1 -- start start 2023-09-30T22:20:57.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5b2b1d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5b2b1d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5ac09f240 0x7fe5ac1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5b2b1d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ac070940 con 0x7fe5ac09f650 2023-09-30T22:20:57.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5b2b1d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac06fd60 con 0x7fe5ac09f240 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5b2b1d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac06fa80 con 0x7fe5ac09e930 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5abfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5ac09f240 0x7fe5ac1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5abfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5ac09f240 0x7fe5ac1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34284/0 (socket says 172.21.15.89:34284) 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5abfff700 1 -- 172.21.15.89:0/3834167768 learned_addr learned my addr 172.21.15.89:0/3834167768 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5b08b9700 1 --2- 172.21.15.89:0/3834167768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5abfff700 1 -- 172.21.15.89:0/3834167768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 msgr2=0x7fe5ac09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5abfff700 1 --2- 172.21.15.89:0/3834167768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.862+0000 7fe5abfff700 1 -- 172.21.15.89:0/3834167768 >> v1:172.21.15.89:6789/0 conn(0x7fe5ac09f650 legacy=0x7fe5ac131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.863+0000 7fe5abfff700 1 -- 172.21.15.89:0/3834167768 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac09fb60 con 0x7fe5ac09f240 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.863+0000 7fe5abfff700 1 --2- 172.21.15.89:0/3834167768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5ac09f240 0x7fe5ac1317f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe5980098f0 tx=0x7fe598009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87985372a4be0d3a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.863+0000 7fe5ab7fe700 1 -- 172.21.15.89:0/3834167768 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598004230 con 0x7fe5ac09f240 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.863+0000 7fe5ab7fe700 1 -- 172.21.15.89:0/3834167768 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe598004390 con 0x7fe5ac09f240 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.863+0000 7fe5ab7fe700 1 -- 172.21.15.89:0/3834167768 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5980036a0 con 0x7fe5ac09f240 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.863+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/3834167768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5ac09f240 msgr2=0x7fe5ac1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.863+0000 7fe5b2b1d700 1 --2- 172.21.15.89:0/3834167768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5ac09f240 0x7fe5ac1317f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe5980098f0 tx=0x7fe598009c00 comp rx=0 tx=0).stop 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.864+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/3834167768 shutdown_connections 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.864+0000 7fe5b2b1d700 1 --2- 172.21.15.89:0/3834167768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.864+0000 7fe5b2b1d700 1 --2- 172.21.15.89:0/3834167768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5ac09f240 0x7fe5ac1317f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.864+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/3834167768 >> 172.21.15.89:0/3834167768 conn(0x7fe5ac0a4210 msgr2=0x7fe5ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.864+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/3834167768 shutdown_connections 2023-09-30T22:20:57.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.864+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/3834167768 wait complete. 2023-09-30T22:20:57.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.865+0000 7fe5b2b1d700 1 Processor -- start 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.865+0000 7fe5b2b1d700 1 -- start start 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b2b1d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b2b1d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5ac09f240 0x7fe5ac1a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b2b1d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5ac09f650 0x7fe5ac1a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b2b1d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac1a7160 con 0x7fe5ac09f650 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b2b1d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac1a78c0 con 0x7fe5ac09f240 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b2b1d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac1a8b10 con 0x7fe5ac09e930 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac1a5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42292/0 (socket says 172.21.15.89:42292) 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 -- 172.21.15.89:0/907602711 learned_addr learned my addr 172.21.15.89:0/907602711 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:57.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5abfff700 1 --2- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5ac09f240 0x7fe5ac1a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 -- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5ac09f240 msgr2=0x7fe5ac1a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 --2- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5ac09f240 0x7fe5ac1a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 -- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5ac09f650 msgr2=0x7fe5ac1a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 --2- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5ac09f650 0x7fe5ac1a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 -- 172.21.15.89:0/907602711 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe598009550 con 0x7fe5ac09e930 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5b08b9700 1 --2- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac1a5cb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000e9e0 tx=0x7fe5a000ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.866+0000 7fe5a97fa700 1 -- 172.21.15.89:0/907602711 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0009710 con 0x7fe5ac09e930 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.867+0000 7fe5a97fa700 1 -- 172.21.15.89:0/907602711 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a000c850 con 0x7fe5ac09e930 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.867+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac0a0190 con 0x7fe5ac09e930 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.867+0000 7fe5a97fa700 1 -- 172.21.15.89:0/907602711 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000cbc0 con 0x7fe5ac09e930 2023-09-30T22:20:57.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.867+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac0a34a0 con 0x7fe5ac09e930 2023-09-30T22:20:57.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.868+0000 7fe5a97fa700 1 -- 172.21.15.89:0/907602711 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5a000cd20 con 0x7fe5ac09e930 2023-09-30T22:20:57.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.868+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac1a2b40 con 0x7fe5ac09e930 2023-09-30T22:20:57.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.868+0000 7fe5a97fa700 1 -- 172.21.15.89:0/907602711 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe5a003f7d0 con 0x7fe5ac09e930 2023-09-30T22:20:57.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.872+0000 7fe5a97fa700 1 -- 172.21.15.89:0/907602711 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5a00232f0 con 0x7fe5ac09e930 2023-09-30T22:20:57.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.911+0000 7fc85b3d0700 1 Processor -- start 2023-09-30T22:20:57.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.911+0000 7fc85b3d0700 1 -- start start 2023-09-30T22:20:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85b3d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 0x7fc85409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85b3d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc85413bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85b3d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85409f960 con 0x7fc85409f160 2023-09-30T22:20:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85b3d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85409ae60 con 0x7fc854134300 2023-09-30T22:20:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85b3d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc85406cb10 con 0x7fc85409e850 2023-09-30T22:20:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85896b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc85409f160 0x7fc854133bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34562/0 (socket says 172.21.15.89:34562) 2023-09-30T22:20:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85896b700 1 -- 172.21.15.89:0/365013834 learned_addr learned my addr 172.21.15.89:0/365013834 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85996d700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc85413bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc85916c700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 0x7fc85409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134161855 0 0) 0x7fc85409f960 con 0x7fc85409f160 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.912+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8400035b0 con 0x7fc85409f160 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 984307377 0 0) 0x7fc8400035b0 con 0x7fc85409f160 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 msgr2=0x7fc85409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 0x7fc85409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 msgr2=0x7fc85413bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc85413bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85406b990 con 0x7fc85409f160 2023-09-30T22:20:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85996d700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc85413bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7fc844001670 con 0x7fc85409f160 2023-09-30T22:20:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 >> v1:172.21.15.89:6789/0 conn(0x7fc85409f160 legacy=0x7fc854133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc840003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 0x7fc840003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc840004850 0x7fc840004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85406cb10 con 0x7fc840004850 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc840005300 con 0x7fc854134300 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400054c0 con 0x7fc85409e850 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85996d700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc840004850 0x7fc840004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85916c700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc840003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85896b700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 0x7fc840003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85996d700 1 -- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 msgr2=0x7fc840003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85996d700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 0x7fc840003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85996d700 1 -- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 msgr2=0x7fc840003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85996d700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc840003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85996d700 1 -- 172.21.15.89:0/365013834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c009a70 con 0x7fc840004850 2023-09-30T22:20:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85896b700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 0x7fc840003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85996d700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc840004850 0x7fc840004cc0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00e350 tx=0x7fc840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3164f896f4b5b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.913+0000 7fc85916c700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc840003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.914+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00edf0 con 0x7fc840004850 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.914+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c00f040 con 0x7fc840004850 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.914+0000 7fc84bfff700 1 -- 172.21.15.89:0/365013834 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c014680 con 0x7fc840004850 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.914+0000 7fc85b3d0700 1 -- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc840004850 msgr2=0x7fc840004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.914+0000 7fc85b3d0700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc840004850 0x7fc840004cc0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00e350 tx=0x7fc840005880 comp rx=0 tx=0).stop 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.915+0000 7fc85b3d0700 1 -- 172.21.15.89:0/365013834 shutdown_connections 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.915+0000 7fc85b3d0700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc840004850 0x7fc840004cc0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.915+0000 7fc85b3d0700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc854134300 0x7fc840003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.915+0000 7fc85b3d0700 1 --2- 172.21.15.89:0/365013834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85409e850 0x7fc840003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.915+0000 7fc85b3d0700 1 -- 172.21.15.89:0/365013834 >> 172.21.15.89:0/365013834 conn(0x7fc8540a3f70 msgr2=0x7fc8540a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:57.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.915+0000 7fc85b3d0700 1 -- 172.21.15.89:0/365013834 shutdown_connections 2023-09-30T22:20:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.915+0000 7fc85b3d0700 1 -- 172.21.15.89:0/365013834 wait complete. 2023-09-30T22:20:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.916+0000 7fc85b3d0700 1 Processor -- start 2023-09-30T22:20:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.916+0000 7fc85b3d0700 1 -- start start 2023-09-30T22:20:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85b3d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc840004850 0x7fc8541cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85b3d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85409e850 0x7fc8541cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85b3d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85409f160 0x7fc85411a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85b3d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85406cda0 con 0x7fc85409f160 2023-09-30T22:20:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85b3d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8541335a0 con 0x7fc85409e850 2023-09-30T22:20:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85b3d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc85411e890 con 0x7fc840004850 2023-09-30T22:20:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85896b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85409e850 0x7fc8541cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85916c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc840004850 0x7fc8541cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85896b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85409e850 0x7fc8541cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54432/0 (socket says 172.21.15.89:54432) 2023-09-30T22:20:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85896b700 1 -- 172.21.15.89:0/1910420901 learned_addr learned my addr 172.21.15.89:0/1910420901 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85996d700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85409f160 0x7fc85411a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85896b700 1 -- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc840004850 msgr2=0x7fc8541cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85896b700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc840004850 0x7fc8541cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85896b700 1 -- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85409f160 msgr2=0x7fc85411a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85896b700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85409f160 0x7fc85411a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85896b700 1 -- 172.21.15.89:0/1910420901 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85411de90 con 0x7fc85409e850 2023-09-30T22:20:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.917+0000 7fc85996d700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85409f160 0x7fc85411a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:20:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.918+0000 7fc85896b700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85409e850 0x7fc8541cccf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e7b0 tx=0x7fc84400eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.918+0000 7fc849ffb700 1 -- 172.21.15.89:0/1910420901 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844012590 con 0x7fc85409e850 2023-09-30T22:20:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.918+0000 7fc849ffb700 1 -- 172.21.15.89:0/1910420901 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8440126f0 con 0x7fc85409e850 2023-09-30T22:20:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.918+0000 7fc849ffb700 1 -- 172.21.15.89:0/1910420901 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844012aa0 con 0x7fc85409e850 2023-09-30T22:20:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.918+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc854133960 con 0x7fc85409e850 2023-09-30T22:20:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.918+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8541cd230 con 0x7fc85409e850 2023-09-30T22:20:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.919+0000 7fc849ffb700 1 -- 172.21.15.89:0/1910420901 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc844012c00 con 0x7fc85409e850 2023-09-30T22:20:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.919+0000 7fc849ffb700 1 -- 172.21.15.89:0/1910420901 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc84403f490 con 0x7fc85409e850 2023-09-30T22:20:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.919+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8541cd3c0 con 0x7fc85409e850 2023-09-30T22:20:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:57.924+0000 7fc849ffb700 1 -- 172.21.15.89:0/1910420901 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc844012eb0 con 0x7fc85409e850 2023-09-30T22:20:58.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.049+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fee04074ea0 con 0x7fee04133fa0 2023-09-30T22:20:58.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.050+0000 7fee017fa700 1 -- 172.21.15.89:0/2278268007 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fedfc011380 con 0x7fee04133fa0 2023-09-30T22:20:58.050 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee04133fa0 msgr2=0x7fee0411f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee04133fa0 0x7fee0411f360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00cb90 tx=0x7fedfc00cea0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 shutdown_connections 2023-09-30T22:20:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee04133fa0 0x7fee0411f360 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee040a0a30 0x7fee0411e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 --2- 172.21.15.89:0/2278268007 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee040a1340 0x7fee0411ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 >> 172.21.15.89:0/2278268007 conn(0x7fee040a6550 msgr2=0x7fee041b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:58.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 shutdown_connections 2023-09-30T22:20:58.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.053+0000 7fee0ac0d700 1 -- 172.21.15.89:0/2278268007 wait complete. 2023-09-30T22:20:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.059+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2bec074ea0 con 0x7f2bec09dc30 2023-09-30T22:20:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.060+0000 7f2be8ff9700 1 -- 172.21.15.89:0/3526728620 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2be0021030 con 0x7f2bec09dc30 2023-09-30T22:20:58.060 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:58.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec09dc30 msgr2=0x7f2bec13a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:58.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 --2- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec09dc30 0x7f2bec13a6c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ebe0 tx=0x7f2be000eef0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 shutdown_connections 2023-09-30T22:20:58.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 --2- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bec13c1e0 0x7f2bec13ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 --2- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2bec06a770 0x7f2bec13e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 --2- 172.21.15.89:0/3526728620 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2bec09dc30 0x7f2bec13a6c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 >> 172.21.15.89:0/3526728620 conn(0x7f2bec066f50 msgr2=0x7f2bec072280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:58.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 shutdown_connections 2023-09-30T22:20:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.062+0000 7f2bf2996700 1 -- 172.21.15.89:0/3526728620 wait complete. 2023-09-30T22:20:58.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:58.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:58.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.097+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe5ac071d20 con 0x7fe5ac09e930 2023-09-30T22:20:58.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.097+0000 7fe5a97fa700 1 -- 172.21.15.89:0/907602711 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5a0021070 con 0x7fe5ac09e930 2023-09-30T22:20:58.097 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 msgr2=0x7fe5ac1a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 --2- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac1a5cb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000e9e0 tx=0x7fe5a000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 shutdown_connections 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 --2- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5ac09f650 0x7fe5ac1a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 --2- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5ac09e930 0x7fe5ac1a5cb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 --2- 172.21.15.89:0/907602711 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5ac09f240 0x7fe5ac1a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 >> 172.21.15.89:0/907602711 conn(0x7fe5ac0a4210 msgr2=0x7fe5ac09de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 shutdown_connections 2023-09-30T22:20:58.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.099+0000 7fe5b2b1d700 1 -- 172.21.15.89:0/907602711 wait complete. 2023-09-30T22:20:58.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:58.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.142+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc8540a1620 con 0x7fc85409e850 2023-09-30T22:20:58.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.142+0000 7fc849ffb700 1 -- 172.21.15.89:0/1910420901 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc844010af0 con 0x7fc85409e850 2023-09-30T22:20:58.142 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:58.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85409e850 msgr2=0x7fc8541cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:58.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85409e850 0x7fc8541cccf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e7b0 tx=0x7fc84400eac0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 shutdown_connections 2023-09-30T22:20:58.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85409f160 0x7fc85411a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc840004850 0x7fc8541cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 --2- 172.21.15.89:0/1910420901 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85409e850 0x7fc8541cccf0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:58.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 >> 172.21.15.89:0/1910420901 conn(0x7fc8540a3f70 msgr2=0x7fc854070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:58.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 shutdown_connections 2023-09-30T22:20:58.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:58.144+0000 7fc85b3d0700 1 -- 172.21.15.89:0/1910420901 wait complete. 2023-09-30T22:20:58.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:20:59.071 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:20:59.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:20:59.111 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:20:59.157 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:20:59.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.218+0000 7f66cccab700 1 Processor -- start 2023-09-30T22:20:59.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.218+0000 7f66cccab700 1 -- start start 2023-09-30T22:20:59.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66cccab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c81316c0 0x7f66c8131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66cccab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c806ae80 0x7f66c8139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66cccab700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c809e130 con 0x7f66c809d9f0 2023-09-30T22:20:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66cccab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81312b0 con 0x7f66c81316c0 2023-09-30T22:20:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66cccab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66c809f6e0 con 0x7f66c806ae80 2023-09-30T22:20:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66c5d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f66c809d9f0 0x7f66c806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34570/0 (socket says 172.21.15.89:34570) 2023-09-30T22:20:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66c659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c81316c0 0x7f66c8131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66c5d9b700 1 -- 172.21.15.89:0/2962662448 learned_addr learned my addr 172.21.15.89:0/2962662448 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:20:59.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66c6d9d700 1 --2- 172.21.15.89:0/2962662448 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c806ae80 0x7f66c8139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66c659c700 1 -- 172.21.15.89:0/2962662448 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c806ae80 msgr2=0x7f66c8139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:59.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66c659c700 1 --2- 172.21.15.89:0/2962662448 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c806ae80 0x7f66c8139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66c659c700 1 -- 172.21.15.89:0/2962662448 >> v1:172.21.15.89:6789/0 conn(0x7f66c809d9f0 legacy=0x7f66c806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.219+0000 7f66c659c700 1 -- 172.21.15.89:0/2962662448 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c809e760 con 0x7f66c81316c0 2023-09-30T22:20:59.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.220+0000 7f66c659c700 1 --2- 172.21.15.89:0/2962662448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c81316c0 0x7f66c8131a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009870 tx=0x7f66b8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7309a4e3329241 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:59.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.220+0000 7f66c559a700 1 -- 172.21.15.89:0/2962662448 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8011830 con 0x7f66c81316c0 2023-09-30T22:20:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.220+0000 7f66c559a700 1 -- 172.21.15.89:0/2962662448 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b8011990 con 0x7f66c81316c0 2023-09-30T22:20:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.220+0000 7f66c559a700 1 -- 172.21.15.89:0/2962662448 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8011d20 con 0x7f66c81316c0 2023-09-30T22:20:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.221+0000 7f66cccab700 1 -- 172.21.15.89:0/2962662448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c81316c0 msgr2=0x7f66c8131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.221+0000 7f66cccab700 1 --2- 172.21.15.89:0/2962662448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c81316c0 0x7f66c8131a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009870 tx=0x7f66b8009b80 comp rx=0 tx=0).stop 2023-09-30T22:20:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.221+0000 7f66cccab700 1 -- 172.21.15.89:0/2962662448 shutdown_connections 2023-09-30T22:20:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.221+0000 7f66cccab700 1 --2- 172.21.15.89:0/2962662448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c81316c0 0x7f66c8131a90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.221+0000 7f66cccab700 1 --2- 172.21.15.89:0/2962662448 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c806ae80 0x7f66c8139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.221+0000 7f66cccab700 1 -- 172.21.15.89:0/2962662448 >> 172.21.15.89:0/2962662448 conn(0x7f66c8066f50 msgr2=0x7f66c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.221+0000 7f66cccab700 1 -- 172.21.15.89:0/2962662448 shutdown_connections 2023-09-30T22:20:59.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.221+0000 7f66cccab700 1 -- 172.21.15.89:0/2962662448 wait complete. 2023-09-30T22:20:59.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.223+0000 7f66cccab700 1 Processor -- start 2023-09-30T22:20:59.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.223+0000 7f66cccab700 1 -- start start 2023-09-30T22:20:59.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.223+0000 7f66cccab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c806ae80 0x7f66c80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.223+0000 7f66cccab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c809d9f0 0x7f66c8071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.223+0000 7f66cccab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c81316c0 0x7f66c8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.223+0000 7f66cccab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81328b0 con 0x7f66c806ae80 2023-09-30T22:20:59.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.223+0000 7f66cccab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8133480 con 0x7f66c809d9f0 2023-09-30T22:20:59.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.223+0000 7f66cccab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66c813a590 con 0x7f66c81316c0 2023-09-30T22:20:59.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c81316c0 0x7f66c8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c81316c0 0x7f66c8071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42320/0 (socket says 172.21.15.89:42320) 2023-09-30T22:20:59.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 -- 172.21.15.89:0/2142525891 learned_addr learned my addr 172.21.15.89:0/2142525891 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:59.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c5d9b700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c809d9f0 0x7f66c8071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c659c700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c806ae80 0x7f66c80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 -- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c809d9f0 msgr2=0x7f66c8071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c809d9f0 0x7f66c8071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 -- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c806ae80 msgr2=0x7f66c80712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c806ae80 0x7f66c80712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 -- 172.21.15.89:0/2142525891 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80094a0 con 0x7f66c81316c0 2023-09-30T22:20:59.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c659c700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c806ae80 0x7f66c80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:59.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c5d9b700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c809d9f0 0x7f66c8071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:59.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66c6d9d700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c81316c0 0x7f66c8071d40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f66c000efc0 tx=0x7f66c000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:59.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66b6ffd700 1 -- 172.21.15.89:0/2142525891 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0009ea0 con 0x7f66c81316c0 2023-09-30T22:20:59.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66b6ffd700 1 -- 172.21.15.89:0/2142525891 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c000f040 con 0x7f66c81316c0 2023-09-30T22:20:59.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.224+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c813a160 con 0x7f66c81316c0 2023-09-30T22:20:59.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.225+0000 7f66b6ffd700 1 -- 172.21.15.89:0/2142525891 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c000ced0 con 0x7f66c81316c0 2023-09-30T22:20:59.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.225+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c8133000 con 0x7f66c81316c0 2023-09-30T22:20:59.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.225+0000 7f66b6ffd700 1 -- 172.21.15.89:0/2142525891 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66c0014480 con 0x7f66c81316c0 2023-09-30T22:20:59.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.226+0000 7f66b6ffd700 1 -- 172.21.15.89:0/2142525891 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f66c003fea0 con 0x7f66c81316c0 2023-09-30T22:20:59.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.226+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c8074ea0 con 0x7f66c81316c0 2023-09-30T22:20:59.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.231+0000 7f66b6ffd700 1 -- 172.21.15.89:0/2142525891 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f66c0018020 con 0x7f66c81316c0 2023-09-30T22:20:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.253+0000 7fd649f4a700 1 Processor -- start 2023-09-30T22:20:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.253+0000 7fd649f4a700 1 -- start start 2023-09-30T22:20:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.253+0000 7fd649f4a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409d6f0 0x7fd64409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.253+0000 7fd649f4a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd64409e410 0x7fd644134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.253+0000 7fd649f4a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64409b120 con 0x7fd64409e000 2023-09-30T22:20:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.253+0000 7fd649f4a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64406b460 con 0x7fd64409e410 2023-09-30T22:20:59.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.253+0000 7fd649f4a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd64409a7e0 con 0x7fd64409d6f0 2023-09-30T22:20:59.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409d6f0 0x7fd64409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409d6f0 0x7fd64409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42332/0 (socket says 172.21.15.89:42332) 2023-09-30T22:20:59.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6437fe700 1 -- 172.21.15.89:0/41105608 learned_addr learned my addr 172.21.15.89:0/41105608 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:59.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd642ffd700 1 --1- 172.21.15.89:0/41105608 >> v1:172.21.15.89:6789/0 conn(0x7fd64409e000 0x7fd6441339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34586/0 (socket says 172.21.15.89:34586) 2023-09-30T22:20:59.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd643fff700 1 --2- 172.21.15.89:0/41105608 >> v2:172.21.15.89:3300/0 conn(0x7fd64409e410 0x7fd644134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6437fe700 1 -- 172.21.15.89:0/41105608 >> v2:172.21.15.89:3300/0 conn(0x7fd64409e410 msgr2=0x7fd644134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:59.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6437fe700 1 --2- 172.21.15.89:0/41105608 >> v2:172.21.15.89:3300/0 conn(0x7fd64409e410 0x7fd644134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6437fe700 1 -- 172.21.15.89:0/41105608 >> v1:172.21.15.89:6789/0 conn(0x7fd64409e000 legacy=0x7fd6441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6437fe700 1 -- 172.21.15.89:0/41105608 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64409a500 con 0x7fd64409d6f0 2023-09-30T22:20:59.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6437fe700 1 --2- 172.21.15.89:0/41105608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409d6f0 0x7fd64409dac0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd634009920 tx=0x7fd634009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6016d3fd751568f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:59.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6427fc700 1 -- 172.21.15.89:0/41105608 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634011830 con 0x7fd64409d6f0 2023-09-30T22:20:59.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6427fc700 1 -- 172.21.15.89:0/41105608 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634011990 con 0x7fd64409d6f0 2023-09-30T22:20:59.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.254+0000 7fd6427fc700 1 -- 172.21.15.89:0/41105608 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634011d20 con 0x7fd64409d6f0 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.255+0000 7fd649f4a700 1 -- 172.21.15.89:0/41105608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409d6f0 msgr2=0x7fd64409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.255+0000 7fd649f4a700 1 --2- 172.21.15.89:0/41105608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409d6f0 0x7fd64409dac0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd634009920 tx=0x7fd634009c30 comp rx=0 tx=0).stop 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.255+0000 7fd649f4a700 1 -- 172.21.15.89:0/41105608 shutdown_connections 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.255+0000 7fd649f4a700 1 --2- 172.21.15.89:0/41105608 >> v2:172.21.15.89:3300/0 conn(0x7fd64409e410 0x7fd644134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.255+0000 7fd649f4a700 1 --2- 172.21.15.89:0/41105608 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409d6f0 0x7fd64409dac0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.255+0000 7fd649f4a700 1 -- 172.21.15.89:0/41105608 >> 172.21.15.89:0/41105608 conn(0x7fd644066f50 msgr2=0x7fd64409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.255+0000 7fd649f4a700 1 -- 172.21.15.89:0/41105608 shutdown_connections 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.255+0000 7fd649f4a700 1 -- 172.21.15.89:0/41105608 wait complete. 2023-09-30T22:20:59.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.256+0000 7fd649f4a700 1 Processor -- start 2023-09-30T22:20:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd649f4a700 1 -- start start 2023-09-30T22:20:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd649f4a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64409d6f0 0x7fd6441a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd649f4a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64409e000 0x7fd6441a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd649f4a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409e410 0x7fd6441a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd649f4a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6441ab880 con 0x7fd64409d6f0 2023-09-30T22:20:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd649f4a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64409c710 con 0x7fd64409e000 2023-09-30T22:20:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd649f4a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6441a9ba0 con 0x7fd64409e410 2023-09-30T22:20:59.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd642ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64409e000 0x7fd6441a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd642ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64409e000 0x7fd6441a85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54460/0 (socket says 172.21.15.89:54460) 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd642ffd700 1 -- 172.21.15.89:0/2955828362 learned_addr learned my addr 172.21.15.89:0/2955828362 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd643fff700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409e410 0x7fd6441a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd6437fe700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64409d6f0 0x7fd6441a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd643fff700 1 -- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64409e000 msgr2=0x7fd6441a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.257+0000 7fd643fff700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64409e000 0x7fd6441a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd643fff700 1 -- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64409d6f0 msgr2=0x7fd6441a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd643fff700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64409d6f0 0x7fd6441a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd643fff700 1 -- 172.21.15.89:0/2955828362 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634009550 con 0x7fd64409e410 2023-09-30T22:20:59.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd642ffd700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64409e000 0x7fd6441a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd643fff700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409e410 0x7fd6441a8af0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009d90 tx=0x7fd63c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd62bfff700 1 -- 172.21.15.89:0/2955828362 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c009c00 con 0x7fd64409e410 2023-09-30T22:20:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6441abae0 con 0x7fd64409e410 2023-09-30T22:20:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd62bfff700 1 -- 172.21.15.89:0/2955828362 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c00f040 con 0x7fd64409e410 2023-09-30T22:20:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd62bfff700 1 -- 172.21.15.89:0/2955828362 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c014480 con 0x7fd64409e410 2023-09-30T22:20:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.258+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6441aa290 con 0x7fd64409e410 2023-09-30T22:20:59.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.259+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd644074ea0 con 0x7fd64409e410 2023-09-30T22:20:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.260+0000 7fd62bfff700 1 -- 172.21.15.89:0/2955828362 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd63c0145e0 con 0x7fd64409e410 2023-09-30T22:20:59.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.260+0000 7fd62bfff700 1 -- 172.21.15.89:0/2955828362 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd63c040300 con 0x7fd64409e410 2023-09-30T22:20:59.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.264+0000 7fe920deb700 1 Processor -- start 2023-09-30T22:20:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.264+0000 7fe920deb700 1 -- start start 2023-09-30T22:20:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe920deb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe91c12cf60 0x7fe91c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe920deb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c130430 0x7fe91c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe920deb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91c12cd90 con 0x7fe91c12f8b0 2023-09-30T22:20:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe920deb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91c12d330 con 0x7fe91c12cf60 2023-09-30T22:20:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe920deb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c071780 con 0x7fe91c130430 2023-09-30T22:20:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe91a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe91c12cf60 0x7fe91c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe91a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe91c12cf60 0x7fe91c06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34352/0 (socket says 172.21.15.89:34352) 2023-09-30T22:20:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe91a59c700 1 -- 172.21.15.89:0/3139749766 learned_addr learned my addr 172.21.15.89:0/3139749766 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:59.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe91a59c700 1 -- 172.21.15.89:0/3139749766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c130430 msgr2=0x7fe91c137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:59.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe91a59c700 1 --2- 172.21.15.89:0/3139749766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c130430 0x7fe91c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe91a59c700 1 -- 172.21.15.89:0/3139749766 >> v1:172.21.15.89:6789/0 conn(0x7fe91c12f8b0 legacy=0x7fe91c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe91a59c700 1 -- 172.21.15.89:0/3139749766 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c071a40 con 0x7fe91c12cf60 2023-09-30T22:20:59.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.265+0000 7fe91a59c700 1 --2- 172.21.15.89:0/3139749766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c12cf60 0x7fe91c06f5d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00aa90 tx=0x7fe90c00ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f58999f4644e23 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:59.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe91959a700 1 -- 172.21.15.89:0/3139749766 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00cb70 con 0x7fe91c12cf60 2023-09-30T22:20:59.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe91959a700 1 -- 172.21.15.89:0/3139749766 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c00ccd0 con 0x7fe91c12cf60 2023-09-30T22:20:59.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe91959a700 1 -- 172.21.15.89:0/3139749766 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c018620 con 0x7fe91c12cf60 2023-09-30T22:20:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe920deb700 1 -- 172.21.15.89:0/3139749766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c12cf60 msgr2=0x7fe91c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe920deb700 1 --2- 172.21.15.89:0/3139749766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c12cf60 0x7fe91c06f5d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00aa90 tx=0x7fe90c00ada0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe920deb700 1 -- 172.21.15.89:0/3139749766 shutdown_connections 2023-09-30T22:20:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe920deb700 1 --2- 172.21.15.89:0/3139749766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c12cf60 0x7fe91c06f5d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe920deb700 1 --2- 172.21.15.89:0/3139749766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c130430 0x7fe91c137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe920deb700 1 -- 172.21.15.89:0/3139749766 >> 172.21.15.89:0/3139749766 conn(0x7fe91c0a2aa0 msgr2=0x7fe91c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe920deb700 1 -- 172.21.15.89:0/3139749766 shutdown_connections 2023-09-30T22:20:59.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.266+0000 7fe920deb700 1 -- 172.21.15.89:0/3139749766 wait complete. 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.268+0000 7fe920deb700 1 Processor -- start 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.268+0000 7fe920deb700 1 -- start start 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.268+0000 7fe920deb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c12cf60 0x7fe91c09be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.268+0000 7fe920deb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe91c12f8b0 0x7fe91c09c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe920deb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c130430 0x7fe91c1c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe920deb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c09ecf0 con 0x7fe91c130430 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe920deb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c129be0 con 0x7fe91c12f8b0 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe920deb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c09ba60 con 0x7fe91c12cf60 2023-09-30T22:20:59.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe91a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c12cf60 0x7fe91c09be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe91a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c12cf60 0x7fe91c09be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42354/0 (socket says 172.21.15.89:42354) 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe91ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c130430 0x7fe91c1c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe91a59c700 1 -- 172.21.15.89:0/2086786468 learned_addr learned my addr 172.21.15.89:0/2086786468 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe919d9b700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe91c12f8b0 0x7fe91c09c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe919d9b700 1 -- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c12cf60 msgr2=0x7fe91c09be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe919d9b700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c12cf60 0x7fe91c09be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe919d9b700 1 -- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c130430 msgr2=0x7fe91c1c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe919d9b700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c130430 0x7fe91c1c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe919d9b700 1 -- 172.21.15.89:0/2086786468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c00a6f0 con 0x7fe91c12f8b0 2023-09-30T22:20:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe91a59c700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c12cf60 0x7fe91c09be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.269+0000 7fe919d9b700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe91c12f8b0 0x7fe91c09c380 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe91000c450 tx=0x7fe91000c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.270+0000 7fe90affd700 1 -- 172.21.15.89:0/2086786468 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100052d0 con 0x7fe91c12f8b0 2023-09-30T22:20:59.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.270+0000 7fd62bfff700 1 -- 172.21.15.89:0/2955828362 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd63c007900 con 0x7fd64409e410 2023-09-30T22:20:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.270+0000 7fe90affd700 1 -- 172.21.15.89:0/2086786468 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910005430 con 0x7fe91c12f8b0 2023-09-30T22:20:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.270+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91c09c920 con 0x7fe91c12f8b0 2023-09-30T22:20:59.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.270+0000 7fe90affd700 1 -- 172.21.15.89:0/2086786468 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100038c0 con 0x7fe91c12f8b0 2023-09-30T22:20:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.270+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91c09dab0 con 0x7fe91c12f8b0 2023-09-30T22:20:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.271+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91c06fbd0 con 0x7fe91c12f8b0 2023-09-30T22:20:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.272+0000 7fe90affd700 1 -- 172.21.15.89:0/2086786468 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe910003a20 con 0x7fe91c12f8b0 2023-09-30T22:20:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.272+0000 7fe90affd700 1 -- 172.21.15.89:0/2086786468 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe9100408e0 con 0x7fe91c12f8b0 2023-09-30T22:20:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.276+0000 7fe90affd700 1 -- 172.21.15.89:0/2086786468 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe910044060 con 0x7fe91c12f8b0 2023-09-30T22:20:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.300+0000 7f765d9c9700 1 Processor -- start 2023-09-30T22:20:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.300+0000 7f765d9c9700 1 -- start start 2023-09-30T22:20:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f765d9c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765809eca0 0x7f765813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f765d9c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f765806f050 0x7f765813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f765d9c9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7658070ee0 con 0x7f765806e570 2023-09-30T22:20:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f765d9c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7658071980 con 0x7f765809eca0 2023-09-30T22:20:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f765d9c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7658071480 con 0x7f765806f050 2023-09-30T22:20:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76567fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765809eca0 0x7f765813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76577fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f765806f050 0x7f765813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76567fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765809eca0 0x7f765813cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54474/0 (socket says 172.21.15.89:54474) 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76577fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f765806f050 0x7f765813f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42362/0 (socket says 172.21.15.89:42362) 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76567fc700 1 -- 172.21.15.89:0/2670673834 learned_addr learned my addr 172.21.15.89:0/2670673834 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76577fe700 1 -- 172.21.15.89:0/2670673834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765809eca0 msgr2=0x7f765813cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76577fe700 1 --2- 172.21.15.89:0/2670673834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765809eca0 0x7f765813cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76577fe700 1 -- 172.21.15.89:0/2670673834 >> v1:172.21.15.89:6789/0 conn(0x7f765806e570 legacy=0x7f765806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76577fe700 1 -- 172.21.15.89:0/2670673834 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658071b90 con 0x7f765806f050 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.301+0000 7f76567fc700 1 --2- 172.21.15.89:0/2670673834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765809eca0 0x7f765813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:20:59.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.302+0000 7f76577fe700 1 --2- 172.21.15.89:0/2670673834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f765806f050 0x7f765813f3f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7650009dc0 tx=0x7f765000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3216692e724de99e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.302+0000 7f7655ffb700 1 -- 172.21.15.89:0/2670673834 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765000e070 con 0x7f765806f050 2023-09-30T22:20:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.302+0000 7f7655ffb700 1 -- 172.21.15.89:0/2670673834 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765000bae0 con 0x7f765806f050 2023-09-30T22:20:59.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.302+0000 7f7655ffb700 1 -- 172.21.15.89:0/2670673834 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765000be90 con 0x7f765806f050 2023-09-30T22:20:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.303+0000 7f765d9c9700 1 -- 172.21.15.89:0/2670673834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f765806f050 msgr2=0x7f765813f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.303+0000 7f765d9c9700 1 --2- 172.21.15.89:0/2670673834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f765806f050 0x7f765813f3f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7650009dc0 tx=0x7f765000b330 comp rx=0 tx=0).stop 2023-09-30T22:20:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.303+0000 7f765d9c9700 1 -- 172.21.15.89:0/2670673834 shutdown_connections 2023-09-30T22:20:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.303+0000 7f765d9c9700 1 --2- 172.21.15.89:0/2670673834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f765806f050 0x7f765813f3f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.303+0000 7f765d9c9700 1 --2- 172.21.15.89:0/2670673834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765809eca0 0x7f765813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.303+0000 7f765d9c9700 1 -- 172.21.15.89:0/2670673834 >> 172.21.15.89:0/2670673834 conn(0x7f76580a2aa0 msgr2=0x7f76580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:59.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.303+0000 7f765d9c9700 1 -- 172.21.15.89:0/2670673834 shutdown_connections 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.303+0000 7f765d9c9700 1 -- 172.21.15.89:0/2670673834 wait complete. 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.304+0000 7f765d9c9700 1 Processor -- start 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.304+0000 7f765d9c9700 1 -- start start 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f765d9c9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765806e570 0x7f76581214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f765d9c9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f765806f050 0x7f7658121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f765d9c9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7658121f60 0x7f76581cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f765d9c9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658123320 con 0x7f765806f050 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f765d9c9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765809a270 con 0x7f765806e570 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f765d9c9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7658070610 con 0x7f7658121f60 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f7656ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765806e570 0x7f76581214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f76577fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7658121f60 0x7f76581cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f76577fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7658121f60 0x7f76581cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42366/0 (socket says 172.21.15.89:42366) 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f7656ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765806e570 0x7f76581214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54488/0 (socket says 172.21.15.89:54488) 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f76577fe700 1 -- 172.21.15.89:0/2805051827 learned_addr learned my addr 172.21.15.89:0/2805051827 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f76567fc700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f765806f050 0x7f7658121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f7656ffd700 1 -- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7658121f60 msgr2=0x7f76581cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f7656ffd700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7658121f60 0x7f76581cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f7656ffd700 1 -- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f765806f050 msgr2=0x7f7658121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.305+0000 7f7656ffd700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f765806f050 0x7f7658121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.306+0000 7f7656ffd700 1 -- 172.21.15.89:0/2805051827 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009550 con 0x7f765806e570 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.306+0000 7f76567fc700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f765806f050 0x7f7658121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:20:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.306+0000 7f7656ffd700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765806e570 0x7f76581214e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f764800c3e0 tx=0x7f764800c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:20:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.306+0000 7f763ffff700 1 -- 172.21.15.89:0/2805051827 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764800cea0 con 0x7f765806e570 2023-09-30T22:20:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.306+0000 7f763ffff700 1 -- 172.21.15.89:0/2805051827 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648005500 con 0x7f765806e570 2023-09-30T22:20:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.306+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7650009a70 con 0x7f765806e570 2023-09-30T22:20:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.306+0000 7f763ffff700 1 -- 172.21.15.89:0/2805051827 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480038c0 con 0x7f765806e570 2023-09-30T22:20:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.306+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7658122930 con 0x7f765806e570 2023-09-30T22:20:59.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.307+0000 7f763ffff700 1 -- 172.21.15.89:0/2805051827 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7648003a20 con 0x7f765806e570 2023-09-30T22:20:59.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.307+0000 7f763ffff700 1 -- 172.21.15.89:0/2805051827 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f764803e780 con 0x7f765806e570 2023-09-30T22:20:59.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.307+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658122360 con 0x7f765806e570 2023-09-30T22:20:59.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.312+0000 7f763ffff700 1 -- 172.21.15.89:0/2805051827 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7648017080 con 0x7f765806e570 2023-09-30T22:20:59.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.447+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f66c806c360 con 0x7f66c81316c0 2023-09-30T22:20:59.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.448+0000 7f66b6ffd700 1 -- 172.21.15.89:0/2142525891 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66c0018020 con 0x7f66c81316c0 2023-09-30T22:20:59.448 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:59.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c81316c0 msgr2=0x7f66c8071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c81316c0 0x7f66c8071d40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f66c000efc0 tx=0x7f66c000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 shutdown_connections 2023-09-30T22:20:59.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66c81316c0 0x7f66c8071d40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c806ae80 0x7f66c80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 --2- 172.21.15.89:0/2142525891 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66c809d9f0 0x7f66c8071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 >> 172.21.15.89:0/2142525891 conn(0x7f66c8066f50 msgr2=0x7f66c809baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:59.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 shutdown_connections 2023-09-30T22:20:59.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.450+0000 7f66cccab700 1 -- 172.21.15.89:0/2142525891 wait complete. 2023-09-30T22:20:59.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:20:59.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.468+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd6440a3410 con 0x7fd64409e410 2023-09-30T22:20:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.468+0000 7fd62bfff700 1 -- 172.21.15.89:0/2955828362 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd63c007330 con 0x7fd64409e410 2023-09-30T22:20:59.468 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409e410 msgr2=0x7fd6441a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409e410 0x7fd6441a8af0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009d90 tx=0x7fd63c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 shutdown_connections 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd64409e410 0x7fd6441a8af0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd64409d6f0 0x7fd6441a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 --2- 172.21.15.89:0/2955828362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd64409e000 0x7fd6441a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 >> 172.21.15.89:0/2955828362 conn(0x7fd644066f50 msgr2=0x7fd64406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 shutdown_connections 2023-09-30T22:20:59.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.471+0000 7fd649f4a700 1 -- 172.21.15.89:0/2955828362 wait complete. 2023-09-30T22:20:59.482 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:20:59.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.529+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe91c09c920 con 0x7fe91c12f8b0 2023-09-30T22:20:59.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.529+0000 7fe90affd700 1 -- 172.21.15.89:0/2086786468 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe91c09c920 con 0x7fe91c12f8b0 2023-09-30T22:20:59.529 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:20:59.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe91c12f8b0 msgr2=0x7fe91c09c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe91c12f8b0 0x7fe91c09c380 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe91000c450 tx=0x7fe91000c760 comp rx=0 tx=0).stop 2023-09-30T22:20:59.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 shutdown_connections 2023-09-30T22:20:59.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91c130430 0x7fe91c1c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe91c12cf60 0x7fe91c09be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 --2- 172.21.15.89:0/2086786468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe91c12f8b0 0x7fe91c09c380 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 >> 172.21.15.89:0/2086786468 conn(0x7fe91c0a2aa0 msgr2=0x7fe91c1b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:59.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 shutdown_connections 2023-09-30T22:20:59.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.531+0000 7fe920deb700 1 -- 172.21.15.89:0/2086786468 wait complete. 2023-09-30T22:20:59.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:20:59.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.562+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7658074ea0 con 0x7f765806e570 2023-09-30T22:20:59.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.562+0000 7f763ffff700 1 -- 172.21.15.89:0/2805051827 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7648018cd0 con 0x7f765806e570 2023-09-30T22:20:59.562 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:20:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765806e570 msgr2=0x7f76581214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:20:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765806e570 0x7f76581214e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f764800c3e0 tx=0x7f764800c6f0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 shutdown_connections 2023-09-30T22:20:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7658121f60 0x7f76581cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f765806e570 0x7f76581214e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 --2- 172.21.15.89:0/2805051827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f765806f050 0x7f7658121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:20:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 >> 172.21.15.89:0/2805051827 conn(0x7f76580a2aa0 msgr2=0x7f76580a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:20:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 shutdown_connections 2023-09-30T22:20:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:20:59.564+0000 7f765d9c9700 1 -- 172.21.15.89:0/2805051827 wait complete. 2023-09-30T22:20:59.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:00.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:00.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:00.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:00.577 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:00.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.612+0000 7f7a01dc5700 1 Processor -- start 2023-09-30T22:21:00.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.612+0000 7f7a01dc5700 1 -- start start 2023-09-30T22:21:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f7a01dc5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc09d630 0x7f79fc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f7a01dc5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79fc138d80 0x7f79fc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f7a01dc5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc0a2a70 con 0x7f79fc0a0a30 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f7a01dc5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0a01d0 con 0x7f79fc138d80 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f7a01dc5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc09e2b0 con 0x7f79fc09d630 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f79faffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc09d630 0x7f79fc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f79faffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc09d630 0x7f79fc0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58164/0 (socket says 172.21.15.89:58164) 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f79faffd700 1 -- 172.21.15.89:0/1560250512 learned_addr learned my addr 172.21.15.89:0/1560250512 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f79fbfff700 1 --2- 172.21.15.89:0/1560250512 >> v2:172.21.15.89:3300/0 conn(0x7f79fc138d80 0x7f79fc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f79faffd700 1 -- 172.21.15.89:0/1560250512 >> v2:172.21.15.89:3300/0 conn(0x7f79fc138d80 msgr2=0x7f79fc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f79faffd700 1 --2- 172.21.15.89:0/1560250512 >> v2:172.21.15.89:3300/0 conn(0x7f79fc138d80 0x7f79fc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f79faffd700 1 -- 172.21.15.89:0/1560250512 >> v1:172.21.15.89:6789/0 conn(0x7f79fc0a0a30 legacy=0x7f79fc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.613+0000 7f79faffd700 1 -- 172.21.15.89:0/1560250512 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc09dc50 con 0x7f79fc09d630 2023-09-30T22:21:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.614+0000 7f79faffd700 1 --2- 172.21.15.89:0/1560250512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc09d630 0x7f79fc0a1510 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f79f00098f0 tx=0x7f79f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad8b9012954afe8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.614+0000 7f79fa7fc700 1 -- 172.21.15.89:0/1560250512 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0011830 con 0x7f79fc09d630 2023-09-30T22:21:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.614+0000 7f79fa7fc700 1 -- 172.21.15.89:0/1560250512 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0011990 con 0x7f79fc09d630 2023-09-30T22:21:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.614+0000 7f79fa7fc700 1 -- 172.21.15.89:0/1560250512 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0011d00 con 0x7f79fc09d630 2023-09-30T22:21:00.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.614+0000 7f7a01dc5700 1 -- 172.21.15.89:0/1560250512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc09d630 msgr2=0x7f79fc0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.614+0000 7f7a01dc5700 1 --2- 172.21.15.89:0/1560250512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc09d630 0x7f79fc0a1510 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f79f00098f0 tx=0x7f79f0009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.615+0000 7f7a01dc5700 1 -- 172.21.15.89:0/1560250512 shutdown_connections 2023-09-30T22:21:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.615+0000 7f7a01dc5700 1 --2- 172.21.15.89:0/1560250512 >> v2:172.21.15.89:3300/0 conn(0x7f79fc138d80 0x7f79fc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.615+0000 7f7a01dc5700 1 --2- 172.21.15.89:0/1560250512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc09d630 0x7f79fc0a1510 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.615+0000 7f7a01dc5700 1 -- 172.21.15.89:0/1560250512 >> 172.21.15.89:0/1560250512 conn(0x7f79fc0a3050 msgr2=0x7f79fc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.615+0000 7f7a01dc5700 1 -- 172.21.15.89:0/1560250512 shutdown_connections 2023-09-30T22:21:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.615+0000 7f7a01dc5700 1 -- 172.21.15.89:0/1560250512 wait complete. 2023-09-30T22:21:00.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.616+0000 7f7a01dc5700 1 Processor -- start 2023-09-30T22:21:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.616+0000 7f7a01dc5700 1 -- start start 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f7a01dc5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79fc09d630 0x7f79fc1337a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f7a01dc5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79fc0a0a30 0x7f79fc133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f7a01dc5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc138d80 0x7f79fc134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f7a01dc5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc13af10 con 0x7f79fc09d630 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f7a01dc5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc1358f0 con 0x7f79fc138d80 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f7a01dc5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0a0410 con 0x7f79fc0a0a30 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79fc09d630 0x7f79fc1337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79fc09d630 0x7f79fc1337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34524/0 (socket says 172.21.15.89:34524) 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 -- 172.21.15.89:0/2197280571 learned_addr learned my addr 172.21.15.89:0/2197280571 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fbfff700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc138d80 0x7f79fc134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 -- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79fc0a0a30 msgr2=0x7f79fc133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79fc0a0a30 0x7f79fc133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 -- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc138d80 msgr2=0x7f79fc134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc138d80 0x7f79fc134220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 -- 172.21.15.89:0/2197280571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009550 con 0x7f79fc09d630 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fbfff700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc138d80 0x7f79fc134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.617+0000 7f79fb7fe700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79fc09d630 0x7f79fc1337a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c4e0 tx=0x7f79ec00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.618+0000 7f79e3fff700 1 -- 172.21.15.89:0/2197280571 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0052d0 con 0x7f79fc09d630 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.618+0000 7f79e3fff700 1 -- 172.21.15.89:0/2197280571 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec005430 con 0x7f79fc09d630 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.618+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc136630 con 0x7f79fc09d630 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.618+0000 7f79e3fff700 1 -- 172.21.15.89:0/2197280571 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0038c0 con 0x7f79fc09d630 2023-09-30T22:21:00.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.618+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc136fa0 con 0x7f79fc09d630 2023-09-30T22:21:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.619+0000 7f79e3fff700 1 -- 172.21.15.89:0/2197280571 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79ec003a20 con 0x7f79fc09d630 2023-09-30T22:21:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.619+0000 7f79e3fff700 1 -- 172.21.15.89:0/2197280571 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f79ec040c30 con 0x7f79fc09d630 2023-09-30T22:21:00.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.619+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc0052e0 con 0x7f79fc09d630 2023-09-30T22:21:00.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.625+0000 7f79e3fff700 1 -- 172.21.15.89:0/2197280571 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79ec017020 con 0x7f79fc09d630 2023-09-30T22:21:00.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.652+0000 7f710edc5700 1 Processor -- start 2023-09-30T22:21:00.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.652+0000 7f710edc5700 1 -- start start 2023-09-30T22:21:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f710edc5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f710809d9f0 0x7f7108136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f710edc5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f7108138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f710edc5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710809e130 con 0x7f71081316c0 2023-09-30T22:21:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f710edc5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71081321a0 con 0x7f710809d9f0 2023-09-30T22:21:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f710edc5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71081312b0 con 0x7f7108136a70 2023-09-30T22:21:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f710d362700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f7108138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f710cb61700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f71081316c0 0x7f7108131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51970/0 (socket says 172.21.15.89:51970) 2023-09-30T22:21:00.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f710cb61700 1 -- 172.21.15.89:0/4109072858 learned_addr learned my addr 172.21.15.89:0/4109072858 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:00.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.653+0000 7f7107fff700 1 --2- 172.21.15.89:0/4109072858 >> v2:172.21.15.89:3300/0 conn(0x7f710809d9f0 0x7f7108136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.654+0000 7f710d362700 1 -- 172.21.15.89:0/4109072858 >> v2:172.21.15.89:3300/0 conn(0x7f710809d9f0 msgr2=0x7f7108136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:00.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.654+0000 7f710d362700 1 --2- 172.21.15.89:0/4109072858 >> v2:172.21.15.89:3300/0 conn(0x7f710809d9f0 0x7f7108136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.654+0000 7f710d362700 1 -- 172.21.15.89:0/4109072858 >> v1:172.21.15.89:6789/0 conn(0x7f71081316c0 legacy=0x7f7108131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.654+0000 7f710d362700 1 -- 172.21.15.89:0/4109072858 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710809f6e0 con 0x7f7108136a70 2023-09-30T22:21:00.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.654+0000 7f710d362700 1 --2- 172.21.15.89:0/4109072858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f7108138e50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7100009dc0 tx=0x7f710000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=901f633a38dbbc24 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:00.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.654+0000 7f71077fe700 1 -- 172.21.15.89:0/4109072858 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000e070 con 0x7f7108136a70 2023-09-30T22:21:00.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.654+0000 7f71077fe700 1 -- 172.21.15.89:0/4109072858 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710000bae0 con 0x7f7108136a70 2023-09-30T22:21:00.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.654+0000 7f71077fe700 1 -- 172.21.15.89:0/4109072858 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000be90 con 0x7f7108136a70 2023-09-30T22:21:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.655+0000 7f710edc5700 1 -- 172.21.15.89:0/4109072858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 msgr2=0x7f7108138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.655+0000 7f710edc5700 1 --2- 172.21.15.89:0/4109072858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f7108138e50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7100009dc0 tx=0x7f710000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.655+0000 7f710edc5700 1 -- 172.21.15.89:0/4109072858 shutdown_connections 2023-09-30T22:21:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.655+0000 7f710edc5700 1 --2- 172.21.15.89:0/4109072858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f7108138e50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.655+0000 7f710edc5700 1 --2- 172.21.15.89:0/4109072858 >> v2:172.21.15.89:3300/0 conn(0x7f710809d9f0 0x7f7108136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.655+0000 7f710edc5700 1 -- 172.21.15.89:0/4109072858 >> 172.21.15.89:0/4109072858 conn(0x7f7108066f50 msgr2=0x7f710809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.655+0000 7f710edc5700 1 -- 172.21.15.89:0/4109072858 shutdown_connections 2023-09-30T22:21:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.655+0000 7f710edc5700 1 -- 172.21.15.89:0/4109072858 wait complete. 2023-09-30T22:21:00.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.656+0000 7f710edc5700 1 Processor -- start 2023-09-30T22:21:00.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.656+0000 7f710edc5700 1 -- start start 2023-09-30T22:21:00.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.657+0000 7f710edc5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710809d9f0 0x7f710812abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.657+0000 7f710edc5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71081316c0 0x7f710812b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.657+0000 7f710edc5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f71081341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.657+0000 7f710edc5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710809c2e0 con 0x7f710809d9f0 2023-09-30T22:21:00.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.657+0000 7f710edc5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710812b900 con 0x7f71081316c0 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.657+0000 7f710edc5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f710809b190 con 0x7f7108136a70 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f7107fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71081316c0 0x7f710812b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710d362700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f71081341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710cb61700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710809d9f0 0x7f710812abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f7107fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71081316c0 0x7f710812b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58174/0 (socket says 172.21.15.89:58174) 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f7107fff700 1 -- 172.21.15.89:0/983281625 learned_addr learned my addr 172.21.15.89:0/983281625 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710d362700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f71081341d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39542/0 (socket says 172.21.15.89:39542) 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710d362700 1 -- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71081316c0 msgr2=0x7f710812b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710d362700 1 --2- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71081316c0 0x7f710812b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710d362700 1 -- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710809d9f0 msgr2=0x7f710812abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710d362700 1 --2- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710809d9f0 0x7f710812abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710d362700 1 -- 172.21.15.89:0/983281625 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100009a70 con 0x7f7108136a70 2023-09-30T22:21:00.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710cb61700 1 --2- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710809d9f0 0x7f710812abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:00.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f710d362700 1 --2- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f71081341d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7100016000 tx=0x7f7100004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:00.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f71057fa700 1 -- 172.21.15.89:0/983281625 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71000045d0 con 0x7f7108136a70 2023-09-30T22:21:00.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f71057fa700 1 -- 172.21.15.89:0/983281625 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7100020070 con 0x7f7108136a70 2023-09-30T22:21:00.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.658+0000 7f71057fa700 1 -- 172.21.15.89:0/983281625 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100027480 con 0x7f7108136a70 2023-09-30T22:21:00.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.659+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710809b840 con 0x7f7108136a70 2023-09-30T22:21:00.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.659+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710809ad90 con 0x7f7108136a70 2023-09-30T22:21:00.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.660+0000 7f71057fa700 1 -- 172.21.15.89:0/983281625 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71000275e0 con 0x7f7108136a70 2023-09-30T22:21:00.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.660+0000 7f71057fa700 1 -- 172.21.15.89:0/983281625 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7100022070 con 0x7f7108136a70 2023-09-30T22:21:00.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.661+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710809beb0 con 0x7f7108136a70 2023-09-30T22:21:00.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.665+0000 7f71057fa700 1 -- 172.21.15.89:0/983281625 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7100027890 con 0x7f7108136a70 2023-09-30T22:21:00.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.711+0000 7f2c4ceb1700 1 Processor -- start 2023-09-30T22:21:00.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.711+0000 7f2c4ceb1700 1 -- start start 2023-09-30T22:21:00.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4ceb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c4809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4ceb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4ceb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2c4809e410 0x7f2c48133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4ceb1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4806b460 con 0x7f2c4809e410 2023-09-30T22:21:00.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4ceb1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4809b120 con 0x7f2c4809e000 2023-09-30T22:21:00.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4ceb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4809a7e0 con 0x7f2c4809d6f0 2023-09-30T22:21:00.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c4809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c4809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39554/0 (socket says 172.21.15.89:39554) 2023-09-30T22:21:00.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c4659c700 1 -- 172.21.15.89:0/365073536 learned_addr learned my addr 172.21.15.89:0/365073536 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:00.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.712+0000 7f2c45d9b700 1 --2- 172.21.15.89:0/365073536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c46d9d700 1 --2- 172.21.15.89:0/365073536 >> v2:172.21.15.89:3300/0 conn(0x7f2c4809e410 0x7f2c48133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4659c700 1 -- 172.21.15.89:0/365073536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 msgr2=0x7f2c481339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4659c700 1 --2- 172.21.15.89:0/365073536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4659c700 1 -- 172.21.15.89:0/365073536 >> v2:172.21.15.89:3300/0 conn(0x7f2c4809e410 msgr2=0x7f2c48133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4659c700 1 --2- 172.21.15.89:0/365073536 >> v2:172.21.15.89:3300/0 conn(0x7f2c4809e410 0x7f2c48133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4659c700 1 -- 172.21.15.89:0/365073536 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4809a500 con 0x7f2c4809d6f0 2023-09-30T22:21:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c46d9d700 1 --2- 172.21.15.89:0/365073536 >> v2:172.21.15.89:3300/0 conn(0x7f2c4809e410 0x7f2c48133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4659c700 1 --2- 172.21.15.89:0/365073536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c4809dac0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009920 tx=0x7f2c38009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d85446c5a89fedb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4559a700 1 -- 172.21.15.89:0/365073536 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38011830 con 0x7f2c4809d6f0 2023-09-30T22:21:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4559a700 1 -- 172.21.15.89:0/365073536 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c38011990 con 0x7f2c4809d6f0 2023-09-30T22:21:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.713+0000 7f2c4559a700 1 -- 172.21.15.89:0/365073536 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38011d20 con 0x7f2c4809d6f0 2023-09-30T22:21:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/365073536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 msgr2=0x7f2c4809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 --2- 172.21.15.89:0/365073536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c4809dac0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009920 tx=0x7f2c38009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/365073536 shutdown_connections 2023-09-30T22:21:00.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 --2- 172.21.15.89:0/365073536 >> v2:172.21.15.89:3300/0 conn(0x7f2c4809e410 0x7f2c48133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 --2- 172.21.15.89:0/365073536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c4809dac0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 --2- 172.21.15.89:0/365073536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/365073536 >> 172.21.15.89:0/365073536 conn(0x7f2c48066f50 msgr2=0x7f2c4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:00.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/365073536 shutdown_connections 2023-09-30T22:21:00.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.714+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/365073536 wait complete. 2023-09-30T22:21:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.715+0000 7f2c4ceb1700 1 Processor -- start 2023-09-30T22:21:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4ceb1700 1 -- start start 2023-09-30T22:21:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4ceb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c481cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4ceb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4ceb1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4809e410 0x7f2c481cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4ceb1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481bd8b0 con 0x7f2c4809e410 2023-09-30T22:21:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4ceb1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481ac740 con 0x7f2c4809e000 2023-09-30T22:21:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4ceb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481bda60 con 0x7f2c4809d6f0 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c481cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c481cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39570/0 (socket says 172.21.15.89:39570) 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c4659c700 1 -- 172.21.15.89:0/463181204 learned_addr learned my addr 172.21.15.89:0/463181204 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.716+0000 7f2c45d9b700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c46d9d700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4809e410 0x7f2c481cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c4659c700 1 -- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 msgr2=0x7f2c481ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c4659c700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c4659c700 1 -- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4809e410 msgr2=0x7f2c481cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c4659c700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4809e410 0x7f2c481cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c4659c700 1 -- 172.21.15.89:0/463181204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c38009550 con 0x7f2c4809d6f0 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c45d9b700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c46d9d700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4809e410 0x7f2c481cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c4659c700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c481cc590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2c38000c00 tx=0x7f2c38004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c36ffd700 1 -- 172.21.15.89:0/463181204 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38004480 con 0x7f2c4809d6f0 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c36ffd700 1 -- 172.21.15.89:0/463181204 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3801f070 con 0x7f2c4809d6f0 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c480a30c0 con 0x7f2c4809d6f0 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c36ffd700 1 -- 172.21.15.89:0/463181204 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c380038e0 con 0x7f2c4809d6f0 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.717+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c481ab300 con 0x7f2c4809d6f0 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.718+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c0052e0 con 0x7f2c4809d6f0 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.718+0000 7f2c36ffd700 1 -- 172.21.15.89:0/463181204 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c38021030 con 0x7f2c4809d6f0 2023-09-30T22:21:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.719+0000 7f2c36ffd700 1 -- 172.21.15.89:0/463181204 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2c38040a60 con 0x7f2c4809d6f0 2023-09-30T22:21:00.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.723+0000 7f2c36ffd700 1 -- 172.21.15.89:0/463181204 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c380239c0 con 0x7f2c4809d6f0 2023-09-30T22:21:00.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.727+0000 7f9d5bbc1700 1 Processor -- start 2023-09-30T22:21:00.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5bbc1700 1 -- start start 2023-09-30T22:21:00.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5bbc1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5bbc1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d5409a580 0x7f9d54139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5bbc1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d541313c0 con 0x7f9d54099a40 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5bbc1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5409b580 con 0x7f9d5409a580 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5bbc1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5406e8c0 con 0x7f9d54131810 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5995d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5995d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39584/0 (socket says 172.21.15.89:39584) 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.728+0000 7f9d5995d700 1 -- 172.21.15.89:0/274345624 learned_addr learned my addr 172.21.15.89:0/274345624 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.729+0000 7f9d5995d700 1 -- 172.21.15.89:0/274345624 >> v2:172.21.15.89:3300/0 conn(0x7f9d5409a580 msgr2=0x7f9d54139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.729+0000 7f9d5995d700 1 --2- 172.21.15.89:0/274345624 >> v2:172.21.15.89:3300/0 conn(0x7f9d5409a580 0x7f9d54139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.729+0000 7f9d5995d700 1 -- 172.21.15.89:0/274345624 >> v1:172.21.15.89:6789/0 conn(0x7f9d54099a40 legacy=0x7f9d54099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.729+0000 7f9d5995d700 1 -- 172.21.15.89:0/274345624 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d54071d20 con 0x7f9d54131810 2023-09-30T22:21:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.729+0000 7f9d5995d700 1 --2- 172.21.15.89:0/274345624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54131be0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009920 tx=0x7f9d50009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb60f8b3494f9260 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:00.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.729+0000 7f9d5895b700 1 -- 172.21.15.89:0/274345624 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50004230 con 0x7f9d54131810 2023-09-30T22:21:00.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.729+0000 7f9d5895b700 1 -- 172.21.15.89:0/274345624 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50004390 con 0x7f9d54131810 2023-09-30T22:21:00.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.729+0000 7f9d5895b700 1 -- 172.21.15.89:0/274345624 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50009e70 con 0x7f9d54131810 2023-09-30T22:21:00.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.730+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/274345624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 msgr2=0x7f9d54131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.730+0000 7f9d5bbc1700 1 --2- 172.21.15.89:0/274345624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54131be0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009920 tx=0x7f9d50009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:00.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.730+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/274345624 shutdown_connections 2023-09-30T22:21:00.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.730+0000 7f9d5bbc1700 1 --2- 172.21.15.89:0/274345624 >> v2:172.21.15.89:3300/0 conn(0x7f9d5409a580 0x7f9d54139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.730+0000 7f9d5bbc1700 1 --2- 172.21.15.89:0/274345624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54131be0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.730+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/274345624 >> 172.21.15.89:0/274345624 conn(0x7f9d540a2aa0 msgr2=0x7f9d540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:00.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.730+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/274345624 shutdown_connections 2023-09-30T22:21:00.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.730+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/274345624 wait complete. 2023-09-30T22:21:00.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5bbc1700 1 Processor -- start 2023-09-30T22:21:00.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5bbc1700 1 -- start start 2023-09-30T22:21:00.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5bbc1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d54099a40 0x7f9d54121920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5bbc1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d5409a580 0x7f9d541201e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5bbc1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54120720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5bbc1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5411df30 con 0x7f9d54099a40 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5bbc1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5411f600 con 0x7f9d5409a580 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5bbc1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54121e60 con 0x7f9d54131810 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5915c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d5409a580 0x7f9d541201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5915c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d5409a580 0x7f9d541201e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58192/0 (socket says 172.21.15.89:58192) 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.731+0000 7f9d5915c700 1 -- 172.21.15.89:0/2274752590 learned_addr learned my addr 172.21.15.89:0/2274752590 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d5915c700 1 -- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 msgr2=0x7f9d54120720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d5915c700 1 --2- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54120720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d5915c700 1 -- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d54099a40 msgr2=0x7f9d54121920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d5915c700 1 --2- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d54099a40 0x7f9d54121920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d5915c700 1 -- 172.21.15.89:0/2274752590 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d50009550 con 0x7f9d5409a580 2023-09-30T22:21:00.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d5915c700 1 --2- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d5409a580 0x7f9d541201e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c4a0 tx=0x7f9d4400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:00.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d4a7fc700 1 -- 172.21.15.89:0/2274752590 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d440054e0 con 0x7f9d5409a580 2023-09-30T22:21:00.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d4a7fc700 1 -- 172.21.15.89:0/2274752590 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d44003680 con 0x7f9d5409a580 2023-09-30T22:21:00.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d4a7fc700 1 -- 172.21.15.89:0/2274752590 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d440039f0 con 0x7f9d5409a580 2023-09-30T22:21:00.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54072250 con 0x7f9d5409a580 2023-09-30T22:21:00.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.732+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5412eaf0 con 0x7f9d5409a580 2023-09-30T22:21:00.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.733+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5404ea50 con 0x7f9d5409a580 2023-09-30T22:21:00.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.734+0000 7f9d4a7fc700 1 -- 172.21.15.89:0/2274752590 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d44003b50 con 0x7f9d5409a580 2023-09-30T22:21:00.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.734+0000 7f9d4a7fc700 1 -- 172.21.15.89:0/2274752590 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d4403e890 con 0x7f9d5409a580 2023-09-30T22:21:00.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.738+0000 7f9d4a7fc700 1 -- 172.21.15.89:0/2274752590 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d44003e00 con 0x7f9d5409a580 2023-09-30T22:21:00.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.827+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f79dc005590 con 0x7f79fc09d630 2023-09-30T22:21:00.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.827+0000 7f79e3fff700 1 -- 172.21.15.89:0/2197280571 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79ec044060 con 0x7f79fc09d630 2023-09-30T22:21:00.827 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.829+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79fc09d630 msgr2=0x7f79fc1337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.829+0000 7f7a01dc5700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79fc09d630 0x7f79fc1337a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c4e0 tx=0x7f79ec00c7f0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.829+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 shutdown_connections 2023-09-30T22:21:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.829+0000 7f7a01dc5700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79fc138d80 0x7f79fc134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.829+0000 7f7a01dc5700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79fc09d630 0x7f79fc1337a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.829+0000 7f7a01dc5700 1 --2- 172.21.15.89:0/2197280571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79fc0a0a30 0x7f79fc133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.829+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 >> 172.21.15.89:0/2197280571 conn(0x7f79fc0a3050 msgr2=0x7f79fc1399e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:00.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.830+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 shutdown_connections 2023-09-30T22:21:00.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.830+0000 7f7a01dc5700 1 -- 172.21.15.89:0/2197280571 wait complete. 2023-09-30T22:21:00.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.903+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7108074ea0 con 0x7f7108136a70 2023-09-30T22:21:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.903+0000 7f71057fa700 1 -- 172.21.15.89:0/983281625 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7100036090 con 0x7f7108136a70 2023-09-30T22:21:00.903 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:00.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 msgr2=0x7f71081341d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 --2- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f71081341d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7100016000 tx=0x7f7100004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 shutdown_connections 2023-09-30T22:21:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 --2- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7108136a70 0x7f71081341d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 --2- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f710809d9f0 0x7f710812abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 --2- 172.21.15.89:0/983281625 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f71081316c0 0x7f710812b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 >> 172.21.15.89:0/983281625 conn(0x7f7108066f50 msgr2=0x7f71081385d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 shutdown_connections 2023-09-30T22:21:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.906+0000 7f710edc5700 1 -- 172.21.15.89:0/983281625 wait complete. 2023-09-30T22:21:00.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:00.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.932+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9d54074ea0 con 0x7f9d5409a580 2023-09-30T22:21:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.932+0000 7f9d4a7fc700 1 -- 172.21.15.89:0/2274752590 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d44017070 con 0x7f9d5409a580 2023-09-30T22:21:00.932 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.935+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d5409a580 msgr2=0x7f9d541201e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.935+0000 7f9d5bbc1700 1 --2- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d5409a580 0x7f9d541201e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c4a0 tx=0x7f9d4400c860 comp rx=0 tx=0).stop 2023-09-30T22:21:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.935+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 shutdown_connections 2023-09-30T22:21:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.935+0000 7f9d5bbc1700 1 --2- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d54131810 0x7f9d54120720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.935+0000 7f9d5bbc1700 1 --2- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d54099a40 0x7f9d54121920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.935+0000 7f9d5bbc1700 1 --2- 172.21.15.89:0/2274752590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d5409a580 0x7f9d541201e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.935+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 >> 172.21.15.89:0/2274752590 conn(0x7f9d540a2aa0 msgr2=0x7f9d540a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.935+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 shutdown_connections 2023-09-30T22:21:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.936+0000 7f9d5bbc1700 1 -- 172.21.15.89:0/2274752590 wait complete. 2023-09-30T22:21:00.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.963+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2c2c005590 con 0x7f2c4809d6f0 2023-09-30T22:21:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.963+0000 7f2c36ffd700 1 -- 172.21.15.89:0/463181204 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c38015070 con 0x7f2c4809d6f0 2023-09-30T22:21:00.963 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:00.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 msgr2=0x7f2c481cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:00.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c481cc590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2c38000c00 tx=0x7f2c38004750 comp rx=0 tx=0).stop 2023-09-30T22:21:00.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 shutdown_connections 2023-09-30T22:21:00.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4809e410 0x7f2c481cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2c4809d6f0 0x7f2c481cc590 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 --2- 172.21.15.89:0/463181204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2c4809e000 0x7f2c481ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:00.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 >> 172.21.15.89:0/463181204 conn(0x7f2c48066f50 msgr2=0x7f2c481c0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:00.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 shutdown_connections 2023-09-30T22:21:00.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:00.965+0000 7f2c4ceb1700 1 -- 172.21.15.89:0/463181204 wait complete. 2023-09-30T22:21:00.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:01.843 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:01.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:01.952 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:01.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76d171d700 1 Processor -- start 2023-09-30T22:21:01.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76d171d700 1 -- start start 2023-09-30T22:21:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76d171d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76d171d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc135e60 0x7f76cc138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:01.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76d171d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76cc0716a0 con 0x7f76cc130540 2023-09-30T22:21:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76d171d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc071290 con 0x7f76cc135e60 2023-09-30T22:21:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76d171d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc130340 con 0x7f76cc133540 2023-09-30T22:21:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76ca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76ca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39616/0 (socket says 172.21.15.89:39616) 2023-09-30T22:21:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76ca7fc700 1 -- 172.21.15.89:0/1279734756 learned_addr learned my addr 172.21.15.89:0/1279734756 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.975+0000 7f76cb7fe700 1 --2- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc135e60 0x7f76cc138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:01.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76ca7fc700 1 -- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc135e60 msgr2=0x7f76cc138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76ca7fc700 1 --2- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc135e60 0x7f76cc138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76ca7fc700 1 -- 172.21.15.89:0/1279734756 >> v1:172.21.15.89:6789/0 conn(0x7f76cc130540 legacy=0x7f76cc130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76ca7fc700 1 -- 172.21.15.89:0/1279734756 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc070b80 con 0x7f76cc133540 2023-09-30T22:21:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76cb7fe700 1 --2- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc135e60 0x7f76cc138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76ca7fc700 1 --2- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc135920 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009920 tx=0x7f76c0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3737d511882c477f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:01.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76c9ffb700 1 -- 172.21.15.89:0/1279734756 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0011830 con 0x7f76cc133540 2023-09-30T22:21:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76c9ffb700 1 -- 172.21.15.89:0/1279734756 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c0011990 con 0x7f76cc133540 2023-09-30T22:21:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.976+0000 7f76c9ffb700 1 -- 172.21.15.89:0/1279734756 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0011d20 con 0x7f76cc133540 2023-09-30T22:21:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.977+0000 7f76d171d700 1 -- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 msgr2=0x7f76cc135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.977+0000 7f76d171d700 1 --2- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc135920 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009920 tx=0x7f76c0009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.977+0000 7f76d171d700 1 -- 172.21.15.89:0/1279734756 shutdown_connections 2023-09-30T22:21:01.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.977+0000 7f76d171d700 1 --2- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc135e60 0x7f76cc138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:01.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.977+0000 7f76d171d700 1 --2- 172.21.15.89:0/1279734756 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc135920 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:01.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.977+0000 7f76d171d700 1 -- 172.21.15.89:0/1279734756 >> 172.21.15.89:0/1279734756 conn(0x7f76cc0a2aa0 msgr2=0x7f76cc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:01.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.977+0000 7f76d171d700 1 -- 172.21.15.89:0/1279734756 shutdown_connections 2023-09-30T22:21:01.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.977+0000 7f76d171d700 1 -- 172.21.15.89:0/1279734756 wait complete. 2023-09-30T22:21:01.982 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:01.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.978+0000 7f76d171d700 1 Processor -- start 2023-09-30T22:21:01.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.978+0000 7f76d171d700 1 -- start start 2023-09-30T22:21:01.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76d171d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc130540 0x7f76cc09a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76d171d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc09aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76d171d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76cc135e60 0x7f76cc09b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76d171d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc0718e0 con 0x7f76cc135e60 2023-09-30T22:21:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76d171d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc1391c0 con 0x7f76cc130540 2023-09-30T22:21:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76d171d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc09c510 con 0x7f76cc133540 2023-09-30T22:21:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76ca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc09aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76caffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc130540 0x7f76cc09a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76ca7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc09aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39624/0 (socket says 172.21.15.89:39624) 2023-09-30T22:21:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76ca7fc700 1 -- 172.21.15.89:0/2967886409 learned_addr learned my addr 172.21.15.89:0/2967886409 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76cb7fe700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76cc135e60 0x7f76cc09b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76caffd700 1 -- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 msgr2=0x7f76cc09aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76caffd700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc09aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76caffd700 1 -- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76cc135e60 msgr2=0x7f76cc09b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76caffd700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76cc135e60 0x7f76cc09b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76caffd700 1 -- 172.21.15.89:0/2967886409 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c4009a70 con 0x7f76cc130540 2023-09-30T22:21:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76ca7fc700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc09aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.979+0000 7f76cb7fe700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76cc135e60 0x7f76cc09b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.980+0000 7f76caffd700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc130540 0x7f76cc09a590 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000f60 tx=0x7f76bc00c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.980+0000 7f76b37fe700 1 -- 172.21.15.89:0/2967886409 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0052d0 con 0x7f76cc130540 2023-09-30T22:21:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.980+0000 7f76b37fe700 1 -- 172.21.15.89:0/2967886409 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc005430 con 0x7f76cc130540 2023-09-30T22:21:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.980+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c0009550 con 0x7f76cc130540 2023-09-30T22:21:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.980+0000 7f76b37fe700 1 -- 172.21.15.89:0/2967886409 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0038a0 con 0x7f76cc130540 2023-09-30T22:21:01.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.980+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc09c920 con 0x7f76cc130540 2023-09-30T22:21:01.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.981+0000 7f76b37fe700 1 -- 172.21.15.89:0/2967886409 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76bc003a00 con 0x7f76cc130540 2023-09-30T22:21:01.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.981+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76cc09db00 con 0x7f76cc130540 2023-09-30T22:21:01.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.981+0000 7f76b37fe700 1 -- 172.21.15.89:0/2967886409 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f76bc03e920 con 0x7f76cc130540 2023-09-30T22:21:01.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:01.986+0000 7f76b37fe700 1 -- 172.21.15.89:0/2967886409 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76bc019d20 con 0x7f76cc130540 2023-09-30T22:21:02.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.066+0000 7f3584497700 1 Processor -- start 2023-09-30T22:21:02.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.066+0000 7f3584497700 1 -- start start 2023-09-30T22:21:02.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3584497700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f357c0a0a30 0x7f357c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3584497700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c09d630 0x7f357c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3584497700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c133930 0x7f357c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3584497700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357c0a2a70 con 0x7f357c0a0a30 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3584497700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357c0a01d0 con 0x7f357c09d630 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3584497700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f357c09e2b0 con 0x7f357c133930 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582233700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f357c0a0a30 0x7f357c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582233700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f357c0a0a30 0x7f357c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34610/0 (socket says 172.21.15.89:34610) 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582233700 1 -- 172.21.15.89:0/3728982081 learned_addr learned my addr 172.21.15.89:0/3728982081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582a34700 1 --2- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c133930 0x7f357c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3581a32700 1 --2- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c09d630 0x7f357c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582a34700 1 -- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c09d630 msgr2=0x7f357c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582a34700 1 --2- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c09d630 0x7f357c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582a34700 1 -- 172.21.15.89:0/3728982081 >> v2:172.21.15.89:3300/0 conn(0x7f357c0a0a30 msgr2=0x7f357c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:02.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582a34700 1 --2- 172.21.15.89:0/3728982081 >> v2:172.21.15.89:3300/0 conn(0x7f357c0a0a30 0x7f357c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582a34700 1 -- 172.21.15.89:0/3728982081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c09dc50 con 0x7f357c133930 2023-09-30T22:21:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3582233700 1 --2- 172.21.15.89:0/3728982081 >> v2:172.21.15.89:3300/0 conn(0x7f357c0a0a30 0x7f357c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.067+0000 7f3581a32700 1 --2- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c09d630 0x7f357c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.068+0000 7f3582a34700 1 --2- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c133930 0x7f357c133db0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3574009d90 tx=0x7f357400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62291945e7d18b85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.068+0000 7f3581231700 1 -- 172.21.15.89:0/3728982081 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400e070 con 0x7f357c133930 2023-09-30T22:21:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.068+0000 7f3581231700 1 -- 172.21.15.89:0/3728982081 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357400bb20 con 0x7f357c133930 2023-09-30T22:21:02.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.068+0000 7f3581231700 1 -- 172.21.15.89:0/3728982081 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574012430 con 0x7f357c133930 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 -- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c133930 msgr2=0x7f357c133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 --2- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c133930 0x7f357c133db0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3574009d90 tx=0x7f357400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 -- 172.21.15.89:0/3728982081 shutdown_connections 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 --2- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c133930 0x7f357c133db0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 --2- 172.21.15.89:0/3728982081 >> v2:172.21.15.89:3300/0 conn(0x7f357c0a0a30 0x7f357c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 --2- 172.21.15.89:0/3728982081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c09d630 0x7f357c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 -- 172.21.15.89:0/3728982081 >> 172.21.15.89:0/3728982081 conn(0x7f357c0a3050 msgr2=0x7f357c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 -- 172.21.15.89:0/3728982081 shutdown_connections 2023-09-30T22:21:02.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.069+0000 7f3584497700 1 -- 172.21.15.89:0/3728982081 wait complete. 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.070+0000 7f3584497700 1 Processor -- start 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.070+0000 7f3584497700 1 -- start start 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3584497700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c09d630 0x7f357c11f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3584497700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f357c0a0a30 0x7f357c11fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3584497700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c133930 0x7f357c1c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3584497700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c122490 con 0x7f357c0a0a30 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3584497700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357c133500 con 0x7f357c133930 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3584497700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f357c120050 con 0x7f357c09d630 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582233700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c09d630 0x7f357c11f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582233700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c09d630 0x7f357c11f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39642/0 (socket says 172.21.15.89:39642) 2023-09-30T22:21:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582233700 1 -- 172.21.15.89:0/3615987386 learned_addr learned my addr 172.21.15.89:0/3615987386 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582a34700 1 --2- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c133930 0x7f357c1c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582233700 1 -- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c133930 msgr2=0x7f357c1c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582233700 1 --2- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c133930 0x7f357c1c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582233700 1 -- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f357c0a0a30 msgr2=0x7f357c11fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582233700 1 --2- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f357c0a0a30 0x7f357c11fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.071+0000 7f3582233700 1 -- 172.21.15.89:0/3615987386 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578009550 con 0x7f357c09d630 2023-09-30T22:21:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.072+0000 7f3582233700 1 --2- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c09d630 0x7f357c11f5d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f357800ec10 tx=0x7f3578009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.072+0000 7f3572ffd700 1 -- 172.21.15.89:0/3615987386 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357800c8e0 con 0x7f357c09d630 2023-09-30T22:21:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.072+0000 7f3572ffd700 1 -- 172.21.15.89:0/3615987386 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357800ca40 con 0x7f357c09d630 2023-09-30T22:21:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.072+0000 7f3572ffd700 1 -- 172.21.15.89:0/3615987386 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357800cdb0 con 0x7f357c09d630 2023-09-30T22:21:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.072+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3574009a70 con 0x7f357c09d630 2023-09-30T22:21:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.072+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c120780 con 0x7f357c09d630 2023-09-30T22:21:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.073+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357c0a4090 con 0x7f357c09d630 2023-09-30T22:21:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.073+0000 7f3572ffd700 1 -- 172.21.15.89:0/3615987386 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3578013470 con 0x7f357c09d630 2023-09-30T22:21:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.074+0000 7f3572ffd700 1 -- 172.21.15.89:0/3615987386 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f357803f840 con 0x7f357c09d630 2023-09-30T22:21:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.082+0000 7f3572ffd700 1 -- 172.21.15.89:0/3615987386 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35780232f0 con 0x7f357c09d630 2023-09-30T22:21:02.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.100+0000 7f7945483700 1 Processor -- start 2023-09-30T22:21:02.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.100+0000 7f7945483700 1 -- start start 2023-09-30T22:21:02.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.100+0000 7f7945483700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009d6f0 0x7f794009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.100+0000 7f7945483700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f794009e000 0x7f7940130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.100+0000 7f7945483700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794009f130 con 0x7f794009e410 2023-09-30T22:21:02.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.100+0000 7f7945483700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794009f520 con 0x7f794009e000 2023-09-30T22:21:02.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.100+0000 7f7945483700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794009e930 con 0x7f794009d6f0 2023-09-30T22:21:02.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.100+0000 7f793e7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f794009e000 0x7f7940130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793e7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f794009e000 0x7f7940130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34620/0 (socket says 172.21.15.89:34620) 2023-09-30T22:21:02.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793e7fc700 1 -- 172.21.15.89:0/310949063 learned_addr learned my addr 172.21.15.89:0/310949063 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:02.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793effd700 1 --2- 172.21.15.89:0/310949063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009d6f0 0x7f794009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793effd700 1 -- 172.21.15.89:0/310949063 >> v2:172.21.15.89:3300/0 conn(0x7f794009e000 msgr2=0x7f7940130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:02.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793effd700 1 --2- 172.21.15.89:0/310949063 >> v2:172.21.15.89:3300/0 conn(0x7f794009e000 0x7f7940130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793effd700 1 -- 172.21.15.89:0/310949063 >> v1:172.21.15.89:6789/0 conn(0x7f794009e410 legacy=0x7f7940130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793effd700 1 -- 172.21.15.89:0/310949063 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794009b2d0 con 0x7f794009d6f0 2023-09-30T22:21:02.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793e7fc700 1 --2- 172.21.15.89:0/310949063 >> v2:172.21.15.89:3300/0 conn(0x7f794009e000 0x7f7940130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793effd700 1 --2- 172.21.15.89:0/310949063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009d6f0 0x7f794009dac0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7930009840 tx=0x7f7930009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e06488cbcf1365a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793dffb700 1 -- 172.21.15.89:0/310949063 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930011830 con 0x7f794009d6f0 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793dffb700 1 -- 172.21.15.89:0/310949063 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7930011990 con 0x7f794009d6f0 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.101+0000 7f793dffb700 1 -- 172.21.15.89:0/310949063 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930011d20 con 0x7f794009d6f0 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.102+0000 7f7945483700 1 -- 172.21.15.89:0/310949063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009d6f0 msgr2=0x7f794009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.102+0000 7f7945483700 1 --2- 172.21.15.89:0/310949063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009d6f0 0x7f794009dac0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7930009840 tx=0x7f7930009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.102+0000 7f7945483700 1 -- 172.21.15.89:0/310949063 shutdown_connections 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.102+0000 7f7945483700 1 --2- 172.21.15.89:0/310949063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009d6f0 0x7f794009dac0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.102+0000 7f7945483700 1 --2- 172.21.15.89:0/310949063 >> v2:172.21.15.89:3300/0 conn(0x7f794009e000 0x7f7940130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.102+0000 7f7945483700 1 -- 172.21.15.89:0/310949063 >> 172.21.15.89:0/310949063 conn(0x7f7940066f50 msgr2=0x7f794009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.102+0000 7f7945483700 1 -- 172.21.15.89:0/310949063 shutdown_connections 2023-09-30T22:21:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.102+0000 7f7945483700 1 -- 172.21.15.89:0/310949063 wait complete. 2023-09-30T22:21:02.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.103+0000 7f7945483700 1 Processor -- start 2023-09-30T22:21:02.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.103+0000 7f7945483700 1 -- start start 2023-09-30T22:21:02.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f7945483700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f794009d6f0 0x7f794011e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f7945483700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009e000 0x7f794011e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f7945483700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794009e410 0x7f794011eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f7945483700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7940121b80 con 0x7f794009e410 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f7945483700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400a1d90 con 0x7f794009e000 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f7945483700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f794009c700 con 0x7f794009d6f0 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f793effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f794009d6f0 0x7f794011e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f793f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794009e410 0x7f794011eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f793effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f794009d6f0 0x7f794011e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39652/0 (socket says 172.21.15.89:39652) 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f793effd700 1 -- 172.21.15.89:0/1584081281 learned_addr learned my addr 172.21.15.89:0/1584081281 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.104+0000 7f793e7fc700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009e000 0x7f794011e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f793effd700 1 -- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009e000 msgr2=0x7f794011e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f793effd700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009e000 0x7f794011e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f793effd700 1 -- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794009e410 msgr2=0x7f794011eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f793effd700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794009e410 0x7f794011eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f793effd700 1 -- 172.21.15.89:0/1584081281 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793400b7f0 con 0x7f794009d6f0 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f793f7fe700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794009e410 0x7f794011eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f793effd700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f794009d6f0 0x7f794011e060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7930009fd0 tx=0x7f7930004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f79277fe700 1 -- 172.21.15.89:0/1584081281 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930004550 con 0x7f794009d6f0 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f79277fe700 1 -- 172.21.15.89:0/1584081281 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793001f070 con 0x7f794009d6f0 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f79277fe700 1 -- 172.21.15.89:0/1584081281 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79300038e0 con 0x7f794009d6f0 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79300094a0 con 0x7f794009d6f0 2023-09-30T22:21:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.105+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7940121310 con 0x7f794009d6f0 2023-09-30T22:21:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.106+0000 7f79277fe700 1 -- 172.21.15.89:0/1584081281 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7930003a40 con 0x7f794009d6f0 2023-09-30T22:21:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.106+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7940120280 con 0x7f794009d6f0 2023-09-30T22:21:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.106+0000 7f79277fe700 1 -- 172.21.15.89:0/1584081281 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7930021070 con 0x7f794009d6f0 2023-09-30T22:21:02.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.111+0000 7f79277fe700 1 -- 172.21.15.89:0/1584081281 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7940120280 con 0x7f794009d6f0 2023-09-30T22:21:02.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.125+0000 7f76bcd3f700 1 Processor -- start 2023-09-30T22:21:02.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.125+0000 7f76bcd3f700 1 -- start start 2023-09-30T22:21:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.126+0000 7f76bcd3f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f76b809d9f0 0x7f76b8136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.126+0000 7f76bcd3f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.126+0000 7f76bcd3f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b809e130 con 0x7f76b81316c0 2023-09-30T22:21:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.126+0000 7f76bcd3f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b81321a0 con 0x7f76b809d9f0 2023-09-30T22:21:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.126+0000 7f76bcd3f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81312b0 con 0x7f76b8136a70 2023-09-30T22:21:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.126+0000 7f76b5d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f76b809d9f0 0x7f76b8136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.126+0000 7f76b5d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f76b809d9f0 0x7f76b8136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34636/0 (socket says 172.21.15.89:34636) 2023-09-30T22:21:02.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.126+0000 7f76b5d9b700 1 -- 172.21.15.89:0/2855365857 learned_addr learned my addr 172.21.15.89:0/2855365857 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:02.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.127+0000 7f76b5d9b700 1 -- 172.21.15.89:0/2855365857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 msgr2=0x7f76b8138e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:21:02.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.127+0000 7f76b5d9b700 1 --2- 172.21.15.89:0/2855365857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8138e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.127+0000 7f76b5d9b700 1 -- 172.21.15.89:0/2855365857 >> v1:172.21.15.89:6789/0 conn(0x7f76b81316c0 legacy=0x7f76b8131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.127+0000 7f76b5d9b700 1 -- 172.21.15.89:0/2855365857 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b809f6e0 con 0x7f76b809d9f0 2023-09-30T22:21:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.127+0000 7f76b5d9b700 1 --2- 172.21.15.89:0/2855365857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b809d9f0 0x7f76b8136530 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0098f0 tx=0x7f76ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae0dfacf778eb7a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.127+0000 7f76b559a700 1 -- 172.21.15.89:0/2855365857 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00b8a0 con 0x7f76b809d9f0 2023-09-30T22:21:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.127+0000 7f76b559a700 1 -- 172.21.15.89:0/2855365857 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac00ba00 con 0x7f76b809d9f0 2023-09-30T22:21:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.127+0000 7f76b559a700 1 -- 172.21.15.89:0/2855365857 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00bd90 con 0x7f76b809d9f0 2023-09-30T22:21:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.128+0000 7f76bcd3f700 1 -- 172.21.15.89:0/2855365857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b809d9f0 msgr2=0x7f76b8136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.128+0000 7f76bcd3f700 1 --2- 172.21.15.89:0/2855365857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b809d9f0 0x7f76b8136530 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0098f0 tx=0x7f76ac009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.128+0000 7f76bcd3f700 1 -- 172.21.15.89:0/2855365857 shutdown_connections 2023-09-30T22:21:02.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.128+0000 7f76bcd3f700 1 --2- 172.21.15.89:0/2855365857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.128+0000 7f76bcd3f700 1 --2- 172.21.15.89:0/2855365857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b809d9f0 0x7f76b8136530 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.128+0000 7f76bcd3f700 1 -- 172.21.15.89:0/2855365857 >> 172.21.15.89:0/2855365857 conn(0x7f76b8066f50 msgr2=0x7f76b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.128+0000 7f76bcd3f700 1 -- 172.21.15.89:0/2855365857 shutdown_connections 2023-09-30T22:21:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.128+0000 7f76bcd3f700 1 -- 172.21.15.89:0/2855365857 wait complete. 2023-09-30T22:21:02.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.129+0000 7f76bcd3f700 1 Processor -- start 2023-09-30T22:21:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.129+0000 7f76bcd3f700 1 -- start start 2023-09-30T22:21:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76bcd3f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b809d9f0 0x7f76b81348f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76bcd3f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b81316c0 0x7f76b8134e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76bcd3f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8135370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76bcd3f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b8133480 con 0x7f76b81316c0 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76bcd3f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b8072540 con 0x7f76b809d9f0 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76bcd3f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76b8133770 con 0x7f76b8136a70 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76b6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8135370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76b6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8135370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39660/0 (socket says 172.21.15.89:39660) 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76b6d9d700 1 -- 172.21.15.89:0/1829594925 learned_addr learned my addr 172.21.15.89:0/1829594925 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.130+0000 7f76b659c700 1 --2- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b809d9f0 0x7f76b81348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76b6d9d700 1 -- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b809d9f0 msgr2=0x7f76b81348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76b6d9d700 1 --2- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b809d9f0 0x7f76b81348f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76b6d9d700 1 -- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b81316c0 msgr2=0x7f76b8134e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76b6d9d700 1 --2- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b81316c0 0x7f76b8134e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76b6d9d700 1 -- 172.21.15.89:0/1829594925 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac009550 con 0x7f76b8136a70 2023-09-30T22:21:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76b6d9d700 1 --2- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8135370 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b9e0 tx=0x7f76b000bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:02.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76a6ffd700 1 -- 172.21.15.89:0/1829594925 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b000c930 con 0x7f76b8136a70 2023-09-30T22:21:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76a6ffd700 1 -- 172.21.15.89:0/1829594925 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b000cad0 con 0x7f76b8136a70 2023-09-30T22:21:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76a6ffd700 1 -- 172.21.15.89:0/1829594925 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b000ce80 con 0x7f76b8136a70 2023-09-30T22:21:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b8133a30 con 0x7f76b8136a70 2023-09-30T22:21:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.131+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b81358b0 con 0x7f76b8136a70 2023-09-30T22:21:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.132+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b8074ea0 con 0x7f76b8136a70 2023-09-30T22:21:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.132+0000 7f76a6ffd700 1 -- 172.21.15.89:0/1829594925 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76b0014440 con 0x7f76b8136a70 2023-09-30T22:21:02.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.132+0000 7f76a6ffd700 1 -- 172.21.15.89:0/1829594925 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f76b0004890 con 0x7f76b8136a70 2023-09-30T22:21:02.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.137+0000 7f76a6ffd700 1 -- 172.21.15.89:0/1829594925 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76b0007910 con 0x7f76b8136a70 2023-09-30T22:21:02.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.229+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f76cc09cd50 con 0x7f76cc130540 2023-09-30T22:21:02.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.230+0000 7f76b37fe700 1 -- 172.21.15.89:0/2967886409 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76bc017020 con 0x7f76cc130540 2023-09-30T22:21:02.230 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:02.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc130540 msgr2=0x7f76cc09a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc130540 0x7f76cc09a590 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000f60 tx=0x7f76bc00c6e0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 shutdown_connections 2023-09-30T22:21:02.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76cc135e60 0x7f76cc09b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76cc130540 0x7f76cc09a590 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 --2- 172.21.15.89:0/2967886409 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76cc133540 0x7f76cc09aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 >> 172.21.15.89:0/2967886409 conn(0x7f76cc0a2aa0 msgr2=0x7f76cc06fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:02.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 shutdown_connections 2023-09-30T22:21:02.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.233+0000 7f76d171d700 1 -- 172.21.15.89:0/2967886409 wait complete. 2023-09-30T22:21:02.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.307+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7940120280 con 0x7f794009d6f0 2023-09-30T22:21:02.307 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.307+0000 7f79277fe700 1 -- 172.21.15.89:0/1584081281 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7940120280 con 0x7f794009d6f0 2023-09-30T22:21:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.309+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f794009d6f0 msgr2=0x7f794011e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.309+0000 7f7945483700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f794009d6f0 0x7f794011e060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7930009fd0 tx=0x7f7930004750 comp rx=0 tx=0).stop 2023-09-30T22:21:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.310+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 shutdown_connections 2023-09-30T22:21:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.310+0000 7f7945483700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794009e410 0x7f794011eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.310+0000 7f7945483700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f794009d6f0 0x7f794011e060 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.310+0000 7f7945483700 1 --2- 172.21.15.89:0/1584081281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f794009e000 0x7f794011e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.310+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 >> 172.21.15.89:0/1584081281 conn(0x7f7940066f50 msgr2=0x7f79400710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.310+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 shutdown_connections 2023-09-30T22:21:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.310+0000 7f7945483700 1 -- 172.21.15.89:0/1584081281 wait complete. 2023-09-30T22:21:02.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:02.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.330+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f357c122050 con 0x7f357c09d630 2023-09-30T22:21:02.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.330+0000 7f3572ffd700 1 -- 172.21.15.89:0/3615987386 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3578021030 con 0x7f357c09d630 2023-09-30T22:21:02.330 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:02.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.332+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f76b809e980 con 0x7f76b8136a70 2023-09-30T22:21:02.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.332+0000 7f76a6ffd700 1 -- 172.21.15.89:0/1829594925 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76b0019020 con 0x7f76b8136a70 2023-09-30T22:21:02.332 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:02.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c09d630 msgr2=0x7f357c11f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 --2- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c09d630 0x7f357c11f5d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f357800ec10 tx=0x7f3578009e70 comp rx=0 tx=0).stop 2023-09-30T22:21:02.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 shutdown_connections 2023-09-30T22:21:02.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 --2- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357c133930 0x7f357c1c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 --2- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f357c09d630 0x7f357c11f5d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 --2- 172.21.15.89:0/3615987386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f357c0a0a30 0x7f357c11fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 >> 172.21.15.89:0/3615987386 conn(0x7f357c0a3050 msgr2=0x7f357c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:02.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 shutdown_connections 2023-09-30T22:21:02.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.333+0000 7f3584497700 1 -- 172.21.15.89:0/3615987386 wait complete. 2023-09-30T22:21:02.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 msgr2=0x7f76b8135370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:02.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 --2- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8135370 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b9e0 tx=0x7f76b000bcf0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 shutdown_connections 2023-09-30T22:21:02.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 --2- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f76b8136a70 0x7f76b8135370 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 --2- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b809d9f0 0x7f76b81348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 --2- 172.21.15.89:0/1829594925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b81316c0 0x7f76b8134e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:02.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 >> 172.21.15.89:0/1829594925 conn(0x7f76b8066f50 msgr2=0x7f76b8136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:02.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 shutdown_connections 2023-09-30T22:21:02.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:02.334+0000 7f76bcd3f700 1 -- 172.21.15.89:0/1829594925 wait complete. 2023-09-30T22:21:02.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:02.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:03.251 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:03.323 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:03.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:03.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:03.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f65937fc700 1 Processor -- start 2023-09-30T22:21:03.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f65937fc700 1 -- start start 2023-09-30T22:21:03.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f65937fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 0x7f658c130420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f65937fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130960 0x7f658c133570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f65937fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f658c130d50 0x7f658c133ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f65937fc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658c099ef0 con 0x7f658c130d50 2023-09-30T22:21:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f65937fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c099500 con 0x7f658c071b40 2023-09-30T22:21:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f65937fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f658c099c10 con 0x7f658c130960 2023-09-30T22:21:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.397+0000 7f6591598700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 0x7f658c130420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6591598700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 0x7f658c130420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58244/0 (socket says 172.21.15.89:58244) 2023-09-30T22:21:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6591598700 1 -- 172.21.15.89:0/1654448484 learned_addr learned my addr 172.21.15.89:0/1654448484 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6590d97700 1 --2- 172.21.15.89:0/1654448484 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130960 0x7f658c133570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6591d99700 1 --2- 172.21.15.89:0/1654448484 >> v2:172.21.15.89:3300/0 conn(0x7f658c130d50 0x7f658c133ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6590d97700 1 -- 172.21.15.89:0/1654448484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 msgr2=0x7f658c130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6590d97700 1 --2- 172.21.15.89:0/1654448484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 0x7f658c130420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6590d97700 1 -- 172.21.15.89:0/1654448484 >> v2:172.21.15.89:3300/0 conn(0x7f658c130d50 msgr2=0x7f658c133ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6590d97700 1 --2- 172.21.15.89:0/1654448484 >> v2:172.21.15.89:3300/0 conn(0x7f658c130d50 0x7f658c133ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6590d97700 1 -- 172.21.15.89:0/1654448484 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c09e370 con 0x7f658c130960 2023-09-30T22:21:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6590d97700 1 --2- 172.21.15.89:0/1654448484 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130960 0x7f658c133570 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f657c009840 tx=0x7f657c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=842a6e786acda9cc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6583fff700 1 -- 172.21.15.89:0/1654448484 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c011830 con 0x7f658c130960 2023-09-30T22:21:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6583fff700 1 -- 172.21.15.89:0/1654448484 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c011990 con 0x7f658c130960 2023-09-30T22:21:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.398+0000 7f6583fff700 1 -- 172.21.15.89:0/1654448484 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c011d20 con 0x7f658c130960 2023-09-30T22:21:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 -- 172.21.15.89:0/1654448484 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130960 msgr2=0x7f658c133570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 --2- 172.21.15.89:0/1654448484 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130960 0x7f658c133570 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f657c009840 tx=0x7f657c009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 -- 172.21.15.89:0/1654448484 shutdown_connections 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 --2- 172.21.15.89:0/1654448484 >> v2:172.21.15.89:3300/0 conn(0x7f658c130d50 0x7f658c133ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 --2- 172.21.15.89:0/1654448484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 0x7f658c130420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 --2- 172.21.15.89:0/1654448484 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130960 0x7f658c133570 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 -- 172.21.15.89:0/1654448484 >> 172.21.15.89:0/1654448484 conn(0x7f658c09e7e0 msgr2=0x7f658c09ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 -- 172.21.15.89:0/1654448484 shutdown_connections 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.399+0000 7f65937fc700 1 -- 172.21.15.89:0/1654448484 wait complete. 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.400+0000 7f65937fc700 1 Processor -- start 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.400+0000 7f65937fc700 1 -- start start 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f65937fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 0x7f658c12ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f65937fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658c130960 0x7f658c12d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f65937fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130d50 0x7f658c12d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f65937fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c08b3a0 con 0x7f658c130960 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f65937fc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c08b680 con 0x7f658c071b40 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f65937fc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f658c09fbe0 con 0x7f658c130d50 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f6590d97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658c130960 0x7f658c12d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f6590d97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658c130960 0x7f658c12d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34670/0 (socket says 172.21.15.89:34670) 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f6590d97700 1 -- 172.21.15.89:0/637144723 learned_addr learned my addr 172.21.15.89:0/637144723 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f6590d97700 1 -- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130d50 msgr2=0x7f658c12d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f6590d97700 1 --2- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130d50 0x7f658c12d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f6590d97700 1 -- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 msgr2=0x7f658c12ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f6590d97700 1 --2- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 0x7f658c12ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.401+0000 7f6590d97700 1 -- 172.21.15.89:0/637144723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0094a0 con 0x7f658c130960 2023-09-30T22:21:03.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.402+0000 7f6590d97700 1 --2- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658c130960 0x7f658c12d340 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f657c0090b0 tx=0x7f657c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:03.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.402+0000 7f6581ffb700 1 -- 172.21.15.89:0/637144723 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c004510 con 0x7f658c130960 2023-09-30T22:21:03.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.402+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c08b0e0 con 0x7f658c130960 2023-09-30T22:21:03.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.402+0000 7f6581ffb700 1 -- 172.21.15.89:0/637144723 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c01f070 con 0x7f658c130960 2023-09-30T22:21:03.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.402+0000 7f6581ffb700 1 -- 172.21.15.89:0/637144723 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c0038e0 con 0x7f658c130960 2023-09-30T22:21:03.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.402+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c12ef80 con 0x7f658c130960 2023-09-30T22:21:03.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.403+0000 7f6581ffb700 1 -- 172.21.15.89:0/637144723 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f657c003a40 con 0x7f658c130960 2023-09-30T22:21:03.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.403+0000 7f6581ffb700 1 -- 172.21.15.89:0/637144723 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f657c021070 con 0x7f658c130960 2023-09-30T22:21:03.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.405+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c08b4e0 con 0x7f658c130960 2023-09-30T22:21:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.414+0000 7f6581ffb700 1 -- 172.21.15.89:0/637144723 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f658c08b4e0 con 0x7f658c130960 2023-09-30T22:21:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.461+0000 7fa7e6164700 1 Processor -- start 2023-09-30T22:21:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.461+0000 7fa7e6164700 1 -- start start 2023-09-30T22:21:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.461+0000 7fa7e6164700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009e930 0x7fa7e009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.461+0000 7fa7e6164700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.461+0000 7fa7e6164700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e0070940 con 0x7fa7e009f650 2023-09-30T22:21:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.461+0000 7fa7e6164700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e006fd60 con 0x7fa7e009e930 2023-09-30T22:21:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.461+0000 7fa7e6164700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e006fa80 con 0x7fa7e009f240 2023-09-30T22:21:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7deffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7df7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009e930 0x7fa7e009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7deffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39686/0 (socket says 172.21.15.89:39686) 2023-09-30T22:21:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7deffd700 1 -- 172.21.15.89:0/1172183701 learned_addr learned my addr 172.21.15.89:0/1172183701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7de7fc700 1 -- 172.21.15.89:0/1172183701 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310958775 0 0) 0x7fa7e0070940 con 0x7fa7e009f650 2023-09-30T22:21:03.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7deffd700 1 -- 172.21.15.89:0/1172183701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009e930 msgr2=0x7fa7e009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:03.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7deffd700 1 --2- 172.21.15.89:0/1172183701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009e930 0x7fa7e009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7deffd700 1 -- 172.21.15.89:0/1172183701 >> v1:172.21.15.89:6789/0 conn(0x7fa7e009f650 legacy=0x7fa7e0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7deffd700 1 -- 172.21.15.89:0/1172183701 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e009fb60 con 0x7fa7e009f240 2023-09-30T22:21:03.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7deffd700 1 --2- 172.21.15.89:0/1172183701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01317f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009920 tx=0x7fa7d4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b1f9037fdf2185 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:03.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7de7fc700 1 -- 172.21.15.89:0/1172183701 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4011830 con 0x7fa7e009f240 2023-09-30T22:21:03.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7de7fc700 1 -- 172.21.15.89:0/1172183701 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d4011990 con 0x7fa7e009f240 2023-09-30T22:21:03.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.462+0000 7fa7de7fc700 1 -- 172.21.15.89:0/1172183701 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4011d20 con 0x7fa7e009f240 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.463+0000 7fa7e6164700 1 -- 172.21.15.89:0/1172183701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 msgr2=0x7fa7e01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.463+0000 7fa7e6164700 1 --2- 172.21.15.89:0/1172183701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01317f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009920 tx=0x7fa7d4009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.463+0000 7fa7e6164700 1 -- 172.21.15.89:0/1172183701 shutdown_connections 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.463+0000 7fa7e6164700 1 --2- 172.21.15.89:0/1172183701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009e930 0x7fa7e009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.463+0000 7fa7e6164700 1 --2- 172.21.15.89:0/1172183701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01317f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.463+0000 7fa7e6164700 1 -- 172.21.15.89:0/1172183701 >> 172.21.15.89:0/1172183701 conn(0x7fa7e00a4210 msgr2=0x7fa7e00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.463+0000 7fa7e6164700 1 -- 172.21.15.89:0/1172183701 shutdown_connections 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.463+0000 7fa7e6164700 1 -- 172.21.15.89:0/1172183701 wait complete. 2023-09-30T22:21:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.464+0000 7fa7e6164700 1 Processor -- start 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.464+0000 7fa7e6164700 1 -- start start 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7e6164700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7e009e930 0x7fa7e01a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7e6164700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7e6164700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009f650 0x7fa7e01a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7e6164700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01a5370 con 0x7fa7e009e930 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7e6164700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01b7200 con 0x7fa7e009f650 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7e6164700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01a6c70 con 0x7fa7e009f240 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7df7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7e009e930 0x7fa7e01a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7df7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7e009e930 0x7fa7e01a3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34684/0 (socket says 172.21.15.89:34684) 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7df7fe700 1 -- 172.21.15.89:0/3242310059 learned_addr learned my addr 172.21.15.89:0/3242310059 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:03.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7dffff700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009f650 0x7fa7e01a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7deffd700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7dffff700 1 -- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 msgr2=0x7fa7e01a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7dffff700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7dffff700 1 -- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7e009e930 msgr2=0x7fa7e01a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7dffff700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7e009e930 0x7fa7e01a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7dffff700 1 -- 172.21.15.89:0/3242310059 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d4009550 con 0x7fa7e009f650 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.465+0000 7fa7df7fe700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7e009e930 0x7fa7e01a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.466+0000 7fa7dffff700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009f650 0x7fa7e01a4920 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa7e009faf0 tx=0x7fa7d800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.466+0000 7fa7c7fff700 1 -- 172.21.15.89:0/3242310059 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8003e30 con 0x7fa7e009f650 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.466+0000 7fa7c7fff700 1 -- 172.21.15.89:0/3242310059 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80050d0 con 0x7fa7e009f650 2023-09-30T22:21:03.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.466+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e00a0190 con 0x7fa7e009f650 2023-09-30T22:21:03.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.466+0000 7fa7c7fff700 1 -- 172.21.15.89:0/3242310059 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8005480 con 0x7fa7e009f650 2023-09-30T22:21:03.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.466+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e01a70c0 con 0x7fa7e009f650 2023-09-30T22:21:03.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.467+0000 7fa7c7fff700 1 -- 172.21.15.89:0/3242310059 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa7d80055e0 con 0x7fa7e009f650 2023-09-30T22:21:03.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.467+0000 7fa7c7fff700 1 -- 172.21.15.89:0/3242310059 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa7d803ec60 con 0x7fa7e009f650 2023-09-30T22:21:03.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.467+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e00a3510 con 0x7fa7e009f650 2023-09-30T22:21:03.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.476+0000 7fa7c7fff700 1 -- 172.21.15.89:0/3242310059 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa7d8017020 con 0x7fa7e009f650 2023-09-30T22:21:03.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.504+0000 7fad38cb1700 1 Processor -- start 2023-09-30T22:21:03.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.504+0000 7fad38cb1700 1 -- start start 2023-09-30T22:21:03.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.504+0000 7fad38cb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad3409c650 0x7fad3409ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.504+0000 7fad38cb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409cf60 0x7fad34132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.504+0000 7fad38cb1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3409ba70 con 0x7fad340a0090 2023-09-30T22:21:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.504+0000 7fad38cb1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3409e8f0 con 0x7fad3409c650 2023-09-30T22:21:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.504+0000 7fad38cb1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340a0760 con 0x7fad3409cf60 2023-09-30T22:21:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad31d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fad340a0090 0x7fad34131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52032/0 (socket says 172.21.15.89:52032) 2023-09-30T22:21:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad31d9b700 1 -- 172.21.15.89:0/3907600742 learned_addr learned my addr 172.21.15.89:0/3907600742 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad3259c700 1 --2- 172.21.15.89:0/3907600742 >> v2:172.21.15.89:3300/0 conn(0x7fad3409c650 0x7fad3409ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad32d9d700 1 --2- 172.21.15.89:0/3907600742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409cf60 0x7fad34132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad32d9d700 1 -- 172.21.15.89:0/3907600742 >> v2:172.21.15.89:3300/0 conn(0x7fad3409c650 msgr2=0x7fad3409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad32d9d700 1 --2- 172.21.15.89:0/3907600742 >> v2:172.21.15.89:3300/0 conn(0x7fad3409c650 0x7fad3409ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad32d9d700 1 -- 172.21.15.89:0/3907600742 >> v1:172.21.15.89:6789/0 conn(0x7fad340a0090 legacy=0x7fad34131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad32d9d700 1 -- 172.21.15.89:0/3907600742 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3409a080 con 0x7fad3409cf60 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad32d9d700 1 --2- 172.21.15.89:0/3907600742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409cf60 0x7fad34132080 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009d90 tx=0x7fad2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=620271e314fdc165 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.505+0000 7fad3159a700 1 -- 172.21.15.89:0/3907600742 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00e070 con 0x7fad3409cf60 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad3159a700 1 -- 172.21.15.89:0/3907600742 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c00bb20 con 0x7fad3409cf60 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad3159a700 1 -- 172.21.15.89:0/3907600742 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c012430 con 0x7fad3409cf60 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad38cb1700 1 -- 172.21.15.89:0/3907600742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409cf60 msgr2=0x7fad34132080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad38cb1700 1 --2- 172.21.15.89:0/3907600742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409cf60 0x7fad34132080 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009d90 tx=0x7fad2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad38cb1700 1 -- 172.21.15.89:0/3907600742 shutdown_connections 2023-09-30T22:21:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad38cb1700 1 --2- 172.21.15.89:0/3907600742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409cf60 0x7fad34132080 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad38cb1700 1 --2- 172.21.15.89:0/3907600742 >> v2:172.21.15.89:3300/0 conn(0x7fad3409c650 0x7fad3409ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad38cb1700 1 -- 172.21.15.89:0/3907600742 >> 172.21.15.89:0/3907600742 conn(0x7fad340a3f70 msgr2=0x7fad340a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad38cb1700 1 -- 172.21.15.89:0/3907600742 shutdown_connections 2023-09-30T22:21:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.506+0000 7fad38cb1700 1 -- 172.21.15.89:0/3907600742 wait complete. 2023-09-30T22:21:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.507+0000 7fad38cb1700 1 Processor -- start 2023-09-30T22:21:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.507+0000 7fad38cb1700 1 -- start start 2023-09-30T22:21:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad38cb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409c650 0x7fad3406b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad38cb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad3409cf60 0x7fad3406bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad38cb1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad340a0090 0x7fad3406c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad38cb1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3409d980 con 0x7fad340a0090 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad38cb1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3412ffb0 con 0x7fad3409c650 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad38cb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad3412d4f0 con 0x7fad3409cf60 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad31d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad3409cf60 0x7fad3406bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad31d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad3409cf60 0x7fad3406bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39704/0 (socket says 172.21.15.89:39704) 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad31d9b700 1 -- 172.21.15.89:0/2865561542 learned_addr learned my addr 172.21.15.89:0/2865561542 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad3259c700 1 --2- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409c650 0x7fad3406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad31d9b700 1 -- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409c650 msgr2=0x7fad3406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad31d9b700 1 --2- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409c650 0x7fad3406b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad31d9b700 1 -- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad340a0090 msgr2=0x7fad3406c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad31d9b700 1 --2- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad340a0090 0x7fad3406c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.508+0000 7fad31d9b700 1 -- 172.21.15.89:0/2865561542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c009a70 con 0x7fad3409cf60 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.509+0000 7fad31d9b700 1 --2- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad3409cf60 0x7fad3406bdc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c550 tx=0x7fad2800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.509+0000 7fad22ffd700 1 -- 172.21.15.89:0/2865561542 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28005d70 con 0x7fad3409cf60 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.509+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3411efa0 con 0x7fad3409cf60 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.509+0000 7fad22ffd700 1 -- 172.21.15.89:0/2865561542 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad280070d0 con 0x7fad3409cf60 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.509+0000 7fad22ffd700 1 -- 172.21.15.89:0/2865561542 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28007440 con 0x7fad3409cf60 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.509+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3411f940 con 0x7fad3409cf60 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.510+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3411f5e0 con 0x7fad3409cf60 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.511+0000 7fad22ffd700 1 -- 172.21.15.89:0/2865561542 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad280075a0 con 0x7fad3409cf60 2023-09-30T22:21:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.511+0000 7fad22ffd700 1 -- 172.21.15.89:0/2865561542 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fad28040c50 con 0x7fad3409cf60 2023-09-30T22:21:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.513+0000 7f9e7b8a2700 1 Processor -- start 2023-09-30T22:21:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7b8a2700 1 -- start start 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7b8a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7b8a2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e7412f8b0 0x7f9e7412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7b8a2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7412cd90 con 0x7f9e74130260 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7b8a2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7412d330 con 0x7f9e7412f8b0 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7b8a2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e741306d0 con 0x7f9e7412cf60 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7963e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7963e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39706/0 (socket says 172.21.15.89:39706) 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.514+0000 7f9e7963e700 1 -- 172.21.15.89:0/599213880 learned_addr learned my addr 172.21.15.89:0/599213880 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.515+0000 7f9e78e3d700 1 --2- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e7412f8b0 0x7f9e7412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.515+0000 7f9e7963e700 1 -- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e7412f8b0 msgr2=0x7f9e7412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.515+0000 7f9e7963e700 1 --2- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e7412f8b0 0x7f9e7412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.515+0000 7f9e7963e700 1 -- 172.21.15.89:0/599213880 >> v1:172.21.15.89:6789/0 conn(0x7f9e74130260 legacy=0x7f9e74136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.515+0000 7f9e7963e700 1 -- 172.21.15.89:0/599213880 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e74071780 con 0x7f9e7412cf60 2023-09-30T22:21:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.515+0000 7f9e78e3d700 1 --2- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e7412f8b0 0x7f9e7412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.515+0000 7f9e7963e700 1 --2- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7406f5d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000bbc0 tx=0x7f9e7000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c340a5b1857d1294 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.516+0000 7f9e6bfff700 1 -- 172.21.15.89:0/599213880 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000dc80 con 0x7f9e7412cf60 2023-09-30T22:21:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.516+0000 7f9e6bfff700 1 -- 172.21.15.89:0/599213880 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7000dde0 con 0x7f9e7412cf60 2023-09-30T22:21:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.516+0000 7f9e6bfff700 1 -- 172.21.15.89:0/599213880 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70017630 con 0x7f9e7412cf60 2023-09-30T22:21:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.516+0000 7fad22ffd700 1 -- 172.21.15.89:0/2865561542 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad28044060 con 0x7fad3409cf60 2023-09-30T22:21:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.516+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 msgr2=0x7f9e7406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.516+0000 7f9e7b8a2700 1 --2- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7406f5d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000bbc0 tx=0x7f9e7000bed0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.517+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/599213880 shutdown_connections 2023-09-30T22:21:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.517+0000 7f9e7b8a2700 1 --2- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7406f5d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.517+0000 7f9e7b8a2700 1 --2- 172.21.15.89:0/599213880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e7412f8b0 0x7f9e7412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.517+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/599213880 >> 172.21.15.89:0/599213880 conn(0x7f9e740a2aa0 msgr2=0x7f9e740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.517+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/599213880 shutdown_connections 2023-09-30T22:21:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.517+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/599213880 wait complete. 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7b8a2700 1 Processor -- start 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7b8a2700 1 -- start start 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7b8a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7409bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7b8a2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e7412f8b0 0x7f9e7409c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7b8a2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e74130260 0x7f9e741c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7b8a2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7409e8e0 con 0x7f9e7412f8b0 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7b8a2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e74129900 con 0x7f9e74130260 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7b8a2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e74129fa0 con 0x7f9e7412cf60 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e78e3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e7412f8b0 0x7f9e7409c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e78e3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e7412f8b0 0x7f9e7409c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34700/0 (socket says 172.21.15.89:34700) 2023-09-30T22:21:03.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e78e3d700 1 -- 172.21.15.89:0/469111303 learned_addr learned my addr 172.21.15.89:0/469111303 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e78e3d700 1 -- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 msgr2=0x7f9e7409bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e7963e700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7409bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e78e3d700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7409bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e78e3d700 1 -- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e74130260 msgr2=0x7f9e741c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.518+0000 7f9e78e3d700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e74130260 0x7f9e741c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.519+0000 7f9e78e3d700 1 -- 172.21.15.89:0/469111303 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7000b7f0 con 0x7f9e7412f8b0 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.519+0000 7f9e7963e700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7409bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.519+0000 7f9e78e3d700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e7412f8b0 0x7f9e7409c130 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400c740 tx=0x7f9e6400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.519+0000 7f9e69ffb700 1 -- 172.21.15.89:0/469111303 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64004220 con 0x7f9e7412f8b0 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.519+0000 7f9e69ffb700 1 -- 172.21.15.89:0/469111303 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64004380 con 0x7f9e7412f8b0 2023-09-30T22:21:03.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.519+0000 7f9e69ffb700 1 -- 172.21.15.89:0/469111303 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e640036a0 con 0x7f9e7412f8b0 2023-09-30T22:21:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.519+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7409b5a0 con 0x7f9e7412f8b0 2023-09-30T22:21:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.519+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7409ccb0 con 0x7f9e7412f8b0 2023-09-30T22:21:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.520+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e74074ea0 con 0x7f9e7412f8b0 2023-09-30T22:21:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.520+0000 7f9e69ffb700 1 -- 172.21.15.89:0/469111303 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e64003800 con 0x7f9e7412f8b0 2023-09-30T22:21:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.521+0000 7f9e69ffb700 1 -- 172.21.15.89:0/469111303 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9e6403ff70 con 0x7f9e7412f8b0 2023-09-30T22:21:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.525+0000 7f9e69ffb700 1 -- 172.21.15.89:0/469111303 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e64021b40 con 0x7f9e7412f8b0 2023-09-30T22:21:03.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.653+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f658c08b4e0 con 0x7f658c130960 2023-09-30T22:21:03.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.654+0000 7f6581ffb700 1 -- 172.21.15.89:0/637144723 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f658c08b4e0 con 0x7f658c130960 2023-09-30T22:21:03.654 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658c130960 msgr2=0x7f658c12d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 --2- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658c130960 0x7f658c12d340 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f657c0090b0 tx=0x7f657c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 shutdown_connections 2023-09-30T22:21:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 --2- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f658c130d50 0x7f658c12d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 --2- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f658c071b40 0x7f658c12ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 --2- 172.21.15.89:0/637144723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f658c130960 0x7f658c12d340 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 >> 172.21.15.89:0/637144723 conn(0x7f658c09e7e0 msgr2=0x7f658c1321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 shutdown_connections 2023-09-30T22:21:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.656+0000 7f65937fc700 1 -- 172.21.15.89:0/637144723 wait complete. 2023-09-30T22:21:03.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:03.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.688+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa7e0071d20 con 0x7fa7e009f650 2023-09-30T22:21:03.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.688+0000 7fa7c7fff700 1 -- 172.21.15.89:0/3242310059 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa7d8019b40 con 0x7fa7e009f650 2023-09-30T22:21:03.688 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:03.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009f650 msgr2=0x7fa7e01a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009f650 0x7fa7e01a4920 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa7e009faf0 tx=0x7fa7d800ca10 comp rx=0 tx=0).stop 2023-09-30T22:21:03.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 shutdown_connections 2023-09-30T22:21:03.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7e009f650 0x7fa7e01a4920 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7e009e930 0x7fa7e01a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 --2- 172.21.15.89:0/3242310059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7e009f240 0x7fa7e01a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 >> 172.21.15.89:0/3242310059 conn(0x7fa7e00a4210 msgr2=0x7fa7e01b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:03.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 shutdown_connections 2023-09-30T22:21:03.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.690+0000 7fa7e6164700 1 -- 172.21.15.89:0/3242310059 wait complete. 2023-09-30T22:21:03.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.718+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e740713a0 con 0x7f9e7412f8b0 2023-09-30T22:21:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.719+0000 7f9e69ffb700 1 -- 172.21.15.89:0/469111303 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e64027070 con 0x7f9e7412f8b0 2023-09-30T22:21:03.720 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e7412f8b0 msgr2=0x7f9e7409c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e7412f8b0 0x7f9e7409c130 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400c740 tx=0x7f9e6400cb00 comp rx=0 tx=0).stop 2023-09-30T22:21:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 shutdown_connections 2023-09-30T22:21:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e74130260 0x7f9e741c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e7412cf60 0x7f9e7409bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 --2- 172.21.15.89:0/469111303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e7412f8b0 0x7f9e7409c130 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 >> 172.21.15.89:0/469111303 conn(0x7f9e740a2aa0 msgr2=0x7f9e7406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 shutdown_connections 2023-09-30T22:21:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.721+0000 7f9e7b8a2700 1 -- 172.21.15.89:0/469111303 wait complete. 2023-09-30T22:21:03.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:03.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.753+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fad340747f0 con 0x7fad3409cf60 2023-09-30T22:21:03.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.753+0000 7fad22ffd700 1 -- 172.21.15.89:0/2865561542 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad28017070 con 0x7fad3409cf60 2023-09-30T22:21:03.754 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.755+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad3409cf60 msgr2=0x7fad3406bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.755+0000 7fad38cb1700 1 --2- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad3409cf60 0x7fad3406bdc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c550 tx=0x7fad2800c910 comp rx=0 tx=0).stop 2023-09-30T22:21:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.756+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 shutdown_connections 2023-09-30T22:21:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.756+0000 7fad38cb1700 1 --2- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad340a0090 0x7fad3406c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.756+0000 7fad38cb1700 1 --2- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad3409c650 0x7fad3406b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.756+0000 7fad38cb1700 1 --2- 172.21.15.89:0/2865561542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad3409cf60 0x7fad3406bdc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.756+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 >> 172.21.15.89:0/2865561542 conn(0x7fad340a3f70 msgr2=0x7fad3412ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.756+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 shutdown_connections 2023-09-30T22:21:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:03.756+0000 7fad38cb1700 1 -- 172.21.15.89:0/2865561542 wait complete. 2023-09-30T22:21:03.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:04.669 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:04.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:04.735 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:04.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.813+0000 7f826c2e4700 1 Processor -- start 2023-09-30T22:21:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.813+0000 7f826c2e4700 1 -- start start 2023-09-30T22:21:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826c2e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826406a770 0x7f826409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826c2e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826409dc30 0x7f826409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826c2e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f826413c1e0 0x7f826413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826c2e4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826406ab40 con 0x7f826413c1e0 2023-09-30T22:21:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826c2e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826409ebc0 con 0x7f826409dc30 2023-09-30T22:21:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826c2e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f826409a810 con 0x7f826406a770 2023-09-30T22:21:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826987f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826409dc30 0x7f826409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826a080700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826406a770 0x7f826409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826987f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826409dc30 0x7f826409e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58314/0 (socket says 172.21.15.89:58314) 2023-09-30T22:21:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.814+0000 7f826987f700 1 -- 172.21.15.89:0/4043018076 learned_addr learned my addr 172.21.15.89:0/4043018076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826a881700 1 --2- 172.21.15.89:0/4043018076 >> v2:172.21.15.89:3300/0 conn(0x7f826413c1e0 0x7f826413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826a080700 1 -- 172.21.15.89:0/4043018076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826409dc30 msgr2=0x7f826409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826a080700 1 --2- 172.21.15.89:0/4043018076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826409dc30 0x7f826409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826a080700 1 -- 172.21.15.89:0/4043018076 >> v2:172.21.15.89:3300/0 conn(0x7f826413c1e0 msgr2=0x7f826413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826a080700 1 --2- 172.21.15.89:0/4043018076 >> v2:172.21.15.89:3300/0 conn(0x7f826413c1e0 0x7f826413e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826a080700 1 -- 172.21.15.89:0/4043018076 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826409e920 con 0x7f826406a770 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826a080700 1 --2- 172.21.15.89:0/4043018076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826406a770 0x7f826409d6f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8260009870 tx=0x7f8260009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51fb501f4ca6a24c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826907e700 1 -- 172.21.15.89:0/4043018076 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260011830 con 0x7f826406a770 2023-09-30T22:21:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826907e700 1 -- 172.21.15.89:0/4043018076 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8260011990 con 0x7f826406a770 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.815+0000 7f826907e700 1 -- 172.21.15.89:0/4043018076 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260011d20 con 0x7f826406a770 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.816+0000 7f826c2e4700 1 -- 172.21.15.89:0/4043018076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826406a770 msgr2=0x7f826409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.816+0000 7f826c2e4700 1 --2- 172.21.15.89:0/4043018076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826406a770 0x7f826409d6f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8260009870 tx=0x7f8260009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.816+0000 7f826c2e4700 1 -- 172.21.15.89:0/4043018076 shutdown_connections 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.816+0000 7f826c2e4700 1 --2- 172.21.15.89:0/4043018076 >> v2:172.21.15.89:3300/0 conn(0x7f826413c1e0 0x7f826413e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.816+0000 7f826c2e4700 1 --2- 172.21.15.89:0/4043018076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826406a770 0x7f826409d6f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.816+0000 7f826c2e4700 1 --2- 172.21.15.89:0/4043018076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826409dc30 0x7f826409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.816+0000 7f826c2e4700 1 -- 172.21.15.89:0/4043018076 >> 172.21.15.89:0/4043018076 conn(0x7f8264066f50 msgr2=0x7f826409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.817+0000 7f826c2e4700 1 -- 172.21.15.89:0/4043018076 shutdown_connections 2023-09-30T22:21:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.817+0000 7f826c2e4700 1 -- 172.21.15.89:0/4043018076 wait complete. 2023-09-30T22:21:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.817+0000 7f826c2e4700 1 Processor -- start 2023-09-30T22:21:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826c2e4700 1 -- start start 2023-09-30T22:21:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826c2e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826406a770 0x7f8264071e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826c2e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826409dc30 0x7f826412dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826c2e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f826413c1e0 0x7f826412e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826c2e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8264131980 con 0x7f826413c1e0 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826c2e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8264131b60 con 0x7f826406a770 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826c2e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82640a19e0 con 0x7f826409dc30 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826a080700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826406a770 0x7f8264071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826a080700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826406a770 0x7f8264071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58328/0 (socket says 172.21.15.89:58328) 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826a080700 1 -- 172.21.15.89:0/955872225 learned_addr learned my addr 172.21.15.89:0/955872225 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826a881700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f826413c1e0 0x7f826412e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826987f700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826409dc30 0x7f826412dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826a881700 1 -- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826409dc30 msgr2=0x7f826412dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.818+0000 7f826a881700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826409dc30 0x7f826412dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f826a881700 1 -- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826406a770 msgr2=0x7f8264071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f826a881700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826406a770 0x7f8264071e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f826a881700 1 -- 172.21.15.89:0/955872225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600094a0 con 0x7f826413c1e0 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f826a881700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f826413c1e0 0x7f826412e290 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f825c009d90 tx=0x7f825c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f825affd700 1 -- 172.21.15.89:0/955872225 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00cc90 con 0x7f826413c1e0 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f825affd700 1 -- 172.21.15.89:0/955872225 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c00f040 con 0x7f826413c1e0 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826409f1a0 con 0x7f826413c1e0 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f825affd700 1 -- 172.21.15.89:0/955872225 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c014480 con 0x7f826413c1e0 2023-09-30T22:21:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.819+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8264130cd0 con 0x7f826413c1e0 2023-09-30T22:21:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.820+0000 7f825affd700 1 -- 172.21.15.89:0/955872225 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f825c0145e0 con 0x7f826413c1e0 2023-09-30T22:21:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.820+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826411cc30 con 0x7f826413c1e0 2023-09-30T22:21:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.820+0000 7f825affd700 1 -- 172.21.15.89:0/955872225 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f825c040160 con 0x7f826413c1e0 2023-09-30T22:21:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.828+0000 7f825affd700 1 -- 172.21.15.89:0/955872225 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f825c018070 con 0x7f826413c1e0 2023-09-30T22:21:04.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.857+0000 7f52bb364700 1 Processor -- start 2023-09-30T22:21:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.857+0000 7f52bb364700 1 -- start start 2023-09-30T22:21:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52bb364700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b41338a0 0x7f52b4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52bb364700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52bb364700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b406ef30 con 0x7f52b409de50 2023-09-30T22:21:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52bb364700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b406fef0 con 0x7f52b409e930 2023-09-30T22:21:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52bb364700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52b41334b0 con 0x7f52b41338a0 2023-09-30T22:21:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52b88ff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f52b409de50 0x7f52b409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52054/0 (socket says 172.21.15.89:52054) 2023-09-30T22:21:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52b88ff700 1 -- 172.21.15.89:0/1568706259 learned_addr learned my addr 172.21.15.89:0/1568706259 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52b9901700 1 --2- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52b9100700 1 --2- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b41338a0 0x7f52b4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52abfff700 1 -- 172.21.15.89:0/1568706259 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111037394 0 0) 0x7f52b406ef30 con 0x7f52b409de50 2023-09-30T22:21:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52abfff700 1 -- 172.21.15.89:0/1568706259 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a00035b0 con 0x7f52b409de50 2023-09-30T22:21:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52b9901700 1 -- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b41338a0 msgr2=0x7f52b4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52b9901700 1 --2- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b41338a0 0x7f52b4133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52b9901700 1 -- 172.21.15.89:0/1568706259 >> v1:172.21.15.89:6789/0 conn(0x7f52b409de50 legacy=0x7f52b409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.858+0000 7f52b9901700 1 -- 172.21.15.89:0/1568706259 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b406fc30 con 0x7f52b409e930 2023-09-30T22:21:04.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.859+0000 7f52b9901700 1 --2- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b413b740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009dc0 tx=0x7f52ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7191ab8c614910 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.859+0000 7f52abfff700 1 -- 172.21.15.89:0/1568706259 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00e070 con 0x7f52b409e930 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.859+0000 7f52abfff700 1 -- 172.21.15.89:0/1568706259 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00bb20 con 0x7f52b409e930 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.859+0000 7f52abfff700 1 -- 172.21.15.89:0/1568706259 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac012430 con 0x7f52b409e930 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.860+0000 7f52bb364700 1 -- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 msgr2=0x7f52b413b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.860+0000 7f52bb364700 1 --2- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b413b740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009dc0 tx=0x7f52ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.860+0000 7f52bb364700 1 -- 172.21.15.89:0/1568706259 shutdown_connections 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.860+0000 7f52bb364700 1 --2- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b413b740 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.860+0000 7f52bb364700 1 --2- 172.21.15.89:0/1568706259 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b41338a0 0x7f52b4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.860+0000 7f52bb364700 1 -- 172.21.15.89:0/1568706259 >> 172.21.15.89:0/1568706259 conn(0x7f52b40a2aa0 msgr2=0x7f52b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.860+0000 7f52bb364700 1 -- 172.21.15.89:0/1568706259 shutdown_connections 2023-09-30T22:21:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.860+0000 7f52bb364700 1 -- 172.21.15.89:0/1568706259 wait complete. 2023-09-30T22:21:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.861+0000 7f52bb364700 1 Processor -- start 2023-09-30T22:21:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.861+0000 7f52bb364700 1 -- start start 2023-09-30T22:21:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52bb364700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b409de50 0x7f52b411d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52bb364700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b411dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52bb364700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52b41338a0 0x7f52b411e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52bb364700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b411f230 con 0x7f52b41338a0 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52bb364700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b406f640 con 0x7f52b409e930 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52bb364700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4120e60 con 0x7f52b409de50 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52b9100700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b409de50 0x7f52b411d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52b88ff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b411dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52b9100700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b409de50 0x7f52b411d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39744/0 (socket says 172.21.15.89:39744) 2023-09-30T22:21:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52b9100700 1 -- 172.21.15.89:0/1513991054 learned_addr learned my addr 172.21.15.89:0/1513991054 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.862+0000 7f52b9901700 1 --2- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52b41338a0 0x7f52b411e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52b9100700 1 -- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 msgr2=0x7f52b411dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52b9100700 1 --2- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b411dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52b9100700 1 -- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52b41338a0 msgr2=0x7f52b411e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52b9100700 1 --2- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52b41338a0 0x7f52b411e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52b9100700 1 -- 172.21.15.89:0/1513991054 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac009a70 con 0x7f52b409de50 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52b9100700 1 --2- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b409de50 0x7f52b411d6a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f52b000e9b0 tx=0x7f52b000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52a9ffb700 1 -- 172.21.15.89:0/1513991054 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0009710 con 0x7f52b409de50 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52a9ffb700 1 -- 172.21.15.89:0/1513991054 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b000ee50 con 0x7f52b409de50 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b40a0c50 con 0x7f52b409de50 2023-09-30T22:21:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b41208d0 con 0x7f52b409de50 2023-09-30T22:21:04.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.863+0000 7f52a9ffb700 1 -- 172.21.15.89:0/1513991054 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000ca50 con 0x7f52b409de50 2023-09-30T22:21:04.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.864+0000 7f52a9ffb700 1 -- 172.21.15.89:0/1513991054 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f52b000cbb0 con 0x7f52b409de50 2023-09-30T22:21:04.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.864+0000 7f52a9ffb700 1 -- 172.21.15.89:0/1513991054 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f52b003f6d0 con 0x7f52b409de50 2023-09-30T22:21:04.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.865+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b4120a60 con 0x7f52b409de50 2023-09-30T22:21:04.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.874+0000 7f52a9ffb700 1 -- 172.21.15.89:0/1513991054 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f52b0013470 con 0x7f52b409de50 2023-09-30T22:21:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.899+0000 7f3f599f1700 1 Processor -- start 2023-09-30T22:21:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.899+0000 7f3f599f1700 1 -- start start 2023-09-30T22:21:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f599f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f54138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f599f1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f54139170 0x7f3f5413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f599f1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5409b120 con 0x7f3f5409d6f0 2023-09-30T22:21:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f599f1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5406b460 con 0x7f3f54139170 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f599f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5409a7e0 con 0x7f3f5409e1d0 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f527fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f54138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f54139170 0x7f3f5413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f527fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f54138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58348/0 (socket says 172.21.15.89:58348) 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f537fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f54139170 0x7f3f5413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34740/0 (socket says 172.21.15.89:34740) 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f527fc700 1 -- 172.21.15.89:0/448354983 learned_addr learned my addr 172.21.15.89:0/448354983 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f527fc700 1 -- 172.21.15.89:0/448354983 >> v2:172.21.15.89:3300/0 conn(0x7f3f54139170 msgr2=0x7f3f5413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f527fc700 1 --2- 172.21.15.89:0/448354983 >> v2:172.21.15.89:3300/0 conn(0x7f3f54139170 0x7f3f5413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f527fc700 1 -- 172.21.15.89:0/448354983 >> v1:172.21.15.89:6789/0 conn(0x7f3f5409d6f0 legacy=0x7f3f5409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f527fc700 1 -- 172.21.15.89:0/448354983 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5409a500 con 0x7f3f5409e1d0 2023-09-30T22:21:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f537fe700 1 --2- 172.21.15.89:0/448354983 >> v2:172.21.15.89:3300/0 conn(0x7f3f54139170 0x7f3f5413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.900+0000 7f3f527fc700 1 --2- 172.21.15.89:0/448354983 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f54138c30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009920 tx=0x7f3f48009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14cd8686af404f04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f51ffb700 1 -- 172.21.15.89:0/448354983 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48011830 con 0x7f3f5409e1d0 2023-09-30T22:21:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f51ffb700 1 -- 172.21.15.89:0/448354983 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48011990 con 0x7f3f5409e1d0 2023-09-30T22:21:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f51ffb700 1 -- 172.21.15.89:0/448354983 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48011d20 con 0x7f3f5409e1d0 2023-09-30T22:21:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f599f1700 1 -- 172.21.15.89:0/448354983 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 msgr2=0x7f3f54138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f599f1700 1 --2- 172.21.15.89:0/448354983 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f54138c30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009920 tx=0x7f3f48009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f599f1700 1 -- 172.21.15.89:0/448354983 shutdown_connections 2023-09-30T22:21:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f599f1700 1 --2- 172.21.15.89:0/448354983 >> v2:172.21.15.89:3300/0 conn(0x7f3f54139170 0x7f3f5413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f599f1700 1 --2- 172.21.15.89:0/448354983 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f54138c30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f599f1700 1 -- 172.21.15.89:0/448354983 >> 172.21.15.89:0/448354983 conn(0x7f3f54066f50 msgr2=0x7f3f5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:04.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.901+0000 7f3f599f1700 1 -- 172.21.15.89:0/448354983 shutdown_connections 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.902+0000 7f3f599f1700 1 -- 172.21.15.89:0/448354983 wait complete. 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.902+0000 7f3f599f1700 1 Processor -- start 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f599f1700 1 -- start start 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f599f1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f5409d6f0 0x7f3f5411e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f599f1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f5411ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f599f1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f54139170 0x7f3f5411f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f599f1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54121be0 con 0x7f3f5409d6f0 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f599f1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5409c710 con 0x7f3f5409e1d0 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f599f1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54120500 con 0x7f3f54139170 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f52ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f5409d6f0 0x7f3f5411e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f54139170 0x7f3f5411f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f52ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f5409d6f0 0x7f3f5411e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34750/0 (socket says 172.21.15.89:34750) 2023-09-30T22:21:04.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f52ffd700 1 -- 172.21.15.89:0/3456239727 learned_addr learned my addr 172.21.15.89:0/3456239727 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:04.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.903+0000 7f3f527fc700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f5411ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f52ffd700 1 -- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f54139170 msgr2=0x7f3f5411f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f52ffd700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f54139170 0x7f3f5411f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f52ffd700 1 -- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 msgr2=0x7f3f5411ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f52ffd700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f5411ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f52ffd700 1 -- 172.21.15.89:0/3456239727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c009a70 con 0x7f3f5409d6f0 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f537fe700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f54139170 0x7f3f5411f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f527fc700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f5411ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f52ffd700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f5409d6f0 0x7f3f5411e930 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400c4e0 tx=0x7f3f4400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f3bfff700 1 -- 172.21.15.89:0/3456239727 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440052d0 con 0x7f3f5409d6f0 2023-09-30T22:21:04.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f48009550 con 0x7f3f5409d6f0 2023-09-30T22:21:04.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f3bfff700 1 -- 172.21.15.89:0/3456239727 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f44005450 con 0x7f3f5409d6f0 2023-09-30T22:21:04.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f3bfff700 1 -- 172.21.15.89:0/3456239727 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440038c0 con 0x7f3f5409d6f0 2023-09-30T22:21:04.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.904+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54134740 con 0x7f3f5409d6f0 2023-09-30T22:21:04.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.905+0000 7f3f3bfff700 1 -- 172.21.15.89:0/3456239727 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f44003a20 con 0x7f3f5409d6f0 2023-09-30T22:21:04.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.905+0000 7f3f3bfff700 1 -- 172.21.15.89:0/3456239727 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3f4403ea10 con 0x7f3f5409d6f0 2023-09-30T22:21:04.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.906+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f54122bc0 con 0x7f3f5409d6f0 2023-09-30T22:21:04.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.910+0000 7f3f3bfff700 1 -- 172.21.15.89:0/3456239727 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f44018260 con 0x7f3f5409d6f0 2023-09-30T22:21:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.937+0000 7fc79aba2700 1 Processor -- start 2023-09-30T22:21:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.937+0000 7fc79aba2700 1 -- start start 2023-09-30T22:21:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc79aba2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc79409e1d0 0x7fc794135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc79aba2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc794135e50 0x7fc794138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc79aba2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79409f130 con 0x7fc79409d6f0 2023-09-30T22:21:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc79aba2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79409f520 con 0x7fc79409e1d0 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc79aba2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79409e930 con 0x7fc794135e50 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc793fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc79409e1d0 0x7fc794135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc793fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc79409e1d0 0x7fc794135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34760/0 (socket says 172.21.15.89:34760) 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc793fff700 1 -- 172.21.15.89:0/14210270 learned_addr learned my addr 172.21.15.89:0/14210270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc79913f700 1 --2- 172.21.15.89:0/14210270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc794135e50 0x7fc794138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc793fff700 1 -- 172.21.15.89:0/14210270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc794135e50 msgr2=0x7fc794138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc793fff700 1 --2- 172.21.15.89:0/14210270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc794135e50 0x7fc794138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc793fff700 1 -- 172.21.15.89:0/14210270 >> v1:172.21.15.89:6789/0 conn(0x7fc79409d6f0 legacy=0x7fc79409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc793fff700 1 -- 172.21.15.89:0/14210270 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79409b2d0 con 0x7fc79409e1d0 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.938+0000 7fc793fff700 1 --2- 172.21.15.89:0/14210270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 0x7fc794135910 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc780009950 tx=0x7fc780009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e787ea2d7a2549d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc7937fe700 1 -- 172.21.15.89:0/14210270 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780009ea0 con 0x7fc79409e1d0 2023-09-30T22:21:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc7937fe700 1 -- 172.21.15.89:0/14210270 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780004230 con 0x7fc79409e1d0 2023-09-30T22:21:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc7937fe700 1 -- 172.21.15.89:0/14210270 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78000b7e0 con 0x7fc79409e1d0 2023-09-30T22:21:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc79aba2700 1 -- 172.21.15.89:0/14210270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 msgr2=0x7fc794135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc79aba2700 1 --2- 172.21.15.89:0/14210270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 0x7fc794135910 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc780009950 tx=0x7fc780009c60 comp rx=0 tx=0).stop 2023-09-30T22:21:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc79aba2700 1 -- 172.21.15.89:0/14210270 shutdown_connections 2023-09-30T22:21:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc79aba2700 1 --2- 172.21.15.89:0/14210270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc794135e50 0x7fc794138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc79aba2700 1 --2- 172.21.15.89:0/14210270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 0x7fc794135910 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc79aba2700 1 -- 172.21.15.89:0/14210270 >> 172.21.15.89:0/14210270 conn(0x7fc794066f50 msgr2=0x7fc79409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc79aba2700 1 -- 172.21.15.89:0/14210270 shutdown_connections 2023-09-30T22:21:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.939+0000 7fc79aba2700 1 -- 172.21.15.89:0/14210270 wait complete. 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.940+0000 7fc79aba2700 1 Processor -- start 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.940+0000 7fc79aba2700 1 -- start start 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79aba2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79409d6f0 0x7fc7941c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79aba2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 0x7fc7941c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79aba2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc794135e50 0x7fc794119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79aba2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79411cf90 con 0x7fc79409e1d0 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79aba2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc794130240 con 0x7fc79409d6f0 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79aba2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc79411b870 con 0x7fc794135e50 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79893e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79409d6f0 0x7fc7941c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79913f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc794135e50 0x7fc794119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79893e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79409d6f0 0x7fc7941c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58392/0 (socket says 172.21.15.89:58392) 2023-09-30T22:21:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79893e700 1 -- 172.21.15.89:0/4002483907 learned_addr learned my addr 172.21.15.89:0/4002483907 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc793fff700 1 --2- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 0x7fc7941c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79893e700 1 -- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc794135e50 msgr2=0x7fc794119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79893e700 1 --2- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc794135e50 0x7fc794119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79893e700 1 -- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 msgr2=0x7fc7941c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79893e700 1 --2- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 0x7fc7941c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.941+0000 7fc79893e700 1 -- 172.21.15.89:0/4002483907 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780009550 con 0x7fc79409d6f0 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.942+0000 7fc79893e700 1 --2- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79409d6f0 0x7fc7941c4360 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c450 tx=0x7fc78800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.942+0000 7fc7917fa700 1 -- 172.21.15.89:0/4002483907 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7880050d0 con 0x7fc79409d6f0 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.942+0000 7fc7917fa700 1 -- 172.21.15.89:0/4002483907 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788005230 con 0x7fc79409d6f0 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.942+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79411b160 con 0x7fc79409d6f0 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.942+0000 7fc7917fa700 1 -- 172.21.15.89:0/4002483907 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788003680 con 0x7fc79409d6f0 2023-09-30T22:21:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.942+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79409c700 con 0x7fc79409d6f0 2023-09-30T22:21:04.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.943+0000 7fc7917fa700 1 -- 172.21.15.89:0/4002483907 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7880037e0 con 0x7fc79409d6f0 2023-09-30T22:21:04.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.943+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc794074ea0 con 0x7fc79409d6f0 2023-09-30T22:21:04.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.943+0000 7fc7917fa700 1 -- 172.21.15.89:0/4002483907 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc78803ea00 con 0x7fc79409d6f0 2023-09-30T22:21:04.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:04.947+0000 7fc7917fa700 1 -- 172.21.15.89:0/4002483907 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc788011910 con 0x7fc79409d6f0 2023-09-30T22:21:05.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.038+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f82641323b0 con 0x7f826413c1e0 2023-09-30T22:21:05.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.038+0000 7f825affd700 1 -- 172.21.15.89:0/955872225 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f825c007920 con 0x7f826413c1e0 2023-09-30T22:21:05.038 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.041+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f826413c1e0 msgr2=0x7f826412e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.041+0000 7f826c2e4700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f826413c1e0 0x7f826412e290 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f825c009d90 tx=0x7f825c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.041+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 shutdown_connections 2023-09-30T22:21:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.041+0000 7f826c2e4700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f826413c1e0 0x7f826412e290 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.041+0000 7f826c2e4700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f826406a770 0x7f8264071e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.041+0000 7f826c2e4700 1 --2- 172.21.15.89:0/955872225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f826409dc30 0x7f826412dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.041+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 >> 172.21.15.89:0/955872225 conn(0x7f8264066f50 msgr2=0x7f826413df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.042+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 shutdown_connections 2023-09-30T22:21:05.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.042+0000 7f826c2e4700 1 -- 172.21.15.89:0/955872225 wait complete. 2023-09-30T22:21:05.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:05.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.123+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f52b411a470 con 0x7f52b409de50 2023-09-30T22:21:05.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.123+0000 7f52a9ffb700 1 -- 172.21.15.89:0/1513991054 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f52b0021030 con 0x7f52b409de50 2023-09-30T22:21:05.123 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:05.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b409de50 msgr2=0x7f52b411d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:05.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 --2- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b409de50 0x7f52b411d6a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f52b000e9b0 tx=0x7f52b000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 shutdown_connections 2023-09-30T22:21:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 --2- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52b41338a0 0x7f52b411e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 --2- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52b409de50 0x7f52b411d6a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 --2- 172.21.15.89:0/1513991054 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52b409e930 0x7f52b411dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 >> 172.21.15.89:0/1513991054 conn(0x7f52b40a2aa0 msgr2=0x7f52b41babf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 shutdown_connections 2023-09-30T22:21:05.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.125+0000 7f52bb364700 1 -- 172.21.15.89:0/1513991054 wait complete. 2023-09-30T22:21:05.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.139+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc79411c8b0 con 0x7fc79409d6f0 2023-09-30T22:21:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.139+0000 7fc7917fa700 1 -- 172.21.15.89:0/4002483907 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7880192c0 con 0x7fc79409d6f0 2023-09-30T22:21:05.139 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79409d6f0 msgr2=0x7fc7941c4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 --2- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79409d6f0 0x7fc7941c4360 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c450 tx=0x7fc78800c760 comp rx=0 tx=0).stop 2023-09-30T22:21:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 shutdown_connections 2023-09-30T22:21:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 --2- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc794135e50 0x7fc794119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 --2- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc79409d6f0 0x7fc7941c4360 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 --2- 172.21.15.89:0/4002483907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc79409e1d0 0x7fc7941c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 >> 172.21.15.89:0/4002483907 conn(0x7fc794066f50 msgr2=0x7fc7941327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 shutdown_connections 2023-09-30T22:21:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.142+0000 7fc79aba2700 1 -- 172.21.15.89:0/4002483907 wait complete. 2023-09-30T22:21:05.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.149+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3f54074ea0 con 0x7f3f5409d6f0 2023-09-30T22:21:05.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.150+0000 7f3f3bfff700 1 -- 172.21.15.89:0/3456239727 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f44017020 con 0x7f3f5409d6f0 2023-09-30T22:21:05.150 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f5409d6f0 msgr2=0x7f3f5411e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f5409d6f0 0x7f3f5411e930 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400c4e0 tx=0x7f3f4400c7f0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 shutdown_connections 2023-09-30T22:21:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f54139170 0x7f3f5411f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f5409d6f0 0x7f3f5411e930 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 --2- 172.21.15.89:0/3456239727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f5409e1d0 0x7f3f5411ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 >> 172.21.15.89:0/3456239727 conn(0x7f3f54066f50 msgr2=0x7f3f540a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 shutdown_connections 2023-09-30T22:21:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:05.152+0000 7f3f599f1700 1 -- 172.21.15.89:0/3456239727 wait complete. 2023-09-30T22:21:05.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:05.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:06.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:06.138 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:06.156 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:06.165 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.192+0000 7f36ed32c700 1 Processor -- start 2023-09-30T22:21:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.192+0000 7f36ed32c700 1 -- start start 2023-09-30T22:21:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36ed32c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36ed32c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36e80a1340 0x7f36e8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36ed32c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e809edd0 con 0x7f36e8133fa0 2023-09-30T22:21:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36ed32c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e809e1f0 con 0x7f36e80a1340 2023-09-30T22:21:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36ed32c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36e809dc50 con 0x7f36e80a0a30 2023-09-30T22:21:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36e67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36e80a1340 0x7f36e8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36e67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36e80a1340 0x7f36e8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34780/0 (socket says 172.21.15.89:34780) 2023-09-30T22:21:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36e67fc700 1 -- 172.21.15.89:0/2227186419 learned_addr learned my addr 172.21.15.89:0/2227186419 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36e6ffd700 1 --2- 172.21.15.89:0/2227186419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36e67fc700 1 -- 172.21.15.89:0/2227186419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 msgr2=0x7f36e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36e67fc700 1 --2- 172.21.15.89:0/2227186419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36e67fc700 1 -- 172.21.15.89:0/2227186419 >> v1:172.21.15.89:6789/0 conn(0x7f36e8133fa0 legacy=0x7f36e813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.193+0000 7f36e67fc700 1 -- 172.21.15.89:0/2227186419 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80a1fe0 con 0x7f36e80a1340 2023-09-30T22:21:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.194+0000 7f36e67fc700 1 --2- 172.21.15.89:0/2227186419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e80a1340 0x7f36e8133a60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0098f0 tx=0x7f36dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68a7deea27d4177 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.194+0000 7f36e5ffb700 1 -- 172.21.15.89:0/2227186419 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc011830 con 0x7f36e80a1340 2023-09-30T22:21:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.194+0000 7f36e5ffb700 1 -- 172.21.15.89:0/2227186419 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc011990 con 0x7f36e80a1340 2023-09-30T22:21:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.194+0000 7f36e5ffb700 1 -- 172.21.15.89:0/2227186419 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc011d20 con 0x7f36e80a1340 2023-09-30T22:21:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.195+0000 7f36ed32c700 1 -- 172.21.15.89:0/2227186419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e80a1340 msgr2=0x7f36e8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.195+0000 7f36ed32c700 1 --2- 172.21.15.89:0/2227186419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e80a1340 0x7f36e8133a60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0098f0 tx=0x7f36dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.195+0000 7f36ed32c700 1 -- 172.21.15.89:0/2227186419 shutdown_connections 2023-09-30T22:21:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.195+0000 7f36ed32c700 1 --2- 172.21.15.89:0/2227186419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.195+0000 7f36ed32c700 1 --2- 172.21.15.89:0/2227186419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e80a1340 0x7f36e8133a60 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.195+0000 7f36ed32c700 1 -- 172.21.15.89:0/2227186419 >> 172.21.15.89:0/2227186419 conn(0x7f36e80a6550 msgr2=0x7f36e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.195+0000 7f36ed32c700 1 -- 172.21.15.89:0/2227186419 shutdown_connections 2023-09-30T22:21:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.195+0000 7f36ed32c700 1 -- 172.21.15.89:0/2227186419 wait complete. 2023-09-30T22:21:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.196+0000 7f36ed32c700 1 Processor -- start 2023-09-30T22:21:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36ed32c700 1 -- start start 2023-09-30T22:21:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36ed32c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e811e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36ed32c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e80a1340 0x7f36e811ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36ed32c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36e8133fa0 0x7f36e811f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36ed32c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8120b70 con 0x7f36e80a1340 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36ed32c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8120420 con 0x7f36e8133fa0 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36ed32c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80a3850 con 0x7f36e80a0a30 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36e6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e811e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36e6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e811e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39768/0 (socket says 172.21.15.89:39768) 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36e6ffd700 1 -- 172.21.15.89:0/1982473866 learned_addr learned my addr 172.21.15.89:0/1982473866 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36e6ffd700 1 -- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36e8133fa0 msgr2=0x7f36e811f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36e6ffd700 1 --2- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36e8133fa0 0x7f36e811f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36e6ffd700 1 -- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e80a1340 msgr2=0x7f36e811ed20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36e6ffd700 1 --2- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e80a1340 0x7f36e811ed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.197+0000 7f36e6ffd700 1 -- 172.21.15.89:0/1982473866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc009550 con 0x7f36e80a0a30 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.198+0000 7f36e6ffd700 1 --2- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e811e7e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e9b0 tx=0x7f36d800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.198+0000 7f36cf7fe700 1 -- 172.21.15.89:0/1982473866 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8009710 con 0x7f36e80a0a30 2023-09-30T22:21:06.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.198+0000 7f36cf7fe700 1 -- 172.21.15.89:0/1982473866 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d800c850 con 0x7f36e80a0a30 2023-09-30T22:21:06.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.198+0000 7f36cf7fe700 1 -- 172.21.15.89:0/1982473866 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800cbc0 con 0x7f36e80a0a30 2023-09-30T22:21:06.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.198+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e80a0220 con 0x7f36e80a0a30 2023-09-30T22:21:06.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.198+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e81331b0 con 0x7f36e80a0a30 2023-09-30T22:21:06.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.199+0000 7f36cf7fe700 1 -- 172.21.15.89:0/1982473866 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f36d800cd20 con 0x7f36e80a0a30 2023-09-30T22:21:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.199+0000 7f36cf7fe700 1 -- 172.21.15.89:0/1982473866 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f36d803fbe0 con 0x7f36e80a0a30 2023-09-30T22:21:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.199+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c80052e0 con 0x7f36e80a0a30 2023-09-30T22:21:06.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.204+0000 7f36cf7fe700 1 -- 172.21.15.89:0/1982473866 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f36d8017020 con 0x7f36e80a0a30 2023-09-30T22:21:06.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.273+0000 7f5c2f119700 1 Processor -- start 2023-09-30T22:21:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.273+0000 7f5c2f119700 1 -- start start 2023-09-30T22:21:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.274+0000 7f5c2f119700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2806e570 0x7f5c2806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.274+0000 7f5c2f119700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2809eca0 0x7f5c2806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.274+0000 7f5c2f119700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c28070ee0 con 0x7f5c28134610 2023-09-30T22:21:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.274+0000 7f5c2f119700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28071480 con 0x7f5c2806e570 2023-09-30T22:21:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.274+0000 7f5c2f119700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2806f3c0 con 0x7f5c2809eca0 2023-09-30T22:21:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.274+0000 7f5c2ceb5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2806e570 0x7f5c2806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.274+0000 7f5c2ceb5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2806e570 0x7f5c2806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58404/0 (socket says 172.21.15.89:58404) 2023-09-30T22:21:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.274+0000 7f5c2ceb5700 1 -- 172.21.15.89:0/903403240 learned_addr learned my addr 172.21.15.89:0/903403240 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c27fff700 1 --2- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2809eca0 0x7f5c2806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c2ceb5700 1 -- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2809eca0 msgr2=0x7f5c2806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c2ceb5700 1 --2- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2809eca0 0x7f5c2806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c2ceb5700 1 -- 172.21.15.89:0/903403240 >> v1:172.21.15.89:6789/0 conn(0x7f5c28134610 legacy=0x7f5c28134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c277fe700 1 -- 172.21.15.89:0/903403240 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085479919 0 0) 0x7f5c28070ee0 con 0x7f5c28134610 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c2ceb5700 1 -- 172.21.15.89:0/903403240 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28071b90 con 0x7f5c2806e570 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c27fff700 1 --2- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2809eca0 0x7f5c2806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c2ceb5700 1 --2- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2806e570 0x7f5c2806e940 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009920 tx=0x7f5c1c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9c5148cccbb061 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c277fe700 1 -- 172.21.15.89:0/903403240 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00ba60 con 0x7f5c2806e570 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c277fe700 1 -- 172.21.15.89:0/903403240 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c00bbc0 con 0x7f5c2806e570 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.275+0000 7f5c277fe700 1 -- 172.21.15.89:0/903403240 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c009e70 con 0x7f5c2806e570 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.276+0000 7f5c2f119700 1 -- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2806e570 msgr2=0x7f5c2806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.276+0000 7f5c2f119700 1 --2- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2806e570 0x7f5c2806e940 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009920 tx=0x7f5c1c009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.276+0000 7f5c2f119700 1 -- 172.21.15.89:0/903403240 shutdown_connections 2023-09-30T22:21:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.276+0000 7f5c2f119700 1 --2- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c2806e570 0x7f5c2806e940 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.276+0000 7f5c2f119700 1 --2- 172.21.15.89:0/903403240 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2809eca0 0x7f5c2806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.276+0000 7f5c2f119700 1 -- 172.21.15.89:0/903403240 >> 172.21.15.89:0/903403240 conn(0x7f5c280a2aa0 msgr2=0x7f5c280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:06.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.276+0000 7f5c2f119700 1 -- 172.21.15.89:0/903403240 shutdown_connections 2023-09-30T22:21:06.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.276+0000 7f5c2f119700 1 -- 172.21.15.89:0/903403240 wait complete. 2023-09-30T22:21:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.277+0000 7f5c2f119700 1 Processor -- start 2023-09-30T22:21:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.277+0000 7f5c2f119700 1 -- start start 2023-09-30T22:21:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.277+0000 7f5c2f119700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2806e570 0x7f5c281214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.277+0000 7f5c2f119700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c28134610 0x7f5c28121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.277+0000 7f5c2f119700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28121f60 0x7f5c281d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.277+0000 7f5c2f119700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28134460 con 0x7f5c28121f60 2023-09-30T22:21:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.277+0000 7f5c2f119700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28124570 con 0x7f5c28134610 2023-09-30T22:21:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.277+0000 7f5c2f119700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280a2170 con 0x7f5c2806e570 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2806e570 0x7f5c281214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2806e570 0x7f5c281214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39792/0 (socket says 172.21.15.89:39792) 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 -- 172.21.15.89:0/3071880027 learned_addr learned my addr 172.21.15.89:0/3071880027 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2d6b6700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28121f60 0x7f5c281d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c27fff700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c28134610 0x7f5c28121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 -- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c28134610 msgr2=0x7f5c28121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c28134610 0x7f5c28121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 -- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28121f60 msgr2=0x7f5c281d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28121f60 0x7f5c281d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 -- 172.21.15.89:0/3071880027 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c009550 con 0x7f5c2806e570 2023-09-30T22:21:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c27fff700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c28134610 0x7f5c28121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c2ceb5700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2806e570 0x7f5c281214e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009c70 tx=0x7f5c1c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c257fa700 1 -- 172.21.15.89:0/3071880027 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c004590 con 0x7f5c2806e570 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c257fa700 1 -- 172.21.15.89:0/3071880027 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c01e070 con 0x7f5c2806e570 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.278+0000 7f5c257fa700 1 -- 172.21.15.89:0/3071880027 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0038c0 con 0x7f5c2806e570 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.279+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c28070610 con 0x7f5c2806e570 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.279+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c28123380 con 0x7f5c2806e570 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.280+0000 7f5c257fa700 1 -- 172.21.15.89:0/3071880027 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c1c003a20 con 0x7f5c2806e570 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.280+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2809b1e0 con 0x7f5c2806e570 2023-09-30T22:21:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.280+0000 7f5c257fa700 1 -- 172.21.15.89:0/3071880027 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5c1c020070 con 0x7f5c2806e570 2023-09-30T22:21:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.289+0000 7f5c257fa700 1 -- 172.21.15.89:0/3071880027 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c1c02a070 con 0x7f5c2806e570 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.314+0000 7f8c5ab2a700 1 Processor -- start 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.314+0000 7f8c5ab2a700 1 -- start start 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c5ab2a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5409e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c5ab2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c5ab2a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5409a810 con 0x7f8c5406a770 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c5ab2a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5406ab40 con 0x7f8c5413d1f0 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c5ab2a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5409ebc0 con 0x7f8c5409de00 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c53fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5409e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c53fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5409e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39802/0 (socket says 172.21.15.89:39802) 2023-09-30T22:21:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c53fff700 1 -- 172.21.15.89:0/760999727 learned_addr learned my addr 172.21.15.89:0/760999727 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c590c7700 1 --2- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c53fff700 1 -- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 msgr2=0x7f8c5413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:06.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c53fff700 1 --2- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5413f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c53fff700 1 -- 172.21.15.89:0/760999727 >> v1:172.21.15.89:6789/0 conn(0x7f8c5406a770 legacy=0x7f8c5409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.315+0000 7f8c537fe700 1 -- 172.21.15.89:0/760999727 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409813012 0 0) 0x7f8c5409a810 con 0x7f8c5406a770 2023-09-30T22:21:06.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c53fff700 1 -- 172.21.15.89:0/760999727 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5409e920 con 0x7f8c5409de00 2023-09-30T22:21:06.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c590c7700 1 --2- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:06.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c53fff700 1 --2- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5409e250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009920 tx=0x7f8c40009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26447c531b6b138b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c537fe700 1 -- 172.21.15.89:0/760999727 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40011830 con 0x7f8c5409de00 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c537fe700 1 -- 172.21.15.89:0/760999727 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c40011990 con 0x7f8c5409de00 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c537fe700 1 -- 172.21.15.89:0/760999727 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40011d20 con 0x7f8c5409de00 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 msgr2=0x7f8c5409e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.316+0000 7f8c5ab2a700 1 --2- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5409e250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009920 tx=0x7f8c40009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.317+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/760999727 shutdown_connections 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.317+0000 7f8c5ab2a700 1 --2- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.317+0000 7f8c5ab2a700 1 --2- 172.21.15.89:0/760999727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5409e250 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.317+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/760999727 >> 172.21.15.89:0/760999727 conn(0x7f8c54066f50 msgr2=0x7f8c5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.317+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/760999727 shutdown_connections 2023-09-30T22:21:06.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.317+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/760999727 wait complete. 2023-09-30T22:21:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.317+0000 7f37f76a2700 1 Processor -- start 2023-09-30T22:21:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.317+0000 7f37f76a2700 1 -- start start 2023-09-30T22:21:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f8c5ab2a700 1 Processor -- start 2023-09-30T22:21:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f76a2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006a770 0x7f37f006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f76a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006b080 0x7f37f0131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f76a2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f009ece0 con 0x7f37f0131b40 2023-09-30T22:21:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f76a2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f009e760 con 0x7f37f006a770 2023-09-30T22:21:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f76a2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37f009e040 con 0x7f37f006b080 2023-09-30T22:21:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f5c3f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f37f0131b40 0x7f37f0138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52100/0 (socket says 172.21.15.89:52100) 2023-09-30T22:21:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f5c3f700 1 -- 172.21.15.89:0/2912232163 learned_addr learned my addr 172.21.15.89:0/2912232163 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f4c3d700 1 --2- 172.21.15.89:0/2912232163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006b080 0x7f37f0131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37e7fff700 1 -- 172.21.15.89:0/2912232163 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879991849 0 0) 0x7f37f009ece0 con 0x7f37f0131b40 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37e7fff700 1 -- 172.21.15.89:0/2912232163 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37dc0035b0 con 0x7f37f0131b40 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f4c3d700 1 -- 172.21.15.89:0/2912232163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006a770 msgr2=0x7f37f006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f4c3d700 1 --2- 172.21.15.89:0/2912232163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006a770 0x7f37f006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f37f4c3d700 1 -- 172.21.15.89:0/2912232163 >> v1:172.21.15.89:6789/0 conn(0x7f37f0131b40 legacy=0x7f37f0138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f37f4c3d700 1 -- 172.21.15.89:0/2912232163 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f009f630 con 0x7f37f006b080 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.318+0000 7f8c5ab2a700 1 -- start start 2023-09-30T22:21:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c5ab2a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c5406a770 0x7f8c5411b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c5ab2a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5411bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c5ab2a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5411c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c5ab2a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5411a8a0 con 0x7f8c5406a770 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c5ab2a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5409f1a0 con 0x7f8c5413d1f0 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c5ab2a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54119ec0 con 0x7f8c5409de00 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5411c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5411c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58444/0 (socket says 172.21.15.89:58444) 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 -- 172.21.15.89:0/61421339 learned_addr learned my addr 172.21.15.89:0/61421339 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:06.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c588c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c5406a770 0x7f8c5411b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f37f4c3d700 1 --2- 172.21.15.89:0/2912232163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006b080 0x7f37f0131600 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009870 tx=0x7f37e0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2d3919474c4828 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c53fff700 1 --2- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5411bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 -- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 msgr2=0x7f8c5411bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 --2- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5411bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 -- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c5406a770 msgr2=0x7f8c5411b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 --2- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c5406a770 0x7f8c5411b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 -- 172.21.15.89:0/61421339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c009a70 con 0x7f8c5413d1f0 2023-09-30T22:21:06.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f37e7fff700 1 -- 172.21.15.89:0/2912232163 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0011830 con 0x7f37f006b080 2023-09-30T22:21:06.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f8c590c7700 1 --2- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5411c1e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00efc0 tx=0x7f8c4c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:06.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f37e7fff700 1 -- 172.21.15.89:0/2912232163 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e0011990 con 0x7f37f006b080 2023-09-30T22:21:06.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.319+0000 7f37e7fff700 1 -- 172.21.15.89:0/2912232163 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0011d20 con 0x7f37f006b080 2023-09-30T22:21:06.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.320+0000 7f8c517fa700 1 -- 172.21.15.89:0/61421339 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c00cb40 con 0x7f8c5413d1f0 2023-09-30T22:21:06.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.320+0000 7f8c517fa700 1 -- 172.21.15.89:0/61421339 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c00f040 con 0x7f8c5413d1f0 2023-09-30T22:21:06.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.320+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c40009550 con 0x7f8c5413d1f0 2023-09-30T22:21:06.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.320+0000 7f8c517fa700 1 -- 172.21.15.89:0/61421339 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c003a20 con 0x7f8c5413d1f0 2023-09-30T22:21:06.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.320+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5411a310 con 0x7f8c5413d1f0 2023-09-30T22:21:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.320+0000 7f37f76a2700 1 -- 172.21.15.89:0/2912232163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006b080 msgr2=0x7f37f0131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.320+0000 7f37f76a2700 1 --2- 172.21.15.89:0/2912232163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006b080 0x7f37f0131600 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009870 tx=0x7f37e0009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.321+0000 7f8c517fa700 1 -- 172.21.15.89:0/61421339 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c4c014480 con 0x7f8c5413d1f0 2023-09-30T22:21:06.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.321+0000 7f8c517fa700 1 -- 172.21.15.89:0/61421339 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8c4c03fff0 con 0x7f8c5413d1f0 2023-09-30T22:21:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.321+0000 7f37f76a2700 1 -- 172.21.15.89:0/2912232163 shutdown_connections 2023-09-30T22:21:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.321+0000 7f37f76a2700 1 --2- 172.21.15.89:0/2912232163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006a770 0x7f37f006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.321+0000 7f37f76a2700 1 --2- 172.21.15.89:0/2912232163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006b080 0x7f37f0131600 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.321+0000 7f37f76a2700 1 -- 172.21.15.89:0/2912232163 >> 172.21.15.89:0/2912232163 conn(0x7f37f0066f50 msgr2=0x7f37f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.322+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c54074ea0 con 0x7f8c5413d1f0 2023-09-30T22:21:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.322+0000 7f37f76a2700 1 -- 172.21.15.89:0/2912232163 shutdown_connections 2023-09-30T22:21:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.322+0000 7f37f76a2700 1 -- 172.21.15.89:0/2912232163 wait complete. 2023-09-30T22:21:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.323+0000 7f37f76a2700 1 Processor -- start 2023-09-30T22:21:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.323+0000 7f37f76a2700 1 -- start start 2023-09-30T22:21:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f76a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006a770 0x7f37f011e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f76a2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006b080 0x7f37f011eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f76a2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0131b40 0x7f37f011f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f76a2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01312f0 con 0x7f37f0131b40 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f76a2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f0071e60 con 0x7f37f006b080 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f76a2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37f011cd70 con 0x7f37f006a770 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f4c3d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006b080 0x7f37f011eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f4c3d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006b080 0x7f37f011eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58446/0 (socket says 172.21.15.89:58446) 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f4c3d700 1 -- 172.21.15.89:0/2121225651 learned_addr learned my addr 172.21.15.89:0/2121225651 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f543e700 1 --2- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006a770 0x7f37f011e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f543e700 1 -- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006b080 msgr2=0x7f37f011eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f543e700 1 --2- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006b080 0x7f37f011eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f543e700 1 -- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0131b40 msgr2=0x7f37f011f3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:21:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f543e700 1 --2- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0131b40 0x7f37f011f3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f543e700 1 -- 172.21.15.89:0/2121225651 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e00094a0 con 0x7f37f006a770 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.324+0000 7f37f543e700 1 --2- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006a770 0x7f37f011e970 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b480 tx=0x7f37ec00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.325+0000 7f37e5ffb700 1 -- 172.21.15.89:0/2121225651 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011830 con 0x7f37f006a770 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.325+0000 7f37e5ffb700 1 -- 172.21.15.89:0/2121225651 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec011990 con 0x7f37f006a770 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.325+0000 7f37e5ffb700 1 -- 172.21.15.89:0/2121225651 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011d40 con 0x7f37f006a770 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.325+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f006ecb0 con 0x7f37f006a770 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.325+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f01206c0 con 0x7f37f006a770 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.326+0000 7f37e5ffb700 1 -- 172.21.15.89:0/2121225651 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f37ec01a400 con 0x7f37f006a770 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.326+0000 7f37e5ffb700 1 -- 172.21.15.89:0/2121225651 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f37ec03f5b0 con 0x7f37f006a770 2023-09-30T22:21:06.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.327+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f006ecb0 con 0x7f37f006a770 2023-09-30T22:21:06.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.328+0000 7f8c517fa700 1 -- 172.21.15.89:0/61421339 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c4c012930 con 0x7f8c5413d1f0 2023-09-30T22:21:06.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.331+0000 7f37e5ffb700 1 -- 172.21.15.89:0/2121225651 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f37f006ecb0 con 0x7f37f006a770 2023-09-30T22:21:06.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.472+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f36c8005150 con 0x7f36e80a0a30 2023-09-30T22:21:06.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.472+0000 7f36cf7fe700 1 -- 172.21.15.89:0/1982473866 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36d802b090 con 0x7f36e80a0a30 2023-09-30T22:21:06.472 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:06.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.475+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 msgr2=0x7f36e811e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.475+0000 7f36ed32c700 1 --2- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e811e7e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e9b0 tx=0x7f36d800ed70 comp rx=0 tx=0).stop 2023-09-30T22:21:06.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.475+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 shutdown_connections 2023-09-30T22:21:06.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.475+0000 7f36ed32c700 1 --2- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36e8133fa0 0x7f36e811f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.475+0000 7f36ed32c700 1 --2- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f36e80a0a30 0x7f36e811e7e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.475+0000 7f36ed32c700 1 --2- 172.21.15.89:0/1982473866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e80a1340 0x7f36e811ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.475+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 >> 172.21.15.89:0/1982473866 conn(0x7f36e80a6550 msgr2=0x7f36e80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:06.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.476+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 shutdown_connections 2023-09-30T22:21:06.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.476+0000 7f36ed32c700 1 -- 172.21.15.89:0/1982473866 wait complete. 2023-09-30T22:21:06.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:06.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.493+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5c28074ea0 con 0x7f5c2806e570 2023-09-30T22:21:06.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.493+0000 7f5c257fa700 1 -- 172.21.15.89:0/3071880027 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c1c03f030 con 0x7f5c2806e570 2023-09-30T22:21:06.493 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:06.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2806e570 msgr2=0x7f5c281214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2806e570 0x7f5c281214e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009c70 tx=0x7f5c1c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:06.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 shutdown_connections 2023-09-30T22:21:06.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28121f60 0x7f5c281d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c2806e570 0x7f5c281214e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 --2- 172.21.15.89:0/3071880027 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c28134610 0x7f5c28121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 >> 172.21.15.89:0/3071880027 conn(0x7f5c280a2aa0 msgr2=0x7f5c281bf110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:06.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 shutdown_connections 2023-09-30T22:21:06.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.495+0000 7f5c2f119700 1 -- 172.21.15.89:0/3071880027 wait complete. 2023-09-30T22:21:06.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:06.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.534+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f37f011d9d0 con 0x7f37f006a770 2023-09-30T22:21:06.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.534+0000 7f37e5ffb700 1 -- 172.21.15.89:0/2121225651 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f37f011d9d0 con 0x7f37f006a770 2023-09-30T22:21:06.534 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006a770 msgr2=0x7f37f011e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 --2- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006a770 0x7f37f011e970 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b480 tx=0x7f37ec00b790 comp rx=0 tx=0).stop 2023-09-30T22:21:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 shutdown_connections 2023-09-30T22:21:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 --2- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0131b40 0x7f37f011f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 --2- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37f006a770 0x7f37f011e970 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 --2- 172.21.15.89:0/2121225651 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f37f006b080 0x7f37f011eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 >> 172.21.15.89:0/2121225651 conn(0x7f37f0066f50 msgr2=0x7f37f006ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 shutdown_connections 2023-09-30T22:21:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.536+0000 7f37f76a2700 1 -- 172.21.15.89:0/2121225651 wait complete. 2023-09-30T22:21:06.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:06.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.599+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8c5409a3d0 con 0x7f8c5413d1f0 2023-09-30T22:21:06.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.599+0000 7f8c517fa700 1 -- 172.21.15.89:0/61421339 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c4c012340 con 0x7f8c5413d1f0 2023-09-30T22:21:06.599 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:06.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 msgr2=0x7f8c5411c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:06.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 --2- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5411c1e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00efc0 tx=0x7f8c4c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 shutdown_connections 2023-09-30T22:21:06.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 --2- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c5413d1f0 0x7f8c5411c1e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 --2- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c5406a770 0x7f8c5411b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 --2- 172.21.15.89:0/61421339 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8c5409de00 0x7f8c5411bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:06.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 >> 172.21.15.89:0/61421339 conn(0x7f8c54066f50 msgr2=0x7f8c5413ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:06.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 shutdown_connections 2023-09-30T22:21:06.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:06.601+0000 7f8c5ab2a700 1 -- 172.21.15.89:0/61421339 wait complete. 2023-09-30T22:21:06.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:07.492 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:07.509 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:07.549 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:07.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.636+0000 7f53f7664700 1 Processor -- start 2023-09-30T22:21:07.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.637+0000 7f53f7664700 1 -- start start 2023-09-30T22:21:07.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.637+0000 7f53f7664700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.637+0000 7f53f7664700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.637+0000 7f53f7664700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53f009e410 0x7f53f0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.637+0000 7f53f7664700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f006b460 con 0x7f53f009e410 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.637+0000 7f53f7664700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f009b120 con 0x7f53f009d6f0 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.637+0000 7f53f7664700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f009a7e0 con 0x7f53f009e000 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58456/0 (socket says 172.21.15.89:58456) 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 -- 172.21.15.89:0/2151549076 learned_addr learned my addr 172.21.15.89:0/2151549076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f4bff700 1 --2- 172.21.15.89:0/2151549076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5c01700 1 --2- 172.21.15.89:0/2151549076 >> v2:172.21.15.89:3300/0 conn(0x7f53f009e410 0x7f53f0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 -- 172.21.15.89:0/2151549076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 msgr2=0x7f53f01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 --2- 172.21.15.89:0/2151549076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 -- 172.21.15.89:0/2151549076 >> v2:172.21.15.89:3300/0 conn(0x7f53f009e410 msgr2=0x7f53f0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:07.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 --2- 172.21.15.89:0/2151549076 >> v2:172.21.15.89:3300/0 conn(0x7f53f009e410 0x7f53f0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 -- 172.21.15.89:0/2151549076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f009a500 con 0x7f53f009d6f0 2023-09-30T22:21:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5c01700 1 --2- 172.21.15.89:0/2151549076 >> v2:172.21.15.89:3300/0 conn(0x7f53f009e410 0x7f53f0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53f5400700 1 --2- 172.21.15.89:0/2151549076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f009dac0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009920 tx=0x7f53ec009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a6b42ed8669c26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53e7fff700 1 -- 172.21.15.89:0/2151549076 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec011830 con 0x7f53f009d6f0 2023-09-30T22:21:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.638+0000 7f53e7fff700 1 -- 172.21.15.89:0/2151549076 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec011990 con 0x7f53f009d6f0 2023-09-30T22:21:07.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53e7fff700 1 -- 172.21.15.89:0/2151549076 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec011d20 con 0x7f53f009d6f0 2023-09-30T22:21:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 -- 172.21.15.89:0/2151549076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 msgr2=0x7f53f009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 --2- 172.21.15.89:0/2151549076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f009dac0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009920 tx=0x7f53ec009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 -- 172.21.15.89:0/2151549076 shutdown_connections 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 --2- 172.21.15.89:0/2151549076 >> v2:172.21.15.89:3300/0 conn(0x7f53f009e410 0x7f53f0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 --2- 172.21.15.89:0/2151549076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f009dac0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 --2- 172.21.15.89:0/2151549076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 -- 172.21.15.89:0/2151549076 >> 172.21.15.89:0/2151549076 conn(0x7f53f0066f50 msgr2=0x7f53f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 -- 172.21.15.89:0/2151549076 shutdown_connections 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.639+0000 7f53f7664700 1 -- 172.21.15.89:0/2151549076 wait complete. 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.640+0000 7f53f7664700 1 Processor -- start 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.640+0000 7f53f7664700 1 -- start start 2023-09-30T22:21:07.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f7664700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f01a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f7664700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f7664700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f009e410 0x7f53f01a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f7664700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01a9840 con 0x7f53f009e410 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f7664700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01a8c10 con 0x7f53f009d6f0 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f7664700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01a7f10 con 0x7f53f009e000 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f5400700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f01a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f4bff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f5400700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f01a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58458/0 (socket says 172.21.15.89:58458) 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f5400700 1 -- 172.21.15.89:0/2744903703 learned_addr learned my addr 172.21.15.89:0/2744903703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f5c01700 1 --2- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f009e410 0x7f53f01a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f4bff700 1 -- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 msgr2=0x7f53f01a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f4bff700 1 --2- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f01a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f4bff700 1 -- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f009e410 msgr2=0x7f53f01a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f4bff700 1 --2- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f009e410 0x7f53f01a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.641+0000 7f53f4bff700 1 -- 172.21.15.89:0/2744903703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec009550 con 0x7f53f009e000 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.642+0000 7f53f4bff700 1 --2- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01a6260 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e8d0 tx=0x7f53e000ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.642+0000 7f53e5ffb700 1 -- 172.21.15.89:0/2744903703 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e000c690 con 0x7f53f009e000 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.642+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f01a8fd0 con 0x7f53f009e000 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.642+0000 7f53e5ffb700 1 -- 172.21.15.89:0/2744903703 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e000c7f0 con 0x7f53f009e000 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.642+0000 7f53e5ffb700 1 -- 172.21.15.89:0/2744903703 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e000cb60 con 0x7f53f009e000 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.642+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f00a3140 con 0x7f53f009e000 2023-09-30T22:21:07.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.643+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f01a93e0 con 0x7f53f009e000 2023-09-30T22:21:07.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.644+0000 7f53e5ffb700 1 -- 172.21.15.89:0/2744903703 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53e000ccc0 con 0x7f53f009e000 2023-09-30T22:21:07.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.644+0000 7f53e5ffb700 1 -- 172.21.15.89:0/2744903703 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f53e003ff10 con 0x7f53f009e000 2023-09-30T22:21:07.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.652+0000 7f53e5ffb700 1 -- 172.21.15.89:0/2744903703 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53e001a360 con 0x7f53f009e000 2023-09-30T22:21:07.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bd62b8700 1 Processor -- start 2023-09-30T22:21:07.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bd62b8700 1 -- start start 2023-09-30T22:21:07.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bd62b8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bd62b8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4bd0135c20 0x7f4bd0138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bd62b8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd006fa40 con 0x7f4bd012cf60 2023-09-30T22:21:07.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bd62b8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd012cd90 con 0x7f4bd0135c20 2023-09-30T22:21:07.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bd62b8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0071780 con 0x7f4bd006f5d0 2023-09-30T22:21:07.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bcf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.673+0000 7f4bd4855700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4bd0135c20 0x7f4bd0138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bcf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd012ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58466/0 (socket says 172.21.15.89:58466) 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bcf7fe700 1 -- 172.21.15.89:0/506260293 learned_addr learned my addr 172.21.15.89:0/506260293 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bcffff700 1 --1- 172.21.15.89:0/506260293 >> v1:172.21.15.89:6789/0 conn(0x7f4bd012cf60 0x7f4bd012f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52110/0 (socket says 172.21.15.89:52110) 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bd4855700 1 -- 172.21.15.89:0/506260293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 msgr2=0x7f4bd012ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bd4855700 1 --2- 172.21.15.89:0/506260293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd012ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bd4855700 1 -- 172.21.15.89:0/506260293 >> v1:172.21.15.89:6789/0 conn(0x7f4bd012cf60 legacy=0x7f4bd012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bd4855700 1 -- 172.21.15.89:0/506260293 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0071a40 con 0x7f4bd0135c20 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bceffd700 1 -- 172.21.15.89:0/506260293 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669814573 0 0) 0x7f4bd006fa40 con 0x7f4bd012cf60 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bcf7fe700 1 --2- 172.21.15.89:0/506260293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bd4855700 1 --2- 172.21.15.89:0/506260293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd0138000 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009d90 tx=0x7f4bc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c5c045068fab9d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bceffd700 1 -- 172.21.15.89:0/506260293 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800e070 con 0x7f4bd0135c20 2023-09-30T22:21:07.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bceffd700 1 -- 172.21.15.89:0/506260293 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc800bb20 con 0x7f4bd0135c20 2023-09-30T22:21:07.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.674+0000 7f4bceffd700 1 -- 172.21.15.89:0/506260293 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8012430 con 0x7f4bd0135c20 2023-09-30T22:21:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.675+0000 7f4bd62b8700 1 -- 172.21.15.89:0/506260293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 msgr2=0x7f4bd0138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.675+0000 7f4bd62b8700 1 --2- 172.21.15.89:0/506260293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd0138000 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009d90 tx=0x7f4bc800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.675+0000 7f4bd62b8700 1 -- 172.21.15.89:0/506260293 shutdown_connections 2023-09-30T22:21:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.675+0000 7f4bd62b8700 1 --2- 172.21.15.89:0/506260293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd0138000 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.675+0000 7f4bd62b8700 1 --2- 172.21.15.89:0/506260293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.675+0000 7f4bd62b8700 1 -- 172.21.15.89:0/506260293 >> 172.21.15.89:0/506260293 conn(0x7f4bd00a2aa0 msgr2=0x7f4bd00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.675+0000 7f4bd62b8700 1 -- 172.21.15.89:0/506260293 shutdown_connections 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.675+0000 7f4bd62b8700 1 -- 172.21.15.89:0/506260293 wait complete. 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.676+0000 7f4bd62b8700 1 Processor -- start 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.676+0000 7f4bd62b8700 1 -- start start 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd62b8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd01c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd62b8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bd012cf60 0x7f4bd01c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd62b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd01c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd62b8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0134b90 con 0x7f4bd0135c20 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd62b8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0129fa0 con 0x7f4bd006f5d0 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd62b8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0132d60 con 0x7f4bd012cf60 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd4855700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd01c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd4855700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd01c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34828/0 (socket says 172.21.15.89:34828) 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd4855700 1 -- 172.21.15.89:0/827668073 learned_addr learned my addr 172.21.15.89:0/827668073 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bcf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bd012cf60 0x7f4bd01c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bcffff700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd01c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd4855700 1 -- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bd012cf60 msgr2=0x7f4bd01c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd4855700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bd012cf60 0x7f4bd01c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd4855700 1 -- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 msgr2=0x7f4bd01c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd4855700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd01c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bd4855700 1 -- 172.21.15.89:0/827668073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc400a6f0 con 0x7f4bd0135c20 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.677+0000 7f4bcf7fe700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bd012cf60 0x7f4bd01c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.678+0000 7f4bd4855700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd01c8d20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8013000 tx=0x7f4bc8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:07.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.678+0000 7f4bccff9700 1 -- 172.21.15.89:0/827668073 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8012430 con 0x7f4bd0135c20 2023-09-30T22:21:07.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.678+0000 7f4bccff9700 1 -- 172.21.15.89:0/827668073 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8020070 con 0x7f4bd0135c20 2023-09-30T22:21:07.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.678+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc8009a70 con 0x7f4bd0135c20 2023-09-30T22:21:07.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.678+0000 7f4bccff9700 1 -- 172.21.15.89:0/827668073 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8004280 con 0x7f4bd0135c20 2023-09-30T22:21:07.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.678+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd009ecc0 con 0x7f4bd0135c20 2023-09-30T22:21:07.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.679+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd0074ea0 con 0x7f4bd0135c20 2023-09-30T22:21:07.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.679+0000 7f4bccff9700 1 -- 172.21.15.89:0/827668073 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bc80043e0 con 0x7f4bd0135c20 2023-09-30T22:21:07.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.680+0000 7f4bccff9700 1 -- 172.21.15.89:0/827668073 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4bc8022070 con 0x7f4bd0135c20 2023-09-30T22:21:07.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.684+0000 7f4bccff9700 1 -- 172.21.15.89:0/827668073 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bc8031430 con 0x7f4bd0135c20 2023-09-30T22:21:07.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.716+0000 7f5a9f38c700 1 Processor -- start 2023-09-30T22:21:07.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.717+0000 7f5a9f38c700 1 -- start start 2023-09-30T22:21:07.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.717+0000 7f5a9f38c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a9809f410 0x7f5a98136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.717+0000 7f5a9f38c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a98136fe0 0x7f5a981393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.717+0000 7f5a9f38c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a98070940 con 0x7f5a9809e930 2023-09-30T22:21:07.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.717+0000 7f5a9f38c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9806fd60 con 0x7f5a9809f410 2023-09-30T22:21:07.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.717+0000 7f5a9f38c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9806fa80 con 0x7f5a98136fe0 2023-09-30T22:21:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9c927700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a9809f410 0x7f5a98136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9d929700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a98136fe0 0x7f5a981393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9c927700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a9809f410 0x7f5a98136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34844/0 (socket says 172.21.15.89:34844) 2023-09-30T22:21:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9d929700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a98136fe0 0x7f5a981393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39856/0 (socket says 172.21.15.89:39856) 2023-09-30T22:21:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9c927700 1 -- 172.21.15.89:0/3290736531 learned_addr learned my addr 172.21.15.89:0/3290736531 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9c927700 1 -- 172.21.15.89:0/3290736531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a98136fe0 msgr2=0x7f5a981393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9c927700 1 --2- 172.21.15.89:0/3290736531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a98136fe0 0x7f5a981393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9c927700 1 -- 172.21.15.89:0/3290736531 >> v1:172.21.15.89:6789/0 conn(0x7f5a9809e930 legacy=0x7f5a9809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9c927700 1 -- 172.21.15.89:0/3290736531 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9809fb60 con 0x7f5a9809f410 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9d929700 1 --2- 172.21.15.89:0/3290736531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a98136fe0 0x7f5a981393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a9c927700 1 --2- 172.21.15.89:0/3290736531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9809f410 0x7f5a98136aa0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5a880098f0 tx=0x7f5a88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ff681fc41fe439 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a8ffff700 1 -- 172.21.15.89:0/3290736531 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88010a90 con 0x7f5a9809f410 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a8ffff700 1 -- 172.21.15.89:0/3290736531 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a88010bf0 con 0x7f5a9809f410 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.718+0000 7f5a8ffff700 1 -- 172.21.15.89:0/3290736531 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88018440 con 0x7f5a9809f410 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.719+0000 7f5a9f38c700 1 -- 172.21.15.89:0/3290736531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9809f410 msgr2=0x7f5a98136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.719+0000 7f5a9f38c700 1 --2- 172.21.15.89:0/3290736531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9809f410 0x7f5a98136aa0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5a880098f0 tx=0x7f5a88009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.719+0000 7f5a9f38c700 1 -- 172.21.15.89:0/3290736531 shutdown_connections 2023-09-30T22:21:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.719+0000 7f5a9f38c700 1 --2- 172.21.15.89:0/3290736531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a98136fe0 0x7f5a981393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.719+0000 7f5a9f38c700 1 --2- 172.21.15.89:0/3290736531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9809f410 0x7f5a98136aa0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.719+0000 7f5a9f38c700 1 -- 172.21.15.89:0/3290736531 >> 172.21.15.89:0/3290736531 conn(0x7f5a980a4210 msgr2=0x7f5a980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.719+0000 7f5a9f38c700 1 -- 172.21.15.89:0/3290736531 shutdown_connections 2023-09-30T22:21:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.719+0000 7f5a9f38c700 1 -- 172.21.15.89:0/3290736531 wait complete. 2023-09-30T22:21:07.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.720+0000 7f5a9f38c700 1 Processor -- start 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9f38c700 1 -- start start 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9f38c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9809e930 0x7f5a98134de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9f38c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9809f410 0x7f5a98135320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9f38c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a98136fe0 0x7f5a98135860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9f38c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9811bcc0 con 0x7f5a98136fe0 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9f38c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9812fd40 con 0x7f5a9809f410 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9f38c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9811bfa0 con 0x7f5a9809e930 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9d128700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9809e930 0x7f5a98134de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9d128700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9809e930 0x7f5a98134de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39872/0 (socket says 172.21.15.89:39872) 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9d128700 1 -- 172.21.15.89:0/1022167208 learned_addr learned my addr 172.21.15.89:0/1022167208 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.721+0000 7f5a9d929700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a98136fe0 0x7f5a98135860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9c927700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9809f410 0x7f5a98135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9d128700 1 -- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9809f410 msgr2=0x7f5a98135320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9d128700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9809f410 0x7f5a98135320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9d128700 1 -- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a98136fe0 msgr2=0x7f5a98135860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9d128700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a98136fe0 0x7f5a98135860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9d128700 1 -- 172.21.15.89:0/1022167208 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a90009a70 con 0x7f5a9809e930 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9c927700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9809f410 0x7f5a98135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9d929700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a98136fe0 0x7f5a98135860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9d128700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9809e930 0x7f5a98134de0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c510 tx=0x7f5a9400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a8dffb700 1 -- 172.21.15.89:0/1022167208 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940052d0 con 0x7f5a9809e930 2023-09-30T22:21:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a8dffb700 1 -- 172.21.15.89:0/1022167208 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a94005430 con 0x7f5a9809e930 2023-09-30T22:21:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a88009550 con 0x7f5a9809e930 2023-09-30T22:21:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a8dffb700 1 -- 172.21.15.89:0/1022167208 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940038c0 con 0x7f5a9809e930 2023-09-30T22:21:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.722+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9811ba00 con 0x7f5a9809e930 2023-09-30T22:21:07.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.723+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9811b7f0 con 0x7f5a9809e930 2023-09-30T22:21:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.724+0000 7f5a8dffb700 1 -- 172.21.15.89:0/1022167208 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a94003a20 con 0x7f5a9809e930 2023-09-30T22:21:07.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.724+0000 7f5a8dffb700 1 -- 172.21.15.89:0/1022167208 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5a9403eb00 con 0x7f5a9809e930 2023-09-30T22:21:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.732+0000 7f5a8dffb700 1 -- 172.21.15.89:0/1022167208 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a94019e60 con 0x7f5a9809e930 2023-09-30T22:21:07.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.759+0000 7fcc87c86700 1 Processor -- start 2023-09-30T22:21:07.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.759+0000 7fcc87c86700 1 -- start start 2023-09-30T22:21:07.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc87c86700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc87c86700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc87c86700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e410 0x7fcc80130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc87c86700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8009f520 con 0x7fcc8009e000 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc87c86700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8009f130 con 0x7fcc8009e410 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc87c86700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8009e930 con 0x7fcc8009d6f0 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc85221700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc85221700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34868/0 (socket says 172.21.15.89:34868) 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc85221700 1 -- 172.21.15.89:0/2087861461 learned_addr learned my addr 172.21.15.89:0/2087861461 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc85a22700 1 --2- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.760+0000 7fcc86223700 1 --2- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e410 0x7fcc80130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc86223700 1 -- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009d6f0 msgr2=0x7fcc8009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc86223700 1 --2- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc86223700 1 -- 172.21.15.89:0/2087861461 >> v2:172.21.15.89:3300/0 conn(0x7fcc8009e000 msgr2=0x7fcc80130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc86223700 1 --2- 172.21.15.89:0/2087861461 >> v2:172.21.15.89:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc86223700 1 -- 172.21.15.89:0/2087861461 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8009b2d0 con 0x7fcc8009e410 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc85221700 1 --2- 172.21.15.89:0/2087861461 >> v2:172.21.15.89:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc86223700 1 --2- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e410 0x7fcc80130ba0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800f370 tx=0x7fcc7800f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a8011ebb269b9c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc84a20700 1 -- 172.21.15.89:0/2087861461 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78012070 con 0x7fcc8009e410 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc84a20700 1 -- 172.21.15.89:0/2087861461 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7800fe70 con 0x7fcc8009e410 2023-09-30T22:21:07.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc84a20700 1 -- 172.21.15.89:0/2087861461 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800d2f0 con 0x7fcc8009e410 2023-09-30T22:21:07.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc87c86700 1 -- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e410 msgr2=0x7fcc80130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.761+0000 7fcc87c86700 1 --2- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e410 0x7fcc80130ba0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800f370 tx=0x7fcc7800f680 comp rx=0 tx=0).stop 2023-09-30T22:21:07.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.762+0000 7fcc87c86700 1 -- 172.21.15.89:0/2087861461 shutdown_connections 2023-09-30T22:21:07.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.762+0000 7fcc87c86700 1 --2- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e410 0x7fcc80130ba0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.762+0000 7fcc87c86700 1 --2- 172.21.15.89:0/2087861461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.762+0000 7fcc87c86700 1 --2- 172.21.15.89:0/2087861461 >> v2:172.21.15.89:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.762+0000 7fcc87c86700 1 -- 172.21.15.89:0/2087861461 >> 172.21.15.89:0/2087861461 conn(0x7fcc80066f50 msgr2=0x7fcc8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.762+0000 7fcc87c86700 1 -- 172.21.15.89:0/2087861461 shutdown_connections 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.762+0000 7fcc87c86700 1 -- 172.21.15.89:0/2087861461 wait complete. 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.763+0000 7fcc87c86700 1 Processor -- start 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.763+0000 7fcc87c86700 1 -- start start 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc87c86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc8009d6f0 0x7fcc8011de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc87c86700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e000 0x7fcc8011e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc87c86700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009e410 0x7fcc8011e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:07.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc87c86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80122270 con 0x7fcc8009d6f0 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc87c86700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801218b0 con 0x7fcc8009e000 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc87c86700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80120bf0 con 0x7fcc8009e410 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85221700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e000 0x7fcc8011e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85221700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e000 0x7fcc8011e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58508/0 (socket says 172.21.15.89:58508) 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85221700 1 -- 172.21.15.89:0/3443313869 learned_addr learned my addr 172.21.15.89:0/3443313869 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc86223700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009e410 0x7fcc8011e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85a22700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc8009d6f0 0x7fcc8011de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85221700 1 -- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009e410 msgr2=0x7fcc8011e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85221700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009e410 0x7fcc8011e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85221700 1 -- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc8009d6f0 msgr2=0x7fcc8011de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85221700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc8009d6f0 0x7fcc8011de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85221700 1 -- 172.21.15.89:0/3443313869 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7000b600 con 0x7fcc8009e000 2023-09-30T22:21:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc86223700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009e410 0x7fcc8011e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.764+0000 7fcc85a22700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc8009d6f0 0x7fcc8011de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.765+0000 7fcc85221700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e000 0x7fcc8011e360 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000fe10 tx=0x7fcc7000d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.765+0000 7fcc767fc700 1 -- 172.21.15.89:0/3443313869 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70018640 con 0x7fcc8009e000 2023-09-30T22:21:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.765+0000 7fcc767fc700 1 -- 172.21.15.89:0/3443313869 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc700187a0 con 0x7fcc8009e000 2023-09-30T22:21:07.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.765+0000 7fcc767fc700 1 -- 172.21.15.89:0/3443313869 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70018b50 con 0x7fcc8009e000 2023-09-30T22:21:07.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.765+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7800f050 con 0x7fcc8009e000 2023-09-30T22:21:07.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.765+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc800a16a0 con 0x7fcc8009e000 2023-09-30T22:21:07.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.766+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc80074ea0 con 0x7fcc8009e000 2023-09-30T22:21:07.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.766+0000 7fcc767fc700 1 -- 172.21.15.89:0/3443313869 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc70018cb0 con 0x7fcc8009e000 2023-09-30T22:21:07.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.767+0000 7fcc767fc700 1 -- 172.21.15.89:0/3443313869 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcc7003ff10 con 0x7fcc8009e000 2023-09-30T22:21:07.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.775+0000 7fcc767fc700 1 -- 172.21.15.89:0/3443313869 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc70010440 con 0x7fcc8009e000 2023-09-30T22:21:07.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.854+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f53f0074ea0 con 0x7f53f009e000 2023-09-30T22:21:07.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.855+0000 7f53e5ffb700 1 -- 172.21.15.89:0/2744903703 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53e0021020 con 0x7f53f009e000 2023-09-30T22:21:07.855 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:07.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 msgr2=0x7f53f01a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 --2- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01a6260 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e8d0 tx=0x7f53e000ebe0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 shutdown_connections 2023-09-30T22:21:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 --2- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f009e410 0x7f53f01a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 --2- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f009d6f0 0x7f53f01a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 --2- 172.21.15.89:0/2744903703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f009e000 0x7f53f01a6260 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 >> 172.21.15.89:0/2744903703 conn(0x7f53f0066f50 msgr2=0x7f53f00a3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 shutdown_connections 2023-09-30T22:21:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.858+0000 7f53f7664700 1 -- 172.21.15.89:0/2744903703 wait complete. 2023-09-30T22:21:07.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:07.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.881+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4bd00713a0 con 0x7f4bd0135c20 2023-09-30T22:21:07.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.882+0000 7f4bccff9700 1 -- 172.21.15.89:0/827668073 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bc8035090 con 0x7f4bd0135c20 2023-09-30T22:21:07.881 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 msgr2=0x7f4bd01c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd01c8d20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8013000 tx=0x7f4bc8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 shutdown_connections 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bd0135c20 0x7f4bd01c8d20 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bd006f5d0 0x7f4bd01c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 --2- 172.21.15.89:0/827668073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bd012cf60 0x7f4bd01c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 >> 172.21.15.89:0/827668073 conn(0x7f4bd00a2aa0 msgr2=0x7f4bd006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 shutdown_connections 2023-09-30T22:21:07.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.884+0000 7f4bd62b8700 1 -- 172.21.15.89:0/827668073 wait complete. 2023-09-30T22:21:07.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:07.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.933+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5a98074ea0 con 0x7f5a9809e930 2023-09-30T22:21:07.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.933+0000 7f5a8dffb700 1 -- 172.21.15.89:0/1022167208 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a94017030 con 0x7f5a9809e930 2023-09-30T22:21:07.934 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:07.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9809e930 msgr2=0x7f5a98134de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9809e930 0x7f5a98134de0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c510 tx=0x7f5a9400c8d0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 shutdown_connections 2023-09-30T22:21:07.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a98136fe0 0x7f5a98135860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9809e930 0x7f5a98134de0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 --2- 172.21.15.89:0/1022167208 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9809f410 0x7f5a98135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 >> 172.21.15.89:0/1022167208 conn(0x7f5a980a4210 msgr2=0x7f5a980a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:07.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 shutdown_connections 2023-09-30T22:21:07.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.935+0000 7f5a9f38c700 1 -- 172.21.15.89:0/1022167208 wait complete. 2023-09-30T22:21:07.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:07.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.977+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcc8006c360 con 0x7fcc8009e000 2023-09-30T22:21:07.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.977+0000 7fcc767fc700 1 -- 172.21.15.89:0/3443313869 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc700173a0 con 0x7fcc8009e000 2023-09-30T22:21:07.978 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e000 msgr2=0x7fcc8011e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e000 0x7fcc8011e360 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000fe10 tx=0x7fcc7000d630 comp rx=0 tx=0).stop 2023-09-30T22:21:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 shutdown_connections 2023-09-30T22:21:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc8009e410 0x7fcc8011e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc8009d6f0 0x7fcc8011de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 --2- 172.21.15.89:0/3443313869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc8009e000 0x7fcc8011e360 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 >> 172.21.15.89:0/3443313869 conn(0x7fcc80066f50 msgr2=0x7fcc80070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 shutdown_connections 2023-09-30T22:21:07.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:07.979+0000 7fcc87c86700 1 -- 172.21.15.89:0/3443313869 wait complete. 2023-09-30T22:21:07.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:08.875 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:08.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:08.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:08.992 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.007+0000 7f029041a700 1 Processor -- start 2023-09-30T22:21:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.007+0000 7f029041a700 1 -- start start 2023-09-30T22:21:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.007+0000 7f029041a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f0288135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.007+0000 7f029041a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0288135e60 0x7f0288138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.007+0000 7f029041a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02880716a0 con 0x7f0288130540 2023-09-30T22:21:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.007+0000 7f029041a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0288071290 con 0x7f0288135e60 2023-09-30T22:21:09.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.007+0000 7f029041a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0288130340 con 0x7f0288133540 2023-09-30T22:21:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d9b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f0288135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028e9b7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0288135e60 0x7f0288138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028e1b6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0288130540 0x7f0288130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52118/0 (socket says 172.21.15.89:52118) 2023-09-30T22:21:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028e1b6700 1 -- 172.21.15.89:0/2909878793 learned_addr learned my addr 172.21.15.89:0/2909878793 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d9b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f0288135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58512/0 (socket says 172.21.15.89:58512) 2023-09-30T22:21:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d9b5700 1 -- 172.21.15.89:0/2909878793 >> v2:172.21.15.89:3300/0 conn(0x7f0288135e60 msgr2=0x7f0288138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:09.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d9b5700 1 --2- 172.21.15.89:0/2909878793 >> v2:172.21.15.89:3300/0 conn(0x7f0288135e60 0x7f0288138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d9b5700 1 -- 172.21.15.89:0/2909878793 >> v1:172.21.15.89:6789/0 conn(0x7f0288130540 legacy=0x7f0288130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d9b5700 1 -- 172.21.15.89:0/2909878793 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288070b80 con 0x7f0288133540 2023-09-30T22:21:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028e9b7700 1 --2- 172.21.15.89:0/2909878793 >> v2:172.21.15.89:3300/0 conn(0x7f0288135e60 0x7f0288138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d9b5700 1 --2- 172.21.15.89:0/2909878793 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f0288135920 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0278009840 tx=0x7f0278009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22df4f225a3fb9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d1b4700 1 -- 172.21.15.89:0/2909878793 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278011830 con 0x7f0288133540 2023-09-30T22:21:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d1b4700 1 -- 172.21.15.89:0/2909878793 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0278011990 con 0x7f0288133540 2023-09-30T22:21:09.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.008+0000 7f028d1b4700 1 -- 172.21.15.89:0/2909878793 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278011d20 con 0x7f0288133540 2023-09-30T22:21:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.009+0000 7f029041a700 1 -- 172.21.15.89:0/2909878793 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 msgr2=0x7f0288135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.009+0000 7f029041a700 1 --2- 172.21.15.89:0/2909878793 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f0288135920 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0278009840 tx=0x7f0278009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.009+0000 7f029041a700 1 -- 172.21.15.89:0/2909878793 shutdown_connections 2023-09-30T22:21:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.009+0000 7f029041a700 1 --2- 172.21.15.89:0/2909878793 >> v2:172.21.15.89:3300/0 conn(0x7f0288135e60 0x7f0288138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.009+0000 7f029041a700 1 --2- 172.21.15.89:0/2909878793 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f0288135920 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.009+0000 7f029041a700 1 -- 172.21.15.89:0/2909878793 >> 172.21.15.89:0/2909878793 conn(0x7f02880a2aa0 msgr2=0x7f02880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.009+0000 7f029041a700 1 -- 172.21.15.89:0/2909878793 shutdown_connections 2023-09-30T22:21:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.009+0000 7f029041a700 1 -- 172.21.15.89:0/2909878793 wait complete. 2023-09-30T22:21:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.010+0000 7f029041a700 1 Processor -- start 2023-09-30T22:21:09.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.010+0000 7f029041a700 1 -- start start 2023-09-30T22:21:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.010+0000 7f029041a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0288130540 0x7f028812d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.010+0000 7f029041a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f028812d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.010+0000 7f029041a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0288135e60 0x7f028809c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.010+0000 7f029041a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028809d770 con 0x7f0288130540 2023-09-30T22:21:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.010+0000 7f029041a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028809ade0 con 0x7f0288133540 2023-09-30T22:21:09.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.010+0000 7f029041a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02880718e0 con 0x7f0288135e60 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e9b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0288135e60 0x7f028809c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0288130540 0x7f028812d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0288130540 0x7f028812d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34886/0 (socket says 172.21.15.89:34886) 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 -- 172.21.15.89:0/3778194384 learned_addr learned my addr 172.21.15.89:0/3778194384 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028d9b5700 1 --2- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f028812d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 -- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0288135e60 msgr2=0x7f028809c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 --2- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0288135e60 0x7f028809c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 -- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 msgr2=0x7f028812d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 --2- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f028812d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 -- 172.21.15.89:0/3778194384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280009a70 con 0x7f0288130540 2023-09-30T22:21:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f028e1b6700 1 --2- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0288130540 0x7f028812d1e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f028400c480 tx=0x7f028400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f027effd700 1 -- 172.21.15.89:0/3778194384 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284005d70 con 0x7f0288130540 2023-09-30T22:21:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f027effd700 1 -- 172.21.15.89:0/3778194384 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02840070d0 con 0x7f0288130540 2023-09-30T22:21:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02780094a0 con 0x7f0288130540 2023-09-30T22:21:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f027effd700 1 -- 172.21.15.89:0/3778194384 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284007440 con 0x7f0288130540 2023-09-30T22:21:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.011+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02880a2360 con 0x7f0288130540 2023-09-30T22:21:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.012+0000 7f027effd700 1 -- 172.21.15.89:0/3778194384 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02840075a0 con 0x7f0288130540 2023-09-30T22:21:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.012+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0288074ea0 con 0x7f0288130540 2023-09-30T22:21:09.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.013+0000 7f027effd700 1 -- 172.21.15.89:0/3778194384 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f028403ec70 con 0x7f0288130540 2023-09-30T22:21:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.021+0000 7f027effd700 1 -- 172.21.15.89:0/3778194384 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0284019d20 con 0x7f0288130540 2023-09-30T22:21:09.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.052+0000 7f81da0f6700 1 Processor -- start 2023-09-30T22:21:09.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.052+0000 7f81da0f6700 1 -- start start 2023-09-30T22:21:09.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.052+0000 7f81da0f6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d41338a0 0x7f81d4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.052+0000 7f81da0f6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409e930 0x7f81d413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.052+0000 7f81da0f6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d406ef30 con 0x7f81d409de50 2023-09-30T22:21:09.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.052+0000 7f81da0f6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d406fef0 con 0x7f81d41338a0 2023-09-30T22:21:09.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.052+0000 7f81da0f6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41334b0 con 0x7f81d409e930 2023-09-30T22:21:09.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d2ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f81d409de50 0x7f81d409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52130/0 (socket says 172.21.15.89:52130) 2023-09-30T22:21:09.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d2ffd700 1 -- 172.21.15.89:0/565435331 learned_addr learned my addr 172.21.15.89:0/565435331 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:09.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d37fe700 1 --2- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d41338a0 0x7f81d4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d3fff700 1 --2- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409e930 0x7f81d413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d37fe700 1 -- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409e930 msgr2=0x7f81d413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:09.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d37fe700 1 --2- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409e930 0x7f81d413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d27fc700 1 -- 172.21.15.89:0/565435331 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958116988 0 0) 0x7f81d406ef30 con 0x7f81d409de50 2023-09-30T22:21:09.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d37fe700 1 -- 172.21.15.89:0/565435331 >> v1:172.21.15.89:6789/0 conn(0x7f81d409de50 legacy=0x7f81d409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d37fe700 1 -- 172.21.15.89:0/565435331 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d406fc30 con 0x7f81d41338a0 2023-09-30T22:21:09.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d37fe700 1 --2- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d41338a0 0x7f81d4133c70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009870 tx=0x7f81c4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64fa45e1c21c4781 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:09.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d27fc700 1 -- 172.21.15.89:0/565435331 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4011830 con 0x7f81d41338a0 2023-09-30T22:21:09.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d27fc700 1 -- 172.21.15.89:0/565435331 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c4011990 con 0x7f81d41338a0 2023-09-30T22:21:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.053+0000 7f81d27fc700 1 -- 172.21.15.89:0/565435331 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4011d20 con 0x7f81d41338a0 2023-09-30T22:21:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.054+0000 7f81da0f6700 1 -- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d41338a0 msgr2=0x7f81d4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.054+0000 7f81da0f6700 1 --2- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d41338a0 0x7f81d4133c70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009870 tx=0x7f81c4009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.054+0000 7f81da0f6700 1 -- 172.21.15.89:0/565435331 shutdown_connections 2023-09-30T22:21:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.054+0000 7f81da0f6700 1 --2- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d41338a0 0x7f81d4133c70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.054+0000 7f81da0f6700 1 --2- 172.21.15.89:0/565435331 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409e930 0x7f81d413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.054+0000 7f81da0f6700 1 -- 172.21.15.89:0/565435331 >> 172.21.15.89:0/565435331 conn(0x7f81d40a2aa0 msgr2=0x7f81d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:09.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.055+0000 7f81da0f6700 1 -- 172.21.15.89:0/565435331 shutdown_connections 2023-09-30T22:21:09.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.055+0000 7f81da0f6700 1 -- 172.21.15.89:0/565435331 wait complete. 2023-09-30T22:21:09.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.055+0000 7f81da0f6700 1 Processor -- start 2023-09-30T22:21:09.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81da0f6700 1 -- start start 2023-09-30T22:21:09.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81da0f6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409de50 0x7f81d411b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81da0f6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d409e930 0x7f81d411bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81da0f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81d41338a0 0x7f81d411c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81da0f6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d411cac0 con 0x7f81d41338a0 2023-09-30T22:21:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81da0f6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d411d220 con 0x7f81d409e930 2023-09-30T22:21:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81da0f6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81d411e470 con 0x7f81d409de50 2023-09-30T22:21:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81d37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409de50 0x7f81d411b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81d37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409de50 0x7f81d411b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39922/0 (socket says 172.21.15.89:39922) 2023-09-30T22:21:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81d37fe700 1 -- 172.21.15.89:0/3565386270 learned_addr learned my addr 172.21.15.89:0/3565386270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81d3fff700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81d41338a0 0x7f81d411c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.056+0000 7f81d2ffd700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d409e930 0x7f81d411bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81d3fff700 1 -- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409de50 msgr2=0x7f81d411b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81d3fff700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409de50 0x7f81d411b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81d3fff700 1 -- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d409e930 msgr2=0x7f81d411bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81d3fff700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d409e930 0x7f81d411bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81d3fff700 1 -- 172.21.15.89:0/3565386270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c40094a0 con 0x7f81d41338a0 2023-09-30T22:21:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81d2ffd700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d409e930 0x7f81d411bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81d3fff700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81d41338a0 0x7f81d411c090 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f81cc007dc0 tx=0x7f81cc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81bbfff700 1 -- 172.21.15.89:0/3565386270 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc018070 con 0x7f81d41338a0 2023-09-30T22:21:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81bbfff700 1 -- 172.21.15.89:0/3565386270 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc00eb30 con 0x7f81d41338a0 2023-09-30T22:21:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d411ee50 con 0x7f81d41338a0 2023-09-30T22:21:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81bbfff700 1 -- 172.21.15.89:0/3565386270 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00eec0 con 0x7f81d41338a0 2023-09-30T22:21:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.057+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d406f640 con 0x7f81d41338a0 2023-09-30T22:21:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.058+0000 7f81bbfff700 1 -- 172.21.15.89:0/3565386270 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81cc003a20 con 0x7f81d41338a0 2023-09-30T22:21:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.058+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d4132a40 con 0x7f81d41338a0 2023-09-30T22:21:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.058+0000 7f81bbfff700 1 -- 172.21.15.89:0/3565386270 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f81cc03f6b0 con 0x7f81d41338a0 2023-09-30T22:21:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.063+0000 7f81bbfff700 1 -- 172.21.15.89:0/3565386270 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81cc009950 con 0x7f81d41338a0 2023-09-30T22:21:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.096+0000 7fd4c8cb2700 1 Processor -- start 2023-09-30T22:21:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.096+0000 7fd4c8cb2700 1 -- start start 2023-09-30T22:21:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c8cb2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c412cf60 0x7fd4c406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c8cb2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd4c4130430 0x7fd4c4137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c8cb2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c412cd90 con 0x7fd4c412f8b0 2023-09-30T22:21:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c8cb2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c412d330 con 0x7fd4c4130430 2023-09-30T22:21:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c8cb2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c4071780 con 0x7fd4c412cf60 2023-09-30T22:21:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c412cf60 0x7fd4c406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c1d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd4c412f8b0 0x7fd4c412fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52140/0 (socket says 172.21.15.89:52140) 2023-09-30T22:21:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c1d9b700 1 -- 172.21.15.89:0/909552005 learned_addr learned my addr 172.21.15.89:0/909552005 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c2d9d700 1 --2- 172.21.15.89:0/909552005 >> v2:172.21.15.89:3300/0 conn(0x7fd4c4130430 0x7fd4c4137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c259c700 1 -- 172.21.15.89:0/909552005 >> v2:172.21.15.89:3300/0 conn(0x7fd4c4130430 msgr2=0x7fd4c4137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c259c700 1 --2- 172.21.15.89:0/909552005 >> v2:172.21.15.89:3300/0 conn(0x7fd4c4130430 0x7fd4c4137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c259c700 1 -- 172.21.15.89:0/909552005 >> v1:172.21.15.89:6789/0 conn(0x7fd4c412f8b0 legacy=0x7fd4c412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c159a700 1 -- 172.21.15.89:0/909552005 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934826008 0 0) 0x7fd4c412cd90 con 0x7fd4c412f8b0 2023-09-30T22:21:09.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.097+0000 7fd4c259c700 1 -- 172.21.15.89:0/909552005 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c4071a40 con 0x7fd4c412cf60 2023-09-30T22:21:09.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.098+0000 7fd4c259c700 1 --2- 172.21.15.89:0/909552005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c412cf60 0x7fd4c406f5d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400bae0 tx=0x7fd4b400bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254815a9a9eca909 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:09.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.098+0000 7fd4c159a700 1 -- 172.21.15.89:0/909552005 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400dc70 con 0x7fd4c412cf60 2023-09-30T22:21:09.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.098+0000 7fd4c159a700 1 -- 172.21.15.89:0/909552005 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b400ddd0 con 0x7fd4c412cf60 2023-09-30T22:21:09.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.098+0000 7fd4c159a700 1 -- 172.21.15.89:0/909552005 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4010690 con 0x7fd4c412cf60 2023-09-30T22:21:09.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.098+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/909552005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c412cf60 msgr2=0x7fd4c406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.098+0000 7fd4c8cb2700 1 --2- 172.21.15.89:0/909552005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c412cf60 0x7fd4c406f5d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400bae0 tx=0x7fd4b400bdf0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.099+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/909552005 shutdown_connections 2023-09-30T22:21:09.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.099+0000 7fd4c8cb2700 1 --2- 172.21.15.89:0/909552005 >> v2:172.21.15.89:3300/0 conn(0x7fd4c4130430 0x7fd4c4137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.099+0000 7fd4c8cb2700 1 --2- 172.21.15.89:0/909552005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c412cf60 0x7fd4c406f5d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.099+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/909552005 >> 172.21.15.89:0/909552005 conn(0x7fd4c40a2aa0 msgr2=0x7fd4c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:09.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.099+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/909552005 shutdown_connections 2023-09-30T22:21:09.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.099+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/909552005 wait complete. 2023-09-30T22:21:09.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c8cb2700 1 Processor -- start 2023-09-30T22:21:09.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c8cb2700 1 -- start start 2023-09-30T22:21:09.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c8cb2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4c412cf60 0x7fd4c409b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c8cb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c412f8b0 0x7fd4c409be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c8cb2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c4130430 0x7fd4c41c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c8cb2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c409e580 con 0x7fd4c412f8b0 2023-09-30T22:21:09.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c8cb2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c412d9d0 con 0x7fd4c412cf60 2023-09-30T22:21:09.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c8cb2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c409d490 con 0x7fd4c4130430 2023-09-30T22:21:09.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c412f8b0 0x7fd4c409be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c2d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c4130430 0x7fd4c41c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c412f8b0 0x7fd4c409be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34924/0 (socket says 172.21.15.89:34924) 2023-09-30T22:21:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.100+0000 7fd4c1d9b700 1 -- 172.21.15.89:0/28211205 learned_addr learned my addr 172.21.15.89:0/28211205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c259c700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4c412cf60 0x7fd4c409b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c1d9b700 1 -- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c4130430 msgr2=0x7fd4c41c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c1d9b700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c4130430 0x7fd4c41c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c1d9b700 1 -- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4c412cf60 msgr2=0x7fd4c409b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c1d9b700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4c412cf60 0x7fd4c409b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c1d9b700 1 -- 172.21.15.89:0/28211205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b400b740 con 0x7fd4c412f8b0 2023-09-30T22:21:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c2d9d700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c4130430 0x7fd4c41c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c1d9b700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c412f8b0 0x7fd4c409be80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800c560 tx=0x7fd4b800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4b2ffd700 1 -- 172.21.15.89:0/28211205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8005e20 con 0x7fd4c412f8b0 2023-09-30T22:21:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c4129fa0 con 0x7fd4c412f8b0 2023-09-30T22:21:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4b2ffd700 1 -- 172.21.15.89:0/28211205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b80070d0 con 0x7fd4c412f8b0 2023-09-30T22:21:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4b2ffd700 1 -- 172.21.15.89:0/28211205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8007480 con 0x7fd4c412f8b0 2023-09-30T22:21:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.101+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c409e200 con 0x7fd4c412f8b0 2023-09-30T22:21:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.102+0000 7fd4b2ffd700 1 -- 172.21.15.89:0/28211205 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4b80075e0 con 0x7fd4c412f8b0 2023-09-30T22:21:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.102+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c4074ea0 con 0x7fd4c412f8b0 2023-09-30T22:21:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.103+0000 7fd4b2ffd700 1 -- 172.21.15.89:0/28211205 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd4b803eb20 con 0x7fd4c412f8b0 2023-09-30T22:21:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.111+0000 7fd4b2ffd700 1 -- 172.21.15.89:0/28211205 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4b8007890 con 0x7fd4c412f8b0 2023-09-30T22:21:09.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.139+0000 7f0b3d684700 1 Processor -- start 2023-09-30T22:21:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.139+0000 7f0b3d684700 1 -- start start 2023-09-30T22:21:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b3d684700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b3809e850 0x7f0b3809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b3d684700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f160 0x7f0b3813c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b3d684700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3809aa30 con 0x7f0b3809f590 2023-09-30T22:21:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b3d684700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3806c930 con 0x7f0b3809e850 2023-09-30T22:21:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b3d684700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3806bf30 con 0x7f0b3809f160 2023-09-30T22:21:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b367fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f160 0x7f0b3813c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b367fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f160 0x7f0b3813c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58570/0 (socket says 172.21.15.89:58570) 2023-09-30T22:21:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b367fc700 1 -- 172.21.15.89:0/878988191 learned_addr learned my addr 172.21.15.89:0/878988191 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b377fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0b3809f590 0x7f0b3813e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52152/0 (socket says 172.21.15.89:52152) 2023-09-30T22:21:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b36ffd700 1 --2- 172.21.15.89:0/878988191 >> v2:172.21.15.89:3300/0 conn(0x7f0b3809e850 0x7f0b3809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b367fc700 1 -- 172.21.15.89:0/878988191 >> v2:172.21.15.89:3300/0 conn(0x7f0b3809e850 msgr2=0x7f0b3809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.140+0000 7f0b367fc700 1 --2- 172.21.15.89:0/878988191 >> v2:172.21.15.89:3300/0 conn(0x7f0b3809e850 0x7f0b3809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b367fc700 1 -- 172.21.15.89:0/878988191 >> v1:172.21.15.89:6789/0 conn(0x7f0b3809f590 legacy=0x7f0b3813e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b367fc700 1 -- 172.21.15.89:0/878988191 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b380a0250 con 0x7f0b3809f160 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b36ffd700 1 --2- 172.21.15.89:0/878988191 >> v2:172.21.15.89:3300/0 conn(0x7f0b3809e850 0x7f0b3809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b367fc700 1 --2- 172.21.15.89:0/878988191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f160 0x7f0b3813c370 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0098f0 tx=0x7f0b2c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab67390b5e0a1cd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b35ffb700 1 -- 172.21.15.89:0/878988191 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c011830 con 0x7f0b3809f160 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b35ffb700 1 -- 172.21.15.89:0/878988191 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c011990 con 0x7f0b3809f160 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b35ffb700 1 -- 172.21.15.89:0/878988191 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c011d20 con 0x7f0b3809f160 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b3d684700 1 -- 172.21.15.89:0/878988191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f160 msgr2=0x7f0b3813c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.141+0000 7f0b3d684700 1 --2- 172.21.15.89:0/878988191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f160 0x7f0b3813c370 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0098f0 tx=0x7f0b2c009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.142+0000 7f0b3d684700 1 -- 172.21.15.89:0/878988191 shutdown_connections 2023-09-30T22:21:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.142+0000 7f0b3d684700 1 --2- 172.21.15.89:0/878988191 >> v2:172.21.15.89:3300/0 conn(0x7f0b3809e850 0x7f0b3809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.142+0000 7f0b3d684700 1 --2- 172.21.15.89:0/878988191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f160 0x7f0b3813c370 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.142+0000 7f0b3d684700 1 -- 172.21.15.89:0/878988191 >> 172.21.15.89:0/878988191 conn(0x7f0b380a3f70 msgr2=0x7f0b380a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.142+0000 7f0b3d684700 1 -- 172.21.15.89:0/878988191 shutdown_connections 2023-09-30T22:21:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.142+0000 7f0b3d684700 1 -- 172.21.15.89:0/878988191 wait complete. 2023-09-30T22:21:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b3d684700 1 Processor -- start 2023-09-30T22:21:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b3d684700 1 -- start start 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b3d684700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b3809e850 0x7f0b381bccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b3d684700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b3809f160 0x7f0b381bd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b3d684700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f590 0x7f0b3811b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b36ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b3809e850 0x7f0b381bccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b3d684700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3811d3f0 con 0x7f0b3809f160 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b3d684700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b381ac120 con 0x7f0b3809f590 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b3d684700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b381ac7c0 con 0x7f0b3809e850 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b367fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b3809f160 0x7f0b381bd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b36ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b3809e850 0x7f0b381bccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39946/0 (socket says 172.21.15.89:39946) 2023-09-30T22:21:09.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b36ffd700 1 -- 172.21.15.89:0/1671276619 learned_addr learned my addr 172.21.15.89:0/1671276619 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.143+0000 7f0b377fe700 1 --2- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f590 0x7f0b3811b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b367fc700 1 -- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b3809e850 msgr2=0x7f0b381bccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b367fc700 1 --2- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b3809e850 0x7f0b381bccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b367fc700 1 -- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f590 msgr2=0x7f0b3811b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b367fc700 1 --2- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f590 0x7f0b3811b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b367fc700 1 -- 172.21.15.89:0/1671276619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b280094a0 con 0x7f0b3809f160 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b367fc700 1 --2- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b3809f160 0x7f0b381bd200 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c016000 tx=0x7f0b2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b1f7fe700 1 -- 172.21.15.89:0/1671276619 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0036a0 con 0x7f0b3809f160 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b1f7fe700 1 -- 172.21.15.89:0/1671276619 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c003800 con 0x7f0b3809f160 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b3d684700 1 -- 172.21.15.89:0/1671276619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c009550 con 0x7f0b3809f160 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b1f7fe700 1 -- 172.21.15.89:0/1671276619 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c003b90 con 0x7f0b3809f160 2023-09-30T22:21:09.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.144+0000 7f0b3d684700 1 -- 172.21.15.89:0/1671276619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b381bd740 con 0x7f0b3809f160 2023-09-30T22:21:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.145+0000 7f0b1f7fe700 1 -- 172.21.15.89:0/1671276619 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b2c021070 con 0x7f0b3809f160 2023-09-30T22:21:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.145+0000 7f0b1d7fa700 1 -- 172.21.15.89:0/1671276619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b100052e0 con 0x7f0b3809f160 2023-09-30T22:21:09.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.145+0000 7f0b1f7fe700 1 -- 172.21.15.89:0/1671276619 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0b2c040ab0 con 0x7f0b3809f160 2023-09-30T22:21:09.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.154+0000 7f0b1f7fe700 1 -- 172.21.15.89:0/1671276619 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b2c02d3a0 con 0x7f0b3809f160 2023-09-30T22:21:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.255+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0288070e70 con 0x7f0288130540 2023-09-30T22:21:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.256+0000 7f027effd700 1 -- 172.21.15.89:0/3778194384 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0284017070 con 0x7f0288130540 2023-09-30T22:21:09.257 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0288130540 msgr2=0x7f028812d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 --2- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0288130540 0x7f028812d1e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f028400c480 tx=0x7f028400c790 comp rx=0 tx=0).stop 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 shutdown_connections 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 --2- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0288135e60 0x7f028809c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 --2- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0288130540 0x7f028812d1e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 --2- 172.21.15.89:0/3778194384 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0288133540 0x7f028812d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 >> 172.21.15.89:0/3778194384 conn(0x7f02880a2aa0 msgr2=0x7f0288131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 shutdown_connections 2023-09-30T22:21:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.258+0000 7f029041a700 1 -- 172.21.15.89:0/3778194384 wait complete. 2023-09-30T22:21:09.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.260+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f81d4074ea0 con 0x7f81d41338a0 2023-09-30T22:21:09.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.261+0000 7f81bbfff700 1 -- 172.21.15.89:0/3565386270 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81cc01cb20 con 0x7f81d41338a0 2023-09-30T22:21:09.262 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.263+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81d41338a0 msgr2=0x7f81d411c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.263+0000 7f81da0f6700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81d41338a0 0x7f81d411c090 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f81cc007dc0 tx=0x7f81cc00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.263+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 shutdown_connections 2023-09-30T22:21:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.263+0000 7f81da0f6700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81d41338a0 0x7f81d411c090 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.263+0000 7f81da0f6700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81d409de50 0x7f81d411b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.263+0000 7f81da0f6700 1 --2- 172.21.15.89:0/3565386270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81d409e930 0x7f81d411bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.263+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 >> 172.21.15.89:0/3565386270 conn(0x7f81d40a2aa0 msgr2=0x7f81d4130630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.264+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 shutdown_connections 2023-09-30T22:21:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.266+0000 7f81da0f6700 1 -- 172.21.15.89:0/3565386270 wait complete. 2023-09-30T22:21:09.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:09.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:09.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.312+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd4c40713a0 con 0x7fd4c412f8b0 2023-09-30T22:21:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.312+0000 7fd4b2ffd700 1 -- 172.21.15.89:0/28211205 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4b8017020 con 0x7fd4c412f8b0 2023-09-30T22:21:09.313 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:09.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c412f8b0 msgr2=0x7fd4c409be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c412f8b0 0x7fd4c409be80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800c560 tx=0x7fd4b800c870 comp rx=0 tx=0).stop 2023-09-30T22:21:09.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 shutdown_connections 2023-09-30T22:21:09.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4c4130430 0x7fd4c41c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4c412cf60 0x7fd4c409b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 --2- 172.21.15.89:0/28211205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4c412f8b0 0x7fd4c409be80 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 >> 172.21.15.89:0/28211205 conn(0x7fd4c40a2aa0 msgr2=0x7fd4c41b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:09.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 shutdown_connections 2023-09-30T22:21:09.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.314+0000 7fd4c8cb2700 1 -- 172.21.15.89:0/28211205 wait complete. 2023-09-30T22:21:09.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:09.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.380+0000 7f0b1d7fa700 1 -- 172.21.15.89:0/1671276619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0b10005590 con 0x7f0b3809f160 2023-09-30T22:21:09.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.380+0000 7f0b1f7fe700 1 -- 172.21.15.89:0/1671276619 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b2c029070 con 0x7f0b3809f160 2023-09-30T22:21:09.381 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:09.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 -- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b3809f160 msgr2=0x7f0b381bd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:09.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 --2- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b3809f160 0x7f0b381bd200 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c016000 tx=0x7f0b2c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:09.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 -- 172.21.15.89:0/1671276619 shutdown_connections 2023-09-30T22:21:09.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 --2- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b3809f590 0x7f0b3811b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 --2- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b3809e850 0x7f0b381bccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 --2- 172.21.15.89:0/1671276619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b3809f160 0x7f0b381bd200 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:09.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 -- 172.21.15.89:0/1671276619 >> 172.21.15.89:0/1671276619 conn(0x7f0b380a3f70 msgr2=0x7f0b381c0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:09.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 -- 172.21.15.89:0/1671276619 shutdown_connections 2023-09-30T22:21:09.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:09.382+0000 7f0b1d7fa700 1 -- 172.21.15.89:0/1671276619 wait complete. 2023-09-30T22:21:09.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:10.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:10.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:10.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:10.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:10.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.411+0000 7f13bc8d1700 1 Processor -- start 2023-09-30T22:21:10.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.412+0000 7f13bc8d1700 1 -- start start 2023-09-30T22:21:10.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.412+0000 7f13bc8d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f13b813ac00 0x7f13b813cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.412+0000 7f13bc8d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813d520 0x7f13b813f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.412+0000 7f13bc8d1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b809e800 con 0x7f13b8095690 2023-09-30T22:21:10.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.412+0000 7f13bc8d1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b8095a60 con 0x7f13b813ac00 2023-09-30T22:21:10.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.412+0000 7f13bc8d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8094bd0 con 0x7f13b813d520 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813d520 0x7f13b813f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813d520 0x7f13b813f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50664/0 (socket says 172.21.15.89:50664) 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b6d9d700 1 -- 172.21.15.89:0/1827477335 learned_addr learned my addr 172.21.15.89:0/1827477335 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b5d9b700 1 --2- 172.21.15.89:0/1827477335 >> v2:172.21.15.89:3300/0 conn(0x7f13b813ac00 0x7f13b813cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b6d9d700 1 -- 172.21.15.89:0/1827477335 >> v2:172.21.15.89:3300/0 conn(0x7f13b813ac00 msgr2=0x7f13b813cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b6d9d700 1 --2- 172.21.15.89:0/1827477335 >> v2:172.21.15.89:3300/0 conn(0x7f13b813ac00 0x7f13b813cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b6d9d700 1 -- 172.21.15.89:0/1827477335 >> v1:172.21.15.89:6789/0 conn(0x7f13b8095690 legacy=0x7f13b8134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b6d9d700 1 -- 172.21.15.89:0/1827477335 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b80940e0 con 0x7f13b813d520 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b5d9b700 1 --2- 172.21.15.89:0/1827477335 >> v2:172.21.15.89:3300/0 conn(0x7f13b813ac00 0x7f13b813cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b6d9d700 1 --2- 172.21.15.89:0/1827477335 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813d520 0x7f13b813f900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009e50 tx=0x7f13b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306409712dc04299 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:10.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.413+0000 7f13b559a700 1 -- 172.21.15.89:0/1827477335 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000e070 con 0x7f13b813d520 2023-09-30T22:21:10.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.414+0000 7f13b559a700 1 -- 172.21.15.89:0/1827477335 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b0003a20 con 0x7f13b813d520 2023-09-30T22:21:10.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.414+0000 7f13b559a700 1 -- 172.21.15.89:0/1827477335 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003dd0 con 0x7f13b813d520 2023-09-30T22:21:10.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.414+0000 7f13bc8d1700 1 -- 172.21.15.89:0/1827477335 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813d520 msgr2=0x7f13b813f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.414+0000 7f13bc8d1700 1 --2- 172.21.15.89:0/1827477335 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813d520 0x7f13b813f900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009e50 tx=0x7f13b000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:10.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.415+0000 7f13bc8d1700 1 -- 172.21.15.89:0/1827477335 shutdown_connections 2023-09-30T22:21:10.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.415+0000 7f13bc8d1700 1 --2- 172.21.15.89:0/1827477335 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813d520 0x7f13b813f900 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.415+0000 7f13bc8d1700 1 --2- 172.21.15.89:0/1827477335 >> v2:172.21.15.89:3300/0 conn(0x7f13b813ac00 0x7f13b813cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.415+0000 7f13bc8d1700 1 -- 172.21.15.89:0/1827477335 >> 172.21.15.89:0/1827477335 conn(0x7f13b80a2360 msgr2=0x7f13b80a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:10.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.415+0000 7f13bc8d1700 1 -- 172.21.15.89:0/1827477335 shutdown_connections 2023-09-30T22:21:10.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.415+0000 7f13bc8d1700 1 -- 172.21.15.89:0/1827477335 wait complete. 2023-09-30T22:21:10.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.417+0000 7f13bc8d1700 1 Processor -- start 2023-09-30T22:21:10.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.417+0000 7f13bc8d1700 1 -- start start 2023-09-30T22:21:10.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13bc8d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13b8095690 0x7f13b8099680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13bc8d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813ac00 0x7f13b8099bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13bc8d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13b813d520 0x7f13b809a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13bc8d1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b809b310 con 0x7f13b813d520 2023-09-30T22:21:10.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13bc8d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b809bce0 con 0x7f13b8095690 2023-09-30T22:21:10.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13bc8d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8140a20 con 0x7f13b813ac00 2023-09-30T22:21:10.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13b8095690 0x7f13b8099680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13b8095690 0x7f13b8099680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36770/0 (socket says 172.21.15.89:36770) 2023-09-30T22:21:10.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b659c700 1 -- 172.21.15.89:0/3782127484 learned_addr learned my addr 172.21.15.89:0/3782127484 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:10.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b6d9d700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13b813d520 0x7f13b809a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b6d9d700 1 -- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813ac00 msgr2=0x7f13b8099bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b6d9d700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813ac00 0x7f13b8099bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b6d9d700 1 -- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13b8095690 msgr2=0x7f13b8099680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b6d9d700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13b8095690 0x7f13b8099680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b6d9d700 1 -- 172.21.15.89:0/3782127484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0009b30 con 0x7f13b813d520 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b659c700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13b8095690 0x7f13b8099680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.418+0000 7f13b6d9d700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13b813d520 0x7f13b809a100 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f13b0014000 tx=0x7f13b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.419+0000 7f13a6ffd700 1 -- 172.21.15.89:0/3782127484 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0004240 con 0x7f13b813d520 2023-09-30T22:21:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.419+0000 7f13a6ffd700 1 -- 172.21.15.89:0/3782127484 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b00043a0 con 0x7f13b813d520 2023-09-30T22:21:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.419+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b8098c60 con 0x7f13b813d520 2023-09-30T22:21:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.419+0000 7f13a6ffd700 1 -- 172.21.15.89:0/3782127484 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0004750 con 0x7f13b813d520 2023-09-30T22:21:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.419+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b809bf30 con 0x7f13b813d520 2023-09-30T22:21:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.420+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b8074ea0 con 0x7f13b813d520 2023-09-30T22:21:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.421+0000 7f13a6ffd700 1 -- 172.21.15.89:0/3782127484 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f13b0019070 con 0x7f13b813d520 2023-09-30T22:21:10.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.421+0000 7f13a6ffd700 1 -- 172.21.15.89:0/3782127484 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f13b0040460 con 0x7f13b813d520 2023-09-30T22:21:10.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.429+0000 7f13a6ffd700 1 -- 172.21.15.89:0/3782127484 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13b0011b90 con 0x7f13b813d520 2023-09-30T22:21:10.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.453+0000 7fa635835700 1 Processor -- start 2023-09-30T22:21:10.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.453+0000 7fa635835700 1 -- start start 2023-09-30T22:21:10.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa635835700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6301316c0 0x7fa630131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa635835700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa63009d9f0 0x7fa63006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa635835700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa63006acb0 0x7fa63006b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa635835700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63006b630 con 0x7fa63009d9f0 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa635835700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63009e130 con 0x7fa6301316c0 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa635835700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6301312b0 con 0x7fa63006acb0 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62e7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa63009d9f0 0x7fa63006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62e7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa63009d9f0 0x7fa63006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33648/0 (socket says 172.21.15.89:33648) 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6301316c0 0x7fa630131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62e7fc700 1 -- 172.21.15.89:0/2585996864 learned_addr learned my addr 172.21.15.89:0/2585996864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:10.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62f7fe700 1 --2- 172.21.15.89:0/2585996864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa63006acb0 0x7fa63006b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62effd700 1 -- 172.21.15.89:0/2585996864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa63006acb0 msgr2=0x7fa63006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62effd700 1 --2- 172.21.15.89:0/2585996864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa63006acb0 0x7fa63006b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62effd700 1 -- 172.21.15.89:0/2585996864 >> v2:172.21.15.89:3300/0 conn(0x7fa63009d9f0 msgr2=0x7fa63006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62effd700 1 --2- 172.21.15.89:0/2585996864 >> v2:172.21.15.89:3300/0 conn(0x7fa63009d9f0 0x7fa63006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.454+0000 7fa62effd700 1 -- 172.21.15.89:0/2585996864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63009f6e0 con 0x7fa6301316c0 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.455+0000 7fa62effd700 1 --2- 172.21.15.89:0/2585996864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6301316c0 0x7fa630131a90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa620009870 tx=0x7fa620009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a7473d19d0e4c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.455+0000 7fa62dffb700 1 -- 172.21.15.89:0/2585996864 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011830 con 0x7fa6301316c0 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.455+0000 7fa62dffb700 1 -- 172.21.15.89:0/2585996864 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa620011990 con 0x7fa6301316c0 2023-09-30T22:21:10.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.455+0000 7fa62dffb700 1 -- 172.21.15.89:0/2585996864 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011d20 con 0x7fa6301316c0 2023-09-30T22:21:10.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 -- 172.21.15.89:0/2585996864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6301316c0 msgr2=0x7fa630131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 --2- 172.21.15.89:0/2585996864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6301316c0 0x7fa630131a90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa620009870 tx=0x7fa620009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:10.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 -- 172.21.15.89:0/2585996864 shutdown_connections 2023-09-30T22:21:10.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 --2- 172.21.15.89:0/2585996864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa63006acb0 0x7fa63006b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 --2- 172.21.15.89:0/2585996864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6301316c0 0x7fa630131a90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 --2- 172.21.15.89:0/2585996864 >> v2:172.21.15.89:3300/0 conn(0x7fa63009d9f0 0x7fa63006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 -- 172.21.15.89:0/2585996864 >> 172.21.15.89:0/2585996864 conn(0x7fa630066f50 msgr2=0x7fa63009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:10.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 -- 172.21.15.89:0/2585996864 shutdown_connections 2023-09-30T22:21:10.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.456+0000 7fa635835700 1 -- 172.21.15.89:0/2585996864 wait complete. 2023-09-30T22:21:10.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.457+0000 7fa635835700 1 Processor -- start 2023-09-30T22:21:10.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.457+0000 7fa635835700 1 -- start start 2023-09-30T22:21:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa635835700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa63006acb0 0x7fa630132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa635835700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63009d9f0 0x7fa630132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa635835700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6301316c0 0x7fa630132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa635835700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630072530 con 0x7fa63009d9f0 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa635835700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa630134bc0 con 0x7fa63006acb0 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa635835700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa630134370 con 0x7fa6301316c0 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa62f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6301316c0 0x7fa630132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa62f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6301316c0 0x7fa630132d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50706/0 (socket says 172.21.15.89:50706) 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa62f7fe700 1 -- 172.21.15.89:0/4280385148 learned_addr learned my addr 172.21.15.89:0/4280385148 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.458+0000 7fa62e7fc700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63009d9f0 0x7fa630132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa62f7fe700 1 -- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa63006acb0 msgr2=0x7fa630132300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa62f7fe700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa63006acb0 0x7fa630132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa62f7fe700 1 -- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63009d9f0 msgr2=0x7fa630132840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa62f7fe700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63009d9f0 0x7fa630132840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa62f7fe700 1 -- 172.21.15.89:0/4280385148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6200094a0 con 0x7fa6301316c0 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa62e7fc700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63009d9f0 0x7fa630132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa62f7fe700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6301316c0 0x7fa630132d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa62800efc0 tx=0x7fa62800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa617fff700 1 -- 172.21.15.89:0/4280385148 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628009ea0 con 0x7fa6301316c0 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa617fff700 1 -- 172.21.15.89:0/4280385148 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62800f040 con 0x7fa6301316c0 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6300a35d0 con 0x7fa6301316c0 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63012b8c0 con 0x7fa6301316c0 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.459+0000 7fa617fff700 1 -- 172.21.15.89:0/4280385148 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628014480 con 0x7fa6301316c0 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.460+0000 7fa617fff700 1 -- 172.21.15.89:0/4280385148 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6280145e0 con 0x7fa6301316c0 2023-09-30T22:21:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.461+0000 7fa617fff700 1 -- 172.21.15.89:0/4280385148 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa628040370 con 0x7fa6301316c0 2023-09-30T22:21:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.462+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6100052e0 con 0x7fa6301316c0 2023-09-30T22:21:10.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.471+0000 7fa617fff700 1 -- 172.21.15.89:0/4280385148 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa628018070 con 0x7fa6301316c0 2023-09-30T22:21:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f2e677700 1 Processor -- start 2023-09-30T22:21:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f2e677700 1 -- start start 2023-09-30T22:21:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f2e677700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806e570 0x7f1f2806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f2e677700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806ee80 0x7f1f28134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f2e677700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f28070ee0 con 0x7f1f2809eca0 2023-09-30T22:21:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f2e677700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f28071480 con 0x7f1f2806ee80 2023-09-30T22:21:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f2e677700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f28071b90 con 0x7f1f2806e570 2023-09-30T22:21:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f277fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1f2809eca0 0x7f1f28134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55152/0 (socket says 172.21.15.89:55152) 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f277fe700 1 -- 172.21.15.89:0/2703013194 learned_addr learned my addr 172.21.15.89:0/2703013194 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f27fff700 1 --2- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806e570 0x7f1f2806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.498+0000 7f1f2cc14700 1 --2- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806ee80 0x7f1f28134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.499+0000 7f1f27fff700 1 -- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806ee80 msgr2=0x7f1f28134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.499+0000 7f1f27fff700 1 --2- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806ee80 0x7f1f28134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.499+0000 7f1f27fff700 1 -- 172.21.15.89:0/2703013194 >> v1:172.21.15.89:6789/0 conn(0x7f1f2809eca0 legacy=0x7f1f28134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.499+0000 7f1f27fff700 1 -- 172.21.15.89:0/2703013194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f280711a0 con 0x7f1f2806e570 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.499+0000 7f1f27fff700 1 --2- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806e570 0x7f1f2806e940 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009870 tx=0x7f1f18009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b272f8246a59bdb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.499+0000 7f1f26ffd700 1 -- 172.21.15.89:0/2703013194 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18011830 con 0x7f1f2806e570 2023-09-30T22:21:10.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.499+0000 7f1f26ffd700 1 -- 172.21.15.89:0/2703013194 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f18011990 con 0x7f1f2806e570 2023-09-30T22:21:10.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.499+0000 7f1f26ffd700 1 -- 172.21.15.89:0/2703013194 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18011d20 con 0x7f1f2806e570 2023-09-30T22:21:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.500+0000 7f1f2e677700 1 -- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806e570 msgr2=0x7f1f2806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.500+0000 7f1f2e677700 1 --2- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806e570 0x7f1f2806e940 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009870 tx=0x7f1f18009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.500+0000 7f1f2e677700 1 -- 172.21.15.89:0/2703013194 shutdown_connections 2023-09-30T22:21:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.500+0000 7f1f2e677700 1 --2- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806ee80 0x7f1f28134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.500+0000 7f1f2e677700 1 --2- 172.21.15.89:0/2703013194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806e570 0x7f1f2806e940 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.500+0000 7f1f2e677700 1 -- 172.21.15.89:0/2703013194 >> 172.21.15.89:0/2703013194 conn(0x7f1f280a2aa0 msgr2=0x7f1f280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.500+0000 7f1f2e677700 1 -- 172.21.15.89:0/2703013194 shutdown_connections 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.500+0000 7f1f2e677700 1 -- 172.21.15.89:0/2703013194 wait complete. 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.501+0000 7f1f2e677700 1 Processor -- start 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.501+0000 7f1f2e677700 1 -- start start 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.501+0000 7f1f2e677700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806e570 0x7f1f2813c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2e677700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806ee80 0x7f1f28133550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2e677700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f28133a90 0x7f1f28133f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2e677700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2813e640 con 0x7f1f28133a90 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2e677700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2813ece0 con 0x7f1f2806e570 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2e677700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2813ccd0 con 0x7f1f2806ee80 2023-09-30T22:21:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f27fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806e570 0x7f1f2813c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f27fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806e570 0x7f1f2813c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36804/0 (socket says 172.21.15.89:36804) 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f27fff700 1 -- 172.21.15.89:0/1656399983 learned_addr learned my addr 172.21.15.89:0/1656399983 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f277fe700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806ee80 0x7f1f28133550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2cc14700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f28133a90 0x7f1f28133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2cc14700 1 -- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806ee80 msgr2=0x7f1f28133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2cc14700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806ee80 0x7f1f28133550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2cc14700 1 -- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806e570 msgr2=0x7f1f2813c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2cc14700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806e570 0x7f1f2813c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2cc14700 1 -- 172.21.15.89:0/1656399983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f180094a0 con 0x7f1f28133a90 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.502+0000 7f1f2cc14700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f28133a90 0x7f1f28133f30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1f20007d90 tx=0x7f1f2000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:10.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.503+0000 7f1f24ff9700 1 -- 172.21.15.89:0/1656399983 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20018070 con 0x7f1f28133a90 2023-09-30T22:21:10.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.503+0000 7f1f24ff9700 1 -- 172.21.15.89:0/1656399983 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2000eb30 con 0x7f1f28133a90 2023-09-30T22:21:10.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.503+0000 7f1f24ff9700 1 -- 172.21.15.89:0/1656399983 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20003a20 con 0x7f1f28133a90 2023-09-30T22:21:10.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.503+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2813df90 con 0x7f1f28133a90 2023-09-30T22:21:10.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.503+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2813dad0 con 0x7f1f28133a90 2023-09-30T22:21:10.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.504+0000 7f1f24ff9700 1 -- 172.21.15.89:0/1656399983 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f20014420 con 0x7f1f28133a90 2023-09-30T22:21:10.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.504+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f28074ea0 con 0x7f1f28133a90 2023-09-30T22:21:10.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.504+0000 7f1f24ff9700 1 -- 172.21.15.89:0/1656399983 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1f2003f5b0 con 0x7f1f28133a90 2023-09-30T22:21:10.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.513+0000 7f1f24ff9700 1 -- 172.21.15.89:0/1656399983 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f200126a0 con 0x7f1f28133a90 2023-09-30T22:21:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.534+0000 7f28540cb700 1 Processor -- start 2023-09-30T22:21:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.535+0000 7f28540cb700 1 -- start start 2023-09-30T22:21:10.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.535+0000 7f28540cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09f410 0x7f284c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.535+0000 7f28540cb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f284c136fe0 0x7f284c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f28540cb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c070940 con 0x7f284c09e930 2023-09-30T22:21:10.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f28540cb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c06fd60 con 0x7f284c136fe0 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f28540cb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f284c06fa80 con 0x7f284c09f410 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2851e67700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f284c09e930 0x7f284c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55164/0 (socket says 172.21.15.89:55164) 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2851e67700 1 -- 172.21.15.89:0/2436615849 learned_addr learned my addr 172.21.15.89:0/2436615849 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2852668700 1 --2- 172.21.15.89:0/2436615849 >> v2:172.21.15.89:3300/0 conn(0x7f284c136fe0 0x7f284c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2851666700 1 --2- 172.21.15.89:0/2436615849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09f410 0x7f284c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2850e65700 1 -- 172.21.15.89:0/2436615849 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110265583 0 0) 0x7f284c070940 con 0x7f284c09e930 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2850e65700 1 -- 172.21.15.89:0/2436615849 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28380035b0 con 0x7f284c09e930 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2852668700 1 -- 172.21.15.89:0/2436615849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09f410 msgr2=0x7f284c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2852668700 1 --2- 172.21.15.89:0/2436615849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09f410 0x7f284c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2852668700 1 -- 172.21.15.89:0/2436615849 >> v1:172.21.15.89:6789/0 conn(0x7f284c09e930 legacy=0x7f284c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2852668700 1 -- 172.21.15.89:0/2436615849 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c09fb60 con 0x7f284c136fe0 2023-09-30T22:21:10.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2852668700 1 --2- 172.21.15.89:0/2436615849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 0x7f284c1393c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2844009d90 tx=0x7f284400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5900bd291b2d95d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:10.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.536+0000 7f2850e65700 1 -- 172.21.15.89:0/2436615849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400e070 con 0x7f284c136fe0 2023-09-30T22:21:10.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.537+0000 7f2850e65700 1 -- 172.21.15.89:0/2436615849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400bb20 con 0x7f284c136fe0 2023-09-30T22:21:10.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.537+0000 7f2850e65700 1 -- 172.21.15.89:0/2436615849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400be90 con 0x7f284c136fe0 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.537+0000 7f28540cb700 1 -- 172.21.15.89:0/2436615849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 msgr2=0x7f284c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.537+0000 7f28540cb700 1 --2- 172.21.15.89:0/2436615849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 0x7f284c1393c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2844009d90 tx=0x7f284400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.538+0000 7f28540cb700 1 -- 172.21.15.89:0/2436615849 shutdown_connections 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.538+0000 7f28540cb700 1 --2- 172.21.15.89:0/2436615849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 0x7f284c1393c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.538+0000 7f28540cb700 1 --2- 172.21.15.89:0/2436615849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09f410 0x7f284c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.538+0000 7f28540cb700 1 -- 172.21.15.89:0/2436615849 >> 172.21.15.89:0/2436615849 conn(0x7f284c0a4210 msgr2=0x7f284c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.538+0000 7f28540cb700 1 -- 172.21.15.89:0/2436615849 shutdown_connections 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.538+0000 7f28540cb700 1 -- 172.21.15.89:0/2436615849 wait complete. 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f28540cb700 1 Processor -- start 2023-09-30T22:21:10.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f28540cb700 1 -- start start 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f28540cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09e930 0x7f284c11b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f28540cb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f284c09f410 0x7f284c11b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f28540cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 0x7f284c11bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f28540cb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c11e350 con 0x7f284c136fe0 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f28540cb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c11e710 con 0x7f284c09f410 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f28540cb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f284c11d890 con 0x7f284c09e930 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f2851e67700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09e930 0x7f284c11b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f2851e67700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09e930 0x7f284c11b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50762/0 (socket says 172.21.15.89:50762) 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f2851e67700 1 -- 172.21.15.89:0/2299299842 learned_addr learned my addr 172.21.15.89:0/2299299842 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f2851666700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f284c09f410 0x7f284c11b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f2852668700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 0x7f284c11bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f2851e67700 1 -- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f284c09f410 msgr2=0x7f284c11b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.539+0000 7f2851e67700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f284c09f410 0x7f284c11b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f2851e67700 1 -- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 msgr2=0x7f284c11bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f2851e67700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 0x7f284c11bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f2851e67700 1 -- 172.21.15.89:0/2299299842 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2844009a70 con 0x7f284c09e930 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f2851666700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f284c09f410 0x7f284c11b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f2851e67700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09e930 0x7f284c11b460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f284800c700 tx=0x7f284800ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f28427fc700 1 -- 172.21.15.89:0/2299299842 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848005d70 con 0x7f284c09e930 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f28427fc700 1 -- 172.21.15.89:0/2299299842 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28480070d0 con 0x7f284c09e930 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c071d20 con 0x7f284c09e930 2023-09-30T22:21:10.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f28427fc700 1 -- 172.21.15.89:0/2299299842 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848007460 con 0x7f284c09e930 2023-09-30T22:21:10.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.540+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c11d070 con 0x7f284c09e930 2023-09-30T22:21:10.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.541+0000 7f28427fc700 1 -- 172.21.15.89:0/2299299842 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28480075c0 con 0x7f284c09e930 2023-09-30T22:21:10.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.541+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c11eb00 con 0x7f284c09e930 2023-09-30T22:21:10.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.541+0000 7f28427fc700 1 -- 172.21.15.89:0/2299299842 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f284803ebf0 con 0x7f284c09e930 2023-09-30T22:21:10.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.550+0000 7f28427fc700 1 -- 172.21.15.89:0/2299299842 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f284c11eb00 con 0x7f284c09e930 2023-09-30T22:21:10.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.630+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f13b806b700 con 0x7f13b813d520 2023-09-30T22:21:10.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.631+0000 7f13a6ffd700 1 -- 172.21.15.89:0/3782127484 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13b0017070 con 0x7f13b813d520 2023-09-30T22:21:10.632 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:10.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13b813d520 msgr2=0x7f13b809a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13b813d520 0x7f13b809a100 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f13b0014000 tx=0x7f13b0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 shutdown_connections 2023-09-30T22:21:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13b813d520 0x7f13b809a100 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f13b8095690 0x7f13b8099680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 --2- 172.21.15.89:0/3782127484 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f13b813ac00 0x7f13b8099bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 >> 172.21.15.89:0/3782127484 conn(0x7f13b80a2360 msgr2=0x7f13b813afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 shutdown_connections 2023-09-30T22:21:10.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.634+0000 7f13bc8d1700 1 -- 172.21.15.89:0/3782127484 wait complete. 2023-09-30T22:21:10.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:10.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.674+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa610005590 con 0x7fa6301316c0 2023-09-30T22:21:10.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.675+0000 7fa617fff700 1 -- 172.21.15.89:0/4280385148 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa62802c090 con 0x7fa6301316c0 2023-09-30T22:21:10.678 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6301316c0 msgr2=0x7fa630132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6301316c0 0x7fa630132d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa62800efc0 tx=0x7fa62800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 shutdown_connections 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6301316c0 0x7fa630132d80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa63006acb0 0x7fa630132300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 --2- 172.21.15.89:0/4280385148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63009d9f0 0x7fa630132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 >> 172.21.15.89:0/4280385148 conn(0x7fa630066f50 msgr2=0x7fa63009bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 shutdown_connections 2023-09-30T22:21:10.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.677+0000 7fa635835700 1 -- 172.21.15.89:0/4280385148 wait complete. 2023-09-30T22:21:10.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.712+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1f2806b9e0 con 0x7f1f28133a90 2023-09-30T22:21:10.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.712+0000 7f1f24ff9700 1 -- 172.21.15.89:0/1656399983 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f200122e0 con 0x7f1f28133a90 2023-09-30T22:21:10.713 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f28133a90 msgr2=0x7f1f28133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f28133a90 0x7f1f28133f30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1f20007d90 tx=0x7f1f2000e3c0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 shutdown_connections 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f28133a90 0x7f1f28133f30 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f2806e570 0x7f1f2813c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 --2- 172.21.15.89:0/1656399983 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f2806ee80 0x7f1f28133550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 >> 172.21.15.89:0/1656399983 conn(0x7f1f280a2aa0 msgr2=0x7f1f28137970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 shutdown_connections 2023-09-30T22:21:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.714+0000 7f1f2e677700 1 -- 172.21.15.89:0/1656399983 wait complete. 2023-09-30T22:21:10.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.753+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f284c11eb00 con 0x7f284c09e930 2023-09-30T22:21:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.754+0000 7f28427fc700 1 -- 172.21.15.89:0/2299299842 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f284c11eb00 con 0x7f284c09e930 2023-09-30T22:21:10.755 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09e930 msgr2=0x7f284c11b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09e930 0x7f284c11b460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f284800c700 tx=0x7f284800ca10 comp rx=0 tx=0).stop 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 shutdown_connections 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f284c136fe0 0x7f284c11bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f284c09e930 0x7f284c11b460 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 --2- 172.21.15.89:0/2299299842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f284c09f410 0x7f284c11b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 >> 172.21.15.89:0/2299299842 conn(0x7f284c0a4210 msgr2=0x7f284c1393c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 shutdown_connections 2023-09-30T22:21:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:10.756+0000 7f28540cb700 1 -- 172.21.15.89:0/2299299842 wait complete. 2023-09-30T22:21:10.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:11.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:11.693 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:11.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:11.769 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.794+0000 7f7486a1a700 1 Processor -- start 2023-09-30T22:21:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.794+0000 7f7486a1a700 1 -- start start 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f7486a1a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7480131810 0x7f7480131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f7486a1a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7480099a40 0x7f7480099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f7486a1a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748009a850 con 0x7f748009a3b0 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f7486a1a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748009b580 con 0x7f7480099a40 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f7486a1a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74801313c0 con 0x7f7480131810 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f747f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7480099a40 0x7f7480099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f747f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7480099a40 0x7f7480099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33692/0 (socket says 172.21.15.89:33692) 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f747f7fe700 1 -- 172.21.15.89:0/14885691 learned_addr learned my addr 172.21.15.89:0/14885691 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.795+0000 7f747ffff700 1 --2- 172.21.15.89:0/14885691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7480131810 0x7f7480131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.796+0000 7f747f7fe700 1 -- 172.21.15.89:0/14885691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7480131810 msgr2=0x7f7480131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:11.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.796+0000 7f747f7fe700 1 --2- 172.21.15.89:0/14885691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7480131810 0x7f7480131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.796+0000 7f747f7fe700 1 -- 172.21.15.89:0/14885691 >> v1:172.21.15.89:6789/0 conn(0x7f748009a3b0 legacy=0x7f7480138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.796+0000 7f747f7fe700 1 -- 172.21.15.89:0/14885691 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748006e8c0 con 0x7f7480099a40 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.796+0000 7f747f7fe700 1 --2- 172.21.15.89:0/14885691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480099a40 0x7f7480099e70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f7474009920 tx=0x7f7474009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fc5448f81472a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.796+0000 7f747effd700 1 -- 172.21.15.89:0/14885691 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474011830 con 0x7f7480099a40 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.796+0000 7f747effd700 1 -- 172.21.15.89:0/14885691 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7474011990 con 0x7f7480099a40 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.796+0000 7f747effd700 1 -- 172.21.15.89:0/14885691 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474011d20 con 0x7f7480099a40 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.797+0000 7f7486a1a700 1 -- 172.21.15.89:0/14885691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480099a40 msgr2=0x7f7480099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.797+0000 7f7486a1a700 1 --2- 172.21.15.89:0/14885691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480099a40 0x7f7480099e70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f7474009920 tx=0x7f7474009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.797+0000 7f7486a1a700 1 -- 172.21.15.89:0/14885691 shutdown_connections 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.797+0000 7f7486a1a700 1 --2- 172.21.15.89:0/14885691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7480131810 0x7f7480131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.797+0000 7f7486a1a700 1 --2- 172.21.15.89:0/14885691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480099a40 0x7f7480099e70 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.797+0000 7f7486a1a700 1 -- 172.21.15.89:0/14885691 >> 172.21.15.89:0/14885691 conn(0x7f74800a2aa0 msgr2=0x7f74800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:11.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.797+0000 7f7486a1a700 1 -- 172.21.15.89:0/14885691 shutdown_connections 2023-09-30T22:21:11.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.797+0000 7f7486a1a700 1 -- 172.21.15.89:0/14885691 wait complete. 2023-09-30T22:21:11.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.798+0000 7f7486a1a700 1 Processor -- start 2023-09-30T22:21:11.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.798+0000 7f7486a1a700 1 -- start start 2023-09-30T22:21:11.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7486a1a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7480099a40 0x7f748011d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7486a1a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f748009a3b0 0x7f748011d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7486a1a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480131810 0x7f748011db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7486a1a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7480120470 con 0x7f7480131810 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7486a1a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748011f750 con 0x7f7480099a40 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7486a1a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74800a1b40 con 0x7f748009a3b0 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7484fb7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480131810 0x7f748011db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f747f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f748009a3b0 0x7f748011d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7484fb7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480131810 0x7f748011db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33708/0 (socket says 172.21.15.89:33708) 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f7484fb7700 1 -- 172.21.15.89:0/1119443542 learned_addr learned my addr 172.21.15.89:0/1119443542 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:11.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.799+0000 7f747ffff700 1 --2- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7480099a40 0x7f748011d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747f7fe700 1 -- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7480099a40 msgr2=0x7f748011d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747f7fe700 1 --2- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7480099a40 0x7f748011d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747f7fe700 1 -- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480131810 msgr2=0x7f748011db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747f7fe700 1 --2- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480131810 0x7f748011db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747f7fe700 1 -- 172.21.15.89:0/1119443542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7474009550 con 0x7f748009a3b0 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747f7fe700 1 --2- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f748009a3b0 0x7f748011d620 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7474016000 tx=0x7f7474004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747cff9700 1 -- 172.21.15.89:0/1119443542 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474004510 con 0x7f748009a3b0 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747cff9700 1 -- 172.21.15.89:0/1119443542 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74740036a0 con 0x7f748009a3b0 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7480120750 con 0x7f748009a3b0 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f747cff9700 1 -- 172.21.15.89:0/1119443542 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474003a10 con 0x7f748009a3b0 2023-09-30T22:21:11.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.800+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7480120c50 con 0x7f748009a3b0 2023-09-30T22:21:11.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.801+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74801202b0 con 0x7f748009a3b0 2023-09-30T22:21:11.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.802+0000 7f747cff9700 1 -- 172.21.15.89:0/1119443542 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7474021020 con 0x7f748009a3b0 2023-09-30T22:21:11.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.802+0000 7f747cff9700 1 -- 172.21.15.89:0/1119443542 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7474040ca0 con 0x7f748009a3b0 2023-09-30T22:21:11.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.806+0000 7f747cff9700 1 -- 172.21.15.89:0/1119443542 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74740199a0 con 0x7f748009a3b0 2023-09-30T22:21:11.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.843+0000 7fb0b753e700 1 Processor -- start 2023-09-30T22:21:11.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.843+0000 7fb0b753e700 1 -- start start 2023-09-30T22:21:11.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b753e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb0b0131810 0x7fb0b0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b753e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009a580 0x7fb0b0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b753e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b01313c0 con 0x7fb0b0099a40 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b753e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b006e8c0 con 0x7fb0b0131810 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b753e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b0071d20 con 0x7fb0b009a580 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b5adb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009a580 0x7fb0b0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b5adb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009a580 0x7fb0b0139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36836/0 (socket says 172.21.15.89:36836) 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b5adb700 1 -- 172.21.15.89:0/1018472240 learned_addr learned my addr 172.21.15.89:0/1018472240 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.844+0000 7fb0b52da700 1 --2- 172.21.15.89:0/1018472240 >> v2:172.21.15.89:3300/0 conn(0x7fb0b0131810 0x7fb0b0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.845+0000 7fb0b5adb700 1 -- 172.21.15.89:0/1018472240 >> v2:172.21.15.89:3300/0 conn(0x7fb0b0131810 msgr2=0x7fb0b0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.845+0000 7fb0b5adb700 1 --2- 172.21.15.89:0/1018472240 >> v2:172.21.15.89:3300/0 conn(0x7fb0b0131810 0x7fb0b0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.845+0000 7fb0b5adb700 1 -- 172.21.15.89:0/1018472240 >> v1:172.21.15.89:6789/0 conn(0x7fb0b0099a40 legacy=0x7fb0b0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.845+0000 7fb0b5adb700 1 -- 172.21.15.89:0/1018472240 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b00721a0 con 0x7fb0b009a580 2023-09-30T22:21:11.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.845+0000 7fb0b5adb700 1 --2- 172.21.15.89:0/1018472240 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009a580 0x7fb0b0139660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009e50 tx=0x7fb0a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f1ab081a609ed23 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:11.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.845+0000 7fb0a7fff700 1 -- 172.21.15.89:0/1018472240 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800e070 con 0x7fb0b009a580 2023-09-30T22:21:11.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.845+0000 7fb0a7fff700 1 -- 172.21.15.89:0/1018472240 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a800b910 con 0x7fb0b009a580 2023-09-30T22:21:11.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.845+0000 7fb0a7fff700 1 -- 172.21.15.89:0/1018472240 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800bcc0 con 0x7fb0b009a580 2023-09-30T22:21:11.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.846+0000 7fb0b753e700 1 -- 172.21.15.89:0/1018472240 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009a580 msgr2=0x7fb0b0139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.846+0000 7fb0b753e700 1 --2- 172.21.15.89:0/1018472240 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009a580 0x7fb0b0139660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009e50 tx=0x7fb0a800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.846+0000 7fb0b753e700 1 -- 172.21.15.89:0/1018472240 shutdown_connections 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.846+0000 7fb0b753e700 1 --2- 172.21.15.89:0/1018472240 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b009a580 0x7fb0b0139660 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.846+0000 7fb0b753e700 1 --2- 172.21.15.89:0/1018472240 >> v2:172.21.15.89:3300/0 conn(0x7fb0b0131810 0x7fb0b0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.846+0000 7fb0b753e700 1 -- 172.21.15.89:0/1018472240 >> 172.21.15.89:0/1018472240 conn(0x7fb0b00a2aa0 msgr2=0x7fb0b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.846+0000 7fb0b753e700 1 -- 172.21.15.89:0/1018472240 shutdown_connections 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.846+0000 7fb0b753e700 1 -- 172.21.15.89:0/1018472240 wait complete. 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.847+0000 7fb0b753e700 1 Processor -- start 2023-09-30T22:21:11.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.847+0000 7fb0b753e700 1 -- start start 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b753e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b0099a40 0x7fb0b011a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b753e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009a580 0x7fb0b0118f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b753e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b0131810 0x7fb0b0119470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b753e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b011af20 con 0x7fb0b009a580 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b753e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b012e720 con 0x7fb0b0099a40 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b753e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b011c470 con 0x7fb0b0131810 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b52da700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b0099a40 0x7fb0b011a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b52da700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b0099a40 0x7fb0b011a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36846/0 (socket says 172.21.15.89:36846) 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b52da700 1 -- 172.21.15.89:0/2639317584 learned_addr learned my addr 172.21.15.89:0/2639317584 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:11.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b4ad9700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009a580 0x7fb0b0118f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b5adb700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b0131810 0x7fb0b0119470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b52da700 1 -- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b0131810 msgr2=0x7fb0b0119470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b52da700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b0131810 0x7fb0b0119470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b52da700 1 -- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009a580 msgr2=0x7fb0b0118f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b52da700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009a580 0x7fb0b0118f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b52da700 1 -- 172.21.15.89:0/2639317584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a8009b30 con 0x7fb0b0099a40 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.848+0000 7fb0b5adb700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b0131810 0x7fb0b0119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.849+0000 7fb0b52da700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b0099a40 0x7fb0b011a900 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9b0 tx=0x7fb0ac00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.849+0000 7fb0a5ffb700 1 -- 172.21.15.89:0/2639317584 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac009710 con 0x7fb0b0099a40 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.849+0000 7fb0a5ffb700 1 -- 172.21.15.89:0/2639317584 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac00c850 con 0x7fb0b0099a40 2023-09-30T22:21:11.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.849+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b0125b60 con 0x7fb0b0099a40 2023-09-30T22:21:11.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.849+0000 7fb0a5ffb700 1 -- 172.21.15.89:0/2639317584 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00cbc0 con 0x7fb0b0099a40 2023-09-30T22:21:11.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.849+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b01199b0 con 0x7fb0b0099a40 2023-09-30T22:21:11.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.850+0000 7fb0a5ffb700 1 -- 172.21.15.89:0/2639317584 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0ac00cd20 con 0x7fb0b0099a40 2023-09-30T22:21:11.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.850+0000 7fb0a5ffb700 1 -- 172.21.15.89:0/2639317584 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb0ac03f8a0 con 0x7fb0b0099a40 2023-09-30T22:21:11.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.850+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b0074ea0 con 0x7fb0b0099a40 2023-09-30T22:21:11.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.855+0000 7fb0a5ffb700 1 -- 172.21.15.89:0/2639317584 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0ac021020 con 0x7fb0b0099a40 2023-09-30T22:21:11.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.877+0000 7f7d8f4f5700 1 Processor -- start 2023-09-30T22:21:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.877+0000 7f7d8f4f5700 1 -- start start 2023-09-30T22:21:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.877+0000 7f7d8f4f5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809d6f0 0x7f7d8809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.877+0000 7f7d8f4f5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d8809e410 0x7f7d88130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.877+0000 7f7d8f4f5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8809f130 con 0x7f7d8809e000 2023-09-30T22:21:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.877+0000 7f7d8f4f5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8809f520 con 0x7f7d8809e410 2023-09-30T22:21:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.877+0000 7f7d8f4f5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8809e930 con 0x7f7d8809d6f0 2023-09-30T22:21:11.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8d291700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809d6f0 0x7f7d8809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8da92700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d8809e410 0x7f7d88130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8d291700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809d6f0 0x7f7d8809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36852/0 (socket says 172.21.15.89:36852) 2023-09-30T22:21:11.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8da92700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7d8809e410 0x7f7d88130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33730/0 (socket says 172.21.15.89:33730) 2023-09-30T22:21:11.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8d291700 1 -- 172.21.15.89:0/2978783706 learned_addr learned my addr 172.21.15.89:0/2978783706 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8da92700 1 -- 172.21.15.89:0/2978783706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809d6f0 msgr2=0x7f7d8809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8da92700 1 --2- 172.21.15.89:0/2978783706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809d6f0 0x7f7d8809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8da92700 1 -- 172.21.15.89:0/2978783706 >> v1:172.21.15.89:6789/0 conn(0x7f7d8809e000 legacy=0x7f7d88130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d7ffff700 1 -- 172.21.15.89:0/2978783706 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146983155 0 0) 0x7f7d8809f130 con 0x7f7d8809e000 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8da92700 1 -- 172.21.15.89:0/2978783706 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8809b2d0 con 0x7f7d8809e410 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8d291700 1 --2- 172.21.15.89:0/2978783706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809d6f0 0x7f7d8809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d8da92700 1 --2- 172.21.15.89:0/2978783706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809e410 0x7f7d88130d70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009d90 tx=0x7f7d8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff0fc1cacaf3dfe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d7ffff700 1 -- 172.21.15.89:0/2978783706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000e070 con 0x7f7d8809e410 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d7ffff700 1 -- 172.21.15.89:0/2978783706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000bb20 con 0x7f7d8809e410 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.878+0000 7f7d7ffff700 1 -- 172.21.15.89:0/2978783706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80012430 con 0x7f7d8809e410 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.879+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/2978783706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809e410 msgr2=0x7f7d88130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.879+0000 7f7d8f4f5700 1 --2- 172.21.15.89:0/2978783706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809e410 0x7f7d88130d70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009d90 tx=0x7f7d8000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.879+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/2978783706 shutdown_connections 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.879+0000 7f7d8f4f5700 1 --2- 172.21.15.89:0/2978783706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809e410 0x7f7d88130d70 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.879+0000 7f7d8f4f5700 1 --2- 172.21.15.89:0/2978783706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809d6f0 0x7f7d8809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.879+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/2978783706 >> 172.21.15.89:0/2978783706 conn(0x7f7d88066f50 msgr2=0x7f7d8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.879+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/2978783706 shutdown_connections 2023-09-30T22:21:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.879+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/2978783706 wait complete. 2023-09-30T22:21:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.880+0000 7f7d8f4f5700 1 Processor -- start 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.880+0000 7f7d8f4f5700 1 -- start start 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8f4f5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809d6f0 0x7f7d8811de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8f4f5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d8809e000 0x7f7d8811e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8f4f5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809e410 0x7f7d8811e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8f4f5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8811ffe0 con 0x7f7d8809d6f0 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8f4f5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d88120ca0 con 0x7f7d8809e410 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8f4f5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880a20b0 con 0x7f7d8809e000 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8d291700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809d6f0 0x7f7d8811de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8d291700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809d6f0 0x7f7d8811de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33736/0 (socket says 172.21.15.89:33736) 2023-09-30T22:21:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8d291700 1 -- 172.21.15.89:0/3505139730 learned_addr learned my addr 172.21.15.89:0/3505139730 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8da92700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809e410 0x7f7d8811e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.881+0000 7f7d8ca90700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d8809e000 0x7f7d8811e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8da92700 1 -- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d8809e000 msgr2=0x7f7d8811e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8da92700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d8809e000 0x7f7d8811e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8da92700 1 -- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809d6f0 msgr2=0x7f7d8811de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8da92700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809d6f0 0x7f7d8811de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8da92700 1 -- 172.21.15.89:0/3505139730 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8400da00 con 0x7f7d8809e410 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8d291700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809d6f0 0x7f7d8811de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8da92700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809e410 0x7f7d8811e8d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7d80016000 tx=0x7f7d80004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d7dffb700 1 -- 172.21.15.89:0/3505139730 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80012430 con 0x7f7d8809e410 2023-09-30T22:21:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d7dffb700 1 -- 172.21.15.89:0/3505139730 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d80020070 con 0x7f7d8809e410 2023-09-30T22:21:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d7dffb700 1 -- 172.21.15.89:0/3505139730 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80004280 con 0x7f7d8809e410 2023-09-30T22:21:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d80009a70 con 0x7f7d8809e410 2023-09-30T22:21:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.882+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d881210b0 con 0x7f7d8809e410 2023-09-30T22:21:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.883+0000 7f7d7dffb700 1 -- 172.21.15.89:0/3505139730 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d800043e0 con 0x7f7d8809e410 2023-09-30T22:21:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.883+0000 7f7d7dffb700 1 -- 172.21.15.89:0/3505139730 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7d80022070 con 0x7f7d8809e410 2023-09-30T22:21:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.883+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d88121290 con 0x7f7d8809e410 2023-09-30T22:21:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.888+0000 7f7d7dffb700 1 -- 172.21.15.89:0/3505139730 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d88121290 con 0x7f7d8809e410 2023-09-30T22:21:11.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.919+0000 7fb06c6a9700 1 Processor -- start 2023-09-30T22:21:11.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.919+0000 7fb06c6a9700 1 -- start start 2023-09-30T22:21:11.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.920+0000 7fb06c6a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a0a30 0x7fb0640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.920+0000 7fb06c6a9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb0640a1340 0x7fb064133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.920+0000 7fb06c6a9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 0x7fb064134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.920+0000 7fb06c6a9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06409edd0 con 0x7fb0640a1340 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.920+0000 7fb06c6a9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06409e1f0 con 0x7fb064133fa0 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.920+0000 7fb06c6a9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb06409dc50 con 0x7fb0640a0a30 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a0a30 0x7fb0640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a0a30 0x7fb0640a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50804/0 (socket says 172.21.15.89:50804) 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 -- 172.21.15.89:0/2212685343 learned_addr learned my addr 172.21.15.89:0/2212685343 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06ac46700 1 --2- 172.21.15.89:0/2212685343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 0x7fb064134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb069c44700 1 --2- 172.21.15.89:0/2212685343 >> v2:172.21.15.89:3300/0 conn(0x7fb0640a1340 0x7fb064133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 -- 172.21.15.89:0/2212685343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 msgr2=0x7fb064134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:11.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 --2- 172.21.15.89:0/2212685343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 0x7fb064134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 -- 172.21.15.89:0/2212685343 >> v2:172.21.15.89:3300/0 conn(0x7fb0640a1340 msgr2=0x7fb064133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:11.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 --2- 172.21.15.89:0/2212685343 >> v2:172.21.15.89:3300/0 conn(0x7fb0640a1340 0x7fb064133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 -- 172.21.15.89:0/2212685343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0640a1fe0 con 0x7fb0640a0a30 2023-09-30T22:21:11.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb06a445700 1 --2- 172.21.15.89:0/2212685343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a0a30 0x7fb0640a0e00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb060009920 tx=0x7fb060009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=543eb137eed03f33 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:11.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb069443700 1 -- 172.21.15.89:0/2212685343 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060014d70 con 0x7fb0640a0a30 2023-09-30T22:21:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb069443700 1 -- 172.21.15.89:0/2212685343 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb060009e70 con 0x7fb0640a0a30 2023-09-30T22:21:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.921+0000 7fb069443700 1 -- 172.21.15.89:0/2212685343 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000b810 con 0x7fb0640a0a30 2023-09-30T22:21:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.922+0000 7fb06c6a9700 1 -- 172.21.15.89:0/2212685343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a0a30 msgr2=0x7fb0640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.922+0000 7fb06c6a9700 1 --2- 172.21.15.89:0/2212685343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a0a30 0x7fb0640a0e00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb060009920 tx=0x7fb060009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.922+0000 7fb06c6a9700 1 -- 172.21.15.89:0/2212685343 shutdown_connections 2023-09-30T22:21:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.922+0000 7fb06c6a9700 1 --2- 172.21.15.89:0/2212685343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 0x7fb064134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.922+0000 7fb06c6a9700 1 --2- 172.21.15.89:0/2212685343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a0a30 0x7fb0640a0e00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.922+0000 7fb06c6a9700 1 --2- 172.21.15.89:0/2212685343 >> v2:172.21.15.89:3300/0 conn(0x7fb0640a1340 0x7fb064133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.922+0000 7fb06c6a9700 1 -- 172.21.15.89:0/2212685343 >> 172.21.15.89:0/2212685343 conn(0x7fb0640a6550 msgr2=0x7fb0640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:11.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.926+0000 7fb06c6a9700 1 -- 172.21.15.89:0/2212685343 shutdown_connections 2023-09-30T22:21:11.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.926+0000 7fb06c6a9700 1 -- 172.21.15.89:0/2212685343 wait complete. 2023-09-30T22:21:11.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.927+0000 7fb06c6a9700 1 Processor -- start 2023-09-30T22:21:11.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.927+0000 7fb06c6a9700 1 -- start start 2023-09-30T22:21:11.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.927+0000 7fb06c6a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0640a0a30 0x7fb06411dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.927+0000 7fb06c6a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a1340 0x7fb06411e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.927+0000 7fb06c6a9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 0x7fb06411ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:11.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.927+0000 7fb06c6a9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064121af0 con 0x7fb0640a0a30 2023-09-30T22:21:11.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.927+0000 7fb06c6a9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb064120ec0 con 0x7fb064133fa0 2023-09-30T22:21:11.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.927+0000 7fb06c6a9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb064120450 con 0x7fb0640a1340 2023-09-30T22:21:11.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a1340 0x7fb06411e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a1340 0x7fb06411e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50814/0 (socket says 172.21.15.89:50814) 2023-09-30T22:21:11.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 -- 172.21.15.89:0/433972227 learned_addr learned my addr 172.21.15.89:0/433972227 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:11.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb06ac46700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 0x7fb06411ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb06a445700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0640a0a30 0x7fb06411dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:11.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 -- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 msgr2=0x7fb06411ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 0x7fb06411ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 -- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0640a0a30 msgr2=0x7fb06411dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:11.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0640a0a30 0x7fb06411dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:11.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 -- 172.21.15.89:0/433972227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb060009550 con 0x7fb0640a1340 2023-09-30T22:21:11.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb069c44700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a1340 0x7fb06411e510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e810 tx=0x7fb05400eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:11.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb05affd700 1 -- 172.21.15.89:0/433972227 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054013610 con 0x7fb0640a1340 2023-09-30T22:21:11.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb05affd700 1 -- 172.21.15.89:0/433972227 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb054013770 con 0x7fb0640a1340 2023-09-30T22:21:11.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb064121280 con 0x7fb0640a1340 2023-09-30T22:21:11.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb05affd700 1 -- 172.21.15.89:0/433972227 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054013b20 con 0x7fb0640a1340 2023-09-30T22:21:11.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.928+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0640a0220 con 0x7fb0640a1340 2023-09-30T22:21:11.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.929+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb064121670 con 0x7fb0640a1340 2023-09-30T22:21:11.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.930+0000 7fb05affd700 1 -- 172.21.15.89:0/433972227 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb054013c80 con 0x7fb0640a1340 2023-09-30T22:21:11.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.930+0000 7fb05affd700 1 -- 172.21.15.89:0/433972227 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb05403f650 con 0x7fb0640a1340 2023-09-30T22:21:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:11.934+0000 7fb05affd700 1 -- 172.21.15.89:0/433972227 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb054024530 con 0x7fb0640a1340 2023-09-30T22:21:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.048+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7480074ea0 con 0x7f748009a3b0 2023-09-30T22:21:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.048+0000 7f747cff9700 1 -- 172.21.15.89:0/1119443542 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7474029030 con 0x7f748009a3b0 2023-09-30T22:21:12.049 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.050+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f748009a3b0 msgr2=0x7f748011d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.050+0000 7f7486a1a700 1 --2- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f748009a3b0 0x7f748011d620 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7474016000 tx=0x7f7474004750 comp rx=0 tx=0).stop 2023-09-30T22:21:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.050+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 shutdown_connections 2023-09-30T22:21:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.050+0000 7f7486a1a700 1 --2- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7480131810 0x7f748011db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.050+0000 7f7486a1a700 1 --2- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7480099a40 0x7f748011d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.050+0000 7f7486a1a700 1 --2- 172.21.15.89:0/1119443542 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f748009a3b0 0x7f748011d620 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.050+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 >> 172.21.15.89:0/1119443542 conn(0x7f74800a2aa0 msgr2=0x7f748012fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:12.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.051+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 shutdown_connections 2023-09-30T22:21:12.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.051+0000 7f7486a1a700 1 -- 172.21.15.89:0/1119443542 wait complete. 2023-09-30T22:21:12.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.081+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb0b006b9e0 con 0x7fb0b0099a40 2023-09-30T22:21:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.082+0000 7fb0a5ffb700 1 -- 172.21.15.89:0/2639317584 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0ac017070 con 0x7fb0b0099a40 2023-09-30T22:21:12.083 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7d88121290 con 0x7f7d8809e410 2023-09-30T22:21:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b0099a40 msgr2=0x7fb0b011a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b0099a40 0x7fb0b011a900 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9b0 tx=0x7fb0ac00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7f7d7dffb700 1 -- 172.21.15.89:0/3505139730 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d88121290 con 0x7f7d8809e410 2023-09-30T22:21:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 shutdown_connections 2023-09-30T22:21:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0b0131810 0x7fb0b0119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0b0099a40 0x7fb0b011a900 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 --2- 172.21.15.89:0/2639317584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0b009a580 0x7fb0b0118f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 >> 172.21.15.89:0/2639317584 conn(0x7fb0b00a2aa0 msgr2=0x7fb0b012fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:12.087 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 shutdown_connections 2023-09-30T22:21:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.085+0000 7fb0b753e700 1 -- 172.21.15.89:0/2639317584 wait complete. 2023-09-30T22:21:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.087+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809e410 msgr2=0x7f7d8811e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.087+0000 7f7d8f4f5700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809e410 0x7f7d8811e8d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7d80016000 tx=0x7f7d80004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.087+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 shutdown_connections 2023-09-30T22:21:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.087+0000 7f7d8f4f5700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7d8809e410 0x7f7d8811e8d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.087+0000 7f7d8f4f5700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8809d6f0 0x7f7d8811de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.087+0000 7f7d8f4f5700 1 --2- 172.21.15.89:0/3505139730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7d8809e000 0x7f7d8811e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.087+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 >> 172.21.15.89:0/3505139730 conn(0x7f7d88066f50 msgr2=0x7f7d88130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:12.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.092+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 shutdown_connections 2023-09-30T22:21:12.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.092+0000 7f7d8f4f5700 1 -- 172.21.15.89:0/3505139730 wait complete. 2023-09-30T22:21:12.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:12.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:12.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.137+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb064121670 con 0x7fb0640a1340 2023-09-30T22:21:12.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.137+0000 7fb05affd700 1 -- 172.21.15.89:0/433972227 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb064121670 con 0x7fb0640a1340 2023-09-30T22:21:12.138 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:12.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a1340 msgr2=0x7fb06411e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:12.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a1340 0x7fb06411e510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e810 tx=0x7fb05400eb20 comp rx=0 tx=0).stop 2023-09-30T22:21:12.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 shutdown_connections 2023-09-30T22:21:12.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb064133fa0 0x7fb06411ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0640a0a30 0x7fb06411dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 --2- 172.21.15.89:0/433972227 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0640a1340 0x7fb06411e510 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:12.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 >> 172.21.15.89:0/433972227 conn(0x7fb0640a6550 msgr2=0x7fb06409c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:12.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 shutdown_connections 2023-09-30T22:21:12.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:12.139+0000 7fb06c6a9700 1 -- 172.21.15.89:0/433972227 wait complete. 2023-09-30T22:21:12.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:13.065 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:13.104 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:13.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:13.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.207+0000 7f21c6ba2700 1 Processor -- start 2023-09-30T22:21:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c6ba2700 1 -- start start 2023-09-30T22:21:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c6ba2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f21c01338a0 0x7f21c0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c6ba2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009de50 0x7f21c009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c6ba2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c006ef30 con 0x7f21c009e760 2023-09-30T22:21:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c6ba2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c006fef0 con 0x7f21c01338a0 2023-09-30T22:21:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c6ba2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c01334b0 con 0x7f21c009de50 2023-09-30T22:21:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c513f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f21c009e760 0x7f21c013a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55200/0 (socket says 172.21.15.89:55200) 2023-09-30T22:21:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c513f700 1 -- 172.21.15.89:0/3900398564 learned_addr learned my addr 172.21.15.89:0/3900398564 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21c493e700 1 --2- 172.21.15.89:0/3900398564 >> v2:172.21.15.89:3300/0 conn(0x7f21c01338a0 0x7f21c0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.208+0000 7f21bffff700 1 --2- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009de50 0x7f21c009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21c493e700 1 -- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009de50 msgr2=0x7f21c009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21c493e700 1 --2- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009de50 0x7f21c009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21c493e700 1 -- 172.21.15.89:0/3900398564 >> v1:172.21.15.89:6789/0 conn(0x7f21c009e760 legacy=0x7f21c013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21bf7fe700 1 -- 172.21.15.89:0/3900398564 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004994813 0 0) 0x7f21c006ef30 con 0x7f21c009e760 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21c493e700 1 -- 172.21.15.89:0/3900398564 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c006fc30 con 0x7f21c01338a0 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21bffff700 1 --2- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009de50 0x7f21c009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21c493e700 1 --2- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c01338a0 0x7f21c0133c70 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009840 tx=0x7f21b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f60b878b08120f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21bf7fe700 1 -- 172.21.15.89:0/3900398564 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4011830 con 0x7f21c01338a0 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21bf7fe700 1 -- 172.21.15.89:0/3900398564 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b4011990 con 0x7f21c01338a0 2023-09-30T22:21:13.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.209+0000 7f21bf7fe700 1 -- 172.21.15.89:0/3900398564 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4011d20 con 0x7f21c01338a0 2023-09-30T22:21:13.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.210+0000 7f21c6ba2700 1 -- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c01338a0 msgr2=0x7f21c0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.210+0000 7f21c6ba2700 1 --2- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c01338a0 0x7f21c0133c70 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009840 tx=0x7f21b4009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.210+0000 7f21c6ba2700 1 -- 172.21.15.89:0/3900398564 shutdown_connections 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.210+0000 7f21c6ba2700 1 --2- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c01338a0 0x7f21c0133c70 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.210+0000 7f21c6ba2700 1 --2- 172.21.15.89:0/3900398564 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009de50 0x7f21c009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.210+0000 7f21c6ba2700 1 -- 172.21.15.89:0/3900398564 >> 172.21.15.89:0/3900398564 conn(0x7f21c00a2aa0 msgr2=0x7f21c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.210+0000 7f21c6ba2700 1 -- 172.21.15.89:0/3900398564 shutdown_connections 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.210+0000 7f21c6ba2700 1 -- 172.21.15.89:0/3900398564 wait complete. 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.211+0000 7f21c6ba2700 1 Processor -- start 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.211+0000 7f21c6ba2700 1 -- start start 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c6ba2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c009de50 0x7f21c011a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c6ba2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009e760 0x7f21c011abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c6ba2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21c01338a0 0x7f21c011b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c6ba2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c00a0d20 con 0x7f21c009de50 2023-09-30T22:21:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c6ba2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c011d8c0 con 0x7f21c009e760 2023-09-30T22:21:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c6ba2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21c006f640 con 0x7f21c01338a0 2023-09-30T22:21:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c513f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21c01338a0 0x7f21c011b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c513f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21c01338a0 0x7f21c011b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50826/0 (socket says 172.21.15.89:50826) 2023-09-30T22:21:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c513f700 1 -- 172.21.15.89:0/705676844 learned_addr learned my addr 172.21.15.89:0/705676844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21bffff700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009e760 0x7f21c011abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21c493e700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c009de50 0x7f21c011a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21bffff700 1 -- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21c01338a0 msgr2=0x7f21c011b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21bffff700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21c01338a0 0x7f21c011b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21bffff700 1 -- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c009de50 msgr2=0x7f21c011a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21bffff700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c009de50 0x7f21c011a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21bffff700 1 -- 172.21.15.89:0/705676844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0094a0 con 0x7f21c009e760 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.212+0000 7f21bffff700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009e760 0x7f21c011abe0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00e900 tx=0x7f21ac00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.213+0000 7f21bd7fa700 1 -- 172.21.15.89:0/705676844 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac013680 con 0x7f21c009e760 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.213+0000 7f21bd7fa700 1 -- 172.21.15.89:0/705676844 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac0137e0 con 0x7f21c009e760 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.213+0000 7f21bd7fa700 1 -- 172.21.15.89:0/705676844 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac013b70 con 0x7f21c009e760 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.213+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b40094a0 con 0x7f21c009e760 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.213+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c011d470 con 0x7f21c009e760 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.214+0000 7f21bd7fa700 1 -- 172.21.15.89:0/705676844 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f21ac013cd0 con 0x7f21c009e760 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.214+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c011dcd0 con 0x7f21c009e760 2023-09-30T22:21:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.214+0000 7f21bd7fa700 1 -- 172.21.15.89:0/705676844 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f21ac03f9d0 con 0x7f21c009e760 2023-09-30T22:21:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.222+0000 7f21bd7fa700 1 -- 172.21.15.89:0/705676844 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21ac011db0 con 0x7f21c009e760 2023-09-30T22:21:13.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.246+0000 7f6c2d20e700 1 Processor -- start 2023-09-30T22:21:13.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.246+0000 7f6c2d20e700 1 -- start start 2023-09-30T22:21:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2d20e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c28095890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2d20e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c28099680 0x7f6c28099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2d20e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c28137590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2d20e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2809aaf0 con 0x7f6c28099680 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2d20e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2809c280 con 0x7f6c28099ff0 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2d20e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2809b960 con 0x7f6c280954c0 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c28099680 0x7f6c28099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c28099680 0x7f6c28099ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33768/0 (socket says 172.21.15.89:33768) 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2659c700 1 -- 172.21.15.89:0/3880014881 learned_addr learned my addr 172.21.15.89:0/3880014881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c26d9d700 1 --2- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c28095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2759e700 1 --2- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c28137590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2759e700 1 -- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 msgr2=0x7f6c28095890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:13.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2759e700 1 --2- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c28095890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2759e700 1 -- 172.21.15.89:0/3880014881 >> v2:172.21.15.89:3300/0 conn(0x7f6c28099680 msgr2=0x7f6c28099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2759e700 1 --2- 172.21.15.89:0/3880014881 >> v2:172.21.15.89:3300/0 conn(0x7f6c28099680 0x7f6c28099ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2759e700 1 -- 172.21.15.89:0/3880014881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2809a7b0 con 0x7f6c28099ff0 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c2659c700 1 --2- 172.21.15.89:0/3880014881 >> v2:172.21.15.89:3300/0 conn(0x7f6c28099680 0x7f6c28099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.247+0000 7f6c26d9d700 1 --2- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c28095890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2759e700 1 --2- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c28137590 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009d90 tx=0x7f6c2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e04799e17ae6d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c25d9b700 1 -- 172.21.15.89:0/3880014881 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000e070 con 0x7f6c28099ff0 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c25d9b700 1 -- 172.21.15.89:0/3880014881 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000bb20 con 0x7f6c28099ff0 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c25d9b700 1 -- 172.21.15.89:0/3880014881 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20012430 con 0x7f6c28099ff0 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2d20e700 1 -- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 msgr2=0x7f6c28137590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2d20e700 1 --2- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c28137590 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009d90 tx=0x7f6c2000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2d20e700 1 -- 172.21.15.89:0/3880014881 shutdown_connections 2023-09-30T22:21:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2d20e700 1 --2- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c28137590 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2d20e700 1 --2- 172.21.15.89:0/3880014881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c28095890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2d20e700 1 --2- 172.21.15.89:0/3880014881 >> v2:172.21.15.89:3300/0 conn(0x7f6c28099680 0x7f6c28099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2d20e700 1 -- 172.21.15.89:0/3880014881 >> 172.21.15.89:0/3880014881 conn(0x7f6c280a2360 msgr2=0x7f6c280a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.248+0000 7f6c2d20e700 1 -- 172.21.15.89:0/3880014881 shutdown_connections 2023-09-30T22:21:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.249+0000 7f6c2d20e700 1 -- 172.21.15.89:0/3880014881 wait complete. 2023-09-30T22:21:13.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2d20e700 1 Processor -- start 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2d20e700 1 -- start start 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2d20e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c280717a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2d20e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c28099680 0x7f6c28071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2d20e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c281c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2d20e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280713d0 con 0x7f6c28099680 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2d20e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28134db0 con 0x7f6c28099ff0 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2d20e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28135010 con 0x7f6c280954c0 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c26d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c280717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c26d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c280717a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50852/0 (socket says 172.21.15.89:50852) 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c26d9d700 1 -- 172.21.15.89:0/2651467850 learned_addr learned my addr 172.21.15.89:0/2651467850 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.250+0000 7f6c2759e700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c281c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2659c700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c28099680 0x7f6c28071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2659c700 1 -- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 msgr2=0x7f6c280717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2659c700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c280717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2659c700 1 -- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 msgr2=0x7f6c281c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2659c700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c281c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2659c700 1 -- 172.21.15.89:0/2651467850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180094a0 con 0x7f6c28099680 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c26d9d700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c280717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2759e700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c281c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2659c700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c28099680 0x7f6c28071ce0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6c2809c840 tx=0x7f6c1c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7fb2b2194700 1 Processor -- start 2023-09-30T22:21:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c0f7fe700 1 -- 172.21.15.89:0/2651467850 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c004240 con 0x7f6c28099680 2023-09-30T22:21:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c0f7fe700 1 -- 172.21.15.89:0/2651467850 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0043a0 con 0x7f6c28099680 2023-09-30T22:21:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c009550 con 0x7f6c28099680 2023-09-30T22:21:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c0f7fe700 1 -- 172.21.15.89:0/2651467850 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0036a0 con 0x7f6c28099680 2023-09-30T22:21:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c20009a70 con 0x7f6c28099680 2023-09-30T22:21:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.251+0000 7fb2b2194700 1 -- start start 2023-09-30T22:21:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2b2194700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 0x7fb2ac134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2b2194700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2ac1378f0 0x7fb2ac137d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2b2194700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac138260 0x7fb2ac13ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2b2194700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ac095a60 con 0x7fb2ac1378f0 2023-09-30T22:21:13.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2b2194700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac09e800 con 0x7fb2ac138260 2023-09-30T22:21:13.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2b2194700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac094bd0 con 0x7fb2ac095690 2023-09-30T22:21:13.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac138260 0x7fb2ac13ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac138260 0x7fb2ac13ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36930/0 (socket says 172.21.15.89:36930) 2023-09-30T22:21:13.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7fb2abfff700 1 -- 172.21.15.89:0/2679746352 learned_addr learned my addr 172.21.15.89:0/2679746352 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:13.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7fb2aaffd700 1 --2- 172.21.15.89:0/2679746352 >> v2:172.21.15.89:3300/0 conn(0x7fb2ac1378f0 0x7fb2ac137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.252+0000 7f6c0f7fe700 1 -- 172.21.15.89:0/2651467850 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c1c003800 con 0x7f6c28099680 2023-09-30T22:21:13.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7f6c0f7fe700 1 -- 172.21.15.89:0/2651467850 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6c1c01bd80 con 0x7f6c28099680 2023-09-30T22:21:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c28133c50 con 0x7f6c28099680 2023-09-30T22:21:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7fb2ab7fe700 1 --2- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 0x7fb2ac134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7fb2aaffd700 1 -- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 msgr2=0x7fb2ac134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7fb2aaffd700 1 --2- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 0x7fb2ac134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7fb2aaffd700 1 -- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac138260 msgr2=0x7fb2ac13ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7fb2aaffd700 1 --2- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac138260 0x7fb2ac13ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7fb2aaffd700 1 -- 172.21.15.89:0/2679746352 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac0940e0 con 0x7fb2ac1378f0 2023-09-30T22:21:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.253+0000 7fb2aaffd700 1 --2- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac1378f0 0x7fb2ac137d20 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009840 tx=0x7fb29c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40b65a4a051f90b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:13.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.262+0000 7fb2aa7fc700 1 -- 172.21.15.89:0/2679746352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c011830 con 0x7fb2ac1378f0 2023-09-30T22:21:13.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.262+0000 7f6c0f7fe700 1 -- 172.21.15.89:0/2651467850 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c1c013470 con 0x7f6c28099680 2023-09-30T22:21:13.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.263+0000 7fb2aa7fc700 1 -- 172.21.15.89:0/2679746352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c011990 con 0x7fb2ac1378f0 2023-09-30T22:21:13.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.263+0000 7fb2aa7fc700 1 -- 172.21.15.89:0/2679746352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c011d20 con 0x7fb2ac1378f0 2023-09-30T22:21:13.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 -- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac1378f0 msgr2=0x7fb2ac137d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 --2- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac1378f0 0x7fb2ac137d20 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009840 tx=0x7fb29c009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 -- 172.21.15.89:0/2679746352 shutdown_connections 2023-09-30T22:21:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 --2- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac138260 0x7fb2ac13ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 --2- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 0x7fb2ac134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 --2- 172.21.15.89:0/2679746352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac1378f0 0x7fb2ac137d20 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 -- 172.21.15.89:0/2679746352 >> 172.21.15.89:0/2679746352 conn(0x7fb2ac0a2360 msgr2=0x7fb2ac0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 -- 172.21.15.89:0/2679746352 shutdown_connections 2023-09-30T22:21:13.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.264+0000 7fb2b2194700 1 -- 172.21.15.89:0/2679746352 wait complete. 2023-09-30T22:21:13.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.265+0000 7fb2b2194700 1 Processor -- start 2023-09-30T22:21:13.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.266+0000 7fb2b2194700 1 -- start start 2023-09-30T22:21:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.266+0000 7fb2b2194700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 0x7fb2ac1d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.266+0000 7fb2b2194700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac1378f0 0x7fb2ac1d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.266+0000 7fb2b2194700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac138260 0x7fb2ac1d5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.266+0000 7fb2b2194700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac071080 con 0x7fb2ac138260 2023-09-30T22:21:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.266+0000 7fb2b2194700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac071290 con 0x7fb2ac1378f0 2023-09-30T22:21:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.266+0000 7fb2b2194700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac072520 con 0x7fb2ac095690 2023-09-30T22:21:13.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac138260 0x7fb2ac1d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac138260 0x7fb2ac1d5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33802/0 (socket says 172.21.15.89:33802) 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 -- 172.21.15.89:0/938684338 learned_addr learned my addr 172.21.15.89:0/938684338 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2aaffd700 1 --2- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac1378f0 0x7fb2ac1d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 -- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 msgr2=0x7fb2ac1d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 --2- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 0x7fb2ac1d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 -- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac1378f0 msgr2=0x7fb2ac1d53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 --2- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac1378f0 0x7fb2ac1d53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 -- 172.21.15.89:0/938684338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0094a0 con 0x7fb2ac138260 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2abfff700 1 --2- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac138260 0x7fb2ac1d5900 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400efc0 tx=0x7fb2a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb293fff700 1 -- 172.21.15.89:0/938684338 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4003c20 con 0x7fb2ac138260 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb293fff700 1 -- 172.21.15.89:0/938684338 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a400f040 con 0x7fb2ac138260 2023-09-30T22:21:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac0946d0 con 0x7fb2ac138260 2023-09-30T22:21:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb293fff700 1 -- 172.21.15.89:0/938684338 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4009e60 con 0x7fb2ac138260 2023-09-30T22:21:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.267+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac071600 con 0x7fb2ac138260 2023-09-30T22:21:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.269+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac074ea0 con 0x7fb2ac138260 2023-09-30T22:21:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.269+0000 7fb293fff700 1 -- 172.21.15.89:0/938684338 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb2a400c930 con 0x7fb2ac138260 2023-09-30T22:21:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.269+0000 7fb293fff700 1 -- 172.21.15.89:0/938684338 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb2a4040750 con 0x7fb2ac138260 2023-09-30T22:21:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.274+0000 7fb293fff700 1 -- 172.21.15.89:0/938684338 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2a4013db0 con 0x7fb2ac138260 2023-09-30T22:21:13.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.303+0000 7fd74eaba700 1 Processor -- start 2023-09-30T22:21:13.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.303+0000 7fd74eaba700 1 -- start start 2023-09-30T22:21:13.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74eaba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd74809e930 0x7fd74809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74eaba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7481342f0 0x7fd74813be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74eaba700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74806c9f0 con 0x7fd74809f240 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74eaba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74806c670 con 0x7fd7481342f0 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74eaba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd74806bc90 con 0x7fd74809e930 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74d057700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7481342f0 0x7fd74813be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74c856700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd74809e930 0x7fd74809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd747fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd74809f240 0x7fd748133be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55214/0 (socket says 172.21.15.89:55214) 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd747fff700 1 -- 172.21.15.89:0/3167130142 learned_addr learned my addr 172.21.15.89:0/3167130142 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74d057700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7481342f0 0x7fd74813be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33806/0 (socket says 172.21.15.89:33806) 2023-09-30T22:21:13.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74c856700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd74809e930 0x7fd74809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50892/0 (socket says 172.21.15.89:50892) 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74c856700 1 -- 172.21.15.89:0/3167130142 >> v2:172.21.15.89:3300/0 conn(0x7fd7481342f0 msgr2=0x7fd74813be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74c856700 1 --2- 172.21.15.89:0/3167130142 >> v2:172.21.15.89:3300/0 conn(0x7fd7481342f0 0x7fd74813be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd7477fe700 1 -- 172.21.15.89:0/3167130142 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712232937 0 0) 0x7fd734001110 con 0x7fd74809f240 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74c856700 1 -- 172.21.15.89:0/3167130142 >> v1:172.21.15.89:6789/0 conn(0x7fd74809f240 legacy=0x7fd748133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.304+0000 7fd74c856700 1 -- 172.21.15.89:0/3167130142 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74806d860 con 0x7fd74809e930 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.305+0000 7fd74c856700 1 --2- 172.21.15.89:0/3167130142 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd74809e930 0x7fd74809ed00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009840 tx=0x7fd73c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3954e22602438f17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.305+0000 7fd7477fe700 1 -- 172.21.15.89:0/3167130142 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c011830 con 0x7fd74809e930 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.305+0000 7fd7477fe700 1 -- 172.21.15.89:0/3167130142 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c011990 con 0x7fd74809e930 2023-09-30T22:21:13.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.305+0000 7fd7477fe700 1 -- 172.21.15.89:0/3167130142 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c011d20 con 0x7fd74809e930 2023-09-30T22:21:13.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.306+0000 7fd74eaba700 1 -- 172.21.15.89:0/3167130142 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd74809e930 msgr2=0x7fd74809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.306+0000 7fd74eaba700 1 --2- 172.21.15.89:0/3167130142 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd74809e930 0x7fd74809ed00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009840 tx=0x7fd73c009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:13.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.306+0000 7fd74eaba700 1 -- 172.21.15.89:0/3167130142 shutdown_connections 2023-09-30T22:21:13.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.306+0000 7fd74eaba700 1 --2- 172.21.15.89:0/3167130142 >> v2:172.21.15.89:3300/0 conn(0x7fd7481342f0 0x7fd74813be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.306+0000 7fd74eaba700 1 --2- 172.21.15.89:0/3167130142 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd74809e930 0x7fd74809ed00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.306+0000 7fd74eaba700 1 -- 172.21.15.89:0/3167130142 >> 172.21.15.89:0/3167130142 conn(0x7fd7480a4210 msgr2=0x7fd7480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:13.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.306+0000 7fd74eaba700 1 -- 172.21.15.89:0/3167130142 shutdown_connections 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.306+0000 7fd74eaba700 1 -- 172.21.15.89:0/3167130142 wait complete. 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.307+0000 7fd74eaba700 1 Processor -- start 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.307+0000 7fd74eaba700 1 -- start start 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.307+0000 7fd74eaba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd74809e930 0x7fd7481d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.307+0000 7fd74eaba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd74809f240 0x7fd7481d58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.307+0000 7fd74eaba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7481342f0 0x7fd7481d1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.307+0000 7fd74eaba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74806cbb0 con 0x7fd74809f240 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.307+0000 7fd74eaba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd748131a90 con 0x7fd74809e930 2023-09-30T22:21:13.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.307+0000 7fd74eaba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7481325e0 con 0x7fd7481342f0 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd74809e930 0x7fd7481d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd74809e930 0x7fd7481d53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36936/0 (socket says 172.21.15.89:36936) 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 -- 172.21.15.89:0/242011466 learned_addr learned my addr 172.21.15.89:0/242011466 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd747fff700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd74809f240 0x7fd7481d58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74d057700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7481342f0 0x7fd7481d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 -- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7481342f0 msgr2=0x7fd7481d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7481342f0 0x7fd7481d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 -- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd74809f240 msgr2=0x7fd7481d58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd74809f240 0x7fd7481d58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 -- 172.21.15.89:0/242011466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c0094a0 con 0x7fd74809e930 2023-09-30T22:21:13.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.308+0000 7fd74c856700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd74809e930 0x7fd7481d53b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009fd0 tx=0x7fd73c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:13.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.309+0000 7fd7457fa700 1 -- 172.21.15.89:0/242011466 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0036a0 con 0x7fd74809e930 2023-09-30T22:21:13.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.309+0000 7fd7457fa700 1 -- 172.21.15.89:0/242011466 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c01f070 con 0x7fd74809e930 2023-09-30T22:21:13.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.309+0000 7fd7457fa700 1 -- 172.21.15.89:0/242011466 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c003a60 con 0x7fd74809e930 2023-09-30T22:21:13.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.309+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd748132b80 con 0x7fd74809e930 2023-09-30T22:21:13.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.309+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd748130e50 con 0x7fd74809e930 2023-09-30T22:21:13.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.310+0000 7fd7457fa700 1 -- 172.21.15.89:0/242011466 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd73c003bc0 con 0x7fd74809e930 2023-09-30T22:21:13.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.310+0000 7fd7457fa700 1 -- 172.21.15.89:0/242011466 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd73c021070 con 0x7fd74809e930 2023-09-30T22:21:13.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.310+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd748130e50 con 0x7fd74809e930 2023-09-30T22:21:13.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.319+0000 7fd7457fa700 1 -- 172.21.15.89:0/242011466 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd73c040e60 con 0x7fd74809e930 2023-09-30T22:21:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.458+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f21c0074ea0 con 0x7f21c009e760 2023-09-30T22:21:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.459+0000 7f21bd7fa700 1 -- 172.21.15.89:0/705676844 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21ac021020 con 0x7f21c009e760 2023-09-30T22:21:13.460 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:13.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.460+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c28074ea0 con 0x7f6c28099680 2023-09-30T22:21:13.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.460+0000 7f6c0f7fe700 1 -- 172.21.15.89:0/2651467850 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c1c01bba0 con 0x7f6c28099680 2023-09-30T22:21:13.461 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:13.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c28099680 msgr2=0x7f6c28071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c28099680 0x7f6c28071ce0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6c2809c840 tx=0x7f6c1c00ec80 comp rx=0 tx=0).stop 2023-09-30T22:21:13.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009e760 msgr2=0x7f21c011abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009e760 0x7f21c011abe0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00e900 tx=0x7f21ac00ec10 comp rx=0 tx=0).stop 2023-09-30T22:21:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 shutdown_connections 2023-09-30T22:21:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c28099ff0 0x7f6c281c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c280954c0 0x7f6c280717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 --2- 172.21.15.89:0/2651467850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c28099680 0x7f6c28071ce0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 >> 172.21.15.89:0/2651467850 conn(0x7f6c280a2360 msgr2=0x7f6c2809d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 shutdown_connections 2023-09-30T22:21:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21c01338a0 0x7f21c011b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21c009de50 0x7f21c011a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 --2- 172.21.15.89:0/705676844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f21c009e760 0x7f21c011abe0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 >> 172.21.15.89:0/705676844 conn(0x7f21c00a2aa0 msgr2=0x7f21c0130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 shutdown_connections 2023-09-30T22:21:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f6c2d20e700 1 -- 172.21.15.89:0/2651467850 wait complete. 2023-09-30T22:21:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 shutdown_connections 2023-09-30T22:21:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.462+0000 7f21c6ba2700 1 -- 172.21.15.89:0/705676844 wait complete. 2023-09-30T22:21:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.475+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb2ac138700 con 0x7fb2ac138260 2023-09-30T22:21:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.476+0000 7fb293fff700 1 -- 172.21.15.89:0/938684338 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2a402c090 con 0x7fb2ac138260 2023-09-30T22:21:13.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:13.477 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac138260 msgr2=0x7fb2ac1d5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 --2- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac138260 0x7fb2ac1d5900 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400efc0 tx=0x7fb2a400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 shutdown_connections 2023-09-30T22:21:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 --2- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ac138260 0x7fb2ac1d5900 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 --2- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2ac095690 0x7fb2ac1d4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 --2- 172.21.15.89:0/938684338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2ac1378f0 0x7fb2ac1d53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 >> 172.21.15.89:0/938684338 conn(0x7fb2ac0a2360 msgr2=0x7fb2ac136db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 shutdown_connections 2023-09-30T22:21:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.478+0000 7fb2b2194700 1 -- 172.21.15.89:0/938684338 wait complete. 2023-09-30T22:21:13.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:13.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:13.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.541+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd748074ea0 con 0x7fd74809e930 2023-09-30T22:21:13.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.541+0000 7fd7457fa700 1 -- 172.21.15.89:0/242011466 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd73c035090 con 0x7fd74809e930 2023-09-30T22:21:13.542 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:13.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd74809e930 msgr2=0x7fd7481d53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:13.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd74809e930 0x7fd7481d53b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009fd0 tx=0x7fd73c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:13.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 shutdown_connections 2023-09-30T22:21:13.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7481342f0 0x7fd7481d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd74809e930 0x7fd7481d53b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 --2- 172.21.15.89:0/242011466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd74809f240 0x7fd7481d58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:13.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 >> 172.21.15.89:0/242011466 conn(0x7fd7480a4210 msgr2=0x7fd7480722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:13.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 shutdown_connections 2023-09-30T22:21:13.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:13.543+0000 7fd74eaba700 1 -- 172.21.15.89:0/242011466 wait complete. 2023-09-30T22:21:13.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:14.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:14.482 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:14.492 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:14.557 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.621+0000 7f8ff0b3a700 1 Processor -- start 2023-09-30T22:21:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.621+0000 7f8ff0b3a700 1 -- start start 2023-09-30T22:21:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.621+0000 7f8ff0b3a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec09e1d0 0x7f8fec138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.621+0000 7f8ff0b3a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec139170 0x7f8fec13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.621+0000 7f8ff0b3a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec09b120 con 0x7f8fec09d6f0 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.621+0000 7f8ff0b3a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec06b460 con 0x7f8fec139170 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.621+0000 7f8ff0b3a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec09a7e0 con 0x7f8fec09e1d0 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec09e1d0 0x7f8fec138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec09e1d0 0x7f8fec138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50922/0 (socket says 172.21.15.89:50922) 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe9d9b700 1 -- 172.21.15.89:0/4106674422 learned_addr learned my addr 172.21.15.89:0/4106674422 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fea59c700 1 --1- 172.21.15.89:0/4106674422 >> v1:172.21.15.89:6789/0 conn(0x7f8fec09d6f0 0x7f8fec09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55222/0 (socket says 172.21.15.89:55222) 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fead9d700 1 --2- 172.21.15.89:0/4106674422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec139170 0x7f8fec13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe9d9b700 1 -- 172.21.15.89:0/4106674422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec139170 msgr2=0x7f8fec13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe9d9b700 1 --2- 172.21.15.89:0/4106674422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec139170 0x7f8fec13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe9d9b700 1 -- 172.21.15.89:0/4106674422 >> v1:172.21.15.89:6789/0 conn(0x7f8fec09d6f0 legacy=0x7f8fec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe9d9b700 1 -- 172.21.15.89:0/4106674422 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec09a500 con 0x7f8fec09e1d0 2023-09-30T22:21:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe959a700 1 -- 172.21.15.89:0/4106674422 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505024205 0 0) 0x7f8fec09b120 con 0x7f8fec09d6f0 2023-09-30T22:21:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe9d9b700 1 --2- 172.21.15.89:0/4106674422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec09e1d0 0x7f8fec138c30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009950 tx=0x7f8fe0009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07a7caa51365d10 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe959a700 1 -- 172.21.15.89:0/4106674422 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0011830 con 0x7f8fec09e1d0 2023-09-30T22:21:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe959a700 1 -- 172.21.15.89:0/4106674422 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe0011990 con 0x7f8fec09e1d0 2023-09-30T22:21:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.622+0000 7f8fe959a700 1 -- 172.21.15.89:0/4106674422 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0011d40 con 0x7f8fec09e1d0 2023-09-30T22:21:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.623+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/4106674422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec09e1d0 msgr2=0x7f8fec138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.623+0000 7f8ff0b3a700 1 --2- 172.21.15.89:0/4106674422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec09e1d0 0x7f8fec138c30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009950 tx=0x7f8fe0009c60 comp rx=0 tx=0).stop 2023-09-30T22:21:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.623+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/4106674422 shutdown_connections 2023-09-30T22:21:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.623+0000 7f8ff0b3a700 1 --2- 172.21.15.89:0/4106674422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec139170 0x7f8fec13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.623+0000 7f8ff0b3a700 1 --2- 172.21.15.89:0/4106674422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec09e1d0 0x7f8fec138c30 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.623+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/4106674422 >> 172.21.15.89:0/4106674422 conn(0x7f8fec066f50 msgr2=0x7f8fec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.624+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/4106674422 shutdown_connections 2023-09-30T22:21:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.624+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/4106674422 wait complete. 2023-09-30T22:21:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.625+0000 7f8ff0b3a700 1 Processor -- start 2023-09-30T22:21:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.625+0000 7f8ff0b3a700 1 -- start start 2023-09-30T22:21:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8ff0b3a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec09d6f0 0x7f8fec121800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8ff0b3a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fec09e1d0 0x7f8fec121d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8ff0b3a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec139170 0x7f8fec122280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8ff0b3a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec125080 con 0x7f8fec09e1d0 2023-09-30T22:21:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8ff0b3a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec125e60 con 0x7f8fec09d6f0 2023-09-30T22:21:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8ff0b3a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec137650 con 0x7f8fec139170 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fead9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec139170 0x7f8fec122280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fead9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec139170 0x7f8fec122280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50924/0 (socket says 172.21.15.89:50924) 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fead9d700 1 -- 172.21.15.89:0/2996099182 learned_addr learned my addr 172.21.15.89:0/2996099182 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fe9d9b700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fec09e1d0 0x7f8fec121d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fea59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec09d6f0 0x7f8fec121800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fead9d700 1 -- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec09d6f0 msgr2=0x7f8fec121800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fead9d700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec09d6f0 0x7f8fec121800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fead9d700 1 -- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fec09e1d0 msgr2=0x7f8fec121d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fead9d700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fec09e1d0 0x7f8fec121d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fead9d700 1 -- 172.21.15.89:0/2996099182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0009550 con 0x7f8fec139170 2023-09-30T22:21:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fe9d9b700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fec09e1d0 0x7f8fec121d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.626+0000 7f8fea59c700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec09d6f0 0x7f8fec121800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.627+0000 7f8fead9d700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec139170 0x7f8fec122280 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009dd0 tx=0x7f8fe400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.627+0000 7f8fdaffd700 1 -- 172.21.15.89:0/2996099182 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4009c40 con 0x7f8fec139170 2023-09-30T22:21:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.627+0000 7f8fdaffd700 1 -- 172.21.15.89:0/2996099182 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400f040 con 0x7f8fec139170 2023-09-30T22:21:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.627+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec123370 con 0x7f8fec139170 2023-09-30T22:21:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.627+0000 7f8fdaffd700 1 -- 172.21.15.89:0/2996099182 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4014480 con 0x7f8fec139170 2023-09-30T22:21:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.627+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec123b60 con 0x7f8fec139170 2023-09-30T22:21:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.628+0000 7f8fdaffd700 1 -- 172.21.15.89:0/2996099182 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8fe40145e0 con 0x7f8fec139170 2023-09-30T22:21:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.628+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec125ad0 con 0x7f8fec139170 2023-09-30T22:21:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.628+0000 7f8fdaffd700 1 -- 172.21.15.89:0/2996099182 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8fe403fee0 con 0x7f8fec139170 2023-09-30T22:21:14.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.633+0000 7f8fdaffd700 1 -- 172.21.15.89:0/2996099182 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8fe40118b0 con 0x7f8fec139170 2023-09-30T22:21:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.664+0000 7ff7338ba700 1 Processor -- start 2023-09-30T22:21:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.664+0000 7ff7338ba700 1 -- start start 2023-09-30T22:21:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.664+0000 7ff7338ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09f410 0x7ff72c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff7338ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff72c136fe0 0x7ff72c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff7338ba700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c070940 con 0x7ff72c09e930 2023-09-30T22:21:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff7338ba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72c06fd60 con 0x7ff72c136fe0 2023-09-30T22:21:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff7338ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c06fa80 con 0x7ff72c09f410 2023-09-30T22:21:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff730e55700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09f410 0x7ff72c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff730e55700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09f410 0x7ff72c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50928/0 (socket says 172.21.15.89:50928) 2023-09-30T22:21:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff730e55700 1 -- 172.21.15.89:0/2036165865 learned_addr learned my addr 172.21.15.89:0/2036165865 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff731e57700 1 --2- 172.21.15.89:0/2036165865 >> v2:172.21.15.89:3300/0 conn(0x7ff72c136fe0 0x7ff72c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff730e55700 1 -- 172.21.15.89:0/2036165865 >> v2:172.21.15.89:3300/0 conn(0x7ff72c136fe0 msgr2=0x7ff72c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff730e55700 1 --2- 172.21.15.89:0/2036165865 >> v2:172.21.15.89:3300/0 conn(0x7ff72c136fe0 0x7ff72c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff730e55700 1 -- 172.21.15.89:0/2036165865 >> v1:172.21.15.89:6789/0 conn(0x7ff72c09e930 legacy=0x7ff72c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff730e55700 1 -- 172.21.15.89:0/2036165865 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c09fb60 con 0x7ff72c09f410 2023-09-30T22:21:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff730e55700 1 --2- 172.21.15.89:0/2036165865 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09f410 0x7ff72c136aa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0098f0 tx=0x7ff71c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c5384918d0577a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff723fff700 1 -- 172.21.15.89:0/2036165865 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c011830 con 0x7ff72c09f410 2023-09-30T22:21:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff723fff700 1 -- 172.21.15.89:0/2036165865 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c011990 con 0x7ff72c09f410 2023-09-30T22:21:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.665+0000 7ff723fff700 1 -- 172.21.15.89:0/2036165865 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c011d20 con 0x7ff72c09f410 2023-09-30T22:21:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.666+0000 7f000f410700 1 Processor -- start 2023-09-30T22:21:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.666+0000 7ff7338ba700 1 -- 172.21.15.89:0/2036165865 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09f410 msgr2=0x7ff72c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.666+0000 7ff7338ba700 1 --2- 172.21.15.89:0/2036165865 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09f410 0x7ff72c136aa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0098f0 tx=0x7ff71c009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.666+0000 7ff7338ba700 1 -- 172.21.15.89:0/2036165865 shutdown_connections 2023-09-30T22:21:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.666+0000 7ff7338ba700 1 --2- 172.21.15.89:0/2036165865 >> v2:172.21.15.89:3300/0 conn(0x7ff72c136fe0 0x7ff72c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.666+0000 7ff7338ba700 1 --2- 172.21.15.89:0/2036165865 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c09f410 0x7ff72c136aa0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.666+0000 7ff7338ba700 1 -- 172.21.15.89:0/2036165865 >> 172.21.15.89:0/2036165865 conn(0x7ff72c0a4210 msgr2=0x7ff72c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7ff7338ba700 1 -- 172.21.15.89:0/2036165865 shutdown_connections 2023-09-30T22:21:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7ff7338ba700 1 -- 172.21.15.89:0/2036165865 wait complete. 2023-09-30T22:21:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.666+0000 7f000f410700 1 -- start start 2023-09-30T22:21:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000f410700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809d6f0 0x7f000809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000f410700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809e000 0x7f0008130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000f410700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000809f130 con 0x7f000809e410 2023-09-30T22:21:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000f410700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000809f520 con 0x7f000809e000 2023-09-30T22:21:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000f410700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f000809e930 con 0x7f000809d6f0 2023-09-30T22:21:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000d1ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809d6f0 0x7f000809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000d1ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809d6f0 0x7f000809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50944/0 (socket says 172.21.15.89:50944) 2023-09-30T22:21:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000d1ac700 1 -- 172.21.15.89:0/4274090299 learned_addr learned my addr 172.21.15.89:0/4274090299 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.667+0000 7f000c9ab700 1 --2- 172.21.15.89:0/4274090299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809e000 0x7f0008130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7f000d1ac700 1 -- 172.21.15.89:0/4274090299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809e000 msgr2=0x7f0008130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7f000d1ac700 1 --2- 172.21.15.89:0/4274090299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809e000 0x7f0008130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7f000d1ac700 1 -- 172.21.15.89:0/4274090299 >> v1:172.21.15.89:6789/0 conn(0x7f000809e410 legacy=0x7f0008130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7f000d1ac700 1 -- 172.21.15.89:0/4274090299 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000809b2d0 con 0x7f000809d6f0 2023-09-30T22:21:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7ff7338ba700 1 Processor -- start 2023-09-30T22:21:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7ff7338ba700 1 -- start start 2023-09-30T22:21:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7f000d1ac700 1 --2- 172.21.15.89:0/4274090299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809d6f0 0x7f000809dac0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0004009950 tx=0x7f0004009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f934037da6d64e2e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7efffffff700 1 -- 172.21.15.89:0/4274090299 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004004230 con 0x7f000809d6f0 2023-09-30T22:21:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7efffffff700 1 -- 172.21.15.89:0/4274090299 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0004004390 con 0x7f000809d6f0 2023-09-30T22:21:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7efffffff700 1 -- 172.21.15.89:0/4274090299 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040036a0 con 0x7f000809d6f0 2023-09-30T22:21:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7ff7338ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09e930 0x7ff72c06c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7ff7338ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09f410 0x7ff72c06c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7ff7338ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c136fe0 0x7ff72c1c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7ff7338ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c06bd70 con 0x7ff72c09f410 2023-09-30T22:21:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7ff7338ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c122080 con 0x7ff72c09e930 2023-09-30T22:21:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.668+0000 7ff7338ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c132420 con 0x7ff72c136fe0 2023-09-30T22:21:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09e930 0x7ff72c06c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09e930 0x7ff72c06c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36960/0 (socket says 172.21.15.89:36960) 2023-09-30T22:21:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 -- 172.21.15.89:0/4101264945 learned_addr learned my addr 172.21.15.89:0/4101264945 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff730e55700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09f410 0x7ff72c06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731e57700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c136fe0 0x7ff72c1c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7f000f410700 1 -- 172.21.15.89:0/4274090299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809d6f0 msgr2=0x7f000809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7f000f410700 1 --2- 172.21.15.89:0/4274090299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809d6f0 0x7f000809dac0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0004009950 tx=0x7f0004009c60 comp rx=0 tx=0).stop 2023-09-30T22:21:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 -- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c136fe0 msgr2=0x7ff72c1c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c136fe0 0x7ff72c1c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 -- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09f410 msgr2=0x7ff72c06c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09f410 0x7ff72c06c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 -- 172.21.15.89:0/4101264945 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c009550 con 0x7ff72c09e930 2023-09-30T22:21:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff730e55700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09f410 0x7ff72c06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7f000f410700 1 -- 172.21.15.89:0/4274090299 shutdown_connections 2023-09-30T22:21:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7f000f410700 1 --2- 172.21.15.89:0/4274090299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809d6f0 0x7f000809dac0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7f000f410700 1 --2- 172.21.15.89:0/4274090299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809e000 0x7f0008130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7f000f410700 1 -- 172.21.15.89:0/4274090299 >> 172.21.15.89:0/4274090299 conn(0x7f0008066f50 msgr2=0x7f000809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7f000f410700 1 -- 172.21.15.89:0/4274090299 shutdown_connections 2023-09-30T22:21:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7f000f410700 1 -- 172.21.15.89:0/4274090299 wait complete. 2023-09-30T22:21:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff731656700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09e930 0x7ff72c06c0e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c4e0 tx=0x7ff72800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.669+0000 7ff721ffb700 1 -- 172.21.15.89:0/4101264945 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7280052d0 con 0x7ff72c09e930 2023-09-30T22:21:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.670+0000 7ff721ffb700 1 -- 172.21.15.89:0/4101264945 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff728005430 con 0x7ff72c09e930 2023-09-30T22:21:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.670+0000 7ff721ffb700 1 -- 172.21.15.89:0/4101264945 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7280038c0 con 0x7ff72c09e930 2023-09-30T22:21:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.670+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c120c40 con 0x7ff72c09e930 2023-09-30T22:21:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.670+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c122af0 con 0x7ff72c09e930 2023-09-30T22:21:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.670+0000 7f000f410700 1 Processor -- start 2023-09-30T22:21:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.670+0000 7f000f410700 1 -- start start 2023-09-30T22:21:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000f410700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809d6f0 0x7f00081b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000f410700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809e000 0x7f00081b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000f410700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f000809e410 0x7f000812d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000f410700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00081b9cf0 con 0x7f000809e410 2023-09-30T22:21:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000f410700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00081b9ed0 con 0x7f000809d6f0 2023-09-30T22:21:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000f410700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00081c6f30 con 0x7f000809e000 2023-09-30T22:21:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f000809e410 0x7f000812d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f000809e410 0x7f000812d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33836/0 (socket says 172.21.15.89:33836) 2023-09-30T22:21:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 -- 172.21.15.89:0/2480133939 learned_addr learned my addr 172.21.15.89:0/2480133939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c121ec0 con 0x7ff72c09e930 2023-09-30T22:21:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000c9ab700 1 --2- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809e000 0x7f00081b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 -- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809e000 msgr2=0x7f00081b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 --2- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809e000 0x7f00081b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 -- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809d6f0 msgr2=0x7f00081b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 --2- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809d6f0 0x7f00081b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 -- 172.21.15.89:0/2480133939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004009550 con 0x7f000809e410 2023-09-30T22:21:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.671+0000 7f000d9ad700 1 --2- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f000809e410 0x7f000812d180 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f000000c7a0 tx=0x7f000000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.672+0000 7efffdffb700 1 -- 172.21.15.89:0/2480133939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000003e10 con 0x7f000809e410 2023-09-30T22:21:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.672+0000 7efffdffb700 1 -- 172.21.15.89:0/2480133939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0000005070 con 0x7f000809e410 2023-09-30T22:21:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.672+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00081b68c0 con 0x7f000809e410 2023-09-30T22:21:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.672+0000 7efffdffb700 1 -- 172.21.15.89:0/2480133939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000005420 con 0x7f000809e410 2023-09-30T22:21:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.672+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000809c700 con 0x7f000809e410 2023-09-30T22:21:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.672+0000 7ff721ffb700 1 -- 172.21.15.89:0/4101264945 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff728003a20 con 0x7ff72c09e930 2023-09-30T22:21:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.672+0000 7ff721ffb700 1 -- 172.21.15.89:0/4101264945 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff72803efe0 con 0x7ff72c09e930 2023-09-30T22:21:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.673+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00081b9b50 con 0x7f000809e410 2023-09-30T22:21:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.673+0000 7efffdffb700 1 -- 172.21.15.89:0/2480133939 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0000005580 con 0x7f000809e410 2023-09-30T22:21:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.674+0000 7efffdffb700 1 -- 172.21.15.89:0/2480133939 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f000003eac0 con 0x7f000809e410 2023-09-30T22:21:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.677+0000 7ff721ffb700 1 -- 172.21.15.89:0/4101264945 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff728018260 con 0x7ff72c09e930 2023-09-30T22:21:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.678+0000 7efffdffb700 1 -- 172.21.15.89:0/2480133939 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f00081b9b50 con 0x7f000809e410 2023-09-30T22:21:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd009f8a700 1 Processor -- start 2023-09-30T22:21:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd009f8a700 1 -- start start 2023-09-30T22:21:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd009f8a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd00409d6f0 0x7fd00409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd009f8a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 0x7fd004130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd009f8a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409e410 0x7fd004130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd009f8a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00409f520 con 0x7fd00409d6f0 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd009f8a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00409f130 con 0x7fd00409e410 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd009f8a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd00409e930 con 0x7fd00409e000 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd003fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409e410 0x7fd004130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd0037fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd00409d6f0 0x7fd00409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd003fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409e410 0x7fd004130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:36984/0 (socket says 172.21.15.89:36984) 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd0037fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd00409d6f0 0x7fd00409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33846/0 (socket says 172.21.15.89:33846) 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd003fff700 1 -- 172.21.15.89:0/3773668902 learned_addr learned my addr 172.21.15.89:0/3773668902 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.714+0000 7fd002ffd700 1 --2- 172.21.15.89:0/3773668902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 0x7fd004130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd003fff700 1 -- 172.21.15.89:0/3773668902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 msgr2=0x7fd004130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd003fff700 1 --2- 172.21.15.89:0/3773668902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 0x7fd004130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd003fff700 1 -- 172.21.15.89:0/3773668902 >> v2:172.21.15.89:3300/0 conn(0x7fd00409d6f0 msgr2=0x7fd00409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd003fff700 1 --2- 172.21.15.89:0/3773668902 >> v2:172.21.15.89:3300/0 conn(0x7fd00409d6f0 0x7fd00409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd003fff700 1 -- 172.21.15.89:0/3773668902 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00409b2d0 con 0x7fd00409e410 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd0037fe700 1 --2- 172.21.15.89:0/3773668902 >> v2:172.21.15.89:3300/0 conn(0x7fd00409d6f0 0x7fd00409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd003fff700 1 --2- 172.21.15.89:0/3773668902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409e410 0x7fd004130ba0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009e50 tx=0x7fcffc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee62949f5a905b44 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd0027fc700 1 -- 172.21.15.89:0/3773668902 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc00e070 con 0x7fd00409e410 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd0027fc700 1 -- 172.21.15.89:0/3773668902 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc00b910 con 0x7fd00409e410 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.715+0000 7fd0027fc700 1 -- 172.21.15.89:0/3773668902 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc00bcc0 con 0x7fd00409e410 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 -- 172.21.15.89:0/3773668902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409e410 msgr2=0x7fd004130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 --2- 172.21.15.89:0/3773668902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409e410 0x7fd004130ba0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009e50 tx=0x7fcffc00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 -- 172.21.15.89:0/3773668902 shutdown_connections 2023-09-30T22:21:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 --2- 172.21.15.89:0/3773668902 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409e410 0x7fd004130ba0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 --2- 172.21.15.89:0/3773668902 >> v2:172.21.15.89:3300/0 conn(0x7fd00409d6f0 0x7fd00409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 --2- 172.21.15.89:0/3773668902 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 0x7fd004130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 -- 172.21.15.89:0/3773668902 >> 172.21.15.89:0/3773668902 conn(0x7fd004066f50 msgr2=0x7fd00409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 -- 172.21.15.89:0/3773668902 shutdown_connections 2023-09-30T22:21:14.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.716+0000 7fd009f8a700 1 -- 172.21.15.89:0/3773668902 wait complete. 2023-09-30T22:21:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.717+0000 7fd009f8a700 1 Processor -- start 2023-09-30T22:21:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.718+0000 7fd009f8a700 1 -- start start 2023-09-30T22:21:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.718+0000 7fd009f8a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409d6f0 0x7fd00412c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.718+0000 7fd009f8a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 0x7fd00412caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.718+0000 7fd009f8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd00409e410 0x7fd00412d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.718+0000 7fd009f8a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004135b80 con 0x7fd00409e410 2023-09-30T22:21:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.718+0000 7fd009f8a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041368f0 con 0x7fd00409d6f0 2023-09-30T22:21:14.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.718+0000 7fd009f8a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd004072560 con 0x7fd00409e000 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.718+0000 7fd003fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd00409e410 0x7fd00412d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd003fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd00409e410 0x7fd00412d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33862/0 (socket says 172.21.15.89:33862) 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd0037fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409d6f0 0x7fd00412c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd003fff700 1 -- 172.21.15.89:0/1353237947 learned_addr learned my addr 172.21.15.89:0/1353237947 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd002ffd700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 0x7fd00412caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd003fff700 1 -- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 msgr2=0x7fd00412caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd003fff700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 0x7fd00412caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd003fff700 1 -- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409d6f0 msgr2=0x7fd00412c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd003fff700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409d6f0 0x7fd00412c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd003fff700 1 -- 172.21.15.89:0/1353237947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40094a0 con 0x7fd00409e410 2023-09-30T22:21:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd0037fe700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409d6f0 0x7fd00412c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd003fff700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd00409e410 0x7fd00412d030 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcffc003f50 tx=0x7fcffc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fcfebfff700 1 -- 172.21.15.89:0/1353237947 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc004450 con 0x7fd00409e410 2023-09-30T22:21:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fcfebfff700 1 -- 172.21.15.89:0/1353237947 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc0045b0 con 0x7fd00409e410 2023-09-30T22:21:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc009b30 con 0x7fd00409e410 2023-09-30T22:21:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fcfebfff700 1 -- 172.21.15.89:0/1353237947 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc004960 con 0x7fd00409e410 2023-09-30T22:21:14.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.719+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd004134a80 con 0x7fd00409e410 2023-09-30T22:21:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.720+0000 7fcfebfff700 1 -- 172.21.15.89:0/1353237947 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcffc01c410 con 0x7fd00409e410 2023-09-30T22:21:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.720+0000 7fcfebfff700 1 -- 172.21.15.89:0/1353237947 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcffc0415d0 con 0x7fd00409e410 2023-09-30T22:21:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.720+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004136650 con 0x7fd00409e410 2023-09-30T22:21:14.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.725+0000 7fcfebfff700 1 -- 172.21.15.89:0/1353237947 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd004136650 con 0x7fd00409e410 2023-09-30T22:21:14.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.868+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8fec074ea0 con 0x7f8fec139170 2023-09-30T22:21:14.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.868+0000 7f8fdaffd700 1 -- 172.21.15.89:0/2996099182 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8fe4018030 con 0x7f8fec139170 2023-09-30T22:21:14.869 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:14.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.870+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec139170 msgr2=0x7f8fec122280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.870+0000 7f8ff0b3a700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec139170 0x7f8fec122280 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009dd0 tx=0x7f8fe400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.870+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 shutdown_connections 2023-09-30T22:21:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.870+0000 7f8ff0b3a700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8fec139170 0x7f8fec122280 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.870+0000 7f8ff0b3a700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8fec09d6f0 0x7f8fec121800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.870+0000 7f8ff0b3a700 1 --2- 172.21.15.89:0/2996099182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fec09e1d0 0x7f8fec121d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.870+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 >> 172.21.15.89:0/2996099182 conn(0x7f8fec066f50 msgr2=0x7f8fec070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.871+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 shutdown_connections 2023-09-30T22:21:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.871+0000 7f8ff0b3a700 1 -- 172.21.15.89:0/2996099182 wait complete. 2023-09-30T22:21:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.874+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f00081b9b50 con 0x7f000809e410 2023-09-30T22:21:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.875+0000 7efffdffb700 1 -- 172.21.15.89:0/2480133939 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f00081b9b50 con 0x7f000809e410 2023-09-30T22:21:14.875 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.877+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f000809e410 msgr2=0x7f000812d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.877+0000 7f000f410700 1 --2- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f000809e410 0x7f000812d180 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f000000c7a0 tx=0x7f000000cb60 comp rx=0 tx=0).stop 2023-09-30T22:21:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.877+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 shutdown_connections 2023-09-30T22:21:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.877+0000 7f000f410700 1 --2- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f000809e410 0x7f000812d180 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.877+0000 7f000f410700 1 --2- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000809d6f0 0x7f00081b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.877+0000 7f000f410700 1 --2- 172.21.15.89:0/2480133939 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f000809e000 0x7f00081b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.877+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 >> 172.21.15.89:0/2480133939 conn(0x7f0008066f50 msgr2=0x7f00080a17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.878+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 shutdown_connections 2023-09-30T22:21:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.879+0000 7f000f410700 1 -- 172.21.15.89:0/2480133939 wait complete. 2023-09-30T22:21:14.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:14.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:14.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.909+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff72c074ea0 con 0x7ff72c09e930 2023-09-30T22:21:14.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.910+0000 7ff721ffb700 1 -- 172.21.15.89:0/4101264945 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff728017030 con 0x7ff72c09e930 2023-09-30T22:21:14.911 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09e930 msgr2=0x7ff72c06c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09e930 0x7ff72c06c0e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c4e0 tx=0x7ff72800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 shutdown_connections 2023-09-30T22:21:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c136fe0 0x7ff72c1c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff72c09e930 0x7ff72c06c0e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 --2- 172.21.15.89:0/4101264945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c09f410 0x7ff72c06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 >> 172.21.15.89:0/4101264945 conn(0x7ff72c0a4210 msgr2=0x7ff72c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:14.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 shutdown_connections 2023-09-30T22:21:14.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.912+0000 7ff7338ba700 1 -- 172.21.15.89:0/4101264945 wait complete. 2023-09-30T22:21:14.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.920+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd004136650 con 0x7fd00409e410 2023-09-30T22:21:14.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.920+0000 7fcfebfff700 1 -- 172.21.15.89:0/1353237947 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd004136650 con 0x7fd00409e410 2023-09-30T22:21:14.921 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:14.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.922+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd00409e410 msgr2=0x7fd00412d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:14.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.922+0000 7fd009f8a700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd00409e410 0x7fd00412d030 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcffc003f50 tx=0x7fcffc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.922+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 shutdown_connections 2023-09-30T22:21:14.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.922+0000 7fd009f8a700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd00409e410 0x7fd00412d030 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.922+0000 7fd009f8a700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd00409d6f0 0x7fd00412c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.922+0000 7fd009f8a700 1 --2- 172.21.15.89:0/1353237947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd00409e000 0x7fd00412caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:14.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.922+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 >> 172.21.15.89:0/1353237947 conn(0x7fd004066f50 msgr2=0x7fd004070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:14.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.923+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 shutdown_connections 2023-09-30T22:21:14.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:14.923+0000 7fd009f8a700 1 -- 172.21.15.89:0/1353237947 wait complete. 2023-09-30T22:21:14.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:14.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:15.886 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:15.893 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:15.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:15.937 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.041+0000 7fbee5dc0700 1 Processor -- start 2023-09-30T22:21:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.041+0000 7fbee5dc0700 1 -- start start 2023-09-30T22:21:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbee5dc0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbee009c650 0x7fbee009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbee5dc0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee009cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbee5dc0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee009ba70 con 0x7fbee0131970 2023-09-30T22:21:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbee5dc0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee009d4a0 con 0x7fbee009c650 2023-09-30T22:21:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbee5dc0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee009e8f0 con 0x7fbee00a0090 2023-09-30T22:21:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbedeffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee009cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbedf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbee009c650 0x7fbee009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbedeffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee009cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37000/0 (socket says 172.21.15.89:37000) 2023-09-30T22:21:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.042+0000 7fbedeffd700 1 -- 172.21.15.89:0/892360967 learned_addr learned my addr 172.21.15.89:0/892360967 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.043+0000 7fbedeffd700 1 -- 172.21.15.89:0/892360967 >> v2:172.21.15.89:3300/0 conn(0x7fbee009c650 msgr2=0x7fbee009ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.043+0000 7fbedeffd700 1 --2- 172.21.15.89:0/892360967 >> v2:172.21.15.89:3300/0 conn(0x7fbee009c650 0x7fbee009ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.043+0000 7fbedeffd700 1 -- 172.21.15.89:0/892360967 >> v1:172.21.15.89:6789/0 conn(0x7fbee0131970 legacy=0x7fbee0131db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.043+0000 7fbedeffd700 1 -- 172.21.15.89:0/892360967 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00a0760 con 0x7fbee00a0090 2023-09-30T22:21:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.043+0000 7fbedeffd700 1 --2- 172.21.15.89:0/892360967 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee009cf60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbed40098f0 tx=0x7fbed4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d2052d2d5bb3fb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.043+0000 7fbede7fc700 1 -- 172.21.15.89:0/892360967 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4011830 con 0x7fbee00a0090 2023-09-30T22:21:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.043+0000 7fbede7fc700 1 -- 172.21.15.89:0/892360967 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed4011990 con 0x7fbee00a0090 2023-09-30T22:21:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.043+0000 7fbede7fc700 1 -- 172.21.15.89:0/892360967 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4011d20 con 0x7fbee00a0090 2023-09-30T22:21:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.044+0000 7fbee5dc0700 1 -- 172.21.15.89:0/892360967 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 msgr2=0x7fbee009cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.044+0000 7fbee5dc0700 1 --2- 172.21.15.89:0/892360967 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee009cf60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbed40098f0 tx=0x7fbed4009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.044+0000 7fbee5dc0700 1 -- 172.21.15.89:0/892360967 shutdown_connections 2023-09-30T22:21:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.044+0000 7fbee5dc0700 1 --2- 172.21.15.89:0/892360967 >> v2:172.21.15.89:3300/0 conn(0x7fbee009c650 0x7fbee009ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.044+0000 7fbee5dc0700 1 --2- 172.21.15.89:0/892360967 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee009cf60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.044+0000 7fbee5dc0700 1 -- 172.21.15.89:0/892360967 >> 172.21.15.89:0/892360967 conn(0x7fbee00a3f70 msgr2=0x7fbee00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.044+0000 7fbee5dc0700 1 -- 172.21.15.89:0/892360967 shutdown_connections 2023-09-30T22:21:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.044+0000 7fbee5dc0700 1 -- 172.21.15.89:0/892360967 wait complete. 2023-09-30T22:21:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.045+0000 7fbee5dc0700 1 Processor -- start 2023-09-30T22:21:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.045+0000 7fbee5dc0700 1 -- start start 2023-09-30T22:21:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbee5dc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbee009c650 0x7fbee011a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbee5dc0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee011ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbee5dc0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0131970 0x7fbee011b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbee5dc0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee011cff0 con 0x7fbee0131970 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbee5dc0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee011c310 con 0x7fbee00a0090 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbee5dc0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbee011bce0 con 0x7fbee009c650 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbee009c650 0x7fbee011a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbee009c650 0x7fbee011a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50978/0 (socket says 172.21.15.89:50978) 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedf7fe700 1 -- 172.21.15.89:0/871517375 learned_addr learned my addr 172.21.15.89:0/871517375 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedffff700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0131970 0x7fbee011b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedeffd700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee011ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedf7fe700 1 -- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 msgr2=0x7fbee011ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedf7fe700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee011ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedf7fe700 1 -- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0131970 msgr2=0x7fbee011b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedf7fe700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0131970 0x7fbee011b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.046+0000 7fbedf7fe700 1 -- 172.21.15.89:0/871517375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4009550 con 0x7fbee009c650 2023-09-30T22:21:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.047+0000 7fbedf7fe700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbee009c650 0x7fbee011a620 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbed000e9e0 tx=0x7fbed000ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.047+0000 7fbec7fff700 1 -- 172.21.15.89:0/871517375 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000ee80 con 0x7fbee009c650 2023-09-30T22:21:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.047+0000 7fbec7fff700 1 -- 172.21.15.89:0/871517375 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed000c850 con 0x7fbee009c650 2023-09-30T22:21:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.047+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee011b980 con 0x7fbee009c650 2023-09-30T22:21:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.047+0000 7fbec7fff700 1 -- 172.21.15.89:0/871517375 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000cbc0 con 0x7fbee009c650 2023-09-30T22:21:16.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.047+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee011d360 con 0x7fbee009c650 2023-09-30T22:21:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.048+0000 7fbec7fff700 1 -- 172.21.15.89:0/871517375 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbed000cd20 con 0x7fbee009c650 2023-09-30T22:21:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.048+0000 7fbec7fff700 1 -- 172.21.15.89:0/871517375 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbed003f810 con 0x7fbee009c650 2023-09-30T22:21:16.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.049+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee00747f0 con 0x7fbee009c650 2023-09-30T22:21:16.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.054+0000 7fbec7fff700 1 -- 172.21.15.89:0/871517375 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbed0021020 con 0x7fbee009c650 2023-09-30T22:21:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.069+0000 7f1a56063700 1 Processor -- start 2023-09-30T22:21:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.069+0000 7f1a56063700 1 -- start start 2023-09-30T22:21:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.069+0000 7f1a56063700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009e930 0x7f1a5009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.069+0000 7f1a56063700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009f240 0x7f1a501317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.069+0000 7f1a56063700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a5009f650 0x7f1a50131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.069+0000 7f1a56063700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a50070940 con 0x7f1a5009f650 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.069+0000 7f1a56063700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5006fd60 con 0x7f1a5009f240 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.069+0000 7f1a56063700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5006fa80 con 0x7f1a5009e930 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a5009f650 0x7f1a50131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a46ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009f240 0x7f1a501317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a5009f650 0x7f1a50131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33888/0 (socket says 172.21.15.89:33888) 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 -- 172.21.15.89:0/1053730616 learned_addr learned my addr 172.21.15.89:0/1053730616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 -- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009e930 msgr2=0x7f1a5009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 --2- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009e930 0x7f1a5009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 -- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009f240 msgr2=0x7f1a501317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 --2- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009f240 0x7f1a501317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 -- 172.21.15.89:0/1053730616 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5009fb60 con 0x7f1a5009f650 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a46ffd700 1 --2- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009f240 0x7f1a501317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4ffff700 1 --2- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f650 0x7f1a50131d30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009e50 tx=0x7f1a4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc34b03edd6dfd74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4effd700 1 -- 172.21.15.89:0/1053730616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800e070 con 0x7f1a5009f650 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4effd700 1 -- 172.21.15.89:0/1053730616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4800b910 con 0x7f1a5009f650 2023-09-30T22:21:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.070+0000 7f1a4effd700 1 -- 172.21.15.89:0/1053730616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800bcc0 con 0x7f1a5009f650 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 -- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f650 msgr2=0x7f1a50131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 --2- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f650 0x7f1a50131d30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009e50 tx=0x7f1a4800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 -- 172.21.15.89:0/1053730616 shutdown_connections 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 --2- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f650 0x7f1a50131d30 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 --2- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009e930 0x7f1a5009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 --2- 172.21.15.89:0/1053730616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009f240 0x7f1a501317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 -- 172.21.15.89:0/1053730616 >> 172.21.15.89:0/1053730616 conn(0x7f1a500a4210 msgr2=0x7f1a500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 -- 172.21.15.89:0/1053730616 shutdown_connections 2023-09-30T22:21:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.071+0000 7f1a56063700 1 -- 172.21.15.89:0/1053730616 wait complete. 2023-09-30T22:21:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.072+0000 7f1a56063700 1 Processor -- start 2023-09-30T22:21:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.072+0000 7f1a56063700 1 -- start start 2023-09-30T22:21:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a56063700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009e930 0x7f1a501ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a56063700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f240 0x7f1a501ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a56063700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009f650 0x7f1a501acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a56063700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501ad6e0 con 0x7f1a5009f240 2023-09-30T22:21:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a56063700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501adb70 con 0x7f1a5009e930 2023-09-30T22:21:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a56063700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501af260 con 0x7f1a5009f650 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a4ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009f650 0x7f1a501acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a4ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009f650 0x7f1a501acc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:50988/0 (socket says 172.21.15.89:50988) 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a4ffff700 1 -- 172.21.15.89:0/1785469938 learned_addr learned my addr 172.21.15.89:0/1785469938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a46ffd700 1 --2- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f240 0x7f1a501ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a4ffff700 1 -- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009e930 msgr2=0x7f1a501ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a4ffff700 1 --2- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009e930 0x7f1a501ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a4ffff700 1 -- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f240 msgr2=0x7f1a501ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a4ffff700 1 --2- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f240 0x7f1a501ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.073+0000 7f1a4ffff700 1 -- 172.21.15.89:0/1785469938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a380094a0 con 0x7f1a5009f650 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.074+0000 7f1a4ffff700 1 --2- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009f650 0x7f1a501acc90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1a48012000 tx=0x7f1a48004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.074+0000 7f1a4cff9700 1 -- 172.21.15.89:0/1785469938 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48004450 con 0x7f1a5009f650 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.074+0000 7f1a4cff9700 1 -- 172.21.15.89:0/1785469938 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a480045b0 con 0x7f1a5009f650 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.074+0000 7f1a4cff9700 1 -- 172.21.15.89:0/1785469938 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48004960 con 0x7f1a5009f650 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.075+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a48009b30 con 0x7f1a5009f650 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.075+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a500a0190 con 0x7f1a5009f650 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.076+0000 7f1a4cff9700 1 -- 172.21.15.89:0/1785469938 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a4801c410 con 0x7f1a5009f650 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.076+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a50074ea0 con 0x7f1a5009f650 2023-09-30T22:21:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.076+0000 7f1a4cff9700 1 -- 172.21.15.89:0/1785469938 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1a480092e0 con 0x7f1a5009f650 2023-09-30T22:21:16.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.081+0000 7f1a4cff9700 1 -- 172.21.15.89:0/1785469938 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a48023770 con 0x7f1a5009f650 2023-09-30T22:21:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.081+0000 7f9b0c80b700 1 Processor -- start 2023-09-30T22:21:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.081+0000 7f9b0c80b700 1 -- start start 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b0c80b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a0a30 0x7f9b080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b0c80b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b08133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b0c80b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0809edd0 con 0x7f9b08133fa0 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b0c80b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0809e1f0 con 0x7f9b080a0a30 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b0c80b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0809dc50 con 0x7f9b080a1340 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a0a30 0x7f9b080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b05d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b08133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a0a30 0x7f9b080a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37046/0 (socket says 172.21.15.89:37046) 2023-09-30T22:21:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.082+0000 7f9b0659c700 1 -- 172.21.15.89:0/960457938 learned_addr learned my addr 172.21.15.89:0/960457938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b05d9b700 1 -- 172.21.15.89:0/960457938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a0a30 msgr2=0x7f9b080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b05d9b700 1 --2- 172.21.15.89:0/960457938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a0a30 0x7f9b080a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b05d9b700 1 -- 172.21.15.89:0/960457938 >> v1:172.21.15.89:6789/0 conn(0x7f9b08133fa0 legacy=0x7f9b0813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b05d9b700 1 -- 172.21.15.89:0/960457938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b080a1fe0 con 0x7f9b080a1340 2023-09-30T22:21:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b0659c700 1 --2- 172.21.15.89:0/960457938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a0a30 0x7f9b080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b05d9b700 1 --2- 172.21.15.89:0/960457938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b08133a60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009870 tx=0x7f9afc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2122194106c83513 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b0559a700 1 -- 172.21.15.89:0/960457938 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc004230 con 0x7f9b080a1340 2023-09-30T22:21:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b0559a700 1 -- 172.21.15.89:0/960457938 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc004390 con 0x7f9b080a1340 2023-09-30T22:21:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.083+0000 7f9b0559a700 1 -- 172.21.15.89:0/960457938 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0036a0 con 0x7f9b080a1340 2023-09-30T22:21:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.084+0000 7f9b0c80b700 1 -- 172.21.15.89:0/960457938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 msgr2=0x7f9b08133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.084+0000 7f9b0c80b700 1 --2- 172.21.15.89:0/960457938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b08133a60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009870 tx=0x7f9afc009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.084+0000 7f9b0c80b700 1 -- 172.21.15.89:0/960457938 shutdown_connections 2023-09-30T22:21:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.084+0000 7f9b0c80b700 1 --2- 172.21.15.89:0/960457938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a0a30 0x7f9b080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.084+0000 7f9b0c80b700 1 --2- 172.21.15.89:0/960457938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b08133a60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.084+0000 7f9b0c80b700 1 -- 172.21.15.89:0/960457938 >> 172.21.15.89:0/960457938 conn(0x7f9b080a6550 msgr2=0x7f9b080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.084+0000 7f9b0c80b700 1 -- 172.21.15.89:0/960457938 shutdown_connections 2023-09-30T22:21:16.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.084+0000 7f9b0c80b700 1 -- 172.21.15.89:0/960457938 wait complete. 2023-09-30T22:21:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.086+0000 7f9b0c80b700 1 Processor -- start 2023-09-30T22:21:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.086+0000 7f9b0c80b700 1 -- start start 2023-09-30T22:21:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.086+0000 7f9b0c80b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b080a0a30 0x7f9b0811e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.086+0000 7f9b0c80b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b0811ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.086+0000 7f9b0c80b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b08133fa0 0x7f9b0811f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.086+0000 7f9b0c80b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b080a0290 con 0x7f9b080a0a30 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.086+0000 7f9b0c80b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b08120590 con 0x7f9b08133fa0 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.086+0000 7f9b0c80b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b08122320 con 0x7f9b080a1340 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b080a0a30 0x7f9b0811e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b080a0a30 0x7f9b0811e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33912/0 (socket says 172.21.15.89:33912) 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b0659c700 1 -- 172.21.15.89:0/78001105 learned_addr learned my addr 172.21.15.89:0/78001105 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b05d9b700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b0811ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b06d9d700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b08133fa0 0x7f9b0811f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b05d9b700 1 -- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b08133fa0 msgr2=0x7f9b0811f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b05d9b700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b08133fa0 0x7f9b0811f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b05d9b700 1 -- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b080a0a30 msgr2=0x7f9b0811e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b05d9b700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b080a0a30 0x7f9b0811e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b05d9b700 1 -- 172.21.15.89:0/78001105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8009550 con 0x7f9b080a1340 2023-09-30T22:21:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b0659c700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b080a0a30 0x7f9b0811e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b05d9b700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b0811ed40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9afc015000 tx=0x7f9afc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9af6ffd700 1 -- 172.21.15.89:0/78001105 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc003e10 con 0x7f9b080a1340 2023-09-30T22:21:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc0094a0 con 0x7f9b080a1340 2023-09-30T22:21:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.087+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b080a25e0 con 0x7f9b080a1340 2023-09-30T22:21:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.089+0000 7f9af6ffd700 1 -- 172.21.15.89:0/78001105 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc011630 con 0x7f9b080a1340 2023-09-30T22:21:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.089+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b08074ea0 con 0x7f9b080a1340 2023-09-30T22:21:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.089+0000 7f9af6ffd700 1 -- 172.21.15.89:0/78001105 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc006c80 con 0x7f9b080a1340 2023-09-30T22:21:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.089+0000 7f9af6ffd700 1 -- 172.21.15.89:0/78001105 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9afc0119e0 con 0x7f9b080a1340 2023-09-30T22:21:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.090+0000 7f9af6ffd700 1 -- 172.21.15.89:0/78001105 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9afc051d40 con 0x7f9b080a1340 2023-09-30T22:21:16.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.094+0000 7f9af6ffd700 1 -- 172.21.15.89:0/78001105 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9afc010d50 con 0x7f9b080a1340 2023-09-30T22:21:16.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.111+0000 7f8e7fb05700 1 Processor -- start 2023-09-30T22:21:16.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.111+0000 7f8e7fb05700 1 -- start start 2023-09-30T22:21:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7fb05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78133a80 0x7f8e78133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7fb05700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e7812ff20 0x7f8e78130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7fb05700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7806bc00 con 0x7f8e78130870 2023-09-30T22:21:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7fb05700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7806c570 con 0x7f8e7812ff20 2023-09-30T22:21:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7fb05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e780a0040 con 0x7f8e78133a80 2023-09-30T22:21:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d8a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78133a80 0x7f8e78133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d8a1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78133a80 0x7f8e78133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37062/0 (socket says 172.21.15.89:37062) 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d8a1700 1 -- 172.21.15.89:0/4001125769 learned_addr learned my addr 172.21.15.89:0/4001125769 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d0a0700 1 --2- 172.21.15.89:0/4001125769 >> v2:172.21.15.89:3300/0 conn(0x7f8e7812ff20 0x7f8e78130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d8a1700 1 -- 172.21.15.89:0/4001125769 >> v2:172.21.15.89:3300/0 conn(0x7f8e7812ff20 msgr2=0x7f8e78130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d8a1700 1 --2- 172.21.15.89:0/4001125769 >> v2:172.21.15.89:3300/0 conn(0x7f8e7812ff20 0x7f8e78130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d8a1700 1 -- 172.21.15.89:0/4001125769 >> v1:172.21.15.89:6789/0 conn(0x7f8e78130870 legacy=0x7f8e7813a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d8a1700 1 -- 172.21.15.89:0/4001125769 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78133860 con 0x7f8e78133a80 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d0a0700 1 --2- 172.21.15.89:0/4001125769 >> v2:172.21.15.89:3300/0 conn(0x7f8e7812ff20 0x7f8e78130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7d8a1700 1 --2- 172.21.15.89:0/4001125769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78133a80 0x7f8e78133e50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8e740098f0 tx=0x7f8e74009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a6acac7a8b36a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.112+0000 7f8e7c89f700 1 -- 172.21.15.89:0/4001125769 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74011830 con 0x7f8e78133a80 2023-09-30T22:21:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7c89f700 1 -- 172.21.15.89:0/4001125769 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e74011990 con 0x7f8e78133a80 2023-09-30T22:21:16.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7c89f700 1 -- 172.21.15.89:0/4001125769 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74011d20 con 0x7f8e78133a80 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7fb05700 1 -- 172.21.15.89:0/4001125769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78133a80 msgr2=0x7f8e78133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7fb05700 1 --2- 172.21.15.89:0/4001125769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78133a80 0x7f8e78133e50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8e740098f0 tx=0x7f8e74009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7fb05700 1 -- 172.21.15.89:0/4001125769 shutdown_connections 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7fb05700 1 --2- 172.21.15.89:0/4001125769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78133a80 0x7f8e78133e50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7fb05700 1 --2- 172.21.15.89:0/4001125769 >> v2:172.21.15.89:3300/0 conn(0x7f8e7812ff20 0x7f8e78130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7fb05700 1 -- 172.21.15.89:0/4001125769 >> 172.21.15.89:0/4001125769 conn(0x7f8e780a4210 msgr2=0x7f8e780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7fb05700 1 -- 172.21.15.89:0/4001125769 shutdown_connections 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.113+0000 7f8e7fb05700 1 -- 172.21.15.89:0/4001125769 wait complete. 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.114+0000 7f8e7fb05700 1 Processor -- start 2023-09-30T22:21:16.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.114+0000 7f8e7fb05700 1 -- start start 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7fb05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e7812ff20 0x7f8e7809c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7fb05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78130870 0x7f8e7809c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7fb05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e78133a80 0x7f8e7809ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7fb05700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7811ee60 con 0x7f8e7812ff20 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7fb05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7811fff0 con 0x7f8e78130870 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7fb05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e78120f50 con 0x7f8e78133a80 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78130870 0x7f8e7809c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d8a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e7812ff20 0x7f8e7809c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78130870 0x7f8e7809c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37074/0 (socket says 172.21.15.89:37074) 2023-09-30T22:21:16.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d8a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e7812ff20 0x7f8e7809c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33930/0 (socket says 172.21.15.89:33930) 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 -- 172.21.15.89:0/3877654969 learned_addr learned my addr 172.21.15.89:0/3877654969 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 -- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e78133a80 msgr2=0x7f8e7809ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 --2- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e78133a80 0x7f8e7809ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 -- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e7812ff20 msgr2=0x7f8e7809c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 --2- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e7812ff20 0x7f8e7809c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 -- 172.21.15.89:0/3877654969 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e680094a0 con 0x7f8e78130870 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d8a1700 1 --2- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e7812ff20 0x7f8e7809c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e7d0a0700 1 --2- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78130870 0x7f8e7809c8e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800ebe0 tx=0x7f8e6800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.115+0000 7f8e6e7fc700 1 -- 172.21.15.89:0/3877654969 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800c910 con 0x7f8e78130870 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.116+0000 7f8e6e7fc700 1 -- 172.21.15.89:0/3877654969 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6800ca70 con 0x7f8e78130870 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.116+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e74009550 con 0x7f8e78130870 2023-09-30T22:21:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.116+0000 7f8e6e7fc700 1 -- 172.21.15.89:0/3877654969 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800ce00 con 0x7f8e78130870 2023-09-30T22:21:16.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.116+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7812db30 con 0x7f8e78130870 2023-09-30T22:21:16.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.116+0000 7f8e6e7fc700 1 -- 172.21.15.89:0/3877654969 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e68013470 con 0x7f8e78130870 2023-09-30T22:21:16.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.117+0000 7f8e6e7fc700 1 -- 172.21.15.89:0/3877654969 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e6803fc00 con 0x7f8e78130870 2023-09-30T22:21:16.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.117+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e78120230 con 0x7f8e78130870 2023-09-30T22:21:16.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.121+0000 7f8e6e7fc700 1 -- 172.21.15.89:0/3877654969 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e6801b3b0 con 0x7f8e78130870 2023-09-30T22:21:16.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.275+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1a5006b250 con 0x7f1a5009f650 2023-09-30T22:21:16.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.275+0000 7f1a4cff9700 1 -- 172.21.15.89:0/1785469938 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a4801b3a0 con 0x7f1a5009f650 2023-09-30T22:21:16.276 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:16.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.277+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009f650 msgr2=0x7f1a501acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.277+0000 7f1a56063700 1 --2- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009f650 0x7f1a501acc90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1a48012000 tx=0x7f1a48004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.277+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 shutdown_connections 2023-09-30T22:21:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.277+0000 7f1a56063700 1 --2- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a5009f650 0x7f1a501acc90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.277+0000 7f1a56063700 1 --2- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a5009e930 0x7f1a501ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.277+0000 7f1a56063700 1 --2- 172.21.15.89:0/1785469938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a5009f240 0x7f1a501ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.277+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 >> 172.21.15.89:0/1785469938 conn(0x7f1a500a4210 msgr2=0x7f1a5009c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.278+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 shutdown_connections 2023-09-30T22:21:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.278+0000 7f1a56063700 1 -- 172.21.15.89:0/1785469938 wait complete. 2023-09-30T22:21:16.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:16.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.315+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8e78074ea0 con 0x7f8e78130870 2023-09-30T22:21:16.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.315+0000 7f8e6e7fc700 1 -- 172.21.15.89:0/3877654969 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e68017030 con 0x7f8e78130870 2023-09-30T22:21:16.316 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:16.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78130870 msgr2=0x7f8e7809c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 --2- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78130870 0x7f8e7809c8e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800ebe0 tx=0x7f8e6800eef0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 shutdown_connections 2023-09-30T22:21:16.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 --2- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e78133a80 0x7f8e7809ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 --2- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e7812ff20 0x7f8e7809c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 --2- 172.21.15.89:0/3877654969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e78130870 0x7f8e7809c8e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 >> 172.21.15.89:0/3877654969 conn(0x7f8e780a4210 msgr2=0x7f8e7809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:16.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 shutdown_connections 2023-09-30T22:21:16.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.317+0000 7f8e7fb05700 1 -- 172.21.15.89:0/3877654969 wait complete. 2023-09-30T22:21:16.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.359+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbee0072840 con 0x7fbee009c650 2023-09-30T22:21:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.359+0000 7fbec7fff700 1 -- 172.21.15.89:0/871517375 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbed002a090 con 0x7fbee009c650 2023-09-30T22:21:16.360 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbee009c650 msgr2=0x7fbee011a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbee009c650 0x7fbee011a620 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbed000e9e0 tx=0x7fbed000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 shutdown_connections 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0131970 0x7fbee011b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbee009c650 0x7fbee011a620 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 --2- 172.21.15.89:0/871517375 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbee00a0090 0x7fbee011ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 >> 172.21.15.89:0/871517375 conn(0x7fbee00a3f70 msgr2=0x7fbee012f050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 shutdown_connections 2023-09-30T22:21:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.362+0000 7fbee5dc0700 1 -- 172.21.15.89:0/871517375 wait complete. 2023-09-30T22:21:16.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.378+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9b08074ea0 con 0x7f9b080a1340 2023-09-30T22:21:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.379+0000 7f9af6ffd700 1 -- 172.21.15.89:0/78001105 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9afc010b70 con 0x7f9b080a1340 2023-09-30T22:21:16.379 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.380+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 msgr2=0x7f9b0811ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.380+0000 7f9b0c80b700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b0811ed40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9afc015000 tx=0x7f9afc0047c0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.381+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 shutdown_connections 2023-09-30T22:21:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.381+0000 7f9b0c80b700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b08133fa0 0x7f9b0811f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.381+0000 7f9b0c80b700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b080a0a30 0x7f9b0811e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.381+0000 7f9b0c80b700 1 --2- 172.21.15.89:0/78001105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b080a1340 0x7f9b0811ed40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.381+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 >> 172.21.15.89:0/78001105 conn(0x7f9b080a6550 msgr2=0x7f9b081b9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.381+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 shutdown_connections 2023-09-30T22:21:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:16.381+0000 7f9b0c80b700 1 -- 172.21.15.89:0/78001105 wait complete. 2023-09-30T22:21:16.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:17.296 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:17.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:17.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:17.394 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.430+0000 7f790e1d5700 1 Processor -- start 2023-09-30T22:21:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.430+0000 7f790e1d5700 1 -- start start 2023-09-30T22:21:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f790e1d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f790e1d5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f790809d9f0 0x7f790806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f790e1d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 0x7f790806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f790e1d5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790806b630 con 0x7f790809d9f0 2023-09-30T22:21:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f790e1d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790809e130 con 0x7f79081316c0 2023-09-30T22:21:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f790e1d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79081312b0 con 0x7f790806acb0 2023-09-30T22:21:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7906ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f790809d9f0 0x7f790806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7906ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f790809d9f0 0x7f790806a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33946/0 (socket says 172.21.15.89:33946) 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7906ffd700 1 -- 172.21.15.89:0/1387509401 learned_addr learned my addr 172.21.15.89:0/1387509401 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7907fff700 1 --2- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 0x7f790806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f79077fe700 1 --2- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7907fff700 1 -- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 msgr2=0x7f7908131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7907fff700 1 --2- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7907fff700 1 -- 172.21.15.89:0/1387509401 >> v2:172.21.15.89:3300/0 conn(0x7f790809d9f0 msgr2=0x7f790806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7907fff700 1 --2- 172.21.15.89:0/1387509401 >> v2:172.21.15.89:3300/0 conn(0x7f790809d9f0 0x7f790806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7907fff700 1 -- 172.21.15.89:0/1387509401 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790809f6e0 con 0x7f790806acb0 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f7906ffd700 1 --2- 172.21.15.89:0/1387509401 >> v2:172.21.15.89:3300/0 conn(0x7f790809d9f0 0x7f790806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.431+0000 7f79077fe700 1 --2- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.432+0000 7f7907fff700 1 --2- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 0x7f790806b0f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7900009dc0 tx=0x7f790000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680ebecb9fe6fc4e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.432+0000 7f79067fc700 1 -- 172.21.15.89:0/1387509401 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000e070 con 0x7f790806acb0 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.432+0000 7f79067fc700 1 -- 172.21.15.89:0/1387509401 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790000bae0 con 0x7f790806acb0 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.432+0000 7f79067fc700 1 -- 172.21.15.89:0/1387509401 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000be90 con 0x7f790806acb0 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 -- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 msgr2=0x7f790806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 --2- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 0x7f790806b0f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7900009dc0 tx=0x7f790000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 -- 172.21.15.89:0/1387509401 shutdown_connections 2023-09-30T22:21:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 --2- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 0x7f790806b0f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 --2- 172.21.15.89:0/1387509401 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 --2- 172.21.15.89:0/1387509401 >> v2:172.21.15.89:3300/0 conn(0x7f790809d9f0 0x7f790806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 -- 172.21.15.89:0/1387509401 >> 172.21.15.89:0/1387509401 conn(0x7f7908066f50 msgr2=0x7f790809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 -- 172.21.15.89:0/1387509401 shutdown_connections 2023-09-30T22:21:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.433+0000 7f790e1d5700 1 -- 172.21.15.89:0/1387509401 wait complete. 2023-09-30T22:21:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.434+0000 7f790e1d5700 1 Processor -- start 2023-09-30T22:21:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.434+0000 7f790e1d5700 1 -- start start 2023-09-30T22:21:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f790e1d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 0x7f7908132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f790e1d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809d9f0 0x7f7908132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f790e1d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f790e1d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7908134950 con 0x7f790809d9f0 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f790e1d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7908134290 con 0x7f79081316c0 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f790e1d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79081355a0 con 0x7f790806acb0 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f7907fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f7907fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908132d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37092/0 (socket says 172.21.15.89:37092) 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f7907fff700 1 -- 172.21.15.89:0/457877761 learned_addr learned my addr 172.21.15.89:0/457877761 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.435+0000 7f7906ffd700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809d9f0 0x7f7908132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f7906ffd700 1 -- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 msgr2=0x7f7908132300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f7906ffd700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 0x7f7908132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f7906ffd700 1 -- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 msgr2=0x7f7908132d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f7906ffd700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908132d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f7906ffd700 1 -- 172.21.15.89:0/457877761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc009550 con 0x7f790809d9f0 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f7907fff700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f7906ffd700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809d9f0 0x7f7908132840 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f790806deb0 tx=0x7f78fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f78effff700 1 -- 172.21.15.89:0/457877761 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00eec0 con 0x7f790809d9f0 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f78effff700 1 -- 172.21.15.89:0/457877761 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc004030 con 0x7f790809d9f0 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7900009a70 con 0x7f790809d9f0 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f78effff700 1 -- 172.21.15.89:0/457877761 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0043a0 con 0x7f790809d9f0 2023-09-30T22:21:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.436+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7908134690 con 0x7f790809d9f0 2023-09-30T22:21:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.437+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7908134b90 con 0x7f790809d9f0 2023-09-30T22:21:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.438+0000 7f78effff700 1 -- 172.21.15.89:0/457877761 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f78fc00c820 con 0x7f790809d9f0 2023-09-30T22:21:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.438+0000 7f78effff700 1 -- 172.21.15.89:0/457877761 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f78fc03fd10 con 0x7f790809d9f0 2023-09-30T22:21:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.446+0000 7f78effff700 1 -- 172.21.15.89:0/457877761 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78fc023350 con 0x7f790809d9f0 2023-09-30T22:21:17.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.469+0000 7f6b6795c700 1 Processor -- start 2023-09-30T22:21:17.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b6795c700 1 -- start start 2023-09-30T22:21:17.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b6795c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60136710 0x7f6b60138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b6795c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b60139030 0x7f6b6013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b6795c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6006ef30 con 0x7f6b601338a0 2023-09-30T22:21:17.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b6795c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6006fef0 con 0x7f6b60139030 2023-09-30T22:21:17.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b6795c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601334b0 con 0x7f6b60136710 2023-09-30T22:21:17.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b65ef9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b60139030 0x7f6b6013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b64ef7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60136710 0x7f6b60138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b65ef9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b60139030 0x7f6b6013b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37096/0 (socket says 172.21.15.89:37096) 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b65ef9700 1 -- 172.21.15.89:0/4080270746 learned_addr learned my addr 172.21.15.89:0/4080270746 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b65ef9700 1 -- 172.21.15.89:0/4080270746 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60136710 msgr2=0x7f6b60138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b65ef9700 1 --2- 172.21.15.89:0/4080270746 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60136710 0x7f6b60138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b65ef9700 1 -- 172.21.15.89:0/4080270746 >> v1:172.21.15.89:6789/0 conn(0x7f6b601338a0 legacy=0x7f6b60133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.470+0000 7f6b65ef9700 1 -- 172.21.15.89:0/4080270746 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6006fc30 con 0x7f6b60139030 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.471+0000 7f6b65ef9700 1 --2- 172.21.15.89:0/4080270746 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b60139030 0x7f6b6013b410 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009e80 tx=0x7f6b5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ad0a0b2828c3571 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.471+0000 7f6b57fff700 1 -- 172.21.15.89:0/4080270746 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800e070 con 0x7f6b60139030 2023-09-30T22:21:17.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.471+0000 7f6b57fff700 1 -- 172.21.15.89:0/4080270746 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b58003a20 con 0x7f6b60139030 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.471+0000 7f6b57fff700 1 -- 172.21.15.89:0/4080270746 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58003dd0 con 0x7f6b60139030 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.472+0000 7f6b6795c700 1 -- 172.21.15.89:0/4080270746 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b60139030 msgr2=0x7f6b6013b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.472+0000 7f6b6795c700 1 --2- 172.21.15.89:0/4080270746 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b60139030 0x7f6b6013b410 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009e80 tx=0x7f6b5800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.472+0000 7f6b6795c700 1 -- 172.21.15.89:0/4080270746 shutdown_connections 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.472+0000 7f6b6795c700 1 --2- 172.21.15.89:0/4080270746 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b60139030 0x7f6b6013b410 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.472+0000 7f6b6795c700 1 --2- 172.21.15.89:0/4080270746 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60136710 0x7f6b60138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.472+0000 7f6b6795c700 1 -- 172.21.15.89:0/4080270746 >> 172.21.15.89:0/4080270746 conn(0x7f6b600a2aa0 msgr2=0x7f6b600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.472+0000 7f6b6795c700 1 -- 172.21.15.89:0/4080270746 shutdown_connections 2023-09-30T22:21:17.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.472+0000 7f6b6795c700 1 -- 172.21.15.89:0/4080270746 wait complete. 2023-09-30T22:21:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b6795c700 1 Processor -- start 2023-09-30T22:21:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b6795c700 1 -- start start 2023-09-30T22:21:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b6795c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b601338a0 0x7f6b60118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b6795c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b60136710 0x7f6b60119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b6795c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60139030 0x7f6b60119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b6795c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6011c450 con 0x7f6b60136710 2023-09-30T22:21:17.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b6795c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6011aad0 con 0x7f6b601338a0 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b6795c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6006f640 con 0x7f6b60139030 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b656f8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b601338a0 0x7f6b60118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b656f8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b601338a0 0x7f6b60118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37112/0 (socket says 172.21.15.89:37112) 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b656f8700 1 -- 172.21.15.89:0/3402931431 learned_addr learned my addr 172.21.15.89:0/3402931431 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.473+0000 7f6b65ef9700 1 --2- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60139030 0x7f6b60119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b656f8700 1 -- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60139030 msgr2=0x7f6b60119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b656f8700 1 --2- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60139030 0x7f6b60119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b656f8700 1 -- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b60136710 msgr2=0x7f6b60119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b656f8700 1 --2- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b60136710 0x7f6b60119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b656f8700 1 -- 172.21.15.89:0/3402931431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58009b30 con 0x7f6b601338a0 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b656f8700 1 --2- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b601338a0 0x7f6b60118cd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c4d0 tx=0x7f6b5c00c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b55ffb700 1 -- 172.21.15.89:0/3402931431 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0054e0 con 0x7f6b601338a0 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b55ffb700 1 -- 172.21.15.89:0/3402931431 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c003680 con 0x7f6b601338a0 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6013b420 con 0x7f6b601338a0 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b55ffb700 1 -- 172.21.15.89:0/3402931431 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c003a30 con 0x7f6b601338a0 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.474+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6011ac90 con 0x7f6b601338a0 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.475+0000 7f6b55ffb700 1 -- 172.21.15.89:0/3402931431 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b5c003b90 con 0x7f6b601338a0 2023-09-30T22:21:17.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.475+0000 7f6b55ffb700 1 -- 172.21.15.89:0/3402931431 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6b5c03e770 con 0x7f6b601338a0 2023-09-30T22:21:17.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.476+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6011bf10 con 0x7f6b601338a0 2023-09-30T22:21:17.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.480+0000 7f6b55ffb700 1 -- 172.21.15.89:0/3402931431 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b5c003e40 con 0x7f6b601338a0 2023-09-30T22:21:17.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.543+0000 7f95b2520700 1 Processor -- start 2023-09-30T22:21:17.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.543+0000 7f95b2520700 1 -- start start 2023-09-30T22:21:17.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95b2520700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95b2520700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95b2520700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac133930 0x7f95ac133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95b2520700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0a2a70 con 0x7f95ac0a0a30 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95b2520700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0a01d0 con 0x7f95ac133930 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95b2520700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac09e2b0 con 0x7f95ac09d630 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33962/0 (socket says 172.21.15.89:33962) 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 -- 172.21.15.89:0/2967231681 learned_addr learned my addr 172.21.15.89:0/2967231681 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95b0abd700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac133930 0x7f95ac133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 -- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 msgr2=0x7f95ac0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 -- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac133930 msgr2=0x7f95ac133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac133930 0x7f95ac133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 -- 172.21.15.89:0/2967231681 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac09dc50 con 0x7f95ac0a0a30 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95ab7fe700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.544+0000 7f95abfff700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac0a0a30 0x7f95ac0a0e00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f959c0098f0 tx=0x7f959c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2d402d60ee4372 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95aaffd700 1 -- 172.21.15.89:0/2967231681 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c011830 con 0x7f95ac0a0a30 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95aaffd700 1 -- 172.21.15.89:0/2967231681 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c011990 con 0x7f95ac0a0a30 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95aaffd700 1 -- 172.21.15.89:0/2967231681 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c011d20 con 0x7f95ac0a0a30 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 -- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac0a0a30 msgr2=0x7f95ac0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac0a0a30 0x7f95ac0a0e00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f959c0098f0 tx=0x7f959c009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:17.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 -- 172.21.15.89:0/2967231681 shutdown_connections 2023-09-30T22:21:17.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac133930 0x7f95ac133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 --2- 172.21.15.89:0/2967231681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 -- 172.21.15.89:0/2967231681 >> 172.21.15.89:0/2967231681 conn(0x7f95ac0a3050 msgr2=0x7f95ac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:17.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 -- 172.21.15.89:0/2967231681 shutdown_connections 2023-09-30T22:21:17.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.545+0000 7f95b2520700 1 -- 172.21.15.89:0/2967231681 wait complete. 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.547+0000 7f95b2520700 1 Processor -- start 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.547+0000 7f95b2520700 1 -- start start 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.547+0000 7f95b2520700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac126020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.547+0000 7f95b2520700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac0a0a30 0x7f95ac126560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.547+0000 7f95b2520700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac133930 0x7f95ac126aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.547+0000 7f95b2520700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1280f0 con 0x7f95ac133930 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.547+0000 7f95b2520700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac127480 con 0x7f95ac0a0a30 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.547+0000 7f95b2520700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac132f80 con 0x7f95ac09d630 2023-09-30T22:21:17.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac126020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac126020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51064/0 (socket says 172.21.15.89:51064) 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 -- 172.21.15.89:0/2849482663 learned_addr learned my addr 172.21.15.89:0/2849482663 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95ab7fe700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac0a0a30 0x7f95ac126560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95b0abd700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac133930 0x7f95ac126aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 -- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac0a0a30 msgr2=0x7f95ac126560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac0a0a30 0x7f95ac126560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 -- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac133930 msgr2=0x7f95ac126aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac133930 0x7f95ac126aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 -- 172.21.15.89:0/2849482663 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00094a0 con 0x7f95ac09d630 2023-09-30T22:21:17.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95abfff700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac126020 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:17.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95a8ff9700 1 -- 172.21.15.89:0/2849482663 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c004450 con 0x7f95ac09d630 2023-09-30T22:21:17.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95a8ff9700 1 -- 172.21.15.89:0/2849482663 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c0045b0 con 0x7f95ac09d630 2023-09-30T22:21:17.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c009550 con 0x7f95ac09d630 2023-09-30T22:21:17.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.548+0000 7f95a8ff9700 1 -- 172.21.15.89:0/2849482663 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c0038e0 con 0x7f95ac09d630 2023-09-30T22:21:17.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.549+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0a0410 con 0x7f95ac09d630 2023-09-30T22:21:17.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.549+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac0747f0 con 0x7f95ac09d630 2023-09-30T22:21:17.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.550+0000 7f95a8ff9700 1 -- 172.21.15.89:0/2849482663 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f959c021070 con 0x7f95ac09d630 2023-09-30T22:21:17.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.550+0000 7f95a8ff9700 1 -- 172.21.15.89:0/2849482663 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f959c041100 con 0x7f95ac09d630 2023-09-30T22:21:17.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.559+0000 7f95a8ff9700 1 -- 172.21.15.89:0/2849482663 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f959c0199a0 con 0x7f95ac09d630 2023-09-30T22:21:17.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe519747700 1 Processor -- start 2023-09-30T22:21:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe519747700 1 -- start start 2023-09-30T22:21:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe519747700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe51409d6f0 0x7fe51409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe519747700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409e000 0x7fe514130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe519747700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51409f130 con 0x7fe51409e410 2023-09-30T22:21:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe519747700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51409f520 con 0x7fe51409d6f0 2023-09-30T22:21:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe519747700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51409e930 con 0x7fe51409e000 2023-09-30T22:21:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe5127fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409e000 0x7fe514130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe512ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe51409d6f0 0x7fe51409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe5127fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409e000 0x7fe514130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37140/0 (socket says 172.21.15.89:37140) 2023-09-30T22:21:17.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.572+0000 7fe5127fc700 1 -- 172.21.15.89:0/3687302155 learned_addr learned my addr 172.21.15.89:0/3687302155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe5127fc700 1 -- 172.21.15.89:0/3687302155 >> v2:172.21.15.89:3300/0 conn(0x7fe51409d6f0 msgr2=0x7fe51409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe5127fc700 1 --2- 172.21.15.89:0/3687302155 >> v2:172.21.15.89:3300/0 conn(0x7fe51409d6f0 0x7fe51409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe5127fc700 1 -- 172.21.15.89:0/3687302155 >> v1:172.21.15.89:6789/0 conn(0x7fe51409e410 legacy=0x7fe514130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe5127fc700 1 -- 172.21.15.89:0/3687302155 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51409b2d0 con 0x7fe51409e000 2023-09-30T22:21:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe511ffb700 1 -- 172.21.15.89:0/3687302155 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129362196 0 0) 0x7fe51409f130 con 0x7fe51409e410 2023-09-30T22:21:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe512ffd700 1 --2- 172.21.15.89:0/3687302155 >> v2:172.21.15.89:3300/0 conn(0x7fe51409d6f0 0x7fe51409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe5127fc700 1 --2- 172.21.15.89:0/3687302155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409e000 0x7fe514130660 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b9a0 tx=0x7fe50800bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55af1653dd0acec5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:17.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe511ffb700 1 -- 172.21.15.89:0/3687302155 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50800cc60 con 0x7fe51409e000 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe511ffb700 1 -- 172.21.15.89:0/3687302155 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50800cdc0 con 0x7fe51409e000 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.573+0000 7fe511ffb700 1 -- 172.21.15.89:0/3687302155 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50800f680 con 0x7fe51409e000 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.574+0000 7fe519747700 1 -- 172.21.15.89:0/3687302155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409e000 msgr2=0x7fe514130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.574+0000 7fe519747700 1 --2- 172.21.15.89:0/3687302155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409e000 0x7fe514130660 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b9a0 tx=0x7fe50800bcb0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.574+0000 7fe519747700 1 -- 172.21.15.89:0/3687302155 shutdown_connections 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.574+0000 7fe519747700 1 --2- 172.21.15.89:0/3687302155 >> v2:172.21.15.89:3300/0 conn(0x7fe51409d6f0 0x7fe51409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.574+0000 7fe519747700 1 --2- 172.21.15.89:0/3687302155 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409e000 0x7fe514130660 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.574+0000 7fe519747700 1 -- 172.21.15.89:0/3687302155 >> 172.21.15.89:0/3687302155 conn(0x7fe514066f50 msgr2=0x7fe51409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:17.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.574+0000 7fe519747700 1 -- 172.21.15.89:0/3687302155 shutdown_connections 2023-09-30T22:21:17.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.574+0000 7fe519747700 1 -- 172.21.15.89:0/3687302155 wait complete. 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.575+0000 7fe519747700 1 Processor -- start 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.575+0000 7fe519747700 1 -- start start 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe519747700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409d6f0 0x7fe5141d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe519747700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe51409e000 0x7fe5141d2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe519747700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe51409e410 0x7fe5141b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe519747700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe514072690 con 0x7fe51409e000 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe519747700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5141b8460 con 0x7fe51409d6f0 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe519747700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe51409c780 con 0x7fe51409e410 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe5127fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe51409e000 0x7fe5141d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe512ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409d6f0 0x7fe5141d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe5127fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe51409e000 0x7fe5141d2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33986/0 (socket says 172.21.15.89:33986) 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe5127fc700 1 -- 172.21.15.89:0/3323352946 learned_addr learned my addr 172.21.15.89:0/3323352946 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe5137fe700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe51409e410 0x7fe5141b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe512ffd700 1 -- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe51409e410 msgr2=0x7fe5141b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe512ffd700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe51409e410 0x7fe5141b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe512ffd700 1 -- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe51409e000 msgr2=0x7fe5141d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe512ffd700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe51409e000 0x7fe5141d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe512ffd700 1 -- 172.21.15.89:0/3323352946 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50800b600 con 0x7fe51409d6f0 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe5127fc700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe51409e000 0x7fe5141d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.576+0000 7fe512ffd700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409d6f0 0x7fe5141d1b40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe50400a700 tx=0x7fe50400aa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.577+0000 7fe4fb7fe700 1 -- 172.21.15.89:0/3323352946 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504004230 con 0x7fe51409d6f0 2023-09-30T22:21:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.577+0000 7fe4fb7fe700 1 -- 172.21.15.89:0/3323352946 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50400c030 con 0x7fe51409d6f0 2023-09-30T22:21:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.577+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51412f170 con 0x7fe51409d6f0 2023-09-30T22:21:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.577+0000 7fe4fb7fe700 1 -- 172.21.15.89:0/3323352946 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504004590 con 0x7fe51409d6f0 2023-09-30T22:21:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.577+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5141d25c0 con 0x7fe51409d6f0 2023-09-30T22:21:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.577+0000 7fe4fb7fe700 1 -- 172.21.15.89:0/3323352946 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5040036a0 con 0x7fe51409d6f0 2023-09-30T22:21:17.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.578+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe514074ea0 con 0x7fe51409d6f0 2023-09-30T22:21:17.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.578+0000 7fe4fb7fe700 1 -- 172.21.15.89:0/3323352946 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe50403eca0 con 0x7fe51409d6f0 2023-09-30T22:21:17.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.586+0000 7fe4fb7fe700 1 -- 172.21.15.89:0/3323352946 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe504043070 con 0x7fe51409d6f0 2023-09-30T22:21:17.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.685+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6b60074ea0 con 0x7f6b601338a0 2023-09-30T22:21:17.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.686+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7908074ea0 con 0x7f790809d9f0 2023-09-30T22:21:17.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.686+0000 7f6b55ffb700 1 -- 172.21.15.89:0/3402931431 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b5c017020 con 0x7f6b601338a0 2023-09-30T22:21:17.688 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:17.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.686+0000 7f78effff700 1 -- 172.21.15.89:0/457877761 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78fc012bc0 con 0x7f790809d9f0 2023-09-30T22:21:17.689 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:17.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b601338a0 msgr2=0x7f6b60118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f6b6795c700 1 --2- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b601338a0 0x7f6b60118cd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c4d0 tx=0x7f6b5c00c7e0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809d9f0 msgr2=0x7f7908132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f790e1d5700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809d9f0 0x7f7908132840 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f790806deb0 tx=0x7f78fc00ec80 comp rx=0 tx=0).stop 2023-09-30T22:21:17.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 shutdown_connections 2023-09-30T22:21:17.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f790e1d5700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081316c0 0x7f7908132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f790e1d5700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f790806acb0 0x7f7908132300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f790e1d5700 1 --2- 172.21.15.89:0/457877761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809d9f0 0x7f7908132840 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.688+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 >> 172.21.15.89:0/457877761 conn(0x7f7908066f50 msgr2=0x7f790809bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:17.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.689+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 shutdown_connections 2023-09-30T22:21:17.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.689+0000 7f6b6795c700 1 --2- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b60139030 0x7f6b60119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.689+0000 7f6b6795c700 1 --2- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b601338a0 0x7f6b60118cd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.689+0000 7f6b6795c700 1 --2- 172.21.15.89:0/3402931431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b60136710 0x7f6b60119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.689+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 >> 172.21.15.89:0/3402931431 conn(0x7f6b600a2aa0 msgr2=0x7f6b600a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:17.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.689+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 shutdown_connections 2023-09-30T22:21:17.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.689+0000 7f790e1d5700 1 -- 172.21.15.89:0/457877761 wait complete. 2023-09-30T22:21:17.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.690+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 shutdown_connections 2023-09-30T22:21:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.690+0000 7f6b6795c700 1 -- 172.21.15.89:0/3402931431 wait complete. 2023-09-30T22:21:17.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:17.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:17.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.765+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f95ac1297b0 con 0x7f95ac09d630 2023-09-30T22:21:17.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.766+0000 7f95a8ff9700 1 -- 172.21.15.89:0/2849482663 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f959c015030 con 0x7f95ac09d630 2023-09-30T22:21:17.766 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:17.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 msgr2=0x7f95ac126020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac126020 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 shutdown_connections 2023-09-30T22:21:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac133930 0x7f95ac126aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95ac09d630 0x7f95ac126020 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 --2- 172.21.15.89:0/2849482663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95ac0a0a30 0x7f95ac126560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 >> 172.21.15.89:0/2849482663 conn(0x7f95ac0a3050 msgr2=0x7f95ac0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 shutdown_connections 2023-09-30T22:21:17.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.768+0000 7f95b2520700 1 -- 172.21.15.89:0/2849482663 wait complete. 2023-09-30T22:21:17.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:17.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.780+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe51406c360 con 0x7fe51409d6f0 2023-09-30T22:21:17.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.780+0000 7fe4fb7fe700 1 -- 172.21.15.89:0/3323352946 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe504027020 con 0x7fe51409d6f0 2023-09-30T22:21:17.781 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409d6f0 msgr2=0x7fe5141d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409d6f0 0x7fe5141d1b40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe50400a700 tx=0x7fe50400aa10 comp rx=0 tx=0).stop 2023-09-30T22:21:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 shutdown_connections 2023-09-30T22:21:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe51409e410 0x7fe5141b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe51409d6f0 0x7fe5141d1b40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 --2- 172.21.15.89:0/3323352946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe51409e000 0x7fe5141d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 >> 172.21.15.89:0/3323352946 conn(0x7fe514066f50 msgr2=0x7fe514130900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:17.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 shutdown_connections 2023-09-30T22:21:17.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:17.782+0000 7fe519747700 1 -- 172.21.15.89:0/3323352946 wait complete. 2023-09-30T22:21:17.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:18.708 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:18.708 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:18.781 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:18.795 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.850+0000 7fcdc6555700 1 Processor -- start 2023-09-30T22:21:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.850+0000 7fcdc6555700 1 -- start start 2023-09-30T22:21:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc6555700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdc012cf60 0x7fcdc006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc6555700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012f8b0 0x7fcdc012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc6555700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcdc0130260 0x7fcdc0136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc6555700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc012d330 con 0x7fcdc0130260 2023-09-30T22:21:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc6555700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc012cd90 con 0x7fcdc012cf60 2023-09-30T22:21:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc6555700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc01306d0 con 0x7fcdc012f8b0 2023-09-30T22:21:18.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc4af2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcdc0130260 0x7fcdc0136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc4af2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcdc0130260 0x7fcdc0136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33996/0 (socket says 172.21.15.89:33996) 2023-09-30T22:21:18.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdc4af2700 1 -- 172.21.15.89:0/3757264880 learned_addr learned my addr 172.21.15.89:0/3757264880 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:18.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.851+0000 7fcdbf7fe700 1 --2- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012f8b0 0x7fcdc012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdc4af2700 1 -- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012f8b0 msgr2=0x7fcdc012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:18.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdc4af2700 1 --2- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012f8b0 0x7fcdc012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdc4af2700 1 -- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdc012cf60 msgr2=0x7fcdc006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:18.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdc4af2700 1 --2- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdc012cf60 0x7fcdc006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdc4af2700 1 -- 172.21.15.89:0/3757264880 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc0071780 con 0x7fcdc0130260 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdc4af2700 1 --2- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc0130260 0x7fcdc0136c20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009e50 tx=0x7fcdb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cda2085f5b257fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdbeffd700 1 -- 172.21.15.89:0/3757264880 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800e070 con 0x7fcdc0130260 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdbeffd700 1 -- 172.21.15.89:0/3757264880 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb800b910 con 0x7fcdc0130260 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.852+0000 7fcdbeffd700 1 -- 172.21.15.89:0/3757264880 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800bcc0 con 0x7fcdc0130260 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 -- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc0130260 msgr2=0x7fcdc0136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 --2- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc0130260 0x7fcdc0136c20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009e50 tx=0x7fcdb800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 -- 172.21.15.89:0/3757264880 shutdown_connections 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 --2- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc0130260 0x7fcdc0136c20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 --2- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdc012cf60 0x7fcdc006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 --2- 172.21.15.89:0/3757264880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012f8b0 0x7fcdc012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 -- 172.21.15.89:0/3757264880 >> 172.21.15.89:0/3757264880 conn(0x7fcdc00a2aa0 msgr2=0x7fcdc00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:18.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 -- 172.21.15.89:0/3757264880 shutdown_connections 2023-09-30T22:21:18.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.853+0000 7fcdc6555700 1 -- 172.21.15.89:0/3757264880 wait complete. 2023-09-30T22:21:18.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.854+0000 7fcdc6555700 1 Processor -- start 2023-09-30T22:21:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.854+0000 7fcdc6555700 1 -- start start 2023-09-30T22:21:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.854+0000 7fcdc6555700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012cf60 0x7fcdc009bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.854+0000 7fcdc6555700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc012f8b0 0x7fcdc009c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.854+0000 7fcdc6555700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdc0130260 0x7fcdc01c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.854+0000 7fcdc6555700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0129990 con 0x7fcdc012f8b0 2023-09-30T22:21:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.854+0000 7fcdc6555700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc012d9b0 con 0x7fcdc0130260 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.854+0000 7fcdc6555700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc009c660 con 0x7fcdc012cf60 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc012f8b0 0x7fcdc009c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc012f8b0 0x7fcdc009c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34012/0 (socket says 172.21.15.89:34012) 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 -- 172.21.15.89:0/1960275287 learned_addr learned my addr 172.21.15.89:0/1960275287 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbffff700 1 --2- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012cf60 0x7fcdc009bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 -- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012cf60 msgr2=0x7fcdc009bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 --2- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012cf60 0x7fcdc009bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 -- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdc0130260 msgr2=0x7fcdc01c3ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 --2- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdc0130260 0x7fcdc01c3ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 -- 172.21.15.89:0/1960275287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8009b30 con 0x7fcdc012f8b0 2023-09-30T22:21:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbf7fe700 1 --2- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc012f8b0 0x7fcdc009c120 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e8e0 tx=0x7fcdb400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbcff9700 1 -- 172.21.15.89:0/1960275287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400c960 con 0x7fcdc012f8b0 2023-09-30T22:21:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbcff9700 1 -- 172.21.15.89:0/1960275287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb400cac0 con 0x7fcdc012f8b0 2023-09-30T22:21:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc009b6d0 con 0x7fcdc012f8b0 2023-09-30T22:21:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.855+0000 7fcdbcff9700 1 -- 172.21.15.89:0/1960275287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400ce30 con 0x7fcdc012f8b0 2023-09-30T22:21:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.856+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc009ea50 con 0x7fcdc012f8b0 2023-09-30T22:21:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.856+0000 7fcdbcff9700 1 -- 172.21.15.89:0/1960275287 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcdb40113f0 con 0x7fcdc012f8b0 2023-09-30T22:21:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.857+0000 7fcdbcff9700 1 -- 172.21.15.89:0/1960275287 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcdb4040dc0 con 0x7fcdc012f8b0 2023-09-30T22:21:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.857+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0074ea0 con 0x7fcdc012f8b0 2023-09-30T22:21:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.858+0000 7f11a7079700 1 Processor -- start 2023-09-30T22:21:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.858+0000 7f11a7079700 1 -- start start 2023-09-30T22:21:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f11a7079700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012cf60 0x7f11a006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f11a7079700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012f8b0 0x7f11a012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f11a7079700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a012cd90 con 0x7f11a0130260 2023-09-30T22:21:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f11a7079700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a012d330 con 0x7f11a012f8b0 2023-09-30T22:21:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f11a7079700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11a01306d0 con 0x7f11a012cf60 2023-09-30T22:21:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f119ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012f8b0 0x7f11a012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f11a5616700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f11a0130260 0x7f11a0136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55286/0 (socket says 172.21.15.89:55286) 2023-09-30T22:21:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f11a5616700 1 -- 172.21.15.89:0/157357834 learned_addr learned my addr 172.21.15.89:0/157357834 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.859+0000 7f11a4e15700 1 --2- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012cf60 0x7f11a006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f11a4e15700 1 -- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012f8b0 msgr2=0x7f11a012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f11a4e15700 1 --2- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012f8b0 0x7f11a012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f11a4e15700 1 -- 172.21.15.89:0/157357834 >> v1:172.21.15.89:6789/0 conn(0x7f11a0130260 legacy=0x7f11a0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f11a4e15700 1 -- 172.21.15.89:0/157357834 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0071780 con 0x7f11a012cf60 2023-09-30T22:21:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f119ffff700 1 --2- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012f8b0 0x7f11a012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f11a4e15700 1 --2- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012cf60 0x7f11a006f5d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f119400bb10 tx=0x7f119400be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6616dccdb59aacdd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f119f7fe700 1 -- 172.21.15.89:0/157357834 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400dc70 con 0x7f11a012cf60 2023-09-30T22:21:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f119f7fe700 1 -- 172.21.15.89:0/157357834 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119400ddd0 con 0x7f11a012cf60 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.860+0000 7f119f7fe700 1 -- 172.21.15.89:0/157357834 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194010690 con 0x7f11a012cf60 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.861+0000 7f11a7079700 1 -- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012cf60 msgr2=0x7f11a006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.861+0000 7f11a7079700 1 --2- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012cf60 0x7f11a006f5d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f119400bb10 tx=0x7f119400be20 comp rx=0 tx=0).stop 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.861+0000 7f11a7079700 1 -- 172.21.15.89:0/157357834 shutdown_connections 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.861+0000 7f11a7079700 1 --2- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012cf60 0x7f11a006f5d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.861+0000 7f11a7079700 1 --2- 172.21.15.89:0/157357834 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012f8b0 0x7f11a012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.861+0000 7f11a7079700 1 -- 172.21.15.89:0/157357834 >> 172.21.15.89:0/157357834 conn(0x7f11a00a2aa0 msgr2=0x7f11a00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.861+0000 7f11a7079700 1 -- 172.21.15.89:0/157357834 shutdown_connections 2023-09-30T22:21:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.861+0000 7f11a7079700 1 -- 172.21.15.89:0/157357834 wait complete. 2023-09-30T22:21:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.862+0000 7f11a7079700 1 Processor -- start 2023-09-30T22:21:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.862+0000 7f11a7079700 1 -- start start 2023-09-30T22:21:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7fcdbcff9700 1 -- 172.21.15.89:0/1960275287 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcdb4044060 con 0x7fcdc012f8b0 2023-09-30T22:21:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a7079700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012cf60 0x7f11a01c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a7079700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012f8b0 0x7f11a01c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a7079700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a0130260 0x7f11a01c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a7079700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009d040 con 0x7f11a0130260 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a7079700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009ce50 con 0x7f11a012cf60 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a7079700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009d640 con 0x7f11a012f8b0 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a4e15700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012cf60 0x7f11a01c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a4e15700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012cf60 0x7f11a01c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37172/0 (socket says 172.21.15.89:37172) 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a4e15700 1 -- 172.21.15.89:0/2370558319 learned_addr learned my addr 172.21.15.89:0/2370558319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f119ffff700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012f8b0 0x7f11a01c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.863+0000 7f11a5616700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a0130260 0x7f11a01c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f11a4e15700 1 -- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012f8b0 msgr2=0x7f11a01c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f11a4e15700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012f8b0 0x7f11a01c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f11a4e15700 1 -- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a0130260 msgr2=0x7f11a01c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f11a4e15700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a0130260 0x7f11a01c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f11a4e15700 1 -- 172.21.15.89:0/2370558319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c00a550 con 0x7f11a012cf60 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f119ffff700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012f8b0 0x7f11a01c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f11a4e15700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012cf60 0x7f11a01c4030 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1194000c00 tx=0x7f1194004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f119d7fa700 1 -- 172.21.15.89:0/2370558319 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194003970 con 0x7f11a012cf60 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f119d7fa700 1 -- 172.21.15.89:0/2370558319 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194003ad0 con 0x7f11a012cf60 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.864+0000 7f119d7fa700 1 -- 172.21.15.89:0/2370558319 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194003e40 con 0x7f11a012cf60 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.865+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119400b740 con 0x7f11a012cf60 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.865+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a0129fa0 con 0x7f11a012cf60 2023-09-30T22:21:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.865+0000 7f119d7fa700 1 -- 172.21.15.89:0/2370558319 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11940214d0 con 0x7f11a012cf60 2023-09-30T22:21:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.866+0000 7f119d7fa700 1 -- 172.21.15.89:0/2370558319 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1194044770 con 0x7f11a012cf60 2023-09-30T22:21:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.866+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a009e720 con 0x7f11a012cf60 2023-09-30T22:21:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.875+0000 7f119d7fa700 1 -- 172.21.15.89:0/2370558319 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1194017300 con 0x7f11a012cf60 2023-09-30T22:21:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.925+0000 7f283a57a700 1 Processor -- start 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.925+0000 7f283a57a700 1 -- start start 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f283a57a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409d6f0 0x7f283409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f283a57a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f283409e410 0x7f2834130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f283a57a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283409f130 con 0x7f283409e000 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f283a57a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283409f520 con 0x7f283409e410 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f283a57a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283409e930 con 0x7f283409d6f0 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f28337fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f283409e000 0x7f2834130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55298/0 (socket says 172.21.15.89:55298) 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f28337fe700 1 -- 172.21.15.89:0/3615380448 learned_addr learned my addr 172.21.15.89:0/3615380448 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2833fff700 1 --2- 172.21.15.89:0/3615380448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409d6f0 0x7f283409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2838b17700 1 --2- 172.21.15.89:0/3615380448 >> v2:172.21.15.89:3300/0 conn(0x7f283409e410 0x7f2834130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2832ffd700 1 -- 172.21.15.89:0/3615380448 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684141419 0 0) 0x7f283409f130 con 0x7f283409e000 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2833fff700 1 -- 172.21.15.89:0/3615380448 >> v2:172.21.15.89:3300/0 conn(0x7f283409e410 msgr2=0x7f2834130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2833fff700 1 --2- 172.21.15.89:0/3615380448 >> v2:172.21.15.89:3300/0 conn(0x7f283409e410 0x7f2834130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2833fff700 1 -- 172.21.15.89:0/3615380448 >> v1:172.21.15.89:6789/0 conn(0x7f283409e000 legacy=0x7f2834130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2833fff700 1 -- 172.21.15.89:0/3615380448 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283409b2d0 con 0x7f283409d6f0 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2838b17700 1 --2- 172.21.15.89:0/3615380448 >> v2:172.21.15.89:3300/0 conn(0x7f283409e410 0x7f2834130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.926+0000 7f2833fff700 1 --2- 172.21.15.89:0/3615380448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409d6f0 0x7f283409dac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2824009840 tx=0x7f2824009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=928fc1e86dca6896 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.927+0000 7f2832ffd700 1 -- 172.21.15.89:0/3615380448 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824011830 con 0x7f283409d6f0 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.927+0000 7f2832ffd700 1 -- 172.21.15.89:0/3615380448 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2824011990 con 0x7f283409d6f0 2023-09-30T22:21:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.927+0000 7f2832ffd700 1 -- 172.21.15.89:0/3615380448 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824011d20 con 0x7f283409d6f0 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.927+0000 7f283a57a700 1 -- 172.21.15.89:0/3615380448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409d6f0 msgr2=0x7f283409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.927+0000 7f283a57a700 1 --2- 172.21.15.89:0/3615380448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409d6f0 0x7f283409dac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2824009840 tx=0x7f2824009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.928+0000 7f283a57a700 1 -- 172.21.15.89:0/3615380448 shutdown_connections 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.928+0000 7f283a57a700 1 --2- 172.21.15.89:0/3615380448 >> v2:172.21.15.89:3300/0 conn(0x7f283409e410 0x7f2834130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.928+0000 7f283a57a700 1 --2- 172.21.15.89:0/3615380448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409d6f0 0x7f283409dac0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.928+0000 7f283a57a700 1 -- 172.21.15.89:0/3615380448 >> 172.21.15.89:0/3615380448 conn(0x7f2834066f50 msgr2=0x7f283409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.928+0000 7f283a57a700 1 -- 172.21.15.89:0/3615380448 shutdown_connections 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.928+0000 7f283a57a700 1 -- 172.21.15.89:0/3615380448 wait complete. 2023-09-30T22:21:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.928+0000 7f283a57a700 1 Processor -- start 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f283a57a700 1 -- start start 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f283a57a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409d6f0 0x7f283411f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f283a57a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283409e000 0x7f283411f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f283a57a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409e410 0x7f283411fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f283a57a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28341223b0 con 0x7f283409d6f0 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f283a57a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2834072670 con 0x7f283409e410 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f283a57a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2834121b30 con 0x7f283409e000 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f2833fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409d6f0 0x7f283411f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f2833fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409d6f0 0x7f283411f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34042/0 (socket says 172.21.15.89:34042) 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f2833fff700 1 -- 172.21.15.89:0/3967709 learned_addr learned my addr 172.21.15.89:0/3967709 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f28337fe700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283409e000 0x7f283411f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.929+0000 7f2838b17700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409e410 0x7f283411fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2833fff700 1 -- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283409e000 msgr2=0x7f283411f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2833fff700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283409e000 0x7f283411f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2833fff700 1 -- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409e410 msgr2=0x7f283411fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2833fff700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409e410 0x7f283411fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2833fff700 1 -- 172.21.15.89:0/3967709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28240094a0 con 0x7f283409d6f0 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2833fff700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409d6f0 0x7f283411f100 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2824014040 tx=0x7f2824004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2830ff9700 1 -- 172.21.15.89:0/3967709 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824004480 con 0x7f283409d6f0 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2830ff9700 1 -- 172.21.15.89:0/3967709 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28240045e0 con 0x7f283409d6f0 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2834122940 con 0x7f283409d6f0 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f2830ff9700 1 -- 172.21.15.89:0/3967709 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28240038c0 con 0x7f283409d6f0 2023-09-30T22:21:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.930+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2834123300 con 0x7f283409d6f0 2023-09-30T22:21:18.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.931+0000 7f2830ff9700 1 -- 172.21.15.89:0/3967709 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2824023070 con 0x7f283409d6f0 2023-09-30T22:21:18.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.931+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283404ea50 con 0x7f283409d6f0 2023-09-30T22:21:18.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.931+0000 7f2830ff9700 1 -- 172.21.15.89:0/3967709 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f28240424c0 con 0x7f283409d6f0 2023-09-30T22:21:18.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.936+0000 7f2830ff9700 1 -- 172.21.15.89:0/3967709 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f282402d7d0 con 0x7f283409d6f0 2023-09-30T22:21:18.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.969+0000 7fd705f1d700 1 Processor -- start 2023-09-30T22:21:18.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.969+0000 7fd705f1d700 1 -- start start 2023-09-30T22:21:18.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd705f1d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd700136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd705f1d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd700136fe0 0x7fd7001393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd705f1d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd700070940 con 0x7fd70009e930 2023-09-30T22:21:18.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd705f1d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70006fd60 con 0x7fd700136fe0 2023-09-30T22:21:18.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd705f1d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd70006fa80 con 0x7fd70009f410 2023-09-30T22:21:18.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd6fffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd700136fe0 0x7fd7001393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd6ff7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd70009e930 0x7fd70009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55304/0 (socket says 172.21.15.89:55304) 2023-09-30T22:21:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd6ff7fe700 1 -- 172.21.15.89:0/82970120 learned_addr learned my addr 172.21.15.89:0/82970120 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.970+0000 7fd6feffd700 1 --2- 172.21.15.89:0/82970120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd700136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.971+0000 7fd6fffff700 1 -- 172.21.15.89:0/82970120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 msgr2=0x7fd700136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.971+0000 7fd6fffff700 1 --2- 172.21.15.89:0/82970120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd700136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.971+0000 7fd6fffff700 1 -- 172.21.15.89:0/82970120 >> v1:172.21.15.89:6789/0 conn(0x7fd70009e930 legacy=0x7fd70009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.971+0000 7fd6fffff700 1 -- 172.21.15.89:0/82970120 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70009fb60 con 0x7fd700136fe0 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.971+0000 7fd6fffff700 1 --2- 172.21.15.89:0/82970120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd700136fe0 0x7fd7001393c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009d90 tx=0x7fd6f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc0201589f4f8f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.971+0000 7fd6fe7fc700 1 -- 172.21.15.89:0/82970120 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800e070 con 0x7fd700136fe0 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.971+0000 7fd6fe7fc700 1 -- 172.21.15.89:0/82970120 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f800bb20 con 0x7fd700136fe0 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.971+0000 7fd6fe7fc700 1 -- 172.21.15.89:0/82970120 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8016840 con 0x7fd700136fe0 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.972+0000 7fd705f1d700 1 -- 172.21.15.89:0/82970120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd700136fe0 msgr2=0x7fd7001393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.972+0000 7fd705f1d700 1 --2- 172.21.15.89:0/82970120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd700136fe0 0x7fd7001393c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009d90 tx=0x7fd6f800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.972+0000 7fd705f1d700 1 -- 172.21.15.89:0/82970120 shutdown_connections 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.972+0000 7fd705f1d700 1 --2- 172.21.15.89:0/82970120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd700136fe0 0x7fd7001393c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.972+0000 7fd705f1d700 1 --2- 172.21.15.89:0/82970120 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd700136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.972+0000 7fd705f1d700 1 -- 172.21.15.89:0/82970120 >> 172.21.15.89:0/82970120 conn(0x7fd7000a4210 msgr2=0x7fd7000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.972+0000 7fd705f1d700 1 -- 172.21.15.89:0/82970120 shutdown_connections 2023-09-30T22:21:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.972+0000 7fd705f1d700 1 -- 172.21.15.89:0/82970120 wait complete. 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.973+0000 7fd705f1d700 1 Processor -- start 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.973+0000 7fd705f1d700 1 -- start start 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd705f1d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd70009e930 0x7fd70013b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd705f1d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd70013b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd705f1d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd700136fe0 0x7fd7001360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd705f1d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70006d720 con 0x7fd70009e930 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd705f1d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70013be80 con 0x7fd700136fe0 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd705f1d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd70013d710 con 0x7fd70009f410 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6feffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd70013b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6feffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd70013b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51126/0 (socket says 172.21.15.89:51126) 2023-09-30T22:21:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6feffd700 1 -- 172.21.15.89:0/1501321599 learned_addr learned my addr 172.21.15.89:0/1501321599 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6fffff700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd700136fe0 0x7fd7001360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6ff7fe700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd70009e930 0x7fd70013b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6feffd700 1 -- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd700136fe0 msgr2=0x7fd7001360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6feffd700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd700136fe0 0x7fd7001360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6feffd700 1 -- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd70009e930 msgr2=0x7fd70013b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6feffd700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd70009e930 0x7fd70013b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6feffd700 1 -- 172.21.15.89:0/1501321599 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8009a70 con 0x7fd70009f410 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.974+0000 7fd6ff7fe700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd70009e930 0x7fd70013b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.975+0000 7fd6feffd700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd70013b940 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e9e0 tx=0x7fd6f400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.975+0000 7fd6e7fff700 1 -- 172.21.15.89:0/1501321599 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4009710 con 0x7fd70009f410 2023-09-30T22:21:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.975+0000 7fd6e7fff700 1 -- 172.21.15.89:0/1501321599 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400c850 con 0x7fd70009f410 2023-09-30T22:21:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.975+0000 7fd6e7fff700 1 -- 172.21.15.89:0/1501321599 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400cbc0 con 0x7fd70009f410 2023-09-30T22:21:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.975+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd700071d20 con 0x7fd70009f410 2023-09-30T22:21:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.975+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70013c6a0 con 0x7fd70009f410 2023-09-30T22:21:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.976+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70013d270 con 0x7fd70009f410 2023-09-30T22:21:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.977+0000 7fd6e7fff700 1 -- 172.21.15.89:0/1501321599 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6f400cd20 con 0x7fd70009f410 2023-09-30T22:21:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.981+0000 7fd6e7fff700 1 -- 172.21.15.89:0/1501321599 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd6f401a360 con 0x7fd70009f410 2023-09-30T22:21:18.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:18.981+0000 7fd6e7fff700 1 -- 172.21.15.89:0/1501321599 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6f40102b0 con 0x7fd70009f410 2023-09-30T22:21:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.095+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcdc009d610 con 0x7fcdc012f8b0 2023-09-30T22:21:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.096+0000 7fcdbcff9700 1 -- 172.21.15.89:0/1960275287 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdb40116a0 con 0x7fcdc012f8b0 2023-09-30T22:21:19.097 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:19.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.097+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc012f8b0 msgr2=0x7fcdc009c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:19.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.097+0000 7fcdc6555700 1 --2- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc012f8b0 0x7fcdc009c120 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e8e0 tx=0x7fcdb400ebf0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.098+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 shutdown_connections 2023-09-30T22:21:19.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.098+0000 7fcdc6555700 1 --2- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdc0130260 0x7fcdc01c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.098+0000 7fcdc6555700 1 --2- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdc012cf60 0x7fcdc009bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.098+0000 7fcdc6555700 1 --2- 172.21.15.89:0/1960275287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdc012f8b0 0x7fcdc009c120 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.098+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 >> 172.21.15.89:0/1960275287 conn(0x7fcdc00a2aa0 msgr2=0x7fcdc00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:19.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.098+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 shutdown_connections 2023-09-30T22:21:19.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.098+0000 7fcdc6555700 1 -- 172.21.15.89:0/1960275287 wait complete. 2023-09-30T22:21:19.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.099+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f11a0074ea0 con 0x7f11a012cf60 2023-09-30T22:21:19.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.099+0000 7f119d7fa700 1 -- 172.21.15.89:0/2370558319 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f119400eb50 con 0x7f11a012cf60 2023-09-30T22:21:19.100 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012cf60 msgr2=0x7f11a01c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012cf60 0x7f11a01c4030 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1194000c00 tx=0x7f1194004750 comp rx=0 tx=0).stop 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 shutdown_connections 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a0130260 0x7f11a01c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a012cf60 0x7f11a01c4030 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 --2- 172.21.15.89:0/2370558319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a012f8b0 0x7f11a01c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 >> 172.21.15.89:0/2370558319 conn(0x7f11a00a2aa0 msgr2=0x7f11a006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 shutdown_connections 2023-09-30T22:21:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.102+0000 7f11a7079700 1 -- 172.21.15.89:0/2370558319 wait complete. 2023-09-30T22:21:19.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:19.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.147+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2834074ea0 con 0x7f283409d6f0 2023-09-30T22:21:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.148+0000 7f2830ff9700 1 -- 172.21.15.89:0/3967709 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f282402a030 con 0x7f283409d6f0 2023-09-30T22:21:19.148 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409d6f0 msgr2=0x7f283411f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409d6f0 0x7f283411f100 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2824014040 tx=0x7f2824004750 comp rx=0 tx=0).stop 2023-09-30T22:21:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 shutdown_connections 2023-09-30T22:21:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f283409e410 0x7f283411fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409d6f0 0x7f283411f100 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 --2- 172.21.15.89:0/3967709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f283409e000 0x7f283411f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 >> 172.21.15.89:0/3967709 conn(0x7f2834066f50 msgr2=0x7f2834130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:19.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 shutdown_connections 2023-09-30T22:21:19.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.150+0000 7f283a57a700 1 -- 172.21.15.89:0/3967709 wait complete. 2023-09-30T22:21:19.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:19.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.206+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd70013e150 con 0x7fd70009f410 2023-09-30T22:21:19.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.207+0000 7fd6e7fff700 1 -- 172.21.15.89:0/1501321599 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6f4017070 con 0x7fd70009f410 2023-09-30T22:21:19.208 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:19.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 msgr2=0x7fd70013b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:19.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd70013b940 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e9e0 tx=0x7fd6f400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 shutdown_connections 2023-09-30T22:21:19.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd700136fe0 0x7fd7001360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd70009e930 0x7fd70013b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 --2- 172.21.15.89:0/1501321599 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd70009f410 0x7fd70013b940 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:19.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 >> 172.21.15.89:0/1501321599 conn(0x7fd7000a4210 msgr2=0x7fd700147750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:19.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 shutdown_connections 2023-09-30T22:21:19.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:19.209+0000 7fd705f1d700 1 -- 172.21.15.89:0/1501321599 wait complete. 2023-09-30T22:21:19.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:20.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:20.117 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:20.164 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:20.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:20.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.243+0000 7fa57e10b700 1 Processor -- start 2023-09-30T22:21:20.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.243+0000 7fa57e10b700 1 -- start start 2023-09-30T22:21:20.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa57e10b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa57806b250 0x7fa578136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa57e10b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa578136d80 0x7fa578139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa57e10b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57809ece0 con 0x7fa57806a770 2023-09-30T22:21:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa57e10b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57806b640 con 0x7fa57806b250 2023-09-30T22:21:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa57e10b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa57809e760 con 0x7fa578136d80 2023-09-30T22:21:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa576ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa57806b250 0x7fa578136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa576ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa57806b250 0x7fa578136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:32942/0 (socket says 172.21.15.89:32942) 2023-09-30T22:21:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa576ffd700 1 -- 172.21.15.89:0/2880401411 learned_addr learned my addr 172.21.15.89:0/2880401411 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.244+0000 7fa577fff700 1 --2- 172.21.15.89:0/2880401411 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa578136d80 0x7fa578139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.245+0000 7fa577fff700 1 -- 172.21.15.89:0/2880401411 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa57806b250 msgr2=0x7fa578136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:20.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.245+0000 7fa577fff700 1 --2- 172.21.15.89:0/2880401411 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa57806b250 0x7fa578136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.245+0000 7fa577fff700 1 -- 172.21.15.89:0/2880401411 >> v1:172.21.15.89:6789/0 conn(0x7fa57806a770 legacy=0x7fa57806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.245+0000 7fa577fff700 1 -- 172.21.15.89:0/2880401411 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57809e040 con 0x7fa578136d80 2023-09-30T22:21:20.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.245+0000 7fa577fff700 1 --2- 172.21.15.89:0/2880401411 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa578136d80 0x7fa578139160 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa570009dc0 tx=0x7fa57000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fdb5bc0f560424 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.245+0000 7fa5767fc700 1 -- 172.21.15.89:0/2880401411 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57000e070 con 0x7fa578136d80 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.245+0000 7fa5767fc700 1 -- 172.21.15.89:0/2880401411 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57000bb20 con 0x7fa578136d80 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.245+0000 7fa5767fc700 1 -- 172.21.15.89:0/2880401411 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570012430 con 0x7fa578136d80 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.246+0000 7fa57e10b700 1 -- 172.21.15.89:0/2880401411 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa578136d80 msgr2=0x7fa578139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.246+0000 7fa57e10b700 1 --2- 172.21.15.89:0/2880401411 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa578136d80 0x7fa578139160 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa570009dc0 tx=0x7fa57000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.246+0000 7fa57e10b700 1 -- 172.21.15.89:0/2880401411 shutdown_connections 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.246+0000 7fa57e10b700 1 --2- 172.21.15.89:0/2880401411 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa578136d80 0x7fa578139160 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.246+0000 7fa57e10b700 1 --2- 172.21.15.89:0/2880401411 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa57806b250 0x7fa578136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.246+0000 7fa57e10b700 1 -- 172.21.15.89:0/2880401411 >> 172.21.15.89:0/2880401411 conn(0x7fa578066f50 msgr2=0x7fa57809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.246+0000 7fa57e10b700 1 -- 172.21.15.89:0/2880401411 shutdown_connections 2023-09-30T22:21:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.247+0000 7fa57e10b700 1 -- 172.21.15.89:0/2880401411 wait complete. 2023-09-30T22:21:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.248+0000 7fa57e10b700 1 Processor -- start 2023-09-30T22:21:20.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.248+0000 7fa57e10b700 1 -- start start 2023-09-30T22:21:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa57e10b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa57806a770 0x7fa57811cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa57e10b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa57806b250 0x7fa57811d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa57e10b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa578136d80 0x7fa5781c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa57e10b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa578071e60 con 0x7fa57806a770 2023-09-30T22:21:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa57e10b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57811da40 con 0x7fa578136d80 2023-09-30T22:21:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa57e10b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa57811cbb0 con 0x7fa57806b250 2023-09-30T22:21:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa57806b250 0x7fa57811d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa57806b250 0x7fa57811d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59550/0 (socket says 172.21.15.89:59550) 2023-09-30T22:21:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 -- 172.21.15.89:0/305843471 learned_addr learned my addr 172.21.15.89:0/305843471 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa5777fe700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa57806a770 0x7fa57811cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa577fff700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa578136d80 0x7fa5781c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 -- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa578136d80 msgr2=0x7fa5781c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa578136d80 0x7fa5781c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 -- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa57806a770 msgr2=0x7fa57811cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa57806a770 0x7fa57811cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 -- 172.21.15.89:0/305843471 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa570009a70 con 0x7fa57806b250 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.249+0000 7fa576ffd700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa57806b250 0x7fa57811d500 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00e940 tx=0x7fa56c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.250+0000 7fa55ffff700 1 -- 172.21.15.89:0/305843471 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00ee90 con 0x7fa57806b250 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.250+0000 7fa55ffff700 1 -- 172.21.15.89:0/305843471 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c00c850 con 0x7fa57806b250 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.250+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57806e8e0 con 0x7fa57806b250 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.250+0000 7fa55ffff700 1 -- 172.21.15.89:0/305843471 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00cbc0 con 0x7fa57806b250 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.250+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57811e9b0 con 0x7fa57806b250 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.251+0000 7fa55ffff700 1 -- 172.21.15.89:0/305843471 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa56c00cd20 con 0x7fa57806b250 2023-09-30T22:21:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.251+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa578120920 con 0x7fa57806b250 2023-09-30T22:21:20.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.251+0000 7fa55ffff700 1 -- 172.21.15.89:0/305843471 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa56c03faa0 con 0x7fa57806b250 2023-09-30T22:21:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.259+0000 7fa55ffff700 1 -- 172.21.15.89:0/305843471 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa56c011b40 con 0x7fa57806b250 2023-09-30T22:21:20.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.291+0000 7f7cf4a05700 1 Processor -- start 2023-09-30T22:21:20.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.291+0000 7f7cf4a05700 1 -- start start 2023-09-30T22:21:20.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.292+0000 7f7cf4a05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.292+0000 7f7cf4a05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e000 0x7f7cf0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.292+0000 7f7cf4a05700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7cf009e410 0x7f7cf0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.292+0000 7f7cf4a05700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf009f520 con 0x7f7cf009e410 2023-09-30T22:21:20.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.292+0000 7f7cf4a05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf009f130 con 0x7f7cf009e000 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.292+0000 7f7cf4a05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf009e930 con 0x7f7cf009d6f0 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7ceed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7cf009e410 0x7f7cf0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7ceed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7cf009e410 0x7f7cf0130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49392/0 (socket says 172.21.15.89:49392) 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7ceed9d700 1 -- 172.21.15.89:0/1630063604 learned_addr learned my addr 172.21.15.89:0/1630063604 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7cedd9b700 1 --2- 172.21.15.89:0/1630063604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e000 0x7f7cf0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7cedd9b700 1 -- 172.21.15.89:0/1630063604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 msgr2=0x7f7cf009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7cedd9b700 1 --2- 172.21.15.89:0/1630063604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7cedd9b700 1 -- 172.21.15.89:0/1630063604 >> v2:172.21.15.89:3300/0 conn(0x7f7cf009e410 msgr2=0x7f7cf0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7cedd9b700 1 --2- 172.21.15.89:0/1630063604 >> v2:172.21.15.89:3300/0 conn(0x7f7cf009e410 0x7f7cf0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7cedd9b700 1 -- 172.21.15.89:0/1630063604 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf009b2d0 con 0x7f7cf009e000 2023-09-30T22:21:20.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7ceed9d700 1 --2- 172.21.15.89:0/1630063604 >> v2:172.21.15.89:3300/0 conn(0x7f7cf009e410 0x7f7cf0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:20.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7cedd9b700 1 --2- 172.21.15.89:0/1630063604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e000 0x7f7cf0130660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009870 tx=0x7f7ce4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7e704e2404acf56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7ced59a700 1 -- 172.21.15.89:0/1630063604 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4011830 con 0x7f7cf009e000 2023-09-30T22:21:20.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7ced59a700 1 -- 172.21.15.89:0/1630063604 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce4011990 con 0x7f7cf009e000 2023-09-30T22:21:20.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.293+0000 7f7ced59a700 1 -- 172.21.15.89:0/1630063604 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4011d20 con 0x7f7cf009e000 2023-09-30T22:21:20.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 -- 172.21.15.89:0/1630063604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e000 msgr2=0x7f7cf0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 --2- 172.21.15.89:0/1630063604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e000 0x7f7cf0130660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009870 tx=0x7f7ce4009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:20.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 -- 172.21.15.89:0/1630063604 shutdown_connections 2023-09-30T22:21:20.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 --2- 172.21.15.89:0/1630063604 >> v2:172.21.15.89:3300/0 conn(0x7f7cf009e410 0x7f7cf0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 --2- 172.21.15.89:0/1630063604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 --2- 172.21.15.89:0/1630063604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e000 0x7f7cf0130660 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 -- 172.21.15.89:0/1630063604 >> 172.21.15.89:0/1630063604 conn(0x7f7cf0066f50 msgr2=0x7f7cf009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 -- 172.21.15.89:0/1630063604 shutdown_connections 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.294+0000 7f7cf4a05700 1 -- 172.21.15.89:0/1630063604 wait complete. 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.296+0000 7f7cf4a05700 1 Processor -- start 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.296+0000 7f7cf4a05700 1 -- start start 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.296+0000 7f7cf4a05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf011dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.296+0000 7f7cf4a05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cf009e000 0x7f7cf011e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.296+0000 7f7cf4a05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e410 0x7f7cf011ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cf4a05700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf011f470 con 0x7f7cf009e000 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cf4a05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf0130380 con 0x7f7cf009e410 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cf4a05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf01201d0 con 0x7f7cf009d6f0 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf011dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf011dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59574/0 (socket says 172.21.15.89:59574) 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 -- 172.21.15.89:0/4107278885 learned_addr learned my addr 172.21.15.89:0/4107278885 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7ceed9d700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e410 0x7f7cf011ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cedd9b700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cf009e000 0x7f7cf011e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 -- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e410 msgr2=0x7f7cf011ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e410 0x7f7cf011ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 -- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cf009e000 msgr2=0x7f7cf011e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cf009e000 0x7f7cf011e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 -- 172.21.15.89:0/4107278885 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8009b30 con 0x7f7cf009d6f0 2023-09-30T22:21:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7ceed9d700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e410 0x7f7cf011ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.297+0000 7f7cee59c700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf011dfa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000b560 tx=0x7f7ce000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.298+0000 7f7cdeffd700 1 -- 172.21.15.89:0/4107278885 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0014d70 con 0x7f7cf009d6f0 2023-09-30T22:21:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.298+0000 7f7cdeffd700 1 -- 172.21.15.89:0/4107278885 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce001c440 con 0x7f7cf009d6f0 2023-09-30T22:21:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.298+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce40094a0 con 0x7f7cf009d6f0 2023-09-30T22:21:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.298+0000 7f7cdeffd700 1 -- 172.21.15.89:0/4107278885 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce001c7f0 con 0x7f7cf009d6f0 2023-09-30T22:21:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.298+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf01212a0 con 0x7f7cf009d6f0 2023-09-30T22:21:20.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.299+0000 7f7cdeffd700 1 -- 172.21.15.89:0/4107278885 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ce0024360 con 0x7f7cf009d6f0 2023-09-30T22:21:20.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.299+0000 7f7cdeffd700 1 -- 172.21.15.89:0/4107278885 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7ce0047310 con 0x7f7cf009d6f0 2023-09-30T22:21:20.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.299+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf0120e60 con 0x7f7cf009d6f0 2023-09-30T22:21:20.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.304+0000 7f7cdeffd700 1 -- 172.21.15.89:0/4107278885 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ce0021080 con 0x7f7cf009d6f0 2023-09-30T22:21:20.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f93027700 1 Processor -- start 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f93027700 1 -- start start 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f93027700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c09f410 0x7f0f8c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f93027700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f8c136fe0 0x7f0f8c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f93027700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c070940 con 0x7f0f8c09e930 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f93027700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c06fd60 con 0x7f0f8c136fe0 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f93027700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c06fa80 con 0x7f0f8c09f410 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f8bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c09f410 0x7f0f8c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f8bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c09f410 0x7f0f8c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:32968/0 (socket says 172.21.15.89:32968) 2023-09-30T22:21:20.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.331+0000 7f0f8bfff700 1 -- 172.21.15.89:0/4245378610 learned_addr learned my addr 172.21.15.89:0/4245378610 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.332+0000 7f0f8bfff700 1 -- 172.21.15.89:0/4245378610 >> v2:172.21.15.89:3300/0 conn(0x7f0f8c136fe0 msgr2=0x7f0f8c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.332+0000 7f0f8bfff700 1 --2- 172.21.15.89:0/4245378610 >> v2:172.21.15.89:3300/0 conn(0x7f0f8c136fe0 0x7f0f8c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.332+0000 7f0f8bfff700 1 -- 172.21.15.89:0/4245378610 >> v1:172.21.15.89:6789/0 conn(0x7f0f8c09e930 legacy=0x7f0f8c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.332+0000 7f0f8bfff700 1 -- 172.21.15.89:0/4245378610 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c09fb60 con 0x7f0f8c09f410 2023-09-30T22:21:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.332+0000 7f0f8bfff700 1 --2- 172.21.15.89:0/4245378610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c09f410 0x7f0f8c136aa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0f780098f0 tx=0x7f0f78009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9443ca0f0c0078f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.332+0000 7f0f8b7fe700 1 -- 172.21.15.89:0/4245378610 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78004230 con 0x7f0f8c09f410 2023-09-30T22:21:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.332+0000 7f0f8b7fe700 1 -- 172.21.15.89:0/4245378610 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f78004390 con 0x7f0f8c09f410 2023-09-30T22:21:20.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.332+0000 7f0f8b7fe700 1 -- 172.21.15.89:0/4245378610 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f780036a0 con 0x7f0f8c09f410 2023-09-30T22:21:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.333+0000 7f0f93027700 1 -- 172.21.15.89:0/4245378610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c09f410 msgr2=0x7f0f8c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.333+0000 7f0f93027700 1 --2- 172.21.15.89:0/4245378610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c09f410 0x7f0f8c136aa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0f780098f0 tx=0x7f0f78009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.333+0000 7f0f93027700 1 -- 172.21.15.89:0/4245378610 shutdown_connections 2023-09-30T22:21:20.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.333+0000 7f0f93027700 1 --2- 172.21.15.89:0/4245378610 >> v2:172.21.15.89:3300/0 conn(0x7f0f8c136fe0 0x7f0f8c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.333+0000 7f0f93027700 1 --2- 172.21.15.89:0/4245378610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c09f410 0x7f0f8c136aa0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.333+0000 7f0f93027700 1 -- 172.21.15.89:0/4245378610 >> 172.21.15.89:0/4245378610 conn(0x7f0f8c0a4210 msgr2=0x7f0f8c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.333+0000 7f0f93027700 1 -- 172.21.15.89:0/4245378610 shutdown_connections 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.333+0000 7f0f93027700 1 -- 172.21.15.89:0/4245378610 wait complete. 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f93027700 1 Processor -- start 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f93027700 1 -- start start 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f93027700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f8c09e930 0x7f0f8c1c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f93027700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c136fe0 0x7f0f8c1c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f93027700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c1c6dc0 0x7f0f8c133460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f93027700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1207d0 con 0x7f0f8c136fe0 2023-09-30T22:21:20.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f93027700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1205e0 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f93027700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1224f0 con 0x7f0f8c09e930 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f915c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c1c6dc0 0x7f0f8c133460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f90dc3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f8c09e930 0x7f0f8c1c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f915c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c1c6dc0 0x7f0f8c133460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:32978/0 (socket says 172.21.15.89:32978) 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f90dc3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f8c09e930 0x7f0f8c1c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59578/0 (socket says 172.21.15.89:59578) 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f915c4700 1 -- 172.21.15.89:0/819669695 learned_addr learned my addr 172.21.15.89:0/819669695 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.334+0000 7f0f8bfff700 1 --2- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c136fe0 0x7f0f8c1c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f915c4700 1 -- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f8c09e930 msgr2=0x7f0f8c1c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f915c4700 1 --2- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f8c09e930 0x7f0f8c1c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f915c4700 1 -- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c136fe0 msgr2=0x7f0f8c1c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f915c4700 1 --2- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c136fe0 0x7f0f8c1c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f915c4700 1 -- 172.21.15.89:0/819669695 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f78009550 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f915c4700 1 --2- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c1c6dc0 0x7f0f8c133460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400b9e0 tx=0x7f0f8400bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f897fa700 1 -- 172.21.15.89:0/819669695 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400d8e0 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f897fa700 1 -- 172.21.15.89:0/819669695 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8400da80 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c0a31d0 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f897fa700 1 -- 172.21.15.89:0/819669695 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400de30 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.335+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c122210 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.336+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c121e90 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.337+0000 7f0f897fa700 1 -- 172.21.15.89:0/819669695 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f84015420 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.337+0000 7f0f897fa700 1 -- 172.21.15.89:0/819669695 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0f84011070 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.341+0000 7f0f897fa700 1 -- 172.21.15.89:0/819669695 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f84013920 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.375+0000 7f84f9aad700 1 Processor -- start 2023-09-30T22:21:20.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.375+0000 7f84f9aad700 1 -- start start 2023-09-30T22:21:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.376+0000 7f84f9aad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84f41316c0 0x7f84f4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.376+0000 7f84f9aad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f406ae80 0x7f84f4139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.376+0000 7f84f9aad700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f409e130 con 0x7f84f409d9f0 2023-09-30T22:21:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.376+0000 7f84f9aad700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f41312b0 con 0x7f84f41316c0 2023-09-30T22:21:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.376+0000 7f84f9aad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f409f6e0 con 0x7f84f406ae80 2023-09-30T22:21:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.376+0000 7f84f37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84f41316c0 0x7f84f4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.376+0000 7f84f37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84f41316c0 0x7f84f4131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49432/0 (socket says 172.21.15.89:49432) 2023-09-30T22:21:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.376+0000 7f84f37fe700 1 -- 172.21.15.89:0/2708215109 learned_addr learned my addr 172.21.15.89:0/2708215109 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.377+0000 7f84f37fe700 1 -- 172.21.15.89:0/2708215109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f406ae80 msgr2=0x7f84f4139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.377+0000 7f84f37fe700 1 --2- 172.21.15.89:0/2708215109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f406ae80 0x7f84f4139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.377+0000 7f84f37fe700 1 -- 172.21.15.89:0/2708215109 >> v1:172.21.15.89:6789/0 conn(0x7f84f409d9f0 legacy=0x7f84f406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.377+0000 7f84f37fe700 1 -- 172.21.15.89:0/2708215109 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f409e760 con 0x7f84f41316c0 2023-09-30T22:21:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.377+0000 7f84f37fe700 1 --2- 172.21.15.89:0/2708215109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f41316c0 0x7f84f4131a90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f84e40098f0 tx=0x7f84e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569b2d2370f59632 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.377+0000 7f84f27fc700 1 -- 172.21.15.89:0/2708215109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e400b8a0 con 0x7f84f41316c0 2023-09-30T22:21:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.377+0000 7f84f27fc700 1 -- 172.21.15.89:0/2708215109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e400ba00 con 0x7f84f41316c0 2023-09-30T22:21:20.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.377+0000 7f84f27fc700 1 -- 172.21.15.89:0/2708215109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e400bd90 con 0x7f84f41316c0 2023-09-30T22:21:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.378+0000 7f84f9aad700 1 -- 172.21.15.89:0/2708215109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f41316c0 msgr2=0x7f84f4131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.378+0000 7f84f9aad700 1 --2- 172.21.15.89:0/2708215109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f41316c0 0x7f84f4131a90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f84e40098f0 tx=0x7f84e4009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.378+0000 7f84f9aad700 1 -- 172.21.15.89:0/2708215109 shutdown_connections 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.378+0000 7f84f9aad700 1 --2- 172.21.15.89:0/2708215109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f406ae80 0x7f84f4139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.378+0000 7f84f9aad700 1 --2- 172.21.15.89:0/2708215109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f41316c0 0x7f84f4131a90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.378+0000 7f84f9aad700 1 -- 172.21.15.89:0/2708215109 >> 172.21.15.89:0/2708215109 conn(0x7f84f4066f50 msgr2=0x7f84f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.378+0000 7f84f9aad700 1 -- 172.21.15.89:0/2708215109 shutdown_connections 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.378+0000 7f84f9aad700 1 -- 172.21.15.89:0/2708215109 wait complete. 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.379+0000 7f84f9aad700 1 Processor -- start 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.379+0000 7f84f9aad700 1 -- start start 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.379+0000 7f84f9aad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84f406ae80 0x7f84f4139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f9aad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f409d9f0 0x7f84f4071220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f9aad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f41316c0 0x7f84f4071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f9aad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f40726f0 con 0x7f84f409d9f0 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f9aad700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f413a230 con 0x7f84f41316c0 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f9aad700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84f41327a0 con 0x7f84f406ae80 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f409d9f0 0x7f84f4071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84f406ae80 0x7f84f4139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f41316c0 0x7f84f4071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f409d9f0 0x7f84f4071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49444/0 (socket says 172.21.15.89:49444) 2023-09-30T22:21:20.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84f406ae80 0x7f84f4139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59584/0 (socket says 172.21.15.89:59584) 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 -- 172.21.15.89:0/1069857861 learned_addr learned my addr 172.21.15.89:0/1069857861 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 -- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84f406ae80 msgr2=0x7f84f4139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 --2- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84f406ae80 0x7f84f4139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 -- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f41316c0 msgr2=0x7f84f4071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 --2- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f41316c0 0x7f84f4071760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 -- 172.21.15.89:0/1069857861 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4009550 con 0x7f84f409d9f0 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f2ffd700 1 --2- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f409d9f0 0x7f84f4071220 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f84e800c4e0 tx=0x7f84e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84dbfff700 1 -- 172.21.15.89:0/1069857861 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e80052d0 con 0x7f84f409d9f0 2023-09-30T22:21:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84dbfff700 1 -- 172.21.15.89:0/1069857861 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8005430 con 0x7f84f409d9f0 2023-09-30T22:21:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.380+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f4132c30 con 0x7f84f409d9f0 2023-09-30T22:21:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.381+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f4133480 con 0x7f84f409d9f0 2023-09-30T22:21:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.381+0000 7f84dbfff700 1 -- 172.21.15.89:0/1069857861 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e80038c0 con 0x7f84f409d9f0 2023-09-30T22:21:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.381+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f4074ea0 con 0x7f84f409d9f0 2023-09-30T22:21:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.382+0000 7f84dbfff700 1 -- 172.21.15.89:0/1069857861 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f84e8003a20 con 0x7f84f409d9f0 2023-09-30T22:21:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.382+0000 7f84dbfff700 1 -- 172.21.15.89:0/1069857861 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f84e803eaf0 con 0x7f84f409d9f0 2023-09-30T22:21:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.390+0000 7f84dbfff700 1 -- 172.21.15.89:0/1069857861 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f84e8011590 con 0x7f84f409d9f0 2023-09-30T22:21:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.497+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa578074ea0 con 0x7fa57806b250 2023-09-30T22:21:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.498+0000 7fa55ffff700 1 -- 172.21.15.89:0/305843471 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa56c017030 con 0x7fa57806b250 2023-09-30T22:21:20.499 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa57806b250 msgr2=0x7fa57811d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa57806b250 0x7fa57811d500 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00e940 tx=0x7fa56c00ec50 comp rx=0 tx=0).stop 2023-09-30T22:21:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 shutdown_connections 2023-09-30T22:21:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa578136d80 0x7fa5781c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa57806a770 0x7fa57811cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 --2- 172.21.15.89:0/305843471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa57806b250 0x7fa57811d500 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 >> 172.21.15.89:0/305843471 conn(0x7fa578066f50 msgr2=0x7fa5780a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 shutdown_connections 2023-09-30T22:21:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.500+0000 7fa57e10b700 1 -- 172.21.15.89:0/305843471 wait complete. 2023-09-30T22:21:20.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:20.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.539+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7cf0074ea0 con 0x7f7cf009d6f0 2023-09-30T22:21:20.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.540+0000 7f7cdeffd700 1 -- 172.21.15.89:0/4107278885 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ce0023e60 con 0x7f7cf009d6f0 2023-09-30T22:21:20.541 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:20.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.541+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0f8c074ea0 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.541+0000 7f0f897fa700 1 -- 172.21.15.89:0/819669695 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f84013330 con 0x7f0f8c1c6dc0 2023-09-30T22:21:20.543 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:20.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 msgr2=0x7f7cf011dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf011dfa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000b560 tx=0x7f7ce000b870 comp rx=0 tx=0).stop 2023-09-30T22:21:20.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 shutdown_connections 2023-09-30T22:21:20.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7cf009e410 0x7f7cf011ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7cf009d6f0 0x7f7cf011dfa0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 --2- 172.21.15.89:0/4107278885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cf009e000 0x7f7cf011e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 >> 172.21.15.89:0/4107278885 conn(0x7f7cf0066f50 msgr2=0x7f7cf006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 shutdown_connections 2023-09-30T22:21:20.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.542+0000 7f7cf4a05700 1 -- 172.21.15.89:0/4107278885 wait complete. 2023-09-30T22:21:20.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c1c6dc0 msgr2=0x7f0f8c133460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 --2- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c1c6dc0 0x7f0f8c133460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400b9e0 tx=0x7f0f8400bcf0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 shutdown_connections 2023-09-30T22:21:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 --2- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f8c1c6dc0 0x7f0f8c133460 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 --2- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f8c09e930 0x7f0f8c1c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 --2- 172.21.15.89:0/819669695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f8c136fe0 0x7f0f8c1c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 >> 172.21.15.89:0/819669695 conn(0x7f0f8c0a4210 msgr2=0x7f0f8c09bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 shutdown_connections 2023-09-30T22:21:20.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.543+0000 7f0f93027700 1 -- 172.21.15.89:0/819669695 wait complete. 2023-09-30T22:21:20.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:20.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:20.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.586+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f84f409e130 con 0x7f84f409d9f0 2023-09-30T22:21:20.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.586+0000 7f84dbfff700 1 -- 172.21.15.89:0/1069857861 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f84f409e130 con 0x7f84f409d9f0 2023-09-30T22:21:20.587 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:20.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f409d9f0 msgr2=0x7f84f4071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 --2- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f409d9f0 0x7f84f4071220 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f84e800c4e0 tx=0x7f84e800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 shutdown_connections 2023-09-30T22:21:20.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 --2- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84f41316c0 0x7f84f4071760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 --2- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84f406ae80 0x7f84f4139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 --2- 172.21.15.89:0/1069857861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f409d9f0 0x7f84f4071220 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:20.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 >> 172.21.15.89:0/1069857861 conn(0x7f84f4066f50 msgr2=0x7f84f409bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 shutdown_connections 2023-09-30T22:21:20.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:20.588+0000 7f84f9aad700 1 -- 172.21.15.89:0/1069857861 wait complete. 2023-09-30T22:21:20.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:21.515 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:21.558 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:21.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:21.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.661+0000 7fbc497d6700 1 Processor -- start 2023-09-30T22:21:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.661+0000 7fbc497d6700 1 -- start start 2023-09-30T22:21:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.661+0000 7fbc497d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 0x7fbc44131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.661+0000 7fbc497d6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc44131f40 0x7fbc4406e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc497d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc4406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc497d6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc44071a00 con 0x7fbc44131f40 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc497d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc44131430 con 0x7fbc44131630 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc497d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4409a510 con 0x7fbc4406e9d0 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc4406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc427fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc44131f40 0x7fbc4406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc4406ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59590/0 (socket says 172.21.15.89:59590) 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 -- 172.21.15.89:0/1584204580 learned_addr learned my addr 172.21.15.89:0/1584204580 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc427fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc44131f40 0x7fbc4406e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49452/0 (socket says 172.21.15.89:49452) 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc42ffd700 1 --2- 172.21.15.89:0/1584204580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 0x7fbc44131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 -- 172.21.15.89:0/1584204580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 msgr2=0x7fbc44131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 --2- 172.21.15.89:0/1584204580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 0x7fbc44131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 -- 172.21.15.89:0/1584204580 >> v2:172.21.15.89:3300/0 conn(0x7fbc44131f40 msgr2=0x7fbc4406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 --2- 172.21.15.89:0/1584204580 >> v2:172.21.15.89:3300/0 conn(0x7fbc44131f40 0x7fbc4406e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 -- 172.21.15.89:0/1584204580 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440721c0 con 0x7fbc4406e9d0 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc427fc700 1 --2- 172.21.15.89:0/1584204580 >> v2:172.21.15.89:3300/0 conn(0x7fbc44131f40 0x7fbc4406e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.662+0000 7fbc437fe700 1 --2- 172.21.15.89:0/1584204580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc4406ee70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009e50 tx=0x7fbc3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ac1a64614381fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.663+0000 7fbc41ffb700 1 -- 172.21.15.89:0/1584204580 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00bb10 con 0x7fbc4406e9d0 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.663+0000 7fbc41ffb700 1 -- 172.21.15.89:0/1584204580 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c00bc70 con 0x7fbc4406e9d0 2023-09-30T22:21:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.663+0000 7fbc41ffb700 1 -- 172.21.15.89:0/1584204580 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0104d0 con 0x7fbc4406e9d0 2023-09-30T22:21:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 -- 172.21.15.89:0/1584204580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 msgr2=0x7fbc4406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 --2- 172.21.15.89:0/1584204580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc4406ee70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009e50 tx=0x7fbc3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 -- 172.21.15.89:0/1584204580 shutdown_connections 2023-09-30T22:21:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 --2- 172.21.15.89:0/1584204580 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc4406ee70 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 --2- 172.21.15.89:0/1584204580 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 0x7fbc44131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 --2- 172.21.15.89:0/1584204580 >> v2:172.21.15.89:3300/0 conn(0x7fbc44131f40 0x7fbc4406e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 -- 172.21.15.89:0/1584204580 >> 172.21.15.89:0/1584204580 conn(0x7fbc440a2aa0 msgr2=0x7fbc440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 -- 172.21.15.89:0/1584204580 shutdown_connections 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.664+0000 7fbc497d6700 1 -- 172.21.15.89:0/1584204580 wait complete. 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.665+0000 7fbc497d6700 1 Processor -- start 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.665+0000 7fbc497d6700 1 -- start start 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc497d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc441197a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc497d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 0x7fbc44119ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc497d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc44131f40 0x7fbc4411a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc497d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4411ac70 con 0x7fbc44131f40 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc497d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4412cb90 con 0x7fbc44131630 2023-09-30T22:21:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc497d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4411b9d0 con 0x7fbc4406e9d0 2023-09-30T22:21:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc44131f40 0x7fbc4411a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc44131f40 0x7fbc4411a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49468/0 (socket says 172.21.15.89:49468) 2023-09-30T22:21:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc437fe700 1 -- 172.21.15.89:0/2777339303 learned_addr learned my addr 172.21.15.89:0/2777339303 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc42ffd700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc441197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.666+0000 7fbc427fc700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 0x7fbc44119ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc42ffd700 1 -- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 msgr2=0x7fbc44119ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc42ffd700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 0x7fbc44119ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc42ffd700 1 -- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc44131f40 msgr2=0x7fbc4411a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc42ffd700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc44131f40 0x7fbc4411a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc42ffd700 1 -- 172.21.15.89:0/2777339303 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc380094a0 con 0x7fbc4406e9d0 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc437fe700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc44131f40 0x7fbc4411a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc42ffd700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc441197a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e930 tx=0x7fbc3400ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc2b7fe700 1 -- 172.21.15.89:0/2777339303 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34013680 con 0x7fbc4406e9d0 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc2b7fe700 1 -- 172.21.15.89:0/2777339303 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc340137e0 con 0x7fbc4406e9d0 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc2b7fe700 1 -- 172.21.15.89:0/2777339303 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34013b50 con 0x7fbc4406e9d0 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c009b30 con 0x7fbc4406e9d0 2023-09-30T22:21:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.667+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4409f7b0 con 0x7fbc4406e9d0 2023-09-30T22:21:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.668+0000 7fbc2b7fe700 1 -- 172.21.15.89:0/2777339303 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc34013cb0 con 0x7fbc4406e9d0 2023-09-30T22:21:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.668+0000 7fbc2b7fe700 1 -- 172.21.15.89:0/2777339303 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbc3403f660 con 0x7fbc4406e9d0 2023-09-30T22:21:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.668+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc44099f80 con 0x7fbc4406e9d0 2023-09-30T22:21:21.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.673+0000 7fbc2b7fe700 1 -- 172.21.15.89:0/2777339303 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc3401bc30 con 0x7fbc4406e9d0 2023-09-30T22:21:21.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.694+0000 7f86a39cb700 1 Processor -- start 2023-09-30T22:21:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a39cb700 1 -- start start 2023-09-30T22:21:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a39cb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f869c06e570 0x7f869c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a39cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c09eca0 0x7f869c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a39cb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c070ee0 con 0x7f869c134610 2023-09-30T22:21:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a39cb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c071480 con 0x7f869c06e570 2023-09-30T22:21:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a39cb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f869c06f3c0 con 0x7f869c09eca0 2023-09-30T22:21:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a0f66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c09eca0 0x7f869c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a0f66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c09eca0 0x7f869c06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59610/0 (socket says 172.21.15.89:59610) 2023-09-30T22:21:21.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a0f66700 1 -- 172.21.15.89:0/4017305286 learned_addr learned my addr 172.21.15.89:0/4017305286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:21.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a1767700 1 --2- 172.21.15.89:0/4017305286 >> v2:172.21.15.89:3300/0 conn(0x7f869c06e570 0x7f869c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a0f66700 1 -- 172.21.15.89:0/4017305286 >> v2:172.21.15.89:3300/0 conn(0x7f869c06e570 msgr2=0x7f869c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a0f66700 1 --2- 172.21.15.89:0/4017305286 >> v2:172.21.15.89:3300/0 conn(0x7f869c06e570 0x7f869c06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.695+0000 7f86a0f66700 1 -- 172.21.15.89:0/4017305286 >> v1:172.21.15.89:6789/0 conn(0x7f869c134610 legacy=0x7f869c134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.696+0000 7f86a0f66700 1 -- 172.21.15.89:0/4017305286 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c071b90 con 0x7f869c09eca0 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.696+0000 7f86a1767700 1 --2- 172.21.15.89:0/4017305286 >> v2:172.21.15.89:3300/0 conn(0x7f869c06e570 0x7f869c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.696+0000 7f86a0f66700 1 --2- 172.21.15.89:0/4017305286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c09eca0 0x7f869c06ee80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f868c0098f0 tx=0x7f868c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7212b31a8b47f9d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.696+0000 7f8693fff700 1 -- 172.21.15.89:0/4017305286 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00b8a0 con 0x7f869c09eca0 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.696+0000 7f8693fff700 1 -- 172.21.15.89:0/4017305286 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00ba00 con 0x7f869c09eca0 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.696+0000 7f8693fff700 1 -- 172.21.15.89:0/4017305286 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00bd90 con 0x7f869c09eca0 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.696+0000 7f86a39cb700 1 -- 172.21.15.89:0/4017305286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c09eca0 msgr2=0x7f869c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.696+0000 7f86a39cb700 1 --2- 172.21.15.89:0/4017305286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c09eca0 0x7f869c06ee80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f868c0098f0 tx=0x7f868c009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.697+0000 7f86a39cb700 1 -- 172.21.15.89:0/4017305286 shutdown_connections 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.697+0000 7f86a39cb700 1 --2- 172.21.15.89:0/4017305286 >> v2:172.21.15.89:3300/0 conn(0x7f869c06e570 0x7f869c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.697+0000 7f86a39cb700 1 --2- 172.21.15.89:0/4017305286 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c09eca0 0x7f869c06ee80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.697+0000 7f86a39cb700 1 -- 172.21.15.89:0/4017305286 >> 172.21.15.89:0/4017305286 conn(0x7f869c0a2aa0 msgr2=0x7f869c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.697+0000 7f86a39cb700 1 -- 172.21.15.89:0/4017305286 shutdown_connections 2023-09-30T22:21:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.697+0000 7f86a39cb700 1 -- 172.21.15.89:0/4017305286 wait complete. 2023-09-30T22:21:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a39cb700 1 Processor -- start 2023-09-30T22:21:21.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a39cb700 1 -- start start 2023-09-30T22:21:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a39cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f869c06e570 0x7f869c13ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a39cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c134610 0x7f869c13ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a39cb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f869c13e370 0x7f869c13f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a39cb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c13cd60 con 0x7f869c06e570 2023-09-30T22:21:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a39cb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c13bf60 con 0x7f869c13e370 2023-09-30T22:21:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a39cb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f869c13de70 con 0x7f869c134610 2023-09-30T22:21:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a0f66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c134610 0x7f869c13ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a0f66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c134610 0x7f869c13ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59618/0 (socket says 172.21.15.89:59618) 2023-09-30T22:21:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a0f66700 1 -- 172.21.15.89:0/1920183146 learned_addr learned my addr 172.21.15.89:0/1920183146 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a1767700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f869c06e570 0x7f869c13ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f86a1f68700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f869c13e370 0x7f869c13f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f3126e6d700 1 Processor -- start 2023-09-30T22:21:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f86a0f66700 1 -- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f869c13e370 msgr2=0x7f869c13f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f86a0f66700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f869c13e370 0x7f869c13f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f86a0f66700 1 -- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f869c06e570 msgr2=0x7f869c13ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f86a0f66700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f869c06e570 0x7f869c13ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f86a0f66700 1 -- 172.21.15.89:0/1920183146 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c009550 con 0x7f869c134610 2023-09-30T22:21:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.698+0000 7f3126e6d700 1 -- start start 2023-09-30T22:21:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f86a0f66700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c134610 0x7f869c13ef90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f868c009c40 tx=0x7f868c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f8691ffb700 1 -- 172.21.15.89:0/1920183146 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c004300 con 0x7f869c134610 2023-09-30T22:21:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f8691ffb700 1 -- 172.21.15.89:0/1920183146 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c01f070 con 0x7f869c134610 2023-09-30T22:21:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c13d8d0 con 0x7f869c134610 2023-09-30T22:21:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f3126e6d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f312006f5d0 0x7f312012cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f8691ffb700 1 -- 172.21.15.89:0/1920183146 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0036a0 con 0x7f869c134610 2023-09-30T22:21:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f3126e6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f312012f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f3126e6d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3120066880 0x7f312012fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f3126e6d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3120071780 con 0x7f312006f5d0 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f3126e6d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312006f9a0 con 0x7f3120066880 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f3126e6d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3120071a40 con 0x7f3120066410 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f312540a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3120066880 0x7f312012fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f3124c09700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f312006f5d0 0x7f312012cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f312540a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3120066880 0x7f312012fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33018/0 (socket says 172.21.15.89:33018) 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f312540a700 1 -- 172.21.15.89:0/4051997149 learned_addr learned my addr 172.21.15.89:0/4051997149 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.699+0000 7f311ffff700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f312012f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.700+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c070610 con 0x7f869c134610 2023-09-30T22:21:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.700+0000 7f3124c09700 1 -- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 msgr2=0x7f312012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.700+0000 7f3124c09700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f312012f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.700+0000 7f3124c09700 1 -- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3120066880 msgr2=0x7f312012fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.700+0000 7f3124c09700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3120066880 0x7f312012fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.700+0000 7f3124c09700 1 -- 172.21.15.89:0/4051997149 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120072220 con 0x7f312006f5d0 2023-09-30T22:21:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.700+0000 7f311ffff700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f312012f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.700+0000 7f3124c09700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f312006f5d0 0x7f312012cd90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3114009840 tx=0x7f3114009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc0b914046d16ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.701+0000 7f311f7fe700 1 -- 172.21.15.89:0/4051997149 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400f8a0 con 0x7f312006f5d0 2023-09-30T22:21:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.701+0000 7f311f7fe700 1 -- 172.21.15.89:0/4051997149 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311400fa00 con 0x7f312006f5d0 2023-09-30T22:21:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.701+0000 7f311f7fe700 1 -- 172.21.15.89:0/4051997149 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400fd90 con 0x7f312006f5d0 2023-09-30T22:21:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.701+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869c074ea0 con 0x7f869c134610 2023-09-30T22:21:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.701+0000 7f8691ffb700 1 -- 172.21.15.89:0/1920183146 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f868c021030 con 0x7f869c134610 2023-09-30T22:21:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.701+0000 7f8691ffb700 1 -- 172.21.15.89:0/1920183146 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f868c0411f0 con 0x7f869c134610 2023-09-30T22:21:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.702+0000 7f3126e6d700 1 -- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f312006f5d0 msgr2=0x7f312012cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.702+0000 7f3126e6d700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f312006f5d0 0x7f312012cd90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3114009840 tx=0x7f3114009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.708+0000 7f3126e6d700 1 -- 172.21.15.89:0/4051997149 shutdown_connections 2023-09-30T22:21:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.708+0000 7f3126e6d700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3120066880 0x7f312012fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.708+0000 7f3126e6d700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f312006f5d0 0x7f312012cd90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.708+0000 7f3126e6d700 1 --2- 172.21.15.89:0/4051997149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f312012f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.708+0000 7f3126e6d700 1 -- 172.21.15.89:0/4051997149 >> 172.21.15.89:0/4051997149 conn(0x7f31200a2aa0 msgr2=0x7f31200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.709+0000 7f3126e6d700 1 -- 172.21.15.89:0/4051997149 shutdown_connections 2023-09-30T22:21:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.709+0000 7f3126e6d700 1 -- 172.21.15.89:0/4051997149 wait complete. 2023-09-30T22:21:21.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.709+0000 7f8691ffb700 1 -- 172.21.15.89:0/1920183146 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f868c0239c0 con 0x7f869c134610 2023-09-30T22:21:21.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.710+0000 7f3126e6d700 1 Processor -- start 2023-09-30T22:21:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.710+0000 7f3126e6d700 1 -- start start 2023-09-30T22:21:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3126e6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f31201c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3126e6d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3120066880 0x7f31201c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3126e6d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f312006f5d0 0x7f31201c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3126e6d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312009cc80 con 0x7f3120066880 2023-09-30T22:21:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3126e6d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312009e4e0 con 0x7f312006f5d0 2023-09-30T22:21:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3126e6d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3120129fa0 con 0x7f3120066410 2023-09-30T22:21:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3124c09700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f31201c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3124c09700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f31201c3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59632/0 (socket says 172.21.15.89:59632) 2023-09-30T22:21:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3124c09700 1 -- 172.21.15.89:0/4229299880 learned_addr learned my addr 172.21.15.89:0/4229299880 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f311ffff700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3120066880 0x7f31201c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f312540a700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f312006f5d0 0x7f31201c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f311ffff700 1 -- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 msgr2=0x7f31201c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f311ffff700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f31201c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f311ffff700 1 -- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f312006f5d0 msgr2=0x7f31201c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f311ffff700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f312006f5d0 0x7f31201c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f311ffff700 1 -- 172.21.15.89:0/4229299880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c0094a0 con 0x7f3120066880 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f3124c09700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f31201c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.711+0000 7f312540a700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f312006f5d0 0x7f31201c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.712+0000 7f311ffff700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3120066880 0x7f31201c4380 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e9b0 tx=0x7f310c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.712+0000 7f311d7fa700 1 -- 172.21.15.89:0/4229299880 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c00ee50 con 0x7f3120066880 2023-09-30T22:21:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.712+0000 7f311d7fa700 1 -- 172.21.15.89:0/4229299880 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c00c850 con 0x7f3120066880 2023-09-30T22:21:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.712+0000 7f311d7fa700 1 -- 172.21.15.89:0/4229299880 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c00cc00 con 0x7f3120066880 2023-09-30T22:21:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.712+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31140094a0 con 0x7f3120066880 2023-09-30T22:21:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.712+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3120129940 con 0x7f3120066880 2023-09-30T22:21:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.713+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3120074ea0 con 0x7f3120066880 2023-09-30T22:21:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.714+0000 7f311d7fa700 1 -- 172.21.15.89:0/4229299880 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f310c013470 con 0x7f3120066880 2023-09-30T22:21:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.714+0000 7f311d7fa700 1 -- 172.21.15.89:0/4229299880 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f310c03ff50 con 0x7f3120066880 2023-09-30T22:21:21.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.718+0000 7f311d7fa700 1 -- 172.21.15.89:0/4229299880 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f310c013720 con 0x7f3120066880 2023-09-30T22:21:21.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.741+0000 7ff34d78f700 1 Processor -- start 2023-09-30T22:21:21.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.741+0000 7ff34d78f700 1 -- start start 2023-09-30T22:21:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.741+0000 7ff34d78f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff34809d6f0 0x7ff34809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.741+0000 7ff34d78f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.741+0000 7ff34d78f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809e410 0x7ff348130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.741+0000 7ff34d78f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34809f520 con 0x7ff34809d6f0 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.741+0000 7ff34d78f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34809f130 con 0x7ff34809e000 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.741+0000 7ff34d78f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff34809e930 con 0x7ff34809e410 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3477fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809e410 0x7ff348130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33040/0 (socket says 172.21.15.89:33040) 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 -- 172.21.15.89:0/2804351864 learned_addr learned my addr 172.21.15.89:0/2804351864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff346ffd700 1 --2- 172.21.15.89:0/2804351864 >> v2:172.21.15.89:3300/0 conn(0x7ff34809d6f0 0x7ff34809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 -- 172.21.15.89:0/2804351864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809e410 msgr2=0x7ff348130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 --2- 172.21.15.89:0/2804351864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809e410 0x7ff348130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 -- 172.21.15.89:0/2804351864 >> v2:172.21.15.89:3300/0 conn(0x7ff34809d6f0 msgr2=0x7ff34809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 --2- 172.21.15.89:0/2804351864 >> v2:172.21.15.89:3300/0 conn(0x7ff34809d6f0 0x7ff34809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 -- 172.21.15.89:0/2804351864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34809b2d0 con 0x7ff34809e000 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3477fe700 1 --2- 172.21.15.89:0/2804351864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809e410 0x7ff348130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff3467fc700 1 --2- 172.21.15.89:0/2804351864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348130660 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0098f0 tx=0x7ff33c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b764b7dd922d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff345ffb700 1 -- 172.21.15.89:0/2804351864 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c011830 con 0x7ff34809e000 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff345ffb700 1 -- 172.21.15.89:0/2804351864 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c011990 con 0x7ff34809e000 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.742+0000 7ff345ffb700 1 -- 172.21.15.89:0/2804351864 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c011d20 con 0x7ff34809e000 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 -- 172.21.15.89:0/2804351864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 msgr2=0x7ff348130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 --2- 172.21.15.89:0/2804351864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348130660 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0098f0 tx=0x7ff33c009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 -- 172.21.15.89:0/2804351864 shutdown_connections 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 --2- 172.21.15.89:0/2804351864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809e410 0x7ff348130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 --2- 172.21.15.89:0/2804351864 >> v2:172.21.15.89:3300/0 conn(0x7ff34809d6f0 0x7ff34809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 --2- 172.21.15.89:0/2804351864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348130660 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 -- 172.21.15.89:0/2804351864 >> 172.21.15.89:0/2804351864 conn(0x7ff348066f50 msgr2=0x7ff34809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 -- 172.21.15.89:0/2804351864 shutdown_connections 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.743+0000 7ff34d78f700 1 -- 172.21.15.89:0/2804351864 wait complete. 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff34d78f700 1 Processor -- start 2023-09-30T22:21:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff34d78f700 1 -- start start 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff34d78f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809d6f0 0x7ff348136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff34d78f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348136ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff34d78f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff34809e410 0x7ff3481373e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff34d78f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348130f70 con 0x7ff34809e410 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff34d78f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff348119eb0 con 0x7ff34809e000 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff34d78f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff34809c700 con 0x7ff34809d6f0 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff3467fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348136ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff3467fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348136ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33054/0 (socket says 172.21.15.89:33054) 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff3467fc700 1 -- 172.21.15.89:0/980808935 learned_addr learned my addr 172.21.15.89:0/980808935 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:21.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.745+0000 7ff346ffd700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809d6f0 0x7ff348136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff3477fe700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff34809e410 0x7ff3481373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff3467fc700 1 -- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809d6f0 msgr2=0x7ff348136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff3467fc700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809d6f0 0x7ff348136960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff3467fc700 1 -- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff34809e410 msgr2=0x7ff3481373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff3467fc700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff34809e410 0x7ff3481373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff3467fc700 1 -- 172.21.15.89:0/980808935 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff340009a70 con 0x7ff34809e000 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff3467fc700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348136ea0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009f50 tx=0x7ff33c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff32f7fe700 1 -- 172.21.15.89:0/980808935 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c004240 con 0x7ff34809e000 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c009550 con 0x7ff34809e000 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff32f7fe700 1 -- 172.21.15.89:0/980808935 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c01f070 con 0x7ff34809e000 2023-09-30T22:21:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff32f7fe700 1 -- 172.21.15.89:0/980808935 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0045c0 con 0x7ff34809e000 2023-09-30T22:21:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.746+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff348138630 con 0x7ff34809e000 2023-09-30T22:21:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.747+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff348074ea0 con 0x7ff34809e000 2023-09-30T22:21:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.747+0000 7ff32f7fe700 1 -- 172.21.15.89:0/980808935 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff33c021030 con 0x7ff34809e000 2023-09-30T22:21:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.748+0000 7ff32f7fe700 1 -- 172.21.15.89:0/980808935 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff33c040700 con 0x7ff34809e000 2023-09-30T22:21:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.756+0000 7ff32f7fe700 1 -- 172.21.15.89:0/980808935 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff33c0197c0 con 0x7ff34809e000 2023-09-30T22:21:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.882+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbc44074ea0 con 0x7fbc4406e9d0 2023-09-30T22:21:21.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.883+0000 7fbc2b7fe700 1 -- 172.21.15.89:0/2777339303 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc34017020 con 0x7fbc4406e9d0 2023-09-30T22:21:21.884 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 msgr2=0x7fbc441197a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc441197a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e930 tx=0x7fbc3400ec40 comp rx=0 tx=0).stop 2023-09-30T22:21:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 shutdown_connections 2023-09-30T22:21:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc44131f40 0x7fbc4411a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbc4406e9d0 0x7fbc441197a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 --2- 172.21.15.89:0/2777339303 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbc44131630 0x7fbc44119ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 >> 172.21.15.89:0/2777339303 conn(0x7fbc440a2aa0 msgr2=0x7fbc4412e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 shutdown_connections 2023-09-30T22:21:21.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.886+0000 7fbc497d6700 1 -- 172.21.15.89:0/2777339303 wait complete. 2023-09-30T22:21:21.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:21.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.914+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f31200713a0 con 0x7f3120066880 2023-09-30T22:21:21.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.914+0000 7f311d7fa700 1 -- 172.21.15.89:0/4229299880 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f310c01ab10 con 0x7f3120066880 2023-09-30T22:21:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:21.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.915+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f869c071100 con 0x7f869c134610 2023-09-30T22:21:21.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.915+0000 7f8691ffb700 1 -- 172.21.15.89:0/1920183146 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f868c015020 con 0x7f869c134610 2023-09-30T22:21:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:21.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3120066880 msgr2=0x7f31201c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3120066880 0x7f31201c4380 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e9b0 tx=0x7f310c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 shutdown_connections 2023-09-30T22:21:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f312006f5d0 0x7f31201c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3120066410 0x7f31201c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 --2- 172.21.15.89:0/4229299880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3120066880 0x7f31201c4380 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 >> 172.21.15.89:0/4229299880 conn(0x7f31200a2aa0 msgr2=0x7f312006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 shutdown_connections 2023-09-30T22:21:21.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.917+0000 7f3126e6d700 1 -- 172.21.15.89:0/4229299880 wait complete. 2023-09-30T22:21:21.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c134610 msgr2=0x7f869c13ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c134610 0x7f869c13ef90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f868c009c40 tx=0x7f868c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 shutdown_connections 2023-09-30T22:21:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f869c13e370 0x7f869c13f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f869c06e570 0x7f869c13ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 --2- 172.21.15.89:0/1920183146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f869c134610 0x7f869c13ef90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 >> 172.21.15.89:0/1920183146 conn(0x7f869c0a2aa0 msgr2=0x7f869c137240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 shutdown_connections 2023-09-30T22:21:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.918+0000 7f86a39cb700 1 -- 172.21.15.89:0/1920183146 wait complete. 2023-09-30T22:21:21.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:21.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.953+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff348071db0 con 0x7ff34809e000 2023-09-30T22:21:21.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.953+0000 7ff32f7fe700 1 -- 172.21.15.89:0/980808935 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff33c015070 con 0x7ff34809e000 2023-09-30T22:21:21.954 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:21.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 msgr2=0x7ff348136ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348136ea0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009f50 tx=0x7ff33c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:21.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 shutdown_connections 2023-09-30T22:21:21.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff34809e410 0x7ff3481373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff34809d6f0 0x7ff348136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 --2- 172.21.15.89:0/980808935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff34809e000 0x7ff348136ea0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:21.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 >> 172.21.15.89:0/980808935 conn(0x7ff348066f50 msgr2=0x7ff348070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 shutdown_connections 2023-09-30T22:21:21.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:21.955+0000 7ff34d78f700 1 -- 172.21.15.89:0/980808935 wait complete. 2023-09-30T22:21:21.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:22.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:22.932 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:22.932 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:22.969 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:23.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92ada4d700 1 Processor -- start 2023-09-30T22:21:23.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92ada4d700 1 -- start start 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92ada4d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92a809d6f0 0x7f92a809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92ada4d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809e440 0x7f92a8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92ada4d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a809b120 con 0x7f92a809e000 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92ada4d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a806b460 con 0x7f92a809d6f0 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92ada4d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92a809a7e0 con 0x7f92a809e440 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92a77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809e440 0x7f92a8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92a77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809e440 0x7f92a8134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59658/0 (socket says 172.21.15.89:59658) 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.046+0000 7f92a77fe700 1 -- 172.21.15.89:0/4162349741 learned_addr learned my addr 172.21.15.89:0/4162349741 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:23.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a6ffd700 1 --2- 172.21.15.89:0/4162349741 >> v2:172.21.15.89:3300/0 conn(0x7f92a809d6f0 0x7f92a809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a77fe700 1 -- 172.21.15.89:0/4162349741 >> v2:172.21.15.89:3300/0 conn(0x7f92a809d6f0 msgr2=0x7f92a809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:23.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a77fe700 1 --2- 172.21.15.89:0/4162349741 >> v2:172.21.15.89:3300/0 conn(0x7f92a809d6f0 0x7f92a809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a77fe700 1 -- 172.21.15.89:0/4162349741 >> v1:172.21.15.89:6789/0 conn(0x7f92a809e000 legacy=0x7f92a81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a77fe700 1 -- 172.21.15.89:0/4162349741 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a809a500 con 0x7f92a809e440 2023-09-30T22:21:23.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a77fe700 1 --2- 172.21.15.89:0/4162349741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809e440 0x7f92a8134100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009e80 tx=0x7f92a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152f105434f64d4f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a5ffb700 1 -- 172.21.15.89:0/4162349741 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a000e070 con 0x7f92a809e440 2023-09-30T22:21:23.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a5ffb700 1 -- 172.21.15.89:0/4162349741 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a000b8d0 con 0x7f92a809e440 2023-09-30T22:21:23.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.047+0000 7f92a5ffb700 1 -- 172.21.15.89:0/4162349741 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a000bc80 con 0x7f92a809e440 2023-09-30T22:21:23.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.048+0000 7f92ada4d700 1 -- 172.21.15.89:0/4162349741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809e440 msgr2=0x7f92a8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.048+0000 7f92ada4d700 1 --2- 172.21.15.89:0/4162349741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809e440 0x7f92a8134100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009e80 tx=0x7f92a000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:23.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.048+0000 7f92ada4d700 1 -- 172.21.15.89:0/4162349741 shutdown_connections 2023-09-30T22:21:23.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.048+0000 7f92ada4d700 1 --2- 172.21.15.89:0/4162349741 >> v2:172.21.15.89:3300/0 conn(0x7f92a809d6f0 0x7f92a809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.048+0000 7f92ada4d700 1 --2- 172.21.15.89:0/4162349741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809e440 0x7f92a8134100 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.048+0000 7f92ada4d700 1 -- 172.21.15.89:0/4162349741 >> 172.21.15.89:0/4162349741 conn(0x7f92a8066f50 msgr2=0x7f92a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.048+0000 7f92ada4d700 1 -- 172.21.15.89:0/4162349741 shutdown_connections 2023-09-30T22:21:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.048+0000 7f92ada4d700 1 -- 172.21.15.89:0/4162349741 wait complete. 2023-09-30T22:21:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.049+0000 7f92ada4d700 1 Processor -- start 2023-09-30T22:21:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92ada4d700 1 -- start start 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92ada4d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809d6f0 0x7f92a81a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92ada4d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a809e000 0x7f92a81a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92ada4d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92a809e440 0x7f92a81a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92ada4d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81a8d50 con 0x7f92a809e000 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92ada4d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81a7330 con 0x7f92a809e440 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92ada4d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81a97a0 con 0x7f92a809d6f0 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92a6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809d6f0 0x7f92a81a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92a6ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809d6f0 0x7f92a81a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59666/0 (socket says 172.21.15.89:59666) 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92a6ffd700 1 -- 172.21.15.89:0/2344784854 learned_addr learned my addr 172.21.15.89:0/2344784854 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.050+0000 7f92a77fe700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92a809e440 0x7f92a81a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a67fc700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a809e000 0x7f92a81a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a6ffd700 1 -- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92a809e440 msgr2=0x7f92a81a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a6ffd700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92a809e440 0x7f92a81a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a6ffd700 1 -- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a809e000 msgr2=0x7f92a81a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a6ffd700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a809e000 0x7f92a81a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a6ffd700 1 -- 172.21.15.89:0/2344784854 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a0009b30 con 0x7f92a809d6f0 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a67fc700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a809e000 0x7f92a81a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a77fe700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92a809e440 0x7f92a81a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92a6ffd700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809d6f0 0x7f92a81a5e80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f929800e9e0 tx=0x7f929800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f928ffff700 1 -- 172.21.15.89:0/2344784854 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298009710 con 0x7f92a809d6f0 2023-09-30T22:21:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f928ffff700 1 -- 172.21.15.89:0/2344784854 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929800ee80 con 0x7f92a809d6f0 2023-09-30T22:21:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a81a8890 con 0x7f92a809d6f0 2023-09-30T22:21:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f928ffff700 1 -- 172.21.15.89:0/2344784854 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929800ca50 con 0x7f92a809d6f0 2023-09-30T22:21:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.051+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a809c710 con 0x7f92a809d6f0 2023-09-30T22:21:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.052+0000 7f928ffff700 1 -- 172.21.15.89:0/2344784854 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f929800cbb0 con 0x7f92a809d6f0 2023-09-30T22:21:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.053+0000 7f928ffff700 1 -- 172.21.15.89:0/2344784854 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f929803f6b0 con 0x7f92a809d6f0 2023-09-30T22:21:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.053+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a8074ea0 con 0x7f92a809d6f0 2023-09-30T22:21:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.057+0000 7f928ffff700 1 -- 172.21.15.89:0/2344784854 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9298021070 con 0x7f92a809d6f0 2023-09-30T22:21:23.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.083+0000 7fec803a0700 1 Processor -- start 2023-09-30T22:21:23.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.083+0000 7fec803a0700 1 -- start start 2023-09-30T22:21:23.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec803a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809d6f0 0x7fec7809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec803a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e410 0x7fec78130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec803a0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7809f130 con 0x7fec7809e000 2023-09-30T22:21:23.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec803a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7809f520 con 0x7fec7809e410 2023-09-30T22:21:23.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec803a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec7809e930 con 0x7fec7809d6f0 2023-09-30T22:21:23.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec7e93d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e410 0x7fec78130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec7e93d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e410 0x7fec78130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33068/0 (socket says 172.21.15.89:33068) 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec7e93d700 1 -- 172.21.15.89:0/177640053 learned_addr learned my addr 172.21.15.89:0/177640053 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec7d93b700 1 --1- 172.21.15.89:0/177640053 >> v1:172.21.15.89:6789/0 conn(0x7fec7809e000 0x7fec78130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45786/0 (socket says 172.21.15.89:45786) 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec7e13c700 1 --2- 172.21.15.89:0/177640053 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809d6f0 0x7fec7809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec7e93d700 1 -- 172.21.15.89:0/177640053 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809d6f0 msgr2=0x7fec7809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec7e93d700 1 --2- 172.21.15.89:0/177640053 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809d6f0 0x7fec7809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.084+0000 7fec7d13a700 1 -- 172.21.15.89:0/177640053 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977479620 0 0) 0x7fec7809f130 con 0x7fec7809e000 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.085+0000 7fec7e93d700 1 -- 172.21.15.89:0/177640053 >> v1:172.21.15.89:6789/0 conn(0x7fec7809e000 legacy=0x7fec78130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.085+0000 7fec7e93d700 1 -- 172.21.15.89:0/177640053 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7809b2d0 con 0x7fec7809e410 2023-09-30T22:21:23.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.085+0000 7fec7e93d700 1 --2- 172.21.15.89:0/177640053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e410 0x7fec78130d70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fec70009e80 tx=0x7fec7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a27e44a79854cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.085+0000 7fec7d13a700 1 -- 172.21.15.89:0/177640053 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000e070 con 0x7fec7809e410 2023-09-30T22:21:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.085+0000 7fec7d13a700 1 -- 172.21.15.89:0/177640053 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7000b910 con 0x7fec7809e410 2023-09-30T22:21:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.085+0000 7fec7d13a700 1 -- 172.21.15.89:0/177640053 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000bcc0 con 0x7fec7809e410 2023-09-30T22:21:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.085+0000 7ff057bf1700 1 Processor -- start 2023-09-30T22:21:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.085+0000 7ff057bf1700 1 -- start start 2023-09-30T22:21:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7fec803a0700 1 -- 172.21.15.89:0/177640053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e410 msgr2=0x7fec78130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7fec803a0700 1 --2- 172.21.15.89:0/177640053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e410 0x7fec78130d70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fec70009e80 tx=0x7fec7000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7fec803a0700 1 -- 172.21.15.89:0/177640053 shutdown_connections 2023-09-30T22:21:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7fec803a0700 1 --2- 172.21.15.89:0/177640053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e410 0x7fec78130d70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7fec803a0700 1 --2- 172.21.15.89:0/177640053 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809d6f0 0x7fec7809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7fec803a0700 1 -- 172.21.15.89:0/177640053 >> 172.21.15.89:0/177640053 conn(0x7fec78066f50 msgr2=0x7fec7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff057bf1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff05009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff057bf1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff05009e000 0x7ff0501339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff057bf1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05009b120 con 0x7ff05009e410 2023-09-30T22:21:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff057bf1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05006b460 con 0x7ff05009e000 2023-09-30T22:21:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff057bf1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff05009a7e0 con 0x7ff05009d6f0 2023-09-30T22:21:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05598d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff05009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05598d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff05009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59672/0 (socket says 172.21.15.89:59672) 2023-09-30T22:21:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05598d700 1 -- 172.21.15.89:0/899712184 learned_addr learned my addr 172.21.15.89:0/899712184 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05518c700 1 --2- 172.21.15.89:0/899712184 >> v2:172.21.15.89:3300/0 conn(0x7ff05009e000 0x7ff0501339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7fec803a0700 1 -- 172.21.15.89:0/177640053 shutdown_connections 2023-09-30T22:21:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05598d700 1 -- 172.21.15.89:0/899712184 >> v2:172.21.15.89:3300/0 conn(0x7ff05009e000 msgr2=0x7ff0501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05598d700 1 --2- 172.21.15.89:0/899712184 >> v2:172.21.15.89:3300/0 conn(0x7ff05009e000 0x7ff0501339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05598d700 1 -- 172.21.15.89:0/899712184 >> v1:172.21.15.89:6789/0 conn(0x7ff05009e410 legacy=0x7ff050133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05598d700 1 -- 172.21.15.89:0/899712184 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05009a500 con 0x7ff05009d6f0 2023-09-30T22:21:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7fec803a0700 1 -- 172.21.15.89:0/177640053 wait complete. 2023-09-30T22:21:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.086+0000 7ff05598d700 1 --2- 172.21.15.89:0/899712184 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff05009dac0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff048009920 tx=0x7ff048009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe1fb82ab06d143c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff05498b700 1 -- 172.21.15.89:0/899712184 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048004230 con 0x7ff05009d6f0 2023-09-30T22:21:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff05498b700 1 -- 172.21.15.89:0/899712184 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff048004390 con 0x7ff05009d6f0 2023-09-30T22:21:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff05498b700 1 -- 172.21.15.89:0/899712184 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0480036a0 con 0x7ff05009d6f0 2023-09-30T22:21:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff057bf1700 1 -- 172.21.15.89:0/899712184 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 msgr2=0x7ff05009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff057bf1700 1 --2- 172.21.15.89:0/899712184 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff05009dac0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff048009920 tx=0x7ff048009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:23.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff057bf1700 1 -- 172.21.15.89:0/899712184 shutdown_connections 2023-09-30T22:21:23.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff057bf1700 1 --2- 172.21.15.89:0/899712184 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff05009dac0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff057bf1700 1 --2- 172.21.15.89:0/899712184 >> v2:172.21.15.89:3300/0 conn(0x7ff05009e000 0x7ff0501339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.087+0000 7ff057bf1700 1 -- 172.21.15.89:0/899712184 >> 172.21.15.89:0/899712184 conn(0x7ff050066f50 msgr2=0x7ff05009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.088+0000 7ff057bf1700 1 -- 172.21.15.89:0/899712184 shutdown_connections 2023-09-30T22:21:23.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.088+0000 7ff057bf1700 1 -- 172.21.15.89:0/899712184 wait complete. 2023-09-30T22:21:23.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.088+0000 7fec803a0700 1 Processor -- start 2023-09-30T22:21:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.088+0000 7ff057bf1700 1 Processor -- start 2023-09-30T22:21:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.088+0000 7ff057bf1700 1 -- start start 2023-09-30T22:21:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec803a0700 1 -- start start 2023-09-30T22:21:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec803a0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7809d6f0 0x7fec781c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec803a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e000 0x7fec781c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec803a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809e410 0x7fec781c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec803a0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec781c8f80 con 0x7fec7809d6f0 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec803a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec78130bc0 con 0x7fec7809e000 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec803a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec780a3860 con 0x7fec7809e410 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec7e93d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809e410 0x7fec781c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec7e93d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809e410 0x7fec781c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59694/0 (socket says 172.21.15.89:59694) 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7fec7e93d700 1 -- 172.21.15.89:0/3610692849 learned_addr learned my addr 172.21.15.89:0/3610692849 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff057bf1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff0501a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff057bf1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05009e000 0x7ff0501a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff057bf1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff05009e410 0x7ff0501a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff057bf1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501a8050 con 0x7ff05009e000 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff057bf1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501a9110 con 0x7ff05009e410 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff057bf1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff05009c710 con 0x7ff05009d6f0 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff0501a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff0501a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59680/0 (socket says 172.21.15.89:59680) 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 -- 172.21.15.89:0/1828040801 learned_addr learned my addr 172.21.15.89:0/1828040801 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05518c700 1 --2- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05009e000 0x7ff0501a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 -- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff05009e410 msgr2=0x7ff0501a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 --2- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff05009e410 0x7ff0501a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 -- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05009e000 msgr2=0x7ff0501a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 --2- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05009e000 0x7ff0501a6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 -- 172.21.15.89:0/1828040801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048009550 con 0x7ff05009d6f0 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.089+0000 7ff05598d700 1 --2- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff0501a5f00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff048009fd0 tx=0x7ff048004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7fec7e93d700 1 -- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e000 msgr2=0x7fec781c8a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7fec7e93d700 1 --2- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e000 0x7fec781c8a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7fec7e93d700 1 -- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7809d6f0 msgr2=0x7fec781c8500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7fec7e93d700 1 --2- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7809d6f0 0x7fec781c8500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7fec7e93d700 1 -- 172.21.15.89:0/3610692849 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70009b30 con 0x7fec7809e410 2023-09-30T22:21:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7ff0467fc700 1 -- 172.21.15.89:0/1828040801 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048003c50 con 0x7ff05009d6f0 2023-09-30T22:21:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7ff0467fc700 1 -- 172.21.15.89:0/1828040801 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04801f070 con 0x7ff05009d6f0 2023-09-30T22:21:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7ff0467fc700 1 -- 172.21.15.89:0/1828040801 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048011630 con 0x7ff05009d6f0 2023-09-30T22:21:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0500a32c0 con 0x7ff05009d6f0 2023-09-30T22:21:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.090+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0501a8cc0 con 0x7ff05009d6f0 2023-09-30T22:21:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.091+0000 7fec7e93d700 1 --2- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809e410 0x7fec781c4c70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fec7809d470 tx=0x7fec70004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.091+0000 7fec6effd700 1 -- 172.21.15.89:0/3610692849 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700092e0 con 0x7fec7809e410 2023-09-30T22:21:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.091+0000 7fec6effd700 1 -- 172.21.15.89:0/3610692849 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7001c410 con 0x7fec7809e410 2023-09-30T22:21:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.091+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7809c700 con 0x7fec7809e410 2023-09-30T22:21:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.091+0000 7fec6effd700 1 -- 172.21.15.89:0/3610692849 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7001c7a0 con 0x7fec7809e410 2023-09-30T22:21:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.092+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0501a9340 con 0x7ff05009d6f0 2023-09-30T22:21:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.092+0000 7ff0467fc700 1 -- 172.21.15.89:0/1828040801 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff048021030 con 0x7ff05009d6f0 2023-09-30T22:21:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.092+0000 7ff0467fc700 1 -- 172.21.15.89:0/1828040801 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff048040cd0 con 0x7ff05009d6f0 2023-09-30T22:21:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.091+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec78132640 con 0x7fec7809e410 2023-09-30T22:21:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.092+0000 7fec6effd700 1 -- 172.21.15.89:0/3610692849 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fec7001c900 con 0x7fec7809e410 2023-09-30T22:21:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.092+0000 7fec6effd700 1 -- 172.21.15.89:0/3610692849 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fec70041570 con 0x7fec7809e410 2023-09-30T22:21:23.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.092+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec78131290 con 0x7fec7809e410 2023-09-30T22:21:23.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.096+0000 7ff0467fc700 1 -- 172.21.15.89:0/1828040801 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0501a9340 con 0x7ff05009d6f0 2023-09-30T22:21:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.105+0000 7fec6effd700 1 -- 172.21.15.89:0/3610692849 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec78131290 con 0x7fec7809e410 2023-09-30T22:21:23.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf46d66700 1 Processor -- start 2023-09-30T22:21:23.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf46d66700 1 -- start start 2023-09-30T22:21:23.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf46d66700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf46d66700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf4009e410 0x7fcf40130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf46d66700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4009f130 con 0x7fcf4009e000 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf46d66700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4009f520 con 0x7fcf4009e410 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf46d66700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4009e930 con 0x7fcf4009d6f0 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf45303700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf4009e410 0x7fcf40130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf45303700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf4009e410 0x7fcf40130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49572/0 (socket says 172.21.15.89:49572) 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf45303700 1 -- 172.21.15.89:0/2588136541 learned_addr learned my addr 172.21.15.89:0/2588136541 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.130+0000 7fcf44b02700 1 --2- 172.21.15.89:0/2588136541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.131+0000 7fcf44b02700 1 -- 172.21.15.89:0/2588136541 >> v2:172.21.15.89:3300/0 conn(0x7fcf4009e410 msgr2=0x7fcf40130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.131+0000 7fcf44b02700 1 --2- 172.21.15.89:0/2588136541 >> v2:172.21.15.89:3300/0 conn(0x7fcf4009e410 0x7fcf40130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.131+0000 7fcf44b02700 1 -- 172.21.15.89:0/2588136541 >> v1:172.21.15.89:6789/0 conn(0x7fcf4009e000 legacy=0x7fcf40130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.131+0000 7fcf44b02700 1 -- 172.21.15.89:0/2588136541 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4009b2d0 con 0x7fcf4009d6f0 2023-09-30T22:21:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.131+0000 7fcf44b02700 1 --2- 172.21.15.89:0/2588136541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4009dac0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009840 tx=0x7fcf34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4964d72c51430899 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.131+0000 7fcf3f7fe700 1 -- 172.21.15.89:0/2588136541 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011830 con 0x7fcf4009d6f0 2023-09-30T22:21:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.131+0000 7fcf3f7fe700 1 -- 172.21.15.89:0/2588136541 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf34011990 con 0x7fcf4009d6f0 2023-09-30T22:21:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.131+0000 7fcf3f7fe700 1 -- 172.21.15.89:0/2588136541 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011d20 con 0x7fcf4009d6f0 2023-09-30T22:21:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.132+0000 7fcf46d66700 1 -- 172.21.15.89:0/2588136541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 msgr2=0x7fcf4009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.132+0000 7fcf46d66700 1 --2- 172.21.15.89:0/2588136541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4009dac0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009840 tx=0x7fcf34009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.132+0000 7fcf46d66700 1 -- 172.21.15.89:0/2588136541 shutdown_connections 2023-09-30T22:21:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.132+0000 7fcf46d66700 1 --2- 172.21.15.89:0/2588136541 >> v2:172.21.15.89:3300/0 conn(0x7fcf4009e410 0x7fcf40130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.132+0000 7fcf46d66700 1 --2- 172.21.15.89:0/2588136541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4009dac0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.132+0000 7fcf46d66700 1 -- 172.21.15.89:0/2588136541 >> 172.21.15.89:0/2588136541 conn(0x7fcf40066f50 msgr2=0x7fcf4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.132+0000 7fcf46d66700 1 -- 172.21.15.89:0/2588136541 shutdown_connections 2023-09-30T22:21:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.132+0000 7fcf46d66700 1 -- 172.21.15.89:0/2588136541 wait complete. 2023-09-30T22:21:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.133+0000 7fcf46d66700 1 Processor -- start 2023-09-30T22:21:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.133+0000 7fcf46d66700 1 -- start start 2023-09-30T22:21:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf46d66700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4011f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf46d66700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4009e000 0x7fcf4011f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf46d66700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4009e410 0x7fcf4011fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf46d66700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4006ea60 con 0x7fcf4009e000 2023-09-30T22:21:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf46d66700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4006b340 con 0x7fcf4009d6f0 2023-09-30T22:21:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf46d66700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40125830 con 0x7fcf4009e410 2023-09-30T22:21:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf3ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4009e000 0x7fcf4011f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf3ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4009e000 0x7fcf4011f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49580/0 (socket says 172.21.15.89:49580) 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf3ffff700 1 -- 172.21.15.89:0/1412628860 learned_addr learned my addr 172.21.15.89:0/1412628860 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.134+0000 7fcf44b02700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4011f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf45303700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4009e410 0x7fcf4011fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf44b02700 1 -- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4009e410 msgr2=0x7fcf4011fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf44b02700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4009e410 0x7fcf4011fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf44b02700 1 -- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4009e000 msgr2=0x7fcf4011f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf44b02700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4009e000 0x7fcf4011f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf44b02700 1 -- 172.21.15.89:0/1412628860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf340094a0 con 0x7fcf4009d6f0 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf3ffff700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4009e000 0x7fcf4011f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf45303700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4009e410 0x7fcf4011fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf44b02700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4011f240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcf34016000 tx=0x7fcf34004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf3d7fa700 1 -- 172.21.15.89:0/1412628860 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf340036a0 con 0x7fcf4009d6f0 2023-09-30T22:21:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf3d7fa700 1 -- 172.21.15.89:0/1412628860 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3401e070 con 0x7fcf4009d6f0 2023-09-30T22:21:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf401303c0 con 0x7fcf4009d6f0 2023-09-30T22:21:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf3d7fa700 1 -- 172.21.15.89:0/1412628860 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34003a40 con 0x7fcf4009d6f0 2023-09-30T22:21:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.135+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf40120740 con 0x7fcf4009d6f0 2023-09-30T22:21:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.136+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf40074ea0 con 0x7fcf4009d6f0 2023-09-30T22:21:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.137+0000 7fcf3d7fa700 1 -- 172.21.15.89:0/1412628860 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf34003ba0 con 0x7fcf4009d6f0 2023-09-30T22:21:23.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.137+0000 7fcf3d7fa700 1 -- 172.21.15.89:0/1412628860 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcf340406b0 con 0x7fcf4009d6f0 2023-09-30T22:21:23.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.141+0000 7fcf3d7fa700 1 -- 172.21.15.89:0/1412628860 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf3402cbc0 con 0x7fcf4009d6f0 2023-09-30T22:21:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.276+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f92a809f420 con 0x7f92a809d6f0 2023-09-30T22:21:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.276+0000 7f928ffff700 1 -- 172.21.15.89:0/2344784854 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92980232f0 con 0x7f92a809d6f0 2023-09-30T22:21:23.277 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809d6f0 msgr2=0x7f92a81a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809d6f0 0x7f92a81a5e80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f929800e9e0 tx=0x7f929800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 shutdown_connections 2023-09-30T22:21:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92a809e440 0x7f92a81a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f92a809d6f0 0x7f92a81a5e80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 --2- 172.21.15.89:0/2344784854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a809e000 0x7f92a81a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 >> 172.21.15.89:0/2344784854 conn(0x7f92a8066f50 msgr2=0x7f92a81bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 shutdown_connections 2023-09-30T22:21:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.279+0000 7f92ada4d700 1 -- 172.21.15.89:0/2344784854 wait complete. 2023-09-30T22:21:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.295+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff0501a9340 con 0x7ff05009d6f0 2023-09-30T22:21:23.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.295+0000 7ff0467fc700 1 -- 172.21.15.89:0/1828040801 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff048046020 con 0x7ff05009d6f0 2023-09-30T22:21:23.296 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:23.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 msgr2=0x7ff0501a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 --2- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff0501a5f00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff048009fd0 tx=0x7ff048004750 comp rx=0 tx=0).stop 2023-09-30T22:21:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 shutdown_connections 2023-09-30T22:21:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 --2- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff05009e410 0x7ff0501a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 --2- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff05009d6f0 0x7ff0501a5f00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 --2- 172.21.15.89:0/1828040801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05009e000 0x7ff0501a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 >> 172.21.15.89:0/1828040801 conn(0x7ff050066f50 msgr2=0x7ff0501bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 shutdown_connections 2023-09-30T22:21:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.297+0000 7ff057bf1700 1 -- 172.21.15.89:0/1828040801 wait complete. 2023-09-30T22:21:23.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:23.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.332+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fec78131290 con 0x7fec7809e410 2023-09-30T22:21:23.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.332+0000 7fec6effd700 1 -- 172.21.15.89:0/3610692849 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec78131290 con 0x7fec7809e410 2023-09-30T22:21:23.333 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809e410 msgr2=0x7fec781c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 --2- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809e410 0x7fec781c4c70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fec7809d470 tx=0x7fec70004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 shutdown_connections 2023-09-30T22:21:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 --2- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec7809e410 0x7fec781c4c70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 --2- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec7809d6f0 0x7fec781c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 --2- 172.21.15.89:0/3610692849 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec7809e000 0x7fec781c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 >> 172.21.15.89:0/3610692849 conn(0x7fec78066f50 msgr2=0x7fec78070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 shutdown_connections 2023-09-30T22:21:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.334+0000 7fec803a0700 1 -- 172.21.15.89:0/3610692849 wait complete. 2023-09-30T22:21:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.335+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcf40125a00 con 0x7fcf4009d6f0 2023-09-30T22:21:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.336+0000 7fcf3d7fa700 1 -- 172.21.15.89:0/1412628860 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf34032020 con 0x7fcf4009d6f0 2023-09-30T22:21:23.337 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 msgr2=0x7fcf4011f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4011f240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcf34016000 tx=0x7fcf34004750 comp rx=0 tx=0).stop 2023-09-30T22:21:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 shutdown_connections 2023-09-30T22:21:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf4009e410 0x7fcf4011fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf4009d6f0 0x7fcf4011f240 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 --2- 172.21.15.89:0/1412628860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf4009e000 0x7fcf4011f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 >> 172.21.15.89:0/1412628860 conn(0x7fcf40066f50 msgr2=0x7fcf400a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 shutdown_connections 2023-09-30T22:21:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:23.338+0000 7fcf46d66700 1 -- 172.21.15.89:0/1412628860 wait complete. 2023-09-30T22:21:23.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:23.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:24.298 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:24.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:24.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:24.353 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.452+0000 7f563fc84700 1 Processor -- start 2023-09-30T22:21:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.452+0000 7f563fc84700 1 -- start start 2023-09-30T22:21:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563fc84700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563fc84700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5638130e50 0x7f563806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563fc84700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563806fb20 0x7f563806ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563fc84700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56380704b0 con 0x7f5638130e50 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563fc84700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563809a390 con 0x7f5638130540 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563fc84700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56380716a0 con 0x7f563806fb20 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563da20700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5638130e50 0x7f563806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5638130e50 0x7f563806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49592/0 (socket says 172.21.15.89:49592) 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563da20700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33106/0 (socket says 172.21.15.89:33106) 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 -- 172.21.15.89:0/21486894 learned_addr learned my addr 172.21.15.89:0/21486894 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 -- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563806fb20 msgr2=0x7f563806ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 --2- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563806fb20 0x7f563806ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 -- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 msgr2=0x7f5638130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 --2- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 -- 172.21.15.89:0/21486894 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5638071290 con 0x7f5638130e50 2023-09-30T22:21:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563da20700 1 --2- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.453+0000 7f563d21f700 1 --2- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5638130e50 0x7f563806f5e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5628009920 tx=0x7f5628009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf18becee6d7a4b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.454+0000 7f563ca1e700 1 -- 172.21.15.89:0/21486894 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628011830 con 0x7f5638130e50 2023-09-30T22:21:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.454+0000 7f563ca1e700 1 -- 172.21.15.89:0/21486894 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5628011990 con 0x7f5638130e50 2023-09-30T22:21:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.454+0000 7f563ca1e700 1 -- 172.21.15.89:0/21486894 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628011d20 con 0x7f5638130e50 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.454+0000 7f563fc84700 1 -- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5638130e50 msgr2=0x7f563806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.454+0000 7f563fc84700 1 --2- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5638130e50 0x7f563806f5e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5628009920 tx=0x7f5628009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.455+0000 7f563fc84700 1 -- 172.21.15.89:0/21486894 shutdown_connections 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.455+0000 7f563fc84700 1 --2- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563806fb20 0x7f563806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.455+0000 7f563fc84700 1 --2- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.455+0000 7f563fc84700 1 --2- 172.21.15.89:0/21486894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5638130e50 0x7f563806f5e0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.455+0000 7f563fc84700 1 -- 172.21.15.89:0/21486894 >> 172.21.15.89:0/21486894 conn(0x7f56380a2aa0 msgr2=0x7f56380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.455+0000 7f563fc84700 1 -- 172.21.15.89:0/21486894 shutdown_connections 2023-09-30T22:21:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.455+0000 7f563fc84700 1 -- 172.21.15.89:0/21486894 wait complete. 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563fc84700 1 Processor -- start 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563fc84700 1 -- start start 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563fc84700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563806fb20 0x7f5638118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563fc84700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563fc84700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5638130e50 0x7f5638119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563fc84700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563811cb30 con 0x7f563806fb20 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563fc84700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563811a280 con 0x7f5638130540 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563fc84700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f563811b520 con 0x7f5638130e50 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563e221700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5638130e50 0x7f5638119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563e221700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5638130e50 0x7f5638119750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59716/0 (socket says 172.21.15.89:59716) 2023-09-30T22:21:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563e221700 1 -- 172.21.15.89:0/2550127953 learned_addr learned my addr 172.21.15.89:0/2550127953 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.456+0000 7f563da20700 1 --2- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563806fb20 0x7f5638118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f563e221700 1 -- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 msgr2=0x7f5638119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f563e221700 1 --2- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f563e221700 1 -- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563806fb20 msgr2=0x7f5638118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f563e221700 1 --2- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563806fb20 0x7f5638118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f563e221700 1 -- 172.21.15.89:0/2550127953 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340094a0 con 0x7f5638130e50 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f563e221700 1 --2- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5638130e50 0x7f5638119750 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f563000b9a0 tx=0x7f563000bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f562e7fc700 1 -- 172.21.15.89:0/2550127953 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630010d40 con 0x7f5638130e50 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f562e7fc700 1 -- 172.21.15.89:0/2550127953 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630018440 con 0x7f5638130e50 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5628009550 con 0x7f5638130e50 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f562e7fc700 1 -- 172.21.15.89:0/2550127953 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56300187f0 con 0x7f5638130e50 2023-09-30T22:21:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.457+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56380718e0 con 0x7f5638130e50 2023-09-30T22:21:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.458+0000 7f562e7fc700 1 -- 172.21.15.89:0/2550127953 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56300213b0 con 0x7f5638130e50 2023-09-30T22:21:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.458+0000 7f562e7fc700 1 -- 172.21.15.89:0/2550127953 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5630006e40 con 0x7f5638130e50 2023-09-30T22:21:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.458+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5638074ea0 con 0x7f5638130e50 2023-09-30T22:21:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.467+0000 7f562e7fc700 1 -- 172.21.15.89:0/2550127953 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f563001d080 con 0x7f5638130e50 2023-09-30T22:21:24.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.490+0000 7fbbdbbf8700 1 Processor -- start 2023-09-30T22:21:24.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.491+0000 7fbbdbbf8700 1 -- start start 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.491+0000 7fbbdbbf8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbd4131810 0x7fbbd4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.491+0000 7fbbdbbf8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4099a40 0x7fbbd4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.491+0000 7fbbdbbf8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.491+0000 7fbbdbbf8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd41313c0 con 0x7fbbd4131810 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.491+0000 7fbbdbbf8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd406e8c0 con 0x7fbbd409a3b0 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.491+0000 7fbbdbbf8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd4071d20 con 0x7fbbd4099a40 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbd9994700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbd4131810 0x7fbbd4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbd9994700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbd4131810 0x7fbbd4131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49620/0 (socket says 172.21.15.89:49620) 2023-09-30T22:21:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbda195700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbd9994700 1 -- 172.21.15.89:0/2956366415 learned_addr learned my addr 172.21.15.89:0/2956366415 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbd9193700 1 --2- 172.21.15.89:0/2956366415 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4099a40 0x7fbbd4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbda195700 1 -- 172.21.15.89:0/2956366415 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4099a40 msgr2=0x7fbbd4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbda195700 1 --2- 172.21.15.89:0/2956366415 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4099a40 0x7fbbd4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbda195700 1 -- 172.21.15.89:0/2956366415 >> v2:172.21.15.89:3300/0 conn(0x7fbbd4131810 msgr2=0x7fbbd4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbda195700 1 --2- 172.21.15.89:0/2956366415 >> v2:172.21.15.89:3300/0 conn(0x7fbbd4131810 0x7fbbd4131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbda195700 1 -- 172.21.15.89:0/2956366415 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd40721a0 con 0x7fbbd409a3b0 2023-09-30T22:21:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbda195700 1 --2- 172.21.15.89:0/2956366415 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4138650 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009d90 tx=0x7fbbcc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a65f6e58f03d13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.492+0000 7fbbd8992700 1 -- 172.21.15.89:0/2956366415 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc00bd20 con 0x7fbbd409a3b0 2023-09-30T22:21:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbd8992700 1 -- 172.21.15.89:0/2956366415 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc00be80 con 0x7fbbd409a3b0 2023-09-30T22:21:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbd8992700 1 -- 172.21.15.89:0/2956366415 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc010710 con 0x7fbbd409a3b0 2023-09-30T22:21:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/2956366415 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 msgr2=0x7fbbd4138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 --2- 172.21.15.89:0/2956366415 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4138650 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009d90 tx=0x7fbbcc00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/2956366415 shutdown_connections 2023-09-30T22:21:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 --2- 172.21.15.89:0/2956366415 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4138650 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 --2- 172.21.15.89:0/2956366415 >> v2:172.21.15.89:3300/0 conn(0x7fbbd4131810 0x7fbbd4131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 --2- 172.21.15.89:0/2956366415 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4099a40 0x7fbbd4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/2956366415 >> 172.21.15.89:0/2956366415 conn(0x7fbbd40a2aa0 msgr2=0x7fbbd40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/2956366415 shutdown_connections 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.493+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/2956366415 wait complete. 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.495+0000 7fbbdbbf8700 1 Processor -- start 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.495+0000 7fbbdbbf8700 1 -- start start 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.495+0000 7fbbdbbf8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd4099a40 0x7fbbd4118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.495+0000 7fbbdbbf8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.495+0000 7fbbdbbf8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4131810 0x7fbbd4119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.495+0000 7fbbdbbf8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd411a6e0 con 0x7fbbd4099a40 2023-09-30T22:21:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.495+0000 7fbbdbbf8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd406eb00 con 0x7fbbd409a3b0 2023-09-30T22:21:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.495+0000 7fbbdbbf8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd412b980 con 0x7fbbd4131810 2023-09-30T22:21:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9994700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd4099a40 0x7fbbd4118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9994700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd4099a40 0x7fbbd4118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49636/0 (socket says 172.21.15.89:49636) 2023-09-30T22:21:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9994700 1 -- 172.21.15.89:0/4196560585 learned_addr learned my addr 172.21.15.89:0/4196560585 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbda195700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4131810 0x7fbbd4119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9193700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9193700 1 -- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4131810 msgr2=0x7fbbd4119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9193700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4131810 0x7fbbd4119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9193700 1 -- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd4099a40 msgr2=0x7fbbd4118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9193700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd4099a40 0x7fbbd4118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9193700 1 -- 172.21.15.89:0/4196560585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc009a70 con 0x7fbbd409a3b0 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbda195700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4131810 0x7fbbd4119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbd9193700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4119210 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400e900 tx=0x7fbbc400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbca7fc700 1 -- 172.21.15.89:0/4196560585 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4013680 con 0x7fbbd409a3b0 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbca7fc700 1 -- 172.21.15.89:0/4196560585 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc40137e0 con 0x7fbbd409a3b0 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.496+0000 7fbbca7fc700 1 -- 172.21.15.89:0/4196560585 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4013b50 con 0x7fbbd409a3b0 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.497+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd412b4c0 con 0x7fbbd409a3b0 2023-09-30T22:21:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.497+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd411c4c0 con 0x7fbbd409a3b0 2023-09-30T22:21:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.498+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd4071600 con 0x7fbbd409a3b0 2023-09-30T22:21:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.498+0000 7fbbca7fc700 1 -- 172.21.15.89:0/4196560585 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbbc4013cb0 con 0x7fbbd409a3b0 2023-09-30T22:21:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.500+0000 7fbbca7fc700 1 -- 172.21.15.89:0/4196560585 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbbc4012e20 con 0x7fbbd409a3b0 2023-09-30T22:21:24.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.503+0000 7fbbca7fc700 1 -- 172.21.15.89:0/4196560585 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbbc4044060 con 0x7fbbd409a3b0 2023-09-30T22:21:24.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.521+0000 7fd8199c5700 1 Processor -- start 2023-09-30T22:21:24.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.521+0000 7fd8199c5700 1 -- start start 2023-09-30T22:21:24.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8199c5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd81406a770 0x7fd81409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8199c5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8199c5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81409a810 con 0x7fd81409dc30 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8199c5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81406ab40 con 0x7fd81406a770 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8199c5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81409ebc0 con 0x7fd81413d1f0 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8137fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8137fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33142/0 (socket says 172.21.15.89:33142) 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8137fe700 1 -- 172.21.15.89:0/128075146 learned_addr learned my addr 172.21.15.89:0/128075146 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd812ffd700 1 --2- 172.21.15.89:0/128075146 >> v2:172.21.15.89:3300/0 conn(0x7fd81406a770 0x7fd81409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8137fe700 1 -- 172.21.15.89:0/128075146 >> v2:172.21.15.89:3300/0 conn(0x7fd81406a770 msgr2=0x7fd81409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8137fe700 1 --2- 172.21.15.89:0/128075146 >> v2:172.21.15.89:3300/0 conn(0x7fd81406a770 0x7fd81409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8137fe700 1 -- 172.21.15.89:0/128075146 >> v1:172.21.15.89:6789/0 conn(0x7fd81409dc30 legacy=0x7fd81409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8137fe700 1 -- 172.21.15.89:0/128075146 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81409e920 con 0x7fd81413d1f0 2023-09-30T22:21:24.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.522+0000 7fd8137fe700 1 --2- 172.21.15.89:0/128075146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413f5d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009e50 tx=0x7fd80c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a5a1b13b9f9a19 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd811ffb700 1 -- 172.21.15.89:0/128075146 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00e070 con 0x7fd81413d1f0 2023-09-30T22:21:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd811ffb700 1 -- 172.21.15.89:0/128075146 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c00b9c0 con 0x7fd81413d1f0 2023-09-30T22:21:24.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd811ffb700 1 -- 172.21.15.89:0/128075146 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00bd70 con 0x7fd81413d1f0 2023-09-30T22:21:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd8199c5700 1 -- 172.21.15.89:0/128075146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 msgr2=0x7fd81413f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd8199c5700 1 --2- 172.21.15.89:0/128075146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413f5d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009e50 tx=0x7fd80c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd8199c5700 1 -- 172.21.15.89:0/128075146 shutdown_connections 2023-09-30T22:21:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd8199c5700 1 --2- 172.21.15.89:0/128075146 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413f5d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd8199c5700 1 --2- 172.21.15.89:0/128075146 >> v2:172.21.15.89:3300/0 conn(0x7fd81406a770 0x7fd81409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd8199c5700 1 -- 172.21.15.89:0/128075146 >> 172.21.15.89:0/128075146 conn(0x7fd814066f50 msgr2=0x7fd81409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd8199c5700 1 -- 172.21.15.89:0/128075146 shutdown_connections 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.523+0000 7fd8199c5700 1 -- 172.21.15.89:0/128075146 wait complete. 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.524+0000 7fd8199c5700 1 Processor -- start 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8199c5700 1 -- start start 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8199c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd81406a770 0x7fd81413a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8199c5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd81409dc30 0x7fd81413abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8199c5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8199c5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81413b750 con 0x7fd81409dc30 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8199c5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8140a3760 con 0x7fd81413d1f0 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8199c5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd81409f1a0 con 0x7fd81406a770 2023-09-30T22:21:24.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8127fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd81409dc30 0x7fd81413abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8127fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd81409dc30 0x7fd81413abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49664/0 (socket says 172.21.15.89:49664) 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8137fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8127fc700 1 -- 172.21.15.89:0/387576377 learned_addr learned my addr 172.21.15.89:0/387576377 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8137fe700 1 -- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd81406a770 msgr2=0x7fd81413a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8137fe700 1 --2- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd81406a770 0x7fd81413a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8137fe700 1 -- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd81409dc30 msgr2=0x7fd81413abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.525+0000 7fd8137fe700 1 --2- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd81409dc30 0x7fd81413abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.526+0000 7fd8137fe700 1 -- 172.21.15.89:0/387576377 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c009b30 con 0x7fd81413d1f0 2023-09-30T22:21:24.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.526+0000 7fd8127fc700 1 --2- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd81409dc30 0x7fd81413abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:24.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.526+0000 7fd8137fe700 1 --2- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413b130 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd80c015000 tx=0x7fd80c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:24.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.526+0000 7fd7fbfff700 1 -- 172.21.15.89:0/387576377 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c004830 con 0x7fd81413d1f0 2023-09-30T22:21:24.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.526+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd814138d80 con 0x7fd81413d1f0 2023-09-30T22:21:24.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.526+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81413a260 con 0x7fd81413d1f0 2023-09-30T22:21:24.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.526+0000 7fd7fbfff700 1 -- 172.21.15.89:0/387576377 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c0092e0 con 0x7fd81413d1f0 2023-09-30T22:21:24.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.526+0000 7fd7fbfff700 1 -- 172.21.15.89:0/387576377 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c01c650 con 0x7fd81413d1f0 2023-09-30T22:21:24.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.527+0000 7fd7fbfff700 1 -- 172.21.15.89:0/387576377 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd80c01c7b0 con 0x7fd81413d1f0 2023-09-30T22:21:24.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.527+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd814139630 con 0x7fd81413d1f0 2023-09-30T22:21:24.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.527+0000 7fd7fbfff700 1 -- 172.21.15.89:0/387576377 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd80c041050 con 0x7fd81413d1f0 2023-09-30T22:21:24.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.532+0000 7ff0f5628700 1 Processor -- start 2023-09-30T22:21:24.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.532+0000 7ff0f5628700 1 -- start start 2023-09-30T22:21:24.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0f5628700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0f009e1d0 0x7ff0f0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0f5628700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f0135e50 0x7ff0f0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0f5628700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f009f130 con 0x7ff0f009d6f0 2023-09-30T22:21:24.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0f5628700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f009f520 con 0x7ff0f009e1d0 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0f5628700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f009e930 con 0x7ff0f0135e50 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0ef7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f0135e50 0x7ff0f0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0ef7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f0135e50 0x7ff0f0138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59750/0 (socket says 172.21.15.89:59750) 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0ef7fe700 1 -- 172.21.15.89:0/2318661704 learned_addr learned my addr 172.21.15.89:0/2318661704 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0ef7fe700 1 -- 172.21.15.89:0/2318661704 >> v2:172.21.15.89:3300/0 conn(0x7ff0f009e1d0 msgr2=0x7ff0f0135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0ef7fe700 1 --2- 172.21.15.89:0/2318661704 >> v2:172.21.15.89:3300/0 conn(0x7ff0f009e1d0 0x7ff0f0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0ef7fe700 1 -- 172.21.15.89:0/2318661704 >> v1:172.21.15.89:6789/0 conn(0x7ff0f009d6f0 legacy=0x7ff0f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0ef7fe700 1 -- 172.21.15.89:0/2318661704 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f009b2d0 con 0x7ff0f0135e50 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.533+0000 7ff0ef7fe700 1 --2- 172.21.15.89:0/2318661704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f0135e50 0x7ff0f0138230 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800d150 tx=0x7ff0e800d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c59669051906aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0edffb700 1 -- 172.21.15.89:0/2318661704 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8010070 con 0x7ff0f0135e50 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0edffb700 1 -- 172.21.15.89:0/2318661704 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e800dd80 con 0x7ff0f0135e50 2023-09-30T22:21:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0edffb700 1 -- 172.21.15.89:0/2318661704 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e800b310 con 0x7ff0f0135e50 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0f5628700 1 -- 172.21.15.89:0/2318661704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f0135e50 msgr2=0x7ff0f0138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0f5628700 1 --2- 172.21.15.89:0/2318661704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f0135e50 0x7ff0f0138230 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800d150 tx=0x7ff0e800d460 comp rx=0 tx=0).stop 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0f5628700 1 -- 172.21.15.89:0/2318661704 shutdown_connections 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0f5628700 1 --2- 172.21.15.89:0/2318661704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f0135e50 0x7ff0f0138230 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0f5628700 1 --2- 172.21.15.89:0/2318661704 >> v2:172.21.15.89:3300/0 conn(0x7ff0f009e1d0 0x7ff0f0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.534+0000 7ff0f5628700 1 -- 172.21.15.89:0/2318661704 >> 172.21.15.89:0/2318661704 conn(0x7ff0f0066f50 msgr2=0x7ff0f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.535+0000 7ff0f5628700 1 -- 172.21.15.89:0/2318661704 shutdown_connections 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.535+0000 7ff0f5628700 1 -- 172.21.15.89:0/2318661704 wait complete. 2023-09-30T22:21:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0f5628700 1 Processor -- start 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0f5628700 1 -- start start 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0f5628700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009d6f0 0x7ff0f01c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0f5628700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e1d0 0x7ff0f01c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0f5628700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f0135e50 0x7ff0f011b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0f5628700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f011d2f0 con 0x7ff0f009d6f0 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0f5628700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f011d100 con 0x7ff0f0135e50 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0f5628700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f01c4e50 con 0x7ff0f009e1d0 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0eeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009d6f0 0x7ff0f01c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0eeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009d6f0 0x7ff0f01c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49692/0 (socket says 172.21.15.89:49692) 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0eeffd700 1 -- 172.21.15.89:0/1385961897 learned_addr learned my addr 172.21.15.89:0/1385961897 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0ef7fe700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f0135e50 0x7ff0f011b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0ee7fc700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e1d0 0x7ff0f01c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0eeffd700 1 -- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e1d0 msgr2=0x7ff0f01c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0eeffd700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e1d0 0x7ff0f01c4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.536+0000 7ff0eeffd700 1 -- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f0135e50 msgr2=0x7ff0f011b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0eeffd700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f0135e50 0x7ff0f011b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0eeffd700 1 -- 172.21.15.89:0/1385961897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e800bd40 con 0x7ff0f009d6f0 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0ee7fc700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e1d0 0x7ff0f01c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0eeffd700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009d6f0 0x7ff0f01c43d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000b4a0 tx=0x7ff0e000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7fd7fbfff700 1 -- 172.21.15.89:0/387576377 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd80c029020 con 0x7fd81413d1f0 2023-09-30T22:21:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0d77fe700 1 -- 172.21.15.89:0/1385961897 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0011830 con 0x7ff0f009d6f0 2023-09-30T22:21:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0d77fe700 1 -- 172.21.15.89:0/1385961897 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e00119b0 con 0x7ff0f009d6f0 2023-09-30T22:21:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0d77fe700 1 -- 172.21.15.89:0/1385961897 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0011d60 con 0x7ff0f009d6f0 2023-09-30T22:21:24.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f00a1f90 con 0x7ff0f009d6f0 2023-09-30T22:21:24.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.537+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f009c760 con 0x7ff0f009d6f0 2023-09-30T22:21:24.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.538+0000 7ff0d77fe700 1 -- 172.21.15.89:0/1385961897 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0e001a400 con 0x7ff0f009d6f0 2023-09-30T22:21:24.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.539+0000 7ff0d77fe700 1 -- 172.21.15.89:0/1385961897 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff0e003f6d0 con 0x7ff0f009d6f0 2023-09-30T22:21:24.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.539+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f011e9d0 con 0x7ff0f009d6f0 2023-09-30T22:21:24.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.548+0000 7ff0d77fe700 1 -- 172.21.15.89:0/1385961897 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0f011e9d0 con 0x7ff0f009d6f0 2023-09-30T22:21:24.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.673+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f563811bf50 con 0x7f5638130e50 2023-09-30T22:21:24.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.674+0000 7f562e7fc700 1 -- 172.21.15.89:0/2550127953 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5630007370 con 0x7f5638130e50 2023-09-30T22:21:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5638130e50 msgr2=0x7f5638119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 --2- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5638130e50 0x7f5638119750 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f563000b9a0 tx=0x7f563000bcb0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 shutdown_connections 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 --2- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5638130e50 0x7f5638119750 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 --2- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563806fb20 0x7f5638118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 --2- 172.21.15.89:0/2550127953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5638130540 0x7f5638119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 >> 172.21.15.89:0/2550127953 conn(0x7f56380a2aa0 msgr2=0x7f563812d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 shutdown_connections 2023-09-30T22:21:24.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.676+0000 7f563fc84700 1 -- 172.21.15.89:0/2550127953 wait complete. 2023-09-30T22:21:24.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:24.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.707+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbbd411c0b0 con 0x7fbbd409a3b0 2023-09-30T22:21:24.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.707+0000 7fbbca7fc700 1 -- 172.21.15.89:0/4196560585 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbd411c0b0 con 0x7fbbd409a3b0 2023-09-30T22:21:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.709+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 msgr2=0x7fbbd4119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.709+0000 7fbbdbbf8700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4119210 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400e900 tx=0x7fbbc400ec10 comp rx=0 tx=0).stop 2023-09-30T22:21:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.710+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 shutdown_connections 2023-09-30T22:21:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.710+0000 7fbbdbbf8700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbd4131810 0x7fbbd4119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.710+0000 7fbbdbbf8700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd4099a40 0x7fbbd4118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.710+0000 7fbbdbbf8700 1 --2- 172.21.15.89:0/4196560585 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbbd409a3b0 0x7fbbd4119210 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.710+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 >> 172.21.15.89:0/4196560585 conn(0x7fbbd40a2aa0 msgr2=0x7fbbd412e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.710+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 shutdown_connections 2023-09-30T22:21:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.710+0000 7fbbdbbf8700 1 -- 172.21.15.89:0/4196560585 wait complete. 2023-09-30T22:21:24.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:24.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.730+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd814074ea0 con 0x7fd81413d1f0 2023-09-30T22:21:24.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.730+0000 7fd7fbfff700 1 -- 172.21.15.89:0/387576377 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd80c033090 con 0x7fd81413d1f0 2023-09-30T22:21:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:24.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 msgr2=0x7fd81413b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 --2- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413b130 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd80c015000 tx=0x7fd80c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 shutdown_connections 2023-09-30T22:21:24.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 --2- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd81413d1f0 0x7fd81413b130 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 --2- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd81406a770 0x7fd81413a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 --2- 172.21.15.89:0/387576377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd81409dc30 0x7fd81413abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 >> 172.21.15.89:0/387576377 conn(0x7fd814066f50 msgr2=0x7fd81413c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:24.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 shutdown_connections 2023-09-30T22:21:24.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.732+0000 7fd8199c5700 1 -- 172.21.15.89:0/387576377 wait complete. 2023-09-30T22:21:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.742+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff0f011e9d0 con 0x7ff0f009d6f0 2023-09-30T22:21:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.743+0000 7ff0d77fe700 1 -- 172.21.15.89:0/1385961897 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0f011e9d0 con 0x7ff0f009d6f0 2023-09-30T22:21:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:24.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009d6f0 msgr2=0x7ff0f01c43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009d6f0 0x7ff0f01c43d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000b4a0 tx=0x7ff0e000b7b0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 shutdown_connections 2023-09-30T22:21:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff0f0135e50 0x7ff0f011b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0f009d6f0 0x7ff0f01c43d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 --2- 172.21.15.89:0/1385961897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0f009e1d0 0x7ff0f01c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 >> 172.21.15.89:0/1385961897 conn(0x7ff0f0066f50 msgr2=0x7ff0f0132c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 shutdown_connections 2023-09-30T22:21:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:24.745+0000 7ff0f5628700 1 -- 172.21.15.89:0/1385961897 wait complete. 2023-09-30T22:21:24.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:25.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:25.724 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:25.745 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:25.759 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.823+0000 7fb46a4fa700 1 Processor -- start 2023-09-30T22:21:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.823+0000 7fb46a4fa700 1 -- start start 2023-09-30T22:21:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb46a4fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412cf60 0x7fb46406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb46a4fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46412f8b0 0x7fb46412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb46a4fa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46412cd90 con 0x7fb464130260 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb46a4fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46412d330 con 0x7fb46412f8b0 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb46a4fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641306d0 con 0x7fb46412cf60 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb468a97700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb464130260 0x7fb464136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45824/0 (socket says 172.21.15.89:45824) 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb468a97700 1 -- 172.21.15.89:0/284603270 learned_addr learned my addr 172.21.15.89:0/284603270 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb463fff700 1 --2- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412cf60 0x7fb46406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb4637fe700 1 --2- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46412f8b0 0x7fb46412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb463fff700 1 -- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46412f8b0 msgr2=0x7fb46412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb463fff700 1 --2- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46412f8b0 0x7fb46412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb463fff700 1 -- 172.21.15.89:0/284603270 >> v1:172.21.15.89:6789/0 conn(0x7fb464130260 legacy=0x7fb464136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.824+0000 7fb463fff700 1 -- 172.21.15.89:0/284603270 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb464071780 con 0x7fb46412cf60 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb463fff700 1 --2- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412cf60 0x7fb46406f5d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb45400bb00 tx=0x7fb45400be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544c215c5315574e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb462ffd700 1 -- 172.21.15.89:0/284603270 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45400dd20 con 0x7fb46412cf60 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb462ffd700 1 -- 172.21.15.89:0/284603270 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45400de80 con 0x7fb46412cf60 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb462ffd700 1 -- 172.21.15.89:0/284603270 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454006d30 con 0x7fb46412cf60 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb46a4fa700 1 -- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412cf60 msgr2=0x7fb46406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb46a4fa700 1 --2- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412cf60 0x7fb46406f5d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb45400bb00 tx=0x7fb45400be10 comp rx=0 tx=0).stop 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb46a4fa700 1 -- 172.21.15.89:0/284603270 shutdown_connections 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb46a4fa700 1 --2- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412cf60 0x7fb46406f5d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb46a4fa700 1 --2- 172.21.15.89:0/284603270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb46412f8b0 0x7fb46412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb46a4fa700 1 -- 172.21.15.89:0/284603270 >> 172.21.15.89:0/284603270 conn(0x7fb4640a2aa0 msgr2=0x7fb4640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:25.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb46a4fa700 1 -- 172.21.15.89:0/284603270 shutdown_connections 2023-09-30T22:21:25.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.825+0000 7fb46a4fa700 1 -- 172.21.15.89:0/284603270 wait complete. 2023-09-30T22:21:25.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.826+0000 7fb46a4fa700 1 Processor -- start 2023-09-30T22:21:25.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb46a4fa700 1 -- start start 2023-09-30T22:21:25.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb46a4fa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46412cf60 0x7fb4641c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb46a4fa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412f8b0 0x7fb4641c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb46a4fa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb464130260 0x7fb4641c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb46a4fa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46409e780 con 0x7fb46412cf60 2023-09-30T22:21:25.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb46a4fa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46409d020 con 0x7fb464130260 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb46a4fa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb46409e030 con 0x7fb46412f8b0 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb463fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46412cf60 0x7fb4641c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb4637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412f8b0 0x7fb4641c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb463fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46412cf60 0x7fb4641c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49694/0 (socket says 172.21.15.89:49694) 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb463fff700 1 -- 172.21.15.89:0/1264217877 learned_addr learned my addr 172.21.15.89:0/1264217877 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb468a97700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb464130260 0x7fb4641c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.827+0000 7fb4637fe700 1 -- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb464130260 msgr2=0x7fb4641c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb4637fe700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb464130260 0x7fb4641c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb4637fe700 1 -- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46412cf60 msgr2=0x7fb4641c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb4637fe700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46412cf60 0x7fb4641c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb4637fe700 1 -- 172.21.15.89:0/1264217877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45400b730 con 0x7fb46412f8b0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb463fff700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46412cf60 0x7fb4641c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb4637fe700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412f8b0 0x7fb4641c4670 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb458009e80 tx=0x7fb45800eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb460ff9700 1 -- 172.21.15.89:0/1264217877 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800c070 con 0x7fb46412f8b0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb460ff9700 1 -- 172.21.15.89:0/1264217877 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458004220 con 0x7fb46412f8b0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46412f660 con 0x7fb46412f8b0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb460ff9700 1 -- 172.21.15.89:0/1264217877 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580138c0 con 0x7fb46412f8b0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.828+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb464119500 con 0x7fb46412f8b0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.829+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb464074ea0 con 0x7fb46412f8b0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.829+0000 7fb460ff9700 1 -- 172.21.15.89:0/1264217877 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb458013a20 con 0x7fb46412f8b0 2023-09-30T22:21:25.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.830+0000 7fb460ff9700 1 -- 172.21.15.89:0/1264217877 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb458042f30 con 0x7fb46412f8b0 2023-09-30T22:21:25.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.838+0000 7fb460ff9700 1 -- 172.21.15.89:0/1264217877 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb45800f040 con 0x7fb46412f8b0 2023-09-30T22:21:25.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.868+0000 7f7900579700 1 Processor -- start 2023-09-30T22:21:25.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.868+0000 7f7900579700 1 -- start start 2023-09-30T22:21:25.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.868+0000 7f7900579700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f806e570 0x7f78f806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.868+0000 7f7900579700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78f809eca0 0x7f78f806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.868+0000 7f7900579700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.868+0000 7f7900579700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f8070ee0 con 0x7f78f809eca0 2023-09-30T22:21:25.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.868+0000 7f7900579700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8071480 con 0x7f78f806e570 2023-09-30T22:21:25.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.868+0000 7f7900579700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78f806f3c0 con 0x7f78f8134610 2023-09-30T22:21:25.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fdb14700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78f809eca0 0x7f78f806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fdb14700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78f809eca0 0x7f78f806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49704/0 (socket says 172.21.15.89:49704) 2023-09-30T22:21:25.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fdb14700 1 -- 172.21.15.89:0/494494318 learned_addr learned my addr 172.21.15.89:0/494494318 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:25.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fe315700 1 --2- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f806e570 0x7f78f806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78feb16700 1 --2- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fe315700 1 -- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 msgr2=0x7f78f8134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:25.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fe315700 1 --2- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fe315700 1 -- 172.21.15.89:0/494494318 >> v2:172.21.15.89:3300/0 conn(0x7f78f809eca0 msgr2=0x7f78f806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:25.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fe315700 1 --2- 172.21.15.89:0/494494318 >> v2:172.21.15.89:3300/0 conn(0x7f78f809eca0 0x7f78f806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fe315700 1 -- 172.21.15.89:0/494494318 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8071b90 con 0x7f78f806e570 2023-09-30T22:21:25.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fe315700 1 --2- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f806e570 0x7f78f806e940 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009840 tx=0x7f78f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db14ddd78190d801 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:25.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fd313700 1 -- 172.21.15.89:0/494494318 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4011830 con 0x7f78f806e570 2023-09-30T22:21:25.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.869+0000 7f78fd313700 1 -- 172.21.15.89:0/494494318 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f4011990 con 0x7f78f806e570 2023-09-30T22:21:25.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f78fd313700 1 -- 172.21.15.89:0/494494318 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4011d20 con 0x7f78f806e570 2023-09-30T22:21:25.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 -- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f806e570 msgr2=0x7f78f806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 --2- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f806e570 0x7f78f806e940 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009840 tx=0x7f78f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:25.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 -- 172.21.15.89:0/494494318 shutdown_connections 2023-09-30T22:21:25.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 --2- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 --2- 172.21.15.89:0/494494318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f806e570 0x7f78f806e940 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 --2- 172.21.15.89:0/494494318 >> v2:172.21.15.89:3300/0 conn(0x7f78f809eca0 0x7f78f806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 -- 172.21.15.89:0/494494318 >> 172.21.15.89:0/494494318 conn(0x7f78f80a2aa0 msgr2=0x7f78f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 -- 172.21.15.89:0/494494318 shutdown_connections 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.870+0000 7f7900579700 1 -- 172.21.15.89:0/494494318 wait complete. 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.871+0000 7f7900579700 1 Processor -- start 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.871+0000 7f7900579700 1 -- start start 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f7900579700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f806e570 0x7f78f8127480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f7900579700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8125ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f7900579700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f81279c0 0x7f78f81263e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f7900579700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f813be10 con 0x7f78f806e570 2023-09-30T22:21:25.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f7900579700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80a20b0 con 0x7f78f81279c0 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f7900579700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8070610 con 0x7f78f8134610 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78fdb14700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8125ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78feb16700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f81279c0 0x7f78f81263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78fe315700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f806e570 0x7f78f8127480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78fdb14700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8125ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59800/0 (socket says 172.21.15.89:59800) 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78fdb14700 1 -- 172.21.15.89:0/1975577577 learned_addr learned my addr 172.21.15.89:0/1975577577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78feb16700 1 -- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 msgr2=0x7f78f8125ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78feb16700 1 --2- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8125ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78feb16700 1 -- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f806e570 msgr2=0x7f78f8127480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78feb16700 1 --2- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f806e570 0x7f78f8127480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78feb16700 1 -- 172.21.15.89:0/1975577577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f40094a0 con 0x7f78f81279c0 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78fdb14700 1 --2- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8125ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.872+0000 7f78feb16700 1 --2- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f81279c0 0x7f78f81263e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f78f000efc0 tx=0x7f78f000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.873+0000 7f78eeffd700 1 -- 172.21.15.89:0/1975577577 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000cd40 con 0x7f78f81279c0 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.873+0000 7f78eeffd700 1 -- 172.21.15.89:0/1975577577 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f000f040 con 0x7f78f81279c0 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.873+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f813e310 con 0x7f78f81279c0 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.873+0000 7f78eeffd700 1 -- 172.21.15.89:0/1975577577 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f00146c0 con 0x7f78f81279c0 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.873+0000 7f78eeffd700 1 -- 172.21.15.89:0/1975577577 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f78f0014820 con 0x7f78f81279c0 2023-09-30T22:21:25.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.874+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f813dec0 con 0x7f78f81279c0 2023-09-30T22:21:25.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.874+0000 7f78eeffd700 1 -- 172.21.15.89:0/1975577577 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f78f001c570 con 0x7f78f81279c0 2023-09-30T22:21:25.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.875+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f8074ea0 con 0x7f78f81279c0 2023-09-30T22:21:25.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.883+0000 7f78eeffd700 1 -- 172.21.15.89:0/1975577577 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78f0018030 con 0x7f78f81279c0 2023-09-30T22:21:25.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.908+0000 7f1b7518b700 1 Processor -- start 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.908+0000 7f1b7518b700 1 -- start start 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b7518b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b701337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b7518b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b70133e30 0x7f1b7012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b7518b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7006b990 con 0x7f1b70134260 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b7518b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b700a0590 con 0x7f1b70133e30 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b7518b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7009fb80 con 0x7f1b70133420 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b701337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b701337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59808/0 (socket says 172.21.15.89:59808) 2023-09-30T22:21:25.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6ed9d700 1 -- 172.21.15.89:0/2723929605 learned_addr learned my addr 172.21.15.89:0/2723929605 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:25.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6e59c700 1 --2- 172.21.15.89:0/2723929605 >> v2:172.21.15.89:3300/0 conn(0x7f1b70133e30 0x7f1b7012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6ed9d700 1 -- 172.21.15.89:0/2723929605 >> v2:172.21.15.89:3300/0 conn(0x7f1b70133e30 msgr2=0x7f1b7012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6ed9d700 1 --2- 172.21.15.89:0/2723929605 >> v2:172.21.15.89:3300/0 conn(0x7f1b70133e30 0x7f1b7012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6ed9d700 1 -- 172.21.15.89:0/2723929605 >> v1:172.21.15.89:6789/0 conn(0x7f1b70134260 legacy=0x7f1b701304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6ed9d700 1 -- 172.21.15.89:0/2723929605 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b700a0030 con 0x7f1b70133420 2023-09-30T22:21:25.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.909+0000 7f1b6ed9d700 1 --2- 172.21.15.89:0/2723929605 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b701337f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1b600098f0 tx=0x7f1b60009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9729e8522429c72 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b6dd9b700 1 -- 172.21.15.89:0/2723929605 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60011830 con 0x7f1b70133420 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b6dd9b700 1 -- 172.21.15.89:0/2723929605 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b60011990 con 0x7f1b70133420 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b6dd9b700 1 -- 172.21.15.89:0/2723929605 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60011d20 con 0x7f1b70133420 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b7518b700 1 -- 172.21.15.89:0/2723929605 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 msgr2=0x7f1b701337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b7518b700 1 --2- 172.21.15.89:0/2723929605 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b701337f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1b600098f0 tx=0x7f1b60009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b7518b700 1 -- 172.21.15.89:0/2723929605 shutdown_connections 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b7518b700 1 --2- 172.21.15.89:0/2723929605 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b701337f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b7518b700 1 --2- 172.21.15.89:0/2723929605 >> v2:172.21.15.89:3300/0 conn(0x7f1b70133e30 0x7f1b7012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b7518b700 1 -- 172.21.15.89:0/2723929605 >> 172.21.15.89:0/2723929605 conn(0x7f1b700a4210 msgr2=0x7f1b700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b7518b700 1 -- 172.21.15.89:0/2723929605 shutdown_connections 2023-09-30T22:21:25.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.910+0000 7f1b7518b700 1 -- 172.21.15.89:0/2723929605 wait complete. 2023-09-30T22:21:25.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b7518b700 1 Processor -- start 2023-09-30T22:21:25.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b7518b700 1 -- start start 2023-09-30T22:21:25.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b7518b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b7009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b7518b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70133e30 0x7f1b7009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b7518b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b70134260 0x7f1b7009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b7518b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7011fc90 con 0x7f1b70133e30 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b7518b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7009dff0 con 0x7f1b70134260 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b7518b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7012da80 con 0x7f1b70133420 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b6ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b7009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b6ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b7009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59814/0 (socket says 172.21.15.89:59814) 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b6ed9d700 1 -- 172.21.15.89:0/360486236 learned_addr learned my addr 172.21.15.89:0/360486236 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b6e59c700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70133e30 0x7f1b7009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.912+0000 7f1b6f59e700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b70134260 0x7f1b7009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b6ed9d700 1 -- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b70134260 msgr2=0x7f1b7009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b6ed9d700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b70134260 0x7f1b7009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b6ed9d700 1 -- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70133e30 msgr2=0x7f1b7009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b6ed9d700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70133e30 0x7f1b7009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b6ed9d700 1 -- 172.21.15.89:0/360486236 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60009550 con 0x7f1b70133420 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b6e59c700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70133e30 0x7f1b7009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b6ed9d700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b7009c480 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1b7009fcc0 tx=0x7f1b60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b577fe700 1 -- 172.21.15.89:0/360486236 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b600036a0 con 0x7f1b70133420 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b577fe700 1 -- 172.21.15.89:0/360486236 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b60003800 con 0x7f1b70133420 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.913+0000 7f1b577fe700 1 -- 172.21.15.89:0/360486236 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60003bb0 con 0x7f1b70133420 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.914+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b70120230 con 0x7f1b70133420 2023-09-30T22:21:25.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.914+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7011fe70 con 0x7f1b70133420 2023-09-30T22:21:25.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.914+0000 7f824b84f700 1 Processor -- start 2023-09-30T22:21:25.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.914+0000 7f824b84f700 1 -- start start 2023-09-30T22:21:25.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f824b84f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8244131810 0x7f8244131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f824b84f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f824409a580 0x7f8244139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f824b84f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82441313c0 con 0x7f8244099a40 2023-09-30T22:21:25.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f824b84f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824406e8c0 con 0x7f8244131810 2023-09-30T22:21:25.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f824b84f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8244071d20 con 0x7f824409a580 2023-09-30T22:21:25.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f1b577fe700 1 -- 172.21.15.89:0/360486236 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b60021070 con 0x7f1b70133420 2023-09-30T22:21:25.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b70120090 con 0x7f1b70133420 2023-09-30T22:21:25.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f82495eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8244131810 0x7f8244131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f8249dec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f824409a580 0x7f8244139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f8248dea700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8244099a40 0x7f8244099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45854/0 (socket says 172.21.15.89:45854) 2023-09-30T22:21:25.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f8248dea700 1 -- 172.21.15.89:0/3220866065 learned_addr learned my addr 172.21.15.89:0/3220866065 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:25.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f1b577fe700 1 -- 172.21.15.89:0/360486236 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1b60040d50 con 0x7f1b70133420 2023-09-30T22:21:25.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f8249dec700 1 -- 172.21.15.89:0/3220866065 >> v2:172.21.15.89:3300/0 conn(0x7f8244131810 msgr2=0x7f8244131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:25.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f8249dec700 1 --2- 172.21.15.89:0/3220866065 >> v2:172.21.15.89:3300/0 conn(0x7f8244131810 0x7f8244131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f8249dec700 1 -- 172.21.15.89:0/3220866065 >> v1:172.21.15.89:6789/0 conn(0x7f8244099a40 legacy=0x7f8244099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f8249dec700 1 -- 172.21.15.89:0/3220866065 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440721a0 con 0x7f824409a580 2023-09-30T22:21:25.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.915+0000 7f8249dec700 1 --2- 172.21.15.89:0/3220866065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f824409a580 0x7f8244139660 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8234009840 tx=0x7f8234009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be78964743c4d7bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:25.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f823bfff700 1 -- 172.21.15.89:0/3220866065 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234011830 con 0x7f824409a580 2023-09-30T22:21:25.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f823bfff700 1 -- 172.21.15.89:0/3220866065 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8234011990 con 0x7f824409a580 2023-09-30T22:21:25.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f823bfff700 1 -- 172.21.15.89:0/3220866065 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234011d20 con 0x7f824409a580 2023-09-30T22:21:25.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f1b577fe700 1 -- 172.21.15.89:0/360486236 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b70120090 con 0x7f1b70133420 2023-09-30T22:21:25.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f824b84f700 1 -- 172.21.15.89:0/3220866065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f824409a580 msgr2=0x7f8244139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f824b84f700 1 --2- 172.21.15.89:0/3220866065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f824409a580 0x7f8244139660 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8234009840 tx=0x7f8234009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:25.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f824b84f700 1 -- 172.21.15.89:0/3220866065 shutdown_connections 2023-09-30T22:21:25.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f824b84f700 1 --2- 172.21.15.89:0/3220866065 >> v2:172.21.15.89:3300/0 conn(0x7f8244131810 0x7f8244131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f824b84f700 1 --2- 172.21.15.89:0/3220866065 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f824409a580 0x7f8244139660 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.920+0000 7f824b84f700 1 -- 172.21.15.89:0/3220866065 >> 172.21.15.89:0/3220866065 conn(0x7f82440a2aa0 msgr2=0x7f82440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:25.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.921+0000 7f824b84f700 1 -- 172.21.15.89:0/3220866065 shutdown_connections 2023-09-30T22:21:25.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.921+0000 7f824b84f700 1 -- 172.21.15.89:0/3220866065 wait complete. 2023-09-30T22:21:25.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.922+0000 7f824b84f700 1 Processor -- start 2023-09-30T22:21:25.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.922+0000 7f824b84f700 1 -- start start 2023-09-30T22:21:25.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.922+0000 7f824b84f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8244099a40 0x7f824411ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.922+0000 7f824b84f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f824409a580 0x7f824411f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f824b84f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8244131810 0x7f824406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f824b84f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824411d0e0 con 0x7f824409a580 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f824b84f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8244071060 con 0x7f8244099a40 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f824b84f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f824411bdb0 con 0x7f8244131810 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f82495eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8244099a40 0x7f824411ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f82495eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8244099a40 0x7f824411ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33198/0 (socket says 172.21.15.89:33198) 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f82495eb700 1 -- 172.21.15.89:0/3750647321 learned_addr learned my addr 172.21.15.89:0/3750647321 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f8248dea700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f824409a580 0x7f824411f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f8249dec700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8244131810 0x7f824406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:25.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f8248dea700 1 -- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8244131810 msgr2=0x7f824406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f8248dea700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8244131810 0x7f824406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f8248dea700 1 -- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8244099a40 msgr2=0x7f824411ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f8248dea700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8244099a40 0x7f824411ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f8248dea700 1 -- 172.21.15.89:0/3750647321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82340094a0 con 0x7f824409a580 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f82495eb700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8244099a40 0x7f824411ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.923+0000 7f8248dea700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f824409a580 0x7f824411f440 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f824000c500 tx=0x7f824000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.924+0000 7f8239ffb700 1 -- 172.21.15.89:0/3750647321 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240005d70 con 0x7f824409a580 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.924+0000 7f8239ffb700 1 -- 172.21.15.89:0/3750647321 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824000ce70 con 0x7f824409a580 2023-09-30T22:21:25.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.924+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824411c2c0 con 0x7f824409a580 2023-09-30T22:21:25.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.924+0000 7f8239ffb700 1 -- 172.21.15.89:0/3750647321 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82400072d0 con 0x7f824409a580 2023-09-30T22:21:25.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.924+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824412ecd0 con 0x7f824409a580 2023-09-30T22:21:25.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.924+0000 7f8239ffb700 1 -- 172.21.15.89:0/3750647321 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8240007430 con 0x7f824409a580 2023-09-30T22:21:25.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.925+0000 7f8239ffb700 1 -- 172.21.15.89:0/3750647321 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f824003e8e0 con 0x7f824409a580 2023-09-30T22:21:25.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.925+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8244074ea0 con 0x7f824409a580 2023-09-30T22:21:25.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:25.930+0000 7f8239ffb700 1 -- 172.21.15.89:0/3750647321 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8240017080 con 0x7f824409a580 2023-09-30T22:21:26.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.073+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb4640713a0 con 0x7fb46412f8b0 2023-09-30T22:21:26.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.074+0000 7fb460ff9700 1 -- 172.21.15.89:0/1264217877 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb458022b30 con 0x7fb46412f8b0 2023-09-30T22:21:26.075 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.076+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412f8b0 msgr2=0x7fb4641c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.076+0000 7fb46a4fa700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412f8b0 0x7fb4641c4670 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb458009e80 tx=0x7fb45800eac0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.077+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 shutdown_connections 2023-09-30T22:21:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.077+0000 7fb46a4fa700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb464130260 0x7fb4641c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.077+0000 7fb46a4fa700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb46412cf60 0x7fb4641c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.077+0000 7fb46a4fa700 1 --2- 172.21.15.89:0/1264217877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46412f8b0 0x7fb4641c4670 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.077+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 >> 172.21.15.89:0/1264217877 conn(0x7fb4640a2aa0 msgr2=0x7fb46406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.077+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 shutdown_connections 2023-09-30T22:21:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.077+0000 7fb46a4fa700 1 -- 172.21.15.89:0/1264217877 wait complete. 2023-09-30T22:21:26.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:26.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.113+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f78f8071100 con 0x7f78f81279c0 2023-09-30T22:21:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.113+0000 7f78eeffd700 1 -- 172.21.15.89:0/1975577577 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78f00076f0 con 0x7f78f81279c0 2023-09-30T22:21:26.114 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:26.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f81279c0 msgr2=0x7f78f81263e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:26.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 --2- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f81279c0 0x7f78f81263e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f78f000efc0 tx=0x7f78f000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 shutdown_connections 2023-09-30T22:21:26.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 --2- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78f81279c0 0x7f78f81263e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 --2- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f806e570 0x7f78f8127480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 --2- 172.21.15.89:0/1975577577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78f8134610 0x7f78f8125ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 >> 172.21.15.89:0/1975577577 conn(0x7f78f80a2aa0 msgr2=0x7f78f8099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:26.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 shutdown_connections 2023-09-30T22:21:26.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.115+0000 7f7900579700 1 -- 172.21.15.89:0/1975577577 wait complete. 2023-09-30T22:21:26.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.124+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f824411cc90 con 0x7f824409a580 2023-09-30T22:21:26.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.125+0000 7f8239ffb700 1 -- 172.21.15.89:0/3750647321 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f824001b3b0 con 0x7f824409a580 2023-09-30T22:21:26.125 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.125+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1b70120090 con 0x7f1b70133420 2023-09-30T22:21:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.125+0000 7f1b577fe700 1 -- 172.21.15.89:0/360486236 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b70120090 con 0x7f1b70133420 2023-09-30T22:21:26.126 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f824409a580 msgr2=0x7f824411f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f824409a580 0x7f824411f440 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f824000c500 tx=0x7f824000c810 comp rx=0 tx=0).stop 2023-09-30T22:21:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 shutdown_connections 2023-09-30T22:21:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8244131810 0x7f824406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8244099a40 0x7f824411ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 --2- 172.21.15.89:0/3750647321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f824409a580 0x7f824411f440 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 >> 172.21.15.89:0/3750647321 conn(0x7f82440a2aa0 msgr2=0x7f82441b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 shutdown_connections 2023-09-30T22:21:26.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.127+0000 7f824b84f700 1 -- 172.21.15.89:0/3750647321 wait complete. 2023-09-30T22:21:26.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:26.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 msgr2=0x7f1b7009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:26.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b7009c480 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1b7009fcc0 tx=0x7f1b60004750 comp rx=0 tx=0).stop 2023-09-30T22:21:26.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 shutdown_connections 2023-09-30T22:21:26.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b70134260 0x7f1b7009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b70133420 0x7f1b7009c480 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 --2- 172.21.15.89:0/360486236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b70133e30 0x7f1b7009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 >> 172.21.15.89:0/360486236 conn(0x7f1b700a4210 msgr2=0x7f1b700a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 shutdown_connections 2023-09-30T22:21:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:26.128+0000 7f1b7518b700 1 -- 172.21.15.89:0/360486236 wait complete. 2023-09-30T22:21:26.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:26.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:27.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:27.130 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:27.142 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:27.142 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.239+0000 7f5b4b8c5700 1 Processor -- start 2023-09-30T22:21:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.239+0000 7f5b4b8c5700 1 -- start start 2023-09-30T22:21:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b4b8c5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409e930 0x7f5b4409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b4b8c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409f650 0x7f5b44131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b4b8c5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b44070940 con 0x7f5b4409f240 2023-09-30T22:21:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b4b8c5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4406fd60 con 0x7f5b4409e930 2023-09-30T22:21:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b4b8c5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4406fa80 con 0x7f5b4409f650 2023-09-30T22:21:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b49e62700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409f650 0x7f5b44131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b49661700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409e930 0x7f5b4409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b49e62700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409f650 0x7f5b44131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59834/0 (socket says 172.21.15.89:59834) 2023-09-30T22:21:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.240+0000 7f5b49e62700 1 -- 172.21.15.89:0/3142517814 learned_addr learned my addr 172.21.15.89:0/3142517814 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b49e62700 1 -- 172.21.15.89:0/3142517814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409e930 msgr2=0x7f5b4409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b49e62700 1 --2- 172.21.15.89:0/3142517814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409e930 0x7f5b4409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b49e62700 1 -- 172.21.15.89:0/3142517814 >> v1:172.21.15.89:6789/0 conn(0x7f5b4409f240 legacy=0x7f5b441317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b49e62700 1 -- 172.21.15.89:0/3142517814 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4409fb60 con 0x7f5b4409f650 2023-09-30T22:21:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b49661700 1 --2- 172.21.15.89:0/3142517814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409e930 0x7f5b4409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b49e62700 1 --2- 172.21.15.89:0/3142517814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409f650 0x7f5b44131f00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009e80 tx=0x7f5b3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab5f8adeedb0fe70 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b3bfff700 1 -- 172.21.15.89:0/3142517814 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00e070 con 0x7f5b4409f650 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b3bfff700 1 -- 172.21.15.89:0/3142517814 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c00b910 con 0x7f5b4409f650 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.241+0000 7f5b3bfff700 1 -- 172.21.15.89:0/3142517814 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00bcc0 con 0x7f5b4409f650 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.242+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/3142517814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409f650 msgr2=0x7f5b44131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.242+0000 7f5b4b8c5700 1 --2- 172.21.15.89:0/3142517814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409f650 0x7f5b44131f00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009e80 tx=0x7f5b3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.242+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/3142517814 shutdown_connections 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.242+0000 7f5b4b8c5700 1 --2- 172.21.15.89:0/3142517814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409e930 0x7f5b4409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.242+0000 7f5b4b8c5700 1 --2- 172.21.15.89:0/3142517814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409f650 0x7f5b44131f00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.242+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/3142517814 >> 172.21.15.89:0/3142517814 conn(0x7f5b440a4210 msgr2=0x7f5b440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.242+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/3142517814 shutdown_connections 2023-09-30T22:21:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.243+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/3142517814 wait complete. 2023-09-30T22:21:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.244+0000 7f5b4b8c5700 1 Processor -- start 2023-09-30T22:21:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.244+0000 7f5b4b8c5700 1 -- start start 2023-09-30T22:21:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.244+0000 7f5b4b8c5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409e930 0x7f5b441a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.244+0000 7f5b4b8c5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4409f240 0x7f5b441a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.244+0000 7f5b4b8c5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409f650 0x7f5b441a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.244+0000 7f5b4b8c5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441a7010 con 0x7f5b4409f240 2023-09-30T22:21:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.244+0000 7f5b4b8c5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441a5930 con 0x7f5b4409f650 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.244+0000 7f5b4b8c5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441a74c0 con 0x7f5b4409e930 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4409f240 0x7f5b441a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b49661700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409e930 0x7f5b441a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4409f240 0x7f5b441a41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49744/0 (socket says 172.21.15.89:49744) 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 -- 172.21.15.89:0/657686035 learned_addr learned my addr 172.21.15.89:0/657686035 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b49e62700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409f650 0x7f5b441a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 -- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409e930 msgr2=0x7f5b441a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409e930 0x7f5b441a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 -- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409f650 msgr2=0x7f5b441a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409f650 0x7f5b441a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 -- 172.21.15.89:0/657686035 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b400094a0 con 0x7f5b4409f240 2023-09-30T22:21:27.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b49e62700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409f650 0x7f5b441a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:27.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b48e60700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4409f240 0x7f5b441a41c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400c4b0 tx=0x7f5b3400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:27.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b39ffb700 1 -- 172.21.15.89:0/657686035 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340052d0 con 0x7f5b4409f240 2023-09-30T22:21:27.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b39ffb700 1 -- 172.21.15.89:0/657686035 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b34005430 con 0x7f5b4409f240 2023-09-30T22:21:27.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.245+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c009b30 con 0x7f5b4409f240 2023-09-30T22:21:27.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.246+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b440a0190 con 0x7f5b4409f240 2023-09-30T22:21:27.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.246+0000 7f5b39ffb700 1 -- 172.21.15.89:0/657686035 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340038c0 con 0x7f5b4409f240 2023-09-30T22:21:27.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.246+0000 7f5b39ffb700 1 -- 172.21.15.89:0/657686035 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b34003a20 con 0x7f5b4409f240 2023-09-30T22:21:27.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.247+0000 7f5b39ffb700 1 -- 172.21.15.89:0/657686035 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5b3403ea20 con 0x7f5b4409f240 2023-09-30T22:21:27.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.247+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b240052e0 con 0x7f5b4409f240 2023-09-30T22:21:27.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.252+0000 7f5b39ffb700 1 -- 172.21.15.89:0/657686035 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b34017080 con 0x7f5b4409f240 2023-09-30T22:21:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.277+0000 7f1bc406c700 1 Processor -- start 2023-09-30T22:21:27.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.277+0000 7f1bc406c700 1 -- start start 2023-09-30T22:21:27.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc406c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc406c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1bbc136fe0 0x7f1bbc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc406c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bbc070940 con 0x7f1bbc09e930 2023-09-30T22:21:27.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc406c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bbc06fd60 con 0x7f1bbc136fe0 2023-09-30T22:21:27.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc406c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc06fa80 con 0x7f1bbc09f410 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc1607700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc2609700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1bbc136fe0 0x7f1bbc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc1607700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33206/0 (socket says 172.21.15.89:33206) 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc2609700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1bbc136fe0 0x7f1bbc1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49746/0 (socket says 172.21.15.89:49746) 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc1607700 1 -- 172.21.15.89:0/117888096 learned_addr learned my addr 172.21.15.89:0/117888096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc2609700 1 -- 172.21.15.89:0/117888096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 msgr2=0x7f1bbc136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc2609700 1 --2- 172.21.15.89:0/117888096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc2609700 1 -- 172.21.15.89:0/117888096 >> v1:172.21.15.89:6789/0 conn(0x7f1bbc09e930 legacy=0x7f1bbc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc2609700 1 -- 172.21.15.89:0/117888096 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc09fb60 con 0x7f1bbc136fe0 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc1607700 1 --2- 172.21.15.89:0/117888096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc2609700 1 --2- 172.21.15.89:0/117888096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc136fe0 0x7f1bbc1393c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009d90 tx=0x7f1bb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb399035b37270e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:27.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.278+0000 7f1bc0e06700 1 -- 172.21.15.89:0/117888096 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb400e070 con 0x7f1bbc136fe0 2023-09-30T22:21:27.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc0e06700 1 -- 172.21.15.89:0/117888096 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb400bb20 con 0x7f1bbc136fe0 2023-09-30T22:21:27.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc0e06700 1 -- 172.21.15.89:0/117888096 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4012430 con 0x7f1bbc136fe0 2023-09-30T22:21:27.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc406c700 1 -- 172.21.15.89:0/117888096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc136fe0 msgr2=0x7f1bbc1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc406c700 1 --2- 172.21.15.89:0/117888096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc136fe0 0x7f1bbc1393c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009d90 tx=0x7f1bb400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc406c700 1 -- 172.21.15.89:0/117888096 shutdown_connections 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc406c700 1 --2- 172.21.15.89:0/117888096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc136fe0 0x7f1bbc1393c0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc406c700 1 --2- 172.21.15.89:0/117888096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc406c700 1 -- 172.21.15.89:0/117888096 >> 172.21.15.89:0/117888096 conn(0x7f1bbc0a4210 msgr2=0x7f1bbc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc406c700 1 -- 172.21.15.89:0/117888096 shutdown_connections 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.279+0000 7f1bc406c700 1 -- 172.21.15.89:0/117888096 wait complete. 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.280+0000 7f1bc406c700 1 Processor -- start 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.280+0000 7f1bc406c700 1 -- start start 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc406c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc09e930 0x7f1bbc11d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc406c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc11d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc406c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bbc136fe0 0x7f1bbc1c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc406c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc11fe50 con 0x7f1bbc09e930 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc406c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc11f130 con 0x7f1bbc09f410 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc406c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc09b220 con 0x7f1bbc136fe0 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc2609700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bbc136fe0 0x7f1bbc1c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc1e08700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc09e930 0x7f1bbc11d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc2609700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bbc136fe0 0x7f1bbc1c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59850/0 (socket says 172.21.15.89:59850) 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc2609700 1 -- 172.21.15.89:0/2683985979 learned_addr learned my addr 172.21.15.89:0/2683985979 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc1607700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc11d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc1e08700 1 -- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bbc136fe0 msgr2=0x7f1bbc1c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc1e08700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bbc136fe0 0x7f1bbc1c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc1e08700 1 -- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 msgr2=0x7f1bbc11d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc1e08700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc11d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc1e08700 1 -- 172.21.15.89:0/2683985979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac009550 con 0x7f1bbc09e930 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.281+0000 7f1bc1607700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc11d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.282+0000 7f1bc1e08700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc09e930 0x7f1bbc11d110 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800c450 tx=0x7f1bb800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.282+0000 7f1bb27fc700 1 -- 172.21.15.89:0/2683985979 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80052d0 con 0x7f1bbc09e930 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.282+0000 7f1bb27fc700 1 -- 172.21.15.89:0/2683985979 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8005430 con 0x7f1bbc09e930 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.282+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb4009a70 con 0x7f1bbc09e930 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.282+0000 7f1bb27fc700 1 -- 172.21.15.89:0/2683985979 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80038c0 con 0x7f1bbc09e930 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.282+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc11ea00 con 0x7f1bbc09e930 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.283+0000 7f1bb27fc700 1 -- 172.21.15.89:0/2683985979 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1bb8003a20 con 0x7f1bbc09e930 2023-09-30T22:21:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.283+0000 7f1bb27fc700 1 -- 172.21.15.89:0/2683985979 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1bb803ea50 con 0x7f1bbc09e930 2023-09-30T22:21:27.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.283+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc074ea0 con 0x7f1bbc09e930 2023-09-30T22:21:27.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.288+0000 7f1bb27fc700 1 -- 172.21.15.89:0/2683985979 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1bb8018260 con 0x7f1bbc09e930 2023-09-30T22:21:27.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.313+0000 7fcddcb7a700 1 Processor -- start 2023-09-30T22:21:27.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.313+0000 7fcddcb7a700 1 -- start start 2023-09-30T22:21:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcddcb7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8131810 0x7fcdd8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcddcb7a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcdd809a580 0x7fcdd8139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcddcb7a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd81313c0 con 0x7fcdd8099a40 2023-09-30T22:21:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcddcb7a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd809b580 con 0x7fcdd809a580 2023-09-30T22:21:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcddcb7a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd806e8c0 con 0x7fcdd8131810 2023-09-30T22:21:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcdd809a580 0x7fcdd8139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcdd809a580 0x7fcdd8139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49778/0 (socket says 172.21.15.89:49778) 2023-09-30T22:21:27.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd6d9d700 1 -- 172.21.15.89:0/71079561 learned_addr learned my addr 172.21.15.89:0/71079561 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd659c700 1 --2- 172.21.15.89:0/71079561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8131810 0x7fcdd8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd6d9d700 1 -- 172.21.15.89:0/71079561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8131810 msgr2=0x7fcdd8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd6d9d700 1 --2- 172.21.15.89:0/71079561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8131810 0x7fcdd8131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd6d9d700 1 -- 172.21.15.89:0/71079561 >> v1:172.21.15.89:6789/0 conn(0x7fcdd8099a40 legacy=0x7fcdd8099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd6d9d700 1 -- 172.21.15.89:0/71079561 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8071d20 con 0x7fcdd809a580 2023-09-30T22:21:27.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.314+0000 7fcdd559a700 1 -- 172.21.15.89:0/71079561 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290158538 0 0) 0x7fcdd81313c0 con 0x7fcdd8099a40 2023-09-30T22:21:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.315+0000 7fcdd6d9d700 1 --2- 172.21.15.89:0/71079561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 0x7fcdd8139660 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009e50 tx=0x7fcdd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6226befa32eecf30 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.315+0000 7fcdd559a700 1 -- 172.21.15.89:0/71079561 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000e070 con 0x7fcdd809a580 2023-09-30T22:21:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.315+0000 7fcdd559a700 1 -- 172.21.15.89:0/71079561 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd000b910 con 0x7fcdd809a580 2023-09-30T22:21:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.315+0000 7fcddcb7a700 1 -- 172.21.15.89:0/71079561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 msgr2=0x7fcdd8139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.315+0000 7fcddcb7a700 1 --2- 172.21.15.89:0/71079561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 0x7fcdd8139660 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009e50 tx=0x7fcdd000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.316+0000 7fcddcb7a700 1 -- 172.21.15.89:0/71079561 shutdown_connections 2023-09-30T22:21:27.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.316+0000 7fcddcb7a700 1 --2- 172.21.15.89:0/71079561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 0x7fcdd8139660 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.316+0000 7fcddcb7a700 1 --2- 172.21.15.89:0/71079561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8131810 0x7fcdd8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.316+0000 7fcddcb7a700 1 -- 172.21.15.89:0/71079561 >> 172.21.15.89:0/71079561 conn(0x7fcdd80a2aa0 msgr2=0x7fcdd80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.316+0000 7fcddcb7a700 1 -- 172.21.15.89:0/71079561 shutdown_connections 2023-09-30T22:21:27.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.316+0000 7fcddcb7a700 1 -- 172.21.15.89:0/71079561 wait complete. 2023-09-30T22:21:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcddcb7a700 1 Processor -- start 2023-09-30T22:21:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcddcb7a700 1 -- start start 2023-09-30T22:21:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcddcb7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8099a40 0x7fcdd81c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcddcb7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 0x7fcdd81c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcddcb7a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd8131810 0x7fcdd81c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcddcb7a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd812aef0 con 0x7fcdd809a580 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcddcb7a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8071890 con 0x7fcdd8099a40 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcddcb7a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd806eb00 con 0x7fcdd8131810 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcdd659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8099a40 0x7fcdd81c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcdd5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 0x7fcdd81c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcdd659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8099a40 0x7fcdd81c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33228/0 (socket says 172.21.15.89:33228) 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcdd659c700 1 -- 172.21.15.89:0/26654584 learned_addr learned my addr 172.21.15.89:0/26654584 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.317+0000 7fcdd6d9d700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd8131810 0x7fcdd81c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdd5d9b700 1 -- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd8131810 msgr2=0x7fcdd81c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdd5d9b700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd8131810 0x7fcdd81c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdd5d9b700 1 -- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8099a40 msgr2=0x7fcdd81c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdd5d9b700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8099a40 0x7fcdd81c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdd5d9b700 1 -- 172.21.15.89:0/26654584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd0009b30 con 0x7fcdd809a580 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdd659c700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8099a40 0x7fcdd81c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdd5d9b700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 0x7fcdd81c86b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00c430 tx=0x7fcdcc00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdc6ffd700 1 -- 172.21.15.89:0/26654584 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0052d0 con 0x7fcdd809a580 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdc6ffd700 1 -- 172.21.15.89:0/26654584 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc005450 con 0x7fcdd809a580 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcdc6ffd700 1 -- 172.21.15.89:0/26654584 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0038c0 con 0x7fcdd809a580 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd8072250 con 0x7fcdd809a580 2023-09-30T22:21:27.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.318+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd8129200 con 0x7fcdd809a580 2023-09-30T22:21:27.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.319+0000 7fcdc6ffd700 1 -- 172.21.15.89:0/26654584 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcdcc003a20 con 0x7fcdd809a580 2023-09-30T22:21:27.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.319+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd812bda0 con 0x7fcdd809a580 2023-09-30T22:21:27.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.320+0000 7fcdc6ffd700 1 -- 172.21.15.89:0/26654584 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcdcc03e9f0 con 0x7fcdd809a580 2023-09-30T22:21:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.324+0000 7fcdc6ffd700 1 -- 172.21.15.89:0/26654584 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcdcc019b40 con 0x7fcdd809a580 2023-09-30T22:21:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.349+0000 7f079db32700 1 Processor -- start 2023-09-30T22:21:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.349+0000 7f079db32700 1 -- start start 2023-09-30T22:21:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f079db32700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f079809e930 0x7f079809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f079db32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f240 0x7f07981317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f079db32700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0798070940 con 0x7f079809f650 2023-09-30T22:21:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f079db32700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079806fd60 con 0x7f079809e930 2023-09-30T22:21:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f079db32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f079806fa80 con 0x7f079809f240 2023-09-30T22:21:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f07977fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f079809e930 0x7f079809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f07977fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f079809e930 0x7f079809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49786/0 (socket says 172.21.15.89:49786) 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f07977fe700 1 -- 172.21.15.89:0/1449192357 learned_addr learned my addr 172.21.15.89:0/1449192357 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f0796ffd700 1 --2- 172.21.15.89:0/1449192357 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f240 0x7f07981317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f07977fe700 1 -- 172.21.15.89:0/1449192357 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f240 msgr2=0x7f07981317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f07977fe700 1 --2- 172.21.15.89:0/1449192357 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f240 0x7f07981317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f07977fe700 1 -- 172.21.15.89:0/1449192357 >> v1:172.21.15.89:6789/0 conn(0x7f079809f650 legacy=0x7f0798131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f07977fe700 1 -- 172.21.15.89:0/1449192357 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079809fb60 con 0x7f079809e930 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.350+0000 7f07977fe700 1 --2- 172.21.15.89:0/1449192357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079809ed00 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f07880098f0 tx=0x7f0788009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f4c79f64e8abce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.351+0000 7f07967fc700 1 -- 172.21.15.89:0/1449192357 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788004230 con 0x7f079809e930 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.351+0000 7f07967fc700 1 -- 172.21.15.89:0/1449192357 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788004390 con 0x7f079809e930 2023-09-30T22:21:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.351+0000 7f07967fc700 1 -- 172.21.15.89:0/1449192357 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880036a0 con 0x7f079809e930 2023-09-30T22:21:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.352+0000 7f079db32700 1 -- 172.21.15.89:0/1449192357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 msgr2=0x7f079809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.352+0000 7f079db32700 1 --2- 172.21.15.89:0/1449192357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079809ed00 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f07880098f0 tx=0x7f0788009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.352+0000 7f079db32700 1 -- 172.21.15.89:0/1449192357 shutdown_connections 2023-09-30T22:21:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.352+0000 7f079db32700 1 --2- 172.21.15.89:0/1449192357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079809ed00 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.352+0000 7f079db32700 1 --2- 172.21.15.89:0/1449192357 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f240 0x7f07981317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.352+0000 7f079db32700 1 -- 172.21.15.89:0/1449192357 >> 172.21.15.89:0/1449192357 conn(0x7f07980a4210 msgr2=0x7f07980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.352+0000 7f079db32700 1 -- 172.21.15.89:0/1449192357 shutdown_connections 2023-09-30T22:21:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.352+0000 7f079db32700 1 -- 172.21.15.89:0/1449192357 wait complete. 2023-09-30T22:21:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.353+0000 7f079db32700 1 Processor -- start 2023-09-30T22:21:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.353+0000 7f079db32700 1 -- start start 2023-09-30T22:21:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f079db32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f079db32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f079809f240 0x7f079806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f079db32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f650 0x7f079806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f079db32700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07981a88b0 con 0x7f079809e930 2023-09-30T22:21:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f079db32700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07981a98f0 con 0x7f079809f240 2023-09-30T22:21:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f079db32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07981a93e0 con 0x7f079809f650 2023-09-30T22:21:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f07977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f07977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49796/0 (socket says 172.21.15.89:49796) 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f07977fe700 1 -- 172.21.15.89:0/692249889 learned_addr learned my addr 172.21.15.89:0/692249889 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f0796ffd700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f079809f240 0x7f079806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.354+0000 7f0797fff700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f650 0x7f079806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f07977fe700 1 -- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f650 msgr2=0x7f079806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f07977fe700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f650 0x7f079806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f07977fe700 1 -- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f079809f240 msgr2=0x7f079806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f07977fe700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f079809f240 0x7f079806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f07977fe700 1 -- 172.21.15.89:0/692249889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0788009550 con 0x7f079809e930 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f0796ffd700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f079809f240 0x7f079806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f07977fe700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079806b8b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f07980a0b00 tx=0x7f0788004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:27.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f077ffff700 1 -- 172.21.15.89:0/692249889 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788003ae0 con 0x7f079809e930 2023-09-30T22:21:27.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f077ffff700 1 -- 172.21.15.89:0/692249889 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078801f070 con 0x7f079809e930 2023-09-30T22:21:27.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f077ffff700 1 -- 172.21.15.89:0/692249889 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788003e40 con 0x7f079809e930 2023-09-30T22:21:27.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07980a0190 con 0x7f079809e930 2023-09-30T22:21:27.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.355+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0798071d20 con 0x7f079809e930 2023-09-30T22:21:27.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.356+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07981a0b30 con 0x7f079809e930 2023-09-30T22:21:27.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.357+0000 7f077ffff700 1 -- 172.21.15.89:0/692249889 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0788021030 con 0x7f079809e930 2023-09-30T22:21:27.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.358+0000 7f077ffff700 1 -- 172.21.15.89:0/692249889 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0788040e60 con 0x7f079809e930 2023-09-30T22:21:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.362+0000 7f077ffff700 1 -- 172.21.15.89:0/692249889 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0788019780 con 0x7f079809e930 2023-09-30T22:21:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.492+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1bbc09e500 con 0x7f1bbc09e930 2023-09-30T22:21:27.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.492+0000 7f1bb27fc700 1 -- 172.21.15.89:0/2683985979 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1bb8017020 con 0x7f1bbc09e930 2023-09-30T22:21:27.495 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.494+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc09e930 msgr2=0x7f1bbc11d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.494+0000 7f1bc406c700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc09e930 0x7f1bbc11d110 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800c450 tx=0x7f1bb800c760 comp rx=0 tx=0).stop 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.494+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 shutdown_connections 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.494+0000 7f1bc406c700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bbc136fe0 0x7f1bbc1c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.494+0000 7f1bc406c700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bbc09e930 0x7f1bbc11d110 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.494+0000 7f1bc406c700 1 --2- 172.21.15.89:0/2683985979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bbc09f410 0x7f1bbc11d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.494+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 >> 172.21.15.89:0/2683985979 conn(0x7f1bbc0a4210 msgr2=0x7f1bbc09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.494+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 shutdown_connections 2023-09-30T22:21:27.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.495+0000 7f1bc406c700 1 -- 172.21.15.89:0/2683985979 wait complete. 2023-09-30T22:21:27.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.496+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b24005590 con 0x7f5b4409f240 2023-09-30T22:21:27.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.496+0000 7f5b39ffb700 1 -- 172.21.15.89:0/657686035 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b34019d20 con 0x7f5b4409f240 2023-09-30T22:21:27.497 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4409f240 msgr2=0x7f5b441a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4409f240 0x7f5b441a41c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400c4b0 tx=0x7f5b3400c7c0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 shutdown_connections 2023-09-30T22:21:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4409f650 0x7f5b441a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4409e930 0x7f5b441a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 --2- 172.21.15.89:0/657686035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4409f240 0x7f5b441a41c0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 >> 172.21.15.89:0/657686035 conn(0x7f5b440a4210 msgr2=0x7f5b441b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 shutdown_connections 2023-09-30T22:21:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.498+0000 7f5b4b8c5700 1 -- 172.21.15.89:0/657686035 wait complete. 2023-09-30T22:21:27.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:27.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.537+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcdd81286a0 con 0x7fcdd809a580 2023-09-30T22:21:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.538+0000 7fcdc6ffd700 1 -- 172.21.15.89:0/26654584 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdcc017030 con 0x7fcdd809a580 2023-09-30T22:21:27.538 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 msgr2=0x7fcdd81c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 0x7fcdd81c86b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00c430 tx=0x7fcdcc00c740 comp rx=0 tx=0).stop 2023-09-30T22:21:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 shutdown_connections 2023-09-30T22:21:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcdd8131810 0x7fcdd81c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcdd8099a40 0x7fcdd81c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 --2- 172.21.15.89:0/26654584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdd809a580 0x7fcdd81c86b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 >> 172.21.15.89:0/26654584 conn(0x7fcdd80a2aa0 msgr2=0x7fcdd81300d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 shutdown_connections 2023-09-30T22:21:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.540+0000 7fcddcb7a700 1 -- 172.21.15.89:0/26654584 wait complete. 2023-09-30T22:21:27.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:27.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.567+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f07981a0b30 con 0x7f079809e930 2023-09-30T22:21:27.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.568+0000 7f077ffff700 1 -- 172.21.15.89:0/692249889 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f07880193c0 con 0x7f079809e930 2023-09-30T22:21:27.568 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 msgr2=0x7f079806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079806b8b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f07980a0b00 tx=0x7f0788004750 comp rx=0 tx=0).stop 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 shutdown_connections 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f079809f650 0x7f079806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f079809e930 0x7f079806b8b0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 --2- 172.21.15.89:0/692249889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f079809f240 0x7f079806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 >> 172.21.15.89:0/692249889 conn(0x7f07980a4210 msgr2=0x7f07981b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 shutdown_connections 2023-09-30T22:21:27.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:27.570+0000 7f079db32700 1 -- 172.21.15.89:0/692249889 wait complete. 2023-09-30T22:21:27.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:28.513 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:28.514 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:28.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:28.583 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.658+0000 7f7f7c428700 1 Processor -- start 2023-09-30T22:21:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.658+0000 7f7f7c428700 1 -- start start 2023-09-30T22:21:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f7c428700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409e930 0x7f7f7409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f7c428700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f7c428700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f74070940 con 0x7f7f7409f650 2023-09-30T22:21:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f7c428700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7406fd60 con 0x7f7f7409f240 2023-09-30T22:21:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f7c428700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7406fa80 con 0x7f7f7409e930 2023-09-30T22:21:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f7a1c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409e930 0x7f7f7409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f7a1c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409e930 0x7f7f7409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59886/0 (socket says 172.21.15.89:59886) 2023-09-30T22:21:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f7a1c4700 1 -- 172.21.15.89:0/184249785 learned_addr learned my addr 172.21.15.89:0/184249785 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.659+0000 7f7f799c3700 1 --2- 172.21.15.89:0/184249785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f791c2700 1 -- 172.21.15.89:0/184249785 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687181991 0 0) 0x7f7f74070940 con 0x7f7f7409f650 2023-09-30T22:21:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f7a1c4700 1 -- 172.21.15.89:0/184249785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 msgr2=0x7f7f741317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f7a1c4700 1 --2- 172.21.15.89:0/184249785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f7a1c4700 1 -- 172.21.15.89:0/184249785 >> v1:172.21.15.89:6789/0 conn(0x7f7f7409f650 legacy=0x7f7f74131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f7a1c4700 1 -- 172.21.15.89:0/184249785 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7409fb60 con 0x7f7f7409e930 2023-09-30T22:21:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f7a1c4700 1 --2- 172.21.15.89:0/184249785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409e930 0x7f7f7409ed00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009920 tx=0x7f7f70009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42552886c091f0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f791c2700 1 -- 172.21.15.89:0/184249785 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f70011830 con 0x7f7f7409e930 2023-09-30T22:21:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f791c2700 1 -- 172.21.15.89:0/184249785 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f70011990 con 0x7f7f7409e930 2023-09-30T22:21:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.660+0000 7f7f791c2700 1 -- 172.21.15.89:0/184249785 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f70011d20 con 0x7f7f7409e930 2023-09-30T22:21:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.661+0000 7f7f7c428700 1 -- 172.21.15.89:0/184249785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409e930 msgr2=0x7f7f7409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.661+0000 7f7f7c428700 1 --2- 172.21.15.89:0/184249785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409e930 0x7f7f7409ed00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009920 tx=0x7f7f70009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.661+0000 7f7f7c428700 1 -- 172.21.15.89:0/184249785 shutdown_connections 2023-09-30T22:21:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.661+0000 7f7f7c428700 1 --2- 172.21.15.89:0/184249785 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409e930 0x7f7f7409ed00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.661+0000 7f7f7c428700 1 --2- 172.21.15.89:0/184249785 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.661+0000 7f7f7c428700 1 -- 172.21.15.89:0/184249785 >> 172.21.15.89:0/184249785 conn(0x7f7f740a4210 msgr2=0x7f7f740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.661+0000 7f7f7c428700 1 -- 172.21.15.89:0/184249785 shutdown_connections 2023-09-30T22:21:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.661+0000 7f7f7c428700 1 -- 172.21.15.89:0/184249785 wait complete. 2023-09-30T22:21:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7c428700 1 Processor -- start 2023-09-30T22:21:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7c428700 1 -- start start 2023-09-30T22:21:28.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7c428700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7409e930 0x7f7f741a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7c428700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7c428700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409f650 0x7f7f741c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7c428700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a5f30 con 0x7f7f7409e930 2023-09-30T22:21:28.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7c428700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a6110 con 0x7f7f7409f240 2023-09-30T22:21:28.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7c428700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741b6f80 con 0x7f7f7409f650 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7a1c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7409e930 0x7f7f741a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7a1c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7409e930 0x7f7f741a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49804/0 (socket says 172.21.15.89:49804) 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f7a1c4700 1 -- 172.21.15.89:0/1343591766 learned_addr learned my addr 172.21.15.89:0/1343591766 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.662+0000 7f7f799c3700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f7a9c5700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409f650 0x7f7f741c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f799c3700 1 -- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409f650 msgr2=0x7f7f741c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f799c3700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409f650 0x7f7f741c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f799c3700 1 -- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7409e930 msgr2=0x7f7f741a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f799c3700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7409e930 0x7f7f741a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f799c3700 1 -- 172.21.15.89:0/1343591766 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f70009550 con 0x7f7f7409f240 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f799c3700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741a3590 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400e900 tx=0x7f7f6400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f6affd700 1 -- 172.21.15.89:0/1343591766 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400ee50 con 0x7f7f7409f240 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f6affd700 1 -- 172.21.15.89:0/1343591766 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6400c850 con 0x7f7f7409f240 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f740a0190 con 0x7f7f7409f240 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f74071d20 con 0x7f7f7409f240 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.663+0000 7f7f6affd700 1 -- 172.21.15.89:0/1343591766 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400cbc0 con 0x7f7f7409f240 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.664+0000 7f7f6affd700 1 -- 172.21.15.89:0/1343591766 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f6400cd20 con 0x7f7f7409f240 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.664+0000 7f7f6affd700 1 -- 172.21.15.89:0/1343591766 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7f6403f670 con 0x7f7f7409f240 2023-09-30T22:21:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.664+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c0052e0 con 0x7f7f7409f240 2023-09-30T22:21:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.666+0000 7fd82cac5700 1 Processor -- start 2023-09-30T22:21:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.666+0000 7fd82cac5700 1 -- start start 2023-09-30T22:21:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82cac5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd82806e570 0x7fd82806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82cac5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82809eca0 0x7fd82806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82cac5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828134a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82cac5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd828070ee0 con 0x7fd82806e570 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82cac5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd828071480 con 0x7fd82809eca0 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82cac5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd82806f3c0 con 0x7fd828134610 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd82806e570 0x7fd82806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd82806e570 0x7fd82806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49818/0 (socket says 172.21.15.89:49818) 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82659c700 1 -- 172.21.15.89:0/1929830869 learned_addr learned my addr 172.21.15.89:0/1929830869 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd826d9d700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd825d9b700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82809eca0 0x7fd82806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82659c700 1 -- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 msgr2=0x7fd828134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.667+0000 7fd82659c700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828134a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.668+0000 7fd82659c700 1 -- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82809eca0 msgr2=0x7fd82806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.668+0000 7fd82659c700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82809eca0 0x7fd82806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.668+0000 7fd82659c700 1 -- 172.21.15.89:0/1929830869 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828071b90 con 0x7fd82806e570 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.668+0000 7fd82659c700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd82806e940 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd8180098f0 tx=0x7fd818009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b667235f836d35 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.668+0000 7fd82559a700 1 -- 172.21.15.89:0/1929830869 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818011830 con 0x7fd82806e570 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.668+0000 7fd82559a700 1 -- 172.21.15.89:0/1929830869 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818011990 con 0x7fd82806e570 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.668+0000 7fd82559a700 1 -- 172.21.15.89:0/1929830869 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818011d20 con 0x7fd82806e570 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 -- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 msgr2=0x7fd82806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd82806e940 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd8180098f0 tx=0x7fd818009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 -- 172.21.15.89:0/1929830869 shutdown_connections 2023-09-30T22:21:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd82806e940 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 --2- 172.21.15.89:0/1929830869 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82809eca0 0x7fd82806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 -- 172.21.15.89:0/1929830869 >> 172.21.15.89:0/1929830869 conn(0x7fd8280a2aa0 msgr2=0x7fd8280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 -- 172.21.15.89:0/1929830869 shutdown_connections 2023-09-30T22:21:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.669+0000 7fd82cac5700 1 -- 172.21.15.89:0/1929830869 wait complete. 2023-09-30T22:21:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.670+0000 7f7f6affd700 1 -- 172.21.15.89:0/1343591766 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f640232f0 con 0x7f7f7409f240 2023-09-30T22:21:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.671+0000 7fd82cac5700 1 Processor -- start 2023-09-30T22:21:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.671+0000 7fd82cac5700 1 -- start start 2023-09-30T22:21:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.671+0000 7fd82cac5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd828127410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.671+0000 7fd82cac5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828125fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.671+0000 7fd82cac5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd828127950 0x7fd8281264f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.671+0000 7fd82cac5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82813b1b0 con 0x7fd82806e570 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.671+0000 7fd82cac5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82813d760 con 0x7fd828127950 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.671+0000 7fd82cac5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd82813be00 con 0x7fd828134610 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd82659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd828127410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd82659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd828127410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49830/0 (socket says 172.21.15.89:49830) 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd82659c700 1 -- 172.21.15.89:0/89024615 learned_addr learned my addr 172.21.15.89:0/89024615 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd825d9b700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828125fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd826d9d700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd828127950 0x7fd8281264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd825d9b700 1 -- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd828127950 msgr2=0x7fd8281264f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd825d9b700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd828127950 0x7fd8281264f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd825d9b700 1 -- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 msgr2=0x7fd828127410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd825d9b700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd828127410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd825d9b700 1 -- 172.21.15.89:0/89024615 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd818009550 con 0x7fd828134610 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd82659c700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd828127410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd825d9b700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828125fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00e900 tx=0x7fd81c00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd816ffd700 1 -- 172.21.15.89:0/89024615 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00ee50 con 0x7fd828134610 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd816ffd700 1 -- 172.21.15.89:0/89024615 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c00c850 con 0x7fd828134610 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd816ffd700 1 -- 172.21.15.89:0/89024615 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00cbc0 con 0x7fd828134610 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.672+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8280a2000 con 0x7fd828134610 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.673+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82813def0 con 0x7fd828134610 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.674+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8280a1aa0 con 0x7fd828134610 2023-09-30T22:21:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.674+0000 7fd816ffd700 1 -- 172.21.15.89:0/89024615 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd81c00cd20 con 0x7fd828134610 2023-09-30T22:21:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.678+0000 7fd816ffd700 1 -- 172.21.15.89:0/89024615 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd81c01a360 con 0x7fd828134610 2023-09-30T22:21:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.679+0000 7fd816ffd700 1 -- 172.21.15.89:0/89024615 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd81c044060 con 0x7fd828134610 2023-09-30T22:21:28.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.695+0000 7f63220ab700 1 Processor -- start 2023-09-30T22:21:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.695+0000 7f63220ab700 1 -- start start 2023-09-30T22:21:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f63220ab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f631c06c0e0 0x7f631c09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f63220ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c1380f0 0x7f631c13fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f63220ab700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631c134980 con 0x7f631c134ec0 2023-09-30T22:21:28.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f63220ab700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c06c4b0 con 0x7f631c06c0e0 2023-09-30T22:21:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f63220ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0a09c0 con 0x7f631c1380f0 2023-09-30T22:21:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c1380f0 0x7f631c13fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c1380f0 0x7f631c13fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59930/0 (socket says 172.21.15.89:59930) 2023-09-30T22:21:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631bfff700 1 -- 172.21.15.89:0/843575778 learned_addr learned my addr 172.21.15.89:0/843575778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631b7fe700 1 --2- 172.21.15.89:0/843575778 >> v2:172.21.15.89:3300/0 conn(0x7f631c06c0e0 0x7f631c09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631bfff700 1 -- 172.21.15.89:0/843575778 >> v2:172.21.15.89:3300/0 conn(0x7f631c06c0e0 msgr2=0x7f631c09ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631bfff700 1 --2- 172.21.15.89:0/843575778 >> v2:172.21.15.89:3300/0 conn(0x7f631c06c0e0 0x7f631c09ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631bfff700 1 -- 172.21.15.89:0/843575778 >> v1:172.21.15.89:6789/0 conn(0x7f631c134ec0 legacy=0x7f631c1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631bfff700 1 -- 172.21.15.89:0/843575778 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0a0600 con 0x7f631c1380f0 2023-09-30T22:21:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.696+0000 7f631bfff700 1 --2- 172.21.15.89:0/843575778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c1380f0 0x7f631c13fd20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6314009e50 tx=0x7f631400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0c3705d9b3c113 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f631affd700 1 -- 172.21.15.89:0/843575778 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631400e070 con 0x7f631c1380f0 2023-09-30T22:21:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f631affd700 1 -- 172.21.15.89:0/843575778 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631400b910 con 0x7f631c1380f0 2023-09-30T22:21:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f631affd700 1 -- 172.21.15.89:0/843575778 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631400bcc0 con 0x7f631c1380f0 2023-09-30T22:21:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f63220ab700 1 -- 172.21.15.89:0/843575778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c1380f0 msgr2=0x7f631c13fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f63220ab700 1 --2- 172.21.15.89:0/843575778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c1380f0 0x7f631c13fd20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6314009e50 tx=0x7f631400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f63220ab700 1 -- 172.21.15.89:0/843575778 shutdown_connections 2023-09-30T22:21:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f63220ab700 1 --2- 172.21.15.89:0/843575778 >> v2:172.21.15.89:3300/0 conn(0x7f631c06c0e0 0x7f631c09ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f63220ab700 1 --2- 172.21.15.89:0/843575778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c1380f0 0x7f631c13fd20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f63220ab700 1 -- 172.21.15.89:0/843575778 >> 172.21.15.89:0/843575778 conn(0x7f631c0a4210 msgr2=0x7f631c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f63220ab700 1 -- 172.21.15.89:0/843575778 shutdown_connections 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.697+0000 7f63220ab700 1 -- 172.21.15.89:0/843575778 wait complete. 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.698+0000 7f63220ab700 1 Processor -- start 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.698+0000 7f63220ab700 1 -- start start 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f63220ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f631c06c0e0 0x7f631c1ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f63220ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c134ec0 0x7f631c1cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f63220ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f631c1380f0 0x7f631c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f63220ab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0a03f0 con 0x7f631c06c0e0 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f63220ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c11df20 con 0x7f631c1380f0 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f63220ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f631c11c5f0 con 0x7f631c134ec0 2023-09-30T22:21:28.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c134ec0 0x7f631c1cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c134ec0 0x7f631c1cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59942/0 (socket says 172.21.15.89:59942) 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f6312ffd700 1 -- 172.21.15.89:0/2191303439 learned_addr learned my addr 172.21.15.89:0/2191303439 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f631b7fe700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f631c06c0e0 0x7f631c1ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f631bfff700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f631c1380f0 0x7f631c11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f6312ffd700 1 -- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f631c1380f0 msgr2=0x7f631c11a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f6312ffd700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f631c1380f0 0x7f631c11a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f6312ffd700 1 -- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f631c06c0e0 msgr2=0x7f631c1ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f6312ffd700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f631c06c0e0 0x7f631c1ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.699+0000 7f6312ffd700 1 -- 172.21.15.89:0/2191303439 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6314009b30 con 0x7f631c134ec0 2023-09-30T22:21:28.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.700+0000 7f6312ffd700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c134ec0 0x7f631c1cd290 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f630400c4e0 tx=0x7f630400c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:28.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.700+0000 7f6318ff9700 1 -- 172.21.15.89:0/2191303439 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040052d0 con 0x7f631c134ec0 2023-09-30T22:21:28.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.700+0000 7f6318ff9700 1 -- 172.21.15.89:0/2191303439 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6304005430 con 0x7f631c134ec0 2023-09-30T22:21:28.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.700+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c11c830 con 0x7f631c134ec0 2023-09-30T22:21:28.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.700+0000 7f6318ff9700 1 -- 172.21.15.89:0/2191303439 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040038c0 con 0x7f631c134ec0 2023-09-30T22:21:28.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.700+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c11ce30 con 0x7f631c134ec0 2023-09-30T22:21:28.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.701+0000 7f6318ff9700 1 -- 172.21.15.89:0/2191303439 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6304003a20 con 0x7f631c134ec0 2023-09-30T22:21:28.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.701+0000 7f6318ff9700 1 -- 172.21.15.89:0/2191303439 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f630403e8c0 con 0x7f631c134ec0 2023-09-30T22:21:28.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.701+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c074ea0 con 0x7f631c134ec0 2023-09-30T22:21:28.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.706+0000 7f6318ff9700 1 -- 172.21.15.89:0/2191303439 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6304018260 con 0x7f631c134ec0 2023-09-30T22:21:28.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.749+0000 7faa02c61700 1 Processor -- start 2023-09-30T22:21:28.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.749+0000 7faa02c61700 1 -- start start 2023-09-30T22:21:28.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa02c61700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09d6f0 0x7fa9fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa02c61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa02c61700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9fc09f130 con 0x7fa9fc09e000 2023-09-30T22:21:28.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa02c61700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc09f520 con 0x7fa9fc09e410 2023-09-30T22:21:28.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa02c61700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc09e930 con 0x7fa9fc09d6f0 2023-09-30T22:21:28.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa011fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7fa9fbfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa9fc09e000 0x7fa9fc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45918/0 (socket says 172.21.15.89:45918) 2023-09-30T22:21:28.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7fa9fbfff700 1 -- 172.21.15.89:0/4005592310 learned_addr learned my addr 172.21.15.89:0/4005592310 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:28.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa009fd700 1 --2- 172.21.15.89:0/4005592310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09d6f0 0x7fa9fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa011fe700 1 -- 172.21.15.89:0/4005592310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09d6f0 msgr2=0x7fa9fc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa011fe700 1 --2- 172.21.15.89:0/4005592310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09d6f0 0x7fa9fc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa011fe700 1 -- 172.21.15.89:0/4005592310 >> v1:172.21.15.89:6789/0 conn(0x7fa9fc09e000 legacy=0x7fa9fc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7fa9fb7fe700 1 -- 172.21.15.89:0/4005592310 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292228494 0 0) 0x7fa9fc09f130 con 0x7fa9fc09e000 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa011fe700 1 -- 172.21.15.89:0/4005592310 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc09b2d0 con 0x7fa9fc09e410 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.750+0000 7faa011fe700 1 --2- 172.21.15.89:0/4005592310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc130d70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009dc0 tx=0x7fa9f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee0356e6829ce6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7fa9fb7fe700 1 -- 172.21.15.89:0/4005592310 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f400bd20 con 0x7fa9fc09e410 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7fa9fb7fe700 1 -- 172.21.15.89:0/4005592310 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f400be80 con 0x7fa9fc09e410 2023-09-30T22:21:28.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7fa9fb7fe700 1 -- 172.21.15.89:0/4005592310 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40116a0 con 0x7fa9fc09e410 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7faa02c61700 1 -- 172.21.15.89:0/4005592310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 msgr2=0x7fa9fc130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7faa02c61700 1 --2- 172.21.15.89:0/4005592310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc130d70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009dc0 tx=0x7fa9f400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7faa02c61700 1 -- 172.21.15.89:0/4005592310 shutdown_connections 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7faa02c61700 1 --2- 172.21.15.89:0/4005592310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc130d70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7faa02c61700 1 --2- 172.21.15.89:0/4005592310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09d6f0 0x7fa9fc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7faa02c61700 1 -- 172.21.15.89:0/4005592310 >> 172.21.15.89:0/4005592310 conn(0x7fa9fc066f50 msgr2=0x7fa9fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7faa02c61700 1 -- 172.21.15.89:0/4005592310 shutdown_connections 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.751+0000 7faa02c61700 1 -- 172.21.15.89:0/4005592310 wait complete. 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.752+0000 7faa02c61700 1 Processor -- start 2023-09-30T22:21:28.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.752+0000 7faa02c61700 1 -- start start 2023-09-30T22:21:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa02c61700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9fc09d6f0 0x7fa9fc11dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa02c61700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09e000 0x7fa9fc11e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa02c61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc11ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa02c61700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc11f490 con 0x7fa9fc09d6f0 2023-09-30T22:21:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa02c61700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc1303a0 con 0x7fa9fc09e410 2023-09-30T22:21:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa02c61700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc1201f0 con 0x7fa9fc09e000 2023-09-30T22:21:28.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa011fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc11ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa011fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc11ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33306/0 (socket says 172.21.15.89:33306) 2023-09-30T22:21:28.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa011fe700 1 -- 172.21.15.89:0/3533507362 learned_addr learned my addr 172.21.15.89:0/3533507362 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:28.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7fa9fbfff700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09e000 0x7fa9fc11e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa009fd700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9fc09d6f0 0x7fa9fc11dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:28.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa011fe700 1 -- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09e000 msgr2=0x7fa9fc11e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa011fe700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09e000 0x7fa9fc11e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa011fe700 1 -- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9fc09d6f0 msgr2=0x7fa9fc11dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa011fe700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9fc09d6f0 0x7fa9fc11dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa011fe700 1 -- 172.21.15.89:0/3533507362 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f4009a70 con 0x7fa9fc09e410 2023-09-30T22:21:28.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.753+0000 7faa009fd700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9fc09d6f0 0x7fa9fc11dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:28.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.754+0000 7faa011fe700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc11ea40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc09d470 tx=0x7fa9f4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:28.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.754+0000 7fa9f97fa700 1 -- 172.21.15.89:0/3533507362 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4004930 con 0x7fa9fc09e410 2023-09-30T22:21:28.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.754+0000 7fa9f97fa700 1 -- 172.21.15.89:0/3533507362 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f4011460 con 0x7fa9fc09e410 2023-09-30T22:21:28.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.754+0000 7fa9f97fa700 1 -- 172.21.15.89:0/3533507362 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40117f0 con 0x7fa9fc09e410 2023-09-30T22:21:28.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.754+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9fc120e20 con 0x7fa9fc09e410 2023-09-30T22:21:28.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.754+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9fc09c700 con 0x7fa9fc09e410 2023-09-30T22:21:28.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.755+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9fc11ac20 con 0x7fa9fc09e410 2023-09-30T22:21:28.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.756+0000 7fa9f97fa700 1 -- 172.21.15.89:0/3533507362 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9f4011950 con 0x7fa9fc09e410 2023-09-30T22:21:28.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.756+0000 7fa9f97fa700 1 -- 172.21.15.89:0/3533507362 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa9f4041950 con 0x7fa9fc09e410 2023-09-30T22:21:28.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.765+0000 7fa9f97fa700 1 -- 172.21.15.89:0/3533507362 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9f402a950 con 0x7fa9fc09e410 2023-09-30T22:21:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.873+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7f5c005590 con 0x7f7f7409f240 2023-09-30T22:21:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.873+0000 7f7f6affd700 1 -- 172.21.15.89:0/1343591766 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f64021020 con 0x7f7f7409f240 2023-09-30T22:21:28.874 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 msgr2=0x7f7f741a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741a3590 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400e900 tx=0x7f7f6400ec10 comp rx=0 tx=0).stop 2023-09-30T22:21:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 shutdown_connections 2023-09-30T22:21:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7f7409f650 0x7f7f741c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f7409e930 0x7f7f741a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 --2- 172.21.15.89:0/1343591766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7409f240 0x7f7f741a3590 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 >> 172.21.15.89:0/1343591766 conn(0x7f7f740a4210 msgr2=0x7f7f7406c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 shutdown_connections 2023-09-30T22:21:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.876+0000 7f7f7c428700 1 -- 172.21.15.89:0/1343591766 wait complete. 2023-09-30T22:21:28.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:28.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.904+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd828074ea0 con 0x7fd828134610 2023-09-30T22:21:28.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.904+0000 7fd816ffd700 1 -- 172.21.15.89:0/89024615 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd81c017070 con 0x7fd828134610 2023-09-30T22:21:28.905 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:28.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 msgr2=0x7fd828125fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828125fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00e900 tx=0x7fd81c00ec10 comp rx=0 tx=0).stop 2023-09-30T22:21:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 shutdown_connections 2023-09-30T22:21:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd828127950 0x7fd8281264f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82806e570 0x7fd828127410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 --2- 172.21.15.89:0/89024615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd828134610 0x7fd828125fb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 >> 172.21.15.89:0/89024615 conn(0x7fd8280a2aa0 msgr2=0x7fd828099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:28.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 shutdown_connections 2023-09-30T22:21:28.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.906+0000 7fd82cac5700 1 -- 172.21.15.89:0/89024615 wait complete. 2023-09-30T22:21:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.911+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f631c06b250 con 0x7f631c134ec0 2023-09-30T22:21:28.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.911+0000 7f6318ff9700 1 -- 172.21.15.89:0/2191303439 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6304017070 con 0x7f631c134ec0 2023-09-30T22:21:28.912 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:28.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.913+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c134ec0 msgr2=0x7f631c1cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.913+0000 7f63220ab700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c134ec0 0x7f631c1cd290 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f630400c4e0 tx=0x7f630400c7f0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.913+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 shutdown_connections 2023-09-30T22:21:28.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.913+0000 7f63220ab700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f631c1380f0 0x7f631c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.913+0000 7f63220ab700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f631c06c0e0 0x7f631c1ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.913+0000 7f63220ab700 1 --2- 172.21.15.89:0/2191303439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f631c134ec0 0x7f631c1cd290 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.913+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 >> 172.21.15.89:0/2191303439 conn(0x7f631c0a4210 msgr2=0x7f631c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:28.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.914+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 shutdown_connections 2023-09-30T22:21:28.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.914+0000 7f63220ab700 1 -- 172.21.15.89:0/2191303439 wait complete. 2023-09-30T22:21:28.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:28.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:28.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.972+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa9fc074ea0 con 0x7fa9fc09e410 2023-09-30T22:21:28.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.972+0000 7fa9f97fa700 1 -- 172.21.15.89:0/3533507362 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9f402a360 con 0x7fa9fc09e410 2023-09-30T22:21:28.973 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 msgr2=0x7fa9fc11ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc11ea40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc09d470 tx=0x7fa9f4004b40 comp rx=0 tx=0).stop 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 shutdown_connections 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9fc09e410 0x7fa9fc11ea40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9fc09d6f0 0x7fa9fc11dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 --2- 172.21.15.89:0/3533507362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa9fc09e000 0x7fa9fc11e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 >> 172.21.15.89:0/3533507362 conn(0x7fa9fc066f50 msgr2=0x7fa9fc130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 shutdown_connections 2023-09-30T22:21:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:28.974+0000 7faa02c61700 1 -- 172.21.15.89:0/3533507362 wait complete. 2023-09-30T22:21:28.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:29.894 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:29.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:29.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:29.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:30.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.036+0000 7faa99c4e700 1 Processor -- start 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa99c4e700 1 -- start start 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa99c4e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa9409eca0 0x7faa9413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa99c4e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 0x7faa9413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa99c4e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa94070ee0 con 0x7faa9406e570 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa99c4e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa94071980 con 0x7faa9409eca0 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa99c4e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa94071480 con 0x7faa9406f050 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa92ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa9409eca0 0x7faa9413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa92ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa9409eca0 0x7faa9413cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49860/0 (socket says 172.21.15.89:49860) 2023-09-30T22:21:30.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.037+0000 7faa92ffd700 1 -- 172.21.15.89:0/1213316072 learned_addr learned my addr 172.21.15.89:0/1213316072 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.038+0000 7faa92ffd700 1 -- 172.21.15.89:0/1213316072 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 msgr2=0x7faa9413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.038+0000 7faa92ffd700 1 --2- 172.21.15.89:0/1213316072 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 0x7faa9413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.038+0000 7faa92ffd700 1 -- 172.21.15.89:0/1213316072 >> v1:172.21.15.89:6789/0 conn(0x7faa9406e570 legacy=0x7faa9406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.038+0000 7faa92ffd700 1 -- 172.21.15.89:0/1213316072 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa94071b90 con 0x7faa9409eca0 2023-09-30T22:21:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.038+0000 7faa92ffd700 1 --2- 172.21.15.89:0/1213316072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9409eca0 0x7faa9413cea0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7faa880098f0 tx=0x7faa88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a08135ce83eeeb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.038+0000 7faa927fc700 1 -- 172.21.15.89:0/1213316072 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88004230 con 0x7faa9409eca0 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.038+0000 7faa927fc700 1 -- 172.21.15.89:0/1213316072 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa88004390 con 0x7faa9409eca0 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.038+0000 7faa927fc700 1 -- 172.21.15.89:0/1213316072 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880036a0 con 0x7faa9409eca0 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.039+0000 7faa99c4e700 1 -- 172.21.15.89:0/1213316072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9409eca0 msgr2=0x7faa9413cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.039+0000 7faa99c4e700 1 --2- 172.21.15.89:0/1213316072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9409eca0 0x7faa9413cea0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7faa880098f0 tx=0x7faa88009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.039+0000 7faa99c4e700 1 -- 172.21.15.89:0/1213316072 shutdown_connections 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.039+0000 7faa99c4e700 1 --2- 172.21.15.89:0/1213316072 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 0x7faa9413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.039+0000 7faa99c4e700 1 --2- 172.21.15.89:0/1213316072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9409eca0 0x7faa9413cea0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.039+0000 7faa99c4e700 1 -- 172.21.15.89:0/1213316072 >> 172.21.15.89:0/1213316072 conn(0x7faa940a2aa0 msgr2=0x7faa940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.039+0000 7faa99c4e700 1 -- 172.21.15.89:0/1213316072 shutdown_connections 2023-09-30T22:21:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.039+0000 7faa99c4e700 1 -- 172.21.15.89:0/1213316072 wait complete. 2023-09-30T22:21:30.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.040+0000 7faa99c4e700 1 Processor -- start 2023-09-30T22:21:30.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.040+0000 7faa99c4e700 1 -- start start 2023-09-30T22:21:30.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.040+0000 7faa99c4e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9406e570 0x7faa9413cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.040+0000 7faa99c4e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 0x7faa9413d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa99c4e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9412e1f0 0x7faa9413d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa99c4e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9412cbe0 con 0x7faa9412e1f0 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa99c4e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9413e030 con 0x7faa9406f050 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa99c4e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa9413e790 con 0x7faa9406e570 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9406e570 0x7faa9413cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9406e570 0x7faa9413cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59968/0 (socket says 172.21.15.89:59968) 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 -- 172.21.15.89:0/3763179073 learned_addr learned my addr 172.21.15.89:0/3763179073 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa93fff700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9412e1f0 0x7faa9413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa92ffd700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 0x7faa9413d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 -- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 msgr2=0x7faa9413d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 0x7faa9413d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 -- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9412e1f0 msgr2=0x7faa9413d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9412e1f0 0x7faa9413d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 -- 172.21.15.89:0/3763179073 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88009550 con 0x7faa9406e570 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa937fe700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9406e570 0x7faa9413cb80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faa8400c4d0 tx=0x7faa8400c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.041+0000 7faa7bfff700 1 -- 172.21.15.89:0/3763179073 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84011790 con 0x7faa9406e570 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.042+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9413ec60 con 0x7faa9406e570 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.042+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9412da10 con 0x7faa9406e570 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.042+0000 7faa7bfff700 1 -- 172.21.15.89:0/3763179073 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa840118f0 con 0x7faa9406e570 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.042+0000 7faa7bfff700 1 -- 172.21.15.89:0/3763179073 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84011ca0 con 0x7faa9406e570 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.043+0000 7faa7bfff700 1 -- 172.21.15.89:0/3763179073 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa8401c460 con 0x7faa9406e570 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.043+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa740052e0 con 0x7faa9406e570 2023-09-30T22:21:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.043+0000 7faa7bfff700 1 -- 172.21.15.89:0/3763179073 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa8403f450 con 0x7faa9406e570 2023-09-30T22:21:30.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.051+0000 7faa7bfff700 1 -- 172.21.15.89:0/3763179073 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa840108d0 con 0x7faa9406e570 2023-09-30T22:21:30.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.077+0000 7ff10f160700 1 Processor -- start 2023-09-30T22:21:30.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.077+0000 7ff10f160700 1 -- start start 2023-09-30T22:21:30.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.077+0000 7ff10f160700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1081316c0 0x7ff108131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.077+0000 7ff10f160700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.077+0000 7ff10f160700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10809e130 con 0x7ff10809d9f0 2023-09-30T22:21:30.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.077+0000 7ff10f160700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1081312b0 con 0x7ff1081316c0 2023-09-30T22:21:30.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.077+0000 7ff10f160700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10809f6e0 con 0x7ff10806ae80 2023-09-30T22:21:30.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.077+0000 7ff10d6fd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10cefc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1081316c0 0x7ff108131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10d6fd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33352/0 (socket says 172.21.15.89:33352) 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10d6fd700 1 -- 172.21.15.89:0/2842578527 learned_addr learned my addr 172.21.15.89:0/2842578527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10d6fd700 1 -- 172.21.15.89:0/2842578527 >> v2:172.21.15.89:3300/0 conn(0x7ff1081316c0 msgr2=0x7ff108131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10d6fd700 1 --2- 172.21.15.89:0/2842578527 >> v2:172.21.15.89:3300/0 conn(0x7ff1081316c0 0x7ff108131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10d6fd700 1 -- 172.21.15.89:0/2842578527 >> v1:172.21.15.89:6789/0 conn(0x7ff10809d9f0 legacy=0x7ff10806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10d6fd700 1 -- 172.21.15.89:0/2842578527 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10809e760 con 0x7ff10806ae80 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10cefc700 1 --2- 172.21.15.89:0/2842578527 >> v2:172.21.15.89:3300/0 conn(0x7ff1081316c0 0x7ff108131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff10d6fd700 1 --2- 172.21.15.89:0/2842578527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108139550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff100009e50 tx=0x7ff10000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f84321489d73893f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff1077fe700 1 -- 172.21.15.89:0/2842578527 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10000e070 con 0x7ff10806ae80 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff1077fe700 1 -- 172.21.15.89:0/2842578527 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10000b910 con 0x7ff10806ae80 2023-09-30T22:21:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.078+0000 7ff1077fe700 1 -- 172.21.15.89:0/2842578527 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10000bcc0 con 0x7ff10806ae80 2023-09-30T22:21:30.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.079+0000 7ff10f160700 1 -- 172.21.15.89:0/2842578527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 msgr2=0x7ff108139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.079+0000 7ff10f160700 1 --2- 172.21.15.89:0/2842578527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108139550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff100009e50 tx=0x7ff10000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.079+0000 7ff10f160700 1 -- 172.21.15.89:0/2842578527 shutdown_connections 2023-09-30T22:21:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.079+0000 7ff10f160700 1 --2- 172.21.15.89:0/2842578527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108139550 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.079+0000 7ff10f160700 1 --2- 172.21.15.89:0/2842578527 >> v2:172.21.15.89:3300/0 conn(0x7ff1081316c0 0x7ff108131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.079+0000 7ff10f160700 1 -- 172.21.15.89:0/2842578527 >> 172.21.15.89:0/2842578527 conn(0x7ff108066f50 msgr2=0x7ff10809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.079+0000 7ff10f160700 1 -- 172.21.15.89:0/2842578527 shutdown_connections 2023-09-30T22:21:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.079+0000 7ff10f160700 1 -- 172.21.15.89:0/2842578527 wait complete. 2023-09-30T22:21:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.081+0000 7ff10f160700 1 Processor -- start 2023-09-30T22:21:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.081+0000 7ff10f160700 1 -- start start 2023-09-30T22:21:30.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.081+0000 7ff10f160700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.081+0000 7ff10f160700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff10809d9f0 0x7ff108071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.081+0000 7ff10f160700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1081316c0 0x7ff1081c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10f160700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10812b900 con 0x7ff10809d9f0 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10f160700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10811a6c0 con 0x7ff10806ae80 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10f160700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff10811b020 con 0x7ff1081316c0 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33354/0 (socket says 172.21.15.89:33354) 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff107fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff10809d9f0 0x7ff108071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 -- 172.21.15.89:0/1055921998 learned_addr learned my addr 172.21.15.89:0/1055921998 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 -- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1081316c0 msgr2=0x7ff1081c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 --2- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1081316c0 0x7ff1081c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 -- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff10809d9f0 msgr2=0x7ff108071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 --2- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff10809d9f0 0x7ff108071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 -- 172.21.15.89:0/1055921998 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc0094a0 con 0x7ff10806ae80 2023-09-30T22:21:30.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10cefc700 1 --2- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108071860 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00ebe0 tx=0x7ff0fc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff1057fa700 1 -- 172.21.15.89:0/1055921998 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc009800 con 0x7ff10806ae80 2023-09-30T22:21:30.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff1057fa700 1 -- 172.21.15.89:0/1055921998 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00c910 con 0x7ff10806ae80 2023-09-30T22:21:30.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff100009b30 con 0x7ff10806ae80 2023-09-30T22:21:30.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff1057fa700 1 -- 172.21.15.89:0/1055921998 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00cc80 con 0x7ff10806ae80 2023-09-30T22:21:30.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.082+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1081196b0 con 0x7ff10806ae80 2023-09-30T22:21:30.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.083+0000 7ff1057fa700 1 -- 172.21.15.89:0/1055921998 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0fc013470 con 0x7ff10806ae80 2023-09-30T22:21:30.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.083+0000 7ff1057fa700 1 -- 172.21.15.89:0/1055921998 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff0fc03f830 con 0x7ff10806ae80 2023-09-30T22:21:30.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.084+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff108074ea0 con 0x7ff10806ae80 2023-09-30T22:21:30.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.088+0000 7ff1057fa700 1 -- 172.21.15.89:0/1055921998 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0fc017080 con 0x7ff10806ae80 2023-09-30T22:21:30.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.092+0000 7f03f3edd700 1 Processor -- start 2023-09-30T22:21:30.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.092+0000 7f03f3edd700 1 -- start start 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f3edd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03ec09d6f0 0x7f03ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f3edd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09e000 0x7f03ec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f3edd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec09b120 con 0x7f03ec09e410 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f3edd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec06b460 con 0x7f03ec09d6f0 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f3edd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec09a7e0 con 0x7f03ec09e000 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f1478700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09e000 0x7f03ec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f1478700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09e000 0x7f03ec1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:59996/0 (socket says 172.21.15.89:59996) 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f1478700 1 -- 172.21.15.89:0/918443431 learned_addr learned my addr 172.21.15.89:0/918443431 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:30.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f1c79700 1 --2- 172.21.15.89:0/918443431 >> v2:172.21.15.89:3300/0 conn(0x7f03ec09d6f0 0x7f03ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f1c79700 1 -- 172.21.15.89:0/918443431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09e000 msgr2=0x7f03ec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:30.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f1c79700 1 --2- 172.21.15.89:0/918443431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09e000 0x7f03ec1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f1c79700 1 -- 172.21.15.89:0/918443431 >> v1:172.21.15.89:6789/0 conn(0x7f03ec09e410 legacy=0x7f03ec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.093+0000 7f03f1c79700 1 -- 172.21.15.89:0/918443431 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec09a500 con 0x7f03ec09d6f0 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.094+0000 7f03f1c79700 1 --2- 172.21.15.89:0/918443431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09d6f0 0x7f03ec09dac0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009840 tx=0x7f03e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e66b41717e31a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.094+0000 7f03f0c77700 1 -- 172.21.15.89:0/918443431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8011830 con 0x7f03ec09d6f0 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.094+0000 7f03f0c77700 1 -- 172.21.15.89:0/918443431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e8011990 con 0x7f03ec09d6f0 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.094+0000 7f03f0c77700 1 -- 172.21.15.89:0/918443431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8011d20 con 0x7f03ec09d6f0 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.094+0000 7f03f3edd700 1 -- 172.21.15.89:0/918443431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09d6f0 msgr2=0x7f03ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.094+0000 7f03f3edd700 1 --2- 172.21.15.89:0/918443431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09d6f0 0x7f03ec09dac0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009840 tx=0x7f03e8009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.095+0000 7f03f3edd700 1 -- 172.21.15.89:0/918443431 shutdown_connections 2023-09-30T22:21:30.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.095+0000 7f03f3edd700 1 --2- 172.21.15.89:0/918443431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09d6f0 0x7f03ec09dac0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.095+0000 7f03f3edd700 1 --2- 172.21.15.89:0/918443431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09e000 0x7f03ec1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.095+0000 7f03f3edd700 1 -- 172.21.15.89:0/918443431 >> 172.21.15.89:0/918443431 conn(0x7f03ec066f50 msgr2=0x7f03ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.095+0000 7f03f3edd700 1 -- 172.21.15.89:0/918443431 shutdown_connections 2023-09-30T22:21:30.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.095+0000 7f03f3edd700 1 -- 172.21.15.89:0/918443431 wait complete. 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.096+0000 7f03f3edd700 1 Processor -- start 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.096+0000 7f03f3edd700 1 -- start start 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f3edd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09d6f0 0x7f03ec1a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f3edd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09e000 0x7f03ec1a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f3edd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03ec09e410 0x7f03ec1a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f3edd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec1a7410 con 0x7f03ec09e000 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f3edd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec1a7b20 con 0x7f03ec09e410 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f3edd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec1a8d70 con 0x7f03ec09d6f0 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f247a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03ec09e410 0x7f03ec1a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1c79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09d6f0 0x7f03ec1a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f247a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03ec09e410 0x7f03ec1a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33358/0 (socket says 172.21.15.89:33358) 2023-09-30T22:21:30.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1c79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09d6f0 0x7f03ec1a5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60006/0 (socket says 172.21.15.89:60006) 2023-09-30T22:21:30.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f247a700 1 -- 172.21.15.89:0/1810460683 learned_addr learned my addr 172.21.15.89:0/1810460683 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:30.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1478700 1 --2- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09e000 0x7f03ec1a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1c79700 1 -- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03ec09e410 msgr2=0x7f03ec1a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1c79700 1 --2- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03ec09e410 0x7f03ec1a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1c79700 1 -- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09e000 msgr2=0x7f03ec1a64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1c79700 1 --2- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09e000 0x7f03ec1a64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1c79700 1 -- 172.21.15.89:0/1810460683 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80094a0 con 0x7f03ec09d6f0 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.097+0000 7f03f1c79700 1 --2- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09d6f0 0x7f03ec1a5f60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009840 tx=0x7f03e8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.098+0000 7f03e27fc700 1 -- 172.21.15.89:0/1810460683 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e80036a0 con 0x7f03ec09d6f0 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.098+0000 7f03e27fc700 1 -- 172.21.15.89:0/1810460683 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e8026070 con 0x7f03ec09d6f0 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.098+0000 7f03e27fc700 1 -- 172.21.15.89:0/1810460683 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8003a20 con 0x7f03ec09d6f0 2023-09-30T22:21:30.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.098+0000 7f03f3edd700 1 -- 172.21.15.89:0/1810460683 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec1a9750 con 0x7f03ec09d6f0 2023-09-30T22:21:30.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.098+0000 7f03f3edd700 1 -- 172.21.15.89:0/1810460683 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec0a2260 con 0x7f03ec09d6f0 2023-09-30T22:21:30.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.099+0000 7f03cffff700 1 -- 172.21.15.89:0/1810460683 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c80052e0 con 0x7f03ec09d6f0 2023-09-30T22:21:30.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.100+0000 7f03e27fc700 1 -- 172.21.15.89:0/1810460683 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03e8003b80 con 0x7f03ec09d6f0 2023-09-30T22:21:30.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.100+0000 7f03e27fc700 1 -- 172.21.15.89:0/1810460683 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f03e80495e0 con 0x7f03ec09d6f0 2023-09-30T22:21:30.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.108+0000 7f03e27fc700 1 -- 172.21.15.89:0/1810460683 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03e8035450 con 0x7f03ec09d6f0 2023-09-30T22:21:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.132+0000 7ff56239d700 1 Processor -- start 2023-09-30T22:21:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.132+0000 7ff56239d700 1 -- start start 2023-09-30T22:21:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff56239d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c09bc50 0x7ff55c09c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff56239d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09c560 0x7ff55c131470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff56239d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55c06b0a0 con 0x7ff55c1319b0 2023-09-30T22:21:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff56239d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c06b2e0 con 0x7ff55c09c560 2023-09-30T22:21:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff56239d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c06e590 con 0x7ff55c09bc50 2023-09-30T22:21:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff561b9c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff55c1319b0 0x7ff55c138940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45940/0 (socket says 172.21.15.89:45940) 2023-09-30T22:21:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff561b9c700 1 -- 172.21.15.89:0/796224023 learned_addr learned my addr 172.21.15.89:0/796224023 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff560b9a700 1 --2- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09c560 0x7ff55c131470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff56139b700 1 --2- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c09bc50 0x7ff55c09c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff553fff700 1 -- 172.21.15.89:0/796224023 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483480438 0 0) 0x7ff55c06b0a0 con 0x7ff55c1319b0 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff553fff700 1 -- 172.21.15.89:0/796224023 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5480079c0 con 0x7ff55c1319b0 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff560b9a700 1 -- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c09bc50 msgr2=0x7ff55c09c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff560b9a700 1 --2- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c09bc50 0x7ff55c09c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff560b9a700 1 -- 172.21.15.89:0/796224023 >> v1:172.21.15.89:6789/0 conn(0x7ff55c1319b0 legacy=0x7ff55c138940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.133+0000 7ff560b9a700 1 -- 172.21.15.89:0/796224023 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c09f370 con 0x7ff55c09c560 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.134+0000 7ff560b9a700 1 --2- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09c560 0x7ff55c131470 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009850 tx=0x7ff54c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b850574fe3e9f760 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.134+0000 7ff553fff700 1 -- 172.21.15.89:0/796224023 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c005e30 con 0x7ff55c09c560 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.134+0000 7ff553fff700 1 -- 172.21.15.89:0/796224023 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c006030 con 0x7ff55c09c560 2023-09-30T22:21:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.134+0000 7ff553fff700 1 -- 172.21.15.89:0/796224023 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0063e0 con 0x7ff55c09c560 2023-09-30T22:21:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.134+0000 7ff56239d700 1 -- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09c560 msgr2=0x7ff55c131470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.134+0000 7ff56239d700 1 --2- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09c560 0x7ff55c131470 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009850 tx=0x7ff54c009b60 comp rx=0 tx=0).stop 2023-09-30T22:21:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.135+0000 7ff56239d700 1 -- 172.21.15.89:0/796224023 shutdown_connections 2023-09-30T22:21:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.135+0000 7ff56239d700 1 --2- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c09bc50 0x7ff55c09c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.135+0000 7ff56239d700 1 --2- 172.21.15.89:0/796224023 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09c560 0x7ff55c131470 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.135+0000 7ff56239d700 1 -- 172.21.15.89:0/796224023 >> 172.21.15.89:0/796224023 conn(0x7ff55c0a2e20 msgr2=0x7ff55c0a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.135+0000 7ff56239d700 1 -- 172.21.15.89:0/796224023 shutdown_connections 2023-09-30T22:21:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.135+0000 7ff56239d700 1 -- 172.21.15.89:0/796224023 wait complete. 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.136+0000 7ff56239d700 1 Processor -- start 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.136+0000 7ff56239d700 1 -- start start 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56239d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09bc50 0x7ff55c1192f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56239d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff55c09c560 0x7ff55c119830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56239d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c1319b0 0x7ff55c119d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56239d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c071e00 con 0x7ff55c09c560 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56239d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c11c480 con 0x7ff55c09bc50 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56239d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c11d020 con 0x7ff55c1319b0 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff560b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff55c09c560 0x7ff55c119830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56139b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09bc50 0x7ff55c1192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff560b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff55c09c560 0x7ff55c119830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49926/0 (socket says 172.21.15.89:49926) 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff560b9a700 1 -- 172.21.15.89:0/1942846960 learned_addr learned my addr 172.21.15.89:0/1942846960 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56139b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09bc50 0x7ff55c1192f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33382/0 (socket says 172.21.15.89:33382) 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff561b9c700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c1319b0 0x7ff55c119d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56139b700 1 -- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c1319b0 msgr2=0x7ff55c119d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56139b700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c1319b0 0x7ff55c119d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56139b700 1 -- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff55c09c560 msgr2=0x7ff55c119830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56139b700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff55c09c560 0x7ff55c119830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56139b700 1 -- 172.21.15.89:0/1942846960 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c009430 con 0x7ff55c09bc50 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff560b9a700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff55c09c560 0x7ff55c119830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff56139b700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09bc50 0x7ff55c1192f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff55800e870 tx=0x7ff55800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.137+0000 7ff551ffb700 1 -- 172.21.15.89:0/1942846960 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558013680 con 0x7ff55c09bc50 2023-09-30T22:21:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.138+0000 7ff551ffb700 1 -- 172.21.15.89:0/1942846960 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5580137e0 con 0x7ff55c09bc50 2023-09-30T22:21:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.138+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c11c680 con 0x7ff55c09bc50 2023-09-30T22:21:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.138+0000 7ff551ffb700 1 -- 172.21.15.89:0/1942846960 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558013b90 con 0x7ff55c09bc50 2023-09-30T22:21:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.138+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c11b500 con 0x7ff55c09bc50 2023-09-30T22:21:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.139+0000 7ff551ffb700 1 -- 172.21.15.89:0/1942846960 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff558013cf0 con 0x7ff55c09bc50 2023-09-30T22:21:30.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.139+0000 7ff551ffb700 1 -- 172.21.15.89:0/1942846960 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff55803f5d0 con 0x7ff55c09bc50 2023-09-30T22:21:30.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.139+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55c04f2a0 con 0x7ff55c09bc50 2023-09-30T22:21:30.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.144+0000 7ff551ffb700 1 -- 172.21.15.89:0/1942846960 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff558017080 con 0x7ff55c09bc50 2023-09-30T22:21:30.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.285+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff10811b470 con 0x7ff10806ae80 2023-09-30T22:21:30.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.285+0000 7ff1057fa700 1 -- 172.21.15.89:0/1055921998 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0fc021020 con 0x7ff10806ae80 2023-09-30T22:21:30.287 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.288+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 msgr2=0x7ff108071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.288+0000 7ff10f160700 1 --2- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108071860 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00ebe0 tx=0x7ff0fc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.289+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 shutdown_connections 2023-09-30T22:21:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.289+0000 7ff10f160700 1 --2- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1081316c0 0x7ff1081c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.289+0000 7ff10f160700 1 --2- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff10806ae80 0x7ff108071860 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.289+0000 7ff10f160700 1 --2- 172.21.15.89:0/1055921998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff10809d9f0 0x7ff108071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.289+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 >> 172.21.15.89:0/1055921998 conn(0x7ff108066f50 msgr2=0x7ff1080a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.289+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 shutdown_connections 2023-09-30T22:21:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.289+0000 7ff10f160700 1 -- 172.21.15.89:0/1055921998 wait complete. 2023-09-30T22:21:30.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.299+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa74005590 con 0x7faa9406e570 2023-09-30T22:21:30.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.300+0000 7faa7bfff700 1 -- 172.21.15.89:0/3763179073 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa840102e0 con 0x7faa9406e570 2023-09-30T22:21:30.301 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9406e570 msgr2=0x7faa9413cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9406e570 0x7faa9413cb80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faa8400c4d0 tx=0x7faa8400c7e0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 shutdown_connections 2023-09-30T22:21:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9412e1f0 0x7faa9413d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9406e570 0x7faa9413cb80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 --2- 172.21.15.89:0/3763179073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9406f050 0x7faa9413d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 >> 172.21.15.89:0/3763179073 conn(0x7faa940a2aa0 msgr2=0x7faa9409cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 shutdown_connections 2023-09-30T22:21:30.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.303+0000 7faa99c4e700 1 -- 172.21.15.89:0/3763179073 wait complete. 2023-09-30T22:21:30.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:30.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:30.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.333+0000 7f03cffff700 1 -- 172.21.15.89:0/1810460683 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f03c8005860 con 0x7f03ec09d6f0 2023-09-30T22:21:30.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.333+0000 7f03e27fc700 1 -- 172.21.15.89:0/1810460683 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03e8034820 con 0x7f03ec09d6f0 2023-09-30T22:21:30.334 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:30.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 -- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09d6f0 msgr2=0x7f03ec1a5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 --2- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09d6f0 0x7f03ec1a5f60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009840 tx=0x7f03e8004750 comp rx=0 tx=0).stop 2023-09-30T22:21:30.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 -- 172.21.15.89:0/1810460683 shutdown_connections 2023-09-30T22:21:30.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 --2- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03ec09e410 0x7f03ec1a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 --2- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03ec09d6f0 0x7f03ec1a5f60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 --2- 172.21.15.89:0/1810460683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03ec09e000 0x7f03ec1a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 -- 172.21.15.89:0/1810460683 >> 172.21.15.89:0/1810460683 conn(0x7f03ec066f50 msgr2=0x7f03ec1baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 -- 172.21.15.89:0/1810460683 shutdown_connections 2023-09-30T22:21:30.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.335+0000 7f03cffff700 1 -- 172.21.15.89:0/1810460683 wait complete. 2023-09-30T22:21:30.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.339+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff55c11c270 con 0x7ff55c09bc50 2023-09-30T22:21:30.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.339+0000 7ff551ffb700 1 -- 172.21.15.89:0/1942846960 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff55801be10 con 0x7ff55c09bc50 2023-09-30T22:21:30.340 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09bc50 msgr2=0x7ff55c1192f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09bc50 0x7ff55c1192f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff55800e870 tx=0x7ff55800eb80 comp rx=0 tx=0).stop 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 shutdown_connections 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff55c1319b0 0x7ff55c119d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff55c09bc50 0x7ff55c1192f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 --2- 172.21.15.89:0/1942846960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff55c09c560 0x7ff55c119830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 >> 172.21.15.89:0/1942846960 conn(0x7ff55c0a2e20 msgr2=0x7ff55c12db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 shutdown_connections 2023-09-30T22:21:30.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:30.342+0000 7ff56239d700 1 -- 172.21.15.89:0/1942846960 wait complete. 2023-09-30T22:21:30.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:30.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:31.306 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:31.317 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:31.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:31.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.450+0000 7f97c5f44700 1 Processor -- start 2023-09-30T22:21:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.450+0000 7f97c5f44700 1 -- start start 2023-09-30T22:21:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97c5f44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c006a770 0x7f97c009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97c5f44700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97c009dc30 0x7f97c009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97c5f44700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c013e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97c5f44700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c006ab40 con 0x7f97c009dc30 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97c5f44700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c009ebc0 con 0x7f97c013c1e0 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97c5f44700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97c009a810 con 0x7f97c006a770 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97beffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97c009dc30 0x7f97c009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97bffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c013e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c006a770 0x7f97c009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97beffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97c009dc30 0x7f97c009e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52134/0 (socket says 172.21.15.89:52134) 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97bffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c013e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51926/0 (socket says 172.21.15.89:51926) 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.451+0000 7f97beffd700 1 -- 172.21.15.89:0/649914174 learned_addr learned my addr 172.21.15.89:0/649914174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97bf7fe700 1 -- 172.21.15.89:0/649914174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 msgr2=0x7f97c013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97bf7fe700 1 --2- 172.21.15.89:0/649914174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c013e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97bf7fe700 1 -- 172.21.15.89:0/649914174 >> v2:172.21.15.89:3300/0 conn(0x7f97c009dc30 msgr2=0x7f97c009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97bf7fe700 1 --2- 172.21.15.89:0/649914174 >> v2:172.21.15.89:3300/0 conn(0x7f97c009dc30 0x7f97c009e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97bf7fe700 1 -- 172.21.15.89:0/649914174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c009e920 con 0x7f97c006a770 2023-09-30T22:21:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97bf7fe700 1 --2- 172.21.15.89:0/649914174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c006a770 0x7f97c009d6f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009dc0 tx=0x7f97b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811a91c66a48c534 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97be7fc700 1 -- 172.21.15.89:0/649914174 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000e070 con 0x7f97c006a770 2023-09-30T22:21:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97be7fc700 1 -- 172.21.15.89:0/649914174 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b000bae0 con 0x7f97c006a770 2023-09-30T22:21:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.452+0000 7f97be7fc700 1 -- 172.21.15.89:0/649914174 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000be90 con 0x7f97c006a770 2023-09-30T22:21:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 -- 172.21.15.89:0/649914174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c006a770 msgr2=0x7f97c009d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 --2- 172.21.15.89:0/649914174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c006a770 0x7f97c009d6f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009dc0 tx=0x7f97b000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 -- 172.21.15.89:0/649914174 shutdown_connections 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 --2- 172.21.15.89:0/649914174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c013e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 --2- 172.21.15.89:0/649914174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c006a770 0x7f97c009d6f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 --2- 172.21.15.89:0/649914174 >> v2:172.21.15.89:3300/0 conn(0x7f97c009dc30 0x7f97c009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 -- 172.21.15.89:0/649914174 >> 172.21.15.89:0/649914174 conn(0x7f97c0066f50 msgr2=0x7f97c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 -- 172.21.15.89:0/649914174 shutdown_connections 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.453+0000 7f97c5f44700 1 -- 172.21.15.89:0/649914174 wait complete. 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.454+0000 7f97c5f44700 1 Processor -- start 2023-09-30T22:21:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97c5f44700 1 -- start start 2023-09-30T22:21:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97c5f44700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97c006a770 0x7f97c0071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97c5f44700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c009dc30 0x7f97c012dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97c5f44700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c012e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97c5f44700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c012ff10 con 0x7f97c006a770 2023-09-30T22:21:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97c5f44700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c012fd20 con 0x7f97c013c1e0 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97c5f44700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97c0130800 con 0x7f97c009dc30 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97c006a770 0x7f97c0071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97c006a770 0x7f97c0071fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52136/0 (socket says 172.21.15.89:52136) 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97bf7fe700 1 -- 172.21.15.89:0/715494832 learned_addr learned my addr 172.21.15.89:0/715494832 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97beffd700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c009dc30 0x7f97c012dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.455+0000 7f97bffff700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c012e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97bf7fe700 1 -- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c009dc30 msgr2=0x7f97c012dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97bf7fe700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c009dc30 0x7f97c012dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97bf7fe700 1 -- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 msgr2=0x7f97c012e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97bf7fe700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c012e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97bf7fe700 1 -- 172.21.15.89:0/715494832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b0009a70 con 0x7f97c006a770 2023-09-30T22:21:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97bffff700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c012e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97bf7fe700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97c006a770 0x7f97c0071fa0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f97b000b370 tx=0x7f97b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97b7fff700 1 -- 172.21.15.89:0/715494832 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b00045d0 con 0x7f97c006a770 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97b7fff700 1 -- 172.21.15.89:0/715494832 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b0020070 con 0x7f97c006a770 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97b7fff700 1 -- 172.21.15.89:0/715494832 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0027480 con 0x7f97c006a770 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c012f4e0 con 0x7f97c006a770 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.456+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c0131950 con 0x7f97c006a770 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.457+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c011cc30 con 0x7f97c006a770 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.457+0000 7f97b7fff700 1 -- 172.21.15.89:0/715494832 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f97b0012430 con 0x7f97c006a770 2023-09-30T22:21:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.458+0000 7f97b7fff700 1 -- 172.21.15.89:0/715494832 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f97b0022070 con 0x7f97c006a770 2023-09-30T22:21:31.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.462+0000 7f97b7fff700 1 -- 172.21.15.89:0/715494832 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97b0025380 con 0x7f97c006a770 2023-09-30T22:21:31.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.495+0000 7fc863ef2700 1 Processor -- start 2023-09-30T22:21:31.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.495+0000 7f47fa96e700 1 Processor -- start 2023-09-30T22:21:31.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.495+0000 7fc863ef2700 1 -- start start 2023-09-30T22:21:31.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc863ef2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c0a09c0 0x7fc85c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc863ef2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc85c0a12d0 0x7fc85c134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc863ef2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85c09ea10 con 0x7fc85c09d5c0 2023-09-30T22:21:31.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc863ef2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c09e210 con 0x7fc85c0a12d0 2023-09-30T22:21:31.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc863ef2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0a26f0 con 0x7fc85c0a09c0 2023-09-30T22:21:31.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.495+0000 7f47fa96e700 1 -- start start 2023-09-30T22:21:31.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc861c8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c0a09c0 0x7fc85c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc861c8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c0a09c0 0x7fc85c0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48752/0 (socket says 172.21.15.89:48752) 2023-09-30T22:21:31.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc861c8e700 1 -- 172.21.15.89:0/1246828990 learned_addr learned my addr 172.21.15.89:0/1246828990 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:31.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7f47fa96e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409e930 0x7f47f409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7f47fa96e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f409f650 0x7f47f4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7f47fa96e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f4070940 con 0x7f47f409f240 2023-09-30T22:21:31.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7f47fa96e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f406fd60 con 0x7f47f409f650 2023-09-30T22:21:31.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7f47fa96e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47f406fa80 con 0x7f47f409e930 2023-09-30T22:21:31.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc861c8e700 1 -- 172.21.15.89:0/1246828990 >> v2:172.21.15.89:3300/0 conn(0x7fc85c0a12d0 msgr2=0x7fc85c134030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:21:31.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc861c8e700 1 --2- 172.21.15.89:0/1246828990 >> v2:172.21.15.89:3300/0 conn(0x7fc85c0a12d0 0x7fc85c134030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc861c8e700 1 -- 172.21.15.89:0/1246828990 >> v1:172.21.15.89:6789/0 conn(0x7fc85c09d5c0 legacy=0x7fc85c133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:31.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7f47f3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409e930 0x7f47f409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7fc861c8e700 1 -- 172.21.15.89:0/1246828990 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c09db70 con 0x7fc85c0a09c0 2023-09-30T22:21:31.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7f47f3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409e930 0x7f47f409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48762/0 (socket says 172.21.15.89:48762) 2023-09-30T22:21:31.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.496+0000 7f47f3fff700 1 -- 172.21.15.89:0/717293439 learned_addr learned my addr 172.21.15.89:0/717293439 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:31.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7fc861c8e700 1 --2- 172.21.15.89:0/1246828990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c0a09c0 0x7fc85c0a0d90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc8580098f0 tx=0x7fc858009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24b34d44dc5f0c3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7f47f3fff700 1 -- 172.21.15.89:0/717293439 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f409f650 msgr2=0x7f47f4131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7f47f3fff700 1 --2- 172.21.15.89:0/717293439 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f409f650 0x7f47f4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7f47f3fff700 1 -- 172.21.15.89:0/717293439 >> v1:172.21.15.89:6789/0 conn(0x7f47f409f240 legacy=0x7f47f41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7f47f3fff700 1 -- 172.21.15.89:0/717293439 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f409fb60 con 0x7f47f409e930 2023-09-30T22:21:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7fc860c8c700 1 -- 172.21.15.89:0/1246828990 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858004230 con 0x7fc85c0a09c0 2023-09-30T22:21:31.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7fc860c8c700 1 -- 172.21.15.89:0/1246828990 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858004390 con 0x7fc85c0a09c0 2023-09-30T22:21:31.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7fc860c8c700 1 -- 172.21.15.89:0/1246828990 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8580036a0 con 0x7fc85c0a09c0 2023-09-30T22:21:31.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7f47f3fff700 1 --2- 172.21.15.89:0/717293439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409e930 0x7f47f409ed00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009920 tx=0x7f47e4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9166c6379f50a4e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7f47f2ffd700 1 -- 172.21.15.89:0/717293439 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4004230 con 0x7f47f409e930 2023-09-30T22:21:31.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7f47f2ffd700 1 -- 172.21.15.89:0/717293439 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e4004390 con 0x7f47f409e930 2023-09-30T22:21:31.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.497+0000 7f47f2ffd700 1 -- 172.21.15.89:0/717293439 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4009e70 con 0x7f47f409e930 2023-09-30T22:21:31.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7fc863ef2700 1 -- 172.21.15.89:0/1246828990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c0a09c0 msgr2=0x7fc85c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7fc863ef2700 1 --2- 172.21.15.89:0/1246828990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c0a09c0 0x7fc85c0a0d90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc8580098f0 tx=0x7fc858009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:31.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7fc863ef2700 1 -- 172.21.15.89:0/1246828990 shutdown_connections 2023-09-30T22:21:31.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7fc863ef2700 1 --2- 172.21.15.89:0/1246828990 >> v2:172.21.15.89:3300/0 conn(0x7fc85c0a12d0 0x7fc85c134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7fc863ef2700 1 --2- 172.21.15.89:0/1246828990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c0a09c0 0x7fc85c0a0d90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7fc863ef2700 1 -- 172.21.15.89:0/1246828990 >> 172.21.15.89:0/1246828990 conn(0x7fc85c0a2fe0 msgr2=0x7fc85c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7fc863ef2700 1 -- 172.21.15.89:0/1246828990 shutdown_connections 2023-09-30T22:21:31.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7f47fa96e700 1 -- 172.21.15.89:0/717293439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409e930 msgr2=0x7f47f409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7f47fa96e700 1 --2- 172.21.15.89:0/717293439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409e930 0x7f47f409ed00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009920 tx=0x7f47e4009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:31.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7f47fa96e700 1 -- 172.21.15.89:0/717293439 shutdown_connections 2023-09-30T22:21:31.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7f47fa96e700 1 --2- 172.21.15.89:0/717293439 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f409f650 0x7f47f4131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7f47fa96e700 1 --2- 172.21.15.89:0/717293439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409e930 0x7f47f409ed00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7f47fa96e700 1 -- 172.21.15.89:0/717293439 >> 172.21.15.89:0/717293439 conn(0x7f47f40a4210 msgr2=0x7f47f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7f47fa96e700 1 -- 172.21.15.89:0/717293439 shutdown_connections 2023-09-30T22:21:31.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7f47fa96e700 1 -- 172.21.15.89:0/717293439 wait complete. 2023-09-30T22:21:31.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.498+0000 7fc863ef2700 1 -- 172.21.15.89:0/1246828990 wait complete. 2023-09-30T22:21:31.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.499+0000 7fc863ef2700 1 Processor -- start 2023-09-30T22:21:31.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.499+0000 7f47fa96e700 1 Processor -- start 2023-09-30T22:21:31.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc863ef2700 1 -- start start 2023-09-30T22:21:31.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc863ef2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c09d5c0 0x7fc85c1d6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc863ef2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85c0a09c0 0x7fc85c1d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc863ef2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85c0a12d0 0x7fc85c1daf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc863ef2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0a5380 con 0x7fc85c0a12d0 2023-09-30T22:21:31.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc863ef2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c09cec0 con 0x7fc85c0a09c0 2023-09-30T22:21:31.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc863ef2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1324d0 con 0x7fc85c09d5c0 2023-09-30T22:21:31.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47fa96e700 1 -- start start 2023-09-30T22:21:31.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47fa96e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f409e930 0x7f47f41a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47fa96e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f409f240 0x7f47f41a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47fa96e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409f650 0x7f47f41a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47fa96e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41b9330 con 0x7f47f409f240 2023-09-30T22:21:31.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47fa96e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41a8170 con 0x7f47f409e930 2023-09-30T22:21:31.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc861c8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c09d5c0 0x7fc85c1d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47fa96e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41a8da0 con 0x7f47f409f650 2023-09-30T22:21:31.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47f8f0b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409f650 0x7f47f41a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47f8f0b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409f650 0x7f47f41a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48768/0 (socket says 172.21.15.89:48768) 2023-09-30T22:21:31.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7f47f8f0b700 1 -- 172.21.15.89:0/1706693645 learned_addr learned my addr 172.21.15.89:0/1706693645 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:31.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc861c8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c09d5c0 0x7fc85c1d6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48764/0 (socket says 172.21.15.89:48764) 2023-09-30T22:21:31.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.500+0000 7fc861c8e700 1 -- 172.21.15.89:0/2528502931 learned_addr learned my addr 172.21.15.89:0/2528502931 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:31.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f8f0b700 1 -- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f409e930 msgr2=0x7f47f41a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f8f0b700 1 --2- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f409e930 0x7f47f41a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f8f0b700 1 -- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f409f240 msgr2=0x7f47f41a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f8f0b700 1 --2- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f409f240 0x7f47f41a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f8f0b700 1 -- 172.21.15.89:0/1706693645 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e4009550 con 0x7f47f409f650 2023-09-30T22:21:31.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc861c8e700 1 -- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85c0a09c0 msgr2=0x7fc85c1d89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc861c8e700 1 --2- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85c0a09c0 0x7fc85c1d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc861c8e700 1 -- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85c0a12d0 msgr2=0x7fc85c1daf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc861c8e700 1 --2- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85c0a12d0 0x7fc85c1daf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc861c8e700 1 -- 172.21.15.89:0/2528502931 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858009550 con 0x7fc85c09d5c0 2023-09-30T22:21:31.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f8f0b700 1 --2- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409f650 0x7f47f41a69c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00b9e0 tx=0x7f47ec00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc861c8e700 1 --2- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c09d5c0 0x7fc85c1d6490 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc858000c00 tx=0x7fc858004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f0ff9700 1 -- 172.21.15.89:0/1706693645 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00c9b0 con 0x7f47f409f650 2023-09-30T22:21:31.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f0ff9700 1 -- 172.21.15.89:0/1706693645 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec00cb50 con 0x7f47f409f650 2023-09-30T22:21:31.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47f0ff9700 1 -- 172.21.15.89:0/1706693645 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec014440 con 0x7f47f409f650 2023-09-30T22:21:31.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f40a0190 con 0x7f47f409f650 2023-09-30T22:21:31.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f41a9240 con 0x7f47f409f650 2023-09-30T22:21:31.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc8527fc700 1 -- 172.21.15.89:0/2528502931 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85800b660 con 0x7fc85c09d5c0 2023-09-30T22:21:31.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc8527fc700 1 -- 172.21.15.89:0/2528502931 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85800b7c0 con 0x7fc85c09d5c0 2023-09-30T22:21:31.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc8527fc700 1 -- 172.21.15.89:0/2528502931 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85800bb50 con 0x7fc85c09d5c0 2023-09-30T22:21:31.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c09cd10 con 0x7fc85c09d5c0 2023-09-30T22:21:31.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.501+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c133200 con 0x7fc85c09d5c0 2023-09-30T22:21:31.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.502+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f41a2b40 con 0x7f47f409f650 2023-09-30T22:21:31.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.503+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8440052e0 con 0x7fc85c09d5c0 2023-09-30T22:21:31.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.503+0000 7f47f0ff9700 1 -- 172.21.15.89:0/1706693645 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f47ec0145a0 con 0x7f47f409f650 2023-09-30T22:21:31.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.503+0000 7fc8527fc700 1 -- 172.21.15.89:0/2528502931 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc85800bcb0 con 0x7fc85c09d5c0 2023-09-30T22:21:31.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.504+0000 7fc8527fc700 1 -- 172.21.15.89:0/2528502931 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc8580411a0 con 0x7fc85c09d5c0 2023-09-30T22:21:31.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.505+0000 7f47f0ff9700 1 -- 172.21.15.89:0/1706693645 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f47ec006e40 con 0x7f47f409f650 2023-09-30T22:21:31.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.508+0000 7fc8527fc700 1 -- 172.21.15.89:0/2528502931 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc85801a460 con 0x7fc85c09d5c0 2023-09-30T22:21:31.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.515+0000 7f47f0ff9700 1 -- 172.21.15.89:0/1706693645 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f47ec01d3b0 con 0x7f47f409f650 2023-09-30T22:21:31.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.540+0000 7fcc157df700 1 Processor -- start 2023-09-30T22:21:31.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.540+0000 7fcc157df700 1 -- start start 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc157df700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc10133540 0x7fcc10135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc157df700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10135e60 0x7fcc10138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc157df700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc100716a0 con 0x7fcc10130540 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc157df700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc10071290 con 0x7fcc10133540 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc157df700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10130340 con 0x7fcc10135e60 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc0e7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc10133540 0x7fcc10135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc0e7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc10133540 0x7fcc10135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52154/0 (socket says 172.21.15.89:52154) 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc0e7fc700 1 -- 172.21.15.89:0/3060640649 learned_addr learned my addr 172.21.15.89:0/3060640649 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc0f7fe700 1 --2- 172.21.15.89:0/3060640649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10135e60 0x7fcc10138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc0e7fc700 1 -- 172.21.15.89:0/3060640649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10135e60 msgr2=0x7fcc10138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:31.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc0e7fc700 1 --2- 172.21.15.89:0/3060640649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10135e60 0x7fcc10138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc0e7fc700 1 -- 172.21.15.89:0/3060640649 >> v1:172.21.15.89:6789/0 conn(0x7fcc10130540 legacy=0x7fcc10130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.541+0000 7fcc0e7fc700 1 -- 172.21.15.89:0/3060640649 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10070b80 con 0x7fcc10133540 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.542+0000 7fcc0e7fc700 1 --2- 172.21.15.89:0/3060640649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10133540 0x7fcc10135920 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcc040098f0 tx=0x7fcc04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235716ae47c83fd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.542+0000 7fcc0dffb700 1 -- 172.21.15.89:0/3060640649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04011830 con 0x7fcc10133540 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.542+0000 7fcc0dffb700 1 -- 172.21.15.89:0/3060640649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04011990 con 0x7fcc10133540 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.542+0000 7fcc0dffb700 1 -- 172.21.15.89:0/3060640649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04011d20 con 0x7fcc10133540 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.543+0000 7fcc157df700 1 -- 172.21.15.89:0/3060640649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10133540 msgr2=0x7fcc10135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.543+0000 7fcc157df700 1 --2- 172.21.15.89:0/3060640649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10133540 0x7fcc10135920 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcc040098f0 tx=0x7fcc04009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.543+0000 7fcc157df700 1 -- 172.21.15.89:0/3060640649 shutdown_connections 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.543+0000 7fcc157df700 1 --2- 172.21.15.89:0/3060640649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10135e60 0x7fcc10138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.543+0000 7fcc157df700 1 --2- 172.21.15.89:0/3060640649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10133540 0x7fcc10135920 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.543+0000 7fcc157df700 1 -- 172.21.15.89:0/3060640649 >> 172.21.15.89:0/3060640649 conn(0x7fcc100a2aa0 msgr2=0x7fcc100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.543+0000 7fcc157df700 1 -- 172.21.15.89:0/3060640649 shutdown_connections 2023-09-30T22:21:31.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.543+0000 7fcc157df700 1 -- 172.21.15.89:0/3060640649 wait complete. 2023-09-30T22:21:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.544+0000 7fcc157df700 1 Processor -- start 2023-09-30T22:21:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.544+0000 7fcc157df700 1 -- start start 2023-09-30T22:21:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc157df700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10130540 0x7fcc101c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc157df700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc10133540 0x7fcc101c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc157df700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10135e60 0x7fcc1009a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc157df700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1009cf20 con 0x7fcc10135e60 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc157df700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc101c4f10 con 0x7fcc10133540 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc157df700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1012b9d0 con 0x7fcc10130540 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc0effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10130540 0x7fcc101c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc0effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10130540 0x7fcc101c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48788/0 (socket says 172.21.15.89:48788) 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc0f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10135e60 0x7fcc1009a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc0effd700 1 -- 172.21.15.89:0/3101062245 learned_addr learned my addr 172.21.15.89:0/3101062245 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.545+0000 7fcc0e7fc700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc10133540 0x7fcc101c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc0effd700 1 -- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc10133540 msgr2=0x7fcc101c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc0effd700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc10133540 0x7fcc101c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc0effd700 1 -- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10135e60 msgr2=0x7fcc1009a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc0effd700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10135e60 0x7fcc1009a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc0effd700 1 -- 172.21.15.89:0/3101062245 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04009550 con 0x7fcc10130540 2023-09-30T22:21:31.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc0f7fe700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10135e60 0x7fcc1009a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc0effd700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10130540 0x7fcc101c4490 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c480 tx=0x7fcc0000c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcbf77fe700 1 -- 172.21.15.89:0/3101062245 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000052d0 con 0x7fcc10130540 2023-09-30T22:21:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcbf77fe700 1 -- 172.21.15.89:0/3101062245 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00005430 con 0x7fcc10130540 2023-09-30T22:21:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcbf77fe700 1 -- 172.21.15.89:0/3101062245 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000038a0 con 0x7fcc10130540 2023-09-30T22:21:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc101baf40 con 0x7fcc10130540 2023-09-30T22:21:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.546+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc100718e0 con 0x7fcc10130540 2023-09-30T22:21:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.547+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc10074ea0 con 0x7fcc10130540 2023-09-30T22:21:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.547+0000 7fcbf77fe700 1 -- 172.21.15.89:0/3101062245 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc00003a00 con 0x7fcc10130540 2023-09-30T22:21:31.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.548+0000 7fcbf77fe700 1 -- 172.21.15.89:0/3101062245 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcc0003e9b0 con 0x7fcc10130540 2023-09-30T22:21:31.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.556+0000 7fcbf77fe700 1 -- 172.21.15.89:0/3101062245 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc00003cb0 con 0x7fcc10130540 2023-09-30T22:21:31.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.665+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f97c012f4e0 con 0x7f97c006a770 2023-09-30T22:21:31.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.666+0000 7f97b7fff700 1 -- 172.21.15.89:0/715494832 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97c012f4e0 con 0x7f97c006a770 2023-09-30T22:21:31.667 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:31.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97c006a770 msgr2=0x7f97c0071fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97c006a770 0x7f97c0071fa0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f97b000b370 tx=0x7f97b0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 shutdown_connections 2023-09-30T22:21:31.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97c013c1e0 0x7f97c012e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97c006a770 0x7f97c0071fa0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 --2- 172.21.15.89:0/715494832 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97c009dc30 0x7f97c012dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 >> 172.21.15.89:0/715494832 conn(0x7f97c0066f50 msgr2=0x7f97c013def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 shutdown_connections 2023-09-30T22:21:31.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.668+0000 7f97c5f44700 1 -- 172.21.15.89:0/715494832 wait complete. 2023-09-30T22:21:31.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.709+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc844005860 con 0x7fc85c09d5c0 2023-09-30T22:21:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.709+0000 7fc8527fc700 1 -- 172.21.15.89:0/2528502931 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc85802b3a0 con 0x7fc85c09d5c0 2023-09-30T22:21:31.710 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c09d5c0 msgr2=0x7fc85c1d6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 --2- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c09d5c0 0x7fc85c1d6490 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc858000c00 tx=0x7fc858004750 comp rx=0 tx=0).stop 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 shutdown_connections 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 --2- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85c0a12d0 0x7fc85c1daf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 --2- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc85c09d5c0 0x7fc85c1d6490 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 --2- 172.21.15.89:0/2528502931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc85c0a09c0 0x7fc85c1d89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 >> 172.21.15.89:0/2528502931 conn(0x7fc85c0a2fe0 msgr2=0x7fc85c133b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 shutdown_connections 2023-09-30T22:21:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.711+0000 7fc863ef2700 1 -- 172.21.15.89:0/2528502931 wait complete. 2023-09-30T22:21:31.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:31.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.725+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f47f41a7400 con 0x7f47f409f650 2023-09-30T22:21:31.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.725+0000 7f47f0ff9700 1 -- 172.21.15.89:0/1706693645 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47ec019020 con 0x7f47f409f650 2023-09-30T22:21:31.726 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:31.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409f650 msgr2=0x7f47f41a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 --2- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409f650 0x7f47f41a69c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00b9e0 tx=0x7f47ec00bda0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 shutdown_connections 2023-09-30T22:21:31.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 --2- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f47f409f650 0x7f47f41a69c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 --2- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f47f409e930 0x7f47f41a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 --2- 172.21.15.89:0/1706693645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47f409f240 0x7f47f41a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 >> 172.21.15.89:0/1706693645 conn(0x7f47f40a4210 msgr2=0x7f47f409ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 shutdown_connections 2023-09-30T22:21:31.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.727+0000 7f47fa96e700 1 -- 172.21.15.89:0/1706693645 wait complete. 2023-09-30T22:21:31.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:31.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.749+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcc1006b9e0 con 0x7fcc10130540 2023-09-30T22:21:31.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.750+0000 7fcbf77fe700 1 -- 172.21.15.89:0/3101062245 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc00017030 con 0x7fcc10130540 2023-09-30T22:21:31.750 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:31.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.751+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10130540 msgr2=0x7fcc101c4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.752+0000 7fcc157df700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10130540 0x7fcc101c4490 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c480 tx=0x7fcc0000c790 comp rx=0 tx=0).stop 2023-09-30T22:21:31.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.752+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 shutdown_connections 2023-09-30T22:21:31.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.752+0000 7fcc157df700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10135e60 0x7fcc1009a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.752+0000 7fcc157df700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc10130540 0x7fcc101c4490 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.752+0000 7fcc157df700 1 --2- 172.21.15.89:0/3101062245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc10133540 0x7fcc101c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:31.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.752+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 >> 172.21.15.89:0/3101062245 conn(0x7fcc100a2aa0 msgr2=0x7fcc10131320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.752+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 shutdown_connections 2023-09-30T22:21:31.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:31.752+0000 7fcc157df700 1 -- 172.21.15.89:0/3101062245 wait complete. 2023-09-30T22:21:31.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:32.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:32.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:32.740 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:32.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:32.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.829+0000 7f1a2b0d7700 1 Processor -- start 2023-09-30T22:21:32.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.829+0000 7f1a2b0d7700 1 -- start start 2023-09-30T22:21:32.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a2b0d7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a24133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a2b0d7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a2b0d7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2406ef30 con 0x7f1a2409e760 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a2b0d7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2406fef0 con 0x7f1a241338a0 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a2b0d7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a241334b0 con 0x7f1a2409de50 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a23fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a23fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48804/0 (socket says 172.21.15.89:48804) 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a23fff700 1 -- 172.21.15.89:0/2529760728 learned_addr learned my addr 172.21.15.89:0/2529760728 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a28e73700 1 --2- 172.21.15.89:0/2529760728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a24133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a28e73700 1 -- 172.21.15.89:0/2529760728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 msgr2=0x7f1a2409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a28e73700 1 --2- 172.21.15.89:0/2529760728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a28e73700 1 -- 172.21.15.89:0/2529760728 >> v1:172.21.15.89:6789/0 conn(0x7f1a2409e760 legacy=0x7f1a2413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.830+0000 7f1a28e73700 1 -- 172.21.15.89:0/2529760728 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2406fc30 con 0x7f1a241338a0 2023-09-30T22:21:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.831+0000 7f1a28e73700 1 --2- 172.21.15.89:0/2529760728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a24133c70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009870 tx=0x7f1a18009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7034b6e9690225b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.831+0000 7f1a237fe700 1 -- 172.21.15.89:0/2529760728 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18011830 con 0x7f1a241338a0 2023-09-30T22:21:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.831+0000 7f1a237fe700 1 -- 172.21.15.89:0/2529760728 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a18011990 con 0x7f1a241338a0 2023-09-30T22:21:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.831+0000 7f1a237fe700 1 -- 172.21.15.89:0/2529760728 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18011d20 con 0x7f1a241338a0 2023-09-30T22:21:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.832+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/2529760728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 msgr2=0x7f1a24133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.832+0000 7f1a2b0d7700 1 --2- 172.21.15.89:0/2529760728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a24133c70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009870 tx=0x7f1a18009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.832+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/2529760728 shutdown_connections 2023-09-30T22:21:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.832+0000 7f1a2b0d7700 1 --2- 172.21.15.89:0/2529760728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a24133c70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.832+0000 7f1a2b0d7700 1 --2- 172.21.15.89:0/2529760728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.832+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/2529760728 >> 172.21.15.89:0/2529760728 conn(0x7f1a240a2aa0 msgr2=0x7f1a240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.832+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/2529760728 shutdown_connections 2023-09-30T22:21:32.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.832+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/2529760728 wait complete. 2023-09-30T22:21:32.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.833+0000 7f1a2b0d7700 1 Processor -- start 2023-09-30T22:21:32.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.833+0000 7f1a2b0d7700 1 -- start start 2023-09-30T22:21:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a2b0d7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2411a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a2b0d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a2409e760 0x7f1a2411abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a2b0d7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a2411b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a2b0d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2411c8e0 con 0x7f1a2409e760 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a2b0d7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2411e1a0 con 0x7f1a241338a0 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a2b0d7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2411d570 con 0x7f1a2409de50 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a28e73700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2411a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a28e73700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2411a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48818/0 (socket says 172.21.15.89:48818) 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a28e73700 1 -- 172.21.15.89:0/3532875332 learned_addr learned my addr 172.21.15.89:0/3532875332 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a23fff700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a2409e760 0x7f1a2411abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.834+0000 7f1a29674700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a2411b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a23fff700 1 -- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 msgr2=0x7f1a2411a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a23fff700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2411a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a23fff700 1 -- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 msgr2=0x7f1a2411b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a23fff700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a2411b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a23fff700 1 -- 172.21.15.89:0/3532875332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180094a0 con 0x7f1a2409e760 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a28e73700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2411a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a23fff700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a2409e760 0x7f1a2411abc0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000e870 tx=0x7f1a1000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a217fa700 1 -- 172.21.15.89:0/3532875332 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10013680 con 0x7f1a2409e760 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a217fa700 1 -- 172.21.15.89:0/3532875332 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a100137e0 con 0x7f1a2409e760 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a217fa700 1 -- 172.21.15.89:0/3532875332 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10013b50 con 0x7f1a2409e760 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2411bae0 con 0x7f1a2409e760 2023-09-30T22:21:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.835+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2411c290 con 0x7f1a2409e760 2023-09-30T22:21:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.836+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a24074ea0 con 0x7f1a2409e760 2023-09-30T22:21:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.837+0000 7f1a217fa700 1 -- 172.21.15.89:0/3532875332 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a10013cb0 con 0x7f1a2409e760 2023-09-30T22:21:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.837+0000 7f1a217fa700 1 -- 172.21.15.89:0/3532875332 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1a1003fcd0 con 0x7f1a2409e760 2023-09-30T22:21:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.845+0000 7f1a217fa700 1 -- 172.21.15.89:0/3532875332 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a10011d90 con 0x7f1a2409e760 2023-09-30T22:21:32.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.879+0000 7fec7146c700 1 Processor -- start 2023-09-30T22:21:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.879+0000 7fec7146c700 1 -- start start 2023-09-30T22:21:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.879+0000 7fec7146c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec7146c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec7146c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c09e130 con 0x7fec6c06acb0 2023-09-30T22:21:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec7146c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c1312b0 con 0x7fec6c1316c0 2023-09-30T22:21:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec7146c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c09f6e0 con 0x7fec6c09d9f0 2023-09-30T22:21:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fec6c06acb0 0x7fec6c138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43306/0 (socket says 172.21.15.89:43306) 2023-09-30T22:21:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48834/0 (socket says 172.21.15.89:48834) 2023-09-30T22:21:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6b7fe700 1 -- 172.21.15.89:0/1207331329 learned_addr learned my addr 172.21.15.89:0/1207331329 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6affd700 1 --2- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6a7fc700 1 -- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 msgr2=0x7fec6c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6a7fc700 1 --2- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6a7fc700 1 -- 172.21.15.89:0/1207331329 >> v1:172.21.15.89:6789/0 conn(0x7fec6c06acb0 legacy=0x7fec6c138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6a7fc700 1 -- 172.21.15.89:0/1207331329 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c09e760 con 0x7fec6c09d9f0 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6affd700 1 --2- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.880+0000 7fec6a7fc700 1 --2- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c06a770 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fec60009870 tx=0x7fec60009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c1a6a94445ae0c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.881+0000 7fec69ffb700 1 -- 172.21.15.89:0/1207331329 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60011830 con 0x7fec6c09d9f0 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.881+0000 7fec69ffb700 1 -- 172.21.15.89:0/1207331329 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec60011990 con 0x7fec6c09d9f0 2023-09-30T22:21:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.881+0000 7fec69ffb700 1 -- 172.21.15.89:0/1207331329 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60011d20 con 0x7fec6c09d9f0 2023-09-30T22:21:32.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.881+0000 7fec7146c700 1 -- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 msgr2=0x7fec6c06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.881+0000 7fec7146c700 1 --2- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c06a770 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fec60009870 tx=0x7fec60009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.882+0000 7fec7146c700 1 -- 172.21.15.89:0/1207331329 shutdown_connections 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.882+0000 7fec7146c700 1 --2- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.882+0000 7fec7146c700 1 --2- 172.21.15.89:0/1207331329 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c06a770 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.882+0000 7fec7146c700 1 -- 172.21.15.89:0/1207331329 >> 172.21.15.89:0/1207331329 conn(0x7fec6c066f50 msgr2=0x7fec6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.882+0000 7fec7146c700 1 -- 172.21.15.89:0/1207331329 shutdown_connections 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.882+0000 7fec7146c700 1 -- 172.21.15.89:0/1207331329 wait complete. 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.883+0000 7fec7146c700 1 Processor -- start 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.883+0000 7fec7146c700 1 -- start start 2023-09-30T22:21:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec7146c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec6c06acb0 0x7fec6c1c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec7146c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c09a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec7146c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c09a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec7146c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c09c1f0 con 0x7fec6c06acb0 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec7146c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c09bfc0 con 0x7fec6c1316c0 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec7146c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0a36d0 con 0x7fec6c09d9f0 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec6c06acb0 0x7fec6c1c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec6c06acb0 0x7fec6c1c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52182/0 (socket says 172.21.15.89:52182) 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6affd700 1 -- 172.21.15.89:0/620943706 learned_addr learned my addr 172.21.15.89:0/620943706 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6a7fc700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c09a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6b7fe700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c09a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6affd700 1 -- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 msgr2=0x7fec6c09a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6affd700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c09a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6affd700 1 -- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 msgr2=0x7fec6c09a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6affd700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c09a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.884+0000 7fec6affd700 1 -- 172.21.15.89:0/620943706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c0094a0 con 0x7fec6c06acb0 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.885+0000 7fec6affd700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec6c06acb0 0x7fec6c1c8980 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00ec10 tx=0x7fec5c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.885+0000 7fec537fe700 1 -- 172.21.15.89:0/620943706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00c850 con 0x7fec6c06acb0 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.885+0000 7fec537fe700 1 -- 172.21.15.89:0/620943706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c00c9b0 con 0x7fec6c06acb0 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.885+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec600094a0 con 0x7fec6c06acb0 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.885+0000 7fec537fe700 1 -- 172.21.15.89:0/620943706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00cd20 con 0x7fec6c06acb0 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.885+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c070790 con 0x7fec6c06acb0 2023-09-30T22:21:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.886+0000 7fec537fe700 1 -- 172.21.15.89:0/620943706 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fec5c013470 con 0x7fec6c06acb0 2023-09-30T22:21:32.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.886+0000 7fec537fe700 1 -- 172.21.15.89:0/620943706 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fec5c03f900 con 0x7fec6c06acb0 2023-09-30T22:21:32.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.886+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c074ea0 con 0x7fec6c06acb0 2023-09-30T22:21:32.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.891+0000 7fec537fe700 1 -- 172.21.15.89:0/620943706 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec5c013720 con 0x7fec6c06acb0 2023-09-30T22:21:32.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.908+0000 7fbb743ba700 1 Processor -- start 2023-09-30T22:21:32.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.908+0000 7fbb743ba700 1 -- start start 2023-09-30T22:21:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.908+0000 7fbb743ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c136830 0x7fbb6c138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.908+0000 7fbb743ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb6c139150 0x7fbb6c13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.908+0000 7fbb743ba700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c06f720 con 0x7fbb6c1338c0 2023-09-30T22:21:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.908+0000 7fbb743ba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6c070c80 con 0x7fbb6c139150 2023-09-30T22:21:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.908+0000 7fbb743ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c06fce0 con 0x7fbb6c136830 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb72957700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb6c139150 0x7fbb6c13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb72156700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbb6c1338c0 0x7fbb6c133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43318/0 (socket says 172.21.15.89:43318) 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb72156700 1 -- 172.21.15.89:0/4193315817 learned_addr learned my addr 172.21.15.89:0/4193315817 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb71955700 1 --2- 172.21.15.89:0/4193315817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c136830 0x7fbb6c138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb72957700 1 -- 172.21.15.89:0/4193315817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c136830 msgr2=0x7fbb6c138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb72957700 1 --2- 172.21.15.89:0/4193315817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c136830 0x7fbb6c138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb72957700 1 -- 172.21.15.89:0/4193315817 >> v1:172.21.15.89:6789/0 conn(0x7fbb6c1338c0 legacy=0x7fbb6c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb72957700 1 -- 172.21.15.89:0/4193315817 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c1336a0 con 0x7fbb6c139150 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb72957700 1 --2- 172.21.15.89:0/4193315817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c13b530 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009d90 tx=0x7fbb6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d0250f3b241ba2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb71154700 1 -- 172.21.15.89:0/4193315817 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400e070 con 0x7fbb6c139150 2023-09-30T22:21:32.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb71154700 1 -- 172.21.15.89:0/4193315817 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6400bb20 con 0x7fbb6c139150 2023-09-30T22:21:32.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.909+0000 7fbb71154700 1 -- 172.21.15.89:0/4193315817 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64012430 con 0x7fbb6c139150 2023-09-30T22:21:32.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.910+0000 7fbb743ba700 1 -- 172.21.15.89:0/4193315817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 msgr2=0x7fbb6c13b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.910+0000 7fbb743ba700 1 --2- 172.21.15.89:0/4193315817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c13b530 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009d90 tx=0x7fbb6400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.910+0000 7fbb743ba700 1 -- 172.21.15.89:0/4193315817 shutdown_connections 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.910+0000 7fbb743ba700 1 --2- 172.21.15.89:0/4193315817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c13b530 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.910+0000 7fbb743ba700 1 --2- 172.21.15.89:0/4193315817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c136830 0x7fbb6c138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.910+0000 7fbb743ba700 1 -- 172.21.15.89:0/4193315817 >> 172.21.15.89:0/4193315817 conn(0x7fbb6c0a2aa0 msgr2=0x7fbb6c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.910+0000 7fbb743ba700 1 -- 172.21.15.89:0/4193315817 shutdown_connections 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.910+0000 7fbb743ba700 1 -- 172.21.15.89:0/4193315817 wait complete. 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.911+0000 7fbb743ba700 1 Processor -- start 2023-09-30T22:21:32.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.911+0000 7fbb743ba700 1 -- start start 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.911+0000 7fbb743ba700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c1338c0 0x7fbb6c1c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.911+0000 7fbb743ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6c136830 0x7fbb6c1c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb743ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb743ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c1be2d0 con 0x7fbb6c139150 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb743ba700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c11c210 con 0x7fbb6c136830 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb743ba700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c06f960 con 0x7fbb6c1338c0 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72957700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72957700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52204/0 (socket says 172.21.15.89:52204) 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72957700 1 -- 172.21.15.89:0/4181993989 learned_addr learned my addr 172.21.15.89:0/4181993989 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72156700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c1338c0 0x7fbb6c1c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb71955700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6c136830 0x7fbb6c1c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72156700 1 -- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6c136830 msgr2=0x7fbb6c1c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72156700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6c136830 0x7fbb6c1c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72156700 1 -- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 msgr2=0x7fbb6c118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72156700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72156700 1 -- 172.21.15.89:0/4181993989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64009a70 con 0x7fbb6c1338c0 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72957700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb72156700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c1338c0 0x7fbb6c1c8620 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c500 tx=0x7fbb6800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.912+0000 7fbb62ffd700 1 -- 172.21.15.89:0/4181993989 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68005d70 con 0x7fbb6c1338c0 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.913+0000 7fbb62ffd700 1 -- 172.21.15.89:0/4181993989 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb680070d0 con 0x7fbb6c1338c0 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.913+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c12eb60 con 0x7fbb6c1338c0 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.913+0000 7fbb62ffd700 1 -- 172.21.15.89:0/4181993989 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68007440 con 0x7fbb6c1338c0 2023-09-30T22:21:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.913+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c1c90a0 con 0x7fbb6c1338c0 2023-09-30T22:21:32.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.913+0000 7fbb62ffd700 1 -- 172.21.15.89:0/4181993989 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbb680075a0 con 0x7fbb6c1338c0 2023-09-30T22:21:32.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.914+0000 7fbb62ffd700 1 -- 172.21.15.89:0/4181993989 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbb6803eb30 con 0x7fbb6c1338c0 2023-09-30T22:21:32.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.914+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c11c430 con 0x7fbb6c1338c0 2023-09-30T22:21:32.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.918+0000 7f2f45c41700 1 Processor -- start 2023-09-30T22:21:32.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.918+0000 7f2f45c41700 1 -- start start 2023-09-30T22:21:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f45c41700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f45c41700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2f4009e410 0x7f2f40130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f45c41700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4009f130 con 0x7f2f4009e000 2023-09-30T22:21:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f45c41700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4009f520 con 0x7f2f4009e410 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f45c41700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4009e930 con 0x7f2f4009d6f0 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f36ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2f4009e000 0x7f2f40130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43324/0 (socket says 172.21.15.89:43324) 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f36ffd700 1 -- 172.21.15.89:0/2075663083 learned_addr learned my addr 172.21.15.89:0/2075663083 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f3ffff700 1 --2- 172.21.15.89:0/2075663083 >> v2:172.21.15.89:3300/0 conn(0x7f2f4009e410 0x7f2f40130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f3ffff700 1 -- 172.21.15.89:0/2075663083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009d6f0 msgr2=0x7f2f4009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f3ffff700 1 --2- 172.21.15.89:0/2075663083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f3ffff700 1 -- 172.21.15.89:0/2075663083 >> v1:172.21.15.89:6789/0 conn(0x7f2f4009e000 legacy=0x7f2f40130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f3ffff700 1 -- 172.21.15.89:0/2075663083 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4009b2d0 con 0x7f2f4009e410 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f3ffff700 1 --2- 172.21.15.89:0/2075663083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 0x7f2f40130d70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009d90 tx=0x7f2f3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56116989ca0b7ec6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.919+0000 7f2f3effd700 1 -- 172.21.15.89:0/2075663083 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800bdd0 con 0x7f2f4009e410 2023-09-30T22:21:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f3effd700 1 -- 172.21.15.89:0/2075663083 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38011460 con 0x7f2f4009e410 2023-09-30T22:21:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f3effd700 1 -- 172.21.15.89:0/2075663083 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011810 con 0x7f2f4009e410 2023-09-30T22:21:32.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f45c41700 1 -- 172.21.15.89:0/2075663083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 msgr2=0x7f2f40130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f45c41700 1 --2- 172.21.15.89:0/2075663083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 0x7f2f40130d70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009d90 tx=0x7f2f3800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:32.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f45c41700 1 -- 172.21.15.89:0/2075663083 shutdown_connections 2023-09-30T22:21:32.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f45c41700 1 --2- 172.21.15.89:0/2075663083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 0x7f2f40130d70 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f45c41700 1 --2- 172.21.15.89:0/2075663083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f45c41700 1 -- 172.21.15.89:0/2075663083 >> 172.21.15.89:0/2075663083 conn(0x7f2f40066f50 msgr2=0x7f2f4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f45c41700 1 -- 172.21.15.89:0/2075663083 shutdown_connections 2023-09-30T22:21:32.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.920+0000 7f2f45c41700 1 -- 172.21.15.89:0/2075663083 wait complete. 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.921+0000 7f2f45c41700 1 Processor -- start 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.921+0000 7f2f45c41700 1 -- start start 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f45c41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4009d6f0 0x7f2f401d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f45c41700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009e000 0x7f2f401d2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f45c41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 0x7f2f401d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f45c41700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f401b7240 con 0x7f2f4009e410 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f45c41700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f401b8ac0 con 0x7f2f4009d6f0 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f45c41700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f401d2990 con 0x7f2f4009e000 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f36ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009e000 0x7f2f401d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f36ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009e000 0x7f2f401d2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48868/0 (socket says 172.21.15.89:48868) 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f36ffd700 1 -- 172.21.15.89:0/2335045629 learned_addr learned my addr 172.21.15.89:0/2335045629 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:32.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f3f7fe700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4009d6f0 0x7f2f401d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f3ffff700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 0x7f2f401d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f36ffd700 1 -- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4009d6f0 msgr2=0x7f2f401d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f36ffd700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4009d6f0 0x7f2f401d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f36ffd700 1 -- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 msgr2=0x7f2f401d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f36ffd700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 0x7f2f401d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.922+0000 7f2f36ffd700 1 -- 172.21.15.89:0/2335045629 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38009a70 con 0x7f2f4009e000 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.923+0000 7f2f36ffd700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009e000 0x7f2f401d2450 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800c500 tx=0x7f2f2800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.923+0000 7f2f3ffff700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 0x7f2f401d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.923+0000 7f2f3cff9700 1 -- 172.21.15.89:0/2335045629 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28005d70 con 0x7f2f4009e000 2023-09-30T22:21:32.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.923+0000 7f2f3cff9700 1 -- 172.21.15.89:0/2335045629 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f280070d0 con 0x7f2f4009e000 2023-09-30T22:21:32.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.923+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f401b83c0 con 0x7f2f4009e000 2023-09-30T22:21:32.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.923+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f401b7050 con 0x7f2f4009e000 2023-09-30T22:21:32.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.924+0000 7fbb62ffd700 1 -- 172.21.15.89:0/4181993989 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb6c11c430 con 0x7fbb6c1338c0 2023-09-30T22:21:32.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.923+0000 7f2f3cff9700 1 -- 172.21.15.89:0/2335045629 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28007440 con 0x7f2f4009e000 2023-09-30T22:21:32.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.924+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f240052e0 con 0x7f2f4009e000 2023-09-30T22:21:32.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.925+0000 7f2f3cff9700 1 -- 172.21.15.89:0/2335045629 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f280075a0 con 0x7f2f4009e000 2023-09-30T22:21:32.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.925+0000 7f2f3cff9700 1 -- 172.21.15.89:0/2335045629 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2f2803eb70 con 0x7f2f4009e000 2023-09-30T22:21:32.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:32.933+0000 7f2f3cff9700 1 -- 172.21.15.89:0/2335045629 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f28019b40 con 0x7f2f4009e000 2023-09-30T22:21:33.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.052+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1a2406b9e0 con 0x7f1a2409e760 2023-09-30T22:21:33.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.053+0000 7f1a217fa700 1 -- 172.21.15.89:0/3532875332 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a10017030 con 0x7f1a2409e760 2023-09-30T22:21:33.054 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a2409e760 msgr2=0x7f1a2411abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a2409e760 0x7f1a2411abc0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000e870 tx=0x7f1a1000eb80 comp rx=0 tx=0).stop 2023-09-30T22:21:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 shutdown_connections 2023-09-30T22:21:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a241338a0 0x7f1a2411b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a2409de50 0x7f1a2411a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 --2- 172.21.15.89:0/3532875332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a2409e760 0x7f1a2411abc0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 >> 172.21.15.89:0/3532875332 conn(0x7f1a240a2aa0 msgr2=0x7f1a241307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 shutdown_connections 2023-09-30T22:21:33.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.056+0000 7f1a2b0d7700 1 -- 172.21.15.89:0/3532875332 wait complete. 2023-09-30T22:21:33.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:33.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.089+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fec6c12eb40 con 0x7fec6c06acb0 2023-09-30T22:21:33.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.089+0000 7fec537fe700 1 -- 172.21.15.89:0/620943706 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec5c01bb60 con 0x7fec6c06acb0 2023-09-30T22:21:33.090 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:33.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec6c06acb0 msgr2=0x7fec6c1c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec6c06acb0 0x7fec6c1c8980 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00ec10 tx=0x7fec5c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:21:33.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 shutdown_connections 2023-09-30T22:21:33.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec6c1316c0 0x7fec6c09a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec6c06acb0 0x7fec6c1c8980 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 --2- 172.21.15.89:0/620943706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec6c09d9f0 0x7fec6c09a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 >> 172.21.15.89:0/620943706 conn(0x7fec6c066f50 msgr2=0x7fec6c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 shutdown_connections 2023-09-30T22:21:33.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.092+0000 7fec7146c700 1 -- 172.21.15.89:0/620943706 wait complete. 2023-09-30T22:21:33.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.128+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb6c11c430 con 0x7fbb6c1338c0 2023-09-30T22:21:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.129+0000 7fbb62ffd700 1 -- 172.21.15.89:0/4181993989 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb68017080 con 0x7fbb6c1338c0 2023-09-30T22:21:33.130 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c1338c0 msgr2=0x7fbb6c1c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c1338c0 0x7fbb6c1c8620 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c500 tx=0x7fbb6800c810 comp rx=0 tx=0).stop 2023-09-30T22:21:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 shutdown_connections 2023-09-30T22:21:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb6c139150 0x7fbb6c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb6c1338c0 0x7fbb6c1c8620 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 --2- 172.21.15.89:0/4181993989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb6c136830 0x7fbb6c1c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 >> 172.21.15.89:0/4181993989 conn(0x7fbb6c0a2aa0 msgr2=0x7fbb6c134610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 shutdown_connections 2023-09-30T22:21:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.132+0000 7fbb743ba700 1 -- 172.21.15.89:0/4181993989 wait complete. 2023-09-30T22:21:33.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:33.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.156+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2f24005590 con 0x7f2f4009e000 2023-09-30T22:21:33.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.156+0000 7f2f3cff9700 1 -- 172.21.15.89:0/2335045629 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f28017030 con 0x7f2f4009e000 2023-09-30T22:21:33.157 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009e000 msgr2=0x7f2f401d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009e000 0x7f2f401d2450 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800c500 tx=0x7f2f2800c810 comp rx=0 tx=0).stop 2023-09-30T22:21:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 shutdown_connections 2023-09-30T22:21:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4009e410 0x7f2f401d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4009d6f0 0x7f2f401d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 --2- 172.21.15.89:0/2335045629 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4009e000 0x7f2f401d2450 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 >> 172.21.15.89:0/2335045629 conn(0x7f2f40066f50 msgr2=0x7f2f400a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 shutdown_connections 2023-09-30T22:21:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:33.158+0000 7f2f45c41700 1 -- 172.21.15.89:0/2335045629 wait complete. 2023-09-30T22:21:33.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:34.074 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:34.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:34.150 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:34.172 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.222+0000 7f0860213700 1 Processor -- start 2023-09-30T22:21:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.222+0000 7f0860213700 1 -- start start 2023-09-30T22:21:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.223+0000 7f0860213700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f0858138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.223+0000 7f0860213700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0858139170 0x7f085813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.223+0000 7f0860213700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085809b120 con 0x7f085809d6f0 2023-09-30T22:21:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.223+0000 7f0860213700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085806b460 con 0x7f0858139170 2023-09-30T22:21:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.223+0000 7f0860213700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f085809a7e0 con 0x7f085809e1d0 2023-09-30T22:21:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085d7ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f0858138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085d7ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f0858138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48882/0 (socket says 172.21.15.89:48882) 2023-09-30T22:21:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085d7ae700 1 -- 172.21.15.89:0/3879286778 learned_addr learned my addr 172.21.15.89:0/3879286778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085d7ae700 1 -- 172.21.15.89:0/3879286778 >> v2:172.21.15.89:3300/0 conn(0x7f0858139170 msgr2=0x7f085813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085d7ae700 1 --2- 172.21.15.89:0/3879286778 >> v2:172.21.15.89:3300/0 conn(0x7f0858139170 0x7f085813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085d7ae700 1 -- 172.21.15.89:0/3879286778 >> v1:172.21.15.89:6789/0 conn(0x7f085809d6f0 legacy=0x7f085809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085d7ae700 1 -- 172.21.15.89:0/3879286778 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085809a500 con 0x7f085809e1d0 2023-09-30T22:21:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085d7ae700 1 --2- 172.21.15.89:0/3879286778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f0858138c30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0848009920 tx=0x7f0848009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9ad06c076a52e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085cfad700 1 -- 172.21.15.89:0/3879286778 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848004230 con 0x7f085809e1d0 2023-09-30T22:21:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085cfad700 1 -- 172.21.15.89:0/3879286778 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0848004390 con 0x7f085809e1d0 2023-09-30T22:21:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.224+0000 7f085cfad700 1 -- 172.21.15.89:0/3879286778 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848009e70 con 0x7f085809e1d0 2023-09-30T22:21:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.225+0000 7f0860213700 1 -- 172.21.15.89:0/3879286778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 msgr2=0x7f0858138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.225+0000 7f0860213700 1 --2- 172.21.15.89:0/3879286778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f0858138c30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0848009920 tx=0x7f0848009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.225+0000 7f0860213700 1 -- 172.21.15.89:0/3879286778 shutdown_connections 2023-09-30T22:21:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.225+0000 7f0860213700 1 --2- 172.21.15.89:0/3879286778 >> v2:172.21.15.89:3300/0 conn(0x7f0858139170 0x7f085813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.225+0000 7f0860213700 1 --2- 172.21.15.89:0/3879286778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f0858138c30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.225+0000 7f0860213700 1 -- 172.21.15.89:0/3879286778 >> 172.21.15.89:0/3879286778 conn(0x7f0858066f50 msgr2=0x7f085809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.225+0000 7f0860213700 1 -- 172.21.15.89:0/3879286778 shutdown_connections 2023-09-30T22:21:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.225+0000 7f0860213700 1 -- 172.21.15.89:0/3879286778 wait complete. 2023-09-30T22:21:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f0860213700 1 Processor -- start 2023-09-30T22:21:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f0860213700 1 -- start start 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f0860213700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f085809d6f0 0x7f08581c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f0860213700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f08581c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f0860213700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0858139170 0x7f085811e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f0860213700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085811fc90 con 0x7f085809d6f0 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f0860213700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085809c710 con 0x7f0858139170 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f0860213700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08580a30b0 con 0x7f085809e1d0 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f085dfaf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f085809d6f0 0x7f08581c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f085dfaf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f085809d6f0 0x7f08581c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52238/0 (socket says 172.21.15.89:52238) 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f085dfaf700 1 -- 172.21.15.89:0/3099983934 learned_addr learned my addr 172.21.15.89:0/3099983934 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.227+0000 7f085d7ae700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f08581c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f085e7b0700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0858139170 0x7f085811e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f085dfaf700 1 -- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 msgr2=0x7f08581c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f085dfaf700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f08581c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f085dfaf700 1 -- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0858139170 msgr2=0x7f085811e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f085dfaf700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0858139170 0x7f085811e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f085dfaf700 1 -- 172.21.15.89:0/3099983934 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0848009550 con 0x7f085809d6f0 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f085d7ae700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f08581c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f085dfaf700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f085809d6f0 0x7f08581c8560 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f085400c4a0 tx=0x7f085400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f084e7fc700 1 -- 172.21.15.89:0/3099983934 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08540054e0 con 0x7f085809d6f0 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f084e7fc700 1 -- 172.21.15.89:0/3099983934 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0854003680 con 0x7f085809d6f0 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f084e7fc700 1 -- 172.21.15.89:0/3099983934 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854003a30 con 0x7f085809d6f0 2023-09-30T22:21:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0858134690 con 0x7f085809d6f0 2023-09-30T22:21:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.228+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0858122a70 con 0x7f085809d6f0 2023-09-30T22:21:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.229+0000 7f084e7fc700 1 -- 172.21.15.89:0/3099983934 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0854003b90 con 0x7f085809d6f0 2023-09-30T22:21:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.230+0000 7f084e7fc700 1 -- 172.21.15.89:0/3099983934 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f085403e740 con 0x7f085809d6f0 2023-09-30T22:21:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.230+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0858074ea0 con 0x7f085809d6f0 2023-09-30T22:21:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.235+0000 7f084e7fc700 1 -- 172.21.15.89:0/3099983934 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0854003e40 con 0x7f085809d6f0 2023-09-30T22:21:34.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.258+0000 7f5558089700 1 Processor -- start 2023-09-30T22:21:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.258+0000 7f5558089700 1 -- start start 2023-09-30T22:21:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5558089700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f555006c0e0 0x7f555009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5558089700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5550134ec0 0x7f55501379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5558089700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5550134980 con 0x7f5550137f20 2023-09-30T22:21:34.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5558089700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555006c4b0 con 0x7f555006c0e0 2023-09-30T22:21:34.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5558089700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55500a09c0 con 0x7f5550134ec0 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5555624700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5550134ec0 0x7f55501379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5555e25700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f555006c0e0 0x7f555009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5555624700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5550134ec0 0x7f55501379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52030/0 (socket says 172.21.15.89:52030) 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5555624700 1 -- 172.21.15.89:0/3358243700 learned_addr learned my addr 172.21.15.89:0/3358243700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5555e25700 1 -- 172.21.15.89:0/3358243700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5550134ec0 msgr2=0x7f55501379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5555e25700 1 --2- 172.21.15.89:0/3358243700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5550134ec0 0x7f55501379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5555e25700 1 -- 172.21.15.89:0/3358243700 >> v1:172.21.15.89:6789/0 conn(0x7f5550137f20 legacy=0x7f555013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.259+0000 7f5555e25700 1 -- 172.21.15.89:0/3358243700 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55500a0600 con 0x7f555006c0e0 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5555e25700 1 --2- 172.21.15.89:0/3358243700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f555006c0e0 0x7f555009ac90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f554c009840 tx=0x7f554c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c5619e8203b0de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5554e23700 1 -- 172.21.15.89:0/3358243700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c011830 con 0x7f555006c0e0 2023-09-30T22:21:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5554e23700 1 -- 172.21.15.89:0/3358243700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c011990 con 0x7f555006c0e0 2023-09-30T22:21:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5554e23700 1 -- 172.21.15.89:0/3358243700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c011d20 con 0x7f555006c0e0 2023-09-30T22:21:34.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5558089700 1 -- 172.21.15.89:0/3358243700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f555006c0e0 msgr2=0x7f555009ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5558089700 1 --2- 172.21.15.89:0/3358243700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f555006c0e0 0x7f555009ac90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f554c009840 tx=0x7f554c009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5558089700 1 -- 172.21.15.89:0/3358243700 shutdown_connections 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5558089700 1 --2- 172.21.15.89:0/3358243700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f555006c0e0 0x7f555009ac90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.260+0000 7f5558089700 1 --2- 172.21.15.89:0/3358243700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5550134ec0 0x7f55501379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.261+0000 7f5558089700 1 -- 172.21.15.89:0/3358243700 >> 172.21.15.89:0/3358243700 conn(0x7f55500a4210 msgr2=0x7f55500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.261+0000 7f5558089700 1 -- 172.21.15.89:0/3358243700 shutdown_connections 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.261+0000 7f5558089700 1 -- 172.21.15.89:0/3358243700 wait complete. 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.261+0000 7f5558089700 1 Processor -- start 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5558089700 1 -- start start 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5558089700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f555006c0e0 0x7f55501ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5558089700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5550134ec0 0x7f55501cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5558089700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5550137f20 0x7f555011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5558089700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55500a0460 con 0x7f5550137f20 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5558089700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555011da50 con 0x7f555006c0e0 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5558089700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5550134010 con 0x7f5550134ec0 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5555e25700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f555006c0e0 0x7f55501ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5556626700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5550137f20 0x7f555011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5555e25700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f555006c0e0 0x7f55501ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52046/0 (socket says 172.21.15.89:52046) 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5556626700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5550137f20 0x7f555011a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52256/0 (socket says 172.21.15.89:52256) 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5555e25700 1 -- 172.21.15.89:0/2163632469 learned_addr learned my addr 172.21.15.89:0/2163632469 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.262+0000 7f5555624700 1 --2- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5550134ec0 0x7f55501cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f5556626700 1 -- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5550134ec0 msgr2=0x7f55501cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f5556626700 1 --2- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5550134ec0 0x7f55501cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f5556626700 1 -- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f555006c0e0 msgr2=0x7f55501ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f5556626700 1 --2- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f555006c0e0 0x7f55501ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f5556626700 1 -- 172.21.15.89:0/2163632469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0094a0 con 0x7f5550137f20 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f5556626700 1 --2- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5550137f20 0x7f555011a680 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f554800cb90 tx=0x7f554800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f55467fc700 1 -- 172.21.15.89:0/2163632469 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548003c00 con 0x7f5550137f20 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f55467fc700 1 -- 172.21.15.89:0/2163632469 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5548003d60 con 0x7f5550137f20 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555011d7a0 con 0x7f5550137f20 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f55467fc700 1 -- 172.21.15.89:0/2163632469 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554800aa80 con 0x7f5550137f20 2023-09-30T22:21:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.263+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5550134350 con 0x7f5550137f20 2023-09-30T22:21:34.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.264+0000 7f55467fc700 1 -- 172.21.15.89:0/2163632469 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f554800abe0 con 0x7f5550137f20 2023-09-30T22:21:34.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.264+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55500a38f0 con 0x7f5550137f20 2023-09-30T22:21:34.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.264+0000 7f55467fc700 1 -- 172.21.15.89:0/2163632469 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f554803f5a0 con 0x7f5550137f20 2023-09-30T22:21:34.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.273+0000 7f55467fc700 1 -- 172.21.15.89:0/2163632469 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55480196c0 con 0x7f5550137f20 2023-09-30T22:21:34.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.294+0000 7fbfec368700 1 Processor -- start 2023-09-30T22:21:34.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.294+0000 7fbfec368700 1 -- start start 2023-09-30T22:21:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.295+0000 7fbfec368700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.295+0000 7fbfec368700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe4138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.295+0000 7fbfec368700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe406fa40 con 0x7fbfe412cf60 2023-09-30T22:21:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.295+0000 7fbfec368700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe412cd90 con 0x7fbfe406f5d0 2023-09-30T22:21:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.295+0000 7fbfec368700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe4071780 con 0x7fbfe4135c20 2023-09-30T22:21:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.295+0000 7fbfea104700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbfe412cf60 0x7fbfe412f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43340/0 (socket says 172.21.15.89:43340) 2023-09-30T22:21:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.295+0000 7fbfea104700 1 -- 172.21.15.89:0/1961495951 learned_addr learned my addr 172.21.15.89:0/1961495951 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:34.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.295+0000 7fbfea905700 1 --2- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe4138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfe9903700 1 --2- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfea905700 1 -- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 msgr2=0x7fbfe412ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfea905700 1 --2- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe412ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfea905700 1 -- 172.21.15.89:0/1961495951 >> v1:172.21.15.89:6789/0 conn(0x7fbfe412cf60 legacy=0x7fbfe412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfe9102700 1 -- 172.21.15.89:0/1961495951 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931707830 0 0) 0x7fbfe406fa40 con 0x7fbfe412cf60 2023-09-30T22:21:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfea905700 1 -- 172.21.15.89:0/1961495951 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe4071a40 con 0x7fbfe4135c20 2023-09-30T22:21:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfea905700 1 --2- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe4138000 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009df0 tx=0x7fbfdc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1224d3cd17077485 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfe9102700 1 -- 172.21.15.89:0/1961495951 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00e070 con 0x7fbfe4135c20 2023-09-30T22:21:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfe9102700 1 -- 172.21.15.89:0/1961495951 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc00bb90 con 0x7fbfe4135c20 2023-09-30T22:21:34.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.296+0000 7fbfe9102700 1 -- 172.21.15.89:0/1961495951 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc012430 con 0x7fbfe4135c20 2023-09-30T22:21:34.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.297+0000 7fbfec368700 1 -- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 msgr2=0x7fbfe4138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.297+0000 7fbfec368700 1 --2- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe4138000 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009df0 tx=0x7fbfdc00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:34.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.297+0000 7fbfec368700 1 -- 172.21.15.89:0/1961495951 shutdown_connections 2023-09-30T22:21:34.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.297+0000 7fbfec368700 1 --2- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe4138000 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.297+0000 7fbfec368700 1 --2- 172.21.15.89:0/1961495951 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe412ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.297+0000 7fbfec368700 1 -- 172.21.15.89:0/1961495951 >> 172.21.15.89:0/1961495951 conn(0x7fbfe40a2aa0 msgr2=0x7fbfe40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.297+0000 7fbfec368700 1 -- 172.21.15.89:0/1961495951 shutdown_connections 2023-09-30T22:21:34.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.297+0000 7fbfec368700 1 -- 172.21.15.89:0/1961495951 wait complete. 2023-09-30T22:21:34.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.299+0000 7fbfec368700 1 Processor -- start 2023-09-30T22:21:34.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.299+0000 7fbfec368700 1 -- start start 2023-09-30T22:21:34.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.299+0000 7fbfec368700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe41c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfec368700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe412cf60 0x7fbfe41c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfec368700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe41c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfec368700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe4099d50 con 0x7fbfe412cf60 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfec368700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe409b390 con 0x7fbfe406f5d0 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfec368700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe409b810 con 0x7fbfe4135c20 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea104700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe41c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea104700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe41c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52062/0 (socket says 172.21.15.89:52062) 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea104700 1 -- 172.21.15.89:0/2612926938 learned_addr learned my addr 172.21.15.89:0/2612926938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea905700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe41c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfe9903700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe412cf60 0x7fbfe41c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea905700 1 -- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 msgr2=0x7fbfe41c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea905700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe41c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea905700 1 -- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe412cf60 msgr2=0x7fbfe41c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea905700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe412cf60 0x7fbfe41c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea905700 1 -- 172.21.15.89:0/2612926938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc009a70 con 0x7fbfe4135c20 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea104700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe41c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfea905700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe41c4880 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc016000 tx=0x7fbfdc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.300+0000 7fbfdaffd700 1 -- 172.21.15.89:0/2612926938 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc012430 con 0x7fbfe4135c20 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.301+0000 7fbfdaffd700 1 -- 172.21.15.89:0/2612926938 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc020070 con 0x7fbfe4135c20 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.301+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe409e9e0 con 0x7fbfe4135c20 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.301+0000 7fbfdaffd700 1 -- 172.21.15.89:0/2612926938 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc004040 con 0x7fbfe4135c20 2023-09-30T22:21:34.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.301+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe409c510 con 0x7fbfe4135c20 2023-09-30T22:21:34.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.302+0000 7fbfdaffd700 1 -- 172.21.15.89:0/2612926938 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbfdc0041a0 con 0x7fbfe4135c20 2023-09-30T22:21:34.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.302+0000 7fbfdaffd700 1 -- 172.21.15.89:0/2612926938 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbfdc022070 con 0x7fbfe4135c20 2023-09-30T22:21:34.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.302+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe4129fe0 con 0x7fbfe4135c20 2023-09-30T22:21:34.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.311+0000 7fbfdaffd700 1 -- 172.21.15.89:0/2612926938 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbfdc046060 con 0x7fbfe4135c20 2023-09-30T22:21:34.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.342+0000 7f531617d700 1 Processor -- start 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.342+0000 7f531617d700 1 -- start start 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f531617d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f531009d6f0 0x7f531009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f531617d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009e000 0x7f5310130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f531617d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531009f130 con 0x7f531009e410 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f531617d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531009f520 con 0x7f531009d6f0 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f531617d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531009e930 con 0x7f531009e000 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f530f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f531009d6f0 0x7f531009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f530f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f531009d6f0 0x7f531009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52274/0 (socket says 172.21.15.89:52274) 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f530f7fe700 1 -- 172.21.15.89:0/2974507968 learned_addr learned my addr 172.21.15.89:0/2974507968 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:34.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f530effd700 1 --2- 172.21.15.89:0/2974507968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009e000 0x7f5310130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f530f7fe700 1 -- 172.21.15.89:0/2974507968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009e000 msgr2=0x7f5310130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f530f7fe700 1 --2- 172.21.15.89:0/2974507968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009e000 0x7f5310130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f530f7fe700 1 -- 172.21.15.89:0/2974507968 >> v1:172.21.15.89:6789/0 conn(0x7f531009e410 legacy=0x7f5310130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.343+0000 7f530f7fe700 1 -- 172.21.15.89:0/2974507968 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531009b2d0 con 0x7f531009d6f0 2023-09-30T22:21:34.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.344+0000 7f530f7fe700 1 --2- 172.21.15.89:0/2974507968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009d6f0 0x7f531009dac0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f530000bb90 tx=0x7f530000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10523da34d50ce28 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.344+0000 7f530e7fc700 1 -- 172.21.15.89:0/2974507968 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000dd30 con 0x7f531009d6f0 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.344+0000 7f530e7fc700 1 -- 172.21.15.89:0/2974507968 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530000de90 con 0x7f531009d6f0 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.344+0000 7f530e7fc700 1 -- 172.21.15.89:0/2974507968 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300010690 con 0x7f531009d6f0 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.345+0000 7f531617d700 1 -- 172.21.15.89:0/2974507968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009d6f0 msgr2=0x7f531009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.345+0000 7f531617d700 1 --2- 172.21.15.89:0/2974507968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009d6f0 0x7f531009dac0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f530000bb90 tx=0x7f530000bea0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.345+0000 7f531617d700 1 -- 172.21.15.89:0/2974507968 shutdown_connections 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.345+0000 7f531617d700 1 --2- 172.21.15.89:0/2974507968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009d6f0 0x7f531009dac0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.345+0000 7f531617d700 1 --2- 172.21.15.89:0/2974507968 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009e000 0x7f5310130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.345+0000 7f531617d700 1 -- 172.21.15.89:0/2974507968 >> 172.21.15.89:0/2974507968 conn(0x7f5310066f50 msgr2=0x7f531009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.345+0000 7f531617d700 1 -- 172.21.15.89:0/2974507968 shutdown_connections 2023-09-30T22:21:34.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.345+0000 7f531617d700 1 -- 172.21.15.89:0/2974507968 wait complete. 2023-09-30T22:21:34.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.346+0000 7f531617d700 1 Processor -- start 2023-09-30T22:21:34.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.346+0000 7f531617d700 1 -- start start 2023-09-30T22:21:34.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f531617d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009d6f0 0x7f531012fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f531617d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009e000 0x7f53101302b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f531617d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531009e410 0x7f53101307f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:34.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f531617d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53101c0cd0 con 0x7f531009e000 2023-09-30T22:21:34.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f531617d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53101be180 con 0x7f531009d6f0 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f531617d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53100a1d00 con 0x7f531009e410 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009d6f0 0x7f531012fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009d6f0 0x7f531012fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52086/0 (socket says 172.21.15.89:52086) 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530f7fe700 1 -- 172.21.15.89:0/425746698 learned_addr learned my addr 172.21.15.89:0/425746698 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530ffff700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531009e410 0x7f53101307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530effd700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009e000 0x7f53101302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530f7fe700 1 -- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531009e410 msgr2=0x7f53101307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530f7fe700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531009e410 0x7f53101307f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530f7fe700 1 -- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009e000 msgr2=0x7f53101302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530f7fe700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009e000 0x7f53101302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.347+0000 7f530f7fe700 1 -- 172.21.15.89:0/425746698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530000b7f0 con 0x7f531009d6f0 2023-09-30T22:21:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.348+0000 7f530f7fe700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009d6f0 0x7f531012fd70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5300000c00 tx=0x7f5300004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.348+0000 7f52f7fff700 1 -- 172.21.15.89:0/425746698 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300003b70 con 0x7f531009d6f0 2023-09-30T22:21:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.348+0000 7f52f7fff700 1 -- 172.21.15.89:0/425746698 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5300003cd0 con 0x7f531009d6f0 2023-09-30T22:21:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.348+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531009c700 con 0x7f531009d6f0 2023-09-30T22:21:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.348+0000 7f52f7fff700 1 -- 172.21.15.89:0/425746698 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000106b0 con 0x7f531009d6f0 2023-09-30T22:21:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.348+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53101bd050 con 0x7f531009d6f0 2023-09-30T22:21:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.349+0000 7f52f7fff700 1 -- 172.21.15.89:0/425746698 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5300019400 con 0x7f531009d6f0 2023-09-30T22:21:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.349+0000 7f52f7fff700 1 -- 172.21.15.89:0/425746698 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5300040a30 con 0x7f531009d6f0 2023-09-30T22:21:34.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.349+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53101bee20 con 0x7f531009d6f0 2023-09-30T22:21:34.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.354+0000 7f52f7fff700 1 -- 172.21.15.89:0/425746698 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5300003e40 con 0x7f531009d6f0 2023-09-30T22:21:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.441+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f085806c360 con 0x7f085809d6f0 2023-09-30T22:21:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.441+0000 7f084e7fc700 1 -- 172.21.15.89:0/3099983934 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0854017030 con 0x7f085809d6f0 2023-09-30T22:21:34.442 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:34.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.443+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f085809d6f0 msgr2=0x7f08581c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.443+0000 7f0860213700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f085809d6f0 0x7f08581c8560 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f085400c4a0 tx=0x7f085400c7b0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.444+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 shutdown_connections 2023-09-30T22:21:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.444+0000 7f0860213700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0858139170 0x7f085811e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.444+0000 7f0860213700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f085809d6f0 0x7f08581c8560 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.444+0000 7f0860213700 1 --2- 172.21.15.89:0/3099983934 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f085809e1d0 0x7f08581c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.444+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 >> 172.21.15.89:0/3099983934 conn(0x7f0858066f50 msgr2=0x7f0858136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.444+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 shutdown_connections 2023-09-30T22:21:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.444+0000 7f0860213700 1 -- 172.21.15.89:0/3099983934 wait complete. 2023-09-30T22:21:34.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:34.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.493+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f555011df50 con 0x7f5550137f20 2023-09-30T22:21:34.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.493+0000 7f55467fc700 1 -- 172.21.15.89:0/2163632469 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5548019300 con 0x7f5550137f20 2023-09-30T22:21:34.494 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:34.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5550137f20 msgr2=0x7f555011a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 --2- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5550137f20 0x7f555011a680 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f554800cb90 tx=0x7f554800cea0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 shutdown_connections 2023-09-30T22:21:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 --2- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5550137f20 0x7f555011a680 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 --2- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f555006c0e0 0x7f55501ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 --2- 172.21.15.89:0/2163632469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5550134ec0 0x7f55501cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 >> 172.21.15.89:0/2163632469 conn(0x7f55500a4210 msgr2=0x7f55500a0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 shutdown_connections 2023-09-30T22:21:34.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.496+0000 7f5558089700 1 -- 172.21.15.89:0/2163632469 wait complete. 2023-09-30T22:21:34.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:34.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.510+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbfe4074ea0 con 0x7fbfe4135c20 2023-09-30T22:21:34.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.510+0000 7fbfdaffd700 1 -- 172.21.15.89:0/2612926938 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbfdc027480 con 0x7fbfe4135c20 2023-09-30T22:21:34.511 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:34.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 msgr2=0x7fbfe41c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe41c4880 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc016000 tx=0x7fbfdc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 shutdown_connections 2023-09-30T22:21:34.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbfe4135c20 0x7fbfe41c4880 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfe406f5d0 0x7fbfe41c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 --2- 172.21.15.89:0/2612926938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfe412cf60 0x7fbfe41c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 >> 172.21.15.89:0/2612926938 conn(0x7fbfe40a2aa0 msgr2=0x7fbfe412a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 shutdown_connections 2023-09-30T22:21:34.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.512+0000 7fbfec368700 1 -- 172.21.15.89:0/2612926938 wait complete. 2023-09-30T22:21:34.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:34.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.568+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f53101beba0 con 0x7f531009d6f0 2023-09-30T22:21:34.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.569+0000 7f52f7fff700 1 -- 172.21.15.89:0/425746698 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5300018360 con 0x7f531009d6f0 2023-09-30T22:21:34.570 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:34.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009d6f0 msgr2=0x7f531012fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009d6f0 0x7f531012fd70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5300000c00 tx=0x7f5300004750 comp rx=0 tx=0).stop 2023-09-30T22:21:34.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 shutdown_connections 2023-09-30T22:21:34.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f531009e410 0x7f53101307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f531009d6f0 0x7f531012fd70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 --2- 172.21.15.89:0/425746698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531009e000 0x7f53101302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:34.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 >> 172.21.15.89:0/425746698 conn(0x7f5310066f50 msgr2=0x7f531006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 shutdown_connections 2023-09-30T22:21:34.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:34.571+0000 7f531617d700 1 -- 172.21.15.89:0/425746698 wait complete. 2023-09-30T22:21:34.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:35.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:35.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:35.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:35.585 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.593+0000 7fdc4a70c700 1 Processor -- start 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.593+0000 7fdc4a70c700 1 -- start start 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc4a70c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc4409d6f0 0x7fdc4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc4a70c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc44134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc4a70c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4409b120 con 0x7fdc4409e000 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc4a70c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4406b460 con 0x7fdc4409d6f0 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc4a70c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4409a7e0 con 0x7fdc4409e440 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc43fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc4409d6f0 0x7fdc4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc48ca9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc44134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc43fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdc4409d6f0 0x7fdc4409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52288/0 (socket says 172.21.15.89:52288) 2023-09-30T22:21:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc43fff700 1 -- 172.21.15.89:0/1584623462 learned_addr learned my addr 172.21.15.89:0/1584623462 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc43fff700 1 -- 172.21.15.89:0/1584623462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 msgr2=0x7fdc44134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc43fff700 1 --2- 172.21.15.89:0/1584623462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc44134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc43fff700 1 -- 172.21.15.89:0/1584623462 >> v1:172.21.15.89:6789/0 conn(0x7fdc4409e000 legacy=0x7fdc441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc43fff700 1 -- 172.21.15.89:0/1584623462 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4409a500 con 0x7fdc4409d6f0 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.594+0000 7fdc42ffd700 1 -- 172.21.15.89:0/1584623462 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623361355 0 0) 0x7fdc4409b120 con 0x7fdc4409e000 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.595+0000 7fdc43fff700 1 --2- 172.21.15.89:0/1584623462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc4409dac0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009920 tx=0x7fdc34009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c116677e4772baf8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.595+0000 7fdc42ffd700 1 -- 172.21.15.89:0/1584623462 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34011830 con 0x7fdc4409d6f0 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.595+0000 7fdc42ffd700 1 -- 172.21.15.89:0/1584623462 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc34011990 con 0x7fdc4409d6f0 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.595+0000 7fdc42ffd700 1 -- 172.21.15.89:0/1584623462 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34011d20 con 0x7fdc4409d6f0 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.595+0000 7fdc4a70c700 1 -- 172.21.15.89:0/1584623462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 msgr2=0x7fdc4409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.595+0000 7fdc4a70c700 1 --2- 172.21.15.89:0/1584623462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc4409dac0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009920 tx=0x7fdc34009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.596+0000 7fdc4a70c700 1 -- 172.21.15.89:0/1584623462 shutdown_connections 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.596+0000 7fdc4a70c700 1 --2- 172.21.15.89:0/1584623462 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc44134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.596+0000 7fdc4a70c700 1 --2- 172.21.15.89:0/1584623462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc4409dac0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.596+0000 7fdc4a70c700 1 -- 172.21.15.89:0/1584623462 >> 172.21.15.89:0/1584623462 conn(0x7fdc44066f50 msgr2=0x7fdc4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.596+0000 7fdc4a70c700 1 -- 172.21.15.89:0/1584623462 shutdown_connections 2023-09-30T22:21:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.596+0000 7fdc4a70c700 1 -- 172.21.15.89:0/1584623462 wait complete. 2023-09-30T22:21:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.597+0000 7fdc4a70c700 1 Processor -- start 2023-09-30T22:21:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.597+0000 7fdc4a70c700 1 -- start start 2023-09-30T22:21:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc4a70c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc441c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc4a70c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc4409e000 0x7fdc441c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc4a70c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc441c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc4a70c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc441a4c90 con 0x7fdc4409d6f0 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc4a70c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc441a8100 con 0x7fdc4409e440 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc4a70c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4409c710 con 0x7fdc4409e000 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc441c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc441c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52298/0 (socket says 172.21.15.89:52298) 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 -- 172.21.15.89:0/13455644 learned_addr learned my addr 172.21.15.89:0/13455644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc48ca9700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc441c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 -- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc4409e000 msgr2=0x7fdc441c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc4409e000 0x7fdc441c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 -- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 msgr2=0x7fdc441c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc441c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 -- 172.21.15.89:0/13455644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34009550 con 0x7fdc4409d6f0 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc48ca9700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc441c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.598+0000 7fdc43fff700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc441c8090 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fdc34000c00 tx=0x7fdc34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.599+0000 7fdc40ff9700 1 -- 172.21.15.89:0/13455644 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34004380 con 0x7fdc4409d6f0 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.599+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc441a63f0 con 0x7fdc4409d6f0 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.599+0000 7fdc40ff9700 1 -- 172.21.15.89:0/13455644 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3401f070 con 0x7fdc4409d6f0 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.599+0000 7fdc40ff9700 1 -- 172.21.15.89:0/13455644 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340036a0 con 0x7fdc4409d6f0 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.599+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc441a7060 con 0x7fdc4409d6f0 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.600+0000 7fdc40ff9700 1 -- 172.21.15.89:0/13455644 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc34021030 con 0x7fdc4409d6f0 2023-09-30T22:21:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.600+0000 7fdc40ff9700 1 -- 172.21.15.89:0/13455644 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdc34040910 con 0x7fdc4409d6f0 2023-09-30T22:21:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.600+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc44074ea0 con 0x7fdc4409d6f0 2023-09-30T22:21:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.608+0000 7fdc40ff9700 1 -- 172.21.15.89:0/13455644 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc34015080 con 0x7fdc4409d6f0 2023-09-30T22:21:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.645+0000 7f5529034700 1 Processor -- start 2023-09-30T22:21:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.645+0000 7f5529034700 1 -- start start 2023-09-30T22:21:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f5529034700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f552412cf60 0x7f552406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f5529034700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f552412f8b0 0x7f552412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f5529034700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552412cd90 con 0x7f5524130260 2023-09-30T22:21:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f5529034700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552412d330 con 0x7f552412cf60 2023-09-30T22:21:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f5529034700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55241306d0 con 0x7f552412f8b0 2023-09-30T22:21:35.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f5522d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f552412cf60 0x7f552406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f552259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f552412f8b0 0x7f552412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f5522d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f552412cf60 0x7f552406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52306/0 (socket says 172.21.15.89:52306) 2023-09-30T22:21:35.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f552259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f552412f8b0 0x7f552412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52114/0 (socket says 172.21.15.89:52114) 2023-09-30T22:21:35.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.646+0000 7f5522d9d700 1 -- 172.21.15.89:0/3457864644 learned_addr learned my addr 172.21.15.89:0/3457864644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:35.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.647+0000 7f5522d9d700 1 -- 172.21.15.89:0/3457864644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f552412f8b0 msgr2=0x7f552412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:35.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.647+0000 7f5522d9d700 1 --2- 172.21.15.89:0/3457864644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f552412f8b0 0x7f552412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.647+0000 7f5522d9d700 1 -- 172.21.15.89:0/3457864644 >> v1:172.21.15.89:6789/0 conn(0x7f5524130260 legacy=0x7f5524136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.647+0000 7f5522d9d700 1 -- 172.21.15.89:0/3457864644 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524071780 con 0x7f552412cf60 2023-09-30T22:21:35.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.647+0000 7f5522d9d700 1 --2- 172.21.15.89:0/3457864644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412cf60 0x7f552406f5d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f551400bb90 tx=0x7f551400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7081bf80edf16f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.647+0000 7f5521d9b700 1 -- 172.21.15.89:0/3457864644 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400dc20 con 0x7f552412cf60 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.647+0000 7f5521d9b700 1 -- 172.21.15.89:0/3457864644 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551400dd80 con 0x7f552412cf60 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.647+0000 7f5521d9b700 1 -- 172.21.15.89:0/3457864644 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514017630 con 0x7f552412cf60 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.648+0000 7f5529034700 1 -- 172.21.15.89:0/3457864644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412cf60 msgr2=0x7f552406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.648+0000 7f5529034700 1 --2- 172.21.15.89:0/3457864644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412cf60 0x7f552406f5d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f551400bb90 tx=0x7f551400bea0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.648+0000 7f5529034700 1 -- 172.21.15.89:0/3457864644 shutdown_connections 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.648+0000 7f5529034700 1 --2- 172.21.15.89:0/3457864644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412cf60 0x7f552406f5d0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.648+0000 7f5529034700 1 --2- 172.21.15.89:0/3457864644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f552412f8b0 0x7f552412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.648+0000 7f5529034700 1 -- 172.21.15.89:0/3457864644 >> 172.21.15.89:0/3457864644 conn(0x7f55240a2aa0 msgr2=0x7f55240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.648+0000 7f5529034700 1 -- 172.21.15.89:0/3457864644 shutdown_connections 2023-09-30T22:21:35.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.648+0000 7f5529034700 1 -- 172.21.15.89:0/3457864644 wait complete. 2023-09-30T22:21:35.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.649+0000 7f5529034700 1 Processor -- start 2023-09-30T22:21:35.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.649+0000 7f5529034700 1 -- start start 2023-09-30T22:21:35.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5529034700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552412cf60 0x7f55241c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5529034700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412f8b0 0x7f55241c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5529034700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524130260 0x7f55241c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5529034700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552409cd00 con 0x7f552412f8b0 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5529034700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552409e4f0 con 0x7f5524130260 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5529034700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f552412f520 con 0x7f552412cf60 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5522d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552412cf60 0x7f55241c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5522d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552412cf60 0x7f55241c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48964/0 (socket says 172.21.15.89:48964) 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f5522d9d700 1 -- 172.21.15.89:0/1842453701 learned_addr learned my addr 172.21.15.89:0/1842453701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f552359e700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524130260 0x7f55241c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.650+0000 7f552259c700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412f8b0 0x7f55241c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f552359e700 1 -- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552412cf60 msgr2=0x7f55241c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f552359e700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552412cf60 0x7f55241c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f552359e700 1 -- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412f8b0 msgr2=0x7f55241c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f552359e700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412f8b0 0x7f55241c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f552359e700 1 -- 172.21.15.89:0/1842453701 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551400b7f0 con 0x7f5524130260 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f5522d9d700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552412cf60 0x7f55241c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f552359e700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524130260 0x7f55241c4a40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f551c00cb90 tx=0x7f551c00cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f550b7fe700 1 -- 172.21.15.89:0/1842453701 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c003c00 con 0x7f5524130260 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f550b7fe700 1 -- 172.21.15.89:0/1842453701 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c003d60 con 0x7f5524130260 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5524129fa0 con 0x7f5524130260 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f550b7fe700 1 -- 172.21.15.89:0/1842453701 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c00aa80 con 0x7f5524130260 2023-09-30T22:21:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.651+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552409ea90 con 0x7f5524130260 2023-09-30T22:21:35.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.652+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552409e6b0 con 0x7f5524130260 2023-09-30T22:21:35.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.653+0000 7f550b7fe700 1 -- 172.21.15.89:0/1842453701 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f551c00abe0 con 0x7f5524130260 2023-09-30T22:21:35.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.653+0000 7f550b7fe700 1 -- 172.21.15.89:0/1842453701 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f551c011540 con 0x7f5524130260 2023-09-30T22:21:35.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.657+0000 7f550b7fe700 1 -- 172.21.15.89:0/1842453701 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f551c0198f0 con 0x7f5524130260 2023-09-30T22:21:35.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.693+0000 7fb1cffc9700 1 Processor -- start 2023-09-30T22:21:35.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.693+0000 7fb1cffc9700 1 -- start start 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1cffc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 0x7fb1c80a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1cffc9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a13e0 0x7fb1c813f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1cffc9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c80a1850 con 0x7fb1c80a0f70 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1cffc9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c80a2550 con 0x7fb1c80a13e0 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1cffc9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c80a1e00 con 0x7fb1c806cc90 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1cd564700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb1c80a0f70 0x7fb1c813c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43368/0 (socket says 172.21.15.89:43368) 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1cd564700 1 -- 172.21.15.89:0/1458324537 learned_addr learned my addr 172.21.15.89:0/1458324537 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1ce566700 1 --2- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a13e0 0x7fb1c813f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.694+0000 7fb1cdd65700 1 --2- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 0x7fb1c80a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ce566700 1 -- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 msgr2=0x7fb1c80a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ce566700 1 --2- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 0x7fb1c80a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ce566700 1 -- 172.21.15.89:0/1458324537 >> v1:172.21.15.89:6789/0 conn(0x7fb1c80a0f70 legacy=0x7fb1c813c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ccd63700 1 -- 172.21.15.89:0/1458324537 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361798299 0 0) 0x7fb1c80a1850 con 0x7fb1c80a0f70 2023-09-30T22:21:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ce566700 1 -- 172.21.15.89:0/1458324537 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c809cfa0 con 0x7fb1c80a13e0 2023-09-30T22:21:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ce566700 1 --2- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a13e0 0x7fb1c813f850 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009dc0 tx=0x7fb1c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e640f309ceea88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ccd63700 1 -- 172.21.15.89:0/1458324537 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c000e070 con 0x7fb1c80a13e0 2023-09-30T22:21:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ccd63700 1 -- 172.21.15.89:0/1458324537 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c000bb20 con 0x7fb1c80a13e0 2023-09-30T22:21:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.695+0000 7fb1ccd63700 1 -- 172.21.15.89:0/1458324537 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0012430 con 0x7fb1c80a13e0 2023-09-30T22:21:35.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.696+0000 7fb1cffc9700 1 -- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a13e0 msgr2=0x7fb1c813f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.696+0000 7fb1cffc9700 1 --2- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a13e0 0x7fb1c813f850 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009dc0 tx=0x7fb1c000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:35.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.696+0000 7fb1cffc9700 1 -- 172.21.15.89:0/1458324537 shutdown_connections 2023-09-30T22:21:35.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.696+0000 7fb1cffc9700 1 --2- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a13e0 0x7fb1c813f850 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.696+0000 7fb1cffc9700 1 --2- 172.21.15.89:0/1458324537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 0x7fb1c80a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.696+0000 7fb1cffc9700 1 -- 172.21.15.89:0/1458324537 >> 172.21.15.89:0/1458324537 conn(0x7fb1c80a3050 msgr2=0x7fb1c80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.696+0000 7fb1cffc9700 1 -- 172.21.15.89:0/1458324537 shutdown_connections 2023-09-30T22:21:35.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.696+0000 7fb1cffc9700 1 -- 172.21.15.89:0/1458324537 wait complete. 2023-09-30T22:21:35.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.697+0000 7fb1cffc9700 1 Processor -- start 2023-09-30T22:21:35.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.697+0000 7fb1cffc9700 1 -- start start 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cffc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 0x7fb1c81cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cffc9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a0f70 0x7fb1c81cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cffc9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1c80a13e0 0x7fb1c81ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cffc9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c80a0070 con 0x7fb1c80a13e0 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cffc9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c81bfa20 con 0x7fb1c80a0f70 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cffc9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c81acc20 con 0x7fb1c806cc90 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cd564700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a0f70 0x7fb1c81cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cd564700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a0f70 0x7fb1c81cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52138/0 (socket says 172.21.15.89:52138) 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cd564700 1 -- 172.21.15.89:0/2862915925 learned_addr learned my addr 172.21.15.89:0/2862915925 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1cdd65700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 0x7fb1c81cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.698+0000 7fb1ce566700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1c80a13e0 0x7fb1c81ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1cd564700 1 -- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 msgr2=0x7fb1c81cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1cd564700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 0x7fb1c81cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1cd564700 1 -- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1c80a13e0 msgr2=0x7fb1c81ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1cd564700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1c80a13e0 0x7fb1c81ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1cd564700 1 -- 172.21.15.89:0/2862915925 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c0009a70 con 0x7fb1c80a0f70 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1cd564700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a0f70 0x7fb1c81cc980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8001eb0 tx=0x7fb1b800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1be7fc700 1 -- 172.21.15.89:0/2862915925 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8004c80 con 0x7fb1c80a0f70 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1be7fc700 1 -- 172.21.15.89:0/2862915925 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8004de0 con 0x7fb1c80a0f70 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c81ae460 con 0x7fb1c80a0f70 2023-09-30T22:21:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1be7fc700 1 -- 172.21.15.89:0/2862915925 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8007310 con 0x7fb1c80a0f70 2023-09-30T22:21:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.699+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c81af190 con 0x7fb1c80a0f70 2023-09-30T22:21:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.700+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c8071e20 con 0x7fb1c80a0f70 2023-09-30T22:21:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.701+0000 7fb1be7fc700 1 -- 172.21.15.89:0/2862915925 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb1b8007470 con 0x7fb1c80a0f70 2023-09-30T22:21:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.701+0000 7fb1be7fc700 1 -- 172.21.15.89:0/2862915925 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb1b803f8d0 con 0x7fb1c80a0f70 2023-09-30T22:21:35.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.705+0000 7fb1be7fc700 1 -- 172.21.15.89:0/2862915925 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb1b80133c0 con 0x7fb1c80a0f70 2023-09-30T22:21:35.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.731+0000 7f1cd77ed700 1 Processor -- start 2023-09-30T22:21:35.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.731+0000 7f1cd77ed700 1 -- start start 2023-09-30T22:21:35.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd77ed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd00a0a30 0x7f1cd00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd77ed700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cd009d630 0x7f1cd00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd77ed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0133930 0x7f1cd0133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd77ed700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd00a2a70 con 0x7f1cd009d630 2023-09-30T22:21:35.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd77ed700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00a01d0 con 0x7f1cd0133930 2023-09-30T22:21:35.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd77ed700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd009e2b0 con 0x7f1cd00a0a30 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd4d88700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cd009d630 0x7f1cd00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd4d88700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cd009d630 0x7f1cd00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52340/0 (socket says 172.21.15.89:52340) 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd4d88700 1 -- 172.21.15.89:0/1284499211 learned_addr learned my addr 172.21.15.89:0/1284499211 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd5d8a700 1 --2- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0133930 0x7f1cd0133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd5589700 1 --2- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd00a0a30 0x7f1cd00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd5589700 1 -- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0133930 msgr2=0x7f1cd0133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd5589700 1 --2- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0133930 0x7f1cd0133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd5589700 1 -- 172.21.15.89:0/1284499211 >> v2:172.21.15.89:3300/0 conn(0x7f1cd009d630 msgr2=0x7f1cd00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.732+0000 7f1cd5589700 1 --2- 172.21.15.89:0/1284499211 >> v2:172.21.15.89:3300/0 conn(0x7f1cd009d630 0x7f1cd00a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.733+0000 7f1cd5589700 1 -- 172.21.15.89:0/1284499211 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd009dc50 con 0x7f1cd00a0a30 2023-09-30T22:21:35.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.733+0000 7f1cd4d88700 1 --2- 172.21.15.89:0/1284499211 >> v2:172.21.15.89:3300/0 conn(0x7f1cd009d630 0x7f1cd00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:35.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.733+0000 7f1cd5589700 1 --2- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd00a0a30 0x7f1cd00a0e00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009840 tx=0x7f1ccc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd570ab185a55f56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.733+0000 7f1cc7fff700 1 -- 172.21.15.89:0/1284499211 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011830 con 0x7f1cd00a0a30 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.733+0000 7f1cc7fff700 1 -- 172.21.15.89:0/1284499211 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc011990 con 0x7f1cd00a0a30 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.733+0000 7f1cc7fff700 1 -- 172.21.15.89:0/1284499211 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011d20 con 0x7f1cd00a0a30 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.733+0000 7f1cd77ed700 1 -- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd00a0a30 msgr2=0x7f1cd00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.733+0000 7f1cd77ed700 1 --2- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd00a0a30 0x7f1cd00a0e00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009840 tx=0x7f1ccc009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.734+0000 7f1cd77ed700 1 -- 172.21.15.89:0/1284499211 shutdown_connections 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.734+0000 7f1cd77ed700 1 --2- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd0133930 0x7f1cd0133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.734+0000 7f1cd77ed700 1 --2- 172.21.15.89:0/1284499211 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd00a0a30 0x7f1cd00a0e00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.734+0000 7f1cd77ed700 1 --2- 172.21.15.89:0/1284499211 >> v2:172.21.15.89:3300/0 conn(0x7f1cd009d630 0x7f1cd00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.734+0000 7f1cd77ed700 1 -- 172.21.15.89:0/1284499211 >> 172.21.15.89:0/1284499211 conn(0x7f1cd00a3050 msgr2=0x7f1cd00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.734+0000 7f1cd77ed700 1 -- 172.21.15.89:0/1284499211 shutdown_connections 2023-09-30T22:21:35.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.734+0000 7f1cd77ed700 1 -- 172.21.15.89:0/1284499211 wait complete. 2023-09-30T22:21:35.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.735+0000 7f1cd77ed700 1 Processor -- start 2023-09-30T22:21:35.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.735+0000 7f1cd77ed700 1 -- start start 2023-09-30T22:21:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd77ed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009d630 0x7f1cd00709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd77ed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd00a0a30 0x7f1cd0070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd77ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd0133930 0x7f1cd01cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd77ed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0071450 con 0x7f1cd0133930 2023-09-30T22:21:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd77ed700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01317e0 con 0x7f1cd00a0a30 2023-09-30T22:21:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd77ed700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00a0410 con 0x7f1cd009d630 2023-09-30T22:21:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd4d88700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd00a0a30 0x7f1cd0070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd4d88700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd00a0a30 0x7f1cd0070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52156/0 (socket says 172.21.15.89:52156) 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd4d88700 1 -- 172.21.15.89:0/878093955 learned_addr learned my addr 172.21.15.89:0/878093955 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd5d8a700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd0133930 0x7f1cd01cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd5589700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009d630 0x7f1cd00709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd4d88700 1 -- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009d630 msgr2=0x7f1cd00709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd4d88700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009d630 0x7f1cd00709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd4d88700 1 -- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd0133930 msgr2=0x7f1cd01cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd4d88700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd0133930 0x7f1cd01cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.736+0000 7f1cd4d88700 1 -- 172.21.15.89:0/878093955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc0009550 con 0x7f1cd00a0a30 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.737+0000 7f1cd4d88700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd00a0a30 0x7f1cd0070f10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000ebe0 tx=0x7f1cc000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.737+0000 7f1cc5ffb700 1 -- 172.21.15.89:0/878093955 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000c850 con 0x7f1cd00a0a30 2023-09-30T22:21:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.737+0000 7f1cc5ffb700 1 -- 172.21.15.89:0/878093955 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc000c9b0 con 0x7f1cd00a0a30 2023-09-30T22:21:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.737+0000 7f1cc5ffb700 1 -- 172.21.15.89:0/878093955 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000cd60 con 0x7f1cd00a0a30 2023-09-30T22:21:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.737+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc0094a0 con 0x7f1cd00a0a30 2023-09-30T22:21:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.737+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd0070620 con 0x7f1cd00a0a30 2023-09-30T22:21:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.738+0000 7f1cc5ffb700 1 -- 172.21.15.89:0/878093955 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1cc0013470 con 0x7f1cd00a0a30 2023-09-30T22:21:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.739+0000 7f1cc5ffb700 1 -- 172.21.15.89:0/878093955 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1cc003f890 con 0x7f1cd00a0a30 2023-09-30T22:21:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.739+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd00747f0 con 0x7f1cd00a0a30 2023-09-30T22:21:35.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.743+0000 7f1cc5ffb700 1 -- 172.21.15.89:0/878093955 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1cc0021080 con 0x7f1cd00a0a30 2023-09-30T22:21:35.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.831+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdc440a1f70 con 0x7fdc4409d6f0 2023-09-30T22:21:35.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.831+0000 7fdc40ff9700 1 -- 172.21.15.89:0/13455644 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc340239c0 con 0x7fdc4409d6f0 2023-09-30T22:21:35.832 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 msgr2=0x7fdc441c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc441c8090 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fdc34000c00 tx=0x7fdc34004750 comp rx=0 tx=0).stop 2023-09-30T22:21:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 shutdown_connections 2023-09-30T22:21:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc4409e440 0x7fdc441c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc4409d6f0 0x7fdc441c8090 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 --2- 172.21.15.89:0/13455644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc4409e000 0x7fdc441c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 >> 172.21.15.89:0/13455644 conn(0x7fdc44066f50 msgr2=0x7fdc4406f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 shutdown_connections 2023-09-30T22:21:35.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.833+0000 7fdc4a70c700 1 -- 172.21.15.89:0/13455644 wait complete. 2023-09-30T22:21:35.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:35.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.858+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5524074ea0 con 0x7f5524130260 2023-09-30T22:21:35.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.858+0000 7f550b7fe700 1 -- 172.21.15.89:0/1842453701 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f551c015120 con 0x7f5524130260 2023-09-30T22:21:35.859 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:35.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.860+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524130260 msgr2=0x7f55241c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.860+0000 7f5529034700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524130260 0x7f55241c4a40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f551c00cb90 tx=0x7f551c00cea0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.861+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 shutdown_connections 2023-09-30T22:21:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.861+0000 7f5529034700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524130260 0x7f55241c4a40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.861+0000 7f5529034700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552412cf60 0x7f55241c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.861+0000 7f5529034700 1 --2- 172.21.15.89:0/1842453701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552412f8b0 0x7f55241c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.861+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 >> 172.21.15.89:0/1842453701 conn(0x7f55240a2aa0 msgr2=0x7f552406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.861+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 shutdown_connections 2023-09-30T22:21:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.861+0000 7f5529034700 1 -- 172.21.15.89:0/1842453701 wait complete. 2023-09-30T22:21:35.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.900+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb1c80747f0 con 0x7fb1c80a0f70 2023-09-30T22:21:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.900+0000 7fb1be7fc700 1 -- 172.21.15.89:0/2862915925 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1b8011360 con 0x7fb1c80a0f70 2023-09-30T22:21:35.901 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a0f70 msgr2=0x7fb1c81cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a0f70 0x7fb1c81cc980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8001eb0 tx=0x7fb1b800c800 comp rx=0 tx=0).stop 2023-09-30T22:21:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 shutdown_connections 2023-09-30T22:21:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1c80a13e0 0x7fb1c81ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1c806cc90 0x7fb1c81cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 --2- 172.21.15.89:0/2862915925 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1c80a0f70 0x7fb1c81cc980 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 >> 172.21.15.89:0/2862915925 conn(0x7fb1c80a3050 msgr2=0x7fb1c809a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 shutdown_connections 2023-09-30T22:21:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.903+0000 7fb1cffc9700 1 -- 172.21.15.89:0/2862915925 wait complete. 2023-09-30T22:21:35.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:35.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.942+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1cd0072980 con 0x7f1cd00a0a30 2023-09-30T22:21:35.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.942+0000 7f1cc5ffb700 1 -- 172.21.15.89:0/878093955 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1cc00232f0 con 0x7f1cd00a0a30 2023-09-30T22:21:35.943 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:35.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.944+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd00a0a30 msgr2=0x7f1cd0070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.944+0000 7f1cd77ed700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd00a0a30 0x7f1cd0070f10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000ebe0 tx=0x7f1cc000eef0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.945+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 shutdown_connections 2023-09-30T22:21:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.945+0000 7f1cd77ed700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cd0133930 0x7f1cd01cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.945+0000 7f1cd77ed700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1cd009d630 0x7f1cd00709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.945+0000 7f1cd77ed700 1 --2- 172.21.15.89:0/878093955 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1cd00a0a30 0x7f1cd0070f10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.945+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 >> 172.21.15.89:0/878093955 conn(0x7f1cd00a3050 msgr2=0x7f1cd009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.945+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 shutdown_connections 2023-09-30T22:21:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:35.945+0000 7f1cd77ed700 1 -- 172.21.15.89:0/878093955 wait complete. 2023-09-30T22:21:35.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:36.848 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:36.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:36.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:36.957 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:36.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.995+0000 7fcb493d8700 1 Processor -- start 2023-09-30T22:21:36.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.995+0000 7fcb493d8700 1 -- start start 2023-09-30T22:21:36.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb493d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb44131810 0x7fcb44131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:36.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb493d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb44099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:36.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb493d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb441313c0 con 0x7fcb4409a3b0 2023-09-30T22:21:36.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb493d8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4409b580 con 0x7fcb44131810 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb493d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4406e8c0 con 0x7fcb44099a40 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb44099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb44099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52170/0 (socket says 172.21.15.89:52170) 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb427fc700 1 -- 172.21.15.89:0/265800391 learned_addr learned my addr 172.21.15.89:0/265800391 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.996+0000 7fcb42ffd700 1 --2- 172.21.15.89:0/265800391 >> v2:172.21.15.89:3300/0 conn(0x7fcb44131810 0x7fcb44131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.997+0000 7fcb427fc700 1 -- 172.21.15.89:0/265800391 >> v2:172.21.15.89:3300/0 conn(0x7fcb44131810 msgr2=0x7fcb44131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.997+0000 7fcb427fc700 1 --2- 172.21.15.89:0/265800391 >> v2:172.21.15.89:3300/0 conn(0x7fcb44131810 0x7fcb44131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.997+0000 7fcb427fc700 1 -- 172.21.15.89:0/265800391 >> v1:172.21.15.89:6789/0 conn(0x7fcb4409a3b0 legacy=0x7fcb44138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.997+0000 7fcb427fc700 1 -- 172.21.15.89:0/265800391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4409a830 con 0x7fcb44099a40 2023-09-30T22:21:36.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.997+0000 7fcb427fc700 1 --2- 172.21.15.89:0/265800391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb44099e70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009920 tx=0x7fcb38009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93bdec29e4c2253b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.997+0000 7fcb41ffb700 1 -- 172.21.15.89:0/265800391 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38011830 con 0x7fcb44099a40 2023-09-30T22:21:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.997+0000 7fcb41ffb700 1 -- 172.21.15.89:0/265800391 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb38011990 con 0x7fcb44099a40 2023-09-30T22:21:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.997+0000 7fcb41ffb700 1 -- 172.21.15.89:0/265800391 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38011d20 con 0x7fcb44099a40 2023-09-30T22:21:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.998+0000 7fcb493d8700 1 -- 172.21.15.89:0/265800391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 msgr2=0x7fcb44099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.998+0000 7fcb493d8700 1 --2- 172.21.15.89:0/265800391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb44099e70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009920 tx=0x7fcb38009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.998+0000 7fcb493d8700 1 -- 172.21.15.89:0/265800391 shutdown_connections 2023-09-30T22:21:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.998+0000 7fcb493d8700 1 --2- 172.21.15.89:0/265800391 >> v2:172.21.15.89:3300/0 conn(0x7fcb44131810 0x7fcb44131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.998+0000 7fcb493d8700 1 --2- 172.21.15.89:0/265800391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb44099e70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.998+0000 7fcb493d8700 1 -- 172.21.15.89:0/265800391 >> 172.21.15.89:0/265800391 conn(0x7fcb440a2aa0 msgr2=0x7fcb440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.998+0000 7fcb493d8700 1 -- 172.21.15.89:0/265800391 shutdown_connections 2023-09-30T22:21:37.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.998+0000 7fcb493d8700 1 -- 172.21.15.89:0/265800391 wait complete. 2023-09-30T22:21:37.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.999+0000 7fcb493d8700 1 Processor -- start 2023-09-30T22:21:37.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:36.999+0000 7fcb493d8700 1 -- start start 2023-09-30T22:21:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb493d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb4411b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb493d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb4409a3b0 0x7fcb4411baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb493d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44131810 0x7fcb4411c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb493d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4411f0d0 con 0x7fcb44131810 2023-09-30T22:21:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb493d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4411e480 con 0x7fcb44099a40 2023-09-30T22:21:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb493d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4411c9d0 con 0x7fcb4409a3b0 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb4409a3b0 0x7fcb4411baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb4409a3b0 0x7fcb4411baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49002/0 (socket says 172.21.15.89:49002) 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb427fc700 1 -- 172.21.15.89:0/650889837 learned_addr learned my addr 172.21.15.89:0/650889837 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb42ffd700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb4411b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb437fe700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44131810 0x7fcb4411c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb42ffd700 1 -- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb4409a3b0 msgr2=0x7fcb4411baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb42ffd700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb4409a3b0 0x7fcb4411baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb42ffd700 1 -- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44131810 msgr2=0x7fcb4411c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb42ffd700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44131810 0x7fcb4411c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb42ffd700 1 -- 172.21.15.89:0/650889837 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38009550 con 0x7fcb44099a40 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb427fc700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb4409a3b0 0x7fcb4411baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.000+0000 7fcb437fe700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44131810 0x7fcb4411c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.001+0000 7fcb42ffd700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb4411b5b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400e900 tx=0x7fcb3400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.001+0000 7fcb2b7fe700 1 -- 172.21.15.89:0/650889837 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400ee50 con 0x7fcb44099a40 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.001+0000 7fcb2b7fe700 1 -- 172.21.15.89:0/650889837 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3400c850 con 0x7fcb44099a40 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.001+0000 7fcb2b7fe700 1 -- 172.21.15.89:0/650889837 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400cbc0 con 0x7fcb44099a40 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.001+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb440a1b80 con 0x7fcb44099a40 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.001+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4411dfc0 con 0x7fcb44099a40 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.002+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb44074ea0 con 0x7fcb44099a40 2023-09-30T22:21:37.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.002+0000 7fcb2b7fe700 1 -- 172.21.15.89:0/650889837 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb3400cd20 con 0x7fcb44099a40 2023-09-30T22:21:37.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.002+0000 7fcb2b7fe700 1 -- 172.21.15.89:0/650889837 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb3403f7c0 con 0x7fcb44099a40 2023-09-30T22:21:37.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.011+0000 7fcb2b7fe700 1 -- 172.21.15.89:0/650889837 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb340232f0 con 0x7fcb44099a40 2023-09-30T22:21:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.040+0000 7f55222d8700 1 Processor -- start 2023-09-30T22:21:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.040+0000 7f55222d8700 1 -- start start 2023-09-30T22:21:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f55222d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1338c0 0x7f551c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f55222d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09e560 0x7f551c13b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f55222d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c06f720 con 0x7f551c1341d0 2023-09-30T22:21:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f55222d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c070c80 con 0x7f551c1338c0 2023-09-30T22:21:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f55222d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f551c06fce0 con 0x7f551c09e560 2023-09-30T22:21:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f5520875700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09e560 0x7f551c13b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f5520875700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09e560 0x7f551c13b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49006/0 (socket says 172.21.15.89:49006) 2023-09-30T22:21:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f5520875700 1 -- 172.21.15.89:0/2198878278 learned_addr learned my addr 172.21.15.89:0/2198878278 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f551b7fe700 1 --1- 172.21.15.89:0/2198878278 >> v1:172.21.15.89:6789/0 conn(0x7f551c1341d0 0x7f551c09de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43380/0 (socket says 172.21.15.89:43380) 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f551affd700 1 -- 172.21.15.89:0/2198878278 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664819332 0 0) 0x7f551c06f720 con 0x7f551c1341d0 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f551affd700 1 -- 172.21.15.89:0/2198878278 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55040035b0 con 0x7f551c1341d0 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f5520875700 1 -- 172.21.15.89:0/2198878278 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1338c0 msgr2=0x7f551c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f5520875700 1 --2- 172.21.15.89:0/2198878278 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1338c0 0x7f551c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f5520875700 1 -- 172.21.15.89:0/2198878278 >> v1:172.21.15.89:6789/0 conn(0x7f551c1341d0 legacy=0x7f551c09de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.041+0000 7f5520875700 1 -- 172.21.15.89:0/2198878278 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c1336a0 con 0x7f551c09e560 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f5520875700 1 --2- 172.21.15.89:0/2198878278 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09e560 0x7f551c13b860 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5514009e80 tx=0x7f551400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=659796eef00f9e8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f551affd700 1 -- 172.21.15.89:0/2198878278 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400e070 con 0x7f551c09e560 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f551affd700 1 -- 172.21.15.89:0/2198878278 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551400b910 con 0x7f551c09e560 2023-09-30T22:21:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f551affd700 1 -- 172.21.15.89:0/2198878278 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400bcc0 con 0x7f551c09e560 2023-09-30T22:21:37.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f55222d8700 1 -- 172.21.15.89:0/2198878278 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09e560 msgr2=0x7f551c13b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f55222d8700 1 --2- 172.21.15.89:0/2198878278 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09e560 0x7f551c13b860 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5514009e80 tx=0x7f551400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f55222d8700 1 -- 172.21.15.89:0/2198878278 shutdown_connections 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f55222d8700 1 --2- 172.21.15.89:0/2198878278 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c1338c0 0x7f551c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f55222d8700 1 --2- 172.21.15.89:0/2198878278 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c09e560 0x7f551c13b860 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.042+0000 7f55222d8700 1 -- 172.21.15.89:0/2198878278 >> 172.21.15.89:0/2198878278 conn(0x7f551c0a2aa0 msgr2=0x7f551c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.043+0000 7f55222d8700 1 -- 172.21.15.89:0/2198878278 shutdown_connections 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.043+0000 7f55222d8700 1 -- 172.21.15.89:0/2198878278 wait complete. 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.044+0000 7f55222d8700 1 Processor -- start 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.044+0000 7f55222d8700 1 -- start start 2023-09-30T22:21:37.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f55222d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c09e560 0x7f551c11a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f55222d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c1338c0 0x7f551c11aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f55222d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c1341d0 0x7f551c11b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f55222d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c11d460 con 0x7f551c1338c0 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f55222d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c11ba40 con 0x7f551c09e560 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f55222d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f551c11deb0 con 0x7f551c1341d0 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c09e560 0x7f551c11a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c1338c0 0x7f551c11aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c09e560 0x7f551c11a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52200/0 (socket says 172.21.15.89:52200) 2023-09-30T22:21:37.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551bfff700 1 -- 172.21.15.89:0/3063627212 learned_addr learned my addr 172.21.15.89:0/3063627212 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f5520875700 1 --2- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c1341d0 0x7f551c11b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551b7fe700 1 -- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c1341d0 msgr2=0x7f551c11b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551b7fe700 1 --2- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c1341d0 0x7f551c11b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551b7fe700 1 -- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c09e560 msgr2=0x7f551c11a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551b7fe700 1 --2- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c09e560 0x7f551c11a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.045+0000 7f551b7fe700 1 -- 172.21.15.89:0/3063627212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514009b30 con 0x7f551c1338c0 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.046+0000 7f551b7fe700 1 --2- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c1338c0 0x7f551c11aad0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f551000c460 tx=0x7f551000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.046+0000 7f5518ff9700 1 -- 172.21.15.89:0/3063627212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510005cb0 con 0x7f551c1338c0 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.046+0000 7f5518ff9700 1 -- 172.21.15.89:0/3063627212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5510005e10 con 0x7f551c1338c0 2023-09-30T22:21:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.046+0000 7f5518ff9700 1 -- 172.21.15.89:0/3063627212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510007310 con 0x7f551c1338c0 2023-09-30T22:21:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.047+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c11cfa0 con 0x7f551c1338c0 2023-09-30T22:21:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.047+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c12fe40 con 0x7f551c1338c0 2023-09-30T22:21:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.048+0000 7f5518ff9700 1 -- 172.21.15.89:0/3063627212 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5510007470 con 0x7f551c1338c0 2023-09-30T22:21:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.048+0000 7f5518ff9700 1 -- 172.21.15.89:0/3063627212 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f551003ebf0 con 0x7f551c1338c0 2023-09-30T22:21:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.048+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c06f9c0 con 0x7f551c1338c0 2023-09-30T22:21:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.057+0000 7f5518ff9700 1 -- 172.21.15.89:0/3063627212 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5510017020 con 0x7f551c1338c0 2023-09-30T22:21:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.076+0000 7fb114433700 1 Processor -- start 2023-09-30T22:21:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.076+0000 7fb114433700 1 -- start start 2023-09-30T22:21:37.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb114433700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb114433700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb114433700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0a2a70 con 0x7fb10c09d630 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb114433700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0a01d0 con 0x7fb10c138d80 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb114433700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c09e2b0 con 0x7fb10c0a0a30 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb1119ce700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb10c09d630 0x7fb10c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43386/0 (socket says 172.21.15.89:43386) 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb1129d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb1119ce700 1 -- 172.21.15.89:0/2048929315 learned_addr learned my addr 172.21.15.89:0/2048929315 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb1121cf700 1 --2- 172.21.15.89:0/2048929315 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb1129d0700 1 -- 172.21.15.89:0/2048929315 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 msgr2=0x7fb10c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb1129d0700 1 --2- 172.21.15.89:0/2048929315 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb1129d0700 1 -- 172.21.15.89:0/2048929315 >> v1:172.21.15.89:6789/0 conn(0x7fb10c09d630 legacy=0x7fb10c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.077+0000 7fb1129d0700 1 -- 172.21.15.89:0/2048929315 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c09dc50 con 0x7fb10c138d80 2023-09-30T22:21:37.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.078+0000 7fb1129d0700 1 --2- 172.21.15.89:0/2048929315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c13b160 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb104009dc0 tx=0x7fb10400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1af65bf5dc7cf6e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.078+0000 7fb1111cd700 1 -- 172.21.15.89:0/2048929315 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400e070 con 0x7fb10c138d80 2023-09-30T22:21:37.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.078+0000 7fb1111cd700 1 -- 172.21.15.89:0/2048929315 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10400bb20 con 0x7fb10c138d80 2023-09-30T22:21:37.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.078+0000 7fb1111cd700 1 -- 172.21.15.89:0/2048929315 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104012430 con 0x7fb10c138d80 2023-09-30T22:21:37.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.079+0000 7fb114433700 1 -- 172.21.15.89:0/2048929315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 msgr2=0x7fb10c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.079+0000 7fb114433700 1 --2- 172.21.15.89:0/2048929315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c13b160 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb104009dc0 tx=0x7fb10400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.079+0000 7fb114433700 1 -- 172.21.15.89:0/2048929315 shutdown_connections 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.079+0000 7fb114433700 1 --2- 172.21.15.89:0/2048929315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c13b160 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.079+0000 7fb114433700 1 --2- 172.21.15.89:0/2048929315 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.079+0000 7fb114433700 1 -- 172.21.15.89:0/2048929315 >> 172.21.15.89:0/2048929315 conn(0x7fb10c0a3050 msgr2=0x7fb10c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.079+0000 7fb114433700 1 -- 172.21.15.89:0/2048929315 shutdown_connections 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.079+0000 7fb114433700 1 -- 172.21.15.89:0/2048929315 wait complete. 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.080+0000 7fb114433700 1 Processor -- start 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.080+0000 7fb114433700 1 -- start start 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.080+0000 7fb114433700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb10c09d630 0x7fb10c1338d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.080+0000 7fb114433700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c133e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.080+0000 7fb114433700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c134350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.080+0000 7fb114433700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c134da0 con 0x7fb10c09d630 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.080+0000 7fb114433700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0a0410 con 0x7fb10c138d80 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.080+0000 7fb114433700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c136d70 con 0x7fb10c0a0a30 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c134350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c134350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52212/0 (socket says 172.21.15.89:52212) 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 -- 172.21.15.89:0/3563471035 learned_addr learned my addr 172.21.15.89:0/3563471035 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1121cf700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb10c09d630 0x7fb10c1338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1119ce700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c133e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 -- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 msgr2=0x7fb10c133e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c133e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 -- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb10c09d630 msgr2=0x7fb10c1338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb10c09d630 0x7fb10c1338d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 -- 172.21.15.89:0/3563471035 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104009a70 con 0x7fb10c138d80 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1119ce700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c133e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb1129d0700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c134350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb10400b370 tx=0x7fb104004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.081+0000 7fb102ffd700 1 -- 172.21.15.89:0/3563471035 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104012430 con 0x7fb10c138d80 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.082+0000 7fb102ffd700 1 -- 172.21.15.89:0/3563471035 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb104020070 con 0x7fb10c138d80 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.082+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c13aee0 con 0x7fb10c138d80 2023-09-30T22:21:37.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.082+0000 7fb102ffd700 1 -- 172.21.15.89:0/3563471035 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104004280 con 0x7fb10c138d80 2023-09-30T22:21:37.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.082+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c135bf0 con 0x7fb10c138d80 2023-09-30T22:21:37.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.082+0000 7fb102ffd700 1 -- 172.21.15.89:0/3563471035 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb1040043e0 con 0x7fb10c138d80 2023-09-30T22:21:37.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.082+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c0747f0 con 0x7fb10c138d80 2023-09-30T22:21:37.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.083+0000 7fb102ffd700 1 -- 172.21.15.89:0/3563471035 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb104022070 con 0x7fb10c138d80 2023-09-30T22:21:37.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.091+0000 7fb102ffd700 1 -- 172.21.15.89:0/3563471035 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb10402c070 con 0x7fb10c138d80 2023-09-30T22:21:37.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.121+0000 7fdf497b7700 1 Processor -- start 2023-09-30T22:21:37.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.121+0000 7fdf497b7700 1 -- start start 2023-09-30T22:21:37.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.121+0000 7fdf497b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409d6f0 0x7fdf4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.121+0000 7fdf497b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e000 0x7fdf441339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.121+0000 7fdf497b7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf4409e410 0x7fdf44133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.121+0000 7fdf497b7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4406b460 con 0x7fdf4409e410 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.121+0000 7fdf497b7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4409b120 con 0x7fdf4409d6f0 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.121+0000 7fdf497b7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4409a7e0 con 0x7fdf4409e000 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf42ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409d6f0 0x7fdf4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf427fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e000 0x7fdf441339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf42ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409d6f0 0x7fdf4409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52214/0 (socket says 172.21.15.89:52214) 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf42ffd700 1 -- 172.21.15.89:0/1733095372 learned_addr learned my addr 172.21.15.89:0/1733095372 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:37.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf427fc700 1 -- 172.21.15.89:0/1733095372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409d6f0 msgr2=0x7fdf4409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf427fc700 1 --2- 172.21.15.89:0/1733095372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409d6f0 0x7fdf4409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf427fc700 1 -- 172.21.15.89:0/1733095372 >> v2:172.21.15.89:3300/0 conn(0x7fdf4409e410 msgr2=0x7fdf44133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf437fe700 1 --2- 172.21.15.89:0/1733095372 >> v2:172.21.15.89:3300/0 conn(0x7fdf4409e410 0x7fdf44133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf427fc700 1 --2- 172.21.15.89:0/1733095372 >> v2:172.21.15.89:3300/0 conn(0x7fdf4409e410 0x7fdf44133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf427fc700 1 -- 172.21.15.89:0/1733095372 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4409a500 con 0x7fdf4409e000 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf42ffd700 1 --2- 172.21.15.89:0/1733095372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409d6f0 0x7fdf4409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf437fe700 1 --2- 172.21.15.89:0/1733095372 >> v2:172.21.15.89:3300/0 conn(0x7fdf4409e410 0x7fdf44133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf427fc700 1 --2- 172.21.15.89:0/1733095372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e000 0x7fdf441339f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009870 tx=0x7fdf38009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e0dcca2bfe6f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf41ffb700 1 -- 172.21.15.89:0/1733095372 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38011830 con 0x7fdf4409e000 2023-09-30T22:21:37.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf41ffb700 1 -- 172.21.15.89:0/1733095372 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf38011990 con 0x7fdf4409e000 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.122+0000 7fdf41ffb700 1 -- 172.21.15.89:0/1733095372 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38011d20 con 0x7fdf4409e000 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 -- 172.21.15.89:0/1733095372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e000 msgr2=0x7fdf441339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 --2- 172.21.15.89:0/1733095372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e000 0x7fdf441339f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009870 tx=0x7fdf38009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 -- 172.21.15.89:0/1733095372 shutdown_connections 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 --2- 172.21.15.89:0/1733095372 >> v2:172.21.15.89:3300/0 conn(0x7fdf4409e410 0x7fdf44133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 --2- 172.21.15.89:0/1733095372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409d6f0 0x7fdf4409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 --2- 172.21.15.89:0/1733095372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e000 0x7fdf441339f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 -- 172.21.15.89:0/1733095372 >> 172.21.15.89:0/1733095372 conn(0x7fdf44066f50 msgr2=0x7fdf4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 -- 172.21.15.89:0/1733095372 shutdown_connections 2023-09-30T22:21:37.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.123+0000 7fdf497b7700 1 -- 172.21.15.89:0/1733095372 wait complete. 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.124+0000 7fdf497b7700 1 Processor -- start 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.124+0000 7fdf497b7700 1 -- start start 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf497b7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf4409d6f0 0x7fdf441c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf497b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409e000 0x7fdf441c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf497b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e410 0x7fdf441c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf497b7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf441a9600 con 0x7fdf4409d6f0 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf497b7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf441a97e0 con 0x7fdf4409e000 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf497b7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf441c5aa0 con 0x7fdf4409e410 2023-09-30T22:21:37.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e410 0x7fdf441c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e410 0x7fdf441c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49052/0 (socket says 172.21.15.89:49052) 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 -- 172.21.15.89:0/3480429193 learned_addr learned my addr 172.21.15.89:0/3480429193 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf427fc700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409e000 0x7fdf441c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf42ffd700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf4409d6f0 0x7fdf441c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 -- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409e000 msgr2=0x7fdf441c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409e000 0x7fdf441c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 -- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf4409d6f0 msgr2=0x7fdf441c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf4409d6f0 0x7fdf441c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 -- 172.21.15.89:0/3480429193 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009550 con 0x7fdf4409e410 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf437fe700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e410 0x7fdf441c8040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00cea0 tx=0x7fdf3c00ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf2b7fe700 1 -- 172.21.15.89:0/3480429193 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00ada0 con 0x7fdf4409e410 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf2b7fe700 1 -- 172.21.15.89:0/3480429193 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c003a20 con 0x7fdf4409e410 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf2b7fe700 1 -- 172.21.15.89:0/3480429193 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c003dd0 con 0x7fdf4409e410 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.125+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf380094a0 con 0x7fdf4409e410 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.126+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf441a9240 con 0x7fdf4409e410 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.127+0000 7fdf2b7fe700 1 -- 172.21.15.89:0/3480429193 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf3c003f30 con 0x7fdf4409e410 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.127+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf441a9420 con 0x7fdf4409e410 2023-09-30T22:21:37.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.127+0000 7fdf2b7fe700 1 -- 172.21.15.89:0/3480429193 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdf3c03fba0 con 0x7fdf4409e410 2023-09-30T22:21:37.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.136+0000 7fdf2b7fe700 1 -- 172.21.15.89:0/3480429193 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf3c014990 con 0x7fdf4409e410 2023-09-30T22:21:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.236+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcb4411d260 con 0x7fcb44099a40 2023-09-30T22:21:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.237+0000 7fcb2b7fe700 1 -- 172.21.15.89:0/650889837 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb34017030 con 0x7fcb44099a40 2023-09-30T22:21:37.237 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 msgr2=0x7fcb4411b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb4411b5b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400e900 tx=0x7fcb3400ec10 comp rx=0 tx=0).stop 2023-09-30T22:21:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 shutdown_connections 2023-09-30T22:21:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb44131810 0x7fcb4411c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb44099a40 0x7fcb4411b5b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 --2- 172.21.15.89:0/650889837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb4409a3b0 0x7fcb4411baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 >> 172.21.15.89:0/650889837 conn(0x7fcb440a2aa0 msgr2=0x7fcb441b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 shutdown_connections 2023-09-30T22:21:37.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.240+0000 7fcb493d8700 1 -- 172.21.15.89:0/650889837 wait complete. 2023-09-30T22:21:37.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:37.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.284+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f551c074ea0 con 0x7f551c1338c0 2023-09-30T22:21:37.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.284+0000 7f5518ff9700 1 -- 172.21.15.89:0/3063627212 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5510007720 con 0x7f551c1338c0 2023-09-30T22:21:37.284 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:37.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c1338c0 msgr2=0x7f551c11aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 --2- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c1338c0 0x7f551c11aad0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f551000c460 tx=0x7f551000c770 comp rx=0 tx=0).stop 2023-09-30T22:21:37.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 shutdown_connections 2023-09-30T22:21:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 --2- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f551c1341d0 0x7f551c11b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 --2- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f551c09e560 0x7f551c11a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 --2- 172.21.15.89:0/3063627212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f551c1338c0 0x7f551c11aad0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 >> 172.21.15.89:0/3063627212 conn(0x7f551c0a2aa0 msgr2=0x7f551c12f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 shutdown_connections 2023-09-30T22:21:37.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.286+0000 7f55222d8700 1 -- 172.21.15.89:0/3063627212 wait complete. 2023-09-30T22:21:37.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.288+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb10c072980 con 0x7fb10c138d80 2023-09-30T22:21:37.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.288+0000 7fb102ffd700 1 -- 172.21.15.89:0/3563471035 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1040048d0 con 0x7fb10c138d80 2023-09-30T22:21:37.288 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 msgr2=0x7fb10c134350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c134350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb10400b370 tx=0x7fb104004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 shutdown_connections 2023-09-30T22:21:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb10c138d80 0x7fb10c134350 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb10c09d630 0x7fb10c1338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 --2- 172.21.15.89:0/3563471035 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb10c0a0a30 0x7fb10c133e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 >> 172.21.15.89:0/3563471035 conn(0x7fb10c0a3050 msgr2=0x7fb10c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 shutdown_connections 2023-09-30T22:21:37.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.290+0000 7fb114433700 1 -- 172.21.15.89:0/3563471035 wait complete. 2023-09-30T22:21:37.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:37.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:37.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.329+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdf4409f420 con 0x7fdf4409e410 2023-09-30T22:21:37.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.329+0000 7fdf2b7fe700 1 -- 172.21.15.89:0/3480429193 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf3c0143a0 con 0x7fdf4409e410 2023-09-30T22:21:37.330 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:37.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e410 msgr2=0x7fdf441c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e410 0x7fdf441c8040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00cea0 tx=0x7fdf3c00ced0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 shutdown_connections 2023-09-30T22:21:37.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdf4409e410 0x7fdf441c8040 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf4409d6f0 0x7fdf441c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 --2- 172.21.15.89:0/3480429193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdf4409e000 0x7fdf441c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:37.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 >> 172.21.15.89:0/3480429193 conn(0x7fdf44066f50 msgr2=0x7fdf441bad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 shutdown_connections 2023-09-30T22:21:37.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:37.331+0000 7fdf497b7700 1 -- 172.21.15.89:0/3480429193 wait complete. 2023-09-30T22:21:37.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:38.257 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:38.301 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:38.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:38.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.403+0000 7f809600f700 1 Processor -- start 2023-09-30T22:21:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.403+0000 7f809600f700 1 -- start start 2023-09-30T22:21:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.403+0000 7f809600f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f809009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.403+0000 7f809600f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f80901379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.403+0000 7f809600f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8090137f20 0x7f8090138390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.403+0000 7f809600f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809006c4b0 con 0x7f8090137f20 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.403+0000 7f809600f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80900a09c0 con 0x7f809006c0e0 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.403+0000 7f809600f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8090134980 con 0x7f8090134ec0 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f80901379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f80901379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49054/0 (socket says 172.21.15.89:49054) 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808effd700 1 -- 172.21.15.89:0/1564794973 learned_addr learned my addr 172.21.15.89:0/1564794973 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808f7fe700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f809009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808ffff700 1 --2- 172.21.15.89:0/1564794973 >> v2:172.21.15.89:3300/0 conn(0x7f8090137f20 0x7f8090138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808ffff700 1 -- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 msgr2=0x7f80901379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808ffff700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f80901379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808ffff700 1 -- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 msgr2=0x7f809009ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808ffff700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f809009ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808ffff700 1 -- 172.21.15.89:0/1564794973 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80900a0600 con 0x7f8090137f20 2023-09-30T22:21:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808f7fe700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f809009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808ffff700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f8090138390 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8088009dc0 tx=0x7f808800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618dd6f6c801f310 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808e7fc700 1 -- 172.21.15.89:0/1564794973 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808800e070 con 0x7f8090137f20 2023-09-30T22:21:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808e7fc700 1 -- 172.21.15.89:0/1564794973 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808800bae0 con 0x7f8090137f20 2023-09-30T22:21:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.404+0000 7f808e7fc700 1 -- 172.21.15.89:0/1564794973 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808800be90 con 0x7f8090137f20 2023-09-30T22:21:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.405+0000 7f809600f700 1 -- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 msgr2=0x7f8090138390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.405+0000 7f809600f700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f8090138390 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8088009dc0 tx=0x7f808800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.405+0000 7f809600f700 1 -- 172.21.15.89:0/1564794973 shutdown_connections 2023-09-30T22:21:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.405+0000 7f809600f700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f8090138390 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.405+0000 7f809600f700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f809009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.405+0000 7f809600f700 1 --2- 172.21.15.89:0/1564794973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f80901379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.405+0000 7f809600f700 1 -- 172.21.15.89:0/1564794973 >> 172.21.15.89:0/1564794973 conn(0x7f80900a4210 msgr2=0x7f80900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.406+0000 7f809600f700 1 -- 172.21.15.89:0/1564794973 shutdown_connections 2023-09-30T22:21:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.406+0000 7f809600f700 1 -- 172.21.15.89:0/1564794973 wait complete. 2023-09-30T22:21:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.407+0000 7f809600f700 1 Processor -- start 2023-09-30T22:21:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.407+0000 7f809600f700 1 -- start start 2023-09-30T22:21:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f809600f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f8090134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f809600f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f809011ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f809600f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f809011b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f809600f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809011c090 con 0x7f8090137f20 2023-09-30T22:21:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f809600f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809006cff0 con 0x7f809006c0e0 2023-09-30T22:21:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f809600f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f809011e5a0 con 0x7f8090134ec0 2023-09-30T22:21:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f8090134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f809011b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f809011b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52424/0 (socket says 172.21.15.89:52424) 2023-09-30T22:21:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f8090134220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52242/0 (socket says 172.21.15.89:52242) 2023-09-30T22:21:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808ffff700 1 -- 172.21.15.89:0/497405909 learned_addr learned my addr 172.21.15.89:0/497405909 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808effd700 1 --2- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f809011ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808ffff700 1 -- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 msgr2=0x7f809011ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808ffff700 1 --2- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f809011ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808ffff700 1 -- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 msgr2=0x7f8090134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808ffff700 1 --2- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f8090134220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.408+0000 7f808ffff700 1 -- 172.21.15.89:0/497405909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800094a0 con 0x7f8090137f20 2023-09-30T22:21:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.409+0000 7f808ffff700 1 --2- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f809011b080 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f809006be20 tx=0x7f8088004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.409+0000 7f8077fff700 1 -- 172.21.15.89:0/497405909 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880045d0 con 0x7f8090137f20 2023-09-30T22:21:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.409+0000 7f8077fff700 1 -- 172.21.15.89:0/497405909 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8088020070 con 0x7f8090137f20 2023-09-30T22:21:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.409+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8088009a70 con 0x7f8090137f20 2023-09-30T22:21:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.409+0000 7f8077fff700 1 -- 172.21.15.89:0/497405909 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088004970 con 0x7f8090137f20 2023-09-30T22:21:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.409+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80900a0440 con 0x7f8090137f20 2023-09-30T22:21:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.410+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8090074ea0 con 0x7f8090137f20 2023-09-30T22:21:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.410+0000 7f8077fff700 1 -- 172.21.15.89:0/497405909 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8088012430 con 0x7f8090137f20 2023-09-30T22:21:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.411+0000 7f8077fff700 1 -- 172.21.15.89:0/497405909 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8088022070 con 0x7f8090137f20 2023-09-30T22:21:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.415+0000 7f8077fff700 1 -- 172.21.15.89:0/497405909 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f808801c400 con 0x7f8090137f20 2023-09-30T22:21:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.440+0000 7fbbac922700 1 Processor -- start 2023-09-30T22:21:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.440+0000 7fbbac922700 1 -- start start 2023-09-30T22:21:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbbac922700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbba809e930 0x7fbba809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbbac922700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f650 0x7fbba8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbbac922700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba8070940 con 0x7fbba809f240 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbbac922700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba806fd60 con 0x7fbba809e930 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbbac922700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba806fa80 con 0x7fbba809f650 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbba6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f650 0x7fbba8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbba6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f650 0x7fbba8131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52258/0 (socket says 172.21.15.89:52258) 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbba6d9d700 1 -- 172.21.15.89:0/801464927 learned_addr learned my addr 172.21.15.89:0/801464927 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbba659c700 1 --2- 172.21.15.89:0/801464927 >> v2:172.21.15.89:3300/0 conn(0x7fbba809e930 0x7fbba809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbba6d9d700 1 -- 172.21.15.89:0/801464927 >> v2:172.21.15.89:3300/0 conn(0x7fbba809e930 msgr2=0x7fbba809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbba6d9d700 1 --2- 172.21.15.89:0/801464927 >> v2:172.21.15.89:3300/0 conn(0x7fbba809e930 0x7fbba809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbba6d9d700 1 -- 172.21.15.89:0/801464927 >> v1:172.21.15.89:6789/0 conn(0x7fbba809f240 legacy=0x7fbba81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.441+0000 7fbba6d9d700 1 -- 172.21.15.89:0/801464927 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba809fb60 con 0x7fbba809f650 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.442+0000 7fbba6d9d700 1 --2- 172.21.15.89:0/801464927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f650 0x7fbba8131f00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009e50 tx=0x7fbba000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c57b06d98a14a26d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.442+0000 7fbba559a700 1 -- 172.21.15.89:0/801464927 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba000e070 con 0x7fbba809f650 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.442+0000 7fbba559a700 1 -- 172.21.15.89:0/801464927 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba000b910 con 0x7fbba809f650 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.442+0000 7fbba559a700 1 -- 172.21.15.89:0/801464927 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba000bcc0 con 0x7fbba809f650 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.442+0000 7fbbac922700 1 -- 172.21.15.89:0/801464927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f650 msgr2=0x7fbba8131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.442+0000 7fbbac922700 1 --2- 172.21.15.89:0/801464927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f650 0x7fbba8131f00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009e50 tx=0x7fbba000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.443+0000 7fbbac922700 1 -- 172.21.15.89:0/801464927 shutdown_connections 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.443+0000 7fbbac922700 1 --2- 172.21.15.89:0/801464927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f650 0x7fbba8131f00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.443+0000 7fbbac922700 1 --2- 172.21.15.89:0/801464927 >> v2:172.21.15.89:3300/0 conn(0x7fbba809e930 0x7fbba809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.443+0000 7fbbac922700 1 -- 172.21.15.89:0/801464927 >> 172.21.15.89:0/801464927 conn(0x7fbba80a4210 msgr2=0x7fbba80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.443+0000 7fbbac922700 1 -- 172.21.15.89:0/801464927 shutdown_connections 2023-09-30T22:21:38.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.443+0000 7fbbac922700 1 -- 172.21.15.89:0/801464927 wait complete. 2023-09-30T22:21:38.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.443+0000 7f40aceec700 1 Processor -- start 2023-09-30T22:21:38.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.443+0000 7f40aceec700 1 -- start start 2023-09-30T22:21:38.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.444+0000 7fbbac922700 1 Processor -- start 2023-09-30T22:21:38.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.444+0000 7fbbac922700 1 -- start start 2023-09-30T22:21:38.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.444+0000 7f40aceec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40a809e850 0x7f40a809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.444+0000 7f40aceec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a8138e30 0x7f40a813b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.444+0000 7f40aceec700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a80a06b0 con 0x7f40a809ffb0 2023-09-30T22:21:38.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.444+0000 7f40aceec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a806b920 con 0x7f40a809e850 2023-09-30T22:21:38.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.444+0000 7f40aceec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40a809aa30 con 0x7f40a8138e30 2023-09-30T22:21:38.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a8138e30 0x7f40a813b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a5d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f40a809ffb0 0x7f40a809f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43412/0 (socket says 172.21.15.89:43412) 2023-09-30T22:21:38.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a5d9b700 1 -- 172.21.15.89:0/1322572493 learned_addr learned my addr 172.21.15.89:0/1322572493 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:38.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a659c700 1 --2- 172.21.15.89:0/1322572493 >> v2:172.21.15.89:3300/0 conn(0x7f40a809e850 0x7f40a809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbbac922700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbba809e930 0x7fbba81a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbbac922700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f240 0x7fbba81a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a6d9d700 1 -- 172.21.15.89:0/1322572493 >> v2:172.21.15.89:3300/0 conn(0x7f40a809e850 msgr2=0x7f40a809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:38.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a6d9d700 1 --2- 172.21.15.89:0/1322572493 >> v2:172.21.15.89:3300/0 conn(0x7f40a809e850 0x7f40a809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a6d9d700 1 -- 172.21.15.89:0/1322572493 >> v1:172.21.15.89:6789/0 conn(0x7f40a809ffb0 legacy=0x7f40a809f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a6d9d700 1 -- 172.21.15.89:0/1322572493 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a806caa0 con 0x7f40a8138e30 2023-09-30T22:21:38.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbbac922700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba809f650 0x7fbba81a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbbac922700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba81a7630 con 0x7fbba809f650 2023-09-30T22:21:38.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbbac922700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba81a7810 con 0x7fbba809f240 2023-09-30T22:21:38.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbbac922700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8071d20 con 0x7fbba809e930 2023-09-30T22:21:38.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbba6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba809f650 0x7fbba81a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbba6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba809f650 0x7fbba81a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52450/0 (socket says 172.21.15.89:52450) 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbba6d9d700 1 -- 172.21.15.89:0/3421813093 learned_addr learned my addr 172.21.15.89:0/3421813093 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbba659c700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbba809e930 0x7fbba81a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7fbba5d9b700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f240 0x7fbba81a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbba6d9d700 1 -- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbba809e930 msgr2=0x7fbba81a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbba6d9d700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbba809e930 0x7fbba81a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbba6d9d700 1 -- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f240 msgr2=0x7fbba81a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbba6d9d700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f240 0x7fbba81a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbba6d9d700 1 -- 172.21.15.89:0/3421813093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0009b30 con 0x7fbba809f650 2023-09-30T22:21:38.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.445+0000 7f40a6d9d700 1 --2- 172.21.15.89:0/1322572493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a8138e30 0x7f40a813b210 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009d90 tx=0x7f40a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e876278fa87b920 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7f40a559a700 1 -- 172.21.15.89:0/1322572493 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a000e070 con 0x7f40a8138e30 2023-09-30T22:21:38.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7f40a559a700 1 -- 172.21.15.89:0/1322572493 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a000bb20 con 0x7f40a8138e30 2023-09-30T22:21:38.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7f40a559a700 1 -- 172.21.15.89:0/1322572493 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0012430 con 0x7f40a8138e30 2023-09-30T22:21:38.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbba6d9d700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba809f650 0x7fbba81a4830 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fbba806fbc0 tx=0x7fbba0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbb96ffd700 1 -- 172.21.15.89:0/3421813093 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0004800 con 0x7fbba809f650 2023-09-30T22:21:38.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbb96ffd700 1 -- 172.21.15.89:0/3421813093 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba0004960 con 0x7fbba809f650 2023-09-30T22:21:38.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbbac922700 1 -- 172.21.15.89:0/3421813093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba80a0190 con 0x7fbba809f650 2023-09-30T22:21:38.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbb96ffd700 1 -- 172.21.15.89:0/3421813093 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba001c680 con 0x7fbba809f650 2023-09-30T22:21:38.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.446+0000 7fbbac922700 1 -- 172.21.15.89:0/3421813093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba81a6690 con 0x7fbba809f650 2023-09-30T22:21:38.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7f40aceec700 1 -- 172.21.15.89:0/1322572493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a8138e30 msgr2=0x7f40a813b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7f40aceec700 1 --2- 172.21.15.89:0/1322572493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a8138e30 0x7f40a813b210 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009d90 tx=0x7f40a000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:38.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7f40aceec700 1 -- 172.21.15.89:0/1322572493 shutdown_connections 2023-09-30T22:21:38.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7f40aceec700 1 --2- 172.21.15.89:0/1322572493 >> v2:172.21.15.89:3300/0 conn(0x7f40a809e850 0x7f40a809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7f40aceec700 1 --2- 172.21.15.89:0/1322572493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a8138e30 0x7f40a813b210 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7f40aceec700 1 -- 172.21.15.89:0/1322572493 >> 172.21.15.89:0/1322572493 conn(0x7f40a80a3f70 msgr2=0x7f40a80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7f40aceec700 1 -- 172.21.15.89:0/1322572493 shutdown_connections 2023-09-30T22:21:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7f40aceec700 1 -- 172.21.15.89:0/1322572493 wait complete. 2023-09-30T22:21:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.447+0000 7fbb94ff9700 1 -- 172.21.15.89:0/3421813093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb800052e0 con 0x7fbba809f650 2023-09-30T22:21:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.448+0000 7f40aceec700 1 Processor -- start 2023-09-30T22:21:38.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.448+0000 7fbb96ffd700 1 -- 172.21.15.89:0/3421813093 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbba001c7e0 con 0x7fbba809f650 2023-09-30T22:21:38.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7fbb96ffd700 1 -- 172.21.15.89:0/3421813093 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbba0041a20 con 0x7fbba809f650 2023-09-30T22:21:38.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.448+0000 7f40aceec700 1 -- start start 2023-09-30T22:21:38.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40aceec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a809e850 0x7f40a81363a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40aceec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40a809ffb0 0x7f40a81368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40aceec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a8138e30 0x7f40a8136e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40aceec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a809d4c0 con 0x7f40a8138e30 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40aceec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8134b00 con 0x7f40a809ffb0 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40aceec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40a806ccc0 con 0x7f40a809e850 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a809e850 0x7f40a81363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a809e850 0x7f40a81363a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49104/0 (socket says 172.21.15.89:49104) 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a8138e30 0x7f40a8136e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a659c700 1 -- 172.21.15.89:0/1882664863 learned_addr learned my addr 172.21.15.89:0/1882664863 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a659c700 1 -- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40a809ffb0 msgr2=0x7f40a81368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a659c700 1 --2- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40a809ffb0 0x7f40a81368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a659c700 1 -- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a8138e30 msgr2=0x7f40a8136e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a659c700 1 --2- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a8138e30 0x7f40a8136e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.449+0000 7f40a659c700 1 -- 172.21.15.89:0/1882664863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a0009a70 con 0x7f40a809e850 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.450+0000 7f40a6d9d700 1 --2- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a8138e30 0x7f40a8136e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.450+0000 7f40a659c700 1 --2- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a809e850 0x7f40a81363a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f409800e9e0 tx=0x7f409800eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.450+0000 7f4096ffd700 1 -- 172.21.15.89:0/1882664863 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098009710 con 0x7f40a809e850 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.450+0000 7f4096ffd700 1 -- 172.21.15.89:0/1882664863 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409800c850 con 0x7f40a809e850 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.450+0000 7f4096ffd700 1 -- 172.21.15.89:0/1882664863 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409800cbc0 con 0x7f40a809e850 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.450+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a8134f50 con 0x7f40a809e850 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.450+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a81342a0 con 0x7f40a809e850 2023-09-30T22:21:38.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.451+0000 7f4096ffd700 1 -- 172.21.15.89:0/1882664863 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f409800cd20 con 0x7f40a809e850 2023-09-30T22:21:38.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.451+0000 7f4096ffd700 1 -- 172.21.15.89:0/1882664863 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f409803f950 con 0x7f40a809e850 2023-09-30T22:21:38.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.452+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a80747f0 con 0x7f40a809e850 2023-09-30T22:21:38.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.453+0000 7fbb96ffd700 1 -- 172.21.15.89:0/3421813093 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbba00257d0 con 0x7fbba809f650 2023-09-30T22:21:38.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.457+0000 7f4096ffd700 1 -- 172.21.15.89:0/1882664863 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4098017080 con 0x7f40a809e850 2023-09-30T22:21:38.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.483+0000 7f48f296a700 1 Processor -- start 2023-09-30T22:21:38.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.483+0000 7f48f296a700 1 -- start start 2023-09-30T22:21:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48f296a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec133b10 0x7f48ec133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48f296a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48ec09e9b0 0x7f48ec13b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48f296a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec1334b0 con 0x7f48ec09de50 2023-09-30T22:21:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48f296a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec134420 con 0x7f48ec09e9b0 2023-09-30T22:21:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48f296a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec070bc0 con 0x7f48ec133b10 2023-09-30T22:21:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48f0f07700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48ec09e9b0 0x7f48ec13b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48f0f07700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48ec09e9b0 0x7f48ec13b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52466/0 (socket says 172.21.15.89:52466) 2023-09-30T22:21:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48f0f07700 1 -- 172.21.15.89:0/680059571 learned_addr learned my addr 172.21.15.89:0/680059571 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:38.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.484+0000 7f48ebfff700 1 --2- 172.21.15.89:0/680059571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec133b10 0x7f48ec133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.485+0000 7f48f0f07700 1 -- 172.21.15.89:0/680059571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec133b10 msgr2=0x7f48ec133ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.485+0000 7f48f0f07700 1 --2- 172.21.15.89:0/680059571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec133b10 0x7f48ec133ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.485+0000 7f48f0f07700 1 -- 172.21.15.89:0/680059571 >> v1:172.21.15.89:6789/0 conn(0x7f48ec09de50 legacy=0x7f48ec09e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.485+0000 7f48f0f07700 1 -- 172.21.15.89:0/680059571 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec070350 con 0x7f48ec09e9b0 2023-09-30T22:21:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.485+0000 7f48f0f07700 1 --2- 172.21.15.89:0/680059571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 0x7f48ec13b7f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009e50 tx=0x7f48e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204cfba43b064dd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.485+0000 7f48eaffd700 1 -- 172.21.15.89:0/680059571 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e400e070 con 0x7f48ec09e9b0 2023-09-30T22:21:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.485+0000 7f48eaffd700 1 -- 172.21.15.89:0/680059571 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e400b910 con 0x7f48ec09e9b0 2023-09-30T22:21:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.485+0000 7f48eaffd700 1 -- 172.21.15.89:0/680059571 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e400bcc0 con 0x7f48ec09e9b0 2023-09-30T22:21:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.486+0000 7f48f296a700 1 -- 172.21.15.89:0/680059571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 msgr2=0x7f48ec13b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.486+0000 7f48f296a700 1 --2- 172.21.15.89:0/680059571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 0x7f48ec13b7f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009e50 tx=0x7f48e400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.486+0000 7f48f296a700 1 -- 172.21.15.89:0/680059571 shutdown_connections 2023-09-30T22:21:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.486+0000 7f48f296a700 1 --2- 172.21.15.89:0/680059571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 0x7f48ec13b7f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.486+0000 7f48f296a700 1 --2- 172.21.15.89:0/680059571 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec133b10 0x7f48ec133ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.486+0000 7f48f296a700 1 -- 172.21.15.89:0/680059571 >> 172.21.15.89:0/680059571 conn(0x7f48ec0a2aa0 msgr2=0x7f48ec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.486+0000 7f48f296a700 1 -- 172.21.15.89:0/680059571 shutdown_connections 2023-09-30T22:21:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.486+0000 7f48f296a700 1 -- 172.21.15.89:0/680059571 wait complete. 2023-09-30T22:21:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.487+0000 7f48f296a700 1 Processor -- start 2023-09-30T22:21:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.487+0000 7f48f296a700 1 -- start start 2023-09-30T22:21:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48f296a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec09de50 0x7f48ec11f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48f296a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 0x7f48ec11ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48f296a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ec133b10 0x7f48ec120470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48f296a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec121710 con 0x7f48ec09e9b0 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48f296a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec123230 con 0x7f48ec133b10 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48f296a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0a13e0 con 0x7f48ec09de50 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48ebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec09de50 0x7f48ec11f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48ebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec09de50 0x7f48ec11f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49112/0 (socket says 172.21.15.89:49112) 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48ebfff700 1 -- 172.21.15.89:0/3128503952 learned_addr learned my addr 172.21.15.89:0/3128503952 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48f0f07700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ec133b10 0x7f48ec120470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48eb7fe700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 0x7f48ec11ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48ebfff700 1 -- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ec133b10 msgr2=0x7f48ec120470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48ebfff700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ec133b10 0x7f48ec120470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48ebfff700 1 -- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 msgr2=0x7f48ec11ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48ebfff700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 0x7f48ec11ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48ebfff700 1 -- 172.21.15.89:0/3128503952 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4009b30 con 0x7f48ec09de50 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.488+0000 7f48f0f07700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ec133b10 0x7f48ec120470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.489+0000 7f48eb7fe700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 0x7f48ec11ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.489+0000 7f48ebfff700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec09de50 0x7f48ec11f9f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00e9b0 tx=0x7f48dc00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.489+0000 7f48e8ff9700 1 -- 172.21.15.89:0/3128503952 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc009710 con 0x7f48ec09de50 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.489+0000 7f48e8ff9700 1 -- 172.21.15.89:0/3128503952 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc00c850 con 0x7f48ec09de50 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.489+0000 7f48e8ff9700 1 -- 172.21.15.89:0/3128503952 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00cbc0 con 0x7f48ec09de50 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.489+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec130510 con 0x7f48ec09de50 2023-09-30T22:21:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.489+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec123ca0 con 0x7f48ec09de50 2023-09-30T22:21:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.490+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec074ea0 con 0x7f48ec09de50 2023-09-30T22:21:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.490+0000 7f48e8ff9700 1 -- 172.21.15.89:0/3128503952 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48dc00cd20 con 0x7f48ec09de50 2023-09-30T22:21:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.490+0000 7f48e8ff9700 1 -- 172.21.15.89:0/3128503952 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f48dc03f880 con 0x7f48ec09de50 2023-09-30T22:21:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.495+0000 7f48e8ff9700 1 -- 172.21.15.89:0/3128503952 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48dc01be10 con 0x7f48ec09de50 2023-09-30T22:21:38.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.619+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f809006b250 con 0x7f8090137f20 2023-09-30T22:21:38.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.619+0000 7f8077fff700 1 -- 172.21.15.89:0/497405909 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8088025380 con 0x7f8090137f20 2023-09-30T22:21:38.619 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:38.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 msgr2=0x7f809011b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 --2- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f809011b080 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f809006be20 tx=0x7f8088004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 shutdown_connections 2023-09-30T22:21:38.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 --2- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8090137f20 0x7f809011b080 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 --2- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f809006c0e0 0x7f8090134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 --2- 172.21.15.89:0/497405909 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8090134ec0 0x7f809011ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 >> 172.21.15.89:0/497405909 conn(0x7f80900a4210 msgr2=0x7f8090099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 shutdown_connections 2023-09-30T22:21:38.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.621+0000 7f809600f700 1 -- 172.21.15.89:0/497405909 wait complete. 2023-09-30T22:21:38.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.649+0000 7fbb94ff9700 1 -- 172.21.15.89:0/3421813093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb80005860 con 0x7fbba809f650 2023-09-30T22:21:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.649+0000 7fbb96ffd700 1 -- 172.21.15.89:0/3421813093 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbba0033070 con 0x7fbba809f650 2023-09-30T22:21:38.649 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:38.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.651+0000 7fbb94ff9700 1 -- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba809f650 msgr2=0x7fbba81a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.651+0000 7fbb94ff9700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba809f650 0x7fbba81a4830 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fbba806fbc0 tx=0x7fbba0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.651+0000 7fbb94ff9700 1 -- 172.21.15.89:0/3421813093 shutdown_connections 2023-09-30T22:21:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.651+0000 7fbb94ff9700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba809f650 0x7fbba81a4830 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.651+0000 7fbb94ff9700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbba809e930 0x7fbba81a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.651+0000 7fbb94ff9700 1 --2- 172.21.15.89:0/3421813093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbba809f240 0x7fbba81a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.651+0000 7fbb94ff9700 1 -- 172.21.15.89:0/3421813093 >> 172.21.15.89:0/3421813093 conn(0x7fbba80a4210 msgr2=0x7fbba81b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.652+0000 7fbb94ff9700 1 -- 172.21.15.89:0/3421813093 shutdown_connections 2023-09-30T22:21:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.652+0000 7fbb94ff9700 1 -- 172.21.15.89:0/3421813093 wait complete. 2023-09-30T22:21:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.655+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f40a8072820 con 0x7f40a809e850 2023-09-30T22:21:38.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.655+0000 7f4096ffd700 1 -- 172.21.15.89:0/1882664863 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4098021020 con 0x7f40a809e850 2023-09-30T22:21:38.655 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:38.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a809e850 msgr2=0x7f40a81363a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 --2- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a809e850 0x7f40a81363a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f409800e9e0 tx=0x7f409800eda0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 shutdown_connections 2023-09-30T22:21:38.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 --2- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a8138e30 0x7f40a8136e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 --2- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40a809e850 0x7f40a81363a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 --2- 172.21.15.89:0/1882664863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40a809ffb0 0x7f40a81368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 >> 172.21.15.89:0/1882664863 conn(0x7f40a80a3f70 msgr2=0x7f40a8132e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 shutdown_connections 2023-09-30T22:21:38.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.658+0000 7f40aceec700 1 -- 172.21.15.89:0/1882664863 wait complete. 2023-09-30T22:21:38.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:38.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.690+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f48ec06b9e0 con 0x7f48ec09de50 2023-09-30T22:21:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.691+0000 7f48e8ff9700 1 -- 172.21.15.89:0/3128503952 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f48dc01b3b0 con 0x7f48ec09de50 2023-09-30T22:21:38.691 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec09de50 msgr2=0x7f48ec11f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec09de50 0x7f48ec11f9f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00e9b0 tx=0x7f48dc00ed70 comp rx=0 tx=0).stop 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 shutdown_connections 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48ec133b10 0x7f48ec120470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48ec09de50 0x7f48ec11f9f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 --2- 172.21.15.89:0/3128503952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48ec09e9b0 0x7f48ec11ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 >> 172.21.15.89:0/3128503952 conn(0x7f48ec0a2aa0 msgr2=0x7f48ec09f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 shutdown_connections 2023-09-30T22:21:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:38.693+0000 7f48f296a700 1 -- 172.21.15.89:0/3128503952 wait complete. 2023-09-30T22:21:38.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:39.634 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:39.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:39.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:39.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.775+0000 7f289554e700 1 Processor -- start 2023-09-30T22:21:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.775+0000 7f289554e700 1 -- start start 2023-09-30T22:21:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f289554e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009eca0 0x7f2890135550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f289554e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f2890137e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f289554e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2890070a50 con 0x7f2890130560 2023-09-30T22:21:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f289554e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28900717d0 con 0x7f289009eca0 2023-09-30T22:21:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f289554e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2890130340 con 0x7f2890135a90 2023-09-30T22:21:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f288effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2890130560 0x7f2890130930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43428/0 (socket says 172.21.15.89:43428) 2023-09-30T22:21:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f288effd700 1 -- 172.21.15.89:0/1241295751 learned_addr learned my addr 172.21.15.89:0/1241295751 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f288f7fe700 1 --2- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f2890137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.776+0000 7f288e7fc700 1 --2- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009eca0 0x7f2890135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288f7fe700 1 -- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009eca0 msgr2=0x7f2890135550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:39.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288f7fe700 1 --2- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009eca0 0x7f2890135550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288f7fe700 1 -- 172.21.15.89:0/1241295751 >> v1:172.21.15.89:6789/0 conn(0x7f2890130560 legacy=0x7f2890130930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288f7fe700 1 -- 172.21.15.89:0/1241295751 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890071060 con 0x7f2890135a90 2023-09-30T22:21:39.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288dffb700 1 -- 172.21.15.89:0/1241295751 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629669791 0 0) 0x7f2890070a50 con 0x7f2890130560 2023-09-30T22:21:39.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288e7fc700 1 --2- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009eca0 0x7f2890135550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:39.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288f7fe700 1 --2- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f2890137e70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2888009dc0 tx=0x7f288800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=296ec224275d3834 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288dffb700 1 -- 172.21.15.89:0/1241295751 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288800e070 con 0x7f2890135a90 2023-09-30T22:21:39.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288dffb700 1 -- 172.21.15.89:0/1241295751 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288800bb20 con 0x7f2890135a90 2023-09-30T22:21:39.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.777+0000 7f288dffb700 1 -- 172.21.15.89:0/1241295751 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888012430 con 0x7f2890135a90 2023-09-30T22:21:39.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.778+0000 7f289554e700 1 -- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 msgr2=0x7f2890137e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.778+0000 7f289554e700 1 --2- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f2890137e70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2888009dc0 tx=0x7f288800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:39.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.778+0000 7f289554e700 1 -- 172.21.15.89:0/1241295751 shutdown_connections 2023-09-30T22:21:39.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.778+0000 7f289554e700 1 --2- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f2890137e70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.778+0000 7f289554e700 1 --2- 172.21.15.89:0/1241295751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f289009eca0 0x7f2890135550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.778+0000 7f289554e700 1 -- 172.21.15.89:0/1241295751 >> 172.21.15.89:0/1241295751 conn(0x7f28900a2aa0 msgr2=0x7f28900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:39.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.778+0000 7f289554e700 1 -- 172.21.15.89:0/1241295751 shutdown_connections 2023-09-30T22:21:39.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.778+0000 7f289554e700 1 -- 172.21.15.89:0/1241295751 wait complete. 2023-09-30T22:21:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.780+0000 7f289554e700 1 Processor -- start 2023-09-30T22:21:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.780+0000 7f289554e700 1 -- start start 2023-09-30T22:21:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.780+0000 7f289554e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f289009eca0 0x7f289009c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.780+0000 7f289554e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2890130560 0x7f289009c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.780+0000 7f289554e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f289009cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.780+0000 7f289554e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289009d610 con 0x7f289009eca0 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.780+0000 7f289554e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2890135570 con 0x7f2890130560 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.780+0000 7f289554e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28900719f0 con 0x7f2890135a90 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2890130560 0x7f289009c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288e7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2890130560 0x7f289009c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52322/0 (socket says 172.21.15.89:52322) 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288e7fc700 1 -- 172.21.15.89:0/157266332 learned_addr learned my addr 172.21.15.89:0/157266332 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288effd700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f289009eca0 0x7f289009c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288f7fe700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f289009cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288effd700 1 -- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 msgr2=0x7f289009cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288effd700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f289009cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288effd700 1 -- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2890130560 msgr2=0x7f289009c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288effd700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2890130560 0x7f289009c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288effd700 1 -- 172.21.15.89:0/157266332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840094a0 con 0x7f289009eca0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f288effd700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f289009eca0 0x7f289009c1b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f288000c470 tx=0x7f288000c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f28777fe700 1 -- 172.21.15.89:0/157266332 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880004c80 con 0x7f289009eca0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f28777fe700 1 -- 172.21.15.89:0/157266332 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2880004de0 con 0x7f289009eca0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2888009a70 con 0x7f289009eca0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f28777fe700 1 -- 172.21.15.89:0/157266332 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880007310 con 0x7f289009eca0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.781+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2890136830 con 0x7f289009eca0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.782+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2890074ea0 con 0x7f289009eca0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.782+0000 7f28777fe700 1 -- 172.21.15.89:0/157266332 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2880007470 con 0x7f289009eca0 2023-09-30T22:21:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.783+0000 7f28777fe700 1 -- 172.21.15.89:0/157266332 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f288003f230 con 0x7f289009eca0 2023-09-30T22:21:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.791+0000 7f28777fe700 1 -- 172.21.15.89:0/157266332 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2880007720 con 0x7f289009eca0 2023-09-30T22:21:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.813+0000 7f4e82be3700 1 Processor -- start 2023-09-30T22:21:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.813+0000 7f4e82be3700 1 -- start start 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e82be3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0a1140 0x7f4e7c13d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e82be3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e7c0a15b0 0x7f4e7c13f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e82be3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7c0a1e00 con 0x7f4e7c06cc90 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e82be3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7c0a2550 con 0x7f4e7c0a15b0 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e82be3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c09cfa0 con 0x7f4e7c0a1140 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e81180700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e7c0a15b0 0x7f4e7c13f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e7bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0a1140 0x7f4e7c13d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e81180700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e7c0a15b0 0x7f4e7c13f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52488/0 (socket says 172.21.15.89:52488) 2023-09-30T22:21:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e81180700 1 -- 172.21.15.89:0/574091526 learned_addr learned my addr 172.21.15.89:0/574091526 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e8097f700 1 --1- 172.21.15.89:0/574091526 >> v1:172.21.15.89:6789/0 conn(0x7f4e7c06cc90 0x7f4e7c0a0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43444/0 (socket says 172.21.15.89:43444) 2023-09-30T22:21:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e7bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0a1140 0x7f4e7c13d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52338/0 (socket says 172.21.15.89:52338) 2023-09-30T22:21:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e7bfff700 1 -- 172.21.15.89:0/574091526 >> v2:172.21.15.89:3300/0 conn(0x7f4e7c0a15b0 msgr2=0x7f4e7c13f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e7bfff700 1 --2- 172.21.15.89:0/574091526 >> v2:172.21.15.89:3300/0 conn(0x7f4e7c0a15b0 0x7f4e7c13f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e7bfff700 1 -- 172.21.15.89:0/574091526 >> v1:172.21.15.89:6789/0 conn(0x7f4e7c06cc90 legacy=0x7f4e7c0a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e7b7fe700 1 -- 172.21.15.89:0/574091526 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727300622 0 0) 0x7f4e7c0a1e00 con 0x7f4e7c06cc90 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.814+0000 7f4e7bfff700 1 -- 172.21.15.89:0/574091526 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c09d8e0 con 0x7f4e7c0a1140 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.815+0000 7f4e7bfff700 1 --2- 172.21.15.89:0/574091526 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0a1140 0x7f4e7c13d130 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009840 tx=0x7f4e68009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c29ee761926f6e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.815+0000 7f4e7b7fe700 1 -- 172.21.15.89:0/574091526 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68011830 con 0x7f4e7c0a1140 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.815+0000 7f4e7b7fe700 1 -- 172.21.15.89:0/574091526 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e68011990 con 0x7f4e7c0a1140 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.815+0000 7f4e7b7fe700 1 -- 172.21.15.89:0/574091526 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68011d20 con 0x7f4e7c0a1140 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.815+0000 7f4e82be3700 1 -- 172.21.15.89:0/574091526 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0a1140 msgr2=0x7f4e7c13d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.815+0000 7f4e82be3700 1 --2- 172.21.15.89:0/574091526 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0a1140 0x7f4e7c13d130 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009840 tx=0x7f4e68009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.816+0000 7f4e82be3700 1 -- 172.21.15.89:0/574091526 shutdown_connections 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.816+0000 7f4e82be3700 1 --2- 172.21.15.89:0/574091526 >> v2:172.21.15.89:3300/0 conn(0x7f4e7c0a15b0 0x7f4e7c13f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.816+0000 7f4e82be3700 1 --2- 172.21.15.89:0/574091526 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0a1140 0x7f4e7c13d130 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.816+0000 7f4e82be3700 1 -- 172.21.15.89:0/574091526 >> 172.21.15.89:0/574091526 conn(0x7f4e7c0a3050 msgr2=0x7f4e7c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.816+0000 7f6af74b8700 1 Processor -- start 2023-09-30T22:21:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.816+0000 7f4e82be3700 1 -- 172.21.15.89:0/574091526 shutdown_connections 2023-09-30T22:21:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.816+0000 7f4e82be3700 1 -- 172.21.15.89:0/574091526 wait complete. 2023-09-30T22:21:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.816+0000 7f6af74b8700 1 -- start start 2023-09-30T22:21:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af74b8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009d6f0 0x7f6af009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af74b8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 0x7f6af0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af74b8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af009f130 con 0x7f6af009e410 2023-09-30T22:21:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af74b8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af009f520 con 0x7f6af009e000 2023-09-30T22:21:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af74b8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6af009e930 con 0x7f6af009d6f0 2023-09-30T22:21:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af4a53700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 0x7f6af0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af5a55700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6af009e410 0x7f6af0130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43460/0 (socket says 172.21.15.89:43460) 2023-09-30T22:21:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af5a55700 1 -- 172.21.15.89:0/2079014482 learned_addr learned my addr 172.21.15.89:0/2079014482 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f6af5254700 1 --2- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009d6f0 0x7f6af009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f4e82be3700 1 Processor -- start 2023-09-30T22:21:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6af4a53700 1 -- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009d6f0 msgr2=0x7f6af009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6af4a53700 1 --2- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009d6f0 0x7f6af009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6af4a53700 1 -- 172.21.15.89:0/2079014482 >> v1:172.21.15.89:6789/0 conn(0x7f6af009e410 legacy=0x7f6af0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6af4a53700 1 -- 172.21.15.89:0/2079014482 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af009b2d0 con 0x7f6af009e000 2023-09-30T22:21:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6ae7fff700 1 -- 172.21.15.89:0/2079014482 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638299064 0 0) 0x7f6af009f130 con 0x7f6af009e410 2023-09-30T22:21:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6af5254700 1 --2- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009d6f0 0x7f6af009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.817+0000 7f4e82be3700 1 -- start start 2023-09-30T22:21:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6af4a53700 1 --2- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 0x7f6af0130660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009870 tx=0x7f6ae0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e82482a9c985562 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6ae7fff700 1 -- 172.21.15.89:0/2079014482 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0011830 con 0x7f6af009e000 2023-09-30T22:21:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6ae7fff700 1 -- 172.21.15.89:0/2079014482 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0011990 con 0x7f6af009e000 2023-09-30T22:21:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f6ae7fff700 1 -- 172.21.15.89:0/2079014482 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0011d20 con 0x7f6af009e000 2023-09-30T22:21:39.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e82be3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c06cc90 0x7f4e7c06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e82be3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0a1140 0x7f4e7c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e82be3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c0a15b0 0x7f4e7c070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e82be3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c134960 con 0x7f4e7c0a15b0 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e82be3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c134f30 con 0x7f4e7c06cc90 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e82be3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c09aa50 con 0x7f4e7c0a1140 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e8097f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c06cc90 0x7f4e7c06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e8097f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c06cc90 0x7f4e7c06fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52350/0 (socket says 172.21.15.89:52350) 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e8097f700 1 -- 172.21.15.89:0/3639215319 learned_addr learned my addr 172.21.15.89:0/3639215319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e7bfff700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0a1140 0x7f4e7c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.818+0000 7f4e81180700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c0a15b0 0x7f4e7c070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e8097f700 1 -- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0a1140 msgr2=0x7f4e7c0703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e8097f700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0a1140 0x7f4e7c0703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e8097f700 1 -- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c0a15b0 msgr2=0x7f4e7c070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e8097f700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c0a15b0 0x7f4e7c070900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e8097f700 1 -- 172.21.15.89:0/3639215319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e680094a0 con 0x7f4e7c06cc90 2023-09-30T22:21:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e8097f700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c06cc90 0x7f4e7c06fe80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c520 tx=0x7f4e7000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f6af74b8700 1 -- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 msgr2=0x7f6af0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f6af74b8700 1 --2- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 0x7f6af0130660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009870 tx=0x7f6ae0009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f6af74b8700 1 -- 172.21.15.89:0/2079014482 shutdown_connections 2023-09-30T22:21:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f6af74b8700 1 --2- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009d6f0 0x7f6af009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f6af74b8700 1 --2- 172.21.15.89:0/2079014482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 0x7f6af0130660 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f6af74b8700 1 -- 172.21.15.89:0/2079014482 >> 172.21.15.89:0/2079014482 conn(0x7f6af0066f50 msgr2=0x7f6af009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e797fa700 1 -- 172.21.15.89:0/3639215319 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004c80 con 0x7f4e7c06cc90 2023-09-30T22:21:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e797fa700 1 -- 172.21.15.89:0/3639215319 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70004de0 con 0x7f4e7c06cc90 2023-09-30T22:21:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e797fa700 1 -- 172.21.15.89:0/3639215319 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70007310 con 0x7f4e7c06cc90 2023-09-30T22:21:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c0a20f0 con 0x7f4e7c06cc90 2023-09-30T22:21:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f6af74b8700 1 -- 172.21.15.89:0/2079014482 shutdown_connections 2023-09-30T22:21:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.819+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7c134b40 con 0x7f4e7c06cc90 2023-09-30T22:21:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.820+0000 7f6af74b8700 1 -- 172.21.15.89:0/2079014482 wait complete. 2023-09-30T22:21:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f6af74b8700 1 Processor -- start 2023-09-30T22:21:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.820+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7c071e20 con 0x7f4e7c06cc90 2023-09-30T22:21:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.820+0000 7f4e797fa700 1 -- 172.21.15.89:0/3639215319 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4e70007470 con 0x7f4e7c06cc90 2023-09-30T22:21:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f4e797fa700 1 -- 172.21.15.89:0/3639215319 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4e7003f280 con 0x7f4e7c06cc90 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f6af74b8700 1 -- start start 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f6af74b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af009d6f0 0x7f6af0121290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f6af74b8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 0x7f6af01217d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f6af74b8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009e410 0x7f6af0121d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f6af74b8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af0123c80 con 0x7f6af009d6f0 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f6af74b8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af006eb10 con 0x7f6af009e000 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.821+0000 7f6af74b8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6af009c700 con 0x7f6af009e410 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af009d6f0 0x7f6af0121290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af009d6f0 0x7f6af0121290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52512/0 (socket says 172.21.15.89:52512) 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 -- 172.21.15.89:0/18338364 learned_addr learned my addr 172.21.15.89:0/18338364 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5a55700 1 --2- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009e410 0x7f6af0121d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 -- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009e410 msgr2=0x7f6af0121d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 --2- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009e410 0x7f6af0121d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 -- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 msgr2=0x7f6af01217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 --2- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 0x7f6af01217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 -- 172.21.15.89:0/18338364 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec0094a0 con 0x7f6af009d6f0 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af5254700 1 --2- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af009d6f0 0x7f6af0121290 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009ec0 tx=0x7f6aec00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6ae5ffb700 1 -- 172.21.15.89:0/18338364 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec013920 con 0x7f6af009d6f0 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6ae5ffb700 1 -- 172.21.15.89:0/18338364 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec013a80 con 0x7f6af009d6f0 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6ae5ffb700 1 -- 172.21.15.89:0/18338364 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec013e10 con 0x7f6af009d6f0 2023-09-30T22:21:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.822+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae00094a0 con 0x7f6af009d6f0 2023-09-30T22:21:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.823+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af0123900 con 0x7f6af009d6f0 2023-09-30T22:21:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.824+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af011c340 con 0x7f6af009d6f0 2023-09-30T22:21:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.825+0000 7f6ae5ffb700 1 -- 172.21.15.89:0/18338364 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6aec00c640 con 0x7f6af009d6f0 2023-09-30T22:21:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.825+0000 7f6ae5ffb700 1 -- 172.21.15.89:0/18338364 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6aec043b10 con 0x7f6af009d6f0 2023-09-30T22:21:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.826+0000 7f4e797fa700 1 -- 172.21.15.89:0/3639215319 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4e700133c0 con 0x7f4e7c06cc90 2023-09-30T22:21:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.830+0000 7f6ae5ffb700 1 -- 172.21.15.89:0/18338364 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6aec00c8f0 con 0x7f6af009d6f0 2023-09-30T22:21:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.856+0000 7ffa40566700 1 Processor -- start 2023-09-30T22:21:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.856+0000 7ffa40566700 1 -- start start 2023-09-30T22:21:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa40566700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffa3809d6f0 0x7ffa3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa40566700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa38130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa40566700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809e410 0x7ffa38130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa40566700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3809f520 con 0x7ffa3809d6f0 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa40566700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3809f130 con 0x7ffa3809e410 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa40566700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3809e930 con 0x7ffa3809e000 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa3e302700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffa3809d6f0 0x7ffa3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa3e302700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffa3809d6f0 0x7ffa3809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52528/0 (socket says 172.21.15.89:52528) 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa3e302700 1 -- 172.21.15.89:0/3278329771 learned_addr learned my addr 172.21.15.89:0/3278329771 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa3eb03700 1 --2- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809e410 0x7ffa38130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.857+0000 7ffa3db01700 1 --2- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa38130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3eb03700 1 -- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 msgr2=0x7ffa38130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3eb03700 1 --2- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa38130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3eb03700 1 -- 172.21.15.89:0/3278329771 >> v2:172.21.15.89:3300/0 conn(0x7ffa3809d6f0 msgr2=0x7ffa3809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3eb03700 1 --2- 172.21.15.89:0/3278329771 >> v2:172.21.15.89:3300/0 conn(0x7ffa3809d6f0 0x7ffa3809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3eb03700 1 -- 172.21.15.89:0/3278329771 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3809b2d0 con 0x7ffa3809e410 2023-09-30T22:21:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3e302700 1 --2- 172.21.15.89:0/3278329771 >> v2:172.21.15.89:3300/0 conn(0x7ffa3809d6f0 0x7ffa3809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3eb03700 1 --2- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809e410 0x7ffa38130ba0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009d90 tx=0x7ffa3000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93e5ddf561092cb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3d300700 1 -- 172.21.15.89:0/3278329771 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000e070 con 0x7ffa3809e410 2023-09-30T22:21:39.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3d300700 1 -- 172.21.15.89:0/3278329771 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3000bb20 con 0x7ffa3809e410 2023-09-30T22:21:39.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.858+0000 7ffa3d300700 1 -- 172.21.15.89:0/3278329771 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30012430 con 0x7ffa3809e410 2023-09-30T22:21:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 -- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809e410 msgr2=0x7ffa38130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 --2- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809e410 0x7ffa38130ba0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009d90 tx=0x7ffa3000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 -- 172.21.15.89:0/3278329771 shutdown_connections 2023-09-30T22:21:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 --2- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809e410 0x7ffa38130ba0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 --2- 172.21.15.89:0/3278329771 >> v2:172.21.15.89:3300/0 conn(0x7ffa3809d6f0 0x7ffa3809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 --2- 172.21.15.89:0/3278329771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa38130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 -- 172.21.15.89:0/3278329771 >> 172.21.15.89:0/3278329771 conn(0x7ffa38066f50 msgr2=0x7ffa3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 -- 172.21.15.89:0/3278329771 shutdown_connections 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.859+0000 7ffa40566700 1 -- 172.21.15.89:0/3278329771 wait complete. 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.860+0000 7ffa40566700 1 Processor -- start 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.860+0000 7ffa40566700 1 -- start start 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa40566700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809d6f0 0x7ffa381221a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa40566700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa381226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa40566700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa3809e410 0x7ffa38122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa40566700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa38125cc0 con 0x7ffa3809e410 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa40566700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa381243d0 con 0x7ffa3809d6f0 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa40566700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3809c700 con 0x7ffa3809e000 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3e302700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809d6f0 0x7ffa381221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3e302700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809d6f0 0x7ffa381221a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52376/0 (socket says 172.21.15.89:52376) 2023-09-30T22:21:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3e302700 1 -- 172.21.15.89:0/3840429508 learned_addr learned my addr 172.21.15.89:0/3840429508 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3eb03700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa3809e410 0x7ffa38122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3db01700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa381226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3db01700 1 -- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809d6f0 msgr2=0x7ffa381221a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3db01700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809d6f0 0x7ffa381221a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3db01700 1 -- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa3809e410 msgr2=0x7ffa38122c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3db01700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa3809e410 0x7ffa38122c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3db01700 1 -- 172.21.15.89:0/3840429508 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3400b7f0 con 0x7ffa3809e000 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3e302700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809d6f0 0x7ffa381221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.861+0000 7ffa3eb03700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa3809e410 0x7ffa38122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.862+0000 7ffa3db01700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa381226e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800e900 tx=0x7ffa2800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.862+0000 7ffa2effd700 1 -- 172.21.15.89:0/3840429508 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28013680 con 0x7ffa3809e000 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.862+0000 7ffa2effd700 1 -- 172.21.15.89:0/3840429508 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa280137e0 con 0x7ffa3809e000 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.862+0000 7ffa2effd700 1 -- 172.21.15.89:0/3840429508 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28013b50 con 0x7ffa3809e000 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.862+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa30009a70 con 0x7ffa3809e000 2023-09-30T22:21:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.862+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa38123e10 con 0x7ffa3809e000 2023-09-30T22:21:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.863+0000 7ffa2effd700 1 -- 172.21.15.89:0/3840429508 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa28013cb0 con 0x7ffa3809e000 2023-09-30T22:21:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.863+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa38074ea0 con 0x7ffa3809e000 2023-09-30T22:21:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.863+0000 7ffa2effd700 1 -- 172.21.15.89:0/3840429508 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffa2803f500 con 0x7ffa3809e000 2023-09-30T22:21:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.867+0000 7ffa2effd700 1 -- 172.21.15.89:0/3840429508 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa2800c770 con 0x7ffa3809e000 2023-09-30T22:21:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.992+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f289006b9e0 con 0x7f289009eca0 2023-09-30T22:21:39.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.993+0000 7f28777fe700 1 -- 172.21.15.89:0/157266332 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2880011e80 con 0x7f289009eca0 2023-09-30T22:21:39.993 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:39.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f289009eca0 msgr2=0x7f289009c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f289009eca0 0x7f289009c1b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f288000c470 tx=0x7f288000c780 comp rx=0 tx=0).stop 2023-09-30T22:21:39.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 shutdown_connections 2023-09-30T22:21:39.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2890135a90 0x7f289009cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f289009eca0 0x7f289009c1b0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 --2- 172.21.15.89:0/157266332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2890130560 0x7f289009c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:39.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 >> 172.21.15.89:0/157266332 conn(0x7f28900a2aa0 msgr2=0x7f289012e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:39.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 shutdown_connections 2023-09-30T22:21:39.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:39.996+0000 7f289554e700 1 -- 172.21.15.89:0/157266332 wait complete. 2023-09-30T22:21:40.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.038+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4e7c133b80 con 0x7f4e7c06cc90 2023-09-30T22:21:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.039+0000 7f4e797fa700 1 -- 172.21.15.89:0/3639215319 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e70011360 con 0x7f4e7c06cc90 2023-09-30T22:21:40.039 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c06cc90 msgr2=0x7f4e7c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c06cc90 0x7f4e7c06fe80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c520 tx=0x7f4e7000c830 comp rx=0 tx=0).stop 2023-09-30T22:21:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 shutdown_connections 2023-09-30T22:21:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c0a15b0 0x7f4e7c070900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c06cc90 0x7f4e7c06fe80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 --2- 172.21.15.89:0/3639215319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0a1140 0x7f4e7c0703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 >> 172.21.15.89:0/3639215319 conn(0x7f4e7c0a3050 msgr2=0x7f4e7c099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 shutdown_connections 2023-09-30T22:21:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.041+0000 7f4e82be3700 1 -- 172.21.15.89:0/3639215319 wait complete. 2023-09-30T22:21:40.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.052+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6af0123640 con 0x7f6af009d6f0 2023-09-30T22:21:40.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.052+0000 7f6ae5ffb700 1 -- 172.21.15.89:0/18338364 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6aec020030 con 0x7f6af009d6f0 2023-09-30T22:21:40.052 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:40.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af009d6f0 msgr2=0x7f6af0121290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 --2- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af009d6f0 0x7f6af0121290 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009ec0 tx=0x7f6aec00eec0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 shutdown_connections 2023-09-30T22:21:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 --2- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6af009e410 0x7f6af0121d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 --2- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af009d6f0 0x7f6af0121290 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 --2- 172.21.15.89:0/18338364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6af009e000 0x7f6af01217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 >> 172.21.15.89:0/18338364 conn(0x7f6af0066f50 msgr2=0x7f6af0119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 shutdown_connections 2023-09-30T22:21:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.054+0000 7f6af74b8700 1 -- 172.21.15.89:0/18338364 wait complete. 2023-09-30T22:21:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.061+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa38123db0 con 0x7ffa3809e000 2023-09-30T22:21:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.061+0000 7ffa2effd700 1 -- 172.21.15.89:0/3840429508 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa28017070 con 0x7ffa3809e000 2023-09-30T22:21:40.061 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 msgr2=0x7ffa381226e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa381226e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800e900 tx=0x7ffa2800ec10 comp rx=0 tx=0).stop 2023-09-30T22:21:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 shutdown_connections 2023-09-30T22:21:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa3809e410 0x7ffa38122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa3809d6f0 0x7ffa381221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 --2- 172.21.15.89:0/3840429508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa3809e000 0x7ffa381226e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 >> 172.21.15.89:0/3840429508 conn(0x7ffa38066f50 msgr2=0x7ffa38070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 shutdown_connections 2023-09-30T22:21:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:40.063+0000 7ffa40566700 1 -- 172.21.15.89:0/3840429508 wait complete. 2023-09-30T22:21:40.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:40.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:41.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:41.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:41.067 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:41.076 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:41.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.162+0000 7fabc59cb700 1 Processor -- start 2023-09-30T22:21:41.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.162+0000 7fabc59cb700 1 -- start start 2023-09-30T22:21:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.162+0000 7fabc59cb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc0138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.162+0000 7fabc59cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 0x7fabc013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.162+0000 7fabc59cb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc006ef30 con 0x7fabc01338a0 2023-09-30T22:21:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabc59cb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc006fef0 con 0x7fabc0136710 2023-09-30T22:21:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabc59cb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabc01334b0 con 0x7fabc0139030 2023-09-30T22:21:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbe7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc0138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbe7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc0138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46024/0 (socket says 172.21.15.89:46024) 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbe7fc700 1 -- 172.21.15.89:0/1281801488 learned_addr learned my addr 172.21.15.89:0/1281801488 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbf7fe700 1 --2- 172.21.15.89:0/1281801488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 0x7fabc013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbe7fc700 1 -- 172.21.15.89:0/1281801488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 msgr2=0x7fabc013b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbe7fc700 1 --2- 172.21.15.89:0/1281801488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 0x7fabc013b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbe7fc700 1 -- 172.21.15.89:0/1281801488 >> v1:172.21.15.89:6789/0 conn(0x7fabc01338a0 legacy=0x7fabc0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbdffb700 1 -- 172.21.15.89:0/1281801488 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550619749 0 0) 0x7fabc006ef30 con 0x7fabc01338a0 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbe7fc700 1 -- 172.21.15.89:0/1281801488 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc006fc30 con 0x7fabc0136710 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbe7fc700 1 --2- 172.21.15.89:0/1281801488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc0138af0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009920 tx=0x7fabb4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4e55c0a488a69b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.163+0000 7fabbdffb700 1 -- 172.21.15.89:0/1281801488 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4011830 con 0x7fabc0136710 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.164+0000 7fabbdffb700 1 -- 172.21.15.89:0/1281801488 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb4011990 con 0x7fabc0136710 2023-09-30T22:21:41.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.164+0000 7fabbdffb700 1 -- 172.21.15.89:0/1281801488 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4011d20 con 0x7fabc0136710 2023-09-30T22:21:41.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.164+0000 7fabc59cb700 1 -- 172.21.15.89:0/1281801488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 msgr2=0x7fabc0138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.164+0000 7fabc59cb700 1 --2- 172.21.15.89:0/1281801488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc0138af0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009920 tx=0x7fabb4009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.165+0000 7fabc59cb700 1 -- 172.21.15.89:0/1281801488 shutdown_connections 2023-09-30T22:21:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.165+0000 7fabc59cb700 1 --2- 172.21.15.89:0/1281801488 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 0x7fabc013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.165+0000 7fabc59cb700 1 --2- 172.21.15.89:0/1281801488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc0138af0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.165+0000 7fabc59cb700 1 -- 172.21.15.89:0/1281801488 >> 172.21.15.89:0/1281801488 conn(0x7fabc00a2aa0 msgr2=0x7fabc00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.165+0000 7fabc59cb700 1 -- 172.21.15.89:0/1281801488 shutdown_connections 2023-09-30T22:21:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.165+0000 7fabc59cb700 1 -- 172.21.15.89:0/1281801488 wait complete. 2023-09-30T22:21:41.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.166+0000 7fabc59cb700 1 Processor -- start 2023-09-30T22:21:41.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.166+0000 7fabc59cb700 1 -- start start 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabc59cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc01338a0 0x7fabc01c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabc59cb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc01c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabc59cb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 0x7fabc01c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabc59cb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc011d730 con 0x7fabc01338a0 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabc59cb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc011d320 con 0x7fabc0136710 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabc59cb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabc011e9d0 con 0x7fabc0139030 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabbeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc01338a0 0x7fabc01c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabbeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc01338a0 0x7fabc01c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46168/0 (socket says 172.21.15.89:46168) 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabbeffd700 1 -- 172.21.15.89:0/1614560898 learned_addr learned my addr 172.21.15.89:0/1614560898 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabbf7fe700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 0x7fabc01c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.167+0000 7fabbe7fc700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc01c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7fabbeffd700 1 -- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 msgr2=0x7fabc01c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7fabbeffd700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 0x7fabc01c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7fabbeffd700 1 -- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 msgr2=0x7fabc01c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7fabbeffd700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc01c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7fabbeffd700 1 -- 172.21.15.89:0/1614560898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4009550 con 0x7fabc01338a0 2023-09-30T22:21:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7fabbeffd700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc01338a0 0x7fabc01c8220 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c520 tx=0x7fabb000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7faba7fff700 1 -- 172.21.15.89:0/1614560898 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00054e0 con 0x7fabc01338a0 2023-09-30T22:21:41.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc00a23f0 con 0x7fabc01338a0 2023-09-30T22:21:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7faba7fff700 1 -- 172.21.15.89:0/1614560898 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb0003680 con 0x7fabc01338a0 2023-09-30T22:21:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.168+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc011e6f0 con 0x7fabc01338a0 2023-09-30T22:21:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.169+0000 7faba7fff700 1 -- 172.21.15.89:0/1614560898 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00039f0 con 0x7fabc01338a0 2023-09-30T22:21:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.169+0000 7faba7fff700 1 -- 172.21.15.89:0/1614560898 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fabb0003bd0 con 0x7fabc01338a0 2023-09-30T22:21:41.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.169+0000 7faba7fff700 1 -- 172.21.15.89:0/1614560898 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fabb003ea40 con 0x7fabc01338a0 2023-09-30T22:21:41.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.170+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba00052e0 con 0x7fabc01338a0 2023-09-30T22:21:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.178+0000 7faba7fff700 1 -- 172.21.15.89:0/1614560898 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fabb0017080 con 0x7fabc01338a0 2023-09-30T22:21:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.203+0000 7fbff8b5c700 1 Processor -- start 2023-09-30T22:21:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.203+0000 7fbff8b5c700 1 -- start start 2023-09-30T22:21:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff8b5c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff41338a0 0x7fbff4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff8b5c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbff409de50 0x7fbff409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff8b5c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff406ef30 con 0x7fbff409e760 2023-09-30T22:21:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff8b5c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff406fef0 con 0x7fbff409de50 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff8b5c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbff41334b0 con 0x7fbff41338a0 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff41338a0 0x7fbff4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff1d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbff409de50 0x7fbff409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff2d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbff409e760 0x7fbff413a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38108/0 (socket says 172.21.15.89:38108) 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff2d9d700 1 -- 172.21.15.89:0/1645115347 learned_addr learned my addr 172.21.15.89:0/1645115347 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff259c700 1 -- 172.21.15.89:0/1645115347 >> v2:172.21.15.89:3300/0 conn(0x7fbff409de50 msgr2=0x7fbff409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff259c700 1 --2- 172.21.15.89:0/1645115347 >> v2:172.21.15.89:3300/0 conn(0x7fbff409de50 0x7fbff409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff259c700 1 -- 172.21.15.89:0/1645115347 >> v1:172.21.15.89:6789/0 conn(0x7fbff409e760 legacy=0x7fbff413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff259c700 1 -- 172.21.15.89:0/1645115347 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff406fc30 con 0x7fbff41338a0 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff1d9b700 1 --2- 172.21.15.89:0/1645115347 >> v2:172.21.15.89:3300/0 conn(0x7fbff409de50 0x7fbff409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff259c700 1 --2- 172.21.15.89:0/1645115347 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff41338a0 0x7fbff4133c70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009840 tx=0x7fbfe4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab273cf68dc08e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.204+0000 7fbff159a700 1 -- 172.21.15.89:0/1645115347 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4011830 con 0x7fbff41338a0 2023-09-30T22:21:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff159a700 1 -- 172.21.15.89:0/1645115347 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4011990 con 0x7fbff41338a0 2023-09-30T22:21:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff159a700 1 -- 172.21.15.89:0/1645115347 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4011d20 con 0x7fbff41338a0 2023-09-30T22:21:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff8b5c700 1 -- 172.21.15.89:0/1645115347 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff41338a0 msgr2=0x7fbff4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff8b5c700 1 --2- 172.21.15.89:0/1645115347 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff41338a0 0x7fbff4133c70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009840 tx=0x7fbfe4009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff8b5c700 1 -- 172.21.15.89:0/1645115347 shutdown_connections 2023-09-30T22:21:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff8b5c700 1 --2- 172.21.15.89:0/1645115347 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff41338a0 0x7fbff4133c70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff8b5c700 1 --2- 172.21.15.89:0/1645115347 >> v2:172.21.15.89:3300/0 conn(0x7fbff409de50 0x7fbff409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff8b5c700 1 -- 172.21.15.89:0/1645115347 >> 172.21.15.89:0/1645115347 conn(0x7fbff40a2aa0 msgr2=0x7fbff40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff8b5c700 1 -- 172.21.15.89:0/1645115347 shutdown_connections 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.205+0000 7fbff8b5c700 1 -- 172.21.15.89:0/1645115347 wait complete. 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.206+0000 7fbff8b5c700 1 Processor -- start 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff8b5c700 1 -- start start 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff8b5c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbff409de50 0x7fbff411d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff8b5c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff409e760 0x7fbff411dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff8b5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbff41338a0 0x7fbff411e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff8b5c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff411f300 con 0x7fbff41338a0 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff8b5c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff411f920 con 0x7fbff409de50 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff8b5c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4120550 con 0x7fbff409e760 2023-09-30T22:21:41.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff1d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff409e760 0x7fbff411dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff1d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff409e760 0x7fbff411dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57546/0 (socket says 172.21.15.89:57546) 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff1d9b700 1 -- 172.21.15.89:0/3330989642 learned_addr learned my addr 172.21.15.89:0/3330989642 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff2d9d700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbff41338a0 0x7fbff411e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff2d9d700 1 -- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff409e760 msgr2=0x7fbff411dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff2d9d700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff409e760 0x7fbff411dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff2d9d700 1 -- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbff409de50 msgr2=0x7fbff411d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff2d9d700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbff409de50 0x7fbff411d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.207+0000 7fbff2d9d700 1 -- 172.21.15.89:0/3330989642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80094a0 con 0x7fbff41338a0 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.208+0000 7fbff1d9b700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff409e760 0x7fbff411dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.208+0000 7fbff2d9d700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbff41338a0 0x7fbff411e170 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00cad0 tx=0x7fbfec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.208+0000 7fbfe2ffd700 1 -- 172.21.15.89:0/3330989642 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00aa20 con 0x7fbff41338a0 2023-09-30T22:21:41.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.208+0000 7fbfe2ffd700 1 -- 172.21.15.89:0/3330989642 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00ab80 con 0x7fbff41338a0 2023-09-30T22:21:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.208+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe40094a0 con 0x7fbff41338a0 2023-09-30T22:21:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.208+0000 7fbfe2ffd700 1 -- 172.21.15.89:0/3330989642 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec012400 con 0x7fbff41338a0 2023-09-30T22:21:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.208+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff40a0fd0 con 0x7fbff41338a0 2023-09-30T22:21:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.209+0000 7fbfe2ffd700 1 -- 172.21.15.89:0/3330989642 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbfec01b3b0 con 0x7fbff41338a0 2023-09-30T22:21:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.209+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff406f6a0 con 0x7fbff41338a0 2023-09-30T22:21:41.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.209+0000 7fbfe2ffd700 1 -- 172.21.15.89:0/3330989642 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbfec03f240 con 0x7fbff41338a0 2023-09-30T22:21:41.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.218+0000 7fbfe2ffd700 1 -- 172.21.15.89:0/3330989642 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbfec0192c0 con 0x7fbff41338a0 2023-09-30T22:21:41.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.242+0000 7fc9e1c4c700 1 Processor -- start 2023-09-30T22:21:41.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.242+0000 7fc9e1c4c700 1 -- start start 2023-09-30T22:21:41.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9e1c4c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc130540 0x7fc9dc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9e1c4c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9dc06fcf0 0x7fc9dc138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9e1c4c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9dc0716a0 con 0x7fc9dc130e50 2023-09-30T22:21:41.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9e1c4c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9dc09a390 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9e1c4c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc071290 con 0x7fc9dc130540 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9db7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc130540 0x7fc9dc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9dbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9dc06fcf0 0x7fc9dc138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9db7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc130540 0x7fc9dc130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57550/0 (socket says 172.21.15.89:57550) 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9db7fe700 1 -- 172.21.15.89:0/2777035504 learned_addr learned my addr 172.21.15.89:0/2777035504 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9daffd700 1 --1- 172.21.15.89:0/2777035504 >> v1:172.21.15.89:6789/0 conn(0x7fc9dc130e50 0x7fc9dc06f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38120/0 (socket says 172.21.15.89:38120) 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9dbfff700 1 -- 172.21.15.89:0/2777035504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc130540 msgr2=0x7fc9dc130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9dbfff700 1 --2- 172.21.15.89:0/2777035504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc130540 0x7fc9dc130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9dbfff700 1 -- 172.21.15.89:0/2777035504 >> v1:172.21.15.89:6789/0 conn(0x7fc9dc130e50 legacy=0x7fc9dc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9dbfff700 1 -- 172.21.15.89:0/2777035504 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc130340 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9da7fc700 1 -- 172.21.15.89:0/2777035504 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942892783 0 0) 0x7fc9dc0716a0 con 0x7fc9dc130e50 2023-09-30T22:21:41.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9dbfff700 1 --2- 172.21.15.89:0/2777035504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc06fcf0 0x7fc9dc138570 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009d90 tx=0x7fc9d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bcb012bea55d04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.243+0000 7fc9da7fc700 1 -- 172.21.15.89:0/2777035504 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400e070 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9da7fc700 1 -- 172.21.15.89:0/2777035504 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d400bb20 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9da7fc700 1 -- 172.21.15.89:0/2777035504 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4012430 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/2777035504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc06fcf0 msgr2=0x7fc9dc138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9e1c4c700 1 --2- 172.21.15.89:0/2777035504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc06fcf0 0x7fc9dc138570 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009d90 tx=0x7fc9d400b330 comp rx=0 tx=0).stop 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/2777035504 shutdown_connections 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9e1c4c700 1 --2- 172.21.15.89:0/2777035504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc06fcf0 0x7fc9dc138570 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9e1c4c700 1 --2- 172.21.15.89:0/2777035504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc130540 0x7fc9dc130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/2777035504 >> 172.21.15.89:0/2777035504 conn(0x7fc9dc0a2aa0 msgr2=0x7fc9dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/2777035504 shutdown_connections 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.244+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/2777035504 wait complete. 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.245+0000 7fc9e1c4c700 1 Processor -- start 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.245+0000 7fc9e1c4c700 1 -- start start 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9e1c4c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc06fcf0 0x7fc9dc1c0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9e1c4c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc130540 0x7fc9dc1c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9e1c4c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9dc130e50 0x7fc9dc1c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9e1c4c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc1c1390 con 0x7fc9dc130540 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9e1c4c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc11deb0 con 0x7fc9dc130e50 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9e1c4c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc11bfa0 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc06fcf0 0x7fc9dc1c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc06fcf0 0x7fc9dc1c0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57566/0 (socket says 172.21.15.89:57566) 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 -- 172.21.15.89:0/3427505319 learned_addr learned my addr 172.21.15.89:0/3427505319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9daffd700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc130540 0x7fc9dc1c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9dbfff700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9dc130e50 0x7fc9dc1c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 -- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9dc130e50 msgr2=0x7fc9dc1c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9dc130e50 0x7fc9dc1c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 -- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc130540 msgr2=0x7fc9dc1c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc130540 0x7fc9dc1c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 -- 172.21.15.89:0/3427505319 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4009a70 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.246+0000 7fc9db7fe700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc06fcf0 0x7fc9dc1c0910 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00bfd0 tx=0x7fc9cc00fe60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.247+0000 7fc9c3fff700 1 -- 172.21.15.89:0/3427505319 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00dc50 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.247+0000 7fc9c3fff700 1 -- 172.21.15.89:0/3427505319 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc00ddb0 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.247+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc0a08a0 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.247+0000 7fc9c3fff700 1 -- 172.21.15.89:0/3427505319 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc018680 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.247+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc071930 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.248+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc11db30 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.248+0000 7fc9c3fff700 1 -- 172.21.15.89:0/3427505319 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc9cc0187e0 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.248+0000 7fc9c3fff700 1 -- 172.21.15.89:0/3427505319 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc9cc03fd50 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.253+0000 7fc9c3fff700 1 -- 172.21.15.89:0/3427505319 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9cc010440 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.271+0000 7f9e06594700 1 Processor -- start 2023-09-30T22:21:41.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.271+0000 7f9e06594700 1 -- start start 2023-09-30T22:21:41.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9e06594700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9e06594700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e00133e30 0x7f9e0012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9e06594700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00134260 0x7f9e001304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9e06594700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0006b990 con 0x7f9e00133e30 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9e06594700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000a0590 con 0x7f9e00133420 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9e06594700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0009fb80 con 0x7f9e00134260 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dff7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e00133e30 0x7f9e0012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dff7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e00133e30 0x7f9e0012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46202/0 (socket says 172.21.15.89:46202) 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dff7fe700 1 -- 172.21.15.89:0/3183484429 learned_addr learned my addr 172.21.15.89:0/3183484429 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dfffff700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9e04b31700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00134260 0x7f9e001304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dff7fe700 1 -- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00134260 msgr2=0x7f9e001304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dff7fe700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00134260 0x7f9e001304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dff7fe700 1 -- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00133420 msgr2=0x7f9e001337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dff7fe700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.272+0000 7f9dff7fe700 1 -- 172.21.15.89:0/3183484429 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000a0030 con 0x7f9e00133e30 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.273+0000 7f9dff7fe700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 0x7f9e0012ff90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9df40131e0 tx=0x7f9df40134f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6d1c0aaae4808a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.273+0000 7f9dfeffd700 1 -- 172.21.15.89:0/3183484429 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4016070 con 0x7f9e00133e30 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.273+0000 7f9dfeffd700 1 -- 172.21.15.89:0/3183484429 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4013d80 con 0x7f9e00133e30 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.273+0000 7f9dfeffd700 1 -- 172.21.15.89:0/3183484429 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40112f0 con 0x7f9e00133e30 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 -- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 msgr2=0x7f9e0012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 0x7f9e0012ff90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9df40131e0 tx=0x7f9df40134f0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 -- 172.21.15.89:0/3183484429 shutdown_connections 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00134260 0x7f9e001304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00133420 0x7f9e001337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 --2- 172.21.15.89:0/3183484429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 0x7f9e0012ff90 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 -- 172.21.15.89:0/3183484429 >> 172.21.15.89:0/3183484429 conn(0x7f9e000a4210 msgr2=0x7f9e000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 -- 172.21.15.89:0/3183484429 shutdown_connections 2023-09-30T22:21:41.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.274+0000 7f9e06594700 1 -- 172.21.15.89:0/3183484429 wait complete. 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.275+0000 7f9e06594700 1 Processor -- start 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.275+0000 7f9e06594700 1 -- start start 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9e06594700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00133420 0x7f9e0009be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9e06594700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 0x7f9e0009c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9e06594700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00134260 0x7f9e0009c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9e06594700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0011c390 con 0x7f9e00133e30 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9e06594700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0011b9d0 con 0x7f9e00134260 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9e06594700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0011b160 con 0x7f9e00133420 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9dfffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00133420 0x7f9e0009be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9dfffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00133420 0x7f9e0009be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57582/0 (socket says 172.21.15.89:57582) 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9dfffff700 1 -- 172.21.15.89:0/308774753 learned_addr learned my addr 172.21.15.89:0/308774753 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9dff7fe700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 0x7f9e0009c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.276+0000 7f9e04b31700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00134260 0x7f9e0009c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dff7fe700 1 -- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00133420 msgr2=0x7f9e0009be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dff7fe700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00133420 0x7f9e0009be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dff7fe700 1 -- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00134260 msgr2=0x7f9e0009c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dff7fe700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00134260 0x7f9e0009c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dff7fe700 1 -- 172.21.15.89:0/308774753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4011df0 con 0x7f9e00133e30 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dff7fe700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 0x7f9e0009c3d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9df401d000 tx=0x7f9df4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dfcff9700 1 -- 172.21.15.89:0/308774753 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400b070 con 0x7f9e00133e30 2023-09-30T22:21:41.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dfcff9700 1 -- 172.21.15.89:0/308774753 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4011c70 con 0x7f9e00133e30 2023-09-30T22:21:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9dfcff9700 1 -- 172.21.15.89:0/308774753 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40036a0 con 0x7f9e00133e30 2023-09-30T22:21:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e00131490 con 0x7f9e00133e30 2023-09-30T22:21:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.277+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0009daa0 con 0x7f9e00133e30 2023-09-30T22:21:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.278+0000 7f9dfcff9700 1 -- 172.21.15.89:0/308774753 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9df4003800 con 0x7f9e00133e30 2023-09-30T22:21:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.279+0000 7f9dfcff9700 1 -- 172.21.15.89:0/308774753 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9df4040550 con 0x7f9e00133e30 2023-09-30T22:21:41.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.279+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0011b390 con 0x7f9e00133e30 2023-09-30T22:21:41.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.288+0000 7f9dfcff9700 1 -- 172.21.15.89:0/308774753 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e0011b390 con 0x7f9e00133e30 2023-09-30T22:21:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.417+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbff411a470 con 0x7fbff41338a0 2023-09-30T22:21:41.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.417+0000 7fbfe2ffd700 1 -- 172.21.15.89:0/3330989642 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbfec017030 con 0x7fbff41338a0 2023-09-30T22:21:41.417 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbff41338a0 msgr2=0x7fbff411e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbff41338a0 0x7fbff411e170 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00cad0 tx=0x7fbfec00cde0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 shutdown_connections 2023-09-30T22:21:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbff41338a0 0x7fbff411e170 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbff409de50 0x7fbff411d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 --2- 172.21.15.89:0/3330989642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbff409e760 0x7fbff411dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 >> 172.21.15.89:0/3330989642 conn(0x7fbff40a2aa0 msgr2=0x7fbff41bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 shutdown_connections 2023-09-30T22:21:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.419+0000 7fbff8b5c700 1 -- 172.21.15.89:0/3330989642 wait complete. 2023-09-30T22:21:41.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.431+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faba0005590 con 0x7fabc01338a0 2023-09-30T22:21:41.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:41.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.432+0000 7faba7fff700 1 -- 172.21.15.89:0/1614560898 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fabb0010b60 con 0x7fabc01338a0 2023-09-30T22:21:41.432 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:41.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.435+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc01338a0 msgr2=0x7fabc01c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.435+0000 7fabc59cb700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc01338a0 0x7fabc01c8220 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c520 tx=0x7fabb000c830 comp rx=0 tx=0).stop 2023-09-30T22:21:41.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.436+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 shutdown_connections 2023-09-30T22:21:41.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.436+0000 7fabc59cb700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabc0139030 0x7fabc01c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.436+0000 7fabc59cb700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc01338a0 0x7fabc01c8220 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.436+0000 7fabc59cb700 1 --2- 172.21.15.89:0/1614560898 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabc0136710 0x7fabc01c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.436+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 >> 172.21.15.89:0/1614560898 conn(0x7fabc00a2aa0 msgr2=0x7fabc00a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.436+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 shutdown_connections 2023-09-30T22:21:41.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.436+0000 7fabc59cb700 1 -- 172.21.15.89:0/1614560898 wait complete. 2023-09-30T22:21:41.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:41.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.465+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc9dc074ea0 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.465+0000 7fc9c3fff700 1 -- 172.21.15.89:0/3427505319 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9cc01d030 con 0x7fc9dc06fcf0 2023-09-30T22:21:41.465 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc06fcf0 msgr2=0x7fc9dc1c0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc06fcf0 0x7fc9dc1c0910 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00bfd0 tx=0x7fc9cc00fe60 comp rx=0 tx=0).stop 2023-09-30T22:21:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 shutdown_connections 2023-09-30T22:21:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9dc130e50 0x7fc9dc1c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc9dc06fcf0 0x7fc9dc1c0910 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 --2- 172.21.15.89:0/3427505319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9dc130540 0x7fc9dc1c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 >> 172.21.15.89:0/3427505319 conn(0x7fc9dc0a2aa0 msgr2=0x7fc9dc12d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 shutdown_connections 2023-09-30T22:21:41.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.467+0000 7fc9e1c4c700 1 -- 172.21.15.89:0/3427505319 wait complete. 2023-09-30T22:21:41.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:41.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.501+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e0011b390 con 0x7f9e00133e30 2023-09-30T22:21:41.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.501+0000 7f9dfcff9700 1 -- 172.21.15.89:0/308774753 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e0011b390 con 0x7f9e00133e30 2023-09-30T22:21:41.501 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:41.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 msgr2=0x7f9e0009c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 0x7f9e0009c3d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9df401d000 tx=0x7f9df4004750 comp rx=0 tx=0).stop 2023-09-30T22:21:41.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 shutdown_connections 2023-09-30T22:21:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e00134260 0x7f9e0009c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e00133420 0x7f9e0009be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 --2- 172.21.15.89:0/308774753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e00133e30 0x7f9e0009c3d0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 >> 172.21.15.89:0/308774753 conn(0x7f9e000a4210 msgr2=0x7f9e001302e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 shutdown_connections 2023-09-30T22:21:41.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:41.503+0000 7f9e06594700 1 -- 172.21.15.89:0/308774753 wait complete. 2023-09-30T22:21:41.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:42.432 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:42.453 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:42.480 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:42.516 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:42.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.574+0000 7f862670d700 1 Processor -- start 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.575+0000 7f862670d700 1 -- start start 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.575+0000 7f862670d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f862009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.575+0000 7f862670d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f862009e000 0x7f86201339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.575+0000 7f862670d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862009b120 con 0x7f862009e410 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.575+0000 7f862670d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862006b460 con 0x7f862009e000 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.575+0000 7f862670d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862009a7e0 con 0x7f862009d6f0 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.575+0000 7f861ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f862009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.575+0000 7f861f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f862009e000 0x7f86201339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f862009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46046/0 (socket says 172.21.15.89:46046) 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f862009e000 0x7f86201339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46228/0 (socket says 172.21.15.89:46228) 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861ffff700 1 -- 172.21.15.89:0/2757999870 learned_addr learned my addr 172.21.15.89:0/2757999870 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861f7fe700 1 -- 172.21.15.89:0/2757999870 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 msgr2=0x7f862009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861f7fe700 1 --2- 172.21.15.89:0/2757999870 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f862009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861f7fe700 1 -- 172.21.15.89:0/2757999870 >> v1:172.21.15.89:6789/0 conn(0x7f862009e410 legacy=0x7f8620133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861f7fe700 1 -- 172.21.15.89:0/2757999870 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862009a500 con 0x7f862009e000 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861effd700 1 -- 172.21.15.89:0/2757999870 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337820184 0 0) 0x7f862009b120 con 0x7f862009e410 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861ffff700 1 --2- 172.21.15.89:0/2757999870 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f862009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861f7fe700 1 --2- 172.21.15.89:0/2757999870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e000 0x7f86201339f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8614009840 tx=0x7f8614009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b809c048331cc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861effd700 1 -- 172.21.15.89:0/2757999870 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614011830 con 0x7f862009e000 2023-09-30T22:21:42.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861effd700 1 -- 172.21.15.89:0/2757999870 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8614011990 con 0x7f862009e000 2023-09-30T22:21:42.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.576+0000 7f861effd700 1 -- 172.21.15.89:0/2757999870 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614011d20 con 0x7f862009e000 2023-09-30T22:21:42.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.577+0000 7f862670d700 1 -- 172.21.15.89:0/2757999870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e000 msgr2=0x7f86201339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.577+0000 7f862670d700 1 --2- 172.21.15.89:0/2757999870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e000 0x7f86201339f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8614009840 tx=0x7f8614009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:42.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.577+0000 7f862670d700 1 -- 172.21.15.89:0/2757999870 shutdown_connections 2023-09-30T22:21:42.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.577+0000 7f862670d700 1 --2- 172.21.15.89:0/2757999870 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f862009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.577+0000 7f862670d700 1 --2- 172.21.15.89:0/2757999870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e000 0x7f86201339f0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.577+0000 7f862670d700 1 -- 172.21.15.89:0/2757999870 >> 172.21.15.89:0/2757999870 conn(0x7f8620066f50 msgr2=0x7f862009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.577+0000 7f862670d700 1 -- 172.21.15.89:0/2757999870 shutdown_connections 2023-09-30T22:21:42.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.577+0000 7f862670d700 1 -- 172.21.15.89:0/2757999870 wait complete. 2023-09-30T22:21:42.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.579+0000 7f862670d700 1 Processor -- start 2023-09-30T22:21:42.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.579+0000 7f862670d700 1 -- start start 2023-09-30T22:21:42.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f862670d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f86201c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f862670d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009e000 0x7f86201c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f862670d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e410 0x7f86201c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f862670d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86201a64d0 con 0x7f862009e410 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f862670d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86201a6b20 con 0x7f862009d6f0 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f862670d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86201a83e0 con 0x7f862009e000 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f861ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f86201c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f861f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009e000 0x7f86201c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f861ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f86201c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46054/0 (socket says 172.21.15.89:46054) 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f861ffff700 1 -- 172.21.15.89:0/157950025 learned_addr learned my addr 172.21.15.89:0/157950025 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:42.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f8624caa700 1 --2- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e410 0x7f86201c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f8624caa700 1 -- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009e000 msgr2=0x7f86201c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f8624caa700 1 --2- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009e000 0x7f86201c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f8624caa700 1 -- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 msgr2=0x7f86201c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f8624caa700 1 --2- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f86201c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f8624caa700 1 -- 172.21.15.89:0/157950025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8610009550 con 0x7f862009e410 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.580+0000 7f8624caa700 1 --2- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e410 0x7f86201c8ad0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f861800c730 tx=0x7f861800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.581+0000 7f861cff9700 1 -- 172.21.15.89:0/157950025 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618003e30 con 0x7f862009e410 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.581+0000 7f861cff9700 1 -- 172.21.15.89:0/157950025 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8618005070 con 0x7f862009e410 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.581+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86140094a0 con 0x7f862009e410 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.581+0000 7f861cff9700 1 -- 172.21.15.89:0/157950025 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618005420 con 0x7f862009e410 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.581+0000 7f861cff9700 1 -- 172.21.15.89:0/157950025 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8618005580 con 0x7f862009e410 2023-09-30T22:21:42.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.582+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86201a8100 con 0x7f862009e410 2023-09-30T22:21:42.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.582+0000 7f861cff9700 1 -- 172.21.15.89:0/157950025 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8618011df0 con 0x7f862009e410 2023-09-30T22:21:42.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.582+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86201a7d80 con 0x7f862009e410 2023-09-30T22:21:42.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.587+0000 7f861cff9700 1 -- 172.21.15.89:0/157950025 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8618010710 con 0x7f862009e410 2023-09-30T22:21:42.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.622+0000 7f0af721d700 1 Processor -- start 2023-09-30T22:21:42.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.623+0000 7f0af721d700 1 -- start start 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.623+0000 7f0af721d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009d6f0 0x7f0af009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.623+0000 7f0af721d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0af009e000 0x7f0af01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.623+0000 7f0af721d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e410 0x7f0af0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.623+0000 7f0af721d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af006b460 con 0x7f0af009e000 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.623+0000 7f0af721d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af009b120 con 0x7f0af009d6f0 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.623+0000 7f0af721d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0af009a7e0 con 0x7f0af009e410 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.623+0000 7f0af4fb9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009d6f0 0x7f0af009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0af4fb9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009d6f0 0x7f0af009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46068/0 (socket says 172.21.15.89:46068) 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0af4fb9700 1 -- 172.21.15.89:0/1022914136 learned_addr learned my addr 172.21.15.89:0/1022914136 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:42.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0aeffff700 1 --2- 172.21.15.89:0/1022914136 >> v2:172.21.15.89:3300/0 conn(0x7f0af009e000 0x7f0af01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0af4fb9700 1 -- 172.21.15.89:0/1022914136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e410 msgr2=0x7f0af0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0af4fb9700 1 --2- 172.21.15.89:0/1022914136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e410 0x7f0af0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0af4fb9700 1 -- 172.21.15.89:0/1022914136 >> v2:172.21.15.89:3300/0 conn(0x7f0af009e000 msgr2=0x7f0af01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0af4fb9700 1 --2- 172.21.15.89:0/1022914136 >> v2:172.21.15.89:3300/0 conn(0x7f0af009e000 0x7f0af01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0af4fb9700 1 -- 172.21.15.89:0/1022914136 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af009a500 con 0x7f0af009d6f0 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0aeffff700 1 --2- 172.21.15.89:0/1022914136 >> v2:172.21.15.89:3300/0 conn(0x7f0af009e000 0x7f0af01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0af4fb9700 1 --2- 172.21.15.89:0/1022914136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009d6f0 0x7f0af009dac0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009920 tx=0x7f0ae4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f38ccda7f3f43685 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0aef7fe700 1 -- 172.21.15.89:0/1022914136 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae400ba20 con 0x7f0af009d6f0 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0aef7fe700 1 -- 172.21.15.89:0/1022914136 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae400bb80 con 0x7f0af009d6f0 2023-09-30T22:21:42.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.624+0000 7f0aef7fe700 1 -- 172.21.15.89:0/1022914136 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4009e70 con 0x7f0af009d6f0 2023-09-30T22:21:42.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 -- 172.21.15.89:0/1022914136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009d6f0 msgr2=0x7f0af009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 --2- 172.21.15.89:0/1022914136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009d6f0 0x7f0af009dac0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009920 tx=0x7f0ae4009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:42.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 -- 172.21.15.89:0/1022914136 shutdown_connections 2023-09-30T22:21:42.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 --2- 172.21.15.89:0/1022914136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e410 0x7f0af0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 --2- 172.21.15.89:0/1022914136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009d6f0 0x7f0af009dac0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 --2- 172.21.15.89:0/1022914136 >> v2:172.21.15.89:3300/0 conn(0x7f0af009e000 0x7f0af01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 -- 172.21.15.89:0/1022914136 >> 172.21.15.89:0/1022914136 conn(0x7f0af0066f50 msgr2=0x7f0af009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 -- 172.21.15.89:0/1022914136 shutdown_connections 2023-09-30T22:21:42.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.625+0000 7f0af721d700 1 -- 172.21.15.89:0/1022914136 wait complete. 2023-09-30T22:21:42.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.627+0000 7f0af721d700 1 Processor -- start 2023-09-30T22:21:42.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.627+0000 7f0af721d700 1 -- start start 2023-09-30T22:21:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af721d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0af009d6f0 0x7f0af01a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af721d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e000 0x7f0af01a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af721d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009e410 0x7f0af01a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af721d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01aa2d0 con 0x7f0af009d6f0 2023-09-30T22:21:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af721d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01a9910 con 0x7f0af009e410 2023-09-30T22:21:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af721d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01a90a0 con 0x7f0af009e000 2023-09-30T22:21:42.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af57ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009e410 0x7f0af01a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af57ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009e410 0x7f0af01a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46084/0 (socket says 172.21.15.89:46084) 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af57ba700 1 -- 172.21.15.89:0/2188078787 learned_addr learned my addr 172.21.15.89:0/2188078787 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0aeffff700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e000 0x7f0af01a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af4fb9700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0af009d6f0 0x7f0af01a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0aeffff700 1 -- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009e410 msgr2=0x7f0af01a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0aeffff700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009e410 0x7f0af01a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0aeffff700 1 -- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0af009d6f0 msgr2=0x7f0af01a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0aeffff700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0af009d6f0 0x7f0af01a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0aeffff700 1 -- 172.21.15.89:0/2188078787 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae4009550 con 0x7f0af009e000 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0af57ba700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009e410 0x7f0af01a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.628+0000 7f0aeffff700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e000 0x7f0af01a6380 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00e810 tx=0x7f0adc00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.629+0000 7f0aed7fa700 1 -- 172.21.15.89:0/2188078787 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00c070 con 0x7f0af009e000 2023-09-30T22:21:42.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.629+0000 7f0aed7fa700 1 -- 172.21.15.89:0/2188078787 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0adc013610 con 0x7f0af009e000 2023-09-30T22:21:42.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.629+0000 7f0aed7fa700 1 -- 172.21.15.89:0/2188078787 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc013980 con 0x7f0af009e000 2023-09-30T22:21:42.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.629+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af01a8c50 con 0x7f0af009e000 2023-09-30T22:21:42.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.629+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af009c710 con 0x7f0af009e000 2023-09-30T22:21:42.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.630+0000 7f0aed7fa700 1 -- 172.21.15.89:0/2188078787 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0adc013ae0 con 0x7f0af009e000 2023-09-30T22:21:42.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.630+0000 7f0aed7fa700 1 -- 172.21.15.89:0/2188078787 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0adc0408b0 con 0x7f0af009e000 2023-09-30T22:21:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.630+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af0074ea0 con 0x7f0af009e000 2023-09-30T22:21:42.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.636+0000 7f0aed7fa700 1 -- 172.21.15.89:0/2188078787 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0adc044060 con 0x7f0af009e000 2023-09-30T22:21:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.660+0000 7fabbc734700 1 Processor -- start 2023-09-30T22:21:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.660+0000 7fabbc734700 1 -- start start 2023-09-30T22:21:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.660+0000 7f2a55873700 1 Processor -- start 2023-09-30T22:21:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.660+0000 7f2a55873700 1 -- start start 2023-09-30T22:21:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a55873700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a5009d6f0 0x7f2a5009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7fabbc734700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409d6f0 0x7fabb409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7fabbc734700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fabb409e000 0x7fabb41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7fabbc734700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb409b120 con 0x7fabb409e410 2023-09-30T22:21:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7fabbc734700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb406b460 con 0x7fabb409e000 2023-09-30T22:21:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7fabbc734700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabb409a7e0 con 0x7fabb409d6f0 2023-09-30T22:21:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a55873700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e440 0x7f2a50134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a55873700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5009b120 con 0x7f2a5009e000 2023-09-30T22:21:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a55873700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5006b460 con 0x7f2a5009d6f0 2023-09-30T22:21:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a55873700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5009a7e0 con 0x7f2a5009e440 2023-09-30T22:21:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a4f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e440 0x7f2a50134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a4f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e440 0x7f2a50134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57648/0 (socket says 172.21.15.89:57648) 2023-09-30T22:21:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a4f7fe700 1 -- 172.21.15.89:0/3513947993 learned_addr learned my addr 172.21.15.89:0/3513947993 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7fabba4d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409d6f0 0x7fabb409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7fabba4d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409d6f0 0x7fabb409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57634/0 (socket says 172.21.15.89:57634) 2023-09-30T22:21:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7fabba4d0700 1 -- 172.21.15.89:0/4028243749 learned_addr learned my addr 172.21.15.89:0/4028243749 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a4f7fe700 1 -- 172.21.15.89:0/3513947993 >> v2:172.21.15.89:3300/0 conn(0x7f2a5009d6f0 msgr2=0x7f2a5009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a4f7fe700 1 --2- 172.21.15.89:0/3513947993 >> v2:172.21.15.89:3300/0 conn(0x7f2a5009d6f0 0x7f2a5009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a4f7fe700 1 -- 172.21.15.89:0/3513947993 >> v1:172.21.15.89:6789/0 conn(0x7f2a5009e000 legacy=0x7f2a501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.661+0000 7f2a4f7fe700 1 -- 172.21.15.89:0/3513947993 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5009a500 con 0x7f2a5009e440 2023-09-30T22:21:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabb9ccf700 1 --2- 172.21.15.89:0/4028243749 >> v2:172.21.15.89:3300/0 conn(0x7fabb409e000 0x7fabb41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7f2a4f7fe700 1 --2- 172.21.15.89:0/3513947993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e440 0x7f2a50134100 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009e80 tx=0x7f2a4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9129004571000508 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabba4d0700 1 -- 172.21.15.89:0/4028243749 >> v2:172.21.15.89:3300/0 conn(0x7fabb409e000 msgr2=0x7fabb41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabba4d0700 1 --2- 172.21.15.89:0/4028243749 >> v2:172.21.15.89:3300/0 conn(0x7fabb409e000 0x7fabb41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabba4d0700 1 -- 172.21.15.89:0/4028243749 >> v1:172.21.15.89:6789/0 conn(0x7fabb409e410 legacy=0x7fabb4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabba4d0700 1 -- 172.21.15.89:0/4028243749 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb409a500 con 0x7fabb409d6f0 2023-09-30T22:21:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabba4d0700 1 --2- 172.21.15.89:0/4028243749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409d6f0 0x7fabb409dac0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009920 tx=0x7fabb0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26efb6d2ebd97d3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7f2a4dffb700 1 -- 172.21.15.89:0/3513947993 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800e070 con 0x7f2a5009e440 2023-09-30T22:21:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7f2a4dffb700 1 -- 172.21.15.89:0/3513947993 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a48003a20 con 0x7f2a5009e440 2023-09-30T22:21:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7f2a4dffb700 1 -- 172.21.15.89:0/3513947993 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48003dd0 con 0x7f2a5009e440 2023-09-30T22:21:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabb94ce700 1 -- 172.21.15.89:0/4028243749 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0011830 con 0x7fabb409d6f0 2023-09-30T22:21:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabb94ce700 1 -- 172.21.15.89:0/4028243749 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb0011990 con 0x7fabb409d6f0 2023-09-30T22:21:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.662+0000 7fabb94ce700 1 -- 172.21.15.89:0/4028243749 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0011d20 con 0x7fabb409d6f0 2023-09-30T22:21:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7fabbc734700 1 -- 172.21.15.89:0/4028243749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409d6f0 msgr2=0x7fabb409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7fabbc734700 1 --2- 172.21.15.89:0/4028243749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409d6f0 0x7fabb409dac0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009920 tx=0x7fabb0009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7f2a55873700 1 -- 172.21.15.89:0/3513947993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e440 msgr2=0x7f2a50134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7f2a55873700 1 --2- 172.21.15.89:0/3513947993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e440 0x7f2a50134100 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009e80 tx=0x7f2a4800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7f2a55873700 1 -- 172.21.15.89:0/3513947993 shutdown_connections 2023-09-30T22:21:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7f2a55873700 1 --2- 172.21.15.89:0/3513947993 >> v2:172.21.15.89:3300/0 conn(0x7f2a5009d6f0 0x7f2a5009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7f2a55873700 1 --2- 172.21.15.89:0/3513947993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e440 0x7f2a50134100 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7f2a55873700 1 -- 172.21.15.89:0/3513947993 >> 172.21.15.89:0/3513947993 conn(0x7f2a50066f50 msgr2=0x7f2a5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7f2a55873700 1 -- 172.21.15.89:0/3513947993 shutdown_connections 2023-09-30T22:21:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7fabbc734700 1 -- 172.21.15.89:0/4028243749 shutdown_connections 2023-09-30T22:21:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7fabbc734700 1 --2- 172.21.15.89:0/4028243749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409d6f0 0x7fabb409dac0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7fabbc734700 1 --2- 172.21.15.89:0/4028243749 >> v2:172.21.15.89:3300/0 conn(0x7fabb409e000 0x7fabb41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7fabbc734700 1 -- 172.21.15.89:0/4028243749 >> 172.21.15.89:0/4028243749 conn(0x7fabb4066f50 msgr2=0x7fabb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7f2a55873700 1 -- 172.21.15.89:0/3513947993 wait complete. 2023-09-30T22:21:42.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7fabbc734700 1 -- 172.21.15.89:0/4028243749 shutdown_connections 2023-09-30T22:21:42.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.663+0000 7fabbc734700 1 -- 172.21.15.89:0/4028243749 wait complete. 2023-09-30T22:21:42.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.664+0000 7f2a55873700 1 Processor -- start 2023-09-30T22:21:42.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.664+0000 7fabbc734700 1 Processor -- start 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.664+0000 7fabbc734700 1 -- start start 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabbc734700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb409d6f0 0x7fabb41a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabbc734700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409e000 0x7fabb41a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.664+0000 7f2a55873700 1 -- start start 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a55873700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a5009d6f0 0x7f2a501196f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a55873700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e000 0x7f2a50119c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a55873700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a5009e440 0x7f2a5011a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a55873700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5011bf30 con 0x7f2a5009d6f0 2023-09-30T22:21:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a55873700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500a1f20 con 0x7f2a5009e440 2023-09-30T22:21:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a55873700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5011cc50 con 0x7f2a5009e000 2023-09-30T22:21:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabbc734700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb409e410 0x7fabb41a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabbc734700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41a89d0 con 0x7fabb409e410 2023-09-30T22:21:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabbc734700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41a7d80 con 0x7fabb409d6f0 2023-09-30T22:21:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabbc734700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41a6360 con 0x7fabb409e000 2023-09-30T22:21:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabba4d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb409d6f0 0x7fabb41a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabba4d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb409d6f0 0x7fabb41a4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46100/0 (socket says 172.21.15.89:46100) 2023-09-30T22:21:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabba4d0700 1 -- 172.21.15.89:0/1093485195 learned_addr learned my addr 172.21.15.89:0/1093485195 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabb9ccf700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409e000 0x7fabb41a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabbacd1700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb409e410 0x7fabb41a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a4e7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e000 0x7f2a50119c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a4e7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e000 0x7f2a50119c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57668/0 (socket says 172.21.15.89:57668) 2023-09-30T22:21:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a4e7fc700 1 -- 172.21.15.89:0/1871107105 learned_addr learned my addr 172.21.15.89:0/1871107105 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a4effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a5009d6f0 0x7f2a501196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7f2a4f7fe700 1 --2- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a5009e440 0x7f2a5011a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabb9ccf700 1 -- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb409d6f0 msgr2=0x7fabb41a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabb9ccf700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb409d6f0 0x7fabb41a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabb9ccf700 1 -- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb409e410 msgr2=0x7fabb41a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.665+0000 7fabb9ccf700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb409e410 0x7fabb41a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7fabb9ccf700 1 -- 172.21.15.89:0/1093485195 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0009550 con 0x7fabb409e000 2023-09-30T22:21:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a4e7fc700 1 -- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a5009e440 msgr2=0x7f2a5011a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a4e7fc700 1 --2- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a5009e440 0x7f2a5011a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a4e7fc700 1 -- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a5009d6f0 msgr2=0x7f2a501196f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a4e7fc700 1 --2- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a5009d6f0 0x7f2a501196f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a4e7fc700 1 -- 172.21.15.89:0/1871107105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48009b30 con 0x7f2a5009e000 2023-09-30T22:21:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7fabb9ccf700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409e000 0x7fabb41a5480 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faba400e8d0 tx=0x7faba400ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a4e7fc700 1 --2- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e000 0x7f2a50119c30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400c4a0 tx=0x7f2a4400c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7fabba4d0700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb409d6f0 0x7fabb41a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7fabaaffd700 1 -- 172.21.15.89:0/1093485195 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4013680 con 0x7fabb409e000 2023-09-30T22:21:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a37fff700 1 -- 172.21.15.89:0/1871107105 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440054e0 con 0x7f2a5009e000 2023-09-30T22:21:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5009c710 con 0x7f2a5009e000 2023-09-30T22:21:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb41a6b60 con 0x7fabb409e000 2023-09-30T22:21:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7fabaaffd700 1 -- 172.21.15.89:0/1093485195 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba40137e0 con 0x7fabb409e000 2023-09-30T22:21:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7fabaaffd700 1 -- 172.21.15.89:0/1093485195 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4013b50 con 0x7fabb409e000 2023-09-30T22:21:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb41a78c0 con 0x7fabb409e000 2023-09-30T22:21:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.666+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5011b8c0 con 0x7f2a5009e000 2023-09-30T22:21:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.667+0000 7f2a37fff700 1 -- 172.21.15.89:0/1871107105 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a44003680 con 0x7f2a5009e000 2023-09-30T22:21:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.667+0000 7f2a37fff700 1 -- 172.21.15.89:0/1871107105 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44003a30 con 0x7f2a5009e000 2023-09-30T22:21:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.668+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb4074ea0 con 0x7fabb409e000 2023-09-30T22:21:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.668+0000 7fabaaffd700 1 -- 172.21.15.89:0/1093485195 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faba4013cb0 con 0x7fabb409e000 2023-09-30T22:21:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.668+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a50074ea0 con 0x7f2a5009e000 2023-09-30T22:21:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.668+0000 7fabaaffd700 1 -- 172.21.15.89:0/1093485195 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faba403f5f0 con 0x7fabb409e000 2023-09-30T22:21:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.668+0000 7f2a37fff700 1 -- 172.21.15.89:0/1871107105 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2a44003b90 con 0x7f2a5009e000 2023-09-30T22:21:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.668+0000 7f2a37fff700 1 -- 172.21.15.89:0/1871107105 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2a4403eb80 con 0x7f2a5009e000 2023-09-30T22:21:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.673+0000 7fabaaffd700 1 -- 172.21.15.89:0/1093485195 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faba401bc30 con 0x7fabb409e000 2023-09-30T22:21:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.677+0000 7f2a37fff700 1 -- 172.21.15.89:0/1871107105 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2a44012400 con 0x7f2a5009e000 2023-09-30T22:21:42.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.793+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f86201a6d20 con 0x7f862009e410 2023-09-30T22:21:42.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.793+0000 7f861cff9700 1 -- 172.21.15.89:0/157950025 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8618037030 con 0x7f862009e410 2023-09-30T22:21:42.793 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:42.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e410 msgr2=0x7f86201c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 --2- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e410 0x7f86201c8ad0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f861800c730 tx=0x7f861800ca40 comp rx=0 tx=0).stop 2023-09-30T22:21:42.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 shutdown_connections 2023-09-30T22:21:42.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 --2- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f862009e410 0x7f86201c8ad0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 --2- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f862009d6f0 0x7f86201c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 --2- 172.21.15.89:0/157950025 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f862009e000 0x7f86201c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 >> 172.21.15.89:0/157950025 conn(0x7f8620066f50 msgr2=0x7f86201bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 shutdown_connections 2023-09-30T22:21:42.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.795+0000 7f862670d700 1 -- 172.21.15.89:0/157950025 wait complete. 2023-09-30T22:21:42.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.838+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0af006c360 con 0x7f0af009e000 2023-09-30T22:21:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.839+0000 7f0aed7fa700 1 -- 172.21.15.89:0/2188078787 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0adc0118e0 con 0x7f0af009e000 2023-09-30T22:21:42.838 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.840+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e000 msgr2=0x7f0af01a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.840+0000 7f0af721d700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e000 0x7f0af01a6380 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00e810 tx=0x7f0adc00eb20 comp rx=0 tx=0).stop 2023-09-30T22:21:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.841+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 shutdown_connections 2023-09-30T22:21:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.841+0000 7f0af721d700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0af009e410 0x7f0af01a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.841+0000 7f0af721d700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0af009d6f0 0x7f0af01a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.841+0000 7f0af721d700 1 --2- 172.21.15.89:0/2188078787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0af009e000 0x7f0af01a6380 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.841+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 >> 172.21.15.89:0/2188078787 conn(0x7f0af0066f50 msgr2=0x7f0af01bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.841+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 shutdown_connections 2023-09-30T22:21:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.841+0000 7f0af721d700 1 -- 172.21.15.89:0/2188078787 wait complete. 2023-09-30T22:21:42.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:42.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.868+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fabb406c360 con 0x7fabb409e000 2023-09-30T22:21:42.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.869+0000 7fabaaffd700 1 -- 172.21.15.89:0/1093485195 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faba4021020 con 0x7fabb409e000 2023-09-30T22:21:42.869 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:42.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2a5009f420 con 0x7f2a5009e000 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409e000 msgr2=0x7fabb41a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409e000 0x7fabb41a5480 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faba400e8d0 tx=0x7faba400ebe0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 shutdown_connections 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabb409e410 0x7fabb41a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabb409d6f0 0x7fabb41a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 --2- 172.21.15.89:0/1093485195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabb409e000 0x7fabb41a5480 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 >> 172.21.15.89:0/1093485195 conn(0x7fabb4066f50 msgr2=0x7fabb41bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 shutdown_connections 2023-09-30T22:21:42.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7fabbc734700 1 -- 172.21.15.89:0/1093485195 wait complete. 2023-09-30T22:21:42.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.871+0000 7f2a37fff700 1 -- 172.21.15.89:0/1871107105 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a44017020 con 0x7f2a5009e000 2023-09-30T22:21:42.872 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:42.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e000 msgr2=0x7f2a50119c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 --2- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e000 0x7f2a50119c30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400c4a0 tx=0x7f2a4400c7b0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 shutdown_connections 2023-09-30T22:21:42.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 --2- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a5009e440 0x7f2a5011a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 --2- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a5009d6f0 0x7f2a501196f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 --2- 172.21.15.89:0/1871107105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a5009e000 0x7f2a50119c30 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:42.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 >> 172.21.15.89:0/1871107105 conn(0x7f2a50066f50 msgr2=0x7f2a501bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 shutdown_connections 2023-09-30T22:21:42.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:42.873+0000 7f2a55873700 1 -- 172.21.15.89:0/1871107105 wait complete. 2023-09-30T22:21:42.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:42.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:43.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:43.853 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:43.885 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:43.886 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.963+0000 7fcff6468700 1 Processor -- start 2023-09-30T22:21:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.963+0000 7fcff6468700 1 -- start start 2023-09-30T22:21:43.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff6468700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcff0133540 0x7fcff0135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:43.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff6468700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0135e60 0x7fcff0138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:43.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff6468700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff00716a0 con 0x7fcff0130540 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff6468700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff0071290 con 0x7fcff0133540 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff6468700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0130340 con 0x7fcff0135e60 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcfef7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcff0133540 0x7fcff0135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff4a05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0135e60 0x7fcff0138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcfeffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcff0130540 0x7fcff0130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38144/0 (socket says 172.21.15.89:38144) 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcfeffff700 1 -- 172.21.15.89:0/607864784 learned_addr learned my addr 172.21.15.89:0/607864784 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff4a05700 1 -- 172.21.15.89:0/607864784 >> v2:172.21.15.89:3300/0 conn(0x7fcff0133540 msgr2=0x7fcff0135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff4a05700 1 --2- 172.21.15.89:0/607864784 >> v2:172.21.15.89:3300/0 conn(0x7fcff0133540 0x7fcff0135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff4a05700 1 -- 172.21.15.89:0/607864784 >> v1:172.21.15.89:6789/0 conn(0x7fcff0130540 legacy=0x7fcff0130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff4a05700 1 -- 172.21.15.89:0/607864784 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff0070b80 con 0x7fcff0135e60 2023-09-30T22:21:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcfef7fe700 1 --2- 172.21.15.89:0/607864784 >> v2:172.21.15.89:3300/0 conn(0x7fcff0133540 0x7fcff0135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:43.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.964+0000 7fcff4a05700 1 --2- 172.21.15.89:0/607864784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0135e60 0x7fcff0138240 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009d90 tx=0x7fcfe800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ed8c24f364d87e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.965+0000 7fcfeeffd700 1 -- 172.21.15.89:0/607864784 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800e070 con 0x7fcff0135e60 2023-09-30T22:21:43.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.965+0000 7fcfeeffd700 1 -- 172.21.15.89:0/607864784 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8003a20 con 0x7fcff0135e60 2023-09-30T22:21:43.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.965+0000 7fcfeeffd700 1 -- 172.21.15.89:0/607864784 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8003dd0 con 0x7fcff0135e60 2023-09-30T22:21:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.966+0000 7fcff6468700 1 -- 172.21.15.89:0/607864784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0135e60 msgr2=0x7fcff0138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.966+0000 7fcff6468700 1 --2- 172.21.15.89:0/607864784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0135e60 0x7fcff0138240 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009d90 tx=0x7fcfe800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.966+0000 7fcff6468700 1 -- 172.21.15.89:0/607864784 shutdown_connections 2023-09-30T22:21:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.966+0000 7fcff6468700 1 --2- 172.21.15.89:0/607864784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0135e60 0x7fcff0138240 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.966+0000 7fcff6468700 1 --2- 172.21.15.89:0/607864784 >> v2:172.21.15.89:3300/0 conn(0x7fcff0133540 0x7fcff0135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.966+0000 7fcff6468700 1 -- 172.21.15.89:0/607864784 >> 172.21.15.89:0/607864784 conn(0x7fcff00a2aa0 msgr2=0x7fcff00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.966+0000 7fcff6468700 1 -- 172.21.15.89:0/607864784 shutdown_connections 2023-09-30T22:21:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.966+0000 7fcff6468700 1 -- 172.21.15.89:0/607864784 wait complete. 2023-09-30T22:21:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.967+0000 7fcff6468700 1 Processor -- start 2023-09-30T22:21:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.967+0000 7fcff6468700 1 -- start start 2023-09-30T22:21:43.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcff6468700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0130540 0x7fcff012d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:43.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcff6468700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcff0133540 0x7fcff012d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcff6468700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcff0135e60 0x7fcff0139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcff6468700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff009a830 con 0x7fcff0135e60 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcff6468700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff009aa10 con 0x7fcff0130540 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcff6468700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0070610 con 0x7fcff0133540 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfeffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0130540 0x7fcff012d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfeffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0130540 0x7fcff012d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46128/0 (socket says 172.21.15.89:46128) 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfeffff700 1 -- 172.21.15.89:0/927073732 learned_addr learned my addr 172.21.15.89:0/927073732 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcff4a05700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcff0135e60 0x7fcff0139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfef7fe700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcff0133540 0x7fcff012d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfeffff700 1 -- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcff0133540 msgr2=0x7fcff012d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfeffff700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcff0133540 0x7fcff012d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfeffff700 1 -- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcff0135e60 msgr2=0x7fcff0139790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfeffff700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcff0135e60 0x7fcff0139790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.968+0000 7fcfeffff700 1 -- 172.21.15.89:0/927073732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe40094a0 con 0x7fcff0130540 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.969+0000 7fcff4a05700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcff0135e60 0x7fcff0139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.969+0000 7fcfeffff700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0130540 0x7fcff012d1e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c780 tx=0x7fcfe000ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.969+0000 7fcfecff9700 1 -- 172.21.15.89:0/927073732 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0004c80 con 0x7fcff0130540 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.969+0000 7fcfecff9700 1 -- 172.21.15.89:0/927073732 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe0004de0 con 0x7fcff0130540 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.969+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe8009a70 con 0x7fcff0130540 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.969+0000 7fcfecff9700 1 -- 172.21.15.89:0/927073732 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0007310 con 0x7fcff0130540 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.969+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff013a8f0 con 0x7fcff0130540 2023-09-30T22:21:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.970+0000 7fcfecff9700 1 -- 172.21.15.89:0/927073732 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfe0007470 con 0x7fcff0130540 2023-09-30T22:21:43.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.970+0000 7fcfecff9700 1 -- 172.21.15.89:0/927073732 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcfe003f330 con 0x7fcff0130540 2023-09-30T22:21:43.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.970+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff009a670 con 0x7fcff0130540 2023-09-30T22:21:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.979+0000 7fcfecff9700 1 -- 172.21.15.89:0/927073732 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcff009a670 con 0x7fcff0130540 2023-09-30T22:21:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.994+0000 7fad62ade700 1 Processor -- start 2023-09-30T22:21:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.994+0000 7fad62ade700 1 -- start start 2023-09-30T22:21:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad62ade700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 0x7fad5c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad62ade700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad5c09e000 0x7fad5c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad62ade700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c09b120 con 0x7fad5c09e410 2023-09-30T22:21:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad62ade700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c06b460 con 0x7fad5c09e000 2023-09-30T22:21:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad62ade700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c09a7e0 con 0x7fad5c09d6f0 2023-09-30T22:21:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad5bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad5c09e000 0x7fad5c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad5bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fad5c09e000 0x7fad5c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46302/0 (socket says 172.21.15.89:46302) 2023-09-30T22:21:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad5bfff700 1 -- 172.21.15.89:0/3838272270 learned_addr learned my addr 172.21.15.89:0/3838272270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.995+0000 7fad6087a700 1 --2- 172.21.15.89:0/3838272270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 0x7fad5c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.996+0000 7fad5bfff700 1 -- 172.21.15.89:0/3838272270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 msgr2=0x7fad5c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.996+0000 7fad5bfff700 1 --2- 172.21.15.89:0/3838272270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 0x7fad5c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.996+0000 7fad5bfff700 1 -- 172.21.15.89:0/3838272270 >> v1:172.21.15.89:6789/0 conn(0x7fad5c09e410 legacy=0x7fad5c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.996+0000 7fad5bfff700 1 -- 172.21.15.89:0/3838272270 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c09a500 con 0x7fad5c09e000 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.996+0000 7fad5bfff700 1 --2- 172.21.15.89:0/3838272270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c1339f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fad480098f0 tx=0x7fad48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88af1d991ef93a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.996+0000 7fad5b7fe700 1 -- 172.21.15.89:0/3838272270 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48011830 con 0x7fad5c09e000 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.996+0000 7fad5b7fe700 1 -- 172.21.15.89:0/3838272270 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad48011990 con 0x7fad5c09e000 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.996+0000 7fad5b7fe700 1 -- 172.21.15.89:0/3838272270 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48011d20 con 0x7fad5c09e000 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.997+0000 7fad62ade700 1 -- 172.21.15.89:0/3838272270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 msgr2=0x7fad5c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.997+0000 7fad62ade700 1 --2- 172.21.15.89:0/3838272270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c1339f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fad480098f0 tx=0x7fad48009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.997+0000 7fad62ade700 1 -- 172.21.15.89:0/3838272270 shutdown_connections 2023-09-30T22:21:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.997+0000 7fad62ade700 1 --2- 172.21.15.89:0/3838272270 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 0x7fad5c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.997+0000 7fad62ade700 1 --2- 172.21.15.89:0/3838272270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c1339f0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.997+0000 7fad62ade700 1 -- 172.21.15.89:0/3838272270 >> 172.21.15.89:0/3838272270 conn(0x7fad5c066f50 msgr2=0x7fad5c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.997+0000 7fad62ade700 1 -- 172.21.15.89:0/3838272270 shutdown_connections 2023-09-30T22:21:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.997+0000 7fad62ade700 1 -- 172.21.15.89:0/3838272270 wait complete. 2023-09-30T22:21:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.998+0000 7fad62ade700 1 Processor -- start 2023-09-30T22:21:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.998+0000 7fad62ade700 1 -- start start 2023-09-30T22:21:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad62ade700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 0x7fad5c127350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad62ade700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c127890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad62ade700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5c09e410 0x7fad5c127dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad62ade700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0a2b90 con 0x7fad5c09e000 2023-09-30T22:21:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad62ade700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c12a8f0 con 0x7fad5c09d6f0 2023-09-30T22:21:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad62ade700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c133fc0 con 0x7fad5c09e410 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46316/0 (socket says 172.21.15.89:46316) 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad5bfff700 1 -- 172.21.15.89:0/431148136 learned_addr learned my addr 172.21.15.89:0/431148136 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad6087a700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 0x7fad5c127350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:43.999+0000 7fad6107b700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5c09e410 0x7fad5c127dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad5bfff700 1 -- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5c09e410 msgr2=0x7fad5c127dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad5bfff700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5c09e410 0x7fad5c127dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad5bfff700 1 -- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 msgr2=0x7fad5c127350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad5bfff700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 0x7fad5c127350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad5bfff700 1 -- 172.21.15.89:0/431148136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad48009550 con 0x7fad5c09e000 2023-09-30T22:21:44.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad5bfff700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c127890 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fad48013000 tx=0x7fad48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad597fa700 1 -- 172.21.15.89:0/431148136 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48004450 con 0x7fad5c09e000 2023-09-30T22:21:44.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad597fa700 1 -- 172.21.15.89:0/431148136 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad480045b0 con 0x7fad5c09e000 2023-09-30T22:21:44.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.000+0000 7fad597fa700 1 -- 172.21.15.89:0/431148136 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480038c0 con 0x7fad5c09e000 2023-09-30T22:21:44.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.001+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c129270 con 0x7fad5c09e000 2023-09-30T22:21:44.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.001+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c1294b0 con 0x7fad5c09e000 2023-09-30T22:21:44.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.001+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c12a730 con 0x7fad5c09e000 2023-09-30T22:21:44.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.003+0000 7fad597fa700 1 -- 172.21.15.89:0/431148136 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad48003a20 con 0x7fad5c09e000 2023-09-30T22:21:44.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.004+0000 7fad597fa700 1 -- 172.21.15.89:0/431148136 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fad480410e0 con 0x7fad5c09e000 2023-09-30T22:21:44.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.007+0000 7fad597fa700 1 -- 172.21.15.89:0/431148136 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad4801a420 con 0x7fad5c09e000 2023-09-30T22:21:44.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f0866b5f700 1 Processor -- start 2023-09-30T22:21:44.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f0866b5f700 1 -- start start 2023-09-30T22:21:44.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f0866b5f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009e930 0x7f086009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f0866b5f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009f650 0x7f0860131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f0866b5f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0860070940 con 0x7f086009f240 2023-09-30T22:21:44.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f0866b5f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086006fd60 con 0x7f086009e930 2023-09-30T22:21:44.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f0866b5f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f086006fa80 con 0x7f086009f650 2023-09-30T22:21:44.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f08650fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009f650 0x7f0860131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f08650fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009f650 0x7f0860131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57690/0 (socket says 172.21.15.89:57690) 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f08650fc700 1 -- 172.21.15.89:0/3438837765 learned_addr learned my addr 172.21.15.89:0/3438837765 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.035+0000 7f085ffff700 1 --1- 172.21.15.89:0/3438837765 >> v1:172.21.15.89:6789/0 conn(0x7f086009f240 0x7f08601317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38156/0 (socket says 172.21.15.89:38156) 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f085f7fe700 1 -- 172.21.15.89:0/3438837765 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203519428 0 0) 0x7f0860070940 con 0x7f086009f240 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f08650fc700 1 -- 172.21.15.89:0/3438837765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009e930 msgr2=0x7f086009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f08650fc700 1 --2- 172.21.15.89:0/3438837765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009e930 0x7f086009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f08650fc700 1 -- 172.21.15.89:0/3438837765 >> v1:172.21.15.89:6789/0 conn(0x7f086009f240 legacy=0x7f08601317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f08650fc700 1 -- 172.21.15.89:0/3438837765 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086009fb60 con 0x7f086009f650 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f08650fc700 1 --2- 172.21.15.89:0/3438837765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009f650 0x7f0860131f00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0858009e80 tx=0x7f085800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57dd8167bf4414a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f085f7fe700 1 -- 172.21.15.89:0/3438837765 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085800e070 con 0x7f086009f650 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f085f7fe700 1 -- 172.21.15.89:0/3438837765 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085800b9c0 con 0x7f086009f650 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.036+0000 7f085f7fe700 1 -- 172.21.15.89:0/3438837765 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085800bd70 con 0x7f086009f650 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.037+0000 7f0866b5f700 1 -- 172.21.15.89:0/3438837765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009f650 msgr2=0x7f0860131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.037+0000 7f0866b5f700 1 --2- 172.21.15.89:0/3438837765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009f650 0x7f0860131f00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0858009e80 tx=0x7f085800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.037+0000 7f0866b5f700 1 -- 172.21.15.89:0/3438837765 shutdown_connections 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.037+0000 7f0866b5f700 1 --2- 172.21.15.89:0/3438837765 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009e930 0x7f086009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.037+0000 7f0866b5f700 1 --2- 172.21.15.89:0/3438837765 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009f650 0x7f0860131f00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.037+0000 7f0866b5f700 1 -- 172.21.15.89:0/3438837765 >> 172.21.15.89:0/3438837765 conn(0x7f08600a4210 msgr2=0x7f08600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.037+0000 7f0866b5f700 1 -- 172.21.15.89:0/3438837765 shutdown_connections 2023-09-30T22:21:44.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.037+0000 7f0866b5f700 1 -- 172.21.15.89:0/3438837765 wait complete. 2023-09-30T22:21:44.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.038+0000 7f0866b5f700 1 Processor -- start 2023-09-30T22:21:44.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.038+0000 7f0866b5f700 1 -- start start 2023-09-30T22:21:44.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f0866b5f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009e930 0x7f086006c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f0866b5f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009f240 0x7f086006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f0866b5f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f086009f650 0x7f08601c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f0866b5f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08601a8080 con 0x7f086009f650 2023-09-30T22:21:44.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f0866b5f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08601b71b0 con 0x7f086009f240 2023-09-30T22:21:44.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f0866b5f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f086006bd90 con 0x7f086009e930 2023-09-30T22:21:44.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f08650fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f086009f650 0x7f08601c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f08650fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f086009f650 0x7f08601c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46320/0 (socket says 172.21.15.89:46320) 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f08650fc700 1 -- 172.21.15.89:0/2812318400 learned_addr learned my addr 172.21.15.89:0/2812318400 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f08648fb700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009e930 0x7f086006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.039+0000 7f085ffff700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009f240 0x7f086006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f08650fc700 1 -- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009e930 msgr2=0x7f086006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f08650fc700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009e930 0x7f086006c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f08650fc700 1 -- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009f240 msgr2=0x7f086006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f08650fc700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009f240 0x7f086006c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f08650fc700 1 -- 172.21.15.89:0/2812318400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858009b30 con 0x7f086009f650 2023-09-30T22:21:44.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f085ffff700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009f240 0x7f086006c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:44.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f08650fc700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f086009f650 0x7f08601c8050 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f086006fbc0 tx=0x7f0858004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f085d7fa700 1 -- 172.21.15.89:0/2812318400 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858004450 con 0x7f086009f650 2023-09-30T22:21:44.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08600a0190 con 0x7f086009f650 2023-09-30T22:21:44.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f085d7fa700 1 -- 172.21.15.89:0/2812318400 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08580045b0 con 0x7f086009f650 2023-09-30T22:21:44.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f085d7fa700 1 -- 172.21.15.89:0/2812318400 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858004940 con 0x7f086009f650 2023-09-30T22:21:44.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.040+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0860071d20 con 0x7f086009f650 2023-09-30T22:21:44.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.041+0000 7f085d7fa700 1 -- 172.21.15.89:0/2812318400 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0858025460 con 0x7f086009f650 2023-09-30T22:21:44.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.041+0000 7f085d7fa700 1 -- 172.21.15.89:0/2812318400 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0858041470 con 0x7f086009f650 2023-09-30T22:21:44.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.041+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0860074ea0 con 0x7f086009f650 2023-09-30T22:21:44.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.043+0000 7f77fbe0f700 1 Processor -- start 2023-09-30T22:21:44.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.043+0000 7f77fbe0f700 1 -- start start 2023-09-30T22:21:44.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.043+0000 7f77fbe0f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f412cf60 0x7f77f406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77fbe0f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f77f4130430 0x7f77f4066410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77fbe0f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f412cd90 con 0x7f77f412f8b0 2023-09-30T22:21:44.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77fbe0f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f412d330 con 0x7f77f4130430 2023-09-30T22:21:44.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77fbe0f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4071780 con 0x7f77f412cf60 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f9bab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f412cf60 0x7f77f406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f9bab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f412cf60 0x7f77f406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57700/0 (socket says 172.21.15.89:57700) 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f9bab700 1 -- 172.21.15.89:0/950385721 learned_addr learned my addr 172.21.15.89:0/950385721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77fa3ac700 1 --2- 172.21.15.89:0/950385721 >> v2:172.21.15.89:3300/0 conn(0x7f77f4130430 0x7f77f4066410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f9bab700 1 -- 172.21.15.89:0/950385721 >> v2:172.21.15.89:3300/0 conn(0x7f77f4130430 msgr2=0x7f77f4066410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f9bab700 1 --2- 172.21.15.89:0/950385721 >> v2:172.21.15.89:3300/0 conn(0x7f77f4130430 0x7f77f4066410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f9bab700 1 -- 172.21.15.89:0/950385721 >> v1:172.21.15.89:6789/0 conn(0x7f77f412f8b0 legacy=0x7f77f412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f9bab700 1 -- 172.21.15.89:0/950385721 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4071a40 con 0x7f77f412cf60 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f9bab700 1 --2- 172.21.15.89:0/950385721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f412cf60 0x7f77f406f5d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098f0 tx=0x7f77f0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863fee06854a76a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f8ba9700 1 -- 172.21.15.89:0/950385721 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0011830 con 0x7f77f412cf60 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f8ba9700 1 -- 172.21.15.89:0/950385721 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f0011990 con 0x7f77f412cf60 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.044+0000 7f77f8ba9700 1 -- 172.21.15.89:0/950385721 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0011d20 con 0x7f77f412cf60 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.045+0000 7f77fbe0f700 1 -- 172.21.15.89:0/950385721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f412cf60 msgr2=0x7f77f406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.045+0000 7f77fbe0f700 1 --2- 172.21.15.89:0/950385721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f412cf60 0x7f77f406f5d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098f0 tx=0x7f77f0009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.045+0000 7f77fbe0f700 1 -- 172.21.15.89:0/950385721 shutdown_connections 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.045+0000 7f77fbe0f700 1 --2- 172.21.15.89:0/950385721 >> v2:172.21.15.89:3300/0 conn(0x7f77f4130430 0x7f77f4066410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.045+0000 7f77fbe0f700 1 --2- 172.21.15.89:0/950385721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f412cf60 0x7f77f406f5d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.045+0000 7f77fbe0f700 1 -- 172.21.15.89:0/950385721 >> 172.21.15.89:0/950385721 conn(0x7f77f40a2aa0 msgr2=0x7f77f40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.045+0000 7f77fbe0f700 1 -- 172.21.15.89:0/950385721 shutdown_connections 2023-09-30T22:21:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.045+0000 7f77fbe0f700 1 -- 172.21.15.89:0/950385721 wait complete. 2023-09-30T22:21:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.047+0000 7f77fbe0f700 1 Processor -- start 2023-09-30T22:21:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.047+0000 7f77fbe0f700 1 -- start start 2023-09-30T22:21:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.047+0000 7f77fbe0f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f412cf60 0x7f77f41c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.047+0000 7f77fbe0f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f412f8b0 0x7f77f41c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.047+0000 7f77fbe0f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f4130430 0x7f77f41c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.047+0000 7f77fbe0f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4124120 con 0x7f77f412f8b0 2023-09-30T22:21:44.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.047+0000 7f77fbe0f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4125ab0 con 0x7f77f412cf60 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.047+0000 7f77fbe0f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41c3d60 con 0x7f77f4130430 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f412cf60 0x7f77f41c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f412cf60 0x7f77f41c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46172/0 (socket says 172.21.15.89:46172) 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 -- 172.21.15.89:0/220964844 learned_addr learned my addr 172.21.15.89:0/220964844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77fa3ac700 1 --2- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f4130430 0x7f77f41c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 -- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f4130430 msgr2=0x7f77f41c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 --2- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f4130430 0x7f77f41c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 -- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f412f8b0 msgr2=0x7f77f41c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 --2- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f412f8b0 0x7f77f41c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 -- 172.21.15.89:0/220964844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0009550 con 0x7f77f412cf60 2023-09-30T22:21:44.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77f9bab700 1 --2- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f412cf60 0x7f77f41c32e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009c40 tx=0x7f77f0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77ea7fc700 1 -- 172.21.15.89:0/220964844 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f00036a0 con 0x7f77f412cf60 2023-09-30T22:21:44.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77ea7fc700 1 -- 172.21.15.89:0/220964844 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f0003800 con 0x7f77f412cf60 2023-09-30T22:21:44.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f4129fa0 con 0x7f77f412cf60 2023-09-30T22:21:44.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77ea7fc700 1 -- 172.21.15.89:0/220964844 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0003b90 con 0x7f77f412cf60 2023-09-30T22:21:44.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.048+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f4126020 con 0x7f77f412cf60 2023-09-30T22:21:44.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.049+0000 7f77ea7fc700 1 -- 172.21.15.89:0/220964844 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f77f0021070 con 0x7f77f412cf60 2023-09-30T22:21:44.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.049+0000 7f77ea7fc700 1 -- 172.21.15.89:0/220964844 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f77f0040ac0 con 0x7f77f412cf60 2023-09-30T22:21:44.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.050+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f4074ea0 con 0x7f77f412cf60 2023-09-30T22:21:44.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.050+0000 7f085d7fa700 1 -- 172.21.15.89:0/2812318400 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f085802a070 con 0x7f086009f650 2023-09-30T22:21:44.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.058+0000 7f77ea7fc700 1 -- 172.21.15.89:0/220964844 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f77f0015080 con 0x7f77f412cf60 2023-09-30T22:21:44.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.179+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcff009a670 con 0x7fcff0130540 2023-09-30T22:21:44.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.179+0000 7fcfecff9700 1 -- 172.21.15.89:0/927073732 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcff009a670 con 0x7fcff0130540 2023-09-30T22:21:44.179 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0130540 msgr2=0x7fcff012d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0130540 0x7fcff012d1e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c780 tx=0x7fcfe000ca90 comp rx=0 tx=0).stop 2023-09-30T22:21:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 shutdown_connections 2023-09-30T22:21:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcff0135e60 0x7fcff0139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcff0130540 0x7fcff012d1e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 --2- 172.21.15.89:0/927073732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcff0133540 0x7fcff012d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 >> 172.21.15.89:0/927073732 conn(0x7fcff00a2aa0 msgr2=0x7fcff006e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 shutdown_connections 2023-09-30T22:21:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.181+0000 7fcff6468700 1 -- 172.21.15.89:0/927073732 wait complete. 2023-09-30T22:21:44.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:44.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.212+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fad5c074ea0 con 0x7fad5c09e000 2023-09-30T22:21:44.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.212+0000 7fad597fa700 1 -- 172.21.15.89:0/431148136 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad48033030 con 0x7fad5c09e000 2023-09-30T22:21:44.212 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:44.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 msgr2=0x7fad5c127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c127890 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fad48013000 tx=0x7fad48004750 comp rx=0 tx=0).stop 2023-09-30T22:21:44.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 shutdown_connections 2023-09-30T22:21:44.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad5c09e410 0x7fad5c127dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad5c09d6f0 0x7fad5c127350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 --2- 172.21.15.89:0/431148136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad5c09e000 0x7fad5c127890 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 >> 172.21.15.89:0/431148136 conn(0x7fad5c066f50 msgr2=0x7fad5c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 shutdown_connections 2023-09-30T22:21:44.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.215+0000 7fad62ade700 1 -- 172.21.15.89:0/431148136 wait complete. 2023-09-30T22:21:44.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.254+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f77f40713a0 con 0x7f77f412cf60 2023-09-30T22:21:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.254+0000 7f77ea7fc700 1 -- 172.21.15.89:0/220964844 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f77f001b950 con 0x7f77f412cf60 2023-09-30T22:21:44.254 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.256+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f412cf60 msgr2=0x7f77f41c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.256+0000 7f77fbe0f700 1 --2- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f412cf60 0x7f77f41c32e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009c40 tx=0x7f77f0004750 comp rx=0 tx=0).stop 2023-09-30T22:21:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.256+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 shutdown_connections 2023-09-30T22:21:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.256+0000 7f77fbe0f700 1 --2- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f77f4130430 0x7f77f41c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.256+0000 7f77fbe0f700 1 --2- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77f412cf60 0x7f77f41c32e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.256+0000 7f77fbe0f700 1 --2- 172.21.15.89:0/220964844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77f412f8b0 0x7f77f41c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.256+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 >> 172.21.15.89:0/220964844 conn(0x7f77f40a2aa0 msgr2=0x7f77f406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.257+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 shutdown_connections 2023-09-30T22:21:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.257+0000 7f77fbe0f700 1 -- 172.21.15.89:0/220964844 wait complete. 2023-09-30T22:21:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.258+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f08601a81c0 con 0x7f086009f650 2023-09-30T22:21:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.258+0000 7f085d7fa700 1 -- 172.21.15.89:0/2812318400 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08601a81c0 con 0x7f086009f650 2023-09-30T22:21:44.258 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:44.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.260+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f086009f650 msgr2=0x7f08601c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.260+0000 7f0866b5f700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f086009f650 0x7f08601c8050 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f086006fbc0 tx=0x7f0858004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.260+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 shutdown_connections 2023-09-30T22:21:44.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.260+0000 7f0866b5f700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f086009f650 0x7f08601c8050 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.260+0000 7f0866b5f700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f086009e930 0x7f086006c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.260+0000 7f0866b5f700 1 --2- 172.21.15.89:0/2812318400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f086009f240 0x7f086006c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:44.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.260+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 >> 172.21.15.89:0/2812318400 conn(0x7f08600a4210 msgr2=0x7f086009bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.261+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 shutdown_connections 2023-09-30T22:21:44.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:44.261+0000 7f0866b5f700 1 -- 172.21.15.89:0/2812318400 wait complete. 2023-09-30T22:21:44.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:44.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:45.194 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:45.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:45.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:45.274 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.347+0000 7fd3d6dae700 1 Processor -- start 2023-09-30T22:21:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.347+0000 7fd3d6dae700 1 -- start start 2023-09-30T22:21:45.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d6dae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d6dae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d6dae700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d01313c0 con 0x7fd3d0099a40 2023-09-30T22:21:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d6dae700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d006e8c0 con 0x7fd3d0131810 2023-09-30T22:21:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d6dae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d0071d20 con 0x7fd3d009a580 2023-09-30T22:21:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d534b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d534b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57716/0 (socket says 172.21.15.89:57716) 2023-09-30T22:21:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d534b700 1 -- 172.21.15.89:0/2507380019 learned_addr learned my addr 172.21.15.89:0/2507380019 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.348+0000 7fd3d4b4a700 1 --2- 172.21.15.89:0/2507380019 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.349+0000 7fd3d534b700 1 -- 172.21.15.89:0/2507380019 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 msgr2=0x7fd3d0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.349+0000 7fd3d534b700 1 --2- 172.21.15.89:0/2507380019 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.349+0000 7fd3d534b700 1 -- 172.21.15.89:0/2507380019 >> v1:172.21.15.89:6789/0 conn(0x7fd3d0099a40 legacy=0x7fd3d0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.349+0000 7fd3d534b700 1 -- 172.21.15.89:0/2507380019 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d00721a0 con 0x7fd3d009a580 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.349+0000 7fd3d534b700 1 --2- 172.21.15.89:0/2507380019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0139660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009e80 tx=0x7fd3c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13bce3735d27379 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.349+0000 7fd3cf7fe700 1 -- 172.21.15.89:0/2507380019 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800e070 con 0x7fd3d009a580 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.349+0000 7fd3cf7fe700 1 -- 172.21.15.89:0/2507380019 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c800b910 con 0x7fd3d009a580 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.349+0000 7fd3cf7fe700 1 -- 172.21.15.89:0/2507380019 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800bcc0 con 0x7fd3d009a580 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.350+0000 7fd3d6dae700 1 -- 172.21.15.89:0/2507380019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 msgr2=0x7fd3d0139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.350+0000 7fd3d6dae700 1 --2- 172.21.15.89:0/2507380019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0139660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009e80 tx=0x7fd3c800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.350+0000 7fd3d6dae700 1 -- 172.21.15.89:0/2507380019 shutdown_connections 2023-09-30T22:21:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.350+0000 7fd3d6dae700 1 --2- 172.21.15.89:0/2507380019 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.350+0000 7fd3d6dae700 1 --2- 172.21.15.89:0/2507380019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0139660 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.350+0000 7fd3d6dae700 1 -- 172.21.15.89:0/2507380019 >> 172.21.15.89:0/2507380019 conn(0x7fd3d00a2aa0 msgr2=0x7fd3d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.350+0000 7fd3d6dae700 1 -- 172.21.15.89:0/2507380019 shutdown_connections 2023-09-30T22:21:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.350+0000 7fd3d6dae700 1 -- 172.21.15.89:0/2507380019 wait complete. 2023-09-30T22:21:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.351+0000 7fd3d6dae700 1 Processor -- start 2023-09-30T22:21:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.352+0000 7fd3d6dae700 1 -- start start 2023-09-30T22:21:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.352+0000 7fd3d6dae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d0099a40 0x7fd3d0119820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.352+0000 7fd3d6dae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0119d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.352+0000 7fd3d6dae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d011a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.352+0000 7fd3d6dae700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d011b3a0 con 0x7fd3d0099a40 2023-09-30T22:21:45.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.352+0000 7fd3d6dae700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d011b9c0 con 0x7fd3d0131810 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.352+0000 7fd3d6dae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d011c5f0 con 0x7fd3d009a580 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0119d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3d534b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d011a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0119d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57728/0 (socket says 172.21.15.89:57728) 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 -- 172.21.15.89:0/3314445650 learned_addr learned my addr 172.21.15.89:0/3314445650 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3d4b4a700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d0099a40 0x7fd3d0119820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 -- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 msgr2=0x7fd3d011a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d011a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 -- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d0099a40 msgr2=0x7fd3d0119820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d0099a40 0x7fd3d0119820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 -- 172.21.15.89:0/3314445650 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c8009b30 con 0x7fd3d009a580 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3d534b700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d011a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3d4b4a700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d0099a40 0x7fd3d0119820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cffff700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0119d60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c4b0 tx=0x7fd3bc00c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.353+0000 7fd3cd7fa700 1 -- 172.21.15.89:0/3314445650 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0052d0 con 0x7fd3d009a580 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.354+0000 7fd3cd7fa700 1 -- 172.21.15.89:0/3314445650 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc005430 con 0x7fd3d009a580 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.354+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d011cfd0 con 0x7fd3d009a580 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.354+0000 7fd3cd7fa700 1 -- 172.21.15.89:0/3314445650 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0038c0 con 0x7fd3d009a580 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.354+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d006eb00 con 0x7fd3d009a580 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.355+0000 7fd3cd7fa700 1 -- 172.21.15.89:0/3314445650 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd3bc003a20 con 0x7fd3d009a580 2023-09-30T22:21:45.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.355+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d0074ea0 con 0x7fd3d009a580 2023-09-30T22:21:45.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.355+0000 7fd3cd7fa700 1 -- 172.21.15.89:0/3314445650 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd3bc03e880 con 0x7fd3d009a580 2023-09-30T22:21:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.364+0000 7fd3cd7fa700 1 -- 172.21.15.89:0/3314445650 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3bc018260 con 0x7fd3d009a580 2023-09-30T22:21:45.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.380+0000 7f3fe169f700 1 Processor -- start 2023-09-30T22:21:45.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fe169f700 1 -- start start 2023-09-30T22:21:45.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fe169f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 0x7f3fdc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fe169f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3fdc09dc30 0x7f3fdc09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fe169f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 0x7f3fdc13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fe169f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fdc06ab40 con 0x7f3fdc09dc30 2023-09-30T22:21:45.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fe169f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc09ebc0 con 0x7f3fdc13c1e0 2023-09-30T22:21:45.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fe169f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc09a810 con 0x7f3fdc06a770 2023-09-30T22:21:45.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fdaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 0x7f3fdc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fda7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3fdc09dc30 0x7f3fdc09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fda7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3fdc09dc30 0x7f3fdc09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46348/0 (socket says 172.21.15.89:46348) 2023-09-30T22:21:45.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fda7fc700 1 -- 172.21.15.89:0/2015390207 learned_addr learned my addr 172.21.15.89:0/2015390207 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:45.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fdaffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 0x7f3fdc09d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57744/0 (socket says 172.21.15.89:57744) 2023-09-30T22:21:45.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.381+0000 7f3fdb7fe700 1 --2- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 0x7f3fdc13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fda7fc700 1 -- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 msgr2=0x7f3fdc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fda7fc700 1 --2- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 0x7f3fdc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fda7fc700 1 -- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 msgr2=0x7f3fdc13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fda7fc700 1 --2- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 0x7f3fdc13e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fda7fc700 1 -- 172.21.15.89:0/2015390207 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc09e920 con 0x7f3fdc09dc30 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fda7fc700 1 --2- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc09e080 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009920 tx=0x7f3fd0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e27425b6e507f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fd9ffb700 1 -- 172.21.15.89:0/2015390207 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0011830 con 0x7f3fdc09dc30 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fd9ffb700 1 -- 172.21.15.89:0/2015390207 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd0011990 con 0x7f3fdc09dc30 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.382+0000 7f3fd9ffb700 1 -- 172.21.15.89:0/2015390207 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0011d20 con 0x7f3fdc09dc30 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 -- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 msgr2=0x7f3fdc09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 --2- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc09e080 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009920 tx=0x7f3fd0009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:45.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 -- 172.21.15.89:0/2015390207 shutdown_connections 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 --2- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 0x7f3fdc13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 --2- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 0x7f3fdc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 --2- 172.21.15.89:0/2015390207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc09e080 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 -- 172.21.15.89:0/2015390207 >> 172.21.15.89:0/2015390207 conn(0x7f3fdc066f50 msgr2=0x7f3fdc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 -- 172.21.15.89:0/2015390207 shutdown_connections 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.383+0000 7f3fe169f700 1 -- 172.21.15.89:0/2015390207 wait complete. 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fe169f700 1 Processor -- start 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fe169f700 1 -- start start 2023-09-30T22:21:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fe169f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 0x7f3fdc11d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fe169f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc11ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fe169f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 0x7f3fdc11e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fe169f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0a38b0 con 0x7f3fdc09dc30 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fe169f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc11fd20 con 0x7f3fdc13c1e0 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fe169f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc09f1a0 con 0x7f3fdc06a770 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.384+0000 7f3fda7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc11ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fda7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc11ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46352/0 (socket says 172.21.15.89:46352) 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fda7fc700 1 -- 172.21.15.89:0/2523975714 learned_addr learned my addr 172.21.15.89:0/2523975714 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fdb7fe700 1 --2- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 0x7f3fdc11e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fda7fc700 1 -- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 msgr2=0x7f3fdc11d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fda7fc700 1 --2- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 0x7f3fdc11d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fda7fc700 1 -- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 msgr2=0x7f3fdc11e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fda7fc700 1 --2- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 0x7f3fdc11e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fda7fc700 1 -- 172.21.15.89:0/2523975714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd0009550 con 0x7f3fdc09dc30 2023-09-30T22:21:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fda7fc700 1 --2- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc11ddd0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009c70 tx=0x7f3fd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fc37fe700 1 -- 172.21.15.89:0/2523975714 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0004240 con 0x7f3fdc09dc30 2023-09-30T22:21:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fc37fe700 1 -- 172.21.15.89:0/2523975714 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd001f070 con 0x7f3fdc09dc30 2023-09-30T22:21:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc138e80 con 0x7f3fdc09dc30 2023-09-30T22:21:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fc37fe700 1 -- 172.21.15.89:0/2523975714 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00045e0 con 0x7f3fdc09dc30 2023-09-30T22:21:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.385+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc13c010 con 0x7f3fdc09dc30 2023-09-30T22:21:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.386+0000 7f3fc37fe700 1 -- 172.21.15.89:0/2523975714 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3fd0021030 con 0x7f3fdc09dc30 2023-09-30T22:21:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.386+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc11cc30 con 0x7f3fdc09dc30 2023-09-30T22:21:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.386+0000 7f3fc37fe700 1 -- 172.21.15.89:0/2523975714 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3fd00407f0 con 0x7f3fdc09dc30 2023-09-30T22:21:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.396+0000 7f3fc37fe700 1 -- 172.21.15.89:0/2523975714 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3fd0003e00 con 0x7f3fdc09dc30 2023-09-30T22:21:45.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fae0c8a7700 1 Processor -- start 2023-09-30T22:21:45.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fae0c8a7700 1 -- start start 2023-09-30T22:21:45.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fae0c8a7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae0809d6f0 0x7fae0809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fae0c8a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fae0c8a7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0809b120 con 0x7fae0809e410 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fae0c8a7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0806b460 con 0x7fae0809d6f0 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fae0c8a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0809a7e0 con 0x7fae0809e000 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fadfffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.428+0000 7fadfffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46238/0 (socket says 172.21.15.89:46238) 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fadfffff700 1 -- 172.21.15.89:0/771442066 learned_addr learned my addr 172.21.15.89:0/771442066 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fae0659c700 1 --2- 172.21.15.89:0/771442066 >> v2:172.21.15.89:3300/0 conn(0x7fae0809d6f0 0x7fae0809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fadfffff700 1 -- 172.21.15.89:0/771442066 >> v2:172.21.15.89:3300/0 conn(0x7fae0809d6f0 msgr2=0x7fae0809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fadfffff700 1 --2- 172.21.15.89:0/771442066 >> v2:172.21.15.89:3300/0 conn(0x7fae0809d6f0 0x7fae0809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fadfffff700 1 -- 172.21.15.89:0/771442066 >> v1:172.21.15.89:6789/0 conn(0x7fae0809e410 legacy=0x7fae08133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fadfffff700 1 -- 172.21.15.89:0/771442066 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0809a500 con 0x7fae0809e000 2023-09-30T22:21:45.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fadfffff700 1 --2- 172.21.15.89:0/771442066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081339f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009920 tx=0x7fadf0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba0ae21fd8de8946 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fae05d9b700 1 -- 172.21.15.89:0/771442066 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0004230 con 0x7fae0809e000 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fae05d9b700 1 -- 172.21.15.89:0/771442066 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0004390 con 0x7fae0809e000 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.429+0000 7fae05d9b700 1 -- 172.21.15.89:0/771442066 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00036a0 con 0x7fae0809e000 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.430+0000 7fae0c8a7700 1 -- 172.21.15.89:0/771442066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 msgr2=0x7fae081339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.430+0000 7fae0c8a7700 1 --2- 172.21.15.89:0/771442066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081339f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009920 tx=0x7fadf0009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.430+0000 7fae0c8a7700 1 -- 172.21.15.89:0/771442066 shutdown_connections 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.430+0000 7fae0c8a7700 1 --2- 172.21.15.89:0/771442066 >> v2:172.21.15.89:3300/0 conn(0x7fae0809d6f0 0x7fae0809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.430+0000 7fae0c8a7700 1 --2- 172.21.15.89:0/771442066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081339f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.430+0000 7fae0c8a7700 1 -- 172.21.15.89:0/771442066 >> 172.21.15.89:0/771442066 conn(0x7fae08066f50 msgr2=0x7fae0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.430+0000 7fae0c8a7700 1 -- 172.21.15.89:0/771442066 shutdown_connections 2023-09-30T22:21:45.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.430+0000 7fae0c8a7700 1 -- 172.21.15.89:0/771442066 wait complete. 2023-09-30T22:21:45.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.432+0000 7fae0c8a7700 1 Processor -- start 2023-09-30T22:21:45.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.432+0000 7fae0c8a7700 1 -- start start 2023-09-30T22:21:45.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.432+0000 7fae0c8a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae0809d6f0 0x7fae081a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.432+0000 7fae0c8a7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.432+0000 7fae0c8a7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae0809e410 0x7fae081a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.432+0000 7fae0c8a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae081a6b00 con 0x7fae0809d6f0 2023-09-30T22:21:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.432+0000 7fae0c8a7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae081a7d50 con 0x7fae0809e000 2023-09-30T22:21:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.432+0000 7fae0c8a7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae081a70f0 con 0x7fae0809e410 2023-09-30T22:21:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae06d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae0809e410 0x7fae081a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae06d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae0809e410 0x7fae081a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57760/0 (socket says 172.21.15.89:57760) 2023-09-30T22:21:45.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae06d9d700 1 -- 172.21.15.89:0/1141282726 learned_addr learned my addr 172.21.15.89:0/1141282726 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0659c700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae0809d6f0 0x7fae081a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fadfffff700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0659c700 1 -- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae0809e410 msgr2=0x7fae081a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0659c700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae0809e410 0x7fae081a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0659c700 1 -- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 msgr2=0x7fae081a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0659c700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0659c700 1 -- 172.21.15.89:0/1141282726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf0009550 con 0x7fae0809d6f0 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0659c700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae0809d6f0 0x7fae081a4ef0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fadf800e870 tx=0x7fadf800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fadfeffd700 1 -- 172.21.15.89:0/1141282726 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8013680 con 0x7fae0809d6f0 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fadfeffd700 1 -- 172.21.15.89:0/1141282726 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf80137e0 con 0x7fae0809d6f0 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae081a81a0 con 0x7fae0809d6f0 2023-09-30T22:21:45.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fadfeffd700 1 -- 172.21.15.89:0/1141282726 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8013b90 con 0x7fae0809d6f0 2023-09-30T22:21:45.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.433+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae081a8730 con 0x7fae0809d6f0 2023-09-30T22:21:45.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.435+0000 7fadfeffd700 1 -- 172.21.15.89:0/1141282726 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fadf8013cf0 con 0x7fae0809d6f0 2023-09-30T22:21:45.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.435+0000 7fadfeffd700 1 -- 172.21.15.89:0/1141282726 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fadf803f960 con 0x7fae0809d6f0 2023-09-30T22:21:45.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.435+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae08074ea0 con 0x7fae0809d6f0 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.438+0000 7f3139009700 1 Processor -- start 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.438+0000 7f3139009700 1 -- start start 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3139009700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f313406a770 0x7f313409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3139009700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313409dc30 0x7f313409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3139009700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313409a810 con 0x7f313413c1e0 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3139009700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313406ab40 con 0x7f313406a770 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3139009700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f313409ebc0 con 0x7f313409dc30 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3132d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f313406a770 0x7f313409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3132d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f313406a770 0x7f313409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46380/0 (socket says 172.21.15.89:46380) 2023-09-30T22:21:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3132d9d700 1 -- 172.21.15.89:0/3459942695 learned_addr learned my addr 172.21.15.89:0/3459942695 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:45.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f313259c700 1 --2- 172.21.15.89:0/3459942695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313409dc30 0x7f313409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3132d9d700 1 -- 172.21.15.89:0/3459942695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313409dc30 msgr2=0x7f313409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3132d9d700 1 --2- 172.21.15.89:0/3459942695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313409dc30 0x7f313409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3132d9d700 1 -- 172.21.15.89:0/3459942695 >> v1:172.21.15.89:6789/0 conn(0x7f313413c1e0 legacy=0x7f313413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.439+0000 7f3132d9d700 1 -- 172.21.15.89:0/3459942695 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313409e920 con 0x7f313406a770 2023-09-30T22:21:45.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.440+0000 7f3132d9d700 1 --2- 172.21.15.89:0/3459942695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313406a770 0x7f313409d6f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f31240098f0 tx=0x7f3124009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce3ef6f7d6bb134 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.441+0000 7f3131d9b700 1 -- 172.21.15.89:0/3459942695 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312400b8a0 con 0x7f313406a770 2023-09-30T22:21:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.441+0000 7f3131d9b700 1 -- 172.21.15.89:0/3459942695 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312400ba00 con 0x7f313406a770 2023-09-30T22:21:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.441+0000 7f3131d9b700 1 -- 172.21.15.89:0/3459942695 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312400bd70 con 0x7f313406a770 2023-09-30T22:21:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.441+0000 7fadfeffd700 1 -- 172.21.15.89:0/1141282726 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fadf8017080 con 0x7fae0809d6f0 2023-09-30T22:21:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.442+0000 7f3139009700 1 -- 172.21.15.89:0/3459942695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313406a770 msgr2=0x7f313409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.442+0000 7f3139009700 1 --2- 172.21.15.89:0/3459942695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313406a770 0x7f313409d6f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f31240098f0 tx=0x7f3124009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.442+0000 7f3139009700 1 -- 172.21.15.89:0/3459942695 shutdown_connections 2023-09-30T22:21:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.442+0000 7f3139009700 1 --2- 172.21.15.89:0/3459942695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313406a770 0x7f313409d6f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.442+0000 7f3139009700 1 --2- 172.21.15.89:0/3459942695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313409dc30 0x7f313409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.442+0000 7f3139009700 1 -- 172.21.15.89:0/3459942695 >> 172.21.15.89:0/3459942695 conn(0x7f3134066f50 msgr2=0x7f313409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.442+0000 7f3139009700 1 -- 172.21.15.89:0/3459942695 shutdown_connections 2023-09-30T22:21:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.442+0000 7f3139009700 1 -- 172.21.15.89:0/3459942695 wait complete. 2023-09-30T22:21:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.443+0000 7f3139009700 1 Processor -- start 2023-09-30T22:21:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.443+0000 7f3139009700 1 -- start start 2023-09-30T22:21:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f3139009700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313406a770 0x7f313413a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f3139009700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313409dc30 0x7f313413a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f3139009700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f313413c1e0 0x7f31341cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f3139009700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134129bd0 con 0x7f313409dc30 2023-09-30T22:21:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f3139009700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313413ac80 con 0x7f313413c1e0 2023-09-30T22:21:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f3139009700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3134127f10 con 0x7f313406a770 2023-09-30T22:21:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313359e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f313413c1e0 0x7f31341cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313359e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f313413c1e0 0x7f31341cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46242/0 (socket says 172.21.15.89:46242) 2023-09-30T22:21:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313359e700 1 -- 172.21.15.89:0/1593736582 learned_addr learned my addr 172.21.15.89:0/1593736582 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313259c700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313409dc30 0x7f313413a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f3132d9d700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313406a770 0x7f313413a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313259c700 1 -- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313406a770 msgr2=0x7f313413a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313259c700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313406a770 0x7f313413a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313259c700 1 -- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f313413c1e0 msgr2=0x7f31341cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313259c700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f313413c1e0 0x7f31341cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313259c700 1 -- 172.21.15.89:0/1593736582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3124009550 con 0x7f313409dc30 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.444+0000 7f313259c700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313409dc30 0x7f313413a740 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f312800e7e0 tx=0x7f312800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.445+0000 7f311b7fe700 1 -- 172.21.15.89:0/1593736582 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312800c070 con 0x7f313409dc30 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.445+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3134072420 con 0x7f313409dc30 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.445+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134128700 con 0x7f313409dc30 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.445+0000 7f311b7fe700 1 -- 172.21.15.89:0/1593736582 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3128013610 con 0x7f313409dc30 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.445+0000 7f311b7fe700 1 -- 172.21.15.89:0/1593736582 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128013980 con 0x7f313409dc30 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.446+0000 7f311b7fe700 1 -- 172.21.15.89:0/1593736582 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3128013ae0 con 0x7f313409dc30 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.446+0000 7f311b7fe700 1 -- 172.21.15.89:0/1593736582 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f312803f440 con 0x7f313409dc30 2023-09-30T22:21:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.446+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31140052e0 con 0x7f313409dc30 2023-09-30T22:21:45.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.451+0000 7f311b7fe700 1 -- 172.21.15.89:0/1593736582 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f312801c400 con 0x7f313409dc30 2023-09-30T22:21:45.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.569+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd3d006b9e0 con 0x7fd3d009a580 2023-09-30T22:21:45.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.569+0000 7fd3cd7fa700 1 -- 172.21.15.89:0/3314445650 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd3bc017090 con 0x7fd3d009a580 2023-09-30T22:21:45.569 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:45.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 msgr2=0x7fd3d0119d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0119d60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c4b0 tx=0x7fd3bc00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 shutdown_connections 2023-09-30T22:21:45.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3d0131810 0x7fd3d011a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d0099a40 0x7fd3d0119820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 --2- 172.21.15.89:0/3314445650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3d009a580 0x7fd3d0119d60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 >> 172.21.15.89:0/3314445650 conn(0x7fd3d00a2aa0 msgr2=0x7fd3d012e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 shutdown_connections 2023-09-30T22:21:45.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.571+0000 7fd3d6dae700 1 -- 172.21.15.89:0/3314445650 wait complete. 2023-09-30T22:21:45.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:45.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.603+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3fdc0723f0 con 0x7f3fdc09dc30 2023-09-30T22:21:45.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.603+0000 7f3fc37fe700 1 -- 172.21.15.89:0/2523975714 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fd0015030 con 0x7f3fdc09dc30 2023-09-30T22:21:45.604 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:45.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 msgr2=0x7f3fdc11ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 --2- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc11ddd0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009c70 tx=0x7f3fd0004750 comp rx=0 tx=0).stop 2023-09-30T22:21:45.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 shutdown_connections 2023-09-30T22:21:45.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 --2- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fdc13c1e0 0x7f3fdc11e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 --2- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fdc06a770 0x7f3fdc11d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 --2- 172.21.15.89:0/2523975714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fdc09dc30 0x7f3fdc11ddd0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 >> 172.21.15.89:0/2523975714 conn(0x7f3fdc066f50 msgr2=0x7f3fdc0a2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 shutdown_connections 2023-09-30T22:21:45.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.606+0000 7f3fe169f700 1 -- 172.21.15.89:0/2523975714 wait complete. 2023-09-30T22:21:45.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:45.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.636+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fae0806c360 con 0x7fae0809d6f0 2023-09-30T22:21:45.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.636+0000 7fadfeffd700 1 -- 172.21.15.89:0/1141282726 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fadf8021020 con 0x7fae0809d6f0 2023-09-30T22:21:45.636 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:45.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.638+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae0809d6f0 msgr2=0x7fae081a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.638+0000 7fae0c8a7700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae0809d6f0 0x7fae081a4ef0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fadf800e870 tx=0x7fadf800eb80 comp rx=0 tx=0).stop 2023-09-30T22:21:45.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.638+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 shutdown_connections 2023-09-30T22:21:45.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.638+0000 7fae0c8a7700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae0809e410 0x7fae081a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.638+0000 7fae0c8a7700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae0809d6f0 0x7fae081a4ef0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.638+0000 7fae0c8a7700 1 --2- 172.21.15.89:0/1141282726 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae0809e000 0x7fae081a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.638+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 >> 172.21.15.89:0/1141282726 conn(0x7fae08066f50 msgr2=0x7fae081b9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.639+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 shutdown_connections 2023-09-30T22:21:45.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.639+0000 7fae0c8a7700 1 -- 172.21.15.89:0/1141282726 wait complete. 2023-09-30T22:21:45.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:45.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.655+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3114005590 con 0x7f313409dc30 2023-09-30T22:21:45.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.656+0000 7f311b7fe700 1 -- 172.21.15.89:0/1593736582 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3128010310 con 0x7f313409dc30 2023-09-30T22:21:45.656 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:45.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313409dc30 msgr2=0x7f313413a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313409dc30 0x7f313413a740 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f312800e7e0 tx=0x7f312800eaf0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 shutdown_connections 2023-09-30T22:21:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f313413c1e0 0x7f31341cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f313406a770 0x7f313413a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 --2- 172.21.15.89:0/1593736582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313409dc30 0x7f313413a740 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 >> 172.21.15.89:0/1593736582 conn(0x7f3134066f50 msgr2=0x7f313413b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 shutdown_connections 2023-09-30T22:21:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:45.658+0000 7f3139009700 1 -- 172.21.15.89:0/1593736582 wait complete. 2023-09-30T22:21:45.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:46.584 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:46.624 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:46.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:46.670 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.734+0000 7fecbc85c700 1 Processor -- start 2023-09-30T22:21:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.734+0000 7fecbc85c700 1 -- start start 2023-09-30T22:21:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecbc85c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fecb812c2f0 0x7fecb8130040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecbc85c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812cff0 0x7fecb8137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecbc85c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb809a510 con 0x7fecb812cc20 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecbc85c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb812c7d0 con 0x7fecb812c2f0 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecbc85c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb809c190 con 0x7fecb812cff0 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecb6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812cff0 0x7fecb8137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecb6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812cff0 0x7fecb8137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46254/0 (socket says 172.21.15.89:46254) 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecb6d9d700 1 -- 172.21.15.89:0/1778017909 learned_addr learned my addr 172.21.15.89:0/1778017909 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecb5d9b700 1 --2- 172.21.15.89:0/1778017909 >> v2:172.21.15.89:3300/0 conn(0x7fecb812c2f0 0x7fecb8130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecb5d9b700 1 -- 172.21.15.89:0/1778017909 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812cff0 msgr2=0x7fecb8137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecb5d9b700 1 --2- 172.21.15.89:0/1778017909 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812cff0 0x7fecb8137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecb5d9b700 1 -- 172.21.15.89:0/1778017909 >> v1:172.21.15.89:6789/0 conn(0x7fecb812cc20 legacy=0x7fecb812f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.735+0000 7fecb5d9b700 1 -- 172.21.15.89:0/1778017909 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb809b540 con 0x7fecb812c2f0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.736+0000 7fecb6d9d700 1 --2- 172.21.15.89:0/1778017909 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812cff0 0x7fecb8137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.736+0000 7fecb5d9b700 1 --2- 172.21.15.89:0/1778017909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812c2f0 0x7fecb8130040 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fecac009840 tx=0x7fecac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f1b60c96085ebd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.736+0000 7fecb559a700 1 -- 172.21.15.89:0/1778017909 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac011830 con 0x7fecb812c2f0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.736+0000 7fecb559a700 1 -- 172.21.15.89:0/1778017909 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac011990 con 0x7fecb812c2f0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.736+0000 7fecb559a700 1 -- 172.21.15.89:0/1778017909 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac011d00 con 0x7fecb812c2f0 2023-09-30T22:21:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.736+0000 7fecbc85c700 1 -- 172.21.15.89:0/1778017909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812c2f0 msgr2=0x7fecb8130040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.736+0000 7fecbc85c700 1 --2- 172.21.15.89:0/1778017909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812c2f0 0x7fecb8130040 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fecac009840 tx=0x7fecac009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.737+0000 7fecbc85c700 1 -- 172.21.15.89:0/1778017909 shutdown_connections 2023-09-30T22:21:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.737+0000 7fecbc85c700 1 --2- 172.21.15.89:0/1778017909 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812cff0 0x7fecb8137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.737+0000 7fecbc85c700 1 --2- 172.21.15.89:0/1778017909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812c2f0 0x7fecb8130040 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.737+0000 7fecbc85c700 1 -- 172.21.15.89:0/1778017909 >> 172.21.15.89:0/1778017909 conn(0x7fecb809f470 msgr2=0x7fecb80a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.737+0000 7fecbc85c700 1 -- 172.21.15.89:0/1778017909 shutdown_connections 2023-09-30T22:21:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.737+0000 7fecbc85c700 1 -- 172.21.15.89:0/1778017909 wait complete. 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.738+0000 7fecbc85c700 1 Processor -- start 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.738+0000 7fecbc85c700 1 -- start start 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecbc85c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812c2f0 0x7fecb813b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecbc85c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812cc20 0x7fecb8070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecbc85c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecb812cff0 0x7fecb8071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecbc85c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb806fa80 con 0x7fecb812cc20 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecbc85c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb806fe20 con 0x7fecb812c2f0 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecbc85c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fecb806f450 con 0x7fecb812cff0 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecb659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812c2f0 0x7fecb813b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecb659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812c2f0 0x7fecb813b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46260/0 (socket says 172.21.15.89:46260) 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecb6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecb812cff0 0x7fecb8071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecb659c700 1 -- 172.21.15.89:0/901964096 learned_addr learned my addr 172.21.15.89:0/901964096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.739+0000 7fecb5d9b700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812cc20 0x7fecb8070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecb659c700 1 -- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecb812cff0 msgr2=0x7fecb8071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecb659c700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecb812cff0 0x7fecb8071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecb659c700 1 -- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812cc20 msgr2=0x7fecb8070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecb659c700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812cc20 0x7fecb8070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecb659c700 1 -- 172.21.15.89:0/901964096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb0009b30 con 0x7fecb812c2f0 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecb6d9d700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecb812cff0 0x7fecb8071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecb659c700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812c2f0 0x7fecb813b370 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7feca800c4e0 tx=0x7feca800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7feca6ffd700 1 -- 172.21.15.89:0/901964096 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca80052d0 con 0x7fecb812c2f0 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7feca6ffd700 1 -- 172.21.15.89:0/901964096 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca800ce90 con 0x7fecb812c2f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7feca6ffd700 1 -- 172.21.15.89:0/901964096 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8003680 con 0x7fecb812c2f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac0094a0 con 0x7fecb812c2f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.740+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb8129f10 con 0x7fecb812c2f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.741+0000 7feca6ffd700 1 -- 172.21.15.89:0/901964096 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feca80037e0 con 0x7fecb812c2f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.741+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb81285a0 con 0x7fecb812c2f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.742+0000 7feca6ffd700 1 -- 172.21.15.89:0/901964096 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feca803ea90 con 0x7fecb812c2f0 2023-09-30T22:21:46.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.746+0000 7feca6ffd700 1 -- 172.21.15.89:0/901964096 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feca8011590 con 0x7fecb812c2f0 2023-09-30T22:21:46.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.766+0000 7f5a2cc0d700 1 Processor -- start 2023-09-30T22:21:46.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.766+0000 7f5a2cc0d700 1 -- start start 2023-09-30T22:21:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.766+0000 7f5a2cc0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a281338a0 0x7f5a28133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.766+0000 7f5a2cc0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a2809e930 0x7f5a2813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.766+0000 7f5a2cc0d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2806ef30 con 0x7f5a2809de50 2023-09-30T22:21:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.766+0000 7f5a2cc0d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2806fef0 con 0x7f5a281338a0 2023-09-30T22:21:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.766+0000 7f5a2cc0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281334b0 con 0x7f5a2809e930 2023-09-30T22:21:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a281338a0 0x7f5a28133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a281338a0 0x7f5a28133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46418/0 (socket says 172.21.15.89:46418) 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2659c700 1 -- 172.21.15.89:0/370499705 learned_addr learned my addr 172.21.15.89:0/370499705 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a26d9d700 1 --2- 172.21.15.89:0/370499705 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a2809e930 0x7f5a2813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2659c700 1 -- 172.21.15.89:0/370499705 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a2809e930 msgr2=0x7f5a2813b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2659c700 1 --2- 172.21.15.89:0/370499705 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a2809e930 0x7f5a2813b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2659c700 1 -- 172.21.15.89:0/370499705 >> v1:172.21.15.89:6789/0 conn(0x7f5a2809de50 legacy=0x7f5a2809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2659c700 1 -- 172.21.15.89:0/370499705 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2806fc30 con 0x7f5a281338a0 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2659c700 1 --2- 172.21.15.89:0/370499705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a281338a0 0x7f5a28133c70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098f0 tx=0x7f5a18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b143031307f45f82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2559a700 1 -- 172.21.15.89:0/370499705 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18011830 con 0x7f5a281338a0 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2559a700 1 -- 172.21.15.89:0/370499705 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a18011990 con 0x7f5a281338a0 2023-09-30T22:21:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.767+0000 7f5a2559a700 1 -- 172.21.15.89:0/370499705 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18011d00 con 0x7f5a281338a0 2023-09-30T22:21:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.768+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/370499705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a281338a0 msgr2=0x7f5a28133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.768+0000 7f5a2cc0d700 1 --2- 172.21.15.89:0/370499705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a281338a0 0x7f5a28133c70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098f0 tx=0x7f5a18009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.768+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/370499705 shutdown_connections 2023-09-30T22:21:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.768+0000 7f5a2cc0d700 1 --2- 172.21.15.89:0/370499705 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a2809e930 0x7f5a2813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.768+0000 7f5a2cc0d700 1 --2- 172.21.15.89:0/370499705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a281338a0 0x7f5a28133c70 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.768+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/370499705 >> 172.21.15.89:0/370499705 conn(0x7f5a280a2aa0 msgr2=0x7f5a280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.768+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/370499705 shutdown_connections 2023-09-30T22:21:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.768+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/370499705 wait complete. 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.769+0000 7f5a2cc0d700 1 Processor -- start 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.769+0000 7f5a2cc0d700 1 -- start start 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2cc0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a2809de50 0x7f5a2811eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2cc0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a2809e930 0x7f5a2811f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2cc0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a281338a0 0x7f5a2811f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2cc0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806f640 con 0x7f5a2809de50 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2cc0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281222b0 con 0x7f5a281338a0 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2cc0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28120790 con 0x7f5a2809e930 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a2809de50 0x7f5a2811eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a2809de50 0x7f5a2811eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46430/0 (socket says 172.21.15.89:46430) 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2659c700 1 -- 172.21.15.89:0/4224532606 learned_addr learned my addr 172.21.15.89:0/4224532606 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a25d9b700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a2809e930 0x7f5a2811f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2659c700 1 -- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a2809e930 msgr2=0x7f5a2811f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2659c700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a2809e930 0x7f5a2811f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2659c700 1 -- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a281338a0 msgr2=0x7f5a2811f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a26d9d700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a281338a0 0x7f5a2811f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2659c700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a281338a0 0x7f5a2811f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a2659c700 1 -- 172.21.15.89:0/4224532606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a18009550 con 0x7f5a2809de50 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a25d9b700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a2809e930 0x7f5a2811f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.770+0000 7f5a26d9d700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a281338a0 0x7f5a2811f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.771+0000 7f5a2659c700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a2809de50 0x7f5a2811eac0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009c40 tx=0x7f5a18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.771+0000 7f5a16ffd700 1 -- 172.21.15.89:0/4224532606 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18004510 con 0x7f5a2809de50 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.771+0000 7f5a16ffd700 1 -- 172.21.15.89:0/4224532606 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a180036a0 con 0x7f5a2809de50 2023-09-30T22:21:46.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.771+0000 7f5a16ffd700 1 -- 172.21.15.89:0/4224532606 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18003a50 con 0x7f5a2809de50 2023-09-30T22:21:46.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.771+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a280a1110 con 0x7f5a2809de50 2023-09-30T22:21:46.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.771+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a28122d20 con 0x7f5a2809de50 2023-09-30T22:21:46.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.772+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a28074ea0 con 0x7f5a2809de50 2023-09-30T22:21:46.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.772+0000 7f5a16ffd700 1 -- 172.21.15.89:0/4224532606 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a18003bb0 con 0x7f5a2809de50 2023-09-30T22:21:46.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.773+0000 7f5a16ffd700 1 -- 172.21.15.89:0/4224532606 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5a18040ac0 con 0x7f5a2809de50 2023-09-30T22:21:46.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.777+0000 7f5a16ffd700 1 -- 172.21.15.89:0/4224532606 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a1801a420 con 0x7f5a2809de50 2023-09-30T22:21:46.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.811+0000 7fa29893b700 1 Processor -- start 2023-09-30T22:21:46.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.811+0000 7fa29893b700 1 -- start start 2023-09-30T22:21:46.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.811+0000 7fa29893b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409d6f0 0x7fa29409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa29893b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 0x7fa294134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa29893b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29409b120 con 0x7fa29409e000 2023-09-30T22:21:46.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa29893b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29406b460 con 0x7fa29409d6f0 2023-09-30T22:21:46.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa29893b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa29409a7e0 con 0x7fa29409e410 2023-09-30T22:21:46.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa29259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409d6f0 0x7fa29409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa291d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa29409e000 0x7fa2941339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38228/0 (socket says 172.21.15.89:38228) 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa291d9b700 1 -- 172.21.15.89:0/2714196242 learned_addr learned my addr 172.21.15.89:0/2714196242 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa292d9d700 1 --2- 172.21.15.89:0/2714196242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 0x7fa294134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa292d9d700 1 -- 172.21.15.89:0/2714196242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409d6f0 msgr2=0x7fa29409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa292d9d700 1 --2- 172.21.15.89:0/2714196242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409d6f0 0x7fa29409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa292d9d700 1 -- 172.21.15.89:0/2714196242 >> v1:172.21.15.89:6789/0 conn(0x7fa29409e000 legacy=0x7fa2941339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa292d9d700 1 -- 172.21.15.89:0/2714196242 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29409a500 con 0x7fa29409e410 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa292d9d700 1 --2- 172.21.15.89:0/2714196242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 0x7fa294134100 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009df0 tx=0x7fa28c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9fd431a1c8f6541 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.812+0000 7fa29159a700 1 -- 172.21.15.89:0/2714196242 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c00e070 con 0x7fa29409e410 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.813+0000 7fa29159a700 1 -- 172.21.15.89:0/2714196242 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c00bae0 con 0x7fa29409e410 2023-09-30T22:21:46.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.813+0000 7fa29159a700 1 -- 172.21.15.89:0/2714196242 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c00be90 con 0x7fa29409e410 2023-09-30T22:21:46.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.813+0000 7fa29893b700 1 -- 172.21.15.89:0/2714196242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 msgr2=0x7fa294134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.813+0000 7fa29893b700 1 --2- 172.21.15.89:0/2714196242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 0x7fa294134100 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009df0 tx=0x7fa28c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:46.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.813+0000 7fa29893b700 1 -- 172.21.15.89:0/2714196242 shutdown_connections 2023-09-30T22:21:46.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.813+0000 7fa29893b700 1 --2- 172.21.15.89:0/2714196242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409d6f0 0x7fa29409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.813+0000 7fa29893b700 1 --2- 172.21.15.89:0/2714196242 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 0x7fa294134100 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.813+0000 7fa29893b700 1 -- 172.21.15.89:0/2714196242 >> 172.21.15.89:0/2714196242 conn(0x7fa294066f50 msgr2=0x7fa29409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.814+0000 7fa29893b700 1 -- 172.21.15.89:0/2714196242 shutdown_connections 2023-09-30T22:21:46.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.814+0000 7fa29893b700 1 -- 172.21.15.89:0/2714196242 wait complete. 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.815+0000 7fa29893b700 1 Processor -- start 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.815+0000 7fa29893b700 1 -- start start 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa29893b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29409d6f0 0x7fa2941224f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa29893b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409e000 0x7fa294122a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa29893b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 0x7fa294122f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa29893b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29409c710 con 0x7fa29409d6f0 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa29893b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940721a0 con 0x7fa29409e000 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa29893b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa294124f20 con 0x7fa29409e410 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa291d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409e000 0x7fa294122a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa291d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409e000 0x7fa294122a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46302/0 (socket says 172.21.15.89:46302) 2023-09-30T22:21:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa291d9b700 1 -- 172.21.15.89:0/4063287752 learned_addr learned my addr 172.21.15.89:0/4063287752 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa29259c700 1 --2- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29409d6f0 0x7fa2941224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa291d9b700 1 -- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 msgr2=0x7fa294122f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa291d9b700 1 --2- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 0x7fa294122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa291d9b700 1 -- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29409d6f0 msgr2=0x7fa2941224f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa291d9b700 1 --2- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29409d6f0 0x7fa2941224f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.816+0000 7fa291d9b700 1 -- 172.21.15.89:0/4063287752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c009a70 con 0x7fa29409e000 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.817+0000 7fa291d9b700 1 --2- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409e000 0x7fa294122a30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa28800c500 tx=0x7fa28800c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.817+0000 7fa282ffd700 1 -- 172.21.15.89:0/4063287752 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa288005d70 con 0x7fa29409e000 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.817+0000 7fa282ffd700 1 -- 172.21.15.89:0/4063287752 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2880070d0 con 0x7fa29409e000 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.817+0000 7fa282ffd700 1 -- 172.21.15.89:0/4063287752 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa288007440 con 0x7fa29409e000 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.817+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2941257a0 con 0x7fa29409e000 2023-09-30T22:21:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.817+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2941267a0 con 0x7fa29409e000 2023-09-30T22:21:46.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.818+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa294074ea0 con 0x7fa29409e000 2023-09-30T22:21:46.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.818+0000 7fa282ffd700 1 -- 172.21.15.89:0/4063287752 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2880075a0 con 0x7fa29409e000 2023-09-30T22:21:46.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.822+0000 7fa282ffd700 1 -- 172.21.15.89:0/4063287752 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa288010b90 con 0x7fa29409e000 2023-09-30T22:21:46.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.822+0000 7eff00b47700 1 Processor -- start 2023-09-30T22:21:46.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.822+0000 7eff00b47700 1 -- start start 2023-09-30T22:21:46.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.822+0000 7eff00b47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efefc06e570 0x7efefc06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.822+0000 7eff00b47700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efefc09eca0 0x7efefc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.822+0000 7eff00b47700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efefc070ee0 con 0x7efefc134610 2023-09-30T22:21:46.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.822+0000 7eff00b47700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc071480 con 0x7efefc09eca0 2023-09-30T22:21:46.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.822+0000 7eff00b47700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efefc06f3c0 con 0x7efefc06e570 2023-09-30T22:21:46.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efefad9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efefc134610 0x7efefc134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38244/0 (socket says 172.21.15.89:38244) 2023-09-30T22:21:46.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efefad9d700 1 -- 172.21.15.89:0/1038000176 learned_addr learned my addr 172.21.15.89:0/1038000176 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:46.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270968568 0 0) 0x7efefc070ee0 con 0x7efefc134610 2023-09-30T22:21:46.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee40035b0 con 0x7efefc134610 2023-09-30T22:21:46.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7fa282ffd700 1 -- 172.21.15.89:0/4063287752 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa288019b40 con 0x7fa29409e000 2023-09-30T22:21:46.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2577589909 0 0) 0x7efee40035b0 con 0x7efefc134610 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efefc06e570 msgr2=0x7efefc06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efefa59c700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efefc06e570 0x7efefc06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef9d9b700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efefc09eca0 0x7efefc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efefc06e570 0x7efefc06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efefc09eca0 msgr2=0x7efefc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efefc09eca0 0x7efefc06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc071b90 con 0x7efefc134610 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7efef40019f0 con 0x7efefc134610 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 >> v1:172.21.15.89:6789/0 conn(0x7efefc134610 legacy=0x7efefc134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efee4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 0x7efee4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 0x7efee400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc071480 con 0x7efee4004850 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc06f3c0 con 0x7efee4003a00 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.823+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efee400c160 con 0x7efee4005250 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefa59c700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efefc06e570 0x7efefc06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efef9d9b700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efefc09eca0 0x7efefc06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefa59c700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efee4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefad9d700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 0x7efee400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efef9d9b700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 0x7efee4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefa59c700 1 -- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 msgr2=0x7efee400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefa59c700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 0x7efee400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefa59c700 1 -- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 msgr2=0x7efee4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefa59c700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 0x7efee4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefa59c700 1 -- 172.21.15.89:0/1038000176 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee40035b0 con 0x7efee4003a00 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efefa59c700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efee4003e10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efefc0a01b0 tx=0x7efee4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20ec840f0bf27e68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec004230 con 0x7efee4003a00 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec004390 con 0x7efee4003a00 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.824+0000 7efef959a700 1 -- 172.21.15.89:0/1038000176 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0036a0 con 0x7efee4003a00 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.825+0000 7eff00b47700 1 -- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 msgr2=0x7efee4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.825+0000 7eff00b47700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efee4003e10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efefc0a01b0 tx=0x7efee4005880 comp rx=0 tx=0).stop 2023-09-30T22:21:46.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.825+0000 7eff00b47700 1 -- 172.21.15.89:0/1038000176 shutdown_connections 2023-09-30T22:21:46.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.825+0000 7eff00b47700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 0x7efee4004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.825+0000 7eff00b47700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efee4003e10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.825+0000 7eff00b47700 1 --2- 172.21.15.89:0/1038000176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 0x7efee400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.825+0000 7eff00b47700 1 -- 172.21.15.89:0/1038000176 >> 172.21.15.89:0/1038000176 conn(0x7efefc0a2aa0 msgr2=0x7efefc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.825+0000 7eff00b47700 1 -- 172.21.15.89:0/1038000176 shutdown_connections 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.826+0000 7eff00b47700 1 -- 172.21.15.89:0/1038000176 wait complete. 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7eff00b47700 1 Processor -- start 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7eff00b47700 1 -- start start 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7eff00b47700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efefc1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7eff00b47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 0x7efefc121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7eff00b47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 0x7efefc121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7eff00b47700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc125200 con 0x7efee4004850 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7eff00b47700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc122930 con 0x7efee4003a00 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7eff00b47700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efefc122f50 con 0x7efee4005250 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7efefa59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efefc1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.827+0000 7efefa59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efefc1214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:46328/0 (socket says 172.21.15.89:46328) 2023-09-30T22:21:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efefa59c700 1 -- 172.21.15.89:0/4109790971 learned_addr learned my addr 172.21.15.89:0/4109790971 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efefad9d700 1 --2- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 0x7efefc121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efefa59c700 1 -- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 msgr2=0x7efefc121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efefa59c700 1 --2- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 0x7efefc121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efefa59c700 1 -- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 msgr2=0x7efefc121a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efefa59c700 1 --2- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 0x7efefc121a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efefa59c700 1 -- 172.21.15.89:0/4109790971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc123b80 con 0x7efee4003a00 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efefa59c700 1 --2- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efefc1214e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efeec00afd0 tx=0x7efeec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efeeaffd700 1 -- 172.21.15.89:0/4109790971 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec01f070 con 0x7efee4003a00 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efeeaffd700 1 -- 172.21.15.89:0/4109790971 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec011600 con 0x7efee4003a00 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc124560 con 0x7efee4003a00 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7efeeaffd700 1 -- 172.21.15.89:0/4109790971 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0119b0 con 0x7efee4003a00 2023-09-30T22:21:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.828+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc070610 con 0x7efee4003a00 2023-09-30T22:21:46.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.829+0000 7efeeaffd700 1 -- 172.21.15.89:0/4109790971 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efeec011b10 con 0x7efee4003a00 2023-09-30T22:21:46.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.829+0000 7efeeaffd700 1 -- 172.21.15.89:0/4109790971 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efeec044490 con 0x7efee4003a00 2023-09-30T22:21:46.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.830+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc09b1e0 con 0x7efee4003a00 2023-09-30T22:21:46.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.834+0000 7efeeaffd700 1 -- 172.21.15.89:0/4109790971 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efeec015070 con 0x7efee4003a00 2023-09-30T22:21:46.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.974+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5a2806b9e0 con 0x7f5a2809de50 2023-09-30T22:21:46.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.975+0000 7f5a16ffd700 1 -- 172.21.15.89:0/4224532606 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a18015020 con 0x7f5a2809de50 2023-09-30T22:21:46.975 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:46.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a2809de50 msgr2=0x7f5a2811eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a2809de50 0x7f5a2811eac0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009c40 tx=0x7f5a18004750 comp rx=0 tx=0).stop 2023-09-30T22:21:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 shutdown_connections 2023-09-30T22:21:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a281338a0 0x7f5a2811f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a2809de50 0x7f5a2811eac0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 --2- 172.21.15.89:0/4224532606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a2809e930 0x7f5a2811f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 >> 172.21.15.89:0/4224532606 conn(0x7f5a280a2aa0 msgr2=0x7f5a2809ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 shutdown_connections 2023-09-30T22:21:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.977+0000 7f5a2cc0d700 1 -- 172.21.15.89:0/4224532606 wait complete. 2023-09-30T22:21:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.986+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fecb80747f0 con 0x7fecb812c2f0 2023-09-30T22:21:46.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.986+0000 7feca6ffd700 1 -- 172.21.15.89:0/901964096 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feca8017070 con 0x7fecb812c2f0 2023-09-30T22:21:46.986 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.988+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812c2f0 msgr2=0x7fecb813b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.988+0000 7fecbc85c700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812c2f0 0x7fecb813b370 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7feca800c4e0 tx=0x7feca800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.988+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 shutdown_connections 2023-09-30T22:21:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.988+0000 7fecbc85c700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecb812cff0 0x7fecb8071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.988+0000 7fecbc85c700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecb812c2f0 0x7fecb813b370 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.988+0000 7fecbc85c700 1 --2- 172.21.15.89:0/901964096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb812cc20 0x7fecb8070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:46.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.988+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 >> 172.21.15.89:0/901964096 conn(0x7fecb809f470 msgr2=0x7fecb812f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.989+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 shutdown_connections 2023-09-30T22:21:46.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:46.989+0000 7fecbc85c700 1 -- 172.21.15.89:0/901964096 wait complete. 2023-09-30T22:21:47.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:47.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.018+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa29406c360 con 0x7fa29409e000 2023-09-30T22:21:47.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.019+0000 7fa282ffd700 1 -- 172.21.15.89:0/4063287752 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa288017030 con 0x7fa29409e000 2023-09-30T22:21:47.019 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:47.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409e000 msgr2=0x7fa294122a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 --2- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409e000 0x7fa294122a30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa28800c500 tx=0x7fa28800c810 comp rx=0 tx=0).stop 2023-09-30T22:21:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 shutdown_connections 2023-09-30T22:21:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 --2- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa29409e410 0x7fa294122f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 --2- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa29409d6f0 0x7fa2941224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:47.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 --2- 172.21.15.89:0/4063287752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa29409e000 0x7fa294122a30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 >> 172.21.15.89:0/4063287752 conn(0x7fa294066f50 msgr2=0x7fa294070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 shutdown_connections 2023-09-30T22:21:47.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.021+0000 7fa29893b700 1 -- 172.21.15.89:0/4063287752 wait complete. 2023-09-30T22:21:47.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:47.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.062+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efefc135140 con 0x7efee4003a00 2023-09-30T22:21:47.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.062+0000 7efeeaffd700 1 -- 172.21.15.89:0/4109790971 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efeec015020 con 0x7efee4003a00 2023-09-30T22:21:47.062 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.064+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 msgr2=0x7efefc1214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.064+0000 7eff00b47700 1 --2- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efefc1214e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efeec00afd0 tx=0x7efeec0047c0 comp rx=0 tx=0).stop 2023-09-30T22:21:47.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.065+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 shutdown_connections 2023-09-30T22:21:47.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.065+0000 7eff00b47700 1 --2- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee4005250 0x7efefc121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:47.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.065+0000 7eff00b47700 1 --2- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4003a00 0x7efefc1214e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:47.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.065+0000 7eff00b47700 1 --2- 172.21.15.89:0/4109790971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efee4004850 0x7efefc121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:47.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.065+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 >> 172.21.15.89:0/4109790971 conn(0x7efefc0a2aa0 msgr2=0x7efefc1bf180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:47.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.065+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 shutdown_connections 2023-09-30T22:21:47.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:47.065+0000 7eff00b47700 1 -- 172.21.15.89:0/4109790971 wait complete. 2023-09-30T22:21:47.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:47.991 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:48.002 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:48.034 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:48.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:48.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f90748aa700 1 Processor -- start 2023-09-30T22:21:48.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f90748aa700 1 -- start start 2023-09-30T22:21:48.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f90748aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070133420 0x7f90701337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f90748aa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9070134260 0x7f90701306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f90748aa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907006b990 con 0x7f9070133e30 2023-09-30T22:21:48.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f90748aa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90700a0590 con 0x7f9070134260 2023-09-30T22:21:48.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f90748aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f907009fb80 con 0x7f9070133420 2023-09-30T22:21:48.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f906ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9070134260 0x7f90701306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f906e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070133420 0x7f90701337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f906ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9070134260 0x7f90701306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46460/0 (socket says 172.21.15.89:46460) 2023-09-30T22:21:48.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.121+0000 7f906ed9d700 1 -- 172.21.15.89:0/3326949182 learned_addr learned my addr 172.21.15.89:0/3326949182 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:48.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906e59c700 1 -- 172.21.15.89:0/3326949182 >> v2:172.21.15.89:3300/0 conn(0x7f9070134260 msgr2=0x7f90701306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906e59c700 1 --2- 172.21.15.89:0/3326949182 >> v2:172.21.15.89:3300/0 conn(0x7f9070134260 0x7f90701306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906e59c700 1 -- 172.21.15.89:0/3326949182 >> v1:172.21.15.89:6789/0 conn(0x7f9070133e30 legacy=0x7f907012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906e59c700 1 -- 172.21.15.89:0/3326949182 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700a0030 con 0x7f9070133420 2023-09-30T22:21:48.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906ed9d700 1 --2- 172.21.15.89:0/3326949182 >> v2:172.21.15.89:3300/0 conn(0x7f9070134260 0x7f90701306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906e59c700 1 --2- 172.21.15.89:0/3326949182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070133420 0x7f90701337f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9060009840 tx=0x7f9060009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20be2222c26e2a5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906d59a700 1 -- 172.21.15.89:0/3326949182 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060011830 con 0x7f9070133420 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906d59a700 1 -- 172.21.15.89:0/3326949182 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9060011990 con 0x7f9070133420 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.122+0000 7f906d59a700 1 -- 172.21.15.89:0/3326949182 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060011d20 con 0x7f9070133420 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.123+0000 7f90748aa700 1 -- 172.21.15.89:0/3326949182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070133420 msgr2=0x7f90701337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.123+0000 7f90748aa700 1 --2- 172.21.15.89:0/3326949182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070133420 0x7f90701337f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9060009840 tx=0x7f9060009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.123+0000 7f90748aa700 1 -- 172.21.15.89:0/3326949182 shutdown_connections 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.123+0000 7f90748aa700 1 --2- 172.21.15.89:0/3326949182 >> v2:172.21.15.89:3300/0 conn(0x7f9070134260 0x7f90701306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.123+0000 7f90748aa700 1 --2- 172.21.15.89:0/3326949182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070133420 0x7f90701337f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.123+0000 7f90748aa700 1 -- 172.21.15.89:0/3326949182 >> 172.21.15.89:0/3326949182 conn(0x7f90700a4210 msgr2=0x7f90700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.123+0000 7f90748aa700 1 -- 172.21.15.89:0/3326949182 shutdown_connections 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.123+0000 7f90748aa700 1 -- 172.21.15.89:0/3326949182 wait complete. 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.124+0000 7f90748aa700 1 Processor -- start 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f90748aa700 1 -- start start 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f90748aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9070133420 0x7f90700709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f90748aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9070133e30 0x7f9070070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f90748aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070134260 0x7f9070071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f90748aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907012d570 con 0x7f9070133e30 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f90748aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907012dae0 con 0x7f9070133420 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f90748aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9070072340 con 0x7f9070134260 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f906ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070134260 0x7f9070071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f906ed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070134260 0x7f9070071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57878/0 (socket says 172.21.15.89:57878) 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f906ed9d700 1 -- 172.21.15.89:0/1232112820 learned_addr learned my addr 172.21.15.89:0/1232112820 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f906dd9b700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9070133e30 0x7f9070070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f906dd9b700 1 -- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070134260 msgr2=0x7f9070071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.125+0000 7f906dd9b700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070134260 0x7f9070071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f906dd9b700 1 -- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9070133420 msgr2=0x7f90700709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f906dd9b700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9070133420 0x7f90700709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f906dd9b700 1 -- 172.21.15.89:0/1232112820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9068009b30 con 0x7f9070133e30 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f906ed9d700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070134260 0x7f9070071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f906dd9b700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9070133e30 0x7f9070070ee0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f907009f2c0 tx=0x7f906400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f905effd700 1 -- 172.21.15.89:0/1232112820 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906400cea0 con 0x7f9070133e30 2023-09-30T22:21:48.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f905effd700 1 -- 172.21.15.89:0/1232112820 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90640052d0 con 0x7f9070133e30 2023-09-30T22:21:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90600094a0 con 0x7f9070133e30 2023-09-30T22:21:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f905effd700 1 -- 172.21.15.89:0/1232112820 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064003680 con 0x7f9070133e30 2023-09-30T22:21:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.126+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9070131980 con 0x7f9070133e30 2023-09-30T22:21:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.127+0000 7f905effd700 1 -- 172.21.15.89:0/1232112820 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f90640037e0 con 0x7f9070133e30 2023-09-30T22:21:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.127+0000 7f905effd700 1 -- 172.21.15.89:0/1232112820 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f906403e9f0 con 0x7f9070133e30 2023-09-30T22:21:48.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.127+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9070074ea0 con 0x7f9070133e30 2023-09-30T22:21:48.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.132+0000 7f905effd700 1 -- 172.21.15.89:0/1232112820 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9064011590 con 0x7f9070133e30 2023-09-30T22:21:48.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.165+0000 7f29cc433700 1 Processor -- start 2023-09-30T22:21:48.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.165+0000 7f29cc433700 1 -- start start 2023-09-30T22:21:48.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.165+0000 7fbebf19e700 1 Processor -- start 2023-09-30T22:21:48.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7fbebf19e700 1 -- start start 2023-09-30T22:21:48.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7fbebf19e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809e670 0x7fbeb809ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7fbebf19e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbeb8134070 0x7fbeb813bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7fbebf19e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb806bb20 con 0x7fbeb809ef80 2023-09-30T22:21:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7fbebf19e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb806cd80 con 0x7fbeb8134070 2023-09-30T22:21:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7fbebf19e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb806b880 con 0x7fbeb809e670 2023-09-30T22:21:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7f29cc433700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f29c409e930 0x7f29c409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7f29cc433700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 0x7f29c41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7f29cc433700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409f650 0x7f29c4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7f29cc433700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c4070940 con 0x7f29c409e930 2023-09-30T22:21:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7f29cc433700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c406fd60 con 0x7f29c409f240 2023-09-30T22:21:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.166+0000 7f29cc433700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29c406fa80 con 0x7f29c409f650 2023-09-30T22:21:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.167+0000 7fbebe19c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809e670 0x7fbeb809ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.167+0000 7fbebe19c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809e670 0x7fbeb809ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57884/0 (socket says 172.21.15.89:57884) 2023-09-30T22:21:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.167+0000 7fbebe19c700 1 -- 172.21.15.89:0/405853050 learned_addr learned my addr 172.21.15.89:0/405853050 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.167+0000 7f29ca9d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409f650 0x7f29c4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.167+0000 7f29ca9d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409f650 0x7f29c4131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57890/0 (socket says 172.21.15.89:57890) 2023-09-30T22:21:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.167+0000 7f29ca9d0700 1 -- 172.21.15.89:0/2201873878 learned_addr learned my addr 172.21.15.89:0/2201873878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29ca1cf700 1 --2- 172.21.15.89:0/2201873878 >> v2:172.21.15.89:3300/0 conn(0x7f29c409e930 0x7f29c409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29ca9d0700 1 -- 172.21.15.89:0/2201873878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 msgr2=0x7f29c41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29ca9d0700 1 --2- 172.21.15.89:0/2201873878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 0x7f29c41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29ca9d0700 1 -- 172.21.15.89:0/2201873878 >> v2:172.21.15.89:3300/0 conn(0x7f29c409e930 msgr2=0x7f29c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29ca9d0700 1 --2- 172.21.15.89:0/2201873878 >> v2:172.21.15.89:3300/0 conn(0x7f29c409e930 0x7f29c409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29ca9d0700 1 -- 172.21.15.89:0/2201873878 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c409fb60 con 0x7f29c409f650 2023-09-30T22:21:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebe99d700 1 --2- 172.21.15.89:0/405853050 >> v2:172.21.15.89:3300/0 conn(0x7fbeb8134070 0x7fbeb813bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebe19c700 1 -- 172.21.15.89:0/405853050 >> v2:172.21.15.89:3300/0 conn(0x7fbeb8134070 msgr2=0x7fbeb813bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebe19c700 1 --2- 172.21.15.89:0/405853050 >> v2:172.21.15.89:3300/0 conn(0x7fbeb8134070 0x7fbeb813bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebe19c700 1 -- 172.21.15.89:0/405853050 >> v1:172.21.15.89:6789/0 conn(0x7fbeb809ef80 legacy=0x7fbeb8133960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebe19c700 1 -- 172.21.15.89:0/405853050 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb80a0360 con 0x7fbeb809e670 2023-09-30T22:21:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebe19c700 1 --2- 172.21.15.89:0/405853050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809e670 0x7fbeb809ea40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40098f0 tx=0x7fbeb4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34afec6cc0d6d7da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29ca9d0700 1 --2- 172.21.15.89:0/2201873878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409f650 0x7f29c4131d30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009e50 tx=0x7f29bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb46edd3ee370892 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebd19a700 1 -- 172.21.15.89:0/405853050 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4004230 con 0x7fbeb809e670 2023-09-30T22:21:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebd19a700 1 -- 172.21.15.89:0/405853050 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb4004390 con 0x7fbeb809e670 2023-09-30T22:21:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7fbebd19a700 1 -- 172.21.15.89:0/405853050 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40036a0 con 0x7fbeb809e670 2023-09-30T22:21:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29c91cd700 1 -- 172.21.15.89:0/2201873878 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00e070 con 0x7f29c409f650 2023-09-30T22:21:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29c91cd700 1 -- 172.21.15.89:0/2201873878 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00b910 con 0x7f29c409f650 2023-09-30T22:21:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.168+0000 7f29c91cd700 1 -- 172.21.15.89:0/2201873878 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00bcc0 con 0x7f29c409f650 2023-09-30T22:21:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7fbebf19e700 1 -- 172.21.15.89:0/405853050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809e670 msgr2=0x7fbeb809ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7fbebf19e700 1 --2- 172.21.15.89:0/405853050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809e670 0x7fbeb809ea40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40098f0 tx=0x7fbeb4009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 -- 172.21.15.89:0/2201873878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409f650 msgr2=0x7f29c4131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 --2- 172.21.15.89:0/2201873878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409f650 0x7f29c4131d30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009e50 tx=0x7f29bc00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 -- 172.21.15.89:0/2201873878 shutdown_connections 2023-09-30T22:21:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 --2- 172.21.15.89:0/2201873878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409f650 0x7f29c4131d30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 --2- 172.21.15.89:0/2201873878 >> v2:172.21.15.89:3300/0 conn(0x7f29c409e930 0x7f29c409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 --2- 172.21.15.89:0/2201873878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 0x7f29c41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 -- 172.21.15.89:0/2201873878 >> 172.21.15.89:0/2201873878 conn(0x7f29c40a4210 msgr2=0x7f29c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7fbebf19e700 1 -- 172.21.15.89:0/405853050 shutdown_connections 2023-09-30T22:21:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7fbebf19e700 1 --2- 172.21.15.89:0/405853050 >> v2:172.21.15.89:3300/0 conn(0x7fbeb8134070 0x7fbeb813bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7fbebf19e700 1 --2- 172.21.15.89:0/405853050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809e670 0x7fbeb809ea40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7fbebf19e700 1 -- 172.21.15.89:0/405853050 >> 172.21.15.89:0/405853050 conn(0x7fbeb80a0c90 msgr2=0x7fbeb80a1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7fbebf19e700 1 -- 172.21.15.89:0/405853050 shutdown_connections 2023-09-30T22:21:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 -- 172.21.15.89:0/2201873878 shutdown_connections 2023-09-30T22:21:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7fbebf19e700 1 -- 172.21.15.89:0/405853050 wait complete. 2023-09-30T22:21:48.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.169+0000 7f29cc433700 1 -- 172.21.15.89:0/2201873878 wait complete. 2023-09-30T22:21:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.170+0000 7fbebf19e700 1 Processor -- start 2023-09-30T22:21:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.170+0000 7fbebf19e700 1 -- start start 2023-09-30T22:21:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7f29cc433700 1 Processor -- start 2023-09-30T22:21:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7f29cc433700 1 -- start start 2023-09-30T22:21:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebf19e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeb809e670 0x7fbeb8122880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebf19e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809ef80 0x7fbeb8122dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebf19e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeb8134070 0x7fbeb8123300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebf19e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8126300 con 0x7fbeb809e670 2023-09-30T22:21:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebf19e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8126ee0 con 0x7fbeb8134070 2023-09-30T22:21:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebf19e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb809b470 con 0x7fbeb809ef80 2023-09-30T22:21:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebe19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeb809e670 0x7fbeb8122880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebd99b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809ef80 0x7fbeb8122dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebe19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeb809e670 0x7fbeb8122880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46490/0 (socket says 172.21.15.89:46490) 2023-09-30T22:21:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebe19c700 1 -- 172.21.15.89:0/2834355604 learned_addr learned my addr 172.21.15.89:0/2834355604 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7fbebe99d700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeb8134070 0x7fbeb8123300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebe19c700 1 -- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809ef80 msgr2=0x7fbeb8122dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebe19c700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809ef80 0x7fbeb8122dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebe19c700 1 -- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeb8134070 msgr2=0x7fbeb8123300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebe19c700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeb8134070 0x7fbeb8123300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebe19c700 1 -- 172.21.15.89:0/2834355604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4009550 con 0x7fbeb809e670 2023-09-30T22:21:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebd99b700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809ef80 0x7fbeb8122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebe99d700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeb8134070 0x7fbeb8123300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:48.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7f29cc433700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409e930 0x7f29c406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7f29cc433700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 0x7f29c406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7f29cc433700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29c409f650 0x7f29c406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7f29cc433700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40a3510 con 0x7f29c409f650 2023-09-30T22:21:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7f29cc433700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41a9050 con 0x7f29c409f240 2023-09-30T22:21:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.171+0000 7f29cc433700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41a9320 con 0x7f29c409e930 2023-09-30T22:21:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409e930 0x7f29c406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409e930 0x7f29c406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57916/0 (socket says 172.21.15.89:57916) 2023-09-30T22:21:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 -- 172.21.15.89:0/2606830257 learned_addr learned my addr 172.21.15.89:0/2606830257 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:48.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebe19c700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeb809e670 0x7fbeb8122880 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fbeb806b9c0 tx=0x7fbeb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29c99ce700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 0x7f29c406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca9d0700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29c409f650 0x7f29c406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 -- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 msgr2=0x7f29c406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 0x7f29c406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 -- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29c409f650 msgr2=0x7f29c406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29c409f650 0x7f29c406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 -- 172.21.15.89:0/2606830257 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc009b30 con 0x7f29c409e930 2023-09-30T22:21:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca9d0700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29c409f650 0x7f29c406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29ca1cf700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409e930 0x7f29c406b8b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f29c000e9b0 tx=0x7f29c000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbeaeffd700 1 -- 172.21.15.89:0/2834355604 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4003c10 con 0x7fbeb809e670 2023-09-30T22:21:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbeaeffd700 1 -- 172.21.15.89:0/2834355604 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb401f070 con 0x7fbeb809e670 2023-09-30T22:21:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbeaeffd700 1 -- 172.21.15.89:0/2834355604 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400b660 con 0x7fbeb809e670 2023-09-30T22:21:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29baffd700 1 -- 172.21.15.89:0/2606830257 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0009710 con 0x7f29c409e930 2023-09-30T22:21:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29baffd700 1 -- 172.21.15.89:0/2606830257 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c000c850 con 0x7f29c409e930 2023-09-30T22:21:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29baffd700 1 -- 172.21.15.89:0/2606830257 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c000cbc0 con 0x7f29c409e930 2023-09-30T22:21:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb806cec0 con 0x7fbeb809e670 2023-09-30T22:21:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb81243f0 con 0x7fbeb809e670 2023-09-30T22:21:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.173+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb8125f40 con 0x7fbeb809e670 2023-09-30T22:21:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c40a0190 con 0x7f29c409e930 2023-09-30T22:21:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.172+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c406d430 con 0x7f29c409e930 2023-09-30T22:21:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.174+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c41a8c90 con 0x7f29c409e930 2023-09-30T22:21:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.174+0000 7f29baffd700 1 -- 172.21.15.89:0/2606830257 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f29c000cd20 con 0x7f29c409e930 2023-09-30T22:21:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.174+0000 7f29baffd700 1 -- 172.21.15.89:0/2606830257 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f29c003fbe0 con 0x7f29c409e930 2023-09-30T22:21:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.174+0000 7fbeaeffd700 1 -- 172.21.15.89:0/2834355604 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbeb4021030 con 0x7fbeb809e670 2023-09-30T22:21:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.174+0000 7fbeaeffd700 1 -- 172.21.15.89:0/2834355604 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbeb4041150 con 0x7fbeb809e670 2023-09-30T22:21:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.179+0000 7fbeaeffd700 1 -- 172.21.15.89:0/2834355604 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbeb400b7c0 con 0x7fbeb809e670 2023-09-30T22:21:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.179+0000 7f29baffd700 1 -- 172.21.15.89:0/2606830257 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29c001b3b0 con 0x7f29c409e930 2023-09-30T22:21:48.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4799d9d700 1 Processor -- start 2023-09-30T22:21:48.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4799d9d700 1 -- start start 2023-09-30T22:21:48.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4799d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 0x7f4794098090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4799d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f479409dd90 0x7f4794139570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4799d9d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479409bd10 con 0x7f479409d230 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4799d9d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4794096630 con 0x7f479409dd90 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4799d9d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47940985d0 con 0x7f4794097cc0 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4793fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f479409d230 0x7f479409d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38268/0 (socket says 172.21.15.89:38268) 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.212+0000 7f4793fff700 1 -- 172.21.15.89:0/1485283170 learned_addr learned my addr 172.21.15.89:0/1485283170 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518167012 0 0) 0x7f479409bd10 con 0x7f479409d230 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47840035b0 con 0x7f479409d230 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f4798d9b700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 0x7f4794098090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373356424 0 0) 0x7f47840035b0 con 0x7f479409d230 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 msgr2=0x7f4794098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 0x7f4794098090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f479409dd90 msgr2=0x7f4794139570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f479409dd90 0x7f4794139570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479409e690 con 0x7f479409d230 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7f4780001670 con 0x7f479409d230 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 >> v1:172.21.15.89:6789/0 conn(0x7f479409d230 legacy=0x7f479409d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f479409dd90 0x7f4784003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4784003fb0 0x7f4784004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 0x7f4784004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47940985d0 con 0x7f4784003fb0 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47840053a0 con 0x7f479409dd90 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4784005510 con 0x7f4794097cc0 2023-09-30T22:21:48.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f4793fff700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4784003fb0 0x7f4784004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.213+0000 7f479959c700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 0x7f4784004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f4793fff700 1 -- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 msgr2=0x7f4784004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f4793fff700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 0x7f4784004d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f4793fff700 1 -- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f479409dd90 msgr2=0x7f4784003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f4793fff700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f479409dd90 0x7f4784003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f4793fff700 1 -- 172.21.15.89:0/1485283170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479409e690 con 0x7f4784003fb0 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f4793fff700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4784003fb0 0x7f4784004850 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4780012bf0 tx=0x7f4784005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a96ab2cc95d84b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780017070 con 0x7f4784003fb0 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4780005960 con 0x7f4784003fb0 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.214+0000 7f47937fe700 1 -- 172.21.15.89:0/1485283170 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780005cf0 con 0x7f4784003fb0 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 -- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4784003fb0 msgr2=0x7f4784004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4784003fb0 0x7f4784004850 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4780012bf0 tx=0x7f4784005880 comp rx=0 tx=0).stop 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 -- 172.21.15.89:0/1485283170 shutdown_connections 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4784003fb0 0x7f4784004850 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f479409dd90 0x7f4784003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 --2- 172.21.15.89:0/1485283170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4794097cc0 0x7f4784004d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 -- 172.21.15.89:0/1485283170 >> 172.21.15.89:0/1485283170 conn(0x7f47940a2540 msgr2=0x7f47940a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 -- 172.21.15.89:0/1485283170 shutdown_connections 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.215+0000 7f4799d9d700 1 -- 172.21.15.89:0/1485283170 wait complete. 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.216+0000 7f4799d9d700 1 Processor -- start 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.216+0000 7f4799d9d700 1 -- start start 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.217+0000 7f4799d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4784003fb0 0x7f47941ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.217+0000 7f4799d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4794097cc0 0x7f47941ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.217+0000 7f4799d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f479409d230 0x7f47941acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:48.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.217+0000 7f4799d9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794098820 con 0x7f4794097cc0 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.217+0000 7f4799d9d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47941ad800 con 0x7f4784003fb0 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.217+0000 7f4799d9d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47941aeee0 con 0x7f479409d230 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.217+0000 7f4793fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4794097cc0 0x7f47941ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.217+0000 7f479959c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f479409d230 0x7f47941acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4793fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4794097cc0 0x7f47941ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46530/0 (socket says 172.21.15.89:46530) 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4793fff700 1 -- 172.21.15.89:0/3858916402 learned_addr learned my addr 172.21.15.89:0/3858916402 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f479959c700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f479409d230 0x7f47941acc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57926/0 (socket says 172.21.15.89:57926) 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4793fff700 1 -- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f479409d230 msgr2=0x7f47941acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4793fff700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f479409d230 0x7f47941acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4793fff700 1 -- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4784003fb0 msgr2=0x7f47941ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4793fff700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4784003fb0 0x7f47941ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4793fff700 1 -- 172.21.15.89:0/3858916402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47941ae2c0 con 0x7f4794097cc0 2023-09-30T22:21:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f479959c700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f479409d230 0x7f47941acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4793fff700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4794097cc0 0x7f47941ac6d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4780014330 tx=0x7f4780001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f47917fa700 1 -- 172.21.15.89:0/3858916402 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780004a80 con 0x7f4794097cc0 2023-09-30T22:21:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f47917fa700 1 -- 172.21.15.89:0/3858916402 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4780004be0 con 0x7f4794097cc0 2023-09-30T22:21:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4799d9d700 1 -- 172.21.15.89:0/3858916402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47941aec20 con 0x7f4794097cc0 2023-09-30T22:21:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f47917fa700 1 -- 172.21.15.89:0/3858916402 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47800120d0 con 0x7f4794097cc0 2023-09-30T22:21:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.218+0000 7f4799d9d700 1 -- 172.21.15.89:0/3858916402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47941ad2c0 con 0x7f4794097cc0 2023-09-30T22:21:48.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.219+0000 7f477effd700 1 -- 172.21.15.89:0/3858916402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c0052e0 con 0x7f4794097cc0 2023-09-30T22:21:48.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.220+0000 7f47917fa700 1 -- 172.21.15.89:0/3858916402 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4780012230 con 0x7f4794097cc0 2023-09-30T22:21:48.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.220+0000 7f47917fa700 1 -- 172.21.15.89:0/3858916402 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4780048a90 con 0x7f4794097cc0 2023-09-30T22:21:48.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.224+0000 7f47917fa700 1 -- 172.21.15.89:0/3858916402 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f478001ca10 con 0x7f4794097cc0 2023-09-30T22:21:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.336+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9070120150 con 0x7f9070133e30 2023-09-30T22:21:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.336+0000 7f905effd700 1 -- 172.21.15.89:0/1232112820 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9070120150 con 0x7f9070133e30 2023-09-30T22:21:48.336 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.338+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9070133e30 msgr2=0x7f9070070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.339+0000 7f90748aa700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9070133e30 0x7f9070070ee0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f907009f2c0 tx=0x7f906400c730 comp rx=0 tx=0).stop 2023-09-30T22:21:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.339+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 shutdown_connections 2023-09-30T22:21:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.339+0000 7f90748aa700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9070134260 0x7f9070071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.339+0000 7f90748aa700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9070133420 0x7f90700709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.339+0000 7f90748aa700 1 --2- 172.21.15.89:0/1232112820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9070133e30 0x7f9070070ee0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.339+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 >> 172.21.15.89:0/1232112820 conn(0x7f90700a4210 msgr2=0x7f90700a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.339+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 shutdown_connections 2023-09-30T22:21:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.339+0000 7f90748aa700 1 -- 172.21.15.89:0/1232112820 wait complete. 2023-09-30T22:21:48.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.377+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbeb8125f40 con 0x7fbeb809e670 2023-09-30T22:21:48.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.377+0000 7fbeaeffd700 1 -- 172.21.15.89:0/2834355604 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbeb40233d0 con 0x7fbeb809e670 2023-09-30T22:21:48.377 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:48.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeb809e670 msgr2=0x7fbeb8122880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeb809e670 0x7fbeb8122880 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fbeb806b9c0 tx=0x7fbeb4004750 comp rx=0 tx=0).stop 2023-09-30T22:21:48.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 shutdown_connections 2023-09-30T22:21:48.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbeb8134070 0x7fbeb8123300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbeb809e670 0x7fbeb8122880 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 --2- 172.21.15.89:0/2834355604 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbeb809ef80 0x7fbeb8122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 >> 172.21.15.89:0/2834355604 conn(0x7fbeb80a0c90 msgr2=0x7fbeb809ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 shutdown_connections 2023-09-30T22:21:48.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7fbebf19e700 1 -- 172.21.15.89:0/2834355604 wait complete. 2023-09-30T22:21:48.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.379+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f29c41a0b30 con 0x7f29c409e930 2023-09-30T22:21:48.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.380+0000 7f29baffd700 1 -- 172.21.15.89:0/2606830257 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29c0017020 con 0x7f29c409e930 2023-09-30T22:21:48.381 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409e930 msgr2=0x7f29c406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409e930 0x7f29c406b8b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f29c000e9b0 tx=0x7f29c000ed70 comp rx=0 tx=0).stop 2023-09-30T22:21:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 shutdown_connections 2023-09-30T22:21:48.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29c409f650 0x7f29c406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29c409e930 0x7f29c406b8b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 --2- 172.21.15.89:0/2606830257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29c409f240 0x7f29c406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 >> 172.21.15.89:0/2606830257 conn(0x7f29c40a4210 msgr2=0x7f29c41b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 shutdown_connections 2023-09-30T22:21:48.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.382+0000 7f29cc433700 1 -- 172.21.15.89:0/2606830257 wait complete. 2023-09-30T22:21:48.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:48.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:48.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.423+0000 7f477effd700 1 -- 172.21.15.89:0/3858916402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f476c005860 con 0x7f4794097cc0 2023-09-30T22:21:48.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.423+0000 7f47917fa700 1 -- 172.21.15.89:0/3858916402 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47800145c0 con 0x7f4794097cc0 2023-09-30T22:21:48.423 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:48.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 -- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4794097cc0 msgr2=0x7f47941ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4794097cc0 0x7f47941ac6d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4780014330 tx=0x7f4780001a00 comp rx=0 tx=0).stop 2023-09-30T22:21:48.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 -- 172.21.15.89:0/3858916402 shutdown_connections 2023-09-30T22:21:48.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f479409d230 0x7f47941acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4784003fb0 0x7f47941ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 --2- 172.21.15.89:0/3858916402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4794097cc0 0x7f47941ac6d0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:48.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 -- 172.21.15.89:0/3858916402 >> 172.21.15.89:0/3858916402 conn(0x7f47940a2540 msgr2=0x7f47941b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 -- 172.21.15.89:0/3858916402 shutdown_connections 2023-09-30T22:21:48.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:48.425+0000 7f477effd700 1 -- 172.21.15.89:0/3858916402 wait complete. 2023-09-30T22:21:48.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:49.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:49.393 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:49.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:49.438 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:49.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.493+0000 7f6e8bb1a700 1 Processor -- start 2023-09-30T22:21:49.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.494+0000 7f6e8bb1a700 1 -- start start 2023-09-30T22:21:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.494+0000 7f6e8bb1a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409d6f0 0x7f6e8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.494+0000 7f6e8bb1a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409e410 0x7f6e84134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.494+0000 7f6e8bb1a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e8409b120 con 0x7f6e8409e000 2023-09-30T22:21:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.494+0000 7f6e8bb1a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8406b460 con 0x7f6e8409e410 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.494+0000 7f6e8bb1a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8409a7e0 con 0x7f6e8409d6f0 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e898b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409d6f0 0x7f6e8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e898b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409d6f0 0x7f6e8409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57938/0 (socket says 172.21.15.89:57938) 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e898b6700 1 -- 172.21.15.89:0/3340451643 learned_addr learned my addr 172.21.15.89:0/3340451643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e8a0b7700 1 --2- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409e410 0x7f6e84134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e888b4700 1 -- 172.21.15.89:0/3340451643 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004278688 0 0) 0x7f6e8409b120 con 0x7f6e8409e000 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e888b4700 1 -- 172.21.15.89:0/3340451643 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e700035b0 con 0x7f6e8409e000 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e898b6700 1 -- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409e410 msgr2=0x7f6e84134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e898b6700 1 --2- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409e410 0x7f6e84134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e898b6700 1 -- 172.21.15.89:0/3340451643 >> v1:172.21.15.89:6789/0 conn(0x7f6e8409e000 legacy=0x7f6e841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e898b6700 1 -- 172.21.15.89:0/3340451643 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8409a500 con 0x7f6e8409d6f0 2023-09-30T22:21:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e8a0b7700 1 --2- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409e410 0x7f6e84134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.495+0000 7f6e898b6700 1 --2- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409d6f0 0x7f6e8409dac0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009950 tx=0x7f6e80009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a834223acc988e22 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e888b4700 1 -- 172.21.15.89:0/3340451643 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80011830 con 0x7f6e8409d6f0 2023-09-30T22:21:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e888b4700 1 -- 172.21.15.89:0/3340451643 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80011990 con 0x7f6e8409d6f0 2023-09-30T22:21:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e888b4700 1 -- 172.21.15.89:0/3340451643 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80011d40 con 0x7f6e8409d6f0 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409d6f0 msgr2=0x7f6e8409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e8bb1a700 1 --2- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409d6f0 0x7f6e8409dac0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009950 tx=0x7f6e80009c60 comp rx=0 tx=0).stop 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/3340451643 shutdown_connections 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e8bb1a700 1 --2- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409e410 0x7f6e84134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e8bb1a700 1 --2- 172.21.15.89:0/3340451643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409d6f0 0x7f6e8409dac0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.496+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/3340451643 >> 172.21.15.89:0/3340451643 conn(0x7f6e84066f50 msgr2=0x7f6e8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.497+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/3340451643 shutdown_connections 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.497+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/3340451643 wait complete. 2023-09-30T22:21:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.498+0000 7f6e8bb1a700 1 Processor -- start 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.498+0000 7f6e8bb1a700 1 -- start start 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.498+0000 7f6e8bb1a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409d6f0 0x7f6e841224f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.498+0000 7f6e8bb1a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409e000 0x7f6e84122a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.498+0000 7f6e8bb1a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e8409e410 0x7f6e84122f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.498+0000 7f6e8bb1a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8409c710 con 0x7f6e8409e410 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.498+0000 7f6e8bb1a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840a1aa0 con 0x7f6e8409d6f0 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.498+0000 7f6e8bb1a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84124f20 con 0x7f6e8409e000 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409e000 0x7f6e84122a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409e000 0x7f6e84122a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57946/0 (socket says 172.21.15.89:57946) 2023-09-30T22:21:49.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 -- 172.21.15.89:0/4120332315 learned_addr learned my addr 172.21.15.89:0/4120332315 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e898b6700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409d6f0 0x7f6e841224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 -- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409d6f0 msgr2=0x7f6e841224f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409d6f0 0x7f6e841224f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 -- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e8409e410 msgr2=0x7f6e84122f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e8409e410 0x7f6e84122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 -- 172.21.15.89:0/4120332315 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c009a70 con 0x7f6e8409e000 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e898b6700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409d6f0 0x7f6e841224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e890b5700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409e000 0x7f6e84122a30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c4f0 tx=0x7f6e7400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e7a7fc700 1 -- 172.21.15.89:0/4120332315 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740052d0 con 0x7f6e8409e000 2023-09-30T22:21:49.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e7a7fc700 1 -- 172.21.15.89:0/4120332315 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e74005430 con 0x7f6e8409e000 2023-09-30T22:21:49.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.499+0000 7f6e7a7fc700 1 -- 172.21.15.89:0/4120332315 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74003880 con 0x7f6e8409e000 2023-09-30T22:21:49.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.500+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e80009550 con 0x7f6e8409e000 2023-09-30T22:21:49.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.500+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e84125d30 con 0x7f6e8409e000 2023-09-30T22:21:49.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.501+0000 7f6e7a7fc700 1 -- 172.21.15.89:0/4120332315 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e740039e0 con 0x7f6e8409e000 2023-09-30T22:21:49.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.501+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8411f030 con 0x7f6e8409e000 2023-09-30T22:21:49.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.501+0000 7f6e7a7fc700 1 -- 172.21.15.89:0/4120332315 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6e74010cb0 con 0x7f6e8409e000 2023-09-30T22:21:49.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.506+0000 7f6e7a7fc700 1 -- 172.21.15.89:0/4120332315 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e74017030 con 0x7f6e8409e000 2023-09-30T22:21:49.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.532+0000 7f681863e700 1 Processor -- start 2023-09-30T22:21:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.532+0000 7f681863e700 1 -- start start 2023-09-30T22:21:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.532+0000 7f567c0e2700 1 Processor -- start 2023-09-30T22:21:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.532+0000 7f567c0e2700 1 -- start start 2023-09-30T22:21:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567c0e2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56741338c0 0x7f5674133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567c0e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741341d0 0x7f567409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567c0e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f567409e390 0x7f567409e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567c0e2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5674070c80 con 0x7f56741338c0 2023-09-30T22:21:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567c0e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567409ed20 con 0x7f56741341d0 2023-09-30T22:21:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567c0e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f567406f720 con 0x7f567409e390 2023-09-30T22:21:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f681863e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810133540 0x7f6810135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f681863e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6810135e60 0x7f6810138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f681863e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68100716a0 con 0x7f6810130540 2023-09-30T22:21:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f681863e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6810071290 con 0x7f6810135e60 2023-09-30T22:21:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f681863e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6810130340 con 0x7f6810133540 2023-09-30T22:21:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567a67f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f567409e390 0x7f567409e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f6815bd9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810133540 0x7f6810135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567a67f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f567409e390 0x7f567409e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57964/0 (socket says 172.21.15.89:57964) 2023-09-30T22:21:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567a67f700 1 -- 172.21.15.89:0/3384641071 learned_addr learned my addr 172.21.15.89:0/3384641071 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f6815bd9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810133540 0x7f6810135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57952/0 (socket says 172.21.15.89:57952) 2023-09-30T22:21:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f6815bd9700 1 -- 172.21.15.89:0/805382583 learned_addr learned my addr 172.21.15.89:0/805382583 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567967d700 1 --2- 172.21.15.89:0/3384641071 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741341d0 0x7f567409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f6815bd9700 1 -- 172.21.15.89:0/805382583 >> v2:172.21.15.89:3300/0 conn(0x7f6810135e60 msgr2=0x7f6810138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f6815bd9700 1 --2- 172.21.15.89:0/805382583 >> v2:172.21.15.89:3300/0 conn(0x7f6810135e60 0x7f6810138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f6815bd9700 1 -- 172.21.15.89:0/805382583 >> v1:172.21.15.89:6789/0 conn(0x7f6810130540 legacy=0x7f6810130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f6815bd9700 1 -- 172.21.15.89:0/805382583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6810070b80 con 0x7f6810133540 2023-09-30T22:21:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567a67f700 1 -- 172.21.15.89:0/3384641071 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741341d0 msgr2=0x7f567409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567a67f700 1 --2- 172.21.15.89:0/3384641071 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741341d0 0x7f567409de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567a67f700 1 -- 172.21.15.89:0/3384641071 >> v2:172.21.15.89:3300/0 conn(0x7f56741338c0 msgr2=0x7f5674133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567a67f700 1 --2- 172.21.15.89:0/3384641071 >> v2:172.21.15.89:3300/0 conn(0x7f56741338c0 0x7f5674133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.533+0000 7f567a67f700 1 -- 172.21.15.89:0/3384641071 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567406fce0 con 0x7f567409e390 2023-09-30T22:21:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.534+0000 7f6815bd9700 1 --2- 172.21.15.89:0/805382583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810133540 0x7f6810135920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f68000098f0 tx=0x7f6800009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5f5e8820f19c20 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.534+0000 7f567a67f700 1 --2- 172.21.15.89:0/3384641071 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f567409e390 0x7f567409e7e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f566c009e80 tx=0x7f566c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d302613f395b5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.534+0000 7f68153d8700 1 -- 172.21.15.89:0/805382583 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800004230 con 0x7f6810133540 2023-09-30T22:21:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.534+0000 7f68153d8700 1 -- 172.21.15.89:0/805382583 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6800004390 con 0x7f6810133540 2023-09-30T22:21:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.534+0000 7f68153d8700 1 -- 172.21.15.89:0/805382583 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68000036a0 con 0x7f6810133540 2023-09-30T22:21:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.534+0000 7f5678e7c700 1 -- 172.21.15.89:0/3384641071 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00e070 con 0x7f567409e390 2023-09-30T22:21:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.534+0000 7f5678e7c700 1 -- 172.21.15.89:0/3384641071 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c00b8d0 con 0x7f567409e390 2023-09-30T22:21:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.534+0000 7f5678e7c700 1 -- 172.21.15.89:0/3384641071 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00bc80 con 0x7f567409e390 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 -- 172.21.15.89:0/3384641071 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f567409e390 msgr2=0x7f567409e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 --2- 172.21.15.89:0/3384641071 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f567409e390 0x7f567409e7e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f566c009e80 tx=0x7f566c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 -- 172.21.15.89:0/3384641071 shutdown_connections 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 --2- 172.21.15.89:0/3384641071 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f567409e390 0x7f567409e7e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 --2- 172.21.15.89:0/3384641071 >> v2:172.21.15.89:3300/0 conn(0x7f56741338c0 0x7f5674133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 --2- 172.21.15.89:0/3384641071 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741341d0 0x7f567409de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 -- 172.21.15.89:0/3384641071 >> 172.21.15.89:0/3384641071 conn(0x7f56740a2aa0 msgr2=0x7f56740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 -- 172.21.15.89:0/3384641071 shutdown_connections 2023-09-30T22:21:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f567c0e2700 1 -- 172.21.15.89:0/3384641071 wait complete. 2023-09-30T22:21:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f681863e700 1 -- 172.21.15.89:0/805382583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810133540 msgr2=0x7f6810135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f681863e700 1 --2- 172.21.15.89:0/805382583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810133540 0x7f6810135920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f68000098f0 tx=0x7f6800009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f681863e700 1 -- 172.21.15.89:0/805382583 shutdown_connections 2023-09-30T22:21:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f681863e700 1 --2- 172.21.15.89:0/805382583 >> v2:172.21.15.89:3300/0 conn(0x7f6810135e60 0x7f6810138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f681863e700 1 --2- 172.21.15.89:0/805382583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810133540 0x7f6810135920 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.535+0000 7f681863e700 1 -- 172.21.15.89:0/805382583 >> 172.21.15.89:0/805382583 conn(0x7f68100a2aa0 msgr2=0x7f68100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.536+0000 7f681863e700 1 -- 172.21.15.89:0/805382583 shutdown_connections 2023-09-30T22:21:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.536+0000 7f681863e700 1 -- 172.21.15.89:0/805382583 wait complete. 2023-09-30T22:21:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.536+0000 7f567c0e2700 1 Processor -- start 2023-09-30T22:21:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.536+0000 7f567c0e2700 1 -- start start 2023-09-30T22:21:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567c0e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f567409e390 0x7f567411ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567c0e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741338c0 0x7f567411ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567c0e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56741341d0 0x7f567411f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567c0e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56741224b0 con 0x7f567409e390 2023-09-30T22:21:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567c0e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567406f960 con 0x7f56741338c0 2023-09-30T22:21:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567c0e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56741205a0 con 0x7f56741341d0 2023-09-30T22:21:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f681863e700 1 Processor -- start 2023-09-30T22:21:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f681863e700 1 -- start start 2023-09-30T22:21:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567a67f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56741341d0 0x7f567411f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567a67f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56741341d0 0x7f567411f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57974/0 (socket says 172.21.15.89:57974) 2023-09-30T22:21:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567a67f700 1 -- 172.21.15.89:0/360768856 learned_addr learned my addr 172.21.15.89:0/360768856 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567967d700 1 --2- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741338c0 0x7f567411ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567a67f700 1 -- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741338c0 msgr2=0x7f567411ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567a67f700 1 --2- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741338c0 0x7f567411ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567a67f700 1 -- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f567409e390 msgr2=0x7f567411ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567a67f700 1 --2- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f567409e390 0x7f567411ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f567a67f700 1 -- 172.21.15.89:0/360768856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c009b30 con 0x7f56741341d0 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f681863e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810130540 0x7f681009c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f681863e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6810133540 0x7f681009c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f681863e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6810135e60 0x7f681009cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f681863e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6810137370 con 0x7f6810135e60 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f681863e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6810136c00 con 0x7f6810133540 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.537+0000 7f681863e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6810137550 con 0x7f6810130540 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f68163da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810130540 0x7f681009c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f68163da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810130540 0x7f681009c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57986/0 (socket says 172.21.15.89:57986) 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f68163da700 1 -- 172.21.15.89:0/743127618 learned_addr learned my addr 172.21.15.89:0/743127618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6815bd9700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6810133540 0x7f681009c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6816bdb700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6810135e60 0x7f681009cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f567a67f700 1 --2- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56741341d0 0x7f567411f4a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f567406f8b0 tx=0x7f566c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6815bd9700 1 -- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810130540 msgr2=0x7f681009c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6815bd9700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810130540 0x7f681009c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6815bd9700 1 -- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6810135e60 msgr2=0x7f681009cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6815bd9700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6810135e60 0x7f681009cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6815bd9700 1 -- 172.21.15.89:0/743127618 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009550 con 0x7f6810133540 2023-09-30T22:21:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6815bd9700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6810133540 0x7f681009c680 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6800015000 tx=0x7f6800004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f566a7fc700 1 -- 172.21.15.89:0/360768856 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0099b0 con 0x7f56741341d0 2023-09-30T22:21:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5674120dc0 con 0x7f56741341d0 2023-09-30T22:21:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f566a7fc700 1 -- 172.21.15.89:0/360768856 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c004250 con 0x7f56741341d0 2023-09-30T22:21:49.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f566a7fc700 1 -- 172.21.15.89:0/360768856 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0045e0 con 0x7f56741341d0 2023-09-30T22:21:49.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6806ffd700 1 -- 172.21.15.89:0/743127618 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000b660 con 0x7f6810133540 2023-09-30T22:21:49.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6806ffd700 1 -- 172.21.15.89:0/743127618 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680000b7c0 con 0x7f6810133540 2023-09-30T22:21:49.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f6806ffd700 1 -- 172.21.15.89:0/743127618 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000bb70 con 0x7f6810133540 2023-09-30T22:21:49.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68100a2360 con 0x7f6810133540 2023-09-30T22:21:49.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.539+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6810137830 con 0x7f6810133540 2023-09-30T22:21:49.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.538+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567412f8e0 con 0x7f56741341d0 2023-09-30T22:21:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.539+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567412f510 con 0x7f56741341d0 2023-09-30T22:21:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.540+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f80052e0 con 0x7f6810133540 2023-09-30T22:21:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.540+0000 7f6806ffd700 1 -- 172.21.15.89:0/743127618 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6800021070 con 0x7f6810133540 2023-09-30T22:21:49.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.540+0000 7f6806ffd700 1 -- 172.21.15.89:0/743127618 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6800040cd0 con 0x7f6810133540 2023-09-30T22:21:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.541+0000 7f566a7fc700 1 -- 172.21.15.89:0/360768856 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f566c004740 con 0x7f56741341d0 2023-09-30T22:21:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.541+0000 7f566a7fc700 1 -- 172.21.15.89:0/360768856 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f566c040e00 con 0x7f56741341d0 2023-09-30T22:21:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.545+0000 7f6806ffd700 1 -- 172.21.15.89:0/743127618 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68000193c0 con 0x7f6810133540 2023-09-30T22:21:49.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.549+0000 7f566a7fc700 1 -- 172.21.15.89:0/360768856 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f566c0117a0 con 0x7f56741341d0 2023-09-30T22:21:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.576+0000 7fda32748700 1 Processor -- start 2023-09-30T22:21:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.576+0000 7fda32748700 1 -- start start 2023-09-30T22:21:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.576+0000 7fda32748700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.576+0000 7fda32748700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda2c0a0090 0x7fda2c09cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.576+0000 7fda32748700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c09ba70 con 0x7fda2c131970 2023-09-30T22:21:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.576+0000 7fda32748700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c09d4a0 con 0x7fda2c0a0090 2023-09-30T22:21:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda32748700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c09e8f0 con 0x7fda2c09c650 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda2c0a0090 0x7fda2c09cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda2c0a0090 0x7fda2c09cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46582/0 (socket says 172.21.15.89:46582) 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2b7fe700 1 -- 172.21.15.89:0/3078178305 learned_addr learned my addr 172.21.15.89:0/3078178305 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2bfff700 1 --2- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2b7fe700 1 -- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 msgr2=0x7fda2c09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2b7fe700 1 --2- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c09ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2b7fe700 1 -- 172.21.15.89:0/3078178305 >> v1:172.21.15.89:6789/0 conn(0x7fda2c131970 legacy=0x7fda2c131dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2b7fe700 1 -- 172.21.15.89:0/3078178305 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c0a0760 con 0x7fda2c0a0090 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2bfff700 1 --2- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2b7fe700 1 --2- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c0a0090 0x7fda2c09cf60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fda200098f0 tx=0x7fda20009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8615ee677b583580 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2affd700 1 -- 172.21.15.89:0/3078178305 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20011830 con 0x7fda2c0a0090 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2affd700 1 -- 172.21.15.89:0/3078178305 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20011990 con 0x7fda2c0a0090 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.577+0000 7fda2affd700 1 -- 172.21.15.89:0/3078178305 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20011d20 con 0x7fda2c0a0090 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.578+0000 7fda32748700 1 -- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c0a0090 msgr2=0x7fda2c09cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.578+0000 7fda32748700 1 --2- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c0a0090 0x7fda2c09cf60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fda200098f0 tx=0x7fda20009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.578+0000 7fda32748700 1 -- 172.21.15.89:0/3078178305 shutdown_connections 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.578+0000 7fda32748700 1 --2- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.578+0000 7fda32748700 1 --2- 172.21.15.89:0/3078178305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c0a0090 0x7fda2c09cf60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.578+0000 7fda32748700 1 -- 172.21.15.89:0/3078178305 >> 172.21.15.89:0/3078178305 conn(0x7fda2c0a3f70 msgr2=0x7fda2c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.578+0000 7fda32748700 1 -- 172.21.15.89:0/3078178305 shutdown_connections 2023-09-30T22:21:49.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.578+0000 7fda32748700 1 -- 172.21.15.89:0/3078178305 wait complete. 2023-09-30T22:21:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.580+0000 7fda32748700 1 Processor -- start 2023-09-30T22:21:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.580+0000 7fda32748700 1 -- start start 2023-09-30T22:21:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.580+0000 7fda32748700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c11bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.580+0000 7fda32748700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2c0a0090 0x7fda2c11c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.580+0000 7fda32748700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c131970 0x7fda2c11c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.580+0000 7fda32748700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c11f1c0 con 0x7fda2c131970 2023-09-30T22:21:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.580+0000 7fda32748700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c09d980 con 0x7fda2c0a0090 2023-09-30T22:21:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.580+0000 7fda32748700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c12ff20 con 0x7fda2c09c650 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c11bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c11bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58000/0 (socket says 172.21.15.89:58000) 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda30ce5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c131970 0x7fda2c11c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda2bfff700 1 -- 172.21.15.89:0/1409759220 learned_addr learned my addr 172.21.15.89:0/1409759220 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda2b7fe700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2c0a0090 0x7fda2c11c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda30ce5700 1 -- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 msgr2=0x7fda2c11bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda30ce5700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c11bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda30ce5700 1 -- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2c0a0090 msgr2=0x7fda2c11c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda30ce5700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2c0a0090 0x7fda2c11c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda30ce5700 1 -- 172.21.15.89:0/1409759220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c00d940 con 0x7fda2c131970 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda2bfff700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c11bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda30ce5700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c131970 0x7fda2c11c990 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fda2400cb90 tx=0x7fda2400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda28ff9700 1 -- 172.21.15.89:0/1409759220 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24003c00 con 0x7fda2c131970 2023-09-30T22:21:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda28ff9700 1 -- 172.21.15.89:0/1409759220 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda24003d60 con 0x7fda2c131970 2023-09-30T22:21:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda28ff9700 1 -- 172.21.15.89:0/1409759220 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400aaa0 con 0x7fda2c131970 2023-09-30T22:21:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.581+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda20009550 con 0x7fda2c131970 2023-09-30T22:21:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.582+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c11ed70 con 0x7fda2c131970 2023-09-30T22:21:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.583+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c0747f0 con 0x7fda2c131970 2023-09-30T22:21:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.583+0000 7fda28ff9700 1 -- 172.21.15.89:0/1409759220 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda2400ac00 con 0x7fda2c131970 2023-09-30T22:21:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.583+0000 7fda28ff9700 1 -- 172.21.15.89:0/1409759220 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fda24040140 con 0x7fda2c131970 2023-09-30T22:21:49.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.588+0000 7fda28ff9700 1 -- 172.21.15.89:0/1409759220 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda240050d0 con 0x7fda2c131970 2023-09-30T22:21:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.709+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6e84074ea0 con 0x7f6e8409e000 2023-09-30T22:21:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.709+0000 7f6e7a7fc700 1 -- 172.21.15.89:0/4120332315 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e74044060 con 0x7f6e8409e000 2023-09-30T22:21:49.709 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:49.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.712+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409e000 msgr2=0x7f6e84122a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.712+0000 7f6e8bb1a700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409e000 0x7f6e84122a30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c4f0 tx=0x7f6e7400c8b0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.713+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 shutdown_connections 2023-09-30T22:21:49.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.713+0000 7f6e8bb1a700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e8409e410 0x7f6e84122f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.713+0000 7f6e8bb1a700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e8409d6f0 0x7f6e841224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.713+0000 7f6e8bb1a700 1 --2- 172.21.15.89:0/4120332315 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e8409e000 0x7f6e84122a30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.713+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 >> 172.21.15.89:0/4120332315 conn(0x7f6e84066f50 msgr2=0x7f6e84133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.713+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 shutdown_connections 2023-09-30T22:21:49.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.713+0000 7f6e8bb1a700 1 -- 172.21.15.89:0/4120332315 wait complete. 2023-09-30T22:21:49.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:49.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.779+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f67f8005590 con 0x7f6810133540 2023-09-30T22:21:49.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.779+0000 7f6806ffd700 1 -- 172.21.15.89:0/743127618 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6800029070 con 0x7f6810133540 2023-09-30T22:21:49.779 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:49.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.781+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fda2c072840 con 0x7fda2c131970 2023-09-30T22:21:49.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.782+0000 7fda28ff9700 1 -- 172.21.15.89:0/1409759220 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda24029090 con 0x7fda2c131970 2023-09-30T22:21:49.782 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:49.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.782+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6810133540 msgr2=0x7f681009c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.782+0000 7f681863e700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6810133540 0x7f681009c680 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6800015000 tx=0x7f6800004750 comp rx=0 tx=0).stop 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.782+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 shutdown_connections 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.782+0000 7f681863e700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6810135e60 0x7f681009cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.782+0000 7f681863e700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6810130540 0x7f681009c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.782+0000 7f681863e700 1 --2- 172.21.15.89:0/743127618 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6810133540 0x7f681009c680 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.782+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 >> 172.21.15.89:0/743127618 conn(0x7f68100a2aa0 msgr2=0x7f6810135170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.783+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 shutdown_connections 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.783+0000 7f681863e700 1 -- 172.21.15.89:0/743127618 wait complete. 2023-09-30T22:21:49.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c131970 msgr2=0x7fda2c11c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c131970 0x7fda2c11c990 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fda2400cb90 tx=0x7fda2400cea0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 shutdown_connections 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda2c131970 0x7fda2c11c990 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda2c09c650 0x7fda2c11bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 --2- 172.21.15.89:0/1409759220 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2c0a0090 0x7fda2c11c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 >> 172.21.15.89:0/1409759220 conn(0x7fda2c0a3f70 msgr2=0x7fda2c0a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 shutdown_connections 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.784+0000 7fda32748700 1 -- 172.21.15.89:0/1409759220 wait complete. 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.786+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5674074ea0 con 0x7f56741341d0 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.787+0000 7f566a7fc700 1 -- 172.21.15.89:0/360768856 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f566c032090 con 0x7f56741341d0 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56741341d0 msgr2=0x7f567411f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 --2- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56741341d0 0x7f567411f4a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f567406f8b0 tx=0x7f566c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 shutdown_connections 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 --2- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f56741341d0 0x7f567411f4a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 --2- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f567409e390 0x7f567411ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 --2- 172.21.15.89:0/360768856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f56741338c0 0x7f567411ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 >> 172.21.15.89:0/360768856 conn(0x7f56740a2aa0 msgr2=0x7f567409f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 shutdown_connections 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:49.789+0000 7f567c0e2700 1 -- 172.21.15.89:0/360768856 wait complete. 2023-09-30T22:21:49.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:49.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:49.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:50.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:50.798 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:50.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:50.803 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:50.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.866+0000 7f78e4282700 1 Processor -- start 2023-09-30T22:21:50.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.866+0000 7f78e4282700 1 -- start start 2023-09-30T22:21:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.867+0000 7f78e4282700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc133420 0x7f78dc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.867+0000 7f78e4282700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78dc133e30 0x7f78dc12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.867+0000 7f78e4282700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc134260 0x7f78dc1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.867+0000 7f78e4282700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc06b990 con 0x7f78dc133e30 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.867+0000 7f78e4282700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0a0590 con 0x7f78dc133420 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.867+0000 7f78e4282700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc09fb80 con 0x7f78dc134260 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e201e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc133420 0x7f78dc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78dc133e30 0x7f78dc12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78dc133e30 0x7f78dc12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42346/0 (socket says 172.21.15.89:42346) 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e201e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc133420 0x7f78dc1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60404/0 (socket says 172.21.15.89:60404) 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 -- 172.21.15.89:0/182297189 learned_addr learned my addr 172.21.15.89:0/182297189 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:50.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e281f700 1 --2- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc134260 0x7f78dc1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 -- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc134260 msgr2=0x7f78dc1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 --2- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc134260 0x7f78dc1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 -- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc133420 msgr2=0x7f78dc1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 --2- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc133420 0x7f78dc1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 -- 172.21.15.89:0/182297189 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0a0030 con 0x7f78dc133e30 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e181d700 1 --2- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133e30 0x7f78dc12ff90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0098f0 tx=0x7f78cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0704718f4d91a66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e101c700 1 -- 172.21.15.89:0/182297189 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011830 con 0x7f78dc133e30 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e101c700 1 -- 172.21.15.89:0/182297189 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc011990 con 0x7f78dc133e30 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.868+0000 7f78e101c700 1 -- 172.21.15.89:0/182297189 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011d20 con 0x7f78dc133e30 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 -- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133e30 msgr2=0x7f78dc12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 --2- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133e30 0x7f78dc12ff90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0098f0 tx=0x7f78cc009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 -- 172.21.15.89:0/182297189 shutdown_connections 2023-09-30T22:21:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 --2- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc134260 0x7f78dc1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 --2- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc133420 0x7f78dc1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 --2- 172.21.15.89:0/182297189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133e30 0x7f78dc12ff90 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 -- 172.21.15.89:0/182297189 >> 172.21.15.89:0/182297189 conn(0x7f78dc0a4210 msgr2=0x7f78dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 -- 172.21.15.89:0/182297189 shutdown_connections 2023-09-30T22:21:50.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.869+0000 7f78e4282700 1 -- 172.21.15.89:0/182297189 wait complete. 2023-09-30T22:21:50.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e4282700 1 Processor -- start 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e4282700 1 -- start start 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e4282700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133420 0x7f78dc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e4282700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc133e30 0x7f78dc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e4282700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc134260 0x7f78dc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e4282700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc12da60 con 0x7f78dc133420 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e4282700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc11ee60 con 0x7f78dc134260 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e4282700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc11fb60 con 0x7f78dc133e30 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e201e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133420 0x7f78dc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e181d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc133e30 0x7f78dc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e201e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133420 0x7f78dc09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42362/0 (socket says 172.21.15.89:42362) 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.871+0000 7f78e201e700 1 -- 172.21.15.89:0/953185776 learned_addr learned my addr 172.21.15.89:0/953185776 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:50.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e281f700 1 --2- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc134260 0x7f78dc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e181d700 1 -- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc134260 msgr2=0x7f78dc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e181d700 1 --2- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc134260 0x7f78dc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e181d700 1 -- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133420 msgr2=0x7f78dc09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e181d700 1 --2- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133420 0x7f78dc09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e181d700 1 -- 172.21.15.89:0/953185776 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc009550 con 0x7f78dc133e30 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e181d700 1 --2- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc133e30 0x7f78dc09c9c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f78cc016000 tx=0x7f78cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78d2ffd700 1 -- 172.21.15.89:0/953185776 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc004450 con 0x7f78dc133e30 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78d2ffd700 1 -- 172.21.15.89:0/953185776 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc0045b0 con 0x7f78dc133e30 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc120190 con 0x7f78dc133e30 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78d2ffd700 1 -- 172.21.15.89:0/953185776 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0038e0 con 0x7f78dc133e30 2023-09-30T22:21:50.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.872+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc09dcb0 con 0x7f78dc133e30 2023-09-30T22:21:50.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.874+0000 7f78d2ffd700 1 -- 172.21.15.89:0/953185776 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f78cc021070 con 0x7f78dc133e30 2023-09-30T22:21:50.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.875+0000 7f78d2ffd700 1 -- 172.21.15.89:0/953185776 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f78cc040c30 con 0x7f78dc133e30 2023-09-30T22:21:50.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.875+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc09de40 con 0x7f78dc133e30 2023-09-30T22:21:50.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.879+0000 7f78d2ffd700 1 -- 172.21.15.89:0/953185776 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78cc015070 con 0x7f78dc133e30 2023-09-30T22:21:50.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.932+0000 7f928d33f700 1 Processor -- start 2023-09-30T22:21:50.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.932+0000 7fb45d939700 1 Processor -- start 2023-09-30T22:21:50.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.932+0000 7f928d33f700 1 -- start start 2023-09-30T22:21:50.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.932+0000 7fb45d939700 1 -- start start 2023-09-30T22:21:50.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f928d33f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9288133b10 0x7f9288133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb45d939700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb45d939700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb45809d630 0x7fb4580a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f928d33f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f928809e9b0 0x7f928813b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f928d33f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92881334b0 con 0x7f928809de50 2023-09-30T22:21:50.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f928d33f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9288134420 con 0x7f928809e9b0 2023-09-30T22:21:50.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f928d33f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9288070bc0 con 0x7f9288133b10 2023-09-30T22:21:50.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb45d939700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4580a2a70 con 0x7fb458133930 2023-09-30T22:21:50.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb45d939700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4580a01d0 con 0x7fb45809d630 2023-09-30T22:21:50.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb45d939700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb45809e2b0 con 0x7fb4580a0a30 2023-09-30T22:21:50.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb456ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb456ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4580a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60424/0 (socket says 172.21.15.89:60424) 2023-09-30T22:21:50.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb456ffd700 1 -- 172.21.15.89:0/1267062384 learned_addr learned my addr 172.21.15.89:0/1267062384 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f9286ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9288133b10 0x7f9288133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f9286ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9288133b10 0x7f9288133ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60440/0 (socket says 172.21.15.89:60440) 2023-09-30T22:21:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f9286ffd700 1 -- 172.21.15.89:0/955338423 learned_addr learned my addr 172.21.15.89:0/955338423 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7fb4567fc700 1 --2- 172.21.15.89:0/1267062384 >> v2:172.21.15.89:3300/0 conn(0x7fb45809d630 0x7fb4580a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7fb456ffd700 1 -- 172.21.15.89:0/1267062384 >> v2:172.21.15.89:3300/0 conn(0x7fb45809d630 msgr2=0x7fb4580a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7fb456ffd700 1 --2- 172.21.15.89:0/1267062384 >> v2:172.21.15.89:3300/0 conn(0x7fb45809d630 0x7fb4580a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.933+0000 7f92877fe700 1 --2- 172.21.15.89:0/955338423 >> v2:172.21.15.89:3300/0 conn(0x7f928809e9b0 0x7f928813b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9286ffd700 1 -- 172.21.15.89:0/955338423 >> v2:172.21.15.89:3300/0 conn(0x7f928809e9b0 msgr2=0x7f928813b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9286ffd700 1 --2- 172.21.15.89:0/955338423 >> v2:172.21.15.89:3300/0 conn(0x7f928809e9b0 0x7f928813b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9286ffd700 1 -- 172.21.15.89:0/955338423 >> v1:172.21.15.89:6789/0 conn(0x7f928809de50 legacy=0x7f928809e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9286ffd700 1 -- 172.21.15.89:0/955338423 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288070350 con 0x7f9288133b10 2023-09-30T22:21:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9285ffb700 1 -- 172.21.15.89:0/955338423 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257868817 0 0) 0x7f92881334b0 con 0x7f928809de50 2023-09-30T22:21:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7fb456ffd700 1 -- 172.21.15.89:0/1267062384 >> v1:172.21.15.89:6789/0 conn(0x7fb458133930 legacy=0x7fb458133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7fb456ffd700 1 -- 172.21.15.89:0/1267062384 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45809dc50 con 0x7fb4580a0a30 2023-09-30T22:21:50.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9286ffd700 1 --2- 172.21.15.89:0/955338423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9288133b10 0x7f9288133ee0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9278009920 tx=0x7f9278009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5529f67517723aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7fb456ffd700 1 --2- 172.21.15.89:0/1267062384 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4580a0e00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb4480098f0 tx=0x7fb448009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=501f9b1b5583413c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9285ffb700 1 -- 172.21.15.89:0/955338423 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278014e90 con 0x7f9288133b10 2023-09-30T22:21:50.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9285ffb700 1 -- 172.21.15.89:0/955338423 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927801c440 con 0x7f9288133b10 2023-09-30T22:21:50.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7f9285ffb700 1 -- 172.21.15.89:0/955338423 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927801c7d0 con 0x7f9288133b10 2023-09-30T22:21:50.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7fb455ffb700 1 -- 172.21.15.89:0/1267062384 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448011830 con 0x7fb4580a0a30 2023-09-30T22:21:50.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7fb455ffb700 1 -- 172.21.15.89:0/1267062384 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448011990 con 0x7fb4580a0a30 2023-09-30T22:21:50.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.934+0000 7fb455ffb700 1 -- 172.21.15.89:0/1267062384 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448011d20 con 0x7fb4580a0a30 2023-09-30T22:21:50.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7f928d33f700 1 -- 172.21.15.89:0/955338423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9288133b10 msgr2=0x7f9288133ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7f928d33f700 1 --2- 172.21.15.89:0/955338423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9288133b10 0x7f9288133ee0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9278009920 tx=0x7f9278009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:50.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7fb45d939700 1 -- 172.21.15.89:0/1267062384 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 msgr2=0x7fb4580a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7fb45d939700 1 --2- 172.21.15.89:0/1267062384 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4580a0e00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb4480098f0 tx=0x7fb448009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:50.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7f928d33f700 1 -- 172.21.15.89:0/955338423 shutdown_connections 2023-09-30T22:21:50.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7f928d33f700 1 --2- 172.21.15.89:0/955338423 >> v2:172.21.15.89:3300/0 conn(0x7f928809e9b0 0x7f928813b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7f928d33f700 1 --2- 172.21.15.89:0/955338423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9288133b10 0x7f9288133ee0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7f928d33f700 1 -- 172.21.15.89:0/955338423 >> 172.21.15.89:0/955338423 conn(0x7f92880a2aa0 msgr2=0x7f92880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7f928d33f700 1 -- 172.21.15.89:0/955338423 shutdown_connections 2023-09-30T22:21:50.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7f928d33f700 1 -- 172.21.15.89:0/955338423 wait complete. 2023-09-30T22:21:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7fb45d939700 1 -- 172.21.15.89:0/1267062384 shutdown_connections 2023-09-30T22:21:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7fb45d939700 1 --2- 172.21.15.89:0/1267062384 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4580a0e00 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7fb45d939700 1 --2- 172.21.15.89:0/1267062384 >> v2:172.21.15.89:3300/0 conn(0x7fb45809d630 0x7fb4580a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7fb45d939700 1 -- 172.21.15.89:0/1267062384 >> 172.21.15.89:0/1267062384 conn(0x7fb4580a3050 msgr2=0x7fb4580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7fb45d939700 1 -- 172.21.15.89:0/1267062384 shutdown_connections 2023-09-30T22:21:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.935+0000 7fb45d939700 1 -- 172.21.15.89:0/1267062384 wait complete. 2023-09-30T22:21:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f928d33f700 1 Processor -- start 2023-09-30T22:21:50.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7fb45d939700 1 Processor -- start 2023-09-30T22:21:50.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7fb45d939700 1 -- start start 2023-09-30T22:21:50.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f928d33f700 1 -- start start 2023-09-30T22:21:50.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f928d33f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f928809de50 0x7f92881c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f928d33f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928809e9b0 0x7f92881c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f928d33f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9288133b10 0x7f92881c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f928d33f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928811a890 con 0x7f928809e9b0 2023-09-30T22:21:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f928d33f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92881c1630 con 0x7f9288133b10 2023-09-30T22:21:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f928d33f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f928811cb80 con 0x7f928809de50 2023-09-30T22:21:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.937+0000 7f9286ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f928809de50 0x7f92881c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f9286ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f928809de50 0x7f92881c0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60456/0 (socket says 172.21.15.89:60456) 2023-09-30T22:21:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f9286ffd700 1 -- 172.21.15.89:0/1191786292 learned_addr learned my addr 172.21.15.89:0/1191786292 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f92877fe700 1 --2- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9288133b10 0x7f92881c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb45d939700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb45809d630 0x7fb458121290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb45d939700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4581217d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb45d939700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb458133930 0x7fb458121d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb45d939700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb458122dc0 con 0x7fb45809d630 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb45d939700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb458123c50 con 0x7fb458133930 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb45d939700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb458124730 con 0x7fb4580a0a30 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4577fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb458133930 0x7fb458121d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4567fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4581217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4577fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb458133930 0x7fb458121d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60426/0 (socket says 172.21.15.89:60426) 2023-09-30T22:21:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4577fe700 1 -- 172.21.15.89:0/3514810009 learned_addr learned my addr 172.21.15.89:0/3514810009 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f9286ffd700 1 -- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9288133b10 msgr2=0x7f92881c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f9286ffd700 1 --2- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9288133b10 0x7f92881c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f9286ffd700 1 -- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928809e9b0 msgr2=0x7f92881c10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f9286ffd700 1 --2- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928809e9b0 0x7f92881c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f9286ffd700 1 -- 172.21.15.89:0/1191786292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278009550 con 0x7f928809de50 2023-09-30T22:21:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7f9286ffd700 1 --2- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f928809de50 0x7f92881c0bb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f927801f040 tx=0x7f9278004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4577fe700 1 -- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 msgr2=0x7fb4581217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4577fe700 1 --2- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4581217d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4577fe700 1 -- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb45809d630 msgr2=0x7fb458121290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4577fe700 1 --2- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb45809d630 0x7fb458121290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4577fe700 1 -- 172.21.15.89:0/3514810009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb448009550 con 0x7fb458133930 2023-09-30T22:21:50.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.938+0000 7fb4567fc700 1 --2- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4581217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7fb4577fe700 1 --2- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb458133930 0x7fb458121d10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb45000cc80 tx=0x7fb45000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7f926f7fe700 1 -- 172.21.15.89:0/1191786292 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780140d0 con 0x7f928809de50 2023-09-30T22:21:50.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928811c1a0 con 0x7f928809de50 2023-09-30T22:21:50.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928811af50 con 0x7f928809de50 2023-09-30T22:21:50.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7fb43ffff700 1 -- 172.21.15.89:0/3514810009 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450003c00 con 0x7fb458133930 2023-09-30T22:21:50.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7fb43ffff700 1 -- 172.21.15.89:0/3514810009 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb450003d60 con 0x7fb458133930 2023-09-30T22:21:50.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7fb43ffff700 1 -- 172.21.15.89:0/3514810009 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45000ab50 con 0x7fb458133930 2023-09-30T22:21:50.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7f926f7fe700 1 -- 172.21.15.89:0/1191786292 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9278014230 con 0x7f928809de50 2023-09-30T22:21:50.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7f926f7fe700 1 -- 172.21.15.89:0/1191786292 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780145a0 con 0x7f928809de50 2023-09-30T22:21:50.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.940+0000 7f926f7fe700 1 -- 172.21.15.89:0/1191786292 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f92780036a0 con 0x7f928809de50 2023-09-30T22:21:50.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.940+0000 7f926f7fe700 1 -- 172.21.15.89:0/1191786292 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f92780412a0 con 0x7f928809de50 2023-09-30T22:21:50.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4581249f0 con 0x7fb458133930 2023-09-30T22:21:50.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.939+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4580a5130 con 0x7fb458133930 2023-09-30T22:21:50.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.940+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92680052e0 con 0x7f928809de50 2023-09-30T22:21:50.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.940+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4380052e0 con 0x7fb458133930 2023-09-30T22:21:50.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.941+0000 7fb43ffff700 1 -- 172.21.15.89:0/3514810009 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb45000acb0 con 0x7fb458133930 2023-09-30T22:21:50.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.941+0000 7fb43ffff700 1 -- 172.21.15.89:0/3514810009 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb4500401e0 con 0x7fb458133930 2023-09-30T22:21:50.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.949+0000 7f926f7fe700 1 -- 172.21.15.89:0/1191786292 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f927802b080 con 0x7f928809de50 2023-09-30T22:21:50.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.949+0000 7fb43ffff700 1 -- 172.21.15.89:0/3514810009 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb450011740 con 0x7fb458133930 2023-09-30T22:21:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.976+0000 7fbcbc3ab700 1 Processor -- start 2023-09-30T22:21:50.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.976+0000 7fbcbc3ab700 1 -- start start 2023-09-30T22:21:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcbc3ab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbcb406f5d0 0x7fbcb412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcbc3ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb4135c20 0x7fbcb4138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcbc3ab700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb406fa40 con 0x7fbcb412cf60 2023-09-30T22:21:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcbc3ab700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb412cd90 con 0x7fbcb406f5d0 2023-09-30T22:21:50.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcbc3ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb4071780 con 0x7fbcb4135c20 2023-09-30T22:21:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcb9946700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbcb406f5d0 0x7fbcb412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcb9946700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbcb406f5d0 0x7fbcb412ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42406/0 (socket says 172.21.15.89:42406) 2023-09-30T22:21:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcb9946700 1 -- 172.21.15.89:0/2813965907 learned_addr learned my addr 172.21.15.89:0/2813965907 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.977+0000 7fbcba948700 1 --2- 172.21.15.89:0/2813965907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb4135c20 0x7fbcb4138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.978+0000 7fbcb9946700 1 -- 172.21.15.89:0/2813965907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb4135c20 msgr2=0x7fbcb4138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.978+0000 7fbcb9946700 1 --2- 172.21.15.89:0/2813965907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb4135c20 0x7fbcb4138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.978+0000 7fbcb9946700 1 -- 172.21.15.89:0/2813965907 >> v1:172.21.15.89:6789/0 conn(0x7fbcb412cf60 legacy=0x7fbcb412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.978+0000 7fbcb9946700 1 -- 172.21.15.89:0/2813965907 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb4071a40 con 0x7fbcb406f5d0 2023-09-30T22:21:50.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.978+0000 7fbcb9946700 1 --2- 172.21.15.89:0/2813965907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb406f5d0 0x7fbcb412ffc0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009920 tx=0x7fbca4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fb57ef66aa0e86 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.978+0000 7fbcb9145700 1 -- 172.21.15.89:0/2813965907 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca400ba20 con 0x7fbcb406f5d0 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.978+0000 7fbcb9145700 1 -- 172.21.15.89:0/2813965907 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca400bb80 con 0x7fbcb406f5d0 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.978+0000 7fbcb9145700 1 -- 172.21.15.89:0/2813965907 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4009e70 con 0x7fbcb406f5d0 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.979+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/2813965907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb406f5d0 msgr2=0x7fbcb412ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.979+0000 7fbcbc3ab700 1 --2- 172.21.15.89:0/2813965907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb406f5d0 0x7fbcb412ffc0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009920 tx=0x7fbca4009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.979+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/2813965907 shutdown_connections 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.979+0000 7fbcbc3ab700 1 --2- 172.21.15.89:0/2813965907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb4135c20 0x7fbcb4138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.979+0000 7fbcbc3ab700 1 --2- 172.21.15.89:0/2813965907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb406f5d0 0x7fbcb412ffc0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.979+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/2813965907 >> 172.21.15.89:0/2813965907 conn(0x7fbcb40a2aa0 msgr2=0x7fbcb40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.979+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/2813965907 shutdown_connections 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.979+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/2813965907 wait complete. 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.980+0000 7fbcbc3ab700 1 Processor -- start 2023-09-30T22:21:50.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.980+0000 7fbcbc3ab700 1 -- start start 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcbc3ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb406f5d0 0x7fbcb409e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcbc3ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcb412cf60 0x7fbcb4099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcbc3ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb4135c20 0x7fbcb409a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcbc3ab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb409c410 con 0x7fbcb4135c20 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcbc3ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb409c5f0 con 0x7fbcb406f5d0 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcbc3ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb409ed80 con 0x7fbcb412cf60 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcb9946700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcb412cf60 0x7fbcb4099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcb9946700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcb412cf60 0x7fbcb4099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60470/0 (socket says 172.21.15.89:60470) 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcb9946700 1 -- 172.21.15.89:0/533694106 learned_addr learned my addr 172.21.15.89:0/533694106 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcba948700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb4135c20 0x7fbcb409a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcba147700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb406f5d0 0x7fbcb409e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcba948700 1 -- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcb412cf60 msgr2=0x7fbcb4099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcba948700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcb412cf60 0x7fbcb4099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcba948700 1 -- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb406f5d0 msgr2=0x7fbcb409e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcba948700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb406f5d0 0x7fbcb409e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcba948700 1 -- 172.21.15.89:0/533694106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca4009550 con 0x7fbcb4135c20 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcb9946700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcb412cf60 0x7fbcb4099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.981+0000 7fbcba948700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb4135c20 0x7fbcb409a290 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fbcac007d60 tx=0x7fbcac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.982+0000 7fbcaaffd700 1 -- 172.21.15.89:0/533694106 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac018070 con 0x7fbcb4135c20 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.982+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb409cf60 con 0x7fbcb4135c20 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.982+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb4129ad0 con 0x7fbcb4135c20 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.982+0000 7fbcaaffd700 1 -- 172.21.15.89:0/533694106 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac00eb30 con 0x7fbcb4135c20 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.982+0000 7fbcaaffd700 1 -- 172.21.15.89:0/533694106 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac003a20 con 0x7fbcb4135c20 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.983+0000 7fbcaaffd700 1 -- 172.21.15.89:0/533694106 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbcac014420 con 0x7fbcb4135c20 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.983+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9c0052e0 con 0x7fbcb4135c20 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.983+0000 7fbcaaffd700 1 -- 172.21.15.89:0/533694106 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbcac03f5a0 con 0x7fbcb4135c20 2023-09-30T22:21:50.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:50.992+0000 7fbcaaffd700 1 -- 172.21.15.89:0/533694106 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbcac0148b0 con 0x7fbcb4135c20 2023-09-30T22:21:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.095+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f78dc074ea0 con 0x7f78dc133e30 2023-09-30T22:21:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.096+0000 7f78d2ffd700 1 -- 172.21.15.89:0/953185776 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78cc0199a0 con 0x7f78dc133e30 2023-09-30T22:21:51.096 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc133e30 msgr2=0x7f78dc09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 --2- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc133e30 0x7f78dc09c9c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f78cc016000 tx=0x7f78cc004750 comp rx=0 tx=0).stop 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 shutdown_connections 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 --2- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78dc134260 0x7f78dc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 --2- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78dc133420 0x7f78dc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 --2- 172.21.15.89:0/953185776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78dc133e30 0x7f78dc09c9c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 >> 172.21.15.89:0/953185776 conn(0x7f78dc0a4210 msgr2=0x7f78dc0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 shutdown_connections 2023-09-30T22:21:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.098+0000 7f78e4282700 1 -- 172.21.15.89:0/953185776 wait complete. 2023-09-30T22:21:51.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:51.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.145+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9268005590 con 0x7f928809de50 2023-09-30T22:21:51.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.145+0000 7f926f7fe700 1 -- 172.21.15.89:0/1191786292 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9278012470 con 0x7f928809de50 2023-09-30T22:21:51.145 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f928809de50 msgr2=0x7f92881c0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 --2- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f928809de50 0x7f92881c0bb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f927801f040 tx=0x7f9278004750 comp rx=0 tx=0).stop 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 shutdown_connections 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 --2- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9288133b10 0x7f92881c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 --2- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f928809de50 0x7f92881c0bb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 --2- 172.21.15.89:0/1191786292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928809e9b0 0x7f92881c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 >> 172.21.15.89:0/1191786292 conn(0x7f92880a2aa0 msgr2=0x7f9288130680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 shutdown_connections 2023-09-30T22:21:51.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.148+0000 7f928d33f700 1 -- 172.21.15.89:0/1191786292 wait complete. 2023-09-30T22:21:51.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.155+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb438005860 con 0x7fb458133930 2023-09-30T22:21:51.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.156+0000 7fb43ffff700 1 -- 172.21.15.89:0/3514810009 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb450011380 con 0x7fb458133930 2023-09-30T22:21:51.155 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb458133930 msgr2=0x7fb458121d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 --2- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb458133930 0x7fb458121d10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb45000cc80 tx=0x7fb45000cf90 comp rx=0 tx=0).stop 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 shutdown_connections 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 --2- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb458133930 0x7fb458121d10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 --2- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb45809d630 0x7fb458121290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 --2- 172.21.15.89:0/3514810009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4580a0a30 0x7fb4581217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 >> 172.21.15.89:0/3514810009 conn(0x7fb4580a3050 msgr2=0x7fb4581331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 shutdown_connections 2023-09-30T22:21:51.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.159+0000 7fb45d939700 1 -- 172.21.15.89:0/3514810009 wait complete. 2023-09-30T22:21:51.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:51.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.188+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbc9c005590 con 0x7fbcb4135c20 2023-09-30T22:21:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.188+0000 7fbcaaffd700 1 -- 172.21.15.89:0/533694106 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbcac0122e0 con 0x7fbcb4135c20 2023-09-30T22:21:51.188 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb4135c20 msgr2=0x7fbcb409a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb4135c20 0x7fbcb409a290 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fbcac007d60 tx=0x7fbcac00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 shutdown_connections 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb4135c20 0x7fbcb409a290 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbcb406f5d0 0x7fbcb409e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 --2- 172.21.15.89:0/533694106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbcb412cf60 0x7fbcb4099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 >> 172.21.15.89:0/533694106 conn(0x7fbcb40a2aa0 msgr2=0x7fbcb412a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 shutdown_connections 2023-09-30T22:21:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:51.190+0000 7fbcbc3ab700 1 -- 172.21.15.89:0/533694106 wait complete. 2023-09-30T22:21:51.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:52.111 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:52.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:52.172 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:52.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.253+0000 7f13390d4700 1 Processor -- start 2023-09-30T22:21:52.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f13390d4700 1 -- start start 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f13390d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f1334138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f13390d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f13390d4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133409b120 con 0x7f133409d6f0 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f13390d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133406b460 con 0x7f133409e1d0 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f13390d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f133409a7e0 con 0x7f1334139170 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f1332d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f133409d6f0 0x7f133409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42756/0 (socket says 172.21.15.89:42756) 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f1332d9d700 1 -- 172.21.15.89:0/3779915684 learned_addr learned my addr 172.21.15.89:0/3779915684 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.254+0000 7f133359e700 1 --2- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f133259c700 1 --2- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f1334138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f1331d9b700 1 -- 172.21.15.89:0/3779915684 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070577150 0 0) 0x7f133409b120 con 0x7f133409d6f0 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f1331d9b700 1 -- 172.21.15.89:0/3779915684 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131c0035b0 con 0x7f133409d6f0 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f133359e700 1 -- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 msgr2=0x7f1334138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f133359e700 1 --2- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f1334138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f133359e700 1 -- 172.21.15.89:0/3779915684 >> v1:172.21.15.89:6789/0 conn(0x7f133409d6f0 legacy=0x7f133409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f133359e700 1 -- 172.21.15.89:0/3779915684 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133409a500 con 0x7f1334139170 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f133359e700 1 --2- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133413b550 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f132c009df0 tx=0x7f132c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2859f940348e7930 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f1331d9b700 1 -- 172.21.15.89:0/3779915684 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c00e070 con 0x7f1334139170 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f1331d9b700 1 -- 172.21.15.89:0/3779915684 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c00bae0 con 0x7f1334139170 2023-09-30T22:21:52.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.255+0000 7f1331d9b700 1 -- 172.21.15.89:0/3779915684 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c00be90 con 0x7f1334139170 2023-09-30T22:21:52.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.256+0000 7f13390d4700 1 -- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 msgr2=0x7f133413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.256+0000 7f13390d4700 1 --2- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133413b550 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f132c009df0 tx=0x7f132c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.256+0000 7f13390d4700 1 -- 172.21.15.89:0/3779915684 shutdown_connections 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.256+0000 7f13390d4700 1 --2- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133413b550 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.256+0000 7f13390d4700 1 --2- 172.21.15.89:0/3779915684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f1334138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.256+0000 7f13390d4700 1 -- 172.21.15.89:0/3779915684 >> 172.21.15.89:0/3779915684 conn(0x7f1334066f50 msgr2=0x7f133409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.256+0000 7f13390d4700 1 -- 172.21.15.89:0/3779915684 shutdown_connections 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.256+0000 7f13390d4700 1 -- 172.21.15.89:0/3779915684 wait complete. 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.257+0000 7f13390d4700 1 Processor -- start 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.257+0000 7f13390d4700 1 -- start start 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f13390d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f133409d6f0 0x7f133411e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f13390d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f133411ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f13390d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133411f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f13390d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13341220a0 con 0x7f133409d6f0 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f13390d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133409c710 con 0x7f133409e1d0 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f13390d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1334120580 con 0x7f1334139170 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f133411ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f133411ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60446/0 (socket says 172.21.15.89:60446) 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133259c700 1 -- 172.21.15.89:0/37169295 learned_addr learned my addr 172.21.15.89:0/37169295 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133359e700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133411f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f1332d9d700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f133409d6f0 0x7f133411e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133359e700 1 -- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 msgr2=0x7f133411ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133359e700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f133411ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133359e700 1 -- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f133409d6f0 msgr2=0x7f133411e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133359e700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f133409d6f0 0x7f133411e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133359e700 1 -- 172.21.15.89:0/37169295 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c009a70 con 0x7f1334139170 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.258+0000 7f133259c700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f133411ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.259+0000 7f133359e700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133411f380 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f132c016000 tx=0x7f132c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.259+0000 7f131b7fe700 1 -- 172.21.15.89:0/37169295 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c004680 con 0x7f1334139170 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.259+0000 7f131b7fe700 1 -- 172.21.15.89:0/37169295 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c020070 con 0x7f1334139170 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.259+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13340a3ab0 con 0x7f1334139170 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.259+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13341347a0 con 0x7f1334139170 2023-09-30T22:21:52.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.259+0000 7f131b7fe700 1 -- 172.21.15.89:0/37169295 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c027480 con 0x7f1334139170 2023-09-30T22:21:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.260+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13140052e0 con 0x7f1334139170 2023-09-30T22:21:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.260+0000 7f131b7fe700 1 -- 172.21.15.89:0/37169295 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f132c012430 con 0x7f1334139170 2023-09-30T22:21:52.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.261+0000 7f131b7fe700 1 -- 172.21.15.89:0/37169295 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f132c022070 con 0x7f1334139170 2023-09-30T22:21:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.269+0000 7f131b7fe700 1 -- 172.21.15.89:0/37169295 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f132c011750 con 0x7f1334139170 2023-09-30T22:21:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9c01988700 1 Processor -- start 2023-09-30T22:21:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9c01988700 1 -- start start 2023-09-30T22:21:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9c01988700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bfc09d6f0 0x7f9bfc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9c01988700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e000 0x7f9bfc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9c01988700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc09f130 con 0x7f9bfc09e410 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9c01988700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc09f520 con 0x7f9bfc09d6f0 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9c01988700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc09e930 con 0x7f9bfc09e000 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9bfaffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bfc09d6f0 0x7f9bfc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9bfa7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e000 0x7f9bfc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9bfa7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e000 0x7f9bfc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60498/0 (socket says 172.21.15.89:60498) 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9bfaffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bfc09d6f0 0x7f9bfc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42428/0 (socket says 172.21.15.89:42428) 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.303+0000 7f9bfa7fc700 1 -- 172.21.15.89:0/550679175 learned_addr learned my addr 172.21.15.89:0/550679175 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.304+0000 7f9bfa7fc700 1 -- 172.21.15.89:0/550679175 >> v2:172.21.15.89:3300/0 conn(0x7f9bfc09d6f0 msgr2=0x7f9bfc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.304+0000 7f9bfa7fc700 1 --2- 172.21.15.89:0/550679175 >> v2:172.21.15.89:3300/0 conn(0x7f9bfc09d6f0 0x7f9bfc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.304+0000 7f9bfa7fc700 1 -- 172.21.15.89:0/550679175 >> v1:172.21.15.89:6789/0 conn(0x7f9bfc09e410 legacy=0x7f9bfc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.304+0000 7f9bfa7fc700 1 -- 172.21.15.89:0/550679175 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc09b2d0 con 0x7f9bfc09e000 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.304+0000 7f9bfa7fc700 1 --2- 172.21.15.89:0/550679175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e000 0x7f9bfc130660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00098f0 tx=0x7f9bf0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbed2ca7f06c52a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.304+0000 7f9bf9ffb700 1 -- 172.21.15.89:0/550679175 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0011830 con 0x7f9bfc09e000 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.304+0000 7f9bf9ffb700 1 -- 172.21.15.89:0/550679175 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0011990 con 0x7f9bfc09e000 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.304+0000 7f9bf9ffb700 1 -- 172.21.15.89:0/550679175 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0011d20 con 0x7f9bfc09e000 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.305+0000 7f9c01988700 1 -- 172.21.15.89:0/550679175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e000 msgr2=0x7f9bfc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.305+0000 7f9c01988700 1 --2- 172.21.15.89:0/550679175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e000 0x7f9bfc130660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00098f0 tx=0x7f9bf0009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.305+0000 7f9c01988700 1 -- 172.21.15.89:0/550679175 shutdown_connections 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.305+0000 7f9c01988700 1 --2- 172.21.15.89:0/550679175 >> v2:172.21.15.89:3300/0 conn(0x7f9bfc09d6f0 0x7f9bfc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.305+0000 7f9c01988700 1 --2- 172.21.15.89:0/550679175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e000 0x7f9bfc130660 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.305+0000 7f9c01988700 1 -- 172.21.15.89:0/550679175 >> 172.21.15.89:0/550679175 conn(0x7f9bfc066f50 msgr2=0x7f9bfc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.305+0000 7f9c01988700 1 -- 172.21.15.89:0/550679175 shutdown_connections 2023-09-30T22:21:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.305+0000 7f9c01988700 1 -- 172.21.15.89:0/550679175 wait complete. 2023-09-30T22:21:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.306+0000 7f9c01988700 1 Processor -- start 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.306+0000 7f9c01988700 1 -- start start 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.306+0000 7f9c01988700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc09d6f0 0x7f9bfc11df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.306+0000 7f9c01988700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bfc09e000 0x7f9bfc11e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.306+0000 7f9c01988700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e410 0x7f9bfc11ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.306+0000 7f9c01988700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc121a60 con 0x7f9bfc09d6f0 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.306+0000 7f9c01988700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc120170 con 0x7f9bfc09e000 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.306+0000 7f9c01988700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc120e30 con 0x7f9bfc09e410 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfa7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bfc09e000 0x7f9bfc11e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e410 0x7f9bfc11ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e410 0x7f9bfc11ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60512/0 (socket says 172.21.15.89:60512) 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfb7fe700 1 -- 172.21.15.89:0/3594766248 learned_addr learned my addr 172.21.15.89:0/3594766248 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfaffd700 1 --2- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc09d6f0 0x7f9bfc11df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfa7fc700 1 -- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e410 msgr2=0x7f9bfc11ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfa7fc700 1 --2- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e410 0x7f9bfc11ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfa7fc700 1 -- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc09d6f0 msgr2=0x7f9bfc11df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfa7fc700 1 --2- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc09d6f0 0x7f9bfc11df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfa7fc700 1 -- 172.21.15.89:0/3594766248 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009550 con 0x7f9bfc09e000 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9bfa7fc700 1 --2- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bfc09e000 0x7f9bfc11e4d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0016000 tx=0x7f9bf0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9be3fff700 1 -- 172.21.15.89:0/3594766248 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0004400 con 0x7f9bfc09e000 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9be3fff700 1 -- 172.21.15.89:0/3594766248 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf001f070 con 0x7f9bfc09e000 2023-09-30T22:21:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9be3fff700 1 -- 172.21.15.89:0/3594766248 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00038e0 con 0x7f9bfc09e000 2023-09-30T22:21:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.307+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc1211f0 con 0x7f9bfc09e000 2023-09-30T22:21:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.308+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc0a1d00 con 0x7f9bfc09e000 2023-09-30T22:21:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.309+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc121420 con 0x7f9bfc09e000 2023-09-30T22:21:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.310+0000 7f9be3fff700 1 -- 172.21.15.89:0/3594766248 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9bf0021030 con 0x7f9bfc09e000 2023-09-30T22:21:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.310+0000 7f9be3fff700 1 -- 172.21.15.89:0/3594766248 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9bf0040bc0 con 0x7f9bfc09e000 2023-09-30T22:21:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.313+0000 7f9be3fff700 1 -- 172.21.15.89:0/3594766248 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9bf0003a40 con 0x7f9bfc09e000 2023-09-30T22:21:52.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.339+0000 7f740f908700 1 Processor -- start 2023-09-30T22:21:52.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.339+0000 7f740f908700 1 -- start start 2023-09-30T22:21:52.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740f908700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f740809ac90 0x7f740813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740f908700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740813d240 0x7f740813f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740f908700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7408134980 con 0x7f740806c0e0 2023-09-30T22:21:52.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740f908700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740806c4b0 con 0x7f740809ac90 2023-09-30T22:21:52.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740f908700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740809b0e0 con 0x7f740813d240 2023-09-30T22:21:52.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740cea3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f740809ac90 0x7f740813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740dea5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740813d240 0x7f740813f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740d6a4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f740806c0e0 0x7f7408134ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42772/0 (socket says 172.21.15.89:42772) 2023-09-30T22:21:52.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740cea3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f740809ac90 0x7f740813cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42456/0 (socket says 172.21.15.89:42456) 2023-09-30T22:21:52.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740d6a4700 1 -- 172.21.15.89:0/3460344021 learned_addr learned my addr 172.21.15.89:0/3460344021 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:52.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740cea3700 1 -- 172.21.15.89:0/3460344021 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740813d240 msgr2=0x7f740813f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740cea3700 1 --2- 172.21.15.89:0/3460344021 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740813d240 0x7f740813f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740cea3700 1 -- 172.21.15.89:0/3460344021 >> v1:172.21.15.89:6789/0 conn(0x7f740806c0e0 legacy=0x7f7408134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740cea3700 1 -- 172.21.15.89:0/3460344021 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080a0600 con 0x7f740809ac90 2023-09-30T22:21:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f740cea3700 1 --2- 172.21.15.89:0/3460344021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740809ac90 0x7f740813cd00 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009840 tx=0x7f73f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f722acc63153d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.340+0000 7f73fffff700 1 -- 172.21.15.89:0/3460344021 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8011830 con 0x7f740809ac90 2023-09-30T22:21:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f73fffff700 1 -- 172.21.15.89:0/3460344021 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f8011990 con 0x7f740809ac90 2023-09-30T22:21:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f73fffff700 1 -- 172.21.15.89:0/3460344021 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8011d20 con 0x7f740809ac90 2023-09-30T22:21:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f740f908700 1 -- 172.21.15.89:0/3460344021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740809ac90 msgr2=0x7f740813cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f740f908700 1 --2- 172.21.15.89:0/3460344021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740809ac90 0x7f740813cd00 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009840 tx=0x7f73f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f740f908700 1 -- 172.21.15.89:0/3460344021 shutdown_connections 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f740f908700 1 --2- 172.21.15.89:0/3460344021 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740813d240 0x7f740813f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f740f908700 1 --2- 172.21.15.89:0/3460344021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740809ac90 0x7f740813cd00 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f740f908700 1 -- 172.21.15.89:0/3460344021 >> 172.21.15.89:0/3460344021 conn(0x7f74080a4210 msgr2=0x7f74080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f740f908700 1 -- 172.21.15.89:0/3460344021 shutdown_connections 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.341+0000 7f740f908700 1 -- 172.21.15.89:0/3460344021 wait complete. 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.342+0000 7f740f908700 1 Processor -- start 2023-09-30T22:21:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.342+0000 7f740f908700 1 -- start start 2023-09-30T22:21:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740f908700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740806c0e0 0x7f7408132c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740f908700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740809ac90 0x7f7408133180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740f908700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f740813d240 0x7f74081336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740f908700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740809ca70 con 0x7f740806c0e0 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740f908700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74081380a0 con 0x7f740809ac90 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740f908700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7408139050 con 0x7f740813d240 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740dea5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f740813d240 0x7f74081336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740dea5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f740813d240 0x7f74081336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60520/0 (socket says 172.21.15.89:60520) 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740dea5700 1 -- 172.21.15.89:0/3969664859 learned_addr learned my addr 172.21.15.89:0/3969664859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740cea3700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740809ac90 0x7f7408133180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740d6a4700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740806c0e0 0x7f7408132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740cea3700 1 -- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f740813d240 msgr2=0x7f74081336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740cea3700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f740813d240 0x7f74081336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740cea3700 1 -- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740806c0e0 msgr2=0x7f7408132c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740cea3700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740806c0e0 0x7f7408132c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740cea3700 1 -- 172.21.15.89:0/3969664859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f80094a0 con 0x7f740809ac90 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740dea5700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f740813d240 0x7f74081336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.343+0000 7f740cea3700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740809ac90 0x7f7408133180 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7408072a30 tx=0x7f73f8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.344+0000 7f73fdffb700 1 -- 172.21.15.89:0/3969664859 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8004550 con 0x7f740809ac90 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.344+0000 7f73fdffb700 1 -- 172.21.15.89:0/3969664859 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f801f070 con 0x7f740809ac90 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.344+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740809c830 con 0x7f740809ac90 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.344+0000 7f73fdffb700 1 -- 172.21.15.89:0/3969664859 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80038e0 con 0x7f740809ac90 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.344+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740813c600 con 0x7f740809ac90 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.345+0000 7f73fdffb700 1 -- 172.21.15.89:0/3969664859 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f73f8003a40 con 0x7f740809ac90 2023-09-30T22:21:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.345+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7408074ea0 con 0x7f740809ac90 2023-09-30T22:21:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.345+0000 7f73fdffb700 1 -- 172.21.15.89:0/3969664859 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f73f8021070 con 0x7f740809ac90 2023-09-30T22:21:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.350+0000 7f73fdffb700 1 -- 172.21.15.89:0/3969664859 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73f8026480 con 0x7f740809ac90 2023-09-30T22:21:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.352+0000 7ff3eaebe700 1 Processor -- start 2023-09-30T22:21:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.352+0000 7ff3eaebe700 1 -- start start 2023-09-30T22:21:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3eaebe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409d6f0 0x7ff3e409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3eaebe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3eaebe700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e409f130 con 0x7ff3e409e410 2023-09-30T22:21:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3eaebe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e409f520 con 0x7ff3e409d6f0 2023-09-30T22:21:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3eaebe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e409e930 con 0x7ff3e409e000 2023-09-30T22:21:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e8c5a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409d6f0 0x7ff3e409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e8c5a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409d6f0 0x7ff3e409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60494/0 (socket says 172.21.15.89:60494) 2023-09-30T22:21:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e8c5a700 1 -- 172.21.15.89:0/823045841 learned_addr learned my addr 172.21.15.89:0/823045841 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e8c5a700 1 -- 172.21.15.89:0/823045841 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 msgr2=0x7ff3e4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e8c5a700 1 --2- 172.21.15.89:0/823045841 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e8c5a700 1 -- 172.21.15.89:0/823045841 >> v1:172.21.15.89:6789/0 conn(0x7ff3e409e410 legacy=0x7ff3e4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e8c5a700 1 -- 172.21.15.89:0/823045841 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e409b2d0 con 0x7ff3e409d6f0 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.353+0000 7ff3e37fe700 1 -- 172.21.15.89:0/823045841 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552314529 0 0) 0x7ff3e409f130 con 0x7ff3e409e410 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.354+0000 7ff3e3fff700 1 --2- 172.21.15.89:0/823045841 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.354+0000 7ff3e8c5a700 1 --2- 172.21.15.89:0/823045841 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409d6f0 0x7ff3e409dac0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009920 tx=0x7ff3d8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2f71cea2f3cde67 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.354+0000 7ff3e37fe700 1 -- 172.21.15.89:0/823045841 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8011830 con 0x7ff3e409d6f0 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.354+0000 7ff3e37fe700 1 -- 172.21.15.89:0/823045841 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d8011990 con 0x7ff3e409d6f0 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.354+0000 7ff3e37fe700 1 -- 172.21.15.89:0/823045841 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8011d20 con 0x7ff3e409d6f0 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.354+0000 7ff3eaebe700 1 -- 172.21.15.89:0/823045841 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409d6f0 msgr2=0x7ff3e409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.354+0000 7ff3eaebe700 1 --2- 172.21.15.89:0/823045841 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409d6f0 0x7ff3e409dac0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009920 tx=0x7ff3d8009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.355+0000 7ff3eaebe700 1 -- 172.21.15.89:0/823045841 shutdown_connections 2023-09-30T22:21:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.355+0000 7ff3eaebe700 1 --2- 172.21.15.89:0/823045841 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409d6f0 0x7ff3e409dac0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.355+0000 7ff3eaebe700 1 --2- 172.21.15.89:0/823045841 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.355+0000 7ff3eaebe700 1 -- 172.21.15.89:0/823045841 >> 172.21.15.89:0/823045841 conn(0x7ff3e4066f50 msgr2=0x7ff3e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.355+0000 7ff3eaebe700 1 -- 172.21.15.89:0/823045841 shutdown_connections 2023-09-30T22:21:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.355+0000 7ff3eaebe700 1 -- 172.21.15.89:0/823045841 wait complete. 2023-09-30T22:21:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.356+0000 7ff3eaebe700 1 Processor -- start 2023-09-30T22:21:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.356+0000 7ff3eaebe700 1 -- start start 2023-09-30T22:21:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3eaebe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3e409d6f0 0x7ff3e411dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3eaebe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e411e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3eaebe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409e410 0x7ff3e411e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3eaebe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e411f930 con 0x7ff3e409d6f0 2023-09-30T22:21:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3eaebe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e411ff50 con 0x7ff3e409e410 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3eaebe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e4120b80 con 0x7ff3e409e000 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e945b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409e410 0x7ff3e411e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e411e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e945b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409e410 0x7ff3e411e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60500/0 (socket says 172.21.15.89:60500) 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e945b700 1 -- 172.21.15.89:0/2297032681 learned_addr learned my addr 172.21.15.89:0/2297032681 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e8c5a700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3e409d6f0 0x7ff3e411dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e3fff700 1 -- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409e410 msgr2=0x7ff3e411e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e3fff700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409e410 0x7ff3e411e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e3fff700 1 -- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3e409d6f0 msgr2=0x7ff3e411dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e3fff700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3e409d6f0 0x7ff3e411dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.357+0000 7ff3e3fff700 1 -- 172.21.15.89:0/2297032681 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d00094a0 con 0x7ff3e409e000 2023-09-30T22:21:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.358+0000 7ff3e945b700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409e410 0x7ff3e411e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.358+0000 7ff3e3fff700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e411e260 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000e9b0 tx=0x7ff3d000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.358+0000 7ff3e17fa700 1 -- 172.21.15.89:0/2297032681 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000ee50 con 0x7ff3e409e000 2023-09-30T22:21:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.358+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d8009550 con 0x7ff3e409e000 2023-09-30T22:21:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.358+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e40a3b80 con 0x7ff3e409e000 2023-09-30T22:21:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.359+0000 7ff3e17fa700 1 -- 172.21.15.89:0/2297032681 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d000c850 con 0x7ff3e409e000 2023-09-30T22:21:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.359+0000 7ff3e17fa700 1 -- 172.21.15.89:0/2297032681 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000cc00 con 0x7ff3e409e000 2023-09-30T22:21:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.359+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e411ac20 con 0x7ff3e409e000 2023-09-30T22:21:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.359+0000 7ff3e17fa700 1 -- 172.21.15.89:0/2297032681 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff3d0013470 con 0x7ff3e409e000 2023-09-30T22:21:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.359+0000 7ff3e17fa700 1 -- 172.21.15.89:0/2297032681 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff3d003f830 con 0x7ff3e409e000 2023-09-30T22:21:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.369+0000 7ff3e17fa700 1 -- 172.21.15.89:0/2297032681 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff3d0013720 con 0x7ff3e409e000 2023-09-30T22:21:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.471+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1314005860 con 0x7f1334139170 2023-09-30T22:21:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.472+0000 7f131b7fe700 1 -- 172.21.15.89:0/37169295 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f132c035090 con 0x7f1334139170 2023-09-30T22:21:52.472 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 msgr2=0x7f133411f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133411f380 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f132c016000 tx=0x7f132c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 shutdown_connections 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1334139170 0x7f133411f380 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f133409d6f0 0x7f133411e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 --2- 172.21.15.89:0/37169295 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f133409e1d0 0x7f133411ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 >> 172.21.15.89:0/37169295 conn(0x7f1334066f50 msgr2=0x7f1334136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 shutdown_connections 2023-09-30T22:21:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.475+0000 7f13390d4700 1 -- 172.21.15.89:0/37169295 wait complete. 2023-09-30T22:21:52.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:52.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.545+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9bfc121420 con 0x7f9bfc09e000 2023-09-30T22:21:52.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.545+0000 7f9be3fff700 1 -- 172.21.15.89:0/3594766248 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9bfc121420 con 0x7f9bfc09e000 2023-09-30T22:21:52.545 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bfc09e000 msgr2=0x7f9bfc11e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 --2- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bfc09e000 0x7f9bfc11e4d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0016000 tx=0x7f9bf0004750 comp rx=0 tx=0).stop 2023-09-30T22:21:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 shutdown_connections 2023-09-30T22:21:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 --2- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bfc09e410 0x7f9bfc11ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 --2- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc09d6f0 0x7f9bfc11df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 --2- 172.21.15.89:0/3594766248 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bfc09e000 0x7f9bfc11e4d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 >> 172.21.15.89:0/3594766248 conn(0x7f9bfc066f50 msgr2=0x7f9bfc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 shutdown_connections 2023-09-30T22:21:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.547+0000 7f9c01988700 1 -- 172.21.15.89:0/3594766248 wait complete. 2023-09-30T22:21:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.549+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f740806b250 con 0x7f740809ac90 2023-09-30T22:21:52.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.549+0000 7f73fdffb700 1 -- 172.21.15.89:0/3969664859 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73f802b030 con 0x7f740809ac90 2023-09-30T22:21:52.549 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:52.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.551+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740809ac90 msgr2=0x7f7408133180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.551+0000 7f740f908700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740809ac90 0x7f7408133180 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7408072a30 tx=0x7f73f8004750 comp rx=0 tx=0).stop 2023-09-30T22:21:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.551+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 shutdown_connections 2023-09-30T22:21:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.551+0000 7f740f908700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f740813d240 0x7f74081336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.551+0000 7f740f908700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740806c0e0 0x7f7408132c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.551+0000 7f740f908700 1 --2- 172.21.15.89:0/3969664859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f740809ac90 0x7f7408133180 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.551+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 >> 172.21.15.89:0/3969664859 conn(0x7f74080a4210 msgr2=0x7f74080a0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.552+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 shutdown_connections 2023-09-30T22:21:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.552+0000 7f740f908700 1 -- 172.21.15.89:0/3969664859 wait complete. 2023-09-30T22:21:52.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:52.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:52.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.580+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff3e409c700 con 0x7ff3e409e000 2023-09-30T22:21:52.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.580+0000 7ff3e17fa700 1 -- 172.21.15.89:0/2297032681 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3d0021020 con 0x7ff3e409e000 2023-09-30T22:21:52.580 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:52.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 msgr2=0x7ff3e411e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e411e260 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000e9b0 tx=0x7ff3d000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 shutdown_connections 2023-09-30T22:21:52.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3e409e410 0x7ff3e411e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3e409d6f0 0x7ff3e411dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 --2- 172.21.15.89:0/2297032681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3e409e000 0x7ff3e411e260 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:52.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 >> 172.21.15.89:0/2297032681 conn(0x7ff3e4066f50 msgr2=0x7ff3e4130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 shutdown_connections 2023-09-30T22:21:52.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:52.582+0000 7ff3eaebe700 1 -- 172.21.15.89:0/2297032681 wait complete. 2023-09-30T22:21:52.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:53.489 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:53.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:53.565 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:53.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:53.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.632+0000 7f9bb32d4700 1 Processor -- start 2023-09-30T22:21:53.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb32d4700 1 -- start start 2023-09-30T22:21:53.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb32d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cc20 0x7f9bac12f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb32d4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9bac12c2f0 0x7f9bac12fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb32d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cff0 0x7f9bac1303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb32d4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bac09a510 con 0x7f9bac12c2f0 2023-09-30T22:21:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb32d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac12c7d0 con 0x7f9bac12cff0 2023-09-30T22:21:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb32d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac09c190 con 0x7f9bac12cc20 2023-09-30T22:21:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb1070700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cc20 0x7f9bac12f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb1070700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cc20 0x7f9bac12f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60552/0 (socket says 172.21.15.89:60552) 2023-09-30T22:21:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb1070700 1 -- 172.21.15.89:0/721311684 learned_addr learned my addr 172.21.15.89:0/721311684 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb086f700 1 --2- 172.21.15.89:0/721311684 >> v2:172.21.15.89:3300/0 conn(0x7f9bac12c2f0 0x7f9bac12fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.633+0000 7f9bb1871700 1 --2- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cff0 0x7f9bac1303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9bb086f700 1 -- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cc20 msgr2=0x7f9bac12f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9bb086f700 1 --2- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cc20 0x7f9bac12f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9bb086f700 1 -- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cff0 msgr2=0x7f9bac1303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:53.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9bb086f700 1 --2- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cff0 0x7f9bac1303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9bb086f700 1 -- 172.21.15.89:0/721311684 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac09b540 con 0x7f9bac12c2f0 2023-09-30T22:21:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9bb086f700 1 --2- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac12fe70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009840 tx=0x7f9b9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9119640defc72ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9ba3fff700 1 -- 172.21.15.89:0/721311684 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c011830 con 0x7f9bac12c2f0 2023-09-30T22:21:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9ba3fff700 1 -- 172.21.15.89:0/721311684 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c011990 con 0x7f9bac12c2f0 2023-09-30T22:21:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.634+0000 7f9ba3fff700 1 -- 172.21.15.89:0/721311684 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c011d20 con 0x7f9bac12c2f0 2023-09-30T22:21:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 -- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 msgr2=0x7f9bac12fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 --2- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac12fe70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009840 tx=0x7f9b9c009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 -- 172.21.15.89:0/721311684 shutdown_connections 2023-09-30T22:21:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 --2- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cff0 0x7f9bac1303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 --2- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cc20 0x7f9bac12f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 --2- 172.21.15.89:0/721311684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac12fe70 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 -- 172.21.15.89:0/721311684 >> 172.21.15.89:0/721311684 conn(0x7f9bac09f470 msgr2=0x7f9bac0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 -- 172.21.15.89:0/721311684 shutdown_connections 2023-09-30T22:21:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.635+0000 7f9bb32d4700 1 -- 172.21.15.89:0/721311684 wait complete. 2023-09-30T22:21:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.636+0000 7f9bb32d4700 1 Processor -- start 2023-09-30T22:21:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.636+0000 7f9bb32d4700 1 -- start start 2023-09-30T22:21:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb32d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac1c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb32d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cc20 0x7f9bac1c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb32d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cff0 0x7f9bac1c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb32d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac09fe50 con 0x7f9bac12c2f0 2023-09-30T22:21:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb32d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac129f10 con 0x7f9bac12cc20 2023-09-30T22:21:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb32d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac070e90 con 0x7f9bac12cff0 2023-09-30T22:21:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac1c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac1c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42494/0 (socket says 172.21.15.89:42494) 2023-09-30T22:21:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 -- 172.21.15.89:0/1746736321 learned_addr learned my addr 172.21.15.89:0/1746736321 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 -- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cff0 msgr2=0x7f9bac1c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 --2- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cff0 0x7f9bac1c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 -- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cc20 msgr2=0x7f9bac1c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 --2- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cc20 0x7f9bac1c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 -- 172.21.15.89:0/1746736321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0094a0 con 0x7f9bac12c2f0 2023-09-30T22:21:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.637+0000 7f9bb1070700 1 --2- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac1c5f00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800e910 tx=0x7f9ba800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.638+0000 7f9ba1ffb700 1 -- 172.21.15.89:0/1746736321 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba80096a0 con 0x7f9bac12c2f0 2023-09-30T22:21:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.638+0000 7f9ba1ffb700 1 -- 172.21.15.89:0/1746736321 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba800c850 con 0x7f9bac12c2f0 2023-09-30T22:21:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.638+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac1b7220 con 0x7f9bac12c2f0 2023-09-30T22:21:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.638+0000 7f9ba1ffb700 1 -- 172.21.15.89:0/1746736321 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800cc00 con 0x7f9bac12c2f0 2023-09-30T22:21:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.638+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac1b5c10 con 0x7f9bac12c2f0 2023-09-30T22:21:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.639+0000 7f9ba1ffb700 1 -- 172.21.15.89:0/1746736321 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ba800cd60 con 0x7f9bac12c2f0 2023-09-30T22:21:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.639+0000 7f9ba1ffb700 1 -- 172.21.15.89:0/1746736321 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9ba803f7e0 con 0x7f9bac12c2f0 2023-09-30T22:21:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.640+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac1b7720 con 0x7f9bac12c2f0 2023-09-30T22:21:53.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.648+0000 7f9ba1ffb700 1 -- 172.21.15.89:0/1746736321 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ba80232f0 con 0x7f9bac12c2f0 2023-09-30T22:21:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.699+0000 7fef3f4f0700 1 Processor -- start 2023-09-30T22:21:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.699+0000 7fef3f4f0700 1 -- start start 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3f4f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef38136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3f4f0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef38136d80 0x7fef38139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3f4f0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3809ece0 con 0x7fef3806a770 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3f4f0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3806b640 con 0x7fef3806b250 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3f4f0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef3809e760 con 0x7fef38136d80 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3d28c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fef3806a770 0x7fef3806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42794/0 (socket says 172.21.15.89:42794) 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3d28c700 1 -- 172.21.15.89:0/69533554 learned_addr learned my addr 172.21.15.89:0/69533554 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3ca8b700 1 --2- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef38136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef3da8d700 1 --2- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef38136d80 0x7fef38139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.700+0000 7fef2ffff700 1 -- 172.21.15.89:0/69533554 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919128531 0 0) 0x7fef3809ece0 con 0x7fef3806a770 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef2ffff700 1 -- 172.21.15.89:0/69533554 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef240035b0 con 0x7fef3806a770 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef3ca8b700 1 -- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef38136d80 msgr2=0x7fef38139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef3ca8b700 1 --2- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef38136d80 0x7fef38139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef3ca8b700 1 -- 172.21.15.89:0/69533554 >> v1:172.21.15.89:6789/0 conn(0x7fef3806a770 legacy=0x7fef3806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef3ca8b700 1 -- 172.21.15.89:0/69533554 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3809e040 con 0x7fef3806b250 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef3ca8b700 1 --2- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef38136840 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fef28009870 tx=0x7fef28009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fcdc2c4fc2684d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef2ffff700 1 -- 172.21.15.89:0/69533554 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28011830 con 0x7fef3806b250 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef2ffff700 1 -- 172.21.15.89:0/69533554 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef28011990 con 0x7fef3806b250 2023-09-30T22:21:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.701+0000 7fef2ffff700 1 -- 172.21.15.89:0/69533554 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28011d20 con 0x7fef3806b250 2023-09-30T22:21:53.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.702+0000 7fef3f4f0700 1 -- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 msgr2=0x7fef38136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.702+0000 7fef3f4f0700 1 --2- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef38136840 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fef28009870 tx=0x7fef28009b80 comp rx=0 tx=0).stop 2023-09-30T22:21:53.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.702+0000 7fef3f4f0700 1 -- 172.21.15.89:0/69533554 shutdown_connections 2023-09-30T22:21:53.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.702+0000 7fef3f4f0700 1 --2- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef38136d80 0x7fef38139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.702+0000 7fef3f4f0700 1 --2- 172.21.15.89:0/69533554 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef38136840 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.702+0000 7fef3f4f0700 1 -- 172.21.15.89:0/69533554 >> 172.21.15.89:0/69533554 conn(0x7fef38066f50 msgr2=0x7fef3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.702+0000 7fef3f4f0700 1 -- 172.21.15.89:0/69533554 shutdown_connections 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.702+0000 7fef3f4f0700 1 -- 172.21.15.89:0/69533554 wait complete. 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.703+0000 7fef3f4f0700 1 Processor -- start 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.703+0000 7fef3f4f0700 1 -- start start 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3f4f0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef3806a770 0x7fef381c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3f4f0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef381c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3f4f0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef38136d80 0x7fef38119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3f4f0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3811aac0 con 0x7fef38136d80 2023-09-30T22:21:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3f4f0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3811bf00 con 0x7fef3806b250 2023-09-30T22:21:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3f4f0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef381302d0 con 0x7fef3806a770 2023-09-30T22:21:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3ca8b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef381c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3ca8b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef381c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60554/0 (socket says 172.21.15.89:60554) 2023-09-30T22:21:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3ca8b700 1 -- 172.21.15.89:0/1445445136 learned_addr learned my addr 172.21.15.89:0/1445445136 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3da8d700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef38136d80 0x7fef38119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.704+0000 7fef3d28c700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef3806a770 0x7fef381c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3ca8b700 1 -- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef3806a770 msgr2=0x7fef381c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3ca8b700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef3806a770 0x7fef381c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3ca8b700 1 -- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef38136d80 msgr2=0x7fef38119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3ca8b700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef38136d80 0x7fef38119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3ca8b700 1 -- 172.21.15.89:0/1445445136 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef280094a0 con 0x7fef3806b250 2023-09-30T22:21:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3da8d700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef38136d80 0x7fef38119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3d28c700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef3806a770 0x7fef381c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3ca8b700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef381c4a50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fef3809ee70 tx=0x7fef28004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef2dffb700 1 -- 172.21.15.89:0/1445445136 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28004450 con 0x7fef3806b250 2023-09-30T22:21:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef2dffb700 1 -- 172.21.15.89:0/1445445136 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2801f070 con 0x7fef3806b250 2023-09-30T22:21:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef38136ad0 con 0x7fef3806b250 2023-09-30T22:21:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef2dffb700 1 -- 172.21.15.89:0/1445445136 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280038e0 con 0x7fef3806b250 2023-09-30T22:21:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.705+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef38071e60 con 0x7fef3806b250 2023-09-30T22:21:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.706+0000 7fef2dffb700 1 -- 172.21.15.89:0/1445445136 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef28003a40 con 0x7fef3806b250 2023-09-30T22:21:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.706+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef38074ea0 con 0x7fef3806b250 2023-09-30T22:21:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.706+0000 7fef2dffb700 1 -- 172.21.15.89:0/1445445136 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef28021070 con 0x7fef3806b250 2023-09-30T22:21:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.715+0000 7fef2dffb700 1 -- 172.21.15.89:0/1445445136 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef280045b0 con 0x7fef3806b250 2023-09-30T22:21:53.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.736+0000 7f88474f2700 1 Processor -- start 2023-09-30T22:21:53.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f88474f2700 1 -- start start 2023-09-30T22:21:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f88474f2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f884009d6f0 0x7f884009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f88474f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e440 0x7f8840134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f88474f2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884009b120 con 0x7f884009e000 2023-09-30T22:21:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f88474f2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884006b460 con 0x7f884009d6f0 2023-09-30T22:21:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f88474f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f884009a7e0 con 0x7f884009e440 2023-09-30T22:21:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f884528e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f884009d6f0 0x7f884009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f884528e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f884009d6f0 0x7f884009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42512/0 (socket says 172.21.15.89:42512) 2023-09-30T22:21:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f884528e700 1 -- 172.21.15.89:0/2729917248 learned_addr learned my addr 172.21.15.89:0/2729917248 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.737+0000 7f8845a8f700 1 --2- 172.21.15.89:0/2729917248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e440 0x7f8840134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.738+0000 7f884528e700 1 -- 172.21.15.89:0/2729917248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e440 msgr2=0x7f8840134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.738+0000 7f884528e700 1 --2- 172.21.15.89:0/2729917248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e440 0x7f8840134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.738+0000 7f884528e700 1 -- 172.21.15.89:0/2729917248 >> v1:172.21.15.89:6789/0 conn(0x7f884009e000 legacy=0x7f88401339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.738+0000 7f884528e700 1 -- 172.21.15.89:0/2729917248 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884009a500 con 0x7f884009d6f0 2023-09-30T22:21:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.738+0000 7f884528e700 1 --2- 172.21.15.89:0/2729917248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009d6f0 0x7f884009dac0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f883c009920 tx=0x7f883c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717a8730809aa22a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.738+0000 7f8837fff700 1 -- 172.21.15.89:0/2729917248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c011830 con 0x7f884009d6f0 2023-09-30T22:21:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.738+0000 7f8837fff700 1 -- 172.21.15.89:0/2729917248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c011990 con 0x7f884009d6f0 2023-09-30T22:21:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.738+0000 7f8837fff700 1 -- 172.21.15.89:0/2729917248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c011d20 con 0x7f884009d6f0 2023-09-30T22:21:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.739+0000 7f88474f2700 1 -- 172.21.15.89:0/2729917248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009d6f0 msgr2=0x7f884009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.739+0000 7f88474f2700 1 --2- 172.21.15.89:0/2729917248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009d6f0 0x7f884009dac0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f883c009920 tx=0x7f883c009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.739+0000 7f88474f2700 1 -- 172.21.15.89:0/2729917248 shutdown_connections 2023-09-30T22:21:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.739+0000 7f88474f2700 1 --2- 172.21.15.89:0/2729917248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009d6f0 0x7f884009dac0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.739+0000 7f88474f2700 1 --2- 172.21.15.89:0/2729917248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e440 0x7f8840134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.739+0000 7f88474f2700 1 -- 172.21.15.89:0/2729917248 >> 172.21.15.89:0/2729917248 conn(0x7f8840066f50 msgr2=0x7f884009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.739+0000 7f88474f2700 1 -- 172.21.15.89:0/2729917248 shutdown_connections 2023-09-30T22:21:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.739+0000 7f88474f2700 1 -- 172.21.15.89:0/2729917248 wait complete. 2023-09-30T22:21:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.740+0000 7f88474f2700 1 Processor -- start 2023-09-30T22:21:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.740+0000 7fe4afb7a700 1 Processor -- start 2023-09-30T22:21:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.740+0000 7f88474f2700 1 -- start start 2023-09-30T22:21:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.740+0000 7fe4afb7a700 1 -- start start 2023-09-30T22:21:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f88474f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f884009d6f0 0x7f88401a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f88474f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e000 0x7f88401a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f88474f2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009e440 0x7f88401a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f88474f2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88401a73b0 con 0x7f884009e440 2023-09-30T22:21:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f88474f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88401a7a80 con 0x7f884009d6f0 2023-09-30T22:21:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f88474f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88401a8cd0 con 0x7f884009e000 2023-09-30T22:21:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4afb7a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a8133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4afb7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a809e560 0x7fe4a813b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4afb7a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a806f720 con 0x7fe4a81341d0 2023-09-30T22:21:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4afb7a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a8070c80 con 0x7fe4a809e560 2023-09-30T22:21:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4afb7a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a806fce0 con 0x7fe4a81338c0 2023-09-30T22:21:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4ad916700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a8133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4ad115700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe4a81341d0 0x7fe4a809de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42806/0 (socket says 172.21.15.89:42806) 2023-09-30T22:21:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4ad115700 1 -- 172.21.15.89:0/278854559 learned_addr learned my addr 172.21.15.89:0/278854559 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4ae117700 1 --2- 172.21.15.89:0/278854559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a809e560 0x7fe4a813b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f8845a8f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009e440 0x7f88401a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f8845a8f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009e440 0x7f88401a6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42526/0 (socket says 172.21.15.89:42526) 2023-09-30T22:21:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f8845a8f700 1 -- 172.21.15.89:0/2409584712 learned_addr learned my addr 172.21.15.89:0/2409584712 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7f8844a8d700 1 --2- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e000 0x7f88401a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4ae117700 1 -- 172.21.15.89:0/278854559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 msgr2=0x7fe4a8133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4ae117700 1 --2- 172.21.15.89:0/278854559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a8133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4ae117700 1 -- 172.21.15.89:0/278854559 >> v1:172.21.15.89:6789/0 conn(0x7fe4a81341d0 legacy=0x7fe4a809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.741+0000 7fe4ae117700 1 -- 172.21.15.89:0/278854559 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a81336a0 con 0x7fe4a809e560 2023-09-30T22:21:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7fe4ae117700 1 --2- 172.21.15.89:0/278854559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a809e560 0x7fe4a813b860 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009dc0 tx=0x7fe4a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ea46a1a1ddba66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8845a8f700 1 -- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e000 msgr2=0x7f88401a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8845a8f700 1 --2- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e000 0x7f88401a6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8845a8f700 1 -- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f884009d6f0 msgr2=0x7f88401a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8845a8f700 1 --2- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f884009d6f0 0x7f88401a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8845a8f700 1 -- 172.21.15.89:0/2409584712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c009550 con 0x7f884009e440 2023-09-30T22:21:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7fe4ac914700 1 -- 172.21.15.89:0/278854559 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a000e070 con 0x7fe4a809e560 2023-09-30T22:21:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7fe4ac914700 1 -- 172.21.15.89:0/278854559 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a000bb20 con 0x7fe4a809e560 2023-09-30T22:21:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7fe4ac914700 1 -- 172.21.15.89:0/278854559 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0012430 con 0x7fe4a809e560 2023-09-30T22:21:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8845a8f700 1 --2- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009e440 0x7f88401a6980 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8838009dd0 tx=0x7f883800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8835ffb700 1 -- 172.21.15.89:0/2409584712 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838009c40 con 0x7f884009e440 2023-09-30T22:21:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8835ffb700 1 -- 172.21.15.89:0/2409584712 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883800f040 con 0x7f884009e440 2023-09-30T22:21:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88401a96b0 con 0x7f884009e440 2023-09-30T22:21:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f8835ffb700 1 -- 172.21.15.89:0/2409584712 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838014480 con 0x7f884009e440 2023-09-30T22:21:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.742+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88400a2220 con 0x7f884009e440 2023-09-30T22:21:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.743+0000 7fe4afb7a700 1 -- 172.21.15.89:0/278854559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a809e560 msgr2=0x7fe4a813b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.743+0000 7fe4afb7a700 1 --2- 172.21.15.89:0/278854559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a809e560 0x7fe4a813b860 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009dc0 tx=0x7fe4a000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.743+0000 7fe4afb7a700 1 -- 172.21.15.89:0/278854559 shutdown_connections 2023-09-30T22:21:53.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.743+0000 7fe4afb7a700 1 --2- 172.21.15.89:0/278854559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a809e560 0x7fe4a813b860 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.743+0000 7fe4afb7a700 1 --2- 172.21.15.89:0/278854559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a8133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.743+0000 7fe4afb7a700 1 -- 172.21.15.89:0/278854559 >> 172.21.15.89:0/278854559 conn(0x7fe4a80a2aa0 msgr2=0x7fe4a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:53.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.743+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88200052e0 con 0x7f884009e440 2023-09-30T22:21:53.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.744+0000 7fe4afb7a700 1 -- 172.21.15.89:0/278854559 shutdown_connections 2023-09-30T22:21:53.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.744+0000 7fe4afb7a700 1 -- 172.21.15.89:0/278854559 wait complete. 2023-09-30T22:21:53.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.744+0000 7f8835ffb700 1 -- 172.21.15.89:0/2409584712 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88380145e0 con 0x7f884009e440 2023-09-30T22:21:53.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.744+0000 7f8835ffb700 1 -- 172.21.15.89:0/2409584712 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f88380401e0 con 0x7f884009e440 2023-09-30T22:21:53.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.745+0000 7fe4afb7a700 1 Processor -- start 2023-09-30T22:21:53.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.745+0000 7fe4afb7a700 1 -- start start 2023-09-30T22:21:53.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.745+0000 7fe4afb7a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4a809e560 0x7fe4a811f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.745+0000 7fe4afb7a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a811fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.745+0000 7fe4afb7a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a81341d0 0x7fe4a8120340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:53.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.745+0000 7fe4afb7a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81214d0 con 0x7fe4a809e560 2023-09-30T22:21:53.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.745+0000 7fe4afb7a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a8120d20 con 0x7fe4a81341d0 2023-09-30T22:21:53.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.745+0000 7fe4afb7a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a806f960 con 0x7fe4a81338c0 2023-09-30T22:21:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a811fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a811fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60626/0 (socket says 172.21.15.89:60626) 2023-09-30T22:21:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 -- 172.21.15.89:0/162950454 learned_addr learned my addr 172.21.15.89:0/162950454 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad916700 1 --2- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4a809e560 0x7fe4a811f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 -- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a81341d0 msgr2=0x7fe4a8120340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 --2- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a81341d0 0x7fe4a8120340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 -- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4a809e560 msgr2=0x7fe4a811f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 --2- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4a809e560 0x7fe4a811f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 -- 172.21.15.89:0/162950454 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a0009a70 con 0x7fe4a81338c0 2023-09-30T22:21:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4ad115700 1 --2- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a811fe00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe49800c530 tx=0x7fe49800c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe49e7fc700 1 -- 172.21.15.89:0/162950454 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498005d70 con 0x7fe4a81338c0 2023-09-30T22:21:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe49e7fc700 1 -- 172.21.15.89:0/162950454 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4980070d0 con 0x7fe4a81338c0 2023-09-30T22:21:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe49e7fc700 1 -- 172.21.15.89:0/162950454 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498007440 con 0x7fe4a81338c0 2023-09-30T22:21:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a8122e40 con 0x7fe4a81338c0 2023-09-30T22:21:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.747+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a81222d0 con 0x7fe4a81338c0 2023-09-30T22:21:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.748+0000 7fe49e7fc700 1 -- 172.21.15.89:0/162950454 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe4980075a0 con 0x7fe4a81338c0 2023-09-30T22:21:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.748+0000 7fe49e7fc700 1 -- 172.21.15.89:0/162950454 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe498011cf0 con 0x7fe4a81338c0 2023-09-30T22:21:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.750+0000 7f8835ffb700 1 -- 172.21.15.89:0/2409584712 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8838007920 con 0x7f884009e440 2023-09-30T22:21:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.750+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a8074ea0 con 0x7fe4a81338c0 2023-09-30T22:21:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.759+0000 7fe49e7fc700 1 -- 172.21.15.89:0/162950454 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe498017070 con 0x7fe4a81338c0 2023-09-30T22:21:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.884+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9bac0747f0 con 0x7f9bac12c2f0 2023-09-30T22:21:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.885+0000 7f9ba1ffb700 1 -- 172.21.15.89:0/1746736321 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ba8011350 con 0x7f9bac12c2f0 2023-09-30T22:21:53.886 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:53.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 msgr2=0x7f9bac1c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 --2- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac1c5f00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800e910 tx=0x7f9ba800ecd0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 shutdown_connections 2023-09-30T22:21:53.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 --2- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bac12cff0 0x7f9bac1c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 --2- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bac12c2f0 0x7f9bac1c5f00 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 --2- 172.21.15.89:0/1746736321 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9bac12cc20 0x7f9bac1c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 >> 172.21.15.89:0/1746736321 conn(0x7f9bac09f470 msgr2=0x7f9bac126e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:53.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 shutdown_connections 2023-09-30T22:21:53.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.888+0000 7f9bb32d4700 1 -- 172.21.15.89:0/1746736321 wait complete. 2023-09-30T22:21:53.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:53.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.943+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8820005590 con 0x7f884009e440 2023-09-30T22:21:53.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.944+0000 7f8835ffb700 1 -- 172.21.15.89:0/2409584712 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8838007330 con 0x7f884009e440 2023-09-30T22:21:53.945 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009e440 msgr2=0x7f88401a6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 --2- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009e440 0x7f88401a6980 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8838009dd0 tx=0x7f883800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 shutdown_connections 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 --2- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f884009e440 0x7f88401a6980 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 --2- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f884009d6f0 0x7f88401a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 --2- 172.21.15.89:0/2409584712 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f884009e000 0x7f88401a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 >> 172.21.15.89:0/2409584712 conn(0x7f8840066f50 msgr2=0x7f88401bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 shutdown_connections 2023-09-30T22:21:53.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.946+0000 7f88474f2700 1 -- 172.21.15.89:0/2409584712 wait complete. 2023-09-30T22:21:53.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.953+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe4a8123060 con 0x7fe4a81338c0 2023-09-30T22:21:53.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.953+0000 7fe49e7fc700 1 -- 172.21.15.89:0/162950454 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe4a8123060 con 0x7fe4a81338c0 2023-09-30T22:21:53.954 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.956+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 msgr2=0x7fe4a811fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.956+0000 7fe4afb7a700 1 --2- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a811fe00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe49800c530 tx=0x7fe49800c840 comp rx=0 tx=0).stop 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.956+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 shutdown_connections 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.956+0000 7fe4afb7a700 1 --2- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4a81341d0 0x7fe4a8120340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.956+0000 7fe4afb7a700 1 --2- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4a809e560 0x7fe4a811f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.956+0000 7fe4afb7a700 1 --2- 172.21.15.89:0/162950454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4a81338c0 0x7fe4a811fe00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.956+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 >> 172.21.15.89:0/162950454 conn(0x7fe4a80a2aa0 msgr2=0x7fe4a809f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.957+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 shutdown_connections 2023-09-30T22:21:53.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.957+0000 7fe4afb7a700 1 -- 172.21.15.89:0/162950454 wait complete. 2023-09-30T22:21:53.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:53.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.961+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fef3809f6b0 con 0x7fef3806b250 2023-09-30T22:21:53.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.961+0000 7fef2dffb700 1 -- 172.21.15.89:0/1445445136 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef28015070 con 0x7fef3806b250 2023-09-30T22:21:53.962 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:53.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 msgr2=0x7fef381c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:53.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef381c4a50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fef3809ee70 tx=0x7fef28004750 comp rx=0 tx=0).stop 2023-09-30T22:21:53.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 shutdown_connections 2023-09-30T22:21:53.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef38136d80 0x7fef38119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef3806a770 0x7fef381c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 --2- 172.21.15.89:0/1445445136 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef3806b250 0x7fef381c4a50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:53.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 >> 172.21.15.89:0/1445445136 conn(0x7fef38066f50 msgr2=0x7fef38132810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:53.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 shutdown_connections 2023-09-30T22:21:53.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:53.965+0000 7fef3f4f0700 1 -- 172.21.15.89:0/1445445136 wait complete. 2023-09-30T22:21:53.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:53.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:54.907 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:54.960 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:54.975 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:54.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:55.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.054+0000 7ffa04fbb700 1 Processor -- start 2023-09-30T22:21:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.054+0000 7ffa04fbb700 1 -- start start 2023-09-30T22:21:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ffa04fbb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa0012cf60 0x7ffa0006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ffa04fbb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa00130430 0x7ffa00137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ffa04fbb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0012cd90 con 0x7ffa0012f8b0 2023-09-30T22:21:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ffa04fbb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0012d330 con 0x7ffa00130430 2023-09-30T22:21:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ffa04fbb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00071780 con 0x7ffa0012cf60 2023-09-30T22:21:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fdd9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ffa0012f8b0 0x7ffa0012fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42822/0 (socket says 172.21.15.89:42822) 2023-09-30T22:21:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fdd9b700 1 -- 172.21.15.89:0/2893172918 learned_addr learned my addr 172.21.15.89:0/2893172918 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fed9d700 1 --2- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa00130430 0x7ffa00137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fe59c700 1 --2- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa0012cf60 0x7ffa0006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fd59a700 1 -- 172.21.15.89:0/2893172918 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681844045 0 0) 0x7ffa0012cd90 con 0x7ffa0012f8b0 2023-09-30T22:21:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fd59a700 1 -- 172.21.15.89:0/2893172918 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e80035b0 con 0x7ffa0012f8b0 2023-09-30T22:21:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fe59c700 1 -- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa00130430 msgr2=0x7ffa00137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fe59c700 1 --2- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa00130430 0x7ffa00137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fe59c700 1 -- 172.21.15.89:0/2893172918 >> v1:172.21.15.89:6789/0 conn(0x7ffa0012f8b0 legacy=0x7ffa0012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.055+0000 7ff9fe59c700 1 -- 172.21.15.89:0/2893172918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00071a40 con 0x7ffa0012cf60 2023-09-30T22:21:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.056+0000 7ff9fe59c700 1 --2- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa0012cf60 0x7ffa0006f5d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000bb10 tx=0x7ff9f000be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edff552774659a2e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.056+0000 7ff9fd59a700 1 -- 172.21.15.89:0/2893172918 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000db60 con 0x7ffa0012cf60 2023-09-30T22:21:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.056+0000 7ff9fd59a700 1 -- 172.21.15.89:0/2893172918 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f000dcc0 con 0x7ffa0012cf60 2023-09-30T22:21:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.056+0000 7ff9fd59a700 1 -- 172.21.15.89:0/2893172918 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0017610 con 0x7ffa0012cf60 2023-09-30T22:21:55.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.056+0000 7ffa04fbb700 1 -- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa0012cf60 msgr2=0x7ffa0006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.056+0000 7ffa04fbb700 1 --2- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa0012cf60 0x7ffa0006f5d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000bb10 tx=0x7ff9f000be20 comp rx=0 tx=0).stop 2023-09-30T22:21:55.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.057+0000 7ffa04fbb700 1 -- 172.21.15.89:0/2893172918 shutdown_connections 2023-09-30T22:21:55.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.057+0000 7ffa04fbb700 1 --2- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa00130430 0x7ffa00137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.057+0000 7ffa04fbb700 1 --2- 172.21.15.89:0/2893172918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa0012cf60 0x7ffa0006f5d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.057+0000 7ffa04fbb700 1 -- 172.21.15.89:0/2893172918 >> 172.21.15.89:0/2893172918 conn(0x7ffa000a2aa0 msgr2=0x7ffa000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.057+0000 7ffa04fbb700 1 -- 172.21.15.89:0/2893172918 shutdown_connections 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.057+0000 7ffa04fbb700 1 -- 172.21.15.89:0/2893172918 wait complete. 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.058+0000 7ffa04fbb700 1 Processor -- start 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.058+0000 7ffa04fbb700 1 -- start start 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.058+0000 7ffa04fbb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa0012cf60 0x7ffa001c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.058+0000 7ffa04fbb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa0012f8b0 0x7ffa001c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.058+0000 7ffa04fbb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa00130430 0x7ffa001c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.058+0000 7ffa04fbb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0009ce50 con 0x7ffa0012f8b0 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.058+0000 7ffa04fbb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0009e130 con 0x7ffa0012cf60 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.058+0000 7ffa04fbb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa001c5f20 con 0x7ffa00130430 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fe59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa0012cf60 0x7ffa001c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa00130430 0x7ffa001c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fe59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa0012cf60 0x7ffa001c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60596/0 (socket says 172.21.15.89:60596) 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fe59c700 1 -- 172.21.15.89:0/1250044205 learned_addr learned my addr 172.21.15.89:0/1250044205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fdd9b700 1 --2- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa0012f8b0 0x7ffa001c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fed9d700 1 -- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa0012cf60 msgr2=0x7ffa001c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fed9d700 1 --2- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa0012cf60 0x7ffa001c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fed9d700 1 -- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa0012f8b0 msgr2=0x7ffa001c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fed9d700 1 --2- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa0012f8b0 0x7ffa001c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fed9d700 1 -- 172.21.15.89:0/1250044205 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f000b740 con 0x7ffa00130430 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9fed9d700 1 --2- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa00130430 0x7ffa001c84c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8007dc0 tx=0x7ff9f800e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9eeffd700 1 -- 172.21.15.89:0/1250044205 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8018070 con 0x7ffa00130430 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0009e4f0 con 0x7ffa00130430 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9eeffd700 1 -- 172.21.15.89:0/1250044205 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f800eb30 con 0x7ffa00130430 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ff9eeffd700 1 -- 172.21.15.89:0/1250044205 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800eec0 con 0x7ffa00130430 2023-09-30T22:21:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.059+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa00129fa0 con 0x7ffa00130430 2023-09-30T22:21:55.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.060+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0006f930 con 0x7ffa00130430 2023-09-30T22:21:55.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.061+0000 7ff9eeffd700 1 -- 172.21.15.89:0/1250044205 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff9f8003a20 con 0x7ffa00130430 2023-09-30T22:21:55.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.061+0000 7ff9eeffd700 1 -- 172.21.15.89:0/1250044205 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff9f803f670 con 0x7ffa00130430 2023-09-30T22:21:55.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.065+0000 7ff9eeffd700 1 -- 172.21.15.89:0/1250044205 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff9f801b720 con 0x7ffa00130430 2023-09-30T22:21:55.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.096+0000 7f17e733d700 1 Processor -- start 2023-09-30T22:21:55.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.096+0000 7f17e733d700 1 -- start start 2023-09-30T22:21:55.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e733d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17e01338a0 0x7f17e0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e733d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009de50 0x7f17e009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e733d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e006ef30 con 0x7f17e009e760 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e733d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e006fef0 con 0x7f17e01338a0 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e733d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17e01334b0 con 0x7f17e009de50 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e50d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17e01338a0 0x7f17e0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e50d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17e01338a0 0x7f17e0133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42560/0 (socket says 172.21.15.89:42560) 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e50d9700 1 -- 172.21.15.89:0/1444541381 learned_addr learned my addr 172.21.15.89:0/1444541381 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e48d8700 1 --2- 172.21.15.89:0/1444541381 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009de50 0x7f17e009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e50d9700 1 -- 172.21.15.89:0/1444541381 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009de50 msgr2=0x7f17e009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e50d9700 1 --2- 172.21.15.89:0/1444541381 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009de50 0x7f17e009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.097+0000 7f17e50d9700 1 -- 172.21.15.89:0/1444541381 >> v1:172.21.15.89:6789/0 conn(0x7f17e009e760 legacy=0x7f17e013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.098+0000 7f17d7fff700 1 -- 172.21.15.89:0/1444541381 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936203909 0 0) 0x7f17e006ef30 con 0x7f17e009e760 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.098+0000 7f17e50d9700 1 -- 172.21.15.89:0/1444541381 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e006fc30 con 0x7f17e01338a0 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.098+0000 7f17e50d9700 1 --2- 172.21.15.89:0/1444541381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 0x7f17e0133c70 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0098f0 tx=0x7f17dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2694c1cf462d9db5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.098+0000 7f17d7fff700 1 -- 172.21.15.89:0/1444541381 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc011830 con 0x7f17e01338a0 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.098+0000 7f17d7fff700 1 -- 172.21.15.89:0/1444541381 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc011990 con 0x7f17e01338a0 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.098+0000 7f17d7fff700 1 -- 172.21.15.89:0/1444541381 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc011d20 con 0x7f17e01338a0 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.099+0000 7f17e733d700 1 -- 172.21.15.89:0/1444541381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 msgr2=0x7f17e0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.099+0000 7f17e733d700 1 --2- 172.21.15.89:0/1444541381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 0x7f17e0133c70 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0098f0 tx=0x7f17dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:55.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.099+0000 7f17e733d700 1 -- 172.21.15.89:0/1444541381 shutdown_connections 2023-09-30T22:21:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.099+0000 7f17e733d700 1 --2- 172.21.15.89:0/1444541381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 0x7f17e0133c70 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.099+0000 7f17e733d700 1 --2- 172.21.15.89:0/1444541381 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009de50 0x7f17e009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.099+0000 7f17e733d700 1 -- 172.21.15.89:0/1444541381 >> 172.21.15.89:0/1444541381 conn(0x7f17e00a2aa0 msgr2=0x7f17e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.099+0000 7f17e733d700 1 -- 172.21.15.89:0/1444541381 shutdown_connections 2023-09-30T22:21:55.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.099+0000 7f17e733d700 1 -- 172.21.15.89:0/1444541381 wait complete. 2023-09-30T22:21:55.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e733d700 1 Processor -- start 2023-09-30T22:21:55.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e733d700 1 -- start start 2023-09-30T22:21:55.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e733d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009de50 0x7f17e011b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e733d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e760 0x7f17e011bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e733d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 0x7f17e011c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e733d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e011cb40 con 0x7f17e01338a0 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e733d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e011d850 con 0x7f17e009de50 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e733d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17e011eba0 con 0x7f17e009e760 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e50d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009de50 0x7f17e011b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e50d9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009de50 0x7f17e011b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60610/0 (socket says 172.21.15.89:60610) 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e50d9700 1 -- 172.21.15.89:0/2369431687 learned_addr learned my addr 172.21.15.89:0/2369431687 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.100+0000 7f17e48d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e760 0x7f17e011bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17e58da700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 0x7f17e011c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17e50d9700 1 -- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e760 msgr2=0x7f17e011bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17e50d9700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e760 0x7f17e011bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17e50d9700 1 -- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 msgr2=0x7f17e011c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17e50d9700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 0x7f17e011c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17e50d9700 1 -- 172.21.15.89:0/2369431687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc009550 con 0x7f17e009de50 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17e58da700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 0x7f17e011c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17e50d9700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009de50 0x7f17e011b690 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009c40 tx=0x7f17dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17d5ffb700 1 -- 172.21.15.89:0/2369431687 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0045d0 con 0x7f17e009de50 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17d5ffb700 1 -- 172.21.15.89:0/2369431687 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc01f070 con 0x7f17e009de50 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.101+0000 7f17d5ffb700 1 -- 172.21.15.89:0/2369431687 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0038c0 con 0x7f17e009de50 2023-09-30T22:21:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.102+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e00a0c70 con 0x7f17e009de50 2023-09-30T22:21:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.102+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e011f140 con 0x7f17e009de50 2023-09-30T22:21:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.103+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e0132a40 con 0x7f17e009de50 2023-09-30T22:21:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.103+0000 7f17d5ffb700 1 -- 172.21.15.89:0/2369431687 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f17dc003a20 con 0x7f17e009de50 2023-09-30T22:21:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.103+0000 7f17d5ffb700 1 -- 172.21.15.89:0/2369431687 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f17dc021070 con 0x7f17e009de50 2023-09-30T22:21:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.112+0000 7f17d5ffb700 1 -- 172.21.15.89:0/2369431687 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17dc02f3a0 con 0x7f17e009de50 2023-09-30T22:21:55.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.143+0000 7f50422fd700 1 Processor -- start 2023-09-30T22:21:55.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.143+0000 7f50422fd700 1 -- start start 2023-09-30T22:21:55.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f50422fd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f503c12cf60 0x7f503c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f50422fd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f50422fd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c12cd90 con 0x7f503c12f8b0 2023-09-30T22:21:55.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f50422fd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503c12d330 con 0x7f503c12cf60 2023-09-30T22:21:55.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f50422fd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f503c071780 con 0x7f503c130430 2023-09-30T22:21:55.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f503bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f503c12cf60 0x7f503c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f503bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f503c12cf60 0x7f503c06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42582/0 (socket says 172.21.15.89:42582) 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f503bfff700 1 -- 172.21.15.89:0/46413706 learned_addr learned my addr 172.21.15.89:0/46413706 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f504089a700 1 --2- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f503bfff700 1 -- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 msgr2=0x7f503c137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f503bfff700 1 --2- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f503bfff700 1 -- 172.21.15.89:0/46413706 >> v1:172.21.15.89:6789/0 conn(0x7f503c12f8b0 legacy=0x7f503c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f503bfff700 1 -- 172.21.15.89:0/46413706 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c071a40 con 0x7f503c12cf60 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.144+0000 7f504089a700 1 --2- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.145+0000 7f503bfff700 1 --2- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c06f5d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f502c00bb90 tx=0x7f502c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f44a4804a070906 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.145+0000 7f503affd700 1 -- 172.21.15.89:0/46413706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c00dc20 con 0x7f503c12cf60 2023-09-30T22:21:55.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.145+0000 7f503affd700 1 -- 172.21.15.89:0/46413706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c00dd80 con 0x7f503c12cf60 2023-09-30T22:21:55.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.145+0000 7f503affd700 1 -- 172.21.15.89:0/46413706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c017630 con 0x7f503c12cf60 2023-09-30T22:21:55.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.146+0000 7f50422fd700 1 -- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 msgr2=0x7f503c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.146+0000 7f50422fd700 1 --2- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c06f5d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f502c00bb90 tx=0x7f502c00bea0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.146+0000 7f50422fd700 1 -- 172.21.15.89:0/46413706 shutdown_connections 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.146+0000 7f50422fd700 1 --2- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c06f5d0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.146+0000 7f50422fd700 1 --2- 172.21.15.89:0/46413706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.146+0000 7f50422fd700 1 -- 172.21.15.89:0/46413706 >> 172.21.15.89:0/46413706 conn(0x7f503c0a2aa0 msgr2=0x7f503c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.146+0000 7f50422fd700 1 -- 172.21.15.89:0/46413706 shutdown_connections 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.146+0000 7f50422fd700 1 -- 172.21.15.89:0/46413706 wait complete. 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.147+0000 7f50422fd700 1 Processor -- start 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.147+0000 7f50422fd700 1 -- start start 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f50422fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c1c1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f50422fd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f503c12f8b0 0x7f503c1c1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f50422fd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c1c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f50422fd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c12f520 con 0x7f503c12cf60 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f50422fd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c09ea10 con 0x7f503c12f8b0 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f50422fd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f503c1c1dc0 con 0x7f503c130430 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c1c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c1c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42588/0 (socket says 172.21.15.89:42588) 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 -- 172.21.15.89:0/3742556367 learned_addr learned my addr 172.21.15.89:0/3742556367 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f504089a700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c1c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 -- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 msgr2=0x7f503c1c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c1c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 -- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f503c12f8b0 msgr2=0x7f503c1c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f503c12f8b0 0x7f503c1c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 -- 172.21.15.89:0/3742556367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5034009a70 con 0x7f503c12cf60 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f504089a700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c1c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.148+0000 7f503bfff700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c1c1340 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d6d0 tx=0x7f502c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7fa366235700 1 Processor -- start 2023-09-30T22:21:55.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7fa366235700 1 -- start start 2023-09-30T22:21:55.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7f5038ff9700 1 -- 172.21.15.89:0/3742556367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c017c30 con 0x7f503c12cf60 2023-09-30T22:21:55.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7f5038ff9700 1 -- 172.21.15.89:0/3742556367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c017d90 con 0x7f503c12cf60 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c00b7f0 con 0x7f503c12cf60 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7f5038ff9700 1 -- 172.21.15.89:0/3742556367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0038e0 con 0x7f503c12cf60 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7fa366235700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3601316c0 0x7fa360131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7fa366235700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa36009d9f0 0x7fa36006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7fa366235700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36009e130 con 0x7fa36006acb0 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7fa366235700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3601312b0 con 0x7fa36009d9f0 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7fa366235700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa36009f6e0 con 0x7fa3601316c0 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.149+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c09d010 con 0x7f503c12cf60 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3601316c0 0x7fa360131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3601316c0 0x7fa360131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60694/0 (socket says 172.21.15.89:60694) 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35f7fe700 1 -- 172.21.15.89:0/1203316632 learned_addr learned my addr 172.21.15.89:0/1203316632 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35f7fe700 1 -- 172.21.15.89:0/1203316632 >> v2:172.21.15.89:3300/0 conn(0x7fa36009d9f0 msgr2=0x7fa36006a770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35f7fe700 1 --2- 172.21.15.89:0/1203316632 >> v2:172.21.15.89:3300/0 conn(0x7fa36009d9f0 0x7fa36006a770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35f7fe700 1 -- 172.21.15.89:0/1203316632 >> v1:172.21.15.89:6789/0 conn(0x7fa36006acb0 legacy=0x7fa360138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:55.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35f7fe700 1 -- 172.21.15.89:0/1203316632 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36009e760 con 0x7fa3601316c0 2023-09-30T22:21:55.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35f7fe700 1 --2- 172.21.15.89:0/1203316632 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3601316c0 0x7fa360131a90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa358009e50 tx=0x7fa35800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7018eca6284f1536 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c09e280 con 0x7f503c12cf60 2023-09-30T22:21:55.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35e7fc700 1 -- 172.21.15.89:0/1203316632 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35800e070 con 0x7fa3601316c0 2023-09-30T22:21:55.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35e7fc700 1 -- 172.21.15.89:0/1203316632 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa358003a20 con 0x7fa3601316c0 2023-09-30T22:21:55.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.150+0000 7fa35e7fc700 1 -- 172.21.15.89:0/1203316632 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358003dd0 con 0x7fa3601316c0 2023-09-30T22:21:55.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.151+0000 7f5038ff9700 1 -- 172.21.15.89:0/3742556367 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f502c015070 con 0x7f503c12cf60 2023-09-30T22:21:55.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.152+0000 7f5038ff9700 1 -- 172.21.15.89:0/3742556367 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f502c043d10 con 0x7f503c12cf60 2023-09-30T22:21:55.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.151+0000 7fa366235700 1 -- 172.21.15.89:0/1203316632 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3601316c0 msgr2=0x7fa360131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.151+0000 7fa366235700 1 --2- 172.21.15.89:0/1203316632 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3601316c0 0x7fa360131a90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa358009e50 tx=0x7fa35800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.151+0000 7fa366235700 1 -- 172.21.15.89:0/1203316632 shutdown_connections 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.151+0000 7fa366235700 1 --2- 172.21.15.89:0/1203316632 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3601316c0 0x7fa360131a90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.151+0000 7fa366235700 1 --2- 172.21.15.89:0/1203316632 >> v2:172.21.15.89:3300/0 conn(0x7fa36009d9f0 0x7fa36006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.151+0000 7fa366235700 1 -- 172.21.15.89:0/1203316632 >> 172.21.15.89:0/1203316632 conn(0x7fa360066f50 msgr2=0x7fa36009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.152+0000 7fa366235700 1 -- 172.21.15.89:0/1203316632 shutdown_connections 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.152+0000 7fa366235700 1 -- 172.21.15.89:0/1203316632 wait complete. 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.153+0000 7fa366235700 1 Processor -- start 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.153+0000 7fa366235700 1 -- start start 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa366235700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa36006acb0 0x7fa360071980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa366235700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa36009d9f0 0x7fa360071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa366235700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3601316c0 0x7fa3601c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa366235700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36012f3c0 con 0x7fa36006acb0 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa366235700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36012b900 con 0x7fa3601316c0 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa366235700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa360133f50 con 0x7fa36009d9f0 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa36009d9f0 0x7fa360071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa36009d9f0 0x7fa360071ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60698/0 (socket says 172.21.15.89:60698) 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35effd700 1 -- 172.21.15.89:0/1914203139 learned_addr learned my addr 172.21.15.89:0/1914203139 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35f7fe700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa36006acb0 0x7fa360071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35f7fe700 1 -- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa36009d9f0 msgr2=0x7fa360071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35f7fe700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa36009d9f0 0x7fa360071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35f7fe700 1 -- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3601316c0 msgr2=0x7fa3601c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35f7fe700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3601316c0 0x7fa3601c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35f7fe700 1 -- 172.21.15.89:0/1914203139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358009b30 con 0x7fa36006acb0 2023-09-30T22:21:55.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.154+0000 7fa35effd700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa36009d9f0 0x7fa360071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:55.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.155+0000 7fa35f7fe700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa36006acb0 0x7fa360071980 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa358014000 tx=0x7fa358004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.157+0000 7fa347fff700 1 -- 172.21.15.89:0/1914203139 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358007000 con 0x7fa36006acb0 2023-09-30T22:21:55.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.157+0000 7fa347fff700 1 -- 172.21.15.89:0/1914203139 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa358004480 con 0x7fa36006acb0 2023-09-30T22:21:55.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.157+0000 7fa347fff700 1 -- 172.21.15.89:0/1914203139 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358004830 con 0x7fa36006acb0 2023-09-30T22:21:55.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.157+0000 7f5038ff9700 1 -- 172.21.15.89:0/3742556367 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f502c01f990 con 0x7f503c12cf60 2023-09-30T22:21:55.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.157+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36009ead0 con 0x7fa36006acb0 2023-09-30T22:21:55.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.157+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3601325e0 con 0x7fa36006acb0 2023-09-30T22:21:55.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.158+0000 7fa347fff700 1 -- 172.21.15.89:0/1914203139 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa358012430 con 0x7fa36006acb0 2023-09-30T22:21:55.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.158+0000 7fa347fff700 1 -- 172.21.15.89:0/1914203139 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa358040e10 con 0x7fa36006acb0 2023-09-30T22:21:55.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.158+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36009e130 con 0x7fa36006acb0 2023-09-30T22:21:55.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.163+0000 7fa347fff700 1 -- 172.21.15.89:0/1914203139 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa36009e130 con 0x7fa36006acb0 2023-09-30T22:21:55.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.305+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffa00074ea0 con 0x7ffa00130430 2023-09-30T22:21:55.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.306+0000 7ff9eeffd700 1 -- 172.21.15.89:0/1250044205 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff9f801c370 con 0x7ffa00130430 2023-09-30T22:21:55.308 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:55.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f17e0074ea0 con 0x7f17e009de50 2023-09-30T22:21:55.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa00130430 msgr2=0x7ffa001c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 --2- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa00130430 0x7ffa001c84c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8007dc0 tx=0x7ff9f800e3c0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 shutdown_connections 2023-09-30T22:21:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 --2- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa00130430 0x7ffa001c84c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 --2- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa0012cf60 0x7ffa001c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 --2- 172.21.15.89:0/1250044205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa0012f8b0 0x7ffa001c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 >> 172.21.15.89:0/1250044205 conn(0x7ffa000a2aa0 msgr2=0x7ffa0006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 shutdown_connections 2023-09-30T22:21:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7ffa04fbb700 1 -- 172.21.15.89:0/1250044205 wait complete. 2023-09-30T22:21:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.308+0000 7f17d5ffb700 1 -- 172.21.15.89:0/2369431687 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17dc02b030 con 0x7f17e009de50 2023-09-30T22:21:55.311 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:55.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.312+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009de50 msgr2=0x7f17e011b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.312+0000 7f17e733d700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009de50 0x7f17e011b690 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009c40 tx=0x7f17dc004750 comp rx=0 tx=0).stop 2023-09-30T22:21:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.313+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 shutdown_connections 2023-09-30T22:21:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.313+0000 7f17e733d700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01338a0 0x7f17e011c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.313+0000 7f17e733d700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17e009de50 0x7f17e011b690 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.313+0000 7f17e733d700 1 --2- 172.21.15.89:0/2369431687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17e009e760 0x7f17e011bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.313+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 >> 172.21.15.89:0/2369431687 conn(0x7f17e00a2aa0 msgr2=0x7f17e0130630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.314+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 shutdown_connections 2023-09-30T22:21:55.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.314+0000 7f17e733d700 1 -- 172.21.15.89:0/2369431687 wait complete. 2023-09-30T22:21:55.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:55.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:55.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.351+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f503c06f7e0 con 0x7f503c12cf60 2023-09-30T22:21:55.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.351+0000 7f5038ff9700 1 -- 172.21.15.89:0/3742556367 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f502c038070 con 0x7f503c12cf60 2023-09-30T22:21:55.352 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:55.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.353+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 msgr2=0x7f503c1c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.353+0000 7f50422fd700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c1c1340 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d6d0 tx=0x7f502c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:55.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.353+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 shutdown_connections 2023-09-30T22:21:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.353+0000 7f50422fd700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f503c130430 0x7f503c1c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.353+0000 7f50422fd700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f503c12cf60 0x7f503c1c1340 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.353+0000 7f50422fd700 1 --2- 172.21.15.89:0/3742556367 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f503c12f8b0 0x7f503c1c1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.353+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 >> 172.21.15.89:0/3742556367 conn(0x7f503c0a2aa0 msgr2=0x7f503c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.354+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 shutdown_connections 2023-09-30T22:21:55.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.354+0000 7f50422fd700 1 -- 172.21.15.89:0/3742556367 wait complete. 2023-09-30T22:21:55.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.394+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa36009e130 con 0x7fa36006acb0 2023-09-30T22:21:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.394+0000 7fa347fff700 1 -- 172.21.15.89:0/1914203139 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa36009e130 con 0x7fa36006acb0 2023-09-30T22:21:55.396 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa36006acb0 msgr2=0x7fa360071980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa36006acb0 0x7fa360071980 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa358014000 tx=0x7fa358004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 shutdown_connections 2023-09-30T22:21:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3601316c0 0x7fa3601c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa36006acb0 0x7fa360071980 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 --2- 172.21.15.89:0/1914203139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa36009d9f0 0x7fa360071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 >> 172.21.15.89:0/1914203139 conn(0x7fa360066f50 msgr2=0x7fa3600a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 shutdown_connections 2023-09-30T22:21:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:55.396+0000 7fa366235700 1 -- 172.21.15.89:0/1914203139 wait complete. 2023-09-30T22:21:55.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:56.323 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:56.334 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:56.367 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:56.410 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.459+0000 7f908301b700 1 Processor -- start 2023-09-30T22:21:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.459+0000 7f908301b700 1 -- start start 2023-09-30T22:21:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.459+0000 7f908301b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c12cf60 0x7f907c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.459+0000 7f908301b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f907c130430 0x7f907c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.459+0000 7f908301b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907c12cd90 con 0x7f907c12f8b0 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.459+0000 7f908301b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907c12d330 con 0x7f907c130430 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.459+0000 7f908301b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f907c071780 con 0x7f907c12cf60 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f9080db7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c12cf60 0x7f907c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f9080db7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c12cf60 0x7f907c06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60702/0 (socket says 172.21.15.89:60702) 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f9080db7700 1 -- 172.21.15.89:0/3848233523 learned_addr learned my addr 172.21.15.89:0/3848233523 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f90815b8700 1 --2- 172.21.15.89:0/3848233523 >> v2:172.21.15.89:3300/0 conn(0x7f907c130430 0x7f907c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f9080db7700 1 -- 172.21.15.89:0/3848233523 >> v2:172.21.15.89:3300/0 conn(0x7f907c130430 msgr2=0x7f907c137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f9080db7700 1 --2- 172.21.15.89:0/3848233523 >> v2:172.21.15.89:3300/0 conn(0x7f907c130430 0x7f907c137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f9080db7700 1 -- 172.21.15.89:0/3848233523 >> v1:172.21.15.89:6789/0 conn(0x7f907c12f8b0 legacy=0x7f907c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f9080db7700 1 -- 172.21.15.89:0/3848233523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c071a40 con 0x7f907c12cf60 2023-09-30T22:21:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f9080db7700 1 --2- 172.21.15.89:0/3848233523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c12cf60 0x7f907c06f5d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f90700098f0 tx=0x7f9070009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406d9253b18e4f52 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f907b7fe700 1 -- 172.21.15.89:0/3848233523 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070004230 con 0x7f907c12cf60 2023-09-30T22:21:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f907b7fe700 1 -- 172.21.15.89:0/3848233523 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9070004390 con 0x7f907c12cf60 2023-09-30T22:21:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.460+0000 7f907b7fe700 1 -- 172.21.15.89:0/3848233523 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90700036a0 con 0x7f907c12cf60 2023-09-30T22:21:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.461+0000 7f908301b700 1 -- 172.21.15.89:0/3848233523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c12cf60 msgr2=0x7f907c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.461+0000 7f908301b700 1 --2- 172.21.15.89:0/3848233523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c12cf60 0x7f907c06f5d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f90700098f0 tx=0x7f9070009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:56.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.461+0000 7f908301b700 1 -- 172.21.15.89:0/3848233523 shutdown_connections 2023-09-30T22:21:56.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.461+0000 7f908301b700 1 --2- 172.21.15.89:0/3848233523 >> v2:172.21.15.89:3300/0 conn(0x7f907c130430 0x7f907c137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.461+0000 7f908301b700 1 --2- 172.21.15.89:0/3848233523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c12cf60 0x7f907c06f5d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.461+0000 7f908301b700 1 -- 172.21.15.89:0/3848233523 >> 172.21.15.89:0/3848233523 conn(0x7f907c0a2aa0 msgr2=0x7f907c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.461+0000 7f908301b700 1 -- 172.21.15.89:0/3848233523 shutdown_connections 2023-09-30T22:21:56.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.461+0000 7f908301b700 1 -- 172.21.15.89:0/3848233523 wait complete. 2023-09-30T22:21:56.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.462+0000 7f908301b700 1 Processor -- start 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.462+0000 7f908301b700 1 -- start start 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f908301b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f907c12cf60 0x7f907c11a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f908301b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c12f8b0 0x7f907c11ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f908301b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c130430 0x7f907c11b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f908301b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c11bb00 con 0x7f907c12f8b0 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f908301b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c11ded0 con 0x7f907c12cf60 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f908301b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f907c11e0d0 con 0x7f907c130430 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f9080db7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f907c12cf60 0x7f907c11a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f9080db7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f907c12cf60 0x7f907c11a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60640/0 (socket says 172.21.15.89:60640) 2023-09-30T22:21:56.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f9080db7700 1 -- 172.21.15.89:0/1510693478 learned_addr learned my addr 172.21.15.89:0/1510693478 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f90815b8700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c130430 0x7f907c11b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.463+0000 7f907bfff700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c12f8b0 0x7f907c11ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90815b8700 1 -- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f907c12cf60 msgr2=0x7f907c11a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90815b8700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f907c12cf60 0x7f907c11a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90815b8700 1 -- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c12f8b0 msgr2=0x7f907c11ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90815b8700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c12f8b0 0x7f907c11ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90815b8700 1 -- 172.21.15.89:0/1510693478 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070009550 con 0x7f907c130430 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f907bfff700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c12f8b0 0x7f907c11ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:56.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f9080db7700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f907c12cf60 0x7f907c11a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90815b8700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c130430 0x7f907c11b0d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9074007c80 tx=0x7f9074007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90797fa700 1 -- 172.21.15.89:0/1510693478 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074017070 con 0x7f907c130430 2023-09-30T22:21:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c11cfc0 con 0x7f907c130430 2023-09-30T22:21:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90797fa700 1 -- 172.21.15.89:0/1510693478 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907400a2e0 con 0x7f907c130430 2023-09-30T22:21:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f90797fa700 1 -- 172.21.15.89:0/1510693478 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907400f040 con 0x7f907c130430 2023-09-30T22:21:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.464+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c11c2b0 con 0x7f907c130430 2023-09-30T22:21:56.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.465+0000 7f90797fa700 1 -- 172.21.15.89:0/1510693478 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f907400d6d0 con 0x7f907c130430 2023-09-30T22:21:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.465+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907c09d700 con 0x7f907c130430 2023-09-30T22:21:56.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.465+0000 7f90797fa700 1 -- 172.21.15.89:0/1510693478 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f907403f1e0 con 0x7f907c130430 2023-09-30T22:21:56.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.470+0000 7f90797fa700 1 -- 172.21.15.89:0/1510693478 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f907401a310 con 0x7f907c130430 2023-09-30T22:21:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.502+0000 7f500d658700 1 Processor -- start 2023-09-30T22:21:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.502+0000 7f500d658700 1 -- start start 2023-09-30T22:21:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.502+0000 7f2b416d0700 1 Processor -- start 2023-09-30T22:21:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.502+0000 7f2b416d0700 1 -- start start 2023-09-30T22:21:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f500d658700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50080a1510 0x7f5008138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f500d658700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5008139250 0x7f500813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f500d658700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500809edd0 con 0x7f50080a0a30 2023-09-30T22:21:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f500d658700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50080a18e0 con 0x7f5008139250 2023-09-30T22:21:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f500d658700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f500809e1f0 con 0x7f50080a1510 2023-09-30T22:21:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b416d0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b3c09e930 0x7f2b3c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b416d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09f240 0x7f2b3c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b416d0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3c070940 con 0x7f2b3c09f650 2023-09-30T22:21:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b416d0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3c06fd60 con 0x7f2b3c09e930 2023-09-30T22:21:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b416d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c06fa80 con 0x7f2b3c09f240 2023-09-30T22:21:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b3b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2b3c09f650 0x7f2b3c131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42878/0 (socket says 172.21.15.89:42878) 2023-09-30T22:21:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b3affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b3c09e930 0x7f2b3c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b3b7fe700 1 -- 172.21.15.89:0/721602664 learned_addr learned my addr 172.21.15.89:0/721602664 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f50077fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5008139250 0x7f500813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f50077fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5008139250 0x7f500813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42624/0 (socket says 172.21.15.89:42624) 2023-09-30T22:21:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f50077fe700 1 -- 172.21.15.89:0/3285808115 learned_addr learned my addr 172.21.15.89:0/3285808115 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f50077fe700 1 -- 172.21.15.89:0/3285808115 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50080a1510 msgr2=0x7f5008138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f50077fe700 1 --2- 172.21.15.89:0/3285808115 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50080a1510 0x7f5008138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f50077fe700 1 -- 172.21.15.89:0/3285808115 >> v1:172.21.15.89:6789/0 conn(0x7f50080a0a30 legacy=0x7f50080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f50077fe700 1 -- 172.21.15.89:0/3285808115 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500809dc50 con 0x7f5008139250 2023-09-30T22:21:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b3affd700 1 -- 172.21.15.89:0/721602664 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09f240 msgr2=0x7f2b3c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b3affd700 1 --2- 172.21.15.89:0/721602664 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09f240 0x7f2b3c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b3affd700 1 -- 172.21.15.89:0/721602664 >> v1:172.21.15.89:6789/0 conn(0x7f2b3c09f650 legacy=0x7f2b3c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.503+0000 7f2b3affd700 1 -- 172.21.15.89:0/721602664 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c09fb60 con 0x7f2b3c09e930 2023-09-30T22:21:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.504+0000 7f50077fe700 1 --2- 172.21.15.89:0/3285808115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5008139250 0x7f500813b630 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f5000009e80 tx=0x7f500000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61b55dec0db0983 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.504+0000 7f2b3affd700 1 --2- 172.21.15.89:0/721602664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09e930 0x7f2b3c09ed00 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009840 tx=0x7f2b2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce31d59a9ab93227 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.504+0000 7f5005ffb700 1 -- 172.21.15.89:0/3285808115 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500000e070 con 0x7f5008139250 2023-09-30T22:21:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.504+0000 7f2b39ffb700 1 -- 172.21.15.89:0/721602664 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c011830 con 0x7f2b3c09e930 2023-09-30T22:21:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.504+0000 7f2b39ffb700 1 -- 172.21.15.89:0/721602664 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c011990 con 0x7f2b3c09e930 2023-09-30T22:21:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.504+0000 7f2b39ffb700 1 -- 172.21.15.89:0/721602664 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c011d20 con 0x7f2b3c09e930 2023-09-30T22:21:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.504+0000 7f5005ffb700 1 -- 172.21.15.89:0/3285808115 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500000b980 con 0x7f5008139250 2023-09-30T22:21:56.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.504+0000 7f5005ffb700 1 -- 172.21.15.89:0/3285808115 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500000bd30 con 0x7f5008139250 2023-09-30T22:21:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f2b416d0700 1 -- 172.21.15.89:0/721602664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09e930 msgr2=0x7f2b3c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f2b416d0700 1 --2- 172.21.15.89:0/721602664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09e930 0x7f2b3c09ed00 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009840 tx=0x7f2b2c009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f500d658700 1 -- 172.21.15.89:0/3285808115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5008139250 msgr2=0x7f500813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f500d658700 1 --2- 172.21.15.89:0/3285808115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5008139250 0x7f500813b630 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f5000009e80 tx=0x7f500000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f500d658700 1 -- 172.21.15.89:0/3285808115 shutdown_connections 2023-09-30T22:21:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f500d658700 1 --2- 172.21.15.89:0/3285808115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5008139250 0x7f500813b630 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f500d658700 1 --2- 172.21.15.89:0/3285808115 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50080a1510 0x7f5008138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f500d658700 1 -- 172.21.15.89:0/3285808115 >> 172.21.15.89:0/3285808115 conn(0x7f50080a6550 msgr2=0x7f50080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f2b416d0700 1 -- 172.21.15.89:0/721602664 shutdown_connections 2023-09-30T22:21:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f2b416d0700 1 --2- 172.21.15.89:0/721602664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09e930 0x7f2b3c09ed00 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f2b416d0700 1 --2- 172.21.15.89:0/721602664 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09f240 0x7f2b3c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f2b416d0700 1 -- 172.21.15.89:0/721602664 >> 172.21.15.89:0/721602664 conn(0x7f2b3c0a4210 msgr2=0x7f2b3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f500d658700 1 -- 172.21.15.89:0/3285808115 shutdown_connections 2023-09-30T22:21:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f500d658700 1 -- 172.21.15.89:0/3285808115 wait complete. 2023-09-30T22:21:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.505+0000 7f2b416d0700 1 -- 172.21.15.89:0/721602664 shutdown_connections 2023-09-30T22:21:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.506+0000 7f500d658700 1 Processor -- start 2023-09-30T22:21:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.506+0000 7f500d658700 1 -- start start 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f500d658700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50080a0a30 0x7f50081c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f500d658700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50080a1510 0x7f50081c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f500d658700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5008139250 0x7f50081216f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f500d658700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008125210 con 0x7f5008139250 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f500d658700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50081be330 con 0x7f50080a1510 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f500d658700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50081375c0 con 0x7f50080a0a30 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50080a1510 0x7f50081c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50080a1510 0x7f50081c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60658/0 (socket says 172.21.15.89:60658) 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 -- 172.21.15.89:0/201801720 learned_addr learned my addr 172.21.15.89:0/201801720 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 -- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50080a0a30 msgr2=0x7f50081c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 --2- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50080a0a30 0x7f50081c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 -- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5008139250 msgr2=0x7f50081216f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:21:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 --2- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5008139250 0x7f50081216f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 -- 172.21.15.89:0/201801720 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000009b30 con 0x7f50080a1510 2023-09-30T22:21:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f50067fc700 1 --2- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50080a1510 0x7f50081c8c30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00b530 tx=0x7f4ffc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.506+0000 7f2b416d0700 1 -- 172.21.15.89:0/721602664 wait complete. 2023-09-30T22:21:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f2b416d0700 1 Processor -- start 2023-09-30T22:21:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f4fef7fe700 1 -- 172.21.15.89:0/201801720 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc011830 con 0x7f50080a1510 2023-09-30T22:21:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f4fef7fe700 1 -- 172.21.15.89:0/201801720 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc011990 con 0x7f50080a1510 2023-09-30T22:21:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f4fef7fe700 1 -- 172.21.15.89:0/201801720 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc011d40 con 0x7f50080a1510 2023-09-30T22:21:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5008138ad0 con 0x7f50080a1510 2023-09-30T22:21:56.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5008123300 con 0x7f50080a1510 2023-09-30T22:21:56.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f4fef7fe700 1 -- 172.21.15.89:0/201801720 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ffc01a400 con 0x7f50080a1510 2023-09-30T22:21:56.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f4fef7fe700 1 -- 172.21.15.89:0/201801720 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4ffc03f840 con 0x7f50080a1510 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.507+0000 7f2b416d0700 1 -- start start 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b416d0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09e930 0x7f2b3c1b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b416d0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b3c09f240 0x7f2b3c1b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b416d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09f650 0x7f2b3c1d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b416d0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c1b4f10 con 0x7f2b3c09f650 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b416d0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c1b50f0 con 0x7f2b3c09e930 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b416d0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c1c5f60 con 0x7f2b3c09f240 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b3affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09e930 0x7f2b3c1b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b3affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09e930 0x7f2b3c1b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60670/0 (socket says 172.21.15.89:60670) 2023-09-30T22:21:56.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b3affd700 1 -- 172.21.15.89:0/1396911698 learned_addr learned my addr 172.21.15.89:0/1396911698 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b3a7fc700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b3c09f240 0x7f2b3c1b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.508+0000 7f2b3b7fe700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09f650 0x7f2b3c1d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b3affd700 1 -- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b3c09f240 msgr2=0x7f2b3c1b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b3affd700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b3c09f240 0x7f2b3c1b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b3affd700 1 -- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09f650 msgr2=0x7f2b3c1d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b3affd700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09f650 0x7f2b3c1d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b3affd700 1 -- 172.21.15.89:0/1396911698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0094a0 con 0x7f2b3c09e930 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b3affd700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09e930 0x7f2b3c1b2030 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000c00 tx=0x7f2b2c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b237fe700 1 -- 172.21.15.89:0/1396911698 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0045d0 con 0x7f2b3c09e930 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b237fe700 1 -- 172.21.15.89:0/1396911698 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c041070 con 0x7f2b3c09e930 2023-09-30T22:21:56.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c0a0190 con 0x7f2b3c09e930 2023-09-30T22:21:56.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b237fe700 1 -- 172.21.15.89:0/1396911698 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0038e0 con 0x7f2b3c09e930 2023-09-30T22:21:56.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c0a3420 con 0x7f2b3c09e930 2023-09-30T22:21:56.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.509+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5008074ea0 con 0x7f50080a1510 2023-09-30T22:21:56.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.510+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3c071d20 con 0x7f2b3c09e930 2023-09-30T22:21:56.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.511+0000 7f2b237fe700 1 -- 172.21.15.89:0/1396911698 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b2c003a40 con 0x7f2b3c09e930 2023-09-30T22:21:56.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.511+0000 7f2b237fe700 1 -- 172.21.15.89:0/1396911698 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2b2c043070 con 0x7f2b3c09e930 2023-09-30T22:21:56.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.515+0000 7f4fef7fe700 1 -- 172.21.15.89:0/201801720 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ffc00f8d0 con 0x7f50080a1510 2023-09-30T22:21:56.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.519+0000 7f2b237fe700 1 -- 172.21.15.89:0/1396911698 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b2c00b3e0 con 0x7f2b3c09e930 2023-09-30T22:21:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.588+0000 7f3a29afc700 1 Processor -- start 2023-09-30T22:21:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.588+0000 7f3a29afc700 1 -- start start 2023-09-30T22:21:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.588+0000 7f3a29afc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a240a0a30 0x7f3a240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.588+0000 7f3a29afc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.588+0000 7f3a29afc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a240a2a70 con 0x7f3a2409d630 2023-09-30T22:21:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.588+0000 7f3a29afc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240a01d0 con 0x7f3a240a0a30 2023-09-30T22:21:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.588+0000 7f3a29afc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2409e2b0 con 0x7f3a24138d80 2023-09-30T22:21:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a237fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a240a0a30 0x7f3a240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a237fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a240a0a30 0x7f3a240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60682/0 (socket says 172.21.15.89:60682) 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a237fe700 1 -- 172.21.15.89:0/1719550872 learned_addr learned my addr 172.21.15.89:0/1719550872 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a23fff700 1 --2- 172.21.15.89:0/1719550872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a23fff700 1 -- 172.21.15.89:0/1719550872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a240a0a30 msgr2=0x7f3a240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a23fff700 1 --2- 172.21.15.89:0/1719550872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a240a0a30 0x7f3a240a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a23fff700 1 -- 172.21.15.89:0/1719550872 >> v1:172.21.15.89:6789/0 conn(0x7f3a2409d630 legacy=0x7f3a240a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a23fff700 1 -- 172.21.15.89:0/1719550872 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2409dc50 con 0x7f3a24138d80 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a23fff700 1 --2- 172.21.15.89:0/1719550872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2413b160 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009dc0 tx=0x7f3a1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9fc52974135e033 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.589+0000 7f3a227fc700 1 -- 172.21.15.89:0/1719550872 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00e070 con 0x7f3a24138d80 2023-09-30T22:21:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a227fc700 1 -- 172.21.15.89:0/1719550872 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c00bb20 con 0x7f3a24138d80 2023-09-30T22:21:56.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a227fc700 1 -- 172.21.15.89:0/1719550872 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c012430 con 0x7f3a24138d80 2023-09-30T22:21:56.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a29afc700 1 -- 172.21.15.89:0/1719550872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 msgr2=0x7f3a2413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a29afc700 1 --2- 172.21.15.89:0/1719550872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2413b160 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009dc0 tx=0x7f3a1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a29afc700 1 -- 172.21.15.89:0/1719550872 shutdown_connections 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a29afc700 1 --2- 172.21.15.89:0/1719550872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a240a0a30 0x7f3a240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a29afc700 1 --2- 172.21.15.89:0/1719550872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2413b160 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a29afc700 1 -- 172.21.15.89:0/1719550872 >> 172.21.15.89:0/1719550872 conn(0x7f3a240a3050 msgr2=0x7f3a240a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.590+0000 7f3a29afc700 1 -- 172.21.15.89:0/1719550872 shutdown_connections 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.591+0000 7f3a29afc700 1 -- 172.21.15.89:0/1719550872 wait complete. 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.591+0000 7f3a29afc700 1 Processor -- start 2023-09-30T22:21:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.592+0000 7f3a29afc700 1 -- start start 2023-09-30T22:21:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.592+0000 7f3a29afc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a2409d630 0x7f3a2411c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.592+0000 7f3a29afc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a240a0a30 0x7f3a2411cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.592+0000 7f3a29afc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2411d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.592+0000 7f3a29afc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240a4d80 con 0x7f3a240a0a30 2023-09-30T22:21:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.592+0000 7f3a29afc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24071870 con 0x7f3a2409d630 2023-09-30T22:21:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.592+0000 7f3a29afc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2411efe0 con 0x7f3a24138d80 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.592+0000 7f3a22ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a240a0a30 0x7f3a2411cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a22ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a240a0a30 0x7f3a2411cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42652/0 (socket says 172.21.15.89:42652) 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a22ffd700 1 -- 172.21.15.89:0/337369033 learned_addr learned my addr 172.21.15.89:0/337369033 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a23fff700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2411d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a237fe700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a2409d630 0x7f3a2411c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a22ffd700 1 -- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 msgr2=0x7f3a2411d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a22ffd700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2411d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a22ffd700 1 -- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a2409d630 msgr2=0x7f3a2411c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a22ffd700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a2409d630 0x7f3a2411c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a22ffd700 1 -- 172.21.15.89:0/337369033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c009a70 con 0x7f3a240a0a30 2023-09-30T22:21:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a22ffd700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a240a0a30 0x7f3a2411cd00 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c4b0 tx=0x7f3a1800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a0bfff700 1 -- 172.21.15.89:0/337369033 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180052d0 con 0x7f3a240a0a30 2023-09-30T22:21:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a0bfff700 1 -- 172.21.15.89:0/337369033 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18005430 con 0x7f3a240a0a30 2023-09-30T22:21:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2411dba0 con 0x7f3a240a0a30 2023-09-30T22:21:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a0bfff700 1 -- 172.21.15.89:0/337369033 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18003880 con 0x7f3a240a0a30 2023-09-30T22:21:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.593+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a24070dd0 con 0x7f3a240a0a30 2023-09-30T22:21:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.595+0000 7f3a0bfff700 1 -- 172.21.15.89:0/337369033 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3a180039e0 con 0x7f3a240a0a30 2023-09-30T22:21:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.595+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a240747f0 con 0x7f3a240a0a30 2023-09-30T22:21:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.596+0000 7f3a0bfff700 1 -- 172.21.15.89:0/337369033 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3a1803eb60 con 0x7f3a240a0a30 2023-09-30T22:21:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.600+0000 7f3a0bfff700 1 -- 172.21.15.89:0/337369033 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a180198b0 con 0x7f3a240a0a30 2023-09-30T22:21:56.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.710+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f50080a28e0 con 0x7f50080a1510 2023-09-30T22:21:56.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.711+0000 7f4fef7fe700 1 -- 172.21.15.89:0/201801720 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ffc00f2e0 con 0x7f50080a1510 2023-09-30T22:21:56.712 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:56.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50080a1510 msgr2=0x7f50081c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 --2- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50080a1510 0x7f50081c8c30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00b530 tx=0x7f4ffc00b8f0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 shutdown_connections 2023-09-30T22:21:56.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 --2- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5008139250 0x7f50081216f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 --2- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50080a0a30 0x7f50081c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 --2- 172.21.15.89:0/201801720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50080a1510 0x7f50081c8c30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 >> 172.21.15.89:0/201801720 conn(0x7f50080a6550 msgr2=0x7f5008137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 shutdown_connections 2023-09-30T22:21:56.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.713+0000 7f500d658700 1 -- 172.21.15.89:0/201801720 wait complete. 2023-09-30T22:21:56.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.716+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2b3c074ea0 con 0x7f2b3c09e930 2023-09-30T22:21:56.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.716+0000 7f2b237fe700 1 -- 172.21.15.89:0/1396911698 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b2c056020 con 0x7f2b3c09e930 2023-09-30T22:21:56.717 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.719+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09e930 msgr2=0x7f2b3c1b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.719+0000 7f2b416d0700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09e930 0x7f2b3c1b2030 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000c00 tx=0x7f2b2c004750 comp rx=0 tx=0).stop 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.720+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 shutdown_connections 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.720+0000 7f2b416d0700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b3c09f650 0x7f2b3c1d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.720+0000 7f2b416d0700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b3c09e930 0x7f2b3c1b2030 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.720+0000 7f2b416d0700 1 --2- 172.21.15.89:0/1396911698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b3c09f240 0x7f2b3c1b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.720+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 >> 172.21.15.89:0/1396911698 conn(0x7f2b3c0a4210 msgr2=0x7f2b3c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.722+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 shutdown_connections 2023-09-30T22:21:56.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.722+0000 7f2b416d0700 1 -- 172.21.15.89:0/1396911698 wait complete. 2023-09-30T22:21:56.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.723+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f907c074ea0 con 0x7f907c130430 2023-09-30T22:21:56.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.723+0000 7f90797fa700 1 -- 172.21.15.89:0/1510693478 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9074012b50 con 0x7f907c130430 2023-09-30T22:21:56.725 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:56.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.727+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c130430 msgr2=0x7f907c11b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.727+0000 7f908301b700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c130430 0x7f907c11b0d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9074007c80 tx=0x7f9074007f90 comp rx=0 tx=0).stop 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.728+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 shutdown_connections 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.728+0000 7f908301b700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f907c130430 0x7f907c11b0d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.728+0000 7f908301b700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f907c12cf60 0x7f907c11a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.728+0000 7f908301b700 1 --2- 172.21.15.89:0/1510693478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907c12f8b0 0x7f907c11ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.728+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 >> 172.21.15.89:0/1510693478 conn(0x7f907c0a2aa0 msgr2=0x7f907c0a1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.728+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 shutdown_connections 2023-09-30T22:21:56.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.728+0000 7f908301b700 1 -- 172.21.15.89:0/1510693478 wait complete. 2023-09-30T22:21:56.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:56.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.794+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3a2411ec80 con 0x7f3a240a0a30 2023-09-30T22:21:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.794+0000 7f3a0bfff700 1 -- 172.21.15.89:0/337369033 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a2411ec80 con 0x7f3a240a0a30 2023-09-30T22:21:56.795 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:56.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a240a0a30 msgr2=0x7f3a2411cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a240a0a30 0x7f3a2411cd00 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c4b0 tx=0x7f3a1800c7c0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 shutdown_connections 2023-09-30T22:21:56.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a24138d80 0x7f3a2411d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a2409d630 0x7f3a2411c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 --2- 172.21.15.89:0/337369033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a240a0a30 0x7f3a2411cd00 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:56.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 >> 172.21.15.89:0/337369033 conn(0x7f3a240a3050 msgr2=0x7f3a24099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 shutdown_connections 2023-09-30T22:21:56.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:56.796+0000 7f3a29afc700 1 -- 172.21.15.89:0/337369033 wait complete. 2023-09-30T22:21:56.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:57.730 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:57.737 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:57.742 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:57.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:57.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.869+0000 7faeac2da700 1 Processor -- start 2023-09-30T22:21:57.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.869+0000 7faeac2da700 1 -- start start 2023-09-30T22:21:57.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeac2da700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faea412cf60 0x7faea406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeac2da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea4130430 0x7faea4137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeac2da700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea412cd90 con 0x7faea412f8b0 2023-09-30T22:21:57.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeac2da700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea412d330 con 0x7faea412cf60 2023-09-30T22:21:57.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeac2da700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faea4071780 con 0x7faea4130430 2023-09-30T22:21:57.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeaa076700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faea412cf60 0x7faea406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeaa076700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faea412cf60 0x7faea406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42668/0 (socket says 172.21.15.89:42668) 2023-09-30T22:21:57.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeaa076700 1 -- 172.21.15.89:0/1187581 learned_addr learned my addr 172.21.15.89:0/1187581 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:57.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeaa877700 1 --2- 172.21.15.89:0/1187581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea4130430 0x7faea4137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeaa877700 1 -- 172.21.15.89:0/1187581 >> v2:172.21.15.89:3300/0 conn(0x7faea412cf60 msgr2=0x7faea406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeaa877700 1 --2- 172.21.15.89:0/1187581 >> v2:172.21.15.89:3300/0 conn(0x7faea412cf60 0x7faea406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeaa877700 1 -- 172.21.15.89:0/1187581 >> v1:172.21.15.89:6789/0 conn(0x7faea412f8b0 legacy=0x7faea412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faeaa877700 1 -- 172.21.15.89:0/1187581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4071a40 con 0x7faea4130430 2023-09-30T22:21:57.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.870+0000 7faea9074700 1 -- 172.21.15.89:0/1187581 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033370788 0 0) 0x7faea412cd90 con 0x7faea412f8b0 2023-09-30T22:21:57.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.871+0000 7faeaa877700 1 --2- 172.21.15.89:0/1187581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea4130430 0x7faea4137c30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009d90 tx=0x7fae9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=352698f99d709e70 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.871+0000 7faea9074700 1 -- 172.21.15.89:0/1187581 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00e070 con 0x7faea4130430 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.871+0000 7faea9074700 1 -- 172.21.15.89:0/1187581 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c00bb20 con 0x7faea4130430 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.871+0000 7faea9074700 1 -- 172.21.15.89:0/1187581 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c012430 con 0x7faea4130430 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.872+0000 7faeac2da700 1 -- 172.21.15.89:0/1187581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea4130430 msgr2=0x7faea4137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.872+0000 7faeac2da700 1 --2- 172.21.15.89:0/1187581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea4130430 0x7faea4137c30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009d90 tx=0x7fae9c00b330 comp rx=0 tx=0).stop 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.872+0000 7faeac2da700 1 -- 172.21.15.89:0/1187581 shutdown_connections 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.872+0000 7faeac2da700 1 --2- 172.21.15.89:0/1187581 >> v2:172.21.15.89:3300/0 conn(0x7faea412cf60 0x7faea406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.872+0000 7faeac2da700 1 --2- 172.21.15.89:0/1187581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea4130430 0x7faea4137c30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.872+0000 7faeac2da700 1 -- 172.21.15.89:0/1187581 >> 172.21.15.89:0/1187581 conn(0x7faea40a2aa0 msgr2=0x7faea40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.872+0000 7faeac2da700 1 -- 172.21.15.89:0/1187581 shutdown_connections 2023-09-30T22:21:57.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.872+0000 7faeac2da700 1 -- 172.21.15.89:0/1187581 wait complete. 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeac2da700 1 Processor -- start 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeac2da700 1 -- start start 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeac2da700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea412cf60 0x7faea41b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeac2da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea412f8b0 0x7faea41b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeac2da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea4130430 0x7faea41c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeac2da700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea41b6190 con 0x7faea4130430 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeac2da700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea41b9070 con 0x7faea412cf60 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeac2da700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faea41b9330 con 0x7faea412f8b0 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faea9875700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea412f8b0 0x7faea41b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faea9875700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea412f8b0 0x7faea41b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60764/0 (socket says 172.21.15.89:60764) 2023-09-30T22:21:57.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faea9875700 1 -- 172.21.15.89:0/4274578594 learned_addr learned my addr 172.21.15.89:0/4274578594 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:57.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.873+0000 7faeaa877700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea4130430 0x7faea41c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faeaa076700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea412cf60 0x7faea41b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faea9875700 1 -- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea412cf60 msgr2=0x7faea41b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faea9875700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea412cf60 0x7faea41b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faea9875700 1 -- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea4130430 msgr2=0x7faea41c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faea9875700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea4130430 0x7faea41c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faea9875700 1 -- 172.21.15.89:0/4274578594 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c009a70 con 0x7faea412f8b0 2023-09-30T22:21:57.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faea9875700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea412f8b0 0x7faea41b6c20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fae9400c520 tx=0x7fae9400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7fae9affd700 1 -- 172.21.15.89:0/4274578594 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940054e0 con 0x7faea412f8b0 2023-09-30T22:21:57.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7fae9affd700 1 -- 172.21.15.89:0/4274578594 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae94003680 con 0x7faea412f8b0 2023-09-30T22:21:57.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea41b8370 con 0x7faea412f8b0 2023-09-30T22:21:57.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7fae9affd700 1 -- 172.21.15.89:0/4274578594 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940039f0 con 0x7faea412f8b0 2023-09-30T22:21:57.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.874+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea41b9610 con 0x7faea412f8b0 2023-09-30T22:21:57.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.875+0000 7fae9affd700 1 -- 172.21.15.89:0/4274578594 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae94003b50 con 0x7faea412f8b0 2023-09-30T22:21:57.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.875+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea4129920 con 0x7faea412f8b0 2023-09-30T22:21:57.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.876+0000 7fae9affd700 1 -- 172.21.15.89:0/4274578594 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fae9403ea10 con 0x7faea412f8b0 2023-09-30T22:21:57.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.884+0000 7fae9affd700 1 -- 172.21.15.89:0/4274578594 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae9401b3b0 con 0x7faea412f8b0 2023-09-30T22:21:57.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.910+0000 7fef5f950700 1 Processor -- start 2023-09-30T22:21:57.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c6597700 1 Processor -- start 2023-09-30T22:21:57.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c6597700 1 -- start start 2023-09-30T22:21:57.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c6597700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0136830 0x7f46c0138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c6597700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c0139150 0x7f46c013b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5f950700 1 -- start start 2023-09-30T22:21:57.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5f950700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 0x7fef58138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5f950700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5f950700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5809ea10 con 0x7fef580a09c0 2023-09-30T22:21:57.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5f950700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5809e210 con 0x7fef580a14a0 2023-09-30T22:21:57.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5f950700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef580a26f0 con 0x7fef5809d5c0 2023-09-30T22:21:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c6597700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c006f720 con 0x7f46c01338c0 2023-09-30T22:21:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c6597700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0070c80 con 0x7f46c0136830 2023-09-30T22:21:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c6597700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46c006fce0 con 0x7f46c0139150 2023-09-30T22:21:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c4b34700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c0139150 0x7f46c013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c4b34700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c0139150 0x7f46c013b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60778/0 (socket says 172.21.15.89:60778) 2023-09-30T22:21:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46c4b34700 1 -- 172.21.15.89:0/3183119048 learned_addr learned my addr 172.21.15.89:0/3183119048 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7f46bf7fe700 1 --2- 172.21.15.89:0/3183119048 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0136830 0x7f46c0138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5deed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5deed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5813b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60710/0 (socket says 172.21.15.89:60710) 2023-09-30T22:21:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5deed700 1 -- 172.21.15.89:0/1970727188 learned_addr learned my addr 172.21.15.89:0/1970727188 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.911+0000 7fef5ceeb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 0x7fef58138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7f46c4b34700 1 -- 172.21.15.89:0/3183119048 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0136830 msgr2=0x7f46c0138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7f46c4b34700 1 --2- 172.21.15.89:0/3183119048 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0136830 0x7f46c0138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7f46c4b34700 1 -- 172.21.15.89:0/3183119048 >> v1:172.21.15.89:6789/0 conn(0x7f46c01338c0 legacy=0x7f46c0133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7f46c4b34700 1 -- 172.21.15.89:0/3183119048 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c01336a0 con 0x7f46c0139150 2023-09-30T22:21:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7fef5deed700 1 -- 172.21.15.89:0/1970727188 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 msgr2=0x7fef58138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7fef5deed700 1 --2- 172.21.15.89:0/1970727188 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 0x7fef58138d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7fef5deed700 1 -- 172.21.15.89:0/1970727188 >> v1:172.21.15.89:6789/0 conn(0x7fef580a09c0 legacy=0x7fef580a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7fef5deed700 1 -- 172.21.15.89:0/1970727188 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5809db70 con 0x7fef580a14a0 2023-09-30T22:21:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7fef5deed700 1 --2- 172.21.15.89:0/1970727188 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5813b2c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fef50009e80 tx=0x7fef5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3dfecdc14f1d14a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7f46c4b34700 1 --2- 172.21.15.89:0/3183119048 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c0139150 0x7f46c013b530 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009e80 tx=0x7f46b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed448999283cd6bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7fef4ffff700 1 -- 172.21.15.89:0/1970727188 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5000e070 con 0x7fef580a14a0 2023-09-30T22:21:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7fef4ffff700 1 -- 172.21.15.89:0/1970727188 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef50003a20 con 0x7fef580a14a0 2023-09-30T22:21:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7fef4ffff700 1 -- 172.21.15.89:0/1970727188 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50003db0 con 0x7fef580a14a0 2023-09-30T22:21:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.912+0000 7f46beffd700 1 -- 172.21.15.89:0/3183119048 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800e070 con 0x7f46c0139150 2023-09-30T22:21:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7f46beffd700 1 -- 172.21.15.89:0/3183119048 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8003a20 con 0x7f46c0139150 2023-09-30T22:21:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7f46beffd700 1 -- 172.21.15.89:0/3183119048 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8003dd0 con 0x7f46c0139150 2023-09-30T22:21:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7fef5f950700 1 -- 172.21.15.89:0/1970727188 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 msgr2=0x7fef5813b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7fef5f950700 1 --2- 172.21.15.89:0/1970727188 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5813b2c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fef50009e80 tx=0x7fef5000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7fef5f950700 1 -- 172.21.15.89:0/1970727188 shutdown_connections 2023-09-30T22:21:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7fef5f950700 1 --2- 172.21.15.89:0/1970727188 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5813b2c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7fef5f950700 1 --2- 172.21.15.89:0/1970727188 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 0x7fef58138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7fef5f950700 1 -- 172.21.15.89:0/1970727188 >> 172.21.15.89:0/1970727188 conn(0x7fef580a2fe0 msgr2=0x7fef580a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7fef5f950700 1 -- 172.21.15.89:0/1970727188 shutdown_connections 2023-09-30T22:21:57.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7fef5f950700 1 -- 172.21.15.89:0/1970727188 wait complete. 2023-09-30T22:21:57.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7f46c6597700 1 -- 172.21.15.89:0/3183119048 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c0139150 msgr2=0x7f46c013b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.913+0000 7f46c6597700 1 --2- 172.21.15.89:0/3183119048 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c0139150 0x7f46c013b530 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009e80 tx=0x7f46b800b330 comp rx=0 tx=0).stop 2023-09-30T22:21:57.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.914+0000 7f46c6597700 1 -- 172.21.15.89:0/3183119048 shutdown_connections 2023-09-30T22:21:57.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.914+0000 7f46c6597700 1 --2- 172.21.15.89:0/3183119048 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c0139150 0x7f46c013b530 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.914+0000 7f46c6597700 1 --2- 172.21.15.89:0/3183119048 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0136830 0x7f46c0138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.914+0000 7f46c6597700 1 -- 172.21.15.89:0/3183119048 >> 172.21.15.89:0/3183119048 conn(0x7f46c00a2aa0 msgr2=0x7f46c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.914+0000 7f46c6597700 1 -- 172.21.15.89:0/3183119048 shutdown_connections 2023-09-30T22:21:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.914+0000 7f46c6597700 1 -- 172.21.15.89:0/3183119048 wait complete. 2023-09-30T22:21:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.914+0000 7fef5f950700 1 Processor -- start 2023-09-30T22:21:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.914+0000 7fef5f950700 1 -- start start 2023-09-30T22:21:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5f950700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 0x7fef5811c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5f950700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef580a09c0 0x7fef5811ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5f950700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5811d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5f950700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5809cec0 con 0x7fef580a09c0 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5f950700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5811e6f0 con 0x7fef580a14a0 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5f950700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef58120020 con 0x7fef5809d5c0 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5811d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5811d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60724/0 (socket says 172.21.15.89:60724) 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 -- 172.21.15.89:0/3297661457 learned_addr learned my addr 172.21.15.89:0/3297661457 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 -- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 msgr2=0x7fef5811c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 --2- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 0x7fef5811c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 -- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef580a09c0 msgr2=0x7fef5811ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 --2- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef580a09c0 0x7fef5811ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 -- 172.21.15.89:0/3297661457 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50009b30 con 0x7fef580a14a0 2023-09-30T22:21:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7f46c6597700 1 Processor -- start 2023-09-30T22:21:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5deed700 1 --2- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5811d230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fef5000b370 tx=0x7fef50004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef4dffb700 1 -- 172.21.15.89:0/3297661457 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50004440 con 0x7fef580a14a0 2023-09-30T22:21:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef4dffb700 1 -- 172.21.15.89:0/3297661457 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef500045a0 con 0x7fef580a14a0 2023-09-30T22:21:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.915+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5809d2b0 con 0x7fef580a14a0 2023-09-30T22:21:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.916+0000 7fef4dffb700 1 -- 172.21.15.89:0/3297661457 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50004910 con 0x7fef580a14a0 2023-09-30T22:21:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.916+0000 7f46c6597700 1 -- start start 2023-09-30T22:21:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.916+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5811e900 con 0x7fef580a14a0 2023-09-30T22:21:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.916+0000 7f46c6597700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c01338c0 0x7f46c011a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.916+0000 7f46c6597700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46c0136830 0x7f46c011aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.916+0000 7f46bf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46c0136830 0x7f46c011aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.916+0000 7f46bf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46c0136830 0x7f46c011aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42682/0 (socket says 172.21.15.89:42682) 2023-09-30T22:21:57.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7fef4dffb700 1 -- 172.21.15.89:0/3297661457 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef50019070 con 0x7fef580a14a0 2023-09-30T22:21:57.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7fef4dffb700 1 -- 172.21.15.89:0/3297661457 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef50040830 con 0x7fef580a14a0 2023-09-30T22:21:57.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7f46c6597700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0139150 0x7f46c01c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7f46c6597700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c011d2b0 con 0x7f46c0136830 2023-09-30T22:21:57.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7f46c6597700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c011d510 con 0x7f46c0139150 2023-09-30T22:21:57.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7f46c6597700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46c006f960 con 0x7f46c01338c0 2023-09-30T22:21:57.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7f46bffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c01338c0 0x7f46c011a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7f46bf7fe700 1 -- 172.21.15.89:0/1631018743 learned_addr learned my addr 172.21.15.89:0/1631018743 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:57.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7f46c4b34700 1 --2- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0139150 0x7f46c01c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.917+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef380052e0 con 0x7fef580a14a0 2023-09-30T22:21:57.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bf7fe700 1 -- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c01338c0 msgr2=0x7f46c011a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bf7fe700 1 --2- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c01338c0 0x7f46c011a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bf7fe700 1 -- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0139150 msgr2=0x7f46c01c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bf7fe700 1 --2- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0139150 0x7f46c01c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bf7fe700 1 -- 172.21.15.89:0/1631018743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8009b30 con 0x7f46c0136830 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bf7fe700 1 --2- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46c0136830 0x7f46c011aef0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f46b400e980 tx=0x7f46b400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bcff9700 1 -- 172.21.15.89:0/1631018743 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4009710 con 0x7f46c0136830 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bcff9700 1 -- 172.21.15.89:0/1631018743 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b400eed0 con 0x7f46c0136830 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c011bc40 con 0x7f46c0136830 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46bcff9700 1 -- 172.21.15.89:0/1631018743 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b400ca90 con 0x7f46c0136830 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.918+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c011a520 con 0x7f46c0136830 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.919+0000 7f46bcff9700 1 -- 172.21.15.89:0/1631018743 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46b400cbf0 con 0x7f46c0136830 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.919+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a00052e0 con 0x7f46c0136830 2023-09-30T22:21:57.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.919+0000 7f46bcff9700 1 -- 172.21.15.89:0/1631018743 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f46b403f6e0 con 0x7f46c0136830 2023-09-30T22:21:57.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.922+0000 7fef4dffb700 1 -- 172.21.15.89:0/3297661457 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef50024e10 con 0x7fef580a14a0 2023-09-30T22:21:57.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.926+0000 7f46bcff9700 1 -- 172.21.15.89:0/1631018743 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46b40232f0 con 0x7f46c0136830 2023-09-30T22:21:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.971+0000 7f9fd1a42700 1 Processor -- start 2023-09-30T22:21:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.971+0000 7f9fd1a42700 1 -- start start 2023-09-30T22:21:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fd1a42700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fd1a42700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fd1a42700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fcc0a2a70 con 0x7f9fcc09d630 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fd1a42700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc0a01d0 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fd1a42700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc09e2b0 con 0x7f9fcc138d80 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fcaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fcaffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60730/0 (socket says 172.21.15.89:60730) 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fcaffd700 1 -- 172.21.15.89:0/3640009255 learned_addr learned my addr 172.21.15.89:0/3640009255 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fcb7fe700 1 --2- 172.21.15.89:0/3640009255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fcaffd700 1 -- 172.21.15.89:0/3640009255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc138d80 msgr2=0x7f9fcc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.972+0000 7f9fcaffd700 1 --2- 172.21.15.89:0/3640009255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.973+0000 7f9fcaffd700 1 -- 172.21.15.89:0/3640009255 >> v1:172.21.15.89:6789/0 conn(0x7f9fcc09d630 legacy=0x7f9fcc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.973+0000 7f9fcaffd700 1 -- 172.21.15.89:0/3640009255 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc09dc50 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.973+0000 7f9fcaffd700 1 --2- 172.21.15.89:0/3640009255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009920 tx=0x7f9fbc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42031a93ab807372 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.973+0000 7f9fc9ffb700 1 -- 172.21.15.89:0/3640009255 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00ba60 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.973+0000 7f9fc9ffb700 1 -- 172.21.15.89:0/3640009255 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc00bbc0 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.973+0000 7f9fc9ffb700 1 -- 172.21.15.89:0/3640009255 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc009e70 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.974+0000 7f9fd1a42700 1 -- 172.21.15.89:0/3640009255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc0a0a30 msgr2=0x7f9fcc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.974+0000 7f9fd1a42700 1 --2- 172.21.15.89:0/3640009255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009920 tx=0x7f9fbc009c30 comp rx=0 tx=0).stop 2023-09-30T22:21:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.974+0000 7f9fd1a42700 1 -- 172.21.15.89:0/3640009255 shutdown_connections 2023-09-30T22:21:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.974+0000 7f9fd1a42700 1 --2- 172.21.15.89:0/3640009255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.974+0000 7f9fd1a42700 1 --2- 172.21.15.89:0/3640009255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.974+0000 7f9fd1a42700 1 -- 172.21.15.89:0/3640009255 >> 172.21.15.89:0/3640009255 conn(0x7f9fcc0a3050 msgr2=0x7f9fcc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.974+0000 7f9fd1a42700 1 -- 172.21.15.89:0/3640009255 shutdown_connections 2023-09-30T22:21:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.974+0000 7f9fd1a42700 1 -- 172.21.15.89:0/3640009255 wait complete. 2023-09-30T22:21:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.975+0000 7f9fd1a42700 1 Processor -- start 2023-09-30T22:21:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.975+0000 7f9fd1a42700 1 -- start start 2023-09-30T22:21:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fd1a42700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc09d630 0x7f9fcc12d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fd1a42700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc12dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fd1a42700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc138d80 0x7f9fcc12e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fd1a42700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc12ee90 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fd1a42700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc130730 con 0x7f9fcc138d80 2023-09-30T22:21:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fd1a42700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc1309f0 con 0x7f9fcc09d630 2023-09-30T22:21:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fca7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc12dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fcb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc138d80 0x7f9fcc12e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fca7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc12dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42696/0 (socket says 172.21.15.89:42696) 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fcb7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc138d80 0x7f9fcc12e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60734/0 (socket says 172.21.15.89:60734) 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fca7fc700 1 -- 172.21.15.89:0/1348492191 learned_addr learned my addr 172.21.15.89:0/1348492191 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fcaffd700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc09d630 0x7f9fcc12d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fca7fc700 1 -- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc09d630 msgr2=0x7f9fcc12d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fca7fc700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc09d630 0x7f9fcc12d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fca7fc700 1 -- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc138d80 msgr2=0x7f9fcc12e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fca7fc700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc138d80 0x7f9fcc12e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fca7fc700 1 -- 172.21.15.89:0/1348492191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc009550 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.976+0000 7f9fcaffd700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc09d630 0x7f9fcc12d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.977+0000 7f9fcb7fe700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc138d80 0x7f9fcc12e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.977+0000 7f9fca7fc700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc12dd10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000c4b0 tx=0x7f9fc000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.977+0000 7f9fb3fff700 1 -- 172.21.15.89:0/1348492191 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00052d0 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.977+0000 7f9fb3fff700 1 -- 172.21.15.89:0/1348492191 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc0005430 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.977+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fcc131500 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.977+0000 7f9fb3fff700 1 -- 172.21.15.89:0/1348492191 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00038a0 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.977+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fcc130cd0 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.978+0000 7f9fb3fff700 1 -- 172.21.15.89:0/1348492191 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9fc0003a00 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.978+0000 7f9fb3fff700 1 -- 172.21.15.89:0/1348492191 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9fc0011ab0 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.978+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fcc0747f0 con 0x7f9fcc0a0a30 2023-09-30T22:21:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:57.987+0000 7f9fb3fff700 1 -- 172.21.15.89:0/1348492191 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9fc0044060 con 0x7f9fcc0a0a30 2023-09-30T22:21:58.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.084+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faea4074ea0 con 0x7faea412f8b0 2023-09-30T22:21:58.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.084+0000 7fae9affd700 1 -- 172.21.15.89:0/4274578594 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae94017070 con 0x7faea412f8b0 2023-09-30T22:21:58.085 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:58.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.086+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea412f8b0 msgr2=0x7faea41b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.086+0000 7faeac2da700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea412f8b0 0x7faea41b6c20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fae9400c520 tx=0x7fae9400c830 comp rx=0 tx=0).stop 2023-09-30T22:21:58.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.086+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 shutdown_connections 2023-09-30T22:21:58.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.086+0000 7faeac2da700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faea4130430 0x7faea41c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.086+0000 7faeac2da700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faea412cf60 0x7faea41b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.086+0000 7faeac2da700 1 --2- 172.21.15.89:0/4274578594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faea412f8b0 0x7faea41b6c20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.087+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 >> 172.21.15.89:0/4274578594 conn(0x7faea40a2aa0 msgr2=0x7faea406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:58.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.087+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 shutdown_connections 2023-09-30T22:21:58.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.087+0000 7faeac2da700 1 -- 172.21.15.89:0/4274578594 wait complete. 2023-09-30T22:21:58.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:58.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.122+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fef38005590 con 0x7fef580a14a0 2023-09-30T22:21:58.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.122+0000 7fef4dffb700 1 -- 172.21.15.89:0/3297661457 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef50017020 con 0x7fef580a14a0 2023-09-30T22:21:58.123 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 msgr2=0x7fef5811d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 --2- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5811d230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fef5000b370 tx=0x7fef50004ad0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 shutdown_connections 2023-09-30T22:21:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 --2- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef580a14a0 0x7fef5811d230 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 --2- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef5809d5c0 0x7fef5811c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 --2- 172.21.15.89:0/3297661457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef580a09c0 0x7fef5811ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 >> 172.21.15.89:0/3297661457 conn(0x7fef580a2fe0 msgr2=0x7fef58134b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:58.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 shutdown_connections 2023-09-30T22:21:58.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.126+0000 7fef5f950700 1 -- 172.21.15.89:0/3297661457 wait complete. 2023-09-30T22:21:58.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:58.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.155+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f46a0005590 con 0x7f46c0136830 2023-09-30T22:21:58.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.155+0000 7f46bcff9700 1 -- 172.21.15.89:0/1631018743 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46b4011350 con 0x7f46c0136830 2023-09-30T22:21:58.157 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:58.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46c0136830 msgr2=0x7f46c011aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 --2- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46c0136830 0x7f46c011aef0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f46b400e980 tx=0x7f46b400ec90 comp rx=0 tx=0).stop 2023-09-30T22:21:58.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 shutdown_connections 2023-09-30T22:21:58.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 --2- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46c0139150 0x7f46c01c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 --2- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46c01338c0 0x7f46c011a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 --2- 172.21.15.89:0/1631018743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46c0136830 0x7f46c011aef0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 >> 172.21.15.89:0/1631018743 conn(0x7f46c00a2aa0 msgr2=0x7f46c009f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 shutdown_connections 2023-09-30T22:21:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.157+0000 7f46c6597700 1 -- 172.21.15.89:0/1631018743 wait complete. 2023-09-30T22:21:58.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:58.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.203+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9fcc072980 con 0x7f9fcc0a0a30 2023-09-30T22:21:58.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.203+0000 7f9fb3fff700 1 -- 172.21.15.89:0/1348492191 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9fc0017030 con 0x7f9fcc0a0a30 2023-09-30T22:21:58.204 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.205+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fcc0a0a30 msgr2=0x7f9fcc12dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.205+0000 7f9fd1a42700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc12dd10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000c4b0 tx=0x7f9fc000c7c0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.205+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 shutdown_connections 2023-09-30T22:21:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.205+0000 7f9fd1a42700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fcc138d80 0x7f9fcc12e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.205+0000 7f9fd1a42700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fcc09d630 0x7f9fcc12d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.205+0000 7f9fd1a42700 1 --2- 172.21.15.89:0/1348492191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc12dd10 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.205+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 >> 172.21.15.89:0/1348492191 conn(0x7f9fcc0a3050 msgr2=0x7f9fcc13a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.205+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 shutdown_connections 2023-09-30T22:21:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:58.206+0000 7f9fd1a42700 1 -- 172.21.15.89:0/1348492191 wait complete. 2023-09-30T22:21:58.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:21:59.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:59.144 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:59.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:59.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:59.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.240+0000 7f0eb04d4700 1 Processor -- start 2023-09-30T22:21:59.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.241+0000 7f0eb04d4700 1 -- start start 2023-09-30T22:21:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.241+0000 7f0eb04d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea8136710 0x7f0ea8138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.241+0000 7f0eb04d4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ea8139030 0x7f0ea813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.241+0000 7f0eb04d4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea806ef30 con 0x7f0ea81338a0 2023-09-30T22:21:59.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.241+0000 7f0eb04d4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea806fef0 con 0x7f0ea8139030 2023-09-30T22:21:59.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.241+0000 7f0eb04d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81334b0 con 0x7f0ea8136710 2023-09-30T22:21:59.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.241+0000 7f0eae270700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0ea81338a0 0x7f0ea8133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42914/0 (socket says 172.21.15.89:42914) 2023-09-30T22:21:59.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0eae270700 1 -- 172.21.15.89:0/962847105 learned_addr learned my addr 172.21.15.89:0/962847105 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:59.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0eaea71700 1 --2- 172.21.15.89:0/962847105 >> v2:172.21.15.89:3300/0 conn(0x7f0ea8139030 0x7f0ea813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0eada6f700 1 --2- 172.21.15.89:0/962847105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea8136710 0x7f0ea8138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0eaea71700 1 -- 172.21.15.89:0/962847105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea8136710 msgr2=0x7f0ea8138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0eaea71700 1 --2- 172.21.15.89:0/962847105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea8136710 0x7f0ea8138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0eaea71700 1 -- 172.21.15.89:0/962847105 >> v1:172.21.15.89:6789/0 conn(0x7f0ea81338a0 legacy=0x7f0ea8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0eaea71700 1 -- 172.21.15.89:0/962847105 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea806fc30 con 0x7f0ea8139030 2023-09-30T22:21:59.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0eaea71700 1 --2- 172.21.15.89:0/962847105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea813b410 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009d90 tx=0x7f0ea000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3376a408963c7c0b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0ead26e700 1 -- 172.21.15.89:0/962847105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000e070 con 0x7f0ea8139030 2023-09-30T22:21:59.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0ead26e700 1 -- 172.21.15.89:0/962847105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea000bb20 con 0x7f0ea8139030 2023-09-30T22:21:59.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.242+0000 7f0ead26e700 1 -- 172.21.15.89:0/962847105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0012430 con 0x7f0ea8139030 2023-09-30T22:21:59.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.243+0000 7f0eb04d4700 1 -- 172.21.15.89:0/962847105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 msgr2=0x7f0ea813b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.243+0000 7f0eb04d4700 1 --2- 172.21.15.89:0/962847105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea813b410 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009d90 tx=0x7f0ea000b330 comp rx=0 tx=0).stop 2023-09-30T22:21:59.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.243+0000 7f0eb04d4700 1 -- 172.21.15.89:0/962847105 shutdown_connections 2023-09-30T22:21:59.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.243+0000 7f0eb04d4700 1 --2- 172.21.15.89:0/962847105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea813b410 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.243+0000 7f0eb04d4700 1 --2- 172.21.15.89:0/962847105 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea8136710 0x7f0ea8138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.243+0000 7f0eb04d4700 1 -- 172.21.15.89:0/962847105 >> 172.21.15.89:0/962847105 conn(0x7f0ea80a2aa0 msgr2=0x7f0ea80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.243+0000 7f0eb04d4700 1 -- 172.21.15.89:0/962847105 shutdown_connections 2023-09-30T22:21:59.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.244+0000 7f0eb04d4700 1 -- 172.21.15.89:0/962847105 wait complete. 2023-09-30T22:21:59.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.244+0000 7f0eb04d4700 1 Processor -- start 2023-09-30T22:21:59.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eb04d4700 1 -- start start 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eb04d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea81338a0 0x7f0ea81c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eb04d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea8136710 0x7f0ea81c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eb04d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea811adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eb04d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81be1b0 con 0x7f0ea8139030 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eb04d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea806f640 con 0x7f0ea8136710 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eb04d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea811e250 con 0x7f0ea81338a0 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eaea71700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea811adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eae270700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea81338a0 0x7f0ea81c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eaea71700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea811adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42716/0 (socket says 172.21.15.89:42716) 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.245+0000 7f0eaea71700 1 -- 172.21.15.89:0/2554745906 learned_addr learned my addr 172.21.15.89:0/2554745906 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:59.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eae270700 1 -- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea8136710 msgr2=0x7f0ea81c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eae270700 1 --2- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea8136710 0x7f0ea81c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eae270700 1 -- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 msgr2=0x7f0ea811adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eae270700 1 --2- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea811adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eae270700 1 -- 172.21.15.89:0/2554745906 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0009a70 con 0x7f0ea81338a0 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eaea71700 1 --2- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea811adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eae270700 1 --2- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea81338a0 0x7f0ea81c8490 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c500 tx=0x7f0ea400c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0e9effd700 1 -- 172.21.15.89:0/2554745906 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4005d70 con 0x7f0ea81338a0 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0e9effd700 1 -- 172.21.15.89:0/2554745906 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea40070d0 con 0x7f0ea81338a0 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea8130ba0 con 0x7f0ea81338a0 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0e9effd700 1 -- 172.21.15.89:0/2554745906 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4007440 con 0x7f0ea81338a0 2023-09-30T22:21:59.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.246+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea811e5c0 con 0x7f0ea81338a0 2023-09-30T22:21:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.247+0000 7f0e9effd700 1 -- 172.21.15.89:0/2554745906 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ea40075a0 con 0x7f0ea81338a0 2023-09-30T22:21:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.247+0000 7f0e9effd700 1 -- 172.21.15.89:0/2554745906 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0ea403ea60 con 0x7f0ea81338a0 2023-09-30T22:21:59.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.247+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea8074ea0 con 0x7f0ea81338a0 2023-09-30T22:21:59.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.256+0000 7f0e9effd700 1 -- 172.21.15.89:0/2554745906 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ea4019d20 con 0x7f0ea81338a0 2023-09-30T22:21:59.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.280+0000 7fb4db979700 1 Processor -- start 2023-09-30T22:21:59.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.280+0000 7fb4db979700 1 -- start start 2023-09-30T22:21:59.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4db979700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d4130540 0x7fb4d4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4db979700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130e50 0x7fb4d406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4db979700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d40716a0 con 0x7fb4d406fb20 2023-09-30T22:21:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4db979700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d409a390 con 0x7fb4d4130e50 2023-09-30T22:21:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4db979700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4071290 con 0x7fb4d4130540 2023-09-30T22:21:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d8f14700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130e50 0x7fb4d406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d9f16700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb4d406fb20 0x7fb4d4137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42922/0 (socket says 172.21.15.89:42922) 2023-09-30T22:21:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d9715700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d4130540 0x7fb4d4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d9f16700 1 -- 172.21.15.89:0/1593395316 learned_addr learned my addr 172.21.15.89:0/1593395316 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d8f14700 1 -- 172.21.15.89:0/1593395316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d4130540 msgr2=0x7fb4d4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d8f14700 1 --2- 172.21.15.89:0/1593395316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d4130540 0x7fb4d4130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d8f14700 1 -- 172.21.15.89:0/1593395316 >> v1:172.21.15.89:6789/0 conn(0x7fb4d406fb20 legacy=0x7fb4d4137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d8f14700 1 -- 172.21.15.89:0/1593395316 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d4130340 con 0x7fb4d4130e50 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.281+0000 7fb4d8f14700 1 --2- 172.21.15.89:0/1593395316 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130e50 0x7fb4d406f5e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40098a0 tx=0x7fb4c4009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612e73165c533249 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4cbfff700 1 -- 172.21.15.89:0/1593395316 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4011830 con 0x7fb4d4130e50 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4cbfff700 1 -- 172.21.15.89:0/1593395316 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c4011990 con 0x7fb4d4130e50 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4cbfff700 1 -- 172.21.15.89:0/1593395316 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4011d20 con 0x7fb4d4130e50 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4db979700 1 -- 172.21.15.89:0/1593395316 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130e50 msgr2=0x7fb4d406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4db979700 1 --2- 172.21.15.89:0/1593395316 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130e50 0x7fb4d406f5e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40098a0 tx=0x7fb4c4009bb0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4db979700 1 -- 172.21.15.89:0/1593395316 shutdown_connections 2023-09-30T22:21:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4db979700 1 --2- 172.21.15.89:0/1593395316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d4130540 0x7fb4d4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4db979700 1 --2- 172.21.15.89:0/1593395316 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130e50 0x7fb4d406f5e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4db979700 1 -- 172.21.15.89:0/1593395316 >> 172.21.15.89:0/1593395316 conn(0x7fb4d40a2aa0 msgr2=0x7fb4d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4db979700 1 -- 172.21.15.89:0/1593395316 shutdown_connections 2023-09-30T22:21:59.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.282+0000 7fb4db979700 1 -- 172.21.15.89:0/1593395316 wait complete. 2023-09-30T22:21:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4db979700 1 Processor -- start 2023-09-30T22:21:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4db979700 1 -- start start 2023-09-30T22:21:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4db979700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d406fb20 0x7fb4d411a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4db979700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130540 0x7fb4d411aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4db979700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4d4130e50 0x7fb4d411afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4db979700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d411de80 con 0x7fb4d4130e50 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4db979700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d411b930 con 0x7fb4d4130540 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4db979700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d411e080 con 0x7fb4d406fb20 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.284+0000 7fb4d9715700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d406fb20 0x7fb4d411a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9715700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d406fb20 0x7fb4d411a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60862/0 (socket says 172.21.15.89:60862) 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9715700 1 -- 172.21.15.89:0/2855699886 learned_addr learned my addr 172.21.15.89:0/2855699886 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9f16700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4d4130e50 0x7fb4d411afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d8f14700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130540 0x7fb4d411aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9715700 1 -- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130540 msgr2=0x7fb4d411aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9715700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130540 0x7fb4d411aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9715700 1 -- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4d4130e50 msgr2=0x7fb4d411afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9715700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4d4130e50 0x7fb4d411afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9715700 1 -- 172.21.15.89:0/2855699886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40094a0 con 0x7fb4d406fb20 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9f16700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4d4130e50 0x7fb4d411afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4d9715700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d406fb20 0x7fb4d411a560 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000ed70 tx=0x7fb4d000d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4c9ffb700 1 -- 172.21.15.89:0/2855699886 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000dd90 con 0x7fb4d406fb20 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4c9ffb700 1 -- 172.21.15.89:0/2855699886 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d0010440 con 0x7fb4d406fb20 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d411cf70 con 0x7fb4d406fb20 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.285+0000 7fb4c9ffb700 1 -- 172.21.15.89:0/2855699886 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00107b0 con 0x7fb4d406fb20 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.286+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d40718e0 con 0x7fb4d406fb20 2023-09-30T22:21:59.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.286+0000 7fb4c9ffb700 1 -- 172.21.15.89:0/2855699886 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb4d0010910 con 0x7fb4d406fb20 2023-09-30T22:21:59.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.287+0000 7fb4c9ffb700 1 -- 172.21.15.89:0/2855699886 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb4d0043240 con 0x7fb4d406fb20 2023-09-30T22:21:59.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.287+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d409cd40 con 0x7fb4d406fb20 2023-09-30T22:21:59.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.292+0000 7fb4c9ffb700 1 -- 172.21.15.89:0/2855699886 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4d000f3d0 con 0x7fb4d406fb20 2023-09-30T22:21:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.323+0000 7f5e20b9e700 1 Processor -- start 2023-09-30T22:21:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e20b9e700 1 -- start start 2023-09-30T22:21:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e20b9e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e1c09e3b0 0x7f5e1c1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e20b9e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c135be0 0x7f5e1c137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e20b9e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1c09b6e0 con 0x7f5e1c09d8d0 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e20b9e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1c09b470 con 0x7f5e1c09e3b0 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e20b9e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c09eb80 con 0x7f5e1c135be0 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e1affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e1c09e3b0 0x7f5e1c1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e1affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e1c09e3b0 0x7f5e1c1356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42736/0 (socket says 172.21.15.89:42736) 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e1affd700 1 -- 172.21.15.89:0/3912015177 learned_addr learned my addr 172.21.15.89:0/3912015177 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.324+0000 7f5e1bfff700 1 --2- 172.21.15.89:0/3912015177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c135be0 0x7f5e1c137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.325+0000 7f5e1affd700 1 -- 172.21.15.89:0/3912015177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c135be0 msgr2=0x7f5e1c137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.325+0000 7f5e1affd700 1 --2- 172.21.15.89:0/3912015177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c135be0 0x7f5e1c137fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.325+0000 7f5e1affd700 1 -- 172.21.15.89:0/3912015177 >> v1:172.21.15.89:6789/0 conn(0x7f5e1c09d8d0 legacy=0x7f5e1c09dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.325+0000 7f5e1affd700 1 -- 172.21.15.89:0/3912015177 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c09ae00 con 0x7f5e1c09e3b0 2023-09-30T22:21:59.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.325+0000 7f5e1affd700 1 --2- 172.21.15.89:0/3912015177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c09e3b0 0x7f5e1c1356a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5e100098f0 tx=0x7f5e10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12332475ee71406e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.326+0000 7f5e1a7fc700 1 -- 172.21.15.89:0/3912015177 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10005e30 con 0x7f5e1c09e3b0 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.326+0000 7f5e1a7fc700 1 -- 172.21.15.89:0/3912015177 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10006030 con 0x7f5e1c09e3b0 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.326+0000 7f5e1a7fc700 1 -- 172.21.15.89:0/3912015177 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e100063c0 con 0x7f5e1c09e3b0 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.327+0000 7f5e20b9e700 1 -- 172.21.15.89:0/3912015177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c09e3b0 msgr2=0x7f5e1c1356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.327+0000 7f5e20b9e700 1 --2- 172.21.15.89:0/3912015177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c09e3b0 0x7f5e1c1356a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5e100098f0 tx=0x7f5e10009c00 comp rx=0 tx=0).stop 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.327+0000 7f5e20b9e700 1 -- 172.21.15.89:0/3912015177 shutdown_connections 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.327+0000 7f5e20b9e700 1 --2- 172.21.15.89:0/3912015177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c135be0 0x7f5e1c137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.327+0000 7f5e20b9e700 1 --2- 172.21.15.89:0/3912015177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c09e3b0 0x7f5e1c1356a0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.327+0000 7f5e20b9e700 1 -- 172.21.15.89:0/3912015177 >> 172.21.15.89:0/3912015177 conn(0x7f5e1c0a2e20 msgr2=0x7f5e1c0a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.327+0000 7f5e20b9e700 1 -- 172.21.15.89:0/3912015177 shutdown_connections 2023-09-30T22:21:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.327+0000 7f5e20b9e700 1 -- 172.21.15.89:0/3912015177 wait complete. 2023-09-30T22:21:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.328+0000 7f5e20b9e700 1 Processor -- start 2023-09-30T22:21:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.329+0000 7f5e20b9e700 1 -- start start 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.329+0000 7f5e20b9e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c09d8d0 0x7f5e1c134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.329+0000 7f5e20b9e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e1c09e3b0 0x7f5e1c134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.329+0000 7f5e20b9e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c135be0 0x7f5e1c1354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.329+0000 7f5e20b9e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1324b0 con 0x7f5e1c135be0 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.329+0000 7f5e20b9e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c138860 con 0x7f5e1c09e3b0 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.329+0000 7f5e20b9e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1338f0 con 0x7f5e1c09d8d0 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.329+0000 7f5e1affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e1c09e3b0 0x7f5e1c134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e1c09e3b0 0x7f5e1c134fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60764/0 (socket says 172.21.15.89:60764) 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1affd700 1 -- 172.21.15.89:0/4256589557 learned_addr learned my addr 172.21.15.89:0/4256589557 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1b7fe700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c09d8d0 0x7f5e1c134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1bfff700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c135be0 0x7f5e1c1354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1bfff700 1 -- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c09d8d0 msgr2=0x7f5e1c134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1bfff700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c09d8d0 0x7f5e1c134a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1bfff700 1 -- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e1c09e3b0 msgr2=0x7f5e1c134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1bfff700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e1c09e3b0 0x7f5e1c134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1bfff700 1 -- 172.21.15.89:0/4256589557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10009500 con 0x7f5e1c135be0 2023-09-30T22:21:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1b7fe700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c09d8d0 0x7f5e1c134a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1affd700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e1c09e3b0 0x7f5e1c134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e1bfff700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c135be0 0x7f5e1c1354f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400efc0 tx=0x7f5e1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e03fff700 1 -- 172.21.15.89:0/4256589557 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400cc90 con 0x7f5e1c135be0 2023-09-30T22:21:59.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e03fff700 1 -- 172.21.15.89:0/4256589557 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1400f040 con 0x7f5e1c135be0 2023-09-30T22:21:59.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1c133e90 con 0x7f5e1c135be0 2023-09-30T22:21:59.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.330+0000 7f5e03fff700 1 -- 172.21.15.89:0/4256589557 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14009e60 con 0x7f5e1c135be0 2023-09-30T22:21:59.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.331+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1c133bb0 con 0x7f5e1c135be0 2023-09-30T22:21:59.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.331+0000 7f5e03fff700 1 -- 172.21.15.89:0/4256589557 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e14014480 con 0x7f5e1c135be0 2023-09-30T22:21:59.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.332+0000 7f5e03fff700 1 -- 172.21.15.89:0/4256589557 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5e14040180 con 0x7f5e1c135be0 2023-09-30T22:21:59.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.332+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1c071370 con 0x7f5e1c135be0 2023-09-30T22:21:59.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.339+0000 7f5e03fff700 1 -- 172.21.15.89:0/4256589557 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e14018030 con 0x7f5e1c135be0 2023-09-30T22:21:59.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07de207700 1 Processor -- start 2023-09-30T22:21:59.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07de207700 1 -- start start 2023-09-30T22:21:59.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07de207700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8133420 0x7f07d81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07de207700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07d8133e30 0x7f07d812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07de207700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d806b990 con 0x7f07d8134260 2023-09-30T22:21:59.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07de207700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d80a0590 con 0x7f07d8133e30 2023-09-30T22:21:59.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07de207700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07d809fb80 con 0x7f07d8133420 2023-09-30T22:21:59.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07d7fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f07d8134260 0x7f07d81304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42944/0 (socket says 172.21.15.89:42944) 2023-09-30T22:21:59.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07d7fff700 1 -- 172.21.15.89:0/1106953047 learned_addr learned my addr 172.21.15.89:0/1106953047 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:21:59.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07d77fe700 1 --2- 172.21.15.89:0/1106953047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8133420 0x7f07d81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.368+0000 7f07d6ffd700 1 --2- 172.21.15.89:0/1106953047 >> v2:172.21.15.89:3300/0 conn(0x7f07d8133e30 0x7f07d812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.369+0000 7f07d6ffd700 1 -- 172.21.15.89:0/1106953047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8133420 msgr2=0x7f07d81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.369+0000 7f07d6ffd700 1 --2- 172.21.15.89:0/1106953047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8133420 0x7f07d81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.369+0000 7f07d6ffd700 1 -- 172.21.15.89:0/1106953047 >> v1:172.21.15.89:6789/0 conn(0x7f07d8134260 legacy=0x7f07d81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.369+0000 7f07d6ffd700 1 -- 172.21.15.89:0/1106953047 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d80a0030 con 0x7f07d8133e30 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.369+0000 7f07d6ffd700 1 --2- 172.21.15.89:0/1106953047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133e30 0x7f07d812ff90 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009840 tx=0x7f07cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d07e9b3f0e2100e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.369+0000 7f07d67fc700 1 -- 172.21.15.89:0/1106953047 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc011830 con 0x7f07d8133e30 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.369+0000 7f07d67fc700 1 -- 172.21.15.89:0/1106953047 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc011990 con 0x7f07d8133e30 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.369+0000 7f07d67fc700 1 -- 172.21.15.89:0/1106953047 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc011d20 con 0x7f07d8133e30 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.370+0000 7f07de207700 1 -- 172.21.15.89:0/1106953047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133e30 msgr2=0x7f07d812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.370+0000 7f07de207700 1 --2- 172.21.15.89:0/1106953047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133e30 0x7f07d812ff90 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009840 tx=0x7f07cc009b50 comp rx=0 tx=0).stop 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.370+0000 7f07de207700 1 -- 172.21.15.89:0/1106953047 shutdown_connections 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.370+0000 7f07de207700 1 --2- 172.21.15.89:0/1106953047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8133420 0x7f07d81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.370+0000 7f07de207700 1 --2- 172.21.15.89:0/1106953047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133e30 0x7f07d812ff90 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.370+0000 7f07de207700 1 -- 172.21.15.89:0/1106953047 >> 172.21.15.89:0/1106953047 conn(0x7f07d80a4210 msgr2=0x7f07d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.370+0000 7f07de207700 1 -- 172.21.15.89:0/1106953047 shutdown_connections 2023-09-30T22:21:59.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.370+0000 7f07de207700 1 -- 172.21.15.89:0/1106953047 wait complete. 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.371+0000 7f07de207700 1 Processor -- start 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.371+0000 7f07de207700 1 -- start start 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07de207700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133420 0x7f07d809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07de207700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07d8133e30 0x7f07d809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07de207700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8134260 0x7f07d809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07de207700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8120010 con 0x7f07d8133420 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07de207700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d81201f0 con 0x7f07d8133e30 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07de207700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07d812da70 con 0x7f07d8134260 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07d77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133420 0x7f07d809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07d77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133420 0x7f07d809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42752/0 (socket says 172.21.15.89:42752) 2023-09-30T22:21:59.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07d77fe700 1 -- 172.21.15.89:0/1285418621 learned_addr learned my addr 172.21.15.89:0/1285418621 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07d6ffd700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07d8133e30 0x7f07d809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.372+0000 7f07d7fff700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8134260 0x7f07d809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07d77fe700 1 -- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8134260 msgr2=0x7f07d809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07d77fe700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8134260 0x7f07d809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07d77fe700 1 -- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07d8133e30 msgr2=0x7f07d809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07d77fe700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07d8133e30 0x7f07d809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07d77fe700 1 -- 172.21.15.89:0/1285418621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0094a0 con 0x7f07d8133420 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07d77fe700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133420 0x7f07d809c480 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f07c800e9e0 tx=0x7f07c800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07bffff700 1 -- 172.21.15.89:0/1285418621 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8009710 con 0x7f07d8133420 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07bffff700 1 -- 172.21.15.89:0/1285418621 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c800ee80 con 0x7f07d8133420 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07bffff700 1 -- 172.21.15.89:0/1285418621 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c800ca50 con 0x7f07d8133420 2023-09-30T22:21:59.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d811fc50 con 0x7f07d8133420 2023-09-30T22:21:59.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.373+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d812d620 con 0x7f07d8133420 2023-09-30T22:21:59.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.374+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d809de40 con 0x7f07d8133420 2023-09-30T22:21:59.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.375+0000 7f07bffff700 1 -- 172.21.15.89:0/1285418621 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f07c800cbb0 con 0x7f07d8133420 2023-09-30T22:21:59.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.375+0000 7f07bffff700 1 -- 172.21.15.89:0/1285418621 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f07c803fea0 con 0x7f07d8133420 2023-09-30T22:21:59.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.379+0000 7f07bffff700 1 -- 172.21.15.89:0/1285418621 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07c8011350 con 0x7f07d8133420 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.463+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0ea811e750 con 0x7f0ea81338a0 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.464+0000 7f0e9effd700 1 -- 172.21.15.89:0/2554745906 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ea4017020 con 0x7f0ea81338a0 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:59.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea81338a0 msgr2=0x7f0ea81c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 --2- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea81338a0 0x7f0ea81c8490 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c500 tx=0x7f0ea400c810 comp rx=0 tx=0).stop 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 shutdown_connections 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 --2- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8139030 0x7f0ea811adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 --2- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ea81338a0 0x7f0ea81c8490 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 --2- 172.21.15.89:0/2554745906 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ea8136710 0x7f0ea81c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 >> 172.21.15.89:0/2554745906 conn(0x7f0ea80a2aa0 msgr2=0x7f0ea81344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 shutdown_connections 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.467+0000 7f0eb04d4700 1 -- 172.21.15.89:0/2554745906 wait complete. 2023-09-30T22:21:59.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:21:59.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.501+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb4d411d490 con 0x7fb4d406fb20 2023-09-30T22:21:59.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.501+0000 7fb4c9ffb700 1 -- 172.21.15.89:0/2855699886 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4d00152b0 con 0x7fb4d406fb20 2023-09-30T22:21:59.502 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:21:59.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d406fb20 msgr2=0x7fb4d411a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d406fb20 0x7fb4d411a560 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000ed70 tx=0x7fb4d000d630 comp rx=0 tx=0).stop 2023-09-30T22:21:59.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 shutdown_connections 2023-09-30T22:21:59.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4d4130e50 0x7fb4d411afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4d406fb20 0x7fb4d411a560 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 --2- 172.21.15.89:0/2855699886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb4d4130540 0x7fb4d411aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 >> 172.21.15.89:0/2855699886 conn(0x7fb4d40a2aa0 msgr2=0x7fb4d412d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 shutdown_connections 2023-09-30T22:21:59.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.503+0000 7fb4db979700 1 -- 172.21.15.89:0/2855699886 wait complete. 2023-09-30T22:21:59.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:21:59.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.551+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5e1c04f2a0 con 0x7f5e1c135be0 2023-09-30T22:21:59.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.552+0000 7f5e03fff700 1 -- 172.21.15.89:0/4256589557 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e14007920 con 0x7f5e1c135be0 2023-09-30T22:21:59.553 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:21:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c135be0 msgr2=0x7f5e1c1354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c135be0 0x7f5e1c1354f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400efc0 tx=0x7f5e1400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 shutdown_connections 2023-09-30T22:21:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e1c135be0 0x7f5e1c1354f0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5e1c09d8d0 0x7f5e1c134a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 --2- 172.21.15.89:0/4256589557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e1c09e3b0 0x7f5e1c134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 >> 172.21.15.89:0/4256589557 conn(0x7f5e1c0a2e20 msgr2=0x7f5e1c06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 shutdown_connections 2023-09-30T22:21:59.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.554+0000 7f5e20b9e700 1 -- 172.21.15.89:0/4256589557 wait complete. 2023-09-30T22:21:59.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:21:59.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.576+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f07d809e330 con 0x7f07d8133420 2023-09-30T22:21:59.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.577+0000 7f07bffff700 1 -- 172.21.15.89:0/1285418621 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f07c802a090 con 0x7f07d8133420 2023-09-30T22:21:59.578 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:21:59.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133420 msgr2=0x7f07d809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133420 0x7f07d809c480 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f07c800e9e0 tx=0x7f07c800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 shutdown_connections 2023-09-30T22:21:59.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07d8134260 0x7f07d809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07d8133420 0x7f07d809c480 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 --2- 172.21.15.89:0/1285418621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07d8133e30 0x7f07d809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:21:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 >> 172.21.15.89:0/1285418621 conn(0x7f07d80a4210 msgr2=0x7f07d80a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 shutdown_connections 2023-09-30T22:21:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:21:59.579+0000 7f07de207700 1 -- 172.21.15.89:0/1285418621 wait complete. 2023-09-30T22:21:59.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:00.485 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:00.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:00.567 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:00.592 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:00.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.632+0000 7f9e1af56700 1 Processor -- start 2023-09-30T22:22:00.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.632+0000 7f9e1af56700 1 -- start start 2023-09-30T22:22:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.632+0000 7f9e1af56700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409d6f0 0x7f9e1409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.632+0000 7f9e1af56700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e1409e410 0x7f9e14134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.632+0000 7f9e1af56700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1409b120 con 0x7f9e1409e000 2023-09-30T22:22:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.632+0000 7f9e1af56700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1406b460 con 0x7f9e1409e410 2023-09-30T22:22:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.632+0000 7f9e1af56700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1409a7e0 con 0x7f9e1409d6f0 2023-09-30T22:22:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e18cf2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409d6f0 0x7f9e1409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e18cf2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409d6f0 0x7f9e1409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60788/0 (socket says 172.21.15.89:60788) 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e18cf2700 1 -- 172.21.15.89:0/570454462 learned_addr learned my addr 172.21.15.89:0/570454462 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e18cf2700 1 -- 172.21.15.89:0/570454462 >> v2:172.21.15.89:3300/0 conn(0x7f9e1409e410 msgr2=0x7f9e14134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e18cf2700 1 --2- 172.21.15.89:0/570454462 >> v2:172.21.15.89:3300/0 conn(0x7f9e1409e410 0x7f9e14134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e18cf2700 1 -- 172.21.15.89:0/570454462 >> v1:172.21.15.89:6789/0 conn(0x7f9e1409e000 legacy=0x7f9e141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e18cf2700 1 -- 172.21.15.89:0/570454462 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1409a500 con 0x7f9e1409d6f0 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e18cf2700 1 --2- 172.21.15.89:0/570454462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409d6f0 0x7f9e1409dac0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009920 tx=0x7f9e08009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f47dcca50c2da5e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e137fe700 1 -- 172.21.15.89:0/570454462 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08004230 con 0x7f9e1409d6f0 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e137fe700 1 -- 172.21.15.89:0/570454462 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e08004390 con 0x7f9e1409d6f0 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.633+0000 7f9e137fe700 1 -- 172.21.15.89:0/570454462 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08009e70 con 0x7f9e1409d6f0 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.634+0000 7f9e1af56700 1 -- 172.21.15.89:0/570454462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409d6f0 msgr2=0x7f9e1409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.634+0000 7f9e1af56700 1 --2- 172.21.15.89:0/570454462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409d6f0 0x7f9e1409dac0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009920 tx=0x7f9e08009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.634+0000 7f9e1af56700 1 -- 172.21.15.89:0/570454462 shutdown_connections 2023-09-30T22:22:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.634+0000 7f9e1af56700 1 --2- 172.21.15.89:0/570454462 >> v2:172.21.15.89:3300/0 conn(0x7f9e1409e410 0x7f9e14134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.634+0000 7f9e1af56700 1 --2- 172.21.15.89:0/570454462 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409d6f0 0x7f9e1409dac0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.634+0000 7f9e1af56700 1 -- 172.21.15.89:0/570454462 >> 172.21.15.89:0/570454462 conn(0x7f9e14066f50 msgr2=0x7f9e1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.634+0000 7f9e1af56700 1 -- 172.21.15.89:0/570454462 shutdown_connections 2023-09-30T22:22:00.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.634+0000 7f9e1af56700 1 -- 172.21.15.89:0/570454462 wait complete. 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.635+0000 7f9e1af56700 1 Processor -- start 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.636+0000 7f9e1af56700 1 -- start start 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.636+0000 7f9e1af56700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e1409d6f0 0x7f9e141a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.636+0000 7f9e1af56700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e1409e000 0x7f9e141a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.636+0000 7f9e1af56700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409e410 0x7f9e141a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.636+0000 7f9e1af56700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141a7bd0 con 0x7f9e1409e000 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.636+0000 7f9e1af56700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141a8e20 con 0x7f9e1409d6f0 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.636+0000 7f9e1af56700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141a81b0 con 0x7f9e1409e410 2023-09-30T22:22:00.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409e410 0x7f9e141a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e1409e000 0x7f9e141a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409e410 0x7f9e141a6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60790/0 (socket says 172.21.15.89:60790) 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 -- 172.21.15.89:0/2674574480 learned_addr learned my addr 172.21.15.89:0/2674574480 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e1409e000 0x7f9e141a6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47966/0 (socket says 172.21.15.89:47966) 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e18cf2700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e1409d6f0 0x7f9e141a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 -- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e1409d6f0 msgr2=0x7f9e141a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e1409d6f0 0x7f9e141a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 -- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e1409e000 msgr2=0x7f9e141a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e1409e000 0x7f9e141a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 -- 172.21.15.89:0/2674574480 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08009550 con 0x7f9e1409e410 2023-09-30T22:22:00.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e13fff700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e1409e000 0x7f9e141a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e194f3700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409e410 0x7f9e141a6a40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00b970 tx=0x7f9e0c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.637+0000 7f9e117fa700 1 -- 172.21.15.89:0/2674574480 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00d860 con 0x7f9e1409e410 2023-09-30T22:22:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.638+0000 7f9e117fa700 1 -- 172.21.15.89:0/2674574480 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c00d9e0 con 0x7f9e1409e410 2023-09-30T22:22:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.638+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e141a9800 con 0x7f9e1409e410 2023-09-30T22:22:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.638+0000 7f9e117fa700 1 -- 172.21.15.89:0/2674574480 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00dd90 con 0x7f9e1409e410 2023-09-30T22:22:00.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.638+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1409c710 con 0x7f9e1409e410 2023-09-30T22:22:00.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.639+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e14074ea0 con 0x7f9e1409e410 2023-09-30T22:22:00.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.639+0000 7f9e117fa700 1 -- 172.21.15.89:0/2674574480 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e0c00e430 con 0x7f9e1409e410 2023-09-30T22:22:00.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.639+0000 7f9e117fa700 1 -- 172.21.15.89:0/2674574480 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9e0c006e40 con 0x7f9e1409e410 2023-09-30T22:22:00.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.644+0000 7f9e117fa700 1 -- 172.21.15.89:0/2674574480 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e0c044060 con 0x7f9e1409e410 2023-09-30T22:22:00.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.665+0000 7ff8a69f6700 1 Processor -- start 2023-09-30T22:22:00.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff8a69f6700 1 -- start start 2023-09-30T22:22:00.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff8a69f6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8a01338a0 0x7ff8a0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff8a69f6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff8a69f6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a006ef30 con 0x7ff8a009e760 2023-09-30T22:22:00.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff8a69f6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a006fef0 con 0x7ff8a01338a0 2023-09-30T22:22:00.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff8a69f6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01334b0 con 0x7ff8a009de50 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff89ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8a01338a0 0x7ff8a0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff89ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8a01338a0 0x7ff8a0133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47972/0 (socket says 172.21.15.89:47972) 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60804/0 (socket says 172.21.15.89:60804) 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.666+0000 7ff89ffff700 1 -- 172.21.15.89:0/616842761 learned_addr learned my addr 172.21.15.89:0/616842761 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89f7fe700 1 -- 172.21.15.89:0/616842761 >> v2:172.21.15.89:3300/0 conn(0x7ff8a01338a0 msgr2=0x7ff8a0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89f7fe700 1 --2- 172.21.15.89:0/616842761 >> v2:172.21.15.89:3300/0 conn(0x7ff8a01338a0 0x7ff8a0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89f7fe700 1 -- 172.21.15.89:0/616842761 >> v1:172.21.15.89:6789/0 conn(0x7ff8a009e760 legacy=0x7ff8a013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89f7fe700 1 -- 172.21.15.89:0/616842761 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a006fc30 con 0x7ff8a009de50 2023-09-30T22:22:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89ffff700 1 --2- 172.21.15.89:0/616842761 >> v2:172.21.15.89:3300/0 conn(0x7ff8a01338a0 0x7ff8a0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:00.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89f7fe700 1 --2- 172.21.15.89:0/616842761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a009e220 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff894009840 tx=0x7ff894009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f52295148e4c6302 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:00.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89effd700 1 -- 172.21.15.89:0/616842761 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894011830 con 0x7ff8a009de50 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89effd700 1 -- 172.21.15.89:0/616842761 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff894011990 con 0x7ff8a009de50 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.667+0000 7ff89effd700 1 -- 172.21.15.89:0/616842761 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894011d20 con 0x7ff8a009de50 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.668+0000 7ff8a69f6700 1 -- 172.21.15.89:0/616842761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 msgr2=0x7ff8a009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.668+0000 7ff8a69f6700 1 --2- 172.21.15.89:0/616842761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a009e220 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff894009840 tx=0x7ff894009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.668+0000 7ff8a69f6700 1 -- 172.21.15.89:0/616842761 shutdown_connections 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.668+0000 7ff8a69f6700 1 --2- 172.21.15.89:0/616842761 >> v2:172.21.15.89:3300/0 conn(0x7ff8a01338a0 0x7ff8a0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.668+0000 7ff8a69f6700 1 --2- 172.21.15.89:0/616842761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a009e220 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.668+0000 7ff8a69f6700 1 -- 172.21.15.89:0/616842761 >> 172.21.15.89:0/616842761 conn(0x7ff8a00a2aa0 msgr2=0x7ff8a00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.668+0000 7ff8a69f6700 1 -- 172.21.15.89:0/616842761 shutdown_connections 2023-09-30T22:22:00.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.668+0000 7ff8a69f6700 1 -- 172.21.15.89:0/616842761 wait complete. 2023-09-30T22:22:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.669+0000 7ff8a69f6700 1 Processor -- start 2023-09-30T22:22:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.669+0000 7ff8a69f6700 1 -- start start 2023-09-30T22:22:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a69f6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a011b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a69f6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8a009e760 0x7ff8a011bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a69f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8a01338a0 0x7ff8a011c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a69f6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a011cd40 con 0x7ff8a01338a0 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a69f6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a012ec60 con 0x7ff8a009e760 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a69f6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a011daa0 con 0x7ff8a009de50 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a4f93700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8a01338a0 0x7ff8a011c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a4f93700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8a01338a0 0x7ff8a011c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47978/0 (socket says 172.21.15.89:47978) 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff8a4f93700 1 -- 172.21.15.89:0/839778351 learned_addr learned my addr 172.21.15.89:0/839778351 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff89ffff700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a011b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.670+0000 7ff89f7fe700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8a009e760 0x7ff8a011bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff8a4f93700 1 -- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 msgr2=0x7ff8a011b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff8a4f93700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a011b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff8a4f93700 1 -- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8a009e760 msgr2=0x7ff8a011bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff8a4f93700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8a009e760 0x7ff8a011bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff8a4f93700 1 -- 172.21.15.89:0/839778351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff890009550 con 0x7ff8a01338a0 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff89ffff700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a011b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff8a4f93700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8a01338a0 0x7ff8a011c2f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff89800cc80 tx=0x7ff89800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff89cff9700 1 -- 172.21.15.89:0/839778351 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898003c00 con 0x7ff8a01338a0 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff89cff9700 1 -- 172.21.15.89:0/839778351 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff898003d60 con 0x7ff8a01338a0 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8940094a0 con 0x7ff8a01338a0 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff89cff9700 1 -- 172.21.15.89:0/839778351 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800aaa0 con 0x7ff8a01338a0 2023-09-30T22:22:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.671+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a011f100 con 0x7ff8a01338a0 2023-09-30T22:22:00.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.672+0000 7ff89cff9700 1 -- 172.21.15.89:0/839778351 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff89800ac00 con 0x7ff8a01338a0 2023-09-30T22:22:00.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.672+0000 7ff89cff9700 1 -- 172.21.15.89:0/839778351 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff89803f4c0 con 0x7ff8a01338a0 2023-09-30T22:22:00.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.674+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a011f100 con 0x7ff8a01338a0 2023-09-30T22:22:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.682+0000 7ff89cff9700 1 -- 172.21.15.89:0/839778351 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8a011f100 con 0x7ff8a01338a0 2023-09-30T22:22:00.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.713+0000 7f2ea2a24700 1 Processor -- start 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.713+0000 7f2ea2a24700 1 -- start start 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea2a24700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea2a24700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea2a24700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c09b4f0 con 0x7f2e9c131810 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea2a24700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c1313c0 con 0x7f2e9c136f50 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea2a24700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c06e8c0 con 0x7f2e9c134630 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea0fc1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea0fc1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35260/0 (socket says 172.21.15.89:35260) 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea0fc1700 1 -- 172.21.15.89:0/1667926871 learned_addr learned my addr 172.21.15.89:0/1667926871 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2e9b7fe700 1 --2- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea0fc1700 1 -- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 msgr2=0x7f2e9c136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea0fc1700 1 --2- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea0fc1700 1 -- 172.21.15.89:0/1667926871 >> v1:172.21.15.89:6789/0 conn(0x7f2e9c131810 legacy=0x7f2e9c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea0fc1700 1 -- 172.21.15.89:0/1667926871 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c071d20 con 0x7f2e9c136f50 2023-09-30T22:22:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2e9b7fe700 1 --2- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.714+0000 7f2ea0fc1700 1 --2- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c139330 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009e80 tx=0x7f2e9400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aafabfecbcb43c6f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2e9affd700 1 -- 172.21.15.89:0/1667926871 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400e070 con 0x7f2e9c136f50 2023-09-30T22:22:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2e9affd700 1 -- 172.21.15.89:0/1667926871 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e94003a20 con 0x7f2e9c136f50 2023-09-30T22:22:00.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2e9affd700 1 -- 172.21.15.89:0/1667926871 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94003dd0 con 0x7f2e9c136f50 2023-09-30T22:22:00.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2ea2a24700 1 -- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 msgr2=0x7f2e9c139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2ea2a24700 1 --2- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c139330 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009e80 tx=0x7f2e9400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2ea2a24700 1 -- 172.21.15.89:0/1667926871 shutdown_connections 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2ea2a24700 1 --2- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c139330 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2ea2a24700 1 --2- 172.21.15.89:0/1667926871 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2ea2a24700 1 -- 172.21.15.89:0/1667926871 >> 172.21.15.89:0/1667926871 conn(0x7f2e9c0a2aa0 msgr2=0x7f2e9c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.715+0000 7f2ea2a24700 1 -- 172.21.15.89:0/1667926871 shutdown_connections 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.716+0000 7f2ea2a24700 1 -- 172.21.15.89:0/1667926871 wait complete. 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.716+0000 7f2ea2a24700 1 Processor -- start 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2ea2a24700 1 -- start start 2023-09-30T22:22:00.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2ea2a24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e9c131810 0x7f2e9c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2ea2a24700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2ea2a24700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2ea2a24700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c11b670 con 0x7f2e9c131810 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2ea2a24700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c12d8f0 con 0x7f2e9c136f50 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2ea2a24700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c130ee0 con 0x7f2e9c134630 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e9c131810 0x7f2e9c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e9c131810 0x7f2e9c118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47992/0 (socket says 172.21.15.89:47992) 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2ea0fc1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2e9bfff700 1 -- 172.21.15.89:0/2582690698 learned_addr learned my addr 172.21.15.89:0/2582690698 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.717+0000 7f2e9b7fe700 1 --2- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e9bfff700 1 -- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 msgr2=0x7f2e9c119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e9bfff700 1 --2- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e9bfff700 1 -- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 msgr2=0x7f2e9c119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e9bfff700 1 --2- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e9bfff700 1 -- 172.21.15.89:0/2582690698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e94009b30 con 0x7f2e9c131810 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e9bfff700 1 --2- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e9c131810 0x7f2e9c118cd0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00c4d0 tx=0x7f2e8c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e98ff9700 1 -- 172.21.15.89:0/2582690698 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c011790 con 0x7f2e9c131810 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e98ff9700 1 -- 172.21.15.89:0/2582690698 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c0118f0 con 0x7f2e9c131810 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2e98ff9700 1 -- 172.21.15.89:0/2582690698 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c011ca0 con 0x7f2e9c131810 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c11a230 con 0x7f2e9c131810 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.718+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c11a750 con 0x7f2e9c131810 2023-09-30T22:22:00.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.719+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c11b310 con 0x7f2e9c131810 2023-09-30T22:22:00.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.719+0000 7f2e98ff9700 1 -- 172.21.15.89:0/2582690698 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e8c01c460 con 0x7f2e9c131810 2023-09-30T22:22:00.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.720+0000 7f2e98ff9700 1 -- 172.21.15.89:0/2582690698 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2e8c03f360 con 0x7f2e9c131810 2023-09-30T22:22:00.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.728+0000 7f2e98ff9700 1 -- 172.21.15.89:0/2582690698 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e9c11b310 con 0x7f2e9c131810 2023-09-30T22:22:00.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.753+0000 7f6907734700 1 Processor -- start 2023-09-30T22:22:00.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.753+0000 7f6907734700 1 -- start start 2023-09-30T22:22:00.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6907734700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f690009c650 0x7f690009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6907734700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f690009cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6907734700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f6900131df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6907734700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690009ba70 con 0x7f690009c650 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6907734700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690009d4a0 con 0x7f6900131970 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6907734700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f690009e8f0 con 0x7f69000a0090 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6905cd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f6900131df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6905cd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f6900131df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35282/0 (socket says 172.21.15.89:35282) 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6905cd1700 1 -- 172.21.15.89:0/1587199310 learned_addr learned my addr 172.21.15.89:0/1587199310 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f6904ccf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f690009cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.754+0000 7f69054d0700 1 --2- 172.21.15.89:0/1587199310 >> v2:172.21.15.89:3300/0 conn(0x7f690009c650 0x7f690009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f6905cd1700 1 -- 172.21.15.89:0/1587199310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 msgr2=0x7f690009cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f6905cd1700 1 --2- 172.21.15.89:0/1587199310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f690009cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f6905cd1700 1 -- 172.21.15.89:0/1587199310 >> v2:172.21.15.89:3300/0 conn(0x7f690009c650 msgr2=0x7f690009ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f6905cd1700 1 --2- 172.21.15.89:0/1587199310 >> v2:172.21.15.89:3300/0 conn(0x7f690009c650 0x7f690009ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f6905cd1700 1 -- 172.21.15.89:0/1587199310 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000a0760 con 0x7f6900131970 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f6904ccf700 1 --2- 172.21.15.89:0/1587199310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f690009cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f6905cd1700 1 --2- 172.21.15.89:0/1587199310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f6900131df0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009e50 tx=0x7f68f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede7007b7a091c5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f68f7fff700 1 -- 172.21.15.89:0/1587199310 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800bb10 con 0x7f6900131970 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f68f7fff700 1 -- 172.21.15.89:0/1587199310 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f800bc70 con 0x7f6900131970 2023-09-30T22:22:00.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.755+0000 7f68f7fff700 1 -- 172.21.15.89:0/1587199310 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f80104d0 con 0x7f6900131970 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 -- 172.21.15.89:0/1587199310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 msgr2=0x7f6900131df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 --2- 172.21.15.89:0/1587199310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f6900131df0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009e50 tx=0x7f68f800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 -- 172.21.15.89:0/1587199310 shutdown_connections 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 --2- 172.21.15.89:0/1587199310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f6900131df0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 --2- 172.21.15.89:0/1587199310 >> v2:172.21.15.89:3300/0 conn(0x7f690009c650 0x7f690009ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 --2- 172.21.15.89:0/1587199310 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f690009cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 -- 172.21.15.89:0/1587199310 >> 172.21.15.89:0/1587199310 conn(0x7f69000a3f70 msgr2=0x7f69000a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 -- 172.21.15.89:0/1587199310 shutdown_connections 2023-09-30T22:22:00.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.756+0000 7f6907734700 1 -- 172.21.15.89:0/1587199310 wait complete. 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.757+0000 7f6907734700 1 Processor -- start 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.757+0000 7f6907734700 1 -- start start 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.757+0000 7f6907734700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690009c650 0x7f69001c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.757+0000 7f6907734700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f69001c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.757+0000 7f6907734700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f690011a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.757+0000 7f6907734700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69001c3ef0 con 0x7f690009c650 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.757+0000 7f6907734700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69001c4d50 con 0x7f6900131970 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.757+0000 7f6907734700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f690009d980 con 0x7f69000a0090 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f690011a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f690011a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35294/0 (socket says 172.21.15.89:35294) 2023-09-30T22:22:00.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 -- 172.21.15.89:0/1045078275 learned_addr learned my addr 172.21.15.89:0/1045078275 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6904ccf700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f69001c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f69054d0700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690009c650 0x7f69001c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 -- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 msgr2=0x7f69001c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f69001c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 -- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690009c650 msgr2=0x7f69001c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690009c650 0x7f69001c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 -- 172.21.15.89:0/1045078275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00094a0 con 0x7f6900131970 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f69054d0700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690009c650 0x7f69001c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6905cd1700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f690011a620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f68f800b370 tx=0x7f68f8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f68f5ffb700 1 -- 172.21.15.89:0/1045078275 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8004450 con 0x7f6900131970 2023-09-30T22:22:00.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f68f5ffb700 1 -- 172.21.15.89:0/1045078275 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f80045b0 con 0x7f6900131970 2023-09-30T22:22:00.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f8009b30 con 0x7f6900131970 2023-09-30T22:22:00.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f68f5ffb700 1 -- 172.21.15.89:0/1045078275 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8004960 con 0x7f6900131970 2023-09-30T22:22:00.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.758+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6900072180 con 0x7f6900131970 2023-09-30T22:22:00.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.759+0000 7f68f5ffb700 1 -- 172.21.15.89:0/1045078275 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68f80104d0 con 0x7f6900131970 2023-09-30T22:22:00.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.760+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69000747f0 con 0x7f6900131970 2023-09-30T22:22:00.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.760+0000 7f68f5ffb700 1 -- 172.21.15.89:0/1045078275 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f68f8045640 con 0x7f6900131970 2023-09-30T22:22:00.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.764+0000 7f68f5ffb700 1 -- 172.21.15.89:0/1045078275 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68f8010780 con 0x7f6900131970 2023-09-30T22:22:00.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.855+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e1406c360 con 0x7f9e1409e410 2023-09-30T22:22:00.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.856+0000 7f9e117fa700 1 -- 172.21.15.89:0/2674574480 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e0c044030 con 0x7f9e1409e410 2023-09-30T22:22:00.870 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:00.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.858+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409e410 msgr2=0x7f9e141a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.858+0000 7f9e1af56700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409e410 0x7f9e141a6a40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00b970 tx=0x7f9e0c00bc80 comp rx=0 tx=0).stop 2023-09-30T22:22:00.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.858+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 shutdown_connections 2023-09-30T22:22:00.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.858+0000 7f9e1af56700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e1409e410 0x7f9e141a6a40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.858+0000 7f9e1af56700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e1409d6f0 0x7f9e141a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.858+0000 7f9e1af56700 1 --2- 172.21.15.89:0/2674574480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e1409e000 0x7f9e141a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.858+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 >> 172.21.15.89:0/2674574480 conn(0x7f9e14066f50 msgr2=0x7f9e141baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:00.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.859+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 shutdown_connections 2023-09-30T22:22:00.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.859+0000 7f9e1af56700 1 -- 172.21.15.89:0/2674574480 wait complete. 2023-09-30T22:22:00.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:00.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.880+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8a011f100 con 0x7ff8a01338a0 2023-09-30T22:22:00.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.881+0000 7ff89cff9700 1 -- 172.21.15.89:0/839778351 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8a011f100 con 0x7ff8a01338a0 2023-09-30T22:22:00.882 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:00.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8a01338a0 msgr2=0x7ff8a011c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8a01338a0 0x7ff8a011c2f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff89800cc80 tx=0x7ff89800cf90 comp rx=0 tx=0).stop 2023-09-30T22:22:00.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 shutdown_connections 2023-09-30T22:22:00.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8a01338a0 0x7ff8a011c2f0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8a009de50 0x7ff8a011b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 --2- 172.21.15.89:0/839778351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8a009e760 0x7ff8a011bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 >> 172.21.15.89:0/839778351 conn(0x7ff8a00a2aa0 msgr2=0x7ff8a0130610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:00.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 shutdown_connections 2023-09-30T22:22:00.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.883+0000 7ff8a69f6700 1 -- 172.21.15.89:0/839778351 wait complete. 2023-09-30T22:22:00.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:00.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.932+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2e9c11b310 con 0x7f2e9c131810 2023-09-30T22:22:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.933+0000 7f2e98ff9700 1 -- 172.21.15.89:0/2582690698 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e9c11b310 con 0x7f2e9c131810 2023-09-30T22:22:00.934 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e9c131810 msgr2=0x7f2e9c118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 --2- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e9c131810 0x7f2e9c118cd0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00c4d0 tx=0x7f2e8c00c7e0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 shutdown_connections 2023-09-30T22:22:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 --2- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e9c136f50 0x7f2e9c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 --2- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e9c131810 0x7f2e9c118cd0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 --2- 172.21.15.89:0/2582690698 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e9c134630 0x7f2e9c119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 >> 172.21.15.89:0/2582690698 conn(0x7f2e9c0a2aa0 msgr2=0x7f2e9c12e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 shutdown_connections 2023-09-30T22:22:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.935+0000 7f2ea2a24700 1 -- 172.21.15.89:0/2582690698 wait complete. 2023-09-30T22:22:00.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:00.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.958+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f69000727e0 con 0x7f6900131970 2023-09-30T22:22:00.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.958+0000 7f68f5ffb700 1 -- 172.21.15.89:0/1045078275 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68f8030030 con 0x7f6900131970 2023-09-30T22:22:00.959 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:00.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 msgr2=0x7f690011a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f690011a620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f68f800b370 tx=0x7f68f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 shutdown_connections 2023-09-30T22:22:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6900131970 0x7f690011a620 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690009c650 0x7f69001c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 --2- 172.21.15.89:0/1045078275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f69000a0090 0x7f69001c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 >> 172.21.15.89:0/1045078275 conn(0x7f69000a3f70 msgr2=0x7f690012ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:00.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 shutdown_connections 2023-09-30T22:22:00.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:00.960+0000 7f6907734700 1 -- 172.21.15.89:0/1045078275 wait complete. 2023-09-30T22:22:00.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:01.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:01.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:01.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:01.973 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febaecc9700 1 Processor -- start 2023-09-30T22:22:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febaecc9700 1 -- start start 2023-09-30T22:22:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febaecc9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febaecc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febaecc9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba809b120 con 0x7feba809e000 2023-09-30T22:22:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febaecc9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba806b460 con 0x7feba809d6f0 2023-09-30T22:22:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febaecc9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feba809a7e0 con 0x7feba809e410 2023-09-30T22:22:02.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febad266700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febad266700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba8134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60876/0 (socket says 172.21.15.89:60876) 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febad266700 1 -- 172.21.15.89:0/3057435246 learned_addr learned my addr 172.21.15.89:0/3057435246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.022+0000 7febaca65700 1 --2- 172.21.15.89:0/3057435246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7feba77fe700 1 -- 172.21.15.89:0/3057435246 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189137855 0 0) 0x7feba809b120 con 0x7feba809e000 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7febaca65700 1 -- 172.21.15.89:0/3057435246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 msgr2=0x7feba8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7febaca65700 1 --2- 172.21.15.89:0/3057435246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7febaca65700 1 -- 172.21.15.89:0/3057435246 >> v1:172.21.15.89:6789/0 conn(0x7feba809e000 legacy=0x7feba81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7febaca65700 1 -- 172.21.15.89:0/3057435246 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba809a500 con 0x7feba809d6f0 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7febaca65700 1 --2- 172.21.15.89:0/3057435246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba809dac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009870 tx=0x7feb9c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4983be07aa5416ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7feba77fe700 1 -- 172.21.15.89:0/3057435246 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c011830 con 0x7feba809d6f0 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7feba77fe700 1 -- 172.21.15.89:0/3057435246 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c011990 con 0x7feba809d6f0 2023-09-30T22:22:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.023+0000 7feba77fe700 1 -- 172.21.15.89:0/3057435246 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c011d20 con 0x7feba809d6f0 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.024+0000 7febaecc9700 1 -- 172.21.15.89:0/3057435246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 msgr2=0x7feba809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.024+0000 7febaecc9700 1 --2- 172.21.15.89:0/3057435246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba809dac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009870 tx=0x7feb9c009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.024+0000 7febaecc9700 1 -- 172.21.15.89:0/3057435246 shutdown_connections 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.024+0000 7febaecc9700 1 --2- 172.21.15.89:0/3057435246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba809dac0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.024+0000 7febaecc9700 1 --2- 172.21.15.89:0/3057435246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.024+0000 7febaecc9700 1 -- 172.21.15.89:0/3057435246 >> 172.21.15.89:0/3057435246 conn(0x7feba8066f50 msgr2=0x7feba809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.024+0000 7febaecc9700 1 -- 172.21.15.89:0/3057435246 shutdown_connections 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.024+0000 7febaecc9700 1 -- 172.21.15.89:0/3057435246 wait complete. 2023-09-30T22:22:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.025+0000 7febaecc9700 1 Processor -- start 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.025+0000 7febaecc9700 1 -- start start 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaecc9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba81c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaecc9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feba809e000 0x7feba81c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaecc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba81c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaecc9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba81a69e0 con 0x7feba809e000 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaecc9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba809c780 con 0x7feba809d6f0 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaecc9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feba81a6540 con 0x7feba809e410 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaca65700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba81c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febad266700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba81c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaca65700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba81c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35326/0 (socket says 172.21.15.89:35326) 2023-09-30T22:22:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febad266700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba81c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60880/0 (socket says 172.21.15.89:60880) 2023-09-30T22:22:02.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.026+0000 7febaca65700 1 -- 172.21.15.89:0/3983675420 learned_addr learned my addr 172.21.15.89:0/3983675420 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7febaca65700 1 -- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 msgr2=0x7feba81c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7febaca65700 1 --2- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba81c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7febaca65700 1 -- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feba809e000 msgr2=0x7feba81c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7febaca65700 1 --2- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feba809e000 0x7feba81c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7febaca65700 1 -- 172.21.15.89:0/3983675420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c0094a0 con 0x7feba809d6f0 2023-09-30T22:22:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7febaca65700 1 --2- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba81c8040 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0090b0 tx=0x7feb9c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7feba57fa700 1 -- 172.21.15.89:0/3983675420 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0045d0 con 0x7feba809d6f0 2023-09-30T22:22:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7feba57fa700 1 -- 172.21.15.89:0/3983675420 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c01f070 con 0x7feba809d6f0 2023-09-30T22:22:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba80a1eb0 con 0x7feba809d6f0 2023-09-30T22:22:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7feba57fa700 1 -- 172.21.15.89:0/3983675420 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0038c0 con 0x7feba809d6f0 2023-09-30T22:22:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.027+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba81a8a60 con 0x7feba809d6f0 2023-09-30T22:22:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.028+0000 7feba57fa700 1 -- 172.21.15.89:0/3983675420 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb9c003a20 con 0x7feba809d6f0 2023-09-30T22:22:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.028+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba8074ea0 con 0x7feba809d6f0 2023-09-30T22:22:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.029+0000 7feba57fa700 1 -- 172.21.15.89:0/3983675420 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feb9c021070 con 0x7feba809d6f0 2023-09-30T22:22:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.034+0000 7feba57fa700 1 -- 172.21.15.89:0/3983675420 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb9c02f3a0 con 0x7feba809d6f0 2023-09-30T22:22:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.071+0000 7f12b51f8700 1 Processor -- start 2023-09-30T22:22:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12b51f8700 1 -- start start 2023-09-30T22:22:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12b51f8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009e930 0x7f12b009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12b51f8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f650 0x7f12b0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12b51f8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b0070940 con 0x7f12b009f240 2023-09-30T22:22:02.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12b51f8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b006fd60 con 0x7f12b009f650 2023-09-30T22:22:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12b51f8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12b006fa80 con 0x7f12b009e930 2023-09-30T22:22:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12aed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009e930 0x7f12b009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12aed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009e930 0x7f12b009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60884/0 (socket says 172.21.15.89:60884) 2023-09-30T22:22:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12aed9d700 1 -- 172.21.15.89:0/3561057854 learned_addr learned my addr 172.21.15.89:0/3561057854 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:02.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.072+0000 7f12af59e700 1 --2- 172.21.15.89:0/3561057854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f650 0x7f12b0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.073+0000 7f12aed9d700 1 -- 172.21.15.89:0/3561057854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f650 msgr2=0x7f12b0131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.073+0000 7f12aed9d700 1 --2- 172.21.15.89:0/3561057854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f650 0x7f12b0131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.073+0000 7f12aed9d700 1 -- 172.21.15.89:0/3561057854 >> v1:172.21.15.89:6789/0 conn(0x7f12b009f240 legacy=0x7f12b01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.073+0000 7f12aed9d700 1 -- 172.21.15.89:0/3561057854 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b009fb60 con 0x7f12b009e930 2023-09-30T22:22:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.073+0000 7f12aed9d700 1 --2- 172.21.15.89:0/3561057854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009e930 0x7f12b009ed00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009920 tx=0x7f12a0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f691cd8de25c399 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:02.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.073+0000 7f12add9b700 1 -- 172.21.15.89:0/3561057854 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0011830 con 0x7f12b009e930 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.073+0000 7f12add9b700 1 -- 172.21.15.89:0/3561057854 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0011990 con 0x7f12b009e930 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.073+0000 7f12add9b700 1 -- 172.21.15.89:0/3561057854 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0011d00 con 0x7f12b009e930 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.074+0000 7f12b51f8700 1 -- 172.21.15.89:0/3561057854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009e930 msgr2=0x7f12b009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.074+0000 7f12b51f8700 1 --2- 172.21.15.89:0/3561057854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009e930 0x7f12b009ed00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009920 tx=0x7f12a0009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.074+0000 7f12b51f8700 1 -- 172.21.15.89:0/3561057854 shutdown_connections 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.074+0000 7f12b51f8700 1 --2- 172.21.15.89:0/3561057854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f650 0x7f12b0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.074+0000 7f12b51f8700 1 --2- 172.21.15.89:0/3561057854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009e930 0x7f12b009ed00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.074+0000 7f12b51f8700 1 -- 172.21.15.89:0/3561057854 >> 172.21.15.89:0/3561057854 conn(0x7f12b00a4210 msgr2=0x7f12b00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.074+0000 7f12b51f8700 1 -- 172.21.15.89:0/3561057854 shutdown_connections 2023-09-30T22:22:02.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.074+0000 7f12b51f8700 1 -- 172.21.15.89:0/3561057854 wait complete. 2023-09-30T22:22:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.075+0000 7f12b51f8700 1 Processor -- start 2023-09-30T22:22:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.075+0000 7f12b51f8700 1 -- start start 2023-09-30T22:22:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.075+0000 7f12b51f8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12b009e930 0x7f12b01a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.075+0000 7f12b51f8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f240 0x7f12b01a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.075+0000 7f12b51f8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009f650 0x7f12b01a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.075+0000 7f12b51f8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01a6000 con 0x7f12b009e930 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.075+0000 7f12b51f8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01a3730 con 0x7f12b009f240 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.075+0000 7f12b51f8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01a3d50 con 0x7f12b009f650 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009f650 0x7f12b01a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009f650 0x7f12b01a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60892/0 (socket says 172.21.15.89:60892) 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 -- 172.21.15.89:0/1225720041 learned_addr learned my addr 172.21.15.89:0/1225720041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12aed9d700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12b009e930 0x7f12b01a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12ae59c700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f240 0x7f12b01a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 -- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f240 msgr2=0x7f12b01a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f240 0x7f12b01a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 -- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12b009e930 msgr2=0x7f12b01a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12b009e930 0x7f12b01a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 -- 172.21.15.89:0/1225720041 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a0009550 con 0x7f12b009f650 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12af59e700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009f650 0x7f12b01a25a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f12a8012450 tx=0x7f12a8012760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12977fe700 1 -- 172.21.15.89:0/1225720041 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800d2d0 con 0x7f12b009f650 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.076+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b00a0190 con 0x7f12b009f650 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.077+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b01a4dd0 con 0x7f12b009f650 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.077+0000 7f12977fe700 1 -- 172.21.15.89:0/1225720041 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a8013040 con 0x7f12b009f650 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.077+0000 7f12977fe700 1 -- 172.21.15.89:0/1225720041 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800b740 con 0x7f12b009f650 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.078+0000 7f12977fe700 1 -- 172.21.15.89:0/1225720041 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12a800b8a0 con 0x7f12b009f650 2023-09-30T22:22:02.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.078+0000 7f12977fe700 1 -- 172.21.15.89:0/1225720041 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f12a8044150 con 0x7f12b009f650 2023-09-30T22:22:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.078+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12900052e0 con 0x7f12b009f650 2023-09-30T22:22:02.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.088+0000 7f12977fe700 1 -- 172.21.15.89:0/1225720041 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12a8016340 con 0x7f12b009f650 2023-09-30T22:22:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.101+0000 7f74d299a700 1 Processor -- start 2023-09-30T22:22:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.101+0000 7f74d299a700 1 -- start start 2023-09-30T22:22:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74d299a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc09e1d0 0x7f74cc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74d299a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc139170 0x7f74cc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74d299a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc09b120 con 0x7f74cc09d6f0 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74d299a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc06b460 con 0x7f74cc139170 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74d299a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc09a7e0 con 0x7f74cc09e1d0 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74cb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc09e1d0 0x7f74cc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74cbfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f74cc09d6f0 0x7f74cc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36934/0 (socket says 172.21.15.89:36934) 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74cbfff700 1 -- 172.21.15.89:0/4225153740 learned_addr learned my addr 172.21.15.89:0/4225153740 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74cb7fe700 1 -- 172.21.15.89:0/4225153740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc139170 msgr2=0x7f74cc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74cb7fe700 1 --2- 172.21.15.89:0/4225153740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc139170 0x7f74cc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74cb7fe700 1 -- 172.21.15.89:0/4225153740 >> v1:172.21.15.89:6789/0 conn(0x7f74cc09d6f0 legacy=0x7f74cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74cb7fe700 1 -- 172.21.15.89:0/4225153740 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc09a500 con 0x7f74cc09e1d0 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.102+0000 7f74cb7fe700 1 --2- 172.21.15.89:0/4225153740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc09e1d0 0x7f74cc138c30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f74c00098a0 tx=0x7f74c0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe840c1e2462f386 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:02.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74caffd700 1 -- 172.21.15.89:0/4225153740 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0011830 con 0x7f74cc09e1d0 2023-09-30T22:22:02.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74caffd700 1 -- 172.21.15.89:0/4225153740 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0011990 con 0x7f74cc09e1d0 2023-09-30T22:22:02.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74caffd700 1 -- 172.21.15.89:0/4225153740 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0011d40 con 0x7f74cc09e1d0 2023-09-30T22:22:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74d299a700 1 -- 172.21.15.89:0/4225153740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc09e1d0 msgr2=0x7f74cc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74d299a700 1 --2- 172.21.15.89:0/4225153740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc09e1d0 0x7f74cc138c30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f74c00098a0 tx=0x7f74c0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74d299a700 1 -- 172.21.15.89:0/4225153740 shutdown_connections 2023-09-30T22:22:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74d299a700 1 --2- 172.21.15.89:0/4225153740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc139170 0x7f74cc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74d299a700 1 --2- 172.21.15.89:0/4225153740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc09e1d0 0x7f74cc138c30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74d299a700 1 -- 172.21.15.89:0/4225153740 >> 172.21.15.89:0/4225153740 conn(0x7f74cc066f50 msgr2=0x7f74cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74d299a700 1 -- 172.21.15.89:0/4225153740 shutdown_connections 2023-09-30T22:22:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.103+0000 7f74d299a700 1 -- 172.21.15.89:0/4225153740 wait complete. 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.104+0000 7f74d299a700 1 Processor -- start 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.104+0000 7f74d299a700 1 -- start start 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d299a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc09d6f0 0x7f74cc11e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d299a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc09e1d0 0x7f74cc11ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d299a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc139170 0x7f74cc11f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d299a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc120660 con 0x7f74cc09d6f0 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d299a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc09c710 con 0x7f74cc09e1d0 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d299a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc120d40 con 0x7f74cc139170 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d0f37700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc139170 0x7f74cc11f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d0f37700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc139170 0x7f74cc11f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60910/0 (socket says 172.21.15.89:60910) 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d0f37700 1 -- 172.21.15.89:0/3703531199 learned_addr learned my addr 172.21.15.89:0/3703531199 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74cb7fe700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc09e1d0 0x7f74cc11ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74cbfff700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc09d6f0 0x7f74cc11e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74cb7fe700 1 -- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc139170 msgr2=0x7f74cc11f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74cb7fe700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc139170 0x7f74cc11f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74cb7fe700 1 -- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc09d6f0 msgr2=0x7f74cc11e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74cb7fe700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc09d6f0 0x7f74cc11e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74cb7fe700 1 -- 172.21.15.89:0/3703531199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00094a0 con 0x7f74cc09e1d0 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74cbfff700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc09d6f0 0x7f74cc11e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.105+0000 7f74d0f37700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc139170 0x7f74cc11f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.106+0000 7f74cb7fe700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc09e1d0 0x7f74cc11ee50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f74c0016000 tx=0x7f74c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.106+0000 7f74c8ff9700 1 -- 172.21.15.89:0/3703531199 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0004510 con 0x7f74cc09e1d0 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.106+0000 7f74c8ff9700 1 -- 172.21.15.89:0/3703531199 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c001f070 con 0x7f74cc09e1d0 2023-09-30T22:22:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.106+0000 7f74c8ff9700 1 -- 172.21.15.89:0/3703531199 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00038e0 con 0x7f74cc09e1d0 2023-09-30T22:22:02.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.106+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc1347c0 con 0x7f74cc09e1d0 2023-09-30T22:22:02.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.106+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc122b80 con 0x7f74cc09e1d0 2023-09-30T22:22:02.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.107+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc072130 con 0x7f74cc09e1d0 2023-09-30T22:22:02.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.107+0000 7f74c8ff9700 1 -- 172.21.15.89:0/3703531199 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74c0003a40 con 0x7f74cc09e1d0 2023-09-30T22:22:02.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.107+0000 7f74c8ff9700 1 -- 172.21.15.89:0/3703531199 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f74c0021070 con 0x7f74cc09e1d0 2023-09-30T22:22:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.116+0000 7f74c8ff9700 1 -- 172.21.15.89:0/3703531199 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74c0006c60 con 0x7f74cc09e1d0 2023-09-30T22:22:02.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.144+0000 7fd1d7f18700 1 Processor -- start 2023-09-30T22:22:02.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.145+0000 7fd1d7f18700 1 -- start start 2023-09-30T22:22:02.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.145+0000 7fd1d7f18700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d0130540 0x7fd1d0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.145+0000 7fd1d7f18700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d006fcf0 0x7fd1d0138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.145+0000 7fd1d7f18700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d00716a0 con 0x7fd1d0130e50 2023-09-30T22:22:02.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.145+0000 7fd1d7f18700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d009a390 con 0x7fd1d006fcf0 2023-09-30T22:22:02.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.145+0000 7fd1d7f18700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0071290 con 0x7fd1d0130540 2023-09-30T22:22:02.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.145+0000 7fd1d5cb4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d0130540 0x7fd1d0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d5cb4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d0130540 0x7fd1d0130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60920/0 (socket says 172.21.15.89:60920) 2023-09-30T22:22:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d64b5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d006fcf0 0x7fd1d0138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d5cb4700 1 -- 172.21.15.89:0/2539136517 learned_addr learned my addr 172.21.15.89:0/2539136517 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d54b3700 1 --1- 172.21.15.89:0/2539136517 >> v1:172.21.15.89:6789/0 conn(0x7fd1d0130e50 0x7fd1d006f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36948/0 (socket says 172.21.15.89:36948) 2023-09-30T22:22:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d4cb2700 1 -- 172.21.15.89:0/2539136517 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796537291 0 0) 0x7fd1d00716a0 con 0x7fd1d0130e50 2023-09-30T22:22:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d5cb4700 1 -- 172.21.15.89:0/2539136517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d006fcf0 msgr2=0x7fd1d0138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d5cb4700 1 --2- 172.21.15.89:0/2539136517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d006fcf0 0x7fd1d0138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d5cb4700 1 -- 172.21.15.89:0/2539136517 >> v1:172.21.15.89:6789/0 conn(0x7fd1d0130e50 legacy=0x7fd1d006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d5cb4700 1 -- 172.21.15.89:0/2539136517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0130340 con 0x7fd1d0130540 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d5cb4700 1 --2- 172.21.15.89:0/2539136517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d0130540 0x7fd1d0130910 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009920 tx=0x7fd1cc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d50fc2caa75f68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d4cb2700 1 -- 172.21.15.89:0/2539136517 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc011830 con 0x7fd1d0130540 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d4cb2700 1 -- 172.21.15.89:0/2539136517 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc011990 con 0x7fd1d0130540 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.146+0000 7fd1d4cb2700 1 -- 172.21.15.89:0/2539136517 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc011d20 con 0x7fd1d0130540 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.147+0000 7fd1d7f18700 1 -- 172.21.15.89:0/2539136517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d0130540 msgr2=0x7fd1d0130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.147+0000 7fd1d7f18700 1 --2- 172.21.15.89:0/2539136517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d0130540 0x7fd1d0130910 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009920 tx=0x7fd1cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.147+0000 7fd1d7f18700 1 -- 172.21.15.89:0/2539136517 shutdown_connections 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.147+0000 7fd1d7f18700 1 --2- 172.21.15.89:0/2539136517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d006fcf0 0x7fd1d0138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.147+0000 7fd1d7f18700 1 --2- 172.21.15.89:0/2539136517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d0130540 0x7fd1d0130910 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.147+0000 7fd1d7f18700 1 -- 172.21.15.89:0/2539136517 >> 172.21.15.89:0/2539136517 conn(0x7fd1d00a2aa0 msgr2=0x7fd1d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.147+0000 7fd1d7f18700 1 -- 172.21.15.89:0/2539136517 shutdown_connections 2023-09-30T22:22:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.147+0000 7fd1d7f18700 1 -- 172.21.15.89:0/2539136517 wait complete. 2023-09-30T22:22:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d7f18700 1 Processor -- start 2023-09-30T22:22:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d7f18700 1 -- start start 2023-09-30T22:22:02.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d7f18700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d006fcf0 0x7fd1d011a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d7f18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d0130540 0x7fd1d011ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d7f18700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d0130e50 0x7fd1d011b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d7f18700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d011e120 con 0x7fd1d0130540 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d7f18700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d011e320 con 0x7fd1d0130e50 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d7f18700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d011d210 con 0x7fd1d006fcf0 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d5cb4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d006fcf0 0x7fd1d011a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d5cb4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d006fcf0 0x7fd1d011a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60930/0 (socket says 172.21.15.89:60930) 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d5cb4700 1 -- 172.21.15.89:0/1483652715 learned_addr learned my addr 172.21.15.89:0/1483652715 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.148+0000 7fd1d64b5700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d0130e50 0x7fd1d011b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d54b3700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d0130540 0x7fd1d011ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d5cb4700 1 -- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d0130e50 msgr2=0x7fd1d011b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d5cb4700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d0130e50 0x7fd1d011b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d5cb4700 1 -- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d0130540 msgr2=0x7fd1d011ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d5cb4700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d0130540 0x7fd1d011ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d5cb4700 1 -- 172.21.15.89:0/1483652715 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc009550 con 0x7fd1d006fcf0 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d5cb4700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d006fcf0 0x7fd1d011a800 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0071420 tx=0x7fd1cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1c67fc700 1 -- 172.21.15.89:0/1483652715 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc004450 con 0x7fd1d006fcf0 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1c67fc700 1 -- 172.21.15.89:0/1483652715 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc0045b0 con 0x7fd1d006fcf0 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d012cca0 con 0x7fd1d006fcf0 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1c67fc700 1 -- 172.21.15.89:0/1483652715 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc01fa40 con 0x7fd1d006fcf0 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.149+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d00a09c0 con 0x7fd1d006fcf0 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.150+0000 7fd1c67fc700 1 -- 172.21.15.89:0/1483652715 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1cc01fba0 con 0x7fd1d006fcf0 2023-09-30T22:22:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.150+0000 7fd1c67fc700 1 -- 172.21.15.89:0/1483652715 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd1cc025030 con 0x7fd1d006fcf0 2023-09-30T22:22:02.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.150+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d004ea50 con 0x7fd1d006fcf0 2023-09-30T22:22:02.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.155+0000 7fd1c67fc700 1 -- 172.21.15.89:0/1483652715 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1cc015080 con 0x7fd1d006fcf0 2023-09-30T22:22:02.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.262+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feba806c360 con 0x7feba809d6f0 2023-09-30T22:22:02.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.262+0000 7feba57fa700 1 -- 172.21.15.89:0/3983675420 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb9c015020 con 0x7feba809d6f0 2023-09-30T22:22:02.263 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:02.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.264+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 msgr2=0x7feba81c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.264+0000 7febaecc9700 1 --2- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba81c8040 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0090b0 tx=0x7feb9c004750 comp rx=0 tx=0).stop 2023-09-30T22:22:02.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.265+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 shutdown_connections 2023-09-30T22:22:02.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.265+0000 7febaecc9700 1 --2- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feba809e410 0x7feba81c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.265+0000 7febaecc9700 1 --2- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feba809d6f0 0x7feba81c8040 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.265+0000 7febaecc9700 1 --2- 172.21.15.89:0/3983675420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feba809e000 0x7feba81c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.265+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 >> 172.21.15.89:0/3983675420 conn(0x7feba8066f50 msgr2=0x7feba806f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:02.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.265+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 shutdown_connections 2023-09-30T22:22:02.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.265+0000 7febaecc9700 1 -- 172.21.15.89:0/3983675420 wait complete. 2023-09-30T22:22:02.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:02.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.349+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd1d0074ea0 con 0x7fd1d006fcf0 2023-09-30T22:22:02.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.349+0000 7fd1c67fc700 1 -- 172.21.15.89:0/1483652715 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1cc0036a0 con 0x7fd1d006fcf0 2023-09-30T22:22:02.350 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:02.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d006fcf0 msgr2=0x7fd1d011a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d006fcf0 0x7fd1d011a800 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0071420 tx=0x7fd1cc004750 comp rx=0 tx=0).stop 2023-09-30T22:22:02.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 shutdown_connections 2023-09-30T22:22:02.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd1d0130e50 0x7fd1d011b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd1d006fcf0 0x7fd1d011a800 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 --2- 172.21.15.89:0/1483652715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d0130540 0x7fd1d011ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 >> 172.21.15.89:0/1483652715 conn(0x7fd1d00a2aa0 msgr2=0x7fd1d012d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:02.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 shutdown_connections 2023-09-30T22:22:02.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.351+0000 7fd1d7f18700 1 -- 172.21.15.89:0/1483652715 wait complete. 2023-09-30T22:22:02.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:02.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.381+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1290005590 con 0x7f12b009f650 2023-09-30T22:22:02.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.381+0000 7f12977fe700 1 -- 172.21.15.89:0/1225720041 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12a8021b60 con 0x7f12b009f650 2023-09-30T22:22:02.382 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:02.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.383+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009f650 msgr2=0x7f12b01a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.383+0000 7f12b51f8700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009f650 0x7f12b01a25a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f12a8012450 tx=0x7f12a8012760 comp rx=0 tx=0).stop 2023-09-30T22:22:02.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.383+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 shutdown_connections 2023-09-30T22:22:02.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.383+0000 7f12b51f8700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f12b009f650 0x7f12b01a25a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.383+0000 7f12b51f8700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12b009e930 0x7f12b01a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.383+0000 7f12b51f8700 1 --2- 172.21.15.89:0/1225720041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f12b009f240 0x7f12b01a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.383+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 >> 172.21.15.89:0/1225720041 conn(0x7f12b00a4210 msgr2=0x7f12b01b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:02.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.384+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 shutdown_connections 2023-09-30T22:22:02.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.384+0000 7f12b51f8700 1 -- 172.21.15.89:0/1225720041 wait complete. 2023-09-30T22:22:02.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:02.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.397+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f74cc06c360 con 0x7f74cc09e1d0 2023-09-30T22:22:02.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.398+0000 7f74c8ff9700 1 -- 172.21.15.89:0/3703531199 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74c002b070 con 0x7f74cc09e1d0 2023-09-30T22:22:02.398 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:02.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.399+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc09e1d0 msgr2=0x7f74cc11ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:02.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.399+0000 7f74d299a700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc09e1d0 0x7f74cc11ee50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f74c0016000 tx=0x7f74c0004750 comp rx=0 tx=0).stop 2023-09-30T22:22:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.399+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 shutdown_connections 2023-09-30T22:22:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.399+0000 7f74d299a700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74cc139170 0x7f74cc11f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.399+0000 7f74d299a700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74cc09d6f0 0x7f74cc11e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.399+0000 7f74d299a700 1 --2- 172.21.15.89:0/3703531199 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74cc09e1d0 0x7f74cc11ee50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.399+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 >> 172.21.15.89:0/3703531199 conn(0x7f74cc066f50 msgr2=0x7f74cc136cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.400+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 shutdown_connections 2023-09-30T22:22:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:02.400+0000 7f74d299a700 1 -- 172.21.15.89:0/3703531199 wait complete. 2023-09-30T22:22:02.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:03.278 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:03.364 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:03.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:03.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.410+0000 7fd7b6d86700 1 Processor -- start 2023-09-30T22:22:03.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.410+0000 7fd7b6d86700 1 -- start start 2023-09-30T22:22:03.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b6d86700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0136710 0x7fd7b0138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b6d86700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b0139030 0x7fd7b013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b6d86700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b006ef30 con 0x7fd7b01338a0 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b6d86700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b006fef0 con 0x7fd7b0136710 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b6d86700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b01334b0 con 0x7fd7b0139030 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b4b22700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd7b01338a0 0x7fd7b0133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36960/0 (socket says 172.21.15.89:36960) 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b4b22700 1 -- 172.21.15.89:0/2033570133 learned_addr learned my addr 172.21.15.89:0/2033570133 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b5323700 1 --2- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b0139030 0x7fd7b013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7affff700 1 --2- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0136710 0x7fd7b0138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7af7fe700 1 -- 172.21.15.89:0/2033570133 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702491586 0 0) 0x7fd7b006ef30 con 0x7fd7b01338a0 2023-09-30T22:22:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7af7fe700 1 -- 172.21.15.89:0/2033570133 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a00035b0 con 0x7fd7b01338a0 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b5323700 1 -- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0136710 msgr2=0x7fd7b0138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b5323700 1 --2- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0136710 0x7fd7b0138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b5323700 1 -- 172.21.15.89:0/2033570133 >> v1:172.21.15.89:6789/0 conn(0x7fd7b01338a0 legacy=0x7fd7b0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.411+0000 7fd7b5323700 1 -- 172.21.15.89:0/2033570133 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b006fc30 con 0x7fd7b0139030 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.412+0000 7fd7affff700 1 --2- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0136710 0x7fd7b0138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.412+0000 7fd7b5323700 1 --2- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b0139030 0x7fd7b013b410 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009dc0 tx=0x7fd7a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ecfeff11019046 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.412+0000 7fd7af7fe700 1 -- 172.21.15.89:0/2033570133 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800e070 con 0x7fd7b0139030 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.412+0000 7fd7af7fe700 1 -- 172.21.15.89:0/2033570133 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a800bb20 con 0x7fd7b0139030 2023-09-30T22:22:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.412+0000 7fd7af7fe700 1 -- 172.21.15.89:0/2033570133 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8012430 con 0x7fd7b0139030 2023-09-30T22:22:03.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.413+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b0139030 msgr2=0x7fd7b013b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.413+0000 7fd7b6d86700 1 --2- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b0139030 0x7fd7b013b410 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009dc0 tx=0x7fd7a800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.413+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2033570133 shutdown_connections 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.413+0000 7fd7b6d86700 1 --2- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b0139030 0x7fd7b013b410 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.413+0000 7fd7b6d86700 1 --2- 172.21.15.89:0/2033570133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0136710 0x7fd7b0138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.413+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2033570133 >> 172.21.15.89:0/2033570133 conn(0x7fd7b00a2aa0 msgr2=0x7fd7b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.413+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2033570133 shutdown_connections 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.413+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2033570133 wait complete. 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.414+0000 7fd7b6d86700 1 Processor -- start 2023-09-30T22:22:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.414+0000 7fd7b6d86700 1 -- start start 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b6d86700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b01338a0 0x7fd7b011ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b6d86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b0136710 0x7fd7b011b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b6d86700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0139030 0x7fd7b011b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b6d86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b011c810 con 0x7fd7b0136710 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b6d86700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b0130af0 con 0x7fd7b0139030 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b6d86700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b011e440 con 0x7fd7b01338a0 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b4b22700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b01338a0 0x7fd7b011ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7affff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b0136710 0x7fd7b011b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b5323700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0139030 0x7fd7b011b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b4b22700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b01338a0 0x7fd7b011ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60956/0 (socket says 172.21.15.89:60956) 2023-09-30T22:22:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b4b22700 1 -- 172.21.15.89:0/2138052660 learned_addr learned my addr 172.21.15.89:0/2138052660 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b4b22700 1 -- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0139030 msgr2=0x7fd7b011b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b4b22700 1 --2- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0139030 0x7fd7b011b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b4b22700 1 -- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b0136710 msgr2=0x7fd7b011b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b4b22700 1 --2- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b0136710 0x7fd7b011b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7b4b22700 1 -- 172.21.15.89:0/2138052660 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c0094a0 con 0x7fd7b01338a0 2023-09-30T22:22:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.415+0000 7fd7affff700 1 --2- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b0136710 0x7fd7b011b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.416+0000 7fd7b4b22700 1 --2- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b01338a0 0x7fd7b011ac00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c700 tx=0x7fd7a400ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.416+0000 7fd7ad7fa700 1 -- 172.21.15.89:0/2138052660 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4005d70 con 0x7fd7b01338a0 2023-09-30T22:22:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.416+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a8009a70 con 0x7fd7b01338a0 2023-09-30T22:22:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.416+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b011d030 con 0x7fd7b01338a0 2023-09-30T22:22:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.416+0000 7fd7ad7fa700 1 -- 172.21.15.89:0/2138052660 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a40070d0 con 0x7fd7b01338a0 2023-09-30T22:22:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.416+0000 7fd7ad7fa700 1 -- 172.21.15.89:0/2138052660 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4007440 con 0x7fd7b01338a0 2023-09-30T22:22:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.417+0000 7fd7ad7fa700 1 -- 172.21.15.89:0/2138052660 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd7a40075a0 con 0x7fd7b01338a0 2023-09-30T22:22:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.417+0000 7fd7ad7fa700 1 -- 172.21.15.89:0/2138052660 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd7a403ea30 con 0x7fd7b01338a0 2023-09-30T22:22:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.417+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7900052e0 con 0x7fd7b01338a0 2023-09-30T22:22:03.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.426+0000 7fd7ad7fa700 1 -- 172.21.15.89:0/2138052660 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd7a4017080 con 0x7fd7b01338a0 2023-09-30T22:22:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.509+0000 7f07ba854700 1 Processor -- start 2023-09-30T22:22:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.509+0000 7f07ba854700 1 -- start start 2023-09-30T22:22:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07ba854700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07b40a0a30 0x7f07b40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07ba854700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b4134170 0x7f07b413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07ba854700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b409edd0 con 0x7f07b40a1340 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07ba854700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b409e1f0 con 0x7f07b40a0a30 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07ba854700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07b409dc50 con 0x7f07b4134170 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07b40a0a30 0x7f07b40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07b40a0a30 0x7f07b40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48044/0 (socket says 172.21.15.89:48044) 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b3fff700 1 -- 172.21.15.89:0/3870289830 learned_addr learned my addr 172.21.15.89:0/3870289830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b8df1700 1 --2- 172.21.15.89:0/3870289830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b4134170 0x7f07b413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b8df1700 1 -- 172.21.15.89:0/3870289830 >> v2:172.21.15.89:3300/0 conn(0x7f07b40a0a30 msgr2=0x7f07b40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b8df1700 1 --2- 172.21.15.89:0/3870289830 >> v2:172.21.15.89:3300/0 conn(0x7f07b40a0a30 0x7f07b40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b8df1700 1 -- 172.21.15.89:0/3870289830 >> v1:172.21.15.89:6789/0 conn(0x7f07b40a1340 legacy=0x7f07b4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b8df1700 1 -- 172.21.15.89:0/3870289830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b40a1fe0 con 0x7f07b4134170 2023-09-30T22:22:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b3fff700 1 --2- 172.21.15.89:0/3870289830 >> v2:172.21.15.89:3300/0 conn(0x7f07b40a0a30 0x7f07b40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.510+0000 7f07b8df1700 1 --2- 172.21.15.89:0/3870289830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b4134170 0x7f07b413bd30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009d90 tx=0x7f07ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b91b7758917178 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.511+0000 7f07b2ffd700 1 -- 172.21.15.89:0/3870289830 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00e070 con 0x7f07b4134170 2023-09-30T22:22:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.511+0000 7f07b2ffd700 1 -- 172.21.15.89:0/3870289830 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac00bb20 con 0x7f07b4134170 2023-09-30T22:22:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.511+0000 7f07b2ffd700 1 -- 172.21.15.89:0/3870289830 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac012430 con 0x7f07b4134170 2023-09-30T22:22:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.512+0000 7f07ba854700 1 -- 172.21.15.89:0/3870289830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b4134170 msgr2=0x7f07b413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.512+0000 7f07ba854700 1 --2- 172.21.15.89:0/3870289830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b4134170 0x7f07b413bd30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009d90 tx=0x7f07ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.512+0000 7f07ba854700 1 -- 172.21.15.89:0/3870289830 shutdown_connections 2023-09-30T22:22:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.512+0000 7f07ba854700 1 --2- 172.21.15.89:0/3870289830 >> v2:172.21.15.89:3300/0 conn(0x7f07b40a0a30 0x7f07b40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.512+0000 7f07ba854700 1 --2- 172.21.15.89:0/3870289830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b4134170 0x7f07b413bd30 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.512+0000 7f07ba854700 1 -- 172.21.15.89:0/3870289830 >> 172.21.15.89:0/3870289830 conn(0x7f07b40a6550 msgr2=0x7f07b40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.512+0000 7f07ba854700 1 -- 172.21.15.89:0/3870289830 shutdown_connections 2023-09-30T22:22:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.512+0000 7f07ba854700 1 -- 172.21.15.89:0/3870289830 wait complete. 2023-09-30T22:22:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.513+0000 7f07ba854700 1 Processor -- start 2023-09-30T22:22:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.513+0000 7f07ba854700 1 -- start start 2023-09-30T22:22:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07ba854700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07b40a0a30 0x7f07b411e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07ba854700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b40a1340 0x7f07b411ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07ba854700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07b4134170 0x7f07b411f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07ba854700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b411fbe0 con 0x7f07b4134170 2023-09-30T22:22:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07ba854700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b41221a0 con 0x7f07b40a0a30 2023-09-30T22:22:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07ba854700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07b4120290 con 0x7f07b40a1340 2023-09-30T22:22:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07b37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b40a1340 0x7f07b411ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07b37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b40a1340 0x7f07b411ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60966/0 (socket says 172.21.15.89:60966) 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07b37fe700 1 -- 172.21.15.89:0/1923340521 learned_addr learned my addr 172.21.15.89:0/1923340521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.514+0000 7f07b8df1700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07b4134170 0x7f07b411f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b3fff700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07b40a0a30 0x7f07b411e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b37fe700 1 -- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07b40a0a30 msgr2=0x7f07b411e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b37fe700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07b40a0a30 0x7f07b411e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b37fe700 1 -- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07b4134170 msgr2=0x7f07b411f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b37fe700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07b4134170 0x7f07b411f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b37fe700 1 -- 172.21.15.89:0/1923340521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a4009550 con 0x7f07b40a1340 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b3fff700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07b40a0a30 0x7f07b411e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b8df1700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07b4134170 0x7f07b411f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b37fe700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b40a1340 0x7f07b411ec50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c510 tx=0x7f07a800c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b0ff9700 1 -- 172.21.15.89:0/1923340521 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80052d0 con 0x7f07b40a1340 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b0ff9700 1 -- 172.21.15.89:0/1923340521 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a8005430 con 0x7f07b40a1340 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac009a70 con 0x7f07b40a1340 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07b0ff9700 1 -- 172.21.15.89:0/1923340521 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80038c0 con 0x7f07b40a1340 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.515+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b40a25e0 con 0x7f07b40a1340 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.516+0000 7f07b0ff9700 1 -- 172.21.15.89:0/1923340521 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f07a8003a20 con 0x7f07b40a1340 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.516+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b4074ea0 con 0x7f07b40a1340 2023-09-30T22:22:03.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.516+0000 7f07b0ff9700 1 -- 172.21.15.89:0/1923340521 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f07a803eb60 con 0x7f07b40a1340 2023-09-30T22:22:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.521+0000 7f07b0ff9700 1 -- 172.21.15.89:0/1923340521 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07a8018260 con 0x7f07b40a1340 2023-09-30T22:22:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.537+0000 7f26607eb700 1 Processor -- start 2023-09-30T22:22:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.537+0000 7f26607eb700 1 -- start start 2023-09-30T22:22:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f26607eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f265809d6f0 0x7f265809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f26607eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f26607eb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265809f130 con 0x7f265809e000 2023-09-30T22:22:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f26607eb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265809f520 con 0x7f265809d6f0 2023-09-30T22:22:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f26607eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f265809e930 con 0x7f265809e410 2023-09-30T22:22:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265ed88700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265ed88700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60970/0 (socket says 172.21.15.89:60970) 2023-09-30T22:22:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265ed88700 1 -- 172.21.15.89:0/3823001986 learned_addr learned my addr 172.21.15.89:0/3823001986 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265ed88700 1 -- 172.21.15.89:0/3823001986 >> v2:172.21.15.89:3300/0 conn(0x7f265809d6f0 msgr2=0x7f265809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265ed88700 1 --2- 172.21.15.89:0/3823001986 >> v2:172.21.15.89:3300/0 conn(0x7f265809d6f0 0x7f265809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265ed88700 1 -- 172.21.15.89:0/3823001986 >> v1:172.21.15.89:6789/0 conn(0x7f265809e000 legacy=0x7f2658130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265ed88700 1 -- 172.21.15.89:0/3823001986 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265809b2d0 con 0x7f265809e410 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265ed88700 1 --2- 172.21.15.89:0/3823001986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658130d70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f265000d150 tx=0x7f265000d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f946a015340974 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265d585700 1 -- 172.21.15.89:0/3823001986 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650010070 con 0x7f265809e410 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265d585700 1 -- 172.21.15.89:0/3823001986 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265000dd80 con 0x7f265809e410 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.538+0000 7f265d585700 1 -- 172.21.15.89:0/3823001986 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265000b310 con 0x7f265809e410 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.539+0000 7f26607eb700 1 -- 172.21.15.89:0/3823001986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 msgr2=0x7f2658130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.539+0000 7f26607eb700 1 --2- 172.21.15.89:0/3823001986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658130d70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f265000d150 tx=0x7f265000d460 comp rx=0 tx=0).stop 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.539+0000 7f26607eb700 1 -- 172.21.15.89:0/3823001986 shutdown_connections 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.539+0000 7f26607eb700 1 --2- 172.21.15.89:0/3823001986 >> v2:172.21.15.89:3300/0 conn(0x7f265809d6f0 0x7f265809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.539+0000 7f26607eb700 1 --2- 172.21.15.89:0/3823001986 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658130d70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.539+0000 7f26607eb700 1 -- 172.21.15.89:0/3823001986 >> 172.21.15.89:0/3823001986 conn(0x7f2658066f50 msgr2=0x7f265809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.539+0000 7f26607eb700 1 -- 172.21.15.89:0/3823001986 shutdown_connections 2023-09-30T22:22:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.539+0000 7f26607eb700 1 -- 172.21.15.89:0/3823001986 wait complete. 2023-09-30T22:22:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.540+0000 7f26607eb700 1 Processor -- start 2023-09-30T22:22:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.540+0000 7f26607eb700 1 -- start start 2023-09-30T22:22:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f26607eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f265809d6f0 0x7f2658122420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f26607eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f265809e000 0x7f2658122960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f26607eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658122ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f26607eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2658125c40 con 0x7f265809e000 2023-09-30T22:22:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f26607eb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2658124120 con 0x7f265809d6f0 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f26607eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26581237f0 con 0x7f265809e410 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265dd86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f265809e000 0x7f2658122960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265dd86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f265809e000 0x7f2658122960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48062/0 (socket says 172.21.15.89:48062) 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265dd86700 1 -- 172.21.15.89:0/2350516743 learned_addr learned my addr 172.21.15.89:0/2350516743 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265e587700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f265809d6f0 0x7f2658122420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265ed88700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658122ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265ed88700 1 -- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f265809d6f0 msgr2=0x7f2658122420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265ed88700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f265809d6f0 0x7f2658122420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265ed88700 1 -- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f265809e000 msgr2=0x7f2658122960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265ed88700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f265809e000 0x7f2658122960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265ed88700 1 -- 172.21.15.89:0/2350516743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265000bd40 con 0x7f265809e410 2023-09-30T22:22:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265dd86700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f265809e000 0x7f2658122960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.541+0000 7f265ed88700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658122ea0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2650016000 tx=0x7f2650004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.542+0000 7f264effd700 1 -- 172.21.15.89:0/2350516743 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650009a10 con 0x7f265809e410 2023-09-30T22:22:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.542+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26580a1e20 con 0x7f265809e410 2023-09-30T22:22:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.542+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26581266b0 con 0x7f265809e410 2023-09-30T22:22:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.542+0000 7f264effd700 1 -- 172.21.15.89:0/2350516743 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2650009bb0 con 0x7f265809e410 2023-09-30T22:22:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.542+0000 7f264effd700 1 -- 172.21.15.89:0/2350516743 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265000b4e0 con 0x7f265809e410 2023-09-30T22:22:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.543+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2658074ea0 con 0x7f265809e410 2023-09-30T22:22:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.543+0000 7f264effd700 1 -- 172.21.15.89:0/2350516743 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2650009050 con 0x7f265809e410 2023-09-30T22:22:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.543+0000 7f264effd700 1 -- 172.21.15.89:0/2350516743 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2650040ad0 con 0x7f265809e410 2023-09-30T22:22:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.545+0000 7f6a2e870700 1 Processor -- start 2023-09-30T22:22:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.545+0000 7f6a2e870700 1 -- start start 2023-09-30T22:22:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2e870700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 0x7f6a2809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2e870700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6a2809f650 0x7f6a28131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2e870700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a28070940 con 0x7f6a2809f240 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2e870700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2806fd60 con 0x7f6a2809f650 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2e870700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2806fa80 con 0x7f6a2809e930 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2ce0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6a2809f650 0x7f6a28131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2ce0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6a2809f650 0x7f6a28131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48066/0 (socket says 172.21.15.89:48066) 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2ce0d700 1 -- 172.21.15.89:0/48979058 learned_addr learned my addr 172.21.15.89:0/48979058 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a27fff700 1 --2- 172.21.15.89:0/48979058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 0x7f6a2809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2ce0d700 1 -- 172.21.15.89:0/48979058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 msgr2=0x7f6a2809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2ce0d700 1 --2- 172.21.15.89:0/48979058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 0x7f6a2809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2ce0d700 1 -- 172.21.15.89:0/48979058 >> v1:172.21.15.89:6789/0 conn(0x7f6a2809f240 legacy=0x7f6a281317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2ce0d700 1 -- 172.21.15.89:0/48979058 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2809fb60 con 0x7f6a2809f650 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a2ce0d700 1 --2- 172.21.15.89:0/48979058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 0x7f6a28131f00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009e50 tx=0x7f6a2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d014f438df32240 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a26ffd700 1 -- 172.21.15.89:0/48979058 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000e070 con 0x7f6a2809f650 2023-09-30T22:22:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a26ffd700 1 -- 172.21.15.89:0/48979058 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2000b9c0 con 0x7f6a2809f650 2023-09-30T22:22:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.546+0000 7f6a26ffd700 1 -- 172.21.15.89:0/48979058 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000bd70 con 0x7f6a2809f650 2023-09-30T22:22:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.547+0000 7f6a2e870700 1 -- 172.21.15.89:0/48979058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 msgr2=0x7f6a28131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.547+0000 7f6a2e870700 1 --2- 172.21.15.89:0/48979058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 0x7f6a28131f00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009e50 tx=0x7f6a2000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.547+0000 7f6a2e870700 1 -- 172.21.15.89:0/48979058 shutdown_connections 2023-09-30T22:22:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.547+0000 7f6a2e870700 1 --2- 172.21.15.89:0/48979058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 0x7f6a28131f00 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.547+0000 7f6a2e870700 1 --2- 172.21.15.89:0/48979058 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 0x7f6a2809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.547+0000 7f6a2e870700 1 -- 172.21.15.89:0/48979058 >> 172.21.15.89:0/48979058 conn(0x7f6a280a4210 msgr2=0x7f6a280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.547+0000 7f6a2e870700 1 -- 172.21.15.89:0/48979058 shutdown_connections 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.547+0000 7f6a2e870700 1 -- 172.21.15.89:0/48979058 wait complete. 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.548+0000 7f6a2e870700 1 Processor -- start 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.548+0000 7f6a2e870700 1 -- start start 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a2e870700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 0x7f6a2806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a2e870700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a2809f240 0x7f6a2806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a2e870700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 0x7f6a2806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a2e870700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a280a3540 con 0x7f6a2809f650 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a2e870700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28071d20 con 0x7f6a2809f240 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a2e870700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a281a8b00 con 0x7f6a2809e930 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a277fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a2809f240 0x7f6a2806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a277fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a2809f240 0x7f6a2806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35438/0 (socket says 172.21.15.89:35438) 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a277fe700 1 -- 172.21.15.89:0/2363201218 learned_addr learned my addr 172.21.15.89:0/2363201218 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a2ce0d700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 0x7f6a2806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a27fff700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 0x7f6a2806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a277fe700 1 -- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 msgr2=0x7f6a2806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a277fe700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 0x7f6a2806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a277fe700 1 -- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 msgr2=0x7f6a2806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a277fe700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 0x7f6a2806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a277fe700 1 -- 172.21.15.89:0/2363201218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a20009b30 con 0x7f6a2809f240 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.549+0000 7f6a2ce0d700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 0x7f6a2806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.550+0000 7f6a277fe700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a2809f240 0x7f6a2806bdf0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00c4b0 tx=0x7f6a1c00c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.550+0000 7f6a24ff9700 1 -- 172.21.15.89:0/2363201218 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0052d0 con 0x7f6a2809f240 2023-09-30T22:22:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.550+0000 7f6a24ff9700 1 -- 172.21.15.89:0/2363201218 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c005430 con 0x7f6a2809f240 2023-09-30T22:22:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.550+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a280a0190 con 0x7f6a2809f240 2023-09-30T22:22:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.550+0000 7f6a24ff9700 1 -- 172.21.15.89:0/2363201218 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0038c0 con 0x7f6a2809f240 2023-09-30T22:22:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.550+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a281a9910 con 0x7f6a2809f240 2023-09-30T22:22:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.551+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2809b430 con 0x7f6a2809f240 2023-09-30T22:22:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.551+0000 7f6a24ff9700 1 -- 172.21.15.89:0/2363201218 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a1c003a20 con 0x7f6a2809f240 2023-09-30T22:22:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.551+0000 7f6a24ff9700 1 -- 172.21.15.89:0/2363201218 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6a1c03ea10 con 0x7f6a2809f240 2023-09-30T22:22:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.552+0000 7f264effd700 1 -- 172.21.15.89:0/2350516743 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2650024ce0 con 0x7f265809e410 2023-09-30T22:22:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.556+0000 7f6a24ff9700 1 -- 172.21.15.89:0/2363201218 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a1c018260 con 0x7f6a2809f240 2023-09-30T22:22:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.644+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd790005590 con 0x7fd7b01338a0 2023-09-30T22:22:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.645+0000 7fd7ad7fa700 1 -- 172.21.15.89:0/2138052660 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7a4011360 con 0x7fd7b01338a0 2023-09-30T22:22:03.646 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.647+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b01338a0 msgr2=0x7fd7b011ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.647+0000 7fd7b6d86700 1 --2- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b01338a0 0x7fd7b011ac00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c700 tx=0x7fd7a400ca10 comp rx=0 tx=0).stop 2023-09-30T22:22:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.648+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 shutdown_connections 2023-09-30T22:22:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.648+0000 7fd7b6d86700 1 --2- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b0139030 0x7fd7b011b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.648+0000 7fd7b6d86700 1 --2- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7b01338a0 0x7fd7b011ac00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.648+0000 7fd7b6d86700 1 --2- 172.21.15.89:0/2138052660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b0136710 0x7fd7b011b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.648+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 >> 172.21.15.89:0/2138052660 conn(0x7fd7b00a2aa0 msgr2=0x7fd7b00a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:03.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.650+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 shutdown_connections 2023-09-30T22:22:03.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.650+0000 7fd7b6d86700 1 -- 172.21.15.89:0/2138052660 wait complete. 2023-09-30T22:22:03.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.724+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f07b40a28e0 con 0x7f07b40a1340 2023-09-30T22:22:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.724+0000 7f07b0ff9700 1 -- 172.21.15.89:0/1923340521 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f07a8017070 con 0x7f07b40a1340 2023-09-30T22:22:03.725 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:03.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b40a1340 msgr2=0x7f07b411ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b40a1340 0x7f07b411ec50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c510 tx=0x7f07a800c820 comp rx=0 tx=0).stop 2023-09-30T22:22:03.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 shutdown_connections 2023-09-30T22:22:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07b4134170 0x7f07b411f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07b40a0a30 0x7f07b411e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 --2- 172.21.15.89:0/1923340521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07b40a1340 0x7f07b411ec50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 >> 172.21.15.89:0/1923340521 conn(0x7f07b40a6550 msgr2=0x7f07b409ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 shutdown_connections 2023-09-30T22:22:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.726+0000 7f07ba854700 1 -- 172.21.15.89:0/1923340521 wait complete. 2023-09-30T22:22:03.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:03.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.769+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f265806c360 con 0x7f265809e410 2023-09-30T22:22:03.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.770+0000 7f264effd700 1 -- 172.21.15.89:0/2350516743 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2650019070 con 0x7f265809e410 2023-09-30T22:22:03.771 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:03.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 msgr2=0x7f2658122ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658122ea0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2650016000 tx=0x7f2650004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 shutdown_connections 2023-09-30T22:22:03.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f265809e410 0x7f2658122ea0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f265809d6f0 0x7f2658122420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 --2- 172.21.15.89:0/2350516743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f265809e000 0x7f2658122960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 >> 172.21.15.89:0/2350516743 conn(0x7f2658066f50 msgr2=0x7f26580a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:03.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 shutdown_connections 2023-09-30T22:22:03.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.773+0000 7f26607eb700 1 -- 172.21.15.89:0/2350516743 wait complete. 2023-09-30T22:22:03.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.777+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6a28074ea0 con 0x7f6a2809f240 2023-09-30T22:22:03.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.778+0000 7f6a24ff9700 1 -- 172.21.15.89:0/2363201218 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a1c017030 con 0x7f6a2809f240 2023-09-30T22:22:03.778 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:03.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.781+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a2809f240 msgr2=0x7f6a2806bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:03.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.781+0000 7f6a2e870700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a2809f240 0x7f6a2806bdf0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00c4b0 tx=0x7f6a1c00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.781+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 shutdown_connections 2023-09-30T22:22:03.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.781+0000 7f6a2e870700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a2809f650 0x7f6a2806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.781+0000 7f6a2e870700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a2809e930 0x7f6a2806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.781+0000 7f6a2e870700 1 --2- 172.21.15.89:0/2363201218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a2809f240 0x7f6a2806bdf0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:03.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.781+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 >> 172.21.15.89:0/2363201218 conn(0x7f6a280a4210 msgr2=0x7f6a281b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:03.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.782+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 shutdown_connections 2023-09-30T22:22:03.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:03.782+0000 7f6a2e870700 1 -- 172.21.15.89:0/2363201218 wait complete. 2023-09-30T22:22:03.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:03.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:04.664 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:04.739 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:04.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:04.796 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.799+0000 7ff403bcb700 1 Processor -- start 2023-09-30T22:22:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.799+0000 7ff403bcb700 1 -- start start 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff403bcb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc1316c0 0x7ff3fc131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff403bcb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3fc09d9f0 0x7ff3fc06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff403bcb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3fc09e130 con 0x7ff3fc06acb0 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff403bcb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3fc1312b0 con 0x7ff3fc09d9f0 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff403bcb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc09f6e0 con 0x7ff3fc1316c0 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff401967700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc1316c0 0x7ff3fc131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff401166700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3fc09d9f0 0x7ff3fc06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff401166700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3fc09d9f0 0x7ff3fc06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48078/0 (socket says 172.21.15.89:48078) 2023-09-30T22:22:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff401967700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc1316c0 0x7ff3fc131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32774/0 (socket says 172.21.15.89:32774) 2023-09-30T22:22:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff401166700 1 -- 172.21.15.89:0/1445731732 learned_addr learned my addr 172.21.15.89:0/1445731732 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff400965700 1 -- 172.21.15.89:0/1445731732 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252642358 0 0) 0x7ff3f4001490 con 0x7ff3fc06acb0 2023-09-30T22:22:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.800+0000 7ff400965700 1 -- 172.21.15.89:0/1445731732 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e80035b0 con 0x7ff3fc06acb0 2023-09-30T22:22:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff401967700 1 -- 172.21.15.89:0/1445731732 >> v2:172.21.15.89:3300/0 conn(0x7ff3fc09d9f0 msgr2=0x7ff3fc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff401967700 1 --2- 172.21.15.89:0/1445731732 >> v2:172.21.15.89:3300/0 conn(0x7ff3fc09d9f0 0x7ff3fc06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff401967700 1 -- 172.21.15.89:0/1445731732 >> v1:172.21.15.89:6789/0 conn(0x7ff3fc06acb0 legacy=0x7ff3fc138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff401967700 1 -- 172.21.15.89:0/1445731732 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc09e760 con 0x7ff3fc1316c0 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff401166700 1 --2- 172.21.15.89:0/1445731732 >> v2:172.21.15.89:3300/0 conn(0x7ff3fc09d9f0 0x7ff3fc06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff401967700 1 --2- 172.21.15.89:0/1445731732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc1316c0 0x7ff3fc131a90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009840 tx=0x7ff3f8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8851fc915c26946 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff400965700 1 -- 172.21.15.89:0/1445731732 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8011830 con 0x7ff3fc1316c0 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff400965700 1 -- 172.21.15.89:0/1445731732 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f8011990 con 0x7ff3fc1316c0 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.801+0000 7ff400965700 1 -- 172.21.15.89:0/1445731732 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8011d00 con 0x7ff3fc1316c0 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.802+0000 7ff403bcb700 1 -- 172.21.15.89:0/1445731732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc1316c0 msgr2=0x7ff3fc131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.802+0000 7ff403bcb700 1 --2- 172.21.15.89:0/1445731732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc1316c0 0x7ff3fc131a90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009840 tx=0x7ff3f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.802+0000 7ff403bcb700 1 -- 172.21.15.89:0/1445731732 shutdown_connections 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.802+0000 7ff403bcb700 1 --2- 172.21.15.89:0/1445731732 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc1316c0 0x7ff3fc131a90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.802+0000 7ff403bcb700 1 --2- 172.21.15.89:0/1445731732 >> v2:172.21.15.89:3300/0 conn(0x7ff3fc09d9f0 0x7ff3fc06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.802+0000 7ff403bcb700 1 -- 172.21.15.89:0/1445731732 >> 172.21.15.89:0/1445731732 conn(0x7ff3fc066f50 msgr2=0x7ff3fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.802+0000 7ff403bcb700 1 -- 172.21.15.89:0/1445731732 shutdown_connections 2023-09-30T22:22:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.802+0000 7ff403bcb700 1 -- 172.21.15.89:0/1445731732 wait complete. 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.803+0000 7ff403bcb700 1 Processor -- start 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff403bcb700 1 -- start start 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff403bcb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3fc06acb0 0x7ff3fc119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff403bcb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc09d9f0 0x7ff3fc1199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff403bcb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3fc1316c0 0x7ff3fc119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff403bcb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc11a740 con 0x7ff3fc06acb0 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff403bcb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc11bb70 con 0x7ff3fc1316c0 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff403bcb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc12b900 con 0x7ff3fc09d9f0 2023-09-30T22:22:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff401967700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3fc06acb0 0x7ff3fc119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff401967700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3fc06acb0 0x7ff3fc119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48086/0 (socket says 172.21.15.89:48086) 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff401967700 1 -- 172.21.15.89:0/3552437856 learned_addr learned my addr 172.21.15.89:0/3552437856 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.804+0000 7ff402168700 1 --2- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3fc1316c0 0x7ff3fc119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff401967700 1 -- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc09d9f0 msgr2=0x7ff3fc1199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff401967700 1 --2- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc09d9f0 0x7ff3fc1199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff401967700 1 -- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3fc1316c0 msgr2=0x7ff3fc119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff401967700 1 --2- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3fc1316c0 0x7ff3fc119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff401967700 1 -- 172.21.15.89:0/3552437856 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec00d9f0 con 0x7ff3fc06acb0 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff401967700 1 --2- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3fc06acb0 0x7ff3fc119480 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8013000 tx=0x7ff3f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff3f27fc700 1 -- 172.21.15.89:0/3552437856 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80045d0 con 0x7ff3fc06acb0 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff3f27fc700 1 -- 172.21.15.89:0/3552437856 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f801f070 con 0x7ff3fc06acb0 2023-09-30T22:22:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f80094a0 con 0x7ff3fc06acb0 2023-09-30T22:22:04.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff3f27fc700 1 -- 172.21.15.89:0/3552437856 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80038a0 con 0x7ff3fc06acb0 2023-09-30T22:22:04.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.805+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc09ead0 con 0x7ff3fc06acb0 2023-09-30T22:22:04.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.806+0000 7ff3f27fc700 1 -- 172.21.15.89:0/3552437856 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff3f8003a00 con 0x7ff3fc06acb0 2023-09-30T22:22:04.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.806+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc11c4e0 con 0x7ff3fc06acb0 2023-09-30T22:22:04.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.806+0000 7ff3f27fc700 1 -- 172.21.15.89:0/3552437856 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff3f8021070 con 0x7ff3fc06acb0 2023-09-30T22:22:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.812+0000 7ff3f27fc700 1 -- 172.21.15.89:0/3552437856 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff3f8024970 con 0x7ff3fc06acb0 2023-09-30T22:22:04.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.874+0000 7f619fcb1700 1 Processor -- start 2023-09-30T22:22:04.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.874+0000 7f619fcb1700 1 -- start start 2023-09-30T22:22:04.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619fcb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f619809d6f0 0x7f619809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619fcb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e440 0x7f6198134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619fcb1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619809b120 con 0x7f619809e000 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619fcb1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619806b460 con 0x7f619809d6f0 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619fcb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f619809a7e0 con 0x7f619809e440 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619da4d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f619809d6f0 0x7f619809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619da4d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f619809d6f0 0x7f619809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48098/0 (socket says 172.21.15.89:48098) 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619da4d700 1 -- 172.21.15.89:0/1800855605 learned_addr learned my addr 172.21.15.89:0/1800855605 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619da4d700 1 -- 172.21.15.89:0/1800855605 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e440 msgr2=0x7f6198134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619da4d700 1 --2- 172.21.15.89:0/1800855605 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e440 0x7f6198134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619da4d700 1 -- 172.21.15.89:0/1800855605 >> v1:172.21.15.89:6789/0 conn(0x7f619809e000 legacy=0x7f61981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619da4d700 1 -- 172.21.15.89:0/1800855605 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619809a500 con 0x7f619809d6f0 2023-09-30T22:22:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.875+0000 7f619da4d700 1 --2- 172.21.15.89:0/1800855605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809d6f0 0x7f619809dac0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f6194009920 tx=0x7f6194009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f38e11601a1b0d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:04.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.876+0000 7f619ca4b700 1 -- 172.21.15.89:0/1800855605 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619400ba20 con 0x7f619809d6f0 2023-09-30T22:22:04.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.876+0000 7f619ca4b700 1 -- 172.21.15.89:0/1800855605 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619400bb80 con 0x7f619809d6f0 2023-09-30T22:22:04.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.876+0000 7f619ca4b700 1 -- 172.21.15.89:0/1800855605 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194009e70 con 0x7f619809d6f0 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.877+0000 7f619fcb1700 1 -- 172.21.15.89:0/1800855605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809d6f0 msgr2=0x7f619809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.877+0000 7f619fcb1700 1 --2- 172.21.15.89:0/1800855605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809d6f0 0x7f619809dac0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f6194009920 tx=0x7f6194009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.877+0000 7f619fcb1700 1 -- 172.21.15.89:0/1800855605 shutdown_connections 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.877+0000 7f619fcb1700 1 --2- 172.21.15.89:0/1800855605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809d6f0 0x7f619809dac0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.877+0000 7f619fcb1700 1 --2- 172.21.15.89:0/1800855605 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e440 0x7f6198134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.877+0000 7f619fcb1700 1 -- 172.21.15.89:0/1800855605 >> 172.21.15.89:0/1800855605 conn(0x7f6198066f50 msgr2=0x7f619809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.877+0000 7f619fcb1700 1 -- 172.21.15.89:0/1800855605 shutdown_connections 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.877+0000 7f619fcb1700 1 -- 172.21.15.89:0/1800855605 wait complete. 2023-09-30T22:22:04.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.878+0000 7f619fcb1700 1 Processor -- start 2023-09-30T22:22:04.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.878+0000 7f619fcb1700 1 -- start start 2023-09-30T22:22:04.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619fcb1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f619809d6f0 0x7f61981a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619fcb1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e000 0x7f61981a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619fcb1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809e440 0x7f61981a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619fcb1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61981a9960 con 0x7f619809e440 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619fcb1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61981a7af0 con 0x7f619809d6f0 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619fcb1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61980a21b0 con 0x7f619809e000 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619d24c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e000 0x7f61981a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619e24e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809e440 0x7f61981a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619d24c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e000 0x7f61981a6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32802/0 (socket says 172.21.15.89:32802) 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619d24c700 1 -- 172.21.15.89:0/2429976202 learned_addr learned my addr 172.21.15.89:0/2429976202 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619e24e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809e440 0x7f61981a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48112/0 (socket says 172.21.15.89:48112) 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619e24e700 1 -- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e000 msgr2=0x7f61981a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619e24e700 1 --2- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e000 0x7f61981a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619e24e700 1 -- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f619809d6f0 msgr2=0x7f61981a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619e24e700 1 --2- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f619809d6f0 0x7f61981a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619e24e700 1 -- 172.21.15.89:0/2429976202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6194009550 con 0x7f619809e440 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.879+0000 7f619e24e700 1 --2- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809e440 0x7f61981a68c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f619000be20 tx=0x7f6190009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.880+0000 7f618e7fc700 1 -- 172.21.15.89:0/2429976202 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000cc10 con 0x7f619809e440 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.880+0000 7f618e7fc700 1 -- 172.21.15.89:0/2429976202 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619000cd90 con 0x7f619809e440 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.880+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619809c710 con 0x7f619809e440 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.880+0000 7f618e7fc700 1 -- 172.21.15.89:0/2429976202 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190014680 con 0x7f619809e440 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.880+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61981a8850 con 0x7f619809e440 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.881+0000 7f618e7fc700 1 -- 172.21.15.89:0/2429976202 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f619001d3b0 con 0x7f619809e440 2023-09-30T22:22:04.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.881+0000 7f618e7fc700 1 -- 172.21.15.89:0/2429976202 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6190006e40 con 0x7f619809e440 2023-09-30T22:22:04.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.881+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6198074ea0 con 0x7f619809e440 2023-09-30T22:22:04.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.886+0000 7f618e7fc700 1 -- 172.21.15.89:0/2429976202 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6190019080 con 0x7f619809e440 2023-09-30T22:22:04.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d20e5c700 1 Processor -- start 2023-09-30T22:22:04.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d20e5c700 1 -- start start 2023-09-30T22:22:04.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d20e5c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d20e5c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c12f8b0 0x7f4d1c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d20e5c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d1c130260 0x7f4d1c136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d20e5c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c12d330 con 0x7f4d1c130260 2023-09-30T22:22:04.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d20e5c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c12cd90 con 0x7f4d1c12f8b0 2023-09-30T22:22:04.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d20e5c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1306d0 con 0x7f4d1c12cf60 2023-09-30T22:22:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.925+0000 7f4d19d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c12f8b0 0x7f4d1c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d19d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c12f8b0 0x7f4d1c12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35476/0 (socket says 172.21.15.89:35476) 2023-09-30T22:22:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d19d9b700 1 -- 172.21.15.89:0/3130809872 learned_addr learned my addr 172.21.15.89:0/3130809872 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d1ad9d700 1 --2- 172.21.15.89:0/3130809872 >> v2:172.21.15.89:3300/0 conn(0x7f4d1c130260 0x7f4d1c136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d1a59c700 1 --2- 172.21.15.89:0/3130809872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d19d9b700 1 -- 172.21.15.89:0/3130809872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 msgr2=0x7f4d1c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d19d9b700 1 --2- 172.21.15.89:0/3130809872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d19d9b700 1 -- 172.21.15.89:0/3130809872 >> v2:172.21.15.89:3300/0 conn(0x7f4d1c130260 msgr2=0x7f4d1c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d19d9b700 1 --2- 172.21.15.89:0/3130809872 >> v2:172.21.15.89:3300/0 conn(0x7f4d1c130260 0x7f4d1c136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d19d9b700 1 -- 172.21.15.89:0/3130809872 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c071780 con 0x7f4d1c12f8b0 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d1ad9d700 1 --2- 172.21.15.89:0/3130809872 >> v2:172.21.15.89:3300/0 conn(0x7f4d1c130260 0x7f4d1c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d19d9b700 1 --2- 172.21.15.89:0/3130809872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c12f8b0 0x7f4d1c12fd20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000aa90 tx=0x7f4d1000ada0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f1cb96d1131b841 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d1959a700 1 -- 172.21.15.89:0/3130809872 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000cb70 con 0x7f4d1c12f8b0 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d1959a700 1 -- 172.21.15.89:0/3130809872 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1000ccd0 con 0x7f4d1c12f8b0 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.926+0000 7f4d1959a700 1 -- 172.21.15.89:0/3130809872 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10018620 con 0x7f4d1c12f8b0 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 -- 172.21.15.89:0/3130809872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c12f8b0 msgr2=0x7f4d1c12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 --2- 172.21.15.89:0/3130809872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c12f8b0 0x7f4d1c12fd20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000aa90 tx=0x7f4d1000ada0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 -- 172.21.15.89:0/3130809872 shutdown_connections 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 --2- 172.21.15.89:0/3130809872 >> v2:172.21.15.89:3300/0 conn(0x7f4d1c130260 0x7f4d1c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 --2- 172.21.15.89:0/3130809872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 --2- 172.21.15.89:0/3130809872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c12f8b0 0x7f4d1c12fd20 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 -- 172.21.15.89:0/3130809872 >> 172.21.15.89:0/3130809872 conn(0x7f4d1c0a2aa0 msgr2=0x7f4d1c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:04.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 -- 172.21.15.89:0/3130809872 shutdown_connections 2023-09-30T22:22:04.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.927+0000 7f4d20e5c700 1 -- 172.21.15.89:0/3130809872 wait complete. 2023-09-30T22:22:04.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.928+0000 7f674cb1d700 1 Processor -- start 2023-09-30T22:22:04.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.928+0000 7f674cb1d700 1 -- start start 2023-09-30T22:22:04.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.928+0000 7f4d20e5c700 1 Processor -- start 2023-09-30T22:22:04.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674cb1d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67481338c0 0x7f6748133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674cb1d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674813b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674cb1d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674806f720 con 0x7f67481341d0 2023-09-30T22:22:04.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674cb1d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6748070c80 con 0x7f67481338c0 2023-09-30T22:22:04.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674cb1d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674806fce0 con 0x7f674809e560 2023-09-30T22:22:04.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67481338c0 0x7f6748133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f6746d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674813b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67481338c0 0x7f6748133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48136/0 (socket says 172.21.15.89:48136) 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674659c700 1 -- 172.21.15.89:0/2580533632 learned_addr learned my addr 172.21.15.89:0/2580533632 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674659c700 1 -- 172.21.15.89:0/2580533632 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 msgr2=0x7f674813b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674659c700 1 --2- 172.21.15.89:0/2580533632 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674813b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674659c700 1 -- 172.21.15.89:0/2580533632 >> v1:172.21.15.89:6789/0 conn(0x7f67481341d0 legacy=0x7f674809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674659c700 1 -- 172.21.15.89:0/2580533632 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67481336a0 con 0x7f67481338c0 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f674659c700 1 --2- 172.21.15.89:0/2580533632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 0x7f6748133c90 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f67380098f0 tx=0x7f6738009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47f23cf1c17035a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:04.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f674559a700 1 -- 172.21.15.89:0/2580533632 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738011830 con 0x7f67481338c0 2023-09-30T22:22:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.929+0000 7f4d20e5c700 1 -- start start 2023-09-30T22:22:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d20e5c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c1c3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d20e5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c12f8b0 0x7f4d1c1c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d20e5c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c130260 0x7f4d1c1c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d20e5c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c11dd90 con 0x7f4d1c12f8b0 2023-09-30T22:22:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d20e5c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c11d2d0 con 0x7f4d1c130260 2023-09-30T22:22:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d20e5c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1c3aa0 con 0x7f4d1c12cf60 2023-09-30T22:22:04.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c1c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c1c3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32826/0 (socket says 172.21.15.89:32826) 2023-09-30T22:22:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1a59c700 1 -- 172.21.15.89:0/1663684539 learned_addr learned my addr 172.21.15.89:0/1663684539 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d19d9b700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c12f8b0 0x7f4d1c1c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f674559a700 1 -- 172.21.15.89:0/2580533632 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6738011990 con 0x7f67481338c0 2023-09-30T22:22:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f674559a700 1 -- 172.21.15.89:0/2580533632 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738011d20 con 0x7f67481338c0 2023-09-30T22:22:04.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1ad9d700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c130260 0x7f4d1c1c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f674cb1d700 1 -- 172.21.15.89:0/2580533632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 msgr2=0x7f6748133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f674cb1d700 1 --2- 172.21.15.89:0/2580533632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 0x7f6748133c90 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f67380098f0 tx=0x7f6738009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1a59c700 1 -- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c130260 msgr2=0x7f4d1c1c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1a59c700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c130260 0x7f4d1c1c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1a59c700 1 -- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c12f8b0 msgr2=0x7f4d1c1c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1a59c700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c12f8b0 0x7f4d1c1c3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.930+0000 7f4d1a59c700 1 -- 172.21.15.89:0/1663684539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14009a70 con 0x7f4d1c12cf60 2023-09-30T22:22:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f674cb1d700 1 -- 172.21.15.89:0/2580533632 shutdown_connections 2023-09-30T22:22:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f674cb1d700 1 --2- 172.21.15.89:0/2580533632 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674813b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f674cb1d700 1 --2- 172.21.15.89:0/2580533632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 0x7f6748133c90 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f674cb1d700 1 -- 172.21.15.89:0/2580533632 >> 172.21.15.89:0/2580533632 conn(0x7f67480a2aa0 msgr2=0x7f67480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f4d1a59c700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c1c3020 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00ec10 tx=0x7f4d0c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f674cb1d700 1 -- 172.21.15.89:0/2580533632 shutdown_connections 2023-09-30T22:22:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f674cb1d700 1 -- 172.21.15.89:0/2580533632 wait complete. 2023-09-30T22:22:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f4d0affd700 1 -- 172.21.15.89:0/1663684539 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c013860 con 0x7f4d1c12cf60 2023-09-30T22:22:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f4d0affd700 1 -- 172.21.15.89:0/1663684539 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c0139c0 con 0x7f4d1c12cf60 2023-09-30T22:22:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f4d0affd700 1 -- 172.21.15.89:0/1663684539 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c013d30 con 0x7f4d1c12cf60 2023-09-30T22:22:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1000a6f0 con 0x7f4d1c12cf60 2023-09-30T22:22:04.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.931+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c12d9d0 con 0x7f4d1c12cf60 2023-09-30T22:22:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f674cb1d700 1 Processor -- start 2023-09-30T22:22:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f674cb1d700 1 -- start start 2023-09-30T22:22:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f674cb1d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674811b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f674cb1d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 0x7f674811baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f674cb1d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67481341d0 0x7f674811bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f674cb1d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674811ca10 con 0x7f67481338c0 2023-09-30T22:22:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f674cb1d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674811d170 con 0x7f674809e560 2023-09-30T22:22:04.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f674cb1d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f674811e3c0 con 0x7f67481341d0 2023-09-30T22:22:04.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67481341d0 0x7f674811bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67481341d0 0x7f674811bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32840/0 (socket says 172.21.15.89:32840) 2023-09-30T22:22:04.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 -- 172.21.15.89:0/1227350496 learned_addr learned my addr 172.21.15.89:0/1227350496 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:04.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f674659c700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674811b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c074ea0 con 0x7f4d1c12cf60 2023-09-30T22:22:04.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f4d0affd700 1 -- 172.21.15.89:0/1663684539 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d0c00c640 con 0x7f4d1c12cf60 2023-09-30T22:22:04.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.932+0000 7f4d0affd700 1 -- 172.21.15.89:0/1663684539 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4d0c043610 con 0x7f4d1c12cf60 2023-09-30T22:22:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6745d9b700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 0x7f674811baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 -- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 msgr2=0x7f674811b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674811b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 -- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 msgr2=0x7f674811baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 0x7f674811baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:04.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 -- 172.21.15.89:0/1227350496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6738009550 con 0x7f67481341d0 2023-09-30T22:22:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f674659c700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674811b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.933+0000 7f6746d9d700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67481341d0 0x7f674811bfe0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6740009dd0 tx=0x7f674000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.937+0000 7f6736ffd700 1 -- 172.21.15.89:0/1227350496 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740009c40 con 0x7f67481341d0 2023-09-30T22:22:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.937+0000 7f6736ffd700 1 -- 172.21.15.89:0/1227350496 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674000f040 con 0x7f67481341d0 2023-09-30T22:22:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.937+0000 7f6736ffd700 1 -- 172.21.15.89:0/1227350496 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740014480 con 0x7f67481341d0 2023-09-30T22:22:04.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.937+0000 7f4d0affd700 1 -- 172.21.15.89:0/1663684539 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d0c01a990 con 0x7f4d1c12cf60 2023-09-30T22:22:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.937+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674811ee00 con 0x7f67481341d0 2023-09-30T22:22:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.937+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67480a1310 con 0x7f67481341d0 2023-09-30T22:22:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.938+0000 7f6736ffd700 1 -- 172.21.15.89:0/1227350496 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67400145e0 con 0x7f67481341d0 2023-09-30T22:22:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.938+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674806f9c0 con 0x7f67481341d0 2023-09-30T22:22:04.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.938+0000 7f6736ffd700 1 -- 172.21.15.89:0/1227350496 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f674003fee0 con 0x7f67481341d0 2023-09-30T22:22:04.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:04.947+0000 7f6736ffd700 1 -- 172.21.15.89:0/1227350496 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f674001a680 con 0x7f67481341d0 2023-09-30T22:22:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.017+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff3fc074ea0 con 0x7ff3fc06acb0 2023-09-30T22:22:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.017+0000 7ff3f27fc700 1 -- 172.21.15.89:0/3552437856 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3f802b030 con 0x7ff3fc06acb0 2023-09-30T22:22:05.018 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.019+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3fc06acb0 msgr2=0x7ff3fc119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.019+0000 7ff403bcb700 1 --2- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3fc06acb0 0x7ff3fc119480 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8013000 tx=0x7ff3f8004750 comp rx=0 tx=0).stop 2023-09-30T22:22:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.020+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 shutdown_connections 2023-09-30T22:22:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.020+0000 7ff403bcb700 1 --2- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3fc1316c0 0x7ff3fc119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.020+0000 7ff403bcb700 1 --2- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3fc06acb0 0x7ff3fc119480 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.020+0000 7ff403bcb700 1 --2- 172.21.15.89:0/3552437856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3fc09d9f0 0x7ff3fc1199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.020+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 >> 172.21.15.89:0/3552437856 conn(0x7ff3fc066f50 msgr2=0x7ff3fc09bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.020+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 shutdown_connections 2023-09-30T22:22:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.020+0000 7ff403bcb700 1 -- 172.21.15.89:0/3552437856 wait complete. 2023-09-30T22:22:05.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:05.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.081+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f619809f420 con 0x7f619809e440 2023-09-30T22:22:05.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.081+0000 7f618e7fc700 1 -- 172.21.15.89:0/2429976202 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f619001c360 con 0x7f619809e440 2023-09-30T22:22:05.082 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:05.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.083+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809e440 msgr2=0x7f61981a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:05.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.083+0000 7f619fcb1700 1 --2- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809e440 0x7f61981a68c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f619000be20 tx=0x7f6190009ed0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.083+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 shutdown_connections 2023-09-30T22:22:05.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.083+0000 7f619fcb1700 1 --2- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f619809e440 0x7f61981a68c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.083+0000 7f619fcb1700 1 --2- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f619809d6f0 0x7f61981a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.083+0000 7f619fcb1700 1 --2- 172.21.15.89:0/2429976202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f619809e000 0x7f61981a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.083+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 >> 172.21.15.89:0/2429976202 conn(0x7f6198066f50 msgr2=0x7f61981bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:05.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.084+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 shutdown_connections 2023-09-30T22:22:05.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.084+0000 7f619fcb1700 1 -- 172.21.15.89:0/2429976202 wait complete. 2023-09-30T22:22:05.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.136+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d1c0713a0 con 0x7f4d1c12cf60 2023-09-30T22:22:05.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.136+0000 7f4d0affd700 1 -- 172.21.15.89:0/1663684539 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d0c020030 con 0x7f4d1c12cf60 2023-09-30T22:22:05.137 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 msgr2=0x7f4d1c1c3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c1c3020 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00ec10 tx=0x7f4d0c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:22:05.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 shutdown_connections 2023-09-30T22:22:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d1c130260 0x7f4d1c1c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d1c12cf60 0x7f4d1c1c3020 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 --2- 172.21.15.89:0/1663684539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d1c12f8b0 0x7f4d1c1c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 >> 172.21.15.89:0/1663684539 conn(0x7f4d1c0a2aa0 msgr2=0x7f4d1c09e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 shutdown_connections 2023-09-30T22:22:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.138+0000 7f4d20e5c700 1 -- 172.21.15.89:0/1663684539 wait complete. 2023-09-30T22:22:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.140+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f674809d1f0 con 0x7f67481341d0 2023-09-30T22:22:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.140+0000 7f6736ffd700 1 -- 172.21.15.89:0/1227350496 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6740018030 con 0x7f67481341d0 2023-09-30T22:22:05.141 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67481341d0 msgr2=0x7f674811bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67481341d0 0x7f674811bfe0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6740009dd0 tx=0x7f674000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 shutdown_connections 2023-09-30T22:22:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67481341d0 0x7f674811bfe0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f674809e560 0x7f674811b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 --2- 172.21.15.89:0/1227350496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67481338c0 0x7f674811baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 >> 172.21.15.89:0/1227350496 conn(0x7f67480a2aa0 msgr2=0x7f6748131d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 shutdown_connections 2023-09-30T22:22:05.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:05.143+0000 7f674cb1d700 1 -- 172.21.15.89:0/1227350496 wait complete. 2023-09-30T22:22:05.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:05.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:06.034 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:06.097 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:06.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:06.157 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.171+0000 7fd227ab1700 1 Processor -- start 2023-09-30T22:22:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.171+0000 7fd227ab1700 1 -- start start 2023-09-30T22:22:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd227ab1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd220133510 0x7fd2201358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd227ab1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 0x7fd220138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd227ab1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd220130260 con 0x7fd220130670 2023-09-30T22:22:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd227ab1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22009a350 con 0x7fd220133510 2023-09-30T22:22:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd227ab1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd220071ff0 con 0x7fd220135e30 2023-09-30T22:22:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd22504c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd220133510 0x7fd2201358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd22504c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd220133510 0x7fd2201358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48172/0 (socket says 172.21.15.89:48172) 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd22504c700 1 -- 172.21.15.89:0/515499117 learned_addr learned my addr 172.21.15.89:0/515499117 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd22604e700 1 --2- 172.21.15.89:0/515499117 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 0x7fd220138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd22504c700 1 -- 172.21.15.89:0/515499117 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 msgr2=0x7fd220138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd22504c700 1 --2- 172.21.15.89:0/515499117 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 0x7fd220138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd22504c700 1 -- 172.21.15.89:0/515499117 >> v1:172.21.15.89:6789/0 conn(0x7fd220130670 legacy=0x7fd220130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.172+0000 7fd22504c700 1 -- 172.21.15.89:0/515499117 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22009ecf0 con 0x7fd220133510 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.173+0000 7fd22504c700 1 --2- 172.21.15.89:0/515499117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220133510 0x7fd2201358f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd2100098f0 tx=0x7fd210009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf2863682cc85e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.173+0000 7fd22484b700 1 -- 172.21.15.89:0/515499117 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210004230 con 0x7fd220133510 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.173+0000 7fd22484b700 1 -- 172.21.15.89:0/515499117 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd210004390 con 0x7fd220133510 2023-09-30T22:22:06.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.173+0000 7fd22484b700 1 -- 172.21.15.89:0/515499117 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100036a0 con 0x7fd220133510 2023-09-30T22:22:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.174+0000 7fd227ab1700 1 -- 172.21.15.89:0/515499117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220133510 msgr2=0x7fd2201358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.174+0000 7fd227ab1700 1 --2- 172.21.15.89:0/515499117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220133510 0x7fd2201358f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd2100098f0 tx=0x7fd210009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.176+0000 7fd227ab1700 1 -- 172.21.15.89:0/515499117 shutdown_connections 2023-09-30T22:22:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.176+0000 7fd227ab1700 1 --2- 172.21.15.89:0/515499117 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 0x7fd220138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.176+0000 7fd227ab1700 1 --2- 172.21.15.89:0/515499117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220133510 0x7fd2201358f0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.176+0000 7fd227ab1700 1 -- 172.21.15.89:0/515499117 >> 172.21.15.89:0/515499117 conn(0x7fd2200a2aa0 msgr2=0x7fd2200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.177+0000 7fd227ab1700 1 -- 172.21.15.89:0/515499117 shutdown_connections 2023-09-30T22:22:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.177+0000 7fd227ab1700 1 -- 172.21.15.89:0/515499117 wait complete. 2023-09-30T22:22:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.178+0000 7fd227ab1700 1 Processor -- start 2023-09-30T22:22:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.178+0000 7fd227ab1700 1 -- start start 2023-09-30T22:22:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd227ab1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220130670 0x7fd2201c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd227ab1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd220133510 0x7fd2201c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd227ab1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 0x7fd22009c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd227ab1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22011b570 con 0x7fd220130670 2023-09-30T22:22:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd227ab1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22011b750 con 0x7fd220135e30 2023-09-30T22:22:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd227ab1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201c6030 con 0x7fd220133510 2023-09-30T22:22:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220130670 0x7fd2201c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220130670 0x7fd2201c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48182/0 (socket says 172.21.15.89:48182) 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 -- 172.21.15.89:0/3942751878 learned_addr learned my addr 172.21.15.89:0/3942751878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22504c700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd220133510 0x7fd2201c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22604e700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 0x7fd22009c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 -- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd220133510 msgr2=0x7fd2201c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd220133510 0x7fd2201c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 -- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 msgr2=0x7fd22009c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 0x7fd22009c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 -- 172.21.15.89:0/3942751878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210009550 con 0x7fd220130670 2023-09-30T22:22:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.179+0000 7fd22584d700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220130670 0x7fd2201c63f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd21c001e50 tx=0x7fd21c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.180+0000 7fd2167fc700 1 -- 172.21.15.89:0/3942751878 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0054e0 con 0x7fd220130670 2023-09-30T22:22:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.180+0000 7fd2167fc700 1 -- 172.21.15.89:0/3942751878 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21c003680 con 0x7fd220130670 2023-09-30T22:22:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.180+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22009e510 con 0x7fd220130670 2023-09-30T22:22:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.180+0000 7fd2167fc700 1 -- 172.21.15.89:0/3942751878 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c003a30 con 0x7fd220130670 2023-09-30T22:22:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.180+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22006f100 con 0x7fd220130670 2023-09-30T22:22:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.181+0000 7fd2167fc700 1 -- 172.21.15.89:0/3942751878 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd21c017020 con 0x7fd220130670 2023-09-30T22:22:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.181+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd220074ea0 con 0x7fd220130670 2023-09-30T22:22:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.181+0000 7fd2167fc700 1 -- 172.21.15.89:0/3942751878 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd21c042cc0 con 0x7fd220130670 2023-09-30T22:22:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.187+0000 7fd2167fc700 1 -- 172.21.15.89:0/3942751878 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd21c018330 con 0x7fd220130670 2023-09-30T22:22:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.238+0000 7f05bb7a6700 1 Processor -- start 2023-09-30T22:22:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.238+0000 7f05bb7a6700 1 -- start start 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05bb7a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f05b41338a0 0x7f05b4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05bb7a6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b409de50 0x7f05b409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05bb7a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b406ef30 con 0x7f05b409e760 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05bb7a6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b406fef0 con 0x7f05b41338a0 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05bb7a6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b41334b0 con 0x7f05b409de50 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05b8d41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b409de50 0x7f05b409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05b8d41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b409de50 0x7f05b409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35542/0 (socket says 172.21.15.89:35542) 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05b8d41700 1 -- 172.21.15.89:0/471672722 learned_addr learned my addr 172.21.15.89:0/471672722 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:06.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.239+0000 7f05b9542700 1 --2- 172.21.15.89:0/471672722 >> v2:172.21.15.89:3300/0 conn(0x7f05b41338a0 0x7f05b4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05b9542700 1 -- 172.21.15.89:0/471672722 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b409de50 msgr2=0x7f05b409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05b9542700 1 --2- 172.21.15.89:0/471672722 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b409de50 0x7f05b409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05b9542700 1 -- 172.21.15.89:0/471672722 >> v1:172.21.15.89:6789/0 conn(0x7f05b409e760 legacy=0x7f05b413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05abfff700 1 -- 172.21.15.89:0/471672722 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154817620 0 0) 0x7f05b406ef30 con 0x7f05b409e760 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05b9542700 1 -- 172.21.15.89:0/471672722 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b406fc30 con 0x7f05b41338a0 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05b9542700 1 --2- 172.21.15.89:0/471672722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b41338a0 0x7f05b4133c70 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009840 tx=0x7f05b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898a718b1a214bce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05abfff700 1 -- 172.21.15.89:0/471672722 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0011830 con 0x7f05b41338a0 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05abfff700 1 -- 172.21.15.89:0/471672722 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0011990 con 0x7f05b41338a0 2023-09-30T22:22:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.240+0000 7f05abfff700 1 -- 172.21.15.89:0/471672722 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0011d20 con 0x7f05b41338a0 2023-09-30T22:22:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.241+0000 7f05bb7a6700 1 -- 172.21.15.89:0/471672722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b41338a0 msgr2=0x7f05b4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.241+0000 7f05bb7a6700 1 --2- 172.21.15.89:0/471672722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b41338a0 0x7f05b4133c70 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009840 tx=0x7f05b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.241+0000 7f05bb7a6700 1 -- 172.21.15.89:0/471672722 shutdown_connections 2023-09-30T22:22:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.241+0000 7f05bb7a6700 1 --2- 172.21.15.89:0/471672722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b41338a0 0x7f05b4133c70 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.241+0000 7f05bb7a6700 1 --2- 172.21.15.89:0/471672722 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b409de50 0x7f05b409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.241+0000 7f05bb7a6700 1 -- 172.21.15.89:0/471672722 >> 172.21.15.89:0/471672722 conn(0x7f05b40a2aa0 msgr2=0x7f05b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.241+0000 7f05bb7a6700 1 -- 172.21.15.89:0/471672722 shutdown_connections 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.241+0000 7f05bb7a6700 1 -- 172.21.15.89:0/471672722 wait complete. 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.242+0000 7f05bb7a6700 1 Processor -- start 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.242+0000 7f05bb7a6700 1 -- start start 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.242+0000 7f05bb7a6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05b409de50 0x7f05b41214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.242+0000 7f05bb7a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b409e760 0x7f05b4121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.242+0000 7f05bb7a6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b41338a0 0x7f05b4121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.242+0000 7f05bb7a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4124440 con 0x7f05b409e760 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.242+0000 7f05bb7a6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b40a0d40 con 0x7f05b41338a0 2023-09-30T22:22:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.242+0000 7f05bb7a6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4124a60 con 0x7f05b409de50 2023-09-30T22:22:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b409e760 0x7f05b4121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b409e760 0x7f05b4121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48208/0 (socket says 172.21.15.89:48208) 2023-09-30T22:22:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 -- 172.21.15.89:0/594789500 learned_addr learned my addr 172.21.15.89:0/594789500 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b9d43700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b41338a0 0x7f05b4121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b9542700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05b409de50 0x7f05b41214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 -- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05b409de50 msgr2=0x7f05b41214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05b409de50 0x7f05b41214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 -- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b41338a0 msgr2=0x7f05b4121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b41338a0 0x7f05b4121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 -- 172.21.15.89:0/594789500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00094a0 con 0x7f05b409e760 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05b8d41700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b409e760 0x7f05b4121a20 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f05a400e8b0 tx=0x7f05a400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05a9ffb700 1 -- 172.21.15.89:0/594789500 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4009600 con 0x7f05b409e760 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05a9ffb700 1 -- 172.21.15.89:0/594789500 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a4013680 con 0x7f05b409e760 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05a9ffb700 1 -- 172.21.15.89:0/594789500 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4013a30 con 0x7f05b409e760 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.243+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b4124cd0 con 0x7f05b409e760 2023-09-30T22:22:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.244+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b406f640 con 0x7f05b409e760 2023-09-30T22:22:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.245+0000 7f05a9ffb700 1 -- 172.21.15.89:0/594789500 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05a4013b90 con 0x7f05b409e760 2023-09-30T22:22:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.245+0000 7f05a9ffb700 1 -- 172.21.15.89:0/594789500 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f05a403f610 con 0x7f05b409e760 2023-09-30T22:22:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.245+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b4074ea0 con 0x7f05b409e760 2023-09-30T22:22:06.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.250+0000 7f05a9ffb700 1 -- 172.21.15.89:0/594789500 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05a4021080 con 0x7f05b409e760 2023-09-30T22:22:06.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd12cfa700 1 Processor -- start 2023-09-30T22:22:06.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd12cfa700 1 -- start start 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd12cfa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd12cfa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e000 0x7fbd0c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd12cfa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd0c09e410 0x7fbd0c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd12cfa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c09f520 con 0x7fbd0c09e410 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd12cfa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c09f130 con 0x7fbd0c09e000 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd12cfa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c09e930 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.299+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e000 0x7fbd0c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e000 0x7fbd0c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35554/0 (socket says 172.21.15.89:35554) 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0bfff700 1 -- 172.21.15.89:0/3347784606 learned_addr learned my addr 172.21.15.89:0/3347784606 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd11297700 1 --2- 172.21.15.89:0/3347784606 >> v2:172.21.15.89:3300/0 conn(0x7fbd0c09e410 0x7fbd0c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd10a96700 1 --2- 172.21.15.89:0/3347784606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0bfff700 1 -- 172.21.15.89:0/3347784606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 msgr2=0x7fbd0c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:06.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0bfff700 1 --2- 172.21.15.89:0/3347784606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0bfff700 1 -- 172.21.15.89:0/3347784606 >> v2:172.21.15.89:3300/0 conn(0x7fbd0c09e410 msgr2=0x7fbd0c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:06.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0bfff700 1 --2- 172.21.15.89:0/3347784606 >> v2:172.21.15.89:3300/0 conn(0x7fbd0c09e410 0x7fbd0c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0bfff700 1 -- 172.21.15.89:0/3347784606 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c09b2d0 con 0x7fbd0c09e000 2023-09-30T22:22:06.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0bfff700 1 --2- 172.21.15.89:0/3347784606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e000 0x7fbd0c130660 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009920 tx=0x7fbcf8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5c63634f932094 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:06.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0b7fe700 1 -- 172.21.15.89:0/3347784606 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8011830 con 0x7fbd0c09e000 2023-09-30T22:22:06.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0b7fe700 1 -- 172.21.15.89:0/3347784606 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf8011990 con 0x7fbd0c09e000 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.300+0000 7fbd0b7fe700 1 -- 172.21.15.89:0/3347784606 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8011d20 con 0x7fbd0c09e000 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 -- 172.21.15.89:0/3347784606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e000 msgr2=0x7fbd0c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 --2- 172.21.15.89:0/3347784606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e000 0x7fbd0c130660 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009920 tx=0x7fbcf8009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 -- 172.21.15.89:0/3347784606 shutdown_connections 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 --2- 172.21.15.89:0/3347784606 >> v2:172.21.15.89:3300/0 conn(0x7fbd0c09e410 0x7fbd0c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 --2- 172.21.15.89:0/3347784606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 --2- 172.21.15.89:0/3347784606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e000 0x7fbd0c130660 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 -- 172.21.15.89:0/3347784606 >> 172.21.15.89:0/3347784606 conn(0x7fbd0c066f50 msgr2=0x7fbd0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 -- 172.21.15.89:0/3347784606 shutdown_connections 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.301+0000 7fbd12cfa700 1 -- 172.21.15.89:0/3347784606 wait complete. 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.302+0000 7fbd12cfa700 1 Processor -- start 2023-09-30T22:22:06.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.302+0000 7fbd12cfa700 1 -- start start 2023-09-30T22:22:06.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.303+0000 7fbd12cfa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c11f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.303+0000 7fbd12cfa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0c09e000 0x7fbd0c11f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.303+0000 7fbd12cfa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e410 0x7fbd0c11fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.303+0000 7fbd12cfa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c09c770 con 0x7fbd0c09e000 2023-09-30T22:22:06.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.303+0000 7fbd12cfa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c06eb60 con 0x7fbd0c09e410 2023-09-30T22:22:06.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.303+0000 7fbd12cfa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c121470 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd11297700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e410 0x7fbd0c11fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd10a96700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c11f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd11297700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e410 0x7fbd0c11fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35562/0 (socket says 172.21.15.89:35562) 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd11297700 1 -- 172.21.15.89:0/1646083181 learned_addr learned my addr 172.21.15.89:0/1646083181 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd0bfff700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0c09e000 0x7fbd0c11f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd10a96700 1 -- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e410 msgr2=0x7fbd0c11fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd10a96700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e410 0x7fbd0c11fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd10a96700 1 -- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0c09e000 msgr2=0x7fbd0c11f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd10a96700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0c09e000 0x7fbd0c11f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd10a96700 1 -- 172.21.15.89:0/1646083181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf8009550 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd11297700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e410 0x7fbd0c11fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd10a96700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c11f040 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000e870 tx=0x7fbd0000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd097fa700 1 -- 172.21.15.89:0/1646083181 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00013680 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd097fa700 1 -- 172.21.15.89:0/1646083181 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd000137e0 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c121ea0 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd097fa700 1 -- 172.21.15.89:0/1646083181 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00013b50 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.304+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c1222f0 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.305+0000 7fbd097fa700 1 -- 172.21.15.89:0/1646083181 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd00013cb0 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.305+0000 7fbd097fa700 1 -- 172.21.15.89:0/1646083181 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbd0003f900 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.306+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c074ea0 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.310+0000 7fbd097fa700 1 -- 172.21.15.89:0/1646083181 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd0001adc0 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.311+0000 7f94bc692700 1 Processor -- start 2023-09-30T22:22:06.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.311+0000 7f94bc692700 1 -- start start 2023-09-30T22:22:06.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.311+0000 7f94bc692700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.311+0000 7f94bc692700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94bc692700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b40a2a70 con 0x7f94b409d630 2023-09-30T22:22:06.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94bc692700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40a01d0 con 0x7f94b4138d80 2023-09-30T22:22:06.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94bc692700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94b409e2b0 con 0x7f94b40a0a30 2023-09-30T22:22:06.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94ba42e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94ba42e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32900/0 (socket says 172.21.15.89:32900) 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94ba42e700 1 -- 172.21.15.89:0/1047932298 learned_addr learned my addr 172.21.15.89:0/1047932298 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94bac2f700 1 --2- 172.21.15.89:0/1047932298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94ba42e700 1 -- 172.21.15.89:0/1047932298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 msgr2=0x7f94b413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94ba42e700 1 --2- 172.21.15.89:0/1047932298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94ba42e700 1 -- 172.21.15.89:0/1047932298 >> v1:172.21.15.89:6789/0 conn(0x7f94b409d630 legacy=0x7f94b40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94ba42e700 1 -- 172.21.15.89:0/1047932298 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b409dc50 con 0x7f94b40a0a30 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94ba42e700 1 --2- 172.21.15.89:0/1047932298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b40a0e00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009920 tx=0x7f94b0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a26dd497279f6d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94b942c700 1 -- 172.21.15.89:0/1047932298 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0004230 con 0x7f94b40a0a30 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94b942c700 1 -- 172.21.15.89:0/1047932298 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b0004390 con 0x7f94b40a0a30 2023-09-30T22:22:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.312+0000 7f94b942c700 1 -- 172.21.15.89:0/1047932298 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00036a0 con 0x7f94b40a0a30 2023-09-30T22:22:06.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.313+0000 7f94bc692700 1 -- 172.21.15.89:0/1047932298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 msgr2=0x7f94b40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.313+0000 7f94bc692700 1 --2- 172.21.15.89:0/1047932298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b40a0e00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009920 tx=0x7f94b0009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.313+0000 7f94bc692700 1 -- 172.21.15.89:0/1047932298 shutdown_connections 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.313+0000 7f94bc692700 1 --2- 172.21.15.89:0/1047932298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.313+0000 7f94bc692700 1 --2- 172.21.15.89:0/1047932298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b40a0e00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.313+0000 7f94bc692700 1 -- 172.21.15.89:0/1047932298 >> 172.21.15.89:0/1047932298 conn(0x7f94b40a3050 msgr2=0x7f94b40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.314+0000 7f94bc692700 1 -- 172.21.15.89:0/1047932298 shutdown_connections 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.314+0000 7f94bc692700 1 -- 172.21.15.89:0/1047932298 wait complete. 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.315+0000 7f94bc692700 1 Processor -- start 2023-09-30T22:22:06.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.315+0000 7f94bc692700 1 -- start start 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94bc692700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94b409d630 0x7f94b4120ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94bc692700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b4121020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94bc692700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b4121560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94bc692700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41246a0 con 0x7f94b409d630 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94bc692700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40a4f50 con 0x7f94b4138d80 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94bc692700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40a0410 con 0x7f94b40a0a30 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94b9c2d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b4121020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94bac2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b4121560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94b9c2d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b4121020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32912/0 (socket says 172.21.15.89:32912) 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94b9c2d700 1 -- 172.21.15.89:0/2341609873 learned_addr learned my addr 172.21.15.89:0/2341609873 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94ba42e700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94b409d630 0x7f94b4120ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94ba42e700 1 -- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 msgr2=0x7f94b4121020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94ba42e700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b4121020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94ba42e700 1 -- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 msgr2=0x7f94b4121560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94ba42e700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b4121560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94ba42e700 1 -- 172.21.15.89:0/2341609873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0009550 con 0x7f94b409d630 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94b9c2d700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b4121020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.316+0000 7f94bac2f700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b4121560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.317+0000 7f94ba42e700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94b409d630 0x7f94b4120ae0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f94b0015000 tx=0x7f94b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.317+0000 7f94aaffd700 1 -- 172.21.15.89:0/2341609873 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0003e60 con 0x7f94b409d630 2023-09-30T22:22:06.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.317+0000 7f94aaffd700 1 -- 172.21.15.89:0/2341609873 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b0011630 con 0x7f94b409d630 2023-09-30T22:22:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.317+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b4123d90 con 0x7f94b409d630 2023-09-30T22:22:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.317+0000 7f94aaffd700 1 -- 172.21.15.89:0/2341609873 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011a00 con 0x7f94b409d630 2023-09-30T22:22:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.317+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b4122ea0 con 0x7f94b409d630 2023-09-30T22:22:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.318+0000 7f94aaffd700 1 -- 172.21.15.89:0/2341609873 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94b0021070 con 0x7f94b409d630 2023-09-30T22:22:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.318+0000 7f94aaffd700 1 -- 172.21.15.89:0/2341609873 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f94b0040c60 con 0x7f94b409d630 2023-09-30T22:22:06.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.318+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b4124160 con 0x7f94b409d630 2023-09-30T22:22:06.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.323+0000 7f94aaffd700 1 -- 172.21.15.89:0/2341609873 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94b0011d40 con 0x7f94b409d630 2023-09-30T22:22:06.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.422+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd22011b890 con 0x7fd220130670 2023-09-30T22:22:06.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.422+0000 7fd2167fc700 1 -- 172.21.15.89:0/3942751878 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd22011b890 con 0x7fd220130670 2023-09-30T22:22:06.423 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:06.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220130670 msgr2=0x7fd2201c63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220130670 0x7fd2201c63f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd21c001e50 tx=0x7fd21c00ca30 comp rx=0 tx=0).stop 2023-09-30T22:22:06.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 shutdown_connections 2023-09-30T22:22:06.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd220135e30 0x7fd22009c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd220130670 0x7fd2201c63f0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 --2- 172.21.15.89:0/3942751878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd220133510 0x7fd2201c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 >> 172.21.15.89:0/3942751878 conn(0x7fd2200a2aa0 msgr2=0x7fd22012d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:06.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 shutdown_connections 2023-09-30T22:22:06.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.424+0000 7fd227ab1700 1 -- 172.21.15.89:0/3942751878 wait complete. 2023-09-30T22:22:06.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:06.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.493+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f05b406b9e0 con 0x7f05b409e760 2023-09-30T22:22:06.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.493+0000 7f05a9ffb700 1 -- 172.21.15.89:0/594789500 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05a4017020 con 0x7f05b409e760 2023-09-30T22:22:06.494 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:06.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.495+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b409e760 msgr2=0x7f05b4121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.495+0000 7f05bb7a6700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b409e760 0x7f05b4121a20 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f05a400e8b0 tx=0x7f05a400ebc0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.495+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 shutdown_connections 2023-09-30T22:22:06.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.495+0000 7f05bb7a6700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b41338a0 0x7f05b4121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.495+0000 7f05bb7a6700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f05b409de50 0x7f05b41214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.495+0000 7f05bb7a6700 1 --2- 172.21.15.89:0/594789500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05b409e760 0x7f05b4121a20 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.495+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 >> 172.21.15.89:0/594789500 conn(0x7f05b40a2aa0 msgr2=0x7f05b41ba560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:06.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.496+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 shutdown_connections 2023-09-30T22:22:06.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.496+0000 7f05bb7a6700 1 -- 172.21.15.89:0/594789500 wait complete. 2023-09-30T22:22:06.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:06.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.509+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbd0c06c360 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.509+0000 7fbd097fa700 1 -- 172.21.15.89:0/1646083181 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd00017020 con 0x7fbd0c09d6f0 2023-09-30T22:22:06.510 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 msgr2=0x7fbd0c11f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c11f040 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000e870 tx=0x7fbd0000eb80 comp rx=0 tx=0).stop 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 shutdown_connections 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbd0c09e410 0x7fbd0c11fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbd0c09d6f0 0x7fbd0c11f040 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 --2- 172.21.15.89:0/1646083181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0c09e000 0x7fbd0c11f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 >> 172.21.15.89:0/1646083181 conn(0x7fbd0c066f50 msgr2=0x7fbd0c119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 shutdown_connections 2023-09-30T22:22:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.512+0000 7fbd12cfa700 1 -- 172.21.15.89:0/1646083181 wait complete. 2023-09-30T22:22:06.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.544+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f94b4123f80 con 0x7f94b409d630 2023-09-30T22:22:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.545+0000 7f94aaffd700 1 -- 172.21.15.89:0/2341609873 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94b000bbd0 con 0x7f94b409d630 2023-09-30T22:22:06.545 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94b409d630 msgr2=0x7f94b4120ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94b409d630 0x7f94b4120ae0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f94b0015000 tx=0x7f94b0004750 comp rx=0 tx=0).stop 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 shutdown_connections 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94b4138d80 0x7f94b4121560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94b409d630 0x7f94b4120ae0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 --2- 172.21.15.89:0/2341609873 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f94b40a0a30 0x7f94b4121020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 >> 172.21.15.89:0/2341609873 conn(0x7f94b40a3050 msgr2=0x7f94b409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 shutdown_connections 2023-09-30T22:22:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:06.547+0000 7f94bc692700 1 -- 172.21.15.89:0/2341609873 wait complete. 2023-09-30T22:22:06.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:07.438 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:07.509 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:07.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:07.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:07.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.577+0000 7f3a7c929700 1 Processor -- start 2023-09-30T22:22:07.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.577+0000 7f3a7c929700 1 -- start start 2023-09-30T22:22:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7c929700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a7809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7c929700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f240 0x7f3a781317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7c929700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a7809f650 0x7f3a78131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7c929700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a78070940 con 0x7f3a7809f650 2023-09-30T22:22:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7c929700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7806fd60 con 0x7f3a7809f240 2023-09-30T22:22:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7c929700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7806fa80 con 0x7f3a7809e930 2023-09-30T22:22:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a75d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f240 0x7f3a781317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a7809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a75d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f240 0x7f3a781317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35576/0 (socket says 172.21.15.89:35576) 2023-09-30T22:22:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a75d9b700 1 -- 172.21.15.89:0/2781215915 learned_addr learned my addr 172.21.15.89:0/2781215915 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a76d9d700 1 --2- 172.21.15.89:0/2781215915 >> v2:172.21.15.89:3300/0 conn(0x7f3a7809f650 0x7f3a78131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a7809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32926/0 (socket says 172.21.15.89:32926) 2023-09-30T22:22:07.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7659c700 1 -- 172.21.15.89:0/2781215915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f240 msgr2=0x7f3a781317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7659c700 1 --2- 172.21.15.89:0/2781215915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f240 0x7f3a781317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7659c700 1 -- 172.21.15.89:0/2781215915 >> v2:172.21.15.89:3300/0 conn(0x7f3a7809f650 msgr2=0x7f3a78131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7659c700 1 --2- 172.21.15.89:0/2781215915 >> v2:172.21.15.89:3300/0 conn(0x7f3a7809f650 0x7f3a78131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7659c700 1 -- 172.21.15.89:0/2781215915 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7809fb60 con 0x7f3a7809e930 2023-09-30T22:22:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a75d9b700 1 --2- 172.21.15.89:0/2781215915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f240 0x7f3a781317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.578+0000 7f3a7659c700 1 --2- 172.21.15.89:0/2781215915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a7809ed00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800dc60 tx=0x7f3a6800df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9241ebe202c536e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.579+0000 7f3a7559a700 1 -- 172.21.15.89:0/2781215915 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68012070 con 0x7f3a7809e930 2023-09-30T22:22:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.579+0000 7f3a7559a700 1 -- 172.21.15.89:0/2781215915 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800d2e0 con 0x7f3a7809e930 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.579+0000 7f3a7559a700 1 -- 172.21.15.89:0/2781215915 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800f900 con 0x7f3a7809e930 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.579+0000 7f3a7c929700 1 -- 172.21.15.89:0/2781215915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 msgr2=0x7f3a7809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.579+0000 7f3a7c929700 1 --2- 172.21.15.89:0/2781215915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a7809ed00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800dc60 tx=0x7f3a6800df70 comp rx=0 tx=0).stop 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.580+0000 7f3a7c929700 1 -- 172.21.15.89:0/2781215915 shutdown_connections 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.580+0000 7f3a7c929700 1 --2- 172.21.15.89:0/2781215915 >> v2:172.21.15.89:3300/0 conn(0x7f3a7809f650 0x7f3a78131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.580+0000 7f3a7c929700 1 --2- 172.21.15.89:0/2781215915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a7809ed00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.580+0000 7f3a7c929700 1 --2- 172.21.15.89:0/2781215915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f240 0x7f3a781317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.580+0000 7f3a7c929700 1 -- 172.21.15.89:0/2781215915 >> 172.21.15.89:0/2781215915 conn(0x7f3a780a4210 msgr2=0x7f3a780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.580+0000 7f3a7c929700 1 -- 172.21.15.89:0/2781215915 shutdown_connections 2023-09-30T22:22:07.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.580+0000 7f3a7c929700 1 -- 172.21.15.89:0/2781215915 wait complete. 2023-09-30T22:22:07.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.582+0000 7f3a7c929700 1 Processor -- start 2023-09-30T22:22:07.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.582+0000 7f3a7c929700 1 -- start start 2023-09-30T22:22:07.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.582+0000 7f3a7c929700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a781c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.582+0000 7f3a7c929700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a7809f240 0x7f3a781c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.582+0000 7f3a7c929700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f650 0x7f3a781c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a7c929700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78071d20 con 0x7f3a7809f240 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a7c929700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781a4fb0 con 0x7f3a7809f650 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a7c929700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781a4340 con 0x7f3a7809e930 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a76d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f650 0x7f3a781c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a76d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f650 0x7f3a781c07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35580/0 (socket says 172.21.15.89:35580) 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a76d9d700 1 -- 172.21.15.89:0/1127660540 learned_addr learned my addr 172.21.15.89:0/1127660540 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a75d9b700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a7809f240 0x7f3a781c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a7659c700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a781c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a75d9b700 1 -- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 msgr2=0x7f3a781c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a75d9b700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a781c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a75d9b700 1 -- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f650 msgr2=0x7f3a781c07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.583+0000 7f3a75d9b700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f650 0x7f3a781c07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.584+0000 7f3a75d9b700 1 -- 172.21.15.89:0/1127660540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c009550 con 0x7f3a7809f240 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.584+0000 7f3a75d9b700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a7809f240 0x7f3a781c4810 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3a7806e6c0 tx=0x7f3a6c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.584+0000 7f3a66ffd700 1 -- 172.21.15.89:0/1127660540 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c017d80 con 0x7f3a7809f240 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.584+0000 7f3a66ffd700 1 -- 172.21.15.89:0/1127660540 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c004030 con 0x7f3a7809f240 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.584+0000 7f3a66ffd700 1 -- 172.21.15.89:0/1127660540 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0043e0 con 0x7f3a7809f240 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.584+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6800d940 con 0x7f3a7809f240 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.584+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a780a0190 con 0x7f3a7809f240 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.585+0000 7f3a66ffd700 1 -- 172.21.15.89:0/1127660540 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3a6c0036a0 con 0x7f3a7809f240 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.585+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a781a51d0 con 0x7f3a7809f240 2023-09-30T22:22:07.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.586+0000 7f3a66ffd700 1 -- 172.21.15.89:0/1127660540 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3a6c047860 con 0x7f3a7809f240 2023-09-30T22:22:07.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.591+0000 7f3a66ffd700 1 -- 172.21.15.89:0/1127660540 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a781a51d0 con 0x7f3a7809f240 2023-09-30T22:22:07.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.652+0000 7fee99b95700 1 Processor -- start 2023-09-30T22:22:07.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.652+0000 7fee99b95700 1 -- start start 2023-09-30T22:22:07.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.652+0000 7fee99b95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.652+0000 7fee99b95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee94135e50 0x7fee94138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.652+0000 7fee99b95700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9409f130 con 0x7fee9409d6f0 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.652+0000 7fee99b95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9409f520 con 0x7fee9409e1d0 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.652+0000 7fee99b95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee9409e930 con 0x7fee94135e50 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.652+0000 7fee92ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee937fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fee9409d6f0 0x7fee9409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37052/0 (socket says 172.21.15.89:37052) 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee937fe700 1 -- 172.21.15.89:0/2407571778 learned_addr learned my addr 172.21.15.89:0/2407571778 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee93fff700 1 --2- 172.21.15.89:0/2407571778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee94135e50 0x7fee94138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee92ffd700 1 -- 172.21.15.89:0/2407571778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee94135e50 msgr2=0x7fee94138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:07.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee92ffd700 1 --2- 172.21.15.89:0/2407571778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee94135e50 0x7fee94138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee927fc700 1 -- 172.21.15.89:0/2407571778 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158465482 0 0) 0x7fee9409f130 con 0x7fee9409d6f0 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee92ffd700 1 -- 172.21.15.89:0/2407571778 >> v1:172.21.15.89:6789/0 conn(0x7fee9409d6f0 legacy=0x7fee9409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee92ffd700 1 -- 172.21.15.89:0/2407571778 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9409b2d0 con 0x7fee9409e1d0 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee92ffd700 1 --2- 172.21.15.89:0/2407571778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94135910 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fee880098a0 tx=0x7fee88009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d72c7cfaf6d3fec2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee927fc700 1 -- 172.21.15.89:0/2407571778 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88011830 con 0x7fee9409e1d0 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee927fc700 1 -- 172.21.15.89:0/2407571778 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee88011990 con 0x7fee9409e1d0 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.653+0000 7fee927fc700 1 -- 172.21.15.89:0/2407571778 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88011d20 con 0x7fee9409e1d0 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.654+0000 7fee99b95700 1 -- 172.21.15.89:0/2407571778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 msgr2=0x7fee94135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.654+0000 7fee99b95700 1 --2- 172.21.15.89:0/2407571778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94135910 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fee880098a0 tx=0x7fee88009bb0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.654+0000 7fee99b95700 1 -- 172.21.15.89:0/2407571778 shutdown_connections 2023-09-30T22:22:07.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.654+0000 7fee99b95700 1 --2- 172.21.15.89:0/2407571778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee94135e50 0x7fee94138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.654+0000 7fee99b95700 1 --2- 172.21.15.89:0/2407571778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94135910 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.654+0000 7fee99b95700 1 -- 172.21.15.89:0/2407571778 >> 172.21.15.89:0/2407571778 conn(0x7fee94066f50 msgr2=0x7fee9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:07.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.654+0000 7fee99b95700 1 -- 172.21.15.89:0/2407571778 shutdown_connections 2023-09-30T22:22:07.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.654+0000 7fee99b95700 1 -- 172.21.15.89:0/2407571778 wait complete. 2023-09-30T22:22:07.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.655+0000 7fee99b95700 1 Processor -- start 2023-09-30T22:22:07.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee99b95700 1 -- start start 2023-09-30T22:22:07.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee99b95700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee9409d6f0 0x7fee941397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee99b95700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee99b95700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee94135e50 0x7fee9413a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee99b95700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee94135b20 con 0x7fee94135e50 2023-09-30T22:22:07.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee99b95700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee94120540 con 0x7fee9409e1d0 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee99b95700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee94120e90 con 0x7fee9409d6f0 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee92ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee92ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35586/0 (socket says 172.21.15.89:35586) 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee92ffd700 1 -- 172.21.15.89:0/1480215763 learned_addr learned my addr 172.21.15.89:0/1480215763 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.656+0000 7fee937fe700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee9409d6f0 0x7fee941397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee93fff700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee94135e50 0x7fee9413a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee92ffd700 1 -- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee9409d6f0 msgr2=0x7fee941397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee92ffd700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee9409d6f0 0x7fee941397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee92ffd700 1 -- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee94135e50 msgr2=0x7fee9413a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee92ffd700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee94135e50 0x7fee9413a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee92ffd700 1 -- 172.21.15.89:0/1480215763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee880094a0 con 0x7fee9409e1d0 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee93fff700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee94135e50 0x7fee9413a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee92ffd700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94139cf0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fee88009870 tx=0x7fee88004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee7bfff700 1 -- 172.21.15.89:0/1480215763 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8801a7d0 con 0x7fee9409e1d0 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee7bfff700 1 -- 172.21.15.89:0/1480215763 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8801a930 con 0x7fee9409e1d0 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee940a3b80 con 0x7fee9409e1d0 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee7bfff700 1 -- 172.21.15.89:0/1480215763 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8801ace0 con 0x7fee9409e1d0 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.657+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee94120bd0 con 0x7fee9409e1d0 2023-09-30T22:22:07.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.658+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9411ff60 con 0x7fee9409e1d0 2023-09-30T22:22:07.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.659+0000 7fee7bfff700 1 -- 172.21.15.89:0/1480215763 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee88022070 con 0x7fee9409e1d0 2023-09-30T22:22:07.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.659+0000 7fee7bfff700 1 -- 172.21.15.89:0/1480215763 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fee88040c30 con 0x7fee9409e1d0 2023-09-30T22:22:07.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.668+0000 7fee7bfff700 1 -- 172.21.15.89:0/1480215763 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee8802d3b0 con 0x7fee9409e1d0 2023-09-30T22:22:07.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.702+0000 7fcec0b41700 1 Processor -- start 2023-09-30T22:22:07.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.702+0000 7fcec0b41700 1 -- start start 2023-09-30T22:22:07.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcec0b41700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcebc09f410 0x7fcebc136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcec0b41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc136fe0 0x7fcebc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcec0b41700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcebc070940 con 0x7fcebc09e930 2023-09-30T22:22:07.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcec0b41700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcebc06fd60 con 0x7fcebc09f410 2023-09-30T22:22:07.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcec0b41700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc06fa80 con 0x7fcebc136fe0 2023-09-30T22:22:07.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcebad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc136fe0 0x7fcebc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcebad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc136fe0 0x7fcebc1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35588/0 (socket says 172.21.15.89:35588) 2023-09-30T22:22:07.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcebad9d700 1 -- 172.21.15.89:0/1014149245 learned_addr learned my addr 172.21.15.89:0/1014149245 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fceb9d9b700 1 --2- 172.21.15.89:0/1014149245 >> v2:172.21.15.89:3300/0 conn(0x7fcebc09f410 0x7fcebc136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcebad9d700 1 -- 172.21.15.89:0/1014149245 >> v2:172.21.15.89:3300/0 conn(0x7fcebc09f410 msgr2=0x7fcebc136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcebad9d700 1 --2- 172.21.15.89:0/1014149245 >> v2:172.21.15.89:3300/0 conn(0x7fcebc09f410 0x7fcebc136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.703+0000 7fcebad9d700 1 -- 172.21.15.89:0/1014149245 >> v1:172.21.15.89:6789/0 conn(0x7fcebc09e930 legacy=0x7fcebc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.704+0000 7fcebad9d700 1 -- 172.21.15.89:0/1014149245 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc09fb60 con 0x7fcebc136fe0 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.704+0000 7fcebad9d700 1 --2- 172.21.15.89:0/1014149245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc136fe0 0x7fcebc1393c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009e50 tx=0x7fceb400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68827be27b1e6a6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.704+0000 7fceb959a700 1 -- 172.21.15.89:0/1014149245 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb400e070 con 0x7fcebc136fe0 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.704+0000 7fceb959a700 1 -- 172.21.15.89:0/1014149245 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb400b9c0 con 0x7fcebc136fe0 2023-09-30T22:22:07.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.704+0000 7fceb959a700 1 -- 172.21.15.89:0/1014149245 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb400bd70 con 0x7fcebc136fe0 2023-09-30T22:22:07.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcec0b41700 1 -- 172.21.15.89:0/1014149245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc136fe0 msgr2=0x7fcebc1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcec0b41700 1 --2- 172.21.15.89:0/1014149245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc136fe0 0x7fcebc1393c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009e50 tx=0x7fceb400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:07.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcec0b41700 1 -- 172.21.15.89:0/1014149245 shutdown_connections 2023-09-30T22:22:07.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcec0b41700 1 --2- 172.21.15.89:0/1014149245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc136fe0 0x7fcebc1393c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcec0b41700 1 --2- 172.21.15.89:0/1014149245 >> v2:172.21.15.89:3300/0 conn(0x7fcebc09f410 0x7fcebc136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcec0b41700 1 -- 172.21.15.89:0/1014149245 >> 172.21.15.89:0/1014149245 conn(0x7fcebc0a4210 msgr2=0x7fcebc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:07.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcec0b41700 1 -- 172.21.15.89:0/1014149245 shutdown_connections 2023-09-30T22:22:07.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcec0b41700 1 -- 172.21.15.89:0/1014149245 wait complete. 2023-09-30T22:22:07.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.705+0000 7fcd23fef700 1 Processor -- start 2023-09-30T22:22:07.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd23fef700 1 -- start start 2023-09-30T22:22:07.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd23fef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd1c09e1d0 0x7fcd1c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd23fef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c139170 0x7fcd1c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd23fef700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c09b120 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd23fef700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c06b460 con 0x7fcd1c09e1d0 2023-09-30T22:22:07.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd23fef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c09a7e0 con 0x7fcd1c139170 2023-09-30T22:22:07.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd2158a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd1c09e1d0 0x7fcd1c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd2158a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd1c09e1d0 0x7fcd1c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48294/0 (socket says 172.21.15.89:48294) 2023-09-30T22:22:07.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcd2158a700 1 -- 172.21.15.89:0/1305497389 learned_addr learned my addr 172.21.15.89:0/1305497389 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:07.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.706+0000 7fcec0b41700 1 Processor -- start 2023-09-30T22:22:07.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcec0b41700 1 -- start start 2023-09-30T22:22:07.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcd2158a700 1 -- 172.21.15.89:0/1305497389 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c139170 msgr2=0x7fcd1c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:07.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcd2158a700 1 --2- 172.21.15.89:0/1305497389 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c139170 0x7fcd1c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcd2158a700 1 -- 172.21.15.89:0/1305497389 >> v1:172.21.15.89:6789/0 conn(0x7fcd1c09d6f0 legacy=0x7fcd1c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcd2158a700 1 -- 172.21.15.89:0/1305497389 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c09a500 con 0x7fcd1c09e1d0 2023-09-30T22:22:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcd2158a700 1 --2- 172.21.15.89:0/1305497389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c09e1d0 0x7fcd1c138c30 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009920 tx=0x7fcd0c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942a0e00d954f2d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcd20d89700 1 -- 172.21.15.89:0/1305497389 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c00ba20 con 0x7fcd1c09e1d0 2023-09-30T22:22:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcd20d89700 1 -- 172.21.15.89:0/1305497389 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c00bb80 con 0x7fcd1c09e1d0 2023-09-30T22:22:07.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcd20d89700 1 -- 172.21.15.89:0/1305497389 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c009e70 con 0x7fcd1c09e1d0 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcec0b41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc09e930 0x7fcebc11c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcec0b41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcebc09f410 0x7fcebc11ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcec0b41700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcebc136fe0 0x7fcebc11cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcec0b41700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc071db0 con 0x7fcebc09f410 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcec0b41700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0a33f0 con 0x7fcebc09e930 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fcec0b41700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc11f820 con 0x7fcebc136fe0 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fceba59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc09e930 0x7fcebc11c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fceba59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc09e930 0x7fcebc11c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35592/0 (socket says 172.21.15.89:35592) 2023-09-30T22:22:07.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.707+0000 7fceba59c700 1 -- 172.21.15.89:0/1831624253 learned_addr learned my addr 172.21.15.89:0/1831624253 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:07.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceba59c700 1 -- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcebc136fe0 msgr2=0x7fcebc11cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcebad9d700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcebc136fe0 0x7fcebc11cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcd23fef700 1 -- 172.21.15.89:0/1305497389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c09e1d0 msgr2=0x7fcd1c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceba59c700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcebc136fe0 0x7fcebc11cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcd23fef700 1 --2- 172.21.15.89:0/1305497389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c09e1d0 0x7fcd1c138c30 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009920 tx=0x7fcd0c009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceba59c700 1 -- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcebc09f410 msgr2=0x7fcebc11ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceba59c700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcebc09f410 0x7fcebc11ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceba59c700 1 -- 172.21.15.89:0/1831624253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb4009b30 con 0x7fcebc09e930 2023-09-30T22:22:07.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcd23fef700 1 -- 172.21.15.89:0/1305497389 shutdown_connections 2023-09-30T22:22:07.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcd23fef700 1 --2- 172.21.15.89:0/1305497389 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c139170 0x7fcd1c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcd23fef700 1 --2- 172.21.15.89:0/1305497389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c09e1d0 0x7fcd1c138c30 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcd23fef700 1 -- 172.21.15.89:0/1305497389 >> 172.21.15.89:0/1305497389 conn(0x7fcd1c066f50 msgr2=0x7fcd1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:07.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcd23fef700 1 -- 172.21.15.89:0/1305497389 shutdown_connections 2023-09-30T22:22:07.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcebad9d700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcebc136fe0 0x7fcebc11cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:07.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceba59c700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc09e930 0x7fcebc11c550 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c510 tx=0x7fceac00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:07.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcd23fef700 1 -- 172.21.15.89:0/1305497389 wait complete. 2023-09-30T22:22:07.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceaaffd700 1 -- 172.21.15.89:0/1831624253 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac0052d0 con 0x7fcebc09e930 2023-09-30T22:22:07.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceaaffd700 1 -- 172.21.15.89:0/1831624253 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac005430 con 0x7fcebc09e930 2023-09-30T22:22:07.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcebc11e9c0 con 0x7fcebc09e930 2023-09-30T22:22:07.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fceaaffd700 1 -- 172.21.15.89:0/1831624253 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac0038c0 con 0x7fcebc09e930 2023-09-30T22:22:07.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.708+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc120800 con 0x7fcebc09e930 2023-09-30T22:22:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.709+0000 7fcd23fef700 1 Processor -- start 2023-09-30T22:22:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.709+0000 7fcd23fef700 1 -- start start 2023-09-30T22:22:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.709+0000 7fceaaffd700 1 -- 172.21.15.89:0/1831624253 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fceac003a20 con 0x7fcebc09e930 2023-09-30T22:22:07.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.709+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcebc11fa30 con 0x7fcebc09e930 2023-09-30T22:22:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd23fef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c09d6f0 0x7fcd1c11f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd23fef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c139170 0x7fcd1c11f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd23fef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c1c8460 0x7fcd1c1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:07.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd23fef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1225e0 con 0x7fcd1c1c8460 2023-09-30T22:22:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd23fef700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c09c710 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd23fef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c121fb0 con 0x7fcd1c139170 2023-09-30T22:22:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c09d6f0 0x7fcd1c11f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c09d6f0 0x7fcd1c11f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35596/0 (socket says 172.21.15.89:35596) 2023-09-30T22:22:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 -- 172.21.15.89:0/3664654419 learned_addr learned my addr 172.21.15.89:0/3664654419 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:07.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fceaaffd700 1 -- 172.21.15.89:0/1831624253 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fceac03ebd0 con 0x7fcebc09e930 2023-09-30T22:22:07.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 -- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c139170 msgr2=0x7fcd1c11f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 --2- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c139170 0x7fcd1c11f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 -- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c1c8460 msgr2=0x7fcd1c1c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 --2- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c1c8460 0x7fcd1c1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 -- 172.21.15.89:0/3664654419 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c009550 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.710+0000 7fcd21d8b700 1 --2- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c09d6f0 0x7fcd1c11f390 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c510 tx=0x7fcd1800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.718+0000 7fcd127fc700 1 -- 172.21.15.89:0/3664654419 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd180052d0 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.718+0000 7fcd127fc700 1 -- 172.21.15.89:0/3664654419 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd18005430 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.718+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c0a2c20 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.718+0000 7fcd127fc700 1 -- 172.21.15.89:0/3664654419 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd180038a0 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.718+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c11f130 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.718+0000 7fceaaffd700 1 -- 172.21.15.89:0/1831624253 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcebc11fa30 con 0x7fcebc09e930 2023-09-30T22:22:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.719+0000 7fcd127fc700 1 -- 172.21.15.89:0/3664654419 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd18003a00 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.719+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c1221d0 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.720+0000 7fcd127fc700 1 -- 172.21.15.89:0/3664654419 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcd1803eb20 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.728+0000 7fcd127fc700 1 -- 172.21.15.89:0/3664654419 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd18003cb0 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.825+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3a781a51d0 con 0x7f3a7809f240 2023-09-30T22:22:07.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.825+0000 7f3a66ffd700 1 -- 172.21.15.89:0/1127660540 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a781a51d0 con 0x7f3a7809f240 2023-09-30T22:22:07.826 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:07.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a7809f240 msgr2=0x7f3a781c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a7809f240 0x7f3a781c4810 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3a7806e6c0 tx=0x7f3a6c00ebf0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 shutdown_connections 2023-09-30T22:22:07.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a7809f650 0x7f3a781c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3a7809e930 0x7f3a781c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 --2- 172.21.15.89:0/1127660540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a7809f240 0x7f3a781c4810 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 >> 172.21.15.89:0/1127660540 conn(0x7f3a780a4210 msgr2=0x7f3a781b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:07.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 shutdown_connections 2023-09-30T22:22:07.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.827+0000 7f3a7c929700 1 -- 172.21.15.89:0/1127660540 wait complete. 2023-09-30T22:22:07.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:07.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.863+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fee94074ea0 con 0x7fee9409e1d0 2023-09-30T22:22:07.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.864+0000 7fee7bfff700 1 -- 172.21.15.89:0/1480215763 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee88029070 con 0x7fee9409e1d0 2023-09-30T22:22:07.865 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 msgr2=0x7fee94139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94139cf0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fee88009870 tx=0x7fee88004750 comp rx=0 tx=0).stop 2023-09-30T22:22:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 shutdown_connections 2023-09-30T22:22:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee94135e50 0x7fee9413a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fee9409d6f0 0x7fee941397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 --2- 172.21.15.89:0/1480215763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee9409e1d0 0x7fee94139cf0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 >> 172.21.15.89:0/1480215763 conn(0x7fee94066f50 msgr2=0x7fee940a2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:07.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 shutdown_connections 2023-09-30T22:22:07.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.866+0000 7fee99b95700 1 -- 172.21.15.89:0/1480215763 wait complete. 2023-09-30T22:22:07.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:07.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.916+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcebc074ea0 con 0x7fcebc09e930 2023-09-30T22:22:07.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.916+0000 7fceaaffd700 1 -- 172.21.15.89:0/1831624253 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fceac017030 con 0x7fcebc09e930 2023-09-30T22:22:07.918 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:07.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.918+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc09e930 msgr2=0x7fcebc11c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.918+0000 7fcec0b41700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc09e930 0x7fcebc11c550 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c510 tx=0x7fceac00c8d0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.919+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 shutdown_connections 2023-09-30T22:22:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.919+0000 7fcec0b41700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcebc136fe0 0x7fcebc11cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.919+0000 7fcec0b41700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcebc09e930 0x7fcebc11c550 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.919+0000 7fcec0b41700 1 --2- 172.21.15.89:0/1831624253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcebc09f410 0x7fcebc11ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.919+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 >> 172.21.15.89:0/1831624253 conn(0x7fcebc0a4210 msgr2=0x7fcebc135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.919+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 shutdown_connections 2023-09-30T22:22:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.919+0000 7fcec0b41700 1 -- 172.21.15.89:0/1831624253 wait complete. 2023-09-30T22:22:07.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.923+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcd1c09f420 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.923+0000 7fcd127fc700 1 -- 172.21.15.89:0/3664654419 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd18017070 con 0x7fcd1c09d6f0 2023-09-30T22:22:07.924 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:07.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c09d6f0 msgr2=0x7fcd1c11f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:07.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 --2- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c09d6f0 0x7fcd1c11f390 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c510 tx=0x7fcd1800c8d0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 shutdown_connections 2023-09-30T22:22:07.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 --2- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c1c8460 0x7fcd1c1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 --2- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c09d6f0 0x7fcd1c11f390 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 --2- 172.21.15.89:0/3664654419 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c139170 0x7fcd1c11f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:07.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 >> 172.21.15.89:0/3664654419 conn(0x7fcd1c066f50 msgr2=0x7fcd1c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:07.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 shutdown_connections 2023-09-30T22:22:07.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:07.925+0000 7fcd23fef700 1 -- 172.21.15.89:0/3664654419 wait complete. 2023-09-30T22:22:07.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:07.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:08.842 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:08.879 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:08.933 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:08.939 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.982+0000 7f868cc17700 1 Processor -- start 2023-09-30T22:22:08.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.982+0000 7f868cc17700 1 -- start start 2023-09-30T22:22:08.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f868cc17700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f868806f5d0 0x7f868812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:08.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f868cc17700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f8688138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:08.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f868cc17700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868806fa40 con 0x7f868812cf60 2023-09-30T22:22:08.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f868cc17700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868812cd90 con 0x7f868806f5d0 2023-09-30T22:22:08.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f868cc17700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8688071780 con 0x7f8688135c20 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f8686d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f8688138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f8686d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f8688138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35612/0 (socket says 172.21.15.89:35612) 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f8686d9d700 1 -- 172.21.15.89:0/1653725950 learned_addr learned my addr 172.21.15.89:0/1653725950 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f8685d9b700 1 --2- 172.21.15.89:0/1653725950 >> v2:172.21.15.89:3300/0 conn(0x7f868806f5d0 0x7f868812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f8686d9d700 1 -- 172.21.15.89:0/1653725950 >> v2:172.21.15.89:3300/0 conn(0x7f868806f5d0 msgr2=0x7f868812ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f8686d9d700 1 --2- 172.21.15.89:0/1653725950 >> v2:172.21.15.89:3300/0 conn(0x7f868806f5d0 0x7f868812ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f8686d9d700 1 -- 172.21.15.89:0/1653725950 >> v1:172.21.15.89:6789/0 conn(0x7f868812cf60 legacy=0x7f868812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.983+0000 7f8686d9d700 1 -- 172.21.15.89:0/1653725950 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688071a40 con 0x7f8688135c20 2023-09-30T22:22:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.984+0000 7f8686d9d700 1 --2- 172.21.15.89:0/1653725950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f8688138000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f868000d150 tx=0x7f868000d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e5e1e091095030e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:08.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.984+0000 7f868559a700 1 -- 172.21.15.89:0/1653725950 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680010070 con 0x7f8688135c20 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.984+0000 7f868559a700 1 -- 172.21.15.89:0/1653725950 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868000dd10 con 0x7f8688135c20 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.984+0000 7f868559a700 1 -- 172.21.15.89:0/1653725950 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000b310 con 0x7f8688135c20 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.985+0000 7f868cc17700 1 -- 172.21.15.89:0/1653725950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 msgr2=0x7f8688138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.985+0000 7f868cc17700 1 --2- 172.21.15.89:0/1653725950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f8688138000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f868000d150 tx=0x7f868000d460 comp rx=0 tx=0).stop 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.985+0000 7f868cc17700 1 -- 172.21.15.89:0/1653725950 shutdown_connections 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.985+0000 7f868cc17700 1 --2- 172.21.15.89:0/1653725950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f8688138000 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.985+0000 7f868cc17700 1 --2- 172.21.15.89:0/1653725950 >> v2:172.21.15.89:3300/0 conn(0x7f868806f5d0 0x7f868812ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.985+0000 7f868cc17700 1 -- 172.21.15.89:0/1653725950 >> 172.21.15.89:0/1653725950 conn(0x7f86880a2aa0 msgr2=0x7f86880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:08.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.985+0000 7f868cc17700 1 -- 172.21.15.89:0/1653725950 shutdown_connections 2023-09-30T22:22:08.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.985+0000 7f868cc17700 1 -- 172.21.15.89:0/1653725950 wait complete. 2023-09-30T22:22:08.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.986+0000 7f868cc17700 1 Processor -- start 2023-09-30T22:22:08.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.986+0000 7f868cc17700 1 -- start start 2023-09-30T22:22:08.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868cc17700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f868806f5d0 0x7f8688129190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868cc17700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f868812cf60 0x7f86881296d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868cc17700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f86881c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868cc17700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868809c920 con 0x7f868812cf60 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868cc17700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868809b2a0 con 0x7f8688135c20 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868cc17700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86880a1620 con 0x7f868806f5d0 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f868806f5d0 0x7f8688129190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f868806f5d0 0x7f8688129190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32986/0 (socket says 172.21.15.89:32986) 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f868659c700 1 -- 172.21.15.89:0/191006699 learned_addr learned my addr 172.21.15.89:0/191006699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f8685d9b700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f868812cf60 0x7f86881296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.987+0000 7f8686d9d700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f86881c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:08.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f868659c700 1 -- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 msgr2=0x7f86881c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f868659c700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f86881c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f868659c700 1 -- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f868812cf60 msgr2=0x7f86881296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f868659c700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f868812cf60 0x7f86881296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f868659c700 1 -- 172.21.15.89:0/191006699 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868000bd40 con 0x7f868806f5d0 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f868659c700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f868806f5d0 0x7f8688129190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f867800c4e0 tx=0x7f867800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f8676ffd700 1 -- 172.21.15.89:0/191006699 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780052d0 con 0x7f868806f5d0 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f8676ffd700 1 -- 172.21.15.89:0/191006699 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8678005430 con 0x7f868806f5d0 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f8676ffd700 1 -- 172.21.15.89:0/191006699 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780038c0 con 0x7f868806f5d0 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868809a060 con 0x7f868806f5d0 2023-09-30T22:22:08.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.988+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8688129c10 con 0x7f868806f5d0 2023-09-30T22:22:08.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.990+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8688099e10 con 0x7f868806f5d0 2023-09-30T22:22:08.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.990+0000 7f8676ffd700 1 -- 172.21.15.89:0/191006699 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8678003a20 con 0x7f868806f5d0 2023-09-30T22:22:08.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.991+0000 7f8676ffd700 1 -- 172.21.15.89:0/191006699 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f867803ea70 con 0x7f868806f5d0 2023-09-30T22:22:08.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:08.996+0000 7f8676ffd700 1 -- 172.21.15.89:0/191006699 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8678040e60 con 0x7f868806f5d0 2023-09-30T22:22:09.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.019+0000 7f2dfc468700 1 Processor -- start 2023-09-30T22:22:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfc468700 1 -- start start 2023-09-30T22:22:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfc468700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409d6f0 0x7f2df409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfc468700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2df409e000 0x7f2df4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfc468700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfc468700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df409f520 con 0x7f2df409e000 2023-09-30T22:22:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfc468700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df409f130 con 0x7f2df409d6f0 2023-09-30T22:22:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfc468700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2df409e930 con 0x7f2df409e410 2023-09-30T22:22:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfaa05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.020+0000 7f2dfa204700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409d6f0 0x7f2df409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2dfaa05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32992/0 (socket says 172.21.15.89:32992) 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2dfaa05700 1 -- 172.21.15.89:0/1352841657 learned_addr learned my addr 172.21.15.89:0/1352841657 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2df9a03700 1 --2- 172.21.15.89:0/1352841657 >> v2:172.21.15.89:3300/0 conn(0x7f2df409e000 0x7f2df4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2dfaa05700 1 -- 172.21.15.89:0/1352841657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409d6f0 msgr2=0x7f2df409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2dfaa05700 1 --2- 172.21.15.89:0/1352841657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409d6f0 0x7f2df409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2dfaa05700 1 -- 172.21.15.89:0/1352841657 >> v2:172.21.15.89:3300/0 conn(0x7f2df409e000 msgr2=0x7f2df4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2dfaa05700 1 --2- 172.21.15.89:0/1352841657 >> v2:172.21.15.89:3300/0 conn(0x7f2df409e000 0x7f2df4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2dfaa05700 1 -- 172.21.15.89:0/1352841657 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df409b2d0 con 0x7f2df409e410 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2df9a03700 1 --2- 172.21.15.89:0/1352841657 >> v2:172.21.15.89:3300/0 conn(0x7f2df409e000 0x7f2df4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2dfaa05700 1 --2- 172.21.15.89:0/1352841657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df4130ba0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009e80 tx=0x7f2dec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62da315cba5647e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:09.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2df9202700 1 -- 172.21.15.89:0/1352841657 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00e070 con 0x7f2df409e410 2023-09-30T22:22:09.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2df9202700 1 -- 172.21.15.89:0/1352841657 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec00b8d0 con 0x7f2df409e410 2023-09-30T22:22:09.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.021+0000 7f2df9202700 1 -- 172.21.15.89:0/1352841657 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00bc80 con 0x7f2df409e410 2023-09-30T22:22:09.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 -- 172.21.15.89:0/1352841657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 msgr2=0x7f2df4130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 --2- 172.21.15.89:0/1352841657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df4130ba0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009e80 tx=0x7f2dec00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 -- 172.21.15.89:0/1352841657 shutdown_connections 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 --2- 172.21.15.89:0/1352841657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df4130ba0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 --2- 172.21.15.89:0/1352841657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409d6f0 0x7f2df409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 --2- 172.21.15.89:0/1352841657 >> v2:172.21.15.89:3300/0 conn(0x7f2df409e000 0x7f2df4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 -- 172.21.15.89:0/1352841657 >> 172.21.15.89:0/1352841657 conn(0x7f2df4066f50 msgr2=0x7f2df409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 -- 172.21.15.89:0/1352841657 shutdown_connections 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.022+0000 7f2dfc468700 1 -- 172.21.15.89:0/1352841657 wait complete. 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.023+0000 7f2dfc468700 1 Processor -- start 2023-09-30T22:22:09.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.023+0000 7f2dfc468700 1 -- start start 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfc468700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df409d6f0 0x7f2df406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfc468700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409e000 0x7f2df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfc468700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df406f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfc468700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4125090 con 0x7f2df409d6f0 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfc468700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4130780 con 0x7f2df409e000 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfc468700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2df409c700 con 0x7f2df409e410 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df406f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df406f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33002/0 (socket says 172.21.15.89:33002) 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 -- 172.21.15.89:0/1299521783 learned_addr learned my addr 172.21.15.89:0/1299521783 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2df9a03700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409e000 0x7f2df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfa204700 1 --2- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df409d6f0 0x7f2df406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 -- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409e000 msgr2=0x7f2df406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 --2- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409e000 0x7f2df406edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 -- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df409d6f0 msgr2=0x7f2df406e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 --2- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df409d6f0 0x7f2df406e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 -- 172.21.15.89:0/1299521783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec009b30 con 0x7f2df409e410 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.024+0000 7f2dfaa05700 1 --2- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df406f2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00b680 tx=0x7f2dec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.025+0000 7f2deaffd700 1 -- 172.21.15.89:0/1299521783 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec004450 con 0x7f2df409e410 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.025+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df4125400 con 0x7f2df409e410 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.025+0000 7f2deaffd700 1 -- 172.21.15.89:0/1299521783 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec0045b0 con 0x7f2df409e410 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.025+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df4124550 con 0x7f2df409e410 2023-09-30T22:22:09.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.025+0000 7f2deaffd700 1 -- 172.21.15.89:0/1299521783 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec004960 con 0x7f2df409e410 2023-09-30T22:22:09.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.026+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc0052e0 con 0x7f2df409e410 2023-09-30T22:22:09.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.027+0000 7f2deaffd700 1 -- 172.21.15.89:0/1299521783 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2dec01c410 con 0x7f2df409e410 2023-09-30T22:22:09.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.027+0000 7f2deaffd700 1 -- 172.21.15.89:0/1299521783 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2dec0418c0 con 0x7f2df409e410 2023-09-30T22:22:09.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.035+0000 7f2deaffd700 1 -- 172.21.15.89:0/1299521783 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2dec01a7d0 con 0x7f2df409e410 2023-09-30T22:22:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.067+0000 7f624e91a700 1 Processor -- start 2023-09-30T22:22:09.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.067+0000 7f624e91a700 1 -- start start 2023-09-30T22:22:09.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f624e91a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f624806f5d0 0x7f624812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f624e91a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f6248138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f624e91a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624806fa40 con 0x7f624812cf60 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f624e91a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624812cd90 con 0x7f624806f5d0 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f624e91a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6248071780 con 0x7f6248135c20 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f62477fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f624806f5d0 0x7f624812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f62477fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f624806f5d0 0x7f624812ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48336/0 (socket says 172.21.15.89:48336) 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f62477fe700 1 -- 172.21.15.89:0/3547696205 learned_addr learned my addr 172.21.15.89:0/3547696205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.068+0000 7f624ceb7700 1 --2- 172.21.15.89:0/3547696205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f6248138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f624ceb7700 1 -- 172.21.15.89:0/3547696205 >> v2:172.21.15.89:3300/0 conn(0x7f624806f5d0 msgr2=0x7f624812ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f624ceb7700 1 --2- 172.21.15.89:0/3547696205 >> v2:172.21.15.89:3300/0 conn(0x7f624806f5d0 0x7f624812ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f624ceb7700 1 -- 172.21.15.89:0/3547696205 >> v1:172.21.15.89:6789/0 conn(0x7f624812cf60 legacy=0x7f624812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f624ceb7700 1 -- 172.21.15.89:0/3547696205 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248071a40 con 0x7f6248135c20 2023-09-30T22:22:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f6246ffd700 1 -- 172.21.15.89:0/3547696205 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71287406 0 0) 0x7f624806fa40 con 0x7f624812cf60 2023-09-30T22:22:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f624ceb7700 1 --2- 172.21.15.89:0/3547696205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f6248138000 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6240009d90 tx=0x7f624000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1ac465463cc450 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f6246ffd700 1 -- 172.21.15.89:0/3547696205 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000e070 con 0x7f6248135c20 2023-09-30T22:22:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f6246ffd700 1 -- 172.21.15.89:0/3547696205 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624000bb20 con 0x7f6248135c20 2023-09-30T22:22:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.069+0000 7f6246ffd700 1 -- 172.21.15.89:0/3547696205 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240012430 con 0x7f6248135c20 2023-09-30T22:22:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.070+0000 7f624e91a700 1 -- 172.21.15.89:0/3547696205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 msgr2=0x7f6248138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.070+0000 7f624e91a700 1 --2- 172.21.15.89:0/3547696205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f6248138000 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6240009d90 tx=0x7f624000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.070+0000 7f624e91a700 1 -- 172.21.15.89:0/3547696205 shutdown_connections 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.070+0000 7f624e91a700 1 --2- 172.21.15.89:0/3547696205 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f6248138000 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.070+0000 7f624e91a700 1 --2- 172.21.15.89:0/3547696205 >> v2:172.21.15.89:3300/0 conn(0x7f624806f5d0 0x7f624812ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.070+0000 7f624e91a700 1 -- 172.21.15.89:0/3547696205 >> 172.21.15.89:0/3547696205 conn(0x7f62480a2aa0 msgr2=0x7f62480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.070+0000 7f624e91a700 1 -- 172.21.15.89:0/3547696205 shutdown_connections 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.070+0000 7f624e91a700 1 -- 172.21.15.89:0/3547696205 wait complete. 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.071+0000 7f624e91a700 1 Processor -- start 2023-09-30T22:22:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.071+0000 7f624e91a700 1 -- start start 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f624e91a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624806f5d0 0x7f6248099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f624e91a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624812cf60 0x7f624809a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f624e91a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f624809a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f624e91a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6248135330 con 0x7f624812cf60 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f624e91a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6248135140 con 0x7f6248135c20 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f624e91a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f624809c460 con 0x7f624806f5d0 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f6247fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624806f5d0 0x7f6248099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f6247fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624806f5d0 0x7f6248099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33014/0 (socket says 172.21.15.89:33014) 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f6247fff700 1 -- 172.21.15.89:0/3163390366 learned_addr learned my addr 172.21.15.89:0/3163390366 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f62477fe700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624812cf60 0x7f624809a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.072+0000 7f624ceb7700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f624809a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f62477fe700 1 -- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624806f5d0 msgr2=0x7f6248099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f62477fe700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624806f5d0 0x7f6248099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f62477fe700 1 -- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 msgr2=0x7f624809a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f62477fe700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f624809a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f62477fe700 1 -- 172.21.15.89:0/3163390366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6240009a70 con 0x7f624812cf60 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f624ceb7700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f624809a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f62477fe700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624812cf60 0x7f624809a290 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f623c00e8e0 tx=0x7f623c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f6244ff9700 1 -- 172.21.15.89:0/3163390366 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c013680 con 0x7f624812cf60 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f6244ff9700 1 -- 172.21.15.89:0/3163390366 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c0137e0 con 0x7f624812cf60 2023-09-30T22:22:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624812a000 con 0x7f624812cf60 2023-09-30T22:22:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f6244ff9700 1 -- 172.21.15.89:0/3163390366 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c013b70 con 0x7f624812cf60 2023-09-30T22:22:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.073+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624809c180 con 0x7f624812cf60 2023-09-30T22:22:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.074+0000 7f6244ff9700 1 -- 172.21.15.89:0/3163390366 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f623c013cd0 con 0x7f624812cf60 2023-09-30T22:22:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.074+0000 7f6244ff9700 1 -- 172.21.15.89:0/3163390366 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f623c03f630 con 0x7f624812cf60 2023-09-30T22:22:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.074+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62280052e0 con 0x7f624812cf60 2023-09-30T22:22:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.083+0000 7f6244ff9700 1 -- 172.21.15.89:0/3163390366 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f623c021020 con 0x7f624812cf60 2023-09-30T22:22:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.117+0000 7f35585cf700 1 Processor -- start 2023-09-30T22:22:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.117+0000 7f35585cf700 1 -- start start 2023-09-30T22:22:09.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f35585cf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009e850 0x7f355009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f35585cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f3550134300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f35585cf700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355006bf30 con 0x7f355009f160 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f35585cf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355009ffb0 con 0x7f355009e850 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f35585cf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f355006cb10 con 0x7f355009f5b0 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f355636b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009e850 0x7f355009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f355636b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009e850 0x7f355009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35668/0 (socket says 172.21.15.89:35668) 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f355636b700 1 -- 172.21.15.89:0/4232001581 learned_addr learned my addr 172.21.15.89:0/4232001581 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f3556b6c700 1 --2- 172.21.15.89:0/4232001581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f3550134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f3556b6c700 1 -- 172.21.15.89:0/4232001581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009e850 msgr2=0x7f355009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f3556b6c700 1 --2- 172.21.15.89:0/4232001581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009e850 0x7f355009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f3556b6c700 1 -- 172.21.15.89:0/4232001581 >> v1:172.21.15.89:6789/0 conn(0x7f355009f160 legacy=0x7f3550133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.118+0000 7f3556b6c700 1 -- 172.21.15.89:0/4232001581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35500a0270 con 0x7f355009f5b0 2023-09-30T22:22:09.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.119+0000 7f3556b6c700 1 --2- 172.21.15.89:0/4232001581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f3550134300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3548009dc0 tx=0x7f354800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b035935c59e7b02d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:09.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.119+0000 7f3555369700 1 -- 172.21.15.89:0/4232001581 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354800e070 con 0x7f355009f5b0 2023-09-30T22:22:09.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.119+0000 7f3555369700 1 -- 172.21.15.89:0/4232001581 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354800bb20 con 0x7f355009f5b0 2023-09-30T22:22:09.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.119+0000 7f3555369700 1 -- 172.21.15.89:0/4232001581 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548012430 con 0x7f355009f5b0 2023-09-30T22:22:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.120+0000 7f35585cf700 1 -- 172.21.15.89:0/4232001581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 msgr2=0x7f3550134300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.120+0000 7f35585cf700 1 --2- 172.21.15.89:0/4232001581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f3550134300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3548009dc0 tx=0x7f354800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.120+0000 7f35585cf700 1 -- 172.21.15.89:0/4232001581 shutdown_connections 2023-09-30T22:22:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.120+0000 7f35585cf700 1 --2- 172.21.15.89:0/4232001581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009e850 0x7f355009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.120+0000 7f35585cf700 1 --2- 172.21.15.89:0/4232001581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f3550134300 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.120+0000 7f35585cf700 1 -- 172.21.15.89:0/4232001581 >> 172.21.15.89:0/4232001581 conn(0x7f35500a3f70 msgr2=0x7f35500a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.120+0000 7f35585cf700 1 -- 172.21.15.89:0/4232001581 shutdown_connections 2023-09-30T22:22:09.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.120+0000 7f35585cf700 1 -- 172.21.15.89:0/4232001581 wait complete. 2023-09-30T22:22:09.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.121+0000 7f35585cf700 1 Processor -- start 2023-09-30T22:22:09.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.121+0000 7f35585cf700 1 -- start start 2023-09-30T22:22:09.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f35585cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355009e850 0x7f3550122720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f35585cf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009f160 0x7f3550122c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f35585cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f35501231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:09.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f35585cf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3550126040 con 0x7f355009e850 2023-09-30T22:22:09.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f35585cf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3550133220 con 0x7f355009f160 2023-09-30T22:22:09.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f35585cf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f355006f810 con 0x7f355009f5b0 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f355636b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355009e850 0x7f3550122720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f3556b6c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f35501231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f355636b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355009e850 0x7f3550122720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48366/0 (socket says 172.21.15.89:48366) 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f355636b700 1 -- 172.21.15.89:0/2258529725 learned_addr learned my addr 172.21.15.89:0/2258529725 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f3555b6a700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009f160 0x7f3550122c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f355636b700 1 -- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 msgr2=0x7f35501231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f355636b700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f35501231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f355636b700 1 -- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009f160 msgr2=0x7f3550122c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f355636b700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009f160 0x7f3550122c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f355636b700 1 -- 172.21.15.89:0/2258529725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009a70 con 0x7f355009e850 2023-09-30T22:22:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.122+0000 7f3556b6c700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f35501231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.123+0000 7f355636b700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355009e850 0x7f3550122720 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f354c00e8b0 tx=0x7f354c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.123+0000 7f3546ffd700 1 -- 172.21.15.89:0/2258529725 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c009600 con 0x7f355009e850 2023-09-30T22:22:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.123+0000 7f3546ffd700 1 -- 172.21.15.89:0/2258529725 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c013680 con 0x7f355009e850 2023-09-30T22:22:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.123+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355006cd90 con 0x7f355009e850 2023-09-30T22:22:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.123+0000 7f3546ffd700 1 -- 172.21.15.89:0/2258529725 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c013a30 con 0x7f355009e850 2023-09-30T22:22:09.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.123+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35501269d0 con 0x7f355009e850 2023-09-30T22:22:09.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.124+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3550125e80 con 0x7f355009e850 2023-09-30T22:22:09.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.124+0000 7f3546ffd700 1 -- 172.21.15.89:0/2258529725 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f354c013b90 con 0x7f355009e850 2023-09-30T22:22:09.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.124+0000 7f3546ffd700 1 -- 172.21.15.89:0/2258529725 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f354c03f9f0 con 0x7f355009e850 2023-09-30T22:22:09.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.133+0000 7f3546ffd700 1 -- 172.21.15.89:0/2258529725 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f354c011db0 con 0x7f355009e850 2023-09-30T22:22:09.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.212+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8688074ea0 con 0x7f868806f5d0 2023-09-30T22:22:09.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.213+0000 7f8676ffd700 1 -- 172.21.15.89:0/191006699 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8678017070 con 0x7f868806f5d0 2023-09-30T22:22:09.213 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:09.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f868806f5d0 msgr2=0x7f8688129190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f868806f5d0 0x7f8688129190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f867800c4e0 tx=0x7f867800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 shutdown_connections 2023-09-30T22:22:09.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8688135c20 0x7f86881c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f868806f5d0 0x7f8688129190 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 --2- 172.21.15.89:0/191006699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f868812cf60 0x7f86881296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 >> 172.21.15.89:0/191006699 conn(0x7f86880a2aa0 msgr2=0x7f868812f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:09.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 shutdown_connections 2023-09-30T22:22:09.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.215+0000 7f868cc17700 1 -- 172.21.15.89:0/191006699 wait complete. 2023-09-30T22:22:09.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:09.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.238+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2ddc005860 con 0x7f2df409e410 2023-09-30T22:22:09.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.238+0000 7f2deaffd700 1 -- 172.21.15.89:0/1299521783 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2dec033090 con 0x7f2df409e410 2023-09-30T22:22:09.239 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:09.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.240+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 msgr2=0x7f2df406f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.240+0000 7f2dfc468700 1 --2- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df406f2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00b680 tx=0x7f2dec004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.240+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 shutdown_connections 2023-09-30T22:22:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.240+0000 7f2dfc468700 1 --2- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2df409e410 0x7f2df406f2f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.240+0000 7f2dfc468700 1 --2- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df409d6f0 0x7f2df406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.240+0000 7f2dfc468700 1 --2- 172.21.15.89:0/1299521783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2df409e000 0x7f2df406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.241+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 >> 172.21.15.89:0/1299521783 conn(0x7f2df4066f50 msgr2=0x7f2df40a25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.241+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 shutdown_connections 2023-09-30T22:22:09.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.241+0000 7f2dfc468700 1 -- 172.21.15.89:0/1299521783 wait complete. 2023-09-30T22:22:09.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.285+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6228005590 con 0x7f624812cf60 2023-09-30T22:22:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.285+0000 7f6244ff9700 1 -- 172.21.15.89:0/3163390366 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f623c017070 con 0x7f624812cf60 2023-09-30T22:22:09.286 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624812cf60 msgr2=0x7f624809a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624812cf60 0x7f624809a290 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f623c00e8e0 tx=0x7f623c00ebf0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 shutdown_connections 2023-09-30T22:22:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248135c20 0x7f624809a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624806f5d0 0x7f6248099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 --2- 172.21.15.89:0/3163390366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624812cf60 0x7f624809a290 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 >> 172.21.15.89:0/3163390366 conn(0x7f62480a2aa0 msgr2=0x7f624812ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 shutdown_connections 2023-09-30T22:22:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.287+0000 7f624e91a700 1 -- 172.21.15.89:0/3163390366 wait complete. 2023-09-30T22:22:09.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:09.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.330+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f35500747f0 con 0x7f355009e850 2023-09-30T22:22:09.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.330+0000 7f3546ffd700 1 -- 172.21.15.89:0/2258529725 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f354c021030 con 0x7f355009e850 2023-09-30T22:22:09.331 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:09.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.332+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355009e850 msgr2=0x7f3550122720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:09.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.332+0000 7f35585cf700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355009e850 0x7f3550122720 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f354c00e8b0 tx=0x7f354c00ebc0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.333+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 shutdown_connections 2023-09-30T22:22:09.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.333+0000 7f35585cf700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f355009f5b0 0x7f35501231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.333+0000 7f35585cf700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355009e850 0x7f3550122720 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.333+0000 7f35585cf700 1 --2- 172.21.15.89:0/2258529725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f355009f160 0x7f3550122c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:09.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.333+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 >> 172.21.15.89:0/2258529725 conn(0x7f35500a3f70 msgr2=0x7f355009d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:09.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.333+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 shutdown_connections 2023-09-30T22:22:09.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:09.333+0000 7f35585cf700 1 -- 172.21.15.89:0/2258529725 wait complete. 2023-09-30T22:22:09.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:10.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:10.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:10.300 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:10.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:10.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.368+0000 7ff8d79f4700 1 Processor -- start 2023-09-30T22:22:10.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.368+0000 7ff8d79f4700 1 -- start start 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d79f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d6f0 0x7ff8d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d79f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009e440 0x7ff8d0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d79f4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d009b120 con 0x7ff8d009e000 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d79f4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d006b460 con 0x7ff8d009d6f0 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d79f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d009a7e0 con 0x7ff8d009e440 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d5f91700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009e440 0x7ff8d0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d4f8f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff8d009e000 0x7ff8d01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36902/0 (socket says 172.21.15.89:36902) 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d4f8f700 1 -- 172.21.15.89:0/1220998954 learned_addr learned my addr 172.21.15.89:0/1220998954 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:10.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d5790700 1 --2- 172.21.15.89:0/1220998954 >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d6f0 0x7ff8d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d5f91700 1 -- 172.21.15.89:0/1220998954 >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d6f0 msgr2=0x7ff8d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:10.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d5f91700 1 --2- 172.21.15.89:0/1220998954 >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d6f0 0x7ff8d009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d5f91700 1 -- 172.21.15.89:0/1220998954 >> v1:172.21.15.89:6789/0 conn(0x7ff8d009e000 legacy=0x7ff8d01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d5f91700 1 -- 172.21.15.89:0/1220998954 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d009a500 con 0x7ff8d009e440 2023-09-30T22:22:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.369+0000 7ff8d5f91700 1 --2- 172.21.15.89:0/1220998954 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009e440 0x7ff8d0134100 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009dc0 tx=0x7ff8c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=718bfd5ef520972d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.370+0000 7ff8c7fff700 1 -- 172.21.15.89:0/1220998954 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800e070 con 0x7ff8d009e440 2023-09-30T22:22:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.370+0000 7ff8c7fff700 1 -- 172.21.15.89:0/1220998954 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c800bae0 con 0x7ff8d009e440 2023-09-30T22:22:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.370+0000 7ff8c7fff700 1 -- 172.21.15.89:0/1220998954 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800be90 con 0x7ff8d009e440 2023-09-30T22:22:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.371+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1220998954 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009e440 msgr2=0x7ff8d0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.371+0000 7ff8d79f4700 1 --2- 172.21.15.89:0/1220998954 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009e440 0x7ff8d0134100 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009dc0 tx=0x7ff8c800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.371+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1220998954 shutdown_connections 2023-09-30T22:22:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.371+0000 7ff8d79f4700 1 --2- 172.21.15.89:0/1220998954 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009e440 0x7ff8d0134100 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.371+0000 7ff8d79f4700 1 --2- 172.21.15.89:0/1220998954 >> v2:172.21.15.89:3300/0 conn(0x7ff8d009d6f0 0x7ff8d009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.371+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1220998954 >> 172.21.15.89:0/1220998954 conn(0x7ff8d0066f50 msgr2=0x7ff8d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.371+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1220998954 shutdown_connections 2023-09-30T22:22:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.371+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1220998954 wait complete. 2023-09-30T22:22:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d79f4700 1 Processor -- start 2023-09-30T22:22:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d79f4700 1 -- start start 2023-09-30T22:22:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d79f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009d6f0 0x7ff8d01cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d79f4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d009e000 0x7ff8d01cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d79f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009e440 0x7ff8d01194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d79f4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d009c710 con 0x7ff8d009e440 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d79f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d011ccf0 con 0x7ff8d009d6f0 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d79f4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d011b350 con 0x7ff8d009e000 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.373+0000 7ff8d4f8f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d009e000 0x7ff8d01cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d4f8f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d009e000 0x7ff8d01cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56644/0 (socket says 172.21.15.89:56644) 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d4f8f700 1 -- 172.21.15.89:0/1080907177 learned_addr learned my addr 172.21.15.89:0/1080907177 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d5790700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009d6f0 0x7ff8d01cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d5f91700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009e440 0x7ff8d01194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d4f8f700 1 -- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009d6f0 msgr2=0x7ff8d01cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d4f8f700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009d6f0 0x7ff8d01cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d4f8f700 1 -- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009e440 msgr2=0x7ff8d01194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d4f8f700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009e440 0x7ff8d01194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d4f8f700 1 -- 172.21.15.89:0/1080907177 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8009a70 con 0x7ff8d009e000 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d5790700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009d6f0 0x7ff8d01cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d5f91700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009e440 0x7ff8d01194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d4f8f700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d009e000 0x7ff8d01cd200 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000c550 tx=0x7ff8c000c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8c5ffb700 1 -- 172.21.15.89:0/1080907177 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0005d70 con 0x7ff8d009e000 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8c5ffb700 1 -- 172.21.15.89:0/1080907177 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c00070d0 con 0x7ff8d009e000 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d00a1f90 con 0x7ff8d009e000 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.374+0000 7ff8c5ffb700 1 -- 172.21.15.89:0/1080907177 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0007440 con 0x7ff8d009e000 2023-09-30T22:22:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.375+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d011cfd0 con 0x7ff8d009e000 2023-09-30T22:22:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.375+0000 7ff8c5ffb700 1 -- 172.21.15.89:0/1080907177 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8c00075a0 con 0x7ff8d009e000 2023-09-30T22:22:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.376+0000 7ff8c5ffb700 1 -- 172.21.15.89:0/1080907177 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff8c003ec80 con 0x7ff8d009e000 2023-09-30T22:22:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.376+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d011ce30 con 0x7ff8d009e000 2023-09-30T22:22:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.381+0000 7ff8c5ffb700 1 -- 172.21.15.89:0/1080907177 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8d011ce30 con 0x7ff8d009e000 2023-09-30T22:22:10.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.424+0000 7f600ea3a700 1 Processor -- start 2023-09-30T22:22:10.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.424+0000 7f600ea3a700 1 -- start start 2023-09-30T22:22:10.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.425+0000 7f600ea3a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60080a0a30 0x7f60080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.425+0000 7f600ea3a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a1340 0x7f6008133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.425+0000 7f600ea3a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600809edd0 con 0x7f6008133fa0 2023-09-30T22:22:10.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.425+0000 7f600ea3a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600809e1f0 con 0x7f60080a0a30 2023-09-30T22:22:10.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.425+0000 7f600ea3a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f600809dc50 con 0x7f60080a1340 2023-09-30T22:22:10.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f60077fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a1340 0x7f6008133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6007fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60080a0a30 0x7f60080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6007fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60080a0a30 0x7f60080a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48692/0 (socket says 172.21.15.89:48692) 2023-09-30T22:22:10.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f60077fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a1340 0x7f6008133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56652/0 (socket says 172.21.15.89:56652) 2023-09-30T22:22:10.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6007fff700 1 -- 172.21.15.89:0/2406749878 learned_addr learned my addr 172.21.15.89:0/2406749878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:10.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6007fff700 1 -- 172.21.15.89:0/2406749878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a1340 msgr2=0x7f6008133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:10.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6007fff700 1 --2- 172.21.15.89:0/2406749878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a1340 0x7f6008133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6007fff700 1 -- 172.21.15.89:0/2406749878 >> v1:172.21.15.89:6789/0 conn(0x7f6008133fa0 legacy=0x7f600813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6006ffd700 1 -- 172.21.15.89:0/2406749878 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690299286 0 0) 0x7f600809edd0 con 0x7f6008133fa0 2023-09-30T22:22:10.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6007fff700 1 -- 172.21.15.89:0/2406749878 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080a1fe0 con 0x7f60080a0a30 2023-09-30T22:22:10.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6007fff700 1 --2- 172.21.15.89:0/2406749878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a0a30 0x7f60080a0e00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80098f0 tx=0x7f5ff8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6aadc6d9c32784d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:10.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6006ffd700 1 -- 172.21.15.89:0/2406749878 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800b8a0 con 0x7f60080a0a30 2023-09-30T22:22:10.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6006ffd700 1 -- 172.21.15.89:0/2406749878 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff800ba00 con 0x7f60080a0a30 2023-09-30T22:22:10.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.426+0000 7f6006ffd700 1 -- 172.21.15.89:0/2406749878 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800bd90 con 0x7f60080a0a30 2023-09-30T22:22:10.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.427+0000 7f600ea3a700 1 -- 172.21.15.89:0/2406749878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a0a30 msgr2=0x7f60080a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.427+0000 7f600ea3a700 1 --2- 172.21.15.89:0/2406749878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a0a30 0x7f60080a0e00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80098f0 tx=0x7f5ff8009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:10.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.427+0000 7f600ea3a700 1 -- 172.21.15.89:0/2406749878 shutdown_connections 2023-09-30T22:22:10.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.427+0000 7f600ea3a700 1 --2- 172.21.15.89:0/2406749878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a0a30 0x7f60080a0e00 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.427+0000 7f600ea3a700 1 --2- 172.21.15.89:0/2406749878 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a1340 0x7f6008133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.427+0000 7f600ea3a700 1 -- 172.21.15.89:0/2406749878 >> 172.21.15.89:0/2406749878 conn(0x7f60080a6550 msgr2=0x7f60080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:10.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.427+0000 7f600ea3a700 1 -- 172.21.15.89:0/2406749878 shutdown_connections 2023-09-30T22:22:10.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.427+0000 7f600ea3a700 1 -- 172.21.15.89:0/2406749878 wait complete. 2023-09-30T22:22:10.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.428+0000 7f600ea3a700 1 Processor -- start 2023-09-30T22:22:10.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.428+0000 7f600ea3a700 1 -- start start 2023-09-30T22:22:10.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f600ea3a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a0a30 0x7f600811e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f600ea3a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a1340 0x7f600811ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f600ea3a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6008133fa0 0x7f600811f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f600ea3a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008121820 con 0x7f60080a1340 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f600ea3a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60080a0220 con 0x7f6008133fa0 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f600ea3a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f600811fdb0 con 0x7f60080a0a30 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f60077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a1340 0x7f600811ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f60077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a1340 0x7f600811ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48708/0 (socket says 172.21.15.89:48708) 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f60077fe700 1 -- 172.21.15.89:0/1545779476 learned_addr learned my addr 172.21.15.89:0/1545779476 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f6007fff700 1 --2- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a0a30 0x7f600811e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f60077fe700 1 -- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a0a30 msgr2=0x7f600811e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f60077fe700 1 --2- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a0a30 0x7f600811e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f60077fe700 1 -- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6008133fa0 msgr2=0x7f600811f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f60077fe700 1 --2- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6008133fa0 0x7f600811f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.429+0000 7f60077fe700 1 -- 172.21.15.89:0/1545779476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8009550 con 0x7f60080a1340 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.430+0000 7f60077fe700 1 --2- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a1340 0x7f600811ee90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00e840 tx=0x7f5ffc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.430+0000 7f6004ff9700 1 -- 172.21.15.89:0/1545779476 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc013610 con 0x7f60080a1340 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.430+0000 7f6004ff9700 1 -- 172.21.15.89:0/1545779476 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc013770 con 0x7f60080a1340 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.430+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6008122470 con 0x7f60080a1340 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.430+0000 7f6004ff9700 1 -- 172.21.15.89:0/1545779476 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc013b20 con 0x7f60080a1340 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.430+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6008122190 con 0x7f60080a1340 2023-09-30T22:22:10.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.431+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60080a1d00 con 0x7f60080a1340 2023-09-30T22:22:10.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.431+0000 7f6004ff9700 1 -- 172.21.15.89:0/1545779476 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ffc013c80 con 0x7f60080a1340 2023-09-30T22:22:10.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.431+0000 7f6004ff9700 1 -- 172.21.15.89:0/1545779476 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ffc03f4d0 con 0x7f60080a1340 2023-09-30T22:22:10.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.436+0000 7f6004ff9700 1 -- 172.21.15.89:0/1545779476 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ffc012370 con 0x7f60080a1340 2023-09-30T22:22:10.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.459+0000 7f1f693e4700 1 Processor -- start 2023-09-30T22:22:10.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.459+0000 7f1f693e4700 1 -- start start 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f693e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f64133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f693e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f6409de50 0x7f1f6409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f693e4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6406ef30 con 0x7f1f6409e760 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f693e4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6406fef0 con 0x7f1f6409de50 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f693e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641334b0 con 0x7f1f641338a0 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f637fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1f6409e760 0x7f1f6413a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36918/0 (socket says 172.21.15.89:36918) 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f637fe700 1 -- 172.21.15.89:0/3427317877 learned_addr learned my addr 172.21.15.89:0/3427317877 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f62ffd700 1 --2- 172.21.15.89:0/3427317877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f64133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f627fc700 1 --2- 172.21.15.89:0/3427317877 >> v2:172.21.15.89:3300/0 conn(0x7f1f6409de50 0x7f1f6409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f62ffd700 1 -- 172.21.15.89:0/3427317877 >> v2:172.21.15.89:3300/0 conn(0x7f1f6409de50 msgr2=0x7f1f6409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f62ffd700 1 --2- 172.21.15.89:0/3427317877 >> v2:172.21.15.89:3300/0 conn(0x7f1f6409de50 0x7f1f6409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f62ffd700 1 -- 172.21.15.89:0/3427317877 >> v1:172.21.15.89:6789/0 conn(0x7f1f6409e760 legacy=0x7f1f6413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f62ffd700 1 -- 172.21.15.89:0/3427317877 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6406fc30 con 0x7f1f641338a0 2023-09-30T22:22:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.460+0000 7f1f62ffd700 1 --2- 172.21.15.89:0/3427317877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f64133c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009840 tx=0x7f1f54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f5b269e9533ecc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f61ffb700 1 -- 172.21.15.89:0/3427317877 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54011830 con 0x7f1f641338a0 2023-09-30T22:22:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f61ffb700 1 -- 172.21.15.89:0/3427317877 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f54011990 con 0x7f1f641338a0 2023-09-30T22:22:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f61ffb700 1 -- 172.21.15.89:0/3427317877 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54011d20 con 0x7f1f641338a0 2023-09-30T22:22:10.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f693e4700 1 -- 172.21.15.89:0/3427317877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 msgr2=0x7f1f64133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f693e4700 1 --2- 172.21.15.89:0/3427317877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f64133c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009840 tx=0x7f1f54009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f693e4700 1 -- 172.21.15.89:0/3427317877 shutdown_connections 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f693e4700 1 --2- 172.21.15.89:0/3427317877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f64133c70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f693e4700 1 --2- 172.21.15.89:0/3427317877 >> v2:172.21.15.89:3300/0 conn(0x7f1f6409de50 0x7f1f6409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f693e4700 1 -- 172.21.15.89:0/3427317877 >> 172.21.15.89:0/3427317877 conn(0x7f1f640a2aa0 msgr2=0x7f1f640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f693e4700 1 -- 172.21.15.89:0/3427317877 shutdown_connections 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.461+0000 7f1f693e4700 1 -- 172.21.15.89:0/3427317877 wait complete. 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f693e4700 1 Processor -- start 2023-09-30T22:22:10.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f693e4700 1 -- start start 2023-09-30T22:22:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f693e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6409de50 0x7f1f6411d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f693e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409e760 0x7f1f6411dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f693e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f6411e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f693e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6411f300 con 0x7f1f6409de50 2023-09-30T22:22:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f693e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6411f920 con 0x7f1f641338a0 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f693e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64120550 con 0x7f1f6409e760 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f6411e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f62ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6409de50 0x7f1f6411d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f6411e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33790/0 (socket says 172.21.15.89:33790) 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f62ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6409de50 0x7f1f6411d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48714/0 (socket says 172.21.15.89:48714) 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f637fe700 1 -- 172.21.15.89:0/2863154326 learned_addr learned my addr 172.21.15.89:0/2863154326 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.463+0000 7f1f627fc700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409e760 0x7f1f6411dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f637fe700 1 -- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409e760 msgr2=0x7f1f6411dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f637fe700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409e760 0x7f1f6411dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f637fe700 1 -- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6409de50 msgr2=0x7f1f6411d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f637fe700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6409de50 0x7f1f6411d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f637fe700 1 -- 172.21.15.89:0/2863154326 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f540094a0 con 0x7f1f641338a0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f62ffd700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6409de50 0x7f1f6411d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f637fe700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f6411e170 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00cbf0 tx=0x7f1f5c00cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f4b7fe700 1 -- 172.21.15.89:0/2863154326 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00aa60 con 0x7f1f641338a0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f4b7fe700 1 -- 172.21.15.89:0/2863154326 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c00abc0 con 0x7f1f641338a0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f4b7fe700 1 -- 172.21.15.89:0/2863154326 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c012400 con 0x7f1f641338a0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f64120f30 con 0x7f1f641338a0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.464+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f64120a00 con 0x7f1f641338a0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.465+0000 7f1f4b7fe700 1 -- 172.21.15.89:0/2863154326 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f5c012560 con 0x7f1f641338a0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.465+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6411a470 con 0x7f1f641338a0 2023-09-30T22:22:10.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.466+0000 7f1f4b7fe700 1 -- 172.21.15.89:0/2863154326 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1f5c012810 con 0x7f1f641338a0 2023-09-30T22:22:10.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.470+0000 7f1f4b7fe700 1 -- 172.21.15.89:0/2863154326 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f5c017020 con 0x7f1f641338a0 2023-09-30T22:22:10.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.493+0000 7f1767513700 1 Processor -- start 2023-09-30T22:22:10.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.493+0000 7f1767513700 1 -- start start 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1767513700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f17600a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1767513700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1760138d80 0x7f176013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1767513700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17600a2a70 con 0x7f17600a0a30 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1767513700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17600a01d0 con 0x7f1760138d80 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1767513700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f176009e2b0 con 0x7f176009d630 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1764aae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f17600a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f17652af700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f17600a0a30 0x7f17600a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36922/0 (socket says 172.21.15.89:36922) 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f17652af700 1 -- 172.21.15.89:0/4930256 learned_addr learned my addr 172.21.15.89:0/4930256 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:10.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213685178 0 0) 0x7f17600a2a70 con 0x7f17600a0a30 2023-09-30T22:22:10.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f174c0035b0 con 0x7f17600a0a30 2023-09-30T22:22:10.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1765ab0700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1760138d80 0x7f176013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 802222704 0 0) 0x7f174c0035b0 con 0x7f17600a0a30 2023-09-30T22:22:10.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 msgr2=0x7f17600a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:10.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.494+0000 7f1757fff700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f17600a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1760138d80 msgr2=0x7f176013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:10.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1760138d80 0x7f176013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176009dc50 con 0x7f17600a0a30 2023-09-30T22:22:10.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7f175c001670 con 0x7f17600a0a30 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 >> v1:172.21.15.89:6789/0 conn(0x7f17600a0a30 legacy=0x7f17600a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c003a00 0x7f174c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f174c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c004d90 0x7f174c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17600a01d0 con 0x7f174c004d90 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176009e2b0 con 0x7f174c003a00 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f174c005a10 con 0x7f176009d630 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1765ab0700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c004d90 0x7f174c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f17652af700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c003a00 0x7f174c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1764aae700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f174c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f17652af700 1 -- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 msgr2=0x7f174c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f17652af700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f174c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f17652af700 1 -- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c004d90 msgr2=0x7f174c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f17652af700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c004d90 0x7f174c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f17652af700 1 -- 172.21.15.89:0/4930256 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176009dc50 con 0x7f174c003a00 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1765ab0700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c004d90 0x7f174c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f17652af700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c003a00 0x7f174c003e10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f175c00c4e0 tx=0x7f174c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5463bbd66df46caa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c005960 con 0x7f174c003a00 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c005ac0 con 0x7f174c003a00 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.495+0000 7f1757fff700 1 -- 172.21.15.89:0/4930256 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c005e50 con 0x7f174c003a00 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 -- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c003a00 msgr2=0x7f174c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c003a00 0x7f174c003e10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f175c00c4e0 tx=0x7f174c005880 comp rx=0 tx=0).stop 2023-09-30T22:22:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 -- 172.21.15.89:0/4930256 shutdown_connections 2023-09-30T22:22:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c004d90 0x7f174c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c003a00 0x7f174c003e10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 --2- 172.21.15.89:0/4930256 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f174c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 -- 172.21.15.89:0/4930256 >> 172.21.15.89:0/4930256 conn(0x7f17600a3050 msgr2=0x7f17600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 -- 172.21.15.89:0/4930256 shutdown_connections 2023-09-30T22:22:10.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.496+0000 7f1767513700 1 -- 172.21.15.89:0/4930256 wait complete. 2023-09-30T22:22:10.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.497+0000 7f1767513700 1 Processor -- start 2023-09-30T22:22:10.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.497+0000 7f1767513700 1 -- start start 2023-09-30T22:22:10.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f1767513700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c003a00 0x7f176011e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f1767513700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c004d90 0x7f176011ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f1767513700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f176011f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:10.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f1767513700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1760120b50 con 0x7f174c003a00 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f1767513700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1760122200 con 0x7f174c004d90 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f1767513700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17600a0410 con 0x7f176009d630 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f17652af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c003a00 0x7f176011e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f17652af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c003a00 0x7f176011e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48726/0 (socket says 172.21.15.89:48726) 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f17652af700 1 -- 172.21.15.89:0/3792748376 learned_addr learned my addr 172.21.15.89:0/3792748376 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f1764aae700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c004d90 0x7f176011ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.498+0000 7f1765ab0700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f176011f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f17652af700 1 -- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 msgr2=0x7f176011f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f17652af700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f176011f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f17652af700 1 -- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c004d90 msgr2=0x7f176011ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f17652af700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c004d90 0x7f176011ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f17652af700 1 -- 172.21.15.89:0/3792748376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17600a4a10 con 0x7f174c003a00 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f17652af700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c003a00 0x7f176011e770 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f175c000c00 tx=0x7f175c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f1755ffb700 1 -- 172.21.15.89:0/3792748376 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c004a80 con 0x7f174c003a00 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f1755ffb700 1 -- 172.21.15.89:0/3792748376 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c004be0 con 0x7f174c003a00 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f1767513700 1 -- 172.21.15.89:0/3792748376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1760121c60 con 0x7f174c003a00 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f1755ffb700 1 -- 172.21.15.89:0/3792748376 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0070d0 con 0x7f174c003a00 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.499+0000 7f1767513700 1 -- 172.21.15.89:0/3792748376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1760121990 con 0x7f174c003a00 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.500+0000 7f1755ffb700 1 -- 172.21.15.89:0/3792748376 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f175c019070 con 0x7f174c003a00 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.500+0000 7f174b7fe700 1 -- 172.21.15.89:0/3792748376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17380052e0 con 0x7f174c003a00 2023-09-30T22:22:10.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.500+0000 7f1755ffb700 1 -- 172.21.15.89:0/3792748376 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f175c040230 con 0x7f174c003a00 2023-09-30T22:22:10.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.505+0000 7f1755ffb700 1 -- 172.21.15.89:0/3792748376 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f175c01b370 con 0x7f174c003a00 2023-09-30T22:22:10.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.602+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff8d011ce30 con 0x7ff8d009e000 2023-09-30T22:22:10.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.602+0000 7ff8c5ffb700 1 -- 172.21.15.89:0/1080907177 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8d011ce30 con 0x7ff8d009e000 2023-09-30T22:22:10.603 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:10.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d009e000 msgr2=0x7ff8d01cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d009e000 0x7ff8d01cd200 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000c550 tx=0x7ff8c000c910 comp rx=0 tx=0).stop 2023-09-30T22:22:10.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 shutdown_connections 2023-09-30T22:22:10.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d009e440 0x7ff8d01194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8d009d6f0 0x7ff8d01cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 --2- 172.21.15.89:0/1080907177 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8d009e000 0x7ff8d01cd200 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 >> 172.21.15.89:0/1080907177 conn(0x7ff8d0066f50 msgr2=0x7ff8d01badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:10.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 shutdown_connections 2023-09-30T22:22:10.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.604+0000 7ff8d79f4700 1 -- 172.21.15.89:0/1080907177 wait complete. 2023-09-30T22:22:10.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:10.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.672+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6008074ea0 con 0x7f60080a1340 2023-09-30T22:22:10.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.672+0000 7f6004ff9700 1 -- 172.21.15.89:0/1545779476 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ffc010310 con 0x7f60080a1340 2023-09-30T22:22:10.673 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:10.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a1340 msgr2=0x7f600811ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 --2- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a1340 0x7f600811ee90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00e840 tx=0x7f5ffc00eb50 comp rx=0 tx=0).stop 2023-09-30T22:22:10.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 shutdown_connections 2023-09-30T22:22:10.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 --2- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6008133fa0 0x7f600811f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 --2- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60080a0a30 0x7f600811e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 --2- 172.21.15.89:0/1545779476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60080a1340 0x7f600811ee90 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 >> 172.21.15.89:0/1545779476 conn(0x7f60080a6550 msgr2=0x7f600809a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:10.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 shutdown_connections 2023-09-30T22:22:10.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.674+0000 7f600ea3a700 1 -- 172.21.15.89:0/1545779476 wait complete. 2023-09-30T22:22:10.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.700+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1f64074ea0 con 0x7f1f641338a0 2023-09-30T22:22:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.701+0000 7f1f4b7fe700 1 -- 172.21.15.89:0/2863154326 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f5c032090 con 0x7f1f641338a0 2023-09-30T22:22:10.702 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.701+0000 7f174b7fe700 1 -- 172.21.15.89:0/3792748376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1738005590 con 0x7f174c003a00 2023-09-30T22:22:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.701+0000 7f1755ffb700 1 -- 172.21.15.89:0/3792748376 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f175c017020 con 0x7f174c003a00 2023-09-30T22:22:10.703 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:10.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 msgr2=0x7f1f6411e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f6411e170 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00cbf0 tx=0x7f1f5c00cf00 comp rx=0 tx=0).stop 2023-09-30T22:22:10.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 shutdown_connections 2023-09-30T22:22:10.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f641338a0 0x7f1f6411e170 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6409de50 0x7f1f6411d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 --2- 172.21.15.89:0/2863154326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6409e760 0x7f1f6411dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 >> 172.21.15.89:0/2863154326 conn(0x7f1f640a2aa0 msgr2=0x7f1f641bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 shutdown_connections 2023-09-30T22:22:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f1f693e4700 1 -- 172.21.15.89:0/2863154326 wait complete. 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 -- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c003a00 msgr2=0x7f176011e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c003a00 0x7f176011e770 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f175c000c00 tx=0x7f175c000fe0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 -- 172.21.15.89:0/3792748376 shutdown_connections 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f176009d630 0x7f176011f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f174c003a00 0x7f176011e770 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 --2- 172.21.15.89:0/3792748376 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f174c004d90 0x7f176011ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 -- 172.21.15.89:0/3792748376 >> 172.21.15.89:0/3792748376 conn(0x7f17600a3050 msgr2=0x7f176009ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 -- 172.21.15.89:0/3792748376 shutdown_connections 2023-09-30T22:22:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:10.703+0000 7f174b7fe700 1 -- 172.21.15.89:0/3792748376 wait complete. 2023-09-30T22:22:10.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:10.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:11.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:11.688 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:11.718 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:11.719 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.763+0000 7fe8d8bb5700 1 Processor -- start 2023-09-30T22:22:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.763+0000 7fe8d8bb5700 1 -- start start 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d8bb5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe8d406e570 0x7fe8d406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d8bb5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d406ee80 0x7fe8d4134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d8bb5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d4070ee0 con 0x7fe8d409eca0 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d8bb5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d4071480 con 0x7fe8d406e570 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d8bb5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4071b90 con 0x7fe8d406ee80 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe8d406e570 0x7fe8d406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe8d406e570 0x7fe8d406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48732/0 (socket says 172.21.15.89:48732) 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d259c700 1 -- 172.21.15.89:0/1135080921 learned_addr learned my addr 172.21.15.89:0/1135080921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d2d9d700 1 --2- 172.21.15.89:0/1135080921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d406ee80 0x7fe8d4134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d2d9d700 1 -- 172.21.15.89:0/1135080921 >> v2:172.21.15.89:3300/0 conn(0x7fe8d406e570 msgr2=0x7fe8d406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:11.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d2d9d700 1 --2- 172.21.15.89:0/1135080921 >> v2:172.21.15.89:3300/0 conn(0x7fe8d406e570 0x7fe8d406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d2d9d700 1 -- 172.21.15.89:0/1135080921 >> v1:172.21.15.89:6789/0 conn(0x7fe8d409eca0 legacy=0x7fe8d4134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d2d9d700 1 -- 172.21.15.89:0/1135080921 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40711a0 con 0x7fe8d406ee80 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.764+0000 7fe8d2d9d700 1 --2- 172.21.15.89:0/1135080921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d406ee80 0x7fe8d4134d20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009d90 tx=0x7fe8cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae2e9e1c3496048 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.765+0000 7fe8d159a700 1 -- 172.21.15.89:0/1135080921 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00e070 con 0x7fe8d406ee80 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.765+0000 7fe8d159a700 1 -- 172.21.15.89:0/1135080921 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc00bb20 con 0x7fe8d406ee80 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.765+0000 7fe8d159a700 1 -- 172.21.15.89:0/1135080921 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc012430 con 0x7fe8d406ee80 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.765+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/1135080921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d406ee80 msgr2=0x7fe8d4134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.765+0000 7fe8d8bb5700 1 --2- 172.21.15.89:0/1135080921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d406ee80 0x7fe8d4134d20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009d90 tx=0x7fe8cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.765+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/1135080921 shutdown_connections 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.765+0000 7fe8d8bb5700 1 --2- 172.21.15.89:0/1135080921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d406ee80 0x7fe8d4134d20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.766+0000 7fe8d8bb5700 1 --2- 172.21.15.89:0/1135080921 >> v2:172.21.15.89:3300/0 conn(0x7fe8d406e570 0x7fe8d406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.766+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/1135080921 >> 172.21.15.89:0/1135080921 conn(0x7fe8d40a2aa0 msgr2=0x7fe8d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.766+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/1135080921 shutdown_connections 2023-09-30T22:22:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.766+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/1135080921 wait complete. 2023-09-30T22:22:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d8bb5700 1 Processor -- start 2023-09-30T22:22:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d8bb5700 1 -- start start 2023-09-30T22:22:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d8bb5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d406e570 0x7fe8d4134880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d8bb5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d406ee80 0x7fe8d41334a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d8bb5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4134dc0 0x7fe8d41339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d8bb5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d413c1f0 con 0x7fe8d406e570 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d8bb5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d413b5c0 con 0x7fe8d4134dc0 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d8bb5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4135700 con 0x7fe8d406ee80 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d406e570 0x7fe8d4134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4134dc0 0x7fe8d41339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4134dc0 0x7fe8d41339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33820/0 (socket says 172.21.15.89:33820) 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d406e570 0x7fe8d4134880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48740/0 (socket says 172.21.15.89:48740) 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.767+0000 7fe8d2d9d700 1 -- 172.21.15.89:0/2591520708 learned_addr learned my addr 172.21.15.89:0/2591520708 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d2d9d700 1 -- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d406ee80 msgr2=0x7fe8d41334a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d2d9d700 1 --2- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d406ee80 0x7fe8d41334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d2d9d700 1 -- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d406e570 msgr2=0x7fe8d4134880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d2d9d700 1 --2- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d406e570 0x7fe8d4134880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d2d9d700 1 -- 172.21.15.89:0/2591520708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc009a70 con 0x7fe8d4134dc0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d259c700 1 --2- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d406e570 0x7fe8d4134880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d2d9d700 1 --2- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4134dc0 0x7fe8d41339e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc014040 tx=0x7fe8cc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8c2ffd700 1 -- 172.21.15.89:0/2591520708 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc012430 con 0x7fe8d4134dc0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8c2ffd700 1 -- 172.21.15.89:0/2591520708 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc020070 con 0x7fe8d4134dc0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8c2ffd700 1 -- 172.21.15.89:0/2591520708 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc003ea0 con 0x7fe8d4134dc0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d4070610 con 0x7fe8d4134dc0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.768+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d409e130 con 0x7fe8d4134dc0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.770+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d4135240 con 0x7fe8d4134dc0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.770+0000 7fe8c2ffd700 1 -- 172.21.15.89:0/2591520708 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe8cc004280 con 0x7fe8d4134dc0 2023-09-30T22:22:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.770+0000 7fe8c2ffd700 1 -- 172.21.15.89:0/2591520708 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe8cc022070 con 0x7fe8d4134dc0 2023-09-30T22:22:11.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.778+0000 7fe8c2ffd700 1 -- 172.21.15.89:0/2591520708 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe8cc011980 con 0x7fe8d4134dc0 2023-09-30T22:22:11.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.830+0000 7faf409d4700 1 Processor -- start 2023-09-30T22:22:11.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.830+0000 7faf409d4700 1 -- start start 2023-09-30T22:22:11.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.831+0000 7faf409d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09e930 0x7faf3c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.831+0000 7faf409d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09f650 0x7faf3c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.831+0000 7faf409d4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c070940 con 0x7faf3c09f240 2023-09-30T22:22:11.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.831+0000 7faf409d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c06fd60 con 0x7faf3c09e930 2023-09-30T22:22:11.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.831+0000 7faf409d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c06fa80 con 0x7faf3c09f650 2023-09-30T22:22:11.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.831+0000 7faf3a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09e930 0x7faf3c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3a59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09e930 0x7faf3c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33836/0 (socket says 172.21.15.89:33836) 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3a59c700 1 -- 172.21.15.89:0/4276425500 learned_addr learned my addr 172.21.15.89:0/4276425500 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3ad9d700 1 --2- 172.21.15.89:0/4276425500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09f650 0x7faf3c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3a59c700 1 -- 172.21.15.89:0/4276425500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09f650 msgr2=0x7faf3c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3a59c700 1 --2- 172.21.15.89:0/4276425500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09f650 0x7faf3c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3a59c700 1 -- 172.21.15.89:0/4276425500 >> v1:172.21.15.89:6789/0 conn(0x7faf3c09f240 legacy=0x7faf3c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3a59c700 1 -- 172.21.15.89:0/4276425500 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c09fb60 con 0x7faf3c09e930 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3a59c700 1 --2- 172.21.15.89:0/4276425500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09e930 0x7faf3c09ed00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009920 tx=0x7faf2c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6937f7568dc076ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3959a700 1 -- 172.21.15.89:0/4276425500 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00ba60 con 0x7faf3c09e930 2023-09-30T22:22:11.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3959a700 1 -- 172.21.15.89:0/4276425500 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c00bbc0 con 0x7faf3c09e930 2023-09-30T22:22:11.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.832+0000 7faf3959a700 1 -- 172.21.15.89:0/4276425500 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c009e70 con 0x7faf3c09e930 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.833+0000 7faf409d4700 1 -- 172.21.15.89:0/4276425500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09e930 msgr2=0x7faf3c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.833+0000 7faf409d4700 1 --2- 172.21.15.89:0/4276425500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09e930 0x7faf3c09ed00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009920 tx=0x7faf2c009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.833+0000 7faf409d4700 1 -- 172.21.15.89:0/4276425500 shutdown_connections 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.833+0000 7faf409d4700 1 --2- 172.21.15.89:0/4276425500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09e930 0x7faf3c09ed00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.833+0000 7faf409d4700 1 --2- 172.21.15.89:0/4276425500 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09f650 0x7faf3c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.833+0000 7faf409d4700 1 -- 172.21.15.89:0/4276425500 >> 172.21.15.89:0/4276425500 conn(0x7faf3c0a4210 msgr2=0x7faf3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.833+0000 7faf409d4700 1 -- 172.21.15.89:0/4276425500 shutdown_connections 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.833+0000 7faf409d4700 1 -- 172.21.15.89:0/4276425500 wait complete. 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf409d4700 1 Processor -- start 2023-09-30T22:22:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf409d4700 1 -- start start 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf409d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09e930 0x7faf3c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf409d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09f240 0x7faf3c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf409d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c09f650 0x7faf3c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf409d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1a9760 con 0x7faf3c09f650 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf409d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c06d5f0 con 0x7faf3c09f240 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf409d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1b6e00 con 0x7faf3c09e930 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.835+0000 7faf3a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09e930 0x7faf3c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09e930 0x7faf3c06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56704/0 (socket says 172.21.15.89:56704) 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3a59c700 1 -- 172.21.15.89:0/3705941504 learned_addr learned my addr 172.21.15.89:0/3705941504 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3ad9d700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c09f650 0x7faf3c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf39d9b700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09f240 0x7faf3c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3a59c700 1 -- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09f240 msgr2=0x7faf3c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3a59c700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09f240 0x7faf3c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3a59c700 1 -- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c09f650 msgr2=0x7faf3c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3a59c700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c09f650 0x7faf3c06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3a59c700 1 -- 172.21.15.89:0/3705941504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c009550 con 0x7faf3c09e930 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3ad9d700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c09f650 0x7faf3c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf3a59c700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09e930 0x7faf3c06b8b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0098f0 tx=0x7faf2c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.836+0000 7faf2affd700 1 -- 172.21.15.89:0/3705941504 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c004510 con 0x7faf3c09e930 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.837+0000 7faf2affd700 1 -- 172.21.15.89:0/3705941504 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c0036a0 con 0x7faf3c09e930 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.837+0000 7faf2affd700 1 -- 172.21.15.89:0/3705941504 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c003a50 con 0x7faf3c09e930 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.837+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c0a0190 con 0x7faf3c09e930 2023-09-30T22:22:11.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.837+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c0a3770 con 0x7faf3c09e930 2023-09-30T22:22:11.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.838+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c099e40 con 0x7faf3c09e930 2023-09-30T22:22:11.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.838+0000 7faf2affd700 1 -- 172.21.15.89:0/3705941504 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf2c021070 con 0x7faf3c09e930 2023-09-30T22:22:11.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.839+0000 7faf2affd700 1 -- 172.21.15.89:0/3705941504 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faf2c040e40 con 0x7faf3c09e930 2023-09-30T22:22:11.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.847+0000 7faf2affd700 1 -- 172.21.15.89:0/3705941504 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf2c019590 con 0x7faf3c09e930 2023-09-30T22:22:11.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.875+0000 7f3551c05700 1 Processor -- start 2023-09-30T22:22:11.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.875+0000 7f3551c05700 1 -- start start 2023-09-30T22:22:11.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f3551c05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f3551c05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 0x7f354c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f3551c05700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f354c09e410 0x7f354c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f3551c05700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c09f520 con 0x7f354c09e410 2023-09-30T22:22:11.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f3551c05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c09f130 con 0x7f354c09d6f0 2023-09-30T22:22:11.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f3551c05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f354c09e930 con 0x7f354c09e000 2023-09-30T22:22:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33848/0 (socket says 172.21.15.89:33848) 2023-09-30T22:22:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354b7fe700 1 -- 172.21.15.89:0/845080677 learned_addr learned my addr 172.21.15.89:0/845080677 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354bfff700 1 --2- 172.21.15.89:0/845080677 >> v2:172.21.15.89:3300/0 conn(0x7f354c09e410 0x7f354c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354affd700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 0x7f354c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354bfff700 1 -- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 msgr2=0x7f354c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354bfff700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 0x7f354c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354bfff700 1 -- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 msgr2=0x7f354c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354bfff700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354bfff700 1 -- 172.21.15.89:0/845080677 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c09b2d0 con 0x7f354c09e410 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354b7fe700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.876+0000 7f354bfff700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c130ba0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f3544009d90 tx=0x7f354400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f10c573afe3e176 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f354a7fc700 1 -- 172.21.15.89:0/845080677 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400e070 con 0x7f354c09e410 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f354a7fc700 1 -- 172.21.15.89:0/845080677 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354400bb20 con 0x7f354c09e410 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f354a7fc700 1 -- 172.21.15.89:0/845080677 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544012430 con 0x7f354c09e410 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 -- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 msgr2=0x7f354c130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c130ba0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f3544009d90 tx=0x7f354400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 -- 172.21.15.89:0/845080677 shutdown_connections 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c130ba0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 --2- 172.21.15.89:0/845080677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 0x7f354c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 -- 172.21.15.89:0/845080677 >> 172.21.15.89:0/845080677 conn(0x7f354c066f50 msgr2=0x7f354c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 -- 172.21.15.89:0/845080677 shutdown_connections 2023-09-30T22:22:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.877+0000 7f3551c05700 1 -- 172.21.15.89:0/845080677 wait complete. 2023-09-30T22:22:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.878+0000 7f3551c05700 1 Processor -- start 2023-09-30T22:22:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.878+0000 7f3551c05700 1 -- start start 2023-09-30T22:22:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f852d4ce700 1 Processor -- start 2023-09-30T22:22:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f852d4ce700 1 -- start start 2023-09-30T22:22:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f3551c05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c1224d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f3551c05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 0x7f354c122a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f3551c05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c122f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f3551c05700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c125f50 con 0x7f354c09e410 2023-09-30T22:22:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f3551c05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c124040 con 0x7f354c09d6f0 2023-09-30T22:22:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f3551c05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0a3a60 con 0x7f354c09e000 2023-09-30T22:22:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f354bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c122f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f354bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c122f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48760/0 (socket says 172.21.15.89:48760) 2023-09-30T22:22:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f354bfff700 1 -- 172.21.15.89:0/3633005202 learned_addr learned my addr 172.21.15.89:0/3633005202 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f852d4ce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f852809e850 0x7f852809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f852d4ce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809f160 0x7f8528133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f852d4ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 0x7f85281341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f852d4ce700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852806bec0 con 0x7f852809e850 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f852d4ce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85280a0030 con 0x7f852809f160 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f852d4ce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85280a0830 con 0x7f852809f5b0 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f354bfff700 1 -- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 msgr2=0x7f354c122a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f354bfff700 1 --2- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 0x7f354c122a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f354bfff700 1 -- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 msgr2=0x7f354c1224d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f354bfff700 1 --2- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c1224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f354bfff700 1 -- 172.21.15.89:0/3633005202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c00b7f0 con 0x7f354c09e410 2023-09-30T22:22:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.879+0000 7f8526ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f852809e850 0x7f852809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f8526ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f852809e850 0x7f852809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48762/0 (socket says 172.21.15.89:48762) 2023-09-30T22:22:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f8526ffd700 1 -- 172.21.15.89:0/2730750302 learned_addr learned my addr 172.21.15.89:0/2730750302 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f85267fc700 1 --2- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809f160 0x7f8528133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f8526ffd700 1 -- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 msgr2=0x7f85281341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f8526ffd700 1 --2- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 0x7f85281341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f8526ffd700 1 -- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809f160 msgr2=0x7f8528133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f8526ffd700 1 --2- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809f160 0x7f8528133c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f354bfff700 1 --2- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c122f50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f354400b680 tx=0x7f3544004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f8526ffd700 1 -- 172.21.15.89:0/2730750302 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852806caa0 con 0x7f852809e850 2023-09-30T22:22:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f3533fff700 1 -- 172.21.15.89:0/3633005202 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544003e10 con 0x7f354c09e410 2023-09-30T22:22:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3544009a70 con 0x7f354c09e410 2023-09-30T22:22:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c130350 con 0x7f354c09e410 2023-09-30T22:22:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f3533fff700 1 -- 172.21.15.89:0/3633005202 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3544020070 con 0x7f354c09e410 2023-09-30T22:22:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f8526ffd700 1 --2- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809e850 0x7f852809ec20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f85180098f0 tx=0x7f8518009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133998480793322f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.881+0000 7f8525ffb700 1 -- 172.21.15.89:0/2730750302 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518011830 con 0x7f852809e850 2023-09-30T22:22:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.881+0000 7f8525ffb700 1 -- 172.21.15.89:0/2730750302 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8518011990 con 0x7f852809e850 2023-09-30T22:22:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.881+0000 7f8525ffb700 1 -- 172.21.15.89:0/2730750302 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518011d20 con 0x7f852809e850 2023-09-30T22:22:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.880+0000 7f3533fff700 1 -- 172.21.15.89:0/3633005202 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544027680 con 0x7f354c09e410 2023-09-30T22:22:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.881+0000 7f3533fff700 1 -- 172.21.15.89:0/3633005202 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3544012430 con 0x7f354c09e410 2023-09-30T22:22:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.881+0000 7f3533fff700 1 -- 172.21.15.89:0/3633005202 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3544022070 con 0x7f354c09e410 2023-09-30T22:22:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.881+0000 7f852d4ce700 1 -- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809e850 msgr2=0x7f852809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.881+0000 7f852d4ce700 1 --2- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809e850 0x7f852809ec20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f85180098f0 tx=0x7f8518009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.882+0000 7f852d4ce700 1 -- 172.21.15.89:0/2730750302 shutdown_connections 2023-09-30T22:22:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.882+0000 7f852d4ce700 1 --2- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 0x7f85281341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.882+0000 7f852d4ce700 1 --2- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809e850 0x7f852809ec20 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.882+0000 7f852d4ce700 1 --2- 172.21.15.89:0/2730750302 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809f160 0x7f8528133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.882+0000 7f852d4ce700 1 -- 172.21.15.89:0/2730750302 >> 172.21.15.89:0/2730750302 conn(0x7f85280a3f70 msgr2=0x7f85280a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.882+0000 7f852d4ce700 1 -- 172.21.15.89:0/2730750302 shutdown_connections 2023-09-30T22:22:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.882+0000 7f852d4ce700 1 -- 172.21.15.89:0/2730750302 wait complete. 2023-09-30T22:22:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.882+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c074ea0 con 0x7f354c09e410 2023-09-30T22:22:11.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f852d4ce700 1 Processor -- start 2023-09-30T22:22:11.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f852d4ce700 1 -- start start 2023-09-30T22:22:11.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f852d4ce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809e850 0x7f8528120630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f852d4ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809f160 0x7f8528120b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f852d4ce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 0x7f85281210b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f852d4ce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8528121b00 con 0x7f852809f160 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f852d4ce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8528133990 con 0x7f852809e850 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f852d4ce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8528123400 con 0x7f852809f5b0 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f8526ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809e850 0x7f8528120630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f8526ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809e850 0x7f8528120630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33892/0 (socket says 172.21.15.89:33892) 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.883+0000 7f8526ffd700 1 -- 172.21.15.89:0/1620218430 learned_addr learned my addr 172.21.15.89:0/1620218430 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f85277fe700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 0x7f85281210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f85267fc700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809f160 0x7f8528120b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f8526ffd700 1 -- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 msgr2=0x7f85281210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f8526ffd700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 0x7f85281210b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f8526ffd700 1 -- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809f160 msgr2=0x7f8528120b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f8526ffd700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809f160 0x7f8528120b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f8526ffd700 1 -- 172.21.15.89:0/1620218430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8518009550 con 0x7f852809e850 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f85277fe700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 0x7f85281210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f85267fc700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809f160 0x7f8528120b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f8526ffd700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809e850 0x7f8528120630 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8518000c00 tx=0x7f8518004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f850f7fe700 1 -- 172.21.15.89:0/1620218430 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518004400 con 0x7f852809e850 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f850f7fe700 1 -- 172.21.15.89:0/1620218430 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851801f070 con 0x7f852809e850 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f852d4ce700 1 -- 172.21.15.89:0/1620218430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8528123de0 con 0x7f852809e850 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f850f7fe700 1 -- 172.21.15.89:0/1620218430 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180038e0 con 0x7f852809e850 2023-09-30T22:22:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.884+0000 7f852d4ce700 1 -- 172.21.15.89:0/1620218430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85280a3570 con 0x7f852809e850 2023-09-30T22:22:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.885+0000 7f850d7fa700 1 -- 172.21.15.89:0/1620218430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85000052e0 con 0x7f852809e850 2023-09-30T22:22:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.886+0000 7f850f7fe700 1 -- 172.21.15.89:0/1620218430 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8518021030 con 0x7f852809e850 2023-09-30T22:22:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.886+0000 7f850f7fe700 1 -- 172.21.15.89:0/1620218430 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8518040ee0 con 0x7f852809e850 2023-09-30T22:22:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.888+0000 7f3533fff700 1 -- 172.21.15.89:0/3633005202 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3544011960 con 0x7f354c09e410 2023-09-30T22:22:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:11.890+0000 7f850f7fe700 1 -- 172.21.15.89:0/1620218430 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f851800b9d0 con 0x7f852809e850 2023-09-30T22:22:12.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.003+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe8d413b420 con 0x7fe8d4134dc0 2023-09-30T22:22:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.004+0000 7fe8c2ffd700 1 -- 172.21.15.89:0/2591520708 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8d413b420 con 0x7fe8d4134dc0 2023-09-30T22:22:12.005 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4134dc0 msgr2=0x7fe8d41339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 --2- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4134dc0 0x7fe8d41339e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc014040 tx=0x7fe8cc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 shutdown_connections 2023-09-30T22:22:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 --2- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4134dc0 0x7fe8d41339e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 --2- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d406e570 0x7fe8d4134880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 --2- 172.21.15.89:0/2591520708 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d406ee80 0x7fe8d41334a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 >> 172.21.15.89:0/2591520708 conn(0x7fe8d40a2aa0 msgr2=0x7fe8d40a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 shutdown_connections 2023-09-30T22:22:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.006+0000 7fe8d8bb5700 1 -- 172.21.15.89:0/2591520708 wait complete. 2023-09-30T22:22:12.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:12.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.058+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faf3c09ffe0 con 0x7faf3c09e930 2023-09-30T22:22:12.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.058+0000 7faf2affd700 1 -- 172.21.15.89:0/3705941504 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf2c029020 con 0x7faf3c09e930 2023-09-30T22:22:12.059 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09e930 msgr2=0x7faf3c06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09e930 0x7faf3c06b8b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0098f0 tx=0x7faf2c004750 comp rx=0 tx=0).stop 2023-09-30T22:22:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 shutdown_connections 2023-09-30T22:22:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf3c09f650 0x7faf3c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf3c09e930 0x7faf3c06b8b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 --2- 172.21.15.89:0/3705941504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf3c09f240 0x7faf3c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 >> 172.21.15.89:0/3705941504 conn(0x7faf3c0a4210 msgr2=0x7faf3c1b8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 shutdown_connections 2023-09-30T22:22:12.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.060+0000 7faf409d4700 1 -- 172.21.15.89:0/3705941504 wait complete. 2023-09-30T22:22:12.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.081+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f354c071d00 con 0x7f354c09e410 2023-09-30T22:22:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.081+0000 7f3533fff700 1 -- 172.21.15.89:0/3633005202 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35440303e0 con 0x7f354c09e410 2023-09-30T22:22:12.082 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 msgr2=0x7f354c122f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 --2- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c122f50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f354400b680 tx=0x7f3544004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 shutdown_connections 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 --2- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f354c09e410 0x7f354c122f50 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 --2- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f354c09d6f0 0x7f354c1224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 --2- 172.21.15.89:0/3633005202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f354c09e000 0x7f354c122a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 >> 172.21.15.89:0/3633005202 conn(0x7f354c066f50 msgr2=0x7f354c130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 shutdown_connections 2023-09-30T22:22:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.083+0000 7f3551c05700 1 -- 172.21.15.89:0/3633005202 wait complete. 2023-09-30T22:22:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.085+0000 7f850d7fa700 1 -- 172.21.15.89:0/1620218430 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8500005860 con 0x7f852809e850 2023-09-30T22:22:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.085+0000 7f850f7fe700 1 -- 172.21.15.89:0/1620218430 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8518015070 con 0x7f852809e850 2023-09-30T22:22:12.086 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 -- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809e850 msgr2=0x7f8528120630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809e850 0x7f8528120630 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8518000c00 tx=0x7f8518004750 comp rx=0 tx=0).stop 2023-09-30T22:22:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 -- 172.21.15.89:0/1620218430 shutdown_connections 2023-09-30T22:22:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f852809f5b0 0x7f85281210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f852809e850 0x7f8528120630 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 --2- 172.21.15.89:0/1620218430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f852809f160 0x7f8528120b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 -- 172.21.15.89:0/1620218430 >> 172.21.15.89:0/1620218430 conn(0x7f85280a3f70 msgr2=0x7f85281b9820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 -- 172.21.15.89:0/1620218430 shutdown_connections 2023-09-30T22:22:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:12.087+0000 7f850d7fa700 1 -- 172.21.15.89:0/1620218430 wait complete. 2023-09-30T22:22:12.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:12.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:13.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:13.074 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:13.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:13.102 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:13.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.167+0000 7f14b9c18700 1 Processor -- start 2023-09-30T22:22:13.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.168+0000 7f14b9c18700 1 -- start start 2023-09-30T22:22:13.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.168+0000 7f14b9c18700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.168+0000 7f14b9c18700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14b409e000 0x7f14b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.168+0000 7f14b9c18700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b409b120 con 0x7f14b409e410 2023-09-30T22:22:13.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.168+0000 7f14b9c18700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b406b460 con 0x7f14b409e000 2023-09-30T22:22:13.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.168+0000 7f14b9c18700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14b409a7e0 con 0x7f14b409d6f0 2023-09-30T22:22:13.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.168+0000 7f14abfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14b409e000 0x7f14b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14b37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14abfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14b409e000 0x7f14b41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48782/0 (socket says 172.21.15.89:48782) 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14b37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56758/0 (socket says 172.21.15.89:56758) 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14abfff700 1 -- 172.21.15.89:0/3399283362 learned_addr learned my addr 172.21.15.89:0/3399283362 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14abfff700 1 -- 172.21.15.89:0/3399283362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409d6f0 msgr2=0x7f14b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14abfff700 1 --2- 172.21.15.89:0/3399283362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14abfff700 1 -- 172.21.15.89:0/3399283362 >> v1:172.21.15.89:6789/0 conn(0x7f14b409e410 legacy=0x7f14b4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14abfff700 1 -- 172.21.15.89:0/3399283362 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b409a500 con 0x7f14b409e000 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14abfff700 1 --2- 172.21.15.89:0/3399283362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409e000 0x7f14b41339f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f149c0098f0 tx=0x7f149c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76092361cbc121a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:13.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14b2ffd700 1 -- 172.21.15.89:0/3399283362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c011830 con 0x7f14b409e000 2023-09-30T22:22:13.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14b2ffd700 1 -- 172.21.15.89:0/3399283362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c011990 con 0x7f14b409e000 2023-09-30T22:22:13.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.169+0000 7f14b2ffd700 1 -- 172.21.15.89:0/3399283362 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c011d20 con 0x7f14b409e000 2023-09-30T22:22:13.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.170+0000 7f14b9c18700 1 -- 172.21.15.89:0/3399283362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409e000 msgr2=0x7f14b41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.170+0000 7f14b9c18700 1 --2- 172.21.15.89:0/3399283362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409e000 0x7f14b41339f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f149c0098f0 tx=0x7f149c009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:13.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.170+0000 7f14b9c18700 1 -- 172.21.15.89:0/3399283362 shutdown_connections 2023-09-30T22:22:13.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.170+0000 7f14b9c18700 1 --2- 172.21.15.89:0/3399283362 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.170+0000 7f14b9c18700 1 --2- 172.21.15.89:0/3399283362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409e000 0x7f14b41339f0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.170+0000 7f14b9c18700 1 -- 172.21.15.89:0/3399283362 >> 172.21.15.89:0/3399283362 conn(0x7f14b4066f50 msgr2=0x7f14b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.170+0000 7f14b9c18700 1 -- 172.21.15.89:0/3399283362 shutdown_connections 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.170+0000 7f14b9c18700 1 -- 172.21.15.89:0/3399283362 wait complete. 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.171+0000 7f14b9c18700 1 Processor -- start 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.171+0000 7f14b9c18700 1 -- start start 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b9c18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409d6f0 0x7f14b41a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b9c18700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b409e000 0x7f14b41a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b9c18700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409e410 0x7f14b41a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b9c18700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41a64c0 con 0x7f14b409d6f0 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b9c18700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41baaf0 con 0x7f14b409e000 2023-09-30T22:22:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b9c18700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40a3160 con 0x7f14b409e410 2023-09-30T22:22:13.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409d6f0 0x7f14b41a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409d6f0 0x7f14b41a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48798/0 (socket says 172.21.15.89:48798) 2023-09-30T22:22:13.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.172+0000 7f14b37fe700 1 -- 172.21.15.89:0/4214183714 learned_addr learned my addr 172.21.15.89:0/4214183714 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:13.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b3fff700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409e410 0x7f14b41a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14abfff700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b409e000 0x7f14b41a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b3fff700 1 -- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b409e000 msgr2=0x7f14b41a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b3fff700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b409e000 0x7f14b41a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b3fff700 1 -- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409d6f0 msgr2=0x7f14b41a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b3fff700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409d6f0 0x7f14b41a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b3fff700 1 -- 172.21.15.89:0/4214183714 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c009550 con 0x7f14b409e410 2023-09-30T22:22:13.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b3fff700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409e410 0x7f14b41a2f70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00cb90 tx=0x7f14ac00cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b0ff9700 1 -- 172.21.15.89:0/4214183714 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac003c00 con 0x7f14b409e410 2023-09-30T22:22:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b0ff9700 1 -- 172.21.15.89:0/4214183714 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac003d60 con 0x7f14b409e410 2023-09-30T22:22:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b409c710 con 0x7f14b409e410 2023-09-30T22:22:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.173+0000 7f14b0ff9700 1 -- 172.21.15.89:0/4214183714 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00aaa0 con 0x7f14b409e410 2023-09-30T22:22:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.174+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b41a4660 con 0x7f14b409e410 2023-09-30T22:22:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.174+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b4074ea0 con 0x7f14b409e410 2023-09-30T22:22:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.175+0000 7f14b0ff9700 1 -- 172.21.15.89:0/4214183714 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f14ac00ac00 con 0x7f14b409e410 2023-09-30T22:22:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.175+0000 7f14b0ff9700 1 -- 172.21.15.89:0/4214183714 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f14ac040250 con 0x7f14b409e410 2023-09-30T22:22:13.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.183+0000 7f14b0ff9700 1 -- 172.21.15.89:0/4214183714 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14ac011740 con 0x7f14b409e410 2023-09-30T22:22:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.210+0000 7f395c5d5700 1 Processor -- start 2023-09-30T22:22:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.210+0000 7f395c5d5700 1 -- start start 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f395c5d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409d6f0 0x7f395409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f395c5d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409e410 0x7f3954130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f395c5d5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f395409f130 con 0x7f395409e000 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f395c5d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395409f520 con 0x7f395409e410 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f395c5d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f395409e930 con 0x7f395409d6f0 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f395ab72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409e410 0x7f3954130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f3959b70700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f395409e000 0x7f3954130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36954/0 (socket says 172.21.15.89:36954) 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f3959b70700 1 -- 172.21.15.89:0/737522556 learned_addr learned my addr 172.21.15.89:0/737522556 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.211+0000 7f395a371700 1 --2- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409d6f0 0x7f395409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395936f700 1 -- 172.21.15.89:0/737522556 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142089827 0 0) 0x7f395409f130 con 0x7f395409e000 2023-09-30T22:22:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395936f700 1 -- 172.21.15.89:0/737522556 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39400035b0 con 0x7f395409e000 2023-09-30T22:22:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395ab72700 1 -- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409d6f0 msgr2=0x7f395409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395ab72700 1 --2- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409d6f0 0x7f395409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395ab72700 1 -- 172.21.15.89:0/737522556 >> v1:172.21.15.89:6789/0 conn(0x7f395409e000 legacy=0x7f3954130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395ab72700 1 -- 172.21.15.89:0/737522556 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395409b2d0 con 0x7f395409e410 2023-09-30T22:22:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395a371700 1 --2- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409d6f0 0x7f395409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395ab72700 1 --2- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409e410 0x7f3954130d70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f394c009dc0 tx=0x7f394c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a13572caf3338095 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395936f700 1 -- 172.21.15.89:0/737522556 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00e070 con 0x7f395409e410 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395936f700 1 -- 172.21.15.89:0/737522556 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c00bb20 con 0x7f395409e410 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.212+0000 7f395936f700 1 -- 172.21.15.89:0/737522556 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00be90 con 0x7f395409e410 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.213+0000 7f395c5d5700 1 -- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409e410 msgr2=0x7f3954130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.213+0000 7f395c5d5700 1 --2- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409e410 0x7f3954130d70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f394c009dc0 tx=0x7f394c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.213+0000 7f395c5d5700 1 -- 172.21.15.89:0/737522556 shutdown_connections 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.213+0000 7f395c5d5700 1 --2- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409e410 0x7f3954130d70 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.213+0000 7f395c5d5700 1 --2- 172.21.15.89:0/737522556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409d6f0 0x7f395409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.213+0000 7f395c5d5700 1 -- 172.21.15.89:0/737522556 >> 172.21.15.89:0/737522556 conn(0x7f3954066f50 msgr2=0x7f395409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.213+0000 7f395c5d5700 1 -- 172.21.15.89:0/737522556 shutdown_connections 2023-09-30T22:22:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.213+0000 7f395c5d5700 1 -- 172.21.15.89:0/737522556 wait complete. 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.215+0000 7f395c5d5700 1 Processor -- start 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.215+0000 7f395c5d5700 1 -- start start 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.215+0000 7f395c5d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409d6f0 0x7f395411dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.215+0000 7f395c5d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409e000 0x7f395411e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.215+0000 7f395c5d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f395409e410 0x7f395411e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.215+0000 7f395c5d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3954122150 con 0x7f395409e410 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.215+0000 7f395c5d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3954071ef0 con 0x7f395409d6f0 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.215+0000 7f395c5d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f395409c700 con 0x7f395409e000 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409d6f0 0x7f395411dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409d6f0 0x7f395411dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33908/0 (socket says 172.21.15.89:33908) 2023-09-30T22:22:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 -- 172.21.15.89:0/1614986174 learned_addr learned my addr 172.21.15.89:0/1614986174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f3959b70700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409e000 0x7f395411e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395ab72700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f395409e410 0x7f395411e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 -- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409e000 msgr2=0x7f395411e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409e000 0x7f395411e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 -- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f395409e410 msgr2=0x7f395411e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f395409e410 0x7f395411e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 -- 172.21.15.89:0/1614986174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395000b740 con 0x7f395409d6f0 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395ab72700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f395409e410 0x7f395411e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395a371700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409d6f0 0x7f395411dc70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3950010f80 tx=0x7f395000e640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f394affd700 1 -- 172.21.15.89:0/1614986174 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950011650 con 0x7f395409d6f0 2023-09-30T22:22:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f394affd700 1 -- 172.21.15.89:0/1614986174 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39500117b0 con 0x7f395409d6f0 2023-09-30T22:22:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c009a70 con 0x7f395409d6f0 2023-09-30T22:22:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.216+0000 7f394affd700 1 -- 172.21.15.89:0/1614986174 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950011b20 con 0x7f395409d6f0 2023-09-30T22:22:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.217+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395411f880 con 0x7f395409d6f0 2023-09-30T22:22:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.217+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395411ac20 con 0x7f395409d6f0 2023-09-30T22:22:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.218+0000 7f394affd700 1 -- 172.21.15.89:0/1614986174 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3950011c80 con 0x7f395409d6f0 2023-09-30T22:22:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.218+0000 7f394affd700 1 -- 172.21.15.89:0/1614986174 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3950040170 con 0x7f395409d6f0 2023-09-30T22:22:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.223+0000 7f394affd700 1 -- 172.21.15.89:0/1614986174 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3950019440 con 0x7f395409d6f0 2023-09-30T22:22:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.251+0000 7fb71b992700 1 Processor -- start 2023-09-30T22:22:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.251+0000 7fb71b992700 1 -- start start 2023-09-30T22:22:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.251+0000 7fb71b992700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7140a0a30 0x7fb7140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.251+0000 7fb71b992700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7140a1340 0x7fb714133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.251+0000 7fb71b992700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb714133fa0 0x7fb714134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.251+0000 7fb71b992700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb71409edd0 con 0x7fb7140a0a30 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.251+0000 7fb71b992700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71409e1f0 con 0x7fb7140a1340 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.251+0000 7fb71b992700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb71409dc50 con 0x7fb714133fa0 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7140a1340 0x7fb714133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7140a1340 0x7fb714133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33916/0 (socket says 172.21.15.89:33916) 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 -- 172.21.15.89:0/3432102433 learned_addr learned my addr 172.21.15.89:0/3432102433 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb71972e700 1 --2- 172.21.15.89:0/3432102433 >> v2:172.21.15.89:3300/0 conn(0x7fb7140a0a30 0x7fb7140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb719f2f700 1 --2- 172.21.15.89:0/3432102433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb714133fa0 0x7fb714134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 -- 172.21.15.89:0/3432102433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb714133fa0 msgr2=0x7fb714134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 --2- 172.21.15.89:0/3432102433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb714133fa0 0x7fb714134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 -- 172.21.15.89:0/3432102433 >> v2:172.21.15.89:3300/0 conn(0x7fb7140a0a30 msgr2=0x7fb7140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 --2- 172.21.15.89:0/3432102433 >> v2:172.21.15.89:3300/0 conn(0x7fb7140a0a30 0x7fb7140a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 -- 172.21.15.89:0/3432102433 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140a1fe0 con 0x7fb7140a1340 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb718f2d700 1 --2- 172.21.15.89:0/3432102433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7140a1340 0x7fb714133a60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb704009920 tx=0x7fb704009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ab593654306b1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb70bfff700 1 -- 172.21.15.89:0/3432102433 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704011830 con 0x7fb7140a1340 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb70bfff700 1 -- 172.21.15.89:0/3432102433 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb704011990 con 0x7fb7140a1340 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.252+0000 7fb70bfff700 1 -- 172.21.15.89:0/3432102433 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704011d00 con 0x7fb7140a1340 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 -- 172.21.15.89:0/3432102433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7140a1340 msgr2=0x7fb714133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 --2- 172.21.15.89:0/3432102433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7140a1340 0x7fb714133a60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb704009920 tx=0x7fb704009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 -- 172.21.15.89:0/3432102433 shutdown_connections 2023-09-30T22:22:13.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 --2- 172.21.15.89:0/3432102433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb714133fa0 0x7fb714134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 --2- 172.21.15.89:0/3432102433 >> v2:172.21.15.89:3300/0 conn(0x7fb7140a0a30 0x7fb7140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 --2- 172.21.15.89:0/3432102433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7140a1340 0x7fb714133a60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 -- 172.21.15.89:0/3432102433 >> 172.21.15.89:0/3432102433 conn(0x7fb7140a6550 msgr2=0x7fb7140a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 -- 172.21.15.89:0/3432102433 shutdown_connections 2023-09-30T22:22:13.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.253+0000 7fb71b992700 1 -- 172.21.15.89:0/3432102433 wait complete. 2023-09-30T22:22:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.254+0000 7fb71b992700 1 Processor -- start 2023-09-30T22:22:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.254+0000 7fb71b992700 1 -- start start 2023-09-30T22:22:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71b992700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7140a0a30 0x7fb714120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71b992700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7140a1340 0x7fb714120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71b992700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb714133fa0 0x7fb7141211e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71b992700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb714122320 con 0x7fb7140a0a30 2023-09-30T22:22:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71b992700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb714123570 con 0x7fb714133fa0 2023-09-30T22:22:13.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71b992700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb714122900 con 0x7fb7140a1340 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb719f2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb714133fa0 0x7fb7141211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71972e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7140a0a30 0x7fb714120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb718f2d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7140a1340 0x7fb714120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71972e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7140a0a30 0x7fb714120760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48832/0 (socket says 172.21.15.89:48832) 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb719f2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb714133fa0 0x7fb7141211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33918/0 (socket says 172.21.15.89:33918) 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb718f2d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7140a1340 0x7fb714120ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56792/0 (socket says 172.21.15.89:56792) 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb71972e700 1 -- 172.21.15.89:0/3092326568 learned_addr learned my addr 172.21.15.89:0/3092326568 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb719f2f700 1 -- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7140a1340 msgr2=0x7fb714120ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb719f2f700 1 --2- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7140a1340 0x7fb714120ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb719f2f700 1 -- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7140a0a30 msgr2=0x7fb714120760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb719f2f700 1 --2- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7140a0a30 0x7fb714120760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb719f2f700 1 -- 172.21.15.89:0/3092326568 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb704009550 con 0x7fb714133fa0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.255+0000 7fb719f2f700 1 --2- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb714133fa0 0x7fb7141211e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00df80 tx=0x7fb70c00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.256+0000 7fb709ffb700 1 -- 172.21.15.89:0/3092326568 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00bc80 con 0x7fb714133fa0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.256+0000 7fb709ffb700 1 -- 172.21.15.89:0/3092326568 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70c00f040 con 0x7fb714133fa0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.256+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb714123f50 con 0x7fb714133fa0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.256+0000 7fb709ffb700 1 -- 172.21.15.89:0/3092326568 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c009e60 con 0x7fb714133fa0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.256+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7140a0220 con 0x7fb714133fa0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.257+0000 7fb709ffb700 1 -- 172.21.15.89:0/3092326568 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb70c013410 con 0x7fb714133fa0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.257+0000 7fb709ffb700 1 -- 172.21.15.89:0/3092326568 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb70c0136c0 con 0x7fb714133fa0 2023-09-30T22:22:13.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.257+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb714074ea0 con 0x7fb714133fa0 2023-09-30T22:22:13.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.266+0000 7f9d4a628700 1 Processor -- start 2023-09-30T22:22:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.266+0000 7f9d4a628700 1 -- start start 2023-09-30T22:22:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.266+0000 7f9d4a628700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409e1d0 0x7f9d44135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.266+0000 7f9d4a628700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9d44135e50 0x7f9d44138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.266+0000 7f9d4a628700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4409f130 con 0x7f9d4409d6f0 2023-09-30T22:22:13.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.266+0000 7f9d4a628700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4409f520 con 0x7f9d44135e50 2023-09-30T22:22:13.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.266+0000 7f9d4a628700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4409e930 con 0x7f9d4409e1d0 2023-09-30T22:22:13.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.266+0000 7fb709ffb700 1 -- 172.21.15.89:0/3092326568 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb70c017080 con 0x7fb714133fa0 2023-09-30T22:22:13.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d437fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409e1d0 0x7f9d44135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d437fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409e1d0 0x7f9d44135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33920/0 (socket says 172.21.15.89:33920) 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d437fe700 1 -- 172.21.15.89:0/2607359893 learned_addr learned my addr 172.21.15.89:0/2607359893 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d48bc5700 1 --2- 172.21.15.89:0/2607359893 >> v2:172.21.15.89:3300/0 conn(0x7f9d44135e50 0x7f9d44138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d437fe700 1 -- 172.21.15.89:0/2607359893 >> v2:172.21.15.89:3300/0 conn(0x7f9d44135e50 msgr2=0x7f9d44138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d437fe700 1 --2- 172.21.15.89:0/2607359893 >> v2:172.21.15.89:3300/0 conn(0x7f9d44135e50 0x7f9d44138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d437fe700 1 -- 172.21.15.89:0/2607359893 >> v1:172.21.15.89:6789/0 conn(0x7f9d4409d6f0 legacy=0x7f9d4409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d437fe700 1 -- 172.21.15.89:0/2607359893 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4409b2d0 con 0x7f9d4409e1d0 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.267+0000 7f9d437fe700 1 --2- 172.21.15.89:0/2607359893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409e1d0 0x7f9d44135910 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800bbc0 tx=0x7f9d3800bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a37b90fd7e4b68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.268+0000 7f9d42ffd700 1 -- 172.21.15.89:0/2607359893 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800dcf0 con 0x7f9d4409e1d0 2023-09-30T22:22:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.268+0000 7f9d42ffd700 1 -- 172.21.15.89:0/2607359893 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3800de50 con 0x7f9d4409e1d0 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.268+0000 7f9d42ffd700 1 -- 172.21.15.89:0/2607359893 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38010690 con 0x7f9d4409e1d0 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.269+0000 7f9d4a628700 1 -- 172.21.15.89:0/2607359893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409e1d0 msgr2=0x7f9d44135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.269+0000 7f9d4a628700 1 --2- 172.21.15.89:0/2607359893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409e1d0 0x7f9d44135910 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800bbc0 tx=0x7f9d3800bed0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.269+0000 7f9d4a628700 1 -- 172.21.15.89:0/2607359893 shutdown_connections 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.269+0000 7f9d4a628700 1 --2- 172.21.15.89:0/2607359893 >> v2:172.21.15.89:3300/0 conn(0x7f9d44135e50 0x7f9d44138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.269+0000 7f9d4a628700 1 --2- 172.21.15.89:0/2607359893 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409e1d0 0x7f9d44135910 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.269+0000 7f9d4a628700 1 -- 172.21.15.89:0/2607359893 >> 172.21.15.89:0/2607359893 conn(0x7f9d44066f50 msgr2=0x7f9d4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.269+0000 7f9d4a628700 1 -- 172.21.15.89:0/2607359893 shutdown_connections 2023-09-30T22:22:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.269+0000 7f9d4a628700 1 -- 172.21.15.89:0/2607359893 wait complete. 2023-09-30T22:22:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.270+0000 7f9d4a628700 1 Processor -- start 2023-09-30T22:22:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.270+0000 7f9d4a628700 1 -- start start 2023-09-30T22:22:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d4a628700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409d6f0 0x7f9d4411a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d4a628700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d4409e1d0 0x7f9d4411a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d4a628700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d44135e50 0x7f9d441c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d4a628700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4409c700 con 0x7f9d4409e1d0 2023-09-30T22:22:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d4a628700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4411ce30 con 0x7f9d4409d6f0 2023-09-30T22:22:13.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d4a628700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d44119c00 con 0x7f9d44135e50 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d4409e1d0 0x7f9d4411a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d4409e1d0 0x7f9d4411a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48860/0 (socket says 172.21.15.89:48860) 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d437fe700 1 -- 172.21.15.89:0/1491626104 learned_addr learned my addr 172.21.15.89:0/1491626104 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d43fff700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409d6f0 0x7f9d4411a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.271+0000 7f9d48bc5700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d44135e50 0x7f9d441c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d48bc5700 1 -- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409d6f0 msgr2=0x7f9d4411a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d48bc5700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409d6f0 0x7f9d4411a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d48bc5700 1 -- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d4409e1d0 msgr2=0x7f9d4411a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d48bc5700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d4409e1d0 0x7f9d4411a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d48bc5700 1 -- 172.21.15.89:0/1491626104 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3800b7f0 con 0x7f9d44135e50 2023-09-30T22:22:13.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d48bc5700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d44135e50 0x7f9d441c3fe0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00ef90 tx=0x7f9d3c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d40ff9700 1 -- 172.21.15.89:0/1491626104 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00cc90 con 0x7f9d44135e50 2023-09-30T22:22:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d40ff9700 1 -- 172.21.15.89:0/1491626104 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00f040 con 0x7f9d44135e50 2023-09-30T22:22:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.272+0000 7f9d40ff9700 1 -- 172.21.15.89:0/1491626104 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c009e60 con 0x7f9d44135e50 2023-09-30T22:22:13.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.273+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d440a1e20 con 0x7f9d44135e50 2023-09-30T22:22:13.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.273+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4411b220 con 0x7f9d44135e50 2023-09-30T22:22:13.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.274+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4411cab0 con 0x7f9d44135e50 2023-09-30T22:22:13.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.274+0000 7f9d40ff9700 1 -- 172.21.15.89:0/1491626104 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d3c014480 con 0x7f9d44135e50 2023-09-30T22:22:13.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.275+0000 7f9d40ff9700 1 -- 172.21.15.89:0/1491626104 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d3c03ff10 con 0x7f9d44135e50 2023-09-30T22:22:13.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.283+0000 7f9d40ff9700 1 -- 172.21.15.89:0/1491626104 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d4411cab0 con 0x7f9d44135e50 2023-09-30T22:22:13.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.389+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f14b406c360 con 0x7f14b409e410 2023-09-30T22:22:13.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.390+0000 7f14b0ff9700 1 -- 172.21.15.89:0/4214183714 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14ac029090 con 0x7f14b409e410 2023-09-30T22:22:13.391 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409e410 msgr2=0x7f14b41a2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409e410 0x7f14b41a2f70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00cb90 tx=0x7f14ac00cea0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 shutdown_connections 2023-09-30T22:22:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b409e410 0x7f14b41a2f70 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b409d6f0 0x7f14b41a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 --2- 172.21.15.89:0/4214183714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b409e000 0x7f14b41a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 >> 172.21.15.89:0/4214183714 conn(0x7f14b4066f50 msgr2=0x7f14b406f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 shutdown_connections 2023-09-30T22:22:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.392+0000 7f14b9c18700 1 -- 172.21.15.89:0/4214183714 wait complete. 2023-09-30T22:22:13.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:13.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.441+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3954120b30 con 0x7f395409d6f0 2023-09-30T22:22:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.441+0000 7f394affd700 1 -- 172.21.15.89:0/1614986174 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f395001e020 con 0x7f395409d6f0 2023-09-30T22:22:13.442 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409d6f0 msgr2=0x7f395411dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409d6f0 0x7f395411dc70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3950010f80 tx=0x7f395000e640 comp rx=0 tx=0).stop 2023-09-30T22:22:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 shutdown_connections 2023-09-30T22:22:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f395409e410 0x7f395411e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f395409d6f0 0x7f395411dc70 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 --2- 172.21.15.89:0/1614986174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f395409e000 0x7f395411e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 >> 172.21.15.89:0/1614986174 conn(0x7f3954066f50 msgr2=0x7f3954130ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:13.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 shutdown_connections 2023-09-30T22:22:13.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.443+0000 7f395c5d5700 1 -- 172.21.15.89:0/1614986174 wait complete. 2023-09-30T22:22:13.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.464+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb7140a28e0 con 0x7fb714133fa0 2023-09-30T22:22:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.464+0000 7fb709ffb700 1 -- 172.21.15.89:0/3092326568 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb70c0118c0 con 0x7fb714133fa0 2023-09-30T22:22:13.465 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:13.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb714133fa0 msgr2=0x7fb7141211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 --2- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb714133fa0 0x7fb7141211e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00df80 tx=0x7fb70c00b390 comp rx=0 tx=0).stop 2023-09-30T22:22:13.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 shutdown_connections 2023-09-30T22:22:13.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 --2- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb714133fa0 0x7fb7141211e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 --2- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7140a0a30 0x7fb714120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 --2- 172.21.15.89:0/3092326568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7140a1340 0x7fb714120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 >> 172.21.15.89:0/3092326568 conn(0x7fb7140a6550 msgr2=0x7fb71409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:13.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 shutdown_connections 2023-09-30T22:22:13.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.466+0000 7fb71b992700 1 -- 172.21.15.89:0/3092326568 wait complete. 2023-09-30T22:22:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.477+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9d4411cab0 con 0x7f9d44135e50 2023-09-30T22:22:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.478+0000 7f9d40ff9700 1 -- 172.21.15.89:0/1491626104 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d4411cab0 con 0x7f9d44135e50 2023-09-30T22:22:13.479 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:13.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d44135e50 msgr2=0x7f9d441c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d44135e50 0x7f9d441c3fe0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00ef90 tx=0x7f9d3c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 shutdown_connections 2023-09-30T22:22:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9d44135e50 0x7f9d441c3fe0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9d4409d6f0 0x7f9d4411a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 --2- 172.21.15.89:0/1491626104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d4409e1d0 0x7f9d4411a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 >> 172.21.15.89:0/1491626104 conn(0x7f9d44066f50 msgr2=0x7f9d4406f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 shutdown_connections 2023-09-30T22:22:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:13.480+0000 7f9d4a628700 1 -- 172.21.15.89:0/1491626104 wait complete. 2023-09-30T22:22:13.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:14.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:14.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:14.480 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:14.494 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.548+0000 7fc5e21a4700 1 Processor -- start 2023-09-30T22:22:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.548+0000 7fc5e21a4700 1 -- start start 2023-09-30T22:22:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5e21a4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc06a770 0x7fc5dc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5e21a4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc13d1f0 0x7fc5dc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5e21a4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5dc09a810 con 0x7fc5dc09dc30 2023-09-30T22:22:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5e21a4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc06ab40 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5e21a4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc09ebc0 con 0x7fc5dc06a770 2023-09-30T22:22:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5daffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc5dc09dc30 0x7fc5dc09e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36972/0 (socket says 172.21.15.89:36972) 2023-09-30T22:22:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5daffd700 1 -- 172.21.15.89:0/3719896635 learned_addr learned my addr 172.21.15.89:0/3719896635 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5db7fe700 1 --2- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc06a770 0x7fc5dc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5dbfff700 1 --2- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc13d1f0 0x7fc5dc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5db7fe700 1 -- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc13d1f0 msgr2=0x7fc5dc13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5db7fe700 1 --2- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc13d1f0 0x7fc5dc13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5db7fe700 1 -- 172.21.15.89:0/3719896635 >> v1:172.21.15.89:6789/0 conn(0x7fc5dc09dc30 legacy=0x7fc5dc09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5da7fc700 1 -- 172.21.15.89:0/3719896635 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462645761 0 0) 0x7fc5dc09a810 con 0x7fc5dc09dc30 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5db7fe700 1 -- 172.21.15.89:0/3719896635 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc09e920 con 0x7fc5dc06a770 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.549+0000 7fc5dbfff700 1 --2- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc13d1f0 0x7fc5dc13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.550+0000 7fc5db7fe700 1 --2- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc06a770 0x7fc5dc09d6f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009870 tx=0x7fc5cc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37bdef5a4e633c23 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.550+0000 7fc5da7fc700 1 -- 172.21.15.89:0/3719896635 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc011830 con 0x7fc5dc06a770 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.550+0000 7fc5da7fc700 1 -- 172.21.15.89:0/3719896635 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc011990 con 0x7fc5dc06a770 2023-09-30T22:22:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.550+0000 7fc5da7fc700 1 -- 172.21.15.89:0/3719896635 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc011d20 con 0x7fc5dc06a770 2023-09-30T22:22:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.551+0000 7fc5e21a4700 1 -- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc06a770 msgr2=0x7fc5dc09d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.551+0000 7fc5e21a4700 1 --2- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc06a770 0x7fc5dc09d6f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009870 tx=0x7fc5cc009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.551+0000 7fc5e21a4700 1 -- 172.21.15.89:0/3719896635 shutdown_connections 2023-09-30T22:22:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.551+0000 7fc5e21a4700 1 --2- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc13d1f0 0x7fc5dc13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.551+0000 7fc5e21a4700 1 --2- 172.21.15.89:0/3719896635 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc06a770 0x7fc5dc09d6f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.551+0000 7fc5e21a4700 1 -- 172.21.15.89:0/3719896635 >> 172.21.15.89:0/3719896635 conn(0x7fc5dc066f50 msgr2=0x7fc5dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.551+0000 7fc5e21a4700 1 -- 172.21.15.89:0/3719896635 shutdown_connections 2023-09-30T22:22:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.551+0000 7fc5e21a4700 1 -- 172.21.15.89:0/3719896635 wait complete. 2023-09-30T22:22:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.552+0000 7fc5e21a4700 1 Processor -- start 2023-09-30T22:22:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.552+0000 7fc5e21a4700 1 -- start start 2023-09-30T22:22:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5e21a4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc06a770 0x7fc5dc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5e21a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5dc09dc30 0x7fc5dc071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5e21a4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc13d1f0 0x7fc5dc0715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5e21a4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc11c620 con 0x7fc5dc09dc30 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5e21a4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc070000 con 0x7fc5dc06a770 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5e21a4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0a1710 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5dbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc13d1f0 0x7fc5dc0715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5db7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc06a770 0x7fc5dc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5db7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc06a770 0x7fc5dc070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:33958/0 (socket says 172.21.15.89:33958) 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5dbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc13d1f0 0x7fc5dc0715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56816/0 (socket says 172.21.15.89:56816) 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5db7fe700 1 -- 172.21.15.89:0/283806416 learned_addr learned my addr 172.21.15.89:0/283806416 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.553+0000 7fc5daffd700 1 --2- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5dc09dc30 0x7fc5dc071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5dbfff700 1 -- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc06a770 msgr2=0x7fc5dc070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5dbfff700 1 --2- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc06a770 0x7fc5dc070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5dbfff700 1 -- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5dc09dc30 msgr2=0x7fc5dc071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5dbfff700 1 --2- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5dc09dc30 0x7fc5dc071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5dbfff700 1 -- 172.21.15.89:0/283806416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4009a70 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5dbfff700 1 --2- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc13d1f0 0x7fc5dc0715c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009e50 tx=0x7fc5d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5c3fff700 1 -- 172.21.15.89:0/283806416 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400cb40 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5c3fff700 1 -- 172.21.15.89:0/283806416 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d400f040 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5c3fff700 1 -- 172.21.15.89:0/283806416 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4003a20 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc0094a0 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.554+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc06f6f0 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.555+0000 7fc5c3fff700 1 -- 172.21.15.89:0/283806416 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5d4014480 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.556+0000 7fc5c3fff700 1 -- 172.21.15.89:0/283806416 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc5d4040010 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.556+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5dc074ea0 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.565+0000 7fc5c3fff700 1 -- 172.21.15.89:0/283806416 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5d4012930 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.600+0000 7f5f54420700 1 Processor -- start 2023-09-30T22:22:14.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.600+0000 7f5f54420700 1 -- start start 2023-09-30T22:22:14.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.600+0000 7f5f54420700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09d6f0 0x7f5f4c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f54420700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e000 0x7f5f4c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f54420700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f4c09e410 0x7f5f4c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f54420700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c06b460 con 0x7f5f4c09e410 2023-09-30T22:22:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f54420700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c09b120 con 0x7f5f4c09e000 2023-09-30T22:22:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f54420700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c09a7e0 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f521bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09d6f0 0x7f5f4c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f4c09e410 0x7f5f4c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5f4c09e410 0x7f5f4c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48886/0 (socket says 172.21.15.89:48886) 2023-09-30T22:22:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 -- 172.21.15.89:0/126551864 learned_addr learned my addr 172.21.15.89:0/126551864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f519bb700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e000 0x7f5f4c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f521bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09d6f0 0x7f5f4c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56828/0 (socket says 172.21.15.89:56828) 2023-09-30T22:22:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 -- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09d6f0 msgr2=0x7f5f4c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09d6f0 0x7f5f4c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 -- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e000 msgr2=0x7f5f4c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e000 0x7f5f4c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 -- 172.21.15.89:0/126551864 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c09a500 con 0x7f5f4c09e410 2023-09-30T22:22:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f521bc700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09d6f0 0x7f5f4c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f519bb700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e000 0x7f5f4c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.601+0000 7f5f529bd700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09e410 0x7f5f4c133f30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009e50 tx=0x7f5f4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999868ce63b26049 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.602+0000 7f5f511ba700 1 -- 172.21.15.89:0/126551864 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400e070 con 0x7f5f4c09e410 2023-09-30T22:22:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.602+0000 7f5f511ba700 1 -- 172.21.15.89:0/126551864 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4400b910 con 0x7f5f4c09e410 2023-09-30T22:22:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.602+0000 7f5f511ba700 1 -- 172.21.15.89:0/126551864 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400bcc0 con 0x7f5f4c09e410 2023-09-30T22:22:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 -- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09e410 msgr2=0x7f5f4c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09e410 0x7f5f4c133f30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009e50 tx=0x7f5f4400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 -- 172.21.15.89:0/126551864 shutdown_connections 2023-09-30T22:22:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09e410 0x7f5f4c133f30 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09d6f0 0x7f5f4c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 --2- 172.21.15.89:0/126551864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e000 0x7f5f4c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 -- 172.21.15.89:0/126551864 >> 172.21.15.89:0/126551864 conn(0x7f5f4c066f50 msgr2=0x7f5f4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 -- 172.21.15.89:0/126551864 shutdown_connections 2023-09-30T22:22:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.603+0000 7f5f54420700 1 -- 172.21.15.89:0/126551864 wait complete. 2023-09-30T22:22:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.604+0000 7f5f54420700 1 Processor -- start 2023-09-30T22:22:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.604+0000 7f5f54420700 1 -- start start 2023-09-30T22:22:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f54420700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09d6f0 0x7f5f4c1c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f54420700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09e000 0x7f5f4c1c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f54420700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e410 0x7f5f4c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f54420700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1a95e0 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f54420700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1a97c0 con 0x7f5f4c09e410 2023-09-30T22:22:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f54420700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1c5aa0 con 0x7f5f4c09e000 2023-09-30T22:22:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09d6f0 0x7f5f4c1c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09d6f0 0x7f5f4c1c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48892/0 (socket says 172.21.15.89:48892) 2023-09-30T22:22:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 -- 172.21.15.89:0/464923133 learned_addr learned my addr 172.21.15.89:0/464923133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f519bb700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09e000 0x7f5f4c1c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f529bd700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e410 0x7f5f4c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 -- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09e000 msgr2=0x7f5f4c1c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09e000 0x7f5f4c1c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 -- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e410 msgr2=0x7f5f4c1c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e410 0x7f5f4c1c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 -- 172.21.15.89:0/464923133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480094a0 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f521bc700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09d6f0 0x7f5f4c1c5020 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800ec10 tx=0x7f5f4800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f42ffd700 1 -- 172.21.15.89:0/464923133 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4800c910 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f42ffd700 1 -- 172.21.15.89:0/464923133 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4800ca70 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c0094a0 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.605+0000 7f5f42ffd700 1 -- 172.21.15.89:0/464923133 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4800cde0 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.606+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f44009b30 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.606+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c074ea0 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.607+0000 7f5f42ffd700 1 -- 172.21.15.89:0/464923133 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f48013470 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.607+0000 7f5f42ffd700 1 -- 172.21.15.89:0/464923133 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5f48040060 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.616+0000 7f5f42ffd700 1 -- 172.21.15.89:0/464923133 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f4801a770 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.637+0000 7f311b970700 1 Processor -- start 2023-09-30T22:22:14.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.637+0000 7f311b970700 1 -- start start 2023-09-30T22:22:14.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311b970700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f311409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311b970700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 0x7f31141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311b970700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f311409e410 0x7f3114133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311b970700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311406b460 con 0x7f311409e410 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311b970700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311409b120 con 0x7f311409e000 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311b970700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f311409a7e0 con 0x7f311409d6f0 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f311409e410 0x7f3114133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311970c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f311409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f311409e410 0x7f3114133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48900/0 (socket says 172.21.15.89:48900) 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311970c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f311409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56858/0 (socket says 172.21.15.89:56858) 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3118f0b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 0x7f31141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 -- 172.21.15.89:0/537671314 learned_addr learned my addr 172.21.15.89:0/537671314 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 -- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 msgr2=0x7f311409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 --2- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f311409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 -- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 msgr2=0x7f31141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 --2- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 0x7f31141339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 -- 172.21.15.89:0/537671314 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311409a500 con 0x7f311409e410 2023-09-30T22:22:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f311970c700 1 --2- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f311409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.638+0000 7f3119f0d700 1 --2- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f3114133f30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f310c009e50 tx=0x7f310c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c14079ebde6fae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f310bfff700 1 -- 172.21.15.89:0/537671314 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c00e070 con 0x7f311409e410 2023-09-30T22:22:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f310bfff700 1 -- 172.21.15.89:0/537671314 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c00b910 con 0x7f311409e410 2023-09-30T22:22:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f310bfff700 1 -- 172.21.15.89:0/537671314 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c00bcc0 con 0x7f311409e410 2023-09-30T22:22:14.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 -- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 msgr2=0x7f3114133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 --2- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f3114133f30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f310c009e50 tx=0x7f310c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 -- 172.21.15.89:0/537671314 shutdown_connections 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 --2- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f3114133f30 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 --2- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f311409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 --2- 172.21.15.89:0/537671314 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 0x7f31141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 -- 172.21.15.89:0/537671314 >> 172.21.15.89:0/537671314 conn(0x7f3114066f50 msgr2=0x7f311409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 -- 172.21.15.89:0/537671314 shutdown_connections 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.639+0000 7f311b970700 1 -- 172.21.15.89:0/537671314 wait complete. 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.640+0000 7f311b970700 1 Processor -- start 2023-09-30T22:22:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.640+0000 7f311b970700 1 -- start start 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f311b970700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f31141c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f311b970700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 0x7f31141c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f311b970700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f31141c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f311b970700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141a7180 con 0x7f311409e410 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f311b970700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31141a97f0 con 0x7f311409e000 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f311b970700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31141c5aa0 con 0x7f311409d6f0 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f31141c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f31141c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48916/0 (socket says 172.21.15.89:48916) 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 -- 172.21.15.89:0/1394895826 learned_addr learned my addr 172.21.15.89:0/1394895826 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f311970c700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f31141c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3118f0b700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 0x7f31141c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 -- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 msgr2=0x7f31141c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f31141c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 -- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 msgr2=0x7f31141c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 0x7f31141c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 -- 172.21.15.89:0/1394895826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100094a0 con 0x7f311409e410 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.641+0000 7f3119f0d700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f31141c8040 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f310c00b680 tx=0x7f310c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.642+0000 7f3109ffb700 1 -- 172.21.15.89:0/1394895826 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c004450 con 0x7f311409e410 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.642+0000 7f3109ffb700 1 -- 172.21.15.89:0/1394895826 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c0045b0 con 0x7f311409e410 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.642+0000 7f311b970700 1 -- 172.21.15.89:0/1394895826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c009b30 con 0x7f311409e410 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.642+0000 7f3109ffb700 1 -- 172.21.15.89:0/1394895826 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c004960 con 0x7f311409e410 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.642+0000 7f311b970700 1 -- 172.21.15.89:0/1394895826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31141a8b30 con 0x7f311409e410 2023-09-30T22:22:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.643+0000 7f30ff7fe700 1 -- 172.21.15.89:0/1394895826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ec0052e0 con 0x7f311409e410 2023-09-30T22:22:14.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.643+0000 7f3109ffb700 1 -- 172.21.15.89:0/1394895826 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f310c025460 con 0x7f311409e410 2023-09-30T22:22:14.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.643+0000 7f3109ffb700 1 -- 172.21.15.89:0/1394895826 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f310c041360 con 0x7f311409e410 2023-09-30T22:22:14.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.648+0000 7f3109ffb700 1 -- 172.21.15.89:0/1394895826 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f310c011780 con 0x7f311409e410 2023-09-30T22:22:14.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.659+0000 7fcf8dda6700 1 Processor -- start 2023-09-30T22:22:14.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.659+0000 7fcf8dda6700 1 -- start start 2023-09-30T22:22:14.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf8dda6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf8dda6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf8809d9f0 0x7fcf8806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf8dda6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8809e130 con 0x7fcf8806acb0 2023-09-30T22:22:14.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf8dda6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf881312b0 con 0x7fcf8809d9f0 2023-09-30T22:22:14.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf8dda6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8809f6e0 con 0x7fcf881316c0 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56876/0 (socket says 172.21.15.89:56876) 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf877fe700 1 -- 172.21.15.89:0/3949639111 learned_addr learned my addr 172.21.15.89:0/3949639111 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.660+0000 7fcf86ffd700 1 --2- 172.21.15.89:0/3949639111 >> v2:172.21.15.89:3300/0 conn(0x7fcf8809d9f0 0x7fcf8806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf877fe700 1 -- 172.21.15.89:0/3949639111 >> v2:172.21.15.89:3300/0 conn(0x7fcf8809d9f0 msgr2=0x7fcf8806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf877fe700 1 --2- 172.21.15.89:0/3949639111 >> v2:172.21.15.89:3300/0 conn(0x7fcf8809d9f0 0x7fcf8806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf877fe700 1 -- 172.21.15.89:0/3949639111 >> v1:172.21.15.89:6789/0 conn(0x7fcf8806acb0 legacy=0x7fcf88138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf877fe700 1 -- 172.21.15.89:0/3949639111 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8809e760 con 0x7fcf881316c0 2023-09-30T22:22:14.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf867fc700 1 -- 172.21.15.89:0/3949639111 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719982704 0 0) 0x7fcf80001490 con 0x7fcf8806acb0 2023-09-30T22:22:14.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf877fe700 1 --2- 172.21.15.89:0/3949639111 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88131a90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcf780098f0 tx=0x7fcf78009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c9f031784c7fa0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:14.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf867fc700 1 -- 172.21.15.89:0/3949639111 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78011830 con 0x7fcf881316c0 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf867fc700 1 -- 172.21.15.89:0/3949639111 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf78011990 con 0x7fcf881316c0 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.661+0000 7fcf867fc700 1 -- 172.21.15.89:0/3949639111 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78011d20 con 0x7fcf881316c0 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.662+0000 7fcf8dda6700 1 -- 172.21.15.89:0/3949639111 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 msgr2=0x7fcf88131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.662+0000 7fcf8dda6700 1 --2- 172.21.15.89:0/3949639111 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88131a90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcf780098f0 tx=0x7fcf78009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.662+0000 7fcf8dda6700 1 -- 172.21.15.89:0/3949639111 shutdown_connections 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.662+0000 7fcf8dda6700 1 --2- 172.21.15.89:0/3949639111 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88131a90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.662+0000 7fcf8dda6700 1 --2- 172.21.15.89:0/3949639111 >> v2:172.21.15.89:3300/0 conn(0x7fcf8809d9f0 0x7fcf8806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.662+0000 7fcf8dda6700 1 -- 172.21.15.89:0/3949639111 >> 172.21.15.89:0/3949639111 conn(0x7fcf88066f50 msgr2=0x7fcf8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.662+0000 7fcf8dda6700 1 -- 172.21.15.89:0/3949639111 shutdown_connections 2023-09-30T22:22:14.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.662+0000 7fcf8dda6700 1 -- 172.21.15.89:0/3949639111 wait complete. 2023-09-30T22:22:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.664+0000 7fcf8dda6700 1 Processor -- start 2023-09-30T22:22:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.664+0000 7fcf8dda6700 1 -- start start 2023-09-30T22:22:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf8dda6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf8806acb0 0x7fcf88070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf8dda6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8809d9f0 0x7fcf880711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf8dda6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf8dda6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8811ae80 con 0x7fcf8809d9f0 2023-09-30T22:22:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf8dda6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8811b060 con 0x7fcf8806acb0 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf8dda6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8811bc40 con 0x7fcf881316c0 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8809d9f0 0x7fcf880711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8809d9f0 0x7fcf880711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48928/0 (socket says 172.21.15.89:48928) 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 -- 172.21.15.89:0/1475707336 learned_addr learned my addr 172.21.15.89:0/1475707336 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf87fff700 1 --2- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 -- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 msgr2=0x7fcf88071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 --2- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 -- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf8806acb0 msgr2=0x7fcf88070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 --2- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf8806acb0 0x7fcf88070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 -- 172.21.15.89:0/1475707336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf78009550 con 0x7fcf8809d9f0 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.665+0000 7fcf86ffd700 1 --2- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8809d9f0 0x7fcf880711d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00e9b0 tx=0x7fcf7c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.666+0000 7fcf6ffff700 1 -- 172.21.15.89:0/1475707336 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0096b0 con 0x7fcf8809d9f0 2023-09-30T22:22:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.666+0000 7fcf6ffff700 1 -- 172.21.15.89:0/1475707336 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c00c7f0 con 0x7fcf8809d9f0 2023-09-30T22:22:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.666+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8809ead0 con 0x7fcf8809d9f0 2023-09-30T22:22:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.666+0000 7fcf6ffff700 1 -- 172.21.15.89:0/1475707336 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00cb80 con 0x7fcf8809d9f0 2023-09-30T22:22:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.666+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8812b900 con 0x7fcf8809d9f0 2023-09-30T22:22:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.667+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf88074ea0 con 0x7fcf8809d9f0 2023-09-30T22:22:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.667+0000 7fcf6ffff700 1 -- 172.21.15.89:0/1475707336 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf7c00cce0 con 0x7fcf8809d9f0 2023-09-30T22:22:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.668+0000 7fcf6ffff700 1 -- 172.21.15.89:0/1475707336 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcf7c03fa90 con 0x7fcf8809d9f0 2023-09-30T22:22:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.676+0000 7fcf6ffff700 1 -- 172.21.15.89:0/1475707336 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf7c01b3b0 con 0x7fcf8809d9f0 2023-09-30T22:22:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.792+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc5dc09a3d0 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.792+0000 7fc5c3fff700 1 -- 172.21.15.89:0/283806416 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5d402b090 con 0x7fc5dc13d1f0 2023-09-30T22:22:14.793 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.795+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc13d1f0 msgr2=0x7fc5dc0715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.795+0000 7fc5e21a4700 1 --2- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc13d1f0 0x7fc5dc0715c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009e50 tx=0x7fc5d400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.795+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 shutdown_connections 2023-09-30T22:22:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.795+0000 7fc5e21a4700 1 --2- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5dc13d1f0 0x7fc5dc0715c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.795+0000 7fc5e21a4700 1 --2- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5dc06a770 0x7fc5dc070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.795+0000 7fc5e21a4700 1 --2- 172.21.15.89:0/283806416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5dc09dc30 0x7fc5dc071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.796+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 >> 172.21.15.89:0/283806416 conn(0x7fc5dc066f50 msgr2=0x7fc5dc13c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.796+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 shutdown_connections 2023-09-30T22:22:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.796+0000 7fc5e21a4700 1 -- 172.21.15.89:0/283806416 wait complete. 2023-09-30T22:22:14.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:14.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.824+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5f4c09f420 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.825+0000 7f5f42ffd700 1 -- 172.21.15.89:0/464923133 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f48021070 con 0x7f5f4c09d6f0 2023-09-30T22:22:14.825 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09d6f0 msgr2=0x7f5f4c1c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09d6f0 0x7f5f4c1c5020 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800ec10 tx=0x7f5f4800ef20 comp rx=0 tx=0).stop 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 shutdown_connections 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5f4c09e410 0x7f5f4c1c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f4c09d6f0 0x7f5f4c1c5020 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 --2- 172.21.15.89:0/464923133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5f4c09e000 0x7f5f4c1c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 >> 172.21.15.89:0/464923133 conn(0x7f5f4c066f50 msgr2=0x7f5f4c1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 shutdown_connections 2023-09-30T22:22:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.827+0000 7f5f54420700 1 -- 172.21.15.89:0/464923133 wait complete. 2023-09-30T22:22:14.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.853+0000 7f30ff7fe700 1 -- 172.21.15.89:0/1394895826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f30ec005860 con 0x7f311409e410 2023-09-30T22:22:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.853+0000 7f3109ffb700 1 -- 172.21.15.89:0/1394895826 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f310c011390 con 0x7f311409e410 2023-09-30T22:22:14.855 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 -- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 msgr2=0x7f31141c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f31141c8040 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f310c00b680 tx=0x7f310c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 -- 172.21.15.89:0/1394895826 shutdown_connections 2023-09-30T22:22:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f311409e410 0x7f31141c8040 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f311409d6f0 0x7f31141c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 --2- 172.21.15.89:0/1394895826 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f311409e000 0x7f31141c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 -- 172.21.15.89:0/1394895826 >> 172.21.15.89:0/1394895826 conn(0x7f3114066f50 msgr2=0x7f31141bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 -- 172.21.15.89:0/1394895826 shutdown_connections 2023-09-30T22:22:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.856+0000 7f30ff7fe700 1 -- 172.21.15.89:0/1394895826 wait complete. 2023-09-30T22:22:14.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.872+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcf88072710 con 0x7fcf8809d9f0 2023-09-30T22:22:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.873+0000 7fcf6ffff700 1 -- 172.21.15.89:0/1475707336 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf7c021070 con 0x7fcf8809d9f0 2023-09-30T22:22:14.874 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8809d9f0 msgr2=0x7fcf880711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 --2- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8809d9f0 0x7fcf880711d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00e9b0 tx=0x7fcf7c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 shutdown_connections 2023-09-30T22:22:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 --2- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf881316c0 0x7fcf88071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 --2- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf8806acb0 0x7fcf88070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 --2- 172.21.15.89:0/1475707336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8809d9f0 0x7fcf880711d0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 >> 172.21.15.89:0/1475707336 conn(0x7fcf88066f50 msgr2=0x7fcf880a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 shutdown_connections 2023-09-30T22:22:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:14.875+0000 7fcf8dda6700 1 -- 172.21.15.89:0/1475707336 wait complete. 2023-09-30T22:22:14.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:15.814 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:15.841 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:15.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:15.889 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:15.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.949+0000 7f11e5579700 1 Processor -- start 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.949+0000 7f11e5579700 1 -- start start 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11e5579700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11e5579700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e013d1f0 0x7f11e013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11e5579700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e009a810 con 0x7f11e009dc30 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11e5579700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e006ab40 con 0x7f11e013d1f0 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11e5579700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11e009ebc0 con 0x7f11e006a770 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11de7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f11e009dc30 0x7f11e009e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:36996/0 (socket says 172.21.15.89:36996) 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11de7fc700 1 -- 172.21.15.89:0/2133990452 learned_addr learned my addr 172.21.15.89:0/2133990452 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11df7fe700 1 --2- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e013d1f0 0x7f11e013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:15.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11deffd700 1 --2- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11ddffb700 1 -- 172.21.15.89:0/2133990452 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893401967 0 0) 0x7f11e009a810 con 0x7f11e009dc30 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11ddffb700 1 -- 172.21.15.89:0/2133990452 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11c80035b0 con 0x7f11e009dc30 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11df7fe700 1 -- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 msgr2=0x7f11e009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11df7fe700 1 --2- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11df7fe700 1 -- 172.21.15.89:0/2133990452 >> v1:172.21.15.89:6789/0 conn(0x7f11e009dc30 legacy=0x7f11e009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.950+0000 7f11df7fe700 1 -- 172.21.15.89:0/2133990452 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e009e920 con 0x7f11e013d1f0 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.951+0000 7f11deffd700 1 --2- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.951+0000 7f11df7fe700 1 --2- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e013d1f0 0x7f11e013f5d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009dc0 tx=0x7f11d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf357c064414bae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.951+0000 7f11ddffb700 1 -- 172.21.15.89:0/2133990452 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800e070 con 0x7f11e013d1f0 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.951+0000 7f11ddffb700 1 -- 172.21.15.89:0/2133990452 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d800bb20 con 0x7f11e013d1f0 2023-09-30T22:22:15.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.951+0000 7f11ddffb700 1 -- 172.21.15.89:0/2133990452 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8012430 con 0x7f11e013d1f0 2023-09-30T22:22:15.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.952+0000 7f11e5579700 1 -- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e013d1f0 msgr2=0x7f11e013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:15.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.952+0000 7f11e5579700 1 --2- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e013d1f0 0x7f11e013f5d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009dc0 tx=0x7f11d800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:15.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.952+0000 7f11e5579700 1 -- 172.21.15.89:0/2133990452 shutdown_connections 2023-09-30T22:22:15.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.952+0000 7f11e5579700 1 --2- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e013d1f0 0x7f11e013f5d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:15.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.952+0000 7f11e5579700 1 --2- 172.21.15.89:0/2133990452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.952+0000 7f11e5579700 1 -- 172.21.15.89:0/2133990452 >> 172.21.15.89:0/2133990452 conn(0x7f11e0066f50 msgr2=0x7f11e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.952+0000 7f11e5579700 1 -- 172.21.15.89:0/2133990452 shutdown_connections 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.952+0000 7f11e5579700 1 -- 172.21.15.89:0/2133990452 wait complete. 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.953+0000 7f11e5579700 1 Processor -- start 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.953+0000 7f11e5579700 1 -- start start 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11e5579700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e011c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11e5579700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e009dc30 0x7f11e013b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11e5579700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11e013d1f0 0x7f11e013b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11e5579700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e011ebb0 con 0x7f11e013d1f0 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11e5579700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0135bd0 con 0x7f11e009dc30 2023-09-30T22:22:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11e5579700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11e013cc70 con 0x7f11e006a770 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11deffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e011c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11deffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e011c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56906/0 (socket says 172.21.15.89:56906) 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11deffd700 1 -- 172.21.15.89:0/3817729048 learned_addr learned my addr 172.21.15.89:0/3817729048 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11df7fe700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11e013d1f0 0x7f11e013b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11df7fe700 1 -- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 msgr2=0x7f11e011c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11df7fe700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e011c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11df7fe700 1 -- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e009dc30 msgr2=0x7f11e013b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11df7fe700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e009dc30 0x7f11e013b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11df7fe700 1 -- 172.21.15.89:0/3817729048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d8009a70 con 0x7f11e013d1f0 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11deffd700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e011c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11df7fe700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11e013d1f0 0x7f11e013b9b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f11e006e4a0 tx=0x7f11d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.954+0000 7f11c77fe700 1 -- 172.21.15.89:0/3817729048 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d80048b0 con 0x7f11e013d1f0 2023-09-30T22:22:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.955+0000 7f11c77fe700 1 -- 172.21.15.89:0/3817729048 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d8020070 con 0x7f11e013d1f0 2023-09-30T22:22:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.955+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e011ee00 con 0x7f11e013d1f0 2023-09-30T22:22:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.955+0000 7f11c77fe700 1 -- 172.21.15.89:0/3817729048 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d80276c0 con 0x7f11e013d1f0 2023-09-30T22:22:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.955+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e011e860 con 0x7f11e013d1f0 2023-09-30T22:22:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.955+0000 7f11c77fe700 1 -- 172.21.15.89:0/3817729048 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11d8012430 con 0x7f11e013d1f0 2023-09-30T22:22:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.956+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e009f1a0 con 0x7f11e013d1f0 2023-09-30T22:22:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.956+0000 7f11c77fe700 1 -- 172.21.15.89:0/3817729048 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f11d8022070 con 0x7f11e013d1f0 2023-09-30T22:22:15.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.964+0000 7f11c77fe700 1 -- 172.21.15.89:0/3817729048 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11d801a790 con 0x7f11e013d1f0 2023-09-30T22:22:15.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.999+0000 7f624f394700 1 Processor -- start 2023-09-30T22:22:16.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:15.999+0000 7f624f394700 1 -- start start 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624f394700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62480a0a30 0x7f62480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624f394700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6248138d80 0x7f624813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624f394700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62480a2a70 con 0x7f624809d630 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624f394700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62480a01d0 con 0x7f6248138d80 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624f394700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f624809e2b0 con 0x7f62480a0a30 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d931700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6248138d80 0x7f624813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d931700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6248138d80 0x7f624813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48934/0 (socket says 172.21.15.89:48934) 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d931700 1 -- 172.21.15.89:0/2167058250 learned_addr learned my addr 172.21.15.89:0/2167058250 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d130700 1 --2- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62480a0a30 0x7f62480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d931700 1 -- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62480a0a30 msgr2=0x7f62480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:16.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d931700 1 --2- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62480a0a30 0x7f62480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d931700 1 -- 172.21.15.89:0/2167058250 >> v1:172.21.15.89:6789/0 conn(0x7f624809d630 legacy=0x7f62480a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d931700 1 -- 172.21.15.89:0/2167058250 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624809dc50 con 0x7f6248138d80 2023-09-30T22:22:16.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.000+0000 7f624d130700 1 --2- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62480a0a30 0x7f62480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:16.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.001+0000 7f624d931700 1 --2- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248138d80 0x7f624813b160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6240009e50 tx=0x7f624000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56511548718fd5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.001+0000 7f623ffff700 1 -- 172.21.15.89:0/2167058250 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000e070 con 0x7f6248138d80 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.001+0000 7f623ffff700 1 -- 172.21.15.89:0/2167058250 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624000b910 con 0x7f6248138d80 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.001+0000 7f623ffff700 1 -- 172.21.15.89:0/2167058250 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000bcc0 con 0x7f6248138d80 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.001+0000 7f624f394700 1 -- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248138d80 msgr2=0x7f624813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.001+0000 7f624f394700 1 --2- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248138d80 0x7f624813b160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6240009e50 tx=0x7f624000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.002+0000 7f624f394700 1 -- 172.21.15.89:0/2167058250 shutdown_connections 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.002+0000 7f624f394700 1 --2- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6248138d80 0x7f624813b160 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.002+0000 7f624f394700 1 --2- 172.21.15.89:0/2167058250 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62480a0a30 0x7f62480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.002+0000 7f624f394700 1 -- 172.21.15.89:0/2167058250 >> 172.21.15.89:0/2167058250 conn(0x7f62480a3050 msgr2=0x7f62480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.002+0000 7f624f394700 1 -- 172.21.15.89:0/2167058250 shutdown_connections 2023-09-30T22:22:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.002+0000 7f624f394700 1 -- 172.21.15.89:0/2167058250 wait complete. 2023-09-30T22:22:16.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624f394700 1 Processor -- start 2023-09-30T22:22:16.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624f394700 1 -- start start 2023-09-30T22:22:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624f394700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624809d630 0x7f6248133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624f394700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62480a0a30 0x7f6248133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624f394700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248138d80 0x7f6248134310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624f394700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62481371b0 con 0x7f62480a0a30 2023-09-30T22:22:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624f394700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6248138030 con 0x7f6248138d80 2023-09-30T22:22:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624f394700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62481373b0 con 0x7f624809d630 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624c92f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62480a0a30 0x7f6248133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624c92f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62480a0a30 0x7f6248133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48940/0 (socket says 172.21.15.89:48940) 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624c92f700 1 -- 172.21.15.89:0/1445305740 learned_addr learned my addr 172.21.15.89:0/1445305740 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624d931700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248138d80 0x7f6248134310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.003+0000 7f624d130700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624809d630 0x7f6248133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f624c92f700 1 -- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624809d630 msgr2=0x7f6248133890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f624c92f700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624809d630 0x7f6248133890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f624c92f700 1 -- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248138d80 msgr2=0x7f6248134310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f624c92f700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248138d80 0x7f6248134310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f624c92f700 1 -- 172.21.15.89:0/1445305740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440094a0 con 0x7f62480a0a30 2023-09-30T22:22:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f624c92f700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62480a0a30 0x7f6248133dd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f623800c4e0 tx=0x7f623800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f623dffb700 1 -- 172.21.15.89:0/1445305740 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380052d0 con 0x7f62480a0a30 2023-09-30T22:22:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f623dffb700 1 -- 172.21.15.89:0/1445305740 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6238005430 con 0x7f62480a0a30 2023-09-30T22:22:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6240009b30 con 0x7f62480a0a30 2023-09-30T22:22:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f623dffb700 1 -- 172.21.15.89:0/1445305740 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380038c0 con 0x7f62480a0a30 2023-09-30T22:22:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.004+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62480a4eb0 con 0x7f62480a0a30 2023-09-30T22:22:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.005+0000 7f623dffb700 1 -- 172.21.15.89:0/1445305740 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6238003a20 con 0x7f62480a0a30 2023-09-30T22:22:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.005+0000 7f623dffb700 1 -- 172.21.15.89:0/1445305740 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f623803ea10 con 0x7f62480a0a30 2023-09-30T22:22:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.006+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62480747f0 con 0x7f62480a0a30 2023-09-30T22:22:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.010+0000 7f623dffb700 1 -- 172.21.15.89:0/1445305740 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6238018260 con 0x7f62480a0a30 2023-09-30T22:22:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.036+0000 7fd36b70c700 1 Processor -- start 2023-09-30T22:22:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.036+0000 7fd36b70c700 1 -- start start 2023-09-30T22:22:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.036+0000 7fd36b70c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd364136710 0x7fd364138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.036+0000 7fd36b70c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.036+0000 7fd36b70c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36406ef30 con 0x7fd3641338a0 2023-09-30T22:22:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.036+0000 7fd36b70c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36406fef0 con 0x7fd364136710 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.036+0000 7fd36b70c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641334b0 con 0x7fd364139030 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd369ca9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd368ca7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd364136710 0x7fd364138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd369ca9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36413b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34070/0 (socket says 172.21.15.89:34070) 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd369ca9700 1 -- 172.21.15.89:0/2793978749 learned_addr learned my addr 172.21.15.89:0/2793978749 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd368ca7700 1 -- 172.21.15.89:0/2793978749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 msgr2=0x7fd36413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd368ca7700 1 --2- 172.21.15.89:0/2793978749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd368ca7700 1 -- 172.21.15.89:0/2793978749 >> v1:172.21.15.89:6789/0 conn(0x7fd3641338a0 legacy=0x7fd364133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd368ca7700 1 -- 172.21.15.89:0/2793978749 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36406fc30 con 0x7fd364136710 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd35bfff700 1 -- 172.21.15.89:0/2793978749 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450416441 0 0) 0x7fd36406ef30 con 0x7fd3641338a0 2023-09-30T22:22:16.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd369ca9700 1 --2- 172.21.15.89:0/2793978749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd368ca7700 1 --2- 172.21.15.89:0/2793978749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 0x7fd364138af0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd354009840 tx=0x7fd354009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912ee3d07623ec3c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd35bfff700 1 -- 172.21.15.89:0/2793978749 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354011830 con 0x7fd364136710 2023-09-30T22:22:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd35bfff700 1 -- 172.21.15.89:0/2793978749 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd354011990 con 0x7fd364136710 2023-09-30T22:22:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.037+0000 7fd35bfff700 1 -- 172.21.15.89:0/2793978749 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354011d20 con 0x7fd364136710 2023-09-30T22:22:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.038+0000 7fd36b70c700 1 -- 172.21.15.89:0/2793978749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 msgr2=0x7fd364138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.038+0000 7fd36b70c700 1 --2- 172.21.15.89:0/2793978749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 0x7fd364138af0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd354009840 tx=0x7fd354009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.038+0000 7fd36b70c700 1 -- 172.21.15.89:0/2793978749 shutdown_connections 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.038+0000 7fd36b70c700 1 --2- 172.21.15.89:0/2793978749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.038+0000 7fd36b70c700 1 --2- 172.21.15.89:0/2793978749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 0x7fd364138af0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.038+0000 7fd36b70c700 1 -- 172.21.15.89:0/2793978749 >> 172.21.15.89:0/2793978749 conn(0x7fd3640a2aa0 msgr2=0x7fd3640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.038+0000 7fd36b70c700 1 -- 172.21.15.89:0/2793978749 shutdown_connections 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.038+0000 7fd36b70c700 1 -- 172.21.15.89:0/2793978749 wait complete. 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.040+0000 7fd36b70c700 1 Processor -- start 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.040+0000 7fd36b70c700 1 -- start start 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.040+0000 7fd36b70c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3641338a0 0x7fd3641c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.040+0000 7fd36b70c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 0x7fd3641c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd36b70c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36411afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd36b70c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36411d060 con 0x7fd364136710 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd36b70c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641be1b0 con 0x7fd364139030 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd36b70c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd36411e3e0 con 0x7fd3641338a0 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd3694a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3641338a0 0x7fd3641c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd3694a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3641338a0 0x7fd3641c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56926/0 (socket says 172.21.15.89:56926) 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd3694a8700 1 -- 172.21.15.89:0/2361870621 learned_addr learned my addr 172.21.15.89:0/2361870621 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd369ca9700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36411afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd368ca7700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 0x7fd3641c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd369ca9700 1 -- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3641338a0 msgr2=0x7fd3641c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd369ca9700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3641338a0 0x7fd3641c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd369ca9700 1 -- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 msgr2=0x7fd3641c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd369ca9700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 0x7fd3641c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd369ca9700 1 -- 172.21.15.89:0/2361870621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c009b30 con 0x7fd364139030 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd369ca9700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36411afb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00efc0 tx=0x7fd35c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd359ffb700 1 -- 172.21.15.89:0/2361870621 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c018070 con 0x7fd364139030 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd359ffb700 1 -- 172.21.15.89:0/2361870621 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c00f040 con 0x7fd364139030 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3540094a0 con 0x7fd364139030 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.041+0000 7fd359ffb700 1 -- 172.21.15.89:0/2361870621 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c003c60 con 0x7fd364139030 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.042+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36406f640 con 0x7fd364139030 2023-09-30T22:22:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.042+0000 7fd359ffb700 1 -- 172.21.15.89:0/2361870621 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd35c00c880 con 0x7fd364139030 2023-09-30T22:22:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.042+0000 7fd359ffb700 1 -- 172.21.15.89:0/2361870621 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd35c03f8a0 con 0x7fd364139030 2023-09-30T22:22:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.043+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36411e9b0 con 0x7fd364139030 2023-09-30T22:22:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.048+0000 7fd359ffb700 1 -- 172.21.15.89:0/2361870621 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd35c003dc0 con 0x7fd364139030 2023-09-30T22:22:16.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.060+0000 7fd71949b700 1 Processor -- start 2023-09-30T22:22:16.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.060+0000 7fd71949b700 1 -- start start 2023-09-30T22:22:16.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd71949b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd71406c0e0 0x7fd71409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd71949b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd714134ec0 0x7fd7141379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd71949b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd7141383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd71949b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71406c4b0 con 0x7fd714134ec0 2023-09-30T22:22:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd71949b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140a09c0 con 0x7fd714137f20 2023-09-30T22:22:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd71949b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd714134980 con 0x7fd71406c0e0 2023-09-30T22:22:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd712ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd71406c0e0 0x7fd71409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd712ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd71406c0e0 0x7fd71409ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56930/0 (socket says 172.21.15.89:56930) 2023-09-30T22:22:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd712ffd700 1 -- 172.21.15.89:0/755740777 learned_addr learned my addr 172.21.15.89:0/755740777 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7137fe700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd7141383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7127fc700 1 --2- 172.21.15.89:0/755740777 >> v2:172.21.15.89:3300/0 conn(0x7fd714134ec0 0x7fd7141379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7127fc700 1 -- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd71406c0e0 msgr2=0x7fd71409ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7127fc700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd71406c0e0 0x7fd71409ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7127fc700 1 -- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 msgr2=0x7fd7141383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7127fc700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd7141383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7127fc700 1 -- 172.21.15.89:0/755740777 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7140a0600 con 0x7fd714134ec0 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7137fe700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd7141383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.061+0000 7fd7127fc700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd714134ec0 0x7fd7141379e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd708009840 tx=0x7fd708009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c699ac428b952eed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd711ffb700 1 -- 172.21.15.89:0/755740777 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708011830 con 0x7fd714134ec0 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd711ffb700 1 -- 172.21.15.89:0/755740777 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd708011990 con 0x7fd714134ec0 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd711ffb700 1 -- 172.21.15.89:0/755740777 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708011d20 con 0x7fd714134ec0 2023-09-30T22:22:16.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 -- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd714134ec0 msgr2=0x7fd7141379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd714134ec0 0x7fd7141379e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd708009840 tx=0x7fd708009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:16.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 -- 172.21.15.89:0/755740777 shutdown_connections 2023-09-30T22:22:16.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd7141383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd71406c0e0 0x7fd71409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 --2- 172.21.15.89:0/755740777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd714134ec0 0x7fd7141379e0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 -- 172.21.15.89:0/755740777 >> 172.21.15.89:0/755740777 conn(0x7fd7140a4210 msgr2=0x7fd7140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:16.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 -- 172.21.15.89:0/755740777 shutdown_connections 2023-09-30T22:22:16.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.062+0000 7fd71949b700 1 -- 172.21.15.89:0/755740777 wait complete. 2023-09-30T22:22:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.063+0000 7fd71949b700 1 Processor -- start 2023-09-30T22:22:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.063+0000 7fd71949b700 1 -- start start 2023-09-30T22:22:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd71949b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd71406c0e0 0x7fd71411a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd71949b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd714134ec0 0x7fd71411ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd71949b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd71411b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd71949b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71411ce10 con 0x7fd71406c0e0 2023-09-30T22:22:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd71949b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71411de70 con 0x7fd714137f20 2023-09-30T22:22:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd71949b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140a1510 con 0x7fd714134ec0 2023-09-30T22:22:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd71411b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd712ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd71406c0e0 0x7fd71411a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd71411b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34114/0 (socket says 172.21.15.89:34114) 2023-09-30T22:22:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7137fe700 1 -- 172.21.15.89:0/3699116278 learned_addr learned my addr 172.21.15.89:0/3699116278 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7127fc700 1 --2- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd714134ec0 0x7fd71411ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7137fe700 1 -- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd714134ec0 msgr2=0x7fd71411ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7137fe700 1 --2- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd714134ec0 0x7fd71411ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7137fe700 1 -- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd71406c0e0 msgr2=0x7fd71411a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7137fe700 1 --2- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd71406c0e0 0x7fd71411a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.064+0000 7fd7137fe700 1 -- 172.21.15.89:0/3699116278 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c009a70 con 0x7fd714137f20 2023-09-30T22:22:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.065+0000 7fd7137fe700 1 --2- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd71411b2e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00efc0 tx=0x7fd70c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.065+0000 7fd6fb7fe700 1 -- 172.21.15.89:0/3699116278 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00cb40 con 0x7fd714137f20 2023-09-30T22:22:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.065+0000 7fd6fb7fe700 1 -- 172.21.15.89:0/3699116278 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c00f040 con 0x7fd714137f20 2023-09-30T22:22:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.065+0000 7fd6fb7fe700 1 -- 172.21.15.89:0/3699116278 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c003a20 con 0x7fd714137f20 2023-09-30T22:22:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.065+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7080094a0 con 0x7fd714137f20 2023-09-30T22:22:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.065+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71411d600 con 0x7fd714137f20 2023-09-30T22:22:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.066+0000 7fd6fb7fe700 1 -- 172.21.15.89:0/3699116278 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd70c014480 con 0x7fd714137f20 2023-09-30T22:22:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.066+0000 7fd6fb7fe700 1 -- 172.21.15.89:0/3699116278 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd70c040040 con 0x7fd714137f20 2023-09-30T22:22:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.066+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71411dab0 con 0x7fd714137f20 2023-09-30T22:22:16.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.071+0000 7fd6fb7fe700 1 -- 172.21.15.89:0/3699116278 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd70c012930 con 0x7fd714137f20 2023-09-30T22:22:16.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.197+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f11e0074ea0 con 0x7f11e013d1f0 2023-09-30T22:22:16.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.198+0000 7f11c77fe700 1 -- 172.21.15.89:0/3817729048 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11d802c030 con 0x7f11e013d1f0 2023-09-30T22:22:16.199 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.201+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11e013d1f0 msgr2=0x7f11e013b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.201+0000 7f11e5579700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11e013d1f0 0x7f11e013b9b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f11e006e4a0 tx=0x7f11d8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.201+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 shutdown_connections 2023-09-30T22:22:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.201+0000 7f11e5579700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11e013d1f0 0x7f11e013b9b0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.201+0000 7f11e5579700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11e006a770 0x7f11e011c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.201+0000 7f11e5579700 1 --2- 172.21.15.89:0/3817729048 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11e009dc30 0x7f11e013b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.201+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 >> 172.21.15.89:0/3817729048 conn(0x7f11e0066f50 msgr2=0x7f11e013ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:16.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.202+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 shutdown_connections 2023-09-30T22:22:16.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.202+0000 7f11e5579700 1 -- 172.21.15.89:0/3817729048 wait complete. 2023-09-30T22:22:16.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:16.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.247+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6248072980 con 0x7f62480a0a30 2023-09-30T22:22:16.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.247+0000 7f623dffb700 1 -- 172.21.15.89:0/1445305740 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6238017020 con 0x7f62480a0a30 2023-09-30T22:22:16.248 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:16.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.249+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62480a0a30 msgr2=0x7f6248133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.249+0000 7f624f394700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62480a0a30 0x7f6248133dd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f623800c4e0 tx=0x7f623800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.249+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 shutdown_connections 2023-09-30T22:22:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.249+0000 7f624f394700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6248138d80 0x7f6248134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.249+0000 7f624f394700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f624809d630 0x7f6248133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.249+0000 7f624f394700 1 --2- 172.21.15.89:0/1445305740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62480a0a30 0x7f6248133dd0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.249+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 >> 172.21.15.89:0/1445305740 conn(0x7f62480a3050 msgr2=0x7f624809f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:16.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.249+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 shutdown_connections 2023-09-30T22:22:16.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.250+0000 7f624f394700 1 -- 172.21.15.89:0/1445305740 wait complete. 2023-09-30T22:22:16.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:16.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.273+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd364074ea0 con 0x7fd364139030 2023-09-30T22:22:16.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.273+0000 7fd359ffb700 1 -- 172.21.15.89:0/2361870621 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd35c02b090 con 0x7fd364139030 2023-09-30T22:22:16.274 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 msgr2=0x7fd36411afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36411afb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00efc0 tx=0x7fd35c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 shutdown_connections 2023-09-30T22:22:16.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd364139030 0x7fd36411afb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3641338a0 0x7fd3641c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 --2- 172.21.15.89:0/2361870621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364136710 0x7fd3641c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 >> 172.21.15.89:0/2361870621 conn(0x7fd3640a2aa0 msgr2=0x7fd3641344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:16.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 shutdown_connections 2023-09-30T22:22:16.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.275+0000 7fd36b70c700 1 -- 172.21.15.89:0/2361870621 wait complete. 2023-09-30T22:22:16.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:16.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.288+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd71411d220 con 0x7fd714137f20 2023-09-30T22:22:16.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.288+0000 7fd6fb7fe700 1 -- 172.21.15.89:0/3699116278 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd70c012340 con 0x7fd714137f20 2023-09-30T22:22:16.292 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:16.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 msgr2=0x7fd71411b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:16.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 --2- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd71411b2e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00efc0 tx=0x7fd70c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 shutdown_connections 2023-09-30T22:22:16.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 --2- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd714137f20 0x7fd71411b2e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 --2- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd71406c0e0 0x7fd71411a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 --2- 172.21.15.89:0/3699116278 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd714134ec0 0x7fd71411ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:16.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 >> 172.21.15.89:0/3699116278 conn(0x7fd7140a4210 msgr2=0x7fd7141369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:16.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 shutdown_connections 2023-09-30T22:22:16.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:16.290+0000 7fd71949b700 1 -- 172.21.15.89:0/3699116278 wait complete. 2023-09-30T22:22:16.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:17.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:17.263 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:17.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:17.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:17.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.359+0000 7f3e196fd700 1 Processor -- start 2023-09-30T22:22:17.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.359+0000 7f3e196fd700 1 -- start start 2023-09-30T22:22:17.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e196fd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e14130540 0x7f3e14130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e196fd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e1406fcf0 0x7f3e14138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e196fd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e140716a0 con 0x7f3e14130e50 2023-09-30T22:22:17.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e196fd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1409a390 con 0x7f3e1406fcf0 2023-09-30T22:22:17.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e196fd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14071290 con 0x7f3e14130540 2023-09-30T22:22:17.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e137fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e1406fcf0 0x7f3e14138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e137fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3e1406fcf0 0x7f3e14138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48984/0 (socket says 172.21.15.89:48984) 2023-09-30T22:22:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e137fe700 1 -- 172.21.15.89:0/3737335094 learned_addr learned my addr 172.21.15.89:0/3737335094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e12ffd700 1 --2- 172.21.15.89:0/3737335094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e14130540 0x7f3e14130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e137fe700 1 -- 172.21.15.89:0/3737335094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e14130540 msgr2=0x7f3e14130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e137fe700 1 --2- 172.21.15.89:0/3737335094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e14130540 0x7f3e14130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e137fe700 1 -- 172.21.15.89:0/3737335094 >> v1:172.21.15.89:6789/0 conn(0x7f3e14130e50 legacy=0x7f3e1406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.360+0000 7f3e137fe700 1 -- 172.21.15.89:0/3737335094 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14130340 con 0x7f3e1406fcf0 2023-09-30T22:22:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.361+0000 7f3e137fe700 1 --2- 172.21.15.89:0/3737335094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e1406fcf0 0x7f3e14138570 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009e50 tx=0x7f3e0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df94d6efcb0f1f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.361+0000 7f3e11ffb700 1 -- 172.21.15.89:0/3737335094 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00e070 con 0x7f3e1406fcf0 2023-09-30T22:22:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.361+0000 7f3e11ffb700 1 -- 172.21.15.89:0/3737335094 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c00b910 con 0x7f3e1406fcf0 2023-09-30T22:22:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.361+0000 7f3e11ffb700 1 -- 172.21.15.89:0/3737335094 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00bcc0 con 0x7f3e1406fcf0 2023-09-30T22:22:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.362+0000 7f3e196fd700 1 -- 172.21.15.89:0/3737335094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e1406fcf0 msgr2=0x7f3e14138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.362+0000 7f3e196fd700 1 --2- 172.21.15.89:0/3737335094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e1406fcf0 0x7f3e14138570 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009e50 tx=0x7f3e0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.362+0000 7f3e196fd700 1 -- 172.21.15.89:0/3737335094 shutdown_connections 2023-09-30T22:22:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.362+0000 7f3e196fd700 1 --2- 172.21.15.89:0/3737335094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e1406fcf0 0x7f3e14138570 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.362+0000 7f3e196fd700 1 --2- 172.21.15.89:0/3737335094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e14130540 0x7f3e14130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.362+0000 7f3e196fd700 1 -- 172.21.15.89:0/3737335094 >> 172.21.15.89:0/3737335094 conn(0x7f3e140a2aa0 msgr2=0x7f3e140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.362+0000 7f3e196fd700 1 -- 172.21.15.89:0/3737335094 shutdown_connections 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.362+0000 7f3e196fd700 1 -- 172.21.15.89:0/3737335094 wait complete. 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.363+0000 7f3e196fd700 1 Processor -- start 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.363+0000 7f3e196fd700 1 -- start start 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e196fd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e1406fcf0 0x7f3e141282f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e196fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e14130540 0x7f3e14128830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e196fd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e14130e50 0x7f3e14128d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e196fd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e141295a0 con 0x7f3e14130540 2023-09-30T22:22:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e196fd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140718e0 con 0x7f3e14130e50 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e196fd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14129f00 con 0x7f3e1406fcf0 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e127fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e14130540 0x7f3e14128830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e127fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e14130540 0x7f3e14128830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48994/0 (socket says 172.21.15.89:48994) 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e127fc700 1 -- 172.21.15.89:0/3357456556 learned_addr learned my addr 172.21.15.89:0/3357456556 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e12ffd700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e1406fcf0 0x7f3e141282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.364+0000 7f3e137fe700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e14130e50 0x7f3e14128d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e12ffd700 1 -- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e14130e50 msgr2=0x7f3e14128d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e12ffd700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e14130e50 0x7f3e14128d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e12ffd700 1 -- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e14130540 msgr2=0x7f3e14128830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e12ffd700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e14130540 0x7f3e14128830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e12ffd700 1 -- 172.21.15.89:0/3357456556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c009b30 con 0x7f3e1406fcf0 2023-09-30T22:22:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e127fc700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e14130540 0x7f3e14128830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e137fe700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e14130e50 0x7f3e14128d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e12ffd700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e1406fcf0 0x7f3e141282f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400e900 tx=0x7f3e0400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3dfb7fe700 1 -- 172.21.15.89:0/3357456556 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04013680 con 0x7f3e1406fcf0 2023-09-30T22:22:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3dfb7fe700 1 -- 172.21.15.89:0/3357456556 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e040137e0 con 0x7f3e1406fcf0 2023-09-30T22:22:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e140a06f0 con 0x7f3e1406fcf0 2023-09-30T22:22:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3dfb7fe700 1 -- 172.21.15.89:0/3357456556 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04013b50 con 0x7f3e1406fcf0 2023-09-30T22:22:17.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.365+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1412abb0 con 0x7f3e1406fcf0 2023-09-30T22:22:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.366+0000 7f3dfb7fe700 1 -- 172.21.15.89:0/3357456556 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3e04013cb0 con 0x7f3e1406fcf0 2023-09-30T22:22:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.366+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e14074ea0 con 0x7f3e1406fcf0 2023-09-30T22:22:17.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.366+0000 7f3dfb7fe700 1 -- 172.21.15.89:0/3357456556 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3e0403f5e0 con 0x7f3e1406fcf0 2023-09-30T22:22:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.375+0000 7f3dfb7fe700 1 -- 172.21.15.89:0/3357456556 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3e0400c910 con 0x7f3e1406fcf0 2023-09-30T22:22:17.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.412+0000 7fd30e27d700 1 Processor -- start 2023-09-30T22:22:17.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.412+0000 7fd30e27d700 1 -- start start 2023-09-30T22:22:17.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30e27d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd308136820 0x7fd308138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30e27d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30813b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30e27d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30806bc00 con 0x7fd308133a80 2023-09-30T22:22:17.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30e27d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30806c570 con 0x7fd308136820 2023-09-30T22:22:17.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30e27d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080a0040 con 0x7fd308139140 2023-09-30T22:22:17.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30c81a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30c81a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30813b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34136/0 (socket says 172.21.15.89:34136) 2023-09-30T22:22:17.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30c81a700 1 -- 172.21.15.89:0/304653116 learned_addr learned my addr 172.21.15.89:0/304653116 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:17.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd3077fe700 1 --2- 172.21.15.89:0/304653116 >> v2:172.21.15.89:3300/0 conn(0x7fd308136820 0x7fd308138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30c81a700 1 -- 172.21.15.89:0/304653116 >> v2:172.21.15.89:3300/0 conn(0x7fd308136820 msgr2=0x7fd308138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:17.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30c81a700 1 --2- 172.21.15.89:0/304653116 >> v2:172.21.15.89:3300/0 conn(0x7fd308136820 0x7fd308138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30c81a700 1 -- 172.21.15.89:0/304653116 >> v1:172.21.15.89:6789/0 conn(0x7fd308133a80 legacy=0x7fd308133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:17.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.413+0000 7fd30c81a700 1 -- 172.21.15.89:0/304653116 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308133860 con 0x7fd308139140 2023-09-30T22:22:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30c81a700 1 --2- 172.21.15.89:0/304653116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30813b520 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd300009e50 tx=0x7fd30000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e64d0647fc85d70 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd306ffd700 1 -- 172.21.15.89:0/304653116 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000e070 con 0x7fd308139140 2023-09-30T22:22:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd306ffd700 1 -- 172.21.15.89:0/304653116 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300003a20 con 0x7fd308139140 2023-09-30T22:22:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd306ffd700 1 -- 172.21.15.89:0/304653116 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300003dd0 con 0x7fd308139140 2023-09-30T22:22:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30e27d700 1 -- 172.21.15.89:0/304653116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 msgr2=0x7fd30813b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30e27d700 1 --2- 172.21.15.89:0/304653116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30813b520 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd300009e50 tx=0x7fd30000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30e27d700 1 -- 172.21.15.89:0/304653116 shutdown_connections 2023-09-30T22:22:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30e27d700 1 --2- 172.21.15.89:0/304653116 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30813b520 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30e27d700 1 --2- 172.21.15.89:0/304653116 >> v2:172.21.15.89:3300/0 conn(0x7fd308136820 0x7fd308138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30e27d700 1 -- 172.21.15.89:0/304653116 >> 172.21.15.89:0/304653116 conn(0x7fd3080a4210 msgr2=0x7fd3080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30e27d700 1 -- 172.21.15.89:0/304653116 shutdown_connections 2023-09-30T22:22:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.414+0000 7fd30e27d700 1 -- 172.21.15.89:0/304653116 wait complete. 2023-09-30T22:22:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.415+0000 7fd30e27d700 1 Processor -- start 2023-09-30T22:22:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd30e27d700 1 -- start start 2023-09-30T22:22:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd30e27d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd308133a80 0x7fd30812cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd30e27d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308136820 0x7fd30812d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd30e27d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30812d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd30e27d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30809c910 con 0x7fd308136820 2023-09-30T22:22:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd30e27d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3081365f0 con 0x7fd308139140 2023-09-30T22:22:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd30e27d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd30812dfb0 con 0x7fd308133a80 2023-09-30T22:22:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd3077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308136820 0x7fd30812d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd3077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308136820 0x7fd30812d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49012/0 (socket says 172.21.15.89:49012) 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd3077fe700 1 -- 172.21.15.89:0/2351224489 learned_addr learned my addr 172.21.15.89:0/2351224489 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd30c81a700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30812d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd3077fe700 1 -- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd308133a80 msgr2=0x7fd30812cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd3077fe700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd308133a80 0x7fd30812cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd3077fe700 1 -- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 msgr2=0x7fd30812d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd3077fe700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30812d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.416+0000 7fd3077fe700 1 -- 172.21.15.89:0/2351224489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300009b30 con 0x7fd308136820 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.417+0000 7fd30c81a700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30812d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.417+0000 7fd3077fe700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308136820 0x7fd30812d140 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00e980 tx=0x7fd2fc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.417+0000 7fd304ff9700 1 -- 172.21.15.89:0/2351224489 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc009710 con 0x7fd308136820 2023-09-30T22:22:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.417+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30809cb50 con 0x7fd308136820 2023-09-30T22:22:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.417+0000 7fd304ff9700 1 -- 172.21.15.89:0/2351224489 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00eed0 con 0x7fd308136820 2023-09-30T22:22:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.417+0000 7fd304ff9700 1 -- 172.21.15.89:0/2351224489 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00ca90 con 0x7fd308136820 2023-09-30T22:22:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.417+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd308071ee0 con 0x7fd308136820 2023-09-30T22:22:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.418+0000 7fd304ff9700 1 -- 172.21.15.89:0/2351224489 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd2fc00cbf0 con 0x7fd308136820 2023-09-30T22:22:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.418+0000 7fd304ff9700 1 -- 172.21.15.89:0/2351224489 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd2fc03f790 con 0x7fd308136820 2023-09-30T22:22:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.418+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd308074ea0 con 0x7fd308136820 2023-09-30T22:22:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.426+0000 7fd304ff9700 1 -- 172.21.15.89:0/2351224489 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd2fc0232f0 con 0x7fd308136820 2023-09-30T22:22:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.442+0000 7fbf0afe7700 1 Processor -- start 2023-09-30T22:22:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.442+0000 7fbf0afe7700 1 -- start start 2023-09-30T22:22:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.442+0000 7fbf0afe7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04134630 0x7fbf04136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.442+0000 7fbf0afe7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbf04136f50 0x7fbf04139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.442+0000 7fbf0afe7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0409b4f0 con 0x7fbf04131810 2023-09-30T22:22:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.442+0000 7fbf0afe7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf041313c0 con 0x7fbf04136f50 2023-09-30T22:22:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.442+0000 7fbf0afe7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0406e8c0 con 0x7fbf04134630 2023-09-30T22:22:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04134630 0x7fbf04136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04134630 0x7fbf04136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:56984/0 (socket says 172.21.15.89:56984) 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf09584700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbf04136f50 0x7fbf04139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf03fff700 1 -- 172.21.15.89:0/692452434 learned_addr learned my addr 172.21.15.89:0/692452434 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf03fff700 1 -- 172.21.15.89:0/692452434 >> v2:172.21.15.89:3300/0 conn(0x7fbf04136f50 msgr2=0x7fbf04139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf03fff700 1 --2- 172.21.15.89:0/692452434 >> v2:172.21.15.89:3300/0 conn(0x7fbf04136f50 0x7fbf04139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf03fff700 1 -- 172.21.15.89:0/692452434 >> v1:172.21.15.89:6789/0 conn(0x7fbf04131810 legacy=0x7fbf04131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf03fff700 1 -- 172.21.15.89:0/692452434 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf04071d20 con 0x7fbf04134630 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf09584700 1 --2- 172.21.15.89:0/692452434 >> v2:172.21.15.89:3300/0 conn(0x7fbf04136f50 0x7fbf04139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf03fff700 1 --2- 172.21.15.89:0/692452434 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04134630 0x7fbf04136a10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbef00098f0 tx=0x7fbef0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9136133425778986 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf037fe700 1 -- 172.21.15.89:0/692452434 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0011830 con 0x7fbf04134630 2023-09-30T22:22:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf037fe700 1 -- 172.21.15.89:0/692452434 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef0011990 con 0x7fbf04134630 2023-09-30T22:22:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.443+0000 7fbf037fe700 1 -- 172.21.15.89:0/692452434 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0011d20 con 0x7fbf04134630 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.444+0000 7fbf0afe7700 1 -- 172.21.15.89:0/692452434 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04134630 msgr2=0x7fbf04136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.444+0000 7fbf0afe7700 1 --2- 172.21.15.89:0/692452434 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04134630 0x7fbf04136a10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbef00098f0 tx=0x7fbef0009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.444+0000 7fbf0afe7700 1 -- 172.21.15.89:0/692452434 shutdown_connections 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.444+0000 7fbf0afe7700 1 --2- 172.21.15.89:0/692452434 >> v2:172.21.15.89:3300/0 conn(0x7fbf04136f50 0x7fbf04139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.444+0000 7fbf0afe7700 1 --2- 172.21.15.89:0/692452434 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04134630 0x7fbf04136a10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.444+0000 7fbf0afe7700 1 -- 172.21.15.89:0/692452434 >> 172.21.15.89:0/692452434 conn(0x7fbf040a2aa0 msgr2=0x7fbf040a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.444+0000 7fbf0afe7700 1 -- 172.21.15.89:0/692452434 shutdown_connections 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.444+0000 7fbf0afe7700 1 -- 172.21.15.89:0/692452434 wait complete. 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.445+0000 7fbf0afe7700 1 Processor -- start 2023-09-30T22:22:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.445+0000 7fbf0afe7700 1 -- start start 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf0afe7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04131810 0x7fbf041c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf0afe7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf04134630 0x7fbf041c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf0afe7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf04136f50 0x7fbf04118d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf0afe7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0411c270 con 0x7fbf04136f50 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf0afe7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf041bc030 con 0x7fbf04134630 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf0afe7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0412ebc0 con 0x7fbf04131810 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf09584700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf04136f50 0x7fbf04118d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf08d83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04131810 0x7fbf041c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf09584700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf04136f50 0x7fbf04118d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49020/0 (socket says 172.21.15.89:49020) 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf09584700 1 -- 172.21.15.89:0/1496489193 learned_addr learned my addr 172.21.15.89:0/1496489193 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf09584700 1 -- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04131810 msgr2=0x7fbf041c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf09584700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04131810 0x7fbf041c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.446+0000 7fbf03fff700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf04134630 0x7fbf041c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf09584700 1 -- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf04134630 msgr2=0x7fbf041c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf09584700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf04134630 0x7fbf041c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf09584700 1 -- 172.21.15.89:0/1496489193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc009a70 con 0x7fbf04136f50 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf03fff700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf04134630 0x7fbf041c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf09584700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf04136f50 0x7fbf04118d40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00efc0 tx=0x7fbefc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf017fa700 1 -- 172.21.15.89:0/1496489193 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc00cd40 con 0x7fbf04136f50 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf017fa700 1 -- 172.21.15.89:0/1496489193 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc00f040 con 0x7fbf04136f50 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef0009550 con 0x7fbf04136f50 2023-09-30T22:22:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf017fa700 1 -- 172.21.15.89:0/1496489193 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0146c0 con 0x7fbf04136f50 2023-09-30T22:22:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.447+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0411cfa0 con 0x7fbf04136f50 2023-09-30T22:22:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.448+0000 7fbf017fa700 1 -- 172.21.15.89:0/1496489193 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbefc014820 con 0x7fbf04136f50 2023-09-30T22:22:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.449+0000 7fbf017fa700 1 -- 172.21.15.89:0/1496489193 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbefc040450 con 0x7fbf04136f50 2023-09-30T22:22:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.449+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0411c3e0 con 0x7fbf04136f50 2023-09-30T22:22:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.453+0000 7fbf017fa700 1 -- 172.21.15.89:0/1496489193 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbefc018030 con 0x7fbf04136f50 2023-09-30T22:22:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.456+0000 7fb6bab79700 1 Processor -- start 2023-09-30T22:22:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.456+0000 7fb6bab79700 1 -- start start 2023-09-30T22:22:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.456+0000 7fb6bab79700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4133420 0x7fb6b41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6bab79700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb6b4133e30 0x7fb6b412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6bab79700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b406b990 con 0x7fb6b4134260 2023-09-30T22:22:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6bab79700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b40a0590 con 0x7fb6b4133e30 2023-09-30T22:22:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6bab79700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b409fb80 con 0x7fb6b4133420 2023-09-30T22:22:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b9116700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb6b4134260 0x7fb6b41304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37066/0 (socket says 172.21.15.89:37066) 2023-09-30T22:22:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b9116700 1 -- 172.21.15.89:0/1387134396 learned_addr learned my addr 172.21.15.89:0/1387134396 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b8915700 1 --2- 172.21.15.89:0/1387134396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4133420 0x7fb6b41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b37fe700 1 -- 172.21.15.89:0/1387134396 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187820842 0 0) 0x7fb6b406b990 con 0x7fb6b4134260 2023-09-30T22:22:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b37fe700 1 -- 172.21.15.89:0/1387134396 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a40035b0 con 0x7fb6b4134260 2023-09-30T22:22:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b8915700 1 -- 172.21.15.89:0/1387134396 >> v2:172.21.15.89:3300/0 conn(0x7fb6b4133e30 msgr2=0x7fb6b412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b8915700 1 --2- 172.21.15.89:0/1387134396 >> v2:172.21.15.89:3300/0 conn(0x7fb6b4133e30 0x7fb6b412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b8915700 1 -- 172.21.15.89:0/1387134396 >> v1:172.21.15.89:6789/0 conn(0x7fb6b4134260 legacy=0x7fb6b41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.457+0000 7fb6b8915700 1 -- 172.21.15.89:0/1387134396 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b40a0030 con 0x7fb6b4133420 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.458+0000 7fb6b8915700 1 --2- 172.21.15.89:0/1387134396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4133420 0x7fb6b41337f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009840 tx=0x7fb6a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c08d0134d0638cac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.458+0000 7fb6b37fe700 1 -- 172.21.15.89:0/1387134396 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8011830 con 0x7fb6b4133420 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.458+0000 7fb6b37fe700 1 -- 172.21.15.89:0/1387134396 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a8011990 con 0x7fb6b4133420 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.458+0000 7fb6b37fe700 1 -- 172.21.15.89:0/1387134396 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8011d20 con 0x7fb6b4133420 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.459+0000 7fb6bab79700 1 -- 172.21.15.89:0/1387134396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4133420 msgr2=0x7fb6b41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.459+0000 7fb6bab79700 1 --2- 172.21.15.89:0/1387134396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4133420 0x7fb6b41337f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009840 tx=0x7fb6a8009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.459+0000 7fb6bab79700 1 -- 172.21.15.89:0/1387134396 shutdown_connections 2023-09-30T22:22:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.459+0000 7fb6bab79700 1 --2- 172.21.15.89:0/1387134396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4133420 0x7fb6b41337f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.459+0000 7fb6bab79700 1 --2- 172.21.15.89:0/1387134396 >> v2:172.21.15.89:3300/0 conn(0x7fb6b4133e30 0x7fb6b412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.459+0000 7fb6bab79700 1 -- 172.21.15.89:0/1387134396 >> 172.21.15.89:0/1387134396 conn(0x7fb6b40a4210 msgr2=0x7fb6b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.459+0000 7fb6bab79700 1 -- 172.21.15.89:0/1387134396 shutdown_connections 2023-09-30T22:22:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.459+0000 7fb6bab79700 1 -- 172.21.15.89:0/1387134396 wait complete. 2023-09-30T22:22:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.460+0000 7fb6bab79700 1 Processor -- start 2023-09-30T22:22:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.460+0000 7fb6bab79700 1 -- start start 2023-09-30T22:22:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6bab79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6b4133420 0x7fb6b409c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6bab79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6b4133e30 0x7fb6b409ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6bab79700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4134260 0x7fb6b409cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6bab79700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b4120ec0 con 0x7fb6b4133e30 2023-09-30T22:22:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6bab79700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b412d6f0 con 0x7fb6b4134260 2023-09-30T22:22:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6bab79700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b4120350 con 0x7fb6b4133420 2023-09-30T22:22:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6b4133e30 0x7fb6b409ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6b4133e30 0x7fb6b409ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49042/0 (socket says 172.21.15.89:49042) 2023-09-30T22:22:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6b3fff700 1 -- 172.21.15.89:0/1167266287 learned_addr learned my addr 172.21.15.89:0/1167266287 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6b9116700 1 --2- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4134260 0x7fb6b409cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.461+0000 7fb6b3fff700 1 -- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6b4133420 msgr2=0x7fb6b409c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6b3fff700 1 --2- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6b4133420 0x7fb6b409c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6b3fff700 1 -- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4134260 msgr2=0x7fb6b409cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6b3fff700 1 --2- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4134260 0x7fb6b409cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6b3fff700 1 -- 172.21.15.89:0/1167266287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80094a0 con 0x7fb6b4133e30 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6b3fff700 1 --2- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6b4133e30 0x7fb6b409ca90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0013f40 tx=0x7fb6a0013f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6b17fa700 1 -- 172.21.15.89:0/1167266287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0014d10 con 0x7fb6b4133e30 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6b17fa700 1 -- 172.21.15.89:0/1167266287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0014e90 con 0x7fb6b4133e30 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b409e2d0 con 0x7fb6b4133e30 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6b17fa700 1 -- 172.21.15.89:0/1167266287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0011310 con 0x7fb6b4133e30 2023-09-30T22:22:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.462+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b41207e0 con 0x7fb6b4133e30 2023-09-30T22:22:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.463+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b4120a90 con 0x7fb6b4133e30 2023-09-30T22:22:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.463+0000 7fb6b17fa700 1 -- 172.21.15.89:0/1167266287 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6a0011470 con 0x7fb6b4133e30 2023-09-30T22:22:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.464+0000 7fb6b17fa700 1 -- 172.21.15.89:0/1167266287 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb6a0047380 con 0x7fb6b4133e30 2023-09-30T22:22:17.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.472+0000 7fb6b17fa700 1 -- 172.21.15.89:0/1167266287 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb6a0019890 con 0x7fb6b4133e30 2023-09-30T22:22:17.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.576+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3e1406b9e0 con 0x7f3e1406fcf0 2023-09-30T22:22:17.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.576+0000 7f3dfb7fe700 1 -- 172.21.15.89:0/3357456556 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e04021070 con 0x7f3e1406fcf0 2023-09-30T22:22:17.577 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.578+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e1406fcf0 msgr2=0x7f3e141282f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.578+0000 7f3e196fd700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e1406fcf0 0x7f3e141282f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400e900 tx=0x7f3e0400ec10 comp rx=0 tx=0).stop 2023-09-30T22:22:17.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.578+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 shutdown_connections 2023-09-30T22:22:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.578+0000 7f3e196fd700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e14130e50 0x7f3e14128d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.578+0000 7f3e196fd700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e1406fcf0 0x7f3e141282f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.578+0000 7f3e196fd700 1 --2- 172.21.15.89:0/3357456556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e14130540 0x7f3e14128830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.579+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 >> 172.21.15.89:0/3357456556 conn(0x7f3e140a2aa0 msgr2=0x7f3e14126fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.579+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 shutdown_connections 2023-09-30T22:22:17.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.579+0000 7f3e196fd700 1 -- 172.21.15.89:0/3357456556 wait complete. 2023-09-30T22:22:17.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.630+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd30806b250 con 0x7fd308136820 2023-09-30T22:22:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.630+0000 7fd304ff9700 1 -- 172.21.15.89:0/2351224489 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd2fc021020 con 0x7fd308136820 2023-09-30T22:22:17.632 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.632+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308136820 msgr2=0x7fd30812d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.632+0000 7fd30e27d700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308136820 0x7fd30812d140 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00e980 tx=0x7fd2fc00ec90 comp rx=0 tx=0).stop 2023-09-30T22:22:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.633+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 shutdown_connections 2023-09-30T22:22:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.633+0000 7fd30e27d700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd308139140 0x7fd30812d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.633+0000 7fd30e27d700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd308133a80 0x7fd30812cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.633+0000 7fd30e27d700 1 --2- 172.21.15.89:0/2351224489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308136820 0x7fd30812d140 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.633+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 >> 172.21.15.89:0/2351224489 conn(0x7fd3080a4210 msgr2=0x7fd308137450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.633+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 shutdown_connections 2023-09-30T22:22:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.633+0000 7fd30e27d700 1 -- 172.21.15.89:0/2351224489 wait complete. 2023-09-30T22:22:17.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.676+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbf04074ea0 con 0x7fbf04136f50 2023-09-30T22:22:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.676+0000 7fbf017fa700 1 -- 172.21.15.89:0/1496489193 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbefc01d400 con 0x7fbf04136f50 2023-09-30T22:22:17.677 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:17.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf04136f50 msgr2=0x7fbf04118d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf04136f50 0x7fbf04118d40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00efc0 tx=0x7fbefc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 shutdown_connections 2023-09-30T22:22:17.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf04136f50 0x7fbf04118d40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbf04131810 0x7fbf041c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 --2- 172.21.15.89:0/1496489193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf04134630 0x7fbf041c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 >> 172.21.15.89:0/1496489193 conn(0x7fbf040a2aa0 msgr2=0x7fbf0409f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 shutdown_connections 2023-09-30T22:22:17.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.678+0000 7fbf0afe7700 1 -- 172.21.15.89:0/1496489193 wait complete. 2023-09-30T22:22:17.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.689+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb6b4074ea0 con 0x7fb6b4133e30 2023-09-30T22:22:17.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.690+0000 7fb6b17fa700 1 -- 172.21.15.89:0/1167266287 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6a0047020 con 0x7fb6b4133e30 2023-09-30T22:22:17.690 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:17.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6b4133e30 msgr2=0x7fb6b409ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 --2- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6b4133e30 0x7fb6b409ca90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0013f40 tx=0x7fb6a0013f70 comp rx=0 tx=0).stop 2023-09-30T22:22:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 shutdown_connections 2023-09-30T22:22:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 --2- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6b4134260 0x7fb6b409cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 --2- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb6b4133420 0x7fb6b409c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 --2- 172.21.15.89:0/1167266287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6b4133e30 0x7fb6b409ca90 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 >> 172.21.15.89:0/1167266287 conn(0x7fb6b40a4210 msgr2=0x7fb6b4130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:17.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 shutdown_connections 2023-09-30T22:22:17.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:17.692+0000 7fb6bab79700 1 -- 172.21.15.89:0/1167266287 wait complete. 2023-09-30T22:22:17.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:18.593 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:18.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:18.692 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:18.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:18.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.736+0000 7faa9726e700 1 Processor -- start 2023-09-30T22:22:18.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.736+0000 7faa9726e700 1 -- start start 2023-09-30T22:22:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9726e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa9012cf60 0x7faa9006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9726e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa9012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9726e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa90136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9726e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9012d330 con 0x7faa9012cf60 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9726e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9012cd90 con 0x7faa90130260 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9726e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa901306d0 con 0x7faa9012f8b0 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa94809700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa9012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9580b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa90136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9500a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa9012cf60 0x7faa9006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa94809700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa9012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57008/0 (socket says 172.21.15.89:57008) 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9580b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa90136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34188/0 (socket says 172.21.15.89:34188) 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa94809700 1 -- 172.21.15.89:0/2361841131 learned_addr learned my addr 172.21.15.89:0/2361841131 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa94809700 1 -- 172.21.15.89:0/2361841131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 msgr2=0x7faa90136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa94809700 1 --2- 172.21.15.89:0/2361841131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa90136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa94809700 1 -- 172.21.15.89:0/2361841131 >> v2:172.21.15.89:3300/0 conn(0x7faa9012cf60 msgr2=0x7faa9006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa94809700 1 --2- 172.21.15.89:0/2361841131 >> v2:172.21.15.89:3300/0 conn(0x7faa9012cf60 0x7faa9006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa94809700 1 -- 172.21.15.89:0/2361841131 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa90071780 con 0x7faa9012f8b0 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.737+0000 7faa9580b700 1 --2- 172.21.15.89:0/2361841131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa90136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.738+0000 7faa94809700 1 --2- 172.21.15.89:0/2361841131 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa9012fd20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7faa8000a9a0 tx=0x7faa8000acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d506b2c6029760 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.738+0000 7faa87fff700 1 -- 172.21.15.89:0/2361841131 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80004230 con 0x7faa9012f8b0 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.738+0000 7faa87fff700 1 -- 172.21.15.89:0/2361841131 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa80004390 con 0x7faa9012f8b0 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.738+0000 7faa87fff700 1 -- 172.21.15.89:0/2361841131 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa800036a0 con 0x7faa9012f8b0 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 -- 172.21.15.89:0/2361841131 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 msgr2=0x7faa9012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 --2- 172.21.15.89:0/2361841131 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa9012fd20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7faa8000a9a0 tx=0x7faa8000acb0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 -- 172.21.15.89:0/2361841131 shutdown_connections 2023-09-30T22:22:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 --2- 172.21.15.89:0/2361841131 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa90136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 --2- 172.21.15.89:0/2361841131 >> v2:172.21.15.89:3300/0 conn(0x7faa9012cf60 0x7faa9006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 --2- 172.21.15.89:0/2361841131 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa9012fd20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 -- 172.21.15.89:0/2361841131 >> 172.21.15.89:0/2361841131 conn(0x7faa900a2aa0 msgr2=0x7faa900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 -- 172.21.15.89:0/2361841131 shutdown_connections 2023-09-30T22:22:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.739+0000 7faa9726e700 1 -- 172.21.15.89:0/2361841131 wait complete. 2023-09-30T22:22:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9726e700 1 Processor -- start 2023-09-30T22:22:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9726e700 1 -- start start 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9726e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9012cf60 0x7faa901c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9726e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa901c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9726e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa901c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9726e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9009ce90 con 0x7faa9012cf60 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9726e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9009cc60 con 0x7faa90130260 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9726e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa9009e5d0 con 0x7faa9012f8b0 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9580b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa901c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9580b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa901c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34200/0 (socket says 172.21.15.89:34200) 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.740+0000 7faa9580b700 1 -- 172.21.15.89:0/800366816 learned_addr learned my addr 172.21.15.89:0/800366816 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa94809700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa901c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa9500a700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9012cf60 0x7faa901c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa9580b700 1 -- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 msgr2=0x7faa901c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa9580b700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa901c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa9580b700 1 -- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9012cf60 msgr2=0x7faa901c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa9580b700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9012cf60 0x7faa901c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa9580b700 1 -- 172.21.15.89:0/800366816 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88009a70 con 0x7faa90130260 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa9500a700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9012cf60 0x7faa901c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa9580b700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa901c4a30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7faa8800ef90 tx=0x7faa8800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa85ffb700 1 -- 172.21.15.89:0/800366816 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8800cd40 con 0x7faa90130260 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa85ffb700 1 -- 172.21.15.89:0/800366816 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8800f040 con 0x7faa90130260 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.741+0000 7faa85ffb700 1 -- 172.21.15.89:0/800366816 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880146c0 con 0x7faa90130260 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.742+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8000a600 con 0x7faa90130260 2023-09-30T22:22:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.742+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9012daa0 con 0x7faa90130260 2023-09-30T22:22:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.743+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa901191b0 con 0x7faa90130260 2023-09-30T22:22:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.743+0000 7faa85ffb700 1 -- 172.21.15.89:0/800366816 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa88014820 con 0x7faa90130260 2023-09-30T22:22:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.744+0000 7faa85ffb700 1 -- 172.21.15.89:0/800366816 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa880402e0 con 0x7faa90130260 2023-09-30T22:22:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.752+0000 7faa85ffb700 1 -- 172.21.15.89:0/800366816 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa88007d90 con 0x7faa90130260 2023-09-30T22:22:18.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.784+0000 7fa70d0ec700 1 Processor -- start 2023-09-30T22:22:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.784+0000 7fa70d0ec700 1 -- start start 2023-09-30T22:22:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa70d0ec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa70806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa70d0ec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa70806ee80 0x7fa708134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa70d0ec700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa708070ee0 con 0x7fa70809eca0 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa70d0ec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa708071480 con 0x7fa70806ee80 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa70d0ec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa708071b90 con 0x7fa70806e570 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa706d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa70806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa706d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa70806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57018/0 (socket says 172.21.15.89:57018) 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa706d9d700 1 -- 172.21.15.89:0/551391153 learned_addr learned my addr 172.21.15.89:0/551391153 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa70759e700 1 --2- 172.21.15.89:0/551391153 >> v2:172.21.15.89:3300/0 conn(0x7fa70806ee80 0x7fa708134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa706d9d700 1 -- 172.21.15.89:0/551391153 >> v2:172.21.15.89:3300/0 conn(0x7fa70806ee80 msgr2=0x7fa708134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa706d9d700 1 --2- 172.21.15.89:0/551391153 >> v2:172.21.15.89:3300/0 conn(0x7fa70806ee80 0x7fa708134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa706d9d700 1 -- 172.21.15.89:0/551391153 >> v1:172.21.15.89:6789/0 conn(0x7fa70809eca0 legacy=0x7fa708134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.785+0000 7fa706d9d700 1 -- 172.21.15.89:0/551391153 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7080711a0 con 0x7fa70806e570 2023-09-30T22:22:18.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.786+0000 7fa706d9d700 1 --2- 172.21.15.89:0/551391153 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa70806e940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80098f0 tx=0x7fa6f8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22602d33272320a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:18.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.786+0000 7fa705d9b700 1 -- 172.21.15.89:0/551391153 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f800b8a0 con 0x7fa70806e570 2023-09-30T22:22:18.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.786+0000 7fa705d9b700 1 -- 172.21.15.89:0/551391153 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f800ba00 con 0x7fa70806e570 2023-09-30T22:22:18.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.786+0000 7fa705d9b700 1 -- 172.21.15.89:0/551391153 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f800bd90 con 0x7fa70806e570 2023-09-30T22:22:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.787+0000 7fa70d0ec700 1 -- 172.21.15.89:0/551391153 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 msgr2=0x7fa70806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.787+0000 7fa70d0ec700 1 --2- 172.21.15.89:0/551391153 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa70806e940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80098f0 tx=0x7fa6f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.787+0000 7fa70d0ec700 1 -- 172.21.15.89:0/551391153 shutdown_connections 2023-09-30T22:22:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.787+0000 7fa70d0ec700 1 --2- 172.21.15.89:0/551391153 >> v2:172.21.15.89:3300/0 conn(0x7fa70806ee80 0x7fa708134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.787+0000 7fa70d0ec700 1 --2- 172.21.15.89:0/551391153 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa70806e940 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.787+0000 7fa70d0ec700 1 -- 172.21.15.89:0/551391153 >> 172.21.15.89:0/551391153 conn(0x7fa7080a2aa0 msgr2=0x7fa7080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.787+0000 7fa70d0ec700 1 -- 172.21.15.89:0/551391153 shutdown_connections 2023-09-30T22:22:18.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.787+0000 7fa70d0ec700 1 -- 172.21.15.89:0/551391153 wait complete. 2023-09-30T22:22:18.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.788+0000 7fa70d0ec700 1 Processor -- start 2023-09-30T22:22:18.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70d0ec700 1 -- start start 2023-09-30T22:22:18.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70d0ec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa708134780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70d0ec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa70806ee80 0x7fa708134cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70d0ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7081389f0 0x7fa708138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70d0ec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70809e920 con 0x7fa7081389f0 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70d0ec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70813b4e0 con 0x7fa70806ee80 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70d0ec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa708135900 con 0x7fa70806e570 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa706d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa708134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa706d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa708134780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57034/0 (socket says 172.21.15.89:57034) 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa706d9d700 1 -- 172.21.15.89:0/2406722079 learned_addr learned my addr 172.21.15.89:0/2406722079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70659c700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa70806ee80 0x7fa708134cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.789+0000 7fa70759e700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7081389f0 0x7fa708138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.790+0000 7fa70659c700 1 -- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 msgr2=0x7fa708134780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.790+0000 7fa70659c700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa708134780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.790+0000 7fa70659c700 1 -- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7081389f0 msgr2=0x7fa708138e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.790+0000 7fa70659c700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7081389f0 0x7fa708138e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.790+0000 7fa70659c700 1 -- 172.21.15.89:0/2406722079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8009550 con 0x7fa70806ee80 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.790+0000 7fa706d9d700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa708134780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.790+0000 7fa70659c700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa70806ee80 0x7fa708134cc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc000f60 tx=0x7fa6fc00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.790+0000 7fa6ef7fe700 1 -- 172.21.15.89:0/2406722079 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0050d0 con 0x7fa70806ee80 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.791+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70809e220 con 0x7fa70806ee80 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.791+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa708139f10 con 0x7fa70806ee80 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.791+0000 7fa6ef7fe700 1 -- 172.21.15.89:0/2406722079 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc005230 con 0x7fa70806ee80 2023-09-30T22:22:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.791+0000 7fa6ef7fe700 1 -- 172.21.15.89:0/2406722079 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc003680 con 0x7fa70806ee80 2023-09-30T22:22:18.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.792+0000 7fa6ef7fe700 1 -- 172.21.15.89:0/2406722079 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6fc0037e0 con 0x7fa70806ee80 2023-09-30T22:22:18.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.792+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e80052e0 con 0x7fa70806ee80 2023-09-30T22:22:18.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.792+0000 7fa6ef7fe700 1 -- 172.21.15.89:0/2406722079 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa6fc03e930 con 0x7fa70806ee80 2023-09-30T22:22:18.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.796+0000 7fa6ef7fe700 1 -- 172.21.15.89:0/2406722079 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6fc011590 con 0x7fa70806ee80 2023-09-30T22:22:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.826+0000 7fa6f9147700 1 Processor -- start 2023-09-30T22:22:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.826+0000 7fa6f9147700 1 -- start start 2023-09-30T22:22:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f9147700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4133540 0x7fa6f4135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f9147700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f4138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f9147700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f40716a0 con 0x7fa6f4130540 2023-09-30T22:22:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f9147700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4071290 con 0x7fa6f4133540 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f9147700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4130340 con 0x7fa6f4135e60 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4133540 0x7fa6f4135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4133540 0x7fa6f4135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34226/0 (socket says 172.21.15.89:34226) 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f259c700 1 -- 172.21.15.89:0/3449981588 learned_addr learned my addr 172.21.15.89:0/3449981588 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f359e700 1 --2- 172.21.15.89:0/3449981588 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f4138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f1d9b700 1 -- 172.21.15.89:0/3449981588 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227408832 0 0) 0x7fa6f40716a0 con 0x7fa6f4130540 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f1d9b700 1 -- 172.21.15.89:0/3449981588 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6dc0035b0 con 0x7fa6f4130540 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f259c700 1 -- 172.21.15.89:0/3449981588 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 msgr2=0x7fa6f4138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f259c700 1 --2- 172.21.15.89:0/3449981588 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f4138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f259c700 1 -- 172.21.15.89:0/3449981588 >> v1:172.21.15.89:6789/0 conn(0x7fa6f4130540 legacy=0x7fa6f4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f259c700 1 -- 172.21.15.89:0/3449981588 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4070b80 con 0x7fa6f4133540 2023-09-30T22:22:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.827+0000 7fa6f259c700 1 --2- 172.21.15.89:0/3449981588 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4133540 0x7fa6f4135920 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800bbc0 tx=0x7fa6e800bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cbd58d6b3321248 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f1d9b700 1 -- 172.21.15.89:0/3449981588 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800dd30 con 0x7fa6f4133540 2023-09-30T22:22:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f1d9b700 1 -- 172.21.15.89:0/3449981588 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e800de90 con 0x7fa6f4133540 2023-09-30T22:22:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f1d9b700 1 -- 172.21.15.89:0/3449981588 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8010690 con 0x7fa6f4133540 2023-09-30T22:22:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f9147700 1 -- 172.21.15.89:0/3449981588 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4133540 msgr2=0x7fa6f4135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f9147700 1 --2- 172.21.15.89:0/3449981588 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4133540 0x7fa6f4135920 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800bbc0 tx=0x7fa6e800bed0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f9147700 1 -- 172.21.15.89:0/3449981588 shutdown_connections 2023-09-30T22:22:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f9147700 1 --2- 172.21.15.89:0/3449981588 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f4138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f9147700 1 --2- 172.21.15.89:0/3449981588 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4133540 0x7fa6f4135920 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f9147700 1 -- 172.21.15.89:0/3449981588 >> 172.21.15.89:0/3449981588 conn(0x7fa6f40a2aa0 msgr2=0x7fa6f40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f9147700 1 -- 172.21.15.89:0/3449981588 shutdown_connections 2023-09-30T22:22:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.828+0000 7fa6f9147700 1 -- 172.21.15.89:0/3449981588 wait complete. 2023-09-30T22:22:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.830+0000 7fa6f9147700 1 Processor -- start 2023-09-30T22:22:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.830+0000 7fa6f9147700 1 -- start start 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f9147700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4130540 0x7fa6f409c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f9147700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f4133540 0x7fa6f409cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f9147700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f409d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f9147700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f411a220 con 0x7fa6f4133540 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f9147700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40718e0 con 0x7fa6f4130540 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f9147700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f41359c0 con 0x7fa6f4135e60 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f359e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f409d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f4133540 0x7fa6f409cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f359e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f409d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57054/0 (socket says 172.21.15.89:57054) 2023-09-30T22:22:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f359e700 1 -- 172.21.15.89:0/120288796 learned_addr learned my addr 172.21.15.89:0/120288796 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f2d9d700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4130540 0x7fa6f409c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f259c700 1 -- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 msgr2=0x7fa6f409d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f259c700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f409d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f259c700 1 -- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4130540 msgr2=0x7fa6f409c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f259c700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4130540 0x7fa6f409c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f259c700 1 -- 172.21.15.89:0/120288796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e800b7f0 con 0x7fa6f4133540 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f359e700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f409d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.831+0000 7fa6f2d9d700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4130540 0x7fa6f409c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.832+0000 7fa6f259c700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f4133540 0x7fa6f409cbf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4070da0 tx=0x7fa6e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.832+0000 7fa6db7fe700 1 -- 172.21.15.89:0/120288796 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8003ab0 con 0x7fa6f4133540 2023-09-30T22:22:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.832+0000 7fa6db7fe700 1 -- 172.21.15.89:0/120288796 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8003c10 con 0x7fa6f4133540 2023-09-30T22:22:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.832+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f4119e70 con 0x7fa6f4133540 2023-09-30T22:22:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.832+0000 7fa6db7fe700 1 -- 172.21.15.89:0/120288796 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8010470 con 0x7fa6f4133540 2023-09-30T22:22:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.832+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f411ae70 con 0x7fa6f4133540 2023-09-30T22:22:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.833+0000 7fa6db7fe700 1 -- 172.21.15.89:0/120288796 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6e8024450 con 0x7fa6f4133540 2023-09-30T22:22:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.833+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f411a060 con 0x7fa6f4133540 2023-09-30T22:22:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.833+0000 7fa6db7fe700 1 -- 172.21.15.89:0/120288796 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa6e8040880 con 0x7fa6f4133540 2023-09-30T22:22:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.842+0000 7fa6db7fe700 1 -- 172.21.15.89:0/120288796 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6e8006bd0 con 0x7fa6f4133540 2023-09-30T22:22:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f3859215700 1 Processor -- start 2023-09-30T22:22:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f3859215700 1 -- start start 2023-09-30T22:22:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f3859215700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38540a0a30 0x7f38540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f3859215700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409d630 0x7f38540a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f3859215700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38540a2a70 con 0x7f3854133930 2023-09-30T22:22:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f3859215700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38540a01d0 con 0x7f38540a0a30 2023-09-30T22:22:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f3859215700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f385409e2b0 con 0x7f385409d630 2023-09-30T22:22:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f385359e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3854133930 0x7f3854133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37084/0 (socket says 172.21.15.89:37084) 2023-09-30T22:22:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f385359e700 1 -- 172.21.15.89:0/356037254 learned_addr learned my addr 172.21.15.89:0/356037254 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.867+0000 7f3852d9d700 1 --2- 172.21.15.89:0/356037254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38540a0a30 0x7f38540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3851d9b700 1 -- 172.21.15.89:0/356037254 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269429135 0 0) 0x7f38540a2a70 con 0x7f3854133930 2023-09-30T22:22:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3852d9d700 1 -- 172.21.15.89:0/356037254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409d630 msgr2=0x7f38540a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3852d9d700 1 --2- 172.21.15.89:0/356037254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409d630 0x7f38540a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3852d9d700 1 -- 172.21.15.89:0/356037254 >> v1:172.21.15.89:6789/0 conn(0x7f3854133930 legacy=0x7f3854133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3852d9d700 1 -- 172.21.15.89:0/356037254 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385409dc50 con 0x7f38540a0a30 2023-09-30T22:22:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3852d9d700 1 --2- 172.21.15.89:0/356037254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38540a0a30 0x7f38540a0e00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3844009870 tx=0x7f3844009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf223f1bc6607ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3851d9b700 1 -- 172.21.15.89:0/356037254 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844011830 con 0x7f38540a0a30 2023-09-30T22:22:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3851d9b700 1 -- 172.21.15.89:0/356037254 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844011990 con 0x7f38540a0a30 2023-09-30T22:22:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.868+0000 7f3851d9b700 1 -- 172.21.15.89:0/356037254 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844011d20 con 0x7f38540a0a30 2023-09-30T22:22:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.869+0000 7f3859215700 1 -- 172.21.15.89:0/356037254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38540a0a30 msgr2=0x7f38540a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.869+0000 7f3859215700 1 --2- 172.21.15.89:0/356037254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38540a0a30 0x7f38540a0e00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3844009870 tx=0x7f3844009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.869+0000 7f3859215700 1 -- 172.21.15.89:0/356037254 shutdown_connections 2023-09-30T22:22:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.869+0000 7f3859215700 1 --2- 172.21.15.89:0/356037254 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38540a0a30 0x7f38540a0e00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.869+0000 7f3859215700 1 --2- 172.21.15.89:0/356037254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409d630 0x7f38540a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.869+0000 7f3859215700 1 -- 172.21.15.89:0/356037254 >> 172.21.15.89:0/356037254 conn(0x7f38540a3050 msgr2=0x7f38540a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.869+0000 7f3859215700 1 -- 172.21.15.89:0/356037254 shutdown_connections 2023-09-30T22:22:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.869+0000 7f3859215700 1 -- 172.21.15.89:0/356037254 wait complete. 2023-09-30T22:22:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.870+0000 7f3859215700 1 Processor -- start 2023-09-30T22:22:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.870+0000 7f3859215700 1 -- start start 2023-09-30T22:22:18.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f3859215700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d630 0x7f38541c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f3859215700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38540a0a30 0x7f38541c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f3859215700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854133930 0x7f38541c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f3859215700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854120800 con 0x7f385409d630 2023-09-30T22:22:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f3859215700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3854120610 con 0x7f3854133930 2023-09-30T22:22:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f3859215700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3854120ee0 con 0x7f38540a0a30 2023-09-30T22:22:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f385259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38540a0a30 0x7f38541c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f3852d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d630 0x7f38541c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f385259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38540a0a30 0x7f38541c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57080/0 (socket says 172.21.15.89:57080) 2023-09-30T22:22:18.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f385359e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854133930 0x7f38541c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:18.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.871+0000 7f385259c700 1 -- 172.21.15.89:0/572303458 learned_addr learned my addr 172.21.15.89:0/572303458 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:18.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f385359e700 1 -- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38540a0a30 msgr2=0x7f38541c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f385359e700 1 --2- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38540a0a30 0x7f38541c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f385359e700 1 -- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d630 msgr2=0x7f38541c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f385359e700 1 --2- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d630 0x7f38541c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f385359e700 1 -- 172.21.15.89:0/572303458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38440094a0 con 0x7f3854133930 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f385259c700 1 --2- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38540a0a30 0x7f38541c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f385359e700 1 --2- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854133930 0x7f38541c8aa0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f384c00cbe0 tx=0x7f384c00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f383b7fe700 1 -- 172.21.15.89:0/572303458 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00aa40 con 0x7f3854133930 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f383b7fe700 1 -- 172.21.15.89:0/572303458 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00abc0 con 0x7f3854133930 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f383b7fe700 1 -- 172.21.15.89:0/572303458 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c0070d0 con 0x7f3854133930 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3854121cd0 con 0x7f3854133930 2023-09-30T22:22:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.872+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854122240 con 0x7f3854133930 2023-09-30T22:22:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.873+0000 7f383b7fe700 1 -- 172.21.15.89:0/572303458 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f384c007230 con 0x7f3854133930 2023-09-30T22:22:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.873+0000 7f383b7fe700 1 -- 172.21.15.89:0/572303458 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f384c03f130 con 0x7f3854133930 2023-09-30T22:22:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.873+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854121e60 con 0x7f3854133930 2023-09-30T22:22:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.878+0000 7f383b7fe700 1 -- 172.21.15.89:0/572303458 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f384c018030 con 0x7f3854133930 2023-09-30T22:22:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.972+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faa90074ea0 con 0x7faa90130260 2023-09-30T22:22:18.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.972+0000 7faa85ffb700 1 -- 172.21.15.89:0/800366816 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa88018020 con 0x7faa90130260 2023-09-30T22:22:18.973 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.975+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 msgr2=0x7faa901c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.975+0000 7faa9726e700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa901c4a30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7faa8800ef90 tx=0x7faa8800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.975+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 shutdown_connections 2023-09-30T22:22:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.975+0000 7faa9726e700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa90130260 0x7faa901c4a30 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.975+0000 7faa9726e700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9012cf60 0x7faa901c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.975+0000 7faa9726e700 1 --2- 172.21.15.89:0/800366816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9012f8b0 0x7faa901c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.975+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 >> 172.21.15.89:0/800366816 conn(0x7faa900a2aa0 msgr2=0x7faa901b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.976+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 shutdown_connections 2023-09-30T22:22:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.976+0000 7faa9726e700 1 -- 172.21.15.89:0/800366816 wait complete. 2023-09-30T22:22:18.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.996+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa6e8005150 con 0x7fa70806ee80 2023-09-30T22:22:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.996+0000 7fa6ef7fe700 1 -- 172.21.15.89:0/2406722079 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6fc0192c0 con 0x7fa70806ee80 2023-09-30T22:22:18.997 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa70806ee80 msgr2=0x7fa708134cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa70806ee80 0x7fa708134cc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc000f60 tx=0x7fa6fc00c730 comp rx=0 tx=0).stop 2023-09-30T22:22:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 shutdown_connections 2023-09-30T22:22:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7081389f0 0x7fa708138e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa70806e570 0x7fa708134780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:18.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 --2- 172.21.15.89:0/2406722079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa70806ee80 0x7fa708134cc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 >> 172.21.15.89:0/2406722079 conn(0x7fa7080a2aa0 msgr2=0x7fa7080a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 shutdown_connections 2023-09-30T22:22:19.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:18.998+0000 7fa70d0ec700 1 -- 172.21.15.89:0/2406722079 wait complete. 2023-09-30T22:22:19.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:19.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.055+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa6f4074ea0 con 0x7fa6f4133540 2023-09-30T22:22:19.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.055+0000 7fa6db7fe700 1 -- 172.21.15.89:0/120288796 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6e8029070 con 0x7fa6f4133540 2023-09-30T22:22:19.056 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:19.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f4133540 msgr2=0x7fa6f409cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f4133540 0x7fa6f409cbf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4070da0 tx=0x7fa6e8004750 comp rx=0 tx=0).stop 2023-09-30T22:22:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 shutdown_connections 2023-09-30T22:22:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa6f4135e60 0x7fa6f409d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa6f4130540 0x7fa6f409c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 --2- 172.21.15.89:0/120288796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f4133540 0x7fa6f409cbf0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 >> 172.21.15.89:0/120288796 conn(0x7fa6f40a2aa0 msgr2=0x7fa6f4131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 shutdown_connections 2023-09-30T22:22:19.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.057+0000 7fa6f9147700 1 -- 172.21.15.89:0/120288796 wait complete. 2023-09-30T22:22:19.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:19.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.088+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3854074770 con 0x7f3854133930 2023-09-30T22:22:19.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.088+0000 7f383b7fe700 1 -- 172.21.15.89:0/572303458 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f384c0074e0 con 0x7f3854133930 2023-09-30T22:22:19.089 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854133930 msgr2=0x7f38541c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 --2- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854133930 0x7f38541c8aa0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f384c00cbe0 tx=0x7f384c00cef0 comp rx=0 tx=0).stop 2023-09-30T22:22:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 shutdown_connections 2023-09-30T22:22:19.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 --2- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854133930 0x7f38541c8aa0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 --2- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d630 0x7f38541c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 --2- 172.21.15.89:0/572303458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38540a0a30 0x7f38541c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 >> 172.21.15.89:0/572303458 conn(0x7f38540a3050 msgr2=0x7f385409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 shutdown_connections 2023-09-30T22:22:19.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:19.090+0000 7f3859215700 1 -- 172.21.15.89:0/572303458 wait complete. 2023-09-30T22:22:19.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:19.994 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:20.011 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:20.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:20.104 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:20.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.134+0000 7f7c5d81d700 1 Processor -- start 2023-09-30T22:22:20.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.134+0000 7f7c5d81d700 1 -- start start 2023-09-30T22:22:20.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c5d81d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7c580a0a30 0x7f7c580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c5d81d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c58138d80 0x7f7c5813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c5d81d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c580a2a70 con 0x7f7c5809d630 2023-09-30T22:22:20.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c5d81d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c580a01d0 con 0x7f7c580a0a30 2023-09-30T22:22:20.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c5d81d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5809e2b0 con 0x7f7c58138d80 2023-09-30T22:22:20.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c577fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c58138d80 0x7f7c5813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c577fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c58138d80 0x7f7c5813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34262/0 (socket says 172.21.15.89:34262) 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c577fe700 1 -- 172.21.15.89:0/243338297 learned_addr learned my addr 172.21.15.89:0/243338297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.135+0000 7f7c56ffd700 1 --2- 172.21.15.89:0/243338297 >> v2:172.21.15.89:3300/0 conn(0x7f7c580a0a30 0x7f7c580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.136+0000 7f7c577fe700 1 -- 172.21.15.89:0/243338297 >> v2:172.21.15.89:3300/0 conn(0x7f7c580a0a30 msgr2=0x7f7c580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.136+0000 7f7c577fe700 1 --2- 172.21.15.89:0/243338297 >> v2:172.21.15.89:3300/0 conn(0x7f7c580a0a30 0x7f7c580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.136+0000 7f7c577fe700 1 -- 172.21.15.89:0/243338297 >> v1:172.21.15.89:6789/0 conn(0x7f7c5809d630 legacy=0x7f7c580a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.136+0000 7f7c577fe700 1 -- 172.21.15.89:0/243338297 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5809dc50 con 0x7f7c58138d80 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.136+0000 7f7c577fe700 1 --2- 172.21.15.89:0/243338297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c58138d80 0x7f7c5813b160 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009e50 tx=0x7f7c5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f06c7292fe1990a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.136+0000 7f7c55ffb700 1 -- 172.21.15.89:0/243338297 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000e070 con 0x7f7c58138d80 2023-09-30T22:22:20.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.136+0000 7f7c55ffb700 1 -- 172.21.15.89:0/243338297 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5000b910 con 0x7f7c58138d80 2023-09-30T22:22:20.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.136+0000 7f7c55ffb700 1 -- 172.21.15.89:0/243338297 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000bcc0 con 0x7f7c58138d80 2023-09-30T22:22:20.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.137+0000 7f7c5d81d700 1 -- 172.21.15.89:0/243338297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c58138d80 msgr2=0x7f7c5813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.137+0000 7f7c5d81d700 1 --2- 172.21.15.89:0/243338297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c58138d80 0x7f7c5813b160 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009e50 tx=0x7f7c5000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.137+0000 7f7c5d81d700 1 -- 172.21.15.89:0/243338297 shutdown_connections 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.137+0000 7f7c5d81d700 1 --2- 172.21.15.89:0/243338297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c58138d80 0x7f7c5813b160 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.137+0000 7f7c5d81d700 1 --2- 172.21.15.89:0/243338297 >> v2:172.21.15.89:3300/0 conn(0x7f7c580a0a30 0x7f7c580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.137+0000 7f7c5d81d700 1 -- 172.21.15.89:0/243338297 >> 172.21.15.89:0/243338297 conn(0x7f7c580a3050 msgr2=0x7f7c580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.137+0000 7f7c5d81d700 1 -- 172.21.15.89:0/243338297 shutdown_connections 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.137+0000 7f7c5d81d700 1 -- 172.21.15.89:0/243338297 wait complete. 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.139+0000 7f7c5d81d700 1 Processor -- start 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.139+0000 7f7c5d81d700 1 -- start start 2023-09-30T22:22:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.139+0000 7f7c5d81d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5809d630 0x7f7c58133770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.139+0000 7f7c5d81d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c580a0a30 0x7f7c58133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.139+0000 7f7c5d81d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c58138d80 0x7f7c581341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c5d81d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58136240 con 0x7f7c58138d80 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c5d81d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c581365f0 con 0x7f7c580a0a30 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c5d81d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58136f60 con 0x7f7c5809d630 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c58138d80 0x7f7c581341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c58138d80 0x7f7c581341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49102/0 (socket says 172.21.15.89:49102) 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 -- 172.21.15.89:0/2260588437 learned_addr learned my addr 172.21.15.89:0/2260588437 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c567fc700 1 --2- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c580a0a30 0x7f7c58133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 -- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5809d630 msgr2=0x7f7c58133770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 --2- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5809d630 0x7f7c58133770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 -- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c580a0a30 msgr2=0x7f7c58133cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 --2- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c580a0a30 0x7f7c58133cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 -- 172.21.15.89:0/2260588437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009b30 con 0x7f7c58138d80 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c577fe700 1 --2- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c58138d80 0x7f7c581341f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f7c50012000 tx=0x7f7c50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.140+0000 7f7c3ffff700 1 -- 172.21.15.89:0/2260588437 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500048b0 con 0x7f7c58138d80 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.141+0000 7f7c3ffff700 1 -- 172.21.15.89:0/2260588437 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500092e0 con 0x7f7c58138d80 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.141+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5813ad00 con 0x7f7c58138d80 2023-09-30T22:22:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.141+0000 7f7c3ffff700 1 -- 172.21.15.89:0/2260588437 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5001c630 con 0x7f7c58138d80 2023-09-30T22:22:20.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.141+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c581379d0 con 0x7f7c58138d80 2023-09-30T22:22:20.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.142+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5813b620 con 0x7f7c58138d80 2023-09-30T22:22:20.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.142+0000 7f7c3ffff700 1 -- 172.21.15.89:0/2260588437 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c5001c790 con 0x7f7c58138d80 2023-09-30T22:22:20.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.142+0000 7f7c3ffff700 1 -- 172.21.15.89:0/2260588437 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c50041680 con 0x7f7c58138d80 2023-09-30T22:22:20.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.147+0000 7f7c3ffff700 1 -- 172.21.15.89:0/2260588437 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c5001ce60 con 0x7f7c58138d80 2023-09-30T22:22:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.174+0000 7f6fb6b90700 1 Processor -- start 2023-09-30T22:22:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.174+0000 7f6fb6b90700 1 -- start start 2023-09-30T22:22:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6fb6b90700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0134630 0x7f6fb0136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6fb6b90700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6fb0136f50 0x7f6fb0139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6fb6b90700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb009b4f0 con 0x7f6fb0131810 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6fb6b90700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb01313c0 con 0x7f6fb0136f50 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6fb6b90700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb006e8c0 con 0x7f6fb0134630 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0134630 0x7f6fb0136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0134630 0x7f6fb0136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34282/0 (socket says 172.21.15.89:34282) 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faffff700 1 -- 172.21.15.89:0/1911964412 learned_addr learned my addr 172.21.15.89:0/1911964412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faffff700 1 -- 172.21.15.89:0/1911964412 >> v2:172.21.15.89:3300/0 conn(0x7f6fb0136f50 msgr2=0x7f6fb0139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faffff700 1 --2- 172.21.15.89:0/1911964412 >> v2:172.21.15.89:3300/0 conn(0x7f6fb0136f50 0x7f6fb0139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faffff700 1 -- 172.21.15.89:0/1911964412 >> v1:172.21.15.89:6789/0 conn(0x7f6fb0131810 legacy=0x7f6fb0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faffff700 1 -- 172.21.15.89:0/1911964412 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb0071d20 con 0x7f6fb0134630 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faffff700 1 --2- 172.21.15.89:0/1911964412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0134630 0x7f6fb0136a10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0098f0 tx=0x7f6f9c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca94d622389cac1d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faf7fe700 1 -- 172.21.15.89:0/1911964412 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c004230 con 0x7f6fb0134630 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faf7fe700 1 -- 172.21.15.89:0/1911964412 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c004390 con 0x7f6fb0134630 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.175+0000 7f6faf7fe700 1 -- 172.21.15.89:0/1911964412 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0036a0 con 0x7f6fb0134630 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.176+0000 7f6fb6b90700 1 -- 172.21.15.89:0/1911964412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0134630 msgr2=0x7f6fb0136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.176+0000 7f6fb6b90700 1 --2- 172.21.15.89:0/1911964412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0134630 0x7f6fb0136a10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0098f0 tx=0x7f6f9c009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.176+0000 7f6fb6b90700 1 -- 172.21.15.89:0/1911964412 shutdown_connections 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.176+0000 7f6fb6b90700 1 --2- 172.21.15.89:0/1911964412 >> v2:172.21.15.89:3300/0 conn(0x7f6fb0136f50 0x7f6fb0139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.176+0000 7f6fb6b90700 1 --2- 172.21.15.89:0/1911964412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0134630 0x7f6fb0136a10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.176+0000 7f6fb6b90700 1 -- 172.21.15.89:0/1911964412 >> 172.21.15.89:0/1911964412 conn(0x7f6fb00a2aa0 msgr2=0x7f6fb00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.176+0000 7f6fb6b90700 1 -- 172.21.15.89:0/1911964412 shutdown_connections 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.176+0000 7f6fb6b90700 1 -- 172.21.15.89:0/1911964412 wait complete. 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.177+0000 7f6fb6b90700 1 Processor -- start 2023-09-30T22:22:20.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.177+0000 7f6fb6b90700 1 -- start start 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb6b90700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb0131810 0x7f6fb0119450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb6b90700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb0134630 0x7f6fb0119990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb6b90700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0136f50 0x7f6fb01c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb6b90700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb011b5c0 con 0x7f6fb0134630 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb6b90700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb012dd00 con 0x7f6fb0136f50 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb6b90700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb00a1f80 con 0x7f6fb0131810 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0136f50 0x7f6fb01c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb492c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb0131810 0x7f6fb0119450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0136f50 0x7f6fb01c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34288/0 (socket says 172.21.15.89:34288) 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 -- 172.21.15.89:0/2669405307 learned_addr learned my addr 172.21.15.89:0/2669405307 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6faffff700 1 --2- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb0134630 0x7f6fb0119990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 -- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb0131810 msgr2=0x7f6fb0119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 --2- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb0131810 0x7f6fb0119450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 -- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb0134630 msgr2=0x7f6fb0119990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 --2- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb0134630 0x7f6fb0119990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 -- 172.21.15.89:0/2669405307 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c009550 con 0x7f6fb0136f50 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.178+0000 7f6fb512d700 1 --2- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0136f50 0x7f6fb01c6240 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800ba10 tx=0x7f6fa800bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.179+0000 7f6fad7fa700 1 -- 172.21.15.89:0/2669405307 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800d8e0 con 0x7f6fb0136f50 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.179+0000 7f6fad7fa700 1 -- 172.21.15.89:0/2669405307 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa800da40 con 0x7f6fb0136f50 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.179+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb011b7f0 con 0x7f6fb0136f50 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.179+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb011a3b0 con 0x7f6fb0136f50 2023-09-30T22:22:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.179+0000 7f6fad7fa700 1 -- 172.21.15.89:0/2669405307 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800ddf0 con 0x7f6fb0136f50 2023-09-30T22:22:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.180+0000 7f6fad7fa700 1 -- 172.21.15.89:0/2669405307 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fa8015420 con 0x7f6fb0136f50 2023-09-30T22:22:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.180+0000 7f6fad7fa700 1 -- 172.21.15.89:0/2669405307 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6fa8011070 con 0x7f6fb0136f50 2023-09-30T22:22:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.180+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f900052e0 con 0x7f6fb0136f50 2023-09-30T22:22:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.185+0000 7f6fad7fa700 1 -- 172.21.15.89:0/2669405307 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fa801c900 con 0x7f6fb0136f50 2023-09-30T22:22:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.245+0000 7fdd50b2c700 1 Processor -- start 2023-09-30T22:22:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.245+0000 7fdd50b2c700 1 -- start start 2023-09-30T22:22:20.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd50b2c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c1367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd50b2c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd4c06f140 0x7fdd4c138d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd50b2c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c099b50 con 0x7fdd4c066410 2023-09-30T22:22:20.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd50b2c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c0667e0 con 0x7fdd4c06f140 2023-09-30T22:22:20.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd50b2c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c099f60 con 0x7fdd4c06ed50 2023-09-30T22:22:20.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdd4c066410 0x7fdd4c06e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47408/0 (socket says 172.21.15.89:47408) 2023-09-30T22:22:20.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4a59c700 1 -- 172.21.15.89:0/1475709553 learned_addr learned my addr 172.21.15.89:0/1475709553 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:20.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd49d9b700 1 --2- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c1367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/1475709553 >> v2:172.21.15.89:3300/0 conn(0x7fdd4c06f140 0x7fdd4c138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4959a700 1 -- 172.21.15.89:0/1475709553 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129492721 0 0) 0x7fdd4c099b50 con 0x7fdd4c066410 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4959a700 1 -- 172.21.15.89:0/1475709553 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd340035b0 con 0x7fdd4c066410 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 msgr2=0x7fdd4c1367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c1367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/1475709553 >> v1:172.21.15.89:6789/0 conn(0x7fdd4c066410 legacy=0x7fdd4c06e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/1475709553 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c09a920 con 0x7fdd4c06f140 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.246+0000 7fdd49d9b700 1 --2- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c1367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.247+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c138d00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009d90 tx=0x7fdd4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d175dd6fdc65b9b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.247+0000 7fdd4959a700 1 -- 172.21.15.89:0/1475709553 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400e070 con 0x7fdd4c06f140 2023-09-30T22:22:20.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.247+0000 7fdd4959a700 1 -- 172.21.15.89:0/1475709553 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4400bb20 con 0x7fdd4c06f140 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.247+0000 7fdd4959a700 1 -- 172.21.15.89:0/1475709553 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44012430 con 0x7fdd4c06f140 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.247+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 msgr2=0x7fdd4c138d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.247+0000 7fdd50b2c700 1 --2- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c138d00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009d90 tx=0x7fdd4400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.248+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1475709553 shutdown_connections 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.248+0000 7fdd50b2c700 1 --2- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c138d00 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.248+0000 7fdd50b2c700 1 --2- 172.21.15.89:0/1475709553 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c1367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.248+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1475709553 >> 172.21.15.89:0/1475709553 conn(0x7fdd4c0a2aa0 msgr2=0x7fdd4c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.248+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1475709553 shutdown_connections 2023-09-30T22:22:20.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.248+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1475709553 wait complete. 2023-09-30T22:22:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.249+0000 7fdd50b2c700 1 Processor -- start 2023-09-30T22:22:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.249+0000 7fdd50b2c700 1 -- start start 2023-09-30T22:22:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd50b2c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd4c066410 0x7fdd4c136500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd50b2c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c136a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd50b2c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c136f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd50b2c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c11a860 con 0x7fdd4c06f140 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd50b2c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c09ba70 con 0x7fdd4c06ed50 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd50b2c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c119f50 con 0x7fdd4c066410 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd4ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c136f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd4ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c136f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40638/0 (socket says 172.21.15.89:40638) 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/1513546993 learned_addr learned my addr 172.21.15.89:0/1513546993 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd49d9b700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd4c066410 msgr2=0x7fdd4c136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd4c066410 0x7fdd4c136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 msgr2=0x7fdd4c136a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c136a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/1513546993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd400094a0 con 0x7fdd4c06f140 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.250+0000 7fdd49d9b700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c136a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.251+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c136f80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdd44013000 tx=0x7fdd44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.251+0000 7fdd3affd700 1 -- 172.21.15.89:0/1513546993 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44012430 con 0x7fdd4c06f140 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.251+0000 7fdd3affd700 1 -- 172.21.15.89:0/1513546993 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd44020070 con 0x7fdd4c06f140 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.251+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd44009a70 con 0x7fdd4c06f140 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.251+0000 7fdd3affd700 1 -- 172.21.15.89:0/1513546993 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44004280 con 0x7fdd4c06f140 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.251+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c11a4e0 con 0x7fdd4c06f140 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.252+0000 7fdd3affd700 1 -- 172.21.15.89:0/1513546993 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdd440043e0 con 0x7fdd4c06f140 2023-09-30T22:22:20.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.252+0000 7fdd3affd700 1 -- 172.21.15.89:0/1513546993 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdd44022070 con 0x7fdd4c06f140 2023-09-30T22:22:20.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.252+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c074ea0 con 0x7fdd4c06f140 2023-09-30T22:22:20.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7f2379c23700 1 Processor -- start 2023-09-30T22:22:20.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7f2379c23700 1 -- start start 2023-09-30T22:22:20.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7fdd3affd700 1 -- 172.21.15.89:0/1513546993 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdd44003e10 con 0x7fdd4c06f140 2023-09-30T22:22:20.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7f2379c23700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23740a0a30 0x7f23740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7f2379c23700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23740a1340 0x7f2374133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7f2379c23700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2374133fa0 0x7f2374134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7f2379c23700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237409edd0 con 0x7f23740a0a30 2023-09-30T22:22:20.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7f2379c23700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237409e1f0 con 0x7f2374133fa0 2023-09-30T22:22:20.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.257+0000 7f2379c23700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f237409dc50 con 0x7f23740a1340 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23740a0a30 0x7f23740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23740a0a30 0x7f23740a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40640/0 (socket says 172.21.15.89:40640) 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 -- 172.21.15.89:0/873647299 learned_addr learned my addr 172.21.15.89:0/873647299 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f2372ffd700 1 --2- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23740a1340 0x7f2374133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 -- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23740a1340 msgr2=0x7f2374133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 --2- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23740a1340 0x7f2374133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 -- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2374133fa0 msgr2=0x7f2374134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 --2- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2374133fa0 0x7f2374134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 -- 172.21.15.89:0/873647299 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740a1fe0 con 0x7f23740a0a30 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23737fe700 1 --2- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 0x7f23740a0e00 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f23640098f0 tx=0x7f2364009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da248f6f43faf4fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23727fc700 1 -- 172.21.15.89:0/873647299 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364011830 con 0x7f23740a0a30 2023-09-30T22:22:20.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23727fc700 1 -- 172.21.15.89:0/873647299 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364011990 con 0x7f23740a0a30 2023-09-30T22:22:20.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.258+0000 7f23727fc700 1 -- 172.21.15.89:0/873647299 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364011d20 con 0x7f23740a0a30 2023-09-30T22:22:20.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 -- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 msgr2=0x7f23740a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 --2- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 0x7f23740a0e00 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f23640098f0 tx=0x7f2364009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 -- 172.21.15.89:0/873647299 shutdown_connections 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 --2- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2374133fa0 0x7f2374134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 --2- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 0x7f23740a0e00 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 --2- 172.21.15.89:0/873647299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f23740a1340 0x7f2374133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 -- 172.21.15.89:0/873647299 >> 172.21.15.89:0/873647299 conn(0x7f23740a6550 msgr2=0x7f23740a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 -- 172.21.15.89:0/873647299 shutdown_connections 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.259+0000 7f2379c23700 1 -- 172.21.15.89:0/873647299 wait complete. 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.261+0000 7f2379c23700 1 Processor -- start 2023-09-30T22:22:20.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.261+0000 7f2379c23700 1 -- start start 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2379c23700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 0x7f237411c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2379c23700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23740a1340 0x7f237411cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2379c23700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2374133fa0 0x7f237411d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2379c23700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740a37e0 con 0x7f23740a0a30 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2379c23700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237411ec30 con 0x7f23740a1340 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2379c23700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23740a0220 con 0x7f2374133fa0 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2373fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2374133fa0 0x7f237411d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23740a1340 0x7f237411cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2373fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2374133fa0 0x7f237411d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60096/0 (socket says 172.21.15.89:60096) 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2373fff700 1 -- 172.21.15.89:0/622045081 learned_addr learned my addr 172.21.15.89:0/622045081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f23737fe700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 0x7f237411c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2373fff700 1 -- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23740a1340 msgr2=0x7f237411cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2373fff700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23740a1340 0x7f237411cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2373fff700 1 -- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 msgr2=0x7f237411c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2373fff700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 0x7f237411c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2373fff700 1 -- 172.21.15.89:0/622045081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364009550 con 0x7f2374133fa0 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.262+0000 7f2372ffd700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23740a1340 0x7f237411cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.263+0000 7f2373fff700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2374133fa0 0x7f237411d220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f236c00b970 tx=0x7f236c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:20.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.263+0000 7f235bfff700 1 -- 172.21.15.89:0/622045081 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00d860 con 0x7f2374133fa0 2023-09-30T22:22:20.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.263+0000 7f235bfff700 1 -- 172.21.15.89:0/622045081 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c00d9e0 con 0x7f2374133fa0 2023-09-30T22:22:20.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.263+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237411e970 con 0x7f2374133fa0 2023-09-30T22:22:20.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.263+0000 7f235bfff700 1 -- 172.21.15.89:0/622045081 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00dd90 con 0x7f2374133fa0 2023-09-30T22:22:20.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.263+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237409ba30 con 0x7f2374133fa0 2023-09-30T22:22:20.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.264+0000 7f235bfff700 1 -- 172.21.15.89:0/622045081 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f236c00e430 con 0x7f2374133fa0 2023-09-30T22:22:20.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.264+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2374074ea0 con 0x7f2374133fa0 2023-09-30T22:22:20.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.264+0000 7f235bfff700 1 -- 172.21.15.89:0/622045081 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f236c006e40 con 0x7f2374133fa0 2023-09-30T22:22:20.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.274+0000 7f235bfff700 1 -- 172.21.15.89:0/622045081 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f236c0203a0 con 0x7f2374133fa0 2023-09-30T22:22:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.389+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6f90005590 con 0x7f6fb0136f50 2023-09-30T22:22:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.390+0000 7f6fad7fa700 1 -- 172.21.15.89:0/2669405307 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fa801c310 con 0x7f6fb0136f50 2023-09-30T22:22:20.391 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.391+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7c580747f0 con 0x7f7c58138d80 2023-09-30T22:22:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.391+0000 7f7c3ffff700 1 -- 172.21.15.89:0/2260588437 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c5002cb80 con 0x7f7c58138d80 2023-09-30T22:22:20.393 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:20.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.393+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0136f50 msgr2=0x7f6fb01c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.393+0000 7f6fb6b90700 1 --2- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0136f50 0x7f6fb01c6240 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800ba10 tx=0x7f6fa800bd20 comp rx=0 tx=0).stop 2023-09-30T22:22:20.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.393+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 shutdown_connections 2023-09-30T22:22:20.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.393+0000 7f6fb6b90700 1 --2- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fb0136f50 0x7f6fb01c6240 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.393+0000 7f6fb6b90700 1 --2- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fb0131810 0x7f6fb0119450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.393+0000 7f6fb6b90700 1 --2- 172.21.15.89:0/2669405307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb0134630 0x7f6fb0119990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.393+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 >> 172.21.15.89:0/2669405307 conn(0x7f6fb00a2aa0 msgr2=0x7f6fb012e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.394+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 shutdown_connections 2023-09-30T22:22:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.394+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c58138d80 msgr2=0x7f7c581341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.394+0000 7f7c5d81d700 1 --2- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c58138d80 0x7f7c581341f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f7c50012000 tx=0x7f7c50004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.394+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 shutdown_connections 2023-09-30T22:22:20.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.394+0000 7f7c5d81d700 1 --2- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c58138d80 0x7f7c581341f0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.394+0000 7f7c5d81d700 1 --2- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5809d630 0x7f7c58133770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.394+0000 7f7c5d81d700 1 --2- 172.21.15.89:0/2260588437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c580a0a30 0x7f7c58133cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.394+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 >> 172.21.15.89:0/2260588437 conn(0x7f7c580a3050 msgr2=0x7f7c5809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:20.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.395+0000 7f6fb6b90700 1 -- 172.21.15.89:0/2669405307 wait complete. 2023-09-30T22:22:20.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.395+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 shutdown_connections 2023-09-30T22:22:20.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.395+0000 7f7c5d81d700 1 -- 172.21.15.89:0/2260588437 wait complete. 2023-09-30T22:22:20.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:20.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:20.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.460+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdd4c06b9e0 con 0x7fdd4c06f140 2023-09-30T22:22:20.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.460+0000 7fdd3affd700 1 -- 172.21.15.89:0/1513546993 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd44011390 con 0x7fdd4c06f140 2023-09-30T22:22:20.461 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 msgr2=0x7fdd4c136f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c136f80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdd44013000 tx=0x7fdd44004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 shutdown_connections 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c06f140 0x7fdd4c136f80 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd4c066410 0x7fdd4c136500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 --2- 172.21.15.89:0/1513546993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd4c06ed50 0x7fdd4c136a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 >> 172.21.15.89:0/1513546993 conn(0x7fdd4c0a2aa0 msgr2=0x7fdd4c09d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 shutdown_connections 2023-09-30T22:22:20.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.462+0000 7fdd50b2c700 1 -- 172.21.15.89:0/1513546993 wait complete. 2023-09-30T22:22:20.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:20.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.482+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f237411ed70 con 0x7f2374133fa0 2023-09-30T22:22:20.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.482+0000 7f235bfff700 1 -- 172.21.15.89:0/622045081 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f236c01c030 con 0x7f2374133fa0 2023-09-30T22:22:20.483 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.484+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2374133fa0 msgr2=0x7f237411d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.484+0000 7f2379c23700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2374133fa0 0x7f237411d220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f236c00b970 tx=0x7f236c00bc80 comp rx=0 tx=0).stop 2023-09-30T22:22:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.484+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 shutdown_connections 2023-09-30T22:22:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.484+0000 7f2379c23700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2374133fa0 0x7f237411d220 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.484+0000 7f2379c23700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23740a0a30 0x7f237411c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.484+0000 7f2379c23700 1 --2- 172.21.15.89:0/622045081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23740a1340 0x7f237411cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.485+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 >> 172.21.15.89:0/622045081 conn(0x7f23740a6550 msgr2=0x7f23740a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.485+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 shutdown_connections 2023-09-30T22:22:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:20.485+0000 7f2379c23700 1 -- 172.21.15.89:0/622045081 wait complete. 2023-09-30T22:22:20.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:21.413 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:21.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:21.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:21.498 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:21.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.558+0000 7fce0ef27700 1 Processor -- start 2023-09-30T22:22:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.558+0000 7f26661a3700 1 Processor -- start 2023-09-30T22:22:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.559+0000 7f26661a3700 1 -- start start 2023-09-30T22:22:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.559+0000 7f26661a3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f266009e1d0 0x7f2660138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.559+0000 7f26661a3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2660139170 0x7f266013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.559+0000 7f26661a3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266009b120 con 0x7f266009d6f0 2023-09-30T22:22:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.559+0000 7f26661a3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266006b460 con 0x7f2660139170 2023-09-30T22:22:21.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.559+0000 7f26661a3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266009a7e0 con 0x7f266009e1d0 2023-09-30T22:22:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.559+0000 7fce0ef27700 1 -- start start 2023-09-30T22:22:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0ef27700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fce080a0a30 0x7fce080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0ef27700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0ef27700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce08133930 0x7fce08133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0ef27700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce080a2a70 con 0x7fce0809d630 2023-09-30T22:22:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0ef27700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce080a01d0 con 0x7fce080a0a30 2023-09-30T22:22:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0ef27700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce0809e2b0 con 0x7fce08133930 2023-09-30T22:22:21.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7f265ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2660139170 0x7f266013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7f265ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2660139170 0x7f266013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40670/0 (socket says 172.21.15.89:40670) 2023-09-30T22:22:21.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7f265ffff700 1 -- 172.21.15.89:0/3781642041 learned_addr learned my addr 172.21.15.89:0/3781642041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:21.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce07fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce07fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40674/0 (socket says 172.21.15.89:40674) 2023-09-30T22:22:21.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0d4c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce08133930 0x7fce08133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce07fff700 1 -- 172.21.15.89:0/1185421167 learned_addr learned my addr 172.21.15.89:0/1185421167 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7f265ffff700 1 -- 172.21.15.89:0/3781642041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f266009e1d0 msgr2=0x7f2660138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7f265ffff700 1 --2- 172.21.15.89:0/3781642041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f266009e1d0 0x7f2660138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7f265ffff700 1 -- 172.21.15.89:0/3781642041 >> v1:172.21.15.89:6789/0 conn(0x7f266009d6f0 legacy=0x7f266009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7f265ffff700 1 -- 172.21.15.89:0/3781642041 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266009a500 con 0x7f2660139170 2023-09-30T22:22:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0d4c4700 1 -- 172.21.15.89:0/1185421167 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fce080a0a30 msgr2=0x7fce080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:21.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0d4c4700 1 --2- 172.21.15.89:0/1185421167 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fce080a0a30 0x7fce080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0d4c4700 1 -- 172.21.15.89:0/1185421167 >> v2:172.21.15.89:3300/0 conn(0x7fce0809d630 msgr2=0x7fce080a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:21.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0d4c4700 1 --2- 172.21.15.89:0/1185421167 >> v2:172.21.15.89:3300/0 conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0d4c4700 1 -- 172.21.15.89:0/1185421167 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0809dc50 con 0x7fce08133930 2023-09-30T22:22:21.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7fce0d4c4700 1 --2- 172.21.15.89:0/1185421167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce08133930 0x7fce08133dd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fce00009d90 tx=0x7fce0000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93562a992eb6fde6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:21.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.560+0000 7f265ffff700 1 --2- 172.21.15.89:0/3781642041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2660139170 0x7f266013b550 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2658009e50 tx=0x7f265800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a35fc05339934d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:21.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.561+0000 7f265e7fc700 1 -- 172.21.15.89:0/3781642041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800e070 con 0x7f2660139170 2023-09-30T22:22:21.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.561+0000 7fce077fe700 1 -- 172.21.15.89:0/1185421167 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0000e070 con 0x7fce08133930 2023-09-30T22:22:21.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.561+0000 7f265e7fc700 1 -- 172.21.15.89:0/3781642041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2658003a20 con 0x7f2660139170 2023-09-30T22:22:21.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.561+0000 7f265e7fc700 1 -- 172.21.15.89:0/3781642041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658003dd0 con 0x7f2660139170 2023-09-30T22:22:21.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.561+0000 7fce077fe700 1 -- 172.21.15.89:0/1185421167 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0000bb20 con 0x7fce08133930 2023-09-30T22:22:21.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.561+0000 7fce077fe700 1 -- 172.21.15.89:0/1185421167 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00012430 con 0x7fce08133930 2023-09-30T22:22:21.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.561+0000 7f26661a3700 1 -- 172.21.15.89:0/3781642041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2660139170 msgr2=0x7f266013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.561+0000 7f26661a3700 1 --2- 172.21.15.89:0/3781642041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2660139170 0x7f266013b550 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2658009e50 tx=0x7f265800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:21.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.562+0000 7fce0ef27700 1 -- 172.21.15.89:0/1185421167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce08133930 msgr2=0x7fce08133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.562+0000 7fce0ef27700 1 --2- 172.21.15.89:0/1185421167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce08133930 0x7fce08133dd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fce00009d90 tx=0x7fce0000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.562+0000 7f26661a3700 1 -- 172.21.15.89:0/3781642041 shutdown_connections 2023-09-30T22:22:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.562+0000 7f26661a3700 1 --2- 172.21.15.89:0/3781642041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2660139170 0x7f266013b550 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.562+0000 7f26661a3700 1 --2- 172.21.15.89:0/3781642041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f266009e1d0 0x7f2660138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.562+0000 7f26661a3700 1 -- 172.21.15.89:0/3781642041 >> 172.21.15.89:0/3781642041 conn(0x7f2660066f50 msgr2=0x7f266009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.562+0000 7f26661a3700 1 -- 172.21.15.89:0/3781642041 shutdown_connections 2023-09-30T22:22:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.562+0000 7f26661a3700 1 -- 172.21.15.89:0/3781642041 wait complete. 2023-09-30T22:22:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7fce0ef27700 1 -- 172.21.15.89:0/1185421167 shutdown_connections 2023-09-30T22:22:21.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f26661a3700 1 Processor -- start 2023-09-30T22:22:21.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7fce0ef27700 1 --2- 172.21.15.89:0/1185421167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce08133930 0x7fce08133dd0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7fce0ef27700 1 --2- 172.21.15.89:0/1185421167 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fce080a0a30 0x7fce080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7fce0ef27700 1 --2- 172.21.15.89:0/1185421167 >> v2:172.21.15.89:3300/0 conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7fce0ef27700 1 -- 172.21.15.89:0/1185421167 >> 172.21.15.89:0/1185421167 conn(0x7fce080a3050 msgr2=0x7fce080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:21.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f26661a3700 1 -- start start 2023-09-30T22:22:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f26661a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266009d6f0 0x7f26601c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f26661a3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f266009e1d0 0x7f26601c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f26661a3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2660139170 0x7f26601c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f26661a3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2660120880 con 0x7f266009d6f0 2023-09-30T22:22:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f26661a3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2660121da0 con 0x7f2660139170 2023-09-30T22:22:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f26661a3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f266009c710 con 0x7f266009e1d0 2023-09-30T22:22:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7fce0ef27700 1 -- 172.21.15.89:0/1185421167 shutdown_connections 2023-09-30T22:22:21.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7fce0ef27700 1 -- 172.21.15.89:0/1185421167 wait complete. 2023-09-30T22:22:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f265ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2660139170 0x7f26601c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f265effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f266009e1d0 0x7f26601c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f265effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f266009e1d0 0x7f26601c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60116/0 (socket says 172.21.15.89:60116) 2023-09-30T22:22:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f265ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2660139170 0x7f26601c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34086/0 (socket says 172.21.15.89:34086) 2023-09-30T22:22:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.563+0000 7f265effd700 1 -- 172.21.15.89:0/721375217 learned_addr learned my addr 172.21.15.89:0/721375217 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:21.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.564+0000 7f265f7fe700 1 --2- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266009d6f0 0x7f26601c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.564+0000 7f265ffff700 1 -- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f266009e1d0 msgr2=0x7f26601c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.564+0000 7f265ffff700 1 --2- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f266009e1d0 0x7f26601c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.564+0000 7f265ffff700 1 -- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266009d6f0 msgr2=0x7f26601c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.564+0000 7f265ffff700 1 --2- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266009d6f0 0x7f26601c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.564+0000 7f265ffff700 1 -- 172.21.15.89:0/721375217 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658009b30 con 0x7f2660139170 2023-09-30T22:22:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.564+0000 7f265ffff700 1 --2- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2660139170 0x7f26601c8ee0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2658009e50 tx=0x7f2658004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.564+0000 7fce0ef27700 1 Processor -- start 2023-09-30T22:22:21.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7fce0ef27700 1 -- start start 2023-09-30T22:22:21.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7f2647fff700 1 -- 172.21.15.89:0/721375217 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658004440 con 0x7f2660139170 2023-09-30T22:22:21.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7f2647fff700 1 -- 172.21.15.89:0/721375217 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26580045c0 con 0x7f2660139170 2023-09-30T22:22:21.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7f2647fff700 1 -- 172.21.15.89:0/721375217 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658004970 con 0x7f2660139170 2023-09-30T22:22:21.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2660122070 con 0x7f2660139170 2023-09-30T22:22:21.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2660122610 con 0x7f2660139170 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7fce0ef27700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fce0809d630 0x7fce0811f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7fce0ef27700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce080a0a30 0x7fce0811fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7fce0ef27700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08133930 0x7fce081c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7fce0ef27700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce081215f0 con 0x7fce08133930 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7fce0ef27700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce080a4030 con 0x7fce0809d630 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.565+0000 7fce0ef27700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce080a0410 con 0x7fce080a0a30 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08133930 0x7fce081c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08133930 0x7fce081c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40692/0 (socket says 172.21.15.89:40692) 2023-09-30T22:22:21.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 -- 172.21.15.89:0/3255990548 learned_addr learned my addr 172.21.15.89:0/3255990548 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce07fff700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce080a0a30 0x7fce0811fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 -- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce080a0a30 msgr2=0x7fce0811fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce080a0a30 0x7fce0811fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 -- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fce0809d630 msgr2=0x7fce0811f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fce0809d630 0x7fce0811f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 -- 172.21.15.89:0/3255990548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce00009a70 con 0x7fce08133930 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce07fff700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce080a0a30 0x7fce0811fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce0d4c4700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08133930 0x7fce081c8270 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fce0000b370 tx=0x7fce00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2660122290 con 0x7f2660139170 2023-09-30T22:22:21.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7f2647fff700 1 -- 172.21.15.89:0/721375217 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2658012430 con 0x7f2660139170 2023-09-30T22:22:21.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.566+0000 7fce057fa700 1 -- 172.21.15.89:0/3255990548 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00012430 con 0x7fce08133930 2023-09-30T22:22:21.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.567+0000 7fce057fa700 1 -- 172.21.15.89:0/3255990548 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce00020070 con 0x7fce08133930 2023-09-30T22:22:21.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.567+0000 7fce057fa700 1 -- 172.21.15.89:0/3255990548 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00004280 con 0x7fce08133930 2023-09-30T22:22:21.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.567+0000 7f2647fff700 1 -- 172.21.15.89:0/721375217 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2658040b20 con 0x7f2660139170 2023-09-30T22:22:21.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.567+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0811f1a0 con 0x7fce08133930 2023-09-30T22:22:21.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.567+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce08122560 con 0x7fce08133930 2023-09-30T22:22:21.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.567+0000 7fce057fa700 1 -- 172.21.15.89:0/3255990548 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce000043e0 con 0x7fce08133930 2023-09-30T22:22:21.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.568+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce08121150 con 0x7fce08133930 2023-09-30T22:22:21.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.568+0000 7fce057fa700 1 -- 172.21.15.89:0/3255990548 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fce00022070 con 0x7fce08133930 2023-09-30T22:22:21.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.573+0000 7fce057fa700 1 -- 172.21.15.89:0/3255990548 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce0001c400 con 0x7fce08133930 2023-09-30T22:22:21.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.575+0000 7f2647fff700 1 -- 172.21.15.89:0/721375217 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2658017080 con 0x7f2660139170 2023-09-30T22:22:21.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.621+0000 7f532a56e700 1 Processor -- start 2023-09-30T22:22:21.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.621+0000 7f532a56e700 1 -- start start 2023-09-30T22:22:21.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.621+0000 7f532a56e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f53240a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.621+0000 7f532a56e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5324138d80 0x7f532413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.621+0000 7f532a56e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53240a2a70 con 0x7f53240a0a30 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.621+0000 7f532a56e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53240a01d0 con 0x7f5324138d80 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.621+0000 7f532a56e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532409e2b0 con 0x7f532409d630 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5323fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f53240a0a30 0x7f53240a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47424/0 (socket says 172.21.15.89:47424) 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5323fff700 1 -- 172.21.15.89:0/2217713843 learned_addr learned my addr 172.21.15.89:0/2217713843 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5328b0b700 1 --2- 172.21.15.89:0/2217713843 >> v2:172.21.15.89:3300/0 conn(0x7f5324138d80 0x7f532413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f53237fe700 1 --2- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f53240a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5328b0b700 1 -- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 msgr2=0x7f53240a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5328b0b700 1 --2- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f53240a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5328b0b700 1 -- 172.21.15.89:0/2217713843 >> v1:172.21.15.89:6789/0 conn(0x7f53240a0a30 legacy=0x7f53240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5328b0b700 1 -- 172.21.15.89:0/2217713843 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532409dc50 con 0x7f5324138d80 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f53237fe700 1 --2- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f53240a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5328b0b700 1 --2- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324138d80 0x7f532413b160 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f531c009d90 tx=0x7f531c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380bc16bb760ea02 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5322ffd700 1 -- 172.21.15.89:0/2217713843 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c00e070 con 0x7f5324138d80 2023-09-30T22:22:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5322ffd700 1 -- 172.21.15.89:0/2217713843 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c00bb20 con 0x7f5324138d80 2023-09-30T22:22:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.622+0000 7f5322ffd700 1 -- 172.21.15.89:0/2217713843 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c012430 con 0x7f5324138d80 2023-09-30T22:22:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.623+0000 7f532a56e700 1 -- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324138d80 msgr2=0x7f532413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.623+0000 7f532a56e700 1 --2- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324138d80 0x7f532413b160 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f531c009d90 tx=0x7f531c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.623+0000 7f532a56e700 1 -- 172.21.15.89:0/2217713843 shutdown_connections 2023-09-30T22:22:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.623+0000 7f532a56e700 1 --2- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324138d80 0x7f532413b160 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.623+0000 7f532a56e700 1 --2- 172.21.15.89:0/2217713843 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f53240a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.623+0000 7f532a56e700 1 -- 172.21.15.89:0/2217713843 >> 172.21.15.89:0/2217713843 conn(0x7f53240a3050 msgr2=0x7f53240a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.623+0000 7f532a56e700 1 -- 172.21.15.89:0/2217713843 shutdown_connections 2023-09-30T22:22:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.623+0000 7f532a56e700 1 -- 172.21.15.89:0/2217713843 wait complete. 2023-09-30T22:22:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.625+0000 7f532a56e700 1 Processor -- start 2023-09-30T22:22:21.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.625+0000 7f532a56e700 1 -- start start 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f532a56e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f5324133860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f532a56e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53240a0a30 0x7f5324133da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f532a56e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5324138d80 0x7f53241342e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f532a56e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53240a0410 con 0x7f53240a0a30 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f532a56e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5324137310 con 0x7f532409d630 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f532a56e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5324136c70 con 0x7f5324138d80 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f5328b0b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5324138d80 0x7f53241342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f53237fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53240a0a30 0x7f5324133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f53237fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53240a0a30 0x7f5324133da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40704/0 (socket says 172.21.15.89:40704) 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f5328b0b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5324138d80 0x7f53241342e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60134/0 (socket says 172.21.15.89:60134) 2023-09-30T22:22:21.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f5323fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f5324133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f53237fe700 1 -- 172.21.15.89:0/736089544 learned_addr learned my addr 172.21.15.89:0/736089544 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f5328b0b700 1 -- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 msgr2=0x7f5324133860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f5328b0b700 1 --2- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f5324133860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f5328b0b700 1 -- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53240a0a30 msgr2=0x7f5324133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f5328b0b700 1 --2- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53240a0a30 0x7f5324133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f5328b0b700 1 -- 172.21.15.89:0/736089544 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180094a0 con 0x7f5324138d80 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.626+0000 7f53237fe700 1 --2- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53240a0a30 0x7f5324133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.627+0000 7f5328b0b700 1 --2- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5324138d80 0x7f53241342e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f531c016000 tx=0x7f531c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.627+0000 7f5320ff9700 1 -- 172.21.15.89:0/736089544 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c012430 con 0x7f5324138d80 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.627+0000 7f5320ff9700 1 -- 172.21.15.89:0/736089544 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c031070 con 0x7f5324138d80 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.627+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c009a70 con 0x7f5324138d80 2023-09-30T22:22:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.627+0000 7f5320ff9700 1 -- 172.21.15.89:0/736089544 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c004040 con 0x7f5324138d80 2023-09-30T22:22:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.627+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5324137e90 con 0x7f5324138d80 2023-09-30T22:22:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.628+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5324137150 con 0x7f5324138d80 2023-09-30T22:22:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.629+0000 7f5320ff9700 1 -- 172.21.15.89:0/736089544 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f531c0041a0 con 0x7f5324138d80 2023-09-30T22:22:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.629+0000 7f5320ff9700 1 -- 172.21.15.89:0/736089544 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f531c033070 con 0x7f5324138d80 2023-09-30T22:22:21.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.638+0000 7f5320ff9700 1 -- 172.21.15.89:0/736089544 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f531c011980 con 0x7f5324138d80 2023-09-30T22:22:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.659+0000 7fd6bdeb6700 1 Processor -- start 2023-09-30T22:22:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.659+0000 7fd6bdeb6700 1 -- start start 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6bdeb6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b80a0a30 0x7fd6b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6bdeb6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd6b8138d80 0x7fd6b813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6bdeb6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b80a2a70 con 0x7fd6b809d630 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6bdeb6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b80a01d0 con 0x7fd6b8138d80 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6bdeb6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b809e2b0 con 0x7fd6b80a0a30 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b80a0a30 0x7fd6b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b80a0a30 0x7fd6b80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34118/0 (socket says 172.21.15.89:34118) 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b77fe700 1 -- 172.21.15.89:0/2597064081 learned_addr learned my addr 172.21.15.89:0/2597064081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b7fff700 1 --2- 172.21.15.89:0/2597064081 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8138d80 0x7fd6b813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b77fe700 1 -- 172.21.15.89:0/2597064081 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8138d80 msgr2=0x7fd6b813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b77fe700 1 --2- 172.21.15.89:0/2597064081 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8138d80 0x7fd6b813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b77fe700 1 -- 172.21.15.89:0/2597064081 >> v1:172.21.15.89:6789/0 conn(0x7fd6b809d630 legacy=0x7fd6b80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b77fe700 1 -- 172.21.15.89:0/2597064081 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b809dc50 con 0x7fd6b80a0a30 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.660+0000 7fd6b7fff700 1 --2- 172.21.15.89:0/2597064081 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8138d80 0x7fd6b813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.661+0000 7fd6b77fe700 1 --2- 172.21.15.89:0/2597064081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b80a0a30 0x7fd6b80a0e00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80098f0 tx=0x7fd6a8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f39ae5beae7d90d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.661+0000 7fd6b67fc700 1 -- 172.21.15.89:0/2597064081 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8011830 con 0x7fd6b80a0a30 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.661+0000 7fd6b67fc700 1 -- 172.21.15.89:0/2597064081 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a8011990 con 0x7fd6b80a0a30 2023-09-30T22:22:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.661+0000 7fd6b67fc700 1 -- 172.21.15.89:0/2597064081 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8011d20 con 0x7fd6b80a0a30 2023-09-30T22:22:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.662+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/2597064081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b80a0a30 msgr2=0x7fd6b80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.662+0000 7fd6bdeb6700 1 --2- 172.21.15.89:0/2597064081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b80a0a30 0x7fd6b80a0e00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80098f0 tx=0x7fd6a8009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.662+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/2597064081 shutdown_connections 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.662+0000 7fd6bdeb6700 1 --2- 172.21.15.89:0/2597064081 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8138d80 0x7fd6b813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.662+0000 7fd6bdeb6700 1 --2- 172.21.15.89:0/2597064081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b80a0a30 0x7fd6b80a0e00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.662+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/2597064081 >> 172.21.15.89:0/2597064081 conn(0x7fd6b80a3050 msgr2=0x7fd6b80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.662+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/2597064081 shutdown_connections 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.662+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/2597064081 wait complete. 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.663+0000 7fd6bdeb6700 1 Processor -- start 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.663+0000 7fd6bdeb6700 1 -- start start 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.663+0000 7fd6bdeb6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b809d630 0x7fd6b8120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.663+0000 7fd6bdeb6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b80a0a30 0x7fd6b8120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.663+0000 7fd6bdeb6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8138d80 0x7fd6b81211e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.663+0000 7fd6bdeb6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8123670 con 0x7fd6b80a0a30 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.663+0000 7fd6bdeb6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8121bc0 con 0x7fd6b809d630 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.663+0000 7fd6bdeb6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8124230 con 0x7fd6b8138d80 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b80a0a30 0x7fd6b8120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b80a0a30 0x7fd6b8120ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40714/0 (socket says 172.21.15.89:40714) 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 -- 172.21.15.89:0/753923619 learned_addr learned my addr 172.21.15.89:0/753923619 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b7fff700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8138d80 0x7fd6b81211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b77fe700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b809d630 0x7fd6b8120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 -- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8138d80 msgr2=0x7fd6b81211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8138d80 0x7fd6b81211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 -- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b809d630 msgr2=0x7fd6b8120760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b809d630 0x7fd6b8120760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 -- 172.21.15.89:0/753923619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b0009a70 con 0x7fd6b80a0a30 2023-09-30T22:22:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6b6ffd700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b80a0a30 0x7fd6b8120ca0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c4b0 tx=0x7fd6ac00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd69ffff700 1 -- 172.21.15.89:0/753923619 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0052d0 con 0x7fd6b80a0a30 2023-09-30T22:22:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd69ffff700 1 -- 172.21.15.89:0/753923619 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac005430 con 0x7fd6b80a0a30 2023-09-30T22:22:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a8009550 con 0x7fd6b80a0a30 2023-09-30T22:22:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.664+0000 7fd69ffff700 1 -- 172.21.15.89:0/753923619 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0038c0 con 0x7fd6b80a0a30 2023-09-30T22:22:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.665+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b80a0410 con 0x7fd6b80a0a30 2023-09-30T22:22:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.665+0000 7fd69ffff700 1 -- 172.21.15.89:0/753923619 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6ac003a20 con 0x7fd6b80a0a30 2023-09-30T22:22:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.666+0000 7fd69ffff700 1 -- 172.21.15.89:0/753923619 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd6ac03e9e0 con 0x7fd6b80a0a30 2023-09-30T22:22:21.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.666+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b8123bf0 con 0x7fd6b80a0a30 2023-09-30T22:22:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.671+0000 7fd69ffff700 1 -- 172.21.15.89:0/753923619 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6b8123bf0 con 0x7fd6b80a0a30 2023-09-30T22:22:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.768+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fce080747f0 con 0x7fce08133930 2023-09-30T22:22:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.768+0000 7fce057fa700 1 -- 172.21.15.89:0/3255990548 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce0002c020 con 0x7fce08133930 2023-09-30T22:22:21.768 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08133930 msgr2=0x7fce081c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08133930 0x7fce081c8270 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fce0000b370 tx=0x7fce00004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 shutdown_connections 2023-09-30T22:22:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08133930 0x7fce081c8270 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fce0809d630 0x7fce0811f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 --2- 172.21.15.89:0/3255990548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fce080a0a30 0x7fce0811fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 >> 172.21.15.89:0/3255990548 conn(0x7fce080a3050 msgr2=0x7fce081bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:21.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 shutdown_connections 2023-09-30T22:22:21.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.770+0000 7fce0ef27700 1 -- 172.21.15.89:0/3255990548 wait complete. 2023-09-30T22:22:21.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.802+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2660074ea0 con 0x7f2660139170 2023-09-30T22:22:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.803+0000 7f2647fff700 1 -- 172.21.15.89:0/721375217 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f26580117a0 con 0x7f2660139170 2023-09-30T22:22:21.803 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2660139170 msgr2=0x7f26601c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 --2- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2660139170 0x7f26601c8ee0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2658009e50 tx=0x7f2658004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 shutdown_connections 2023-09-30T22:22:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 --2- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2660139170 0x7f26601c8ee0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 --2- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f266009d6f0 0x7f26601c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 --2- 172.21.15.89:0/721375217 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f266009e1d0 0x7f26601c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 >> 172.21.15.89:0/721375217 conn(0x7f2660066f50 msgr2=0x7f2660138220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 shutdown_connections 2023-09-30T22:22:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.806+0000 7f26661a3700 1 -- 172.21.15.89:0/721375217 wait complete. 2023-09-30T22:22:21.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:21.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.852+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5324137150 con 0x7f5324138d80 2023-09-30T22:22:21.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.852+0000 7f5320ff9700 1 -- 172.21.15.89:0/736089544 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5324137150 con 0x7f5324138d80 2023-09-30T22:22:21.852 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5324138d80 msgr2=0x7f53241342e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 --2- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5324138d80 0x7f53241342e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f531c016000 tx=0x7f531c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 shutdown_connections 2023-09-30T22:22:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 --2- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5324138d80 0x7f53241342e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 --2- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f532409d630 0x7f5324133860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 --2- 172.21.15.89:0/736089544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53240a0a30 0x7f5324133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 >> 172.21.15.89:0/736089544 conn(0x7f53240a3050 msgr2=0x7f532409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 shutdown_connections 2023-09-30T22:22:21.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.855+0000 7f532a56e700 1 -- 172.21.15.89:0/736089544 wait complete. 2023-09-30T22:22:21.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:21.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.879+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd6b8123bf0 con 0x7fd6b80a0a30 2023-09-30T22:22:21.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.879+0000 7fd69ffff700 1 -- 172.21.15.89:0/753923619 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6b8123bf0 con 0x7fd6b80a0a30 2023-09-30T22:22:21.879 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b80a0a30 msgr2=0x7fd6b8120ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b80a0a30 0x7fd6b8120ca0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c4b0 tx=0x7fd6ac00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 shutdown_connections 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8138d80 0x7fd6b81211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6b809d630 0x7fd6b8120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 --2- 172.21.15.89:0/753923619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6b80a0a30 0x7fd6b8120ca0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 >> 172.21.15.89:0/753923619 conn(0x7fd6b80a3050 msgr2=0x7fd6b8139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 shutdown_connections 2023-09-30T22:22:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:21.881+0000 7fd6bdeb6700 1 -- 172.21.15.89:0/753923619 wait complete. 2023-09-30T22:22:21.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:22.787 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:22.820 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:22.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:22.894 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.928+0000 7f9f4c952700 1 Processor -- start 2023-09-30T22:22:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.928+0000 7f9f4c952700 1 -- start start 2023-09-30T22:22:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f4c952700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f4809eca0 0x7f9f4813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f4c952700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f4813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f4c952700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f48070ee0 con 0x7f9f4806e570 2023-09-30T22:22:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f4c952700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f48071980 con 0x7f9f4809eca0 2023-09-30T22:22:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f4c952700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f48071480 con 0x7f9f4806f050 2023-09-30T22:22:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f45d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f4809eca0 0x7f9f4813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f45d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f4809eca0 0x7f9f4813cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40728/0 (socket says 172.21.15.89:40728) 2023-09-30T22:22:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f45d9b700 1 -- 172.21.15.89:0/2770796235 learned_addr learned my addr 172.21.15.89:0/2770796235 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f4659c700 1 --1- 172.21.15.89:0/2770796235 >> v1:172.21.15.89:6789/0 conn(0x7f9f4806e570 0x7f9f4806e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47454/0 (socket says 172.21.15.89:47454) 2023-09-30T22:22:22.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f45d9b700 1 -- 172.21.15.89:0/2770796235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 msgr2=0x7f9f4813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f45d9b700 1 --2- 172.21.15.89:0/2770796235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f4813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f45d9b700 1 -- 172.21.15.89:0/2770796235 >> v1:172.21.15.89:6789/0 conn(0x7f9f4806e570 legacy=0x7f9f4806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.929+0000 7f9f45d9b700 1 -- 172.21.15.89:0/2770796235 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48071b90 con 0x7f9f4809eca0 2023-09-30T22:22:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.930+0000 7f9f45d9b700 1 --2- 172.21.15.89:0/2770796235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4809eca0 0x7f9f4813cea0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0098f0 tx=0x7f9f3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5601f621dd85eaa6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.930+0000 7f9f4559a700 1 -- 172.21.15.89:0/2770796235 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00b8a0 con 0x7f9f4809eca0 2023-09-30T22:22:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.930+0000 7f9f4559a700 1 -- 172.21.15.89:0/2770796235 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c00ba00 con 0x7f9f4809eca0 2023-09-30T22:22:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.930+0000 7f9f4559a700 1 -- 172.21.15.89:0/2770796235 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00bd90 con 0x7f9f4809eca0 2023-09-30T22:22:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.931+0000 7f9f4c952700 1 -- 172.21.15.89:0/2770796235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4809eca0 msgr2=0x7f9f4813cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.931+0000 7f9f4c952700 1 --2- 172.21.15.89:0/2770796235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4809eca0 0x7f9f4813cea0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0098f0 tx=0x7f9f3c009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.931+0000 7f9f4c952700 1 -- 172.21.15.89:0/2770796235 shutdown_connections 2023-09-30T22:22:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.931+0000 7f9f4c952700 1 --2- 172.21.15.89:0/2770796235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f4813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.931+0000 7f9f4c952700 1 --2- 172.21.15.89:0/2770796235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4809eca0 0x7f9f4813cea0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.931+0000 7f9f4c952700 1 -- 172.21.15.89:0/2770796235 >> 172.21.15.89:0/2770796235 conn(0x7f9f480a2aa0 msgr2=0x7f9f480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.931+0000 7f9f4c952700 1 -- 172.21.15.89:0/2770796235 shutdown_connections 2023-09-30T22:22:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.931+0000 7f9f4c952700 1 -- 172.21.15.89:0/2770796235 wait complete. 2023-09-30T22:22:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.933+0000 7f9f4c952700 1 Processor -- start 2023-09-30T22:22:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.933+0000 7f9f4c952700 1 -- start start 2023-09-30T22:22:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.933+0000 7f9f4c952700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4806e570 0x7f9f481ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.933+0000 7f9f4c952700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f481cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.933+0000 7f9f4c952700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f4813aa10 0x7f9f4813ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.933+0000 7f9f4c952700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4813ded0 con 0x7f9f4806e570 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.933+0000 7f9f4c952700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4813e470 con 0x7f9f4813aa10 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.933+0000 7f9f4c952700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4813eaa0 con 0x7f9f4806f050 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f481cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f481cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60164/0 (socket says 172.21.15.89:60164) 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 -- 172.21.15.89:0/1705692255 learned_addr learned my addr 172.21.15.89:0/1705692255 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f46d9d700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f4813aa10 0x7f9f4813ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f4659c700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4806e570 0x7f9f481ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 -- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f4813aa10 msgr2=0x7f9f4813ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f4813aa10 0x7f9f4813ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 -- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4806e570 msgr2=0x7f9f481ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4806e570 0x7f9f481ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 -- 172.21.15.89:0/1705692255 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c009550 con 0x7f9f4806f050 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.934+0000 7f9f45d9b700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f481cd1b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c000c00 tx=0x7f9f3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.935+0000 7f9f36ffd700 1 -- 172.21.15.89:0/1705692255 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c004510 con 0x7f9f4806f050 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.935+0000 7f9f36ffd700 1 -- 172.21.15.89:0/1705692255 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c0036a0 con 0x7f9f4806f050 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.935+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4809a470 con 0x7f9f4806f050 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.935+0000 7f9f36ffd700 1 -- 172.21.15.89:0/1705692255 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c003a10 con 0x7f9f4806f050 2023-09-30T22:22:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.935+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4813cd50 con 0x7f9f4806f050 2023-09-30T22:22:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.936+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4813e0f0 con 0x7f9f4806f050 2023-09-30T22:22:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.936+0000 7f9f36ffd700 1 -- 172.21.15.89:0/1705692255 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f3c003b70 con 0x7f9f4806f050 2023-09-30T22:22:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.937+0000 7f9f36ffd700 1 -- 172.21.15.89:0/1705692255 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9f3c021030 con 0x7f9f4806f050 2023-09-30T22:22:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.945+0000 7f9f36ffd700 1 -- 172.21.15.89:0/1705692255 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f3c01a430 con 0x7f9f4806f050 2023-09-30T22:22:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.973+0000 7f5229d0e700 1 Processor -- start 2023-09-30T22:22:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.973+0000 7f5229d0e700 1 -- start start 2023-09-30T22:22:22.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5229d0e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52241316c0 0x7f5224131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5229d0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522406ae80 0x7f5224139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5229d0e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522409e130 con 0x7f522409d9f0 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5229d0e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52241312b0 con 0x7f52241316c0 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5229d0e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f522409f6e0 con 0x7f522406ae80 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5223fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522406ae80 0x7f5224139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5223fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522406ae80 0x7f5224139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60166/0 (socket says 172.21.15.89:60166) 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5223fff700 1 -- 172.21.15.89:0/3502793319 learned_addr learned my addr 172.21.15.89:0/3502793319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f52237fe700 1 --2- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52241316c0 0x7f5224131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5223fff700 1 -- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52241316c0 msgr2=0x7f5224131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5223fff700 1 --2- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52241316c0 0x7f5224131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5223fff700 1 -- 172.21.15.89:0/3502793319 >> v1:172.21.15.89:6789/0 conn(0x7f522409d9f0 legacy=0x7f522406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5223fff700 1 -- 172.21.15.89:0/3502793319 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522409e760 con 0x7f522406ae80 2023-09-30T22:22:22.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f52237fe700 1 --2- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52241316c0 0x7f5224131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:22.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.974+0000 7f5223fff700 1 --2- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522406ae80 0x7f5224139550 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f521c009e80 tx=0x7f521c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf64b1c7da199a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:22.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.975+0000 7f52227fc700 1 -- 172.21.15.89:0/3502793319 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c00e070 con 0x7f522406ae80 2023-09-30T22:22:22.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.975+0000 7f52227fc700 1 -- 172.21.15.89:0/3502793319 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c00b910 con 0x7f522406ae80 2023-09-30T22:22:22.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.975+0000 7f52227fc700 1 -- 172.21.15.89:0/3502793319 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c00bcc0 con 0x7f522406ae80 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.976+0000 7f5229d0e700 1 -- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522406ae80 msgr2=0x7f5224139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.976+0000 7f5229d0e700 1 --2- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522406ae80 0x7f5224139550 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f521c009e80 tx=0x7f521c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.976+0000 7f5229d0e700 1 -- 172.21.15.89:0/3502793319 shutdown_connections 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.976+0000 7f5229d0e700 1 --2- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f52241316c0 0x7f5224131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.976+0000 7f5229d0e700 1 --2- 172.21.15.89:0/3502793319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522406ae80 0x7f5224139550 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.976+0000 7f5229d0e700 1 -- 172.21.15.89:0/3502793319 >> 172.21.15.89:0/3502793319 conn(0x7f5224066f50 msgr2=0x7f522409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.976+0000 7f5229d0e700 1 -- 172.21.15.89:0/3502793319 shutdown_connections 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.976+0000 7f5229d0e700 1 -- 172.21.15.89:0/3502793319 wait complete. 2023-09-30T22:22:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.977+0000 7f5229d0e700 1 Processor -- start 2023-09-30T22:22:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.977+0000 7f5229d0e700 1 -- start start 2023-09-30T22:22:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5229d0e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f522406ae80 0x7f5224071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5229d0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522409d9f0 0x7f5224071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5229d0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52241316c0 0x7f52241c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5229d0e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52241331e0 con 0x7f52241316c0 2023-09-30T22:22:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5229d0e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5224133580 con 0x7f522406ae80 2023-09-30T22:22:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5229d0e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5224133de0 con 0x7f522409d9f0 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522409d9f0 0x7f5224071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f52237fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f522406ae80 0x7f5224071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522409d9f0 0x7f5224071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60174/0 (socket says 172.21.15.89:60174) 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5222ffd700 1 -- 172.21.15.89:0/2402580745 learned_addr learned my addr 172.21.15.89:0/2402580745 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f5223fff700 1 --2- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52241316c0 0x7f52241c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f52237fe700 1 -- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522409d9f0 msgr2=0x7f5224071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f52237fe700 1 --2- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522409d9f0 0x7f5224071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f52237fe700 1 -- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52241316c0 msgr2=0x7f52241c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.978+0000 7f52237fe700 1 --2- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52241316c0 0x7f52241c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.979+0000 7f52237fe700 1 -- 172.21.15.89:0/2402580745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52140094a0 con 0x7f522406ae80 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.979+0000 7f52237fe700 1 --2- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f522406ae80 0x7f5224071650 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f521400e9e0 tx=0x7f521400ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.979+0000 7f520bfff700 1 -- 172.21.15.89:0/2402580745 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400ee80 con 0x7f522406ae80 2023-09-30T22:22:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.979+0000 7f520bfff700 1 -- 172.21.15.89:0/2402580745 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521400c850 con 0x7f522406ae80 2023-09-30T22:22:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.979+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c009b30 con 0x7f522406ae80 2023-09-30T22:22:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.979+0000 7f520bfff700 1 -- 172.21.15.89:0/2402580745 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400cc00 con 0x7f522406ae80 2023-09-30T22:22:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.979+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5224132470 con 0x7f522406ae80 2023-09-30T22:22:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.980+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5224074ea0 con 0x7f522406ae80 2023-09-30T22:22:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.980+0000 7f520bfff700 1 -- 172.21.15.89:0/2402580745 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f521400cd60 con 0x7f522406ae80 2023-09-30T22:22:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.980+0000 7f520bfff700 1 -- 172.21.15.89:0/2402580745 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f521403f770 con 0x7f522406ae80 2023-09-30T22:22:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:22.989+0000 7f520bfff700 1 -- 172.21.15.89:0/2402580745 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f521401bde0 con 0x7f522406ae80 2023-09-30T22:22:23.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.013+0000 7f0573a2f700 1 Processor -- start 2023-09-30T22:22:23.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.013+0000 7f0573a2f700 1 -- start start 2023-09-30T22:22:23.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0573a2f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f056c131810 0x7f056c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0573a2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c09a580 0x7f056c139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0573a2f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c1313c0 con 0x7f056c099a40 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0573a2f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056c06e8c0 con 0x7f056c131810 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0573a2f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c071d20 con 0x7f056c09a580 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0571fcc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c09a580 0x7f056c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f05717cb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f056c131810 0x7f056c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f05717cb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f056c131810 0x7f056c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40750/0 (socket says 172.21.15.89:40750) 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0571fcc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c09a580 0x7f056c139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34152/0 (socket says 172.21.15.89:34152) 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f05717cb700 1 -- 172.21.15.89:0/2442910085 learned_addr learned my addr 172.21.15.89:0/2442910085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f05717cb700 1 -- 172.21.15.89:0/2442910085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c09a580 msgr2=0x7f056c139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f05717cb700 1 --2- 172.21.15.89:0/2442910085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c09a580 0x7f056c139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f05717cb700 1 -- 172.21.15.89:0/2442910085 >> v1:172.21.15.89:6789/0 conn(0x7f056c099a40 legacy=0x7f056c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f05717cb700 1 -- 172.21.15.89:0/2442910085 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0721a0 con 0x7f056c131810 2023-09-30T22:22:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0563fff700 1 -- 172.21.15.89:0/2442910085 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230762573 0 0) 0x7f056c1313c0 con 0x7f056c099a40 2023-09-30T22:22:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.014+0000 7f0571fcc700 1 --2- 172.21.15.89:0/2442910085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c09a580 0x7f056c139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.015+0000 7f05717cb700 1 --2- 172.21.15.89:0/2442910085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c131810 0x7f056c131be0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f05680098f0 tx=0x7f0568009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59142eb493bd3fa5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.015+0000 7f0563fff700 1 -- 172.21.15.89:0/2442910085 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568011830 con 0x7f056c131810 2023-09-30T22:22:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.015+0000 7f0563fff700 1 -- 172.21.15.89:0/2442910085 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568011990 con 0x7f056c131810 2023-09-30T22:22:23.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.015+0000 7f0563fff700 1 -- 172.21.15.89:0/2442910085 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568011d20 con 0x7f056c131810 2023-09-30T22:22:23.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.015+0000 7f0573a2f700 1 -- 172.21.15.89:0/2442910085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c131810 msgr2=0x7f056c131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.015+0000 7f0573a2f700 1 --2- 172.21.15.89:0/2442910085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c131810 0x7f056c131be0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f05680098f0 tx=0x7f0568009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:23.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.016+0000 7f0573a2f700 1 -- 172.21.15.89:0/2442910085 shutdown_connections 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.016+0000 7f0573a2f700 1 --2- 172.21.15.89:0/2442910085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c09a580 0x7f056c139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.016+0000 7f0573a2f700 1 --2- 172.21.15.89:0/2442910085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c131810 0x7f056c131be0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.016+0000 7f0573a2f700 1 -- 172.21.15.89:0/2442910085 >> 172.21.15.89:0/2442910085 conn(0x7f056c0a2aa0 msgr2=0x7f056c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.016+0000 7f0573a2f700 1 -- 172.21.15.89:0/2442910085 shutdown_connections 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.016+0000 7f0573a2f700 1 -- 172.21.15.89:0/2442910085 wait complete. 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0573a2f700 1 Processor -- start 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0573a2f700 1 -- start start 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0573a2f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c099a40 0x7f056c11b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0573a2f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c09a580 0x7f056c11bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0573a2f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f056c131810 0x7f056c11c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0573a2f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c12ec70 con 0x7f056c09a580 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0573a2f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c11dab0 con 0x7f056c099a40 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0573a2f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f056c11e6e0 con 0x7f056c131810 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0570fca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c09a580 0x7f056c11bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0570fca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c09a580 0x7f056c11bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40752/0 (socket says 172.21.15.89:40752) 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0570fca700 1 -- 172.21.15.89:0/3346343173 learned_addr learned my addr 172.21.15.89:0/3346343173 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f05717cb700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c099a40 0x7f056c11b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.017+0000 7f0571fcc700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f056c131810 0x7f056c11c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0570fca700 1 -- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f056c131810 msgr2=0x7f056c11c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0570fca700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f056c131810 0x7f056c11c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0570fca700 1 -- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c099a40 msgr2=0x7f056c11b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0570fca700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c099a40 0x7f056c11b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0570fca700 1 -- 172.21.15.89:0/3346343173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056400f050 con 0x7f056c09a580 2023-09-30T22:22:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0571fcc700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f056c131810 0x7f056c11c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0570fca700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c09a580 0x7f056c11bdc0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c550 tx=0x7f055c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0561ffb700 1 -- 172.21.15.89:0/3346343173 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0054e0 con 0x7f056c09a580 2023-09-30T22:22:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0568009550 con 0x7f056c09a580 2023-09-30T22:22:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0561ffb700 1 -- 172.21.15.89:0/3346343173 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c003680 con 0x7f056c09a580 2023-09-30T22:22:23.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0561ffb700 1 -- 172.21.15.89:0/3346343173 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c003a10 con 0x7f056c09a580 2023-09-30T22:22:23.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.018+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c06eb00 con 0x7f056c09a580 2023-09-30T22:22:23.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.019+0000 7f0561ffb700 1 -- 172.21.15.89:0/3346343173 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f055c003b70 con 0x7f056c09a580 2023-09-30T22:22:23.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.019+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c04f2a0 con 0x7f056c09a580 2023-09-30T22:22:23.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.019+0000 7f0561ffb700 1 -- 172.21.15.89:0/3346343173 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f055c03ecd0 con 0x7f056c09a580 2023-09-30T22:22:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.028+0000 7f0561ffb700 1 -- 172.21.15.89:0/3346343173 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f055c019b40 con 0x7f056c09a580 2023-09-30T22:22:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.050+0000 7fd88e2f2700 1 Processor -- start 2023-09-30T22:22:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.050+0000 7fd88e2f2700 1 -- start start 2023-09-30T22:22:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd88e2f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8881338a0 0x7fd888133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd88e2f2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd88809de50 0x7fd88809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd88e2f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd88809e760 0x7fd88813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd88e2f2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88806fef0 con 0x7fd88809de50 2023-09-30T22:22:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd88e2f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88806ef30 con 0x7fd88809e760 2023-09-30T22:22:23.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd88e2f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881334b0 con 0x7fd8881338a0 2023-09-30T22:22:23.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd8877fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd88809de50 0x7fd88809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd8877fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd88809de50 0x7fd88809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40762/0 (socket says 172.21.15.89:40762) 2023-09-30T22:22:23.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd8877fe700 1 -- 172.21.15.89:0/4279593749 learned_addr learned my addr 172.21.15.89:0/4279593749 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:23.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd887fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8881338a0 0x7fd888133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.051+0000 7fd88c88f700 1 --2- 172.21.15.89:0/4279593749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd88809e760 0x7fd88813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd887fff700 1 -- 172.21.15.89:0/4279593749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd88809e760 msgr2=0x7fd88813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd887fff700 1 --2- 172.21.15.89:0/4279593749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd88809e760 0x7fd88813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd887fff700 1 -- 172.21.15.89:0/4279593749 >> v2:172.21.15.89:3300/0 conn(0x7fd88809de50 msgr2=0x7fd88809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd887fff700 1 --2- 172.21.15.89:0/4279593749 >> v2:172.21.15.89:3300/0 conn(0x7fd88809de50 0x7fd88809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd887fff700 1 -- 172.21.15.89:0/4279593749 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88806fc30 con 0x7fd8881338a0 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd8877fe700 1 --2- 172.21.15.89:0/4279593749 >> v2:172.21.15.89:3300/0 conn(0x7fd88809de50 0x7fd88809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd887fff700 1 --2- 172.21.15.89:0/4279593749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8881338a0 0x7fd888133c70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd878009840 tx=0x7fd878009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f24e93e241dfe6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd886ffd700 1 -- 172.21.15.89:0/4279593749 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878011830 con 0x7fd8881338a0 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd886ffd700 1 -- 172.21.15.89:0/4279593749 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878011990 con 0x7fd8881338a0 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.052+0000 7fd886ffd700 1 -- 172.21.15.89:0/4279593749 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878011d20 con 0x7fd8881338a0 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 -- 172.21.15.89:0/4279593749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8881338a0 msgr2=0x7fd888133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 --2- 172.21.15.89:0/4279593749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8881338a0 0x7fd888133c70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd878009840 tx=0x7fd878009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 -- 172.21.15.89:0/4279593749 shutdown_connections 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 --2- 172.21.15.89:0/4279593749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd88809e760 0x7fd88813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 --2- 172.21.15.89:0/4279593749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8881338a0 0x7fd888133c70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 --2- 172.21.15.89:0/4279593749 >> v2:172.21.15.89:3300/0 conn(0x7fd88809de50 0x7fd88809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 -- 172.21.15.89:0/4279593749 >> 172.21.15.89:0/4279593749 conn(0x7fd8880a2aa0 msgr2=0x7fd8880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 -- 172.21.15.89:0/4279593749 shutdown_connections 2023-09-30T22:22:23.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.053+0000 7fd88e2f2700 1 -- 172.21.15.89:0/4279593749 wait complete. 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.054+0000 7fd88e2f2700 1 Processor -- start 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.054+0000 7fd88e2f2700 1 -- start start 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd88e2f2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd88809de50 0x7fd88811d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd88e2f2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd88809e760 0x7fd88811dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd88e2f2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8881338a0 0x7fd88811e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd88e2f2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd888121370 con 0x7fd88809de50 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd88e2f2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88811fa40 con 0x7fd8881338a0 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd88e2f2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd888120b00 con 0x7fd88809e760 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd8877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd88809e760 0x7fd88811dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd8877fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd88809e760 0x7fd88811dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60192/0 (socket says 172.21.15.89:60192) 2023-09-30T22:22:23.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd8877fe700 1 -- 172.21.15.89:0/2797777399 learned_addr learned my addr 172.21.15.89:0/2797777399 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd887fff700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd88809de50 0x7fd88811d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd88c88f700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8881338a0 0x7fd88811e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd887fff700 1 -- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd88809e760 msgr2=0x7fd88811dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd887fff700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd88809e760 0x7fd88811dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd887fff700 1 -- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8881338a0 msgr2=0x7fd88811e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd887fff700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8881338a0 0x7fd88811e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd887fff700 1 -- 172.21.15.89:0/2797777399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c009550 con 0x7fd88809de50 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd88c88f700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8881338a0 0x7fd88811e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.055+0000 7fd8877fe700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd88809e760 0x7fd88811dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.056+0000 7fd887fff700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd88809de50 0x7fd88811d850 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd88806dc90 tx=0x7fd878004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:23.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.056+0000 7fd884ff9700 1 -- 172.21.15.89:0/2797777399 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878004510 con 0x7fd88809de50 2023-09-30T22:22:23.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.056+0000 7fd884ff9700 1 -- 172.21.15.89:0/2797777399 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8780036a0 con 0x7fd88809de50 2023-09-30T22:22:23.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.056+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8780094a0 con 0x7fd88809de50 2023-09-30T22:22:23.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.056+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8881206b0 con 0x7fd88809de50 2023-09-30T22:22:23.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.056+0000 7fd884ff9700 1 -- 172.21.15.89:0/2797777399 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878003a50 con 0x7fd88809de50 2023-09-30T22:22:23.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.057+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd888120ef0 con 0x7fd88809de50 2023-09-30T22:22:23.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.057+0000 7fd884ff9700 1 -- 172.21.15.89:0/2797777399 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd878003bb0 con 0x7fd88809de50 2023-09-30T22:22:23.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.057+0000 7fd884ff9700 1 -- 172.21.15.89:0/2797777399 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd8780408e0 con 0x7fd88809de50 2023-09-30T22:22:23.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.066+0000 7fd884ff9700 1 -- 172.21.15.89:0/2797777399 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd87801a420 con 0x7fd88809de50 2023-09-30T22:22:23.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.154+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9f48074ea0 con 0x7f9f4806f050 2023-09-30T22:22:23.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.155+0000 7f9f36ffd700 1 -- 172.21.15.89:0/1705692255 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f3c015020 con 0x7f9f4806f050 2023-09-30T22:22:23.155 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 msgr2=0x7f9f481cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f481cd1b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c000c00 tx=0x7f9f3c004750 comp rx=0 tx=0).stop 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 shutdown_connections 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f4813aa10 0x7f9f4813ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f4806e570 0x7f9f481ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 --2- 172.21.15.89:0/1705692255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9f4806f050 0x7f9f481cd1b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 >> 172.21.15.89:0/1705692255 conn(0x7f9f480a2aa0 msgr2=0x7f9f4813f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 shutdown_connections 2023-09-30T22:22:23.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.157+0000 7f9f4c952700 1 -- 172.21.15.89:0/1705692255 wait complete. 2023-09-30T22:22:23.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.186+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f522406c360 con 0x7f522406ae80 2023-09-30T22:22:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.186+0000 7f520bfff700 1 -- 172.21.15.89:0/2402580745 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5214017070 con 0x7f522406ae80 2023-09-30T22:22:23.186 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f522406ae80 msgr2=0x7f5224071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 --2- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f522406ae80 0x7f5224071650 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f521400e9e0 tx=0x7f521400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 shutdown_connections 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 --2- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52241316c0 0x7f52241c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 --2- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f522406ae80 0x7f5224071650 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 --2- 172.21.15.89:0/2402580745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f522409d9f0 0x7f5224071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 >> 172.21.15.89:0/2402580745 conn(0x7f5224066f50 msgr2=0x7f522409b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 shutdown_connections 2023-09-30T22:22:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.189+0000 7f5229d0e700 1 -- 172.21.15.89:0/2402580745 wait complete. 2023-09-30T22:22:23.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.225+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f056c074ea0 con 0x7f056c09a580 2023-09-30T22:22:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.225+0000 7f0561ffb700 1 -- 172.21.15.89:0/3346343173 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f055c017030 con 0x7f056c09a580 2023-09-30T22:22:23.225 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.227+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c09a580 msgr2=0x7f056c11bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.227+0000 7f0573a2f700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c09a580 0x7f056c11bdc0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c550 tx=0x7f055c00c860 comp rx=0 tx=0).stop 2023-09-30T22:22:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.227+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 shutdown_connections 2023-09-30T22:22:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.227+0000 7f0573a2f700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f056c131810 0x7f056c11c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.227+0000 7f0573a2f700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f056c099a40 0x7f056c11b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.227+0000 7f0573a2f700 1 --2- 172.21.15.89:0/3346343173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f056c09a580 0x7f056c11bdc0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.227+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 >> 172.21.15.89:0/3346343173 conn(0x7f056c0a2aa0 msgr2=0x7f056c070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.228+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 shutdown_connections 2023-09-30T22:22:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.228+0000 7f0573a2f700 1 -- 172.21.15.89:0/3346343173 wait complete. 2023-09-30T22:22:23.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.258+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd8880a0780 con 0x7fd88809de50 2023-09-30T22:22:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.259+0000 7fd884ff9700 1 -- 172.21.15.89:0/2797777399 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd878015020 con 0x7fd88809de50 2023-09-30T22:22:23.259 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd88809de50 msgr2=0x7fd88811d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd88809de50 0x7fd88811d850 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd88806dc90 tx=0x7fd878004750 comp rx=0 tx=0).stop 2023-09-30T22:22:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 shutdown_connections 2023-09-30T22:22:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8881338a0 0x7fd88811e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd88809de50 0x7fd88811d850 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 --2- 172.21.15.89:0/2797777399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd88809e760 0x7fd88811dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 >> 172.21.15.89:0/2797777399 conn(0x7fd8880a2aa0 msgr2=0x7fd88809ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 shutdown_connections 2023-09-30T22:22:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:23.261+0000 7fd88e2f2700 1 -- 172.21.15.89:0/2797777399 wait complete. 2023-09-30T22:22:23.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:24.170 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:24.201 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:24.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:24.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:24.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.317+0000 7fa250bf0700 1 Processor -- start 2023-09-30T22:22:24.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.317+0000 7fa250bf0700 1 -- start start 2023-09-30T22:22:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa250bf0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa24c09d6f0 0x7fa24c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa250bf0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e000 0x7fa24c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa250bf0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa24c09b120 con 0x7fa24c09e410 2023-09-30T22:22:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa250bf0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24c06b460 con 0x7fa24c09d6f0 2023-09-30T22:22:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa250bf0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c09a7e0 con 0x7fa24c09e000 2023-09-30T22:22:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa24a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa24c09d6f0 0x7fa24c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e000 0x7fa24c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa24a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa24c09d6f0 0x7fa24c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40790/0 (socket says 172.21.15.89:40790) 2023-09-30T22:22:24.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa24a59c700 1 -- 172.21.15.89:0/3180405900 learned_addr learned my addr 172.21.15.89:0/3180405900 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa249d9b700 1 -- 172.21.15.89:0/3180405900 >> v2:172.21.15.89:3300/0 conn(0x7fa24c09d6f0 msgr2=0x7fa24c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:24.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa249d9b700 1 --2- 172.21.15.89:0/3180405900 >> v2:172.21.15.89:3300/0 conn(0x7fa24c09d6f0 0x7fa24c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa249d9b700 1 -- 172.21.15.89:0/3180405900 >> v1:172.21.15.89:6789/0 conn(0x7fa24c09e410 legacy=0x7fa24c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa249d9b700 1 -- 172.21.15.89:0/3180405900 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c09a500 con 0x7fa24c09e000 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa24a59c700 1 --2- 172.21.15.89:0/3180405900 >> v2:172.21.15.89:3300/0 conn(0x7fa24c09d6f0 0x7fa24c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.318+0000 7fa249d9b700 1 --2- 172.21.15.89:0/3180405900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e000 0x7fa24c1339f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa240009870 tx=0x7fa240009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddc1f72c5bb5db6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.319+0000 7fa24959a700 1 -- 172.21.15.89:0/3180405900 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240011830 con 0x7fa24c09e000 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.319+0000 7fa24959a700 1 -- 172.21.15.89:0/3180405900 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa240011990 con 0x7fa24c09e000 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.319+0000 7fa24959a700 1 -- 172.21.15.89:0/3180405900 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240011d20 con 0x7fa24c09e000 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.319+0000 7fa250bf0700 1 -- 172.21.15.89:0/3180405900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e000 msgr2=0x7fa24c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.319+0000 7fa250bf0700 1 --2- 172.21.15.89:0/3180405900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e000 0x7fa24c1339f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa240009870 tx=0x7fa240009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.319+0000 7fa250bf0700 1 -- 172.21.15.89:0/3180405900 shutdown_connections 2023-09-30T22:22:24.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.320+0000 7fa250bf0700 1 --2- 172.21.15.89:0/3180405900 >> v2:172.21.15.89:3300/0 conn(0x7fa24c09d6f0 0x7fa24c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.320+0000 7fa250bf0700 1 --2- 172.21.15.89:0/3180405900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e000 0x7fa24c1339f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.320+0000 7fa250bf0700 1 -- 172.21.15.89:0/3180405900 >> 172.21.15.89:0/3180405900 conn(0x7fa24c066f50 msgr2=0x7fa24c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.320+0000 7fa250bf0700 1 -- 172.21.15.89:0/3180405900 shutdown_connections 2023-09-30T22:22:24.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.320+0000 7fa250bf0700 1 -- 172.21.15.89:0/3180405900 wait complete. 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.321+0000 7fa250bf0700 1 Processor -- start 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.321+0000 7fa250bf0700 1 -- start start 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa250bf0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa24c09d6f0 0x7fa24c1a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa250bf0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa24c09e000 0x7fa24c1a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa250bf0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e410 0x7fa24c1a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa250bf0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c1a6b00 con 0x7fa24c09e000 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa250bf0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c1a8690 con 0x7fa24c09e410 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa250bf0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c1a7cb0 con 0x7fa24c09d6f0 2023-09-30T22:22:24.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa24c09e000 0x7fa24c1a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa24c09e000 0x7fa24c1a5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40796/0 (socket says 172.21.15.89:40796) 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa249d9b700 1 -- 172.21.15.89:0/2677595780 learned_addr learned my addr 172.21.15.89:0/2677595780 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa24ad9d700 1 --2- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e410 0x7fa24c1a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa249d9b700 1 -- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa24c09d6f0 msgr2=0x7fa24c1a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.322+0000 7fa249d9b700 1 --2- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa24c09d6f0 0x7fa24c1a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa249d9b700 1 -- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e410 msgr2=0x7fa24c1a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa249d9b700 1 --2- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e410 0x7fa24c1a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa249d9b700 1 -- 172.21.15.89:0/2677595780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c009550 con 0x7fa24c09e000 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa249d9b700 1 --2- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa24c09e000 0x7fa24c1a5390 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa240000c00 tx=0x7fa240004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa23affd700 1 -- 172.21.15.89:0/2677595780 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2400038a0 con 0x7fa24c09e000 2023-09-30T22:22:24.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa23affd700 1 -- 172.21.15.89:0/2677595780 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24001f070 con 0x7fa24c09e000 2023-09-30T22:22:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa23affd700 1 -- 172.21.15.89:0/2677595780 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240003c40 con 0x7fa24c09e000 2023-09-30T22:22:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2400094a0 con 0x7fa24c09e000 2023-09-30T22:22:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.323+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c09c710 con 0x7fa24c09e000 2023-09-30T22:22:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.324+0000 7fa23affd700 1 -- 172.21.15.89:0/2677595780 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa240026480 con 0x7fa24c09e000 2023-09-30T22:22:24.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.324+0000 7fa23affd700 1 -- 172.21.15.89:0/2677595780 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa240021070 con 0x7fa24c09e000 2023-09-30T22:22:24.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.324+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24c1a8330 con 0x7fa24c09e000 2023-09-30T22:22:24.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.330+0000 7fa23affd700 1 -- 172.21.15.89:0/2677595780 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa24002b070 con 0x7fa24c09e000 2023-09-30T22:22:24.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.358+0000 7f5a9b71e700 1 Processor -- start 2023-09-30T22:22:24.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.359+0000 7f5a9b71e700 1 -- start start 2023-09-30T22:22:24.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.359+0000 7f5a9b71e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9409e1d0 0x7f5a94135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.359+0000 7f5a9b71e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a94135e50 0x7f5a94138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.359+0000 7f5a9b71e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9409f130 con 0x7f5a9409d6f0 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.359+0000 7f5a9b71e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9409f520 con 0x7f5a94135e50 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.359+0000 7f5a9b71e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9409e930 con 0x7f5a9409e1d0 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.359+0000 7f5a994ba700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5a9409d6f0 0x7f5a9409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47480/0 (socket says 172.21.15.89:47480) 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.359+0000 7f5a994ba700 1 -- 172.21.15.89:0/2710138702 learned_addr learned my addr 172.21.15.89:0/2710138702 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a98cb9700 1 --2- 172.21.15.89:0/2710138702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9409e1d0 0x7f5a94135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a99cbb700 1 --2- 172.21.15.89:0/2710138702 >> v2:172.21.15.89:3300/0 conn(0x7f5a94135e50 0x7f5a94138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a8bfff700 1 -- 172.21.15.89:0/2710138702 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788339752 0 0) 0x7f5a9409f130 con 0x7f5a9409d6f0 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a8bfff700 1 -- 172.21.15.89:0/2710138702 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a800035b0 con 0x7f5a9409d6f0 2023-09-30T22:22:24.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a98cb9700 1 -- 172.21.15.89:0/2710138702 >> v2:172.21.15.89:3300/0 conn(0x7f5a94135e50 msgr2=0x7f5a94138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:24.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a98cb9700 1 --2- 172.21.15.89:0/2710138702 >> v2:172.21.15.89:3300/0 conn(0x7f5a94135e50 0x7f5a94138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a98cb9700 1 -- 172.21.15.89:0/2710138702 >> v1:172.21.15.89:6789/0 conn(0x7f5a9409d6f0 legacy=0x7f5a9409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a98cb9700 1 -- 172.21.15.89:0/2710138702 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9409b2d0 con 0x7f5a9409e1d0 2023-09-30T22:22:24.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.360+0000 7f5a98cb9700 1 --2- 172.21.15.89:0/2710138702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9409e1d0 0x7f5a94135910 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009870 tx=0x7f5a84009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf0765cb4174207 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:24.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.361+0000 7f5a8bfff700 1 -- 172.21.15.89:0/2710138702 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84011830 con 0x7f5a9409e1d0 2023-09-30T22:22:24.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.361+0000 7f5a8bfff700 1 -- 172.21.15.89:0/2710138702 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a84011990 con 0x7f5a9409e1d0 2023-09-30T22:22:24.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.361+0000 7f5a8bfff700 1 -- 172.21.15.89:0/2710138702 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84011d20 con 0x7f5a9409e1d0 2023-09-30T22:22:24.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.361+0000 7f5a9b71e700 1 -- 172.21.15.89:0/2710138702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9409e1d0 msgr2=0x7f5a94135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.361+0000 7f5a9b71e700 1 --2- 172.21.15.89:0/2710138702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9409e1d0 0x7f5a94135910 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009870 tx=0x7f5a84009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:24.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.362+0000 7f5a9b71e700 1 -- 172.21.15.89:0/2710138702 shutdown_connections 2023-09-30T22:22:24.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.362+0000 7f5a9b71e700 1 --2- 172.21.15.89:0/2710138702 >> v2:172.21.15.89:3300/0 conn(0x7f5a94135e50 0x7f5a94138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.362+0000 7f5a9b71e700 1 --2- 172.21.15.89:0/2710138702 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a9409e1d0 0x7f5a94135910 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.362+0000 7f5a9b71e700 1 -- 172.21.15.89:0/2710138702 >> 172.21.15.89:0/2710138702 conn(0x7f5a94066f50 msgr2=0x7f5a9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:24.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.362+0000 7f5a9b71e700 1 -- 172.21.15.89:0/2710138702 shutdown_connections 2023-09-30T22:22:24.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.362+0000 7f5a9b71e700 1 -- 172.21.15.89:0/2710138702 wait complete. 2023-09-30T22:22:24.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.363+0000 7f5a9b71e700 1 Processor -- start 2023-09-30T22:22:24.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.363+0000 7f5a9b71e700 1 -- start start 2023-09-30T22:22:24.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a9b71e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9409d6f0 0x7f5a941397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a9b71e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9409e1d0 0x7f5a94139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a9b71e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a94135e50 0x7f5a9413a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a9b71e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94135b60 con 0x7f5a9409d6f0 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a9b71e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94120580 con 0x7f5a94135e50 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a9b71e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9411fb90 con 0x7f5a9409e1d0 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a98cb9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9409e1d0 0x7f5a94139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a98cb9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9409e1d0 0x7f5a94139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60202/0 (socket says 172.21.15.89:60202) 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a98cb9700 1 -- 172.21.15.89:0/3927107409 learned_addr learned my addr 172.21.15.89:0/3927107409 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a99cbb700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a94135e50 0x7f5a9413a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a994ba700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9409d6f0 0x7f5a941397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a98cb9700 1 -- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a94135e50 msgr2=0x7f5a9413a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a98cb9700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a94135e50 0x7f5a9413a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.364+0000 7f5a98cb9700 1 -- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9409d6f0 msgr2=0x7f5a941397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.365+0000 7f5a98cb9700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9409d6f0 0x7f5a941397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.365+0000 7f5a98cb9700 1 -- 172.21.15.89:0/3927107409 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a840094a0 con 0x7f5a9409e1d0 2023-09-30T22:22:24.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.365+0000 7f5a98cb9700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9409e1d0 0x7f5a94139cf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5a84000c00 tx=0x7f5a84004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:24.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.365+0000 7f5a89ffb700 1 -- 172.21.15.89:0/3927107409 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a840036a0 con 0x7f5a9409e1d0 2023-09-30T22:22:24.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.365+0000 7f5a89ffb700 1 -- 172.21.15.89:0/3927107409 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8401f070 con 0x7f5a9409e1d0 2023-09-30T22:22:24.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.365+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a94120f60 con 0x7f5a9409e1d0 2023-09-30T22:22:24.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.365+0000 7f5a89ffb700 1 -- 172.21.15.89:0/3927107409 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84003a00 con 0x7f5a9409e1d0 2023-09-30T22:22:24.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.365+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a940a2090 con 0x7f5a9409e1d0 2023-09-30T22:22:24.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.366+0000 7f5a89ffb700 1 -- 172.21.15.89:0/3927107409 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a84003b60 con 0x7f5a9409e1d0 2023-09-30T22:22:24.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.366+0000 7f5a89ffb700 1 -- 172.21.15.89:0/3927107409 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5a84021070 con 0x7f5a9409e1d0 2023-09-30T22:22:24.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.366+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9409c760 con 0x7f5a9409e1d0 2023-09-30T22:22:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.375+0000 7f5a89ffb700 1 -- 172.21.15.89:0/3927107409 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a8402b080 con 0x7f5a9409e1d0 2023-09-30T22:22:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.395+0000 7fbb5cb47700 1 Processor -- start 2023-09-30T22:22:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.395+0000 7fbb5cb47700 1 -- start start 2023-09-30T22:22:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb5cb47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5806a770 0x7fbb5809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb5cb47700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb5813d1f0 0x7fbb5813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb5cb47700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5809a810 con 0x7fbb5809dc30 2023-09-30T22:22:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb5cb47700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5806ab40 con 0x7fbb5813d1f0 2023-09-30T22:22:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb5cb47700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5809ebc0 con 0x7fbb5806a770 2023-09-30T22:22:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb56d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb5813d1f0 0x7fbb5813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb56d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb5813d1f0 0x7fbb5813f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40816/0 (socket says 172.21.15.89:40816) 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb56d9d700 1 -- 172.21.15.89:0/3353238254 learned_addr learned my addr 172.21.15.89:0/3353238254 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb55d9b700 1 --1- 172.21.15.89:0/3353238254 >> v1:172.21.15.89:6789/0 conn(0x7fbb5809dc30 0x7fbb5809e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47488/0 (socket says 172.21.15.89:47488) 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.396+0000 7fbb5659c700 1 --2- 172.21.15.89:0/3353238254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5806a770 0x7fbb5809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.397+0000 7fbb56d9d700 1 -- 172.21.15.89:0/3353238254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5806a770 msgr2=0x7fbb5809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.397+0000 7fbb56d9d700 1 --2- 172.21.15.89:0/3353238254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5806a770 0x7fbb5809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.397+0000 7fbb56d9d700 1 -- 172.21.15.89:0/3353238254 >> v1:172.21.15.89:6789/0 conn(0x7fbb5809dc30 legacy=0x7fbb5809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.397+0000 7fbb56d9d700 1 -- 172.21.15.89:0/3353238254 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5809e920 con 0x7fbb5813d1f0 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.397+0000 7fbb56d9d700 1 --2- 172.21.15.89:0/3353238254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 0x7fbb5813f5d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009e80 tx=0x7fbb5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbb9c30ad289b5e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.397+0000 7fbb5559a700 1 -- 172.21.15.89:0/3353238254 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000e070 con 0x7fbb5813d1f0 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.397+0000 7fbb5559a700 1 -- 172.21.15.89:0/3353238254 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5000b980 con 0x7fbb5813d1f0 2023-09-30T22:22:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.397+0000 7fbb5559a700 1 -- 172.21.15.89:0/3353238254 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000bd30 con 0x7fbb5813d1f0 2023-09-30T22:22:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.398+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3353238254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 msgr2=0x7fbb5813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.398+0000 7fbb5cb47700 1 --2- 172.21.15.89:0/3353238254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 0x7fbb5813f5d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009e80 tx=0x7fbb5000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.398+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3353238254 shutdown_connections 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.398+0000 7fbb5cb47700 1 --2- 172.21.15.89:0/3353238254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 0x7fbb5813f5d0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.398+0000 7fbb5cb47700 1 --2- 172.21.15.89:0/3353238254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5806a770 0x7fbb5809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.398+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3353238254 >> 172.21.15.89:0/3353238254 conn(0x7fbb58066f50 msgr2=0x7fbb5809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.398+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3353238254 shutdown_connections 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.398+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3353238254 wait complete. 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb5cb47700 1 Processor -- start 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb5cb47700 1 -- start start 2023-09-30T22:22:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb5cb47700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb5806a770 0x7fbb58071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb5cb47700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5809dc30 0x7fbb5813b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb5cb47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 0x7fbb5813bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb5cb47700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5809f1a0 con 0x7fbb5813d1f0 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb5cb47700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5811f040 con 0x7fbb5806a770 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb5cb47700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5811e380 con 0x7fbb5809dc30 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb55d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5809dc30 0x7fbb5813b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb55d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5809dc30 0x7fbb5813b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60214/0 (socket says 172.21.15.89:60214) 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb55d9b700 1 -- 172.21.15.89:0/3719240150 learned_addr learned my addr 172.21.15.89:0/3719240150 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.399+0000 7fbb56d9d700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 0x7fbb5813bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb5659c700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb5806a770 0x7fbb58071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb55d9b700 1 -- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb5806a770 msgr2=0x7fbb58071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb55d9b700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb5806a770 0x7fbb58071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb55d9b700 1 -- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 msgr2=0x7fbb5813bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb55d9b700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 0x7fbb5813bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb55d9b700 1 -- 172.21.15.89:0/3719240150 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50009b30 con 0x7fbb5809dc30 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb55d9b700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5809dc30 0x7fbb5813b6b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c4f0 tx=0x7fbb4c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb46ffd700 1 -- 172.21.15.89:0/3719240150 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c005cb0 con 0x7fbb5809dc30 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb46ffd700 1 -- 172.21.15.89:0/3719240150 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c005e10 con 0x7fbb5809dc30 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb46ffd700 1 -- 172.21.15.89:0/3719240150 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0072d0 con 0x7fbb5809dc30 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5811eaa0 con 0x7fbb5809dc30 2023-09-30T22:22:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.400+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5811ed60 con 0x7fbb5809dc30 2023-09-30T22:22:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.401+0000 7fbb46ffd700 1 -- 172.21.15.89:0/3719240150 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbb4c007430 con 0x7fbb5809dc30 2023-09-30T22:22:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.401+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb58074ea0 con 0x7fbb5809dc30 2023-09-30T22:22:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.402+0000 7fbb46ffd700 1 -- 172.21.15.89:0/3719240150 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbb4c03eab0 con 0x7fbb5809dc30 2023-09-30T22:22:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.406+0000 7fbb46ffd700 1 -- 172.21.15.89:0/3719240150 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb4c01b3b0 con 0x7fbb5809dc30 2023-09-30T22:22:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.455+0000 7fc94e9f3700 1 Processor -- start 2023-09-30T22:22:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.455+0000 7fc94e9f3700 1 -- start start 2023-09-30T22:22:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94e9f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806e570 0x7fc94806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94e9f3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc94806ee80 0x7fc948134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94e9f3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc948070ee0 con 0x7fc94809eca0 2023-09-30T22:22:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94e9f3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc948071480 con 0x7fc94806ee80 2023-09-30T22:22:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94e9f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc948071b90 con 0x7fc94806e570 2023-09-30T22:22:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc9477fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc94809eca0 0x7fc948134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47502/0 (socket says 172.21.15.89:47502) 2023-09-30T22:22:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc9477fe700 1 -- 172.21.15.89:0/2028583538 learned_addr learned my addr 172.21.15.89:0/2028583538 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:24.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94cf90700 1 --2- 172.21.15.89:0/2028583538 >> v2:172.21.15.89:3300/0 conn(0x7fc94806ee80 0x7fc948134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94cf90700 1 -- 172.21.15.89:0/2028583538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806e570 msgr2=0x7fc94806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94cf90700 1 --2- 172.21.15.89:0/2028583538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806e570 0x7fc94806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94cf90700 1 -- 172.21.15.89:0/2028583538 >> v1:172.21.15.89:6789/0 conn(0x7fc94809eca0 legacy=0x7fc948134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.456+0000 7fc94cf90700 1 -- 172.21.15.89:0/2028583538 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480711a0 con 0x7fc94806ee80 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc94cf90700 1 --2- 172.21.15.89:0/2028583538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806ee80 0x7fc948134d20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc940009d90 tx=0x7fc94000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5492b63985de087 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc946ffd700 1 -- 172.21.15.89:0/2028583538 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94000e070 con 0x7fc94806ee80 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc946ffd700 1 -- 172.21.15.89:0/2028583538 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94000bbd0 con 0x7fc94806ee80 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc946ffd700 1 -- 172.21.15.89:0/2028583538 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940012430 con 0x7fc94806ee80 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2028583538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806ee80 msgr2=0x7fc948134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc94e9f3700 1 --2- 172.21.15.89:0/2028583538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806ee80 0x7fc948134d20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc940009d90 tx=0x7fc94000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2028583538 shutdown_connections 2023-09-30T22:22:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc94e9f3700 1 --2- 172.21.15.89:0/2028583538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806ee80 0x7fc948134d20 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc94e9f3700 1 --2- 172.21.15.89:0/2028583538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806e570 0x7fc94806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.457+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2028583538 >> 172.21.15.89:0/2028583538 conn(0x7fc9480a2aa0 msgr2=0x7fc9480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.458+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2028583538 shutdown_connections 2023-09-30T22:22:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.458+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2028583538 wait complete. 2023-09-30T22:22:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.458+0000 7fc94e9f3700 1 Processor -- start 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc94e9f3700 1 -- start start 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc94e9f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806e570 0x7fc94813bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc94e9f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806ee80 0x7fc948134920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc94e9f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc948134e60 0x7fc9481389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc94e9f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94813b4d0 con 0x7fc94806e570 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc94e9f3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc948070610 con 0x7fc948134e60 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc94e9f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc94813b210 con 0x7fc94806ee80 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc9477fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806ee80 0x7fc948134920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc9477fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806ee80 0x7fc948134920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60238/0 (socket says 172.21.15.89:60238) 2023-09-30T22:22:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc9477fe700 1 -- 172.21.15.89:0/2783994583 learned_addr learned my addr 172.21.15.89:0/2783994583 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.459+0000 7fc947fff700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806e570 0x7fc94813bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc9477fe700 1 -- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc948134e60 msgr2=0x7fc9481389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc9477fe700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc948134e60 0x7fc9481389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc9477fe700 1 -- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806e570 msgr2=0x7fc94813bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc9477fe700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806e570 0x7fc94813bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc9477fe700 1 -- 172.21.15.89:0/2783994583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc940009a70 con 0x7fc94806ee80 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc947fff700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806e570 0x7fc94813bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc9477fe700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806ee80 0x7fc948134920 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00c4d0 tx=0x7fc93c00c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc944ff9700 1 -- 172.21.15.89:0/2783994583 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c005d70 con 0x7fc94806ee80 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc944ff9700 1 -- 172.21.15.89:0/2783994583 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c0070d0 con 0x7fc94806ee80 2023-09-30T22:22:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94809e250 con 0x7fc94806ee80 2023-09-30T22:22:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc944ff9700 1 -- 172.21.15.89:0/2783994583 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c007440 con 0x7fc94806ee80 2023-09-30T22:22:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.460+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc948139110 con 0x7fc94806ee80 2023-09-30T22:22:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.461+0000 7fc944ff9700 1 -- 172.21.15.89:0/2783994583 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc93c0075a0 con 0x7fc94806ee80 2023-09-30T22:22:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.461+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94813b620 con 0x7fc94806ee80 2023-09-30T22:22:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.461+0000 7fc944ff9700 1 -- 172.21.15.89:0/2783994583 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc93c03ea60 con 0x7fc94806ee80 2023-09-30T22:22:24.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.470+0000 7fc944ff9700 1 -- 172.21.15.89:0/2783994583 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc93c011be0 con 0x7fc94806ee80 2023-09-30T22:22:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.535+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa24c074ea0 con 0x7fa24c09e000 2023-09-30T22:22:24.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.535+0000 7fa23affd700 1 -- 172.21.15.89:0/2677595780 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa240026730 con 0x7fa24c09e000 2023-09-30T22:22:24.535 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:24.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.537+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa24c09e000 msgr2=0x7fa24c1a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.537+0000 7fa250bf0700 1 --2- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa24c09e000 0x7fa24c1a5390 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa240000c00 tx=0x7fa240004750 comp rx=0 tx=0).stop 2023-09-30T22:22:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.538+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 shutdown_connections 2023-09-30T22:22:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.538+0000 7fa250bf0700 1 --2- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa24c09e410 0x7fa24c1a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.538+0000 7fa250bf0700 1 --2- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa24c09d6f0 0x7fa24c1a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.538+0000 7fa250bf0700 1 --2- 172.21.15.89:0/2677595780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa24c09e000 0x7fa24c1a5390 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.538+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 >> 172.21.15.89:0/2677595780 conn(0x7fa24c066f50 msgr2=0x7fa24c1baf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.539+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 shutdown_connections 2023-09-30T22:22:24.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.539+0000 7fa250bf0700 1 -- 172.21.15.89:0/2677595780 wait complete. 2023-09-30T22:22:24.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:24.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.598+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5a94070ff0 con 0x7f5a9409e1d0 2023-09-30T22:22:24.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.598+0000 7f5a89ffb700 1 -- 172.21.15.89:0/3927107409 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a84035090 con 0x7f5a9409e1d0 2023-09-30T22:22:24.598 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:24.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.600+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9409e1d0 msgr2=0x7f5a94139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.600+0000 7f5a9b71e700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9409e1d0 0x7f5a94139cf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5a84000c00 tx=0x7f5a84004750 comp rx=0 tx=0).stop 2023-09-30T22:22:24.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.600+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 shutdown_connections 2023-09-30T22:22:24.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.600+0000 7f5a9b71e700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a94135e50 0x7f5a9413a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.600+0000 7f5a9b71e700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a9409d6f0 0x7f5a941397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.600+0000 7f5a9b71e700 1 --2- 172.21.15.89:0/3927107409 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a9409e1d0 0x7f5a94139cf0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.600+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 >> 172.21.15.89:0/3927107409 conn(0x7f5a94066f50 msgr2=0x7f5a940a23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:24.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.601+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 shutdown_connections 2023-09-30T22:22:24.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.601+0000 7f5a9b71e700 1 -- 172.21.15.89:0/3927107409 wait complete. 2023-09-30T22:22:24.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.605+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb5809a3d0 con 0x7fbb5809dc30 2023-09-30T22:22:24.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.606+0000 7fbb46ffd700 1 -- 172.21.15.89:0/3719240150 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb4c0192c0 con 0x7fbb5809dc30 2023-09-30T22:22:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5809dc30 msgr2=0x7fbb5813b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5809dc30 0x7fbb5813b6b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c4f0 tx=0x7fbb4c00c800 comp rx=0 tx=0).stop 2023-09-30T22:22:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 shutdown_connections 2023-09-30T22:22:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb5813d1f0 0x7fbb5813bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb5806a770 0x7fbb58071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 --2- 172.21.15.89:0/3719240150 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb5809dc30 0x7fbb5813b6b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 >> 172.21.15.89:0/3719240150 conn(0x7fbb58066f50 msgr2=0x7fbb5813d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:24.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 shutdown_connections 2023-09-30T22:22:24.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.608+0000 7fbb5cb47700 1 -- 172.21.15.89:0/3719240150 wait complete. 2023-09-30T22:22:24.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:24.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.665+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc948074ea0 con 0x7fc94806ee80 2023-09-30T22:22:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.665+0000 7fc944ff9700 1 -- 172.21.15.89:0/2783994583 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc93c017020 con 0x7fc94806ee80 2023-09-30T22:22:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.667+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806ee80 msgr2=0x7fc948134920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.667+0000 7fc94e9f3700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806ee80 0x7fc948134920 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00c4d0 tx=0x7fc93c00c7e0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.667+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 shutdown_connections 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.667+0000 7fc94e9f3700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc948134e60 0x7fc9481389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.668+0000 7fc94e9f3700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc94806e570 0x7fc94813bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.668+0000 7fc94e9f3700 1 --2- 172.21.15.89:0/2783994583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc94806ee80 0x7fc948134920 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.668+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 >> 172.21.15.89:0/2783994583 conn(0x7fc9480a2aa0 msgr2=0x7fc9480a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.668+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 shutdown_connections 2023-09-30T22:22:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:24.668+0000 7fc94e9f3700 1 -- 172.21.15.89:0/2783994583 wait complete. 2023-09-30T22:22:24.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:25.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:25.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:25.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:25.680 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.692+0000 7f683b070700 1 Processor -- start 2023-09-30T22:22:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.692+0000 7f683b070700 1 -- start start 2023-09-30T22:22:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.692+0000 7f683b070700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f68341337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f683b070700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834133e30 0x7f683412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f683b070700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6834134260 0x7f68341304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f683b070700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683406b990 con 0x7f6834134260 2023-09-30T22:22:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f683b070700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68340a0590 con 0x7f6834133e30 2023-09-30T22:22:25.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f683b070700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f683409fb80 con 0x7f6834133420 2023-09-30T22:22:25.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6838e0c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f68341337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6833fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834133e30 0x7f683412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6838e0c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f68341337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60240/0 (socket says 172.21.15.89:60240) 2023-09-30T22:22:25.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6838e0c700 1 -- 172.21.15.89:0/2522950589 learned_addr learned my addr 172.21.15.89:0/2522950589 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:25.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6833fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834133e30 0x7f683412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34224/0 (socket says 172.21.15.89:34224) 2023-09-30T22:22:25.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f683960d700 1 --2- 172.21.15.89:0/2522950589 >> v2:172.21.15.89:3300/0 conn(0x7f6834134260 0x7f68341304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6833fff700 1 -- 172.21.15.89:0/2522950589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 msgr2=0x7f68341337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6833fff700 1 --2- 172.21.15.89:0/2522950589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f68341337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6833fff700 1 -- 172.21.15.89:0/2522950589 >> v2:172.21.15.89:3300/0 conn(0x7f6834134260 msgr2=0x7f68341304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6833fff700 1 --2- 172.21.15.89:0/2522950589 >> v2:172.21.15.89:3300/0 conn(0x7f6834134260 0x7f68341304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6833fff700 1 -- 172.21.15.89:0/2522950589 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68340a0030 con 0x7f6834133e30 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f6833fff700 1 --2- 172.21.15.89:0/2522950589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834133e30 0x7f683412ff90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6820009840 tx=0x7f6820009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79e00085db067dfe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f68337fe700 1 -- 172.21.15.89:0/2522950589 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820011830 con 0x7f6834133e30 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f68337fe700 1 -- 172.21.15.89:0/2522950589 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6820011990 con 0x7f6834133e30 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.693+0000 7f68337fe700 1 -- 172.21.15.89:0/2522950589 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820011d20 con 0x7f6834133e30 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 -- 172.21.15.89:0/2522950589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834133e30 msgr2=0x7f683412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 --2- 172.21.15.89:0/2522950589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834133e30 0x7f683412ff90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6820009840 tx=0x7f6820009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 -- 172.21.15.89:0/2522950589 shutdown_connections 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 --2- 172.21.15.89:0/2522950589 >> v2:172.21.15.89:3300/0 conn(0x7f6834134260 0x7f68341304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 --2- 172.21.15.89:0/2522950589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f68341337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 --2- 172.21.15.89:0/2522950589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834133e30 0x7f683412ff90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 -- 172.21.15.89:0/2522950589 >> 172.21.15.89:0/2522950589 conn(0x7f68340a4210 msgr2=0x7f68340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 -- 172.21.15.89:0/2522950589 shutdown_connections 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.694+0000 7f683b070700 1 -- 172.21.15.89:0/2522950589 wait complete. 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.695+0000 7f683b070700 1 Processor -- start 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.695+0000 7f683b070700 1 -- start start 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f683b070700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f683409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f683b070700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6834133e30 0x7f683409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f683b070700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834134260 0x7f683409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f683b070700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683411f400 con 0x7f6834133e30 2023-09-30T22:22:25.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f683b070700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683411ef60 con 0x7f6834134260 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f683b070700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f683411ff70 con 0x7f6834133420 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f6838e0c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f683409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f6833fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6834133e30 0x7f683409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f6838e0c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f683409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60246/0 (socket says 172.21.15.89:60246) 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f6838e0c700 1 -- 172.21.15.89:0/1783771783 learned_addr learned my addr 172.21.15.89:0/1783771783 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f683960d700 1 --2- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834134260 0x7f683409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f6838e0c700 1 -- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834134260 msgr2=0x7f683409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.696+0000 7f6838e0c700 1 --2- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834134260 0x7f683409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f6838e0c700 1 -- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6834133e30 msgr2=0x7f683409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f6838e0c700 1 --2- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6834133e30 0x7f683409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f6838e0c700 1 -- 172.21.15.89:0/1783771783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68200094a0 con 0x7f6834133420 2023-09-30T22:22:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f6838e0c700 1 --2- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f683409c480 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6828009330 tx=0x7f682800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f68317fa700 1 -- 172.21.15.89:0/1783771783 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280096a0 con 0x7f6834133420 2023-09-30T22:22:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f68317fa700 1 -- 172.21.15.89:0/1783771783 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800ee60 con 0x7f6834133420 2023-09-30T22:22:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f68317fa700 1 -- 172.21.15.89:0/1783771783 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800ca70 con 0x7f6834133420 2023-09-30T22:22:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683409d8e0 con 0x7f6834133420 2023-09-30T22:22:25.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.697+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6834131490 con 0x7f6834133420 2023-09-30T22:22:25.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.698+0000 7f68317fa700 1 -- 172.21.15.89:0/1783771783 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f682800cbd0 con 0x7f6834133420 2023-09-30T22:22:25.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.698+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6834074ea0 con 0x7f6834133420 2023-09-30T22:22:25.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.698+0000 7f68317fa700 1 -- 172.21.15.89:0/1783771783 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f68280506f0 con 0x7f6834133420 2023-09-30T22:22:25.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.703+0000 7f68317fa700 1 -- 172.21.15.89:0/1783771783 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68280342f0 con 0x7f6834133420 2023-09-30T22:22:25.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.757+0000 7f98c8bde700 1 Processor -- start 2023-09-30T22:22:25.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.757+0000 7f98c8bde700 1 -- start start 2023-09-30T22:22:25.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c8bde700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f98c409f410 0x7f98c4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c8bde700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c4136fe0 0x7f98c41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c8bde700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c4070940 con 0x7f98c409e930 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c8bde700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c406fd60 con 0x7f98c409f410 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c8bde700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98c406fa80 con 0x7f98c4136fe0 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c2d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c4136fe0 0x7f98c41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f98c409e930 0x7f98c409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47510/0 (socket says 172.21.15.89:47510) 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c259c700 1 -- 172.21.15.89:0/441541076 learned_addr learned my addr 172.21.15.89:0/441541076 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.758+0000 7f98c1d9b700 1 --2- 172.21.15.89:0/441541076 >> v2:172.21.15.89:3300/0 conn(0x7f98c409f410 0x7f98c4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.759+0000 7f98c2d9d700 1 -- 172.21.15.89:0/441541076 >> v2:172.21.15.89:3300/0 conn(0x7f98c409f410 msgr2=0x7f98c4136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.759+0000 7f98c2d9d700 1 --2- 172.21.15.89:0/441541076 >> v2:172.21.15.89:3300/0 conn(0x7f98c409f410 0x7f98c4136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.759+0000 7f98c159a700 1 -- 172.21.15.89:0/441541076 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703084191 0 0) 0x7f98c4070940 con 0x7f98c409e930 2023-09-30T22:22:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.759+0000 7f98c2d9d700 1 -- 172.21.15.89:0/441541076 >> v1:172.21.15.89:6789/0 conn(0x7f98c409e930 legacy=0x7f98c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.759+0000 7f98c2d9d700 1 -- 172.21.15.89:0/441541076 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c409fb60 con 0x7f98c4136fe0 2023-09-30T22:22:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.759+0000 7f98c2d9d700 1 --2- 172.21.15.89:0/441541076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c4136fe0 0x7f98c41393c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009d90 tx=0x7f98bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11ebd980de7d8bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.760+0000 7f98c159a700 1 -- 172.21.15.89:0/441541076 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc00e070 con 0x7f98c4136fe0 2023-09-30T22:22:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.760+0000 7f98c159a700 1 -- 172.21.15.89:0/441541076 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc00bb20 con 0x7f98c4136fe0 2023-09-30T22:22:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.760+0000 7f98c159a700 1 -- 172.21.15.89:0/441541076 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc012430 con 0x7f98c4136fe0 2023-09-30T22:22:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.761+0000 7f98c8bde700 1 -- 172.21.15.89:0/441541076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c4136fe0 msgr2=0x7f98c41393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.761+0000 7f98c8bde700 1 --2- 172.21.15.89:0/441541076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c4136fe0 0x7f98c41393c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009d90 tx=0x7f98bc00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.761+0000 7f98c8bde700 1 -- 172.21.15.89:0/441541076 shutdown_connections 2023-09-30T22:22:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.761+0000 7f98c8bde700 1 --2- 172.21.15.89:0/441541076 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c4136fe0 0x7f98c41393c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.761+0000 7f98c8bde700 1 --2- 172.21.15.89:0/441541076 >> v2:172.21.15.89:3300/0 conn(0x7f98c409f410 0x7f98c4136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.761+0000 7f98c8bde700 1 -- 172.21.15.89:0/441541076 >> 172.21.15.89:0/441541076 conn(0x7f98c40a4210 msgr2=0x7f98c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.761+0000 7f98c8bde700 1 -- 172.21.15.89:0/441541076 shutdown_connections 2023-09-30T22:22:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.761+0000 7f98c8bde700 1 -- 172.21.15.89:0/441541076 wait complete. 2023-09-30T22:22:25.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.762+0000 7f98c8bde700 1 Processor -- start 2023-09-30T22:22:25.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.762+0000 7f98c8bde700 1 -- start start 2023-09-30T22:22:25.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c8bde700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c409e930 0x7f98c411f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c8bde700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c409f410 0x7f98c411fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c8bde700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98c4136fe0 0x7f98c4120100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c8bde700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4071d20 con 0x7f98c409f410 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c8bde700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4121a50 con 0x7f98c4136fe0 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c8bde700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98c41354d0 con 0x7f98c409e930 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c409f410 0x7f98c411fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98c4136fe0 0x7f98c4120100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c409f410 0x7f98c411fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40878/0 (socket says 172.21.15.89:40878) 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98c4136fe0 0x7f98c4120100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34244/0 (socket says 172.21.15.89:34244) 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c1d9b700 1 -- 172.21.15.89:0/2657975586 learned_addr learned my addr 172.21.15.89:0/2657975586 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:25.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.763+0000 7f98c259c700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c409e930 0x7f98c411f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c1d9b700 1 -- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c409e930 msgr2=0x7f98c411f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c1d9b700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c409e930 0x7f98c411f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c1d9b700 1 -- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98c4136fe0 msgr2=0x7f98c4120100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c1d9b700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98c4136fe0 0x7f98c4120100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c1d9b700 1 -- 172.21.15.89:0/2657975586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc009a70 con 0x7f98c409f410 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c2d9d700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98c4136fe0 0x7f98c4120100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c1d9b700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c409f410 0x7f98c411fbc0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f98b800e9b0 tx=0x7f98b800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98b2ffd700 1 -- 172.21.15.89:0/2657975586 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8009710 con 0x7f98c409f410 2023-09-30T22:22:25.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98b2ffd700 1 -- 172.21.15.89:0/2657975586 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b800ee50 con 0x7f98c409f410 2023-09-30T22:22:25.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c4121290 con 0x7f98c409f410 2023-09-30T22:22:25.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98b2ffd700 1 -- 172.21.15.89:0/2657975586 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800ca50 con 0x7f98c409f410 2023-09-30T22:22:25.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.764+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c41227f0 con 0x7f98c409f410 2023-09-30T22:22:25.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.766+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c4074ea0 con 0x7f98c409f410 2023-09-30T22:22:25.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.766+0000 7f98b2ffd700 1 -- 172.21.15.89:0/2657975586 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98b800cbb0 con 0x7f98c409f410 2023-09-30T22:22:25.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.766+0000 7f98b2ffd700 1 -- 172.21.15.89:0/2657975586 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f98b803ff40 con 0x7f98c409f410 2023-09-30T22:22:25.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.775+0000 7f98b2ffd700 1 -- 172.21.15.89:0/2657975586 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98b801a360 con 0x7f98c409f410 2023-09-30T22:22:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.803+0000 7f7c593e4700 1 Processor -- start 2023-09-30T22:22:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.803+0000 7f7c593e4700 1 -- start start 2023-09-30T22:22:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.803+0000 7f7c593e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130560 0x7f7c54130930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.803+0000 7f7c593e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.803+0000 7f7c593e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7c54130e70 0x7f7c5406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.803+0000 7f7c593e4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c54070a50 con 0x7f7c54130e70 2023-09-30T22:22:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.803+0000 7f7c593e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540717d0 con 0x7f7c54130560 2023-09-30T22:22:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.803+0000 7f7c593e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c54130340 con 0x7f7c5409eca0 2023-09-30T22:22:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c52ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130560 0x7f7c54130930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c52ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130560 0x7f7c54130930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34250/0 (socket says 172.21.15.89:34250) 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c52ffd700 1 -- 172.21.15.89:0/3106932222 learned_addr learned my addr 172.21.15.89:0/3106932222 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c527fc700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c537fe700 1 --2- 172.21.15.89:0/3106932222 >> v2:172.21.15.89:3300/0 conn(0x7f7c54130e70 0x7f7c5406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c537fe700 1 -- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 msgr2=0x7f7c5406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c537fe700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c537fe700 1 -- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130560 msgr2=0x7f7c54130930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c537fe700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130560 0x7f7c54130930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c537fe700 1 -- 172.21.15.89:0/3106932222 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c54071060 con 0x7f7c54130e70 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c527fc700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:25.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c52ffd700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130560 0x7f7c54130930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:25.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c537fe700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130e70 0x7f7c5406fb60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009d90 tx=0x7f7c4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3672eb2a0b8f334a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:25.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c51ffb700 1 -- 172.21.15.89:0/3106932222 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00e070 con 0x7f7c54130e70 2023-09-30T22:22:25.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c51ffb700 1 -- 172.21.15.89:0/3106932222 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c00bb20 con 0x7f7c54130e70 2023-09-30T22:22:25.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.804+0000 7f7c51ffb700 1 -- 172.21.15.89:0/3106932222 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c012430 con 0x7f7c54130e70 2023-09-30T22:22:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.805+0000 7f7c593e4700 1 -- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130e70 msgr2=0x7f7c5406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.805+0000 7f7c593e4700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130e70 0x7f7c5406fb60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009d90 tx=0x7f7c4c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.805+0000 7f7c593e4700 1 -- 172.21.15.89:0/3106932222 shutdown_connections 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.805+0000 7f7c593e4700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130e70 0x7f7c5406fb60 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.805+0000 7f7c593e4700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130560 0x7f7c54130930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.805+0000 7f7c593e4700 1 --2- 172.21.15.89:0/3106932222 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.805+0000 7f7c593e4700 1 -- 172.21.15.89:0/3106932222 >> 172.21.15.89:0/3106932222 conn(0x7f7c540a2aa0 msgr2=0x7f7c540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.805+0000 7f7c593e4700 1 -- 172.21.15.89:0/3106932222 shutdown_connections 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.806+0000 7f7c593e4700 1 -- 172.21.15.89:0/3106932222 wait complete. 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.807+0000 7f7c593e4700 1 Processor -- start 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.807+0000 7f7c593e4700 1 -- start start 2023-09-30T22:22:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c593e4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5411d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c593e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130560 0x7f7c5411d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c593e4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130e70 0x7f7c5411db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c593e4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5412e7b0 con 0x7f7c54130560 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c593e4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5412b9c0 con 0x7f7c54130e70 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c593e4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5411e740 con 0x7f7c5409eca0 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5411d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5411d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60286/0 (socket says 172.21.15.89:60286) 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 -- 172.21.15.89:0/281813982 learned_addr learned my addr 172.21.15.89:0/281813982 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c527fc700 1 --2- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130560 0x7f7c5411d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 -- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130e70 msgr2=0x7f7c5411db60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:22:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 --2- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130e70 0x7f7c5411db60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 -- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130560 msgr2=0x7f7c5411d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 --2- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130560 0x7f7c5411d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 -- 172.21.15.89:0/281813982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c480094a0 con 0x7f7c5409eca0 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.808+0000 7f7c52ffd700 1 --2- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5411d0e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400ebe0 tx=0x7f7c4400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.809+0000 7f7c3b7fe700 1 -- 172.21.15.89:0/281813982 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400c850 con 0x7f7c5409eca0 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.809+0000 7f7c3b7fe700 1 -- 172.21.15.89:0/281813982 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4400c9b0 con 0x7f7c5409eca0 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.809+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c44009550 con 0x7f7c5409eca0 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.809+0000 7f7c3b7fe700 1 -- 172.21.15.89:0/281813982 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400cd20 con 0x7f7c5409eca0 2023-09-30T22:22:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.809+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c009a70 con 0x7f7c5409eca0 2023-09-30T22:22:25.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.810+0000 7f7c3b7fe700 1 -- 172.21.15.89:0/281813982 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c44013470 con 0x7f7c5409eca0 2023-09-30T22:22:25.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.810+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5406d4a0 con 0x7f7c5409eca0 2023-09-30T22:22:25.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.810+0000 7f7c3b7fe700 1 -- 172.21.15.89:0/281813982 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c4403f7a0 con 0x7f7c5409eca0 2023-09-30T22:22:25.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.815+0000 7f7c3b7fe700 1 -- 172.21.15.89:0/281813982 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c440232f0 con 0x7f7c5409eca0 2023-09-30T22:22:25.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.841+0000 7fb1a8b51700 1 Processor -- start 2023-09-30T22:22:25.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a8b51700 1 -- start start 2023-09-30T22:22:25.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a8b51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a8b51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a8b51700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40a2a70 con 0x7fb1a409d630 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a8b51700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40a01d0 con 0x7fb1a4138d80 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a8b51700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a409e2b0 con 0x7fb1a40a0a30 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34252/0 (socket says 172.21.15.89:34252) 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.842+0000 7fb1a2d9d700 1 -- 172.21.15.89:0/1571184977 learned_addr learned my addr 172.21.15.89:0/1571184977 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a2d9d700 1 -- 172.21.15.89:0/1571184977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 msgr2=0x7fb1a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a2d9d700 1 --2- 172.21.15.89:0/1571184977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a2d9d700 1 -- 172.21.15.89:0/1571184977 >> v1:172.21.15.89:6789/0 conn(0x7fb1a409d630 legacy=0x7fb1a40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a159a700 1 -- 172.21.15.89:0/1571184977 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786271230 0 0) 0x7fb1a40a2a70 con 0x7fb1a409d630 2023-09-30T22:22:25.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a2d9d700 1 -- 172.21.15.89:0/1571184977 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a409dc50 con 0x7fb1a4138d80 2023-09-30T22:22:25.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a2d9d700 1 --2- 172.21.15.89:0/1571184977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a413b160 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009e80 tx=0x7fb19c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d931e08dce570253 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:25.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a159a700 1 -- 172.21.15.89:0/1571184977 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00e070 con 0x7fb1a4138d80 2023-09-30T22:22:25.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a159a700 1 -- 172.21.15.89:0/1571184977 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00b910 con 0x7fb1a4138d80 2023-09-30T22:22:25.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.843+0000 7fb1a159a700 1 -- 172.21.15.89:0/1571184977 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00bcc0 con 0x7fb1a4138d80 2023-09-30T22:22:25.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.844+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1571184977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 msgr2=0x7fb1a413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.844+0000 7fb1a8b51700 1 --2- 172.21.15.89:0/1571184977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a413b160 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009e80 tx=0x7fb19c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:25.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.844+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1571184977 shutdown_connections 2023-09-30T22:22:25.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.844+0000 7fb1a8b51700 1 --2- 172.21.15.89:0/1571184977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a413b160 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.844+0000 7fb1a8b51700 1 --2- 172.21.15.89:0/1571184977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.844+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1571184977 >> 172.21.15.89:0/1571184977 conn(0x7fb1a40a3050 msgr2=0x7fb1a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:25.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.844+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1571184977 shutdown_connections 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.844+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1571184977 wait complete. 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.845+0000 7fb1a8b51700 1 Processor -- start 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.845+0000 7fb1a8b51700 1 -- start start 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a8b51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a409d630 0x7fb1a4133530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a8b51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a4133a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a8b51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a4133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a8b51700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4136ce0 con 0x7fb1a409d630 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a8b51700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4135fc0 con 0x7fb1a4138d80 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a8b51700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4136370 con 0x7fb1a40a0a30 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a1d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a4133a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a1d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a4133a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60318/0 (socket says 172.21.15.89:60318) 2023-09-30T22:22:25.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a1d9b700 1 -- 172.21.15.89:0/1041193748 learned_addr learned my addr 172.21.15.89:0/1041193748 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a2d9d700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a4133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a259c700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a409d630 0x7fb1a4133530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a1d9b700 1 -- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 msgr2=0x7fb1a4133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a1d9b700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a4133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a1d9b700 1 -- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a409d630 msgr2=0x7fb1a4133530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a1d9b700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a409d630 0x7fb1a4133530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.846+0000 7fb1a1d9b700 1 -- 172.21.15.89:0/1041193748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c009b30 con 0x7fb1a40a0a30 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.847+0000 7fb1a1d9b700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a4133a70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb19800c520 tx=0x7fb19800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:25.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.847+0000 7fb192ffd700 1 -- 172.21.15.89:0/1041193748 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980054e0 con 0x7fb1a40a0a30 2023-09-30T22:22:25.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.847+0000 7fb192ffd700 1 -- 172.21.15.89:0/1041193748 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198003680 con 0x7fb1a40a0a30 2023-09-30T22:22:25.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.847+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a413abc0 con 0x7fb1a40a0a30 2023-09-30T22:22:25.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.847+0000 7fb192ffd700 1 -- 172.21.15.89:0/1041193748 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198003a30 con 0x7fb1a40a0a30 2023-09-30T22:22:25.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.847+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a4136fc0 con 0x7fb1a40a0a30 2023-09-30T22:22:25.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.848+0000 7fb192ffd700 1 -- 172.21.15.89:0/1041193748 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb198003b90 con 0x7fb1a40a0a30 2023-09-30T22:22:25.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.848+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a40747f0 con 0x7fb1a40a0a30 2023-09-30T22:22:25.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.848+0000 7fb192ffd700 1 -- 172.21.15.89:0/1041193748 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb19803eb90 con 0x7fb1a40a0a30 2023-09-30T22:22:25.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.853+0000 7fb192ffd700 1 -- 172.21.15.89:0/1041193748 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb198003e40 con 0x7fb1a40a0a30 2023-09-30T22:22:25.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.903+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f683406b250 con 0x7f6834133420 2023-09-30T22:22:25.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.904+0000 7f68317fa700 1 -- 172.21.15.89:0/1783771783 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6828032070 con 0x7f6834133420 2023-09-30T22:22:25.904 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:25.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.906+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 msgr2=0x7f683409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.906+0000 7f683b070700 1 --2- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f683409c480 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6828009330 tx=0x7f682800ec20 comp rx=0 tx=0).stop 2023-09-30T22:22:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.907+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 shutdown_connections 2023-09-30T22:22:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.907+0000 7f683b070700 1 --2- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6834134260 0x7f683409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.907+0000 7f683b070700 1 --2- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6834133420 0x7f683409c480 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.907+0000 7f683b070700 1 --2- 172.21.15.89:0/1783771783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6834133e30 0x7f683409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.907+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 >> 172.21.15.89:0/1783771783 conn(0x7f68340a4210 msgr2=0x7f68340a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.907+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 shutdown_connections 2023-09-30T22:22:25.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.907+0000 7f683b070700 1 -- 172.21.15.89:0/1783771783 wait complete. 2023-09-30T22:22:25.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:25.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.987+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f98c4135710 con 0x7f98c409f410 2023-09-30T22:22:25.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.987+0000 7f98b2ffd700 1 -- 172.21.15.89:0/2657975586 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98b8017070 con 0x7f98c409f410 2023-09-30T22:22:25.987 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:25.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c409f410 msgr2=0x7f98c411fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:25.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c409f410 0x7f98c411fbc0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f98b800e9b0 tx=0x7f98b800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 shutdown_connections 2023-09-30T22:22:25.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98c4136fe0 0x7f98c4120100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f98c409e930 0x7f98c411f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 --2- 172.21.15.89:0/2657975586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c409f410 0x7f98c411fbc0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:25.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 >> 172.21.15.89:0/2657975586 conn(0x7f98c40a4210 msgr2=0x7f98c4135b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:25.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 shutdown_connections 2023-09-30T22:22:25.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:25.990+0000 7f98c8bde700 1 -- 172.21.15.89:0/2657975586 wait complete. 2023-09-30T22:22:26.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:26.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.039+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7c54074ea0 con 0x7f7c5409eca0 2023-09-30T22:22:26.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.039+0000 7f7c3b7fe700 1 -- 172.21.15.89:0/281813982 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c44021070 con 0x7f7c5409eca0 2023-09-30T22:22:26.040 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:26.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 msgr2=0x7f7c5411d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:26.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 --2- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5411d0e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400ebe0 tx=0x7f7c4400eef0 comp rx=0 tx=0).stop 2023-09-30T22:22:26.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 shutdown_connections 2023-09-30T22:22:26.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 --2- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7c54130e70 0x7f7c5411db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:26.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 --2- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c5409eca0 0x7f7c5411d0e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:26.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 --2- 172.21.15.89:0/281813982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c54130560 0x7f7c5411d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:26.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 >> 172.21.15.89:0/281813982 conn(0x7f7c540a2aa0 msgr2=0x7f7c5412d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:26.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 shutdown_connections 2023-09-30T22:22:26.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.042+0000 7f7c593e4700 1 -- 172.21.15.89:0/281813982 wait complete. 2023-09-30T22:22:26.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.075+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb1a4072980 con 0x7fb1a40a0a30 2023-09-30T22:22:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.076+0000 7fb192ffd700 1 -- 172.21.15.89:0/1041193748 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb198017020 con 0x7fb1a40a0a30 2023-09-30T22:22:26.076 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 msgr2=0x7fb1a4133a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a4133a70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb19800c520 tx=0x7fb19800c830 comp rx=0 tx=0).stop 2023-09-30T22:22:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 shutdown_connections 2023-09-30T22:22:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1a4138d80 0x7fb1a4133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a409d630 0x7fb1a4133530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 --2- 172.21.15.89:0/1041193748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a0a30 0x7fb1a4133a70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:26.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 >> 172.21.15.89:0/1041193748 conn(0x7fb1a40a3050 msgr2=0x7fb1a409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:26.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 shutdown_connections 2023-09-30T22:22:26.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:26.078+0000 7fb1a8b51700 1 -- 172.21.15.89:0/1041193748 wait complete. 2023-09-30T22:22:26.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:26.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:27.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:27.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.053+0000 7f04d0112700 1 Processor -- start 2023-09-30T22:22:27.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.053+0000 7f04d0112700 1 -- start start 2023-09-30T22:22:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.053+0000 7f04d0112700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a0a30 0x7f04c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.053+0000 7f04d0112700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a1340 0x7f04c8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.053+0000 7f04d0112700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04c8133fa0 0x7f04c8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.053+0000 7f04d0112700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c809edd0 con 0x7f04c8133fa0 2023-09-30T22:22:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.053+0000 7f04d0112700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c809e1f0 con 0x7f04c80a1340 2023-09-30T22:22:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.053+0000 7f04d0112700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04c809dc50 con 0x7f04c80a0a30 2023-09-30T22:22:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a0a30 0x7f04c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cd6ad700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a1340 0x7f04c8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04ce6af700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04c8133fa0 0x7f04c8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a0a30 0x7f04c80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60328/0 (socket says 172.21.15.89:60328) 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 -- 172.21.15.89:0/2404806990 learned_addr learned my addr 172.21.15.89:0/2404806990 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 -- 172.21.15.89:0/2404806990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a1340 msgr2=0x7f04c8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 --2- 172.21.15.89:0/2404806990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a1340 0x7f04c8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 -- 172.21.15.89:0/2404806990 >> v2:172.21.15.89:3300/0 conn(0x7f04c8133fa0 msgr2=0x7f04c8134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 --2- 172.21.15.89:0/2404806990 >> v2:172.21.15.89:3300/0 conn(0x7f04c8133fa0 0x7f04c8134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 -- 172.21.15.89:0/2404806990 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c80a1fe0 con 0x7f04c80a0a30 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cd6ad700 1 --2- 172.21.15.89:0/2404806990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a1340 0x7f04c8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cdeae700 1 --2- 172.21.15.89:0/2404806990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a0a30 0x7f04c80a0e00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f04c40098f0 tx=0x7f04c4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46db746cc841c62 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cceac700 1 -- 172.21.15.89:0/2404806990 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4011830 con 0x7f04c80a0a30 2023-09-30T22:22:27.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cceac700 1 -- 172.21.15.89:0/2404806990 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c4011990 con 0x7f04c80a0a30 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.054+0000 7f04cceac700 1 -- 172.21.15.89:0/2404806990 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4011d20 con 0x7f04c80a0a30 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 -- 172.21.15.89:0/2404806990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a0a30 msgr2=0x7f04c80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 --2- 172.21.15.89:0/2404806990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a0a30 0x7f04c80a0e00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f04c40098f0 tx=0x7f04c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 -- 172.21.15.89:0/2404806990 shutdown_connections 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 --2- 172.21.15.89:0/2404806990 >> v2:172.21.15.89:3300/0 conn(0x7f04c8133fa0 0x7f04c8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 --2- 172.21.15.89:0/2404806990 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a0a30 0x7f04c80a0e00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 --2- 172.21.15.89:0/2404806990 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a1340 0x7f04c8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 -- 172.21.15.89:0/2404806990 >> 172.21.15.89:0/2404806990 conn(0x7f04c80a6550 msgr2=0x7f04c80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 -- 172.21.15.89:0/2404806990 shutdown_connections 2023-09-30T22:22:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.055+0000 7f04d0112700 1 -- 172.21.15.89:0/2404806990 wait complete. 2023-09-30T22:22:27.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:27.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04d0112700 1 Processor -- start 2023-09-30T22:22:27.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04d0112700 1 -- start start 2023-09-30T22:22:27.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04d0112700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a0a30 0x7f04c81c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04d0112700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a1340 0x7f04c81c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04d0112700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c8133fa0 0x7f04c81c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04d0112700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c811d7b0 con 0x7f04c8133fa0 2023-09-30T22:22:27.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04d0112700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81c18a0 con 0x7f04c80a0a30 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04d0112700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04c8070e60 con 0x7f04c80a1340 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04cd6ad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a1340 0x7f04c81c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04ce6af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c8133fa0 0x7f04c81c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04cd6ad700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a1340 0x7f04c81c1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60330/0 (socket says 172.21.15.89:60330) 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04cdeae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a0a30 0x7f04c81c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.056+0000 7f04cd6ad700 1 -- 172.21.15.89:0/4261911956 learned_addr learned my addr 172.21.15.89:0/4261911956 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04cd6ad700 1 -- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a0a30 msgr2=0x7f04c81c0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04cd6ad700 1 --2- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a0a30 0x7f04c81c0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04cd6ad700 1 -- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c8133fa0 msgr2=0x7f04c81c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04cd6ad700 1 --2- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c8133fa0 0x7f04c81c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04cd6ad700 1 -- 172.21.15.89:0/4261911956 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80094a0 con 0x7f04c80a1340 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04ce6af700 1 --2- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c8133fa0 0x7f04c81c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04cd6ad700 1 --2- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a1340 0x7f04c81c1360 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f04b800ec10 tx=0x7f04b800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04be7fc700 1 -- 172.21.15.89:0/4261911956 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b800c850 con 0x7f04c80a1340 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04be7fc700 1 -- 172.21.15.89:0/4261911956 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b800c9b0 con 0x7f04c80a1340 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c4009550 con 0x7f04c80a1340 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04be7fc700 1 -- 172.21.15.89:0/4261911956 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b800cd20 con 0x7f04c80a1340 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.057+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c8071650 con 0x7f04c80a1340 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.058+0000 7f04be7fc700 1 -- 172.21.15.89:0/4261911956 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f04b8013470 con 0x7f04c80a1340 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.058+0000 7f04be7fc700 1 -- 172.21.15.89:0/4261911956 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f04b803fb60 con 0x7f04c80a1340 2023-09-30T22:22:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.058+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c8074ea0 con 0x7f04c80a1340 2023-09-30T22:22:27.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.063+0000 7f04be7fc700 1 -- 172.21.15.89:0/4261911956 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f04b8021080 con 0x7f04c80a1340 2023-09-30T22:22:27.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:27.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.137+0000 7f664fecf700 1 Processor -- start 2023-09-30T22:22:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.137+0000 7f664fecf700 1 -- start start 2023-09-30T22:22:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.137+0000 7f664fecf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f664806e570 0x7f664806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.137+0000 7f664fecf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664809eca0 0x7f664806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.137+0000 7f664fecf700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6648070ee0 con 0x7f6648134610 2023-09-30T22:22:27.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.137+0000 7f664fecf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6648071480 con 0x7f664806e570 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.137+0000 7f664fecf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664806f3c0 con 0x7f664809eca0 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664dc6b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f664806e570 0x7f664806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664dc6b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f664806e570 0x7f664806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40918/0 (socket says 172.21.15.89:40918) 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664dc6b700 1 -- 172.21.15.89:0/3286897615 learned_addr learned my addr 172.21.15.89:0/3286897615 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664d46a700 1 --2- 172.21.15.89:0/3286897615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664809eca0 0x7f664806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664dc6b700 1 -- 172.21.15.89:0/3286897615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664809eca0 msgr2=0x7f664806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664dc6b700 1 --2- 172.21.15.89:0/3286897615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664809eca0 0x7f664806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664dc6b700 1 -- 172.21.15.89:0/3286897615 >> v1:172.21.15.89:6789/0 conn(0x7f6648134610 legacy=0x7f6648134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664dc6b700 1 -- 172.21.15.89:0/3286897615 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6648071b90 con 0x7f664806e570 2023-09-30T22:22:27.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664dc6b700 1 --2- 172.21.15.89:0/3286897615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f664806e940 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f66440098f0 tx=0x7f6644009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da301924aec8a3fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:27.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664cc69700 1 -- 172.21.15.89:0/3286897615 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644011830 con 0x7f664806e570 2023-09-30T22:22:27.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664cc69700 1 -- 172.21.15.89:0/3286897615 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644011990 con 0x7f664806e570 2023-09-30T22:22:27.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.138+0000 7f664cc69700 1 -- 172.21.15.89:0/3286897615 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644011d20 con 0x7f664806e570 2023-09-30T22:22:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.139+0000 7f664fecf700 1 -- 172.21.15.89:0/3286897615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 msgr2=0x7f664806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.139+0000 7f664fecf700 1 --2- 172.21.15.89:0/3286897615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f664806e940 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f66440098f0 tx=0x7f6644009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.139+0000 7f664fecf700 1 -- 172.21.15.89:0/3286897615 shutdown_connections 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.139+0000 7f664fecf700 1 --2- 172.21.15.89:0/3286897615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f664806e940 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.139+0000 7f664fecf700 1 --2- 172.21.15.89:0/3286897615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664809eca0 0x7f664806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.139+0000 7f664fecf700 1 -- 172.21.15.89:0/3286897615 >> 172.21.15.89:0/3286897615 conn(0x7f66480a2aa0 msgr2=0x7f66480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.139+0000 7f664fecf700 1 -- 172.21.15.89:0/3286897615 shutdown_connections 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.140+0000 7f664fecf700 1 -- 172.21.15.89:0/3286897615 wait complete. 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.140+0000 7f664fecf700 1 Processor -- start 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.140+0000 7f664fecf700 1 -- start start 2023-09-30T22:22:27.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664fecf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f66481340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664fecf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6648134610 0x7f6648132e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664fecf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66481333b0 0x7f6648133820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664fecf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664813d820 con 0x7f664806e570 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664fecf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664813bd40 con 0x7f66481333b0 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664fecf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66480a2130 con 0x7f6648134610 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664dc6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f66481340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664dc6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f66481340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40922/0 (socket says 172.21.15.89:40922) 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664dc6b700 1 -- 172.21.15.89:0/198648113 learned_addr learned my addr 172.21.15.89:0/198648113 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664e46c700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66481333b0 0x7f6648133820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664dc6b700 1 -- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6648134610 msgr2=0x7f6648132e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664dc6b700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6648134610 0x7f6648132e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664dc6b700 1 -- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66481333b0 msgr2=0x7f6648133820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664dc6b700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66481333b0 0x7f6648133820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664dc6b700 1 -- 172.21.15.89:0/198648113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644009550 con 0x7f664806e570 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.141+0000 7f664e46c700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66481333b0 0x7f6648133820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.142+0000 7f664dc6b700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f66481340b0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6644009c40 tx=0x7f6644004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.142+0000 7f663e7fc700 1 -- 172.21.15.89:0/198648113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644004530 con 0x7f664806e570 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.142+0000 7f663e7fc700 1 -- 172.21.15.89:0/198648113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664401f070 con 0x7f664806e570 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.142+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6648070610 con 0x7f664806e570 2023-09-30T22:22:27.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.142+0000 7f663e7fc700 1 -- 172.21.15.89:0/198648113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66440038c0 con 0x7f664806e570 2023-09-30T22:22:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.142+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664813cce0 con 0x7f664806e570 2023-09-30T22:22:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.143+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6648074ea0 con 0x7f664806e570 2023-09-30T22:22:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.143+0000 7f663e7fc700 1 -- 172.21.15.89:0/198648113 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6644021030 con 0x7f664806e570 2023-09-30T22:22:27.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.144+0000 7f663e7fc700 1 -- 172.21.15.89:0/198648113 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6644040f80 con 0x7f664806e570 2023-09-30T22:22:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.152+0000 7f663e7fc700 1 -- 172.21.15.89:0/198648113 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6644019980 con 0x7f664806e570 2023-09-30T22:22:27.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.202+0000 7f9ff0fca700 1 Processor -- start 2023-09-30T22:22:27.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.202+0000 7f9ff0fca700 1 -- start start 2023-09-30T22:22:27.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9ff0fca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9ff0fca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9fec09e930 0x7f9fec13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9ff0fca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec06ef30 con 0x7f9fec09de50 2023-09-30T22:22:27.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9ff0fca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec06fef0 con 0x7f9fec09e930 2023-09-30T22:22:27.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9ff0fca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1334b0 con 0x7f9fec1338a0 2023-09-30T22:22:27.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9fea59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9fead9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9fec09e930 0x7f9fec13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9fe9d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9fec09de50 0x7f9fec09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47534/0 (socket says 172.21.15.89:47534) 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9fea59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34286/0 (socket says 172.21.15.89:34286) 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9fe9d9b700 1 -- 172.21.15.89:0/1914364619 learned_addr learned my addr 172.21.15.89:0/1914364619 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.203+0000 7f9fead9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9fec09e930 0x7f9fec13b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40934/0 (socket says 172.21.15.89:40934) 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.204+0000 7f9fea59c700 1 -- 172.21.15.89:0/1914364619 >> v2:172.21.15.89:3300/0 conn(0x7f9fec09e930 msgr2=0x7f9fec13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.204+0000 7f9fea59c700 1 --2- 172.21.15.89:0/1914364619 >> v2:172.21.15.89:3300/0 conn(0x7f9fec09e930 0x7f9fec13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.204+0000 7f9fea59c700 1 -- 172.21.15.89:0/1914364619 >> v1:172.21.15.89:6789/0 conn(0x7f9fec09de50 legacy=0x7f9fec09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.204+0000 7f9fea59c700 1 -- 172.21.15.89:0/1914364619 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec06fc30 con 0x7f9fec1338a0 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.204+0000 7f9fea59c700 1 --2- 172.21.15.89:0/1914364619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec133c70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009840 tx=0x7f9fdc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ceaff87dcb23db1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.204+0000 7f9fe959a700 1 -- 172.21.15.89:0/1914364619 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011830 con 0x7f9fec1338a0 2023-09-30T22:22:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.204+0000 7f9fe959a700 1 -- 172.21.15.89:0/1914364619 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc011990 con 0x7f9fec1338a0 2023-09-30T22:22:27.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.204+0000 7f9fe959a700 1 -- 172.21.15.89:0/1914364619 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011d20 con 0x7f9fec1338a0 2023-09-30T22:22:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.205+0000 7f9ff0fca700 1 -- 172.21.15.89:0/1914364619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 msgr2=0x7f9fec133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.205+0000 7f9ff0fca700 1 --2- 172.21.15.89:0/1914364619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec133c70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009840 tx=0x7f9fdc009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.205+0000 7f9ff0fca700 1 -- 172.21.15.89:0/1914364619 shutdown_connections 2023-09-30T22:22:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.205+0000 7f9ff0fca700 1 --2- 172.21.15.89:0/1914364619 >> v2:172.21.15.89:3300/0 conn(0x7f9fec09e930 0x7f9fec13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.205+0000 7f9ff0fca700 1 --2- 172.21.15.89:0/1914364619 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec133c70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.205+0000 7f9ff0fca700 1 -- 172.21.15.89:0/1914364619 >> 172.21.15.89:0/1914364619 conn(0x7f9fec0a2aa0 msgr2=0x7f9fec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.205+0000 7f9ff0fca700 1 -- 172.21.15.89:0/1914364619 shutdown_connections 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.205+0000 7f9ff0fca700 1 -- 172.21.15.89:0/1914364619 wait complete. 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9ff0fca700 1 Processor -- start 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9ff0fca700 1 -- start start 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9ff0fca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fec09de50 0x7f9fec11d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9ff0fca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fec09e930 0x7f9fec11dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9ff0fca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec11e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9ff0fca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1212a0 con 0x7f9fec09e930 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9ff0fca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec11f9b0 con 0x7f9fec1338a0 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9ff0fca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec120a30 con 0x7f9fec09de50 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9fe9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fec09e930 0x7f9fec11dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.207+0000 7f9fe9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fec09e930 0x7f9fec11dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40946/0 (socket says 172.21.15.89:40946) 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fe9d9b700 1 -- 172.21.15.89:0/391301175 learned_addr learned my addr 172.21.15.89:0/391301175 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fea59c700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fec09de50 0x7f9fec11d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fead9d700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec11e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fe9d9b700 1 -- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fec09de50 msgr2=0x7f9fec11d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fe9d9b700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fec09de50 0x7f9fec11d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fe9d9b700 1 -- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 msgr2=0x7f9fec11e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fe9d9b700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec11e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fe9d9b700 1 -- 172.21.15.89:0/391301175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0094a0 con 0x7f9fec09e930 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fea59c700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fec09de50 0x7f9fec11d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fead9d700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec11e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fe9d9b700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fec09e930 0x7f9fec11dd50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000c450 tx=0x7f9fe000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fdaffd700 1 -- 172.21.15.89:0/391301175 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0005d70 con 0x7f9fec09e930 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fdaffd700 1 -- 172.21.15.89:0/391301175 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe00070d0 con 0x7f9fec09e930 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9fdaffd700 1 -- 172.21.15.89:0/391301175 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0007440 con 0x7f9fec09e930 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.208+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec06f640 con 0x7f9fec09e930 2023-09-30T22:22:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.209+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec0a0fc0 con 0x7f9fec09e930 2023-09-30T22:22:27.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.209+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fec120e20 con 0x7f9fec09e930 2023-09-30T22:22:27.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.210+0000 7f9fdaffd700 1 -- 172.21.15.89:0/391301175 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9fe00075a0 con 0x7f9fec09e930 2023-09-30T22:22:27.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.211+0000 7f9fdaffd700 1 -- 172.21.15.89:0/391301175 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9fe003edb0 con 0x7f9fec09e930 2023-09-30T22:22:27.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.214+0000 7f9fdaffd700 1 -- 172.21.15.89:0/391301175 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9fe001b3b0 con 0x7f9fec09e930 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.237+0000 7f9e35e72700 1 Processor -- start 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.237+0000 7f9e35e72700 1 -- start start 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e35e72700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009d6f0 0x7f9e3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e35e72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e410 0x7f9e30134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e35e72700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3009b120 con 0x7f9e3009e000 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e35e72700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3006b460 con 0x7f9e3009e410 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e35e72700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3009a7e0 con 0x7f9e3009d6f0 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e2effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9e3009e000 0x7f9e301339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47550/0 (socket says 172.21.15.89:47550) 2023-09-30T22:22:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e2effd700 1 -- 172.21.15.89:0/3773692682 learned_addr learned my addr 172.21.15.89:0/3773692682 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e2ffff700 1 --2- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e410 0x7f9e30134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e2f7fe700 1 --2- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009d6f0 0x7f9e3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e2e7fc700 1 -- 172.21.15.89:0/3773692682 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405836143 0 0) 0x7f9e3009b120 con 0x7f9e3009e000 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.238+0000 7f9e2e7fc700 1 -- 172.21.15.89:0/3773692682 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e180035b0 con 0x7f9e3009e000 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.239+0000 7f9e2ffff700 1 -- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009d6f0 msgr2=0x7f9e3009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.239+0000 7f9e2ffff700 1 --2- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009d6f0 0x7f9e3009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.239+0000 7f9e2ffff700 1 -- 172.21.15.89:0/3773692682 >> v1:172.21.15.89:6789/0 conn(0x7f9e3009e000 legacy=0x7f9e301339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.239+0000 7f9e2ffff700 1 -- 172.21.15.89:0/3773692682 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3009a500 con 0x7f9e3009e410 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.239+0000 7f9e2ffff700 1 --2- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e410 0x7f9e30134100 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009dc0 tx=0x7f9e2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2367376e0d33f769 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.239+0000 7f9e2e7fc700 1 -- 172.21.15.89:0/3773692682 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800e070 con 0x7f9e3009e410 2023-09-30T22:22:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.239+0000 7f9e2e7fc700 1 -- 172.21.15.89:0/3773692682 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2800bb20 con 0x7f9e3009e410 2023-09-30T22:22:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.239+0000 7f9e2e7fc700 1 -- 172.21.15.89:0/3773692682 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800be90 con 0x7f9e3009e410 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.240+0000 7f9e35e72700 1 -- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e410 msgr2=0x7f9e30134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.240+0000 7f9e35e72700 1 --2- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e410 0x7f9e30134100 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009dc0 tx=0x7f9e2800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.240+0000 7f9e35e72700 1 -- 172.21.15.89:0/3773692682 shutdown_connections 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.240+0000 7f9e35e72700 1 --2- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e410 0x7f9e30134100 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.240+0000 7f9e35e72700 1 --2- 172.21.15.89:0/3773692682 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009d6f0 0x7f9e3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.240+0000 7f9e35e72700 1 -- 172.21.15.89:0/3773692682 >> 172.21.15.89:0/3773692682 conn(0x7f9e30066f50 msgr2=0x7f9e3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.240+0000 7f9e35e72700 1 -- 172.21.15.89:0/3773692682 shutdown_connections 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.240+0000 7f9e35e72700 1 -- 172.21.15.89:0/3773692682 wait complete. 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e35e72700 1 Processor -- start 2023-09-30T22:22:27.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e35e72700 1 -- start start 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e35e72700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3009d6f0 0x7f9e301b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e35e72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e000 0x7f9e301b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e35e72700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009e410 0x7f9e301a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e35e72700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e301a6340 con 0x7f9e3009d6f0 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e35e72700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300a1d70 con 0x7f9e3009e000 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e35e72700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e301a8470 con 0x7f9e3009e410 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e2f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3009d6f0 0x7f9e301b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e2ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009e410 0x7f9e301a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e2effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e000 0x7f9e301b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e2f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3009d6f0 0x7f9e301b8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40950/0 (socket says 172.21.15.89:40950) 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.241+0000 7f9e2f7fe700 1 -- 172.21.15.89:0/3871497465 learned_addr learned my addr 172.21.15.89:0/3871497465 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e2f7fe700 1 -- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009e410 msgr2=0x7f9e301a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e2f7fe700 1 --2- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009e410 0x7f9e301a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e2f7fe700 1 -- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e000 msgr2=0x7f9e301b8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e2f7fe700 1 --2- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e000 0x7f9e301b8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e2f7fe700 1 -- 172.21.15.89:0/3871497465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e28009a70 con 0x7f9e3009d6f0 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e2effd700 1 --2- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e000 0x7f9e301b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e2f7fe700 1 --2- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3009d6f0 0x7f9e301b8530 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000b3f0 tx=0x7f9e2000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e17fff700 1 -- 172.21.15.89:0/3871497465 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20019070 con 0x7f9e3009d6f0 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e17fff700 1 -- 172.21.15.89:0/3871497465 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000be10 con 0x7f9e3009d6f0 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e301a7f70 con 0x7f9e3009d6f0 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e17fff700 1 -- 172.21.15.89:0/3871497465 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20014af0 con 0x7f9e3009d6f0 2023-09-30T22:22:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.242+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e301a6940 con 0x7f9e3009d6f0 2023-09-30T22:22:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.243+0000 7f9e17fff700 1 -- 172.21.15.89:0/3871497465 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e2001d610 con 0x7f9e3009d6f0 2023-09-30T22:22:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.243+0000 7f9e17fff700 1 -- 172.21.15.89:0/3871497465 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9e200472f0 con 0x7f9e3009d6f0 2023-09-30T22:22:27.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.244+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e30074ea0 con 0x7f9e3009d6f0 2023-09-30T22:22:27.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.249+0000 7f9e17fff700 1 -- 172.21.15.89:0/3871497465 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e200263b0 con 0x7f9e3009d6f0 2023-09-30T22:22:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.282+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f04c811da70 con 0x7f04c80a1340 2023-09-30T22:22:27.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.283+0000 7f04be7fc700 1 -- 172.21.15.89:0/4261911956 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04b800ce80 con 0x7f04c80a1340 2023-09-30T22:22:27.283 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a1340 msgr2=0x7f04c81c1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 --2- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a1340 0x7f04c81c1360 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f04b800ec10 tx=0x7f04b800ef20 comp rx=0 tx=0).stop 2023-09-30T22:22:27.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 shutdown_connections 2023-09-30T22:22:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 --2- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c8133fa0 0x7f04c81c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 --2- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f04c80a0a30 0x7f04c81c0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 --2- 172.21.15.89:0/4261911956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04c80a1340 0x7f04c81c1360 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 >> 172.21.15.89:0/4261911956 conn(0x7f04c80a6550 msgr2=0x7f04c8132fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 shutdown_connections 2023-09-30T22:22:27.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.285+0000 7f04d0112700 1 -- 172.21.15.89:0/4261911956 wait complete. 2023-09-30T22:22:27.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.366+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6648071100 con 0x7f664806e570 2023-09-30T22:22:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.366+0000 7f663e7fc700 1 -- 172.21.15.89:0/198648113 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6644015030 con 0x7f664806e570 2023-09-30T22:22:27.366 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 msgr2=0x7f66481340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f66481340b0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6644009c40 tx=0x7f6644004750 comp rx=0 tx=0).stop 2023-09-30T22:22:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 shutdown_connections 2023-09-30T22:22:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66481333b0 0x7f6648133820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664806e570 0x7f66481340b0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 --2- 172.21.15.89:0/198648113 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6648134610 0x7f6648132e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 >> 172.21.15.89:0/198648113 conn(0x7f66480a2aa0 msgr2=0x7f66480a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 shutdown_connections 2023-09-30T22:22:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.368+0000 7f664fecf700 1 -- 172.21.15.89:0/198648113 wait complete. 2023-09-30T22:22:27.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:27.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.427+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9fec120e20 con 0x7f9fec09e930 2023-09-30T22:22:27.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.428+0000 7f9fdaffd700 1 -- 172.21.15.89:0/391301175 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9fec120e20 con 0x7f9fec09e930 2023-09-30T22:22:27.428 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:27.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.429+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fec09e930 msgr2=0x7f9fec11dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.429+0000 7f9ff0fca700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fec09e930 0x7f9fec11dd50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000c450 tx=0x7f9fe000c760 comp rx=0 tx=0).stop 2023-09-30T22:22:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.429+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 shutdown_connections 2023-09-30T22:22:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.429+0000 7f9ff0fca700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9fec1338a0 0x7f9fec11e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.430+0000 7f9ff0fca700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9fec09de50 0x7f9fec11d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.430+0000 7f9ff0fca700 1 --2- 172.21.15.89:0/391301175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fec09e930 0x7f9fec11dd50 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.430+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 >> 172.21.15.89:0/391301175 conn(0x7f9fec0a2aa0 msgr2=0x7f9fec1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.430+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 shutdown_connections 2023-09-30T22:22:27.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.430+0000 7f9ff0fca700 1 -- 172.21.15.89:0/391301175 wait complete. 2023-09-30T22:22:27.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:27.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.459+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9e3006c360 con 0x7f9e3009d6f0 2023-09-30T22:22:27.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.459+0000 7f9e17fff700 1 -- 172.21.15.89:0/3871497465 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e20022020 con 0x7f9e3009d6f0 2023-09-30T22:22:27.459 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3009d6f0 msgr2=0x7f9e301b8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 --2- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3009d6f0 0x7f9e301b8530 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000b3f0 tx=0x7f9e2000b700 comp rx=0 tx=0).stop 2023-09-30T22:22:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 shutdown_connections 2023-09-30T22:22:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 --2- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e3009e410 0x7f9e301a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 --2- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e3009d6f0 0x7f9e301b8530 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 --2- 172.21.15.89:0/3871497465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e3009e000 0x7f9e301b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 >> 172.21.15.89:0/3871497465 conn(0x7f9e30066f50 msgr2=0x7f9e301b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 shutdown_connections 2023-09-30T22:22:27.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:27.461+0000 7f9e35e72700 1 -- 172.21.15.89:0/3871497465 wait complete. 2023-09-30T22:22:27.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:28.298 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:28.382 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:28.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.428+0000 7fb564c05700 1 Processor -- start 2023-09-30T22:22:28.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.428+0000 7fb564c05700 1 -- start start 2023-09-30T22:22:28.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb564c05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 0x7fb56009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb564c05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f240 0x7fb5601317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb564c05700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb560070940 con 0x7fb56009f650 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb564c05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56006fd60 con 0x7fb56009f240 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb564c05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb56006fa80 con 0x7fb56009e930 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55dd9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f240 0x7fb5601317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb56009f650 0x7fb560131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47552/0 (socket says 172.21.15.89:47552) 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 0x7fb56009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55ed9d700 1 -- 172.21.15.89:0/2975849815 learned_addr learned my addr 172.21.15.89:0/2975849815 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55dd9b700 1 -- 172.21.15.89:0/2975849815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 msgr2=0x7fb56009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55dd9b700 1 --2- 172.21.15.89:0/2975849815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 0x7fb56009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55dd9b700 1 -- 172.21.15.89:0/2975849815 >> v1:172.21.15.89:6789/0 conn(0x7fb56009f650 legacy=0x7fb560131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55dd9b700 1 -- 172.21.15.89:0/2975849815 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56009fb60 con 0x7fb56009f240 2023-09-30T22:22:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55d59a700 1 -- 172.21.15.89:0/2975849815 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060276216 0 0) 0x7fb560070940 con 0x7fb56009f650 2023-09-30T22:22:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.429+0000 7fb55dd9b700 1 --2- 172.21.15.89:0/2975849815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f240 0x7fb5601317f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb554009870 tx=0x7fb554009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee41e10e9573b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb55d59a700 1 -- 172.21.15.89:0/2975849815 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554011830 con 0x7fb56009f240 2023-09-30T22:22:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb55d59a700 1 -- 172.21.15.89:0/2975849815 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb554011990 con 0x7fb56009f240 2023-09-30T22:22:28.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb55d59a700 1 -- 172.21.15.89:0/2975849815 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554011d20 con 0x7fb56009f240 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb564c05700 1 -- 172.21.15.89:0/2975849815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f240 msgr2=0x7fb5601317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb564c05700 1 --2- 172.21.15.89:0/2975849815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f240 0x7fb5601317f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb554009870 tx=0x7fb554009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb564c05700 1 -- 172.21.15.89:0/2975849815 shutdown_connections 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb564c05700 1 --2- 172.21.15.89:0/2975849815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 0x7fb56009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb564c05700 1 --2- 172.21.15.89:0/2975849815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f240 0x7fb5601317f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb564c05700 1 -- 172.21.15.89:0/2975849815 >> 172.21.15.89:0/2975849815 conn(0x7fb5600a4210 msgr2=0x7fb5600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb564c05700 1 -- 172.21.15.89:0/2975849815 shutdown_connections 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.430+0000 7fb564c05700 1 -- 172.21.15.89:0/2975849815 wait complete. 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.431+0000 7fb564c05700 1 Processor -- start 2023-09-30T22:22:28.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.431+0000 7fb564c05700 1 -- start start 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb564c05700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 0x7fb5601a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb564c05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb56009f240 0x7fb5601a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb564c05700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f650 0x7fb5601a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb564c05700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601a5de0 con 0x7fb56009f240 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb564c05700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601a76a0 con 0x7fb56009f650 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb564c05700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601a6a70 con 0x7fb56009e930 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb56009f240 0x7fb5601a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb56009f240 0x7fb5601a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40962/0 (socket says 172.21.15.89:40962) 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55dd9b700 1 -- 172.21.15.89:0/3626306426 learned_addr learned my addr 172.21.15.89:0/3626306426 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55ed9d700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f650 0x7fb5601a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55e59c700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 0x7fb5601a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55dd9b700 1 -- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 msgr2=0x7fb5601a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55dd9b700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 0x7fb5601a3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55dd9b700 1 -- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f650 msgr2=0x7fb5601a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55dd9b700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f650 0x7fb5601a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.432+0000 7fb55dd9b700 1 -- 172.21.15.89:0/3626306426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540094a0 con 0x7fb56009f240 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.433+0000 7fb55dd9b700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb56009f240 0x7fb5601a4160 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fb554013000 tx=0x7fb554004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.433+0000 7fb54effd700 1 -- 172.21.15.89:0/3626306426 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554004510 con 0x7fb56009f240 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.433+0000 7fb54effd700 1 -- 172.21.15.89:0/3626306426 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55401f070 con 0x7fb56009f240 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.433+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5600a0190 con 0x7fb56009f240 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.433+0000 7fb54effd700 1 -- 172.21.15.89:0/3626306426 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540038c0 con 0x7fb56009f240 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.433+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5601a6e80 con 0x7fb56009f240 2023-09-30T22:22:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.433+0000 7fb54effd700 1 -- 172.21.15.89:0/3626306426 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb554003a20 con 0x7fb56009f240 2023-09-30T22:22:28.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.434+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5601a7220 con 0x7fb56009f240 2023-09-30T22:22:28.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.434+0000 7fb54effd700 1 -- 172.21.15.89:0/3626306426 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb554021070 con 0x7fb56009f240 2023-09-30T22:22:28.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.442+0000 7fb54effd700 1 -- 172.21.15.89:0/3626306426 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5601a7220 con 0x7fb56009f240 2023-09-30T22:22:28.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:28.474 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:28.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.515+0000 7fb62a5e7700 1 Processor -- start 2023-09-30T22:22:28.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.515+0000 7fb62a5e7700 1 -- start start 2023-09-30T22:22:28.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.516+0000 7fb62a5e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb624130670 0x7fb624130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.516+0000 7fb62a5e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406f680 0x7fb62406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.516+0000 7fb62a5e7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb624130260 con 0x7fb62406ffd0 2023-09-30T22:22:28.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.516+0000 7fb62a5e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62409a350 con 0x7fb62406f680 2023-09-30T22:22:28.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.516+0000 7fb62a5e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb624071ff0 con 0x7fb624130670 2023-09-30T22:22:28.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.516+0000 7fb628b84700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb62406ffd0 0x7fb624137530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47564/0 (socket says 172.21.15.89:47564) 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.516+0000 7fb628b84700 1 -- 172.21.15.89:0/267837630 learned_addr learned my addr 172.21.15.89:0/267837630 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.516+0000 7fb6237fe700 1 --2- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406f680 0x7fb62406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb623fff700 1 --2- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb624130670 0x7fb624130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb622ffd700 1 -- 172.21.15.89:0/267837630 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862393886 0 0) 0x7fb624130260 con 0x7fb62406ffd0 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb622ffd700 1 -- 172.21.15.89:0/267837630 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb60c0035b0 con 0x7fb62406ffd0 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb6237fe700 1 -- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb624130670 msgr2=0x7fb624130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb6237fe700 1 --2- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb624130670 0x7fb624130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb6237fe700 1 -- 172.21.15.89:0/267837630 >> v1:172.21.15.89:6789/0 conn(0x7fb62406ffd0 legacy=0x7fb624137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb6237fe700 1 -- 172.21.15.89:0/267837630 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624070470 con 0x7fb62406f680 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb623fff700 1 --2- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb624130670 0x7fb624130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:28.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb6237fe700 1 --2- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406f680 0x7fb62406fa90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb618009870 tx=0x7fb618009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6828cb8d7bf5f27d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb622ffd700 1 -- 172.21.15.89:0/267837630 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618011830 con 0x7fb62406f680 2023-09-30T22:22:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb622ffd700 1 -- 172.21.15.89:0/267837630 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618011990 con 0x7fb62406f680 2023-09-30T22:22:28.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.517+0000 7fb622ffd700 1 -- 172.21.15.89:0/267837630 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618011d20 con 0x7fb62406f680 2023-09-30T22:22:28.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.518+0000 7fb62a5e7700 1 -- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406f680 msgr2=0x7fb62406fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.518+0000 7fb62a5e7700 1 --2- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406f680 0x7fb62406fa90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb618009870 tx=0x7fb618009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.518+0000 7fb62a5e7700 1 -- 172.21.15.89:0/267837630 shutdown_connections 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.518+0000 7fb62a5e7700 1 --2- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb624130670 0x7fb624130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.518+0000 7fb62a5e7700 1 --2- 172.21.15.89:0/267837630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406f680 0x7fb62406fa90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.518+0000 7fb62a5e7700 1 -- 172.21.15.89:0/267837630 >> 172.21.15.89:0/267837630 conn(0x7fb6240a2aa0 msgr2=0x7fb6240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.518+0000 7fb62a5e7700 1 -- 172.21.15.89:0/267837630 shutdown_connections 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.518+0000 7fb62a5e7700 1 -- 172.21.15.89:0/267837630 wait complete. 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.519+0000 7fb62a5e7700 1 Processor -- start 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.519+0000 7fb62a5e7700 1 -- start start 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb62a5e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb62406f680 0x7fb624138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb62a5e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406ffd0 0x7fb624133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb62a5e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb624130670 0x7fb624134130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb62a5e7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6241383d0 con 0x7fb624130670 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb62a5e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb624139350 con 0x7fb62406ffd0 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb62a5e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb624137820 con 0x7fb62406f680 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb623fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb62406f680 0x7fb624138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb623fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb62406f680 0x7fb624138e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60408/0 (socket says 172.21.15.89:60408) 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb623fff700 1 -- 172.21.15.89:0/29032517 learned_addr learned my addr 172.21.15.89:0/29032517 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb628b84700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb624130670 0x7fb624134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb6237fe700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406ffd0 0x7fb624133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb623fff700 1 -- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406ffd0 msgr2=0x7fb624133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb623fff700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406ffd0 0x7fb624133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb623fff700 1 -- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb624130670 msgr2=0x7fb624134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb623fff700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb624130670 0x7fb624134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb623fff700 1 -- 172.21.15.89:0/29032517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6140094a0 con 0x7fb62406f680 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.520+0000 7fb628b84700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb624130670 0x7fb624134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.521+0000 7fb623fff700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb62406f680 0x7fb624138e10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb614009ef0 tx=0x7fb61400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.521+0000 7fb620ff9700 1 -- 172.21.15.89:0/29032517 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614013860 con 0x7fb62406f680 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.521+0000 7fb620ff9700 1 -- 172.21.15.89:0/29032517 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6140139c0 con 0x7fb62406f680 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.521+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6180094a0 con 0x7fb62406f680 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.521+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb624137170 con 0x7fb62406f680 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.521+0000 7fb620ff9700 1 -- 172.21.15.89:0/29032517 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614013d50 con 0x7fb62406f680 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.522+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6040052e0 con 0x7fb62406f680 2023-09-30T22:22:28.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.522+0000 7fb620ff9700 1 -- 172.21.15.89:0/29032517 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb61400c640 con 0x7fb62406f680 2023-09-30T22:22:28.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.523+0000 7fb620ff9700 1 -- 172.21.15.89:0/29032517 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb614043b90 con 0x7fb62406f680 2023-09-30T22:22:28.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.532+0000 7fb620ff9700 1 -- 172.21.15.89:0/29032517 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb61401b440 con 0x7fb62406f680 2023-09-30T22:22:28.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.580+0000 7f65d5905700 1 Processor -- start 2023-09-30T22:22:28.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.580+0000 7f65d5905700 1 -- start start 2023-09-30T22:22:28.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65d5905700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65d5905700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65d5905700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d00716a0 con 0x7f65d0130e50 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65d5905700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d009a390 con 0x7f65d0130540 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65d5905700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0071290 con 0x7f65d006fcf0 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65ceffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65ceffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40976/0 (socket says 172.21.15.89:40976) 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65ceffd700 1 -- 172.21.15.89:0/1954527442 learned_addr learned my addr 172.21.15.89:0/1954527442 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65cf7fe700 1 --2- 172.21.15.89:0/1954527442 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65ceffd700 1 -- 172.21.15.89:0/1954527442 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 msgr2=0x7f65d0138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65ceffd700 1 --2- 172.21.15.89:0/1954527442 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65ceffd700 1 -- 172.21.15.89:0/1954527442 >> v1:172.21.15.89:6789/0 conn(0x7f65d0130e50 legacy=0x7f65d006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65ce7fc700 1 -- 172.21.15.89:0/1954527442 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89120673 0 0) 0x7f65d00716a0 con 0x7f65d0130e50 2023-09-30T22:22:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.581+0000 7f65ceffd700 1 -- 172.21.15.89:0/1954527442 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d0130340 con 0x7f65d0130540 2023-09-30T22:22:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.582+0000 7f65ceffd700 1 --2- 172.21.15.89:0/1954527442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 0x7f65d0130910 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f65c000bb90 tx=0x7f65c000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a67e87dee0949c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.582+0000 7f65ce7fc700 1 -- 172.21.15.89:0/1954527442 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000dd50 con 0x7f65d0130540 2023-09-30T22:22:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.582+0000 7f65ce7fc700 1 -- 172.21.15.89:0/1954527442 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c0010470 con 0x7f65d0130540 2023-09-30T22:22:28.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.582+0000 7f65ce7fc700 1 -- 172.21.15.89:0/1954527442 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0010800 con 0x7f65d0130540 2023-09-30T22:22:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.583+0000 7f65d5905700 1 -- 172.21.15.89:0/1954527442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 msgr2=0x7f65d0130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.583+0000 7f65d5905700 1 --2- 172.21.15.89:0/1954527442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 0x7f65d0130910 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f65c000bb90 tx=0x7f65c000bea0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.583+0000 7f65d5905700 1 -- 172.21.15.89:0/1954527442 shutdown_connections 2023-09-30T22:22:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.583+0000 7f65d5905700 1 --2- 172.21.15.89:0/1954527442 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.583+0000 7f65d5905700 1 --2- 172.21.15.89:0/1954527442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.583+0000 7f65d5905700 1 -- 172.21.15.89:0/1954527442 >> 172.21.15.89:0/1954527442 conn(0x7f65d00a2aa0 msgr2=0x7f65d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:28.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.583+0000 7f65d5905700 1 -- 172.21.15.89:0/1954527442 shutdown_connections 2023-09-30T22:22:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.583+0000 7f65d5905700 1 -- 172.21.15.89:0/1954527442 wait complete. 2023-09-30T22:22:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.584+0000 7f65d5905700 1 Processor -- start 2023-09-30T22:22:28.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.584+0000 7f65d5905700 1 -- start start 2023-09-30T22:22:28.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65d5905700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d006fcf0 0x7f65d011e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65d5905700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130540 0x7f65d011edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65d5905700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d0130e50 0x7f65d011f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65d5905700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0122060 con 0x7f65d006fcf0 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65d5905700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d012cc10 con 0x7f65d0130540 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65d5905700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0121cc0 con 0x7f65d0130e50 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65ceffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d006fcf0 0x7f65d011e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65ceffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d006fcf0 0x7f65d011e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40986/0 (socket says 172.21.15.89:40986) 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65ceffd700 1 -- 172.21.15.89:0/2710568609 learned_addr learned my addr 172.21.15.89:0/2710568609 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65c67fc700 1 --2- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130540 0x7f65d011edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65ceffd700 1 -- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d0130e50 msgr2=0x7f65d011f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.585+0000 7f65ceffd700 1 --2- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d0130e50 0x7f65d011f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65ceffd700 1 -- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130540 msgr2=0x7f65d011edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65ceffd700 1 --2- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130540 0x7f65d011edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65ceffd700 1 -- 172.21.15.89:0/2710568609 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c000b7f0 con 0x7f65d006fcf0 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65ceffd700 1 --2- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d006fcf0 0x7f65d011e870 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f65c000d5d0 tx=0x7f65c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65ccff9700 1 -- 172.21.15.89:0/2710568609 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c001d070 con 0x7f65d006fcf0 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65ccff9700 1 -- 172.21.15.89:0/2710568609 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c0006ab0 con 0x7f65d006fcf0 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00a06a0 con 0x7f65d006fcf0 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65ccff9700 1 -- 172.21.15.89:0/2710568609 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0004470 con 0x7f65d006fcf0 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.586+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d0120fc0 con 0x7f65d006fcf0 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.587+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d0074ea0 con 0x7f65d006fcf0 2023-09-30T22:22:28.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.587+0000 7f65ccff9700 1 -- 172.21.15.89:0/2710568609 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f65c00036a0 con 0x7f65d006fcf0 2023-09-30T22:22:28.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.587+0000 7f65ccff9700 1 -- 172.21.15.89:0/2710568609 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f65c0040ca0 con 0x7f65d006fcf0 2023-09-30T22:22:28.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.596+0000 7f65ccff9700 1 -- 172.21.15.89:0/2710568609 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65c0003950 con 0x7f65d006fcf0 2023-09-30T22:22:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.622+0000 7f99b44ca700 1 Processor -- start 2023-09-30T22:22:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.622+0000 7f99b44ca700 1 -- start start 2023-09-30T22:22:28.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b44ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99ac06a770 0x7f99ac09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b44ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b44ca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac09a810 con 0x7f99ac09dc30 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b44ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac06ab40 con 0x7f99ac06a770 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b44ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac09ebc0 con 0x7f99ac13d1f0 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b2266700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99ac06a770 0x7f99ac09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b2266700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99ac06a770 0x7f99ac09d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40998/0 (socket says 172.21.15.89:40998) 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b2266700 1 -- 172.21.15.89:0/3476971619 learned_addr learned my addr 172.21.15.89:0/3476971619 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b2a67700 1 --2- 172.21.15.89:0/3476971619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b2266700 1 -- 172.21.15.89:0/3476971619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 msgr2=0x7f99ac13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b2266700 1 --2- 172.21.15.89:0/3476971619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b2266700 1 -- 172.21.15.89:0/3476971619 >> v1:172.21.15.89:6789/0 conn(0x7f99ac09dc30 legacy=0x7f99ac09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.623+0000 7f99b2266700 1 -- 172.21.15.89:0/3476971619 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac09e920 con 0x7f99ac06a770 2023-09-30T22:22:28.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b2266700 1 --2- 172.21.15.89:0/3476971619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 0x7f99ac09d6f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f99a80098f0 tx=0x7f99a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4277d14e86eff1e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:28.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b1264700 1 -- 172.21.15.89:0/3476971619 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8011830 con 0x7f99ac06a770 2023-09-30T22:22:28.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b1264700 1 -- 172.21.15.89:0/3476971619 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a8011990 con 0x7f99ac06a770 2023-09-30T22:22:28.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b1264700 1 -- 172.21.15.89:0/3476971619 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8011d20 con 0x7f99ac06a770 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b44ca700 1 -- 172.21.15.89:0/3476971619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 msgr2=0x7f99ac09d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b44ca700 1 --2- 172.21.15.89:0/3476971619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 0x7f99ac09d6f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f99a80098f0 tx=0x7f99a8009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b44ca700 1 -- 172.21.15.89:0/3476971619 shutdown_connections 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b44ca700 1 --2- 172.21.15.89:0/3476971619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 0x7f99ac09d6f0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b44ca700 1 --2- 172.21.15.89:0/3476971619 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b44ca700 1 -- 172.21.15.89:0/3476971619 >> 172.21.15.89:0/3476971619 conn(0x7f99ac066f50 msgr2=0x7f99ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.624+0000 7f99b44ca700 1 -- 172.21.15.89:0/3476971619 shutdown_connections 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.625+0000 7f99b44ca700 1 -- 172.21.15.89:0/3476971619 wait complete. 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.625+0000 7f99b44ca700 1 Processor -- start 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.625+0000 7f99b44ca700 1 -- start start 2023-09-30T22:22:28.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b44ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 0x7f99ac1cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b44ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99ac09dc30 0x7f99ac1ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b44ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac1cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b44ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac127790 con 0x7f99ac06a770 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b44ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac128880 con 0x7f99ac09dc30 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b44ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac11ce20 con 0x7f99ac13d1f0 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2a67700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac1cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2a67700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac1cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60434/0 (socket says 172.21.15.89:60434) 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2a67700 1 -- 172.21.15.89:0/2465835341 learned_addr learned my addr 172.21.15.89:0/2465835341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2266700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 0x7f99ac1cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b1a65700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99ac09dc30 0x7f99ac1ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:28.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2a67700 1 -- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99ac09dc30 msgr2=0x7f99ac1ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2a67700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99ac09dc30 0x7f99ac1ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2a67700 1 -- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 msgr2=0x7f99ac1cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2a67700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 0x7f99ac1cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2a67700 1 -- 172.21.15.89:0/2465835341 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8009550 con 0x7f99ac13d1f0 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.626+0000 7f99b2266700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 0x7f99ac1cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.627+0000 7f99b2a67700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac1cd440 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f99a400efc0 tx=0x7f99a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.627+0000 7f99a2ffd700 1 -- 172.21.15.89:0/2465835341 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4009ca0 con 0x7f99ac13d1f0 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.627+0000 7f99a2ffd700 1 -- 172.21.15.89:0/2465835341 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a400f040 con 0x7f99ac13d1f0 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.627+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac0a3990 con 0x7f99ac13d1f0 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.627+0000 7f99a2ffd700 1 -- 172.21.15.89:0/2465835341 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4014480 con 0x7f99ac13d1f0 2023-09-30T22:22:28.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.627+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac127160 con 0x7f99ac13d1f0 2023-09-30T22:22:28.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.628+0000 7f99a2ffd700 1 -- 172.21.15.89:0/2465835341 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99a40145e0 con 0x7f99ac13d1f0 2023-09-30T22:22:28.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.628+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac074ea0 con 0x7f99ac13d1f0 2023-09-30T22:22:28.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.628+0000 7f99a2ffd700 1 -- 172.21.15.89:0/2465835341 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f99a4040390 con 0x7f99ac13d1f0 2023-09-30T22:22:28.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.633+0000 7f99a2ffd700 1 -- 172.21.15.89:0/2465835341 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99a4007920 con 0x7f99ac13d1f0 2023-09-30T22:22:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.655+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb5601a0b30 con 0x7fb56009f240 2023-09-30T22:22:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.656+0000 7fb54effd700 1 -- 172.21.15.89:0/3626306426 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb55401a430 con 0x7fb56009f240 2023-09-30T22:22:28.656 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:28.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb56009f240 msgr2=0x7fb5601a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb56009f240 0x7fb5601a4160 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fb554013000 tx=0x7fb554004750 comp rx=0 tx=0).stop 2023-09-30T22:22:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 shutdown_connections 2023-09-30T22:22:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb56009f650 0x7fb5601a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb56009e930 0x7fb5601a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 --2- 172.21.15.89:0/3626306426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb56009f240 0x7fb5601a4160 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 >> 172.21.15.89:0/3626306426 conn(0x7fb5600a4210 msgr2=0x7fb5601b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 shutdown_connections 2023-09-30T22:22:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.658+0000 7fb564c05700 1 -- 172.21.15.89:0/3626306426 wait complete. 2023-09-30T22:22:28.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:28.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.733+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb604005150 con 0x7fb62406f680 2023-09-30T22:22:28.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.734+0000 7fb620ff9700 1 -- 172.21.15.89:0/29032517 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb614020030 con 0x7fb62406f680 2023-09-30T22:22:28.733 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:28.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb62406f680 msgr2=0x7fb624138e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb62406f680 0x7fb624138e10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb614009ef0 tx=0x7fb61400ee90 comp rx=0 tx=0).stop 2023-09-30T22:22:28.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 shutdown_connections 2023-09-30T22:22:28.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb624130670 0x7fb624134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb62406f680 0x7fb624138e10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 --2- 172.21.15.89:0/29032517 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb62406ffd0 0x7fb624133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 >> 172.21.15.89:0/29032517 conn(0x7fb6240a2aa0 msgr2=0x7fb6240a1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:28.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 shutdown_connections 2023-09-30T22:22:28.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.736+0000 7fb62a5e7700 1 -- 172.21.15.89:0/29032517 wait complete. 2023-09-30T22:22:28.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.799+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f65d006b9e0 con 0x7f65d006fcf0 2023-09-30T22:22:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.799+0000 7f65ccff9700 1 -- 172.21.15.89:0/2710568609 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65c0027070 con 0x7f65d006fcf0 2023-09-30T22:22:28.799 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.801+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d006fcf0 msgr2=0x7f65d011e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.801+0000 7f65d5905700 1 --2- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d006fcf0 0x7f65d011e870 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f65c000d5d0 tx=0x7f65c0004750 comp rx=0 tx=0).stop 2023-09-30T22:22:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.802+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 shutdown_connections 2023-09-30T22:22:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.802+0000 7f65d5905700 1 --2- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d0130e50 0x7f65d011f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.802+0000 7f65d5905700 1 --2- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d006fcf0 0x7f65d011e870 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.802+0000 7f65d5905700 1 --2- 172.21.15.89:0/2710568609 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130540 0x7f65d011edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.802+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 >> 172.21.15.89:0/2710568609 conn(0x7f65d00a2aa0 msgr2=0x7f65d012d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.802+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 shutdown_connections 2023-09-30T22:22:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.802+0000 7f65d5905700 1 -- 172.21.15.89:0/2710568609 wait complete. 2023-09-30T22:22:28.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.827+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f99ac09a3d0 con 0x7f99ac13d1f0 2023-09-30T22:22:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.828+0000 7f99a2ffd700 1 -- 172.21.15.89:0/2465835341 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99a4018020 con 0x7f99ac13d1f0 2023-09-30T22:22:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 msgr2=0x7f99ac1cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac1cd440 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f99a400efc0 tx=0x7f99a400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 shutdown_connections 2023-09-30T22:22:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99ac13d1f0 0x7f99ac1cd440 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99ac06a770 0x7f99ac1cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 --2- 172.21.15.89:0/2465835341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99ac09dc30 0x7f99ac1ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 >> 172.21.15.89:0/2465835341 conn(0x7f99ac066f50 msgr2=0x7f99ac0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 shutdown_connections 2023-09-30T22:22:28.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:28.830+0000 7f99b44ca700 1 -- 172.21.15.89:0/2465835341 wait complete. 2023-09-30T22:22:28.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:29.676 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:29.748 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.805+0000 7fe56e994700 1 Processor -- start 2023-09-30T22:22:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.805+0000 7fe56e994700 1 -- start start 2023-09-30T22:22:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe56e994700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe56809d630 0x7fe5680a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe56e994700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe568138d80 0x7fe56813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe56e994700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5680a2a70 con 0x7fe5680a0a30 2023-09-30T22:22:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe56e994700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5680a01d0 con 0x7fe56809d630 2023-09-30T22:22:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe56e994700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe56809e2b0 con 0x7fe568138d80 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe5677fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe56809d630 0x7fe5680a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe56cf31700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe568138d80 0x7fe56813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe5677fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe56809d630 0x7fe5680a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41012/0 (socket says 172.21.15.89:41012) 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe5677fe700 1 -- 172.21.15.89:0/3599982405 learned_addr learned my addr 172.21.15.89:0/3599982405 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe5677fe700 1 -- 172.21.15.89:0/3599982405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe568138d80 msgr2=0x7fe56813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe5677fe700 1 --2- 172.21.15.89:0/3599982405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe568138d80 0x7fe56813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe5677fe700 1 -- 172.21.15.89:0/3599982405 >> v1:172.21.15.89:6789/0 conn(0x7fe5680a0a30 legacy=0x7fe5680a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe5677fe700 1 -- 172.21.15.89:0/3599982405 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56809dc50 con 0x7fe56809d630 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe5677fe700 1 --2- 172.21.15.89:0/3599982405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56809d630 0x7fe5680a1510 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098f0 tx=0x7fe55c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dd7df7dd7fd003 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe566ffd700 1 -- 172.21.15.89:0/3599982405 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c011830 con 0x7fe56809d630 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe566ffd700 1 -- 172.21.15.89:0/3599982405 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c011990 con 0x7fe56809d630 2023-09-30T22:22:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.806+0000 7fe566ffd700 1 -- 172.21.15.89:0/3599982405 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c011d20 con 0x7fe56809d630 2023-09-30T22:22:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.807+0000 7fe56e994700 1 -- 172.21.15.89:0/3599982405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56809d630 msgr2=0x7fe5680a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.807+0000 7fe56e994700 1 --2- 172.21.15.89:0/3599982405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56809d630 0x7fe5680a1510 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098f0 tx=0x7fe55c009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.807+0000 7fe56e994700 1 -- 172.21.15.89:0/3599982405 shutdown_connections 2023-09-30T22:22:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.807+0000 7fe56e994700 1 --2- 172.21.15.89:0/3599982405 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe568138d80 0x7fe56813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.807+0000 7fe56e994700 1 --2- 172.21.15.89:0/3599982405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56809d630 0x7fe5680a1510 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.807+0000 7fe56e994700 1 -- 172.21.15.89:0/3599982405 >> 172.21.15.89:0/3599982405 conn(0x7fe5680a3050 msgr2=0x7fe5680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.807+0000 7fe56e994700 1 -- 172.21.15.89:0/3599982405 shutdown_connections 2023-09-30T22:22:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.807+0000 7fe56e994700 1 -- 172.21.15.89:0/3599982405 wait complete. 2023-09-30T22:22:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.808+0000 7fe56e994700 1 Processor -- start 2023-09-30T22:22:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.808+0000 7fe56e994700 1 -- start start 2023-09-30T22:22:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.808+0000 7fe56e994700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe56809d630 0x7fe568120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.808+0000 7fe56e994700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5680a0a30 0x7fe568120d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.808+0000 7fe56e994700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe568138d80 0x7fe568121280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe56e994700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe568123fb0 con 0x7fe568138d80 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe56e994700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680a4e00 con 0x7fe5680a0a30 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe56e994700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680a0410 con 0x7fe56809d630 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5680a0a30 0x7fe568120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe567fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe56809d630 0x7fe568120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5680a0a30 0x7fe568120d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34394/0 (socket says 172.21.15.89:34394) 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 -- 172.21.15.89:0/3682541593 learned_addr learned my addr 172.21.15.89:0/3682541593 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe56cf31700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe568138d80 0x7fe568121280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 -- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe56809d630 msgr2=0x7fe568120800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe56809d630 0x7fe568120800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 -- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe568138d80 msgr2=0x7fe568121280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe568138d80 0x7fe568121280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 -- 172.21.15.89:0/3682541593 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c009550 con 0x7fe5680a0a30 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe567fff700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe56809d630 0x7fe568120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe5677fe700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5680a0a30 0x7fe568120d40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe55c000c00 tx=0x7fe55c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe564ff9700 1 -- 172.21.15.89:0/3682541593 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0036a0 con 0x7fe5680a0a30 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe564ff9700 1 -- 172.21.15.89:0/3682541593 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c003800 con 0x7fe5680a0a30 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe568122b70 con 0x7fe5680a0a30 2023-09-30T22:22:29.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe564ff9700 1 -- 172.21.15.89:0/3682541593 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c003bb0 con 0x7fe5680a0a30 2023-09-30T22:22:29.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.809+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe568124290 con 0x7fe5680a0a30 2023-09-30T22:22:29.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.810+0000 7fe564ff9700 1 -- 172.21.15.89:0/3682541593 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe55c021070 con 0x7fe5680a0a30 2023-09-30T22:22:29.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.810+0000 7fe564ff9700 1 -- 172.21.15.89:0/3682541593 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe55c040a10 con 0x7fe5680a0a30 2023-09-30T22:22:29.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.810+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5680747f0 con 0x7fe5680a0a30 2023-09-30T22:22:29.816 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:29.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.815+0000 7fe564ff9700 1 -- 172.21.15.89:0/3682541593 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe55c015020 con 0x7fe5680a0a30 2023-09-30T22:22:29.842 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:29.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.878+0000 7f3ec53ab700 1 Processor -- start 2023-09-30T22:22:29.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.878+0000 7f3ec53ab700 1 -- start start 2023-09-30T22:22:29.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.878+0000 7f3ec53ab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3ec009eca0 0x7f3ec013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.878+0000 7f3ec53ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec006f050 0x7f3ec013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.878+0000 7f3ec53ab700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec0070ee0 con 0x7f3ec006e570 2023-09-30T22:22:29.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.878+0000 7f3ec53ab700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec0071980 con 0x7f3ec009eca0 2023-09-30T22:22:29.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.878+0000 7f3ec53ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec0071480 con 0x7f3ec006f050 2023-09-30T22:22:29.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebe7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3ec009eca0 0x7f3ec013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec006f050 0x7f3ec013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebe7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3ec009eca0 0x7f3ec013cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41038/0 (socket says 172.21.15.89:41038) 2023-09-30T22:22:29.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebe7fc700 1 -- 172.21.15.89:0/846228016 learned_addr learned my addr 172.21.15.89:0/846228016 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:29.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebe7fc700 1 -- 172.21.15.89:0/846228016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec006f050 msgr2=0x7f3ec013f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:29.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebe7fc700 1 --2- 172.21.15.89:0/846228016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec006f050 0x7f3ec013f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebe7fc700 1 -- 172.21.15.89:0/846228016 >> v1:172.21.15.89:6789/0 conn(0x7f3ec006e570 legacy=0x7f3ec006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebdffb700 1 -- 172.21.15.89:0/846228016 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632634070 0 0) 0x7f3ec0070ee0 con 0x7f3ec006e570 2023-09-30T22:22:29.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebe7fc700 1 -- 172.21.15.89:0/846228016 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec0071b90 con 0x7f3ec009eca0 2023-09-30T22:22:29.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebe7fc700 1 --2- 172.21.15.89:0/846228016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec009eca0 0x7f3ec013cea0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098f0 tx=0x7f3eb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bb5d65caf4865a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:29.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebdffb700 1 -- 172.21.15.89:0/846228016 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400b8a0 con 0x7f3ec009eca0 2023-09-30T22:22:29.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebdffb700 1 -- 172.21.15.89:0/846228016 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb400ba00 con 0x7f3ec009eca0 2023-09-30T22:22:29.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.879+0000 7f3ebdffb700 1 -- 172.21.15.89:0/846228016 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400bd90 con 0x7f3ec009eca0 2023-09-30T22:22:29.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.880+0000 7f3ec53ab700 1 -- 172.21.15.89:0/846228016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec009eca0 msgr2=0x7f3ec013cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.880+0000 7f3ec53ab700 1 --2- 172.21.15.89:0/846228016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec009eca0 0x7f3ec013cea0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098f0 tx=0x7f3eb4009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:29.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.880+0000 7f3ec53ab700 1 -- 172.21.15.89:0/846228016 shutdown_connections 2023-09-30T22:22:29.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.880+0000 7f3ec53ab700 1 --2- 172.21.15.89:0/846228016 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec006f050 0x7f3ec013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.880+0000 7f3ec53ab700 1 --2- 172.21.15.89:0/846228016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec009eca0 0x7f3ec013cea0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.880+0000 7f3ec53ab700 1 -- 172.21.15.89:0/846228016 >> 172.21.15.89:0/846228016 conn(0x7f3ec00a2aa0 msgr2=0x7f3ec00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:29.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.881+0000 7f3ec53ab700 1 -- 172.21.15.89:0/846228016 shutdown_connections 2023-09-30T22:22:29.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.881+0000 7f3ec53ab700 1 -- 172.21.15.89:0/846228016 wait complete. 2023-09-30T22:22:29.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ec53ab700 1 Processor -- start 2023-09-30T22:22:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ec53ab700 1 -- start start 2023-09-30T22:22:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ec53ab700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ec006e570 0x7f3ec01364e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ec53ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec006f050 0x7f3ec0136a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ec53ab700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec013a500 0x7f3ec0136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ec53ab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec0138270 con 0x7f3ec006f050 2023-09-30T22:22:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ec53ab700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec0139340 con 0x7f3ec013a500 2023-09-30T22:22:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ec53ab700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec013a000 con 0x7f3ec006e570 2023-09-30T22:22:29.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ebeffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ec006e570 0x7f3ec01364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ebeffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ec006e570 0x7f3ec01364e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:60460/0 (socket says 172.21.15.89:60460) 2023-09-30T22:22:29.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ebeffd700 1 -- 172.21.15.89:0/4132914776 learned_addr learned my addr 172.21.15.89:0/4132914776 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:29.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ebe7fc700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec006f050 0x7f3ec0136a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.882+0000 7f3ebe7fc700 1 -- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ec006e570 msgr2=0x7f3ec01364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ebe7fc700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ec006e570 0x7f3ec01364e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ebe7fc700 1 -- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec013a500 msgr2=0x7f3ec0136f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ebe7fc700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec013a500 0x7f3ec0136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ebe7fc700 1 -- 172.21.15.89:0/4132914776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb4009550 con 0x7f3ec006f050 2023-09-30T22:22:29.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ebeffd700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ec006e570 0x7f3ec01364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:29.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ebe7fc700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec006f050 0x7f3ec0136a20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0070170 tx=0x7f3eb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:29.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ea77fe700 1 -- 172.21.15.89:0/4132914776 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4004510 con 0x7f3ec006f050 2023-09-30T22:22:29.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ea77fe700 1 -- 172.21.15.89:0/4132914776 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb40036a0 con 0x7f3ec006f050 2023-09-30T22:22:29.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.883+0000 7f3ea77fe700 1 -- 172.21.15.89:0/4132914776 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4003a10 con 0x7f3ec006f050 2023-09-30T22:22:29.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.884+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec0070610 con 0x7f3ec006f050 2023-09-30T22:22:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.884+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec01388f0 con 0x7f3ec006f050 2023-09-30T22:22:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.884+0000 7f3ea77fe700 1 -- 172.21.15.89:0/4132914776 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3eb4003b70 con 0x7f3ec006f050 2023-09-30T22:22:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.885+0000 7f3ea77fe700 1 -- 172.21.15.89:0/4132914776 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3eb4021030 con 0x7f3ec006f050 2023-09-30T22:22:29.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.885+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec006eb50 con 0x7f3ec006f050 2023-09-30T22:22:29.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.894+0000 7f3ea77fe700 1 -- 172.21.15.89:0/4132914776 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3eb402b080 con 0x7f3ec006f050 2023-09-30T22:22:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.949+0000 7f07769f6700 1 Processor -- start 2023-09-30T22:22:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.949+0000 7f07769f6700 1 -- start start 2023-09-30T22:22:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f07769f6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07701305f0 0x7f07701309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f07769f6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077006fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f07769f6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07701301c0 con 0x7f077006ff90 2023-09-30T22:22:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f07769f6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0770071ff0 con 0x7f07701305f0 2023-09-30T22:22:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f07769f6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0770070e70 con 0x7f077006f660 2023-09-30T22:22:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f076ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07701305f0 0x7f07701309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f076ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07701305f0 0x7f07701309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41066/0 (socket says 172.21.15.89:41066) 2023-09-30T22:22:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f076ffff700 1 -- 172.21.15.89:0/501895364 learned_addr learned my addr 172.21.15.89:0/501895364 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.950+0000 7f076f7fe700 1 --2- 172.21.15.89:0/501895364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077006fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.951+0000 7f076ffff700 1 -- 172.21.15.89:0/501895364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 msgr2=0x7f077006fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.951+0000 7f076ffff700 1 --2- 172.21.15.89:0/501895364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077006fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.951+0000 7f076ffff700 1 -- 172.21.15.89:0/501895364 >> v1:172.21.15.89:6789/0 conn(0x7f077006ff90 legacy=0x7f0770137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.951+0000 7f076ffff700 1 -- 172.21.15.89:0/501895364 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700722b0 con 0x7f07701305f0 2023-09-30T22:22:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.951+0000 7f076ffff700 1 --2- 172.21.15.89:0/501895364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07701305f0 0x7f07701309c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f07600098f0 tx=0x7f0760009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216a5fac17b5514b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.951+0000 7f076effd700 1 -- 172.21.15.89:0/501895364 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760011830 con 0x7f07701305f0 2023-09-30T22:22:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.951+0000 7f076effd700 1 -- 172.21.15.89:0/501895364 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0760011990 con 0x7f07701305f0 2023-09-30T22:22:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.951+0000 7f076effd700 1 -- 172.21.15.89:0/501895364 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760011d20 con 0x7f07701305f0 2023-09-30T22:22:29.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.952+0000 7f07769f6700 1 -- 172.21.15.89:0/501895364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07701305f0 msgr2=0x7f07701309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.952+0000 7f07769f6700 1 --2- 172.21.15.89:0/501895364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07701305f0 0x7f07701309c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f07600098f0 tx=0x7f0760009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:29.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.952+0000 7f07769f6700 1 -- 172.21.15.89:0/501895364 shutdown_connections 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.952+0000 7f07769f6700 1 --2- 172.21.15.89:0/501895364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07701305f0 0x7f07701309c0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.952+0000 7f07769f6700 1 --2- 172.21.15.89:0/501895364 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077006fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.952+0000 7f07769f6700 1 -- 172.21.15.89:0/501895364 >> 172.21.15.89:0/501895364 conn(0x7f07700a2aa0 msgr2=0x7f07700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.952+0000 7f07769f6700 1 -- 172.21.15.89:0/501895364 shutdown_connections 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.952+0000 7f07769f6700 1 -- 172.21.15.89:0/501895364 wait complete. 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.953+0000 7f07769f6700 1 Processor -- start 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.953+0000 7f07769f6700 1 -- start start 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.953+0000 7f07769f6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077011ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.953+0000 7f07769f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077006ff90 0x7f077011cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.953+0000 7f07769f6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07701305f0 0x7f077011d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.953+0000 7f07769f6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07701243b0 con 0x7f077006ff90 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.953+0000 7f07769f6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0770123230 con 0x7f077006f660 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.953+0000 7f07769f6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0770071820 con 0x7f07701305f0 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077006ff90 0x7f077011cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077006ff90 0x7f077011cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41076/0 (socket says 172.21.15.89:41076) 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 -- 172.21.15.89:0/1909829472 learned_addr learned my addr 172.21.15.89:0/1909829472 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076ffff700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077011ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f0774f93700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07701305f0 0x7f077011d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:29.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 -- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07701305f0 msgr2=0x7f077011d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07701305f0 0x7f077011d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 -- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 msgr2=0x7f077011ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:29.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077011ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:29.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 -- 172.21.15.89:0/1909829472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0760009550 con 0x7f077006ff90 2023-09-30T22:22:29.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076ffff700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:29.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f0774f93700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07701305f0 0x7f077011d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:29.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076f7fe700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077006ff90 0x7f077011cfc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9b0 tx=0x7f076400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:29.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076cff9700 1 -- 172.21.15.89:0/1909829472 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764009710 con 0x7f077006ff90 2023-09-30T22:22:29.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076cff9700 1 -- 172.21.15.89:0/1909829472 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076400ee50 con 0x7f077006ff90 2023-09-30T22:22:29.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.954+0000 7f076cff9700 1 -- 172.21.15.89:0/1909829472 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076400ca50 con 0x7f077006ff90 2023-09-30T22:22:29.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.955+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07701234f0 con 0x7f077006ff90 2023-09-30T22:22:29.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.955+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077006ef80 con 0x7f077006ff90 2023-09-30T22:22:29.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.956+0000 7f076cff9700 1 -- 172.21.15.89:0/1909829472 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f076400cbb0 con 0x7f077006ff90 2023-09-30T22:22:29.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.956+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0770074ea0 con 0x7f077006ff90 2023-09-30T22:22:29.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.956+0000 7f076cff9700 1 -- 172.21.15.89:0/1909829472 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f076403f7a0 con 0x7f077006ff90 2023-09-30T22:22:29.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.965+0000 7f076cff9700 1 -- 172.21.15.89:0/1909829472 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07640232f0 con 0x7f077006ff90 2023-09-30T22:22:29.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.998+0000 7f9a1c322700 1 Processor -- start 2023-09-30T22:22:29.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.998+0000 7f9a1c322700 1 -- start start 2023-09-30T22:22:29.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1c322700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9a140a0a30 0x7f9a140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:29.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1c322700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a1409d630 0x7f9a140a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1c322700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a14133930 0x7f9a14133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1c322700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a140a2a70 con 0x7f9a140a0a30 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1c322700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140a01d0 con 0x7f9a1409d630 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1c322700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1409e2b0 con 0x7f9a14133930 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a0be700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9a140a0a30 0x7f9a140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a0be700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9a140a0a30 0x7f9a140a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41086/0 (socket says 172.21.15.89:41086) 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a0be700 1 -- 172.21.15.89:0/1801545921 learned_addr learned my addr 172.21.15.89:0/1801545921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a8bf700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a14133930 0x7f9a14133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a198bd700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a1409d630 0x7f9a140a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a0be700 1 -- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a14133930 msgr2=0x7f9a14133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:30.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a0be700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a14133930 0x7f9a14133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a0be700 1 -- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a1409d630 msgr2=0x7f9a140a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:30.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a0be700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a1409d630 0x7f9a140a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:29.999+0000 7f9a1a0be700 1 -- 172.21.15.89:0/1801545921 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1409dc50 con 0x7f9a140a0a30 2023-09-30T22:22:30.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.000+0000 7f9a1a0be700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 0x7f9a140a0e00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f9a100098f0 tx=0x7f9a10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e98e14d7383cb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:30.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.000+0000 7f9a190bc700 1 -- 172.21.15.89:0/1801545921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10011830 con 0x7f9a140a0a30 2023-09-30T22:22:30.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.000+0000 7f9a190bc700 1 -- 172.21.15.89:0/1801545921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a10011990 con 0x7f9a140a0a30 2023-09-30T22:22:30.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.000+0000 7f9a190bc700 1 -- 172.21.15.89:0/1801545921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10011d20 con 0x7f9a140a0a30 2023-09-30T22:22:30.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.000+0000 7f9a1c322700 1 -- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 msgr2=0x7f9a140a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:30.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.000+0000 7f9a1c322700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 0x7f9a140a0e00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f9a100098f0 tx=0x7f9a10009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:30.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.001+0000 7f9a1c322700 1 -- 172.21.15.89:0/1801545921 shutdown_connections 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.001+0000 7f9a1c322700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a14133930 0x7f9a14133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.001+0000 7f9a1c322700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 0x7f9a140a0e00 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.001+0000 7f9a1c322700 1 --2- 172.21.15.89:0/1801545921 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a1409d630 0x7f9a140a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.001+0000 7f9a1c322700 1 -- 172.21.15.89:0/1801545921 >> 172.21.15.89:0/1801545921 conn(0x7f9a140a3050 msgr2=0x7f9a140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.001+0000 7f9a1c322700 1 -- 172.21.15.89:0/1801545921 shutdown_connections 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.001+0000 7f9a1c322700 1 -- 172.21.15.89:0/1801545921 wait complete. 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.001+0000 7f9a1c322700 1 Processor -- start 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1c322700 1 -- start start 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1c322700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a1409d630 0x7f9a1411f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1c322700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 0x7f9a1411f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1c322700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a14133930 0x7f9a141c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:30.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1c322700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1411efa0 con 0x7f9a140a0a30 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1c322700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14133690 con 0x7f9a14133930 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1c322700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14133450 con 0x7f9a1409d630 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a8bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a14133930 0x7f9a141c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a8bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a14133930 0x7f9a141c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34436/0 (socket says 172.21.15.89:34436) 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a8bf700 1 -- 172.21.15.89:0/3286293490 learned_addr learned my addr 172.21.15.89:0/3286293490 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a0be700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a1409d630 0x7f9a1411f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a8bf700 1 -- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a1409d630 msgr2=0x7f9a1411f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a8bf700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a1409d630 0x7f9a1411f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a198bd700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 0x7f9a1411f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a8bf700 1 -- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 msgr2=0x7f9a1411f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:30.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a8bf700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 0x7f9a1411f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a1a8bf700 1 -- 172.21.15.89:0/3286293490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a10009550 con 0x7f9a14133930 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.002+0000 7f9a198bd700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 0x7f9a1411f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.003+0000 7f9a1a8bf700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a14133930 0x7f9a141c8090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009dd0 tx=0x7f9a0c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.003+0000 7f9a0affd700 1 -- 172.21.15.89:0/3286293490 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00cd40 con 0x7f9a14133930 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.003+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a14120ec0 con 0x7f9a14133930 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.003+0000 7f9a0affd700 1 -- 172.21.15.89:0/3286293490 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00f040 con 0x7f9a14133930 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.003+0000 7f9a0affd700 1 -- 172.21.15.89:0/3286293490 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0146c0 con 0x7f9a14133930 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.003+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a14122360 con 0x7f9a14133930 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.004+0000 7f9a0affd700 1 -- 172.21.15.89:0/3286293490 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9a0c014820 con 0x7f9a14133930 2023-09-30T22:22:30.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.004+0000 7f9a0affd700 1 -- 172.21.15.89:0/3286293490 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9a0c03ff00 con 0x7f9a14133930 2023-09-30T22:22:30.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.004+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a140747f0 con 0x7f9a14133930 2023-09-30T22:22:30.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.013+0000 7f9a0affd700 1 -- 172.21.15.89:0/3286293490 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a0c018020 con 0x7f9a14133930 2023-09-30T22:22:30.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.018+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe568072980 con 0x7fe5680a0a30 2023-09-30T22:22:30.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.018+0000 7fe564ff9700 1 -- 172.21.15.89:0/3682541593 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe55c00b3e0 con 0x7fe5680a0a30 2023-09-30T22:22:30.018 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:30.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.020+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5680a0a30 msgr2=0x7fe568120d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:30.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.020+0000 7fe56e994700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5680a0a30 0x7fe568120d40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe55c000c00 tx=0x7fe55c004750 comp rx=0 tx=0).stop 2023-09-30T22:22:30.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.020+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 shutdown_connections 2023-09-30T22:22:30.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.020+0000 7fe56e994700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe568138d80 0x7fe568121280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.020+0000 7fe56e994700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe56809d630 0x7fe568120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.020+0000 7fe56e994700 1 --2- 172.21.15.89:0/3682541593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5680a0a30 0x7fe568120d40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.020+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 >> 172.21.15.89:0/3682541593 conn(0x7fe5680a3050 msgr2=0x7fe56809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:30.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.021+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 shutdown_connections 2023-09-30T22:22:30.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.021+0000 7fe56e994700 1 -- 172.21.15.89:0/3682541593 wait complete. 2023-09-30T22:22:30.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:30.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.101+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ec0074ea0 con 0x7f3ec006f050 2023-09-30T22:22:30.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.101+0000 7f3ea77fe700 1 -- 172.21.15.89:0/4132914776 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3eb4015020 con 0x7f3ec006f050 2023-09-30T22:22:30.101 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:30.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.103+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec006f050 msgr2=0x7f3ec0136a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:30.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.103+0000 7f3ec53ab700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec006f050 0x7f3ec0136a20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0070170 tx=0x7f3eb4004750 comp rx=0 tx=0).stop 2023-09-30T22:22:30.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.103+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 shutdown_connections 2023-09-30T22:22:30.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.103+0000 7f3ec53ab700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3ec013a500 0x7f3ec0136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.103+0000 7f3ec53ab700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3ec006e570 0x7f3ec01364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.103+0000 7f3ec53ab700 1 --2- 172.21.15.89:0/4132914776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec006f050 0x7f3ec0136a20 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.103+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 >> 172.21.15.89:0/4132914776 conn(0x7f3ec00a2aa0 msgr2=0x7f3ec009ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:30.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.104+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 shutdown_connections 2023-09-30T22:22:30.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.104+0000 7f3ec53ab700 1 -- 172.21.15.89:0/4132914776 wait complete. 2023-09-30T22:22:30.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:30.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.170+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f077006b9e0 con 0x7f077006ff90 2023-09-30T22:22:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.171+0000 7f076cff9700 1 -- 172.21.15.89:0/1909829472 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0764017030 con 0x7f077006ff90 2023-09-30T22:22:30.171 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077006ff90 msgr2=0x7f077011cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077006ff90 0x7f077011cfc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9b0 tx=0x7f076400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 shutdown_connections 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07701305f0 0x7f077011d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f077006f660 0x7f077011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 --2- 172.21.15.89:0/1909829472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f077006ff90 0x7f077011cfc0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 >> 172.21.15.89:0/1909829472 conn(0x7f07700a2aa0 msgr2=0x7f077012e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 shutdown_connections 2023-09-30T22:22:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.173+0000 7f07769f6700 1 -- 172.21.15.89:0/1909829472 wait complete. 2023-09-30T22:22:30.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:30.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.209+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9a14072980 con 0x7f9a14133930 2023-09-30T22:22:30.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.209+0000 7f9a0affd700 1 -- 172.21.15.89:0/3286293490 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a0c01a8b0 con 0x7f9a14133930 2023-09-30T22:22:30.209 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a14133930 msgr2=0x7f9a141c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a14133930 0x7f9a141c8090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009dd0 tx=0x7f9a0c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 shutdown_connections 2023-09-30T22:22:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a14133930 0x7f9a141c8090 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a1409d630 0x7f9a1411f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 --2- 172.21.15.89:0/3286293490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a140a0a30 0x7f9a1411f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 >> 172.21.15.89:0/3286293490 conn(0x7f9a140a3050 msgr2=0x7f9a1409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 shutdown_connections 2023-09-30T22:22:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:30.211+0000 7f9a1c322700 1 -- 172.21.15.89:0/3286293490 wait complete. 2023-09-30T22:22:30.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:31.035 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:31.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1d154f700 1 Processor -- start 2023-09-30T22:22:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1d154f700 1 -- start start 2023-09-30T22:22:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1d154f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa1cc09f410 0x7fa1cc136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1d154f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc136fe0 0x7fa1cc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1d154f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1cc070940 con 0x7fa1cc09e930 2023-09-30T22:22:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1d154f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1cc06fd60 con 0x7fa1cc09f410 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1d154f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc06fa80 con 0x7fa1cc136fe0 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1ca7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa1cc09f410 0x7fa1cc136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.173+0000 7fa1cb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc136fe0 0x7fa1cc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1ca7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa1cc09f410 0x7fa1cc136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48392/0 (socket says 172.21.15.89:48392) 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1ca7fc700 1 -- 172.21.15.89:0/2792242471 learned_addr learned my addr 172.21.15.89:0/2792242471 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1cb7fe700 1 -- 172.21.15.89:0/2792242471 >> v2:172.21.15.89:3300/0 conn(0x7fa1cc09f410 msgr2=0x7fa1cc136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1cb7fe700 1 --2- 172.21.15.89:0/2792242471 >> v2:172.21.15.89:3300/0 conn(0x7fa1cc09f410 0x7fa1cc136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1cb7fe700 1 -- 172.21.15.89:0/2792242471 >> v1:172.21.15.89:6789/0 conn(0x7fa1cc09e930 legacy=0x7fa1cc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1cb7fe700 1 -- 172.21.15.89:0/2792242471 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc09fb60 con 0x7fa1cc136fe0 2023-09-30T22:22:31.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1cb7fe700 1 --2- 172.21.15.89:0/2792242471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc136fe0 0x7fa1cc1393c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009d90 tx=0x7fa1c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2632686155fc7984 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:31.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1c9ffb700 1 -- 172.21.15.89:0/2792242471 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400e070 con 0x7fa1cc136fe0 2023-09-30T22:22:31.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1c9ffb700 1 -- 172.21.15.89:0/2792242471 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c400bb20 con 0x7fa1cc136fe0 2023-09-30T22:22:31.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.174+0000 7fa1c9ffb700 1 -- 172.21.15.89:0/2792242471 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4012430 con 0x7fa1cc136fe0 2023-09-30T22:22:31.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.175+0000 7fa1d154f700 1 -- 172.21.15.89:0/2792242471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc136fe0 msgr2=0x7fa1cc1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.175+0000 7fa1d154f700 1 --2- 172.21.15.89:0/2792242471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc136fe0 0x7fa1cc1393c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009d90 tx=0x7fa1c400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.175+0000 7fa1d154f700 1 -- 172.21.15.89:0/2792242471 shutdown_connections 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.175+0000 7fa1d154f700 1 --2- 172.21.15.89:0/2792242471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc136fe0 0x7fa1cc1393c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.175+0000 7fa1d154f700 1 --2- 172.21.15.89:0/2792242471 >> v2:172.21.15.89:3300/0 conn(0x7fa1cc09f410 0x7fa1cc136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.175+0000 7fa1d154f700 1 -- 172.21.15.89:0/2792242471 >> 172.21.15.89:0/2792242471 conn(0x7fa1cc0a4210 msgr2=0x7fa1cc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.175+0000 7fa1d154f700 1 -- 172.21.15.89:0/2792242471 shutdown_connections 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.175+0000 7fa1d154f700 1 -- 172.21.15.89:0/2792242471 wait complete. 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1d154f700 1 Processor -- start 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1d154f700 1 -- start start 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1d154f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc09e930 0x7fa1cc1c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1d154f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1cc09f410 0x7fa1cc1c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1d154f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1cc136fe0 0x7fa1cc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1d154f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc1bc0c0 con 0x7fa1cc136fe0 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1d154f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc1224b0 con 0x7fa1cc09f410 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1d154f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc1320d0 con 0x7fa1cc09e930 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1cb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1cc136fe0 0x7fa1cc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1cb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1cc136fe0 0x7fa1cc06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48396/0 (socket says 172.21.15.89:48396) 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1cb7fe700 1 -- 172.21.15.89:0/2938050144 learned_addr learned my addr 172.21.15.89:0/2938050144 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1caffd700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc09e930 0x7fa1cc1c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.176+0000 7fa1ca7fc700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1cc09f410 0x7fa1cc1c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1cb7fe700 1 -- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc09e930 msgr2=0x7fa1cc1c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1cb7fe700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc09e930 0x7fa1cc1c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1cb7fe700 1 -- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1cc09f410 msgr2=0x7fa1cc1c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1cb7fe700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1cc09f410 0x7fa1cc1c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1cb7fe700 1 -- 172.21.15.89:0/2938050144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4009a70 con 0x7fa1cc136fe0 2023-09-30T22:22:31.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1caffd700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc09e930 0x7fa1cc1c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:31.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1cb7fe700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1cc136fe0 0x7fa1cc06b8b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4014040 tx=0x7fa1c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:31.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1b37fe700 1 -- 172.21.15.89:0/2938050144 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4012430 con 0x7fa1cc136fe0 2023-09-30T22:22:31.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1b37fe700 1 -- 172.21.15.89:0/2938050144 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c4020070 con 0x7fa1cc136fe0 2023-09-30T22:22:31.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1b37fe700 1 -- 172.21.15.89:0/2938050144 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4003ea0 con 0x7fa1cc136fe0 2023-09-30T22:22:31.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc121d40 con 0x7fa1cc136fe0 2023-09-30T22:22:31.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.177+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc06d4c0 con 0x7fa1cc136fe0 2023-09-30T22:22:31.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.178+0000 7fa1b37fe700 1 -- 172.21.15.89:0/2938050144 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa1c4004040 con 0x7fa1cc136fe0 2023-09-30T22:22:31.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.178+0000 7fa1b37fe700 1 -- 172.21.15.89:0/2938050144 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa1c4022070 con 0x7fa1cc136fe0 2023-09-30T22:22:31.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.178+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1cc121490 con 0x7fa1cc136fe0 2023-09-30T22:22:31.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.183+0000 7fa1b37fe700 1 -- 172.21.15.89:0/2938050144 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa1c402c080 con 0x7fa1cc136fe0 2023-09-30T22:22:31.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:31.224 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.251+0000 7fb0e2ed7700 1 Processor -- start 2023-09-30T22:22:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.251+0000 7fb0e2ed7700 1 -- start start 2023-09-30T22:22:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e2ed7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb0dc09e930 0x7fb0dc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e2ed7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09f240 0x7fb0dc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e2ed7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f650 0x7fb0dc131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e2ed7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc070940 con 0x7fb0dc09e930 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e2ed7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc06fd60 con 0x7fb0dc09f650 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e2ed7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc06fa80 con 0x7fb0dc09f240 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e0c73700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb0dc09e930 0x7fb0dc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e0c73700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb0dc09e930 0x7fb0dc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48412/0 (socket says 172.21.15.89:48412) 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e0c73700 1 -- 172.21.15.89:0/1920560436 learned_addr learned my addr 172.21.15.89:0/1920560436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e1474700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f650 0x7fb0dc131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0dbfff700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09f240 0x7fb0dc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e0c73700 1 -- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09f240 msgr2=0x7fb0dc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e0c73700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09f240 0x7fb0dc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e0c73700 1 -- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f650 msgr2=0x7fb0dc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e0c73700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f650 0x7fb0dc131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.252+0000 7fb0e0c73700 1 -- 172.21.15.89:0/1920560436 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc09fb60 con 0x7fb0dc09e930 2023-09-30T22:22:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.253+0000 7fb0e0c73700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09e930 0x7fb0dc09ed00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00098f0 tx=0x7fb0d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a83fd58f4f6d61 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.253+0000 7fb0db7fe700 1 -- 172.21.15.89:0/1920560436 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0011830 con 0x7fb0dc09e930 2023-09-30T22:22:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.253+0000 7fb0db7fe700 1 -- 172.21.15.89:0/1920560436 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d0011990 con 0x7fb0dc09e930 2023-09-30T22:22:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.253+0000 7fb0db7fe700 1 -- 172.21.15.89:0/1920560436 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0011d20 con 0x7fb0dc09e930 2023-09-30T22:22:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09e930 msgr2=0x7fb0dc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09e930 0x7fb0dc09ed00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00098f0 tx=0x7fb0d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/1920560436 shutdown_connections 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f650 0x7fb0dc131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09e930 0x7fb0dc09ed00 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 --2- 172.21.15.89:0/1920560436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09f240 0x7fb0dc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/1920560436 >> 172.21.15.89:0/1920560436 conn(0x7fb0dc0a4210 msgr2=0x7fb0dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/1920560436 shutdown_connections 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.254+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/1920560436 wait complete. 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.255+0000 7fb0e2ed7700 1 Processor -- start 2023-09-30T22:22:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.255+0000 7fb0e2ed7700 1 -- start start 2023-09-30T22:22:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e2ed7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09e930 0x7fb0dc1a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e2ed7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f240 0x7fb0dc1a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e2ed7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09f650 0x7fb0dc1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e2ed7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1a80d0 con 0x7fb0dc09f650 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e2ed7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1a7430 con 0x7fb0dc09f240 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e2ed7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0a34a0 con 0x7fb0dc09e930 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e0c73700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09e930 0x7fb0dc1a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e0c73700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09e930 0x7fb0dc1a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32830/0 (socket says 172.21.15.89:32830) 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f240 0x7fb0dc1a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e0c73700 1 -- 172.21.15.89:0/4229944749 learned_addr learned my addr 172.21.15.89:0/4229944749 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.256+0000 7fb0e1474700 1 --2- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09f650 0x7fb0dc1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0e0c73700 1 -- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f240 msgr2=0x7fb0dc1a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0e0c73700 1 --2- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f240 0x7fb0dc1a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0e0c73700 1 -- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09f650 msgr2=0x7fb0dc1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0e0c73700 1 --2- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09f650 0x7fb0dc1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0e0c73700 1 -- 172.21.15.89:0/4229944749 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0009550 con 0x7fb0dc09e930 2023-09-30T22:22:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0e0c73700 1 --2- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09e930 0x7fb0dc1a3c40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009c40 tx=0x7fb0d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0d97fa700 1 -- 172.21.15.89:0/4229944749 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0004240 con 0x7fb0dc09e930 2023-09-30T22:22:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0d97fa700 1 -- 172.21.15.89:0/4229944749 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d001f070 con 0x7fb0dc09e930 2023-09-30T22:22:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc0a0190 con 0x7fb0dc09e930 2023-09-30T22:22:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0d97fa700 1 -- 172.21.15.89:0/4229944749 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00045e0 con 0x7fb0dc09e930 2023-09-30T22:22:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.257+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc1a6a50 con 0x7fb0dc09e930 2023-09-30T22:22:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.258+0000 7fb0d97fa700 1 -- 172.21.15.89:0/4229944749 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0d0021030 con 0x7fb0dc09e930 2023-09-30T22:22:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.258+0000 7fb0d97fa700 1 -- 172.21.15.89:0/4229944749 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb0d0040800 con 0x7fb0dc09e930 2023-09-30T22:22:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.258+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc1a7570 con 0x7fb0dc09e930 2023-09-30T22:22:31.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.263+0000 7fb0d97fa700 1 -- 172.21.15.89:0/4229944749 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0dc1a7570 con 0x7fb0dc09e930 2023-09-30T22:22:31.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.333+0000 7f374d9b7700 1 Processor -- start 2023-09-30T22:22:31.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f374d9b7700 1 -- start start 2023-09-30T22:22:31.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f374d9b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809d6f0 0x7f374809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f374d9b7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f374809e000 0x7f3748130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f374d9b7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374809f130 con 0x7f374809e410 2023-09-30T22:22:31.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f374d9b7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374809f520 con 0x7f374809e000 2023-09-30T22:22:31.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f374d9b7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f374809e930 con 0x7f374809d6f0 2023-09-30T22:22:31.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f37467fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f374809e000 0x7f3748130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f37467fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f374809e000 0x7f3748130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48424/0 (socket says 172.21.15.89:48424) 2023-09-30T22:22:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f37467fc700 1 -- 172.21.15.89:0/3988011726 learned_addr learned my addr 172.21.15.89:0/3988011726 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.334+0000 7f3746ffd700 1 --2- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809d6f0 0x7f374809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f37467fc700 1 -- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809d6f0 msgr2=0x7f374809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f37467fc700 1 --2- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809d6f0 0x7f374809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f37467fc700 1 -- 172.21.15.89:0/3988011726 >> v1:172.21.15.89:6789/0 conn(0x7f374809e410 legacy=0x7f3748130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f37467fc700 1 -- 172.21.15.89:0/3988011726 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374809b2d0 con 0x7f374809e000 2023-09-30T22:22:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f3746ffd700 1 --2- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809d6f0 0x7f374809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:31.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f37467fc700 1 --2- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e000 0x7f3748130660 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f373c0098f0 tx=0x7f373c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e92999c7b236c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:31.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f3745ffb700 1 -- 172.21.15.89:0/3988011726 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c004230 con 0x7f374809e000 2023-09-30T22:22:31.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f3745ffb700 1 -- 172.21.15.89:0/3988011726 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c004390 con 0x7f374809e000 2023-09-30T22:22:31.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.335+0000 7f3745ffb700 1 -- 172.21.15.89:0/3988011726 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c0036a0 con 0x7f374809e000 2023-09-30T22:22:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.336+0000 7f374d9b7700 1 -- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e000 msgr2=0x7f3748130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.336+0000 7f374d9b7700 1 --2- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e000 0x7f3748130660 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f373c0098f0 tx=0x7f373c009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.336+0000 7f374d9b7700 1 -- 172.21.15.89:0/3988011726 shutdown_connections 2023-09-30T22:22:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.336+0000 7f374d9b7700 1 --2- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809d6f0 0x7f374809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.336+0000 7f374d9b7700 1 --2- 172.21.15.89:0/3988011726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e000 0x7f3748130660 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.336+0000 7f374d9b7700 1 -- 172.21.15.89:0/3988011726 >> 172.21.15.89:0/3988011726 conn(0x7f3748066f50 msgr2=0x7f374809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.336+0000 7f374d9b7700 1 -- 172.21.15.89:0/3988011726 shutdown_connections 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.336+0000 7f374d9b7700 1 -- 172.21.15.89:0/3988011726 wait complete. 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.337+0000 7f374d9b7700 1 Processor -- start 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.337+0000 7f374d9b7700 1 -- start start 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f374d9b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f374809d6f0 0x7f3748122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f374d9b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809e000 0x7f3748122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f374d9b7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e410 0x7f37481232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f374d9b7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37481262f0 con 0x7f374809e410 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f374d9b7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37481243e0 con 0x7f374809d6f0 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f374d9b7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37480a1d00 con 0x7f374809e000 2023-09-30T22:22:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f37477fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e410 0x7f37481232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f37467fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809e000 0x7f3748122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f37477fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e410 0x7f37481232e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48438/0 (socket says 172.21.15.89:48438) 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.338+0000 7f37477fe700 1 -- 172.21.15.89:0/4074394529 learned_addr learned my addr 172.21.15.89:0/4074394529 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f37467fc700 1 -- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f374809d6f0 msgr2=0x7f3748122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f37467fc700 1 --2- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f374809d6f0 0x7f3748122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f37467fc700 1 -- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e410 msgr2=0x7f37481232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f37467fc700 1 --2- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e410 0x7f37481232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f37467fc700 1 -- 172.21.15.89:0/4074394529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380094a0 con 0x7f374809e000 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f37477fe700 1 --2- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e410 0x7f37481232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f37467fc700 1 --2- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809e000 0x7f3748122da0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f373c015000 tx=0x7f373c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:31.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f372ffff700 1 -- 172.21.15.89:0/4074394529 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c003c10 con 0x7f374809e000 2023-09-30T22:22:31.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f372ffff700 1 -- 172.21.15.89:0/4074394529 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c01f070 con 0x7f374809e000 2023-09-30T22:22:31.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c009550 con 0x7f374809e000 2023-09-30T22:22:31.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.339+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3748123c50 con 0x7f374809e000 2023-09-30T22:22:31.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.340+0000 7f372ffff700 1 -- 172.21.15.89:0/4074394529 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c011630 con 0x7f374809e000 2023-09-30T22:22:31.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.341+0000 7f372ffff700 1 -- 172.21.15.89:0/4074394529 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f373c021030 con 0x7f374809e000 2023-09-30T22:22:31.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.341+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37280052e0 con 0x7f374809e000 2023-09-30T22:22:31.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.341+0000 7f372ffff700 1 -- 172.21.15.89:0/4074394529 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f373c040d00 con 0x7f374809e000 2023-09-30T22:22:31.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.350+0000 7f372ffff700 1 -- 172.21.15.89:0/4074394529 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f373c00b7a0 con 0x7f374809e000 2023-09-30T22:22:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.362+0000 7f20cf267700 1 Processor -- start 2023-09-30T22:22:31.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.363+0000 7f20cf267700 1 -- start start 2023-09-30T22:22:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.363+0000 7f20cf267700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20c80a0a30 0x7f20c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.363+0000 7f20cf267700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c809d630 0x7f20c80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.363+0000 7f20cf267700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c8133930 0x7f20c8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.363+0000 7f20cf267700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c80a2a70 con 0x7f20c80a0a30 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.363+0000 7f20cf267700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80a01d0 con 0x7f20c8133930 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.363+0000 7f20cf267700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20c809e2b0 con 0x7f20c809d630 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cc802700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c809d630 0x7f20c80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd003700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20c80a0a30 0x7f20c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd804700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c8133930 0x7f20c8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd003700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20c80a0a30 0x7f20c80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48454/0 (socket says 172.21.15.89:48454) 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd003700 1 -- 172.21.15.89:0/2856083041 learned_addr learned my addr 172.21.15.89:0/2856083041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd804700 1 -- 172.21.15.89:0/2856083041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c809d630 msgr2=0x7f20c80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:31.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd804700 1 --2- 172.21.15.89:0/2856083041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c809d630 0x7f20c80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd804700 1 -- 172.21.15.89:0/2856083041 >> v2:172.21.15.89:3300/0 conn(0x7f20c80a0a30 msgr2=0x7f20c80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd804700 1 --2- 172.21.15.89:0/2856083041 >> v2:172.21.15.89:3300/0 conn(0x7f20c80a0a30 0x7f20c80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd804700 1 -- 172.21.15.89:0/2856083041 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c809dc50 con 0x7f20c8133930 2023-09-30T22:22:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.364+0000 7f20cd804700 1 --2- 172.21.15.89:0/2856083041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c8133930 0x7f20c8133db0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009d90 tx=0x7f20c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39466ac196fc9e7b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.365+0000 7f20bffff700 1 -- 172.21.15.89:0/2856083041 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c000e070 con 0x7f20c8133930 2023-09-30T22:22:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.365+0000 7f20bffff700 1 -- 172.21.15.89:0/2856083041 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c000bb20 con 0x7f20c8133930 2023-09-30T22:22:31.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.365+0000 7f20bffff700 1 -- 172.21.15.89:0/2856083041 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0012430 con 0x7f20c8133930 2023-09-30T22:22:31.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.365+0000 7f20cf267700 1 -- 172.21.15.89:0/2856083041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c8133930 msgr2=0x7f20c8133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.365+0000 7f20cf267700 1 --2- 172.21.15.89:0/2856083041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c8133930 0x7f20c8133db0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009d90 tx=0x7f20c000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:31.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.366+0000 7f20cf267700 1 -- 172.21.15.89:0/2856083041 shutdown_connections 2023-09-30T22:22:31.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.366+0000 7f20cf267700 1 --2- 172.21.15.89:0/2856083041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c8133930 0x7f20c8133db0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.366+0000 7f20cf267700 1 --2- 172.21.15.89:0/2856083041 >> v2:172.21.15.89:3300/0 conn(0x7f20c80a0a30 0x7f20c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.366+0000 7f20cf267700 1 --2- 172.21.15.89:0/2856083041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c809d630 0x7f20c80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.366+0000 7f20cf267700 1 -- 172.21.15.89:0/2856083041 >> 172.21.15.89:0/2856083041 conn(0x7f20c80a3050 msgr2=0x7f20c80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:31.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.366+0000 7f20cf267700 1 -- 172.21.15.89:0/2856083041 shutdown_connections 2023-09-30T22:22:31.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.366+0000 7f20cf267700 1 -- 172.21.15.89:0/2856083041 wait complete. 2023-09-30T22:22:31.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.367+0000 7f20cf267700 1 Processor -- start 2023-09-30T22:22:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.367+0000 7f20cf267700 1 -- start start 2023-09-30T22:22:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cf267700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c809d630 0x7f20c806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cf267700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20c80a0a30 0x7f20c806e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cf267700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c8133930 0x7f20c806e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cf267700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8070650 con 0x7f20c80a0a30 2023-09-30T22:22:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cf267700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80a0410 con 0x7f20c809d630 2023-09-30T22:22:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cf267700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20c806f7e0 con 0x7f20c8133930 2023-09-30T22:22:31.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c809d630 0x7f20c806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c809d630 0x7f20c806dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39554/0 (socket says 172.21.15.89:39554) 2023-09-30T22:22:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 -- 172.21.15.89:0/260571456 learned_addr learned my addr 172.21.15.89:0/260571456 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd804700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c8133930 0x7f20c806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cc802700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20c80a0a30 0x7f20c806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:31.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 -- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c8133930 msgr2=0x7f20c806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c8133930 0x7f20c806e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 -- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20c80a0a30 msgr2=0x7f20c806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20c80a0a30 0x7f20c806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 -- 172.21.15.89:0/260571456 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c0009a70 con 0x7f20c809d630 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cc802700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20c80a0a30 0x7f20c806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.368+0000 7f20cd003700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c809d630 0x7f20c806dac0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f20c400e940 tx=0x7f20c400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.369+0000 7f20bdffb700 1 -- 172.21.15.89:0/260571456 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40096a0 con 0x7f20c809d630 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.369+0000 7f20bdffb700 1 -- 172.21.15.89:0/260571456 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c400ee90 con 0x7f20c809d630 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.369+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c80a4150 con 0x7f20c809d630 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.369+0000 7f20bdffb700 1 -- 172.21.15.89:0/260571456 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c400ca90 con 0x7f20c809d630 2023-09-30T22:22:31.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.369+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c81cc350 con 0x7f20c809d630 2023-09-30T22:22:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.370+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c80747f0 con 0x7f20c809d630 2023-09-30T22:22:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.371+0000 7f20bdffb700 1 -- 172.21.15.89:0/260571456 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20c400cbf0 con 0x7f20c809d630 2023-09-30T22:22:31.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.371+0000 7f20bdffb700 1 -- 172.21.15.89:0/260571456 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f20c403faa0 con 0x7f20c809d630 2023-09-30T22:22:31.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.380+0000 7f20bdffb700 1 -- 172.21.15.89:0/260571456 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20c401b3b0 con 0x7f20c809d630 2023-09-30T22:22:31.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.397+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa1cc074ea0 con 0x7fa1cc136fe0 2023-09-30T22:22:31.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.397+0000 7fa1b37fe700 1 -- 172.21.15.89:0/2938050144 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa1c401a970 con 0x7fa1cc136fe0 2023-09-30T22:22:31.397 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:31.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.399+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1cc136fe0 msgr2=0x7fa1cc06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.399+0000 7fa1d154f700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1cc136fe0 0x7fa1cc06b8b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4014040 tx=0x7fa1c4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.399+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 shutdown_connections 2023-09-30T22:22:31.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.399+0000 7fa1d154f700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1cc136fe0 0x7fa1cc06b8b0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.399+0000 7fa1d154f700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa1cc09e930 0x7fa1cc1c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.399+0000 7fa1d154f700 1 --2- 172.21.15.89:0/2938050144 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa1cc09f410 0x7fa1cc1c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.399+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 >> 172.21.15.89:0/2938050144 conn(0x7fa1cc0a4210 msgr2=0x7fa1cc135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:31.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.402+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 shutdown_connections 2023-09-30T22:22:31.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.402+0000 7fa1d154f700 1 -- 172.21.15.89:0/2938050144 wait complete. 2023-09-30T22:22:31.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:31.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.491+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb0dc1a7570 con 0x7fb0dc09e930 2023-09-30T22:22:31.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.492+0000 7fb0d97fa700 1 -- 172.21.15.89:0/4229944749 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0dc1a7570 con 0x7fb0dc09e930 2023-09-30T22:22:31.492 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:31.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09e930 msgr2=0x7fb0dc1a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 --2- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09e930 0x7fb0dc1a3c40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009c40 tx=0x7fb0d0004750 comp rx=0 tx=0).stop 2023-09-30T22:22:31.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 shutdown_connections 2023-09-30T22:22:31.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 --2- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0dc09f650 0x7fb0dc1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 --2- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0dc09e930 0x7fb0dc1a3c40 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 --2- 172.21.15.89:0/4229944749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0dc09f240 0x7fb0dc1a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 >> 172.21.15.89:0/4229944749 conn(0x7fb0dc0a4210 msgr2=0x7fb0dc1b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:31.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 shutdown_connections 2023-09-30T22:22:31.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.494+0000 7fb0e2ed7700 1 -- 172.21.15.89:0/4229944749 wait complete. 2023-09-30T22:22:31.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.559+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3728005590 con 0x7f374809e000 2023-09-30T22:22:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.560+0000 7f372ffff700 1 -- 172.21.15.89:0/4074394529 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f373c00b3e0 con 0x7f374809e000 2023-09-30T22:22:31.653 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809e000 msgr2=0x7f3748122da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 --2- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809e000 0x7f3748122da0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f373c015000 tx=0x7f373c004750 comp rx=0 tx=0).stop 2023-09-30T22:22:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 shutdown_connections 2023-09-30T22:22:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 --2- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f374809e410 0x7f37481232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 --2- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f374809d6f0 0x7f3748122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 --2- 172.21.15.89:0/4074394529 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f374809e000 0x7f3748122da0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 >> 172.21.15.89:0/4074394529 conn(0x7f3748066f50 msgr2=0x7f374806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 shutdown_connections 2023-09-30T22:22:31.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.563+0000 7f374d9b7700 1 -- 172.21.15.89:0/4074394529 wait complete. 2023-09-30T22:22:31.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:31.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.587+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f20c8072980 con 0x7f20c809d630 2023-09-30T22:22:31.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.587+0000 7f20bdffb700 1 -- 172.21.15.89:0/260571456 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20c4021070 con 0x7f20c809d630 2023-09-30T22:22:31.657 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:31.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c809d630 msgr2=0x7f20c806dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:31.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c809d630 0x7f20c806dac0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f20c400e940 tx=0x7f20c400ec50 comp rx=0 tx=0).stop 2023-09-30T22:22:31.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 shutdown_connections 2023-09-30T22:22:31.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20c8133930 0x7f20c806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20c809d630 0x7f20c806dac0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 --2- 172.21.15.89:0/260571456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20c80a0a30 0x7f20c806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:31.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 >> 172.21.15.89:0/260571456 conn(0x7f20c80a3050 msgr2=0x7f20c81bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:31.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 shutdown_connections 2023-09-30T22:22:31.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:31.589+0000 7f20cf267700 1 -- 172.21.15.89:0/260571456 wait complete. 2023-09-30T22:22:31.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:32.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:32.506 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:32.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81d23d700 1 Processor -- start 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81d23d700 1 -- start start 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81d23d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809f410 0x7fe818136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81d23d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe818136fe0 0x7fe8181393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81d23d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe818070940 con 0x7fe81809e930 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81d23d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81806fd60 con 0x7fe818136fe0 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81d23d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe81806fa80 con 0x7fe81809f410 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809f410 0x7fe818136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809f410 0x7fe818136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32872/0 (socket says 172.21.15.89:32872) 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81659c700 1 -- 172.21.15.89:0/303760534 learned_addr learned my addr 172.21.15.89:0/303760534 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:32.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.547+0000 7fe81759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe818136fe0 0x7fe8181393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe81659c700 1 -- 172.21.15.89:0/303760534 >> v2:172.21.15.89:3300/0 conn(0x7fe818136fe0 msgr2=0x7fe8181393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe81659c700 1 --2- 172.21.15.89:0/303760534 >> v2:172.21.15.89:3300/0 conn(0x7fe818136fe0 0x7fe8181393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe81659c700 1 -- 172.21.15.89:0/303760534 >> v1:172.21.15.89:6789/0 conn(0x7fe81809e930 legacy=0x7fe81809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe81659c700 1 -- 172.21.15.89:0/303760534 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81809fb60 con 0x7fe81809f410 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe81759e700 1 --2- 172.21.15.89:0/303760534 >> v2:172.21.15.89:3300/0 conn(0x7fe818136fe0 0x7fe8181393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe81659c700 1 --2- 172.21.15.89:0/303760534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809f410 0x7fe818136aa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0098f0 tx=0x7fe80c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f01325f532138a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe815d9b700 1 -- 172.21.15.89:0/303760534 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c011830 con 0x7fe81809f410 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe815d9b700 1 -- 172.21.15.89:0/303760534 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c011990 con 0x7fe81809f410 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe815d9b700 1 -- 172.21.15.89:0/303760534 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c011d20 con 0x7fe81809f410 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe81d23d700 1 -- 172.21.15.89:0/303760534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809f410 msgr2=0x7fe818136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.548+0000 7fe81d23d700 1 --2- 172.21.15.89:0/303760534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809f410 0x7fe818136aa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0098f0 tx=0x7fe80c009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.549+0000 7fe81d23d700 1 -- 172.21.15.89:0/303760534 shutdown_connections 2023-09-30T22:22:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.549+0000 7fe81d23d700 1 --2- 172.21.15.89:0/303760534 >> v2:172.21.15.89:3300/0 conn(0x7fe818136fe0 0x7fe8181393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.549+0000 7fe81d23d700 1 --2- 172.21.15.89:0/303760534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809f410 0x7fe818136aa0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.549+0000 7fe81d23d700 1 -- 172.21.15.89:0/303760534 >> 172.21.15.89:0/303760534 conn(0x7fe8180a4210 msgr2=0x7fe8180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.549+0000 7fe81d23d700 1 -- 172.21.15.89:0/303760534 shutdown_connections 2023-09-30T22:22:32.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.549+0000 7fe81d23d700 1 -- 172.21.15.89:0/303760534 wait complete. 2023-09-30T22:22:32.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.550+0000 7fe81d23d700 1 Processor -- start 2023-09-30T22:22:32.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.550+0000 7fe81d23d700 1 -- start start 2023-09-30T22:22:32.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81d23d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809e930 0x7fe81806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81d23d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe81809f410 0x7fe81806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81d23d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe818136fe0 0x7fe8181c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81d23d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe818122230 con 0x7fe81809f410 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81d23d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe818122410 con 0x7fe818136fe0 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81d23d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe818132270 con 0x7fe81809e930 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe816d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809e930 0x7fe81806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81759e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe818136fe0 0x7fe8181c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe816d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809e930 0x7fe81806c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32886/0 (socket says 172.21.15.89:32886) 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81759e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe818136fe0 0x7fe8181c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39568/0 (socket says 172.21.15.89:39568) 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe816d9d700 1 -- 172.21.15.89:0/875314397 learned_addr learned my addr 172.21.15.89:0/875314397 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81659c700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe81809f410 0x7fe81806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81759e700 1 -- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809e930 msgr2=0x7fe81806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81759e700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809e930 0x7fe81806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81759e700 1 -- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe81809f410 msgr2=0x7fe81806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81759e700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe81809f410 0x7fe81806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81759e700 1 -- 172.21.15.89:0/875314397 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810009a70 con 0x7fe818136fe0 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe816d9d700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809e930 0x7fe81806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81659c700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe81809f410 0x7fe81806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.551+0000 7fe81759e700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe818136fe0 0x7fe8181c82f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ef90 tx=0x7fe81000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.552+0000 7fe7ff7fe700 1 -- 172.21.15.89:0/875314397 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000cc90 con 0x7fe818136fe0 2023-09-30T22:22:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.552+0000 7fe7ff7fe700 1 -- 172.21.15.89:0/875314397 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81000f040 con 0x7fe818136fe0 2023-09-30T22:22:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.552+0000 7fe7ff7fe700 1 -- 172.21.15.89:0/875314397 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100146c0 con 0x7fe818136fe0 2023-09-30T22:22:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.552+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c009550 con 0x7fe818136fe0 2023-09-30T22:22:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.552+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe818071d20 con 0x7fe818136fe0 2023-09-30T22:22:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.553+0000 7fe7ff7fe700 1 -- 172.21.15.89:0/875314397 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe810014820 con 0x7fe818136fe0 2023-09-30T22:22:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.553+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe818074ea0 con 0x7fe818136fe0 2023-09-30T22:22:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.553+0000 7fe7ff7fe700 1 -- 172.21.15.89:0/875314397 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe810040270 con 0x7fe818136fe0 2023-09-30T22:22:32.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.557+0000 7fe7ff7fe700 1 -- 172.21.15.89:0/875314397 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe810007920 con 0x7fe818136fe0 2023-09-30T22:22:32.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.633+0000 7f215c6e6700 1 Processor -- start 2023-09-30T22:22:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.633+0000 7f215c6e6700 1 -- start start 2023-09-30T22:22:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215c6e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409d6f0 0x7f215409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215c6e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409e410 0x7f2154130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215c6e6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215409f130 con 0x7f215409e000 2023-09-30T22:22:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215c6e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215409f520 con 0x7f215409e410 2023-09-30T22:22:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215c6e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f215409e930 con 0x7f215409d6f0 2023-09-30T22:22:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215a482700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409d6f0 0x7f215409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f2159c81700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f215409e000 0x7f2154130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47868/0 (socket says 172.21.15.89:47868) 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f2159c81700 1 -- 172.21.15.89:0/3825005331 learned_addr learned my addr 172.21.15.89:0/3825005331 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215ac83700 1 --2- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409e410 0x7f2154130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215a482700 1 -- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409e410 msgr2=0x7f2154130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215a482700 1 --2- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409e410 0x7f2154130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215a482700 1 -- 172.21.15.89:0/3825005331 >> v1:172.21.15.89:6789/0 conn(0x7f215409e000 legacy=0x7f2154130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f2159480700 1 -- 172.21.15.89:0/3825005331 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770295396 0 0) 0x7f215409f130 con 0x7f215409e000 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215a482700 1 -- 172.21.15.89:0/3825005331 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215409b2d0 con 0x7f215409d6f0 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.634+0000 7f215ac83700 1 --2- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409e410 0x7f2154130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215a482700 1 --2- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409d6f0 0x7f215409dac0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2144009870 tx=0x7f2144009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a464e2752518cfe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:32.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f2159480700 1 -- 172.21.15.89:0/3825005331 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144011830 con 0x7f215409d6f0 2023-09-30T22:22:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f2159480700 1 -- 172.21.15.89:0/3825005331 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2144011990 con 0x7f215409d6f0 2023-09-30T22:22:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f2159480700 1 -- 172.21.15.89:0/3825005331 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144011d20 con 0x7f215409d6f0 2023-09-30T22:22:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215c6e6700 1 -- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409d6f0 msgr2=0x7f215409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215c6e6700 1 --2- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409d6f0 0x7f215409dac0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2144009870 tx=0x7f2144009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215c6e6700 1 -- 172.21.15.89:0/3825005331 shutdown_connections 2023-09-30T22:22:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215c6e6700 1 --2- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409e410 0x7f2154130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215c6e6700 1 --2- 172.21.15.89:0/3825005331 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409d6f0 0x7f215409dac0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215c6e6700 1 -- 172.21.15.89:0/3825005331 >> 172.21.15.89:0/3825005331 conn(0x7f2154066f50 msgr2=0x7f215409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215c6e6700 1 -- 172.21.15.89:0/3825005331 shutdown_connections 2023-09-30T22:22:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.635+0000 7f215c6e6700 1 -- 172.21.15.89:0/3825005331 wait complete. 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.636+0000 7f215c6e6700 1 Processor -- start 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.636+0000 7f215c6e6700 1 -- start start 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215c6e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409d6f0 0x7f215411f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215c6e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f215409e000 0x7f215411f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215c6e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409e410 0x7f215411fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215c6e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2154125790 con 0x7f215409e000 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215c6e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21540725f0 con 0x7f215409d6f0 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215c6e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f215409c700 con 0x7f215409e410 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409d6f0 0x7f215411f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215ac83700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409e410 0x7f215411fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409d6f0 0x7f215411f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39598/0 (socket says 172.21.15.89:39598) 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f2159c81700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f215409e000 0x7f215411f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 -- 172.21.15.89:0/3919047258 learned_addr learned my addr 172.21.15.89:0/3919047258 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 -- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409e410 msgr2=0x7f215411fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 --2- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409e410 0x7f215411fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 -- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f215409e000 msgr2=0x7f215411f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 --2- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f215409e000 0x7f215411f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 -- 172.21.15.89:0/3919047258 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c009a70 con 0x7f215409d6f0 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215ac83700 1 --2- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409e410 0x7f215411fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f215a482700 1 --2- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409d6f0 0x7f215411f130 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2144009840 tx=0x7f2144004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f214affd700 1 -- 172.21.15.89:0/3919047258 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440044d0 con 0x7f215409d6f0 2023-09-30T22:22:32.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.637+0000 7f214affd700 1 -- 172.21.15.89:0/3919047258 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214401f070 con 0x7f215409d6f0 2023-09-30T22:22:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.638+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21440094a0 con 0x7f215409d6f0 2023-09-30T22:22:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.638+0000 7f214affd700 1 -- 172.21.15.89:0/3919047258 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440038e0 con 0x7f215409d6f0 2023-09-30T22:22:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.638+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2154120600 con 0x7f215409d6f0 2023-09-30T22:22:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.638+0000 7f214affd700 1 -- 172.21.15.89:0/3919047258 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2144003a40 con 0x7f215409d6f0 2023-09-30T22:22:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.638+0000 7f214affd700 1 -- 172.21.15.89:0/3919047258 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2144021070 con 0x7f215409d6f0 2023-09-30T22:22:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.639+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215409dd30 con 0x7f215409d6f0 2023-09-30T22:22:32.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.643+0000 7f214affd700 1 -- 172.21.15.89:0/3919047258 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2144015080 con 0x7f215409d6f0 2023-09-30T22:22:32.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:32.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.761+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe818122090 con 0x7fe818136fe0 2023-09-30T22:22:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.762+0000 7fe7ff7fe700 1 -- 172.21.15.89:0/875314397 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe818122090 con 0x7fe818136fe0 2023-09-30T22:22:32.762 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.764+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe818136fe0 msgr2=0x7fe8181c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.764+0000 7fe81d23d700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe818136fe0 0x7fe8181c82f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ef90 tx=0x7fe81000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.764+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 shutdown_connections 2023-09-30T22:22:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.764+0000 7fe81d23d700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe818136fe0 0x7fe8181c82f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.764+0000 7fe81d23d700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe81809e930 0x7fe81806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.764+0000 7fe81d23d700 1 --2- 172.21.15.89:0/875314397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe81809f410 0x7fe81806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.764+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 >> 172.21.15.89:0/875314397 conn(0x7fe8180a4210 msgr2=0x7fe818135d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.766+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 shutdown_connections 2023-09-30T22:22:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.766+0000 7fe81d23d700 1 -- 172.21.15.89:0/875314397 wait complete. 2023-09-30T22:22:32.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:32.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.790+0000 7f0e205e2700 1 Processor -- start 2023-09-30T22:22:32.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.790+0000 7f0e205e2700 1 -- start start 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e205e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e181338a0 0x7f0e18133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e205e2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e1809de50 0x7f0e1809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e205e2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1806ef30 con 0x7f0e1809e760 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e205e2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1806fef0 con 0x7f0e1809de50 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e205e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e181334b0 con 0x7f0e181338a0 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1e37e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e181338a0 0x7f0e18133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1e37e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e181338a0 0x7f0e18133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32898/0 (socket says 172.21.15.89:32898) 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1e37e700 1 -- 172.21.15.89:0/3002831999 learned_addr learned my addr 172.21.15.89:0/3002831999 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1db7d700 1 --2- 172.21.15.89:0/3002831999 >> v2:172.21.15.89:3300/0 conn(0x7f0e1809de50 0x7f0e1809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1e37e700 1 -- 172.21.15.89:0/3002831999 >> v2:172.21.15.89:3300/0 conn(0x7f0e1809de50 msgr2=0x7f0e1809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1e37e700 1 --2- 172.21.15.89:0/3002831999 >> v2:172.21.15.89:3300/0 conn(0x7f0e1809de50 0x7f0e1809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1e37e700 1 -- 172.21.15.89:0/3002831999 >> v1:172.21.15.89:6789/0 conn(0x7f0e1809e760 legacy=0x7f0e1813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1e37e700 1 -- 172.21.15.89:0/3002831999 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1806fc30 con 0x7f0e181338a0 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1e37e700 1 --2- 172.21.15.89:0/3002831999 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e181338a0 0x7f0e18133c70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0e140098f0 tx=0x7f0e14009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7182d13ac4759e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1d37c700 1 -- 172.21.15.89:0/3002831999 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14011830 con 0x7f0e181338a0 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1d37c700 1 -- 172.21.15.89:0/3002831999 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14011990 con 0x7f0e181338a0 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.791+0000 7f0e1d37c700 1 -- 172.21.15.89:0/3002831999 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14011d20 con 0x7f0e181338a0 2023-09-30T22:22:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.792+0000 7f0e205e2700 1 -- 172.21.15.89:0/3002831999 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e181338a0 msgr2=0x7f0e18133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.792+0000 7f0e205e2700 1 --2- 172.21.15.89:0/3002831999 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e181338a0 0x7f0e18133c70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0e140098f0 tx=0x7f0e14009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.792+0000 7f0e205e2700 1 -- 172.21.15.89:0/3002831999 shutdown_connections 2023-09-30T22:22:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.792+0000 7f0e205e2700 1 --2- 172.21.15.89:0/3002831999 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e181338a0 0x7f0e18133c70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.792+0000 7f0e205e2700 1 --2- 172.21.15.89:0/3002831999 >> v2:172.21.15.89:3300/0 conn(0x7f0e1809de50 0x7f0e1809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.792+0000 7f0e205e2700 1 -- 172.21.15.89:0/3002831999 >> 172.21.15.89:0/3002831999 conn(0x7f0e180a2aa0 msgr2=0x7f0e180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.792+0000 7f0e205e2700 1 -- 172.21.15.89:0/3002831999 shutdown_connections 2023-09-30T22:22:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.792+0000 7f0e205e2700 1 -- 172.21.15.89:0/3002831999 wait complete. 2023-09-30T22:22:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e205e2700 1 Processor -- start 2023-09-30T22:22:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e205e2700 1 -- start start 2023-09-30T22:22:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e205e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e1809de50 0x7f0e1811d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e205e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e1809e760 0x7f0e1811dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e205e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e181338a0 0x7f0e1811e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e205e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e181211d0 con 0x7f0e1809de50 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e205e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1811eb10 con 0x7f0e181338a0 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e205e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1811f360 con 0x7f0e1809e760 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e1db7d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e1809e760 0x7f0e1811dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e1db7d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e1809e760 0x7f0e1811dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32902/0 (socket says 172.21.15.89:32902) 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e1db7d700 1 -- 172.21.15.89:0/1697139470 learned_addr learned my addr 172.21.15.89:0/1697139470 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.793+0000 7f0e1e37e700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e1809de50 0x7f0e1811d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e1eb7f700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e181338a0 0x7f0e1811e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e1e37e700 1 -- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e1809e760 msgr2=0x7f0e1811dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e1e37e700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e1809e760 0x7f0e1811dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e1e37e700 1 -- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e181338a0 msgr2=0x7f0e1811e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e1e37e700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e181338a0 0x7f0e1811e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e1e37e700 1 -- 172.21.15.89:0/1697139470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14009550 con 0x7f0e1809de50 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e1e37e700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e1809de50 0x7f0e1811d6b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f0e14016000 tx=0x7f0e14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e0effd700 1 -- 172.21.15.89:0/1697139470 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14004400 con 0x7f0e1809de50 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e0effd700 1 -- 172.21.15.89:0/1697139470 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1401f070 con 0x7f0e1809de50 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e180a1010 con 0x7f0e1809de50 2023-09-30T22:22:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e0effd700 1 -- 172.21.15.89:0/1697139470 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140038e0 con 0x7f0e1809de50 2023-09-30T22:22:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.794+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e18121960 con 0x7f0e1809de50 2023-09-30T22:22:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.795+0000 7f0e0effd700 1 -- 172.21.15.89:0/1697139470 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e14021030 con 0x7f0e1809de50 2023-09-30T22:22:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.795+0000 7f0e0effd700 1 -- 172.21.15.89:0/1697139470 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0e14040ab0 con 0x7f0e1809de50 2023-09-30T22:22:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.796+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1811a470 con 0x7f0e1809de50 2023-09-30T22:22:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.800+0000 7f0e0effd700 1 -- 172.21.15.89:0/1697139470 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e1400b9d0 con 0x7f0e1809de50 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.842+0000 7fde7acd9700 1 Processor -- start 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.842+0000 7fde7acd9700 1 -- start start 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.842+0000 7fde7acd9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde7409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.842+0000 7fde7acd9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde7409f160 0x7fde74133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.842+0000 7fde7acd9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde7406bf30 con 0x7fde7409f5b0 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.842+0000 7fde7acd9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7409ffb0 con 0x7fde7409f160 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.842+0000 7fde7acd9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7406cb10 con 0x7fde7409e850 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde73fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde7409f160 0x7fde74133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde78a75700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde7409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde78a75700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde7409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39612/0 (socket says 172.21.15.89:39612) 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde73fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde7409f160 0x7fde74133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48522/0 (socket says 172.21.15.89:48522) 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde78a75700 1 -- 172.21.15.89:0/1401656569 learned_addr learned my addr 172.21.15.89:0/1401656569 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde78a75700 1 -- 172.21.15.89:0/1401656569 >> v2:172.21.15.89:3300/0 conn(0x7fde7409f160 msgr2=0x7fde74133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde78a75700 1 --2- 172.21.15.89:0/1401656569 >> v2:172.21.15.89:3300/0 conn(0x7fde7409f160 0x7fde74133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde78a75700 1 -- 172.21.15.89:0/1401656569 >> v1:172.21.15.89:6789/0 conn(0x7fde7409f5b0 legacy=0x7fde74134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde78a75700 1 -- 172.21.15.89:0/1401656569 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde740a0270 con 0x7fde7409e850 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde78a75700 1 --2- 172.21.15.89:0/1401656569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde7409ec20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fde68009920 tx=0x7fde68009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4efb4c416cde727 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde737fe700 1 -- 172.21.15.89:0/1401656569 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68011830 con 0x7fde7409e850 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde737fe700 1 -- 172.21.15.89:0/1401656569 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde68011990 con 0x7fde7409e850 2023-09-30T22:22:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.843+0000 7fde737fe700 1 -- 172.21.15.89:0/1401656569 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68011d20 con 0x7fde7409e850 2023-09-30T22:22:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.844+0000 7fde7acd9700 1 -- 172.21.15.89:0/1401656569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 msgr2=0x7fde7409ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.844+0000 7fde7acd9700 1 --2- 172.21.15.89:0/1401656569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde7409ec20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fde68009920 tx=0x7fde68009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.844+0000 7fde7acd9700 1 -- 172.21.15.89:0/1401656569 shutdown_connections 2023-09-30T22:22:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.844+0000 7fde7acd9700 1 --2- 172.21.15.89:0/1401656569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde7409ec20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.844+0000 7fde7acd9700 1 --2- 172.21.15.89:0/1401656569 >> v2:172.21.15.89:3300/0 conn(0x7fde7409f160 0x7fde74133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.844+0000 7fde7acd9700 1 -- 172.21.15.89:0/1401656569 >> 172.21.15.89:0/1401656569 conn(0x7fde740a3f70 msgr2=0x7fde740a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.844+0000 7fde7acd9700 1 -- 172.21.15.89:0/1401656569 shutdown_connections 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.844+0000 7fde7acd9700 1 -- 172.21.15.89:0/1401656569 wait complete. 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.845+0000 7fde7acd9700 1 Processor -- start 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.845+0000 7fde7acd9700 1 -- start start 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde7acd9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde741228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde7acd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7409f160 0x7fde74122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde7acd9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7409f5b0 0x7fde74123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde7acd9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde74125a30 con 0x7fde7409f160 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde7acd9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde741333e0 con 0x7fde7409e850 2023-09-30T22:22:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde7acd9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde74124440 con 0x7fde7409f5b0 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7409f160 0x7fde74122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7409f160 0x7fde74122e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48530/0 (socket says 172.21.15.89:48530) 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde73fff700 1 -- 172.21.15.89:0/4015744176 learned_addr learned my addr 172.21.15.89:0/4015744176 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.846+0000 7fde79276700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7409f5b0 0x7fde74123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde73fff700 1 -- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7409f5b0 msgr2=0x7fde74123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde73fff700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7409f5b0 0x7fde74123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde73fff700 1 -- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 msgr2=0x7fde741228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde78a75700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde741228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde73fff700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde741228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde73fff700 1 -- 172.21.15.89:0/4015744176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde68009550 con 0x7fde7409f160 2023-09-30T22:22:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde78a75700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde741228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde73fff700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7409f160 0x7fde74122e00 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fde6000e9b0 tx=0x7fde6000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde717fa700 1 -- 172.21.15.89:0/4015744176 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60009710 con 0x7fde7409f160 2023-09-30T22:22:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde717fa700 1 -- 172.21.15.89:0/4015744176 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6000c850 con 0x7fde7409f160 2023-09-30T22:22:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7406f9a0 con 0x7fde7409f160 2023-09-30T22:22:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde717fa700 1 -- 172.21.15.89:0/4015744176 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6000cbe0 con 0x7fde7409f160 2023-09-30T22:22:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.847+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde74125db0 con 0x7fde7409f160 2023-09-30T22:22:32.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.848+0000 7fde717fa700 1 -- 172.21.15.89:0/4015744176 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde6000cd40 con 0x7fde7409f160 2023-09-30T22:22:32.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.848+0000 7fde717fa700 1 -- 172.21.15.89:0/4015744176 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fde6003f780 con 0x7fde7409f160 2023-09-30T22:22:32.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.848+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde740747f0 con 0x7fde7409f160 2023-09-30T22:22:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.851+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2154074ea0 con 0x7f215409d6f0 2023-09-30T22:22:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.852+0000 7f214affd700 1 -- 172.21.15.89:0/3919047258 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f214402b020 con 0x7f215409d6f0 2023-09-30T22:22:32.852 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.853+0000 7fde717fa700 1 -- 172.21.15.89:0/4015744176 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde600232f0 con 0x7fde7409f160 2023-09-30T22:22:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.854+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409d6f0 msgr2=0x7f215411f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.854+0000 7f215c6e6700 1 --2- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409d6f0 0x7f215411f130 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2144009840 tx=0x7f2144004750 comp rx=0 tx=0).stop 2023-09-30T22:22:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.854+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 shutdown_connections 2023-09-30T22:22:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.854+0000 7f215c6e6700 1 --2- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f215409e410 0x7f215411fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.854+0000 7f215c6e6700 1 --2- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f215409d6f0 0x7f215411f130 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.854+0000 7f215c6e6700 1 --2- 172.21.15.89:0/3919047258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f215409e000 0x7f215411f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.854+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 >> 172.21.15.89:0/3919047258 conn(0x7f2154066f50 msgr2=0x7f21540a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.856+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 shutdown_connections 2023-09-30T22:22:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.856+0000 7f215c6e6700 1 -- 172.21.15.89:0/3919047258 wait complete. 2023-09-30T22:22:32.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:32.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.996+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0e18074ea0 con 0x7f0e1809de50 2023-09-30T22:22:32.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.996+0000 7f0e0effd700 1 -- 172.21.15.89:0/1697139470 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e140193b0 con 0x7f0e1809de50 2023-09-30T22:22:32.996 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:32.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e1809de50 msgr2=0x7f0e1811d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:32.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e1809de50 0x7f0e1811d6b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f0e14016000 tx=0x7f0e14004750 comp rx=0 tx=0).stop 2023-09-30T22:22:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 shutdown_connections 2023-09-30T22:22:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e181338a0 0x7f0e1811e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e1809de50 0x7f0e1811d6b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 --2- 172.21.15.89:0/1697139470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e1809e760 0x7f0e1811dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 >> 172.21.15.89:0/1697139470 conn(0x7f0e180a2aa0 msgr2=0x7f0e181baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 shutdown_connections 2023-09-30T22:22:32.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:32.998+0000 7f0e205e2700 1 -- 172.21.15.89:0/1697139470 wait complete. 2023-09-30T22:22:33.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:33.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.054+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fde74123d20 con 0x7fde7409f160 2023-09-30T22:22:33.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.054+0000 7fde717fa700 1 -- 172.21.15.89:0/4015744176 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde60011350 con 0x7fde7409f160 2023-09-30T22:22:33.055 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:33.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7409f160 msgr2=0x7fde74122e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7409f160 0x7fde74122e00 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fde6000e9b0 tx=0x7fde6000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 shutdown_connections 2023-09-30T22:22:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7409f5b0 0x7fde74123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7409e850 0x7fde741228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 --2- 172.21.15.89:0/4015744176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7409f160 0x7fde74122e00 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 >> 172.21.15.89:0/4015744176 conn(0x7fde740a3f70 msgr2=0x7fde740a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 shutdown_connections 2023-09-30T22:22:33.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.056+0000 7fde7acd9700 1 -- 172.21.15.89:0/4015744176 wait complete. 2023-09-30T22:22:33.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:33.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:33.873 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:33.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.906+0000 7fc555856700 1 Processor -- start 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.906+0000 7fc555856700 1 -- start start 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc555856700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc5500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc555856700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc550134170 0x7fc55013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc555856700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55009edd0 con 0x7fc5500a1340 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc555856700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55009e1f0 con 0x7fc550134170 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc555856700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55009dc50 con 0x7fc5500a0a30 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc5500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc550134170 0x7fc55013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc5500a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39638/0 (socket says 172.21.15.89:39638) 2023-09-30T22:22:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc550134170 0x7fc55013bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48544/0 (socket says 172.21.15.89:48544) 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54effd700 1 -- 172.21.15.89:0/3335930812 learned_addr learned my addr 172.21.15.89:0/3335930812 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54f7fe700 1 -- 172.21.15.89:0/3335930812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 msgr2=0x7fc5500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54f7fe700 1 --2- 172.21.15.89:0/3335930812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc5500a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54f7fe700 1 -- 172.21.15.89:0/3335930812 >> v1:172.21.15.89:6789/0 conn(0x7fc5500a1340 legacy=0x7fc550133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54f7fe700 1 -- 172.21.15.89:0/3335930812 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5500a1fe0 con 0x7fc550134170 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54effd700 1 --2- 172.21.15.89:0/3335930812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc5500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.907+0000 7fc54dffb700 1 -- 172.21.15.89:0/3335930812 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043322689 0 0) 0x7fc55009edd0 con 0x7fc5500a1340 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc54f7fe700 1 --2- 172.21.15.89:0/3335930812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 0x7fc55013bd30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc548009d90 tx=0x7fc54800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeeb478375d59e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc54dffb700 1 -- 172.21.15.89:0/3335930812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800e070 con 0x7fc550134170 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc54dffb700 1 -- 172.21.15.89:0/3335930812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54800bb20 con 0x7fc550134170 2023-09-30T22:22:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc54dffb700 1 -- 172.21.15.89:0/3335930812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548012430 con 0x7fc550134170 2023-09-30T22:22:33.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc555856700 1 -- 172.21.15.89:0/3335930812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 msgr2=0x7fc55013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:33.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc555856700 1 --2- 172.21.15.89:0/3335930812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 0x7fc55013bd30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc548009d90 tx=0x7fc54800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:33.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc555856700 1 -- 172.21.15.89:0/3335930812 shutdown_connections 2023-09-30T22:22:33.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc555856700 1 --2- 172.21.15.89:0/3335930812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 0x7fc55013bd30 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc555856700 1 --2- 172.21.15.89:0/3335930812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc5500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc555856700 1 -- 172.21.15.89:0/3335930812 >> 172.21.15.89:0/3335930812 conn(0x7fc5500a6550 msgr2=0x7fc5500a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:33.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc555856700 1 -- 172.21.15.89:0/3335930812 shutdown_connections 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.908+0000 7fc555856700 1 -- 172.21.15.89:0/3335930812 wait complete. 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.909+0000 7fc555856700 1 Processor -- start 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc555856700 1 -- start start 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc555856700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc550122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc555856700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5500a1340 0x7fc550122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc555856700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 0x7fc550123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc555856700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc550125f30 con 0x7fc550134170 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc555856700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5500a4dc0 con 0x7fc5500a0a30 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc555856700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5500a0220 con 0x7fc5500a1340 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc550122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54e7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5500a1340 0x7fc550122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc550122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39650/0 (socket says 172.21.15.89:39650) 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54effd700 1 -- 172.21.15.89:0/3099085850 learned_addr learned my addr 172.21.15.89:0/3099085850 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54f7fe700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 0x7fc550123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54f7fe700 1 -- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5500a1340 msgr2=0x7fc550122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54f7fe700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5500a1340 0x7fc550122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54f7fe700 1 -- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 msgr2=0x7fc550122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54f7fe700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc550122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54f7fe700 1 -- 172.21.15.89:0/3099085850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc540009550 con 0x7fc550134170 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54e7fc700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5500a1340 0x7fc550122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.910+0000 7fc54f7fe700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 0x7fc550123200 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc548016000 tx=0x7fc548004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.911+0000 7fc537fff700 1 -- 172.21.15.89:0/3099085850 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548012430 con 0x7fc550134170 2023-09-30T22:22:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.911+0000 7fc537fff700 1 -- 172.21.15.89:0/3099085850 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548020070 con 0x7fc550134170 2023-09-30T22:22:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.911+0000 7fc537fff700 1 -- 172.21.15.89:0/3099085850 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548004280 con 0x7fc550134170 2023-09-30T22:22:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.911+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc548009a70 con 0x7fc550134170 2023-09-30T22:22:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.911+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501269a0 con 0x7fc550134170 2023-09-30T22:22:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.912+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc550125bd0 con 0x7fc550134170 2023-09-30T22:22:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.912+0000 7fc537fff700 1 -- 172.21.15.89:0/3099085850 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5480043e0 con 0x7fc550134170 2023-09-30T22:22:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.912+0000 7fc537fff700 1 -- 172.21.15.89:0/3099085850 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc548022070 con 0x7fc550134170 2023-09-30T22:22:33.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.916+0000 7fc537fff700 1 -- 172.21.15.89:0/3099085850 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc548012e60 con 0x7fc550134170 2023-09-30T22:22:33.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.998+0000 7f9c0ae38700 1 Processor -- start 2023-09-30T22:22:33.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.998+0000 7f9c0ae38700 1 -- start start 2023-09-30T22:22:34.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c0ae38700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c04131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c0ae38700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c0406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c0ae38700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0409e130 con 0x7f9c0406acb0 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c0ae38700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041312b0 con 0x7f9c041316c0 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c0ae38700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0409f6e0 con 0x7f9c0409d9f0 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c08bd4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c04131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c0406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c093d5700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9c0406acb0 0x7f9c04138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47928/0 (socket says 172.21.15.89:47928) 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:33.999+0000 7f9c093d5700 1 -- 172.21.15.89:0/1466590448 learned_addr learned my addr 172.21.15.89:0/1466590448 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c03fff700 1 -- 172.21.15.89:0/1466590448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 msgr2=0x7f9c04131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c037fe700 1 -- 172.21.15.89:0/1466590448 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319579955 0 0) 0x7f9bfc001490 con 0x7f9c0406acb0 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c03fff700 1 --2- 172.21.15.89:0/1466590448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c04131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c03fff700 1 -- 172.21.15.89:0/1466590448 >> v1:172.21.15.89:6789/0 conn(0x7f9c0406acb0 legacy=0x7f9c04138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c03fff700 1 -- 172.21.15.89:0/1466590448 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0409e760 con 0x7f9c0409d9f0 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c08bd4700 1 --2- 172.21.15.89:0/1466590448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c04131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c03fff700 1 --2- 172.21.15.89:0/1466590448 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c0406a770 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009870 tx=0x7f9bf0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315739f165de0a38 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c037fe700 1 -- 172.21.15.89:0/1466590448 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0011830 con 0x7f9c0409d9f0 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c037fe700 1 -- 172.21.15.89:0/1466590448 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0011990 con 0x7f9c0409d9f0 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c037fe700 1 -- 172.21.15.89:0/1466590448 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0011d20 con 0x7f9c0409d9f0 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c0ae38700 1 -- 172.21.15.89:0/1466590448 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 msgr2=0x7f9c0406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c0ae38700 1 --2- 172.21.15.89:0/1466590448 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c0406a770 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009870 tx=0x7f9bf0009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.000+0000 7f9c0ae38700 1 -- 172.21.15.89:0/1466590448 shutdown_connections 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.001+0000 7f9c0ae38700 1 --2- 172.21.15.89:0/1466590448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c04131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.001+0000 7f9c0ae38700 1 --2- 172.21.15.89:0/1466590448 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c0406a770 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.001+0000 7f9c0ae38700 1 -- 172.21.15.89:0/1466590448 >> 172.21.15.89:0/1466590448 conn(0x7f9c04066f50 msgr2=0x7f9c0409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:34.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.001+0000 7f9c0ae38700 1 -- 172.21.15.89:0/1466590448 shutdown_connections 2023-09-30T22:22:34.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.001+0000 7f9c0ae38700 1 -- 172.21.15.89:0/1466590448 wait complete. 2023-09-30T22:22:34.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.001+0000 7f9c0ae38700 1 Processor -- start 2023-09-30T22:22:34.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.001+0000 7f9c0ae38700 1 -- start start 2023-09-30T22:22:34.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c0ae38700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c0406acb0 0x7f9c041c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c0ae38700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c041c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c0ae38700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c041c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c0ae38700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0412b900 con 0x7f9c0406acb0 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c0ae38700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04071690 con 0x7f9c041316c0 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c0ae38700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0412b640 con 0x7f9c0409d9f0 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c08bd4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c0406acb0 0x7f9c041c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c041c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c08bd4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c0406acb0 0x7f9c041c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48560/0 (socket says 172.21.15.89:48560) 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c041c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32930/0 (socket says 172.21.15.89:32930) 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c08bd4700 1 -- 172.21.15.89:0/2504672326 learned_addr learned my addr 172.21.15.89:0/2504672326 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c093d5700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c041c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c08bd4700 1 -- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 msgr2=0x7f9c041c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c08bd4700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c041c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c08bd4700 1 -- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 msgr2=0x7f9c041c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.002+0000 7f9c08bd4700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c041c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c08bd4700 1 -- 172.21.15.89:0/2504672326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80094a0 con 0x7f9c0406acb0 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c03fff700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c041c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c093d5700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c041c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c08bd4700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c0406acb0 0x7f9c041c83c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800ebe0 tx=0x7f9bf800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c017fa700 1 -- 172.21.15.89:0/2504672326 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800c850 con 0x7f9c0406acb0 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c017fa700 1 -- 172.21.15.89:0/2504672326 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf800c9b0 con 0x7f9c0406acb0 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c017fa700 1 -- 172.21.15.89:0/2504672326 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800cd40 con 0x7f9c0406acb0 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf00094a0 con 0x7f9c0406acb0 2023-09-30T22:22:34.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.003+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c04070ff0 con 0x7f9c0406acb0 2023-09-30T22:22:34.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.004+0000 7f9c017fa700 1 -- 172.21.15.89:0/2504672326 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9bf8013470 con 0x7f9c0406acb0 2023-09-30T22:22:34.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.004+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c040a0690 con 0x7f9c0406acb0 2023-09-30T22:22:34.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.004+0000 7f9c017fa700 1 -- 172.21.15.89:0/2504672326 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9bf8042220 con 0x7f9c0406acb0 2023-09-30T22:22:34.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.009+0000 7f9c017fa700 1 -- 172.21.15.89:0/2504672326 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9bf8013720 con 0x7f9c0406acb0 2023-09-30T22:22:34.011 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:34.069 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:34.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.118+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc550074ea0 con 0x7fc550134170 2023-09-30T22:22:34.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.118+0000 7fc537fff700 1 -- 172.21.15.89:0/3099085850 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc54802c020 con 0x7fc550134170 2023-09-30T22:22:34.118 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:34.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.120+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 msgr2=0x7fc550123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.120+0000 7fc555856700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 0x7fc550123200 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc548016000 tx=0x7fc548004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.121+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 shutdown_connections 2023-09-30T22:22:34.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.121+0000 7fc555856700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc550134170 0x7fc550123200 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.121+0000 7fc555856700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5500a0a30 0x7fc550122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.121+0000 7fc555856700 1 --2- 172.21.15.89:0/3099085850 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5500a1340 0x7fc550122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.121+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 >> 172.21.15.89:0/3099085850 conn(0x7fc5500a6550 msgr2=0x7fc55009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:34.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.121+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 shutdown_connections 2023-09-30T22:22:34.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.121+0000 7fc555856700 1 -- 172.21.15.89:0/3099085850 wait complete. 2023-09-30T22:22:34.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:34.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.139+0000 7fae3f97f700 1 Processor -- start 2023-09-30T22:22:34.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.139+0000 7fae3f97f700 1 -- start start 2023-09-30T22:22:34.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3f97f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae3809d9f0 0x7fae38136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3f97f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae38136a70 0x7fae38138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3f97f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3809e130 con 0x7fae381316c0 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3f97f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae381321a0 con 0x7fae3809d9f0 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3f97f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae381312b0 con 0x7fae38136a70 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3cf1a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae3809d9f0 0x7fae38136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3cf1a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae3809d9f0 0x7fae38136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48574/0 (socket says 172.21.15.89:48574) 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3cf1a700 1 -- 172.21.15.89:0/1458036354 learned_addr learned my addr 172.21.15.89:0/1458036354 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3df1c700 1 --2- 172.21.15.89:0/1458036354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae38136a70 0x7fae38138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3cf1a700 1 -- 172.21.15.89:0/1458036354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae38136a70 msgr2=0x7fae38138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3cf1a700 1 --2- 172.21.15.89:0/1458036354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae38136a70 0x7fae38138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3cf1a700 1 -- 172.21.15.89:0/1458036354 >> v1:172.21.15.89:6789/0 conn(0x7fae381316c0 legacy=0x7fae38131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.140+0000 7fae3cf1a700 1 -- 172.21.15.89:0/1458036354 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3809f6e0 con 0x7fae3809d9f0 2023-09-30T22:22:34.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3cf1a700 1 --2- 172.21.15.89:0/1458036354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3809d9f0 0x7fae38136530 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fae280098f0 tx=0x7fae28009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c477a7ca095bc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:34.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae2ffff700 1 -- 172.21.15.89:0/1458036354 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28011830 con 0x7fae3809d9f0 2023-09-30T22:22:34.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae2ffff700 1 -- 172.21.15.89:0/1458036354 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae28011990 con 0x7fae3809d9f0 2023-09-30T22:22:34.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae2ffff700 1 -- 172.21.15.89:0/1458036354 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28011d20 con 0x7fae3809d9f0 2023-09-30T22:22:34.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3f97f700 1 -- 172.21.15.89:0/1458036354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3809d9f0 msgr2=0x7fae38136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3f97f700 1 --2- 172.21.15.89:0/1458036354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3809d9f0 0x7fae38136530 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fae280098f0 tx=0x7fae28009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:34.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3f97f700 1 -- 172.21.15.89:0/1458036354 shutdown_connections 2023-09-30T22:22:34.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3f97f700 1 --2- 172.21.15.89:0/1458036354 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae38136a70 0x7fae38138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3f97f700 1 --2- 172.21.15.89:0/1458036354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3809d9f0 0x7fae38136530 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3f97f700 1 -- 172.21.15.89:0/1458036354 >> 172.21.15.89:0/1458036354 conn(0x7fae38066f50 msgr2=0x7fae3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:34.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3f97f700 1 -- 172.21.15.89:0/1458036354 shutdown_connections 2023-09-30T22:22:34.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.141+0000 7fae3f97f700 1 -- 172.21.15.89:0/1458036354 wait complete. 2023-09-30T22:22:34.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.142+0000 7fae3f97f700 1 Processor -- start 2023-09-30T22:22:34.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.142+0000 7fae3f97f700 1 -- start start 2023-09-30T22:22:34.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3f97f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae3809d9f0 0x7fae381c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3f97f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae381316c0 0x7fae381c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3f97f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae38136a70 0x7fae381c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3f97f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38135700 con 0x7fae381316c0 2023-09-30T22:22:34.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3f97f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae38072220 con 0x7fae3809d9f0 2023-09-30T22:22:34.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3f97f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae38136550 con 0x7fae38136a70 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3d71b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae3809d9f0 0x7fae381c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3cf1a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae381316c0 0x7fae381c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3d71b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae3809d9f0 0x7fae381c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39692/0 (socket says 172.21.15.89:39692) 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3d71b700 1 -- 172.21.15.89:0/1201656238 learned_addr learned my addr 172.21.15.89:0/1201656238 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3df1c700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae38136a70 0x7fae381c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3cf1a700 1 -- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae38136a70 msgr2=0x7fae381c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3cf1a700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae38136a70 0x7fae381c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3cf1a700 1 -- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae3809d9f0 msgr2=0x7fae381c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3cf1a700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae3809d9f0 0x7fae381c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3cf1a700 1 -- 172.21.15.89:0/1201656238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae28009550 con 0x7fae381316c0 2023-09-30T22:22:34.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3df1c700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae38136a70 0x7fae381c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3d71b700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae3809d9f0 0x7fae381c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.143+0000 7fae3cf1a700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae381316c0 0x7fae381c6870 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fae280098f0 tx=0x7fae28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.144+0000 7fae2dffb700 1 -- 172.21.15.89:0/1201656238 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28004400 con 0x7fae381316c0 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.144+0000 7fae2dffb700 1 -- 172.21.15.89:0/1201656238 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2801f070 con 0x7fae381316c0 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.144+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3812b900 con 0x7fae381316c0 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.144+0000 7fae2dffb700 1 -- 172.21.15.89:0/1201656238 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae280038e0 con 0x7fae381316c0 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.144+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae380702f0 con 0x7fae381316c0 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.145+0000 7fae2dffb700 1 -- 172.21.15.89:0/1201656238 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae28021030 con 0x7fae381316c0 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.145+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae38074ea0 con 0x7fae381316c0 2023-09-30T22:22:34.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.145+0000 7fae2dffb700 1 -- 172.21.15.89:0/1201656238 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fae28040a90 con 0x7fae381316c0 2023-09-30T22:22:34.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.149+0000 7fae2dffb700 1 -- 172.21.15.89:0/1201656238 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae280239c0 con 0x7fae381316c0 2023-09-30T22:22:34.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.196+0000 7fae50eb2700 1 Processor -- start 2023-09-30T22:22:34.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.196+0000 7fae50eb2700 1 -- start start 2023-09-30T22:22:34.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.196+0000 7fae50eb2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.196+0000 7fae50eb2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae4c138d80 0x7fae4c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.196+0000 7fae50eb2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4c0a2a70 con 0x7fae4c09d630 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.196+0000 7fae50eb2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4c0a01d0 con 0x7fae4c138d80 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.196+0000 7fae50eb2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c09e2b0 con 0x7fae4c0a0a30 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.196+0000 7fae4ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fae4c138d80 0x7fae4c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae49d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fae4c09d630 0x7fae4c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47932/0 (socket says 172.21.15.89:47932) 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae49d9b700 1 -- 172.21.15.89:0/2786867858 learned_addr learned my addr 172.21.15.89:0/2786867858 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4a59c700 1 --2- 172.21.15.89:0/2786867858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4ad9d700 1 -- 172.21.15.89:0/2786867858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 msgr2=0x7fae4c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4ad9d700 1 --2- 172.21.15.89:0/2786867858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4ad9d700 1 -- 172.21.15.89:0/2786867858 >> v1:172.21.15.89:6789/0 conn(0x7fae4c09d630 legacy=0x7fae4c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4ad9d700 1 -- 172.21.15.89:0/2786867858 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c09dc50 con 0x7fae4c138d80 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4959a700 1 -- 172.21.15.89:0/2786867858 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498520664 0 0) 0x7fae4c0a2a70 con 0x7fae4c09d630 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4ad9d700 1 --2- 172.21.15.89:0/2786867858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c138d80 0x7fae4c13b160 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fae44009d90 tx=0x7fae4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5ff5d94e8dc951 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4959a700 1 -- 172.21.15.89:0/2786867858 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4400e070 con 0x7fae4c138d80 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4959a700 1 -- 172.21.15.89:0/2786867858 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4400bb20 con 0x7fae4c138d80 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.197+0000 7fae4959a700 1 -- 172.21.15.89:0/2786867858 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44012430 con 0x7fae4c138d80 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.198+0000 7fae50eb2700 1 -- 172.21.15.89:0/2786867858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c138d80 msgr2=0x7fae4c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.198+0000 7fae50eb2700 1 --2- 172.21.15.89:0/2786867858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c138d80 0x7fae4c13b160 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fae44009d90 tx=0x7fae4400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.198+0000 7fae50eb2700 1 -- 172.21.15.89:0/2786867858 shutdown_connections 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.198+0000 7fae50eb2700 1 --2- 172.21.15.89:0/2786867858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c138d80 0x7fae4c13b160 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.198+0000 7fae50eb2700 1 --2- 172.21.15.89:0/2786867858 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.198+0000 7fae50eb2700 1 -- 172.21.15.89:0/2786867858 >> 172.21.15.89:0/2786867858 conn(0x7fae4c0a3050 msgr2=0x7fae4c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:34.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.198+0000 7fae50eb2700 1 -- 172.21.15.89:0/2786867858 shutdown_connections 2023-09-30T22:22:34.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.198+0000 7fae50eb2700 1 -- 172.21.15.89:0/2786867858 wait complete. 2023-09-30T22:22:34.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.199+0000 7fae50eb2700 1 Processor -- start 2023-09-30T22:22:34.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.199+0000 7fae50eb2700 1 -- start start 2023-09-30T22:22:34.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.199+0000 7fae50eb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c09d630 0x7fae4c133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.199+0000 7fae50eb2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.199+0000 7fae50eb2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae4c138d80 0x7fae4c1342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:34.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.199+0000 7fae50eb2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c134d00 con 0x7fae4c09d630 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.199+0000 7fae50eb2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c136700 con 0x7fae4c138d80 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.199+0000 7fae50eb2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c137070 con 0x7fae4c0a0a30 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c09d630 0x7fae4c133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c09d630 0x7fae4c133830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48600/0 (socket says 172.21.15.89:48600) 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 -- 172.21.15.89:0/3108296962 learned_addr learned my addr 172.21.15.89:0/3108296962 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae49d9b700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4ad9d700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae4c138d80 0x7fae4c1342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 -- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 msgr2=0x7fae4c133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 -- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae4c138d80 msgr2=0x7fae4c1342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae4c138d80 0x7fae4c1342b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 -- 172.21.15.89:0/3108296962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae44009a70 con 0x7fae4c09d630 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae49d9b700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae4a59c700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c09d630 0x7fae4c133830 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00e9e0 tx=0x7fae3c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae3affd700 1 -- 172.21.15.89:0/3108296962 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c009710 con 0x7fae4c09d630 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae3affd700 1 -- 172.21.15.89:0/3108296962 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c00ee80 con 0x7fae4c09d630 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae3affd700 1 -- 172.21.15.89:0/3108296962 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00ca50 con 0x7fae4c09d630 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c0a0410 con 0x7fae4c09d630 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.200+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c136ae0 con 0x7fae4c09d630 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.201+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c135ce0 con 0x7fae4c09d630 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.201+0000 7fae3affd700 1 -- 172.21.15.89:0/3108296962 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae3c00cbb0 con 0x7fae4c09d630 2023-09-30T22:22:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.201+0000 7fae3affd700 1 -- 172.21.15.89:0/3108296962 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fae3c03f720 con 0x7fae4c09d630 2023-09-30T22:22:34.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.209+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9c040714d0 con 0x7f9c0406acb0 2023-09-30T22:22:34.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.210+0000 7f9c017fa700 1 -- 172.21.15.89:0/2504672326 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c040714d0 con 0x7f9c0406acb0 2023-09-30T22:22:34.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.210+0000 7fae3affd700 1 -- 172.21.15.89:0/3108296962 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae3c0232f0 con 0x7fae4c09d630 2023-09-30T22:22:34.210 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:34.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.212+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c0406acb0 msgr2=0x7f9c041c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.212+0000 7f9c0ae38700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c0406acb0 0x7f9c041c83c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800ebe0 tx=0x7f9bf800eef0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.212+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 shutdown_connections 2023-09-30T22:22:34.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.212+0000 7f9c0ae38700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c041316c0 0x7f9c041c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.212+0000 7f9c0ae38700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c0406acb0 0x7f9c041c83c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.212+0000 7f9c0ae38700 1 --2- 172.21.15.89:0/2504672326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c0409d9f0 0x7f9c041c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.212+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 >> 172.21.15.89:0/2504672326 conn(0x7f9c04066f50 msgr2=0x7f9c04130150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:34.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.213+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 shutdown_connections 2023-09-30T22:22:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.215+0000 7f9c0ae38700 1 -- 172.21.15.89:0/2504672326 wait complete. 2023-09-30T22:22:34.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.346+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fae3809e980 con 0x7fae381316c0 2023-09-30T22:22:34.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.346+0000 7fae2dffb700 1 -- 172.21.15.89:0/1201656238 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae28015020 con 0x7fae381316c0 2023-09-30T22:22:34.346 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae381316c0 msgr2=0x7fae381c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae381316c0 0x7fae381c6870 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fae280098f0 tx=0x7fae28004750 comp rx=0 tx=0).stop 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 shutdown_connections 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae38136a70 0x7fae381c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae3809d9f0 0x7fae381c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 --2- 172.21.15.89:0/1201656238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae381316c0 0x7fae381c6870 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 >> 172.21.15.89:0/1201656238 conn(0x7fae38066f50 msgr2=0x7fae381386e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 shutdown_connections 2023-09-30T22:22:34.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.349+0000 7fae3f97f700 1 -- 172.21.15.89:0/1201656238 wait complete. 2023-09-30T22:22:34.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:34.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.403+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae4c137f30 con 0x7fae4c09d630 2023-09-30T22:22:34.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.403+0000 7fae3affd700 1 -- 172.21.15.89:0/3108296962 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae3c01ae80 con 0x7fae4c09d630 2023-09-30T22:22:34.403 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:34.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c09d630 msgr2=0x7fae4c133830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:34.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c09d630 0x7fae4c133830 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00e9e0 tx=0x7fae3c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 shutdown_connections 2023-09-30T22:22:34.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fae4c138d80 0x7fae4c1342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae4c09d630 0x7fae4c133830 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 --2- 172.21.15.89:0/3108296962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fae4c0a0a30 0x7fae4c133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:34.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 >> 172.21.15.89:0/3108296962 conn(0x7fae4c0a3050 msgr2=0x7fae4c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:34.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 shutdown_connections 2023-09-30T22:22:34.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:34.405+0000 7fae50eb2700 1 -- 172.21.15.89:0/3108296962 wait complete. 2023-09-30T22:22:34.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:35.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:35.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.272+0000 7fa83c7f3700 1 Processor -- start 2023-09-30T22:22:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.272+0000 7fa83c7f3700 1 -- start start 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.272+0000 7fa83c7f3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8340a0a30 0x7fa8340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.272+0000 7fa83c7f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8340a1340 0x7fa834133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.272+0000 7fa83c7f3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83409edd0 con 0x7fa834133fa0 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83c7f3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83409e1f0 con 0x7fa8340a0a30 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83c7f3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83409dc50 con 0x7fa8340a1340 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83a58f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8340a0a30 0x7fa8340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83a58f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8340a0a30 0x7fa8340a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48604/0 (socket says 172.21.15.89:48604) 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa839d8e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8340a1340 0x7fa834133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83a58f700 1 -- 172.21.15.89:0/3403935803 learned_addr learned my addr 172.21.15.89:0/3403935803 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa839d8e700 1 -- 172.21.15.89:0/3403935803 >> v2:172.21.15.89:3300/0 conn(0x7fa8340a0a30 msgr2=0x7fa8340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa839d8e700 1 --2- 172.21.15.89:0/3403935803 >> v2:172.21.15.89:3300/0 conn(0x7fa8340a0a30 0x7fa8340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa839d8e700 1 -- 172.21.15.89:0/3403935803 >> v1:172.21.15.89:6789/0 conn(0x7fa834133fa0 legacy=0x7fa83413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa839d8e700 1 -- 172.21.15.89:0/3403935803 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8340a1fe0 con 0x7fa8340a1340 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83a58f700 1 --2- 172.21.15.89:0/3403935803 >> v2:172.21.15.89:3300/0 conn(0x7fa8340a0a30 0x7fa8340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa839d8e700 1 --2- 172.21.15.89:0/3403935803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8340a1340 0x7fa834133a60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa824009840 tx=0x7fa824009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=901d922a4a5a5f28 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83958d700 1 -- 172.21.15.89:0/3403935803 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824011830 con 0x7fa8340a1340 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83958d700 1 -- 172.21.15.89:0/3403935803 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa824011990 con 0x7fa8340a1340 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.273+0000 7fa83958d700 1 -- 172.21.15.89:0/3403935803 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824011d20 con 0x7fa8340a1340 2023-09-30T22:22:35.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.274+0000 7fa83c7f3700 1 -- 172.21.15.89:0/3403935803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8340a1340 msgr2=0x7fa834133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.274+0000 7fa83c7f3700 1 --2- 172.21.15.89:0/3403935803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8340a1340 0x7fa834133a60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa824009840 tx=0x7fa824009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.274+0000 7fa83c7f3700 1 -- 172.21.15.89:0/3403935803 shutdown_connections 2023-09-30T22:22:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.274+0000 7fa83c7f3700 1 --2- 172.21.15.89:0/3403935803 >> v2:172.21.15.89:3300/0 conn(0x7fa8340a0a30 0x7fa8340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.274+0000 7fa83c7f3700 1 --2- 172.21.15.89:0/3403935803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8340a1340 0x7fa834133a60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.274+0000 7fa83c7f3700 1 -- 172.21.15.89:0/3403935803 >> 172.21.15.89:0/3403935803 conn(0x7fa8340a6550 msgr2=0x7fa8340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.274+0000 7fa83c7f3700 1 -- 172.21.15.89:0/3403935803 shutdown_connections 2023-09-30T22:22:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.274+0000 7fa83c7f3700 1 -- 172.21.15.89:0/3403935803 wait complete. 2023-09-30T22:22:35.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83c7f3700 1 Processor -- start 2023-09-30T22:22:35.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83c7f3700 1 -- start start 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83c7f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a0a30 0x7fa83411e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83c7f3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a1340 0x7fa83411ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83c7f3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133fa0 0x7fa83411f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83c7f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa834120bc0 con 0x7fa8340a0a30 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83c7f3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340a3760 con 0x7fa834133fa0 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83c7f3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340a0220 con 0x7fa8340a1340 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83a58f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a0a30 0x7fa83411e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa839d8e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a1340 0x7fa83411ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.275+0000 7fa83ad90700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133fa0 0x7fa83411f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa83a58f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a0a30 0x7fa83411e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48616/0 (socket says 172.21.15.89:48616) 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa83a58f700 1 -- 172.21.15.89:0/498740777 learned_addr learned my addr 172.21.15.89:0/498740777 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa839d8e700 1 -- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133fa0 msgr2=0x7fa83411f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa839d8e700 1 --2- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133fa0 0x7fa83411f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa839d8e700 1 -- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a0a30 msgr2=0x7fa83411e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa839d8e700 1 --2- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a0a30 0x7fa83411e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa839d8e700 1 -- 172.21.15.89:0/498740777 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830009550 con 0x7fa8340a1340 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa83a58f700 1 --2- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a0a30 0x7fa83411e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa839d8e700 1 --2- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a1340 0x7fa83411ec90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa824016000 tx=0x7fa824004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa82affd700 1 -- 172.21.15.89:0/498740777 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240036a0 con 0x7fa8340a1340 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa82affd700 1 -- 172.21.15.89:0/498740777 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82401f070 con 0x7fa8340a1340 2023-09-30T22:22:35.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8240094a0 con 0x7fa8340a1340 2023-09-30T22:22:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa82affd700 1 -- 172.21.15.89:0/498740777 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824003a60 con 0x7fa8340a1340 2023-09-30T22:22:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.276+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa834122a00 con 0x7fa8340a1340 2023-09-30T22:22:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.277+0000 7fa82affd700 1 -- 172.21.15.89:0/498740777 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa824003bc0 con 0x7fa8340a1340 2023-09-30T22:22:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.277+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa834074ea0 con 0x7fa8340a1340 2023-09-30T22:22:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.277+0000 7fa82affd700 1 -- 172.21.15.89:0/498740777 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa824021070 con 0x7fa8340a1340 2023-09-30T22:22:35.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.282+0000 7fa82affd700 1 -- 172.21.15.89:0/498740777 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa824010380 con 0x7fa8340a1340 2023-09-30T22:22:35.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.358+0000 7f6441b9f700 1 Processor -- start 2023-09-30T22:22:35.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.358+0000 7f6441b9f700 1 -- start start 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.358+0000 7f6441b9f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f643c0a0a30 0x7f643c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f6441b9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c09d630 0x7f643c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f6441b9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c133930 0x7f643c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f6441b9f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c0a2a70 con 0x7f643c0a0a30 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f6441b9f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0a01d0 con 0x7f643c133930 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f6441b9f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f643c09e2b0 con 0x7f643c09d630 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f643c0a0a30 0x7f643c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c133930 0x7f643c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c09d630 0x7f643c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c133930 0x7f643c133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39722/0 (socket says 172.21.15.89:39722) 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f643c0a0a30 0x7f643c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48632/0 (socket says 172.21.15.89:48632) 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 -- 172.21.15.89:0/789429623 learned_addr learned my addr 172.21.15.89:0/789429623 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 -- 172.21.15.89:0/789429623 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c09d630 msgr2=0x7f643c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 --2- 172.21.15.89:0/789429623 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c09d630 0x7f643c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 -- 172.21.15.89:0/789429623 >> v2:172.21.15.89:3300/0 conn(0x7f643c0a0a30 msgr2=0x7f643c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 --2- 172.21.15.89:0/789429623 >> v2:172.21.15.89:3300/0 conn(0x7f643c0a0a30 0x7f643c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 -- 172.21.15.89:0/789429623 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c09dc50 con 0x7f643c133930 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643affd700 1 --2- 172.21.15.89:0/789429623 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c09d630 0x7f643c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643bfff700 1 --2- 172.21.15.89:0/789429623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c133930 0x7f643c133db0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e50 tx=0x7f643400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9145f0994858796e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643a7fc700 1 -- 172.21.15.89:0/789429623 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400e070 con 0x7f643c133930 2023-09-30T22:22:35.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643a7fc700 1 -- 172.21.15.89:0/789429623 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643400b910 con 0x7f643c133930 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.359+0000 7f643a7fc700 1 -- 172.21.15.89:0/789429623 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400bcc0 con 0x7f643c133930 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 -- 172.21.15.89:0/789429623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c133930 msgr2=0x7f643c133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 --2- 172.21.15.89:0/789429623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c133930 0x7f643c133db0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e50 tx=0x7f643400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 -- 172.21.15.89:0/789429623 shutdown_connections 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 --2- 172.21.15.89:0/789429623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c133930 0x7f643c133db0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 --2- 172.21.15.89:0/789429623 >> v2:172.21.15.89:3300/0 conn(0x7f643c0a0a30 0x7f643c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 --2- 172.21.15.89:0/789429623 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c09d630 0x7f643c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 -- 172.21.15.89:0/789429623 >> 172.21.15.89:0/789429623 conn(0x7f643c0a3050 msgr2=0x7f643c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 -- 172.21.15.89:0/789429623 shutdown_connections 2023-09-30T22:22:35.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.360+0000 7f6441b9f700 1 -- 172.21.15.89:0/789429623 wait complete. 2023-09-30T22:22:35.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:35.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.361+0000 7f6441b9f700 1 Processor -- start 2023-09-30T22:22:35.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.361+0000 7f6441b9f700 1 -- start start 2023-09-30T22:22:35.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f6441b9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c09d630 0x7f643c1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f6441b9f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c0a0a30 0x7f643c1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f6441b9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f643c133930 0x7f643c1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f6441b9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0a0410 con 0x7f643c133930 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f6441b9f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c120630 con 0x7f643c09d630 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f6441b9f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0a4030 con 0x7f643c0a0a30 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c0a0a30 0x7f643c1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c0a0a30 0x7f643c1c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32978/0 (socket says 172.21.15.89:32978) 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643affd700 1 -- 172.21.15.89:0/1109391422 learned_addr learned my addr 172.21.15.89:0/1109391422 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643b7fe700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c09d630 0x7f643c1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643bfff700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f643c133930 0x7f643c1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643b7fe700 1 -- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c0a0a30 msgr2=0x7f643c1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643b7fe700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c0a0a30 0x7f643c1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643b7fe700 1 -- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f643c133930 msgr2=0x7f643c1c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643b7fe700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f643c133930 0x7f643c1c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643b7fe700 1 -- 172.21.15.89:0/1109391422 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300094a0 con 0x7f643c09d630 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.362+0000 7f643affd700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c0a0a30 0x7f643c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.363+0000 7f643b7fe700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c09d630 0x7f643c1c7e80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f642c00e900 tx=0x7f642c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.363+0000 7f6423fff700 1 -- 172.21.15.89:0/1109391422 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c013680 con 0x7f643c09d630 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.363+0000 7f6423fff700 1 -- 172.21.15.89:0/1109391422 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c0137e0 con 0x7f643c09d630 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.363+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6434009b30 con 0x7f643c09d630 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.363+0000 7f6423fff700 1 -- 172.21.15.89:0/1109391422 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c013b50 con 0x7f643c09d630 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.363+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c120ca0 con 0x7f643c09d630 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.364+0000 7f6423fff700 1 -- 172.21.15.89:0/1109391422 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f642c013cb0 con 0x7f643c09d630 2023-09-30T22:22:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.364+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c0747f0 con 0x7f643c09d630 2023-09-30T22:22:35.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.364+0000 7f6423fff700 1 -- 172.21.15.89:0/1109391422 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f642c03f840 con 0x7f643c09d630 2023-09-30T22:22:35.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.368+0000 7f6423fff700 1 -- 172.21.15.89:0/1109391422 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f642c01b7c0 con 0x7f643c09d630 2023-09-30T22:22:35.418 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:35.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.481+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa8340a28e0 con 0x7fa8340a1340 2023-09-30T22:22:35.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.481+0000 7fa82affd700 1 -- 172.21.15.89:0/498740777 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa824040e60 con 0x7fa8340a1340 2023-09-30T22:22:35.481 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a1340 msgr2=0x7fa83411ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 --2- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a1340 0x7fa83411ec90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa824016000 tx=0x7fa824004750 comp rx=0 tx=0).stop 2023-09-30T22:22:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 shutdown_connections 2023-09-30T22:22:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 --2- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133fa0 0x7fa83411f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 --2- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a0a30 0x7fa83411e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 --2- 172.21.15.89:0/498740777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a1340 0x7fa83411ec90 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 >> 172.21.15.89:0/498740777 conn(0x7fa8340a6550 msgr2=0x7fa8340a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:35.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 shutdown_connections 2023-09-30T22:22:35.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.485+0000 7fa83c7f3700 1 -- 172.21.15.89:0/498740777 wait complete. 2023-09-30T22:22:35.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.498+0000 7f53fca19700 1 Processor -- start 2023-09-30T22:22:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.498+0000 7f53fca19700 1 -- start start 2023-09-30T22:22:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.498+0000 7f53fca19700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130540 0x7f53f8130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.498+0000 7f53fca19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53f806fcf0 0x7f53f8138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53fca19700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f80716a0 con 0x7f53f8130e50 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53fca19700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f809a390 con 0x7f53f806fcf0 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53fca19700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f8071290 con 0x7f53f8130540 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130540 0x7f53f8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53effff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f53f8130e50 0x7f53f806f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47948/0 (socket says 172.21.15.89:47948) 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53effff700 1 -- 172.21.15.89:0/4151748459 learned_addr learned my addr 172.21.15.89:0/4151748459 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f6d9d700 1 --2- 172.21.15.89:0/4151748459 >> v2:172.21.15.89:3300/0 conn(0x7f53f806fcf0 0x7f53f8138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f5d9b700 1 -- 172.21.15.89:0/4151748459 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203859918 0 0) 0x7f53f80716a0 con 0x7f53f8130e50 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f5d9b700 1 -- 172.21.15.89:0/4151748459 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e40035b0 con 0x7f53f8130e50 2023-09-30T22:22:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f659c700 1 -- 172.21.15.89:0/4151748459 >> v2:172.21.15.89:3300/0 conn(0x7f53f806fcf0 msgr2=0x7f53f8138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f659c700 1 --2- 172.21.15.89:0/4151748459 >> v2:172.21.15.89:3300/0 conn(0x7f53f806fcf0 0x7f53f8138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f659c700 1 -- 172.21.15.89:0/4151748459 >> v1:172.21.15.89:6789/0 conn(0x7f53f8130e50 legacy=0x7f53f806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f659c700 1 -- 172.21.15.89:0/4151748459 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8130340 con 0x7f53f8130540 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f6d9d700 1 --2- 172.21.15.89:0/4151748459 >> v2:172.21.15.89:3300/0 conn(0x7f53f806fcf0 0x7f53f8138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f659c700 1 --2- 172.21.15.89:0/4151748459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130540 0x7f53f8130910 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009870 tx=0x7f53e8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=494745f92769f6bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f5d9b700 1 -- 172.21.15.89:0/4151748459 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8011830 con 0x7f53f8130540 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f5d9b700 1 -- 172.21.15.89:0/4151748459 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e8011990 con 0x7f53f8130540 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.499+0000 7f53f5d9b700 1 -- 172.21.15.89:0/4151748459 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8011d20 con 0x7f53f8130540 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.500+0000 7f53fca19700 1 -- 172.21.15.89:0/4151748459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130540 msgr2=0x7f53f8130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.500+0000 7f53fca19700 1 --2- 172.21.15.89:0/4151748459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130540 0x7f53f8130910 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009870 tx=0x7f53e8009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.500+0000 7f53fca19700 1 -- 172.21.15.89:0/4151748459 shutdown_connections 2023-09-30T22:22:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.500+0000 7f53fca19700 1 --2- 172.21.15.89:0/4151748459 >> v2:172.21.15.89:3300/0 conn(0x7f53f806fcf0 0x7f53f8138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.500+0000 7f53fca19700 1 --2- 172.21.15.89:0/4151748459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130540 0x7f53f8130910 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.500+0000 7f53fca19700 1 -- 172.21.15.89:0/4151748459 >> 172.21.15.89:0/4151748459 conn(0x7f53f80a2aa0 msgr2=0x7f53f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.500+0000 7f53fca19700 1 -- 172.21.15.89:0/4151748459 shutdown_connections 2023-09-30T22:22:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.500+0000 7f53fca19700 1 -- 172.21.15.89:0/4151748459 wait complete. 2023-09-30T22:22:35.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.501+0000 7f53fca19700 1 Processor -- start 2023-09-30T22:22:35.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.501+0000 7f53fca19700 1 -- start start 2023-09-30T22:22:35.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53fca19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f806fcf0 0x7f53f811a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53fca19700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f8130540 0x7f53f811aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53fca19700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130e50 0x7f53f811af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53fca19700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f811da80 con 0x7f53f806fcf0 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53fca19700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80a0a50 con 0x7f53f8130e50 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53fca19700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f811d3d0 con 0x7f53f8130540 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53effff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f8130540 0x7f53f811aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53f6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130e50 0x7f53f811af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53f6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130e50 0x7f53f811af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39760/0 (socket says 172.21.15.89:39760) 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53effff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f8130540 0x7f53f811aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:32988/0 (socket says 172.21.15.89:32988) 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53f6d9d700 1 -- 172.21.15.89:0/3701766085 learned_addr learned my addr 172.21.15.89:0/3701766085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53f659c700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f806fcf0 0x7f53f811a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53effff700 1 -- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130e50 msgr2=0x7f53f811af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53effff700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130e50 0x7f53f811af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53effff700 1 -- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f806fcf0 msgr2=0x7f53f811a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53effff700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f806fcf0 0x7f53f811a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53effff700 1 -- 172.21.15.89:0/3701766085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0009a70 con 0x7f53f8130540 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53f659c700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f806fcf0 0x7f53f811a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53f6d9d700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130e50 0x7f53f811af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53effff700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f8130540 0x7f53f811aa40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c460 tx=0x7f53e000c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.502+0000 7f53eeffd700 1 -- 172.21.15.89:0/3701766085 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0005d70 con 0x7f53f8130540 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.503+0000 7f53eeffd700 1 -- 172.21.15.89:0/3701766085 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e00070d0 con 0x7f53f8130540 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.503+0000 7f53eeffd700 1 -- 172.21.15.89:0/3701766085 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0007440 con 0x7f53f8130540 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.503+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e80094a0 con 0x7f53f8130540 2023-09-30T22:22:35.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.503+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f811cf10 con 0x7f53f8130540 2023-09-30T22:22:35.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.504+0000 7f53eeffd700 1 -- 172.21.15.89:0/3701766085 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53e00075a0 con 0x7f53f8130540 2023-09-30T22:22:35.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.504+0000 7f53eeffd700 1 -- 172.21.15.89:0/3701766085 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f53e003e700 con 0x7f53f8130540 2023-09-30T22:22:35.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.504+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f809cd40 con 0x7f53f8130540 2023-09-30T22:22:35.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.508+0000 7f53eeffd700 1 -- 172.21.15.89:0/3701766085 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53e0017020 con 0x7f53f8130540 2023-09-30T22:22:35.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.548+0000 7f6ea2bf8700 1 Processor -- start 2023-09-30T22:22:35.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.548+0000 7f6ea2bf8700 1 -- start start 2023-09-30T22:22:35.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea2bf8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e9c131630 0x7f6e9c131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea2bf8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131f40 0x7f6e9c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea2bf8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c071a00 con 0x7f6e9c06e9d0 2023-09-30T22:22:35.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea2bf8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c131430 con 0x7f6e9c131630 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea2bf8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c09a510 con 0x7f6e9c131f40 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea0994700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e9c131630 0x7f6e9c131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea0994700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e9c131630 0x7f6e9c131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48678/0 (socket says 172.21.15.89:48678) 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea0994700 1 -- 172.21.15.89:0/4169322738 learned_addr learned my addr 172.21.15.89:0/4169322738 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6e9bfff700 1 --2- 172.21.15.89:0/4169322738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131f40 0x7f6e9c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea0994700 1 -- 172.21.15.89:0/4169322738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131f40 msgr2=0x7f6e9c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea0994700 1 --2- 172.21.15.89:0/4169322738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131f40 0x7f6e9c06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea0994700 1 -- 172.21.15.89:0/4169322738 >> v1:172.21.15.89:6789/0 conn(0x7f6e9c06e9d0 legacy=0x7f6e9c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea0994700 1 -- 172.21.15.89:0/4169322738 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0721c0 con 0x7f6e9c131630 2023-09-30T22:22:35.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6ea0994700 1 --2- 172.21.15.89:0/4169322738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131630 0x7f6e9c131a00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6e900098f0 tx=0x7f6e90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578b81b631b542a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:35.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.549+0000 7f6e9b7fe700 1 -- 172.21.15.89:0/4169322738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90011830 con 0x7f6e9c131630 2023-09-30T22:22:35.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6e9b7fe700 1 -- 172.21.15.89:0/4169322738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e90011990 con 0x7f6e9c131630 2023-09-30T22:22:35.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6e9b7fe700 1 -- 172.21.15.89:0/4169322738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90011d20 con 0x7f6e9c131630 2023-09-30T22:22:35.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/4169322738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131630 msgr2=0x7f6e9c131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6ea2bf8700 1 --2- 172.21.15.89:0/4169322738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131630 0x7f6e9c131a00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6e900098f0 tx=0x7f6e90009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:35.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/4169322738 shutdown_connections 2023-09-30T22:22:35.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6ea2bf8700 1 --2- 172.21.15.89:0/4169322738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131630 0x7f6e9c131a00 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6ea2bf8700 1 --2- 172.21.15.89:0/4169322738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131f40 0x7f6e9c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/4169322738 >> 172.21.15.89:0/4169322738 conn(0x7f6e9c0a2aa0 msgr2=0x7f6e9c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:35.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/4169322738 shutdown_connections 2023-09-30T22:22:35.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.550+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/4169322738 wait complete. 2023-09-30T22:22:35.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.551+0000 7f6ea2bf8700 1 Processor -- start 2023-09-30T22:22:35.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.551+0000 7f6ea2bf8700 1 -- start start 2023-09-30T22:22:35.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea2bf8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e9c06e9d0 0x7f6e9c11b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea2bf8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131630 0x7f6e9c11bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea2bf8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131f40 0x7f6e9c11c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:35.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea2bf8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c11f2d0 con 0x7f6e9c131f40 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea2bf8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c11d9e0 con 0x7f6e9c06e9d0 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea2bf8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c11e6a0 con 0x7f6e9c131630 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea1195700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131f40 0x7f6e9c11c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6e9bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131630 0x7f6e9c11bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea1195700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131f40 0x7f6e9c11c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48694/0 (socket says 172.21.15.89:48694) 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea1195700 1 -- 172.21.15.89:0/1057272458 learned_addr learned my addr 172.21.15.89:0/1057272458 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea0994700 1 --2- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e9c06e9d0 0x7f6e9c11b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.552+0000 7f6ea1195700 1 -- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131630 msgr2=0x7f6e9c11bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6ea1195700 1 --2- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131630 0x7f6e9c11bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6ea1195700 1 -- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e9c06e9d0 msgr2=0x7f6e9c11b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6ea1195700 1 --2- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e9c06e9d0 0x7f6e9c11b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6ea1195700 1 -- 172.21.15.89:0/1057272458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e90009550 con 0x7f6e9c131f40 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6ea1195700 1 --2- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131f40 0x7f6e9c11c2c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400cc50 tx=0x7f6e9400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6e997fa700 1 -- 172.21.15.89:0/1057272458 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94003c00 con 0x7f6e9c131f40 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6e997fa700 1 -- 172.21.15.89:0/1057272458 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e94003d60 con 0x7f6e9c131f40 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9c11ea60 con 0x7f6e9c131f40 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6e997fa700 1 -- 172.21.15.89:0/1057272458 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400aab0 con 0x7f6e9c131f40 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.553+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c0a1b10 con 0x7f6e9c131f40 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.554+0000 7f6e997fa700 1 -- 172.21.15.89:0/1057272458 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e9400ac10 con 0x7f6e9c131f40 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.554+0000 7f6e997fa700 1 -- 172.21.15.89:0/1057272458 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6e9403f640 con 0x7f6e9c131f40 2023-09-30T22:22:35.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.554+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9c074ea0 con 0x7f6e9c131f40 2023-09-30T22:22:35.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.559+0000 7f6e997fa700 1 -- 172.21.15.89:0/1057272458 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e940198f0 con 0x7f6e9c131f40 2023-09-30T22:22:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.571+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f643c121eb0 con 0x7f643c09d630 2023-09-30T22:22:35.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.571+0000 7f6423fff700 1 -- 172.21.15.89:0/1109391422 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f642c017030 con 0x7f643c09d630 2023-09-30T22:22:35.571 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c09d630 msgr2=0x7f643c1c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c09d630 0x7f643c1c7e80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f642c00e900 tx=0x7f642c00ec10 comp rx=0 tx=0).stop 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 shutdown_connections 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f643c133930 0x7f643c1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f643c09d630 0x7f643c1c7e80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 --2- 172.21.15.89:0/1109391422 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f643c0a0a30 0x7f643c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 >> 172.21.15.89:0/1109391422 conn(0x7f643c0a3050 msgr2=0x7f643c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 shutdown_connections 2023-09-30T22:22:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.574+0000 7f6441b9f700 1 -- 172.21.15.89:0/1109391422 wait complete. 2023-09-30T22:22:35.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:35.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.712+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53f8074ea0 con 0x7f53f8130540 2023-09-30T22:22:35.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.712+0000 7f53eeffd700 1 -- 172.21.15.89:0/3701766085 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53e0018af0 con 0x7f53f8130540 2023-09-30T22:22:35.713 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:35.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.714+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f8130540 msgr2=0x7f53f811aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.714+0000 7f53fca19700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f8130540 0x7f53f811aa40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c460 tx=0x7f53e000c770 comp rx=0 tx=0).stop 2023-09-30T22:22:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.715+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 shutdown_connections 2023-09-30T22:22:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.715+0000 7f53fca19700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53f8130e50 0x7f53f811af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.715+0000 7f53fca19700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f806fcf0 0x7f53f811a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.715+0000 7f53fca19700 1 --2- 172.21.15.89:0/3701766085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53f8130540 0x7f53f811aa40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.715+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 >> 172.21.15.89:0/3701766085 conn(0x7f53f80a2aa0 msgr2=0x7f53f812d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.715+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 shutdown_connections 2023-09-30T22:22:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.715+0000 7f53fca19700 1 -- 172.21.15.89:0/3701766085 wait complete. 2023-09-30T22:22:35.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:35.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.749+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6e9c06b9e0 con 0x7f6e9c131f40 2023-09-30T22:22:35.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.750+0000 7f6e997fa700 1 -- 172.21.15.89:0/1057272458 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e94019300 con 0x7f6e9c131f40 2023-09-30T22:22:35.750 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:35.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131f40 msgr2=0x7f6e9c11c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:35.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 --2- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131f40 0x7f6e9c11c2c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400cc50 tx=0x7f6e9400cf60 comp rx=0 tx=0).stop 2023-09-30T22:22:35.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 shutdown_connections 2023-09-30T22:22:35.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 --2- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e9c131f40 0x7f6e9c11c2c0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 --2- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e9c06e9d0 0x7f6e9c11b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 --2- 172.21.15.89:0/1057272458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e9c131630 0x7f6e9c11bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:35.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 >> 172.21.15.89:0/1057272458 conn(0x7f6e9c0a2aa0 msgr2=0x7f6e9c1b8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:35.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 shutdown_connections 2023-09-30T22:22:35.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:35.752+0000 7f6ea2bf8700 1 -- 172.21.15.89:0/1057272458 wait complete. 2023-09-30T22:22:35.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:36.499 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:36.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:36.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bf4d5b700 1 Processor -- start 2023-09-30T22:22:36.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bf4d5b700 1 -- start start 2023-09-30T22:22:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bf4d5b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bf4d5b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bf4d5b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf009ece0 con 0x7f4bf006b080 2023-09-30T22:22:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bf4d5b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf009e760 con 0x7f4bf006a770 2023-09-30T22:22:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bf4d5b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf009e040 con 0x7f4bf0131d10 2023-09-30T22:22:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4beed9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bedd9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4bf006b080 0x7f4bf0131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47962/0 (socket says 172.21.15.89:47962) 2023-09-30T22:22:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bedd9b700 1 -- 172.21.15.89:0/2127595918 learned_addr learned my addr 172.21.15.89:0/2127595918 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.637+0000 7f4bee59c700 1 --2- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bed59a700 1 -- 172.21.15.89:0/2127595918 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669855416 0 0) 0x7f4bf009ece0 con 0x7f4bf006b080 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bee59c700 1 -- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 msgr2=0x7f4bf0139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bee59c700 1 --2- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bee59c700 1 -- 172.21.15.89:0/2127595918 >> v1:172.21.15.89:6789/0 conn(0x7f4bf006b080 legacy=0x7f4bf0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bee59c700 1 -- 172.21.15.89:0/2127595918 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf009f630 con 0x7f4bf006a770 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4beed9d700 1 --2- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bee59c700 1 --2- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009870 tx=0x7f4be0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34aba45e0eba567f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:36.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bed59a700 1 -- 172.21.15.89:0/2127595918 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0011830 con 0x7f4bf006a770 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bed59a700 1 -- 172.21.15.89:0/2127595918 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0011990 con 0x7f4bf006a770 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.638+0000 7f4bed59a700 1 -- 172.21.15.89:0/2127595918 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0011d20 con 0x7f4bf006a770 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.639+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 msgr2=0x7f4bf006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.639+0000 7f4bf4d5b700 1 --2- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009870 tx=0x7f4be0009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.639+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/2127595918 shutdown_connections 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.639+0000 7f4bf4d5b700 1 --2- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.639+0000 7f4bf4d5b700 1 --2- 172.21.15.89:0/2127595918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.639+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/2127595918 >> 172.21.15.89:0/2127595918 conn(0x7f4bf0066f50 msgr2=0x7f4bf009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.639+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/2127595918 shutdown_connections 2023-09-30T22:22:36.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.639+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/2127595918 wait complete. 2023-09-30T22:22:36.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bf4d5b700 1 Processor -- start 2023-09-30T22:22:36.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bf4d5b700 1 -- start start 2023-09-30T22:22:36.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bf4d5b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf01c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bf4d5b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf006b080 0x7f4bf01c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bf4d5b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf01c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bf4d5b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0131240 con 0x7f4bf006b080 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bf4d5b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf009c180 con 0x7f4bf006a770 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bf4d5b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0125b60 con 0x7f4bf0131d10 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.640+0000 7f4bedd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf006b080 0x7f4bf01c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bedd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf006b080 0x7f4bf01c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48710/0 (socket says 172.21.15.89:48710) 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bedd9b700 1 -- 172.21.15.89:0/1638943774 learned_addr learned my addr 172.21.15.89:0/1638943774 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bee59c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf01c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4beed9d700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf01c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bedd9b700 1 -- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 msgr2=0x7f4bf01c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bedd9b700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf01c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bedd9b700 1 -- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 msgr2=0x7f4bf01c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bedd9b700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf01c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bedd9b700 1 -- 172.21.15.89:0/1638943774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8009a70 con 0x7f4bf006b080 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bee59c700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf01c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bedd9b700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf006b080 0x7f4bf01c85d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4be4001eb0 tx=0x7f4be400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bdeffd700 1 -- 172.21.15.89:0/1638943774 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4004c80 con 0x7f4bf006b080 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bdeffd700 1 -- 172.21.15.89:0/1638943774 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4004de0 con 0x7f4bf006b080 2023-09-30T22:22:36.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bdeffd700 1 -- 172.21.15.89:0/1638943774 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4007310 con 0x7f4bf006b080 2023-09-30T22:22:36.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be00094a0 con 0x7f4bf006b080 2023-09-30T22:22:36.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.641+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf0126170 con 0x7f4bf006b080 2023-09-30T22:22:36.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.642+0000 7f4bdeffd700 1 -- 172.21.15.89:0/1638943774 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4be4007470 con 0x7f4bf006b080 2023-09-30T22:22:36.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.642+0000 7f4bdeffd700 1 -- 172.21.15.89:0/1638943774 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4be403f250 con 0x7f4bf006b080 2023-09-30T22:22:36.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.642+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0074ea0 con 0x7f4bf006b080 2023-09-30T22:22:36.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.647+0000 7f4bdeffd700 1 -- 172.21.15.89:0/1638943774 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4be40123b0 con 0x7f4bf006b080 2023-09-30T22:22:36.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:36.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.735+0000 7fcc81f8a700 1 Processor -- start 2023-09-30T22:22:36.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.735+0000 7fcc81f8a700 1 -- start start 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc81f8a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc7c134630 0x7fcc7c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc81f8a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c136f50 0x7fcc7c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc81f8a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c09b4f0 con 0x7fcc7c131810 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc81f8a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c1313c0 con 0x7fcc7c134630 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc81f8a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c06e8c0 con 0x7fcc7c136f50 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc7bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c136f50 0x7fcc7c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc7affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc7c134630 0x7fcc7c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc7affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc7c134630 0x7fcc7c136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48716/0 (socket says 172.21.15.89:48716) 2023-09-30T22:22:36.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc7bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c136f50 0x7fcc7c139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39798/0 (socket says 172.21.15.89:39798) 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc7affd700 1 -- 172.21.15.89:0/1526698415 learned_addr learned my addr 172.21.15.89:0/1526698415 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc7b7fe700 1 --1- 172.21.15.89:0/1526698415 >> v1:172.21.15.89:6789/0 conn(0x7fcc7c131810 0x7fcc7c131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47968/0 (socket says 172.21.15.89:47968) 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc7affd700 1 -- 172.21.15.89:0/1526698415 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c136f50 msgr2=0x7fcc7c139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.736+0000 7fcc7affd700 1 --2- 172.21.15.89:0/1526698415 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c136f50 0x7fcc7c139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.737+0000 7fcc7affd700 1 -- 172.21.15.89:0/1526698415 >> v1:172.21.15.89:6789/0 conn(0x7fcc7c131810 legacy=0x7fcc7c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.737+0000 7fcc7affd700 1 -- 172.21.15.89:0/1526698415 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c071d20 con 0x7fcc7c134630 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.737+0000 7fcc7bfff700 1 --2- 172.21.15.89:0/1526698415 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c136f50 0x7fcc7c139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.737+0000 7fcc7affd700 1 --2- 172.21.15.89:0/1526698415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c134630 0x7fcc7c136a10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcc700098f0 tx=0x7fcc70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bae18b89703ed3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.737+0000 7fcc7a7fc700 1 -- 172.21.15.89:0/1526698415 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70011830 con 0x7fcc7c134630 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.737+0000 7fcc7a7fc700 1 -- 172.21.15.89:0/1526698415 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc70011990 con 0x7fcc7c134630 2023-09-30T22:22:36.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.737+0000 7fcc7a7fc700 1 -- 172.21.15.89:0/1526698415 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70011d20 con 0x7fcc7c134630 2023-09-30T22:22:36.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.738+0000 7fcc81f8a700 1 -- 172.21.15.89:0/1526698415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c134630 msgr2=0x7fcc7c136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.738+0000 7fcc81f8a700 1 --2- 172.21.15.89:0/1526698415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c134630 0x7fcc7c136a10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcc700098f0 tx=0x7fcc70009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:36.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.738+0000 7fcc81f8a700 1 -- 172.21.15.89:0/1526698415 shutdown_connections 2023-09-30T22:22:36.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.738+0000 7fcc81f8a700 1 --2- 172.21.15.89:0/1526698415 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c136f50 0x7fcc7c139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.738+0000 7fcc81f8a700 1 --2- 172.21.15.89:0/1526698415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c134630 0x7fcc7c136a10 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.738+0000 7fcc81f8a700 1 -- 172.21.15.89:0/1526698415 >> 172.21.15.89:0/1526698415 conn(0x7fcc7c0a2aa0 msgr2=0x7fcc7c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.738+0000 7fcc81f8a700 1 -- 172.21.15.89:0/1526698415 shutdown_connections 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.738+0000 7fcc81f8a700 1 -- 172.21.15.89:0/1526698415 wait complete. 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc81f8a700 1 Processor -- start 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc81f8a700 1 -- start start 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc81f8a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c131810 0x7fcc7c1196a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc81f8a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7c134630 0x7fcc7c119be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc81f8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c136f50 0x7fcc7c1c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc81f8a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c119380 con 0x7fcc7c136f50 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc81f8a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c12ed10 con 0x7fcc7c131810 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc81f8a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c11c350 con 0x7fcc7c134630 2023-09-30T22:22:36.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc7b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c131810 0x7fcc7c1196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc7b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c131810 0x7fcc7c1196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39800/0 (socket says 172.21.15.89:39800) 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc7b7fe700 1 -- 172.21.15.89:0/2110321311 learned_addr learned my addr 172.21.15.89:0/2110321311 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.739+0000 7fcc7affd700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7c134630 0x7fcc7c119be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc7bfff700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c136f50 0x7fcc7c1c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc7b7fe700 1 -- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7c134630 msgr2=0x7fcc7c119be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc7b7fe700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7c134630 0x7fcc7c119be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc7b7fe700 1 -- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c136f50 msgr2=0x7fcc7c1c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc7b7fe700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c136f50 0x7fcc7c1c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc7b7fe700 1 -- 172.21.15.89:0/2110321311 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc74009a70 con 0x7fcc7c131810 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc7b7fe700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c131810 0x7fcc7c1196a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c530 tx=0x7fcc6c00c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc63fff700 1 -- 172.21.15.89:0/2110321311 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c005cb0 con 0x7fcc7c131810 2023-09-30T22:22:36.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc63fff700 1 -- 172.21.15.89:0/2110321311 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c005e10 con 0x7fcc7c131810 2023-09-30T22:22:36.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc70009550 con 0x7fcc7c131810 2023-09-30T22:22:36.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc63fff700 1 -- 172.21.15.89:0/2110321311 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0072d0 con 0x7fcc7c131810 2023-09-30T22:22:36.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.740+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c06eb00 con 0x7fcc7c131810 2023-09-30T22:22:36.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.741+0000 7fcc63fff700 1 -- 172.21.15.89:0/2110321311 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc6c007430 con 0x7fcc7c131810 2023-09-30T22:22:36.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.741+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c11d140 con 0x7fcc7c131810 2023-09-30T22:22:36.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.741+0000 7fcc63fff700 1 -- 172.21.15.89:0/2110321311 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcc6c03ea50 con 0x7fcc7c131810 2023-09-30T22:22:36.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.746+0000 7fcc63fff700 1 -- 172.21.15.89:0/2110321311 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc6c0076e0 con 0x7fcc7c131810 2023-09-30T22:22:36.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.856+0000 7fcc80706700 1 Processor -- start 2023-09-30T22:22:36.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.856+0000 7fcc80706700 1 -- start start 2023-09-30T22:22:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc80706700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc78138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc80706700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc78139170 0x7fcc7813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc80706700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7809b120 con 0x7fcc7809d6f0 2023-09-30T22:22:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc80706700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7806b460 con 0x7fcc78139170 2023-09-30T22:22:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc80706700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7809a7e0 con 0x7fcc7809e1d0 2023-09-30T22:22:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc7eca3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc78139170 0x7fcc7813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc7e4a2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcc7809d6f0 0x7fcc7809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47972/0 (socket says 172.21.15.89:47972) 2023-09-30T22:22:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc7e4a2700 1 -- 172.21.15.89:0/443442857 learned_addr learned my addr 172.21.15.89:0/443442857 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.857+0000 7fcc7dca1700 1 --2- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc78138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7eca3700 1 -- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 msgr2=0x7fcc78138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7eca3700 1 --2- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc78138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7eca3700 1 -- 172.21.15.89:0/443442857 >> v1:172.21.15.89:6789/0 conn(0x7fcc7809d6f0 legacy=0x7fcc7809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7d4a0700 1 -- 172.21.15.89:0/443442857 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608956244 0 0) 0x7fcc7809b120 con 0x7fcc7809d6f0 2023-09-30T22:22:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7eca3700 1 -- 172.21.15.89:0/443442857 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7809a500 con 0x7fcc78139170 2023-09-30T22:22:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7dca1700 1 --2- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc78138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7eca3700 1 --2- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc78139170 0x7fcc7813b550 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009d90 tx=0x7fcc7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcca296745dc910 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7d4a0700 1 -- 172.21.15.89:0/443442857 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7000e070 con 0x7fcc78139170 2023-09-30T22:22:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7d4a0700 1 -- 172.21.15.89:0/443442857 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7000bb20 con 0x7fcc78139170 2023-09-30T22:22:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.858+0000 7fcc7d4a0700 1 -- 172.21.15.89:0/443442857 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70012430 con 0x7fcc78139170 2023-09-30T22:22:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.859+0000 7fcc80706700 1 -- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc78139170 msgr2=0x7fcc7813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.859+0000 7fcc80706700 1 --2- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc78139170 0x7fcc7813b550 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009d90 tx=0x7fcc7000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.859+0000 7fcc80706700 1 -- 172.21.15.89:0/443442857 shutdown_connections 2023-09-30T22:22:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.859+0000 7fcc80706700 1 --2- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc78139170 0x7fcc7813b550 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.859+0000 7fcc80706700 1 --2- 172.21.15.89:0/443442857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc78138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.859+0000 7fcc80706700 1 -- 172.21.15.89:0/443442857 >> 172.21.15.89:0/443442857 conn(0x7fcc78066f50 msgr2=0x7fcc7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.859+0000 7fcc80706700 1 -- 172.21.15.89:0/443442857 shutdown_connections 2023-09-30T22:22:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.859+0000 7fcc80706700 1 -- 172.21.15.89:0/443442857 wait complete. 2023-09-30T22:22:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc80706700 1 Processor -- start 2023-09-30T22:22:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc80706700 1 -- start start 2023-09-30T22:22:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc80706700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7809d6f0 0x7fcc7811e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc80706700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc7811edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc80706700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc78139170 0x7fcc7811f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc80706700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7809c710 con 0x7fcc7809d6f0 2023-09-30T22:22:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc80706700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78134740 con 0x7fcc78139170 2023-09-30T22:22:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc80706700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78120570 con 0x7fcc7809e1d0 2023-09-30T22:22:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc7dca1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc7811edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc7dca1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc7811edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33048/0 (socket says 172.21.15.89:33048) 2023-09-30T22:22:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.861+0000 7fcc7dca1700 1 -- 172.21.15.89:0/558553068 learned_addr learned my addr 172.21.15.89:0/558553068 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc7eca3700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc78139170 0x7fcc7811f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc7e4a2700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7809d6f0 0x7fcc7811e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc7dca1700 1 -- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc78139170 msgr2=0x7fcc7811f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc7dca1700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc78139170 0x7fcc7811f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc7dca1700 1 -- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7809d6f0 msgr2=0x7fcc7811e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc7dca1700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7809d6f0 0x7fcc7811e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc7dca1700 1 -- 172.21.15.89:0/558553068 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680094a0 con 0x7fcc7809e1d0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc7dca1700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc7811edd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800ebe0 tx=0x7fcc6800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc6effd700 1 -- 172.21.15.89:0/558553068 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800c850 con 0x7fcc7809e1d0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc6effd700 1 -- 172.21.15.89:0/558553068 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6800c9b0 con 0x7fcc7809e1d0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc6effd700 1 -- 172.21.15.89:0/558553068 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800cd40 con 0x7fcc7809e1d0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc70009a70 con 0x7fcc7809e1d0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.862+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc78122b00 con 0x7fcc7809e1d0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.863+0000 7fcc6effd700 1 -- 172.21.15.89:0/558553068 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc68013470 con 0x7fcc7809e1d0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.864+0000 7fcc6effd700 1 -- 172.21.15.89:0/558553068 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcc6803f770 con 0x7fcc7809e1d0 2023-09-30T22:22:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.864+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc78074ea0 con 0x7fcc7809e1d0 2023-09-30T22:22:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.868+0000 7fcc6effd700 1 -- 172.21.15.89:0/558553068 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc68021020 con 0x7fcc7809e1d0 2023-09-30T22:22:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.869+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4bf0124e60 con 0x7f4bf006b080 2023-09-30T22:22:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.869+0000 7f4bdeffd700 1 -- 172.21.15.89:0/1638943774 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4be4011360 con 0x7f4bf006b080 2023-09-30T22:22:36.871 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.871+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf006b080 msgr2=0x7f4bf01c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.871+0000 7f4bf4d5b700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf006b080 0x7f4bf01c85d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4be4001eb0 tx=0x7f4be400c800 comp rx=0 tx=0).stop 2023-09-30T22:22:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.872+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 shutdown_connections 2023-09-30T22:22:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.872+0000 7f4bf4d5b700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4bf0131d10 0x7f4bf01c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.872+0000 7f4bf4d5b700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4bf006a770 0x7f4bf01c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.872+0000 7f4bf4d5b700 1 --2- 172.21.15.89:0/1638943774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bf006b080 0x7f4bf01c85d0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.872+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 >> 172.21.15.89:0/1638943774 conn(0x7f4bf0066f50 msgr2=0x7f4bf00700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.872+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 shutdown_connections 2023-09-30T22:22:36.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.872+0000 7f4bf4d5b700 1 -- 172.21.15.89:0/1638943774 wait complete. 2023-09-30T22:22:36.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:36.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.893+0000 7f36989db700 1 Processor -- start 2023-09-30T22:22:36.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.893+0000 7f36989db700 1 -- start start 2023-09-30T22:22:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.894+0000 7f36989db700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409d6f0 0x7f369409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.894+0000 7f36989db700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f369409e000 0x7f3694130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.894+0000 7f36989db700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369409f130 con 0x7f369409e410 2023-09-30T22:22:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.894+0000 7f36989db700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369409f520 con 0x7f369409e000 2023-09-30T22:22:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.894+0000 7f36989db700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369409e930 con 0x7f369409d6f0 2023-09-30T22:22:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f369259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409d6f0 0x7f369409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f3691d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f369409e000 0x7f3694130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f369259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409d6f0 0x7f369409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39812/0 (socket says 172.21.15.89:39812) 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f3691d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f369409e000 0x7f3694130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48758/0 (socket says 172.21.15.89:48758) 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f369259c700 1 -- 172.21.15.89:0/1630610734 learned_addr learned my addr 172.21.15.89:0/1630610734 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f3691d9b700 1 -- 172.21.15.89:0/1630610734 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409d6f0 msgr2=0x7f369409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f3691d9b700 1 --2- 172.21.15.89:0/1630610734 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409d6f0 0x7f369409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f3691d9b700 1 -- 172.21.15.89:0/1630610734 >> v1:172.21.15.89:6789/0 conn(0x7f369409e410 legacy=0x7f3694130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f3691d9b700 1 -- 172.21.15.89:0/1630610734 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369409b2d0 con 0x7f369409e000 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f369259c700 1 --2- 172.21.15.89:0/1630610734 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409d6f0 0x7f369409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f3691d9b700 1 --2- 172.21.15.89:0/1630610734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409e000 0x7f3694130660 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3688009840 tx=0x7f3688009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcec205eb109bbe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f369159a700 1 -- 172.21.15.89:0/1630610734 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688011830 con 0x7f369409e000 2023-09-30T22:22:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f369159a700 1 -- 172.21.15.89:0/1630610734 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3688011990 con 0x7f369409e000 2023-09-30T22:22:36.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.895+0000 7f369159a700 1 -- 172.21.15.89:0/1630610734 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688011d20 con 0x7f369409e000 2023-09-30T22:22:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.896+0000 7f36989db700 1 -- 172.21.15.89:0/1630610734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409e000 msgr2=0x7f3694130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.896+0000 7f36989db700 1 --2- 172.21.15.89:0/1630610734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409e000 0x7f3694130660 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3688009840 tx=0x7f3688009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.896+0000 7f36989db700 1 -- 172.21.15.89:0/1630610734 shutdown_connections 2023-09-30T22:22:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.896+0000 7f36989db700 1 --2- 172.21.15.89:0/1630610734 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409d6f0 0x7f369409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.896+0000 7f36989db700 1 --2- 172.21.15.89:0/1630610734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409e000 0x7f3694130660 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.896+0000 7f36989db700 1 -- 172.21.15.89:0/1630610734 >> 172.21.15.89:0/1630610734 conn(0x7f3694066f50 msgr2=0x7f369409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.896+0000 7f36989db700 1 -- 172.21.15.89:0/1630610734 shutdown_connections 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.896+0000 7f36989db700 1 -- 172.21.15.89:0/1630610734 wait complete. 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.897+0000 7f36989db700 1 Processor -- start 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.897+0000 7f36989db700 1 -- start start 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f36989db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409d6f0 0x7f3694138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f36989db700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f369409e000 0x7f3694138e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f36989db700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409e410 0x7f36941393b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f36989db700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694131100 con 0x7f369409d6f0 2023-09-30T22:22:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f36989db700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3694130130 con 0x7f369409e410 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f36989db700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3694072670 con 0x7f369409e000 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f369259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409d6f0 0x7f3694138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f3691d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f369409e000 0x7f3694138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f369259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409d6f0 0x7f3694138930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48772/0 (socket says 172.21.15.89:48772) 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f369259c700 1 -- 172.21.15.89:0/559898412 learned_addr learned my addr 172.21.15.89:0/559898412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f3692d9d700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409e410 0x7f36941393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f3691d9b700 1 -- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409e410 msgr2=0x7f36941393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f3691d9b700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409e410 0x7f36941393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f3691d9b700 1 -- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409d6f0 msgr2=0x7f3694138930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f3691d9b700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409d6f0 0x7f3694138930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f3691d9b700 1 -- 172.21.15.89:0/559898412 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684009550 con 0x7f369409e000 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f369259c700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409d6f0 0x7f3694138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.898+0000 7f3691d9b700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f369409e000 0x7f3694138e70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3688016000 tx=0x7f3688004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.899+0000 7f3682ffd700 1 -- 172.21.15.89:0/559898412 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880036a0 con 0x7f369409e000 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.899+0000 7f3682ffd700 1 -- 172.21.15.89:0/559898412 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368801f070 con 0x7f369409e000 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.899+0000 7f36989db700 1 -- 172.21.15.89:0/559898412 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36880094a0 con 0x7f369409e000 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.899+0000 7f3682ffd700 1 -- 172.21.15.89:0/559898412 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688003a60 con 0x7f369409e000 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.899+0000 7f36989db700 1 -- 172.21.15.89:0/559898412 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369412eef0 con 0x7f369409e000 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.899+0000 7f3682ffd700 1 -- 172.21.15.89:0/559898412 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3688003bc0 con 0x7f369409e000 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.899+0000 7f3682ffd700 1 -- 172.21.15.89:0/559898412 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3688021070 con 0x7f369409e000 2023-09-30T22:22:36.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.900+0000 7f3680ff9700 1 -- 172.21.15.89:0/559898412 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c0052e0 con 0x7f369409e000 2023-09-30T22:22:36.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.904+0000 7f3682ffd700 1 -- 172.21.15.89:0/559898412 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3688015020 con 0x7f369409e000 2023-09-30T22:22:36.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.958+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcc7c11c830 con 0x7fcc7c131810 2023-09-30T22:22:36.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.959+0000 7fcc63fff700 1 -- 172.21.15.89:0/2110321311 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc6c017030 con 0x7fcc7c131810 2023-09-30T22:22:36.959 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:36.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c131810 msgr2=0x7fcc7c1196a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:36.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c131810 0x7fcc7c1196a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c530 tx=0x7fcc6c00c840 comp rx=0 tx=0).stop 2023-09-30T22:22:36.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 shutdown_connections 2023-09-30T22:22:36.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7c136f50 0x7fcc7c1c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc7c131810 0x7fcc7c1196a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 --2- 172.21.15.89:0/2110321311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7c134630 0x7fcc7c119be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:36.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 >> 172.21.15.89:0/2110321311 conn(0x7fcc7c0a2aa0 msgr2=0x7fcc7c12f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:36.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 shutdown_connections 2023-09-30T22:22:36.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:36.961+0000 7fcc81f8a700 1 -- 172.21.15.89:0/2110321311 wait complete. 2023-09-30T22:22:36.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:37.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.078+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcc78121b00 con 0x7fcc7809e1d0 2023-09-30T22:22:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.078+0000 7fcc6effd700 1 -- 172.21.15.89:0/558553068 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc6800cea0 con 0x7fcc7809e1d0 2023-09-30T22:22:37.078 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 msgr2=0x7fcc7811edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc7811edd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800ebe0 tx=0x7fcc6800eef0 comp rx=0 tx=0).stop 2023-09-30T22:22:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 shutdown_connections 2023-09-30T22:22:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcc78139170 0x7fcc7811f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc7809d6f0 0x7fcc7811e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 --2- 172.21.15.89:0/558553068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcc7809e1d0 0x7fcc7811edd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:37.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 >> 172.21.15.89:0/558553068 conn(0x7fcc78066f50 msgr2=0x7fcc780a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:37.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 shutdown_connections 2023-09-30T22:22:37.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.080+0000 7fcc80706700 1 -- 172.21.15.89:0/558553068 wait complete. 2023-09-30T22:22:37.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:37.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.099+0000 7f3680ff9700 1 -- 172.21.15.89:0/559898412 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f366c005590 con 0x7f369409e000 2023-09-30T22:22:37.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.099+0000 7f3682ffd700 1 -- 172.21.15.89:0/559898412 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3688026480 con 0x7f369409e000 2023-09-30T22:22:37.099 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:37.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.101+0000 7f3680ff9700 1 -- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f369409e000 msgr2=0x7f3694138e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:37.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.101+0000 7f3680ff9700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f369409e000 0x7f3694138e70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3688016000 tx=0x7f3688004750 comp rx=0 tx=0).stop 2023-09-30T22:22:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.101+0000 7f3680ff9700 1 -- 172.21.15.89:0/559898412 shutdown_connections 2023-09-30T22:22:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.101+0000 7f3680ff9700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f369409e410 0x7f36941393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.101+0000 7f3680ff9700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369409d6f0 0x7f3694138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.101+0000 7f3680ff9700 1 --2- 172.21.15.89:0/559898412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f369409e000 0x7f3694138e70 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.101+0000 7f3680ff9700 1 -- 172.21.15.89:0/559898412 >> 172.21.15.89:0/559898412 conn(0x7f3694066f50 msgr2=0x7f36940a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.102+0000 7f3680ff9700 1 -- 172.21.15.89:0/559898412 shutdown_connections 2023-09-30T22:22:37.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:37.102+0000 7f3680ff9700 1 -- 172.21.15.89:0/559898412 wait complete. 2023-09-30T22:22:37.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:37.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:37.974 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b1013f700 1 Processor -- start 2023-09-30T22:22:38.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b1013f700 1 -- start start 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b1013f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b081337f0 0x7f2b08138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b1013f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08138dc0 0x7f2b0813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b1013f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0806b990 con 0x7f2b08133420 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b1013f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0809b0e0 con 0x7f2b081337f0 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b1013f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08133c20 con 0x7f2b08138dc0 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b0d6da700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b081337f0 0x7f2b08138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b0e6dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08138dc0 0x7f2b0813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b0d6da700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b081337f0 0x7f2b08138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48774/0 (socket says 172.21.15.89:48774) 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b0e6dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08138dc0 0x7f2b0813b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39832/0 (socket says 172.21.15.89:39832) 2023-09-30T22:22:38.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.022+0000 7f2b0d6da700 1 -- 172.21.15.89:0/4005778753 learned_addr learned my addr 172.21.15.89:0/4005778753 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0e6dc700 1 -- 172.21.15.89:0/4005778753 >> v2:172.21.15.89:3300/0 conn(0x7f2b081337f0 msgr2=0x7f2b08138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0e6dc700 1 --2- 172.21.15.89:0/4005778753 >> v2:172.21.15.89:3300/0 conn(0x7f2b081337f0 0x7f2b08138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0e6dc700 1 -- 172.21.15.89:0/4005778753 >> v1:172.21.15.89:6789/0 conn(0x7f2b08133420 legacy=0x7f2b08133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0e6dc700 1 -- 172.21.15.89:0/4005778753 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080a0590 con 0x7f2b08138dc0 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0d6da700 1 --2- 172.21.15.89:0/4005778753 >> v2:172.21.15.89:3300/0 conn(0x7f2b081337f0 0x7f2b08138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0e6dc700 1 --2- 172.21.15.89:0/4005778753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08138dc0 0x7f2b0813b1a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009d90 tx=0x7f2b0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=432ee323e9ca1e45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0ced9700 1 -- 172.21.15.89:0/4005778753 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000e070 con 0x7f2b08138dc0 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0ced9700 1 -- 172.21.15.89:0/4005778753 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0000bb20 con 0x7f2b08138dc0 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.023+0000 7f2b0ced9700 1 -- 172.21.15.89:0/4005778753 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00012430 con 0x7f2b08138dc0 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.024+0000 7f2b1013f700 1 -- 172.21.15.89:0/4005778753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08138dc0 msgr2=0x7f2b0813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.024+0000 7f2b1013f700 1 --2- 172.21.15.89:0/4005778753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08138dc0 0x7f2b0813b1a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009d90 tx=0x7f2b0000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.024+0000 7f2b1013f700 1 -- 172.21.15.89:0/4005778753 shutdown_connections 2023-09-30T22:22:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.024+0000 7f2b1013f700 1 --2- 172.21.15.89:0/4005778753 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08138dc0 0x7f2b0813b1a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.024+0000 7f2b1013f700 1 --2- 172.21.15.89:0/4005778753 >> v2:172.21.15.89:3300/0 conn(0x7f2b081337f0 0x7f2b08138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.024+0000 7f2b1013f700 1 -- 172.21.15.89:0/4005778753 >> 172.21.15.89:0/4005778753 conn(0x7f2b080a4210 msgr2=0x7f2b080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.024+0000 7f2b1013f700 1 -- 172.21.15.89:0/4005778753 shutdown_connections 2023-09-30T22:22:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.024+0000 7f2b1013f700 1 -- 172.21.15.89:0/4005778753 wait complete. 2023-09-30T22:22:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b1013f700 1 Processor -- start 2023-09-30T22:22:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b1013f700 1 -- start start 2023-09-30T22:22:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b1013f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08133420 0x7f2b0812f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b1013f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b081337f0 0x7f2b0812f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b1013f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b08138dc0 0x7f2b0812fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b1013f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b081304f0 con 0x7f2b081337f0 2023-09-30T22:22:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b1013f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0809c510 con 0x7f2b08133420 2023-09-30T22:22:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b1013f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0809c720 con 0x7f2b08138dc0 2023-09-30T22:22:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b0dedb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08133420 0x7f2b0812f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b0dedb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08133420 0x7f2b0812f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39834/0 (socket says 172.21.15.89:39834) 2023-09-30T22:22:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b0d6da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b081337f0 0x7f2b0812f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b0dedb700 1 -- 172.21.15.89:0/2079845630 learned_addr learned my addr 172.21.15.89:0/2079845630 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.025+0000 7f2b0e6dc700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b08138dc0 0x7f2b0812fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b0dedb700 1 -- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b08138dc0 msgr2=0x7f2b0812fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b0dedb700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b08138dc0 0x7f2b0812fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b0dedb700 1 -- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b081337f0 msgr2=0x7f2b0812f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b0dedb700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b081337f0 0x7f2b0812f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b0dedb700 1 -- 172.21.15.89:0/2079845630 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af8009550 con 0x7f2b08133420 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b0d6da700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b081337f0 0x7f2b0812f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b0dedb700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08133420 0x7f2b0812f0b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c4e0 tx=0x7f2b0400c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2afe7fc700 1 -- 172.21.15.89:0/2079845630 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b040052d0 con 0x7f2b08133420 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2afe7fc700 1 -- 172.21.15.89:0/2079845630 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b04005430 con 0x7f2b08133420 2023-09-30T22:22:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2afe7fc700 1 -- 172.21.15.89:0/2079845630 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b040038c0 con 0x7f2b08133420 2023-09-30T22:22:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b00009a70 con 0x7f2b08133420 2023-09-30T22:22:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.026+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b081308e0 con 0x7f2b08133420 2023-09-30T22:22:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.027+0000 7f2afe7fc700 1 -- 172.21.15.89:0/2079845630 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b04003a20 con 0x7f2b08133420 2023-09-30T22:22:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.027+0000 7f2afe7fc700 1 -- 172.21.15.89:0/2079845630 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2b0403eb20 con 0x7f2b08133420 2023-09-30T22:22:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.027+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b08074ea0 con 0x7f2b08133420 2023-09-30T22:22:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.032+0000 7f2afe7fc700 1 -- 172.21.15.89:0/2079845630 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b04017070 con 0x7f2b08133420 2023-09-30T22:22:38.093 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.103+0000 7f851d3d1700 1 Processor -- start 2023-09-30T22:22:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.103+0000 7f851d3d1700 1 -- start start 2023-09-30T22:22:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f851d3d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518133540 0x7f8518135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f851d3d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8518135e60 0x7f8518138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f851d3d1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85180716a0 con 0x7f8518130540 2023-09-30T22:22:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f851d3d1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8518071290 con 0x7f8518135e60 2023-09-30T22:22:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f851d3d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8518130340 con 0x7f8518133540 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f8516ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8518130540 0x7f8518130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48018/0 (socket says 172.21.15.89:48018) 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f8516ffd700 1 -- 172.21.15.89:0/2511485052 learned_addr learned my addr 172.21.15.89:0/2511485052 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f85167fc700 1 --2- 172.21.15.89:0/2511485052 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518133540 0x7f8518135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f85177fe700 1 --2- 172.21.15.89:0/2511485052 >> v2:172.21.15.89:3300/0 conn(0x7f8518135e60 0x7f8518138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f85167fc700 1 -- 172.21.15.89:0/2511485052 >> v2:172.21.15.89:3300/0 conn(0x7f8518135e60 msgr2=0x7f8518138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f85167fc700 1 --2- 172.21.15.89:0/2511485052 >> v2:172.21.15.89:3300/0 conn(0x7f8518135e60 0x7f8518138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f85167fc700 1 -- 172.21.15.89:0/2511485052 >> v1:172.21.15.89:6789/0 conn(0x7f8518130540 legacy=0x7f8518130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f85167fc700 1 -- 172.21.15.89:0/2511485052 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8518070b80 con 0x7f8518133540 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.104+0000 7f85167fc700 1 --2- 172.21.15.89:0/2511485052 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518133540 0x7f8518135920 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f850c009840 tx=0x7f850c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c5c6fcce01bea6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f8515ffb700 1 -- 172.21.15.89:0/2511485052 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c004230 con 0x7f8518133540 2023-09-30T22:22:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f8515ffb700 1 -- 172.21.15.89:0/2511485052 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c004390 con 0x7f8518133540 2023-09-30T22:22:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f8515ffb700 1 -- 172.21.15.89:0/2511485052 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c0036a0 con 0x7f8518133540 2023-09-30T22:22:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f851d3d1700 1 -- 172.21.15.89:0/2511485052 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518133540 msgr2=0x7f8518135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f851d3d1700 1 --2- 172.21.15.89:0/2511485052 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518133540 0x7f8518135920 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f850c009840 tx=0x7f850c009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f851d3d1700 1 -- 172.21.15.89:0/2511485052 shutdown_connections 2023-09-30T22:22:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f851d3d1700 1 --2- 172.21.15.89:0/2511485052 >> v2:172.21.15.89:3300/0 conn(0x7f8518135e60 0x7f8518138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f851d3d1700 1 --2- 172.21.15.89:0/2511485052 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518133540 0x7f8518135920 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f851d3d1700 1 -- 172.21.15.89:0/2511485052 >> 172.21.15.89:0/2511485052 conn(0x7f85180a2aa0 msgr2=0x7f85180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f851d3d1700 1 -- 172.21.15.89:0/2511485052 shutdown_connections 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.105+0000 7f851d3d1700 1 -- 172.21.15.89:0/2511485052 wait complete. 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.106+0000 7f851d3d1700 1 Processor -- start 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.106+0000 7f851d3d1700 1 -- start start 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f851d3d1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518130540 0x7f8518134460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f851d3d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8518133540 0x7f85181349a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f851d3d1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8518135e60 0x7f85181c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f851d3d1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851812d8b0 con 0x7f8518133540 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f851d3d1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8518133ef0 con 0x7f8518135e60 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f851d3d1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f851811ea60 con 0x7f8518130540 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f85167fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8518133540 0x7f85181349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f85167fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8518133540 0x7f85181349a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48804/0 (socket says 172.21.15.89:48804) 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f85167fc700 1 -- 172.21.15.89:0/3945369656 learned_addr learned my addr 172.21.15.89:0/3945369656 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f8516ffd700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518130540 0x7f8518134460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f85177fe700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8518135e60 0x7f85181c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f8516ffd700 1 -- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8518135e60 msgr2=0x7f85181c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f8516ffd700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8518135e60 0x7f85181c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f8516ffd700 1 -- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8518133540 msgr2=0x7f85181349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f8516ffd700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8518133540 0x7f85181349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f8516ffd700 1 -- 172.21.15.89:0/3945369656 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0094a0 con 0x7f8518130540 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f85167fc700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8518133540 0x7f85181349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.107+0000 7f8516ffd700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518130540 0x7f8518134460 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f850800c780 tx=0x7f850800ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.108+0000 7f84ff7fe700 1 -- 172.21.15.89:0/3945369656 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508004c80 con 0x7f8518130540 2023-09-30T22:22:38.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.108+0000 7f84ff7fe700 1 -- 172.21.15.89:0/3945369656 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508004de0 con 0x7f8518130540 2023-09-30T22:22:38.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.108+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851811d3b0 con 0x7f8518130540 2023-09-30T22:22:38.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.108+0000 7f84ff7fe700 1 -- 172.21.15.89:0/3945369656 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508007310 con 0x7f8518130540 2023-09-30T22:22:38.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.108+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85180718e0 con 0x7f8518130540 2023-09-30T22:22:38.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.109+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8518074ea0 con 0x7f8518130540 2023-09-30T22:22:38.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.109+0000 7f84ff7fe700 1 -- 172.21.15.89:0/3945369656 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8508007470 con 0x7f8518130540 2023-09-30T22:22:38.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.109+0000 7f84ff7fe700 1 -- 172.21.15.89:0/3945369656 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f850803f370 con 0x7f8518130540 2023-09-30T22:22:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.114+0000 7f84ff7fe700 1 -- 172.21.15.89:0/3945369656 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8508007720 con 0x7f8518130540 2023-09-30T22:22:38.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f4276db6700 1 Processor -- start 2023-09-30T22:22:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f4276db6700 1 -- start start 2023-09-30T22:22:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f4276db6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270133540 0x7f4270135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f4276db6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4270135e60 0x7f4270138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f4276db6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42700716a0 con 0x7f4270130540 2023-09-30T22:22:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f4276db6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4270071290 con 0x7f4270135e60 2023-09-30T22:22:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f4276db6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4270130340 con 0x7f4270133540 2023-09-30T22:22:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f426ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270133540 0x7f4270135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.221+0000 7f4275353700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4270135e60 0x7f4270138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270133540 0x7f4270135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33086/0 (socket says 172.21.15.89:33086) 2023-09-30T22:22:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426ffff700 1 -- 172.21.15.89:0/1334687727 learned_addr learned my addr 172.21.15.89:0/1334687727 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426ffff700 1 -- 172.21.15.89:0/1334687727 >> v2:172.21.15.89:3300/0 conn(0x7f4270135e60 msgr2=0x7f4270138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426ffff700 1 --2- 172.21.15.89:0/1334687727 >> v2:172.21.15.89:3300/0 conn(0x7f4270135e60 0x7f4270138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426ffff700 1 -- 172.21.15.89:0/1334687727 >> v1:172.21.15.89:6789/0 conn(0x7f4270130540 legacy=0x7f4270130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426ffff700 1 -- 172.21.15.89:0/1334687727 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270070b80 con 0x7f4270133540 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426ffff700 1 --2- 172.21.15.89:0/1334687727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270133540 0x7f4270135920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f425c00bb90 tx=0x7f425c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa85f23af5c0311 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426f7fe700 1 -- 172.21.15.89:0/1334687727 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00dd30 con 0x7f4270133540 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426f7fe700 1 -- 172.21.15.89:0/1334687727 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c00de90 con 0x7f4270133540 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.222+0000 7f426f7fe700 1 -- 172.21.15.89:0/1334687727 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c010690 con 0x7f4270133540 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.223+0000 7f4276db6700 1 -- 172.21.15.89:0/1334687727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270133540 msgr2=0x7f4270135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.223+0000 7f4276db6700 1 --2- 172.21.15.89:0/1334687727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270133540 0x7f4270135920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f425c00bb90 tx=0x7f425c00bea0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.223+0000 7f4276db6700 1 -- 172.21.15.89:0/1334687727 shutdown_connections 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.223+0000 7f4276db6700 1 --2- 172.21.15.89:0/1334687727 >> v2:172.21.15.89:3300/0 conn(0x7f4270135e60 0x7f4270138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.223+0000 7f4276db6700 1 --2- 172.21.15.89:0/1334687727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270133540 0x7f4270135920 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.223+0000 7f4276db6700 1 -- 172.21.15.89:0/1334687727 >> 172.21.15.89:0/1334687727 conn(0x7f42700a2aa0 msgr2=0x7f42700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.223+0000 7f4276db6700 1 -- 172.21.15.89:0/1334687727 shutdown_connections 2023-09-30T22:22:38.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.223+0000 7f4276db6700 1 -- 172.21.15.89:0/1334687727 wait complete. 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.224+0000 7f4276db6700 1 Processor -- start 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.224+0000 7f4276db6700 1 -- start start 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4276db6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270130540 0x7f427009d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4276db6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4270133540 0x7f427009d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4276db6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4270135e60 0x7f42701c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4276db6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42700718e0 con 0x7f4270135e60 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4276db6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427012de50 con 0x7f4270133540 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4276db6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f427009cb30 con 0x7f4270130540 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270130540 0x7f427009d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270130540 0x7f427009d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33090/0 (socket says 172.21.15.89:33090) 2023-09-30T22:22:38.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 -- 172.21.15.89:0/882867127 learned_addr learned my addr 172.21.15.89:0/882867127 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f426ffff700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4270133540 0x7f427009d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4275353700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4270135e60 0x7f42701c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 -- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4270133540 msgr2=0x7f427009d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4270133540 0x7f427009d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 -- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4270135e60 msgr2=0x7f42701c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4270135e60 0x7f42701c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 -- 172.21.15.89:0/882867127 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c00b7f0 con 0x7f4270130540 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4275353700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4270135e60 0x7f42701c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.225+0000 7f4274b52700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270130540 0x7f427009d170 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f426400c510 tx=0x7f426400c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.226+0000 7f426d7fa700 1 -- 172.21.15.89:0/882867127 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42640052d0 con 0x7f4270130540 2023-09-30T22:22:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.226+0000 7f426d7fa700 1 -- 172.21.15.89:0/882867127 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4264005430 con 0x7f4270130540 2023-09-30T22:22:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.226+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427011e5c0 con 0x7f4270130540 2023-09-30T22:22:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.226+0000 7f426d7fa700 1 -- 172.21.15.89:0/882867127 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42640038c0 con 0x7f4270130540 2023-09-30T22:22:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.226+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427009e3f0 con 0x7f4270130540 2023-09-30T22:22:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.227+0000 7f426d7fa700 1 -- 172.21.15.89:0/882867127 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4264003a20 con 0x7f4270130540 2023-09-30T22:22:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.227+0000 7f426d7fa700 1 -- 172.21.15.89:0/882867127 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f426403e970 con 0x7f4270130540 2023-09-30T22:22:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.227+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427009e3f0 con 0x7f4270130540 2023-09-30T22:22:38.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.236+0000 7f426d7fa700 1 -- 172.21.15.89:0/882867127 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f427009e3f0 con 0x7f4270130540 2023-09-30T22:22:38.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.236+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2b0806b250 con 0x7f2b08133420 2023-09-30T22:22:38.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.236+0000 7f2afe7fc700 1 -- 172.21.15.89:0/2079845630 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b04017020 con 0x7f2b08133420 2023-09-30T22:22:38.236 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:38.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08133420 msgr2=0x7f2b0812f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08133420 0x7f2b0812f0b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c4e0 tx=0x7f2b0400c7f0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 shutdown_connections 2023-09-30T22:22:38.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2b08138dc0 0x7f2b0812fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b08133420 0x7f2b0812f0b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 --2- 172.21.15.89:0/2079845630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b081337f0 0x7f2b0812f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 >> 172.21.15.89:0/2079845630 conn(0x7f2b080a4210 msgr2=0x7f2b080a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:38.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 shutdown_connections 2023-09-30T22:22:38.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.239+0000 7f2b1013f700 1 -- 172.21.15.89:0/2079845630 wait complete. 2023-09-30T22:22:38.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.244+0000 7fecc32d5700 1 Processor -- start 2023-09-30T22:22:38.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.244+0000 7fecc32d5700 1 -- start start 2023-09-30T22:22:38.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc32d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc32d5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fecbc13d1f0 0x7fecbc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc32d5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc09a810 con 0x7fecbc06a770 2023-09-30T22:22:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc32d5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc06ab40 con 0x7fecbc13d1f0 2023-09-30T22:22:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc32d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc09ebc0 con 0x7fecbc09de00 2023-09-30T22:22:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc1071700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fecbc06a770 0x7fecbc09d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48044/0 (socket says 172.21.15.89:48044) 2023-09-30T22:22:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc1071700 1 -- 172.21.15.89:0/3558909630 learned_addr learned my addr 172.21.15.89:0/3558909630 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:38.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc1872700 1 --2- 172.21.15.89:0/3558909630 >> v2:172.21.15.89:3300/0 conn(0x7fecbc13d1f0 0x7fecbc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc1872700 1 -- 172.21.15.89:0/3558909630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 msgr2=0x7fecbc09e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc1872700 1 --2- 172.21.15.89:0/3558909630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc1872700 1 -- 172.21.15.89:0/3558909630 >> v1:172.21.15.89:6789/0 conn(0x7fecbc06a770 legacy=0x7fecbc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc1872700 1 -- 172.21.15.89:0/3558909630 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc09e920 con 0x7fecbc13d1f0 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecc1872700 1 --2- 172.21.15.89:0/3558909630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 0x7fecbc13f5d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009d90 tx=0x7fecb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9472265b52a24cc3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.245+0000 7fecb3fff700 1 -- 172.21.15.89:0/3558909630 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb400e070 con 0x7fecbc13d1f0 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecb3fff700 1 -- 172.21.15.89:0/3558909630 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb400bbd0 con 0x7fecbc13d1f0 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecb3fff700 1 -- 172.21.15.89:0/3558909630 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4012430 con 0x7fecbc13d1f0 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecc32d5700 1 -- 172.21.15.89:0/3558909630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 msgr2=0x7fecbc13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecc32d5700 1 --2- 172.21.15.89:0/3558909630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 0x7fecbc13f5d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009d90 tx=0x7fecb400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecc32d5700 1 -- 172.21.15.89:0/3558909630 shutdown_connections 2023-09-30T22:22:38.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecc32d5700 1 --2- 172.21.15.89:0/3558909630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 0x7fecbc13f5d0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecc32d5700 1 --2- 172.21.15.89:0/3558909630 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc09e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecc32d5700 1 -- 172.21.15.89:0/3558909630 >> 172.21.15.89:0/3558909630 conn(0x7fecbc066f50 msgr2=0x7fecbc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:38.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecc32d5700 1 -- 172.21.15.89:0/3558909630 shutdown_connections 2023-09-30T22:22:38.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.246+0000 7fecc32d5700 1 -- 172.21.15.89:0/3558909630 wait complete. 2023-09-30T22:22:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.247+0000 7fecc32d5700 1 Processor -- start 2023-09-30T22:22:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.247+0000 7fecc32d5700 1 -- start start 2023-09-30T22:22:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc32d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecbc06a770 0x7fecbc13b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc32d5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc13bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc32d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 0x7fecbc13c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc32d5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc11ed00 con 0x7fecbc13d1f0 2023-09-30T22:22:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc32d5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc09f210 con 0x7fecbc09de00 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc32d5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc11d170 con 0x7fecbc06a770 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc1872700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 0x7fecbc13c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc0870700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc13bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc0870700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc13bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39876/0 (socket says 172.21.15.89:39876) 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc0870700 1 -- 172.21.15.89:0/1241378469 learned_addr learned my addr 172.21.15.89:0/1241378469 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc1071700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecbc06a770 0x7fecbc13b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc1872700 1 -- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecbc06a770 msgr2=0x7fecbc13b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc1872700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecbc06a770 0x7fecbc13b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc1872700 1 -- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 msgr2=0x7fecbc13bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc1872700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc13bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc1872700 1 -- 172.21.15.89:0/1241378469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4009a70 con 0x7fecbc13d1f0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc0870700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc13bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.248+0000 7fecc1071700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecbc06a770 0x7fecbc13b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.249+0000 7fecc1872700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 0x7fecbc13c0f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fecb4016000 tx=0x7fecb4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.249+0000 7fecb1ffb700 1 -- 172.21.15.89:0/1241378469 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4012430 con 0x7fecbc13d1f0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.249+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc11e390 con 0x7fecbc13d1f0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.249+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc11ded0 con 0x7fecbc13d1f0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.249+0000 7fecb1ffb700 1 -- 172.21.15.89:0/1241378469 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb4020070 con 0x7fecbc13d1f0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.249+0000 7fecb1ffb700 1 -- 172.21.15.89:0/1241378469 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4004040 con 0x7fecbc13d1f0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.250+0000 7fecb1ffb700 1 -- 172.21.15.89:0/1241378469 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fecb40041a0 con 0x7fecbc13d1f0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.250+0000 7fecb1ffb700 1 -- 172.21.15.89:0/1241378469 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fecb4022070 con 0x7fecbc13d1f0 2023-09-30T22:22:38.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.250+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c0052e0 con 0x7fecbc13d1f0 2023-09-30T22:22:38.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:38.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.256+0000 7fecb1ffb700 1 -- 172.21.15.89:0/1241378469 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fecb401b400 con 0x7fecbc13d1f0 2023-09-30T22:22:38.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.316+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f851811f220 con 0x7f8518130540 2023-09-30T22:22:38.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.316+0000 7f84ff7fe700 1 -- 172.21.15.89:0/3945369656 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8508011360 con 0x7f8518130540 2023-09-30T22:22:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518130540 msgr2=0x7f8518134460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518130540 0x7f8518134460 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f850800c780 tx=0x7f850800ca90 comp rx=0 tx=0).stop 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 shutdown_connections 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8518135e60 0x7f85181c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8518130540 0x7f8518134460 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 --2- 172.21.15.89:0/3945369656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8518133540 0x7f85181349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 >> 172.21.15.89:0/3945369656 conn(0x7f85180a2aa0 msgr2=0x7f851812e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 shutdown_connections 2023-09-30T22:22:38.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.318+0000 7f851d3d1700 1 -- 172.21.15.89:0/3945369656 wait complete. 2023-09-30T22:22:38.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:38.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.435+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f427009e3f0 con 0x7f4270130540 2023-09-30T22:22:38.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.435+0000 7f426d7fa700 1 -- 172.21.15.89:0/882867127 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f427009e3f0 con 0x7f4270130540 2023-09-30T22:22:38.435 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:38.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270130540 msgr2=0x7f427009d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270130540 0x7f427009d170 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f426400c510 tx=0x7f426400c820 comp rx=0 tx=0).stop 2023-09-30T22:22:38.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 shutdown_connections 2023-09-30T22:22:38.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4270135e60 0x7f42701c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4270130540 0x7f427009d170 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 --2- 172.21.15.89:0/882867127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4270133540 0x7f427009d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 >> 172.21.15.89:0/882867127 conn(0x7f42700a2aa0 msgr2=0x7f4270131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:38.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 shutdown_connections 2023-09-30T22:22:38.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.437+0000 7f4276db6700 1 -- 172.21.15.89:0/882867127 wait complete. 2023-09-30T22:22:38.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:38.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.453+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fec9c005590 con 0x7fecbc13d1f0 2023-09-30T22:22:38.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.453+0000 7fecb1ffb700 1 -- 172.21.15.89:0/1241378469 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fecb4035090 con 0x7fecbc13d1f0 2023-09-30T22:22:38.453 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 msgr2=0x7fecbc13c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 0x7fecbc13c0f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fecb4016000 tx=0x7fecb4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 shutdown_connections 2023-09-30T22:22:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecbc13d1f0 0x7fecbc13c0f0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecbc06a770 0x7fecbc13b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 --2- 172.21.15.89:0/1241378469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecbc09de00 0x7fecbc13bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:38.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 >> 172.21.15.89:0/1241378469 conn(0x7fecbc066f50 msgr2=0x7fecbc09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:38.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 shutdown_connections 2023-09-30T22:22:38.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:38.455+0000 7fecc32d5700 1 -- 172.21.15.89:0/1241378469 wait complete. 2023-09-30T22:22:38.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:39.255 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:39.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6d165700 1 Processor -- start 2023-09-30T22:22:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6d165700 1 -- start start 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6d165700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809e930 0x7f7a6809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6d165700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a6809f240 0x7f7a681317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6d165700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809f650 0x7f7a68131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6d165700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a68070940 con 0x7f7a6809f240 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6d165700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806fd60 con 0x7f7a6809e930 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6d165700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806fa80 con 0x7f7a6809f650 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a66d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809e930 0x7f7a6809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6759e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809f650 0x7f7a68131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.394+0000 7f7a6659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a6809f240 0x7f7a681317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a66d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809e930 0x7f7a6809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39880/0 (socket says 172.21.15.89:39880) 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a6759e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809f650 0x7f7a68131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33106/0 (socket says 172.21.15.89:33106) 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a66d9d700 1 -- 172.21.15.89:0/1747810847 learned_addr learned my addr 172.21.15.89:0/1747810847 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a6659c700 1 -- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809f650 msgr2=0x7f7a68131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a6659c700 1 --2- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809f650 0x7f7a68131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a6659c700 1 -- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809e930 msgr2=0x7f7a6809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a6659c700 1 --2- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809e930 0x7f7a6809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a6659c700 1 -- 172.21.15.89:0/1747810847 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6809fb60 con 0x7f7a6809f240 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a66d9d700 1 --2- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809e930 0x7f7a6809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a6659c700 1 --2- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 0x7f7a681317f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009840 tx=0x7f7a5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5d3c17d2b76c8d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a65d9b700 1 -- 172.21.15.89:0/1747810847 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c011830 con 0x7f7a6809f240 2023-09-30T22:22:39.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a65d9b700 1 -- 172.21.15.89:0/1747810847 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c011990 con 0x7f7a6809f240 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.395+0000 7f7a65d9b700 1 -- 172.21.15.89:0/1747810847 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c011d20 con 0x7f7a6809f240 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 -- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 msgr2=0x7f7a681317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 --2- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 0x7f7a681317f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009840 tx=0x7f7a5c009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 -- 172.21.15.89:0/1747810847 shutdown_connections 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 --2- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809f650 0x7f7a68131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 --2- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809e930 0x7f7a6809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 --2- 172.21.15.89:0/1747810847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 0x7f7a681317f0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 -- 172.21.15.89:0/1747810847 >> 172.21.15.89:0/1747810847 conn(0x7f7a680a4210 msgr2=0x7f7a680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 -- 172.21.15.89:0/1747810847 shutdown_connections 2023-09-30T22:22:39.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.396+0000 7f7a6d165700 1 -- 172.21.15.89:0/1747810847 wait complete. 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.397+0000 7f7a6d165700 1 Processor -- start 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.397+0000 7f7a6d165700 1 -- start start 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6d165700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809e930 0x7f7a681b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6d165700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 0x7f7a681b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6d165700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809f650 0x7f7a681a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6d165700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681a7d20 con 0x7f7a6809f240 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6d165700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681a44c0 con 0x7f7a6809f650 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6d165700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681a4280 con 0x7f7a6809e930 2023-09-30T22:22:39.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a66d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809e930 0x7f7a681b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a66d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809e930 0x7f7a681b67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33122/0 (socket says 172.21.15.89:33122) 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a66d9d700 1 -- 172.21.15.89:0/1872069695 learned_addr learned my addr 172.21.15.89:0/1872069695 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6659c700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 0x7f7a681b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6759e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809f650 0x7f7a681a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6659c700 1 -- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809e930 msgr2=0x7f7a681b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6659c700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809e930 0x7f7a681b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6659c700 1 -- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809f650 msgr2=0x7f7a681a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6659c700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809f650 0x7f7a681a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6659c700 1 -- 172.21.15.89:0/1872069695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a58009550 con 0x7f7a6809f240 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6759e700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809f650 0x7f7a681a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6659c700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 0x7f7a681b6d10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c016000 tx=0x7f7a5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a4f7fe700 1 -- 172.21.15.89:0/1872069695 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c004450 con 0x7f7a6809f240 2023-09-30T22:22:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a4f7fe700 1 -- 172.21.15.89:0/1872069695 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c01f070 con 0x7f7a6809f240 2023-09-30T22:22:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c0094a0 con 0x7f7a6809f240 2023-09-30T22:22:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.398+0000 7f7a4f7fe700 1 -- 172.21.15.89:0/1872069695 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0038e0 con 0x7f7a6809f240 2023-09-30T22:22:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.399+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a680a0190 con 0x7f7a6809f240 2023-09-30T22:22:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.399+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a68071d80 con 0x7f7a6809f240 2023-09-30T22:22:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.400+0000 7f7a4f7fe700 1 -- 172.21.15.89:0/1872069695 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a5c003a40 con 0x7f7a6809f240 2023-09-30T22:22:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.400+0000 7f7a4f7fe700 1 -- 172.21.15.89:0/1872069695 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7a5c021070 con 0x7f7a6809f240 2023-09-30T22:22:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.404+0000 7f7a4f7fe700 1 -- 172.21.15.89:0/1872069695 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a5c0045b0 con 0x7f7a6809f240 2023-09-30T22:22:39.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:39.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.463+0000 7fd4a2995700 1 Processor -- start 2023-09-30T22:22:39.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.463+0000 7fd4a2995700 1 -- start start 2023-09-30T22:22:39.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a2995700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd49c09d6f0 0x7fd49c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a2995700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a2995700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49c09b120 con 0x7fd49c09e000 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a2995700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49c06b460 con 0x7fd49c09d6f0 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a2995700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c09a7e0 con 0x7fd49c09e440 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a0f32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd49bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd49c09d6f0 0x7fd49c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a0f32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39884/0 (socket says 172.21.15.89:39884) 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd49bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd49c09d6f0 0x7fd49c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48868/0 (socket says 172.21.15.89:48868) 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a0f32700 1 -- 172.21.15.89:0/1655700633 learned_addr learned my addr 172.21.15.89:0/1655700633 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:39.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a0f32700 1 -- 172.21.15.89:0/1655700633 >> v2:172.21.15.89:3300/0 conn(0x7fd49c09d6f0 msgr2=0x7fd49c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a0f32700 1 --2- 172.21.15.89:0/1655700633 >> v2:172.21.15.89:3300/0 conn(0x7fd49c09d6f0 0x7fd49c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a0f32700 1 -- 172.21.15.89:0/1655700633 >> v1:172.21.15.89:6789/0 conn(0x7fd49c09e000 legacy=0x7fd49c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.464+0000 7fd4a0f32700 1 -- 172.21.15.89:0/1655700633 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c09a500 con 0x7fd49c09e440 2023-09-30T22:22:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd4a0f32700 1 --2- 172.21.15.89:0/1655700633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c134100 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd494009e80 tx=0x7fd49400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4bae471e116202 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd49affd700 1 -- 172.21.15.89:0/1655700633 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49400e070 con 0x7fd49c09e440 2023-09-30T22:22:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd49affd700 1 -- 172.21.15.89:0/1655700633 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49400b8d0 con 0x7fd49c09e440 2023-09-30T22:22:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd49affd700 1 -- 172.21.15.89:0/1655700633 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49400bc80 con 0x7fd49c09e440 2023-09-30T22:22:39.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd4a2995700 1 -- 172.21.15.89:0/1655700633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 msgr2=0x7fd49c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd4a2995700 1 --2- 172.21.15.89:0/1655700633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c134100 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd494009e80 tx=0x7fd49400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:39.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd4a2995700 1 -- 172.21.15.89:0/1655700633 shutdown_connections 2023-09-30T22:22:39.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd4a2995700 1 --2- 172.21.15.89:0/1655700633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c134100 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd4a2995700 1 --2- 172.21.15.89:0/1655700633 >> v2:172.21.15.89:3300/0 conn(0x7fd49c09d6f0 0x7fd49c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd4a2995700 1 -- 172.21.15.89:0/1655700633 >> 172.21.15.89:0/1655700633 conn(0x7fd49c066f50 msgr2=0x7fd49c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.465+0000 7fd4a2995700 1 -- 172.21.15.89:0/1655700633 shutdown_connections 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.466+0000 7fd4a2995700 1 -- 172.21.15.89:0/1655700633 wait complete. 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.467+0000 7fd4a2995700 1 Processor -- start 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.467+0000 7fd4a2995700 1 -- start start 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.467+0000 7fd4a2995700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c09d6f0 0x7fd49c1c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.467+0000 7fd4a2995700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c09e000 0x7fd49c1c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.467+0000 7fd4a2995700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c1c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd4a2995700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c1a3ca0 con 0x7fd49c09d6f0 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd4a2995700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0a1f90 con 0x7fd49c09e440 2023-09-30T22:22:39.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd4a2995700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c1b5140 con 0x7fd49c09e000 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c09d6f0 0x7fd49c1c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c09d6f0 0x7fd49c1c4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48878/0 (socket says 172.21.15.89:48878) 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 -- 172.21.15.89:0/4176484022 learned_addr learned my addr 172.21.15.89:0/4176484022 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49b7fe700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c09e000 0x7fd49c1c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd4a0f32700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c1c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 -- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c09e000 msgr2=0x7fd49c1c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c09e000 0x7fd49c1c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 -- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 msgr2=0x7fd49c1c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c1c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 -- 172.21.15.89:0/4176484022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd494009b30 con 0x7fd49c09d6f0 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd4a0f32700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c1c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:39.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49b7fe700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c09e000 0x7fd49c1c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd49bfff700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c09d6f0 0x7fd49c1c4150 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd48c017100 tx=0x7fd48c017410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd498ff9700 1 -- 172.21.15.89:0/4176484022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c013c80 con 0x7fd49c09d6f0 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd498ff9700 1 -- 172.21.15.89:0/4176484022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c019040 con 0x7fd49c09d6f0 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c1a4570 con 0x7fd49c09d6f0 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd498ff9700 1 -- 172.21.15.89:0/4176484022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c01f600 con 0x7fd49c09d6f0 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.468+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c1a35b0 con 0x7fd49c09d6f0 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.469+0000 7fd498ff9700 1 -- 172.21.15.89:0/4176484022 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd48c01f760 con 0x7fd49c09d6f0 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.469+0000 7fd498ff9700 1 -- 172.21.15.89:0/4176484022 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd48c047740 con 0x7fd49c09d6f0 2023-09-30T22:22:39.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.469+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c1a35b0 con 0x7fd49c09d6f0 2023-09-30T22:22:39.473 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:39.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.474+0000 7fd498ff9700 1 -- 172.21.15.89:0/4176484022 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd49c1a35b0 con 0x7fd49c09d6f0 2023-09-30T22:22:39.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.586+0000 7f64fb6d8700 1 Processor -- start 2023-09-30T22:22:39.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.586+0000 7f64fb6d8700 1 -- start start 2023-09-30T22:22:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64fb6d8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64f409d6f0 0x7f64f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64fb6d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 0x7f64f41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64fb6d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f409b120 con 0x7f64f409e410 2023-09-30T22:22:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64fb6d8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f406b460 con 0x7f64f409d6f0 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64fb6d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64f409a7e0 con 0x7f64f409e000 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64f9c75700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f64f409e410 0x7f64f4133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48060/0 (socket says 172.21.15.89:48060) 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64f9c75700 1 -- 172.21.15.89:0/3083554205 learned_addr learned my addr 172.21.15.89:0/3083554205 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64f9474700 1 --2- 172.21.15.89:0/3083554205 >> v2:172.21.15.89:3300/0 conn(0x7f64f409d6f0 0x7f64f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.587+0000 7f64f8c73700 1 --2- 172.21.15.89:0/3083554205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 0x7f64f41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64f9474700 1 -- 172.21.15.89:0/3083554205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 msgr2=0x7f64f41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64f9474700 1 --2- 172.21.15.89:0/3083554205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 0x7f64f41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64f9474700 1 -- 172.21.15.89:0/3083554205 >> v1:172.21.15.89:6789/0 conn(0x7f64f409e410 legacy=0x7f64f4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64f9474700 1 -- 172.21.15.89:0/3083554205 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f409a500 con 0x7f64f409d6f0 2023-09-30T22:22:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64ebfff700 1 -- 172.21.15.89:0/3083554205 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331635360 0 0) 0x7f64f409b120 con 0x7f64f409e410 2023-09-30T22:22:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64f9474700 1 --2- 172.21.15.89:0/3083554205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409d6f0 0x7f64f409dac0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009840 tx=0x7f64f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03b4b5de8bac96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64ebfff700 1 -- 172.21.15.89:0/3083554205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f0011830 con 0x7f64f409d6f0 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64ebfff700 1 -- 172.21.15.89:0/3083554205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f0011990 con 0x7f64f409d6f0 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.588+0000 7f64ebfff700 1 -- 172.21.15.89:0/3083554205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f0011d20 con 0x7f64f409d6f0 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.589+0000 7f64fb6d8700 1 -- 172.21.15.89:0/3083554205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409d6f0 msgr2=0x7f64f409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.589+0000 7f64fb6d8700 1 --2- 172.21.15.89:0/3083554205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409d6f0 0x7f64f409dac0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009840 tx=0x7f64f0009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.589+0000 7f64fb6d8700 1 -- 172.21.15.89:0/3083554205 shutdown_connections 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.589+0000 7f64fb6d8700 1 --2- 172.21.15.89:0/3083554205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409d6f0 0x7f64f409dac0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.589+0000 7f64fb6d8700 1 --2- 172.21.15.89:0/3083554205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 0x7f64f41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.589+0000 7f64fb6d8700 1 -- 172.21.15.89:0/3083554205 >> 172.21.15.89:0/3083554205 conn(0x7f64f4066f50 msgr2=0x7f64f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.589+0000 7f64fb6d8700 1 -- 172.21.15.89:0/3083554205 shutdown_connections 2023-09-30T22:22:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.589+0000 7f64fb6d8700 1 -- 172.21.15.89:0/3083554205 wait complete. 2023-09-30T22:22:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64fb6d8700 1 Processor -- start 2023-09-30T22:22:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64fb6d8700 1 -- start start 2023-09-30T22:22:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64fb6d8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64f409d6f0 0x7f64f41c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64fb6d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 0x7f64f41c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64fb6d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409e410 0x7f64f41c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64fb6d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f41aaae0 con 0x7f64f409e410 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64fb6d8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f40a1f00 con 0x7f64f409d6f0 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64fb6d8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64f409c710 con 0x7f64f409e000 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64f9474700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64f409d6f0 0x7f64f41c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64f9474700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64f409d6f0 0x7f64f41c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39900/0 (socket says 172.21.15.89:39900) 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.591+0000 7f64f9474700 1 -- 172.21.15.89:0/2760885054 learned_addr learned my addr 172.21.15.89:0/2760885054 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64f8c73700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 0x7f64f41c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64f9c75700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409e410 0x7f64f41c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64f9474700 1 -- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 msgr2=0x7f64f41c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64f9474700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 0x7f64f41c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64f9474700 1 -- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409e410 msgr2=0x7f64f41c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64f9474700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409e410 0x7f64f41c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64f9474700 1 -- 172.21.15.89:0/2760885054 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00094a0 con 0x7f64f409d6f0 2023-09-30T22:22:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64f9474700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64f409d6f0 0x7f64f41c87e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f64f0000c00 tx=0x7f64f0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64e9ffb700 1 -- 172.21.15.89:0/2760885054 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f001a860 con 0x7f64f409d6f0 2023-09-30T22:22:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64e9ffb700 1 -- 172.21.15.89:0/2760885054 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f001a9c0 con 0x7f64f409d6f0 2023-09-30T22:22:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64fb6d8700 1 -- 172.21.15.89:0/2760885054 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f41a9cc0 con 0x7f64f409d6f0 2023-09-30T22:22:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64e9ffb700 1 -- 172.21.15.89:0/2760885054 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f001ad30 con 0x7f64f409d6f0 2023-09-30T22:22:39.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.592+0000 7f64fb6d8700 1 -- 172.21.15.89:0/2760885054 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f41ac290 con 0x7f64f409d6f0 2023-09-30T22:22:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.593+0000 7f64e9ffb700 1 -- 172.21.15.89:0/2760885054 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64f0022070 con 0x7f64f409d6f0 2023-09-30T22:22:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.594+0000 7f64e9ffb700 1 -- 172.21.15.89:0/2760885054 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f64f002c9b0 con 0x7f64f409d6f0 2023-09-30T22:22:39.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.594+0000 7f64df7fe700 1 -- 172.21.15.89:0/2760885054 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f409e920 con 0x7f64f409d6f0 2023-09-30T22:22:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.598+0000 7f64e9ffb700 1 -- 172.21.15.89:0/2760885054 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64f0029070 con 0x7f64f409d6f0 2023-09-30T22:22:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c6f18700 1 Processor -- start 2023-09-30T22:22:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c6f18700 1 -- start start 2023-09-30T22:22:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c6f18700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0136710 0x7f43c0138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c6f18700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f43c0139030 0x7f43c013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c6f18700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c006ef30 con 0x7f43c01338a0 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c6f18700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c006fef0 con 0x7f43c0139030 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c6f18700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01334b0 con 0x7f43c0136710 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c4cb4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f43c01338a0 0x7f43c0133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48064/0 (socket says 172.21.15.89:48064) 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43c4cb4700 1 -- 172.21.15.89:0/2271016010 learned_addr learned my addr 172.21.15.89:0/2271016010 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.604+0000 7f43bffff700 1 --2- 172.21.15.89:0/2271016010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0136710 0x7f43c0138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43c54b5700 1 --2- 172.21.15.89:0/2271016010 >> v2:172.21.15.89:3300/0 conn(0x7f43c0139030 0x7f43c013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43bffff700 1 -- 172.21.15.89:0/2271016010 >> v2:172.21.15.89:3300/0 conn(0x7f43c0139030 msgr2=0x7f43c013b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43bffff700 1 --2- 172.21.15.89:0/2271016010 >> v2:172.21.15.89:3300/0 conn(0x7f43c0139030 0x7f43c013b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43bffff700 1 -- 172.21.15.89:0/2271016010 >> v1:172.21.15.89:6789/0 conn(0x7f43c01338a0 legacy=0x7f43c0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43bffff700 1 -- 172.21.15.89:0/2271016010 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c006fc30 con 0x7f43c0136710 2023-09-30T22:22:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43bffff700 1 --2- 172.21.15.89:0/2271016010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0136710 0x7f43c0138af0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009840 tx=0x7f43ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0486c700481e12 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43bf7fe700 1 -- 172.21.15.89:0/2271016010 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac011830 con 0x7f43c0136710 2023-09-30T22:22:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43bf7fe700 1 -- 172.21.15.89:0/2271016010 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac011990 con 0x7f43c0136710 2023-09-30T22:22:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.605+0000 7f43bf7fe700 1 -- 172.21.15.89:0/2271016010 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac011d20 con 0x7f43c0136710 2023-09-30T22:22:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.606+0000 7f43c6f18700 1 -- 172.21.15.89:0/2271016010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0136710 msgr2=0x7f43c0138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.606+0000 7f43c6f18700 1 --2- 172.21.15.89:0/2271016010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0136710 0x7f43c0138af0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009840 tx=0x7f43ac009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.606+0000 7f43c6f18700 1 -- 172.21.15.89:0/2271016010 shutdown_connections 2023-09-30T22:22:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.606+0000 7f43c6f18700 1 --2- 172.21.15.89:0/2271016010 >> v2:172.21.15.89:3300/0 conn(0x7f43c0139030 0x7f43c013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.606+0000 7f43c6f18700 1 --2- 172.21.15.89:0/2271016010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0136710 0x7f43c0138af0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.606+0000 7f43c6f18700 1 -- 172.21.15.89:0/2271016010 >> 172.21.15.89:0/2271016010 conn(0x7f43c00a2aa0 msgr2=0x7f43c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.606+0000 7f43c6f18700 1 -- 172.21.15.89:0/2271016010 shutdown_connections 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.606+0000 7f43c6f18700 1 -- 172.21.15.89:0/2271016010 wait complete. 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c6f18700 1 Processor -- start 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c6f18700 1 -- start start 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c6f18700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43c01338a0 0x7f43c0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c6f18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43c0136710 0x7f43c011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c6f18700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0139030 0x7f43c011aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c6f18700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c006f640 con 0x7f43c0136710 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c6f18700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c012fbc0 con 0x7f43c01338a0 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c6f18700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43c011d250 con 0x7f43c0139030 2023-09-30T22:22:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c4cb4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43c01338a0 0x7f43c0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c4cb4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43c01338a0 0x7f43c0119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39912/0 (socket says 172.21.15.89:39912) 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c4cb4700 1 -- 172.21.15.89:0/1314365160 learned_addr learned my addr 172.21.15.89:0/1314365160 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.607+0000 7f43c54b5700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0139030 0x7f43c011aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43bffff700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43c0136710 0x7f43c011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43c4cb4700 1 -- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0139030 msgr2=0x7f43c011aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43c4cb4700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0139030 0x7f43c011aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43c4cb4700 1 -- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43c0136710 msgr2=0x7f43c011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43c4cb4700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43c0136710 0x7f43c011a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43c4cb4700 1 -- 172.21.15.89:0/1314365160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0094a0 con 0x7f43c01338a0 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43c4cb4700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43c01338a0 0x7f43c0119f80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f43b400c520 tx=0x7f43b400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43bd7fa700 1 -- 172.21.15.89:0/1314365160 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4005d70 con 0x7f43c01338a0 2023-09-30T22:22:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43bd7fa700 1 -- 172.21.15.89:0/1314365160 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b40070d0 con 0x7f43c01338a0 2023-09-30T22:22:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43bd7fa700 1 -- 172.21.15.89:0/1314365160 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4007440 con 0x7f43c01338a0 2023-09-30T22:22:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c011e0c0 con 0x7f43c01338a0 2023-09-30T22:22:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.608+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c011d7f0 con 0x7f43c01338a0 2023-09-30T22:22:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.609+0000 7f43bd7fa700 1 -- 172.21.15.89:0/1314365160 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f43b40075a0 con 0x7f43c01338a0 2023-09-30T22:22:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.609+0000 7f43bd7fa700 1 -- 172.21.15.89:0/1314365160 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f43b403ea00 con 0x7f43c01338a0 2023-09-30T22:22:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.609+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c0074ea0 con 0x7f43c01338a0 2023-09-30T22:22:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.614+0000 7f43bd7fa700 1 -- 172.21.15.89:0/1314365160 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43b4019b40 con 0x7f43c01338a0 2023-09-30T22:22:39.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.637+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7a68074ea0 con 0x7f7a6809f240 2023-09-30T22:22:39.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.637+0000 7f7a4f7fe700 1 -- 172.21.15.89:0/1872069695 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a5c015030 con 0x7f7a6809f240 2023-09-30T22:22:39.638 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:39.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.640+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 msgr2=0x7f7a681b6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.640+0000 7f7a6d165700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 0x7f7a681b6d10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c016000 tx=0x7f7a5c004750 comp rx=0 tx=0).stop 2023-09-30T22:22:39.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.641+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 shutdown_connections 2023-09-30T22:22:39.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.641+0000 7f7a6d165700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a6809f650 0x7f7a681a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.641+0000 7f7a6d165700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a6809e930 0x7f7a681b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.641+0000 7f7a6d165700 1 --2- 172.21.15.89:0/1872069695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a6809f240 0x7f7a681b6d10 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.641+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 >> 172.21.15.89:0/1872069695 conn(0x7f7a680a4210 msgr2=0x7f7a681b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:39.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.641+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 shutdown_connections 2023-09-30T22:22:39.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.641+0000 7f7a6d165700 1 -- 172.21.15.89:0/1872069695 wait complete. 2023-09-30T22:22:39.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:39.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.684+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd49c1a35b0 con 0x7fd49c09d6f0 2023-09-30T22:22:39.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.684+0000 7fd498ff9700 1 -- 172.21.15.89:0/4176484022 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd49c1a35b0 con 0x7fd49c09d6f0 2023-09-30T22:22:39.685 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c09d6f0 msgr2=0x7fd49c1c4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c09d6f0 0x7fd49c1c4150 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd48c017100 tx=0x7fd48c017410 comp rx=0 tx=0).stop 2023-09-30T22:22:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 shutdown_connections 2023-09-30T22:22:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd49c09e440 0x7fd49c1c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c09d6f0 0x7fd49c1c4150 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 --2- 172.21.15.89:0/4176484022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd49c09e000 0x7fd49c1c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 >> 172.21.15.89:0/4176484022 conn(0x7fd49c066f50 msgr2=0x7fd49c1b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 shutdown_connections 2023-09-30T22:22:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.686+0000 7fd4a2995700 1 -- 172.21.15.89:0/4176484022 wait complete. 2023-09-30T22:22:39.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.806+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f43c011d470 con 0x7f43c01338a0 2023-09-30T22:22:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.806+0000 7f43bd7fa700 1 -- 172.21.15.89:0/1314365160 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43c011d470 con 0x7f43c01338a0 2023-09-30T22:22:39.807 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:39.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.808+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43c01338a0 msgr2=0x7f43c0119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.808+0000 7f43c6f18700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43c01338a0 0x7f43c0119f80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f43b400c520 tx=0x7f43b400c830 comp rx=0 tx=0).stop 2023-09-30T22:22:39.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.808+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 shutdown_connections 2023-09-30T22:22:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.808+0000 7f43c6f18700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43c0139030 0x7f43c011aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.808+0000 7f43c6f18700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f43c01338a0 0x7f43c0119f80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.808+0000 7f43c6f18700 1 --2- 172.21.15.89:0/1314365160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43c0136710 0x7f43c011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.808+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 >> 172.21.15.89:0/1314365160 conn(0x7f43c00a2aa0 msgr2=0x7f43c0130050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.809+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 shutdown_connections 2023-09-30T22:22:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.809+0000 7f43c6f18700 1 -- 172.21.15.89:0/1314365160 wait complete. 2023-09-30T22:22:39.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.810+0000 7f64df7fe700 1 -- 172.21.15.89:0/2760885054 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f64f41a9e50 con 0x7f64f409d6f0 2023-09-30T22:22:39.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.810+0000 7f64e9ffb700 1 -- 172.21.15.89:0/2760885054 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64f0038090 con 0x7f64f409d6f0 2023-09-30T22:22:39.811 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.812+0000 7f64df7fe700 1 -- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64f409d6f0 msgr2=0x7f64f41c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.812+0000 7f64df7fe700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64f409d6f0 0x7f64f41c87e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f64f0000c00 tx=0x7f64f0004750 comp rx=0 tx=0).stop 2023-09-30T22:22:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.813+0000 7f64df7fe700 1 -- 172.21.15.89:0/2760885054 shutdown_connections 2023-09-30T22:22:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.813+0000 7f64df7fe700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409e410 0x7f64f41c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.813+0000 7f64df7fe700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f64f409d6f0 0x7f64f41c87e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.813+0000 7f64df7fe700 1 --2- 172.21.15.89:0/2760885054 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f409e000 0x7f64f41c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.813+0000 7f64df7fe700 1 -- 172.21.15.89:0/2760885054 >> 172.21.15.89:0/2760885054 conn(0x7f64f4066f50 msgr2=0x7f64f4071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.813+0000 7f64df7fe700 1 -- 172.21.15.89:0/2760885054 shutdown_connections 2023-09-30T22:22:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:39.813+0000 7f64df7fe700 1 -- 172.21.15.89:0/2760885054 wait complete. 2023-09-30T22:22:39.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:39.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:40.659 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:40.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:40.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.798+0000 7fb3331fb700 1 Processor -- start 2023-09-30T22:22:40.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.798+0000 7fb3331fb700 1 -- start start 2023-09-30T22:22:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb3331fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb3331fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c138d80 0x7fb32c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb3331fb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32c0a2a70 con 0x7fb32c09d630 2023-09-30T22:22:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb3331fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0a01d0 con 0x7fb32c138d80 2023-09-30T22:22:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb3331fb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c09e2b0 con 0x7fb32c0a0a30 2023-09-30T22:22:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb330f97700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb331798700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c138d80 0x7fb32c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb330f97700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55540/0 (socket says 172.21.15.89:55540) 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb331798700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c138d80 0x7fb32c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51164/0 (socket says 172.21.15.89:51164) 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb330f97700 1 -- 172.21.15.89:0/3988220492 learned_addr learned my addr 172.21.15.89:0/3988220492 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb32b7fe700 1 -- 172.21.15.89:0/3988220492 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986213557 0 0) 0x7fb32c0a2a70 con 0x7fb32c09d630 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb330f97700 1 -- 172.21.15.89:0/3988220492 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c138d80 msgr2=0x7fb32c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb330f97700 1 --2- 172.21.15.89:0/3988220492 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c138d80 0x7fb32c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb330f97700 1 -- 172.21.15.89:0/3988220492 >> v1:172.21.15.89:6789/0 conn(0x7fb32c09d630 legacy=0x7fb32c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb330f97700 1 -- 172.21.15.89:0/3988220492 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c09dc50 con 0x7fb32c0a0a30 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb331798700 1 --2- 172.21.15.89:0/3988220492 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c138d80 0x7fb32c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.799+0000 7fb330f97700 1 --2- 172.21.15.89:0/3988220492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c0a0e00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb320009920 tx=0x7fb320009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc46b23519a3b248 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb32b7fe700 1 -- 172.21.15.89:0/3988220492 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320011830 con 0x7fb32c0a0a30 2023-09-30T22:22:40.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb32b7fe700 1 -- 172.21.15.89:0/3988220492 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb320011990 con 0x7fb32c0a0a30 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb32b7fe700 1 -- 172.21.15.89:0/3988220492 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320011d20 con 0x7fb32c0a0a30 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb3331fb700 1 -- 172.21.15.89:0/3988220492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 msgr2=0x7fb32c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb3331fb700 1 --2- 172.21.15.89:0/3988220492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c0a0e00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb320009920 tx=0x7fb320009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb3331fb700 1 -- 172.21.15.89:0/3988220492 shutdown_connections 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb3331fb700 1 --2- 172.21.15.89:0/3988220492 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c138d80 0x7fb32c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb3331fb700 1 --2- 172.21.15.89:0/3988220492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c0a0e00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb3331fb700 1 -- 172.21.15.89:0/3988220492 >> 172.21.15.89:0/3988220492 conn(0x7fb32c0a3050 msgr2=0x7fb32c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb3331fb700 1 -- 172.21.15.89:0/3988220492 shutdown_connections 2023-09-30T22:22:40.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.800+0000 7fb3331fb700 1 -- 172.21.15.89:0/3988220492 wait complete. 2023-09-30T22:22:40.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.801+0000 7fb3331fb700 1 Processor -- start 2023-09-30T22:22:40.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.801+0000 7fb3331fb700 1 -- start start 2023-09-30T22:22:40.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb3331fb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c09d630 0x7fb32c1337d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb3331fb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c133d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb3331fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb32c138d80 0x7fb32c134250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb3331fb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0a0410 con 0x7fb32c138d80 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb3331fb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c136cc0 con 0x7fb32c09d630 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb3331fb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0a4b00 con 0x7fb32c0a0a30 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb331798700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb32c138d80 0x7fb32c134250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb32bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb330f97700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c09d630 0x7fb32c1337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb331798700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb32c138d80 0x7fb32c134250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56876/0 (socket says 172.21.15.89:56876) 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb331798700 1 -- 172.21.15.89:0/3287443989 learned_addr learned my addr 172.21.15.89:0/3287443989 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb32bfff700 1 -- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c09d630 msgr2=0x7fb32c1337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb32bfff700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c09d630 0x7fb32c1337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb32bfff700 1 -- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb32c138d80 msgr2=0x7fb32c134250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb32bfff700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb32c138d80 0x7fb32c134250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb32bfff700 1 -- 172.21.15.89:0/3287443989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb324009a70 con 0x7fb32c0a0a30 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb330f97700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c09d630 0x7fb32c1337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb331798700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb32c138d80 0x7fb32c134250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.802+0000 7fb32bfff700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c133d10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb32c06d980 tx=0x7fb31800c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.803+0000 7fb3297fa700 1 -- 172.21.15.89:0/3287443989 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3180054e0 con 0x7fb32c0a0a30 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.803+0000 7fb3297fa700 1 -- 172.21.15.89:0/3287443989 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31800ce10 con 0x7fb32c0a0a30 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.803+0000 7fb3297fa700 1 -- 172.21.15.89:0/3287443989 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3180038c0 con 0x7fb32c0a0a30 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.803+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb320009550 con 0x7fb32c0a0a30 2023-09-30T22:22:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.803+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c135af0 con 0x7fb32c0a0a30 2023-09-30T22:22:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.803+0000 7fb3297fa700 1 -- 172.21.15.89:0/3287443989 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb318003a20 con 0x7fb32c0a0a30 2023-09-30T22:22:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.804+0000 7fb3297fa700 1 -- 172.21.15.89:0/3287443989 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb31803e980 con 0x7fb32c0a0a30 2023-09-30T22:22:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.804+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c0747f0 con 0x7fb32c0a0a30 2023-09-30T22:22:40.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.808+0000 7fb3297fa700 1 -- 172.21.15.89:0/3287443989 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb318017080 con 0x7fb32c0a0a30 2023-09-30T22:22:40.823 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:40.827 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.834+0000 7f108c4af700 1 Processor -- start 2023-09-30T22:22:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.834+0000 7f108c4af700 1 -- start start 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108c4af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f108409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108c4af700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108c4af700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f108409e410 0x7f1084130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108c4af700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108409f520 con 0x7f108409e410 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108c4af700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108409f130 con 0x7f108409e000 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108c4af700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f108409e930 con 0x7f108409d6f0 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108a24b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f108409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f108409e410 0x7f1084130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f108409e410 0x7f1084130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56880/0 (socket says 172.21.15.89:56880) 2023-09-30T22:22:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108a24b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f108409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55566/0 (socket says 172.21.15.89:55566) 2023-09-30T22:22:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 -- 172.21.15.89:0/1953845969 learned_addr learned my addr 172.21.15.89:0/1953845969 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f1089a4a700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 -- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 msgr2=0x7f108409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f108409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 -- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 msgr2=0x7f1084130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 -- 172.21.15.89:0/1953845969 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108409b2d0 con 0x7f108409e410 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f1089a4a700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.835+0000 7f108aa4c700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 0x7f1084130ba0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f107c00d150 tx=0x7f107c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859389de1c34c8bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f1089249700 1 -- 172.21.15.89:0/1953845969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c010070 con 0x7f108409e410 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f1089249700 1 -- 172.21.15.89:0/1953845969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c00dd10 con 0x7f108409e410 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f1089249700 1 -- 172.21.15.89:0/1953845969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c00b310 con 0x7f108409e410 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f108c4af700 1 -- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 msgr2=0x7f1084130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f108c4af700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 0x7f1084130ba0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f107c00d150 tx=0x7f107c00d460 comp rx=0 tx=0).stop 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f108c4af700 1 -- 172.21.15.89:0/1953845969 shutdown_connections 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f108c4af700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 0x7f1084130ba0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f108c4af700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f108409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f108c4af700 1 --2- 172.21.15.89:0/1953845969 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.836+0000 7f108c4af700 1 -- 172.21.15.89:0/1953845969 >> 172.21.15.89:0/1953845969 conn(0x7f1084066f50 msgr2=0x7f108409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.837+0000 7f108c4af700 1 -- 172.21.15.89:0/1953845969 shutdown_connections 2023-09-30T22:22:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.837+0000 7f108c4af700 1 -- 172.21.15.89:0/1953845969 wait complete. 2023-09-30T22:22:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108c4af700 1 Processor -- start 2023-09-30T22:22:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108c4af700 1 -- start start 2023-09-30T22:22:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108c4af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f1084122220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108c4af700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084122760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108c4af700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 0x7f1084122ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108c4af700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840a16b0 con 0x7f108409e410 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108c4af700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1084124630 con 0x7f108409e000 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108c4af700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f108409c700 con 0x7f108409d6f0 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f1089a4a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084122760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f1089a4a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084122760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51202/0 (socket says 172.21.15.89:51202) 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f1089a4a700 1 -- 172.21.15.89:0/793835977 learned_addr learned my addr 172.21.15.89:0/793835977 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108a24b700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f1084122220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.838+0000 7f108aa4c700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 0x7f1084122ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f1089a4a700 1 -- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 msgr2=0x7f1084122220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f1089a4a700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f1084122220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f1089a4a700 1 -- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 msgr2=0x7f1084122ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f1089a4a700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 0x7f1084122ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f1089a4a700 1 -- 172.21.15.89:0/793835977 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10740094a0 con 0x7f108409e000 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f1089a4a700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084122760 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f107400ebb0 tx=0x7f107400eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f107affd700 1 -- 172.21.15.89:0/793835977 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400c850 con 0x7f108409e000 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f107affd700 1 -- 172.21.15.89:0/793835977 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107400c9b0 con 0x7f108409e000 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107c00bd40 con 0x7f108409e000 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1084125d20 con 0x7f108409e000 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.839+0000 7f107affd700 1 -- 172.21.15.89:0/793835977 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400cd40 con 0x7f108409e000 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.840+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c0052e0 con 0x7f108409e000 2023-09-30T22:22:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.841+0000 7f107affd700 1 -- 172.21.15.89:0/793835977 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1074013470 con 0x7f108409e000 2023-09-30T22:22:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.841+0000 7f107affd700 1 -- 172.21.15.89:0/793835977 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f107403ff60 con 0x7f108409e000 2023-09-30T22:22:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.848+0000 7f107affd700 1 -- 172.21.15.89:0/793835977 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f107406b750 con 0x7f108409e000 2023-09-30T22:22:40.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.951+0000 7f1693337700 1 Processor -- start 2023-09-30T22:22:40.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.951+0000 7f1693337700 1 -- start start 2023-09-30T22:22:40.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.952+0000 7f1693337700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f168c130540 0x7f168c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.952+0000 7f1693337700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.952+0000 7f1693337700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168c0716a0 con 0x7f168c130e50 2023-09-30T22:22:40.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.952+0000 7f1693337700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168c09a390 con 0x7f168c130540 2023-09-30T22:22:40.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.952+0000 7f1693337700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c071290 con 0x7f168c06fcf0 2023-09-30T22:22:40.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f16918d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f16918d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51218/0 (socket says 172.21.15.89:51218) 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f16918d4700 1 -- 172.21.15.89:0/1743954435 learned_addr learned my addr 172.21.15.89:0/1743954435 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f16918d4700 1 -- 172.21.15.89:0/1743954435 >> v2:172.21.15.89:3300/0 conn(0x7f168c130540 msgr2=0x7f168c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f16918d4700 1 --2- 172.21.15.89:0/1743954435 >> v2:172.21.15.89:3300/0 conn(0x7f168c130540 0x7f168c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f16918d4700 1 -- 172.21.15.89:0/1743954435 >> v1:172.21.15.89:6789/0 conn(0x7f168c130e50 legacy=0x7f168c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f16918d4700 1 -- 172.21.15.89:0/1743954435 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c130340 con 0x7f168c06fcf0 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f16918d4700 1 --2- 172.21.15.89:0/1743954435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c138570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1684009e80 tx=0x7f168400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2abd39aaaf3599ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f1683fff700 1 -- 172.21.15.89:0/1743954435 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168400e070 con 0x7f168c06fcf0 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f1683fff700 1 -- 172.21.15.89:0/1743954435 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1684003a20 con 0x7f168c06fcf0 2023-09-30T22:22:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.953+0000 7f1683fff700 1 -- 172.21.15.89:0/1743954435 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684003dd0 con 0x7f168c06fcf0 2023-09-30T22:22:40.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.954+0000 7f1693337700 1 -- 172.21.15.89:0/1743954435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 msgr2=0x7f168c138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.954+0000 7f1693337700 1 --2- 172.21.15.89:0/1743954435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c138570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1684009e80 tx=0x7f168400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:40.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.954+0000 7f1693337700 1 -- 172.21.15.89:0/1743954435 shutdown_connections 2023-09-30T22:22:40.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.954+0000 7f1693337700 1 --2- 172.21.15.89:0/1743954435 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c138570 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.954+0000 7f1693337700 1 --2- 172.21.15.89:0/1743954435 >> v2:172.21.15.89:3300/0 conn(0x7f168c130540 0x7f168c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.954+0000 7f1693337700 1 -- 172.21.15.89:0/1743954435 >> 172.21.15.89:0/1743954435 conn(0x7f168c0a2aa0 msgr2=0x7f168c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:40.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.954+0000 7f1693337700 1 -- 172.21.15.89:0/1743954435 shutdown_connections 2023-09-30T22:22:40.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.954+0000 7f1693337700 1 -- 172.21.15.89:0/1743954435 wait complete. 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.955+0000 7f1693337700 1 Processor -- start 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.955+0000 7f1693337700 1 -- start start 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f1693337700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f1693337700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f168c130540 0x7f168c13a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f1693337700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f168c130e50 0x7f168c13a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f1693337700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c13acd0 con 0x7f168c130e50 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f1693337700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c0a0830 con 0x7f168c06fcf0 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f1693337700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f168c138d40 con 0x7f168c130540 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16910d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c139ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16910d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c139ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51226/0 (socket says 172.21.15.89:51226) 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16910d3700 1 -- 172.21.15.89:0/516301218 learned_addr learned my addr 172.21.15.89:0/516301218 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16908d2700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f168c130540 0x7f168c13a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16918d4700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f168c130e50 0x7f168c13a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16910d3700 1 -- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f168c130540 msgr2=0x7f168c13a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16910d3700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f168c130540 0x7f168c13a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16910d3700 1 -- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f168c130e50 msgr2=0x7f168c13a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16910d3700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f168c130e50 0x7f168c13a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.956+0000 7f16910d3700 1 -- 172.21.15.89:0/516301218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009b30 con 0x7f168c06fcf0 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.957+0000 7f16910d3700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c139ac0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f168800b530 tx=0x7f168800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.957+0000 7f1681ffb700 1 -- 172.21.15.89:0/516301218 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011830 con 0x7f168c06fcf0 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.957+0000 7f1681ffb700 1 -- 172.21.15.89:0/516301218 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1688011990 con 0x7f168c06fcf0 2023-09-30T22:22:40.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.957+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c13b4b0 con 0x7f168c06fcf0 2023-09-30T22:22:40.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.957+0000 7f1681ffb700 1 -- 172.21.15.89:0/516301218 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011d40 con 0x7f168c06fcf0 2023-09-30T22:22:40.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.957+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168c119170 con 0x7f168c06fcf0 2023-09-30T22:22:40.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.958+0000 7f1681ffb700 1 -- 172.21.15.89:0/516301218 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f168801a400 con 0x7f168c06fcf0 2023-09-30T22:22:40.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.958+0000 7f1681ffb700 1 -- 172.21.15.89:0/516301218 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f168803f6b0 con 0x7f168c06fcf0 2023-09-30T22:22:40.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168c074ea0 con 0x7f168c06fcf0 2023-09-30T22:22:40.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f430d38c700 1 Processor -- start 2023-09-30T22:22:40.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f430d38c700 1 -- start start 2023-09-30T22:22:40.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f430d38c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4308131630 0x7f4308131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f430d38c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4308131f40 0x7f430806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f430d38c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f430806e9d0 0x7f430806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f430d38c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4308071a00 con 0x7f430806e9d0 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f430d38c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4308131430 con 0x7f4308131630 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f430d38c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f430809a510 con 0x7f4308131f40 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f4306ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4308131630 0x7f4308131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.961+0000 7f4306ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4308131630 0x7f4308131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51234/0 (socket says 172.21.15.89:51234) 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.962+0000 7f4306ffd700 1 -- 172.21.15.89:0/268103982 learned_addr learned my addr 172.21.15.89:0/268103982 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.962+0000 7f4306ffd700 1 -- 172.21.15.89:0/268103982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4308131f40 msgr2=0x7f430806e490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.962+0000 7f4306ffd700 1 --2- 172.21.15.89:0/268103982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4308131f40 0x7f430806e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.962+0000 7f4306ffd700 1 -- 172.21.15.89:0/268103982 >> v2:172.21.15.89:3300/0 conn(0x7f430806e9d0 msgr2=0x7f430806ee70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:22:40.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.962+0000 7f4306ffd700 1 --2- 172.21.15.89:0/268103982 >> v2:172.21.15.89:3300/0 conn(0x7f430806e9d0 0x7f430806ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.962+0000 7f4306ffd700 1 -- 172.21.15.89:0/268103982 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43080721c0 con 0x7f4308131630 2023-09-30T22:22:40.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.962+0000 7f4306ffd700 1 --2- 172.21.15.89:0/268103982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4308131630 0x7f4308131a00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f42f80098f0 tx=0x7f42f8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56165f30d52ef611 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.965+0000 7f4305ffb700 1 -- 172.21.15.89:0/268103982 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800b8a0 con 0x7f4308131630 2023-09-30T22:22:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.965+0000 7f4305ffb700 1 -- 172.21.15.89:0/268103982 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f800ba00 con 0x7f4308131630 2023-09-30T22:22:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.965+0000 7f4305ffb700 1 -- 172.21.15.89:0/268103982 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800bd90 con 0x7f4308131630 2023-09-30T22:22:40.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.965+0000 7f1681ffb700 1 -- 172.21.15.89:0/516301218 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1688015080 con 0x7f168c06fcf0 2023-09-30T22:22:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.966+0000 7f430d38c700 1 -- 172.21.15.89:0/268103982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4308131630 msgr2=0x7f4308131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.966+0000 7f430d38c700 1 --2- 172.21.15.89:0/268103982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4308131630 0x7f4308131a00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f42f80098f0 tx=0x7f42f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.966+0000 7f430d38c700 1 -- 172.21.15.89:0/268103982 shutdown_connections 2023-09-30T22:22:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.966+0000 7f430d38c700 1 --2- 172.21.15.89:0/268103982 >> v2:172.21.15.89:3300/0 conn(0x7f430806e9d0 0x7f430806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.966+0000 7f430d38c700 1 --2- 172.21.15.89:0/268103982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4308131630 0x7f4308131a00 secure :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f42f80098f0 tx=0x7f42f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:40.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.966+0000 7f430d38c700 1 --2- 172.21.15.89:0/268103982 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4308131f40 0x7f430806e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.966+0000 7f430d38c700 1 -- 172.21.15.89:0/268103982 >> 172.21.15.89:0/268103982 conn(0x7f43080a2aa0 msgr2=0x7f43080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:40.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.971+0000 7f430d38c700 1 -- 172.21.15.89:0/268103982 shutdown_connections 2023-09-30T22:22:40.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.971+0000 7f430d38c700 1 -- 172.21.15.89:0/268103982 wait complete. 2023-09-30T22:22:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f430d38c700 1 Processor -- start 2023-09-30T22:22:40.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f430d38c700 1 -- start start 2023-09-30T22:22:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f430d38c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430806e9d0 0x7f43081353b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f430d38c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4308131f40 0x7f43081358f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f430d38c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43081376e0 0x7f4308135e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f430d38c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430809fb80 con 0x7f4308131f40 2023-09-30T22:22:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f430d38c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43081195e0 con 0x7f430806e9d0 2023-09-30T22:22:40.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f430d38c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43081367a0 con 0x7f43081376e0 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f4306ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430806e9d0 0x7f43081353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.972+0000 7f43077fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43081376e0 0x7f4308135e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f4306ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430806e9d0 0x7f43081353b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51248/0 (socket says 172.21.15.89:51248) 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f4306ffd700 1 -- 172.21.15.89:0/407314831 learned_addr learned my addr 172.21.15.89:0/407314831 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f43067fc700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4308131f40 0x7f43081358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f43077fe700 1 -- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430806e9d0 msgr2=0x7f43081353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f43077fe700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430806e9d0 0x7f43081353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f43077fe700 1 -- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4308131f40 msgr2=0x7f43081358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f43077fe700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4308131f40 0x7f43081358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f43077fe700 1 -- 172.21.15.89:0/407314831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f8009550 con 0x7f43081376e0 2023-09-30T22:22:40.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f43067fc700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4308131f40 0x7f43081358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f43077fe700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43081376e0 0x7f4308135e30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f430000be60 tx=0x7f4300009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f42ef7fe700 1 -- 172.21.15.89:0/407314831 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430000cc10 con 0x7f43081376e0 2023-09-30T22:22:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f42ef7fe700 1 -- 172.21.15.89:0/407314831 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430000cd90 con 0x7f43081376e0 2023-09-30T22:22:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4308118f30 con 0x7f43081376e0 2023-09-30T22:22:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f42ef7fe700 1 -- 172.21.15.89:0/407314831 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300014680 con 0x7f43081376e0 2023-09-30T22:22:40.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.973+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4308119c60 con 0x7f43081376e0 2023-09-30T22:22:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.975+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4308074ea0 con 0x7f43081376e0 2023-09-30T22:22:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.975+0000 7f42ef7fe700 1 -- 172.21.15.89:0/407314831 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f430001d3b0 con 0x7f43081376e0 2023-09-30T22:22:40.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.975+0000 7f42ef7fe700 1 -- 172.21.15.89:0/407314831 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f430003fa40 con 0x7f43081376e0 2023-09-30T22:22:40.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:40.980+0000 7f42ef7fe700 1 -- 172.21.15.89:0/407314831 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f430001c360 con 0x7f43081376e0 2023-09-30T22:22:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.014+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb32c072980 con 0x7fb32c0a0a30 2023-09-30T22:22:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.014+0000 7fb3297fa700 1 -- 172.21.15.89:0/3287443989 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb318019b20 con 0x7fb32c0a0a30 2023-09-30T22:22:41.015 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 msgr2=0x7fb32c133d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c133d10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb32c06d980 tx=0x7fb31800c710 comp rx=0 tx=0).stop 2023-09-30T22:22:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 shutdown_connections 2023-09-30T22:22:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb32c138d80 0x7fb32c134250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb32c09d630 0x7fb32c1337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 --2- 172.21.15.89:0/3287443989 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb32c0a0a30 0x7fb32c133d10 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 >> 172.21.15.89:0/3287443989 conn(0x7fb32c0a3050 msgr2=0x7fb32c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 shutdown_connections 2023-09-30T22:22:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.016+0000 7fb3331fb700 1 -- 172.21.15.89:0/3287443989 wait complete. 2023-09-30T22:22:41.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:41.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.046+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f106c005860 con 0x7f108409e000 2023-09-30T22:22:41.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.046+0000 7f107affd700 1 -- 172.21.15.89:0/793835977 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1074021030 con 0x7f108409e000 2023-09-30T22:22:41.047 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 msgr2=0x7f1084122760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084122760 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f107400ebb0 tx=0x7f107400eec0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 shutdown_connections 2023-09-30T22:22:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f108409e410 0x7f1084122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f108409d6f0 0x7f1084122220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 --2- 172.21.15.89:0/793835977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f108409e000 0x7f1084122760 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 >> 172.21.15.89:0/793835977 conn(0x7f1084066f50 msgr2=0x7f1084130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 shutdown_connections 2023-09-30T22:22:41.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.048+0000 7f108c4af700 1 -- 172.21.15.89:0/793835977 wait complete. 2023-09-30T22:22:41.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.176+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f168c06b9e0 con 0x7f168c06fcf0 2023-09-30T22:22:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.177+0000 7f1681ffb700 1 -- 172.21.15.89:0/516301218 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f168800f8d0 con 0x7f168c06fcf0 2023-09-30T22:22:41.177 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:41.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 msgr2=0x7f168c139ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c139ac0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f168800b530 tx=0x7f168800b840 comp rx=0 tx=0).stop 2023-09-30T22:22:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 shutdown_connections 2023-09-30T22:22:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f168c130e50 0x7f168c13a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f168c06fcf0 0x7f168c139ac0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 --2- 172.21.15.89:0/516301218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f168c130540 0x7f168c13a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 >> 172.21.15.89:0/516301218 conn(0x7f168c0a2aa0 msgr2=0x7f168c12e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 shutdown_connections 2023-09-30T22:22:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.179+0000 7f1693337700 1 -- 172.21.15.89:0/516301218 wait complete. 2023-09-30T22:22:41.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.188+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f430806b9e0 con 0x7f43081376e0 2023-09-30T22:22:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.188+0000 7f42ef7fe700 1 -- 172.21.15.89:0/407314831 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4300019030 con 0x7f43081376e0 2023-09-30T22:22:41.189 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:41.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.190+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43081376e0 msgr2=0x7f4308135e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.190+0000 7f430d38c700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43081376e0 0x7f4308135e30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f430000be60 tx=0x7f4300009ed0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.191+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 shutdown_connections 2023-09-30T22:22:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.191+0000 7f430d38c700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f43081376e0 0x7f4308135e30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.191+0000 7f430d38c700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430806e9d0 0x7f43081353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.191+0000 7f430d38c700 1 --2- 172.21.15.89:0/407314831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4308131f40 0x7f43081358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.191+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 >> 172.21.15.89:0/407314831 conn(0x7f43080a2aa0 msgr2=0x7f43080a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.191+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 shutdown_connections 2023-09-30T22:22:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:41.191+0000 7f430d38c700 1 -- 172.21.15.89:0/407314831 wait complete. 2023-09-30T22:22:41.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:42.030 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:42.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:42.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.169+0000 7fd9ad8aa700 1 Processor -- start 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.169+0000 7fd9ad8aa700 1 -- start start 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9ad8aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a0a30 0x7fd9a80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9ad8aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a1340 0x7fd9a8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9ad8aa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd9a8133fa0 0x7fd9a8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9ad8aa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a809edd0 con 0x7fd9a8133fa0 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9ad8aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a809e1f0 con 0x7fd9a80a0a30 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9ad8aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a809dc50 con 0x7fd9a80a1340 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd9a8133fa0 0x7fd9a8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a0a30 0x7fd9a80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a67fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a1340 0x7fd9a8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd9a8133fa0 0x7fd9a8134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56914/0 (socket says 172.21.15.89:56914) 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a0a30 0x7fd9a80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51252/0 (socket says 172.21.15.89:51252) 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 -- 172.21.15.89:0/1743360974 learned_addr learned my addr 172.21.15.89:0/1743360974 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 -- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a1340 msgr2=0x7fd9a8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 --2- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a1340 0x7fd9a8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 -- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a0a30 msgr2=0x7fd9a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 --2- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a0a30 0x7fd9a80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 -- 172.21.15.89:0/1743360974 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a80a1fe0 con 0x7fd9a8133fa0 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a6ffd700 1 --2- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a0a30 0x7fd9a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.170+0000 7fd9a77fe700 1 --2- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 0x7fd9a8134410 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009e50 tx=0x7fd9a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2afe4cda925a8f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9a5ffb700 1 -- 172.21.15.89:0/1743360974 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000e070 con 0x7fd9a8133fa0 2023-09-30T22:22:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9a5ffb700 1 -- 172.21.15.89:0/1743360974 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a000b910 con 0x7fd9a8133fa0 2023-09-30T22:22:42.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9a5ffb700 1 -- 172.21.15.89:0/1743360974 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000bcc0 con 0x7fd9a8133fa0 2023-09-30T22:22:42.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 msgr2=0x7fd9a8134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 --2- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 0x7fd9a8134410 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009e50 tx=0x7fd9a000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:42.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/1743360974 shutdown_connections 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 --2- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 0x7fd9a8134410 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 --2- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a0a30 0x7fd9a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 --2- 172.21.15.89:0/1743360974 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a1340 0x7fd9a8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/1743360974 >> 172.21.15.89:0/1743360974 conn(0x7fd9a80a6550 msgr2=0x7fd9a80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/1743360974 shutdown_connections 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.171+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/1743360974 wait complete. 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.172+0000 7fd9ad8aa700 1 Processor -- start 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.172+0000 7fd9ad8aa700 1 -- start start 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9ad8aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a0a30 0x7fd9a81c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9ad8aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a1340 0x7fd9a81c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9ad8aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 0x7fd9a81c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9ad8aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81337e0 con 0x7fd9a8133fa0 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9ad8aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80a6d90 con 0x7fd9a80a1340 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9ad8aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80a0220 con 0x7fd9a80a0a30 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a1340 0x7fd9a81c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a67fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a1340 0x7fd9a81c13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51258/0 (socket says 172.21.15.89:51258) 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a67fc700 1 -- 172.21.15.89:0/418117178 learned_addr learned my addr 172.21.15.89:0/418117178 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a77fe700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 0x7fd9a81c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a6ffd700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a0a30 0x7fd9a81c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a67fc700 1 -- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a0a30 msgr2=0x7fd9a81c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a67fc700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a0a30 0x7fd9a81c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a67fc700 1 -- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 msgr2=0x7fd9a81c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a67fc700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 0x7fd9a81c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a67fc700 1 -- 172.21.15.89:0/418117178 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980094a0 con 0x7fd9a80a1340 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a77fe700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 0x7fd9a81c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.173+0000 7fd9a6ffd700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a0a30 0x7fd9a81c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.174+0000 7fd9a67fc700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a1340 0x7fd9a81c13c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd99c014360 tx=0x7fd99c014670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.174+0000 7fd98ffff700 1 -- 172.21.15.89:0/418117178 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c016050 con 0x7fd9a80a1340 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.174+0000 7fd98ffff700 1 -- 172.21.15.89:0/418117178 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c017040 con 0x7fd9a80a1340 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.174+0000 7fd98ffff700 1 -- 172.21.15.89:0/418117178 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0112d0 con 0x7fd9a80a1340 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.174+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a0009b30 con 0x7fd9a80a1340 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.174+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a8121680 con 0x7fd9a80a1340 2023-09-30T22:22:42.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.175+0000 7fd98ffff700 1 -- 172.21.15.89:0/418117178 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd99c011430 con 0x7fd9a80a1340 2023-09-30T22:22:42.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.175+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a80a1cc0 con 0x7fd9a80a1340 2023-09-30T22:22:42.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.175+0000 7fd98ffff700 1 -- 172.21.15.89:0/418117178 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd99c047540 con 0x7fd9a80a1340 2023-09-30T22:22:42.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.179+0000 7fd98ffff700 1 -- 172.21.15.89:0/418117178 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd99c00f850 con 0x7fd9a80a1340 2023-09-30T22:22:42.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:42.204 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:42.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.208+0000 7f5d9a574700 1 Processor -- start 2023-09-30T22:22:42.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.208+0000 7f5d9a574700 1 -- start start 2023-09-30T22:22:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d9a574700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d941316c0 0x7f5d94131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d9a574700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d94139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d9a574700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9409e130 con 0x7f5d9409d9f0 2023-09-30T22:22:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d9a574700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941312b0 con 0x7f5d9406ae80 2023-09-30T22:22:42.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d9a574700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9409f6e0 con 0x7f5d941316c0 2023-09-30T22:22:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d93fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d941316c0 0x7f5d94131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d98b11700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d94139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d93fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d941316c0 0x7f5d94131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55622/0 (socket says 172.21.15.89:55622) 2023-09-30T22:22:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d98b11700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d94139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51260/0 (socket says 172.21.15.89:51260) 2023-09-30T22:22:42.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d93fff700 1 -- 172.21.15.89:0/584076476 learned_addr learned my addr 172.21.15.89:0/584076476 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:42.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d98b11700 1 -- 172.21.15.89:0/584076476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d941316c0 msgr2=0x7f5d94131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:42.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d98b11700 1 --2- 172.21.15.89:0/584076476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d941316c0 0x7f5d94131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d98b11700 1 -- 172.21.15.89:0/584076476 >> v1:172.21.15.89:6789/0 conn(0x7f5d9409d9f0 legacy=0x7f5d9406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d98b11700 1 -- 172.21.15.89:0/584076476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9409e760 con 0x7f5d9406ae80 2023-09-30T22:22:42.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d98b11700 1 --2- 172.21.15.89:0/584076476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d94139550 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009dc0 tx=0x7f5d8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1fc45b8ea9c442d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:42.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.209+0000 7f5d92ffd700 1 -- 172.21.15.89:0/584076476 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00e070 con 0x7f5d9406ae80 2023-09-30T22:22:42.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.210+0000 7f5d92ffd700 1 -- 172.21.15.89:0/584076476 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c00bb20 con 0x7f5d9406ae80 2023-09-30T22:22:42.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.210+0000 7f5d92ffd700 1 -- 172.21.15.89:0/584076476 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c012430 con 0x7f5d9406ae80 2023-09-30T22:22:42.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.210+0000 7f5d9a574700 1 -- 172.21.15.89:0/584076476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 msgr2=0x7f5d94139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.210+0000 7f5d9a574700 1 --2- 172.21.15.89:0/584076476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d94139550 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009dc0 tx=0x7f5d8c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:42.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.210+0000 7f5d9a574700 1 -- 172.21.15.89:0/584076476 shutdown_connections 2023-09-30T22:22:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.210+0000 7f5d9a574700 1 --2- 172.21.15.89:0/584076476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d94139550 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.210+0000 7f5d9a574700 1 --2- 172.21.15.89:0/584076476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d941316c0 0x7f5d94131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.210+0000 7f5d9a574700 1 -- 172.21.15.89:0/584076476 >> 172.21.15.89:0/584076476 conn(0x7f5d94066f50 msgr2=0x7f5d9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.211+0000 7f5d9a574700 1 -- 172.21.15.89:0/584076476 shutdown_connections 2023-09-30T22:22:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.211+0000 7f5d9a574700 1 -- 172.21.15.89:0/584076476 wait complete. 2023-09-30T22:22:42.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.212+0000 7f5d9a574700 1 Processor -- start 2023-09-30T22:22:42.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.212+0000 7f5d9a574700 1 -- start start 2023-09-30T22:22:42.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d9a574700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d9411a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d9a574700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d9409d9f0 0x7f5d9411a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d9a574700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d941316c0 0x7f5d9411aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d9a574700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9411cdf0 con 0x7f5d941316c0 2023-09-30T22:22:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d9a574700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9411bff0 con 0x7f5d9406ae80 2023-09-30T22:22:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d9a574700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940a3760 con 0x7f5d9409d9f0 2023-09-30T22:22:42.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d93fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d9411a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d93fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d9411a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51268/0 (socket says 172.21.15.89:51268) 2023-09-30T22:22:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d93fff700 1 -- 172.21.15.89:0/1219134257 learned_addr learned my addr 172.21.15.89:0/1219134257 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d98b11700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d941316c0 0x7f5d9411aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d937fe700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d9409d9f0 0x7f5d9411a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d98b11700 1 -- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d9409d9f0 msgr2=0x7f5d9411a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d98b11700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d9409d9f0 0x7f5d9411a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d98b11700 1 -- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 msgr2=0x7f5d9411a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d98b11700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d9411a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d98b11700 1 -- 172.21.15.89:0/1219134257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c009a70 con 0x7f5d941316c0 2023-09-30T22:22:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.213+0000 7f5d98b11700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d941316c0 0x7f5d9411aeb0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f5d9406b800 tx=0x7f5d8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:42.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.214+0000 7f5d90ff9700 1 -- 172.21.15.89:0/1219134257 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c004800 con 0x7f5d941316c0 2023-09-30T22:22:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.214+0000 7f5d90ff9700 1 -- 172.21.15.89:0/1219134257 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c012430 con 0x7f5d941316c0 2023-09-30T22:22:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.214+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9412b900 con 0x7f5d941316c0 2023-09-30T22:22:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.214+0000 7f5d90ff9700 1 -- 172.21.15.89:0/1219134257 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0127e0 con 0x7f5d941316c0 2023-09-30T22:22:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.214+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9411dc20 con 0x7f5d941316c0 2023-09-30T22:22:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.215+0000 7f5d90ff9700 1 -- 172.21.15.89:0/1219134257 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d8c012940 con 0x7f5d941316c0 2023-09-30T22:22:42.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.215+0000 7f5d90ff9700 1 -- 172.21.15.89:0/1219134257 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5d8c041310 con 0x7f5d941316c0 2023-09-30T22:22:42.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.215+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9411de10 con 0x7f5d941316c0 2023-09-30T22:22:42.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.221+0000 7f5d90ff9700 1 -- 172.21.15.89:0/1219134257 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d8c012bf0 con 0x7f5d941316c0 2023-09-30T22:22:42.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.334+0000 7f9ed896c700 1 Processor -- start 2023-09-30T22:22:42.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.334+0000 7f9ed896c700 1 -- start start 2023-09-30T22:22:42.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed896c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133420 0x7f9ed41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed896c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ed4133e30 0x7f9ed412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed896c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed406b990 con 0x7f9ed4134260 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed896c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed40a0590 con 0x7f9ed4133e30 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed896c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed409fb80 con 0x7f9ed4133420 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed1d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ed4133e30 0x7f9ed412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed1d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ed4133e30 0x7f9ed412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56936/0 (socket says 172.21.15.89:56936) 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed1d9b700 1 -- 172.21.15.89:0/859147701 learned_addr learned my addr 172.21.15.89:0/859147701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed2d9d700 1 --1- 172.21.15.89:0/859147701 >> v1:172.21.15.89:6789/0 conn(0x7f9ed4134260 0x7f9ed41304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57890/0 (socket says 172.21.15.89:57890) 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.335+0000 7f9ed259c700 1 --2- 172.21.15.89:0/859147701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133420 0x7f9ed41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.336+0000 7f9ed1d9b700 1 -- 172.21.15.89:0/859147701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133420 msgr2=0x7f9ed41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:42.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.336+0000 7f9ed1d9b700 1 --2- 172.21.15.89:0/859147701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133420 0x7f9ed41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.336+0000 7f9ed1d9b700 1 -- 172.21.15.89:0/859147701 >> v1:172.21.15.89:6789/0 conn(0x7f9ed4134260 legacy=0x7f9ed41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.336+0000 7f9ed1d9b700 1 -- 172.21.15.89:0/859147701 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40a0030 con 0x7f9ed4133e30 2023-09-30T22:22:42.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.336+0000 7f9ed1d9b700 1 --2- 172.21.15.89:0/859147701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133e30 0x7f9ed412ff90 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80098f0 tx=0x7f9ec8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54777d198f91e20 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:42.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.337+0000 7f9ed159a700 1 -- 172.21.15.89:0/859147701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8011830 con 0x7f9ed4133e30 2023-09-30T22:22:42.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.337+0000 7f9ed159a700 1 -- 172.21.15.89:0/859147701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec8011990 con 0x7f9ed4133e30 2023-09-30T22:22:42.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.337+0000 7f9ed159a700 1 -- 172.21.15.89:0/859147701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8011d20 con 0x7f9ed4133e30 2023-09-30T22:22:42.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.337+0000 7f9ed896c700 1 -- 172.21.15.89:0/859147701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133e30 msgr2=0x7f9ed412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.337+0000 7f9ed896c700 1 --2- 172.21.15.89:0/859147701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133e30 0x7f9ed412ff90 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80098f0 tx=0x7f9ec8009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:42.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.338+0000 7f9ed896c700 1 -- 172.21.15.89:0/859147701 shutdown_connections 2023-09-30T22:22:42.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.338+0000 7f9ed896c700 1 --2- 172.21.15.89:0/859147701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133420 0x7f9ed41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.338+0000 7f9ed896c700 1 --2- 172.21.15.89:0/859147701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133e30 0x7f9ed412ff90 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.338+0000 7f9ed896c700 1 -- 172.21.15.89:0/859147701 >> 172.21.15.89:0/859147701 conn(0x7f9ed40a4210 msgr2=0x7f9ed40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:42.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.338+0000 7f9ed896c700 1 -- 172.21.15.89:0/859147701 shutdown_connections 2023-09-30T22:22:42.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.338+0000 7f9ed896c700 1 -- 172.21.15.89:0/859147701 wait complete. 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.339+0000 7f9ed896c700 1 Processor -- start 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.339+0000 7f9ed896c700 1 -- start start 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed896c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133420 0x7f9ed409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed896c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133e30 0x7f9ed409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed896c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9ed4134260 0x7f9ed409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed896c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed4131a90 con 0x7f9ed4133420 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed896c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed409de30 con 0x7f9ed4134260 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed896c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed411fcc0 con 0x7f9ed4133e30 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed1d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133e30 0x7f9ed409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9ed4134260 0x7f9ed409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed1d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133e30 0x7f9ed409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55636/0 (socket says 172.21.15.89:55636) 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed1d9b700 1 -- 172.21.15.89:0/296330886 learned_addr learned my addr 172.21.15.89:0/296330886 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9ed4134260 0x7f9ed409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51276/0 (socket says 172.21.15.89:51276) 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed259c700 1 --2- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133420 0x7f9ed409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed1d9b700 1 -- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9ed4134260 msgr2=0x7f9ed409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed1d9b700 1 --2- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9ed4134260 0x7f9ed409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed1d9b700 1 -- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133420 msgr2=0x7f9ed409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed1d9b700 1 --2- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133420 0x7f9ed409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.340+0000 7f9ed1d9b700 1 -- 172.21.15.89:0/296330886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8009550 con 0x7f9ed4133e30 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.341+0000 7f9ed1d9b700 1 --2- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133e30 0x7f9ed409c9c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8013000 tx=0x7f9ec8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.341+0000 7f9ec2ffd700 1 -- 172.21.15.89:0/296330886 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8004550 con 0x7f9ed4133e30 2023-09-30T22:22:42.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.341+0000 7f9ec2ffd700 1 -- 172.21.15.89:0/296330886 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec80036a0 con 0x7f9ed4133e30 2023-09-30T22:22:42.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.341+0000 7f9ec2ffd700 1 -- 172.21.15.89:0/296330886 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8003a30 con 0x7f9ed4133e30 2023-09-30T22:22:42.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.341+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed412d670 con 0x7f9ed4133e30 2023-09-30T22:22:42.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.341+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed41209f0 con 0x7f9ed4133e30 2023-09-30T22:22:42.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.342+0000 7f9ec2ffd700 1 -- 172.21.15.89:0/296330886 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ec8021070 con 0x7f9ed4133e30 2023-09-30T22:22:42.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.342+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed4074ea0 con 0x7f9ed4133e30 2023-09-30T22:22:42.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.342+0000 7f9ec2ffd700 1 -- 172.21.15.89:0/296330886 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9ec8040af0 con 0x7f9ed4133e30 2023-09-30T22:22:42.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.344+0000 7f78bb76c700 1 Processor -- start 2023-09-30T22:22:42.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.344+0000 7f78bb76c700 1 -- start start 2023-09-30T22:22:42.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78bb76c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b41305f0 0x7f78b41309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78bb76c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78b406f660 0x7f78b406fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78bb76c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b406ff90 0x7f78b4137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78bb76c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b41301c0 con 0x7f78b406f660 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78bb76c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4071ff0 con 0x7f78b41305f0 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78bb76c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4070e70 con 0x7f78b406ff90 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78b8d07700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78b406f660 0x7f78b406fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78b8d07700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f78b406f660 0x7f78b406fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56954/0 (socket says 172.21.15.89:56954) 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78b8d07700 1 -- 172.21.15.89:0/933376301 learned_addr learned my addr 172.21.15.89:0/933376301 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78b9508700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b41305f0 0x7f78b41309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.345+0000 7f78b9d09700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b406ff90 0x7f78b4137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78b8d07700 1 -- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b406ff90 msgr2=0x7f78b4137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78b8d07700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b406ff90 0x7f78b4137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78b8d07700 1 -- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b41305f0 msgr2=0x7f78b41309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78b8d07700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b41305f0 0x7f78b41309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78b8d07700 1 -- 172.21.15.89:0/933376301 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40722b0 con 0x7f78b406f660 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78b8d07700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b406fa50 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f78a400bb90 tx=0x7f78a400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d14a451b8486de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78abfff700 1 -- 172.21.15.89:0/933376301 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400dc70 con 0x7f78b406f660 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78abfff700 1 -- 172.21.15.89:0/933376301 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a400ddd0 con 0x7f78b406f660 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.346+0000 7f78abfff700 1 -- 172.21.15.89:0/933376301 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4012990 con 0x7f78b406f660 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 -- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 msgr2=0x7f78b406fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b406fa50 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f78a400bb90 tx=0x7f78a400bea0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 -- 172.21.15.89:0/933376301 shutdown_connections 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b406ff90 0x7f78b4137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b41305f0 0x7f78b41309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 --2- 172.21.15.89:0/933376301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b406fa50 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 -- 172.21.15.89:0/933376301 >> 172.21.15.89:0/933376301 conn(0x7f78b40a2aa0 msgr2=0x7f78b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:42.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 -- 172.21.15.89:0/933376301 shutdown_connections 2023-09-30T22:22:42.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.347+0000 7f78bb76c700 1 -- 172.21.15.89:0/933376301 wait complete. 2023-09-30T22:22:42.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78bb76c700 1 Processor -- start 2023-09-30T22:22:42.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78bb76c700 1 -- start start 2023-09-30T22:22:42.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78bb76c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b411b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78bb76c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b406ff90 0x7f78b411bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78bb76c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b41305f0 0x7f78b411c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78bb76c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b411f0a0 con 0x7f78b406f660 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78bb76c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b411dd00 con 0x7f78b406ff90 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78bb76c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78b412ecf0 con 0x7f78b41305f0 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b9508700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b411b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b9508700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b411b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56964/0 (socket says 172.21.15.89:56964) 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b9508700 1 -- 172.21.15.89:0/2341315721 learned_addr learned my addr 172.21.15.89:0/2341315721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b9d09700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b41305f0 0x7f78b411c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b8d07700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b406ff90 0x7f78b411bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:42.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b9508700 1 -- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b41305f0 msgr2=0x7f78b411c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b9508700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b41305f0 0x7f78b411c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b9508700 1 -- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b406ff90 msgr2=0x7f78b411bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.348+0000 7f78b9508700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b406ff90 0x7f78b411bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.349+0000 7f78b9508700 1 -- 172.21.15.89:0/2341315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a400b7f0 con 0x7f78b406f660 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.349+0000 7f78b8d07700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b406ff90 0x7f78b411bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.349+0000 7f78b9508700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b411b8a0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f78b000b3f0 tx=0x7f78b000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.349+0000 7f78a9ffb700 1 -- 172.21.15.89:0/2341315721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0019070 con 0x7f78b406f660 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.349+0000 7f78a9ffb700 1 -- 172.21.15.89:0/2341315721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b001d410 con 0x7f78b406f660 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.349+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b411cc90 con 0x7f78b406f660 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.349+0000 7f78a9ffb700 1 -- 172.21.15.89:0/2341315721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0014af0 con 0x7f78b406f660 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.349+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b411eb10 con 0x7f78b406f660 2023-09-30T22:22:42.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.350+0000 7f78a9ffb700 1 -- 172.21.15.89:0/2341315721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f78b001d780 con 0x7f78b406f660 2023-09-30T22:22:42.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.350+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b4074ea0 con 0x7f78b406f660 2023-09-30T22:22:42.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.350+0000 7f78a9ffb700 1 -- 172.21.15.89:0/2341315721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f78b0047300 con 0x7f78b406f660 2023-09-30T22:22:42.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.351+0000 7f9ec2ffd700 1 -- 172.21.15.89:0/296330886 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ec801b430 con 0x7f9ed4133e30 2023-09-30T22:22:42.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.359+0000 7f78a9ffb700 1 -- 172.21.15.89:0/2341315721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78b001b900 con 0x7f78b406f660 2023-09-30T22:22:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.425+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd9a8074ea0 con 0x7fd9a80a1340 2023-09-30T22:22:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.426+0000 7fd98ffff700 1 -- 172.21.15.89:0/418117178 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd99c022310 con 0x7fd9a80a1340 2023-09-30T22:22:42.427 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:42.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.428+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a1340 msgr2=0x7fd9a81c13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.428+0000 7fd9ad8aa700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a1340 0x7fd9a81c13c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd99c014360 tx=0x7fd99c014670 comp rx=0 tx=0).stop 2023-09-30T22:22:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.429+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 shutdown_connections 2023-09-30T22:22:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.429+0000 7fd9ad8aa700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9a8133fa0 0x7fd9a81c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.429+0000 7fd9ad8aa700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9a80a0a30 0x7fd9a81c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.429+0000 7fd9ad8aa700 1 --2- 172.21.15.89:0/418117178 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9a80a1340 0x7fd9a81c13c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.429+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 >> 172.21.15.89:0/418117178 conn(0x7fd9a80a6550 msgr2=0x7fd9a809c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.430+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 shutdown_connections 2023-09-30T22:22:42.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.431+0000 7fd9ad8aa700 1 -- 172.21.15.89:0/418117178 wait complete. 2023-09-30T22:22:42.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.437+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5d9409eaa0 con 0x7f5d941316c0 2023-09-30T22:22:42.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.438+0000 7f5d90ff9700 1 -- 172.21.15.89:0/1219134257 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d8c01ab70 con 0x7f5d941316c0 2023-09-30T22:22:42.438 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:42.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.441+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d941316c0 msgr2=0x7f5d9411aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.441+0000 7f5d9a574700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d941316c0 0x7f5d9411aeb0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f5d9406b800 tx=0x7f5d8c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.441+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 shutdown_connections 2023-09-30T22:22:42.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.441+0000 7f5d9a574700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d941316c0 0x7f5d9411aeb0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.441+0000 7f5d9a574700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d9406ae80 0x7f5d9411a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.441+0000 7f5d9a574700 1 --2- 172.21.15.89:0/1219134257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d9409d9f0 0x7f5d9411a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.441+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 >> 172.21.15.89:0/1219134257 conn(0x7f5d94066f50 msgr2=0x7f5d940a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:42.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.442+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 shutdown_connections 2023-09-30T22:22:42.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.442+0000 7f5d9a574700 1 -- 172.21.15.89:0/1219134257 wait complete. 2023-09-30T22:22:42.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:42.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:42.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.557+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9ed406b250 con 0x7f9ed4133e30 2023-09-30T22:22:42.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.557+0000 7f9ec2ffd700 1 -- 172.21.15.89:0/296330886 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ec8040e60 con 0x7f9ed4133e30 2023-09-30T22:22:42.558 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:42.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133e30 msgr2=0x7f9ed409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 --2- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133e30 0x7f9ed409c9c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8013000 tx=0x7f9ec8004750 comp rx=0 tx=0).stop 2023-09-30T22:22:42.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 shutdown_connections 2023-09-30T22:22:42.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 --2- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9ed4134260 0x7f9ed409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 --2- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ed4133420 0x7f9ed409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 --2- 172.21.15.89:0/296330886 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed4133e30 0x7f9ed409c9c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 >> 172.21.15.89:0/296330886 conn(0x7f9ed40a4210 msgr2=0x7f9ed40a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 shutdown_connections 2023-09-30T22:22:42.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.559+0000 7f9ed896c700 1 -- 172.21.15.89:0/296330886 wait complete. 2023-09-30T22:22:42.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:42.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.595+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f78b411d9c0 con 0x7f78b406f660 2023-09-30T22:22:42.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.596+0000 7f78a9ffb700 1 -- 172.21.15.89:0/2341315721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78b0022030 con 0x7f78b406f660 2023-09-30T22:22:42.597 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 msgr2=0x7f78b411b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b411b8a0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f78b000b3f0 tx=0x7f78b000b700 comp rx=0 tx=0).stop 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 shutdown_connections 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78b41305f0 0x7f78b411c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b406f660 0x7f78b411b8a0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 --2- 172.21.15.89:0/2341315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78b406ff90 0x7f78b411bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 >> 172.21.15.89:0/2341315721 conn(0x7f78b40a2aa0 msgr2=0x7f78b406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 shutdown_connections 2023-09-30T22:22:42.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:42.598+0000 7f78bb76c700 1 -- 172.21.15.89:0/2341315721 wait complete. 2023-09-30T22:22:42.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:43.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:43.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:43.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:43.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.588+0000 7f0dffe71700 1 Processor -- start 2023-09-30T22:22:43.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.588+0000 7f0dffe71700 1 -- start start 2023-09-30T22:22:43.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dffe71700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0df8131810 0x7f0df8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dffe71700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8099a40 0x7f0df8099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dffe71700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df81313c0 con 0x7f0df809a3b0 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dffe71700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df809b580 con 0x7f0df8131810 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dffe71700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df806e8c0 con 0x7f0df8099a40 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dfdc0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0df8131810 0x7f0df8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dfdc0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0df8131810 0x7f0df8131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56972/0 (socket says 172.21.15.89:56972) 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dfdc0d700 1 -- 172.21.15.89:0/540856873 learned_addr learned my addr 172.21.15.89:0/540856873 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dfd40c700 1 --2- 172.21.15.89:0/540856873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8099a40 0x7f0df8099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dfdc0d700 1 -- 172.21.15.89:0/540856873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8099a40 msgr2=0x7f0df8099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:43.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dfdc0d700 1 --2- 172.21.15.89:0/540856873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8099a40 0x7f0df8099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dfdc0d700 1 -- 172.21.15.89:0/540856873 >> v1:172.21.15.89:6789/0 conn(0x7f0df809a3b0 legacy=0x7f0df8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.589+0000 7f0dfdc0d700 1 -- 172.21.15.89:0/540856873 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df809a830 con 0x7f0df8131810 2023-09-30T22:22:43.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.590+0000 7f0dfdc0d700 1 --2- 172.21.15.89:0/540856873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8131810 0x7f0df8131be0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f0df40098f0 tx=0x7f0df4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ee5119c8642b27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:43.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.590+0000 7f0dfcc0b700 1 -- 172.21.15.89:0/540856873 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4011830 con 0x7f0df8131810 2023-09-30T22:22:43.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.590+0000 7f0dfcc0b700 1 -- 172.21.15.89:0/540856873 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df4011990 con 0x7f0df8131810 2023-09-30T22:22:43.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.590+0000 7f0dfcc0b700 1 -- 172.21.15.89:0/540856873 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4011d20 con 0x7f0df8131810 2023-09-30T22:22:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.591+0000 7f0dffe71700 1 -- 172.21.15.89:0/540856873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8131810 msgr2=0x7f0df8131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.591+0000 7f0dffe71700 1 --2- 172.21.15.89:0/540856873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8131810 0x7f0df8131be0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f0df40098f0 tx=0x7f0df4009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:43.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.591+0000 7f0dffe71700 1 -- 172.21.15.89:0/540856873 shutdown_connections 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.591+0000 7f0dffe71700 1 --2- 172.21.15.89:0/540856873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8131810 0x7f0df8131be0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.591+0000 7f0dffe71700 1 --2- 172.21.15.89:0/540856873 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8099a40 0x7f0df8099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.591+0000 7f0dffe71700 1 -- 172.21.15.89:0/540856873 >> 172.21.15.89:0/540856873 conn(0x7f0df80a2aa0 msgr2=0x7f0df80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.591+0000 7f0dffe71700 1 -- 172.21.15.89:0/540856873 shutdown_connections 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.591+0000 7f0dffe71700 1 -- 172.21.15.89:0/540856873 wait complete. 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.592+0000 7f0dffe71700 1 Processor -- start 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.592+0000 7f0dffe71700 1 -- start start 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dffe71700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8099a40 0x7f0df811b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dffe71700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0df809a3b0 0x7f0df811bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dffe71700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8131810 0x7f0df811c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dffe71700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df811f1b0 con 0x7f0df8099a40 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dffe71700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df811d880 con 0x7f0df8131810 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dffe71700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0df811e940 con 0x7f0df809a3b0 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dfd40c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0df809a3b0 0x7f0df811bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dfd40c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0df809a3b0 0x7f0df811bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55666/0 (socket says 172.21.15.89:55666) 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dfd40c700 1 -- 172.21.15.89:0/490721373 learned_addr learned my addr 172.21.15.89:0/490721373 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dfdc0d700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8099a40 0x7f0df811b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.593+0000 7f0dfe40e700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8131810 0x7f0df811c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dfd40c700 1 -- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8131810 msgr2=0x7f0df811c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dfd40c700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8131810 0x7f0df811c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dfd40c700 1 -- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8099a40 msgr2=0x7f0df811b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dfd40c700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8099a40 0x7f0df811b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dfd40c700 1 -- 172.21.15.89:0/490721373 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df4009550 con 0x7f0df809a3b0 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dfe40e700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8131810 0x7f0df811c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dfdc0d700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8099a40 0x7f0df811b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dfd40c700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0df809a3b0 0x7f0df811bbd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0de800e9e0 tx=0x7f0de800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dee7fc700 1 -- 172.21.15.89:0/490721373 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8009710 con 0x7f0df809a3b0 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dee7fc700 1 -- 172.21.15.89:0/490721373 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de800c850 con 0x7f0df809a3b0 2023-09-30T22:22:43.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df806eb00 con 0x7f0df809a3b0 2023-09-30T22:22:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dee7fc700 1 -- 172.21.15.89:0/490721373 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800cbc0 con 0x7f0df809a3b0 2023-09-30T22:22:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.594+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df811d2a0 con 0x7f0df809a3b0 2023-09-30T22:22:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.595+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df811e550 con 0x7f0df809a3b0 2023-09-30T22:22:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.596+0000 7f0dee7fc700 1 -- 172.21.15.89:0/490721373 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0de800cd20 con 0x7f0df809a3b0 2023-09-30T22:22:43.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.596+0000 7f0dee7fc700 1 -- 172.21.15.89:0/490721373 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0de803ff60 con 0x7f0df809a3b0 2023-09-30T22:22:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.601+0000 7f0dee7fc700 1 -- 172.21.15.89:0/490721373 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0de806b750 con 0x7f0df809a3b0 2023-09-30T22:22:43.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.627+0000 7ff424441700 1 Processor -- start 2023-09-30T22:22:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.627+0000 7ff424441700 1 -- start start 2023-09-30T22:22:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff424441700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff424441700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff424441700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff41c09a3b0 0x7ff41c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff424441700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c09b580 con 0x7ff41c09a3b0 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff424441700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c1313c0 con 0x7ff41c099a40 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff424441700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c06e8c0 con 0x7ff41c131810 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4219dc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55676/0 (socket says 172.21.15.89:55676) 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 -- 172.21.15.89:0/408410933 learned_addr learned my addr 172.21.15.89:0/408410933 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 -- 172.21.15.89:0/408410933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 msgr2=0x7ff41c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 --2- 172.21.15.89:0/408410933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 -- 172.21.15.89:0/408410933 >> v2:172.21.15.89:3300/0 conn(0x7ff41c09a3b0 msgr2=0x7ff41c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:43.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 --2- 172.21.15.89:0/408410933 >> v2:172.21.15.89:3300/0 conn(0x7ff41c09a3b0 0x7ff41c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 -- 172.21.15.89:0/408410933 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c09a820 con 0x7ff41c131810 2023-09-30T22:22:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4219dc700 1 --2- 172.21.15.89:0/408410933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.628+0000 7ff4221dd700 1 --2- 172.21.15.89:0/408410933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c131be0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff4180098f0 tx=0x7ff418009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e1a5d36916ede0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.629+0000 7ff4211db700 1 -- 172.21.15.89:0/408410933 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418011830 con 0x7ff41c131810 2023-09-30T22:22:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.629+0000 7ff4211db700 1 -- 172.21.15.89:0/408410933 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff418011990 con 0x7ff41c131810 2023-09-30T22:22:43.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.629+0000 7ff4211db700 1 -- 172.21.15.89:0/408410933 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418011d20 con 0x7ff41c131810 2023-09-30T22:22:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 -- 172.21.15.89:0/408410933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 msgr2=0x7ff41c131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 --2- 172.21.15.89:0/408410933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c131be0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff4180098f0 tx=0x7ff418009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 -- 172.21.15.89:0/408410933 shutdown_connections 2023-09-30T22:22:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 --2- 172.21.15.89:0/408410933 >> v2:172.21.15.89:3300/0 conn(0x7ff41c09a3b0 0x7ff41c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 --2- 172.21.15.89:0/408410933 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c131be0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 --2- 172.21.15.89:0/408410933 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 -- 172.21.15.89:0/408410933 >> 172.21.15.89:0/408410933 conn(0x7ff41c0a2aa0 msgr2=0x7ff41c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 -- 172.21.15.89:0/408410933 shutdown_connections 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.630+0000 7ff424441700 1 -- 172.21.15.89:0/408410933 wait complete. 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff424441700 1 Processor -- start 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff424441700 1 -- start start 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff424441700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c06f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff424441700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff41c09a3b0 0x7ff41c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff424441700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff424441700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c120320 con 0x7ff41c09a3b0 2023-09-30T22:22:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff424441700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c121360 con 0x7ff41c099a40 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff424441700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c120af0 con 0x7ff41c131810 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff4221dd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c06f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff4221dd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c06f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51320/0 (socket says 172.21.15.89:51320) 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff4221dd700 1 -- 172.21.15.89:0/1754475306 learned_addr learned my addr 172.21.15.89:0/1754475306 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.631+0000 7ff4229de700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff4221dd700 1 -- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 msgr2=0x7ff41c070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff4221dd700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff4221dd700 1 -- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff41c09a3b0 msgr2=0x7ff41c06fb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff4221dd700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff41c09a3b0 0x7ff41c06fb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff4221dd700 1 -- 172.21.15.89:0/1754475306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0094a0 con 0x7ff41c099a40 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff4229de700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff4221dd700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c06f610 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff418009c40 tx=0x7ff418004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff412ffd700 1 -- 172.21.15.89:0/1754475306 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418004590 con 0x7ff41c099a40 2023-09-30T22:22:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff412ffd700 1 -- 172.21.15.89:0/1754475306 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4180036a0 con 0x7ff41c099a40 2023-09-30T22:22:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff418009550 con 0x7ff41c099a40 2023-09-30T22:22:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff412ffd700 1 -- 172.21.15.89:0/1754475306 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418003a50 con 0x7ff41c099a40 2023-09-30T22:22:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.632+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c06eb00 con 0x7ff41c099a40 2023-09-30T22:22:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.633+0000 7ff412ffd700 1 -- 172.21.15.89:0/1754475306 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff418003bb0 con 0x7ff41c099a40 2023-09-30T22:22:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.633+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c074ea0 con 0x7ff41c099a40 2023-09-30T22:22:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.633+0000 7ff412ffd700 1 -- 172.21.15.89:0/1754475306 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff4180407e0 con 0x7ff41c099a40 2023-09-30T22:22:43.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.642+0000 7ff412ffd700 1 -- 172.21.15.89:0/1754475306 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff41801a420 con 0x7ff41c099a40 2023-09-30T22:22:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.713+0000 7f39f554c700 1 Processor -- start 2023-09-30T22:22:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.713+0000 7f39f554c700 1 -- start start 2023-09-30T22:22:43.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39f554c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39f009e1d0 0x7f39f0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39f554c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39f554c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f009f130 con 0x7f39f009d6f0 2023-09-30T22:22:43.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39f554c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f009f520 con 0x7f39f009e1d0 2023-09-30T22:22:43.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39f554c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39f009e930 con 0x7f39f0135e50 2023-09-30T22:22:43.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39ef7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39ee7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39f009e1d0 0x7f39f0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39ef7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f0138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55688/0 (socket says 172.21.15.89:55688) 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39eeffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f39f009d6f0 0x7f39f009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57916/0 (socket says 172.21.15.89:57916) 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.714+0000 7f39ef7fe700 1 -- 172.21.15.89:0/602508070 learned_addr learned my addr 172.21.15.89:0/602508070 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39ef7fe700 1 -- 172.21.15.89:0/602508070 >> v2:172.21.15.89:3300/0 conn(0x7f39f009e1d0 msgr2=0x7f39f0135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39ef7fe700 1 --2- 172.21.15.89:0/602508070 >> v2:172.21.15.89:3300/0 conn(0x7f39f009e1d0 0x7f39f0135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39ef7fe700 1 -- 172.21.15.89:0/602508070 >> v1:172.21.15.89:6789/0 conn(0x7f39f009d6f0 legacy=0x7f39f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39ef7fe700 1 -- 172.21.15.89:0/602508070 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f009b2d0 con 0x7f39f0135e50 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39edffb700 1 -- 172.21.15.89:0/602508070 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223832950 0 0) 0x7f39f009f130 con 0x7f39f009d6f0 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39ef7fe700 1 --2- 172.21.15.89:0/602508070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f0138230 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009e50 tx=0x7f39e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748b3c5df0254cf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39edffb700 1 -- 172.21.15.89:0/602508070 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e800e070 con 0x7f39f0135e50 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39edffb700 1 -- 172.21.15.89:0/602508070 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e800b910 con 0x7f39f0135e50 2023-09-30T22:22:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.715+0000 7f39edffb700 1 -- 172.21.15.89:0/602508070 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e800bcc0 con 0x7f39f0135e50 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.716+0000 7f39f554c700 1 -- 172.21.15.89:0/602508070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 msgr2=0x7f39f0138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.716+0000 7f39f554c700 1 --2- 172.21.15.89:0/602508070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f0138230 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009e50 tx=0x7f39e800b330 comp rx=0 tx=0).stop 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.716+0000 7f39f554c700 1 -- 172.21.15.89:0/602508070 shutdown_connections 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.716+0000 7f39f554c700 1 --2- 172.21.15.89:0/602508070 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f0138230 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.716+0000 7f39f554c700 1 --2- 172.21.15.89:0/602508070 >> v2:172.21.15.89:3300/0 conn(0x7f39f009e1d0 0x7f39f0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.716+0000 7f39f554c700 1 -- 172.21.15.89:0/602508070 >> 172.21.15.89:0/602508070 conn(0x7f39f0066f50 msgr2=0x7f39f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.716+0000 7f39f554c700 1 -- 172.21.15.89:0/602508070 shutdown_connections 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.716+0000 7f39f554c700 1 -- 172.21.15.89:0/602508070 wait complete. 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.717+0000 7f39f554c700 1 Processor -- start 2023-09-30T22:22:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39f554c700 1 -- start start 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39f554c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f009d6f0 0x7f39f011d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39f554c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39f009e1d0 0x7f39f011dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39f554c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f011dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39f554c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f011e530 con 0x7f39f009d6f0 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39f554c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f01243f0 con 0x7f39f009e1d0 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39f554c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39f011f500 con 0x7f39f0135e50 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39eeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f009d6f0 0x7f39f011d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39eeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f009d6f0 0x7f39f011d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57026/0 (socket says 172.21.15.89:57026) 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.718+0000 7f39eeffd700 1 -- 172.21.15.89:0/3868486531 learned_addr learned my addr 172.21.15.89:0/3868486531 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39ef7fe700 1 --2- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f011dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39eeffd700 1 -- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 msgr2=0x7f39f011dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39eeffd700 1 --2- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f011dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39eeffd700 1 -- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39f009e1d0 msgr2=0x7f39f011dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39eeffd700 1 --2- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39f009e1d0 0x7f39f011dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39eeffd700 1 -- 172.21.15.89:0/3868486531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e8009b30 con 0x7f39f009d6f0 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39eeffd700 1 --2- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f009d6f0 0x7f39f011d570 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c520 tx=0x7f39e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39d77fe700 1 -- 172.21.15.89:0/3868486531 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0004c80 con 0x7f39f009d6f0 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39d77fe700 1 -- 172.21.15.89:0/3868486531 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e0004de0 con 0x7f39f009d6f0 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f0137ad0 con 0x7f39f009d6f0 2023-09-30T22:22:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39d77fe700 1 -- 172.21.15.89:0/3868486531 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0007310 con 0x7f39f009d6f0 2023-09-30T22:22:43.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.719+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f0124110 con 0x7f39f009d6f0 2023-09-30T22:22:43.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.720+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f0074ea0 con 0x7f39f009d6f0 2023-09-30T22:22:43.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.721+0000 7f39d77fe700 1 -- 172.21.15.89:0/3868486531 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f39e0007470 con 0x7f39f009d6f0 2023-09-30T22:22:43.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.721+0000 7f39d77fe700 1 -- 172.21.15.89:0/3868486531 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f39e003f920 con 0x7f39f009d6f0 2023-09-30T22:22:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.726+0000 7f39d77fe700 1 -- 172.21.15.89:0/3868486531 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39e00133c0 con 0x7f39f009d6f0 2023-09-30T22:22:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.744+0000 7f451b86b700 1 Processor -- start 2023-09-30T22:22:43.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.744+0000 7f451b86b700 1 -- start start 2023-09-30T22:22:43.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.744+0000 7f451b86b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f451409f410 0x7f4514136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.744+0000 7f451b86b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4514136fe0 0x7f45141393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.744+0000 7f451b86b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4514070940 con 0x7f451409e930 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.744+0000 7f451b86b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451406fd60 con 0x7f4514136fe0 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f451b86b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451406fa80 con 0x7f451409f410 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4519607700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f451409e930 0x7f451409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57930/0 (socket says 172.21.15.89:57930) 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4519607700 1 -- 172.21.15.89:0/2498085444 learned_addr learned my addr 172.21.15.89:0/2498085444 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4518e06700 1 --2- 172.21.15.89:0/2498085444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f451409f410 0x7f4514136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4519e08700 1 --2- 172.21.15.89:0/2498085444 >> v2:172.21.15.89:3300/0 conn(0x7f4514136fe0 0x7f45141393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4519e08700 1 -- 172.21.15.89:0/2498085444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f451409f410 msgr2=0x7f4514136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:43.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4519e08700 1 --2- 172.21.15.89:0/2498085444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f451409f410 0x7f4514136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4519e08700 1 -- 172.21.15.89:0/2498085444 >> v1:172.21.15.89:6789/0 conn(0x7f451409e930 legacy=0x7f451409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4519e08700 1 -- 172.21.15.89:0/2498085444 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451409fb60 con 0x7f4514136fe0 2023-09-30T22:22:43.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f4519e08700 1 --2- 172.21.15.89:0/2498085444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4514136fe0 0x7f45141393c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f450c009d90 tx=0x7f450c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcf1655764b8ba0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.745+0000 7f450bfff700 1 -- 172.21.15.89:0/2498085444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c003c20 con 0x7f4514136fe0 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f450bfff700 1 -- 172.21.15.89:0/2498085444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c003d80 con 0x7f4514136fe0 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f450bfff700 1 -- 172.21.15.89:0/2498085444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00bc60 con 0x7f4514136fe0 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f451b86b700 1 -- 172.21.15.89:0/2498085444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4514136fe0 msgr2=0x7f45141393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f451b86b700 1 --2- 172.21.15.89:0/2498085444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4514136fe0 0x7f45141393c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f450c009d90 tx=0x7f450c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f451b86b700 1 -- 172.21.15.89:0/2498085444 shutdown_connections 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f451b86b700 1 --2- 172.21.15.89:0/2498085444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4514136fe0 0x7f45141393c0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f451b86b700 1 --2- 172.21.15.89:0/2498085444 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f451409f410 0x7f4514136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f451b86b700 1 -- 172.21.15.89:0/2498085444 >> 172.21.15.89:0/2498085444 conn(0x7f45140a4210 msgr2=0x7f45140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:43.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f451b86b700 1 -- 172.21.15.89:0/2498085444 shutdown_connections 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.746+0000 7f451b86b700 1 -- 172.21.15.89:0/2498085444 wait complete. 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.747+0000 7f451b86b700 1 Processor -- start 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.747+0000 7f451b86b700 1 -- start start 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f451b86b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f451409e930 0x7f451411b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f451b86b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f451409f410 0x7f451411bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f451b86b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4514136fe0 0x7f451411c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f451b86b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451411f220 con 0x7f451409f410 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f451b86b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451406d780 con 0x7f4514136fe0 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f451b86b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4514071d20 con 0x7f451409e930 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f451409f410 0x7f451411bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f451409f410 0x7f451411bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57036/0 (socket says 172.21.15.89:57036) 2023-09-30T22:22:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 -- 172.21.15.89:0/774577675 learned_addr learned my addr 172.21.15.89:0/774577675 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4519e08700 1 --2- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4514136fe0 0x7f451411c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 -- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f451409e930 msgr2=0x7f451411b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 --2- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f451409e930 0x7f451411b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 -- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4514136fe0 msgr2=0x7f451411c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 --2- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4514136fe0 0x7f451411c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 -- 172.21.15.89:0/774577675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c009a70 con 0x7f451409f410 2023-09-30T22:22:43.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4518e06700 1 --2- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f451409f410 0x7f451411bba0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f450400e9b0 tx=0x7f450400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f4509ffb700 1 -- 172.21.15.89:0/774577675 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450400c850 con 0x7f451409f410 2023-09-30T22:22:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.748+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451411e910 con 0x7f451409f410 2023-09-30T22:22:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.749+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451411da20 con 0x7f451409f410 2023-09-30T22:22:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.749+0000 7f4509ffb700 1 -- 172.21.15.89:0/774577675 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450400c9b0 con 0x7f451409f410 2023-09-30T22:22:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.749+0000 7f4509ffb700 1 -- 172.21.15.89:0/774577675 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450400cd20 con 0x7f451409f410 2023-09-30T22:22:43.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.750+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f40052e0 con 0x7f451409f410 2023-09-30T22:22:43.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.750+0000 7f4509ffb700 1 -- 172.21.15.89:0/774577675 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4504013470 con 0x7f451409f410 2023-09-30T22:22:43.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.758+0000 7f4509ffb700 1 -- 172.21.15.89:0/774577675 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4504024380 con 0x7f451409f410 2023-09-30T22:22:43.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.759+0000 7f4509ffb700 1 -- 172.21.15.89:0/774577675 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4504011350 con 0x7f451409f410 2023-09-30T22:22:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.805+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0df8074ea0 con 0x7f0df809a3b0 2023-09-30T22:22:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.805+0000 7f0dee7fc700 1 -- 172.21.15.89:0/490721373 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0de8017070 con 0x7f0df809a3b0 2023-09-30T22:22:43.806 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.807+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0df809a3b0 msgr2=0x7f0df811bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.807+0000 7f0dffe71700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0df809a3b0 0x7f0df811bbd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0de800e9e0 tx=0x7f0de800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.807+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 shutdown_connections 2023-09-30T22:22:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.807+0000 7f0dffe71700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0df8131810 0x7f0df811c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.807+0000 7f0dffe71700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df8099a40 0x7f0df811b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.807+0000 7f0dffe71700 1 --2- 172.21.15.89:0/490721373 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0df809a3b0 0x7f0df811bbd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.807+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 >> 172.21.15.89:0/490721373 conn(0x7f0df80a2aa0 msgr2=0x7f0df81b8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.808+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 shutdown_connections 2023-09-30T22:22:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.808+0000 7f0dffe71700 1 -- 172.21.15.89:0/490721373 wait complete. 2023-09-30T22:22:43.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:43.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.841+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff41c071d80 con 0x7ff41c099a40 2023-09-30T22:22:43.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.842+0000 7ff412ffd700 1 -- 172.21.15.89:0/1754475306 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff418015020 con 0x7ff41c099a40 2023-09-30T22:22:43.842 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:43.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 msgr2=0x7ff41c06f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c06f610 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff418009c40 tx=0x7ff418004750 comp rx=0 tx=0).stop 2023-09-30T22:22:43.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 shutdown_connections 2023-09-30T22:22:43.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff41c131810 0x7ff41c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff41c099a40 0x7ff41c06f610 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 --2- 172.21.15.89:0/1754475306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff41c09a3b0 0x7ff41c06fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 >> 172.21.15.89:0/1754475306 conn(0x7ff41c0a2aa0 msgr2=0x7ff41c1b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:43.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 shutdown_connections 2023-09-30T22:22:43.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.844+0000 7ff424441700 1 -- 172.21.15.89:0/1754475306 wait complete. 2023-09-30T22:22:43.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:43.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.925+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f39f006c360 con 0x7f39f009d6f0 2023-09-30T22:22:43.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.926+0000 7f39d77fe700 1 -- 172.21.15.89:0/3868486531 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39e0011360 con 0x7f39f009d6f0 2023-09-30T22:22:43.927 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f009d6f0 msgr2=0x7f39f011d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 --2- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f009d6f0 0x7f39f011d570 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c520 tx=0x7f39e000c830 comp rx=0 tx=0).stop 2023-09-30T22:22:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 shutdown_connections 2023-09-30T22:22:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 --2- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f39f0135e50 0x7f39f011dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 --2- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f009d6f0 0x7f39f011d570 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 --2- 172.21.15.89:0/3868486531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39f009e1d0 0x7f39f011dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 >> 172.21.15.89:0/3868486531 conn(0x7f39f0066f50 msgr2=0x7f39f0070850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:43.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 shutdown_connections 2023-09-30T22:22:43.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.928+0000 7f39f554c700 1 -- 172.21.15.89:0/3868486531 wait complete. 2023-09-30T22:22:43.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:43.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.953+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f44f4005860 con 0x7f451409f410 2023-09-30T22:22:43.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.954+0000 7f4509ffb700 1 -- 172.21.15.89:0/774577675 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4504021020 con 0x7f451409f410 2023-09-30T22:22:43.955 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f451409f410 msgr2=0x7f451411bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 --2- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f451409f410 0x7f451411bba0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f450400e9b0 tx=0x7f450400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 shutdown_connections 2023-09-30T22:22:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 --2- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4514136fe0 0x7f451411c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 --2- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f451409e930 0x7f451411b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 --2- 172.21.15.89:0/774577675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f451409f410 0x7f451411bba0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 >> 172.21.15.89:0/774577675 conn(0x7f45140a4210 msgr2=0x7f45140a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:43.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 shutdown_connections 2023-09-30T22:22:43.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:43.956+0000 7f451b86b700 1 -- 172.21.15.89:0/774577675 wait complete. 2023-09-30T22:22:43.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:44.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:44.858 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:44.942 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:44.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99df023700 1 Processor -- start 2023-09-30T22:22:44.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99df023700 1 -- start start 2023-09-30T22:22:44.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99df023700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8134630 0x7f99d8136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:44.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99df023700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 0x7f99d8139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:44.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99df023700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d809b4f0 con 0x7f99d8131810 2023-09-30T22:22:44.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99df023700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d81313c0 con 0x7f99d8136f50 2023-09-30T22:22:44.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99df023700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99d806e8c0 con 0x7f99d8134630 2023-09-30T22:22:44.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8134630 0x7f99d8136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8134630 0x7f99d8136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55714/0 (socket says 172.21.15.89:55714) 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99dcdbf700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f99d8131810 0x7f99d8131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57938/0 (socket says 172.21.15.89:57938) 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.957+0000 7f99d7fff700 1 -- 172.21.15.89:0/2521356050 learned_addr learned my addr 172.21.15.89:0/2521356050 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99dd5c0700 1 --2- 172.21.15.89:0/2521356050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 0x7f99d8139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d77fe700 1 -- 172.21.15.89:0/2521356050 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270706745 0 0) 0x7f99d809b4f0 con 0x7f99d8131810 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d77fe700 1 -- 172.21.15.89:0/2521356050 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c80035b0 con 0x7f99d8131810 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d7fff700 1 -- 172.21.15.89:0/2521356050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 msgr2=0x7f99d8139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d7fff700 1 --2- 172.21.15.89:0/2521356050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 0x7f99d8139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d7fff700 1 -- 172.21.15.89:0/2521356050 >> v1:172.21.15.89:6789/0 conn(0x7f99d8131810 legacy=0x7f99d8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d7fff700 1 -- 172.21.15.89:0/2521356050 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d8071d20 con 0x7f99d8134630 2023-09-30T22:22:44.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d7fff700 1 --2- 172.21.15.89:0/2521356050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8134630 0x7f99d8136a10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009920 tx=0x7f99c4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3fb58384509764 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:44.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d77fe700 1 -- 172.21.15.89:0/2521356050 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4011830 con 0x7f99d8134630 2023-09-30T22:22:44.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d77fe700 1 -- 172.21.15.89:0/2521356050 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c4011990 con 0x7f99d8134630 2023-09-30T22:22:44.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.958+0000 7f99d77fe700 1 -- 172.21.15.89:0/2521356050 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4011d20 con 0x7f99d8134630 2023-09-30T22:22:44.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.959+0000 7f99df023700 1 -- 172.21.15.89:0/2521356050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8134630 msgr2=0x7f99d8136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:44.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.959+0000 7f99df023700 1 --2- 172.21.15.89:0/2521356050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8134630 0x7f99d8136a10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009920 tx=0x7f99c4009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:44.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.959+0000 7f99df023700 1 -- 172.21.15.89:0/2521356050 shutdown_connections 2023-09-30T22:22:44.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.959+0000 7f99df023700 1 --2- 172.21.15.89:0/2521356050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 0x7f99d8139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.959+0000 7f99df023700 1 --2- 172.21.15.89:0/2521356050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8134630 0x7f99d8136a10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.959+0000 7f99df023700 1 -- 172.21.15.89:0/2521356050 >> 172.21.15.89:0/2521356050 conn(0x7f99d80a2aa0 msgr2=0x7f99d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.959+0000 7f99df023700 1 -- 172.21.15.89:0/2521356050 shutdown_connections 2023-09-30T22:22:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.959+0000 7f99df023700 1 -- 172.21.15.89:0/2521356050 wait complete. 2023-09-30T22:22:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.960+0000 7f99df023700 1 Processor -- start 2023-09-30T22:22:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.960+0000 7f99df023700 1 -- start start 2023-09-30T22:22:44.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.961+0000 7f99df023700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8131810 0x7f99d8118d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.961+0000 7f99df023700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d8134630 0x7f99d8119260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.961+0000 7f99df023700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 0x7f99d81197a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.961+0000 7f99df023700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d811aeb0 con 0x7f99d8134630 2023-09-30T22:22:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.961+0000 7f99df023700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d811c770 con 0x7f99d8136f50 2023-09-30T22:22:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.961+0000 7f99df023700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8130cb0 con 0x7f99d8131810 2023-09-30T22:22:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.961+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d8134630 0x7f99d8119260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99dcdbf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8131810 0x7f99d8118d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:44.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d8134630 0x7f99d8119260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57052/0 (socket says 172.21.15.89:57052) 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d7fff700 1 -- 172.21.15.89:0/115315721 learned_addr learned my addr 172.21.15.89:0/115315721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99dd5c0700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 0x7f99d81197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d7fff700 1 -- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8131810 msgr2=0x7f99d8118d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d7fff700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8131810 0x7f99d8118d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d7fff700 1 -- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 msgr2=0x7f99d81197a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d7fff700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 0x7f99d81197a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d7fff700 1 -- 172.21.15.89:0/115315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4009550 con 0x7f99d8134630 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99dcdbf700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8131810 0x7f99d8118d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d7fff700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d8134630 0x7f99d8119260 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f99c40098f0 tx=0x7f99c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d57fa700 1 -- 172.21.15.89:0/115315721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4004450 con 0x7f99d8134630 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d57fa700 1 -- 172.21.15.89:0/115315721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c40045b0 con 0x7f99d8134630 2023-09-30T22:22:44.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99d57fa700 1 -- 172.21.15.89:0/115315721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c40038e0 con 0x7f99d8134630 2023-09-30T22:22:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d811c1d0 con 0x7f99d8134630 2023-09-30T22:22:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.962+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d811b050 con 0x7f99d8134630 2023-09-30T22:22:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.963+0000 7f99d57fa700 1 -- 172.21.15.89:0/115315721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99c4003a40 con 0x7f99d8134630 2023-09-30T22:22:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.963+0000 7f99d57fa700 1 -- 172.21.15.89:0/115315721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f99c4021030 con 0x7f99d8134630 2023-09-30T22:22:44.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.963+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d811c5a0 con 0x7f99d8134630 2023-09-30T22:22:44.969 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:44.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:44.970+0000 7f99d57fa700 1 -- 172.21.15.89:0/115315721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99c4003cf0 con 0x7f99d8134630 2023-09-30T22:22:45.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabf2018700 1 Processor -- start 2023-09-30T22:22:45.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabf2018700 1 -- start start 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabf2018700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec12cf60 0x7fabec06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabf2018700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 0x7fabec12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabf2018700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec12cd90 con 0x7fabec130260 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabf2018700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec12d330 con 0x7fabec12f8b0 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabf2018700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabec1306d0 con 0x7fabec12cf60 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabebfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fabec130260 0x7fabec136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57942/0 (socket says 172.21.15.89:57942) 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabebfff700 1 -- 172.21.15.89:0/3558814851 learned_addr learned my addr 172.21.15.89:0/3558814851 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.001+0000 7fabeb7fe700 1 --2- 172.21.15.89:0/3558814851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec12cf60 0x7fabec06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabea7fc700 1 -- 172.21.15.89:0/3558814851 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731155601 0 0) 0x7fabec12cd90 con 0x7fabec130260 2023-09-30T22:22:45.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabea7fc700 1 -- 172.21.15.89:0/3558814851 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd40035b0 con 0x7fabec130260 2023-09-30T22:22:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabeb7fe700 1 -- 172.21.15.89:0/3558814851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 msgr2=0x7fabec12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabeb7fe700 1 --2- 172.21.15.89:0/3558814851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 0x7fabec12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabeb7fe700 1 -- 172.21.15.89:0/3558814851 >> v1:172.21.15.89:6789/0 conn(0x7fabec130260 legacy=0x7fabec136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabeb7fe700 1 -- 172.21.15.89:0/3558814851 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec071780 con 0x7fabec12cf60 2023-09-30T22:22:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabeb7fe700 1 --2- 172.21.15.89:0/3558814851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec12cf60 0x7fabec06f5d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009870 tx=0x7fabdc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b792b0d520e6e60f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabea7fc700 1 -- 172.21.15.89:0/3558814851 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc010800 con 0x7fabec12cf60 2023-09-30T22:22:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabea7fc700 1 -- 172.21.15.89:0/3558814851 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc010960 con 0x7fabec12cf60 2023-09-30T22:22:45.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.002+0000 7fabea7fc700 1 -- 172.21.15.89:0/3558814851 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc010cd0 con 0x7fabec12cf60 2023-09-30T22:22:45.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.003+0000 7fabf2018700 1 -- 172.21.15.89:0/3558814851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec12cf60 msgr2=0x7fabec06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.003+0000 7fabf2018700 1 --2- 172.21.15.89:0/3558814851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec12cf60 0x7fabec06f5d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009870 tx=0x7fabdc009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:45.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.003+0000 7fabf2018700 1 -- 172.21.15.89:0/3558814851 shutdown_connections 2023-09-30T22:22:45.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.003+0000 7fabf2018700 1 --2- 172.21.15.89:0/3558814851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec12cf60 0x7fabec06f5d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.003+0000 7fabf2018700 1 --2- 172.21.15.89:0/3558814851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 0x7fabec12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.003+0000 7fabf2018700 1 -- 172.21.15.89:0/3558814851 >> 172.21.15.89:0/3558814851 conn(0x7fabec0a2aa0 msgr2=0x7fabec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.003+0000 7fabf2018700 1 -- 172.21.15.89:0/3558814851 shutdown_connections 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.003+0000 7fabf2018700 1 -- 172.21.15.89:0/3558814851 wait complete. 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.004+0000 7fabf2018700 1 Processor -- start 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.004+0000 7fabf2018700 1 -- start start 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.004+0000 7fabf2018700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabec12cf60 0x7fabec099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.004+0000 7fabf2018700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 0x7fabec09a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.004+0000 7fabf2018700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec130260 0x7fabec09a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.004+0000 7fabf2018700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec09d040 con 0x7fabec12cf60 2023-09-30T22:22:45.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.004+0000 7fabf2018700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec09c6d0 con 0x7fabec12f8b0 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.004+0000 7fabf2018700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabec12d8d0 con 0x7fabec130260 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec130260 0x7fabec09a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec130260 0x7fabec09a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55728/0 (socket says 172.21.15.89:55728) 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 -- 172.21.15.89:0/2217575399 learned_addr learned my addr 172.21.15.89:0/2217575399 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabeb7fe700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabec12cf60 0x7fabec099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabeaffd700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 0x7fabec09a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 -- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 msgr2=0x7fabec09a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 0x7fabec09a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 -- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabec12cf60 msgr2=0x7fabec099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabec12cf60 0x7fabec099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 -- 172.21.15.89:0/2217575399 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0094a0 con 0x7fabec130260 2023-09-30T22:22:45.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.005+0000 7fabebfff700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec130260 0x7fabec09a7d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fabe400cbe0 tx=0x7fabe400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.006+0000 7fabd3fff700 1 -- 172.21.15.89:0/2217575399 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe400ab10 con 0x7fabec130260 2023-09-30T22:22:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.006+0000 7fabd3fff700 1 -- 172.21.15.89:0/2217575399 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe400ac70 con 0x7fabec130260 2023-09-30T22:22:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.006+0000 7fabd3fff700 1 -- 172.21.15.89:0/2217575399 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4012400 con 0x7fabec130260 2023-09-30T22:22:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.006+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec129fa0 con 0x7fabec130260 2023-09-30T22:22:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.006+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec09d820 con 0x7fabec130260 2023-09-30T22:22:45.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.007+0000 7fabd3fff700 1 -- 172.21.15.89:0/2217575399 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fabe4012560 con 0x7fabec130260 2023-09-30T22:22:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.007+0000 7fabd3fff700 1 -- 172.21.15.89:0/2217575399 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fabe4012810 con 0x7fabec130260 2023-09-30T22:22:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.007+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec09d180 con 0x7fabec130260 2023-09-30T22:22:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.016+0000 7fabd3fff700 1 -- 172.21.15.89:0/2217575399 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fabec09d180 con 0x7fabec130260 2023-09-30T22:22:45.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.081+0000 7f2e25dbd700 1 Processor -- start 2023-09-30T22:22:45.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.081+0000 7f2e25dbd700 1 -- start start 2023-09-30T22:22:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.082+0000 7f2e25dbd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012cf60 0x7f2e2006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.082+0000 7f2e25dbd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e2012f8b0 0x7f2e2012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.082+0000 7f2e25dbd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2012cd90 con 0x7f2e20130260 2023-09-30T22:22:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.082+0000 7f2e25dbd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2012d330 con 0x7f2e2012f8b0 2023-09-30T22:22:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.082+0000 7f2e25dbd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201306e0 con 0x7f2e2012cf60 2023-09-30T22:22:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.082+0000 7f2e1f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012cf60 0x7f2e2006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.082+0000 7f2e1f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012cf60 0x7f2e2006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51400/0 (socket says 172.21.15.89:51400) 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.082+0000 7f2e1f7fe700 1 -- 172.21.15.89:0/1360384274 learned_addr learned my addr 172.21.15.89:0/1360384274 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1effd700 1 --2- 172.21.15.89:0/1360384274 >> v2:172.21.15.89:3300/0 conn(0x7f2e2012f8b0 0x7f2e2012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1f7fe700 1 -- 172.21.15.89:0/1360384274 >> v2:172.21.15.89:3300/0 conn(0x7f2e2012f8b0 msgr2=0x7f2e2012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1f7fe700 1 --2- 172.21.15.89:0/1360384274 >> v2:172.21.15.89:3300/0 conn(0x7f2e2012f8b0 0x7f2e2012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1f7fe700 1 -- 172.21.15.89:0/1360384274 >> v1:172.21.15.89:6789/0 conn(0x7f2e20130260 legacy=0x7f2e20136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1f7fe700 1 -- 172.21.15.89:0/1360384274 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e20071780 con 0x7f2e2012cf60 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1f7fe700 1 --2- 172.21.15.89:0/1360384274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012cf60 0x7f2e2006f5d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000bb90 tx=0x7f2e1000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2061769c3bdec19b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1e7fc700 1 -- 172.21.15.89:0/1360384274 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000dc20 con 0x7f2e2012cf60 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1e7fc700 1 -- 172.21.15.89:0/1360384274 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1000dd80 con 0x7f2e2012cf60 2023-09-30T22:22:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.083+0000 7f2e1e7fc700 1 -- 172.21.15.89:0/1360384274 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e10017630 con 0x7f2e2012cf60 2023-09-30T22:22:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.084+0000 7f2e25dbd700 1 -- 172.21.15.89:0/1360384274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012cf60 msgr2=0x7f2e2006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.084+0000 7f2e25dbd700 1 --2- 172.21.15.89:0/1360384274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012cf60 0x7f2e2006f5d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000bb90 tx=0x7f2e1000bea0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.084+0000 7f2e25dbd700 1 -- 172.21.15.89:0/1360384274 shutdown_connections 2023-09-30T22:22:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.084+0000 7f2e25dbd700 1 --2- 172.21.15.89:0/1360384274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012cf60 0x7f2e2006f5d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.084+0000 7f2e25dbd700 1 --2- 172.21.15.89:0/1360384274 >> v2:172.21.15.89:3300/0 conn(0x7f2e2012f8b0 0x7f2e2012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.084+0000 7f2e25dbd700 1 -- 172.21.15.89:0/1360384274 >> 172.21.15.89:0/1360384274 conn(0x7f2e200a2aa0 msgr2=0x7f2e200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.084+0000 7f2e25dbd700 1 -- 172.21.15.89:0/1360384274 shutdown_connections 2023-09-30T22:22:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.084+0000 7f2e25dbd700 1 -- 172.21.15.89:0/1360384274 wait complete. 2023-09-30T22:22:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.085+0000 7f2e25dbd700 1 Processor -- start 2023-09-30T22:22:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.086+0000 7f2e25dbd700 1 -- start start 2023-09-30T22:22:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.086+0000 7f2e25dbd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2012cf60 0x7f2e201c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.086+0000 7f2e25dbd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012f8b0 0x7f2e201c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.086+0000 7f2e25dbd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e20130260 0x7f2e201c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.086+0000 7f2e25dbd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2012d820 con 0x7f2e20130260 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.086+0000 7f2e25dbd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2009c6d0 con 0x7f2e2012f8b0 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.086+0000 7f2e25dbd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e20129fa0 con 0x7f2e2012cf60 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.086+0000 7f2e1effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012f8b0 0x7f2e201c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1effd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012f8b0 0x7f2e201c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51402/0 (socket says 172.21.15.89:51402) 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1effd700 1 -- 172.21.15.89:0/710272775 learned_addr learned my addr 172.21.15.89:0/710272775 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1f7fe700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2012cf60 0x7f2e201c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1ffff700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e20130260 0x7f2e201c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1effd700 1 -- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2012cf60 msgr2=0x7f2e201c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1effd700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2012cf60 0x7f2e201c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1effd700 1 -- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e20130260 msgr2=0x7f2e201c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1effd700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e20130260 0x7f2e201c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1effd700 1 -- 172.21.15.89:0/710272775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1000b7f0 con 0x7f2e2012f8b0 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e1effd700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012f8b0 0x7f2e201c5940 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400e9b0 tx=0x7f2e1400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e07fff700 1 -- 172.21.15.89:0/710272775 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14009710 con 0x7f2e2012f8b0 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e07fff700 1 -- 172.21.15.89:0/710272775 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1400c850 con 0x7f2e2012f8b0 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.087+0000 7f2e07fff700 1 -- 172.21.15.89:0/710272775 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400cbe0 con 0x7f2e2012f8b0 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.088+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2009db90 con 0x7f2e2012f8b0 2023-09-30T22:22:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.088+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2009cf70 con 0x7f2e2012f8b0 2023-09-30T22:22:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.088+0000 7f2e07fff700 1 -- 172.21.15.89:0/710272775 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e1400cd40 con 0x7f2e2012f8b0 2023-09-30T22:22:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.089+0000 7f2e07fff700 1 -- 172.21.15.89:0/710272775 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2e1403f640 con 0x7f2e2012f8b0 2023-09-30T22:22:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.089+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2009cf70 con 0x7f2e2012f8b0 2023-09-30T22:22:45.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.093+0000 7f2e07fff700 1 -- 172.21.15.89:0/710272775 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e2009cf70 con 0x7f2e2012f8b0 2023-09-30T22:22:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd91eb6700 1 Processor -- start 2023-09-30T22:22:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd91eb6700 1 -- start start 2023-09-30T22:22:45.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd91eb6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd8c09d6f0 0x7fcd8c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd91eb6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09e000 0x7fcd8c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd91eb6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd91eb6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c09f520 con 0x7fcd8c09d6f0 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd91eb6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c09f130 con 0x7fcd8c09e410 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd91eb6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c09e930 con 0x7fcd8c09e000 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd8b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd8c09d6f0 0x7fcd8c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd8b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd8c09d6f0 0x7fcd8c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57104/0 (socket says 172.21.15.89:57104) 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd8bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.107+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/3931480616 learned_addr learned my addr 172.21.15.89:0/3931480616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8affd700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09e000 0x7fcd8c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09e000 msgr2=0x7fcd8c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8b7fe700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09e000 0x7fcd8c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 msgr2=0x7fcd8c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8b7fe700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8b7fe700 1 -- 172.21.15.89:0/3931480616 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c09b2d0 con 0x7fcd8c09d6f0 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8b7fe700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09d6f0 0x7fcd8c09dac0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009920 tx=0x7fcd7c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c98d810fafa83 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8a7fc700 1 -- 172.21.15.89:0/3931480616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c010a90 con 0x7fcd8c09d6f0 2023-09-30T22:22:45.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8a7fc700 1 -- 172.21.15.89:0/3931480616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7c010bf0 con 0x7fcd8c09d6f0 2023-09-30T22:22:45.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.108+0000 7fcd8a7fc700 1 -- 172.21.15.89:0/3931480616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c018440 con 0x7fcd8c09d6f0 2023-09-30T22:22:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09d6f0 msgr2=0x7fcd8c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09d6f0 0x7fcd8c09dac0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009920 tx=0x7fcd7c009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3931480616 shutdown_connections 2023-09-30T22:22:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09d6f0 0x7fcd8c09dac0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 --2- 172.21.15.89:0/3931480616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09e000 0x7fcd8c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3931480616 >> 172.21.15.89:0/3931480616 conn(0x7fcd8c066f50 msgr2=0x7fcd8c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3931480616 shutdown_connections 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.109+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3931480616 wait complete. 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd91eb6700 1 Processor -- start 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd91eb6700 1 -- start start 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd91eb6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09d6f0 0x7fcd8c11e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd91eb6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09e000 0x7fcd8c11e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd91eb6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c11eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd91eb6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c121c60 con 0x7fcd8c09e000 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd91eb6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c11f540 con 0x7fcd8c09e410 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd91eb6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1216b0 con 0x7fcd8c09d6f0 2023-09-30T22:22:45.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09e000 0x7fcd8c11e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09e000 0x7fcd8c11e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57108/0 (socket says 172.21.15.89:57108) 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd8affd700 1 -- 172.21.15.89:0/3149333200 learned_addr learned my addr 172.21.15.89:0/3149333200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd8b7fe700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09d6f0 0x7fcd8c11e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.110+0000 7fcd8bfff700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c11eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd8affd700 1 -- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09d6f0 msgr2=0x7fcd8c11e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd8affd700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09d6f0 0x7fcd8c11e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd8affd700 1 -- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 msgr2=0x7fcd8c11eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd8affd700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c11eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd8affd700 1 -- 172.21.15.89:0/3149333200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84009a70 con 0x7fcd8c09e000 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd8affd700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09e000 0x7fcd8c11e620 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000d9b0 tx=0x7fcd8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:45.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd73fff700 1 -- 172.21.15.89:0/3149333200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8000a9d0 con 0x7fcd8c09e000 2023-09-30T22:22:45.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c009550 con 0x7fcd8c09e000 2023-09-30T22:22:45.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1218e0 con 0x7fcd8c09e000 2023-09-30T22:22:45.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd73fff700 1 -- 172.21.15.89:0/3149333200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8000ab30 con 0x7fcd8c09e000 2023-09-30T22:22:45.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.111+0000 7fcd73fff700 1 -- 172.21.15.89:0/3149333200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8000aea0 con 0x7fcd8c09e000 2023-09-30T22:22:45.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.112+0000 7fcd73fff700 1 -- 172.21.15.89:0/3149333200 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd80019400 con 0x7fcd8c09e000 2023-09-30T22:22:45.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.112+0000 7fcd73fff700 1 -- 172.21.15.89:0/3149333200 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcd8003f980 con 0x7fcd8c09e000 2023-09-30T22:22:45.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.112+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c0052e0 con 0x7fcd8c09e000 2023-09-30T22:22:45.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.117+0000 7fcd73fff700 1 -- 172.21.15.89:0/3149333200 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd800178f0 con 0x7fcd8c09e000 2023-09-30T22:22:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.248+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f99d812eb40 con 0x7f99d8134630 2023-09-30T22:22:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.249+0000 7f99d57fa700 1 -- 172.21.15.89:0/115315721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99c4015020 con 0x7f99d8134630 2023-09-30T22:22:45.249 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:45.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.250+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d8134630 msgr2=0x7f99d8119260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.250+0000 7f99df023700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d8134630 0x7f99d8119260 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f99c40098f0 tx=0x7f99c4004750 comp rx=0 tx=0).stop 2023-09-30T22:22:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.251+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 shutdown_connections 2023-09-30T22:22:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.251+0000 7f99df023700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99d8136f50 0x7f99d81197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.251+0000 7f99df023700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f99d8131810 0x7f99d8118d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.251+0000 7f99df023700 1 --2- 172.21.15.89:0/115315721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99d8134630 0x7f99d8119260 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.251+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 >> 172.21.15.89:0/115315721 conn(0x7f99d80a2aa0 msgr2=0x7f99d8133120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.251+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 shutdown_connections 2023-09-30T22:22:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.251+0000 7f99df023700 1 -- 172.21.15.89:0/115315721 wait complete. 2023-09-30T22:22:45.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:45.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.283+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fabec09d180 con 0x7fabec130260 2023-09-30T22:22:45.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.283+0000 7fabd3fff700 1 -- 172.21.15.89:0/2217575399 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fabec09d180 con 0x7fabec130260 2023-09-30T22:22:45.284 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.285+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec130260 msgr2=0x7fabec09a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.285+0000 7fabf2018700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec130260 0x7fabec09a7d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fabe400cbe0 tx=0x7fabe400cef0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.285+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 shutdown_connections 2023-09-30T22:22:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.285+0000 7fabf2018700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fabec130260 0x7fabec09a7d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.285+0000 7fabf2018700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabec12cf60 0x7fabec099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.285+0000 7fabf2018700 1 --2- 172.21.15.89:0/2217575399 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fabec12f8b0 0x7fabec09a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.285+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 >> 172.21.15.89:0/2217575399 conn(0x7fabec0a2aa0 msgr2=0x7fabec06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.286+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 shutdown_connections 2023-09-30T22:22:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.286+0000 7fabf2018700 1 -- 172.21.15.89:0/2217575399 wait complete. 2023-09-30T22:22:45.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:45.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.303+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2e2009cf70 con 0x7f2e2012f8b0 2023-09-30T22:22:45.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.303+0000 7f2e07fff700 1 -- 172.21.15.89:0/710272775 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e2009cf70 con 0x7f2e2012f8b0 2023-09-30T22:22:45.304 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012f8b0 msgr2=0x7f2e201c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012f8b0 0x7f2e201c5940 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400e9b0 tx=0x7f2e1400ed70 comp rx=0 tx=0).stop 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 shutdown_connections 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e20130260 0x7f2e201c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2e2012cf60 0x7f2e201c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 --2- 172.21.15.89:0/710272775 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2e2012f8b0 0x7f2e201c5940 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 >> 172.21.15.89:0/710272775 conn(0x7f2e200a2aa0 msgr2=0x7f2e2006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 shutdown_connections 2023-09-30T22:22:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.306+0000 7f2e25dbd700 1 -- 172.21.15.89:0/710272775 wait complete. 2023-09-30T22:22:45.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:45.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.325+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcd6c005590 con 0x7fcd8c09e000 2023-09-30T22:22:45.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.325+0000 7fcd73fff700 1 -- 172.21.15.89:0/3149333200 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd8000e360 con 0x7fcd8c09e000 2023-09-30T22:22:45.326 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09e000 msgr2=0x7fcd8c11e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09e000 0x7fcd8c11e620 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000d9b0 tx=0x7fcd8000dcc0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 shutdown_connections 2023-09-30T22:22:45.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd8c09e410 0x7fcd8c11eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd8c09d6f0 0x7fcd8c11e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 --2- 172.21.15.89:0/3149333200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd8c09e000 0x7fcd8c11e620 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 >> 172.21.15.89:0/3149333200 conn(0x7fcd8c066f50 msgr2=0x7fcd8c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 shutdown_connections 2023-09-30T22:22:45.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:45.327+0000 7fcd91eb6700 1 -- 172.21.15.89:0/3149333200 wait complete. 2023-09-30T22:22:45.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:46.265 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:46.299 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:46.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:46.341 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:46.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.408+0000 7f655367e700 1 Processor -- start 2023-09-30T22:22:46.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.409+0000 7f655367e700 1 -- start start 2023-09-30T22:22:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.409+0000 7f655367e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c1338a0 0x7f654c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.409+0000 7f655367e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c09e930 0x7f654c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.409+0000 7f655367e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654c06ef30 con 0x7f654c09de50 2023-09-30T22:22:46.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.409+0000 7f655367e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c06fef0 con 0x7f654c1338a0 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.409+0000 7f655367e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f654c1334b0 con 0x7f654c09e930 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.409+0000 7f6550c19700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f654c09de50 0x7f654c09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57956/0 (socket says 172.21.15.89:57956) 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.409+0000 7f6550c19700 1 -- 172.21.15.89:0/4125379161 learned_addr learned my addr 172.21.15.89:0/4125379161 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f655141a700 1 --2- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c1338a0 0x7f654c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f6551c1b700 1 --2- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c09e930 0x7f654c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f6543fff700 1 -- 172.21.15.89:0/4125379161 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667891999 0 0) 0x7f654c06ef30 con 0x7f654c09de50 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f6543fff700 1 -- 172.21.15.89:0/4125379161 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65380035b0 con 0x7f654c09de50 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f655141a700 1 -- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c09e930 msgr2=0x7f654c13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f655141a700 1 --2- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c09e930 0x7f654c13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f655141a700 1 -- 172.21.15.89:0/4125379161 >> v1:172.21.15.89:6789/0 conn(0x7f654c09de50 legacy=0x7f654c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f655141a700 1 -- 172.21.15.89:0/4125379161 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c06fc30 con 0x7f654c1338a0 2023-09-30T22:22:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f655141a700 1 --2- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c1338a0 0x7f654c133c70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6548009870 tx=0x7f6548009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b064eeb6a36a621 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f6543fff700 1 -- 172.21.15.89:0/4125379161 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548011830 con 0x7f654c1338a0 2023-09-30T22:22:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f6543fff700 1 -- 172.21.15.89:0/4125379161 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6548011990 con 0x7f654c1338a0 2023-09-30T22:22:46.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.410+0000 7f6543fff700 1 -- 172.21.15.89:0/4125379161 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548011d00 con 0x7f654c1338a0 2023-09-30T22:22:46.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.411+0000 7f655367e700 1 -- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c1338a0 msgr2=0x7f654c133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.411+0000 7f655367e700 1 --2- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c1338a0 0x7f654c133c70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6548009870 tx=0x7f6548009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:46.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.411+0000 7f655367e700 1 -- 172.21.15.89:0/4125379161 shutdown_connections 2023-09-30T22:22:46.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.411+0000 7f655367e700 1 --2- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c1338a0 0x7f654c133c70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.411+0000 7f655367e700 1 --2- 172.21.15.89:0/4125379161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c09e930 0x7f654c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.411+0000 7f655367e700 1 -- 172.21.15.89:0/4125379161 >> 172.21.15.89:0/4125379161 conn(0x7f654c0a2aa0 msgr2=0x7f654c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.411+0000 7f655367e700 1 -- 172.21.15.89:0/4125379161 shutdown_connections 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.411+0000 7f655367e700 1 -- 172.21.15.89:0/4125379161 wait complete. 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f655367e700 1 Processor -- start 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f655367e700 1 -- start start 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f655367e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654c09de50 0x7f654c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f655367e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c09e930 0x7f654c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f655367e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c1338a0 0x7f654c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f655367e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c11b4d0 con 0x7f654c09de50 2023-09-30T22:22:46.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f655367e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c11be20 con 0x7f654c09e930 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f655367e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f654c11a0a0 con 0x7f654c1338a0 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f6550c19700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c09e930 0x7f654c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f6550c19700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c09e930 0x7f654c119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51428/0 (socket says 172.21.15.89:51428) 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.413+0000 7f6550c19700 1 -- 172.21.15.89:0/488230252 learned_addr learned my addr 172.21.15.89:0/488230252 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f655141a700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654c09de50 0x7f654c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6551c1b700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c1338a0 0x7f654c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6550c19700 1 -- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c1338a0 msgr2=0x7f654c119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6550c19700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c1338a0 0x7f654c119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6550c19700 1 -- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654c09de50 msgr2=0x7f654c118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6550c19700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654c09de50 0x7f654c118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6550c19700 1 -- 172.21.15.89:0/488230252 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480094a0 con 0x7f654c09e930 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6550c19700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c09e930 0x7f654c119210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f653c005910 tx=0x7f653c00c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6541ffb700 1 -- 172.21.15.89:0/488230252 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c005d70 con 0x7f654c09e930 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6541ffb700 1 -- 172.21.15.89:0/488230252 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c00ce80 con 0x7f654c09e930 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654c0a0be0 con 0x7f654c09e930 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.414+0000 7f6541ffb700 1 -- 172.21.15.89:0/488230252 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c0072d0 con 0x7f654c09e930 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.415+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654c06f640 con 0x7f654c09e930 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.415+0000 7f6541ffb700 1 -- 172.21.15.89:0/488230252 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f653c007430 con 0x7f654c09e930 2023-09-30T22:22:46.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.416+0000 7f6541ffb700 1 -- 172.21.15.89:0/488230252 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f653c012920 con 0x7f654c09e930 2023-09-30T22:22:46.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.416+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654c11baa0 con 0x7f654c09e930 2023-09-30T22:22:46.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.425+0000 7f6541ffb700 1 -- 172.21.15.89:0/488230252 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f654c11baa0 con 0x7f654c09e930 2023-09-30T22:22:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.447+0000 7f786ff6d700 1 Processor -- start 2023-09-30T22:22:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786ff6d700 1 -- start start 2023-09-30T22:22:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786ff6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7868131810 0x7f7868131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786ff6d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f786809a580 0x7f7868139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786ff6d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78681313c0 con 0x7f7868099a40 2023-09-30T22:22:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786ff6d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786806e8c0 con 0x7f786809a580 2023-09-30T22:22:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786ff6d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7868071d20 con 0x7f7868131810 2023-09-30T22:22:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786dd09700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7868131810 0x7f7868131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786dd09700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7868131810 0x7f7868131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55772/0 (socket says 172.21.15.89:55772) 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786dd09700 1 -- 172.21.15.89:0/1572161617 learned_addr learned my addr 172.21.15.89:0/1572161617 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.448+0000 7f786e50a700 1 --2- 172.21.15.89:0/1572161617 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f786809a580 0x7f7868139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.449+0000 7f786dd09700 1 -- 172.21.15.89:0/1572161617 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f786809a580 msgr2=0x7f7868139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.449+0000 7f786dd09700 1 --2- 172.21.15.89:0/1572161617 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f786809a580 0x7f7868139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.449+0000 7f786dd09700 1 -- 172.21.15.89:0/1572161617 >> v1:172.21.15.89:6789/0 conn(0x7f7868099a40 legacy=0x7f7868099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.449+0000 7f786dd09700 1 -- 172.21.15.89:0/1572161617 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680721a0 con 0x7f7868131810 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.449+0000 7f786dd09700 1 --2- 172.21.15.89:0/1572161617 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7868131810 0x7f7868131be0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7864009920 tx=0x7f7864009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6318830bb02671b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.449+0000 7f786cd07700 1 -- 172.21.15.89:0/1572161617 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400ba60 con 0x7f7868131810 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.449+0000 7f786cd07700 1 -- 172.21.15.89:0/1572161617 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400bbc0 con 0x7f7868131810 2023-09-30T22:22:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.449+0000 7f786cd07700 1 -- 172.21.15.89:0/1572161617 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864009e70 con 0x7f7868131810 2023-09-30T22:22:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.450+0000 7f786ff6d700 1 -- 172.21.15.89:0/1572161617 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7868131810 msgr2=0x7f7868131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.450+0000 7f786ff6d700 1 --2- 172.21.15.89:0/1572161617 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7868131810 0x7f7868131be0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7864009920 tx=0x7f7864009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.450+0000 7f786ff6d700 1 -- 172.21.15.89:0/1572161617 shutdown_connections 2023-09-30T22:22:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.450+0000 7f786ff6d700 1 --2- 172.21.15.89:0/1572161617 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f786809a580 0x7f7868139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.450+0000 7f786ff6d700 1 --2- 172.21.15.89:0/1572161617 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7868131810 0x7f7868131be0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.450+0000 7f786ff6d700 1 -- 172.21.15.89:0/1572161617 >> 172.21.15.89:0/1572161617 conn(0x7f78680a2aa0 msgr2=0x7f78680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:46.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.450+0000 7f786ff6d700 1 -- 172.21.15.89:0/1572161617 shutdown_connections 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.450+0000 7f786ff6d700 1 -- 172.21.15.89:0/1572161617 wait complete. 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.451+0000 7f786ff6d700 1 Processor -- start 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.451+0000 7f786ff6d700 1 -- start start 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.451+0000 7f786ff6d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7868099a40 0x7f786811f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.451+0000 7f786ff6d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f786809a580 0x7f786811f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.451+0000 7f786ff6d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7868131810 0x7f786811faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.451+0000 7f786ff6d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7868122b00 con 0x7f7868099a40 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.451+0000 7f786ff6d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7868120bf0 con 0x7f7868131810 2023-09-30T22:22:46.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.451+0000 7f786ff6d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78680a1c40 con 0x7f786809a580 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786e50a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7868131810 0x7f786811faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786e50a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7868131810 0x7f786811faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51442/0 (socket says 172.21.15.89:51442) 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786e50a700 1 -- 172.21.15.89:0/3774797545 learned_addr learned my addr 172.21.15.89:0/3774797545 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786d508700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f786809a580 0x7f786811f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786dd09700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7868099a40 0x7f786811f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786dd09700 1 -- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f786809a580 msgr2=0x7f786811f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786dd09700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f786809a580 0x7f786811f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786dd09700 1 -- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7868131810 msgr2=0x7f786811faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786dd09700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7868131810 0x7f786811faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786dd09700 1 -- 172.21.15.89:0/3774797545 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009550 con 0x7f7868099a40 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786e50a700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7868131810 0x7f786811faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f786dd09700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7868099a40 0x7f786811f070 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7864016000 tx=0x7f7864004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.452+0000 7f785e7fc700 1 -- 172.21.15.89:0/3774797545 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864004300 con 0x7f7868099a40 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.453+0000 7f785e7fc700 1 -- 172.21.15.89:0/3774797545 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786401f070 con 0x7f7868099a40 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.453+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786806eb00 con 0x7f7868099a40 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.453+0000 7f785e7fc700 1 -- 172.21.15.89:0/3774797545 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640036a0 con 0x7f7868099a40 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.453+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7868123000 con 0x7f7868099a40 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.453+0000 7f785e7fc700 1 -- 172.21.15.89:0/3774797545 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7864021030 con 0x7f7868099a40 2023-09-30T22:22:46.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.454+0000 7f785e7fc700 1 -- 172.21.15.89:0/3774797545 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f78640405e0 con 0x7f7868099a40 2023-09-30T22:22:46.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.454+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78681224c0 con 0x7f7868099a40 2023-09-30T22:22:46.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.459+0000 7f785e7fc700 1 -- 172.21.15.89:0/3774797545 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78681224c0 con 0x7f7868099a40 2023-09-30T22:22:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8cd0d700 1 Processor -- start 2023-09-30T22:22:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8cd0d700 1 -- start start 2023-09-30T22:22:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8cd0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8cd0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c88134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8cd0d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8809b120 con 0x7f9c8809e000 2023-09-30T22:22:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8cd0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8806b460 con 0x7f9c8809d6f0 2023-09-30T22:22:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8cd0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8809a7e0 con 0x7f9c8809e410 2023-09-30T22:22:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51450/0 (socket says 172.21.15.89:51450) 2023-09-30T22:22:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.486+0000 7f9c8659c700 1 -- 172.21.15.89:0/3047042351 learned_addr learned my addr 172.21.15.89:0/3047042351 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c86d9d700 1 --2- 172.21.15.89:0/3047042351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c88134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8559a700 1 -- 172.21.15.89:0/3047042351 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640640005 0 0) 0x7f9c8809b120 con 0x7f9c8809e000 2023-09-30T22:22:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8559a700 1 -- 172.21.15.89:0/3047042351 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c700035b0 con 0x7f9c8809e000 2023-09-30T22:22:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8659c700 1 -- 172.21.15.89:0/3047042351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 msgr2=0x7f9c88134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:46.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8659c700 1 --2- 172.21.15.89:0/3047042351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c88134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8659c700 1 -- 172.21.15.89:0/3047042351 >> v1:172.21.15.89:6789/0 conn(0x7f9c8809e000 legacy=0x7f9c881339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8659c700 1 -- 172.21.15.89:0/3047042351 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8809a500 con 0x7f9c8809d6f0 2023-09-30T22:22:46.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8659c700 1 --2- 172.21.15.89:0/3047042351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809d6f0 0x7f9c8809dac0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9c78013210 tx=0x7f9c78013520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b10d69fd0eefdf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:46.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8559a700 1 -- 172.21.15.89:0/3047042351 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78016070 con 0x7f9c8809d6f0 2023-09-30T22:22:46.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8559a700 1 -- 172.21.15.89:0/3047042351 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78013db0 con 0x7f9c8809d6f0 2023-09-30T22:22:46.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.487+0000 7f9c8559a700 1 -- 172.21.15.89:0/3047042351 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c780112d0 con 0x7f9c8809d6f0 2023-09-30T22:22:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.488+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/3047042351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809d6f0 msgr2=0x7f9c8809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.488+0000 7f9c8cd0d700 1 --2- 172.21.15.89:0/3047042351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809d6f0 0x7f9c8809dac0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9c78013210 tx=0x7f9c78013520 comp rx=0 tx=0).stop 2023-09-30T22:22:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.488+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/3047042351 shutdown_connections 2023-09-30T22:22:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.488+0000 7f9c8cd0d700 1 --2- 172.21.15.89:0/3047042351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.488+0000 7f9c8cd0d700 1 --2- 172.21.15.89:0/3047042351 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c88134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.488+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/3047042351 >> 172.21.15.89:0/3047042351 conn(0x7f9c88066f50 msgr2=0x7f9c8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.488+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/3047042351 shutdown_connections 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.488+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/3047042351 wait complete. 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.489+0000 7f9c8cd0d700 1 Processor -- start 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.489+0000 7f9c8cd0d700 1 -- start start 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8cd0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c8809d6f0 0x7f9c881c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8cd0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809e000 0x7f9c881c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8cd0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c881c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8cd0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8809c780 con 0x7f9c8809d6f0 2023-09-30T22:22:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8cd0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8811e2f0 con 0x7f9c8809e000 2023-09-30T22:22:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8cd0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c881c1610 con 0x7f9c8809e410 2023-09-30T22:22:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c8809d6f0 0x7f9c881c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c86d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c881c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c8809d6f0 0x7f9c881c0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57136/0 (socket says 172.21.15.89:57136) 2023-09-30T22:22:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.490+0000 7f9c8659c700 1 -- 172.21.15.89:0/123712207 learned_addr learned my addr 172.21.15.89:0/123712207 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c86d9d700 1 -- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809e000 msgr2=0x7f9c881c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c86d9d700 1 --2- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809e000 0x7f9c881c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c86d9d700 1 -- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c8809d6f0 msgr2=0x7f9c881c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c86d9d700 1 --2- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c8809d6f0 0x7f9c881c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c86d9d700 1 -- 172.21.15.89:0/123712207 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78011df0 con 0x7f9c8809e410 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c86d9d700 1 --2- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c881c3bd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9c80007d90 tx=0x7f9c8000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c76ffd700 1 -- 172.21.15.89:0/123712207 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80018070 con 0x7f9c8809e410 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c76ffd700 1 -- 172.21.15.89:0/123712207 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8000ebe0 con 0x7f9c8809e410 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c76ffd700 1 -- 172.21.15.89:0/123712207 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80003a20 con 0x7f9c8809e410 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8811d330 con 0x7f9c8809e410 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.491+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8811c6c0 con 0x7f9c8809e410 2023-09-30T22:22:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.492+0000 7f9c76ffd700 1 -- 172.21.15.89:0/123712207 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c80014420 con 0x7f9c8809e410 2023-09-30T22:22:46.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.492+0000 7f9c76ffd700 1 -- 172.21.15.89:0/123712207 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9c8003f690 con 0x7f9c8809e410 2023-09-30T22:22:46.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.492+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c88074ea0 con 0x7f9c8809e410 2023-09-30T22:22:46.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.497+0000 7f9c76ffd700 1 -- 172.21.15.89:0/123712207 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c800128d0 con 0x7f9c8809e410 2023-09-30T22:22:46.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.500+0000 7f115c353700 1 Processor -- start 2023-09-30T22:22:46.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.500+0000 7f115c353700 1 -- start start 2023-09-30T22:22:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f115c353700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11541316c0 0x7f1154131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f115c353700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f115409d9f0 0x7f115406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f115c353700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115409e130 con 0x7f115406acb0 2023-09-30T22:22:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f115c353700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11541312b0 con 0x7f11541316c0 2023-09-30T22:22:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f115c353700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f115409f6e0 con 0x7f115409d9f0 2023-09-30T22:22:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f11598ee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f115409d9f0 0x7f115406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f115a0ef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11541316c0 0x7f1154131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f11598ee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f115409d9f0 0x7f115406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55818/0 (socket says 172.21.15.89:55818) 2023-09-30T22:22:46.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f11598ee700 1 -- 172.21.15.89:0/991018010 learned_addr learned my addr 172.21.15.89:0/991018010 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f11598ee700 1 -- 172.21.15.89:0/991018010 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11541316c0 msgr2=0x7f1154131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f11598ee700 1 --2- 172.21.15.89:0/991018010 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11541316c0 0x7f1154131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f11598ee700 1 -- 172.21.15.89:0/991018010 >> v1:172.21.15.89:6789/0 conn(0x7f115406acb0 legacy=0x7f1154138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.501+0000 7f11598ee700 1 -- 172.21.15.89:0/991018010 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115409e760 con 0x7f115409d9f0 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.502+0000 7f11598ee700 1 --2- 172.21.15.89:0/991018010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f115409d9f0 0x7f115406a770 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1144009920 tx=0x7f1144009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6102372b2ce09f2e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.502+0000 7f11590ed700 1 -- 172.21.15.89:0/991018010 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144011830 con 0x7f115409d9f0 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.502+0000 7f11590ed700 1 -- 172.21.15.89:0/991018010 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1144011990 con 0x7f115409d9f0 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.502+0000 7f11590ed700 1 -- 172.21.15.89:0/991018010 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144011d20 con 0x7f115409d9f0 2023-09-30T22:22:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.502+0000 7f115c353700 1 -- 172.21.15.89:0/991018010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f115409d9f0 msgr2=0x7f115406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.502+0000 7f115c353700 1 --2- 172.21.15.89:0/991018010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f115409d9f0 0x7f115406a770 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1144009920 tx=0x7f1144009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:46.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.503+0000 7f115c353700 1 -- 172.21.15.89:0/991018010 shutdown_connections 2023-09-30T22:22:46.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.503+0000 7f115c353700 1 --2- 172.21.15.89:0/991018010 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11541316c0 0x7f1154131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.503+0000 7f115c353700 1 --2- 172.21.15.89:0/991018010 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f115409d9f0 0x7f115406a770 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.503+0000 7f115c353700 1 -- 172.21.15.89:0/991018010 >> 172.21.15.89:0/991018010 conn(0x7f1154066f50 msgr2=0x7f115409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:46.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.503+0000 7f115c353700 1 -- 172.21.15.89:0/991018010 shutdown_connections 2023-09-30T22:22:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.503+0000 7f115c353700 1 -- 172.21.15.89:0/991018010 wait complete. 2023-09-30T22:22:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.504+0000 7f115c353700 1 Processor -- start 2023-09-30T22:22:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.504+0000 7f115c353700 1 -- start start 2023-09-30T22:22:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115c353700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f115406acb0 0x7f1154072050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115c353700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f115409d9f0 0x7f1154127660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115c353700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11541316c0 0x7f1154127ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115c353700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115412b900 con 0x7f115406acb0 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115c353700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115412b380 con 0x7f115409d9f0 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115c353700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11541b7680 con 0x7f11541316c0 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11541316c0 0x7f1154127ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a0ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f115406acb0 0x7f1154072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11541316c0 0x7f1154127ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55830/0 (socket says 172.21.15.89:55830) 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f11598ee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f115409d9f0 0x7f1154127660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 -- 172.21.15.89:0/2654118038 learned_addr learned my addr 172.21.15.89:0/2654118038 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:46.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 -- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f115409d9f0 msgr2=0x7f1154127660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 --2- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f115409d9f0 0x7f1154127660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 -- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f115406acb0 msgr2=0x7f1154072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 --2- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f115406acb0 0x7f1154072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 -- 172.21.15.89:0/2654118038 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1144009550 con 0x7f11541316c0 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f115a8f0700 1 --2- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11541316c0 0x7f1154127ba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f114c00cc50 tx=0x7f114c00cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f114affd700 1 -- 172.21.15.89:0/2654118038 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c003c00 con 0x7f11541316c0 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f114affd700 1 -- 172.21.15.89:0/2654118038 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c003d60 con 0x7f11541316c0 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.505+0000 7f114affd700 1 -- 172.21.15.89:0/2654118038 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c00aa50 con 0x7f11541316c0 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.506+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11541b87a0 con 0x7f11541316c0 2023-09-30T22:22:46.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.506+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11541b9b80 con 0x7f11541316c0 2023-09-30T22:22:46.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.506+0000 7f114affd700 1 -- 172.21.15.89:0/2654118038 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f114c00abb0 con 0x7f11541316c0 2023-09-30T22:22:46.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.506+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115412d2b0 con 0x7f11541316c0 2023-09-30T22:22:46.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.507+0000 7f114affd700 1 -- 172.21.15.89:0/2654118038 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f114c03f4c0 con 0x7f11541316c0 2023-09-30T22:22:46.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.515+0000 7f114affd700 1 -- 172.21.15.89:0/2654118038 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f114c019710 con 0x7f11541316c0 2023-09-30T22:22:46.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.678+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f654c11baa0 con 0x7f654c09e930 2023-09-30T22:22:46.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.679+0000 7f6541ffb700 1 -- 172.21.15.89:0/488230252 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f654c11baa0 con 0x7f654c09e930 2023-09-30T22:22:46.680 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:46.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.682+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c09e930 msgr2=0x7f654c119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.682+0000 7f655367e700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c09e930 0x7f654c119210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f653c005910 tx=0x7f653c00c7b0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.683+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 shutdown_connections 2023-09-30T22:22:46.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.683+0000 7f655367e700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f654c1338a0 0x7f654c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.683+0000 7f655367e700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654c09de50 0x7f654c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.683+0000 7f655367e700 1 --2- 172.21.15.89:0/488230252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f654c09e930 0x7f654c119210 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.683+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 >> 172.21.15.89:0/488230252 conn(0x7f654c0a2aa0 msgr2=0x7f654c12f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:46.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.683+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 shutdown_connections 2023-09-30T22:22:46.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.683+0000 7f655367e700 1 -- 172.21.15.89:0/488230252 wait complete. 2023-09-30T22:22:46.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.697+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f78681224c0 con 0x7f7868099a40 2023-09-30T22:22:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.697+0000 7f785e7fc700 1 -- 172.21.15.89:0/3774797545 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78681224c0 con 0x7f7868099a40 2023-09-30T22:22:46.698 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9c8811e430 con 0x7f9c8809e410 2023-09-30T22:22:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7868099a40 msgr2=0x7f786811f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7868099a40 0x7f786811f070 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7864016000 tx=0x7f7864004750 comp rx=0 tx=0).stop 2023-09-30T22:22:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 shutdown_connections 2023-09-30T22:22:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7868131810 0x7f786811faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7868099a40 0x7f786811f070 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 --2- 172.21.15.89:0/3774797545 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f786809a580 0x7f786811f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 >> 172.21.15.89:0/3774797545 conn(0x7f78680a2aa0 msgr2=0x7f786812ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 shutdown_connections 2023-09-30T22:22:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f786ff6d700 1 -- 172.21.15.89:0/3774797545 wait complete. 2023-09-30T22:22:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.699+0000 7f9c76ffd700 1 -- 172.21.15.89:0/123712207 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c800122e0 con 0x7f9c8809e410 2023-09-30T22:22:46.701 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 msgr2=0x7f9c881c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 --2- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c881c3bd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9c80007d90 tx=0x7f9c8000e3c0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 shutdown_connections 2023-09-30T22:22:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 --2- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c8809e410 0x7f9c881c3bd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 --2- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c8809d6f0 0x7f9c881c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 --2- 172.21.15.89:0/123712207 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9c8809e000 0x7f9c881c10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 >> 172.21.15.89:0/123712207 conn(0x7f9c88066f50 msgr2=0x7f9c88133c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 shutdown_connections 2023-09-30T22:22:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.702+0000 7f9c8cd0d700 1 -- 172.21.15.89:0/123712207 wait complete. 2023-09-30T22:22:46.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:46.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.712+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f115409e130 con 0x7f11541316c0 2023-09-30T22:22:46.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.712+0000 7f114affd700 1 -- 172.21.15.89:0/2654118038 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f115409e130 con 0x7f11541316c0 2023-09-30T22:22:46.714 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11541316c0 msgr2=0x7f1154127ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 --2- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11541316c0 0x7f1154127ba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f114c00cc50 tx=0x7f114c00cf60 comp rx=0 tx=0).stop 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 shutdown_connections 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 --2- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11541316c0 0x7f1154127ba0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 --2- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f115406acb0 0x7f1154072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 --2- 172.21.15.89:0/2654118038 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f115409d9f0 0x7f1154127660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 >> 172.21.15.89:0/2654118038 conn(0x7f1154066f50 msgr2=0x7f115412f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 shutdown_connections 2023-09-30T22:22:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:46.715+0000 7f115c353700 1 -- 172.21.15.89:0/2654118038 wait complete. 2023-09-30T22:22:46.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:46.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:47.697 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:47.714 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:47.718 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:47.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.852+0000 7f7590de5700 1 Processor -- start 2023-09-30T22:22:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.852+0000 7f7590de5700 1 -- start start 2023-09-30T22:22:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.853+0000 7f7590de5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f758c09d6f0 0x7f758c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.853+0000 7f7590de5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.853+0000 7f7590de5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 0x7f758c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.853+0000 7f7590de5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c09f520 con 0x7f758c09d6f0 2023-09-30T22:22:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.853+0000 7f7590de5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c09f130 con 0x7f758c09e000 2023-09-30T22:22:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.853+0000 7f7590de5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f758c09e930 con 0x7f758c09e410 2023-09-30T22:22:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f758a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f758c09d6f0 0x7f758c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f7589d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f758a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f758c09d6f0 0x7f758c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57166/0 (socket says 172.21.15.89:57166) 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f758a59c700 1 -- 172.21.15.89:0/266476952 learned_addr learned my addr 172.21.15.89:0/266476952 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f758ad9d700 1 --2- 172.21.15.89:0/266476952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 0x7f758c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f7589d9b700 1 -- 172.21.15.89:0/266476952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 msgr2=0x7f758c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f7589d9b700 1 --2- 172.21.15.89:0/266476952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 0x7f758c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f7589d9b700 1 -- 172.21.15.89:0/266476952 >> v2:172.21.15.89:3300/0 conn(0x7f758c09d6f0 msgr2=0x7f758c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f7589d9b700 1 --2- 172.21.15.89:0/266476952 >> v2:172.21.15.89:3300/0 conn(0x7f758c09d6f0 0x7f758c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f7589d9b700 1 -- 172.21.15.89:0/266476952 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c09b2d0 con 0x7f758c09e000 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f758a59c700 1 --2- 172.21.15.89:0/266476952 >> v2:172.21.15.89:3300/0 conn(0x7f758c09d6f0 0x7f758c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f7589d9b700 1 --2- 172.21.15.89:0/266476952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c130660 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7580009840 tx=0x7f7580009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98766ac62ae0d8d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f758959a700 1 -- 172.21.15.89:0/266476952 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580011830 con 0x7f758c09e000 2023-09-30T22:22:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f758959a700 1 -- 172.21.15.89:0/266476952 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7580011990 con 0x7f758c09e000 2023-09-30T22:22:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.854+0000 7f758959a700 1 -- 172.21.15.89:0/266476952 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580011d20 con 0x7f758c09e000 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 -- 172.21.15.89:0/266476952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 msgr2=0x7f758c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 --2- 172.21.15.89:0/266476952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c130660 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7580009840 tx=0x7f7580009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 -- 172.21.15.89:0/266476952 shutdown_connections 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 --2- 172.21.15.89:0/266476952 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 0x7f758c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 --2- 172.21.15.89:0/266476952 >> v2:172.21.15.89:3300/0 conn(0x7f758c09d6f0 0x7f758c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 --2- 172.21.15.89:0/266476952 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c130660 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 -- 172.21.15.89:0/266476952 >> 172.21.15.89:0/266476952 conn(0x7f758c066f50 msgr2=0x7f758c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 -- 172.21.15.89:0/266476952 shutdown_connections 2023-09-30T22:22:47.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.855+0000 7f7590de5700 1 -- 172.21.15.89:0/266476952 wait complete. 2023-09-30T22:22:47.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.857+0000 7f7590de5700 1 Processor -- start 2023-09-30T22:22:47.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.857+0000 7f7590de5700 1 -- start start 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7590de5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f758c09d6f0 0x7f758c121120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7590de5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c121660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7590de5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 0x7f758c121ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7590de5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c124520 con 0x7f758c09d6f0 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7590de5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c122270 con 0x7f758c09e000 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7590de5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0a2a90 con 0x7f758c09e410 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f758a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f758c09d6f0 0x7f758c121120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7589d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f758a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f758c09d6f0 0x7f758c121120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57174/0 (socket says 172.21.15.89:57174) 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f758a59c700 1 -- 172.21.15.89:0/1277829864 learned_addr learned my addr 172.21.15.89:0/1277829864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7589d9b700 1 -- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 msgr2=0x7f758c121ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7589d9b700 1 --2- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 0x7f758c121ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7589d9b700 1 -- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f758c09d6f0 msgr2=0x7f758c121120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7589d9b700 1 --2- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f758c09d6f0 0x7f758c121120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7589d9b700 1 -- 172.21.15.89:0/1277829864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c009550 con 0x7f758c09e000 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f758a59c700 1 --2- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f758c09d6f0 0x7f758c121120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.858+0000 7f7589d9b700 1 --2- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c121660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7580009840 tx=0x7f7580004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.859+0000 7f757affd700 1 -- 172.21.15.89:0/1277829864 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580004590 con 0x7f758c09e000 2023-09-30T22:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.859+0000 7f757affd700 1 -- 172.21.15.89:0/1277829864 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758001d7c0 con 0x7f758c09e000 2023-09-30T22:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.859+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75800094a0 con 0x7f758c09e000 2023-09-30T22:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.859+0000 7f757affd700 1 -- 172.21.15.89:0/1277829864 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758001db30 con 0x7f758c09e000 2023-09-30T22:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.859+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c122ef0 con 0x7f758c09e000 2023-09-30T22:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.860+0000 7f757affd700 1 -- 172.21.15.89:0/1277829864 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f758001dc90 con 0x7f758c09e000 2023-09-30T22:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.860+0000 7f757affd700 1 -- 172.21.15.89:0/1277829864 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7580044970 con 0x7f758c09e000 2023-09-30T22:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.860+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c074ea0 con 0x7f758c09e000 2023-09-30T22:22:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.869+0000 7f757affd700 1 -- 172.21.15.89:0/1277829864 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7580031450 con 0x7f758c09e000 2023-09-30T22:22:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.877+0000 7fa49774e700 1 Processor -- start 2023-09-30T22:22:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa49774e700 1 -- start start 2023-09-30T22:22:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa49774e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa49774e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa49774e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa49009e410 0x7fa490130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa49774e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49009f520 con 0x7fa49009e410 2023-09-30T22:22:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa49774e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49009f130 con 0x7fa49009d6f0 2023-09-30T22:22:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa49774e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa49009e930 con 0x7fa49009e000 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa4954ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa494ce9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa494ce9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55852/0 (socket says 172.21.15.89:55852) 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa494ce9700 1 -- 172.21.15.89:0/3177862696 learned_addr learned my addr 172.21.15.89:0/3177862696 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa4954ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51502/0 (socket says 172.21.15.89:51502) 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.878+0000 7fa495ceb700 1 --2- 172.21.15.89:0/3177862696 >> v2:172.21.15.89:3300/0 conn(0x7fa49009e410 0x7fa490130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa4954ea700 1 -- 172.21.15.89:0/3177862696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 msgr2=0x7fa490130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa4954ea700 1 --2- 172.21.15.89:0/3177862696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa4954ea700 1 -- 172.21.15.89:0/3177862696 >> v2:172.21.15.89:3300/0 conn(0x7fa49009e410 msgr2=0x7fa490130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa4954ea700 1 --2- 172.21.15.89:0/3177862696 >> v2:172.21.15.89:3300/0 conn(0x7fa49009e410 0x7fa490130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa4954ea700 1 -- 172.21.15.89:0/3177862696 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49009b2d0 con 0x7fa49009d6f0 2023-09-30T22:22:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa495ceb700 1 --2- 172.21.15.89:0/3177862696 >> v2:172.21.15.89:3300/0 conn(0x7fa49009e410 0x7fa490130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa494ce9700 1 --2- 172.21.15.89:0/3177862696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa4954ea700 1 --2- 172.21.15.89:0/3177862696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49009dac0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa48c010be0 tx=0x7fa48c010ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b94673cd1ba26d24 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa487fff700 1 -- 172.21.15.89:0/3177862696 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00be30 con 0x7fa49009d6f0 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa487fff700 1 -- 172.21.15.89:0/3177862696 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c018430 con 0x7fa49009d6f0 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.879+0000 7fa487fff700 1 -- 172.21.15.89:0/3177862696 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0187c0 con 0x7fa49009d6f0 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 -- 172.21.15.89:0/3177862696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 msgr2=0x7fa49009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 --2- 172.21.15.89:0/3177862696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49009dac0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa48c010be0 tx=0x7fa48c010ef0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 -- 172.21.15.89:0/3177862696 shutdown_connections 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 --2- 172.21.15.89:0/3177862696 >> v2:172.21.15.89:3300/0 conn(0x7fa49009e410 0x7fa490130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 --2- 172.21.15.89:0/3177862696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49009dac0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 --2- 172.21.15.89:0/3177862696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa490130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 -- 172.21.15.89:0/3177862696 >> 172.21.15.89:0/3177862696 conn(0x7fa490066f50 msgr2=0x7fa49009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 -- 172.21.15.89:0/3177862696 shutdown_connections 2023-09-30T22:22:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.880+0000 7fa49774e700 1 -- 172.21.15.89:0/3177862696 wait complete. 2023-09-30T22:22:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa49774e700 1 Processor -- start 2023-09-30T22:22:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa49774e700 1 -- start start 2023-09-30T22:22:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7f8e8687e700 1 Processor -- start 2023-09-30T22:22:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa49774e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49011e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa49774e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa49011e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa49774e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa49011ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa49774e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490121bf0 con 0x7fa49009e410 2023-09-30T22:22:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa49774e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49011f730 con 0x7fa49009d6f0 2023-09-30T22:22:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa49774e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa490121dd0 con 0x7fa49009e000 2023-09-30T22:22:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa494ce9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa49011e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa494ce9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa49011e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55856/0 (socket says 172.21.15.89:55856) 2023-09-30T22:22:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7fa494ce9700 1 -- 172.21.15.89:0/3637273558 learned_addr learned my addr 172.21.15.89:0/3637273558 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.881+0000 7f8e8687e700 1 -- start start 2023-09-30T22:22:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa495ceb700 1 --2- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa49011ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa494ce9700 1 -- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 msgr2=0x7fa49011e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa494ce9700 1 --2- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49011e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa494ce9700 1 -- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 msgr2=0x7fa49011ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa494ce9700 1 --2- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa49011ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa494ce9700 1 -- 172.21.15.89:0/3637273558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa480009550 con 0x7fa49009e000 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa494ce9700 1 --2- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa49011e830 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa48000ebb0 tx=0x7fa48000ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa485ffb700 1 -- 172.21.15.89:0/3637273558 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480004240 con 0x7fa49009e000 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa485ffb700 1 -- 172.21.15.89:0/3637273558 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4800043a0 con 0x7fa49009e000 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7fa485ffb700 1 -- 172.21.15.89:0/3637273558 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4800036a0 con 0x7fa49009e000 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa488009a70 con 0x7fa49009e000 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c010810 con 0x7fa49009e000 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7fa485ffb700 1 -- 172.21.15.89:0/3637273558 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa480003800 con 0x7fa49009e000 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7fa485ffb700 1 -- 172.21.15.89:0/3637273558 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa48003f810 con 0x7fa49009e000 2023-09-30T22:22:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4900a2a50 con 0x7fa49009e000 2023-09-30T22:22:47.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e8687e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130660 0x7f8e80130a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e8687e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e80130f70 0x7f8e8006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e8687e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e8006acb0 0x7f8e8006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e8687e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8009f520 con 0x7f8e80130f70 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e8687e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8009f130 con 0x7f8e80130660 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e8687e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8006b690 con 0x7f8e8006acb0 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e7f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e80130f70 0x7f8e8006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e7f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e80130f70 0x7f8e8006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57208/0 (socket says 172.21.15.89:57208) 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e7f7fe700 1 -- 172.21.15.89:0/2491805557 learned_addr learned my addr 172.21.15.89:0/2491805557 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e84e1b700 1 --2- 172.21.15.89:0/2491805557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e8006acb0 0x7f8e8006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e84e1b700 1 -- 172.21.15.89:0/2491805557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130660 msgr2=0x7f8e80130a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.882+0000 7f8e84e1b700 1 --2- 172.21.15.89:0/2491805557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130660 0x7f8e80130a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7f8e84e1b700 1 -- 172.21.15.89:0/2491805557 >> v2:172.21.15.89:3300/0 conn(0x7f8e80130f70 msgr2=0x7f8e8006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7f8e84e1b700 1 --2- 172.21.15.89:0/2491805557 >> v2:172.21.15.89:3300/0 conn(0x7f8e80130f70 0x7f8e8006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7f8e84e1b700 1 -- 172.21.15.89:0/2491805557 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80133900 con 0x7f8e8006acb0 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7f8e7f7fe700 1 --2- 172.21.15.89:0/2491805557 >> v2:172.21.15.89:3300/0 conn(0x7f8e80130f70 0x7f8e8006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7f8e84e1b700 1 --2- 172.21.15.89:0/2491805557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e8006acb0 0x7f8e8006b150 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009db0 tx=0x7f8e7801c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f227c6d7da43ccf7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7f8e7effd700 1 -- 172.21.15.89:0/2491805557 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7801cae0 con 0x7f8e8006acb0 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7f8e7effd700 1 -- 172.21.15.89:0/2491805557 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7801f070 con 0x7f8e8006acb0 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.883+0000 7f8e7effd700 1 -- 172.21.15.89:0/2491805557 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7801ce80 con 0x7f8e8006acb0 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7f8e8687e700 1 -- 172.21.15.89:0/2491805557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e8006acb0 msgr2=0x7f8e8006b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7f8e8687e700 1 --2- 172.21.15.89:0/2491805557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e8006acb0 0x7f8e8006b150 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009db0 tx=0x7f8e7801c340 comp rx=0 tx=0).stop 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7f8e8687e700 1 -- 172.21.15.89:0/2491805557 shutdown_connections 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7f8e8687e700 1 --2- 172.21.15.89:0/2491805557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e8006acb0 0x7f8e8006b150 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7f8e8687e700 1 --2- 172.21.15.89:0/2491805557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130660 0x7f8e80130a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7f8e8687e700 1 --2- 172.21.15.89:0/2491805557 >> v2:172.21.15.89:3300/0 conn(0x7f8e80130f70 0x7f8e8006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.884+0000 7f8e8687e700 1 -- 172.21.15.89:0/2491805557 >> 172.21.15.89:0/2491805557 conn(0x7f8e80066f50 msgr2=0x7f8e8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.886+0000 7f8e8687e700 1 -- 172.21.15.89:0/2491805557 shutdown_connections 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.886+0000 7f8e8687e700 1 -- 172.21.15.89:0/2491805557 wait complete. 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.887+0000 7f8e8687e700 1 Processor -- start 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.887+0000 7f8e8687e700 1 -- start start 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e8687e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8006acb0 0x7f8e8006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e8687e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e80130660 0x7f8e8006ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e8687e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130f70 0x7f8e8006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e8687e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80070850 con 0x7f8e8006acb0 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e8687e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8009c700 con 0x7f8e80130f70 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e8687e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8006fb40 con 0x7f8e80130660 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e7f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e80130660 0x7f8e8006ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e84e1b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130f70 0x7f8e8006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e7f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e80130660 0x7f8e8006ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55864/0 (socket says 172.21.15.89:55864) 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e7f7fe700 1 -- 172.21.15.89:0/3124850663 learned_addr learned my addr 172.21.15.89:0/3124850663 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e7ffff700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8006acb0 0x7f8e8006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e84e1b700 1 -- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e80130660 msgr2=0x7f8e8006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e84e1b700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e80130660 0x7f8e8006ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e84e1b700 1 -- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8006acb0 msgr2=0x7f8e8006e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e84e1b700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8006acb0 0x7f8e8006e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e84e1b700 1 -- 172.21.15.89:0/3124850663 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e74009550 con 0x7f8e80130f70 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e7ffff700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8006acb0 0x7f8e8006e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.888+0000 7f8e7f7fe700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e80130660 0x7f8e8006ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.889+0000 7f8e84e1b700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130f70 0x7f8e8006f2c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8e78027000 tx=0x7f8e78004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.889+0000 7f8e7cff9700 1 -- 172.21.15.89:0/3124850663 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78004450 con 0x7f8e80130f70 2023-09-30T22:22:47.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.889+0000 7f8e7cff9700 1 -- 172.21.15.89:0/3124850663 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e780045f0 con 0x7f8e80130f70 2023-09-30T22:22:47.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.889+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e78009a70 con 0x7f8e80130f70 2023-09-30T22:22:47.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.889+0000 7f8e7cff9700 1 -- 172.21.15.89:0/3124850663 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78023440 con 0x7f8e80130f70 2023-09-30T22:22:47.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.890+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e80071460 con 0x7f8e80130f70 2023-09-30T22:22:47.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.891+0000 7fa485ffb700 1 -- 172.21.15.89:0/3637273558 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa48000c5e0 con 0x7fa49009e000 2023-09-30T22:22:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.891+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e800715f0 con 0x7f8e80130f70 2023-09-30T22:22:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.892+0000 7f8e7cff9700 1 -- 172.21.15.89:0/3124850663 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e780235a0 con 0x7f8e80130f70 2023-09-30T22:22:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.892+0000 7f8e7cff9700 1 -- 172.21.15.89:0/3124850663 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e78040ea0 con 0x7f8e80130f70 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.893+0000 7f3706bf4700 1 Processor -- start 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.893+0000 7f3706bf4700 1 -- start start 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3706bf4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009d6f0 0x7f370009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3706bf4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f370009e000 0x7f3700130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3706bf4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370009f130 con 0x7f370009e410 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3706bf4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370009f520 con 0x7f370009e000 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3706bf4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370009e930 con 0x7f370009d6f0 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3704990700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009d6f0 0x7f370009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3704990700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009d6f0 0x7f370009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51542/0 (socket says 172.21.15.89:51542) 2023-09-30T22:22:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3704990700 1 -- 172.21.15.89:0/3384395939 learned_addr learned my addr 172.21.15.89:0/3384395939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f36fffff700 1 --2- 172.21.15.89:0/3384395939 >> v2:172.21.15.89:3300/0 conn(0x7f370009e000 0x7f3700130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f36fffff700 1 -- 172.21.15.89:0/3384395939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009d6f0 msgr2=0x7f370009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f36fffff700 1 --2- 172.21.15.89:0/3384395939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009d6f0 0x7f370009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f36fffff700 1 -- 172.21.15.89:0/3384395939 >> v1:172.21.15.89:6789/0 conn(0x7f370009e410 legacy=0x7f3700130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f36fffff700 1 -- 172.21.15.89:0/3384395939 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370009b2d0 con 0x7f370009e000 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f3704990700 1 --2- 172.21.15.89:0/3384395939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009d6f0 0x7f370009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.894+0000 7f36fffff700 1 --2- 172.21.15.89:0/3384395939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009e000 0x7f3700130660 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009840 tx=0x7f36ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fa5079d38b6f73 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.895+0000 7f36ff7fe700 1 -- 172.21.15.89:0/3384395939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec004230 con 0x7f370009e000 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.895+0000 7f36ff7fe700 1 -- 172.21.15.89:0/3384395939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec004390 con 0x7f370009e000 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.895+0000 7f36ff7fe700 1 -- 172.21.15.89:0/3384395939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0036a0 con 0x7f370009e000 2023-09-30T22:22:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.895+0000 7f3706bf4700 1 -- 172.21.15.89:0/3384395939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009e000 msgr2=0x7f3700130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.895+0000 7f3706bf4700 1 --2- 172.21.15.89:0/3384395939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009e000 0x7f3700130660 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009840 tx=0x7f36ec009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.896+0000 7f3706bf4700 1 -- 172.21.15.89:0/3384395939 shutdown_connections 2023-09-30T22:22:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.896+0000 7f3706bf4700 1 --2- 172.21.15.89:0/3384395939 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009d6f0 0x7f370009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.896+0000 7f3706bf4700 1 --2- 172.21.15.89:0/3384395939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009e000 0x7f3700130660 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.896+0000 7f3706bf4700 1 -- 172.21.15.89:0/3384395939 >> 172.21.15.89:0/3384395939 conn(0x7f3700066f50 msgr2=0x7f370009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.896+0000 7f3706bf4700 1 -- 172.21.15.89:0/3384395939 shutdown_connections 2023-09-30T22:22:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.896+0000 7f3706bf4700 1 -- 172.21.15.89:0/3384395939 wait complete. 2023-09-30T22:22:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.897+0000 7f3706bf4700 1 Processor -- start 2023-09-30T22:22:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.897+0000 7f8e7cff9700 1 -- 172.21.15.89:0/3124850663 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e7802c400 con 0x7f8e80130f70 2023-09-30T22:22:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.897+0000 7f3706bf4700 1 -- start start 2023-09-30T22:22:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3706bf4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009d6f0 0x7f370011df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3706bf4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f370009e000 0x7f370011e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3706bf4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009e410 0x7f370011e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3706bf4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001217b0 con 0x7f370009d6f0 2023-09-30T22:22:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3706bf4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37001302b0 con 0x7f370009e410 2023-09-30T22:22:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3706bf4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3700120d30 con 0x7f370009e000 2023-09-30T22:22:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3704990700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009d6f0 0x7f370011df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3704990700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009d6f0 0x7f370011df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57230/0 (socket says 172.21.15.89:57230) 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3704990700 1 -- 172.21.15.89:0/3986573956 learned_addr learned my addr 172.21.15.89:0/3986573956 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.898+0000 7f3705191700 1 --2- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009e410 0x7f370011e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.899+0000 7f3704990700 1 -- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f370009e000 msgr2=0x7f370011e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.899+0000 7f3704990700 1 --2- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f370009e000 0x7f370011e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.899+0000 7f3704990700 1 -- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009e410 msgr2=0x7f370011e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.899+0000 7f3704990700 1 --2- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009e410 0x7f370011e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.899+0000 7f3704990700 1 -- 172.21.15.89:0/3986573956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f4009550 con 0x7f370009d6f0 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.899+0000 7f3704990700 1 --2- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009d6f0 0x7f370011df20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f36f400ec10 tx=0x7f36f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.899+0000 7f36fd7fa700 1 -- 172.21.15.89:0/3986573956 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400c850 con 0x7f370009d6f0 2023-09-30T22:22:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.900+0000 7f36fd7fa700 1 -- 172.21.15.89:0/3986573956 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f400c9b0 con 0x7f370009d6f0 2023-09-30T22:22:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.900+0000 7f36fd7fa700 1 -- 172.21.15.89:0/3986573956 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400cd20 con 0x7f370009d6f0 2023-09-30T22:22:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.900+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec0094a0 con 0x7f370009d6f0 2023-09-30T22:22:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.900+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37000a1af0 con 0x7f370009d6f0 2023-09-30T22:22:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.901+0000 7f36fd7fa700 1 -- 172.21.15.89:0/3986573956 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f36f4013470 con 0x7f370009d6f0 2023-09-30T22:22:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.901+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370009c760 con 0x7f370009d6f0 2023-09-30T22:22:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.901+0000 7f36fd7fa700 1 -- 172.21.15.89:0/3986573956 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f36f4045060 con 0x7f370009d6f0 2023-09-30T22:22:47.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:47.910+0000 7f36fd7fa700 1 -- 172.21.15.89:0/3986573956 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f36f40102b0 con 0x7f370009d6f0 2023-09-30T22:22:48.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.088+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8e80074ea0 con 0x7f8e80130f70 2023-09-30T22:22:48.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.088+0000 7f8e7cff9700 1 -- 172.21.15.89:0/3124850663 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e78019020 con 0x7f8e80130f70 2023-09-30T22:22:48.089 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130f70 msgr2=0x7f8e8006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130f70 0x7f8e8006f2c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8e78027000 tx=0x7f8e78004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 shutdown_connections 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e80130f70 0x7f8e8006f2c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8006acb0 0x7f8e8006e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 --2- 172.21.15.89:0/3124850663 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e80130660 0x7f8e8006ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 >> 172.21.15.89:0/3124850663 conn(0x7f8e80066f50 msgr2=0x7f8e8012b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 shutdown_connections 2023-09-30T22:22:48.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.090+0000 7f8e8687e700 1 -- 172.21.15.89:0/3124850663 wait complete. 2023-09-30T22:22:48.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.092+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa490074ea0 con 0x7fa49009e000 2023-09-30T22:22:48.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.095+0000 7fa485ffb700 1 -- 172.21.15.89:0/3637273558 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa48001bba0 con 0x7fa49009e000 2023-09-30T22:22:48.096 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.097+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 msgr2=0x7fa49011e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.097+0000 7fa49774e700 1 --2- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa49011e830 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa48000ebb0 tx=0x7fa48000ef70 comp rx=0 tx=0).stop 2023-09-30T22:22:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.097+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 shutdown_connections 2023-09-30T22:22:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.097+0000 7fa49774e700 1 --2- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa49009e410 0x7fa49011ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.097+0000 7fa49774e700 1 --2- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa49009d6f0 0x7fa49011e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.097+0000 7fa49774e700 1 --2- 172.21.15.89:0/3637273558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa49009e000 0x7fa49011e830 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.097+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 >> 172.21.15.89:0/3637273558 conn(0x7fa490066f50 msgr2=0x7fa4900a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:48.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.099+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 shutdown_connections 2023-09-30T22:22:48.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.099+0000 7fa49774e700 1 -- 172.21.15.89:0/3637273558 wait complete. 2023-09-30T22:22:48.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:48.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.104+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f758c122490 con 0x7f758c09e000 2023-09-30T22:22:48.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.104+0000 7f757affd700 1 -- 172.21.15.89:0/1277829864 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7580010380 con 0x7f758c09e000 2023-09-30T22:22:48.105 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:48.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 msgr2=0x7f758c121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:48.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 --2- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c121660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7580009840 tx=0x7f7580004750 comp rx=0 tx=0).stop 2023-09-30T22:22:48.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 shutdown_connections 2023-09-30T22:22:48.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 --2- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f758c09e410 0x7f758c121ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 --2- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f758c09d6f0 0x7f758c121120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 --2- 172.21.15.89:0/1277829864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f758c09e000 0x7f758c121660 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 >> 172.21.15.89:0/1277829864 conn(0x7f758c066f50 msgr2=0x7f758c119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:48.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 shutdown_connections 2023-09-30T22:22:48.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.106+0000 7f7590de5700 1 -- 172.21.15.89:0/1277829864 wait complete. 2023-09-30T22:22:48.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:48.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.135+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3700074ea0 con 0x7f370009d6f0 2023-09-30T22:22:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.136+0000 7f36fd7fa700 1 -- 172.21.15.89:0/3986573956 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36f4017020 con 0x7f370009d6f0 2023-09-30T22:22:48.136 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009d6f0 msgr2=0x7f370011df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 --2- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009d6f0 0x7f370011df20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f36f400ec10 tx=0x7f36f400ef20 comp rx=0 tx=0).stop 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 shutdown_connections 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 --2- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f370009e410 0x7f370011e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 --2- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370009d6f0 0x7f370011df20 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 --2- 172.21.15.89:0/3986573956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f370009e000 0x7f370011e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 >> 172.21.15.89:0/3986573956 conn(0x7f3700066f50 msgr2=0x7f370006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 shutdown_connections 2023-09-30T22:22:48.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:48.138+0000 7f3706bf4700 1 -- 172.21.15.89:0/3986573956 wait complete. 2023-09-30T22:22:48.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:49.105 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:49.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:49.121 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:49.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.259+0000 7f40c9939700 1 Processor -- start 2023-09-30T22:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.260+0000 7f40c9939700 1 -- start start 2023-09-30T22:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.260+0000 7f40c9939700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40c4136710 0x7f40c4138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.260+0000 7f40c9939700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4139030 0x7f40c413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.260+0000 7f40c9939700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c406ef30 con 0x7f40c41338a0 2023-09-30T22:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.260+0000 7f40c9939700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c406fef0 con 0x7f40c4136710 2023-09-30T22:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.260+0000 7f40c9939700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40c41334b0 con 0x7f40c4139030 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c27fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40c4136710 0x7f40c4138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c27fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40c4136710 0x7f40c4138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57244/0 (socket says 172.21.15.89:57244) 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4139030 0x7f40c413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c27fc700 1 -- 172.21.15.89:0/777396961 learned_addr learned my addr 172.21.15.89:0/777396961 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c27fc700 1 -- 172.21.15.89:0/777396961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4139030 msgr2=0x7f40c413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c27fc700 1 --2- 172.21.15.89:0/777396961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4139030 0x7f40c413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c27fc700 1 -- 172.21.15.89:0/777396961 >> v1:172.21.15.89:6789/0 conn(0x7f40c41338a0 legacy=0x7f40c4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c27fc700 1 -- 172.21.15.89:0/777396961 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c406fc30 con 0x7f40c4136710 2023-09-30T22:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c37fe700 1 --2- 172.21.15.89:0/777396961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4139030 0x7f40c413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c27fc700 1 --2- 172.21.15.89:0/777396961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c4136710 0x7f40c4138af0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f40b80098f0 tx=0x7f40b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4695b701044987 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c1ffb700 1 -- 172.21.15.89:0/777396961 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8011830 con 0x7f40c4136710 2023-09-30T22:22:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c1ffb700 1 -- 172.21.15.89:0/777396961 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b8011990 con 0x7f40c4136710 2023-09-30T22:22:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.261+0000 7f40c1ffb700 1 -- 172.21.15.89:0/777396961 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8011d20 con 0x7f40c4136710 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.262+0000 7f40c9939700 1 -- 172.21.15.89:0/777396961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c4136710 msgr2=0x7f40c4138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.262+0000 7f40c9939700 1 --2- 172.21.15.89:0/777396961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c4136710 0x7f40c4138af0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f40b80098f0 tx=0x7f40b8009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.262+0000 7f40c9939700 1 -- 172.21.15.89:0/777396961 shutdown_connections 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.262+0000 7f40c9939700 1 --2- 172.21.15.89:0/777396961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4139030 0x7f40c413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.262+0000 7f40c9939700 1 --2- 172.21.15.89:0/777396961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c4136710 0x7f40c4138af0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.262+0000 7f40c9939700 1 -- 172.21.15.89:0/777396961 >> 172.21.15.89:0/777396961 conn(0x7f40c40a2aa0 msgr2=0x7f40c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.262+0000 7f40c9939700 1 -- 172.21.15.89:0/777396961 shutdown_connections 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.262+0000 7f40c9939700 1 -- 172.21.15.89:0/777396961 wait complete. 2023-09-30T22:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.264+0000 7f40c9939700 1 Processor -- start 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.264+0000 7f40c9939700 1 -- start start 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.264+0000 7f40c9939700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c41338a0 0x7f40c411ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c9939700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4136710 0x7f40c411b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c9939700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40c4139030 0x7f40c411b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c9939700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c411cad0 con 0x7f40c41338a0 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c9939700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c411e330 con 0x7f40c4139030 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c9939700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40c4138b00 con 0x7f40c4136710 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c27fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4136710 0x7f40c411b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c41338a0 0x7f40c411ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c27fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4136710 0x7f40c411b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55896/0 (socket says 172.21.15.89:55896) 2023-09-30T22:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c27fc700 1 -- 172.21.15.89:0/3604839144 learned_addr learned my addr 172.21.15.89:0/3604839144 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c37fe700 1 --2- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40c4139030 0x7f40c411b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c2ffd700 1 -- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4136710 msgr2=0x7f40c411b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c2ffd700 1 --2- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4136710 0x7f40c411b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c2ffd700 1 -- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40c4139030 msgr2=0x7f40c411b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c2ffd700 1 --2- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40c4139030 0x7f40c411b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c2ffd700 1 -- 172.21.15.89:0/3604839144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc009a70 con 0x7f40c41338a0 2023-09-30T22:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40c2ffd700 1 --2- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c41338a0 0x7f40c411ae50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000f60 tx=0x7f40b400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.265+0000 7f40abfff700 1 -- 172.21.15.89:0/3604839144 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4011790 con 0x7f40c41338a0 2023-09-30T22:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.266+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b8009550 con 0x7f40c41338a0 2023-09-30T22:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.266+0000 7f40abfff700 1 -- 172.21.15.89:0/3604839144 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b40118f0 con 0x7f40c41338a0 2023-09-30T22:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.266+0000 7f40abfff700 1 -- 172.21.15.89:0/3604839144 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4011c60 con 0x7f40c41338a0 2023-09-30T22:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.266+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c411f060 con 0x7f40c41338a0 2023-09-30T22:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.267+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c4074ea0 con 0x7f40c41338a0 2023-09-30T22:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.267+0000 7f40abfff700 1 -- 172.21.15.89:0/3604839144 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f40b4005080 con 0x7f40c41338a0 2023-09-30T22:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.267+0000 7f40abfff700 1 -- 172.21.15.89:0/3604839144 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f40b403eea0 con 0x7f40c41338a0 2023-09-30T22:22:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.272+0000 7f40abfff700 1 -- 172.21.15.89:0/3604839144 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f40b4019b90 con 0x7f40c41338a0 2023-09-30T22:22:49.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.283+0000 7fcd2119c700 1 Processor -- start 2023-09-30T22:22:49.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.283+0000 7fcd2119c700 1 -- start start 2023-09-30T22:22:49.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.283+0000 7fde2279b700 1 Processor -- start 2023-09-30T22:22:49.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.283+0000 7fde2279b700 1 -- start start 2023-09-30T22:22:49.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd2119c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1338c0 0x7fcd1c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd2119c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd1c1341d0 0x7fcd1c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd2119c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c06f720 con 0x7fcd1c09e390 2023-09-30T22:22:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd2119c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c070c80 con 0x7fcd1c1341d0 2023-09-30T22:22:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd2119c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c06fce0 con 0x7fcd1c1338c0 2023-09-30T22:22:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde2279b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0700e0 0x7fde1c0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde2279b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c0709f0 0x7fde1c12cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde2279b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde1c12d2e0 0x7fde1c12d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde2279b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c09a4a0 con 0x7fde1c12d2e0 2023-09-30T22:22:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde2279b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c071c70 con 0x7fde1c0700e0 2023-09-30T22:22:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde2279b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c066680 con 0x7fde1c0709f0 2023-09-30T22:22:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0700e0 0x7fde1c0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd1ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1338c0 0x7fcd1c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd1ad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1338c0 0x7fcd1c133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51586/0 (socket says 172.21.15.89:51586) 2023-09-30T22:22:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd1ad9d700 1 -- 172.21.15.89:0/1333336402 learned_addr learned my addr 172.21.15.89:0/1333336402 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fcd1b59e700 1 --1- 172.21.15.89:0/1333336402 >> v1:172.21.15.89:6789/0 conn(0x7fcd1c09e390 0x7fcd1c13a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58020/0 (socket says 172.21.15.89:58020) 2023-09-30T22:22:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c0709f0 0x7fde1c12cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0700e0 0x7fde1c0704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:51600/0 (socket says 172.21.15.89:51600) 2023-09-30T22:22:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1bfff700 1 -- 172.21.15.89:0/2149080337 learned_addr learned my addr 172.21.15.89:0/2149080337 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1bfff700 1 -- 172.21.15.89:0/2149080337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c0709f0 msgr2=0x7fde1c12cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1bfff700 1 --2- 172.21.15.89:0/2149080337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c0709f0 0x7fde1c12cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1bfff700 1 -- 172.21.15.89:0/2149080337 >> v2:172.21.15.89:3300/0 conn(0x7fde1c12d2e0 msgr2=0x7fde1c12d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1bfff700 1 --2- 172.21.15.89:0/2149080337 >> v2:172.21.15.89:3300/0 conn(0x7fde1c12d2e0 0x7fde1c12d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.284+0000 7fde1bfff700 1 -- 172.21.15.89:0/2149080337 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c099ce0 con 0x7fde1c0700e0 2023-09-30T22:22:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd1a59c700 1 --2- 172.21.15.89:0/1333336402 >> v2:172.21.15.89:3300/0 conn(0x7fcd1c1341d0 0x7fcd1c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd19d9b700 1 -- 172.21.15.89:0/1333336402 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642835440 0 0) 0x7fcd1c06f720 con 0x7fcd1c09e390 2023-09-30T22:22:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd19d9b700 1 -- 172.21.15.89:0/1333336402 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd040035b0 con 0x7fcd1c09e390 2023-09-30T22:22:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd1ad9d700 1 -- 172.21.15.89:0/1333336402 >> v2:172.21.15.89:3300/0 conn(0x7fcd1c1341d0 msgr2=0x7fcd1c09de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd1ad9d700 1 --2- 172.21.15.89:0/1333336402 >> v2:172.21.15.89:3300/0 conn(0x7fcd1c1341d0 0x7fcd1c09de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd1ad9d700 1 -- 172.21.15.89:0/1333336402 >> v1:172.21.15.89:6789/0 conn(0x7fcd1c09e390 legacy=0x7fcd1c13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd1ad9d700 1 -- 172.21.15.89:0/1333336402 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c1336a0 con 0x7fcd1c1338c0 2023-09-30T22:22:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fde1bfff700 1 --2- 172.21.15.89:0/2149080337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0700e0 0x7fde1c0704b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0098f0 tx=0x7fde0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28e8e93102e3a6e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd1a59c700 1 --2- 172.21.15.89:0/1333336402 >> v2:172.21.15.89:3300/0 conn(0x7fcd1c1341d0 0x7fcd1c09de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fde1affd700 1 -- 172.21.15.89:0/2149080337 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c011830 con 0x7fde1c0700e0 2023-09-30T22:22:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd1ad9d700 1 --2- 172.21.15.89:0/1333336402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1338c0 0x7fcd1c133c90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0098f0 tx=0x7fcd0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b3e5fba2b063423 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fde1affd700 1 -- 172.21.15.89:0/2149080337 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c011990 con 0x7fde1c0700e0 2023-09-30T22:22:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fde1affd700 1 -- 172.21.15.89:0/2149080337 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c011d20 con 0x7fde1c0700e0 2023-09-30T22:22:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd19d9b700 1 -- 172.21.15.89:0/1333336402 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c011830 con 0x7fcd1c1338c0 2023-09-30T22:22:49.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd19d9b700 1 -- 172.21.15.89:0/1333336402 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c011990 con 0x7fcd1c1338c0 2023-09-30T22:22:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.285+0000 7fcd19d9b700 1 -- 172.21.15.89:0/1333336402 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c011d00 con 0x7fcd1c1338c0 2023-09-30T22:22:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 -- 172.21.15.89:0/2149080337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0700e0 msgr2=0x7fde1c0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 --2- 172.21.15.89:0/2149080337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0700e0 0x7fde1c0704b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0098f0 tx=0x7fde0c009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 -- 172.21.15.89:0/2149080337 shutdown_connections 2023-09-30T22:22:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 --2- 172.21.15.89:0/2149080337 >> v2:172.21.15.89:3300/0 conn(0x7fde1c12d2e0 0x7fde1c12d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 --2- 172.21.15.89:0/2149080337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0700e0 0x7fde1c0704b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 --2- 172.21.15.89:0/2149080337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c0709f0 0x7fde1c12cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 -- 172.21.15.89:0/2149080337 >> 172.21.15.89:0/2149080337 conn(0x7fde1c0a27f0 msgr2=0x7fde1c0a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fcd2119c700 1 -- 172.21.15.89:0/1333336402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1338c0 msgr2=0x7fcd1c133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fcd2119c700 1 --2- 172.21.15.89:0/1333336402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1338c0 0x7fcd1c133c90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0098f0 tx=0x7fcd0c009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 -- 172.21.15.89:0/2149080337 shutdown_connections 2023-09-30T22:22:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fde2279b700 1 -- 172.21.15.89:0/2149080337 wait complete. 2023-09-30T22:22:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fcd2119c700 1 -- 172.21.15.89:0/1333336402 shutdown_connections 2023-09-30T22:22:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fcd2119c700 1 --2- 172.21.15.89:0/1333336402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1338c0 0x7fcd1c133c90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fcd2119c700 1 --2- 172.21.15.89:0/1333336402 >> v2:172.21.15.89:3300/0 conn(0x7fcd1c1341d0 0x7fcd1c09de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.286+0000 7fcd2119c700 1 -- 172.21.15.89:0/1333336402 >> 172.21.15.89:0/1333336402 conn(0x7fcd1c0a2aa0 msgr2=0x7fcd1c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.287+0000 7fcd2119c700 1 -- 172.21.15.89:0/1333336402 shutdown_connections 2023-09-30T22:22:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.287+0000 7fcd2119c700 1 -- 172.21.15.89:0/1333336402 wait complete. 2023-09-30T22:22:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.287+0000 7fde2279b700 1 Processor -- start 2023-09-30T22:22:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.287+0000 7fde2279b700 1 -- start start 2023-09-30T22:22:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.287+0000 7fde2279b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1c0700e0 0x7fde1c13c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde2279b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0709f0 0x7fde1c12b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde2279b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c12d2e0 0x7fde1c12be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde2279b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c129730 con 0x7fde1c0700e0 2023-09-30T22:22:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde2279b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c06e570 con 0x7fde1c0709f0 2023-09-30T22:22:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde2279b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c129dc0 con 0x7fde1c12d2e0 2023-09-30T22:22:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c12d2e0 0x7fde1c12be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c12d2e0 0x7fde1c12be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55908/0 (socket says 172.21.15.89:55908) 2023-09-30T22:22:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 -- 172.21.15.89:0/4140914676 learned_addr learned my addr 172.21.15.89:0/4140914676 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde1bfff700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1c0700e0 0x7fde1c13c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde1b7fe700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0709f0 0x7fde1c12b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 -- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0709f0 msgr2=0x7fde1c12b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0709f0 0x7fde1c12b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 -- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1c0700e0 msgr2=0x7fde1c13c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1c0700e0 0x7fde1c13c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 -- 172.21.15.89:0/4140914676 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c009550 con 0x7fde1c12d2e0 2023-09-30T22:22:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fcd2119c700 1 Processor -- start 2023-09-30T22:22:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde20d38700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c12d2e0 0x7fde1c12be70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fde1400b970 tx=0x7fde1400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde18ff9700 1 -- 172.21.15.89:0/4140914676 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1400c930 con 0x7fde1c12d2e0 2023-09-30T22:22:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde18ff9700 1 -- 172.21.15.89:0/4140914676 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1400cab0 con 0x7fde1c12d2e0 2023-09-30T22:22:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fde18ff9700 1 -- 172.21.15.89:0/4140914676 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1400ce60 con 0x7fde1c12d2e0 2023-09-30T22:22:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.288+0000 7fcd2119c700 1 -- start start 2023-09-30T22:22:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd2119c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c09e390 0x7fcd1c1cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd2119c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c1338c0 0x7fcd1c1cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd2119c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1341d0 0x7fcd1c1c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd2119c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c09c120 con 0x7fcd1c09e390 2023-09-30T22:22:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd2119c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0a10b0 con 0x7fcd1c1341d0 2023-09-30T22:22:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd2119c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c06f960 con 0x7fcd1c1338c0 2023-09-30T22:22:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd1a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c1338c0 0x7fcd1c1cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd1a59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c1338c0 0x7fcd1c1cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55922/0 (socket says 172.21.15.89:55922) 2023-09-30T22:22:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd1a59c700 1 -- 172.21.15.89:0/201744095 learned_addr learned my addr 172.21.15.89:0/201744095 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c070f60 con 0x7fde1c12d2e0 2023-09-30T22:22:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c12a7b0 con 0x7fde1c12d2e0 2023-09-30T22:22:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.289+0000 7fcd1b59e700 1 --2- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1341d0 0x7fcd1c1c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fcd1a59c700 1 -- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1341d0 msgr2=0x7fcd1c1c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fcd1a59c700 1 --2- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1341d0 0x7fcd1c1c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fcd1a59c700 1 -- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c09e390 msgr2=0x7fcd1c1cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fcd1a59c700 1 --2- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c09e390 0x7fcd1c1cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c04f2a0 con 0x7fde1c12d2e0 2023-09-30T22:22:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fcd1a59c700 1 -- 172.21.15.89:0/201744095 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c009550 con 0x7fcd1c1338c0 2023-09-30T22:22:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fcd1a59c700 1 --2- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c1338c0 0x7fcd1c1cc920 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000c740 tx=0x7fcd1000cb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fde18ff9700 1 -- 172.21.15.89:0/4140914676 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde1401d3b0 con 0x7fde1c12d2e0 2023-09-30T22:22:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.290+0000 7fde18ff9700 1 -- 172.21.15.89:0/4140914676 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fde14006e40 con 0x7fde1c12d2e0 2023-09-30T22:22:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.294+0000 7fcd037fe700 1 -- 172.21.15.89:0/201744095 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10004220 con 0x7fcd1c1338c0 2023-09-30T22:22:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.294+0000 7fcd037fe700 1 -- 172.21.15.89:0/201744095 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10004380 con 0x7fcd1c1338c0 2023-09-30T22:22:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.295+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c09b790 con 0x7fcd1c1338c0 2023-09-30T22:22:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.295+0000 7fde18ff9700 1 -- 172.21.15.89:0/4140914676 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde1401de60 con 0x7fde1c12d2e0 2023-09-30T22:22:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.295+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c1cce60 con 0x7fcd1c1338c0 2023-09-30T22:22:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.295+0000 7fcd037fe700 1 -- 172.21.15.89:0/201744095 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100036a0 con 0x7fcd1c1338c0 2023-09-30T22:22:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.295+0000 7fcd037fe700 1 -- 172.21.15.89:0/201744095 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd10003800 con 0x7fcd1c1338c0 2023-09-30T22:22:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.295+0000 7fcd037fe700 1 -- 172.21.15.89:0/201744095 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcd1003f960 con 0x7fcd1c1338c0 2023-09-30T22:22:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.296+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccfc0052e0 con 0x7fcd1c1338c0 2023-09-30T22:22:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.300+0000 7fcd037fe700 1 -- 172.21.15.89:0/201744095 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd10007320 con 0x7fcd1c1338c0 2023-09-30T22:22:49.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.365+0000 7fe58bbce700 1 Processor -- start 2023-09-30T22:22:49.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.365+0000 7fe58bbce700 1 -- start start 2023-09-30T22:22:49.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58bbce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe58409e850 0x7fe58409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58bbce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe584134290 0x7fe58413be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58bbce700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58406bc50 con 0x7fe58409f160 2023-09-30T22:22:49.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58bbce700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58409ae60 con 0x7fe584134290 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58bbce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe58406b990 con 0x7fe58409e850 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58996a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe58409e850 0x7fe58409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58a16b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe584134290 0x7fe58413be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58996a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe58409e850 0x7fe58409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55928/0 (socket says 172.21.15.89:55928) 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58996a700 1 -- 172.21.15.89:0/225495612 learned_addr learned my addr 172.21.15.89:0/225495612 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58a16b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe584134290 0x7fe58413be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57292/0 (socket says 172.21.15.89:57292) 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58996a700 1 -- 172.21.15.89:0/225495612 >> v2:172.21.15.89:3300/0 conn(0x7fe584134290 msgr2=0x7fe58413be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58996a700 1 --2- 172.21.15.89:0/225495612 >> v2:172.21.15.89:3300/0 conn(0x7fe584134290 0x7fe58413be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58996a700 1 -- 172.21.15.89:0/225495612 >> v1:172.21.15.89:6789/0 conn(0x7fe58409f160 legacy=0x7fe584133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58996a700 1 -- 172.21.15.89:0/225495612 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58406cb10 con 0x7fe58409e850 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58a16b700 1 --2- 172.21.15.89:0/225495612 >> v2:172.21.15.89:3300/0 conn(0x7fe584134290 0x7fe58413be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.366+0000 7fe58996a700 1 --2- 172.21.15.89:0/225495612 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe58409e850 0x7fe58409ec20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe5800098f0 tx=0x7fe580009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=673123ff79a1d873 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe588968700 1 -- 172.21.15.89:0/225495612 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580011830 con 0x7fe58409e850 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe588968700 1 -- 172.21.15.89:0/225495612 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe580011990 con 0x7fe58409e850 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe588968700 1 -- 172.21.15.89:0/225495612 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580011d20 con 0x7fe58409e850 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe58bbce700 1 -- 172.21.15.89:0/225495612 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe58409e850 msgr2=0x7fe58409ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe58bbce700 1 --2- 172.21.15.89:0/225495612 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe58409e850 0x7fe58409ec20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe5800098f0 tx=0x7fe580009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe58bbce700 1 -- 172.21.15.89:0/225495612 shutdown_connections 2023-09-30T22:22:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe58bbce700 1 --2- 172.21.15.89:0/225495612 >> v2:172.21.15.89:3300/0 conn(0x7fe584134290 0x7fe58413be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe58bbce700 1 --2- 172.21.15.89:0/225495612 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe58409e850 0x7fe58409ec20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe58bbce700 1 -- 172.21.15.89:0/225495612 >> 172.21.15.89:0/225495612 conn(0x7fe5840a3f70 msgr2=0x7fe5840a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:49.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe58bbce700 1 -- 172.21.15.89:0/225495612 shutdown_connections 2023-09-30T22:22:49.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.367+0000 7fe58bbce700 1 -- 172.21.15.89:0/225495612 wait complete. 2023-09-30T22:22:49.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.369+0000 7fe58bbce700 1 Processor -- start 2023-09-30T22:22:49.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.369+0000 7fe58bbce700 1 -- start start 2023-09-30T22:22:49.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.369+0000 7fe58bbce700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe58409e850 0x7fe584123470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.369+0000 7fe58bbce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe58409f160 0x7fe5841239b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.369+0000 7fe58bbce700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe584134290 0x7fe5841c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.369+0000 7fe58bbce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58406cd30 con 0x7fe58409f160 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.369+0000 7fe58bbce700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5841261c0 con 0x7fe58409e850 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.369+0000 7fe58bbce700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe584123180 con 0x7fe584134290 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe584134290 0x7fe5841c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe589169700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe58409f160 0x7fe5841239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe584134290 0x7fe5841c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55940/0 (socket says 172.21.15.89:55940) 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 -- 172.21.15.89:0/3776088662 learned_addr learned my addr 172.21.15.89:0/3776088662 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58996a700 1 --2- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe58409e850 0x7fe584123470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 -- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe58409e850 msgr2=0x7fe584123470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 --2- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe58409e850 0x7fe584123470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 -- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe58409f160 msgr2=0x7fe5841239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 --2- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe58409f160 0x7fe5841239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 -- 172.21.15.89:0/3776088662 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009a70 con 0x7fe584134290 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58a16b700 1 --2- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe584134290 0x7fe5841c84d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00efc0 tx=0x7fe57c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe57a7fc700 1 -- 172.21.15.89:0/3776088662 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00cb40 con 0x7fe584134290 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe57a7fc700 1 -- 172.21.15.89:0/3776088662 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00f040 con 0x7fe584134290 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe580009550 con 0x7fe584134290 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe57a7fc700 1 -- 172.21.15.89:0/3776088662 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00cea0 con 0x7fe584134290 2023-09-30T22:22:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.370+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe584123ef0 con 0x7fe584134290 2023-09-30T22:22:49.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.371+0000 7fe57a7fc700 1 -- 172.21.15.89:0/3776088662 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe57c003a20 con 0x7fe584134290 2023-09-30T22:22:49.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.372+0000 7fe57a7fc700 1 -- 172.21.15.89:0/3776088662 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe57c040130 con 0x7fe584134290 2023-09-30T22:22:49.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.372+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5840747f0 con 0x7fe584134290 2023-09-30T22:22:49.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.376+0000 7fe57a7fc700 1 -- 172.21.15.89:0/3776088662 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe57c01cba0 con 0x7fe584134290 2023-09-30T22:22:49.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.492+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fde1c06b9a0 con 0x7fde1c12d2e0 2023-09-30T22:22:49.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.492+0000 7fde18ff9700 1 -- 172.21.15.89:0/4140914676 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde14019020 con 0x7fde1c12d2e0 2023-09-30T22:22:49.493 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.494+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c12d2e0 msgr2=0x7fde1c12be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.494+0000 7fde2279b700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c12d2e0 0x7fde1c12be70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fde1400b970 tx=0x7fde1400bc80 comp rx=0 tx=0).stop 2023-09-30T22:22:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.494+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 shutdown_connections 2023-09-30T22:22:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.494+0000 7fde2279b700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde1c12d2e0 0x7fde1c12be70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.494+0000 7fde2279b700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1c0700e0 0x7fde1c13c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.494+0000 7fde2279b700 1 --2- 172.21.15.89:0/4140914676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde1c0709f0 0x7fde1c12b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.494+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 >> 172.21.15.89:0/4140914676 conn(0x7fde1c0a27f0 msgr2=0x7fde1c09d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.495+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 shutdown_connections 2023-09-30T22:22:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.495+0000 7fde2279b700 1 -- 172.21.15.89:0/4140914676 wait complete. 2023-09-30T22:22:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.495+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fccfc005590 con 0x7fcd1c1338c0 2023-09-30T22:22:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.496+0000 7fcd037fe700 1 -- 172.21.15.89:0/201744095 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd10010b40 con 0x7fcd1c1338c0 2023-09-30T22:22:49.497 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c1338c0 msgr2=0x7fcd1c1cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 --2- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c1338c0 0x7fcd1c1cc920 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000c740 tx=0x7fcd1000cb00 comp rx=0 tx=0).stop 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 shutdown_connections 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 --2- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd1c1341d0 0x7fcd1c1c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 --2- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd1c09e390 0x7fcd1c1cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 --2- 172.21.15.89:0/201744095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcd1c1338c0 0x7fcd1c1cc920 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 >> 172.21.15.89:0/201744095 conn(0x7fcd1c0a2aa0 msgr2=0x7fcd1c12ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 shutdown_connections 2023-09-30T22:22:49.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.499+0000 7fcd2119c700 1 -- 172.21.15.89:0/201744095 wait complete. 2023-09-30T22:22:49.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:49.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.523+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f40c406b9e0 con 0x7f40c41338a0 2023-09-30T22:22:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.523+0000 7f40abfff700 1 -- 172.21.15.89:0/3604839144 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40b4019780 con 0x7f40c41338a0 2023-09-30T22:22:49.524 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c41338a0 msgr2=0x7f40c411ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 --2- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c41338a0 0x7f40c411ae50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000f60 tx=0x7f40b400c730 comp rx=0 tx=0).stop 2023-09-30T22:22:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 shutdown_connections 2023-09-30T22:22:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 --2- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40c4139030 0x7f40c411b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 --2- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c41338a0 0x7f40c411ae50 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 --2- 172.21.15.89:0/3604839144 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40c4136710 0x7f40c411b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 >> 172.21.15.89:0/3604839144 conn(0x7f40c40a2aa0 msgr2=0x7f40c4137360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 shutdown_connections 2023-09-30T22:22:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.525+0000 7f40c9939700 1 -- 172.21.15.89:0/3604839144 wait complete. 2023-09-30T22:22:49.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.585+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe584072840 con 0x7fe584134290 2023-09-30T22:22:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.585+0000 7fe57a7fc700 1 -- 172.21.15.89:0/3776088662 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe57c02b090 con 0x7fe584134290 2023-09-30T22:22:49.586 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe584134290 msgr2=0x7fe5841c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 --2- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe584134290 0x7fe5841c84d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00efc0 tx=0x7fe57c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 shutdown_connections 2023-09-30T22:22:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 --2- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe584134290 0x7fe5841c84d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 --2- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe58409e850 0x7fe584123470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 --2- 172.21.15.89:0/3776088662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe58409f160 0x7fe5841239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 >> 172.21.15.89:0/3776088662 conn(0x7fe5840a3f70 msgr2=0x7fe5840a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 shutdown_connections 2023-09-30T22:22:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:49.587+0000 7fe58bbce700 1 -- 172.21.15.89:0/3776088662 wait complete. 2023-09-30T22:22:49.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:50.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:50.514 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:50.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:50.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:50.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.664+0000 7fab3cc6c700 1 Processor -- start 2023-09-30T22:22:50.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.664+0000 7fab3cc6c700 1 -- start start 2023-09-30T22:22:50.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3cc6c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3cc6c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab3809d630 0x7fab380a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3cc6c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab38133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3cc6c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab380a2a70 con 0x7fab3809d630 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3cc6c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab380a01d0 con 0x7fab38133930 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3cc6c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab3809e2b0 con 0x7fab380a0a30 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab35d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab3809d630 0x7fab380a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab380a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54076/0 (socket says 172.21.15.89:54076) 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab35d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fab3809d630 0x7fab380a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40506/0 (socket says 172.21.15.89:40506) 2023-09-30T22:22:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3659c700 1 -- 172.21.15.89:0/2387730953 learned_addr learned my addr 172.21.15.89:0/2387730953 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab36d9d700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab38133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab35d9b700 1 -- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 msgr2=0x7fab380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab35d9b700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab380a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab35d9b700 1 -- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 msgr2=0x7fab38133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab35d9b700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab38133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab35d9b700 1 -- 172.21.15.89:0/2387730953 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3809dc50 con 0x7fab3809d630 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab36d9d700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab38133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.665+0000 7fab3659c700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.666+0000 7fab35d9b700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 0x7fab380a1340 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009840 tx=0x7fab2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea09e98e3964ef3f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.666+0000 7fab3559a700 1 -- 172.21.15.89:0/2387730953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c011830 con 0x7fab3809d630 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.666+0000 7fab3559a700 1 -- 172.21.15.89:0/2387730953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c011990 con 0x7fab3809d630 2023-09-30T22:22:50.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.666+0000 7fab3559a700 1 -- 172.21.15.89:0/2387730953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c011d20 con 0x7fab3809d630 2023-09-30T22:22:50.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 msgr2=0x7fab380a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 0x7fab380a1340 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009840 tx=0x7fab2c009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:50.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2387730953 shutdown_connections 2023-09-30T22:22:50.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab38133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 --2- 172.21.15.89:0/2387730953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 0x7fab380a1340 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2387730953 >> 172.21.15.89:0/2387730953 conn(0x7fab380a3050 msgr2=0x7fab380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2387730953 shutdown_connections 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.667+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2387730953 wait complete. 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.668+0000 7fab3cc6c700 1 Processor -- start 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.668+0000 7fab3cc6c700 1 -- start start 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab3cc6c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 0x7fab3811f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab3cc6c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab3811f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab3cc6c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab381c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab3cc6c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38122400 con 0x7fab3809d630 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab3cc6c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab38133270 con 0x7fab38133930 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab3cc6c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab3811edc0 con 0x7fab380a0a30 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab381c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab3659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 0x7fab3811f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab381c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60356/0 (socket says 172.21.15.89:60356) 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab3659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 0x7fab3811f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40522/0 (socket says 172.21.15.89:40522) 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 -- 172.21.15.89:0/2827661819 learned_addr learned my addr 172.21.15.89:0/2827661819 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab35d9b700 1 --2- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab3811f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 -- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 msgr2=0x7fab3811f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 --2- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab3811f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 -- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 msgr2=0x7fab3811f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 --2- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 0x7fab3811f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 -- 172.21.15.89:0/2827661819 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009a70 con 0x7fab38133930 2023-09-30T22:22:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.669+0000 7fab36d9d700 1 --2- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab381c7fb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fab3000efc0 tx=0x7fab3000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.670+0000 7fab26ffd700 1 -- 172.21.15.89:0/2827661819 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000ca90 con 0x7fab38133930 2023-09-30T22:22:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.670+0000 7fab26ffd700 1 -- 172.21.15.89:0/2827661819 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000f040 con 0x7fab38133930 2023-09-30T22:22:50.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.670+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c0094a0 con 0x7fab38133930 2023-09-30T22:22:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.670+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab38120ce0 con 0x7fab38133930 2023-09-30T22:22:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.670+0000 7fab26ffd700 1 -- 172.21.15.89:0/2827661819 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000ce10 con 0x7fab38133930 2023-09-30T22:22:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.671+0000 7fab26ffd700 1 -- 172.21.15.89:0/2827661819 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab30014480 con 0x7fab38133930 2023-09-30T22:22:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.671+0000 7fab26ffd700 1 -- 172.21.15.89:0/2827661819 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab3003fc50 con 0x7fab38133930 2023-09-30T22:22:50.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.671+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0052e0 con 0x7fab38133930 2023-09-30T22:22:50.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.680+0000 7fab26ffd700 1 -- 172.21.15.89:0/2827661819 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab30012930 con 0x7fab38133930 2023-09-30T22:22:50.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.690+0000 7f06c471e700 1 Processor -- start 2023-09-30T22:22:50.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c471e700 1 -- start start 2023-09-30T22:22:50.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c471e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09e1d0 0x7f06bc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c471e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f06bc139170 0x7f06bc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c471e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc09b120 con 0x7f06bc09d6f0 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c471e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc06b460 con 0x7f06bc139170 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c471e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc09a7e0 con 0x7f06bc09e1d0 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c24ba700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f06bc09d6f0 0x7f06bc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52826/0 (socket says 172.21.15.89:52826) 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c24ba700 1 -- 172.21.15.89:0/2071169166 learned_addr learned my addr 172.21.15.89:0/2071169166 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c2cbb700 1 --2- 172.21.15.89:0/2071169166 >> v2:172.21.15.89:3300/0 conn(0x7f06bc139170 0x7f06bc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.691+0000 7f06c1cb9700 1 --2- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09e1d0 0x7f06bc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c2cbb700 1 -- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09e1d0 msgr2=0x7f06bc138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c2cbb700 1 --2- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09e1d0 0x7f06bc138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c2cbb700 1 -- 172.21.15.89:0/2071169166 >> v1:172.21.15.89:6789/0 conn(0x7f06bc09d6f0 legacy=0x7f06bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c2cbb700 1 -- 172.21.15.89:0/2071169166 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc09a500 con 0x7f06bc139170 2023-09-30T22:22:50.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c14b8700 1 -- 172.21.15.89:0/2071169166 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267272836 0 0) 0x7f06bc09b120 con 0x7f06bc09d6f0 2023-09-30T22:22:50.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c1cb9700 1 --2- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09e1d0 0x7f06bc138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c2cbb700 1 --2- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc139170 0x7f06bc13b550 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009d90 tx=0x7f06b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9609644b2d0c05c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c14b8700 1 -- 172.21.15.89:0/2071169166 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b400e070 con 0x7f06bc139170 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c14b8700 1 -- 172.21.15.89:0/2071169166 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b400bb20 con 0x7f06bc139170 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.692+0000 7f06c14b8700 1 -- 172.21.15.89:0/2071169166 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4012430 con 0x7f06bc139170 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.693+0000 7f06c471e700 1 -- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc139170 msgr2=0x7f06bc13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.693+0000 7f06c471e700 1 --2- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc139170 0x7f06bc13b550 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009d90 tx=0x7f06b400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.693+0000 7f06c471e700 1 -- 172.21.15.89:0/2071169166 shutdown_connections 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.693+0000 7f06c471e700 1 --2- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc139170 0x7f06bc13b550 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.693+0000 7f06c471e700 1 --2- 172.21.15.89:0/2071169166 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09e1d0 0x7f06bc138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.693+0000 7f06c471e700 1 -- 172.21.15.89:0/2071169166 >> 172.21.15.89:0/2071169166 conn(0x7f06bc066f50 msgr2=0x7f06bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.693+0000 7f06c471e700 1 -- 172.21.15.89:0/2071169166 shutdown_connections 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.693+0000 7f06c471e700 1 -- 172.21.15.89:0/2071169166 wait complete. 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c471e700 1 Processor -- start 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c471e700 1 -- start start 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c471e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09d6f0 0x7f06bc1207a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c471e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc09e1d0 0x7f06bc120ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c471e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06bc139170 0x7f06bc121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c471e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc1229e0 con 0x7f06bc09e1d0 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c471e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc1224c0 con 0x7f06bc09d6f0 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c471e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc123fe0 con 0x7f06bc139170 2023-09-30T22:22:50.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.694+0000 7f06c24ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09d6f0 0x7f06bc1207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c24ba700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09d6f0 0x7f06bc1207a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60376/0 (socket says 172.21.15.89:60376) 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c24ba700 1 -- 172.21.15.89:0/992602009 learned_addr learned my addr 172.21.15.89:0/992602009 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c1cb9700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc09e1d0 0x7f06bc120ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c2cbb700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06bc139170 0x7f06bc121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c24ba700 1 -- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06bc139170 msgr2=0x7f06bc121220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c24ba700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06bc139170 0x7f06bc121220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c24ba700 1 -- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc09e1d0 msgr2=0x7f06bc120ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c24ba700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc09e1d0 0x7f06bc120ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c24ba700 1 -- 172.21.15.89:0/992602009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0094a0 con 0x7f06bc09d6f0 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c24ba700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09d6f0 0x7f06bc1207a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f06b800c550 tx=0x7f06b800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:50.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c2cbb700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06bc139170 0x7f06bc121220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:50.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06b2ffd700 1 -- 172.21.15.89:0/992602009 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8004c80 con 0x7f06bc09d6f0 2023-09-30T22:22:50.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06b2ffd700 1 -- 172.21.15.89:0/992602009 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b8004e00 con 0x7f06bc09d6f0 2023-09-30T22:22:50.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b4009a70 con 0x7f06bc09d6f0 2023-09-30T22:22:50.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc122ba0 con 0x7f06bc09d6f0 2023-09-30T22:22:50.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.696+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc074ea0 con 0x7f06bc09d6f0 2023-09-30T22:22:50.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.695+0000 7f06b2ffd700 1 -- 172.21.15.89:0/992602009 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8007310 con 0x7f06bc09d6f0 2023-09-30T22:22:50.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.697+0000 7f06b2ffd700 1 -- 172.21.15.89:0/992602009 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f06b8007470 con 0x7f06bc09d6f0 2023-09-30T22:22:50.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.697+0000 7f06b2ffd700 1 -- 172.21.15.89:0/992602009 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f06b8040e40 con 0x7f06bc09d6f0 2023-09-30T22:22:50.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.698+0000 7f7998c14700 1 Processor -- start 2023-09-30T22:22:50.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.698+0000 7f7998c14700 1 -- start start 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7998c14700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7998c14700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e000 0x7f79941339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7998c14700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f799409e410 0x7f7994133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7998c14700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799406b460 con 0x7f799409e410 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7998c14700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799409b120 con 0x7f799409d6f0 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7998c14700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f799409a7e0 con 0x7f799409e000 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e000 0x7f79941339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e000 0x7f79941339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54106/0 (socket says 172.21.15.89:54106) 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 -- 172.21.15.89:0/3167530093 learned_addr learned my addr 172.21.15.89:0/3167530093 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:50.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7992d9d700 1 --2- 172.21.15.89:0/3167530093 >> v2:172.21.15.89:3300/0 conn(0x7f799409e410 0x7f7994133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 -- 172.21.15.89:0/3167530093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 msgr2=0x7f799409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 --2- 172.21.15.89:0/3167530093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 -- 172.21.15.89:0/3167530093 >> v2:172.21.15.89:3300/0 conn(0x7f799409e410 msgr2=0x7f7994133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 --2- 172.21.15.89:0/3167530093 >> v2:172.21.15.89:3300/0 conn(0x7f799409e410 0x7f7994133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 -- 172.21.15.89:0/3167530093 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799409a500 con 0x7f799409e000 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.699+0000 7f7991d9b700 1 --2- 172.21.15.89:0/3167530093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e000 0x7f79941339f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7988009920 tx=0x7f7988009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6b70422f808ea52 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f799159a700 1 -- 172.21.15.89:0/3167530093 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988011830 con 0x7f799409e000 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f799159a700 1 -- 172.21.15.89:0/3167530093 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7988011990 con 0x7f799409e000 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f799159a700 1 -- 172.21.15.89:0/3167530093 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988011d20 con 0x7f799409e000 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f7998c14700 1 -- 172.21.15.89:0/3167530093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e000 msgr2=0x7f79941339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f7998c14700 1 --2- 172.21.15.89:0/3167530093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e000 0x7f79941339f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7988009920 tx=0x7f7988009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f7998c14700 1 -- 172.21.15.89:0/3167530093 shutdown_connections 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f7998c14700 1 --2- 172.21.15.89:0/3167530093 >> v2:172.21.15.89:3300/0 conn(0x7f799409e410 0x7f7994133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f7998c14700 1 --2- 172.21.15.89:0/3167530093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f7998c14700 1 --2- 172.21.15.89:0/3167530093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e000 0x7f79941339f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.700+0000 7f7998c14700 1 -- 172.21.15.89:0/3167530093 >> 172.21.15.89:0/3167530093 conn(0x7f7994066f50 msgr2=0x7f799409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.701+0000 7f7998c14700 1 -- 172.21.15.89:0/3167530093 shutdown_connections 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.701+0000 7f7998c14700 1 -- 172.21.15.89:0/3167530093 wait complete. 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.702+0000 7f7998c14700 1 Processor -- start 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.702+0000 7f7998c14700 1 -- start start 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7998c14700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409d6f0 0x7f79941a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7998c14700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409e000 0x7f79941a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7998c14700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e410 0x7f79941a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7998c14700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79941a8610 con 0x7f799409e000 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7998c14700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79941a9c50 con 0x7f799409e410 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7998c14700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79940a3180 con 0x7f799409d6f0 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7991d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409e000 0x7f79941a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7991d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409e000 0x7f79941a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40558/0 (socket says 172.21.15.89:40558) 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7991d9b700 1 -- 172.21.15.89:0/4235949975 learned_addr learned my addr 172.21.15.89:0/4235949975 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7992d9d700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e410 0x7f79941a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f06b2ffd700 1 -- 172.21.15.89:0/992602009 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f06b8044060 con 0x7f06bc09d6f0 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f799259c700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409d6f0 0x7f79941a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7992d9d700 1 -- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409d6f0 msgr2=0x7f79941a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7992d9d700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409d6f0 0x7f79941a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7992d9d700 1 -- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409e000 msgr2=0x7f79941a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7992d9d700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409e000 0x7f79941a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7992d9d700 1 -- 172.21.15.89:0/4235949975 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7988009550 con 0x7f799409e410 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7991d9b700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409e000 0x7f79941a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.703+0000 7f7992d9d700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e410 0x7f79941a6bb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f798c00ef90 tx=0x7f798c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.704+0000 7f7982ffd700 1 -- 172.21.15.89:0/4235949975 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c00cc90 con 0x7f799409e410 2023-09-30T22:22:50.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.704+0000 7f7982ffd700 1 -- 172.21.15.89:0/4235949975 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c00f040 con 0x7f799409e410 2023-09-30T22:22:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.704+0000 7f7982ffd700 1 -- 172.21.15.89:0/4235949975 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c0146c0 con 0x7f799409e410 2023-09-30T22:22:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.704+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79941a93e0 con 0x7f799409e410 2023-09-30T22:22:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.704+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79941a8350 con 0x7f799409e410 2023-09-30T22:22:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.705+0000 7f7982ffd700 1 -- 172.21.15.89:0/4235949975 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f798c014820 con 0x7f799409e410 2023-09-30T22:22:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.705+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7994074ea0 con 0x7f799409e410 2023-09-30T22:22:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.705+0000 7f7982ffd700 1 -- 172.21.15.89:0/4235949975 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f798c0401d0 con 0x7f799409e410 2023-09-30T22:22:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.710+0000 7f7982ffd700 1 -- 172.21.15.89:0/4235949975 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f798c014ad0 con 0x7f799409e410 2023-09-30T22:22:50.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.755+0000 7f434e6bf700 1 Processor -- start 2023-09-30T22:22:50.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.755+0000 7f434e6bf700 1 -- start start 2023-09-30T22:22:50.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f434e6bf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f434809d6f0 0x7f434809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f434e6bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e000 0x7f43481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f434e6bf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e410 0x7f4348133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f434e6bf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f434806b460 con 0x7f434809d6f0 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f434e6bf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434809b120 con 0x7f434809e000 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f434e6bf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f434809a7e0 con 0x7f434809e410 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f43477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e000 0x7f43481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f43477fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e000 0x7f43481339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60396/0 (socket says 172.21.15.89:60396) 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f43477fe700 1 -- 172.21.15.89:0/3992899797 learned_addr learned my addr 172.21.15.89:0/3992899797 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f434cc5c700 1 --2- 172.21.15.89:0/3992899797 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e410 0x7f4348133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f43477fe700 1 -- 172.21.15.89:0/3992899797 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e410 msgr2=0x7f4348133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f43477fe700 1 --2- 172.21.15.89:0/3992899797 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e410 0x7f4348133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f43477fe700 1 -- 172.21.15.89:0/3992899797 >> v2:172.21.15.89:3300/0 conn(0x7f434809d6f0 msgr2=0x7f434809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f43477fe700 1 --2- 172.21.15.89:0/3992899797 >> v2:172.21.15.89:3300/0 conn(0x7f434809d6f0 0x7f434809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.756+0000 7f43477fe700 1 -- 172.21.15.89:0/3992899797 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434809a500 con 0x7f434809e000 2023-09-30T22:22:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.757+0000 7f43477fe700 1 --2- 172.21.15.89:0/3992899797 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e000 0x7f43481339f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f433c009920 tx=0x7f433c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8467256d0e25c5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f4346ffd700 1 -- 172.21.15.89:0/3992899797 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011830 con 0x7f434809e000 2023-09-30T22:22:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f4346ffd700 1 -- 172.21.15.89:0/3992899797 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c011990 con 0x7f434809e000 2023-09-30T22:22:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f4346ffd700 1 -- 172.21.15.89:0/3992899797 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011d20 con 0x7f434809e000 2023-09-30T22:22:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 -- 172.21.15.89:0/3992899797 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e000 msgr2=0x7f43481339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 --2- 172.21.15.89:0/3992899797 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e000 0x7f43481339f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f433c009920 tx=0x7f433c009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 -- 172.21.15.89:0/3992899797 shutdown_connections 2023-09-30T22:22:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 --2- 172.21.15.89:0/3992899797 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e410 0x7f4348133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 --2- 172.21.15.89:0/3992899797 >> v2:172.21.15.89:3300/0 conn(0x7f434809d6f0 0x7f434809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 --2- 172.21.15.89:0/3992899797 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e000 0x7f43481339f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 -- 172.21.15.89:0/3992899797 >> 172.21.15.89:0/3992899797 conn(0x7f4348066f50 msgr2=0x7f434809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 -- 172.21.15.89:0/3992899797 shutdown_connections 2023-09-30T22:22:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.759+0000 7f434e6bf700 1 -- 172.21.15.89:0/3992899797 wait complete. 2023-09-30T22:22:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.761+0000 7f434e6bf700 1 Processor -- start 2023-09-30T22:22:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.761+0000 7f434e6bf700 1 -- start start 2023-09-30T22:22:50.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f434e6bf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434809d6f0 0x7f43481a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f434e6bf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e000 0x7f43481a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f434e6bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e410 0x7f43481a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f434e6bf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43481aa320 con 0x7f434809d6f0 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f434e6bf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43481a9960 con 0x7f434809e410 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f434e6bf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43480a2200 con 0x7f434809e000 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f4347fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434809d6f0 0x7f43481a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f4347fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434809d6f0 0x7f43481a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40574/0 (socket says 172.21.15.89:40574) 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f4347fff700 1 -- 172.21.15.89:0/725531231 learned_addr learned my addr 172.21.15.89:0/725531231 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f434cc5c700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e410 0x7f43481a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f43477fe700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e000 0x7f43481a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f4347fff700 1 -- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e000 msgr2=0x7f43481a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f4347fff700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e000 0x7f43481a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f4347fff700 1 -- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e410 msgr2=0x7f43481a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f4347fff700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e410 0x7f43481a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f4347fff700 1 -- 172.21.15.89:0/725531231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c009550 con 0x7f434809d6f0 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.762+0000 7f434cc5c700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e410 0x7f43481a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.763+0000 7f4347fff700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434809d6f0 0x7f43481a5e40 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f433800b530 tx=0x7f433800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.763+0000 7f4344ff9700 1 -- 172.21.15.89:0/725531231 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338011830 con 0x7f434809d6f0 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.763+0000 7f4344ff9700 1 -- 172.21.15.89:0/725531231 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4338011990 con 0x7f434809d6f0 2023-09-30T22:22:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.763+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43481a8ca0 con 0x7f434809d6f0 2023-09-30T22:22:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.763+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434809c760 con 0x7f434809d6f0 2023-09-30T22:22:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.763+0000 7f4344ff9700 1 -- 172.21.15.89:0/725531231 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338011d40 con 0x7f434809d6f0 2023-09-30T22:22:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.764+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43280052e0 con 0x7f434809d6f0 2023-09-30T22:22:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.764+0000 7f4344ff9700 1 -- 172.21.15.89:0/725531231 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f433801a400 con 0x7f434809d6f0 2023-09-30T22:22:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.765+0000 7f4344ff9700 1 -- 172.21.15.89:0/725531231 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f433803fc50 con 0x7f434809d6f0 2023-09-30T22:22:50.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.773+0000 7f4344ff9700 1 -- 172.21.15.89:0/725531231 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f433803fe60 con 0x7f434809d6f0 2023-09-30T22:22:50.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.898+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f06bc06c360 con 0x7f06bc09d6f0 2023-09-30T22:22:50.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.898+0000 7f06b2ffd700 1 -- 172.21.15.89:0/992602009 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f06b8044030 con 0x7f06bc09d6f0 2023-09-30T22:22:50.899 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:50.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.900+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09d6f0 msgr2=0x7f06bc1207a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.900+0000 7f06c471e700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09d6f0 0x7f06bc1207a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f06b800c550 tx=0x7f06b800c860 comp rx=0 tx=0).stop 2023-09-30T22:22:50.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.901+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 shutdown_connections 2023-09-30T22:22:50.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.901+0000 7f06c471e700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f06bc139170 0x7f06bc121220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.901+0000 7f06c471e700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06bc09d6f0 0x7f06bc1207a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.901+0000 7f06c471e700 1 --2- 172.21.15.89:0/992602009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06bc09e1d0 0x7f06bc120ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.901+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 >> 172.21.15.89:0/992602009 conn(0x7f06bc066f50 msgr2=0x7f06bc136a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:50.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.901+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 shutdown_connections 2023-09-30T22:22:50.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.901+0000 7f06c471e700 1 -- 172.21.15.89:0/992602009 wait complete. 2023-09-30T22:22:50.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.906+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f79941a82f0 con 0x7f799409e410 2023-09-30T22:22:50.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.907+0000 7f7982ffd700 1 -- 172.21.15.89:0/4235949975 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f798c012340 con 0x7f799409e410 2023-09-30T22:22:50.908 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:50.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.907+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fab1c005590 con 0x7fab38133930 2023-09-30T22:22:50.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.907+0000 7fab26ffd700 1 -- 172.21.15.89:0/2827661819 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab3002b090 con 0x7fab38133930 2023-09-30T22:22:50.908 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:50.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e410 msgr2=0x7f79941a6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e410 0x7f79941a6bb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f798c00ef90 tx=0x7f798c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 shutdown_connections 2023-09-30T22:22:50.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e410 0x7f79941a6bb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409d6f0 0x7f79941a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 --2- 172.21.15.89:0/4235949975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409e000 0x7f79941a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 >> 172.21.15.89:0/4235949975 conn(0x7f7994066f50 msgr2=0x7f79941bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:50.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 shutdown_connections 2023-09-30T22:22:50.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.909+0000 7f7998c14700 1 -- 172.21.15.89:0/4235949975 wait complete. 2023-09-30T22:22:50.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.911+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 msgr2=0x7fab381c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.911+0000 7fab3cc6c700 1 --2- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab381c7fb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fab3000efc0 tx=0x7fab3000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.911+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 shutdown_connections 2023-09-30T22:22:50.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.911+0000 7fab3cc6c700 1 --2- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fab38133930 0x7fab381c7fb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.911+0000 7fab3cc6c700 1 --2- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3809d630 0x7fab3811f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.911+0000 7fab3cc6c700 1 --2- 172.21.15.89:0/2827661819 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fab380a0a30 0x7fab3811f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.911+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 >> 172.21.15.89:0/2827661819 conn(0x7fab380a3050 msgr2=0x7fab3809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.915+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 shutdown_connections 2023-09-30T22:22:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.915+0000 7fab3cc6c700 1 -- 172.21.15.89:0/2827661819 wait complete. 2023-09-30T22:22:50.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:50.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:50.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:50.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.969+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4328005150 con 0x7f434809d6f0 2023-09-30T22:22:50.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.969+0000 7f4344ff9700 1 -- 172.21.15.89:0/725531231 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4338028090 con 0x7f434809d6f0 2023-09-30T22:22:50.970 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:50.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434809d6f0 msgr2=0x7f43481a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:50.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434809d6f0 0x7f43481a5e40 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f433800b530 tx=0x7f433800b840 comp rx=0 tx=0).stop 2023-09-30T22:22:50.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 shutdown_connections 2023-09-30T22:22:50.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f434809e410 0x7f43481a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f434809d6f0 0x7f43481a5e40 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 --2- 172.21.15.89:0/725531231 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f434809e000 0x7f43481a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 >> 172.21.15.89:0/725531231 conn(0x7f4348066f50 msgr2=0x7f43481baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 shutdown_connections 2023-09-30T22:22:50.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:50.971+0000 7f434e6bf700 1 -- 172.21.15.89:0/725531231 wait complete. 2023-09-30T22:22:50.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:51.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:51.924 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:51.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:51.984 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:52.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.059+0000 7fe604064700 1 Processor -- start 2023-09-30T22:22:52.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe604064700 1 -- start start 2023-09-30T22:22:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe604064700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5fc09d6f0 0x7fe5fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe604064700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e440 0x7fe5fc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe604064700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc09b120 con 0x7fe5fc09e000 2023-09-30T22:22:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe604064700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5fc06b460 con 0x7fe5fc09d6f0 2023-09-30T22:22:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe604064700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc09a7e0 con 0x7fe5fc09e440 2023-09-30T22:22:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe601e00700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5fc09d6f0 0x7fe5fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe601e00700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5fc09d6f0 0x7fe5fc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40580/0 (socket says 172.21.15.89:40580) 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe6015ff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe5fc09e000 0x7fe5fc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52836/0 (socket says 172.21.15.89:52836) 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.060+0000 7fe601e00700 1 -- 172.21.15.89:0/2580422899 learned_addr learned my addr 172.21.15.89:0/2580422899 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe602601700 1 --2- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e440 0x7fe5fc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe601e00700 1 -- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e440 msgr2=0x7fe5fc134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe601e00700 1 --2- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e440 0x7fe5fc134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe601e00700 1 -- 172.21.15.89:0/2580422899 >> v1:172.21.15.89:6789/0 conn(0x7fe5fc09e000 legacy=0x7fe5fc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe601e00700 1 -- 172.21.15.89:0/2580422899 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc09a500 con 0x7fe5fc09d6f0 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe602601700 1 --2- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e440 0x7fe5fc134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe601e00700 1 --2- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09d6f0 0x7fe5fc09dac0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8013de0 tx=0x7fe5f8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e682d8a596cd371 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe600dfe700 1 -- 172.21.15.89:0/2580422899 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80128e0 con 0x7fe5fc09d6f0 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe600dfe700 1 -- 172.21.15.89:0/2580422899 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8012a40 con 0x7fe5fc09d6f0 2023-09-30T22:22:52.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.061+0000 7fe600dfe700 1 -- 172.21.15.89:0/2580422899 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8012dd0 con 0x7fe5fc09d6f0 2023-09-30T22:22:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.062+0000 7fe604064700 1 -- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09d6f0 msgr2=0x7fe5fc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.062+0000 7fe604064700 1 --2- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09d6f0 0x7fe5fc09dac0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8013de0 tx=0x7fe5f8009710 comp rx=0 tx=0).stop 2023-09-30T22:22:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.062+0000 7fe604064700 1 -- 172.21.15.89:0/2580422899 shutdown_connections 2023-09-30T22:22:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.062+0000 7fe604064700 1 --2- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e440 0x7fe5fc134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.062+0000 7fe604064700 1 --2- 172.21.15.89:0/2580422899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09d6f0 0x7fe5fc09dac0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.062+0000 7fe604064700 1 -- 172.21.15.89:0/2580422899 >> 172.21.15.89:0/2580422899 conn(0x7fe5fc066f50 msgr2=0x7fe5fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.062+0000 7fe604064700 1 -- 172.21.15.89:0/2580422899 shutdown_connections 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.062+0000 7fe604064700 1 -- 172.21.15.89:0/2580422899 wait complete. 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe604064700 1 Processor -- start 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe604064700 1 -- start start 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe604064700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5fc09d6f0 0x7fe5fc1c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe604064700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e000 0x7fe5fc1c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe604064700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09e440 0x7fe5fc1c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe604064700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc09c710 con 0x7fe5fc09e440 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe604064700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1a1120 con 0x7fe5fc09e000 2023-09-30T22:22:52.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe604064700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1b3ed0 con 0x7fe5fc09d6f0 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe6015ff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e000 0x7fe5fc1c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe6015ff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e000 0x7fe5fc1c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60432/0 (socket says 172.21.15.89:60432) 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.064+0000 7fe6015ff700 1 -- 172.21.15.89:0/901647160 learned_addr learned my addr 172.21.15.89:0/901647160 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe601e00700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5fc09d6f0 0x7fe5fc1c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe602601700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09e440 0x7fe5fc1c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe6015ff700 1 -- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5fc09d6f0 msgr2=0x7fe5fc1c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe6015ff700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5fc09d6f0 0x7fe5fc1c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe6015ff700 1 -- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09e440 msgr2=0x7fe5fc1c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe6015ff700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09e440 0x7fe5fc1c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe6015ff700 1 -- 172.21.15.89:0/901647160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f8013a10 con 0x7fe5fc09e000 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe601e00700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5fc09d6f0 0x7fe5fc1c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:52.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe602601700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09e440 0x7fe5fc1c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe6015ff700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e000 0x7fe5fc1c4330 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00c520 tx=0x7fe5ec00c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe5f27fc700 1 -- 172.21.15.89:0/901647160 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec005cb0 con 0x7fe5fc09e000 2023-09-30T22:22:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe5f27fc700 1 -- 172.21.15.89:0/901647160 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec005e10 con 0x7fe5fc09e000 2023-09-30T22:22:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc1a2910 con 0x7fe5fc09e000 2023-09-30T22:22:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe5f27fc700 1 -- 172.21.15.89:0/901647160 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0072d0 con 0x7fe5fc09e000 2023-09-30T22:22:52.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.065+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc1a3640 con 0x7fe5fc09e000 2023-09-30T22:22:52.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.067+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc074ea0 con 0x7fe5fc09e000 2023-09-30T22:22:52.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.067+0000 7fe5f27fc700 1 -- 172.21.15.89:0/901647160 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5ec007430 con 0x7fe5fc09e000 2023-09-30T22:22:52.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.068+0000 7fe5f27fc700 1 -- 172.21.15.89:0/901647160 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe5ec03f020 con 0x7fe5fc09e000 2023-09-30T22:22:52.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.072+0000 7fe5f27fc700 1 -- 172.21.15.89:0/901647160 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5ec01b3b0 con 0x7fe5fc09e000 2023-09-30T22:22:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.099+0000 7f0f79a6a700 1 Processor -- start 2023-09-30T22:22:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.099+0000 7f0f79a6a700 1 -- start start 2023-09-30T22:22:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f79a6a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f7412cf60 0x7f0f7406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f79a6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f74130430 0x7f0f74137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f79a6a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7412cd90 con 0x7f0f7412f8b0 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f79a6a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7412d330 con 0x7f0f7412cf60 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f79a6a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f74071780 con 0x7f0f74130430 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f737fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f7412cf60 0x7f0f7406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f737fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f7412cf60 0x7f0f7406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40596/0 (socket says 172.21.15.89:40596) 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f737fe700 1 -- 172.21.15.89:0/3667981360 learned_addr learned my addr 172.21.15.89:0/3667981360 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f72ffd700 1 --1- 172.21.15.89:0/3667981360 >> v1:172.21.15.89:6789/0 conn(0x7f0f7412f8b0 0x7f0f7412fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52842/0 (socket says 172.21.15.89:52842) 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f73fff700 1 --2- 172.21.15.89:0/3667981360 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f74130430 0x7f0f74137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.101+0000 7f0f737fe700 1 -- 172.21.15.89:0/3667981360 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f74130430 msgr2=0x7f0f74137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.100+0000 7f0f727fc700 1 -- 172.21.15.89:0/3667981360 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404117678 0 0) 0x7f0f7412cd90 con 0x7f0f7412f8b0 2023-09-30T22:22:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.101+0000 7f0f737fe700 1 --2- 172.21.15.89:0/3667981360 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f74130430 0x7f0f74137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.101+0000 7f0f737fe700 1 -- 172.21.15.89:0/3667981360 >> v1:172.21.15.89:6789/0 conn(0x7f0f7412f8b0 legacy=0x7f0f7412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.101+0000 7f0f737fe700 1 -- 172.21.15.89:0/3667981360 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f74071a40 con 0x7f0f7412cf60 2023-09-30T22:22:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.101+0000 7f0f737fe700 1 --2- 172.21.15.89:0/3667981360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412cf60 0x7f0f7406f5d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0f640098f0 tx=0x7f0f64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959fd16a085e2ba9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.101+0000 7f0f727fc700 1 -- 172.21.15.89:0/3667981360 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64011830 con 0x7f0f7412cf60 2023-09-30T22:22:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.101+0000 7f0f727fc700 1 -- 172.21.15.89:0/3667981360 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f64011990 con 0x7f0f7412cf60 2023-09-30T22:22:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.101+0000 7f0f727fc700 1 -- 172.21.15.89:0/3667981360 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64011d20 con 0x7f0f7412cf60 2023-09-30T22:22:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.102+0000 7f0f79a6a700 1 -- 172.21.15.89:0/3667981360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412cf60 msgr2=0x7f0f7406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.102+0000 7f0f79a6a700 1 --2- 172.21.15.89:0/3667981360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412cf60 0x7f0f7406f5d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0f640098f0 tx=0x7f0f64009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.102+0000 7f0f79a6a700 1 -- 172.21.15.89:0/3667981360 shutdown_connections 2023-09-30T22:22:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.102+0000 7f0f79a6a700 1 --2- 172.21.15.89:0/3667981360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412cf60 0x7f0f7406f5d0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.102+0000 7f0f79a6a700 1 --2- 172.21.15.89:0/3667981360 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f74130430 0x7f0f74137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.102+0000 7f0f79a6a700 1 -- 172.21.15.89:0/3667981360 >> 172.21.15.89:0/3667981360 conn(0x7f0f740a2aa0 msgr2=0x7f0f740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.102+0000 7f0f79a6a700 1 -- 172.21.15.89:0/3667981360 shutdown_connections 2023-09-30T22:22:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.102+0000 7f0f79a6a700 1 -- 172.21.15.89:0/3667981360 wait complete. 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f79a6a700 1 Processor -- start 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f79a6a700 1 -- start start 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f79a6a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7412cf60 0x7f0f7409bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f79a6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412f8b0 0x7f0f7409c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f79a6a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f74130430 0x7f0f741c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f79a6a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7409e9b0 con 0x7f0f7412f8b0 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f79a6a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f74067c90 con 0x7f0f74130430 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f79a6a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7409b6b0 con 0x7f0f7412cf60 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f737fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7412cf60 0x7f0f7409bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f737fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7412cf60 0x7f0f7409bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54146/0 (socket says 172.21.15.89:54146) 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.103+0000 7f0f737fe700 1 -- 172.21.15.89:0/2106674728 learned_addr learned my addr 172.21.15.89:0/2106674728 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f72ffd700 1 --2- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412f8b0 0x7f0f7409c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f737fe700 1 -- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f74130430 msgr2=0x7f0f741c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f737fe700 1 --2- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f74130430 0x7f0f741c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f737fe700 1 -- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412f8b0 msgr2=0x7f0f7409c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f737fe700 1 --2- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412f8b0 0x7f0f7409c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f737fe700 1 -- 172.21.15.89:0/2106674728 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64009550 con 0x7f0f7412cf60 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f737fe700 1 --2- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7412cf60 0x7f0f7409bb00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009c40 tx=0x7f0f64004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f5bfff700 1 -- 172.21.15.89:0/2106674728 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f640036a0 con 0x7f0f7412cf60 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f5bfff700 1 -- 172.21.15.89:0/2106674728 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f64003800 con 0x7f0f7412cf60 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7409b3e0 con 0x7f0f7412cf60 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f5bfff700 1 -- 172.21.15.89:0/2106674728 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64003b70 con 0x7f0f7412cf60 2023-09-30T22:22:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.104+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7411d500 con 0x7f0f7412cf60 2023-09-30T22:22:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.105+0000 7f0f5bfff700 1 -- 172.21.15.89:0/2106674728 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f64021070 con 0x7f0f7412cf60 2023-09-30T22:22:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.105+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f74129fa0 con 0x7f0f7412cf60 2023-09-30T22:22:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.105+0000 7f0f5bfff700 1 -- 172.21.15.89:0/2106674728 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0f64040b90 con 0x7f0f7412cf60 2023-09-30T22:22:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.111+0000 7f0f5bfff700 1 -- 172.21.15.89:0/2106674728 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f64010380 con 0x7f0f7412cf60 2023-09-30T22:22:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.111+0000 7f2f6edb3700 1 Processor -- start 2023-09-30T22:22:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.111+0000 7f2f6edb3700 1 -- start start 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f6edb3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809d6f0 0x7f2f6809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f6edb3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809e000 0x7f2f68130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f6edb3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6809f130 con 0x7f2f6809e410 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f6edb3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6809f520 con 0x7f2f6809d6f0 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f6edb3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6809e930 con 0x7f2f6809e000 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f67fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809e000 0x7f2f68130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f6cb4f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809d6f0 0x7f2f6809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f6cb4f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809d6f0 0x7f2f6809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60448/0 (socket says 172.21.15.89:60448) 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f6cb4f700 1 -- 172.21.15.89:0/2095011024 learned_addr learned my addr 172.21.15.89:0/2095011024 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.112+0000 7f2f67fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809e000 0x7f2f68130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54160/0 (socket says 172.21.15.89:54160) 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f67fff700 1 -- 172.21.15.89:0/2095011024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809d6f0 msgr2=0x7f2f6809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f67fff700 1 --2- 172.21.15.89:0/2095011024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809d6f0 0x7f2f6809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f67fff700 1 -- 172.21.15.89:0/2095011024 >> v1:172.21.15.89:6789/0 conn(0x7f2f6809e410 legacy=0x7f2f68130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f67fff700 1 -- 172.21.15.89:0/2095011024 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6809b2d0 con 0x7f2f6809e000 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f6cb4f700 1 --2- 172.21.15.89:0/2095011024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809d6f0 0x7f2f6809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f67fff700 1 --2- 172.21.15.89:0/2095011024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809e000 0x7f2f68130660 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098a0 tx=0x7f2f54009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb8cf32eb732dab1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f677fe700 1 -- 172.21.15.89:0/2095011024 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54011830 con 0x7f2f6809e000 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f677fe700 1 -- 172.21.15.89:0/2095011024 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f54011990 con 0x7f2f6809e000 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.113+0000 7f2f677fe700 1 -- 172.21.15.89:0/2095011024 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54011d20 con 0x7f2f6809e000 2023-09-30T22:22:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.114+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2095011024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809e000 msgr2=0x7f2f68130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.114+0000 7f2f6edb3700 1 --2- 172.21.15.89:0/2095011024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809e000 0x7f2f68130660 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098a0 tx=0x7f2f54009bb0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.114+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2095011024 shutdown_connections 2023-09-30T22:22:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.114+0000 7f2f6edb3700 1 --2- 172.21.15.89:0/2095011024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809d6f0 0x7f2f6809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.114+0000 7f2f6edb3700 1 --2- 172.21.15.89:0/2095011024 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809e000 0x7f2f68130660 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.114+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2095011024 >> 172.21.15.89:0/2095011024 conn(0x7f2f68066f50 msgr2=0x7f2f6809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.114+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2095011024 shutdown_connections 2023-09-30T22:22:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.114+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2095011024 wait complete. 2023-09-30T22:22:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.116+0000 7f2f6edb3700 1 Processor -- start 2023-09-30T22:22:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.117+0000 7f2f6edb3700 1 -- start start 2023-09-30T22:22:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.117+0000 7f2f6edb3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809d6f0 0x7f2f6811e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.117+0000 7f2f6edb3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809e000 0x7f2f6811e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.117+0000 7f2f6edb3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6809e410 0x7f2f6811eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.117+0000 7f2f6edb3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f68121bb0 con 0x7f2f6809e410 2023-09-30T22:22:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.117+0000 7f2f6edb3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f680a1d00 con 0x7f2f6809e000 2023-09-30T22:22:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.117+0000 7f2f6edb3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6809c700 con 0x7f2f6809d6f0 2023-09-30T22:22:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809d6f0 0x7f2f6811e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809d6f0 0x7f2f6811e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54162/0 (socket says 172.21.15.89:54162) 2023-09-30T22:22:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 -- 172.21.15.89:0/2160836789 learned_addr learned my addr 172.21.15.89:0/2160836789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f67fff700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809e000 0x7f2f6811e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6d350700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6809e410 0x7f2f6811eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 -- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809e000 msgr2=0x7f2f6811e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809e000 0x7f2f6811e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 -- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6809e410 msgr2=0x7f2f6811eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6809e410 0x7f2f6811eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 -- 172.21.15.89:0/2160836789 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c00d9f0 con 0x7f2f6809d6f0 2023-09-30T22:22:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.118+0000 7f2f6cb4f700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809d6f0 0x7f2f6811e090 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c012fc0 tx=0x7f2f5c0103a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.119+0000 7f2f657fa700 1 -- 172.21.15.89:0/2160836789 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c010b00 con 0x7f2f6809d6f0 2023-09-30T22:22:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.119+0000 7f2f657fa700 1 -- 172.21.15.89:0/2160836789 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c013040 con 0x7f2f6809d6f0 2023-09-30T22:22:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.119+0000 7f2f657fa700 1 -- 172.21.15.89:0/2160836789 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c010e60 con 0x7f2f6809d6f0 2023-09-30T22:22:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.119+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f540094a0 con 0x7f2f6809d6f0 2023-09-30T22:22:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.119+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f68121340 con 0x7f2f6809d6f0 2023-09-30T22:22:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.120+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f68120f80 con 0x7f2f6809d6f0 2023-09-30T22:22:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.120+0000 7f2f657fa700 1 -- 172.21.15.89:0/2160836789 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f5c018480 con 0x7f2f6809d6f0 2023-09-30T22:22:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.120+0000 7f2f657fa700 1 -- 172.21.15.89:0/2160836789 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2f5c043540 con 0x7f2f6809d6f0 2023-09-30T22:22:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.125+0000 7f2f657fa700 1 -- 172.21.15.89:0/2160836789 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f5c016520 con 0x7f2f6809d6f0 2023-09-30T22:22:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.140+0000 7f5271d78700 1 Processor -- start 2023-09-30T22:22:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.140+0000 7f5271d78700 1 -- start start 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f5271d78700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f5271d78700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f5271d78700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526c12cd90 con 0x7f526c12f8b0 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f5271d78700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526c12d330 con 0x7f526c130430 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f5271d78700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f526c071780 con 0x7f526c12cf60 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f526bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f526bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60472/0 (socket says 172.21.15.89:60472) 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f526bfff700 1 -- 172.21.15.89:0/644250193 learned_addr learned my addr 172.21.15.89:0/644250193 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.141+0000 7f526b7fe700 1 --2- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526bfff700 1 -- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 msgr2=0x7f526c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526bfff700 1 --2- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526bfff700 1 -- 172.21.15.89:0/644250193 >> v1:172.21.15.89:6789/0 conn(0x7f526c12f8b0 legacy=0x7f526c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526bfff700 1 -- 172.21.15.89:0/644250193 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c071a40 con 0x7f526c130430 2023-09-30T22:22:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526b7fe700 1 --2- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526bfff700 1 --2- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c137c30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f526400bf20 tx=0x7f526400a4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96dfcb66a1c8a56e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526a7fc700 1 -- 172.21.15.89:0/644250193 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400a2e0 con 0x7f526c130430 2023-09-30T22:22:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526a7fc700 1 -- 172.21.15.89:0/644250193 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526400cb50 con 0x7f526c130430 2023-09-30T22:22:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.142+0000 7f526a7fc700 1 -- 172.21.15.89:0/644250193 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264014480 con 0x7f526c130430 2023-09-30T22:22:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.143+0000 7f5271d78700 1 -- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 msgr2=0x7f526c137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.143+0000 7f5271d78700 1 --2- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c137c30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f526400bf20 tx=0x7f526400a4c0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.143+0000 7f5271d78700 1 -- 172.21.15.89:0/644250193 shutdown_connections 2023-09-30T22:22:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.143+0000 7f5271d78700 1 --2- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c137c30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.143+0000 7f5271d78700 1 --2- 172.21.15.89:0/644250193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.143+0000 7f5271d78700 1 -- 172.21.15.89:0/644250193 >> 172.21.15.89:0/644250193 conn(0x7f526c0a2aa0 msgr2=0x7f526c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.143+0000 7f5271d78700 1 -- 172.21.15.89:0/644250193 shutdown_connections 2023-09-30T22:22:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.143+0000 7f5271d78700 1 -- 172.21.15.89:0/644250193 wait complete. 2023-09-30T22:22:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.144+0000 7f5271d78700 1 Processor -- start 2023-09-30T22:22:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.145+0000 7f5271d78700 1 -- start start 2023-09-30T22:22:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.145+0000 7f5271d78700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c09bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.145+0000 7f5271d78700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c12f8b0 0x7f526c09c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.145+0000 7f5271d78700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c1c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.145+0000 7f5271d78700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c129820 con 0x7f526c12f8b0 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.145+0000 7f5271d78700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526c09b580 con 0x7f526c130430 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.145+0000 7f5271d78700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f526c09e650 con 0x7f526c12cf60 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c09bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c09bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54174/0 (socket says 172.21.15.89:54174) 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 -- 172.21.15.89:0/166436064 learned_addr learned my addr 172.21.15.89:0/166436064 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c1c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526affd700 1 --2- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c12f8b0 0x7f526c09c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 -- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 msgr2=0x7f526c1c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 --2- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c1c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 -- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c12f8b0 msgr2=0x7f526c09c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 --2- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c12f8b0 0x7f526c09c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 -- 172.21.15.89:0/166436064 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c00d940 con 0x7f526c12cf60 2023-09-30T22:22:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.146+0000 7f526b7fe700 1 --2- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c09bb90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f525c012fc0 tx=0x7f525c0103a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.147+0000 7f5253fff700 1 -- 172.21.15.89:0/166436064 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c010bb0 con 0x7f526c12cf60 2023-09-30T22:22:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.147+0000 7f5253fff700 1 -- 172.21.15.89:0/166436064 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c013040 con 0x7f526c12cf60 2023-09-30T22:22:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.147+0000 7f5253fff700 1 -- 172.21.15.89:0/166436064 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c00d2e0 con 0x7f526c12cf60 2023-09-30T22:22:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.147+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526400bb50 con 0x7f526c12cf60 2023-09-30T22:22:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.147+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c12f610 con 0x7f526c12cf60 2023-09-30T22:22:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.148+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526c09eb30 con 0x7f526c12cf60 2023-09-30T22:22:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.148+0000 7f5253fff700 1 -- 172.21.15.89:0/166436064 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f525c018480 con 0x7f526c12cf60 2023-09-30T22:22:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.148+0000 7f5253fff700 1 -- 172.21.15.89:0/166436064 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f525c0442f0 con 0x7f526c12cf60 2023-09-30T22:22:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.157+0000 7f5253fff700 1 -- 172.21.15.89:0/166436064 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f525c020960 con 0x7f526c12cf60 2023-09-30T22:22:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.286+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe5fc09f420 con 0x7fe5fc09e000 2023-09-30T22:22:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.287+0000 7fe5f27fc700 1 -- 172.21.15.89:0/901647160 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5ec017020 con 0x7fe5fc09e000 2023-09-30T22:22:52.288 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e000 msgr2=0x7fe5fc1c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e000 0x7fe5fc1c4330 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00c520 tx=0x7fe5ec00c8e0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 shutdown_connections 2023-09-30T22:22:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5fc09e440 0x7fe5fc1c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5fc09d6f0 0x7fe5fc1c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 --2- 172.21.15.89:0/901647160 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5fc09e000 0x7fe5fc1c4330 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 >> 172.21.15.89:0/901647160 conn(0x7fe5fc066f50 msgr2=0x7fe5fc06f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 shutdown_connections 2023-09-30T22:22:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.289+0000 7fe604064700 1 -- 172.21.15.89:0/901647160 wait complete. 2023-09-30T22:22:52.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.308+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0f74074ea0 con 0x7f0f7412cf60 2023-09-30T22:22:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.309+0000 7f0f5bfff700 1 -- 172.21.15.89:0/2106674728 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f64015030 con 0x7f0f7412cf60 2023-09-30T22:22:52.310 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7412cf60 msgr2=0x7f0f7409bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 --2- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7412cf60 0x7f0f7409bb00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009c40 tx=0x7f0f64004750 comp rx=0 tx=0).stop 2023-09-30T22:22:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 shutdown_connections 2023-09-30T22:22:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 --2- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f74130430 0x7f0f741c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 --2- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7412cf60 0x7f0f7409bb00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 --2- 172.21.15.89:0/2106674728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7412f8b0 0x7f0f7409c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 >> 172.21.15.89:0/2106674728 conn(0x7f0f740a2aa0 msgr2=0x7f0f740a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 shutdown_connections 2023-09-30T22:22:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.311+0000 7f0f79a6a700 1 -- 172.21.15.89:0/2106674728 wait complete. 2023-09-30T22:22:52.324 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:52.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.323+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2f68074ea0 con 0x7f2f6809d6f0 2023-09-30T22:22:52.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.324+0000 7f2f657fa700 1 -- 172.21.15.89:0/2160836789 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f5c016340 con 0x7f2f6809d6f0 2023-09-30T22:22:52.325 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809d6f0 msgr2=0x7f2f6811e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809d6f0 0x7f2f6811e090 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c012fc0 tx=0x7f2f5c0103a0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 shutdown_connections 2023-09-30T22:22:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f6809e410 0x7f2f6811eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6809d6f0 0x7f2f6811e090 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 --2- 172.21.15.89:0/2160836789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f6809e000 0x7f2f6811e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 >> 172.21.15.89:0/2160836789 conn(0x7f2f68066f50 msgr2=0x7f2f680a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 shutdown_connections 2023-09-30T22:22:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.327+0000 7f2f6edb3700 1 -- 172.21.15.89:0/2160836789 wait complete. 2023-09-30T22:22:52.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:52.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.360+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f526c09eb30 con 0x7f526c12cf60 2023-09-30T22:22:52.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.360+0000 7f5253fff700 1 -- 172.21.15.89:0/166436064 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f526c09eb30 con 0x7f526c12cf60 2023-09-30T22:22:52.361 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 msgr2=0x7f526c09bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 --2- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c09bb90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f525c012fc0 tx=0x7f525c0103a0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 shutdown_connections 2023-09-30T22:22:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 --2- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f526c130430 0x7f526c1c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 --2- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f526c12cf60 0x7f526c09bb90 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 --2- 172.21.15.89:0/166436064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c12f8b0 0x7f526c09c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 >> 172.21.15.89:0/166436064 conn(0x7f526c0a2aa0 msgr2=0x7f526c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 shutdown_connections 2023-09-30T22:22:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:52.362+0000 7f5271d78700 1 -- 172.21.15.89:0/166436064 wait complete. 2023-09-30T22:22:52.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:53.303 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:53.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:53.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:53.375 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:53.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.445+0000 7fe8da274700 1 Processor -- start 2023-09-30T22:22:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.445+0000 7fe8da274700 1 -- start start 2023-09-30T22:22:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.445+0000 7fe8da274700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d412cf60 0x7fe8d406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.445+0000 7fe8da274700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412f8b0 0x7fe8d412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.445+0000 7fe8da274700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d412cd90 con 0x7fe8d4130260 2023-09-30T22:22:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.445+0000 7fe8da274700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d412d330 con 0x7fe8d412cf60 2023-09-30T22:22:53.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.445+0000 7fe8da274700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d41306d0 con 0x7fe8d412f8b0 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412f8b0 0x7fe8d412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d412cf60 0x7fe8d406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412f8b0 0x7fe8d412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54176/0 (socket says 172.21.15.89:54176) 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d412cf60 0x7fe8d406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60494/0 (socket says 172.21.15.89:60494) 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d37fe700 1 -- 172.21.15.89:0/3058577429 learned_addr learned my addr 172.21.15.89:0/3058577429 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d3fff700 1 -- 172.21.15.89:0/3058577429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412f8b0 msgr2=0x7fe8d412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d3fff700 1 --2- 172.21.15.89:0/3058577429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412f8b0 0x7fe8d412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d3fff700 1 -- 172.21.15.89:0/3058577429 >> v1:172.21.15.89:6789/0 conn(0x7fe8d4130260 legacy=0x7fe8d4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d3fff700 1 -- 172.21.15.89:0/3058577429 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4071780 con 0x7fe8d412cf60 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d37fe700 1 --2- 172.21.15.89:0/3058577429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412f8b0 0x7fe8d412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:53.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d3fff700 1 --2- 172.21.15.89:0/3058577429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d412cf60 0x7fe8d406f5d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009870 tx=0x7fe8c4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6d0d47d95378bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d2ffd700 1 -- 172.21.15.89:0/3058577429 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4011830 con 0x7fe8d412cf60 2023-09-30T22:22:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d2ffd700 1 -- 172.21.15.89:0/3058577429 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4011990 con 0x7fe8d412cf60 2023-09-30T22:22:53.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.446+0000 7fe8d2ffd700 1 -- 172.21.15.89:0/3058577429 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4011d20 con 0x7fe8d412cf60 2023-09-30T22:22:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.447+0000 7fe8da274700 1 -- 172.21.15.89:0/3058577429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d412cf60 msgr2=0x7fe8d406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.447+0000 7fe8da274700 1 --2- 172.21.15.89:0/3058577429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d412cf60 0x7fe8d406f5d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009870 tx=0x7fe8c4009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.447+0000 7fe8da274700 1 -- 172.21.15.89:0/3058577429 shutdown_connections 2023-09-30T22:22:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.447+0000 7fe8da274700 1 --2- 172.21.15.89:0/3058577429 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d412cf60 0x7fe8d406f5d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.447+0000 7fe8da274700 1 --2- 172.21.15.89:0/3058577429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412f8b0 0x7fe8d412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.447+0000 7fe8da274700 1 -- 172.21.15.89:0/3058577429 >> 172.21.15.89:0/3058577429 conn(0x7fe8d40a2aa0 msgr2=0x7fe8d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.447+0000 7fe8da274700 1 -- 172.21.15.89:0/3058577429 shutdown_connections 2023-09-30T22:22:53.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.447+0000 7fe8da274700 1 -- 172.21.15.89:0/3058577429 wait complete. 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.448+0000 7fe8da274700 1 Processor -- start 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.448+0000 7fe8da274700 1 -- start start 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8da274700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412cf60 0x7fe8d4099d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8da274700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d412f8b0 0x7fe8d409a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8da274700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4130260 0x7fe8d409a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8da274700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4129fa0 con 0x7fe8d412f8b0 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8da274700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d409ba50 con 0x7fe8d4130260 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8da274700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4120070 con 0x7fe8d412cf60 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412cf60 0x7fe8d4099d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412cf60 0x7fe8d4099d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54182/0 (socket says 172.21.15.89:54182) 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d3fff700 1 -- 172.21.15.89:0/1450797170 learned_addr learned my addr 172.21.15.89:0/1450797170 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d8811700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4130260 0x7fe8d409a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d37fe700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d412f8b0 0x7fe8d409a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d3fff700 1 -- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4130260 msgr2=0x7fe8d409a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d3fff700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4130260 0x7fe8d409a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d3fff700 1 -- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d412f8b0 msgr2=0x7fe8d409a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d3fff700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d412f8b0 0x7fe8d409a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.449+0000 7fe8d3fff700 1 -- 172.21.15.89:0/1450797170 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c800a6f0 con 0x7fe8d412cf60 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.450+0000 7fe8d3fff700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412cf60 0x7fe8d4099d80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009bc0 tx=0x7fe8c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.450+0000 7fe8d0ff9700 1 -- 172.21.15.89:0/1450797170 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4004300 con 0x7fe8d412cf60 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.450+0000 7fe8d0ff9700 1 -- 172.21.15.89:0/1450797170 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c401f070 con 0x7fe8d412cf60 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.450+0000 7fe8d0ff9700 1 -- 172.21.15.89:0/1450797170 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40036a0 con 0x7fe8d412cf60 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.450+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c40094a0 con 0x7fe8d412cf60 2023-09-30T22:22:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.450+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d411ec30 con 0x7fe8d412cf60 2023-09-30T22:22:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.451+0000 7fe8d0ff9700 1 -- 172.21.15.89:0/1450797170 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe8c4021030 con 0x7fe8d412cf60 2023-09-30T22:22:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.451+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d4074ea0 con 0x7fe8d412cf60 2023-09-30T22:22:53.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.451+0000 7fe8d0ff9700 1 -- 172.21.15.89:0/1450797170 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe8c4040790 con 0x7fe8d412cf60 2023-09-30T22:22:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.460+0000 7fe8d0ff9700 1 -- 172.21.15.89:0/1450797170 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe8c401a450 con 0x7fe8d412cf60 2023-09-30T22:22:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc96a6e3700 1 Processor -- start 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc96a6e3700 1 -- start start 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc96a6e3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc96409e930 0x7fc96409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc96a6e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409f240 0x7fc9641317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc96a6e3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f650 0x7fc964131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc96a6e3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc964070940 con 0x7fc96409e930 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc96a6e3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96406fd60 con 0x7fc96409f650 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc96a6e3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc96406fa80 con 0x7fc96409f240 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc9637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409f240 0x7fc9641317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc968c80700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f650 0x7fc964131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc968c80700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f650 0x7fc964131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60516/0 (socket says 172.21.15.89:60516) 2023-09-30T22:22:53.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc9637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409f240 0x7fc9641317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54184/0 (socket says 172.21.15.89:54184) 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc968c80700 1 -- 172.21.15.89:0/786484482 learned_addr learned my addr 172.21.15.89:0/786484482 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.483+0000 7fc963fff700 1 --2- 172.21.15.89:0/786484482 >> v2:172.21.15.89:3300/0 conn(0x7fc96409e930 0x7fc96409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc9637fe700 1 -- 172.21.15.89:0/786484482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f650 msgr2=0x7fc964131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc9637fe700 1 --2- 172.21.15.89:0/786484482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f650 0x7fc964131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc9637fe700 1 -- 172.21.15.89:0/786484482 >> v2:172.21.15.89:3300/0 conn(0x7fc96409e930 msgr2=0x7fc96409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc9637fe700 1 --2- 172.21.15.89:0/786484482 >> v2:172.21.15.89:3300/0 conn(0x7fc96409e930 0x7fc96409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc9637fe700 1 -- 172.21.15.89:0/786484482 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96409fb60 con 0x7fc96409f240 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc9637fe700 1 --2- 172.21.15.89:0/786484482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409f240 0x7fc9641317f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc958009840 tx=0x7fc958009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5eafb53b48b1804 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc962ffd700 1 -- 172.21.15.89:0/786484482 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958011830 con 0x7fc96409f240 2023-09-30T22:22:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc962ffd700 1 -- 172.21.15.89:0/786484482 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958011990 con 0x7fc96409f240 2023-09-30T22:22:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.484+0000 7fc962ffd700 1 -- 172.21.15.89:0/786484482 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958011d20 con 0x7fc96409f240 2023-09-30T22:22:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 -- 172.21.15.89:0/786484482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409f240 msgr2=0x7fc9641317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 --2- 172.21.15.89:0/786484482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409f240 0x7fc9641317f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc958009840 tx=0x7fc958009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 -- 172.21.15.89:0/786484482 shutdown_connections 2023-09-30T22:22:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 --2- 172.21.15.89:0/786484482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f650 0x7fc964131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 --2- 172.21.15.89:0/786484482 >> v2:172.21.15.89:3300/0 conn(0x7fc96409e930 0x7fc96409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 --2- 172.21.15.89:0/786484482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409f240 0x7fc9641317f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 -- 172.21.15.89:0/786484482 >> 172.21.15.89:0/786484482 conn(0x7fc9640a4210 msgr2=0x7fc9640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 -- 172.21.15.89:0/786484482 shutdown_connections 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.485+0000 7fc96a6e3700 1 -- 172.21.15.89:0/786484482 wait complete. 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc96a6e3700 1 Processor -- start 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc96a6e3700 1 -- start start 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc96a6e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409e930 0x7fc9641a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc96a6e3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f240 0x7fc9641a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc96a6e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc96409f650 0x7fc9641a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc96a6e3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641a58a0 con 0x7fc96409f650 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc96a6e3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641a52c0 con 0x7fc96409f240 2023-09-30T22:22:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc96a6e3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641a7310 con 0x7fc96409e930 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.486+0000 7fc9637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f240 0x7fc9641a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc9637fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f240 0x7fc9641a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60526/0 (socket says 172.21.15.89:60526) 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc9637fe700 1 -- 172.21.15.89:0/1909911578 learned_addr learned my addr 172.21.15.89:0/1909911578 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc963fff700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409e930 0x7fc9641a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc968c80700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc96409f650 0x7fc9641a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc9637fe700 1 -- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409e930 msgr2=0x7fc9641a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc9637fe700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409e930 0x7fc9641a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc9637fe700 1 -- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc96409f650 msgr2=0x7fc9641a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc9637fe700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc96409f650 0x7fc9641a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc9637fe700 1 -- 172.21.15.89:0/1909911578 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9580094a0 con 0x7fc96409f240 2023-09-30T22:22:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc968c80700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc96409f650 0x7fc9641a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc9637fe700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f240 0x7fc9641a4350 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc9580090b0 tx=0x7fc958004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc960ff9700 1 -- 172.21.15.89:0/1909911578 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958004510 con 0x7fc96409f240 2023-09-30T22:22:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7f627c9e0700 1 Processor -- start 2023-09-30T22:22:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7f627c9e0700 1 -- start start 2023-09-30T22:22:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9640a0190 con 0x7fc96409f240 2023-09-30T22:22:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.487+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9641a60a0 con 0x7fc96409f240 2023-09-30T22:22:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7fc960ff9700 1 -- 172.21.15.89:0/1909911578 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95801f070 con 0x7fc96409f240 2023-09-30T22:22:53.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7fc960ff9700 1 -- 172.21.15.89:0/1909911578 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9580038c0 con 0x7fc96409f240 2023-09-30T22:22:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7f627c9e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f627812cf60 0x7f627806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7f627c9e0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f627812f8b0 0x7f627812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7f627c9e0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627812cd90 con 0x7f6278130260 2023-09-30T22:22:53.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7f627c9e0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627812d330 con 0x7f627812f8b0 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7f627c9e0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62781306d0 con 0x7f627812cf60 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7f6276d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6278130260 0x7f6278136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52890/0 (socket says 172.21.15.89:52890) 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.488+0000 7f6276d9d700 1 -- 172.21.15.89:0/1264956958 learned_addr learned my addr 172.21.15.89:0/1264956958 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193308843 0 0) 0x7f627812cd90 con 0x7f6278130260 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62600035b0 con 0x7f6278130260 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627659c700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f627812cf60 0x7f627806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3602678887 0 0) 0x7f62600035b0 con 0x7f6278130260 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f627812cf60 msgr2=0x7f627806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f627812cf60 0x7f627806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f627812f8b0 msgr2=0x7f627812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f627812f8b0 0x7f627812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278071780 con 0x7f6278130260 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7f62700019f0 con 0x7f6278130260 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 >> v1:172.21.15.89:6789/0 conn(0x7f6278130260 legacy=0x7f6278136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260003a00 0x7f6260003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260004350 0x7f6260004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 0x7f6260010020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627812d330 con 0x7f6260004350 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62781306d0 con 0x7f6260005010 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6260010560 con 0x7f6260003a00 2023-09-30T22:22:53.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f627659c700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260003a00 0x7f6260003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9440052e0 con 0x7fc96409f240 2023-09-30T22:22:53.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7fc960ff9700 1 -- 172.21.15.89:0/1909911578 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc958003a20 con 0x7fc96409f240 2023-09-30T22:22:53.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f6275d9b700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260004350 0x7f6260004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.489+0000 7f6276d9d700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 0x7f6260010020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7fc960ff9700 1 -- 172.21.15.89:0/1909911578 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc958021070 con 0x7fc96409f240 2023-09-30T22:22:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627659c700 1 -- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 msgr2=0x7f6260010020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627659c700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 0x7f6260010020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627659c700 1 -- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260004350 msgr2=0x7f6260004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627659c700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260004350 0x7f6260004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627659c700 1 -- 172.21.15.89:0/1264956958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600035b0 con 0x7f6260003a00 2023-09-30T22:22:53.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627659c700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260003a00 0x7f6260003e10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f626c009a00 tx=0x7f6260005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac608a60baba1f66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c004020 con 0x7f6260003a00 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c004180 con 0x7f6260003a00 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.490+0000 7f627559a700 1 -- 172.21.15.89:0/1264956958 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c0044f0 con 0x7f6260003a00 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 -- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260003a00 msgr2=0x7f6260003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260003a00 0x7f6260003e10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f626c009a00 tx=0x7f6260005880 comp rx=0 tx=0).stop 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 -- 172.21.15.89:0/1264956958 shutdown_connections 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 0x7f6260010020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260004350 0x7f6260004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 --2- 172.21.15.89:0/1264956958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260003a00 0x7f6260003e10 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 -- 172.21.15.89:0/1264956958 >> 172.21.15.89:0/1264956958 conn(0x7f62780a2aa0 msgr2=0x7f62780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 -- 172.21.15.89:0/1264956958 shutdown_connections 2023-09-30T22:22:53.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.491+0000 7f627c9e0700 1 -- 172.21.15.89:0/1264956958 wait complete. 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.492+0000 7f627c9e0700 1 Processor -- start 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.492+0000 7f627c9e0700 1 -- start start 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f627c9e0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260003a00 0x7f627809bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f627c9e0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260004350 0x7f627809c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f627c9e0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 0x7f62781c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f627c9e0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627809b810 con 0x7f6260003a00 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f627c9e0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6278129fa0 con 0x7f6260005010 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f627c9e0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6278129900 con 0x7f6260004350 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f6275d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260004350 0x7f627809c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f6275d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260004350 0x7f627809c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54228/0 (socket says 172.21.15.89:54228) 2023-09-30T22:22:53.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f6275d9b700 1 -- 172.21.15.89:0/1171716793 learned_addr learned my addr 172.21.15.89:0/1171716793 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f627659c700 1 --2- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260003a00 0x7f627809bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f6275d9b700 1 -- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 msgr2=0x7f62781c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f6275d9b700 1 --2- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 0x7f62781c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f6275d9b700 1 -- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260003a00 msgr2=0x7f627809bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f6275d9b700 1 --2- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260003a00 0x7f627809bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.493+0000 7f6275d9b700 1 -- 172.21.15.89:0/1171716793 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627809e720 con 0x7f6260004350 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.494+0000 7f6275d9b700 1 --2- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260004350 0x7f627809c2a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6268009d10 tx=0x7f62680173b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.494+0000 7f6266ffd700 1 -- 172.21.15.89:0/1171716793 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268013aa0 con 0x7f6260004350 2023-09-30T22:22:53.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.494+0000 7f6266ffd700 1 -- 172.21.15.89:0/1171716793 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268019040 con 0x7f6260004350 2023-09-30T22:22:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.494+0000 7f6266ffd700 1 -- 172.21.15.89:0/1171716793 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626801f400 con 0x7f6260004350 2023-09-30T22:22:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.495+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627809ecc0 con 0x7f6260004350 2023-09-30T22:22:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.495+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6278119220 con 0x7f6260004350 2023-09-30T22:22:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.495+0000 7fc960ff9700 1 -- 172.21.15.89:0/1909911578 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc958006bd0 con 0x7fc96409f240 2023-09-30T22:22:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.496+0000 7f6266ffd700 1 -- 172.21.15.89:0/1171716793 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f62680283b0 con 0x7f6260004350 2023-09-30T22:22:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.496+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6278074ea0 con 0x7f6260004350 2023-09-30T22:22:53.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.496+0000 7f6266ffd700 1 -- 172.21.15.89:0/1171716793 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6268047680 con 0x7f6260004350 2023-09-30T22:22:53.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.509+0000 7f6266ffd700 1 -- 172.21.15.89:0/1171716793 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f626801e3a0 con 0x7f6260004350 2023-09-30T22:22:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.544+0000 7f28af472700 1 Processor -- start 2023-09-30T22:22:53.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.544+0000 7f28af472700 1 -- start start 2023-09-30T22:22:53.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.544+0000 7f28af472700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28a809e1d0 0x7f28a8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.544+0000 7f28af472700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a8135e50 0x7f28a8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.544+0000 7f28af472700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a809f130 con 0x7f28a809d6f0 2023-09-30T22:22:53.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.544+0000 7f28af472700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a809f520 con 0x7f28a809e1d0 2023-09-30T22:22:53.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.544+0000 7f28af472700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a809e930 con 0x7f28a8135e50 2023-09-30T22:22:53.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28ada0f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a8135e50 0x7f28a8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28ada0f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a8135e50 0x7f28a8138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60572/0 (socket says 172.21.15.89:60572) 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28ada0f700 1 -- 172.21.15.89:0/1567409979 learned_addr learned my addr 172.21.15.89:0/1567409979 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28aca0d700 1 --2- 172.21.15.89:0/1567409979 >> v2:172.21.15.89:3300/0 conn(0x7f28a809e1d0 0x7f28a8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28ada0f700 1 -- 172.21.15.89:0/1567409979 >> v2:172.21.15.89:3300/0 conn(0x7f28a809e1d0 msgr2=0x7f28a8135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28ada0f700 1 --2- 172.21.15.89:0/1567409979 >> v2:172.21.15.89:3300/0 conn(0x7f28a809e1d0 0x7f28a8135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28ada0f700 1 -- 172.21.15.89:0/1567409979 >> v1:172.21.15.89:6789/0 conn(0x7f28a809d6f0 legacy=0x7f28a809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28ada0f700 1 -- 172.21.15.89:0/1567409979 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a809b2d0 con 0x7f28a8135e50 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f28ada0f700 1 --2- 172.21.15.89:0/1567409979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a8135e50 0x7f28a8138230 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d150 tx=0x7f28a000d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82400c13ecb3c948 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f289ffff700 1 -- 172.21.15.89:0/1567409979 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0010070 con 0x7f28a8135e50 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f289ffff700 1 -- 172.21.15.89:0/1567409979 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a000dd10 con 0x7f28a8135e50 2023-09-30T22:22:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.545+0000 7f289ffff700 1 -- 172.21.15.89:0/1567409979 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a000b310 con 0x7f28a8135e50 2023-09-30T22:22:53.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.546+0000 7f28af472700 1 -- 172.21.15.89:0/1567409979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a8135e50 msgr2=0x7f28a8138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.546+0000 7f28af472700 1 --2- 172.21.15.89:0/1567409979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a8135e50 0x7f28a8138230 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d150 tx=0x7f28a000d460 comp rx=0 tx=0).stop 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.546+0000 7f28af472700 1 -- 172.21.15.89:0/1567409979 shutdown_connections 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.546+0000 7f28af472700 1 --2- 172.21.15.89:0/1567409979 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a8135e50 0x7f28a8138230 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.546+0000 7f28af472700 1 --2- 172.21.15.89:0/1567409979 >> v2:172.21.15.89:3300/0 conn(0x7f28a809e1d0 0x7f28a8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.546+0000 7f28af472700 1 -- 172.21.15.89:0/1567409979 >> 172.21.15.89:0/1567409979 conn(0x7f28a8066f50 msgr2=0x7f28a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.546+0000 7f28af472700 1 -- 172.21.15.89:0/1567409979 shutdown_connections 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.546+0000 7f28af472700 1 -- 172.21.15.89:0/1567409979 wait complete. 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28af472700 1 Processor -- start 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28af472700 1 -- start start 2023-09-30T22:22:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28af472700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a809d6f0 0x7f28a81c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28af472700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28a809e1d0 0x7f28a81c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28af472700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28a8135e50 0x7f28a8119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28af472700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a811b1f0 con 0x7f28a8135e50 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28af472700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a811b000 con 0x7f28a809d6f0 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28af472700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80a1e20 con 0x7f28a809e1d0 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.547+0000 7f28ad20e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a809d6f0 0x7f28a81c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28ad20e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a809d6f0 0x7f28a81c4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60588/0 (socket says 172.21.15.89:60588) 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28ad20e700 1 -- 172.21.15.89:0/4090458158 learned_addr learned my addr 172.21.15.89:0/4090458158 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28aca0d700 1 --2- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28a809e1d0 0x7f28a81c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28ad20e700 1 -- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28a809e1d0 msgr2=0x7f28a81c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28ad20e700 1 --2- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28a809e1d0 0x7f28a81c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28ad20e700 1 -- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28a8135e50 msgr2=0x7f28a8119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28ad20e700 1 --2- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28a8135e50 0x7f28a8119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28ad20e700 1 -- 172.21.15.89:0/4090458158 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a000bd40 con 0x7f28a809d6f0 2023-09-30T22:22:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28ad20e700 1 --2- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a809d6f0 0x7f28a81c4500 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f28a400c480 tx=0x7f28a400c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f289dffb700 1 -- 172.21.15.89:0/4090458158 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a40052d0 con 0x7f28a809d6f0 2023-09-30T22:22:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f289dffb700 1 -- 172.21.15.89:0/4090458158 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a4005430 con 0x7f28a809d6f0 2023-09-30T22:22:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a811c970 con 0x7f28a809d6f0 2023-09-30T22:22:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.548+0000 7f289dffb700 1 -- 172.21.15.89:0/4090458158 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a40038c0 con 0x7f28a809d6f0 2023-09-30T22:22:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.549+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a811cc30 con 0x7f28a809d6f0 2023-09-30T22:22:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.549+0000 7f289dffb700 1 -- 172.21.15.89:0/4090458158 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28a4003a20 con 0x7f28a809d6f0 2023-09-30T22:22:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.549+0000 7f289dffb700 1 -- 172.21.15.89:0/4090458158 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f28a4040aa0 con 0x7f28a809d6f0 2023-09-30T22:22:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.550+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a8074ea0 con 0x7f28a809d6f0 2023-09-30T22:22:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.554+0000 7f289dffb700 1 -- 172.21.15.89:0/4090458158 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28a4044060 con 0x7f28a809d6f0 2023-09-30T22:22:53.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.665+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe8d40713a0 con 0x7fe8d412cf60 2023-09-30T22:22:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.665+0000 7fe8d0ff9700 1 -- 172.21.15.89:0/1450797170 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8c401b430 con 0x7fe8d412cf60 2023-09-30T22:22:53.666 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412cf60 msgr2=0x7fe8d4099d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412cf60 0x7fe8d4099d80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009bc0 tx=0x7fe8c4004750 comp rx=0 tx=0).stop 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 shutdown_connections 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8d4130260 0x7fe8d409a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe8d412cf60 0x7fe8d4099d80 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 --2- 172.21.15.89:0/1450797170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d412f8b0 0x7fe8d409a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 >> 172.21.15.89:0/1450797170 conn(0x7fe8d40a2aa0 msgr2=0x7fe8d409e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 shutdown_connections 2023-09-30T22:22:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.667+0000 7fe8da274700 1 -- 172.21.15.89:0/1450797170 wait complete. 2023-09-30T22:22:53.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.705+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6278070e90 con 0x7f6260004350 2023-09-30T22:22:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.706+0000 7f6266ffd700 1 -- 172.21.15.89:0/1171716793 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6268024020 con 0x7f6260004350 2023-09-30T22:22:53.707 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260004350 msgr2=0x7f627809c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 --2- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260004350 0x7f627809c2a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6268009d10 tx=0x7f62680173b0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 shutdown_connections 2023-09-30T22:22:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 --2- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6260005010 0x7f62781c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 --2- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6260003a00 0x7f627809bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 --2- 172.21.15.89:0/1171716793 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6260004350 0x7f627809c2a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 >> 172.21.15.89:0/1171716793 conn(0x7f62780a2aa0 msgr2=0x7f62780a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 shutdown_connections 2023-09-30T22:22:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.708+0000 7f627c9e0700 1 -- 172.21.15.89:0/1171716793 wait complete. 2023-09-30T22:22:53.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.712+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc944005590 con 0x7fc96409f240 2023-09-30T22:22:53.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.712+0000 7fc960ff9700 1 -- 172.21.15.89:0/1909911578 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc958015070 con 0x7fc96409f240 2023-09-30T22:22:53.713 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.716+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f240 msgr2=0x7fc9641a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.716+0000 7fc96a6e3700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f240 0x7fc9641a4350 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc9580090b0 tx=0x7fc958004750 comp rx=0 tx=0).stop 2023-09-30T22:22:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.716+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 shutdown_connections 2023-09-30T22:22:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.716+0000 7fc96a6e3700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc96409f650 0x7fc9641a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.716+0000 7fc96a6e3700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc96409e930 0x7fc9641a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.716+0000 7fc96a6e3700 1 --2- 172.21.15.89:0/1909911578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc96409f240 0x7fc9641a4350 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.716+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 >> 172.21.15.89:0/1909911578 conn(0x7fc9640a4210 msgr2=0x7fc9641b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.717+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 shutdown_connections 2023-09-30T22:22:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.717+0000 7fc96a6e3700 1 -- 172.21.15.89:0/1909911578 wait complete. 2023-09-30T22:22:53.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:53.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.749+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28a806c360 con 0x7f28a809d6f0 2023-09-30T22:22:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.749+0000 7f289dffb700 1 -- 172.21.15.89:0/4090458158 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28a4017070 con 0x7f28a809d6f0 2023-09-30T22:22:53.750 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a809d6f0 msgr2=0x7f28a81c4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 --2- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a809d6f0 0x7f28a81c4500 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f28a400c480 tx=0x7f28a400c790 comp rx=0 tx=0).stop 2023-09-30T22:22:53.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 shutdown_connections 2023-09-30T22:22:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 --2- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28a8135e50 0x7f28a8119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 --2- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f28a809d6f0 0x7f28a81c4500 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 --2- 172.21.15.89:0/4090458158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f28a809e1d0 0x7f28a81c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 >> 172.21.15.89:0/4090458158 conn(0x7f28a8066f50 msgr2=0x7f28a806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 shutdown_connections 2023-09-30T22:22:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:53.751+0000 7f28af472700 1 -- 172.21.15.89:0/4090458158 wait complete. 2023-09-30T22:22:53.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:54.681 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:54.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:54.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:54.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:54.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.819+0000 7f049253b700 1 Processor -- start 2023-09-30T22:22:54.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.819+0000 7f049253b700 1 -- start start 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.819+0000 7f049253b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.819+0000 7f049253b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f048c136fe0 0x7f048c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.819+0000 7f049253b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c070940 con 0x7f048c09e930 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.819+0000 7f049253b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c06fd60 con 0x7f048c136fe0 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.819+0000 7f049253b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f048c06fa80 con 0x7f048c09f410 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f0490ad8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f048c136fe0 0x7f048c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54246/0 (socket says 172.21.15.89:54246) 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f0490ad8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f048c136fe0 0x7f048c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40682/0 (socket says 172.21.15.89:40682) 2023-09-30T22:22:54.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048b7fe700 1 -- 172.21.15.89:0/1339556316 learned_addr learned my addr 172.21.15.89:0/1339556316 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:54.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048b7fe700 1 -- 172.21.15.89:0/1339556316 >> v2:172.21.15.89:3300/0 conn(0x7f048c136fe0 msgr2=0x7f048c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:54.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048b7fe700 1 --2- 172.21.15.89:0/1339556316 >> v2:172.21.15.89:3300/0 conn(0x7f048c136fe0 0x7f048c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048b7fe700 1 -- 172.21.15.89:0/1339556316 >> v1:172.21.15.89:6789/0 conn(0x7f048c09e930 legacy=0x7f048c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048b7fe700 1 -- 172.21.15.89:0/1339556316 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c09fb60 con 0x7f048c09f410 2023-09-30T22:22:54.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048b7fe700 1 --2- 172.21.15.89:0/1339556316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c136aa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f04800098f0 tx=0x7f0480009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ed34848273209 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:54.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048affd700 1 -- 172.21.15.89:0/1339556316 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480010a90 con 0x7f048c09f410 2023-09-30T22:22:54.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048affd700 1 -- 172.21.15.89:0/1339556316 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480010bf0 con 0x7f048c09f410 2023-09-30T22:22:54.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.820+0000 7f048affd700 1 -- 172.21.15.89:0/1339556316 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480018440 con 0x7f048c09f410 2023-09-30T22:22:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.821+0000 7f049253b700 1 -- 172.21.15.89:0/1339556316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 msgr2=0x7f048c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.821+0000 7f049253b700 1 --2- 172.21.15.89:0/1339556316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c136aa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f04800098f0 tx=0x7f0480009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.821+0000 7f049253b700 1 -- 172.21.15.89:0/1339556316 shutdown_connections 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.821+0000 7f049253b700 1 --2- 172.21.15.89:0/1339556316 >> v2:172.21.15.89:3300/0 conn(0x7f048c136fe0 0x7f048c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.821+0000 7f049253b700 1 --2- 172.21.15.89:0/1339556316 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c136aa0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.821+0000 7f049253b700 1 -- 172.21.15.89:0/1339556316 >> 172.21.15.89:0/1339556316 conn(0x7f048c0a4210 msgr2=0x7f048c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.821+0000 7f049253b700 1 -- 172.21.15.89:0/1339556316 shutdown_connections 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.821+0000 7f049253b700 1 -- 172.21.15.89:0/1339556316 wait complete. 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.822+0000 7f049253b700 1 Processor -- start 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.822+0000 7f049253b700 1 -- start start 2023-09-30T22:22:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f049253b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048c09e930 0x7f048c134e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f049253b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c135380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f049253b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048c136fe0 0x7f048c1358c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f049253b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c071d20 con 0x7f048c136fe0 2023-09-30T22:22:54.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f049253b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0a2f40 con 0x7f048c09e930 2023-09-30T22:22:54.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f049253b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f048c11b300 con 0x7f048c09f410 2023-09-30T22:22:54.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f048b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c135380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f048b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c135380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54258/0 (socket says 172.21.15.89:54258) 2023-09-30T22:22:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f048b7fe700 1 -- 172.21.15.89:0/511876567 learned_addr learned my addr 172.21.15.89:0/511876567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f0490ad8700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048c136fe0 0x7f048c1358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f048bfff700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048c09e930 0x7f048c134e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f0490ad8700 1 -- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 msgr2=0x7f048c135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f0490ad8700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c135380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f0490ad8700 1 -- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048c09e930 msgr2=0x7f048c134e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f0490ad8700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048c09e930 0x7f048c134e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f0490ad8700 1 -- 172.21.15.89:0/511876567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480009550 con 0x7f048c136fe0 2023-09-30T22:22:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.823+0000 7f0490ad8700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048c136fe0 0x7f048c1358c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f048400ef90 tx=0x7f048400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.824+0000 7f048b7fe700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c135380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.824+0000 7f0488ff9700 1 -- 172.21.15.89:0/511876567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048400cc90 con 0x7f048c136fe0 2023-09-30T22:22:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.824+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c11b750 con 0x7f048c136fe0 2023-09-30T22:22:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.824+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c11c750 con 0x7f048c136fe0 2023-09-30T22:22:54.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.824+0000 7f0488ff9700 1 -- 172.21.15.89:0/511876567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048400f040 con 0x7f048c136fe0 2023-09-30T22:22:54.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.824+0000 7f0488ff9700 1 -- 172.21.15.89:0/511876567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484009e60 con 0x7f048c136fe0 2023-09-30T22:22:54.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.825+0000 7f0488ff9700 1 -- 172.21.15.89:0/511876567 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0484014480 con 0x7f048c136fe0 2023-09-30T22:22:54.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.825+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c074ea0 con 0x7f048c136fe0 2023-09-30T22:22:54.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.826+0000 7f0488ff9700 1 -- 172.21.15.89:0/511876567 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0484040160 con 0x7f048c136fe0 2023-09-30T22:22:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.834+0000 7f0488ff9700 1 -- 172.21.15.89:0/511876567 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0484007920 con 0x7f048c136fe0 2023-09-30T22:22:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.871+0000 7f1a0dfee700 1 Processor -- start 2023-09-30T22:22:54.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.871+0000 7f1a0dfee700 1 -- start start 2023-09-30T22:22:54.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.872+0000 7f1a0dfee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a081337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.872+0000 7f1a0dfee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.872+0000 7f1a0dfee700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0806b990 con 0x7f1a08134260 2023-09-30T22:22:54.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.872+0000 7f1a0dfee700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080a0590 con 0x7f1a08133420 2023-09-30T22:22:54.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.872+0000 7f1a0dfee700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0809fb80 con 0x7f1a08133e30 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.872+0000 7f1a07fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1a08134260 0x7f1a081304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52918/0 (socket says 172.21.15.89:52918) 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.872+0000 7f1a07fff700 1 -- 172.21.15.89:0/3251407543 learned_addr learned my addr 172.21.15.89:0/3251407543 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.872+0000 7f1a077fe700 1 --2- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a081337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a06ffd700 1 --2- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a067fc700 1 -- 172.21.15.89:0/3251407543 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928033950 0 0) 0x7f1a0806b990 con 0x7f1a08134260 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a077fe700 1 -- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 msgr2=0x7f1a0812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a077fe700 1 --2- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a077fe700 1 -- 172.21.15.89:0/3251407543 >> v1:172.21.15.89:6789/0 conn(0x7f1a08134260 legacy=0x7f1a081304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a077fe700 1 -- 172.21.15.89:0/3251407543 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080a0030 con 0x7f1a08133420 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a077fe700 1 --2- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a081337f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009870 tx=0x7f19f8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e3c69d967a2af1a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a067fc700 1 -- 172.21.15.89:0/3251407543 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8011830 con 0x7f1a08133420 2023-09-30T22:22:54.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a067fc700 1 -- 172.21.15.89:0/3251407543 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f8011990 con 0x7f1a08133420 2023-09-30T22:22:54.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.873+0000 7f1a067fc700 1 -- 172.21.15.89:0/3251407543 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8011d20 con 0x7f1a08133420 2023-09-30T22:22:54.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.874+0000 7f1a0dfee700 1 -- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 msgr2=0x7f1a081337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.874+0000 7f1a0dfee700 1 --2- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a081337f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009870 tx=0x7f19f8009b80 comp rx=0 tx=0).stop 2023-09-30T22:22:54.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.874+0000 7f1a0dfee700 1 -- 172.21.15.89:0/3251407543 shutdown_connections 2023-09-30T22:22:54.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.874+0000 7f1a0dfee700 1 --2- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a081337f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.874+0000 7f1a0dfee700 1 --2- 172.21.15.89:0/3251407543 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.874+0000 7f1a0dfee700 1 -- 172.21.15.89:0/3251407543 >> 172.21.15.89:0/3251407543 conn(0x7f1a080a4210 msgr2=0x7f1a080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:54.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.874+0000 7f1a0dfee700 1 -- 172.21.15.89:0/3251407543 shutdown_connections 2023-09-30T22:22:54.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.874+0000 7f1a0dfee700 1 -- 172.21.15.89:0/3251407543 wait complete. 2023-09-30T22:22:54.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a0dfee700 1 Processor -- start 2023-09-30T22:22:54.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a0dfee700 1 -- start start 2023-09-30T22:22:54.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a0dfee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a0809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a0dfee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a0dfee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a08134260 0x7f1a0809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a0dfee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0811c700 con 0x7f1a08134260 2023-09-30T22:22:54.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a0dfee700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0809d850 con 0x7f1a08133420 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a0dfee700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0811bb20 con 0x7f1a08133e30 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a077fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a0809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a077fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a0809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60622/0 (socket says 172.21.15.89:60622) 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.876+0000 7f1a077fe700 1 -- 172.21.15.89:0/200693221 learned_addr learned my addr 172.21.15.89:0/200693221 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a06ffd700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a07fff700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a08134260 0x7f1a0809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a077fe700 1 -- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 msgr2=0x7f1a0809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a077fe700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a077fe700 1 -- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a08134260 msgr2=0x7f1a0809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a077fe700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a08134260 0x7f1a0809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a077fe700 1 -- 172.21.15.89:0/200693221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80094a0 con 0x7f1a08133420 2023-09-30T22:22:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a06ffd700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a07fff700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a08134260 0x7f1a0809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a077fe700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a0809c480 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f19f80090b0 tx=0x7f19f8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f19effff700 1 -- 172.21.15.89:0/200693221 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8004510 con 0x7f1a08133420 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0809db50 con 0x7f1a08133420 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f19effff700 1 -- 172.21.15.89:0/200693221 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f801f070 con 0x7f1a08133420 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f19effff700 1 -- 172.21.15.89:0/200693221 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f80038c0 con 0x7f1a08133420 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.877+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0812d7f0 con 0x7f1a08133420 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.878+0000 7f19effff700 1 -- 172.21.15.89:0/200693221 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19f8003a20 con 0x7f1a08133420 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.879+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0811c370 con 0x7f1a08133420 2023-09-30T22:22:54.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.879+0000 7f19effff700 1 -- 172.21.15.89:0/200693221 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f19f8021070 con 0x7f1a08133420 2023-09-30T22:22:54.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.888+0000 7f19effff700 1 -- 172.21.15.89:0/200693221 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19f8024930 con 0x7f1a08133420 2023-09-30T22:22:54.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.900+0000 7f118b811700 1 Processor -- start 2023-09-30T22:22:54.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.901+0000 7f118b811700 1 -- start start 2023-09-30T22:22:54.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.901+0000 7f118b811700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1184130540 0x7f1184130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.901+0000 7f118b811700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f118406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.901+0000 7f118b811700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f118406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.901+0000 7f118b811700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118409a390 con 0x7f1184130540 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.901+0000 7f118b811700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840716a0 con 0x7f118406fb20 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.901+0000 7f118b811700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1184071290 con 0x7f1184130e50 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f11895ad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1184130540 0x7f1184130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1188dac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f118406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f118406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f118406ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60626/0 (socket says 172.21.15.89:60626) 2023-09-30T22:22:54.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f11895ad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1184130540 0x7f1184130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40706/0 (socket says 172.21.15.89:40706) 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 -- 172.21.15.89:0/3816198173 learned_addr learned my addr 172.21.15.89:0/3816198173 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1188dac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f118406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54290/0 (socket says 172.21.15.89:54290) 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 -- 172.21.15.89:0/3816198173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 msgr2=0x7f118406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 --2- 172.21.15.89:0/3816198173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f118406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 -- 172.21.15.89:0/3816198173 >> v2:172.21.15.89:3300/0 conn(0x7f1184130540 msgr2=0x7f1184130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 --2- 172.21.15.89:0/3816198173 >> v2:172.21.15.89:3300/0 conn(0x7f1184130540 0x7f1184130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 -- 172.21.15.89:0/3816198173 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184130340 con 0x7f118406fb20 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1188dac700 1 --2- 172.21.15.89:0/3816198173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f118406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:54.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f1189dae700 1 --2- 172.21.15.89:0/3816198173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f118406ffa0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=554126f9fbc89050 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f117bfff700 1 -- 172.21.15.89:0/3816198173 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00e070 con 0x7f118406fb20 2023-09-30T22:22:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f117bfff700 1 -- 172.21.15.89:0/3816198173 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00b910 con 0x7f118406fb20 2023-09-30T22:22:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.902+0000 7f117bfff700 1 -- 172.21.15.89:0/3816198173 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00bcc0 con 0x7f118406fb20 2023-09-30T22:22:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.903+0000 7f118b811700 1 -- 172.21.15.89:0/3816198173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 msgr2=0x7f118406ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.903+0000 7f118b811700 1 --2- 172.21.15.89:0/3816198173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f118406ffa0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c00b330 comp rx=0 tx=0).stop 2023-09-30T22:22:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.903+0000 7f118b811700 1 -- 172.21.15.89:0/3816198173 shutdown_connections 2023-09-30T22:22:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.903+0000 7f118b811700 1 --2- 172.21.15.89:0/3816198173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f118406ffa0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.903+0000 7f118b811700 1 --2- 172.21.15.89:0/3816198173 >> v2:172.21.15.89:3300/0 conn(0x7f1184130540 0x7f1184130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.903+0000 7f118b811700 1 --2- 172.21.15.89:0/3816198173 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f118406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.903+0000 7f118b811700 1 -- 172.21.15.89:0/3816198173 >> 172.21.15.89:0/3816198173 conn(0x7f11840a2aa0 msgr2=0x7f11840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.904+0000 7f118b811700 1 -- 172.21.15.89:0/3816198173 shutdown_connections 2023-09-30T22:22:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.904+0000 7f118b811700 1 -- 172.21.15.89:0/3816198173 wait complete. 2023-09-30T22:22:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.905+0000 7f118b811700 1 Processor -- start 2023-09-30T22:22:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.905+0000 7f118b811700 1 -- start start 2023-09-30T22:22:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f118b811700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f1184123580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f118b811700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1184130540 0x7f1184123ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f118b811700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f1184124000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f118b811700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184126ab0 con 0x7f1184130540 2023-09-30T22:22:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f118b811700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1184127150 con 0x7f118406fb20 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f118b811700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1184127db0 con 0x7f1184130e50 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f1188dac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1184130540 0x7f1184123ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f1188dac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1184130540 0x7f1184123ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40712/0 (socket says 172.21.15.89:40712) 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f1188dac700 1 -- 172.21.15.89:0/3775498752 learned_addr learned my addr 172.21.15.89:0/3775498752 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f11895ad700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f1184123580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f1189dae700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f1184124000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f11895ad700 1 -- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 msgr2=0x7f1184124000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f11895ad700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f1184124000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f11895ad700 1 -- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1184130540 msgr2=0x7f1184123ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f11895ad700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1184130540 0x7f1184123ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f11895ad700 1 -- 172.21.15.89:0/3775498752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740094a0 con 0x7f118406fb20 2023-09-30T22:22:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f1188dac700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1184130540 0x7f1184123ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.906+0000 7f11895ad700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f1184123580 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f118000eb10 tx=0x7f118000ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.907+0000 7f1179ffb700 1 -- 172.21.15.89:0/3775498752 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000dd10 con 0x7f118406fb20 2023-09-30T22:22:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.907+0000 7f1179ffb700 1 -- 172.21.15.89:0/3775498752 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118000de70 con 0x7f118406fb20 2023-09-30T22:22:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.907+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c009b30 con 0x7f118406fb20 2023-09-30T22:22:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.907+0000 7f1179ffb700 1 -- 172.21.15.89:0/3775498752 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180010640 con 0x7f118406fb20 2023-09-30T22:22:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.907+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840718e0 con 0x7f118406fb20 2023-09-30T22:22:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.908+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1184074ea0 con 0x7f118406fb20 2023-09-30T22:22:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.908+0000 7f1179ffb700 1 -- 172.21.15.89:0/3775498752 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11800107a0 con 0x7f118406fb20 2023-09-30T22:22:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.909+0000 7f1179ffb700 1 -- 172.21.15.89:0/3775498752 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1180014030 con 0x7f118406fb20 2023-09-30T22:22:54.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.913+0000 7ff739286700 1 Processor -- start 2023-09-30T22:22:54.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.913+0000 7ff739286700 1 -- start start 2023-09-30T22:22:54.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.913+0000 7ff739286700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff73406a770 0x7ff73409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.913+0000 7ff739286700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73409dc30 0x7ff73409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.913+0000 7ff739286700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73413c1e0 0x7ff73413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.913+0000 7ff739286700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73406ab40 con 0x7ff73406a770 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.913+0000 7ff739286700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73409ebc0 con 0x7ff73413c1e0 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.913+0000 7ff739286700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff73409a810 con 0x7ff73409dc30 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73409dc30 0x7ff73409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73409dc30 0x7ff73409e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54320/0 (socket says 172.21.15.89:54320) 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 -- 172.21.15.89:0/2865374631 learned_addr learned my addr 172.21.15.89:0/2865374631 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7337fe700 1 --2- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73413c1e0 0x7ff73413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff732ffd700 1 --2- 172.21.15.89:0/2865374631 >> v2:172.21.15.89:3300/0 conn(0x7ff73406a770 0x7ff73409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 -- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73413c1e0 msgr2=0x7ff73413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:54.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 --2- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73413c1e0 0x7ff73413e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 -- 172.21.15.89:0/2865374631 >> v2:172.21.15.89:3300/0 conn(0x7ff73406a770 msgr2=0x7ff73409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:54.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 --2- 172.21.15.89:0/2865374631 >> v2:172.21.15.89:3300/0 conn(0x7ff73406a770 0x7ff73409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 -- 172.21.15.89:0/2865374631 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73409e920 con 0x7ff73409dc30 2023-09-30T22:22:54.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7337fe700 1 --2- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73413c1e0 0x7ff73413e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:54.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.914+0000 7ff7327fc700 1 --2- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73409dc30 0x7ff73409e080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff728009920 tx=0x7ff728009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa1e3cdca600995 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:54.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff731ffb700 1 -- 172.21.15.89:0/2865374631 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728011830 con 0x7ff73409dc30 2023-09-30T22:22:54.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff731ffb700 1 -- 172.21.15.89:0/2865374631 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff728011990 con 0x7ff73409dc30 2023-09-30T22:22:54.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff731ffb700 1 -- 172.21.15.89:0/2865374631 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728011d20 con 0x7ff73409dc30 2023-09-30T22:22:54.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff739286700 1 -- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73409dc30 msgr2=0x7ff73409e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff739286700 1 --2- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73409dc30 0x7ff73409e080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff728009920 tx=0x7ff728009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:54.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff739286700 1 -- 172.21.15.89:0/2865374631 shutdown_connections 2023-09-30T22:22:54.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff739286700 1 --2- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73413c1e0 0x7ff73413e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff739286700 1 --2- 172.21.15.89:0/2865374631 >> v2:172.21.15.89:3300/0 conn(0x7ff73406a770 0x7ff73409d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff739286700 1 --2- 172.21.15.89:0/2865374631 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73409dc30 0x7ff73409e080 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.915+0000 7ff739286700 1 -- 172.21.15.89:0/2865374631 >> 172.21.15.89:0/2865374631 conn(0x7ff734066f50 msgr2=0x7ff73409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:54.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.916+0000 7ff739286700 1 -- 172.21.15.89:0/2865374631 shutdown_connections 2023-09-30T22:22:54.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.916+0000 7ff739286700 1 -- 172.21.15.89:0/2865374631 wait complete. 2023-09-30T22:22:54.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.917+0000 7f1179ffb700 1 -- 172.21.15.89:0/3775498752 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11800223d0 con 0x7f118406fb20 2023-09-30T22:22:54.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.917+0000 7ff739286700 1 Processor -- start 2023-09-30T22:22:54.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.917+0000 7ff739286700 1 -- start start 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff739286700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73406a770 0x7ff73413b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff739286700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73409dc30 0x7ff73413a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff739286700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73413c1e0 0x7ff73413aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff739286700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340a1c60 con 0x7ff73406a770 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff739286700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73409f1a0 con 0x7ff73409dc30 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff739286700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff73411f380 con 0x7ff73413c1e0 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73409dc30 0x7ff73413a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73409dc30 0x7ff73413a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60650/0 (socket says 172.21.15.89:60650) 2023-09-30T22:22:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 -- 172.21.15.89:0/2146224112 learned_addr learned my addr 172.21.15.89:0/2146224112 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff732ffd700 1 --2- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73406a770 0x7ff73413b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 -- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73413c1e0 msgr2=0x7ff73413aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 --2- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73413c1e0 0x7ff73413aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 -- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73406a770 msgr2=0x7ff73413b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 --2- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73406a770 0x7ff73413b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 -- 172.21.15.89:0/2146224112 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c009a70 con 0x7ff73409dc30 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.918+0000 7ff7327fc700 1 --2- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73409dc30 0x7ff73413a590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff728000c00 tx=0x7ff728004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.919+0000 7ff71b7fe700 1 -- 172.21.15.89:0/2146224112 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728004590 con 0x7ff73409dc30 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.919+0000 7ff71b7fe700 1 -- 172.21.15.89:0/2146224112 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7280036a0 con 0x7ff73409dc30 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.919+0000 7ff739286700 1 -- 172.21.15.89:0/2146224112 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff728009550 con 0x7ff73409dc30 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.919+0000 7ff71b7fe700 1 -- 172.21.15.89:0/2146224112 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728003a50 con 0x7ff73409dc30 2023-09-30T22:22:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.919+0000 7ff739286700 1 -- 172.21.15.89:0/2146224112 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff734071940 con 0x7ff73409dc30 2023-09-30T22:22:54.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.920+0000 7ff7197fa700 1 -- 172.21.15.89:0/2146224112 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c0052e0 con 0x7ff73409dc30 2023-09-30T22:22:54.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.920+0000 7ff71b7fe700 1 -- 172.21.15.89:0/2146224112 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff728021070 con 0x7ff73409dc30 2023-09-30T22:22:54.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.920+0000 7ff71b7fe700 1 -- 172.21.15.89:0/2146224112 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff728040ea0 con 0x7ff73409dc30 2023-09-30T22:22:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:54.929+0000 7ff71b7fe700 1 -- 172.21.15.89:0/2146224112 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff72800b3e0 con 0x7ff73409dc30 2023-09-30T22:22:55.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.046+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f048c06b250 con 0x7f048c136fe0 2023-09-30T22:22:55.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.046+0000 7f0488ff9700 1 -- 172.21.15.89:0/511876567 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0484018020 con 0x7f048c136fe0 2023-09-30T22:22:55.047 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048c136fe0 msgr2=0x7f048c1358c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048c136fe0 0x7f048c1358c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f048400ef90 tx=0x7f048400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 shutdown_connections 2023-09-30T22:22:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f048c136fe0 0x7f048c1358c0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f048c09e930 0x7f048c134e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 --2- 172.21.15.89:0/511876567 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f048c09f410 0x7f048c135380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 >> 172.21.15.89:0/511876567 conn(0x7f048c0a4210 msgr2=0x7f048c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 shutdown_connections 2023-09-30T22:22:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.048+0000 7f049253b700 1 -- 172.21.15.89:0/511876567 wait complete. 2023-09-30T22:22:55.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:55.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.090+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1a08074ea0 con 0x7f1a08133420 2023-09-30T22:22:55.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.091+0000 7f19effff700 1 -- 172.21.15.89:0/200693221 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19f802b070 con 0x7f1a08133420 2023-09-30T22:22:55.092 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 msgr2=0x7f1a0809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a0809c480 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f19f80090b0 tx=0x7f19f8004750 comp rx=0 tx=0).stop 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 shutdown_connections 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a08134260 0x7f1a0809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1a08133420 0x7f1a0809c480 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 --2- 172.21.15.89:0/200693221 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1a08133e30 0x7f1a0809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 >> 172.21.15.89:0/200693221 conn(0x7f1a080a4210 msgr2=0x7f1a08130350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 shutdown_connections 2023-09-30T22:22:55.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.093+0000 7f1a0dfee700 1 -- 172.21.15.89:0/200693221 wait complete. 2023-09-30T22:22:55.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.112+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f11841278a0 con 0x7f118406fb20 2023-09-30T22:22:55.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.113+0000 7f1179ffb700 1 -- 172.21.15.89:0/3775498752 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1180020ae0 con 0x7f118406fb20 2023-09-30T22:22:55.114 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 msgr2=0x7f1184123580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f1184123580 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f118000eb10 tx=0x7f118000ee20 comp rx=0 tx=0).stop 2023-09-30T22:22:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 shutdown_connections 2023-09-30T22:22:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1184130e50 0x7f1184124000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f118406fb20 0x7f1184123580 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 --2- 172.21.15.89:0/3775498752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1184130540 0x7f1184123ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 >> 172.21.15.89:0/3775498752 conn(0x7f11840a2aa0 msgr2=0x7f118409a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 shutdown_connections 2023-09-30T22:22:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.116+0000 7f118b811700 1 -- 172.21.15.89:0/3775498752 wait complete. 2023-09-30T22:22:55.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:55.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.132+0000 7ff7197fa700 1 -- 172.21.15.89:0/2146224112 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff70c005860 con 0x7ff73409dc30 2023-09-30T22:22:55.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.133+0000 7ff71b7fe700 1 -- 172.21.15.89:0/2146224112 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff728015020 con 0x7ff73409dc30 2023-09-30T22:22:55.134 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:55.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 -- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73409dc30 msgr2=0x7ff73413a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:55.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 --2- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73409dc30 0x7ff73413a590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff728000c00 tx=0x7ff728004750 comp rx=0 tx=0).stop 2023-09-30T22:22:55.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 -- 172.21.15.89:0/2146224112 shutdown_connections 2023-09-30T22:22:55.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 --2- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff73413c1e0 0x7ff73413aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 --2- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff73406a770 0x7ff73413b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 --2- 172.21.15.89:0/2146224112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff73409dc30 0x7ff73413a590 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:55.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 -- 172.21.15.89:0/2146224112 >> 172.21.15.89:0/2146224112 conn(0x7ff734066f50 msgr2=0x7ff73409b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:55.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 -- 172.21.15.89:0/2146224112 shutdown_connections 2023-09-30T22:22:55.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:55.135+0000 7ff7197fa700 1 -- 172.21.15.89:0/2146224112 wait complete. 2023-09-30T22:22:55.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:56.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:56.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:56.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:56.149 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.203+0000 7f54b4305700 1 Processor -- start 2023-09-30T22:22:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.203+0000 7f54b4305700 1 -- start start 2023-09-30T22:22:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b4305700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09d6f0 0x7f54ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b4305700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e410 0x7f54ac134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b4305700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac09b120 con 0x7f54ac09e000 2023-09-30T22:22:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b4305700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac06b460 con 0x7f54ac09e410 2023-09-30T22:22:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b4305700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac09a7e0 con 0x7f54ac09d6f0 2023-09-30T22:22:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b18a0700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f54ac09e000 0x7f54ac1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52932/0 (socket says 172.21.15.89:52932) 2023-09-30T22:22:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b18a0700 1 -- 172.21.15.89:0/4170752509 learned_addr learned my addr 172.21.15.89:0/4170752509 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b20a1700 1 --2- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09d6f0 0x7f54ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.204+0000 7f54b28a2700 1 --2- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e410 0x7f54ac134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b20a1700 1 -- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e410 msgr2=0x7f54ac134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b20a1700 1 --2- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e410 0x7f54ac134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b20a1700 1 -- 172.21.15.89:0/4170752509 >> v1:172.21.15.89:6789/0 conn(0x7f54ac09e000 legacy=0x7f54ac1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b109f700 1 -- 172.21.15.89:0/4170752509 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253543374 0 0) 0x7f54ac09b120 con 0x7f54ac09e000 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b20a1700 1 -- 172.21.15.89:0/4170752509 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac09a500 con 0x7f54ac09d6f0 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b20a1700 1 --2- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09d6f0 0x7f54ac09dac0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f54a80098a0 tx=0x7f54a8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2740b1e973f2a44 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b109f700 1 -- 172.21.15.89:0/4170752509 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8011830 con 0x7f54ac09d6f0 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b109f700 1 -- 172.21.15.89:0/4170752509 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8011990 con 0x7f54ac09d6f0 2023-09-30T22:22:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.205+0000 7f54b109f700 1 -- 172.21.15.89:0/4170752509 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8011d20 con 0x7f54ac09d6f0 2023-09-30T22:22:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.206+0000 7f54b4305700 1 -- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09d6f0 msgr2=0x7f54ac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.206+0000 7f54b4305700 1 --2- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09d6f0 0x7f54ac09dac0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f54a80098a0 tx=0x7f54a8009bb0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.206+0000 7f54b4305700 1 -- 172.21.15.89:0/4170752509 shutdown_connections 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.206+0000 7f54b4305700 1 --2- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e410 0x7f54ac134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.206+0000 7f54b4305700 1 --2- 172.21.15.89:0/4170752509 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09d6f0 0x7f54ac09dac0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.206+0000 7f54b4305700 1 -- 172.21.15.89:0/4170752509 >> 172.21.15.89:0/4170752509 conn(0x7f54ac066f50 msgr2=0x7f54ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.206+0000 7f54b4305700 1 -- 172.21.15.89:0/4170752509 shutdown_connections 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.206+0000 7f54b4305700 1 -- 172.21.15.89:0/4170752509 wait complete. 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.207+0000 7f54b4305700 1 Processor -- start 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.207+0000 7f54b4305700 1 -- start start 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b4305700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54ac09d6f0 0x7f54ac1a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b4305700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e000 0x7f54ac1c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b4305700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09e410 0x7f54ac1c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b4305700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac1a7080 con 0x7f54ac09d6f0 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b4305700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac1a4d20 con 0x7f54ac09e000 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b4305700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac1a7c80 con 0x7f54ac09e410 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09e410 0x7f54ac1c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09e410 0x7f54ac1c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54342/0 (socket says 172.21.15.89:54342) 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 -- 172.21.15.89:0/100995461 learned_addr learned my addr 172.21.15.89:0/100995461 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b18a0700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e000 0x7f54ac1c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b20a1700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54ac09d6f0 0x7f54ac1a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 -- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e000 msgr2=0x7f54ac1c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e000 0x7f54ac1c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 -- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54ac09d6f0 msgr2=0x7f54ac1a5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54ac09d6f0 0x7f54ac1a5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 -- 172.21.15.89:0/100995461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a80094a0 con 0x7f54ac09e410 2023-09-30T22:22:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.208+0000 7f54b28a2700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09e410 0x7f54ac1c84f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f54a400efc0 tx=0x7f54a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.209+0000 7f54a2ffd700 1 -- 172.21.15.89:0/100995461 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4009ea0 con 0x7f54ac09e410 2023-09-30T22:22:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.209+0000 7f54a2ffd700 1 -- 172.21.15.89:0/100995461 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a400f040 con 0x7f54ac09e410 2023-09-30T22:22:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.209+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac1a7f40 con 0x7f54ac09e410 2023-09-30T22:22:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.209+0000 7f54a2ffd700 1 -- 172.21.15.89:0/100995461 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a400ceb0 con 0x7f54ac09e410 2023-09-30T22:22:56.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.209+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac1a8220 con 0x7f54ac09e410 2023-09-30T22:22:56.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.210+0000 7f54a2ffd700 1 -- 172.21.15.89:0/100995461 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f54a4014480 con 0x7f54ac09e410 2023-09-30T22:22:56.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.210+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac074ea0 con 0x7f54ac09e410 2023-09-30T22:22:56.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.210+0000 7f54a2ffd700 1 -- 172.21.15.89:0/100995461 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f54a403fe30 con 0x7f54ac09e410 2023-09-30T22:22:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.214+0000 7f54a2ffd700 1 -- 172.21.15.89:0/100995461 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f54a40118b0 con 0x7f54ac09e410 2023-09-30T22:22:56.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.242+0000 7eff2fb0d700 1 Processor -- start 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.242+0000 7eff2fb0d700 1 -- start start 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2fb0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809e930 0x7eff2809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2fb0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff281317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2fb0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7eff2809f650 0x7eff28131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2fb0d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff28070940 con 0x7eff2809f650 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2fb0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2806fd60 con 0x7eff2809f240 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2fb0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff2806fa80 con 0x7eff2809e930 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d0a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff281317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d8a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809e930 0x7eff2809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2e0aa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7eff2809f650 0x7eff28131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d0a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff281317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60672/0 (socket says 172.21.15.89:60672) 2023-09-30T22:22:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d0a8700 1 -- 172.21.15.89:0/1319446830 learned_addr learned my addr 172.21.15.89:0/1319446830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d0a8700 1 -- 172.21.15.89:0/1319446830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809e930 msgr2=0x7eff2809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d0a8700 1 --2- 172.21.15.89:0/1319446830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809e930 0x7eff2809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d0a8700 1 -- 172.21.15.89:0/1319446830 >> v2:172.21.15.89:3300/0 conn(0x7eff2809f650 msgr2=0x7eff28131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d0a8700 1 --2- 172.21.15.89:0/1319446830 >> v2:172.21.15.89:3300/0 conn(0x7eff2809f650 0x7eff28131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.243+0000 7eff2d0a8700 1 -- 172.21.15.89:0/1319446830 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2809fb60 con 0x7eff2809f240 2023-09-30T22:22:56.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.244+0000 7eff2d0a8700 1 --2- 172.21.15.89:0/1319446830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff281317f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7eff180098f0 tx=0x7eff18009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c18ba87b6998fb92 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:56.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.244+0000 7eff2c8a7700 1 -- 172.21.15.89:0/1319446830 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18011830 con 0x7eff2809f240 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.244+0000 7eff2c8a7700 1 -- 172.21.15.89:0/1319446830 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff18011990 con 0x7eff2809f240 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.244+0000 7eff2c8a7700 1 -- 172.21.15.89:0/1319446830 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18011d20 con 0x7eff2809f240 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.244+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1319446830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 msgr2=0x7eff281317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.244+0000 7eff2fb0d700 1 --2- 172.21.15.89:0/1319446830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff281317f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7eff180098f0 tx=0x7eff18009c00 comp rx=0 tx=0).stop 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.245+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1319446830 shutdown_connections 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.245+0000 7eff2fb0d700 1 --2- 172.21.15.89:0/1319446830 >> v2:172.21.15.89:3300/0 conn(0x7eff2809f650 0x7eff28131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.245+0000 7eff2fb0d700 1 --2- 172.21.15.89:0/1319446830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809e930 0x7eff2809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.245+0000 7eff2fb0d700 1 --2- 172.21.15.89:0/1319446830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff281317f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.245+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1319446830 >> 172.21.15.89:0/1319446830 conn(0x7eff280a4210 msgr2=0x7eff280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.245+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1319446830 shutdown_connections 2023-09-30T22:22:56.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.245+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1319446830 wait complete. 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2fb0d700 1 Processor -- start 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2fb0d700 1 -- start start 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2fb0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff2809e930 0x7eff2812f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2fb0d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff2812f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2fb0d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809f650 0x7eff2812fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2fb0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff28131490 con 0x7eff2809e930 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2fb0d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff28131e20 con 0x7eff2809f240 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2fb0d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff280a34c0 con 0x7eff2809f650 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2d8a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff2809e930 0x7eff2812f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2d8a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff2809e930 0x7eff2812f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40756/0 (socket says 172.21.15.89:40756) 2023-09-30T22:22:56.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2d8a9700 1 -- 172.21.15.89:0/1440756391 learned_addr learned my addr 172.21.15.89:0/1440756391 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.246+0000 7eff2d0a8700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff2812f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2e0aa700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809f650 0x7eff2812fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2d0a8700 1 -- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809f650 msgr2=0x7eff2812fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2d0a8700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809f650 0x7eff2812fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2d0a8700 1 -- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff2809e930 msgr2=0x7eff2812f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2d0a8700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff2809e930 0x7eff2812f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2d0a8700 1 -- 172.21.15.89:0/1440756391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18009550 con 0x7eff2809f240 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2d0a8700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff2812f600 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7eff18016000 tx=0x7eff18004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff1e7fc700 1 -- 172.21.15.89:0/1440756391 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18004240 con 0x7eff2809f240 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff1e7fc700 1 -- 172.21.15.89:0/1440756391 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1801f070 con 0x7eff2809f240 2023-09-30T22:22:56.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff1e7fc700 1 -- 172.21.15.89:0/1440756391 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180045e0 con 0x7eff2809f240 2023-09-30T22:22:56.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff280a0190 con 0x7eff2809f240 2023-09-30T22:22:56.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.247+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff28071d20 con 0x7eff2809f240 2023-09-30T22:22:56.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.248+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff28132a10 con 0x7eff2809f240 2023-09-30T22:22:56.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.248+0000 7eff1e7fc700 1 -- 172.21.15.89:0/1440756391 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff18021030 con 0x7eff2809f240 2023-09-30T22:22:56.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.249+0000 7eff1e7fc700 1 -- 172.21.15.89:0/1440756391 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7eff180406c0 con 0x7eff2809f240 2023-09-30T22:22:56.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.253+0000 7eff1e7fc700 1 -- 172.21.15.89:0/1440756391 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff28132a10 con 0x7eff2809f240 2023-09-30T22:22:56.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.285+0000 7f4a215eb700 1 Processor -- start 2023-09-30T22:22:56.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.285+0000 7f4a215eb700 1 -- start start 2023-09-30T22:22:56.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a215eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a215eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4a1c13d1f0 0x7f4a1c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a215eb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c09a810 con 0x7f4a1c09dc30 2023-09-30T22:22:56.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a215eb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c06ab40 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a215eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c09ebc0 con 0x7f4a1c06a770 2023-09-30T22:22:56.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a1b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4a1c13d1f0 0x7f4a1c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a1b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4a1c13d1f0 0x7f4a1c13f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40764/0 (socket says 172.21.15.89:40764) 2023-09-30T22:22:56.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a1b7fe700 1 -- 172.21.15.89:0/1805795485 learned_addr learned my addr 172.21.15.89:0/1805795485 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:56.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.286+0000 7f4a1a7fc700 1 --1- 172.21.15.89:0/1805795485 >> v1:172.21.15.89:6789/0 conn(0x7f4a1c09dc30 0x7f4a1c09e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52942/0 (socket says 172.21.15.89:52942) 2023-09-30T22:22:56.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a1affd700 1 --2- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a1b7fe700 1 -- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 msgr2=0x7f4a1c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:56.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a1b7fe700 1 --2- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a1b7fe700 1 -- 172.21.15.89:0/1805795485 >> v1:172.21.15.89:6789/0 conn(0x7f4a1c09dc30 legacy=0x7f4a1c09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a1b7fe700 1 -- 172.21.15.89:0/1805795485 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c09e920 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a1affd700 1 --2- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:56.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a1b7fe700 1 --2- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13f5d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009e80 tx=0x7f4a1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f00e3f18b113a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:56.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a19ffb700 1 -- 172.21.15.89:0/1805795485 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400e070 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a19ffb700 1 -- 172.21.15.89:0/1805795485 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1400b980 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.287+0000 7f4a19ffb700 1 -- 172.21.15.89:0/1805795485 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400bd30 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.288+0000 7f4a215eb700 1 -- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 msgr2=0x7f4a1c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.288+0000 7f4a215eb700 1 --2- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13f5d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009e80 tx=0x7f4a1400b330 comp rx=0 tx=0).stop 2023-09-30T22:22:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.288+0000 7f4a215eb700 1 -- 172.21.15.89:0/1805795485 shutdown_connections 2023-09-30T22:22:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.288+0000 7f4a215eb700 1 --2- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13f5d0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.288+0000 7f4a215eb700 1 --2- 172.21.15.89:0/1805795485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.288+0000 7f4a215eb700 1 -- 172.21.15.89:0/1805795485 >> 172.21.15.89:0/1805795485 conn(0x7f4a1c066f50 msgr2=0x7f4a1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.288+0000 7f4a215eb700 1 -- 172.21.15.89:0/1805795485 shutdown_connections 2023-09-30T22:22:56.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.288+0000 7f4a215eb700 1 -- 172.21.15.89:0/1805795485 wait complete. 2023-09-30T22:22:56.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.289+0000 7f4a215eb700 1 Processor -- start 2023-09-30T22:22:56.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.289+0000 7f4a215eb700 1 -- start start 2023-09-30T22:22:56.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.289+0000 7f4a215eb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c11c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.289+0000 7f4a215eb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a1c09dc30 0x7f4a1c13b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.289+0000 7f4a215eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.289+0000 7f4a215eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c11e480 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.289+0000 7f4a215eb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c09f1a0 con 0x7f4a1c09dc30 2023-09-30T22:22:56.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.289+0000 7f4a215eb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c11cee0 con 0x7f4a1c06a770 2023-09-30T22:22:56.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c11c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c11c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54382/0 (socket says 172.21.15.89:54382) 2023-09-30T22:22:56.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40778/0 (socket says 172.21.15.89:40778) 2023-09-30T22:22:56.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1affd700 1 -- 172.21.15.89:0/3089997271 learned_addr learned my addr 172.21.15.89:0/3089997271 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1a7fc700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a1c09dc30 0x7f4a1c13b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1b7fe700 1 -- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 msgr2=0x7f4a1c11c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1b7fe700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c11c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1b7fe700 1 -- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a1c09dc30 msgr2=0x7f4a1c13b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1b7fe700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a1c09dc30 0x7f4a1c13b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1b7fe700 1 -- 172.21.15.89:0/3089997271 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14009b30 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1a7fc700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a1c09dc30 0x7f4a1c13b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a1b7fe700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13bcf0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4a14015000 tx=0x7f4a14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:56.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a037fe700 1 -- 172.21.15.89:0/3089997271 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140048b0 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a037fe700 1 -- 172.21.15.89:0/3089997271 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140092e0 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c11edd0 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a037fe700 1 -- 172.21.15.89:0/3089997271 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1401c680 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.290+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c11f840 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.291+0000 7f4a037fe700 1 -- 172.21.15.89:0/3089997271 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a1401c7e0 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.291+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c074ea0 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.292+0000 7f4a037fe700 1 -- 172.21.15.89:0/3089997271 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4a140410a0 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.295+0000 7f14b5f08700 1 Processor -- start 2023-09-30T22:22:56.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.295+0000 7f14b5f08700 1 -- start start 2023-09-30T22:22:56.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.295+0000 7f14b5f08700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b012cf60 0x7f14b006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.295+0000 7f14b5f08700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012f8b0 0x7f14b012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.295+0000 7f14b5f08700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b012cd90 con 0x7f14b0130260 2023-09-30T22:22:56.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.295+0000 7f14b5f08700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b012d330 con 0x7f14b012f8b0 2023-09-30T22:22:56.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.295+0000 7f14b5f08700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01306d0 con 0x7f14b012cf60 2023-09-30T22:22:56.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14af7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b012cf60 0x7f14b006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14af7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b012cf60 0x7f14b006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54390/0 (socket says 172.21.15.89:54390) 2023-09-30T22:22:56.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14af7fe700 1 -- 172.21.15.89:0/3462353590 learned_addr learned my addr 172.21.15.89:0/3462353590 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:56.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14af7fe700 1 -- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012f8b0 msgr2=0x7f14b012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:56.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14aeffd700 1 --2- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012f8b0 0x7f14b012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14af7fe700 1 --2- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012f8b0 0x7f14b012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14af7fe700 1 -- 172.21.15.89:0/3462353590 >> v1:172.21.15.89:6789/0 conn(0x7f14b0130260 legacy=0x7f14b0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14af7fe700 1 -- 172.21.15.89:0/3462353590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b0071780 con 0x7f14b012cf60 2023-09-30T22:22:56.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14aeffd700 1 --2- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012f8b0 0x7f14b012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:22:56.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14af7fe700 1 --2- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b012cf60 0x7f14b006f5d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009920 tx=0x7f14a0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1cd854d70ba00 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:56.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.296+0000 7f14ae7fc700 1 -- 172.21.15.89:0/3462353590 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000ba60 con 0x7f14b012cf60 2023-09-30T22:22:56.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f14ae7fc700 1 -- 172.21.15.89:0/3462353590 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a000bbc0 con 0x7f14b012cf60 2023-09-30T22:22:56.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f14ae7fc700 1 -- 172.21.15.89:0/3462353590 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0009e10 con 0x7f14b012cf60 2023-09-30T22:22:56.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f14b5f08700 1 -- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b012cf60 msgr2=0x7f14b006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f14b5f08700 1 --2- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b012cf60 0x7f14b006f5d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009920 tx=0x7f14a0009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:56.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f14b5f08700 1 -- 172.21.15.89:0/3462353590 shutdown_connections 2023-09-30T22:22:56.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f14b5f08700 1 --2- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b012cf60 0x7f14b006f5d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f14b5f08700 1 --2- 172.21.15.89:0/3462353590 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012f8b0 0x7f14b012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f14b5f08700 1 -- 172.21.15.89:0/3462353590 >> 172.21.15.89:0/3462353590 conn(0x7f14b00a2aa0 msgr2=0x7f14b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:56.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.298+0000 7f14b5f08700 1 -- 172.21.15.89:0/3462353590 shutdown_connections 2023-09-30T22:22:56.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.297+0000 7f4a037fe700 1 -- 172.21.15.89:0/3089997271 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a14072750 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.298+0000 7f14b5f08700 1 -- 172.21.15.89:0/3462353590 wait complete. 2023-09-30T22:22:56.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.299+0000 7f14b5f08700 1 Processor -- start 2023-09-30T22:22:56.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.299+0000 7f14b5f08700 1 -- start start 2023-09-30T22:22:56.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14b5f08700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012cf60 0x7f14b009ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14b5f08700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b012f8b0 0x7f14b009bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14b5f08700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b0130260 0x7f14b01c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:56.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14b5f08700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b009b580 con 0x7f14b012f8b0 2023-09-30T22:22:56.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14b5f08700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b009ec60 con 0x7f14b012cf60 2023-09-30T22:22:56.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14b5f08700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14b012f610 con 0x7f14b0130260 2023-09-30T22:22:56.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14af7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012cf60 0x7f14b009ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14af7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012cf60 0x7f14b009ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60712/0 (socket says 172.21.15.89:60712) 2023-09-30T22:22:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14af7fe700 1 -- 172.21.15.89:0/49117121 learned_addr learned my addr 172.21.15.89:0/49117121 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14affff700 1 --2- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b0130260 0x7f14b01c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14affff700 1 -- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012cf60 msgr2=0x7f14b009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.300+0000 7f14affff700 1 --2- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012cf60 0x7f14b009ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f14affff700 1 -- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b012f8b0 msgr2=0x7f14b009bf60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:22:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f14affff700 1 --2- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b012f8b0 0x7f14b009bf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f14affff700 1 -- 172.21.15.89:0/49117121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a0009550 con 0x7f14b0130260 2023-09-30T22:22:56.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f14affff700 1 --2- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b0130260 0x7f14b01c3f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f14b0071fd0 tx=0x7f14a800ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:56.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f1497fff700 1 -- 172.21.15.89:0/49117121 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8003c00 con 0x7f14b0130260 2023-09-30T22:22:56.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f1497fff700 1 -- 172.21.15.89:0/49117121 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8003d60 con 0x7f14b0130260 2023-09-30T22:22:56.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f1497fff700 1 -- 172.21.15.89:0/49117121 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800aa80 con 0x7f14b0130260 2023-09-30T22:22:56.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b0129fa0 con 0x7f14b0130260 2023-09-30T22:22:56.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.301+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b009d3d0 con 0x7f14b0130260 2023-09-30T22:22:56.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.302+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b0074ea0 con 0x7f14b0130260 2023-09-30T22:22:56.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.303+0000 7f1497fff700 1 -- 172.21.15.89:0/49117121 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f14a800abe0 con 0x7f14b0130260 2023-09-30T22:22:56.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.303+0000 7f1497fff700 1 -- 172.21.15.89:0/49117121 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f14a803f330 con 0x7f14b0130260 2023-09-30T22:22:56.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.311+0000 7f1497fff700 1 -- 172.21.15.89:0/49117121 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14a801c440 con 0x7f14b0130260 2023-09-30T22:22:56.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.420+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f54ac1b91b0 con 0x7f54ac09e410 2023-09-30T22:22:56.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.420+0000 7f54a2ffd700 1 -- 172.21.15.89:0/100995461 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f54a4018030 con 0x7f54ac09e410 2023-09-30T22:22:56.421 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:56.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09e410 msgr2=0x7f54ac1c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09e410 0x7f54ac1c84f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f54a400efc0 tx=0x7f54a400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 shutdown_connections 2023-09-30T22:22:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54ac09e410 0x7f54ac1c84f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54ac09d6f0 0x7f54ac1a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 --2- 172.21.15.89:0/100995461 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54ac09e000 0x7f54ac1c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 >> 172.21.15.89:0/100995461 conn(0x7f54ac066f50 msgr2=0x7f54ac0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 shutdown_connections 2023-09-30T22:22:56.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.423+0000 7f54b4305700 1 -- 172.21.15.89:0/100995461 wait complete. 2023-09-30T22:22:56.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.476+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7eff28132a10 con 0x7eff2809f240 2023-09-30T22:22:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.476+0000 7eff1e7fc700 1 -- 172.21.15.89:0/1440756391 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff28132a10 con 0x7eff2809f240 2023-09-30T22:22:56.477 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 msgr2=0x7eff2812f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff2812f600 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7eff18016000 tx=0x7eff18004750 comp rx=0 tx=0).stop 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 shutdown_connections 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff2809f650 0x7eff2812fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff2809e930 0x7eff2812f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 --2- 172.21.15.89:0/1440756391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7eff2809f240 0x7eff2812f600 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 >> 172.21.15.89:0/1440756391 conn(0x7eff280a4210 msgr2=0x7eff2806c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 shutdown_connections 2023-09-30T22:22:56.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.478+0000 7eff2fb0d700 1 -- 172.21.15.89:0/1440756391 wait complete. 2023-09-30T22:22:56.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:56.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.502+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4a1c09a3d0 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.502+0000 7f4a037fe700 1 -- 172.21.15.89:0/3089997271 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a140249a0 con 0x7f4a1c13d1f0 2023-09-30T22:22:56.503 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 msgr2=0x7f4a1c13bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13bcf0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4a14015000 tx=0x7f4a14004ad0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 shutdown_connections 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a1c13d1f0 0x7f4a1c13bcf0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1c06a770 0x7f4a1c11c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 --2- 172.21.15.89:0/3089997271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a1c09dc30 0x7f4a1c13b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 >> 172.21.15.89:0/3089997271 conn(0x7f4a1c066f50 msgr2=0x7f4a1c13d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 shutdown_connections 2023-09-30T22:22:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.505+0000 7f4a215eb700 1 -- 172.21.15.89:0/3089997271 wait complete. 2023-09-30T22:22:56.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:56.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.532+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f14b00713a0 con 0x7f14b0130260 2023-09-30T22:22:56.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.532+0000 7f1497fff700 1 -- 172.21.15.89:0/49117121 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14a801b3b0 con 0x7f14b0130260 2023-09-30T22:22:56.533 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.534+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b0130260 msgr2=0x7f14b01c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.535+0000 7f14b5f08700 1 --2- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b0130260 0x7f14b01c3f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f14b0071fd0 tx=0x7f14a800ce70 comp rx=0 tx=0).stop 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.535+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 shutdown_connections 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.535+0000 7f14b5f08700 1 --2- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14b0130260 0x7f14b01c3f20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.535+0000 7f14b5f08700 1 --2- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f14b012cf60 0x7f14b009ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.535+0000 7f14b5f08700 1 --2- 172.21.15.89:0/49117121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14b012f8b0 0x7f14b009bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.535+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 >> 172.21.15.89:0/49117121 conn(0x7f14b00a2aa0 msgr2=0x7f14b006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.535+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 shutdown_connections 2023-09-30T22:22:56.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:56.535+0000 7f14b5f08700 1 -- 172.21.15.89:0/49117121 wait complete. 2023-09-30T22:22:56.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:57.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:57.491 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:57.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:57.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:57.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.589+0000 7f00680ef700 1 Processor -- start 2023-09-30T22:22:57.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.589+0000 7f00680ef700 1 -- start start 2023-09-30T22:22:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f00680ef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f006006a770 0x7f006006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f00680ef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f0060131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f00680ef700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006009ece0 con 0x7f0060131b40 2023-09-30T22:22:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f00680ef700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006009e760 con 0x7f006006a770 2023-09-30T22:22:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f00680ef700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006009e040 con 0x7f006006b080 2023-09-30T22:22:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f0065e8b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f006006a770 0x7f006006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f006568a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f0060131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f0065e8b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f006006a770 0x7f006006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40788/0 (socket says 172.21.15.89:40788) 2023-09-30T22:22:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f0065e8b700 1 -- 172.21.15.89:0/1142520930 learned_addr learned my addr 172.21.15.89:0/1142520930 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f006568a700 1 -- 172.21.15.89:0/1142520930 >> v2:172.21.15.89:3300/0 conn(0x7f006006a770 msgr2=0x7f006006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f006568a700 1 --2- 172.21.15.89:0/1142520930 >> v2:172.21.15.89:3300/0 conn(0x7f006006a770 0x7f006006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f006568a700 1 -- 172.21.15.89:0/1142520930 >> v1:172.21.15.89:6789/0 conn(0x7f0060131b40 legacy=0x7f0060138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f006568a700 1 -- 172.21.15.89:0/1142520930 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006009f630 con 0x7f006006b080 2023-09-30T22:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.590+0000 7f006568a700 1 --2- 172.21.15.89:0/1142520930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f0060131600 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0050009840 tx=0x7f0050009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e873dfdefd119409 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f0064e89700 1 -- 172.21.15.89:0/1142520930 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050011830 con 0x7f006006b080 2023-09-30T22:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f0064e89700 1 -- 172.21.15.89:0/1142520930 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0050011990 con 0x7f006006b080 2023-09-30T22:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f0064e89700 1 -- 172.21.15.89:0/1142520930 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050011d20 con 0x7f006006b080 2023-09-30T22:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f00680ef700 1 -- 172.21.15.89:0/1142520930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 msgr2=0x7f0060131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f00680ef700 1 --2- 172.21.15.89:0/1142520930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f0060131600 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0050009840 tx=0x7f0050009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f00680ef700 1 -- 172.21.15.89:0/1142520930 shutdown_connections 2023-09-30T22:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f00680ef700 1 --2- 172.21.15.89:0/1142520930 >> v2:172.21.15.89:3300/0 conn(0x7f006006a770 0x7f006006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f00680ef700 1 --2- 172.21.15.89:0/1142520930 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f0060131600 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f00680ef700 1 -- 172.21.15.89:0/1142520930 >> 172.21.15.89:0/1142520930 conn(0x7f0060066f50 msgr2=0x7f006009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.591+0000 7f00680ef700 1 -- 172.21.15.89:0/1142520930 shutdown_connections 2023-09-30T22:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.592+0000 7f00680ef700 1 -- 172.21.15.89:0/1142520930 wait complete. 2023-09-30T22:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.593+0000 7f00680ef700 1 Processor -- start 2023-09-30T22:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.593+0000 7f00680ef700 1 -- start start 2023-09-30T22:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.593+0000 7f00680ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006006a770 0x7f00601c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.593+0000 7f00680ef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f00601c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.593+0000 7f00680ef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131b40 0x7f00601c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.593+0000 7f00680ef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006011fd50 con 0x7f006006a770 2023-09-30T22:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.593+0000 7f00680ef700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0060131520 con 0x7f006006b080 2023-09-30T22:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.593+0000 7f00680ef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f006011fa90 con 0x7f0060131b40 2023-09-30T22:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006668c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131b40 0x7f00601c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006668c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131b40 0x7f00601c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54406/0 (socket says 172.21.15.89:54406) 2023-09-30T22:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006668c700 1 -- 172.21.15.89:0/1442279400 learned_addr learned my addr 172.21.15.89:0/1442279400 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006568a700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f00601c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f0065e8b700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006006a770 0x7f00601c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006568a700 1 -- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131b40 msgr2=0x7f00601c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006568a700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131b40 0x7f00601c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006568a700 1 -- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006006a770 msgr2=0x7f00601c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006568a700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006006a770 0x7f00601c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.594+0000 7f006568a700 1 -- 172.21.15.89:0/1442279400 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00500094a0 con 0x7f006006b080 2023-09-30T22:22:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.595+0000 7f006568a700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f00601c6510 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0050016000 tx=0x7f0050004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.595+0000 7f00567fc700 1 -- 172.21.15.89:0/1442279400 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00500036a0 con 0x7f006006b080 2023-09-30T22:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.595+0000 7f00567fc700 1 -- 172.21.15.89:0/1442279400 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005001f070 con 0x7f006006b080 2023-09-30T22:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.595+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0060071e60 con 0x7f006006b080 2023-09-30T22:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.595+0000 7f00567fc700 1 -- 172.21.15.89:0/1442279400 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050003a60 con 0x7f006006b080 2023-09-30T22:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.595+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006011e120 con 0x7f006006b080 2023-09-30T22:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.597+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006011fea0 con 0x7f006006b080 2023-09-30T22:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.597+0000 7f00567fc700 1 -- 172.21.15.89:0/1442279400 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0050003bc0 con 0x7f006006b080 2023-09-30T22:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.597+0000 7f00567fc700 1 -- 172.21.15.89:0/1442279400 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0050021070 con 0x7f006006b080 2023-09-30T22:22:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.605+0000 7f00567fc700 1 -- 172.21.15.89:0/1442279400 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0050019d80 con 0x7f006006b080 2023-09-30T22:22:57.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.636+0000 7f646f580700 1 Processor -- start 2023-09-30T22:22:57.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.636+0000 7f646f580700 1 -- start start 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.636+0000 7f646f580700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646f580700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f6468130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646f580700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646809f130 con 0x7f646809e410 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646f580700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646809f520 con 0x7f646809e000 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646f580700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f646809e930 con 0x7f646809d6f0 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646cb1b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f6468130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646cb1b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f6468130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60722/0 (socket says 172.21.15.89:60722) 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646cb1b700 1 -- 172.21.15.89:0/1779942224 learned_addr learned my addr 172.21.15.89:0/1779942224 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:57.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646d31c700 1 --2- 172.21.15.89:0/1779942224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646cb1b700 1 -- 172.21.15.89:0/1779942224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 msgr2=0x7f646809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:57.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646cb1b700 1 --2- 172.21.15.89:0/1779942224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646cb1b700 1 -- 172.21.15.89:0/1779942224 >> v1:172.21.15.89:6789/0 conn(0x7f646809e410 legacy=0x7f6468130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646cb1b700 1 -- 172.21.15.89:0/1779942224 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646809b2d0 con 0x7f646809e000 2023-09-30T22:22:57.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f646cb1b700 1 --2- 172.21.15.89:0/1779942224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f6468130660 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6458009920 tx=0x7f6458009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac1429958f1e436a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:57.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.637+0000 7f645ffff700 1 -- 172.21.15.89:0/1779942224 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800ba60 con 0x7f646809e000 2023-09-30T22:22:57.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.638+0000 7f645ffff700 1 -- 172.21.15.89:0/1779942224 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645800bbc0 con 0x7f646809e000 2023-09-30T22:22:57.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.638+0000 7f645ffff700 1 -- 172.21.15.89:0/1779942224 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458009e70 con 0x7f646809e000 2023-09-30T22:22:57.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.638+0000 7f646f580700 1 -- 172.21.15.89:0/1779942224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 msgr2=0x7f6468130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.638+0000 7f646f580700 1 --2- 172.21.15.89:0/1779942224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f6468130660 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6458009920 tx=0x7f6458009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:57.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.639+0000 7f646f580700 1 -- 172.21.15.89:0/1779942224 shutdown_connections 2023-09-30T22:22:57.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.639+0000 7f646f580700 1 --2- 172.21.15.89:0/1779942224 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.639+0000 7f646f580700 1 --2- 172.21.15.89:0/1779942224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f6468130660 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.639+0000 7f646f580700 1 -- 172.21.15.89:0/1779942224 >> 172.21.15.89:0/1779942224 conn(0x7f6468066f50 msgr2=0x7f646809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:57.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.639+0000 7f646f580700 1 -- 172.21.15.89:0/1779942224 shutdown_connections 2023-09-30T22:22:57.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.639+0000 7f646f580700 1 -- 172.21.15.89:0/1779942224 wait complete. 2023-09-30T22:22:57.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.640+0000 7f646f580700 1 Processor -- start 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.640+0000 7f646f580700 1 -- start start 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646f580700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646812fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646f580700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f64681300a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646f580700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f646809e410 0x7f64681305e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646f580700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468139e10 con 0x7f646809e410 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646f580700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646813a430 con 0x7f646809e000 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646f580700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6468072370 con 0x7f646809d6f0 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646d31c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646812fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646cb1b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f64681300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646d31c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646812fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54412/0 (socket says 172.21.15.89:54412) 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646d31c700 1 -- 172.21.15.89:0/1794116888 learned_addr learned my addr 172.21.15.89:0/1794116888 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646db1d700 1 --2- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f646809e410 0x7f64681305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646cb1b700 1 -- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 msgr2=0x7f646812fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646cb1b700 1 --2- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646812fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646cb1b700 1 -- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f646809e410 msgr2=0x7f64681305e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646cb1b700 1 --2- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f646809e410 0x7f64681305e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.641+0000 7f646cb1b700 1 -- 172.21.15.89:0/1794116888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6458009550 con 0x7f646809e000 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.642+0000 7f646cb1b700 1 --2- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f64681300a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f64680a3e20 tx=0x7f6458004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.642+0000 7f645dffb700 1 -- 172.21.15.89:0/1794116888 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645801a900 con 0x7f646809e000 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.642+0000 7f645dffb700 1 -- 172.21.15.89:0/1794116888 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645801f070 con 0x7f646809e000 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.642+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646809c700 con 0x7f646809e000 2023-09-30T22:22:57.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.642+0000 7f645dffb700 1 -- 172.21.15.89:0/1794116888 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645801ac80 con 0x7f646809e000 2023-09-30T22:22:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.642+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646813a600 con 0x7f646809e000 2023-09-30T22:22:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.643+0000 7f645dffb700 1 -- 172.21.15.89:0/1794116888 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6458026480 con 0x7f646809e000 2023-09-30T22:22:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.643+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6468074ea0 con 0x7f646809e000 2023-09-30T22:22:57.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.644+0000 7f645dffb700 1 -- 172.21.15.89:0/1794116888 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6458021070 con 0x7f646809e000 2023-09-30T22:22:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.651+0000 7f645dffb700 1 -- 172.21.15.89:0/1794116888 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f645802b020 con 0x7f646809e000 2023-09-30T22:22:57.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.686+0000 7f200b3ea700 1 Processor -- start 2023-09-30T22:22:57.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.686+0000 7f200b3ea700 1 -- start start 2023-09-30T22:22:57.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f200b3ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f200b3ea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f200409d630 0x7f20040a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f200b3ea700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20040a2a70 con 0x7f2004133930 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f200b3ea700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20040a01d0 con 0x7f200409d630 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f200b3ea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200409e2b0 con 0x7f20040a0a30 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f2008985700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f200409d630 0x7f20040a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f2009186700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f2008985700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f200409d630 0x7f20040a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40806/0 (socket says 172.21.15.89:40806) 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f2009186700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20040a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60736/0 (socket says 172.21.15.89:60736) 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f2008985700 1 -- 172.21.15.89:0/1803011720 learned_addr learned my addr 172.21.15.89:0/1803011720 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.687+0000 7f2009987700 1 --1- 172.21.15.89:0/1803011720 >> v1:172.21.15.89:6789/0 conn(0x7f2004133930 0x7f2004133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52988/0 (socket says 172.21.15.89:52988) 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.688+0000 7f2009186700 1 -- 172.21.15.89:0/1803011720 >> v2:172.21.15.89:3300/0 conn(0x7f200409d630 msgr2=0x7f20040a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:57.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.688+0000 7f2009186700 1 --2- 172.21.15.89:0/1803011720 >> v2:172.21.15.89:3300/0 conn(0x7f200409d630 0x7f20040a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.688+0000 7f2009186700 1 -- 172.21.15.89:0/1803011720 >> v1:172.21.15.89:6789/0 conn(0x7f2004133930 legacy=0x7f2004133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.688+0000 7f2009186700 1 -- 172.21.15.89:0/1803011720 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200409dc50 con 0x7f20040a0a30 2023-09-30T22:22:57.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.688+0000 7f2009186700 1 --2- 172.21.15.89:0/1803011720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20040a0e00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2000009840 tx=0x7f2000009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3cbbbf47a205f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.688+0000 7f1ffbfff700 1 -- 172.21.15.89:0/1803011720 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000011830 con 0x7f20040a0a30 2023-09-30T22:22:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.688+0000 7f1ffbfff700 1 -- 172.21.15.89:0/1803011720 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2000011990 con 0x7f20040a0a30 2023-09-30T22:22:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.688+0000 7f1ffbfff700 1 -- 172.21.15.89:0/1803011720 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000011d20 con 0x7f20040a0a30 2023-09-30T22:22:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.689+0000 7f200b3ea700 1 -- 172.21.15.89:0/1803011720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 msgr2=0x7f20040a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.689+0000 7f200b3ea700 1 --2- 172.21.15.89:0/1803011720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20040a0e00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2000009840 tx=0x7f2000009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.689+0000 7f200b3ea700 1 -- 172.21.15.89:0/1803011720 shutdown_connections 2023-09-30T22:22:57.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.689+0000 7f200b3ea700 1 --2- 172.21.15.89:0/1803011720 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20040a0e00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.689+0000 7f200b3ea700 1 --2- 172.21.15.89:0/1803011720 >> v2:172.21.15.89:3300/0 conn(0x7f200409d630 0x7f20040a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.689+0000 7f200b3ea700 1 -- 172.21.15.89:0/1803011720 >> 172.21.15.89:0/1803011720 conn(0x7f20040a3050 msgr2=0x7f20040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.689+0000 7f200b3ea700 1 -- 172.21.15.89:0/1803011720 shutdown_connections 2023-09-30T22:22:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.689+0000 7f200b3ea700 1 -- 172.21.15.89:0/1803011720 wait complete. 2023-09-30T22:22:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.690+0000 7f200b3ea700 1 Processor -- start 2023-09-30T22:22:57.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.690+0000 7f200b3ea700 1 -- start start 2023-09-30T22:22:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f200b3ea700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f200409d630 0x7f2004121290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f200b3ea700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20041217d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f200b3ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2004133930 0x7f2004121d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f200b3ea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2004122dc0 con 0x7f2004133930 2023-09-30T22:22:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f200b3ea700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2004123c50 con 0x7f20040a0a30 2023-09-30T22:22:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f200b3ea700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2004124730 con 0x7f200409d630 2023-09-30T22:22:57.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f2008985700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20041217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f2008985700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20041217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60748/0 (socket says 172.21.15.89:60748) 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f2008985700 1 -- 172.21.15.89:0/72040624 learned_addr learned my addr 172.21.15.89:0/72040624 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f2009987700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2004133930 0x7f2004121d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.691+0000 7f2009186700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f200409d630 0x7f2004121290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f2008985700 1 -- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f200409d630 msgr2=0x7f2004121290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f2008985700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f200409d630 0x7f2004121290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f2008985700 1 -- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2004133930 msgr2=0x7f2004121d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f2008985700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2004133930 0x7f2004121d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f2008985700 1 -- 172.21.15.89:0/72040624 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000094a0 con 0x7f20040a0a30 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f2009987700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2004133930 0x7f2004121d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f2009186700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f200409d630 0x7f2004121290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f2008985700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20041217d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400e910 tx=0x7f1ff400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f1ff9ffb700 1 -- 172.21.15.89:0/72040624 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400c850 con 0x7f20040a0a30 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f1ff9ffb700 1 -- 172.21.15.89:0/72040624 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff400c9b0 con 0x7f20040a0a30 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20041249f0 con 0x7f20040a0a30 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f1ff9ffb700 1 -- 172.21.15.89:0/72040624 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400cd20 con 0x7f20040a0a30 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.692+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2004124460 con 0x7f20040a0a30 2023-09-30T22:22:57.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.693+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2004124010 con 0x7f20040a0a30 2023-09-30T22:22:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.694+0000 7f1ff9ffb700 1 -- 172.21.15.89:0/72040624 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ff4013470 con 0x7f20040a0a30 2023-09-30T22:22:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.694+0000 7f1ff9ffb700 1 -- 172.21.15.89:0/72040624 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1ff403fb80 con 0x7f20040a0a30 2023-09-30T22:22:57.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.703+0000 7f1ff9ffb700 1 -- 172.21.15.89:0/72040624 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ff4013720 con 0x7f20040a0a30 2023-09-30T22:22:57.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.717+0000 7f5ca8d69700 1 Processor -- start 2023-09-30T22:22:57.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.717+0000 7f5ca8d69700 1 -- start start 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca8d69700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca8d69700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ca409dc30 0x7f5ca409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca8d69700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca409a810 con 0x7f5ca413c1e0 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca8d69700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca406ab40 con 0x7f5ca409dc30 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca8d69700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca409ebc0 con 0x7f5ca406a770 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60766/0 (socket says 172.21.15.89:60766) 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca259c700 1 -- 172.21.15.89:0/1663981186 learned_addr learned my addr 172.21.15.89:0/1663981186 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.718+0000 7f5ca1d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ca409dc30 0x7f5ca409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.719+0000 7f5ca259c700 1 -- 172.21.15.89:0/1663981186 >> v2:172.21.15.89:3300/0 conn(0x7f5ca409dc30 msgr2=0x7f5ca409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:57.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.719+0000 7f5ca259c700 1 --2- 172.21.15.89:0/1663981186 >> v2:172.21.15.89:3300/0 conn(0x7f5ca409dc30 0x7f5ca409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.719+0000 7f5ca259c700 1 -- 172.21.15.89:0/1663981186 >> v1:172.21.15.89:6789/0 conn(0x7f5ca413c1e0 legacy=0x7f5ca413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.719+0000 7f5ca259c700 1 -- 172.21.15.89:0/1663981186 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca409e920 con 0x7f5ca406a770 2023-09-30T22:22:57.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.719+0000 7f5ca259c700 1 --2- 172.21.15.89:0/1663981186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca409d6f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009920 tx=0x7f5c94009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886742c8c5083eea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:57.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.719+0000 7f5ca159a700 1 -- 172.21.15.89:0/1663981186 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94011830 con 0x7f5ca406a770 2023-09-30T22:22:57.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.719+0000 7f5ca159a700 1 -- 172.21.15.89:0/1663981186 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94011990 con 0x7f5ca406a770 2023-09-30T22:22:57.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.719+0000 7f5ca159a700 1 -- 172.21.15.89:0/1663981186 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94011d20 con 0x7f5ca406a770 2023-09-30T22:22:57.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.720+0000 7f5ca8d69700 1 -- 172.21.15.89:0/1663981186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 msgr2=0x7f5ca409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.720+0000 7f5ca8d69700 1 --2- 172.21.15.89:0/1663981186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca409d6f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009920 tx=0x7f5c94009c30 comp rx=0 tx=0).stop 2023-09-30T22:22:57.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.720+0000 7f5ca8d69700 1 -- 172.21.15.89:0/1663981186 shutdown_connections 2023-09-30T22:22:57.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.720+0000 7f5ca8d69700 1 --2- 172.21.15.89:0/1663981186 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca409d6f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.720+0000 7f5ca8d69700 1 --2- 172.21.15.89:0/1663981186 >> v2:172.21.15.89:3300/0 conn(0x7f5ca409dc30 0x7f5ca409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.720+0000 7f5ca8d69700 1 -- 172.21.15.89:0/1663981186 >> 172.21.15.89:0/1663981186 conn(0x7f5ca4066f50 msgr2=0x7f5ca409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.720+0000 7f5ca8d69700 1 -- 172.21.15.89:0/1663981186 shutdown_connections 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.720+0000 7f5ca8d69700 1 -- 172.21.15.89:0/1663981186 wait complete. 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.721+0000 7f5ca8d69700 1 Processor -- start 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.721+0000 7f5ca8d69700 1 -- start start 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.721+0000 7f5ca8d69700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca413e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.721+0000 7f5ca8d69700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ca409dc30 0x7f5ca413a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.721+0000 7f5ca8d69700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ca413c1e0 0x7f5ca413a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.721+0000 7f5ca8d69700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4134b90 con 0x7f5ca413c1e0 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.721+0000 7f5ca8d69700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca413bd40 con 0x7f5ca406a770 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.721+0000 7f5ca8d69700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca409f1a0 con 0x7f5ca409dc30 2023-09-30T22:22:57.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ca413c1e0 0x7f5ca413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ca413c1e0 0x7f5ca413a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40828/0 (socket says 172.21.15.89:40828) 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 -- 172.21.15.89:0/3426372447 learned_addr learned my addr 172.21.15.89:0/3426372447 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca259c700 1 --2- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca413e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 -- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ca409dc30 msgr2=0x7f5ca413a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 --2- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ca409dc30 0x7f5ca413a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 -- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 msgr2=0x7f5ca413e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 --2- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca413e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 -- 172.21.15.89:0/3426372447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c94009550 con 0x7f5ca413c1e0 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca2d9d700 1 --2- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ca413c1e0 0x7f5ca413a990 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00cbc0 tx=0x7f5c9c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3426372447 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c003c00 con 0x7f5ca413c1e0 2023-09-30T22:22:57.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca411dae0 con 0x7f5ca413c1e0 2023-09-30T22:22:57.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca413fa80 con 0x7f5ca413c1e0 2023-09-30T22:22:57.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3426372447 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c003d60 con 0x7f5ca413c1e0 2023-09-30T22:22:57.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.722+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3426372447 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00ab40 con 0x7f5ca413c1e0 2023-09-30T22:22:57.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.723+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c880052e0 con 0x7f5ca413c1e0 2023-09-30T22:22:57.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.724+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3426372447 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c9c00aca0 con 0x7f5ca413c1e0 2023-09-30T22:22:57.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.724+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3426372447 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5c9c03f830 con 0x7f5ca413c1e0 2023-09-30T22:22:57.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.729+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3426372447 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c9c0196c0 con 0x7f5ca413c1e0 2023-09-30T22:22:57.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.813+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0060074ea0 con 0x7f006006b080 2023-09-30T22:22:57.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.813+0000 7f00567fc700 1 -- 172.21.15.89:0/1442279400 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f005002b030 con 0x7f006006b080 2023-09-30T22:22:57.814 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:57.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 msgr2=0x7f00601c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f00601c6510 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0050016000 tx=0x7f0050004750 comp rx=0 tx=0).stop 2023-09-30T22:22:57.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 shutdown_connections 2023-09-30T22:22:57.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0060131b40 0x7f00601c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f006006a770 0x7f00601c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 --2- 172.21.15.89:0/1442279400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f006006b080 0x7f00601c6510 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 >> 172.21.15.89:0/1442279400 conn(0x7f0060066f50 msgr2=0x7f00600a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:57.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 shutdown_connections 2023-09-30T22:22:57.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.816+0000 7f00680ef700 1 -- 172.21.15.89:0/1442279400 wait complete. 2023-09-30T22:22:57.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.850+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f646806c360 con 0x7f646809e000 2023-09-30T22:22:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.850+0000 7f645dffb700 1 -- 172.21.15.89:0/1794116888 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6458026730 con 0x7f646809e000 2023-09-30T22:22:57.851 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 msgr2=0x7f64681300a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 --2- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f64681300a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f64680a3e20 tx=0x7f6458004750 comp rx=0 tx=0).stop 2023-09-30T22:22:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 shutdown_connections 2023-09-30T22:22:57.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 --2- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f646809e410 0x7f64681305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 --2- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f646809d6f0 0x7f646812fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 --2- 172.21.15.89:0/1794116888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f646809e000 0x7f64681300a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 >> 172.21.15.89:0/1794116888 conn(0x7f6468066f50 msgr2=0x7f64680a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 shutdown_connections 2023-09-30T22:22:57.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.852+0000 7f646f580700 1 -- 172.21.15.89:0/1794116888 wait complete. 2023-09-30T22:22:57.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.913+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f200411df60 con 0x7f20040a0a30 2023-09-30T22:22:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.913+0000 7f1ff9ffb700 1 -- 172.21.15.89:0/72040624 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ff4021070 con 0x7f20040a0a30 2023-09-30T22:22:57.914 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.915+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 msgr2=0x7f20041217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.916+0000 7f200b3ea700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20041217d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400e910 tx=0x7f1ff400ecd0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.916+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 shutdown_connections 2023-09-30T22:22:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.916+0000 7f200b3ea700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2004133930 0x7f2004121d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.916+0000 7f200b3ea700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f200409d630 0x7f2004121290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.916+0000 7f200b3ea700 1 --2- 172.21.15.89:0/72040624 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20040a0a30 0x7f20041217d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.916+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 >> 172.21.15.89:0/72040624 conn(0x7f20040a3050 msgr2=0x7f20041331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.916+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 shutdown_connections 2023-09-30T22:22:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.916+0000 7f200b3ea700 1 -- 172.21.15.89:0/72040624 wait complete. 2023-09-30T22:22:57.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:57.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.946+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5c88005150 con 0x7f5ca413c1e0 2023-09-30T22:22:57.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.946+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3426372447 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c9c011380 con 0x7f5ca413c1e0 2023-09-30T22:22:57.947 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:57.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ca413c1e0 msgr2=0x7f5ca413a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:57.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 --2- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ca413c1e0 0x7f5ca413a990 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00cbc0 tx=0x7f5c9c00ced0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 shutdown_connections 2023-09-30T22:22:57.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 --2- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ca413c1e0 0x7f5ca413a990 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 --2- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5ca406a770 0x7f5ca413e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 --2- 172.21.15.89:0/3426372447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5ca409dc30 0x7f5ca413a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:57.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 >> 172.21.15.89:0/3426372447 conn(0x7f5ca4066f50 msgr2=0x7f5ca4072240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:57.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 shutdown_connections 2023-09-30T22:22:57.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:57.948+0000 7f5ca8d69700 1 -- 172.21.15.89:0/3426372447 wait complete. 2023-09-30T22:22:57.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:22:58.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:22:58.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:22:58.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:22:58.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:22:58.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.970+0000 7f68f043a700 1 Processor -- start 2023-09-30T22:22:58.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.970+0000 7f68f043a700 1 -- start start 2023-09-30T22:22:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68f043a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f68e81316c0 0x7f68e8131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68f043a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e809d9f0 0x7f68e806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68f043a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e806b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68f043a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e809e130 con 0x7f68e81316c0 2023-09-30T22:22:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68f043a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e81312b0 con 0x7f68e806acb0 2023-09-30T22:22:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68f043a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68e809f6e0 con 0x7f68e809d9f0 2023-09-30T22:22:58.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ed9d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e809d9f0 0x7f68e806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:58.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ee1d6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f68e81316c0 0x7f68e8131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ed9d5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e809d9f0 0x7f68e806a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54444/0 (socket says 172.21.15.89:54444) 2023-09-30T22:22:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ed9d5700 1 -- 172.21.15.89:0/3327405996 learned_addr learned my addr 172.21.15.89:0/3327405996 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ee9d7700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e806b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ee1d6700 1 -- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e809d9f0 msgr2=0x7f68e806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ee1d6700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e809d9f0 0x7f68e806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ee1d6700 1 -- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 msgr2=0x7f68e806b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ee1d6700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e806b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ee1d6700 1 -- 172.21.15.89:0/3327405996 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e809e760 con 0x7f68e81316c0 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ed9d5700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e809d9f0 0x7f68e806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ee1d6700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e81316c0 0x7f68e8131a90 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009840 tx=0x7f68e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a674e02d6c82cd96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.971+0000 7f68ed1d4700 1 -- 172.21.15.89:0/3327405996 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4011830 con 0x7f68e81316c0 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68ed1d4700 1 -- 172.21.15.89:0/3327405996 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e4011990 con 0x7f68e81316c0 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68ed1d4700 1 -- 172.21.15.89:0/3327405996 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4011d20 con 0x7f68e81316c0 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68f043a700 1 -- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e81316c0 msgr2=0x7f68e8131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68f043a700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e81316c0 0x7f68e8131a90 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009840 tx=0x7f68e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68f043a700 1 -- 172.21.15.89:0/3327405996 shutdown_connections 2023-09-30T22:22:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68f043a700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e806b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68f043a700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e81316c0 0x7f68e8131a90 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68f043a700 1 --2- 172.21.15.89:0/3327405996 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e809d9f0 0x7f68e806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68f043a700 1 -- 172.21.15.89:0/3327405996 >> 172.21.15.89:0/3327405996 conn(0x7f68e8066f50 msgr2=0x7f68e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.972+0000 7f68f043a700 1 -- 172.21.15.89:0/3327405996 shutdown_connections 2023-09-30T22:22:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.973+0000 7f68f043a700 1 -- 172.21.15.89:0/3327405996 wait complete. 2023-09-30T22:22:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.974+0000 7f68f043a700 1 Processor -- start 2023-09-30T22:22:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.974+0000 7f68f043a700 1 -- start start 2023-09-30T22:22:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.974+0000 7f68f043a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e8132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68f043a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e809d9f0 0x7f68e8132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68f043a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e81316c0 0x7f68e8132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68f043a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e812b900 con 0x7f68e809d9f0 2023-09-30T22:22:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68f043a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8134be0 con 0x7f68e806acb0 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68f043a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80a3540 con 0x7f68e81316c0 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e8132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e8132300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60790/0 (socket says 172.21.15.89:60790) 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 -- 172.21.15.89:0/2989357780 learned_addr learned my addr 172.21.15.89:0/2989357780 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ed9d5700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e809d9f0 0x7f68e8132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee9d7700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e81316c0 0x7f68e8132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 -- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e81316c0 msgr2=0x7f68e8132d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e81316c0 0x7f68e8132d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 -- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e809d9f0 msgr2=0x7f68e8132840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e809d9f0 0x7f68e8132840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 -- 172.21.15.89:0/2989357780 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d8009550 con 0x7f68e806acb0 2023-09-30T22:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ed9d5700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e809d9f0 0x7f68e8132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.975+0000 7f68ee1d6700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e8132300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f68e40090b0 tx=0x7f68e4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.976+0000 7f68deffd700 1 -- 172.21.15.89:0/2989357780 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4004450 con 0x7f68e806acb0 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.976+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e40094a0 con 0x7f68e806acb0 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.976+0000 7f68deffd700 1 -- 172.21.15.89:0/2989357780 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e401f070 con 0x7f68e806acb0 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.976+0000 7f68deffd700 1 -- 172.21.15.89:0/2989357780 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e40038e0 con 0x7f68e806acb0 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.976+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e8134540 con 0x7f68e806acb0 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.977+0000 7f68deffd700 1 -- 172.21.15.89:0/2989357780 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68e4003a40 con 0x7f68e806acb0 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.977+0000 7f68deffd700 1 -- 172.21.15.89:0/2989357780 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f68e4021070 con 0x7f68e806acb0 2023-09-30T22:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.977+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e8074ea0 con 0x7f68e806acb0 2023-09-30T22:22:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:58.982+0000 7f68deffd700 1 -- 172.21.15.89:0/2989357780 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68e401a430 con 0x7f68e806acb0 2023-09-30T22:22:59.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f81b8a700 1 Processor -- start 2023-09-30T22:22:59.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f81b8a700 1 -- start start 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f81b8a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f7c09e1d0 0x7f0f7c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f81b8a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c135e50 0x7f0f7c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f81b8a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7c09f130 con 0x7f0f7c09d6f0 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f81b8a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7c09f520 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f81b8a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c09e930 con 0x7f0f7c135e50 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f7b7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0f7c09d6f0 0x7f0f7c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52996/0 (socket says 172.21.15.89:52996) 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f7b7fe700 1 -- 172.21.15.89:0/3970349185 learned_addr learned my addr 172.21.15.89:0/3970349185 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.015+0000 7f0f7affd700 1 --2- 172.21.15.89:0/3970349185 >> v2:172.21.15.89:3300/0 conn(0x7f0f7c09e1d0 0x7f0f7c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7bfff700 1 --2- 172.21.15.89:0/3970349185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c135e50 0x7f0f7c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7affd700 1 -- 172.21.15.89:0/3970349185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c135e50 msgr2=0x7f0f7c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:59.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7affd700 1 --2- 172.21.15.89:0/3970349185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c135e50 0x7f0f7c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7affd700 1 -- 172.21.15.89:0/3970349185 >> v1:172.21.15.89:6789/0 conn(0x7f0f7c09d6f0 legacy=0x7f0f7c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7affd700 1 -- 172.21.15.89:0/3970349185 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c09b2d0 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7affd700 1 --2- 172.21.15.89:0/3970349185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 0x7f0f7c135910 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009840 tx=0x7f0f70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91dd92008bc4d6cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:59.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7a7fc700 1 -- 172.21.15.89:0/3970349185 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70011830 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7a7fc700 1 -- 172.21.15.89:0/3970349185 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f70011990 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.016+0000 7f0f7a7fc700 1 -- 172.21.15.89:0/3970349185 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70011d20 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.017+0000 7f0f81b8a700 1 -- 172.21.15.89:0/3970349185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 msgr2=0x7f0f7c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.017+0000 7f0f81b8a700 1 --2- 172.21.15.89:0/3970349185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 0x7f0f7c135910 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009840 tx=0x7f0f70009b50 comp rx=0 tx=0).stop 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.017+0000 7f0f81b8a700 1 -- 172.21.15.89:0/3970349185 shutdown_connections 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.017+0000 7f0f81b8a700 1 --2- 172.21.15.89:0/3970349185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c135e50 0x7f0f7c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.017+0000 7f0f81b8a700 1 --2- 172.21.15.89:0/3970349185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 0x7f0f7c135910 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.017+0000 7f0f81b8a700 1 -- 172.21.15.89:0/3970349185 >> 172.21.15.89:0/3970349185 conn(0x7f0f7c066f50 msgr2=0x7f0f7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.017+0000 7f0f81b8a700 1 -- 172.21.15.89:0/3970349185 shutdown_connections 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.017+0000 7f0f81b8a700 1 -- 172.21.15.89:0/3970349185 wait complete. 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.018+0000 7f0f81b8a700 1 Processor -- start 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.018+0000 7f0f81b8a700 1 -- start start 2023-09-30T22:22:59.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f81b8a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c09d6f0 0x7f0f7c1c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f81b8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 0x7f0f7c1c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f81b8a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7c135e50 0x7f0f7c1c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f81b8a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c1211f0 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f81b8a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0a1cf0 con 0x7f0f7c09d6f0 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f81b8a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c09c700 con 0x7f0f7c135e50 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c09d6f0 0x7f0f7c1c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 0x7f0f7c1c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c09d6f0 0x7f0f7c1c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60808/0 (socket says 172.21.15.89:60808) 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7b7fe700 1 -- 172.21.15.89:0/2343868277 learned_addr learned my addr 172.21.15.89:0/2343868277 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7bfff700 1 --2- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7c135e50 0x7f0f7c1c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7affd700 1 -- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7c135e50 msgr2=0x7f0f7c1c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7affd700 1 --2- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7c135e50 0x7f0f7c1c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7affd700 1 -- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c09d6f0 msgr2=0x7f0f7c1c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7affd700 1 --2- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c09d6f0 0x7f0f7c1c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.019+0000 7f0f7affd700 1 -- 172.21.15.89:0/2343868277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700094a0 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.020+0000 7f0f7affd700 1 --2- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 0x7f0f7c1c6590 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0f70016000 tx=0x7f0f70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.020+0000 7f0f63fff700 1 -- 172.21.15.89:0/2343868277 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70004550 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.020+0000 7f0f63fff700 1 -- 172.21.15.89:0/2343868277 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7001f070 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.020+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c122e20 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.020+0000 7f0f63fff700 1 -- 172.21.15.89:0/2343868277 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f700038e0 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.020+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c123890 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.021+0000 7f0f63fff700 1 -- 172.21.15.89:0/2343868277 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f70003a40 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.021+0000 7f0f63fff700 1 -- 172.21.15.89:0/2343868277 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0f70021070 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.021+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c122ac0 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.026+0000 7f0f63fff700 1 -- 172.21.15.89:0/2343868277 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f7c122ac0 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.071+0000 7f10cf3dd700 1 Processor -- start 2023-09-30T22:22:59.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.072+0000 7f10cf3dd700 1 -- start start 2023-09-30T22:22:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.072+0000 7f10cf3dd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f10c80a0a30 0x7f10c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.072+0000 7f10cf3dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c8138d80 0x7f10c813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.072+0000 7f10cf3dd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c80a2a70 con 0x7f10c809d630 2023-09-30T22:22:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.072+0000 7f10cf3dd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c80a01d0 con 0x7f10c80a0a30 2023-09-30T22:22:59.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.072+0000 7f10cf3dd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10c809e2b0 con 0x7f10c8138d80 2023-09-30T22:22:59.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd97a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c8138d80 0x7f10c813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd179700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f10c80a0a30 0x7f10c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd179700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f10c80a0a30 0x7f10c80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40886/0 (socket says 172.21.15.89:40886) 2023-09-30T22:22:59.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd97a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c8138d80 0x7f10c813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54460/0 (socket says 172.21.15.89:54460) 2023-09-30T22:22:59.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd179700 1 -- 172.21.15.89:0/894593962 learned_addr learned my addr 172.21.15.89:0/894593962 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:59.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd97a700 1 -- 172.21.15.89:0/894593962 >> v2:172.21.15.89:3300/0 conn(0x7f10c80a0a30 msgr2=0x7f10c80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:59.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd97a700 1 --2- 172.21.15.89:0/894593962 >> v2:172.21.15.89:3300/0 conn(0x7f10c80a0a30 0x7f10c80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd97a700 1 -- 172.21.15.89:0/894593962 >> v1:172.21.15.89:6789/0 conn(0x7f10c809d630 legacy=0x7f10c80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:59.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd97a700 1 -- 172.21.15.89:0/894593962 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c809dc50 con 0x7f10c8138d80 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10cd97a700 1 --2- 172.21.15.89:0/894593962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c8138d80 0x7f10c813b160 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009d90 tx=0x7f10c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed6d46298cade90 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10bffff700 1 -- 172.21.15.89:0/894593962 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c000e070 con 0x7f10c8138d80 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10bffff700 1 -- 172.21.15.89:0/894593962 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c000bae0 con 0x7f10c8138d80 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.073+0000 7f10bffff700 1 -- 172.21.15.89:0/894593962 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c000be90 con 0x7f10c8138d80 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.074+0000 7f10cf3dd700 1 -- 172.21.15.89:0/894593962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c8138d80 msgr2=0x7f10c813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.074+0000 7f10cf3dd700 1 --2- 172.21.15.89:0/894593962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c8138d80 0x7f10c813b160 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009d90 tx=0x7f10c000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.074+0000 7f10cf3dd700 1 -- 172.21.15.89:0/894593962 shutdown_connections 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.074+0000 7f10cf3dd700 1 --2- 172.21.15.89:0/894593962 >> v2:172.21.15.89:3300/0 conn(0x7f10c80a0a30 0x7f10c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.074+0000 7f10cf3dd700 1 --2- 172.21.15.89:0/894593962 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c8138d80 0x7f10c813b160 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.074+0000 7f10cf3dd700 1 -- 172.21.15.89:0/894593962 >> 172.21.15.89:0/894593962 conn(0x7f10c80a3050 msgr2=0x7f10c80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.074+0000 7f10cf3dd700 1 -- 172.21.15.89:0/894593962 shutdown_connections 2023-09-30T22:22:59.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.075+0000 7f10cf3dd700 1 -- 172.21.15.89:0/894593962 wait complete. 2023-09-30T22:22:59.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.076+0000 7f10cf3dd700 1 Processor -- start 2023-09-30T22:22:59.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.076+0000 7f10cf3dd700 1 -- start start 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cf3dd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c809d630 0x7f10c81336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cf3dd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c80a0a30 0x7f10c8133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cf3dd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10c8138d80 0x7f10c8134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cf3dd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c81365c0 con 0x7f10c809d630 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cf3dd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c8136f30 con 0x7f10c8138d80 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cf3dd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10c80a4c70 con 0x7f10c80a0a30 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c809d630 0x7f10c81336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cc978700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c80a0a30 0x7f10c8133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c809d630 0x7f10c81336f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40894/0 (socket says 172.21.15.89:40894) 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 -- 172.21.15.89:0/670587862 learned_addr learned my addr 172.21.15.89:0/670587862 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:59.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd97a700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10c8138d80 0x7f10c8134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 -- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c80a0a30 msgr2=0x7f10c8133c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c80a0a30 0x7f10c8133c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 -- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10c8138d80 msgr2=0x7f10c8134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10c8138d80 0x7f10c8134170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 -- 172.21.15.89:0/670587862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0009a70 con 0x7f10c809d630 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cc978700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c80a0a30 0x7f10c8133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.077+0000 7f10cd179700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c809d630 0x7f10c81336f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f10c400e910 tx=0x7f10c400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.078+0000 7f10bdffb700 1 -- 172.21.15.89:0/670587862 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c40096a0 con 0x7f10c809d630 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.078+0000 7f10bdffb700 1 -- 172.21.15.89:0/670587862 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c4013680 con 0x7f10c809d630 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.078+0000 7f10bdffb700 1 -- 172.21.15.89:0/670587862 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4013a30 con 0x7f10c809d630 2023-09-30T22:22:59.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.078+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c80a0410 con 0x7f10c809d630 2023-09-30T22:22:59.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.078+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c8137210 con 0x7f10c809d630 2023-09-30T22:22:59.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.079+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c8136d70 con 0x7f10c809d630 2023-09-30T22:22:59.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.079+0000 7f10bdffb700 1 -- 172.21.15.89:0/670587862 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f10c4013b90 con 0x7f10c809d630 2023-09-30T22:22:59.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.079+0000 7f10bdffb700 1 -- 172.21.15.89:0/670587862 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f10c403f670 con 0x7f10c809d630 2023-09-30T22:22:59.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.084+0000 7f10bdffb700 1 -- 172.21.15.89:0/670587862 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10c401bdf0 con 0x7f10c809d630 2023-09-30T22:22:59.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.101+0000 7fececcd1700 1 Processor -- start 2023-09-30T22:22:59.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.101+0000 7fececcd1700 1 -- start start 2023-09-30T22:22:59.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fececcd1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece809de00 0x7fece809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fececcd1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fece813d1f0 0x7fece813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fececcd1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece809a810 con 0x7fece806a770 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fececcd1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece806ab40 con 0x7fece813d1f0 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fececcd1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fece809ebc0 con 0x7fece809de00 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fece5d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece809de00 0x7fece809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fece5d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece809de00 0x7fece809e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54474/0 (socket says 172.21.15.89:54474) 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fece6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fece813d1f0 0x7fece813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.102+0000 7fece5d9b700 1 -- 172.21.15.89:0/2426142940 learned_addr learned my addr 172.21.15.89:0/2426142940 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:59.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece6d9d700 1 -- 172.21.15.89:0/2426142940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece809de00 msgr2=0x7fece809e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:22:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece6d9d700 1 --2- 172.21.15.89:0/2426142940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece809de00 0x7fece809e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece6d9d700 1 -- 172.21.15.89:0/2426142940 >> v1:172.21.15.89:6789/0 conn(0x7fece806a770 legacy=0x7fece809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece6d9d700 1 -- 172.21.15.89:0/2426142940 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece809e920 con 0x7fece813d1f0 2023-09-30T22:22:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece559a700 1 -- 172.21.15.89:0/2426142940 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825858631 0 0) 0x7fece809a810 con 0x7fece806a770 2023-09-30T22:22:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece5d9b700 1 --2- 172.21.15.89:0/2426142940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece809de00 0x7fece809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:22:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece6d9d700 1 --2- 172.21.15.89:0/2426142940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece813d1f0 0x7fece813f5d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fece0009d90 tx=0x7fece000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad5b074bb131a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece559a700 1 -- 172.21.15.89:0/2426142940 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece000e070 con 0x7fece813d1f0 2023-09-30T22:22:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece559a700 1 -- 172.21.15.89:0/2426142940 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece000bb20 con 0x7fece813d1f0 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.103+0000 7fece559a700 1 -- 172.21.15.89:0/2426142940 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0012430 con 0x7fece813d1f0 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.104+0000 7fececcd1700 1 -- 172.21.15.89:0/2426142940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece813d1f0 msgr2=0x7fece813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.104+0000 7fececcd1700 1 --2- 172.21.15.89:0/2426142940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece813d1f0 0x7fece813f5d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fece0009d90 tx=0x7fece000b330 comp rx=0 tx=0).stop 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.104+0000 7fececcd1700 1 -- 172.21.15.89:0/2426142940 shutdown_connections 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.104+0000 7fececcd1700 1 --2- 172.21.15.89:0/2426142940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece813d1f0 0x7fece813f5d0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.104+0000 7fececcd1700 1 --2- 172.21.15.89:0/2426142940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece809de00 0x7fece809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.104+0000 7fececcd1700 1 -- 172.21.15.89:0/2426142940 >> 172.21.15.89:0/2426142940 conn(0x7fece8066f50 msgr2=0x7fece809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.104+0000 7fececcd1700 1 -- 172.21.15.89:0/2426142940 shutdown_connections 2023-09-30T22:22:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.104+0000 7fececcd1700 1 -- 172.21.15.89:0/2426142940 wait complete. 2023-09-30T22:22:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fececcd1700 1 Processor -- start 2023-09-30T22:22:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fececcd1700 1 -- start start 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fececcd1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece806a770 0x7fece813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fececcd1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece809de00 0x7fece813ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fececcd1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece813d1f0 0x7fece813b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fececcd1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece8139b60 con 0x7fece806a770 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fececcd1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece8139650 con 0x7fece809de00 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fececcd1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fece81392f0 con 0x7fece813d1f0 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fece659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece806a770 0x7fece813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fece659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece806a770 0x7fece813a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40914/0 (socket says 172.21.15.89:40914) 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.105+0000 7fece659c700 1 -- 172.21.15.89:0/3533262908 learned_addr learned my addr 172.21.15.89:0/3533262908 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:22:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fece6d9d700 1 --2- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece813d1f0 0x7fece813b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fece659c700 1 -- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece813d1f0 msgr2=0x7fece813b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fece659c700 1 --2- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece813d1f0 0x7fece813b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fece659c700 1 -- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece809de00 msgr2=0x7fece813ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fece659c700 1 --2- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece809de00 0x7fece813ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fece659c700 1 -- 172.21.15.89:0/3533262908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc009550 con 0x7fece806a770 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fece659c700 1 --2- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece806a770 0x7fece813a740 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fecd8001eb0 tx=0x7fecd800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fecd6ffd700 1 -- 172.21.15.89:0/3533262908 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd80054e0 con 0x7fece806a770 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fecd6ffd700 1 -- 172.21.15.89:0/3533262908 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd8003680 con 0x7fece806a770 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece0009a70 con 0x7fece806a770 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fecd6ffd700 1 -- 172.21.15.89:0/3533262908 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd80039f0 con 0x7fece806a770 2023-09-30T22:22:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.106+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece8138ea0 con 0x7fece806a770 2023-09-30T22:22:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.107+0000 7fecd6ffd700 1 -- 172.21.15.89:0/3533262908 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fecd8003b50 con 0x7fece806a770 2023-09-30T22:22:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.107+0000 7fecd6ffd700 1 -- 172.21.15.89:0/3533262908 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fecd803eca0 con 0x7fece806a770 2023-09-30T22:22:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.107+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece813b7e0 con 0x7fece806a770 2023-09-30T22:22:59.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.116+0000 7fecd6ffd700 1 -- 172.21.15.89:0/3533262908 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fecd8017020 con 0x7fece806a770 2023-09-30T22:22:59.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.204+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f68e809e980 con 0x7f68e806acb0 2023-09-30T22:22:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.205+0000 7f68deffd700 1 -- 172.21.15.89:0/2989357780 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68e4015030 con 0x7f68e806acb0 2023-09-30T22:22:59.206 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 msgr2=0x7f68e8132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e8132300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f68e40090b0 tx=0x7f68e4004750 comp rx=0 tx=0).stop 2023-09-30T22:22:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 shutdown_connections 2023-09-30T22:22:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68e81316c0 0x7f68e8132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68e806acb0 0x7f68e8132300 secure :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f68e40090b0 tx=0x7f68e4004750 comp rx=0 tx=0).stop 2023-09-30T22:22:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 --2- 172.21.15.89:0/2989357780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68e809d9f0 0x7f68e8132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 >> 172.21.15.89:0/2989357780 conn(0x7f68e8066f50 msgr2=0x7f68e8130150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 shutdown_connections 2023-09-30T22:22:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.208+0000 7f68f043a700 1 -- 172.21.15.89:0/2989357780 wait complete. 2023-09-30T22:22:59.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:22:59.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.239+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0f7c122ac0 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.240+0000 7f0f63fff700 1 -- 172.21.15.89:0/2343868277 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f7c122ac0 con 0x7f0f7c09e1d0 2023-09-30T22:22:59.241 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 msgr2=0x7f0f7c1c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 --2- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 0x7f0f7c1c6590 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0f70016000 tx=0x7f0f70004750 comp rx=0 tx=0).stop 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 shutdown_connections 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 --2- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0f7c135e50 0x7f0f7c1c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 --2- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0f7c09d6f0 0x7f0f7c1c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 --2- 172.21.15.89:0/2343868277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f7c09e1d0 0x7f0f7c1c6590 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 >> 172.21.15.89:0/2343868277 conn(0x7f0f7c066f50 msgr2=0x7f0f7c134840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 shutdown_connections 2023-09-30T22:22:59.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.242+0000 7f0f81b8a700 1 -- 172.21.15.89:0/2343868277 wait complete. 2023-09-30T22:22:59.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:22:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.296+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f10c80747f0 con 0x7f10c809d630 2023-09-30T22:22:59.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.296+0000 7f10bdffb700 1 -- 172.21.15.89:0/670587862 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10c4017070 con 0x7f10c809d630 2023-09-30T22:22:59.297 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:22:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c809d630 msgr2=0x7f10c81336f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c809d630 0x7f10c81336f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f10c400e910 tx=0x7f10c400ec20 comp rx=0 tx=0).stop 2023-09-30T22:22:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 shutdown_connections 2023-09-30T22:22:59.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10c8138d80 0x7f10c8134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c809d630 0x7f10c81336f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 --2- 172.21.15.89:0/670587862 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10c80a0a30 0x7f10c8133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 >> 172.21.15.89:0/670587862 conn(0x7f10c80a3050 msgr2=0x7f10c809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 shutdown_connections 2023-09-30T22:22:59.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.298+0000 7f10cf3dd700 1 -- 172.21.15.89:0/670587862 wait complete. 2023-09-30T22:22:59.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:22:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.310+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fece8071700 con 0x7fece806a770 2023-09-30T22:22:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.311+0000 7fecd6ffd700 1 -- 172.21.15.89:0/3533262908 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fecd8012400 con 0x7fece806a770 2023-09-30T22:22:59.312 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:22:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.312+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece806a770 msgr2=0x7fece813a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.312+0000 7fececcd1700 1 --2- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece806a770 0x7fece813a740 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fecd8001eb0 tx=0x7fecd800c800 comp rx=0 tx=0).stop 2023-09-30T22:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.313+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 shutdown_connections 2023-09-30T22:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.313+0000 7fececcd1700 1 --2- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece813d1f0 0x7fece813b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.313+0000 7fececcd1700 1 --2- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece806a770 0x7fece813a740 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.313+0000 7fececcd1700 1 --2- 172.21.15.89:0/3533262908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece809de00 0x7fece813ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.313+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 >> 172.21.15.89:0/3533262908 conn(0x7fece8066f50 msgr2=0x7fece806e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.313+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 shutdown_connections 2023-09-30T22:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:22:59.313+0000 7fececcd1700 1 -- 172.21.15.89:0/3533262908 wait complete. 2023-09-30T22:22:59.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:00.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:00.255 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:00.312 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:00.326 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:00.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.363+0000 7f036dd03700 1 Processor -- start 2023-09-30T22:23:00.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.363+0000 7f036dd03700 1 -- start start 2023-09-30T22:23:00.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f036dd03700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368134630 0x7f0368136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f036dd03700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368136f50 0x7f0368139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f036dd03700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036809b4f0 con 0x7f0368131810 2023-09-30T22:23:00.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f036dd03700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03681313c0 con 0x7f0368134630 2023-09-30T22:23:00.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f036dd03700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f036806e8c0 con 0x7f0368136f50 2023-09-30T22:23:00.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f03677fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0368131810 0x7f0368131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35036/0 (socket says 172.21.15.89:35036) 2023-09-30T22:23:00.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f03677fe700 1 -- 172.21.15.89:0/2110925482 learned_addr learned my addr 172.21.15.89:0/2110925482 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:00.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f0367fff700 1 --2- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368136f50 0x7f0368139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f0366ffd700 1 --2- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368134630 0x7f0368136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f03667fc700 1 -- 172.21.15.89:0/2110925482 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801714241 0 0) 0x7f036809b4f0 con 0x7f0368131810 2023-09-30T22:23:00.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f03667fc700 1 -- 172.21.15.89:0/2110925482 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03500035b0 con 0x7f0368131810 2023-09-30T22:23:00.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f0367fff700 1 -- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368134630 msgr2=0x7f0368136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:00.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f0367fff700 1 --2- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368134630 0x7f0368136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f0367fff700 1 -- 172.21.15.89:0/2110925482 >> v1:172.21.15.89:6789/0 conn(0x7f0368131810 legacy=0x7f0368131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f0367fff700 1 -- 172.21.15.89:0/2110925482 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0368071d20 con 0x7f0368136f50 2023-09-30T22:23:00.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.364+0000 7f0366ffd700 1 --2- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368134630 0x7f0368136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:00.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.365+0000 7f0367fff700 1 --2- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368136f50 0x7f0368139330 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0360009dc0 tx=0x7f036000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e5d4255dff8adc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:00.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.365+0000 7f03667fc700 1 -- 172.21.15.89:0/2110925482 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036000e070 con 0x7f0368136f50 2023-09-30T22:23:00.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.365+0000 7f03667fc700 1 -- 172.21.15.89:0/2110925482 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036000bb20 con 0x7f0368136f50 2023-09-30T22:23:00.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.365+0000 7f03667fc700 1 -- 172.21.15.89:0/2110925482 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360012430 con 0x7f0368136f50 2023-09-30T22:23:00.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.366+0000 7f036dd03700 1 -- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368136f50 msgr2=0x7f0368139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.366+0000 7f036dd03700 1 --2- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368136f50 0x7f0368139330 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0360009dc0 tx=0x7f036000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:00.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.366+0000 7f036dd03700 1 -- 172.21.15.89:0/2110925482 shutdown_connections 2023-09-30T22:23:00.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.366+0000 7f036dd03700 1 --2- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368136f50 0x7f0368139330 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.366+0000 7f036dd03700 1 --2- 172.21.15.89:0/2110925482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368134630 0x7f0368136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.366+0000 7f036dd03700 1 -- 172.21.15.89:0/2110925482 >> 172.21.15.89:0/2110925482 conn(0x7f03680a2aa0 msgr2=0x7f03680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:00.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.366+0000 7f036dd03700 1 -- 172.21.15.89:0/2110925482 shutdown_connections 2023-09-30T22:23:00.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.366+0000 7f036dd03700 1 -- 172.21.15.89:0/2110925482 wait complete. 2023-09-30T22:23:00.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.368+0000 7f036dd03700 1 Processor -- start 2023-09-30T22:23:00.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.368+0000 7f036dd03700 1 -- start start 2023-09-30T22:23:00.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.368+0000 7f036dd03700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368131810 0x7f0368118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.368+0000 7f036dd03700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0368134630 0x7f0368119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.368+0000 7f036dd03700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368136f50 0x7f0368119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.368+0000 7f036dd03700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036811c3c0 con 0x7f0368134630 2023-09-30T22:23:00.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.368+0000 7f036dd03700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680a1eb0 con 0x7f0368136f50 2023-09-30T22:23:00.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.368+0000 7f036dd03700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f036806eb00 con 0x7f0368131810 2023-09-30T22:23:00.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f0366ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0368134630 0x7f0368119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f03677fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368131810 0x7f0368118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f0366ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0368134630 0x7f0368119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46428/0 (socket says 172.21.15.89:46428) 2023-09-30T22:23:00.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f0366ffd700 1 -- 172.21.15.89:0/1480689655 learned_addr learned my addr 172.21.15.89:0/1480689655 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:00.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f0367fff700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368136f50 0x7f0368119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f03677fe700 1 -- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368136f50 msgr2=0x7f0368119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f03677fe700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368136f50 0x7f0368119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f03677fe700 1 -- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0368134630 msgr2=0x7f0368119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f03677fe700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0368134630 0x7f0368119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f03677fe700 1 -- 172.21.15.89:0/1480689655 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0094a0 con 0x7f0368131810 2023-09-30T22:23:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f0366ffd700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0368134630 0x7f0368119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.369+0000 7f03677fe700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368131810 0x7f0368118cd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f035800c490 tx=0x7f035800c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:00.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.370+0000 7f034ffff700 1 -- 172.21.15.89:0/1480689655 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358005d70 con 0x7f0368131810 2023-09-30T22:23:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.370+0000 7f034ffff700 1 -- 172.21.15.89:0/1480689655 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03580070d0 con 0x7f0368131810 2023-09-30T22:23:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.370+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0360009a70 con 0x7f0368131810 2023-09-30T22:23:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.370+0000 7f034ffff700 1 -- 172.21.15.89:0/1480689655 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358007440 con 0x7f0368131810 2023-09-30T22:23:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.370+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036811ce30 con 0x7f0368131810 2023-09-30T22:23:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.371+0000 7f034ffff700 1 -- 172.21.15.89:0/1480689655 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03580075a0 con 0x7f0368131810 2023-09-30T22:23:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.371+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0368074ea0 con 0x7f0368131810 2023-09-30T22:23:00.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.371+0000 7f034ffff700 1 -- 172.21.15.89:0/1480689655 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f035803e7f0 con 0x7f0368131810 2023-09-30T22:23:00.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.380+0000 7f034ffff700 1 -- 172.21.15.89:0/1480689655 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0358017020 con 0x7f0368131810 2023-09-30T22:23:00.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.410+0000 7f529db35700 1 Processor -- start 2023-09-30T22:23:00.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.410+0000 7f529db35700 1 -- start start 2023-09-30T22:23:00.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f529db35700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 0x7f529809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f529db35700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f529809e000 0x7f5298130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f529db35700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529809f130 con 0x7f529809e410 2023-09-30T22:23:00.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f529db35700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529809f520 con 0x7f529809e000 2023-09-30T22:23:00.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f529db35700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529809e930 con 0x7f529809d6f0 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f5297fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f529809e410 0x7f5298130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35040/0 (socket says 172.21.15.89:35040) 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f5297fff700 1 -- 172.21.15.89:0/3048632525 learned_addr learned my addr 172.21.15.89:0/3048632525 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f52977fe700 1 --2- 172.21.15.89:0/3048632525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 0x7f529809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f5296ffd700 1 --2- 172.21.15.89:0/3048632525 >> v2:172.21.15.89:3300/0 conn(0x7f529809e000 0x7f5298130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.411+0000 7f52967fc700 1 -- 172.21.15.89:0/3048632525 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524661109 0 0) 0x7f529809f130 con 0x7f529809e410 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52967fc700 1 -- 172.21.15.89:0/3048632525 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52800035b0 con 0x7f529809e410 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52977fe700 1 -- 172.21.15.89:0/3048632525 >> v2:172.21.15.89:3300/0 conn(0x7f529809e000 msgr2=0x7f5298130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52977fe700 1 --2- 172.21.15.89:0/3048632525 >> v2:172.21.15.89:3300/0 conn(0x7f529809e000 0x7f5298130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52977fe700 1 -- 172.21.15.89:0/3048632525 >> v1:172.21.15.89:6789/0 conn(0x7f529809e410 legacy=0x7f5298130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52977fe700 1 -- 172.21.15.89:0/3048632525 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529809b2d0 con 0x7f529809d6f0 2023-09-30T22:23:00.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f5296ffd700 1 --2- 172.21.15.89:0/3048632525 >> v2:172.21.15.89:3300/0 conn(0x7f529809e000 0x7f5298130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:00.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52977fe700 1 --2- 172.21.15.89:0/3048632525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 0x7f529809dac0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5288009840 tx=0x7f5288009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5ffb51675bb491 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:00.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52967fc700 1 -- 172.21.15.89:0/3048632525 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011830 con 0x7f529809d6f0 2023-09-30T22:23:00.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52967fc700 1 -- 172.21.15.89:0/3048632525 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288011990 con 0x7f529809d6f0 2023-09-30T22:23:00.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.412+0000 7f52967fc700 1 -- 172.21.15.89:0/3048632525 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011d00 con 0x7f529809d6f0 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.413+0000 7f529db35700 1 -- 172.21.15.89:0/3048632525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 msgr2=0x7f529809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.413+0000 7f529db35700 1 --2- 172.21.15.89:0/3048632525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 0x7f529809dac0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5288009840 tx=0x7f5288009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.413+0000 7f529db35700 1 -- 172.21.15.89:0/3048632525 shutdown_connections 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.413+0000 7f529db35700 1 --2- 172.21.15.89:0/3048632525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 0x7f529809dac0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.413+0000 7f529db35700 1 --2- 172.21.15.89:0/3048632525 >> v2:172.21.15.89:3300/0 conn(0x7f529809e000 0x7f5298130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.413+0000 7f529db35700 1 -- 172.21.15.89:0/3048632525 >> 172.21.15.89:0/3048632525 conn(0x7f5298066f50 msgr2=0x7f529809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.413+0000 7f529db35700 1 -- 172.21.15.89:0/3048632525 shutdown_connections 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.413+0000 7f529db35700 1 -- 172.21.15.89:0/3048632525 wait complete. 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.414+0000 7f529db35700 1 Processor -- start 2023-09-30T22:23:00.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.415+0000 7f529db35700 1 -- start start 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.415+0000 7f529db35700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 0x7f52981c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.415+0000 7f529db35700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f529809e000 0x7f52981c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.415+0000 7f529db35700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f529809e410 0x7f52981c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.415+0000 7f529db35700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529809c700 con 0x7f529809e000 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.415+0000 7f529db35700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5298132520 con 0x7f529809d6f0 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.415+0000 7f529db35700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5298133190 con 0x7f529809e410 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5297fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f529809e410 0x7f52981c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f529809e000 0x7f52981c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f529809e000 0x7f52981c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46448/0 (socket says 172.21.15.89:46448) 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5297fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f529809e410 0x7f52981c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44494/0 (socket says 172.21.15.89:44494) 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 -- 172.21.15.89:0/2089078237 learned_addr learned my addr 172.21.15.89:0/2089078237 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:00.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 -- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f529809e410 msgr2=0x7f52981c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 --2- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f529809e410 0x7f52981c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 -- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 msgr2=0x7f52981c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 --2- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 0x7f52981c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 -- 172.21.15.89:0/2089078237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0094a0 con 0x7f529809e000 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f5296ffd700 1 --2- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f529809e000 0x7f52981c62c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f528c00ebb0 tx=0x7f528c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f527ffff700 1 -- 172.21.15.89:0/2089078237 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00c850 con 0x7f529809e000 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f527ffff700 1 -- 172.21.15.89:0/2089078237 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c00c9b0 con 0x7f529809e000 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52880094a0 con 0x7f529809e000 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f527ffff700 1 -- 172.21.15.89:0/2089078237 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00cd20 con 0x7f529809e000 2023-09-30T22:23:00.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.416+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5298131820 con 0x7f529809e000 2023-09-30T22:23:00.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.417+0000 7f527ffff700 1 -- 172.21.15.89:0/2089078237 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f528c013470 con 0x7f529809e000 2023-09-30T22:23:00.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.418+0000 7f527ffff700 1 -- 172.21.15.89:0/2089078237 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f528c03f810 con 0x7f529809e000 2023-09-30T22:23:00.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.418+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52780052e0 con 0x7f529809e000 2023-09-30T22:23:00.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.423+0000 7f527ffff700 1 -- 172.21.15.89:0/2089078237 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f528c0232f0 con 0x7f529809e000 2023-09-30T22:23:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.448+0000 7f7a4c09a700 1 Processor -- start 2023-09-30T22:23:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.448+0000 7f7a4c09a700 1 -- start start 2023-09-30T22:23:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4c09a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a44131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4c09a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a4406ae80 0x7f7a44139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4c09a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4409e130 con 0x7f7a4409d9f0 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4c09a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a441312b0 con 0x7f7a4406ae80 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4c09a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4409f6e0 con 0x7f7a441316c0 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4a637700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a4406ae80 0x7f7a44139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4a637700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a4406ae80 0x7f7a44139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46464/0 (socket says 172.21.15.89:46464) 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4a637700 1 -- 172.21.15.89:0/1846316351 learned_addr learned my addr 172.21.15.89:0/1846316351 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a49e36700 1 --2- 172.21.15.89:0/1846316351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a44131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4a637700 1 -- 172.21.15.89:0/1846316351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 msgr2=0x7f7a44131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.449+0000 7f7a4a637700 1 --2- 172.21.15.89:0/1846316351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a44131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.450+0000 7f7a4a637700 1 -- 172.21.15.89:0/1846316351 >> v1:172.21.15.89:6789/0 conn(0x7f7a4409d9f0 legacy=0x7f7a4406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.450+0000 7f7a4a637700 1 -- 172.21.15.89:0/1846316351 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4409e760 con 0x7f7a4406ae80 2023-09-30T22:23:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.450+0000 7f7a4a637700 1 --2- 172.21.15.89:0/1846316351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 0x7f7a44139550 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009e50 tx=0x7f7a3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e333f7c644d3ac6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.450+0000 7f7a48e34700 1 -- 172.21.15.89:0/1846316351 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00e070 con 0x7f7a4406ae80 2023-09-30T22:23:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.450+0000 7f7a48e34700 1 -- 172.21.15.89:0/1846316351 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c003a20 con 0x7f7a4406ae80 2023-09-30T22:23:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.450+0000 7f7a48e34700 1 -- 172.21.15.89:0/1846316351 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c003dd0 con 0x7f7a4406ae80 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.451+0000 7f7a4c09a700 1 -- 172.21.15.89:0/1846316351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 msgr2=0x7f7a44139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.451+0000 7f7a4c09a700 1 --2- 172.21.15.89:0/1846316351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 0x7f7a44139550 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009e50 tx=0x7f7a3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.451+0000 7f7a4c09a700 1 -- 172.21.15.89:0/1846316351 shutdown_connections 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.451+0000 7f7a4c09a700 1 --2- 172.21.15.89:0/1846316351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 0x7f7a44139550 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.451+0000 7f7a4c09a700 1 --2- 172.21.15.89:0/1846316351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a44131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.451+0000 7f7a4c09a700 1 -- 172.21.15.89:0/1846316351 >> 172.21.15.89:0/1846316351 conn(0x7f7a44066f50 msgr2=0x7f7a4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.451+0000 7f7a4c09a700 1 -- 172.21.15.89:0/1846316351 shutdown_connections 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.451+0000 7f7a4c09a700 1 -- 172.21.15.89:0/1846316351 wait complete. 2023-09-30T22:23:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.452+0000 7f7a4c09a700 1 Processor -- start 2023-09-30T22:23:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.452+0000 7f7a4c09a700 1 -- start start 2023-09-30T22:23:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a4c09a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 0x7f7a4412b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a4c09a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a4409d9f0 0x7f7a4406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a4c09a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a4406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a4c09a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440724d0 con 0x7f7a4406ae80 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a4c09a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a44071500 con 0x7f7a441316c0 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a4c09a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4413a590 con 0x7f7a4409d9f0 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a4409d9f0 0x7f7a4406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a4409d9f0 0x7f7a4406e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44510/0 (socket says 172.21.15.89:44510) 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 -- 172.21.15.89:0/9657436 learned_addr learned my addr 172.21.15.89:0/9657436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49e36700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 0x7f7a4412b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a4a637700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a4406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 -- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 msgr2=0x7f7a4406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a4406ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 -- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 msgr2=0x7f7a4412b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 0x7f7a4412b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 -- 172.21.15.89:0/9657436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c009b30 con 0x7f7a4409d9f0 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49e36700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 0x7f7a4412b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a4a637700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a4406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.453+0000 7f7a49635700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a4409d9f0 0x7f7a4406e8e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400c7f0 tx=0x7f7a3400cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.454+0000 7f7a3a7fc700 1 -- 172.21.15.89:0/9657436 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34005480 con 0x7f7a4409d9f0 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.454+0000 7f7a3a7fc700 1 -- 172.21.15.89:0/9657436 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a34003680 con 0x7f7a4409d9f0 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.454+0000 7f7a3a7fc700 1 -- 172.21.15.89:0/9657436 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34003a10 con 0x7f7a4409d9f0 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.454+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a44072690 con 0x7f7a4409d9f0 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.454+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4412e3a0 con 0x7f7a4409d9f0 2023-09-30T22:23:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.455+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a44074ea0 con 0x7f7a4409d9f0 2023-09-30T22:23:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.455+0000 7f7a3a7fc700 1 -- 172.21.15.89:0/9657436 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a34003b70 con 0x7f7a4409d9f0 2023-09-30T22:23:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.455+0000 7f7a3a7fc700 1 -- 172.21.15.89:0/9657436 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7a3403f440 con 0x7f7a4409d9f0 2023-09-30T22:23:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.460+0000 7f7a3a7fc700 1 -- 172.21.15.89:0/9657436 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a3400a950 con 0x7f7a4409d9f0 2023-09-30T22:23:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d99ffb700 1 Processor -- start 2023-09-30T22:23:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d99ffb700 1 -- start start 2023-09-30T22:23:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d99ffb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d9406a770 0x7f1d9406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d99ffb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1d94131d10 0x7f1d94139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d99ffb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9409ece0 con 0x7f1d9406b080 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d99ffb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9409e760 con 0x7f1d94131d10 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d99ffb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9409e040 con 0x7f1d9406a770 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d937fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d9406a770 0x7f1d9406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d937fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d9406a770 0x7f1d9406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44522/0 (socket says 172.21.15.89:44522) 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.524+0000 7f1d937fe700 1 -- 172.21.15.89:0/1298253897 learned_addr learned my addr 172.21.15.89:0/1298253897 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.525+0000 7f1d937fe700 1 -- 172.21.15.89:0/1298253897 >> v2:172.21.15.89:3300/0 conn(0x7f1d94131d10 msgr2=0x7f1d94139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.525+0000 7f1d937fe700 1 --2- 172.21.15.89:0/1298253897 >> v2:172.21.15.89:3300/0 conn(0x7f1d94131d10 0x7f1d94139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.525+0000 7f1d937fe700 1 -- 172.21.15.89:0/1298253897 >> v1:172.21.15.89:6789/0 conn(0x7f1d9406b080 legacy=0x7f1d94131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.525+0000 7f1d937fe700 1 -- 172.21.15.89:0/1298253897 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9409f630 con 0x7f1d9406a770 2023-09-30T22:23:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.525+0000 7f1d937fe700 1 --2- 172.21.15.89:0/1298253897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d9406a770 0x7f1d9406ab40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1d840098f0 tx=0x7f1d84009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0f8b2716b366c07 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.525+0000 7f1d927fc700 1 -- 172.21.15.89:0/1298253897 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84004230 con 0x7f1d9406a770 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.525+0000 7f1d927fc700 1 -- 172.21.15.89:0/1298253897 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d84004390 con 0x7f1d9406a770 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.525+0000 7f1d927fc700 1 -- 172.21.15.89:0/1298253897 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840036a0 con 0x7f1d9406a770 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.526+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1298253897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d9406a770 msgr2=0x7f1d9406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.526+0000 7f1d99ffb700 1 --2- 172.21.15.89:0/1298253897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d9406a770 0x7f1d9406ab40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1d840098f0 tx=0x7f1d84009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.526+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1298253897 shutdown_connections 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.526+0000 7f1d99ffb700 1 --2- 172.21.15.89:0/1298253897 >> v2:172.21.15.89:3300/0 conn(0x7f1d94131d10 0x7f1d94139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.526+0000 7f1d99ffb700 1 --2- 172.21.15.89:0/1298253897 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d9406a770 0x7f1d9406ab40 secure :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1d840098f0 tx=0x7f1d84009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.526+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1298253897 >> 172.21.15.89:0/1298253897 conn(0x7f1d94066f50 msgr2=0x7f1d9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.526+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1298253897 shutdown_connections 2023-09-30T22:23:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.526+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1298253897 wait complete. 2023-09-30T22:23:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.527+0000 7f1d99ffb700 1 Processor -- start 2023-09-30T22:23:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d99ffb700 1 -- start start 2023-09-30T22:23:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d99ffb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9406b080 0x7f1d941c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d99ffb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94131d10 0x7f1d941c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d99ffb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d941c6c50 0x7f1d9412ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d99ffb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9411fdf0 con 0x7f1d941c6c50 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d99ffb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9411e1c0 con 0x7f1d9406b080 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d99ffb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9411fb30 con 0x7f1d94131d10 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d92ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94131d10 0x7f1d941c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.528+0000 7f1d93fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d941c6c50 0x7f1d9412ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d92ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94131d10 0x7f1d941c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44524/0 (socket says 172.21.15.89:44524) 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d92ffd700 1 -- 172.21.15.89:0/1861495771 learned_addr learned my addr 172.21.15.89:0/1861495771 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d92ffd700 1 -- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9406b080 msgr2=0x7f1d941c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d92ffd700 1 --2- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9406b080 0x7f1d941c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d92ffd700 1 -- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d941c6c50 msgr2=0x7f1d9412ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d92ffd700 1 --2- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d941c6c50 0x7f1d9412ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d92ffd700 1 -- 172.21.15.89:0/1861495771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009550 con 0x7f1d94131d10 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d93fff700 1 --2- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d941c6c50 0x7f1d9412ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d92ffd700 1 --2- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94131d10 0x7f1d941c6710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800c500 tx=0x7f1d8800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d7bfff700 1 -- 172.21.15.89:0/1861495771 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880054e0 con 0x7f1d94131d10 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d7bfff700 1 -- 172.21.15.89:0/1861495771 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d88003680 con 0x7f1d94131d10 2023-09-30T22:23:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d7bfff700 1 -- 172.21.15.89:0/1861495771 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88003a10 con 0x7f1d94131d10 2023-09-30T22:23:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9411eec0 con 0x7f1d94131d10 2023-09-30T22:23:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.529+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d94131060 con 0x7f1d94131d10 2023-09-30T22:23:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.530+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d94074ea0 con 0x7f1d94131d10 2023-09-30T22:23:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.531+0000 7f1d7bfff700 1 -- 172.21.15.89:0/1861495771 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d88003b70 con 0x7f1d94131d10 2023-09-30T22:23:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.531+0000 7f1d7bfff700 1 -- 172.21.15.89:0/1861495771 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d8803ee50 con 0x7f1d94131d10 2023-09-30T22:23:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.535+0000 7f1d7bfff700 1 -- 172.21.15.89:0/1861495771 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d88006820 con 0x7f1d94131d10 2023-09-30T22:23:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.584+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f036806b9e0 con 0x7f0368131810 2023-09-30T22:23:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.585+0000 7f034ffff700 1 -- 172.21.15.89:0/1480689655 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0358018af0 con 0x7f0368131810 2023-09-30T22:23:00.587 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368131810 msgr2=0x7f0368118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368131810 0x7f0368118cd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f035800c490 tx=0x7f035800c7a0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 shutdown_connections 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0368136f50 0x7f0368119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0368131810 0x7f0368118cd0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 --2- 172.21.15.89:0/1480689655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0368134630 0x7f0368119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 >> 172.21.15.89:0/1480689655 conn(0x7f03680a2aa0 msgr2=0x7f03680a09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 shutdown_connections 2023-09-30T22:23:00.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.587+0000 7f036dd03700 1 -- 172.21.15.89:0/1480689655 wait complete. 2023-09-30T22:23:00.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:00.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.656+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7a4409e130 con 0x7f7a4409d9f0 2023-09-30T22:23:00.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.656+0000 7f7a3a7fc700 1 -- 172.21.15.89:0/9657436 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a34017020 con 0x7f7a4409d9f0 2023-09-30T22:23:00.657 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a4409d9f0 msgr2=0x7f7a4406e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a4409d9f0 0x7f7a4406e8e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400c7f0 tx=0x7f7a3400cbb0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 shutdown_connections 2023-09-30T22:23:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a441316c0 0x7f7a4406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a4406ae80 0x7f7a4412b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 --2- 172.21.15.89:0/9657436 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7a4409d9f0 0x7f7a4406e8e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 >> 172.21.15.89:0/9657436 conn(0x7f7a44066f50 msgr2=0x7f7a441301b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:00.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 shutdown_connections 2023-09-30T22:23:00.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.658+0000 7f7a4c09a700 1 -- 172.21.15.89:0/9657436 wait complete. 2023-09-30T22:23:00.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.694+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5278005590 con 0x7f529809e000 2023-09-30T22:23:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.694+0000 7f527ffff700 1 -- 172.21.15.89:0/2089078237 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f528c01bb60 con 0x7f529809e000 2023-09-30T22:23:00.695 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:00.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.696+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f529809e000 msgr2=0x7f52981c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.696+0000 7f529db35700 1 --2- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f529809e000 0x7f52981c62c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f528c00ebb0 tx=0x7f528c00eec0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.696+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 shutdown_connections 2023-09-30T22:23:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.696+0000 7f529db35700 1 --2- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f529809e410 0x7f52981c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.696+0000 7f529db35700 1 --2- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f529809d6f0 0x7f52981c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.696+0000 7f529db35700 1 --2- 172.21.15.89:0/2089078237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f529809e000 0x7f52981c62c0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.696+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 >> 172.21.15.89:0/2089078237 conn(0x7f5298066f50 msgr2=0x7f529812a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.697+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 shutdown_connections 2023-09-30T22:23:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.697+0000 7f529db35700 1 -- 172.21.15.89:0/2089078237 wait complete. 2023-09-30T22:23:00.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:00.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.762+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1d9406c360 con 0x7f1d94131d10 2023-09-30T22:23:00.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.762+0000 7f1d7bfff700 1 -- 172.21.15.89:0/1861495771 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d88017030 con 0x7f1d94131d10 2023-09-30T22:23:00.763 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:00.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.764+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94131d10 msgr2=0x7f1d941c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.764+0000 7f1d99ffb700 1 --2- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94131d10 0x7f1d941c6710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800c500 tx=0x7f1d8800c810 comp rx=0 tx=0).stop 2023-09-30T22:23:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.764+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 shutdown_connections 2023-09-30T22:23:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.764+0000 7f1d99ffb700 1 --2- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d941c6c50 0x7f1d9412ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.764+0000 7f1d99ffb700 1 --2- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9406b080 0x7f1d941c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.764+0000 7f1d99ffb700 1 --2- 172.21.15.89:0/1861495771 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94131d10 0x7f1d941c6710 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.764+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 >> 172.21.15.89:0/1861495771 conn(0x7f1d94066f50 msgr2=0x7f1d940a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.765+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 shutdown_connections 2023-09-30T22:23:00.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:00.765+0000 7f1d99ffb700 1 -- 172.21.15.89:0/1861495771 wait complete. 2023-09-30T22:23:00.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:01.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:01.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:01.710 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.735+0000 7f9e12369700 1 Processor -- start 2023-09-30T22:23:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.735+0000 7f9e12369700 1 -- start start 2023-09-30T22:23:01.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.735+0000 7f9e12369700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130540 0x7f9e0c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.735+0000 7f9e12369700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c130e50 0x7f9e0c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.735+0000 7f9e12369700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e0c06fb20 0x7f9e0c06ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.735+0000 7f9e12369700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0c09a390 con 0x7f9e0c06fb20 2023-09-30T22:23:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.735+0000 7f9e12369700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0716a0 con 0x7f9e0c130e50 2023-09-30T22:23:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.735+0000 7f9e12369700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c071290 con 0x7f9e0c130540 2023-09-30T22:23:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130540 0x7f9e0c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e10906700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e0c06fb20 0x7f9e0c06ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0b7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c130e50 0x7f9e0c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e10906700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9e0c06fb20 0x7f9e0c06ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46514/0 (socket says 172.21.15.89:46514) 2023-09-30T22:23:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130540 0x7f9e0c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44528/0 (socket says 172.21.15.89:44528) 2023-09-30T22:23:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e10906700 1 -- 172.21.15.89:0/3997783616 learned_addr learned my addr 172.21.15.89:0/3997783616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0b7fe700 1 -- 172.21.15.89:0/3997783616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130540 msgr2=0x7f9e0c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0b7fe700 1 --2- 172.21.15.89:0/3997783616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130540 0x7f9e0c130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0b7fe700 1 -- 172.21.15.89:0/3997783616 >> v2:172.21.15.89:3300/0 conn(0x7f9e0c06fb20 msgr2=0x7f9e0c06ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:01.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0b7fe700 1 --2- 172.21.15.89:0/3997783616 >> v2:172.21.15.89:3300/0 conn(0x7f9e0c06fb20 0x7f9e0c06ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0b7fe700 1 -- 172.21.15.89:0/3997783616 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c130340 con 0x7f9e0c130e50 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e10906700 1 --2- 172.21.15.89:0/3997783616 >> v2:172.21.15.89:3300/0 conn(0x7f9e0c06fb20 0x7f9e0c06ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0bfff700 1 --2- 172.21.15.89:0/3997783616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130540 0x7f9e0c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0b7fe700 1 --2- 172.21.15.89:0/3997783616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c130e50 0x7f9e0c06f5e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000dc90 tx=0x7f9e0000f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740951dbcf96fcdd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0affd700 1 -- 172.21.15.89:0/3997783616 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00012070 con 0x7f9e0c130e50 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0affd700 1 -- 172.21.15.89:0/3997783616 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0000d2e0 con 0x7f9e0c130e50 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.736+0000 7f9e0affd700 1 -- 172.21.15.89:0/3997783616 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000f8b0 con 0x7f9e0c130e50 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 -- 172.21.15.89:0/3997783616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c130e50 msgr2=0x7f9e0c06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 --2- 172.21.15.89:0/3997783616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c130e50 0x7f9e0c06f5e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000dc90 tx=0x7f9e0000f050 comp rx=0 tx=0).stop 2023-09-30T22:23:01.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 -- 172.21.15.89:0/3997783616 shutdown_connections 2023-09-30T22:23:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 --2- 172.21.15.89:0/3997783616 >> v2:172.21.15.89:3300/0 conn(0x7f9e0c06fb20 0x7f9e0c06ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 --2- 172.21.15.89:0/3997783616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130540 0x7f9e0c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 --2- 172.21.15.89:0/3997783616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c130e50 0x7f9e0c06f5e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 -- 172.21.15.89:0/3997783616 >> 172.21.15.89:0/3997783616 conn(0x7f9e0c0a2aa0 msgr2=0x7f9e0c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 -- 172.21.15.89:0/3997783616 shutdown_connections 2023-09-30T22:23:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.737+0000 7f9e12369700 1 -- 172.21.15.89:0/3997783616 wait complete. 2023-09-30T22:23:01.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.738+0000 7f9e12369700 1 Processor -- start 2023-09-30T22:23:01.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.738+0000 7f9e12369700 1 -- start start 2023-09-30T22:23:01.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e12369700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c06fb20 0x7f9e0c11a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e12369700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e0c130540 0x7f9e0c11ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e12369700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130e50 0x7f9e0c11b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e12369700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c11c330 con 0x7f9e0c130540 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e12369700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c11d580 con 0x7f9e0c06fb20 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e12369700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c11c910 con 0x7f9e0c130e50 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130e50 0x7f9e0c11b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130e50 0x7f9e0c11b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44538/0 (socket says 172.21.15.89:44538) 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 -- 172.21.15.89:0/4079413981 learned_addr learned my addr 172.21.15.89:0/4079413981 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e0b7fe700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e0c130540 0x7f9e0c11ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 -- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c06fb20 msgr2=0x7f9e0c11a720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c06fb20 0x7f9e0c11a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 -- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e0c130540 msgr2=0x7f9e0c11ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e0c130540 0x7f9e0c11ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 -- 172.21.15.89:0/4079413981 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e04009b30 con 0x7f9e0c130e50 2023-09-30T22:23:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e0b7fe700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e0c130540 0x7f9e0c11ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.739+0000 7f9e10906700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130e50 0x7f9e0c11b1a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400efc0 tx=0x7f9e04009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.740+0000 7f9e08ff9700 1 -- 172.21.15.89:0/4079413981 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04018070 con 0x7f9e0c130e50 2023-09-30T22:23:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.740+0000 7f9e08ff9700 1 -- 172.21.15.89:0/4079413981 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0400f040 con 0x7f9e0c130e50 2023-09-30T22:23:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.740+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc0094a0 con 0x7f9e0c130e50 2023-09-30T22:23:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.740+0000 7f9e08ff9700 1 -- 172.21.15.89:0/4079413981 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04003c40 con 0x7f9e0c130e50 2023-09-30T22:23:01.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.740+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0000d940 con 0x7f9e0c130e50 2023-09-30T22:23:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.740+0000 7f9e08ff9700 1 -- 172.21.15.89:0/4079413981 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e0400c9e0 con 0x7f9e0c130e50 2023-09-30T22:23:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.741+0000 7f9e08ff9700 1 -- 172.21.15.89:0/4079413981 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9e0403f8d0 con 0x7f9e0c130e50 2023-09-30T22:23:01.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.741+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0c0a0910 con 0x7f9e0c130e50 2023-09-30T22:23:01.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.746+0000 7f9e08ff9700 1 -- 172.21.15.89:0/4079413981 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e04012340 con 0x7f9e0c130e50 2023-09-30T22:23:01.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:01.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.811+0000 7f0a917c4700 1 Processor -- start 2023-09-30T22:23:01.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.811+0000 7f0a917c4700 1 -- start start 2023-09-30T22:23:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a917c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a917c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 0x7f0a8c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a917c4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0a8c09e410 0x7f0a8c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a917c4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8c06b460 con 0x7f0a8c09e410 2023-09-30T22:23:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a917c4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c09b120 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a917c4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c09a7e0 con 0x7f0a8c09e000 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a8affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a8a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 0x7f0a8c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a8affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58166/0 (socket says 172.21.15.89:58166) 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a8affd700 1 -- 172.21.15.89:0/4100237641 learned_addr learned my addr 172.21.15.89:0/4100237641 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.812+0000 7f0a8b7fe700 1 --2- 172.21.15.89:0/4100237641 >> v2:172.21.15.89:3300/0 conn(0x7f0a8c09e410 0x7f0a8c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a8affd700 1 -- 172.21.15.89:0/4100237641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 msgr2=0x7f0a8c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a8affd700 1 --2- 172.21.15.89:0/4100237641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 0x7f0a8c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a8affd700 1 -- 172.21.15.89:0/4100237641 >> v2:172.21.15.89:3300/0 conn(0x7f0a8c09e410 msgr2=0x7f0a8c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:01.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a8affd700 1 --2- 172.21.15.89:0/4100237641 >> v2:172.21.15.89:3300/0 conn(0x7f0a8c09e410 0x7f0a8c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a8affd700 1 -- 172.21.15.89:0/4100237641 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c09a500 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a8affd700 1 --2- 172.21.15.89:0/4100237641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c09dac0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009920 tx=0x7f0a7c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cd0ba3423a11d13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a89ffb700 1 -- 172.21.15.89:0/4100237641 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c011830 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a89ffb700 1 -- 172.21.15.89:0/4100237641 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c011990 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.813+0000 7f0a89ffb700 1 -- 172.21.15.89:0/4100237641 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c011d20 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 -- 172.21.15.89:0/4100237641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 msgr2=0x7f0a8c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 --2- 172.21.15.89:0/4100237641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c09dac0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009920 tx=0x7f0a7c009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 -- 172.21.15.89:0/4100237641 shutdown_connections 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 --2- 172.21.15.89:0/4100237641 >> v2:172.21.15.89:3300/0 conn(0x7f0a8c09e410 0x7f0a8c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 --2- 172.21.15.89:0/4100237641 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c09dac0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 --2- 172.21.15.89:0/4100237641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 0x7f0a8c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 -- 172.21.15.89:0/4100237641 >> 172.21.15.89:0/4100237641 conn(0x7f0a8c066f50 msgr2=0x7f0a8c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 -- 172.21.15.89:0/4100237641 shutdown_connections 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.814+0000 7f0a917c4700 1 -- 172.21.15.89:0/4100237641 wait complete. 2023-09-30T22:23:01.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.816+0000 7f0a917c4700 1 Processor -- start 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.816+0000 7f0a917c4700 1 -- start start 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.816+0000 7f0a917c4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c1a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.816+0000 7f0a917c4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 0x7f0a8c1a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.816+0000 7f0a917c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a8c09e410 0x7f0a8c1a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.816+0000 7f0a917c4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1a9a60 con 0x7f0a8c09e410 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.816+0000 7f0a917c4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1a8130 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.816+0000 7f0a917c4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1a91f0 con 0x7f0a8c09e000 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c1a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 0x7f0a8c1a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c1a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58172/0 (socket says 172.21.15.89:58172) 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 -- 172.21.15.89:0/1153611080 learned_addr learned my addr 172.21.15.89:0/1153611080 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8b7fe700 1 --2- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a8c09e410 0x7f0a8c1a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 -- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 msgr2=0x7f0a8c1a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 --2- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 0x7f0a8c1a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 -- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a8c09e410 msgr2=0x7f0a8c1a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 --2- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a8c09e410 0x7f0a8c1a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 -- 172.21.15.89:0/1153611080 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c009550 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a8affd700 1 --2- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c1a6020 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c000c00 tx=0x7f0a7c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a737fe700 1 -- 172.21.15.89:0/1153611080 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c004380 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a737fe700 1 -- 172.21.15.89:0/1153611080 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c01f070 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.817+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c09c710 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.818+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c0a3190 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.818+0000 7f0a737fe700 1 -- 172.21.15.89:0/1153611080 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0036a0 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.819+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c0052e0 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.820+0000 7f0a737fe700 1 -- 172.21.15.89:0/1153611080 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a7c021030 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.820+0000 7f0a737fe700 1 -- 172.21.15.89:0/1153611080 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a7c040d00 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.827+0000 7f0a737fe700 1 -- 172.21.15.89:0/1153611080 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a7c00b9d0 con 0x7f0a8c09d6f0 2023-09-30T22:23:01.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.866+0000 7f17046e1700 1 Processor -- start 2023-09-30T22:23:01.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.866+0000 7f17046e1700 1 -- start start 2023-09-30T22:23:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.867+0000 7f17046e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc0a0a30 0x7f16fc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.867+0000 7f17046e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.867+0000 7f17046e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f16fc133930 0x7f16fc133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.867+0000 7f17046e1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc0a2a70 con 0x7f16fc133930 2023-09-30T22:23:01.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.867+0000 7f17046e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0a01d0 con 0x7f16fc0a0a30 2023-09-30T22:23:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.867+0000 7f17046e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc09e2b0 con 0x7f16fc09d630 2023-09-30T22:23:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.867+0000 7f1701c7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1701c7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44572/0 (socket says 172.21.15.89:44572) 2023-09-30T22:23:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1701c7c700 1 -- 172.21.15.89:0/3159596009 learned_addr learned my addr 172.21.15.89:0/3159596009 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f170247d700 1 --2- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc0a0a30 0x7f16fc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1702c7e700 1 --2- 172.21.15.89:0/3159596009 >> v2:172.21.15.89:3300/0 conn(0x7f16fc133930 0x7f16fc133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1701c7c700 1 -- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc0a0a30 msgr2=0x7f16fc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1701c7c700 1 --2- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc0a0a30 0x7f16fc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1701c7c700 1 -- 172.21.15.89:0/3159596009 >> v2:172.21.15.89:3300/0 conn(0x7f16fc133930 msgr2=0x7f16fc133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1701c7c700 1 --2- 172.21.15.89:0/3159596009 >> v2:172.21.15.89:3300/0 conn(0x7f16fc133930 0x7f16fc133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1701c7c700 1 -- 172.21.15.89:0/3159596009 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc09dc50 con 0x7f16fc09d630 2023-09-30T22:23:01.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f170247d700 1 --2- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc0a0a30 0x7f16fc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:01.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f1701c7c700 1 --2- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc0a1340 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098f0 tx=0x7f16ec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=401bcf35c3138e52 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:01.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f170147b700 1 -- 172.21.15.89:0/3159596009 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec011830 con 0x7f16fc09d630 2023-09-30T22:23:01.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f170147b700 1 -- 172.21.15.89:0/3159596009 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec011990 con 0x7f16fc09d630 2023-09-30T22:23:01.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.868+0000 7f170147b700 1 -- 172.21.15.89:0/3159596009 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec011d20 con 0x7f16fc09d630 2023-09-30T22:23:01.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.869+0000 7f17046e1700 1 -- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 msgr2=0x7f16fc0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.869+0000 7f17046e1700 1 --2- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc0a1340 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098f0 tx=0x7f16ec009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:01.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.869+0000 7f17046e1700 1 -- 172.21.15.89:0/3159596009 shutdown_connections 2023-09-30T22:23:01.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.869+0000 7f17046e1700 1 --2- 172.21.15.89:0/3159596009 >> v2:172.21.15.89:3300/0 conn(0x7f16fc133930 0x7f16fc133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.869+0000 7f17046e1700 1 --2- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc0a0a30 0x7f16fc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.869+0000 7f17046e1700 1 --2- 172.21.15.89:0/3159596009 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc0a1340 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.869+0000 7f17046e1700 1 -- 172.21.15.89:0/3159596009 >> 172.21.15.89:0/3159596009 conn(0x7f16fc0a3050 msgr2=0x7f16fc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:01.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.873+0000 7f17046e1700 1 -- 172.21.15.89:0/3159596009 shutdown_connections 2023-09-30T22:23:01.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.873+0000 7f17046e1700 1 -- 172.21.15.89:0/3159596009 wait complete. 2023-09-30T22:23:01.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.874+0000 7f17046e1700 1 Processor -- start 2023-09-30T22:23:01.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.874+0000 7f17046e1700 1 -- start start 2023-09-30T22:23:01.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.874+0000 7f17046e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc1c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.874+0000 7f17046e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16fc0a0a30 0x7f16fc1c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.874+0000 7f17046e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc133930 0x7f16fc1c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.874+0000 7f17046e1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc1206b0 con 0x7f16fc0a0a30 2023-09-30T22:23:01.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.874+0000 7f17046e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc1204c0 con 0x7f16fc133930 2023-09-30T22:23:01.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.874+0000 7f17046e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc1223d0 con 0x7f16fc09d630 2023-09-30T22:23:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16fc0a0a30 0x7f16fc1c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16fc0a0a30 0x7f16fc1c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46552/0 (socket says 172.21.15.89:46552) 2023-09-30T22:23:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 -- 172.21.15.89:0/637589943 learned_addr learned my addr 172.21.15.89:0/637589943 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f170247d700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc1c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1702c7e700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc133930 0x7f16fc1c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 -- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 msgr2=0x7f16fc1c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc1c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 -- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc133930 msgr2=0x7f16fc1c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc133930 0x7f16fc1c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 -- 172.21.15.89:0/637589943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f80094a0 con 0x7f16fc0a0a30 2023-09-30T22:23:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f1701c7c700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16fc0a0a30 0x7f16fc1c8640 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000c00 tx=0x7f16ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f16f2ffd700 1 -- 172.21.15.89:0/637589943 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec004240 con 0x7f16fc0a0a30 2023-09-30T22:23:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f16f2ffd700 1 -- 172.21.15.89:0/637589943 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec01f070 con 0x7f16fc0a0a30 2023-09-30T22:23:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec009550 con 0x7f16fc0a0a30 2023-09-30T22:23:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.875+0000 7f16f2ffd700 1 -- 172.21.15.89:0/637589943 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0045e0 con 0x7f16fc0a0a30 2023-09-30T22:23:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.876+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16fc1220f0 con 0x7f16fc0a0a30 2023-09-30T22:23:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.877+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16fc121d90 con 0x7f16fc0a0a30 2023-09-30T22:23:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.877+0000 7f16f2ffd700 1 -- 172.21.15.89:0/637589943 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f16ec021030 con 0x7f16fc0a0a30 2023-09-30T22:23:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.877+0000 7f16f2ffd700 1 -- 172.21.15.89:0/637589943 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f16ec040a90 con 0x7f16fc0a0a30 2023-09-30T22:23:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.885+0000 7f16f2ffd700 1 -- 172.21.15.89:0/637589943 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16ec0093b0 con 0x7f16fc0a0a30 2023-09-30T22:23:01.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.909+0000 7f40fbec4700 1 Processor -- start 2023-09-30T22:23:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.909+0000 7f40fbec4700 1 -- start start 2023-09-30T22:23:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fbec4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409e850 0x7f40f409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fbec4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40f409f590 0x7f40f413faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fbec4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f409aa30 con 0x7f40f409f160 2023-09-30T22:23:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fbec4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f406c930 con 0x7f40f409f590 2023-09-30T22:23:01.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fbec4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f406bf30 con 0x7f40f409e850 2023-09-30T22:23:01.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fa461700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40f409f590 0x7f40f413faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fa461700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40f409f590 0x7f40f413faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46562/0 (socket says 172.21.15.89:46562) 2023-09-30T22:23:01.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fa461700 1 -- 172.21.15.89:0/1715389187 learned_addr learned my addr 172.21.15.89:0/1715389187 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:01.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40f9c60700 1 --2- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409e850 0x7f40f409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fa461700 1 -- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409e850 msgr2=0x7f40f409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:01.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fa461700 1 --2- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409e850 0x7f40f409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fa461700 1 -- 172.21.15.89:0/1715389187 >> v1:172.21.15.89:6789/0 conn(0x7f40f409f160 legacy=0x7f40f413c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fa461700 1 -- 172.21.15.89:0/1715389187 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f40a0250 con 0x7f40f409f590 2023-09-30T22:23:01.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40f9c60700 1 --2- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409e850 0x7f40f409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:01.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.910+0000 7f40fa461700 1 --2- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 0x7f40f413faa0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009e50 tx=0x7f40ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda58e4809ae10c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:01.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40f8c5e700 1 -- 172.21.15.89:0/1715389187 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00e070 con 0x7f40f409f590 2023-09-30T22:23:01.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40f8c5e700 1 -- 172.21.15.89:0/1715389187 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec00b910 con 0x7f40f409f590 2023-09-30T22:23:01.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40f8c5e700 1 -- 172.21.15.89:0/1715389187 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00bcc0 con 0x7f40f409f590 2023-09-30T22:23:01.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40fbec4700 1 -- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 msgr2=0x7f40f413faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40fbec4700 1 --2- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 0x7f40f413faa0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009e50 tx=0x7f40ec00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:01.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40fbec4700 1 -- 172.21.15.89:0/1715389187 shutdown_connections 2023-09-30T22:23:01.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40fbec4700 1 --2- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 0x7f40f413faa0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40fbec4700 1 --2- 172.21.15.89:0/1715389187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409e850 0x7f40f409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40fbec4700 1 -- 172.21.15.89:0/1715389187 >> 172.21.15.89:0/1715389187 conn(0x7f40f40a3f70 msgr2=0x7f40f40a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:01.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40fbec4700 1 -- 172.21.15.89:0/1715389187 shutdown_connections 2023-09-30T22:23:01.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.911+0000 7f40fbec4700 1 -- 172.21.15.89:0/1715389187 wait complete. 2023-09-30T22:23:01.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.913+0000 7f40fbec4700 1 Processor -- start 2023-09-30T22:23:01.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.913+0000 7f40fbec4700 1 -- start start 2023-09-30T22:23:01.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40fbec4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40f409e850 0x7f40f411bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40fbec4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409f160 0x7f40f411bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40fbec4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 0x7f40f411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:01.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40fbec4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40a0500 con 0x7f40f409f590 2023-09-30T22:23:01.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40fbec4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f406f8a0 con 0x7f40f409f160 2023-09-30T22:23:01.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40fbec4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40f411d600 con 0x7f40f409e850 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40f409e850 0x7f40f411bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40f409e850 0x7f40f411bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44592/0 (socket says 172.21.15.89:44592) 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 -- 172.21.15.89:0/4116778924 learned_addr learned my addr 172.21.15.89:0/4116778924 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40fa461700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 0x7f40f411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f945f700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409f160 0x7f40f411bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 -- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409f160 msgr2=0x7f40f411bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409f160 0x7f40f411bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 -- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 msgr2=0x7f40f411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 0x7f40f411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 -- 172.21.15.89:0/4116778924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00094a0 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f9c60700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40f409e850 0x7f40f411bab0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f40f000ebe0 tx=0x7f40f000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40f945f700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409f160 0x7f40f411bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40ea7fc700 1 -- 172.21.15.89:0/4116778924 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000c910 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40ea7fc700 1 -- 172.21.15.89:0/4116778924 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f000ca70 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.914+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec009b30 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.915+0000 7f40ea7fc700 1 -- 172.21.15.89:0/4116778924 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000cde0 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.915+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f411deb0 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.916+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f40747f0 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.916+0000 7f40ea7fc700 1 -- 172.21.15.89:0/4116778924 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f40f0013470 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.916+0000 7f40ea7fc700 1 -- 172.21.15.89:0/4116778924 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f40f003f8e0 con 0x7f40f409e850 2023-09-30T22:23:01.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.920+0000 7f40ea7fc700 1 -- 172.21.15.89:0/4116778924 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f40f00232f0 con 0x7f40f409e850 2023-09-30T22:23:01.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.944+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e0c0718e0 con 0x7f9e0c130e50 2023-09-30T22:23:01.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.945+0000 7f9e08ff9700 1 -- 172.21.15.89:0/4079413981 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e0401cbb0 con 0x7f9e0c130e50 2023-09-30T22:23:01.946 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:01.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.947+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130e50 msgr2=0x7f9e0c11b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:01.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.947+0000 7f9e12369700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130e50 0x7f9e0c11b1a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400efc0 tx=0x7f9e04009f90 comp rx=0 tx=0).stop 2023-09-30T22:23:01.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.947+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 shutdown_connections 2023-09-30T22:23:01.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.947+0000 7f9e12369700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e0c130e50 0x7f9e0c11b1a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.947+0000 7f9e12369700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e0c06fb20 0x7f9e0c11a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.947+0000 7f9e12369700 1 --2- 172.21.15.89:0/4079413981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e0c130540 0x7f9e0c11ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:01.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.947+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 >> 172.21.15.89:0/4079413981 conn(0x7f9e0c0a2aa0 msgr2=0x7f9e0c12d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:01.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.948+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 shutdown_connections 2023-09-30T22:23:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:01.952+0000 7f9e12369700 1 -- 172.21.15.89:0/4079413981 wait complete. 2023-09-30T22:23:01.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.079+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0a6c005860 con 0x7f0a8c09d6f0 2023-09-30T22:23:02.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.080+0000 7f0a737fe700 1 -- 172.21.15.89:0/1153611080 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a7c015070 con 0x7f0a8c09d6f0 2023-09-30T22:23:02.082 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 msgr2=0x7f0a8c1a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 --2- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c1a6020 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c000c00 tx=0x7f0a7c004750 comp rx=0 tx=0).stop 2023-09-30T22:23:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 shutdown_connections 2023-09-30T22:23:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 --2- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a8c09e410 0x7f0a8c1a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 --2- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a8c09d6f0 0x7f0a8c1a6020 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 --2- 172.21.15.89:0/1153611080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a8c09e000 0x7f0a8c1a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 >> 172.21.15.89:0/1153611080 conn(0x7f0a8c066f50 msgr2=0x7f0a8c1bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:02.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 shutdown_connections 2023-09-30T22:23:02.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.082+0000 7f0a917c4700 1 -- 172.21.15.89:0/1153611080 wait complete. 2023-09-30T22:23:02.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:02.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.112+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f40f406b250 con 0x7f40f409e850 2023-09-30T22:23:02.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.112+0000 7f40ea7fc700 1 -- 172.21.15.89:0/4116778924 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40f0017070 con 0x7f40f409e850 2023-09-30T22:23:02.113 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.114+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40f409e850 msgr2=0x7f40f411bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.114+0000 7f40fbec4700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40f409e850 0x7f40f411bab0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f40f000ebe0 tx=0x7f40f000eef0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.115+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 shutdown_connections 2023-09-30T22:23:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.115+0000 7f40fbec4700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40f409f590 0x7f40f411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.115+0000 7f40fbec4700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40f409e850 0x7f40f411bab0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.115+0000 7f40fbec4700 1 --2- 172.21.15.89:0/4116778924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40f409f160 0x7f40f411bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.115+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 >> 172.21.15.89:0/4116778924 conn(0x7f40f40a3f70 msgr2=0x7f40f40a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.115+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 shutdown_connections 2023-09-30T22:23:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.115+0000 7f40fbec4700 1 -- 172.21.15.89:0/4116778924 wait complete. 2023-09-30T22:23:02.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.122+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f16fc121d90 con 0x7f16fc0a0a30 2023-09-30T22:23:02.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.123+0000 7f16f2ffd700 1 -- 172.21.15.89:0/637589943 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16fc121d90 con 0x7f16fc0a0a30 2023-09-30T22:23:02.123 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:02.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16fc0a0a30 msgr2=0x7f16fc1c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:02.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16fc0a0a30 0x7f16fc1c8640 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000c00 tx=0x7f16ec004750 comp rx=0 tx=0).stop 2023-09-30T22:23:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 shutdown_connections 2023-09-30T22:23:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f16fc133930 0x7f16fc1c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16fc09d630 0x7f16fc1c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 --2- 172.21.15.89:0/637589943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16fc0a0a30 0x7f16fc1c8640 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 >> 172.21.15.89:0/637589943 conn(0x7f16fc0a3050 msgr2=0x7f16fc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 shutdown_connections 2023-09-30T22:23:02.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:02.125+0000 7f17046e1700 1 -- 172.21.15.89:0/637589943 wait complete. 2023-09-30T22:23:02.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:02.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:02.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:03.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46ba24b700 1 Processor -- start 2023-09-30T22:23:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46ba24b700 1 -- start start 2023-09-30T22:23:03.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46ba24b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46b412cf60 0x7f46b406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46ba24b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46ba24b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4130260 0x7f46b4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46ba24b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b412d330 con 0x7f46b412cf60 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46ba24b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b412cd90 con 0x7f46b412f8b0 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46ba24b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41306d0 con 0x7f46b4130260 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.101+0000 7f46b37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46b412cf60 0x7f46b406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46b412cf60 0x7f46b406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46580/0 (socket says 172.21.15.89:46580) 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b37fe700 1 -- 172.21.15.89:0/1920629744 learned_addr learned my addr 172.21.15.89:0/1920629744 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b3fff700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4130260 0x7f46b4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b2ffd700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b37fe700 1 -- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4130260 msgr2=0x7f46b4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b37fe700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4130260 0x7f46b4136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b37fe700 1 -- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 msgr2=0x7f46b412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b37fe700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b37fe700 1 -- 172.21.15.89:0/1920629744 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b4071780 con 0x7f46b412cf60 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b37fe700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b412cf60 0x7f46b406f5d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f46a400d180 tx=0x7f46a400d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd42050afdadec18 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b27fc700 1 -- 172.21.15.89:0/1920629744 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4010070 con 0x7f46b412cf60 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b27fc700 1 -- 172.21.15.89:0/1920629744 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a400daf0 con 0x7f46b412cf60 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.102+0000 7f46b27fc700 1 -- 172.21.15.89:0/1920629744 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a400dea0 con 0x7f46b412cf60 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 -- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b412cf60 msgr2=0x7f46b406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b412cf60 0x7f46b406f5d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f46a400d180 tx=0x7f46a400d490 comp rx=0 tx=0).stop 2023-09-30T22:23:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 -- 172.21.15.89:0/1920629744 shutdown_connections 2023-09-30T22:23:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b4130260 0x7f46b4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b412cf60 0x7f46b406f5d0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 --2- 172.21.15.89:0/1920629744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 -- 172.21.15.89:0/1920629744 >> 172.21.15.89:0/1920629744 conn(0x7f46b40a2aa0 msgr2=0x7f46b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 -- 172.21.15.89:0/1920629744 shutdown_connections 2023-09-30T22:23:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.103+0000 7f46ba24b700 1 -- 172.21.15.89:0/1920629744 wait complete. 2023-09-30T22:23:03.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46ba24b700 1 Processor -- start 2023-09-30T22:23:03.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46ba24b700 1 -- start start 2023-09-30T22:23:03.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46ba24b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b412cf60 0x7f46b409b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46ba24b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b409b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46ba24b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b4130260 0x7f46b409bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46ba24b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b409db20 con 0x7f46b4130260 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46ba24b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b412da40 con 0x7f46b412f8b0 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46ba24b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46b409eb50 con 0x7f46b412cf60 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.104+0000 7f46b2ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b409b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b2ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b409b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58220/0 (socket says 172.21.15.89:58220) 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b4130260 0x7f46b409bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b2ffd700 1 -- 172.21.15.89:0/4292502536 learned_addr learned my addr 172.21.15.89:0/4292502536 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b37fe700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b412cf60 0x7f46b409b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b2ffd700 1 -- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b412cf60 msgr2=0x7f46b409b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b2ffd700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b412cf60 0x7f46b409b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b2ffd700 1 -- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b4130260 msgr2=0x7f46b409bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b2ffd700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b4130260 0x7f46b409bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b2ffd700 1 -- 172.21.15.89:0/4292502536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a400bd40 con 0x7f46b412f8b0 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b3fff700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b4130260 0x7f46b409bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46b2ffd700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b409b890 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f46a800e9b0 tx=0x7f46a800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f469bfff700 1 -- 172.21.15.89:0/4292502536 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8009710 con 0x7f46b412f8b0 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f469bfff700 1 -- 172.21.15.89:0/4292502536 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a800ee50 con 0x7f46b412f8b0 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b411d0e0 con 0x7f46b412f8b0 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f469bfff700 1 -- 172.21.15.89:0/4292502536 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a800ca50 con 0x7f46b412f8b0 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.105+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b409cfa0 con 0x7f46b412f8b0 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.106+0000 7f469bfff700 1 -- 172.21.15.89:0/4292502536 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46a800cbb0 con 0x7f46b412f8b0 2023-09-30T22:23:03.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.106+0000 7f469bfff700 1 -- 172.21.15.89:0/4292502536 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f46a803f690 con 0x7f46b412f8b0 2023-09-30T22:23:03.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.106+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b40679c0 con 0x7f46b412f8b0 2023-09-30T22:23:03.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.111+0000 7f469bfff700 1 -- 172.21.15.89:0/4292502536 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46a80232f0 con 0x7f46b412f8b0 2023-09-30T22:23:03.128 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:03.139 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:03.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.223+0000 7f971e130700 1 Processor -- start 2023-09-30T22:23:03.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.223+0000 7f971e130700 1 -- start start 2023-09-30T22:23:03.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f971e130700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131630 0x7f9718131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f971e130700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f971806eba0 0x7f9718139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f971e130700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9718071a00 con 0x7f9718131f40 2023-09-30T22:23:03.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f971e130700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9718131430 con 0x7f9718131630 2023-09-30T22:23:03.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f971e130700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f971809a510 con 0x7f971806eba0 2023-09-30T22:23:03.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f9716ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9718131f40 0x7f971806e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35066/0 (socket says 172.21.15.89:35066) 2023-09-30T22:23:03.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f9716ffd700 1 -- 172.21.15.89:0/1124146689 learned_addr learned my addr 172.21.15.89:0/1124146689 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:03.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f9717fff700 1 --2- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f971806eba0 0x7f9718139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f97177fe700 1 --2- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131630 0x7f9718131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f97177fe700 1 -- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f971806eba0 msgr2=0x7f9718139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:03.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f97177fe700 1 --2- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f971806eba0 0x7f9718139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f97177fe700 1 -- 172.21.15.89:0/1124146689 >> v1:172.21.15.89:6789/0 conn(0x7f9718131f40 legacy=0x7f971806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.224+0000 7f97177fe700 1 -- 172.21.15.89:0/1124146689 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97180721c0 con 0x7f9718131630 2023-09-30T22:23:03.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.225+0000 7f97177fe700 1 --2- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131630 0x7f9718131a00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9708009870 tx=0x7f9708009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf036d31f7f88f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:03.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.225+0000 7f97167fc700 1 -- 172.21.15.89:0/1124146689 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708011830 con 0x7f9718131630 2023-09-30T22:23:03.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.225+0000 7f97167fc700 1 -- 172.21.15.89:0/1124146689 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9708011990 con 0x7f9718131630 2023-09-30T22:23:03.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.225+0000 7f97167fc700 1 -- 172.21.15.89:0/1124146689 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708011d20 con 0x7f9718131630 2023-09-30T22:23:03.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.226+0000 7f971e130700 1 -- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131630 msgr2=0x7f9718131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.226+0000 7f971e130700 1 --2- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131630 0x7f9718131a00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9708009870 tx=0x7f9708009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:03.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.226+0000 7f971e130700 1 -- 172.21.15.89:0/1124146689 shutdown_connections 2023-09-30T22:23:03.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.226+0000 7f971e130700 1 --2- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131630 0x7f9718131a00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.226+0000 7f971e130700 1 --2- 172.21.15.89:0/1124146689 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f971806eba0 0x7f9718139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.226+0000 7f971e130700 1 -- 172.21.15.89:0/1124146689 >> 172.21.15.89:0/1124146689 conn(0x7f97180a2aa0 msgr2=0x7f97180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:03.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.226+0000 7f971e130700 1 -- 172.21.15.89:0/1124146689 shutdown_connections 2023-09-30T22:23:03.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.226+0000 7f971e130700 1 -- 172.21.15.89:0/1124146689 wait complete. 2023-09-30T22:23:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.227+0000 7f971e130700 1 Processor -- start 2023-09-30T22:23:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.227+0000 7f971e130700 1 -- start start 2023-09-30T22:23:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f971e130700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f971806eba0 0x7f9718128290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f971e130700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9718131630 0x7f97181287d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f971e130700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131f40 0x7f9718128d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f971e130700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971812b310 con 0x7f971806eba0 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f971e130700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971812aec0 con 0x7f9718131f40 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f971e130700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f971812ac30 con 0x7f9718131630 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f9717fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131f40 0x7f9718128d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f9716ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9718131630 0x7f97181287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f9717fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131f40 0x7f9718128d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58236/0 (socket says 172.21.15.89:58236) 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f9717fff700 1 -- 172.21.15.89:0/2223288778 learned_addr learned my addr 172.21.15.89:0/2223288778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f9716ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9718131630 0x7f97181287d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44608/0 (socket says 172.21.15.89:44608) 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.228+0000 7f97177fe700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f971806eba0 0x7f9718128290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f9716ffd700 1 -- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131f40 msgr2=0x7f9718128d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f9716ffd700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131f40 0x7f9718128d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f9716ffd700 1 -- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f971806eba0 msgr2=0x7f9718128290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f9716ffd700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f971806eba0 0x7f9718128290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f9716ffd700 1 -- 172.21.15.89:0/2223288778 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080094a0 con 0x7f9718131630 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f9717fff700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131f40 0x7f9718128d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f9716ffd700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9718131630 0x7f97181287d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f970c00c4d0 tx=0x7f970c00c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f96fffff700 1 -- 172.21.15.89:0/2223288778 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c005d70 con 0x7f9718131630 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f96fffff700 1 -- 172.21.15.89:0/2223288778 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c00ce80 con 0x7f9718131630 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f96fffff700 1 -- 172.21.15.89:0/2223288778 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c0072d0 con 0x7f9718131630 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971812a9d0 con 0x7f9718131630 2023-09-30T22:23:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.229+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971812a3d0 con 0x7f9718131630 2023-09-30T22:23:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.230+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971812b100 con 0x7f9718131630 2023-09-30T22:23:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.230+0000 7f96fffff700 1 -- 172.21.15.89:0/2223288778 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f970c007430 con 0x7f9718131630 2023-09-30T22:23:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.231+0000 7f96fffff700 1 -- 172.21.15.89:0/2223288778 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f970c03e9e0 con 0x7f9718131630 2023-09-30T22:23:03.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.235+0000 7f96fffff700 1 -- 172.21.15.89:0/2223288778 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f970c01b3b0 con 0x7f9718131630 2023-09-30T22:23:03.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.269+0000 7f423ff8b700 1 Processor -- start 2023-09-30T22:23:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.270+0000 7f423ff8b700 1 -- start start 2023-09-30T22:23:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.270+0000 7f423ff8b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809e930 0x7f423809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.270+0000 7f423ff8b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f423809f240 0x7f42381317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.270+0000 7f423ff8b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4238070940 con 0x7f423809f650 2023-09-30T22:23:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.270+0000 7f423ff8b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423806fd60 con 0x7f423809f240 2023-09-30T22:23:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.270+0000 7f423ff8b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f423806fa80 con 0x7f423809e930 2023-09-30T22:23:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.270+0000 7f423dd27700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809e930 0x7f423809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423d526700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f423809f240 0x7f42381317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423dd27700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809e930 0x7f423809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44624/0 (socket says 172.21.15.89:44624) 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423dd27700 1 -- 172.21.15.89:0/424387398 learned_addr learned my addr 172.21.15.89:0/424387398 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423dd27700 1 -- 172.21.15.89:0/424387398 >> v2:172.21.15.89:3300/0 conn(0x7f423809f240 msgr2=0x7f42381317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423dd27700 1 --2- 172.21.15.89:0/424387398 >> v2:172.21.15.89:3300/0 conn(0x7f423809f240 0x7f42381317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423dd27700 1 -- 172.21.15.89:0/424387398 >> v1:172.21.15.89:6789/0 conn(0x7f423809f650 legacy=0x7f4238131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423dd27700 1 -- 172.21.15.89:0/424387398 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423809fb60 con 0x7f423809e930 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423dd27700 1 --2- 172.21.15.89:0/424387398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809e930 0x7f423809ed00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f42340098f0 tx=0x7f4234009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25923b34b092ecd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423cd25700 1 -- 172.21.15.89:0/424387398 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234004230 con 0x7f423809e930 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423cd25700 1 -- 172.21.15.89:0/424387398 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234004390 con 0x7f423809e930 2023-09-30T22:23:03.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.271+0000 7f423cd25700 1 -- 172.21.15.89:0/424387398 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42340036a0 con 0x7f423809e930 2023-09-30T22:23:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.272+0000 7f423ff8b700 1 -- 172.21.15.89:0/424387398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809e930 msgr2=0x7f423809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.272+0000 7f423ff8b700 1 --2- 172.21.15.89:0/424387398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809e930 0x7f423809ed00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f42340098f0 tx=0x7f4234009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.272+0000 7f423ff8b700 1 -- 172.21.15.89:0/424387398 shutdown_connections 2023-09-30T22:23:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.272+0000 7f423ff8b700 1 --2- 172.21.15.89:0/424387398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809e930 0x7f423809ed00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.272+0000 7f423ff8b700 1 --2- 172.21.15.89:0/424387398 >> v2:172.21.15.89:3300/0 conn(0x7f423809f240 0x7f42381317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.272+0000 7f423ff8b700 1 -- 172.21.15.89:0/424387398 >> 172.21.15.89:0/424387398 conn(0x7f42380a4210 msgr2=0x7f42380a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.272+0000 7f423ff8b700 1 -- 172.21.15.89:0/424387398 shutdown_connections 2023-09-30T22:23:03.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.272+0000 7f423ff8b700 1 -- 172.21.15.89:0/424387398 wait complete. 2023-09-30T22:23:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.273+0000 7f423ff8b700 1 Processor -- start 2023-09-30T22:23:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.273+0000 7f423ff8b700 1 -- start start 2023-09-30T22:23:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423ff8b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423809e930 0x7f42381c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423ff8b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f423809f240 0x7f42381c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423ff8b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809f650 0x7f42381c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423ff8b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42381a7c90 con 0x7f423809e930 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423ff8b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42381a7aa0 con 0x7f423809f240 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423ff8b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42380a3510 con 0x7f423809f650 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423e528700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809f650 0x7f42381c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423e528700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809f650 0x7f42381c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44632/0 (socket says 172.21.15.89:44632) 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423d526700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f423809f240 0x7f42381c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423e528700 1 -- 172.21.15.89:0/4211564539 learned_addr learned my addr 172.21.15.89:0/4211564539 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423dd27700 1 --2- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423809e930 0x7f42381c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423e528700 1 -- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f423809f240 msgr2=0x7f42381c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423e528700 1 --2- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f423809f240 0x7f42381c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423e528700 1 -- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423809e930 msgr2=0x7f42381c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423e528700 1 --2- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423809e930 0x7f42381c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.274+0000 7f423e528700 1 -- 172.21.15.89:0/4211564539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234009550 con 0x7f423809f650 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.275+0000 7f423e528700 1 --2- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809f650 0x7f42381c8230 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f423000cc10 tx=0x7f423000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.275+0000 7f422e7fc700 1 -- 172.21.15.89:0/4211564539 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000aa60 con 0x7f423809f650 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.275+0000 7f422e7fc700 1 -- 172.21.15.89:0/4211564539 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423000abc0 con 0x7f423809f650 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.275+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42380a0190 con 0x7f423809f650 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.275+0000 7f422e7fc700 1 -- 172.21.15.89:0/4211564539 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4230012420 con 0x7f423809f650 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.275+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42381a8cf0 con 0x7f423809f650 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.276+0000 7f422e7fc700 1 -- 172.21.15.89:0/4211564539 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4230012580 con 0x7f423809f650 2023-09-30T22:23:03.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.276+0000 7f422e7fc700 1 -- 172.21.15.89:0/4211564539 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f423003f920 con 0x7f423809f650 2023-09-30T22:23:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.276+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4238074ea0 con 0x7f423809f650 2023-09-30T22:23:03.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.281+0000 7f422e7fc700 1 -- 172.21.15.89:0/4211564539 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f42300196d0 con 0x7f423809f650 2023-09-30T22:23:03.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.302+0000 7f829d023700 1 Processor -- start 2023-09-30T22:23:03.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.302+0000 7f829d023700 1 -- start start 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.302+0000 7f829d023700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f829809d9f0 0x7f8298136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.302+0000 7f829d023700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8298136a70 0x7f8298138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.302+0000 7f829d023700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829809e130 con 0x7f82981316c0 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.302+0000 7f829d023700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82981321a0 con 0x7f8298136a70 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.302+0000 7f829d023700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82981312b0 con 0x7f829809d9f0 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f829809d9f0 0x7f8298136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829759e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8298136a70 0x7f8298138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f829809d9f0 0x7f8298136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44638/0 (socket says 172.21.15.89:44638) 2023-09-30T22:23:03.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829659c700 1 -- 172.21.15.89:0/114551085 learned_addr learned my addr 172.21.15.89:0/114551085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:03.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829659c700 1 -- 172.21.15.89:0/114551085 >> v2:172.21.15.89:3300/0 conn(0x7f8298136a70 msgr2=0x7f8298138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:03.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829659c700 1 --2- 172.21.15.89:0/114551085 >> v2:172.21.15.89:3300/0 conn(0x7f8298136a70 0x7f8298138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829659c700 1 -- 172.21.15.89:0/114551085 >> v1:172.21.15.89:6789/0 conn(0x7f82981316c0 legacy=0x7f8298131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:03.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829659c700 1 -- 172.21.15.89:0/114551085 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829809f6e0 con 0x7f829809d9f0 2023-09-30T22:23:03.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f829659c700 1 --2- 172.21.15.89:0/114551085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f829809d9f0 0x7f8298136530 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f828c0098f0 tx=0x7f828c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d934fc7be687df22 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:03.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f8295d9b700 1 -- 172.21.15.89:0/114551085 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c004230 con 0x7f829809d9f0 2023-09-30T22:23:03.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f8295d9b700 1 -- 172.21.15.89:0/114551085 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c004390 con 0x7f829809d9f0 2023-09-30T22:23:03.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.303+0000 7f8295d9b700 1 -- 172.21.15.89:0/114551085 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0036a0 con 0x7f829809d9f0 2023-09-30T22:23:03.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.304+0000 7f829d023700 1 -- 172.21.15.89:0/114551085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f829809d9f0 msgr2=0x7f8298136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.304+0000 7f829d023700 1 --2- 172.21.15.89:0/114551085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f829809d9f0 0x7f8298136530 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f828c0098f0 tx=0x7f828c009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.304+0000 7f829d023700 1 -- 172.21.15.89:0/114551085 shutdown_connections 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.304+0000 7f829d023700 1 --2- 172.21.15.89:0/114551085 >> v2:172.21.15.89:3300/0 conn(0x7f8298136a70 0x7f8298138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.304+0000 7f829d023700 1 --2- 172.21.15.89:0/114551085 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f829809d9f0 0x7f8298136530 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.304+0000 7f829d023700 1 -- 172.21.15.89:0/114551085 >> 172.21.15.89:0/114551085 conn(0x7f8298066f50 msgr2=0x7f829809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.304+0000 7f829d023700 1 -- 172.21.15.89:0/114551085 shutdown_connections 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.304+0000 7f829d023700 1 -- 172.21.15.89:0/114551085 wait complete. 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.305+0000 7f829d023700 1 Processor -- start 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.305+0000 7f829d023700 1 -- start start 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829d023700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f829809d9f0 0x7f8298071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829d023700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82981316c0 0x7f82980719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829d023700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8298136a70 0x7f8298071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:03.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829d023700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829806ea80 con 0x7f829809d9f0 2023-09-30T22:23:03.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829d023700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980a3ab0 con 0x7f82981316c0 2023-09-30T22:23:03.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829d023700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f829812b900 con 0x7f8298136a70 2023-09-30T22:23:03.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82981316c0 0x7f82980719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8298136a70 0x7f8298071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:03.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8298136a70 0x7f8298071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44644/0 (socket says 172.21.15.89:44644) 2023-09-30T22:23:03.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 -- 172.21.15.89:0/1955801680 learned_addr learned my addr 172.21.15.89:0/1955801680 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:03.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82981316c0 0x7f82980719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58246/0 (socket says 172.21.15.89:58246) 2023-09-30T22:23:03.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 -- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82981316c0 msgr2=0x7f82980719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 --2- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82981316c0 0x7f82980719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 -- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f829809d9f0 msgr2=0x7f8298071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 --2- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f829809d9f0 0x7f8298071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 -- 172.21.15.89:0/1955801680 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c009550 con 0x7f8298136a70 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f829759e700 1 --2- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8298136a70 0x7f8298071ee0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f82900095b0 tx=0x7f82900098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.306+0000 7f827f7fe700 1 -- 172.21.15.89:0/1955801680 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290007300 con 0x7f8298136a70 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.307+0000 7f827f7fe700 1 -- 172.21.15.89:0/1955801680 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290003a20 con 0x7f8298136a70 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.307+0000 7f827f7fe700 1 -- 172.21.15.89:0/1955801680 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290003dd0 con 0x7f8298136a70 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.307+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82980705a0 con 0x7f8298136a70 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.307+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8298070880 con 0x7f8298136a70 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.308+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8298074ea0 con 0x7f8298136a70 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.308+0000 7f827f7fe700 1 -- 172.21.15.89:0/1955801680 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8290014030 con 0x7f8298136a70 2023-09-30T22:23:03.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.308+0000 7f827f7fe700 1 -- 172.21.15.89:0/1955801680 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f829003ff50 con 0x7f8298136a70 2023-09-30T22:23:03.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.317+0000 7f827f7fe700 1 -- 172.21.15.89:0/1955801680 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f829000d110 con 0x7f8298136a70 2023-09-30T22:23:03.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.323+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f46b404ea50 con 0x7f46b412f8b0 2023-09-30T22:23:03.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.323+0000 7f469bfff700 1 -- 172.21.15.89:0/4292502536 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46a8021030 con 0x7f46b412f8b0 2023-09-30T22:23:03.324 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:03.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.325+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 msgr2=0x7f46b409b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.325+0000 7f46ba24b700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b409b890 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f46a800e9b0 tx=0x7f46a800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.325+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 shutdown_connections 2023-09-30T22:23:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.325+0000 7f46ba24b700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b4130260 0x7f46b409bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.325+0000 7f46ba24b700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46b412cf60 0x7f46b409b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.325+0000 7f46ba24b700 1 --2- 172.21.15.89:0/4292502536 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46b412f8b0 0x7f46b409b890 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.325+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 >> 172.21.15.89:0/4292502536 conn(0x7f46b40a2aa0 msgr2=0x7f46b41b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.326+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 shutdown_connections 2023-09-30T22:23:03.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.326+0000 7f46ba24b700 1 -- 172.21.15.89:0/4292502536 wait complete. 2023-09-30T22:23:03.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:03.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.450+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9718074ea0 con 0x7f9718131630 2023-09-30T22:23:03.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.450+0000 7f96fffff700 1 -- 172.21.15.89:0/2223288778 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f970c017030 con 0x7f9718131630 2023-09-30T22:23:03.451 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9718131630 msgr2=0x7f97181287d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9718131630 0x7f97181287d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f970c00c4d0 tx=0x7f970c00c7e0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 shutdown_connections 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9718131f40 0x7f9718128d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f971806eba0 0x7f9718128290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 --2- 172.21.15.89:0/2223288778 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9718131630 0x7f97181287d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 >> 172.21.15.89:0/2223288778 conn(0x7f97180a2aa0 msgr2=0x7f9718126f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 shutdown_connections 2023-09-30T22:23:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.452+0000 7f971e130700 1 -- 172.21.15.89:0/2223288778 wait complete. 2023-09-30T22:23:03.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:03.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.481+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f423809ffe0 con 0x7f423809f650 2023-09-30T22:23:03.483 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:03.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.482+0000 7f422e7fc700 1 -- 172.21.15.89:0/4211564539 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4230019310 con 0x7f423809f650 2023-09-30T22:23:03.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809f650 msgr2=0x7f42381c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 --2- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809f650 0x7f42381c8230 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f423000cc10 tx=0x7f423000cf20 comp rx=0 tx=0).stop 2023-09-30T22:23:03.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 shutdown_connections 2023-09-30T22:23:03.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 --2- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f423809f650 0x7f42381c8230 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 --2- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f423809e930 0x7f42381c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 --2- 172.21.15.89:0/4211564539 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f423809f240 0x7f42381c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 >> 172.21.15.89:0/4211564539 conn(0x7f42380a4210 msgr2=0x7f42381bae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:03.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 shutdown_connections 2023-09-30T22:23:03.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.484+0000 7f423ff8b700 1 -- 172.21.15.89:0/4211564539 wait complete. 2023-09-30T22:23:03.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.514+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f829809e980 con 0x7f8298136a70 2023-09-30T22:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.514+0000 7f827f7fe700 1 -- 172.21.15.89:0/1955801680 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82900163d0 con 0x7f8298136a70 2023-09-30T22:23:03.515 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8298136a70 msgr2=0x7f8298071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 --2- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8298136a70 0x7f8298071ee0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f82900095b0 tx=0x7f82900098c0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 shutdown_connections 2023-09-30T22:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 --2- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8298136a70 0x7f8298071ee0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 --2- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f829809d9f0 0x7f8298071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 --2- 172.21.15.89:0/1955801680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82981316c0 0x7f82980719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 >> 172.21.15.89:0/1955801680 conn(0x7f8298066f50 msgr2=0x7f829809a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 shutdown_connections 2023-09-30T22:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:03.516+0000 7f829d023700 1 -- 172.21.15.89:0/1955801680 wait complete. 2023-09-30T22:23:03.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:04.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:04.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.474+0000 7f90ffdaa700 1 Processor -- start 2023-09-30T22:23:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.474+0000 7f90ffdaa700 1 -- start start 2023-09-30T22:23:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90ffdaa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809d6f0 0x7f90f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90ffdaa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90f809e000 0x7f90f8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90ffdaa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f809f130 con 0x7f90f809e410 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90ffdaa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f809f520 con 0x7f90f809e000 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90ffdaa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f809e930 con 0x7f90f809d6f0 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fd345700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90f809e000 0x7f90f8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fdb46700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809d6f0 0x7f90f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fd345700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90f809e000 0x7f90f8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46644/0 (socket says 172.21.15.89:46644) 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fdb46700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809d6f0 0x7f90f809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58254/0 (socket says 172.21.15.89:58254) 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fd345700 1 -- 172.21.15.89:0/3636990387 learned_addr learned my addr 172.21.15.89:0/3636990387 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fdb46700 1 -- 172.21.15.89:0/3636990387 >> v2:172.21.15.89:3300/0 conn(0x7f90f809e000 msgr2=0x7f90f8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fdb46700 1 --2- 172.21.15.89:0/3636990387 >> v2:172.21.15.89:3300/0 conn(0x7f90f809e000 0x7f90f8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fdb46700 1 -- 172.21.15.89:0/3636990387 >> v1:172.21.15.89:6789/0 conn(0x7f90f809e410 legacy=0x7f90f8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fdb46700 1 -- 172.21.15.89:0/3636990387 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f809b2d0 con 0x7f90f809d6f0 2023-09-30T22:23:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.475+0000 7f90fd345700 1 --2- 172.21.15.89:0/3636990387 >> v2:172.21.15.89:3300/0 conn(0x7f90f809e000 0x7f90f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.476+0000 7f90fdb46700 1 --2- 172.21.15.89:0/3636990387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809d6f0 0x7f90f809dac0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009840 tx=0x7f90f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b94286b2984caa5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.476+0000 7f90fcb44700 1 -- 172.21.15.89:0/3636990387 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4011830 con 0x7f90f809d6f0 2023-09-30T22:23:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.476+0000 7f90fcb44700 1 -- 172.21.15.89:0/3636990387 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f4011990 con 0x7f90f809d6f0 2023-09-30T22:23:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.476+0000 7f90fcb44700 1 -- 172.21.15.89:0/3636990387 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4011d20 con 0x7f90f809d6f0 2023-09-30T22:23:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.477+0000 7f90ffdaa700 1 -- 172.21.15.89:0/3636990387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809d6f0 msgr2=0x7f90f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.477+0000 7f90ffdaa700 1 --2- 172.21.15.89:0/3636990387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809d6f0 0x7f90f809dac0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009840 tx=0x7f90f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.477+0000 7f90ffdaa700 1 -- 172.21.15.89:0/3636990387 shutdown_connections 2023-09-30T22:23:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.477+0000 7f90ffdaa700 1 --2- 172.21.15.89:0/3636990387 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809d6f0 0x7f90f809dac0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.477+0000 7f90ffdaa700 1 --2- 172.21.15.89:0/3636990387 >> v2:172.21.15.89:3300/0 conn(0x7f90f809e000 0x7f90f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.477+0000 7f90ffdaa700 1 -- 172.21.15.89:0/3636990387 >> 172.21.15.89:0/3636990387 conn(0x7f90f8066f50 msgr2=0x7f90f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.477+0000 7f90ffdaa700 1 -- 172.21.15.89:0/3636990387 shutdown_connections 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.477+0000 7f90ffdaa700 1 -- 172.21.15.89:0/3636990387 wait complete. 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90ffdaa700 1 Processor -- start 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90ffdaa700 1 -- start start 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90ffdaa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90f809d6f0 0x7f90f8122430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90ffdaa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809e000 0x7f90f8122970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90ffdaa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90f809e410 0x7f90f8122eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90ffdaa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8124640 con 0x7f90f809e410 2023-09-30T22:23:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90ffdaa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8125f00 con 0x7f90f809e000 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90ffdaa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80a1d90 con 0x7f90f809d6f0 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fd345700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809e000 0x7f90f8122970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fe347700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90f809e410 0x7f90f8122eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fd345700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809e000 0x7f90f8122970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58264/0 (socket says 172.21.15.89:58264) 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fd345700 1 -- 172.21.15.89:0/4080511713 learned_addr learned my addr 172.21.15.89:0/4080511713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fdb46700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90f809d6f0 0x7f90f8122430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fd345700 1 -- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90f809d6f0 msgr2=0x7f90f8122430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fd345700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90f809d6f0 0x7f90f8122430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fd345700 1 -- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90f809e410 msgr2=0x7f90f8122eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.478+0000 7f90fd345700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90f809e410 0x7f90f8122eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.479+0000 7f90fd345700 1 -- 172.21.15.89:0/4080511713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e800b7f0 con 0x7f90f809e000 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.479+0000 7f90fe347700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90f809e410 0x7f90f8122eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.479+0000 7f90fd345700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809e000 0x7f90f8122970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f90e800ed70 tx=0x7f90e800d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.479+0000 7f90ee7fc700 1 -- 172.21.15.89:0/4080511713 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800ee60 con 0x7f90f809e000 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.479+0000 7f90ee7fc700 1 -- 172.21.15.89:0/4080511713 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e8010440 con 0x7f90f809e000 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.479+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f40094a0 con 0x7f90f809e000 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.479+0000 7f90ee7fc700 1 -- 172.21.15.89:0/4080511713 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e80107f0 con 0x7f90f809e000 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.479+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f8130300 con 0x7f90f809e000 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.480+0000 7f90ee7fc700 1 -- 172.21.15.89:0/4080511713 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f90e8010950 con 0x7f90f809e000 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.480+0000 7f90ee7fc700 1 -- 172.21.15.89:0/4080511713 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f90e803fda0 con 0x7f90f809e000 2023-09-30T22:23:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.480+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f8074ea0 con 0x7f90f809e000 2023-09-30T22:23:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.485+0000 7f90ee7fc700 1 -- 172.21.15.89:0/4080511713 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f90e8017360 con 0x7f90f809e000 2023-09-30T22:23:04.498 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:04.529 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.596+0000 7ff98cd0c700 1 Processor -- start 2023-09-30T22:23:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.596+0000 7ff98cd0c700 1 -- start start 2023-09-30T22:23:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff98cd0c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809e1d0 0x7ff988135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff98cd0c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff98cd0c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98809f130 con 0x7ff98809d6f0 2023-09-30T22:23:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff98cd0c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98809f520 con 0x7ff98809e1d0 2023-09-30T22:23:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff98cd0c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff98809e930 con 0x7ff988135e50 2023-09-30T22:23:04.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff98659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff98809d6f0 0x7ff98809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35102/0 (socket says 172.21.15.89:35102) 2023-09-30T22:23:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff985d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809e1d0 0x7ff988135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff986d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.597+0000 7ff98659c700 1 -- 172.21.15.89:0/442628337 learned_addr learned my addr 172.21.15.89:0/442628337 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:04.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff98559a700 1 -- 172.21.15.89:0/442628337 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975225045 0 0) 0x7ff98809f130 con 0x7ff98809d6f0 2023-09-30T22:23:04.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff985d9b700 1 -- 172.21.15.89:0/442628337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 msgr2=0x7ff988138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:04.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff985d9b700 1 --2- 172.21.15.89:0/442628337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff985d9b700 1 -- 172.21.15.89:0/442628337 >> v1:172.21.15.89:6789/0 conn(0x7ff98809d6f0 legacy=0x7ff98809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff985d9b700 1 -- 172.21.15.89:0/442628337 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98809b2d0 con 0x7ff98809e1d0 2023-09-30T22:23:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff985d9b700 1 --2- 172.21.15.89:0/442628337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809e1d0 0x7ff988135910 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009870 tx=0x7ff97c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeeb79fb0f807ac3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff98559a700 1 -- 172.21.15.89:0/442628337 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c011830 con 0x7ff98809e1d0 2023-09-30T22:23:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff98559a700 1 -- 172.21.15.89:0/442628337 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c011990 con 0x7ff98809e1d0 2023-09-30T22:23:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.598+0000 7ff98559a700 1 -- 172.21.15.89:0/442628337 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c011d20 con 0x7ff98809e1d0 2023-09-30T22:23:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.599+0000 7ff98cd0c700 1 -- 172.21.15.89:0/442628337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809e1d0 msgr2=0x7ff988135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.599+0000 7ff98cd0c700 1 --2- 172.21.15.89:0/442628337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809e1d0 0x7ff988135910 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009870 tx=0x7ff97c009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.599+0000 7ff98cd0c700 1 -- 172.21.15.89:0/442628337 shutdown_connections 2023-09-30T22:23:04.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.599+0000 7ff98cd0c700 1 --2- 172.21.15.89:0/442628337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.599+0000 7ff98cd0c700 1 --2- 172.21.15.89:0/442628337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809e1d0 0x7ff988135910 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.599+0000 7ff98cd0c700 1 -- 172.21.15.89:0/442628337 >> 172.21.15.89:0/442628337 conn(0x7ff988066f50 msgr2=0x7ff98809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:04.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.599+0000 7ff98cd0c700 1 -- 172.21.15.89:0/442628337 shutdown_connections 2023-09-30T22:23:04.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.599+0000 7ff98cd0c700 1 -- 172.21.15.89:0/442628337 wait complete. 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.600+0000 7ff98cd0c700 1 Processor -- start 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.600+0000 7ff98cd0c700 1 -- start start 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.600+0000 7ff98cd0c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809d6f0 0x7ff988119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.600+0000 7ff98cd0c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff98809e1d0 0x7ff9881199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.600+0000 7ff98cd0c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.600+0000 7ff98cd0c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98811c8e0 con 0x7ff98809e1d0 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.600+0000 7ff98cd0c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98809c700 con 0x7ff98809d6f0 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.600+0000 7ff98cd0c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff988072570 con 0x7ff988135e50 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff98659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809d6f0 0x7ff988119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44686/0 (socket says 172.21.15.89:44686) 2023-09-30T22:23:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 -- 172.21.15.89:0/1460989194 learned_addr learned my addr 172.21.15.89:0/1460989194 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff985d9b700 1 --2- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff98809e1d0 0x7ff9881199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 -- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809d6f0 msgr2=0x7ff988119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 --2- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809d6f0 0x7ff988119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 -- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff98809e1d0 msgr2=0x7ff9881199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 --2- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff98809e1d0 0x7ff9881199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 -- 172.21.15.89:0/1460989194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c0094a0 con 0x7ff988135e50 2023-09-30T22:23:04.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff986d9d700 1 --2- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988119f00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff980009e30 tx=0x7ff98000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.601+0000 7ff976ffd700 1 -- 172.21.15.89:0/1460989194 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980009ca0 con 0x7ff988135e50 2023-09-30T22:23:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.602+0000 7ff976ffd700 1 -- 172.21.15.89:0/1460989194 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98000f040 con 0x7ff988135e50 2023-09-30T22:23:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.602+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98811c530 con 0x7ff988135e50 2023-09-30T22:23:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.602+0000 7ff976ffd700 1 -- 172.21.15.89:0/1460989194 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980014480 con 0x7ff988135e50 2023-09-30T22:23:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.602+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98811b680 con 0x7ff988135e50 2023-09-30T22:23:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.603+0000 7ff976ffd700 1 -- 172.21.15.89:0/1460989194 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff9800145e0 con 0x7ff988135e50 2023-09-30T22:23:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.603+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98811c720 con 0x7ff988135e50 2023-09-30T22:23:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.603+0000 7ff976ffd700 1 -- 172.21.15.89:0/1460989194 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff980040290 con 0x7ff988135e50 2023-09-30T22:23:04.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.611+0000 7ff976ffd700 1 -- 172.21.15.89:0/1460989194 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff980007900 con 0x7ff988135e50 2023-09-30T22:23:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.630+0000 7f5a0f2e7700 1 Processor -- start 2023-09-30T22:23:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.630+0000 7f5a0f2e7700 1 -- start start 2023-09-30T22:23:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0f2e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131630 0x7f5a08131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0f2e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a08131f40 0x7f5a0806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0f2e7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a08071a00 con 0x7f5a0806e9d0 2023-09-30T22:23:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0f2e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08131430 con 0x7f5a08131630 2023-09-30T22:23:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0f2e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0809a510 con 0x7f5a08131f40 2023-09-30T22:23:04.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0c882700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a08131f40 0x7f5a0806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0c882700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a08131f40 0x7f5a0806e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44688/0 (socket says 172.21.15.89:44688) 2023-09-30T22:23:04.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0c882700 1 -- 172.21.15.89:0/625994404 learned_addr learned my addr 172.21.15.89:0/625994404 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:04.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0d083700 1 --2- 172.21.15.89:0/625994404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131630 0x7f5a08131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0c882700 1 -- 172.21.15.89:0/625994404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131630 msgr2=0x7f5a08131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:04.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0c882700 1 --2- 172.21.15.89:0/625994404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131630 0x7f5a08131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0c882700 1 -- 172.21.15.89:0/625994404 >> v1:172.21.15.89:6789/0 conn(0x7f5a0806e9d0 legacy=0x7f5a08138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f59fffff700 1 -- 172.21.15.89:0/625994404 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052737381 0 0) 0x7f5a08071a00 con 0x7f5a0806e9d0 2023-09-30T22:23:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.631+0000 7f5a0c882700 1 -- 172.21.15.89:0/625994404 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080721c0 con 0x7f5a08131f40 2023-09-30T22:23:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.632+0000 7f5a0c882700 1 --2- 172.21.15.89:0/625994404 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a08131f40 0x7f5a0806e490 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009920 tx=0x7f59f8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ae8767bbbfc53f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.632+0000 7f59fffff700 1 -- 172.21.15.89:0/625994404 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8011830 con 0x7f5a08131f40 2023-09-30T22:23:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.632+0000 7f59fffff700 1 -- 172.21.15.89:0/625994404 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f8011990 con 0x7f5a08131f40 2023-09-30T22:23:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.632+0000 7f59fffff700 1 -- 172.21.15.89:0/625994404 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8011d20 con 0x7f5a08131f40 2023-09-30T22:23:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.632+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/625994404 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a08131f40 msgr2=0x7f5a0806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.632+0000 7f5a0f2e7700 1 --2- 172.21.15.89:0/625994404 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a08131f40 0x7f5a0806e490 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009920 tx=0x7f59f8009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:04.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.633+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/625994404 shutdown_connections 2023-09-30T22:23:04.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.633+0000 7f5a0f2e7700 1 --2- 172.21.15.89:0/625994404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131630 0x7f5a08131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.633+0000 7f5a0f2e7700 1 --2- 172.21.15.89:0/625994404 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a08131f40 0x7f5a0806e490 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.633+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/625994404 >> 172.21.15.89:0/625994404 conn(0x7f5a080a2aa0 msgr2=0x7f5a080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:04.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.633+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/625994404 shutdown_connections 2023-09-30T22:23:04.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.633+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/625994404 wait complete. 2023-09-30T22:23:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.634+0000 7f5a0f2e7700 1 Processor -- start 2023-09-30T22:23:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.634+0000 7f5a0f2e7700 1 -- start start 2023-09-30T22:23:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.634+0000 7f5a0f2e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a0806e9d0 0x7f5a0811b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0f2e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a08131630 0x7f5a0811bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0f2e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131f40 0x7f5a0811c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0f2e7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0811cb50 con 0x7f5a08131630 2023-09-30T22:23:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0f2e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0811d260 con 0x7f5a08131f40 2023-09-30T22:23:04.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0f2e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0811e4b0 con 0x7f5a0806e9d0 2023-09-30T22:23:04.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a08131630 0x7f5a0811bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0d083700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a0806e9d0 0x7f5a0811b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a08131630 0x7f5a0811bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46656/0 (socket says 172.21.15.89:46656) 2023-09-30T22:23:04.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 -- 172.21.15.89:0/3325767524 learned_addr learned my addr 172.21.15.89:0/3325767524 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:04.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0d884700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131f40 0x7f5a0811c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 -- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a0806e9d0 msgr2=0x7f5a0811b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a0806e9d0 0x7f5a0811b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 -- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131f40 msgr2=0x7f5a0811c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131f40 0x7f5a0811c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 -- 172.21.15.89:0/3325767524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009550 con 0x7f5a08131630 2023-09-30T22:23:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0d884700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131f40 0x7f5a0811c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.635+0000 7f5a0c882700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a08131630 0x7f5a0811bbe0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000c00 tx=0x7f59f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.636+0000 7f59fdffb700 1 -- 172.21.15.89:0/3325767524 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8004450 con 0x7f5a08131630 2023-09-30T22:23:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.636+0000 7f59fdffb700 1 -- 172.21.15.89:0/3325767524 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f80045b0 con 0x7f5a08131630 2023-09-30T22:23:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.636+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0811ee90 con 0x7f5a08131630 2023-09-30T22:23:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.636+0000 7f59fdffb700 1 -- 172.21.15.89:0/3325767524 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80038c0 con 0x7f5a08131630 2023-09-30T22:23:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.636+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0811e950 con 0x7f5a08131630 2023-09-30T22:23:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.637+0000 7f59fdffb700 1 -- 172.21.15.89:0/3325767524 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59f8003a20 con 0x7f5a08131630 2023-09-30T22:23:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.637+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a08074ea0 con 0x7f5a08131630 2023-09-30T22:23:04.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.637+0000 7f59fdffb700 1 -- 172.21.15.89:0/3325767524 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f59f8040600 con 0x7f5a08131630 2023-09-30T22:23:04.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.646+0000 7f59fdffb700 1 -- 172.21.15.89:0/3325767524 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59f802b3b0 con 0x7f5a08131630 2023-09-30T22:23:04.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.674+0000 7fb6699b9700 1 Processor -- start 2023-09-30T22:23:04.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.674+0000 7fb6699b9700 1 -- start start 2023-09-30T22:23:04.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6699b9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb66409e1d0 0x7fb664138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6699b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb664139170 0x7fb66413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6699b9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66409b120 con 0x7fb66409d6f0 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6699b9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66406b460 con 0x7fb66409e1d0 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6699b9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb66409a7e0 con 0x7fb664139170 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb664139170 0x7fb66413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb664139170 0x7fb66413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44704/0 (socket says 172.21.15.89:44704) 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6637fe700 1 -- 172.21.15.89:0/3416886815 learned_addr learned my addr 172.21.15.89:0/3416886815 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6627fc700 1 --2- 172.21.15.89:0/3416886815 >> v2:172.21.15.89:3300/0 conn(0x7fb66409e1d0 0x7fb664138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6637fe700 1 -- 172.21.15.89:0/3416886815 >> v2:172.21.15.89:3300/0 conn(0x7fb66409e1d0 msgr2=0x7fb664138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:04.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6637fe700 1 --2- 172.21.15.89:0/3416886815 >> v2:172.21.15.89:3300/0 conn(0x7fb66409e1d0 0x7fb664138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6637fe700 1 -- 172.21.15.89:0/3416886815 >> v1:172.21.15.89:6789/0 conn(0x7fb66409d6f0 legacy=0x7fb66409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6637fe700 1 -- 172.21.15.89:0/3416886815 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66409a500 con 0x7fb664139170 2023-09-30T22:23:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb6637fe700 1 --2- 172.21.15.89:0/3416886815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb664139170 0x7fb66413b550 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009e80 tx=0x7fb65c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affb4530730e5ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.675+0000 7fb661ffb700 1 -- 172.21.15.89:0/3416886815 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00e070 con 0x7fb664139170 2023-09-30T22:23:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.676+0000 7fb661ffb700 1 -- 172.21.15.89:0/3416886815 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c003a20 con 0x7fb664139170 2023-09-30T22:23:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.676+0000 7fb661ffb700 1 -- 172.21.15.89:0/3416886815 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c003dd0 con 0x7fb664139170 2023-09-30T22:23:04.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.676+0000 7fb6699b9700 1 -- 172.21.15.89:0/3416886815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb664139170 msgr2=0x7fb66413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.676+0000 7fb6699b9700 1 --2- 172.21.15.89:0/3416886815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb664139170 0x7fb66413b550 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009e80 tx=0x7fb65c00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:04.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.676+0000 7fb6699b9700 1 -- 172.21.15.89:0/3416886815 shutdown_connections 2023-09-30T22:23:04.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.676+0000 7fb6699b9700 1 --2- 172.21.15.89:0/3416886815 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb664139170 0x7fb66413b550 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.676+0000 7fb6699b9700 1 --2- 172.21.15.89:0/3416886815 >> v2:172.21.15.89:3300/0 conn(0x7fb66409e1d0 0x7fb664138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.676+0000 7fb6699b9700 1 -- 172.21.15.89:0/3416886815 >> 172.21.15.89:0/3416886815 conn(0x7fb664066f50 msgr2=0x7fb66409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:04.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.677+0000 7fb6699b9700 1 -- 172.21.15.89:0/3416886815 shutdown_connections 2023-09-30T22:23:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.677+0000 7fb6699b9700 1 -- 172.21.15.89:0/3416886815 wait complete. 2023-09-30T22:23:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.678+0000 7fb6699b9700 1 Processor -- start 2023-09-30T22:23:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.678+0000 7fb6699b9700 1 -- start start 2023-09-30T22:23:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6699b9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb66409d6f0 0x7fb66411e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6699b9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb66409e1d0 0x7fb66411ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6699b9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb664139170 0x7fb66411f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6699b9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664121ee0 con 0x7fb664139170 2023-09-30T22:23:04.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6699b9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641220c0 con 0x7fb66409e1d0 2023-09-30T22:23:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6699b9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640a2b90 con 0x7fb66409d6f0 2023-09-30T22:23:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb664139170 0x7fb66411f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb664139170 0x7fb66411f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46678/0 (socket says 172.21.15.89:46678) 2023-09-30T22:23:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 -- 172.21.15.89:0/1042001859 learned_addr learned my addr 172.21.15.89:0/1042001859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 -- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb66409d6f0 msgr2=0x7fb66411e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 --2- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb66409d6f0 0x7fb66411e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 -- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb66409e1d0 msgr2=0x7fb66411ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 --2- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb66409e1d0 0x7fb66411ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 -- 172.21.15.89:0/1042001859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c009b30 con 0x7fb664139170 2023-09-30T22:23:04.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.679+0000 7fb6637fe700 1 --2- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb664139170 0x7fb66411f390 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00b370 tx=0x7fb65c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:04.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.680+0000 7fb64bfff700 1 -- 172.21.15.89:0/1042001859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c004440 con 0x7fb664139170 2023-09-30T22:23:04.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.680+0000 7fb64bfff700 1 -- 172.21.15.89:0/1042001859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c0045a0 con 0x7fb664139170 2023-09-30T22:23:04.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.680+0000 7fb64bfff700 1 -- 172.21.15.89:0/1042001859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c004950 con 0x7fb664139170 2023-09-30T22:23:04.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.681+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66409c710 con 0x7fb664139170 2023-09-30T22:23:04.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.681+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb664122b30 con 0x7fb664139170 2023-09-30T22:23:04.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.681+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb664074ea0 con 0x7fb664139170 2023-09-30T22:23:04.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.682+0000 7fb64bfff700 1 -- 172.21.15.89:0/1042001859 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb65c012430 con 0x7fb664139170 2023-09-30T22:23:04.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.682+0000 7fb64bfff700 1 -- 172.21.15.89:0/1042001859 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb65c040a20 con 0x7fb664139170 2023-09-30T22:23:04.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.688+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f90f8125a80 con 0x7f90f809e000 2023-09-30T22:23:04.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.688+0000 7f90ee7fc700 1 -- 172.21.15.89:0/4080511713 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f90f8125a80 con 0x7f90f809e000 2023-09-30T22:23:04.689 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:04.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.690+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809e000 msgr2=0x7f90f8122970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.690+0000 7f90ffdaa700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809e000 0x7f90f8122970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f90e800ed70 tx=0x7f90e800d630 comp rx=0 tx=0).stop 2023-09-30T22:23:04.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.690+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 shutdown_connections 2023-09-30T22:23:04.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.690+0000 7f90ffdaa700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90f809e410 0x7f90f8122eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.690+0000 7f90ffdaa700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90f809d6f0 0x7f90f8122430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.690+0000 7f90ffdaa700 1 --2- 172.21.15.89:0/4080511713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90f809e000 0x7f90f8122970 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.690+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 >> 172.21.15.89:0/4080511713 conn(0x7f90f8066f50 msgr2=0x7f90f80710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:04.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.691+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 shutdown_connections 2023-09-30T22:23:04.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.691+0000 7fb64bfff700 1 -- 172.21.15.89:0/1042001859 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb65c011390 con 0x7fb664139170 2023-09-30T22:23:04.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.692+0000 7f90ffdaa700 1 -- 172.21.15.89:0/4080511713 wait complete. 2023-09-30T22:23:04.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.813+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff988074ea0 con 0x7ff988135e50 2023-09-30T22:23:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.814+0000 7ff976ffd700 1 -- 172.21.15.89:0/1460989194 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff980018020 con 0x7ff988135e50 2023-09-30T22:23:04.815 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 msgr2=0x7ff988119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 --2- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988119f00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff980009e30 tx=0x7ff98000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 shutdown_connections 2023-09-30T22:23:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 --2- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff988135e50 0x7ff988119f00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 --2- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff98809d6f0 0x7ff988119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 --2- 172.21.15.89:0/1460989194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff98809e1d0 0x7ff9881199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 >> 172.21.15.89:0/1460989194 conn(0x7ff988066f50 msgr2=0x7ff9880a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 shutdown_connections 2023-09-30T22:23:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.816+0000 7ff98cd0c700 1 -- 172.21.15.89:0/1460989194 wait complete. 2023-09-30T22:23:04.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:04.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.847+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5a0811d840 con 0x7f5a08131630 2023-09-30T22:23:04.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.847+0000 7f59fdffb700 1 -- 172.21.15.89:0/3325767524 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59f8015030 con 0x7f5a08131630 2023-09-30T22:23:04.848 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:04.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.849+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a08131630 msgr2=0x7f5a0811bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.849+0000 7f5a0f2e7700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a08131630 0x7f5a0811bbe0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000c00 tx=0x7f59f8004750 comp rx=0 tx=0).stop 2023-09-30T22:23:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.849+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 shutdown_connections 2023-09-30T22:23:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.849+0000 7f5a0f2e7700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5a08131f40 0x7f5a0811c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.849+0000 7f5a0f2e7700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5a0806e9d0 0x7f5a0811b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.849+0000 7f5a0f2e7700 1 --2- 172.21.15.89:0/3325767524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a08131630 0x7f5a0811bbe0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.849+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 >> 172.21.15.89:0/3325767524 conn(0x7f5a080a2aa0 msgr2=0x7f5a081b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.850+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 shutdown_connections 2023-09-30T22:23:04.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.850+0000 7f5a0f2e7700 1 -- 172.21.15.89:0/3325767524 wait complete. 2023-09-30T22:23:04.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:04.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.894+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb66406c360 con 0x7fb664139170 2023-09-30T22:23:04.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.894+0000 7fb64bfff700 1 -- 172.21.15.89:0/1042001859 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb65c017020 con 0x7fb664139170 2023-09-30T22:23:04.895 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb664139170 msgr2=0x7fb66411f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 --2- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb664139170 0x7fb66411f390 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00b370 tx=0x7fb65c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 shutdown_connections 2023-09-30T22:23:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 --2- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb664139170 0x7fb66411f390 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 --2- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb66409d6f0 0x7fb66411e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 --2- 172.21.15.89:0/1042001859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb66409e1d0 0x7fb66411ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 >> 172.21.15.89:0/1042001859 conn(0x7fb664066f50 msgr2=0x7fb6640a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 shutdown_connections 2023-09-30T22:23:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:04.896+0000 7fb6699b9700 1 -- 172.21.15.89:0/1042001859 wait complete. 2023-09-30T22:23:04.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:05.711 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:05.830 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.835+0000 7f5c80e1d700 1 Processor -- start 2023-09-30T22:23:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.835+0000 7f5c80e1d700 1 -- start start 2023-09-30T22:23:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c80e1d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5c7c09d6f0 0x7f5c7c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c80e1d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09e000 0x7f5c7c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c80e1d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c09b120 con 0x7f5c7c09e410 2023-09-30T22:23:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c80e1d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7c06b460 con 0x7f5c7c09d6f0 2023-09-30T22:23:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c80e1d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c09a7e0 con 0x7f5c7c09e000 2023-09-30T22:23:05.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09e000 0x7f5c7c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09e000 0x7f5c7c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58334/0 (socket says 172.21.15.89:58334) 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c71d9b700 1 -- 172.21.15.89:0/325926150 learned_addr learned my addr 172.21.15.89:0/325926150 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c7a59c700 1 --2- 172.21.15.89:0/325926150 >> v2:172.21.15.89:3300/0 conn(0x7f5c7c09d6f0 0x7f5c7c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c71d9b700 1 -- 172.21.15.89:0/325926150 >> v2:172.21.15.89:3300/0 conn(0x7f5c7c09d6f0 msgr2=0x7f5c7c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c71d9b700 1 --2- 172.21.15.89:0/325926150 >> v2:172.21.15.89:3300/0 conn(0x7f5c7c09d6f0 0x7f5c7c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c71d9b700 1 -- 172.21.15.89:0/325926150 >> v1:172.21.15.89:6789/0 conn(0x7f5c7c09e410 legacy=0x7f5c7c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c71d9b700 1 -- 172.21.15.89:0/325926150 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c09a500 con 0x7f5c7c09e000 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c7a59c700 1 --2- 172.21.15.89:0/325926150 >> v2:172.21.15.89:3300/0 conn(0x7f5c7c09d6f0 0x7f5c7c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c71d9b700 1 --2- 172.21.15.89:0/325926150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09e000 0x7f5c7c1339f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009920 tx=0x7f5c64009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393d7b8801fd0b53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c79d9b700 1 -- 172.21.15.89:0/325926150 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004230 con 0x7f5c7c09e000 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c79d9b700 1 -- 172.21.15.89:0/325926150 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c64004390 con 0x7f5c7c09e000 2023-09-30T22:23:05.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.836+0000 7f5c79d9b700 1 -- 172.21.15.89:0/325926150 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64009e70 con 0x7f5c7c09e000 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.837+0000 7f5c80e1d700 1 -- 172.21.15.89:0/325926150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09e000 msgr2=0x7f5c7c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.837+0000 7f5c80e1d700 1 --2- 172.21.15.89:0/325926150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09e000 0x7f5c7c1339f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009920 tx=0x7f5c64009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.837+0000 7f5c80e1d700 1 -- 172.21.15.89:0/325926150 shutdown_connections 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.837+0000 7f5c80e1d700 1 --2- 172.21.15.89:0/325926150 >> v2:172.21.15.89:3300/0 conn(0x7f5c7c09d6f0 0x7f5c7c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.837+0000 7f5c80e1d700 1 --2- 172.21.15.89:0/325926150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09e000 0x7f5c7c1339f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.837+0000 7f5c80e1d700 1 -- 172.21.15.89:0/325926150 >> 172.21.15.89:0/325926150 conn(0x7f5c7c066f50 msgr2=0x7f5c7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.837+0000 7f5c80e1d700 1 -- 172.21.15.89:0/325926150 shutdown_connections 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.837+0000 7f5c80e1d700 1 -- 172.21.15.89:0/325926150 wait complete. 2023-09-30T22:23:05.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.838+0000 7f5c80e1d700 1 Processor -- start 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.838+0000 7f5c80e1d700 1 -- start start 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c80e1d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09d6f0 0x7f5c7c1a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c80e1d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c7c09e000 0x7f5c7c1a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c80e1d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c7c09e410 0x7f5c7c1a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c80e1d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a8030 con 0x7f5c7c09e410 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c80e1d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a90f0 con 0x7f5c7c09d6f0 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c80e1d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c09c710 con 0x7f5c7c09e000 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c7c09e000 0x7f5c7c1a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c7c09e410 0x7f5c7c1a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c7c09e000 0x7f5c7c1a6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44730/0 (socket says 172.21.15.89:44730) 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c71d9b700 1 -- 172.21.15.89:0/2156207201 learned_addr learned my addr 172.21.15.89:0/2156207201 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7a59c700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09d6f0 0x7f5c7c1a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7ad9d700 1 -- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c7c09e000 msgr2=0x7f5c7c1a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7ad9d700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c7c09e000 0x7f5c7c1a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7ad9d700 1 -- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09d6f0 msgr2=0x7f5c7c1a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7ad9d700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09d6f0 0x7f5c7c1a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7ad9d700 1 -- 172.21.15.89:0/2156207201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64009550 con 0x7f5c7c09e410 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c71d9b700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c7c09e000 0x7f5c7c1a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7a59c700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09d6f0 0x7f5c7c1a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:05.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.839+0000 7f5c7ad9d700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c7c09e410 0x7f5c7c1a68c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400cb10 tx=0x7f5c7400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.840+0000 7f5c737fe700 1 -- 172.21.15.89:0/2156207201 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400a9e0 con 0x7f5c7c09e410 2023-09-30T22:23:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.840+0000 7f5c737fe700 1 -- 172.21.15.89:0/2156207201 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7400ab40 con 0x7f5c7c09e410 2023-09-30T22:23:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.840+0000 7f5c737fe700 1 -- 172.21.15.89:0/2156207201 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74012420 con 0x7f5c7c09e410 2023-09-30T22:23:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.840+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c0a31f0 con 0x7f5c7c09e410 2023-09-30T22:23:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.840+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a8ca0 con 0x7f5c7c09e410 2023-09-30T22:23:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.840+0000 7f5c737fe700 1 -- 172.21.15.89:0/2156207201 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c74012580 con 0x7f5c7c09e410 2023-09-30T22:23:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.841+0000 7f5c737fe700 1 -- 172.21.15.89:0/2156207201 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5c7403fc00 con 0x7f5c7c09e410 2023-09-30T22:23:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.841+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c074ea0 con 0x7f5c7c09e410 2023-09-30T22:23:05.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.846+0000 7f5c737fe700 1 -- 172.21.15.89:0/2156207201 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c740228f0 con 0x7f5c7c09e410 2023-09-30T22:23:05.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:05.910 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.958+0000 7f0ba41c8700 1 Processor -- start 2023-09-30T22:23:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba41c8700 1 -- start start 2023-09-30T22:23:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba41c8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c09de00 0x7f0b9c09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba41c8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c13d1f0 0x7f0b9c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba41c8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c09a810 con 0x7f0b9c06a770 2023-09-30T22:23:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba41c8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c06ab40 con 0x7f0b9c09de00 2023-09-30T22:23:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba41c8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c09ebc0 con 0x7f0b9c13d1f0 2023-09-30T22:23:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba1763700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c09de00 0x7f0b9c09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba1763700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c09de00 0x7f0b9c09e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58356/0 (socket says 172.21.15.89:58356) 2023-09-30T22:23:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba1763700 1 -- 172.21.15.89:0/678422788 learned_addr learned my addr 172.21.15.89:0/678422788 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.959+0000 7f0ba2765700 1 --2- 172.21.15.89:0/678422788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c13d1f0 0x7f0b9c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba1763700 1 -- 172.21.15.89:0/678422788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c13d1f0 msgr2=0x7f0b9c13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba1763700 1 --2- 172.21.15.89:0/678422788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c13d1f0 0x7f0b9c13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba1763700 1 -- 172.21.15.89:0/678422788 >> v1:172.21.15.89:6789/0 conn(0x7f0b9c06a770 legacy=0x7f0b9c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba0f62700 1 -- 172.21.15.89:0/678422788 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416056207 0 0) 0x7f0b9c09a810 con 0x7f0b9c06a770 2023-09-30T22:23:05.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba1763700 1 -- 172.21.15.89:0/678422788 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c09e920 con 0x7f0b9c09de00 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba1763700 1 --2- 172.21.15.89:0/678422788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c09de00 0x7f0b9c09e250 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009920 tx=0x7f0b8c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b702590b9a4168bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba0f62700 1 -- 172.21.15.89:0/678422788 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c011830 con 0x7f0b9c09de00 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba0f62700 1 -- 172.21.15.89:0/678422788 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c011990 con 0x7f0b9c09de00 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.960+0000 7f0ba0f62700 1 -- 172.21.15.89:0/678422788 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c011d20 con 0x7f0b9c09de00 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.961+0000 7f0ba41c8700 1 -- 172.21.15.89:0/678422788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c09de00 msgr2=0x7f0b9c09e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.961+0000 7f0ba41c8700 1 --2- 172.21.15.89:0/678422788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c09de00 0x7f0b9c09e250 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009920 tx=0x7f0b8c009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.961+0000 7f0ba41c8700 1 -- 172.21.15.89:0/678422788 shutdown_connections 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.961+0000 7f0ba41c8700 1 --2- 172.21.15.89:0/678422788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c13d1f0 0x7f0b9c13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.961+0000 7f0ba41c8700 1 --2- 172.21.15.89:0/678422788 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c09de00 0x7f0b9c09e250 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.961+0000 7f0ba41c8700 1 -- 172.21.15.89:0/678422788 >> 172.21.15.89:0/678422788 conn(0x7f0b9c066f50 msgr2=0x7f0b9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.961+0000 7f0ba41c8700 1 -- 172.21.15.89:0/678422788 shutdown_connections 2023-09-30T22:23:05.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.961+0000 7f0ba41c8700 1 -- 172.21.15.89:0/678422788 wait complete. 2023-09-30T22:23:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.962+0000 7f0ba41c8700 1 Processor -- start 2023-09-30T22:23:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.962+0000 7f0ba41c8700 1 -- start start 2023-09-30T22:23:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.962+0000 7f0ba41c8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c06a770 0x7f0b9c071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.962+0000 7f0ba41c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b9c09de00 0x7f0b9c13b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.962+0000 7f0ba41c8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c13d1f0 0x7f0b9c13ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.962+0000 7f0ba41c8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c09f1a0 con 0x7f0b9c09de00 2023-09-30T22:23:05.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.962+0000 7f0ba41c8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c11e930 con 0x7f0b9c13d1f0 2023-09-30T22:23:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.962+0000 7f0ba41c8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c13f850 con 0x7f0b9c06a770 2023-09-30T22:23:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1763700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b9c09de00 0x7f0b9c13b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1f64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c06a770 0x7f0b9c071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1763700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b9c09de00 0x7f0b9c13b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46706/0 (socket says 172.21.15.89:46706) 2023-09-30T22:23:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1f64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c06a770 0x7f0b9c071550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44750/0 (socket says 172.21.15.89:44750) 2023-09-30T22:23:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1763700 1 -- 172.21.15.89:0/3292449154 learned_addr learned my addr 172.21.15.89:0/3292449154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:05.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba2765700 1 --2- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c13d1f0 0x7f0b9c13ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1f64700 1 -- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c13d1f0 msgr2=0x7f0b9c13ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1f64700 1 --2- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c13d1f0 0x7f0b9c13ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1f64700 1 -- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b9c09de00 msgr2=0x7f0b9c13b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1f64700 1 --2- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b9c09de00 0x7f0b9c13b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1f64700 1 -- 172.21.15.89:0/3292449154 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c009550 con 0x7f0b9c06a770 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba1f64700 1 --2- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c06a770 0x7f0b9c071550 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800c430 tx=0x7f0b9800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0b927fc700 1 -- 172.21.15.89:0/3292449154 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b980054e0 con 0x7f0b9c06a770 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0b927fc700 1 -- 172.21.15.89:0/3292449154 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9800ce40 con 0x7f0b9c06a770 2023-09-30T22:23:05.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c0722d0 con 0x7f0b9c06a770 2023-09-30T22:23:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.963+0000 7f0b927fc700 1 -- 172.21.15.89:0/3292449154 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b980038c0 con 0x7f0b9c06a770 2023-09-30T22:23:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.964+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c11e210 con 0x7f0b9c06a770 2023-09-30T22:23:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.964+0000 7f0b927fc700 1 -- 172.21.15.89:0/3292449154 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b98003a20 con 0x7f0b9c06a770 2023-09-30T22:23:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.964+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c074ea0 con 0x7f0b9c06a770 2023-09-30T22:23:05.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.965+0000 7f0b927fc700 1 -- 172.21.15.89:0/3292449154 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0b9803e9a0 con 0x7f0b9c06a770 2023-09-30T22:23:05.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.969+0000 7f0b927fc700 1 -- 172.21.15.89:0/3292449154 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b98003cd0 con 0x7f0b9c06a770 2023-09-30T22:23:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa38ef9a700 1 Processor -- start 2023-09-30T22:23:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa38ef9a700 1 -- start start 2023-09-30T22:23:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa38ef9a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa38806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa38ef9a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa388137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa38ef9a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38812cd90 con 0x7fa38812f8b0 2023-09-30T22:23:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa38ef9a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38812d330 con 0x7fa38812cf60 2023-09-30T22:23:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa38ef9a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa388071780 con 0x7fa388130430 2023-09-30T22:23:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa387fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa38812f8b0 0x7fa38812fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35142/0 (socket says 172.21.15.89:35142) 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.990+0000 7fa387fff700 1 -- 172.21.15.89:0/746234961 learned_addr learned my addr 172.21.15.89:0/746234961 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa38cd36700 1 --2- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa38806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa3877fe700 1 -- 172.21.15.89:0/746234961 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332309838 0 0) 0x7fa38812cd90 con 0x7fa38812f8b0 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa3877fe700 1 -- 172.21.15.89:0/746234961 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3780035b0 con 0x7fa38812f8b0 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa38d537700 1 --2- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa388137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa38cd36700 1 -- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 msgr2=0x7fa388137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa38cd36700 1 --2- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa388137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa38cd36700 1 -- 172.21.15.89:0/746234961 >> v1:172.21.15.89:6789/0 conn(0x7fa38812f8b0 legacy=0x7fa38812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa38cd36700 1 -- 172.21.15.89:0/746234961 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388071a40 con 0x7fa38812cf60 2023-09-30T22:23:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa38d537700 1 --2- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa388137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa38cd36700 1 --2- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa38806f5d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00bb10 tx=0x7fa37c00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24275d68982e2217 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa3877fe700 1 -- 172.21.15.89:0/746234961 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c009030 con 0x7fa38812cf60 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa3877fe700 1 -- 172.21.15.89:0/746234961 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c00b050 con 0x7fa38812cf60 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.991+0000 7fa3877fe700 1 -- 172.21.15.89:0/746234961 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c00b1c0 con 0x7fa38812cf60 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.992+0000 7fa38ef9a700 1 -- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 msgr2=0x7fa38806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.992+0000 7fa38ef9a700 1 --2- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa38806f5d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00bb10 tx=0x7fa37c00be20 comp rx=0 tx=0).stop 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.992+0000 7fa38ef9a700 1 -- 172.21.15.89:0/746234961 shutdown_connections 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.992+0000 7fa38ef9a700 1 --2- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa38806f5d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.992+0000 7fa38ef9a700 1 --2- 172.21.15.89:0/746234961 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa388137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.992+0000 7fa38ef9a700 1 -- 172.21.15.89:0/746234961 >> 172.21.15.89:0/746234961 conn(0x7fa3880a2aa0 msgr2=0x7fa3880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.992+0000 7fa38ef9a700 1 -- 172.21.15.89:0/746234961 shutdown_connections 2023-09-30T22:23:05.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.992+0000 7fa38ef9a700 1 -- 172.21.15.89:0/746234961 wait complete. 2023-09-30T22:23:05.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.993+0000 7fa38ef9a700 1 Processor -- start 2023-09-30T22:23:05.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.993+0000 7fa38ef9a700 1 -- start start 2023-09-30T22:23:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.994+0000 7fa38ef9a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa3881c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.994+0000 7fa38ef9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38812f8b0 0x7fa3881c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.994+0000 7fa38ef9a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa3881c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.994+0000 7fa38ef9a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38809ea40 con 0x7fa38812f8b0 2023-09-30T22:23:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.994+0000 7fa38ef9a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38809cd30 con 0x7fa38812cf60 2023-09-30T22:23:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.994+0000 7fa38ef9a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa388129990 con 0x7fa388130430 2023-09-30T22:23:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa3881c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa3881c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58392/0 (socket says 172.21.15.89:58392) 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 -- 172.21.15.89:0/2789650864 learned_addr learned my addr 172.21.15.89:0/2789650864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38d537700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa3881c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa387fff700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38812f8b0 0x7fa3881c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 -- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 msgr2=0x7fa3881c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa3881c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 -- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38812f8b0 msgr2=0x7fa3881c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38812f8b0 0x7fa3881c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 -- 172.21.15.89:0/2789650864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c00b740 con 0x7fa38812cf60 2023-09-30T22:23:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa387fff700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38812f8b0 0x7fa3881c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38cd36700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa3881c40c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa38812d470 tx=0x7fa37c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa3857fa700 1 -- 172.21.15.89:0/2789650864 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c009040 con 0x7fa38812cf60 2023-09-30T22:23:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa3857fa700 1 -- 172.21.15.89:0/2789650864 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c00b1c0 con 0x7fa38812cf60 2023-09-30T22:23:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38812dad0 con 0x7fa38812cf60 2023-09-30T22:23:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa3857fa700 1 -- 172.21.15.89:0/2789650864 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c006de0 con 0x7fa38812cf60 2023-09-30T22:23:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.995+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa388129ff0 con 0x7fa38812cf60 2023-09-30T22:23:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.996+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38804ea50 con 0x7fa38812cf60 2023-09-30T22:23:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.997+0000 7fa3857fa700 1 -- 172.21.15.89:0/2789650864 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa37c01e050 con 0x7fa38812cf60 2023-09-30T22:23:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:05.997+0000 7fa3857fa700 1 -- 172.21.15.89:0/2789650864 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa37c00a070 con 0x7fa38812cf60 2023-09-30T22:23:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.005+0000 7fa3857fa700 1 -- 172.21.15.89:0/2789650864 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa37c020680 con 0x7fa38812cf60 2023-09-30T22:23:06.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.040+0000 7f03d9fdb700 1 Processor -- start 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.040+0000 7f03d9fdb700 1 -- start start 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d9fdb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409d6f0 0x7f03d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d9fdb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e410 0x7f03d4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d9fdb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d409b120 con 0x7f03d409e000 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d9fdb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d406b460 con 0x7f03d409d6f0 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d9fdb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03d409a7e0 con 0x7f03d409e410 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e410 0x7f03d4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409d6f0 0x7f03d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d3fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e410 0x7f03d4134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44774/0 (socket says 172.21.15.89:44774) 2023-09-30T22:23:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409d6f0 0x7f03d409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58398/0 (socket says 172.21.15.89:58398) 2023-09-30T22:23:06.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d3fff700 1 -- 172.21.15.89:0/843962548 learned_addr learned my addr 172.21.15.89:0/843962548 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d3fff700 1 -- 172.21.15.89:0/843962548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409d6f0 msgr2=0x7f03d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d3fff700 1 --2- 172.21.15.89:0/843962548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409d6f0 0x7f03d409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d3fff700 1 -- 172.21.15.89:0/843962548 >> v1:172.21.15.89:6789/0 conn(0x7f03d409e000 legacy=0x7f03d41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d3fff700 1 -- 172.21.15.89:0/843962548 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d409a500 con 0x7f03d409e410 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d3fff700 1 --2- 172.21.15.89:0/843962548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e410 0x7f03d4134100 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009eb0 tx=0x7f03cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=831bcdc7a058e432 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.041+0000 7f03d27fc700 1 -- 172.21.15.89:0/843962548 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00e070 con 0x7f03d409e410 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d27fc700 1 -- 172.21.15.89:0/843962548 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc003a20 con 0x7f03d409e410 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d27fc700 1 -- 172.21.15.89:0/843962548 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc003dd0 con 0x7f03d409e410 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d9fdb700 1 -- 172.21.15.89:0/843962548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e410 msgr2=0x7f03d4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d9fdb700 1 --2- 172.21.15.89:0/843962548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e410 0x7f03d4134100 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009eb0 tx=0x7f03cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:06.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d9fdb700 1 -- 172.21.15.89:0/843962548 shutdown_connections 2023-09-30T22:23:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d9fdb700 1 --2- 172.21.15.89:0/843962548 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409d6f0 0x7f03d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d9fdb700 1 --2- 172.21.15.89:0/843962548 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e410 0x7f03d4134100 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d9fdb700 1 -- 172.21.15.89:0/843962548 >> 172.21.15.89:0/843962548 conn(0x7f03d4066f50 msgr2=0x7f03d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.042+0000 7f03d9fdb700 1 -- 172.21.15.89:0/843962548 shutdown_connections 2023-09-30T22:23:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.043+0000 7f03d9fdb700 1 -- 172.21.15.89:0/843962548 wait complete. 2023-09-30T22:23:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.044+0000 7f03d9fdb700 1 Processor -- start 2023-09-30T22:23:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.044+0000 7f03d9fdb700 1 -- start start 2023-09-30T22:23:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d9fdb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d409d6f0 0x7f03d41224a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d9fdb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e000 0x7f03d41229e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:06.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d9fdb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409e410 0x7f03d4122f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d9fdb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4125ce0 con 0x7f03d409d6f0 2023-09-30T22:23:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d9fdb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4125fc0 con 0x7f03d409e410 2023-09-30T22:23:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d9fdb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40a1d00 con 0x7f03d409e000 2023-09-30T22:23:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d409d6f0 0x7f03d41224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d409d6f0 0x7f03d41224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46724/0 (socket says 172.21.15.89:46724) 2023-09-30T22:23:06.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d37fe700 1 -- 172.21.15.89:0/3201620139 learned_addr learned my addr 172.21.15.89:0/3201620139 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d2ffd700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e000 0x7f03d41229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d3fff700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409e410 0x7f03d4122f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d37fe700 1 -- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e000 msgr2=0x7f03d41229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d37fe700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e000 0x7f03d41229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d37fe700 1 -- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409e410 msgr2=0x7f03d4122f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:06.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d37fe700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409e410 0x7f03d4122f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d37fe700 1 -- 172.21.15.89:0/3201620139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc009b30 con 0x7f03d409d6f0 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.045+0000 7f03d2ffd700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e000 0x7f03d41229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.046+0000 7f03d37fe700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d409d6f0 0x7f03d41224a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f03c400e9b0 tx=0x7f03c400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.046+0000 7f03bbfff700 1 -- 172.21.15.89:0/3201620139 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4009710 con 0x7f03d409d6f0 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.046+0000 7f03bbfff700 1 -- 172.21.15.89:0/3201620139 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c400ee50 con 0x7f03d409d6f0 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.046+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d409c710 con 0x7f03d409d6f0 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.046+0000 7f03bbfff700 1 -- 172.21.15.89:0/3201620139 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400ca50 con 0x7f03d409d6f0 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.046+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d4125750 con 0x7f03d409d6f0 2023-09-30T22:23:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.047+0000 7f03bbfff700 1 -- 172.21.15.89:0/3201620139 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03c400cbb0 con 0x7f03d409d6f0 2023-09-30T22:23:06.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.047+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d411f030 con 0x7f03d409d6f0 2023-09-30T22:23:06.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.047+0000 7f03bbfff700 1 -- 172.21.15.89:0/3201620139 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f03c403fa90 con 0x7f03d409d6f0 2023-09-30T22:23:06.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.052+0000 7f03bbfff700 1 -- 172.21.15.89:0/3201620139 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03c401b3b0 con 0x7f03d409d6f0 2023-09-30T22:23:06.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.054+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5c7c1a9400 con 0x7f5c7c09e410 2023-09-30T22:23:06.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.054+0000 7f5c737fe700 1 -- 172.21.15.89:0/2156207201 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c74029070 con 0x7f5c7c09e410 2023-09-30T22:23:06.055 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:06.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.057+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c7c09e410 msgr2=0x7f5c7c1a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:06.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.057+0000 7f5c80e1d700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c7c09e410 0x7f5c7c1a68c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400cb10 tx=0x7f5c7400ce20 comp rx=0 tx=0).stop 2023-09-30T22:23:06.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.057+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 shutdown_connections 2023-09-30T22:23:06.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.057+0000 7f5c80e1d700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c7c09e410 0x7f5c7c1a68c0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.057+0000 7f5c80e1d700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c7c09d6f0 0x7f5c7c1a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.057+0000 7f5c80e1d700 1 --2- 172.21.15.89:0/2156207201 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c7c09e000 0x7f5c7c1a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.057+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 >> 172.21.15.89:0/2156207201 conn(0x7f5c7c066f50 msgr2=0x7f5c7c1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:06.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.058+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 shutdown_connections 2023-09-30T22:23:06.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.060+0000 7f5c80e1d700 1 -- 172.21.15.89:0/2156207201 wait complete. 2023-09-30T22:23:06.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:06.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.166+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0b9c09a3d0 con 0x7f0b9c06a770 2023-09-30T22:23:06.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.167+0000 7f0b927fc700 1 -- 172.21.15.89:0/3292449154 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b98017030 con 0x7f0b9c06a770 2023-09-30T22:23:06.168 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c06a770 msgr2=0x7f0b9c071550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 --2- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c06a770 0x7f0b9c071550 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800c430 tx=0x7f0b9800c740 comp rx=0 tx=0).stop 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 shutdown_connections 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 --2- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b9c13d1f0 0x7f0b9c13ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 --2- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b9c06a770 0x7f0b9c071550 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 --2- 172.21.15.89:0/3292449154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b9c09de00 0x7f0b9c13b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 >> 172.21.15.89:0/3292449154 conn(0x7f0b9c066f50 msgr2=0x7f0b9c0a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 shutdown_connections 2023-09-30T22:23:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.169+0000 7f0ba41c8700 1 -- 172.21.15.89:0/3292449154 wait complete. 2023-09-30T22:23:06.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:06.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.203+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa388074ea0 con 0x7fa38812cf60 2023-09-30T22:23:06.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.203+0000 7fa3857fa700 1 -- 172.21.15.89:0/2789650864 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa37c0293c0 con 0x7fa38812cf60 2023-09-30T22:23:06.204 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:06.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 msgr2=0x7fa3881c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:06.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa3881c40c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa38812d470 tx=0x7fa37c004750 comp rx=0 tx=0).stop 2023-09-30T22:23:06.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 shutdown_connections 2023-09-30T22:23:06.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa388130430 0x7fa3881c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa38812cf60 0x7fa3881c40c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 --2- 172.21.15.89:0/2789650864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38812f8b0 0x7fa3881c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 >> 172.21.15.89:0/2789650864 conn(0x7fa3880a2aa0 msgr2=0x7fa3881b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:06.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 shutdown_connections 2023-09-30T22:23:06.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.205+0000 7fa38ef9a700 1 -- 172.21.15.89:0/2789650864 wait complete. 2023-09-30T22:23:06.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.248+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f03d4125940 con 0x7f03d409d6f0 2023-09-30T22:23:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.248+0000 7f03bbfff700 1 -- 172.21.15.89:0/3201620139 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03c400d1c0 con 0x7f03d409d6f0 2023-09-30T22:23:06.249 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.250+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d409d6f0 msgr2=0x7f03d41224a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.250+0000 7f03d9fdb700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d409d6f0 0x7f03d41224a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f03c400e9b0 tx=0x7f03c400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.250+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 shutdown_connections 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.250+0000 7f03d9fdb700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f03d409e410 0x7f03d4122f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.250+0000 7f03d9fdb700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d409d6f0 0x7f03d41224a0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.250+0000 7f03d9fdb700 1 --2- 172.21.15.89:0/3201620139 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f03d409e000 0x7f03d41229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.250+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 >> 172.21.15.89:0/3201620139 conn(0x7f03d4066f50 msgr2=0x7f03d409fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.251+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 shutdown_connections 2023-09-30T22:23:06.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:06.251+0000 7f03d9fdb700 1 -- 172.21.15.89:0/3201620139 wait complete. 2023-09-30T22:23:06.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:07.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:07.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:07.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd26aca5700 1 Processor -- start 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd26aca5700 1 -- start start 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd26aca5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd26409d6f0 0x7fd26409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd26aca5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409e440 0x7fd264134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd26aca5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26409b120 con 0x7fd26409e000 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd26aca5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26406b460 con 0x7fd26409d6f0 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd26aca5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd26409a7e0 con 0x7fd26409e440 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd269242700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409e440 0x7fd264134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd268a41700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd26409d6f0 0x7fd26409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd269242700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409e440 0x7fd264134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44796/0 (socket says 172.21.15.89:44796) 2023-09-30T22:23:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd268a41700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd26409d6f0 0x7fd26409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46736/0 (socket says 172.21.15.89:46736) 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd269242700 1 -- 172.21.15.89:0/1166704854 learned_addr learned my addr 172.21.15.89:0/1166704854 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd268a41700 1 -- 172.21.15.89:0/1166704854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409e440 msgr2=0x7fd264134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.211+0000 7fd268a41700 1 --2- 172.21.15.89:0/1166704854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409e440 0x7fd264134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd268a41700 1 -- 172.21.15.89:0/1166704854 >> v1:172.21.15.89:6789/0 conn(0x7fd26409e000 legacy=0x7fd2641339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd268a41700 1 -- 172.21.15.89:0/1166704854 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26409a500 con 0x7fd26409d6f0 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd2637fe700 1 -- 172.21.15.89:0/1166704854 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801935168 0 0) 0x7fd26409b120 con 0x7fd26409e000 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd269242700 1 --2- 172.21.15.89:0/1166704854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409e440 0x7fd264134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd268a41700 1 --2- 172.21.15.89:0/1166704854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd26409dac0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd258009870 tx=0x7fd258009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30a0719628be54e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd2637fe700 1 -- 172.21.15.89:0/1166704854 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258011830 con 0x7fd26409d6f0 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd2637fe700 1 -- 172.21.15.89:0/1166704854 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd258011990 con 0x7fd26409d6f0 2023-09-30T22:23:07.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd2637fe700 1 -- 172.21.15.89:0/1166704854 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258011d20 con 0x7fd26409d6f0 2023-09-30T22:23:07.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd26aca5700 1 -- 172.21.15.89:0/1166704854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 msgr2=0x7fd26409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.212+0000 7fd26aca5700 1 --2- 172.21.15.89:0/1166704854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd26409dac0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd258009870 tx=0x7fd258009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:07.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.213+0000 7fd26aca5700 1 -- 172.21.15.89:0/1166704854 shutdown_connections 2023-09-30T22:23:07.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.213+0000 7fd26aca5700 1 --2- 172.21.15.89:0/1166704854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409d6f0 0x7fd26409dac0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.213+0000 7fd26aca5700 1 --2- 172.21.15.89:0/1166704854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409e440 0x7fd264134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.213+0000 7fd26aca5700 1 -- 172.21.15.89:0/1166704854 >> 172.21.15.89:0/1166704854 conn(0x7fd264066f50 msgr2=0x7fd26409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:07.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.213+0000 7fd26aca5700 1 -- 172.21.15.89:0/1166704854 shutdown_connections 2023-09-30T22:23:07.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.213+0000 7fd26aca5700 1 -- 172.21.15.89:0/1166704854 wait complete. 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.213+0000 7fd26aca5700 1 Processor -- start 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd26aca5700 1 -- start start 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd26aca5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409d6f0 0x7fd2641a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd26aca5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409e000 0x7fd2641a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd26aca5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e440 0x7fd2641c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd26aca5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640a1e70 con 0x7fd26409e000 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd26aca5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2641a8050 con 0x7fd26409e440 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd26aca5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2641b92c0 con 0x7fd26409d6f0 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd263fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409e000 0x7fd2641a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd269242700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e440 0x7fd2641c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd263fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409e000 0x7fd2641a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46752/0 (socket says 172.21.15.89:46752) 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd263fff700 1 -- 172.21.15.89:0/1196418931 learned_addr learned my addr 172.21.15.89:0/1196418931 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.214+0000 7fd268a41700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409d6f0 0x7fd2641a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd263fff700 1 -- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409d6f0 msgr2=0x7fd2641a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd263fff700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409d6f0 0x7fd2641a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd263fff700 1 -- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e440 msgr2=0x7fd2641c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd263fff700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e440 0x7fd2641c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd263fff700 1 -- 172.21.15.89:0/1196418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c009b30 con 0x7fd26409e000 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd269242700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e440 0x7fd2641c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd268a41700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409d6f0 0x7fd2641a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd263fff700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409e000 0x7fd2641a5750 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd250001eb0 tx=0x7fd25000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd2617fa700 1 -- 172.21.15.89:0/1196418931 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500054e0 con 0x7fd26409e000 2023-09-30T22:23:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd2617fa700 1 -- 172.21.15.89:0/1196418931 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250003680 con 0x7fd26409e000 2023-09-30T22:23:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd2617fa700 1 -- 172.21.15.89:0/1196418931 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500039f0 con 0x7fd26409e000 2023-09-30T22:23:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2580094a0 con 0x7fd26409e000 2023-09-30T22:23:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.215+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26409c760 con 0x7fd26409e000 2023-09-30T22:23:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.216+0000 7fd2617fa700 1 -- 172.21.15.89:0/1196418931 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd250003b50 con 0x7fd26409e000 2023-09-30T22:23:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.216+0000 7fd2617fa700 1 -- 172.21.15.89:0/1196418931 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd25003ea30 con 0x7fd26409e000 2023-09-30T22:23:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.216+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd264074ea0 con 0x7fd26409e000 2023-09-30T22:23:07.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.225+0000 7fd2617fa700 1 -- 172.21.15.89:0/1196418931 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd250017020 con 0x7fd26409e000 2023-09-30T22:23:07.265 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.315+0000 7faaa2dc7700 1 Processor -- start 2023-09-30T22:23:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.315+0000 7faaa2dc7700 1 -- start start 2023-09-30T22:23:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa2dc7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa2dc7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa2dc7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa9c09e410 0x7faa9c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa2dc7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9c06b460 con 0x7faa9c09e410 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa2dc7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c09b120 con 0x7faa9c09e000 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa2dc7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c09a7e0 con 0x7faa9c09d6f0 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faa9bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa1364700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa9c09e410 0x7faa9c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faa9bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58434/0 (socket says 172.21.15.89:58434) 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa1364700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa9c09e410 0x7faa9c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46760/0 (socket says 172.21.15.89:46760) 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faa9bfff700 1 -- 172.21.15.89:0/1364137355 learned_addr learned my addr 172.21.15.89:0/1364137355 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa0b63700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa1364700 1 -- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 msgr2=0x7faa9c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa1364700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa1364700 1 -- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 msgr2=0x7faa9c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa1364700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa1364700 1 -- 172.21.15.89:0/1364137355 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c09a500 con 0x7faa9c09e410 2023-09-30T22:23:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa0b63700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.316+0000 7faaa1364700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c133f30 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7faa94009d90 tx=0x7faa9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223e376dbc59b493 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faa9b7fe700 1 -- 172.21.15.89:0/1364137355 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9400e070 con 0x7faa9c09e410 2023-09-30T22:23:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faa9b7fe700 1 -- 172.21.15.89:0/1364137355 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9400bb20 con 0x7faa9c09e410 2023-09-30T22:23:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faa9b7fe700 1 -- 172.21.15.89:0/1364137355 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94012430 con 0x7faa9c09e410 2023-09-30T22:23:07.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faaa2dc7700 1 -- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 msgr2=0x7faa9c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faaa2dc7700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c133f30 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7faa94009d90 tx=0x7faa9400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faaa2dc7700 1 -- 172.21.15.89:0/1364137355 shutdown_connections 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faaa2dc7700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c133f30 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faaa2dc7700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faaa2dc7700 1 --2- 172.21.15.89:0/1364137355 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.317+0000 7faaa2dc7700 1 -- 172.21.15.89:0/1364137355 >> 172.21.15.89:0/1364137355 conn(0x7faa9c066f50 msgr2=0x7faa9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.318+0000 7faaa2dc7700 1 -- 172.21.15.89:0/1364137355 shutdown_connections 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.318+0000 7faaa2dc7700 1 -- 172.21.15.89:0/1364137355 wait complete. 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.318+0000 7faaa2dc7700 1 Processor -- start 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.318+0000 7faaa2dc7700 1 -- start start 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa2dc7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c1a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa2dc7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa2dc7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c1a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa2dc7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1aa1f0 con 0x7faa9c09e410 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa2dc7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1a7920 con 0x7faa9c09e000 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa2dc7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1a7f40 con 0x7faa9c09d6f0 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa1364700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c1a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa1364700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c1a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46764/0 (socket says 172.21.15.89:46764) 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa1364700 1 -- 172.21.15.89:0/4053268236 learned_addr learned my addr 172.21.15.89:0/4053268236 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faaa0b63700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c1a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.319+0000 7faa9bfff700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faaa1364700 1 -- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 msgr2=0x7faa9c1a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faaa1364700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c1a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faaa1364700 1 -- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 msgr2=0x7faa9c1a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faaa1364700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faaa1364700 1 -- 172.21.15.89:0/4053268236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa94009a70 con 0x7faa9c09e410 2023-09-30T22:23:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faaa1364700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c1a6820 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b680 tx=0x7faa94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faa997fa700 1 -- 172.21.15.89:0/4053268236 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94004920 con 0x7faa9c09e410 2023-09-30T22:23:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faa997fa700 1 -- 172.21.15.89:0/4053268236 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa94020070 con 0x7faa9c09e410 2023-09-30T22:23:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9c1a8b70 con 0x7faa9c09e410 2023-09-30T22:23:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faa997fa700 1 -- 172.21.15.89:0/4053268236 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa940276c0 con 0x7faa9c09e410 2023-09-30T22:23:07.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.320+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c0a2f60 con 0x7faa9c09e410 2023-09-30T22:23:07.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.321+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9c09c710 con 0x7faa9c09e410 2023-09-30T22:23:07.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.321+0000 7faa997fa700 1 -- 172.21.15.89:0/4053268236 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa94027820 con 0x7faa9c09e410 2023-09-30T22:23:07.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.322+0000 7faa997fa700 1 -- 172.21.15.89:0/4053268236 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa94022070 con 0x7faa9c09e410 2023-09-30T22:23:07.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.330+0000 7faa997fa700 1 -- 172.21.15.89:0/4053268236 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa94027ad0 con 0x7faa9c09e410 2023-09-30T22:23:07.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.350+0000 7f0cc8c32700 1 Processor -- start 2023-09-30T22:23:07.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.351+0000 7f0cc8c32700 1 -- start start 2023-09-30T22:23:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.351+0000 7f0cc8c32700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0cc409d9f0 0x7f0cc4136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.351+0000 7f0cc8c32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc4136a70 0x7f0cc4138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc8c32700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc409e130 con 0x7f0cc41316c0 2023-09-30T22:23:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc8c32700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc41321a0 con 0x7f0cc409d9f0 2023-09-30T22:23:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc8c32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41312b0 con 0x7f0cc4136a70 2023-09-30T22:23:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0cc41316c0 0x7f0cc4131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35170/0 (socket says 172.21.15.89:35170) 2023-09-30T22:23:07.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc259c700 1 -- 172.21.15.89:0/875630064 learned_addr learned my addr 172.21.15.89:0/875630064 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc2d9d700 1 --2- 172.21.15.89:0/875630064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc4136a70 0x7f0cc4138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc1d9b700 1 --2- 172.21.15.89:0/875630064 >> v2:172.21.15.89:3300/0 conn(0x7f0cc409d9f0 0x7f0cc4136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc2d9d700 1 -- 172.21.15.89:0/875630064 >> v2:172.21.15.89:3300/0 conn(0x7f0cc409d9f0 msgr2=0x7f0cc4136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc2d9d700 1 --2- 172.21.15.89:0/875630064 >> v2:172.21.15.89:3300/0 conn(0x7f0cc409d9f0 0x7f0cc4136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc2d9d700 1 -- 172.21.15.89:0/875630064 >> v1:172.21.15.89:6789/0 conn(0x7f0cc41316c0 legacy=0x7f0cc4131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc2d9d700 1 -- 172.21.15.89:0/875630064 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc409f6e0 con 0x7f0cc4136a70 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc2d9d700 1 --2- 172.21.15.89:0/875630064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc4136a70 0x7f0cc4138e50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009dc0 tx=0x7f0cbc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea2bdcec22c3c53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc159a700 1 -- 172.21.15.89:0/875630064 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00e070 con 0x7f0cc4136a70 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc159a700 1 -- 172.21.15.89:0/875630064 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc00bae0 con 0x7f0cc4136a70 2023-09-30T22:23:07.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.352+0000 7f0cc159a700 1 -- 172.21.15.89:0/875630064 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00be90 con 0x7f0cc4136a70 2023-09-30T22:23:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.353+0000 7f0cc8c32700 1 -- 172.21.15.89:0/875630064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc4136a70 msgr2=0x7f0cc4138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.353+0000 7f0cc8c32700 1 --2- 172.21.15.89:0/875630064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc4136a70 0x7f0cc4138e50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009dc0 tx=0x7f0cbc00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:07.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.353+0000 7f0cc8c32700 1 -- 172.21.15.89:0/875630064 shutdown_connections 2023-09-30T22:23:07.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.353+0000 7f0cc8c32700 1 --2- 172.21.15.89:0/875630064 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc4136a70 0x7f0cc4138e50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.353+0000 7f0cc8c32700 1 --2- 172.21.15.89:0/875630064 >> v2:172.21.15.89:3300/0 conn(0x7f0cc409d9f0 0x7f0cc4136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.353+0000 7f0cc8c32700 1 -- 172.21.15.89:0/875630064 >> 172.21.15.89:0/875630064 conn(0x7f0cc4066f50 msgr2=0x7f0cc409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:07.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.353+0000 7f0cc8c32700 1 -- 172.21.15.89:0/875630064 shutdown_connections 2023-09-30T22:23:07.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.353+0000 7f0cc8c32700 1 -- 172.21.15.89:0/875630064 wait complete. 2023-09-30T22:23:07.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc8c32700 1 Processor -- start 2023-09-30T22:23:07.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc8c32700 1 -- start start 2023-09-30T22:23:07.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc8c32700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc409d9f0 0x7f0cc4072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc8c32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc41316c0 0x7f0cc41341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc8c32700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cc4136a70 0x7f0cc4134710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc8c32700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc409b8b0 con 0x7f0cc41316c0 2023-09-30T22:23:07.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc8c32700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc409c3a0 con 0x7f0cc4136a70 2023-09-30T22:23:07.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc8c32700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc412b300 con 0x7f0cc409d9f0 2023-09-30T22:23:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.355+0000 7f0cc259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc409d9f0 0x7f0cc4072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc409d9f0 0x7f0cc4072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44850/0 (socket says 172.21.15.89:44850) 2023-09-30T22:23:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc259c700 1 -- 172.21.15.89:0/2263239577 learned_addr learned my addr 172.21.15.89:0/2263239577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc1d9b700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc41316c0 0x7f0cc41341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc2d9d700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cc4136a70 0x7f0cc4134710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc2d9d700 1 -- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc409d9f0 msgr2=0x7f0cc4072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc2d9d700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc409d9f0 0x7f0cc4072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc2d9d700 1 -- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc41316c0 msgr2=0x7f0cc41341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc2d9d700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc41316c0 0x7f0cc41341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc2d9d700 1 -- 172.21.15.89:0/2263239577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc009a70 con 0x7f0cc4136a70 2023-09-30T22:23:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc2d9d700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cc4136a70 0x7f0cc4134710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc016000 tx=0x7f0cbc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cb2ffd700 1 -- 172.21.15.89:0/2263239577 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0045d0 con 0x7f0cc4136a70 2023-09-30T22:23:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cb2ffd700 1 -- 172.21.15.89:0/2263239577 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc020070 con 0x7f0cc4136a70 2023-09-30T22:23:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cb2ffd700 1 -- 172.21.15.89:0/2263239577 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc027480 con 0x7f0cc4136a70 2023-09-30T22:23:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc409bb60 con 0x7f0cc4136a70 2023-09-30T22:23:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.356+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc40728b0 con 0x7f0cc4136a70 2023-09-30T22:23:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.357+0000 7f0cb2ffd700 1 -- 172.21.15.89:0/2263239577 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0cbc0275e0 con 0x7f0cc4136a70 2023-09-30T22:23:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.358+0000 7f0cb2ffd700 1 -- 172.21.15.89:0/2263239577 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0cbc022070 con 0x7f0cc4136a70 2023-09-30T22:23:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.358+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc4074ea0 con 0x7f0cc4136a70 2023-09-30T22:23:07.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.367+0000 7f0cb2ffd700 1 -- 172.21.15.89:0/2263239577 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0cbc01a740 con 0x7f0cc4136a70 2023-09-30T22:23:07.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.408+0000 7fb5c3fc8700 1 Processor -- start 2023-09-30T22:23:07.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.408+0000 7fb5c3fc8700 1 -- start start 2023-09-30T22:23:07.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c3fc8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc0a0a30 0x7fb5bc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c3fc8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc138d80 0x7fb5bc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c3fc8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc0a2a70 con 0x7fb5bc09d630 2023-09-30T22:23:07.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c3fc8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0a01d0 con 0x7fb5bc138d80 2023-09-30T22:23:07.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c3fc8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc09e2b0 con 0x7fb5bc0a0a30 2023-09-30T22:23:07.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c2565700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc138d80 0x7fb5bc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c2565700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc138d80 0x7fb5bc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58458/0 (socket says 172.21.15.89:58458) 2023-09-30T22:23:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c2565700 1 -- 172.21.15.89:0/591348743 learned_addr learned my addr 172.21.15.89:0/591348743 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c1d64700 1 --2- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc0a0a30 0x7fb5bc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c2565700 1 -- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc0a0a30 msgr2=0x7fb5bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c2565700 1 --2- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc0a0a30 0x7fb5bc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c2565700 1 -- 172.21.15.89:0/591348743 >> v1:172.21.15.89:6789/0 conn(0x7fb5bc09d630 legacy=0x7fb5bc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:07.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c2565700 1 -- 172.21.15.89:0/591348743 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc09dc50 con 0x7fb5bc138d80 2023-09-30T22:23:07.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c1d64700 1 --2- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc0a0a30 0x7fb5bc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:07.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.409+0000 7fb5c2565700 1 --2- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc138d80 0x7fb5bc13b160 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009e80 tx=0x7fb5b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19a9b3af423da04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:07.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c0d62700 1 -- 172.21.15.89:0/591348743 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400e070 con 0x7fb5bc138d80 2023-09-30T22:23:07.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c0d62700 1 -- 172.21.15.89:0/591348743 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b4003a20 con 0x7fb5bc138d80 2023-09-30T22:23:07.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c0d62700 1 -- 172.21.15.89:0/591348743 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4003dd0 con 0x7fb5bc138d80 2023-09-30T22:23:07.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc138d80 msgr2=0x7fb5bc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c3fc8700 1 --2- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc138d80 0x7fb5bc13b160 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009e80 tx=0x7fb5b400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:07.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/591348743 shutdown_connections 2023-09-30T22:23:07.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c3fc8700 1 --2- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc138d80 0x7fb5bc13b160 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c3fc8700 1 --2- 172.21.15.89:0/591348743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc0a0a30 0x7fb5bc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/591348743 >> 172.21.15.89:0/591348743 conn(0x7fb5bc0a3050 msgr2=0x7fb5bc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:07.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/591348743 shutdown_connections 2023-09-30T22:23:07.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.410+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/591348743 wait complete. 2023-09-30T22:23:07.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.411+0000 7fb5c3fc8700 1 Processor -- start 2023-09-30T22:23:07.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.411+0000 7fb5c3fc8700 1 -- start start 2023-09-30T22:23:07.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c3fc8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc09d630 0x7fb5bc133820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c3fc8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc0a0a30 0x7fb5bc133d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c3fc8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc138d80 0x7fb5bc1342a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:07.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c3fc8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc1372b0 con 0x7fb5bc138d80 2023-09-30T22:23:07.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c3fc8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc1365f0 con 0x7fb5bc0a0a30 2023-09-30T22:23:07.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c3fc8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0a4c00 con 0x7fb5bc09d630 2023-09-30T22:23:07.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc09d630 0x7fb5bc133820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc09d630 0x7fb5bc133820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44862/0 (socket says 172.21.15.89:44862) 2023-09-30T22:23:07.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 -- 172.21.15.89:0/4236839766 learned_addr learned my addr 172.21.15.89:0/4236839766 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:07.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1563700 1 --2- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc0a0a30 0x7fb5bc133d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:07.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 -- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc0a0a30 msgr2=0x7fb5bc133d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 --2- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc0a0a30 0x7fb5bc133d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 -- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc138d80 msgr2=0x7fb5bc1342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 --2- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc138d80 0x7fb5bc1342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 -- 172.21.15.89:0/4236839766 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b4009b30 con 0x7fb5bc09d630 2023-09-30T22:23:07.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.412+0000 7fb5c1d64700 1 --2- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc09d630 0x7fb5bc133820 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800e810 tx=0x7fb5b800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:07.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.413+0000 7fb5b27fc700 1 -- 172.21.15.89:0/4236839766 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8013610 con 0x7fb5bc09d630 2023-09-30T22:23:07.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.413+0000 7fb5b27fc700 1 -- 172.21.15.89:0/4236839766 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b8013770 con 0x7fb5bc09d630 2023-09-30T22:23:07.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.413+0000 7fb5b27fc700 1 -- 172.21.15.89:0/4236839766 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8013ae0 con 0x7fb5bc09d630 2023-09-30T22:23:07.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.413+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc0a0410 con 0x7fb5bc09d630 2023-09-30T22:23:07.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.413+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc136a40 con 0x7fb5bc09d630 2023-09-30T22:23:07.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.414+0000 7fb5b27fc700 1 -- 172.21.15.89:0/4236839766 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb5b8013c40 con 0x7fb5bc09d630 2023-09-30T22:23:07.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.414+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5bc135c00 con 0x7fb5bc09d630 2023-09-30T22:23:07.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.414+0000 7fb5b27fc700 1 -- 172.21.15.89:0/4236839766 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb5b803f3c0 con 0x7fb5bc09d630 2023-09-30T22:23:07.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.423+0000 7fb5b27fc700 1 -- 172.21.15.89:0/4236839766 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5b8012370 con 0x7fb5bc09d630 2023-09-30T22:23:07.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.446+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd26409f420 con 0x7fd26409e000 2023-09-30T22:23:07.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.446+0000 7fd2617fa700 1 -- 172.21.15.89:0/1196418931 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd25001b3b0 con 0x7fd26409e000 2023-09-30T22:23:07.446 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:07.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.449+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409e000 msgr2=0x7fd2641a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.449+0000 7fd26aca5700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409e000 0x7fd2641a5750 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd250001eb0 tx=0x7fd25000c800 comp rx=0 tx=0).stop 2023-09-30T22:23:07.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.449+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 shutdown_connections 2023-09-30T22:23:07.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.449+0000 7fd26aca5700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd26409e440 0x7fd2641c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.449+0000 7fd26aca5700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd26409d6f0 0x7fd2641a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.449+0000 7fd26aca5700 1 --2- 172.21.15.89:0/1196418931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd26409e000 0x7fd2641a5750 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.449+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 >> 172.21.15.89:0/1196418931 conn(0x7fd264066f50 msgr2=0x7fd2640a2610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:07.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.450+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 shutdown_connections 2023-09-30T22:23:07.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.450+0000 7fd26aca5700 1 -- 172.21.15.89:0/1196418931 wait complete. 2023-09-30T22:23:07.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:07.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.529+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa9c09f420 con 0x7faa9c09e410 2023-09-30T22:23:07.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.529+0000 7faa997fa700 1 -- 172.21.15.89:0/4053268236 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa94011390 con 0x7faa9c09e410 2023-09-30T22:23:07.529 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:07.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.531+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 msgr2=0x7faa9c1a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.531+0000 7faaa2dc7700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c1a6820 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b680 tx=0x7faa94004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.531+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 shutdown_connections 2023-09-30T22:23:07.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.532+0000 7faaa2dc7700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa9c09e410 0x7faa9c1a6820 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.532+0000 7faaa2dc7700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa9c09d6f0 0x7faa9c1a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.532+0000 7faaa2dc7700 1 --2- 172.21.15.89:0/4053268236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa9c09e000 0x7faa9c1a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.532+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 >> 172.21.15.89:0/4053268236 conn(0x7faa9c066f50 msgr2=0x7faa9c1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:07.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.532+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 shutdown_connections 2023-09-30T22:23:07.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.532+0000 7faaa2dc7700 1 -- 172.21.15.89:0/4053268236 wait complete. 2023-09-30T22:23:07.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:07.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.566+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0cc409e980 con 0x7f0cc4136a70 2023-09-30T22:23:07.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.566+0000 7f0cb2ffd700 1 -- 172.21.15.89:0/2263239577 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0cbc01a380 con 0x7f0cc4136a70 2023-09-30T22:23:07.566 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cc4136a70 msgr2=0x7f0cc4134710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cc4136a70 0x7f0cc4134710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc016000 tx=0x7f0cbc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 shutdown_connections 2023-09-30T22:23:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0cc4136a70 0x7f0cc4134710 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0cc409d9f0 0x7f0cc4072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 --2- 172.21.15.89:0/2263239577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0cc41316c0 0x7f0cc41341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 >> 172.21.15.89:0/2263239577 conn(0x7f0cc4066f50 msgr2=0x7f0cc41385d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 shutdown_connections 2023-09-30T22:23:07.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.568+0000 7f0cc8c32700 1 -- 172.21.15.89:0/2263239577 wait complete. 2023-09-30T22:23:07.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.617+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb5bc0747f0 con 0x7fb5bc09d630 2023-09-30T22:23:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.618+0000 7fb5b27fc700 1 -- 172.21.15.89:0/4236839766 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5b8021070 con 0x7fb5bc09d630 2023-09-30T22:23:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc09d630 msgr2=0x7fb5bc133820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 --2- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc09d630 0x7fb5bc133820 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800e810 tx=0x7fb5b800eb20 comp rx=0 tx=0).stop 2023-09-30T22:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 shutdown_connections 2023-09-30T22:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 --2- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc138d80 0x7fb5bc1342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 --2- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb5bc09d630 0x7fb5bc133820 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 --2- 172.21.15.89:0/4236839766 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb5bc0a0a30 0x7fb5bc133d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 >> 172.21.15.89:0/4236839766 conn(0x7fb5bc0a3050 msgr2=0x7fb5bc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 shutdown_connections 2023-09-30T22:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:07.620+0000 7fb5c3fc8700 1 -- 172.21.15.89:0/4236839766 wait complete. 2023-09-30T22:23:07.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:08.464 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:08.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:08.580 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:08.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.602+0000 7faa2ffec700 1 Processor -- start 2023-09-30T22:23:08.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.602+0000 7faa2ffec700 1 -- start start 2023-09-30T22:23:08.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2ffec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28133420 0x7faa281337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2ffec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa28134260 0x7faa281306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2ffec700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2806b990 con 0x7faa28133e30 2023-09-30T22:23:08.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2ffec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa280a0590 con 0x7faa28134260 2023-09-30T22:23:08.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2ffec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2809fb80 con 0x7faa28133420 2023-09-30T22:23:08.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2d587700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faa28133e30 0x7faa2812ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35196/0 (socket says 172.21.15.89:35196) 2023-09-30T22:23:08.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2d587700 1 -- 172.21.15.89:0/19863757 learned_addr learned my addr 172.21.15.89:0/19863757 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:08.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2dd88700 1 --2- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28133420 0x7faa281337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2e589700 1 --2- 172.21.15.89:0/19863757 >> v2:172.21.15.89:3300/0 conn(0x7faa28134260 0x7faa281306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2e589700 1 -- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28133420 msgr2=0x7faa281337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:08.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2e589700 1 --2- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28133420 0x7faa281337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2e589700 1 -- 172.21.15.89:0/19863757 >> v1:172.21.15.89:6789/0 conn(0x7faa28133e30 legacy=0x7faa2812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2e589700 1 -- 172.21.15.89:0/19863757 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa280a0030 con 0x7faa28134260 2023-09-30T22:23:08.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2dd88700 1 --2- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28133420 0x7faa281337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:08.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.603+0000 7faa2e589700 1 --2- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28134260 0x7faa281306a0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7faa20009d90 tx=0x7faa2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a4af59f15668f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:08.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2cd86700 1 -- 172.21.15.89:0/19863757 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2000e070 con 0x7faa28134260 2023-09-30T22:23:08.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2cd86700 1 -- 172.21.15.89:0/19863757 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2000bb20 con 0x7faa28134260 2023-09-30T22:23:08.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2cd86700 1 -- 172.21.15.89:0/19863757 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20012430 con 0x7faa28134260 2023-09-30T22:23:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2ffec700 1 -- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28134260 msgr2=0x7faa281306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2ffec700 1 --2- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28134260 0x7faa281306a0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7faa20009d90 tx=0x7faa2000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2ffec700 1 -- 172.21.15.89:0/19863757 shutdown_connections 2023-09-30T22:23:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2ffec700 1 --2- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28134260 0x7faa281306a0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2ffec700 1 --2- 172.21.15.89:0/19863757 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28133420 0x7faa281337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2ffec700 1 -- 172.21.15.89:0/19863757 >> 172.21.15.89:0/19863757 conn(0x7faa280a4210 msgr2=0x7faa280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2ffec700 1 -- 172.21.15.89:0/19863757 shutdown_connections 2023-09-30T22:23:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.604+0000 7faa2ffec700 1 -- 172.21.15.89:0/19863757 wait complete. 2023-09-30T22:23:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2ffec700 1 Processor -- start 2023-09-30T22:23:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2ffec700 1 -- start start 2023-09-30T22:23:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2ffec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa28133420 0x7faa2809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2ffec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28133e30 0x7faa2809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2ffec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28134260 0x7faa2809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2ffec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa281202f0 con 0x7faa28133e30 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2ffec700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2812da20 con 0x7faa28134260 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2ffec700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa2811ff20 con 0x7faa28133420 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2d587700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28133e30 0x7faa2809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2d587700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28133e30 0x7faa2809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46806/0 (socket says 172.21.15.89:46806) 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.606+0000 7faa2d587700 1 -- 172.21.15.89:0/4191749842 learned_addr learned my addr 172.21.15.89:0/4191749842 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2e589700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28134260 0x7faa2809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2dd88700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa28133420 0x7faa2809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2d587700 1 -- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa28133420 msgr2=0x7faa2809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2d587700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa28133420 0x7faa2809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2d587700 1 -- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28134260 msgr2=0x7faa2809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2d587700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28134260 0x7faa2809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2d587700 1 -- 172.21.15.89:0/4191749842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa240094a0 con 0x7faa28133e30 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2e589700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28134260 0x7faa2809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2dd88700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa28133420 0x7faa2809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2d587700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28133e30 0x7faa2809c9c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7faa1800c530 tx=0x7faa1800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa1e7fc700 1 -- 172.21.15.89:0/4191749842 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18005d70 con 0x7faa28133e30 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa1e7fc700 1 -- 172.21.15.89:0/4191749842 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa180070d0 con 0x7faa28133e30 2023-09-30T22:23:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa1e7fc700 1 -- 172.21.15.89:0/4191749842 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18007440 con 0x7faa28133e30 2023-09-30T22:23:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa20009a70 con 0x7faa28133e30 2023-09-30T22:23:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.607+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2809ddb0 con 0x7faa28133e30 2023-09-30T22:23:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.608+0000 7faa1e7fc700 1 -- 172.21.15.89:0/4191749842 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa180075a0 con 0x7faa28133e30 2023-09-30T22:23:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.608+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa28074ea0 con 0x7faa28133e30 2023-09-30T22:23:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.608+0000 7faa1e7fc700 1 -- 172.21.15.89:0/4191749842 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa1803e880 con 0x7faa28133e30 2023-09-30T22:23:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.613+0000 7faa1e7fc700 1 -- 172.21.15.89:0/4191749842 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa1801b3b0 con 0x7faa28133e30 2023-09-30T22:23:08.632 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.679+0000 7fc41b4ef700 1 Processor -- start 2023-09-30T22:23:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc41b4ef700 1 -- start start 2023-09-30T22:23:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc41b4ef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc41406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc41b4ef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc414131d10 0x7fc414139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc41b4ef700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41409ece0 con 0x7fc41406b080 2023-09-30T22:23:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc41b4ef700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41409e760 con 0x7fc41406a770 2023-09-30T22:23:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc41b4ef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc41409e040 con 0x7fc414131d10 2023-09-30T22:23:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc419a8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc414131d10 0x7fc414139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc419a8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc414131d10 0x7fc414139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44870/0 (socket says 172.21.15.89:44870) 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.680+0000 7fc419a8c700 1 -- 172.21.15.89:0/114374648 learned_addr learned my addr 172.21.15.89:0/114374648 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc41928b700 1 --2- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc41406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc40bfff700 1 -- 172.21.15.89:0/114374648 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004607551 0 0) 0x7fc41409ece0 con 0x7fc41406b080 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc40bfff700 1 -- 172.21.15.89:0/114374648 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4000035b0 con 0x7fc41406b080 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc419a8c700 1 -- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 msgr2=0x7fc41406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc419a8c700 1 --2- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc41406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc419a8c700 1 -- 172.21.15.89:0/114374648 >> v1:172.21.15.89:6789/0 conn(0x7fc41406b080 legacy=0x7fc414131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc419a8c700 1 -- 172.21.15.89:0/114374648 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41409f630 con 0x7fc414131d10 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc41928b700 1 --2- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc41406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc419a8c700 1 --2- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc414131d10 0x7fc414139860 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009e80 tx=0x7fc40c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636350338cc13794 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc40bfff700 1 -- 172.21.15.89:0/114374648 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c00e070 con 0x7fc414131d10 2023-09-30T22:23:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc40bfff700 1 -- 172.21.15.89:0/114374648 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c00b910 con 0x7fc414131d10 2023-09-30T22:23:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.681+0000 7fc40bfff700 1 -- 172.21.15.89:0/114374648 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c00bcc0 con 0x7fc414131d10 2023-09-30T22:23:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.682+0000 7fc41b4ef700 1 -- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc414131d10 msgr2=0x7fc414139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.682+0000 7fc41b4ef700 1 --2- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc414131d10 0x7fc414139860 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009e80 tx=0x7fc40c00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.682+0000 7fc41b4ef700 1 -- 172.21.15.89:0/114374648 shutdown_connections 2023-09-30T22:23:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.682+0000 7fc41b4ef700 1 --2- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc41406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.682+0000 7fc41b4ef700 1 --2- 172.21.15.89:0/114374648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc414131d10 0x7fc414139860 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.682+0000 7fc41b4ef700 1 -- 172.21.15.89:0/114374648 >> 172.21.15.89:0/114374648 conn(0x7fc414066f50 msgr2=0x7fc41409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.682+0000 7fc41b4ef700 1 -- 172.21.15.89:0/114374648 shutdown_connections 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.682+0000 7fc41b4ef700 1 -- 172.21.15.89:0/114374648 wait complete. 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.683+0000 7fc41b4ef700 1 Processor -- start 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41b4ef700 1 -- start start 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41b4ef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc414120450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41b4ef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41406b080 0x7fc414120990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41b4ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc414131d10 0x7fc414120ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41b4ef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414122120 con 0x7fc414131d10 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41b4ef700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc414071e60 con 0x7fc41406a770 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41b4ef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc414123980 con 0x7fc41406b080 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41928b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc414120450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41928b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc414120450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58482/0 (socket says 172.21.15.89:58482) 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.684+0000 7fc41928b700 1 -- 172.21.15.89:0/2618537583 learned_addr learned my addr 172.21.15.89:0/2618537583 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc419a8c700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc414131d10 0x7fc414120ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc418a8a700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41406b080 0x7fc414120990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc41928b700 1 -- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41406b080 msgr2=0x7fc414120990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc41928b700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41406b080 0x7fc414120990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc41928b700 1 -- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc414131d10 msgr2=0x7fc414120ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc41928b700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc414131d10 0x7fc414120ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc41928b700 1 -- 172.21.15.89:0/2618537583 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100094a0 con 0x7fc41406a770 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc418a8a700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41406b080 0x7fc414120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc41928b700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc414120450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc41000ebe0 tx=0x7fc41000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc409ffb700 1 -- 172.21.15.89:0/2618537583 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410009800 con 0x7fc41406a770 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc409ffb700 1 -- 172.21.15.89:0/2618537583 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41000c910 con 0x7fc41406a770 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c009b30 con 0x7fc41406a770 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc409ffb700 1 -- 172.21.15.89:0/2618537583 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41000cc80 con 0x7fc41406a770 2023-09-30T22:23:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.685+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4141246b0 con 0x7fc41406a770 2023-09-30T22:23:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.686+0000 7fc409ffb700 1 -- 172.21.15.89:0/2618537583 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc410013470 con 0x7fc41406a770 2023-09-30T22:23:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.686+0000 7fc409ffb700 1 -- 172.21.15.89:0/2618537583 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc41003fc00 con 0x7fc41406a770 2023-09-30T22:23:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.687+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc414123af0 con 0x7fc41406a770 2023-09-30T22:23:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.691+0000 7fc409ffb700 1 -- 172.21.15.89:0/2618537583 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc410017020 con 0x7fc41406a770 2023-09-30T22:23:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.721+0000 7fdab3d84700 1 Processor -- start 2023-09-30T22:23:08.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.721+0000 7fdab3d84700 1 -- start start 2023-09-30T22:23:08.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab3d84700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdaac136710 0x7fdaac138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab3d84700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac139030 0x7fdaac13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab3d84700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaac06ef30 con 0x7fdaac1338a0 2023-09-30T22:23:08.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab3d84700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaac06fef0 con 0x7fdaac136710 2023-09-30T22:23:08.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab3d84700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac1334b0 con 0x7fdaac139030 2023-09-30T22:23:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab2321700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac139030 0x7fdaac13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab2321700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac139030 0x7fdaac13b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58496/0 (socket says 172.21.15.89:58496) 2023-09-30T22:23:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab2321700 1 -- 172.21.15.89:0/4270028848 learned_addr learned my addr 172.21.15.89:0/4270028848 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab2321700 1 -- 172.21.15.89:0/4270028848 >> v2:172.21.15.89:3300/0 conn(0x7fdaac136710 msgr2=0x7fdaac138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab2321700 1 --2- 172.21.15.89:0/4270028848 >> v2:172.21.15.89:3300/0 conn(0x7fdaac136710 0x7fdaac138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab2321700 1 -- 172.21.15.89:0/4270028848 >> v1:172.21.15.89:6789/0 conn(0x7fdaac1338a0 legacy=0x7fdaac133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.722+0000 7fdab2321700 1 -- 172.21.15.89:0/4270028848 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac06fc30 con 0x7fdaac139030 2023-09-30T22:23:08.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab2321700 1 --2- 172.21.15.89:0/4270028848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac139030 0x7fdaac13b410 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009e50 tx=0x7fdaa400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c48f01d3ff648c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:08.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab0b1e700 1 -- 172.21.15.89:0/4270028848 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400e070 con 0x7fdaac139030 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab0b1e700 1 -- 172.21.15.89:0/4270028848 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa4003a20 con 0x7fdaac139030 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab0b1e700 1 -- 172.21.15.89:0/4270028848 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4003dd0 con 0x7fdaac139030 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab3d84700 1 -- 172.21.15.89:0/4270028848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac139030 msgr2=0x7fdaac13b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab3d84700 1 --2- 172.21.15.89:0/4270028848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac139030 0x7fdaac13b410 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009e50 tx=0x7fdaa400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab3d84700 1 -- 172.21.15.89:0/4270028848 shutdown_connections 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab3d84700 1 --2- 172.21.15.89:0/4270028848 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac139030 0x7fdaac13b410 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab3d84700 1 --2- 172.21.15.89:0/4270028848 >> v2:172.21.15.89:3300/0 conn(0x7fdaac136710 0x7fdaac138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.723+0000 7fdab3d84700 1 -- 172.21.15.89:0/4270028848 >> 172.21.15.89:0/4270028848 conn(0x7fdaac0a2aa0 msgr2=0x7fdaac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.724+0000 7fdab3d84700 1 -- 172.21.15.89:0/4270028848 shutdown_connections 2023-09-30T22:23:08.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.724+0000 7fdab3d84700 1 -- 172.21.15.89:0/4270028848 wait complete. 2023-09-30T22:23:08.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.725+0000 7fdab3d84700 1 Processor -- start 2023-09-30T22:23:08.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.725+0000 7fdab3d84700 1 -- start start 2023-09-30T22:23:08.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.725+0000 7fdab3d84700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdaac1338a0 0x7fdaac118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.725+0000 7fdab3d84700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac136710 0x7fdaac119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab3d84700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaac139030 0x7fdaac119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab3d84700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac11b7c0 con 0x7fdaac139030 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab3d84700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac11ad60 con 0x7fdaac136710 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab3d84700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac13b420 con 0x7fdaac1338a0 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac136710 0x7fdaac119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac136710 0x7fdaac119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58498/0 (socket says 172.21.15.89:58498) 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 -- 172.21.15.89:0/2450379202 learned_addr learned my addr 172.21.15.89:0/2450379202 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab1b20700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdaac1338a0 0x7fdaac118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab2321700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaac139030 0x7fdaac119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 -- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdaac1338a0 msgr2=0x7fdaac118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdaac1338a0 0x7fdaac118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 -- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaac139030 msgr2=0x7fdaac119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaac139030 0x7fdaac119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 -- 172.21.15.89:0/2450379202 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4009b30 con 0x7fdaac136710 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.726+0000 7fdab131f700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac136710 0x7fdaac119210 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00b530 tx=0x7fda9c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.727+0000 7fdaa27fc700 1 -- 172.21.15.89:0/2450379202 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c011830 con 0x7fdaac136710 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.727+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac06f640 con 0x7fdaac136710 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.727+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac11c4b0 con 0x7fdaac136710 2023-09-30T22:23:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.727+0000 7fdaa27fc700 1 -- 172.21.15.89:0/2450379202 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c011990 con 0x7fdaac136710 2023-09-30T22:23:08.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.727+0000 7fdaa27fc700 1 -- 172.21.15.89:0/2450379202 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c011d00 con 0x7fdaac136710 2023-09-30T22:23:08.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.728+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda940052e0 con 0x7fdaac136710 2023-09-30T22:23:08.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.730+0000 7fdaa27fc700 1 -- 172.21.15.89:0/2450379202 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda9c024440 con 0x7fdaac136710 2023-09-30T22:23:08.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.730+0000 7fdaa27fc700 1 -- 172.21.15.89:0/2450379202 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fda9c03fd20 con 0x7fdaac136710 2023-09-30T22:23:08.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.737+0000 7fdaa27fc700 1 -- 172.21.15.89:0/2450379202 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda9c022740 con 0x7fdaac136710 2023-09-30T22:23:08.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.772+0000 7f385b1cf700 1 Processor -- start 2023-09-30T22:23:08.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.772+0000 7f385b1cf700 1 -- start start 2023-09-30T22:23:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f385b1cf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f385409e1d0 0x7f3854138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f385b1cf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3854139170 0x7f385413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f385b1cf700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385409b120 con 0x7f385409d6f0 2023-09-30T22:23:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f385b1cf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385406b460 con 0x7f3854139170 2023-09-30T22:23:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f385b1cf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385409a7e0 con 0x7f385409e1d0 2023-09-30T22:23:08.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f3853fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f385409e1d0 0x7f3854138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f3853fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f385409e1d0 0x7f3854138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58500/0 (socket says 172.21.15.89:58500) 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f3853fff700 1 -- 172.21.15.89:0/3976190519 learned_addr learned my addr 172.21.15.89:0/3976190519 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f385976c700 1 --2- 172.21.15.89:0/3976190519 >> v2:172.21.15.89:3300/0 conn(0x7f3854139170 0x7f385413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f3853fff700 1 -- 172.21.15.89:0/3976190519 >> v2:172.21.15.89:3300/0 conn(0x7f3854139170 msgr2=0x7f385413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f3853fff700 1 --2- 172.21.15.89:0/3976190519 >> v2:172.21.15.89:3300/0 conn(0x7f3854139170 0x7f385413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f3853fff700 1 -- 172.21.15.89:0/3976190519 >> v1:172.21.15.89:6789/0 conn(0x7f385409d6f0 legacy=0x7f385409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f3853fff700 1 -- 172.21.15.89:0/3976190519 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385409a500 con 0x7f385409e1d0 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f3853fff700 1 --2- 172.21.15.89:0/3976190519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f385409e1d0 0x7f3854138c30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3848009920 tx=0x7f3848009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d65df3980ae491c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:08.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f38537fe700 1 -- 172.21.15.89:0/3976190519 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848004230 con 0x7f385409e1d0 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f38537fe700 1 -- 172.21.15.89:0/3976190519 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848004390 con 0x7f385409e1d0 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.773+0000 7f38537fe700 1 -- 172.21.15.89:0/3976190519 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848009e70 con 0x7f385409e1d0 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.774+0000 7f385b1cf700 1 -- 172.21.15.89:0/3976190519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f385409e1d0 msgr2=0x7f3854138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.774+0000 7f385b1cf700 1 --2- 172.21.15.89:0/3976190519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f385409e1d0 0x7f3854138c30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3848009920 tx=0x7f3848009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.774+0000 7f385b1cf700 1 -- 172.21.15.89:0/3976190519 shutdown_connections 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.774+0000 7f385b1cf700 1 --2- 172.21.15.89:0/3976190519 >> v2:172.21.15.89:3300/0 conn(0x7f3854139170 0x7f385413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.774+0000 7f385b1cf700 1 --2- 172.21.15.89:0/3976190519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f385409e1d0 0x7f3854138c30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.774+0000 7f385b1cf700 1 -- 172.21.15.89:0/3976190519 >> 172.21.15.89:0/3976190519 conn(0x7f3854066f50 msgr2=0x7f385409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.774+0000 7f385b1cf700 1 -- 172.21.15.89:0/3976190519 shutdown_connections 2023-09-30T22:23:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.774+0000 7f385b1cf700 1 -- 172.21.15.89:0/3976190519 wait complete. 2023-09-30T22:23:08.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.775+0000 7f385b1cf700 1 Processor -- start 2023-09-30T22:23:08.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.776+0000 7f385b1cf700 1 -- start start 2023-09-30T22:23:08.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.776+0000 7f385b1cf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d6f0 0x7f38541c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.776+0000 7f385b1cf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409e1d0 0x7f38541c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.776+0000 7f385b1cf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854139170 0x7f385411e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.776+0000 7f385b1cf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38540a2c00 con 0x7f385409d6f0 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.776+0000 7f385b1cf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38541be2f0 con 0x7f3854139170 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.776+0000 7f385b1cf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3854122370 con 0x7f385409e1d0 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.776+0000 7f3858f6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d6f0 0x7f38541c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f3858f6b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d6f0 0x7f38541c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46852/0 (socket says 172.21.15.89:46852) 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f3858f6b700 1 -- 172.21.15.89:0/275191942 learned_addr learned my addr 172.21.15.89:0/275191942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385976c700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854139170 0x7f385411e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f3853fff700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409e1d0 0x7f38541c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:08.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385976c700 1 -- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409e1d0 msgr2=0x7f38541c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385976c700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409e1d0 0x7f38541c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385976c700 1 -- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d6f0 msgr2=0x7f38541c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385976c700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d6f0 0x7f38541c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385976c700 1 -- 172.21.15.89:0/275191942 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848009550 con 0x7f3854139170 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f3853fff700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409e1d0 0x7f38541c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f3858f6b700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d6f0 0x7f38541c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385976c700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854139170 0x7f385411e850 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f384c007d90 tx=0x7f384c00e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f38517fa700 1 -- 172.21.15.89:0/275191942 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c018070 con 0x7f3854139170 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f38517fa700 1 -- 172.21.15.89:0/275191942 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00ebe0 con 0x7f3854139170 2023-09-30T22:23:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f38517fa700 1 -- 172.21.15.89:0/275191942 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c003a20 con 0x7f3854139170 2023-09-30T22:23:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3854134720 con 0x7f3854139170 2023-09-30T22:23:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.777+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854122b00 con 0x7f3854139170 2023-09-30T22:23:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.778+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854121cb0 con 0x7f3854139170 2023-09-30T22:23:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.779+0000 7f38517fa700 1 -- 172.21.15.89:0/275191942 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f384c014420 con 0x7f3854139170 2023-09-30T22:23:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.779+0000 7f38517fa700 1 -- 172.21.15.89:0/275191942 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f384c03f7d0 con 0x7f3854139170 2023-09-30T22:23:08.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.787+0000 7f38517fa700 1 -- 172.21.15.89:0/275191942 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f384c0146d0 con 0x7f3854139170 2023-09-30T22:23:08.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.821+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faa2809dd50 con 0x7faa28133e30 2023-09-30T22:23:08.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.821+0000 7faa1e7fc700 1 -- 172.21.15.89:0/4191749842 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa18017030 con 0x7faa28133e30 2023-09-30T22:23:08.821 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28133e30 msgr2=0x7faa2809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28133e30 0x7faa2809c9c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7faa1800c530 tx=0x7faa1800c840 comp rx=0 tx=0).stop 2023-09-30T22:23:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 shutdown_connections 2023-09-30T22:23:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa28134260 0x7faa2809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa28133420 0x7faa2809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 --2- 172.21.15.89:0/4191749842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa28133e30 0x7faa2809c9c0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 >> 172.21.15.89:0/4191749842 conn(0x7faa280a4210 msgr2=0x7faa280a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 shutdown_connections 2023-09-30T22:23:08.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.823+0000 7faa2ffec700 1 -- 172.21.15.89:0/4191749842 wait complete. 2023-09-30T22:23:08.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:08.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.900+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc414074ea0 con 0x7fc41406a770 2023-09-30T22:23:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.901+0000 7fc409ffb700 1 -- 172.21.15.89:0/2618537583 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc41000ce80 con 0x7fc41406a770 2023-09-30T22:23:08.901 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:08.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 msgr2=0x7fc414120450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc414120450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc41000ebe0 tx=0x7fc41000eef0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 shutdown_connections 2023-09-30T22:23:08.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc414131d10 0x7fc414120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc41406a770 0x7fc414120450 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 --2- 172.21.15.89:0/2618537583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc41406b080 0x7fc414120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 >> 172.21.15.89:0/2618537583 conn(0x7fc414066f50 msgr2=0x7fc41406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 shutdown_connections 2023-09-30T22:23:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.903+0000 7fc41b4ef700 1 -- 172.21.15.89:0/2618537583 wait complete. 2023-09-30T22:23:08.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.934+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fda94005150 con 0x7fdaac136710 2023-09-30T22:23:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.934+0000 7fdaa27fc700 1 -- 172.21.15.89:0/2450379202 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda9c028030 con 0x7fdaac136710 2023-09-30T22:23:08.934 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.936+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac136710 msgr2=0x7fdaac119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.936+0000 7fdab3d84700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac136710 0x7fdaac119210 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00b530 tx=0x7fda9c00b840 comp rx=0 tx=0).stop 2023-09-30T22:23:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.936+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 shutdown_connections 2023-09-30T22:23:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.936+0000 7fdab3d84700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaac139030 0x7fdaac119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.936+0000 7fdab3d84700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdaac1338a0 0x7fdaac118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.936+0000 7fdab3d84700 1 --2- 172.21.15.89:0/2450379202 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdaac136710 0x7fdaac119210 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.936+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 >> 172.21.15.89:0/2450379202 conn(0x7fdaac0a2aa0 msgr2=0x7fdaac1344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.937+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 shutdown_connections 2023-09-30T22:23:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.937+0000 7fdab3d84700 1 -- 172.21.15.89:0/2450379202 wait complete. 2023-09-30T22:23:08.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.982+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3854074ea0 con 0x7f3854139170 2023-09-30T22:23:08.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.983+0000 7f38517fa700 1 -- 172.21.15.89:0/275191942 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f384c0122e0 con 0x7f3854139170 2023-09-30T22:23:08.983 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854139170 msgr2=0x7f385411e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854139170 0x7f385411e850 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f384c007d90 tx=0x7f384c00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 shutdown_connections 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3854139170 0x7f385411e850 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f385409d6f0 0x7f38541c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 --2- 172.21.15.89:0/275191942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f385409e1d0 0x7f38541c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 >> 172.21.15.89:0/275191942 conn(0x7f3854066f50 msgr2=0x7f3854070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 shutdown_connections 2023-09-30T22:23:08.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:08.985+0000 7f385b1cf700 1 -- 172.21.15.89:0/275191942 wait complete. 2023-09-30T22:23:08.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:09.838 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:09.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:09.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:09.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.975+0000 7ff1c7da9700 1 Processor -- start 2023-09-30T22:23:09.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.975+0000 7ff1c7da9700 1 -- start start 2023-09-30T22:23:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c7da9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0133540 0x7ff1c0135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c7da9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0135e60 0x7ff1c0138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c7da9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c00716a0 con 0x7ff1c0130540 2023-09-30T22:23:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c7da9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0071290 con 0x7ff1c0135e60 2023-09-30T22:23:09.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c7da9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0130340 con 0x7ff1c0133540 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c6346700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0135e60 0x7ff1c0138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c5344700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0133540 0x7ff1c0135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c6346700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0135e60 0x7ff1c0138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58516/0 (socket says 172.21.15.89:58516) 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c6346700 1 -- 172.21.15.89:0/3121242984 learned_addr learned my addr 172.21.15.89:0/3121242984 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c5344700 1 -- 172.21.15.89:0/3121242984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0135e60 msgr2=0x7ff1c0138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c5344700 1 --2- 172.21.15.89:0/3121242984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0135e60 0x7ff1c0138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c5344700 1 -- 172.21.15.89:0/3121242984 >> v1:172.21.15.89:6789/0 conn(0x7ff1c0130540 legacy=0x7ff1c0130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c5344700 1 -- 172.21.15.89:0/3121242984 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c0070b80 con 0x7ff1c0133540 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c6346700 1 --2- 172.21.15.89:0/3121242984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0135e60 0x7ff1c0138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c5344700 1 --2- 172.21.15.89:0/3121242984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0133540 0x7ff1c0135920 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000dc90 tx=0x7ff1b000f050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f7cb2dc6992eb30 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:09.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.976+0000 7ff1c4b43700 1 -- 172.21.15.89:0/3121242984 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0012070 con 0x7ff1c0133540 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c4b43700 1 -- 172.21.15.89:0/3121242984 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b000d2e0 con 0x7ff1c0133540 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c4b43700 1 -- 172.21.15.89:0/3121242984 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000f8f0 con 0x7ff1c0133540 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c7da9700 1 -- 172.21.15.89:0/3121242984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0133540 msgr2=0x7ff1c0135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c7da9700 1 --2- 172.21.15.89:0/3121242984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0133540 0x7ff1c0135920 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000dc90 tx=0x7ff1b000f050 comp rx=0 tx=0).stop 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c7da9700 1 -- 172.21.15.89:0/3121242984 shutdown_connections 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c7da9700 1 --2- 172.21.15.89:0/3121242984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0135e60 0x7ff1c0138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c7da9700 1 --2- 172.21.15.89:0/3121242984 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0133540 0x7ff1c0135920 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c7da9700 1 -- 172.21.15.89:0/3121242984 >> 172.21.15.89:0/3121242984 conn(0x7ff1c00a2aa0 msgr2=0x7ff1c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c7da9700 1 -- 172.21.15.89:0/3121242984 shutdown_connections 2023-09-30T22:23:09.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.977+0000 7ff1c7da9700 1 -- 172.21.15.89:0/3121242984 wait complete. 2023-09-30T22:23:09.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.978+0000 7ff1c7da9700 1 Processor -- start 2023-09-30T22:23:09.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.978+0000 7ff1c7da9700 1 -- start start 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c7da9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0130540 0x7ff1c009c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c7da9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0133540 0x7ff1c009cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c7da9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1c0135e60 0x7ff1c009d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c7da9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0119b90 con 0x7ff1c0135e60 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c7da9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0119110 con 0x7ff1c0133540 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c7da9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0135930 con 0x7ff1c0130540 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5b45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0130540 0x7ff1c009c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5344700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0133540 0x7ff1c009cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5b45700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0130540 0x7ff1c009c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44924/0 (socket says 172.21.15.89:44924) 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5b45700 1 -- 172.21.15.89:0/612182495 learned_addr learned my addr 172.21.15.89:0/612182495 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c6346700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1c0135e60 0x7ff1c009d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5b45700 1 -- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0133540 msgr2=0x7ff1c009cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5b45700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0133540 0x7ff1c009cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5b45700 1 -- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1c0135e60 msgr2=0x7ff1c009d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5b45700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1c0135e60 0x7ff1c009d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5b45700 1 -- 172.21.15.89:0/612182495 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009b30 con 0x7ff1c0130540 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.979+0000 7ff1c5344700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0133540 0x7ff1c009cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.980+0000 7ff1c5b45700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0130540 0x7ff1c009c710 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00c450 tx=0x7ff1bc00c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.980+0000 7ff1b67fc700 1 -- 172.21.15.89:0/612182495 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0052d0 con 0x7ff1c0130540 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.980+0000 7ff1b67fc700 1 -- 172.21.15.89:0/612182495 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc005430 con 0x7ff1c0130540 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.980+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b000d940 con 0x7ff1c0130540 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.980+0000 7ff1b67fc700 1 -- 172.21.15.89:0/612182495 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0038c0 con 0x7ff1c0130540 2023-09-30T22:23:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.980+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c011a500 con 0x7ff1c0130540 2023-09-30T22:23:09.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.981+0000 7ff1b67fc700 1 -- 172.21.15.89:0/612182495 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1bc003a20 con 0x7ff1c0130540 2023-09-30T22:23:09.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.981+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c0074ea0 con 0x7ff1c0130540 2023-09-30T22:23:09.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.981+0000 7ff1b67fc700 1 -- 172.21.15.89:0/612182495 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff1bc03e9e0 con 0x7ff1c0130540 2023-09-30T22:23:09.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:09.986+0000 7ff1b67fc700 1 -- 172.21.15.89:0/612182495 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1bc011590 con 0x7ff1c0130540 2023-09-30T22:23:09.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:10.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.051+0000 7fa2bcd0e700 1 Processor -- start 2023-09-30T22:23:10.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2bcd0e700 1 -- start start 2023-09-30T22:23:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2bcd0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809d6f0 0x7fa2b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2bcd0e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809e000 0x7fa2b8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2bcd0e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b809f130 con 0x7fa2b809e410 2023-09-30T22:23:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2bcd0e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809f520 con 0x7fa2b809e000 2023-09-30T22:23:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2bcd0e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809e930 con 0x7fa2b809d6f0 2023-09-30T22:23:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2b5d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809e000 0x7fa2b8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2b6d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa2b809e410 0x7fa2b8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35252/0 (socket says 172.21.15.89:35252) 2023-09-30T22:23:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2b6d9d700 1 -- 172.21.15.89:0/1049808311 learned_addr learned my addr 172.21.15.89:0/1049808311 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.052+0000 7fa2b659c700 1 --2- 172.21.15.89:0/1049808311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809d6f0 0x7fa2b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b5d9b700 1 -- 172.21.15.89:0/1049808311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809d6f0 msgr2=0x7fa2b809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b5d9b700 1 --2- 172.21.15.89:0/1049808311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809d6f0 0x7fa2b809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b5d9b700 1 -- 172.21.15.89:0/1049808311 >> v1:172.21.15.89:6789/0 conn(0x7fa2b809e410 legacy=0x7fa2b8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b559a700 1 -- 172.21.15.89:0/1049808311 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090923237 0 0) 0x7fa2b809f130 con 0x7fa2b809e410 2023-09-30T22:23:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b5d9b700 1 -- 172.21.15.89:0/1049808311 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b809b2d0 con 0x7fa2b809e000 2023-09-30T22:23:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b5d9b700 1 --2- 172.21.15.89:0/1049808311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809e000 0x7fa2b8130660 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009870 tx=0x7fa2ac009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808d3b1b6e143635 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b559a700 1 -- 172.21.15.89:0/1049808311 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac012b80 con 0x7fa2b809e000 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b559a700 1 -- 172.21.15.89:0/1049808311 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ac012ce0 con 0x7fa2b809e000 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.053+0000 7fa2b559a700 1 -- 172.21.15.89:0/1049808311 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac01b620 con 0x7fa2b809e000 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.054+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1049808311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809e000 msgr2=0x7fa2b8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.054+0000 7fa2bcd0e700 1 --2- 172.21.15.89:0/1049808311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809e000 0x7fa2b8130660 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009870 tx=0x7fa2ac009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.054+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1049808311 shutdown_connections 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.054+0000 7fa2bcd0e700 1 --2- 172.21.15.89:0/1049808311 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809d6f0 0x7fa2b809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.054+0000 7fa2bcd0e700 1 --2- 172.21.15.89:0/1049808311 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809e000 0x7fa2b8130660 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.054+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1049808311 >> 172.21.15.89:0/1049808311 conn(0x7fa2b8066f50 msgr2=0x7fa2b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.054+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1049808311 shutdown_connections 2023-09-30T22:23:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.054+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1049808311 wait complete. 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.055+0000 7fa2bcd0e700 1 Processor -- start 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2bcd0e700 1 -- start start 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2bcd0e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809d6f0 0x7fa2b811de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2bcd0e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809e000 0x7fa2b811e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2bcd0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b809e410 0x7fa2b811e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2bcd0e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809c700 con 0x7fa2b809e410 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2bcd0e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b811ff90 con 0x7fa2b809d6f0 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2bcd0e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8121070 con 0x7fa2b809e000 2023-09-30T22:23:10.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2b659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809d6f0 0x7fa2b811de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2b659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809d6f0 0x7fa2b811de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58546/0 (socket says 172.21.15.89:58546) 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.056+0000 7fa2b659c700 1 -- 172.21.15.89:0/1725214854 learned_addr learned my addr 172.21.15.89:0/1725214854 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b6d9d700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b809e410 0x7fa2b811e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b5d9b700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809e000 0x7fa2b811e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b659c700 1 -- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809e000 msgr2=0x7fa2b811e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b659c700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809e000 0x7fa2b811e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b659c700 1 -- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b809e410 msgr2=0x7fa2b811e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b659c700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b809e410 0x7fa2b811e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b659c700 1 -- 172.21.15.89:0/1725214854 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac0094a0 con 0x7fa2b809d6f0 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b6d9d700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b809e410 0x7fa2b811e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:10.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2b659c700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809d6f0 0x7fa2b811de60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e9b0 tx=0x7fa2a800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:10.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2a6ffd700 1 -- 172.21.15.89:0/1725214854 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8009710 con 0x7fa2b809d6f0 2023-09-30T22:23:10.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2a6ffd700 1 -- 172.21.15.89:0/1725214854 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a800c850 con 0x7fa2b809d6f0 2023-09-30T22:23:10.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b80a2020 con 0x7fa2b809d6f0 2023-09-30T22:23:10.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2a6ffd700 1 -- 172.21.15.89:0/1725214854 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800cbc0 con 0x7fa2b809d6f0 2023-09-30T22:23:10.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.057+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b8120c20 con 0x7fa2b809d6f0 2023-09-30T22:23:10.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.058+0000 7fa2a6ffd700 1 -- 172.21.15.89:0/1725214854 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2a800cd20 con 0x7fa2b809d6f0 2023-09-30T22:23:10.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.058+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b811ac20 con 0x7fa2b809d6f0 2023-09-30T22:23:10.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.058+0000 7fa2a6ffd700 1 -- 172.21.15.89:0/1725214854 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa2a803fbe0 con 0x7fa2b809d6f0 2023-09-30T22:23:10.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.067+0000 7fa2a6ffd700 1 -- 172.21.15.89:0/1725214854 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa2a801b3b0 con 0x7fa2b809d6f0 2023-09-30T22:23:10.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.101+0000 7f65d6575700 1 Processor -- start 2023-09-30T22:23:10.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.101+0000 7f65d6575700 1 -- start start 2023-09-30T22:23:10.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.101+0000 7f65d6575700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d6575700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d6575700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d00716a0 con 0x7f65d0130e50 2023-09-30T22:23:10.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d6575700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d009a390 con 0x7f65d0130540 2023-09-30T22:23:10.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d6575700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0071290 con 0x7f65d006fcf0 2023-09-30T22:23:10.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65cffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65cffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46888/0 (socket says 172.21.15.89:46888) 2023-09-30T22:23:10.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65cffff700 1 -- 172.21.15.89:0/2307867290 learned_addr learned my addr 172.21.15.89:0/2307867290 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:10.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d4b12700 1 --2- 172.21.15.89:0/2307867290 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65ceffd700 1 -- 172.21.15.89:0/2307867290 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551428541 0 0) 0x7f65d00716a0 con 0x7f65d0130e50 2023-09-30T22:23:10.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65ceffd700 1 -- 172.21.15.89:0/2307867290 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b80057c0 con 0x7f65d0130e50 2023-09-30T22:23:10.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d4b12700 1 -- 172.21.15.89:0/2307867290 >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 msgr2=0x7f65d0130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:10.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d4b12700 1 --2- 172.21.15.89:0/2307867290 >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d4b12700 1 -- 172.21.15.89:0/2307867290 >> v1:172.21.15.89:6789/0 conn(0x7f65d0130e50 legacy=0x7f65d006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d4b12700 1 -- 172.21.15.89:0/2307867290 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d0130340 con 0x7f65d006fcf0 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65cffff700 1 --2- 172.21.15.89:0/2307867290 >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.102+0000 7f65d4b12700 1 --2- 172.21.15.89:0/2307867290 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009dc0 tx=0x7f65c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dccabe61920aa14c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65ceffd700 1 -- 172.21.15.89:0/2307867290 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800e070 con 0x7f65d006fcf0 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65ceffd700 1 -- 172.21.15.89:0/2307867290 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c800bae0 con 0x7f65d006fcf0 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65ceffd700 1 -- 172.21.15.89:0/2307867290 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800be90 con 0x7f65d006fcf0 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65d6575700 1 -- 172.21.15.89:0/2307867290 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 msgr2=0x7f65d0138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65d6575700 1 --2- 172.21.15.89:0/2307867290 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009dc0 tx=0x7f65c800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65d6575700 1 -- 172.21.15.89:0/2307867290 shutdown_connections 2023-09-30T22:23:10.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65d6575700 1 --2- 172.21.15.89:0/2307867290 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d006fcf0 0x7f65d0138570 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65d6575700 1 --2- 172.21.15.89:0/2307867290 >> v2:172.21.15.89:3300/0 conn(0x7f65d0130540 0x7f65d0130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65d6575700 1 -- 172.21.15.89:0/2307867290 >> 172.21.15.89:0/2307867290 conn(0x7f65d00a2aa0 msgr2=0x7f65d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:10.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65d6575700 1 -- 172.21.15.89:0/2307867290 shutdown_connections 2023-09-30T22:23:10.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.103+0000 7f65d6575700 1 -- 172.21.15.89:0/2307867290 wait complete. 2023-09-30T22:23:10.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.104+0000 7f65d6575700 1 Processor -- start 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.104+0000 7f65d6575700 1 -- start start 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d6575700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d006fcf0 0x7f65d011a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d6575700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 0x7f65d011ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d6575700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130e50 0x7f65d011b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d6575700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d011c150 con 0x7f65d0130540 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d6575700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d011bae0 con 0x7f65d0130e50 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d6575700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65d011dbc0 con 0x7f65d006fcf0 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d4b12700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130e50 0x7f65d011b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d4b12700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130e50 0x7f65d011b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58570/0 (socket says 172.21.15.89:58570) 2023-09-30T22:23:10.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d4b12700 1 -- 172.21.15.89:0/2526652642 learned_addr learned my addr 172.21.15.89:0/2526652642 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65cffff700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d006fcf0 0x7f65d011a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65cf7fe700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 0x7f65d011ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d4b12700 1 -- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d006fcf0 msgr2=0x7f65d011a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d4b12700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d006fcf0 0x7f65d011a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d4b12700 1 -- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 msgr2=0x7f65d011ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d4b12700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 0x7f65d011ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65d4b12700 1 -- 172.21.15.89:0/2526652642 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0009550 con 0x7f65d0130e50 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.105+0000 7f65cffff700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d006fcf0 0x7f65d011a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.106+0000 7f65d4b12700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130e50 0x7f65d011b0b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f65c800b370 tx=0x7f65c8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.106+0000 7f65ccff9700 1 -- 172.21.15.89:0/2526652642 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8004960 con 0x7f65d0130e50 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.106+0000 7f65ccff9700 1 -- 172.21.15.89:0/2526652642 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c8020070 con 0x7f65d0130e50 2023-09-30T22:23:10.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.106+0000 7f65ccff9700 1 -- 172.21.15.89:0/2526652642 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8026650 con 0x7f65d0130e50 2023-09-30T22:23:10.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.106+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c8009a70 con 0x7f65d0130e50 2023-09-30T22:23:10.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.106+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d011c950 con 0x7f65d0130e50 2023-09-30T22:23:10.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.107+0000 7f65ccff9700 1 -- 172.21.15.89:0/2526652642 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f65c8012430 con 0x7f65d0130e50 2023-09-30T22:23:10.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.107+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d011dd50 con 0x7f65d0130e50 2023-09-30T22:23:10.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.107+0000 7f65ccff9700 1 -- 172.21.15.89:0/2526652642 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f65c8022070 con 0x7f65d0130e50 2023-09-30T22:23:10.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.112+0000 7f65ccff9700 1 -- 172.21.15.89:0/2526652642 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65d011dd50 con 0x7f65d0130e50 2023-09-30T22:23:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.137+0000 7f1d98a38700 1 Processor -- start 2023-09-30T22:23:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.137+0000 7f1d98a38700 1 -- start start 2023-09-30T22:23:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d98a38700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1d9412cf60 0x7f1d9406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d98a38700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412f8b0 0x7f1d9412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d98a38700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9412cd90 con 0x7f1d94130260 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d98a38700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9412d330 con 0x7f1d9412cf60 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d98a38700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d941306d0 con 0x7f1d9412f8b0 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d91d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412f8b0 0x7f1d9412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d9259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1d9412cf60 0x7f1d9406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d92d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1d94130260 0x7f1d94136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35270/0 (socket says 172.21.15.89:35270) 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d91d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412f8b0 0x7f1d9412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58572/0 (socket says 172.21.15.89:58572) 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d92d9d700 1 -- 172.21.15.89:0/4038988656 learned_addr learned my addr 172.21.15.89:0/4038988656 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d9259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1d9412cf60 0x7f1d9406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46914/0 (socket says 172.21.15.89:46914) 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d91d9b700 1 -- 172.21.15.89:0/4038988656 >> v2:172.21.15.89:3300/0 conn(0x7f1d9412cf60 msgr2=0x7f1d9406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d91d9b700 1 --2- 172.21.15.89:0/4038988656 >> v2:172.21.15.89:3300/0 conn(0x7f1d9412cf60 0x7f1d9406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d91d9b700 1 -- 172.21.15.89:0/4038988656 >> v1:172.21.15.89:6789/0 conn(0x7f1d94130260 legacy=0x7f1d94136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d91d9b700 1 -- 172.21.15.89:0/4038988656 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94071780 con 0x7f1d9412f8b0 2023-09-30T22:23:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.138+0000 7f1d91d9b700 1 --2- 172.21.15.89:0/4038988656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412f8b0 0x7f1d9412fd20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800dc60 tx=0x7f1d8800df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd665a775fe4c0a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d9159a700 1 -- 172.21.15.89:0/4038988656 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88012070 con 0x7f1d9412f8b0 2023-09-30T22:23:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d9159a700 1 -- 172.21.15.89:0/4038988656 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8800d2e0 con 0x7f1d9412f8b0 2023-09-30T22:23:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d9159a700 1 -- 172.21.15.89:0/4038988656 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800f900 con 0x7f1d9412f8b0 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d98a38700 1 -- 172.21.15.89:0/4038988656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412f8b0 msgr2=0x7f1d9412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d98a38700 1 --2- 172.21.15.89:0/4038988656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412f8b0 0x7f1d9412fd20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800dc60 tx=0x7f1d8800df70 comp rx=0 tx=0).stop 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d98a38700 1 -- 172.21.15.89:0/4038988656 shutdown_connections 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d98a38700 1 --2- 172.21.15.89:0/4038988656 >> v2:172.21.15.89:3300/0 conn(0x7f1d9412cf60 0x7f1d9406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d98a38700 1 --2- 172.21.15.89:0/4038988656 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412f8b0 0x7f1d9412fd20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d98a38700 1 -- 172.21.15.89:0/4038988656 >> 172.21.15.89:0/4038988656 conn(0x7f1d940a2aa0 msgr2=0x7f1d940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.139+0000 7f1d98a38700 1 -- 172.21.15.89:0/4038988656 shutdown_connections 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.140+0000 7f1d98a38700 1 -- 172.21.15.89:0/4038988656 wait complete. 2023-09-30T22:23:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d98a38700 1 Processor -- start 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d98a38700 1 -- start start 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d98a38700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412cf60 0x7f1d941c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d98a38700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9412f8b0 0x7f1d941c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d98a38700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94130260 0x7f1d9409b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d98a38700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d941b8ea0 con 0x7f1d9412f8b0 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d98a38700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94129930 con 0x7f1d9412cf60 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d98a38700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9409d170 con 0x7f1d94130260 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d9259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412cf60 0x7f1d941c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d9259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412cf60 0x7f1d941c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58574/0 (socket says 172.21.15.89:58574) 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d91d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9412f8b0 0x7f1d941c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d9259c700 1 -- 172.21.15.89:0/2583231717 learned_addr learned my addr 172.21.15.89:0/2583231717 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.141+0000 7f1d92d9d700 1 --2- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94130260 0x7f1d9409b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d91d9b700 1 -- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94130260 msgr2=0x7f1d9409b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d91d9b700 1 --2- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94130260 0x7f1d9409b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d91d9b700 1 -- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412cf60 msgr2=0x7f1d941c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d91d9b700 1 --2- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412cf60 0x7f1d941c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d91d9b700 1 -- 172.21.15.89:0/2583231717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8800d940 con 0x7f1d9412f8b0 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d91d9b700 1 --2- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9412f8b0 0x7f1d941c4850 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f1d88019000 tx=0x7f1d88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d82ffd700 1 -- 172.21.15.89:0/2583231717 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880036a0 con 0x7f1d9412f8b0 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d82ffd700 1 -- 172.21.15.89:0/2583231717 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d88003800 con 0x7f1d9412f8b0 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9412f520 con 0x7f1d9412f8b0 2023-09-30T22:23:10.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d82ffd700 1 -- 172.21.15.89:0/2583231717 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88003bb0 con 0x7f1d9412f8b0 2023-09-30T22:23:10.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.142+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9409cbc0 con 0x7f1d9412f8b0 2023-09-30T22:23:10.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.143+0000 7f1d82ffd700 1 -- 172.21.15.89:0/2583231717 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d88003d10 con 0x7f1d9412f8b0 2023-09-30T22:23:10.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.143+0000 7f1d82ffd700 1 -- 172.21.15.89:0/2583231717 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d880447b0 con 0x7f1d9412f8b0 2023-09-30T22:23:10.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.143+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9409e490 con 0x7f1d9412f8b0 2023-09-30T22:23:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.152+0000 7f1d82ffd700 1 -- 172.21.15.89:0/2583231717 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d8800b970 con 0x7f1d9412f8b0 2023-09-30T22:23:10.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.197+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff1c011a1a0 con 0x7ff1c0130540 2023-09-30T22:23:10.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.197+0000 7ff1b67fc700 1 -- 172.21.15.89:0/612182495 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1bc017070 con 0x7ff1c0130540 2023-09-30T22:23:10.199 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.200+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0130540 msgr2=0x7ff1c009c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.200+0000 7ff1c7da9700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0130540 0x7ff1c009c710 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00c450 tx=0x7ff1bc00c760 comp rx=0 tx=0).stop 2023-09-30T22:23:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.201+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 shutdown_connections 2023-09-30T22:23:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.201+0000 7ff1c7da9700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1c0135e60 0x7ff1c009d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.201+0000 7ff1c7da9700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff1c0130540 0x7ff1c009c710 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.201+0000 7ff1c7da9700 1 --2- 172.21.15.89:0/612182495 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff1c0133540 0x7ff1c009cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.201+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 >> 172.21.15.89:0/612182495 conn(0x7ff1c00a2aa0 msgr2=0x7ff1c0131320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.201+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 shutdown_connections 2023-09-30T22:23:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.201+0000 7ff1c7da9700 1 -- 172.21.15.89:0/612182495 wait complete. 2023-09-30T22:23:10.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.268+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa2b8074ea0 con 0x7fa2b809d6f0 2023-09-30T22:23:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.268+0000 7fa2a6ffd700 1 -- 172.21.15.89:0/1725214854 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2a8017020 con 0x7fa2b809d6f0 2023-09-30T22:23:10.268 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:10.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.271+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809d6f0 msgr2=0x7fa2b811de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.271+0000 7fa2bcd0e700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809d6f0 0x7fa2b811de60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e9b0 tx=0x7fa2a800ed70 comp rx=0 tx=0).stop 2023-09-30T22:23:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.271+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 shutdown_connections 2023-09-30T22:23:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.271+0000 7fa2bcd0e700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b809e410 0x7fa2b811e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.271+0000 7fa2bcd0e700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa2b809d6f0 0x7fa2b811de60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.271+0000 7fa2bcd0e700 1 --2- 172.21.15.89:0/1725214854 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2b809e000 0x7fa2b811e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.271+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 >> 172.21.15.89:0/1725214854 conn(0x7fa2b8066f50 msgr2=0x7fa2b8130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.271+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 shutdown_connections 2023-09-30T22:23:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.272+0000 7fa2bcd0e700 1 -- 172.21.15.89:0/1725214854 wait complete. 2023-09-30T22:23:10.288 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.328+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f65d011dd00 con 0x7f65d0130e50 2023-09-30T22:23:10.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.329+0000 7f65ccff9700 1 -- 172.21.15.89:0/2526652642 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65c802b070 con 0x7f65d0130e50 2023-09-30T22:23:10.329 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130e50 msgr2=0x7f65d011b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130e50 0x7f65d011b0b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f65c800b370 tx=0x7f65c8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 shutdown_connections 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f65d0130e50 0x7f65d011b0b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f65d006fcf0 0x7f65d011a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 --2- 172.21.15.89:0/2526652642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65d0130540 0x7f65d011ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 >> 172.21.15.89:0/2526652642 conn(0x7f65d00a2aa0 msgr2=0x7f65d012d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 shutdown_connections 2023-09-30T22:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.331+0000 7f65d6575700 1 -- 172.21.15.89:0/2526652642 wait complete. 2023-09-30T22:23:10.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:10.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.355+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1d94074ea0 con 0x7f1d9412f8b0 2023-09-30T22:23:10.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.355+0000 7f1d82ffd700 1 -- 172.21.15.89:0/2583231717 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d8802e030 con 0x7f1d9412f8b0 2023-09-30T22:23:10.355 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:10.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9412f8b0 msgr2=0x7f1d941c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:10.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 --2- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9412f8b0 0x7f1d941c4850 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f1d88019000 tx=0x7f1d88004750 comp rx=0 tx=0).stop 2023-09-30T22:23:10.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 shutdown_connections 2023-09-30T22:23:10.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 --2- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1d94130260 0x7f1d9409b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 --2- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d9412cf60 0x7f1d941c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 --2- 172.21.15.89:0/2583231717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9412f8b0 0x7f1d941c4850 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:10.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 >> 172.21.15.89:0/2583231717 conn(0x7f1d940a2aa0 msgr2=0x7f1d9406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:10.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 shutdown_connections 2023-09-30T22:23:10.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:10.357+0000 7f1d98a38700 1 -- 172.21.15.89:0/2583231717 wait complete. 2023-09-30T22:23:10.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:11.218 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:11.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:11.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:11.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.361+0000 7f79d0f34700 1 Processor -- start 2023-09-30T22:23:11.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.361+0000 7f79d0f34700 1 -- start start 2023-09-30T22:23:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.361+0000 7f79d0f34700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.361+0000 7f79d0f34700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79cc135e50 0x7f79cc138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.361+0000 7f79d0f34700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79cc09f130 con 0x7f79cc09d6f0 2023-09-30T22:23:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.361+0000 7f79d0f34700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79cc09f520 con 0x7f79cc135e50 2023-09-30T22:23:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.361+0000 7f79d0f34700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc09e930 con 0x7f79cc09e1d0 2023-09-30T22:23:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47762/0 (socket says 172.21.15.89:47762) 2023-09-30T22:23:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79ca59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f79cc09d6f0 0x7f79cc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47870/0 (socket says 172.21.15.89:47870) 2023-09-30T22:23:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c9d9b700 1 -- 172.21.15.89:0/3395004730 learned_addr learned my addr 172.21.15.89:0/3395004730 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79cad9d700 1 --2- 172.21.15.89:0/3395004730 >> v2:172.21.15.89:3300/0 conn(0x7f79cc135e50 0x7f79cc138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c9d9b700 1 -- 172.21.15.89:0/3395004730 >> v2:172.21.15.89:3300/0 conn(0x7f79cc135e50 msgr2=0x7f79cc138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c9d9b700 1 --2- 172.21.15.89:0/3395004730 >> v2:172.21.15.89:3300/0 conn(0x7f79cc135e50 0x7f79cc138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c9d9b700 1 -- 172.21.15.89:0/3395004730 >> v1:172.21.15.89:6789/0 conn(0x7f79cc09d6f0 legacy=0x7f79cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c9d9b700 1 -- 172.21.15.89:0/3395004730 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc09b2d0 con 0x7f79cc09e1d0 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79cad9d700 1 --2- 172.21.15.89:0/3395004730 >> v2:172.21.15.89:3300/0 conn(0x7f79cc135e50 0x7f79cc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c9d9b700 1 --2- 172.21.15.89:0/3395004730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc135910 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009920 tx=0x7f79c0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd2039f6c8fd47a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c959a700 1 -- 172.21.15.89:0/3395004730 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000f8a0 con 0x7f79cc09e1d0 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c959a700 1 -- 172.21.15.89:0/3395004730 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c000fa00 con 0x7f79cc09e1d0 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.362+0000 7f79c959a700 1 -- 172.21.15.89:0/3395004730 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000fd90 con 0x7f79cc09e1d0 2023-09-30T22:23:11.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.363+0000 7f79d0f34700 1 -- 172.21.15.89:0/3395004730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 msgr2=0x7f79cc135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.363+0000 7f79d0f34700 1 --2- 172.21.15.89:0/3395004730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc135910 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009920 tx=0x7f79c0009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.363+0000 7f79d0f34700 1 -- 172.21.15.89:0/3395004730 shutdown_connections 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.363+0000 7f79d0f34700 1 --2- 172.21.15.89:0/3395004730 >> v2:172.21.15.89:3300/0 conn(0x7f79cc135e50 0x7f79cc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.363+0000 7f79d0f34700 1 --2- 172.21.15.89:0/3395004730 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc135910 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.363+0000 7f79d0f34700 1 -- 172.21.15.89:0/3395004730 >> 172.21.15.89:0/3395004730 conn(0x7f79cc066f50 msgr2=0x7f79cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.363+0000 7f79d0f34700 1 -- 172.21.15.89:0/3395004730 shutdown_connections 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.363+0000 7f79d0f34700 1 -- 172.21.15.89:0/3395004730 wait complete. 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.364+0000 7f79d0f34700 1 Processor -- start 2023-09-30T22:23:11.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.364+0000 7f79d0f34700 1 -- start start 2023-09-30T22:23:11.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79d0f34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79cc09d6f0 0x7f79cc1c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79d0f34700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc1c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79d0f34700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79cc135e50 0x7f79cc11b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79d0f34700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc11d260 con 0x7f79cc09d6f0 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79d0f34700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc11d070 con 0x7f79cc135e50 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79d0f34700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc0a1f90 con 0x7f79cc09e1d0 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79cc135e50 0x7f79cc11b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79c9d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc1c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79cc135e50 0x7f79cc11b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37180/0 (socket says 172.21.15.89:37180) 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 -- 172.21.15.89:0/3432743644 learned_addr learned my addr 172.21.15.89:0/3432743644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79c9d9b700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc1c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47770/0 (socket says 172.21.15.89:47770) 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 -- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 msgr2=0x7f79cc1c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc1c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 -- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79cc09d6f0 msgr2=0x7f79cc1c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79cc09d6f0 0x7f79cc1c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 -- 172.21.15.89:0/3432743644 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c4009a70 con 0x7f79cc135e50 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79c9d9b700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc1c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.365+0000 7f79cad9d700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79cc135e50 0x7f79cc11b460 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009ea0 tx=0x7f79c400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.366+0000 7f79baffd700 1 -- 172.21.15.89:0/3432743644 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c400cd40 con 0x7f79cc135e50 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.366+0000 7f79baffd700 1 -- 172.21.15.89:0/3432743644 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c400f040 con 0x7f79cc135e50 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.366+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c0009550 con 0x7f79cc135e50 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.366+0000 7f79baffd700 1 -- 172.21.15.89:0/3432743644 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c40146c0 con 0x7f79cc135e50 2023-09-30T22:23:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.366+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc132240 con 0x7f79cc135e50 2023-09-30T22:23:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.367+0000 7f79baffd700 1 -- 172.21.15.89:0/3432743644 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79c4014820 con 0x7f79cc135e50 2023-09-30T22:23:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.367+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79cc074ea0 con 0x7f79cc135e50 2023-09-30T22:23:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.367+0000 7f79baffd700 1 -- 172.21.15.89:0/3432743644 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f79c4040450 con 0x7f79cc135e50 2023-09-30T22:23:11.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.372+0000 7f79baffd700 1 -- 172.21.15.89:0/3432743644 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79c401d400 con 0x7f79cc135e50 2023-09-30T22:23:11.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.433+0000 7f8957150700 1 Processor -- start 2023-09-30T22:23:11.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.433+0000 7f8957150700 1 -- start start 2023-09-30T22:23:11.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f8957150700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950133540 0x7f8950135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f8957150700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f8957150700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89500716a0 con 0x7f8950130540 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f8957150700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8950071290 con 0x7f8950133540 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f8957150700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8950130340 con 0x7f8950135e60 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f8954eec700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8950130540 0x7f8950130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47880/0 (socket says 172.21.15.89:47880) 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f894ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950133540 0x7f8950135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f8954eec700 1 -- 172.21.15.89:0/606151501 learned_addr learned my addr 172.21.15.89:0/606151501 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f89556ed700 1 --2- 172.21.15.89:0/606151501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f894f7fe700 1 -- 172.21.15.89:0/606151501 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777830648 0 0) 0x7f89500716a0 con 0x7f8950130540 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.434+0000 7f894f7fe700 1 -- 172.21.15.89:0/606151501 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89400035b0 con 0x7f8950130540 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.435+0000 7f89556ed700 1 -- 172.21.15.89:0/606151501 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950133540 msgr2=0x7f8950135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.435+0000 7f89556ed700 1 --2- 172.21.15.89:0/606151501 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950133540 0x7f8950135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.435+0000 7f89556ed700 1 -- 172.21.15.89:0/606151501 >> v1:172.21.15.89:6789/0 conn(0x7f8950130540 legacy=0x7f8950130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.435+0000 7f89556ed700 1 -- 172.21.15.89:0/606151501 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950070b80 con 0x7f8950135e60 2023-09-30T22:23:11.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.435+0000 7f89556ed700 1 --2- 172.21.15.89:0/606151501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138240 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8948009dc0 tx=0x7f894800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fb5c0ff8c4ee74f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:11.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.435+0000 7f894f7fe700 1 -- 172.21.15.89:0/606151501 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894800e070 con 0x7f8950135e60 2023-09-30T22:23:11.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.435+0000 7f894f7fe700 1 -- 172.21.15.89:0/606151501 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894800bb20 con 0x7f8950135e60 2023-09-30T22:23:11.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.435+0000 7f894f7fe700 1 -- 172.21.15.89:0/606151501 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948012430 con 0x7f8950135e60 2023-09-30T22:23:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.436+0000 7f8957150700 1 -- 172.21.15.89:0/606151501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 msgr2=0x7f8950138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.436+0000 7f8957150700 1 --2- 172.21.15.89:0/606151501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138240 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8948009dc0 tx=0x7f894800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.436+0000 7f8957150700 1 -- 172.21.15.89:0/606151501 shutdown_connections 2023-09-30T22:23:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.436+0000 7f8957150700 1 --2- 172.21.15.89:0/606151501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138240 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.436+0000 7f8957150700 1 --2- 172.21.15.89:0/606151501 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950133540 0x7f8950135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.436+0000 7f8957150700 1 -- 172.21.15.89:0/606151501 >> 172.21.15.89:0/606151501 conn(0x7f89500a2aa0 msgr2=0x7f89500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.436+0000 7f8957150700 1 -- 172.21.15.89:0/606151501 shutdown_connections 2023-09-30T22:23:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.436+0000 7f8957150700 1 -- 172.21.15.89:0/606151501 wait complete. 2023-09-30T22:23:11.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.437+0000 7f8957150700 1 Processor -- start 2023-09-30T22:23:11.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.437+0000 7f8957150700 1 -- start start 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.438+0000 7f8957150700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950130540 0x7f8950137bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.438+0000 7f8957150700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8950133540 0x7f89501380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.438+0000 7f8957150700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.438+0000 7f8957150700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895009c470 con 0x7f8950133540 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.438+0000 7f8957150700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895009e4a0 con 0x7f8950130540 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.438+0000 7f8957150700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f895009c730 con 0x7f8950135e60 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f8954eec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950130540 0x7f8950137bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47792/0 (socket says 172.21.15.89:47792) 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f8954eec700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950130540 0x7f8950137bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37200/0 (socket says 172.21.15.89:37200) 2023-09-30T22:23:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 -- 172.21.15.89:0/1067635718 learned_addr learned my addr 172.21.15.89:0/1067635718 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f894ffff700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8950133540 0x7f89501380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 -- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950130540 msgr2=0x7f8950137bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950130540 0x7f8950137bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 -- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8950133540 msgr2=0x7f89501380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8950133540 0x7f89501380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 -- 172.21.15.89:0/1067635718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8948009a70 con 0x7f8950135e60 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f894ffff700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8950133540 0x7f89501380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f89556ed700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138630 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8948014040 tx=0x7f8948004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f894d7fa700 1 -- 172.21.15.89:0/1067635718 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948012430 con 0x7f8950135e60 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f894d7fa700 1 -- 172.21.15.89:0/1067635718 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8948020070 con 0x7f8950135e60 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895009d5f0 con 0x7f8950135e60 2023-09-30T22:23:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.439+0000 7f894d7fa700 1 -- 172.21.15.89:0/1067635718 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948012e40 con 0x7f8950135e60 2023-09-30T22:23:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.440+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895009ab00 con 0x7f8950135e60 2023-09-30T22:23:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.440+0000 7f894d7fa700 1 -- 172.21.15.89:0/1067635718 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8948004040 con 0x7f8950135e60 2023-09-30T22:23:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.441+0000 7f894d7fa700 1 -- 172.21.15.89:0/1067635718 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8948022070 con 0x7f8950135e60 2023-09-30T22:23:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.441+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895009acf0 con 0x7f8950135e60 2023-09-30T22:23:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.449+0000 7f894d7fa700 1 -- 172.21.15.89:0/1067635718 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f894801c400 con 0x7f8950135e60 2023-09-30T22:23:11.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.492+0000 7f66f733d700 1 Processor -- start 2023-09-30T22:23:11.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.492+0000 7f66f733d700 1 -- start start 2023-09-30T22:23:11.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f733d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66f01338c0 0x7f66f0133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f733d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 0x7f66f009de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f733d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f009e390 0x7f66f009e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f733d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f0070c80 con 0x7f66f01338c0 2023-09-30T22:23:11.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f733d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f009ed20 con 0x7f66f01341d0 2023-09-30T22:23:11.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f733d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66f006f720 con 0x7f66f009e390 2023-09-30T22:23:11.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f50d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66f01338c0 0x7f66f0133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f58da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f009e390 0x7f66f009e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f50d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66f01338c0 0x7f66f0133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45780/0 (socket says 172.21.15.89:45780) 2023-09-30T22:23:11.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.493+0000 7f66f50d9700 1 -- 172.21.15.89:0/3514386470 learned_addr learned my addr 172.21.15.89:0/3514386470 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:11.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66f48d8700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 0x7f66f009de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66f50d9700 1 -- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f009e390 msgr2=0x7f66f009e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66f50d9700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f009e390 0x7f66f009e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66f50d9700 1 -- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 msgr2=0x7f66f009de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66f50d9700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 0x7f66f009de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66f50d9700 1 -- 172.21.15.89:0/3514386470 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f006fce0 con 0x7f66f01338c0 2023-09-30T22:23:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66f48d8700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 0x7f66f009de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66f50d9700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f01338c0 0x7f66f0133c90 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0098f0 tx=0x7f66ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985e77bc629679d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:11.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66e7fff700 1 -- 172.21.15.89:0/3514386470 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec011830 con 0x7f66f01338c0 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66e7fff700 1 -- 172.21.15.89:0/3514386470 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec011990 con 0x7f66f01338c0 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.494+0000 7f66e7fff700 1 -- 172.21.15.89:0/3514386470 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec011d20 con 0x7f66f01338c0 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.495+0000 7f66f733d700 1 -- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f01338c0 msgr2=0x7f66f0133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.495+0000 7f66f733d700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f01338c0 0x7f66f0133c90 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0098f0 tx=0x7f66ec009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.495+0000 7f66f733d700 1 -- 172.21.15.89:0/3514386470 shutdown_connections 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.495+0000 7f66f733d700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f009e390 0x7f66f009e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.495+0000 7f66f733d700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f01338c0 0x7f66f0133c90 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.495+0000 7f66f733d700 1 --2- 172.21.15.89:0/3514386470 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 0x7f66f009de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.495+0000 7f66f733d700 1 -- 172.21.15.89:0/3514386470 >> 172.21.15.89:0/3514386470 conn(0x7f66f00a2aa0 msgr2=0x7f66f00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:11.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.499+0000 7f66f733d700 1 -- 172.21.15.89:0/3514386470 shutdown_connections 2023-09-30T22:23:11.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.499+0000 7f66f733d700 1 -- 172.21.15.89:0/3514386470 wait complete. 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f733d700 1 Processor -- start 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f733d700 1 -- start start 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f733d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f009e390 0x7f66f01214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f733d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f01338c0 0x7f66f0121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f733d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 0x7f66f0121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f733d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01229b0 con 0x7f66f009e390 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f733d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0130b50 con 0x7f66f01341d0 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f733d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66f006f960 con 0x7f66f01338c0 2023-09-30T22:23:11.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f50d9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f009e390 0x7f66f01214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f50d9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f009e390 0x7f66f01214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45790/0 (socket says 172.21.15.89:45790) 2023-09-30T22:23:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f50d9700 1 -- 172.21.15.89:0/3587493704 learned_addr learned my addr 172.21.15.89:0/3587493704 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f48d8700 1 --2- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f01338c0 0x7f66f0121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f50d9700 1 -- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f01338c0 msgr2=0x7f66f0121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f50d9700 1 --2- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f01338c0 0x7f66f0121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f50d9700 1 -- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 msgr2=0x7f66f0121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.500+0000 7f66f50d9700 1 --2- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 0x7f66f0121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.501+0000 7f66f50d9700 1 -- 172.21.15.89:0/3587493704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e00094a0 con 0x7f66f009e390 2023-09-30T22:23:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.501+0000 7f66f50d9700 1 --2- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f009e390 0x7f66f01214e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009f50 tx=0x7f66ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.501+0000 7f66e5ffb700 1 -- 172.21.15.89:0/3587493704 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec004240 con 0x7f66f009e390 2023-09-30T22:23:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.501+0000 7f66e5ffb700 1 -- 172.21.15.89:0/3587493704 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec041070 con 0x7f66f009e390 2023-09-30T22:23:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.501+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ec009550 con 0x7f66f009e390 2023-09-30T22:23:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.501+0000 7f66e5ffb700 1 -- 172.21.15.89:0/3587493704 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0045e0 con 0x7f66f009e390 2023-09-30T22:23:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.501+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f0124460 con 0x7f66f009e390 2023-09-30T22:23:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.502+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f0074ea0 con 0x7f66f009e390 2023-09-30T22:23:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.503+0000 7f66e5ffb700 1 -- 172.21.15.89:0/3587493704 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66ec043030 con 0x7f66f009e390 2023-09-30T22:23:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.503+0000 7f66e5ffb700 1 -- 172.21.15.89:0/3587493704 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f66ec062760 con 0x7f66f009e390 2023-09-30T22:23:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.505+0000 7f56415bf700 1 Processor -- start 2023-09-30T22:23:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.505+0000 7f56415bf700 1 -- start start 2023-09-30T22:23:11.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.506+0000 7f56415bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.506+0000 7f56415bf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f563c09e000 0x7f563c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.506+0000 7f56415bf700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c09b120 con 0x7f563c09e410 2023-09-30T22:23:11.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.506+0000 7f56415bf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c06b460 con 0x7f563c09e000 2023-09-30T22:23:11.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.506+0000 7f56415bf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c09a7e0 con 0x7f563c09d6f0 2023-09-30T22:23:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.506+0000 7f563affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.506+0000 7f563affd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37218/0 (socket says 172.21.15.89:37218) 2023-09-30T22:23:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.506+0000 7f563affd700 1 -- 172.21.15.89:0/564107570 learned_addr learned my addr 172.21.15.89:0/564107570 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f563a7fc700 1 --2- 172.21.15.89:0/564107570 >> v2:172.21.15.89:3300/0 conn(0x7f563c09e000 0x7f563c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f563affd700 1 -- 172.21.15.89:0/564107570 >> v2:172.21.15.89:3300/0 conn(0x7f563c09e000 msgr2=0x7f563c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f563affd700 1 --2- 172.21.15.89:0/564107570 >> v2:172.21.15.89:3300/0 conn(0x7f563c09e000 0x7f563c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f563affd700 1 -- 172.21.15.89:0/564107570 >> v1:172.21.15.89:6789/0 conn(0x7f563c09e410 legacy=0x7f563c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f563affd700 1 -- 172.21.15.89:0/564107570 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c09a500 con 0x7f563c09d6f0 2023-09-30T22:23:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f563affd700 1 --2- 172.21.15.89:0/564107570 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c09dac0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f562c009920 tx=0x7f562c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dec4f67baf1428d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f5639ffb700 1 -- 172.21.15.89:0/564107570 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c011830 con 0x7f563c09d6f0 2023-09-30T22:23:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f5639ffb700 1 -- 172.21.15.89:0/564107570 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c011990 con 0x7f563c09d6f0 2023-09-30T22:23:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.507+0000 7f5639ffb700 1 -- 172.21.15.89:0/564107570 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c011d20 con 0x7f563c09d6f0 2023-09-30T22:23:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f56415bf700 1 -- 172.21.15.89:0/564107570 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 msgr2=0x7f563c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f56415bf700 1 --2- 172.21.15.89:0/564107570 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c09dac0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f562c009920 tx=0x7f562c009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f56415bf700 1 -- 172.21.15.89:0/564107570 shutdown_connections 2023-09-30T22:23:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f56415bf700 1 --2- 172.21.15.89:0/564107570 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c09dac0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f56415bf700 1 --2- 172.21.15.89:0/564107570 >> v2:172.21.15.89:3300/0 conn(0x7f563c09e000 0x7f563c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f56415bf700 1 -- 172.21.15.89:0/564107570 >> 172.21.15.89:0/564107570 conn(0x7f563c066f50 msgr2=0x7f563c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f66e5ffb700 1 -- 172.21.15.89:0/3587493704 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f66ec003e00 con 0x7f66f009e390 2023-09-30T22:23:11.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f56415bf700 1 -- 172.21.15.89:0/564107570 shutdown_connections 2023-09-30T22:23:11.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.508+0000 7f56415bf700 1 -- 172.21.15.89:0/564107570 wait complete. 2023-09-30T22:23:11.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.510+0000 7f56415bf700 1 Processor -- start 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.510+0000 7f56415bf700 1 -- start start 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56415bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c1a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56415bf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563c09e000 0x7f563c1a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56415bf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563c09e410 0x7f563c1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56415bf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1a9a60 con 0x7f563c09e000 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56415bf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1a7bf0 con 0x7f563c09d6f0 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56415bf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0a33d0 con 0x7f563c09e410 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563c09e410 0x7f563c1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563c09e410 0x7f563c1a6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47820/0 (socket says 172.21.15.89:47820) 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 -- 172.21.15.89:0/2967949877 learned_addr learned my addr 172.21.15.89:0/2967949877 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 -- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 msgr2=0x7f563c1a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 --2- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c1a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 -- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563c09e000 msgr2=0x7f563c1a6510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 --2- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563c09e000 0x7f563c1a6510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 -- 172.21.15.89:0/2967949877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c009550 con 0x7f563c09e410 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f563b7fe700 1 --2- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563c09e410 0x7f563c1a6a50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f563400cc50 tx=0x7f563400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56237fe700 1 -- 172.21.15.89:0/2967949877 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563400ad80 con 0x7f563c09e410 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56237fe700 1 -- 172.21.15.89:0/2967949877 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563400ce50 con 0x7f563c09e410 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.511+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c09c710 con 0x7f563c09e410 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.512+0000 7f56237fe700 1 -- 172.21.15.89:0/2967949877 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634003c40 con 0x7f563c09e410 2023-09-30T22:23:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.512+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c1a8950 con 0x7f563c09e410 2023-09-30T22:23:11.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.512+0000 7f56237fe700 1 -- 172.21.15.89:0/2967949877 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56340050d0 con 0x7f563c09e410 2023-09-30T22:23:11.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.513+0000 7f56237fe700 1 -- 172.21.15.89:0/2967949877 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f563403f7a0 con 0x7f563c09e410 2023-09-30T22:23:11.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.513+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c074ea0 con 0x7f563c09e410 2023-09-30T22:23:11.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.522+0000 7f56237fe700 1 -- 172.21.15.89:0/2967949877 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5634005380 con 0x7f563c09e410 2023-09-30T22:23:11.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.591+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f79cc06c360 con 0x7f79cc135e50 2023-09-30T22:23:11.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.591+0000 7f79baffd700 1 -- 172.21.15.89:0/3432743644 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79c4037020 con 0x7f79cc135e50 2023-09-30T22:23:11.591 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.593+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79cc135e50 msgr2=0x7f79cc11b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.593+0000 7f79d0f34700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79cc135e50 0x7f79cc11b460 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009ea0 tx=0x7f79c400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.593+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 shutdown_connections 2023-09-30T22:23:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.593+0000 7f79d0f34700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79cc135e50 0x7f79cc11b460 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.593+0000 7f79d0f34700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79cc09d6f0 0x7f79cc1c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.593+0000 7f79d0f34700 1 --2- 172.21.15.89:0/3432743644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79cc09e1d0 0x7f79cc1c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.593+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 >> 172.21.15.89:0/3432743644 conn(0x7f79cc066f50 msgr2=0x7f79cc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.594+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 shutdown_connections 2023-09-30T22:23:11.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.594+0000 7f79d0f34700 1 -- 172.21.15.89:0/3432743644 wait complete. 2023-09-30T22:23:11.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:11.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.651+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8950074ea0 con 0x7f8950135e60 2023-09-30T22:23:11.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.651+0000 7f894d7fa700 1 -- 172.21.15.89:0/1067635718 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f894802c020 con 0x7f8950135e60 2023-09-30T22:23:11.651 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:11.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.653+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 msgr2=0x7f8950138630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.653+0000 7f8957150700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138630 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8948014040 tx=0x7f8948004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.654+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 shutdown_connections 2023-09-30T22:23:11.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.654+0000 7f8957150700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8950135e60 0x7f8950138630 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.654+0000 7f8957150700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8950130540 0x7f8950137bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.654+0000 7f8957150700 1 --2- 172.21.15.89:0/1067635718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8950133540 0x7f89501380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.654+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 >> 172.21.15.89:0/1067635718 conn(0x7f89500a2aa0 msgr2=0x7f8950134f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:11.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.654+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 shutdown_connections 2023-09-30T22:23:11.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.654+0000 7f8957150700 1 -- 172.21.15.89:0/1067635718 wait complete. 2023-09-30T22:23:11.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:11.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.710+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f66f006b9e0 con 0x7f66f009e390 2023-09-30T22:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.710+0000 7f66e5ffb700 1 -- 172.21.15.89:0/3587493704 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66ec015070 con 0x7f66f009e390 2023-09-30T22:23:11.710 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f009e390 msgr2=0x7f66f01214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 --2- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f009e390 0x7f66f01214e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009f50 tx=0x7f66ec004750 comp rx=0 tx=0).stop 2023-09-30T22:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 shutdown_connections 2023-09-30T22:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 --2- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66f01341d0 0x7f66f0121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 --2- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66f009e390 0x7f66f01214e0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 --2- 172.21.15.89:0/3587493704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66f01338c0 0x7f66f0121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 >> 172.21.15.89:0/3587493704 conn(0x7f66f00a2aa0 msgr2=0x7f66f0132240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:11.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 shutdown_connections 2023-09-30T22:23:11.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.712+0000 7f66f733d700 1 -- 172.21.15.89:0/3587493704 wait complete. 2023-09-30T22:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.718+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f563c06c360 con 0x7f563c09e410 2023-09-30T22:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.718+0000 7f56237fe700 1 -- 172.21.15.89:0/2967949877 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5634019300 con 0x7f563c09e410 2023-09-30T22:23:11.718 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:11.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.720+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563c09e410 msgr2=0x7f563c1a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:11.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.720+0000 7f56415bf700 1 --2- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563c09e410 0x7f563c1a6a50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f563400cc50 tx=0x7f563400a340 comp rx=0 tx=0).stop 2023-09-30T22:23:11.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.721+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 shutdown_connections 2023-09-30T22:23:11.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.721+0000 7f56415bf700 1 --2- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f563c09e410 0x7f563c1a6a50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.721+0000 7f56415bf700 1 --2- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f563c09d6f0 0x7f563c1a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.721+0000 7f56415bf700 1 --2- 172.21.15.89:0/2967949877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f563c09e000 0x7f563c1a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:11.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.721+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 >> 172.21.15.89:0/2967949877 conn(0x7f563c066f50 msgr2=0x7f563c1bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:11.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.721+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 shutdown_connections 2023-09-30T22:23:11.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:11.721+0000 7f56415bf700 1 -- 172.21.15.89:0/2967949877 wait complete. 2023-09-30T22:23:11.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:11.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:12.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:12.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:12.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:12.733 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:12.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.746+0000 7fd0d2362700 1 Processor -- start 2023-09-30T22:23:12.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.746+0000 7fd0d2362700 1 -- start start 2023-09-30T22:23:12.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d2362700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd0cc09d6f0 0x7fd0cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d2362700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e440 0x7fd0cc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d2362700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc09b120 con 0x7fd0cc09e000 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d2362700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0cc06b460 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d2362700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc09a7e0 con 0x7fd0cc09e440 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0cbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd0cc09d6f0 0x7fd0cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0cbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd0cc09d6f0 0x7fd0cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45808/0 (socket says 172.21.15.89:45808) 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d08ff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e440 0x7fd0cc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0cbfff700 1 -- 172.21.15.89:0/747015739 learned_addr learned my addr 172.21.15.89:0/747015739 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0cb7fe700 1 --1- 172.21.15.89:0/747015739 >> v1:172.21.15.89:6789/0 conn(0x7fd0cc09e000 0x7fd0cc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47900/0 (socket says 172.21.15.89:47900) 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d08ff700 1 -- 172.21.15.89:0/747015739 >> v2:172.21.15.89:3300/0 conn(0x7fd0cc09d6f0 msgr2=0x7fd0cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d08ff700 1 --2- 172.21.15.89:0/747015739 >> v2:172.21.15.89:3300/0 conn(0x7fd0cc09d6f0 0x7fd0cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d08ff700 1 -- 172.21.15.89:0/747015739 >> v1:172.21.15.89:6789/0 conn(0x7fd0cc09e000 legacy=0x7fd0cc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d08ff700 1 -- 172.21.15.89:0/747015739 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc09a500 con 0x7fd0cc09e440 2023-09-30T22:23:12.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.747+0000 7fd0d08ff700 1 --2- 172.21.15.89:0/747015739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e440 0x7fd0cc134100 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009dc0 tx=0x7fd0c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a8d1dcae31d164 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0caffd700 1 -- 172.21.15.89:0/747015739 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400e070 con 0x7fd0cc09e440 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0caffd700 1 -- 172.21.15.89:0/747015739 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c400bae0 con 0x7fd0cc09e440 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0caffd700 1 -- 172.21.15.89:0/747015739 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400be90 con 0x7fd0cc09e440 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0d2362700 1 -- 172.21.15.89:0/747015739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e440 msgr2=0x7fd0cc134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0d2362700 1 --2- 172.21.15.89:0/747015739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e440 0x7fd0cc134100 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009dc0 tx=0x7fd0c400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0d2362700 1 -- 172.21.15.89:0/747015739 shutdown_connections 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0d2362700 1 --2- 172.21.15.89:0/747015739 >> v2:172.21.15.89:3300/0 conn(0x7fd0cc09d6f0 0x7fd0cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0d2362700 1 --2- 172.21.15.89:0/747015739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e440 0x7fd0cc134100 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0d2362700 1 -- 172.21.15.89:0/747015739 >> 172.21.15.89:0/747015739 conn(0x7fd0cc066f50 msgr2=0x7fd0cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.748+0000 7fd0d2362700 1 -- 172.21.15.89:0/747015739 shutdown_connections 2023-09-30T22:23:12.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.749+0000 7fd0d2362700 1 -- 172.21.15.89:0/747015739 wait complete. 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.750+0000 7fd0d2362700 1 Processor -- start 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.750+0000 7fd0d2362700 1 -- start start 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0d2362700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc09d6f0 0x7fd0cc1a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0d2362700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e000 0x7fd0cc1a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0d2362700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0cc09e440 0x7fd0cc1a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0d2362700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc1aa360 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0d2362700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc1a99a0 con 0x7fd0cc09e440 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0d2362700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc1a8ce0 con 0x7fd0cc09e000 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc09d6f0 0x7fd0cc1a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc09d6f0 0x7fd0cc1a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45818/0 (socket says 172.21.15.89:45818) 2023-09-30T22:23:12.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cbfff700 1 -- 172.21.15.89:0/2902126481 learned_addr learned my addr 172.21.15.89:0/2902126481 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cb7fe700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e000 0x7fd0cc1a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0d08ff700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0cc09e440 0x7fd0cc1a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cbfff700 1 -- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e000 msgr2=0x7fd0cc1a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cbfff700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e000 0x7fd0cc1a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cbfff700 1 -- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0cc09e440 msgr2=0x7fd0cc1a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cbfff700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0cc09e440 0x7fd0cc1a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cbfff700 1 -- 172.21.15.89:0/2902126481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c4009a70 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.751+0000 7fd0cb7fe700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e000 0x7fd0cc1a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.752+0000 7fd0cbfff700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc09d6f0 0x7fd0cc1a5ed0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00e910 tx=0x7fd0bc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:12.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.752+0000 7fd0c8ff9700 1 -- 172.21.15.89:0/2902126481 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0096a0 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.752+0000 7fd0c8ff9700 1 -- 172.21.15.89:0/2902126481 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc00ee60 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.752+0000 7fd0c8ff9700 1 -- 172.21.15.89:0/2902126481 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00ca90 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.753+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc0a2000 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.753+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc1a7a90 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.754+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc074ea0 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.754+0000 7fd0c8ff9700 1 -- 172.21.15.89:0/2902126481 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd0bc00cbf0 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.754+0000 7fd0c8ff9700 1 -- 172.21.15.89:0/2902126481 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd0bc03fe60 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.760+0000 7fd0c8ff9700 1 -- 172.21.15.89:0/2902126481 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd0bc013470 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.804+0000 7f038544d700 1 Processor -- start 2023-09-30T22:23:12.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.804+0000 7f038544d700 1 -- start start 2023-09-30T22:23:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f038544d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f038012cf60 0x7f038006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f038544d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012f8b0 0x7f038012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f038544d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038012cd90 con 0x7f0380130260 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f038544d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038012d330 con 0x7f038012cf60 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f038544d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03801306d0 con 0x7f038012f8b0 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f038012cf60 0x7f038006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037e7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012f8b0 0x7f038012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037effd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f038012cf60 0x7f038006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45822/0 (socket says 172.21.15.89:45822) 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037effd700 1 -- 172.21.15.89:0/2893106212 learned_addr learned my addr 172.21.15.89:0/2893106212 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037effd700 1 -- 172.21.15.89:0/2893106212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012f8b0 msgr2=0x7f038012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037effd700 1 --2- 172.21.15.89:0/2893106212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012f8b0 0x7f038012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037effd700 1 -- 172.21.15.89:0/2893106212 >> v1:172.21.15.89:6789/0 conn(0x7f0380130260 legacy=0x7f0380136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037effd700 1 -- 172.21.15.89:0/2893106212 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0380071780 con 0x7f038012cf60 2023-09-30T22:23:12.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.805+0000 7f037e7fc700 1 --2- 172.21.15.89:0/2893106212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012f8b0 0x7f038012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.806+0000 7f037effd700 1 --2- 172.21.15.89:0/2893106212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012cf60 0x7f038006f5d0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f03700098f0 tx=0x7f0370009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c9c02075c72ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.806+0000 7f037dffb700 1 -- 172.21.15.89:0/2893106212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370011830 con 0x7f038012cf60 2023-09-30T22:23:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.806+0000 7f037dffb700 1 -- 172.21.15.89:0/2893106212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0370011990 con 0x7f038012cf60 2023-09-30T22:23:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.806+0000 7f037dffb700 1 -- 172.21.15.89:0/2893106212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370011d20 con 0x7f038012cf60 2023-09-30T22:23:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.806+0000 7f038544d700 1 -- 172.21.15.89:0/2893106212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012cf60 msgr2=0x7f038006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.806+0000 7f038544d700 1 --2- 172.21.15.89:0/2893106212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012cf60 0x7f038006f5d0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f03700098f0 tx=0x7f0370009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.806+0000 7f038544d700 1 -- 172.21.15.89:0/2893106212 shutdown_connections 2023-09-30T22:23:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.807+0000 7f038544d700 1 --2- 172.21.15.89:0/2893106212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012cf60 0x7f038006f5d0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.807+0000 7f038544d700 1 --2- 172.21.15.89:0/2893106212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012f8b0 0x7f038012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.807+0000 7f038544d700 1 -- 172.21.15.89:0/2893106212 >> 172.21.15.89:0/2893106212 conn(0x7f03800a2aa0 msgr2=0x7f03800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.807+0000 7f038544d700 1 -- 172.21.15.89:0/2893106212 shutdown_connections 2023-09-30T22:23:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.807+0000 7f038544d700 1 -- 172.21.15.89:0/2893106212 wait complete. 2023-09-30T22:23:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.808+0000 7f038544d700 1 Processor -- start 2023-09-30T22:23:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.808+0000 7f038544d700 1 -- start start 2023-09-30T22:23:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f038544d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012cf60 0x7f03801c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f038544d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012f8b0 0x7f03801c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f038544d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0380130260 0x7f038013a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f038544d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038013b640 con 0x7f038012f8b0 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f038544d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0380135070 con 0x7f0380130260 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f038544d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03801c8460 con 0x7f038012cf60 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012f8b0 0x7f03801c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0380130260 0x7f038013a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012f8b0 0x7f03801c8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45832/0 (socket says 172.21.15.89:45832) 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037e7fc700 1 -- 172.21.15.89:0/4094518783 learned_addr learned my addr 172.21.15.89:0/4094518783 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037effd700 1 --2- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012cf60 0x7f03801c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037e7fc700 1 -- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012cf60 msgr2=0x7f03801c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037e7fc700 1 --2- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012cf60 0x7f03801c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037e7fc700 1 -- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0380130260 msgr2=0x7f038013a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037e7fc700 1 --2- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0380130260 0x7f038013a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.809+0000 7f037e7fc700 1 -- 172.21.15.89:0/4094518783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03740094a0 con 0x7f038012f8b0 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.810+0000 7f037e7fc700 1 --2- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012f8b0 0x7f03801c8f10 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f037400ec10 tx=0x7f037400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.810+0000 7f03677fe700 1 -- 172.21.15.89:0/4094518783 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037400c850 con 0x7f038012f8b0 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.810+0000 7f03677fe700 1 -- 172.21.15.89:0/4094518783 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037400c9b0 con 0x7f038012f8b0 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.810+0000 7f03677fe700 1 -- 172.21.15.89:0/4094518783 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037400cd20 con 0x7f038012f8b0 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.810+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0370009550 con 0x7f038012f8b0 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.810+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0380135430 con 0x7f038012f8b0 2023-09-30T22:23:12.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.811+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038012da60 con 0x7f038012f8b0 2023-09-30T22:23:12.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.811+0000 7f03677fe700 1 -- 172.21.15.89:0/4094518783 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0374013470 con 0x7f038012f8b0 2023-09-30T22:23:12.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.812+0000 7f03677fe700 1 -- 172.21.15.89:0/4094518783 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f037403fc20 con 0x7f038012f8b0 2023-09-30T22:23:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.816+0000 7f03677fe700 1 -- 172.21.15.89:0/4094518783 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f038012da60 con 0x7f038012f8b0 2023-09-30T22:23:12.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.860+0000 7f7facb7f700 1 Processor -- start 2023-09-30T22:23:12.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.860+0000 7f7facb7f700 1 -- start start 2023-09-30T22:23:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.860+0000 7f7facb7f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7fa81338a0 0x7f7fa8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.860+0000 7f7facb7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809e930 0x7f7fa813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.860+0000 7f7facb7f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa806ef30 con 0x7f7fa809de50 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.860+0000 7f7facb7f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa806fef0 con 0x7f7fa81338a0 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.860+0000 7f7facb7f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81334b0 con 0x7f7fa809e930 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa5d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7fa809de50 0x7f7fa809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47918/0 (socket says 172.21.15.89:47918) 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa5d9b700 1 -- 172.21.15.89:0/1226120541 learned_addr learned my addr 172.21.15.89:0/1226120541 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa6d9d700 1 --2- 172.21.15.89:0/1226120541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809e930 0x7f7fa813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa659c700 1 --2- 172.21.15.89:0/1226120541 >> v2:172.21.15.89:3300/0 conn(0x7f7fa81338a0 0x7f7fa8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa559a700 1 -- 172.21.15.89:0/1226120541 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476419440 0 0) 0x7f7fa806ef30 con 0x7f7fa809de50 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa559a700 1 -- 172.21.15.89:0/1226120541 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f900035b0 con 0x7f7fa809de50 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa6d9d700 1 -- 172.21.15.89:0/1226120541 >> v2:172.21.15.89:3300/0 conn(0x7f7fa81338a0 msgr2=0x7f7fa8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa6d9d700 1 --2- 172.21.15.89:0/1226120541 >> v2:172.21.15.89:3300/0 conn(0x7f7fa81338a0 0x7f7fa8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa6d9d700 1 -- 172.21.15.89:0/1226120541 >> v1:172.21.15.89:6789/0 conn(0x7f7fa809de50 legacy=0x7f7fa809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa6d9d700 1 -- 172.21.15.89:0/1226120541 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa806fc30 con 0x7f7fa809e930 2023-09-30T22:23:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.861+0000 7f7fa6d9d700 1 --2- 172.21.15.89:0/1226120541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809e930 0x7f7fa813b740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009d90 tx=0x7f7fa000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ffbbac1af5fb54 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7fa559a700 1 -- 172.21.15.89:0/1226120541 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000e070 con 0x7f7fa809e930 2023-09-30T22:23:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7fa559a700 1 -- 172.21.15.89:0/1226120541 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa000bb20 con 0x7f7fa809e930 2023-09-30T22:23:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7fa559a700 1 -- 172.21.15.89:0/1226120541 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0012430 con 0x7f7fa809e930 2023-09-30T22:23:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7facb7f700 1 -- 172.21.15.89:0/1226120541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809e930 msgr2=0x7f7fa813b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7facb7f700 1 --2- 172.21.15.89:0/1226120541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809e930 0x7f7fa813b740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009d90 tx=0x7f7fa000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7facb7f700 1 -- 172.21.15.89:0/1226120541 shutdown_connections 2023-09-30T22:23:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7facb7f700 1 --2- 172.21.15.89:0/1226120541 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809e930 0x7f7fa813b740 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7facb7f700 1 --2- 172.21.15.89:0/1226120541 >> v2:172.21.15.89:3300/0 conn(0x7f7fa81338a0 0x7f7fa8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.862+0000 7f7facb7f700 1 -- 172.21.15.89:0/1226120541 >> 172.21.15.89:0/1226120541 conn(0x7f7fa80a2aa0 msgr2=0x7f7fa80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.863+0000 7f7facb7f700 1 -- 172.21.15.89:0/1226120541 shutdown_connections 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.863+0000 7f7facb7f700 1 -- 172.21.15.89:0/1226120541 wait complete. 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7facb7f700 1 Processor -- start 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7facb7f700 1 -- start start 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7facb7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809de50 0x7f7fa811d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7facb7f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fa809e930 0x7f7fa811db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7facb7f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa81338a0 0x7f7fa811e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7facb7f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa8120df0 con 0x7f7fa81338a0 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7facb7f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa811f2d0 con 0x7f7fa809de50 2023-09-30T22:23:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7facb7f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80a0d20 con 0x7f7fa809e930 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7fa5d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fa809e930 0x7f7fa811db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7fa6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa81338a0 0x7f7fa811e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7fa5d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fa809e930 0x7f7fa811db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47854/0 (socket says 172.21.15.89:47854) 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7fa6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa81338a0 0x7f7fa811e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45836/0 (socket says 172.21.15.89:45836) 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7fa5d9b700 1 -- 172.21.15.89:0/2454608589 learned_addr learned my addr 172.21.15.89:0/2454608589 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.864+0000 7f7fa659c700 1 --2- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809de50 0x7f7fa811d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7fa5d9b700 1 -- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809de50 msgr2=0x7f7fa811d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7fa5d9b700 1 --2- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809de50 0x7f7fa811d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7fa5d9b700 1 -- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa81338a0 msgr2=0x7f7fa811e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7fa5d9b700 1 --2- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa81338a0 0x7f7fa811e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7fa5d9b700 1 -- 172.21.15.89:0/2454608589 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0009a70 con 0x7f7fa809e930 2023-09-30T22:23:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7fa5d9b700 1 --2- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fa809e930 0x7f7fa811db80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c001eb0 tx=0x7f7f9c00c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7f96ffd700 1 -- 172.21.15.89:0/2454608589 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c005d70 con 0x7f7fa809e930 2023-09-30T22:23:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7f96ffd700 1 -- 172.21.15.89:0/2454608589 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c0070d0 con 0x7f7fa809e930 2023-09-30T22:23:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa806f640 con 0x7f7fa809e930 2023-09-30T22:23:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7f96ffd700 1 -- 172.21.15.89:0/2454608589 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c007440 con 0x7f7fa809e930 2023-09-30T22:23:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.865+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa81210d0 con 0x7f7fa809e930 2023-09-30T22:23:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.866+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa811a470 con 0x7f7fa809e930 2023-09-30T22:23:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.866+0000 7f7f96ffd700 1 -- 172.21.15.89:0/2454608589 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f9c0075a0 con 0x7f7fa809e930 2023-09-30T22:23:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.867+0000 7f7f96ffd700 1 -- 172.21.15.89:0/2454608589 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7f9c03fa20 con 0x7f7fa809e930 2023-09-30T22:23:12.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.871+0000 7f7f96ffd700 1 -- 172.21.15.89:0/2454608589 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f9c019b40 con 0x7f7fa809e930 2023-09-30T22:23:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.877+0000 7f858c67a700 1 Processor -- start 2023-09-30T22:23:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.877+0000 7f858c67a700 1 -- start start 2023-09-30T22:23:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f858c67a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f85841338a0 0x7f8584133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f858c67a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f858409de50 0x7f858409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f858c67a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858406ef30 con 0x7f858409e760 2023-09-30T22:23:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f858c67a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858406fef0 con 0x7f85841338a0 2023-09-30T22:23:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f858c67a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85841334b0 con 0x7f858409de50 2023-09-30T22:23:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f8589c15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f858409de50 0x7f858409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f8589c15700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f858409de50 0x7f858409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47856/0 (socket says 172.21.15.89:47856) 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f8589c15700 1 -- 172.21.15.89:0/2781271955 learned_addr learned my addr 172.21.15.89:0/2781271955 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f858a416700 1 --2- 172.21.15.89:0/2781271955 >> v2:172.21.15.89:3300/0 conn(0x7f85841338a0 0x7f8584133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f8589c15700 1 -- 172.21.15.89:0/2781271955 >> v2:172.21.15.89:3300/0 conn(0x7f85841338a0 msgr2=0x7f8584133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f8589c15700 1 --2- 172.21.15.89:0/2781271955 >> v2:172.21.15.89:3300/0 conn(0x7f85841338a0 0x7f8584133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f8589c15700 1 -- 172.21.15.89:0/2781271955 >> v1:172.21.15.89:6789/0 conn(0x7f858409e760 legacy=0x7f858413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f8589c15700 1 -- 172.21.15.89:0/2781271955 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858406fc30 con 0x7f858409de50 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.878+0000 7f8589c15700 1 --2- 172.21.15.89:0/2781271955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f858409de50 0x7f858409e220 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f85740098f0 tx=0x7f8574009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c54a6d701d9a474 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f8589414700 1 -- 172.21.15.89:0/2781271955 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857400b8a0 con 0x7f858409de50 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f8589414700 1 -- 172.21.15.89:0/2781271955 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857400ba00 con 0x7f858409de50 2023-09-30T22:23:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f8589414700 1 -- 172.21.15.89:0/2781271955 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857400bd90 con 0x7f858409de50 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f858c67a700 1 -- 172.21.15.89:0/2781271955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f858409de50 msgr2=0x7f858409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f858c67a700 1 --2- 172.21.15.89:0/2781271955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f858409de50 0x7f858409e220 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f85740098f0 tx=0x7f8574009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f858c67a700 1 -- 172.21.15.89:0/2781271955 shutdown_connections 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f858c67a700 1 --2- 172.21.15.89:0/2781271955 >> v2:172.21.15.89:3300/0 conn(0x7f85841338a0 0x7f8584133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f858c67a700 1 --2- 172.21.15.89:0/2781271955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f858409de50 0x7f858409e220 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f858c67a700 1 -- 172.21.15.89:0/2781271955 >> 172.21.15.89:0/2781271955 conn(0x7f85840a2aa0 msgr2=0x7f85840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f858c67a700 1 -- 172.21.15.89:0/2781271955 shutdown_connections 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.879+0000 7f858c67a700 1 -- 172.21.15.89:0/2781271955 wait complete. 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.880+0000 7f858c67a700 1 Processor -- start 2023-09-30T22:23:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858c67a700 1 -- start start 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858c67a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f858409de50 0x7f858411ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858c67a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f858409e760 0x7f858411ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858c67a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85841338a0 0x7f858411f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858c67a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8584120ea0 con 0x7f858409de50 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858c67a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8584120720 con 0x7f858409e760 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858c67a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8584122240 con 0x7f85841338a0 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858a416700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f858409de50 0x7f858411ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f8589c15700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f858409e760 0x7f858411ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858a416700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f858409de50 0x7f858411ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45848/0 (socket says 172.21.15.89:45848) 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858a416700 1 -- 172.21.15.89:0/244237039 learned_addr learned my addr 172.21.15.89:0/244237039 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f858ac17700 1 --2- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85841338a0 0x7f858411f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f8589c15700 1 -- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85841338a0 msgr2=0x7f858411f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f8589c15700 1 --2- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85841338a0 0x7f858411f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f8589c15700 1 -- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f858409de50 msgr2=0x7f858411ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f8589c15700 1 --2- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f858409de50 0x7f858411ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.881+0000 7f8589c15700 1 -- 172.21.15.89:0/244237039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574009550 con 0x7f858409e760 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.882+0000 7f8589c15700 1 --2- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f858409e760 0x7f858411ef40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8574016000 tx=0x7f8574004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.882+0000 7f857affd700 1 -- 172.21.15.89:0/244237039 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574004590 con 0x7f858409e760 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.882+0000 7f857affd700 1 -- 172.21.15.89:0/244237039 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857401e070 con 0x7f858409e760 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.882+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85840a1020 con 0x7f858409e760 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.882+0000 7f857affd700 1 -- 172.21.15.89:0/244237039 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85740038a0 con 0x7f858409e760 2023-09-30T22:23:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.882+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8584122cb0 con 0x7f858409e760 2023-09-30T22:23:12.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.883+0000 7f857affd700 1 -- 172.21.15.89:0/244237039 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8574003ac0 con 0x7f858409e760 2023-09-30T22:23:12.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.883+0000 7f857affd700 1 -- 172.21.15.89:0/244237039 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8574020070 con 0x7f858409e760 2023-09-30T22:23:12.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.883+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8584074ea0 con 0x7f858409e760 2023-09-30T22:23:12.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.892+0000 7f857affd700 1 -- 172.21.15.89:0/244237039 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8574015020 con 0x7f858409e760 2023-09-30T22:23:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.965+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd0cc06c360 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.965+0000 7fd0c8ff9700 1 -- 172.21.15.89:0/2902126481 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd0bc021070 con 0x7fd0cc09d6f0 2023-09-30T22:23:12.965 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:12.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.967+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc09d6f0 msgr2=0x7fd0cc1a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:12.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.967+0000 7fd0d2362700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc09d6f0 0x7fd0cc1a5ed0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00e910 tx=0x7fd0bc00ec20 comp rx=0 tx=0).stop 2023-09-30T22:23:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.968+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 shutdown_connections 2023-09-30T22:23:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.968+0000 7fd0d2362700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0cc09e440 0x7fd0cc1a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.968+0000 7fd0d2362700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc09d6f0 0x7fd0cc1a5ed0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.968+0000 7fd0d2362700 1 --2- 172.21.15.89:0/2902126481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0cc09e000 0x7fd0cc1a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.968+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 >> 172.21.15.89:0/2902126481 conn(0x7fd0cc066f50 msgr2=0x7fd0cc1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.968+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 shutdown_connections 2023-09-30T22:23:12.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:12.968+0000 7fd0d2362700 1 -- 172.21.15.89:0/2902126481 wait complete. 2023-09-30T22:23:12.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:13.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.069+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7fa8120f30 con 0x7f7fa809e930 2023-09-30T22:23:13.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.069+0000 7f7f96ffd700 1 -- 172.21.15.89:0/2454608589 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7fa8120f30 con 0x7f7fa809e930 2023-09-30T22:23:13.069 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fa809e930 msgr2=0x7f7fa811db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 --2- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fa809e930 0x7f7fa811db80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c001eb0 tx=0x7f7f9c00c7b0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 shutdown_connections 2023-09-30T22:23:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 --2- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa81338a0 0x7f7fa811e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 --2- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7fa809de50 0x7f7fa811d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 --2- 172.21.15.89:0/2454608589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7fa809e930 0x7f7fa811db80 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 >> 172.21.15.89:0/2454608589 conn(0x7f7fa80a2aa0 msgr2=0x7f7fa8132af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:13.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 shutdown_connections 2023-09-30T22:23:13.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.071+0000 7f7facb7f700 1 -- 172.21.15.89:0/2454608589 wait complete. 2023-09-30T22:23:13.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.073+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f038012da60 con 0x7f038012f8b0 2023-09-30T22:23:13.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.073+0000 7f03677fe700 1 -- 172.21.15.89:0/4094518783 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f038012da60 con 0x7f038012f8b0 2023-09-30T22:23:13.073 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:13.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012f8b0 msgr2=0x7f03801c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:13.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 --2- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012f8b0 0x7f03801c8f10 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f037400ec10 tx=0x7f037400ef20 comp rx=0 tx=0).stop 2023-09-30T22:23:13.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 shutdown_connections 2023-09-30T22:23:13.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 --2- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0380130260 0x7f038013a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 --2- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f038012cf60 0x7f03801c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 --2- 172.21.15.89:0/4094518783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f038012f8b0 0x7f03801c8f10 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 >> 172.21.15.89:0/4094518783 conn(0x7f03800a2aa0 msgr2=0x7f03800a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:13.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 shutdown_connections 2023-09-30T22:23:13.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.075+0000 7f038544d700 1 -- 172.21.15.89:0/4094518783 wait complete. 2023-09-30T22:23:13.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:13.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.143+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f858406b9e0 con 0x7f858409e760 2023-09-30T22:23:13.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.143+0000 7f857affd700 1 -- 172.21.15.89:0/244237039 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8574033070 con 0x7f858409e760 2023-09-30T22:23:13.143 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:13.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f858409e760 msgr2=0x7f858411ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:13.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 --2- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f858409e760 0x7f858411ef40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8574016000 tx=0x7f8574004750 comp rx=0 tx=0).stop 2023-09-30T22:23:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 shutdown_connections 2023-09-30T22:23:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 --2- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85841338a0 0x7f858411f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 --2- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f858409de50 0x7f858411ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 --2- 172.21.15.89:0/244237039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f858409e760 0x7f858411ef40 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 >> 172.21.15.89:0/244237039 conn(0x7f85840a2aa0 msgr2=0x7f858409ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 shutdown_connections 2023-09-30T22:23:13.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:13.145+0000 7f858c67a700 1 -- 172.21.15.89:0/244237039 wait complete. 2023-09-30T22:23:13.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:13.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:14.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:14.088 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.119+0000 7f2dc6480700 1 Processor -- start 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.119+0000 7f2dc6480700 1 -- start start 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc6480700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012c9a0 0x7f2dc012cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc6480700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2dc0130380 0x7f2dc0137c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc6480700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc009a420 con 0x7f2dc012f880 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc6480700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc009b890 con 0x7f2dc0130380 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc6480700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc009b2f0 con 0x7f2dc012c9a0 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc4a1d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2dc0130380 0x7f2dc0137c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc4a1d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2dc0130380 0x7f2dc0137c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45858/0 (socket says 172.21.15.89:45858) 2023-09-30T22:23:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dbffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012c9a0 0x7f2dc012cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc4a1d700 1 -- 172.21.15.89:0/2762827074 learned_addr learned my addr 172.21.15.89:0/2762827074 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc4a1d700 1 -- 172.21.15.89:0/2762827074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012c9a0 msgr2=0x7f2dc012cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc4a1d700 1 --2- 172.21.15.89:0/2762827074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012c9a0 0x7f2dc012cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc4a1d700 1 -- 172.21.15.89:0/2762827074 >> v1:172.21.15.89:6789/0 conn(0x7f2dc012f880 legacy=0x7f2dc012fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dc4a1d700 1 -- 172.21.15.89:0/2762827074 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc009a870 con 0x7f2dc0130380 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dbeffd700 1 -- 172.21.15.89:0/2762827074 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427516588 0 0) 0x7f2dc009a420 con 0x7f2dc012f880 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.120+0000 7f2dbffff700 1 --2- 172.21.15.89:0/2762827074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012c9a0 0x7f2dc012cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dc4a1d700 1 --2- 172.21.15.89:0/2762827074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 0x7f2dc0137c20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009e50 tx=0x7f2db800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4951f288a0cd46c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dbeffd700 1 -- 172.21.15.89:0/2762827074 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800e070 con 0x7f2dc0130380 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dbeffd700 1 -- 172.21.15.89:0/2762827074 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db800b910 con 0x7f2dc0130380 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dbeffd700 1 -- 172.21.15.89:0/2762827074 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800bcc0 con 0x7f2dc0130380 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dc6480700 1 -- 172.21.15.89:0/2762827074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 msgr2=0x7f2dc0137c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dc6480700 1 --2- 172.21.15.89:0/2762827074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 0x7f2dc0137c20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009e50 tx=0x7f2db800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dc6480700 1 -- 172.21.15.89:0/2762827074 shutdown_connections 2023-09-30T22:23:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dc6480700 1 --2- 172.21.15.89:0/2762827074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 0x7f2dc0137c20 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dc6480700 1 --2- 172.21.15.89:0/2762827074 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012c9a0 0x7f2dc012cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.121+0000 7f2dc6480700 1 -- 172.21.15.89:0/2762827074 >> 172.21.15.89:0/2762827074 conn(0x7f2dc00a2360 msgr2=0x7f2dc00a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.122+0000 7f2dc6480700 1 -- 172.21.15.89:0/2762827074 shutdown_connections 2023-09-30T22:23:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.122+0000 7f2dc6480700 1 -- 172.21.15.89:0/2762827074 wait complete. 2023-09-30T22:23:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.122+0000 7f2dc6480700 1 Processor -- start 2023-09-30T22:23:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dc6480700 1 -- start start 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dc6480700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2dc012c9a0 0x7f2dc0098b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dc6480700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012f880 0x7f2dc0070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dc6480700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 0x7f2dc0071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dc6480700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc0127e00 con 0x7f2dc0130380 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dc6480700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc0072340 con 0x7f2dc012f880 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dc6480700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc009ad80 con 0x7f2dc012c9a0 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dbf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012f880 0x7f2dc0070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dbf7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012f880 0x7f2dc0070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37296/0 (socket says 172.21.15.89:37296) 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dbf7fe700 1 -- 172.21.15.89:0/2425998657 learned_addr learned my addr 172.21.15.89:0/2425998657 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dc4a1d700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 0x7f2dc0071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.123+0000 7f2dbffff700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2dc012c9a0 0x7f2dc0098b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbf7fe700 1 -- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2dc012c9a0 msgr2=0x7f2dc0098b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbf7fe700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2dc012c9a0 0x7f2dc0098b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbf7fe700 1 -- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 msgr2=0x7f2dc0071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbf7fe700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 0x7f2dc0071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbf7fe700 1 -- 172.21.15.89:0/2425998657 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db000a5b0 con 0x7f2dc012f880 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbf7fe700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012f880 0x7f2dc0070ee0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2db400c550 tx=0x7f2db400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbcff9700 1 -- 172.21.15.89:0/2425998657 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40054e0 con 0x7f2dc012f880 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbcff9700 1 -- 172.21.15.89:0/2425998657 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4003680 con 0x7f2dc012f880 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db8009b30 con 0x7f2dc012f880 2023-09-30T22:23:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dbcff9700 1 -- 172.21.15.89:0/2425998657 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4003a30 con 0x7f2dc012f880 2023-09-30T22:23:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.124+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc009c5f0 con 0x7f2dc012f880 2023-09-30T22:23:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.125+0000 7f2dbcff9700 1 -- 172.21.15.89:0/2425998657 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2db4003b90 con 0x7f2dc012f880 2023-09-30T22:23:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.125+0000 7f2dbcff9700 1 -- 172.21.15.89:0/2425998657 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2db403eb10 con 0x7f2dc012f880 2023-09-30T22:23:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.125+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc0074ea0 con 0x7f2dc012f880 2023-09-30T22:23:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.130+0000 7f2dbcff9700 1 -- 172.21.15.89:0/2425998657 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2db4003e40 con 0x7f2dc012f880 2023-09-30T22:23:14.158 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.216+0000 7f89683a6700 1 Processor -- start 2023-09-30T22:23:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.216+0000 7f89683a6700 1 -- start start 2023-09-30T22:23:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.216+0000 7f9e56929700 1 Processor -- start 2023-09-30T22:23:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.216+0000 7f9e56929700 1 -- start start 2023-09-30T22:23:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e56929700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 0x7f9e50135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e56929700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e50135e50 0x7f9e50138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e56929700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5009f130 con 0x7f9e5009d6f0 2023-09-30T22:23:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e56929700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5009f520 con 0x7f9e5009e1d0 2023-09-30T22:23:14.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e56929700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5009e930 con 0x7f9e50135e50 2023-09-30T22:23:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f89683a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f89600a09c0 0x7f89600a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f89683a6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f896009d5c0 0x7f89600a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f89683a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89600a1810 con 0x7f8960133920 2023-09-30T22:23:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f89683a6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896009ea10 con 0x7f89600a09c0 2023-09-30T22:23:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f89683a6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f896009e210 con 0x7f896009d5c0 2023-09-30T22:23:14.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8965941700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f896009d5c0 0x7f89600a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8965941700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f896009d5c0 0x7f89600a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47888/0 (socket says 172.21.15.89:47888) 2023-09-30T22:23:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8965941700 1 -- 172.21.15.89:0/86278823 learned_addr learned my addr 172.21.15.89:0/86278823 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e54ec6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e50135e50 0x7f9e50138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e54ec6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e50135e50 0x7f9e50138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47896/0 (socket says 172.21.15.89:47896) 2023-09-30T22:23:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e54ec6700 1 -- 172.21.15.89:0/1463115776 learned_addr learned my addr 172.21.15.89:0/1463115776 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8966142700 1 --2- 172.21.15.89:0/86278823 >> v2:172.21.15.89:3300/0 conn(0x7f89600a09c0 0x7f89600a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8965941700 1 -- 172.21.15.89:0/86278823 >> v2:172.21.15.89:3300/0 conn(0x7f89600a09c0 msgr2=0x7f89600a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8965941700 1 --2- 172.21.15.89:0/86278823 >> v2:172.21.15.89:3300/0 conn(0x7f89600a09c0 0x7f89600a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8965941700 1 -- 172.21.15.89:0/86278823 >> v1:172.21.15.89:6789/0 conn(0x7f8960133920 legacy=0x7f8960133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8965941700 1 -- 172.21.15.89:0/86278823 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89600a26f0 con 0x7f896009d5c0 2023-09-30T22:23:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e4f7fe700 1 --2- 172.21.15.89:0/1463115776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 0x7f9e50135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e54ec6700 1 -- 172.21.15.89:0/1463115776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 msgr2=0x7f9e50135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e54ec6700 1 --2- 172.21.15.89:0/1463115776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 0x7f9e50135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f9e54ec6700 1 -- 172.21.15.89:0/1463115776 >> v1:172.21.15.89:6789/0 conn(0x7f9e5009d6f0 legacy=0x7f9e5009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f9e54ec6700 1 -- 172.21.15.89:0/1463115776 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5009b2d0 con 0x7f9e50135e50 2023-09-30T22:23:14.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.217+0000 7f8966142700 1 --2- 172.21.15.89:0/86278823 >> v2:172.21.15.89:3300/0 conn(0x7f89600a09c0 0x7f89600a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:14.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f8965941700 1 --2- 172.21.15.89:0/86278823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f896009d5c0 0x7f89600a12d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f89500098f0 tx=0x7f8950009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00adb90d1822137 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:14.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f8965140700 1 -- 172.21.15.89:0/86278823 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000b8a0 con 0x7f896009d5c0 2023-09-30T22:23:14.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f8965140700 1 -- 172.21.15.89:0/86278823 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895000ba00 con 0x7f896009d5c0 2023-09-30T22:23:14.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f8965140700 1 -- 172.21.15.89:0/86278823 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000bd90 con 0x7f896009d5c0 2023-09-30T22:23:14.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f9e54ec6700 1 --2- 172.21.15.89:0/1463115776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e50135e50 0x7f9e50138230 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009e80 tx=0x7f9e4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b28853895ee11e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:14.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f9e4effd700 1 -- 172.21.15.89:0/1463115776 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4800e070 con 0x7f9e50135e50 2023-09-30T22:23:14.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f9e4effd700 1 -- 172.21.15.89:0/1463115776 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4800b910 con 0x7f9e50135e50 2023-09-30T22:23:14.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.218+0000 7f9e4effd700 1 -- 172.21.15.89:0/1463115776 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4800bcc0 con 0x7f9e50135e50 2023-09-30T22:23:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f89683a6700 1 -- 172.21.15.89:0/86278823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f896009d5c0 msgr2=0x7f89600a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f89683a6700 1 --2- 172.21.15.89:0/86278823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f896009d5c0 0x7f89600a12d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f89500098f0 tx=0x7f8950009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f89683a6700 1 -- 172.21.15.89:0/86278823 shutdown_connections 2023-09-30T22:23:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f89683a6700 1 --2- 172.21.15.89:0/86278823 >> v2:172.21.15.89:3300/0 conn(0x7f89600a09c0 0x7f89600a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f89683a6700 1 --2- 172.21.15.89:0/86278823 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f896009d5c0 0x7f89600a12d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f89683a6700 1 -- 172.21.15.89:0/86278823 >> 172.21.15.89:0/86278823 conn(0x7f89600a2fe0 msgr2=0x7f89600a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f89683a6700 1 -- 172.21.15.89:0/86278823 shutdown_connections 2023-09-30T22:23:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f89683a6700 1 -- 172.21.15.89:0/86278823 wait complete. 2023-09-30T22:23:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f9e56929700 1 -- 172.21.15.89:0/1463115776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e50135e50 msgr2=0x7f9e50138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f9e56929700 1 --2- 172.21.15.89:0/1463115776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e50135e50 0x7f9e50138230 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009e80 tx=0x7f9e4800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f9e56929700 1 -- 172.21.15.89:0/1463115776 shutdown_connections 2023-09-30T22:23:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f9e56929700 1 --2- 172.21.15.89:0/1463115776 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e50135e50 0x7f9e50138230 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f9e56929700 1 --2- 172.21.15.89:0/1463115776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 0x7f9e50135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f9e56929700 1 -- 172.21.15.89:0/1463115776 >> 172.21.15.89:0/1463115776 conn(0x7f9e50066f50 msgr2=0x7f9e5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f9e56929700 1 -- 172.21.15.89:0/1463115776 shutdown_connections 2023-09-30T22:23:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.219+0000 7f9e56929700 1 -- 172.21.15.89:0/1463115776 wait complete. 2023-09-30T22:23:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.220+0000 7f9e56929700 1 Processor -- start 2023-09-30T22:23:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.220+0000 7f89683a6700 1 Processor -- start 2023-09-30T22:23:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.220+0000 7f89683a6700 1 -- start start 2023-09-30T22:23:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.220+0000 7f9e56929700 1 -- start start 2023-09-30T22:23:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e56929700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e5009d6f0 0x7f9e50119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e56929700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 0x7f9e501199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e56929700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e50135e50 0x7f9e50119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e56929700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5011b0b0 con 0x7f9e50135e50 2023-09-30T22:23:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e56929700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5011cbb0 con 0x7f9e5009e1d0 2023-09-30T22:23:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e56929700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500a1d00 con 0x7f9e5009d6f0 2023-09-30T22:23:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f89683a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896009d5c0 0x7f89601211d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f89683a6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89600a09c0 0x7f8960121710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f89683a6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8960133920 0x7f8960121c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f89683a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8960123fd0 con 0x7f896009d5c0 2023-09-30T22:23:14.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f89683a6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896009cec0 con 0x7f8960133920 2023-09-30T22:23:14.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e5009d6f0 0x7f9e50119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e5009d6f0 0x7f9e50119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47910/0 (socket says 172.21.15.89:47910) 2023-09-30T22:23:14.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4ffff700 1 -- 172.21.15.89:0/1095746835 learned_addr learned my addr 172.21.15.89:0/1095746835 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:14.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f89683a6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8960122e20 con 0x7f89600a09c0 2023-09-30T22:23:14.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8966943700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8960133920 0x7f8960121c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8966943700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8960133920 0x7f8960121c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37324/0 (socket says 172.21.15.89:37324) 2023-09-30T22:23:14.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8966943700 1 -- 172.21.15.89:0/2353465820 learned_addr learned my addr 172.21.15.89:0/2353465820 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:14.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4f7fe700 1 --2- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 0x7f9e501199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4ffff700 1 -- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 msgr2=0x7f9e501199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4ffff700 1 --2- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 0x7f9e501199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4ffff700 1 -- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e50135e50 msgr2=0x7f9e50119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8965941700 1 --2- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89600a09c0 0x7f8960121710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4ffff700 1 --2- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e50135e50 0x7f9e50119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f9e4ffff700 1 -- 172.21.15.89:0/1095746835 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e48009b30 con 0x7f9e5009d6f0 2023-09-30T22:23:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8966943700 1 -- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89600a09c0 msgr2=0x7f8960121710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8966943700 1 --2- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89600a09c0 0x7f8960121710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8966943700 1 -- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896009d5c0 msgr2=0x7f89601211d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8966943700 1 --2- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896009d5c0 0x7f89601211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.221+0000 7f8966943700 1 -- 172.21.15.89:0/2353465820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009550 con 0x7f8960133920 2023-09-30T22:23:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f9e4ffff700 1 --2- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e5009d6f0 0x7f9e50119480 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000c450 tx=0x7f9e4000c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f8966943700 1 --2- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8960133920 0x7f8960121c50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f895800cc50 tx=0x7f895800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f9e4cff9700 1 -- 172.21.15.89:0/1095746835 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e400052d0 con 0x7f9e5009d6f0 2023-09-30T22:23:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f8956ffd700 1 -- 172.21.15.89:0/2353465820 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895800ad80 con 0x7f8960133920 2023-09-30T22:23:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f8956ffd700 1 -- 172.21.15.89:0/2353465820 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895800ce50 con 0x7f8960133920 2023-09-30T22:23:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f8956ffd700 1 -- 172.21.15.89:0/2353465820 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958003c40 con 0x7f8960133920 2023-09-30T22:23:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5011b770 con 0x7f9e5009d6f0 2023-09-30T22:23:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f9e4cff9700 1 -- 172.21.15.89:0/1095746835 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e40005430 con 0x7f9e5009d6f0 2023-09-30T22:23:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5011ce90 con 0x7f9e5009d6f0 2023-09-30T22:23:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8960124680 con 0x7f8960133920 2023-09-30T22:23:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.222+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8960124c20 con 0x7f8960133920 2023-09-30T22:23:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.223+0000 7f9e4cff9700 1 -- 172.21.15.89:0/1095746835 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e400038c0 con 0x7f9e5009d6f0 2023-09-30T22:23:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.223+0000 7f9e4cff9700 1 -- 172.21.15.89:0/1095746835 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e40003a20 con 0x7f9e5009d6f0 2023-09-30T22:23:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.223+0000 7f8956ffd700 1 -- 172.21.15.89:0/2353465820 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89580050d0 con 0x7f8960133920 2023-09-30T22:23:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.224+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89480052e0 con 0x7f8960133920 2023-09-30T22:23:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.223+0000 7f9e4cff9700 1 -- 172.21.15.89:0/1095746835 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9e40010e00 con 0x7f9e5009d6f0 2023-09-30T22:23:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.224+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e300052e0 con 0x7f9e5009d6f0 2023-09-30T22:23:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.224+0000 7f8956ffd700 1 -- 172.21.15.89:0/2353465820 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f895803f7f0 con 0x7f8960133920 2023-09-30T22:23:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.228+0000 7f8956ffd700 1 -- 172.21.15.89:0/2353465820 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8958003da0 con 0x7f8960133920 2023-09-30T22:23:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.229+0000 7f9e4cff9700 1 -- 172.21.15.89:0/1095746835 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e40017080 con 0x7f9e5009d6f0 2023-09-30T22:23:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.287+0000 7f9918e7b700 1 Processor -- start 2023-09-30T22:23:14.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.287+0000 7f9918e7b700 1 -- start start 2023-09-30T22:23:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.287+0000 7f9918e7b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409d6f0 0x7f991409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.287+0000 7f9918e7b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f991409e000 0x7f99141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.287+0000 7f9918e7b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f9914133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.287+0000 7f9918e7b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991406b460 con 0x7f991409e000 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.287+0000 7f9918e7b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991409b120 con 0x7f991409e410 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.287+0000 7f9918e7b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f991409a7e0 con 0x7f991409d6f0 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f991409e000 0x7f99141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9912d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f9914133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f991409e000 0x7f99141339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45890/0 (socket says 172.21.15.89:45890) 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 -- 172.21.15.89:0/4135851246 learned_addr learned my addr 172.21.15.89:0/4135851246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f991259c700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409d6f0 0x7f991409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 -- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409d6f0 msgr2=0x7f991409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409d6f0 0x7f991409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 -- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 msgr2=0x7f9914133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f9914133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 -- 172.21.15.89:0/4135851246 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991409a500 con 0x7f991409e000 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f991259c700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409d6f0 0x7f991409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9912d9d700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f9914133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f9911d9b700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409e000 0x7f99141339f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f99080098f0 tx=0x7f9908009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8f7c5f3a214c9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f991159a700 1 -- 172.21.15.89:0/4135851246 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908011830 con 0x7f991409e000 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f991159a700 1 -- 172.21.15.89:0/4135851246 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9908011990 con 0x7f991409e000 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.288+0000 7f991159a700 1 -- 172.21.15.89:0/4135851246 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908011d20 con 0x7f991409e000 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 -- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409e000 msgr2=0x7f99141339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409e000 0x7f99141339f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f99080098f0 tx=0x7f9908009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 -- 172.21.15.89:0/4135851246 shutdown_connections 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f9914133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409d6f0 0x7f991409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 --2- 172.21.15.89:0/4135851246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409e000 0x7f99141339f0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 -- 172.21.15.89:0/4135851246 >> 172.21.15.89:0/4135851246 conn(0x7f9914066f50 msgr2=0x7f991409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 -- 172.21.15.89:0/4135851246 shutdown_connections 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.289+0000 7f9918e7b700 1 -- 172.21.15.89:0/4135851246 wait complete. 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.290+0000 7f9918e7b700 1 Processor -- start 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.290+0000 7f9918e7b700 1 -- start start 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9918e7b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409d6f0 0x7f99141c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9918e7b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409e000 0x7f99141c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9918e7b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f99141c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9918e7b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99141a71a0 con 0x7f991409d6f0 2023-09-30T22:23:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9918e7b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99141c5ad0 con 0x7f991409e410 2023-09-30T22:23:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9918e7b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99141a9810 con 0x7f991409e000 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f991259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409d6f0 0x7f99141c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f991259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409d6f0 0x7f99141c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45896/0 (socket says 172.21.15.89:45896) 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f991259c700 1 -- 172.21.15.89:0/2089289182 learned_addr learned my addr 172.21.15.89:0/2089289182 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9912d9d700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f99141c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9911d9b700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409e000 0x7f99141c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f991259c700 1 -- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409e000 msgr2=0x7f99141c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f991259c700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409e000 0x7f99141c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f991259c700 1 -- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 msgr2=0x7f99141c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f991259c700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f99141c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f991259c700 1 -- 172.21.15.89:0/2089289182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c009a70 con 0x7f991409d6f0 2023-09-30T22:23:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.291+0000 7f9911d9b700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409e000 0x7f99141c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.292+0000 7f991259c700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409d6f0 0x7f99141c5050 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f990400e810 tx=0x7f990400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.292+0000 7f9902ffd700 1 -- 172.21.15.89:0/2089289182 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904013610 con 0x7f991409d6f0 2023-09-30T22:23:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.292+0000 7f9902ffd700 1 -- 172.21.15.89:0/2089289182 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9904013770 con 0x7f991409d6f0 2023-09-30T22:23:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.292+0000 7f9902ffd700 1 -- 172.21.15.89:0/2089289182 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904013ae0 con 0x7f991409d6f0 2023-09-30T22:23:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.292+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9908009550 con 0x7f991409d6f0 2023-09-30T22:23:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.292+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99141a8b50 con 0x7f991409d6f0 2023-09-30T22:23:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.293+0000 7f9902ffd700 1 -- 172.21.15.89:0/2089289182 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9904013c40 con 0x7f991409d6f0 2023-09-30T22:23:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.293+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9914074ea0 con 0x7f991409d6f0 2023-09-30T22:23:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.293+0000 7f9902ffd700 1 -- 172.21.15.89:0/2089289182 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f990403f3a0 con 0x7f991409d6f0 2023-09-30T22:23:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.302+0000 7f9902ffd700 1 -- 172.21.15.89:0/2089289182 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f990400c3f0 con 0x7f991409d6f0 2023-09-30T22:23:14.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.351+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2dc009a1a0 con 0x7f2dc012f880 2023-09-30T22:23:14.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.351+0000 7f2dbcff9700 1 -- 172.21.15.89:0/2425998657 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2db40192c0 con 0x7f2dc012f880 2023-09-30T22:23:14.351 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:14.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.354+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012f880 msgr2=0x7f2dc0070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.354+0000 7f2dc6480700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012f880 0x7f2dc0070ee0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2db400c550 tx=0x7f2db400c860 comp rx=0 tx=0).stop 2023-09-30T22:23:14.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.355+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 shutdown_connections 2023-09-30T22:23:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.355+0000 7f2dc6480700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0130380 0x7f2dc0071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.355+0000 7f2dc6480700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2dc012c9a0 0x7f2dc0098b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.355+0000 7f2dc6480700 1 --2- 172.21.15.89:0/2425998657 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2dc012f880 0x7f2dc0070ee0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.355+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 >> 172.21.15.89:0/2425998657 conn(0x7f2dc00a2360 msgr2=0x7f2dc01b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.355+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 shutdown_connections 2023-09-30T22:23:14.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.355+0000 7f2dc6480700 1 -- 172.21.15.89:0/2425998657 wait complete. 2023-09-30T22:23:14.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.432+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e30005150 con 0x7f9e5009d6f0 2023-09-30T22:23:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.432+0000 7f9e4cff9700 1 -- 172.21.15.89:0/1095746835 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e40044060 con 0x7f9e5009d6f0 2023-09-30T22:23:14.432 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e5009d6f0 msgr2=0x7f9e50119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 --2- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e5009d6f0 0x7f9e50119480 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000c450 tx=0x7f9e4000c760 comp rx=0 tx=0).stop 2023-09-30T22:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 shutdown_connections 2023-09-30T22:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 --2- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e50135e50 0x7f9e50119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 --2- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e5009d6f0 0x7f9e50119480 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 --2- 172.21.15.89:0/1095746835 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5009e1d0 0x7f9e501199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 >> 172.21.15.89:0/1095746835 conn(0x7f9e50066f50 msgr2=0x7f9e500a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 shutdown_connections 2023-09-30T22:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.435+0000 7f9e56929700 1 -- 172.21.15.89:0/1095746835 wait complete. 2023-09-30T22:23:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.445+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8948005590 con 0x7f8960133920 2023-09-30T22:23:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.446+0000 7f8956ffd700 1 -- 172.21.15.89:0/2353465820 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8958001cb0 con 0x7f8960133920 2023-09-30T22:23:14.446 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8960133920 msgr2=0x7f8960121c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 --2- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8960133920 0x7f8960121c50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f895800cc50 tx=0x7f895800a340 comp rx=0 tx=0).stop 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 shutdown_connections 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 --2- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8960133920 0x7f8960121c50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 --2- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896009d5c0 0x7f89601211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 --2- 172.21.15.89:0/2353465820 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89600a09c0 0x7f8960121710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 >> 172.21.15.89:0/2353465820 conn(0x7f89600a2fe0 msgr2=0x7f8960133190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 shutdown_connections 2023-09-30T22:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.448+0000 7f89683a6700 1 -- 172.21.15.89:0/2353465820 wait complete. 2023-09-30T22:23:14.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:14.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.504+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f99141a7900 con 0x7f991409d6f0 2023-09-30T22:23:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.504+0000 7f9902ffd700 1 -- 172.21.15.89:0/2089289182 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9904021030 con 0x7f991409d6f0 2023-09-30T22:23:14.504 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.506+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409d6f0 msgr2=0x7f99141c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.506+0000 7f9918e7b700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409d6f0 0x7f99141c5050 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f990400e810 tx=0x7f990400ebd0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.506+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 shutdown_connections 2023-09-30T22:23:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.506+0000 7f9918e7b700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f991409e410 0x7f99141c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.507+0000 7f9918e7b700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991409d6f0 0x7f99141c5050 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.507+0000 7f9918e7b700 1 --2- 172.21.15.89:0/2089289182 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991409e000 0x7f99141c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.507+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 >> 172.21.15.89:0/2089289182 conn(0x7f9914066f50 msgr2=0x7f99141badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.507+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 shutdown_connections 2023-09-30T22:23:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:14.507+0000 7f9918e7b700 1 -- 172.21.15.89:0/2089289182 wait complete. 2023-09-30T22:23:14.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:15.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:15.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:15.461 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:15.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f63251d6700 1 Processor -- start 2023-09-30T22:23:15.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f63251d6700 1 -- start start 2023-09-30T22:23:15.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f63251d6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f632009d6f0 0x7f632009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f63251d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f63251d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009e410 0x7f6320133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f63251d6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632006b460 con 0x7f632009d6f0 2023-09-30T22:23:15.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f63251d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632009b120 con 0x7f632009e410 2023-09-30T22:23:15.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f63251d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f632009a7e0 con 0x7f632009e000 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f631f59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009e410 0x7f6320133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f631e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f631f59e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009e410 0x7f6320133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37338/0 (socket says 172.21.15.89:37338) 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.512+0000 7f631f59e700 1 -- 172.21.15.89:0/21251475 learned_addr learned my addr 172.21.15.89:0/21251475 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631ed9d700 1 --2- 172.21.15.89:0/21251475 >> v2:172.21.15.89:3300/0 conn(0x7f632009d6f0 0x7f632009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631f59e700 1 -- 172.21.15.89:0/21251475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 msgr2=0x7f63201339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631f59e700 1 --2- 172.21.15.89:0/21251475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631f59e700 1 -- 172.21.15.89:0/21251475 >> v2:172.21.15.89:3300/0 conn(0x7f632009d6f0 msgr2=0x7f632009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:15.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631f59e700 1 --2- 172.21.15.89:0/21251475 >> v2:172.21.15.89:3300/0 conn(0x7f632009d6f0 0x7f632009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631f59e700 1 -- 172.21.15.89:0/21251475 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632009a500 con 0x7f632009e410 2023-09-30T22:23:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631e59c700 1 --2- 172.21.15.89:0/21251475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631ed9d700 1 --2- 172.21.15.89:0/21251475 >> v2:172.21.15.89:3300/0 conn(0x7f632009d6f0 0x7f632009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631f59e700 1 --2- 172.21.15.89:0/21251475 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009e410 0x7f6320133f30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6318009e80 tx=0x7f631800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e043c75a029e6631 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631dd9b700 1 -- 172.21.15.89:0/21251475 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800e070 con 0x7f632009e410 2023-09-30T22:23:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631dd9b700 1 -- 172.21.15.89:0/21251475 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631800b8d0 con 0x7f632009e410 2023-09-30T22:23:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.513+0000 7f631dd9b700 1 -- 172.21.15.89:0/21251475 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800bc80 con 0x7f632009e410 2023-09-30T22:23:15.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 -- 172.21.15.89:0/21251475 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009e410 msgr2=0x7f6320133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 --2- 172.21.15.89:0/21251475 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009e410 0x7f6320133f30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6318009e80 tx=0x7f631800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:15.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 -- 172.21.15.89:0/21251475 shutdown_connections 2023-09-30T22:23:15.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 --2- 172.21.15.89:0/21251475 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009e410 0x7f6320133f30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 --2- 172.21.15.89:0/21251475 >> v2:172.21.15.89:3300/0 conn(0x7f632009d6f0 0x7f632009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 --2- 172.21.15.89:0/21251475 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 -- 172.21.15.89:0/21251475 >> 172.21.15.89:0/21251475 conn(0x7f6320066f50 msgr2=0x7f632009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 -- 172.21.15.89:0/21251475 shutdown_connections 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.514+0000 7f63251d6700 1 -- 172.21.15.89:0/21251475 wait complete. 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.515+0000 7f63251d6700 1 Processor -- start 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.515+0000 7f63251d6700 1 -- start start 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f63251d6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009d6f0 0x7f63201a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f63251d6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f63251d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632009e410 0x7f63201a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f63251d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63201a5260 con 0x7f632009e410 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f63251d6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63200a2000 con 0x7f632009d6f0 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f63251d6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f632009c710 con 0x7f632009e000 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632009e410 0x7f63201a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632009e410 0x7f63201a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45914/0 (socket says 172.21.15.89:45914) 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 -- 172.21.15.89:0/125502115 learned_addr learned my addr 172.21.15.89:0/125502115 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631e59c700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631ed9d700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009d6f0 0x7f63201a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 -- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 msgr2=0x7f63201a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201a3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 -- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009d6f0 msgr2=0x7f63201a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009d6f0 0x7f63201a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 -- 172.21.15.89:0/125502115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100094a0 con 0x7f632009e410 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.516+0000 7f631f59e700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632009e410 0x7f63201a4170 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f632009b3c0 tx=0x7f6318004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.517+0000 7f63077fe700 1 -- 172.21.15.89:0/125502115 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318004450 con 0x7f632009e410 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.517+0000 7f63077fe700 1 -- 172.21.15.89:0/125502115 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63180045b0 con 0x7f632009e410 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.517+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63140094a0 con 0x7f632009e410 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.517+0000 7f63077fe700 1 -- 172.21.15.89:0/125502115 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318004960 con 0x7f632009e410 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.517+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6318009b30 con 0x7f632009e410 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.517+0000 7f63077fe700 1 -- 172.21.15.89:0/125502115 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f631801c410 con 0x7f632009e410 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.518+0000 7f63077fe700 1 -- 172.21.15.89:0/125502115 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6318040f50 con 0x7f632009e410 2023-09-30T22:23:15.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.518+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63201a6b30 con 0x7f632009e410 2023-09-30T22:23:15.523 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:15.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.523+0000 7f63077fe700 1 -- 172.21.15.89:0/125502115 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f63201a6b30 con 0x7f632009e410 2023-09-30T22:23:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.602+0000 7fa94ebc1700 1 Processor -- start 2023-09-30T22:23:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.602+0000 7fa94ebc1700 1 -- start start 2023-09-30T22:23:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa94ebc1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa94812cf60 0x7fa94806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa94ebc1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa948137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa94ebc1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94812cd90 con 0x7fa94812f8b0 2023-09-30T22:23:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa94ebc1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94812d330 con 0x7fa94812cf60 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa94ebc1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa948071780 con 0x7fa948130430 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa947fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa94812f8b0 0x7fa94812fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47964/0 (socket says 172.21.15.89:47964) 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa94d15e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa948137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa94c95d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa94812cf60 0x7fa94806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.603+0000 7fa947fff700 1 -- 172.21.15.89:0/2648228281 learned_addr learned my addr 172.21.15.89:0/2648228281 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.604+0000 7fa94c95d700 1 -- 172.21.15.89:0/2648228281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 msgr2=0x7fa948137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.604+0000 7fa94c95d700 1 --2- 172.21.15.89:0/2648228281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa948137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.604+0000 7fa94c95d700 1 -- 172.21.15.89:0/2648228281 >> v1:172.21.15.89:6789/0 conn(0x7fa94812f8b0 legacy=0x7fa94812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.604+0000 7fa94c95d700 1 -- 172.21.15.89:0/2648228281 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948071a40 con 0x7fa94812cf60 2023-09-30T22:23:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.604+0000 7fa94c95d700 1 --2- 172.21.15.89:0/2648228281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812cf60 0x7fa94806f5d0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00bad0 tx=0x7fa93c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06fd845bc02d6c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.604+0000 7fa9477fe700 1 -- 172.21.15.89:0/2648228281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c006d10 con 0x7fa94812cf60 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.604+0000 7fa9477fe700 1 -- 172.21.15.89:0/2648228281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c006e70 con 0x7fa94812cf60 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.604+0000 7fa9477fe700 1 -- 172.21.15.89:0/2648228281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00dc80 con 0x7fa94812cf60 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.605+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2648228281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812cf60 msgr2=0x7fa94806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.605+0000 7fa94ebc1700 1 --2- 172.21.15.89:0/2648228281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812cf60 0x7fa94806f5d0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00bad0 tx=0x7fa93c00bde0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.605+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2648228281 shutdown_connections 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.605+0000 7fa94ebc1700 1 --2- 172.21.15.89:0/2648228281 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa948137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.605+0000 7fa94ebc1700 1 --2- 172.21.15.89:0/2648228281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812cf60 0x7fa94806f5d0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.605+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2648228281 >> 172.21.15.89:0/2648228281 conn(0x7fa9480a2aa0 msgr2=0x7fa9480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.605+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2648228281 shutdown_connections 2023-09-30T22:23:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.605+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2648228281 wait complete. 2023-09-30T22:23:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.606+0000 7fa94ebc1700 1 Processor -- start 2023-09-30T22:23:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.606+0000 7fa94ebc1700 1 -- start start 2023-09-30T22:23:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa94ebc1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94812cf60 0x7fa9481c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa94ebc1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812f8b0 0x7fa9481c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa94ebc1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa9481c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa94ebc1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94809e810 con 0x7fa94812f8b0 2023-09-30T22:23:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa94ebc1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94812f590 con 0x7fa948130430 2023-09-30T22:23:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa94ebc1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa94809e580 con 0x7fa94812cf60 2023-09-30T22:23:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa947fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812f8b0 0x7fa9481c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa947fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812f8b0 0x7fa9481c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45926/0 (socket says 172.21.15.89:45926) 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa947fff700 1 -- 172.21.15.89:0/2821018751 learned_addr learned my addr 172.21.15.89:0/2821018751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa94d15e700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa9481c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.607+0000 7fa94c95d700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94812cf60 0x7fa9481c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa947fff700 1 -- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94812cf60 msgr2=0x7fa9481c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa947fff700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94812cf60 0x7fa9481c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa947fff700 1 -- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 msgr2=0x7fa9481c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa947fff700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa9481c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa947fff700 1 -- 172.21.15.89:0/2821018751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c00b730 con 0x7fa94812f8b0 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa94c95d700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94812cf60 0x7fa9481c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa94d15e700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa9481c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa947fff700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812f8b0 0x7fa9481c8850 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fa93400c550 tx=0x7fa93400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa9457fa700 1 -- 172.21.15.89:0/2821018751 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934005d70 con 0x7fa94812f8b0 2023-09-30T22:23:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa9457fa700 1 -- 172.21.15.89:0/2821018751 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9340070d0 con 0x7fa94812f8b0 2023-09-30T22:23:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94809cbe0 con 0x7fa94812f8b0 2023-09-30T22:23:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa9457fa700 1 -- 172.21.15.89:0/2821018751 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934007440 con 0x7fa94812f8b0 2023-09-30T22:23:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.608+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94809e1f0 con 0x7fa94812f8b0 2023-09-30T22:23:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.609+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa948074ea0 con 0x7fa94812f8b0 2023-09-30T22:23:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.610+0000 7fa9457fa700 1 -- 172.21.15.89:0/2821018751 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9340075a0 con 0x7fa94812f8b0 2023-09-30T22:23:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.610+0000 7fa9457fa700 1 -- 172.21.15.89:0/2821018751 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa93403eef0 con 0x7fa94812f8b0 2023-09-30T22:23:15.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.618+0000 7fa9457fa700 1 -- 172.21.15.89:0/2821018751 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa934019b40 con 0x7fa94812f8b0 2023-09-30T22:23:15.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.630+0000 7f6ea7e2b700 1 Processor -- start 2023-09-30T22:23:15.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.630+0000 7f6ea7e2b700 1 -- start start 2023-09-30T22:23:15.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea7e2b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0133420 0x7f6ea01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea7e2b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0134260 0x7f6ea01306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea7e2b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea006b990 con 0x7f6ea0133e30 2023-09-30T22:23:15.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea7e2b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea00a0590 con 0x7f6ea0134260 2023-09-30T22:23:15.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea7e2b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea009fb80 con 0x7f6ea0133420 2023-09-30T22:23:15.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea53c6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6ea0133e30 0x7f6ea012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47966/0 (socket says 172.21.15.89:47966) 2023-09-30T22:23:15.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea53c6700 1 -- 172.21.15.89:0/2315066083 learned_addr learned my addr 172.21.15.89:0/2315066083 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:15.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea5bc7700 1 --2- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0133420 0x7f6ea01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea63c8700 1 --2- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0134260 0x7f6ea01306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea5bc7700 1 -- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0134260 msgr2=0x7f6ea01306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea5bc7700 1 --2- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0134260 0x7f6ea01306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea5bc7700 1 -- 172.21.15.89:0/2315066083 >> v1:172.21.15.89:6789/0 conn(0x7f6ea0133e30 legacy=0x7f6ea012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea5bc7700 1 -- 172.21.15.89:0/2315066083 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea00a0030 con 0x7f6ea0133420 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea4bc5700 1 -- 172.21.15.89:0/2315066083 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849099869 0 0) 0x7f6ea006b990 con 0x7f6ea0133e30 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.631+0000 7f6ea5bc7700 1 --2- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0133420 0x7f6ea01337f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009870 tx=0x7f6e9c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af03e8d69749d237 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea4bc5700 1 -- 172.21.15.89:0/2315066083 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c011830 con 0x7f6ea0133420 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea4bc5700 1 -- 172.21.15.89:0/2315066083 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c011990 con 0x7f6ea0133420 2023-09-30T22:23:15.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea4bc5700 1 -- 172.21.15.89:0/2315066083 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c011d20 con 0x7f6ea0133420 2023-09-30T22:23:15.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0133420 msgr2=0x7f6ea01337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea7e2b700 1 --2- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0133420 0x7f6ea01337f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009870 tx=0x7f6e9c009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:15.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/2315066083 shutdown_connections 2023-09-30T22:23:15.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea7e2b700 1 --2- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0134260 0x7f6ea01306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea7e2b700 1 --2- 172.21.15.89:0/2315066083 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0133420 0x7f6ea01337f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/2315066083 >> 172.21.15.89:0/2315066083 conn(0x7f6ea00a4210 msgr2=0x7f6ea00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/2315066083 shutdown_connections 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.632+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/2315066083 wait complete. 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.633+0000 7f6ea7e2b700 1 Processor -- start 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea7e2b700 1 -- start start 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea7e2b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0133420 0x7f6ea009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea7e2b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ea0133e30 0x7f6ea009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea7e2b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0134260 0x7f6ea009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea7e2b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea009de30 con 0x7f6ea0133e30 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea7e2b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea009dbd0 con 0x7f6ea0133420 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea7e2b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea011fe20 con 0x7f6ea0134260 2023-09-30T22:23:15.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea53c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ea0133e30 0x7f6ea009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea53c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ea0133e30 0x7f6ea009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45936/0 (socket says 172.21.15.89:45936) 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea53c6700 1 -- 172.21.15.89:0/555707106 learned_addr learned my addr 172.21.15.89:0/555707106 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea5bc7700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0133420 0x7f6ea009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.634+0000 7f6ea63c8700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0134260 0x7f6ea009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6ea53c6700 1 -- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0134260 msgr2=0x7f6ea009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6ea53c6700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0134260 0x7f6ea009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6ea53c6700 1 -- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0133420 msgr2=0x7f6ea009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6ea53c6700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0133420 0x7f6ea009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6ea53c6700 1 -- 172.21.15.89:0/555707106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0094a0 con 0x7f6ea0133e30 2023-09-30T22:23:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6ea53c6700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ea0133e30 0x7f6ea009c9c0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6e90001eb0 tx=0x7f6e9000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:15.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6e967fc700 1 -- 172.21.15.89:0/555707106 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90004c80 con 0x7f6ea0133e30 2023-09-30T22:23:15.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6e967fc700 1 -- 172.21.15.89:0/555707106 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e90004de0 con 0x7f6ea0133e30 2023-09-30T22:23:15.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea0120090 con 0x7f6ea0133e30 2023-09-30T22:23:15.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.635+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea011ed80 con 0x7f6ea0133e30 2023-09-30T22:23:15.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.636+0000 7f6e967fc700 1 -- 172.21.15.89:0/555707106 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90007310 con 0x7f6ea0133e30 2023-09-30T22:23:15.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.636+0000 7f6e967fc700 1 -- 172.21.15.89:0/555707106 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e900074f0 con 0x7f6ea0133e30 2023-09-30T22:23:15.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.636+0000 7f6e967fc700 1 -- 172.21.15.89:0/555707106 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6e9003f380 con 0x7f6ea0133e30 2023-09-30T22:23:15.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.637+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea011fc60 con 0x7f6ea0133e30 2023-09-30T22:23:15.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.646+0000 7f6e967fc700 1 -- 172.21.15.89:0/555707106 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e9003f030 con 0x7f6ea0133e30 2023-09-30T22:23:15.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.660+0000 7fd9df6a6700 1 Processor -- start 2023-09-30T22:23:15.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.660+0000 7fd9df6a6700 1 -- start start 2023-09-30T22:23:15.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.660+0000 7fd9df6a6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 0x7fd9d809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.660+0000 7fd9df6a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd9d809e000 0x7fd9d81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9df6a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d809b120 con 0x7fd9d809e410 2023-09-30T22:23:15.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9df6a6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d806b460 con 0x7fd9d809e000 2023-09-30T22:23:15.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9df6a6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d809a7e0 con 0x7fd9d809d6f0 2023-09-30T22:23:15.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9ddc43700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd9d809e410 0x7fd9d8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47976/0 (socket says 172.21.15.89:47976) 2023-09-30T22:23:15.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9dcc41700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd9d809e000 0x7fd9d81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9ddc43700 1 -- 172.21.15.89:0/1269096133 learned_addr learned my addr 172.21.15.89:0/1269096133 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:15.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9dd442700 1 --2- 172.21.15.89:0/1269096133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 0x7fd9d809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9dcc41700 1 -- 172.21.15.89:0/1269096133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 msgr2=0x7fd9d809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:15.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9dcc41700 1 --2- 172.21.15.89:0/1269096133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 0x7fd9d809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9dcc41700 1 -- 172.21.15.89:0/1269096133 >> v1:172.21.15.89:6789/0 conn(0x7fd9d809e410 legacy=0x7fd9d8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9cffff700 1 -- 172.21.15.89:0/1269096133 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139157834 0 0) 0x7fd9d809b120 con 0x7fd9d809e410 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9dcc41700 1 -- 172.21.15.89:0/1269096133 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d809a500 con 0x7fd9d809e000 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.661+0000 7fd9dcc41700 1 --2- 172.21.15.89:0/1269096133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e000 0x7fd9d81339f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009840 tx=0x7fd9c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bce1a72811f2b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9cffff700 1 -- 172.21.15.89:0/1269096133 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8011830 con 0x7fd9d809e000 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9cffff700 1 -- 172.21.15.89:0/1269096133 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c8011990 con 0x7fd9d809e000 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9cffff700 1 -- 172.21.15.89:0/1269096133 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8011d20 con 0x7fd9d809e000 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9df6a6700 1 -- 172.21.15.89:0/1269096133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e000 msgr2=0x7fd9d81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9df6a6700 1 --2- 172.21.15.89:0/1269096133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e000 0x7fd9d81339f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009840 tx=0x7fd9c8009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9df6a6700 1 -- 172.21.15.89:0/1269096133 shutdown_connections 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9df6a6700 1 --2- 172.21.15.89:0/1269096133 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 0x7fd9d809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9df6a6700 1 --2- 172.21.15.89:0/1269096133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e000 0x7fd9d81339f0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9df6a6700 1 -- 172.21.15.89:0/1269096133 >> 172.21.15.89:0/1269096133 conn(0x7fd9d8066f50 msgr2=0x7fd9d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:15.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9df6a6700 1 -- 172.21.15.89:0/1269096133 shutdown_connections 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.662+0000 7fd9df6a6700 1 -- 172.21.15.89:0/1269096133 wait complete. 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.663+0000 7fd9df6a6700 1 Processor -- start 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.663+0000 7fd9df6a6700 1 -- start start 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9df6a6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 0x7fd9d81a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9df6a6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9d809e000 0x7fd9d81a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9df6a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e410 0x7fd9d81c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9df6a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81a8100 con 0x7fd9d809e410 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9df6a6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81a6a00 con 0x7fd9d809e000 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9df6a6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81a82e0 con 0x7fd9d809d6f0 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9dcc41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9d809e000 0x7fd9d81a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9dcc41700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9d809e000 0x7fd9d81a56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37378/0 (socket says 172.21.15.89:37378) 2023-09-30T22:23:15.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9dcc41700 1 -- 172.21.15.89:0/2063141447 learned_addr learned my addr 172.21.15.89:0/2063141447 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9ddc43700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e410 0x7fd9d81c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9dd442700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 0x7fd9d81a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9ddc43700 1 -- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 msgr2=0x7fd9d81a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9ddc43700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 0x7fd9d81a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9ddc43700 1 -- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9d809e000 msgr2=0x7fd9d81a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9ddc43700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9d809e000 0x7fd9d81a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9ddc43700 1 -- 172.21.15.89:0/2063141447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c80094a0 con 0x7fd9d809e410 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9dcc41700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9d809e000 0x7fd9d81a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.664+0000 7fd9ddc43700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e410 0x7fd9d81c8040 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000cbe0 tx=0x7fd9d000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.665+0000 7fd9cdffb700 1 -- 172.21.15.89:0/2063141447 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000aa40 con 0x7fd9d809e410 2023-09-30T22:23:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.665+0000 7fd9cdffb700 1 -- 172.21.15.89:0/2063141447 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d000abc0 con 0x7fd9d809e410 2023-09-30T22:23:15.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.665+0000 7fd9cdffb700 1 -- 172.21.15.89:0/2063141447 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00070d0 con 0x7fd9d809e410 2023-09-30T22:23:15.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.665+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d81a4d10 con 0x7fd9d809e410 2023-09-30T22:23:15.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.665+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d80a2d10 con 0x7fd9d809e410 2023-09-30T22:23:15.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.666+0000 7fd9cdffb700 1 -- 172.21.15.89:0/2063141447 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9d0007230 con 0x7fd9d809e410 2023-09-30T22:23:15.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.666+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d81a7f60 con 0x7fd9d809e410 2023-09-30T22:23:15.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.666+0000 7fd9cdffb700 1 -- 172.21.15.89:0/2063141447 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd9d003f020 con 0x7fd9d809e410 2023-09-30T22:23:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.675+0000 7fd9cdffb700 1 -- 172.21.15.89:0/2063141447 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9d81a7f60 con 0x7fd9d809e410 2023-09-30T22:23:15.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.738+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f63201a6b30 con 0x7f632009e410 2023-09-30T22:23:15.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.739+0000 7f63077fe700 1 -- 172.21.15.89:0/125502115 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f63201a6b30 con 0x7f632009e410 2023-09-30T22:23:15.739 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.741+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632009e410 msgr2=0x7f63201a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.741+0000 7f63251d6700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632009e410 0x7f63201a4170 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f632009b3c0 tx=0x7f6318004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.742+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 shutdown_connections 2023-09-30T22:23:15.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.742+0000 7f63251d6700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632009e410 0x7f63201a4170 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.742+0000 7f63251d6700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f632009d6f0 0x7f63201a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.742+0000 7f63251d6700 1 --2- 172.21.15.89:0/125502115 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632009e000 0x7f63201a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.742+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 >> 172.21.15.89:0/125502115 conn(0x7f6320066f50 msgr2=0x7f632006f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:15.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.743+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 shutdown_connections 2023-09-30T22:23:15.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.743+0000 7f63251d6700 1 -- 172.21.15.89:0/125502115 wait complete. 2023-09-30T22:23:15.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.823+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa9480713a0 con 0x7fa94812f8b0 2023-09-30T22:23:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.823+0000 7fa9457fa700 1 -- 172.21.15.89:0/2821018751 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa934017030 con 0x7fa94812f8b0 2023-09-30T22:23:15.823 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812f8b0 msgr2=0x7fa9481c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812f8b0 0x7fa9481c8850 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fa93400c550 tx=0x7fa93400c860 comp rx=0 tx=0).stop 2023-09-30T22:23:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 shutdown_connections 2023-09-30T22:23:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa948130430 0x7fa9481c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa94812cf60 0x7fa9481c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 --2- 172.21.15.89:0/2821018751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94812f8b0 0x7fa9481c8850 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 >> 172.21.15.89:0/2821018751 conn(0x7fa9480a2aa0 msgr2=0x7fa94806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 shutdown_connections 2023-09-30T22:23:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.825+0000 7fa94ebc1700 1 -- 172.21.15.89:0/2821018751 wait complete. 2023-09-30T22:23:15.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:15.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.843+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6ea0074ea0 con 0x7f6ea0133e30 2023-09-30T22:23:15.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.843+0000 7f6e967fc700 1 -- 172.21.15.89:0/555707106 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e900123b0 con 0x7f6ea0133e30 2023-09-30T22:23:15.843 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ea0133e30 msgr2=0x7f6ea009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ea0133e30 0x7f6ea009c9c0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6e90001eb0 tx=0x7f6e9000c800 comp rx=0 tx=0).stop 2023-09-30T22:23:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 shutdown_connections 2023-09-30T22:23:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ea0134260 0x7f6ea009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ea0133420 0x7f6ea009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 --2- 172.21.15.89:0/555707106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ea0133e30 0x7f6ea009c9c0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 >> 172.21.15.89:0/555707106 conn(0x7f6ea00a4210 msgr2=0x7f6ea01303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 shutdown_connections 2023-09-30T22:23:15.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.845+0000 7f6ea7e2b700 1 -- 172.21.15.89:0/555707106 wait complete. 2023-09-30T22:23:15.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:15.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.872+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd9d81a7f60 con 0x7fd9d809e410 2023-09-30T22:23:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.872+0000 7fd9cdffb700 1 -- 172.21.15.89:0/2063141447 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9d81a7f60 con 0x7fd9d809e410 2023-09-30T22:23:15.872 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e410 msgr2=0x7fd9d81c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e410 0x7fd9d81c8040 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000cbe0 tx=0x7fd9d000cef0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 shutdown_connections 2023-09-30T22:23:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9d809e410 0x7fd9d81c8040 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd9d809d6f0 0x7fd9d81a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 --2- 172.21.15.89:0/2063141447 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9d809e000 0x7fd9d81a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:15.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 >> 172.21.15.89:0/2063141447 conn(0x7fd9d8066f50 msgr2=0x7fd9d806f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:15.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 shutdown_connections 2023-09-30T22:23:15.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:15.874+0000 7fd9df6a6700 1 -- 172.21.15.89:0/2063141447 wait complete. 2023-09-30T22:23:15.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:16.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:16.838 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:16.858 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:16.887 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.897+0000 7f7568121700 1 Processor -- start 2023-09-30T22:23:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.897+0000 7f7568121700 1 -- start start 2023-09-30T22:23:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f7568121700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f75601305f0 0x7f75601309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f7568121700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006f660 0x7f756006fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f7568121700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75601301c0 con 0x7f756006ff90 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f7568121700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7560071ff0 con 0x7f75601305f0 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f7568121700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7560070e70 con 0x7f756006f660 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f7565ebd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f75601305f0 0x7f75601309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f75666be700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f756006ff90 0x7f7560137530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47982/0 (socket says 172.21.15.89:47982) 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f75666be700 1 -- 172.21.15.89:0/1723029816 learned_addr learned my addr 172.21.15.89:0/1723029816 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f75656bc700 1 --2- 172.21.15.89:0/1723029816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006f660 0x7f756006fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f75656bc700 1 -- 172.21.15.89:0/1723029816 >> v2:172.21.15.89:3300/0 conn(0x7f75601305f0 msgr2=0x7f75601309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f75656bc700 1 --2- 172.21.15.89:0/1723029816 >> v2:172.21.15.89:3300/0 conn(0x7f75601305f0 0x7f75601309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f75656bc700 1 -- 172.21.15.89:0/1723029816 >> v1:172.21.15.89:6789/0 conn(0x7f756006ff90 legacy=0x7f7560137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f75656bc700 1 -- 172.21.15.89:0/1723029816 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600722b0 con 0x7f756006f660 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.898+0000 7f7565ebd700 1 --2- 172.21.15.89:0/1723029816 >> v2:172.21.15.89:3300/0 conn(0x7f75601305f0 0x7f75601309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.899+0000 7f75656bc700 1 --2- 172.21.15.89:0/1723029816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006f660 0x7f756006fa50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7550009840 tx=0x7f7550009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7717b9a591137a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.899+0000 7f7564ebb700 1 -- 172.21.15.89:0/1723029816 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550010a90 con 0x7f756006f660 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.899+0000 7f7564ebb700 1 -- 172.21.15.89:0/1723029816 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550010bf0 con 0x7f756006f660 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.899+0000 7f7564ebb700 1 -- 172.21.15.89:0/1723029816 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550018440 con 0x7f756006f660 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.899+0000 7f7568121700 1 -- 172.21.15.89:0/1723029816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006f660 msgr2=0x7f756006fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.899+0000 7f7568121700 1 --2- 172.21.15.89:0/1723029816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006f660 0x7f756006fa50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7550009840 tx=0x7f7550009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.900+0000 7f7568121700 1 -- 172.21.15.89:0/1723029816 shutdown_connections 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.900+0000 7f7568121700 1 --2- 172.21.15.89:0/1723029816 >> v2:172.21.15.89:3300/0 conn(0x7f75601305f0 0x7f75601309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.900+0000 7f7568121700 1 --2- 172.21.15.89:0/1723029816 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006f660 0x7f756006fa50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.900+0000 7f7568121700 1 -- 172.21.15.89:0/1723029816 >> 172.21.15.89:0/1723029816 conn(0x7f75600a2aa0 msgr2=0x7f75600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.900+0000 7f7568121700 1 -- 172.21.15.89:0/1723029816 shutdown_connections 2023-09-30T22:23:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.900+0000 7f7568121700 1 -- 172.21.15.89:0/1723029816 wait complete. 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.901+0000 7f7568121700 1 Processor -- start 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.901+0000 7f7568121700 1 -- start start 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f7568121700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756006f660 0x7f756011b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f7568121700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006ff90 0x7f756011bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f7568121700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75601305f0 0x7f756011c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f7568121700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756011dd60 con 0x7f756006f660 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f7568121700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756011da40 con 0x7f75601305f0 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f7568121700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f756012ecc0 con 0x7f756006ff90 2023-09-30T22:23:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75666be700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75601305f0 0x7f756011c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75656bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006ff90 0x7f756011bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75666be700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75601305f0 0x7f756011c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37392/0 (socket says 172.21.15.89:37392) 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75666be700 1 -- 172.21.15.89:0/1105360292 learned_addr learned my addr 172.21.15.89:0/1105360292 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75656bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006ff90 0x7f756011bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48020/0 (socket says 172.21.15.89:48020) 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f7565ebd700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756006f660 0x7f756011b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75666be700 1 -- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006ff90 msgr2=0x7f756011bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75666be700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006ff90 0x7f756011bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75666be700 1 -- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756006f660 msgr2=0x7f756011b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75666be700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756006f660 0x7f756011b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f75666be700 1 -- 172.21.15.89:0/1105360292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c0094a0 con 0x7f75601305f0 2023-09-30T22:23:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.902+0000 7f7565ebd700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756006f660 0x7f756011b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.903+0000 7f75666be700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75601305f0 0x7f756011c300 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f755800cbc0 tx=0x7f755800ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.903+0000 7f75567fc700 1 -- 172.21.15.89:0/1105360292 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755800aaa0 con 0x7f75601305f0 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.903+0000 7f75567fc700 1 -- 172.21.15.89:0/1105360292 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755800ac00 con 0x7f75601305f0 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.903+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75500094a0 con 0x7f75601305f0 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.903+0000 7f75567fc700 1 -- 172.21.15.89:0/1105360292 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558012400 con 0x7f75601305f0 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.903+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756011f440 con 0x7f75601305f0 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.904+0000 7f75567fc700 1 -- 172.21.15.89:0/1105360292 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f755801b3b0 con 0x7f75601305f0 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.904+0000 7f75567fc700 1 -- 172.21.15.89:0/1105360292 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f755803f340 con 0x7f75601305f0 2023-09-30T22:23:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.904+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756011ed10 con 0x7f75601305f0 2023-09-30T22:23:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.913+0000 7f75567fc700 1 -- 172.21.15.89:0/1105360292 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7558017020 con 0x7f75601305f0 2023-09-30T22:23:16.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67fdd19700 1 Processor -- start 2023-09-30T22:23:16.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67fdd19700 1 -- start start 2023-09-30T22:23:16.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67fdd19700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f81338a0 0x7f67f8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67fdd19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67f809e930 0x7f67f813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67fdd19700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f806ef30 con 0x7f67f809de50 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67fdd19700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f806fef0 con 0x7f67f809e930 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67fdd19700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f81334b0 con 0x7f67f81338a0 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f81338a0 0x7f67f8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67f77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f81338a0 0x7f67f8133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37406/0 (socket says 172.21.15.89:37406) 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67f77fe700 1 -- 172.21.15.89:0/3747174810 learned_addr learned my addr 172.21.15.89:0/3747174810 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.974+0000 7f67f7fff700 1 --2- 172.21.15.89:0/3747174810 >> v2:172.21.15.89:3300/0 conn(0x7f67f809e930 0x7f67f813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.975+0000 7f67f77fe700 1 -- 172.21.15.89:0/3747174810 >> v2:172.21.15.89:3300/0 conn(0x7f67f809e930 msgr2=0x7f67f813b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.975+0000 7f67f77fe700 1 --2- 172.21.15.89:0/3747174810 >> v2:172.21.15.89:3300/0 conn(0x7f67f809e930 0x7f67f813b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.975+0000 7f67f77fe700 1 -- 172.21.15.89:0/3747174810 >> v1:172.21.15.89:6789/0 conn(0x7f67f809de50 legacy=0x7f67f809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.975+0000 7f67f77fe700 1 -- 172.21.15.89:0/3747174810 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f806fc30 con 0x7f67f81338a0 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.975+0000 7f67f77fe700 1 --2- 172.21.15.89:0/3747174810 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f81338a0 0x7f67f8133c70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f67e80098f0 tx=0x7f67e8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19ef696a8226548a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.975+0000 7f67f67fc700 1 -- 172.21.15.89:0/3747174810 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8011830 con 0x7f67f81338a0 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.975+0000 7f67f67fc700 1 -- 172.21.15.89:0/3747174810 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8011990 con 0x7f67f81338a0 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.975+0000 7f67f67fc700 1 -- 172.21.15.89:0/3747174810 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8011d20 con 0x7f67f81338a0 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.976+0000 7f67fdd19700 1 -- 172.21.15.89:0/3747174810 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f81338a0 msgr2=0x7f67f8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.976+0000 7f67fdd19700 1 --2- 172.21.15.89:0/3747174810 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f81338a0 0x7f67f8133c70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f67e80098f0 tx=0x7f67e8009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.976+0000 7f67fdd19700 1 -- 172.21.15.89:0/3747174810 shutdown_connections 2023-09-30T22:23:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.976+0000 7f67fdd19700 1 --2- 172.21.15.89:0/3747174810 >> v2:172.21.15.89:3300/0 conn(0x7f67f809e930 0x7f67f813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.976+0000 7f67fdd19700 1 --2- 172.21.15.89:0/3747174810 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f81338a0 0x7f67f8133c70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.976+0000 7f67fdd19700 1 -- 172.21.15.89:0/3747174810 >> 172.21.15.89:0/3747174810 conn(0x7f67f80a2aa0 msgr2=0x7f67f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:16.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.976+0000 7f67fdd19700 1 -- 172.21.15.89:0/3747174810 shutdown_connections 2023-09-30T22:23:16.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.976+0000 7f67fdd19700 1 -- 172.21.15.89:0/3747174810 wait complete. 2023-09-30T22:23:16.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.977+0000 7f67fdd19700 1 Processor -- start 2023-09-30T22:23:16.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.977+0000 7f67fdd19700 1 -- start start 2023-09-30T22:23:16.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.977+0000 7f67fdd19700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67f809de50 0x7f67f811eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.977+0000 7f67fdd19700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f809e930 0x7f67f811f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.977+0000 7f67fdd19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f81338a0 0x7f67f811f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.977+0000 7f67fdd19700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f8121cf0 con 0x7f67f81338a0 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.977+0000 7f67fdd19700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f812f6a0 con 0x7f67f809e930 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.977+0000 7f67fdd19700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67f8120700 con 0x7f67f809de50 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67f809de50 0x7f67f811eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67f809de50 0x7f67f811eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48022/0 (socket says 172.21.15.89:48022) 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 -- 172.21.15.89:0/1300833125 learned_addr learned my addr 172.21.15.89:0/1300833125 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f7fff700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f81338a0 0x7f67f811f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f6ffd700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f809e930 0x7f67f811f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:16.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 -- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f809e930 msgr2=0x7f67f811f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f809e930 0x7f67f811f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 -- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f81338a0 msgr2=0x7f67f811f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f81338a0 0x7f67f811f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 -- 172.21.15.89:0/1300833125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e8009550 con 0x7f67f809de50 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f7fff700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f81338a0 0x7f67f811f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67f77fe700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67f809de50 0x7f67f811eaf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009c40 tx=0x7f67e8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67dffff700 1 -- 172.21.15.89:0/1300833125 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8004480 con 0x7f67f809de50 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67dffff700 1 -- 172.21.15.89:0/1300833125 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e801f070 con 0x7f67f809de50 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f80a10d0 con 0x7f67f809de50 2023-09-30T22:23:16.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.978+0000 7f67dffff700 1 -- 172.21.15.89:0/1300833125 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e80038c0 con 0x7f67f809de50 2023-09-30T22:23:16.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.979+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f8122070 con 0x7f67f809de50 2023-09-30T22:23:16.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.979+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f8074ea0 con 0x7f67f809de50 2023-09-30T22:23:16.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.980+0000 7f67dffff700 1 -- 172.21.15.89:0/1300833125 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67e8021030 con 0x7f67f809de50 2023-09-30T22:23:16.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.980+0000 7f67dffff700 1 -- 172.21.15.89:0/1300833125 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f67e8040a20 con 0x7f67f809de50 2023-09-30T22:23:16.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:16.985+0000 7f67dffff700 1 -- 172.21.15.89:0/1300833125 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67e80199a0 con 0x7f67f809de50 2023-09-30T22:23:17.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4f242700 1 Processor -- start 2023-09-30T22:23:17.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4f242700 1 -- start start 2023-09-30T22:23:17.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4f242700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809d6f0 0x7f2f4809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4f242700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809e410 0x7f2f48130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4f242700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4809f130 con 0x7f2f4809e000 2023-09-30T22:23:17.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4f242700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4809f520 con 0x7f2f4809e410 2023-09-30T22:23:17.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4f242700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4809e930 con 0x7f2f4809d6f0 2023-09-30T22:23:17.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4cfde700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809d6f0 0x7f2f4809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4cfde700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809d6f0 0x7f2f4809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48024/0 (socket says 172.21.15.89:48024) 2023-09-30T22:23:17.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4cfde700 1 -- 172.21.15.89:0/420897110 learned_addr learned my addr 172.21.15.89:0/420897110 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:17.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.021+0000 7f2f4d7df700 1 --2- 172.21.15.89:0/420897110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809e410 0x7f2f48130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.022+0000 7f2f4cfde700 1 -- 172.21.15.89:0/420897110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809e410 msgr2=0x7f2f48130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:17.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.022+0000 7f2f4cfde700 1 --2- 172.21.15.89:0/420897110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809e410 0x7f2f48130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.022+0000 7f2f4cfde700 1 -- 172.21.15.89:0/420897110 >> v1:172.21.15.89:6789/0 conn(0x7f2f4809e000 legacy=0x7f2f48130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.022+0000 7f2f4cfde700 1 -- 172.21.15.89:0/420897110 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4809b2d0 con 0x7f2f4809d6f0 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.022+0000 7f2f4cfde700 1 --2- 172.21.15.89:0/420897110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809d6f0 0x7f2f4809dac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00bbc0 tx=0x7f2f3c00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba3c2e48a1178853 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.022+0000 7f2f477fe700 1 -- 172.21.15.89:0/420897110 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00dcf0 con 0x7f2f4809d6f0 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.022+0000 7f2f477fe700 1 -- 172.21.15.89:0/420897110 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c00de50 con 0x7f2f4809d6f0 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.022+0000 7f2f477fe700 1 -- 172.21.15.89:0/420897110 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c004250 con 0x7f2f4809d6f0 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.023+0000 7f2f4f242700 1 -- 172.21.15.89:0/420897110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809d6f0 msgr2=0x7f2f4809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.023+0000 7f2f4f242700 1 --2- 172.21.15.89:0/420897110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809d6f0 0x7f2f4809dac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00bbc0 tx=0x7f2f3c00bed0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.023+0000 7f2f4f242700 1 -- 172.21.15.89:0/420897110 shutdown_connections 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.023+0000 7f2f4f242700 1 --2- 172.21.15.89:0/420897110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809e410 0x7f2f48130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.023+0000 7f2f4f242700 1 --2- 172.21.15.89:0/420897110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809d6f0 0x7f2f4809dac0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.023+0000 7f2f4f242700 1 -- 172.21.15.89:0/420897110 >> 172.21.15.89:0/420897110 conn(0x7f2f48066f50 msgr2=0x7f2f4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.023+0000 7f2f4f242700 1 -- 172.21.15.89:0/420897110 shutdown_connections 2023-09-30T22:23:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.023+0000 7f2f4f242700 1 -- 172.21.15.89:0/420897110 wait complete. 2023-09-30T22:23:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.024+0000 7f2f4f242700 1 Processor -- start 2023-09-30T22:23:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.024+0000 7f2f4f242700 1 -- start start 2023-09-30T22:23:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.024+0000 7f2f4f242700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809d6f0 0x7f2f4811e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.024+0000 7f2f4f242700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809e000 0x7f2f4811e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.024+0000 7f2f4f242700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4809e410 0x7f2f4811ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.024+0000 7f2f4f242700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4809c700 con 0x7f2f4809e410 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.024+0000 7f2f4f242700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48121d90 con 0x7f2f4809d6f0 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.024+0000 7f2f4f242700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48120550 con 0x7f2f4809e000 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809e000 0x7f2f4811e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f4cfde700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809d6f0 0x7f2f4811e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809e000 0x7f2f4811e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48040/0 (socket says 172.21.15.89:48040) 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 -- 172.21.15.89:0/1858925675 learned_addr learned my addr 172.21.15.89:0/1858925675 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f4d7df700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4809e410 0x7f2f4811ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 -- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809d6f0 msgr2=0x7f2f4811e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809d6f0 0x7f2f4811e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 -- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4809e410 msgr2=0x7f2f4811ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4809e410 0x7f2f4811ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 -- 172.21.15.89:0/1858925675 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c00b7f0 con 0x7f2f4809e000 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f4cfde700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809d6f0 0x7f2f4811e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f47fff700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809e000 0x7f2f4811e730 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400c4a0 tx=0x7f2f3400c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f457fa700 1 -- 172.21.15.89:0/1858925675 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34011790 con 0x7f2f4809e000 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f457fa700 1 -- 172.21.15.89:0/1858925675 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f340118f0 con 0x7f2f4809e000 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4812fca0 con 0x7f2f4809e000 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f457fa700 1 -- 172.21.15.89:0/1858925675 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34011ca0 con 0x7f2f4809e000 2023-09-30T22:23:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.025+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f480a1cc0 con 0x7f2f4809e000 2023-09-30T22:23:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.026+0000 7f2f457fa700 1 -- 172.21.15.89:0/1858925675 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f3401c460 con 0x7f2f4809e000 2023-09-30T22:23:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.026+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f48072380 con 0x7f2f4809e000 2023-09-30T22:23:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.027+0000 7f2f457fa700 1 -- 172.21.15.89:0/1858925675 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2f3403f400 con 0x7f2f4809e000 2023-09-30T22:23:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.027+0000 7fef879e6700 1 Processor -- start 2023-09-30T22:23:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef879e6700 1 -- start start 2023-09-30T22:23:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef879e6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fef80095690 0x7fef80134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef879e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8013fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef879e6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8009e800 con 0x7fef801378f0 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef879e6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef80095a60 con 0x7fef80095690 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef879e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef80094bd0 con 0x7fef80138430 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef85782700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fef80095690 0x7fef80134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef85782700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fef80095690 0x7fef80134cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46012/0 (socket says 172.21.15.89:46012) 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef85782700 1 -- 172.21.15.89:0/771090374 learned_addr learned my addr 172.21.15.89:0/771090374 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.028+0000 7fef84f81700 1 --1- 172.21.15.89:0/771090374 >> v1:172.21.15.89:6789/0 conn(0x7fef801378f0 0x7fef80137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48008/0 (socket says 172.21.15.89:48008) 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef85f83700 1 --2- 172.21.15.89:0/771090374 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8013fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef85f83700 1 -- 172.21.15.89:0/771090374 >> v2:172.21.15.89:3300/0 conn(0x7fef80095690 msgr2=0x7fef80134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef85f83700 1 --2- 172.21.15.89:0/771090374 >> v2:172.21.15.89:3300/0 conn(0x7fef80095690 0x7fef80134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef85f83700 1 -- 172.21.15.89:0/771090374 >> v1:172.21.15.89:6789/0 conn(0x7fef801378f0 legacy=0x7fef80137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef77fff700 1 -- 172.21.15.89:0/771090374 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400419442 0 0) 0x7fef8009e800 con 0x7fef801378f0 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef85f83700 1 -- 172.21.15.89:0/771090374 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800940e0 con 0x7fef80138430 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef85f83700 1 --2- 172.21.15.89:0/771090374 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8013fc30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fef78009d90 tx=0x7fef7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b994fa5ce1303e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef77fff700 1 -- 172.21.15.89:0/771090374 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7800e070 con 0x7fef80138430 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef77fff700 1 -- 172.21.15.89:0/771090374 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7800bbd0 con 0x7fef80138430 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.029+0000 7fef77fff700 1 -- 172.21.15.89:0/771090374 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78012430 con 0x7fef80138430 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.030+0000 7fef879e6700 1 -- 172.21.15.89:0/771090374 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 msgr2=0x7fef8013fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.030+0000 7fef879e6700 1 --2- 172.21.15.89:0/771090374 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8013fc30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fef78009d90 tx=0x7fef7800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.031+0000 7fef879e6700 1 -- 172.21.15.89:0/771090374 shutdown_connections 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.031+0000 7fef879e6700 1 --2- 172.21.15.89:0/771090374 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8013fc30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.031+0000 7fef879e6700 1 --2- 172.21.15.89:0/771090374 >> v2:172.21.15.89:3300/0 conn(0x7fef80095690 0x7fef80134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.031+0000 7fef879e6700 1 -- 172.21.15.89:0/771090374 >> 172.21.15.89:0/771090374 conn(0x7fef800a2360 msgr2=0x7fef800a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.031+0000 7fef879e6700 1 -- 172.21.15.89:0/771090374 shutdown_connections 2023-09-30T22:23:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.032+0000 7f2f457fa700 1 -- 172.21.15.89:0/1858925675 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f34011e00 con 0x7f2f4809e000 2023-09-30T22:23:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.031+0000 7fef879e6700 1 -- 172.21.15.89:0/771090374 wait complete. 2023-09-30T22:23:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.032+0000 7fef879e6700 1 Processor -- start 2023-09-30T22:23:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.032+0000 7fef879e6700 1 -- start start 2023-09-30T22:23:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef879e6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef80095690 0x7fef80071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef879e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef801378f0 0x7fef800715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef879e6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef879e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef80070920 con 0x7fef801378f0 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef879e6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef800a0740 con 0x7fef80138430 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef879e6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef80138af0 con 0x7fef80095690 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef84f81700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef801378f0 0x7fef800715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef84f81700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef801378f0 0x7fef800715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46028/0 (socket says 172.21.15.89:46028) 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef84f81700 1 -- 172.21.15.89:0/2621954501 learned_addr learned my addr 172.21.15.89:0/2621954501 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef85782700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef80095690 0x7fef80071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.033+0000 7fef85f83700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef84f81700 1 -- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef80095690 msgr2=0x7fef80071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef84f81700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef80095690 0x7fef80071060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef84f81700 1 -- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 msgr2=0x7fef8006eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef84f81700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8006eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef84f81700 1 -- 172.21.15.89:0/2621954501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78009a70 con 0x7fef801378f0 2023-09-30T22:23:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef85f83700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef84f81700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef801378f0 0x7fef800715a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c550 tx=0x7fef7000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef75ffb700 1 -- 172.21.15.89:0/2621954501 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70004c80 con 0x7fef801378f0 2023-09-30T22:23:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef75ffb700 1 -- 172.21.15.89:0/2621954501 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70004e00 con 0x7fef801378f0 2023-09-30T22:23:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.034+0000 7fef75ffb700 1 -- 172.21.15.89:0/2621954501 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70007310 con 0x7fef801378f0 2023-09-30T22:23:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.035+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef800946d0 con 0x7fef801378f0 2023-09-30T22:23:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.035+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef80072300 con 0x7fef801378f0 2023-09-30T22:23:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.036+0000 7fef75ffb700 1 -- 172.21.15.89:0/2621954501 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef70007470 con 0x7fef801378f0 2023-09-30T22:23:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.037+0000 7fef75ffb700 1 -- 172.21.15.89:0/2621954501 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef7003f270 con 0x7fef801378f0 2023-09-30T22:23:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.037+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef80074ea0 con 0x7fef801378f0 2023-09-30T22:23:17.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.046+0000 7fef75ffb700 1 -- 172.21.15.89:0/2621954501 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef700123b0 con 0x7fef801378f0 2023-09-30T22:23:17.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.132+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7560074ea0 con 0x7f75601305f0 2023-09-30T22:23:17.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.132+0000 7f75567fc700 1 -- 172.21.15.89:0/1105360292 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75580192c0 con 0x7f75601305f0 2023-09-30T22:23:17.132 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75601305f0 msgr2=0x7f756011c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75601305f0 0x7f756011c300 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f755800cbc0 tx=0x7f755800ced0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 shutdown_connections 2023-09-30T22:23:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f75601305f0 0x7f756011c300 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f756006f660 0x7f756011b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 --2- 172.21.15.89:0/1105360292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f756006ff90 0x7f756011bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 >> 172.21.15.89:0/1105360292 conn(0x7f75600a2aa0 msgr2=0x7f756009f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 shutdown_connections 2023-09-30T22:23:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.135+0000 7f7568121700 1 -- 172.21.15.89:0/1105360292 wait complete. 2023-09-30T22:23:17.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:17.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.193+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f67f806b9e0 con 0x7f67f809de50 2023-09-30T22:23:17.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.194+0000 7f67dffff700 1 -- 172.21.15.89:0/1300833125 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67e8015030 con 0x7f67f809de50 2023-09-30T22:23:17.194 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67f809de50 msgr2=0x7f67f811eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67f809de50 0x7f67f811eaf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009c40 tx=0x7f67e8004750 comp rx=0 tx=0).stop 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 shutdown_connections 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f81338a0 0x7f67f811f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67f809de50 0x7f67f811eaf0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 --2- 172.21.15.89:0/1300833125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67f809e930 0x7f67f811f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 >> 172.21.15.89:0/1300833125 conn(0x7f67f80a2aa0 msgr2=0x7f67f809ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 shutdown_connections 2023-09-30T22:23:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.196+0000 7f67fdd19700 1 -- 172.21.15.89:0/1300833125 wait complete. 2023-09-30T22:23:17.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:17.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.225+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2f48074ea0 con 0x7f2f4809e000 2023-09-30T22:23:17.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.226+0000 7f2f457fa700 1 -- 172.21.15.89:0/1858925675 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f340102e0 con 0x7f2f4809e000 2023-09-30T22:23:17.226 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:17.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809e000 msgr2=0x7f2f4811e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809e000 0x7f2f4811e730 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400c4a0 tx=0x7f2f3400c7b0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 shutdown_connections 2023-09-30T22:23:17.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f4809e410 0x7f2f4811ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2f4809d6f0 0x7f2f4811e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 --2- 172.21.15.89:0/1858925675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f4809e000 0x7f2f4811e730 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 >> 172.21.15.89:0/1858925675 conn(0x7f2f48066f50 msgr2=0x7f2f480a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:17.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 shutdown_connections 2023-09-30T22:23:17.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.229+0000 7f2f4f242700 1 -- 172.21.15.89:0/1858925675 wait complete. 2023-09-30T22:23:17.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.248+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fef8006b700 con 0x7fef801378f0 2023-09-30T22:23:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.248+0000 7fef75ffb700 1 -- 172.21.15.89:0/2621954501 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef70011360 con 0x7fef801378f0 2023-09-30T22:23:17.248 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:17.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef801378f0 msgr2=0x7fef800715a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:17.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef801378f0 0x7fef800715a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c550 tx=0x7fef7000c860 comp rx=0 tx=0).stop 2023-09-30T22:23:17.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 shutdown_connections 2023-09-30T22:23:17.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fef80138430 0x7fef8006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fef80095690 0x7fef80071060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 --2- 172.21.15.89:0/2621954501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef801378f0 0x7fef800715a0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:17.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 >> 172.21.15.89:0/2621954501 conn(0x7fef800a2360 msgr2=0x7fef8009b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:17.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 shutdown_connections 2023-09-30T22:23:17.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:17.250+0000 7fef879e6700 1 -- 172.21.15.89:0/2621954501 wait complete. 2023-09-30T22:23:17.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:18.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:18.208 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:18.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:18.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:18.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.295+0000 7feba37aa700 1 Processor -- start 2023-09-30T22:23:18.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.295+0000 7feba37aa700 1 -- start start 2023-09-30T22:23:18.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba37aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba37aa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feb9c06b080 0x7feb9c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba37aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c131b40 0x7feb9c131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba37aa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9c09ece0 con 0x7feb9c06b080 2023-09-30T22:23:18.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba37aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c09e760 con 0x7feb9c131b40 2023-09-30T22:23:18.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba37aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c09e040 con 0x7feb9c06a770 2023-09-30T22:23:18.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1546700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1546700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48058/0 (socket says 172.21.15.89:48058) 2023-09-30T22:23:18.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1546700 1 -- 172.21.15.89:0/439170400 learned_addr learned my addr 172.21.15.89:0/439170400 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:18.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1d47700 1 --2- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c131b40 0x7feb9c131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba0d45700 1 --2- 172.21.15.89:0/439170400 >> v2:172.21.15.89:3300/0 conn(0x7feb9c06b080 0x7feb9c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1546700 1 -- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c131b40 msgr2=0x7feb9c131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:18.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1546700 1 --2- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c131b40 0x7feb9c131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1546700 1 -- 172.21.15.89:0/439170400 >> v2:172.21.15.89:3300/0 conn(0x7feb9c06b080 msgr2=0x7feb9c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1546700 1 --2- 172.21.15.89:0/439170400 >> v2:172.21.15.89:3300/0 conn(0x7feb9c06b080 0x7feb9c131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1546700 1 -- 172.21.15.89:0/439170400 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c09f630 con 0x7feb9c06a770 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.296+0000 7feba1d47700 1 --2- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c131b40 0x7feb9c131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.297+0000 7feba1546700 1 --2- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c06ab40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7feb980098f0 tx=0x7feb98009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=359b3b21ee22381d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.297+0000 7feb93fff700 1 -- 172.21.15.89:0/439170400 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98010a90 con 0x7feb9c06a770 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.297+0000 7feb93fff700 1 -- 172.21.15.89:0/439170400 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb98010bf0 con 0x7feb9c06a770 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.297+0000 7feb93fff700 1 -- 172.21.15.89:0/439170400 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98018440 con 0x7feb9c06a770 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.297+0000 7feba37aa700 1 -- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 msgr2=0x7feb9c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.297+0000 7feba37aa700 1 --2- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c06ab40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7feb980098f0 tx=0x7feb98009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.298+0000 7feba37aa700 1 -- 172.21.15.89:0/439170400 shutdown_connections 2023-09-30T22:23:18.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.298+0000 7feba37aa700 1 --2- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c131b40 0x7feb9c131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.298+0000 7feba37aa700 1 --2- 172.21.15.89:0/439170400 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c06ab40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.298+0000 7feba37aa700 1 --2- 172.21.15.89:0/439170400 >> v2:172.21.15.89:3300/0 conn(0x7feb9c06b080 0x7feb9c131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.298+0000 7feba37aa700 1 -- 172.21.15.89:0/439170400 >> 172.21.15.89:0/439170400 conn(0x7feb9c066f50 msgr2=0x7feb9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:18.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.298+0000 7feba37aa700 1 -- 172.21.15.89:0/439170400 shutdown_connections 2023-09-30T22:23:18.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.298+0000 7feba37aa700 1 -- 172.21.15.89:0/439170400 wait complete. 2023-09-30T22:23:18.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.299+0000 7feba37aa700 1 Processor -- start 2023-09-30T22:23:18.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.299+0000 7feba37aa700 1 -- start start 2023-09-30T22:23:18.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.299+0000 7feba37aa700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c126aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.299+0000 7feba37aa700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c06b080 0x7feb9c126fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.299+0000 7feba37aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb9c131b40 0x7feb9c1cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.299+0000 7feba37aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c129980 con 0x7feb9c131b40 2023-09-30T22:23:18.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.299+0000 7feba37aa700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c129b80 con 0x7feb9c06b080 2023-09-30T22:23:18.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.299+0000 7feba37aa700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c127c70 con 0x7feb9c06a770 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba1d47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb9c131b40 0x7feb9c1cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba1546700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c126aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba1546700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c126aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48064/0 (socket says 172.21.15.89:48064) 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba1d47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb9c131b40 0x7feb9c1cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46046/0 (socket says 172.21.15.89:46046) 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba1546700 1 -- 172.21.15.89:0/170319026 learned_addr learned my addr 172.21.15.89:0/170319026 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba0d45700 1 --2- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c06b080 0x7feb9c126fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba0d45700 1 -- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 msgr2=0x7feb9c126aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba0d45700 1 --2- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c126aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba0d45700 1 -- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb9c131b40 msgr2=0x7feb9c1cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba0d45700 1 --2- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb9c131b40 0x7feb9c1cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba0d45700 1 -- 172.21.15.89:0/170319026 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb98009550 con 0x7feb9c06b080 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.300+0000 7feba0d45700 1 --2- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c06b080 0x7feb9c126fe0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e900 tx=0x7feb8c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.301+0000 7feb91ffb700 1 -- 172.21.15.89:0/170319026 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c013680 con 0x7feb9c06b080 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.301+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9c071e60 con 0x7feb9c06b080 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.301+0000 7feb91ffb700 1 -- 172.21.15.89:0/170319026 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8c0137e0 con 0x7feb9c06b080 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.301+0000 7feb91ffb700 1 -- 172.21.15.89:0/170319026 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c013b50 con 0x7feb9c06b080 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.301+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9c12a310 con 0x7feb9c06b080 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.302+0000 7feb91ffb700 1 -- 172.21.15.89:0/170319026 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb8c013cb0 con 0x7feb9c06b080 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.302+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9c0728e0 con 0x7feb9c06b080 2023-09-30T22:23:18.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.302+0000 7feb91ffb700 1 -- 172.21.15.89:0/170319026 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feb8c011e80 con 0x7feb9c06b080 2023-09-30T22:23:18.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.311+0000 7feb91ffb700 1 -- 172.21.15.89:0/170319026 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb8c017030 con 0x7feb9c06b080 2023-09-30T22:23:18.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.347+0000 7fcfd9551700 1 Processor -- start 2023-09-30T22:23:18.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.347+0000 7fcfd9551700 1 -- start start 2023-09-30T22:23:18.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.347+0000 7fcfd9551700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd41338a0 0x7fcfd4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.347+0000 7fcfd9551700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcfd409de50 0x7fcfd409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.347+0000 7fcfd9551700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd409e760 0x7fcfd413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.347+0000 7fcfd9551700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd406fef0 con 0x7fcfd409de50 2023-09-30T22:23:18.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.347+0000 7fcfd9551700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd406ef30 con 0x7fcfd41338a0 2023-09-30T22:23:18.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.347+0000 7fcfd9551700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41334b0 con 0x7fcfd409e760 2023-09-30T22:23:18.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd409e760 0x7fcfd413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd409e760 0x7fcfd413a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48078/0 (socket says 172.21.15.89:48078) 2023-09-30T22:23:18.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd37fe700 1 -- 172.21.15.89:0/1102124701 learned_addr learned my addr 172.21.15.89:0/1102124701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd2ffd700 1 --2- 172.21.15.89:0/1102124701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd41338a0 0x7fcfd4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd27fc700 1 --2- 172.21.15.89:0/1102124701 >> v2:172.21.15.89:3300/0 conn(0x7fcfd409de50 0x7fcfd409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd2ffd700 1 -- 172.21.15.89:0/1102124701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd409e760 msgr2=0x7fcfd413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd2ffd700 1 --2- 172.21.15.89:0/1102124701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd409e760 0x7fcfd413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd2ffd700 1 -- 172.21.15.89:0/1102124701 >> v2:172.21.15.89:3300/0 conn(0x7fcfd409de50 msgr2=0x7fcfd409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd2ffd700 1 --2- 172.21.15.89:0/1102124701 >> v2:172.21.15.89:3300/0 conn(0x7fcfd409de50 0x7fcfd409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.348+0000 7fcfd2ffd700 1 -- 172.21.15.89:0/1102124701 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd406fc30 con 0x7fcfd41338a0 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.349+0000 7fcfd2ffd700 1 --2- 172.21.15.89:0/1102124701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd41338a0 0x7fcfd4133c70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009840 tx=0x7fcfc4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4877f3fa5486c532 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.349+0000 7fcfd1ffb700 1 -- 172.21.15.89:0/1102124701 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4011830 con 0x7fcfd41338a0 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.349+0000 7fcfd1ffb700 1 -- 172.21.15.89:0/1102124701 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc4011990 con 0x7fcfd41338a0 2023-09-30T22:23:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.349+0000 7fcfd1ffb700 1 -- 172.21.15.89:0/1102124701 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4011d20 con 0x7fcfd41338a0 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 -- 172.21.15.89:0/1102124701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd41338a0 msgr2=0x7fcfd4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 --2- 172.21.15.89:0/1102124701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd41338a0 0x7fcfd4133c70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009840 tx=0x7fcfc4009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 -- 172.21.15.89:0/1102124701 shutdown_connections 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 --2- 172.21.15.89:0/1102124701 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd409e760 0x7fcfd413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 --2- 172.21.15.89:0/1102124701 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd41338a0 0x7fcfd4133c70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 --2- 172.21.15.89:0/1102124701 >> v2:172.21.15.89:3300/0 conn(0x7fcfd409de50 0x7fcfd409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 -- 172.21.15.89:0/1102124701 >> 172.21.15.89:0/1102124701 conn(0x7fcfd40a2aa0 msgr2=0x7fcfd40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 -- 172.21.15.89:0/1102124701 shutdown_connections 2023-09-30T22:23:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.350+0000 7fcfd9551700 1 -- 172.21.15.89:0/1102124701 wait complete. 2023-09-30T22:23:18.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.352+0000 7fcfd9551700 1 Processor -- start 2023-09-30T22:23:18.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.352+0000 7fcfd9551700 1 -- start start 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.352+0000 7fcfd9551700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd409de50 0x7fcfd411b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.352+0000 7fcfd9551700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd409e760 0x7fcfd411bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.352+0000 7fcfd9551700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd41338a0 0x7fcfd411c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.352+0000 7fcfd9551700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd411ef80 con 0x7fcfd409e760 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.352+0000 7fcfd9551700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd411d2f0 con 0x7fcfd409de50 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.352+0000 7fcfd9551700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd411f1d0 con 0x7fcfd41338a0 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd2ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd409de50 0x7fcfd411b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd41338a0 0x7fcfd411c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd2ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd409de50 0x7fcfd411b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37486/0 (socket says 172.21.15.89:37486) 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd2ffd700 1 -- 172.21.15.89:0/1141159279 learned_addr learned my addr 172.21.15.89:0/1141159279 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd27fc700 1 --2- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd409e760 0x7fcfd411bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd37fe700 1 -- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd409de50 msgr2=0x7fcfd411b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd37fe700 1 --2- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd409de50 0x7fcfd411b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd37fe700 1 -- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd409e760 msgr2=0x7fcfd411bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd37fe700 1 --2- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd409e760 0x7fcfd411bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd37fe700 1 -- 172.21.15.89:0/1141159279 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40094a0 con 0x7fcfd41338a0 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.353+0000 7fcfd37fe700 1 --2- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd41338a0 0x7fcfd411c1b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc007a70 tx=0x7fcfcc007d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.354+0000 7fcfbb7fe700 1 -- 172.21.15.89:0/1141159279 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00d9d0 con 0x7fcfd41338a0 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.354+0000 7fcfbb7fe700 1 -- 172.21.15.89:0/1141159279 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00f040 con 0x7fcfd41338a0 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.354+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd40a0f70 con 0x7fcfd41338a0 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.354+0000 7fcfbb7fe700 1 -- 172.21.15.89:0/1141159279 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00dd70 con 0x7fcfd41338a0 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.354+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd411da00 con 0x7fcfd41338a0 2023-09-30T22:23:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.355+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd411ede0 con 0x7fcfd41338a0 2023-09-30T22:23:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.355+0000 7fcfbb7fe700 1 -- 172.21.15.89:0/1141159279 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfcc01e3b0 con 0x7fcfd41338a0 2023-09-30T22:23:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.356+0000 7fcfbb7fe700 1 -- 172.21.15.89:0/1141159279 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcfcc03f480 con 0x7fcfd41338a0 2023-09-30T22:23:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.364+0000 7fcfbb7fe700 1 -- 172.21.15.89:0/1141159279 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcfd411ede0 con 0x7fcfd41338a0 2023-09-30T22:23:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.389+0000 7f560713b700 1 Processor -- start 2023-09-30T22:23:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.389+0000 7f560713b700 1 -- start start 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f560713b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f560009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f560713b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f560009e000 0x7f5600130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f560713b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f5600130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f560713b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560009f520 con 0x7f560009e000 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f560713b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560009f130 con 0x7f560009d6f0 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f560713b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f560009e930 con 0x7f560009e410 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f56056d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f5600130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f56056d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f5600130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48086/0 (socket says 172.21.15.89:48086) 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f56056d8700 1 -- 172.21.15.89:0/1011251087 learned_addr learned my addr 172.21.15.89:0/1011251087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:18.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f5604ed7700 1 --2- 172.21.15.89:0/1011251087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f560009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.390+0000 7f55fffff700 1 --2- 172.21.15.89:0/1011251087 >> v2:172.21.15.89:3300/0 conn(0x7f560009e000 0x7f5600130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f56056d8700 1 -- 172.21.15.89:0/1011251087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 msgr2=0x7f560009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f56056d8700 1 --2- 172.21.15.89:0/1011251087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f560009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f56056d8700 1 -- 172.21.15.89:0/1011251087 >> v2:172.21.15.89:3300/0 conn(0x7f560009e000 msgr2=0x7f5600130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f56056d8700 1 --2- 172.21.15.89:0/1011251087 >> v2:172.21.15.89:3300/0 conn(0x7f560009e000 0x7f5600130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f56056d8700 1 -- 172.21.15.89:0/1011251087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560009b2d0 con 0x7f560009e410 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f56056d8700 1 --2- 172.21.15.89:0/1011251087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f5600130ba0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f55f800d180 tx=0x7f55f800d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27a4c0b2c466071 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f55ff7fe700 1 -- 172.21.15.89:0/1011251087 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8010070 con 0x7f560009e410 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f55ff7fe700 1 -- 172.21.15.89:0/1011251087 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f800ddb0 con 0x7f560009e410 2023-09-30T22:23:18.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.391+0000 7f55ff7fe700 1 -- 172.21.15.89:0/1011251087 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800b310 con 0x7f560009e410 2023-09-30T22:23:18.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 -- 172.21.15.89:0/1011251087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 msgr2=0x7f5600130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 --2- 172.21.15.89:0/1011251087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f5600130ba0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f55f800d180 tx=0x7f55f800d490 comp rx=0 tx=0).stop 2023-09-30T22:23:18.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 -- 172.21.15.89:0/1011251087 shutdown_connections 2023-09-30T22:23:18.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 --2- 172.21.15.89:0/1011251087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f5600130ba0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 --2- 172.21.15.89:0/1011251087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f560009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 --2- 172.21.15.89:0/1011251087 >> v2:172.21.15.89:3300/0 conn(0x7f560009e000 0x7f5600130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 -- 172.21.15.89:0/1011251087 >> 172.21.15.89:0/1011251087 conn(0x7f5600066f50 msgr2=0x7f560009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:18.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 -- 172.21.15.89:0/1011251087 shutdown_connections 2023-09-30T22:23:18.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.392+0000 7f560713b700 1 -- 172.21.15.89:0/1011251087 wait complete. 2023-09-30T22:23:18.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.393+0000 7f560713b700 1 Processor -- start 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.393+0000 7f560713b700 1 -- start start 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f560713b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f5600127df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f560713b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560009e000 0x7f5600128330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f560713b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f56001cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f560713b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56001279d0 con 0x7f560009e000 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f560713b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5600129640 con 0x7f560009d6f0 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f560713b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5600128870 con 0x7f560009e410 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f5604ed7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f5600127df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f5604ed7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f5600127df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37504/0 (socket says 172.21.15.89:37504) 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f5604ed7700 1 -- 172.21.15.89:0/757506569 learned_addr learned my addr 172.21.15.89:0/757506569 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:18.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.394+0000 7f56056d8700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f56001cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f55fffff700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560009e000 0x7f5600128330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f56056d8700 1 -- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 msgr2=0x7f5600127df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f56056d8700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f5600127df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f56056d8700 1 -- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560009e000 msgr2=0x7f5600128330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f56056d8700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560009e000 0x7f5600128330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f56056d8700 1 -- 172.21.15.89:0/757506569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f800bd40 con 0x7f560009e410 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f5604ed7700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f5600127df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f56056d8700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f56001cc630 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f56000a3e50 tx=0x7f55f8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f55fd7fa700 1 -- 172.21.15.89:0/757506569 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8009a10 con 0x7f560009e410 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560012b560 con 0x7f560009e410 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560012add0 con 0x7f560009e410 2023-09-30T22:23:18.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f55fd7fa700 1 -- 172.21.15.89:0/757506569 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f8009b70 con 0x7f560009e410 2023-09-30T22:23:18.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.395+0000 7f55fd7fa700 1 -- 172.21.15.89:0/757506569 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8003e10 con 0x7f560009e410 2023-09-30T22:23:18.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.396+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e00052e0 con 0x7f560009e410 2023-09-30T22:23:18.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.397+0000 7f55fd7fa700 1 -- 172.21.15.89:0/757506569 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f55f8004040 con 0x7f560009e410 2023-09-30T22:23:18.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.397+0000 7f55fd7fa700 1 -- 172.21.15.89:0/757506569 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f55f80410c0 con 0x7f560009e410 2023-09-30T22:23:18.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.406+0000 7f55fd7fa700 1 -- 172.21.15.89:0/757506569 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55f8013980 con 0x7f560009e410 2023-09-30T22:23:18.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f494ad55700 1 Processor -- start 2023-09-30T22:23:18.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f494ad55700 1 -- start start 2023-09-30T22:23:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f494ad55700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f494406e570 0x7f494406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f494ad55700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494409eca0 0x7f494406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f494ad55700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4944134610 0x7f4944134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f494ad55700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4944070ee0 con 0x7f4944134610 2023-09-30T22:23:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f494ad55700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944071480 con 0x7f494409eca0 2023-09-30T22:23:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f494ad55700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f494406f3c0 con 0x7f494406e570 2023-09-30T22:23:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f49492f2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4944134610 0x7f4944134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f4948af1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f494406e570 0x7f494406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f49492f2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4944134610 0x7f4944134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46070/0 (socket says 172.21.15.89:46070) 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.432+0000 7f49492f2700 1 -- 172.21.15.89:0/739168476 learned_addr learned my addr 172.21.15.89:0/739168476 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f4943fff700 1 --2- 172.21.15.89:0/739168476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494409eca0 0x7f494406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f4943fff700 1 -- 172.21.15.89:0/739168476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f494406e570 msgr2=0x7f494406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f4943fff700 1 --2- 172.21.15.89:0/739168476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f494406e570 0x7f494406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f4943fff700 1 -- 172.21.15.89:0/739168476 >> v2:172.21.15.89:3300/0 conn(0x7f4944134610 msgr2=0x7f4944134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f4943fff700 1 --2- 172.21.15.89:0/739168476 >> v2:172.21.15.89:3300/0 conn(0x7f4944134610 0x7f4944134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f4943fff700 1 -- 172.21.15.89:0/739168476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944071b90 con 0x7f494409eca0 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f49492f2700 1 --2- 172.21.15.89:0/739168476 >> v2:172.21.15.89:3300/0 conn(0x7f4944134610 0x7f4944134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f4943fff700 1 --2- 172.21.15.89:0/739168476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494409eca0 0x7f494406ee80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4930009840 tx=0x7f4930009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e4a9962ce17080 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:18.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f49437fe700 1 -- 172.21.15.89:0/739168476 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930011830 con 0x7f494409eca0 2023-09-30T22:23:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f49437fe700 1 -- 172.21.15.89:0/739168476 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4930011990 con 0x7f494409eca0 2023-09-30T22:23:18.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.433+0000 7f49437fe700 1 -- 172.21.15.89:0/739168476 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930011d20 con 0x7f494409eca0 2023-09-30T22:23:18.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 -- 172.21.15.89:0/739168476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494409eca0 msgr2=0x7f494406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 --2- 172.21.15.89:0/739168476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494409eca0 0x7f494406ee80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4930009840 tx=0x7f4930009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 -- 172.21.15.89:0/739168476 shutdown_connections 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 --2- 172.21.15.89:0/739168476 >> v2:172.21.15.89:3300/0 conn(0x7f4944134610 0x7f4944134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 --2- 172.21.15.89:0/739168476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f494406e570 0x7f494406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 --2- 172.21.15.89:0/739168476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494409eca0 0x7f494406ee80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 -- 172.21.15.89:0/739168476 >> 172.21.15.89:0/739168476 conn(0x7f49440a2aa0 msgr2=0x7f49440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 -- 172.21.15.89:0/739168476 shutdown_connections 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.434+0000 7f494ad55700 1 -- 172.21.15.89:0/739168476 wait complete. 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.435+0000 7f494ad55700 1 Processor -- start 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.435+0000 7f494ad55700 1 -- start start 2023-09-30T22:23:18.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f494ad55700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494406e570 0x7f49441214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f494ad55700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4944134610 0x7f4944121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f494ad55700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f494409e4c0 0x7f4944121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f494ad55700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944122f50 con 0x7f494409e4c0 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f494ad55700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944124010 con 0x7f494406e570 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f494ad55700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4944070610 con 0x7f4944134610 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494406e570 0x7f49441214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494406e570 0x7f49441214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37518/0 (socket says 172.21.15.89:37518) 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 -- 172.21.15.89:0/313056247 learned_addr learned my addr 172.21.15.89:0/313056247 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4943fff700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4944134610 0x7f4944121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f49492f2700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f494409e4c0 0x7f4944121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:18.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 -- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4944134610 msgr2=0x7f4944121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4944134610 0x7f4944121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 -- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f494409e4c0 msgr2=0x7f4944121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f494409e4c0 0x7f4944121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 -- 172.21.15.89:0/313056247 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c009b30 con 0x7f494406e570 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f4948af1700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494406e570 0x7f49441214e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f493800e9b0 tx=0x7f493800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f49417fa700 1 -- 172.21.15.89:0/313056247 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938009710 con 0x7f494406e570 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f49417fa700 1 -- 172.21.15.89:0/313056247 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493800ee50 con 0x7f494406e570 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.436+0000 7f49417fa700 1 -- 172.21.15.89:0/313056247 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493800ca50 con 0x7f494406e570 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.437+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300094a0 con 0x7f494406e570 2023-09-30T22:23:18.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.437+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4944122970 con 0x7f494406e570 2023-09-30T22:23:18.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.437+0000 7f49417fa700 1 -- 172.21.15.89:0/313056247 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f493800cbb0 con 0x7f494406e570 2023-09-30T22:23:18.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.438+0000 7f49417fa700 1 -- 172.21.15.89:0/313056247 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f493803f750 con 0x7f494406e570 2023-09-30T22:23:18.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.438+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494409b1e0 con 0x7f494406e570 2023-09-30T22:23:18.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.442+0000 7f49417fa700 1 -- 172.21.15.89:0/313056247 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4938021030 con 0x7f494406e570 2023-09-30T22:23:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.544+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feb9c074ea0 con 0x7feb9c06b080 2023-09-30T22:23:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.545+0000 7feb91ffb700 1 -- 172.21.15.89:0/170319026 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb8c024070 con 0x7feb9c06b080 2023-09-30T22:23:18.545 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c06b080 msgr2=0x7feb9c126fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 --2- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c06b080 0x7feb9c126fe0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e900 tx=0x7feb8c00ec10 comp rx=0 tx=0).stop 2023-09-30T22:23:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 shutdown_connections 2023-09-30T22:23:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 --2- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb9c131b40 0x7feb9c1cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 --2- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7feb9c06a770 0x7feb9c126aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 --2- 172.21.15.89:0/170319026 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feb9c06b080 0x7feb9c126fe0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 >> 172.21.15.89:0/170319026 conn(0x7feb9c066f50 msgr2=0x7feb9c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:18.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 shutdown_connections 2023-09-30T22:23:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.548+0000 7feba37aa700 1 -- 172.21.15.89:0/170319026 wait complete. 2023-09-30T22:23:18.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.567+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcfd411ede0 con 0x7fcfd41338a0 2023-09-30T22:23:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.568+0000 7fcfbb7fe700 1 -- 172.21.15.89:0/1141159279 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcfd411ede0 con 0x7fcfd41338a0 2023-09-30T22:23:18.567 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd41338a0 msgr2=0x7fcfd411c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 --2- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd41338a0 0x7fcfd411c1b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc007a70 tx=0x7fcfcc007d80 comp rx=0 tx=0).stop 2023-09-30T22:23:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 shutdown_connections 2023-09-30T22:23:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 --2- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcfd41338a0 0x7fcfd411c1b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 --2- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcfd409de50 0x7fcfd411b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 --2- 172.21.15.89:0/1141159279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd409e760 0x7fcfd411bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 >> 172.21.15.89:0/1141159279 conn(0x7fcfd40a2aa0 msgr2=0x7fcfd41306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 shutdown_connections 2023-09-30T22:23:18.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.570+0000 7fcfd9551700 1 -- 172.21.15.89:0/1141159279 wait complete. 2023-09-30T22:23:18.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:18.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.633+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f55e0005150 con 0x7f560009e410 2023-09-30T22:23:18.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.633+0000 7f55fd7fa700 1 -- 172.21.15.89:0/757506569 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55f8035090 con 0x7f560009e410 2023-09-30T22:23:18.633 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:18.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 msgr2=0x7f56001cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f56001cc630 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f56000a3e50 tx=0x7f55f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 shutdown_connections 2023-09-30T22:23:18.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f560009e410 0x7f56001cc630 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f560009d6f0 0x7f5600127df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 --2- 172.21.15.89:0/757506569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f560009e000 0x7f5600128330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 >> 172.21.15.89:0/757506569 conn(0x7f5600066f50 msgr2=0x7f5600070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:18.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 shutdown_connections 2023-09-30T22:23:18.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.635+0000 7f560713b700 1 -- 172.21.15.89:0/757506569 wait complete. 2023-09-30T22:23:18.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.637+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4944074ea0 con 0x7f494406e570 2023-09-30T22:23:18.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.637+0000 7f49417fa700 1 -- 172.21.15.89:0/313056247 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f493801be10 con 0x7f494406e570 2023-09-30T22:23:18.637 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:18.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494406e570 msgr2=0x7f49441214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:18.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494406e570 0x7f49441214e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f493800e9b0 tx=0x7f493800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 shutdown_connections 2023-09-30T22:23:18.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f494409e4c0 0x7f4944121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f494406e570 0x7f49441214e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 --2- 172.21.15.89:0/313056247 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4944134610 0x7f4944121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:18.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 >> 172.21.15.89:0/313056247 conn(0x7f49440a2aa0 msgr2=0x7f49441bf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:18.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 shutdown_connections 2023-09-30T22:23:18.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:18.639+0000 7f494ad55700 1 -- 172.21.15.89:0/313056247 wait complete. 2023-09-30T22:23:18.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:18.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:19.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:19.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:19.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:19.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.702+0000 7fd734ca2700 1 Processor -- start 2023-09-30T22:23:19.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.702+0000 7fd734ca2700 1 -- start start 2023-09-30T22:23:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.702+0000 7fd734ca2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133420 0x7fd7301337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.702+0000 7fd734ca2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133e30 0x7fd73012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.702+0000 7fd734ca2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73006b990 con 0x7fd730134260 2023-09-30T22:23:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.702+0000 7fd734ca2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300a0590 con 0x7fd730133e30 2023-09-30T22:23:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.702+0000 7fd734ca2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd73009fb80 con 0x7fd730133420 2023-09-30T22:23:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72e59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133420 0x7fd7301337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd730134260 0x7fd7301304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48022/0 (socket says 172.21.15.89:48022) 2023-09-30T22:23:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72ed9d700 1 -- 172.21.15.89:0/3665359692 learned_addr learned my addr 172.21.15.89:0/3665359692 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72dd9b700 1 --2- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133e30 0x7fd73012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72d59a700 1 -- 172.21.15.89:0/3665359692 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746090925 0 0) 0x7fd73006b990 con 0x7fd730134260 2023-09-30T22:23:19.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72d59a700 1 -- 172.21.15.89:0/3665359692 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7180035b0 con 0x7fd730134260 2023-09-30T22:23:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72e59c700 1 -- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133e30 msgr2=0x7fd73012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72e59c700 1 --2- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133e30 0x7fd73012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72e59c700 1 -- 172.21.15.89:0/3665359692 >> v1:172.21.15.89:6789/0 conn(0x7fd730134260 legacy=0x7fd7301304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72e59c700 1 -- 172.21.15.89:0/3665359692 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7300a0030 con 0x7fd730133420 2023-09-30T22:23:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72dd9b700 1 --2- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133e30 0x7fd73012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.703+0000 7fd72e59c700 1 --2- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133420 0x7fd7301337f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd720009870 tx=0x7fd720009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471fbb83cc21ce10 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:19.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.704+0000 7fd72d59a700 1 -- 172.21.15.89:0/3665359692 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720011830 con 0x7fd730133420 2023-09-30T22:23:19.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.704+0000 7fd72d59a700 1 -- 172.21.15.89:0/3665359692 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd720011990 con 0x7fd730133420 2023-09-30T22:23:19.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.704+0000 7fd72d59a700 1 -- 172.21.15.89:0/3665359692 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720011d20 con 0x7fd730133420 2023-09-30T22:23:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.704+0000 7fd734ca2700 1 -- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133420 msgr2=0x7fd7301337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.704+0000 7fd734ca2700 1 --2- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133420 0x7fd7301337f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd720009870 tx=0x7fd720009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.705+0000 7fd734ca2700 1 -- 172.21.15.89:0/3665359692 shutdown_connections 2023-09-30T22:23:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.705+0000 7fd734ca2700 1 --2- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133420 0x7fd7301337f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.705+0000 7fd734ca2700 1 --2- 172.21.15.89:0/3665359692 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133e30 0x7fd73012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.705+0000 7fd734ca2700 1 -- 172.21.15.89:0/3665359692 >> 172.21.15.89:0/3665359692 conn(0x7fd7300a4210 msgr2=0x7fd7300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.705+0000 7fd734ca2700 1 -- 172.21.15.89:0/3665359692 shutdown_connections 2023-09-30T22:23:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.705+0000 7fd734ca2700 1 -- 172.21.15.89:0/3665359692 wait complete. 2023-09-30T22:23:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.706+0000 7fd734ca2700 1 Processor -- start 2023-09-30T22:23:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.706+0000 7fd734ca2700 1 -- start start 2023-09-30T22:23:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd734ca2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133420 0x7fd7300709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd734ca2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133e30 0x7fd730070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd734ca2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd730134260 0x7fd730071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd734ca2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd730131700 con 0x7fd730134260 2023-09-30T22:23:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd734ca2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73011f7b0 con 0x7fd730133420 2023-09-30T22:23:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd734ca2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd730071d90 con 0x7fd730133e30 2023-09-30T22:23:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72dd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133e30 0x7fd730070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72dd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133e30 0x7fd730070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48144/0 (socket says 172.21.15.89:48144) 2023-09-30T22:23:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72dd9b700 1 -- 172.21.15.89:0/2028647493 learned_addr learned my addr 172.21.15.89:0/2028647493 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72ed9d700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd730134260 0x7fd730071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72e59c700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133420 0x7fd7300709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72ed9d700 1 -- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133e30 msgr2=0x7fd730070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72ed9d700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133e30 0x7fd730070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72ed9d700 1 -- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133420 msgr2=0x7fd7300709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72ed9d700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133420 0x7fd7300709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72ed9d700 1 -- 172.21.15.89:0/2028647493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7200094a0 con 0x7fd730134260 2023-09-30T22:23:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.707+0000 7fd72dd9b700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133e30 0x7fd730070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.708+0000 7fd72ed9d700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd730134260 0x7fd730071420 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fd72800cab0 tx=0x7fd72800cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:19.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.708+0000 7fd71effd700 1 -- 172.21.15.89:0/2028647493 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72800a9a0 con 0x7fd730134260 2023-09-30T22:23:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.708+0000 7fd71effd700 1 -- 172.21.15.89:0/2028647493 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72800ab00 con 0x7fd730134260 2023-09-30T22:23:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.708+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd730120520 con 0x7fd730134260 2023-09-30T22:23:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.708+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd730072210 con 0x7fd730134260 2023-09-30T22:23:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.708+0000 7fd71effd700 1 -- 172.21.15.89:0/2028647493 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72800ae70 con 0x7fd730134260 2023-09-30T22:23:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.709+0000 7fd71effd700 1 -- 172.21.15.89:0/2028647493 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd72801b3b0 con 0x7fd730134260 2023-09-30T22:23:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.709+0000 7fd71effd700 1 -- 172.21.15.89:0/2028647493 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd72803f1c0 con 0x7fd730134260 2023-09-30T22:23:19.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.709+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7140052e0 con 0x7fd730134260 2023-09-30T22:23:19.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.714+0000 7fd71effd700 1 -- 172.21.15.89:0/2028647493 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd728017080 con 0x7fd730134260 2023-09-30T22:23:19.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.743+0000 7f48946b7700 1 Processor -- start 2023-09-30T22:23:19.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.743+0000 7f48946b7700 1 -- start start 2023-09-30T22:23:19.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f48946b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c13cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f48946b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c13d240 0x7f488c13f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f48946b7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c134980 con 0x7f488c06c0e0 2023-09-30T22:23:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f48946b7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c06c4b0 con 0x7f488c13d240 2023-09-30T22:23:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f48946b7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f488c09b0e0 con 0x7f488c09ac90 2023-09-30T22:23:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f4891c52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c13cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f4891c52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c13cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48156/0 (socket says 172.21.15.89:48156) 2023-09-30T22:23:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f4891c52700 1 -- 172.21.15.89:0/2758841425 learned_addr learned my addr 172.21.15.89:0/2758841425 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f4892c54700 1 --2- 172.21.15.89:0/2758841425 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c13d240 0x7f488c13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f4891c52700 1 -- 172.21.15.89:0/2758841425 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c13d240 msgr2=0x7f488c13f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f4891c52700 1 --2- 172.21.15.89:0/2758841425 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c13d240 0x7f488c13f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f4891c52700 1 -- 172.21.15.89:0/2758841425 >> v1:172.21.15.89:6789/0 conn(0x7f488c06c0e0 legacy=0x7f488c134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.744+0000 7f4891c52700 1 -- 172.21.15.89:0/2758841425 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c0a0600 con 0x7f488c09ac90 2023-09-30T22:23:19.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.745+0000 7f4891c52700 1 --2- 172.21.15.89:0/2758841425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c13cd00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f487c009920 tx=0x7f487c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aa1f2d2121203ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.745+0000 7f4891451700 1 -- 172.21.15.89:0/2758841425 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c011830 con 0x7f488c09ac90 2023-09-30T22:23:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.745+0000 7f4891451700 1 -- 172.21.15.89:0/2758841425 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487c011990 con 0x7f488c09ac90 2023-09-30T22:23:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.745+0000 7f4891451700 1 -- 172.21.15.89:0/2758841425 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c011d20 con 0x7f488c09ac90 2023-09-30T22:23:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.745+0000 7f48946b7700 1 -- 172.21.15.89:0/2758841425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 msgr2=0x7f488c13cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.745+0000 7f48946b7700 1 --2- 172.21.15.89:0/2758841425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c13cd00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f487c009920 tx=0x7f487c009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.746+0000 7f48946b7700 1 -- 172.21.15.89:0/2758841425 shutdown_connections 2023-09-30T22:23:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.746+0000 7f48946b7700 1 --2- 172.21.15.89:0/2758841425 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c13d240 0x7f488c13f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.746+0000 7f48946b7700 1 --2- 172.21.15.89:0/2758841425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c13cd00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.746+0000 7f48946b7700 1 -- 172.21.15.89:0/2758841425 >> 172.21.15.89:0/2758841425 conn(0x7f488c0a4210 msgr2=0x7f488c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.746+0000 7f48946b7700 1 -- 172.21.15.89:0/2758841425 shutdown_connections 2023-09-30T22:23:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.746+0000 7f48946b7700 1 -- 172.21.15.89:0/2758841425 wait complete. 2023-09-30T22:23:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f48946b7700 1 Processor -- start 2023-09-30T22:23:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f48946b7700 1 -- start start 2023-09-30T22:23:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f48946b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c06c0e0 0x7f488c09b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f48946b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c09b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f48946b7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f488c13d240 0x7f488c09beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f48946b7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c072590 con 0x7f488c13d240 2023-09-30T22:23:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f48946b7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c09e430 con 0x7f488c06c0e0 2023-09-30T22:23:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f48946b7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f488c13f630 con 0x7f488c09ac90 2023-09-30T22:23:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f4891c52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c09b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f4892453700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c06c0e0 0x7f488c09b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f4891c52700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c09b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48172/0 (socket says 172.21.15.89:48172) 2023-09-30T22:23:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f4891c52700 1 -- 172.21.15.89:0/3195562537 learned_addr learned my addr 172.21.15.89:0/3195562537 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.747+0000 7f4892453700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c06c0e0 0x7f488c09b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37550/0 (socket says 172.21.15.89:37550) 2023-09-30T22:23:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4892c54700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f488c13d240 0x7f488c09beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4892453700 1 -- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 msgr2=0x7f488c09b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4892453700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c09b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4892453700 1 -- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f488c13d240 msgr2=0x7f488c09beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4892453700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f488c13d240 0x7f488c09beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4892453700 1 -- 172.21.15.89:0/3195562537 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c009550 con 0x7f488c06c0e0 2023-09-30T22:23:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4892c54700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f488c13d240 0x7f488c09beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4892453700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c06c0e0 0x7f488c09b430 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f488800c400 tx=0x7f488800c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4882ffd700 1 -- 172.21.15.89:0/3195562537 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880052d0 con 0x7f488c06c0e0 2023-09-30T22:23:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4882ffd700 1 -- 172.21.15.89:0/3195562537 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4888005430 con 0x7f488c06c0e0 2023-09-30T22:23:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c0a03f0 con 0x7f488c06c0e0 2023-09-30T22:23:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f4882ffd700 1 -- 172.21.15.89:0/3195562537 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880038c0 con 0x7f488c06c0e0 2023-09-30T22:23:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.748+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c1391b0 con 0x7f488c06c0e0 2023-09-30T22:23:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.749+0000 7f4882ffd700 1 -- 172.21.15.89:0/3195562537 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4888003a20 con 0x7f488c06c0e0 2023-09-30T22:23:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.749+0000 7f4882ffd700 1 -- 172.21.15.89:0/3195562537 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f488803ea10 con 0x7f488c06c0e0 2023-09-30T22:23:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.749+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c074ea0 con 0x7f488c06c0e0 2023-09-30T22:23:19.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.754+0000 7f4882ffd700 1 -- 172.21.15.89:0/3195562537 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4888017020 con 0x7f488c06c0e0 2023-09-30T22:23:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.784+0000 7f11175b7700 1 Processor -- start 2023-09-30T22:23:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.784+0000 7f11175b7700 1 -- start start 2023-09-30T22:23:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f11175b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009d6f0 0x7f111009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f11175b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009e000 0x7f11101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f11175b7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f111009e410 0x7f1110133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f11175b7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111006b460 con 0x7f111009e410 2023-09-30T22:23:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f11175b7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111009b120 con 0x7f111009d6f0 2023-09-30T22:23:19.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f11175b7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111009a7e0 con 0x7f111009e000 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f111009e410 0x7f1110133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f111009e410 0x7f1110133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46082/0 (socket says 172.21.15.89:46082) 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 -- 172.21.15.89:0/547680633 learned_addr learned my addr 172.21.15.89:0/547680633 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1114b52700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009e000 0x7f11101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115353700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009d6f0 0x7f111009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 -- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009e000 msgr2=0x7f11101339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009e000 0x7f11101339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 -- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009d6f0 msgr2=0x7f111009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009d6f0 0x7f111009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 -- 172.21.15.89:0/547680633 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111009a500 con 0x7f111009e410 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.785+0000 7f1115b54700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e410 0x7f1110133f30 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1108009e50 tx=0x7f110800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e773b845203b8c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:19.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f1107fff700 1 -- 172.21.15.89:0/547680633 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110800e070 con 0x7f111009e410 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f1107fff700 1 -- 172.21.15.89:0/547680633 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110800b910 con 0x7f111009e410 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f1107fff700 1 -- 172.21.15.89:0/547680633 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110800bcc0 con 0x7f111009e410 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f11175b7700 1 -- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e410 msgr2=0x7f1110133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f11175b7700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e410 0x7f1110133f30 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1108009e50 tx=0x7f110800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f11175b7700 1 -- 172.21.15.89:0/547680633 shutdown_connections 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f11175b7700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e410 0x7f1110133f30 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f11175b7700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009d6f0 0x7f111009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f11175b7700 1 --2- 172.21.15.89:0/547680633 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009e000 0x7f11101339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f11175b7700 1 -- 172.21.15.89:0/547680633 >> 172.21.15.89:0/547680633 conn(0x7f1110066f50 msgr2=0x7f111009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.786+0000 7f11175b7700 1 -- 172.21.15.89:0/547680633 shutdown_connections 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.787+0000 7f11175b7700 1 -- 172.21.15.89:0/547680633 wait complete. 2023-09-30T22:23:19.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.787+0000 7f11175b7700 1 Processor -- start 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.787+0000 7f11175b7700 1 -- start start 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f11175b7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009d6f0 0x7f11101a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f11175b7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e000 0x7f11101a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f11175b7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009e410 0x7f11101c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f11175b7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11101b9110 con 0x7f111009e000 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f11175b7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11101a4c60 con 0x7f111009e410 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f11175b7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11101a6390 con 0x7f111009d6f0 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1115353700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009d6f0 0x7f11101a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1115353700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009d6f0 0x7f11101a5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48186/0 (socket says 172.21.15.89:48186) 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1115353700 1 -- 172.21.15.89:0/2963351643 learned_addr learned my addr 172.21.15.89:0/2963351643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1114b52700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e000 0x7f11101a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1115b54700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009e410 0x7f11101c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1114b52700 1 -- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009d6f0 msgr2=0x7f11101a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1114b52700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009d6f0 0x7f11101a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1114b52700 1 -- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009e410 msgr2=0x7f11101c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1114b52700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009e410 0x7f11101c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1114b52700 1 -- 172.21.15.89:0/2963351643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1108009b30 con 0x7f111009e000 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.788+0000 7f1115353700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009d6f0 0x7f11101a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.789+0000 7f1114b52700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e000 0x7f11101a5590 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f110000e8d0 tx=0x7f110000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.789+0000 7f1105ffb700 1 -- 172.21.15.89:0/2963351643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100013680 con 0x7f111009e000 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.789+0000 7f1105ffb700 1 -- 172.21.15.89:0/2963351643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11000137e0 con 0x7f111009e000 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.789+0000 7f1105ffb700 1 -- 172.21.15.89:0/2963351643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100013b50 con 0x7f111009e000 2023-09-30T22:23:19.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.789+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11101a7d00 con 0x7f111009e000 2023-09-30T22:23:19.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.790+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111009c710 con 0x7f111009e000 2023-09-30T22:23:19.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.790+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11101a70c0 con 0x7f111009e000 2023-09-30T22:23:19.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.791+0000 7f1105ffb700 1 -- 172.21.15.89:0/2963351643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1100013cb0 con 0x7f111009e000 2023-09-30T22:23:19.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.792+0000 7f1105ffb700 1 -- 172.21.15.89:0/2963351643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f110001be60 con 0x7f111009e000 2023-09-30T22:23:19.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.796+0000 7f1105ffb700 1 -- 172.21.15.89:0/2963351643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1100044060 con 0x7f111009e000 2023-09-30T22:23:19.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.830+0000 7f95e2dff700 1 Processor -- start 2023-09-30T22:23:19.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.830+0000 7f95e2dff700 1 -- start start 2023-09-30T22:23:19.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e2dff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc1316c0 0x7f95dc131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e2dff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc06ae80 0x7f95dc139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e2dff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc09e130 con 0x7f95dc09d9f0 2023-09-30T22:23:19.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e2dff700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc1312b0 con 0x7f95dc06ae80 2023-09-30T22:23:19.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e2dff700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc09f6e0 con 0x7f95dc1316c0 2023-09-30T22:23:19.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e0b9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc1316c0 0x7f95dc131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e139c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc06ae80 0x7f95dc139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e0b9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc1316c0 0x7f95dc131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48196/0 (socket says 172.21.15.89:48196) 2023-09-30T22:23:19.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e139c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc06ae80 0x7f95dc139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37584/0 (socket says 172.21.15.89:37584) 2023-09-30T22:23:19.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95e0b9b700 1 -- 172.21.15.89:0/993584293 learned_addr learned my addr 172.21.15.89:0/993584293 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:19.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.831+0000 7f95dbfff700 1 --1- 172.21.15.89:0/993584293 >> v1:172.21.15.89:6789/0 conn(0x7f95dc09d9f0 0x7f95dc06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48036/0 (socket says 172.21.15.89:48036) 2023-09-30T22:23:19.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.832+0000 7f95e139c700 1 -- 172.21.15.89:0/993584293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc1316c0 msgr2=0x7f95dc131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:19.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.832+0000 7f95e139c700 1 --2- 172.21.15.89:0/993584293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc1316c0 0x7f95dc131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.832+0000 7f95e139c700 1 -- 172.21.15.89:0/993584293 >> v1:172.21.15.89:6789/0 conn(0x7f95dc09d9f0 legacy=0x7f95dc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.832+0000 7f95e139c700 1 -- 172.21.15.89:0/993584293 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc09e760 con 0x7f95dc06ae80 2023-09-30T22:23:19.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.832+0000 7f95e139c700 1 --2- 172.21.15.89:0/993584293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc06ae80 0x7f95dc139550 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009dc0 tx=0x7f95d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6866f879deebf49a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:19.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.832+0000 7f95db7fe700 1 -- 172.21.15.89:0/993584293 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d400e070 con 0x7f95dc06ae80 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.832+0000 7f95db7fe700 1 -- 172.21.15.89:0/993584293 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d400bb20 con 0x7f95dc06ae80 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.832+0000 7f95db7fe700 1 -- 172.21.15.89:0/993584293 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4012430 con 0x7f95dc06ae80 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.833+0000 7f95e2dff700 1 -- 172.21.15.89:0/993584293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc06ae80 msgr2=0x7f95dc139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.833+0000 7f95e2dff700 1 --2- 172.21.15.89:0/993584293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc06ae80 0x7f95dc139550 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009dc0 tx=0x7f95d400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.833+0000 7f95e2dff700 1 -- 172.21.15.89:0/993584293 shutdown_connections 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.833+0000 7f95e2dff700 1 --2- 172.21.15.89:0/993584293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc06ae80 0x7f95dc139550 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.833+0000 7f95e2dff700 1 --2- 172.21.15.89:0/993584293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc1316c0 0x7f95dc131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.833+0000 7f95e2dff700 1 -- 172.21.15.89:0/993584293 >> 172.21.15.89:0/993584293 conn(0x7f95dc066f50 msgr2=0x7f95dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:19.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.833+0000 7f95e2dff700 1 -- 172.21.15.89:0/993584293 shutdown_connections 2023-09-30T22:23:19.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.833+0000 7f95e2dff700 1 -- 172.21.15.89:0/993584293 wait complete. 2023-09-30T22:23:19.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.834+0000 7f95e2dff700 1 Processor -- start 2023-09-30T22:23:19.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.835+0000 7f95e2dff700 1 -- start start 2023-09-30T22:23:19.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.835+0000 7f95e2dff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc06ae80 0x7f95dc127fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.835+0000 7f95e2dff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc09d9f0 0x7f95dc06e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.835+0000 7f95e2dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95dc1316c0 0x7f95dc06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.835+0000 7f95e2dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc070a60 con 0x7f95dc1316c0 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.835+0000 7f95e2dff700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc06fcd0 con 0x7f95dc09d9f0 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.835+0000 7f95e2dff700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc071640 con 0x7f95dc06ae80 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc09d9f0 0x7f95dc06e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc09d9f0 0x7f95dc06e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37586/0 (socket says 172.21.15.89:37586) 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 -- 172.21.15.89:0/2232827658 learned_addr learned my addr 172.21.15.89:0/2232827658 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95e0b9b700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc06ae80 0x7f95dc127fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95e139c700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95dc1316c0 0x7f95dc06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 -- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc06ae80 msgr2=0x7f95dc127fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc06ae80 0x7f95dc127fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 -- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95dc1316c0 msgr2=0x7f95dc06ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95dc1316c0 0x7f95dc06ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 -- 172.21.15.89:0/2232827658 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d4009a70 con 0x7f95dc09d9f0 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95dbfff700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc09d9f0 0x7f95dc06e8e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c4d0 tx=0x7f95c800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95d97fa700 1 -- 172.21.15.89:0/2232827658 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8005cb0 con 0x7f95dc09d9f0 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95d97fa700 1 -- 172.21.15.89:0/2232827658 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8005e10 con 0x7f95dc09d9f0 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95d97fa700 1 -- 172.21.15.89:0/2232827658 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80072d0 con 0x7f95dc09d9f0 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc071900 con 0x7f95dc09d9f0 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.836+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc0727c0 con 0x7f95dc09d9f0 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.837+0000 7f95d97fa700 1 -- 172.21.15.89:0/2232827658 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f95c8007430 con 0x7f95dc09d9f0 2023-09-30T22:23:19.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.837+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc074ea0 con 0x7f95dc09d9f0 2023-09-30T22:23:19.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.837+0000 7f95d97fa700 1 -- 172.21.15.89:0/2232827658 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f95c803e930 con 0x7f95dc09d9f0 2023-09-30T22:23:19.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.842+0000 7f95d97fa700 1 -- 172.21.15.89:0/2232827658 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f95c8011950 con 0x7f95dc09d9f0 2023-09-30T22:23:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.932+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd714005590 con 0x7fd730134260 2023-09-30T22:23:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.933+0000 7fd71effd700 1 -- 172.21.15.89:0/2028647493 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7280192c0 con 0x7fd730134260 2023-09-30T22:23:19.933 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd730134260 msgr2=0x7fd730071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd730134260 0x7fd730071420 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fd72800cab0 tx=0x7fd72800cdc0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 shutdown_connections 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd730134260 0x7fd730071420 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd730133420 0x7fd7300709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 --2- 172.21.15.89:0/2028647493 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd730133e30 0x7fd730070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 >> 172.21.15.89:0/2028647493 conn(0x7fd7300a4210 msgr2=0x7fd7300a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 shutdown_connections 2023-09-30T22:23:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.935+0000 7fd734ca2700 1 -- 172.21.15.89:0/2028647493 wait complete. 2023-09-30T22:23:19.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:19.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.960+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f488c06b250 con 0x7f488c06c0e0 2023-09-30T22:23:19.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.961+0000 7f4882ffd700 1 -- 172.21.15.89:0/3195562537 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4888018260 con 0x7f488c06c0e0 2023-09-30T22:23:19.961 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:19.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c06c0e0 msgr2=0x7f488c09b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:19.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c06c0e0 0x7f488c09b430 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f488800c400 tx=0x7f488800c710 comp rx=0 tx=0).stop 2023-09-30T22:23:19.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 shutdown_connections 2023-09-30T22:23:19.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f488c13d240 0x7f488c09beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f488c06c0e0 0x7f488c09b430 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 --2- 172.21.15.89:0/3195562537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f488c09ac90 0x7f488c09b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:19.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 >> 172.21.15.89:0/3195562537 conn(0x7f488c0a4210 msgr2=0x7f488c0a0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:19.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 shutdown_connections 2023-09-30T22:23:19.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:19.964+0000 7f48946b7700 1 -- 172.21.15.89:0/3195562537 wait complete. 2023-09-30T22:23:19.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:20.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.020+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f11101a8200 con 0x7f111009e000 2023-09-30T22:23:20.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.021+0000 7f1105ffb700 1 -- 172.21.15.89:0/2963351643 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11101a8200 con 0x7f111009e000 2023-09-30T22:23:20.021 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:20.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e000 msgr2=0x7f11101a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:20.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e000 0x7f11101a5590 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f110000e8d0 tx=0x7f110000ebe0 comp rx=0 tx=0).stop 2023-09-30T22:23:20.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 shutdown_connections 2023-09-30T22:23:20.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111009e410 0x7f11101c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:20.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111009d6f0 0x7f11101a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:20.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 --2- 172.21.15.89:0/2963351643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111009e000 0x7f11101a5590 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:20.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 >> 172.21.15.89:0/2963351643 conn(0x7f1110066f50 msgr2=0x7f11101bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:20.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 shutdown_connections 2023-09-30T22:23:20.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.023+0000 7f11175b7700 1 -- 172.21.15.89:0/2963351643 wait complete. 2023-09-30T22:23:20.035 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:20.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.036+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f95dc09e130 con 0x7f95dc09d9f0 2023-09-30T22:23:20.037 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:20.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.037+0000 7f95d97fa700 1 -- 172.21.15.89:0/2232827658 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f95dc09e130 con 0x7f95dc09d9f0 2023-09-30T22:23:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc09d9f0 msgr2=0x7f95dc06e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc09d9f0 0x7f95dc06e8e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c4d0 tx=0x7f95c800c890 comp rx=0 tx=0).stop 2023-09-30T22:23:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 shutdown_connections 2023-09-30T22:23:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95dc1316c0 0x7f95dc06ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95dc06ae80 0x7f95dc127fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:20.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 --2- 172.21.15.89:0/2232827658 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95dc09d9f0 0x7f95dc06e8e0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 >> 172.21.15.89:0/2232827658 conn(0x7f95dc066f50 msgr2=0x7f95dc0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 shutdown_connections 2023-09-30T22:23:20.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:20.039+0000 7f95e2dff700 1 -- 172.21.15.89:0/2232827658 wait complete. 2023-09-30T22:23:20.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:20.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:20.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:21.036 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:21.052 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:21.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.100+0000 7fe06a9f7700 1 Processor -- start 2023-09-30T22:23:21.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.101+0000 7fe06a9f7700 1 -- start start 2023-09-30T22:23:21.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.101+0000 7fe06a9f7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe064130670 0x7fe064130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.101+0000 7fe06a9f7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406f680 0x7fe06406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.101+0000 7fe06a9f7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe06406ffd0 0x7fe064137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.101+0000 7fe06a9f7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06409a350 con 0x7fe06406ffd0 2023-09-30T22:23:21.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.101+0000 7fe06a9f7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe064130260 con 0x7fe064130670 2023-09-30T22:23:21.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.101+0000 7fe06a9f7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe064071ff0 con 0x7fe06406f680 2023-09-30T22:23:21.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe0637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406f680 0x7fe06406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe0637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406f680 0x7fe06406fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47286/0 (socket says 172.21.15.89:47286) 2023-09-30T22:23:21.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe0637fe700 1 -- 172.21.15.89:0/3784073056 learned_addr learned my addr 172.21.15.89:0/3784073056 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe063fff700 1 --2- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe064130670 0x7fe064130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe068f94700 1 --2- 172.21.15.89:0/3784073056 >> v2:172.21.15.89:3300/0 conn(0x7fe06406ffd0 0x7fe064137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe068f94700 1 -- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406f680 msgr2=0x7fe06406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe068f94700 1 --2- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406f680 0x7fe06406fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe068f94700 1 -- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe064130670 msgr2=0x7fe064130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:21.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe068f94700 1 --2- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe064130670 0x7fe064130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe068f94700 1 -- 172.21.15.89:0/3784073056 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe064070470 con 0x7fe06406ffd0 2023-09-30T22:23:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe068f94700 1 --2- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe06406ffd0 0x7fe064137530 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009d90 tx=0x7fe05c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d839eaeda3f92115 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe062ffd700 1 -- 172.21.15.89:0/3784073056 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00e070 con 0x7fe06406ffd0 2023-09-30T22:23:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe062ffd700 1 -- 172.21.15.89:0/3784073056 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c00bb20 con 0x7fe06406ffd0 2023-09-30T22:23:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.102+0000 7fe062ffd700 1 -- 172.21.15.89:0/3784073056 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c012430 con 0x7fe06406ffd0 2023-09-30T22:23:21.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe06406ffd0 msgr2=0x7fe064137530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 --2- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe06406ffd0 0x7fe064137530 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009d90 tx=0x7fe05c00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3784073056 shutdown_connections 2023-09-30T22:23:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 --2- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe06406ffd0 0x7fe064137530 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 --2- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe064130670 0x7fe064130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 --2- 172.21.15.89:0/3784073056 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406f680 0x7fe06406fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3784073056 >> 172.21.15.89:0/3784073056 conn(0x7fe0640a2aa0 msgr2=0x7fe0640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:21.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3784073056 shutdown_connections 2023-09-30T22:23:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.103+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3784073056 wait complete. 2023-09-30T22:23:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.105+0000 7fe06a9f7700 1 Processor -- start 2023-09-30T22:23:21.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.105+0000 7fe06a9f7700 1 -- start start 2023-09-30T22:23:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe06a9f7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe06406f680 0x7fe06409ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe06a9f7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406ffd0 0x7fe06409d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe06a9f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe064130670 0x7fe06409d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe06a9f7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06411e8d0 con 0x7fe064130670 2023-09-30T22:23:21.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe06a9f7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06409e2c0 con 0x7fe06406f680 2023-09-30T22:23:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe06a9f7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe064071820 con 0x7fe06406ffd0 2023-09-30T22:23:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe063fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe06406f680 0x7fe06409ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe0637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406ffd0 0x7fe06409d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe063fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe06406f680 0x7fe06409ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60394/0 (socket says 172.21.15.89:60394) 2023-09-30T22:23:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe0637fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406ffd0 0x7fe06409d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47288/0 (socket says 172.21.15.89:47288) 2023-09-30T22:23:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe063fff700 1 -- 172.21.15.89:0/3529564260 learned_addr learned my addr 172.21.15.89:0/3529564260 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe068f94700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe064130670 0x7fe06409d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe0637fe700 1 -- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe06406f680 msgr2=0x7fe06409ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe0637fe700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe06406f680 0x7fe06409ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe0637fe700 1 -- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe064130670 msgr2=0x7fe06409d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe0637fe700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe064130670 0x7fe06409d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe0637fe700 1 -- 172.21.15.89:0/3529564260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c009a70 con 0x7fe06406ffd0 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe068f94700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe064130670 0x7fe06409d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.106+0000 7fe063fff700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe06406f680 0x7fe06409ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.107+0000 7fe0637fe700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406ffd0 0x7fe06409d3c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe05800ed70 tx=0x7fe05800d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.107+0000 7fe060ff9700 1 -- 172.21.15.89:0/3529564260 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05800dd90 con 0x7fe06406ffd0 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.107+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06412ea80 con 0x7fe06406ffd0 2023-09-30T22:23:21.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.107+0000 7fe060ff9700 1 -- 172.21.15.89:0/3529564260 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe058010440 con 0x7fe06406ffd0 2023-09-30T22:23:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.107+0000 7fe060ff9700 1 -- 172.21.15.89:0/3529564260 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0580107b0 con 0x7fe06406ffd0 2023-09-30T22:23:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.107+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06411f1d0 con 0x7fe06406ffd0 2023-09-30T22:23:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.108+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06409ecb0 con 0x7fe06406ffd0 2023-09-30T22:23:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.108+0000 7fe060ff9700 1 -- 172.21.15.89:0/3529564260 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe058010910 con 0x7fe06406ffd0 2023-09-30T22:23:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.109+0000 7fe060ff9700 1 -- 172.21.15.89:0/3529564260 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe058016e30 con 0x7fe06406ffd0 2023-09-30T22:23:21.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.114+0000 7fe060ff9700 1 -- 172.21.15.89:0/3529564260 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe058018400 con 0x7fe06406ffd0 2023-09-30T22:23:21.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.132+0000 7f39172e5700 1 Processor -- start 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f39172e5700 1 -- start start 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f39172e5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391006a770 0x7f391009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f39172e5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391009dc30 0x7f391009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f39172e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f391013c1e0 0x7f391013e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f39172e5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391006ab40 con 0x7f391013c1e0 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f39172e5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391009ebc0 con 0x7f391006a770 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f39172e5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f391009a810 con 0x7f391009dc30 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f3915081700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391006a770 0x7f391009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f3915882700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f391013c1e0 0x7f391013e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f3915882700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f391013c1e0 0x7f391013e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34910/0 (socket says 172.21.15.89:34910) 2023-09-30T22:23:21.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f3915081700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391006a770 0x7f391009d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60398/0 (socket says 172.21.15.89:60398) 2023-09-30T22:23:21.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f3915882700 1 -- 172.21.15.89:0/2539622041 learned_addr learned my addr 172.21.15.89:0/2539622041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:21.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.133+0000 7f3914880700 1 --2- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391009dc30 0x7f391009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3915882700 1 -- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391009dc30 msgr2=0x7f391009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3915882700 1 --2- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391009dc30 0x7f391009e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3915882700 1 -- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391006a770 msgr2=0x7f391009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3915882700 1 --2- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391006a770 0x7f391009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3915882700 1 -- 172.21.15.89:0/2539622041 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391009e920 con 0x7f391013c1e0 2023-09-30T22:23:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3915882700 1 --2- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391013c1e0 0x7f391013e5c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3908009e50 tx=0x7f390800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee86742fc05d5c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3907fff700 1 -- 172.21.15.89:0/2539622041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800e070 con 0x7f391013c1e0 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3907fff700 1 -- 172.21.15.89:0/2539622041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390800b910 con 0x7f391013c1e0 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.134+0000 7f3907fff700 1 -- 172.21.15.89:0/2539622041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800bcc0 con 0x7f391013c1e0 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 -- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391013c1e0 msgr2=0x7f391013e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 --2- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391013c1e0 0x7f391013e5c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3908009e50 tx=0x7f390800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 -- 172.21.15.89:0/2539622041 shutdown_connections 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 --2- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391013c1e0 0x7f391013e5c0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 --2- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391006a770 0x7f391009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 --2- 172.21.15.89:0/2539622041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391009dc30 0x7f391009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 -- 172.21.15.89:0/2539622041 >> 172.21.15.89:0/2539622041 conn(0x7f3910066f50 msgr2=0x7f391009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 -- 172.21.15.89:0/2539622041 shutdown_connections 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.135+0000 7f39172e5700 1 -- 172.21.15.89:0/2539622041 wait complete. 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f39172e5700 1 Processor -- start 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f39172e5700 1 -- start start 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f39172e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391006a770 0x7f3910071e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f39172e5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391009dc30 0x7f391012dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f39172e5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391013c1e0 0x7f391012e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f39172e5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391012fdd0 con 0x7f391006a770 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f39172e5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100a35f0 con 0x7f391009dc30 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f39172e5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f391009f1a0 con 0x7f391013c1e0 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f3914880700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391009dc30 0x7f391012dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f3914880700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391009dc30 0x7f391012dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60400/0 (socket says 172.21.15.89:60400) 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f3914880700 1 -- 172.21.15.89:0/545838008 learned_addr learned my addr 172.21.15.89:0/545838008 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f3915882700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391013c1e0 0x7f391012e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.136+0000 7f3915081700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391006a770 0x7f3910071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3914880700 1 -- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391013c1e0 msgr2=0x7f391012e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3914880700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391013c1e0 0x7f391012e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3914880700 1 -- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391006a770 msgr2=0x7f3910071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3914880700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391006a770 0x7f3910071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3914880700 1 -- 172.21.15.89:0/545838008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908009b30 con 0x7f391009dc30 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3914880700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391009dc30 0x7f391012dcc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f390000e7e0 tx=0x7f390000eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3905ffb700 1 -- 172.21.15.89:0/545838008 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900013610 con 0x7f391009dc30 2023-09-30T22:23:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3905ffb700 1 -- 172.21.15.89:0/545838008 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900013770 con 0x7f391009dc30 2023-09-30T22:23:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f3905ffb700 1 -- 172.21.15.89:0/545838008 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900013ae0 con 0x7f391009dc30 2023-09-30T22:23:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39101315c0 con 0x7f391009dc30 2023-09-30T22:23:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.137+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39101322f0 con 0x7f391009dc30 2023-09-30T22:23:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.138+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391011cc30 con 0x7f391009dc30 2023-09-30T22:23:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.139+0000 7f3905ffb700 1 -- 172.21.15.89:0/545838008 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3900013c40 con 0x7f391009dc30 2023-09-30T22:23:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.139+0000 7f3905ffb700 1 -- 172.21.15.89:0/545838008 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f390003f780 con 0x7f391009dc30 2023-09-30T22:23:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.143+0000 7f3905ffb700 1 -- 172.21.15.89:0/545838008 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3900026480 con 0x7f391009dc30 2023-09-30T22:23:21.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.192+0000 7f4ae7f3c700 1 Processor -- start 2023-09-30T22:23:21.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.192+0000 7f4ae7f3c700 1 -- start start 2023-09-30T22:23:21.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae7f3c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae7f3c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae7f3c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ae009e410 0x7f4ae0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae7f3c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae006b460 con 0x7f4ae009e410 2023-09-30T22:23:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae7f3c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae009b120 con 0x7f4ae009e000 2023-09-30T22:23:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae7f3c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae009a7e0 con 0x7f4ae009d6f0 2023-09-30T22:23:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ae009e410 0x7f4ae0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae5cd8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ae009e410 0x7f4ae0133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34938/0 (socket says 172.21.15.89:34938) 2023-09-30T22:23:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 -- 172.21.15.89:0/1455885760 learned_addr learned my addr 172.21.15.89:0/1455885760 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae54d7700 1 --2- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 -- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 msgr2=0x7f4ae009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 --2- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 -- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 msgr2=0x7f4ae01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 --2- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 -- 172.21.15.89:0/1455885760 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae009a500 con 0x7f4ae009e410 2023-09-30T22:23:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.193+0000 7f4ae64d9700 1 --2- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 0x7f4ae0133f30 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009e50 tx=0x7f4ad800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ef5a272052b9c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae4cd6700 1 -- 172.21.15.89:0/1455885760 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800e070 con 0x7f4ae009e410 2023-09-30T22:23:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae4cd6700 1 -- 172.21.15.89:0/1455885760 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad800b910 con 0x7f4ae009e410 2023-09-30T22:23:21.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae4cd6700 1 -- 172.21.15.89:0/1455885760 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800bcc0 con 0x7f4ae009e410 2023-09-30T22:23:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 msgr2=0x7f4ae0133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae7f3c700 1 --2- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 0x7f4ae0133f30 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009e50 tx=0x7f4ad800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/1455885760 shutdown_connections 2023-09-30T22:23:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae7f3c700 1 --2- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 0x7f4ae0133f30 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae7f3c700 1 --2- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae7f3c700 1 --2- 172.21.15.89:0/1455885760 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.194+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/1455885760 >> 172.21.15.89:0/1455885760 conn(0x7f4ae0066f50 msgr2=0x7f4ae009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.195+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/1455885760 shutdown_connections 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.195+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/1455885760 wait complete. 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.195+0000 7f4ae7f3c700 1 Processor -- start 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae7f3c700 1 -- start start 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae7f3c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae01c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae7f3c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae7f3c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 0x7f4ae01c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae7f3c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae01a9620 con 0x7f4ae009e410 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae7f3c700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae01a8b60 con 0x7f4ae009e000 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae7f3c700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae01c5aa0 con 0x7f4ae009d6f0 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae5cd8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae01c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae54d7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae5cd8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae01c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47338/0 (socket says 172.21.15.89:47338) 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae54d7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60424/0 (socket says 172.21.15.89:60424) 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae5cd8700 1 -- 172.21.15.89:0/2642111163 learned_addr learned my addr 172.21.15.89:0/2642111163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.196+0000 7f4ae64d9700 1 --2- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 0x7f4ae01c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ae54d7700 1 -- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 msgr2=0x7f4ae01c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ae54d7700 1 --2- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae01c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ae54d7700 1 -- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 msgr2=0x7f4ae01c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ae54d7700 1 --2- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 0x7f4ae01c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ae54d7700 1 -- 172.21.15.89:0/2642111163 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8009b30 con 0x7f4ae009e000 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ae54d7700 1 --2- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01c5560 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000e8d0 tx=0x7f4ad000ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ad67fc700 1 -- 172.21.15.89:0/2642111163 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000c690 con 0x7f4ae009e000 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ad67fc700 1 -- 172.21.15.89:0/2642111163 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad000c7f0 con 0x7f4ae009e000 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.197+0000 7f4ad67fc700 1 -- 172.21.15.89:0/2642111163 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000cb60 con 0x7f4ae009e000 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.198+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae01a7910 con 0x7f4ae009e000 2023-09-30T22:23:21.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.198+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae01a9820 con 0x7f4ae009e000 2023-09-30T22:23:21.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.199+0000 7f4ad67fc700 1 -- 172.21.15.89:0/2642111163 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ad000ccc0 con 0x7f4ae009e000 2023-09-30T22:23:21.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.199+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae0074ea0 con 0x7f4ae009e000 2023-09-30T22:23:21.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.199+0000 7f4ad67fc700 1 -- 172.21.15.89:0/2642111163 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4ad003f6a0 con 0x7f4ae009e000 2023-09-30T22:23:21.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.208+0000 7f4ad67fc700 1 -- 172.21.15.89:0/2642111163 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ad001bde0 con 0x7f4ae009e000 2023-09-30T22:23:21.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.217+0000 7f50307e1700 1 Processor -- start 2023-09-30T22:23:21.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.217+0000 7f50307e1700 1 -- start start 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f50307e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f502809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f50307e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5028134ec0 0x7f50281379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f50307e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 0x7f50281383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f50307e1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502806c4b0 con 0x7f5028134ec0 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f50307e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50280a09c0 con 0x7f502806c0e0 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f50307e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5028134980 con 0x7f5028137f20 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f502e57d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f502809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f502e57d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f502809ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60434/0 (socket says 172.21.15.89:60434) 2023-09-30T22:23:21.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f502e57d700 1 -- 172.21.15.89:0/2127922416 learned_addr learned my addr 172.21.15.89:0/2127922416 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f502dd7c700 1 --2- 172.21.15.89:0/2127922416 >> v2:172.21.15.89:3300/0 conn(0x7f5028134ec0 0x7f50281379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.218+0000 7f502ed7e700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 0x7f50281383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502dd7c700 1 -- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 msgr2=0x7f50281383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502dd7c700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 0x7f50281383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502dd7c700 1 -- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 msgr2=0x7f502809ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502dd7c700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f502809ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502dd7c700 1 -- 172.21.15.89:0/2127922416 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280a0600 con 0x7f5028134ec0 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502e57d700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f502809ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502dd7c700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281379e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f5018009840 tx=0x7f5018009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0495b491eee13d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502d57b700 1 -- 172.21.15.89:0/2127922416 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018011830 con 0x7f5028134ec0 2023-09-30T22:23:21.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502d57b700 1 -- 172.21.15.89:0/2127922416 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5018011990 con 0x7f5028134ec0 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.219+0000 7f502d57b700 1 -- 172.21.15.89:0/2127922416 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018011d20 con 0x7f5028134ec0 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 -- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 msgr2=0x7f50281379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281379e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f5018009840 tx=0x7f5018009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 -- 172.21.15.89:0/2127922416 shutdown_connections 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 0x7f50281383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f502809ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 --2- 172.21.15.89:0/2127922416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281379e0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 -- 172.21.15.89:0/2127922416 >> 172.21.15.89:0/2127922416 conn(0x7f50280a4210 msgr2=0x7f50280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 -- 172.21.15.89:0/2127922416 shutdown_connections 2023-09-30T22:23:21.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.220+0000 7f50307e1700 1 -- 172.21.15.89:0/2127922416 wait complete. 2023-09-30T22:23:21.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f50307e1700 1 Processor -- start 2023-09-30T22:23:21.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f50307e1700 1 -- start start 2023-09-30T22:23:21.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f50307e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f50281ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f50307e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f50307e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 0x7f502811a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:21.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f50307e1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50281339d0 con 0x7f5028134ec0 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f50307e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502811c430 con 0x7f502806c0e0 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f50307e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f502811d780 con 0x7f5028137f20 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f502dd7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f502dd7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34964/0 (socket says 172.21.15.89:34964) 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f502dd7c700 1 -- 172.21.15.89:0/3132053960 learned_addr learned my addr 172.21.15.89:0/3132053960 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.221+0000 7f502e57d700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f50281ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f502dd7c700 1 -- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 msgr2=0x7f502811a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f502dd7c700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 0x7f502811a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f502dd7c700 1 -- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 msgr2=0x7f50281ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f502dd7c700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f50281ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f502dd7c700 1 -- 172.21.15.89:0/3132053960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024009550 con 0x7f5028134ec0 2023-09-30T22:23:21.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f502e57d700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f50281ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f502dd7c700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281cd290 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f5018000c00 tx=0x7f5018004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f501effd700 1 -- 172.21.15.89:0/3132053960 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018004240 con 0x7f5028134ec0 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f501effd700 1 -- 172.21.15.89:0/3132053960 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501801f070 con 0x7f5028134ec0 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.222+0000 7f501effd700 1 -- 172.21.15.89:0/3132053960 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50180036a0 con 0x7f5028134ec0 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.223+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50180094a0 con 0x7f5028134ec0 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.223+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50280a0450 con 0x7f5028134ec0 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.224+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5028074ea0 con 0x7f5028134ec0 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.224+0000 7f501effd700 1 -- 172.21.15.89:0/3132053960 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5018021030 con 0x7f5028134ec0 2023-09-30T22:23:21.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.225+0000 7f501effd700 1 -- 172.21.15.89:0/3132053960 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5018040bc0 con 0x7f5028134ec0 2023-09-30T22:23:21.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.233+0000 7f501effd700 1 -- 172.21.15.89:0/3132053960 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f50180239c0 con 0x7f5028134ec0 2023-09-30T22:23:21.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.342+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe064074ea0 con 0x7fe06406ffd0 2023-09-30T22:23:21.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.343+0000 7fe060ff9700 1 -- 172.21.15.89:0/3529564260 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe05800c750 con 0x7fe06406ffd0 2023-09-30T22:23:21.343 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:21.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.344+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406ffd0 msgr2=0x7fe06409d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.344+0000 7fe06a9f7700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406ffd0 0x7fe06409d3c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe05800ed70 tx=0x7fe05800d630 comp rx=0 tx=0).stop 2023-09-30T22:23:21.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 shutdown_connections 2023-09-30T22:23:21.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7fe06a9f7700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe064130670 0x7fe06409d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7fe06a9f7700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe06406f680 0x7fe06409ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7fe06a9f7700 1 --2- 172.21.15.89:0/3529564260 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe06406ffd0 0x7fe06409d3c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 >> 172.21.15.89:0/3529564260 conn(0x7fe0640a2aa0 msgr2=0x7fe06412d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:21.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 shutdown_connections 2023-09-30T22:23:21.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7fe06a9f7700 1 -- 172.21.15.89:0/3529564260 wait complete. 2023-09-30T22:23:21.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f39101317b0 con 0x7f391009dc30 2023-09-30T22:23:21.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.345+0000 7f3905ffb700 1 -- 172.21.15.89:0/545838008 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3900012e60 con 0x7f391009dc30 2023-09-30T22:23:21.346 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.348+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391009dc30 msgr2=0x7f391012dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.348+0000 7f39172e5700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391009dc30 0x7f391012dcc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f390000e7e0 tx=0x7f390000eaf0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.349+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 shutdown_connections 2023-09-30T22:23:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.349+0000 7f39172e5700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f391013c1e0 0x7f391012e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.349+0000 7f39172e5700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f391006a770 0x7f3910071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.349+0000 7f39172e5700 1 --2- 172.21.15.89:0/545838008 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f391009dc30 0x7f391012dcc0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.349+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 >> 172.21.15.89:0/545838008 conn(0x7f3910066f50 msgr2=0x7f391013de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.349+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 shutdown_connections 2023-09-30T22:23:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.349+0000 7f39172e5700 1 -- 172.21.15.89:0/545838008 wait complete. 2023-09-30T22:23:21.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:21.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:21.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.415+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ae009c710 con 0x7f4ae009e000 2023-09-30T22:23:21.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.416+0000 7f4ad67fc700 1 -- 172.21.15.89:0/2642111163 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ad0021070 con 0x7f4ae009e000 2023-09-30T22:23:21.416 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:21.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.417+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 msgr2=0x7f4ae01c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.417+0000 7f4ae7f3c700 1 --2- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01c5560 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000e8d0 tx=0x7f4ad000ebe0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.418+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 shutdown_connections 2023-09-30T22:23:21.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.418+0000 7f4ae7f3c700 1 --2- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ae009e410 0x7f4ae01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.418+0000 7f4ae7f3c700 1 --2- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ae009d6f0 0x7f4ae01c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.418+0000 7f4ae7f3c700 1 --2- 172.21.15.89:0/2642111163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ae009e000 0x7f4ae01c5560 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.418+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 >> 172.21.15.89:0/2642111163 conn(0x7f4ae0066f50 msgr2=0x7f4ae01bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:21.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.418+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 shutdown_connections 2023-09-30T22:23:21.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.418+0000 7f4ae7f3c700 1 -- 172.21.15.89:0/2642111163 wait complete. 2023-09-30T22:23:21.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.434+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f502811dc10 con 0x7f5028134ec0 2023-09-30T22:23:21.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.434+0000 7f501effd700 1 -- 172.21.15.89:0/3132053960 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5018015070 con 0x7f5028134ec0 2023-09-30T22:23:21.434 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 msgr2=0x7f50281cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281cd290 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f5018000c00 tx=0x7f5018004750 comp rx=0 tx=0).stop 2023-09-30T22:23:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 shutdown_connections 2023-09-30T22:23:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5028137f20 0x7f502811a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f502806c0e0 0x7f50281ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 --2- 172.21.15.89:0/3132053960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5028134ec0 0x7f50281cd290 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 >> 172.21.15.89:0/3132053960 conn(0x7f50280a4210 msgr2=0x7f5028099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 shutdown_connections 2023-09-30T22:23:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:21.436+0000 7f50307e1700 1 -- 172.21.15.89:0/3132053960 wait complete. 2023-09-30T22:23:21.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:22.360 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:22.367 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:22.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:22.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:22.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.504+0000 7fc607530700 1 Processor -- start 2023-09-30T22:23:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.504+0000 7fc607530700 1 -- start start 2023-09-30T22:23:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc607530700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012cf60 0x7fc60006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc607530700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc60012f8b0 0x7fc60012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc607530700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc600130260 0x7fc600136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc607530700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60012d330 con 0x7fc600130260 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc607530700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60012cd90 con 0x7fc60012f8b0 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc607530700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001306d0 con 0x7fc60012cf60 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc6052cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012cf60 0x7fc60006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc604acb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc60012f8b0 0x7fc60012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc6052cc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012cf60 0x7fc60006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47376/0 (socket says 172.21.15.89:47376) 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc6052cc700 1 -- 172.21.15.89:0/1233975292 learned_addr learned my addr 172.21.15.89:0/1233975292 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.505+0000 7fc605acd700 1 --2- 172.21.15.89:0/1233975292 >> v2:172.21.15.89:3300/0 conn(0x7fc600130260 0x7fc600136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc6052cc700 1 -- 172.21.15.89:0/1233975292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc60012f8b0 msgr2=0x7fc60012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc6052cc700 1 --2- 172.21.15.89:0/1233975292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc60012f8b0 0x7fc60012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc6052cc700 1 -- 172.21.15.89:0/1233975292 >> v2:172.21.15.89:3300/0 conn(0x7fc600130260 msgr2=0x7fc600136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc6052cc700 1 --2- 172.21.15.89:0/1233975292 >> v2:172.21.15.89:3300/0 conn(0x7fc600130260 0x7fc600136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc6052cc700 1 -- 172.21.15.89:0/1233975292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600071780 con 0x7fc60012cf60 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc604acb700 1 --2- 172.21.15.89:0/1233975292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc60012f8b0 0x7fc60012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc6052cc700 1 --2- 172.21.15.89:0/1233975292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012cf60 0x7fc60006f5d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00bb90 tx=0x7fc5fc00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f4a67e67ec3de9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc5f7fff700 1 -- 172.21.15.89:0/1233975292 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc00dc20 con 0x7fc60012cf60 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc5f7fff700 1 -- 172.21.15.89:0/1233975292 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc00dd80 con 0x7fc60012cf60 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.506+0000 7fc5f7fff700 1 -- 172.21.15.89:0/1233975292 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc017660 con 0x7fc60012cf60 2023-09-30T22:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 -- 172.21.15.89:0/1233975292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012cf60 msgr2=0x7fc60006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 --2- 172.21.15.89:0/1233975292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012cf60 0x7fc60006f5d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00bb90 tx=0x7fc5fc00bea0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 -- 172.21.15.89:0/1233975292 shutdown_connections 2023-09-30T22:23:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 --2- 172.21.15.89:0/1233975292 >> v2:172.21.15.89:3300/0 conn(0x7fc600130260 0x7fc600136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 --2- 172.21.15.89:0/1233975292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012cf60 0x7fc60006f5d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 --2- 172.21.15.89:0/1233975292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc60012f8b0 0x7fc60012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 -- 172.21.15.89:0/1233975292 >> 172.21.15.89:0/1233975292 conn(0x7fc6000a2aa0 msgr2=0x7fc6000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:22.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 -- 172.21.15.89:0/1233975292 shutdown_connections 2023-09-30T22:23:22.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.507+0000 7fc607530700 1 -- 172.21.15.89:0/1233975292 wait complete. 2023-09-30T22:23:22.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc607530700 1 Processor -- start 2023-09-30T22:23:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc607530700 1 -- start start 2023-09-30T22:23:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc607530700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60012cf60 0x7fc6001c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc607530700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012f8b0 0x7fc6001c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc607530700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc600130260 0x7fc6001c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc607530700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60009d320 con 0x7fc60012cf60 2023-09-30T22:23:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc607530700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60009ccd0 con 0x7fc600130260 2023-09-30T22:23:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc607530700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc60009e900 con 0x7fc60012f8b0 2023-09-30T22:23:22.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc6052cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60012cf60 0x7fc6001c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc6052cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60012cf60 0x7fc6001c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34978/0 (socket says 172.21.15.89:34978) 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.509+0000 7fc6052cc700 1 -- 172.21.15.89:0/3299215606 learned_addr learned my addr 172.21.15.89:0/3299215606 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc605acd700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc600130260 0x7fc6001c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc604acb700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012f8b0 0x7fc6001c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc6052cc700 1 -- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012f8b0 msgr2=0x7fc6001c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc6052cc700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012f8b0 0x7fc6001c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc6052cc700 1 -- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc600130260 msgr2=0x7fc6001c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc6052cc700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc600130260 0x7fc6001c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc6052cc700 1 -- 172.21.15.89:0/3299215606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00094a0 con 0x7fc60012cf60 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc604acb700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012f8b0 0x7fc6001c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:22.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc6052cc700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60012cf60 0x7fc6001c3e70 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc019040 tx=0x7fc5fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:22.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc5f5ffb700 1 -- 172.21.15.89:0/3299215606 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc003ab0 con 0x7fc60012cf60 2023-09-30T22:23:22.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc5f5ffb700 1 -- 172.21.15.89:0/3299215606 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc003c30 con 0x7fc60012cf60 2023-09-30T22:23:22.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc5f5ffb700 1 -- 172.21.15.89:0/3299215606 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0045a0 con 0x7fc60012cf60 2023-09-30T22:23:22.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5fc00b7f0 con 0x7fc60012cf60 2023-09-30T22:23:22.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.510+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc600129fa0 con 0x7fc60012cf60 2023-09-30T22:23:22.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.511+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc600074ea0 con 0x7fc60012cf60 2023-09-30T22:23:22.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.512+0000 7fc5f5ffb700 1 -- 172.21.15.89:0/3299215606 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5fc015070 con 0x7fc60012cf60 2023-09-30T22:23:22.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.512+0000 7fc5f5ffb700 1 -- 172.21.15.89:0/3299215606 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc5fc042e20 con 0x7fc60012cf60 2023-09-30T22:23:22.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.520+0000 7fc5f5ffb700 1 -- 172.21.15.89:0/3299215606 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5fc02a7d0 con 0x7fc60012cf60 2023-09-30T22:23:22.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.548+0000 7fb1bb311700 1 Processor -- start 2023-09-30T22:23:22.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.548+0000 7fb1bb311700 1 -- start start 2023-09-30T22:23:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.548+0000 7fb1bb311700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb1b4133420 0x7fb1b41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1bb311700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b41306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1bb311700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b406b990 con 0x7fb1b4133e30 2023-09-30T22:23:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1bb311700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b40a0590 con 0x7fb1b4133420 2023-09-30T22:23:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1bb311700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b409fb80 con 0x7fb1b4134260 2023-09-30T22:23:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b98ae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b41306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b88ac700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb1b4133e30 0x7fb1b412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40032/0 (socket says 172.21.15.89:40032) 2023-09-30T22:23:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b88ac700 1 -- 172.21.15.89:0/1333718156 learned_addr learned my addr 172.21.15.89:0/1333718156 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b90ad700 1 --2- 172.21.15.89:0/1333718156 >> v2:172.21.15.89:3300/0 conn(0x7fb1b4133420 0x7fb1b41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b98ae700 1 -- 172.21.15.89:0/1333718156 >> v2:172.21.15.89:3300/0 conn(0x7fb1b4133420 msgr2=0x7fb1b41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b98ae700 1 --2- 172.21.15.89:0/1333718156 >> v2:172.21.15.89:3300/0 conn(0x7fb1b4133420 0x7fb1b41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b98ae700 1 -- 172.21.15.89:0/1333718156 >> v1:172.21.15.89:6789/0 conn(0x7fb1b4133e30 legacy=0x7fb1b412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b98ae700 1 -- 172.21.15.89:0/1333718156 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40a0030 con 0x7fb1b4134260 2023-09-30T22:23:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1b98ae700 1 --2- 172.21.15.89:0/1333718156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b41306a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009d90 tx=0x7fb1ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa44694d59acebc0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.549+0000 7fb1abfff700 1 -- 172.21.15.89:0/1333718156 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00e070 con 0x7fb1b4134260 2023-09-30T22:23:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1abfff700 1 -- 172.21.15.89:0/1333718156 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac00bb20 con 0x7fb1b4134260 2023-09-30T22:23:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1abfff700 1 -- 172.21.15.89:0/1333718156 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac012430 con 0x7fb1b4134260 2023-09-30T22:23:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1bb311700 1 -- 172.21.15.89:0/1333718156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 msgr2=0x7fb1b41306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1bb311700 1 --2- 172.21.15.89:0/1333718156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b41306a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009d90 tx=0x7fb1ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1bb311700 1 -- 172.21.15.89:0/1333718156 shutdown_connections 2023-09-30T22:23:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1bb311700 1 --2- 172.21.15.89:0/1333718156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b41306a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1bb311700 1 --2- 172.21.15.89:0/1333718156 >> v2:172.21.15.89:3300/0 conn(0x7fb1b4133420 0x7fb1b41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1bb311700 1 -- 172.21.15.89:0/1333718156 >> 172.21.15.89:0/1333718156 conn(0x7fb1b40a4210 msgr2=0x7fb1b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1bb311700 1 -- 172.21.15.89:0/1333718156 shutdown_connections 2023-09-30T22:23:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.550+0000 7fb1bb311700 1 -- 172.21.15.89:0/1333718156 wait complete. 2023-09-30T22:23:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.552+0000 7fb1bb311700 1 Processor -- start 2023-09-30T22:23:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.552+0000 7fb1bb311700 1 -- start start 2023-09-30T22:23:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.552+0000 7fb1bb311700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b4133420 0x7fb1b409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.552+0000 7fb1bb311700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1b4133e30 0x7fb1b409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.552+0000 7fb1bb311700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.552+0000 7fb1bb311700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b411fc20 con 0x7fb1b4133420 2023-09-30T22:23:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.552+0000 7fb1bb311700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b411fee0 con 0x7fb1b4134260 2023-09-30T22:23:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.552+0000 7fb1bb311700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b411f910 con 0x7fb1b4133e30 2023-09-30T22:23:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b98ae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b90ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b4133420 0x7fb1b409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b98ae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60470/0 (socket says 172.21.15.89:60470) 2023-09-30T22:23:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b98ae700 1 -- 172.21.15.89:0/389333830 learned_addr learned my addr 172.21.15.89:0/389333830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b88ac700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1b4133e30 0x7fb1b409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b88ac700 1 -- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 msgr2=0x7fb1b409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b88ac700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b88ac700 1 -- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b4133420 msgr2=0x7fb1b409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b88ac700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b4133420 0x7fb1b409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b88ac700 1 -- 172.21.15.89:0/389333830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac009a70 con 0x7fb1b4133e30 2023-09-30T22:23:22.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b90ad700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b4133420 0x7fb1b409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:22.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1b88ac700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1b4133e30 0x7fb1b409c9c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400c500 tx=0x7fb1a400c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:22.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1a9ffb700 1 -- 172.21.15.89:0/389333830 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4005d70 con 0x7fb1b4133e30 2023-09-30T22:23:22.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1a9ffb700 1 -- 172.21.15.89:0/389333830 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a40070d0 con 0x7fb1b4133e30 2023-09-30T22:23:22.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b4120950 con 0x7fb1b4133e30 2023-09-30T22:23:22.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.553+0000 7fb1a9ffb700 1 -- 172.21.15.89:0/389333830 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4007440 con 0x7fb1b4133e30 2023-09-30T22:23:22.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.554+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b412dab0 con 0x7fb1b4133e30 2023-09-30T22:23:22.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.554+0000 7fb1a9ffb700 1 -- 172.21.15.89:0/389333830 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb1a40075a0 con 0x7fb1b4133e30 2023-09-30T22:23:22.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.554+0000 7fb1a9ffb700 1 -- 172.21.15.89:0/389333830 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb1a4040a60 con 0x7fb1b4133e30 2023-09-30T22:23:22.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.555+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b4074ea0 con 0x7fb1b4133e30 2023-09-30T22:23:22.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.560+0000 7fb1a9ffb700 1 -- 172.21.15.89:0/389333830 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb1a4017070 con 0x7fb1b4133e30 2023-09-30T22:23:22.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a9657700 1 Processor -- start 2023-09-30T22:23:22.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a9657700 1 -- start start 2023-09-30T22:23:22.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a9657700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a9657700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79a40a1340 0x7f79a4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a9657700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a9657700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a409edd0 con 0x7f79a40a1340 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a9657700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a409e1f0 con 0x7f79a4133fa0 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a9657700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79a409dc50 con 0x7f79a40a0a30 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a27fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79a40a1340 0x7f79a4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47390/0 (socket says 172.21.15.89:47390) 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a27fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79a40a1340 0x7f79a4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35014/0 (socket says 172.21.15.89:35014) 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a4134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60476/0 (socket says 172.21.15.89:60476) 2023-09-30T22:23:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.581+0000 7f79a2ffd700 1 -- 172.21.15.89:0/1514355676 learned_addr learned my addr 172.21.15.89:0/1514355676 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:22.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a27fc700 1 -- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 msgr2=0x7f79a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:22.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a27fc700 1 --2- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a27fc700 1 -- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 msgr2=0x7f79a4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:22.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a27fc700 1 --2- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a27fc700 1 -- 172.21.15.89:0/1514355676 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40a1fe0 con 0x7f79a40a1340 2023-09-30T22:23:22.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a37fe700 1 --2- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a27fc700 1 --2- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a4133a60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f7998009840 tx=0x7f7998009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b2302e9e2d153e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a1ffb700 1 -- 172.21.15.89:0/1514355676 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998011830 con 0x7f79a40a1340 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a1ffb700 1 -- 172.21.15.89:0/1514355676 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7998011990 con 0x7f79a40a1340 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.582+0000 7f79a1ffb700 1 -- 172.21.15.89:0/1514355676 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998011d00 con 0x7f79a40a1340 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 -- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 msgr2=0x7f79a4133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 --2- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a4133a60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f7998009840 tx=0x7f7998009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 -- 172.21.15.89:0/1514355676 shutdown_connections 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 --2- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 --2- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 --2- 172.21.15.89:0/1514355676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a4133a60 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 -- 172.21.15.89:0/1514355676 >> 172.21.15.89:0/1514355676 conn(0x7f79a40a6550 msgr2=0x7f79a40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:22.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 -- 172.21.15.89:0/1514355676 shutdown_connections 2023-09-30T22:23:22.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.583+0000 7f79a9657700 1 -- 172.21.15.89:0/1514355676 wait complete. 2023-09-30T22:23:22.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.584+0000 7f79a9657700 1 Processor -- start 2023-09-30T22:23:22.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.584+0000 7f79a9657700 1 -- start start 2023-09-30T22:23:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a9657700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a411e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a9657700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a411e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a9657700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a411ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a9657700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4120280 con 0x7f79a40a1340 2023-09-30T22:23:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a9657700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4121340 con 0x7f79a4133fa0 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a9657700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79a40a0220 con 0x7f79a40a0a30 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a411e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a411e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35022/0 (socket says 172.21.15.89:35022) 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a27fc700 1 -- 172.21.15.89:0/2609281498 learned_addr learned my addr 172.21.15.89:0/2609281498 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.585+0000 7f79a2ffd700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a411e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a37fe700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a411ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a27fc700 1 -- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 msgr2=0x7f79a411e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a27fc700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a411e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a27fc700 1 -- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 msgr2=0x7f79a411ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a27fc700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a411ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a27fc700 1 -- 172.21.15.89:0/2609281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c009a70 con 0x7f79a40a1340 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a37fe700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a411ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a27fc700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a411e670 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7998009fd0 tx=0x7f7998004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f798b7fe700 1 -- 172.21.15.89:0/2609281498 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79980036a0 con 0x7f79a40a1340 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79980094a0 con 0x7f79a40a1340 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f798b7fe700 1 -- 172.21.15.89:0/2609281498 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799801f070 con 0x7f79a40a1340 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f798b7fe700 1 -- 172.21.15.89:0/2609281498 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998003a60 con 0x7f79a40a1340 2023-09-30T22:23:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.586+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a411fca0 con 0x7f79a40a1340 2023-09-30T22:23:22.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.588+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a4121570 con 0x7f79a40a1340 2023-09-30T22:23:22.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.588+0000 7f798b7fe700 1 -- 172.21.15.89:0/2609281498 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7998003bc0 con 0x7f79a40a1340 2023-09-30T22:23:22.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.590+0000 7f798b7fe700 1 -- 172.21.15.89:0/2609281498 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7998021070 con 0x7f79a40a1340 2023-09-30T22:23:22.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.596+0000 7fca2ce2d700 1 Processor -- start 2023-09-30T22:23:22.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.596+0000 7fca2ce2d700 1 -- start start 2023-09-30T22:23:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.596+0000 7fca2ce2d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca28130540 0x7fca28130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.596+0000 7fca2ce2d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca2806fcf0 0x7fca28138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.596+0000 7fca2ce2d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca280716a0 con 0x7fca28130e50 2023-09-30T22:23:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.596+0000 7fca2ce2d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2809a390 con 0x7fca2806fcf0 2023-09-30T22:23:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.596+0000 7fca2ce2d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca28071290 con 0x7fca28130540 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca2659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca28130540 0x7fca28130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca2659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca28130540 0x7fca28130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47410/0 (socket says 172.21.15.89:47410) 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca2659c700 1 -- 172.21.15.89:0/2174153483 learned_addr learned my addr 172.21.15.89:0/2174153483 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca26d9d700 1 --2- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca2806fcf0 0x7fca28138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca26d9d700 1 -- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca28130540 msgr2=0x7fca28130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca26d9d700 1 --2- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca28130540 0x7fca28130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca26d9d700 1 -- 172.21.15.89:0/2174153483 >> v1:172.21.15.89:6789/0 conn(0x7fca28130e50 legacy=0x7fca2806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca26d9d700 1 -- 172.21.15.89:0/2174153483 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca28130340 con 0x7fca2806fcf0 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca2659c700 1 --2- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca28130540 0x7fca28130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca26d9d700 1 --2- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca2806fcf0 0x7fca28138570 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fca20009df0 tx=0x7fca2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ceb9ff4cf1e68a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca2559a700 1 -- 172.21.15.89:0/2174153483 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000e070 con 0x7fca2806fcf0 2023-09-30T22:23:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca2559a700 1 -- 172.21.15.89:0/2174153483 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000bae0 con 0x7fca2806fcf0 2023-09-30T22:23:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7fca2559a700 1 -- 172.21.15.89:0/2174153483 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000be90 con 0x7fca2806fcf0 2023-09-30T22:23:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.597+0000 7f798b7fe700 1 -- 172.21.15.89:0/2609281498 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79a4121570 con 0x7f79a40a1340 2023-09-30T22:23:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.598+0000 7fca2ce2d700 1 -- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca2806fcf0 msgr2=0x7fca28138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.598+0000 7fca2ce2d700 1 --2- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca2806fcf0 0x7fca28138570 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fca20009df0 tx=0x7fca2000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.598+0000 7fca2ce2d700 1 -- 172.21.15.89:0/2174153483 shutdown_connections 2023-09-30T22:23:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.598+0000 7fca2ce2d700 1 --2- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca2806fcf0 0x7fca28138570 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.598+0000 7fca2ce2d700 1 --2- 172.21.15.89:0/2174153483 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca28130540 0x7fca28130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.598+0000 7fca2ce2d700 1 -- 172.21.15.89:0/2174153483 >> 172.21.15.89:0/2174153483 conn(0x7fca280a2aa0 msgr2=0x7fca280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.598+0000 7fca2ce2d700 1 -- 172.21.15.89:0/2174153483 shutdown_connections 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.598+0000 7fca2ce2d700 1 -- 172.21.15.89:0/2174153483 wait complete. 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2ce2d700 1 Processor -- start 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2ce2d700 1 -- start start 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2ce2d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca2806fcf0 0x7fca2809cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2ce2d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca28130540 0x7fca2809d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2ce2d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca28130e50 0x7fca2809d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2ce2d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca28119940 con 0x7fca28130e50 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2ce2d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca28129d50 con 0x7fca28130540 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2ce2d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca281191f0 con 0x7fca2806fcf0 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca25d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca28130540 0x7fca2809d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca25d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca28130540 0x7fca2809d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60500/0 (socket says 172.21.15.89:60500) 2023-09-30T22:23:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca25d9b700 1 -- 172.21.15.89:0/1052345950 learned_addr learned my addr 172.21.15.89:0/1052345950 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.600+0000 7fca2659c700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca2806fcf0 0x7fca2809cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca25d9b700 1 -- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca2806fcf0 msgr2=0x7fca2809cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca25d9b700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca2806fcf0 0x7fca2809cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca25d9b700 1 -- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca28130e50 msgr2=0x7fca2809d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca25d9b700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca28130e50 0x7fca2809d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca25d9b700 1 -- 172.21.15.89:0/1052345950 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1800d9f0 con 0x7fca28130540 2023-09-30T22:23:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca2659c700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca2806fcf0 0x7fca2809cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca25d9b700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca28130540 0x7fca2809d460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00c4b0 tx=0x7fca1c00c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca16ffd700 1 -- 172.21.15.89:0/1052345950 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0052d0 con 0x7fca28130540 2023-09-30T22:23:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca16ffd700 1 -- 172.21.15.89:0/1052345950 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c005430 con 0x7fca28130540 2023-09-30T22:23:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca20009a70 con 0x7fca28130540 2023-09-30T22:23:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca16ffd700 1 -- 172.21.15.89:0/1052345950 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0038c0 con 0x7fca28130540 2023-09-30T22:23:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.601+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca280a0b50 con 0x7fca28130540 2023-09-30T22:23:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.602+0000 7fca16ffd700 1 -- 172.21.15.89:0/1052345950 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca1c003a20 con 0x7fca28130540 2023-09-30T22:23:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.602+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2812f8d0 con 0x7fca28130540 2023-09-30T22:23:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.603+0000 7fca16ffd700 1 -- 172.21.15.89:0/1052345950 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fca1c03ead0 con 0x7fca28130540 2023-09-30T22:23:22.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.611+0000 7fca16ffd700 1 -- 172.21.15.89:0/1052345950 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca1c017020 con 0x7fca28130540 2023-09-30T22:23:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.719+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc6000713a0 con 0x7fc60012cf60 2023-09-30T22:23:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.719+0000 7fc5f5ffb700 1 -- 172.21.15.89:0/3299215606 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5fc02a3c0 con 0x7fc60012cf60 2023-09-30T22:23:22.719 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60012cf60 msgr2=0x7fc6001c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60012cf60 0x7fc6001c3e70 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc019040 tx=0x7fc5fc004750 comp rx=0 tx=0).stop 2023-09-30T22:23:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 shutdown_connections 2023-09-30T22:23:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc600130260 0x7fc6001c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60012cf60 0x7fc6001c3e70 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 --2- 172.21.15.89:0/3299215606 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60012f8b0 0x7fc6001c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 >> 172.21.15.89:0/3299215606 conn(0x7fc6000a2aa0 msgr2=0x7fc6000a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 shutdown_connections 2023-09-30T22:23:22.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.722+0000 7fc607530700 1 -- 172.21.15.89:0/3299215606 wait complete. 2023-09-30T22:23:22.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:22.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.764+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb1b406b250 con 0x7fb1b4133e30 2023-09-30T22:23:22.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.764+0000 7fb1a9ffb700 1 -- 172.21.15.89:0/389333830 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1a4017020 con 0x7fb1b4133e30 2023-09-30T22:23:22.764 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:22.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1b4133e30 msgr2=0x7fb1b409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1b4133e30 0x7fb1b409c9c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400c500 tx=0x7fb1a400c810 comp rx=0 tx=0).stop 2023-09-30T22:23:22.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 shutdown_connections 2023-09-30T22:23:22.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1b4134260 0x7fb1b409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b4133420 0x7fb1b409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 --2- 172.21.15.89:0/389333830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1b4133e30 0x7fb1b409c9c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 >> 172.21.15.89:0/389333830 conn(0x7fb1b40a4210 msgr2=0x7fb1b40a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:22.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 shutdown_connections 2023-09-30T22:23:22.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.766+0000 7fb1bb311700 1 -- 172.21.15.89:0/389333830 wait complete. 2023-09-30T22:23:22.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.797+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f79a4121570 con 0x7f79a40a1340 2023-09-30T22:23:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.797+0000 7f798b7fe700 1 -- 172.21.15.89:0/2609281498 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79a4121570 con 0x7f79a40a1340 2023-09-30T22:23:22.797 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 msgr2=0x7f79a411e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a411e670 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7998009fd0 tx=0x7f7998004750 comp rx=0 tx=0).stop 2023-09-30T22:23:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 shutdown_connections 2023-09-30T22:23:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79a4133fa0 0x7f79a411ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79a40a0a30 0x7f79a411e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 --2- 172.21.15.89:0/2609281498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79a40a1340 0x7f79a411e670 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 >> 172.21.15.89:0/2609281498 conn(0x7f79a40a6550 msgr2=0x7f79a41330f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 shutdown_connections 2023-09-30T22:23:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.800+0000 7f79a9657700 1 -- 172.21.15.89:0/2609281498 wait complete. 2023-09-30T22:23:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.809+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fca28074ea0 con 0x7fca28130540 2023-09-30T22:23:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.809+0000 7fca16ffd700 1 -- 172.21.15.89:0/1052345950 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca1c018260 con 0x7fca28130540 2023-09-30T22:23:22.809 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca28130540 msgr2=0x7fca2809d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca28130540 0x7fca2809d460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00c4b0 tx=0x7fca1c00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 shutdown_connections 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca28130e50 0x7fca2809d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca2806fcf0 0x7fca2809cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 --2- 172.21.15.89:0/1052345950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca28130540 0x7fca2809d460 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 >> 172.21.15.89:0/1052345950 conn(0x7fca280a2aa0 msgr2=0x7fca280a0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 shutdown_connections 2023-09-30T22:23:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:22.812+0000 7fca2ce2d700 1 -- 172.21.15.89:0/1052345950 wait complete. 2023-09-30T22:23:22.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:22.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:23.735 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:23.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:23.814 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:23.824 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.878+0000 7fd489bf8700 1 Processor -- start 2023-09-30T22:23:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.878+0000 7fd489bf8700 1 -- start start 2023-09-30T22:23:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.878+0000 7fd489bf8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd48409d6f0 0x7fd48409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.878+0000 7fd489bf8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409e410 0x7fd484130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd489bf8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48409f130 con 0x7fd48409e000 2023-09-30T22:23:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd489bf8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48409f520 con 0x7fd48409d6f0 2023-09-30T22:23:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd489bf8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd48409e930 con 0x7fd48409e410 2023-09-30T22:23:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4837fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd48409d6f0 0x7fd48409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4837fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd48409d6f0 0x7fd48409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35052/0 (socket says 172.21.15.89:35052) 2023-09-30T22:23:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4837fe700 1 -- 172.21.15.89:0/1246320022 learned_addr learned my addr 172.21.15.89:0/1246320022 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd483fff700 1 --2- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409e410 0x7fd484130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4837fe700 1 -- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409e410 msgr2=0x7fd484130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4837fe700 1 --2- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409e410 0x7fd484130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4837fe700 1 -- 172.21.15.89:0/1246320022 >> v1:172.21.15.89:6789/0 conn(0x7fd48409e000 legacy=0x7fd484130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4827fc700 1 -- 172.21.15.89:0/1246320022 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412187645 0 0) 0x7fd48409f130 con 0x7fd48409e000 2023-09-30T22:23:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4837fe700 1 -- 172.21.15.89:0/1246320022 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48409b2d0 con 0x7fd48409d6f0 2023-09-30T22:23:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd483fff700 1 --2- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409e410 0x7fd484130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.879+0000 7fd4837fe700 1 --2- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409d6f0 0x7fd48409dac0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098f0 tx=0x7fd474009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eb497eba9fa756 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd4827fc700 1 -- 172.21.15.89:0/1246320022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474011830 con 0x7fd48409d6f0 2023-09-30T22:23:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd4827fc700 1 -- 172.21.15.89:0/1246320022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474011990 con 0x7fd48409d6f0 2023-09-30T22:23:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd4827fc700 1 -- 172.21.15.89:0/1246320022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474011d20 con 0x7fd48409d6f0 2023-09-30T22:23:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd489bf8700 1 -- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409d6f0 msgr2=0x7fd48409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd489bf8700 1 --2- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409d6f0 0x7fd48409dac0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098f0 tx=0x7fd474009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd489bf8700 1 -- 172.21.15.89:0/1246320022 shutdown_connections 2023-09-30T22:23:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd489bf8700 1 --2- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409d6f0 0x7fd48409dac0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd489bf8700 1 --2- 172.21.15.89:0/1246320022 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409e410 0x7fd484130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.880+0000 7fd489bf8700 1 -- 172.21.15.89:0/1246320022 >> 172.21.15.89:0/1246320022 conn(0x7fd484066f50 msgr2=0x7fd48409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.881+0000 7fd489bf8700 1 -- 172.21.15.89:0/1246320022 shutdown_connections 2023-09-30T22:23:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.881+0000 7fd489bf8700 1 -- 172.21.15.89:0/1246320022 wait complete. 2023-09-30T22:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.881+0000 7fd489bf8700 1 Processor -- start 2023-09-30T22:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd489bf8700 1 -- start start 2023-09-30T22:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd489bf8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409d6f0 0x7fd484139150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd489bf8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd48409e000 0x7fd484139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd489bf8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409e410 0x7fd484139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd489bf8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4841372f0 con 0x7fd48409e410 2023-09-30T22:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd489bf8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48413abe0 con 0x7fd48409e000 2023-09-30T22:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd489bf8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd48409c770 con 0x7fd48409d6f0 2023-09-30T22:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd4837fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409d6f0 0x7fd484139150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd4837fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409d6f0 0x7fd484139150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47440/0 (socket says 172.21.15.89:47440) 2023-09-30T22:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd4837fe700 1 -- 172.21.15.89:0/580293581 learned_addr learned my addr 172.21.15.89:0/580293581 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.882+0000 7fd483fff700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409e410 0x7fd484139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd4837fe700 1 -- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd48409e000 msgr2=0x7fd484139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd4837fe700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd48409e000 0x7fd484139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd4837fe700 1 -- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409e410 msgr2=0x7fd484139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd4837fe700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409e410 0x7fd484139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd4837fe700 1 -- 172.21.15.89:0/580293581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c009a70 con 0x7fd48409d6f0 2023-09-30T22:23:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd483fff700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409e410 0x7fd484139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd4837fe700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409d6f0 0x7fd484139150 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd4840a3e80 tx=0x7fd474004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd46bfff700 1 -- 172.21.15.89:0/580293581 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474004470 con 0x7fd48409d6f0 2023-09-30T22:23:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd46bfff700 1 -- 172.21.15.89:0/580293581 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47401f070 con 0x7fd48409d6f0 2023-09-30T22:23:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd46bfff700 1 -- 172.21.15.89:0/580293581 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4740038e0 con 0x7fd48409d6f0 2023-09-30T22:23:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd474009550 con 0x7fd48409d6f0 2023-09-30T22:23:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.883+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd484136390 con 0x7fd48409d6f0 2023-09-30T22:23:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.884+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484074ea0 con 0x7fd48409d6f0 2023-09-30T22:23:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.884+0000 7fd46bfff700 1 -- 172.21.15.89:0/580293581 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd474021030 con 0x7fd48409d6f0 2023-09-30T22:23:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.885+0000 7fd46bfff700 1 -- 172.21.15.89:0/580293581 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd474040af0 con 0x7fd48409d6f0 2023-09-30T22:23:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.889+0000 7fd46bfff700 1 -- 172.21.15.89:0/580293581 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4740199a0 con 0x7fd48409d6f0 2023-09-30T22:23:23.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.926+0000 7fe0586e1700 1 Processor -- start 2023-09-30T22:23:23.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.926+0000 7fe0586e1700 1 -- start start 2023-09-30T22:23:23.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe0586e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009d630 0x7fe0500a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe0586e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe050138d80 0x7fe05013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe0586e1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0500a2a70 con 0x7fe0500a0a30 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe0586e1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0500a01d0 con 0x7fe050138d80 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe0586e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe05009e2b0 con 0x7fe05009d630 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe056c7e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe050138d80 0x7fe05013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe056c7e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe050138d80 0x7fe05013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35066/0 (socket says 172.21.15.89:35066) 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe05647d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe0500a0a30 0x7fe0500a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40062/0 (socket says 172.21.15.89:40062) 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe056c7e700 1 -- 172.21.15.89:0/3422287618 learned_addr learned my addr 172.21.15.89:0/3422287618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe055c7c700 1 --2- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009d630 0x7fe0500a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe056c7e700 1 -- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009d630 msgr2=0x7fe0500a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe056c7e700 1 --2- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009d630 0x7fe0500a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe056c7e700 1 -- 172.21.15.89:0/3422287618 >> v1:172.21.15.89:6789/0 conn(0x7fe0500a0a30 legacy=0x7fe0500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe056c7e700 1 -- 172.21.15.89:0/3422287618 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05009dc50 con 0x7fe050138d80 2023-09-30T22:23:23.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.927+0000 7fe05547b700 1 -- 172.21.15.89:0/3422287618 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265891675 0 0) 0x7fe0500a2a70 con 0x7fe0500a0a30 2023-09-30T22:23:23.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.928+0000 7fe055c7c700 1 --2- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009d630 0x7fe0500a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:23.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.928+0000 7fe056c7e700 1 --2- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe050138d80 0x7fe05013b160 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe048009e50 tx=0x7fe04800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d0e13ac5a4b353 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:23.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.928+0000 7fe05547b700 1 -- 172.21.15.89:0/3422287618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800e070 con 0x7fe050138d80 2023-09-30T22:23:23.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.928+0000 7fe05547b700 1 -- 172.21.15.89:0/3422287618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04800b910 con 0x7fe050138d80 2023-09-30T22:23:23.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.928+0000 7fe05547b700 1 -- 172.21.15.89:0/3422287618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800bcc0 con 0x7fe050138d80 2023-09-30T22:23:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.929+0000 7fe0586e1700 1 -- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe050138d80 msgr2=0x7fe05013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.929+0000 7fe0586e1700 1 --2- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe050138d80 0x7fe05013b160 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe048009e50 tx=0x7fe04800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.929+0000 7fe0586e1700 1 -- 172.21.15.89:0/3422287618 shutdown_connections 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.929+0000 7fe0586e1700 1 --2- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe050138d80 0x7fe05013b160 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.929+0000 7fe0586e1700 1 --2- 172.21.15.89:0/3422287618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe05009d630 0x7fe0500a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.929+0000 7fe0586e1700 1 -- 172.21.15.89:0/3422287618 >> 172.21.15.89:0/3422287618 conn(0x7fe0500a3050 msgr2=0x7fe0500a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.929+0000 7fe0586e1700 1 -- 172.21.15.89:0/3422287618 shutdown_connections 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.929+0000 7fe0586e1700 1 -- 172.21.15.89:0/3422287618 wait complete. 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.930+0000 7fe0586e1700 1 Processor -- start 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.930+0000 7fe0586e1700 1 -- start start 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe0586e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05009d630 0x7fe05011c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe0586e1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe0500a0a30 0x7fe05011cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe0586e1700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe050138d80 0x7fe05011d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe0586e1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05011f190 con 0x7fe05009d630 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe0586e1700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05009d390 con 0x7fe050138d80 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe0586e1700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe05011fde0 con 0x7fe0500a0a30 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe055c7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe0500a0a30 0x7fe05011cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe055c7c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe0500a0a30 0x7fe05011cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47472/0 (socket says 172.21.15.89:47472) 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe055c7c700 1 -- 172.21.15.89:0/2572943584 learned_addr learned my addr 172.21.15.89:0/2572943584 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe05647d700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05009d630 0x7fe05011c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe056c7e700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe050138d80 0x7fe05011d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe055c7c700 1 -- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe050138d80 msgr2=0x7fe05011d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe055c7c700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe050138d80 0x7fe05011d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe055c7c700 1 -- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05009d630 msgr2=0x7fe05011c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe055c7c700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05009d630 0x7fe05011c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe055c7c700 1 -- 172.21.15.89:0/2572943584 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0400094a0 con 0x7fe0500a0a30 2023-09-30T22:23:23.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.931+0000 7fe05647d700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05009d630 0x7fe05011c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:23.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.932+0000 7fe055c7c700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe0500a0a30 0x7fe05011cd00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe04000ebe0 tx=0x7fe04000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:23.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.932+0000 7fe046ffd700 1 -- 172.21.15.89:0/2572943584 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04000c7f0 con 0x7fe0500a0a30 2023-09-30T22:23:23.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.932+0000 7fe046ffd700 1 -- 172.21.15.89:0/2572943584 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04000c950 con 0x7fe0500a0a30 2023-09-30T22:23:23.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.932+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe048009b30 con 0x7fe0500a0a30 2023-09-30T22:23:23.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.932+0000 7fe046ffd700 1 -- 172.21.15.89:0/2572943584 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04000cce0 con 0x7fe0500a0a30 2023-09-30T22:23:23.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.932+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05011fb00 con 0x7fe0500a0a30 2023-09-30T22:23:23.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.933+0000 7fe046ffd700 1 -- 172.21.15.89:0/2572943584 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe040013470 con 0x7fe0500a0a30 2023-09-30T22:23:23.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.933+0000 7fe046ffd700 1 -- 172.21.15.89:0/2572943584 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe04003f7d0 con 0x7fe0500a0a30 2023-09-30T22:23:23.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.933+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0500747f0 con 0x7fe0500a0a30 2023-09-30T22:23:23.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.938+0000 7fe046ffd700 1 -- 172.21.15.89:0/2572943584 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe040013720 con 0x7fe0500a0a30 2023-09-30T22:23:23.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.958+0000 7f1123865700 1 Processor -- start 2023-09-30T22:23:23.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.958+0000 7f61dbe64700 1 Processor -- start 2023-09-30T22:23:23.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f61dbe64700 1 -- start start 2023-09-30T22:23:23.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.958+0000 7f1123865700 1 -- start start 2023-09-30T22:23:23.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f1123865700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c0a0a30 0x7f111c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f1123865700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f111c09d630 0x7f111c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f1123865700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c133930 0x7f111c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f1123865700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c0a2a70 con 0x7f111c09d630 2023-09-30T22:23:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f1123865700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0a01d0 con 0x7f111c133930 2023-09-30T22:23:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f1123865700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111c09e2b0 con 0x7f111c0a0a30 2023-09-30T22:23:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f61dbe64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409d6f0 0x7f61d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f61dbe64700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f61d409e410 0x7f61d4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f61dbe64700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d409b120 con 0x7f61d409e000 2023-09-30T22:23:23.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f61dbe64700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d406b460 con 0x7f61d409e410 2023-09-30T22:23:23.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.959+0000 7f61dbe64700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d409a7e0 con 0x7f61d409d6f0 2023-09-30T22:23:23.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f111c09d630 0x7f111c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f111c09d630 0x7f111c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35074/0 (socket says 172.21.15.89:35074) 2023-09-30T22:23:23.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 -- 172.21.15.89:0/131961998 learned_addr learned my addr 172.21.15.89:0/131961998 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:23.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1121e02700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c133930 0x7f111c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d93ff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61d409e000 0x7f61d41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40076/0 (socket says 172.21.15.89:40076) 2023-09-30T22:23:23.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d93ff700 1 -- 172.21.15.89:0/1205337479 learned_addr learned my addr 172.21.15.89:0/1205337479 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:23.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1121601700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c0a0a30 0x7f111c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 -- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c0a0a30 msgr2=0x7f111c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:23.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c0a0a30 0x7f111c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 -- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c133930 msgr2=0x7f111c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:23.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c133930 0x7f111c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 -- 172.21.15.89:0/131961998 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c09dc50 con 0x7f111c09d630 2023-09-30T22:23:23.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61da401700 1 --2- 172.21.15.89:0/1205337479 >> v2:172.21.15.89:3300/0 conn(0x7f61d409e410 0x7f61d4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d9c00700 1 --2- 172.21.15.89:0/1205337479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409d6f0 0x7f61d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1121e02700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c133930 0x7f111c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:23.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d8bfe700 1 -- 172.21.15.89:0/1205337479 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198105105 0 0) 0x7f61d409b120 con 0x7f61d409e000 2023-09-30T22:23:23.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d8bfe700 1 -- 172.21.15.89:0/1205337479 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c00035b0 con 0x7f61d409e000 2023-09-30T22:23:23.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d9c00700 1 -- 172.21.15.89:0/1205337479 >> v2:172.21.15.89:3300/0 conn(0x7f61d409e410 msgr2=0x7f61d4134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:23.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d9c00700 1 --2- 172.21.15.89:0/1205337479 >> v2:172.21.15.89:3300/0 conn(0x7f61d409e410 0x7f61d4134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d9c00700 1 -- 172.21.15.89:0/1205337479 >> v1:172.21.15.89:6789/0 conn(0x7f61d409e000 legacy=0x7f61d41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f61d9c00700 1 -- 172.21.15.89:0/1205337479 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d409a500 con 0x7f61d409d6f0 2023-09-30T22:23:23.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.960+0000 7f1120e00700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c09d630 0x7f111c0a1340 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f110c0098f0 tx=0x7f110c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f057af805847fdf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:23.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1113fff700 1 -- 172.21.15.89:0/131961998 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c011830 con 0x7f111c09d630 2023-09-30T22:23:23.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f61d9c00700 1 --2- 172.21.15.89:0/1205337479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409d6f0 0x7f61d409dac0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009870 tx=0x7f61d0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b102857777d5ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:23.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1113fff700 1 -- 172.21.15.89:0/131961998 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c011990 con 0x7f111c09d630 2023-09-30T22:23:23.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1113fff700 1 -- 172.21.15.89:0/131961998 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c011d20 con 0x7f111c09d630 2023-09-30T22:23:23.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f61d8bfe700 1 -- 172.21.15.89:0/1205337479 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0011830 con 0x7f61d409d6f0 2023-09-30T22:23:23.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f61d8bfe700 1 -- 172.21.15.89:0/1205337479 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0011990 con 0x7f61d409d6f0 2023-09-30T22:23:23.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f61d8bfe700 1 -- 172.21.15.89:0/1205337479 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0011d20 con 0x7f61d409d6f0 2023-09-30T22:23:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1123865700 1 -- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c09d630 msgr2=0x7f111c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1123865700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c09d630 0x7f111c0a1340 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f110c0098f0 tx=0x7f110c009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1123865700 1 -- 172.21.15.89:0/131961998 shutdown_connections 2023-09-30T22:23:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1123865700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c133930 0x7f111c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1123865700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c0a0a30 0x7f111c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1123865700 1 --2- 172.21.15.89:0/131961998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c09d630 0x7f111c0a1340 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.961+0000 7f1123865700 1 -- 172.21.15.89:0/131961998 >> 172.21.15.89:0/131961998 conn(0x7f111c0a3050 msgr2=0x7f111c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f61dbe64700 1 -- 172.21.15.89:0/1205337479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409d6f0 msgr2=0x7f61d409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f61dbe64700 1 --2- 172.21.15.89:0/1205337479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409d6f0 0x7f61d409dac0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009870 tx=0x7f61d0009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f61dbe64700 1 -- 172.21.15.89:0/1205337479 shutdown_connections 2023-09-30T22:23:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f61dbe64700 1 --2- 172.21.15.89:0/1205337479 >> v2:172.21.15.89:3300/0 conn(0x7f61d409e410 0x7f61d4134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f61dbe64700 1 --2- 172.21.15.89:0/1205337479 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409d6f0 0x7f61d409dac0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f61dbe64700 1 -- 172.21.15.89:0/1205337479 >> 172.21.15.89:0/1205337479 conn(0x7f61d4066f50 msgr2=0x7f61d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:23.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f1123865700 1 -- 172.21.15.89:0/131961998 shutdown_connections 2023-09-30T22:23:23.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f61dbe64700 1 -- 172.21.15.89:0/1205337479 shutdown_connections 2023-09-30T22:23:23.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f61dbe64700 1 -- 172.21.15.89:0/1205337479 wait complete. 2023-09-30T22:23:23.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.962+0000 7f1123865700 1 -- 172.21.15.89:0/131961998 wait complete. 2023-09-30T22:23:23.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1123865700 1 Processor -- start 2023-09-30T22:23:23.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f61dbe64700 1 Processor -- start 2023-09-30T22:23:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1123865700 1 -- start start 2023-09-30T22:23:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1123865700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c09d630 0x7f111c1c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1123865700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c0a0a30 0x7f111c1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1123865700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c133930 0x7f111c1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1123865700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0a0410 con 0x7f111c0a0a30 2023-09-30T22:23:23.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1123865700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c12f620 con 0x7f111c09d630 2023-09-30T22:23:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1123865700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111c130ef0 con 0x7f111c133930 2023-09-30T22:23:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1120e00700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c0a0a30 0x7f111c1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f61dbe64700 1 -- start start 2023-09-30T22:23:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f61dbe64700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61d409d6f0 0x7f61d41cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f61dbe64700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61d409e000 0x7f61d41194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f61dbe64700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409e410 0x7f61d41199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:23.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f61dbe64700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d411c280 con 0x7f61d409d6f0 2023-09-30T22:23:23.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f61dbe64700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d411a910 con 0x7f61d409e410 2023-09-30T22:23:23.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f61dbe64700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61d411d4d0 con 0x7f61d409e000 2023-09-30T22:23:23.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1120e00700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c0a0a30 0x7f111c1c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35082/0 (socket says 172.21.15.89:35082) 2023-09-30T22:23:23.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.963+0000 7f1120e00700 1 -- 172.21.15.89:0/463518091 learned_addr learned my addr 172.21.15.89:0/463518091 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1121e02700 1 --2- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c133930 0x7f111c1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1120e00700 1 -- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c133930 msgr2=0x7f111c1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1120e00700 1 --2- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c133930 0x7f111c1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1120e00700 1 -- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c09d630 msgr2=0x7f111c1c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1120e00700 1 --2- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c09d630 0x7f111c1c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61d409e000 0x7f61d41194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61d409e000 0x7f61d41194b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47500/0 (socket says 172.21.15.89:47500) 2023-09-30T22:23:23.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 -- 172.21.15.89:0/3950219437 learned_addr learned my addr 172.21.15.89:0/3950219437 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1120e00700 1 -- 172.21.15.89:0/463518091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1114009a70 con 0x7f111c0a0a30 2023-09-30T22:23:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1120e00700 1 --2- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c0a0a30 0x7f111c1c7e80 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f110c000c00 tx=0x7f110c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1111ffb700 1 -- 172.21.15.89:0/463518091 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c004450 con 0x7f111c0a0a30 2023-09-30T22:23:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1111ffb700 1 -- 172.21.15.89:0/463518091 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c0045b0 con 0x7f111c0a0a30 2023-09-30T22:23:23.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1111ffb700 1 -- 172.21.15.89:0/463518091 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c0038c0 con 0x7f111c0a0a30 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61da401700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409e410 0x7f61d41199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d9c00700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61d409d6f0 0x7f61d41cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 -- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409e410 msgr2=0x7f61d41199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409e410 0x7f61d41199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 -- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61d409d6f0 msgr2=0x7f61d41cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61d409d6f0 0x7f61d41cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 -- 172.21.15.89:0/3950219437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00094a0 con 0x7f61d409e000 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f61d93ff700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61d409e000 0x7f61d41194b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f61c4005910 tx=0x7f61c400c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.965+0000 7f61ca7fc700 1 -- 172.21.15.89:0/3950219437 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4005d70 con 0x7f61d409e000 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.965+0000 7f61ca7fc700 1 -- 172.21.15.89:0/3950219437 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c40070d0 con 0x7f61d409e000 2023-09-30T22:23:23.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.965+0000 7f61ca7fc700 1 -- 172.21.15.89:0/3950219437 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4007440 con 0x7f61d409e000 2023-09-30T22:23:23.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.964+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c009550 con 0x7f111c0a0a30 2023-09-30T22:23:23.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.965+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c0a5000 con 0x7f111c0a0a30 2023-09-30T22:23:23.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.965+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d411b0c0 con 0x7f61d409e000 2023-09-30T22:23:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.965+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d411ccf0 con 0x7f61d409e000 2023-09-30T22:23:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.966+0000 7f61ca7fc700 1 -- 172.21.15.89:0/3950219437 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f61c40075a0 con 0x7f61d409e000 2023-09-30T22:23:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.966+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d4074ea0 con 0x7f61d409e000 2023-09-30T22:23:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.966+0000 7f61ca7fc700 1 -- 172.21.15.89:0/3950219437 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f61c403eb50 con 0x7f61d409e000 2023-09-30T22:23:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.966+0000 7f1111ffb700 1 -- 172.21.15.89:0/463518091 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f110c021070 con 0x7f111c0a0a30 2023-09-30T22:23:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.966+0000 7f1111ffb700 1 -- 172.21.15.89:0/463518091 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f110c0408b0 con 0x7f111c0a0a30 2023-09-30T22:23:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.967+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c0747f0 con 0x7f111c0a0a30 2023-09-30T22:23:23.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.970+0000 7f61ca7fc700 1 -- 172.21.15.89:0/3950219437 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f61c4017030 con 0x7f61d409e000 2023-09-30T22:23:23.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:23.972+0000 7f1111ffb700 1 -- 172.21.15.89:0/463518091 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f110c029080 con 0x7f111c0a0a30 2023-09-30T22:23:24.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.147+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd48413b340 con 0x7fd48409d6f0 2023-09-30T22:23:24.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.147+0000 7fd46bfff700 1 -- 172.21.15.89:0/580293581 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd474040e60 con 0x7fd48409d6f0 2023-09-30T22:23:24.148 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:24.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409d6f0 msgr2=0x7fd484139150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:24.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409d6f0 0x7fd484139150 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd4840a3e80 tx=0x7fd474004750 comp rx=0 tx=0).stop 2023-09-30T22:23:24.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 shutdown_connections 2023-09-30T22:23:24.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48409e410 0x7fd484139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd48409d6f0 0x7fd484139150 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 --2- 172.21.15.89:0/580293581 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd48409e000 0x7fd484139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 >> 172.21.15.89:0/580293581 conn(0x7fd484066f50 msgr2=0x7fd484070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:24.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 shutdown_connections 2023-09-30T22:23:24.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.150+0000 7fd489bf8700 1 -- 172.21.15.89:0/580293581 wait complete. 2023-09-30T22:23:24.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:24.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.163+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f61d406c360 con 0x7f61d409e000 2023-09-30T22:23:24.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.163+0000 7f61ca7fc700 1 -- 172.21.15.89:0/3950219437 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f61c4024070 con 0x7f61d409e000 2023-09-30T22:23:24.164 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:24.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61d409e000 msgr2=0x7f61d41194b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:24.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61d409e000 0x7f61d41194b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f61c4005910 tx=0x7f61c400c9e0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 shutdown_connections 2023-09-30T22:23:24.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61d409e410 0x7f61d41199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61d409d6f0 0x7f61d41cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 --2- 172.21.15.89:0/3950219437 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61d409e000 0x7f61d41194b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 >> 172.21.15.89:0/3950219437 conn(0x7f61d4066f50 msgr2=0x7f61d41bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:24.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 shutdown_connections 2023-09-30T22:23:24.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.165+0000 7f61dbe64700 1 -- 172.21.15.89:0/3950219437 wait complete. 2023-09-30T22:23:24.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.173+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f111c072980 con 0x7f111c0a0a30 2023-09-30T22:23:24.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.173+0000 7f1111ffb700 1 -- 172.21.15.89:0/463518091 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f110c015020 con 0x7f111c0a0a30 2023-09-30T22:23:24.174 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c0a0a30 msgr2=0x7f111c1c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 --2- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c0a0a30 0x7f111c1c7e80 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f110c000c00 tx=0x7f110c004750 comp rx=0 tx=0).stop 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 shutdown_connections 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 --2- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f111c133930 0x7f111c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 --2- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f111c09d630 0x7f111c1c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 --2- 172.21.15.89:0/463518091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f111c0a0a30 0x7f111c1c7e80 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 >> 172.21.15.89:0/463518091 conn(0x7f111c0a3050 msgr2=0x7f111c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 shutdown_connections 2023-09-30T22:23:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.176+0000 7f1123865700 1 -- 172.21.15.89:0/463518091 wait complete. 2023-09-30T22:23:24.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:24.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.187+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe0500a4e30 con 0x7fe0500a0a30 2023-09-30T22:23:24.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.187+0000 7fe046ffd700 1 -- 172.21.15.89:0/2572943584 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe04001b3b0 con 0x7fe0500a0a30 2023-09-30T22:23:24.188 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:24.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe0500a0a30 msgr2=0x7fe05011cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe0500a0a30 0x7fe05011cd00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe04000ebe0 tx=0x7fe04000eef0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 shutdown_connections 2023-09-30T22:23:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe050138d80 0x7fe05011d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe05009d630 0x7fe05011c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 --2- 172.21.15.89:0/2572943584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe0500a0a30 0x7fe05011cd00 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 >> 172.21.15.89:0/2572943584 conn(0x7fe0500a3050 msgr2=0x7fe05009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 shutdown_connections 2023-09-30T22:23:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:24.189+0000 7fe0586e1700 1 -- 172.21.15.89:0/2572943584 wait complete. 2023-09-30T22:23:24.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:25.165 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:25.179 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:25.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:25.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.344+0000 7f73b1cfd700 1 Processor -- start 2023-09-30T22:23:25.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.344+0000 7f73b1cfd700 1 -- start start 2023-09-30T22:23:25.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73b1cfd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09e930 0x7f73ac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73b1cfd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09f240 0x7f73ac1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73b1cfd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73ac09f650 0x7f73ac131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73b1cfd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac070940 con 0x7f73ac09f650 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73b1cfd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac06fd60 con 0x7f73ac09e930 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73b1cfd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac06fa80 con 0x7f73ac09f240 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73ab7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09e930 0x7f73ac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73ab7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09e930 0x7f73ac09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60568/0 (socket says 172.21.15.89:60568) 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73ab7fe700 1 -- 172.21.15.89:0/1596779276 learned_addr learned my addr 172.21.15.89:0/1596779276 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:25.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73aaffd700 1 --2- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09f240 0x7f73ac1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73abfff700 1 --2- 172.21.15.89:0/1596779276 >> v2:172.21.15.89:3300/0 conn(0x7f73ac09f650 0x7f73ac131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73aaffd700 1 -- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09e930 msgr2=0x7f73ac09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73aaffd700 1 --2- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09e930 0x7f73ac09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.345+0000 7f73aaffd700 1 -- 172.21.15.89:0/1596779276 >> v2:172.21.15.89:3300/0 conn(0x7f73ac09f650 msgr2=0x7f73ac131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f73aaffd700 1 --2- 172.21.15.89:0/1596779276 >> v2:172.21.15.89:3300/0 conn(0x7f73ac09f650 0x7f73ac131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f73aaffd700 1 -- 172.21.15.89:0/1596779276 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac09fb60 con 0x7f73ac09f240 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f73ab7fe700 1 --2- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09e930 0x7f73ac09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f73abfff700 1 --2- 172.21.15.89:0/1596779276 >> v2:172.21.15.89:3300/0 conn(0x7f73ac09f650 0x7f73ac131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f73aaffd700 1 --2- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09f240 0x7f73ac1317f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009840 tx=0x7f73a0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694265a5e7370833 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f73aa7fc700 1 -- 172.21.15.89:0/1596779276 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0011830 con 0x7f73ac09f240 2023-09-30T22:23:25.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f73aa7fc700 1 -- 172.21.15.89:0/1596779276 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a0011990 con 0x7f73ac09f240 2023-09-30T22:23:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f73aa7fc700 1 -- 172.21.15.89:0/1596779276 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0011d20 con 0x7f73ac09f240 2023-09-30T22:23:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f8a92f51700 1 Processor -- start 2023-09-30T22:23:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.346+0000 7f8a92f51700 1 -- start start 2023-09-30T22:23:25.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 -- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09f240 msgr2=0x7f73ac1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 --2- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09f240 0x7f73ac1317f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009840 tx=0x7f73a0009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a92f51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09d6f0 0x7f8a8c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a92f51700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a8c09e410 0x7f8a8c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a92f51700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8c09f130 con 0x7f8a8c09e000 2023-09-30T22:23:25.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a92f51700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8c09f520 con 0x7f8a8c09e410 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a92f51700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c09e930 con 0x7f8a8c09d6f0 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a914ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a8c09e410 0x7f8a8c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a90ced700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09d6f0 0x7f8a8c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a914ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a8c09e410 0x7f8a8c130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35100/0 (socket says 172.21.15.89:35100) 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a914ee700 1 -- 172.21.15.89:0/3190463730 learned_addr learned my addr 172.21.15.89:0/3190463730 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 -- 172.21.15.89:0/1596779276 shutdown_connections 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 --2- 172.21.15.89:0/1596779276 >> v2:172.21.15.89:3300/0 conn(0x7f73ac09f650 0x7f73ac131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 --2- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09e930 0x7f73ac09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 --2- 172.21.15.89:0/1596779276 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09f240 0x7f73ac1317f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 -- 172.21.15.89:0/1596779276 >> 172.21.15.89:0/1596779276 conn(0x7f73ac0a4210 msgr2=0x7f73ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 -- 172.21.15.89:0/1596779276 shutdown_connections 2023-09-30T22:23:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f73b1cfd700 1 -- 172.21.15.89:0/1596779276 wait complete. 2023-09-30T22:23:25.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a90ced700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09d6f0 0x7f8a8c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60584/0 (socket says 172.21.15.89:60584) 2023-09-30T22:23:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a90ced700 1 -- 172.21.15.89:0/3190463730 >> v2:172.21.15.89:3300/0 conn(0x7f8a8c09e410 msgr2=0x7f8a8c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a90ced700 1 --2- 172.21.15.89:0/3190463730 >> v2:172.21.15.89:3300/0 conn(0x7f8a8c09e410 0x7f8a8c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a90ced700 1 -- 172.21.15.89:0/3190463730 >> v1:172.21.15.89:6789/0 conn(0x7f8a8c09e000 legacy=0x7f8a8c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.347+0000 7f8a90ced700 1 -- 172.21.15.89:0/3190463730 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c09b2d0 con 0x7f8a8c09d6f0 2023-09-30T22:23:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.348+0000 7f8a90ced700 1 --2- 172.21.15.89:0/3190463730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09d6f0 0x7f8a8c09dac0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00bae0 tx=0x7f8a7c00bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a5bab0fa723bb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:25.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.348+0000 7f8a8b7fe700 1 -- 172.21.15.89:0/3190463730 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00dc70 con 0x7f8a8c09d6f0 2023-09-30T22:23:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.348+0000 7f8a8b7fe700 1 -- 172.21.15.89:0/3190463730 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c00ddd0 con 0x7f8a8c09d6f0 2023-09-30T22:23:25.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.348+0000 7f8a8b7fe700 1 -- 172.21.15.89:0/3190463730 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c010690 con 0x7f8a8c09d6f0 2023-09-30T22:23:25.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.348+0000 7f73b1cfd700 1 Processor -- start 2023-09-30T22:23:25.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73b1cfd700 1 -- start start 2023-09-30T22:23:25.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f8a92f51700 1 -- 172.21.15.89:0/3190463730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09d6f0 msgr2=0x7f8a8c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f8a92f51700 1 --2- 172.21.15.89:0/3190463730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09d6f0 0x7f8a8c09dac0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00bae0 tx=0x7f8a7c00bdf0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f8a92f51700 1 -- 172.21.15.89:0/3190463730 shutdown_connections 2023-09-30T22:23:25.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f8a92f51700 1 --2- 172.21.15.89:0/3190463730 >> v2:172.21.15.89:3300/0 conn(0x7f8a8c09e410 0x7f8a8c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f8a92f51700 1 --2- 172.21.15.89:0/3190463730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09d6f0 0x7f8a8c09dac0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f8a92f51700 1 -- 172.21.15.89:0/3190463730 >> 172.21.15.89:0/3190463730 conn(0x7f8a8c066f50 msgr2=0x7f8a8c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:25.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f8a92f51700 1 -- 172.21.15.89:0/3190463730 shutdown_connections 2023-09-30T22:23:25.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f8a92f51700 1 -- 172.21.15.89:0/3190463730 wait complete. 2023-09-30T22:23:25.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73b1cfd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09e930 0x7f73ac1c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73b1cfd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73ac09f240 0x7f73ac1c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73b1cfd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09f650 0x7f73ac1c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73b1cfd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac1a43d0 con 0x7f73ac09f240 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73b1cfd700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac1a47e0 con 0x7f73ac09f650 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73b1cfd700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac1a4fd0 con 0x7f73ac09e930 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73aaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73ac09f240 0x7f73ac1c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73aaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73ac09f240 0x7f73ac1c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35104/0 (socket says 172.21.15.89:35104) 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.349+0000 7f73aaffd700 1 -- 172.21.15.89:0/58185285 learned_addr learned my addr 172.21.15.89:0/58185285 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f73aaffd700 1 -- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09e930 msgr2=0x7f73ac1c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f73aaffd700 1 --2- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09e930 0x7f73ac1c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f73aaffd700 1 -- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09f650 msgr2=0x7f73ac1c07b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:25.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f73aaffd700 1 --2- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09f650 0x7f73ac1c07b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f73aaffd700 1 -- 172.21.15.89:0/58185285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c009550 con 0x7f73ac09f240 2023-09-30T22:23:25.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f73aaffd700 1 --2- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73ac09f240 0x7f73ac1c4880 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f73a00090b0 tx=0x7f73a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:25.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f8a92f51700 1 Processor -- start 2023-09-30T22:23:25.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f7393fff700 1 -- 172.21.15.89:0/58185285 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0004590 con 0x7f73ac09f240 2023-09-30T22:23:25.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a400bc50 con 0x7f73ac09f240 2023-09-30T22:23:25.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f7393fff700 1 -- 172.21.15.89:0/58185285 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a001f070 con 0x7f73ac09f240 2023-09-30T22:23:25.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f7393fff700 1 -- 172.21.15.89:0/58185285 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00038e0 con 0x7f73ac09f240 2023-09-30T22:23:25.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a00094a0 con 0x7f73ac09f240 2023-09-30T22:23:25.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.350+0000 7f8a92f51700 1 -- start start 2023-09-30T22:23:25.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a92f51700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a8c09d6f0 0x7f8a8c11e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a92f51700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a8c09e000 0x7f8a8c11e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a92f51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09e410 0x7f8a8c11eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a92f51700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c11fd10 con 0x7f8a8c09e000 2023-09-30T22:23:25.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a92f51700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0a1e90 con 0x7f8a8c09e410 2023-09-30T22:23:25.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a92f51700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c09c700 con 0x7f8a8c09d6f0 2023-09-30T22:23:25.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac074ea0 con 0x7f73ac09f240 2023-09-30T22:23:25.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a8bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a8c09e000 0x7f8a8c11e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a8bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a8c09e000 0x7f8a8c11e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35106/0 (socket says 172.21.15.89:35106) 2023-09-30T22:23:25.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a8bfff700 1 -- 172.21.15.89:0/171246769 learned_addr learned my addr 172.21.15.89:0/171246769 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:25.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.351+0000 7f8a914ee700 1 --2- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09e410 0x7f8a8c11eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a914ee700 1 -- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a8c09d6f0 msgr2=0x7f8a8c11e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a914ee700 1 --2- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a8c09d6f0 0x7f8a8c11e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a914ee700 1 -- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a8c09e000 msgr2=0x7f8a8c11e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a914ee700 1 --2- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a8c09e000 0x7f8a8c11e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a914ee700 1 -- 172.21.15.89:0/171246769 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c00b740 con 0x7f8a8c09e410 2023-09-30T22:23:25.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a914ee700 1 --2- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09e410 0x7f8a8c11eac0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400ef90 tx=0x7f8a8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:25.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f7393fff700 1 -- 172.21.15.89:0/58185285 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f73a0003a40 con 0x7f73ac09f240 2023-09-30T22:23:25.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a897fa700 1 -- 172.21.15.89:0/171246769 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84003c20 con 0x7f8a8c09e410 2023-09-30T22:23:25.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c121b10 con 0x7f8a8c09e410 2023-09-30T22:23:25.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a897fa700 1 -- 172.21.15.89:0/171246769 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8400f040 con 0x7f8a8c09e410 2023-09-30T22:23:25.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c1cc670 con 0x7f8a8c09e410 2023-09-30T22:23:25.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f8a897fa700 1 -- 172.21.15.89:0/171246769 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84009e60 con 0x7f8a8c09e410 2023-09-30T22:23:25.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.352+0000 7f7393fff700 1 -- 172.21.15.89:0/58185285 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f73a0021070 con 0x7f73ac09f240 2023-09-30T22:23:25.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.353+0000 7f6b26b5b700 1 Processor -- start 2023-09-30T22:23:25.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.353+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c0052e0 con 0x7f8a8c09e410 2023-09-30T22:23:25.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.353+0000 7f6b26b5b700 1 -- start start 2023-09-30T22:23:25.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.353+0000 7f8a897fa700 1 -- 172.21.15.89:0/171246769 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a8400c930 con 0x7f8a8c09e410 2023-09-30T22:23:25.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f8a897fa700 1 -- 172.21.15.89:0/171246769 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8a8403f950 con 0x7f8a8c09e410 2023-09-30T22:23:25.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b26b5b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b26b5b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 0x7f6b2013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b26b5b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b200a2a70 con 0x7f6b2009d630 2023-09-30T22:23:25.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b26b5b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200a01d0 con 0x7f6b200a0a30 2023-09-30T22:23:25.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b26b5b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2009e2b0 con 0x7f6b20138d80 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b1ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6b2009d630 0x7f6b200a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40098/0 (socket says 172.21.15.89:40098) 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b1ffff700 1 -- 172.21.15.89:0/2439160914 learned_addr learned my addr 172.21.15.89:0/2439160914 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b248f7700 1 --2- 172.21.15.89:0/2439160914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b248f7700 1 -- 172.21.15.89:0/2439160914 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 msgr2=0x7f6b2013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b248f7700 1 --2- 172.21.15.89:0/2439160914 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 0x7f6b2013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b248f7700 1 -- 172.21.15.89:0/2439160914 >> v1:172.21.15.89:6789/0 conn(0x7f6b2009d630 legacy=0x7f6b200a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.354+0000 7f6b248f7700 1 -- 172.21.15.89:0/2439160914 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2009dc50 con 0x7f6b200a0a30 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.355+0000 7f6b248f7700 1 --2- 172.21.15.89:0/2439160914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b200a0e00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009870 tx=0x7f6b0c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2f5ed4213821fd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:25.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.357+0000 7f7393fff700 1 -- 172.21.15.89:0/58185285 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73a0030480 con 0x7f73ac09f240 2023-09-30T22:23:25.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.358+0000 7f6b1f7fe700 1 -- 172.21.15.89:0/2439160914 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c011830 con 0x7f6b200a0a30 2023-09-30T22:23:25.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.358+0000 7f6b1f7fe700 1 -- 172.21.15.89:0/2439160914 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c011990 con 0x7f6b200a0a30 2023-09-30T22:23:25.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.358+0000 7f6b1f7fe700 1 -- 172.21.15.89:0/2439160914 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c011d20 con 0x7f6b200a0a30 2023-09-30T22:23:25.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.358+0000 7f8a897fa700 1 -- 172.21.15.89:0/171246769 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a8401cb50 con 0x7f8a8c09e410 2023-09-30T22:23:25.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.359+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2439160914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 msgr2=0x7f6b200a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.359+0000 7f6b26b5b700 1 --2- 172.21.15.89:0/2439160914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b200a0e00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009870 tx=0x7f6b0c009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:25.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.359+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2439160914 shutdown_connections 2023-09-30T22:23:25.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.359+0000 7f6b26b5b700 1 --2- 172.21.15.89:0/2439160914 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b200a0e00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.360+0000 7f6b26b5b700 1 --2- 172.21.15.89:0/2439160914 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 0x7f6b2013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.360+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2439160914 >> 172.21.15.89:0/2439160914 conn(0x7f6b200a3050 msgr2=0x7f6b200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.363+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2439160914 shutdown_connections 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.363+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2439160914 wait complete. 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.364+0000 7f6b26b5b700 1 Processor -- start 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.364+0000 7f6b26b5b700 1 -- start start 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b26b5b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2009d630 0x7f6b201206f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b26b5b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b20120c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b26b5b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 0x7f6b20121170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b26b5b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b20123ea0 con 0x7f6b2009d630 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b26b5b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b20123530 con 0x7f6b200a0a30 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b26b5b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200a4d90 con 0x7f6b20138d80 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b1ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b20120c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b1ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b20120c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60608/0 (socket says 172.21.15.89:60608) 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b1ffff700 1 -- 172.21.15.89:0/2092555110 learned_addr learned my addr 172.21.15.89:0/2092555110 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b250f8700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 0x7f6b20121170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b1ffff700 1 -- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 msgr2=0x7f6b20121170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b1ffff700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 0x7f6b20121170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b1ffff700 1 -- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2009d630 msgr2=0x7f6b201206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b1ffff700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2009d630 0x7f6b201206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b1ffff700 1 -- 172.21.15.89:0/2092555110 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0094a0 con 0x7f6b200a0a30 2023-09-30T22:23:25.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.365+0000 7f6b250f8700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 0x7f6b20121170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:25.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.366+0000 7f6b1ffff700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b20120c30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c500 tx=0x7f6b1400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:25.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.366+0000 7f6b1d7fa700 1 -- 172.21.15.89:0/2092555110 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14005d70 con 0x7f6b200a0a30 2023-09-30T22:23:25.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.366+0000 7f6b1d7fa700 1 -- 172.21.15.89:0/2092555110 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b140070d0 con 0x7f6b200a0a30 2023-09-30T22:23:25.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.366+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b200a0410 con 0x7f6b200a0a30 2023-09-30T22:23:25.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.366+0000 7f6b1d7fa700 1 -- 172.21.15.89:0/2092555110 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14007440 con 0x7f6b200a0a30 2023-09-30T22:23:25.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.366+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b20124180 con 0x7f6b200a0a30 2023-09-30T22:23:25.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.367+0000 7f6b1d7fa700 1 -- 172.21.15.89:0/2092555110 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b140075a0 con 0x7f6b200a0a30 2023-09-30T22:23:25.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.367+0000 7f6b1d7fa700 1 -- 172.21.15.89:0/2092555110 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6b1403ec80 con 0x7f6b200a0a30 2023-09-30T22:23:25.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.367+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b200747f0 con 0x7f6b200a0a30 2023-09-30T22:23:25.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.372+0000 7f6b1d7fa700 1 -- 172.21.15.89:0/2092555110 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b14017080 con 0x7f6b200a0a30 2023-09-30T22:23:25.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.387+0000 7f62e256a700 1 Processor -- start 2023-09-30T22:23:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.387+0000 7f62e256a700 1 -- start start 2023-09-30T22:23:25.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.387+0000 7f62e256a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f62dc06a770 0x7f62dc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.387+0000 7f62e256a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc13d1f0 0x7f62dc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.387+0000 7f62e256a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62dc09a810 con 0x7f62dc09dc30 2023-09-30T22:23:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.387+0000 7f62e256a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc06ab40 con 0x7f62dc06a770 2023-09-30T22:23:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.387+0000 7f62e256a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc09ebc0 con 0x7f62dc13d1f0 2023-09-30T22:23:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62e0b07700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc13d1f0 0x7f62dc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62dbfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f62dc06a770 0x7f62dc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62e0b07700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc13d1f0 0x7f62dc13f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60616/0 (socket says 172.21.15.89:60616) 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62e0b07700 1 -- 172.21.15.89:0/2279007509 learned_addr learned my addr 172.21.15.89:0/2279007509 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62e0b07700 1 -- 172.21.15.89:0/2279007509 >> v2:172.21.15.89:3300/0 conn(0x7f62dc06a770 msgr2=0x7f62dc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62e0b07700 1 --2- 172.21.15.89:0/2279007509 >> v2:172.21.15.89:3300/0 conn(0x7f62dc06a770 0x7f62dc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62e0b07700 1 -- 172.21.15.89:0/2279007509 >> v1:172.21.15.89:6789/0 conn(0x7f62dc09dc30 legacy=0x7f62dc09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62e0b07700 1 -- 172.21.15.89:0/2279007509 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc09e920 con 0x7f62dc13d1f0 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62dbfff700 1 --2- 172.21.15.89:0/2279007509 >> v2:172.21.15.89:3300/0 conn(0x7f62dc06a770 0x7f62dc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.388+0000 7f62e0b07700 1 --2- 172.21.15.89:0/2279007509 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc13d1f0 0x7f62dc13f5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009e50 tx=0x7f62d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8572dd9cda9d792a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62daffd700 1 -- 172.21.15.89:0/2279007509 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d400e070 con 0x7f62dc13d1f0 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62daffd700 1 -- 172.21.15.89:0/2279007509 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d400b8d0 con 0x7f62dc13d1f0 2023-09-30T22:23:25.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62daffd700 1 -- 172.21.15.89:0/2279007509 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d400bc40 con 0x7f62dc13d1f0 2023-09-30T22:23:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62e256a700 1 -- 172.21.15.89:0/2279007509 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc13d1f0 msgr2=0x7f62dc13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62e256a700 1 --2- 172.21.15.89:0/2279007509 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc13d1f0 0x7f62dc13f5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009e50 tx=0x7f62d400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:25.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62e256a700 1 -- 172.21.15.89:0/2279007509 shutdown_connections 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62e256a700 1 --2- 172.21.15.89:0/2279007509 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc13d1f0 0x7f62dc13f5d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62e256a700 1 --2- 172.21.15.89:0/2279007509 >> v2:172.21.15.89:3300/0 conn(0x7f62dc06a770 0x7f62dc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62e256a700 1 -- 172.21.15.89:0/2279007509 >> 172.21.15.89:0/2279007509 conn(0x7f62dc066f50 msgr2=0x7f62dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62e256a700 1 -- 172.21.15.89:0/2279007509 shutdown_connections 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.389+0000 7f62e256a700 1 -- 172.21.15.89:0/2279007509 wait complete. 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.390+0000 7f62e256a700 1 Processor -- start 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.390+0000 7f62e256a700 1 -- start start 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e256a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62dc06a770 0x7f62dc071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e256a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc09dc30 0x7f62dc13b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e256a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62dc13d1f0 0x7f62dc13bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:25.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e256a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc072570 con 0x7f62dc13d1f0 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e256a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc11e460 con 0x7f62dc09dc30 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e256a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc11f190 con 0x7f62dc06a770 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62dbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62dc06a770 0x7f62dc071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62dbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62dc06a770 0x7f62dc071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47540/0 (socket says 172.21.15.89:47540) 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62dbfff700 1 -- 172.21.15.89:0/3937642836 learned_addr learned my addr 172.21.15.89:0/3937642836 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e0b07700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62dc13d1f0 0x7f62dc13bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62db7fe700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc09dc30 0x7f62dc13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e0b07700 1 -- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62dc06a770 msgr2=0x7f62dc071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e0b07700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62dc06a770 0x7f62dc071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e0b07700 1 -- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc09dc30 msgr2=0x7f62dc13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e0b07700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc09dc30 0x7f62dc13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.391+0000 7f62e0b07700 1 -- 172.21.15.89:0/3937642836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0094a0 con 0x7f62dc13d1f0 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.392+0000 7f62e0b07700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62dc13d1f0 0x7f62dc13bc80 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f62d4015000 tx=0x7f62d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.392+0000 7f62d8ff9700 1 -- 172.21.15.89:0/3937642836 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4004450 con 0x7f62dc13d1f0 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.392+0000 7f62d8ff9700 1 -- 172.21.15.89:0/3937642836 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d40045d0 con 0x7f62dc13d1f0 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.392+0000 7f62e256a700 1 -- 172.21.15.89:0/3937642836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d4009b30 con 0x7f62dc13d1f0 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.392+0000 7f62d8ff9700 1 -- 172.21.15.89:0/3937642836 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40094d0 con 0x7f62dc13d1f0 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.392+0000 7f62e256a700 1 -- 172.21.15.89:0/3937642836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc11f840 con 0x7f62dc13d1f0 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.393+0000 7f62c27fc700 1 -- 172.21.15.89:0/3937642836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b40052e0 con 0x7f62dc13d1f0 2023-09-30T22:23:25.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.393+0000 7f62d8ff9700 1 -- 172.21.15.89:0/3937642836 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f62d401c410 con 0x7f62dc13d1f0 2023-09-30T22:23:25.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.397+0000 7f62d8ff9700 1 -- 172.21.15.89:0/3937642836 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f62d401ce20 con 0x7f62dc13d1f0 2023-09-30T22:23:25.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.398+0000 7f62d8ff9700 1 -- 172.21.15.89:0/3937642836 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f62d40123f0 con 0x7f62dc13d1f0 2023-09-30T22:23:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.555+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8a6c005150 con 0x7f8a8c09e410 2023-09-30T22:23:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.555+0000 7f8a897fa700 1 -- 172.21.15.89:0/171246769 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a8401cb50 con 0x7f8a8c09e410 2023-09-30T22:23:25.556 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f73ac06b250 con 0x7f73ac09f240 2023-09-30T22:23:25.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f7393fff700 1 -- 172.21.15.89:0/58185285 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73a0015070 con 0x7f73ac09f240 2023-09-30T22:23:25.558 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09e410 msgr2=0x7f8a8c11eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 --2- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09e410 0x7f8a8c11eac0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400ef90 tx=0x7f8a8400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 shutdown_connections 2023-09-30T22:23:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 --2- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a8c09e410 0x7f8a8c11eac0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 --2- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a8c09d6f0 0x7f8a8c11e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 --2- 172.21.15.89:0/171246769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a8c09e000 0x7f8a8c11e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 >> 172.21.15.89:0/171246769 conn(0x7f8a8c066f50 msgr2=0x7f8a8c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 shutdown_connections 2023-09-30T22:23:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.557+0000 7f8a92f51700 1 -- 172.21.15.89:0/171246769 wait complete. 2023-09-30T22:23:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.559+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73ac09f240 msgr2=0x7f73ac1c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.559+0000 7f73b1cfd700 1 --2- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73ac09f240 0x7f73ac1c4880 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f73a00090b0 tx=0x7f73a0004750 comp rx=0 tx=0).stop 2023-09-30T22:23:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.560+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 shutdown_connections 2023-09-30T22:23:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.560+0000 7f73b1cfd700 1 --2- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73ac09f650 0x7f73ac1c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.560+0000 7f73b1cfd700 1 --2- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73ac09e930 0x7f73ac1c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.560+0000 7f73b1cfd700 1 --2- 172.21.15.89:0/58185285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73ac09f240 0x7f73ac1c4880 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.560+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 >> 172.21.15.89:0/58185285 conn(0x7f73ac0a4210 msgr2=0x7f73ac1b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:25.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.560+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 shutdown_connections 2023-09-30T22:23:25.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.560+0000 7f73b1cfd700 1 -- 172.21.15.89:0/58185285 wait complete. 2023-09-30T22:23:25.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:25.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.573+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6b20072980 con 0x7f6b200a0a30 2023-09-30T22:23:25.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.574+0000 7f6b1d7fa700 1 -- 172.21.15.89:0/2092555110 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b14019d20 con 0x7f6b200a0a30 2023-09-30T22:23:25.575 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:25.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 msgr2=0x7f6b20120c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b20120c30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c500 tx=0x7f6b1400c810 comp rx=0 tx=0).stop 2023-09-30T22:23:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 shutdown_connections 2023-09-30T22:23:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6b20138d80 0x7f6b20121170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b2009d630 0x7f6b201206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 --2- 172.21.15.89:0/2092555110 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b200a0a30 0x7f6b20120c30 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 >> 172.21.15.89:0/2092555110 conn(0x7f6b200a3050 msgr2=0x7f6b2009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 shutdown_connections 2023-09-30T22:23:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.576+0000 7f6b26b5b700 1 -- 172.21.15.89:0/2092555110 wait complete. 2023-09-30T22:23:25.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.597+0000 7f62c27fc700 1 -- 172.21.15.89:0/3937642836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f62b4005590 con 0x7f62dc13d1f0 2023-09-30T22:23:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.598+0000 7f62d8ff9700 1 -- 172.21.15.89:0/3937642836 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f62d402e3e0 con 0x7f62dc13d1f0 2023-09-30T22:23:25.599 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 -- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62dc13d1f0 msgr2=0x7f62dc13bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62dc13d1f0 0x7f62dc13bc80 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f62d4015000 tx=0x7f62d4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 -- 172.21.15.89:0/3937642836 shutdown_connections 2023-09-30T22:23:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62dc13d1f0 0x7f62dc13bc80 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62dc06a770 0x7f62dc071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 --2- 172.21.15.89:0/3937642836 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62dc09dc30 0x7f62dc13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 -- 172.21.15.89:0/3937642836 >> 172.21.15.89:0/3937642836 conn(0x7f62dc066f50 msgr2=0x7f62dc09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 -- 172.21.15.89:0/3937642836 shutdown_connections 2023-09-30T22:23:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:25.600+0000 7f62c27fc700 1 -- 172.21.15.89:0/3937642836 wait complete. 2023-09-30T22:23:25.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:26.575 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:26.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:26.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:26.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.719+0000 7f74c9dc9700 1 Processor -- start 2023-09-30T22:23:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.719+0000 7f7456daf700 1 Processor -- start 2023-09-30T22:23:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.719+0000 7f74c9dc9700 1 -- start start 2023-09-30T22:23:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.719+0000 7f7456daf700 1 -- start start 2023-09-30T22:23:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7456daf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74501338a0 0x7f7450133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7456daf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f745009de50 0x7f745009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7456daf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f745009e760 0x7f745013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7456daf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745006fef0 con 0x7f745009de50 2023-09-30T22:23:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7456daf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745006ef30 con 0x7f74501338a0 2023-09-30T22:23:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7456daf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74501334b0 con 0x7f745009e760 2023-09-30T22:23:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f74c9dc9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f74c409e930 0x7f74c409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f74c9dc9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409f650 0x7f74c4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f74c9dc9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c4070940 con 0x7f74c409f240 2023-09-30T22:23:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f74c9dc9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c406fd60 con 0x7f74c409e930 2023-09-30T22:23:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f74c9dc9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c406fa80 con 0x7f74c409f650 2023-09-30T22:23:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7454b4b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74501338a0 0x7f7450133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7454b4b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74501338a0 0x7f7450133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60634/0 (socket says 172.21.15.89:60634) 2023-09-30T22:23:26.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f7454b4b700 1 -- 172.21.15.89:0/986849151 learned_addr learned my addr 172.21.15.89:0/986849151 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f744ffff700 1 --2- 172.21.15.89:0/986849151 >> v2:172.21.15.89:3300/0 conn(0x7f745009de50 0x7f745009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f74c2ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f74c409f240 0x7f74c41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40112/0 (socket says 172.21.15.89:40112) 2023-09-30T22:23:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f74c2ffd700 1 -- 172.21.15.89:0/3654990530 learned_addr learned my addr 172.21.15.89:0/3654990530 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.720+0000 7f745534c700 1 --2- 172.21.15.89:0/986849151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f745009e760 0x7f745013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f7454b4b700 1 -- 172.21.15.89:0/986849151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f745009e760 msgr2=0x7f745013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f7454b4b700 1 --2- 172.21.15.89:0/986849151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f745009e760 0x7f745013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f7454b4b700 1 -- 172.21.15.89:0/986849151 >> v2:172.21.15.89:3300/0 conn(0x7f745009de50 msgr2=0x7f745009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f7454b4b700 1 --2- 172.21.15.89:0/986849151 >> v2:172.21.15.89:3300/0 conn(0x7f745009de50 0x7f745009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f7454b4b700 1 -- 172.21.15.89:0/986849151 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745006fc30 con 0x7f74501338a0 2023-09-30T22:23:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f74c3fff700 1 --2- 172.21.15.89:0/3654990530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409f650 0x7f74c4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f7454b4b700 1 --2- 172.21.15.89:0/986849151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74501338a0 0x7f7450133c70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f74440098f0 tx=0x7f7444009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b510bc61eaa954 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:26.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f74c3fff700 1 -- 172.21.15.89:0/3654990530 >> v2:172.21.15.89:3300/0 conn(0x7f74c409e930 msgr2=0x7f74c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:26.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f74c3fff700 1 --2- 172.21.15.89:0/3654990530 >> v2:172.21.15.89:3300/0 conn(0x7f74c409e930 0x7f74c409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f74c3fff700 1 -- 172.21.15.89:0/3654990530 >> v1:172.21.15.89:6789/0 conn(0x7f74c409f240 legacy=0x7f74c41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f74c3fff700 1 -- 172.21.15.89:0/3654990530 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c409fb60 con 0x7f74c409f650 2023-09-30T22:23:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.721+0000 7f74c3fff700 1 --2- 172.21.15.89:0/3654990530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409f650 0x7f74c4131f00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009d90 tx=0x7f74bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db3ccdc0fce0ae29 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f74c27fc700 1 -- 172.21.15.89:0/3654990530 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00e070 con 0x7f74c409f650 2023-09-30T22:23:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f744f7fe700 1 -- 172.21.15.89:0/986849151 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444011830 con 0x7f74501338a0 2023-09-30T22:23:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f744f7fe700 1 -- 172.21.15.89:0/986849151 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7444011990 con 0x7f74501338a0 2023-09-30T22:23:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f744f7fe700 1 -- 172.21.15.89:0/986849151 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444011d20 con 0x7f74501338a0 2023-09-30T22:23:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f74c27fc700 1 -- 172.21.15.89:0/3654990530 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc00bbd0 con 0x7f74c409f650 2023-09-30T22:23:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f74c27fc700 1 -- 172.21.15.89:0/3654990530 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc012430 con 0x7f74c409f650 2023-09-30T22:23:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f7456daf700 1 -- 172.21.15.89:0/986849151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74501338a0 msgr2=0x7f7450133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f7456daf700 1 --2- 172.21.15.89:0/986849151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74501338a0 0x7f7450133c70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f74440098f0 tx=0x7f7444009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f7456daf700 1 -- 172.21.15.89:0/986849151 shutdown_connections 2023-09-30T22:23:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f7456daf700 1 --2- 172.21.15.89:0/986849151 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f745009e760 0x7f745013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f7456daf700 1 --2- 172.21.15.89:0/986849151 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74501338a0 0x7f7450133c70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f7456daf700 1 --2- 172.21.15.89:0/986849151 >> v2:172.21.15.89:3300/0 conn(0x7f745009de50 0x7f745009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.722+0000 7f7456daf700 1 -- 172.21.15.89:0/986849151 >> 172.21.15.89:0/986849151 conn(0x7f74500a2aa0 msgr2=0x7f74500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f7456daf700 1 -- 172.21.15.89:0/986849151 shutdown_connections 2023-09-30T22:23:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f7456daf700 1 -- 172.21.15.89:0/986849151 wait complete. 2023-09-30T22:23:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f74c9dc9700 1 -- 172.21.15.89:0/3654990530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409f650 msgr2=0x7f74c4131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f74c9dc9700 1 --2- 172.21.15.89:0/3654990530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409f650 0x7f74c4131f00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009d90 tx=0x7f74bc00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f74c9dc9700 1 -- 172.21.15.89:0/3654990530 shutdown_connections 2023-09-30T22:23:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f74c9dc9700 1 --2- 172.21.15.89:0/3654990530 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409f650 0x7f74c4131f00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f74c9dc9700 1 --2- 172.21.15.89:0/3654990530 >> v2:172.21.15.89:3300/0 conn(0x7f74c409e930 0x7f74c409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f74c9dc9700 1 -- 172.21.15.89:0/3654990530 >> 172.21.15.89:0/3654990530 conn(0x7f74c40a4210 msgr2=0x7f74c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f74c9dc9700 1 -- 172.21.15.89:0/3654990530 shutdown_connections 2023-09-30T22:23:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f74c9dc9700 1 -- 172.21.15.89:0/3654990530 wait complete. 2023-09-30T22:23:26.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.723+0000 7f7456daf700 1 Processor -- start 2023-09-30T22:23:26.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f7456daf700 1 -- start start 2023-09-30T22:23:26.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f7456daf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f745009de50 0x7f745011eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f7456daf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f745009e760 0x7f745011f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f7456daf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74501338a0 0x7f745011f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f7456daf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745006f640 con 0x7f745009e760 2023-09-30T22:23:26.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f7456daf700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7450121e30 con 0x7f745009de50 2023-09-30T22:23:26.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f7456daf700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74500a1020 con 0x7f74501338a0 2023-09-30T22:23:26.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f74c9dc9700 1 Processor -- start 2023-09-30T22:23:26.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f745534c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74501338a0 0x7f745011f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f745534c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74501338a0 0x7f745011f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47550/0 (socket says 172.21.15.89:47550) 2023-09-30T22:23:26.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.724+0000 7f745534c700 1 -- 172.21.15.89:0/2454996394 learned_addr learned my addr 172.21.15.89:0/2454996394 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:26.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c9dc9700 1 -- start start 2023-09-30T22:23:26.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f744ffff700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f745009e760 0x7f745011f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f7454b4b700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f745009de50 0x7f745011eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f745534c700 1 -- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f745009de50 msgr2=0x7f745011eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f745534c700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f745009de50 0x7f745011eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f745534c700 1 -- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f745009e760 msgr2=0x7f745011f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f745534c700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f745009e760 0x7f745011f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f745534c700 1 -- 172.21.15.89:0/2454996394 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7444009550 con 0x7f74501338a0 2023-09-30T22:23:26.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c9dc9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409e930 0x7f74c41a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c9dc9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74c409f240 0x7f74c41a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c9dc9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74c409f650 0x7f74c41a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c9dc9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c41a6f00 con 0x7f74c409f240 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c9dc9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c41a70e0 con 0x7f74c409e930 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c9dc9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74c41a4dd0 con 0x7f74c409f650 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74c409f240 0x7f74c41a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74c409f240 0x7f74c41a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35170/0 (socket says 172.21.15.89:35170) 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409e930 0x7f74c41a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c2ffd700 1 -- 172.21.15.89:0/4285524126 learned_addr learned my addr 172.21.15.89:0/4285524126 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f74c3fff700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74c409f650 0x7f74c41a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.725+0000 7f745534c700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74501338a0 0x7f745011f620 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f744800efc0 tx=0x7f744800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:26.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c2ffd700 1 -- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74c409f650 msgr2=0x7f74c41a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c2ffd700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74c409f650 0x7f74c41a4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c2ffd700 1 -- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409e930 msgr2=0x7f74c41a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c2ffd700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409e930 0x7f74c41a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c2ffd700 1 -- 172.21.15.89:0/4285524126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009a70 con 0x7f74c409f240 2023-09-30T22:23:26.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c3fff700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74c409f650 0x7f74c41a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:26.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f744d7fa700 1 -- 172.21.15.89:0/2454996394 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448009ea0 con 0x7f74501338a0 2023-09-30T22:23:26.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f744d7fa700 1 -- 172.21.15.89:0/2454996394 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744800f040 con 0x7f74501338a0 2023-09-30T22:23:26.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f744d7fa700 1 -- 172.21.15.89:0/2454996394 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744800ced0 con 0x7f74501338a0 2023-09-30T22:23:26.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7450120850 con 0x7f74501338a0 2023-09-30T22:23:26.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c2ffd700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74c409f240 0x7f74c41a3f40 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c550 tx=0x7f74b800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:26.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7450122e50 con 0x7f74501338a0 2023-09-30T22:23:26.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74abfff700 1 -- 172.21.15.89:0/4285524126 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8005d70 con 0x7f74c409f240 2023-09-30T22:23:26.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74abfff700 1 -- 172.21.15.89:0/4285524126 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b80070d0 con 0x7f74c409f240 2023-09-30T22:23:26.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c40a0190 con 0x7f74c409f240 2023-09-30T22:23:26.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c41a55c0 con 0x7f74c409f240 2023-09-30T22:23:26.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.726+0000 7f74abfff700 1 -- 172.21.15.89:0/4285524126 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8007440 con 0x7f74c409f240 2023-09-30T22:23:26.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.727+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7450122300 con 0x7f74501338a0 2023-09-30T22:23:26.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.727+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a40052e0 con 0x7f74c409f240 2023-09-30T22:23:26.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.728+0000 7f74abfff700 1 -- 172.21.15.89:0/4285524126 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74b80075a0 con 0x7f74c409f240 2023-09-30T22:23:26.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.728+0000 7f744d7fa700 1 -- 172.21.15.89:0/2454996394 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7448014480 con 0x7f74501338a0 2023-09-30T22:23:26.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.728+0000 7f744d7fa700 1 -- 172.21.15.89:0/2454996394 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7448040810 con 0x7f74501338a0 2023-09-30T22:23:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.728+0000 7f74abfff700 1 -- 172.21.15.89:0/4285524126 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f74b803ead0 con 0x7f74c409f240 2023-09-30T22:23:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.733+0000 7f74abfff700 1 -- 172.21.15.89:0/4285524126 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74b8019b40 con 0x7f74c409f240 2023-09-30T22:23:26.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.737+0000 7f744d7fa700 1 -- 172.21.15.89:0/2454996394 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7448012930 con 0x7f74501338a0 2023-09-30T22:23:26.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.757+0000 7ff69bf3d700 1 Processor -- start 2023-09-30T22:23:26.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff69bf3d700 1 -- start start 2023-09-30T22:23:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff69bf3d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff69409e930 0x7ff69409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff69bf3d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff69bf3d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff694070940 con 0x7ff69409f650 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff69bf3d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69406fd60 con 0x7ff69409e930 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff69bf3d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff69406fa80 con 0x7ff69409f240 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff699cd9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff69409e930 0x7ff69409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff699cd9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff69409e930 0x7ff69409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35180/0 (socket says 172.21.15.89:35180) 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff699cd9700 1 -- 172.21.15.89:0/4158404733 learned_addr learned my addr 172.21.15.89:0/4158404733 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff6994d8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff6994d8700 1 -- 172.21.15.89:0/4158404733 >> v2:172.21.15.89:3300/0 conn(0x7ff69409e930 msgr2=0x7ff69409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff6994d8700 1 --2- 172.21.15.89:0/4158404733 >> v2:172.21.15.89:3300/0 conn(0x7ff69409e930 0x7ff69409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff6994d8700 1 -- 172.21.15.89:0/4158404733 >> v1:172.21.15.89:6789/0 conn(0x7ff69409f650 legacy=0x7ff694131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff698cd7700 1 -- 172.21.15.89:0/4158404733 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305975243 0 0) 0x7ff694070940 con 0x7ff69409f650 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.758+0000 7ff6994d8700 1 -- 172.21.15.89:0/4158404733 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69409fb60 con 0x7ff69409f240 2023-09-30T22:23:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.759+0000 7ff699cd9700 1 --2- 172.21.15.89:0/4158404733 >> v2:172.21.15.89:3300/0 conn(0x7ff69409e930 0x7ff69409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.759+0000 7ff6994d8700 1 --2- 172.21.15.89:0/4158404733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941317f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff684009840 tx=0x7ff684009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33396e76b2214998 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.759+0000 7ff698cd7700 1 -- 172.21.15.89:0/4158404733 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684011830 con 0x7ff69409f240 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.759+0000 7ff698cd7700 1 -- 172.21.15.89:0/4158404733 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff684011990 con 0x7ff69409f240 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.759+0000 7ff698cd7700 1 -- 172.21.15.89:0/4158404733 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684011d20 con 0x7ff69409f240 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.759+0000 7ff69bf3d700 1 -- 172.21.15.89:0/4158404733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 msgr2=0x7ff6941317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.759+0000 7ff69bf3d700 1 --2- 172.21.15.89:0/4158404733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941317f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff684009840 tx=0x7ff684009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.760+0000 7ff69bf3d700 1 -- 172.21.15.89:0/4158404733 shutdown_connections 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.760+0000 7ff69bf3d700 1 --2- 172.21.15.89:0/4158404733 >> v2:172.21.15.89:3300/0 conn(0x7ff69409e930 0x7ff69409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.760+0000 7ff69bf3d700 1 --2- 172.21.15.89:0/4158404733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941317f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.760+0000 7ff69bf3d700 1 -- 172.21.15.89:0/4158404733 >> 172.21.15.89:0/4158404733 conn(0x7ff6940a4210 msgr2=0x7ff6940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.760+0000 7ff69bf3d700 1 -- 172.21.15.89:0/4158404733 shutdown_connections 2023-09-30T22:23:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.760+0000 7ff69bf3d700 1 -- 172.21.15.89:0/4158404733 wait complete. 2023-09-30T22:23:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69bf3d700 1 Processor -- start 2023-09-30T22:23:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69bf3d700 1 -- start start 2023-09-30T22:23:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69bf3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69409e930 0x7ff6941c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69bf3d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69bf3d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff69409f650 0x7ff6941c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69bf3d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6941a7320 con 0x7ff69409e930 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69bf3d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6941a6540 con 0x7ff69409f650 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69bf3d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6941a6920 con 0x7ff69409f240 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff699cd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69409e930 0x7ff6941c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff699cd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69409e930 0x7ff6941c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35194/0 (socket says 172.21.15.89:35194) 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff699cd9700 1 -- 172.21.15.89:0/3783299876 learned_addr learned my addr 172.21.15.89:0/3783299876 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff6994d8700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff69a4da700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff69409f650 0x7ff6941c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff699cd9700 1 -- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 msgr2=0x7ff6941c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff699cd9700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff699cd9700 1 -- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff69409f650 msgr2=0x7ff6941c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff699cd9700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff69409f650 0x7ff6941c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.761+0000 7ff699cd9700 1 -- 172.21.15.89:0/3783299876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690009550 con 0x7ff69409e930 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.762+0000 7ff69a4da700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff69409f650 0x7ff6941c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.762+0000 7ff699cd9700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69409e930 0x7ff6941c63d0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7ff69000ec10 tx=0x7ff690009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.762+0000 7ff68a7fc700 1 -- 172.21.15.89:0/3783299876 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69000c8e0 con 0x7ff69409e930 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.762+0000 7ff68a7fc700 1 -- 172.21.15.89:0/3783299876 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69000ca40 con 0x7ff69409e930 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.762+0000 7ff68a7fc700 1 -- 172.21.15.89:0/3783299876 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69000cdb0 con 0x7ff69409e930 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.762+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6840094a0 con 0x7ff69409e930 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.762+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6940a0190 con 0x7ff69409e930 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.763+0000 7ff68a7fc700 1 -- 172.21.15.89:0/3783299876 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff690013470 con 0x7ff69409e930 2023-09-30T22:23:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.763+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff694074ea0 con 0x7ff69409e930 2023-09-30T22:23:26.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.763+0000 7ff68a7fc700 1 -- 172.21.15.89:0/3783299876 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff69003f9a0 con 0x7ff69409e930 2023-09-30T22:23:26.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.768+0000 7ff68a7fc700 1 -- 172.21.15.89:0/3783299876 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6900232f0 con 0x7ff69409e930 2023-09-30T22:23:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.775+0000 7f8a17e24700 1 Processor -- start 2023-09-30T22:23:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.775+0000 7f8a17e24700 1 -- start start 2023-09-30T22:23:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a17e24700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a1009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a17e24700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a100a0090 0x7f8a1009cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a17e24700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a10131970 0x7f8a10131e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a17e24700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1009ba70 con 0x7f8a10131970 2023-09-30T22:23:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a17e24700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1009d4a0 con 0x7f8a100a0090 2023-09-30T22:23:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a17e24700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1009e8f0 con 0x7f8a1009c650 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a1009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a153bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a100a0090 0x7f8a1009cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a1009ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47590/0 (socket says 172.21.15.89:47590) 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 -- 172.21.15.89:0/2488813679 learned_addr learned my addr 172.21.15.89:0/2488813679 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a153bf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a100a0090 0x7f8a1009cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60682/0 (socket says 172.21.15.89:60682) 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a163c1700 1 --2- 172.21.15.89:0/2488813679 >> v2:172.21.15.89:3300/0 conn(0x7f8a10131970 0x7f8a10131e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 -- 172.21.15.89:0/2488813679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a100a0090 msgr2=0x7f8a1009cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 --2- 172.21.15.89:0/2488813679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a100a0090 0x7f8a1009cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 -- 172.21.15.89:0/2488813679 >> v2:172.21.15.89:3300/0 conn(0x7f8a10131970 msgr2=0x7f8a10131e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 --2- 172.21.15.89:0/2488813679 >> v2:172.21.15.89:3300/0 conn(0x7f8a10131970 0x7f8a10131e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 -- 172.21.15.89:0/2488813679 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a100a0760 con 0x7f8a1009c650 2023-09-30T22:23:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.776+0000 7f8a15bc0700 1 --2- 172.21.15.89:0/2488813679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a1009ca20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009920 tx=0x7f8a0c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23eff775aeff50a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.777+0000 7f8a14bbe700 1 -- 172.21.15.89:0/2488813679 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c011830 con 0x7f8a1009c650 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.777+0000 7f8a14bbe700 1 -- 172.21.15.89:0/2488813679 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c011990 con 0x7f8a1009c650 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.777+0000 7f8a14bbe700 1 -- 172.21.15.89:0/2488813679 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c011d20 con 0x7f8a1009c650 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.777+0000 7f8a17e24700 1 -- 172.21.15.89:0/2488813679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 msgr2=0x7f8a1009ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.778+0000 7f8a17e24700 1 --2- 172.21.15.89:0/2488813679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a1009ca20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009920 tx=0x7f8a0c009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.778+0000 7f8a17e24700 1 -- 172.21.15.89:0/2488813679 shutdown_connections 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.778+0000 7f8a17e24700 1 --2- 172.21.15.89:0/2488813679 >> v2:172.21.15.89:3300/0 conn(0x7f8a10131970 0x7f8a10131e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.778+0000 7f8a17e24700 1 --2- 172.21.15.89:0/2488813679 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a1009ca20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.778+0000 7f8a17e24700 1 --2- 172.21.15.89:0/2488813679 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a100a0090 0x7f8a1009cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.778+0000 7f8a17e24700 1 -- 172.21.15.89:0/2488813679 >> 172.21.15.89:0/2488813679 conn(0x7f8a100a3f70 msgr2=0x7f8a100a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.778+0000 7f8a17e24700 1 -- 172.21.15.89:0/2488813679 shutdown_connections 2023-09-30T22:23:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.778+0000 7f8a17e24700 1 -- 172.21.15.89:0/2488813679 wait complete. 2023-09-30T22:23:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.779+0000 7f8a17e24700 1 Processor -- start 2023-09-30T22:23:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.779+0000 7f8a17e24700 1 -- start start 2023-09-30T22:23:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a17e24700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a10071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a17e24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a100a0090 0x7f8a100716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a17e24700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a10131970 0x7f8a10071be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a17e24700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a100725c0 con 0x7f8a100a0090 2023-09-30T22:23:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a17e24700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1012ff00 con 0x7f8a10131970 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a17e24700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1012c4a0 con 0x7f8a1009c650 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a15bc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a10071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a15bc0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a10071160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47604/0 (socket says 172.21.15.89:47604) 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a15bc0700 1 -- 172.21.15.89:0/1057726887 learned_addr learned my addr 172.21.15.89:0/1057726887 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a153bf700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a100a0090 0x7f8a100716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a163c1700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a10131970 0x7f8a10071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a15bc0700 1 -- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a10131970 msgr2=0x7f8a10071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a15bc0700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a10131970 0x7f8a10071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a15bc0700 1 -- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a100a0090 msgr2=0x7f8a100716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a15bc0700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a100a0090 0x7f8a100716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.780+0000 7f8a15bc0700 1 -- 172.21.15.89:0/1057726887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c009550 con 0x7f8a1009c650 2023-09-30T22:23:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.781+0000 7f8a15bc0700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a10071160 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c000c00 tx=0x7f8a0c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.781+0000 7f8a067fc700 1 -- 172.21.15.89:0/1057726887 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c004240 con 0x7f8a1009c650 2023-09-30T22:23:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.781+0000 7f8a067fc700 1 -- 172.21.15.89:0/1057726887 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c01f070 con 0x7f8a1009c650 2023-09-30T22:23:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.781+0000 7f8a067fc700 1 -- 172.21.15.89:0/1057726887 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0045c0 con 0x7f8a1009c650 2023-09-30T22:23:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.781+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1006c320 con 0x7f8a1009c650 2023-09-30T22:23:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.781+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1006c560 con 0x7f8a1009c650 2023-09-30T22:23:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.782+0000 7f8a067fc700 1 -- 172.21.15.89:0/1057726887 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a0c021030 con 0x7f8a1009c650 2023-09-30T22:23:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.782+0000 7f8a067fc700 1 -- 172.21.15.89:0/1057726887 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8a0c019570 con 0x7f8a1009c650 2023-09-30T22:23:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.782+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1006d7e0 con 0x7f8a1009c650 2023-09-30T22:23:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.791+0000 7f8a067fc700 1 -- 172.21.15.89:0/1057726887 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a0c015070 con 0x7f8a1009c650 2023-09-30T22:23:26.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.931+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f74a4005590 con 0x7f74c409f240 2023-09-30T22:23:26.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.931+0000 7f74abfff700 1 -- 172.21.15.89:0/4285524126 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74b8017070 con 0x7f74c409f240 2023-09-30T22:23:26.933 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.933+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74c409f240 msgr2=0x7f74c41a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.933+0000 7f74c9dc9700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74c409f240 0x7f74c41a3f40 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c550 tx=0x7f74b800c860 comp rx=0 tx=0).stop 2023-09-30T22:23:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.934+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 shutdown_connections 2023-09-30T22:23:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.934+0000 7f74c9dc9700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74c409f650 0x7f74c41a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.934+0000 7f74c9dc9700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74c409e930 0x7f74c41a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.934+0000 7f74c9dc9700 1 --2- 172.21.15.89:0/4285524126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74c409f240 0x7f74c41a3f40 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.934+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 >> 172.21.15.89:0/4285524126 conn(0x7f74c40a4210 msgr2=0x7f74c406cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:26.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.934+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 shutdown_connections 2023-09-30T22:23:26.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.934+0000 7f74c9dc9700 1 -- 172.21.15.89:0/4285524126 wait complete. 2023-09-30T22:23:26.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.966+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff6941a5810 con 0x7ff69409e930 2023-09-30T22:23:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.966+0000 7ff68a7fc700 1 -- 172.21.15.89:0/3783299876 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff690021020 con 0x7ff69409e930 2023-09-30T22:23:26.967 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69409e930 msgr2=0x7ff6941c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69409e930 0x7ff6941c63d0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7ff69000ec10 tx=0x7ff690009e70 comp rx=0 tx=0).stop 2023-09-30T22:23:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 shutdown_connections 2023-09-30T22:23:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff69409f650 0x7ff6941c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69409e930 0x7ff6941c63d0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 --2- 172.21.15.89:0/3783299876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69409f240 0x7ff6941c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 >> 172.21.15.89:0/3783299876 conn(0x7ff6940a4210 msgr2=0x7ff6941b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 shutdown_connections 2023-09-30T22:23:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:26.968+0000 7ff69bf3d700 1 -- 172.21.15.89:0/3783299876 wait complete. 2023-09-30T22:23:26.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.005+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f745006f950 con 0x7f74501338a0 2023-09-30T22:23:27.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.006+0000 7f744d7fa700 1 -- 172.21.15.89:0/2454996394 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f744802a090 con 0x7f74501338a0 2023-09-30T22:23:27.006 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.007+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74501338a0 msgr2=0x7f745011f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.007+0000 7f7456daf700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74501338a0 0x7f745011f620 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f744800efc0 tx=0x7f744800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.008+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 shutdown_connections 2023-09-30T22:23:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.008+0000 7f7456daf700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74501338a0 0x7f745011f620 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.008+0000 7f7456daf700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f745009de50 0x7f745011eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.008+0000 7f7456daf700 1 --2- 172.21.15.89:0/2454996394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f745009e760 0x7f745011f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.008+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 >> 172.21.15.89:0/2454996394 conn(0x7f74500a2aa0 msgr2=0x7f745009ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.008+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 shutdown_connections 2023-09-30T22:23:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.008+0000 7f7456daf700 1 -- 172.21.15.89:0/2454996394 wait complete. 2023-09-30T22:23:27.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:27.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.049+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8a100747f0 con 0x7f8a1009c650 2023-09-30T22:23:27.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.049+0000 7f8a067fc700 1 -- 172.21.15.89:0/1057726887 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a0c015020 con 0x7f8a1009c650 2023-09-30T22:23:27.050 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:27.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.051+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 msgr2=0x7f8a10071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:27.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.051+0000 7f8a17e24700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a10071160 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c000c00 tx=0x7f8a0c004750 comp rx=0 tx=0).stop 2023-09-30T22:23:27.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.051+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 shutdown_connections 2023-09-30T22:23:27.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.051+0000 7f8a17e24700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8a10131970 0x7f8a10071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:27.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.051+0000 7f8a17e24700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8a1009c650 0x7f8a10071160 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:27.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.051+0000 7f8a17e24700 1 --2- 172.21.15.89:0/1057726887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a100a0090 0x7f8a100716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:27.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.051+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 >> 172.21.15.89:0/1057726887 conn(0x7f8a100a3f70 msgr2=0x7f8a1012f050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:27.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.051+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 shutdown_connections 2023-09-30T22:23:27.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:27.052+0000 7f8a17e24700 1 -- 172.21.15.89:0/1057726887 wait complete. 2023-09-30T22:23:27.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:27.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:27.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:28.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:28.065 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:28.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f06807bc700 1 Processor -- start 2023-09-30T22:23:28.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f06807bc700 1 -- start start 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f06807bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f0678138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f06807bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678139030 0x7f067813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f06807bc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067806ef30 con 0x7f06781338a0 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f06807bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067806fef0 con 0x7f0678139030 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f06807bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06781334b0 con 0x7f0678136710 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f067e558700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f06781338a0 0x7f0678133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40128/0 (socket says 172.21.15.89:40128) 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f067e558700 1 -- 172.21.15.89:0/230540341 learned_addr learned my addr 172.21.15.89:0/230540341 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.095+0000 7f067dd57700 1 --2- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f0678138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067ed59700 1 --2- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678139030 0x7f067813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067dd57700 1 -- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678139030 msgr2=0x7f067813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:28.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067dd57700 1 --2- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678139030 0x7f067813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067dd57700 1 -- 172.21.15.89:0/230540341 >> v1:172.21.15.89:6789/0 conn(0x7f06781338a0 legacy=0x7f0678133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067dd57700 1 -- 172.21.15.89:0/230540341 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067806fc30 con 0x7f0678136710 2023-09-30T22:23:28.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067dd57700 1 --2- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f0678138af0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0668009870 tx=0x7f0668009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd1f47e1ad0ea73 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:28.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067d556700 1 -- 172.21.15.89:0/230540341 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668011830 con 0x7f0678136710 2023-09-30T22:23:28.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067d556700 1 -- 172.21.15.89:0/230540341 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0668011990 con 0x7f0678136710 2023-09-30T22:23:28.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.096+0000 7f067d556700 1 -- 172.21.15.89:0/230540341 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668011d20 con 0x7f0678136710 2023-09-30T22:23:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.097+0000 7f06807bc700 1 -- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 msgr2=0x7f0678138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.097+0000 7f06807bc700 1 --2- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f0678138af0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0668009870 tx=0x7f0668009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.097+0000 7f06807bc700 1 -- 172.21.15.89:0/230540341 shutdown_connections 2023-09-30T22:23:28.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.097+0000 7f06807bc700 1 --2- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0678139030 0x7f067813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.097+0000 7f06807bc700 1 --2- 172.21.15.89:0/230540341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f0678138af0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.097+0000 7f06807bc700 1 -- 172.21.15.89:0/230540341 >> 172.21.15.89:0/230540341 conn(0x7f06780a2aa0 msgr2=0x7f06780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.097+0000 7f06807bc700 1 -- 172.21.15.89:0/230540341 shutdown_connections 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.097+0000 7f06807bc700 1 -- 172.21.15.89:0/230540341 wait complete. 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.098+0000 7f06807bc700 1 Processor -- start 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.098+0000 7f06807bc700 1 -- start start 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.098+0000 7f06807bc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06781338a0 0x7f06781c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.098+0000 7f06807bc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f06781c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.098+0000 7f06807bc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678139030 0x7f067811ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.098+0000 7f06807bc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067811e690 con 0x7f0678139030 2023-09-30T22:23:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.098+0000 7f06807bc700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06781c9160 con 0x7f06781338a0 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.098+0000 7f06807bc700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0678138b00 con 0x7f0678136710 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f06781c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f06781c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47626/0 (socket says 172.21.15.89:47626) 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 -- 172.21.15.89:0/4047260449 learned_addr learned my addr 172.21.15.89:0/4047260449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067ed59700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678139030 0x7f067811ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067e558700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06781338a0 0x7f06781c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 -- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06781338a0 msgr2=0x7f06781c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06781338a0 0x7f06781c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 -- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678139030 msgr2=0x7f067811ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678139030 0x7f067811ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 -- 172.21.15.89:0/4047260449 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06680094a0 con 0x7f0678136710 2023-09-30T22:23:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f067dd57700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f06781c8c20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f06680090b0 tx=0x7f0668004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f066effd700 1 -- 172.21.15.89:0/4047260449 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668004590 con 0x7f0678136710 2023-09-30T22:23:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f066effd700 1 -- 172.21.15.89:0/4047260449 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066801f070 con 0x7f0678136710 2023-09-30T22:23:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f066effd700 1 -- 172.21.15.89:0/4047260449 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06680038e0 con 0x7f0678136710 2023-09-30T22:23:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067806f6a0 con 0x7f0678136710 2023-09-30T22:23:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.099+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067811e9e0 con 0x7f0678136710 2023-09-30T22:23:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.100+0000 7f066effd700 1 -- 172.21.15.89:0/4047260449 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0668003a40 con 0x7f0678136710 2023-09-30T22:23:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.101+0000 7f066effd700 1 -- 172.21.15.89:0/4047260449 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0668021070 con 0x7f0678136710 2023-09-30T22:23:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.101+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0678074ea0 con 0x7f0678136710 2023-09-30T22:23:28.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.106+0000 7f066effd700 1 -- 172.21.15.89:0/4047260449 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f066802b020 con 0x7f0678136710 2023-09-30T22:23:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.126+0000 7fb76e3ca700 1 Processor -- start 2023-09-30T22:23:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.126+0000 7fb76e3ca700 1 -- start start 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb76e3ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb76812cf60 0x7fb76806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb76e3ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb76e3ca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76812cd90 con 0x7fb76812f8b0 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb76e3ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76812d330 con 0x7fb76812cf60 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb76e3ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb768071780 con 0x7fb768130430 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb767fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb76812cf60 0x7fb76806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb767fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb76812cf60 0x7fb76806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35212/0 (socket says 172.21.15.89:35212) 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb7677fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb76812f8b0 0x7fb76812fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40140/0 (socket says 172.21.15.89:40140) 2023-09-30T22:23:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb767fff700 1 -- 172.21.15.89:0/484941095 learned_addr learned my addr 172.21.15.89:0/484941095 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.127+0000 7fb76c967700 1 --2- 172.21.15.89:0/484941095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb76c967700 1 -- 172.21.15.89:0/484941095 >> v2:172.21.15.89:3300/0 conn(0x7fb76812cf60 msgr2=0x7fb76806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb76c967700 1 --2- 172.21.15.89:0/484941095 >> v2:172.21.15.89:3300/0 conn(0x7fb76812cf60 0x7fb76806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb76c967700 1 -- 172.21.15.89:0/484941095 >> v1:172.21.15.89:6789/0 conn(0x7fb76812f8b0 legacy=0x7fb76812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb766ffd700 1 -- 172.21.15.89:0/484941095 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123031779 0 0) 0x7fb76812cd90 con 0x7fb76812f8b0 2023-09-30T22:23:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb76c967700 1 -- 172.21.15.89:0/484941095 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb768071a40 con 0x7fb768130430 2023-09-30T22:23:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb767fff700 1 --2- 172.21.15.89:0/484941095 >> v2:172.21.15.89:3300/0 conn(0x7fb76812cf60 0x7fb76806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb76c967700 1 --2- 172.21.15.89:0/484941095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768137c30 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb760009d90 tx=0x7fb76000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f23a4b8b5986036d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb766ffd700 1 -- 172.21.15.89:0/484941095 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000e070 con 0x7fb768130430 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb766ffd700 1 -- 172.21.15.89:0/484941095 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76000bb20 con 0x7fb768130430 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.128+0000 7fb766ffd700 1 -- 172.21.15.89:0/484941095 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760012430 con 0x7fb768130430 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.129+0000 7fb76e3ca700 1 -- 172.21.15.89:0/484941095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 msgr2=0x7fb768137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.129+0000 7fb76e3ca700 1 --2- 172.21.15.89:0/484941095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768137c30 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb760009d90 tx=0x7fb76000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.129+0000 7fb76e3ca700 1 -- 172.21.15.89:0/484941095 shutdown_connections 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.129+0000 7fb76e3ca700 1 --2- 172.21.15.89:0/484941095 >> v2:172.21.15.89:3300/0 conn(0x7fb76812cf60 0x7fb76806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.129+0000 7fb76e3ca700 1 --2- 172.21.15.89:0/484941095 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768137c30 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.129+0000 7fb76e3ca700 1 -- 172.21.15.89:0/484941095 >> 172.21.15.89:0/484941095 conn(0x7fb7680a2aa0 msgr2=0x7fb7680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.129+0000 7fb76e3ca700 1 -- 172.21.15.89:0/484941095 shutdown_connections 2023-09-30T22:23:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.129+0000 7fb76e3ca700 1 -- 172.21.15.89:0/484941095 wait complete. 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.130+0000 7fb76e3ca700 1 Processor -- start 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.130+0000 7fb76e3ca700 1 -- start start 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.130+0000 7fb76e3ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb76812cf60 0x7fb7681277b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.130+0000 7fb76e3ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb76812f8b0 0x7fb768127cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.130+0000 7fb76e3ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768128230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.130+0000 7fb76e3ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768129700 con 0x7fb76812cf60 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.130+0000 7fb76e3ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb768125dd0 con 0x7fb76812f8b0 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.130+0000 7fb76e3ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb768128860 con 0x7fb768130430 2023-09-30T22:23:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb76812cf60 0x7fb7681277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb76812cf60 0x7fb7681277b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35220/0 (socket says 172.21.15.89:35220) 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 -- 172.21.15.89:0/3551643837 learned_addr learned my addr 172.21.15.89:0/3551643837 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb76c967700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768128230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 -- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 msgr2=0x7fb768128230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768128230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 -- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb76812f8b0 msgr2=0x7fb768127cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb76812f8b0 0x7fb768127cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 -- 172.21.15.89:0/3551643837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758009550 con 0x7fb76812cf60 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb76c967700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768128230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb767fff700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb76812cf60 0x7fb7681277b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb75800ebe0 tx=0x7fb75800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb764ff9700 1 -- 172.21.15.89:0/3551643837 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75800c850 con 0x7fb76812cf60 2023-09-30T22:23:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb764ff9700 1 -- 172.21.15.89:0/3551643837 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75800c9b0 con 0x7fb76812cf60 2023-09-30T22:23:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb764ff9700 1 -- 172.21.15.89:0/3551643837 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75800cd20 con 0x7fb76812cf60 2023-09-30T22:23:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.131+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb760009a70 con 0x7fb76812cf60 2023-09-30T22:23:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.132+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb768129fa0 con 0x7fb76812cf60 2023-09-30T22:23:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.132+0000 7fb764ff9700 1 -- 172.21.15.89:0/3551643837 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb758013470 con 0x7fb76812cf60 2023-09-30T22:23:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.133+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb768074ea0 con 0x7fb76812cf60 2023-09-30T22:23:28.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.133+0000 7fb764ff9700 1 -- 172.21.15.89:0/3551643837 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb75803fc80 con 0x7fb76812cf60 2023-09-30T22:23:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.142+0000 7fb764ff9700 1 -- 172.21.15.89:0/3551643837 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb758021020 con 0x7fb76812cf60 2023-09-30T22:23:28.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.181+0000 7f610b3d3700 1 Processor -- start 2023-09-30T22:23:28.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.181+0000 7f610b3d3700 1 -- start start 2023-09-30T22:23:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f610b3d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f610409d630 0x7f61040a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f610b3d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6104138d80 0x7f610413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f610b3d3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61040a2a70 con 0x7f61040a0a30 2023-09-30T22:23:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f610b3d3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61040a01d0 con 0x7f610409d630 2023-09-30T22:23:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f610b3d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f610409e2b0 con 0x7f6104138d80 2023-09-30T22:23:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f610916f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61040a0a30 0x7f61040a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40148/0 (socket says 172.21.15.89:40148) 2023-09-30T22:23:28.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f610916f700 1 -- 172.21.15.89:0/264688337 learned_addr learned my addr 172.21.15.89:0/264688337 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:28.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f610896e700 1 --2- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f610409d630 0x7f61040a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.182+0000 7f6109970700 1 --2- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6104138d80 0x7f610413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f610896e700 1 -- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6104138d80 msgr2=0x7f610413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f610896e700 1 --2- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6104138d80 0x7f610413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f610896e700 1 -- 172.21.15.89:0/264688337 >> v1:172.21.15.89:6789/0 conn(0x7f61040a0a30 legacy=0x7f61040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f60fbfff700 1 -- 172.21.15.89:0/264688337 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644140594 0 0) 0x7f61040a2a70 con 0x7f61040a0a30 2023-09-30T22:23:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f610896e700 1 -- 172.21.15.89:0/264688337 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610409dc50 con 0x7f610409d630 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f610896e700 1 --2- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f610409d630 0x7f61040a1510 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009870 tx=0x7f60f4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=822bfca8230351a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f60fbfff700 1 -- 172.21.15.89:0/264688337 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f4011830 con 0x7f610409d630 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f60fbfff700 1 -- 172.21.15.89:0/264688337 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f4011990 con 0x7f610409d630 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.183+0000 7f60fbfff700 1 -- 172.21.15.89:0/264688337 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f4011d20 con 0x7f610409d630 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.184+0000 7f610b3d3700 1 -- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f610409d630 msgr2=0x7f61040a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.184+0000 7f610b3d3700 1 --2- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f610409d630 0x7f61040a1510 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009870 tx=0x7f60f4009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.184+0000 7f610b3d3700 1 -- 172.21.15.89:0/264688337 shutdown_connections 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.184+0000 7f610b3d3700 1 --2- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6104138d80 0x7f610413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.184+0000 7f610b3d3700 1 --2- 172.21.15.89:0/264688337 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f610409d630 0x7f61040a1510 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.184+0000 7f610b3d3700 1 -- 172.21.15.89:0/264688337 >> 172.21.15.89:0/264688337 conn(0x7f61040a3050 msgr2=0x7f61040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.184+0000 7f610b3d3700 1 -- 172.21.15.89:0/264688337 shutdown_connections 2023-09-30T22:23:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.184+0000 7f610b3d3700 1 -- 172.21.15.89:0/264688337 wait complete. 2023-09-30T22:23:28.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.185+0000 7f610b3d3700 1 Processor -- start 2023-09-30T22:23:28.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.185+0000 7f610b3d3700 1 -- start start 2023-09-30T22:23:28.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f610b3d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f610409d630 0x7f610411c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f610b3d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61040a0a30 0x7f610411cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f610b3d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6104138d80 0x7f610411d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f610b3d3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610413adb0 con 0x7f6104138d80 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f610b3d3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610412f6b0 con 0x7f61040a0a30 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f610b3d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61040a0410 con 0x7f610409d630 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f6109970700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6104138d80 0x7f610411d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f6109970700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6104138d80 0x7f610411d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35236/0 (socket says 172.21.15.89:35236) 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f6109970700 1 -- 172.21.15.89:0/3492926397 learned_addr learned my addr 172.21.15.89:0/3492926397 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f610916f700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f610409d630 0x7f610411c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.186+0000 7f610896e700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61040a0a30 0x7f610411cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f610916f700 1 -- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61040a0a30 msgr2=0x7f610411cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f610916f700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61040a0a30 0x7f610411cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f610916f700 1 -- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6104138d80 msgr2=0x7f610411d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f610916f700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6104138d80 0x7f610411d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f610916f700 1 -- 172.21.15.89:0/3492926397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f40094a0 con 0x7f610409d630 2023-09-30T22:23:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f610916f700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f610409d630 0x7f610411c7c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f610000c470 tx=0x7f610000c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f60f9ffb700 1 -- 172.21.15.89:0/3492926397 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6100004c80 con 0x7f610409d630 2023-09-30T22:23:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61040716e0 con 0x7f610409d630 2023-09-30T22:23:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f60f9ffb700 1 -- 172.21.15.89:0/3492926397 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6100004de0 con 0x7f610409d630 2023-09-30T22:23:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f60f9ffb700 1 -- 172.21.15.89:0/3492926397 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61000072f0 con 0x7f610409d630 2023-09-30T22:23:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.187+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610411f1e0 con 0x7f610409d630 2023-09-30T22:23:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.188+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61040747f0 con 0x7f610409d630 2023-09-30T22:23:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.189+0000 7f60f9ffb700 1 -- 172.21.15.89:0/3492926397 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6100007450 con 0x7f610409d630 2023-09-30T22:23:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.189+0000 7f60f9ffb700 1 -- 172.21.15.89:0/3492926397 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f610003f4a0 con 0x7f610409d630 2023-09-30T22:23:28.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.198+0000 7f60f9ffb700 1 -- 172.21.15.89:0/3492926397 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6100007700 con 0x7f610409d630 2023-09-30T22:23:28.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.227+0000 7f703d35e700 1 Processor -- start 2023-09-30T22:23:28.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.227+0000 7f703d35e700 1 -- start start 2023-09-30T22:23:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f703d35e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7038130540 0x7f7038130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f703d35e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7038130e50 0x7f703806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f703d35e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70380716a0 con 0x7f703806fb20 2023-09-30T22:23:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f703d35e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703809a390 con 0x7f7038130540 2023-09-30T22:23:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f703d35e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7038071290 con 0x7f7038130e50 2023-09-30T22:23:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f7036ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7038130540 0x7f7038130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f70367fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7038130e50 0x7f703806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f7036ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7038130540 0x7f7038130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35252/0 (socket says 172.21.15.89:35252) 2023-09-30T22:23:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f7036ffd700 1 -- 172.21.15.89:0/3991766563 learned_addr learned my addr 172.21.15.89:0/3991766563 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:28.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f70367fc700 1 -- 172.21.15.89:0/3991766563 >> v2:172.21.15.89:3300/0 conn(0x7f7038130540 msgr2=0x7f7038130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f70367fc700 1 --2- 172.21.15.89:0/3991766563 >> v2:172.21.15.89:3300/0 conn(0x7f7038130540 0x7f7038130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f70367fc700 1 -- 172.21.15.89:0/3991766563 >> v1:172.21.15.89:6789/0 conn(0x7f703806fb20 legacy=0x7f7038137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f70367fc700 1 -- 172.21.15.89:0/3991766563 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038130340 con 0x7f7038130e50 2023-09-30T22:23:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f7036ffd700 1 --2- 172.21.15.89:0/3991766563 >> v2:172.21.15.89:3300/0 conn(0x7f7038130540 0x7f7038130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f70367fc700 1 --2- 172.21.15.89:0/3991766563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7038130e50 0x7f703806f5e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f702c009870 tx=0x7f702c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9be18b7d7995f637 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.228+0000 7f7035ffb700 1 -- 172.21.15.89:0/3991766563 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c010880 con 0x7f7038130e50 2023-09-30T22:23:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f7035ffb700 1 -- 172.21.15.89:0/3991766563 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c0109e0 con 0x7f7038130e50 2023-09-30T22:23:28.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f7035ffb700 1 -- 172.21.15.89:0/3991766563 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c010d70 con 0x7f7038130e50 2023-09-30T22:23:28.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f703d35e700 1 -- 172.21.15.89:0/3991766563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7038130e50 msgr2=0x7f703806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f703d35e700 1 --2- 172.21.15.89:0/3991766563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7038130e50 0x7f703806f5e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f702c009870 tx=0x7f702c009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:28.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f703d35e700 1 -- 172.21.15.89:0/3991766563 shutdown_connections 2023-09-30T22:23:28.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f703d35e700 1 --2- 172.21.15.89:0/3991766563 >> v2:172.21.15.89:3300/0 conn(0x7f7038130540 0x7f7038130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f703d35e700 1 --2- 172.21.15.89:0/3991766563 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7038130e50 0x7f703806f5e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f703d35e700 1 -- 172.21.15.89:0/3991766563 >> 172.21.15.89:0/3991766563 conn(0x7f70380a2aa0 msgr2=0x7f70380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.229+0000 7f703d35e700 1 -- 172.21.15.89:0/3991766563 shutdown_connections 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.230+0000 7f703d35e700 1 -- 172.21.15.89:0/3991766563 wait complete. 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.231+0000 7f703d35e700 1 Processor -- start 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.231+0000 7f703d35e700 1 -- start start 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f703d35e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f703806fb20 0x7f703811a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f703d35e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038130540 0x7f703811ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f703d35e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7038130e50 0x7f703811b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f703d35e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703811ba00 con 0x7f7038130540 2023-09-30T22:23:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f703d35e700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703811d020 con 0x7f7038130e50 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f703d35e700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f703811c2c0 con 0x7f703806fb20 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f703806fb20 0x7f703811a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f703806fb20 0x7f703811a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47668/0 (socket says 172.21.15.89:47668) 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 -- 172.21.15.89:0/1398692300 learned_addr learned my addr 172.21.15.89:0/1398692300 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f70367fc700 1 --2- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038130540 0x7f703811ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 -- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7038130e50 msgr2=0x7f703811b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 --2- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7038130e50 0x7f703811b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 -- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038130540 msgr2=0x7f703811ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 --2- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038130540 0x7f703811ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 -- 172.21.15.89:0/1398692300 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028009550 con 0x7f703806fb20 2023-09-30T22:23:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f7036ffd700 1 --2- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f703806fb20 0x7f703811a610 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7028010ef0 tx=0x7f702800b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.232+0000 7f701f7fe700 1 -- 172.21.15.89:0/1398692300 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028015070 con 0x7f703806fb20 2023-09-30T22:23:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.233+0000 7f701f7fe700 1 -- 172.21.15.89:0/1398692300 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702800bc40 con 0x7f703806fb20 2023-09-30T22:23:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.233+0000 7f701f7fe700 1 -- 172.21.15.89:0/1398692300 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028019410 con 0x7f703806fb20 2023-09-30T22:23:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.233+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702c0094a0 con 0x7f703806fb20 2023-09-30T22:23:28.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.233+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70380718e0 con 0x7f703806fb20 2023-09-30T22:23:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.234+0000 7f701f7fe700 1 -- 172.21.15.89:0/1398692300 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7028019570 con 0x7f703806fb20 2023-09-30T22:23:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.234+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703809cd40 con 0x7f703806fb20 2023-09-30T22:23:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.234+0000 7f701f7fe700 1 -- 172.21.15.89:0/1398692300 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7028043210 con 0x7f703806fb20 2023-09-30T22:23:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.239+0000 7f701f7fe700 1 -- 172.21.15.89:0/1398692300 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7028019820 con 0x7f703806fb20 2023-09-30T22:23:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.316+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f067806b9e0 con 0x7f0678136710 2023-09-30T22:23:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.317+0000 7f066effd700 1 -- 172.21.15.89:0/4047260449 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0668015070 con 0x7f0678136710 2023-09-30T22:23:28.318 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.319+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 msgr2=0x7f06781c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.319+0000 7f06807bc700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f06781c8c20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f06680090b0 tx=0x7f0668004750 comp rx=0 tx=0).stop 2023-09-30T22:23:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.319+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 shutdown_connections 2023-09-30T22:23:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.319+0000 7f06807bc700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0678139030 0x7f067811ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.319+0000 7f06807bc700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f06781338a0 0x7f06781c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.319+0000 7f06807bc700 1 --2- 172.21.15.89:0/4047260449 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0678136710 0x7f06781c8c20 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.319+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 >> 172.21.15.89:0/4047260449 conn(0x7f06780a2aa0 msgr2=0x7f0678137270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.320+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 shutdown_connections 2023-09-30T22:23:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.320+0000 7f06807bc700 1 -- 172.21.15.89:0/4047260449 wait complete. 2023-09-30T22:23:28.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:28.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.343+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb7680713a0 con 0x7fb76812cf60 2023-09-30T22:23:28.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.343+0000 7fb764ff9700 1 -- 172.21.15.89:0/3551643837 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb758017070 con 0x7fb76812cf60 2023-09-30T22:23:28.344 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:28.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb76812cf60 msgr2=0x7fb7681277b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb76812cf60 0x7fb7681277b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb75800ebe0 tx=0x7fb75800eef0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 shutdown_connections 2023-09-30T22:23:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb768130430 0x7fb768128230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb76812cf60 0x7fb7681277b0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 --2- 172.21.15.89:0/3551643837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb76812f8b0 0x7fb768127cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 >> 172.21.15.89:0/3551643837 conn(0x7fb7680a2aa0 msgr2=0x7fb76806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:28.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 shutdown_connections 2023-09-30T22:23:28.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.345+0000 7fb76e3ca700 1 -- 172.21.15.89:0/3551643837 wait complete. 2023-09-30T22:23:28.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.409+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6104072980 con 0x7f610409d630 2023-09-30T22:23:28.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.409+0000 7f60f9ffb700 1 -- 172.21.15.89:0/3492926397 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6100011e80 con 0x7f610409d630 2023-09-30T22:23:28.410 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f610409d630 msgr2=0x7f610411c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f610409d630 0x7f610411c7c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f610000c470 tx=0x7f610000c780 comp rx=0 tx=0).stop 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 shutdown_connections 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6104138d80 0x7f610411d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f610409d630 0x7f610411c7c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 --2- 172.21.15.89:0/3492926397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61040a0a30 0x7f610411cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 >> 172.21.15.89:0/3492926397 conn(0x7f61040a3050 msgr2=0x7f610409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 shutdown_connections 2023-09-30T22:23:28.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.412+0000 7f610b3d3700 1 -- 172.21.15.89:0/3492926397 wait complete. 2023-09-30T22:23:28.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.435+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7038074ea0 con 0x7f703806fb20 2023-09-30T22:23:28.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.436+0000 7f701f7fe700 1 -- 172.21.15.89:0/1398692300 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f702801e090 con 0x7f703806fb20 2023-09-30T22:23:28.436 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:28.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.437+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f703806fb20 msgr2=0x7f703811a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:28.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.437+0000 7f703d35e700 1 --2- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f703806fb20 0x7f703811a610 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7028010ef0 tx=0x7f702800b420 comp rx=0 tx=0).stop 2023-09-30T22:23:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.437+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 shutdown_connections 2023-09-30T22:23:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.437+0000 7f703d35e700 1 --2- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7038130e50 0x7f703811b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.437+0000 7f703d35e700 1 --2- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f703806fb20 0x7f703811a610 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.437+0000 7f703d35e700 1 --2- 172.21.15.89:0/1398692300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7038130540 0x7f703811ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.437+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 >> 172.21.15.89:0/1398692300 conn(0x7f70380a2aa0 msgr2=0x7f703812d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.438+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 shutdown_connections 2023-09-30T22:23:28.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:28.438+0000 7f703d35e700 1 -- 172.21.15.89:0/1398692300 wait complete. 2023-09-30T22:23:28.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:29.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:29.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:29.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:29.451 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:29.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.486+0000 7f30c4188700 1 Processor -- start 2023-09-30T22:23:29.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c4188700 1 -- start start 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c4188700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30bc06a770 0x7f30bc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c4188700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 0x7f30bc139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c4188700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30bc09ece0 con 0x7f30bc06b080 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c4188700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30bc09e760 con 0x7f30bc06a770 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c4188700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc09e040 con 0x7f30bc131d10 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c1f24700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30bc06a770 0x7f30bc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c1f24700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30bc06a770 0x7f30bc06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35274/0 (socket says 172.21.15.89:35274) 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c1f24700 1 -- 172.21.15.89:0/1247123416 learned_addr learned my addr 172.21.15.89:0/1247123416 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.487+0000 7f30c2725700 1 --2- 172.21.15.89:0/1247123416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 0x7f30bc139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c0f22700 1 -- 172.21.15.89:0/1247123416 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999547682 0 0) 0x7f30bc09ece0 con 0x7f30bc06b080 2023-09-30T22:23:29.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c1f24700 1 -- 172.21.15.89:0/1247123416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 msgr2=0x7f30bc139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:29.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c1f24700 1 --2- 172.21.15.89:0/1247123416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 0x7f30bc139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c1f24700 1 -- 172.21.15.89:0/1247123416 >> v1:172.21.15.89:6789/0 conn(0x7f30bc06b080 legacy=0x7f30bc131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c1f24700 1 -- 172.21.15.89:0/1247123416 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc09f630 con 0x7f30bc06a770 2023-09-30T22:23:29.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c1f24700 1 --2- 172.21.15.89:0/1247123416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 0x7f30bc06ab40 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f30b800dd10 tx=0x7f30b800f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9cd693dd257969 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:29.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c0f22700 1 -- 172.21.15.89:0/1247123416 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8012070 con 0x7f30bc06a770 2023-09-30T22:23:29.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c0f22700 1 -- 172.21.15.89:0/1247123416 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b800d2e0 con 0x7f30bc06a770 2023-09-30T22:23:29.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.488+0000 7f30c0f22700 1 -- 172.21.15.89:0/1247123416 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800fb10 con 0x7f30bc06a770 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.489+0000 7f30c4188700 1 -- 172.21.15.89:0/1247123416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 msgr2=0x7f30bc06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.489+0000 7f30c4188700 1 --2- 172.21.15.89:0/1247123416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 0x7f30bc06ab40 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f30b800dd10 tx=0x7f30b800f330 comp rx=0 tx=0).stop 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.489+0000 7f30c4188700 1 -- 172.21.15.89:0/1247123416 shutdown_connections 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.489+0000 7f30c4188700 1 --2- 172.21.15.89:0/1247123416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 0x7f30bc139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.489+0000 7f30c4188700 1 --2- 172.21.15.89:0/1247123416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 0x7f30bc06ab40 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.489+0000 7f30c4188700 1 -- 172.21.15.89:0/1247123416 >> 172.21.15.89:0/1247123416 conn(0x7f30bc066f50 msgr2=0x7f30bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.489+0000 7f30c4188700 1 -- 172.21.15.89:0/1247123416 shutdown_connections 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.489+0000 7f30c4188700 1 -- 172.21.15.89:0/1247123416 wait complete. 2023-09-30T22:23:29.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c4188700 1 Processor -- start 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c4188700 1 -- start start 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c4188700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 0x7f30bc11e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c4188700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30bc06b080 0x7f30bc11e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c4188700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 0x7f30bc11ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c4188700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc11f760 con 0x7f30bc06a770 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c4188700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc121ad0 con 0x7f30bc131d10 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c4188700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc11ffb0 con 0x7f30bc06b080 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.490+0000 7f30c1723700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30bc06b080 0x7f30bc11e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1f24700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 0x7f30bc11e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1723700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30bc06b080 0x7f30bc11e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47674/0 (socket says 172.21.15.89:47674) 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1723700 1 -- 172.21.15.89:0/3426538138 learned_addr learned my addr 172.21.15.89:0/3426538138 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c2725700 1 --2- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 0x7f30bc11ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1723700 1 -- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 msgr2=0x7f30bc11ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1723700 1 --2- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 0x7f30bc11ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1723700 1 -- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 msgr2=0x7f30bc11e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1723700 1 --2- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 0x7f30bc11e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1723700 1 -- 172.21.15.89:0/3426538138 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b800d9f0 con 0x7f30bc06b080 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30c1723700 1 --2- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30bc06b080 0x7f30bc11e7d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c750 tx=0x7f30ac00ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30b27fc700 1 -- 172.21.15.89:0/3426538138 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0054e0 con 0x7f30bc06b080 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30b27fc700 1 -- 172.21.15.89:0/3426538138 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac003680 con 0x7f30bc06b080 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.491+0000 7f30b27fc700 1 -- 172.21.15.89:0/3426538138 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac003a30 con 0x7f30bc06b080 2023-09-30T22:23:29.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.492+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc1204b0 con 0x7f30bc06b080 2023-09-30T22:23:29.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.492+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc122540 con 0x7f30bc06b080 2023-09-30T22:23:29.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.493+0000 7f30b27fc700 1 -- 172.21.15.89:0/3426538138 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f30ac003b90 con 0x7f30bc06b080 2023-09-30T22:23:29.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.493+0000 7f30b27fc700 1 -- 172.21.15.89:0/3426538138 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f30ac03ea10 con 0x7f30bc06b080 2023-09-30T22:23:29.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.493+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30bc120450 con 0x7f30bc06b080 2023-09-30T22:23:29.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.498+0000 7f30b27fc700 1 -- 172.21.15.89:0/3426538138 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f30ac017080 con 0x7f30bc06b080 2023-09-30T22:23:29.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.527+0000 7f169b0d3700 1 Processor -- start 2023-09-30T22:23:29.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.527+0000 7f169b0d3700 1 -- start start 2023-09-30T22:23:29.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f169b0d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f169406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f169b0d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f1694137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f169b0d3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169412cd90 con 0x7f169412f8b0 2023-09-30T22:23:29.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f169b0d3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169412d330 con 0x7f1694130430 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f169b0d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1694071780 con 0x7f169412cf60 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f1693fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f169412f8b0 0x7f169412fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40164/0 (socket says 172.21.15.89:40164) 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f1693fff700 1 -- 172.21.15.89:0/920517379 learned_addr learned my addr 172.21.15.89:0/920517379 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f1699670700 1 --2- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f1694137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.528+0000 7f1698e6f700 1 --2- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f169406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f16937fe700 1 -- 172.21.15.89:0/920517379 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108794268 0 0) 0x7f169412cd90 con 0x7f169412f8b0 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f16937fe700 1 -- 172.21.15.89:0/920517379 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16840035b0 con 0x7f169412f8b0 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f1699670700 1 -- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 msgr2=0x7f169406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f1699670700 1 --2- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f169406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f1699670700 1 -- 172.21.15.89:0/920517379 >> v1:172.21.15.89:6789/0 conn(0x7f169412f8b0 legacy=0x7f169412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f1699670700 1 -- 172.21.15.89:0/920517379 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1694071a40 con 0x7f1694130430 2023-09-30T22:23:29.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f1698e6f700 1 --2- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f169406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:29.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f1699670700 1 --2- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f1694137c30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f168c009df0 tx=0x7f168c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c030bdf3ff4e1a29 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:29.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f16937fe700 1 -- 172.21.15.89:0/920517379 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c00e070 con 0x7f1694130430 2023-09-30T22:23:29.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f16937fe700 1 -- 172.21.15.89:0/920517379 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c00bae0 con 0x7f1694130430 2023-09-30T22:23:29.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.529+0000 7f16937fe700 1 -- 172.21.15.89:0/920517379 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c00be90 con 0x7f1694130430 2023-09-30T22:23:29.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.530+0000 7f169b0d3700 1 -- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 msgr2=0x7f1694137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.530+0000 7f169b0d3700 1 --2- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f1694137c30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f168c009df0 tx=0x7f168c00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.530+0000 7f169b0d3700 1 -- 172.21.15.89:0/920517379 shutdown_connections 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.530+0000 7f169b0d3700 1 --2- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f1694137c30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.530+0000 7f169b0d3700 1 --2- 172.21.15.89:0/920517379 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f169406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.530+0000 7f169b0d3700 1 -- 172.21.15.89:0/920517379 >> 172.21.15.89:0/920517379 conn(0x7f16940a2aa0 msgr2=0x7f16940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.530+0000 7f169b0d3700 1 -- 172.21.15.89:0/920517379 shutdown_connections 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.530+0000 7f169b0d3700 1 -- 172.21.15.89:0/920517379 wait complete. 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.532+0000 7f169b0d3700 1 Processor -- start 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.532+0000 7f169b0d3700 1 -- start start 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.532+0000 7f169b0d3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f16941c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.532+0000 7f169b0d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169412f8b0 0x7f16941c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.532+0000 7f169b0d3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f169409b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.532+0000 7f169b0d3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16941299c0 con 0x7f169412f8b0 2023-09-30T22:23:29.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f169b0d3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1694067b50 con 0x7f1694130430 2023-09-30T22:23:29.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f169b0d3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1694129fa0 con 0x7f169412cf60 2023-09-30T22:23:29.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f169409b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f169409b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60780/0 (socket says 172.21.15.89:60780) 2023-09-30T22:23:29.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 -- 172.21.15.89:0/2554151109 learned_addr learned my addr 172.21.15.89:0/2554151109 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:29.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1693fff700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169412f8b0 0x7f16941c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1698e6f700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f16941c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 -- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 msgr2=0x7f16941c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f16941c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 -- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169412f8b0 msgr2=0x7f16941c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169412f8b0 0x7f16941c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 -- 172.21.15.89:0/2554151109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16880094a0 con 0x7f1694130430 2023-09-30T22:23:29.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1699670700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f169409b1e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f168c00b370 tx=0x7f168c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:29.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f1698e6f700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f16941c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:29.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f16917fa700 1 -- 172.21.15.89:0/2554151109 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c0046a0 con 0x7f1694130430 2023-09-30T22:23:29.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.533+0000 7f16917fa700 1 -- 172.21.15.89:0/2554151109 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c020070 con 0x7f1694130430 2023-09-30T22:23:29.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.534+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c009a70 con 0x7f1694130430 2023-09-30T22:23:29.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.534+0000 7f16917fa700 1 -- 172.21.15.89:0/2554151109 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c003e10 con 0x7f1694130430 2023-09-30T22:23:29.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.534+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16941c4c10 con 0x7f1694130430 2023-09-30T22:23:29.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.534+0000 7f16917fa700 1 -- 172.21.15.89:0/2554151109 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f168c022030 con 0x7f1694130430 2023-09-30T22:23:29.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.535+0000 7f16917fa700 1 -- 172.21.15.89:0/2554151109 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f168c0414b0 con 0x7f1694130430 2023-09-30T22:23:29.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.535+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169409eb70 con 0x7f1694130430 2023-09-30T22:23:29.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.539+0000 7f16917fa700 1 -- 172.21.15.89:0/2554151109 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f168c0247e0 con 0x7f1694130430 2023-09-30T22:23:29.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.566+0000 7fc4fcc74700 1 Processor -- start 2023-09-30T22:23:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.566+0000 7fc4fcc74700 1 -- start start 2023-09-30T22:23:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.566+0000 7fc4fcc74700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f809de00 0x7fc4f809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.566+0000 7fc4fcc74700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.566+0000 7fc4fcc74700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f809a810 con 0x7fc4f806a770 2023-09-30T22:23:29.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.566+0000 7fc4fcc74700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f806ab40 con 0x7fc4f809de00 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.566+0000 7fc4fcc74700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f809ebc0 con 0x7fc4f813d1f0 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.566+0000 7fc4f6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f5d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f809de00 0x7fc4f809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f6d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47702/0 (socket says 172.21.15.89:47702) 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f6d9d700 1 -- 172.21.15.89:0/4193894828 learned_addr learned my addr 172.21.15.89:0/4193894828 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f5d9b700 1 -- 172.21.15.89:0/4193894828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 msgr2=0x7fc4f813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f5d9b700 1 --2- 172.21.15.89:0/4193894828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f5d9b700 1 -- 172.21.15.89:0/4193894828 >> v1:172.21.15.89:6789/0 conn(0x7fc4f806a770 legacy=0x7fc4f809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f5d9b700 1 -- 172.21.15.89:0/4193894828 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f809e920 con 0x7fc4f809de00 2023-09-30T22:23:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f5d9b700 1 --2- 172.21.15.89:0/4193894828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f809de00 0x7fc4f809e250 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009870 tx=0x7fc4ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a950d2134ceaf93 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f559a700 1 -- 172.21.15.89:0/4193894828 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec011830 con 0x7fc4f809de00 2023-09-30T22:23:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f559a700 1 -- 172.21.15.89:0/4193894828 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec011990 con 0x7fc4f809de00 2023-09-30T22:23:29.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.567+0000 7fc4f559a700 1 -- 172.21.15.89:0/4193894828 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec011d20 con 0x7fc4f809de00 2023-09-30T22:23:29.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.568+0000 7fc4fcc74700 1 -- 172.21.15.89:0/4193894828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f809de00 msgr2=0x7fc4f809e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.568+0000 7fc4fcc74700 1 --2- 172.21.15.89:0/4193894828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f809de00 0x7fc4f809e250 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009870 tx=0x7fc4ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.568+0000 7fc4fcc74700 1 -- 172.21.15.89:0/4193894828 shutdown_connections 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.568+0000 7fc4fcc74700 1 --2- 172.21.15.89:0/4193894828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.568+0000 7fc4fcc74700 1 --2- 172.21.15.89:0/4193894828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f809de00 0x7fc4f809e250 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.568+0000 7fc4fcc74700 1 -- 172.21.15.89:0/4193894828 >> 172.21.15.89:0/4193894828 conn(0x7fc4f8066f50 msgr2=0x7fc4f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.568+0000 7fc4fcc74700 1 -- 172.21.15.89:0/4193894828 shutdown_connections 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.568+0000 7fc4fcc74700 1 -- 172.21.15.89:0/4193894828 wait complete. 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.569+0000 7fc4fcc74700 1 Processor -- start 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.569+0000 7fc4fcc74700 1 -- start start 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4fcc74700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f806a770 0x7fc4f813b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4fcc74700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4f809de00 0x7fc4f813bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4fcc74700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4fcc74700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8135e00 con 0x7fc4f809de00 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4fcc74700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f811df20 con 0x7fc4f806a770 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4fcc74700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8072370 con 0x7fc4f813d1f0 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f806a770 0x7fc4f813b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f806a770 0x7fc4f813b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60794/0 (socket says 172.21.15.89:60794) 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 -- 172.21.15.89:0/3116782431 learned_addr learned my addr 172.21.15.89:0/3116782431 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f6d9d700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f5d9b700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4f809de00 0x7fc4f813bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 -- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 msgr2=0x7fc4f813c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 -- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4f809de00 msgr2=0x7fc4f813bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4f809de00 0x7fc4f813bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 -- 172.21.15.89:0/3116782431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec0094a0 con 0x7fc4f806a770 2023-09-30T22:23:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4f659c700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f806a770 0x7fc4f813b6c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800c510 tx=0x7fc4e800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4e6ffd700 1 -- 172.21.15.89:0/3116782431 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80052d0 con 0x7fc4f806a770 2023-09-30T22:23:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4e6ffd700 1 -- 172.21.15.89:0/3116782431 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8005430 con 0x7fc4f806a770 2023-09-30T22:23:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.570+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f811f130 con 0x7fc4f806a770 2023-09-30T22:23:29.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.571+0000 7fc4e6ffd700 1 -- 172.21.15.89:0/3116782431 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80038c0 con 0x7fc4f806a770 2023-09-30T22:23:29.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.571+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f811ea90 con 0x7fc4f806a770 2023-09-30T22:23:29.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.571+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f8074ea0 con 0x7fc4f806a770 2023-09-30T22:23:29.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.572+0000 7fc4e6ffd700 1 -- 172.21.15.89:0/3116782431 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc4e8003a20 con 0x7fc4f806a770 2023-09-30T22:23:29.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.572+0000 7fc4e6ffd700 1 -- 172.21.15.89:0/3116782431 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc4e803eb40 con 0x7fc4f806a770 2023-09-30T22:23:29.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.576+0000 7fc4e6ffd700 1 -- 172.21.15.89:0/3116782431 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc4e8018260 con 0x7fc4f806a770 2023-09-30T22:23:29.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.629+0000 7f799a34b700 1 Processor -- start 2023-09-30T22:23:29.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.629+0000 7f799a34b700 1 -- start start 2023-09-30T22:23:29.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f799a34b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f799a34b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f799409e000 0x7f79941339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f799a34b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799409b120 con 0x7f799409e410 2023-09-30T22:23:29.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f799a34b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799406b460 con 0x7f799409e000 2023-09-30T22:23:29.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f799a34b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799409a7e0 con 0x7f799409d6f0 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f7993fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f7993fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60810/0 (socket says 172.21.15.89:60810) 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f7993fff700 1 -- 172.21.15.89:0/2690287638 learned_addr learned my addr 172.21.15.89:0/2690287638 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f79937fe700 1 --2- 172.21.15.89:0/2690287638 >> v2:172.21.15.89:3300/0 conn(0x7f799409e000 0x7f79941339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f7993fff700 1 -- 172.21.15.89:0/2690287638 >> v2:172.21.15.89:3300/0 conn(0x7f799409e000 msgr2=0x7f79941339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f7993fff700 1 --2- 172.21.15.89:0/2690287638 >> v2:172.21.15.89:3300/0 conn(0x7f799409e000 0x7f79941339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f7993fff700 1 -- 172.21.15.89:0/2690287638 >> v1:172.21.15.89:6789/0 conn(0x7f799409e410 legacy=0x7f7994133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f7993fff700 1 -- 172.21.15.89:0/2690287638 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799409a500 con 0x7f799409d6f0 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.630+0000 7f7993fff700 1 --2- 172.21.15.89:0/2690287638 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7984009920 tx=0x7f7984009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef13f435d1c000fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f7992ffd700 1 -- 172.21.15.89:0/2690287638 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984011830 con 0x7f799409d6f0 2023-09-30T22:23:29.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f7992ffd700 1 -- 172.21.15.89:0/2690287638 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7984011990 con 0x7f799409d6f0 2023-09-30T22:23:29.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f7992ffd700 1 -- 172.21.15.89:0/2690287638 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984011d20 con 0x7f799409d6f0 2023-09-30T22:23:29.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f799a34b700 1 -- 172.21.15.89:0/2690287638 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 msgr2=0x7f799409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f799a34b700 1 --2- 172.21.15.89:0/2690287638 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7984009920 tx=0x7f7984009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:29.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f799a34b700 1 -- 172.21.15.89:0/2690287638 shutdown_connections 2023-09-30T22:23:29.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f799a34b700 1 --2- 172.21.15.89:0/2690287638 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409d6f0 0x7f799409dac0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f799a34b700 1 --2- 172.21.15.89:0/2690287638 >> v2:172.21.15.89:3300/0 conn(0x7f799409e000 0x7f79941339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.631+0000 7f799a34b700 1 -- 172.21.15.89:0/2690287638 >> 172.21.15.89:0/2690287638 conn(0x7f7994066f50 msgr2=0x7f799409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:29.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.632+0000 7f799a34b700 1 -- 172.21.15.89:0/2690287638 shutdown_connections 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.632+0000 7f799a34b700 1 -- 172.21.15.89:0/2690287638 wait complete. 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.632+0000 7f799a34b700 1 Processor -- start 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f799a34b700 1 -- start start 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f799a34b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409d6f0 0x7f79941a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f799a34b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e000 0x7f79941a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f799a34b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e410 0x7f79941a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f799a34b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79941a9a60 con 0x7f799409d6f0 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f799a34b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79941a7bf0 con 0x7f799409e000 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f799a34b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79940a33d0 con 0x7f799409e410 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f79988e8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e410 0x7f79941a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f79937fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e000 0x7f79941a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f79988e8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e410 0x7f79941a6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:47710/0 (socket says 172.21.15.89:47710) 2023-09-30T22:23:29.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f79988e8700 1 -- 172.21.15.89:0/2617927876 learned_addr learned my addr 172.21.15.89:0/2617927876 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:29.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.633+0000 7f7993fff700 1 --2- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409d6f0 0x7f79941a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:29.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f79988e8700 1 -- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e000 msgr2=0x7f79941a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f79988e8700 1 --2- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e000 0x7f79941a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f79988e8700 1 -- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409d6f0 msgr2=0x7f79941a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f79988e8700 1 --2- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409d6f0 0x7f79941a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f79988e8700 1 -- 172.21.15.89:0/2617927876 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7984009550 con 0x7f799409e410 2023-09-30T22:23:29.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f79988e8700 1 --2- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e410 0x7f79941a6a50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f798c00cc50 tx=0x7f798c00cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:29.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f7990ff9700 1 -- 172.21.15.89:0/2617927876 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c003c00 con 0x7f799409e410 2023-09-30T22:23:29.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f7990ff9700 1 -- 172.21.15.89:0/2617927876 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c003d60 con 0x7f799409e410 2023-09-30T22:23:29.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.634+0000 7f7990ff9700 1 -- 172.21.15.89:0/2617927876 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c00aab0 con 0x7f799409e410 2023-09-30T22:23:29.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.635+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799409c710 con 0x7f799409e410 2023-09-30T22:23:29.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.635+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79941a8950 con 0x7f799409e410 2023-09-30T22:23:29.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.636+0000 7f7990ff9700 1 -- 172.21.15.89:0/2617927876 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f798c00ac10 con 0x7f799409e410 2023-09-30T22:23:29.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.636+0000 7f7990ff9700 1 -- 172.21.15.89:0/2617927876 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f798c011540 con 0x7f799409e410 2023-09-30T22:23:29.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.636+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7994074ea0 con 0x7f799409e410 2023-09-30T22:23:29.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.645+0000 7f7990ff9700 1 -- 172.21.15.89:0/2617927876 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f798c0198f0 con 0x7f799409e410 2023-09-30T22:23:29.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.703+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f30bc074ea0 con 0x7f30bc06b080 2023-09-30T22:23:29.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.703+0000 7f30b27fc700 1 -- 172.21.15.89:0/3426538138 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f30ac003e40 con 0x7f30bc06b080 2023-09-30T22:23:29.704 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:29.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.705+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30bc06b080 msgr2=0x7f30bc11e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.705+0000 7f30c4188700 1 --2- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30bc06b080 0x7f30bc11e7d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c750 tx=0x7f30ac00ca60 comp rx=0 tx=0).stop 2023-09-30T22:23:29.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.706+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 shutdown_connections 2023-09-30T22:23:29.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.706+0000 7f30c4188700 1 --2- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30bc131d10 0x7f30bc11ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.706+0000 7f30c4188700 1 --2- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30bc06a770 0x7f30bc11e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.706+0000 7f30c4188700 1 --2- 172.21.15.89:0/3426538138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30bc06b080 0x7f30bc11e7d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.706+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 >> 172.21.15.89:0/3426538138 conn(0x7f30bc066f50 msgr2=0x7f30bc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:29.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.706+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 shutdown_connections 2023-09-30T22:23:29.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.706+0000 7f30c4188700 1 -- 172.21.15.89:0/3426538138 wait complete. 2023-09-30T22:23:29.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.758+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1694074ea0 con 0x7f1694130430 2023-09-30T22:23:29.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.758+0000 7f16917fa700 1 -- 172.21.15.89:0/2554151109 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f168c0243d0 con 0x7f1694130430 2023-09-30T22:23:29.759 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:29.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 msgr2=0x7f169409b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f169409b1e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f168c00b370 tx=0x7f168c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 shutdown_connections 2023-09-30T22:23:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1694130430 0x7f169409b1e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f169412cf60 0x7f16941c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 --2- 172.21.15.89:0/2554151109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f169412f8b0 0x7f16941c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 >> 172.21.15.89:0/2554151109 conn(0x7f16940a2aa0 msgr2=0x7f16941b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 shutdown_connections 2023-09-30T22:23:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.762+0000 7f169b0d3700 1 -- 172.21.15.89:0/2554151109 wait complete. 2023-09-30T22:23:29.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:29.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.804+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc4f809a3d0 con 0x7fc4f806a770 2023-09-30T22:23:29.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.804+0000 7fc4e6ffd700 1 -- 172.21.15.89:0/3116782431 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc4e8017070 con 0x7fc4f806a770 2023-09-30T22:23:29.805 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f806a770 msgr2=0x7fc4f813b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f806a770 0x7fc4f813b6c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800c510 tx=0x7fc4e800c820 comp rx=0 tx=0).stop 2023-09-30T22:23:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 shutdown_connections 2023-09-30T22:23:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4f813d1f0 0x7fc4f813c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4f806a770 0x7fc4f813b6c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 --2- 172.21.15.89:0/3116782431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4f809de00 0x7fc4f813bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 >> 172.21.15.89:0/3116782431 conn(0x7fc4f8066f50 msgr2=0x7fc4f809b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 shutdown_connections 2023-09-30T22:23:29.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.806+0000 7fc4fcc74700 1 -- 172.21.15.89:0/3116782431 wait complete. 2023-09-30T22:23:29.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:29.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.868+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f799406c360 con 0x7f799409e410 2023-09-30T22:23:29.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.868+0000 7f7990ff9700 1 -- 172.21.15.89:0/2617927876 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f798c011380 con 0x7f799409e410 2023-09-30T22:23:29.869 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:29.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.870+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e410 msgr2=0x7f79941a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:29.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.870+0000 7f799a34b700 1 --2- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e410 0x7f79941a6a50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f798c00cc50 tx=0x7f798c00cf60 comp rx=0 tx=0).stop 2023-09-30T22:23:29.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.870+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 shutdown_connections 2023-09-30T22:23:29.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.870+0000 7f799a34b700 1 --2- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f799409e410 0x7f79941a6a50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.870+0000 7f799a34b700 1 --2- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f799409d6f0 0x7f79941a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.870+0000 7f799a34b700 1 --2- 172.21.15.89:0/2617927876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f799409e000 0x7f79941a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:29.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.870+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 >> 172.21.15.89:0/2617927876 conn(0x7f7994066f50 msgr2=0x7f79941bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:29.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.871+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 shutdown_connections 2023-09-30T22:23:29.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:29.871+0000 7f799a34b700 1 -- 172.21.15.89:0/2617927876 wait complete. 2023-09-30T22:23:29.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:30.721 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:30.779 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:30.820 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.863+0000 7fd4bd23b700 1 Processor -- start 2023-09-30T22:23:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4bd23b700 1 -- start start 2023-09-30T22:23:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4bd23b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130540 0x7fd4b8130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4bd23b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b806fcf0 0x7fd4b8138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4bd23b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b80716a0 con 0x7fd4b8130e50 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4bd23b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b809a390 con 0x7fd4b806fcf0 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4bd23b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b8071290 con 0x7fd4b8130540 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4b659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd4b8130e50 0x7fd4b806f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58954/0 (socket says 172.21.15.89:58954) 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4b659c700 1 -- 172.21.15.89:0/1941895141 learned_addr learned my addr 172.21.15.89:0/1941895141 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4b759e700 1 --2- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b806fcf0 0x7fd4b8138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.864+0000 7fd4b6d9d700 1 --2- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130540 0x7fd4b8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b5d9b700 1 -- 172.21.15.89:0/1941895141 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161850266 0 0) 0x7fd4b80716a0 con 0x7fd4b8130e50 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b759e700 1 -- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130540 msgr2=0x7fd4b8130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b759e700 1 --2- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130540 0x7fd4b8130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b759e700 1 -- 172.21.15.89:0/1941895141 >> v1:172.21.15.89:6789/0 conn(0x7fd4b8130e50 legacy=0x7fd4b806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b759e700 1 -- 172.21.15.89:0/1941895141 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b8130340 con 0x7fd4b806fcf0 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b6d9d700 1 --2- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130540 0x7fd4b8130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b759e700 1 --2- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b806fcf0 0x7fd4b8138570 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009dc0 tx=0x7fd4b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca67a44e8b0d0a4c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b5d9b700 1 -- 172.21.15.89:0/1941895141 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000e070 con 0x7fd4b806fcf0 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b5d9b700 1 -- 172.21.15.89:0/1941895141 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b000bb20 con 0x7fd4b806fcf0 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4b5d9b700 1 -- 172.21.15.89:0/1941895141 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0012430 con 0x7fd4b806fcf0 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.865+0000 7fd4bd23b700 1 -- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b806fcf0 msgr2=0x7fd4b8138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.866+0000 7fd4bd23b700 1 --2- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b806fcf0 0x7fd4b8138570 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009dc0 tx=0x7fd4b000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.866+0000 7fd4bd23b700 1 -- 172.21.15.89:0/1941895141 shutdown_connections 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.866+0000 7fd4bd23b700 1 --2- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b806fcf0 0x7fd4b8138570 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.866+0000 7fd4bd23b700 1 --2- 172.21.15.89:0/1941895141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130540 0x7fd4b8130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.866+0000 7fd4bd23b700 1 -- 172.21.15.89:0/1941895141 >> 172.21.15.89:0/1941895141 conn(0x7fd4b80a2aa0 msgr2=0x7fd4b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:30.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.866+0000 7fd4bd23b700 1 -- 172.21.15.89:0/1941895141 shutdown_connections 2023-09-30T22:23:30.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.866+0000 7fd4bd23b700 1 -- 172.21.15.89:0/1941895141 wait complete. 2023-09-30T22:23:30.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.867+0000 7fd4bd23b700 1 Processor -- start 2023-09-30T22:23:30.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.867+0000 7fd4bd23b700 1 -- start start 2023-09-30T22:23:30.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.867+0000 7fd4bd23b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4b806fcf0 0x7fd4b811e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.867+0000 7fd4bd23b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b8130540 0x7fd4b811ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.867+0000 7fd4bd23b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130e50 0x7fd4b811f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.867+0000 7fd4bd23b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80a19f0 con 0x7fd4b806fcf0 2023-09-30T22:23:30.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.867+0000 7fd4bd23b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80718e0 con 0x7fd4b8130540 2023-09-30T22:23:30.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.867+0000 7fd4bd23b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b8120b20 con 0x7fd4b8130e50 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b8130540 0x7fd4b811ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b8130540 0x7fd4b811ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39794/0 (socket says 172.21.15.89:39794) 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 -- 172.21.15.89:0/175916683 learned_addr learned my addr 172.21.15.89:0/175916683 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b759e700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130e50 0x7fd4b811f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b6d9d700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4b806fcf0 0x7fd4b811e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 -- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130e50 msgr2=0x7fd4b811f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130e50 0x7fd4b811f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 -- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4b806fcf0 msgr2=0x7fd4b811e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4b806fcf0 0x7fd4b811e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 -- 172.21.15.89:0/175916683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a80094a0 con 0x7fd4b8130540 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b759e700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130e50 0x7fd4b811f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:30.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4b659c700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b8130540 0x7fd4b811ef20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac001eb0 tx=0x7fd4ac00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:30.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd49f7fe700 1 -- 172.21.15.89:0/175916683 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac004c80 con 0x7fd4b8130540 2023-09-30T22:23:30.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd49f7fe700 1 -- 172.21.15.89:0/175916683 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac004de0 con 0x7fd4b8130540 2023-09-30T22:23:30.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b0009a70 con 0x7fd4b8130540 2023-09-30T22:23:30.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd49f7fe700 1 -- 172.21.15.89:0/175916683 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac007310 con 0x7fd4b8130540 2023-09-30T22:23:30.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.868+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b811fc70 con 0x7fd4b8130540 2023-09-30T22:23:30.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.869+0000 7fd49f7fe700 1 -- 172.21.15.89:0/175916683 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4ac007470 con 0x7fd4b8130540 2023-09-30T22:23:30.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.869+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b8074ea0 con 0x7fd4b8130540 2023-09-30T22:23:30.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.869+0000 7fd49f7fe700 1 -- 172.21.15.89:0/175916683 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd4ac03f220 con 0x7fd4b8130540 2023-09-30T22:23:30.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.878+0000 7fd49f7fe700 1 -- 172.21.15.89:0/175916683 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4ac0133c0 con 0x7fd4b8130540 2023-09-30T22:23:30.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.916+0000 7fa556cb4700 1 Processor -- start 2023-09-30T22:23:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.916+0000 7fa556cb4700 1 -- start start 2023-09-30T22:23:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa556cb4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5501338a0 0x7fa550133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa556cb4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55009e930 0x7fa55013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa556cb4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55006ef30 con 0x7fa55009de50 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa556cb4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55006fef0 con 0x7fa5501338a0 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa556cb4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5501334b0 con 0x7fa55009e930 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa555251700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55009e930 0x7fa55013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa54ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa55009de50 0x7fa55009e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58956/0 (socket says 172.21.15.89:58956) 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa54ffff700 1 -- 172.21.15.89:0/3508788071 learned_addr learned my addr 172.21.15.89:0/3508788071 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.917+0000 7fa554a50700 1 --2- 172.21.15.89:0/3508788071 >> v2:172.21.15.89:3300/0 conn(0x7fa5501338a0 0x7fa550133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.918+0000 7fa555251700 1 -- 172.21.15.89:0/3508788071 >> v2:172.21.15.89:3300/0 conn(0x7fa5501338a0 msgr2=0x7fa550133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.918+0000 7fa555251700 1 --2- 172.21.15.89:0/3508788071 >> v2:172.21.15.89:3300/0 conn(0x7fa5501338a0 0x7fa550133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.918+0000 7fa555251700 1 -- 172.21.15.89:0/3508788071 >> v1:172.21.15.89:6789/0 conn(0x7fa55009de50 legacy=0x7fa55009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.918+0000 7fa555251700 1 -- 172.21.15.89:0/3508788071 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55006fc30 con 0x7fa55009e930 2023-09-30T22:23:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.918+0000 7fa555251700 1 --2- 172.21.15.89:0/3508788071 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55009e930 0x7fa55013b740 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa548009d90 tx=0x7fa54800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d20bdceac9930f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.918+0000 7fa54f7fe700 1 -- 172.21.15.89:0/3508788071 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54800e070 con 0x7fa55009e930 2023-09-30T22:23:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.918+0000 7fa54f7fe700 1 -- 172.21.15.89:0/3508788071 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54800bb20 con 0x7fa55009e930 2023-09-30T22:23:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.918+0000 7fa54f7fe700 1 -- 172.21.15.89:0/3508788071 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548012430 con 0x7fa55009e930 2023-09-30T22:23:30.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.919+0000 7fa556cb4700 1 -- 172.21.15.89:0/3508788071 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55009e930 msgr2=0x7fa55013b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.919+0000 7fa556cb4700 1 --2- 172.21.15.89:0/3508788071 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55009e930 0x7fa55013b740 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa548009d90 tx=0x7fa54800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.919+0000 7fa556cb4700 1 -- 172.21.15.89:0/3508788071 shutdown_connections 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.919+0000 7fa556cb4700 1 --2- 172.21.15.89:0/3508788071 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa55009e930 0x7fa55013b740 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.919+0000 7fa556cb4700 1 --2- 172.21.15.89:0/3508788071 >> v2:172.21.15.89:3300/0 conn(0x7fa5501338a0 0x7fa550133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.919+0000 7fa556cb4700 1 -- 172.21.15.89:0/3508788071 >> 172.21.15.89:0/3508788071 conn(0x7fa5500a2aa0 msgr2=0x7fa5500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.919+0000 7fa556cb4700 1 -- 172.21.15.89:0/3508788071 shutdown_connections 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.919+0000 7fa556cb4700 1 -- 172.21.15.89:0/3508788071 wait complete. 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.920+0000 7fa556cb4700 1 Processor -- start 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.920+0000 7fa556cb4700 1 -- start start 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa556cb4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55009de50 0x7fa55011b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa556cb4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa55009e930 0x7fa55011bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa556cb4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5501338a0 0x7fa55011c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa556cb4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55011f390 con 0x7fa55009e930 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa556cb4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55006f6b0 con 0x7fa5501338a0 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa556cb4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500a0fb0 con 0x7fa55009de50 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55009de50 0x7fa55011b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55009de50 0x7fa55011b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53630/0 (socket says 172.21.15.89:53630) 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 -- 172.21.15.89:0/1364142091 learned_addr learned my addr 172.21.15.89:0/1364142091 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa54ffff700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa55009e930 0x7fa55011bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa555251700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5501338a0 0x7fa55011c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 -- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5501338a0 msgr2=0x7fa55011c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5501338a0 0x7fa55011c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 -- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa55009e930 msgr2=0x7fa55011bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa55009e930 0x7fa55011bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 -- 172.21.15.89:0/1364142091 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548009a70 con 0x7fa55009de50 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa555251700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5501338a0 0x7fa55011c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa54ffff700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa55009e930 0x7fa55011bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.921+0000 7fa554a50700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55009de50 0x7fa55011b870 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa54400e9b0 tx=0x7fa54400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.922+0000 7fa54d7fa700 1 -- 172.21.15.89:0/1364142091 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544009710 con 0x7fa55009de50 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.922+0000 7fa54d7fa700 1 -- 172.21.15.89:0/1364142091 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54400c7f0 con 0x7fa55009de50 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.922+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55011dce0 con 0x7fa55009de50 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.922+0000 7fa54d7fa700 1 -- 172.21.15.89:0/1364142091 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54400cb60 con 0x7fa55009de50 2023-09-30T22:23:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.922+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55011eb20 con 0x7fa55009de50 2023-09-30T22:23:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.923+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5500a0580 con 0x7fa55009de50 2023-09-30T22:23:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.923+0000 7fa54d7fa700 1 -- 172.21.15.89:0/1364142091 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa54400ccc0 con 0x7fa55009de50 2023-09-30T22:23:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.924+0000 7fa54d7fa700 1 -- 172.21.15.89:0/1364142091 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa544040040 con 0x7fa55009de50 2023-09-30T22:23:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.928+0000 7fa54d7fa700 1 -- 172.21.15.89:0/1364142091 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa544011350 con 0x7fa55009de50 2023-09-30T22:23:30.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.960+0000 7f0da4320700 1 Processor -- start 2023-09-30T22:23:30.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.960+0000 7f0da4320700 1 -- start start 2023-09-30T22:23:30.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.960+0000 7f0da4320700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d9c06c0e0 0x7f0d9c09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.960+0000 7f0da4320700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c134ec0 0x7f0d9c1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.960+0000 7f0da4320700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9c134980 con 0x7f0d9c137f20 2023-09-30T22:23:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.960+0000 7f0da4320700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9c06c4b0 con 0x7f0d9c06c0e0 2023-09-30T22:23:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.960+0000 7f0da4320700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c0a09c0 con 0x7f0d9c134ec0 2023-09-30T22:23:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da20bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d9c06c0e0 0x7f0d9c09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da20bc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d9c06c0e0 0x7f0d9c09ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41290/0 (socket says 172.21.15.89:41290) 2023-09-30T22:23:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da20bc700 1 -- 172.21.15.89:0/4054501811 learned_addr learned my addr 172.21.15.89:0/4054501811 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da18bb700 1 --2- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c134ec0 0x7f0d9c1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da20bc700 1 -- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c134ec0 msgr2=0x7f0d9c1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da20bc700 1 --2- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c134ec0 0x7f0d9c1379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da20bc700 1 -- 172.21.15.89:0/4054501811 >> v1:172.21.15.89:6789/0 conn(0x7f0d9c137f20 legacy=0x7f0d9c13ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da20bc700 1 -- 172.21.15.89:0/4054501811 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c0a0600 con 0x7f0d9c06c0e0 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da18bb700 1 --2- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c134ec0 0x7f0d9c1379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da20bc700 1 --2- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c06c0e0 0x7f0d9c09ac90 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0d980098f0 tx=0x7f0d98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8284300d5f0b1f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da10ba700 1 -- 172.21.15.89:0/4054501811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98011830 con 0x7f0d9c06c0e0 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.961+0000 7f0da10ba700 1 -- 172.21.15.89:0/4054501811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98011990 con 0x7f0d9c06c0e0 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da10ba700 1 -- 172.21.15.89:0/4054501811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98011d20 con 0x7f0d9c06c0e0 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da4320700 1 -- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c06c0e0 msgr2=0x7f0d9c09ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da4320700 1 --2- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c06c0e0 0x7f0d9c09ac90 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0d980098f0 tx=0x7f0d98009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da4320700 1 -- 172.21.15.89:0/4054501811 shutdown_connections 2023-09-30T22:23:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da4320700 1 --2- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c06c0e0 0x7f0d9c09ac90 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da4320700 1 --2- 172.21.15.89:0/4054501811 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c134ec0 0x7f0d9c1379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da4320700 1 -- 172.21.15.89:0/4054501811 >> 172.21.15.89:0/4054501811 conn(0x7f0d9c0a4210 msgr2=0x7f0d9c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da4320700 1 -- 172.21.15.89:0/4054501811 shutdown_connections 2023-09-30T22:23:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.962+0000 7f0da4320700 1 -- 172.21.15.89:0/4054501811 wait complete. 2023-09-30T22:23:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.963+0000 7f0da4320700 1 Processor -- start 2023-09-30T22:23:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.963+0000 7f0da4320700 1 -- start start 2023-09-30T22:23:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da4320700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c06c0e0 0x7f0d9c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da4320700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c134ec0 0x7f0d9c11abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da4320700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d9c137f20 0x7f0d9c11b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da4320700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c11e550 con 0x7f0d9c134ec0 2023-09-30T22:23:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da4320700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c11db00 con 0x7f0d9c06c0e0 2023-09-30T22:23:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da4320700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c137900 con 0x7f0d9c137f20 2023-09-30T22:23:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c134ec0 0x7f0d9c11abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c134ec0 0x7f0d9c11abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41300/0 (socket says 172.21.15.89:41300) 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 -- 172.21.15.89:0/469395744 learned_addr learned my addr 172.21.15.89:0/469395744 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da28bd700 1 --2- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d9c137f20 0x7f0d9c11b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 -- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d9c137f20 msgr2=0x7f0d9c11b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 --2- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d9c137f20 0x7f0d9c11b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 -- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c06c0e0 msgr2=0x7f0d9c11a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 --2- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c06c0e0 0x7f0d9c11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 -- 172.21.15.89:0/469395744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c0094a0 con 0x7f0d9c134ec0 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da18bb700 1 --2- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c134ec0 0x7f0d9c11abc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00ec10 tx=0x7f0d8c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:30.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0d92ffd700 1 -- 172.21.15.89:0/469395744 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00c940 con 0x7f0d9c134ec0 2023-09-30T22:23:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0d92ffd700 1 -- 172.21.15.89:0/469395744 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c00caa0 con 0x7f0d9c134ec0 2023-09-30T22:23:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.964+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d98009550 con 0x7f0d9c134ec0 2023-09-30T22:23:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.965+0000 7f0d92ffd700 1 -- 172.21.15.89:0/469395744 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00ce10 con 0x7f0d9c134ec0 2023-09-30T22:23:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.965+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c0a03f0 con 0x7f0d9c134ec0 2023-09-30T22:23:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.965+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9c074ea0 con 0x7f0d9c134ec0 2023-09-30T22:23:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.966+0000 7f0d92ffd700 1 -- 172.21.15.89:0/469395744 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d8c013470 con 0x7f0d9c134ec0 2023-09-30T22:23:30.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.966+0000 7f0d92ffd700 1 -- 172.21.15.89:0/469395744 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0d8c03f8e0 con 0x7f0d9c134ec0 2023-09-30T22:23:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:30.970+0000 7f0d92ffd700 1 -- 172.21.15.89:0/469395744 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d8c0232f0 con 0x7f0d9c134ec0 2023-09-30T22:23:31.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.018+0000 7f1bc91a8700 1 Processor -- start 2023-09-30T22:23:31.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.018+0000 7f1bc91a8700 1 -- start start 2023-09-30T22:23:31.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc91a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc406e570 0x7f1bc406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc91a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc409eca0 0x7f1bc406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc91a8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc4070ee0 con 0x7f1bc4134610 2023-09-30T22:23:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc91a8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc4071480 con 0x7f1bc406e570 2023-09-30T22:23:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc91a8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc406f3c0 con 0x7f1bc409eca0 2023-09-30T22:23:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc2d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc406e570 0x7f1bc406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc359e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1bc4134610 0x7f1bc4134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58970/0 (socket says 172.21.15.89:58970) 2023-09-30T22:23:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc409eca0 0x7f1bc406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc359e700 1 -- 172.21.15.89:0/1893120700 learned_addr learned my addr 172.21.15.89:0/1893120700 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:31.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc259c700 1 -- 172.21.15.89:0/1893120700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc406e570 msgr2=0x7f1bc406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc259c700 1 --2- 172.21.15.89:0/1893120700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc406e570 0x7f1bc406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc259c700 1 -- 172.21.15.89:0/1893120700 >> v1:172.21.15.89:6789/0 conn(0x7f1bc4134610 legacy=0x7f1bc4134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc259c700 1 -- 172.21.15.89:0/1893120700 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc4071b90 con 0x7f1bc409eca0 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.019+0000 7f1bc259c700 1 --2- 172.21.15.89:0/1893120700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc409eca0 0x7f1bc406ee80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009870 tx=0x7f1bb8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d782c651374c759c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc1d9b700 1 -- 172.21.15.89:0/1893120700 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8011830 con 0x7f1bc409eca0 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc1d9b700 1 -- 172.21.15.89:0/1893120700 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8011990 con 0x7f1bc409eca0 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc1d9b700 1 -- 172.21.15.89:0/1893120700 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8011d20 con 0x7f1bc409eca0 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc91a8700 1 -- 172.21.15.89:0/1893120700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc409eca0 msgr2=0x7f1bc406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc91a8700 1 --2- 172.21.15.89:0/1893120700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc409eca0 0x7f1bc406ee80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009870 tx=0x7f1bb8009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc91a8700 1 -- 172.21.15.89:0/1893120700 shutdown_connections 2023-09-30T22:23:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc91a8700 1 --2- 172.21.15.89:0/1893120700 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc406e570 0x7f1bc406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc91a8700 1 --2- 172.21.15.89:0/1893120700 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc409eca0 0x7f1bc406ee80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc91a8700 1 -- 172.21.15.89:0/1893120700 >> 172.21.15.89:0/1893120700 conn(0x7f1bc40a2aa0 msgr2=0x7f1bc40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc91a8700 1 -- 172.21.15.89:0/1893120700 shutdown_connections 2023-09-30T22:23:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.020+0000 7f1bc91a8700 1 -- 172.21.15.89:0/1893120700 wait complete. 2023-09-30T22:23:31.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc91a8700 1 Processor -- start 2023-09-30T22:23:31.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc91a8700 1 -- start start 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc91a8700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc406e570 0x7f1bc413ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc91a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bc4134610 0x7f1bc413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc91a8700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc413f700 0x7f1bc4132e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc91a8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc413d0a0 con 0x7f1bc4134610 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc91a8700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc413fb70 con 0x7f1bc413f700 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc91a8700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc413ddc0 con 0x7f1bc406e570 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc359e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc413f700 0x7f1bc4132e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc359e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc413f700 0x7f1bc4132e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39866/0 (socket says 172.21.15.89:39866) 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc359e700 1 -- 172.21.15.89:0/212857745 learned_addr learned my addr 172.21.15.89:0/212857745 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc2d9d700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc406e570 0x7f1bc413ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.022+0000 7f1bc259c700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bc4134610 0x7f1bc413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc359e700 1 -- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc406e570 msgr2=0x7f1bc413ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc359e700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc406e570 0x7f1bc413ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc359e700 1 -- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bc4134610 msgr2=0x7f1bc413f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc359e700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bc4134610 0x7f1bc413f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc359e700 1 -- 172.21.15.89:0/212857745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb80094a0 con 0x7f1bc413f700 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc2d9d700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc406e570 0x7f1bc413ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc359e700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc413f700 0x7f1bc4132e70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00cbc0 tx=0x7f1bbc00ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bab7fe700 1 -- 172.21.15.89:0/212857745 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00aaa0 con 0x7f1bc413f700 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bab7fe700 1 -- 172.21.15.89:0/212857745 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00ac00 con 0x7f1bc413f700 2023-09-30T22:23:31.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc40a1fe0 con 0x7f1bc413f700 2023-09-30T22:23:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bab7fe700 1 -- 172.21.15.89:0/212857745 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc012400 con 0x7f1bc413f700 2023-09-30T22:23:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.023+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc413db00 con 0x7f1bc413f700 2023-09-30T22:23:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.024+0000 7f1bab7fe700 1 -- 172.21.15.89:0/212857745 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1bbc01b3b0 con 0x7f1bc413f700 2023-09-30T22:23:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.024+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc413e8f0 con 0x7f1bc413f700 2023-09-30T22:23:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.024+0000 7f1bab7fe700 1 -- 172.21.15.89:0/212857745 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1bbc03f300 con 0x7f1bc413f700 2023-09-30T22:23:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.033+0000 7f1bab7fe700 1 -- 172.21.15.89:0/212857745 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1bbc0192c0 con 0x7f1bc413f700 2023-09-30T22:23:31.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.079+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd4b806b9e0 con 0x7fd4b8130540 2023-09-30T22:23:31.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.079+0000 7fd49f7fe700 1 -- 172.21.15.89:0/175916683 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4ac0102c0 con 0x7fd4b8130540 2023-09-30T22:23:31.080 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:31.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b8130540 msgr2=0x7fd4b811ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:31.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b8130540 0x7fd4b811ef20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac001eb0 tx=0x7fd4ac00c800 comp rx=0 tx=0).stop 2023-09-30T22:23:31.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 shutdown_connections 2023-09-30T22:23:31.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd4b8130e50 0x7fd4b811f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4b806fcf0 0x7fd4b811e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 --2- 172.21.15.89:0/175916683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4b8130540 0x7fd4b811ef20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 >> 172.21.15.89:0/175916683 conn(0x7fd4b80a2aa0 msgr2=0x7fd4b812ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:31.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 shutdown_connections 2023-09-30T22:23:31.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.081+0000 7fd4bd23b700 1 -- 172.21.15.89:0/175916683 wait complete. 2023-09-30T22:23:31.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:31.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.129+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa550074ea0 con 0x7fa55009de50 2023-09-30T22:23:31.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.129+0000 7fa54d7fa700 1 -- 172.21.15.89:0/1364142091 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa544024b00 con 0x7fa55009de50 2023-09-30T22:23:31.130 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:31.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55009de50 msgr2=0x7fa55011b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:31.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55009de50 0x7fa55011b870 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa54400e9b0 tx=0x7fa54400ed70 comp rx=0 tx=0).stop 2023-09-30T22:23:31.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 shutdown_connections 2023-09-30T22:23:31.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5501338a0 0x7fa55011c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa55009de50 0x7fa55011b870 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 --2- 172.21.15.89:0/1364142091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa55009e930 0x7fa55011bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 >> 172.21.15.89:0/1364142091 conn(0x7fa5500a2aa0 msgr2=0x7fa5501305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:31.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 shutdown_connections 2023-09-30T22:23:31.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.131+0000 7fa556cb4700 1 -- 172.21.15.89:0/1364142091 wait complete. 2023-09-30T22:23:31.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.170+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0d9c06b250 con 0x7f0d9c134ec0 2023-09-30T22:23:31.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.171+0000 7f0d92ffd700 1 -- 172.21.15.89:0/469395744 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d8c021020 con 0x7f0d9c134ec0 2023-09-30T22:23:31.172 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c134ec0 msgr2=0x7f0d9c11abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 --2- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c134ec0 0x7f0d9c11abc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00ec10 tx=0x7f0d8c009ec0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 shutdown_connections 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 --2- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d9c137f20 0x7f0d9c11b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 --2- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0d9c06c0e0 0x7f0d9c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 --2- 172.21.15.89:0/469395744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d9c134ec0 0x7f0d9c11abc0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 >> 172.21.15.89:0/469395744 conn(0x7f0d9c0a4210 msgr2=0x7f0d9c0a0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 shutdown_connections 2023-09-30T22:23:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.173+0000 7f0da4320700 1 -- 172.21.15.89:0/469395744 wait complete. 2023-09-30T22:23:31.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:31.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.225+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1bc4074ea0 con 0x7f1bc413f700 2023-09-30T22:23:31.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.226+0000 7f1bab7fe700 1 -- 172.21.15.89:0/212857745 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1bbc017020 con 0x7f1bc413f700 2023-09-30T22:23:31.227 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:31.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc413f700 msgr2=0x7f1bc4132e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc413f700 0x7f1bc4132e70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00cbc0 tx=0x7f1bbc00ced0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 shutdown_connections 2023-09-30T22:23:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1bc413f700 0x7f1bc4132e70 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1bc406e570 0x7f1bc413ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 --2- 172.21.15.89:0/212857745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bc4134610 0x7f1bc413f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 >> 172.21.15.89:0/212857745 conn(0x7f1bc40a2aa0 msgr2=0x7f1bc40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 shutdown_connections 2023-09-30T22:23:31.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:31.228+0000 7f1bc91a8700 1 -- 172.21.15.89:0/212857745 wait complete. 2023-09-30T22:23:31.240 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:32.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:32.145 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:32.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.236+0000 7f7e744d4700 1 Processor -- start 2023-09-30T22:23:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.236+0000 7f7e744d4700 1 -- start start 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e744d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c09d9f0 0x7f7e6c136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e744d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c136a70 0x7f7e6c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e744d4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c09e130 con 0x7f7e6c1316c0 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e744d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1321a0 con 0x7f7e6c09d9f0 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e744d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1312b0 con 0x7f7e6c136a70 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e71a6f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c09d9f0 0x7f7e6c136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e72270700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7e6c1316c0 0x7f7e6c131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58972/0 (socket says 172.21.15.89:58972) 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e72270700 1 -- 172.21.15.89:0/863706842 learned_addr learned my addr 172.21.15.89:0/863706842 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.237+0000 7f7e72a71700 1 --2- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c136a70 0x7f7e6c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e72a71700 1 -- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c09d9f0 msgr2=0x7f7e6c136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e72a71700 1 --2- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c09d9f0 0x7f7e6c136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e72a71700 1 -- 172.21.15.89:0/863706842 >> v1:172.21.15.89:6789/0 conn(0x7f7e6c1316c0 legacy=0x7f7e6c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e72a71700 1 -- 172.21.15.89:0/863706842 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c09f6e0 con 0x7f7e6c136a70 2023-09-30T22:23:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e71a6f700 1 --2- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c09d9f0 0x7f7e6c136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e72a71700 1 --2- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c136a70 0x7f7e6c138e50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009df0 tx=0x7f7e6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284bec83fffa96ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e7126e700 1 -- 172.21.15.89:0/863706842 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400e070 con 0x7f7e6c136a70 2023-09-30T22:23:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e7126e700 1 -- 172.21.15.89:0/863706842 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6400bae0 con 0x7f7e6c136a70 2023-09-30T22:23:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.238+0000 7f7e7126e700 1 -- 172.21.15.89:0/863706842 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400be90 con 0x7f7e6c136a70 2023-09-30T22:23:32.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.239+0000 7f7e744d4700 1 -- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c136a70 msgr2=0x7f7e6c138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.239+0000 7f7e744d4700 1 --2- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c136a70 0x7f7e6c138e50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009df0 tx=0x7f7e6400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.239+0000 7f7e744d4700 1 -- 172.21.15.89:0/863706842 shutdown_connections 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.239+0000 7f7e744d4700 1 --2- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c136a70 0x7f7e6c138e50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.239+0000 7f7e744d4700 1 --2- 172.21.15.89:0/863706842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c09d9f0 0x7f7e6c136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.239+0000 7f7e744d4700 1 -- 172.21.15.89:0/863706842 >> 172.21.15.89:0/863706842 conn(0x7f7e6c066f50 msgr2=0x7f7e6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.239+0000 7f7e744d4700 1 -- 172.21.15.89:0/863706842 shutdown_connections 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.239+0000 7f7e744d4700 1 -- 172.21.15.89:0/863706842 wait complete. 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e744d4700 1 Processor -- start 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e744d4700 1 -- start start 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e744d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e6c09d9f0 0x7f7e6c1c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e744d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c1316c0 0x7f7e6c1c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e744d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c136a70 0x7f7e6c134300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e744d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c135f10 con 0x7f7e6c09d9f0 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e744d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0722d0 con 0x7f7e6c136a70 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e744d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c072700 con 0x7f7e6c1316c0 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e71a6f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c1316c0 0x7f7e6c1c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e71a6f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c1316c0 0x7f7e6c1c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53672/0 (socket says 172.21.15.89:53672) 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e72270700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e6c09d9f0 0x7f7e6c1c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e71a6f700 1 -- 172.21.15.89:0/258243535 learned_addr learned my addr 172.21.15.89:0/258243535 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.240+0000 7f7e72a71700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c136a70 0x7f7e6c134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e72a71700 1 -- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c1316c0 msgr2=0x7f7e6c1c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e72a71700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c1316c0 0x7f7e6c1c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e72a71700 1 -- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e6c09d9f0 msgr2=0x7f7e6c1c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e72a71700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e6c09d9f0 0x7f7e6c1c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e72a71700 1 -- 172.21.15.89:0/258243535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0094a0 con 0x7f7e6c136a70 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e72270700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e6c09d9f0 0x7f7e6c1c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e72a71700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c136a70 0x7f7e6c134300 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7e64016000 tx=0x7f7e64004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e62ffd700 1 -- 172.21.15.89:0/258243535 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64004800 con 0x7f7e6c136a70 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e62ffd700 1 -- 172.21.15.89:0/258243535 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e64020070 con 0x7f7e6c136a70 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e64009a70 con 0x7f7e6c136a70 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e62ffd700 1 -- 172.21.15.89:0/258243535 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e640276c0 con 0x7f7e6c136a70 2023-09-30T22:23:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.241+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c1365a0 con 0x7f7e6c136a70 2023-09-30T22:23:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.242+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c09b8a0 con 0x7f7e6c136a70 2023-09-30T22:23:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.242+0000 7f7e62ffd700 1 -- 172.21.15.89:0/258243535 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7e64012430 con 0x7f7e6c136a70 2023-09-30T22:23:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.242+0000 7f7e62ffd700 1 -- 172.21.15.89:0/258243535 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7e64022070 con 0x7f7e6c136a70 2023-09-30T22:23:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.247+0000 7f7e62ffd700 1 -- 172.21.15.89:0/258243535 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e6c09b8a0 con 0x7f7e6c136a70 2023-09-30T22:23:32.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.281+0000 7f84727ca700 1 Processor -- start 2023-09-30T22:23:32.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.281+0000 7f84727ca700 1 -- start start 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f84727ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f846c133510 0x7f846c1358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f84727ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c135e30 0x7f846c138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f84727ca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846c130260 con 0x7f846c130670 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f84727ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c09a350 con 0x7f846c133510 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f84727ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f846c071ff0 con 0x7f846c135e30 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f846b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f846c133510 0x7f846c1358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f846b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f846c133510 0x7f846c1358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41334/0 (socket says 172.21.15.89:41334) 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f846b7fe700 1 -- 172.21.15.89:0/409538414 learned_addr learned my addr 172.21.15.89:0/409538414 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.282+0000 7f8470d67700 1 --2- 172.21.15.89:0/409538414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c135e30 0x7f846c138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f8470d67700 1 -- 172.21.15.89:0/409538414 >> v2:172.21.15.89:3300/0 conn(0x7f846c133510 msgr2=0x7f846c1358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f8470d67700 1 --2- 172.21.15.89:0/409538414 >> v2:172.21.15.89:3300/0 conn(0x7f846c133510 0x7f846c1358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f8470d67700 1 -- 172.21.15.89:0/409538414 >> v1:172.21.15.89:6789/0 conn(0x7f846c130670 legacy=0x7f846c130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f846affd700 1 -- 172.21.15.89:0/409538414 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552993782 0 0) 0x7f846c130260 con 0x7f846c130670 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f8470d67700 1 -- 172.21.15.89:0/409538414 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c09ecf0 con 0x7f846c135e30 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f846b7fe700 1 --2- 172.21.15.89:0/409538414 >> v2:172.21.15.89:3300/0 conn(0x7f846c133510 0x7f846c1358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f8470d67700 1 --2- 172.21.15.89:0/409538414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c135e30 0x7f846c138210 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8464009d90 tx=0x7f846400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7867023f83f1fc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f846affd700 1 -- 172.21.15.89:0/409538414 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846400e070 con 0x7f846c135e30 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f846affd700 1 -- 172.21.15.89:0/409538414 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846400bb20 con 0x7f846c135e30 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.283+0000 7f846affd700 1 -- 172.21.15.89:0/409538414 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464012430 con 0x7f846c135e30 2023-09-30T22:23:32.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.284+0000 7f84727ca700 1 -- 172.21.15.89:0/409538414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c135e30 msgr2=0x7f846c138210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.284+0000 7f84727ca700 1 --2- 172.21.15.89:0/409538414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c135e30 0x7f846c138210 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8464009d90 tx=0x7f846400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.284+0000 7f84727ca700 1 -- 172.21.15.89:0/409538414 shutdown_connections 2023-09-30T22:23:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.284+0000 7f84727ca700 1 --2- 172.21.15.89:0/409538414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c135e30 0x7f846c138210 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.284+0000 7f84727ca700 1 --2- 172.21.15.89:0/409538414 >> v2:172.21.15.89:3300/0 conn(0x7f846c133510 0x7f846c1358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.284+0000 7f84727ca700 1 -- 172.21.15.89:0/409538414 >> 172.21.15.89:0/409538414 conn(0x7f846c0a2aa0 msgr2=0x7f846c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.284+0000 7f84727ca700 1 -- 172.21.15.89:0/409538414 shutdown_connections 2023-09-30T22:23:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.284+0000 7f84727ca700 1 -- 172.21.15.89:0/409538414 wait complete. 2023-09-30T22:23:32.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.285+0000 7f84727ca700 1 Processor -- start 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.285+0000 7f84727ca700 1 -- start start 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f84727ca700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c130670 0x7f846c09e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f84727ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c133510 0x7f846c09a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f84727ca700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f846c135e30 0x7f846c09aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f84727ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c09d740 con 0x7f846c133510 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f84727ca700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c09d520 con 0x7f846c135e30 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f84727ca700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f846c1388d0 con 0x7f846c130670 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c130670 0x7f846c09e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c133510 0x7f846c09a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c130670 0x7f846c09e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53692/0 (socket says 172.21.15.89:53692) 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 -- 172.21.15.89:0/4139560514 learned_addr learned my addr 172.21.15.89:0/4139560514 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:32.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f8470d67700 1 --2- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f846c135e30 0x7f846c09aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 -- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f846c135e30 msgr2=0x7f846c09aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 --2- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f846c135e30 0x7f846c09aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 -- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c133510 msgr2=0x7f846c09a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 --2- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c133510 0x7f846c09a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 -- 172.21.15.89:0/4139560514 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460009550 con 0x7f846c130670 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f846bfff700 1 --2- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c130670 0x7f846c09e150 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f845c001eb0 tx=0x7f845c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.286+0000 7f8468ff9700 1 -- 172.21.15.89:0/4139560514 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c0054e0 con 0x7f846c130670 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.287+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8464009a70 con 0x7f846c130670 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.287+0000 7f8468ff9700 1 -- 172.21.15.89:0/4139560514 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c003680 con 0x7f846c130670 2023-09-30T22:23:32.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.287+0000 7f8468ff9700 1 -- 172.21.15.89:0/4139560514 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c0039f0 con 0x7f846c130670 2023-09-30T22:23:32.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.287+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c09c610 con 0x7f846c130670 2023-09-30T22:23:32.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.288+0000 7f8468ff9700 1 -- 172.21.15.89:0/4139560514 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f845c003b50 con 0x7f846c130670 2023-09-30T22:23:32.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.288+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c074ea0 con 0x7f846c130670 2023-09-30T22:23:32.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.288+0000 7f8468ff9700 1 -- 172.21.15.89:0/4139560514 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f845c03eab0 con 0x7f846c130670 2023-09-30T22:23:32.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.296+0000 7f8468ff9700 1 -- 172.21.15.89:0/4139560514 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f845c01b3b0 con 0x7f846c130670 2023-09-30T22:23:32.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.321+0000 7f6bd47d4700 1 Processor -- start 2023-09-30T22:23:32.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.321+0000 7f6bd47d4700 1 -- start start 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd47d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09e850 0x7f6bcc09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd47d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f590 0x7f6bcc13faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd47d4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc09aa30 con 0x7f6bcc09f160 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd47d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc06c930 con 0x7f6bcc09f590 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd47d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc06bf30 con 0x7f6bcc09e850 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd1d6f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6bcc09f160 0x7f6bcc13c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58992/0 (socket says 172.21.15.89:58992) 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd1d6f700 1 -- 172.21.15.89:0/1078389763 learned_addr learned my addr 172.21.15.89:0/1078389763 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd2570700 1 --2- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09e850 0x7f6bcc09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.322+0000 7f6bd2d71700 1 --2- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f590 0x7f6bcc13faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd156e700 1 -- 172.21.15.89:0/1078389763 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519801103 0 0) 0x7f6bcc09aa30 con 0x7f6bcc09f160 2023-09-30T22:23:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd156e700 1 -- 172.21.15.89:0/1078389763 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb80035b0 con 0x7f6bcc09f160 2023-09-30T22:23:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd2570700 1 -- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f590 msgr2=0x7f6bcc13faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd2570700 1 --2- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f590 0x7f6bcc13faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd2570700 1 -- 172.21.15.89:0/1078389763 >> v1:172.21.15.89:6789/0 conn(0x7f6bcc09f160 legacy=0x7f6bcc13c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd2570700 1 -- 172.21.15.89:0/1078389763 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc0a0250 con 0x7f6bcc09e850 2023-09-30T22:23:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd2570700 1 --2- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09e850 0x7f6bcc09ec20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009870 tx=0x7f6bc8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7e4b7edd242e97 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd156e700 1 -- 172.21.15.89:0/1078389763 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8011830 con 0x7f6bcc09e850 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd156e700 1 -- 172.21.15.89:0/1078389763 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8011990 con 0x7f6bcc09e850 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.323+0000 7f6bd156e700 1 -- 172.21.15.89:0/1078389763 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8011d20 con 0x7f6bcc09e850 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.324+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09e850 msgr2=0x7f6bcc09ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.324+0000 7f6bd47d4700 1 --2- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09e850 0x7f6bcc09ec20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009870 tx=0x7f6bc8009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.324+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1078389763 shutdown_connections 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.324+0000 7f6bd47d4700 1 --2- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f590 0x7f6bcc13faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.324+0000 7f6bd47d4700 1 --2- 172.21.15.89:0/1078389763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09e850 0x7f6bcc09ec20 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.324+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1078389763 >> 172.21.15.89:0/1078389763 conn(0x7f6bcc0a3f70 msgr2=0x7f6bcc0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.324+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1078389763 shutdown_connections 2023-09-30T22:23:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.324+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1078389763 wait complete. 2023-09-30T22:23:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd47d4700 1 Processor -- start 2023-09-30T22:23:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd47d4700 1 -- start start 2023-09-30T22:23:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd47d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bcc09e850 0x7f6bcc11c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd47d4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f160 0x7f6bcc11cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd47d4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09f590 0x7f6bcc11d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd47d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc13f010 con 0x7f6bcc09e850 2023-09-30T22:23:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd47d4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc11bcf0 con 0x7f6bcc09f160 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd47d4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc11b450 con 0x7f6bcc09f590 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd2570700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bcc09e850 0x7f6bcc11c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd2570700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bcc09e850 0x7f6bcc11c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41360/0 (socket says 172.21.15.89:41360) 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd2570700 1 -- 172.21.15.89:0/1219416814 learned_addr learned my addr 172.21.15.89:0/1219416814 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd2d71700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09f590 0x7f6bcc11d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.325+0000 7f6bd1d6f700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f160 0x7f6bcc11cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd2d71700 1 -- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f160 msgr2=0x7f6bcc11cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd2d71700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f160 0x7f6bcc11cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd2d71700 1 -- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bcc09e850 msgr2=0x7f6bcc11c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd2d71700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bcc09e850 0x7f6bcc11c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd2d71700 1 -- 172.21.15.89:0/1219416814 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc80094a0 con 0x7f6bcc09f590 2023-09-30T22:23:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd1d6f700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f160 0x7f6bcc11cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd2d71700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09f590 0x7f6bcc11d2b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400efc0 tx=0x7f6bc400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bc2ffd700 1 -- 172.21.15.89:0/1219416814 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400cd40 con 0x7f6bcc09f590 2023-09-30T22:23:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bc2ffd700 1 -- 172.21.15.89:0/1219416814 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc400f040 con 0x7f6bcc09f590 2023-09-30T22:23:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc0a0490 con 0x7f6bcc09f590 2023-09-30T22:23:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.326+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc11b970 con 0x7f6bcc09f590 2023-09-30T22:23:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.327+0000 7f6bc2ffd700 1 -- 172.21.15.89:0/1219416814 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40146c0 con 0x7f6bcc09f590 2023-09-30T22:23:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.328+0000 7f6bc2ffd700 1 -- 172.21.15.89:0/1219416814 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6bc4014820 con 0x7f6bcc09f590 2023-09-30T22:23:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.328+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb40052e0 con 0x7f6bcc09f590 2023-09-30T22:23:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.328+0000 7f6bc2ffd700 1 -- 172.21.15.89:0/1219416814 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6bc4040230 con 0x7f6bcc09f590 2023-09-30T22:23:32.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.333+0000 7f6bc2ffd700 1 -- 172.21.15.89:0/1219416814 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bc4018030 con 0x7f6bcc09f590 2023-09-30T22:23:32.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.383+0000 7f1f722b6700 1 Processor -- start 2023-09-30T22:23:32.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.383+0000 7f1f722b6700 1 -- start start 2023-09-30T22:23:32.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.383+0000 7f1f722b6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.383+0000 7f1f722b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09e410 0x7f1f6c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.383+0000 7f1f722b6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c09f130 con 0x7f1f6c09e000 2023-09-30T22:23:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.383+0000 7f1f722b6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c09f520 con 0x7f1f6c09d6f0 2023-09-30T22:23:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.383+0000 7f1f722b6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c09e930 con 0x7f1f6c09e410 2023-09-30T22:23:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f70853700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09e410 0x7f1f6c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41366/0 (socket says 172.21.15.89:41366) 2023-09-30T22:23:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6bfff700 1 -- 172.21.15.89:0/1433827996 learned_addr learned my addr 172.21.15.89:0/1433827996 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6bfff700 1 -- 172.21.15.89:0/1433827996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09e410 msgr2=0x7f1f6c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6bfff700 1 --2- 172.21.15.89:0/1433827996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09e410 0x7f1f6c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6bfff700 1 -- 172.21.15.89:0/1433827996 >> v1:172.21.15.89:6789/0 conn(0x7f1f6c09e000 legacy=0x7f1f6c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6affd700 1 -- 172.21.15.89:0/1433827996 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944969609 0 0) 0x7f1f6c09f130 con 0x7f1f6c09e000 2023-09-30T22:23:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6bfff700 1 -- 172.21.15.89:0/1433827996 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c09b2d0 con 0x7f1f6c09d6f0 2023-09-30T22:23:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f70853700 1 --2- 172.21.15.89:0/1433827996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09e410 0x7f1f6c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6bfff700 1 --2- 172.21.15.89:0/1433827996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00b9a0 tx=0x7f1f5c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396cc508c2454dce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.384+0000 7f1f6affd700 1 -- 172.21.15.89:0/1433827996 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00caa0 con 0x7f1f6c09d6f0 2023-09-30T22:23:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f6affd700 1 -- 172.21.15.89:0/1433827996 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c00cc00 con 0x7f1f6c09d6f0 2023-09-30T22:23:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f6affd700 1 -- 172.21.15.89:0/1433827996 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00f460 con 0x7f1f6c09d6f0 2023-09-30T22:23:32.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f722b6700 1 -- 172.21.15.89:0/1433827996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09d6f0 msgr2=0x7f1f6c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f722b6700 1 --2- 172.21.15.89:0/1433827996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00b9a0 tx=0x7f1f5c00bcb0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f722b6700 1 -- 172.21.15.89:0/1433827996 shutdown_connections 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f722b6700 1 --2- 172.21.15.89:0/1433827996 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09e410 0x7f1f6c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f722b6700 1 --2- 172.21.15.89:0/1433827996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f722b6700 1 -- 172.21.15.89:0/1433827996 >> 172.21.15.89:0/1433827996 conn(0x7f1f6c066f50 msgr2=0x7f1f6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f722b6700 1 -- 172.21.15.89:0/1433827996 shutdown_connections 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.385+0000 7f1f722b6700 1 -- 172.21.15.89:0/1433827996 wait complete. 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.387+0000 7f1f722b6700 1 Processor -- start 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.387+0000 7f1f722b6700 1 -- start start 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.387+0000 7f1f722b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09d6f0 0x7f1f6c11df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.387+0000 7f1f722b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6c09e000 0x7f1f6c11e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.387+0000 7f1f722b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09e410 0x7f1f6c11e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.387+0000 7f1f722b6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c1302d0 con 0x7f1f6c09e410 2023-09-30T22:23:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.387+0000 7f1f722b6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c120d50 con 0x7f1f6c09d6f0 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.387+0000 7f1f722b6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c11f3a0 con 0x7f1f6c09e000 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09e410 0x7f1f6c11e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6c09e000 0x7f1f6c11e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09e410 0x7f1f6c11e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41380/0 (socket says 172.21.15.89:41380) 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 -- 172.21.15.89:0/3340966977 learned_addr learned my addr 172.21.15.89:0/3340966977 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 -- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6c09e000 msgr2=0x7f1f6c11e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 --2- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6c09e000 0x7f1f6c11e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 -- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09d6f0 msgr2=0x7f1f6c11df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 --2- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09d6f0 0x7f1f6c11df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 -- 172.21.15.89:0/3340966977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6400f050 con 0x7f1f6c09e410 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f6b7fe700 1 --2- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6c09e000 0x7f1f6c11e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f70853700 1 --2- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09e410 0x7f1f6c11e9c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400f530 tx=0x7f1f6400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f68ff9700 1 -- 172.21.15.89:0/3340966977 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6401c070 con 0x7f1f6c09e410 2023-09-30T22:23:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f68ff9700 1 -- 172.21.15.89:0/3340966977 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f64013040 con 0x7f1f6c09e410 2023-09-30T22:23:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c00b600 con 0x7f1f6c09e410 2023-09-30T22:23:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f68ff9700 1 -- 172.21.15.89:0/3340966977 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64012070 con 0x7f1f6c09e410 2023-09-30T22:23:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.388+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c1211a0 con 0x7f1f6c09e410 2023-09-30T22:23:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.390+0000 7f1f68ff9700 1 -- 172.21.15.89:0/3340966977 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f64018460 con 0x7f1f6c09e410 2023-09-30T22:23:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.390+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c09c700 con 0x7f1f6c09e410 2023-09-30T22:23:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.390+0000 7f1f68ff9700 1 -- 172.21.15.89:0/3340966977 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1f64043b60 con 0x7f1f6c09e410 2023-09-30T22:23:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.394+0000 7f1f68ff9700 1 -- 172.21.15.89:0/3340966977 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f64018710 con 0x7f1f6c09e410 2023-09-30T22:23:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.448+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7e6c09b8a0 con 0x7f7e6c136a70 2023-09-30T22:23:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.449+0000 7f7e62ffd700 1 -- 172.21.15.89:0/258243535 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e6c09b8a0 con 0x7f7e6c136a70 2023-09-30T22:23:32.449 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:32.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.452+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c136a70 msgr2=0x7f7e6c134300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.452+0000 7f7e744d4700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c136a70 0x7f7e6c134300 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7e64016000 tx=0x7f7e64004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.452+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 shutdown_connections 2023-09-30T22:23:32.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.452+0000 7f7e744d4700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e6c136a70 0x7f7e6c134300 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.452+0000 7f7e744d4700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e6c09d9f0 0x7f7e6c1c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.452+0000 7f7e744d4700 1 --2- 172.21.15.89:0/258243535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e6c1316c0 0x7f7e6c1c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.452+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 >> 172.21.15.89:0/258243535 conn(0x7f7e6c066f50 msgr2=0x7f7e6c136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:32.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.453+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 shutdown_connections 2023-09-30T22:23:32.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.453+0000 7f7e744d4700 1 -- 172.21.15.89:0/258243535 wait complete. 2023-09-30T22:23:32.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:32.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.501+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f846c072210 con 0x7f846c130670 2023-09-30T22:23:32.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.503+0000 7f8468ff9700 1 -- 172.21.15.89:0/4139560514 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f845c017070 con 0x7f846c130670 2023-09-30T22:23:32.503 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:32.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c130670 msgr2=0x7f846c09e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 --2- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c130670 0x7f846c09e150 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f845c001eb0 tx=0x7f845c00c800 comp rx=0 tx=0).stop 2023-09-30T22:23:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 shutdown_connections 2023-09-30T22:23:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 --2- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f846c135e30 0x7f846c09aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 --2- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f846c130670 0x7f846c09e150 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 --2- 172.21.15.89:0/4139560514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c133510 0x7f846c09a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 >> 172.21.15.89:0/4139560514 conn(0x7f846c0a2aa0 msgr2=0x7f846c06fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 shutdown_connections 2023-09-30T22:23:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.505+0000 7f84727ca700 1 -- 172.21.15.89:0/4139560514 wait complete. 2023-09-30T22:23:32.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:32.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.536+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6bb4005590 con 0x7f6bcc09f590 2023-09-30T22:23:32.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.536+0000 7f6bc2ffd700 1 -- 172.21.15.89:0/1219416814 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bc4007920 con 0x7f6bcc09f590 2023-09-30T22:23:32.537 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:32.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.538+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09f590 msgr2=0x7f6bcc11d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.538+0000 7f6bd47d4700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09f590 0x7f6bcc11d2b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400efc0 tx=0x7f6bc400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.539+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 shutdown_connections 2023-09-30T22:23:32.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.539+0000 7f6bd47d4700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bcc09f590 0x7f6bcc11d2b0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.539+0000 7f6bd47d4700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bcc09e850 0x7f6bcc11c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.539+0000 7f6bd47d4700 1 --2- 172.21.15.89:0/1219416814 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6bcc09f160 0x7f6bcc11cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.539+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 >> 172.21.15.89:0/1219416814 conn(0x7f6bcc0a3f70 msgr2=0x7f6bcc0a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:32.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.539+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 shutdown_connections 2023-09-30T22:23:32.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.539+0000 7f6bd47d4700 1 -- 172.21.15.89:0/1219416814 wait complete. 2023-09-30T22:23:32.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.593+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1f6c074ea0 con 0x7f1f6c09e410 2023-09-30T22:23:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.594+0000 7f1f68ff9700 1 -- 172.21.15.89:0/3340966977 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f6401f310 con 0x7f1f6c09e410 2023-09-30T22:23:32.594 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09e410 msgr2=0x7f1f6c11e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 --2- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09e410 0x7f1f6c11e9c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400f530 tx=0x7f1f6400f840 comp rx=0 tx=0).stop 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 shutdown_connections 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 --2- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f6c09e410 0x7f1f6c11e9c0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 --2- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1f6c09d6f0 0x7f1f6c11df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 --2- 172.21.15.89:0/3340966977 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1f6c09e000 0x7f1f6c11e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 >> 172.21.15.89:0/3340966977 conn(0x7f1f6c066f50 msgr2=0x7f1f6c070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 shutdown_connections 2023-09-30T22:23:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:32.596+0000 7f1f722b6700 1 -- 172.21.15.89:0/3340966977 wait complete. 2023-09-30T22:23:32.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:33.470 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:33.518 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:33.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:33.609 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.621+0000 7fa85d2fe700 1 Processor -- start 2023-09-30T22:23:33.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.621+0000 7fa85d2fe700 1 -- start start 2023-09-30T22:23:33.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.621+0000 7fa85d2fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85812cf60 0x7fa85806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.621+0000 7fa85d2fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa85812f8b0 0x7fa85812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.621+0000 7fa85d2fe700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85812cd90 con 0x7fa858130260 2023-09-30T22:23:33.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.621+0000 7fa85d2fe700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85812d330 con 0x7fa85812f8b0 2023-09-30T22:23:33.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.621+0000 7fa85d2fe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581306e0 con 0x7fa85812cf60 2023-09-30T22:23:33.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa856ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85812cf60 0x7fa85806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa856ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85812cf60 0x7fa85806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39942/0 (socket says 172.21.15.89:39942) 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa8567fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa85812f8b0 0x7fa85812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa856ffd700 1 -- 172.21.15.89:0/4262055459 learned_addr learned my addr 172.21.15.89:0/4262055459 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa856ffd700 1 -- 172.21.15.89:0/4262055459 >> v2:172.21.15.89:3300/0 conn(0x7fa85812f8b0 msgr2=0x7fa85812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa856ffd700 1 --2- 172.21.15.89:0/4262055459 >> v2:172.21.15.89:3300/0 conn(0x7fa85812f8b0 0x7fa85812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa856ffd700 1 -- 172.21.15.89:0/4262055459 >> v1:172.21.15.89:6789/0 conn(0x7fa858130260 legacy=0x7fa858136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa856ffd700 1 -- 172.21.15.89:0/4262055459 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa858071780 con 0x7fa85812cf60 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa856ffd700 1 --2- 172.21.15.89:0/4262055459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85812cf60 0x7fa85806f5d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa848014fb0 tx=0x7fa84800b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3252b762dfc16289 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.622+0000 7fa855ffb700 1 -- 172.21.15.89:0/4262055459 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848004230 con 0x7fa85812cf60 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.623+0000 7fa855ffb700 1 -- 172.21.15.89:0/4262055459 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa848004390 con 0x7fa85812cf60 2023-09-30T22:23:33.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.623+0000 7fa855ffb700 1 -- 172.21.15.89:0/4262055459 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84800be80 con 0x7fa85812cf60 2023-09-30T22:23:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.623+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4262055459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85812cf60 msgr2=0x7fa85806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.623+0000 7fa85d2fe700 1 --2- 172.21.15.89:0/4262055459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85812cf60 0x7fa85806f5d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa848014fb0 tx=0x7fa84800b7c0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.624+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4262055459 shutdown_connections 2023-09-30T22:23:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.624+0000 7fa85d2fe700 1 --2- 172.21.15.89:0/4262055459 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa85812cf60 0x7fa85806f5d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.624+0000 7fa85d2fe700 1 --2- 172.21.15.89:0/4262055459 >> v2:172.21.15.89:3300/0 conn(0x7fa85812f8b0 0x7fa85812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.624+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4262055459 >> 172.21.15.89:0/4262055459 conn(0x7fa8580a2aa0 msgr2=0x7fa8580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.624+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4262055459 shutdown_connections 2023-09-30T22:23:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.624+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4262055459 wait complete. 2023-09-30T22:23:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.625+0000 7fa85d2fe700 1 Processor -- start 2023-09-30T22:23:33.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa85d2fe700 1 -- start start 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa85d2fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa85812cf60 0x7fa85809bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa85d2fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85812f8b0 0x7fa85809c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa85d2fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa858130260 0x7fa8581c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa85d2fe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85809d740 con 0x7fa85812f8b0 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa85d2fe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85809b810 con 0x7fa858130260 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa85d2fe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa85809e810 con 0x7fa85812cf60 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa856ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa85812cf60 0x7fa85809bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.626+0000 7fa856ffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa85812cf60 0x7fa85809bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53718/0 (socket says 172.21.15.89:53718) 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa856ffd700 1 -- 172.21.15.89:0/4244963999 learned_addr learned my addr 172.21.15.89:0/4244963999 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa8567fc700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85812f8b0 0x7fa85809c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa8577fe700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa858130260 0x7fa8581c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa856ffd700 1 -- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa858130260 msgr2=0x7fa8581c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa856ffd700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa858130260 0x7fa8581c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa856ffd700 1 -- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85812f8b0 msgr2=0x7fa85809c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa856ffd700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85812f8b0 0x7fa85809c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa856ffd700 1 -- 172.21.15.89:0/4244963999 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848014c10 con 0x7fa85812cf60 2023-09-30T22:23:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.627+0000 7fa856ffd700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa85812cf60 0x7fa85809bbf0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa84801e040 tx=0x7fa848004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:33.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.628+0000 7fa83f7fe700 1 -- 172.21.15.89:0/4244963999 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848026450 con 0x7fa85812cf60 2023-09-30T22:23:33.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.628+0000 7fa83f7fe700 1 -- 172.21.15.89:0/4244963999 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8480265b0 con 0x7fa85812cf60 2023-09-30T22:23:33.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.628+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85812d7b0 con 0x7fa85812cf60 2023-09-30T22:23:33.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.628+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa858129ff0 con 0x7fa85812cf60 2023-09-30T22:23:33.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.628+0000 7fa83f7fe700 1 -- 172.21.15.89:0/4244963999 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848026940 con 0x7fa85812cf60 2023-09-30T22:23:33.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.629+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8380052e0 con 0x7fa85812cf60 2023-09-30T22:23:33.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.629+0000 7fa83f7fe700 1 -- 172.21.15.89:0/4244963999 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa848026aa0 con 0x7fa85812cf60 2023-09-30T22:23:33.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.630+0000 7fa83f7fe700 1 -- 172.21.15.89:0/4244963999 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa848049cc0 con 0x7fa85812cf60 2023-09-30T22:23:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.638+0000 7fa83f7fe700 1 -- 172.21.15.89:0/4244963999 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa84802f3b0 con 0x7fa85812cf60 2023-09-30T22:23:33.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.659+0000 7f2d7d1ed700 1 Processor -- start 2023-09-30T22:23:33.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.659+0000 7f2d7d1ed700 1 -- start start 2023-09-30T22:23:33.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7d1ed700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2d7806e570 0x7f2d7806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7d1ed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d7806ee80 0x7f2d78134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7d1ed700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d78070ee0 con 0x7f2d7809eca0 2023-09-30T22:23:33.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7d1ed700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d78071480 con 0x7f2d7806e570 2023-09-30T22:23:33.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7d1ed700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78071b90 con 0x7f2d7806ee80 2023-09-30T22:23:33.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d76d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2d7806e570 0x7f2d7806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7759e700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d7806ee80 0x7f2d78134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d76d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2d7806e570 0x7f2d7806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41416/0 (socket says 172.21.15.89:41416) 2023-09-30T22:23:33.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d76d9d700 1 -- 172.21.15.89:0/3055196835 learned_addr learned my addr 172.21.15.89:0/3055196835 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:33.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7759e700 1 --2- 172.21.15.89:0/3055196835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d7806ee80 0x7f2d78134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53730/0 (socket says 172.21.15.89:53730) 2023-09-30T22:23:33.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7759e700 1 -- 172.21.15.89:0/3055196835 >> v2:172.21.15.89:3300/0 conn(0x7f2d7806e570 msgr2=0x7f2d7806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:33.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7759e700 1 --2- 172.21.15.89:0/3055196835 >> v2:172.21.15.89:3300/0 conn(0x7f2d7806e570 0x7f2d7806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7759e700 1 -- 172.21.15.89:0/3055196835 >> v1:172.21.15.89:6789/0 conn(0x7f2d7809eca0 legacy=0x7f2d78134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7759e700 1 -- 172.21.15.89:0/3055196835 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780711a0 con 0x7f2d7806ee80 2023-09-30T22:23:33.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d76d9d700 1 --2- 172.21.15.89:0/3055196835 >> v2:172.21.15.89:3300/0 conn(0x7f2d7806e570 0x7f2d7806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:33.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.660+0000 7f2d7759e700 1 --2- 172.21.15.89:0/3055196835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d7806ee80 0x7f2d78134d20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009d90 tx=0x7f2d7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f8e98063714e65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:33.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d75d9b700 1 -- 172.21.15.89:0/3055196835 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000e070 con 0x7f2d7806ee80 2023-09-30T22:23:33.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d75d9b700 1 -- 172.21.15.89:0/3055196835 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000bb20 con 0x7f2d7806ee80 2023-09-30T22:23:33.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d75d9b700 1 -- 172.21.15.89:0/3055196835 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70012430 con 0x7f2d7806ee80 2023-09-30T22:23:33.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/3055196835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d7806ee80 msgr2=0x7f2d78134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d7d1ed700 1 --2- 172.21.15.89:0/3055196835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d7806ee80 0x7f2d78134d20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009d90 tx=0x7f2d7000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/3055196835 shutdown_connections 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d7d1ed700 1 --2- 172.21.15.89:0/3055196835 >> v2:172.21.15.89:3300/0 conn(0x7f2d7806e570 0x7f2d7806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d7d1ed700 1 --2- 172.21.15.89:0/3055196835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d7806ee80 0x7f2d78134d20 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.661+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/3055196835 >> 172.21.15.89:0/3055196835 conn(0x7f2d780a2aa0 msgr2=0x7f2d780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.662+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/3055196835 shutdown_connections 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.662+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/3055196835 wait complete. 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7d1ed700 1 Processor -- start 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7d1ed700 1 -- start start 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7d1ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d7806e570 0x7f2d78123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7d1ed700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d7806ee80 0x7f2d78123780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7d1ed700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d78123cc0 0x7f2d781d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7d1ed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78124710 con 0x7f2d7806e570 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7d1ed700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78126d60 con 0x7f2d7806ee80 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7d1ed700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78124e50 con 0x7f2d78123cc0 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d7806ee80 0x7f2d78123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d7806ee80 0x7f2d78123780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39952/0 (socket says 172.21.15.89:39952) 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d7659c700 1 -- 172.21.15.89:0/1171110743 learned_addr learned my addr 172.21.15.89:0/1171110743 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.663+0000 7f2d76d9d700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d7806e570 0x7f2d78123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7759e700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d78123cc0 0x7f2d781d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7659c700 1 -- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d78123cc0 msgr2=0x7f2d781d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7659c700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d78123cc0 0x7f2d781d2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7659c700 1 -- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d7806e570 msgr2=0x7f2d78123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7659c700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d7806e570 0x7f2d78123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7659c700 1 -- 172.21.15.89:0/1171110743 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d68009550 con 0x7f2d7806ee80 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7659c700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d7806ee80 0x7f2d78123780 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c480 tx=0x7f2d6c00c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d5f7fe700 1 -- 172.21.15.89:0/1171110743 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0052d0 con 0x7f2d7806ee80 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d5f7fe700 1 -- 172.21.15.89:0/1171110743 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c00ce90 con 0x7f2d7806ee80 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d5f7fe700 1 -- 172.21.15.89:0/1171110743 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c003680 con 0x7f2d7806ee80 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d70009a70 con 0x7f2d7806ee80 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.664+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d78134190 con 0x7f2d7806ee80 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.665+0000 7f2d5f7fe700 1 -- 172.21.15.89:0/1171110743 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2d6c0037e0 con 0x7f2d7806ee80 2023-09-30T22:23:33.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.665+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d78074ea0 con 0x7f2d7806ee80 2023-09-30T22:23:33.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.666+0000 7f2d5f7fe700 1 -- 172.21.15.89:0/1171110743 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2d6c03ea30 con 0x7f2d7806ee80 2023-09-30T22:23:33.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.670+0000 7f2d5f7fe700 1 -- 172.21.15.89:0/1171110743 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2d6c011590 con 0x7f2d7806ee80 2023-09-30T22:23:33.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f6426e67700 1 Processor -- start 2023-09-30T22:23:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f6426e67700 1 -- start start 2023-09-30T22:23:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f6426e67700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009d6f0 0x7f642009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f6426e67700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f642009e000 0x7f6420130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f6426e67700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642009f130 con 0x7f642009e410 2023-09-30T22:23:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f6426e67700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642009f520 con 0x7f642009e000 2023-09-30T22:23:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f6426e67700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642009e930 con 0x7f642009d6f0 2023-09-30T22:23:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f641ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f642009e000 0x7f6420130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f641ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f642009e000 0x7f6420130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41444/0 (socket says 172.21.15.89:41444) 2023-09-30T22:23:33.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f641ffff700 1 -- 172.21.15.89:0/125495812 learned_addr learned my addr 172.21.15.89:0/125495812 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:33.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.699+0000 7f6425404700 1 --1- 172.21.15.89:0/125495812 >> v1:172.21.15.89:6789/0 conn(0x7f642009e410 0x7f6420130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59014/0 (socket says 172.21.15.89:59014) 2023-09-30T22:23:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f6424c03700 1 --2- 172.21.15.89:0/125495812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009d6f0 0x7f642009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f641ffff700 1 -- 172.21.15.89:0/125495812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009d6f0 msgr2=0x7f642009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f641ffff700 1 --2- 172.21.15.89:0/125495812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009d6f0 0x7f642009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f641ffff700 1 -- 172.21.15.89:0/125495812 >> v1:172.21.15.89:6789/0 conn(0x7f642009e410 legacy=0x7f6420130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f641ffff700 1 -- 172.21.15.89:0/125495812 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642009b2d0 con 0x7f642009e000 2023-09-30T22:23:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f641ffff700 1 --2- 172.21.15.89:0/125495812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 0x7f6420130660 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f640c00bb90 tx=0x7f640c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63da837c2e742483 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f641f7fe700 1 -- 172.21.15.89:0/125495812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00dd30 con 0x7f642009e000 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f641f7fe700 1 -- 172.21.15.89:0/125495812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c00de90 con 0x7f642009e000 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.700+0000 7f641f7fe700 1 -- 172.21.15.89:0/125495812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c010670 con 0x7f642009e000 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.701+0000 7f6426e67700 1 -- 172.21.15.89:0/125495812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 msgr2=0x7f6420130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.701+0000 7f6426e67700 1 --2- 172.21.15.89:0/125495812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 0x7f6420130660 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f640c00bb90 tx=0x7f640c00bea0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.701+0000 7f6426e67700 1 -- 172.21.15.89:0/125495812 shutdown_connections 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.701+0000 7f6426e67700 1 --2- 172.21.15.89:0/125495812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009d6f0 0x7f642009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.701+0000 7f6426e67700 1 --2- 172.21.15.89:0/125495812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 0x7f6420130660 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.701+0000 7f6426e67700 1 -- 172.21.15.89:0/125495812 >> 172.21.15.89:0/125495812 conn(0x7f6420066f50 msgr2=0x7f642009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.701+0000 7f6426e67700 1 -- 172.21.15.89:0/125495812 shutdown_connections 2023-09-30T22:23:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.701+0000 7f6426e67700 1 -- 172.21.15.89:0/125495812 wait complete. 2023-09-30T22:23:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.702+0000 7f6426e67700 1 Processor -- start 2023-09-30T22:23:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.702+0000 7f6426e67700 1 -- start start 2023-09-30T22:23:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.702+0000 7f6426e67700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f642009d6f0 0x7f642011e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.702+0000 7f6426e67700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 0x7f642011e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6426e67700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009e410 0x7f64201cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6426e67700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642011fc30 con 0x7f642009e000 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6426e67700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6420120250 con 0x7f642009e410 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6426e67700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f642009c700 con 0x7f642009d6f0 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6425404700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009e410 0x7f64201cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6425404700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009e410 0x7f64201cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39980/0 (socket says 172.21.15.89:39980) 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6425404700 1 -- 172.21.15.89:0/3733109518 learned_addr learned my addr 172.21.15.89:0/3733109518 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f641ffff700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 0x7f642011e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6424c03700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f642009d6f0 0x7f642011e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6425404700 1 -- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f642009d6f0 msgr2=0x7f642011e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6425404700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f642009d6f0 0x7f642011e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6425404700 1 -- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 msgr2=0x7f642011e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6425404700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 0x7f642011e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f6425404700 1 -- 172.21.15.89:0/3733109518 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c00b7f0 con 0x7f642009e410 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.703+0000 7f641ffff700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 0x7f642011e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.704+0000 7f6425404700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009e410 0x7f64201cc600 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f641800cba0 tx=0x7f641800ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.704+0000 7f641d7fa700 1 -- 172.21.15.89:0/3733109518 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641800aad0 con 0x7f642009e410 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.704+0000 7f641d7fa700 1 -- 172.21.15.89:0/3733109518 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641800ac30 con 0x7f642009e410 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.704+0000 7f641d7fa700 1 -- 172.21.15.89:0/3733109518 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418007070 con 0x7f642009e410 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.705+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64200a1d90 con 0x7f642009e410 2023-09-30T22:23:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.705+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6420120420 con 0x7f642009e410 2023-09-30T22:23:33.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.706+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6420121500 con 0x7f642009e410 2023-09-30T22:23:33.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.706+0000 7f641d7fa700 1 -- 172.21.15.89:0/3733109518 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64180071d0 con 0x7f642009e410 2023-09-30T22:23:33.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.706+0000 7f641d7fa700 1 -- 172.21.15.89:0/3733109518 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f641803f430 con 0x7f642009e410 2023-09-30T22:23:33.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.715+0000 7f641d7fa700 1 -- 172.21.15.89:0/3733109518 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6420121500 con 0x7f642009e410 2023-09-30T22:23:33.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.750+0000 7f29ef551700 1 Processor -- start 2023-09-30T22:23:33.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.751+0000 7f29ef551700 1 -- start start 2023-09-30T22:23:33.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.751+0000 7f29ef551700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e80a0a30 0x7f29e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.751+0000 7f29ef551700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e8138d80 0x7f29e813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.751+0000 7f29ef551700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e80a2a70 con 0x7f29e809d630 2023-09-30T22:23:33.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.751+0000 7f29ef551700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80a01d0 con 0x7f29e8138d80 2023-09-30T22:23:33.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.751+0000 7f29ef551700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29e809e2b0 con 0x7f29e80a0a30 2023-09-30T22:23:33.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29edaee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e8138d80 0x7f29e813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29edaee700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e8138d80 0x7f29e813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:39982/0 (socket says 172.21.15.89:39982) 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29edaee700 1 -- 172.21.15.89:0/2791470916 learned_addr learned my addr 172.21.15.89:0/2791470916 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29ed2ed700 1 --2- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e80a0a30 0x7f29e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29ed2ed700 1 -- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e8138d80 msgr2=0x7f29e813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29ed2ed700 1 --2- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e8138d80 0x7f29e813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29ed2ed700 1 -- 172.21.15.89:0/2791470916 >> v1:172.21.15.89:6789/0 conn(0x7f29e809d630 legacy=0x7f29e80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29ed2ed700 1 -- 172.21.15.89:0/2791470916 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e809dc50 con 0x7f29e80a0a30 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29edaee700 1 --2- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e8138d80 0x7f29e813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29ed2ed700 1 --2- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e80a0a30 0x7f29e80a0e00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009870 tx=0x7f29e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaff54cb16352045 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29dffff700 1 -- 172.21.15.89:0/2791470916 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4011830 con 0x7f29e80a0a30 2023-09-30T22:23:33.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29dffff700 1 -- 172.21.15.89:0/2791470916 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e4011990 con 0x7f29e80a0a30 2023-09-30T22:23:33.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.752+0000 7f29dffff700 1 -- 172.21.15.89:0/2791470916 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4011d20 con 0x7f29e80a0a30 2023-09-30T22:23:33.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.753+0000 7f29ef551700 1 -- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e80a0a30 msgr2=0x7f29e80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.753+0000 7f29ef551700 1 --2- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e80a0a30 0x7f29e80a0e00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009870 tx=0x7f29e4009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:33.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.753+0000 7f29ef551700 1 -- 172.21.15.89:0/2791470916 shutdown_connections 2023-09-30T22:23:33.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.753+0000 7f29ef551700 1 --2- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e8138d80 0x7f29e813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.753+0000 7f29ef551700 1 --2- 172.21.15.89:0/2791470916 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e80a0a30 0x7f29e80a0e00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.753+0000 7f29ef551700 1 -- 172.21.15.89:0/2791470916 >> 172.21.15.89:0/2791470916 conn(0x7f29e80a3050 msgr2=0x7f29e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:33.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.753+0000 7f29ef551700 1 -- 172.21.15.89:0/2791470916 shutdown_connections 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.753+0000 7f29ef551700 1 -- 172.21.15.89:0/2791470916 wait complete. 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.754+0000 7f29ef551700 1 Processor -- start 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.754+0000 7f29ef551700 1 -- start start 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ef551700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e809d630 0x7f29e81ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ef551700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e80a0a30 0x7f29e81cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ef551700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e8138d80 0x7f29e8124fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ef551700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e8128450 con 0x7f29e80a0a30 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ef551700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e8138730 con 0x7f29e809d630 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ef551700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29e813aff0 con 0x7f29e8138d80 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ecaec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e80a0a30 0x7f29e81cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29edaee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e8138d80 0x7f29e8124fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ecaec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e80a0a30 0x7f29e81cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41472/0 (socket says 172.21.15.89:41472) 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29edaee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e8138d80 0x7f29e8124fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53758/0 (socket says 172.21.15.89:53758) 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.755+0000 7f29ecaec700 1 -- 172.21.15.89:0/3420957087 learned_addr learned my addr 172.21.15.89:0/3420957087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29ed2ed700 1 --2- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e809d630 0x7f29e81ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29edaee700 1 -- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e809d630 msgr2=0x7f29e81ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29edaee700 1 --2- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e809d630 0x7f29e81ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29edaee700 1 -- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e80a0a30 msgr2=0x7f29e81cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29edaee700 1 --2- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e80a0a30 0x7f29e81cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29edaee700 1 -- 172.21.15.89:0/3420957087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e0009b30 con 0x7f29e8138d80 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29edaee700 1 --2- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e8138d80 0x7f29e8124fd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f29e80a23a0 tx=0x7f29e000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:33.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29ddffb700 1 -- 172.21.15.89:0/3420957087 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000c9e0 con 0x7f29e8138d80 2023-09-30T22:23:33.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e40094a0 con 0x7f29e8138d80 2023-09-30T22:23:33.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e81c06d0 con 0x7f29e8138d80 2023-09-30T22:23:33.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29ddffb700 1 -- 172.21.15.89:0/3420957087 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e000f040 con 0x7f29e8138d80 2023-09-30T22:23:33.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.756+0000 7f29ddffb700 1 -- 172.21.15.89:0/3420957087 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000cd80 con 0x7f29e8138d80 2023-09-30T22:23:33.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.757+0000 7f29ddffb700 1 -- 172.21.15.89:0/3420957087 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f29e0014480 con 0x7f29e8138d80 2023-09-30T22:23:33.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.758+0000 7f29ddffb700 1 -- 172.21.15.89:0/3420957087 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f29e003fdb0 con 0x7f29e8138d80 2023-09-30T22:23:33.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.757+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c80052e0 con 0x7f29e8138d80 2023-09-30T22:23:33.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.763+0000 7f29ddffb700 1 -- 172.21.15.89:0/3420957087 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29e001c930 con 0x7f29e8138d80 2023-09-30T22:23:33.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.837+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa838005590 con 0x7fa85812cf60 2023-09-30T22:23:33.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.837+0000 7fa83f7fe700 1 -- 172.21.15.89:0/4244963999 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa84802b070 con 0x7fa85812cf60 2023-09-30T22:23:33.839 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:33.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.841+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa85812cf60 msgr2=0x7fa85809bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.841+0000 7fa85d2fe700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa85812cf60 0x7fa85809bbf0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa84801e040 tx=0x7fa848004750 comp rx=0 tx=0).stop 2023-09-30T22:23:33.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.842+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 shutdown_connections 2023-09-30T22:23:33.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.842+0000 7fa85d2fe700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa858130260 0x7fa8581c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.842+0000 7fa85d2fe700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa85812cf60 0x7fa85809bbf0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.842+0000 7fa85d2fe700 1 --2- 172.21.15.89:0/4244963999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85812f8b0 0x7fa85809c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.842+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 >> 172.21.15.89:0/4244963999 conn(0x7fa8580a2aa0 msgr2=0x7fa85806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:33.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.842+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 shutdown_connections 2023-09-30T22:23:33.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.842+0000 7fa85d2fe700 1 -- 172.21.15.89:0/4244963999 wait complete. 2023-09-30T22:23:33.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:33.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.871+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2d7806b9e0 con 0x7f2d7806ee80 2023-09-30T22:23:33.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.871+0000 7f2d5f7fe700 1 -- 172.21.15.89:0/1171110743 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d6c0192c0 con 0x7f2d7806ee80 2023-09-30T22:23:33.872 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d7806ee80 msgr2=0x7f2d78123780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d7806ee80 0x7f2d78123780 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c480 tx=0x7f2d6c00c790 comp rx=0 tx=0).stop 2023-09-30T22:23:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 shutdown_connections 2023-09-30T22:23:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d78123cc0 0x7f2d781d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d7806e570 0x7f2d78123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 --2- 172.21.15.89:0/1171110743 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d7806ee80 0x7f2d78123780 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 >> 172.21.15.89:0/1171110743 conn(0x7f2d780a2aa0 msgr2=0x7f2d780a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:33.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 shutdown_connections 2023-09-30T22:23:33.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.873+0000 7f2d7d1ed700 1 -- 172.21.15.89:0/1171110743 wait complete. 2023-09-30T22:23:33.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:33.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.924+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6420121500 con 0x7f642009e410 2023-09-30T22:23:33.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.925+0000 7f641d7fa700 1 -- 172.21.15.89:0/3733109518 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6420121500 con 0x7f642009e410 2023-09-30T22:23:33.925 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:33.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.926+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009e410 msgr2=0x7f64201cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.926+0000 7f6426e67700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009e410 0x7f64201cc600 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f641800cba0 tx=0x7f641800ceb0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.927+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 shutdown_connections 2023-09-30T22:23:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.927+0000 7f6426e67700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f642009e410 0x7f64201cc600 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.927+0000 7f6426e67700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f642009d6f0 0x7f642011e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.927+0000 7f6426e67700 1 --2- 172.21.15.89:0/3733109518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f642009e000 0x7f642011e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.927+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 >> 172.21.15.89:0/3733109518 conn(0x7f6420066f50 msgr2=0x7f642006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.927+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 shutdown_connections 2023-09-30T22:23:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.927+0000 7f6426e67700 1 -- 172.21.15.89:0/3733109518 wait complete. 2023-09-30T22:23:33.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:33.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.971+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f29c8005590 con 0x7f29e8138d80 2023-09-30T22:23:33.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.971+0000 7f29ddffb700 1 -- 172.21.15.89:0/3420957087 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29e002c090 con 0x7f29e8138d80 2023-09-30T22:23:33.972 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:33.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.973+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e8138d80 msgr2=0x7f29e8124fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:33.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.973+0000 7f29ef551700 1 --2- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e8138d80 0x7f29e8124fd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f29e80a23a0 tx=0x7f29e000ef90 comp rx=0 tx=0).stop 2023-09-30T22:23:33.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.973+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 shutdown_connections 2023-09-30T22:23:33.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.973+0000 7f29ef551700 1 --2- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29e8138d80 0x7f29e8124fd0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.973+0000 7f29ef551700 1 --2- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29e809d630 0x7f29e81ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.973+0000 7f29ef551700 1 --2- 172.21.15.89:0/3420957087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e80a0a30 0x7f29e81cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:33.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.974+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 >> 172.21.15.89:0/3420957087 conn(0x7f29e80a3050 msgr2=0x7f29e8139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:33.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.974+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 shutdown_connections 2023-09-30T22:23:33.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:33.974+0000 7f29ef551700 1 -- 172.21.15.89:0/3420957087 wait complete. 2023-09-30T22:23:33.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:34.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:34.887 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:34.940 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:34.986 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:35.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.012+0000 7fad1e274700 1 Processor -- start 2023-09-30T22:23:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.012+0000 7fad1e274700 1 -- start start 2023-09-30T22:23:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.012+0000 7fad1e274700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad18130200 0x7fad1806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.012+0000 7fad1e274700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad181305d0 0x7fad1809de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.012+0000 7fad1e274700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1806fc00 con 0x7fad18130a20 2023-09-30T22:23:35.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.012+0000 7fad1e274700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1812fdd0 con 0x7fad181305d0 2023-09-30T22:23:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.012+0000 7fad1e274700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad18066680 con 0x7fad18130200 2023-09-30T22:23:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad1c811700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fad18130a20 0x7fad1809e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59028/0 (socket says 172.21.15.89:59028) 2023-09-30T22:23:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad1c811700 1 -- 172.21.15.89:0/2413726956 learned_addr learned my addr 172.21.15.89:0/2413726956 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad17fff700 1 --2- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad18130200 0x7fad1806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad177fe700 1 --2- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad181305d0 0x7fad1809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad177fe700 1 -- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad18130200 msgr2=0x7fad1806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad177fe700 1 --2- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad18130200 0x7fad1806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad177fe700 1 -- 172.21.15.89:0/2413726956 >> v1:172.21.15.89:6789/0 conn(0x7fad18130a20 legacy=0x7fad1809e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad177fe700 1 -- 172.21.15.89:0/2413726956 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1806fee0 con 0x7fad181305d0 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad17fff700 1 --2- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad18130200 0x7fad1806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad177fe700 1 --2- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad181305d0 0x7fad1809de50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009870 tx=0x7fad0c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b291118ddaa20a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.013+0000 7fad16ffd700 1 -- 172.21.15.89:0/2413726956 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c011830 con 0x7fad181305d0 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.014+0000 7fad16ffd700 1 -- 172.21.15.89:0/2413726956 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c011990 con 0x7fad181305d0 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.014+0000 7fad16ffd700 1 -- 172.21.15.89:0/2413726956 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c011d20 con 0x7fad181305d0 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.014+0000 7fad1e274700 1 -- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad181305d0 msgr2=0x7fad1809de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.014+0000 7fad1e274700 1 --2- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad181305d0 0x7fad1809de50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009870 tx=0x7fad0c009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:35.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.014+0000 7fad1e274700 1 -- 172.21.15.89:0/2413726956 shutdown_connections 2023-09-30T22:23:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.014+0000 7fad1e274700 1 --2- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad18130200 0x7fad1806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.014+0000 7fad1e274700 1 --2- 172.21.15.89:0/2413726956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad181305d0 0x7fad1809de50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.014+0000 7fad1e274700 1 -- 172.21.15.89:0/2413726956 >> 172.21.15.89:0/2413726956 conn(0x7fad180a27f0 msgr2=0x7fad180a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.015+0000 7fad1e274700 1 -- 172.21.15.89:0/2413726956 shutdown_connections 2023-09-30T22:23:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.015+0000 7fad1e274700 1 -- 172.21.15.89:0/2413726956 wait complete. 2023-09-30T22:23:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.016+0000 7fad1e274700 1 Processor -- start 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.016+0000 7fad1e274700 1 -- start start 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1e274700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad18130200 0x7fad18118c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1e274700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad181305d0 0x7fad181191a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1e274700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad18130a20 0x7fad181196e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1e274700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1811ad40 con 0x7fad18130a20 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1e274700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1811af20 con 0x7fad18130200 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1e274700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad18139e60 con 0x7fad181305d0 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad18130a20 0x7fad181196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad18130a20 0x7fad181196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41480/0 (socket says 172.21.15.89:41480) 2023-09-30T22:23:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 -- 172.21.15.89:0/1649280649 learned_addr learned my addr 172.21.15.89:0/1649280649 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad177fe700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad181305d0 0x7fad181191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad17fff700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad18130200 0x7fad18118c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 -- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad181305d0 msgr2=0x7fad181191a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad181305d0 0x7fad181191a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 -- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad18130200 msgr2=0x7fad18118c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad18130200 0x7fad18118c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 -- 172.21.15.89:0/1649280649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080094a0 con 0x7fad18130a20 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.017+0000 7fad1c811700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad18130a20 0x7fad181196e0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fad1000cbc0 tx=0x7fad1000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.018+0000 7fad14ff9700 1 -- 172.21.15.89:0/1649280649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000a9e0 con 0x7fad18130a20 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.018+0000 7fad14ff9700 1 -- 172.21.15.89:0/1649280649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1000ab40 con 0x7fad18130a20 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.018+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c0094a0 con 0x7fad18130a20 2023-09-30T22:23:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.018+0000 7fad14ff9700 1 -- 172.21.15.89:0/1649280649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10012400 con 0x7fad18130a20 2023-09-30T22:23:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.018+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad18072000 con 0x7fad18130a20 2023-09-30T22:23:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.019+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad181395e0 con 0x7fad18130a20 2023-09-30T22:23:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.020+0000 7fad14ff9700 1 -- 172.21.15.89:0/1649280649 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad1001b3b0 con 0x7fad18130a20 2023-09-30T22:23:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.020+0000 7fad14ff9700 1 -- 172.21.15.89:0/1649280649 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fad1003f3a0 con 0x7fad18130a20 2023-09-30T22:23:35.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.029+0000 7fad14ff9700 1 -- 172.21.15.89:0/1649280649 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad100192c0 con 0x7fad18130a20 2023-09-30T22:23:35.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.052+0000 7f420c877700 1 Processor -- start 2023-09-30T22:23:35.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.052+0000 7f420c877700 1 -- start start 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f420c877700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f420809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f420c877700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e410 0x7f4208130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f420c877700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420809f130 con 0x7f420809e000 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f420c877700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420809f520 con 0x7f420809d6f0 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f420c877700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f420809e930 con 0x7f420809e410 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f420659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f420809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f420659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f420809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40002/0 (socket says 172.21.15.89:40002) 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f420659c700 1 -- 172.21.15.89:0/2893408299 learned_addr learned my addr 172.21.15.89:0/2893408299 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.053+0000 7f4206d9d700 1 --2- 172.21.15.89:0/2893408299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e410 0x7f4208130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420659c700 1 -- 172.21.15.89:0/2893408299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e410 msgr2=0x7f4208130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420659c700 1 --2- 172.21.15.89:0/2893408299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e410 0x7f4208130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420559a700 1 -- 172.21.15.89:0/2893408299 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410547054 0 0) 0x7f420809f130 con 0x7f420809e000 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420659c700 1 -- 172.21.15.89:0/2893408299 >> v1:172.21.15.89:6789/0 conn(0x7f420809e000 legacy=0x7f4208130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420659c700 1 -- 172.21.15.89:0/2893408299 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420809b2d0 con 0x7f420809d6f0 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420659c700 1 --2- 172.21.15.89:0/2893408299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f420809dac0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f41f800bbc0 tx=0x7f41f800bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4a228f391d726f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420559a700 1 -- 172.21.15.89:0/2893408299 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800dd30 con 0x7f420809d6f0 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420559a700 1 -- 172.21.15.89:0/2893408299 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f800de90 con 0x7f420809d6f0 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.054+0000 7f420559a700 1 -- 172.21.15.89:0/2893408299 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8010690 con 0x7f420809d6f0 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.055+0000 7f420c877700 1 -- 172.21.15.89:0/2893408299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 msgr2=0x7f420809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.055+0000 7f420c877700 1 --2- 172.21.15.89:0/2893408299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f420809dac0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f41f800bbc0 tx=0x7f41f800bed0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.055+0000 7f420c877700 1 -- 172.21.15.89:0/2893408299 shutdown_connections 2023-09-30T22:23:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.055+0000 7f420c877700 1 --2- 172.21.15.89:0/2893408299 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f420809dac0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.055+0000 7f420c877700 1 --2- 172.21.15.89:0/2893408299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e410 0x7f4208130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.055+0000 7f420c877700 1 -- 172.21.15.89:0/2893408299 >> 172.21.15.89:0/2893408299 conn(0x7f4208066f50 msgr2=0x7f420809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.055+0000 7f420c877700 1 -- 172.21.15.89:0/2893408299 shutdown_connections 2023-09-30T22:23:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.055+0000 7f420c877700 1 -- 172.21.15.89:0/2893408299 wait complete. 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.056+0000 7f420c877700 1 Processor -- start 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.056+0000 7f420c877700 1 -- start start 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.056+0000 7f420c877700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f42081222a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.056+0000 7f420c877700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e000 0x7f42081227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420c877700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f420809e410 0x7f4208122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420c877700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4208125820 con 0x7f420809e410 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420c877700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4208125dc0 con 0x7f420809d6f0 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420c877700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42081246d0 con 0x7f420809e000 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f42081222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f42081222a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40016/0 (socket says 172.21.15.89:40016) 2023-09-30T22:23:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 -- 172.21.15.89:0/3690323482 learned_addr learned my addr 172.21.15.89:0/3690323482 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f4205d9b700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e000 0x7f42081227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f4206d9d700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f420809e410 0x7f4208122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 -- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e000 msgr2=0x7f42081227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e000 0x7f42081227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 -- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f420809e410 msgr2=0x7f4208122d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f420809e410 0x7f4208122d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 -- 172.21.15.89:0/3690323482 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f800b7f0 con 0x7f420809d6f0 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f4205d9b700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e000 0x7f42081227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420659c700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f42081222a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000c00 tx=0x7f41f8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f41f6ffd700 1 -- 172.21.15.89:0/3690323482 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8003b70 con 0x7f420809d6f0 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f41f6ffd700 1 -- 172.21.15.89:0/3690323482 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f8003cd0 con 0x7f420809d6f0 2023-09-30T22:23:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f41f6ffd700 1 -- 172.21.15.89:0/3690323482 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8010690 con 0x7f420809d6f0 2023-09-30T22:23:35.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.057+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42080a1d00 con 0x7f420809d6f0 2023-09-30T22:23:35.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.058+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4208125ae0 con 0x7f420809d6f0 2023-09-30T22:23:35.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.059+0000 7f41f6ffd700 1 -- 172.21.15.89:0/3690323482 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f41f8019400 con 0x7f420809d6f0 2023-09-30T22:23:35.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.059+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4208074ea0 con 0x7f420809d6f0 2023-09-30T22:23:35.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.059+0000 7f41f6ffd700 1 -- 172.21.15.89:0/3690323482 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f41f8021760 con 0x7f420809d6f0 2023-09-30T22:23:35.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.068+0000 7f41f6ffd700 1 -- 172.21.15.89:0/3690323482 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41f8046060 con 0x7f420809d6f0 2023-09-30T22:23:35.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.087+0000 7f0e79817700 1 Processor -- start 2023-09-30T22:23:35.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.087+0000 7f0e79817700 1 -- start start 2023-09-30T22:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e79817700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e74131810 0x7f0e74131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e79817700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e74099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e79817700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e74138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e79817700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e741313c0 con 0x7f0e74131810 2023-09-30T22:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e79817700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7406e8c0 con 0x7f0e74099a40 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e79817700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e74071d20 con 0x7f0e7409a3b0 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e727fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e74099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e737fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e74138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e72ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e74131810 0x7f0e74131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e727fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e74099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40028/0 (socket says 172.21.15.89:40028) 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e737fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e74138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53792/0 (socket says 172.21.15.89:53792) 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e72ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e74131810 0x7f0e74131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41488/0 (socket says 172.21.15.89:41488) 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e727fc700 1 -- 172.21.15.89:0/463295046 learned_addr learned my addr 172.21.15.89:0/463295046 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e72ffd700 1 -- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 msgr2=0x7f0e74138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e72ffd700 1 --2- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e74138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e72ffd700 1 -- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 msgr2=0x7f0e74099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e72ffd700 1 --2- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e74099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e72ffd700 1 -- 172.21.15.89:0/463295046 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740721a0 con 0x7f0e74131810 2023-09-30T22:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e737fe700 1 --2- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e74138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.088+0000 7f0e72ffd700 1 --2- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 0x7f0e74131be0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009840 tx=0x7f0e64009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0e75a88064f65e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.089+0000 7f0e71ffb700 1 -- 172.21.15.89:0/463295046 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64011830 con 0x7f0e74131810 2023-09-30T22:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.089+0000 7f0e71ffb700 1 -- 172.21.15.89:0/463295046 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e64011990 con 0x7f0e74131810 2023-09-30T22:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.089+0000 7f0e71ffb700 1 -- 172.21.15.89:0/463295046 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64011d20 con 0x7f0e74131810 2023-09-30T22:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 -- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 msgr2=0x7f0e74131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 --2- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 0x7f0e74131be0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009840 tx=0x7f0e64009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 -- 172.21.15.89:0/463295046 shutdown_connections 2023-09-30T22:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 --2- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e74138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 --2- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 0x7f0e74131be0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 --2- 172.21.15.89:0/463295046 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e74099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 -- 172.21.15.89:0/463295046 >> 172.21.15.89:0/463295046 conn(0x7f0e740a2aa0 msgr2=0x7f0e740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 -- 172.21.15.89:0/463295046 shutdown_connections 2023-09-30T22:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.090+0000 7f0e79817700 1 -- 172.21.15.89:0/463295046 wait complete. 2023-09-30T22:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.091+0000 7f0e79817700 1 Processor -- start 2023-09-30T22:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.091+0000 7f0e79817700 1 -- start start 2023-09-30T22:23:35.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e79817700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e7411b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e79817700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e7411bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e79817700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 0x7f0e7411c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e79817700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7411d2b0 con 0x7f0e74131810 2023-09-30T22:23:35.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e79817700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7406eb00 con 0x7f0e74099a40 2023-09-30T22:23:35.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e79817700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7411eee0 con 0x7f0e7409a3b0 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e72ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e7411b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e72ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e7411b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40036/0 (socket says 172.21.15.89:40036) 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e72ffd700 1 -- 172.21.15.89:0/1607579253 learned_addr learned my addr 172.21.15.89:0/1607579253 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e737fe700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 0x7f0e7411c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.092+0000 7f0e727fc700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e7411bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e72ffd700 1 -- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 msgr2=0x7f0e7411bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e72ffd700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e7411bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e72ffd700 1 -- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 msgr2=0x7f0e7411c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e72ffd700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 0x7f0e7411c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e72ffd700 1 -- 172.21.15.89:0/1607579253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c009a70 con 0x7f0e74099a40 2023-09-30T22:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e72ffd700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e7411b6e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e640090b0 tx=0x7f0e64004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e5bfff700 1 -- 172.21.15.89:0/1607579253 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64004450 con 0x7f0e74099a40 2023-09-30T22:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e5bfff700 1 -- 172.21.15.89:0/1607579253 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6401f070 con 0x7f0e74099a40 2023-09-30T22:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e640094a0 con 0x7f0e74099a40 2023-09-30T22:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e5bfff700 1 -- 172.21.15.89:0/1607579253 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e640038e0 con 0x7f0e74099a40 2023-09-30T22:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.093+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7411e500 con 0x7f0e74099a40 2023-09-30T22:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.094+0000 7f0e5bfff700 1 -- 172.21.15.89:0/1607579253 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e64003a40 con 0x7f0e74099a40 2023-09-30T22:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.094+0000 7f0e5bfff700 1 -- 172.21.15.89:0/1607579253 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0e64021070 con 0x7f0e74099a40 2023-09-30T22:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.094+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e74074ea0 con 0x7f0e74099a40 2023-09-30T22:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.099+0000 7f0e5bfff700 1 -- 172.21.15.89:0/1607579253 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e640253e0 con 0x7f0e74099a40 2023-09-30T22:23:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.127+0000 7f7df6b79700 1 Processor -- start 2023-09-30T22:23:35.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.127+0000 7f7df6b79700 1 -- start start 2023-09-30T22:23:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7df6b79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7df6b79700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df0139170 0x7f7df013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7df6b79700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df009b120 con 0x7f7df009d6f0 2023-09-30T22:23:35.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7df6b79700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df006b460 con 0x7f7df0139170 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7df6b79700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7df009a7e0 con 0x7f7df009e1d0 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7deffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7deffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53816/0 (socket says 172.21.15.89:53816) 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7deffff700 1 -- 172.21.15.89:0/14952777 learned_addr learned my addr 172.21.15.89:0/14952777 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7df5116700 1 --2- 172.21.15.89:0/14952777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df0139170 0x7f7df013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7deffff700 1 -- 172.21.15.89:0/14952777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df0139170 msgr2=0x7f7df013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7deffff700 1 --2- 172.21.15.89:0/14952777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df0139170 0x7f7df013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7deffff700 1 -- 172.21.15.89:0/14952777 >> v1:172.21.15.89:6789/0 conn(0x7f7df009d6f0 legacy=0x7f7df009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7deffff700 1 -- 172.21.15.89:0/14952777 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df009a500 con 0x7f7df009e1d0 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7def7fe700 1 -- 172.21.15.89:0/14952777 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864928408 0 0) 0x7f7df009b120 con 0x7f7df009d6f0 2023-09-30T22:23:35.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7deffff700 1 --2- 172.21.15.89:0/14952777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0138c30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009950 tx=0x7f7ddc009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f229d3df679f893 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.128+0000 7f7def7fe700 1 -- 172.21.15.89:0/14952777 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc011830 con 0x7f7df009e1d0 2023-09-30T22:23:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7def7fe700 1 -- 172.21.15.89:0/14952777 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc011990 con 0x7f7df009e1d0 2023-09-30T22:23:35.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7def7fe700 1 -- 172.21.15.89:0/14952777 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc011d40 con 0x7f7df009e1d0 2023-09-30T22:23:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7df6b79700 1 -- 172.21.15.89:0/14952777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 msgr2=0x7f7df0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7df6b79700 1 --2- 172.21.15.89:0/14952777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0138c30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009950 tx=0x7f7ddc009c60 comp rx=0 tx=0).stop 2023-09-30T22:23:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7df6b79700 1 -- 172.21.15.89:0/14952777 shutdown_connections 2023-09-30T22:23:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7df6b79700 1 --2- 172.21.15.89:0/14952777 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df0139170 0x7f7df013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7df6b79700 1 --2- 172.21.15.89:0/14952777 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0138c30 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7df6b79700 1 -- 172.21.15.89:0/14952777 >> 172.21.15.89:0/14952777 conn(0x7f7df0066f50 msgr2=0x7f7df009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7df6b79700 1 -- 172.21.15.89:0/14952777 shutdown_connections 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.129+0000 7f7df6b79700 1 -- 172.21.15.89:0/14952777 wait complete. 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.130+0000 7f7df6b79700 1 Processor -- start 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.130+0000 7f7df6b79700 1 -- start start 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df6b79700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df009d6f0 0x7f7df0121870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df6b79700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0121db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df6b79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7df0139170 0x7f7df01222f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df6b79700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0123d10 con 0x7f7df0139170 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df6b79700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0125350 con 0x7f7df009d6f0 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df6b79700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0071f00 con 0x7f7df009e1d0 2023-09-30T22:23:35.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7deffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0121db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7deffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0121db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53826/0 (socket says 172.21.15.89:53826) 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7deffff700 1 -- 172.21.15.89:0/273138790 learned_addr learned my addr 172.21.15.89:0/273138790 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df5116700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7df0139170 0x7f7df01222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df4915700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df009d6f0 0x7f7df0121870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df5116700 1 -- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 msgr2=0x7f7df0121db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df5116700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0121db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df5116700 1 -- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df009d6f0 msgr2=0x7f7df0121870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df5116700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df009d6f0 0x7f7df0121870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df5116700 1 -- 172.21.15.89:0/273138790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc009550 con 0x7f7df0139170 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.131+0000 7f7df4915700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df009d6f0 0x7f7df0121870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:35.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.132+0000 7f7df5116700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7df0139170 0x7f7df01222f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f7de800efc0 tx=0x7f7de800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.132+0000 7f7ded7fa700 1 -- 172.21.15.89:0/273138790 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de800cc90 con 0x7f7df0139170 2023-09-30T22:23:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.132+0000 7f7ded7fa700 1 -- 172.21.15.89:0/273138790 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de800f040 con 0x7f7df0139170 2023-09-30T22:23:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.132+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df0124ae0 con 0x7f7df0139170 2023-09-30T22:23:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.132+0000 7f7ded7fa700 1 -- 172.21.15.89:0/273138790 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de80146c0 con 0x7f7df0139170 2023-09-30T22:23:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.132+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df0123440 con 0x7f7df0139170 2023-09-30T22:23:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.133+0000 7f7ded7fa700 1 -- 172.21.15.89:0/273138790 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7de8014820 con 0x7f7df0139170 2023-09-30T22:23:35.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.133+0000 7f7ded7fa700 1 -- 172.21.15.89:0/273138790 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7de8040140 con 0x7f7df0139170 2023-09-30T22:23:35.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.133+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df0125180 con 0x7f7df0139170 2023-09-30T22:23:35.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.138+0000 7f7ded7fa700 1 -- 172.21.15.89:0/273138790 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7de801d400 con 0x7f7df0139170 2023-09-30T22:23:35.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.266+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f420806c360 con 0x7f420809d6f0 2023-09-30T22:23:35.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.266+0000 7f41f6ffd700 1 -- 172.21.15.89:0/3690323482 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f41f8003e40 con 0x7f420809d6f0 2023-09-30T22:23:35.267 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:35.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.268+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 msgr2=0x7f42081222a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.268+0000 7f420c877700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f42081222a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000c00 tx=0x7f41f8004750 comp rx=0 tx=0).stop 2023-09-30T22:23:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.269+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 shutdown_connections 2023-09-30T22:23:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.269+0000 7f420c877700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f420809e410 0x7f4208122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.269+0000 7f420c877700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f420809d6f0 0x7f42081222a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.269+0000 7f420c877700 1 --2- 172.21.15.89:0/3690323482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f420809e000 0x7f42081227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.269+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 >> 172.21.15.89:0/3690323482 conn(0x7f4208066f50 msgr2=0x7f4208130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.269+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 shutdown_connections 2023-09-30T22:23:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.269+0000 7f420c877700 1 -- 172.21.15.89:0/3690323482 wait complete. 2023-09-30T22:23:35.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:35.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.292+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fad1804f2a0 con 0x7fad18130a20 2023-09-30T22:23:35.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.293+0000 7fad14ff9700 1 -- 172.21.15.89:0/1649280649 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad10017070 con 0x7fad18130a20 2023-09-30T22:23:35.294 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad18130a20 msgr2=0x7fad181196e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad18130a20 0x7fad181196e0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fad1000cbc0 tx=0x7fad1000ced0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 shutdown_connections 2023-09-30T22:23:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad18130a20 0x7fad181196e0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad18130200 0x7fad18118c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 --2- 172.21.15.89:0/1649280649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad181305d0 0x7fad181191a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 >> 172.21.15.89:0/1649280649 conn(0x7fad180a27f0 msgr2=0x7fad18070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:35.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 shutdown_connections 2023-09-30T22:23:35.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.295+0000 7fad1e274700 1 -- 172.21.15.89:0/1649280649 wait complete. 2023-09-30T22:23:35.308 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:35.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.330+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7df0074ea0 con 0x7f7df0139170 2023-09-30T22:23:35.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.330+0000 7f7ded7fa700 1 -- 172.21.15.89:0/273138790 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7de802b090 con 0x7f7df0139170 2023-09-30T22:23:35.331 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:35.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7df0139170 msgr2=0x7f7df01222f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7df0139170 0x7f7df01222f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f7de800efc0 tx=0x7f7de800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 shutdown_connections 2023-09-30T22:23:35.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7df0139170 0x7f7df01222f0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df009d6f0 0x7f7df0121870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 --2- 172.21.15.89:0/273138790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df009e1d0 0x7f7df0121db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 >> 172.21.15.89:0/273138790 conn(0x7f7df0066f50 msgr2=0x7f7df0070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:35.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 shutdown_connections 2023-09-30T22:23:35.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.332+0000 7f7df6b79700 1 -- 172.21.15.89:0/273138790 wait complete. 2023-09-30T22:23:35.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:35.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.347+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0e7406b9e0 con 0x7f0e74099a40 2023-09-30T22:23:35.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.347+0000 7f0e5bfff700 1 -- 172.21.15.89:0/1607579253 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e64010b70 con 0x7f0e74099a40 2023-09-30T22:23:35.348 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:35.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.349+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 msgr2=0x7f0e7411b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:35.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.349+0000 7f0e79817700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e7411b6e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e640090b0 tx=0x7f0e64004750 comp rx=0 tx=0).stop 2023-09-30T22:23:35.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.349+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 shutdown_connections 2023-09-30T22:23:35.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.349+0000 7f0e79817700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e74131810 0x7f0e7411c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.349+0000 7f0e79817700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e74099a40 0x7f0e7411b6e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.349+0000 7f0e79817700 1 --2- 172.21.15.89:0/1607579253 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e7409a3b0 0x7f0e7411bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:35.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.349+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 >> 172.21.15.89:0/1607579253 conn(0x7f0e740a2aa0 msgr2=0x7f0e741b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:35.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.349+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 shutdown_connections 2023-09-30T22:23:35.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:35.350+0000 7f0e79817700 1 -- 172.21.15.89:0/1607579253 wait complete. 2023-09-30T22:23:35.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:36.287 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:36.308 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:36.347 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:36.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:36.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.430+0000 7fd0362c2700 1 Processor -- start 2023-09-30T22:23:36.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.430+0000 7fd0362c2700 1 -- start start 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd0362c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009e1d0 0x7fd030138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd0362c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 0x7fd03013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd0362c2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03009b120 con 0x7fd03009d6f0 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd0362c2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03006b460 con 0x7fd030139170 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd0362c2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd03009a7e0 con 0x7fd03009e1d0 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd02f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009e1d0 0x7fd030138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd02f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009e1d0 0x7fd030138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53838/0 (socket says 172.21.15.89:53838) 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd02f7fe700 1 -- 172.21.15.89:0/2303420043 learned_addr learned my addr 172.21.15.89:0/2303420043 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:36.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd03485f700 1 --2- 172.21.15.89:0/2303420043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 0x7fd03013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd02f7fe700 1 -- 172.21.15.89:0/2303420043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 msgr2=0x7fd03013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:36.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.431+0000 7fd02f7fe700 1 --2- 172.21.15.89:0/2303420043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 0x7fd03013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.432+0000 7fd02f7fe700 1 -- 172.21.15.89:0/2303420043 >> v1:172.21.15.89:6789/0 conn(0x7fd03009d6f0 legacy=0x7fd03009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.432+0000 7fd02f7fe700 1 -- 172.21.15.89:0/2303420043 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03009a500 con 0x7fd03009e1d0 2023-09-30T22:23:36.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.432+0000 7fd02f7fe700 1 --2- 172.21.15.89:0/2303420043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009e1d0 0x7fd030138c30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd024009950 tx=0x7fd024009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465d1bca7c23f6cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:36.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.432+0000 7fd02effd700 1 -- 172.21.15.89:0/2303420043 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024011830 con 0x7fd03009e1d0 2023-09-30T22:23:36.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.432+0000 7fd02effd700 1 -- 172.21.15.89:0/2303420043 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd024011990 con 0x7fd03009e1d0 2023-09-30T22:23:36.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.432+0000 7fd02effd700 1 -- 172.21.15.89:0/2303420043 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024011d40 con 0x7fd03009e1d0 2023-09-30T22:23:36.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.433+0000 7fd0362c2700 1 -- 172.21.15.89:0/2303420043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009e1d0 msgr2=0x7fd030138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.433+0000 7fd0362c2700 1 --2- 172.21.15.89:0/2303420043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009e1d0 0x7fd030138c30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd024009950 tx=0x7fd024009c60 comp rx=0 tx=0).stop 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.433+0000 7fd0362c2700 1 -- 172.21.15.89:0/2303420043 shutdown_connections 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.433+0000 7fd0362c2700 1 --2- 172.21.15.89:0/2303420043 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 0x7fd03013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.433+0000 7fd0362c2700 1 --2- 172.21.15.89:0/2303420043 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009e1d0 0x7fd030138c30 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.433+0000 7fd0362c2700 1 -- 172.21.15.89:0/2303420043 >> 172.21.15.89:0/2303420043 conn(0x7fd030066f50 msgr2=0x7fd03009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.433+0000 7fd0362c2700 1 -- 172.21.15.89:0/2303420043 shutdown_connections 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.433+0000 7fd0362c2700 1 -- 172.21.15.89:0/2303420043 wait complete. 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.434+0000 7fd0362c2700 1 Processor -- start 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.434+0000 7fd0362c2700 1 -- start start 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.434+0000 7fd0362c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009d6f0 0x7fd03011e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.434+0000 7fd0362c2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd03009e1d0 0x7fd03011ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.434+0000 7fd0362c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 0x7fd03011f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.434+0000 7fd0362c2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd030121a80 con 0x7fd03009e1d0 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.434+0000 7fd0362c2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd030120400 con 0x7fd030139170 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.434+0000 7fd0362c2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300a2020 con 0x7fd03009d6f0 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd03009e1d0 0x7fd03011ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02ffff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009d6f0 0x7fd03011e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd03009e1d0 0x7fd03011ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41526/0 (socket says 172.21.15.89:41526) 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02f7fe700 1 -- 172.21.15.89:0/4117631271 learned_addr learned my addr 172.21.15.89:0/4117631271 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd03485f700 1 --2- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 0x7fd03011f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02ffff700 1 -- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 msgr2=0x7fd03011f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02ffff700 1 --2- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 0x7fd03011f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02ffff700 1 -- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd03009e1d0 msgr2=0x7fd03011ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02ffff700 1 --2- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd03009e1d0 0x7fd03011ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02ffff700 1 -- 172.21.15.89:0/4117631271 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd024009550 con 0x7fd03009d6f0 2023-09-30T22:23:36.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02ffff700 1 --2- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009d6f0 0x7fd03011e7f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd020000f60 tx=0x7fd02000c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:36.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02cff9700 1 -- 172.21.15.89:0/4117631271 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0200052d0 con 0x7fd03009d6f0 2023-09-30T22:23:36.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02cff9700 1 -- 172.21.15.89:0/4117631271 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd020005430 con 0x7fd03009d6f0 2023-09-30T22:23:36.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03009c710 con 0x7fd03009d6f0 2023-09-30T22:23:36.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd02cff9700 1 -- 172.21.15.89:0/4117631271 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0200038c0 con 0x7fd03009d6f0 2023-09-30T22:23:36.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.435+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd030122810 con 0x7fd03009d6f0 2023-09-30T22:23:36.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.437+0000 7fd02cff9700 1 -- 172.21.15.89:0/4117631271 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd020003a20 con 0x7fd03009d6f0 2023-09-30T22:23:36.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.437+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd030121e80 con 0x7fd03009d6f0 2023-09-30T22:23:36.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.437+0000 7fd02cff9700 1 -- 172.21.15.89:0/4117631271 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd02003e9f0 con 0x7fd03009d6f0 2023-09-30T22:23:36.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.445+0000 7fd02cff9700 1 -- 172.21.15.89:0/4117631271 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd020040e60 con 0x7fd03009d6f0 2023-09-30T22:23:36.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.477+0000 7f9307e33700 1 Processor -- start 2023-09-30T22:23:36.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.477+0000 7f9307e33700 1 -- start start 2023-09-30T22:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f9307e33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009d6f0 0x7f930009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f9307e33700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009e410 0x7f9300134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f9307e33700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930009b120 con 0x7f930009e000 2023-09-30T22:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f9307e33700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930006b460 con 0x7f930009d6f0 2023-09-30T22:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f9307e33700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f930009a7e0 con 0x7f930009e410 2023-09-30T22:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f9305bcf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009d6f0 0x7f930009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f93053ce700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f930009e000 0x7f93001339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59058/0 (socket says 172.21.15.89:59058) 2023-09-30T22:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f9305bcf700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009d6f0 0x7f930009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40084/0 (socket says 172.21.15.89:40084) 2023-09-30T22:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f93053ce700 1 -- 172.21.15.89:0/534644257 learned_addr learned my addr 172.21.15.89:0/534644257 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.478+0000 7f93063d0700 1 --2- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009e410 0x7f9300134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9305bcf700 1 -- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009e410 msgr2=0x7f9300134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9305bcf700 1 --2- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009e410 0x7f9300134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9305bcf700 1 -- 172.21.15.89:0/534644257 >> v1:172.21.15.89:6789/0 conn(0x7f930009e000 legacy=0x7f93001339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9304bcd700 1 -- 172.21.15.89:0/534644257 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661827769 0 0) 0x7f930009b120 con 0x7f930009e000 2023-09-30T22:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9305bcf700 1 -- 172.21.15.89:0/534644257 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930009a500 con 0x7f930009d6f0 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f93063d0700 1 --2- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009e410 0x7f9300134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9305bcf700 1 --2- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009d6f0 0x7f930009dac0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009870 tx=0x7f92fc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e5b0dbb1e59adf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9304bcd700 1 -- 172.21.15.89:0/534644257 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc011830 con 0x7f930009d6f0 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9304bcd700 1 -- 172.21.15.89:0/534644257 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc011990 con 0x7f930009d6f0 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.479+0000 7f9304bcd700 1 -- 172.21.15.89:0/534644257 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc011d20 con 0x7f930009d6f0 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.480+0000 7f9307e33700 1 -- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009d6f0 msgr2=0x7f930009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.480+0000 7f9307e33700 1 --2- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009d6f0 0x7f930009dac0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009870 tx=0x7f92fc009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.480+0000 7f9307e33700 1 -- 172.21.15.89:0/534644257 shutdown_connections 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.480+0000 7f9307e33700 1 --2- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009d6f0 0x7f930009dac0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.480+0000 7f9307e33700 1 --2- 172.21.15.89:0/534644257 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009e410 0x7f9300134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.480+0000 7f9307e33700 1 -- 172.21.15.89:0/534644257 >> 172.21.15.89:0/534644257 conn(0x7f9300066f50 msgr2=0x7f930009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.480+0000 7f9307e33700 1 -- 172.21.15.89:0/534644257 shutdown_connections 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.480+0000 7f9307e33700 1 -- 172.21.15.89:0/534644257 wait complete. 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.481+0000 7f9307e33700 1 Processor -- start 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.481+0000 7f9307e33700 1 -- start start 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.481+0000 7f9307e33700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009d6f0 0x7f93001c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.481+0000 7f9307e33700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009e000 0x7f93001c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.481+0000 7f9307e33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f930009e410 0x7f93001c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.481+0000 7f9307e33700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93001c48c0 con 0x7f930009e410 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.481+0000 7f9307e33700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93000a1f20 con 0x7f930009e000 2023-09-30T22:23:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.481+0000 7f9307e33700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f930009c710 con 0x7f930009d6f0 2023-09-30T22:23:36.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f9305bcf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009d6f0 0x7f93001c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f930009e410 0x7f93001c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f930009e410 0x7f93001c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41534/0 (socket says 172.21.15.89:41534) 2023-09-30T22:23:36.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 -- 172.21.15.89:0/2678642170 learned_addr learned my addr 172.21.15.89:0/2678642170 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:36.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f9305bcf700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009d6f0 0x7f93001c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53870/0 (socket says 172.21.15.89:53870) 2023-09-30T22:23:36.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93053ce700 1 --2- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009e000 0x7f93001c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 -- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009d6f0 msgr2=0x7f93001c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 --2- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009d6f0 0x7f93001c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 -- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009e000 msgr2=0x7f93001c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 --2- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009e000 0x7f93001c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 -- 172.21.15.89:0/2678642170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f8009a70 con 0x7f930009e410 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f93063d0700 1 --2- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f930009e410 0x7f93001c7f70 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f92f800efc0 tx=0x7f92f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f92f67fc700 1 -- 172.21.15.89:0/2678642170 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f800cb40 con 0x7f930009e410 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f92f67fc700 1 -- 172.21.15.89:0/2678642170 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f800f040 con 0x7f930009e410 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc0094a0 con 0x7f930009e410 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.482+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93001a9ae0 con 0x7f930009e410 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.483+0000 7f92f67fc700 1 -- 172.21.15.89:0/2678642170 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8003a20 con 0x7f930009e410 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.483+0000 7f92f67fc700 1 -- 172.21.15.89:0/2678642170 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f92f8014480 con 0x7f930009e410 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.484+0000 7f92f67fc700 1 -- 172.21.15.89:0/2678642170 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f92f80401d0 con 0x7f930009e410 2023-09-30T22:23:36.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.484+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e80052e0 con 0x7f930009e410 2023-09-30T22:23:36.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.493+0000 7f92f67fc700 1 -- 172.21.15.89:0/2678642170 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92f8012930 con 0x7f930009e410 2023-09-30T22:23:36.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.520+0000 7f0479de5700 1 Processor -- start 2023-09-30T22:23:36.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.520+0000 7f0479de5700 1 -- start start 2023-09-30T22:23:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0479de5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f047409d6f0 0x7f047409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0479de5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0479de5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047409f130 con 0x7f047409e000 2023-09-30T22:23:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0479de5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047409f520 con 0x7f047409d6f0 2023-09-30T22:23:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0479de5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f047409e930 con 0x7f047409e410 2023-09-30T22:23:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0473fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0473fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53880/0 (socket says 172.21.15.89:53880) 2023-09-30T22:23:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0473fff700 1 -- 172.21.15.89:0/1727191421 learned_addr learned my addr 172.21.15.89:0/1727191421 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f04737fe700 1 --2- 172.21.15.89:0/1727191421 >> v2:172.21.15.89:3300/0 conn(0x7f047409d6f0 0x7f047409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0473fff700 1 -- 172.21.15.89:0/1727191421 >> v2:172.21.15.89:3300/0 conn(0x7f047409d6f0 msgr2=0x7f047409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0473fff700 1 --2- 172.21.15.89:0/1727191421 >> v2:172.21.15.89:3300/0 conn(0x7f047409d6f0 0x7f047409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0473fff700 1 -- 172.21.15.89:0/1727191421 >> v1:172.21.15.89:6789/0 conn(0x7f047409e000 legacy=0x7f0474130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f0473fff700 1 -- 172.21.15.89:0/1727191421 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047409b2d0 con 0x7f047409e410 2023-09-30T22:23:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.521+0000 7f04737fe700 1 --2- 172.21.15.89:0/1727191421 >> v2:172.21.15.89:3300/0 conn(0x7f047409d6f0 0x7f047409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f0473fff700 1 --2- 172.21.15.89:0/1727191421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474130d70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f046c00d150 tx=0x7f046c00d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5acc6fccdc4f9bfe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f04727fc700 1 -- 172.21.15.89:0/1727191421 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c010070 con 0x7f047409e410 2023-09-30T22:23:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f04727fc700 1 -- 172.21.15.89:0/1727191421 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c00dd10 con 0x7f047409e410 2023-09-30T22:23:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f04727fc700 1 -- 172.21.15.89:0/1727191421 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00b310 con 0x7f047409e410 2023-09-30T22:23:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7ff6dc314700 1 Processor -- start 2023-09-30T22:23:36.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f0479de5700 1 -- 172.21.15.89:0/1727191421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 msgr2=0x7f0474130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f0479de5700 1 --2- 172.21.15.89:0/1727191421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474130d70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f046c00d150 tx=0x7f046c00d460 comp rx=0 tx=0).stop 2023-09-30T22:23:36.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7ff6dc314700 1 -- start start 2023-09-30T22:23:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f0479de5700 1 -- 172.21.15.89:0/1727191421 shutdown_connections 2023-09-30T22:23:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f0479de5700 1 --2- 172.21.15.89:0/1727191421 >> v2:172.21.15.89:3300/0 conn(0x7f047409d6f0 0x7f047409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f0479de5700 1 --2- 172.21.15.89:0/1727191421 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474130d70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.522+0000 7f0479de5700 1 -- 172.21.15.89:0/1727191421 >> 172.21.15.89:0/1727191421 conn(0x7f0474066f50 msgr2=0x7f047409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7f0479de5700 1 -- 172.21.15.89:0/1727191421 shutdown_connections 2023-09-30T22:23:36.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7f0479de5700 1 -- 172.21.15.89:0/1727191421 wait complete. 2023-09-30T22:23:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7ff6dc314700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6d412cf60 0x7ff6d406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7ff6dc314700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d412f8b0 0x7ff6d412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7ff6dc314700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d4130260 0x7ff6d4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7ff6dc314700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d412d330 con 0x7ff6d412cf60 2023-09-30T22:23:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7ff6dc314700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d412cd90 con 0x7ff6d4130260 2023-09-30T22:23:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7ff6dc314700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41306d0 con 0x7ff6d412f8b0 2023-09-30T22:23:36.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6d412cf60 0x7ff6d406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6d412cf60 0x7ff6d406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41554/0 (socket says 172.21.15.89:41554) 2023-09-30T22:23:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 -- 172.21.15.89:0/2225301945 learned_addr learned my addr 172.21.15.89:0/2225301945 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.523+0000 7f0479de5700 1 Processor -- start 2023-09-30T22:23:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da8b1700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d4130260 0x7ff6d4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 -- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d412f8b0 msgr2=0x7ff6d412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:36.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6d98af700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d412f8b0 0x7ff6d412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7f0479de5700 1 -- start start 2023-09-30T22:23:36.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d412f8b0 0x7ff6d412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 -- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d4130260 msgr2=0x7ff6d4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:36.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d4130260 0x7ff6d4136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 -- 172.21.15.89:0/2225301945 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4071780 con 0x7ff6d412cf60 2023-09-30T22:23:36.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da8b1700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d4130260 0x7ff6d4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:36.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6d98af700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d412f8b0 0x7ff6d412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:36.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6da0b0700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d412cf60 0x7ff6d406f5d0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009920 tx=0x7ff6d0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f15e7f5d208a2b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:36.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7ff6d90ae700 1 -- 172.21.15.89:0/2225301945 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000ba20 con 0x7ff6d412cf60 2023-09-30T22:23:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.524+0000 7f0479de5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f047409d6f0 0x7f0474122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0479de5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f047409e000 0x7f0474122bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0479de5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474123100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0479de5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0474125c00 con 0x7f047409d6f0 2023-09-30T22:23:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0479de5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04741261a0 con 0x7f047409e000 2023-09-30T22:23:36.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0479de5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0474124ab0 con 0x7f047409e410 2023-09-30T22:23:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7ff6d90ae700 1 -- 172.21.15.89:0/2225301945 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d000bb80 con 0x7ff6d412cf60 2023-09-30T22:23:36.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7ff6d90ae700 1 -- 172.21.15.89:0/2225301945 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0009e70 con 0x7ff6d412cf60 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f04737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f047409d6f0 0x7f0474122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0472ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f047409e000 0x7f0474122bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0472ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f047409e000 0x7f0474122bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40098/0 (socket says 172.21.15.89:40098) 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f04737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f047409d6f0 0x7f0474122680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41566/0 (socket says 172.21.15.89:41566) 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0472ffd700 1 -- 172.21.15.89:0/1242808433 learned_addr learned my addr 172.21.15.89:0/1242808433 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f04737fe700 1 -- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 msgr2=0x7f0474123100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f04737fe700 1 --2- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474123100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f04737fe700 1 -- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f047409e000 msgr2=0x7f0474122bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f04737fe700 1 --2- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f047409e000 0x7f0474122bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f04737fe700 1 -- 172.21.15.89:0/1242808433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04640094a0 con 0x7f047409d6f0 2023-09-30T22:23:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f04737fe700 1 --2- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f047409d6f0 0x7f0474122680 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f046400e9b0 tx=0x7f046400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f045bfff700 1 -- 172.21.15.89:0/1242808433 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046400c850 con 0x7f047409d6f0 2023-09-30T22:23:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046c00bd40 con 0x7f047409d6f0 2023-09-30T22:23:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f045bfff700 1 -- 172.21.15.89:0/1242808433 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046400c9b0 con 0x7f047409d6f0 2023-09-30T22:23:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f045bfff700 1 -- 172.21.15.89:0/1242808433 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046400cd60 con 0x7f047409d6f0 2023-09-30T22:23:36.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.525+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047409c700 con 0x7f047409d6f0 2023-09-30T22:23:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 -- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d412cf60 msgr2=0x7ff6d406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d412cf60 0x7ff6d406f5d0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009920 tx=0x7ff6d0009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 -- 172.21.15.89:0/2225301945 shutdown_connections 2023-09-30T22:23:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d4130260 0x7ff6d4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d412cf60 0x7ff6d406f5d0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 --2- 172.21.15.89:0/2225301945 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d412f8b0 0x7ff6d412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 -- 172.21.15.89:0/2225301945 >> 172.21.15.89:0/2225301945 conn(0x7ff6d40a2aa0 msgr2=0x7ff6d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:36.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 -- 172.21.15.89:0/2225301945 shutdown_connections 2023-09-30T22:23:36.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.526+0000 7ff6dc314700 1 -- 172.21.15.89:0/2225301945 wait complete. 2023-09-30T22:23:36.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.527+0000 7ff6dc314700 1 Processor -- start 2023-09-30T22:23:36.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.527+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0474074ea0 con 0x7f047409d6f0 2023-09-30T22:23:36.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.527+0000 7f045bfff700 1 -- 172.21.15.89:0/1242808433 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0464013470 con 0x7f047409d6f0 2023-09-30T22:23:36.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.527+0000 7f045bfff700 1 -- 172.21.15.89:0/1242808433 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f046403fb90 con 0x7f047409d6f0 2023-09-30T22:23:36.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.527+0000 7ff6dc314700 1 -- start start 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6dc314700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d412cf60 0x7ff6d41309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6dc314700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d412f8b0 0x7ff6d4130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6dc314700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d4130260 0x7ff6d4131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6dc314700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4133030 con 0x7ff6d412f8b0 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6dc314700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d409e920 con 0x7ff6d412cf60 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6dc314700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4129fa0 con 0x7ff6d4130260 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6da8b1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d4130260 0x7ff6d4131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6da8b1700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d4130260 0x7ff6d4131460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53908/0 (socket says 172.21.15.89:53908) 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6da8b1700 1 -- 172.21.15.89:0/2249006414 learned_addr learned my addr 172.21.15.89:0/2249006414 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:36.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.528+0000 7ff6da0b0700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d412cf60 0x7ff6d41309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.529+0000 7ff6da0b0700 1 -- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d4130260 msgr2=0x7ff6d4131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.529+0000 7ff6da0b0700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d4130260 0x7ff6d4131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.529+0000 7ff6da0b0700 1 -- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d412f8b0 msgr2=0x7ff6d4130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.529+0000 7ff6da0b0700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d412f8b0 0x7ff6d4130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.529+0000 7ff6da0b0700 1 -- 172.21.15.89:0/2249006414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d0009550 con 0x7ff6d412cf60 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.529+0000 7ff6da8b1700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d4130260 0x7ff6d4131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.529+0000 7ff6da0b0700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d412cf60 0x7ff6d41309e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0016000 tx=0x7ff6d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.531+0000 7ff6caffd700 1 -- 172.21.15.89:0/2249006414 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0004510 con 0x7ff6d412cf60 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.531+0000 7ff6caffd700 1 -- 172.21.15.89:0/2249006414 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d00036a0 con 0x7ff6d412cf60 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.531+0000 7ff6caffd700 1 -- 172.21.15.89:0/2249006414 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0003a50 con 0x7ff6d412cf60 2023-09-30T22:23:36.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.531+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d4132d70 con 0x7ff6d412cf60 2023-09-30T22:23:36.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.531+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d409def0 con 0x7ff6d412cf60 2023-09-30T22:23:36.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.532+0000 7ff6caffd700 1 -- 172.21.15.89:0/2249006414 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff6d0021070 con 0x7ff6d412cf60 2023-09-30T22:23:36.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.532+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d40701b0 con 0x7ff6d412cf60 2023-09-30T22:23:36.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.532+0000 7ff6caffd700 1 -- 172.21.15.89:0/2249006414 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff6d0040ac0 con 0x7ff6d412cf60 2023-09-30T22:23:36.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.536+0000 7f045bfff700 1 -- 172.21.15.89:0/1242808433 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f046401b3b0 con 0x7f047409d6f0 2023-09-30T22:23:36.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.536+0000 7ff6caffd700 1 -- 172.21.15.89:0/2249006414 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6d0042e60 con 0x7ff6d412cf60 2023-09-30T22:23:36.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.651+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd030074ea0 con 0x7fd03009d6f0 2023-09-30T22:23:36.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.652+0000 7fd02cff9700 1 -- 172.21.15.89:0/4117631271 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd020017020 con 0x7fd03009d6f0 2023-09-30T22:23:36.653 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:36.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009d6f0 msgr2=0x7fd03011e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 --2- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009d6f0 0x7fd03011e7f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd020000f60 tx=0x7fd02000c730 comp rx=0 tx=0).stop 2023-09-30T22:23:36.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 shutdown_connections 2023-09-30T22:23:36.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 --2- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd030139170 0x7fd03011f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 --2- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd03009d6f0 0x7fd03011e7f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 --2- 172.21.15.89:0/4117631271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd03009e1d0 0x7fd03011ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 >> 172.21.15.89:0/4117631271 conn(0x7fd030066f50 msgr2=0x7fd030136c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:36.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 shutdown_connections 2023-09-30T22:23:36.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.654+0000 7fd0362c2700 1 -- 172.21.15.89:0/4117631271 wait complete. 2023-09-30T22:23:36.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:36.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.698+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f92e8005590 con 0x7f930009e410 2023-09-30T22:23:36.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.699+0000 7f92f67fc700 1 -- 172.21.15.89:0/2678642170 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92f802b090 con 0x7f930009e410 2023-09-30T22:23:36.700 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:36.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f930009e410 msgr2=0x7f93001c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 --2- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f930009e410 0x7f93001c7f70 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f92f800efc0 tx=0x7f92f800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 shutdown_connections 2023-09-30T22:23:36.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 --2- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f930009e410 0x7f93001c7f70 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 --2- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f930009d6f0 0x7f93001c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 --2- 172.21.15.89:0/2678642170 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f930009e000 0x7f93001c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 >> 172.21.15.89:0/2678642170 conn(0x7f9300066f50 msgr2=0x7f930006f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:36.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 shutdown_connections 2023-09-30T22:23:36.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.701+0000 7f9307e33700 1 -- 172.21.15.89:0/2678642170 wait complete. 2023-09-30T22:23:36.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:36.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.746+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0474125f20 con 0x7f047409d6f0 2023-09-30T22:23:36.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.746+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff6d4074ea0 con 0x7ff6d412cf60 2023-09-30T22:23:36.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.746+0000 7f045bfff700 1 -- 172.21.15.89:0/1242808433 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0464021030 con 0x7f047409d6f0 2023-09-30T22:23:36.748 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:36.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.746+0000 7ff6caffd700 1 -- 172.21.15.89:0/2249006414 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6d0029020 con 0x7ff6d412cf60 2023-09-30T22:23:36.748 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:36.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.748+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f047409d6f0 msgr2=0x7f0474122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.748+0000 7f0479de5700 1 --2- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f047409d6f0 0x7f0474122680 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f046400e9b0 tx=0x7f046400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.748+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d412cf60 msgr2=0x7ff6d41309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:36.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.748+0000 7ff6dc314700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d412cf60 0x7ff6d41309e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0016000 tx=0x7ff6d0004750 comp rx=0 tx=0).stop 2023-09-30T22:23:36.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.748+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 shutdown_connections 2023-09-30T22:23:36.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.748+0000 7f0479de5700 1 --2- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f047409e410 0x7f0474123100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.748+0000 7f0479de5700 1 --2- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f047409d6f0 0x7f0474122680 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.748+0000 7f0479de5700 1 --2- 172.21.15.89:0/1242808433 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f047409e000 0x7f0474122bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 >> 172.21.15.89:0/1242808433 conn(0x7f0474066f50 msgr2=0x7f04740a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:36.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 shutdown_connections 2023-09-30T22:23:36.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7ff6dc314700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6d4130260 0x7ff6d4131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7ff6dc314700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6d412cf60 0x7ff6d41309e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7ff6dc314700 1 --2- 172.21.15.89:0/2249006414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d412f8b0 0x7ff6d4130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:36.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 >> 172.21.15.89:0/2249006414 conn(0x7ff6d40a2aa0 msgr2=0x7ff6d412a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:36.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 shutdown_connections 2023-09-30T22:23:36.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7ff6dc314700 1 -- 172.21.15.89:0/2249006414 wait complete. 2023-09-30T22:23:36.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 shutdown_connections 2023-09-30T22:23:36.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:36.749+0000 7f0479de5700 1 -- 172.21.15.89:0/1242808433 wait complete. 2023-09-30T22:23:36.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:36.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:37.668 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:37.716 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:37.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:37.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.814+0000 7fd3a2ca3700 1 Processor -- start 2023-09-30T22:23:37.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.814+0000 7fd3a2ca3700 1 -- start start 2023-09-30T22:23:37.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd3a2ca3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd39c06f5d0 0x7fd39c12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd3a2ca3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd3a2ca3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39c06fa40 con 0x7fd39c12cf60 2023-09-30T22:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd3a2ca3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39c12cd90 con 0x7fd39c06f5d0 2023-09-30T22:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd3a2ca3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c071780 con 0x7fd39c135c20 2023-09-30T22:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd3a1240700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd3a1240700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53912/0 (socket says 172.21.15.89:53912) 2023-09-30T22:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd3a1240700 1 -- 172.21.15.89:0/747094649 learned_addr learned my addr 172.21.15.89:0/747094649 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.815+0000 7fd39bfff700 1 --2- 172.21.15.89:0/747094649 >> v2:172.21.15.89:3300/0 conn(0x7fd39c06f5d0 0x7fd39c12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd3a1240700 1 -- 172.21.15.89:0/747094649 >> v2:172.21.15.89:3300/0 conn(0x7fd39c06f5d0 msgr2=0x7fd39c12ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd3a1240700 1 --2- 172.21.15.89:0/747094649 >> v2:172.21.15.89:3300/0 conn(0x7fd39c06f5d0 0x7fd39c12ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd3a1240700 1 -- 172.21.15.89:0/747094649 >> v1:172.21.15.89:6789/0 conn(0x7fd39c12cf60 legacy=0x7fd39c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd3a1240700 1 -- 172.21.15.89:0/747094649 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c071a40 con 0x7fd39c135c20 2023-09-30T22:23:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd39bfff700 1 --2- 172.21.15.89:0/747094649 >> v2:172.21.15.89:3300/0 conn(0x7fd39c06f5d0 0x7fd39c12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:37.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd3a1240700 1 --2- 172.21.15.89:0/747094649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c138000 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d150 tx=0x7fd39400d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5516e14c4de8a37e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:37.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd39b7fe700 1 -- 172.21.15.89:0/747094649 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394010070 con 0x7fd39c135c20 2023-09-30T22:23:37.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd39b7fe700 1 -- 172.21.15.89:0/747094649 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39400db00 con 0x7fd39c135c20 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.816+0000 7fd39b7fe700 1 -- 172.21.15.89:0/747094649 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394016400 con 0x7fd39c135c20 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.817+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/747094649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 msgr2=0x7fd39c138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.817+0000 7fd3a2ca3700 1 --2- 172.21.15.89:0/747094649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c138000 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d150 tx=0x7fd39400d460 comp rx=0 tx=0).stop 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.817+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/747094649 shutdown_connections 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.817+0000 7fd3a2ca3700 1 --2- 172.21.15.89:0/747094649 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c138000 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.817+0000 7fd3a2ca3700 1 --2- 172.21.15.89:0/747094649 >> v2:172.21.15.89:3300/0 conn(0x7fd39c06f5d0 0x7fd39c12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.817+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/747094649 >> 172.21.15.89:0/747094649 conn(0x7fd39c0a2aa0 msgr2=0x7fd39c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.817+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/747094649 shutdown_connections 2023-09-30T22:23:37.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.817+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/747094649 wait complete. 2023-09-30T22:23:37.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.819+0000 7fd3a2ca3700 1 Processor -- start 2023-09-30T22:23:37.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.819+0000 7fd3a2ca3700 1 -- start start 2023-09-30T22:23:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.819+0000 7fd3a2ca3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd39c06f5d0 0x7fd39c127100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.819+0000 7fd3a2ca3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd39c12cf60 0x7fd39c127640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.819+0000 7fd3a2ca3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c1c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.819+0000 7fd3a2ca3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c131470 con 0x7fd39c12cf60 2023-09-30T22:23:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd3a2ca3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c09c4f0 con 0x7fd39c06f5d0 2023-09-30T22:23:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd3a2ca3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c09c790 con 0x7fd39c135c20 2023-09-30T22:23:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd3a1240700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c1c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd3a1240700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c1c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53916/0 (socket says 172.21.15.89:53916) 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd3a1240700 1 -- 172.21.15.89:0/455749594 learned_addr learned my addr 172.21.15.89:0/455749594 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd39bfff700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd39c12cf60 0x7fd39c127640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd39bfff700 1 -- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 msgr2=0x7fd39c1c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd39bfff700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c1c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd39bfff700 1 -- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd39c06f5d0 msgr2=0x7fd39c127100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd39bfff700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd39c06f5d0 0x7fd39c127100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd39bfff700 1 -- 172.21.15.89:0/455749594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880094a0 con 0x7fd39c12cf60 2023-09-30T22:23:37.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd3a1240700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c1c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:37.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd39bfff700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd39c12cf60 0x7fd39c127640 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fd38800e9b0 tx=0x7fd38800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:37.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd3997fa700 1 -- 172.21.15.89:0/455749594 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800c850 con 0x7fd39c12cf60 2023-09-30T22:23:37.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.820+0000 7fd3997fa700 1 -- 172.21.15.89:0/455749594 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38800c9b0 con 0x7fd39c12cf60 2023-09-30T22:23:37.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.821+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39400bd40 con 0x7fd39c12cf60 2023-09-30T22:23:37.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.821+0000 7fd3997fa700 1 -- 172.21.15.89:0/455749594 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800cd60 con 0x7fd39c12cf60 2023-09-30T22:23:37.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.821+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c1311b0 con 0x7fd39c12cf60 2023-09-30T22:23:37.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.822+0000 7fd3997fa700 1 -- 172.21.15.89:0/455749594 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd388013470 con 0x7fd39c12cf60 2023-09-30T22:23:37.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.822+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39c074ea0 con 0x7fd39c12cf60 2023-09-30T22:23:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.822+0000 7fd3997fa700 1 -- 172.21.15.89:0/455749594 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd38803f890 con 0x7fd39c12cf60 2023-09-30T22:23:37.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.827+0000 7fd3997fa700 1 -- 172.21.15.89:0/455749594 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3880232f0 con 0x7fd39c12cf60 2023-09-30T22:23:37.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.857+0000 7ff300b14700 1 Processor -- start 2023-09-30T22:23:37.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.857+0000 7ff300b14700 1 -- start start 2023-09-30T22:23:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.857+0000 7ff300b14700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff300b14700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff300b14700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2fc09f130 con 0x7ff2fc09e410 2023-09-30T22:23:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff300b14700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2fc09f520 con 0x7ff2fc09e000 2023-09-30T22:23:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff300b14700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc09e930 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2fa59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2fa59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53924/0 (socket says 172.21.15.89:53924) 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2fa59c700 1 -- 172.21.15.89:0/2646261299 learned_addr learned my addr 172.21.15.89:0/2646261299 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2f9d9b700 1 --2- 172.21.15.89:0/2646261299 >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2fa59c700 1 -- 172.21.15.89:0/2646261299 >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09e000 msgr2=0x7ff2fc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2fa59c700 1 --2- 172.21.15.89:0/2646261299 >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2fa59c700 1 -- 172.21.15.89:0/2646261299 >> v1:172.21.15.89:6789/0 conn(0x7ff2fc09e410 legacy=0x7ff2fc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2fa59c700 1 -- 172.21.15.89:0/2646261299 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc09b2d0 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2fa59c700 1 --2- 172.21.15.89:0/2646261299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0098f0 tx=0x7ff2ec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506b84c1c404ff2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2f959a700 1 -- 172.21.15.89:0/2646261299 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec011830 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2f959a700 1 -- 172.21.15.89:0/2646261299 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec011990 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.858+0000 7ff2f959a700 1 -- 172.21.15.89:0/2646261299 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec011d20 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.859+0000 7ff300b14700 1 -- 172.21.15.89:0/2646261299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09d6f0 msgr2=0x7ff2fc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.859+0000 7ff300b14700 1 --2- 172.21.15.89:0/2646261299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0098f0 tx=0x7ff2ec009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.859+0000 7ff300b14700 1 -- 172.21.15.89:0/2646261299 shutdown_connections 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.859+0000 7ff300b14700 1 --2- 172.21.15.89:0/2646261299 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09d6f0 0x7ff2fc09dac0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.859+0000 7ff300b14700 1 --2- 172.21.15.89:0/2646261299 >> v2:172.21.15.89:3300/0 conn(0x7ff2fc09e000 0x7ff2fc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.859+0000 7ff300b14700 1 -- 172.21.15.89:0/2646261299 >> 172.21.15.89:0/2646261299 conn(0x7ff2fc066f50 msgr2=0x7ff2fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:37.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.859+0000 7ff300b14700 1 -- 172.21.15.89:0/2646261299 shutdown_connections 2023-09-30T22:23:37.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.859+0000 7ff300b14700 1 -- 172.21.15.89:0/2646261299 wait complete. 2023-09-30T22:23:37.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.860+0000 7ff300b14700 1 Processor -- start 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.860+0000 7ff300b14700 1 -- start start 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff300b14700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff300b14700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc11e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff300b14700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc11ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff300b14700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc11f610 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff300b14700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc1303f0 con 0x7ff2fc09e000 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff300b14700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc120240 con 0x7ff2fc09e410 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41624/0 (socket says 172.21.15.89:41624) 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 -- 172.21.15.89:0/1856859645 learned_addr learned my addr 172.21.15.89:0/1856859645 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fad9d700 1 --2- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc11ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 -- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 msgr2=0x7ff2fc11ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 --2- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc11ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 -- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 msgr2=0x7ff2fc11e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 --2- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc11e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 -- 172.21.15.89:0/1856859645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec009550 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.861+0000 7ff2fa59c700 1 --2- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11e140 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec014040 tx=0x7ff2ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:37.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.862+0000 7ff2eaffd700 1 -- 172.21.15.89:0/1856859645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004590 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.862+0000 7ff2eaffd700 1 -- 172.21.15.89:0/1856859645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec01e070 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.862+0000 7ff2eaffd700 1 -- 172.21.15.89:0/1856859645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0038e0 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.862+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc120e70 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.862+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc0a1d00 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.863+0000 7ff2eaffd700 1 -- 172.21.15.89:0/1856859645 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff2ec003a40 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.863+0000 7ff2eaffd700 1 -- 172.21.15.89:0/1856859645 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff2ec040750 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.863+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc09c760 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.868+0000 7ff2eaffd700 1 -- 172.21.15.89:0/1856859645 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2ec015080 con 0x7ff2fc09d6f0 2023-09-30T22:23:37.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.899+0000 7fc4ab7c3700 1 Processor -- start 2023-09-30T22:23:37.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.900+0000 7fc4ab7c3700 1 -- start start 2023-09-30T22:23:37.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.900+0000 7fc4ab7c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a41338a0 0x7fc4a4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.900+0000 7fc4ab7c3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc4a409e930 0x7fc4a413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.900+0000 7fc4ab7c3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a406ef30 con 0x7fc4a409de50 2023-09-30T22:23:37.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.900+0000 7fc4ab7c3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a406fef0 con 0x7fc4a409e930 2023-09-30T22:23:37.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.900+0000 7fc4ab7c3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a41334b0 con 0x7fc4a41338a0 2023-09-30T22:23:37.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a955f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a41338a0 0x7fc4a4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a955f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a41338a0 0x7fc4a4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53938/0 (socket says 172.21.15.89:53938) 2023-09-30T22:23:37.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a955f700 1 -- 172.21.15.89:0/1700790642 learned_addr learned my addr 172.21.15.89:0/1700790642 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:37.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a9d60700 1 --2- 172.21.15.89:0/1700790642 >> v2:172.21.15.89:3300/0 conn(0x7fc4a409e930 0x7fc4a413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc49bfff700 1 -- 172.21.15.89:0/1700790642 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259744542 0 0) 0x7fc4a406ef30 con 0x7fc4a409de50 2023-09-30T22:23:37.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a955f700 1 -- 172.21.15.89:0/1700790642 >> v2:172.21.15.89:3300/0 conn(0x7fc4a409e930 msgr2=0x7fc4a413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:37.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a955f700 1 --2- 172.21.15.89:0/1700790642 >> v2:172.21.15.89:3300/0 conn(0x7fc4a409e930 0x7fc4a413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a955f700 1 -- 172.21.15.89:0/1700790642 >> v1:172.21.15.89:6789/0 conn(0x7fc4a409de50 legacy=0x7fc4a409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a955f700 1 -- 172.21.15.89:0/1700790642 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a406fc30 con 0x7fc4a41338a0 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc4a955f700 1 --2- 172.21.15.89:0/1700790642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a41338a0 0x7fc4a4133c70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00098f0 tx=0x7fc4a0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46048ced116f51ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc49bfff700 1 -- 172.21.15.89:0/1700790642 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0011830 con 0x7fc4a41338a0 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc49bfff700 1 -- 172.21.15.89:0/1700790642 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a0011990 con 0x7fc4a41338a0 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.901+0000 7fc49bfff700 1 -- 172.21.15.89:0/1700790642 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0011d20 con 0x7fc4a41338a0 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.902+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/1700790642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a41338a0 msgr2=0x7fc4a4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.902+0000 7fc4ab7c3700 1 --2- 172.21.15.89:0/1700790642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a41338a0 0x7fc4a4133c70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00098f0 tx=0x7fc4a0009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:37.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.902+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/1700790642 shutdown_connections 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.902+0000 7fc4ab7c3700 1 --2- 172.21.15.89:0/1700790642 >> v2:172.21.15.89:3300/0 conn(0x7fc4a409e930 0x7fc4a413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.902+0000 7fc4ab7c3700 1 --2- 172.21.15.89:0/1700790642 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a41338a0 0x7fc4a4133c70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.902+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/1700790642 >> 172.21.15.89:0/1700790642 conn(0x7fc4a40a2aa0 msgr2=0x7fc4a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.902+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/1700790642 shutdown_connections 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.902+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/1700790642 wait complete. 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.903+0000 7fc4ab7c3700 1 Processor -- start 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.903+0000 7fc4ab7c3700 1 -- start start 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4ab7c3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a409de50 0x7fc4a411b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4ab7c3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4a409e930 0x7fc4a411bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4ab7c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4a41338a0 0x7fc4a411c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4ab7c3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a411cb20 con 0x7fc4a41338a0 2023-09-30T22:23:37.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4ab7c3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a411d280 con 0x7fc4a409e930 2023-09-30T22:23:37.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4ab7c3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a411e4d0 con 0x7fc4a409de50 2023-09-30T22:23:37.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4a409e930 0x7fc4a411bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4a409e930 0x7fc4a411bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40134/0 (socket says 172.21.15.89:40134) 2023-09-30T22:23:37.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 -- 172.21.15.89:0/3796517980 learned_addr learned my addr 172.21.15.89:0/3796517980 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:37.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a955f700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a409de50 0x7fc4a411b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a9d60700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4a41338a0 0x7fc4a411c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 -- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a409de50 msgr2=0x7fc4a411b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a409de50 0x7fc4a411b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 -- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4a41338a0 msgr2=0x7fc4a411c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4a41338a0 0x7fc4a411c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 -- 172.21.15.89:0/3796517980 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a0009550 con 0x7fc4a409e930 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc4a8d5e700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4a409e930 0x7fc4a411bbb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc49400c450 tx=0x7fc49400c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc499ffb700 1 -- 172.21.15.89:0/3796517980 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4940050d0 con 0x7fc4a409e930 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.904+0000 7fc499ffb700 1 -- 172.21.15.89:0/3796517980 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc494005230 con 0x7fc4a409e930 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.905+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a411eeb0 con 0x7fc4a409e930 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.905+0000 7fc499ffb700 1 -- 172.21.15.89:0/3796517980 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494003680 con 0x7fc4a409e930 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.905+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a406f640 con 0x7fc4a409e930 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.906+0000 7fc499ffb700 1 -- 172.21.15.89:0/3796517980 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc4940037e0 con 0x7fc4a409e930 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.906+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a406ea40 con 0x7fc4a409e930 2023-09-30T22:23:37.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.906+0000 7fc499ffb700 1 -- 172.21.15.89:0/3796517980 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc49404f840 con 0x7fc4a409e930 2023-09-30T22:23:37.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.911+0000 7fc499ffb700 1 -- 172.21.15.89:0/3796517980 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc49402c3b0 con 0x7fc4a409e930 2023-09-30T22:23:37.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.928+0000 7f75822a9700 1 Processor -- start 2023-09-30T22:23:37.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.928+0000 7f75822a9700 1 -- start start 2023-09-30T22:23:37.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.928+0000 7f75822a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c131810 0x7f757c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.928+0000 7f75822a9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c099a40 0x7f757c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.928+0000 7f75822a9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c1313c0 con 0x7f757c09a3b0 2023-09-30T22:23:37.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.928+0000 7f75822a9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c09b580 con 0x7f757c099a40 2023-09-30T22:23:37.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.928+0000 7f75822a9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f757c06e8c0 con 0x7f757c131810 2023-09-30T22:23:37.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c131810 0x7f757c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c131810 0x7f757c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53958/0 (socket says 172.21.15.89:53958) 2023-09-30T22:23:37.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757bfff700 1 -- 172.21.15.89:0/2318250284 learned_addr learned my addr 172.21.15.89:0/2318250284 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:37.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757b7fe700 1 --2- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c099a40 0x7f757c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757bfff700 1 -- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c099a40 msgr2=0x7f757c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:37.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757bfff700 1 --2- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c099a40 0x7f757c099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757bfff700 1 -- 172.21.15.89:0/2318250284 >> v1:172.21.15.89:6789/0 conn(0x7f757c09a3b0 legacy=0x7f757c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757bfff700 1 -- 172.21.15.89:0/2318250284 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c09a820 con 0x7f757c131810 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757b7fe700 1 --2- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c099a40 0x7f757c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757bfff700 1 --2- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c131810 0x7f757c131be0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f756c009920 tx=0x7f756c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a90df9fe84fa39aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757affd700 1 -- 172.21.15.89:0/2318250284 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c00ba60 con 0x7f757c131810 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757affd700 1 -- 172.21.15.89:0/2318250284 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c00bbc0 con 0x7f757c131810 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.929+0000 7f757affd700 1 -- 172.21.15.89:0/2318250284 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c009e70 con 0x7f757c131810 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.930+0000 7f75822a9700 1 -- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c131810 msgr2=0x7f757c131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.930+0000 7f75822a9700 1 --2- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c131810 0x7f757c131be0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f756c009920 tx=0x7f756c009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.930+0000 7f75822a9700 1 -- 172.21.15.89:0/2318250284 shutdown_connections 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.930+0000 7f75822a9700 1 --2- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c131810 0x7f757c131be0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.930+0000 7f75822a9700 1 --2- 172.21.15.89:0/2318250284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c099a40 0x7f757c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.930+0000 7f75822a9700 1 -- 172.21.15.89:0/2318250284 >> 172.21.15.89:0/2318250284 conn(0x7f757c0a2aa0 msgr2=0x7f757c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.930+0000 7f75822a9700 1 -- 172.21.15.89:0/2318250284 shutdown_connections 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.930+0000 7f75822a9700 1 -- 172.21.15.89:0/2318250284 wait complete. 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.931+0000 7f75822a9700 1 Processor -- start 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.931+0000 7f75822a9700 1 -- start start 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f75822a9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c099a40 0x7f757c06f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f75822a9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c09a3b0 0x7f757c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f75822a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757c131810 0x7f757c070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f75822a9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c121180 con 0x7f757c131810 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f75822a9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c120070 con 0x7f757c09a3b0 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f75822a9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0712c0 con 0x7f757c099a40 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c099a40 0x7f757c06f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c099a40 0x7f757c06f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53968/0 (socket says 172.21.15.89:53968) 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 -- 172.21.15.89:0/508771844 learned_addr learned my addr 172.21.15.89:0/508771844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f7580846700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757c131810 0x7f757c070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757b7fe700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c09a3b0 0x7f757c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:37.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 -- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c09a3b0 msgr2=0x7f757c06fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c09a3b0 0x7f757c06fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 -- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757c131810 msgr2=0x7f757c070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757c131810 0x7f757c070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 -- 172.21.15.89:0/508771844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009550 con 0x7f757c099a40 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757b7fe700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c09a3b0 0x7f757c06fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.932+0000 7f757bfff700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c099a40 0x7f757c06f610 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f756c0098f0 tx=0x7f756c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.933+0000 7f7578ff9700 1 -- 172.21.15.89:0/508771844 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c004590 con 0x7f757c099a40 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.933+0000 7f7578ff9700 1 -- 172.21.15.89:0/508771844 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c0036a0 con 0x7f757c099a40 2023-09-30T22:23:37.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.933+0000 7f7578ff9700 1 -- 172.21.15.89:0/508771844 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c003a50 con 0x7f757c099a40 2023-09-30T22:23:37.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.933+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c120be0 con 0x7f757c099a40 2023-09-30T22:23:37.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.933+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c06eb00 con 0x7f757c099a40 2023-09-30T22:23:37.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.934+0000 7f7578ff9700 1 -- 172.21.15.89:0/508771844 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f756c003bb0 con 0x7f757c099a40 2023-09-30T22:23:37.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.934+0000 7f7578ff9700 1 -- 172.21.15.89:0/508771844 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f756c040d80 con 0x7f757c099a40 2023-09-30T22:23:37.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.934+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c074ea0 con 0x7f757c099a40 2023-09-30T22:23:37.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:37.939+0000 7f7578ff9700 1 -- 172.21.15.89:0/508771844 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f756c015020 con 0x7f757c099a40 2023-09-30T22:23:38.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.062+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd39c0713a0 con 0x7fd39c12cf60 2023-09-30T22:23:38.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.063+0000 7fd3997fa700 1 -- 172.21.15.89:0/455749594 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd388021090 con 0x7fd39c12cf60 2023-09-30T22:23:38.064 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd39c12cf60 msgr2=0x7fd39c127640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd39c12cf60 0x7fd39c127640 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fd38800e9b0 tx=0x7fd38800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 shutdown_connections 2023-09-30T22:23:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd39c135c20 0x7fd39c1c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd39c06f5d0 0x7fd39c127100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 --2- 172.21.15.89:0/455749594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd39c12cf60 0x7fd39c127640 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 >> 172.21.15.89:0/455749594 conn(0x7fd39c0a2aa0 msgr2=0x7fd39c12a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 shutdown_connections 2023-09-30T22:23:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7fd3a2ca3700 1 -- 172.21.15.89:0/455749594 wait complete. 2023-09-30T22:23:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.065+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff2fc074ea0 con 0x7ff2fc09d6f0 2023-09-30T22:23:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.066+0000 7ff2eaffd700 1 -- 172.21.15.89:0/1856859645 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2ec003cf0 con 0x7ff2fc09d6f0 2023-09-30T22:23:38.068 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 msgr2=0x7ff2fc11e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 --2- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11e140 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec014040 tx=0x7ff2ec004750 comp rx=0 tx=0).stop 2023-09-30T22:23:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 shutdown_connections 2023-09-30T22:23:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 --2- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2fc09e410 0x7ff2fc11ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 --2- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2fc09d6f0 0x7ff2fc11e140 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 --2- 172.21.15.89:0/1856859645 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2fc09e000 0x7ff2fc11e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 >> 172.21.15.89:0/1856859645 conn(0x7ff2fc066f50 msgr2=0x7ff2fc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 shutdown_connections 2023-09-30T22:23:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.069+0000 7ff300b14700 1 -- 172.21.15.89:0/1856859645 wait complete. 2023-09-30T22:23:38.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:38.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.116+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc4a4074ea0 con 0x7fc4a409e930 2023-09-30T22:23:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.117+0000 7fc499ffb700 1 -- 172.21.15.89:0/3796517980 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc494028030 con 0x7fc4a409e930 2023-09-30T22:23:38.118 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4a409e930 msgr2=0x7fc4a411bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4a409e930 0x7fc4a411bbb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc49400c450 tx=0x7fc49400c760 comp rx=0 tx=0).stop 2023-09-30T22:23:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 shutdown_connections 2023-09-30T22:23:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4a41338a0 0x7fc4a411c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4a409de50 0x7fc4a411b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 --2- 172.21.15.89:0/3796517980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4a409e930 0x7fc4a411bbb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 >> 172.21.15.89:0/3796517980 conn(0x7fc4a40a2aa0 msgr2=0x7fc4a41306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 shutdown_connections 2023-09-30T22:23:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.119+0000 7fc4ab7c3700 1 -- 172.21.15.89:0/3796517980 wait complete. 2023-09-30T22:23:38.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:38.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.139+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f757c06b9e0 con 0x7f757c099a40 2023-09-30T22:23:38.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.139+0000 7f7578ff9700 1 -- 172.21.15.89:0/508771844 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f756c003e60 con 0x7f757c099a40 2023-09-30T22:23:38.140 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c099a40 msgr2=0x7f757c06f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c099a40 0x7f757c06f610 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f756c0098f0 tx=0x7f756c004750 comp rx=0 tx=0).stop 2023-09-30T22:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 shutdown_connections 2023-09-30T22:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f757c131810 0x7f757c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f757c099a40 0x7f757c06f610 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 --2- 172.21.15.89:0/508771844 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f757c09a3b0 0x7f757c06fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 >> 172.21.15.89:0/508771844 conn(0x7f757c0a2aa0 msgr2=0x7f757c1b8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 shutdown_connections 2023-09-30T22:23:38.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:38.141+0000 7f75822a9700 1 -- 172.21.15.89:0/508771844 wait complete. 2023-09-30T22:23:38.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:39.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:39.089 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:39.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:39.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.227+0000 7fa470785700 1 Processor -- start 2023-09-30T22:23:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7fa470785700 1 -- start start 2023-09-30T22:23:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7f2ea933b700 1 Processor -- start 2023-09-30T22:23:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7f2ea933b700 1 -- start start 2023-09-30T22:23:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7f2ea933b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ea409d6f0 0x7f2ea409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7f2ea933b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7f2ea933b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea409b120 con 0x7f2ea409e000 2023-09-30T22:23:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7f2ea933b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea406b460 con 0x7f2ea409d6f0 2023-09-30T22:23:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7f2ea933b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea409a7e0 con 0x7f2ea409e440 2023-09-30T22:23:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7fa470785700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa468136710 0x7fa468138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7fa470785700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468139030 0x7fa46813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7fa470785700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46806ef30 con 0x7fa4681338a0 2023-09-30T22:23:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7fa470785700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46806fef0 con 0x7fa468139030 2023-09-30T22:23:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.228+0000 7fa470785700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4681334b0 con 0x7fa468136710 2023-09-30T22:23:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46dd20700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa468136710 0x7fa468138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46dd20700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa468136710 0x7fa468138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53982/0 (socket says 172.21.15.89:53982) 2023-09-30T22:23:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46dd20700 1 -- 172.21.15.89:0/41772204 learned_addr learned my addr 172.21.15.89:0/41772204 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7f2ea37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7f2ea37fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea4134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40154/0 (socket says 172.21.15.89:40154) 2023-09-30T22:23:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7f2ea37fe700 1 -- 172.21.15.89:0/1488788923 learned_addr learned my addr 172.21.15.89:0/1488788923 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46ed22700 1 --2- 172.21.15.89:0/41772204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468139030 0x7fa46813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46dd20700 1 -- 172.21.15.89:0/41772204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468139030 msgr2=0x7fa46813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46dd20700 1 --2- 172.21.15.89:0/41772204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468139030 0x7fa46813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46dd20700 1 -- 172.21.15.89:0/41772204 >> v1:172.21.15.89:6789/0 conn(0x7fa4681338a0 legacy=0x7fa468133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46dd20700 1 -- 172.21.15.89:0/41772204 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46806fc30 con 0x7fa468136710 2023-09-30T22:23:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7f2ea2ffd700 1 --2- 172.21.15.89:0/1488788923 >> v2:172.21.15.89:3300/0 conn(0x7f2ea409d6f0 0x7f2ea409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7fa46dd20700 1 --2- 172.21.15.89:0/41772204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa468136710 0x7fa468138af0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa458009920 tx=0x7fa458009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ac0413e1d333d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7f2ea37fe700 1 -- 172.21.15.89:0/1488788923 >> v2:172.21.15.89:3300/0 conn(0x7f2ea409d6f0 msgr2=0x7f2ea409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7f2ea37fe700 1 --2- 172.21.15.89:0/1488788923 >> v2:172.21.15.89:3300/0 conn(0x7f2ea409d6f0 0x7f2ea409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7f2ea37fe700 1 -- 172.21.15.89:0/1488788923 >> v1:172.21.15.89:6789/0 conn(0x7f2ea409e000 legacy=0x7f2ea41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.229+0000 7f2ea37fe700 1 -- 172.21.15.89:0/1488788923 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea409a500 con 0x7f2ea409e440 2023-09-30T22:23:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7f2ea37fe700 1 --2- 172.21.15.89:0/1488788923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea4134100 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009e80 tx=0x7f2e9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594e42775ca3bf3f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa46d51f700 1 -- 172.21.15.89:0/41772204 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458004230 con 0x7fa468136710 2023-09-30T22:23:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa46d51f700 1 -- 172.21.15.89:0/41772204 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458004390 con 0x7fa468136710 2023-09-30T22:23:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa46d51f700 1 -- 172.21.15.89:0/41772204 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458009e70 con 0x7fa468136710 2023-09-30T22:23:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7f2ea1ffb700 1 -- 172.21.15.89:0/1488788923 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00e070 con 0x7f2ea409e440 2023-09-30T22:23:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7f2ea1ffb700 1 -- 172.21.15.89:0/1488788923 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c003a20 con 0x7f2ea409e440 2023-09-30T22:23:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7f2ea1ffb700 1 -- 172.21.15.89:0/1488788923 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c003dd0 con 0x7f2ea409e440 2023-09-30T22:23:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa470785700 1 -- 172.21.15.89:0/41772204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa468136710 msgr2=0x7fa468138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa470785700 1 --2- 172.21.15.89:0/41772204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa468136710 0x7fa468138af0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa458009920 tx=0x7fa458009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa470785700 1 -- 172.21.15.89:0/41772204 shutdown_connections 2023-09-30T22:23:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa470785700 1 --2- 172.21.15.89:0/41772204 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468139030 0x7fa46813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa470785700 1 --2- 172.21.15.89:0/41772204 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa468136710 0x7fa468138af0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.230+0000 7fa470785700 1 -- 172.21.15.89:0/41772204 >> 172.21.15.89:0/41772204 conn(0x7fa4680a2aa0 msgr2=0x7fa4680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7fa470785700 1 -- 172.21.15.89:0/41772204 shutdown_connections 2023-09-30T22:23:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7fa470785700 1 -- 172.21.15.89:0/41772204 wait complete. 2023-09-30T22:23:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7f2ea933b700 1 -- 172.21.15.89:0/1488788923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 msgr2=0x7f2ea4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7f2ea933b700 1 --2- 172.21.15.89:0/1488788923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea4134100 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009e80 tx=0x7f2e9c00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7f2ea933b700 1 -- 172.21.15.89:0/1488788923 shutdown_connections 2023-09-30T22:23:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7f2ea933b700 1 --2- 172.21.15.89:0/1488788923 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea4134100 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7f2ea933b700 1 --2- 172.21.15.89:0/1488788923 >> v2:172.21.15.89:3300/0 conn(0x7f2ea409d6f0 0x7f2ea409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7f2ea933b700 1 -- 172.21.15.89:0/1488788923 >> 172.21.15.89:0/1488788923 conn(0x7f2ea4066f50 msgr2=0x7f2ea409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7f2ea933b700 1 -- 172.21.15.89:0/1488788923 shutdown_connections 2023-09-30T22:23:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.231+0000 7f2ea933b700 1 -- 172.21.15.89:0/1488788923 wait complete. 2023-09-30T22:23:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa470785700 1 Processor -- start 2023-09-30T22:23:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa470785700 1 -- start start 2023-09-30T22:23:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa470785700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4681338a0 0x7fa468119d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa470785700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468136710 0x7fa46811a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa470785700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa468139030 0x7fa46811a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa470785700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46806f6b0 con 0x7fa468139030 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa470785700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46811bb00 con 0x7fa468136710 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa470785700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa46811bf60 con 0x7fa4681338a0 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa46e521700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4681338a0 0x7fa468119d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa46e521700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4681338a0 0x7fa468119d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53988/0 (socket says 172.21.15.89:53988) 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa46e521700 1 -- 172.21.15.89:0/3949968611 learned_addr learned my addr 172.21.15.89:0/3949968611 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.232+0000 7fa46ed22700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa468139030 0x7fa46811a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa46dd20700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468136710 0x7fa46811a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa46e521700 1 -- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468136710 msgr2=0x7fa46811a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa46e521700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468136710 0x7fa46811a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa46e521700 1 -- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa468139030 msgr2=0x7fa46811a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa46e521700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa468139030 0x7fa46811a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa46e521700 1 -- 172.21.15.89:0/3949968611 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458009550 con 0x7fa4681338a0 2023-09-30T22:23:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7f2ea933b700 1 Processor -- start 2023-09-30T22:23:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa46ed22700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa468139030 0x7fa46811a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa46e521700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4681338a0 0x7fa468119d70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c4a0 tx=0x7fa46400c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa45effd700 1 -- 172.21.15.89:0/3949968611 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4640054e0 con 0x7fa4681338a0 2023-09-30T22:23:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa45effd700 1 -- 172.21.15.89:0/3949968611 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa464003680 con 0x7fa4681338a0 2023-09-30T22:23:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa45effd700 1 -- 172.21.15.89:0/3949968611 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464003a30 con 0x7fa4681338a0 2023-09-30T22:23:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46811d5b0 con 0x7fa4681338a0 2023-09-30T22:23:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46811d070 con 0x7fa4681338a0 2023-09-30T22:23:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7f2ea933b700 1 -- start start 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7f2ea933b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ea409d6f0 0x7f2ea4119720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7f2ea933b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea409e000 0x7f2ea4119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7f2ea933b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea411a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.233+0000 7f2ea933b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea411be30 con 0x7f2ea409e000 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea933b700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea411c9a0 con 0x7f2ea409e440 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea933b700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea411b890 con 0x7f2ea409d6f0 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea409e000 0x7f2ea4119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea409e000 0x7f2ea4119c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41666/0 (socket says 172.21.15.89:41666) 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea27fc700 1 -- 172.21.15.89:0/666105584 learned_addr learned my addr 172.21.15.89:0/666105584 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea2ffd700 1 --2- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ea409d6f0 0x7f2ea4119720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea27fc700 1 -- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ea409d6f0 msgr2=0x7f2ea4119720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea27fc700 1 --2- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ea409d6f0 0x7f2ea4119720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea27fc700 1 -- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 msgr2=0x7f2ea411a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea27fc700 1 --2- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea411a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.234+0000 7f2ea27fc700 1 -- 172.21.15.89:0/666105584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c009b30 con 0x7f2ea409e000 2023-09-30T22:23:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.235+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa468074ea0 con 0x7fa4681338a0 2023-09-30T22:23:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.235+0000 7fa45effd700 1 -- 172.21.15.89:0/3949968611 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa464003b90 con 0x7fa4681338a0 2023-09-30T22:23:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.235+0000 7fa45effd700 1 -- 172.21.15.89:0/3949968611 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa46403e680 con 0x7fa4681338a0 2023-09-30T22:23:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.235+0000 7f2ea27fc700 1 --2- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea409e000 0x7f2ea4119c60 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800c500 tx=0x7f2e9800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.235+0000 7f2e8b7fe700 1 -- 172.21.15.89:0/666105584 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98011790 con 0x7f2ea409e000 2023-09-30T22:23:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.235+0000 7f2e8b7fe700 1 -- 172.21.15.89:0/666105584 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e980118f0 con 0x7f2ea409e000 2023-09-30T22:23:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.235+0000 7f2e8b7fe700 1 -- 172.21.15.89:0/666105584 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98011ca0 con 0x7f2ea409e000 2023-09-30T22:23:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.242+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea411cfa0 con 0x7f2ea409e000 2023-09-30T22:23:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.242+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea411cc60 con 0x7f2ea409e000 2023-09-30T22:23:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.242+0000 7f2e8b7fe700 1 -- 172.21.15.89:0/666105584 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e9801c460 con 0x7f2ea409e000 2023-09-30T22:23:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.242+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea4074ea0 con 0x7f2ea409e000 2023-09-30T22:23:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.243+0000 7f2e8b7fe700 1 -- 172.21.15.89:0/666105584 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2e9803f430 con 0x7f2ea409e000 2023-09-30T22:23:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.244+0000 7fa45effd700 1 -- 172.21.15.89:0/3949968611 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa464003e40 con 0x7fa4681338a0 2023-09-30T22:23:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.248+0000 7f2e8b7fe700 1 -- 172.21.15.89:0/666105584 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e980108d0 con 0x7f2ea409e000 2023-09-30T22:23:39.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.272+0000 7f860f349700 1 Processor -- start 2023-09-30T22:23:39.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.272+0000 7f860f349700 1 -- start start 2023-09-30T22:23:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.272+0000 7f860f349700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f86080a0a30 0x7f86080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.272+0000 7f860f349700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f860809d630 0x7f86080a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.272+0000 7f860f349700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8608133930 0x7f8608133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.272+0000 7f860f349700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86080a2a70 con 0x7f86080a0a30 2023-09-30T22:23:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.272+0000 7f860f349700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86080a01d0 con 0x7f8608133930 2023-09-30T22:23:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.272+0000 7f860f349700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f860809e2b0 con 0x7f860809d630 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860d0e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f86080a0a30 0x7f86080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860d0e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f86080a0a30 0x7f86080a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41680/0 (socket says 172.21.15.89:41680) 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860d0e5700 1 -- 172.21.15.89:0/2560312296 learned_addr learned my addr 172.21.15.89:0/2560312296 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860c8e4700 1 --2- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f860809d630 0x7f86080a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860d8e6700 1 --2- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8608133930 0x7f8608133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860c8e4700 1 -- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8608133930 msgr2=0x7f8608133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860c8e4700 1 --2- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8608133930 0x7f8608133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860c8e4700 1 -- 172.21.15.89:0/2560312296 >> v2:172.21.15.89:3300/0 conn(0x7f86080a0a30 msgr2=0x7f86080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860c8e4700 1 --2- 172.21.15.89:0/2560312296 >> v2:172.21.15.89:3300/0 conn(0x7f86080a0a30 0x7f86080a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860c8e4700 1 -- 172.21.15.89:0/2560312296 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860809dc50 con 0x7f860809d630 2023-09-30T22:23:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860d0e5700 1 --2- 172.21.15.89:0/2560312296 >> v2:172.21.15.89:3300/0 conn(0x7f86080a0a30 0x7f86080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860d8e6700 1 --2- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8608133930 0x7f8608133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f860c8e4700 1 --2- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f860809d630 0x7f86080a1340 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009840 tx=0x7f85f8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b08d6de8e822c2a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f85fffff700 1 -- 172.21.15.89:0/2560312296 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8011830 con 0x7f860809d630 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f85fffff700 1 -- 172.21.15.89:0/2560312296 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f8011990 con 0x7f860809d630 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.273+0000 7f85fffff700 1 -- 172.21.15.89:0/2560312296 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8011d20 con 0x7f860809d630 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 -- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f860809d630 msgr2=0x7f86080a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 --2- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f860809d630 0x7f86080a1340 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009840 tx=0x7f85f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 -- 172.21.15.89:0/2560312296 shutdown_connections 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 --2- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8608133930 0x7f8608133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 --2- 172.21.15.89:0/2560312296 >> v2:172.21.15.89:3300/0 conn(0x7f86080a0a30 0x7f86080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 --2- 172.21.15.89:0/2560312296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f860809d630 0x7f86080a1340 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 -- 172.21.15.89:0/2560312296 >> 172.21.15.89:0/2560312296 conn(0x7f86080a3050 msgr2=0x7f86080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:39.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 -- 172.21.15.89:0/2560312296 shutdown_connections 2023-09-30T22:23:39.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.274+0000 7f860f349700 1 -- 172.21.15.89:0/2560312296 wait complete. 2023-09-30T22:23:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.275+0000 7f860f349700 1 Processor -- start 2023-09-30T22:23:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.275+0000 7f860f349700 1 -- start start 2023-09-30T22:23:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860f349700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860809d630 0x7f86081c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860f349700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f86080a0a30 0x7f86081c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860f349700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8608133930 0x7f86081c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860f349700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608120e20 con 0x7f860809d630 2023-09-30T22:23:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860f349700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86081206d0 con 0x7f86080a0a30 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860f349700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86080a4030 con 0x7f8608133930 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d0e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860809d630 0x7f86081c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d0e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860809d630 0x7f86081c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41690/0 (socket says 172.21.15.89:41690) 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d0e5700 1 -- 172.21.15.89:0/2254316735 learned_addr learned my addr 172.21.15.89:0/2254316735 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d8e6700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8608133930 0x7f86081c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860c8e4700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f86080a0a30 0x7f86081c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d8e6700 1 -- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f86080a0a30 msgr2=0x7f86081c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d8e6700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f86080a0a30 0x7f86081c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d8e6700 1 -- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860809d630 msgr2=0x7f86081c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d8e6700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860809d630 0x7f86081c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d8e6700 1 -- 172.21.15.89:0/2254316735 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604009550 con 0x7f8608133930 2023-09-30T22:23:39.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860c8e4700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f86080a0a30 0x7f86081c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.276+0000 7f860d8e6700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8608133930 0x7f86081c8b80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8600009e50 tx=0x7f860000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.277+0000 7f85fdffb700 1 -- 172.21.15.89:0/2254316735 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860000cb40 con 0x7f8608133930 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.277+0000 7f85fdffb700 1 -- 172.21.15.89:0/2254316735 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860000f040 con 0x7f8608133930 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.277+0000 7f85fdffb700 1 -- 172.21.15.89:0/2254316735 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600003a20 con 0x7f8608133930 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.277+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8600009a70 con 0x7f8608133930 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.277+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f80094a0 con 0x7f8608133930 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.278+0000 7f85fdffb700 1 -- 172.21.15.89:0/2254316735 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8600014480 con 0x7f8608133930 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.278+0000 7f85fdffb700 1 -- 172.21.15.89:0/2254316735 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f860003ffb0 con 0x7f8608133930 2023-09-30T22:23:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.278+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86080a0470 con 0x7f8608133930 2023-09-30T22:23:39.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.282+0000 7f85fdffb700 1 -- 172.21.15.89:0/2254316735 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8600012930 con 0x7f8608133930 2023-09-30T22:23:39.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.314+0000 7f561c929700 1 Processor -- start 2023-09-30T22:23:39.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.314+0000 7f561c929700 1 -- start start 2023-09-30T22:23:39.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561c929700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f561806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561c929700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f5618139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561c929700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561809ece0 con 0x7f561806b080 2023-09-30T22:23:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561c929700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561809e760 con 0x7f5618131d10 2023-09-30T22:23:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561c929700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f561809e040 con 0x7f561806a770 2023-09-30T22:23:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f561806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f5616d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f5618139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f561806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54032/0 (socket says 172.21.15.89:54032) 2023-09-30T22:23:39.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561659c700 1 -- 172.21.15.89:0/2377543669 learned_addr learned my addr 172.21.15.89:0/2377543669 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:39.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f5616d9d700 1 -- 172.21.15.89:0/2377543669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 msgr2=0x7f561806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:39.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f5616d9d700 1 --2- 172.21.15.89:0/2377543669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f561806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f5616d9d700 1 -- 172.21.15.89:0/2377543669 >> v1:172.21.15.89:6789/0 conn(0x7f561806b080 legacy=0x7f5618131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f5616d9d700 1 -- 172.21.15.89:0/2377543669 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561809f630 con 0x7f5618131d10 2023-09-30T22:23:39.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f561559a700 1 -- 172.21.15.89:0/2377543669 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336494427 0 0) 0x7f561809ece0 con 0x7f561806b080 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.315+0000 7f5616d9d700 1 --2- 172.21.15.89:0/2377543669 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f5618139860 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5610009dc0 tx=0x7f561000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6207c3c9cf70618b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561559a700 1 -- 172.21.15.89:0/2377543669 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000e070 con 0x7f5618131d10 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561559a700 1 -- 172.21.15.89:0/2377543669 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561000bb20 con 0x7f5618131d10 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561559a700 1 -- 172.21.15.89:0/2377543669 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610012430 con 0x7f5618131d10 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561c929700 1 -- 172.21.15.89:0/2377543669 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 msgr2=0x7f5618139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561c929700 1 --2- 172.21.15.89:0/2377543669 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f5618139860 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5610009dc0 tx=0x7f561000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561c929700 1 -- 172.21.15.89:0/2377543669 shutdown_connections 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561c929700 1 --2- 172.21.15.89:0/2377543669 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f5618139860 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561c929700 1 --2- 172.21.15.89:0/2377543669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f561806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561c929700 1 -- 172.21.15.89:0/2377543669 >> 172.21.15.89:0/2377543669 conn(0x7f5618066f50 msgr2=0x7f561809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561c929700 1 -- 172.21.15.89:0/2377543669 shutdown_connections 2023-09-30T22:23:39.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.316+0000 7f561c929700 1 -- 172.21.15.89:0/2377543669 wait complete. 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.317+0000 7f561c929700 1 Processor -- start 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.317+0000 7f561c929700 1 -- start start 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f561c929700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f56181c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f561c929700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f561806b080 0x7f56181c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f561c929700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f56181c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f561c929700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5618071e60 con 0x7f561806b080 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f561c929700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56181386d0 con 0x7f5618131d10 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f561c929700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f561811a560 con 0x7f561806a770 2023-09-30T22:23:39.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f56181c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f56181c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40196/0 (socket says 172.21.15.89:40196) 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 -- 172.21.15.89:0/2216741079 learned_addr learned my addr 172.21.15.89:0/2216741079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5615d9b700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f561806b080 0x7f56181c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f561659c700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f56181c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 -- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 msgr2=0x7f56181c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f56181c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 -- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f561806b080 msgr2=0x7f56181c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f561806b080 0x7f56181c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 -- 172.21.15.89:0/2216741079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5610009a70 con 0x7f5618131d10 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.318+0000 7f5616d9d700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f56181c8b00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5610014040 tx=0x7f5610004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.319+0000 7f5606ffd700 1 -- 172.21.15.89:0/2216741079 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610004680 con 0x7f5618131d10 2023-09-30T22:23:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.319+0000 7f5606ffd700 1 -- 172.21.15.89:0/2216741079 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610031070 con 0x7f5618131d10 2023-09-30T22:23:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.319+0000 7f5606ffd700 1 -- 172.21.15.89:0/2216741079 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56100098f0 con 0x7f5618131d10 2023-09-30T22:23:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.319+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561809a6d0 con 0x7f5618131d10 2023-09-30T22:23:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.319+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561811afd0 con 0x7f5618131d10 2023-09-30T22:23:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.320+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5618074ea0 con 0x7f5618131d10 2023-09-30T22:23:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.320+0000 7f5606ffd700 1 -- 172.21.15.89:0/2216741079 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5610012430 con 0x7f5618131d10 2023-09-30T22:23:39.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.321+0000 7f5606ffd700 1 -- 172.21.15.89:0/2216741079 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5610033070 con 0x7f5618131d10 2023-09-30T22:23:39.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.325+0000 7f5606ffd700 1 -- 172.21.15.89:0/2216741079 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f561002d410 con 0x7f5618131d10 2023-09-30T22:23:39.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.476+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86081220a0 con 0x7f8608133930 2023-09-30T22:23:39.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.476+0000 7f85fdffb700 1 -- 172.21.15.89:0/2254316735 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8600012340 con 0x7f8608133930 2023-09-30T22:23:39.477 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:39.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.478+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8608133930 msgr2=0x7f86081c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.478+0000 7f860f349700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8608133930 0x7f86081c8b80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8600009e50 tx=0x7f860000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.478+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 shutdown_connections 2023-09-30T22:23:39.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.478+0000 7f860f349700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8608133930 0x7f86081c8b80 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.478+0000 7f860f349700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860809d630 0x7f86081c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.478+0000 7f860f349700 1 --2- 172.21.15.89:0/2254316735 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f86080a0a30 0x7f86081c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.478+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 >> 172.21.15.89:0/2254316735 conn(0x7f86080a3050 msgr2=0x7f86081bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:39.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.479+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 shutdown_connections 2023-09-30T22:23:39.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.479+0000 7f860f349700 1 -- 172.21.15.89:0/2254316735 wait complete. 2023-09-30T22:23:39.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.486+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa46811d3a0 con 0x7fa4681338a0 2023-09-30T22:23:39.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.487+0000 7fa45effd700 1 -- 172.21.15.89:0/3949968611 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa464017020 con 0x7fa4681338a0 2023-09-30T22:23:39.487 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:39.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4681338a0 msgr2=0x7fa468119d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4681338a0 0x7fa468119d70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c4a0 tx=0x7fa46400c7b0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 shutdown_connections 2023-09-30T22:23:39.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa468139030 0x7fa46811a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4681338a0 0x7fa468119d70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 --2- 172.21.15.89:0/3949968611 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa468136710 0x7fa46811a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 >> 172.21.15.89:0/3949968611 conn(0x7fa4680a2aa0 msgr2=0x7fa468130050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 shutdown_connections 2023-09-30T22:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.490+0000 7fa470785700 1 -- 172.21.15.89:0/3949968611 wait complete. 2023-09-30T22:23:39.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.491+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2ea409f420 con 0x7f2ea409e000 2023-09-30T22:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.491+0000 7f2e8b7fe700 1 -- 172.21.15.89:0/666105584 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e980102e0 con 0x7f2ea409e000 2023-09-30T22:23:39.493 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea409e000 msgr2=0x7f2ea4119c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 --2- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea409e000 0x7f2ea4119c60 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800c500 tx=0x7f2e9800c810 comp rx=0 tx=0).stop 2023-09-30T22:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 shutdown_connections 2023-09-30T22:23:39.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 --2- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2ea409e440 0x7f2ea411a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 --2- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ea409d6f0 0x7f2ea4119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 --2- 172.21.15.89:0/666105584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea409e000 0x7f2ea4119c60 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 >> 172.21.15.89:0/666105584 conn(0x7f2ea4066f50 msgr2=0x7f2ea41bae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:39.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 shutdown_connections 2023-09-30T22:23:39.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.493+0000 7f2ea933b700 1 -- 172.21.15.89:0/666105584 wait complete. 2023-09-30T22:23:39.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:39.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:39.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.529+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f561806c360 con 0x7f5618131d10 2023-09-30T22:23:39.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.530+0000 7f5606ffd700 1 -- 172.21.15.89:0/2216741079 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f561001a380 con 0x7f5618131d10 2023-09-30T22:23:39.532 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:39.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 msgr2=0x7f56181c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:39.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f56181c8b00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5610014040 tx=0x7f5610004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 shutdown_connections 2023-09-30T22:23:39.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5618131d10 0x7f56181c8b00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f561806a770 0x7f56181c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 --2- 172.21.15.89:0/2216741079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f561806b080 0x7f56181c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:39.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 >> 172.21.15.89:0/2216741079 conn(0x7f5618066f50 msgr2=0x7f5618139500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:39.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 shutdown_connections 2023-09-30T22:23:39.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:39.532+0000 7f561c929700 1 -- 172.21.15.89:0/2216741079 wait complete. 2023-09-30T22:23:39.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:40.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:40.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:40.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:40.546 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:40.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.630+0000 7f48a0327700 1 Processor -- start 2023-09-30T22:23:40.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.630+0000 7f48a0327700 1 -- start start 2023-09-30T22:23:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.630+0000 7f48a0327700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809e1d0 0x7f4898135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.630+0000 7f48a0327700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4898135e50 0x7f4898138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.630+0000 7f48a0327700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489809f130 con 0x7f489809d6f0 2023-09-30T22:23:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.630+0000 7f48a0327700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489809f520 con 0x7f4898135e50 2023-09-30T22:23:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.630+0000 7f48a0327700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f489809e930 con 0x7f489809e1d0 2023-09-30T22:23:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.630+0000 7f489d8c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809e1d0 0x7f4898135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d8c2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809e1d0 0x7f4898135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48196/0 (socket says 172.21.15.89:48196) 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489e0c3700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f489809d6f0 0x7f489809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49556/0 (socket says 172.21.15.89:49556) 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d8c2700 1 -- 172.21.15.89:0/2076735662 learned_addr learned my addr 172.21.15.89:0/2076735662 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489e8c4700 1 --2- 172.21.15.89:0/2076735662 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4898135e50 0x7f4898138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d8c2700 1 -- 172.21.15.89:0/2076735662 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4898135e50 msgr2=0x7f4898138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d8c2700 1 --2- 172.21.15.89:0/2076735662 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4898135e50 0x7f4898138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d8c2700 1 -- 172.21.15.89:0/2076735662 >> v1:172.21.15.89:6789/0 conn(0x7f489809d6f0 legacy=0x7f489809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d8c2700 1 -- 172.21.15.89:0/2076735662 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489809b2d0 con 0x7f489809e1d0 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d8c2700 1 --2- 172.21.15.89:0/2076735662 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809e1d0 0x7f4898135910 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4888009920 tx=0x7f4888009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821f0db70100ecea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d0c1700 1 -- 172.21.15.89:0/2076735662 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888011830 con 0x7f489809e1d0 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d0c1700 1 -- 172.21.15.89:0/2076735662 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4888011990 con 0x7f489809e1d0 2023-09-30T22:23:40.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.631+0000 7f489d0c1700 1 -- 172.21.15.89:0/2076735662 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888011d20 con 0x7f489809e1d0 2023-09-30T22:23:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.632+0000 7f48a0327700 1 -- 172.21.15.89:0/2076735662 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809e1d0 msgr2=0x7f4898135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.632+0000 7f48a0327700 1 --2- 172.21.15.89:0/2076735662 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809e1d0 0x7f4898135910 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4888009920 tx=0x7f4888009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.632+0000 7f48a0327700 1 -- 172.21.15.89:0/2076735662 shutdown_connections 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.632+0000 7f48a0327700 1 --2- 172.21.15.89:0/2076735662 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4898135e50 0x7f4898138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.632+0000 7f48a0327700 1 --2- 172.21.15.89:0/2076735662 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809e1d0 0x7f4898135910 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.632+0000 7f48a0327700 1 -- 172.21.15.89:0/2076735662 >> 172.21.15.89:0/2076735662 conn(0x7f4898066f50 msgr2=0x7f489809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.632+0000 7f48a0327700 1 -- 172.21.15.89:0/2076735662 shutdown_connections 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.632+0000 7f48a0327700 1 -- 172.21.15.89:0/2076735662 wait complete. 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.633+0000 7f48a0327700 1 Processor -- start 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.633+0000 7f48a0327700 1 -- start start 2023-09-30T22:23:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f48a0327700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809d6f0 0x7f48981329e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f48a0327700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f489809e1d0 0x7f4898132f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f48a0327700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4898135e50 0x7f4898133460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f48a0327700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4898119590 con 0x7f4898135e50 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f48a0327700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4898134b80 con 0x7f489809e1d0 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f48a0327700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4898119850 con 0x7f489809d6f0 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e8c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4898135e50 0x7f4898133460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489d8c2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f489809e1d0 0x7f4898132f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e8c4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4898135e50 0x7f4898133460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45128/0 (socket says 172.21.15.89:45128) 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e8c4700 1 -- 172.21.15.89:0/3195917338 learned_addr learned my addr 172.21.15.89:0/3195917338 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e0c3700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809d6f0 0x7f48981329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e8c4700 1 -- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809d6f0 msgr2=0x7f48981329e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e8c4700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809d6f0 0x7f48981329e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e8c4700 1 -- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f489809e1d0 msgr2=0x7f4898132f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e8c4700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f489809e1d0 0x7f4898132f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489e8c4700 1 -- 172.21.15.89:0/3195917338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4888009550 con 0x7f4898135e50 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.634+0000 7f489d8c2700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f489809e1d0 0x7f4898132f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.635+0000 7f489e8c4700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4898135e50 0x7f4898133460 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f489000efc0 tx=0x7f489000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.635+0000 7f488effd700 1 -- 172.21.15.89:0/3195917338 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000cc90 con 0x7f4898135e50 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.635+0000 7f488effd700 1 -- 172.21.15.89:0/3195917338 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489000f040 con 0x7f4898135e50 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.635+0000 7f488effd700 1 -- 172.21.15.89:0/3195917338 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900146c0 con 0x7f4898135e50 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.635+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4898134de0 con 0x7f4898135e50 2023-09-30T22:23:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.635+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48981cc730 con 0x7f4898135e50 2023-09-30T22:23:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.636+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4898071700 con 0x7f4898135e50 2023-09-30T22:23:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.637+0000 7f488effd700 1 -- 172.21.15.89:0/3195917338 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4890014820 con 0x7f4898135e50 2023-09-30T22:23:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.637+0000 7f488effd700 1 -- 172.21.15.89:0/3195917338 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4890040920 con 0x7f4898135e50 2023-09-30T22:23:40.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.645+0000 7f488effd700 1 -- 172.21.15.89:0/3195917338 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4890012910 con 0x7f4898135e50 2023-09-30T22:23:40.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bf8c6700 1 Processor -- start 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bf8c6700 1 -- start start 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bf8c6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b809eca0 0x7f50b813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bf8c6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806f050 0x7f50b813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bf8c6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b8070ee0 con 0x7f50b806e570 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bf8c6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b8071980 con 0x7f50b806f050 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bf8c6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50b8071480 con 0x7f50b809eca0 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bde63700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806f050 0x7f50b813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bde63700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806f050 0x7f50b813f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58134/0 (socket says 172.21.15.89:58134) 2023-09-30T22:23:40.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bd662700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f50b806e570 0x7f50b806e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49570/0 (socket says 172.21.15.89:49570) 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.671+0000 7f50bde63700 1 -- 172.21.15.89:0/60599223 learned_addr learned my addr 172.21.15.89:0/60599223 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50bce61700 1 --2- 172.21.15.89:0/60599223 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b809eca0 0x7f50b813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50bde63700 1 -- 172.21.15.89:0/60599223 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b809eca0 msgr2=0x7f50b813cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50bde63700 1 --2- 172.21.15.89:0/60599223 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b809eca0 0x7f50b813cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50bde63700 1 -- 172.21.15.89:0/60599223 >> v1:172.21.15.89:6789/0 conn(0x7f50b806e570 legacy=0x7f50b806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50bde63700 1 -- 172.21.15.89:0/60599223 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8071b90 con 0x7f50b806f050 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50bde63700 1 --2- 172.21.15.89:0/60599223 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806f050 0x7f50b813f3f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009e80 tx=0x7f50b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f759642c3e21a4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50affff700 1 -- 172.21.15.89:0/60599223 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b000e070 con 0x7f50b806f050 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50affff700 1 -- 172.21.15.89:0/60599223 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b000b910 con 0x7f50b806f050 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.672+0000 7f50affff700 1 -- 172.21.15.89:0/60599223 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b000bcc0 con 0x7f50b806f050 2023-09-30T22:23:40.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.673+0000 7f50bf8c6700 1 -- 172.21.15.89:0/60599223 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806f050 msgr2=0x7f50b813f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.673+0000 7f50bf8c6700 1 --2- 172.21.15.89:0/60599223 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806f050 0x7f50b813f3f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009e80 tx=0x7f50b000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:40.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.673+0000 7f50bf8c6700 1 -- 172.21.15.89:0/60599223 shutdown_connections 2023-09-30T22:23:40.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.673+0000 7f50bf8c6700 1 --2- 172.21.15.89:0/60599223 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806f050 0x7f50b813f3f0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.673+0000 7f50bf8c6700 1 --2- 172.21.15.89:0/60599223 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b809eca0 0x7f50b813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.673+0000 7f50bf8c6700 1 -- 172.21.15.89:0/60599223 >> 172.21.15.89:0/60599223 conn(0x7f50b80a2aa0 msgr2=0x7f50b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:40.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.673+0000 7f50bf8c6700 1 -- 172.21.15.89:0/60599223 shutdown_connections 2023-09-30T22:23:40.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.673+0000 7f50bf8c6700 1 -- 172.21.15.89:0/60599223 wait complete. 2023-09-30T22:23:40.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.674+0000 7f50bf8c6700 1 Processor -- start 2023-09-30T22:23:40.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.674+0000 7f50bf8c6700 1 -- start start 2023-09-30T22:23:40.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bf8c6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806e570 0x7f50b813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bf8c6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b806f050 0x7f50b813d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bf8c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50b812e180 0x7f50b813d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bf8c6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b813dff0 con 0x7f50b812e180 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bf8c6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b813ec60 con 0x7f50b806e570 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bf8c6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50b8070610 con 0x7f50b806f050 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50b812e180 0x7f50b813d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50b812e180 0x7f50b813d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45144/0 (socket says 172.21.15.89:45144) 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 -- 172.21.15.89:0/3934466758 learned_addr learned my addr 172.21.15.89:0/3934466758 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 -- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b806f050 msgr2=0x7f50b813d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 --2- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b806f050 0x7f50b813d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 -- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806e570 msgr2=0x7f50b813cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 --2- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806e570 0x7f50b813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 -- 172.21.15.89:0/3934466758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b0009b30 con 0x7f50b812e180 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bde63700 1 --2- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50b812e180 0x7f50b813d5c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f50b8071070 tx=0x7f50b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50adffb700 1 -- 172.21.15.89:0/3934466758 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0004250 con 0x7f50b812e180 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50adffb700 1 -- 172.21.15.89:0/3934466758 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b00043b0 con 0x7f50b812e180 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b812d410 con 0x7f50b812e180 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.675+0000 7f50adffb700 1 -- 172.21.15.89:0/3934466758 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0004760 con 0x7f50b812e180 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.676+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b812c440 con 0x7f50b812e180 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.677+0000 7f50adffb700 1 -- 172.21.15.89:0/3934466758 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f50b001c440 con 0x7f50b812e180 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.677+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b812c5d0 con 0x7f50b812e180 2023-09-30T22:23:40.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.677+0000 7f50adffb700 1 -- 172.21.15.89:0/3934466758 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f50b0040f90 con 0x7f50b812e180 2023-09-30T22:23:40.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.682+0000 7f2166972700 1 Processor -- start 2023-09-30T22:23:40.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.682+0000 7f2166972700 1 -- start start 2023-09-30T22:23:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.682+0000 7f2166972700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009d6f0 0x7f216009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.682+0000 7f2166972700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009e410 0x7f2160130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.682+0000 7f2166972700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216009f130 con 0x7f216009e000 2023-09-30T22:23:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.682+0000 7f2166972700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216009f520 con 0x7f216009e410 2023-09-30T22:23:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.682+0000 7f2166972700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f216009e930 con 0x7f216009d6f0 2023-09-30T22:23:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f2164f0f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009e410 0x7f2160130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f2164f0f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009e410 0x7f2160130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58146/0 (socket says 172.21.15.89:58146) 2023-09-30T22:23:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f2164f0f700 1 -- 172.21.15.89:0/3643483613 learned_addr learned my addr 172.21.15.89:0/3643483613 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f2164f0f700 1 -- 172.21.15.89:0/3643483613 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009d6f0 msgr2=0x7f216009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f2164f0f700 1 --2- 172.21.15.89:0/3643483613 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009d6f0 0x7f216009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f2164f0f700 1 -- 172.21.15.89:0/3643483613 >> v1:172.21.15.89:6789/0 conn(0x7f216009e000 legacy=0x7f2160130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f2164f0f700 1 -- 172.21.15.89:0/3643483613 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216009b2d0 con 0x7f216009e410 2023-09-30T22:23:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f2164f0f700 1 --2- 172.21.15.89:0/3643483613 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009e410 0x7f2160130d70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f215800d180 tx=0x7f215800d490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6872afd46122444f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f215effd700 1 -- 172.21.15.89:0/3643483613 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158010070 con 0x7f216009e410 2023-09-30T22:23:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f215effd700 1 -- 172.21.15.89:0/3643483613 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215800ddf0 con 0x7f216009e410 2023-09-30T22:23:40.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.683+0000 7f215effd700 1 -- 172.21.15.89:0/3643483613 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215800b310 con 0x7f216009e410 2023-09-30T22:23:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.684+0000 7f2166972700 1 -- 172.21.15.89:0/3643483613 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009e410 msgr2=0x7f2160130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.684+0000 7f2166972700 1 --2- 172.21.15.89:0/3643483613 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009e410 0x7f2160130d70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f215800d180 tx=0x7f215800d490 comp rx=0 tx=0).stop 2023-09-30T22:23:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.684+0000 7f2166972700 1 -- 172.21.15.89:0/3643483613 shutdown_connections 2023-09-30T22:23:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.684+0000 7f2166972700 1 --2- 172.21.15.89:0/3643483613 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009e410 0x7f2160130d70 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.684+0000 7f2166972700 1 --2- 172.21.15.89:0/3643483613 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009d6f0 0x7f216009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.684+0000 7f2166972700 1 -- 172.21.15.89:0/3643483613 >> 172.21.15.89:0/3643483613 conn(0x7f2160066f50 msgr2=0x7f216009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.684+0000 7f2166972700 1 -- 172.21.15.89:0/3643483613 shutdown_connections 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.684+0000 7f2166972700 1 -- 172.21.15.89:0/3643483613 wait complete. 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.685+0000 7f2166972700 1 Processor -- start 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.685+0000 7f2166972700 1 -- start start 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f2166972700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009d6f0 0x7f2160122870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f2166972700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009e000 0x7f2160122db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f2166972700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216009e410 0x7f21601232f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f2166972700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160126410 con 0x7f216009e410 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f2166972700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2160072670 con 0x7f216009d6f0 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f2166972700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2160125d70 con 0x7f216009e000 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009d6f0 0x7f2160122870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009d6f0 0x7f2160122870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58150/0 (socket says 172.21.15.89:58150) 2023-09-30T22:23:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009e000 0x7f2160122db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 -- 172.21.15.89:0/2247068458 learned_addr learned my addr 172.21.15.89:0/2247068458 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f2164f0f700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216009e410 0x7f21601232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 -- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009e000 msgr2=0x7f2160122db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009e000 0x7f2160122db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 -- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216009e410 msgr2=0x7f21601232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216009e410 0x7f21601232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 -- 172.21.15.89:0/2247068458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215800bd40 con 0x7f216009d6f0 2023-09-30T22:23:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f50adffb700 1 -- 172.21.15.89:0/3934466758 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f50b00048c0 con 0x7f50b812e180 2023-09-30T22:23:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215f7fe700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009e000 0x7f2160122db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.686+0000 7f215ffff700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009d6f0 0x7f2160122870 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f215000b530 tx=0x7f215000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.687+0000 7f215cff9700 1 -- 172.21.15.89:0/2247068458 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150011830 con 0x7f216009d6f0 2023-09-30T22:23:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.687+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2160124390 con 0x7f216009d6f0 2023-09-30T22:23:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.687+0000 7f215cff9700 1 -- 172.21.15.89:0/2247068458 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21500119b0 con 0x7f216009d6f0 2023-09-30T22:23:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.687+0000 7f215cff9700 1 -- 172.21.15.89:0/2247068458 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150011d60 con 0x7f216009d6f0 2023-09-30T22:23:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.687+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2160126a10 con 0x7f216009d6f0 2023-09-30T22:23:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.688+0000 7f215cff9700 1 -- 172.21.15.89:0/2247068458 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f215001a400 con 0x7f216009d6f0 2023-09-30T22:23:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.688+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2160131260 con 0x7f216009d6f0 2023-09-30T22:23:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.688+0000 7f215cff9700 1 -- 172.21.15.89:0/2247068458 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2150040cc0 con 0x7f216009d6f0 2023-09-30T22:23:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.693+0000 7f215cff9700 1 -- 172.21.15.89:0/2247068458 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2150044060 con 0x7f216009d6f0 2023-09-30T22:23:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.717+0000 7f676375a700 1 Processor -- start 2023-09-30T22:23:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.717+0000 7f676375a700 1 -- start start 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f676375a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f675c09d6f0 0x7f675c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f676375a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f676375a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675c09f130 con 0x7f675c09e410 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f676375a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675c09f520 con 0x7f675c09d6f0 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f676375a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c09e930 con 0x7f675c09e000 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f67614f6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f675c09d6f0 0x7f675c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f6760cf5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f67614f6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f675c09d6f0 0x7f675c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45164/0 (socket says 172.21.15.89:45164) 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f67614f6700 1 -- 172.21.15.89:0/1925959781 learned_addr learned my addr 172.21.15.89:0/1925959781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.718+0000 7f6761cf7700 1 --1- 172.21.15.89:0/1925959781 >> v1:172.21.15.89:6789/0 conn(0x7f675c09e410 0x7f675c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49592/0 (socket says 172.21.15.89:49592) 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.719+0000 7f6760cf5700 1 -- 172.21.15.89:0/1925959781 >> v2:172.21.15.89:3300/0 conn(0x7f675c09d6f0 msgr2=0x7f675c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.719+0000 7f6760cf5700 1 --2- 172.21.15.89:0/1925959781 >> v2:172.21.15.89:3300/0 conn(0x7f675c09d6f0 0x7f675c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.719+0000 7f6760cf5700 1 -- 172.21.15.89:0/1925959781 >> v1:172.21.15.89:6789/0 conn(0x7f675c09e410 legacy=0x7f675c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.719+0000 7f6760cf5700 1 -- 172.21.15.89:0/1925959781 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c09b2d0 con 0x7f675c09e000 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.719+0000 7f6760cf5700 1 --2- 172.21.15.89:0/1925959781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c130660 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f674c009840 tx=0x7f674c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c73d51f5fc1a73 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.719+0000 7f6753fff700 1 -- 172.21.15.89:0/1925959781 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c011830 con 0x7f675c09e000 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.719+0000 7f6753fff700 1 -- 172.21.15.89:0/1925959781 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c011990 con 0x7f675c09e000 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.719+0000 7f6753fff700 1 -- 172.21.15.89:0/1925959781 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c011d20 con 0x7f675c09e000 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.720+0000 7f676375a700 1 -- 172.21.15.89:0/1925959781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 msgr2=0x7f675c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.720+0000 7f676375a700 1 --2- 172.21.15.89:0/1925959781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c130660 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f674c009840 tx=0x7f674c009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.720+0000 7f676375a700 1 -- 172.21.15.89:0/1925959781 shutdown_connections 2023-09-30T22:23:40.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.720+0000 7f676375a700 1 --2- 172.21.15.89:0/1925959781 >> v2:172.21.15.89:3300/0 conn(0x7f675c09d6f0 0x7f675c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.720+0000 7f676375a700 1 --2- 172.21.15.89:0/1925959781 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c130660 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.720+0000 7f676375a700 1 -- 172.21.15.89:0/1925959781 >> 172.21.15.89:0/1925959781 conn(0x7f675c066f50 msgr2=0x7f675c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:40.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.720+0000 7f676375a700 1 -- 172.21.15.89:0/1925959781 shutdown_connections 2023-09-30T22:23:40.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.720+0000 7f676375a700 1 -- 172.21.15.89:0/1925959781 wait complete. 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.721+0000 7f676375a700 1 Processor -- start 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.721+0000 7f676375a700 1 -- start start 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f676375a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f675c09d6f0 0x7f675c138a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f676375a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c138f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f676375a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f675c09e410 0x7f675c139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f676375a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c12f8a0 con 0x7f675c09d6f0 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f676375a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c12f300 con 0x7f675c09e000 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f676375a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f675c12e720 con 0x7f675c09e410 2023-09-30T22:23:40.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f6761cf7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f675c09e410 0x7f675c139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f67614f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f675c09d6f0 0x7f675c138a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f6761cf7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f675c09e410 0x7f675c139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48254/0 (socket says 172.21.15.89:48254) 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f6761cf7700 1 -- 172.21.15.89:0/3074284156 learned_addr learned my addr 172.21.15.89:0/3074284156 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.722+0000 7f6760cf5700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6761cf7700 1 -- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 msgr2=0x7f675c138f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6761cf7700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c138f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6761cf7700 1 -- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f675c09d6f0 msgr2=0x7f675c138a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6761cf7700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f675c09d6f0 0x7f675c138a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6761cf7700 1 -- 172.21.15.89:0/3074284156 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0094a0 con 0x7f675c09e410 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f67614f6700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f675c09d6f0 0x7f675c138a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6760cf5700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c138f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6761cf7700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f675c09e410 0x7f675c139940 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f675400cbd0 tx=0x7f675400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:40.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6751ffb700 1 -- 172.21.15.89:0/3074284156 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400abb0 con 0x7f675c09e410 2023-09-30T22:23:40.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6751ffb700 1 -- 172.21.15.89:0/3074284156 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675400ad10 con 0x7f675c09e410 2023-09-30T22:23:40.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f6751ffb700 1 -- 172.21.15.89:0/3074284156 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754007270 con 0x7f675c09e410 2023-09-30T22:23:40.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c0725e0 con 0x7f675c09e410 2023-09-30T22:23:40.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.723+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c1394b0 con 0x7f675c09e410 2023-09-30T22:23:40.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.724+0000 7f6751ffb700 1 -- 172.21.15.89:0/3074284156 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67540073d0 con 0x7f675c09e410 2023-09-30T22:23:40.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.725+0000 7f6751ffb700 1 -- 172.21.15.89:0/3074284156 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f675403f660 con 0x7f675c09e410 2023-09-30T22:23:40.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.725+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c139730 con 0x7f675c09e410 2023-09-30T22:23:40.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.734+0000 7f6751ffb700 1 -- 172.21.15.89:0/3074284156 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6754017080 con 0x7f675c09e410 2023-09-30T22:23:40.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.864+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f489809c700 con 0x7f4898135e50 2023-09-30T22:23:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.864+0000 7f488effd700 1 -- 172.21.15.89:0/3195917338 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f489002a020 con 0x7f4898135e50 2023-09-30T22:23:40.865 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4898135e50 msgr2=0x7f4898133460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4898135e50 0x7f4898133460 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f489000efc0 tx=0x7f489000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 shutdown_connections 2023-09-30T22:23:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4898135e50 0x7f4898133460 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f489809d6f0 0x7f48981329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 --2- 172.21.15.89:0/3195917338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f489809e1d0 0x7f4898132f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 >> 172.21.15.89:0/3195917338 conn(0x7f4898066f50 msgr2=0x7f489806e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 shutdown_connections 2023-09-30T22:23:40.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.866+0000 7f48a0327700 1 -- 172.21.15.89:0/3195917338 wait complete. 2023-09-30T22:23:40.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:40.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.880+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f50b8074ea0 con 0x7f50b812e180 2023-09-30T22:23:40.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.881+0000 7f50adffb700 1 -- 172.21.15.89:0/3934466758 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f50b001bb90 con 0x7f50b812e180 2023-09-30T22:23:40.881 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50b812e180 msgr2=0x7f50b813d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 --2- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50b812e180 0x7f50b813d5c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f50b8071070 tx=0x7f50b0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 shutdown_connections 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 --2- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50b812e180 0x7f50b813d5c0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 --2- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f50b806e570 0x7f50b813cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 --2- 172.21.15.89:0/3934466758 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50b806f050 0x7f50b813d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 >> 172.21.15.89:0/3934466758 conn(0x7f50b80a2aa0 msgr2=0x7f50b809cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 shutdown_connections 2023-09-30T22:23:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.883+0000 7f50bf8c6700 1 -- 172.21.15.89:0/3934466758 wait complete. 2023-09-30T22:23:40.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:40.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.907+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2160074ea0 con 0x7f216009d6f0 2023-09-30T22:23:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.907+0000 7f215cff9700 1 -- 172.21.15.89:0/2247068458 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2150015070 con 0x7f216009d6f0 2023-09-30T22:23:40.908 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009d6f0 msgr2=0x7f2160122870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009d6f0 0x7f2160122870 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f215000b530 tx=0x7f215000b840 comp rx=0 tx=0).stop 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 shutdown_connections 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216009e410 0x7f21601232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f216009d6f0 0x7f2160122870 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 --2- 172.21.15.89:0/2247068458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f216009e000 0x7f2160122db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 >> 172.21.15.89:0/2247068458 conn(0x7f2160066f50 msgr2=0x7f216006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 shutdown_connections 2023-09-30T22:23:40.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.909+0000 7f2166972700 1 -- 172.21.15.89:0/2247068458 wait complete. 2023-09-30T22:23:40.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:40.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.947+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f675c074ea0 con 0x7f675c09e410 2023-09-30T22:23:40.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.948+0000 7f6751ffb700 1 -- 172.21.15.89:0/3074284156 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67540108f0 con 0x7f675c09e410 2023-09-30T22:23:40.948 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:40.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f675c09e410 msgr2=0x7f675c139940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:40.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f675c09e410 0x7f675c139940 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f675400cbd0 tx=0x7f675400cf90 comp rx=0 tx=0).stop 2023-09-30T22:23:40.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 shutdown_connections 2023-09-30T22:23:40.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f675c09e410 0x7f675c139940 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f675c09d6f0 0x7f675c138a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 --2- 172.21.15.89:0/3074284156 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f675c09e000 0x7f675c138f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:40.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 >> 172.21.15.89:0/3074284156 conn(0x7f675c066f50 msgr2=0x7f675c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:40.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 shutdown_connections 2023-09-30T22:23:40.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:40.950+0000 7f676375a700 1 -- 172.21.15.89:0/3074284156 wait complete. 2023-09-30T22:23:40.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:41.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:41.897 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:41.924 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:41.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:42.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.022+0000 7f5c99ded700 1 Processor -- start 2023-09-30T22:23:42.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.022+0000 7f5c99ded700 1 -- start start 2023-09-30T22:23:42.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.022+0000 7f5c99ded700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5c9409d6f0 0x7f5c9409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.022+0000 7f5c99ded700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409e000 0x7f5c94130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.022+0000 7f5c99ded700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e410 0x7f5c94130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.022+0000 7f5c99ded700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9409f520 con 0x7f5c9409d6f0 2023-09-30T22:23:42.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.022+0000 7f5c99ded700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9409f130 con 0x7f5c9409e000 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.022+0000 7f5c99ded700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9409e930 con 0x7f5c9409e410 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409e000 0x7f5c94130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409e000 0x7f5c94130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58170/0 (socket says 172.21.15.89:58170) 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3775113190 learned_addr learned my addr 172.21.15.89:0/3775113190 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c93fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e410 0x7f5c94130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3775113190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e410 msgr2=0x7f5c94130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 --2- 172.21.15.89:0/3775113190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e410 0x7f5c94130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3775113190 >> v2:172.21.15.89:3300/0 conn(0x7f5c9409d6f0 msgr2=0x7f5c9409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 --2- 172.21.15.89:0/3775113190 >> v2:172.21.15.89:3300/0 conn(0x7f5c9409d6f0 0x7f5c9409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 -- 172.21.15.89:0/3775113190 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9409b2d0 con 0x7f5c9409e000 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c92ffd700 1 --2- 172.21.15.89:0/3775113190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409e000 0x7f5c94130660 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800bb90 tx=0x7f5c8800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ba5ffdbd00cb77 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:42.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c927fc700 1 -- 172.21.15.89:0/3775113190 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800dd30 con 0x7f5c9409e000 2023-09-30T22:23:42.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c927fc700 1 -- 172.21.15.89:0/3775113190 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8800de90 con 0x7f5c9409e000 2023-09-30T22:23:42.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.023+0000 7f5c927fc700 1 -- 172.21.15.89:0/3775113190 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88010670 con 0x7f5c9409e000 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 -- 172.21.15.89:0/3775113190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409e000 msgr2=0x7f5c94130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 --2- 172.21.15.89:0/3775113190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409e000 0x7f5c94130660 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800bb90 tx=0x7f5c8800bea0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 -- 172.21.15.89:0/3775113190 shutdown_connections 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 --2- 172.21.15.89:0/3775113190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e410 0x7f5c94130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 --2- 172.21.15.89:0/3775113190 >> v2:172.21.15.89:3300/0 conn(0x7f5c9409d6f0 0x7f5c9409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 --2- 172.21.15.89:0/3775113190 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409e000 0x7f5c94130660 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 -- 172.21.15.89:0/3775113190 >> 172.21.15.89:0/3775113190 conn(0x7f5c94066f50 msgr2=0x7f5c9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 -- 172.21.15.89:0/3775113190 shutdown_connections 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.024+0000 7f5c99ded700 1 -- 172.21.15.89:0/3775113190 wait complete. 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.026+0000 7f5c99ded700 1 Processor -- start 2023-09-30T22:23:42.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.026+0000 7f5c99ded700 1 -- start start 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c99ded700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409d6f0 0x7f5c9411e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c99ded700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e000 0x7f5c9411e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c99ded700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c9409e410 0x7f5c9411eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c99ded700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9409c700 con 0x7f5c9409e410 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c99ded700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94120240 con 0x7f5c9409d6f0 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c99ded700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94072640 con 0x7f5c9409e000 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c937fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409d6f0 0x7f5c9411e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c937fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409d6f0 0x7f5c9411e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58174/0 (socket says 172.21.15.89:58174) 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c937fe700 1 -- 172.21.15.89:0/2117104752 learned_addr learned my addr 172.21.15.89:0/2117104752 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:42.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c93fff700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c9409e410 0x7f5c9411eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c92ffd700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e000 0x7f5c9411e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c93fff700 1 -- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e000 msgr2=0x7f5c9411e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c93fff700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e000 0x7f5c9411e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c93fff700 1 -- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409d6f0 msgr2=0x7f5c9411e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c93fff700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409d6f0 0x7f5c9411e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.027+0000 7f5c93fff700 1 -- 172.21.15.89:0/2117104752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8800b7f0 con 0x7f5c9409e410 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.028+0000 7f5c93fff700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c9409e410 0x7f5c9411eb90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00efc0 tx=0x7f5c8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.028+0000 7f5c7bfff700 1 -- 172.21.15.89:0/2117104752 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00cc90 con 0x7f5c9409e410 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.028+0000 7f5c7bfff700 1 -- 172.21.15.89:0/2117104752 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c00f040 con 0x7f5c9409e410 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.028+0000 7f5c7bfff700 1 -- 172.21.15.89:0/2117104752 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0146c0 con 0x7f5c9409e410 2023-09-30T22:23:42.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.028+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c94121330 con 0x7f5c9409e410 2023-09-30T22:23:42.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.028+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c941218c0 con 0x7f5c9409e410 2023-09-30T22:23:42.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.029+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c941214c0 con 0x7f5c9409e410 2023-09-30T22:23:42.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.029+0000 7f5c7bfff700 1 -- 172.21.15.89:0/2117104752 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c8c014820 con 0x7f5c9409e410 2023-09-30T22:23:42.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.030+0000 7f5c7bfff700 1 -- 172.21.15.89:0/2117104752 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5c8c040210 con 0x7f5c9409e410 2023-09-30T22:23:42.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.038+0000 7f5c7bfff700 1 -- 172.21.15.89:0/2117104752 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c8c007920 con 0x7f5c9409e410 2023-09-30T22:23:42.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.061+0000 7f96b346a700 1 Processor -- start 2023-09-30T22:23:42.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.061+0000 7f96b346a700 1 -- start start 2023-09-30T22:23:42.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b346a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f96ac09d9f0 0x7f96ac136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b346a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac136a70 0x7f96ac138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b346a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ac09e130 con 0x7f96ac1316c0 2023-09-30T22:23:42.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b346a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ac1321a0 con 0x7f96ac09d9f0 2023-09-30T22:23:42.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b346a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac1312b0 con 0x7f96ac136a70 2023-09-30T22:23:42.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b1a07700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac136a70 0x7f96ac138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b0a05700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f96ac09d9f0 0x7f96ac136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b1a07700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac136a70 0x7f96ac138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48280/0 (socket says 172.21.15.89:48280) 2023-09-30T22:23:42.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.062+0000 7f96b1a07700 1 -- 172.21.15.89:0/4201484581 learned_addr learned my addr 172.21.15.89:0/4201484581 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:42.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.063+0000 7f96b0a05700 1 -- 172.21.15.89:0/4201484581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac136a70 msgr2=0x7f96ac138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:42.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.063+0000 7f96b0a05700 1 --2- 172.21.15.89:0/4201484581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac136a70 0x7f96ac138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.063+0000 7f96b0a05700 1 -- 172.21.15.89:0/4201484581 >> v1:172.21.15.89:6789/0 conn(0x7f96ac1316c0 legacy=0x7f96ac131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.063+0000 7f96b0a05700 1 -- 172.21.15.89:0/4201484581 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac09f6e0 con 0x7f96ac09d9f0 2023-09-30T22:23:42.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.063+0000 7f96b0a05700 1 --2- 172.21.15.89:0/4201484581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac09d9f0 0x7f96ac136530 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009840 tx=0x7f96a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8adb3f4e553dc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:42.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.063+0000 7f96a3fff700 1 -- 172.21.15.89:0/4201484581 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8011830 con 0x7f96ac09d9f0 2023-09-30T22:23:42.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.063+0000 7f96a3fff700 1 -- 172.21.15.89:0/4201484581 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8011990 con 0x7f96ac09d9f0 2023-09-30T22:23:42.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.063+0000 7f96a3fff700 1 -- 172.21.15.89:0/4201484581 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8011d20 con 0x7f96ac09d9f0 2023-09-30T22:23:42.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f3c16a72700 1 Processor -- start 2023-09-30T22:23:42.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f96b346a700 1 -- 172.21.15.89:0/4201484581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac09d9f0 msgr2=0x7f96ac136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f96b346a700 1 --2- 172.21.15.89:0/4201484581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac09d9f0 0x7f96ac136530 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009840 tx=0x7f96a8009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:42.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f3c16a72700 1 -- start start 2023-09-30T22:23:42.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f96b346a700 1 -- 172.21.15.89:0/4201484581 shutdown_connections 2023-09-30T22:23:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f96b346a700 1 --2- 172.21.15.89:0/4201484581 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac136a70 0x7f96ac138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f96b346a700 1 --2- 172.21.15.89:0/4201484581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac09d9f0 0x7f96ac136530 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f96b346a700 1 -- 172.21.15.89:0/4201484581 >> 172.21.15.89:0/4201484581 conn(0x7f96ac066f50 msgr2=0x7f96ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f3c16a72700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c100a09c0 0x7f3c100a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f3c16a72700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c1009d5c0 0x7f3c100a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f3c16a72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c10133920 0x7f3c10133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f3c16a72700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c100a1810 con 0x7f3c1009d5c0 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f3c16a72700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1009ea10 con 0x7f3c10133920 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.064+0000 7f3c16a72700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1009e210 con 0x7f3c100a09c0 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c0ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c1009d5c0 0x7f3c100a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c0ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3c1009d5c0 0x7f3c100a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45218/0 (socket says 172.21.15.89:45218) 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c0ffff700 1 -- 172.21.15.89:0/1622093175 learned_addr learned my addr 172.21.15.89:0/1622093175 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c1500f700 1 --2- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c10133920 0x7f3c10133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c1480e700 1 --2- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c100a09c0 0x7f3c100a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f96b346a700 1 -- 172.21.15.89:0/4201484581 shutdown_connections 2023-09-30T22:23:42.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f96b346a700 1 -- 172.21.15.89:0/4201484581 wait complete. 2023-09-30T22:23:42.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c1500f700 1 -- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c100a09c0 msgr2=0x7f3c100a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:42.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c1500f700 1 --2- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c100a09c0 0x7f3c100a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c1500f700 1 -- 172.21.15.89:0/1622093175 >> v2:172.21.15.89:3300/0 conn(0x7f3c1009d5c0 msgr2=0x7f3c100a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:42.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c1500f700 1 --2- 172.21.15.89:0/1622093175 >> v2:172.21.15.89:3300/0 conn(0x7f3c1009d5c0 0x7f3c100a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.065+0000 7f3c1500f700 1 -- 172.21.15.89:0/1622093175 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c100a26f0 con 0x7f3c10133920 2023-09-30T22:23:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f3c1500f700 1 --2- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c10133920 0x7f3c10133dc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009d90 tx=0x7f3c0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b4bdc10159edf7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f3c0f7fe700 1 -- 172.21.15.89:0/1622093175 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800e070 con 0x7f3c10133920 2023-09-30T22:23:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f3c0f7fe700 1 -- 172.21.15.89:0/1622093175 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0800bb20 con 0x7f3c10133920 2023-09-30T22:23:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f3c0f7fe700 1 -- 172.21.15.89:0/1622093175 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08012430 con 0x7f3c10133920 2023-09-30T22:23:42.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f96b346a700 1 Processor -- start 2023-09-30T22:23:42.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f96b346a700 1 -- start start 2023-09-30T22:23:42.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f96b346a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac09d9f0 0x7f96ac0718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f96b346a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac1316c0 0x7f96ac071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f96b346a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ac136a70 0x7f96ac1d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f96b346a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac06ea60 con 0x7f96ac1316c0 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f96b346a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac06f1b0 con 0x7f96ac136a70 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.066+0000 7f96b346a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0a3620 con 0x7f96ac09d9f0 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac1316c0 0x7f96ac071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac1316c0 0x7f96ac071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45234/0 (socket says 172.21.15.89:45234) 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 -- 172.21.15.89:0/407784280 learned_addr learned my addr 172.21.15.89:0/407784280 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 -- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac09d9f0 msgr2=0x7f96ac0718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 --2- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac09d9f0 0x7f96ac0718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 -- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ac136a70 msgr2=0x7f96ac1d4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 --2- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ac136a70 0x7f96ac1d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 -- 172.21.15.89:0/407784280 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80094a0 con 0x7f96ac1316c0 2023-09-30T22:23:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b0a05700 1 --2- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac1316c0 0x7f96ac071df0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009fd0 tx=0x7f96a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96a1ffb700 1 -- 172.21.15.89:0/407784280 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80038a0 con 0x7f96ac1316c0 2023-09-30T22:23:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96a1ffb700 1 -- 172.21.15.89:0/407784280 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a801f070 con 0x7f96ac1316c0 2023-09-30T22:23:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96a1ffb700 1 -- 172.21.15.89:0/407784280 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8003c40 con 0x7f96ac1316c0 2023-09-30T22:23:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f3c16a72700 1 -- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c10133920 msgr2=0x7f3c10133dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f3c16a72700 1 --2- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c10133920 0x7f3c10133dc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009d90 tx=0x7f3c0800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.067+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac070250 con 0x7f96ac1316c0 2023-09-30T22:23:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.068+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ac072720 con 0x7f96ac1316c0 2023-09-30T22:23:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.068+0000 7f3c16a72700 1 -- 172.21.15.89:0/1622093175 shutdown_connections 2023-09-30T22:23:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.068+0000 7f3c16a72700 1 --2- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c10133920 0x7f3c10133dc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.068+0000 7f3c16a72700 1 --2- 172.21.15.89:0/1622093175 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c100a09c0 0x7f3c100a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.068+0000 7f3c16a72700 1 --2- 172.21.15.89:0/1622093175 >> v2:172.21.15.89:3300/0 conn(0x7f3c1009d5c0 0x7f3c100a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.068+0000 7f3c16a72700 1 -- 172.21.15.89:0/1622093175 >> 172.21.15.89:0/1622093175 conn(0x7f3c100a2fe0 msgr2=0x7f3c100a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:42.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.068+0000 7f96a1ffb700 1 -- 172.21.15.89:0/407784280 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f96a8026480 con 0x7f96ac1316c0 2023-09-30T22:23:42.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.069+0000 7f3c16a72700 1 -- 172.21.15.89:0/1622093175 shutdown_connections 2023-09-30T22:23:42.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.069+0000 7f3c16a72700 1 -- 172.21.15.89:0/1622093175 wait complete. 2023-09-30T22:23:42.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.069+0000 7f96a1ffb700 1 -- 172.21.15.89:0/407784280 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f96a8021070 con 0x7f96ac1316c0 2023-09-30T22:23:42.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.069+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac074ea0 con 0x7f96ac1316c0 2023-09-30T22:23:42.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.070+0000 7f3c16a72700 1 Processor -- start 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.070+0000 7f3c16a72700 1 -- start start 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c16a72700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1009d5c0 0x7f3c10126010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c16a72700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c100a09c0 0x7f3c10126550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c16a72700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c10133920 0x7f3c10126a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c16a72700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1009cec0 con 0x7f3c1009d5c0 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c16a72700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10129060 con 0x7f3c100a09c0 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c16a72700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101289b0 con 0x7f3c10133920 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1009d5c0 0x7f3c10126010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1009d5c0 0x7f3c10126010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45236/0 (socket says 172.21.15.89:45236) 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 -- 172.21.15.89:0/1797448464 learned_addr learned my addr 172.21.15.89:0/1797448464 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1500f700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c10133920 0x7f3c10126a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c0ffff700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c100a09c0 0x7f3c10126550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 -- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c10133920 msgr2=0x7f3c10126a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c10133920 0x7f3c10126a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 -- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c100a09c0 msgr2=0x7f3c10126550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c100a09c0 0x7f3c10126550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 -- 172.21.15.89:0/1797448464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c08009a70 con 0x7f3c1009d5c0 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.071+0000 7f3c1480e700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1009d5c0 0x7f3c10126010 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400e9b0 tx=0x7f3c0400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.073+0000 7f3c0d7fa700 1 -- 172.21.15.89:0/1797448464 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04009710 con 0x7f3c1009d5c0 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.073+0000 7f3c0d7fa700 1 -- 172.21.15.89:0/1797448464 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0400c850 con 0x7f3c1009d5c0 2023-09-30T22:23:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.073+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c10132e90 con 0x7f3c1009d5c0 2023-09-30T22:23:42.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.073+0000 7f96a1ffb700 1 -- 172.21.15.89:0/407784280 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96a8015030 con 0x7f96ac1316c0 2023-09-30T22:23:42.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.073+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c10129600 con 0x7f3c1009d5c0 2023-09-30T22:23:42.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.073+0000 7f3c0d7fa700 1 -- 172.21.15.89:0/1797448464 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400cbc0 con 0x7f3c1009d5c0 2023-09-30T22:23:42.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.075+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf00052e0 con 0x7f3c1009d5c0 2023-09-30T22:23:42.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.077+0000 7f3c0d7fa700 1 -- 172.21.15.89:0/1797448464 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c0400cd20 con 0x7f3c1009d5c0 2023-09-30T22:23:42.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.077+0000 7f3c0d7fa700 1 -- 172.21.15.89:0/1797448464 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3c0403fff0 con 0x7f3c1009d5c0 2023-09-30T22:23:42.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.086+0000 7f3c0d7fa700 1 -- 172.21.15.89:0/1797448464 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c0401a360 con 0x7f3c1009d5c0 2023-09-30T22:23:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.132+0000 7f319c4b6700 1 Processor -- start 2023-09-30T22:23:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.132+0000 7f319c4b6700 1 -- start start 2023-09-30T22:23:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.132+0000 7f319c4b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f319406f5d0 0x7f319412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.132+0000 7f319c4b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3194135c20 0x7f3194138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.132+0000 7f319c4b6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319406fa40 con 0x7f319412cf60 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.132+0000 7f319c4b6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319412cd90 con 0x7f319406f5d0 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.132+0000 7f319c4b6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3194071780 con 0x7f3194135c20 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f319a252700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f319412cf60 0x7f319412f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49610/0 (socket says 172.21.15.89:49610) 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f319a252700 1 -- 172.21.15.89:0/732366589 learned_addr learned my addr 172.21.15.89:0/732366589 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f3199a51700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f319406f5d0 0x7f319412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f319aa53700 1 --2- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3194135c20 0x7f3194138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f3199250700 1 -- 172.21.15.89:0/732366589 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458762483 0 0) 0x7f319406fa40 con 0x7f319412cf60 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f3199a51700 1 -- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3194135c20 msgr2=0x7f3194138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f3199a51700 1 --2- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3194135c20 0x7f3194138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f3199a51700 1 -- 172.21.15.89:0/732366589 >> v1:172.21.15.89:6789/0 conn(0x7f319412cf60 legacy=0x7f319412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f3199a51700 1 -- 172.21.15.89:0/732366589 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3194071a40 con 0x7f319406f5d0 2023-09-30T22:23:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f319aa53700 1 --2- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3194135c20 0x7f3194138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f3199a51700 1 --2- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f319406f5d0 0x7f319412ffc0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f318400aa10 tx=0x7f318400ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340d5e95e344d20d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.133+0000 7f3199250700 1 -- 172.21.15.89:0/732366589 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318400cb70 con 0x7f319406f5d0 2023-09-30T22:23:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.134+0000 7f3199250700 1 -- 172.21.15.89:0/732366589 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318400ccd0 con 0x7f319406f5d0 2023-09-30T22:23:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.134+0000 7f3199250700 1 -- 172.21.15.89:0/732366589 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184018620 con 0x7f319406f5d0 2023-09-30T22:23:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.134+0000 7f319c4b6700 1 -- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f319406f5d0 msgr2=0x7f319412ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.134+0000 7f319c4b6700 1 --2- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f319406f5d0 0x7f319412ffc0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f318400aa10 tx=0x7f318400ad20 comp rx=0 tx=0).stop 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.135+0000 7f319c4b6700 1 -- 172.21.15.89:0/732366589 shutdown_connections 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.135+0000 7f319c4b6700 1 --2- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3194135c20 0x7f3194138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.135+0000 7f319c4b6700 1 --2- 172.21.15.89:0/732366589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f319406f5d0 0x7f319412ffc0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.135+0000 7f319c4b6700 1 -- 172.21.15.89:0/732366589 >> 172.21.15.89:0/732366589 conn(0x7f31940a2aa0 msgr2=0x7f31940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.135+0000 7f319c4b6700 1 -- 172.21.15.89:0/732366589 shutdown_connections 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.135+0000 7f319c4b6700 1 -- 172.21.15.89:0/732366589 wait complete. 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.136+0000 7f319c4b6700 1 Processor -- start 2023-09-30T22:23:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.136+0000 7f319c4b6700 1 -- start start 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319c4b6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f319406f5d0 0x7f3194132e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319c4b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319412cf60 0x7f3194133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319c4b6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3194135c20 0x7f3194133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319c4b6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3194129fa0 con 0x7f319412cf60 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319c4b6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319409a910 con 0x7f3194135c20 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319c4b6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3194134cc0 con 0x7f319406f5d0 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319a252700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f319406f5d0 0x7f3194132e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319a252700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f319406f5d0 0x7f3194132e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48320/0 (socket says 172.21.15.89:48320) 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319a252700 1 -- 172.21.15.89:0/2388308149 learned_addr learned my addr 172.21.15.89:0/2388308149 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f3199a51700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319412cf60 0x7f3194133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319aa53700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3194135c20 0x7f3194133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319a252700 1 -- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3194135c20 msgr2=0x7f3194133880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319a252700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3194135c20 0x7f3194133880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319a252700 1 -- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319412cf60 msgr2=0x7f3194133340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319a252700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319412cf60 0x7f3194133340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.137+0000 7f319a252700 1 -- 172.21.15.89:0/2388308149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c009a70 con 0x7f319406f5d0 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.138+0000 7f319a252700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f319406f5d0 0x7f3194132e00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f319000c520 tx=0x7f319000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.138+0000 7f318affd700 1 -- 172.21.15.89:0/2388308149 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190004c80 con 0x7f319406f5d0 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.138+0000 7f318affd700 1 -- 172.21.15.89:0/2388308149 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3190004e00 con 0x7f319406f5d0 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.138+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318400a640 con 0x7f319406f5d0 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.138+0000 7f318affd700 1 -- 172.21.15.89:0/2388308149 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190007310 con 0x7f319406f5d0 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.138+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31940a1660 con 0x7f319406f5d0 2023-09-30T22:23:42.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.139+0000 7f318affd700 1 -- 172.21.15.89:0/2388308149 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3190007470 con 0x7f319406f5d0 2023-09-30T22:23:42.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.139+0000 7f318affd700 1 -- 172.21.15.89:0/2388308149 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f319003f300 con 0x7f319406f5d0 2023-09-30T22:23:42.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.139+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3194074ea0 con 0x7f319406f5d0 2023-09-30T22:23:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.144+0000 7f318affd700 1 -- 172.21.15.89:0/2388308149 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31900123b0 con 0x7f319406f5d0 2023-09-30T22:23:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.285+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f96ac12b900 con 0x7f96ac1316c0 2023-09-30T22:23:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.286+0000 7f96a1ffb700 1 -- 172.21.15.89:0/407784280 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96a8026730 con 0x7f96ac1316c0 2023-09-30T22:23:42.287 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac1316c0 msgr2=0x7f96ac071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 --2- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac1316c0 0x7f96ac071df0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009fd0 tx=0x7f96a8004750 comp rx=0 tx=0).stop 2023-09-30T22:23:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 shutdown_connections 2023-09-30T22:23:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 --2- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96ac136a70 0x7f96ac1d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 --2- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96ac09d9f0 0x7f96ac0718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 --2- 172.21.15.89:0/407784280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96ac1316c0 0x7f96ac071df0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 >> 172.21.15.89:0/407784280 conn(0x7f96ac066f50 msgr2=0x7f96ac09a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 shutdown_connections 2023-09-30T22:23:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.289+0000 7f96b346a700 1 -- 172.21.15.89:0/407784280 wait complete. 2023-09-30T22:23:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.291+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3bf0005860 con 0x7f3c1009d5c0 2023-09-30T22:23:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.291+0000 7f3c0d7fa700 1 -- 172.21.15.89:0/1797448464 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c04021090 con 0x7f3c1009d5c0 2023-09-30T22:23:42.292 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:42.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.294+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1009d5c0 msgr2=0x7f3c10126010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.294+0000 7f3c16a72700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1009d5c0 0x7f3c10126010 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400e9b0 tx=0x7f3c0400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.294+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 shutdown_connections 2023-09-30T22:23:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.294+0000 7f3c16a72700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c10133920 0x7f3c10126a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.294+0000 7f3c16a72700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1009d5c0 0x7f3c10126010 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.294+0000 7f3c16a72700 1 --2- 172.21.15.89:0/1797448464 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c100a09c0 0x7f3c10126550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.294+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 >> 172.21.15.89:0/1797448464 conn(0x7f3c100a2fe0 msgr2=0x7f3c1009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.295+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 shutdown_connections 2023-09-30T22:23:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.295+0000 7f3c16a72700 1 -- 172.21.15.89:0/1797448464 wait complete. 2023-09-30T22:23:42.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.297+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5c94074ea0 con 0x7f5c9409e410 2023-09-30T22:23:42.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.298+0000 7f5c7bfff700 1 -- 172.21.15.89:0/2117104752 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c8c018020 con 0x7f5c9409e410 2023-09-30T22:23:42.299 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c9409e410 msgr2=0x7f5c9411eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c9409e410 0x7f5c9411eb90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00efc0 tx=0x7f5c8c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 shutdown_connections 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c9409e410 0x7f5c9411eb90 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c9409d6f0 0x7f5c9411e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 --2- 172.21.15.89:0/2117104752 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c9409e000 0x7f5c9411e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 >> 172.21.15.89:0/2117104752 conn(0x7f5c94066f50 msgr2=0x7f5c94070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 shutdown_connections 2023-09-30T22:23:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.300+0000 7f5c99ded700 1 -- 172.21.15.89:0/2117104752 wait complete. 2023-09-30T22:23:42.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:42.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:42.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:42.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.368+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f319409a4d0 con 0x7f319406f5d0 2023-09-30T22:23:42.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.369+0000 7f318affd700 1 -- 172.21.15.89:0/2388308149 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3190011360 con 0x7f319406f5d0 2023-09-30T22:23:42.369 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f319406f5d0 msgr2=0x7f3194132e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f319406f5d0 0x7f3194132e00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f319000c520 tx=0x7f319000c830 comp rx=0 tx=0).stop 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 shutdown_connections 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3194135c20 0x7f3194133880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f319406f5d0 0x7f3194132e00 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 --2- 172.21.15.89:0/2388308149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f319412cf60 0x7f3194133340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 >> 172.21.15.89:0/2388308149 conn(0x7f31940a2aa0 msgr2=0x7f319412bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 shutdown_connections 2023-09-30T22:23:42.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:42.371+0000 7f319c4b6700 1 -- 172.21.15.89:0/2388308149 wait complete. 2023-09-30T22:23:42.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:43.305 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:43.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:43.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:43.384 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:43.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090ffdb700 1 Processor -- start 2023-09-30T22:23:43.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090ffdb700 1 -- start start 2023-09-30T22:23:43.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090ffdb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809d6f0 0x7f090809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090ffdb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f090809e000 0x7f0908130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090ffdb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090809f130 con 0x7f090809e410 2023-09-30T22:23:43.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090ffdb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090809f520 con 0x7f090809e000 2023-09-30T22:23:43.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090ffdb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f090809e930 con 0x7f090809d6f0 2023-09-30T22:23:43.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090d576700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f090809e000 0x7f0908130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090d576700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f090809e000 0x7f0908130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45244/0 (socket says 172.21.15.89:45244) 2023-09-30T22:23:43.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090d576700 1 -- 172.21.15.89:0/2963138200 learned_addr learned my addr 172.21.15.89:0/2963138200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:43.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.443+0000 7f090dd77700 1 --2- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809d6f0 0x7f090809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090d576700 1 -- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809d6f0 msgr2=0x7f090809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:43.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090d576700 1 --2- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809d6f0 0x7f090809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090d576700 1 -- 172.21.15.89:0/2963138200 >> v1:172.21.15.89:6789/0 conn(0x7f090809e410 legacy=0x7f0908130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090d576700 1 -- 172.21.15.89:0/2963138200 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090809b2d0 con 0x7f090809e000 2023-09-30T22:23:43.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090dd77700 1 --2- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809d6f0 0x7f090809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:43.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090d576700 1 --2- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f0908130660 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f08f800bb90 tx=0x7f08f800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dcea3e31bb74c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:43.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090cd75700 1 -- 172.21.15.89:0/2963138200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f800dd30 con 0x7f090809e000 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090cd75700 1 -- 172.21.15.89:0/2963138200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f800de90 con 0x7f090809e000 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.444+0000 7f090cd75700 1 -- 172.21.15.89:0/2963138200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8010690 con 0x7f090809e000 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.445+0000 7f090ffdb700 1 -- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 msgr2=0x7f0908130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.445+0000 7f090ffdb700 1 --2- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f0908130660 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f08f800bb90 tx=0x7f08f800bea0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.445+0000 7f090ffdb700 1 -- 172.21.15.89:0/2963138200 shutdown_connections 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.445+0000 7f090ffdb700 1 --2- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809d6f0 0x7f090809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.445+0000 7f090ffdb700 1 --2- 172.21.15.89:0/2963138200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f0908130660 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.445+0000 7f090ffdb700 1 -- 172.21.15.89:0/2963138200 >> 172.21.15.89:0/2963138200 conn(0x7f0908066f50 msgr2=0x7f090809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.445+0000 7f090ffdb700 1 -- 172.21.15.89:0/2963138200 shutdown_connections 2023-09-30T22:23:43.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.445+0000 7f090ffdb700 1 -- 172.21.15.89:0/2963138200 wait complete. 2023-09-30T22:23:43.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.446+0000 7f090ffdb700 1 Processor -- start 2023-09-30T22:23:43.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.446+0000 7f090ffdb700 1 -- start start 2023-09-30T22:23:43.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.446+0000 7f090ffdb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f090809d6f0 0x7f090811dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.446+0000 7f090ffdb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f090811e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.446+0000 7f090ffdb700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809e410 0x7f090811ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.446+0000 7f090ffdb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090811f4a0 con 0x7f090809e000 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.446+0000 7f090ffdb700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0908121af0 con 0x7f090809d6f0 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.446+0000 7f090ffdb700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0908120e30 con 0x7f090809e410 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f090811e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f090811e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45258/0 (socket says 172.21.15.89:45258) 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 -- 172.21.15.89:0/3786781591 learned_addr learned my addr 172.21.15.89:0/3786781591 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090dd77700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f090809d6f0 0x7f090811dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090e578700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809e410 0x7f090811ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 -- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809e410 msgr2=0x7f090811ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809e410 0x7f090811ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 -- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f090809d6f0 msgr2=0x7f090811dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f090809d6f0 0x7f090811dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 -- 172.21.15.89:0/3786781591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040094a0 con 0x7f090809e000 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090dd77700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f090809d6f0 0x7f090811dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090d576700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f090811e510 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f08f800bee0 tx=0x7f08f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f08fe7fc700 1 -- 172.21.15.89:0/3786781591 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8003c70 con 0x7f090809e000 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f08fe7fc700 1 -- 172.21.15.89:0/3786781591 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f8003df0 con 0x7f090809e000 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f08fe7fc700 1 -- 172.21.15.89:0/3786781591 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80106b0 con 0x7f090809e000 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f800b7f0 con 0x7f090809e000 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.447+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0908121280 con 0x7f090809e000 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.448+0000 7f08fe7fc700 1 -- 172.21.15.89:0/3786781591 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08f8019400 con 0x7f090809e000 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.449+0000 7f08fe7fc700 1 -- 172.21.15.89:0/3786781591 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f08f8040b10 con 0x7f090809e000 2023-09-30T22:23:43.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.449+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09080a1d90 con 0x7f090809e000 2023-09-30T22:23:43.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.454+0000 7f08fe7fc700 1 -- 172.21.15.89:0/3786781591 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08f8018950 con 0x7f090809e000 2023-09-30T22:23:43.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.482+0000 7fec68ad6700 1 Processor -- start 2023-09-30T22:23:43.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.482+0000 7fec68ad6700 1 -- start start 2023-09-30T22:23:43.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec68ad6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec64136710 0x7fec64138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec68ad6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fec64139030 0x7fec6413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec68ad6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6406ef30 con 0x7fec641338a0 2023-09-30T22:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec68ad6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6406fef0 con 0x7fec64139030 2023-09-30T22:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec68ad6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec641334b0 con 0x7fec64136710 2023-09-30T22:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec62d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fec64139030 0x7fec6413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec62d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fec64139030 0x7fec6413b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45268/0 (socket says 172.21.15.89:45268) 2023-09-30T22:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec62d9d700 1 -- 172.21.15.89:0/4039080579 learned_addr learned my addr 172.21.15.89:0/4039080579 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec61d9b700 1 --2- 172.21.15.89:0/4039080579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec64136710 0x7fec64138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec62d9d700 1 -- 172.21.15.89:0/4039080579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec64136710 msgr2=0x7fec64138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec62d9d700 1 --2- 172.21.15.89:0/4039080579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec64136710 0x7fec64138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec62d9d700 1 -- 172.21.15.89:0/4039080579 >> v1:172.21.15.89:6789/0 conn(0x7fec641338a0 legacy=0x7fec64133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec6159a700 1 -- 172.21.15.89:0/4039080579 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015838452 0 0) 0x7fec6406ef30 con 0x7fec641338a0 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.483+0000 7fec62d9d700 1 -- 172.21.15.89:0/4039080579 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6406fc30 con 0x7fec64139030 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec62d9d700 1 --2- 172.21.15.89:0/4039080579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64139030 0x7fec6413b410 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009e50 tx=0x7fec5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8368b23a8fa6e8e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec6159a700 1 -- 172.21.15.89:0/4039080579 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00e070 con 0x7fec64139030 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec6159a700 1 -- 172.21.15.89:0/4039080579 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c00b910 con 0x7fec64139030 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec6159a700 1 -- 172.21.15.89:0/4039080579 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00bcc0 con 0x7fec64139030 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec68ad6700 1 -- 172.21.15.89:0/4039080579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64139030 msgr2=0x7fec6413b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec68ad6700 1 --2- 172.21.15.89:0/4039080579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64139030 0x7fec6413b410 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009e50 tx=0x7fec5c00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec68ad6700 1 -- 172.21.15.89:0/4039080579 shutdown_connections 2023-09-30T22:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec68ad6700 1 --2- 172.21.15.89:0/4039080579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64139030 0x7fec6413b410 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec68ad6700 1 --2- 172.21.15.89:0/4039080579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec64136710 0x7fec64138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.484+0000 7fec68ad6700 1 -- 172.21.15.89:0/4039080579 >> 172.21.15.89:0/4039080579 conn(0x7fec640a2aa0 msgr2=0x7fec640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.485+0000 7fec68ad6700 1 -- 172.21.15.89:0/4039080579 shutdown_connections 2023-09-30T22:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.485+0000 7fec68ad6700 1 -- 172.21.15.89:0/4039080579 wait complete. 2023-09-30T22:23:43.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.486+0000 7fec68ad6700 1 Processor -- start 2023-09-30T22:23:43.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.486+0000 7fec68ad6700 1 -- start start 2023-09-30T22:23:43.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec68ad6700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec641338a0 0x7fec6411abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec68ad6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64136710 0x7fec6411b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec68ad6700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec64139030 0x7fec6411b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec68ad6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6411e770 con 0x7fec64136710 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec68ad6700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec64138cb0 con 0x7fec64139030 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec68ad6700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec6406f640 con 0x7fec641338a0 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec641338a0 0x7fec6411abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec641338a0 0x7fec6411abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48344/0 (socket says 172.21.15.89:48344) 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 -- 172.21.15.89:0/2492526738 learned_addr learned my addr 172.21.15.89:0/2492526738 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec61d9b700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64136710 0x7fec6411b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec62d9d700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec64139030 0x7fec6411b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 -- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec64139030 msgr2=0x7fec6411b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec64139030 0x7fec6411b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 -- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64136710 msgr2=0x7fec6411b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64136710 0x7fec6411b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 -- 172.21.15.89:0/2492526738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c009b30 con 0x7fec641338a0 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.487+0000 7fec6259c700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec641338a0 0x7fec6411abb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fec54001eb0 tx=0x7fec5400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.488+0000 7fec52ffd700 1 -- 172.21.15.89:0/2492526738 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540054e0 con 0x7fec641338a0 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.488+0000 7fec52ffd700 1 -- 172.21.15.89:0/2492526738 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec54003680 con 0x7fec641338a0 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.488+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6411de60 con 0x7fec641338a0 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.488+0000 7fec52ffd700 1 -- 172.21.15.89:0/2492526738 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540039f0 con 0x7fec641338a0 2023-09-30T22:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.488+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6411e3f0 con 0x7fec641338a0 2023-09-30T22:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.489+0000 7fec52ffd700 1 -- 172.21.15.89:0/2492526738 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fec54003b50 con 0x7fec641338a0 2023-09-30T22:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.489+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64130aa0 con 0x7fec641338a0 2023-09-30T22:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.489+0000 7fec52ffd700 1 -- 172.21.15.89:0/2492526738 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fec5403ea50 con 0x7fec641338a0 2023-09-30T22:23:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.492+0000 7faa668ef700 1 Processor -- start 2023-09-30T22:23:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.492+0000 7faa668ef700 1 -- start start 2023-09-30T22:23:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa668ef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa6006e570 0x7faa6006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa668ef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006ee80 0x7faa60134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa668ef700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa60070ee0 con 0x7faa6009eca0 2023-09-30T22:23:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa668ef700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa60071480 con 0x7faa6006e570 2023-09-30T22:23:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa668ef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa60071b90 con 0x7faa6006ee80 2023-09-30T22:23:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa64e8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006ee80 0x7faa60134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa64e8c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006ee80 0x7faa60134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48352/0 (socket says 172.21.15.89:48352) 2023-09-30T22:23:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa64e8c700 1 -- 172.21.15.89:0/2321507880 learned_addr learned my addr 172.21.15.89:0/2321507880 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa64e8c700 1 -- 172.21.15.89:0/2321507880 >> v2:172.21.15.89:3300/0 conn(0x7faa6006e570 msgr2=0x7faa6006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa64e8c700 1 --2- 172.21.15.89:0/2321507880 >> v2:172.21.15.89:3300/0 conn(0x7faa6006e570 0x7faa6006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa64e8c700 1 -- 172.21.15.89:0/2321507880 >> v1:172.21.15.89:6789/0 conn(0x7faa6009eca0 legacy=0x7faa60134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa64e8c700 1 -- 172.21.15.89:0/2321507880 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa600711a0 con 0x7faa6006ee80 2023-09-30T22:23:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.493+0000 7faa64e8c700 1 --2- 172.21.15.89:0/2321507880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006ee80 0x7faa60134d20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7faa58009e50 tx=0x7faa5800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fa8ac0236c7c4d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.497+0000 7faa5effd700 1 -- 172.21.15.89:0/2321507880 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800e070 con 0x7faa6006ee80 2023-09-30T22:23:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa5effd700 1 -- 172.21.15.89:0/2321507880 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58003a20 con 0x7faa6006ee80 2023-09-30T22:23:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa5effd700 1 -- 172.21.15.89:0/2321507880 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58003dd0 con 0x7faa6006ee80 2023-09-30T22:23:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7fec52ffd700 1 -- 172.21.15.89:0/2492526738 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec5401b730 con 0x7fec641338a0 2023-09-30T22:23:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa668ef700 1 -- 172.21.15.89:0/2321507880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006ee80 msgr2=0x7faa60134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa668ef700 1 --2- 172.21.15.89:0/2321507880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006ee80 0x7faa60134d20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7faa58009e50 tx=0x7faa5800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa668ef700 1 -- 172.21.15.89:0/2321507880 shutdown_connections 2023-09-30T22:23:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa668ef700 1 --2- 172.21.15.89:0/2321507880 >> v2:172.21.15.89:3300/0 conn(0x7faa6006e570 0x7faa6006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa668ef700 1 --2- 172.21.15.89:0/2321507880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006ee80 0x7faa60134d20 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa668ef700 1 -- 172.21.15.89:0/2321507880 >> 172.21.15.89:0/2321507880 conn(0x7faa600a2aa0 msgr2=0x7faa600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa668ef700 1 -- 172.21.15.89:0/2321507880 shutdown_connections 2023-09-30T22:23:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.498+0000 7faa668ef700 1 -- 172.21.15.89:0/2321507880 wait complete. 2023-09-30T22:23:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.500+0000 7faa668ef700 1 Processor -- start 2023-09-30T22:23:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.500+0000 7faa668ef700 1 -- start start 2023-09-30T22:23:43.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa668ef700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006e570 0x7faa60134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa668ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa6006ee80 0x7faa601389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa668ef700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa60138f30 0x7faa601393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa668ef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa601405f0 con 0x7faa6006ee80 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa668ef700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa60135950 con 0x7faa60138f30 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa668ef700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa6013b600 con 0x7faa6006e570 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa64e8c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa60138f30 0x7faa601393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa64e8c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa60138f30 0x7faa601393d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58216/0 (socket says 172.21.15.89:58216) 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa64e8c700 1 -- 172.21.15.89:0/2142634734 learned_addr learned my addr 172.21.15.89:0/2142634734 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa5f7fe700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa6006ee80 0x7faa601389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa5ffff700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006e570 0x7faa60134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa5ffff700 1 -- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa60138f30 msgr2=0x7faa601393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa5ffff700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa60138f30 0x7faa601393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa5ffff700 1 -- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa6006ee80 msgr2=0x7faa601389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa5ffff700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa6006ee80 0x7faa601389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa5ffff700 1 -- 172.21.15.89:0/2142634734 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa58009b30 con 0x7faa6006e570 2023-09-30T22:23:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.501+0000 7faa5ffff700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006e570 0x7faa60134980 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b450 tx=0x7faa5000b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.502+0000 7faa5cff9700 1 -- 172.21.15.89:0/2142634734 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50011830 con 0x7faa6006e570 2023-09-30T22:23:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.502+0000 7faa5cff9700 1 -- 172.21.15.89:0/2142634734 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa50011990 con 0x7faa6006e570 2023-09-30T22:23:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.502+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6009e980 con 0x7faa6006e570 2023-09-30T22:23:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.502+0000 7faa5cff9700 1 -- 172.21.15.89:0/2142634734 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50011d40 con 0x7faa6006e570 2023-09-30T22:23:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.502+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa60139ee0 con 0x7faa6006e570 2023-09-30T22:23:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.503+0000 7faa5cff9700 1 -- 172.21.15.89:0/2142634734 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa5001a400 con 0x7faa6006e570 2023-09-30T22:23:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.503+0000 7faa5cff9700 1 -- 172.21.15.89:0/2142634734 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa5000fdc0 con 0x7faa6006e570 2023-09-30T22:23:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.504+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa60074ea0 con 0x7faa6006e570 2023-09-30T22:23:43.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.508+0000 7faa5cff9700 1 -- 172.21.15.89:0/2142634734 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa50015070 con 0x7faa6006e570 2023-09-30T22:23:43.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.531+0000 7fa660451700 1 Processor -- start 2023-09-30T22:23:43.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.531+0000 7fa660451700 1 -- start start 2023-09-30T22:23:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.532+0000 7fa660451700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa65809e930 0x7fa65809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.532+0000 7fa660451700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f650 0x7fa658131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.532+0000 7fa660451700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa658070940 con 0x7fa65809f240 2023-09-30T22:23:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.532+0000 7fa660451700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65806fd60 con 0x7fa65809e930 2023-09-30T22:23:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.532+0000 7fa660451700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa65806fa80 con 0x7fa65809f650 2023-09-30T22:23:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.532+0000 7fa65e9ee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f650 0x7fa658131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.532+0000 7fa65e9ee700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f650 0x7fa658131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48366/0 (socket says 172.21.15.89:48366) 2023-09-30T22:23:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.532+0000 7fa65e9ee700 1 -- 172.21.15.89:0/4292438795 learned_addr learned my addr 172.21.15.89:0/4292438795 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.533+0000 7fa65e9ee700 1 -- 172.21.15.89:0/4292438795 >> v2:172.21.15.89:3300/0 conn(0x7fa65809e930 msgr2=0x7fa65809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.533+0000 7fa65e9ee700 1 --2- 172.21.15.89:0/4292438795 >> v2:172.21.15.89:3300/0 conn(0x7fa65809e930 0x7fa65809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.533+0000 7fa65e9ee700 1 -- 172.21.15.89:0/4292438795 >> v1:172.21.15.89:6789/0 conn(0x7fa65809f240 legacy=0x7fa6581317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.533+0000 7fa65e9ee700 1 -- 172.21.15.89:0/4292438795 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65809fb60 con 0x7fa65809f650 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.533+0000 7fa65e9ee700 1 --2- 172.21.15.89:0/4292438795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f650 0x7fa658131f00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa650009e50 tx=0x7fa65000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce21e150ec39bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.533+0000 7fa65d1eb700 1 -- 172.21.15.89:0/4292438795 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65000e070 con 0x7fa65809f650 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.533+0000 7fa65d1eb700 1 -- 172.21.15.89:0/4292438795 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa650003a20 con 0x7fa65809f650 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.533+0000 7fa65d1eb700 1 -- 172.21.15.89:0/4292438795 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650003dd0 con 0x7fa65809f650 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.534+0000 7fa660451700 1 -- 172.21.15.89:0/4292438795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f650 msgr2=0x7fa658131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.534+0000 7fa660451700 1 --2- 172.21.15.89:0/4292438795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f650 0x7fa658131f00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa650009e50 tx=0x7fa65000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.534+0000 7fa660451700 1 -- 172.21.15.89:0/4292438795 shutdown_connections 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.534+0000 7fa660451700 1 --2- 172.21.15.89:0/4292438795 >> v2:172.21.15.89:3300/0 conn(0x7fa65809e930 0x7fa65809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.534+0000 7fa660451700 1 --2- 172.21.15.89:0/4292438795 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f650 0x7fa658131f00 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.534+0000 7fa660451700 1 -- 172.21.15.89:0/4292438795 >> 172.21.15.89:0/4292438795 conn(0x7fa6580a4210 msgr2=0x7fa6580a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:43.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.534+0000 7fa660451700 1 -- 172.21.15.89:0/4292438795 shutdown_connections 2023-09-30T22:23:43.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.534+0000 7fa660451700 1 -- 172.21.15.89:0/4292438795 wait complete. 2023-09-30T22:23:43.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.535+0000 7fa660451700 1 Processor -- start 2023-09-30T22:23:43.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.535+0000 7fa660451700 1 -- start start 2023-09-30T22:23:43.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa660451700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa65809e930 0x7fa6581abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa660451700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f240 0x7fa6581ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa660451700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65809f650 0x7fa6581aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa660451700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6581adc40 con 0x7fa65809f650 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa660451700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580a3450 con 0x7fa65809e930 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa660451700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6581aefd0 con 0x7fa65809f240 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65d9ec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f240 0x7fa6581ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65e9ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65809f650 0x7fa6581aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65d9ec700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f240 0x7fa6581ac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48380/0 (socket says 172.21.15.89:48380) 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65e9ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65809f650 0x7fa6581aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45324/0 (socket says 172.21.15.89:45324) 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65d9ec700 1 -- 172.21.15.89:0/1811617017 learned_addr learned my addr 172.21.15.89:0/1811617017 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65e1ed700 1 --2- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa65809e930 0x7fa6581abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65d9ec700 1 -- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa65809e930 msgr2=0x7fa6581abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65d9ec700 1 --2- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa65809e930 0x7fa6581abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65d9ec700 1 -- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65809f650 msgr2=0x7fa6581aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65d9ec700 1 --2- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65809f650 0x7fa6581aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.536+0000 7fa65d9ec700 1 -- 172.21.15.89:0/1811617017 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650009b30 con 0x7fa65809f240 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.537+0000 7fa65d9ec700 1 --2- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f240 0x7fa6581ac4f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c4a0 tx=0x7fa64800c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.537+0000 7fa64effd700 1 -- 172.21.15.89:0/1811617017 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480054e0 con 0x7fa65809f240 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.537+0000 7fa64effd700 1 -- 172.21.15.89:0/1811617017 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa648003680 con 0x7fa65809f240 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.537+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6580a0190 con 0x7fa65809f240 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.537+0000 7fa64effd700 1 -- 172.21.15.89:0/1811617017 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648003a30 con 0x7fa65809f240 2023-09-30T22:23:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.537+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa658071ce0 con 0x7fa65809f240 2023-09-30T22:23:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.538+0000 7fa64effd700 1 -- 172.21.15.89:0/1811617017 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa648003b90 con 0x7fa65809f240 2023-09-30T22:23:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.538+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa658074ea0 con 0x7fa65809f240 2023-09-30T22:23:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.538+0000 7fa64effd700 1 -- 172.21.15.89:0/1811617017 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa64803eba0 con 0x7fa65809f240 2023-09-30T22:23:43.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.543+0000 7fa64effd700 1 -- 172.21.15.89:0/1811617017 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa648003e40 con 0x7fa65809f240 2023-09-30T22:23:43.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.663+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0908074ea0 con 0x7f090809e000 2023-09-30T22:23:43.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.663+0000 7f08fe7fc700 1 -- 172.21.15.89:0/3786781591 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08f8018360 con 0x7f090809e000 2023-09-30T22:23:43.664 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 msgr2=0x7f090811e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f090811e510 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f08f800bee0 tx=0x7f08f8004750 comp rx=0 tx=0).stop 2023-09-30T22:23:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 shutdown_connections 2023-09-30T22:23:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f090809e410 0x7f090811ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f090809d6f0 0x7f090811dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 --2- 172.21.15.89:0/3786781591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f090809e000 0x7f090811e510 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 >> 172.21.15.89:0/3786781591 conn(0x7f0908066f50 msgr2=0x7f090806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:43.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 shutdown_connections 2023-09-30T22:23:43.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.665+0000 7f090ffdb700 1 -- 172.21.15.89:0/3786781591 wait complete. 2023-09-30T22:23:43.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:43.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.696+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fec64074ea0 con 0x7fec641338a0 2023-09-30T22:23:43.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.697+0000 7fec52ffd700 1 -- 172.21.15.89:0/2492526738 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec54017070 con 0x7fec641338a0 2023-09-30T22:23:43.697 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec641338a0 msgr2=0x7fec6411abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec641338a0 0x7fec6411abb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fec54001eb0 tx=0x7fec5400c800 comp rx=0 tx=0).stop 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 shutdown_connections 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fec64139030 0x7fec6411b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fec641338a0 0x7fec6411abb0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 --2- 172.21.15.89:0/2492526738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec64136710 0x7fec6411b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 >> 172.21.15.89:0/2492526738 conn(0x7fec640a2aa0 msgr2=0x7fec64136e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 shutdown_connections 2023-09-30T22:23:43.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.699+0000 7fec68ad6700 1 -- 172.21.15.89:0/2492526738 wait complete. 2023-09-30T22:23:43.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.707+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa6006b9e0 con 0x7faa6006e570 2023-09-30T22:23:43.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.707+0000 7faa5cff9700 1 -- 172.21.15.89:0/2142634734 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa50015030 con 0x7faa6006e570 2023-09-30T22:23:43.708 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006e570 msgr2=0x7faa60134980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006e570 0x7faa60134980 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b450 tx=0x7faa5000b760 comp rx=0 tx=0).stop 2023-09-30T22:23:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 shutdown_connections 2023-09-30T22:23:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faa60138f30 0x7faa601393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faa6006e570 0x7faa60134980 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 --2- 172.21.15.89:0/2142634734 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa6006ee80 0x7faa601389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 >> 172.21.15.89:0/2142634734 conn(0x7faa600a2aa0 msgr2=0x7faa600a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 shutdown_connections 2023-09-30T22:23:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.709+0000 7faa668ef700 1 -- 172.21.15.89:0/2142634734 wait complete. 2023-09-30T22:23:43.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:43.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.739+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa6581bae20 con 0x7fa65809f240 2023-09-30T22:23:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.739+0000 7fa64effd700 1 -- 172.21.15.89:0/1811617017 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa648017070 con 0x7fa65809f240 2023-09-30T22:23:43.740 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f240 msgr2=0x7fa6581ac4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 --2- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f240 0x7fa6581ac4f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c4a0 tx=0x7fa64800c7b0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 shutdown_connections 2023-09-30T22:23:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 --2- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa65809f650 0x7fa6581aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 --2- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa65809e930 0x7fa6581abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 --2- 172.21.15.89:0/1811617017 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa65809f240 0x7fa6581ac4f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 >> 172.21.15.89:0/1811617017 conn(0x7fa6580a4210 msgr2=0x7fa65806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 shutdown_connections 2023-09-30T22:23:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:43.741+0000 7fa660451700 1 -- 172.21.15.89:0/1811617017 wait complete. 2023-09-30T22:23:43.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:44.680 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:44.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:44.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:44.754 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.832+0000 7fd072ce3700 1 Processor -- start 2023-09-30T22:23:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.832+0000 7fd072ce3700 1 -- start start 2023-09-30T22:23:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd072ce3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd06c0a0a30 0x7fd06c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd072ce3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd072ce3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0a2a70 con 0x7fd06c133930 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd072ce3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0a01d0 con 0x7fd06c0a0a30 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd072ce3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c09e2b0 con 0x7fd06c09d630 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd070a7f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd06c0a0a30 0x7fd06c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd070a7f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd06c0a0a30 0x7fd06c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45336/0 (socket says 172.21.15.89:45336) 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58242/0 (socket says 172.21.15.89:58242) 2023-09-30T22:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd070a7f700 1 -- 172.21.15.89:0/3894108538 learned_addr learned my addr 172.21.15.89:0/3894108538 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd06bfff700 1 -- 172.21.15.89:0/3894108538 >> v2:172.21.15.89:3300/0 conn(0x7fd06c0a0a30 msgr2=0x7fd06c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd06bfff700 1 --2- 172.21.15.89:0/3894108538 >> v2:172.21.15.89:3300/0 conn(0x7fd06c0a0a30 0x7fd06c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd06bfff700 1 -- 172.21.15.89:0/3894108538 >> v1:172.21.15.89:6789/0 conn(0x7fd06c133930 legacy=0x7fd06c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd06bfff700 1 -- 172.21.15.89:0/3894108538 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c09dc50 con 0x7fd06c09d630 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.833+0000 7fd070a7f700 1 --2- 172.21.15.89:0/3894108538 >> v2:172.21.15.89:3300/0 conn(0x7fd06c0a0a30 0x7fd06c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.834+0000 7fd06bfff700 1 --2- 172.21.15.89:0/3894108538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c0a1340 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd058009840 tx=0x7fd058009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc2f37cb2f4cf4b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.834+0000 7fd06b7fe700 1 -- 172.21.15.89:0/3894108538 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011830 con 0x7fd06c09d630 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.834+0000 7fd06b7fe700 1 -- 172.21.15.89:0/3894108538 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058011990 con 0x7fd06c09d630 2023-09-30T22:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.834+0000 7fd06b7fe700 1 -- 172.21.15.89:0/3894108538 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011d20 con 0x7fd06c09d630 2023-09-30T22:23:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.834+0000 7fd072ce3700 1 -- 172.21.15.89:0/3894108538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 msgr2=0x7fd06c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.835+0000 7fd072ce3700 1 --2- 172.21.15.89:0/3894108538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c0a1340 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd058009840 tx=0x7fd058009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.835+0000 7fd072ce3700 1 -- 172.21.15.89:0/3894108538 shutdown_connections 2023-09-30T22:23:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.835+0000 7fd072ce3700 1 --2- 172.21.15.89:0/3894108538 >> v2:172.21.15.89:3300/0 conn(0x7fd06c0a0a30 0x7fd06c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.835+0000 7fd072ce3700 1 --2- 172.21.15.89:0/3894108538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c0a1340 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.835+0000 7fd072ce3700 1 -- 172.21.15.89:0/3894108538 >> 172.21.15.89:0/3894108538 conn(0x7fd06c0a3050 msgr2=0x7fd06c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.835+0000 7fd072ce3700 1 -- 172.21.15.89:0/3894108538 shutdown_connections 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.835+0000 7fd072ce3700 1 -- 172.21.15.89:0/3894108538 wait complete. 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.836+0000 7fd072ce3700 1 Processor -- start 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.836+0000 7fd072ce3700 1 -- start start 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd072ce3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c121100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd072ce3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c0a0a30 0x7fd06c121640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd072ce3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd06c133930 0x7fd06c121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd072ce3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c1225b0 con 0x7fd06c0a0a30 2023-09-30T22:23:44.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd072ce3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c124bd0 con 0x7fd06c09d630 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd072ce3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c124630 con 0x7fd06c133930 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd070a7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c121100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c0a0a30 0x7fd06c121640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd070a7f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c121100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58246/0 (socket says 172.21.15.89:58246) 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd070a7f700 1 -- 172.21.15.89:0/2638575668 learned_addr learned my addr 172.21.15.89:0/2638575668 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.837+0000 7fd071280700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd06c133930 0x7fd06c121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd06bfff700 1 -- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd06c133930 msgr2=0x7fd06c121b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd06bfff700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd06c133930 0x7fd06c121b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd06bfff700 1 -- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 msgr2=0x7fd06c121100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd06bfff700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c121100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd06bfff700 1 -- 172.21.15.89:0/2638575668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd060009550 con 0x7fd06c0a0a30 2023-09-30T22:23:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd070a7f700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd06bfff700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c0a0a30 0x7fd06c121640 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fd058016000 tx=0x7fd058004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd0697fa700 1 -- 172.21.15.89:0/2638575668 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580036a0 con 0x7fd06c0a0a30 2023-09-30T22:23:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd0697fa700 1 -- 172.21.15.89:0/2638575668 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05801f070 con 0x7fd06c0a0a30 2023-09-30T22:23:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0580094a0 con 0x7fd06c0a0a30 2023-09-30T22:23:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd0697fa700 1 -- 172.21.15.89:0/2638575668 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058003a60 con 0x7fd06c0a0a30 2023-09-30T22:23:44.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.838+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c0a0410 con 0x7fd06c0a0a30 2023-09-30T22:23:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.839+0000 7fd0697fa700 1 -- 172.21.15.89:0/2638575668 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd058003bc0 con 0x7fd06c0a0a30 2023-09-30T22:23:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.839+0000 7fd0697fa700 1 -- 172.21.15.89:0/2638575668 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd058021070 con 0x7fd06c0a0a30 2023-09-30T22:23:44.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.839+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c11df60 con 0x7fd06c0a0a30 2023-09-30T22:23:44.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.844+0000 7fd0697fa700 1 -- 172.21.15.89:0/2638575668 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd058015030 con 0x7fd06c0a0a30 2023-09-30T22:23:44.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece63a2700 1 Processor -- start 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece63a2700 1 -- start start 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece63a2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece013ac10 0x7fece013cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece63a2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fece013d530 0x7fece013f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece63a2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece009ebc0 con 0x7fece01348a0 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece63a2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece0095cb0 con 0x7fece013d530 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece63a2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece009e980 con 0x7fece013ac10 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece4b9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece013ac10 0x7fece013cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece4b9f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece013ac10 0x7fece013cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58256/0 (socket says 172.21.15.89:58256) 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece4b9f700 1 -- 172.21.15.89:0/1581578194 learned_addr learned my addr 172.21.15.89:0/1581578194 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.860+0000 7fece5ba1700 1 --2- 172.21.15.89:0/1581578194 >> v2:172.21.15.89:3300/0 conn(0x7fece013d530 0x7fece013f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.861+0000 7fece4b9f700 1 -- 172.21.15.89:0/1581578194 >> v2:172.21.15.89:3300/0 conn(0x7fece013d530 msgr2=0x7fece013f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:44.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.861+0000 7fece4b9f700 1 --2- 172.21.15.89:0/1581578194 >> v2:172.21.15.89:3300/0 conn(0x7fece013d530 0x7fece013f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.861+0000 7fece4b9f700 1 -- 172.21.15.89:0/1581578194 >> v1:172.21.15.89:6789/0 conn(0x7fece01348a0 legacy=0x7fece0134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.861+0000 7fece4b9f700 1 -- 172.21.15.89:0/1581578194 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0094ec0 con 0x7fece013ac10 2023-09-30T22:23:44.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.861+0000 7fece4b9f700 1 --2- 172.21.15.89:0/1581578194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece013ac10 0x7fece013cff0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fecd00098f0 tx=0x7fecd0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79901a8061392f95 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:44.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.861+0000 7fecd7fff700 1 -- 172.21.15.89:0/1581578194 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0011830 con 0x7fece013ac10 2023-09-30T22:23:44.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.861+0000 7fecd7fff700 1 -- 172.21.15.89:0/1581578194 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd0011990 con 0x7fece013ac10 2023-09-30T22:23:44.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.861+0000 7fecd7fff700 1 -- 172.21.15.89:0/1581578194 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0011d20 con 0x7fece013ac10 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.862+0000 7fece63a2700 1 -- 172.21.15.89:0/1581578194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece013ac10 msgr2=0x7fece013cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.862+0000 7fece63a2700 1 --2- 172.21.15.89:0/1581578194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece013ac10 0x7fece013cff0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fecd00098f0 tx=0x7fecd0009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.862+0000 7fece63a2700 1 -- 172.21.15.89:0/1581578194 shutdown_connections 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.862+0000 7fece63a2700 1 --2- 172.21.15.89:0/1581578194 >> v2:172.21.15.89:3300/0 conn(0x7fece013d530 0x7fece013f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.862+0000 7fece63a2700 1 --2- 172.21.15.89:0/1581578194 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece013ac10 0x7fece013cff0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.862+0000 7fece63a2700 1 -- 172.21.15.89:0/1581578194 >> 172.21.15.89:0/1581578194 conn(0x7fece00a2540 msgr2=0x7fece00a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.862+0000 7fece63a2700 1 -- 172.21.15.89:0/1581578194 shutdown_connections 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.862+0000 7fece63a2700 1 -- 172.21.15.89:0/1581578194 wait complete. 2023-09-30T22:23:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece63a2700 1 Processor -- start 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece63a2700 1 -- start start 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece63a2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece01348a0 0x7fece01403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece63a2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece013ac10 0x7fece0132a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece63a2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece013d530 0x7fece0132fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece63a2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece009a4d0 con 0x7fece013ac10 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece63a2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece013d000 con 0x7fece01348a0 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece63a2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fece0094080 con 0x7fece013d530 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece4b9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece013ac10 0x7fece0132a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece4b9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece013ac10 0x7fece0132a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45356/0 (socket says 172.21.15.89:45356) 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.863+0000 7fece4b9f700 1 -- 172.21.15.89:0/3312171889 learned_addr learned my addr 172.21.15.89:0/3312171889 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece53a0700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece01348a0 0x7fece01403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece5ba1700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece013d530 0x7fece0132fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece4b9f700 1 -- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece013d530 msgr2=0x7fece0132fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece4b9f700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece013d530 0x7fece0132fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece4b9f700 1 -- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece01348a0 msgr2=0x7fece01403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece4b9f700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece01348a0 0x7fece01403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece4b9f700 1 -- 172.21.15.89:0/3312171889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0009550 con 0x7fece013ac10 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece4b9f700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece013ac10 0x7fece0132a80 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fecd0000c00 tx=0x7fecd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fecd5ffb700 1 -- 172.21.15.89:0/3312171889 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00036a0 con 0x7fece013ac10 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fecd5ffb700 1 -- 172.21.15.89:0/3312171889 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd0003800 con 0x7fece013ac10 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece009b550 con 0x7fece013ac10 2023-09-30T22:23:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fecd5ffb700 1 -- 172.21.15.89:0/3312171889 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0003bb0 con 0x7fece013ac10 2023-09-30T22:23:44.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.864+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece009a850 con 0x7fece013ac10 2023-09-30T22:23:44.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.865+0000 7fecd5ffb700 1 -- 172.21.15.89:0/3312171889 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fecd0021070 con 0x7fece013ac10 2023-09-30T22:23:44.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.866+0000 7fecd5ffb700 1 -- 172.21.15.89:0/3312171889 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fecd0040c20 con 0x7fece013ac10 2023-09-30T22:23:44.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.866+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece009ad40 con 0x7fece013ac10 2023-09-30T22:23:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.875+0000 7fecd5ffb700 1 -- 172.21.15.89:0/3312171889 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fecd0029080 con 0x7fece013ac10 2023-09-30T22:23:44.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f50616e2700 1 Processor -- start 2023-09-30T22:23:44.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f50616e2700 1 -- start start 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f50616e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09d6f0 0x7f505c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f50616e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e000 0x7f505c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f50616e2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f505c09e410 0x7f505c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f50616e2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c06b460 con 0x7f505c09e410 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f50616e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c09b120 con 0x7f505c09d6f0 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f50616e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f505c09a7e0 con 0x7f505c09e000 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f505a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e000 0x7f505c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f505a7fc700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e000 0x7f505c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48406/0 (socket says 172.21.15.89:48406) 2023-09-30T22:23:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.876+0000 7f505a7fc700 1 -- 172.21.15.89:0/1506022465 learned_addr learned my addr 172.21.15.89:0/1506022465 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505a7fc700 1 -- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09d6f0 msgr2=0x7f505c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505affd700 1 --2- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09d6f0 0x7f505c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505a7fc700 1 --2- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09d6f0 0x7f505c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505a7fc700 1 -- 172.21.15.89:0/1506022465 >> v2:172.21.15.89:3300/0 conn(0x7f505c09e410 msgr2=0x7f505c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505b7fe700 1 --2- 172.21.15.89:0/1506022465 >> v2:172.21.15.89:3300/0 conn(0x7f505c09e410 0x7f505c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505a7fc700 1 --2- 172.21.15.89:0/1506022465 >> v2:172.21.15.89:3300/0 conn(0x7f505c09e410 0x7f505c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505a7fc700 1 -- 172.21.15.89:0/1506022465 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c09a500 con 0x7f505c09e000 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505affd700 1 --2- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09d6f0 0x7f505c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505b7fe700 1 --2- 172.21.15.89:0/1506022465 >> v2:172.21.15.89:3300/0 conn(0x7f505c09e410 0x7f505c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f505a7fc700 1 --2- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e000 0x7f505c1339f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5050009920 tx=0x7f5050009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fb5dd2b3ac1a071 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f5059ffb700 1 -- 172.21.15.89:0/1506022465 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000ba20 con 0x7f505c09e000 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f5059ffb700 1 -- 172.21.15.89:0/1506022465 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505000bb80 con 0x7f505c09e000 2023-09-30T22:23:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.877+0000 7f5059ffb700 1 -- 172.21.15.89:0/1506022465 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050009e70 con 0x7f505c09e000 2023-09-30T22:23:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 -- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e000 msgr2=0x7f505c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 --2- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e000 0x7f505c1339f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5050009920 tx=0x7f5050009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 -- 172.21.15.89:0/1506022465 shutdown_connections 2023-09-30T22:23:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 --2- 172.21.15.89:0/1506022465 >> v2:172.21.15.89:3300/0 conn(0x7f505c09e410 0x7f505c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 --2- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09d6f0 0x7f505c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 --2- 172.21.15.89:0/1506022465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e000 0x7f505c1339f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 -- 172.21.15.89:0/1506022465 >> 172.21.15.89:0/1506022465 conn(0x7f505c066f50 msgr2=0x7f505c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 -- 172.21.15.89:0/1506022465 shutdown_connections 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.878+0000 7f50616e2700 1 -- 172.21.15.89:0/1506022465 wait complete. 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.879+0000 7f50616e2700 1 Processor -- start 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.879+0000 7f50616e2700 1 -- start start 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f50616e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c09d6f0 0x7f505c1c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f50616e2700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09e000 0x7f505c1c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f50616e2700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e410 0x7f505c1c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f50616e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c1a85f0 con 0x7f505c09d6f0 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f50616e2700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c1a8a00 con 0x7f505c09e000 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f50616e2700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f505c1a91f0 con 0x7f505c09e410 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e410 0x7f505c1c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09e000 0x7f505c1c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e410 0x7f505c1c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48418/0 (socket says 172.21.15.89:48418) 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505a7fc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09e000 0x7f505c1c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58270/0 (socket says 172.21.15.89:58270) 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 -- 172.21.15.89:0/688931324 learned_addr learned my addr 172.21.15.89:0/688931324 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505affd700 1 --2- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c09d6f0 0x7f505c1c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 -- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09e000 msgr2=0x7f505c1c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 --2- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09e000 0x7f505c1c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 -- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c09d6f0 msgr2=0x7f505c1c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 --2- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c09d6f0 0x7f505c1c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 -- 172.21.15.89:0/688931324 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5050009550 con 0x7f505c09e410 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f505b7fe700 1 --2- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e410 0x7f505c1c49d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f505400ced0 tx=0x7f505400cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.880+0000 7f50437fe700 1 -- 172.21.15.89:0/688931324 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505400ad00 con 0x7f505c09e410 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.881+0000 7f50437fe700 1 -- 172.21.15.89:0/688931324 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505400ae80 con 0x7f505c09e410 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.881+0000 7f50437fe700 1 -- 172.21.15.89:0/688931324 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054003c60 con 0x7f505c09e410 2023-09-30T22:23:44.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.881+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c1a94b0 con 0x7f505c09e410 2023-09-30T22:23:44.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.881+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c1a7a60 con 0x7f505c09e410 2023-09-30T22:23:44.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.882+0000 7f50437fe700 1 -- 172.21.15.89:0/688931324 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5054015440 con 0x7f505c09e410 2023-09-30T22:23:44.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.882+0000 7f50437fe700 1 -- 172.21.15.89:0/688931324 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f505403f800 con 0x7f505c09e410 2023-09-30T22:23:44.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.882+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c1a7a60 con 0x7f505c09e410 2023-09-30T22:23:44.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.891+0000 7f50437fe700 1 -- 172.21.15.89:0/688931324 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f505c1a7a60 con 0x7f505c09e410 2023-09-30T22:23:44.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.905+0000 7fccd5db0700 1 Processor -- start 2023-09-30T22:23:44.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.905+0000 7fccd5db0700 1 -- start start 2023-09-30T22:23:44.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccd5db0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd00a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccd5db0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd00a0f70 0x7fccd00a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccd5db0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fccd013c120 0x7fccd013e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccd5db0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd006d060 con 0x7fccd013c120 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccd5db0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00a2550 con 0x7fccd00a0f70 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccd5db0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00a1e00 con 0x7fccd006cc90 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccceffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd00a0f70 0x7fccd00a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccceffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd00a0f70 0x7fccd00a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58286/0 (socket says 172.21.15.89:58286) 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fccceffd700 1 -- 172.21.15.89:0/446990200 learned_addr learned my addr 172.21.15.89:0/446990200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.906+0000 7fcccffff700 1 --2- 172.21.15.89:0/446990200 >> v2:172.21.15.89:3300/0 conn(0x7fccd013c120 0x7fccd013e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fcccf7fe700 1 --2- 172.21.15.89:0/446990200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd00a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccceffd700 1 -- 172.21.15.89:0/446990200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 msgr2=0x7fccd00a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccceffd700 1 --2- 172.21.15.89:0/446990200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd00a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccceffd700 1 -- 172.21.15.89:0/446990200 >> v2:172.21.15.89:3300/0 conn(0x7fccd013c120 msgr2=0x7fccd013e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccceffd700 1 --2- 172.21.15.89:0/446990200 >> v2:172.21.15.89:3300/0 conn(0x7fccd013c120 0x7fccd013e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccceffd700 1 -- 172.21.15.89:0/446990200 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd009cfa0 con 0x7fccd00a0f70 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccceffd700 1 --2- 172.21.15.89:0/446990200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd00a0f70 0x7fccd00a13e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fccc40098f0 tx=0x7fccc4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a95f6412c5e796c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccce7fc700 1 -- 172.21.15.89:0/446990200 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4011830 con 0x7fccd00a0f70 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccce7fc700 1 -- 172.21.15.89:0/446990200 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc4011990 con 0x7fccd00a0f70 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.907+0000 7fccce7fc700 1 -- 172.21.15.89:0/446990200 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4011d20 con 0x7fccd00a0f70 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 -- 172.21.15.89:0/446990200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd00a0f70 msgr2=0x7fccd00a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 --2- 172.21.15.89:0/446990200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd00a0f70 0x7fccd00a13e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fccc40098f0 tx=0x7fccc4009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 -- 172.21.15.89:0/446990200 shutdown_connections 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 --2- 172.21.15.89:0/446990200 >> v2:172.21.15.89:3300/0 conn(0x7fccd013c120 0x7fccd013e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 --2- 172.21.15.89:0/446990200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd00a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 --2- 172.21.15.89:0/446990200 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd00a0f70 0x7fccd00a13e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 -- 172.21.15.89:0/446990200 >> 172.21.15.89:0/446990200 conn(0x7fccd00a3050 msgr2=0x7fccd00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:44.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 -- 172.21.15.89:0/446990200 shutdown_connections 2023-09-30T22:23:44.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.908+0000 7fccd5db0700 1 -- 172.21.15.89:0/446990200 wait complete. 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.909+0000 7fccd5db0700 1 Processor -- start 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.909+0000 7fccd5db0700 1 -- start start 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.909+0000 7fccd5db0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd013ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.909+0000 7fccd5db0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd00a0f70 0x7fccd013a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.909+0000 7fccd5db0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd013c120 0x7fccd013ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.909+0000 7fccd5db0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd011fbf0 con 0x7fccd00a0f70 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.909+0000 7fccd5db0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd011ea90 con 0x7fccd013c120 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.909+0000 7fccd5db0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccd013bf80 con 0x7fccd006cc90 2023-09-30T22:23:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd013ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd013ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48438/0 (socket says 172.21.15.89:48438) 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 -- 172.21.15.89:0/3340175472 learned_addr learned my addr 172.21.15.89:0/3340175472 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fccceffd700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd00a0f70 0x7fccd013a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccffff700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd013c120 0x7fccd013ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 -- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd013c120 msgr2=0x7fccd013ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd013c120 0x7fccd013ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 -- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd00a0f70 msgr2=0x7fccd013a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd00a0f70 0x7fccd013a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 -- 172.21.15.89:0/3340175472 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4009550 con 0x7fccd006cc90 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fcccf7fe700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd013ba40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e870 tx=0x7fccc000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fccb7fff700 1 -- 172.21.15.89:0/3340175472 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0009590 con 0x7fccd006cc90 2023-09-30T22:23:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fccb7fff700 1 -- 172.21.15.89:0/3340175472 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc0013680 con 0x7fccd006cc90 2023-09-30T22:23:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fccb7fff700 1 -- 172.21.15.89:0/3340175472 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc00139f0 con 0x7fccd006cc90 2023-09-30T22:23:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.910+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd013e340 con 0x7fccd006cc90 2023-09-30T22:23:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.911+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd011fdd0 con 0x7fccd006cc90 2023-09-30T22:23:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.911+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd0071e20 con 0x7fccd006cc90 2023-09-30T22:23:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.912+0000 7fccb7fff700 1 -- 172.21.15.89:0/3340175472 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fccc0013b50 con 0x7fccd006cc90 2023-09-30T22:23:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.912+0000 7fccb7fff700 1 -- 172.21.15.89:0/3340175472 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fccc003fed0 con 0x7fccd006cc90 2023-09-30T22:23:44.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:44.920+0000 7fccb7fff700 1 -- 172.21.15.89:0/3340175472 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fccc0026480 con 0x7fccd006cc90 2023-09-30T22:23:45.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.077+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fece004f2a0 con 0x7fece013ac10 2023-09-30T22:23:45.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.078+0000 7fecd5ffb700 1 -- 172.21.15.89:0/3312171889 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fecd0015020 con 0x7fece013ac10 2023-09-30T22:23:45.078 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:45.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.079+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece013ac10 msgr2=0x7fece0132a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:45.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.079+0000 7fece63a2700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece013ac10 0x7fece0132a80 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fecd0000c00 tx=0x7fecd0004750 comp rx=0 tx=0).stop 2023-09-30T22:23:45.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.080+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 shutdown_connections 2023-09-30T22:23:45.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.080+0000 7fece63a2700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fece013d530 0x7fece0132fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.080+0000 7fece63a2700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fece01348a0 0x7fece01403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.080+0000 7fece63a2700 1 --2- 172.21.15.89:0/3312171889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fece013ac10 0x7fece0132a80 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.080+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 >> 172.21.15.89:0/3312171889 conn(0x7fece00a2540 msgr2=0x7fece013c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:45.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.080+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 shutdown_connections 2023-09-30T22:23:45.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.080+0000 7fece63a2700 1 -- 172.21.15.89:0/3312171889 wait complete. 2023-09-30T22:23:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.084+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd06c0747f0 con 0x7fd06c0a0a30 2023-09-30T22:23:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.085+0000 7fd0697fa700 1 -- 172.21.15.89:0/2638575668 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd058026480 con 0x7fd06c0a0a30 2023-09-30T22:23:45.085 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c0a0a30 msgr2=0x7fd06c121640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c0a0a30 0x7fd06c121640 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fd058016000 tx=0x7fd058004750 comp rx=0 tx=0).stop 2023-09-30T22:23:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 shutdown_connections 2023-09-30T22:23:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd06c133930 0x7fd06c121b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd06c09d630 0x7fd06c121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 --2- 172.21.15.89:0/2638575668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd06c0a0a30 0x7fd06c121640 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 >> 172.21.15.89:0/2638575668 conn(0x7fd06c0a3050 msgr2=0x7fd06c133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 shutdown_connections 2023-09-30T22:23:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.087+0000 7fd072ce3700 1 -- 172.21.15.89:0/2638575668 wait complete. 2023-09-30T22:23:45.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:45.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:45.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.115+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fccd011fa20 con 0x7fccd006cc90 2023-09-30T22:23:45.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.115+0000 7fccb7fff700 1 -- 172.21.15.89:0/3340175472 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fccc0017020 con 0x7fccd006cc90 2023-09-30T22:23:45.116 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:45.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.117+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 msgr2=0x7fccd013ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.117+0000 7fccd5db0700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd013ba40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e870 tx=0x7fccc000eb80 comp rx=0 tx=0).stop 2023-09-30T22:23:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.117+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 shutdown_connections 2023-09-30T22:23:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.117+0000 7fccd5db0700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fccd013c120 0x7fccd013ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.117+0000 7fccd5db0700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fccd006cc90 0x7fccd013ba40 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.117+0000 7fccd5db0700 1 --2- 172.21.15.89:0/3340175472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd00a0f70 0x7fccd013a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.117+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 >> 172.21.15.89:0/3340175472 conn(0x7fccd00a3050 msgr2=0x7fccd009aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.118+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 shutdown_connections 2023-09-30T22:23:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.118+0000 7fccd5db0700 1 -- 172.21.15.89:0/3340175472 wait complete. 2023-09-30T22:23:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.123+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f505c1a7a60 con 0x7f505c09e410 2023-09-30T22:23:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.123+0000 7f50437fe700 1 -- 172.21.15.89:0/688931324 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f505c1a7a60 con 0x7f505c09e410 2023-09-30T22:23:45.124 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e410 msgr2=0x7f505c1c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 --2- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e410 0x7f505c1c49d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f505400ced0 tx=0x7f505400cf00 comp rx=0 tx=0).stop 2023-09-30T22:23:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 shutdown_connections 2023-09-30T22:23:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 --2- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f505c09e410 0x7f505c1c49d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 --2- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f505c09d6f0 0x7f505c1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 --2- 172.21.15.89:0/688931324 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f505c09e000 0x7f505c1c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 >> 172.21.15.89:0/688931324 conn(0x7f505c066f50 msgr2=0x7f505c1babc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 shutdown_connections 2023-09-30T22:23:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:45.126+0000 7f50616e2700 1 -- 172.21.15.89:0/688931324 wait complete. 2023-09-30T22:23:45.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:45.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:46.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:46.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:46.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:46.140 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:46.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.242+0000 7f0db07f4700 1 Processor -- start 2023-09-30T22:23:46.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.242+0000 7f0db07f4700 1 -- start start 2023-09-30T22:23:46.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.242+0000 7f0db07f4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0da81337f0 0x7f0da8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0db07f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da8138dc0 0x7f0da813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0db07f4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da806b990 con 0x7f0da8133420 2023-09-30T22:23:46.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0db07f4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da809b0e0 con 0x7f0da81337f0 2023-09-30T22:23:46.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0db07f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8133c20 con 0x7f0da8138dc0 2023-09-30T22:23:46.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dadd8f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0da81337f0 0x7f0da8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0daed91700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da8138dc0 0x7f0da813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dadd8f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0da81337f0 0x7f0da8138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45418/0 (socket says 172.21.15.89:45418) 2023-09-30T22:23:46.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dadd8f700 1 -- 172.21.15.89:0/350255607 learned_addr learned my addr 172.21.15.89:0/350255607 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:46.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0daed91700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da8138dc0 0x7f0da813b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58296/0 (socket says 172.21.15.89:58296) 2023-09-30T22:23:46.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dadd8f700 1 -- 172.21.15.89:0/350255607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da8138dc0 msgr2=0x7f0da813b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:46.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dadd8f700 1 --2- 172.21.15.89:0/350255607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da8138dc0 0x7f0da813b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dadd8f700 1 -- 172.21.15.89:0/350255607 >> v1:172.21.15.89:6789/0 conn(0x7f0da8133420 legacy=0x7f0da8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dad58e700 1 -- 172.21.15.89:0/350255607 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701180955 0 0) 0x7f0da806b990 con 0x7f0da8133420 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dadd8f700 1 -- 172.21.15.89:0/350255607 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da80a0590 con 0x7f0da81337f0 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.243+0000 7f0dadd8f700 1 --2- 172.21.15.89:0/350255607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da81337f0 0x7f0da8138880 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f0d980098f0 tx=0x7f0d98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b35b376f85c7505 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0dad58e700 1 -- 172.21.15.89:0/350255607 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98011830 con 0x7f0da81337f0 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0dad58e700 1 -- 172.21.15.89:0/350255607 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98011990 con 0x7f0da81337f0 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0dad58e700 1 -- 172.21.15.89:0/350255607 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98011d20 con 0x7f0da81337f0 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0db07f4700 1 -- 172.21.15.89:0/350255607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da81337f0 msgr2=0x7f0da8138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0db07f4700 1 --2- 172.21.15.89:0/350255607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da81337f0 0x7f0da8138880 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f0d980098f0 tx=0x7f0d98009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0db07f4700 1 -- 172.21.15.89:0/350255607 shutdown_connections 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0db07f4700 1 --2- 172.21.15.89:0/350255607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da8138dc0 0x7f0da813b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0db07f4700 1 --2- 172.21.15.89:0/350255607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da81337f0 0x7f0da8138880 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0db07f4700 1 -- 172.21.15.89:0/350255607 >> 172.21.15.89:0/350255607 conn(0x7f0da80a4210 msgr2=0x7f0da80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:46.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0db07f4700 1 -- 172.21.15.89:0/350255607 shutdown_connections 2023-09-30T22:23:46.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.244+0000 7f0db07f4700 1 -- 172.21.15.89:0/350255607 wait complete. 2023-09-30T22:23:46.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.245+0000 7f0db07f4700 1 Processor -- start 2023-09-30T22:23:46.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.245+0000 7f0db07f4700 1 -- start start 2023-09-30T22:23:46.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0db07f4700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0da8133420 0x7f0da812ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0db07f4700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da81337f0 0x7f0da812f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0db07f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da8138dc0 0x7f0da812f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0db07f4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da809c920 con 0x7f0da8138dc0 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0db07f4700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da809e400 con 0x7f0da81337f0 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0db07f4700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0da81389c0 con 0x7f0da8133420 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dae590700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0da8133420 0x7f0da812ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dae590700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0da8133420 0x7f0da812ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48446/0 (socket says 172.21.15.89:48446) 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dae590700 1 -- 172.21.15.89:0/2671942123 learned_addr learned my addr 172.21.15.89:0/2671942123 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dadd8f700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da81337f0 0x7f0da812f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0daed91700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da8138dc0 0x7f0da812f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dae590700 1 -- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da81337f0 msgr2=0x7f0da812f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dae590700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da81337f0 0x7f0da812f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dae590700 1 -- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da8138dc0 msgr2=0x7f0da812f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dae590700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da8138dc0 0x7f0da812f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.246+0000 7f0dae590700 1 -- 172.21.15.89:0/2671942123 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d98009550 con 0x7f0da8133420 2023-09-30T22:23:46.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.247+0000 7f0dae590700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0da8133420 0x7f0da812ee50 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0da4001eb0 tx=0x7f0da400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:46.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.247+0000 7f0d9effd700 1 -- 172.21.15.89:0/2671942123 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40054e0 con 0x7f0da8133420 2023-09-30T22:23:46.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.247+0000 7f0d9effd700 1 -- 172.21.15.89:0/2671942123 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da4003680 con 0x7f0da8133420 2023-09-30T22:23:46.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.247+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da809beb0 con 0x7f0da8133420 2023-09-30T22:23:46.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.247+0000 7f0d9effd700 1 -- 172.21.15.89:0/2671942123 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40039f0 con 0x7f0da8133420 2023-09-30T22:23:46.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.247+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da809d390 con 0x7f0da8133420 2023-09-30T22:23:46.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.248+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da809c590 con 0x7f0da8133420 2023-09-30T22:23:46.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.248+0000 7f0d9effd700 1 -- 172.21.15.89:0/2671942123 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0da4003b50 con 0x7f0da8133420 2023-09-30T22:23:46.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.248+0000 7f0d9effd700 1 -- 172.21.15.89:0/2671942123 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0da403eb20 con 0x7f0da8133420 2023-09-30T22:23:46.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.250+0000 7f4af4a16700 1 Processor -- start 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.250+0000 7f4af4a16700 1 -- start start 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4af4a16700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012cf60 0x7f4af006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4af4a16700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4af012f8b0 0x7f4af012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4af4a16700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af012cd90 con 0x7f4af0130260 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4af4a16700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af012d330 con 0x7f4af012f8b0 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4af4a16700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01306e0 con 0x7f4af012cf60 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4aee59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012cf60 0x7f4af006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4aee59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012cf60 0x7f4af006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48452/0 (socket says 172.21.15.89:48452) 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4aedd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4af012f8b0 0x7f4af012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4aee59c700 1 -- 172.21.15.89:0/148549739 learned_addr learned my addr 172.21.15.89:0/148549739 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:46.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4aee59c700 1 -- 172.21.15.89:0/148549739 >> v2:172.21.15.89:3300/0 conn(0x7f4af012f8b0 msgr2=0x7f4af012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:46.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4aee59c700 1 --2- 172.21.15.89:0/148549739 >> v2:172.21.15.89:3300/0 conn(0x7f4af012f8b0 0x7f4af012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4aee59c700 1 -- 172.21.15.89:0/148549739 >> v1:172.21.15.89:6789/0 conn(0x7f4af0130260 legacy=0x7f4af0136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.251+0000 7f4aee59c700 1 -- 172.21.15.89:0/148549739 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af0071780 con 0x7f4af012cf60 2023-09-30T22:23:46.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.252+0000 7f4aedd9b700 1 --2- 172.21.15.89:0/148549739 >> v2:172.21.15.89:3300/0 conn(0x7f4af012f8b0 0x7f4af012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:46.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.252+0000 7f4aee59c700 1 --2- 172.21.15.89:0/148549739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012cf60 0x7f4af006f5d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00098f0 tx=0x7f4ae0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2a8e41bf1aaebd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:46.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.257+0000 7f4aed59a700 1 -- 172.21.15.89:0/148549739 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011830 con 0x7f4af012cf60 2023-09-30T22:23:46.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.257+0000 7f4aed59a700 1 -- 172.21.15.89:0/148549739 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae0011990 con 0x7f4af012cf60 2023-09-30T22:23:46.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.257+0000 7f4aed59a700 1 -- 172.21.15.89:0/148549739 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011d20 con 0x7f4af012cf60 2023-09-30T22:23:46.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.257+0000 7f0d9effd700 1 -- 172.21.15.89:0/2671942123 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0da401b3b0 con 0x7f0da8133420 2023-09-30T22:23:46.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.258+0000 7f4af4a16700 1 -- 172.21.15.89:0/148549739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012cf60 msgr2=0x7f4af006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.258+0000 7f4af4a16700 1 --2- 172.21.15.89:0/148549739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012cf60 0x7f4af006f5d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00098f0 tx=0x7f4ae0009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:46.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.258+0000 7f4af4a16700 1 -- 172.21.15.89:0/148549739 shutdown_connections 2023-09-30T22:23:46.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.258+0000 7f4af4a16700 1 --2- 172.21.15.89:0/148549739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012cf60 0x7f4af006f5d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.258+0000 7f4af4a16700 1 --2- 172.21.15.89:0/148549739 >> v2:172.21.15.89:3300/0 conn(0x7f4af012f8b0 0x7f4af012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.258+0000 7f4af4a16700 1 -- 172.21.15.89:0/148549739 >> 172.21.15.89:0/148549739 conn(0x7f4af00a2aa0 msgr2=0x7f4af00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:46.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.258+0000 7f4af4a16700 1 -- 172.21.15.89:0/148549739 shutdown_connections 2023-09-30T22:23:46.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.258+0000 7f4af4a16700 1 -- 172.21.15.89:0/148549739 wait complete. 2023-09-30T22:23:46.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.259+0000 7f4af4a16700 1 Processor -- start 2023-09-30T22:23:46.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.259+0000 7f4af4a16700 1 -- start start 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4af4a16700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4af012cf60 0x7f4af0129120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4af4a16700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012f8b0 0x7f4af0129660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4af4a16700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0130260 0x7f4af0138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4af4a16700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af009e850 con 0x7f4af0130260 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4af4a16700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af012d9d0 con 0x7f4af012cf60 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4af4a16700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0129fa0 con 0x7f4af012f8b0 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4aedd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012f8b0 0x7f4af0129660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4aeed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0130260 0x7f4af0138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4aedd9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012f8b0 0x7f4af0129660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48454/0 (socket says 172.21.15.89:48454) 2023-09-30T22:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.260+0000 7f4aedd9b700 1 -- 172.21.15.89:0/3291150843 learned_addr learned my addr 172.21.15.89:0/3291150843 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:46.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4aedd9b700 1 -- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4af012cf60 msgr2=0x7f4af0129120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4aedd9b700 1 --2- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4af012cf60 0x7f4af0129120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4aedd9b700 1 -- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0130260 msgr2=0x7f4af0138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4aedd9b700 1 --2- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0130260 0x7f4af0138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4aedd9b700 1 -- 172.21.15.89:0/3291150843 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae40094a0 con 0x7f4af012f8b0 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4aeed9d700 1 --2- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0130260 0x7f4af0138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4aedd9b700 1 --2- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012f8b0 0x7f4af0129660 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400ebb0 tx=0x7f4ae400eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4adeffd700 1 -- 172.21.15.89:0/3291150843 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400c850 con 0x7f4af012f8b0 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae0009550 con 0x7f4af012f8b0 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4adeffd700 1 -- 172.21.15.89:0/3291150843 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae400c9b0 con 0x7f4af012f8b0 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.261+0000 7f4adeffd700 1 -- 172.21.15.89:0/3291150843 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400cd40 con 0x7f4af012f8b0 2023-09-30T22:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.262+0000 7f4adeffd700 1 -- 172.21.15.89:0/3291150843 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ae4013470 con 0x7f4af012f8b0 2023-09-30T22:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.262+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af009dfa0 con 0x7f4af012f8b0 2023-09-30T22:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.262+0000 7f4adeffd700 1 -- 172.21.15.89:0/3291150843 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4ae4011350 con 0x7f4af012f8b0 2023-09-30T22:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.263+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af0074ea0 con 0x7f4af012f8b0 2023-09-30T22:23:46.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.271+0000 7f4adeffd700 1 -- 172.21.15.89:0/3291150843 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ae4017020 con 0x7f4af012f8b0 2023-09-30T22:23:46.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.284+0000 7fd309765700 1 Processor -- start 2023-09-30T22:23:46.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd309765700 1 -- start start 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd309765700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409e1d0 0x7fd304135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd309765700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd309765700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30409f130 con 0x7fd30409d6f0 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd309765700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30409f520 con 0x7fd30409e1d0 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd309765700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd30409e930 con 0x7fd304135e50 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd3037fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd302ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd30409d6f0 0x7fd30409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49698/0 (socket says 172.21.15.89:49698) 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd302ffd700 1 -- 172.21.15.89:0/2088920687 learned_addr learned my addr 172.21.15.89:0/2088920687 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:46.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.285+0000 7fd3037fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48462/0 (socket says 172.21.15.89:48462) 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd3037fe700 1 -- 172.21.15.89:0/2088920687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409e1d0 msgr2=0x7fd304135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd3037fe700 1 --2- 172.21.15.89:0/2088920687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409e1d0 0x7fd304135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd3037fe700 1 -- 172.21.15.89:0/2088920687 >> v1:172.21.15.89:6789/0 conn(0x7fd30409d6f0 legacy=0x7fd30409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd3037fe700 1 -- 172.21.15.89:0/2088920687 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30409b2d0 con 0x7fd304135e50 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd301ffb700 1 -- 172.21.15.89:0/2088920687 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394645569 0 0) 0x7fd30409f130 con 0x7fd30409d6f0 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd3037fe700 1 --2- 172.21.15.89:0/2088920687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304138230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009dc0 tx=0x7fd2fc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f28a6824ef6516e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd301ffb700 1 -- 172.21.15.89:0/2088920687 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00e070 con 0x7fd304135e50 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd301ffb700 1 -- 172.21.15.89:0/2088920687 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00bbd0 con 0x7fd304135e50 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.286+0000 7fd301ffb700 1 -- 172.21.15.89:0/2088920687 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc012430 con 0x7fd304135e50 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.287+0000 7fd309765700 1 -- 172.21.15.89:0/2088920687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 msgr2=0x7fd304138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.287+0000 7fd309765700 1 --2- 172.21.15.89:0/2088920687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304138230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009dc0 tx=0x7fd2fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.287+0000 7fd309765700 1 -- 172.21.15.89:0/2088920687 shutdown_connections 2023-09-30T22:23:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.287+0000 7fd309765700 1 --2- 172.21.15.89:0/2088920687 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304138230 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.287+0000 7fd309765700 1 --2- 172.21.15.89:0/2088920687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409e1d0 0x7fd304135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.287+0000 7fd309765700 1 -- 172.21.15.89:0/2088920687 >> 172.21.15.89:0/2088920687 conn(0x7fd304066f50 msgr2=0x7fd30409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.287+0000 7fd309765700 1 -- 172.21.15.89:0/2088920687 shutdown_connections 2023-09-30T22:23:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.287+0000 7fd309765700 1 -- 172.21.15.89:0/2088920687 wait complete. 2023-09-30T22:23:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.288+0000 7fd309765700 1 Processor -- start 2023-09-30T22:23:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.288+0000 7fd309765700 1 -- start start 2023-09-30T22:23:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd309765700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409d6f0 0x7fd30411b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd309765700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30409e1d0 0x7fd304133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd309765700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd309765700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30411ce60 con 0x7fd30409e1d0 2023-09-30T22:23:46.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd309765700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040a20b0 con 0x7fd30409d6f0 2023-09-30T22:23:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd309765700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd30409c700 con 0x7fd304135e50 2023-09-30T22:23:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd3037fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd3037fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304134400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48472/0 (socket says 172.21.15.89:48472) 2023-09-30T22:23:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd3027fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30409e1d0 0x7fd304133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd3037fe700 1 -- 172.21.15.89:0/2591740519 learned_addr learned my addr 172.21.15.89:0/2591740519 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.289+0000 7fd302ffd700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409d6f0 0x7fd30411b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd3037fe700 1 -- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409d6f0 msgr2=0x7fd30411b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd3037fe700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409d6f0 0x7fd30411b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd3037fe700 1 -- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30409e1d0 msgr2=0x7fd304133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd3037fe700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30409e1d0 0x7fd304133ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd3037fe700 1 -- 172.21.15.89:0/2591740519 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc009a70 con 0x7fd304135e50 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd3027fc700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30409e1d0 0x7fd304133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd3037fe700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304134400 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc013000 tx=0x7fd2fc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd2eb7fe700 1 -- 172.21.15.89:0/2591740519 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc004880 con 0x7fd304135e50 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd2eb7fe700 1 -- 172.21.15.89:0/2591740519 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc020070 con 0x7fd304135e50 2023-09-30T22:23:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30411c8c0 con 0x7fd304135e50 2023-09-30T22:23:46.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd2eb7fe700 1 -- 172.21.15.89:0/2591740519 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0276a0 con 0x7fd304135e50 2023-09-30T22:23:46.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.290+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30411d5f0 con 0x7fd304135e50 2023-09-30T22:23:46.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.291+0000 7f8e41ef9700 1 Processor -- start 2023-09-30T22:23:46.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.291+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd304074ea0 con 0x7fd304135e50 2023-09-30T22:23:46.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.291+0000 7f8e41ef9700 1 -- start start 2023-09-30T22:23:46.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.292+0000 7f8e41ef9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12cf60 0x7f8e3c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.292+0000 7f8e41ef9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.292+0000 7f8e41ef9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c12cd90 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.292+0000 7f8e41ef9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c12d330 con 0x7f8e3c130430 2023-09-30T22:23:46.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.292+0000 7f8e41ef9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c071780 con 0x7f8e3c12cf60 2023-09-30T22:23:46.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.292+0000 7f8e3affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8e3c12f8b0 0x7f8e3c12fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49710/0 (socket says 172.21.15.89:49710) 2023-09-30T22:23:46.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.292+0000 7f8e3affd700 1 -- 172.21.15.89:0/2219380755 learned_addr learned my addr 172.21.15.89:0/2219380755 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:46.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.293+0000 7f8e3bfff700 1 --2- 172.21.15.89:0/2219380755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.293+0000 7f8e3bfff700 1 -- 172.21.15.89:0/2219380755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12cf60 msgr2=0x7f8e3c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:46.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.293+0000 7f8e3bfff700 1 --2- 172.21.15.89:0/2219380755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12cf60 0x7f8e3c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.293+0000 7f8e3bfff700 1 -- 172.21.15.89:0/2219380755 >> v1:172.21.15.89:6789/0 conn(0x7f8e3c12f8b0 legacy=0x7f8e3c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.293+0000 7f8e3bfff700 1 -- 172.21.15.89:0/2219380755 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c071a40 con 0x7f8e3c130430 2023-09-30T22:23:46.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.293+0000 7f8e3bfff700 1 --2- 172.21.15.89:0/2219380755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c137c30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009dc0 tx=0x7f8e3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfbb1fed6d740b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:46.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.294+0000 7fd2eb7fe700 1 -- 172.21.15.89:0/2591740519 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd2fc031430 con 0x7fd304135e50 2023-09-30T22:23:46.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.294+0000 7fd2eb7fe700 1 -- 172.21.15.89:0/2591740519 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd2fc022070 con 0x7fd304135e50 2023-09-30T22:23:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/2219380755 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400e070 con 0x7f8e3c130430 2023-09-30T22:23:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/2219380755 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3400bad0 con 0x7f8e3c130430 2023-09-30T22:23:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e3a7fc700 1 -- 172.21.15.89:0/2219380755 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400be80 con 0x7f8e3c130430 2023-09-30T22:23:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e41ef9700 1 -- 172.21.15.89:0/2219380755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 msgr2=0x7f8e3c137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e41ef9700 1 --2- 172.21.15.89:0/2219380755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c137c30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009dc0 tx=0x7f8e3400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e41ef9700 1 -- 172.21.15.89:0/2219380755 shutdown_connections 2023-09-30T22:23:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e41ef9700 1 --2- 172.21.15.89:0/2219380755 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c137c30 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e41ef9700 1 --2- 172.21.15.89:0/2219380755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12cf60 0x7f8e3c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.295+0000 7f8e41ef9700 1 -- 172.21.15.89:0/2219380755 >> 172.21.15.89:0/2219380755 conn(0x7f8e3c0a2aa0 msgr2=0x7f8e3c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:46.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.296+0000 7f8e41ef9700 1 -- 172.21.15.89:0/2219380755 shutdown_connections 2023-09-30T22:23:46.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.296+0000 7f8e41ef9700 1 -- 172.21.15.89:0/2219380755 wait complete. 2023-09-30T22:23:46.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.298+0000 7f8e41ef9700 1 Processor -- start 2023-09-30T22:23:46.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.298+0000 7f8e41ef9700 1 -- start start 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.298+0000 7f8e41ef9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c12cf60 0x7f8e3c09ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.298+0000 7f8e41ef9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12f8b0 0x7f8e3c09b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.298+0000 7f8e41ef9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c09b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.298+0000 7f8e41ef9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c09def0 con 0x7f8e3c12cf60 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.298+0000 7f8e41ef9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c09eb50 con 0x7f8e3c130430 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.298+0000 7f8e41ef9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c072660 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c12cf60 0x7f8e3c09ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3affd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12f8b0 0x7f8e3c09b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c12cf60 0x7f8e3c09ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45470/0 (socket says 172.21.15.89:45470) 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3b7fe700 1 -- 172.21.15.89:0/1528273086 learned_addr learned my addr 172.21.15.89:0/1528273086 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3bfff700 1 --2- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c09b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3affd700 1 -- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 msgr2=0x7f8e3c09b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3affd700 1 --2- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c09b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3affd700 1 -- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c12cf60 msgr2=0x7f8e3c09ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3affd700 1 --2- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c12cf60 0x7f8e3c09ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3affd700 1 -- 172.21.15.89:0/1528273086 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e34009a70 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7fd2eb7fe700 1 -- 172.21.15.89:0/2591740519 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd2fc012430 con 0x7fd304135e50 2023-09-30T22:23:46.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.299+0000 7f8e3affd700 1 --2- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12f8b0 0x7f8e3c09b2e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c750 tx=0x7f8e3000ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:46.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.300+0000 7f8e23fff700 1 -- 172.21.15.89:0/1528273086 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30004c80 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.300+0000 7f8e23fff700 1 -- 172.21.15.89:0/1528273086 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e30004de0 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.300+0000 7f8e23fff700 1 -- 172.21.15.89:0/1528273086 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30007310 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.301+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c129fa0 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.301+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c129720 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.301+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c09dd50 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.302+0000 7f8e23fff700 1 -- 172.21.15.89:0/1528273086 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e30007470 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.302+0000 7f8e23fff700 1 -- 172.21.15.89:0/1528273086 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e3003f930 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.310+0000 7f8e23fff700 1 -- 172.21.15.89:0/1528273086 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e3001bba0 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.456+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0da8074ea0 con 0x7f0da8133420 2023-09-30T22:23:46.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.456+0000 7f0d9effd700 1 -- 172.21.15.89:0/2671942123 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0da4017030 con 0x7f0da8133420 2023-09-30T22:23:46.457 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0da8133420 msgr2=0x7f0da812ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0da8133420 0x7f0da812ee50 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0da4001eb0 tx=0x7f0da400c800 comp rx=0 tx=0).stop 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 shutdown_connections 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da8138dc0 0x7f0da812f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0da8133420 0x7f0da812ee50 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 --2- 172.21.15.89:0/2671942123 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0da81337f0 0x7f0da812f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 >> 172.21.15.89:0/2671942123 conn(0x7f0da80a4210 msgr2=0x7f0da80a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 shutdown_connections 2023-09-30T22:23:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.459+0000 7f0db07f4700 1 -- 172.21.15.89:0/2671942123 wait complete. 2023-09-30T22:23:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.465+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4af00713a0 con 0x7f4af012f8b0 2023-09-30T22:23:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.465+0000 7f4adeffd700 1 -- 172.21.15.89:0/3291150843 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ae4044060 con 0x7f4af012f8b0 2023-09-30T22:23:46.466 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.469+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012f8b0 msgr2=0x7f4af0129660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.469+0000 7f4af4a16700 1 --2- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012f8b0 0x7f4af0129660 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400ebb0 tx=0x7f4ae400eec0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.469+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 shutdown_connections 2023-09-30T22:23:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.469+0000 7f4af4a16700 1 --2- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0130260 0x7f4af0138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.469+0000 7f4af4a16700 1 --2- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4af012cf60 0x7f4af0129120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.469+0000 7f4af4a16700 1 --2- 172.21.15.89:0/3291150843 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4af012f8b0 0x7f4af0129660 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.469+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 >> 172.21.15.89:0/3291150843 conn(0x7f4af00a2aa0 msgr2=0x7f4af006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.470+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 shutdown_connections 2023-09-30T22:23:46.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.471+0000 7f4af4a16700 1 -- 172.21.15.89:0/3291150843 wait complete. 2023-09-30T22:23:46.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:46.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:46.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.498+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd30411c6a0 con 0x7fd304135e50 2023-09-30T22:23:46.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.498+0000 7fd2eb7fe700 1 -- 172.21.15.89:0/2591740519 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd2fc02c020 con 0x7fd304135e50 2023-09-30T22:23:46.499 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 msgr2=0x7fd304134400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304134400 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc013000 tx=0x7fd2fc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 shutdown_connections 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd304135e50 0x7fd304134400 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd30409d6f0 0x7fd30411b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 --2- 172.21.15.89:0/2591740519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd30409e1d0 0x7fd304133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 >> 172.21.15.89:0/2591740519 conn(0x7fd304066f50 msgr2=0x7fd304070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 shutdown_connections 2023-09-30T22:23:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.501+0000 7fd309765700 1 -- 172.21.15.89:0/2591740519 wait complete. 2023-09-30T22:23:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.503+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8e3c09dd50 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.503+0000 7f8e23fff700 1 -- 172.21.15.89:0/1528273086 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e3c09dd50 con 0x7f8e3c12f8b0 2023-09-30T22:23:46.504 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12f8b0 msgr2=0x7f8e3c09b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 --2- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12f8b0 0x7f8e3c09b2e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c750 tx=0x7f8e3000ca60 comp rx=0 tx=0).stop 2023-09-30T22:23:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 shutdown_connections 2023-09-30T22:23:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 --2- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8e3c130430 0x7f8e3c09b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 --2- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e3c12cf60 0x7f8e3c09ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 --2- 172.21.15.89:0/1528273086 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e3c12f8b0 0x7f8e3c09b2e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 >> 172.21.15.89:0/1528273086 conn(0x7f8e3c0a2aa0 msgr2=0x7f8e3c06e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 shutdown_connections 2023-09-30T22:23:46.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:46.505+0000 7f8e41ef9700 1 -- 172.21.15.89:0/1528273086 wait complete. 2023-09-30T22:23:46.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:46.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:47.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:47.485 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:47.516 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:47.521 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:47.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.617+0000 7fb9b555f700 1 Processor -- start 2023-09-30T22:23:47.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.617+0000 7fb9b555f700 1 -- start start 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.617+0000 7fb9b555f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012cf60 0x7fb9b006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9b555f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9b0130430 0x7fb9b0137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9b555f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b012cd90 con 0x7fb9b012f8b0 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9b555f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b012d330 con 0x7fb9b0130430 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9b555f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b0071780 con 0x7fb9b012cf60 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9aeffd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012cf60 0x7fb9b006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9ae7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb9b012f8b0 0x7fb9b012fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49712/0 (socket says 172.21.15.89:49712) 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9ae7fc700 1 -- 172.21.15.89:0/15341895 learned_addr learned my addr 172.21.15.89:0/15341895 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9af7fe700 1 --2- 172.21.15.89:0/15341895 >> v2:172.21.15.89:3300/0 conn(0x7fb9b0130430 0x7fb9b0137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9af7fe700 1 -- 172.21.15.89:0/15341895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012cf60 msgr2=0x7fb9b006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:47.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9af7fe700 1 --2- 172.21.15.89:0/15341895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012cf60 0x7fb9b006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9af7fe700 1 -- 172.21.15.89:0/15341895 >> v1:172.21.15.89:6789/0 conn(0x7fb9b012f8b0 legacy=0x7fb9b012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9af7fe700 1 -- 172.21.15.89:0/15341895 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0071a40 con 0x7fb9b0130430 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.618+0000 7fb9af7fe700 1 --2- 172.21.15.89:0/15341895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 0x7fb9b0137c30 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009d90 tx=0x7fb9a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96426a8d6bbb9b13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.619+0000 7fb9adffb700 1 -- 172.21.15.89:0/15341895 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800e070 con 0x7fb9b0130430 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.619+0000 7fb9adffb700 1 -- 172.21.15.89:0/15341895 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a800bb20 con 0x7fb9b0130430 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.619+0000 7fb9adffb700 1 -- 172.21.15.89:0/15341895 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8012430 con 0x7fb9b0130430 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.620+0000 7fb9b555f700 1 -- 172.21.15.89:0/15341895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 msgr2=0x7fb9b0137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.620+0000 7fb9b555f700 1 --2- 172.21.15.89:0/15341895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 0x7fb9b0137c30 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009d90 tx=0x7fb9a800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.620+0000 7fb9b555f700 1 -- 172.21.15.89:0/15341895 shutdown_connections 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.620+0000 7fb9b555f700 1 --2- 172.21.15.89:0/15341895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 0x7fb9b0137c30 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.620+0000 7fb9b555f700 1 --2- 172.21.15.89:0/15341895 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012cf60 0x7fb9b006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.620+0000 7fb9b555f700 1 -- 172.21.15.89:0/15341895 >> 172.21.15.89:0/15341895 conn(0x7fb9b00a2aa0 msgr2=0x7fb9b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.620+0000 7fb9b555f700 1 -- 172.21.15.89:0/15341895 shutdown_connections 2023-09-30T22:23:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.620+0000 7fb9b555f700 1 -- 172.21.15.89:0/15341895 wait complete. 2023-09-30T22:23:47.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.621+0000 7fb9b555f700 1 Processor -- start 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.621+0000 7fb9b555f700 1 -- start start 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.621+0000 7fb9b555f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9b012cf60 0x7fb9b011a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.621+0000 7fb9b555f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012f8b0 0x7fb9b011acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.621+0000 7fb9b555f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 0x7fb9b011b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.621+0000 7fb9b555f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b0129b50 con 0x7fb9b0130430 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.621+0000 7fb9b555f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b011e2a0 con 0x7fb9b012cf60 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.621+0000 7fb9b555f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b011dd00 con 0x7fb9b012f8b0 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9aeffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9b012cf60 0x7fb9b011a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9aeffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9b012cf60 0x7fb9b011a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58368/0 (socket says 172.21.15.89:58368) 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9aeffd700 1 -- 172.21.15.89:0/2010222646 learned_addr learned my addr 172.21.15.89:0/2010222646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9af7fe700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 0x7fb9b011b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9ae7fc700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012f8b0 0x7fb9b011acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9af7fe700 1 -- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012f8b0 msgr2=0x7fb9b011acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9af7fe700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012f8b0 0x7fb9b011acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9af7fe700 1 -- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9b012cf60 msgr2=0x7fb9b011a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9af7fe700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9b012cf60 0x7fb9b011a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9af7fe700 1 -- 172.21.15.89:0/2010222646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8009a70 con 0x7fb9b0130430 2023-09-30T22:23:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9aeffd700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9b012cf60 0x7fb9b011a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.622+0000 7fb9af7fe700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 0x7fb9b011b200 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8016000 tx=0x7fb9a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.623+0000 7fb9977fe700 1 -- 172.21.15.89:0/2010222646 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8012430 con 0x7fb9b0130430 2023-09-30T22:23:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.623+0000 7fb9977fe700 1 -- 172.21.15.89:0/2010222646 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a8020070 con 0x7fb9b0130430 2023-09-30T22:23:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.623+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b012f4b0 con 0x7fb9b0130430 2023-09-30T22:23:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.623+0000 7fb9977fe700 1 -- 172.21.15.89:0/2010222646 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8004280 con 0x7fb9b0130430 2023-09-30T22:23:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.623+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b0129ff0 con 0x7fb9b0130430 2023-09-30T22:23:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.624+0000 7fb9977fe700 1 -- 172.21.15.89:0/2010222646 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb9a80043e0 con 0x7fb9b0130430 2023-09-30T22:23:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.624+0000 7fb9977fe700 1 -- 172.21.15.89:0/2010222646 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb9a8022070 con 0x7fb9b0130430 2023-09-30T22:23:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.624+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b00705f0 con 0x7fb9b0130430 2023-09-30T22:23:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.633+0000 7fb9977fe700 1 -- 172.21.15.89:0/2010222646 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb9a8012e60 con 0x7fb9b0130430 2023-09-30T22:23:47.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321e7e7700 1 Processor -- start 2023-09-30T22:23:47.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321e7e7700 1 -- start start 2023-09-30T22:23:47.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321e7e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809d6f0 0x7f321809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321e7e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809e410 0x7f3218130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321e7e7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321809f130 con 0x7f321809e000 2023-09-30T22:23:47.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321e7e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321809f520 con 0x7f321809d6f0 2023-09-30T22:23:47.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321e7e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f321809e930 con 0x7f321809e410 2023-09-30T22:23:47.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321cd84700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809e410 0x7f3218130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f3217fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809d6f0 0x7f321809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321cd84700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809e410 0x7f3218130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48524/0 (socket says 172.21.15.89:48524) 2023-09-30T22:23:47.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.655+0000 7f321cd84700 1 -- 172.21.15.89:0/2927500163 learned_addr learned my addr 172.21.15.89:0/2927500163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f3217fff700 1 -- 172.21.15.89:0/2927500163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809e410 msgr2=0x7f3218130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f3217fff700 1 --2- 172.21.15.89:0/2927500163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809e410 0x7f3218130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f3217fff700 1 -- 172.21.15.89:0/2927500163 >> v1:172.21.15.89:6789/0 conn(0x7f321809e000 legacy=0x7f3218130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f3217fff700 1 -- 172.21.15.89:0/2927500163 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321809b2d0 con 0x7f321809d6f0 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f321cd84700 1 --2- 172.21.15.89:0/2927500163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809e410 0x7f3218130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f3217fff700 1 --2- 172.21.15.89:0/2927500163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809d6f0 0x7f321809dac0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3208009870 tx=0x7f3208009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0d4da6aa95a82c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f3216ffd700 1 -- 172.21.15.89:0/2927500163 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208010a90 con 0x7f321809d6f0 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f3216ffd700 1 -- 172.21.15.89:0/2927500163 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208010bf0 con 0x7f321809d6f0 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.656+0000 7f3216ffd700 1 -- 172.21.15.89:0/2927500163 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208018440 con 0x7f321809d6f0 2023-09-30T22:23:47.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.657+0000 7f321e7e7700 1 -- 172.21.15.89:0/2927500163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809d6f0 msgr2=0x7f321809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.657+0000 7f321e7e7700 1 --2- 172.21.15.89:0/2927500163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809d6f0 0x7f321809dac0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3208009870 tx=0x7f3208009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:47.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.657+0000 7f321e7e7700 1 -- 172.21.15.89:0/2927500163 shutdown_connections 2023-09-30T22:23:47.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.657+0000 7f321e7e7700 1 --2- 172.21.15.89:0/2927500163 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809d6f0 0x7f321809dac0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.657+0000 7f321e7e7700 1 --2- 172.21.15.89:0/2927500163 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809e410 0x7f3218130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.657+0000 7f321e7e7700 1 -- 172.21.15.89:0/2927500163 >> 172.21.15.89:0/2927500163 conn(0x7f3218066f50 msgr2=0x7f321809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:47.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.657+0000 7f321e7e7700 1 -- 172.21.15.89:0/2927500163 shutdown_connections 2023-09-30T22:23:47.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.657+0000 7f321e7e7700 1 -- 172.21.15.89:0/2927500163 wait complete. 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.658+0000 7f321e7e7700 1 Processor -- start 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.658+0000 7f321e7e7700 1 -- start start 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.658+0000 7f321e7e7700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809d6f0 0x7f321811e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.658+0000 7f321e7e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f321809e000 0x7f321811e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.658+0000 7f321e7e7700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809e410 0x7f321811ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f321e7e7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3218121bc0 con 0x7f321809e000 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f321e7e7700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3218120260 con 0x7f321809e410 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f321e7e7700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32180a1e20 con 0x7f321809d6f0 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f321cd84700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809e410 0x7f321811ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f3217fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809d6f0 0x7f321811e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f321cd84700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809e410 0x7f321811ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58384/0 (socket says 172.21.15.89:58384) 2023-09-30T22:23:47.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f321cd84700 1 -- 172.21.15.89:0/3340512738 learned_addr learned my addr 172.21.15.89:0/3340512738 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:47.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f32177fe700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f321809e000 0x7f321811e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f1e531db700 1 Processor -- start 2023-09-30T22:23:47.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f3217fff700 1 -- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809e410 msgr2=0x7f321811ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f3217fff700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809e410 0x7f321811ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f3217fff700 1 -- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f321809e000 msgr2=0x7f321811e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f3217fff700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f321809e000 0x7f321811e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f3217fff700 1 -- 172.21.15.89:0/3340512738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210009b30 con 0x7f321809d6f0 2023-09-30T22:23:47.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f321cd84700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809e410 0x7f321811ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:47.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f1e531db700 1 -- start start 2023-09-30T22:23:47.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.659+0000 7f3217fff700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809d6f0 0x7f321811e130 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f320801a000 tx=0x7f3208004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:47.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f3214ff9700 1 -- 172.21.15.89:0/3340512738 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208003cf0 con 0x7f321809d6f0 2023-09-30T22:23:47.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f3214ff9700 1 -- 172.21.15.89:0/3340512738 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208003e70 con 0x7f321809d6f0 2023-09-30T22:23:47.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f3214ff9700 1 -- 172.21.15.89:0/3340512738 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208023670 con 0x7f321809d6f0 2023-09-30T22:23:47.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32080094a0 con 0x7f321809d6f0 2023-09-30T22:23:47.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3218121620 con 0x7f321809d6f0 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e531db700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c136710 0x7f1e4c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e531db700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e4c139030 0x7f1e4c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e531db700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c06ef30 con 0x7f1e4c1338a0 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e531db700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4c06fef0 con 0x7f1e4c139030 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e531db700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c1334b0 con 0x7f1e4c136710 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e51778700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e4c139030 0x7f1e4c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e51778700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e4c139030 0x7f1e4c13b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45500/0 (socket says 172.21.15.89:45500) 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e50f77700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1e4c1338a0 0x7f1e4c133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49730/0 (socket says 172.21.15.89:49730) 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e51778700 1 -- 172.21.15.89:0/2929714900 learned_addr learned my addr 172.21.15.89:0/2929714900 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:47.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e4bfff700 1 --2- 172.21.15.89:0/2929714900 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c136710 0x7f1e4c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e51778700 1 -- 172.21.15.89:0/2929714900 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c136710 msgr2=0x7f1e4c138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:47.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e51778700 1 --2- 172.21.15.89:0/2929714900 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c136710 0x7f1e4c138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.660+0000 7f1e51778700 1 -- 172.21.15.89:0/2929714900 >> v1:172.21.15.89:6789/0 conn(0x7f1e4c1338a0 legacy=0x7f1e4c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.661+0000 7f1e51778700 1 -- 172.21.15.89:0/2929714900 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c06fc30 con 0x7f1e4c139030 2023-09-30T22:23:47.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.661+0000 7f1e51778700 1 --2- 172.21.15.89:0/2929714900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c139030 0x7f1e4c13b410 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009e50 tx=0x7f1e4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05c2836a33a4558 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:47.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.661+0000 7f3214ff9700 1 -- 172.21.15.89:0/3340512738 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f320801f070 con 0x7f321809d6f0 2023-09-30T22:23:47.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.661+0000 7f1e4b7fe700 1 -- 172.21.15.89:0/2929714900 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400e070 con 0x7f1e4c139030 2023-09-30T22:23:47.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.661+0000 7f1e4b7fe700 1 -- 172.21.15.89:0/2929714900 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4400b9c0 con 0x7f1e4c139030 2023-09-30T22:23:47.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.661+0000 7f1e4b7fe700 1 -- 172.21.15.89:0/2929714900 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400bd30 con 0x7f1e4c139030 2023-09-30T22:23:47.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.661+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3218120f90 con 0x7f321809d6f0 2023-09-30T22:23:47.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.661+0000 7f3214ff9700 1 -- 172.21.15.89:0/3340512738 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3208046970 con 0x7f321809d6f0 2023-09-30T22:23:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.662+0000 7f1e531db700 1 -- 172.21.15.89:0/2929714900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c139030 msgr2=0x7f1e4c13b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.662+0000 7f1e531db700 1 --2- 172.21.15.89:0/2929714900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c139030 0x7f1e4c13b410 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009e50 tx=0x7f1e4400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.662+0000 7f1e531db700 1 -- 172.21.15.89:0/2929714900 shutdown_connections 2023-09-30T22:23:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.662+0000 7f1e531db700 1 --2- 172.21.15.89:0/2929714900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c139030 0x7f1e4c13b410 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.662+0000 7f1e531db700 1 --2- 172.21.15.89:0/2929714900 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c136710 0x7f1e4c138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.662+0000 7f1e531db700 1 -- 172.21.15.89:0/2929714900 >> 172.21.15.89:0/2929714900 conn(0x7f1e4c0a2aa0 msgr2=0x7f1e4c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.662+0000 7f1e531db700 1 -- 172.21.15.89:0/2929714900 shutdown_connections 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.662+0000 7f1e531db700 1 -- 172.21.15.89:0/2929714900 wait complete. 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.663+0000 7f1e531db700 1 Processor -- start 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.663+0000 7f1e531db700 1 -- start start 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e531db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c1338a0 0x7f1e4c1c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e531db700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e4c136710 0x7f1e4c1c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e531db700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c139030 0x7f1e4c11ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e531db700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c1be1b0 con 0x7f1e4c1338a0 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e531db700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c06f640 con 0x7f1e4c136710 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e531db700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c130bb0 con 0x7f1e4c139030 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c1338a0 0x7f1e4c1c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c1338a0 0x7f1e4c1c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45502/0 (socket says 172.21.15.89:45502) 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 -- 172.21.15.89:0/854776511 learned_addr learned my addr 172.21.15.89:0/854776511 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e4bfff700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e4c136710 0x7f1e4c1c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e51778700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c139030 0x7f1e4c11ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 -- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c139030 msgr2=0x7f1e4c11ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c139030 0x7f1e4c11ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 -- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e4c136710 msgr2=0x7f1e4c1c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e4c136710 0x7f1e4c1c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 -- 172.21.15.89:0/854776511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e44009b30 con 0x7f1e4c1338a0 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e4bfff700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e4c136710 0x7f1e4c1c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e51778700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c139030 0x7f1e4c11ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.664+0000 7f1e50f77700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c1338a0 0x7f1e4c1c83e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000c520 tx=0x7f1e4000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.665+0000 7f1e497fa700 1 -- 172.21.15.89:0/854776511 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40005cb0 con 0x7f1e4c1338a0 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.665+0000 7f1e497fa700 1 -- 172.21.15.89:0/854776511 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e40005e10 con 0x7f1e4c1338a0 2023-09-30T22:23:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.665+0000 7f1e497fa700 1 -- 172.21.15.89:0/854776511 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400072d0 con 0x7f1e4c1338a0 2023-09-30T22:23:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.665+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c11e260 con 0x7f1e4c1338a0 2023-09-30T22:23:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.665+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c11ef90 con 0x7f1e4c1338a0 2023-09-30T22:23:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.665+0000 7f1e497fa700 1 -- 172.21.15.89:0/854776511 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e40007430 con 0x7f1e4c1338a0 2023-09-30T22:23:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.666+0000 7f1e497fa700 1 -- 172.21.15.89:0/854776511 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e4003eb20 con 0x7f1e4c1338a0 2023-09-30T22:23:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.666+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c074ea0 con 0x7f1e4c1338a0 2023-09-30T22:23:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.670+0000 7f3214ff9700 1 -- 172.21.15.89:0/3340512738 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3208027020 con 0x7f321809d6f0 2023-09-30T22:23:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.671+0000 7f1e497fa700 1 -- 172.21.15.89:0/854776511 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e40017070 con 0x7f1e4c1338a0 2023-09-30T22:23:47.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.701+0000 7f8539168700 1 Processor -- start 2023-09-30T22:23:47.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.701+0000 7f8539168700 1 -- start start 2023-09-30T22:23:47.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.701+0000 7f8539168700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8534130540 0x7f8534130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.701+0000 7f8539168700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8534130e50 0x7f853406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.701+0000 7f8539168700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85340716a0 con 0x7f853406fb20 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.701+0000 7f8539168700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853409a390 con 0x7f8534130e50 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.701+0000 7f8539168700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8534071290 con 0x7f8534130540 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8532d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8534130540 0x7f8534130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8532d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8534130540 0x7f8534130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58406/0 (socket says 172.21.15.89:58406) 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8532d9d700 1 -- 172.21.15.89:0/1665529730 learned_addr learned my addr 172.21.15.89:0/1665529730 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8532d9d700 1 -- 172.21.15.89:0/1665529730 >> v2:172.21.15.89:3300/0 conn(0x7f8534130e50 msgr2=0x7f853406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8532d9d700 1 --2- 172.21.15.89:0/1665529730 >> v2:172.21.15.89:3300/0 conn(0x7f8534130e50 0x7f853406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8531d9b700 1 -- 172.21.15.89:0/1665529730 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670105772 0 0) 0x7f85340716a0 con 0x7f853406fb20 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8532d9d700 1 -- 172.21.15.89:0/1665529730 >> v1:172.21.15.89:6789/0 conn(0x7f853406fb20 legacy=0x7f8534137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8532d9d700 1 -- 172.21.15.89:0/1665529730 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8534130340 con 0x7f8534130540 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8532d9d700 1 --2- 172.21.15.89:0/1665529730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8534130540 0x7f8534130910 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8524009920 tx=0x7f8524009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1453522a032194a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8531d9b700 1 -- 172.21.15.89:0/1665529730 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400ba20 con 0x7f8534130540 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8531d9b700 1 -- 172.21.15.89:0/1665529730 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852400bb80 con 0x7f8534130540 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.702+0000 7f8531d9b700 1 -- 172.21.15.89:0/1665529730 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524009e70 con 0x7f8534130540 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.703+0000 7f8539168700 1 -- 172.21.15.89:0/1665529730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8534130540 msgr2=0x7f8534130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.703+0000 7f8539168700 1 --2- 172.21.15.89:0/1665529730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8534130540 0x7f8534130910 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8524009920 tx=0x7f8524009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.703+0000 7f8539168700 1 -- 172.21.15.89:0/1665529730 shutdown_connections 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.703+0000 7f8539168700 1 --2- 172.21.15.89:0/1665529730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8534130540 0x7f8534130910 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.703+0000 7f8539168700 1 --2- 172.21.15.89:0/1665529730 >> v2:172.21.15.89:3300/0 conn(0x7f8534130e50 0x7f853406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.703+0000 7f8539168700 1 -- 172.21.15.89:0/1665529730 >> 172.21.15.89:0/1665529730 conn(0x7f85340a2aa0 msgr2=0x7f85340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.703+0000 7f8539168700 1 -- 172.21.15.89:0/1665529730 shutdown_connections 2023-09-30T22:23:47.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.703+0000 7f8539168700 1 -- 172.21.15.89:0/1665529730 wait complete. 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.704+0000 7f8539168700 1 Processor -- start 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.704+0000 7f8539168700 1 -- start start 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8539168700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f853406fb20 0x7f8534118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8539168700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8534130540 0x7f8534119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8539168700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8534130e50 0x7f8534119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8539168700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341b8fa0 con 0x7f8534130540 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8539168700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853411a2f0 con 0x7f853406fb20 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8539168700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85340718e0 con 0x7f8534130e50 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8532d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f853406fb20 0x7f8534118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8532d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f853406fb20 0x7f8534118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58412/0 (socket says 172.21.15.89:58412) 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f853259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8534130540 0x7f8534119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f8532d9d700 1 -- 172.21.15.89:0/4284746859 learned_addr learned my addr 172.21.15.89:0/4284746859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f853259c700 1 -- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8534130e50 msgr2=0x7f8534119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f853259c700 1 --2- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8534130e50 0x7f8534119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f853259c700 1 -- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f853406fb20 msgr2=0x7f8534118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f853259c700 1 --2- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f853406fb20 0x7f8534118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.705+0000 7f853259c700 1 -- 172.21.15.89:0/4284746859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8524009550 con 0x7f8534130540 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.706+0000 7f853259c700 1 --2- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8534130540 0x7f8534119210 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f852800b450 tx=0x7f852800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.706+0000 7f851b7fe700 1 -- 172.21.15.89:0/4284746859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528011830 con 0x7f8534130540 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.706+0000 7f851b7fe700 1 -- 172.21.15.89:0/4284746859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528011990 con 0x7f8534130540 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.706+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853411b490 con 0x7f8534130540 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.706+0000 7f851b7fe700 1 -- 172.21.15.89:0/4284746859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528011d40 con 0x7f8534130540 2023-09-30T22:23:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.706+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853411c080 con 0x7f8534130540 2023-09-30T22:23:47.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.707+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534074ea0 con 0x7f8534130540 2023-09-30T22:23:47.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.707+0000 7f851b7fe700 1 -- 172.21.15.89:0/4284746859 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f852801a400 con 0x7f8534130540 2023-09-30T22:23:47.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.708+0000 7f851b7fe700 1 -- 172.21.15.89:0/4284746859 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f852803fba0 con 0x7f8534130540 2023-09-30T22:23:47.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.712+0000 7f851b7fe700 1 -- 172.21.15.89:0/4284746859 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f852801ae60 con 0x7f8534130540 2023-09-30T22:23:47.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.865+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3218074ea0 con 0x7f321809d6f0 2023-09-30T22:23:47.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.865+0000 7f3214ff9700 1 -- 172.21.15.89:0/3340512738 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f320804a060 con 0x7f321809d6f0 2023-09-30T22:23:47.866 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.867+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809d6f0 msgr2=0x7f321811e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.867+0000 7f321e7e7700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809d6f0 0x7f321811e130 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f320801a000 tx=0x7f3208004750 comp rx=0 tx=0).stop 2023-09-30T22:23:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.867+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 shutdown_connections 2023-09-30T22:23:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.867+0000 7f321e7e7700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f321809e410 0x7f321811ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.867+0000 7f321e7e7700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f321809d6f0 0x7f321811e130 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.867+0000 7f321e7e7700 1 --2- 172.21.15.89:0/3340512738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f321809e000 0x7f321811e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.867+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 >> 172.21.15.89:0/3340512738 conn(0x7f3218066f50 msgr2=0x7f321806f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.867+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 shutdown_connections 2023-09-30T22:23:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.868+0000 7f321e7e7700 1 -- 172.21.15.89:0/3340512738 wait complete. 2023-09-30T22:23:47.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.895+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb9b0074ea0 con 0x7fb9b0130430 2023-09-30T22:23:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.896+0000 7fb9977fe700 1 -- 172.21.15.89:0/2010222646 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb9a80263e0 con 0x7fb9b0130430 2023-09-30T22:23:47.897 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 msgr2=0x7fb9b011b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 0x7fb9b011b200 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8016000 tx=0x7fb9a8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 shutdown_connections 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b0130430 0x7fb9b011b200 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb9b012cf60 0x7fb9b011a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 --2- 172.21.15.89:0/2010222646 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9b012f8b0 0x7fb9b011acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 >> 172.21.15.89:0/2010222646 conn(0x7fb9b00a2aa0 msgr2=0x7fb9b009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 shutdown_connections 2023-09-30T22:23:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.898+0000 7fb9b555f700 1 -- 172.21.15.89:0/2010222646 wait complete. 2023-09-30T22:23:47.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.907+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f853406b9e0 con 0x7f8534130540 2023-09-30T22:23:47.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.907+0000 7f851b7fe700 1 -- 172.21.15.89:0/4284746859 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8528028030 con 0x7f8534130540 2023-09-30T22:23:47.908 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8534130540 msgr2=0x7f8534119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 --2- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8534130540 0x7f8534119210 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f852800b450 tx=0x7f852800b760 comp rx=0 tx=0).stop 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 shutdown_connections 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 --2- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8534130e50 0x7f8534119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 --2- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f853406fb20 0x7f8534118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 --2- 172.21.15.89:0/4284746859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8534130540 0x7f8534119210 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 >> 172.21.15.89:0/4284746859 conn(0x7f85340a2aa0 msgr2=0x7f853412d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 shutdown_connections 2023-09-30T22:23:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.910+0000 7f8539168700 1 -- 172.21.15.89:0/4284746859 wait complete. 2023-09-30T22:23:47.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:47.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:47.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.939+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1e4c11e760 con 0x7f1e4c1338a0 2023-09-30T22:23:47.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.939+0000 7f1e497fa700 1 -- 172.21.15.89:0/854776511 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e4001b3b0 con 0x7f1e4c1338a0 2023-09-30T22:23:47.940 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c1338a0 msgr2=0x7f1e4c1c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c1338a0 0x7f1e4c1c83e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000c520 tx=0x7f1e4000c830 comp rx=0 tx=0).stop 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 shutdown_connections 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e4c139030 0x7f1e4c11ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c1338a0 0x7f1e4c1c83e0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 --2- 172.21.15.89:0/854776511 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e4c136710 0x7f1e4c1c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 >> 172.21.15.89:0/854776511 conn(0x7f1e4c0a2aa0 msgr2=0x7f1e4c1344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 shutdown_connections 2023-09-30T22:23:47.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:47.941+0000 7f1e531db700 1 -- 172.21.15.89:0/854776511 wait complete. 2023-09-30T22:23:47.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:48.885 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:48.913 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:48.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:48.954 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.025+0000 7f81f0819700 1 Processor -- start 2023-09-30T22:23:49.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.025+0000 7f81f0819700 1 -- start start 2023-09-30T22:23:49.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81f0819700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec09e1d0 0x7f81ec135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81f0819700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec135e50 0x7f81ec138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81f0819700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec09f130 con 0x7f81ec09d6f0 2023-09-30T22:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81f0819700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec09f520 con 0x7f81ec09e1d0 2023-09-30T22:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81f0819700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec09e930 con 0x7f81ec135e50 2023-09-30T22:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec09e1d0 0x7f81ec135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e9d9b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec09e1d0 0x7f81ec135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58426/0 (socket says 172.21.15.89:58426) 2023-09-30T22:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e9d9b700 1 -- 172.21.15.89:0/3296005297 learned_addr learned my addr 172.21.15.89:0/3296005297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81ead9d700 1 --2- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec135e50 0x7f81ec138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e9d9b700 1 -- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec135e50 msgr2=0x7f81ec138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e9d9b700 1 --2- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec135e50 0x7f81ec138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e9d9b700 1 -- 172.21.15.89:0/3296005297 >> v1:172.21.15.89:6789/0 conn(0x7f81ec09d6f0 legacy=0x7f81ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e959a700 1 -- 172.21.15.89:0/3296005297 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036089034 0 0) 0x7f81ec09f130 con 0x7f81ec09d6f0 2023-09-30T22:23:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e9d9b700 1 -- 172.21.15.89:0/3296005297 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec09b2d0 con 0x7f81ec09e1d0 2023-09-30T22:23:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81ead9d700 1 --2- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec135e50 0x7f81ec138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e9d9b700 1 --2- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec09e1d0 0x7f81ec135910 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009920 tx=0x7f81e0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f69af2e8131abe27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.026+0000 7f81e959a700 1 -- 172.21.15.89:0/3296005297 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0011830 con 0x7f81ec09e1d0 2023-09-30T22:23:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.027+0000 7f81e959a700 1 -- 172.21.15.89:0/3296005297 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e0011990 con 0x7f81ec09e1d0 2023-09-30T22:23:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.027+0000 7f81e959a700 1 -- 172.21.15.89:0/3296005297 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0011d20 con 0x7f81ec09e1d0 2023-09-30T22:23:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.027+0000 7f81f0819700 1 -- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec09e1d0 msgr2=0x7f81ec135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.027+0000 7f81f0819700 1 --2- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec09e1d0 0x7f81ec135910 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009920 tx=0x7f81e0009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.028+0000 7f81f0819700 1 -- 172.21.15.89:0/3296005297 shutdown_connections 2023-09-30T22:23:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.028+0000 7f81f0819700 1 --2- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec135e50 0x7f81ec138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.028+0000 7f81f0819700 1 --2- 172.21.15.89:0/3296005297 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec09e1d0 0x7f81ec135910 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.028+0000 7f81f0819700 1 -- 172.21.15.89:0/3296005297 >> 172.21.15.89:0/3296005297 conn(0x7f81ec066f50 msgr2=0x7f81ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.028+0000 7f81f0819700 1 -- 172.21.15.89:0/3296005297 shutdown_connections 2023-09-30T22:23:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.028+0000 7f81f0819700 1 -- 172.21.15.89:0/3296005297 wait complete. 2023-09-30T22:23:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.029+0000 7f81f0819700 1 Processor -- start 2023-09-30T22:23:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.029+0000 7f81f0819700 1 -- start start 2023-09-30T22:23:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.029+0000 7f81f0819700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec09d6f0 0x7f81ec1327a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.029+0000 7f81f0819700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec09e1d0 0x7f81ec132ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.029+0000 7f81f0819700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec135e50 0x7f81ec133220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.029+0000 7f81f0819700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec119900 con 0x7f81ec09e1d0 2023-09-30T22:23:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.029+0000 7f81f0819700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec09c700 con 0x7f81ec135e50 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.029+0000 7f81f0819700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0a21d0 con 0x7f81ec09d6f0 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81ea59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec09d6f0 0x7f81ec1327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81ea59c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec09d6f0 0x7f81ec1327a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48566/0 (socket says 172.21.15.89:48566) 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81ea59c700 1 -- 172.21.15.89:0/181596887 learned_addr learned my addr 172.21.15.89:0/181596887 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81e9d9b700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec09e1d0 0x7f81ec132ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81ead9d700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec135e50 0x7f81ec133220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81e9d9b700 1 -- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec09d6f0 msgr2=0x7f81ec1327a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81e9d9b700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec09d6f0 0x7f81ec1327a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81e9d9b700 1 -- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec135e50 msgr2=0x7f81ec133220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81e9d9b700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec135e50 0x7f81ec133220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81e9d9b700 1 -- 172.21.15.89:0/181596887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e4009a70 con 0x7f81ec09e1d0 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81ea59c700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec09d6f0 0x7f81ec1327a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81e9d9b700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec09e1d0 0x7f81ec132ce0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f81e0016000 tx=0x7f81e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81daffd700 1 -- 172.21.15.89:0/181596887 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0004510 con 0x7f81ec09e1d0 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81daffd700 1 -- 172.21.15.89:0/181596887 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e001f070 con 0x7f81ec09e1d0 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81daffd700 1 -- 172.21.15.89:0/181596887 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e00038c0 con 0x7f81ec09e1d0 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.030+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e0009550 con 0x7f81ec09e1d0 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.031+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec135a50 con 0x7f81ec09e1d0 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.032+0000 7f81daffd700 1 -- 172.21.15.89:0/181596887 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81e0003a20 con 0x7f81ec09e1d0 2023-09-30T22:23:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.032+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec071700 con 0x7f81ec09e1d0 2023-09-30T22:23:49.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.032+0000 7f81daffd700 1 -- 172.21.15.89:0/181596887 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f81e0021070 con 0x7f81ec09e1d0 2023-09-30T22:23:49.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.041+0000 7f81daffd700 1 -- 172.21.15.89:0/181596887 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81e001a430 con 0x7f81ec09e1d0 2023-09-30T22:23:49.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.066+0000 7f03020e3700 1 Processor -- start 2023-09-30T22:23:49.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.066+0000 7f03020e3700 1 -- start start 2023-09-30T22:23:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f03020e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09d6f0 0x7f02fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f03020e3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e000 0x7f02fc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f03020e3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f02fc09e410 0x7f02fc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f03020e3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02fc06b460 con 0x7f02fc09e410 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f03020e3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc09b120 con 0x7f02fc09e000 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f03020e3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc09a7e0 con 0x7f02fc09d6f0 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02faffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e000 0x7f02fc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02faffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e000 0x7f02fc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:58432/0 (socket says 172.21.15.89:58432) 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02faffd700 1 -- 172.21.15.89:0/591624402 learned_addr learned my addr 172.21.15.89:0/591624402 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02fb7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09d6f0 0x7f02fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02fbfff700 1 --2- 172.21.15.89:0/591624402 >> v2:172.21.15.89:3300/0 conn(0x7f02fc09e410 0x7f02fc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02fb7fe700 1 -- 172.21.15.89:0/591624402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e000 msgr2=0x7f02fc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02fb7fe700 1 --2- 172.21.15.89:0/591624402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e000 0x7f02fc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02fb7fe700 1 -- 172.21.15.89:0/591624402 >> v2:172.21.15.89:3300/0 conn(0x7f02fc09e410 msgr2=0x7f02fc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02fb7fe700 1 --2- 172.21.15.89:0/591624402 >> v2:172.21.15.89:3300/0 conn(0x7f02fc09e410 0x7f02fc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02fb7fe700 1 -- 172.21.15.89:0/591624402 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc09a500 con 0x7f02fc09d6f0 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02faffd700 1 --2- 172.21.15.89:0/591624402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e000 0x7f02fc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.067+0000 7f02fb7fe700 1 --2- 172.21.15.89:0/591624402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09d6f0 0x7f02fc09dac0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009870 tx=0x7f02ec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b78d46e85a2b3be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f02fa7fc700 1 -- 172.21.15.89:0/591624402 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec011830 con 0x7f02fc09d6f0 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f02fa7fc700 1 -- 172.21.15.89:0/591624402 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec011990 con 0x7f02fc09d6f0 2023-09-30T22:23:49.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f02fa7fc700 1 -- 172.21.15.89:0/591624402 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec011d20 con 0x7f02fc09d6f0 2023-09-30T22:23:49.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 -- 172.21.15.89:0/591624402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09d6f0 msgr2=0x7f02fc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 --2- 172.21.15.89:0/591624402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09d6f0 0x7f02fc09dac0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009870 tx=0x7f02ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:49.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 -- 172.21.15.89:0/591624402 shutdown_connections 2023-09-30T22:23:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 --2- 172.21.15.89:0/591624402 >> v2:172.21.15.89:3300/0 conn(0x7f02fc09e410 0x7f02fc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 --2- 172.21.15.89:0/591624402 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09d6f0 0x7f02fc09dac0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 --2- 172.21.15.89:0/591624402 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e000 0x7f02fc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 -- 172.21.15.89:0/591624402 >> 172.21.15.89:0/591624402 conn(0x7f02fc066f50 msgr2=0x7f02fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 -- 172.21.15.89:0/591624402 shutdown_connections 2023-09-30T22:23:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.068+0000 7f03020e3700 1 -- 172.21.15.89:0/591624402 wait complete. 2023-09-30T22:23:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.069+0000 7f03020e3700 1 Processor -- start 2023-09-30T22:23:49.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.069+0000 7f03020e3700 1 -- start start 2023-09-30T22:23:49.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f03020e3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02fc09d6f0 0x7f02fc1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f03020e3700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09e000 0x7f02fc1a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f03020e3700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e410 0x7f02fc1a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f03020e3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1aa270 con 0x7f02fc09d6f0 2023-09-30T22:23:49.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f03020e3700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1a79a0 con 0x7f02fc09e410 2023-09-30T22:23:49.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f03020e3700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1a7fc0 con 0x7f02fc09e000 2023-09-30T22:23:49.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02fc09d6f0 0x7f02fc1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02fc09d6f0 0x7f02fc1a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45546/0 (socket says 172.21.15.89:45546) 2023-09-30T22:23:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fb7fe700 1 -- 172.21.15.89:0/3344864764 learned_addr learned my addr 172.21.15.89:0/3344864764 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02faffd700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09e000 0x7f02fc1a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fbfff700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e410 0x7f02fc1a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fb7fe700 1 -- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09e000 msgr2=0x7f02fc1a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fb7fe700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09e000 0x7f02fc1a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fb7fe700 1 -- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e410 msgr2=0x7f02fc1a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fb7fe700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e410 0x7f02fc1a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02fb7fe700 1 -- 172.21.15.89:0/3344864764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009550 con 0x7f02fc09d6f0 2023-09-30T22:23:49.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.070+0000 7f02faffd700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09e000 0x7f02fc1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:49.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.071+0000 7f02fb7fe700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02fc09d6f0 0x7f02fc1a5de0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009ed0 tx=0x7f02ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.073+0000 7f02e3fff700 1 -- 172.21.15.89:0/3344864764 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec004450 con 0x7f02fc09d6f0 2023-09-30T22:23:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.073+0000 7f02e3fff700 1 -- 172.21.15.89:0/3344864764 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec01f070 con 0x7f02fc09d6f0 2023-09-30T22:23:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.073+0000 7f02e3fff700 1 -- 172.21.15.89:0/3344864764 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec0038e0 con 0x7f02fc09d6f0 2023-09-30T22:23:49.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.074+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec0094a0 con 0x7f02fc09d6f0 2023-09-30T22:23:49.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.074+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc0a3130 con 0x7f02fc09d6f0 2023-09-30T22:23:49.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.074+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02fc09c770 con 0x7f02fc09d6f0 2023-09-30T22:23:49.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.075+0000 7f02e3fff700 1 -- 172.21.15.89:0/3344864764 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02ec003a40 con 0x7f02fc09d6f0 2023-09-30T22:23:49.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.075+0000 7f02e3fff700 1 -- 172.21.15.89:0/3344864764 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f02ec021070 con 0x7f02fc09d6f0 2023-09-30T22:23:49.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.079+0000 7f02e3fff700 1 -- 172.21.15.89:0/3344864764 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02ec010380 con 0x7f02fc09d6f0 2023-09-30T22:23:49.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.081+0000 7f61b8f4a700 1 Processor -- start 2023-09-30T22:23:49.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.081+0000 7f61b8f4a700 1 -- start start 2023-09-30T22:23:49.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b8f4a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61b412cf60 0x7f61b406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b8f4a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412f8b0 0x7f61b412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b8f4a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b412cd90 con 0x7f61b4130260 2023-09-30T22:23:49.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b8f4a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b412d330 con 0x7f61b412f8b0 2023-09-30T22:23:49.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b8f4a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61b41306d0 con 0x7f61b412cf60 2023-09-30T22:23:49.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b2d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61b4130260 0x7f61b4136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49760/0 (socket says 172.21.15.89:49760) 2023-09-30T22:23:49.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b2d9d700 1 -- 172.21.15.89:0/3449487538 learned_addr learned my addr 172.21.15.89:0/3449487538 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:49.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b1d9b700 1 --2- 172.21.15.89:0/3449487538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412f8b0 0x7f61b412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b159a700 1 -- 172.21.15.89:0/3449487538 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020067604 0 0) 0x7f61b412cd90 con 0x7f61b4130260 2023-09-30T22:23:49.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.082+0000 7f61b159a700 1 -- 172.21.15.89:0/3449487538 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f619c0035b0 con 0x7f61b4130260 2023-09-30T22:23:49.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.083+0000 7f61b1d9b700 1 -- 172.21.15.89:0/3449487538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61b412cf60 msgr2=0x7f61b406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.083+0000 7f61b1d9b700 1 --2- 172.21.15.89:0/3449487538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61b412cf60 0x7f61b406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.083+0000 7f61b1d9b700 1 -- 172.21.15.89:0/3449487538 >> v1:172.21.15.89:6789/0 conn(0x7f61b4130260 legacy=0x7f61b4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.083+0000 7f61b1d9b700 1 -- 172.21.15.89:0/3449487538 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4071780 con 0x7f61b412f8b0 2023-09-30T22:23:49.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.083+0000 7f61b1d9b700 1 --2- 172.21.15.89:0/3449487538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412f8b0 0x7f61b412fd20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009870 tx=0x7f61a8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea35d1b53de1ffb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.083+0000 7f61b159a700 1 -- 172.21.15.89:0/3449487538 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8011830 con 0x7f61b412f8b0 2023-09-30T22:23:49.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.083+0000 7f61b159a700 1 -- 172.21.15.89:0/3449487538 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a8011990 con 0x7f61b412f8b0 2023-09-30T22:23:49.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.083+0000 7f61b159a700 1 -- 172.21.15.89:0/3449487538 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8011d20 con 0x7f61b412f8b0 2023-09-30T22:23:49.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.084+0000 7f61b8f4a700 1 -- 172.21.15.89:0/3449487538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412f8b0 msgr2=0x7f61b412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.084+0000 7f61b8f4a700 1 --2- 172.21.15.89:0/3449487538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412f8b0 0x7f61b412fd20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009870 tx=0x7f61a8009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:49.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.084+0000 7f61b8f4a700 1 -- 172.21.15.89:0/3449487538 shutdown_connections 2023-09-30T22:23:49.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.084+0000 7f61b8f4a700 1 --2- 172.21.15.89:0/3449487538 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61b412cf60 0x7f61b406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.084+0000 7f61b8f4a700 1 --2- 172.21.15.89:0/3449487538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412f8b0 0x7f61b412fd20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.084+0000 7f61b8f4a700 1 -- 172.21.15.89:0/3449487538 >> 172.21.15.89:0/3449487538 conn(0x7f61b40a2aa0 msgr2=0x7f61b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.084+0000 7f61b8f4a700 1 -- 172.21.15.89:0/3449487538 shutdown_connections 2023-09-30T22:23:49.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.084+0000 7f61b8f4a700 1 -- 172.21.15.89:0/3449487538 wait complete. 2023-09-30T22:23:49.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.086+0000 7f61b8f4a700 1 Processor -- start 2023-09-30T22:23:49.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.086+0000 7f61b8f4a700 1 -- start start 2023-09-30T22:23:49.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b8f4a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412cf60 0x7f61b41309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b8f4a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b412f8b0 0x7f61b4130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b8f4a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61b4130260 0x7f61b4131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b8f4a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4133d30 con 0x7f61b412f8b0 2023-09-30T22:23:49.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b8f4a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4129fa0 con 0x7f61b412cf60 2023-09-30T22:23:49.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b8f4a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4133070 con 0x7f61b4130260 2023-09-30T22:23:49.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b412f8b0 0x7f61b4130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412cf60 0x7f61b41309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b412f8b0 0x7f61b4130f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45562/0 (socket says 172.21.15.89:45562) 2023-09-30T22:23:49.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 -- 172.21.15.89:0/2773316125 learned_addr learned my addr 172.21.15.89:0/2773316125 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:49.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 -- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61b4130260 msgr2=0x7f61b4131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 --2- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61b4130260 0x7f61b4131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 -- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412cf60 msgr2=0x7f61b41309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 --2- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412cf60 0x7f61b41309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 -- 172.21.15.89:0/2773316125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a80094a0 con 0x7f61b412f8b0 2023-09-30T22:23:49.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b259c700 1 --2- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412cf60 0x7f61b41309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:49.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61b1d9b700 1 --2- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b412f8b0 0x7f61b4130f20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f61a8013000 tx=0x7f61a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.087+0000 7f61a2ffd700 1 -- 172.21.15.89:0/2773316125 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8004380 con 0x7f61b412f8b0 2023-09-30T22:23:49.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.088+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b4132ac0 con 0x7f61b412f8b0 2023-09-30T22:23:49.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.088+0000 7f61a2ffd700 1 -- 172.21.15.89:0/2773316125 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a801f070 con 0x7f61b412f8b0 2023-09-30T22:23:49.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.088+0000 7f61a2ffd700 1 -- 172.21.15.89:0/2773316125 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80036a0 con 0x7f61b412f8b0 2023-09-30T22:23:49.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.088+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b41392c0 con 0x7f61b412f8b0 2023-09-30T22:23:49.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.089+0000 7f61a2ffd700 1 -- 172.21.15.89:0/2773316125 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f61a8021030 con 0x7f61b412f8b0 2023-09-30T22:23:49.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.089+0000 7f61a2ffd700 1 -- 172.21.15.89:0/2773316125 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f61a80409e0 con 0x7f61b412f8b0 2023-09-30T22:23:49.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.089+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b406d4a0 con 0x7f61b412f8b0 2023-09-30T22:23:49.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.098+0000 7f61a2ffd700 1 -- 172.21.15.89:0/2773316125 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f61a8015080 con 0x7f61b412f8b0 2023-09-30T22:23:49.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.104+0000 7fa52aaa0700 1 Processor -- start 2023-09-30T22:23:49.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.104+0000 7fa52aaa0700 1 -- start start 2023-09-30T22:23:49.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52aaa0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa524133420 0x7fa5241337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52aaa0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133e30 0x7fa52412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52aaa0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524134260 0x7fa5241304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52aaa0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52406b990 con 0x7fa524133420 2023-09-30T22:23:49.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52aaa0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5240a0590 con 0x7fa524134260 2023-09-30T22:23:49.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52aaa0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa52409fb80 con 0x7fa524133e30 2023-09-30T22:23:49.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa523fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133e30 0x7fa52412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa523fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133e30 0x7fa52412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48608/0 (socket says 172.21.15.89:48608) 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa523fff700 1 -- 172.21.15.89:0/2084542471 learned_addr learned my addr 172.21.15.89:0/2084542471 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52903d700 1 --2- 172.21.15.89:0/2084542471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524134260 0x7fa5241304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa523fff700 1 -- 172.21.15.89:0/2084542471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524134260 msgr2=0x7fa5241304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa523fff700 1 --2- 172.21.15.89:0/2084542471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524134260 0x7fa5241304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52883c700 1 --2- 172.21.15.89:0/2084542471 >> v2:172.21.15.89:3300/0 conn(0x7fa524133420 0x7fa5241337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa523fff700 1 -- 172.21.15.89:0/2084542471 >> v2:172.21.15.89:3300/0 conn(0x7fa524133420 msgr2=0x7fa5241337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa523fff700 1 --2- 172.21.15.89:0/2084542471 >> v2:172.21.15.89:3300/0 conn(0x7fa524133420 0x7fa5241337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa523fff700 1 -- 172.21.15.89:0/2084542471 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5240a0030 con 0x7fa524133e30 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.105+0000 7fa52883c700 1 --2- 172.21.15.89:0/2084542471 >> v2:172.21.15.89:3300/0 conn(0x7fa524133420 0x7fa5241337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:49.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.106+0000 7fa523fff700 1 --2- 172.21.15.89:0/2084542471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133e30 0x7fa52412ff90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa5100098f0 tx=0x7fa510009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f7ec2dc185ca234 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.106+0000 7fa5237fe700 1 -- 172.21.15.89:0/2084542471 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510011830 con 0x7fa524133e30 2023-09-30T22:23:49.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.106+0000 7fa5237fe700 1 -- 172.21.15.89:0/2084542471 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa510011990 con 0x7fa524133e30 2023-09-30T22:23:49.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.106+0000 7fa5237fe700 1 -- 172.21.15.89:0/2084542471 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510011d20 con 0x7fa524133e30 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.106+0000 7fa52aaa0700 1 -- 172.21.15.89:0/2084542471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133e30 msgr2=0x7fa52412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.106+0000 7fa52aaa0700 1 --2- 172.21.15.89:0/2084542471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133e30 0x7fa52412ff90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa5100098f0 tx=0x7fa510009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.107+0000 7fa52aaa0700 1 -- 172.21.15.89:0/2084542471 shutdown_connections 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.107+0000 7fa52aaa0700 1 --2- 172.21.15.89:0/2084542471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524134260 0x7fa5241304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.107+0000 7fa52aaa0700 1 --2- 172.21.15.89:0/2084542471 >> v2:172.21.15.89:3300/0 conn(0x7fa524133420 0x7fa5241337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.107+0000 7fa52aaa0700 1 --2- 172.21.15.89:0/2084542471 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133e30 0x7fa52412ff90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.107+0000 7fa52aaa0700 1 -- 172.21.15.89:0/2084542471 >> 172.21.15.89:0/2084542471 conn(0x7fa5240a4210 msgr2=0x7fa5240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.107+0000 7fa52aaa0700 1 -- 172.21.15.89:0/2084542471 shutdown_connections 2023-09-30T22:23:49.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.107+0000 7fa52aaa0700 1 -- 172.21.15.89:0/2084542471 wait complete. 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.107+0000 7fa52aaa0700 1 Processor -- start 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52aaa0700 1 -- start start 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52aaa0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133420 0x7fa52409b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52aaa0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524133e30 0x7fa52409bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52aaa0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa524134260 0x7fa52409c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52aaa0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52409d0c0 con 0x7fa524134260 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52aaa0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52409ca80 con 0x7fa524133e30 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52aaa0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa52411ae70 con 0x7fa524133420 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52903d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa524134260 0x7fa52409c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52903d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa524134260 0x7fa52409c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45572/0 (socket says 172.21.15.89:45572) 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa52903d700 1 -- 172.21.15.89:0/204238319 learned_addr learned my addr 172.21.15.89:0/204238319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.108+0000 7fa523fff700 1 --2- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524133e30 0x7fa52409bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa52903d700 1 -- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133420 msgr2=0x7fa52409b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa52903d700 1 --2- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133420 0x7fa52409b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa52903d700 1 -- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524133e30 msgr2=0x7fa52409bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa52903d700 1 --2- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524133e30 0x7fa52409bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa52903d700 1 -- 172.21.15.89:0/204238319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa510009550 con 0x7fa524134260 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa52903d700 1 --2- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa524134260 0x7fa52409c110 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009e30 tx=0x7fa51c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa5217fa700 1 -- 172.21.15.89:0/204238319 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c009ca0 con 0x7fa524134260 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa5217fa700 1 -- 172.21.15.89:0/204238319 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c00f040 con 0x7fa524134260 2023-09-30T22:23:49.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa5217fa700 1 -- 172.21.15.89:0/204238319 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c014480 con 0x7fa524134260 2023-09-30T22:23:49.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52411b0c0 con 0x7fa524134260 2023-09-30T22:23:49.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.109+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52411a990 con 0x7fa524134260 2023-09-30T22:23:49.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.110+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa524074ea0 con 0x7fa524134260 2023-09-30T22:23:49.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.111+0000 7fa5217fa700 1 -- 172.21.15.89:0/204238319 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa51c0145e0 con 0x7fa524134260 2023-09-30T22:23:49.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.111+0000 7fa5217fa700 1 -- 172.21.15.89:0/204238319 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa51c040890 con 0x7fa524134260 2023-09-30T22:23:49.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.120+0000 7fa5217fa700 1 -- 172.21.15.89:0/204238319 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa51c012930 con 0x7fa524134260 2023-09-30T22:23:49.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.251+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f81ec074ea0 con 0x7f81ec09e1d0 2023-09-30T22:23:49.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.251+0000 7f81daffd700 1 -- 172.21.15.89:0/181596887 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81e0015070 con 0x7f81ec09e1d0 2023-09-30T22:23:49.253 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.254+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec09e1d0 msgr2=0x7f81ec132ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.254+0000 7f81f0819700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec09e1d0 0x7f81ec132ce0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f81e0016000 tx=0x7f81e0004750 comp rx=0 tx=0).stop 2023-09-30T22:23:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.255+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 shutdown_connections 2023-09-30T22:23:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.255+0000 7f81f0819700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81ec135e50 0x7f81ec133220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.255+0000 7f81f0819700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81ec09d6f0 0x7f81ec1327a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.255+0000 7f81f0819700 1 --2- 172.21.15.89:0/181596887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec09e1d0 0x7f81ec132ce0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.255+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 >> 172.21.15.89:0/181596887 conn(0x7f81ec066f50 msgr2=0x7f81ec06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.255+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 shutdown_connections 2023-09-30T22:23:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.255+0000 7f81f0819700 1 -- 172.21.15.89:0/181596887 wait complete. 2023-09-30T22:23:49.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.1 2023-09-30T22:23:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.295+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f61b4074ea0 con 0x7f61b412f8b0 2023-09-30T22:23:49.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.295+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f02fc074ea0 con 0x7f02fc09d6f0 2023-09-30T22:23:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.295+0000 7f61a2ffd700 1 -- 172.21.15.89:0/2773316125 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f61a80199a0 con 0x7f61b412f8b0 2023-09-30T22:23:49.296 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:49.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.295+0000 7f02e3fff700 1 -- 172.21.15.89:0/3344864764 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02ec015020 con 0x7f02fc09d6f0 2023-09-30T22:23:49.297 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b412f8b0 msgr2=0x7f61b4130f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 --2- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b412f8b0 0x7f61b4130f20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f61a8013000 tx=0x7f61a8004750 comp rx=0 tx=0).stop 2023-09-30T22:23:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 shutdown_connections 2023-09-30T22:23:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 --2- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f61b4130260 0x7f61b4131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 --2- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61b412cf60 0x7f61b41309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 --2- 172.21.15.89:0/2773316125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61b412f8b0 0x7f61b4130f20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 >> 172.21.15.89:0/2773316125 conn(0x7f61b40a2aa0 msgr2=0x7f61b412a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 shutdown_connections 2023-09-30T22:23:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.297+0000 7f61b8f4a700 1 -- 172.21.15.89:0/2773316125 wait complete. 2023-09-30T22:23:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02fc09d6f0 msgr2=0x7f02fc1a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02fc09d6f0 0x7f02fc1a5de0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009ed0 tx=0x7f02ec004750 comp rx=0 tx=0).stop 2023-09-30T22:23:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 shutdown_connections 2023-09-30T22:23:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02fc09e410 0x7f02fc1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02fc09d6f0 0x7f02fc1a5de0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 --2- 172.21.15.89:0/3344864764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02fc09e000 0x7f02fc1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 >> 172.21.15.89:0/3344864764 conn(0x7f02fc066f50 msgr2=0x7f02fc0a3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 shutdown_connections 2023-09-30T22:23:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.298+0000 7f03020e3700 1 -- 172.21.15.89:0/3344864764 wait complete. 2023-09-30T22:23:49.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:49.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:23:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.323+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa52411b8b0 con 0x7fa524134260 2023-09-30T22:23:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.323+0000 7fa5217fa700 1 -- 172.21.15.89:0/204238319 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa51c02a070 con 0x7fa524134260 2023-09-30T22:23:49.324 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:49.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa524134260 msgr2=0x7fa52409c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 --2- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa524134260 0x7fa52409c110 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009e30 tx=0x7fa51c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 shutdown_connections 2023-09-30T22:23:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 --2- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa524134260 0x7fa52409c110 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 --2- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa524133420 0x7fa52409b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 --2- 172.21.15.89:0/204238319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa524133e30 0x7fa52409bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 >> 172.21.15.89:0/204238319 conn(0x7fa5240a4210 msgr2=0x7fa52412de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 shutdown_connections 2023-09-30T22:23:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:49.325+0000 7fa52aaa0700 1 -- 172.21.15.89:0/204238319 wait complete. 2023-09-30T22:23:49.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:50.273 INFO:tasks.ceph:ignoring flush pg stats error, probably testing upgrade: timed out waiting for mon to be updated with osd.1: 133143986178 < 133143986180 2023-09-30T22:23:50.547 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:23:50.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:23:50.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:50.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:50.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:50.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.691+0000 7f734df39700 1 Processor -- start 2023-09-30T22:23:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f734df39700 1 -- start start 2023-09-30T22:23:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f734df39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f734df39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f734813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f734df39700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734809edd0 con 0x7f73480a1340 2023-09-30T22:23:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f734df39700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734809e1f0 con 0x7f7348134170 2023-09-30T22:23:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f734df39700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f734809dc50 con 0x7f73480a0a30 2023-09-30T22:23:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f32cdc98700 1 Processor -- start 2023-09-30T22:23:50.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f7347fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f734813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:50.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f73477fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:50.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f7347fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f734813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59868/0 (socket says 172.21.15.89:59868) 2023-09-30T22:23:50.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f7347fff700 1 -- 172.21.15.89:0/3849737851 learned_addr learned my addr 172.21.15.89:0/3849737851 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:50.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.692+0000 7f32cdc98700 1 -- start start 2023-09-30T22:23:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f7347fff700 1 -- 172.21.15.89:0/3849737851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 msgr2=0x7f73480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f7347fff700 1 --2- 172.21.15.89:0/3849737851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f7347fff700 1 -- 172.21.15.89:0/3849737851 >> v1:172.21.15.89:6789/0 conn(0x7f73480a1340 legacy=0x7f7348133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f7347fff700 1 -- 172.21.15.89:0/3849737851 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480a1fe0 con 0x7f7348134170 2023-09-30T22:23:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f7347fff700 1 --2- 172.21.15.89:0/3849737851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f734813bd30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7340009e80 tx=0x7f734000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed38c26c9e03233 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f32cdc98700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809eca0 0x7f32c813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f32cdc98700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806f050 0x7f32c813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f32cdc98700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c8070ee0 con 0x7f32c806e570 2023-09-30T22:23:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f32cdc98700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8071980 con 0x7f32c806f050 2023-09-30T22:23:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f32cdc98700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8071480 con 0x7f32c809eca0 2023-09-30T22:23:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f73467fc700 1 -- 172.21.15.89:0/3849737851 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000e070 con 0x7f7348134170 2023-09-30T22:23:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f73467fc700 1 -- 172.21.15.89:0/3849737851 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734000b910 con 0x7f7348134170 2023-09-30T22:23:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f73467fc700 1 -- 172.21.15.89:0/3849737851 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000bcc0 con 0x7f7348134170 2023-09-30T22:23:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f32c77fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f32c806e570 0x7f32c806e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48494/0 (socket says 172.21.15.89:48494) 2023-09-30T22:23:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f32c77fe700 1 -- 172.21.15.89:0/1403553842 learned_addr learned my addr 172.21.15.89:0/1403553842 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.693+0000 7f32c6ffd700 1 --2- 172.21.15.89:0/1403553842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809eca0 0x7f32c813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f32c6ffd700 1 -- 172.21.15.89:0/1403553842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806f050 msgr2=0x7f32c813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f32c6ffd700 1 --2- 172.21.15.89:0/1403553842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806f050 0x7f32c813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f32c6ffd700 1 -- 172.21.15.89:0/1403553842 >> v1:172.21.15.89:6789/0 conn(0x7f32c806e570 legacy=0x7f32c806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f32c6ffd700 1 -- 172.21.15.89:0/1403553842 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8071b90 con 0x7f32c809eca0 2023-09-30T22:23:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f32c67fc700 1 -- 172.21.15.89:0/1403553842 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221965073 0 0) 0x7f32c8070ee0 con 0x7f32c806e570 2023-09-30T22:23:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f32c6ffd700 1 --2- 172.21.15.89:0/1403553842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809eca0 0x7f32c813cea0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009870 tx=0x7f32b8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=413b5d08bbc58777 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f734df39700 1 -- 172.21.15.89:0/3849737851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 msgr2=0x7f734813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f734df39700 1 --2- 172.21.15.89:0/3849737851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f734813bd30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7340009e80 tx=0x7f734000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f734df39700 1 -- 172.21.15.89:0/3849737851 shutdown_connections 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f734df39700 1 --2- 172.21.15.89:0/3849737851 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f734813bd30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f734df39700 1 --2- 172.21.15.89:0/3849737851 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f734df39700 1 -- 172.21.15.89:0/3849737851 >> 172.21.15.89:0/3849737851 conn(0x7f73480a6550 msgr2=0x7f73480a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f734df39700 1 -- 172.21.15.89:0/3849737851 shutdown_connections 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.694+0000 7f734df39700 1 -- 172.21.15.89:0/3849737851 wait complete. 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.695+0000 7f32c67fc700 1 -- 172.21.15.89:0/1403553842 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8011830 con 0x7f32c809eca0 2023-09-30T22:23:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.695+0000 7f32c67fc700 1 -- 172.21.15.89:0/1403553842 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b8011990 con 0x7f32c809eca0 2023-09-30T22:23:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.695+0000 7f32c67fc700 1 -- 172.21.15.89:0/1403553842 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8011d20 con 0x7f32c809eca0 2023-09-30T22:23:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.695+0000 7f734df39700 1 Processor -- start 2023-09-30T22:23:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.695+0000 7f32cdc98700 1 -- 172.21.15.89:0/1403553842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809eca0 msgr2=0x7f32c813cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.695+0000 7f32cdc98700 1 --2- 172.21.15.89:0/1403553842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809eca0 0x7f32c813cea0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009870 tx=0x7f32b8009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.695+0000 7f734df39700 1 -- start start 2023-09-30T22:23:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7ff674130700 1 Processor -- start 2023-09-30T22:23:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7ff674130700 1 -- start start 2023-09-30T22:23:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f734df39700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73481231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f734df39700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73480a1340 0x7f73481236e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f734df39700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f73481c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f734df39700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7348125dc0 con 0x7f73480a1340 2023-09-30T22:23:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f734df39700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7348126460 con 0x7f7348134170 2023-09-30T22:23:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f734df39700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7348124450 con 0x7f73480a0a30 2023-09-30T22:23:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7ff674130700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09d6f0 0x7ff66c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7ff674130700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff66c09e000 0x7ff66c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7ff674130700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 0x7ff66c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7ff674130700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66c06b460 con 0x7ff66c09e000 2023-09-30T22:23:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7ff674130700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c09b120 con 0x7ff66c09d6f0 2023-09-30T22:23:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7ff674130700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c09a7e0 con 0x7ff66c09e410 2023-09-30T22:23:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f73477fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73481231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f7346ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73480a1340 0x7f73481236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f73477fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73481231a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33794/0 (socket says 172.21.15.89:33794) 2023-09-30T22:23:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.696+0000 7f73477fe700 1 -- 172.21.15.89:0/2108898882 learned_addr learned my addr 172.21.15.89:0/2108898882 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f7347fff700 1 --2- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f73481c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff671ecc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09d6f0 0x7ff66c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff671ecc700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09d6f0 0x7ff66c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59898/0 (socket says 172.21.15.89:59898) 2023-09-30T22:23:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff671ecc700 1 -- 172.21.15.89:0/2383177610 learned_addr learned my addr 172.21.15.89:0/2383177610 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:50.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6726cd700 1 --2- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 0x7ff66c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:50.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f73477fe700 1 -- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 msgr2=0x7f73481c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f73477fe700 1 --2- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f73481c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f73477fe700 1 -- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73480a1340 msgr2=0x7f73481236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f73477fe700 1 --2- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73480a1340 0x7f73481236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f73477fe700 1 -- 172.21.15.89:0/2108898882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7340009b30 con 0x7f73480a0a30 2023-09-30T22:23:51.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6716cb700 1 --2- 172.21.15.89:0/2383177610 >> v2:172.21.15.89:3300/0 conn(0x7ff66c09e000 0x7ff66c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6726cd700 1 -- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09d6f0 msgr2=0x7ff66c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6726cd700 1 --2- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09d6f0 0x7ff66c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6726cd700 1 -- 172.21.15.89:0/2383177610 >> v2:172.21.15.89:3300/0 conn(0x7ff66c09e000 msgr2=0x7ff66c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6726cd700 1 --2- 172.21.15.89:0/2383177610 >> v2:172.21.15.89:3300/0 conn(0x7ff66c09e000 0x7ff66c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6726cd700 1 -- 172.21.15.89:0/2383177610 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c09a500 con 0x7ff66c09e410 2023-09-30T22:23:51.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6716cb700 1 --2- 172.21.15.89:0/2383177610 >> v2:172.21.15.89:3300/0 conn(0x7ff66c09e000 0x7ff66c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:51.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f73477fe700 1 --2- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73481231a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f733800e9b0 tx=0x7f733800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff671ecc700 1 --2- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09d6f0 0x7ff66c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:51.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff6726cd700 1 --2- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 0x7ff66c133f30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff664009dc0 tx=0x7ff66400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6357a47c90b9d07 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f732ffff700 1 -- 172.21.15.89:0/2108898882 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338009710 con 0x7f73480a0a30 2023-09-30T22:23:51.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73480a25e0 con 0x7f73480a0a30 2023-09-30T22:23:51.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f732ffff700 1 -- 172.21.15.89:0/2108898882 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733800ee50 con 0x7f73480a0a30 2023-09-30T22:23:51.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f732ffff700 1 -- 172.21.15.89:0/2108898882 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800ca50 con 0x7f73480a0a30 2023-09-30T22:23:51.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7348122d20 con 0x7f73480a0a30 2023-09-30T22:23:51.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.697+0000 7ff670eca700 1 -- 172.21.15.89:0/2383177610 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66400e070 con 0x7ff66c09e410 2023-09-30T22:23:51.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7ff670eca700 1 -- 172.21.15.89:0/2383177610 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66400bb90 con 0x7ff66c09e410 2023-09-30T22:23:51.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7ff670eca700 1 -- 172.21.15.89:0/2383177610 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664012430 con 0x7ff66c09e410 2023-09-30T22:23:51.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7f32cdc98700 1 -- 172.21.15.89:0/1403553842 shutdown_connections 2023-09-30T22:23:51.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7f32cdc98700 1 --2- 172.21.15.89:0/1403553842 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806f050 0x7f32c813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7f32cdc98700 1 --2- 172.21.15.89:0/1403553842 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c809eca0 0x7f32c813cea0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7f32cdc98700 1 -- 172.21.15.89:0/1403553842 >> 172.21.15.89:0/1403553842 conn(0x7f32c80a2aa0 msgr2=0x7f32c80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7f32cdc98700 1 -- 172.21.15.89:0/1403553842 shutdown_connections 2023-09-30T22:23:51.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7f32cdc98700 1 -- 172.21.15.89:0/1403553842 wait complete. 2023-09-30T22:23:51.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7f732ffff700 1 -- 172.21.15.89:0/2108898882 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f733800cbb0 con 0x7f73480a0a30 2023-09-30T22:23:51.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7ff674130700 1 -- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 msgr2=0x7ff66c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.698+0000 7ff674130700 1 --2- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 0x7ff66c133f30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff664009dc0 tx=0x7ff66400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:51.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7ff674130700 1 -- 172.21.15.89:0/2383177610 shutdown_connections 2023-09-30T22:23:51.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7ff674130700 1 --2- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 0x7ff66c133f30 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7ff674130700 1 --2- 172.21.15.89:0/2383177610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09d6f0 0x7ff66c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7ff674130700 1 --2- 172.21.15.89:0/2383177610 >> v2:172.21.15.89:3300/0 conn(0x7ff66c09e000 0x7ff66c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7ff674130700 1 -- 172.21.15.89:0/2383177610 >> 172.21.15.89:0/2383177610 conn(0x7ff66c066f50 msgr2=0x7ff66c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7f732ffff700 1 -- 172.21.15.89:0/2108898882 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f733803f720 con 0x7f73480a0a30 2023-09-30T22:23:51.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7ff674130700 1 -- 172.21.15.89:0/2383177610 shutdown_connections 2023-09-30T22:23:51.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7ff674130700 1 -- 172.21.15.89:0/2383177610 wait complete. 2023-09-30T22:23:51.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7f32cdc98700 1 Processor -- start 2023-09-30T22:23:51.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.699+0000 7f32cdc98700 1 -- start start 2023-09-30T22:23:51.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7da6e9700 1 Processor -- start 2023-09-30T22:23:51.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7ff674130700 1 Processor -- start 2023-09-30T22:23:51.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7ff674130700 1 -- start start 2023-09-30T22:23:51.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7348074ea0 con 0x7f73480a0a30 2023-09-30T22:23:51.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32cdc98700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806e570 0x7f32c81214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32cdc98700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c806f050 0x7f32c8121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32cdc98700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c8121f60 0x7f32c81cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32cdc98700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8122e90 con 0x7f32c8121f60 2023-09-30T22:23:51.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32cdc98700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8070610 con 0x7f32c806e570 2023-09-30T22:23:51.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32cdc98700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8123bb0 con 0x7f32c806f050 2023-09-30T22:23:51.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32c77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806e570 0x7f32c81214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32c77fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806e570 0x7f32c81214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59900/0 (socket says 172.21.15.89:59900) 2023-09-30T22:23:51.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32c77fe700 1 -- 172.21.15.89:0/1006858082 learned_addr learned my addr 172.21.15.89:0/1006858082 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:51.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7f32c7fff700 1 --2- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c8121f60 0x7f32c81cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7da6e9700 1 -- start start 2023-09-30T22:23:51.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7da6e9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa7d406e570 0x7fa7d406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7da6e9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7d409eca0 0x7fa7d406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7da6e9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d4070ee0 con 0x7fa7d4134610 2023-09-30T22:23:51.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7da6e9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d4071480 con 0x7fa7d406e570 2023-09-30T22:23:51.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7da6e9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d406f3c0 con 0x7fa7d409eca0 2023-09-30T22:23:51.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7d8c86700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa7d4134610 0x7fa7d4134a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48500/0 (socket says 172.21.15.89:48500) 2023-09-30T22:23:51.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.700+0000 7fa7d8c86700 1 -- 172.21.15.89:0/143589059 learned_addr learned my addr 172.21.15.89:0/143589059 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:23:51.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7f32c77fe700 1 -- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c806f050 msgr2=0x7f32c8121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7f32c77fe700 1 --2- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c806f050 0x7f32c8121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7f32c77fe700 1 -- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c8121f60 msgr2=0x7f32c81cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7f32c77fe700 1 --2- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c8121f60 0x7f32c81cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7f32c77fe700 1 -- 172.21.15.89:0/1006858082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80094a0 con 0x7f32c806e570 2023-09-30T22:23:51.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048176323 0 0) 0x7fa7d4070ee0 con 0x7fa7d4134610 2023-09-30T22:23:51.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc0035b0 con 0x7fa7d4134610 2023-09-30T22:23:51.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d37fe700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7d409eca0 0x7fa7d406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2033978426 0 0) 0x7fa7bc0035b0 con 0x7fa7d4134610 2023-09-30T22:23:51.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff674130700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff66c09d6f0 0x7ff66c1a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff674130700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09e000 0x7ff66c1a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff674130700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 0x7ff66c1a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff674130700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1a7310 con 0x7ff66c09d6f0 2023-09-30T22:23:51.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff674130700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1a7a70 con 0x7ff66c09e000 2023-09-30T22:23:51.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff674130700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1a8cc0 con 0x7ff66c09e410 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7d409eca0 msgr2=0x7fa7d406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7d409eca0 0x7fa7d406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 >> v2:172.21.15.89:3300/0 conn(0x7fa7d406e570 msgr2=0x7fa7d406e940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 --2- 172.21.15.89:0/143589059 >> v2:172.21.15.89:3300/0 conn(0x7fa7d406e570 0x7fa7d406e940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d4071b90 con 0x7fa7d4134610 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1945586405 0 0) 0x7fa7cc0019f0 con 0x7fa7d4134610 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 >> v1:172.21.15.89:6789/0 conn(0x7fa7d4134610 legacy=0x7fa7d4134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7bc003a00 0x7fa7bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7bc004850 0x7fa7bc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7bc005200 0x7fa7bc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4071480 con 0x7fa7bc003a00 2023-09-30T22:23:51.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d406f3c0 con 0x7fa7bc004850 2023-09-30T22:23:51.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc00c110 con 0x7fa7bc005200 2023-09-30T22:23:51.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7f32c77fe700 1 --2- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806e570 0x7f32c81214e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00c550 tx=0x7f32bc00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d37fe700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7bc004850 0x7fa7bc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7fa7d8c86700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7bc005200 0x7fa7bc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7fa7d8c86700 1 -- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7bc004850 msgr2=0x7fa7bc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7fa7d8c86700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7bc004850 0x7fa7bc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7fa7d8c86700 1 -- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7bc003a00 msgr2=0x7fa7bc003e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:23:51.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff671ecc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff66c09d6f0 0x7ff66c1a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff671ecc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff66c09d6f0 0x7ff66c1a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60996/0 (socket says 172.21.15.89:60996) 2023-09-30T22:23:51.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.701+0000 7ff671ecc700 1 -- 172.21.15.89:0/2783108789 learned_addr learned my addr 172.21.15.89:0/2783108789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff671ecc700 1 -- 172.21.15.89:0/2783108789 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 msgr2=0x7ff66c1a68e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff671ecc700 1 --2- 172.21.15.89:0/2783108789 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff66c09e410 0x7ff66c1a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff671ecc700 1 -- 172.21.15.89:0/2783108789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09e000 msgr2=0x7ff66c1a63a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff671ecc700 1 --2- 172.21.15.89:0/2783108789 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff66c09e000 0x7ff66c1a63a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff671ecc700 1 -- 172.21.15.89:0/2783108789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664009a70 con 0x7ff66c09d6f0 2023-09-30T22:23:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7f32affff700 1 -- 172.21.15.89:0/1006858082 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc004c80 con 0x7f32c806e570 2023-09-30T22:23:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7fa7d8c86700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7bc003a00 0x7fa7bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7fa7d8c86700 1 -- 172.21.15.89:0/143589059 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0035b0 con 0x7fa7bc005200 2023-09-30T22:23:51.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7f32affff700 1 -- 172.21.15.89:0/1006858082 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc004e00 con 0x7f32c806e570 2023-09-30T22:23:51.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7f32affff700 1 -- 172.21.15.89:0/1006858082 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc007310 con 0x7f32c806e570 2023-09-30T22:23:51.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c8124800 con 0x7f32c806e570 2023-09-30T22:23:51.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff671ecc700 1 --2- 172.21.15.89:0/2783108789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff66c09d6f0 0x7ff66c1a5e60 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff66800d800 tx=0x7ff66800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c8122770 con 0x7f32c806e570 2023-09-30T22:23:51.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff6627fc700 1 -- 172.21.15.89:0/2783108789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668004030 con 0x7ff66c09d6f0 2023-09-30T22:23:51.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff6627fc700 1 -- 172.21.15.89:0/2783108789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66801a070 con 0x7ff66c09d6f0 2023-09-30T22:23:51.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff6627fc700 1 -- 172.21.15.89:0/2783108789 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6680043d0 con 0x7ff66c09d6f0 2023-09-30T22:23:51.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7fa7d8c86700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7bc005200 0x7fa7bc00ba10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00a7b0 tx=0x7fa7bc005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=774d5f349417254b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.702+0000 7ff674130700 1 -- 172.21.15.89:0/2783108789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c1a96a0 con 0x7ff66c09d6f0 2023-09-30T22:23:51.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.703+0000 7ff674130700 1 -- 172.21.15.89:0/2783108789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c0a1e90 con 0x7ff66c09d6f0 2023-09-30T22:23:51.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.704+0000 7ff6627fc700 1 -- 172.21.15.89:0/2783108789 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff6680036a0 con 0x7ff66c09d6f0 2023-09-30T22:23:51.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.704+0000 7ff6627fc700 1 -- 172.21.15.89:0/2783108789 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff6680475a0 con 0x7ff66c09d6f0 2023-09-30T22:23:51.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.703+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8074ea0 con 0x7f32c806e570 2023-09-30T22:23:51.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.703+0000 7f32affff700 1 -- 172.21.15.89:0/1006858082 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f32bc007470 con 0x7f32c806e570 2023-09-30T22:23:51.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.703+0000 7f32affff700 1 -- 172.21.15.89:0/1006858082 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f32bc03f370 con 0x7f32c806e570 2023-09-30T22:23:51.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.705+0000 7ff674130700 1 -- 172.21.15.89:0/2783108789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c1a8050 con 0x7ff66c09d6f0 2023-09-30T22:23:51.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc012600 con 0x7fa7bc005200 2023-09-30T22:23:51.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc0127a0 con 0x7fa7bc005200 2023-09-30T22:23:51.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7fa7d2ffd700 1 -- 172.21.15.89:0/143589059 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc012b50 con 0x7fa7bc005200 2023-09-30T22:23:51.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7f732ffff700 1 -- 172.21.15.89:0/2108898882 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73380232f0 con 0x7f73480a0a30 2023-09-30T22:23:51.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7fa7da6e9700 1 -- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7bc005200 msgr2=0x7fa7bc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7fa7da6e9700 1 --2- 172.21.15.89:0/143589059 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7bc005200 0x7fa7bc00ba10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00a7b0 tx=0x7fa7bc005880 comp rx=0 tx=0).stop 2023-09-30T22:23:51.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7fa7d8c86700 1 -- 172.21.15.89:0/143589059 reap_dead start 2023-09-30T22:23:51.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7fa7da6e9700 1 -- 172.21.15.89:0/143589059 shutdown_connections 2023-09-30T22:23:51.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.709+0000 7fa7da6e9700 1 -- 172.21.15.89:0/143589059 >> 172.21.15.89:0/143589059 conn(0x7fa7d40a2aa0 msgr2=0x7fa7d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.710+0000 7fa7da6e9700 1 -- 172.21.15.89:0/143589059 shutdown_connections 2023-09-30T22:23:51.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.710+0000 7fa7da6e9700 1 -- 172.21.15.89:0/143589059 wait complete. 2023-09-30T22:23:51.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.710+0000 7ff6627fc700 1 -- 172.21.15.89:0/2783108789 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff668027930 con 0x7ff66c09d6f0 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7da6e9700 1 Processor -- start 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7da6e9700 1 -- start start 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7da6e9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7d41404d0 0x7fa7d413ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7da6e9700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7d413f100 0x7fa7d413f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7da6e9700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7d4133080 0x7fa7d41334f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7da6e9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d413baa0 con 0x7fa7d41404d0 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7da6e9700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d413fa50 con 0x7fa7d4133080 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7da6e9700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d413cfe0 con 0x7fa7d413f100 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7d413f100 0x7fa7d413f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7d413f100 0x7fa7d413f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33824/0 (socket says 172.21.15.89:33824) 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7fa7d37fe700 1 -- 172.21.15.89:0/787068763 learned_addr learned my addr 172.21.15.89:0/787068763 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d37fe700 1 -- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7d4133080 msgr2=0x7fa7d41334f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d37fe700 1 --2- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7d4133080 0x7fa7d41334f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d37fe700 1 -- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7d41404d0 msgr2=0x7fa7d413ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d37fe700 1 --2- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7d41404d0 0x7fa7d413ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d37fe700 1 -- 172.21.15.89:0/787068763 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d40a1f80 con 0x7fa7d413f100 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d37fe700 1 --2- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7d413f100 0x7fa7d413f510 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40115c0 tx=0x7fa7c4011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d0ff9700 1 -- 172.21.15.89:0/787068763 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c400c070 con 0x7fa7d413f100 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d0ff9700 1 -- 172.21.15.89:0/787068763 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c4015930 con 0x7fa7d413f100 2023-09-30T22:23:51.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d4070610 con 0x7fa7d413f100 2023-09-30T22:23:51.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7d0ff9700 1 -- 172.21.15.89:0/787068763 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4015cc0 con 0x7fa7d413f100 2023-09-30T22:23:51.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.712+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d413e770 con 0x7fa7d413f100 2023-09-30T22:23:51.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.713+0000 7fa7d0ff9700 1 -- 172.21.15.89:0/787068763 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa7c4013400 con 0x7fa7d413f100 2023-09-30T22:23:51.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.713+0000 7fa7d0ff9700 1 -- 172.21.15.89:0/787068763 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa7c403fb10 con 0x7fa7d413f100 2023-09-30T22:23:51.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.713+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d409d1f0 con 0x7fa7d413f100 2023-09-30T22:23:51.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.711+0000 7f32affff700 1 -- 172.21.15.89:0/1006858082 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32bc0133c0 con 0x7f32c806e570 2023-09-30T22:23:51.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.718+0000 7fa7d0ff9700 1 -- 172.21.15.89:0/787068763 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa7c4023080 con 0x7fa7d413f100 2023-09-30T22:23:51.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.900+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f73480a28e0 con 0x7f73480a0a30 2023-09-30T22:23:51.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.901+0000 7f732ffff700 1 -- 172.21.15.89:0/2108898882 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7338017020 con 0x7f73480a0a30 2023-09-30T22:23:51.094 INFO:teuthology.orchestra.run.smithi089.stdout:137438953474 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 msgr2=0x7f73481231a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 --2- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73481231a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f733800e9b0 tx=0x7f733800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 shutdown_connections 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 --2- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7348134170 0x7f73481c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 --2- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73480a0a30 0x7f73481231a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 --2- 172.21.15.89:0/2108898882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73480a1340 0x7f73481236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 >> 172.21.15.89:0/2108898882 conn(0x7f73480a6550 msgr2=0x7f734809ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 shutdown_connections 2023-09-30T22:23:51.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.903+0000 7f734df39700 1 -- 172.21.15.89:0/2108898882 wait complete. 2023-09-30T22:23:51.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.911+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa7d4074ea0 con 0x7fa7d413f100 2023-09-30T22:23:51.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.912+0000 7fa7d0ff9700 1 -- 172.21.15.89:0/787068763 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa7c401d820 con 0x7fa7d413f100 2023-09-30T22:23:51.096 INFO:teuthology.orchestra.run.smithi089.stdout:133143986178 2023-09-30T22:23:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.914+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7d413f100 msgr2=0x7fa7d413f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.914+0000 7fa7da6e9700 1 --2- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7d413f100 0x7fa7d413f510 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40115c0 tx=0x7fa7c4011980 comp rx=0 tx=0).stop 2023-09-30T22:23:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.914+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 shutdown_connections 2023-09-30T22:23:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.914+0000 7fa7da6e9700 1 --2- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa7d4133080 0x7fa7d41334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.914+0000 7fa7da6e9700 1 --2- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7d41404d0 0x7fa7d413ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.914+0000 7fa7da6e9700 1 --2- 172.21.15.89:0/787068763 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa7d413f100 0x7fa7d413f510 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.914+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 >> 172.21.15.89:0/787068763 conn(0x7fa7d40a2aa0 msgr2=0x7fa7d4138100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.914+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 shutdown_connections 2023-09-30T22:23:51.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:23:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.915+0000 7fa7da6e9700 1 -- 172.21.15.89:0/787068763 wait complete. 2023-09-30T22:23:51.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:23:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.932+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f32c806b9e0 con 0x7f32c806e570 2023-09-30T22:23:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.932+0000 7f32affff700 1 -- 172.21.15.89:0/1006858082 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32bc011360 con 0x7f32c806e570 2023-09-30T22:23:51.100 INFO:teuthology.orchestra.run.smithi089.stdout:124554051586 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.934+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806e570 msgr2=0x7f32c81214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.934+0000 7f32cdc98700 1 --2- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806e570 0x7f32c81214e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00c550 tx=0x7f32bc00c860 comp rx=0 tx=0).stop 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.934+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 shutdown_connections 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.934+0000 7f32cdc98700 1 --2- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32c8121f60 0x7f32c81cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.934+0000 7f32cdc98700 1 --2- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f32c806e570 0x7f32c81214e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.934+0000 7f32cdc98700 1 --2- 172.21.15.89:0/1006858082 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32c806f050 0x7f32c8121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.934+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 >> 172.21.15.89:0/1006858082 conn(0x7f32c80a2aa0 msgr2=0x7f32c80a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.935+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 shutdown_connections 2023-09-30T22:23:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:23:50.935+0000 7f32cdc98700 1 -- 172.21.15.89:0/1006858082 wait complete. 2023-09-30T22:23:51.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.0 2023-09-30T22:25:50.592 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-30T22:25:50.593 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:25:50.671 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=38e6cdfdfd0a40f1b6744f4631b74a54 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:25:50.675 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:25:50.686 DEBUG:teuthology.run_tasks:Unwinding manager ceph.stop 2023-09-30T22:25:50.722 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:25:50.735 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:25:50.746 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2023-09-30T22:25:50.756 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-09-30T22:25:50.771 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-09-30T22:25:50.772 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:25:50.772 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:25:50.811 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:25:50.811 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:25:50.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:25:51.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.007+0000 7f60d2fc5700 1 Processor -- start 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.007+0000 7f60d2fc5700 1 -- start start 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d2fc5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d2fc5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc06b080 0x7f60cc131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d2fc5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60cc131b40 0x7f60cc131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d2fc5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc09ece0 con 0x7f60cc131b40 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d2fc5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc09e760 con 0x7f60cc06a770 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d2fc5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc09e040 con 0x7f60cc06b080 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d0d61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d1562700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60cc131b40 0x7f60cc131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60cbfff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc06b080 0x7f60cc131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d1562700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60cc131b40 0x7f60cc131fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44266/0 (socket says 172.21.15.89:44266) 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d0d61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52352/0 (socket says 172.21.15.89:52352) 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d1562700 1 -- 172.21.15.89:0/1588748911 learned_addr learned my addr 172.21.15.89:0/1588748911 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60cbfff700 1 --2- 172.21.15.89:0/1588748911 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc06b080 0x7f60cc131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58180/0 (socket says 172.21.15.89:58180) 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d0d61700 1 -- 172.21.15.89:0/1588748911 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc06b080 msgr2=0x7f60cc131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d0d61700 1 --2- 172.21.15.89:0/1588748911 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc06b080 0x7f60cc131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d0d61700 1 -- 172.21.15.89:0/1588748911 >> v2:172.21.15.89:3300/0 conn(0x7f60cc131b40 msgr2=0x7f60cc131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d0d61700 1 --2- 172.21.15.89:0/1588748911 >> v2:172.21.15.89:3300/0 conn(0x7f60cc131b40 0x7f60cc131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d0d61700 1 -- 172.21.15.89:0/1588748911 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc09f630 con 0x7f60cc06a770 2023-09-30T22:25:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d1562700 1 --2- 172.21.15.89:0/1588748911 >> v2:172.21.15.89:3300/0 conn(0x7f60cc131b40 0x7f60cc131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60d0d61700 1 --2- 172.21.15.89:0/1588748911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc06ab40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009840 tx=0x7f60c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0c4846f889608d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60cb7fe700 1 -- 172.21.15.89:0/1588748911 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0011830 con 0x7f60cc06a770 2023-09-30T22:25:51.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60cb7fe700 1 -- 172.21.15.89:0/1588748911 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c0011990 con 0x7f60cc06a770 2023-09-30T22:25:51.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.008+0000 7f60cb7fe700 1 -- 172.21.15.89:0/1588748911 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0011d20 con 0x7f60cc06a770 2023-09-30T22:25:51.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 -- 172.21.15.89:0/1588748911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 msgr2=0x7f60cc06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 --2- 172.21.15.89:0/1588748911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc06ab40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009840 tx=0x7f60c0009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:51.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 -- 172.21.15.89:0/1588748911 shutdown_connections 2023-09-30T22:25:51.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 --2- 172.21.15.89:0/1588748911 >> v2:172.21.15.89:3300/0 conn(0x7f60cc131b40 0x7f60cc131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 --2- 172.21.15.89:0/1588748911 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc06ab40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 --2- 172.21.15.89:0/1588748911 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc06b080 0x7f60cc131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 -- 172.21.15.89:0/1588748911 >> 172.21.15.89:0/1588748911 conn(0x7f60cc066f50 msgr2=0x7f60cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 -- 172.21.15.89:0/1588748911 shutdown_connections 2023-09-30T22:25:51.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.009+0000 7f60d2fc5700 1 -- 172.21.15.89:0/1588748911 wait complete. 2023-09-30T22:25:51.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.010+0000 7f60d2fc5700 1 Processor -- start 2023-09-30T22:25:51.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.010+0000 7f60d2fc5700 1 -- start start 2023-09-30T22:25:51.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d2fc5700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc120070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d2fc5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60cc06b080 0x7f60cc1205b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d2fc5700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc131b40 0x7f60cc120af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d2fc5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc122b90 con 0x7f60cc06b080 2023-09-30T22:25:51.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d2fc5700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc1214d0 con 0x7f60cc06a770 2023-09-30T22:25:51.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d2fc5700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0a3960 con 0x7f60cc131b40 2023-09-30T22:25:51.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc120070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc120070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52356/0 (socket says 172.21.15.89:52356) 2023-09-30T22:25:51.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 -- 172.21.15.89:0/565977361 learned_addr learned my addr 172.21.15.89:0/565977361 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:51.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60cc06b080 0x7f60cc1205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d1562700 1 --2- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc131b40 0x7f60cc120af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 -- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc131b40 msgr2=0x7f60cc120af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 --2- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc131b40 0x7f60cc120af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 -- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60cc06b080 msgr2=0x7f60cc1205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 --2- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60cc06b080 0x7f60cc1205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 -- 172.21.15.89:0/565977361 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c4009b30 con 0x7f60cc06a770 2023-09-30T22:25:51.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d0d61700 1 --2- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc120070 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f60c00090b0 tx=0x7f60c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60c97fa700 1 -- 172.21.15.89:0/565977361 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0004450 con 0x7f60cc06a770 2023-09-30T22:25:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60c97fa700 1 -- 172.21.15.89:0/565977361 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c001f070 con 0x7f60cc06a770 2023-09-30T22:25:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c00094a0 con 0x7f60cc06a770 2023-09-30T22:25:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60c97fa700 1 -- 172.21.15.89:0/565977361 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c00038e0 con 0x7f60cc06a770 2023-09-30T22:25:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.011+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc122470 con 0x7f60cc06a770 2023-09-30T22:25:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.012+0000 7f60c97fa700 1 -- 172.21.15.89:0/565977361 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f60c0003a40 con 0x7f60cc06a770 2023-09-30T22:25:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.012+0000 7f60c97fa700 1 -- 172.21.15.89:0/565977361 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f60c0021070 con 0x7f60cc06a770 2023-09-30T22:25:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.012+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc074ea0 con 0x7f60cc06a770 2023-09-30T22:25:51.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.017+0000 7f60c97fa700 1 -- 172.21.15.89:0/565977361 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60c001a430 con 0x7f60cc06a770 2023-09-30T22:25:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.200+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60cc06c360 con 0x7f60cc06a770 2023-09-30T22:25:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.201+0000 7f60c97fa700 1 -- 172.21.15.89:0/565977361 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13399 (secure 0 0 0) 0x7f60c0015030 con 0x7f60cc06a770 2023-09-30T22:25:51.202 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:25:51.202 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":40,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:17:18.222304+0000","last_up_change":"2023-09-30T22:16:38.305512+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":129669}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":129669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":129669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":129669}]},"public_addr":"172.21.15.89:6804/129669","cluster_addr":"172.21.15.89:6805/129669","heartbeat_back_addr":"172.21.15.89:6807/129669","heartbeat_front_addr":"172.21.15.89:6806/129669","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":129767}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":129767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":129767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":129767}]},"public_addr":"172.21.15.89:6800/129767","cluster_addr":"172.21.15.89:6801/129767","heartbeat_back_addr":"172.21.15.89:6803/129767","heartbeat_front_addr":"172.21.15.89:6802/129767","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":129877}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":129877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":129877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":129877}]},"public_addr":"172.21.15.89:6808/129877","cluster_addr":"172.21.15.89:6809/129877","heartbeat_back_addr":"172.21.15.89:6811/129877","heartbeat_front_addr":"172.21.15.89:6810/129877","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":130217}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":130217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":130217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":130217}]},"public_addr":"172.21.15.89:6812/130217","cluster_addr":"172.21.15.89:6813/130217","heartbeat_back_addr":"172.21.15.89:6815/130217","heartbeat_front_addr":"172.21.15.89:6814/130217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:16:32.255166+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/2456756709":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/4019526010":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000","172.21.15.89:6817/3539463463":"2023-10-01T22:16:43.298028+0000","172.21.15.89:0/2736305244":"2023-10-01T22:17:18.222274+0000","172.21.15.89:6816/126953":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/4100667825":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/1785021332":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/1295992953":"2023-10-01T22:17:18.222274+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:25:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 msgr2=0x7f60cc120070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 --2- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc120070 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f60c00090b0 tx=0x7f60c0004750 comp rx=0 tx=0).stop 2023-09-30T22:25:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 shutdown_connections 2023-09-30T22:25:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 --2- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60cc131b40 0x7f60cc120af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 --2- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60cc06a770 0x7f60cc120070 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 --2- 172.21.15.89:0/565977361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60cc06b080 0x7f60cc1205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 >> 172.21.15.89:0/565977361 conn(0x7f60cc066f50 msgr2=0x7f60cc071480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 shutdown_connections 2023-09-30T22:25:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.203+0000 7f60d2fc5700 1 -- 172.21.15.89:0/565977361 wait complete. 2023-09-30T22:25:51.216 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:14:00.533280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:14:01.011104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:25:51.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:25:51.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.340+0000 7f53e00ae700 1 Processor -- start 2023-09-30T22:25:51.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.340+0000 7f53e00ae700 1 -- start start 2023-09-30T22:25:51.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53e00ae700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53d809d7d0 0x7f53d809dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53e00ae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d8130e80 0x7f53d8138a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53e00ae700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d809b800 con 0x7f53d809e0e0 2023-09-30T22:25:51.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53e00ae700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d809b620 con 0x7f53d809d7d0 2023-09-30T22:25:51.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53e00ae700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d809f370 con 0x7f53d8130e80 2023-09-30T22:25:51.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53dde4a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53d809d7d0 0x7f53d809dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53de64b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d8130e80 0x7f53d8138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53dde4a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53d809d7d0 0x7f53d809dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44296/0 (socket says 172.21.15.89:44296) 2023-09-30T22:25:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53dde4a700 1 -- 172.21.15.89:0/113428516 learned_addr learned my addr 172.21.15.89:0/113428516 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53dde4a700 1 -- 172.21.15.89:0/113428516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d8130e80 msgr2=0x7f53d8138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53dde4a700 1 --2- 172.21.15.89:0/113428516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d8130e80 0x7f53d8138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53dde4a700 1 -- 172.21.15.89:0/113428516 >> v1:172.21.15.89:6789/0 conn(0x7f53d809e0e0 legacy=0x7f53d8130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53dde4a700 1 -- 172.21.15.89:0/113428516 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d809ef80 con 0x7f53d809d7d0 2023-09-30T22:25:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53de64b700 1 --2- 172.21.15.89:0/113428516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d8130e80 0x7f53d8138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.341+0000 7f53dde4a700 1 --2- 172.21.15.89:0/113428516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d809d7d0 0x7f53d809dba0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f53d40098f0 tx=0x7f53d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3eda77e7f6ef5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53dce48700 1 -- 172.21.15.89:0/113428516 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4011830 con 0x7f53d809d7d0 2023-09-30T22:25:51.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53dce48700 1 -- 172.21.15.89:0/113428516 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d4011990 con 0x7f53d809d7d0 2023-09-30T22:25:51.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53dce48700 1 -- 172.21.15.89:0/113428516 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4011d20 con 0x7f53d809d7d0 2023-09-30T22:25:51.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53e00ae700 1 -- 172.21.15.89:0/113428516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d809d7d0 msgr2=0x7f53d809dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53e00ae700 1 --2- 172.21.15.89:0/113428516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d809d7d0 0x7f53d809dba0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f53d40098f0 tx=0x7f53d4009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:51.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53e00ae700 1 -- 172.21.15.89:0/113428516 shutdown_connections 2023-09-30T22:25:51.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53e00ae700 1 --2- 172.21.15.89:0/113428516 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d8130e80 0x7f53d8138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53e00ae700 1 --2- 172.21.15.89:0/113428516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d809d7d0 0x7f53d809dba0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53e00ae700 1 -- 172.21.15.89:0/113428516 >> 172.21.15.89:0/113428516 conn(0x7f53d8066d70 msgr2=0x7f53d80a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53e00ae700 1 -- 172.21.15.89:0/113428516 shutdown_connections 2023-09-30T22:25:51.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.342+0000 7f53e00ae700 1 -- 172.21.15.89:0/113428516 wait complete. 2023-09-30T22:25:51.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.343+0000 7f53e00ae700 1 Processor -- start 2023-09-30T22:25:51.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53e00ae700 1 -- start start 2023-09-30T22:25:51.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53e00ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d809d7d0 0x7f53d81270a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53e00ae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d809e0e0 0x7f53d81275e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53e00ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d8130e80 0x7f53d81cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53e00ae700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d812a000 con 0x7f53d8130e80 2023-09-30T22:25:51.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53e00ae700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8129370 con 0x7f53d809e0e0 2023-09-30T22:25:51.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53e00ae700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53d812a1e0 con 0x7f53d809d7d0 2023-09-30T22:25:51.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53dde4a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d809d7d0 0x7f53d81270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53dde4a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d809d7d0 0x7f53d81270a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58208/0 (socket says 172.21.15.89:58208) 2023-09-30T22:25:51.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53dde4a700 1 -- 172.21.15.89:0/3484676966 learned_addr learned my addr 172.21.15.89:0/3484676966 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:51.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.344+0000 7f53dd649700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d809e0e0 0x7f53d81275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53de64b700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d8130e80 0x7f53d81cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53dd649700 1 -- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d809d7d0 msgr2=0x7f53d81270a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53dd649700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d809d7d0 0x7f53d81270a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53dd649700 1 -- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d8130e80 msgr2=0x7f53d81cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53dd649700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d8130e80 0x7f53d81cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53dd649700 1 -- 172.21.15.89:0/3484676966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009a70 con 0x7f53d809e0e0 2023-09-30T22:25:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53de64b700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d8130e80 0x7f53d81cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:25:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53dde4a700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d809d7d0 0x7f53d81270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53dd649700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d809e0e0 0x7f53d81275e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f53c8000f60 tx=0x7f53c800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53ce7fc700 1 -- 172.21.15.89:0/3484676966 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80052d0 con 0x7f53d809e0e0 2023-09-30T22:25:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53ce7fc700 1 -- 172.21.15.89:0/3484676966 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8005430 con 0x7f53d809e0e0 2023-09-30T22:25:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d4009550 con 0x7f53d809e0e0 2023-09-30T22:25:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53ce7fc700 1 -- 172.21.15.89:0/3484676966 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80038c0 con 0x7f53d809e0e0 2023-09-30T22:25:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.345+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d8127b20 con 0x7f53d809e0e0 2023-09-30T22:25:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.346+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d804f2a0 con 0x7f53d809e0e0 2023-09-30T22:25:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.346+0000 7f53ce7fc700 1 -- 172.21.15.89:0/3484676966 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53c8003a20 con 0x7f53d809e0e0 2023-09-30T22:25:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.347+0000 7f53ce7fc700 1 -- 172.21.15.89:0/3484676966 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f53c803eed0 con 0x7f53d809e0e0 2023-09-30T22:25:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.351+0000 7f53ce7fc700 1 -- 172.21.15.89:0/3484676966 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53c8018260 con 0x7f53d809e0e0 2023-09-30T22:25:51.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.532+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53d8126c20 con 0x7f53d809e0e0 2023-09-30T22:25:51.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.533+0000 7f53ce7fc700 1 -- 172.21.15.89:0/3484676966 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53c8017030 con 0x7f53d809e0e0 2023-09-30T22:25:51.534 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:25:51.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d809e0e0 msgr2=0x7f53d81275e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d809e0e0 0x7f53d81275e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f53c8000f60 tx=0x7f53c800c730 comp rx=0 tx=0).stop 2023-09-30T22:25:51.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 shutdown_connections 2023-09-30T22:25:51.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53d8130e80 0x7f53d81cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f53d809d7d0 0x7f53d81270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 --2- 172.21.15.89:0/3484676966 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53d809e0e0 0x7f53d81275e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 >> 172.21.15.89:0/3484676966 conn(0x7f53d8066d70 msgr2=0x7f53d80a2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 shutdown_connections 2023-09-30T22:25:51.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.535+0000 7f53e00ae700 1 -- 172.21.15.89:0/3484676966 wait complete. 2023-09-30T22:25:51.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:25:51.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.670+0000 7fecf115a700 1 Processor -- start 2023-09-30T22:25:51.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecf115a700 1 -- start start 2023-09-30T22:25:51.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecf115a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec09dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecf115a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fecec130e80 0x7fecec138a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecf115a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec09b800 con 0x7fecec09e0e0 2023-09-30T22:25:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecf115a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec09b620 con 0x7fecec130e80 2023-09-30T22:25:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecf115a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec09f370 con 0x7fecec09d7d0 2023-09-30T22:25:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7feceb59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fecec130e80 0x7fecec138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecead9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec09dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecead9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec09dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52388/0 (socket says 172.21.15.89:52388) 2023-09-30T22:25:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7feceb59e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fecec130e80 0x7fecec138a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44312/0 (socket says 172.21.15.89:44312) 2023-09-30T22:25:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecead9d700 1 -- 172.21.15.89:0/595662875 learned_addr learned my addr 172.21.15.89:0/595662875 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.671+0000 7fecea59c700 1 --1- 172.21.15.89:0/595662875 >> v1:172.21.15.89:6789/0 conn(0x7fecec09e0e0 0x7fecec130770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34998/0 (socket says 172.21.15.89:34998) 2023-09-30T22:25:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7feceb59e700 1 -- 172.21.15.89:0/595662875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 msgr2=0x7fecec09dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7feceb59e700 1 --2- 172.21.15.89:0/595662875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec09dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7feceb59e700 1 -- 172.21.15.89:0/595662875 >> v1:172.21.15.89:6789/0 conn(0x7fecec09e0e0 legacy=0x7fecec130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7feceb59e700 1 -- 172.21.15.89:0/595662875 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec09ef80 con 0x7fecec130e80 2023-09-30T22:25:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7fece9d9b700 1 -- 172.21.15.89:0/595662875 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113540038 0 0) 0x7fecec09b800 con 0x7fecec09e0e0 2023-09-30T22:25:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7fecead9d700 1 --2- 172.21.15.89:0/595662875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec09dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7feceb59e700 1 --2- 172.21.15.89:0/595662875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec130e80 0x7fecec138a40 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fece4009d90 tx=0x7fece400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491469ea921cba7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7fece9d9b700 1 -- 172.21.15.89:0/595662875 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece400e070 con 0x7fecec130e80 2023-09-30T22:25:51.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7fece9d9b700 1 -- 172.21.15.89:0/595662875 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece400bb20 con 0x7fecec130e80 2023-09-30T22:25:51.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7fece9d9b700 1 -- 172.21.15.89:0/595662875 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4012430 con 0x7fecec130e80 2023-09-30T22:25:51.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.672+0000 7fecf115a700 1 -- 172.21.15.89:0/595662875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec130e80 msgr2=0x7fecec138a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.673+0000 7fecf115a700 1 --2- 172.21.15.89:0/595662875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec130e80 0x7fecec138a40 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fece4009d90 tx=0x7fece400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:51.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.673+0000 7fecf115a700 1 -- 172.21.15.89:0/595662875 shutdown_connections 2023-09-30T22:25:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.673+0000 7fecf115a700 1 --2- 172.21.15.89:0/595662875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec130e80 0x7fecec138a40 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.673+0000 7fecf115a700 1 --2- 172.21.15.89:0/595662875 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec09dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.673+0000 7fecf115a700 1 -- 172.21.15.89:0/595662875 >> 172.21.15.89:0/595662875 conn(0x7fecec066d70 msgr2=0x7fecec0a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.673+0000 7fecf115a700 1 -- 172.21.15.89:0/595662875 shutdown_connections 2023-09-30T22:25:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.673+0000 7fecf115a700 1 -- 172.21.15.89:0/595662875 wait complete. 2023-09-30T22:25:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.674+0000 7fecf115a700 1 Processor -- start 2023-09-30T22:25:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.674+0000 7fecf115a700 1 -- start start 2023-09-30T22:25:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecf115a700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec11c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecf115a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec09e0e0 0x7fecec11c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecf115a700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec130e80 0x7fecec11cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecf115a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec11fb10 con 0x7fecec09e0e0 2023-09-30T22:25:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecf115a700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec11dff0 con 0x7fecec09d7d0 2023-09-30T22:25:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecf115a700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0a20d0 con 0x7fecec130e80 2023-09-30T22:25:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec11c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec09e0e0 0x7fecec11c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec11c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52402/0 (socket says 172.21.15.89:52402) 2023-09-30T22:25:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec09e0e0 0x7fecec11c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44326/0 (socket says 172.21.15.89:44326) 2023-09-30T22:25:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 -- 172.21.15.89:0/60004093 learned_addr learned my addr 172.21.15.89:0/60004093 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7feceb59e700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec130e80 0x7fecec11cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 -- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec130e80 msgr2=0x7fecec11cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec130e80 0x7fecec11cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 -- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec09e0e0 msgr2=0x7fecec11c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec09e0e0 0x7fecec11c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 -- 172.21.15.89:0/60004093 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc009550 con 0x7fecec09d7d0 2023-09-30T22:25:51.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7feceb59e700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec130e80 0x7fecec11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecea59c700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec09e0e0 0x7fecec11c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecead9d700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec11c2d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00e9b0 tx=0x7fecdc00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.675+0000 7fecd37fe700 1 -- 172.21.15.89:0/60004093 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00ee50 con 0x7fecec09d7d0 2023-09-30T22:25:51.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.676+0000 7fecd37fe700 1 -- 172.21.15.89:0/60004093 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc00c850 con 0x7fecec09d7d0 2023-09-30T22:25:51.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.676+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece4009a70 con 0x7fecec09d7d0 2023-09-30T22:25:51.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.676+0000 7fecd37fe700 1 -- 172.21.15.89:0/60004093 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00cbc0 con 0x7fecec09d7d0 2023-09-30T22:25:51.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.676+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec12de00 con 0x7fecec09d7d0 2023-09-30T22:25:51.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.676+0000 7fecd37fe700 1 -- 172.21.15.89:0/60004093 <== mon.1 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fecdc00cd20 con 0x7fecec09d7d0 2023-09-30T22:25:51.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.676+0000 7fecd37fe700 1 -- 172.21.15.89:0/60004093 <== mon.1 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fecdc03fac0 con 0x7fecec09d7d0 2023-09-30T22:25:51.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.676+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec120640 con 0x7fecec09d7d0 2023-09-30T22:25:51.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.681+0000 7fecd37fe700 1 -- 172.21.15.89:0/60004093 <== mon.1 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fecdc01b3b0 con 0x7fecec09d7d0 2023-09-30T22:25:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.863+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fecec04f2a0 con 0x7fecec09d7d0 2023-09-30T22:25:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.863+0000 7fecd37fe700 1 -- 172.21.15.89:0/60004093 <== mon.1 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fecdc01ab50 con 0x7fecec09d7d0 2023-09-30T22:25:51.864 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:25:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 msgr2=0x7fecec11c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec11c2d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00e9b0 tx=0x7fecdc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 shutdown_connections 2023-09-30T22:25:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecec130e80 0x7fecec11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fecec09d7d0 0x7fecec11c2d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 --2- 172.21.15.89:0/60004093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecec09e0e0 0x7fecec11c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 >> 172.21.15.89:0/60004093 conn(0x7fecec066d70 msgr2=0x7fecec12e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 shutdown_connections 2023-09-30T22:25:51.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:51.865+0000 7fecf115a700 1 -- 172.21.15.89:0/60004093 wait complete. 2023-09-30T22:25:51.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:25:52.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa9304a0700 1 Processor -- start 2023-09-30T22:25:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa9304a0700 1 -- start start 2023-09-30T22:25:52.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa9304a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9281339a0 0x7fa928133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa9304a0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa92806a770 0x7fa92806ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa9304a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806b0c0 0x7fa92813a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa9304a0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92809e770 con 0x7fa92806a770 2023-09-30T22:25:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa9304a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92809e900 con 0x7fa9281339a0 2023-09-30T22:25:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa9304a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281337a0 con 0x7fa92806b0c0 2023-09-30T22:25:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa92e23c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9281339a0 0x7fa928133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa92da3b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa92806a770 0x7fa92806ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa92ea3d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806b0c0 0x7fa92813a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa92da3b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa92806a770 0x7fa92806ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44328/0 (socket says 172.21.15.89:44328) 2023-09-30T22:25:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa92e23c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9281339a0 0x7fa928133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52404/0 (socket says 172.21.15.89:52404) 2023-09-30T22:25:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa92ea3d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806b0c0 0x7fa92813a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58222/0 (socket says 172.21.15.89:58222) 2023-09-30T22:25:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.001+0000 7fa92e23c700 1 -- 172.21.15.89:0/3888277616 learned_addr learned my addr 172.21.15.89:0/3888277616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92e23c700 1 -- 172.21.15.89:0/3888277616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806b0c0 msgr2=0x7fa92813a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92e23c700 1 --2- 172.21.15.89:0/3888277616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806b0c0 0x7fa92813a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92e23c700 1 -- 172.21.15.89:0/3888277616 >> v2:172.21.15.89:3300/0 conn(0x7fa92806a770 msgr2=0x7fa92806ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92e23c700 1 --2- 172.21.15.89:0/3888277616 >> v2:172.21.15.89:3300/0 conn(0x7fa92806a770 0x7fa92806ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92e23c700 1 -- 172.21.15.89:0/3888277616 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92809f670 con 0x7fa9281339a0 2023-09-30T22:25:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92da3b700 1 --2- 172.21.15.89:0/3888277616 >> v2:172.21.15.89:3300/0 conn(0x7fa92806a770 0x7fa92806ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92ea3d700 1 --2- 172.21.15.89:0/3888277616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806b0c0 0x7fa92813a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92e23c700 1 --2- 172.21.15.89:0/3888277616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9281339a0 0x7fa928133d70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa9240098f0 tx=0x7fa924009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429f3bbf3d9b4d25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92d23a700 1 -- 172.21.15.89:0/3888277616 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924011830 con 0x7fa9281339a0 2023-09-30T22:25:52.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92d23a700 1 -- 172.21.15.89:0/3888277616 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa924011990 con 0x7fa9281339a0 2023-09-30T22:25:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa92d23a700 1 -- 172.21.15.89:0/3888277616 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924011d20 con 0x7fa9281339a0 2023-09-30T22:25:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.002+0000 7fa9304a0700 1 -- 172.21.15.89:0/3888277616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9281339a0 msgr2=0x7fa928133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 --2- 172.21.15.89:0/3888277616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9281339a0 0x7fa928133d70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa9240098f0 tx=0x7fa924009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 -- 172.21.15.89:0/3888277616 shutdown_connections 2023-09-30T22:25:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 --2- 172.21.15.89:0/3888277616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806b0c0 0x7fa92813a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 --2- 172.21.15.89:0/3888277616 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa9281339a0 0x7fa928133d70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 --2- 172.21.15.89:0/3888277616 >> v2:172.21.15.89:3300/0 conn(0x7fa92806a770 0x7fa92806ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 -- 172.21.15.89:0/3888277616 >> 172.21.15.89:0/3888277616 conn(0x7fa928066d70 msgr2=0x7fa9280a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 -- 172.21.15.89:0/3888277616 shutdown_connections 2023-09-30T22:25:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 -- 172.21.15.89:0/3888277616 wait complete. 2023-09-30T22:25:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.003+0000 7fa9304a0700 1 Processor -- start 2023-09-30T22:25:52.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa9304a0700 1 -- start start 2023-09-30T22:25:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa9304a0700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806a770 0x7fa92811d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa9304a0700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa92806b0c0 0x7fa92811de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa9304a0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9281339a0 0x7fa92811e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa9304a0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281310e0 con 0x7fa9281339a0 2023-09-30T22:25:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa9304a0700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92811ff70 con 0x7fa92806b0c0 2023-09-30T22:25:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa9304a0700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281208e0 con 0x7fa92806a770 2023-09-30T22:25:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9281339a0 0x7fa92811e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92da3b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa92806b0c0 0x7fa92811de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9281339a0 0x7fa92811e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44340/0 (socket says 172.21.15.89:44340) 2023-09-30T22:25:52.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92da3b700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa92806b0c0 0x7fa92811de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52412/0 (socket says 172.21.15.89:52412) 2023-09-30T22:25:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 -- 172.21.15.89:0/1667277940 learned_addr learned my addr 172.21.15.89:0/1667277940 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92e23c700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806a770 0x7fa92811d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 -- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806a770 msgr2=0x7fa92811d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806a770 0x7fa92811d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 -- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa92806b0c0 msgr2=0x7fa92811de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa92806b0c0 0x7fa92811de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 -- 172.21.15.89:0/1667277940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9180094a0 con 0x7fa9281339a0 2023-09-30T22:25:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92da3b700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa92806b0c0 0x7fa92811de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.004+0000 7fa92ea3d700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9281339a0 0x7fa92811e3b0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa92000efc0 tx=0x7fa92000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.005+0000 7fa91effd700 1 -- 172.21.15.89:0/1667277940 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000ca90 con 0x7fa9281339a0 2023-09-30T22:25:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.005+0000 7fa91effd700 1 -- 172.21.15.89:0/1667277940 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92000f040 con 0x7fa9281339a0 2023-09-30T22:25:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.005+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa920009a70 con 0x7fa9281339a0 2023-09-30T22:25:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.005+0000 7fa91effd700 1 -- 172.21.15.89:0/1667277940 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000ce10 con 0x7fa9281339a0 2023-09-30T22:25:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.005+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa924009550 con 0x7fa9281339a0 2023-09-30T22:25:52.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.005+0000 7fa91effd700 1 -- 172.21.15.89:0/1667277940 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa920014480 con 0x7fa9281339a0 2023-09-30T22:25:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.006+0000 7fa91effd700 1 -- 172.21.15.89:0/1667277940 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa92003fc90 con 0x7fa9281339a0 2023-09-30T22:25:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.006+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92804f2a0 con 0x7fa9281339a0 2023-09-30T22:25:52.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.010+0000 7fa91effd700 1 -- 172.21.15.89:0/1667277940 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa920012930 con 0x7fa9281339a0 2023-09-30T22:25:52.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.192+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa92811e9f0 con 0x7fa9281339a0 2023-09-30T22:25:52.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.193+0000 7fa91effd700 1 -- 172.21.15.89:0/1667277940 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa920012340 con 0x7fa9281339a0 2023-09-30T22:25:52.194 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-09-30T22:25:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.194+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9281339a0 msgr2=0x7fa92811e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.194+0000 7fa9304a0700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9281339a0 0x7fa92811e3b0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa92000efc0 tx=0x7fa92000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.194+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 shutdown_connections 2023-09-30T22:25:52.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.194+0000 7fa9304a0700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9281339a0 0x7fa92811e3b0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.194+0000 7fa9304a0700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa92806a770 0x7fa92811d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.194+0000 7fa9304a0700 1 --2- 172.21.15.89:0/1667277940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa92806b0c0 0x7fa92811de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.194+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 >> 172.21.15.89:0/1667277940 conn(0x7fa928066d70 msgr2=0x7fa92812fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.194+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 shutdown_connections 2023-09-30T22:25:52.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.195+0000 7fa9304a0700 1 -- 172.21.15.89:0/1667277940 wait complete. 2023-09-30T22:25:52.207 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:25:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.331+0000 7fe54527d700 1 Processor -- start 2023-09-30T22:25:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe54527d700 1 -- start start 2023-09-30T22:25:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe54527d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe540136b80 0x7fe540138f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe54527d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5401394a0 0x7fe54013b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe54527d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5400a0280 con 0x7fe540071d60 2023-09-30T22:25:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe54527d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400a0020 con 0x7fe540136b80 2023-09-30T22:25:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe54527d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe54009f860 con 0x7fe5401394a0 2023-09-30T22:25:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53f7fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5401394a0 0x7fe54013b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53effd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe540071d60 0x7fe540072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35012/0 (socket says 172.21.15.89:35012) 2023-09-30T22:25:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53effd700 1 -- 172.21.15.89:0/858653480 learned_addr learned my addr 172.21.15.89:0/858653480 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:25:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53e7fc700 1 --2- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe540136b80 0x7fe540138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53f7fe700 1 -- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe540136b80 msgr2=0x7fe540138f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53f7fe700 1 --2- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe540136b80 0x7fe540138f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53f7fe700 1 -- 172.21.15.89:0/858653480 >> v1:172.21.15.89:6789/0 conn(0x7fe540071d60 legacy=0x7fe540072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53dffb700 1 -- 172.21.15.89:0/858653480 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790071378 0 0) 0x7fe5400a0280 con 0x7fe540071d60 2023-09-30T22:25:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53f7fe700 1 -- 172.21.15.89:0/858653480 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54006ba00 con 0x7fe5401394a0 2023-09-30T22:25:52.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.332+0000 7fe53e7fc700 1 --2- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe540136b80 0x7fe540138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe53f7fe700 1 --2- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5401394a0 0x7fe54013b880 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe538009dc0 tx=0x7fe53800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8f5d7e544c33c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe53dffb700 1 -- 172.21.15.89:0/858653480 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53800e070 con 0x7fe5401394a0 2023-09-30T22:25:52.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe53dffb700 1 -- 172.21.15.89:0/858653480 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53800bb20 con 0x7fe5401394a0 2023-09-30T22:25:52.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe53dffb700 1 -- 172.21.15.89:0/858653480 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538012430 con 0x7fe5401394a0 2023-09-30T22:25:52.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe54527d700 1 -- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5401394a0 msgr2=0x7fe54013b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe54527d700 1 --2- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5401394a0 0x7fe54013b880 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe538009dc0 tx=0x7fe53800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:52.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe54527d700 1 -- 172.21.15.89:0/858653480 shutdown_connections 2023-09-30T22:25:52.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe54527d700 1 --2- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe5401394a0 0x7fe54013b880 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe54527d700 1 --2- 172.21.15.89:0/858653480 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe540136b80 0x7fe540138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe54527d700 1 -- 172.21.15.89:0/858653480 >> 172.21.15.89:0/858653480 conn(0x7fe5400a4130 msgr2=0x7fe5400a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe54527d700 1 -- 172.21.15.89:0/858653480 shutdown_connections 2023-09-30T22:25:52.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.333+0000 7fe54527d700 1 -- 172.21.15.89:0/858653480 wait complete. 2023-09-30T22:25:52.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.334+0000 7fe54527d700 1 Processor -- start 2023-09-30T22:25:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.334+0000 7fe54527d700 1 -- start start 2023-09-30T22:25:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe54527d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe540071d60 0x7fe540120870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe54527d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540136b80 0x7fe540120db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe54527d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5401394a0 0x7fe5401212f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe54527d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe540136130 con 0x7fe540136b80 2023-09-30T22:25:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe54527d700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe540136710 con 0x7fe5401394a0 2023-09-30T22:25:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe54527d700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe54009f4e0 con 0x7fe540071d60 2023-09-30T22:25:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe540071d60 0x7fe540120870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540136b80 0x7fe540120db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540136b80 0x7fe540120db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44356/0 (socket says 172.21.15.89:44356) 2023-09-30T22:25:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53effd700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe540071d60 0x7fe540120870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58246/0 (socket says 172.21.15.89:58246) 2023-09-30T22:25:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53f7fe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5401394a0 0x7fe5401212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 -- 172.21.15.89:0/1880453915 learned_addr learned my addr 172.21.15.89:0/1880453915 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 -- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe540071d60 msgr2=0x7fe540120870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe540071d60 0x7fe540120870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 -- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5401394a0 msgr2=0x7fe5401212f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5401394a0 0x7fe5401212f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 -- 172.21.15.89:0/1880453915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300094a0 con 0x7fe540136b80 2023-09-30T22:25:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53f7fe700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5401394a0 0x7fe5401212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53effd700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe540071d60 0x7fe540120870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe53e7fc700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540136b80 0x7fe540120db0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe53000ebe0 tx=0x7fe53000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.335+0000 7fe5277fe700 1 -- 172.21.15.89:0/1880453915 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000c850 con 0x7fe540136b80 2023-09-30T22:25:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.336+0000 7fe5277fe700 1 -- 172.21.15.89:0/1880453915 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53000c9b0 con 0x7fe540136b80 2023-09-30T22:25:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.336+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe538009a70 con 0x7fe540136b80 2023-09-30T22:25:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.336+0000 7fe5277fe700 1 -- 172.21.15.89:0/1880453915 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000cd40 con 0x7fe540136b80 2023-09-30T22:25:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.336+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe540122c90 con 0x7fe540136b80 2023-09-30T22:25:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.336+0000 7fe5277fe700 1 -- 172.21.15.89:0/1880453915 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe530013470 con 0x7fe540136b80 2023-09-30T22:25:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.336+0000 7fe5277fe700 1 -- 172.21.15.89:0/1880453915 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe53003fb30 con 0x7fe540136b80 2023-09-30T22:25:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.337+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54004f2a0 con 0x7fe540136b80 2023-09-30T22:25:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.341+0000 7fe5277fe700 1 -- 172.21.15.89:0/1880453915 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe530013720 con 0x7fe540136b80 2023-09-30T22:25:52.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.523+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5400728b0 con 0x7fe540136b80 2023-09-30T22:25:52.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.523+0000 7fe5277fe700 1 -- 172.21.15.89:0/1880453915 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe530021030 con 0x7fe540136b80 2023-09-30T22:25:52.524 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-09-30T22:25:52.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540136b80 msgr2=0x7fe540120db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540136b80 0x7fe540120db0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe53000ebe0 tx=0x7fe53000eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 shutdown_connections 2023-09-30T22:25:52.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe5401394a0 0x7fe5401212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe540071d60 0x7fe540120870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 --2- 172.21.15.89:0/1880453915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe540136b80 0x7fe540120db0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 >> 172.21.15.89:0/1880453915 conn(0x7fe5400a4130 msgr2=0x7fe5400a1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 shutdown_connections 2023-09-30T22:25:52.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.525+0000 7fe54527d700 1 -- 172.21.15.89:0/1880453915 wait complete. 2023-09-30T22:25:52.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-30T22:25:52.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.659+0000 7fbedcc16700 1 Processor -- start 2023-09-30T22:25:52.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.659+0000 7fbedcc16700 1 -- start start 2023-09-30T22:25:52.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbedcc16700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbed809d6f0 0x7fbed809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbedcc16700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbedcc16700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed809f130 con 0x7fbed809e000 2023-09-30T22:25:52.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbedcc16700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed809f520 con 0x7fbed809d6f0 2023-09-30T22:25:52.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbedcc16700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed809e930 con 0x7fbed809e410 2023-09-30T22:25:52.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbed809d6f0 0x7fbed809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed6d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52442/0 (socket says 172.21.15.89:52442) 2023-09-30T22:25:52.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbed809d6f0 0x7fbed809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44358/0 (socket says 172.21.15.89:44358) 2023-09-30T22:25:52.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed6d9d700 1 -- 172.21.15.89:0/1280356522 learned_addr learned my addr 172.21.15.89:0/1280356522 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:52.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed6d9d700 1 -- 172.21.15.89:0/1280356522 >> v2:172.21.15.89:3300/0 conn(0x7fbed809d6f0 msgr2=0x7fbed809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed6d9d700 1 --2- 172.21.15.89:0/1280356522 >> v2:172.21.15.89:3300/0 conn(0x7fbed809d6f0 0x7fbed809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed6d9d700 1 -- 172.21.15.89:0/1280356522 >> v1:172.21.15.89:6789/0 conn(0x7fbed809e000 legacy=0x7fbed8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed6d9d700 1 -- 172.21.15.89:0/1280356522 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed809b2d0 con 0x7fbed809e410 2023-09-30T22:25:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.660+0000 7fbed659c700 1 --2- 172.21.15.89:0/1280356522 >> v2:172.21.15.89:3300/0 conn(0x7fbed809d6f0 0x7fbed809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbed6d9d700 1 --2- 172.21.15.89:0/1280356522 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8130d70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009e50 tx=0x7fbed000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a02e3ad3f585775 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbed559a700 1 -- 172.21.15.89:0/1280356522 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000e070 con 0x7fbed809e410 2023-09-30T22:25:52.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbed559a700 1 -- 172.21.15.89:0/1280356522 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed000b910 con 0x7fbed809e410 2023-09-30T22:25:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbed559a700 1 -- 172.21.15.89:0/1280356522 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000bcc0 con 0x7fbed809e410 2023-09-30T22:25:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbedcc16700 1 -- 172.21.15.89:0/1280356522 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 msgr2=0x7fbed8130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbedcc16700 1 --2- 172.21.15.89:0/1280356522 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8130d70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009e50 tx=0x7fbed000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbedcc16700 1 -- 172.21.15.89:0/1280356522 shutdown_connections 2023-09-30T22:25:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbedcc16700 1 --2- 172.21.15.89:0/1280356522 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8130d70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbedcc16700 1 --2- 172.21.15.89:0/1280356522 >> v2:172.21.15.89:3300/0 conn(0x7fbed809d6f0 0x7fbed809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbedcc16700 1 -- 172.21.15.89:0/1280356522 >> 172.21.15.89:0/1280356522 conn(0x7fbed8066f50 msgr2=0x7fbed809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbedcc16700 1 -- 172.21.15.89:0/1280356522 shutdown_connections 2023-09-30T22:25:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.661+0000 7fbedcc16700 1 -- 172.21.15.89:0/1280356522 wait complete. 2023-09-30T22:25:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.662+0000 7fbedcc16700 1 Processor -- start 2023-09-30T22:25:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.662+0000 7fbedcc16700 1 -- start start 2023-09-30T22:25:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbedcc16700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbed809d6f0 0x7fbed8121110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbedcc16700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbed809e000 0x7fbed8121650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbedcc16700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8121b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbedcc16700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed81245f0 con 0x7fbed809d6f0 2023-09-30T22:25:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbedcc16700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed809c700 con 0x7fbed809e410 2023-09-30T22:25:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbedcc16700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8072640 con 0x7fbed809e000 2023-09-30T22:25:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbed809d6f0 0x7fbed8121110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbed809d6f0 0x7fbed8121110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44366/0 (socket says 172.21.15.89:44366) 2023-09-30T22:25:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 -- 172.21.15.89:0/1089746942 learned_addr learned my addr 172.21.15.89:0/1089746942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed6d9d700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8121b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed5d9b700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbed809e000 0x7fbed8121650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 -- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbed809e000 msgr2=0x7fbed8121650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbed809e000 0x7fbed8121650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 -- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 msgr2=0x7fbed8121b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8121b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 -- 172.21.15.89:0/1089746942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0094a0 con 0x7fbed809d6f0 2023-09-30T22:25:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.663+0000 7fbed659c700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbed809d6f0 0x7fbed8121110 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00ebe0 tx=0x7fbecc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.664+0000 7fbec6ffd700 1 -- 172.21.15.89:0/1089746942 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00c7f0 con 0x7fbed809d6f0 2023-09-30T22:25:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.664+0000 7fbec6ffd700 1 -- 172.21.15.89:0/1089746942 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc00c950 con 0x7fbed809d6f0 2023-09-30T22:25:52.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.664+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed0009b30 con 0x7fbed809d6f0 2023-09-30T22:25:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.664+0000 7fbec6ffd700 1 -- 172.21.15.89:0/1089746942 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00cce0 con 0x7fbed809d6f0 2023-09-30T22:25:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.664+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed8123cb0 con 0x7fbed809d6f0 2023-09-30T22:25:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.664+0000 7fbec6ffd700 1 -- 172.21.15.89:0/1089746942 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbecc013470 con 0x7fbed809d6f0 2023-09-30T22:25:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.664+0000 7fbec6ffd700 1 -- 172.21.15.89:0/1089746942 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbecc03fb40 con 0x7fbed809d6f0 2023-09-30T22:25:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.664+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed8074ea0 con 0x7fbed809d6f0 2023-09-30T22:25:52.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.669+0000 7fbec6ffd700 1 -- 172.21.15.89:0/1089746942 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbecc017030 con 0x7fbed809d6f0 2023-09-30T22:25:52.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.864+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbed81224f0 con 0x7fbed809d6f0 2023-09-30T22:25:52.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.865+0000 7fbec6ffd700 1 -- 172.21.15.89:0/1089746942 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v40) v1 ==== 81+0+639 (secure 0 0 0) 0x7fbecc02b080 con 0x7fbed809d6f0 2023-09-30T22:25:52.866 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2456756709 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:52.866 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4019526010 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:52.867 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2579157783 2023-10-01T22:16:19.404121+0000 2023-09-30T22:25:52.867 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6816/118161 2023-10-01T22:16:19.404121+0000 2023-09-30T22:25:52.867 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1269574253 2023-10-01T22:16:19.404121+0000 2023-09-30T22:25:52.867 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6817/3539463463 2023-10-01T22:16:43.298028+0000 2023-09-30T22:25:52.867 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2736305244 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:52.867 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6816/126953 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:52.867 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4100667825 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:52.867 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1785021332 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:52.868 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1295992953 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:52.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbed809d6f0 msgr2=0x7fbed8121110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbed809d6f0 0x7fbed8121110 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00ebe0 tx=0x7fbecc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 shutdown_connections 2023-09-30T22:25:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbed809e410 0x7fbed8121b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbed809d6f0 0x7fbed8121110 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 --2- 172.21.15.89:0/1089746942 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbed809e000 0x7fbed8121650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 >> 172.21.15.89:0/1089746942 conn(0x7fbed8066f50 msgr2=0x7fbed80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 shutdown_connections 2023-09-30T22:25:52.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:52.867+0000 7fbedcc16700 1 -- 172.21.15.89:0/1089746942 wait complete. 2023-09-30T22:25:52.870 INFO:teuthology.orchestra.run.smithi089.stderr:listed 11 entries 2023-09-30T22:25:52.880 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-09-30T22:25:52.880 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:25:52.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:25:53.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.021+0000 7f6a4013f700 1 Processor -- start 2023-09-30T22:25:53.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.021+0000 7f6a4013f700 1 -- start start 2023-09-30T22:25:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a4013f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809d6f0 0x7f6a3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a4013f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6a3809e410 0x7f6a38134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a4013f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3809b120 con 0x7f6a3809e000 2023-09-30T22:25:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a4013f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3806b460 con 0x7f6a3809e410 2023-09-30T22:25:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a4013f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3809a7e0 con 0x7f6a3809d6f0 2023-09-30T22:25:53.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3d6da700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6a3809e000 0x7f6a381339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35028/0 (socket says 172.21.15.89:35028) 2023-09-30T22:25:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3d6da700 1 -- 172.21.15.89:0/628219931 learned_addr learned my addr 172.21.15.89:0/628219931 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-09-30T22:25:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3dedb700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809d6f0 0x7f6a3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3e6dc700 1 --2- 172.21.15.89:0/628219931 >> v2:172.21.15.89:3300/0 conn(0x7f6a3809e410 0x7f6a38134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3dedb700 1 -- 172.21.15.89:0/628219931 >> v2:172.21.15.89:3300/0 conn(0x7f6a3809e410 msgr2=0x7f6a38134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3dedb700 1 --2- 172.21.15.89:0/628219931 >> v2:172.21.15.89:3300/0 conn(0x7f6a3809e410 0x7f6a38134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3dedb700 1 -- 172.21.15.89:0/628219931 >> v1:172.21.15.89:6789/0 conn(0x7f6a3809e000 legacy=0x7f6a381339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3dedb700 1 -- 172.21.15.89:0/628219931 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3809a500 con 0x7f6a3809d6f0 2023-09-30T22:25:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3e6dc700 1 --2- 172.21.15.89:0/628219931 >> v2:172.21.15.89:3300/0 conn(0x7f6a3809e410 0x7f6a38134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.022+0000 7f6a3dedb700 1 --2- 172.21.15.89:0/628219931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809d6f0 0x7f6a3809dac0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009870 tx=0x7f6a28009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e58f2d4c9fb9e8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a3ced9700 1 -- 172.21.15.89:0/628219931 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28011830 con 0x7f6a3809d6f0 2023-09-30T22:25:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a3ced9700 1 -- 172.21.15.89:0/628219931 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a28011990 con 0x7f6a3809d6f0 2023-09-30T22:25:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a3ced9700 1 -- 172.21.15.89:0/628219931 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28011d20 con 0x7f6a3809d6f0 2023-09-30T22:25:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a4013f700 1 -- 172.21.15.89:0/628219931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809d6f0 msgr2=0x7f6a3809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a4013f700 1 --2- 172.21.15.89:0/628219931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809d6f0 0x7f6a3809dac0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009870 tx=0x7f6a28009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:53.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a4013f700 1 -- 172.21.15.89:0/628219931 shutdown_connections 2023-09-30T22:25:53.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a4013f700 1 --2- 172.21.15.89:0/628219931 >> v2:172.21.15.89:3300/0 conn(0x7f6a3809e410 0x7f6a38134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a4013f700 1 --2- 172.21.15.89:0/628219931 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809d6f0 0x7f6a3809dac0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a4013f700 1 -- 172.21.15.89:0/628219931 >> 172.21.15.89:0/628219931 conn(0x7f6a38066f50 msgr2=0x7f6a3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a4013f700 1 -- 172.21.15.89:0/628219931 shutdown_connections 2023-09-30T22:25:53.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.023+0000 7f6a4013f700 1 -- 172.21.15.89:0/628219931 wait complete. 2023-09-30T22:25:53.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.024+0000 7f6a4013f700 1 Processor -- start 2023-09-30T22:25:53.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a4013f700 1 -- start start 2023-09-30T22:25:53.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a4013f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a3809d6f0 0x7f6a381a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a4013f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a3809e000 0x7f6a381a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a4013f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809e410 0x7f6a381a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a4013f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a381a9b00 con 0x7f6a3809d6f0 2023-09-30T22:25:53.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a4013f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a381a81d0 con 0x7f6a3809e410 2023-09-30T22:25:53.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a4013f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a381a9290 con 0x7f6a3809e000 2023-09-30T22:25:53.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a3dedb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a3809d6f0 0x7f6a381a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a3dedb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a3809d6f0 0x7f6a381a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44386/0 (socket says 172.21.15.89:44386) 2023-09-30T22:25:53.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a3dedb700 1 -- 172.21.15.89:0/3328014161 learned_addr learned my addr 172.21.15.89:0/3328014161 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:53.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a3d6da700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a3809e000 0x7f6a381a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.025+0000 7f6a3e6dc700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809e410 0x7f6a381a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a3dedb700 1 -- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a3809e000 msgr2=0x7f6a381a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a3dedb700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a3809e000 0x7f6a381a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a3dedb700 1 -- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809e410 msgr2=0x7f6a381a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a3dedb700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809e410 0x7f6a381a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a3dedb700 1 -- 172.21.15.89:0/3328014161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30009a70 con 0x7f6a3809d6f0 2023-09-30T22:25:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a3d6da700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a3809e000 0x7f6a381a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a3e6dc700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809e410 0x7f6a381a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a3dedb700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a3809d6f0 0x7f6a381a6070 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f6a280097c0 tx=0x7f6a28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a2e7fc700 1 -- 172.21.15.89:0/3328014161 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a280045d0 con 0x7f6a3809d6f0 2023-09-30T22:25:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a2e7fc700 1 -- 172.21.15.89:0/3328014161 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2801f070 con 0x7f6a3809d6f0 2023-09-30T22:25:53.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a280094a0 con 0x7f6a3809d6f0 2023-09-30T22:25:53.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a2e7fc700 1 -- 172.21.15.89:0/3328014161 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a280038a0 con 0x7f6a3809d6f0 2023-09-30T22:25:53.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.026+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3809c760 con 0x7f6a3809d6f0 2023-09-30T22:25:53.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.027+0000 7f6a2e7fc700 1 -- 172.21.15.89:0/3328014161 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a28003a00 con 0x7f6a3809d6f0 2023-09-30T22:25:53.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.027+0000 7f6a2e7fc700 1 -- 172.21.15.89:0/3328014161 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6a28021070 con 0x7f6a3809d6f0 2023-09-30T22:25:53.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.027+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a38074ea0 con 0x7f6a3809d6f0 2023-09-30T22:25:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.032+0000 7f6a2e7fc700 1 -- 172.21.15.89:0/3328014161 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a2802b070 con 0x7f6a3809d6f0 2023-09-30T22:25:53.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.218+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a381a9680 con 0x7f6a3809d6f0 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.219+0000 7f6a2e7fc700 1 -- 172.21.15.89:0/3328014161 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13399 (secure 0 0 0) 0x7f6a381a9680 con 0x7f6a3809d6f0 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":40,"fsid":"389f6e95-ea58-47bd-a145-778ba0724a9e","created":"2023-09-30T22:13:45.747992+0000","modified":"2023-09-30T22:17:18.222304+0000","last_up_change":"2023-09-30T22:16:38.305512+0000","last_in_change":"2023-09-30T22:13:47.831690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:13:54.564869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:13:55.740519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:14:00.533280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:14:01.011104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96fbca95-1fa5-4074-9fbb-d4f206bb1aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6804","nonce":129669}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6805","nonce":129669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6807","nonce":129669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6806","nonce":129669}]},"public_addr":"172.21.15.89:6804/129669","cluster_addr":"172.21.15.89:6805/129669","heartbeat_back_addr":"172.21.15.89:6807/129669","heartbeat_front_addr":"172.21.15.89:6806/129669","state":["exists","up"]},{"osd":1,"uuid":"bca437d5-1c37-4cda-a2f2-86f7957e0ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6800","nonce":129767}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6801","nonce":129767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6803","nonce":129767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6802","nonce":129767}]},"public_addr":"172.21.15.89:6800/129767","cluster_addr":"172.21.15.89:6801/129767","heartbeat_back_addr":"172.21.15.89:6803/129767","heartbeat_front_addr":"172.21.15.89:6802/129767","state":["exists","up"]},{"osd":2,"uuid":"0fb339c9-0000-43f3-bcbc-9cc5b8b65359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6808","nonce":129877}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6809","nonce":129877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6811","nonce":129877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6810","nonce":129877}]},"public_addr":"172.21.15.89:6808/129877","cluster_addr":"172.21.15.89:6809/129877","heartbeat_back_addr":"172.21.15.89:6811/129877","heartbeat_front_addr":"172.21.15.89:6810/129877","state":["exists","up"]},{"osd":3,"uuid":"4010d897-d6fa-4f8c-ab79-7415116d796d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6812","nonce":130217}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6813","nonce":130217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6815","nonce":130217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.89:6814","nonce":130217}]},"public_addr":"172.21.15.89:6812/130217","cluster_addr":"172.21.15.89:6813/130217","heartbeat_back_addr":"172.21.15.89:6815/130217","heartbeat_front_addr":"172.21.15.89:6814/130217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.964504+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:53.039111+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.738991+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:16:32.255166+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:13:52.958026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.28","mappings":[{"from":3,"to":0}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]},{"pgid":"4.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/2456756709":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/4019526010":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/2579157783":"2023-10-01T22:16:19.404121+0000","172.21.15.89:6816/118161":"2023-10-01T22:16:19.404121+0000","172.21.15.89:0/1269574253":"2023-10-01T22:16:19.404121+0000","172.21.15.89:6817/3539463463":"2023-10-01T22:16:43.298028+0000","172.21.15.89:0/2736305244":"2023-10-01T22:17:18.222274+0000","172.21.15.89:6816/126953":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/4100667825":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/1785021332":"2023-10-01T22:17:18.222274+0000","172.21.15.89:0/1295992953":"2023-10-01T22:17:18.222274+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:25:53.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a3809d6f0 msgr2=0x7f6a381a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a3809d6f0 0x7f6a381a6070 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f6a280097c0 tx=0x7f6a28004750 comp rx=0 tx=0).stop 2023-09-30T22:25:53.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 shutdown_connections 2023-09-30T22:25:53.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6a3809e410 0x7f6a381a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a3809d6f0 0x7f6a381a6070 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 --2- 172.21.15.89:0/3328014161 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a3809e000 0x7f6a381a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 >> 172.21.15.89:0/3328014161 conn(0x7f6a38066f50 msgr2=0x7f6a381baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 shutdown_connections 2023-09-30T22:25:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.221+0000 7f6a4013f700 1 -- 172.21.15.89:0/3328014161 wait complete. 2023-09-30T22:25:53.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:13:54.564869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:13:55.740519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:14:00.533280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:14:01.011104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:25:53.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:25:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.360+0000 7faf0e65f700 1 Processor -- start 2023-09-30T22:25:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.360+0000 7faf0e65f700 1 -- start start 2023-09-30T22:25:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0e65f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf08133cc0 0x7faf08134090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0e65f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf080a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0e65f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faf080a0f70 0x7faf080a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0e65f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf080a0640 con 0x7faf080a0f70 2023-09-30T22:25:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0e65f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0809e1f0 con 0x7faf08133cc0 2023-09-30T22:25:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0e65f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf080a23b0 con 0x7faf081345d0 2023-09-30T22:25:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faf080a0f70 0x7faf080a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf077fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf080a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf07fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf08133cc0 0x7faf08134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faf080a0f70 0x7faf080a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44402/0 (socket says 172.21.15.89:44402) 2023-09-30T22:25:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 -- 172.21.15.89:0/2790344857 learned_addr learned my addr 172.21.15.89:0/2790344857 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf07fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf08133cc0 0x7faf08134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52482/0 (socket says 172.21.15.89:52482) 2023-09-30T22:25:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf077fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf080a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58266/0 (socket says 172.21.15.89:58266) 2023-09-30T22:25:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 -- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 msgr2=0x7faf080a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf080a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 -- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf08133cc0 msgr2=0x7faf08134090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf08133cc0 0x7faf08134090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 -- 172.21.15.89:0/2790344857 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0809df30 con 0x7faf080a0f70 2023-09-30T22:25:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf077fe700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf080a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf07fff700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf08133cc0 0x7faf08134090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.361+0000 7faf0cbfc700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf080a0f70 0x7faf080a1410 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7faf00009e80 tx=0x7faf0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32380dffb35e812a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf06ffd700 1 -- 172.21.15.89:0/2790344857 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0000e070 con 0x7faf080a0f70 2023-09-30T22:25:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf06ffd700 1 -- 172.21.15.89:0/2790344857 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0000b8d0 con 0x7faf080a0f70 2023-09-30T22:25:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf06ffd700 1 -- 172.21.15.89:0/2790344857 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0000bc80 con 0x7faf080a0f70 2023-09-30T22:25:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 -- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf080a0f70 msgr2=0x7faf080a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf080a0f70 0x7faf080a1410 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7faf00009e80 tx=0x7faf0000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:53.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 -- 172.21.15.89:0/2790344857 shutdown_connections 2023-09-30T22:25:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf080a0f70 0x7faf080a1410 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf08133cc0 0x7faf08134090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 --2- 172.21.15.89:0/2790344857 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf080a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 -- 172.21.15.89:0/2790344857 >> 172.21.15.89:0/2790344857 conn(0x7faf080a3050 msgr2=0x7faf080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 -- 172.21.15.89:0/2790344857 shutdown_connections 2023-09-30T22:25:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.362+0000 7faf0e65f700 1 -- 172.21.15.89:0/2790344857 wait complete. 2023-09-30T22:25:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.363+0000 7faf0e65f700 1 Processor -- start 2023-09-30T22:25:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.363+0000 7faf0e65f700 1 -- start start 2023-09-30T22:25:53.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0e65f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf080a0f70 0x7faf0811e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0e65f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf08133cc0 0x7faf0811e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0e65f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf0811ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0e65f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf08121050 con 0x7faf08133cc0 2023-09-30T22:25:53.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0e65f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0811f630 con 0x7faf080a0f70 2023-09-30T22:25:53.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0e65f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf08121aa0 con 0x7faf081345d0 2023-09-30T22:25:53.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf07fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf080a0f70 0x7faf0811e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf08133cc0 0x7faf0811e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf07fff700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf080a0f70 0x7faf0811e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52494/0 (socket says 172.21.15.89:52494) 2023-09-30T22:25:53.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf077fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf08133cc0 0x7faf0811e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44418/0 (socket says 172.21.15.89:44418) 2023-09-30T22:25:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf07fff700 1 -- 172.21.15.89:0/3096505963 learned_addr learned my addr 172.21.15.89:0/3096505963 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0cbfc700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf0811ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0cbfc700 1 -- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf080a0f70 msgr2=0x7faf0811e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0cbfc700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf080a0f70 0x7faf0811e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0cbfc700 1 -- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf08133cc0 msgr2=0x7faf0811e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0cbfc700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf08133cc0 0x7faf0811e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0cbfc700 1 -- 172.21.15.89:0/3096505963 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0094a0 con 0x7faf081345d0 2023-09-30T22:25:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf07fff700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf080a0f70 0x7faf0811e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf077fe700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf08133cc0 0x7faf0811e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0cbfc700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf0811ec00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faf00015000 tx=0x7faf00004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf04ff9700 1 -- 172.21.15.89:0/3096505963 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00003e10 con 0x7faf081345d0 2023-09-30T22:25:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf04ff9700 1 -- 172.21.15.89:0/3096505963 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf00004040 con 0x7faf081345d0 2023-09-30T22:25:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef80094a0 con 0x7faf081345d0 2023-09-30T22:25:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf04ff9700 1 -- 172.21.15.89:0/3096505963 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000043f0 con 0x7faf081345d0 2023-09-30T22:25:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.364+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf00009b30 con 0x7faf081345d0 2023-09-30T22:25:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.365+0000 7faf04ff9700 1 -- 172.21.15.89:0/3096505963 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf00004550 con 0x7faf081345d0 2023-09-30T22:25:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.365+0000 7faf04ff9700 1 -- 172.21.15.89:0/3096505963 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faf00040460 con 0x7faf081345d0 2023-09-30T22:25:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.365+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf080a57f0 con 0x7faf081345d0 2023-09-30T22:25:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.370+0000 7faf04ff9700 1 -- 172.21.15.89:0/3096505963 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf00019320 con 0x7faf081345d0 2023-09-30T22:25:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.552+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf0809f870 con 0x7faf081345d0 2023-09-30T22:25:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.552+0000 7faf04ff9700 1 -- 172.21.15.89:0/3096505963 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf00011b80 con 0x7faf081345d0 2023-09-30T22:25:53.553 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-09-30T22:25:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 msgr2=0x7faf0811ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf0811ec00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faf00015000 tx=0x7faf00004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 shutdown_connections 2023-09-30T22:25:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf081345d0 0x7faf0811ec00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faf080a0f70 0x7faf0811e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 --2- 172.21.15.89:0/3096505963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf08133cc0 0x7faf0811e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 >> 172.21.15.89:0/3096505963 conn(0x7faf080a3050 msgr2=0x7faf081331e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 shutdown_connections 2023-09-30T22:25:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.554+0000 7faf0e65f700 1 -- 172.21.15.89:0/3096505963 wait complete. 2023-09-30T22:25:53.568 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:25:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.690+0000 7fa3c8312700 1 Processor -- start 2023-09-30T22:25:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.690+0000 7fa3c8312700 1 -- start start 2023-09-30T22:25:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c8312700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c0130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c8312700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa3c006a770 0x7fa3c006ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c8312700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c009f620 con 0x7fa3c006b0c0 2023-09-30T22:25:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c8312700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c0130280 con 0x7fa3c006a770 2023-09-30T22:25:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c8312700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c009e1b0 con 0x7fa3c01303c0 2023-09-30T22:25:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c60ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c0130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c60ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c0130790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58274/0 (socket says 172.21.15.89:58274) 2023-09-30T22:25:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c58ad700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa3c006a770 0x7fa3c006ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c60ae700 1 -- 172.21.15.89:0/2291766598 learned_addr learned my addr 172.21.15.89:0/2291766598 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c58ad700 1 -- 172.21.15.89:0/2291766598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 msgr2=0x7fa3c0130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c58ad700 1 --2- 172.21.15.89:0/2291766598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c0130790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c58ad700 1 -- 172.21.15.89:0/2291766598 >> v1:172.21.15.89:6789/0 conn(0x7fa3c006b0c0 legacy=0x7fa3c013a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c58ad700 1 -- 172.21.15.89:0/2291766598 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c006b540 con 0x7fa3c006a770 2023-09-30T22:25:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c60ae700 1 --2- 172.21.15.89:0/2291766598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c0130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.691+0000 7fa3c58ad700 1 --2- 172.21.15.89:0/2291766598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006a770 0x7fa3c006ab80 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009840 tx=0x7fa3bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc0b27a4109bc09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c50ac700 1 -- 172.21.15.89:0/2291766598 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc011830 con 0x7fa3c006a770 2023-09-30T22:25:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c50ac700 1 -- 172.21.15.89:0/2291766598 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc011990 con 0x7fa3c006a770 2023-09-30T22:25:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c50ac700 1 -- 172.21.15.89:0/2291766598 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc011d20 con 0x7fa3c006a770 2023-09-30T22:25:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c8312700 1 -- 172.21.15.89:0/2291766598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006a770 msgr2=0x7fa3c006ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c8312700 1 --2- 172.21.15.89:0/2291766598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006a770 0x7fa3c006ab80 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009840 tx=0x7fa3bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c8312700 1 -- 172.21.15.89:0/2291766598 shutdown_connections 2023-09-30T22:25:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c8312700 1 --2- 172.21.15.89:0/2291766598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c0130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c8312700 1 --2- 172.21.15.89:0/2291766598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006a770 0x7fa3c006ab80 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c8312700 1 -- 172.21.15.89:0/2291766598 >> 172.21.15.89:0/2291766598 conn(0x7fa3c0066d70 msgr2=0x7fa3c00a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c8312700 1 -- 172.21.15.89:0/2291766598 shutdown_connections 2023-09-30T22:25:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.692+0000 7fa3c8312700 1 -- 172.21.15.89:0/2291766598 wait complete. 2023-09-30T22:25:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.693+0000 7fa3c8312700 1 Processor -- start 2023-09-30T22:25:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.693+0000 7fa3c8312700 1 -- start start 2023-09-30T22:25:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c8312700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c006a770 0x7fa3c011d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c8312700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006b0c0 0x7fa3c011de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c8312700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c8312700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c012d110 con 0x7fa3c006b0c0 2023-09-30T22:25:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c8312700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00721d0 con 0x7fa3c006a770 2023-09-30T22:25:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c8312700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c011fb70 con 0x7fa3c01303c0 2023-09-30T22:25:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c58ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006b0c0 0x7fa3c011de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c68af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c58ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006b0c0 0x7fa3c011de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44434/0 (socket says 172.21.15.89:44434) 2023-09-30T22:25:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c60ae700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c006a770 0x7fa3c011d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c58ad700 1 -- 172.21.15.89:0/882163149 learned_addr learned my addr 172.21.15.89:0/882163149 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c68af700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c011e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58286/0 (socket says 172.21.15.89:58286) 2023-09-30T22:25:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c68af700 1 -- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c006a770 msgr2=0x7fa3c011d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c68af700 1 --2- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c006a770 0x7fa3c011d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c68af700 1 -- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006b0c0 msgr2=0x7fa3c011de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c68af700 1 --2- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006b0c0 0x7fa3c011de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c68af700 1 -- 172.21.15.89:0/882163149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0009550 con 0x7fa3c01303c0 2023-09-30T22:25:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c58ad700 1 --2- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006b0c0 0x7fa3c011de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c68af700 1 --2- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c011e3b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800cb90 tx=0x7fa3b800cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3b6ffd700 1 -- 172.21.15.89:0/882163149 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8003c00 con 0x7fa3c01303c0 2023-09-30T22:25:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3b6ffd700 1 -- 172.21.15.89:0/882163149 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b8003d60 con 0x7fa3c01303c0 2023-09-30T22:25:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc0094a0 con 0x7fa3c01303c0 2023-09-30T22:25:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3b6ffd700 1 -- 172.21.15.89:0/882163149 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b800aa80 con 0x7fa3c01303c0 2023-09-30T22:25:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.694+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c0120110 con 0x7fa3c01303c0 2023-09-30T22:25:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.695+0000 7fa3b6ffd700 1 -- 172.21.15.89:0/882163149 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa3b800abe0 con 0x7fa3c01303c0 2023-09-30T22:25:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.695+0000 7fa3b6ffd700 1 -- 172.21.15.89:0/882163149 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa3b803f500 con 0x7fa3c01303c0 2023-09-30T22:25:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.695+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c004f2a0 con 0x7fa3c01303c0 2023-09-30T22:25:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.700+0000 7fa3b6ffd700 1 -- 172.21.15.89:0/882163149 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3b80198f0 con 0x7fa3c01303c0 2023-09-30T22:25:53.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.881+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa3c006c380 con 0x7fa3c01303c0 2023-09-30T22:25:53.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.882+0000 7fa3b6ffd700 1 -- 172.21.15.89:0/882163149 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa3b8019300 con 0x7fa3c01303c0 2023-09-30T22:25:53.883 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-09-30T22:25:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 msgr2=0x7fa3c011e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 --2- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c011e3b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800cb90 tx=0x7fa3b800cea0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 shutdown_connections 2023-09-30T22:25:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 --2- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa3c01303c0 0x7fa3c011e3b0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 --2- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c006a770 0x7fa3c011d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 --2- 172.21.15.89:0/882163149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c006b0c0 0x7fa3c011de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 >> 172.21.15.89:0/882163149 conn(0x7fa3c0066d70 msgr2=0x7fa3c012eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 shutdown_connections 2023-09-30T22:25:53.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:53.884+0000 7fa3c8312700 1 -- 172.21.15.89:0/882163149 wait complete. 2023-09-30T22:25:53.897 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:25:54.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.019+0000 7f4d7c92f700 1 Processor -- start 2023-09-30T22:25:54.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.019+0000 7f4d7c92f700 1 -- start start 2023-09-30T22:25:54.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d7c92f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d7809e2b0 0x7f4d78135a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d7c92f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d78135f60 0x7f4d78138340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d7c92f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7809b800 con 0x7f4d7809d7d0 2023-09-30T22:25:54.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d7c92f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7809b620 con 0x7f4d7809e2b0 2023-09-30T22:25:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d7c92f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7809f370 con 0x7f4d78135f60 2023-09-30T22:25:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d76d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d78135f60 0x7f4d78138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d75d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d7809e2b0 0x7f4d78135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d75d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4d7809e2b0 0x7f4d78135a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44436/0 (socket says 172.21.15.89:44436) 2023-09-30T22:25:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d76d9d700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d78135f60 0x7f4d78138340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58296/0 (socket says 172.21.15.89:58296) 2023-09-30T22:25:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d75d9b700 1 -- 172.21.15.89:0/416828218 learned_addr learned my addr 172.21.15.89:0/416828218 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d76d9d700 1 -- 172.21.15.89:0/416828218 >> v2:172.21.15.89:3300/0 conn(0x7f4d7809e2b0 msgr2=0x7f4d78135a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d76d9d700 1 --2- 172.21.15.89:0/416828218 >> v2:172.21.15.89:3300/0 conn(0x7f4d7809e2b0 0x7f4d78135a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d76d9d700 1 -- 172.21.15.89:0/416828218 >> v1:172.21.15.89:6789/0 conn(0x7f4d7809d7d0 legacy=0x7f4d7809dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d76d9d700 1 -- 172.21.15.89:0/416828218 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7809ef80 con 0x7f4d78135f60 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d75d9b700 1 --2- 172.21.15.89:0/416828218 >> v2:172.21.15.89:3300/0 conn(0x7f4d7809e2b0 0x7f4d78135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d76d9d700 1 --2- 172.21.15.89:0/416828218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d78135f60 0x7f4d78138340 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009d90 tx=0x7f4d7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b020b287a4d1b385 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d7559a700 1 -- 172.21.15.89:0/416828218 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000e070 con 0x7f4d78135f60 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d7559a700 1 -- 172.21.15.89:0/416828218 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7000bb20 con 0x7f4d78135f60 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.020+0000 7f4d7559a700 1 -- 172.21.15.89:0/416828218 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70012430 con 0x7f4d78135f60 2023-09-30T22:25:54.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.021+0000 7f4d7c92f700 1 -- 172.21.15.89:0/416828218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d78135f60 msgr2=0x7f4d78138340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.021+0000 7f4d7c92f700 1 --2- 172.21.15.89:0/416828218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d78135f60 0x7f4d78138340 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009d90 tx=0x7f4d7000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:54.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.021+0000 7f4d7c92f700 1 -- 172.21.15.89:0/416828218 shutdown_connections 2023-09-30T22:25:54.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.021+0000 7f4d7c92f700 1 --2- 172.21.15.89:0/416828218 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d78135f60 0x7f4d78138340 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.021+0000 7f4d7c92f700 1 --2- 172.21.15.89:0/416828218 >> v2:172.21.15.89:3300/0 conn(0x7f4d7809e2b0 0x7f4d78135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.021+0000 7f4d7c92f700 1 -- 172.21.15.89:0/416828218 >> 172.21.15.89:0/416828218 conn(0x7f4d78066d70 msgr2=0x7f4d780a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.021+0000 7f4d7c92f700 1 -- 172.21.15.89:0/416828218 shutdown_connections 2023-09-30T22:25:54.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.021+0000 7f4d7c92f700 1 -- 172.21.15.89:0/416828218 wait complete. 2023-09-30T22:25:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.022+0000 7f4d7c92f700 1 Processor -- start 2023-09-30T22:25:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.022+0000 7f4d7c92f700 1 -- start start 2023-09-30T22:25:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.022+0000 7f4d7c92f700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d7809d7d0 0x7f4d7811e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.022+0000 7f4d7c92f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7809e2b0 0x7f4d7811ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.022+0000 7f4d7c92f700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d78135f60 0x7f4d7811efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.022+0000 7f4d7c92f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78134f70 con 0x7f4d7809e2b0 2023-09-30T22:25:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.022+0000 7f4d7c92f700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78121d70 con 0x7f4d78135f60 2023-09-30T22:25:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.022+0000 7f4d7c92f700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78135310 con 0x7f4d7809d7d0 2023-09-30T22:25:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d7809d7d0 0x7f4d7811e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d76d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d78135f60 0x7f4d7811efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d7809d7d0 0x7f4d7811e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58308/0 (socket says 172.21.15.89:58308) 2023-09-30T22:25:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 -- 172.21.15.89:0/919637572 learned_addr learned my addr 172.21.15.89:0/919637572 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d75d9b700 1 --2- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7809e2b0 0x7f4d7811ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 -- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d78135f60 msgr2=0x7f4d7811efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 --2- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d78135f60 0x7f4d7811efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 -- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7809e2b0 msgr2=0x7f4d7811ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 --2- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7809e2b0 0x7f4d7811ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 -- 172.21.15.89:0/919637572 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6800b7f0 con 0x7f4d7809d7d0 2023-09-30T22:25:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7659c700 1 --2- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d7809d7d0 0x7f4d7811e530 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00c450 tx=0x7f4d6c00c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d66ffd700 1 -- 172.21.15.89:0/919637572 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0052d0 con 0x7f4d7809d7d0 2023-09-30T22:25:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d66ffd700 1 -- 172.21.15.89:0/919637572 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c005430 con 0x7f4d7809d7d0 2023-09-30T22:25:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d70009a70 con 0x7f4d7809d7d0 2023-09-30T22:25:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d66ffd700 1 -- 172.21.15.89:0/919637572 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0038c0 con 0x7f4d7809d7d0 2023-09-30T22:25:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.023+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d781358c0 con 0x7f4d7809d7d0 2023-09-30T22:25:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.024+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7804f2a0 con 0x7f4d7809d7d0 2023-09-30T22:25:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.025+0000 7f4d66ffd700 1 -- 172.21.15.89:0/919637572 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d6c003a20 con 0x7f4d7809d7d0 2023-09-30T22:25:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.025+0000 7f4d66ffd700 1 -- 172.21.15.89:0/919637572 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4d6c03ee10 con 0x7f4d7809d7d0 2023-09-30T22:25:54.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.029+0000 7f4d66ffd700 1 -- 172.21.15.89:0/919637572 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d6c011590 con 0x7f4d7809d7d0 2023-09-30T22:25:54.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.207+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d7806c380 con 0x7f4d7809d7d0 2023-09-30T22:25:54.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.208+0000 7f4d66ffd700 1 -- 172.21.15.89:0/919637572 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d6c017070 con 0x7f4d7809d7d0 2023-09-30T22:25:54.209 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-09-30T22:25:54.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d7809d7d0 msgr2=0x7f4d7811e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 --2- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d7809d7d0 0x7f4d7811e530 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00c450 tx=0x7f4d6c00c760 comp rx=0 tx=0).stop 2023-09-30T22:25:54.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 shutdown_connections 2023-09-30T22:25:54.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 --2- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4d78135f60 0x7f4d7811efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 --2- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4d7809d7d0 0x7f4d7811e530 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 --2- 172.21.15.89:0/919637572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7809e2b0 0x7f4d7811ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 >> 172.21.15.89:0/919637572 conn(0x7f4d78066d70 msgr2=0x7f4d780a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 shutdown_connections 2023-09-30T22:25:54.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.210+0000 7f4d7c92f700 1 -- 172.21.15.89:0/919637572 wait complete. 2023-09-30T22:25:54.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.346+0000 7f5270011700 1 Processor -- start 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.346+0000 7f5270011700 1 -- start start 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f5270011700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5268136b80 0x7f5268138f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f5270011700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52681394a0 0x7f526813b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f5270011700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526806bcc0 con 0x7f5268071cd0 2023-09-30T22:25:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f5270011700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526806cbf0 con 0x7f5268136b80 2023-09-30T22:25:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f5270011700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f526806ba00 con 0x7f52681394a0 2023-09-30T22:25:54.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526e5ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52681394a0 0x7f526813b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526d5ac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5268136b80 0x7f5268138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526e5ae700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52681394a0 0x7f526813b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58318/0 (socket says 172.21.15.89:58318) 2023-09-30T22:25:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526e5ae700 1 -- 172.21.15.89:0/750117709 learned_addr learned my addr 172.21.15.89:0/750117709 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526d5ac700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5268136b80 0x7f5268138f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44454/0 (socket says 172.21.15.89:44454) 2023-09-30T22:25:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526e5ae700 1 -- 172.21.15.89:0/750117709 >> v2:172.21.15.89:3300/0 conn(0x7f5268136b80 msgr2=0x7f5268138f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526e5ae700 1 --2- 172.21.15.89:0/750117709 >> v2:172.21.15.89:3300/0 conn(0x7f5268136b80 0x7f5268138f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526e5ae700 1 -- 172.21.15.89:0/750117709 >> v1:172.21.15.89:6789/0 conn(0x7f5268071cd0 legacy=0x7f52680720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526e5ae700 1 -- 172.21.15.89:0/750117709 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526809f860 con 0x7f52681394a0 2023-09-30T22:25:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526cdab700 1 -- 172.21.15.89:0/750117709 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820192638 0 0) 0x7f526806bcc0 con 0x7f5268071cd0 2023-09-30T22:25:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526d5ac700 1 --2- 172.21.15.89:0/750117709 >> v2:172.21.15.89:3300/0 conn(0x7f5268136b80 0x7f5268138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526e5ae700 1 --2- 172.21.15.89:0/750117709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52681394a0 0x7f526813b880 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5260009e50 tx=0x7f526000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d5bd0dc587db9b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526cdab700 1 -- 172.21.15.89:0/750117709 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000e070 con 0x7f52681394a0 2023-09-30T22:25:54.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526cdab700 1 -- 172.21.15.89:0/750117709 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000b910 con 0x7f52681394a0 2023-09-30T22:25:54.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.347+0000 7f526cdab700 1 -- 172.21.15.89:0/750117709 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000bcc0 con 0x7f52681394a0 2023-09-30T22:25:54.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.348+0000 7f5270011700 1 -- 172.21.15.89:0/750117709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52681394a0 msgr2=0x7f526813b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.348+0000 7f5270011700 1 --2- 172.21.15.89:0/750117709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52681394a0 0x7f526813b880 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5260009e50 tx=0x7f526000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:54.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.348+0000 7f5270011700 1 -- 172.21.15.89:0/750117709 shutdown_connections 2023-09-30T22:25:54.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.348+0000 7f5270011700 1 --2- 172.21.15.89:0/750117709 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f52681394a0 0x7f526813b880 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.348+0000 7f5270011700 1 --2- 172.21.15.89:0/750117709 >> v2:172.21.15.89:3300/0 conn(0x7f5268136b80 0x7f5268138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.348+0000 7f5270011700 1 -- 172.21.15.89:0/750117709 >> 172.21.15.89:0/750117709 conn(0x7f52680a4130 msgr2=0x7f52680a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.348+0000 7f5270011700 1 -- 172.21.15.89:0/750117709 shutdown_connections 2023-09-30T22:25:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.348+0000 7f5270011700 1 -- 172.21.15.89:0/750117709 wait complete. 2023-09-30T22:25:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f5270011700 1 Processor -- start 2023-09-30T22:25:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f5270011700 1 -- start start 2023-09-30T22:25:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f5270011700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5268071cd0 0x7f526811ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f5270011700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5268136b80 0x7f526811ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f5270011700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52681394a0 0x7f526811f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f5270011700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526811ff00 con 0x7f52681394a0 2023-09-30T22:25:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f5270011700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5268121130 con 0x7f5268071cd0 2023-09-30T22:25:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f5270011700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f526813ba90 con 0x7f5268136b80 2023-09-30T22:25:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f526d5ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5268136b80 0x7f526811ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f526e5ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52681394a0 0x7f526811f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f526d5ac700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5268136b80 0x7f526811ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58332/0 (socket says 172.21.15.89:58332) 2023-09-30T22:25:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f526e5ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52681394a0 0x7f526811f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44470/0 (socket says 172.21.15.89:44470) 2023-09-30T22:25:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f526d5ac700 1 -- 172.21.15.89:0/1989276830 learned_addr learned my addr 172.21.15.89:0/1989276830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.349+0000 7f526ddad700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5268071cd0 0x7f526811ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f526d5ac700 1 -- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5268071cd0 msgr2=0x7f526811ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f526d5ac700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5268071cd0 0x7f526811ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f526d5ac700 1 -- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52681394a0 msgr2=0x7f526811f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f526d5ac700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52681394a0 0x7f526811f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f526d5ac700 1 -- 172.21.15.89:0/1989276830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52640094a0 con 0x7f5268136b80 2023-09-30T22:25:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f526e5ae700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52681394a0 0x7f526811f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f526d5ac700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5268136b80 0x7f526811ef50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f526400ebe0 tx=0x7f526400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f525e7fc700 1 -- 172.21.15.89:0/1989276830 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400c850 con 0x7f5268136b80 2023-09-30T22:25:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f525e7fc700 1 -- 172.21.15.89:0/1989276830 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526400c9b0 con 0x7f5268136b80 2023-09-30T22:25:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5260009b30 con 0x7f5268136b80 2023-09-30T22:25:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f525e7fc700 1 -- 172.21.15.89:0/1989276830 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400cd40 con 0x7f5268136b80 2023-09-30T22:25:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.350+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526806c0f0 con 0x7f5268136b80 2023-09-30T22:25:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.351+0000 7f525e7fc700 1 -- 172.21.15.89:0/1989276830 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5264013470 con 0x7f5268136b80 2023-09-30T22:25:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.351+0000 7f525e7fc700 1 -- 172.21.15.89:0/1989276830 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f526403fb10 con 0x7f5268136b80 2023-09-30T22:25:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.351+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52681386f0 con 0x7f5268136b80 2023-09-30T22:25:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.355+0000 7f525e7fc700 1 -- 172.21.15.89:0/1989276830 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5264021020 con 0x7f5268136b80 2023-09-30T22:25:54.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.537+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5268121580 con 0x7f5268136b80 2023-09-30T22:25:54.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.537+0000 7f525e7fc700 1 -- 172.21.15.89:0/1989276830 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5264017070 con 0x7f5268136b80 2023-09-30T22:25:54.538 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-09-30T22:25:54.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5268136b80 msgr2=0x7f526811ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5268136b80 0x7f526811ef50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f526400ebe0 tx=0x7f526400eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 shutdown_connections 2023-09-30T22:25:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52681394a0 0x7f526811f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5268071cd0 0x7f526811ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 --2- 172.21.15.89:0/1989276830 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5268136b80 0x7f526811ef50 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 >> 172.21.15.89:0/1989276830 conn(0x7f52680a4130 msgr2=0x7f52680712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 shutdown_connections 2023-09-30T22:25:54.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.539+0000 7f5270011700 1 -- 172.21.15.89:0/1989276830 wait complete. 2023-09-30T22:25:54.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-30T22:25:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.673+0000 7f6cdddfe700 1 Processor -- start 2023-09-30T22:25:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.673+0000 7f6cdddfe700 1 -- start start 2023-09-30T22:25:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.673+0000 7f6cdddfe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6cd809d6f0 0x7f6cd809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.673+0000 7f6cdddfe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e000 0x7f6cd8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.673+0000 7f6cdddfe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809e410 0x7f6cd8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.673+0000 7f6cdddfe700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd809f520 con 0x7f6cd809d6f0 2023-09-30T22:25:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.673+0000 7f6cdddfe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd809f130 con 0x7f6cd809e000 2023-09-30T22:25:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.673+0000 7f6cdddfe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd809e930 con 0x7f6cd809e410 2023-09-30T22:25:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809e410 0x7f6cd8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6cd809d6f0 0x7f6cd809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd6ffd700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e000 0x7f6cd8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6cd809d6f0 0x7f6cd809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44486/0 (socket says 172.21.15.89:44486) 2023-09-30T22:25:54.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 -- 172.21.15.89:0/1358439882 learned_addr learned my addr 172.21.15.89:0/1358439882 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd7fff700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809e410 0x7f6cd8130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58346/0 (socket says 172.21.15.89:58346) 2023-09-30T22:25:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 -- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809e410 msgr2=0x7f6cd8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809e410 0x7f6cd8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 -- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e000 msgr2=0x7f6cd8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e000 0x7f6cd8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 -- 172.21.15.89:0/1358439882 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd809b2d0 con 0x7f6cd809d6f0 2023-09-30T22:25:54.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd6ffd700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e000 0x7f6cd8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd7fff700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809e410 0x7f6cd8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd77fe700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809d6f0 0x7f6cd809dac0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009920 tx=0x7f6ccc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04f9e61951364e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd67fc700 1 -- 172.21.15.89:0/1358439882 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc011830 con 0x7f6cd809d6f0 2023-09-30T22:25:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd67fc700 1 -- 172.21.15.89:0/1358439882 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc011990 con 0x7f6cd809d6f0 2023-09-30T22:25:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.674+0000 7f6cd67fc700 1 -- 172.21.15.89:0/1358439882 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc011d20 con 0x7f6cd809d6f0 2023-09-30T22:25:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 -- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809d6f0 msgr2=0x7f6cd809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809d6f0 0x7f6cd809dac0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009920 tx=0x7f6ccc009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:54.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 -- 172.21.15.89:0/1358439882 shutdown_connections 2023-09-30T22:25:54.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809e410 0x7f6cd8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809d6f0 0x7f6cd809dac0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 --2- 172.21.15.89:0/1358439882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e000 0x7f6cd8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 -- 172.21.15.89:0/1358439882 >> 172.21.15.89:0/1358439882 conn(0x7f6cd8066f50 msgr2=0x7f6cd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 -- 172.21.15.89:0/1358439882 shutdown_connections 2023-09-30T22:25:54.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.675+0000 7f6cdddfe700 1 -- 172.21.15.89:0/1358439882 wait complete. 2023-09-30T22:25:54.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cdddfe700 1 Processor -- start 2023-09-30T22:25:54.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cdddfe700 1 -- start start 2023-09-30T22:25:54.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cdddfe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809d6f0 0x7f6cd81c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cdddfe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809e000 0x7f6cd81c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cdddfe700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e410 0x7f6cd81c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cdddfe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8130130 con 0x7f6cd809e000 2023-09-30T22:25:54.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cdddfe700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd806b450 con 0x7f6cd809e410 2023-09-30T22:25:54.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cdddfe700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8123770 con 0x7f6cd809d6f0 2023-09-30T22:25:54.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cd77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809d6f0 0x7f6cd81c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cd77fe700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809d6f0 0x7f6cd81c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58360/0 (socket says 172.21.15.89:58360) 2023-09-30T22:25:54.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cd77fe700 1 -- 172.21.15.89:0/516506747 learned_addr learned my addr 172.21.15.89:0/516506747 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:25:54.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cd7fff700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e410 0x7f6cd81c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.676+0000 7f6cd6ffd700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809e000 0x7f6cd81c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cd77fe700 1 -- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e410 msgr2=0x7f6cd81c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cd77fe700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e410 0x7f6cd81c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cd77fe700 1 -- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809e000 msgr2=0x7f6cd81c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cd77fe700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809e000 0x7f6cd81c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cd77fe700 1 -- 172.21.15.89:0/516506747 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80094a0 con 0x7f6cd809d6f0 2023-09-30T22:25:54.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cd77fe700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809d6f0 0x7f6cd81c6460 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0098f0 tx=0x7f6ccc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cbffff700 1 -- 172.21.15.89:0/516506747 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc004240 con 0x7f6cd809d6f0 2023-09-30T22:25:54.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cbffff700 1 -- 172.21.15.89:0/516506747 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc01f070 con 0x7f6cd809d6f0 2023-09-30T22:25:54.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd0009a70 con 0x7f6cd809d6f0 2023-09-30T22:25:54.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cbffff700 1 -- 172.21.15.89:0/516506747 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0045e0 con 0x7f6cd809d6f0 2023-09-30T22:25:54.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.677+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc009550 con 0x7f6cd809d6f0 2023-09-30T22:25:54.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.678+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd8074ea0 con 0x7f6cd809d6f0 2023-09-30T22:25:54.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.678+0000 7f6cbffff700 1 -- 172.21.15.89:0/516506747 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ccc021030 con 0x7f6cd809d6f0 2023-09-30T22:25:54.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.678+0000 7f6cbffff700 1 -- 172.21.15.89:0/516506747 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6ccc040d60 con 0x7f6cd809d6f0 2023-09-30T22:25:54.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.682+0000 7f6cbffff700 1 -- 172.21.15.89:0/516506747 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ccc0237e0 con 0x7f6cd809d6f0 2023-09-30T22:25:54.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.878+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6cd806c360 con 0x7f6cd809d6f0 2023-09-30T22:25:54.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.879+0000 7f6cbffff700 1 -- 172.21.15.89:0/516506747 <== mon.2 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v40) v1 ==== 81+0+639 (secure 0 0 0) 0x7f6ccc015070 con 0x7f6cd809d6f0 2023-09-30T22:25:54.880 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2456756709 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:54.880 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4019526010 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:54.881 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2579157783 2023-10-01T22:16:19.404121+0000 2023-09-30T22:25:54.881 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6816/118161 2023-10-01T22:16:19.404121+0000 2023-09-30T22:25:54.881 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1269574253 2023-10-01T22:16:19.404121+0000 2023-09-30T22:25:54.881 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6817/3539463463 2023-10-01T22:16:43.298028+0000 2023-09-30T22:25:54.881 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2736305244 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:54.881 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6816/126953 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:54.881 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4100667825 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:54.882 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1785021332 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:54.882 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1295992953 2023-10-01T22:17:18.222274+0000 2023-09-30T22:25:54.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809d6f0 msgr2=0x7f6cd81c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809d6f0 0x7f6cd81c6460 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0098f0 tx=0x7f6ccc004750 comp rx=0 tx=0).stop 2023-09-30T22:25:54.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 shutdown_connections 2023-09-30T22:25:54.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6cd809e410 0x7f6cd81c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6cd809d6f0 0x7f6cd81c6460 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 --2- 172.21.15.89:0/516506747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cd809e000 0x7f6cd81c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 >> 172.21.15.89:0/516506747 conn(0x7f6cd8066f50 msgr2=0x7f6cd8119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 shutdown_connections 2023-09-30T22:25:54.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:25:54.881+0000 7f6cdddfe700 1 -- 172.21.15.89:0/516506747 wait complete. 2023-09-30T22:25:54.884 INFO:teuthology.orchestra.run.smithi089.stderr:listed 11 entries 2023-09-30T22:25:54.894 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi134.front.sepia.ceph.com... 2023-09-30T22:25:54.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:25:54.894 DEBUG:teuthology.orchestra.run.smithi134:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-30T22:25:54.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi134.stderr:ceph-fuse[112275]: fuse finished with error 0 and tester_r 0 2023-09-30T22:25:54.942 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:26:01.044 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2023-09-30T22:26:01.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:26:01.045 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-30T22:26:01.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:26:01.080 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2023-09-30T22:26:01.142 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-30T22:26:01.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:26:01.143 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:26:01.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:26:01.233 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2023-09-30T22:26:01.299 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-30T22:26:01.320 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:26:01.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:26:01.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.454+0000 7f0b588da700 1 Processor -- start 2023-09-30T22:26:01.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.454+0000 7f0b588da700 1 -- start start 2023-09-30T22:26:01.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b588da700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b5409e1d0 0x7f0b54138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b588da700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b54139170 0x7f0b5413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b588da700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5409b120 con 0x7f0b5409d6f0 2023-09-30T22:26:01.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b588da700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5406b460 con 0x7f0b5409e1d0 2023-09-30T22:26:01.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b588da700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5409a7e0 con 0x7f0b54139170 2023-09-30T22:26:01.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b51d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b5409e1d0 0x7f0b54138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b52d9d700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b54139170 0x7f0b5413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b51d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0b5409e1d0 0x7f0b54138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41948/0 (socket says 172.21.15.89:41948) 2023-09-30T22:26:01.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.455+0000 7f0b51d9b700 1 -- 172.21.15.89:0/2089703052 learned_addr learned my addr 172.21.15.89:0/2089703052 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:26:01.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b52d9d700 1 -- 172.21.15.89:0/2089703052 >> v2:172.21.15.89:3300/0 conn(0x7f0b5409e1d0 msgr2=0x7f0b54138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:01.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b52d9d700 1 --2- 172.21.15.89:0/2089703052 >> v2:172.21.15.89:3300/0 conn(0x7f0b5409e1d0 0x7f0b54138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b52d9d700 1 -- 172.21.15.89:0/2089703052 >> v1:172.21.15.89:6789/0 conn(0x7f0b5409d6f0 legacy=0x7f0b5409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b52d9d700 1 -- 172.21.15.89:0/2089703052 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5409a500 con 0x7f0b54139170 2023-09-30T22:26:01.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b51d9b700 1 --2- 172.21.15.89:0/2089703052 >> v2:172.21.15.89:3300/0 conn(0x7f0b5409e1d0 0x7f0b54138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b52d9d700 1 --2- 172.21.15.89:0/2089703052 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b54139170 0x7f0b5413b550 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009dc0 tx=0x7f0b4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8052c75b742ba98 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b5159a700 1 -- 172.21.15.89:0/2089703052 <== mon.1 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c00e070 con 0x7f0b54139170 2023-09-30T22:26:01.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b5159a700 1 -- 172.21.15.89:0/2089703052 <== mon.1 v2:172.21.15.89:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c00bae0 con 0x7f0b54139170 2023-09-30T22:26:01.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b5159a700 1 -- 172.21.15.89:0/2089703052 <== mon.1 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c00be90 con 0x7f0b54139170 2023-09-30T22:26:01.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b588da700 1 -- 172.21.15.89:0/2089703052 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b54139170 msgr2=0x7f0b5413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.456+0000 7f0b588da700 1 --2- 172.21.15.89:0/2089703052 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b54139170 0x7f0b5413b550 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009dc0 tx=0x7f0b4c00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:01.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.457+0000 7f0b588da700 1 -- 172.21.15.89:0/2089703052 shutdown_connections 2023-09-30T22:26:01.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.457+0000 7f0b588da700 1 --2- 172.21.15.89:0/2089703052 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b54139170 0x7f0b5413b550 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.457+0000 7f0b588da700 1 --2- 172.21.15.89:0/2089703052 >> v2:172.21.15.89:3300/0 conn(0x7f0b5409e1d0 0x7f0b54138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.457+0000 7f0b588da700 1 -- 172.21.15.89:0/2089703052 >> 172.21.15.89:0/2089703052 conn(0x7f0b54066f50 msgr2=0x7f0b5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:01.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.457+0000 7f0b588da700 1 -- 172.21.15.89:0/2089703052 shutdown_connections 2023-09-30T22:26:01.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.457+0000 7f0b588da700 1 -- 172.21.15.89:0/2089703052 wait complete. 2023-09-30T22:26:01.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b588da700 1 Processor -- start 2023-09-30T22:26:01.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b588da700 1 -- start start 2023-09-30T22:26:01.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b588da700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b5409d6f0 0x7f0b541c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b588da700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b5409e1d0 0x7f0b541c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b588da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b54139170 0x7f0b5411e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b588da700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54122110 con 0x7f0b54139170 2023-09-30T22:26:01.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b588da700 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54120400 con 0x7f0b5409d6f0 2023-09-30T22:26:01.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b588da700 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54121d70 con 0x7f0b5409e1d0 2023-09-30T22:26:01.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b52d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b54139170 0x7f0b5411e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b51d9b700 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b5409e1d0 0x7f0b541c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b52d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b54139170 0x7f0b5411e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41962/0 (socket says 172.21.15.89:41962) 2023-09-30T22:26:01.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b5259c700 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b5409d6f0 0x7f0b541c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.458+0000 7f0b52d9d700 1 -- 172.21.15.89:0/3359361167 learned_addr learned my addr 172.21.15.89:0/3359361167 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-30T22:26:01.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b51d9b700 1 -- 172.21.15.89:0/3359361167 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b5409d6f0 msgr2=0x7f0b541c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b51d9b700 1 --2- 172.21.15.89:0/3359361167 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0b5409d6f0 0x7f0b541c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b51d9b700 1 -- 172.21.15.89:0/3359361167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b54139170 msgr2=0x7f0b5411e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b51d9b700 1 --2- 172.21.15.89:0/3359361167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b54139170 0x7f0b5411e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b51d9b700 1 -- 172.21.15.89:0/3359361167 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b44009550 con 0x7f0b5409e1d0 2023-09-30T22:26:01.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b52d9d700 1 --2- 172.21.15.89:0/3359361167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b54139170 0x7f0b5411e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b51d9b700 1 --2- 172.21.15.89:0/3359361167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0b5409e1d0 0x7f0b541c8a10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400e9a0 tx=0x7f0b4400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b42ffd700 1 -- 172.21.15.89:0/3359361167 <== mon.2 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44004030 con 0x7f0b5409e1d0 2023-09-30T22:26:01.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b42ffd700 1 -- 172.21.15.89:0/3359361167 <== mon.2 v2:172.21.15.89:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44004190 con 0x7f0b5409e1d0 2023-09-30T22:26:01.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b588da700 1 -- 172.21.15.89:0/3359361167 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c009a70 con 0x7f0b5409e1d0 2023-09-30T22:26:01.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b42ffd700 1 -- 172.21.15.89:0/3359361167 <== mon.2 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44004520 con 0x7f0b5409e1d0 2023-09-30T22:26:01.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.459+0000 7f0b588da700 1 -- 172.21.15.89:0/3359361167 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b54122aa0 con 0x7f0b5409e1d0 2023-09-30T22:26:01.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.460+0000 7f0b42ffd700 1 -- 172.21.15.89:0/3359361167 <== mon.2 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b440036a0 con 0x7f0b5409e1d0 2023-09-30T22:26:01.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.460+0000 7f0b588da700 1 -- 172.21.15.89:0/3359361167 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54074ea0 con 0x7f0b5409e1d0 2023-09-30T22:26:01.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.460+0000 7f0b42ffd700 1 -- 172.21.15.89:0/3359361167 <== mon.2 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0b4403f8c0 con 0x7f0b5409e1d0 2023-09-30T22:26:01.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-30T22:26:01.465+0000 7f0b42ffd700 1 -- 172.21.15.89:0/3359361167 <== mon.2 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b44011db0 con 0x7f0b5409e1d0 2023-09-30T22:28:01.360 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-30T22:28:01.361 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:28:01.370 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-30T22:28:01.370 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-30T22:28:01.371 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.387 INFO:tasks.ceph.mds.a:Stopped 2023-09-30T22:28:01.387 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-30T22:28:01.388 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-30T22:28:01.388 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.456 INFO:tasks.ceph.osd.0:Stopped 2023-09-30T22:28:01.457 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-30T22:28:01.457 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.527 INFO:tasks.ceph.osd.1:Stopped 2023-09-30T22:28:01.528 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-30T22:28:01.528 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.594 INFO:tasks.ceph.osd.2:Stopped 2023-09-30T22:28:01.594 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-30T22:28:01.595 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.661 INFO:tasks.ceph.osd.3:Stopped 2023-09-30T22:28:01.661 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-30T22:28:01.661 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-30T22:28:01.661 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:28:01.662 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2023-09-30T22:28:01.673 INFO:tasks.ceph.mgr.x:Stopped 2023-09-30T22:28:01.673 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-30T22:28:01.673 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-30T22:28:01.673 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.714 INFO:tasks.ceph.mon.a:Stopped 2023-09-30T22:28:01.714 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-30T22:28:01.715 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.767 INFO:tasks.ceph.mon.b:Stopped 2023-09-30T22:28:01.767 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-30T22:28:01.767 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:28:01.820 INFO:tasks.ceph.mon.c:Stopped 2023-09-30T22:28:01.820 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-30T22:28:01.820 DEBUG:teuthology.orchestra.run.smithi089:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'scrub mismatch' | egrep -v ScrubResult | egrep -v 'wrongly marked' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_MSGR2_NOT_ENABLED\)' | egrep -v 'slow request' | head -n 1 2023-09-30T22:28:01.900 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:28:01.901 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-30T22:28:02.148 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:28:02.148 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-30T22:28:02.262 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:28:02.262 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-30T22:28:02.320 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:28:02.320 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-30T22:28:02.377 INFO:tasks.ceph:Archiving mon data... 2023-09-30T22:28:02.378 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/data/mon.a.tgz 2023-09-30T22:28:02.378 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-09-30T22:28:02.395 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.HqRLfsxZaI 2023-09-30T22:28:02.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.HqRLfsxZaI 2023-09-30T22:28:02.536 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.HqRLfsxZaI 2023-09-30T22:28:02.624 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.HqRLfsxZaI is 316KB 2023-09-30T22:28:02.721 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.HqRLfsxZaI 2023-09-30T22:28:02.739 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/data/mon.b.tgz 2023-09-30T22:28:02.739 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.Wrdz1vIR0X 2023-09-30T22:28:02.796 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Wrdz1vIR0X 2023-09-30T22:28:02.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.Wrdz1vIR0X 2023-09-30T22:28:03.008 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.Wrdz1vIR0X is 318KB 2023-09-30T22:28:03.103 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.Wrdz1vIR0X 2023-09-30T22:28:03.121 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/data/mon.c.tgz 2023-09-30T22:28:03.121 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-09-30T22:28:03.178 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.3fRbiiMUMP 2023-09-30T22:28:03.178 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.3fRbiiMUMP 2023-09-30T22:28:03.303 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.3fRbiiMUMP 2023-09-30T22:28:03.391 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.3fRbiiMUMP is 318KB 2023-09-30T22:28:03.486 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.3fRbiiMUMP 2023-09-30T22:28:03.504 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-30T22:28:03.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T22:28:03.547 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T22:28:03.583 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-30T22:28:03.583 DEBUG:teuthology.orchestra.run.smithi089:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T22:28:03.614 DEBUG:teuthology.orchestra.run.smithi134:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T22:28:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T22:28:03.656 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T22:28:03.663 INFO:tasks.ceph:Archiving crash dumps... 2023-09-30T22:28:03.665 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/remote/smithi089/crash 2023-09-30T22:28:03.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T22:28:03.712 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/remote/smithi134/crash 2023-09-30T22:28:03.713 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T22:28:03.758 INFO:tasks.ceph:Compressing logs... 2023-09-30T22:28:03.759 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T22:28:03.761 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T22:28:03.787 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191676.log 2023-09-30T22:28:03.787 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135389.log 2023-09-30T22:28:03.794 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112275.log 2023-09-30T22:28:03.801 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.0.112275.log: 96.6% -- replaced with /var/log/ceph/ceph-client.0.112275.log.gz 2023-09-30T22:28:03.801 INFO:teuthology.orchestra.run.smithi134.stderr: 2023-09-30T22:28:03.802 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m0.019s 2023-09-30T22:28:03.802 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m0.018s 2023-09-30T22:28:03.802 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.009s 2023-09-30T22:28:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143407.log 2023-09-30T22:28:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135389.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191676.log.gz 2023-09-30T22:28:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187464.log 2023-09-30T22:28:03.946 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.135389.log.gz 2023-09-30T22:28:03.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192881.log 2023-09-30T22:28:03.946 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143407.log.gz 2023-09-30T22:28:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153278.log 2023-09-30T22:28:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192881.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.187464.log.gz 2023-09-30T22:28:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148513.log 2023-09-30T22:28:03.947 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192881.log.gz 2023-09-30T22:28:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194305.log 2023-09-30T22:28:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153278.log.gz 2023-09-30T22:28:03.947 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186972.log 2023-09-30T22:28:03.947 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148513.log.gz 2023-09-30T22:28:03.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166589.log 2023-09-30T22:28:03.948 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.194305.log.gz 2023-09-30T22:28:03.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151138.log 2023-09-30T22:28:03.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166589.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.186972.log.gz 2023-09-30T22:28:03.948 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124774.log 2023-09-30T22:28:03.948 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166589.log.gz 2023-09-30T22:28:03.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187952.log 2023-09-30T22:28:03.948 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.151138.log.gz 2023-09-30T22:28:03.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156827.log 2023-09-30T22:28:03.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187952.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124774.log.gz 2023-09-30T22:28:03.949 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129106.log 2023-09-30T22:28:03.949 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.187952.log.gz 2023-09-30T22:28:03.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140713.log 2023-09-30T22:28:03.949 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156827.log.gz 2023-09-30T22:28:03.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149963.log 2023-09-30T22:28:03.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140713.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129106.log.gz 2023-09-30T22:28:03.950 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177802.log 2023-09-30T22:28:03.950 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140713.log.gz 2023-09-30T22:28:03.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122799.log 2023-09-30T22:28:03.950 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.149963.log.gz 2023-09-30T22:28:03.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127864.log 2023-09-30T22:28:03.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122799.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177802.log.gz 2023-09-30T22:28:03.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186266.log 2023-09-30T22:28:03.953 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122799.log.gz 2023-09-30T22:28:03.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184535.log 2023-09-30T22:28:03.954 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127864.log.gz 2023-09-30T22:28:03.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176449.log 2023-09-30T22:28:03.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184535.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.186266.log.gz 2023-09-30T22:28:03.954 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128714.log 2023-09-30T22:28:03.954 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.184535.log.gz 2023-09-30T22:28:03.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156779.log 2023-09-30T22:28:03.954 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176449.log.gz 2023-09-30T22:28:03.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145991.log 2023-09-30T22:28:03.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156779.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128714.log.gz 2023-09-30T22:28:03.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118879.log 2023-09-30T22:28:03.955 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156779.log.gz 2023-09-30T22:28:03.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192068.log 2023-09-30T22:28:03.955 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145991.log.gz 2023-09-30T22:28:03.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.118879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127838.log 2023-09-30T22:28:03.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192068.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118879.log.gz 2023-09-30T22:28:03.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124872.log 2023-09-30T22:28:03.956 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192068.log.gz 2023-09-30T22:28:03.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137774.log 2023-09-30T22:28:03.956 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127838.log.gz 2023-09-30T22:28:03.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164629.log 2023-09-30T22:28:03.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137774.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124872.log.gz 2023-09-30T22:28:03.956 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142696.log 2023-09-30T22:28:03.956 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137774.log.gz 2023-09-30T22:28:03.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195328.log 2023-09-30T22:28:03.957 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164629.log.gz 2023-09-30T22:28:03.957 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190002.log 2023-09-30T22:28:03.957 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.195328.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142696.log.gz 2023-09-30T22:28:03.957 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138487.log 2023-09-30T22:28:03.957 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.195328.log.gz 2023-09-30T22:28:03.957 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184710.log 2023-09-30T22:28:03.957 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.190002.log.gz 2023-09-30T22:28:03.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189031.log 2023-09-30T22:28:03.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184710.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138487.log.gz 2023-09-30T22:28:03.958 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125629.log 2023-09-30T22:28:03.958 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.184710.log.gz 2023-09-30T22:28:03.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169995.log 2023-09-30T22:28:03.958 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189031.log.gz 2023-09-30T22:28:03.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148003.log 2023-09-30T22:28:03.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169995.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125629.log.gz 2023-09-30T22:28:03.959 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168183.log 2023-09-30T22:28:03.959 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169995.log.gz 2023-09-30T22:28:03.959 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178731.log 2023-09-30T22:28:03.959 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148003.log.gz 2023-09-30T22:28:03.959 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181724.log 2023-09-30T22:28:03.959 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168183.log: /var/log/ceph/ceph-client.admin.178731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157589.log 2023-09-30T22:28:03.959 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168183.log.gz 2023-09-30T22:28:03.959 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146218.log 2023-09-30T22:28:03.960 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178731.log.gz 2023-09-30T22:28:03.960 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157589.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.181724.log.gz 2023-09-30T22:28:03.960 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171767.log 2023-09-30T22:28:03.960 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157589.log.gz 2023-09-30T22:28:03.960 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137531.log 2023-09-30T22:28:03.960 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146218.log.gz 2023-09-30T22:28:03.960 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179027.log 2023-09-30T22:28:03.960 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.171767.log.gz 2023-09-30T22:28:03.961 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156975.log 2023-09-30T22:28:03.961 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179027.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137531.log.gz 2023-09-30T22:28:03.961 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193247.log 2023-09-30T22:28:03.961 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179027.log.gz 2023-09-30T22:28:03.961 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162886.log 2023-09-30T22:28:03.961 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156975.log: /var/log/ceph/ceph-client.admin.193247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151727.log 2023-09-30T22:28:03.961 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156975.log.gz 2023-09-30T22:28:03.961 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193247.log.gz 2023-09-30T22:28:03.962 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161100.log 2023-09-30T22:28:03.962 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151727.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.162886.log.gz-5 2023-09-30T22:28:03.962 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.156241.log 2023-09-30T22:28:03.962 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151727.log.gz 2023-09-30T22:28:03.962 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140491.log 2023-09-30T22:28:03.962 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156241.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161100.log.gz 2023-09-30T22:28:03.962 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150745.log 2023-09-30T22:28:03.963 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156241.log.gz 2023-09-30T22:28:03.963 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168013.log 2023-09-30T22:28:03.963 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140491.log.gz 2023-09-30T22:28:03.963 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169806.log 2023-09-30T22:28:03.963 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150745.log.gz 2023-09-30T22:28:03.963 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171301.log 2023-09-30T22:28:03.963 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.168013.log.gz 2023-09-30T22:28:03.963 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180780.log 2023-09-30T22:28:03.964 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169806.log.gz 2023-09-30T22:28:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178026.log 2023-09-30T22:28:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180780.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171301.log.gz 2023-09-30T22:28:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156753.log 2023-09-30T22:28:03.964 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.180780.log.gz 2023-09-30T22:28:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145181.log 2023-09-30T22:28:03.964 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178026.log.gz 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178614.log 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145181.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.156753.log.gz 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192933.log 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.145181.log.gz 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149374.log 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178614.log.gz 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168719.log 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149374.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192933.log.gz 2023-09-30T22:28:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127185.log 2023-09-30T22:28:03.966 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149374.log.gz 2023-09-30T22:28:03.966 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162126.log 2023-09-30T22:28:03.966 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127185.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168719.log.gz 2023-09-30T22:28:03.966 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125313.log 2023-09-30T22:28:03.966 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127185.log.gz 2023-09-30T22:28:03.966 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165975.log 2023-09-30T22:28:03.966 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125313.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162126.log.gz 2023-09-30T22:28:03.967 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138360.log 2023-09-30T22:28:03.967 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125313.log.gz 2023-09-30T22:28:03.967 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125166.log 2023-09-30T22:28:03.967 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.165975.log.gz 2023-09-30T22:28:03.967 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138262.log 2023-09-30T22:28:03.967 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.138360.log.gz 2023-09-30T22:28:03.967 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156165.log 2023-09-30T22:28:03.967 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169803.log 2023-09-30T22:28:03.968 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125166.log.gz 2023-09-30T22:28:03.968 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138262.log.gz 2023-09-30T22:28:03.968 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137504.log 2023-09-30T22:28:03.968 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156165.log.gz 2023-09-30T22:28:03.968 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193639.log 2023-09-30T22:28:03.968 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169803.log: /var/log/ceph/ceph-client.admin.137504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151478.log 2023-09-30T22:28:03.968 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.169803.log.gz 2023-09-30T22:28:03.968 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137504.log.gz 2023-09-30T22:28:03.969 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125776.log 2023-09-30T22:28:03.969 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151478.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.193639.log.gz 2023-09-30T22:28:03.969 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149493.log 2023-09-30T22:28:03.969 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.151478.log.gz 2023-09-30T22:28:03.969 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136890.log 2023-09-30T22:28:03.969 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125776.log.gz 2023-09-30T22:28:03.969 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125117.log 2023-09-30T22:28:03.969 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149493.log: /var/log/ceph/ceph-client.admin.136890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172894.log 2023-09-30T22:28:03.970 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149493.log.gz 2023-09-30T22:28:03.970 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136890.log.gz 2023-09-30T22:28:03.970 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182677.log 2023-09-30T22:28:03.970 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172894.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.125117.log.gz 2023-09-30T22:28:03.970 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163035.log 2023-09-30T22:28:03.970 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172894.log.gz 2023-09-30T22:28:03.970 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158203.log 2023-09-30T22:28:03.970 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.182677.log.gz 2023-09-30T22:28:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186070.log 2023-09-30T22:28:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158203.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.163035.log.gz 2023-09-30T22:28:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189520.log 2023-09-30T22:28:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186070.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158203.log.gz 2023-09-30T22:28:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175587.log 2023-09-30T22:28:03.971 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.186070.log.gz 2023-09-30T22:28:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136772.log 2023-09-30T22:28:03.971 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175587.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189520.log.gz 2023-09-30T22:28:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160904.log 2023-09-30T22:28:03.972 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175587.log.gz 2023-09-30T22:28:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149937.log 2023-09-30T22:28:03.972 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136772.log.gz 2023-09-30T22:28:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141275.log 2023-09-30T22:28:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160904.log: /var/log/ceph/ceph-client.admin.149937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171719.log 2023-09-30T22:28:03.972 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160904.log.gz 2023-09-30T22:28:03.972 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141275.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149937.log.gz 2023-09-30T22:28:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167986.log 2023-09-30T22:28:03.973 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141275.log.gz 2023-09-30T22:28:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173039.log 2023-09-30T22:28:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167986.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.171719.log.gz 2023-09-30T22:28:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189227.log 2023-09-30T22:28:03.973 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167986.log.gz 2023-09-30T22:28:03.973 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162546.log 2023-09-30T22:28:03.973 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173039.log.gz 2023-09-30T22:28:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123304.log 2023-09-30T22:28:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189227.log: /var/log/ceph/ceph-client.admin.162546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138117.log 2023-09-30T22:28:03.974 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.189227.log.gz 2023-09-30T22:28:03.974 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162546.log.gz 2023-09-30T22:28:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146408.log 2023-09-30T22:28:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138117.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123304.log.gz 2023-09-30T22:28:03.974 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135935.log 2023-09-30T22:28:03.974 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138117.log.gz 2023-09-30T22:28:03.975 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140687.log 2023-09-30T22:28:03.975 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146408.log.gz 2023-09-30T22:28:03.975 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160339.log 2023-09-30T22:28:03.975 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135935.log.gz 2023-09-30T22:28:03.975 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190971.log 2023-09-30T22:28:03.975 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160339.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140687.log.gz 2023-09-30T22:28:03.975 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144248.log 2023-09-30T22:28:03.975 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160339.log.gz 2023-09-30T22:28:03.976 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188221.log 2023-09-30T22:28:03.976 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.190971.log.gz 2023-09-30T22:28:03.976 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165217.log 2023-09-30T22:28:03.976 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188221.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144248.log.gz 2023-09-30T22:28:03.976 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183921.log 2023-09-30T22:28:03.976 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188221.log.gz 2023-09-30T22:28:03.976 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135463.log 2023-09-30T22:28:03.976 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.165217.log.gz 2023-09-30T22:28:03.977 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153839.log 2023-09-30T22:28:03.977 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.183921.log.gz 2023-09-30T22:28:03.977 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168379.log 2023-09-30T22:28:03.977 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180594.log 2023-09-30T22:28:03.977 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135463.log.gz 2023-09-30T22:28:03.977 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153839.log.gz 2023-09-30T22:28:03.977 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170295.log 2023-09-30T22:28:03.977 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168379.log.gz 2023-09-30T22:28:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136771.log 2023-09-30T22:28:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177947.log 2023-09-30T22:28:03.978 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.180594.log.gz 2023-09-30T22:28:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136771.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170295.log.gz 2023-09-30T22:28:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171863.log 2023-09-30T22:28:03.978 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136771.log.gz 2023-09-30T22:28:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123451.log 2023-09-30T22:28:03.978 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177947.log: /var/log/ceph/ceph-client.admin.171863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149178.log 2023-09-30T22:28:03.979 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177947.log.gz 2023-09-30T22:28:03.979 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171863.log.gz 2023-09-30T22:28:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135316.log 2023-09-30T22:28:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123451.log: /var/log/ceph/ceph-client.admin.149178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175417.log 2023-09-30T22:28:03.979 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123451.log.gz 2023-09-30T22:28:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135316.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149178.log.gz 2023-09-30T22:28:03.979 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173336.log 2023-09-30T22:28:03.979 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135316.log.gz 2023-09-30T22:28:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149741.log 2023-09-30T22:28:03.980 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.175417.log.gz 2023-09-30T22:28:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190970.log 2023-09-30T22:28:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149741.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173336.log.gz 2023-09-30T22:28:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135936.log 2023-09-30T22:28:03.980 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149741.log.gz 2023-09-30T22:28:03.980 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163452.log 2023-09-30T22:28:03.980 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.190970.log.gz 2023-09-30T22:28:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151649.log 2023-09-30T22:28:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158963.log 2023-09-30T22:28:03.981 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135936.log.gz 2023-09-30T22:28:03.981 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163452.log.gz 2023-09-30T22:28:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144666.log 2023-09-30T22:28:03.981 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151649.log.gz 2023-09-30T22:28:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185029.log 2023-09-30T22:28:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190774.log 2023-09-30T22:28:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144666.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158963.log.gz 2023-09-30T22:28:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185029.log: gzip -5 --verbose 88.2%-- /var/log/ceph/ceph-client.admin.153670.log -- replaced with /var/log/ceph/ceph-client.admin.144666.log.gz 2023-09-30T22:28:03.982 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:03.982 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.185029.log.gz 2023-09-30T22:28:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189575.log 2023-09-30T22:28:03.982 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.190774.log.gz-5 2023-09-30T22:28:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.141079.log 2023-09-30T22:28:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189575.log: /var/log/ceph/ceph-client.admin.194057.log 2023-09-30T22:28:03.983 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153670.log.gz 2023-09-30T22:28:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141079.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189575.log.gz 2023-09-30T22:28:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178025.log 2023-09-30T22:28:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194057.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141079.log.gz 2023-09-30T22:28:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175883.log 2023-09-30T22:28:03.983 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.194057.log.gz 2023-09-30T22:28:03.983 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149935.log 2023-09-30T22:28:03.984 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178025.log.gz 2023-09-30T22:28:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145795.log 2023-09-30T22:28:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149935.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.175883.log.gz 2023-09-30T22:28:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191728.log 2023-09-30T22:28:03.984 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149935.log.gz 2023-09-30T22:28:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143434.log 2023-09-30T22:28:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191728.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145795.log.gz 2023-09-30T22:28:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136576.log 2023-09-30T22:28:03.985 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.191728.log.gz 2023-09-30T22:28:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142650.log 2023-09-30T22:28:03.985 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143434.log.gz 2023-09-30T22:28:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129449.log 2023-09-30T22:28:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142650.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136576.log.gz 2023-09-30T22:28:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155407.log 2023-09-30T22:28:03.985 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142650.log.gz 2023-09-30T22:28:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176397.log 2023-09-30T22:28:03.986 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129449.log.gz 2023-09-30T22:28:03.986 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148002.log 2023-09-30T22:28:03.986 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155407.log.gz 2023-09-30T22:28:03.986 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172698.log 2023-09-30T22:28:03.986 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176397.log.gz 2023-09-30T22:28:03.986 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155240.log 2023-09-30T22:28:03.986 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148002.log.gz 2023-09-30T22:28:03.986 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136720.log 2023-09-30T22:28:03.987 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172698.log.gz 2023-09-30T22:28:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153740.log 2023-09-30T22:28:03.987 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155240.log.gz 2023-09-30T22:28:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165659.log 2023-09-30T22:28:03.987 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136720.log.gz 2023-09-30T22:28:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178232.log 2023-09-30T22:28:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165659.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153740.log.gz 2023-09-30T22:28:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135118.log 2023-09-30T22:28:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178232.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165659.log.gz 2023-09-30T22:28:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151896.log 2023-09-30T22:28:03.988 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.178232.log.gz 2023-09-30T22:28:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180325.log 2023-09-30T22:28:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151896.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135118.log.gz 2023-09-30T22:28:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160399.log 2023-09-30T22:28:03.988 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151896.log.gz/var/log/ceph/ceph-client.admin.180325.log: 2023-09-30T22:28:03.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170543.log 2023-09-30T22:28:03.989 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.180325.log.gz 2023-09-30T22:28:03.989 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167373.log 2023-09-30T22:28:03.989 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.160399.log.gz 2023-09-30T22:28:03.989 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170687.log 2023-09-30T22:28:03.989 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170543.log.gz 2023-09-30T22:28:03.989 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157733.log 2023-09-30T22:28:03.989 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.167373.log.gz 2023-09-30T22:28:03.989 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128910.log 2023-09-30T22:28:03.990 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170687.log.gz 2023-09-30T22:28:03.990 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183508.log 2023-09-30T22:28:03.990 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157733.log.gz 2023-09-30T22:28:03.990 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181358.log 2023-09-30T22:28:03.990 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128910.log.gz 2023-09-30T22:28:03.990 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182259.log 2023-09-30T22:28:03.990 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183508.log.gz 2023-09-30T22:28:03.990 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170713.log 2023-09-30T22:28:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182259.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.181358.log.gz 2023-09-30T22:28:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170374.log 2023-09-30T22:28:03.991 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.182259.log.gz 2023-09-30T22:28:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175273.log 2023-09-30T22:28:03.991 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.170713.log.gz 2023-09-30T22:28:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185118.log 2023-09-30T22:28:03.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155969.log 2023-09-30T22:28:03.991 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170374.log.gz 2023-09-30T22:28:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185118.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175273.log.gz 2023-09-30T22:28:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159301.log 2023-09-30T22:28:03.992 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.185118.log.gz 2023-09-30T22:28:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157784.log 2023-09-30T22:28:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159301.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155969.log.gz 2023-09-30T22:28:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173874.log 2023-09-30T22:28:03.992 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159301.log.gz 2023-09-30T22:28:03.992 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164603.log 2023-09-30T22:28:03.993 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157784.log.gz 2023-09-30T22:28:03.993 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144936.log 2023-09-30T22:28:03.993 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173874.log.gz 2023-09-30T22:28:03.993 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175391.log 2023-09-30T22:28:03.993 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164603.log.gz 2023-09-30T22:28:03.993 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183070.log 2023-09-30T22:28:03.993 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.144936.log.gz 2023-09-30T22:28:03.993 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.175391.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.166074.log 2023-09-30T22:28:03.994 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183070.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175391.log.gz 2023-09-30T22:28:03.994 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181553.log 2023-09-30T22:28:03.994 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.183070.log.gz 2023-09-30T22:28:03.994 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148983.log 2023-09-30T22:28:03.994 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166074.log.gz 2023-09-30T22:28:03.994 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125019.log 2023-09-30T22:28:03.994 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181553.log.gz 2023-09-30T22:28:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169611.log 2023-09-30T22:28:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123820.log 2023-09-30T22:28:03.995 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148983.log.gz 2023-09-30T22:28:03.995 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.125019.log.gz 2023-09-30T22:28:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151139.log 2023-09-30T22:28:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123820.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169611.log.gz 2023-09-30T22:28:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150329.log 2023-09-30T22:28:03.995 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151139.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.123820.log.gz 2023-09-30T22:28:03.996 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129204.log 2023-09-30T22:28:03.996 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.151139.log.gz 2023-09-30T22:28:03.996 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129547.log 2023-09-30T22:28:03.996 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150329.log: /var/log/ceph/ceph-client.admin.129204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158372.log 2023-09-30T22:28:03.996 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150329.log.gz 2023-09-30T22:28:03.996 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164331.log 2023-09-30T22:28:03.996 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129204.log.gz 2023-09-30T22:28:03.996 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183725.log 2023-09-30T22:28:03.997 INFO:teuthology.orchestra.run.smithi089.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.129547.log.gz 2023-09-30T22:28:03.997 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158372.log.gz 2023-09-30T22:28:03.997 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176031.log 2023-09-30T22:28:03.997 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158373.log 2023-09-30T22:28:03.997 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164331.log.gz 2023-09-30T22:28:03.997 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170982.log 2023-09-30T22:28:03.997 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183725.log.gz 2023-09-30T22:28:03.997 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176031.log.gz 2023-09-30T22:28:03.998 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152713.log 2023-09-30T22:28:03.998 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158373.log.gz 2023-09-30T22:28:03.998 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154871.log 2023-09-30T22:28:03.998 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170982.log: /var/log/ceph/ceph-client.admin.152713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156388.log 2023-09-30T22:28:03.998 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170982.log.gz 2023-09-30T22:28:03.998 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152713.log.gz 2023-09-30T22:28:03.998 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146187.log 2023-09-30T22:28:03.998 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154871.log.gz 2023-09-30T22:28:03.999 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165779.log 2023-09-30T22:28:03.999 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146187.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156388.log.gz 2023-09-30T22:28:03.999 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192367.log 2023-09-30T22:28:03.999 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146187.log.gz 2023-09-30T22:28:03.999 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124137.log 2023-09-30T22:28:03.999 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165779.log.gz 2023-09-30T22:28:03.999 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157811.log 2023-09-30T22:28:03.999 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124137.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.192367.log.gz 2023-09-30T22:28:04.000 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127560.log 2023-09-30T22:28:04.000 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124137.log.gz 2023-09-30T22:28:04.000 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159834.log 2023-09-30T22:28:04.000 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157811.log.gz 2023-09-30T22:28:04.000 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136106.log 2023-09-30T22:28:04.000 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127560.log: /var/log/ceph/ceph-client.admin.159834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124019.log 2023-09-30T22:28:04.000 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127560.log.gz 2023-09-30T22:28:04.000 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126021.log 2023-09-30T22:28:04.001 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159834.log.gz 2023-09-30T22:28:04.001 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124019.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136106.log.gz 2023-09-30T22:28:04.001 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150746.log 2023-09-30T22:28:04.001 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124019.log.gz 2023-09-30T22:28:04.001 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.126021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171327.log 2023-09-30T22:28:04.001 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172502.log 2023-09-30T22:28:04.001 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150746.log.gz 2023-09-30T22:28:04.001 INFO:teuthology.orchestra.run.smithi089.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.126021.log.gz 2023-09-30T22:28:04.002 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159915.log 2023-09-30T22:28:04.002 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.171327.log.gz 2023-09-30T22:28:04.002 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172451.log 2023-09-30T22:28:04.002 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172502.log: /var/log/ceph/ceph-client.admin.159915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179542.log 2023-09-30T22:28:04.002 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.172502.log.gz 2023-09-30T22:28:04.002 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159915.log.gz 2023-09-30T22:28:04.002 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172451.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-09-30T22:28:04.002 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.172451.log.gz/var/log/ceph/ceph-client.admin.179542.log: 2023-09-30T22:28:04.003 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162108.log 2023-09-30T22:28:04.003 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.2.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.179542.log.gz 2023-09-30T22:28:04.003 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177181.log 2023-09-30T22:28:04.003 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191480.log 2023-09-30T22:28:04.003 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162108.log.gz 2023-09-30T22:28:04.003 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160997.log 2023-09-30T22:28:04.003 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.177181.log.gz 2023-09-30T22:28:04.003 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191165.log 2023-09-30T22:28:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191480.log: /var/log/ceph/ceph-client.admin.160997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187390.log 2023-09-30T22:28:04.004 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191480.log.gz 2023-09-30T22:28:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191165.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160997.log.gz 2023-09-30T22:28:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141986.log 2023-09-30T22:28:04.004 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.191165.log.gz 2023-09-30T22:28:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174045.log 2023-09-30T22:28:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188613.log 2023-09-30T22:28:04.004 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.187390.log.gz 2023-09-30T22:28:04.005 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141986.log.gz 2023-09-30T22:28:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156191.log 2023-09-30T22:28:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188613.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.174045.log.gz 2023-09-30T22:28:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138557.log 2023-09-30T22:28:04.005 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.188613.log.gz 2023-09-30T22:28:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176763.log 2023-09-30T22:28:04.005 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156191.log.gz 2023-09-30T22:28:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152517.log 2023-09-30T22:28:04.006 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138557.log.gz 2023-09-30T22:28:04.006 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173875.log 2023-09-30T22:28:04.006 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176763.log: /var/log/ceph/ceph-client.admin.152517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123255.log 2023-09-30T22:28:04.006 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176763.log.gz 2023-09-30T22:28:04.006 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173875.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.152517.log.gz 2023-09-30T22:28:04.006 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179711.log 2023-09-30T22:28:04.006 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173875.log.gz 2023-09-30T22:28:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149297.log 2023-09-30T22:28:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123255.log: /var/log/ceph/ceph-client.admin.179711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153129.log 2023-09-30T22:28:04.007 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123255.log.gz 2023-09-30T22:28:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149297.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179711.log.gz 2023-09-30T22:28:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142280.log 2023-09-30T22:28:04.007 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149297.log.gz 2023-09-30T22:28:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153521.log 2023-09-30T22:28:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153129.log: /var/log/ceph/ceph-client.admin.142280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180966.log 2023-09-30T22:28:04.008 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153129.log.gz 2023-09-30T22:28:04.008 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142280.log.gz 2023-09-30T22:28:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181501.log 2023-09-30T22:28:04.008 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153521.log.gz 2023-09-30T22:28:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192264.log 2023-09-30T22:28:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180966.log: /var/log/ceph/ceph-client.admin.181501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122950.log 2023-09-30T22:28:04.008 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.180966.log.gz 2023-09-30T22:28:04.008 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.181501.log.gz 2023-09-30T22:28:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150157.log 2023-09-30T22:28:04.009 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192264.log.gz 2023-09-30T22:28:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183555.log 2023-09-30T22:28:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150157.log: gzip -5 --verbose -- 87.5%/var/log/ceph/ceph-client.admin.182089.log 2023-09-30T22:28:04.009 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.122950.log.gz 2023-09-30T22:28:04.009 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150157.log.gz 2023-09-30T22:28:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154260.log 2023-09-30T22:28:04.009 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.183555.log.gz 2023-09-30T22:28:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138654.log 2023-09-30T22:28:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154260.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182089.log.gz 2023-09-30T22:28:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193326.log 2023-09-30T22:28:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138654.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154260.log.gz 2023-09-30T22:28:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145208.log 2023-09-30T22:28:04.010 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138654.log.gz 2023-09-30T22:28:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186632.log 2023-09-30T22:28:04.010 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193326.log.gz 2023-09-30T22:28:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174854.log 2023-09-30T22:28:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186632.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.145208.log.gz-5 2023-09-30T22:28:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.179807.log 2023-09-30T22:28:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174854.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.186632.log.gz 2023-09-30T22:28:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176030.log 2023-09-30T22:28:04.011 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.174854.log.gz 2023-09-30T22:28:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147141.log 2023-09-30T22:28:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176030.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.179807.log.gz 2023-09-30T22:28:04.012 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136184.log 2023-09-30T22:28:04.012 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.176030.log.gz 2023-09-30T22:28:04.012 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127976.log 2023-09-30T22:28:04.012 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147141.log.gz 2023-09-30T22:28:04.012 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175686.log 2023-09-30T22:28:04.012 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148121.log 2023-09-30T22:28:04.012 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136184.log.gz 2023-09-30T22:28:04.012 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127976.log.gz 2023-09-30T22:28:04.013 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149936.log 2023-09-30T22:28:04.013 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175686.log.gz 2023-09-30T22:28:04.013 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.148121.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.164703.log 2023-09-30T22:28:04.013 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118454.log 2023-09-30T22:28:04.013 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148121.log.gz 2023-09-30T22:28:04.013 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164703.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149936.log.gz 2023-09-30T22:28:04.013 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148197.log 2023-09-30T22:28:04.013 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164703.log.gz 2023-09-30T22:28:04.014 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.118454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152884.log 2023-09-30T22:28:04.014 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140909.log 2023-09-30T22:28:04.014 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118454.log.gz 2023-09-30T22:28:04.014 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148197.log.gz 2023-09-30T22:28:04.014 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172921.log 2023-09-30T22:28:04.014 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152884.log.gz 2023-09-30T22:28:04.014 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155887.log 2023-09-30T22:28:04.014 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140909.log.gz 2023-09-30T22:28:04.015 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147240.log 2023-09-30T22:28:04.015 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172921.log.gz 2023-09-30T22:28:04.015 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191362.log 2023-09-30T22:28:04.015 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147240.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.155887.log.gz 2023-09-30T22:28:04.015 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151333.log 2023-09-30T22:28:04.015 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147240.log.gz 2023-09-30T22:28:04.015 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144470.log 2023-09-30T22:28:04.015 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.191362.log.gz 2023-09-30T22:28:04.016 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152322.log 2023-09-30T22:28:04.016 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144470.log: gzip -5 --verbose 89.3%-- /var/log/ceph/ceph-client.admin.137478.log -- replaced with /var/log/ceph/ceph-client.admin.151333.log.gz 2023-09-30T22:28:04.016 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:04.016 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152322.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144470.log.gz 2023-09-30T22:28:04.016 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175195.log 2023-09-30T22:28:04.016 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152322.log.gz 2023-09-30T22:28:04.016 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188736.log 2023-09-30T22:28:04.017 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175195.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137478.log.gz 2023-09-30T22:28:04.017 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144201.log 2023-09-30T22:28:04.017 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175195.log.gz 2023-09-30T22:28:04.017 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170002.log 2023-09-30T22:28:04.017 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188736.log.gz 2023-09-30T22:28:04.017 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146974.log 2023-09-30T22:28:04.017 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6%/var/log/ceph/ceph-client.admin.170002.log: -- replaced with /var/log/ceph/ceph-client.admin.144201.log.gz 2023-09-30T22:28:04.017 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138948.log 2023-09-30T22:28:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146974.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.170002.log.gz 2023-09-30T22:28:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188247.log 2023-09-30T22:28:04.018 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146974.log.gz 2023-09-30T22:28:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189324.log 2023-09-30T22:28:04.018 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138948.log.gz 2023-09-30T22:28:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179319.log 2023-09-30T22:28:04.018 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.188247.log.gz 2023-09-30T22:28:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168209.log 2023-09-30T22:28:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179319.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.189324.log.gz 2023-09-30T22:28:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168586.log 2023-09-30T22:28:04.019 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179319.log.gz 2023-09-30T22:28:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123823.log 2023-09-30T22:28:04.019 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168209.log.gz 2023-09-30T22:28:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174999.log 2023-09-30T22:28:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123823.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.168586.log.gz 2023-09-30T22:28:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129620.log 2023-09-30T22:28:04.020 INFO:teuthology.orchestra.run.smithi089.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.123823.log.gz 2023-09-30T22:28:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124970.log 2023-09-30T22:28:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129620.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174999.log.gz 2023-09-30T22:28:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154009.log 2023-09-30T22:28:04.020 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5%/var/log/ceph/ceph-client.admin.124970.log: -- replaced with /var/log/ceph/ceph-client.admin.129620.log.gz 2023-09-30T22:28:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147363.log 2023-09-30T22:28:04.020 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124970.log.gz 2023-09-30T22:28:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185272.log 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147363.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154009.log.gz 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163700.log 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147363.log.gz 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124215.log 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.185272.log.gz 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150158.log 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163700.log.gz 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186240.log 2023-09-30T22:28:04.022 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8%/var/log/ceph/ceph-client.admin.150158.log: -- replaced with /var/log/ceph/ceph-client.admin.124215.log.gz 2023-09-30T22:28:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158909.log 2023-09-30T22:28:04.022 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.150158.log.gz 2023-09-30T22:28:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143042.log 2023-09-30T22:28:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158909.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.186240.log.gz 2023-09-30T22:28:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186580.log 2023-09-30T22:28:04.022 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158909.log.gz 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159693.log 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143042.log.gz 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187560.log 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159693.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.186580.log.gz 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155773.log 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.159693.log.gz 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178979.log 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.187560.log.gz--verbose 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.184519.log 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155773.log: /var/log/ceph/ceph-client.admin.178979.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155773.log.gz 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159354.log 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.178979.log.gz 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153055.log 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184519.log: /var/log/ceph/ceph-client.admin.159354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185395.log 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.184519.log.gz 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.159354.log.gz 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142229.log 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4%/var/log/ceph/ceph-client.admin.185395.log: -- replaced with /var/log/ceph/ceph-client.admin.153055.log.gz 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183751.log 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.185395.log.gz 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173065.log 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142229.log: /var/log/ceph/ceph-client.admin.183751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129057.log 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142229.log.gz 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183751.log.gz 2023-09-30T22:28:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179223.log 2023-09-30T22:28:04.026 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.173065.log.gz 2023-09-30T22:28:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163304.log 2023-09-30T22:28:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158713.log 2023-09-30T22:28:04.026 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129057.log.gz 2023-09-30T22:28:04.026 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179223.log.gz 2023-09-30T22:28:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180717.log 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163304.log.gz 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140222.log 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173627.log 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158713.log.gz 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.180717.log.gz 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139321.log 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140222.log.gz 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173627.log: /var/log/ceph/ceph-client.admin.182705.log 2023-09-30T22:28:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162643.log 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.173627.log.gz 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139321.log.gz 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175835.log 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125825.log 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.182705.log.gz 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162643.log.gz 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143750.log 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1%/var/log/ceph/ceph-client.admin.125825.log: -- replaced with /var/log/ceph/ceph-client.admin.175835.log.gz 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173287.log 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125825.log.gz 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170787.log 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143750.log.gz 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194509.log 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173287.log.gz 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160877.log 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170787.log.gz 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162912.log 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194509.log: /var/log/ceph/ceph-client.admin.160877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148761.log 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.194509.log.gz 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1%/var/log/ceph/ceph-client.admin.162912.log: -- replaced with /var/log/ceph/ceph-client.admin.160877.log.gz 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138458.log 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148761.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162912.log.gz 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189742.log 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148761.log.gz 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128254.log 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138458.log.gz 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169023.log 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128254.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189742.log.gz 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175443.log 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128254.log.gz 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154819.log 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169023.log.gz 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160971.log 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154819.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175443.log.gz 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174633.log 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.154819.log.gz 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186239.log 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi089.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.160971.log.gz 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146796.log 2023-09-30T22:28:04.033 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186239.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.174633.log.gz 2023-09-30T22:28:04.033 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179345.log 2023-09-30T22:28:04.033 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.186239.log.gz 2023-09-30T22:28:04.033 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154989.log 2023-09-30T22:28:04.033 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146796.log.gz 2023-09-30T22:28:04.033 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136379.log 2023-09-30T22:28:04.033 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.179345.log.gz 2023-09-30T22:28:04.033 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141202.log 2023-09-30T22:28:04.034 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136379.log: gzip 88.5%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.154989.log.gz/var/log/ceph/ceph-client.admin.147168.log 2023-09-30T22:28:04.034 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:04.034 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136379.log.gz 2023-09-30T22:28:04.034 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161661.log 2023-09-30T22:28:04.034 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147168.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141202.log.gz 2023-09-30T22:28:04.034 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173260.log 2023-09-30T22:28:04.034 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147168.log.gz 2023-09-30T22:28:04.034 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122505.log 2023-09-30T22:28:04.035 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161661.log.gz 2023-09-30T22:28:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122652.log 2023-09-30T22:28:04.035 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.173260.log.gz 2023-09-30T22:28:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155494.log 2023-09-30T22:28:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122505.log: /var/log/ceph/ceph-client.admin.122652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184339.log 2023-09-30T22:28:04.035 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122505.log.gz 2023-09-30T22:28:04.035 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122652.log.gz 2023-09-30T22:28:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160111.log 2023-09-30T22:28:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171172.log 2023-09-30T22:28:04.036 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155494.log.gz 2023-09-30T22:28:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184339.log: /var/log/ceph/ceph-client.admin.160111.log: gzip 88.5%-5 -- replaced with /var/log/ceph/ceph-client.admin.184339.log.gz--verbose 2023-09-30T22:28:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.165465.log 2023-09-30T22:28:04.036 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160111.log.gz 2023-09-30T22:28:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176645.log 2023-09-30T22:28:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171172.log: /var/log/ceph/ceph-client.admin.165465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163260.log 2023-09-30T22:28:04.036 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.171172.log.gz 2023-09-30T22:28:04.037 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.165465.log.gz 2023-09-30T22:28:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140418.log 2023-09-30T22:28:04.037 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176645.log.gz 2023-09-30T22:28:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139721.log 2023-09-30T22:28:04.037 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163260.log.gz 2023-09-30T22:28:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124529.log 2023-09-30T22:28:04.037 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140418.log.gz 2023-09-30T22:28:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182285.log 2023-09-30T22:28:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139721.log: /var/log/ceph/ceph-client.admin.124529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174019.log 2023-09-30T22:28:04.038 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139721.log.gz 2023-09-30T22:28:04.038 INFO:teuthology.orchestra.run.smithi089.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.124529.log.gz 2023-09-30T22:28:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128308.log 2023-09-30T22:28:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175272.log 88.3% 2023-09-30T22:28:04.038 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.182285.log.gz 2023-09-30T22:28:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174019.log: /var/log/ceph/ceph-client.admin.128308.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174019.log.gz 2023-09-30T22:28:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168923.log 2023-09-30T22:28:04.039 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128308.log.gz 2023-09-30T22:28:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190722.log 2023-09-30T22:28:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189005.log 2023-09-30T22:28:04.039 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.175272.log.gz 2023-09-30T22:28:04.039 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.168923.log.gz 2023-09-30T22:28:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177359.log 2023-09-30T22:28:04.039 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190722.log.gz 2023-09-30T22:28:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147436.log 2023-09-30T22:28:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182651.log 2023-09-30T22:28:04.040 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189005.log.gz 2023-09-30T22:28:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147436.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177359.log.gz 2023-09-30T22:28:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173823.log 2023-09-30T22:28:04.040 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147436.log.gz 2023-09-30T22:28:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137086.log 2023-09-30T22:28:04.040 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.182651.log.gz 2023-09-30T22:28:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180913.log 2023-09-30T22:28:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173823.log: /var/log/ceph/ceph-client.admin.137086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138066.log 2023-09-30T22:28:04.041 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173823.log.gz 2023-09-30T22:28:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180913.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.137086.log.gz 2023-09-30T22:28:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174685.log 2023-09-30T22:28:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191088.log 2023-09-30T22:28:04.041 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180913.log.gz 2023-09-30T22:28:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138066.log: /var/log/ceph/ceph-client.admin.174685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169172.log 2023-09-30T22:28:04.042 INFO:teuthology.orchestra.run.smithi089.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.138066.log.gz 2023-09-30T22:28:04.042 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.174685.log.gz 2023-09-30T22:28:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139122.log 2023-09-30T22:28:04.042 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.191088.log.gz 2023-09-30T22:28:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159131.log 2023-09-30T22:28:04.042 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169172.log.gz 2023-09-30T22:28:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139537.log 2023-09-30T22:28:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159131.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139122.log.gz 2023-09-30T22:28:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166955.log 2023-09-30T22:28:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139537.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.159131.log.gz 2023-09-30T22:28:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167425.log 2023-09-30T22:28:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166955.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139537.log.gz 2023-09-30T22:28:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173679.log 2023-09-30T22:28:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167425.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166955.log.gz 2023-09-30T22:28:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153866.log 2023-09-30T22:28:04.043 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167425.log.gz 2023-09-30T22:28:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122362.log 2023-09-30T22:28:04.044 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.173679.log.gz 2023-09-30T22:28:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184637.log 2023-09-30T22:28:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122362.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153866.log.gz 2023-09-30T22:28:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150473.log 2023-09-30T22:28:04.044 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122362.log.gz 2023-09-30T22:28:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147586.log 2023-09-30T22:28:04.044 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.184637.log.gz 2023-09-30T22:28:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124823.log 2023-09-30T22:28:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147586.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.150473.log.gz 2023-09-30T22:28:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151479.log 2023-09-30T22:28:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124823.log: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.147586.log.gz--verbose 2023-09-30T22:28:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.174882.log 2023-09-30T22:28:04.045 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124823.log.gz 2023-09-30T22:28:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163701.log 2023-09-30T22:28:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174882.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151479.log.gz 2023-09-30T22:28:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152911.log 2023-09-30T22:28:04.046 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174882.log.gz 2023-09-30T22:28:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.163701.log: -- /var/log/ceph/ceph-client.admin.158321.log 2023-09-30T22:28:04.046 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163701.log.gz 2023-09-30T22:28:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147784.log 2023-09-30T22:28:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158321.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152911.log.gz 2023-09-30T22:28:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165269.log 2023-09-30T22:28:04.047 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158321.log.gz 2023-09-30T22:28:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172281.log 2023-09-30T22:28:04.047 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147784.log.gz 2023-09-30T22:28:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165269.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-09-30T22:28:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172281.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165269.log.gz 2023-09-30T22:28:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170321.log 2023-09-30T22:28:04.047 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172281.log.gz 2023-09-30T22:28:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149571.log 2023-09-30T22:28:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182115.log 2023-09-30T22:28:04.048 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170321.log.gz 2023-09-30T22:28:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188391.log 2023-09-30T22:28:04.048 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.149571.log.gz 2023-09-30T22:28:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188540.log 2023-09-30T22:28:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182115.log: /var/log/ceph/ceph-client.admin.188391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165583.log 2023-09-30T22:28:04.048 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.182115.log.gz 2023-09-30T22:28:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188540.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188391.log.gz 2023-09-30T22:28:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173261.log 2023-09-30T22:28:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165583.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.188540.log.gz 2023-09-30T22:28:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137333.log 2023-09-30T22:28:04.049 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165583.log.gz 2023-09-30T22:28:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135789.log 2023-09-30T22:28:04.049 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.173261.log.gz 2023-09-30T22:28:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168820.log 2023-09-30T22:28:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169367.log 2023-09-30T22:28:04.050 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137333.log.gz 2023-09-30T22:28:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135789.log: /var/log/ceph/ceph-client.admin.168820.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.135789.log.gz 2023-09-30T22:28:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139489.log 2023-09-30T22:28:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146409.log 2023-09-30T22:28:04.050 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168820.log.gz 2023-09-30T22:28:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169367.log: /var/log/ceph/ceph-client.admin.139489.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169367.log.gz 2023-09-30T22:28:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150355.log 2023-09-30T22:28:04.050 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139489.log.gz 2023-09-30T22:28:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171569.log 2023-09-30T22:28:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146409.log: /var/log/ceph/ceph-client.admin.150355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127367.log 2023-09-30T22:28:04.051 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146409.log.gz 2023-09-30T22:28:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171569.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150355.log.gz 2023-09-30T22:28:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160485.log 2023-09-30T22:28:04.051 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171569.log.gz 2023-09-30T22:28:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190552.log 2023-09-30T22:28:04.051 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.127367.log.gz 2023-09-30T22:28:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164432.log 2023-09-30T22:28:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189128.log 2023-09-30T22:28:04.052 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160485.log.gz 2023-09-30T22:28:04.052 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190552.log.gz 2023-09-30T22:28:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161956.log 2023-09-30T22:28:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.189128.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.163155.log 2023-09-30T22:28:04.052 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164432.log.gz 2023-09-30T22:28:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161956.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189128.log.gz 2023-09-30T22:28:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170739.log 2023-09-30T22:28:04.053 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161956.log.gz 2023-09-30T22:28:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143632.log 2023-09-30T22:28:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163155.log: /var/log/ceph/ceph-client.admin.170739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128379.log 2023-09-30T22:28:04.053 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163155.log.gz 2023-09-30T22:28:04.053 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170739.log.gz 2023-09-30T22:28:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163081.log 2023-09-30T22:28:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152833.log 2023-09-30T22:28:04.054 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143632.log.gz 2023-09-30T22:28:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128379.log: /var/log/ceph/ceph-client.admin.163081.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128379.log.gz 2023-09-30T22:28:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152833.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163081.log.gz 2023-09-30T22:28:04.054 INFO:teuthology.orchestra.run.smithi089.stderr: 95.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-30T22:28:04.054 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152833.log.gz 2023-09-30T22:28:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184023.log 2023-09-30T22:28:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128069.log 2023-09-30T22:28:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144815.log 2023-09-30T22:28:04.055 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.184023.log.gz 2023-09-30T22:28:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128069.log: /var/log/ceph/ceph-client.admin.185668.log 2023-09-30T22:28:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144815.log: gzip -5 --verbose -- 88.3%/var/log/ceph/ceph-client.admin.145012.log 2023-09-30T22:28:04.055 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.128069.log.gz 2023-09-30T22:28:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185668.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144815.log.gz 2023-09-30T22:28:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184442.log 2023-09-30T22:28:04.055 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.185668.log.gz 2023-09-30T22:28:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179611.log 2023-09-30T22:28:04.056 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145012.log.gz 2023-09-30T22:28:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129400.log 2023-09-30T22:28:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179611.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.184442.log.gz 2023-09-30T22:28:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194704.log 2023-09-30T22:28:04.056 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179611.log.gz 2023-09-30T22:28:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157562.log 2023-09-30T22:28:04.056 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129400.log.gz 2023-09-30T22:28:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150524.log 2023-09-30T22:28:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157562.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.194704.log.gz 2023-09-30T22:28:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184050.log 2023-09-30T22:28:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150524.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.157562.log.gz 2023-09-30T22:28:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144649.log 2023-09-30T22:28:04.057 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150524.log.gz 2023-09-30T22:28:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146308.log 2023-09-30T22:28:04.057 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.184050.log.gz 2023-09-30T22:28:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142966.log 2023-09-30T22:28:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146308.log: gzip 88.1%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.144649.log.gz-- 2023-09-30T22:28:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146652.log 2023-09-30T22:28:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142966.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146308.log.gz 2023-09-30T22:28:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159167.log 2023-09-30T22:28:04.058 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142966.log.gz 2023-09-30T22:28:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178586.log 2023-09-30T22:28:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159167.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146652.log.gz 2023-09-30T22:28:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157171.log 2023-09-30T22:28:04.059 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159167.log.gz 2023-09-30T22:28:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125874.log 2023-09-30T22:28:04.059 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.178586.log.gz 2023-09-30T22:28:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171916.log 2023-09-30T22:28:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125874.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157171.log.gz 2023-09-30T22:28:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174095.log 2023-09-30T22:28:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171916.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125874.log.gz 2023-09-30T22:28:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139634.log 2023-09-30T22:28:04.060 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.171916.log.gz 2023-09-30T22:28:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151726.log 2023-09-30T22:28:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139634.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.174095.log.gz 2023-09-30T22:28:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181162.log 2023-09-30T22:28:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151726.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139634.log.gz 2023-09-30T22:28:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123157.log 2023-09-30T22:28:04.060 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151726.log.gz 2023-09-30T22:28:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172843.log 2023-09-30T22:28:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123157.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.181162.log.gz 2023-09-30T22:28:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170099.log 2023-09-30T22:28:04.061 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123157.log.gz 2023-09-30T22:28:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166860.log 2023-09-30T22:28:04.061 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172843.log.gz 2023-09-30T22:28:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185003.log 2023-09-30T22:28:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166860.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.170099.log.gz 2023-09-30T22:28:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134261.log 2023-09-30T22:28:04.062 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.166860.log.gz 2023-09-30T22:28:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160085.log 2023-09-30T22:28:04.062 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.185003.log.gz 2023-09-30T22:28:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.134261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159941.log 2023-09-30T22:28:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160085.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.134261.log.gz 2023-09-30T22:28:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174411.log 2023-09-30T22:28:04.062 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160085.log.gz 2023-09-30T22:28:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163845.log 2023-09-30T22:28:04.063 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159941.log.gz 2023-09-30T22:28:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155087.log 2023-09-30T22:28:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163845.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174411.log.gz 2023-09-30T22:28:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194108.log 2023-09-30T22:28:04.063 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163845.log.gz 2023-09-30T22:28:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172895.log 2023-09-30T22:28:04.063 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155087.log.gz 2023-09-30T22:28:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170383.log 2023-09-30T22:28:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172895.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.194108.log.gz 2023-09-30T22:28:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144862.log 2023-09-30T22:28:04.064 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.172895.log.gz 2023-09-30T22:28:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188762.log 2023-09-30T22:28:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137282.log 2023-09-30T22:28:04.064 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.170383.log.gz 2023-09-30T22:28:04.064 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144862.log.gz 2023-09-30T22:28:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.188762.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.144274.log 2023-09-30T22:28:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137282.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188762.log.gz 2023-09-30T22:28:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142574.log 2023-09-30T22:28:04.065 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137282.log.gz 2023-09-30T22:28:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179737.log 2023-09-30T22:28:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142574.log: 89.2%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.144274.log.gz-- 2023-09-30T22:28:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123206.log 2023-09-30T22:28:04.066 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142574.log.gz 2023-09-30T22:28:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154401.log 2023-09-30T22:28:04.066 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179737.log.gz 2023-09-30T22:28:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165046.log 2023-09-30T22:28:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154401.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123206.log.gz 2023-09-30T22:28:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188417.log 2023-09-30T22:28:04.066 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154401.log.gz 2023-09-30T22:28:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194227.log 2023-09-30T22:28:04.067 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.165046.log.gz 2023-09-30T22:28:04.067 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128462.log 2023-09-30T22:28:04.067 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194227.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.188417.log.gz 2023-09-30T22:28:04.067 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159745.log 2023-09-30T22:28:04.067 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.194227.log.gz 2023-09-30T22:28:04.067 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136380.log 2023-09-30T22:28:04.067 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128462.log.gz 2023-09-30T22:28:04.067 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179810.log 2023-09-30T22:28:04.068 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136380.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159745.log.gz 2023-09-30T22:28:04.068 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153643.log 2023-09-30T22:28:04.068 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.136380.log.gz 2023-09-30T22:28:04.068 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166563.log 2023-09-30T22:28:04.068 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179810.log.gz 2023-09-30T22:28:04.068 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167621.log 2023-09-30T22:28:04.068 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153643.log.gz 2023-09-30T22:28:04.068 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156583.log 2023-09-30T22:28:04.069 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166563.log.gz 2023-09-30T22:28:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152715.log 2023-09-30T22:28:04.069 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167621.log.gz 2023-09-30T22:28:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154205.log 2023-09-30T22:28:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152715.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156583.log.gz 2023-09-30T22:28:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147018.log 2023-09-30T22:28:04.069 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152715.log.gz 2023-09-30T22:28:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166981.log 2023-09-30T22:28:04.070 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154205.log.gz 2023-09-30T22:28:04.070 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147018.log: /var/log/ceph/ceph-client.admin.188195.log 2023-09-30T22:28:04.070 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166981.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147018.log.gz 2023-09-30T22:28:04.070 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165464.log 2023-09-30T22:28:04.070 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166981.log.gz 2023-09-30T22:28:04.070 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195377.log 2023-09-30T22:28:04.070 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.188195.log.gz 2023-09-30T22:28:04.071 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159105.log 2023-09-30T22:28:04.071 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.195377.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165464.log.gz 2023-09-30T22:28:04.071 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123822.log 2023-09-30T22:28:04.071 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.195377.log.gz 2023-09-30T22:28:04.071 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176840.log 2023-09-30T22:28:04.071 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125531.log 2023-09-30T22:28:04.071 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159105.log.gz 2023-09-30T22:28:04.071 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123822.log: /var/log/ceph/ceph-client.admin.176840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138682.log 2023-09-30T22:28:04.072 INFO:teuthology.orchestra.run.smithi089.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.123822.log.gz 2023-09-30T22:28:04.072 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176840.log.gz 2023-09-30T22:28:04.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155088.log 2023-09-30T22:28:04.072 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125531.log.gz 2023-09-30T22:28:04.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123647.log 2023-09-30T22:28:04.072 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138682.log.gz 2023-09-30T22:28:04.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140125.log 2023-09-30T22:28:04.072 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.155088.log.gz 2023-09-30T22:28:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187829.log 2023-09-30T22:28:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140125.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123647.log.gz 2023-09-30T22:28:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156631.log 2023-09-30T22:28:04.073 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140125.log.gz 2023-09-30T22:28:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170491.log 2023-09-30T22:28:04.073 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.187829.log.gz 2023-09-30T22:28:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122848.log 2023-09-30T22:28:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170491.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156631.log.gz 2023-09-30T22:28:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178587.log 2023-09-30T22:28:04.074 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170491.log.gz 2023-09-30T22:28:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151923.log 2023-09-30T22:28:04.074 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122848.log.gz 2023-09-30T22:28:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163649.log 2023-09-30T22:28:04.074 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.178587.log.gz 2023-09-30T22:28:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186462.log 2023-09-30T22:28:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163649.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151923.log.gz 2023-09-30T22:28:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151531.log 2023-09-30T22:28:04.075 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163649.log.gz 2023-09-30T22:28:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122554.log 2023-09-30T22:28:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151531.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.186462.log.gz 2023-09-30T22:28:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179371.log 2023-09-30T22:28:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122554.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151531.log.gz 2023-09-30T22:28:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166395.log 2023-09-30T22:28:04.075 INFO:teuthology.orchestra.run.smithi089.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122554.log.gz 2023-09-30T22:28:04.076 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157981.log 2023-09-30T22:28:04.076 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166395.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179371.log.gz 2023-09-30T22:28:04.076 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188577.log 2023-09-30T22:28:04.076 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.166395.log.gz 2023-09-30T22:28:04.076 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140661.log 2023-09-30T22:28:04.076 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157981.log.gz 2023-09-30T22:28:04.076 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181109.log 2023-09-30T22:28:04.076 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140661.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.188577.log.gz 2023-09-30T22:28:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129253.log 2023-09-30T22:28:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181109.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140661.log.gz 2023-09-30T22:28:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149740.log 2023-09-30T22:28:04.077 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.181109.log.gz 2023-09-30T22:28:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158664.log 2023-09-30T22:28:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149740.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129253.log.gz 2023-09-30T22:28:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154674.log 2023-09-30T22:28:04.077 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149740.log.gz 2023-09-30T22:28:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190011.log 2023-09-30T22:28:04.078 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158664.log.gz 2023-09-30T22:28:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161538.log 2023-09-30T22:28:04.078 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154674.log.gz 2023-09-30T22:28:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169537.log 2023-09-30T22:28:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161538.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190011.log.gz 2023-09-30T22:28:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-09-30T22:28:04.078 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161538.log.gz 2023-09-30T22:28:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155211.log 2023-09-30T22:28:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.3.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.169537.log.gz 2023-09-30T22:28:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140465.log 2023-09-30T22:28:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191974.log 2023-09-30T22:28:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140465.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155211.log.gz 2023-09-30T22:28:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164015.log 2023-09-30T22:28:04.079 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140465.log.gz 2023-09-30T22:28:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182455.log 2023-09-30T22:28:04.080 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191974.log.gz 2023-09-30T22:28:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153447.log 2023-09-30T22:28:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164015.log: /var/log/ceph/ceph-client.admin.182455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184219.log 2023-09-30T22:28:04.080 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164015.log.gz 2023-09-30T22:28:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153447.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182455.log.gz 2023-09-30T22:28:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175077.log 2023-09-30T22:28:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149767.log 2023-09-30T22:28:04.081 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184219.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.153447.log.gz 2023-09-30T22:28:04.081 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175077.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.184219.log.gz 2023-09-30T22:28:04.081 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174632.log 2023-09-30T22:28:04.081 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175077.log.gz 2023-09-30T22:28:04.081 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168774.log 2023-09-30T22:28:04.081 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149767.log: /var/log/ceph/ceph-client.admin.174632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192489.log 2023-09-30T22:28:04.081 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.149767.log.gz 2023-09-30T22:28:04.081 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174632.log.gz 2023-09-30T22:28:04.082 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150865.log 2023-09-30T22:28:04.082 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192489.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168774.log.gz 2023-09-30T22:28:04.082 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183461.log 2023-09-30T22:28:04.082 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.192489.log.gz 2023-09-30T22:28:04.082 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141398.log 2023-09-30T22:28:04.082 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183461.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150865.log.gz 2023-09-30T22:28:04.082 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146498.log 2023-09-30T22:28:04.082 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.183461.log.gz 2023-09-30T22:28:04.083 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139919.log 2023-09-30T22:28:04.083 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141398.log.gz 2023-09-30T22:28:04.083 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157415.log 2023-09-30T22:28:04.083 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146498.log.gz 2023-09-30T22:28:04.083 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168327.log 2023-09-30T22:28:04.083 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139919.log.gz 2023-09-30T22:28:04.083 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163895.log 2023-09-30T22:28:04.083 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157415.log.gz 2023-09-30T22:28:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128861.log 2023-09-30T22:28:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168327.log: /var/log/ceph/ceph-client.admin.163895.log: gzip 88.2%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.168327.log.gz/var/log/ceph/ceph-client.admin.173431.log 2023-09-30T22:28:04.084 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:04.084 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163895.log.gz 2023-09-30T22:28:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181357.log 2023-09-30T22:28:04.084 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128861.log.gz 2023-09-30T22:28:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177607.log 2023-09-30T22:28:04.084 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173431.log.gz 2023-09-30T22:28:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154110.log 2023-09-30T22:28:04.085 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181357.log.gz 2023-09-30T22:28:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191580.log 2023-09-30T22:28:04.085 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.177607.log.gz 2023-09-30T22:28:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194431.log 2023-09-30T22:28:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186998.log 2023-09-30T22:28:04.085 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154110.log.gz 2023-09-30T22:28:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191580.log: /var/log/ceph/ceph-client.admin.194431.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.191580.log.gz 2023-09-30T22:28:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141641.log 2023-09-30T22:28:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160681.log /var/log/ceph/ceph-client.admin.186998.log: 2023-09-30T22:28:04.086 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.194431.log.gz 2023-09-30T22:28:04.086 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.186998.log.gz 2023-09-30T22:28:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195132.log 2023-09-30T22:28:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148198.log 2023-09-30T22:28:04.086 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141641.log.gz 2023-09-30T22:28:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160681.log: /var/log/ceph/ceph-client.admin.195132.log: gzip 89.3%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.160681.log.gz/var/log/ceph/ceph-client.admin.124188.log 2023-09-30T22:28:04.087 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:04.087 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.195132.log.gz 2023-09-30T22:28:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148001.log 2023-09-30T22:28:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148198.log: /var/log/ceph/ceph-client.admin.124188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143872.log 2023-09-30T22:28:04.087 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148198.log.gz 2023-09-30T22:28:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148001.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124188.log.gz 2023-09-30T22:28:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141667.log 2023-09-30T22:28:04.088 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148001.log.gz 2023-09-30T22:28:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188836.log 2023-09-30T22:28:04.088 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143872.log.gz 2023-09-30T22:28:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171375.log 2023-09-30T22:28:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141667.log: /var/log/ceph/ceph-client.admin.188836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141249.log 2023-09-30T22:28:04.088 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141667.log.gz 2023-09-30T22:28:04.088 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188836.log.gz 2023-09-30T22:28:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129807.log 2023-09-30T22:28:04.089 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171375.log.gz 2023-09-30T22:28:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195230.log 2023-09-30T22:28:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141249.log: /var/log/ceph/ceph-client.admin.129807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191359.log 2023-09-30T22:28:04.089 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141249.log.gz 2023-09-30T22:28:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152245.log 2023-09-30T22:28:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.195230.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129807.log.gz 2023-09-30T22:28:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191359.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.195230.log.gz 2023-09-30T22:28:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165047.log 2023-09-30T22:28:04.090 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191359.log.gz 2023-09-30T22:28:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-09-30T22:28:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152245.log: /var/log/ceph/ceph-client.admin.165047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142500.log 2023-09-30T22:28:04.090 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152245.log.gz 2023-09-30T22:28:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.0.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165047.log.gz 2023-09-30T22:28:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140810.log 2023-09-30T22:28:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164237.log 2023-09-30T22:28:04.091 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142500.log.gz 2023-09-30T22:28:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140321.log 2023-09-30T22:28:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164237.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140810.log.gz 2023-09-30T22:28:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194887.log 2023-09-30T22:28:04.091 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164237.log.gz 2023-09-30T22:28:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140321.log: /var/log/ceph/ceph-client.admin.194887.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140321.log.gz 2023-09-30T22:28:04.091 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.194887.log.gz 2023-09-30T22:28:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173531.log 2023-09-30T22:28:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138850.log 2023-09-30T22:28:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167203.log 2023-09-30T22:28:04.092 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173531.log.gz 2023-09-30T22:28:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138289.log 2023-09-30T22:28:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175490.log 2023-09-30T22:28:04.092 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138850.log.gz 2023-09-30T22:28:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167203.log: /var/log/ceph/ceph-client.admin.138289.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167203.log.gz 2023-09-30T22:28:04.093 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0%/var/log/ceph/ceph-client.admin.175490.log: -- replaced with /var/log/ceph/ceph-client.admin.138289.log.gz 2023-09-30T22:28:04.093 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175490.log.gz 2023-09-30T22:28:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178810.log 2023-09-30T22:28:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192685.log 2023-09-30T22:28:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140291.log 2023-09-30T22:28:04.093 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.178810.log.gz 2023-09-30T22:28:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127591.log 2023-09-30T22:28:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148905.log 2023-09-30T22:28:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140291.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192685.log.gz 2023-09-30T22:28:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127591.log: gzip -5 89.1%--verbose -- /var/log/ceph/ceph-client.admin.141471.log -- replaced with /var/log/ceph/ceph-client.admin.140291.log.gz 2023-09-30T22:28:04.094 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:04.094 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127591.log.gz 2023-09-30T22:28:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170909.log 2023-09-30T22:28:04.094 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148905.log.gz 2023-09-30T22:28:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141471.log: /var/log/ceph/ceph-client.admin.170909.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141471.log.gz 2023-09-30T22:28:04.094 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170909.log.gz 2023-09-30T22:28:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148370.log 2023-09-30T22:28:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160281.log 2023-09-30T22:28:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189423.log 2023-09-30T22:28:04.095 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148370.log.gz 2023-09-30T22:28:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128022.log 2023-09-30T22:28:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180521.log 2023-09-30T22:28:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189423.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160281.log.gz 2023-09-30T22:28:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128022.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.189423.log.gzgzip 2023-09-30T22:28:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.184807.log 2023-09-30T22:28:04.096 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.128022.log.gz 2023-09-30T22:28:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180521.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180521.log.gz 2023-09-30T22:28:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184807.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.184807.log.gz 2023-09-30T22:28:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141594.log 2023-09-30T22:28:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154035.log 2023-09-30T22:28:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165661.log 2023-09-30T22:28:04.096 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4%/var/log/ceph/ceph-client.admin.154035.log: -- replaced with /var/log/ceph/ceph-client.admin.141594.log.gz 2023-09-30T22:28:04.097 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154035.log.gz 2023-09-30T22:28:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165661.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.165661.log.gz 2023-09-30T22:28:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125482.log 2023-09-30T22:28:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190134.log 2023-09-30T22:28:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191702.log 2023-09-30T22:28:04.194 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125482.log.gz/var/log/ceph/ceph-client.admin.190134.log: 2023-09-30T22:28:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185617.log 2023-09-30T22:28:04.195 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190134.log.gz 2023-09-30T22:28:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176985.log 2023-09-30T22:28:04.195 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.191702.log.gz 2023-09-30T22:28:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148735.log 2023-09-30T22:28:04.195 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.185617.log.gz 2023-09-30T22:28:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176985.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.176985.log.gz 2023-09-30T22:28:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148735.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148735.log.gz 2023-09-30T22:28:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192121.log 2023-09-30T22:28:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178953.log 2023-09-30T22:28:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157954.log 2023-09-30T22:28:04.196 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192121.log.gz/var/log/ceph/ceph-client.admin.178953.log: 2023-09-30T22:28:04.196 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178953.log.gz 2023-09-30T22:28:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136967.log 2023-09-30T22:28:04.196 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157954.log.gz 2023-09-30T22:28:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154623.log 2023-09-30T22:28:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148539.log 2023-09-30T22:28:04.197 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136967.log.gz 2023-09-30T22:28:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154623.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.154623.log.gz 2023-09-30T22:28:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148539.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148539.log.gz 2023-09-30T22:28:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118159.log 2023-09-30T22:28:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183043.log 2023-09-30T22:28:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.118159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127949.log 2023-09-30T22:28:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183043.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.118159.log.gz 2023-09-30T22:28:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127949.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183043.log.gz 2023-09-30T22:28:04.198 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127949.log.gz 2023-09-30T22:28:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160538.log 2023-09-30T22:28:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185473.log 2023-09-30T22:28:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182874.log 2023-09-30T22:28:04.198 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8%/var/log/ceph/ceph-client.admin.185473.log: -- replaced with /var/log/ceph/ceph-client.admin.160538.log.gz 2023-09-30T22:28:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122603.log 2023-09-30T22:28:04.199 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.185473.log.gz 2023-09-30T22:28:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192562.log 2023-09-30T22:28:04.199 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182874.log.gz 2023-09-30T22:28:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128280.log 2023-09-30T22:28:04.199 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122603.log.gz 2023-09-30T22:28:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160202.log 2023-09-30T22:28:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128280.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.192562.log.gz 2023-09-30T22:28:04.199 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128280.log.gz 2023-09-30T22:28:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160202.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160202.log.gz 2023-09-30T22:28:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172726.log 2023-09-30T22:28:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155697.log 2023-09-30T22:28:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190330.log 2023-09-30T22:28:04.200 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.172726.log.gz 2023-09-30T22:28:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146553.log 2023-09-30T22:28:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152518.log 2023-09-30T22:28:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190330.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.155697.log.gz 2023-09-30T22:28:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187756.log 2023-09-30T22:28:04.201 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.190330.log.gz 2023-09-30T22:28:04.201 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146553.log.gz 2023-09-30T22:28:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160587.log 2023-09-30T22:28:04.201 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152518.log.gz 2023-09-30T22:28:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187756.log: /var/log/ceph/ceph-client.admin.160587.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.187756.log.gz 2023-09-30T22:28:04.201 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.160587.log.gz 2023-09-30T22:28:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145573.log 2023-09-30T22:28:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148199.log 2023-09-30T22:28:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181967.log 2023-09-30T22:28:04.202 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145573.log.gz 2023-09-30T22:28:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169511.log 2023-09-30T22:28:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172085.log 2023-09-30T22:28:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181967.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148199.log.gz 2023-09-30T22:28:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169511.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.181967.log.gz 2023-09-30T22:28:04.203 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9%/var/log/ceph/ceph-client.admin.172085.log: -- replaced with /var/log/ceph/ceph-client.admin.169511.log.gz 2023-09-30T22:28:04.203 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.172085.log.gz 2023-09-30T22:28:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146945.log 2023-09-30T22:28:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181475.log 2023-09-30T22:28:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129155.log 2023-09-30T22:28:04.203 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.146945.log.gz 2023-09-30T22:28:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172355.log 2023-09-30T22:28:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148317.log 2023-09-30T22:28:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129155.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.181475.log.gz 2023-09-30T22:28:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172355.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129155.log.gz 2023-09-30T22:28:04.204 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.172355.log.gz/var/log/ceph/ceph-client.admin.148317.log: 2023-09-30T22:28:04.204 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.148317.log.gz 2023-09-30T22:28:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128763.log 2023-09-30T22:28:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135269.log 2023-09-30T22:28:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136498.log 2023-09-30T22:28:04.205 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.128763.log.gz 2023-09-30T22:28:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164825.log 2023-09-30T22:28:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157955.log 2023-09-30T22:28:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136498.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135269.log.gz 2023-09-30T22:28:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151725.log 2023-09-30T22:28:04.205 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136498.log.gz 2023-09-30T22:28:04.205 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.164825.log.gz 2023-09-30T22:28:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157955.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157955.log.gz 2023-09-30T22:28:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151725.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151725.log.gz 2023-09-30T22:28:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129498.log 2023-09-30T22:28:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164066.log 2023-09-30T22:28:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 /var/log/ceph/ceph-client.admin.129498.log: --verbose -- /var/log/ceph/ceph-client.admin.185871.log 2023-09-30T22:28:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164066.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129498.log.gz 2023-09-30T22:28:04.206 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164066.log.gz 2023-09-30T22:28:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185871.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.185871.log.gz 2023-09-30T22:28:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190108.log 2023-09-30T22:28:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147559.log 2023-09-30T22:28:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160709.log 2023-09-30T22:28:04.207 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2%/var/log/ceph/ceph-client.admin.147559.log: -- replaced with /var/log/ceph/ceph-client.admin.190108.log.gz 2023-09-30T22:28:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192659.log 2023-09-30T22:28:04.207 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147559.log.gz 2023-09-30T22:28:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184833.log 2023-09-30T22:28:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158569.log 2023-09-30T22:28:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192659.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160709.log.gz 2023-09-30T22:28:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192463.log 2023-09-30T22:28:04.208 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.192659.log.gz 2023-09-30T22:28:04.208 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.184833.log.gz 2023-09-30T22:28:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158569.log: /var/log/ceph/ceph-client.admin.192463.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158569.log.gz 2023-09-30T22:28:04.208 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192463.log.gz 2023-09-30T22:28:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123106.log 2023-09-30T22:28:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193887.log 2023-09-30T22:28:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142621.log 2023-09-30T22:28:04.209 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0%/var/log/ceph/ceph-client.admin.193887.log: -- replaced with /var/log/ceph/ceph-client.admin.123106.log.gz 2023-09-30T22:28:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185787.log 2023-09-30T22:28:04.209 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193887.log.gz 2023-09-30T22:28:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142621.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142621.log.gz 2023-09-30T22:28:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185787.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.185787.log.gz 2023-09-30T22:28:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187417.log 2023-09-30T22:28:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155694.log 2023-09-30T22:28:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154482.log 2023-09-30T22:28:04.210 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.187417.log.gz 2023-09-30T22:28:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135665.log 2023-09-30T22:28:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154482.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.155694.log.gz 2023-09-30T22:28:04.210 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154482.log.gz 2023-09-30T22:28:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135665.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.135665.log.gz 2023-09-30T22:28:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141498.log 2023-09-30T22:28:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195181.log 2023-09-30T22:28:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169707.log 2023-09-30T22:28:04.211 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141498.log.gz 2023-09-30T22:28:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.195181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124725.log 2023-09-30T22:28:04.211 INFO:teuthology.orchestra.run.smithi089.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.195181.log.gz 2023-09-30T22:28:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169707.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.169707.log.gz 2023-09-30T22:28:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124725.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124725.log.gz 2023-09-30T22:28:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127059.log 2023-09-30T22:28:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190356.log 2023-09-30T22:28:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168405.log 2023-09-30T22:28:04.212 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127059.log.gz 2023-09-30T22:28:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177037.log 2023-09-30T22:28:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168405.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190356.log.gz 2023-09-30T22:28:04.212 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168405.log.gz 2023-09-30T22:28:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164898.log 2023-09-30T22:28:04.212 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177037.log.gz 2023-09-30T22:28:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168182.log 2023-09-30T22:28:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194705.log 2023-09-30T22:28:04.213 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164898.log.gz 2023-09-30T22:28:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164459.log 2023-09-30T22:28:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194705.log: gzip -5 88.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.168182.log.gz/var/log/ceph/ceph-client.admin.145842.log 2023-09-30T22:28:04.213 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:04.213 INFO:teuthology.orchestra.run.smithi089.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.194705.log.gz 2023-09-30T22:28:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164459.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164459.log.gz 2023-09-30T22:28:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145842.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145842.log.gz 2023-09-30T22:28:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179515.log 2023-09-30T22:28:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180887.log 2023-09-30T22:28:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161465.log 2023-09-30T22:28:04.214 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179515.log.gz 2023-09-30T22:28:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148343.log 2023-09-30T22:28:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161465.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180887.log.gz 2023-09-30T22:28:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141046.log 2023-09-30T22:28:04.215 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.161465.log.gz 2023-09-30T22:28:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135365.log 2023-09-30T22:28:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141046.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.148343.log.gz 2023-09-30T22:28:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135365.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141046.log.gz 2023-09-30T22:28:04.215 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.135365.log.gz 2023-09-30T22:28:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145524.log 2023-09-30T22:28:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180129.log 2023-09-30T22:28:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140053.log 2023-09-30T22:28:04.216 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145524.log.gz 2023-09-30T22:28:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177231.log 2023-09-30T22:28:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.111882.log 2023-09-30T22:28:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140053.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180129.log.gz 2023-09-30T22:28:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160754.log 2023-09-30T22:28:04.216 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.140053.log.gz 2023-09-30T22:28:04.216 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.177231.log.gz 2023-09-30T22:28:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.tmp-client.admin.111882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193443.log 2023-09-30T22:28:04.217 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.111882.log.gz 2023-09-30T22:28:04.217 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141694.log 2023-09-30T22:28:04.217 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160754.log: /var/log/ceph/ceph-client.admin.193443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184313.log 2023-09-30T22:28:04.217 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160754.log.gz 2023-09-30T22:28:04.217 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.193443.log.gz 2023-09-30T22:28:04.217 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141694.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141694.log.gz 2023-09-30T22:28:04.217 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184313.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.184313.log.gz 2023-09-30T22:28:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176448.log 2023-09-30T22:28:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184927.log 2023-09-30T22:28:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176448.log: /var/log/ceph/ceph-client.admin.184927.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176448.log.gz 2023-09-30T22:28:04.218 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.184927.log.gz 2023-09-30T22:28:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182920.log 2023-09-30T22:28:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142378.log 2023-09-30T22:28:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187221.log 2023-09-30T22:28:04.218 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.182920.log.gz 2023-09-30T22:28:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142378.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142378.log.gz 2023-09-30T22:28:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187221.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.187221.log.gz 2023-09-30T22:28:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137700.log 2023-09-30T22:28:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162152.log 2023-09-30T22:28:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172699.log 2023-09-30T22:28:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162152.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137700.log.gz 2023-09-30T22:28:04.219 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162152.log.gz 2023-09-30T22:28:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172699.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.172699.log.gz 2023-09-30T22:28:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137726.log 2023-09-30T22:28:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159889.log 2023-09-30T22:28:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143554.log 2023-09-30T22:28:04.220 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4%/var/log/ceph/ceph-client.admin.159889.log: -- replaced with /var/log/ceph/ceph-client.admin.137726.log.gz 2023-09-30T22:28:04.220 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159889.log.gz 2023-09-30T22:28:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178535.log 2023-09-30T22:28:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162940.log 2023-09-30T22:28:04.220 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.143554.log.gz 2023-09-30T22:28:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125580.log 2023-09-30T22:28:04.221 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178535.log.gz 2023-09-30T22:28:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186043.log 2023-09-30T22:28:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163351.log 2023-09-30T22:28:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125580.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162940.log.gz 2023-09-30T22:28:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166663.log 2023-09-30T22:28:04.221 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125580.log.gz 2023-09-30T22:28:04.221 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.186043.log.gz 2023-09-30T22:28:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163351.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163351.log.gz 2023-09-30T22:28:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166663.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.166663.log.gz 2023-09-30T22:28:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175613.log 2023-09-30T22:28:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152885.log 2023-09-30T22:28:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194279.log 2023-09-30T22:28:04.222 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3%/var/log/ceph/ceph-client.admin.152885.log: -- replaced with /var/log/ceph/ceph-client.admin.175613.log.gz 2023-09-30T22:28:04.222 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189413.log 2023-09-30T22:28:04.222 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152885.log.gz 2023-09-30T22:28:04.223 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194279.log: /var/log/ceph/ceph-client.admin.189413.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.194279.log.gz 2023-09-30T22:28:04.223 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.189413.log.gz 2023-09-30T22:28:04.223 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192168.log 2023-09-30T22:28:04.223 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169903.log 2023-09-30T22:28:04.223 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151895.log 2023-09-30T22:28:04.223 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.192168.log.gz 2023-09-30T22:28:04.223 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169903.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169903.log.gz 2023-09-30T22:28:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151895.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151895.log.gz 2023-09-30T22:28:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167543.log 2023-09-30T22:28:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151282.log 2023-09-30T22:28:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157145.log 2023-09-30T22:28:04.224 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.167543.log.gz 2023-09-30T22:28:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151282.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151282.log.gz 2023-09-30T22:28:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157145.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157145.log.gz 2023-09-30T22:28:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189716.log 2023-09-30T22:28:04.225 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161193.log 2023-09-30T22:28:04.225 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167230.log 2023-09-30T22:28:04.225 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.189716.log.gz/var/log/ceph/ceph-client.admin.161193.log: 2023-09-30T22:28:04.225 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182949.log 2023-09-30T22:28:04.225 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161193.log.gz 2023-09-30T22:28:04.225 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167230.log: /var/log/ceph/ceph-client.admin.182949.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167230.log.gz 2023-09-30T22:28:04.225 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180202.log 2023-09-30T22:28:04.225 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.182949.log.gz 2023-09-30T22:28:04.226 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168624.log 2023-09-30T22:28:04.226 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137969.log 2023-09-30T22:28:04.226 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.180202.log.gz 2023-09-30T22:28:04.226 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161298.log 2023-09-30T22:28:04.226 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143088.log 2023-09-30T22:28:04.226 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168624.log.gz 2023-09-30T22:28:04.226 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137969.log: /var/log/ceph/ceph-client.admin.161298.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137969.log.gz 2023-09-30T22:28:04.226 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.161298.log.gz 2023-09-30T22:28:04.227 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143088.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143088.log.gz 2023-09-30T22:28:04.227 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174241.log 2023-09-30T22:28:04.227 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190918.log 2023-09-30T22:28:04.227 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169119.log 2023-09-30T22:28:04.227 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.174241.log.gz 2023-09-30T22:28:04.227 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124578.log 2023-09-30T22:28:04.227 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145058.log 2023-09-30T22:28:04.227 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169119.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.190918.log.gz 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124578.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169119.log.gz 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi089.stderr: 87.0%/var/log/ceph/ceph-client.admin.145058.log: -- replaced with /var/log/ceph/ceph-client.admin.124578.log.gz 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145058.log.gz 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155471.log 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125727.log 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165463.log 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155471.log.gz 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181893.log 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149544.log 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125727.log.gz 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181893.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165463.log.gz 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181893.log.gz 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149544.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149544.log.gz 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179149.log 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190892.log 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135791.log 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190892.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.179149.log.gz 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135791.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190892.log.gz 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi089.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.135791.log.gz 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180965.log 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191309.log 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151283.log 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.180965.log.gz 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191309.log: /var/log/ceph/ceph-client.admin.151283.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.191309.log.gz 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151283.log.gz 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192094.log 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145132.log 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127655.log 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192094.log.gz 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135790.log 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184117.log 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145132.log.gz 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135790.log: gzip -5 --verbose 87.6%-- /var/log/ceph/ceph-client.admin.185873.log -- replaced with /var/log/ceph/ceph-client.admin.127655.log.gz 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184117.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.135790.log.gz 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185873.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184117.log.gz 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.185873.log.gz 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156949.log 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181771.log 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156949.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156949.log.gz 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181771.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.181771.log.gz 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148612.log 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166466.log 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193913.log 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166466.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148612.log.gz 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193913.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166466.log.gz 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.193913.log.gz 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193690.log 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138554.log 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173901.log 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.193690.log.gz 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138554.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138554.log.gz 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173901.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.173901.log.gz 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179614.log 2023-09-30T22:28:04.235 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153225.log 2023-09-30T22:28:04.235 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181161.log 2023-09-30T22:28:04.235 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179614.log.gz 2023-09-30T22:28:04.235 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153225.log: /var/log/ceph/ceph-client.admin.181161.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.153225.log.gz 2023-09-30T22:28:04.235 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.181161.log.gz 2023-09-30T22:28:04.235 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180352.log 2023-09-30T22:28:04.235 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157615.log 2023-09-30T22:28:04.235 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164995.log 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157615.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180352.log.gz 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164995.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157615.log.gz 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140518.log 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164995.log.gz 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139490.log 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144740.log 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140518.log.gz 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139490.log: /var/log/ceph/ceph-client.admin.144740.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139490.log.gz 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144740.log.gz 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177606.log 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152128.log 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172059.log 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177606.log.gz 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174803.log 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150551.log 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172059.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.152128.log.gz 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174803.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172059.log.gz 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174803.log.gz 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150551.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150551.log.gz 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161493.log 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156361.log 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147435.log 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156361.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161493.log.gz 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147435.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156361.log.gz 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147435.log.gz 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164527.log 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193494.log 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148931.log 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164527.log.gz 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191532.log 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.193494.log.gz 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148931.log: /var/log/ceph/ceph-client.admin.191532.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148931.log.gz 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191532.log.gz 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177751.log 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137674.log 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138949.log 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177751.log.gz 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152519.log 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136694.log 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138949.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137674.log.gz 2023-09-30T22:28:04.241 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166269.log 2023-09-30T22:28:04.241 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138949.log.gz 2023-09-30T22:28:04.241 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152519.log.gz 2023-09-30T22:28:04.241 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156481.log 2023-09-30T22:28:04.241 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166269.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136694.log.gz 2023-09-30T22:28:04.241 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156481.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166269.log.gz 2023-09-30T22:28:04.241 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156481.log.gz 2023-09-30T22:28:04.242 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136183.log 2023-09-30T22:28:04.242 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183658.log 2023-09-30T22:28:04.242 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162716.log 2023-09-30T22:28:04.242 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136183.log.gz 2023-09-30T22:28:04.242 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145404.log 2023-09-30T22:28:04.242 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185872.log 2023-09-30T22:28:04.242 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.183658.log.gz 2023-09-30T22:28:04.242 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162716.log: /var/log/ceph/ceph-client.admin.145404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190208.log 2023-09-30T22:28:04.243 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162716.log.gz 2023-09-30T22:28:04.243 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185872.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.145404.log.gz 2023-09-30T22:28:04.243 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.185872.log.gz 2023-09-30T22:28:04.243 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190208.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.190208.log.gz 2023-09-30T22:28:04.243 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123992.log 2023-09-30T22:28:04.243 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166270.log 2023-09-30T22:28:04.243 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123992.log: 87.3%/var/log/ceph/ceph-client.admin.166270.log: -- replaced with /var/log/ceph/ceph-client.admin.123992.log.gz 2023-09-30T22:28:04.243 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166270.log.gz 2023-09-30T22:28:04.244 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183359.log 2023-09-30T22:28:04.244 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144789.log 2023-09-30T22:28:04.244 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141790.log 2023-09-30T22:28:04.244 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.183359.log.gz 2023-09-30T22:28:04.244 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157419.log 2023-09-30T22:28:04.244 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146407.log 2023-09-30T22:28:04.244 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144789.log.gz 2023-09-30T22:28:04.244 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141790.log: /var/log/ceph/ceph-client.admin.157419.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141790.log.gz 2023-09-30T22:28:04.245 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157419.log.gz 2023-09-30T22:28:04.245 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146407.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146407.log.gz 2023-09-30T22:28:04.245 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189815.log 2023-09-30T22:28:04.245 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173092.log 2023-09-30T22:28:04.245 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143603.log 2023-09-30T22:28:04.245 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.189815.log.gz 2023-09-30T22:28:04.245 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143284.log 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173092.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173092.log.gz 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143603.log: /var/log/ceph/ceph-client.admin.143284.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143603.log.gz 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143284.log.gz 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153617.log 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190161.log 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154793.log 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190161.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153617.log.gz 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171570.log 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190161.log.gz 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171497.log 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154793.log.gz 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157265.log 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171570.log: /var/log/ceph/ceph-client.admin.171497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156557.log 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.171570.log.gz 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171497.log.gz 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124333.log 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157265.log.gz 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194702.log 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147632.log 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156557.log.gz 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124333.log.gz 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194702.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.194702.log.gz 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147632.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147632.log.gz 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122313.log 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151061.log 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165270.log 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122313.log.gz/var/log/ceph/ceph-client.admin.151061.log: 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165270.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151061.log.gz 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161687.log 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185123.log 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165270.log.gz 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127184.log 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161687.log.gz 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147045.log 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2%/var/log/ceph/ceph-client.admin.127184.log: -- replaced with /var/log/ceph/ceph-client.admin.185123.log.gz 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140121.log 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147045.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127184.log.gz 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140121.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147045.log.gz 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140121.log.gz 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153081.log 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158517.log 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171173.log 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153081.log.gz 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146848.log 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163505.log 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171173.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158517.log.gz 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146848.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171173.log.gz 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146848.log.gz 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163505.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163505.log.gz 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167935.log 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176959.log 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185591.log 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167935.log.gz 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136968.log 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5%gzip /var/log/ceph/ceph-client.admin.185591.log: -5 -- replaced with /var/log/ceph/ceph-client.admin.176959.log.gz--verbose 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:-- /var/log/ceph/ceph-client.admin.165658.log 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136968.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.185591.log.gz 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165658.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136968.log.gz 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165658.log.gz 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188958.log 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189204.log 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161759.log 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.188958.log.gz 2023-09-30T22:28:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184611.log 2023-09-30T22:28:04.254 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.189204.log.gz 2023-09-30T22:28:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161759.log: /var/log/ceph/ceph-client.admin.184611.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161759.log.gz 2023-09-30T22:28:04.254 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.184611.log.gz 2023-09-30T22:28:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155185.log 2023-09-30T22:28:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171471.log 2023-09-30T22:28:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143946.log 2023-09-30T22:28:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171471.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155185.log.gz 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171471.log.gz/var/log/ceph/ceph-client.admin.143946.log: 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171275.log 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.143946.log.gz 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166616.log 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191164.log 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.171275.log.gz 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167987.log 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194056.log 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166616.log.gz 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167987.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191164.log.gz 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167987.log.gz 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194056.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.194056.log.gz 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155603.log 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153933.log 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135616.log 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155603.log.gz 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153933.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153933.log.gz 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135616.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.135616.log.gz-5 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:--verbose -- /var/log/ceph/ceph-client.admin.138752.log 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188148.log 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146161.log 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188148.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138752.log.gz 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.188148.log.gz 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146161.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146161.log.gz 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186802.log 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162520.log 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186802.log: /var/log/ceph/ceph-client.admin.162520.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.186802.log.gz 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162520.log.gz 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150081.log 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137112.log 2023-09-30T22:28:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157239.log 2023-09-30T22:28:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137112.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150081.log.gz 2023-09-30T22:28:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157239.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137112.log.gz 2023-09-30T22:28:04.259 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157239.log.gz 2023-09-30T22:28:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135413.log 2023-09-30T22:28:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183435.log 2023-09-30T22:28:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135413.log: /var/log/ceph/ceph-client.admin.183435.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135413.log.gz 2023-09-30T22:28:04.259 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183435.log.gz 2023-09-30T22:28:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162690.log 2023-09-30T22:28:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138753.log 2023-09-30T22:28:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162690.log: /var/log/ceph/ceph-client.admin.138753.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162690.log.gz 2023-09-30T22:28:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172503.log 2023-09-30T22:28:04.260 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138753.log.gz 2023-09-30T22:28:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124921.log 2023-09-30T22:28:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181671.log 2023-09-30T22:28:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124921.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.172503.log.gz 2023-09-30T22:28:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181671.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124921.log.gz 2023-09-30T22:28:04.261 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.181671.log.gz 2023-09-30T22:28:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142770.log 2023-09-30T22:28:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151335.log 2023-09-30T22:28:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142770.log: /var/log/ceph/ceph-client.admin.151335.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142770.log.gz 2023-09-30T22:28:04.261 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151335.log.gz 2023-09-30T22:28:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158007.log 2023-09-30T22:28:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189912.log 2023-09-30T22:28:04.262 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148416.log 2023-09-30T22:28:04.262 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189912.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158007.log.gz 2023-09-30T22:28:04.262 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148416.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189912.log.gz 2023-09-30T22:28:04.262 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148416.log.gz 2023-09-30T22:28:04.262 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142059.log 2023-09-30T22:28:04.262 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135792.log 2023-09-30T22:28:04.262 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183947.log 2023-09-30T22:28:04.262 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135792.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142059.log.gz 2023-09-30T22:28:04.263 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183947.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135792.log.gz 2023-09-30T22:28:04.263 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183947.log.gz 2023-09-30T22:28:04.263 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153251.log 2023-09-30T22:28:04.263 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164655.log 2023-09-30T22:28:04.263 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157341.log 2023-09-30T22:28:04.263 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164655.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153251.log.gz 2023-09-30T22:28:04.263 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157341.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.164655.log.gz 2023-09-30T22:28:04.263 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157341.log.gz 2023-09-30T22:28:04.264 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195034.log 2023-09-30T22:28:04.264 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149179.log 2023-09-30T22:28:04.264 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.195034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122436.log 2023-09-30T22:28:04.264 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149179.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.195034.log.gz 2023-09-30T22:28:04.264 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4%/var/log/ceph/ceph-client.admin.122436.log: -- replaced with /var/log/ceph/ceph-client.admin.149179.log.gz 2023-09-30T22:28:04.264 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122436.log.gz 2023-09-30T22:28:04.264 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186435.log 2023-09-30T22:28:04.264 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187268.log 2023-09-30T22:28:04.265 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128492.log 2023-09-30T22:28:04.265 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187268.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.186435.log.gz 2023-09-30T22:28:04.265 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.187268.log.gz/var/log/ceph/ceph-client.admin.128492.log: 2023-09-30T22:28:04.265 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128492.log.gz 2023-09-30T22:28:04.265 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156434.log 2023-09-30T22:28:04.265 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138159.log 2023-09-30T22:28:04.265 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143480.log 2023-09-30T22:28:04.266 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138159.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156434.log.gz 2023-09-30T22:28:04.266 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138159.log.gz 2023-09-30T22:28:04.266 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143480.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143480.log.gz 2023-09-30T22:28:04.266 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145720.log 2023-09-30T22:28:04.266 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148709.log 2023-09-30T22:28:04.266 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176593.log 2023-09-30T22:28:04.266 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148709.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145720.log.gz 2023-09-30T22:28:04.266 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148709.log.gz 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176593.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.176593.log.gz 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186631.log 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180299.log 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167054.log 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180299.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.186631.log.gz 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167054.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180299.log.gz 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167054.log.gz 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155890.log 2023-09-30T22:28:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135192.log 2023-09-30T22:28:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150669.log 2023-09-30T22:28:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135192.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.155890.log.gz 2023-09-30T22:28:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150669.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135192.log.gz 2023-09-30T22:28:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183631.log 2023-09-30T22:28:04.268 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150669.log.gz 2023-09-30T22:28:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168549.log 2023-09-30T22:28:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190600.log 2023-09-30T22:28:04.269 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.183631.log.gz 2023-09-30T22:28:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168549.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168549.log.gz 2023-09-30T22:28:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190600.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.190600.log.gz 2023-09-30T22:28:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127369.log 2023-09-30T22:28:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175221.log 2023-09-30T22:28:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161167.log 2023-09-30T22:28:04.269 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127369.log.gz 2023-09-30T22:28:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175221.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175221.log.gz/var/log/ceph/ceph-client.admin.161167.log: 2023-09-30T22:28:04.270 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161167.log.gz 2023-09-30T22:28:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178339.log 2023-09-30T22:28:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181552.log 2023-09-30T22:28:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178339.log: /var/log/ceph/ceph-client.admin.181552.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.178339.log.gz 2023-09-30T22:28:04.270 INFO:teuthology.orchestra.run.smithi089.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.181552.log.gz 2023-09-30T22:28:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154597.log 2023-09-30T22:28:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172255.log 2023-09-30T22:28:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191776.log 2023-09-30T22:28:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172255.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154597.log.gz 2023-09-30T22:28:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191776.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172255.log.gz 2023-09-30T22:28:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177411.log 2023-09-30T22:28:04.271 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191776.log.gz 2023-09-30T22:28:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154870.log 2023-09-30T22:28:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177411.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.177411.log.gz 2023-09-30T22:28:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154870.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154870.log.gz 2023-09-30T22:28:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154675.log 2023-09-30T22:28:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149689.log 2023-09-30T22:28:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:gzip /var/log/ceph/ceph-client.admin.154675.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.192737.log 2023-09-30T22:28:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149689.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154675.log.gz 2023-09-30T22:28:04.272 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149689.log.gz 2023-09-30T22:28:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192737.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.192737.log.gz 2023-09-30T22:28:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161073.log 2023-09-30T22:28:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177038.log 2023-09-30T22:28:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161073.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161073.log.gz 2023-09-30T22:28:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162350.log 2023-09-30T22:28:04.273 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177038.log.gz 2023-09-30T22:28:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127768.log 2023-09-30T22:28:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167426.log 2023-09-30T22:28:04.273 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162350.log.gz/var/log/ceph/ceph-client.admin.127768.log: 2023-09-30T22:28:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167426.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127768.log.gz 2023-09-30T22:28:04.273 INFO:teuthology.orchestra.run.smithi089.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-09-30T22:28:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167347.log 2023-09-30T22:28:04.274 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.167426.log.gz 2023-09-30T22:28:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139242.log 2023-09-30T22:28:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184143.log 2023-09-30T22:28:04.274 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167347.log.gz 2023-09-30T22:28:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124411.log 2023-09-30T22:28:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160307.log 2023-09-30T22:28:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184143.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139242.log.gz 2023-09-30T22:28:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124411.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.184143.log.gz 2023-09-30T22:28:04.275 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124411.log.gz 2023-09-30T22:28:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160307.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160307.log.gz 2023-09-30T22:28:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122750.log 2023-09-30T22:28:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124359.log 2023-09-30T22:28:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175809.log 2023-09-30T22:28:04.275 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122750.log.gz 2023-09-30T22:28:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136575.log 2023-09-30T22:28:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192855.log 2023-09-30T22:28:04.276 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124359.log.gz 2023-09-30T22:28:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136575.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175809.log.gz 2023-09-30T22:28:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123057.log 2023-09-30T22:28:04.276 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136575.log.gz 2023-09-30T22:28:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192855.log: /var/log/ceph/ceph-client.admin.123057.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192855.log.gz 2023-09-30T22:28:04.276 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123057.log.gz 2023-09-30T22:28:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174489.log 2023-09-30T22:28:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167817.log 2023-09-30T22:28:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165805.log 2023-09-30T22:28:04.277 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.174489.log.gz 2023-09-30T22:28:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167817.log: /var/log/ceph/ceph-client.admin.165805.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.167817.log.gz 2023-09-30T22:28:04.277 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165805.log.gz 2023-09-30T22:28:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161937.log 2023-09-30T22:28:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182358.log 2023-09-30T22:28:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173457.log 2023-09-30T22:28:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182358.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161937.log.gz 2023-09-30T22:28:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173457.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.182358.log.gz 2023-09-30T22:28:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176567.log 2023-09-30T22:28:04.278 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173457.log.gz 2023-09-30T22:28:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143995.log 2023-09-30T22:28:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123353.log 2023-09-30T22:28:04.278 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176567.log.gz 2023-09-30T22:28:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143995.log: /var/log/ceph/ceph-client.admin.123353.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143995.log.gz 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi089.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.123353.log.gz 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150941.log 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183827.log 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186044.log 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150941.log.gz 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182088.log 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186044.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.183827.log.gz 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182088.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.186044.log.gz 2023-09-30T22:28:04.280 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.182088.log.gz 2023-09-30T22:28:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148984.log 2023-09-30T22:28:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176841.log 2023-09-30T22:28:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180770.log 2023-09-30T22:28:04.280 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148984.log.gz 2023-09-30T22:28:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190500.log 2023-09-30T22:28:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181083.log 2023-09-30T22:28:04.280 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176841.log.gz 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180770.log: /var/log/ceph/ceph-client.admin.190500.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.180770.log.gz 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.190500.log.gz/var/log/ceph/ceph-client.admin.181083.log: 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.181083.log.gz 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195279.log 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140883.log 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.195279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139660.log 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.195279.log.gz 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152441.log 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152100.log 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140883.log.gz 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139660.log: /var/log/ceph/ceph-client.admin.152441.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139660.log.gz 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152441.log.gz 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169341.log 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152100.log.gz 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164852.log 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183116.log 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.169341.log.gz 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142846.log 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145646.log 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.164852.log.gz 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142846.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.183116.log.gz 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170145.log 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142846.log.gz 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118503.log 2023-09-30T22:28:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170145.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145646.log.gz 2023-09-30T22:28:04.284 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.170145.log.gz 2023-09-30T22:28:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.118503.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118503.log.gz 2023-09-30T22:28:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165858.log 2023-09-30T22:28:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182555.log 2023-09-30T22:28:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153473.log 2023-09-30T22:28:04.284 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165858.log.gz 2023-09-30T22:28:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182555.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.182555.log.gz 2023-09-30T22:28:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153473.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153473.log.gz 2023-09-30T22:28:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178417.log 2023-09-30T22:28:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142281.log 2023-09-30T22:28:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189574.log 2023-09-30T22:28:04.285 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178417.log.gz 2023-09-30T22:28:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142281.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142281.log.gz 2023-09-30T22:28:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189574.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189574.log.gz 2023-09-30T22:28:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138118.log 2023-09-30T22:28:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136574.log 2023-09-30T22:28:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170569.log 2023-09-30T22:28:04.286 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138118.log.gz 2023-09-30T22:28:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136574.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136574.log.gz 2023-09-30T22:28:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170569.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.170569.log.gz 2023-09-30T22:28:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163453.log 2023-09-30T22:28:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186436.log 2023-09-30T22:28:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166785.log 2023-09-30T22:28:04.287 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163453.log.gz 2023-09-30T22:28:04.287 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186436.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.186436.log.gz 2023-09-30T22:28:04.287 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166785.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166785.log.gz 2023-09-30T22:28:04.287 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176226.log 2023-09-30T22:28:04.287 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174293.log 2023-09-30T22:28:04.287 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176253.log 2023-09-30T22:28:04.287 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176226.log.gz/var/log/ceph/ceph-client.admin.174293.log: 2023-09-30T22:28:04.287 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.174293.log.gz 2023-09-30T22:28:04.288 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176253.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.176253.log.gz 2023-09-30T22:28:04.288 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191872.log 2023-09-30T22:28:04.288 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147214.log 2023-09-30T22:28:04.288 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164093.log 2023-09-30T22:28:04.288 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191872.log.gz 2023-09-30T22:28:04.288 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147214.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147214.log.gz 2023-09-30T22:28:04.288 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164093.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164093.log.gz 2023-09-30T22:28:04.288 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181697.log 2023-09-30T22:28:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149545.log 2023-09-30T22:28:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179175.log 2023-09-30T22:28:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149545.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.181697.log.gz 2023-09-30T22:28:04.289 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.149545.log.gz 2023-09-30T22:28:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179175.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.179175.log.gz 2023-09-30T22:28:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191166.log 2023-09-30T22:28:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137165.log 2023-09-30T22:28:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161857.log 2023-09-30T22:28:04.290 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191166.log.gz 2023-09-30T22:28:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137165.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137165.log.gz 2023-09-30T22:28:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161857.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161857.log.gz 2023-09-30T22:28:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174490.log 2023-09-30T22:28:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138361.log 2023-09-30T22:28:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148000.log 2023-09-30T22:28:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138361.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174490.log.gz 2023-09-30T22:28:04.291 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138361.log.gz 2023-09-30T22:28:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148000.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148000.log.gz 2023-09-30T22:28:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182507.log 2023-09-30T22:28:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124189.log 2023-09-30T22:28:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144593.log 2023-09-30T22:28:04.291 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182507.log.gz 2023-09-30T22:28:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124189.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124189.log.gz 2023-09-30T22:28:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144593.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.144593.log.gz 2023-09-30T22:28:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183856.log 2023-09-30T22:28:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166171.log 2023-09-30T22:28:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169958.log 2023-09-30T22:28:04.292 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.183856.log.gz 2023-09-30T22:28:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166171.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166171.log.gz 2023-09-30T22:28:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163231.log 2023-09-30T22:28:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-09-30T22:28:04.292 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169958.log.gz 2023-09-30T22:28:04.293 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145769.log 2023-09-30T22:28:04.293 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163231.log.gz 2023-09-30T22:28:04.293 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183312.log 2023-09-30T22:28:04.293 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187612.log 2023-09-30T22:28:04.293 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183312.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145769.log.gz 2023-09-30T22:28:04.293 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.183312.log.gz 2023-09-30T22:28:04.293 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187612.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.187612.log.gz 2023-09-30T22:28:04.293 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168378.log 2023-09-30T22:28:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172111.log 2023-09-30T22:28:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162743.log 2023-09-30T22:28:04.294 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168378.log.gz 2023-09-30T22:28:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172111.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.172111.log.gz 2023-09-30T22:28:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188025.log 2023-09-30T22:28:04.294 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162743.log.gz 2023-09-30T22:28:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186188.log 2023-09-30T22:28:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194838.log 2023-09-30T22:28:04.295 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188025.log.gz 2023-09-30T22:28:04.295 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186188.log: /var/log/ceph/ceph-client.admin.194838.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.186188.log.gz 2023-09-30T22:28:04.295 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.194838.log.gz 2023-09-30T22:28:04.295 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125264.log 2023-09-30T22:28:04.295 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178783.log 2023-09-30T22:28:04.295 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193131.log 2023-09-30T22:28:04.295 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178783.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125264.log.gz 2023-09-30T22:28:04.295 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178783.log.gz 2023-09-30T22:28:04.296 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193131.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.193131.log.gz 2023-09-30T22:28:04.296 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158739.log 2023-09-30T22:28:04.296 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141302.log 2023-09-30T22:28:04.296 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193718.log 2023-09-30T22:28:04.296 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141302.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158739.log.gz 2023-09-30T22:28:04.296 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193718.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141302.log.gz 2023-09-30T22:28:04.296 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.193718.log.gz 2023-09-30T22:28:04.296 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156264.log 2023-09-30T22:28:04.297 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164067.log 2023-09-30T22:28:04.297 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118766.log 2023-09-30T22:28:04.297 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164067.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156264.log.gz 2023-09-30T22:28:04.297 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.118766.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164067.log.gz 2023-09-30T22:28:04.297 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118766.log.gz 2023-09-30T22:28:04.297 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185669.log 2023-09-30T22:28:04.297 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-09-30T22:28:04.297 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185669.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-09-30T22:28:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mds.a.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.185669.log.gz 2023-09-30T22:28:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159523.log 2023-09-30T22:28:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159523.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159523.log.gz 2023-09-30T22:28:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152321.log 2023-09-30T22:28:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152321.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152321.log.gz 2023-09-30T22:28:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141863.log 2023-09-30T22:28:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141863.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141863.log.gz 2023-09-30T22:28:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143211.log 2023-09-30T22:28:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143211.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143211.log.gz 2023-09-30T22:28:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172647.log 2023-09-30T22:28:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172647.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172647.log.gz 2023-09-30T22:28:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189791.log 2023-09-30T22:28:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189791.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.189791.log.gz 2023-09-30T22:28:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163897.log 2023-09-30T22:28:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163897.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163897.log.gz 2023-09-30T22:28:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167151.log 2023-09-30T22:28:04.300 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167151.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.167151.log.gz 2023-09-30T22:28:04.300 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144544.log 2023-09-30T22:28:04.300 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144544.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144544.log.gz 2023-09-30T22:28:04.300 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165857.log 2023-09-30T22:28:04.300 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165857.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165857.log.gz 2023-09-30T22:28:04.300 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123549.log 2023-09-30T22:28:04.300 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123549.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.123549.log.gz 2023-09-30T22:28:04.300 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173705.log 2023-09-30T22:28:04.301 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173705.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.173705.log.gz 2023-09-30T22:28:04.301 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185225.log 2023-09-30T22:28:04.301 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185225.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.185225.log.gz 2023-09-30T22:28:04.301 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186776.log 2023-09-30T22:28:04.301 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186776.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.186776.log.gz 2023-09-30T22:28:04.301 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154231.log 2023-09-30T22:28:04.301 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154231.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.154231.log.gz 2023-09-30T22:28:04.301 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188640.log 2023-09-30T22:28:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188640.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.188640.log.gz 2023-09-30T22:28:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194703.log 2023-09-30T22:28:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194703.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.194703.log.gz 2023-09-30T22:28:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127366.log 2023-09-30T22:28:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127366.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127366.log.gz 2023-09-30T22:28:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174437.log 2023-09-30T22:28:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174437.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174437.log.gz 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182481.log 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182481.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.182481.log.gz 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171079.log 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171079.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.171079.log.gz 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139320.log 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139320.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.139320.log.gz 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178782.log 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178782.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178782.log.gz 2023-09-30T22:28:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191506.log 2023-09-30T22:28:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191506.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.191506.log.gz 2023-09-30T22:28:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137578.log 2023-09-30T22:28:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137578.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137578.log.gz 2023-09-30T22:28:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165387.log 2023-09-30T22:28:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165387.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165387.log.gz 2023-09-30T22:28:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195426.log 2023-09-30T22:28:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.195426.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.195426.log.gz 2023-09-30T22:28:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187586.log 2023-09-30T22:28:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187586.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.187586.log.gz 2023-09-30T22:28:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136378.log 2023-09-30T22:28:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136378.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136378.log.gz 2023-09-30T22:28:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187782.log 2023-09-30T22:28:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187782.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.187782.log.gz 2023-09-30T22:28:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141445.log 2023-09-30T22:28:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141445.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141445.log.gz 2023-09-30T22:28:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150525.log 2023-09-30T22:28:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150525.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150525.log.gz 2023-09-30T22:28:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-09-30T22:28:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158766.log 2023-09-30T22:28:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158766.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158766.log.gz 2023-09-30T22:28:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193521.log 2023-09-30T22:28:04.306 INFO:teuthology.orchestra.run.smithi089.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-09-30T22:28:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193521.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.193521.log.gz 2023-09-30T22:28:04.431 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144397.log 2023-09-30T22:28:04.431 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144397.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144397.log.gz 2023-09-30T22:28:04.431 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193103.log 2023-09-30T22:28:04.431 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193103.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.193103.log.gz 2023-09-30T22:28:04.431 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143015.log 2023-09-30T22:28:04.431 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143015.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.143015.log.gz 2023-09-30T22:28:04.432 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160009.log 2023-09-30T22:28:04.432 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160009.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160009.log.gz 2023-09-30T22:28:04.432 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130032.log 2023-09-30T22:28:04.432 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.130032.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130032.log.gz 2023-09-30T22:28:04.432 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159052.log 2023-09-30T22:28:04.432 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159052.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159052.log.gz 2023-09-30T22:28:04.432 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158568.log 2023-09-30T22:28:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158568.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158568.log.gz 2023-09-30T22:28:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180691.log 2023-09-30T22:28:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180691.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180691.log.gz 2023-09-30T22:28:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128959.log 2023-09-30T22:28:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128959.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128959.log.gz 2023-09-30T22:28:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143358.log 2023-09-30T22:28:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143358.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.143358.log.gz 2023-09-30T22:28:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127534.log 2023-09-30T22:28:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127534.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127534.log.gz 2023-09-30T22:28:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148809.log 2023-09-30T22:28:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148809.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148809.log.gz 2023-09-30T22:28:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158935.log 2023-09-30T22:28:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158935.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.158935.log.gz 2023-09-30T22:28:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184415.log 2023-09-30T22:28:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184415.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.184415.log.gz 2023-09-30T22:28:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145916.log 2023-09-30T22:28:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145916.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145916.log.gz 2023-09-30T22:28:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152101.log 2023-09-30T22:28:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152101.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152101.log.gz 2023-09-30T22:28:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179418.log 2023-09-30T22:28:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179418.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179418.log.gz 2023-09-30T22:28:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180548.log 2023-09-30T22:28:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180548.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180548.log.gz 2023-09-30T22:28:04.436 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177155.log 2023-09-30T22:28:04.436 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177155.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177155.log.gz 2023-09-30T22:28:04.436 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179123.log 2023-09-30T22:28:04.436 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179123.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179123.log.gz 2023-09-30T22:28:04.436 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182143.log 2023-09-30T22:28:04.436 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182143.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182143.log.gz 2023-09-30T22:28:04.436 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136302.log 2023-09-30T22:28:04.436 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.136302.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136302.log.gz 2023-09-30T22:28:04.437 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171889.log 2023-09-30T22:28:04.437 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171889.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171889.log.gz 2023-09-30T22:28:04.437 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146822.log 2023-09-30T22:28:04.437 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146822.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146822.log.gz 2023-09-30T22:28:04.437 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156873.log 2023-09-30T22:28:04.437 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156873.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156873.log.gz 2023-09-30T22:28:04.437 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194109.log 2023-09-30T22:28:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194109.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.194109.log.gz 2023-09-30T22:28:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143846.log 2023-09-30T22:28:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143846.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143846.log.gz 2023-09-30T22:28:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157563.log 2023-09-30T22:28:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157563.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157563.log.gz 2023-09-30T22:28:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142086.log 2023-09-30T22:28:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142086.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142086.log.gz 2023-09-30T22:28:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143162.log 2023-09-30T22:28:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143162.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143162.log.gz 2023-09-30T22:28:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146038.log 2023-09-30T22:28:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146038.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146038.log.gz 2023-09-30T22:28:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145450.log 2023-09-30T22:28:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145450.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.145450.log.gz 2023-09-30T22:28:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147533.log 2023-09-30T22:28:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147533.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147533.log.gz 2023-09-30T22:28:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189619.log 2023-09-30T22:28:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189619.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.189619.log.gz 2023-09-30T22:28:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181919.log 2023-09-30T22:28:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.181919.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.181919.log.gz 2023-09-30T22:28:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123991.log 2023-09-30T22:28:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123991.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123991.log.gz 2023-09-30T22:28:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193495.log 2023-09-30T22:28:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187857.log 2023-09-30T22:28:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193495.log: /var/log/ceph/ceph-client.admin.187857.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193495.log.gz 2023-09-30T22:28:04.441 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.187857.log.gz 2023-09-30T22:28:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123500.log 2023-09-30T22:28:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123500.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123500.log.gz 2023-09-30T22:28:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157416.log 2023-09-30T22:28:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157416.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157416.log.gz 2023-09-30T22:28:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139124.log 2023-09-30T22:28:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139124.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139124.log.gz 2023-09-30T22:28:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187660.log 2023-09-30T22:28:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187660.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.187660.log.gz 2023-09-30T22:28:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145377.log 2023-09-30T22:28:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145377.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145377.log.gz 2023-09-30T22:28:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185421.log 2023-09-30T22:28:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185421.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.185421.log.gz 2023-09-30T22:28:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164236.log 2023-09-30T22:28:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164236.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.164236.log.gz 2023-09-30T22:28:04.443 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185199.log 2023-09-30T22:28:04.443 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185199.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.185199.log.gz 2023-09-30T22:28:04.443 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174215.log 2023-09-30T22:28:04.443 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181305.log 2023-09-30T22:28:04.443 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174215.log: /var/log/ceph/ceph-client.admin.181305.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174215.log.gz 2023-09-30T22:28:04.443 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.181305.log.gz 2023-09-30T22:28:04.443 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141910.log 2023-09-30T22:28:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141910.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141910.log.gz 2023-09-30T22:28:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118550.log 2023-09-30T22:28:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.118550.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118550.log.gz 2023-09-30T22:28:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175051.log 2023-09-30T22:28:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175051.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175051.log.gz 2023-09-30T22:28:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175979.log 2023-09-30T22:28:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175979.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.175979.log.gz 2023-09-30T22:28:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125433.log 2023-09-30T22:28:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125433.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125433.log.gz 2023-09-30T22:28:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-30T22:28:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.log: 94.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-09-30T22:28:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179996.log 2023-09-30T22:28:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179996.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.179996.log.gz 2023-09-30T22:28:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135567.log 2023-09-30T22:28:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135567.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135567.log.gz 2023-09-30T22:28:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129302.log 2023-09-30T22:28:04.446 INFO:teuthology.orchestra.run.smithi089.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-09-30T22:28:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129302.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129302.log.gz 2023-09-30T22:28:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123402.log 2023-09-30T22:28:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152046.log 2023-09-30T22:28:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123402.log: /var/log/ceph/ceph-client.admin.152046.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123402.log.gz 2023-09-30T22:28:04.446 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152046.log.gz 2023-09-30T22:28:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150328.log 2023-09-30T22:28:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122244.log 2023-09-30T22:28:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150328.log: /var/log/ceph/ceph-client.admin.122244.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150328.log.gz 2023-09-30T22:28:04.447 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122244.log.gz 2023-09-30T22:28:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155284.log 2023-09-30T22:28:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176371.log 2023-09-30T22:28:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155284.log: /var/log/ceph/ceph-client.admin.176371.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155284.log.gz 2023-09-30T22:28:04.447 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176371.log.gz 2023-09-30T22:28:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144985.log 2023-09-30T22:28:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151137.log 2023-09-30T22:28:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144985.log: /var/log/ceph/ceph-client.admin.151137.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144985.log.gz 2023-09-30T22:28:04.448 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151137.log.gz 2023-09-30T22:28:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137362.log 2023-09-30T22:28:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175050.log 2023-09-30T22:28:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137362.log: /var/log/ceph/ceph-client.admin.175050.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137362.log.gz 2023-09-30T22:28:04.448 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175050.log.gz 2023-09-30T22:28:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170542.log 2023-09-30T22:28:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181867.log 2023-09-30T22:28:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170542.log: /var/log/ceph/ceph-client.admin.181867.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170542.log.gz 2023-09-30T22:28:04.449 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.181867.log.gz 2023-09-30T22:28:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176789.log 2023-09-30T22:28:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142454.log 2023-09-30T22:28:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176789.log: /var/log/ceph/ceph-client.admin.142454.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.176789.log.gz 2023-09-30T22:28:04.449 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142454.log.gz 2023-09-30T22:28:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145328.log 2023-09-30T22:28:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191925.log 2023-09-30T22:28:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145328.log: /var/log/ceph/ceph-client.admin.191925.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145328.log.gz 2023-09-30T22:28:04.450 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191925.log.gz 2023-09-30T22:28:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127767.log 2023-09-30T22:28:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159550.log 2023-09-30T22:28:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127767.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127767.log.gz 2023-09-30T22:28:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159550.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159550.log.gz 2023-09-30T22:28:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194936.log 2023-09-30T22:28:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139438.log 2023-09-30T22:28:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194936.log: /var/log/ceph/ceph-client.admin.139438.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.194936.log.gz 2023-09-30T22:28:04.451 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139438.log.gz 2023-09-30T22:28:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135987.log 2023-09-30T22:28:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153029.log 2023-09-30T22:28:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135987.log: /var/log/ceph/ceph-client.admin.153029.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135987.log.gz 2023-09-30T22:28:04.452 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153029.log.gz 2023-09-30T22:28:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173140.log 2023-09-30T22:28:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177410.log 2023-09-30T22:28:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173140.log: /var/log/ceph/ceph-client.admin.177410.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.173140.log.gz 2023-09-30T22:28:04.452 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177410.log.gz 2023-09-30T22:28:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159327.log 2023-09-30T22:28:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158468.log 2023-09-30T22:28:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159327.log: /var/log/ceph/ceph-client.admin.158468.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159327.log.gz 2023-09-30T22:28:04.453 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158468.log.gz 2023-09-30T22:28:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185319.log 2023-09-30T22:28:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144151.log 2023-09-30T22:28:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185319.log: /var/log/ceph/ceph-client.admin.144151.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.185319.log.gz 2023-09-30T22:28:04.453 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.144151.log.gz 2023-09-30T22:28:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125972.log 2023-09-30T22:28:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139830.log 2023-09-30T22:28:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125972.log: /var/log/ceph/ceph-client.admin.139830.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125972.log.gz 2023-09-30T22:28:04.454 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139830.log.gz 2023-09-30T22:28:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142817.log 2023-09-30T22:28:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169219.log 2023-09-30T22:28:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142817.log: /var/log/ceph/ceph-client.admin.169219.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142817.log.gz 2023-09-30T22:28:04.454 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169219.log.gz 2023-09-30T22:28:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159497.log 2023-09-30T22:28:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136916.log 2023-09-30T22:28:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159497.log: /var/log/ceph/ceph-client.admin.136916.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.159497.log.gz 2023-09-30T22:28:04.455 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136916.log.gz 2023-09-30T22:28:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166811.log 2023-09-30T22:28:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159442.log 2023-09-30T22:28:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166811.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.166811.log.gz 2023-09-30T22:28:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159442.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159442.log.gz 2023-09-30T22:28:04.455 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155577.log 2023-09-30T22:28:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187168.log 2023-09-30T22:28:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155577.log: /var/log/ceph/ceph-client.admin.187168.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155577.log.gz 2023-09-30T22:28:04.456 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.187168.log.gz 2023-09-30T22:28:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143799.log 2023-09-30T22:28:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166759.log 2023-09-30T22:28:04.456 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143799.log: /var/log/ceph/ceph-client.admin.166759.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143799.log.gz 2023-09-30T22:28:04.456 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166759.log.gz 2023-09-30T22:28:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147337.log 2023-09-30T22:28:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173484.log 2023-09-30T22:28:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147337.log: /var/log/ceph/ceph-client.admin.173484.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147337.log.gz 2023-09-30T22:28:04.457 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173484.log.gz 2023-09-30T22:28:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169315.log 2023-09-30T22:28:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187050.log 2023-09-30T22:28:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169315.log: /var/log/ceph/ceph-client.admin.187050.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169315.log.gz 2023-09-30T22:28:04.457 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.187050.log.gz 2023-09-30T22:28:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179933.log 2023-09-30T22:28:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141006.log 2023-09-30T22:28:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179933.log: /var/log/ceph/ceph-client.admin.141006.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179933.log.gz 2023-09-30T22:28:04.458 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141006.log.gz 2023-09-30T22:28:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139858.log 2023-09-30T22:28:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182847.log 2023-09-30T22:28:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139858.log: /var/log/ceph/ceph-client.admin.182847.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139858.log.gz 2023-09-30T22:28:04.459 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.182847.log.gz 2023-09-30T22:28:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170198.log 2023-09-30T22:28:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186855.log 2023-09-30T22:28:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170198.log: /var/log/ceph/ceph-client.admin.186855.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.170198.log.gz 2023-09-30T22:28:04.459 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.186855.log.gz 2023-09-30T22:28:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127010.log 2023-09-30T22:28:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163844.log 2023-09-30T22:28:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127010.log: /var/log/ceph/ceph-client.admin.163844.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127010.log.gz 2023-09-30T22:28:04.460 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163844.log.gz 2023-09-30T22:28:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174294.log 2023-09-30T22:28:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169415.log 2023-09-30T22:28:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174294.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174294.log.gz 2023-09-30T22:28:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169415.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169415.log.gz 2023-09-30T22:28:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128523.log 2023-09-30T22:28:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150277.log 2023-09-30T22:28:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128523.log: /var/log/ceph/ceph-client.admin.150277.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.128523.log.gz 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150277.log.gz 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192293.log 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174097.log 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192293.log: /var/log/ceph/ceph-client.admin.174097.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.192293.log.gz 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.174097.log.gz 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178927.log 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157025.log 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178927.log: /var/log/ceph/ceph-client.admin.157025.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.178927.log.gz 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157025.log.gz 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147828.log 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144348.log 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147828.log: /var/log/ceph/ceph-client.admin.144348.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147828.log.gz 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144348.log.gz 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140614.log 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188443.log 2023-09-30T22:28:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140614.log: /var/log/ceph/ceph-client.admin.188443.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140614.log.gz 2023-09-30T22:28:04.463 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.188443.log.gz 2023-09-30T22:28:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191898.log 2023-09-30T22:28:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146614.log 2023-09-30T22:28:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.191898.log: /var/log/ceph/ceph-client.admin.146614.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191898.log.gz 2023-09-30T22:28:04.463 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146614.log.gz 2023-09-30T22:28:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168949.log 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152323.log 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168949.log: /var/log/ceph/ceph-client.admin.152323.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.168949.log.gz 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152323.log.gz 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127183.log 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187194.log 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127183.log: /var/log/ceph/ceph-client.admin.187194.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127183.log.gz 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.187194.log.gz 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178365.log 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172159.log 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178365.log: /var/log/ceph/ceph-client.admin.172159.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178365.log.gz 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172159.log.gz 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189938.log 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160511.log 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189938.log: /var/log/ceph/ceph-client.admin.160511.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189938.log.gz 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160511.log.gz 2023-09-30T22:28:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150159.log 2023-09-30T22:28:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149375.log 2023-09-30T22:28:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150159.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150159.log.gz 2023-09-30T22:28:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149375.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149375.log.gz 2023-09-30T22:28:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168523.log 2023-09-30T22:28:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161883.log 2023-09-30T22:28:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168523.log: /var/log/ceph/ceph-client.admin.161883.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168523.log.gz 2023-09-30T22:28:04.466 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161883.log.gz 2023-09-30T22:28:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137922.log 2023-09-30T22:28:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186384.log 2023-09-30T22:28:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137922.log: /var/log/ceph/ceph-client.admin.186384.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137922.log.gz 2023-09-30T22:28:04.467 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186384.log.gz 2023-09-30T22:28:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146626.log 2023-09-30T22:28:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129709.log 2023-09-30T22:28:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146626.log: /var/log/ceph/ceph-client.admin.129709.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.146626.log.gz 2023-09-30T22:28:04.468 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129709.log.gz 2023-09-30T22:28:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168131.log 2023-09-30T22:28:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135439.log 2023-09-30T22:28:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168131.log: /var/log/ceph/ceph-client.admin.135439.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168131.log.gz 2023-09-30T22:28:04.468 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135439.log.gz 2023-09-30T22:28:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190526.log 2023-09-30T22:28:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193299.log 2023-09-30T22:28:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190526.log: /var/log/ceph/ceph-client.admin.193299.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.190526.log.gz 2023-09-30T22:28:04.469 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.193299.log.gz 2023-09-30T22:28:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193298.log 2023-09-30T22:28:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193691.log 2023-09-30T22:28:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193298.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.193298.log.gz 2023-09-30T22:28:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193691.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193691.log.gz 2023-09-30T22:28:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188809.log 2023-09-30T22:28:04.470 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128616.log 2023-09-30T22:28:04.470 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188809.log: /var/log/ceph/ceph-client.admin.128616.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.188809.log.gz 2023-09-30T22:28:04.470 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128616.log.gz 2023-09-30T22:28:04.470 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176175.log 2023-09-30T22:28:04.470 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181279.log 2023-09-30T22:28:04.470 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176175.log: /var/log/ceph/ceph-client.admin.181279.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176175.log.gz 2023-09-30T22:28:04.470 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.181279.log.gz 2023-09-30T22:28:04.470 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194278.log 2023-09-30T22:28:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193076.log 2023-09-30T22:28:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194278.log: /var/log/ceph/ceph-client.admin.193076.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.194278.log.gz 2023-09-30T22:28:04.471 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.193076.log.gz 2023-09-30T22:28:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146112.log 2023-09-30T22:28:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165191.log 2023-09-30T22:28:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146112.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.146112.log.gz 2023-09-30T22:28:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.165191.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.165191.log.gz 2023-09-30T22:28:04.471 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170972.log 2023-09-30T22:28:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183239.log 2023-09-30T22:28:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170972.log: /var/log/ceph/ceph-client.admin.183239.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170972.log.gz 2023-09-30T22:28:04.472 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183239.log.gz 2023-09-30T22:28:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169733.log 2023-09-30T22:28:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136182.log 2023-09-30T22:28:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169733.log: /var/log/ceph/ceph-client.admin.136182.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169733.log.gz 2023-09-30T22:28:04.472 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136182.log.gz 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173678.log 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135518.log 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.173678.log: /var/log/ceph/ceph-client.admin.135518.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173678.log.gz 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.135518.log.gz 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143676.log 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187364.log 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143676.log: /var/log/ceph/ceph-client.admin.187364.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143676.log.gz 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.187364.log.gz 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186658.log 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184731.log 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186658.log: /var/log/ceph/ceph-client.admin.184731.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186658.log.gz 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.184731.log.gz 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160137.log 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155995.log 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160137.log: /var/log/ceph/ceph-client.admin.155995.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160137.log.gz 2023-09-30T22:28:04.475 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155995.log.gz 2023-09-30T22:28:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172307.log 2023-09-30T22:28:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158856.log 2023-09-30T22:28:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172307.log: /var/log/ceph/ceph-client.admin.158856.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172307.log.gz 2023-09-30T22:28:04.475 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158856.log.gz 2023-09-30T22:28:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144023.log 2023-09-30T22:28:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188932.log 2023-09-30T22:28:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.144023.log: /var/log/ceph/ceph-client.admin.188932.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144023.log.gz 2023-09-30T22:28:04.476 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188932.log.gz 2023-09-30T22:28:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176057.log 2023-09-30T22:28:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177438.log 2023-09-30T22:28:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176057.log: /var/log/ceph/ceph-client.admin.177438.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.176057.log.gz 2023-09-30T22:28:04.476 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.177438.log.gz 2023-09-30T22:28:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189154.log 2023-09-30T22:28:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167765.log 2023-09-30T22:28:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189154.log: /var/log/ceph/ceph-client.admin.167765.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.189154.log.gz 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167765.log.gz 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170182.log 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180103.log 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170182.log: /var/log/ceph/ceph-client.admin.180103.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170182.log.gz 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.180103.log.gz 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124627.log 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118614.log 2023-09-30T22:28:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124627.log: /var/log/ceph/ceph-client.admin.118614.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124627.log.gz 2023-09-30T22:28:04.478 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118614.log.gz 2023-09-30T22:28:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194985.log 2023-09-30T22:28:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123993.log 2023-09-30T22:28:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194985.log: /var/log/ceph/ceph-client.admin.123993.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.194985.log.gz 2023-09-30T22:28:04.478 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123993.log.gz 2023-09-30T22:28:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159638.log 2023-09-30T22:28:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125923.log 2023-09-30T22:28:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159638.log: /var/log/ceph/ceph-client.admin.125923.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159638.log.gz 2023-09-30T22:28:04.479 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125923.log.gz 2023-09-30T22:28:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187999.log 2023-09-30T22:28:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141106.log 2023-09-30T22:28:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.187999.log: /var/log/ceph/ceph-client.admin.141106.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.187999.log.gz 2023-09-30T22:28:04.479 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141106.log.gz 2023-09-30T22:28:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137334.log 2023-09-30T22:28:04.479 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125362.log 2023-09-30T22:28:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137334.log: /var/log/ceph/ceph-client.admin.125362.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137334.log.gz 2023-09-30T22:28:04.480 INFO:teuthology.orchestra.run.smithi089.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.125362.log.gz 2023-09-30T22:28:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185992.log 2023-09-30T22:28:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162516.log 2023-09-30T22:28:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185992.log: /var/log/ceph/ceph-client.admin.162516.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.185992.log.gz 2023-09-30T22:28:04.480 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162516.log.gz 2023-09-30T22:28:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193886.log 2023-09-30T22:28:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167619.log 2023-09-30T22:28:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193886.log: /var/log/ceph/ceph-client.admin.167619.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.193886.log.gz 2023-09-30T22:28:04.481 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.167619.log.gz 2023-09-30T22:28:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124412.log 2023-09-30T22:28:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140295.log 2023-09-30T22:28:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124412.log: /var/log/ceph/ceph-client.admin.140295.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124412.log.gz 2023-09-30T22:28:04.481 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140295.log.gz 2023-09-30T22:28:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168745.log 2023-09-30T22:28:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161564.log 2023-09-30T22:28:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.168745.log: /var/log/ceph/ceph-client.admin.161564.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.168745.log.gz 2023-09-30T22:28:04.482 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.161564.log.gz 2023-09-30T22:28:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156677.log 2023-09-30T22:28:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191310.log 2023-09-30T22:28:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156677.log: /var/log/ceph/ceph-client.admin.191310.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156677.log.gz 2023-09-30T22:28:04.482 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.191310.log.gz 2023-09-30T22:28:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145600.log 2023-09-30T22:28:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137896.log 2023-09-30T22:28:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145600.log: /var/log/ceph/ceph-client.admin.137896.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145600.log.gz 2023-09-30T22:28:04.483 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137896.log.gz 2023-09-30T22:28:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183153.log 2023-09-30T22:28:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192934.log 2023-09-30T22:28:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183153.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183153.log.gz 2023-09-30T22:28:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192934.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.192934.log.gz 2023-09-30T22:28:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156069.log 2023-09-30T22:28:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177973.log 2023-09-30T22:28:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156069.log: /var/log/ceph/ceph-client.admin.177973.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156069.log.gz 2023-09-30T22:28:04.484 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.177973.log.gz 2023-09-30T22:28:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123821.log 2023-09-30T22:28:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192515.log 2023-09-30T22:28:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123821.log: /var/log/ceph/ceph-client.admin.192515.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123821.log.gz 2023-09-30T22:28:04.484 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.192515.log.gz 2023-09-30T22:28:04.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154427.log 2023-09-30T22:28:04.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178169.log 2023-09-30T22:28:04.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154427.log: /var/log/ceph/ceph-client.admin.178169.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154427.log.gz 2023-09-30T22:28:04.485 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178169.log.gz 2023-09-30T22:28:04.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142892.log 2023-09-30T22:28:04.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182311.log 2023-09-30T22:28:04.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142892.log: /var/log/ceph/ceph-client.admin.182311.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142892.log.gz 2023-09-30T22:28:04.485 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.182311.log.gz 2023-09-30T22:28:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155381.log 2023-09-30T22:28:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164263.log 2023-09-30T22:28:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155381.log: /var/log/ceph/ceph-client.admin.164263.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155381.log.gz 2023-09-30T22:28:04.486 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164263.log.gz 2023-09-30T22:28:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166456.log 2023-09-30T22:28:04.486 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147755.log 2023-09-30T22:28:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166456.log: /var/log/ceph/ceph-client.admin.147755.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166456.log.gz 2023-09-30T22:28:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147729.log 2023-09-30T22:28:04.487 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147755.log.gz 2023-09-30T22:28:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144444.log 2023-09-30T22:28:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.147729.log: /var/log/ceph/ceph-client.admin.144444.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147729.log.gz 2023-09-30T22:28:04.487 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.144444.log.gz 2023-09-30T22:28:04.487 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178418.log 2023-09-30T22:28:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153421.log 2023-09-30T22:28:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178418.log: /var/log/ceph/ceph-client.admin.153421.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178418.log.gz 2023-09-30T22:28:04.488 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153421.log.gz 2023-09-30T22:28:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184246.log 2023-09-30T22:28:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151530.log 2023-09-30T22:28:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184246.log: /var/log/ceph/ceph-client.admin.151530.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.184246.log.gz 2023-09-30T22:28:04.488 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151530.log.gz 2023-09-30T22:28:04.488 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148565.log 2023-09-30T22:28:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122701.log 2023-09-30T22:28:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.148565.log: /var/log/ceph/ceph-client.admin.122701.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148565.log.gz 2023-09-30T22:28:04.489 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122701.log.gz 2023-09-30T22:28:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162447.log 2023-09-30T22:28:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125068.log 2023-09-30T22:28:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162447.log: /var/log/ceph/ceph-client.admin.125068.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162447.log.gz 2023-09-30T22:28:04.489 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.125068.log.gz 2023-09-30T22:28:04.489 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162839.log 2023-09-30T22:28:04.490 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165074.log 2023-09-30T22:28:04.490 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162839.log: /var/log/ceph/ceph-client.admin.165074.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162839.log.gz 2023-09-30T22:28:04.490 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165074.log.gz 2023-09-30T22:28:04.490 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164799.log 2023-09-30T22:28:04.490 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193835.log 2023-09-30T22:28:04.490 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164799.log: /var/log/ceph/ceph-client.admin.193835.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164799.log.gz 2023-09-30T22:28:04.490 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.193835.log.gz 2023-09-30T22:28:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159246.log 2023-09-30T22:28:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154481.log 2023-09-30T22:28:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159246.log: /var/log/ceph/ceph-client.admin.154481.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.159246.log.gz 2023-09-30T22:28:04.491 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154481.log.gz 2023-09-30T22:28:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176644.log 2023-09-30T22:28:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123598.log 2023-09-30T22:28:04.491 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176644.log: /var/log/ceph/ceph-client.admin.123598.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176644.log.gz 2023-09-30T22:28:04.491 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123598.log.gz 2023-09-30T22:28:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169145.log 2023-09-30T22:28:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166073.log 2023-09-30T22:28:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169145.log: /var/log/ceph/ceph-client.admin.166073.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169145.log.gz 2023-09-30T22:28:04.492 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166073.log.gz 2023-09-30T22:28:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139123.log 2023-09-30T22:28:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187051.log 2023-09-30T22:28:04.492 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139123.log: /var/log/ceph/ceph-client.admin.187051.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139123.log.gz 2023-09-30T22:28:04.493 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.187051.log.gz 2023-09-30T22:28:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145254.log 2023-09-30T22:28:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129351.log 2023-09-30T22:28:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145254.log: /var/log/ceph/ceph-client.admin.129351.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145254.log.gz 2023-09-30T22:28:04.493 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129351.log.gz 2023-09-30T22:28:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171693.log 2023-09-30T22:28:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140026.log 2023-09-30T22:28:04.493 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171693.log: /var/log/ceph/ceph-client.admin.140026.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171693.log.gz 2023-09-30T22:28:04.494 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.140026.log.gz 2023-09-30T22:28:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190304.log 2023-09-30T22:28:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167620.log 2023-09-30T22:28:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190304.log: /var/log/ceph/ceph-client.admin.167620.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.190304.log.gz 2023-09-30T22:28:04.494 INFO:teuthology.orchestra.run.smithi089.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.167620.log.gz 2023-09-30T22:28:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188051.log 2023-09-30T22:28:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163648.log 2023-09-30T22:28:04.494 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188051.log: /var/log/ceph/ceph-client.admin.163648.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188051.log.gz 2023-09-30T22:28:04.495 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.163648.log.gz 2023-09-30T22:28:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175640.log 2023-09-30T22:28:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163109.log 2023-09-30T22:28:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175640.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175640.log.gz 2023-09-30T22:28:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163109.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163109.log.gz 2023-09-30T22:28:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157069.log 2023-09-30T22:28:04.495 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161342.log 2023-09-30T22:28:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157069.log: /var/log/ceph/ceph-client.admin.161342.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157069.log.gz 2023-09-30T22:28:04.496 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.161342.log.gz 2023-09-30T22:28:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172530.log 2023-09-30T22:28:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142033.log 2023-09-30T22:28:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.172530.log: /var/log/ceph/ceph-client.admin.142033.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.172530.log.gz 2023-09-30T22:28:04.496 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142033.log.gz 2023-09-30T22:28:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126941.log 2023-09-30T22:28:04.496 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168976.log 2023-09-30T22:28:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.126941.log: /var/log/ceph/ceph-client.admin.168976.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.126941.log.gz 2023-09-30T22:28:04.497 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.168976.log.gz 2023-09-30T22:28:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125678.log 2023-09-30T22:28:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144068.log 2023-09-30T22:28:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.125678.log: /var/log/ceph/ceph-client.admin.144068.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125678.log.gz 2023-09-30T22:28:04.497 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144068.log.gz 2023-09-30T22:28:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177633.log 2023-09-30T22:28:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161269.log 2023-09-30T22:28:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177633.log: /var/log/ceph/ceph-client.admin.161269.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.177633.log.gz 2023-09-30T22:28:04.498 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161269.log.gz 2023-09-30T22:28:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190404.log 2023-09-30T22:28:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154304.log 2023-09-30T22:28:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190404.log: /var/log/ceph/ceph-client.admin.154304.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.190404.log.gz 2023-09-30T22:28:04.498 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154304.log.gz 2023-09-30T22:28:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129008.log 2023-09-30T22:28:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180495.log 2023-09-30T22:28:04.499 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129008.log: /var/log/ceph/ceph-client.admin.180495.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129008.log.gz 2023-09-30T22:28:04.499 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180495.log.gz 2023-09-30T22:28:04.499 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166070.log 2023-09-30T22:28:04.499 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139268.log 2023-09-30T22:28:04.499 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166070.log: /var/log/ceph/ceph-client.admin.139268.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166070.log.gz 2023-09-30T22:28:04.499 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139268.log.gz 2023-09-30T22:28:04.499 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152714.log 2023-09-30T22:28:04.499 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150747.log 2023-09-30T22:28:04.500 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152714.log: /var/log/ceph/ceph-client.admin.150747.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.152714.log.gz 2023-09-30T22:28:04.500 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150747.log.gz 2023-09-30T22:28:04.500 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149373.log 2023-09-30T22:28:04.500 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125215.log 2023-09-30T22:28:04.500 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149373.log: /var/log/ceph/ceph-client.admin.125215.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149373.log.gz 2023-09-30T22:28:04.500 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125215.log.gz 2023-09-30T22:28:04.500 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-09-30T22:28:04.500 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177234.log 2023-09-30T22:28:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.177234.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177234.log.gz 2023-09-30T22:28:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151897.log 2023-09-30T22:28:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.151897.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.151897.log.gz 2023-09-30T22:28:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171667.log 2023-09-30T22:28:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171667.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.171667.log.gz 2023-09-30T22:28:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171105.log 2023-09-30T22:28:04.501 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171105.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171105.log.gz 2023-09-30T22:28:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192736.log 2023-09-30T22:28:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192736.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192736.log.gz 2023-09-30T22:28:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137164.log 2023-09-30T22:28:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137164.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137164.log.gz 2023-09-30T22:28:04.502 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159719.log 2023-09-30T22:28:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.159719.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159719.log.gz 2023-09-30T22:28:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150942.log 2023-09-30T22:28:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150942.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.150942.log.gz 2023-09-30T22:28:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194483.log 2023-09-30T22:28:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194483.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.194483.log.gz 2023-09-30T22:28:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145965.log 2023-09-30T22:28:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.145965.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145965.log.gz 2023-09-30T22:28:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169563.log 2023-09-30T22:28:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.169563.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169563.log.gz 2023-09-30T22:28:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177803.log 2023-09-30T22:28:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177803.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177803.log.gz 2023-09-30T22:28:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123008.log 2023-09-30T22:28:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123008.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123008.log.gz 2023-09-30T22:28:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152637.log 2023-09-30T22:28:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.152637.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.152637.log.gz 2023-09-30T22:28:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182750.log 2023-09-30T22:28:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.182750.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182750.log.gz 2023-09-30T22:28:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160801.log 2023-09-30T22:28:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.160801.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160801.log.gz 2023-09-30T22:28:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179907.log 2023-09-30T22:28:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.179907.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179907.log.gz 2023-09-30T22:28:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143238.log 2023-09-30T22:28:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.143238.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143238.log.gz 2023-09-30T22:28:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189350.log 2023-09-30T22:28:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.189350.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.189350.log.gz 2023-09-30T22:28:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194482.log 2023-09-30T22:28:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.194482.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.194482.log.gz 2023-09-30T22:28:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157785.log 2023-09-30T22:28:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.157785.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157785.log.gz 2023-09-30T22:28:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124676.log 2023-09-30T22:28:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.124676.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124676.log.gz 2023-09-30T22:28:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135988.log 2023-09-30T22:28:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.135988.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135988.log.gz 2023-09-30T22:28:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164433.log 2023-09-30T22:28:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.164433.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.164433.log.gz 2023-09-30T22:28:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178143.log 2023-09-30T22:28:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178143.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.178143.log.gz 2023-09-30T22:28:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170883.log 2023-09-30T22:28:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.170883.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.170883.log.gz 2023-09-30T22:28:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188344.log 2023-09-30T22:28:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.188344.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.188344.log.gz 2023-09-30T22:28:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193077.log 2023-09-30T22:28:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.193077.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.193077.log.gz 2023-09-30T22:28:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129917.log 2023-09-30T22:28:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.129917.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129917.log.gz 2023-09-30T22:28:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142425.log 2023-09-30T22:28:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142425.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142425.log.gz 2023-09-30T22:28:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192319.log 2023-09-30T22:28:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.192319.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192319.log.gz 2023-09-30T22:28:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162053.log 2023-09-30T22:28:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162053.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162053.log.gz 2023-09-30T22:28:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122899.log 2023-09-30T22:28:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.122899.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122899.log.gz 2023-09-30T22:28:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180181.log 2023-09-30T22:28:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180181.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180181.log.gz 2023-09-30T22:28:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183265.log 2023-09-30T22:28:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.183265.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.183265.log.gz 2023-09-30T22:28:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186854.log 2023-09-30T22:28:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.186854.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.186854.log.gz 2023-09-30T22:28:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150943.log 2023-09-30T22:28:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.150943.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.150943.log.gz 2023-09-30T22:28:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166198.log 2023-09-30T22:28:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166198.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166198.log.gz 2023-09-30T22:28:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140857.log 2023-09-30T22:28:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.140857.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140857.log.gz 2023-09-30T22:28:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176227.log 2023-09-30T22:28:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.176227.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.176227.log.gz 2023-09-30T22:28:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.195083.log 2023-09-30T22:28:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.195083.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.195083.log.gz 2023-09-30T22:28:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190696.log 2023-09-30T22:28:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190696.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190696.log.gz 2023-09-30T22:28:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137870.log 2023-09-30T22:28:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.137870.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137870.log.gz 2023-09-30T22:28:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128812.log 2023-09-30T22:28:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.128812.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128812.log.gz 2023-09-30T22:28:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158177.log 2023-09-30T22:28:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158177.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158177.log.gz 2023-09-30T22:28:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127186.log 2023-09-30T22:28:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127186.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127186.log.gz 2023-09-30T22:28:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127470.log 2023-09-30T22:28:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.127470.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127470.log.gz 2023-09-30T22:28:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174659.log 2023-09-30T22:28:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174659.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.174659.log.gz 2023-09-30T22:28:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138877.log 2023-09-30T22:28:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.138877.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138877.log.gz 2023-09-30T22:28:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161760.log 2023-09-30T22:28:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161760.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161760.log.gz 2023-09-30T22:28:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146749.log 2023-09-30T22:28:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.146749.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146749.log.gz 2023-09-30T22:28:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167739.log 2023-09-30T22:28:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167739.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167739.log.gz 2023-09-30T22:28:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162325.log 2023-09-30T22:28:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162325.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.162325.log.gz 2023-09-30T22:28:04.536 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185497.log 2023-09-30T22:28:04.536 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.185497.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.185497.log.gz 2023-09-30T22:28:04.536 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158150.log 2023-09-30T22:28:04.536 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158150.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158150.log.gz 2023-09-30T22:28:04.536 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190775.log 2023-09-30T22:28:04.536 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.190775.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.190775.log.gz 2023-09-30T22:28:04.536 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139733.log 2023-09-30T22:28:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139733.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139733.log.gz 2023-09-30T22:28:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153813.log 2023-09-30T22:28:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153813.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153813.log.gz 2023-09-30T22:28:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180399.log 2023-09-30T22:28:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180399.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180399.log.gz 2023-09-30T22:28:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141837.log 2023-09-30T22:28:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.141837.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141837.log.gz 2023-09-30T22:28:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153325.log 2023-09-30T22:28:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.153325.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.153325.log.gz 2023-09-30T22:28:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180006.log 2023-09-30T22:28:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.180006.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.180006.log.gz 2023-09-30T22:28:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166367.log 2023-09-30T22:28:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.166367.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.166367.log.gz 2023-09-30T22:28:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171963.log 2023-09-30T22:28:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.171963.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171963.log.gz 2023-09-30T22:28:04.539 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156051.log 2023-09-30T22:28:04.539 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.156051.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156051.log.gz 2023-09-30T22:28:04.539 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178240.log 2023-09-30T22:28:04.539 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.178240.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178240.log.gz 2023-09-30T22:28:04.539 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139929.log 2023-09-30T22:28:04.539 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139929.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139929.log.gz 2023-09-30T22:28:04.539 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161369.log 2023-09-30T22:28:04.539 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177555.log 2023-09-30T22:28:04.540 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.161369.log: /var/log/ceph/ceph-client.admin.177555.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161369.log.gz 2023-09-30T22:28:04.540 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177555.log.gz 2023-09-30T22:28:04.540 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139046.log 2023-09-30T22:28:04.540 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.139046.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139046.log.gz 2023-09-30T22:28:04.540 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162251.log 2023-09-30T22:28:04.540 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162251.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162251.log.gz 2023-09-30T22:28:04.540 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142182.log 2023-09-30T22:28:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.142182.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142182.log.gz 2023-09-30T22:28:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162279.log 2023-09-30T22:28:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.162279.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162279.log.gz 2023-09-30T22:28:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123696.log 2023-09-30T22:28:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.123696.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123696.log.gz 2023-09-30T22:28:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167202.log 2023-09-30T22:28:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167202.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167202.log.gz 2023-09-30T22:28:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167816.log 2023-09-30T22:28:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167816.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167816.log.gz 2023-09-30T22:28:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149127.log 2023-09-30T22:28:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.149127.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149127.log.gz 2023-09-30T22:28:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158151.log 2023-09-30T22:28:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.158151.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158151.log.gz 2023-09-30T22:28:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155799.log 2023-09-30T22:28:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155799.log: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-09-30T22:28:04.542 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155799.log.gz 2023-09-30T22:28:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174855.log 2023-09-30T22:28:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163503.log 2023-09-30T22:28:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.174855.log: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-09-30T22:28:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.163503.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174855.log.gz 2023-09-30T22:28:04.543 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163503.log.gz 2023-09-30T22:28:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167007.log 2023-09-30T22:28:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155015.log 2023-09-30T22:28:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.167007.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167007.log.gz 2023-09-30T22:28:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.155015.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155015.log.gz 2023-09-30T22:28:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154062.log 2023-09-30T22:28:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177850.log 2023-09-30T22:28:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.154062.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.154062.log.gz 2023-09-30T22:28:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.177850.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177850.log.gz 2023-09-30T22:28:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184923.log 2023-09-30T22:28:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175783.log 2023-09-30T22:28:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.184923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149101.log 2023-09-30T22:28:04.545 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.184923.log.gz 2023-09-30T22:28:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.175783.log: /var/log/ceph/ceph-client.admin.149101.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175783.log.gz 2023-09-30T22:28:04.545 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149101.log.gz 2023-09-30T22:28:04.551 INFO:teuthology.orchestra.run.smithi089.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-09-30T22:28:04.555 INFO:teuthology.orchestra.run.smithi089.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-09-30T22:28:04.710 INFO:teuthology.orchestra.run.smithi089.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-09-30T22:28:04.784 INFO:teuthology.orchestra.run.smithi089.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-09-30T22:28:04.786 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-30T22:28:04.786 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m1.009s 2023-09-30T22:28:04.786 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m3.813s 2023-09-30T22:28:04.786 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m0.779s 2023-09-30T22:28:04.786 INFO:tasks.ceph:Archiving logs... 2023-09-30T22:28:04.787 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/remote/smithi089/log 2023-09-30T22:28:04.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T22:28:06.395 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/remote/smithi134/log 2023-09-30T22:28:06.396 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T22:28:06.431 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:28:06.445 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-30T22:28:06.464 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:28:06.469 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-30T22:28:06.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T22:28:06.475 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T22:28:06.556 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-30T22:28:06.569 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-30T22:28:06.569 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:28:06.571 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:28:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-09-30T22:28:06.590 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-09-30T22:28:06.593 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:28:06.593 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-09-30T22:28:06.593 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 -95us[ -80us] +/- 89ms 2023-09-30T22:28:06.594 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -449us[ -434us] +/- 49ms 2023-09-30T22:28:06.594 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +839us[ +839us] +/- 83ms 2023-09-30T22:28:06.594 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:28:06.596 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:28:06.596 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-09-30T22:28:06.596 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 -86us[ -86us] +/- 89ms 2023-09-30T22:28:06.596 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv02.front.sepia.ceph.com 3 6 377 48 -465us[ -467us] +/- 49ms 2023-09-30T22:28:06.596 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +827us[ +827us] +/- 83ms 2023-09-30T22:28:06.596 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:28:06.597 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-30T22:28:06.609 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-30T22:28:06.609 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-30T22:28:06.622 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T22:28:06.687 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T22:28:06.777 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:28:06.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:28:06.809 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:28:06.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:28:06.841 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-30T22:28:06.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-30T22:28:06.868 INFO:teuthology.task.internal:Duration was 1463.525702 seconds 2023-09-30T22:28:06.869 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-30T22:28:06.880 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-30T22:28:06.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T22:28:06.883 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T22:28:06.930 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:28:06.930 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:28:07.334 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-30T22:28:07.334 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2023-09-30T22:28:07.334 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T22:28:07.358 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2023-09-30T22:28:07.359 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T22:28:07.382 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-30T22:28:07.382 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T22:28:07.400 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T22:28:07.460 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-30T22:28:07.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:28:07.462 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:28:07.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-30T22:28:07.577 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-30T22:28:07.577 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T22:28:07.605 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T22:28:07.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-30T22:28:07.645 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:28:07.648 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:28:07.672 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2023-09-30T22:28:07.700 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2023-09-30T22:28:07.713 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:28:07.765 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi089.front.sepia.ceph.com, flagging run as failed 2023-09-30T22:28:07.765 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:28:07.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:28:07.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-30T22:28:07.796 INFO:teuthology.task.internal:Transferring archived files... 2023-09-30T22:28:07.796 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/remote/smithi089 2023-09-30T22:28:07.796 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T22:29:36.079 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2023-09-30T22:29:36.505 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035/remote/smithi089/coredump/1696112205.126953.core.gz: gzip compressed data, was "1696112205.126953.core", last modified: Sat Sep 30 22:28:07 2023, max compression, original size modulo 2^32 1365295104 2023-09-30T22:29:36.505 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2023-09-30T22:29:42.285 INFO:teuthology.task.internal: core looks like: /tmp/tmpprhagtfe: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mgr -f --cluster ceph -i x', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mgr', platform: 'x86_64' 2023-09-30T22:29:42.427 INFO:teuthology.task.internal: dump_program: ceph-mgr -f --cluster ceph -i x 2023-09-30T22:29:42.428 DEBUG:teuthology.orchestra.run.smithi089:> which 'ceph-mgr -f --cluster ceph -i x' 2023-09-30T22:29:42.450 INFO:teuthology.orchestra.run.smithi089.stderr:/usr/bin/which: invalid option -- 'f' 2023-09-30T22:29:42.450 INFO:teuthology.orchestra.run.smithi089.stderr:/usr/bin/which: unrecognized option '--cluster' 2023-09-30T22:29:42.451 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-mgr 2023-09-30T22:29:42.451 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph 2023-09-30T22:29:42.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:29:42.452 INFO:teuthology.orchestra.run.smithi089.stderr:/usr/bin/which: no x in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2023-09-30T22:29:42.452 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 1: "which 'ceph-mgr -f --cluster ceph -i x'" 2023-09-30T22:29:42.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-30T22:29:42.483 INFO:teuthology.task.internal:Not uploading archives. 2023-09-30T22:29:42.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-30T22:29:42.540 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-30T22:29:42.540 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T22:29:42.556 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T22:29:42.568 INFO:teuthology.orchestra.run.smithi089.stdout: 393224 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 30 22:28 /home/ubuntu/cephtest 2023-09-30T22:29:42.568 INFO:teuthology.orchestra.run.smithi089.stdout: 393225 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Sep 30 22:28 /home/ubuntu/cephtest/archive 2023-09-30T22:29:42.568 INFO:teuthology.orchestra.run.smithi089.stdout: 393226 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:16 /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:29:42.568 INFO:teuthology.orchestra.run.smithi089.stdout: 393232 362148 -rw------- 1 root root 1365295104 Sep 30 22:16 /home/ubuntu/cephtest/archive/coredump/1696112205.126953.core 2023-09-30T22:29:42.568 INFO:teuthology.orchestra.run.smithi089.stdout: 655784 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 30 22:28 /home/ubuntu/cephtest/archive/audit 2023-09-30T22:29:42.569 INFO:teuthology.orchestra.run.smithi089.stdout: 655822 28 -rw------- 1 ubuntu ubuntu 25507 Sep 30 22:28 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-09-30T22:29:42.569 INFO:teuthology.orchestra.run.smithi089.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:28 /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:29:42.569 INFO:teuthology.orchestra.run.smithi089.stdout: 525491 4 -rw-rw-rw- 1 ubuntu ubuntu 773 Sep 30 22:28 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-09-30T22:29:42.569 INFO:teuthology.orchestra.run.smithi089.stdout: 525359 112 -rw-rw-r-- 1 ubuntu ubuntu 113174 Sep 30 22:28 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:29:42.569 INFO:teuthology.orchestra.run.smithi089.stdout: 525358 132 -rw-rw-rw- 1 ubuntu ubuntu 133213 Sep 30 22:28 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-09-30T22:29:42.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:29:42.570 INFO:teuthology.orchestra.run.smithi089.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-09-30T22:29:42.570 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi089 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-09-30T22:29:42.583 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-30T22:29:42.596 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-30T22:29:42.609 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:29:42.609 INFO:teuthology.nuke:Checking targets against current locks 2023-09-30T22:29:42.617 INFO:teuthology.orchestra.run.smithi134.stdout: 393224 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 30 22:28 /home/ubuntu/cephtest 2023-09-30T22:29:42.617 INFO:teuthology.orchestra.run.smithi134.stdout: 393225 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Sep 30 22:28 /home/ubuntu/cephtest/archive 2023-09-30T22:29:42.617 INFO:teuthology.orchestra.run.smithi134.stdout: 655784 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 30 22:28 /home/ubuntu/cephtest/archive/audit 2023-09-30T22:29:42.617 INFO:teuthology.orchestra.run.smithi134.stdout: 655786 220 -rw------- 1 ubuntu ubuntu 223563 Sep 30 22:28 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-09-30T22:29:42.618 INFO:teuthology.orchestra.run.smithi134.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:28 /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:29:42.618 INFO:teuthology.orchestra.run.smithi134.stdout: 525491 4 -rw-rw-rw- 1 ubuntu ubuntu 898 Sep 30 22:26 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-09-30T22:29:42.618 INFO:teuthology.orchestra.run.smithi134.stdout: 525359 92 -rw-rw-r-- 1 ubuntu ubuntu 92143 Sep 30 22:28 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:29:42.618 INFO:teuthology.orchestra.run.smithi134.stdout: 525358 88 -rw-rw-rw- 1 ubuntu ubuntu 86198 Sep 30 22:28 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-09-30T22:29:42.621 INFO:teuthology.orchestra.run.smithi134.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-09-30T22:29:42.645 DEBUG:teuthology.nuke:shortname: smithi089 2023-09-30T22:29:42.646 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:29:42.666 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 21:55:08.688025', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHT6UO4ioFtlIcI3t7G0ooNN2C7WcTF/HmMzTJpxs4Rz3xSMq8jsklewTvcYjzA7i+VieIsSZql1hajeiQ40DdU='} 2023-09-30T22:29:42.676 DEBUG:teuthology.nuke:shortname: smithi134 2023-09-30T22:29:42.676 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:29:42.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 21:55:08.689429', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGySYt+c1ONBPxgHRkiQTLkeNpBlx2n5G1xBpVB2OApOEJ7I2MiR9s06RV4zXlQcYi699ouZk420QaSwHM7Enqc='} 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.console.smithi089:Power off 2023-09-30T22:29:42.713 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T22:29:42.736 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2023-09-30T22:29:42.736 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:29:42.756 INFO:teuthology.orchestra.console.smithi134:Power off 2023-09-30T22:29:42.757 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T22:29:42.766 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-09-30T22:29:42.779 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2023-09-30T22:29:42.779 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:29:42.801 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2023-09-30T22:29:46.767 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:29:46.803 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:29:46.878 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-09-30T22:29:46.915 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2023-09-30T22:29:50.882 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:29:50.916 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:29:50.994 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-09-30T22:29:51.028 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2023-09-30T22:29:51.095 INFO:teuthology.orchestra.console.smithi089:Power off completed 2023-09-30T22:29:55.030 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:29:55.141 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2023-09-30T22:29:55.242 INFO:teuthology.orchestra.console.smithi134:Power off completed 2023-09-30T22:29:55.347 INFO:teuthology.run:Summary data: description: fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/fsstress}}} duration: 1463.5257019996643 failure_reason: 'Command failed on smithi089 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json''' flavor: default owner: scheduled_dparmar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=38e6cdfdfd0a40f1b6744f4631b74a54 status: fail success: false 2023-09-30T22:29:55.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T22:29:55.429 INFO:teuthology.run:FAIL